From patchwork Wed Mar 18 15:50:43 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Andr=C3=A9_Erdmann?= X-Patchwork-Id: 451512 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from whitealder.osuosl.org (whitealder.osuosl.org [140.211.166.138]) by ozlabs.org (Postfix) with ESMTP id 14897140077 for ; Thu, 19 Mar 2015 02:51:29 +1100 (AEDT) Authentication-Results: ozlabs.org; dkim=fail reason="verification failed; unprotected key" header.d=mailerd.de header.i=@mailerd.de header.b=QQEqkc+f; dkim-adsp=none (unprotected policy); dkim-atps=neutral Received: from localhost (localhost [127.0.0.1]) by whitealder.osuosl.org (Postfix) with ESMTP id 4634791ECA; Wed, 18 Mar 2015 15:51:28 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from whitealder.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id mY5Cts3J+-sF; Wed, 18 Mar 2015 15:51:27 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by whitealder.osuosl.org (Postfix) with ESMTP id E0F6A91EC0; Wed, 18 Mar 2015 15:51:26 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from fraxinus.osuosl.org (fraxinus.osuosl.org [140.211.166.137]) by ash.osuosl.org (Postfix) with ESMTP id E2FDB1C2439 for ; Wed, 18 Mar 2015 15:51:20 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by fraxinus.osuosl.org (Postfix) with ESMTP id D99A9A3763 for ; Wed, 18 Mar 2015 15:51:20 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from fraxinus.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id BALLwwR-5l_k for ; Wed, 18 Mar 2015 15:51:17 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.7.6 Received: from mail-wg0-f48.google.com (mail-wg0-f48.google.com [74.125.82.48]) by fraxinus.osuosl.org (Postfix) with ESMTPS id C1C9EA3728 for ; Wed, 18 Mar 2015 15:51:11 +0000 (UTC) Received: by wggv3 with SMTP id v3so38981292wgg.1 for ; Wed, 18 Mar 2015 08:51:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mailerd.de; s=mails; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-type:content-transfer-encoding; bh=yJzUhzcybrcKNkQEA5dnshOcKF2KmjpK9oUh94F3eUc=; b=QQEqkc+fDzwGtGwE9IDiQyNGaik6JJ3XpBThRfmRFgBPuC87RpVIBkf8glppsle5cs 5IjSySAvXBRHLeAxID2B5rQNpcNqIrRlRxl0PpjtKxYoOcHpblfJnsss8ReywkzUFWz7 gb79+PUlGmXuEwSQsBzbGjkrWoMFILaAEHFR8= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references:mime-version:content-type :content-transfer-encoding; bh=yJzUhzcybrcKNkQEA5dnshOcKF2KmjpK9oUh94F3eUc=; b=USNN7dsT0enWZJw8CNrSLP6WKoQN2YGLqXxVKCqowvWpSKmgfLF2TkbBL0oS+uScIz zww/uDdlwj1ktKyvf81D2K2INf+ox1kEYljQ7aWBbc9WguPvIEoS0J6PJDpNty1ZY1Dv aikr0zSvRbLlTrXqkwFTRFrvFry03pntiJMkJOnd63GHmv/qh3yFlFpuktWJ5fIIStf6 9uM6lFENVaZyQKbvXDtq/8LBe349d/dfwXk3Xe3oe9o8lUabCmlmJ8lgSoc6GNaMllzz ajwZQyOJzrprEW2FZK4c8QlqIQ/U27UtZsFWKVSVqqWObCXIyiEOlFcTjHO0Kqvl2kG8 L+zw== X-Gm-Message-State: ALoCoQmFQLOIBiIGBPl6ElrgB5Pfj8zvVWFiTf7EarJR+iKI9ONDY6hFuvSNrAtrkJdkk5R8lYjc X-Received: by 10.194.61.244 with SMTP id t20mr137737023wjr.83.1426693870311; Wed, 18 Mar 2015 08:51:10 -0700 (PDT) Received: from localhost.localdomain (dslb-092-074-053-028.092.074.pools.vodafone-ip.de. [92.74.53.28]) by mx.google.com with ESMTPSA id u16sm25024794wjr.5.2015.03.18.08.51.09 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Wed, 18 Mar 2015 08:51:09 -0700 (PDT) From: =?UTF-8?q?Andr=C3=A9=20Erdmann?= To: buildroot@buildroot.org Date: Wed, 18 Mar 2015 16:50:43 +0100 Message-Id: <1426693843-28792-11-git-send-email-dywi@mailerd.de> X-Mailer: git-send-email 2.3.2 In-Reply-To: <1426693843-28792-1-git-send-email-dywi@mailerd.de> References: <1426693843-28792-1-git-send-email-dywi@mailerd.de> MIME-Version: 1.0 Cc: thomas.petazzoni@free-electrons.com Subject: [Buildroot] [PATCH v2 10/10] autobuild-run: set locale to en_US or C X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: buildroot-bounces@busybox.net Sender: "buildroot" some python scripts break if the locale is set to C, try en_US{.UTF-8,} first Additionally, drop all locale env vars (LC_*, LANG[GUAGE]) when setting the new locale. Signed-off-by: André Erdmann --- Just for reference, a small python example that breaks if LANG is set to C: $ LANG=C python -c "print(u'Andr\xe9')" UnicodeEncodeError: 'ascii' codec can't encode character u'\xe9' in position 4: ordinal not in range(128) --- scripts/autobuild-run | 48 ++++++++++++++++++++++++++++++++++++++++++++---- 1 file changed, 44 insertions(+), 4 deletions(-) diff --git a/scripts/autobuild-run b/scripts/autobuild-run index e1a3398..a1e947d 100755 --- a/scripts/autobuild-run +++ b/scripts/autobuild-run @@ -168,6 +168,12 @@ class SystemInfo: DEFAULT_NEEDED_PROGS = ["make", "git", "gcc", "timeout"] DEFAULT_OPTIONAL_PROGS = ["bzr", "java", "javac", "jar"] + # list of default locales (in lowercase, without "-", descending order) + # some python scripts break if the locale is set to C, try en_US first + TRY_LOCALES = ['en_us.utf8', 'en_us', 'c'] + # list of locale environment variables that should be (re-)set by set_locale() + LOCALE_KEYS = ['LANG'] + def __init__(self): self.needed_progs = list(self.__class__.DEFAULT_NEEDED_PROGS) self.optional_progs = list(self.__class__.DEFAULT_OPTIONAL_PROGS) @@ -231,6 +237,41 @@ class SystemInfo: return not missing_requirements + def set_locale(self): + def is_locale_env_varname(w): + # w[:4] == 'LANG' and (not w[4:] or w[4:] == 'UAGE') ... + return w[:3] == 'LC_' or w == 'LANG' or w == 'LANGUAGE' + + ret, locales_str = self.run_cmd_get_stdout(["locale", "-a"]) + if ret != os.EX_OK: + return False + + # create a dict + # (as listed in TRY_LOCALES) => + locales = dict(( + (k.lower().replace("-", ""), k) for k in locales_str.split(None) + )) + + for loc_key in filter(lambda x: x in locales, self.TRY_LOCALES): + # cannot modify self.env while iterating over it, + # create intermediate list + env_old_locale_keys = [ + k for k in self.env.keys() if is_locale_env_varname(k) + ] + for k in env_old_locale_keys: + del self.env[k] + + # set new locale once + for vname in self.LOCALE_KEYS: + self.env[vname] = locales[loc_key] + return True + # -- end for + # practically impossible to reach this return if 'c' is in TRY_LOCALES: + return None + + def sanitize_env(self): + self.set_locale() + def popen(self, cmdv, **kwargs): kwargs.setdefault('stdin', self.devnull) kwargs.setdefault('stdout', self.devnull) @@ -789,12 +830,11 @@ def merge(dict_1, dict_2): def main(): - # Avoid locale settings of autobuilder machine leaking in, for example - # showing error messages in another language. - os.environ['LC_ALL'] = 'C' - check_version() sysinfo = SystemInfo() + # Avoid locale settings of autobuilder machine leaking in, for example + # showing error messages in another language. + sysinfo.sanitize_env() args = docopt.docopt(doc, version=VERSION)