From patchwork Wed Feb 25 21:17:28 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Andr=C3=A9_Erdmann?= X-Patchwork-Id: 443664 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from hemlock.osuosl.org (hemlock.osuosl.org [140.211.166.133]) by ozlabs.org (Postfix) with ESMTP id 2247C140083 for ; Thu, 26 Feb 2015 08:24:51 +1100 (AEDT) Authentication-Results: ozlabs.org; dkim=fail reason="verification failed; unprotected key" header.d=mailerd.de header.i=@mailerd.de header.b=NrCn/xzq; dkim-adsp=none (unprotected policy); dkim-atps=neutral Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id 407CF955B6; Wed, 25 Feb 2015 21:24:50 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id HHOBgLS7IdxB; Wed, 25 Feb 2015 21:24:48 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by hemlock.osuosl.org (Postfix) with ESMTP id 8A24A955A8; Wed, 25 Feb 2015 21:24:48 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from hemlock.osuosl.org (hemlock.osuosl.org [140.211.166.133]) by ash.osuosl.org (Postfix) with ESMTP id A72821C2214 for ; Wed, 25 Feb 2015 21:24:47 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id A1FAE955A8 for ; Wed, 25 Feb 2015 21:24:47 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Dap+8pmwzson for ; Wed, 25 Feb 2015 21:24:46 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.7.6 Received: from mail-wi0-f182.google.com (mail-wi0-f182.google.com [209.85.212.182]) by hemlock.osuosl.org (Postfix) with ESMTPS id 71EF095585 for ; Wed, 25 Feb 2015 21:24:46 +0000 (UTC) Received: by mail-wi0-f182.google.com with SMTP id l15so8211336wiw.3 for ; Wed, 25 Feb 2015 13:24:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mailerd.de; s=mails; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-type:content-transfer-encoding; bh=ZTKG3aI+Z01FFwiWHwCH5MEA/0p//SAxROepGXkX4xk=; b=NrCn/xzqhrNxG+sqVJ/531qhgcAecuFXbCSogXivHMLRV/XXEU6l0Euog6+JGoCrxD OWSbFLL720a3t0o/8Ed6/4DoqNcyaXSo3+rW0LNJLPjjePrPJQksgtQa+xiBbJcL2qBl Z8dFanw/DqfNoJMgxK8UsDl3SYILaWjfNmdX4= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references:mime-version:content-type :content-transfer-encoding; bh=ZTKG3aI+Z01FFwiWHwCH5MEA/0p//SAxROepGXkX4xk=; b=LrAMxyaJEFcRp/lHCoS1eiR8cqdP1t9+5ZmSrWO7cYRsnVnbHEHlf6O98PJU9ni9Yf o+o35XhtR6Q9/p9NlwRmCZe/o5xt2Ba6CAiQaTVR06e0SGv4NHSAtQAst0Wum6RXxDz8 sop5HkrBp29sEXIqmsubrMpN1kiDX6PzsM/6ZLcM1Zz/e5H9hmPM5uFW0BouxYxOCFQ5 T0T+1GCCUgp/s7OJudN/yRqVg7ybRrE6hMzPJmvIsyzYcCMmucrd3tKQWpxD/kZhljyU FZQtTCexYPw18b+Om4vBtmczVEgdQUEk7ft01qx8BtrEoKaIW/QLm+WLYAoF8cbQEfAI UYHg== X-Gm-Message-State: ALoCoQnrOAqbi2nHCNNCJTBuZPYe8hsGPSk6NvUJls+m3EerKrokAcIVGK8XuMMChDEoxSK35tmy X-Received: by 10.180.77.233 with SMTP id v9mr11508992wiw.28.1424899081134; Wed, 25 Feb 2015 13:18:01 -0800 (PST) Received: from localhost.localdomain (dslb-188-105-009-107.188.105.pools.vodafone-ip.de. [188.105.9.107]) by mx.google.com with ESMTPSA id fo9sm26800926wib.16.2015.02.25.13.18.00 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Wed, 25 Feb 2015 13:18:00 -0800 (PST) From: =?UTF-8?q?Andr=C3=A9=20Erdmann?= To: buildroot@buildroot.org Date: Wed, 25 Feb 2015 22:17:28 +0100 Message-Id: <1424899050-24932-12-git-send-email-dywi@mailerd.de> X-Mailer: git-send-email 2.3.0 In-Reply-To: <1424899050-24932-1-git-send-email-dywi@mailerd.de> References: <1424899050-24932-1-git-send-email-dywi@mailerd.de> MIME-Version: 1.0 Cc: thomas.petazzoni@free-electrons.com Subject: [Buildroot] [PATCH 11/13] autobuild-run: move check_requirements() to SystemInfo X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: buildroot-bounces@busybox.net Sender: "buildroot" * build up the list of (additional) essential programs in main() rather than passing the http credentials to check_requirements() * can check for optional progs in fixup_config() without having to list them in sysinfo * it'd be quite easy to dump the prog list to the logfile (for name, have_prog in sysinfo.progs.items(): ...) Signed-off-by: André Erdmann --- scripts/autobuild-run | 120 ++++++++++++++++++++++++++++++-------------------- 1 file changed, 72 insertions(+), 48 deletions(-) diff --git a/scripts/autobuild-run b/scripts/autobuild-run index 29d1bbb..9c78cc5 100755 --- a/scripts/autobuild-run +++ b/scripts/autobuild-run @@ -109,51 +109,69 @@ def check_version(): print("ERROR: script version too old, please upgrade.") sys.exit(1) -def has_prog(name, flags=os.X_OK, env=os.environ): - if not name or name[0] == os.sep: raise ValueError(name) - - prog_path = env.get("PATH", None) - # for windows compatibility, we'd need to take PATHEXT into account - - if prog_path: - for prog_dir in filter(None, prog_path.split(os.pathsep)): - # os.join() not necessary: non-empty prog_dir and name[0] != os.sep - prog = prog_dir + os.sep + name - if os.access(prog, flags): - return True - # -- - return False - -def check_requirements(http_login, http_password): - needed_progs = ["make", "git", "gcc", "timeout"] - missing_requirements = False - - if http_login and http_password: - needed_progs.append("curl") - - for prog in needed_progs: - if not has_prog(prog): - print("ERROR: your system lacks the '%s' program" % prog) - missing_requirements = True - - if missing_requirements: - sys.exit(1) - class SystemInfo: - def __init__(self): - devnull = open(os.devnull, "w") - # _grep_gcj :: str -> int - _grep_gcj = lambda prog: \ - subprocess.call("%s -version | grep gcj" % prog, shell=True, - stdout=devnull, stderr=devnull) + DEFAULT_NEEDED_PROGS = ["make", "git", "gcc", "timeout"] + DEFAULT_OPTIONAL_PROGS = ["bzr", "java", "javac", "jar"] - self.has_bzr = has_prog("bzr") - - self.has_java = has_prog("java") and _grep_gcj("java") == 1 - - self.has_javac = has_prog("javac") and _grep_gcj("javac") == 1 + def __init__(self): + self.needed_progs = list(self.__class__.DEFAULT_NEEDED_PROGS) + self.optional_progs = list(self.__class__.DEFAULT_OPTIONAL_PROGS) + self.progs = {} + + def find_prog(self, name, flags=os.X_OK, env=os.environ): + if not name or name[0] == os.sep: raise ValueError(name) + + prog_path = env.get("PATH", None) + # for windows compatibility, we'd need to take PATHEXT into account + + if prog_path: + for prog_dir in filter(None, prog_path.split(os.pathsep)): + # os.join() not necessary: non-empty prog_dir + # and name[0] != os.sep + prog = prog_dir + os.sep + name + if os.access(prog, flags): + return prog + # -- + return None + + def has(self, prog): + """Checks whether a program is available. + Lazily evaluates missing entries. + + Returns: None if prog not found, else path to the program [evaluates to True] + """ + try: + return self.progs[prog] + except KeyError: + pass + + have_it = self.find_prog(prog) + # java[c] needs special care + if have_it and prog in ('java', 'javac'): + with open(os.devnull, "w") as devnull: + if subprocess.call("%s -version | grep gcj" % prog, shell=True, + stdout=devnull, stderr=devnull) != 1: + have_it = False + # -- + self.progs[prog] = have_it + return have_it + + def check_requirements(self): + do_check_has_prog = self.has + + missing_requirements = False + for prog in self.needed_progs: + if not do_check_has_prog(prog): + print("ERROR: your system lacks the '%s' program" % prog) + missing_requirements = True + + # check optional programs here, + # else they'd get checked by each worker instance + for prog in self.optional_progs: + do_check_has_prog(prog) + + return not missing_requirements - self.has_jar = has_prog("jar") def get_toolchain_configs(): """Fetch and return the possible toolchain configurations @@ -279,14 +297,14 @@ def fixup_config(instance, sysinfo): if "BR2_PACKAGE_MROUTED=y\n" in configlines and \ "BR2_TOOLCHAIN_USES_UCLIBC=y\n" in configlines: configlines.remove("BR2_PACKAGE_MROUTED=y\n") - if "BR2_NEEDS_HOST_JAVA=y\n" in configlines and not sysinfo.has_java: + if "BR2_NEEDS_HOST_JAVA=y\n" in configlines and not sysinfo.has("java"): return False - if "BR2_NEEDS_HOST_JAVAC=y\n" in configlines and not sysinfo.has_javac: + if "BR2_NEEDS_HOST_JAVAC=y\n" in configlines and not sysinfo.has("javac"): return False - if "BR2_NEEDS_HOST_JAR=y\n" in configlines and not sysinfo.has_jar: + if "BR2_NEEDS_HOST_JAR=y\n" in configlines and not sysinfo.has("jar"): return False # python-nfc needs bzr - if 'BR2_PACKAGE_PYTHON_NFC=y\n' in configlines and not sysinfo.has_bzr: + if 'BR2_PACKAGE_PYTHON_NFC=y\n' in configlines and not sysinfo.has("bzr"): return False # The ctng toolchain is affected by PR58854 if 'BR2_PACKAGE_LTTNG_TOOLS=y\n' in configlines and \ @@ -637,10 +655,16 @@ def main(): check_version() sysinfo = SystemInfo() (ninstances, njobs, http_login, http_password, submitter) = config_get() - check_requirements(http_login, http_password) - if http_login is None or http_password is None: + + if http_login and http_password: + sysinfo.needed_progs.append("curl") + else: print("WARN: due to the lack of http login/password details, results will not be submitted") print("WARN: tarballs of results will be kept locally only") + + if not sysinfo.check_requirements(): + sys.exit(1) + def sigterm_handler(signum, frame): os.killpg(os.getpgid(os.getpid()), signal.SIGTERM) sys.exit(1)