From patchwork Thu Aug 1 02:46:15 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Atharva Lele X-Patchwork-Id: 1140197 Return-Path: X-Original-To: incoming-buildroot@patchwork.ozlabs.org Delivered-To: patchwork-incoming-buildroot@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=busybox.net (client-ip=140.211.166.137; helo=fraxinus.osuosl.org; envelope-from=buildroot-bounces@busybox.net; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="G4fAdGGS"; dkim-atps=neutral Received: from fraxinus.osuosl.org (smtp4.osuosl.org [140.211.166.137]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 45zZSj4vdKz9sNF for ; Thu, 1 Aug 2019 12:47:01 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by fraxinus.osuosl.org (Postfix) with ESMTP id 462E785FCB; Thu, 1 Aug 2019 02:47:00 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from fraxinus.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id MnPkpS_d68Z1; Thu, 1 Aug 2019 02:46:59 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by fraxinus.osuosl.org (Postfix) with ESMTP id 78D9585FB6; Thu, 1 Aug 2019 02:46:59 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from fraxinus.osuosl.org (smtp4.osuosl.org [140.211.166.137]) by ash.osuosl.org (Postfix) with ESMTP id B48361BF59D for ; Thu, 1 Aug 2019 02:46:58 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by fraxinus.osuosl.org (Postfix) with ESMTP id AF43C85FB0 for ; Thu, 1 Aug 2019 02:46:58 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from fraxinus.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id O58hJm4KbObQ for ; Thu, 1 Aug 2019 02:46:57 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mail-pg1-f193.google.com (mail-pg1-f193.google.com [209.85.215.193]) by fraxinus.osuosl.org (Postfix) with ESMTPS id 7607585FAE for ; Thu, 1 Aug 2019 02:46:57 +0000 (UTC) Received: by mail-pg1-f193.google.com with SMTP id i18so33181169pgl.11 for ; Wed, 31 Jul 2019 19:46:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=UYtWAsuXso4Wp1+oJXG44fKqkDyv+IXv4cj4CkeG24I=; b=G4fAdGGSMMQZCdWwKcKlDMwjtG5GewxoQslTk3kijFU82zKSBQDhYPP73KGXCtW4Kp g3QMnMTFnpSx5eKJARggCMPXl5berl8c7Z5eqic4io8OzJHq5yvaOghKd5mkz22C4Vgk LSvs+i7FnEQCEWD49INcpUZdJuxNF19cLoHGtNMCV3D+IC1WbK1auZ+NdckByPfuXSNw 872XNOTPQQV+ho8eeLfcS6KGvseNU8cPgwTN0EERlJHwLoNs357MsAZSYEt8DNatVG8G Kj0lMlEBbWhcN7IRB1S5YeWt/cWVqDzhazSrAhAMu++s55WsdFPi3tHOJfpX0+Y/o9oO o3yQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=UYtWAsuXso4Wp1+oJXG44fKqkDyv+IXv4cj4CkeG24I=; b=kozZvWDHZ1vJ9Ge3PWUMix1x/NjEikUoQocrvrT8DoF4KoZNFp/0mfSBbjZRSXFwsv u+Ub4AsPxJe531xvDw2uWwVl34xHfGAL257/DpfST/lLixGyt55CWsH5uKcTK2Mpe+Fn V4NYvlGvNsLlFitSLIQTRgkLXsBQ7Uolx97PM+CJb3+X/sued/Ngm/7epWlLUAEvo+HR yj9N11/MEyWyCrYgTinV6UAUZm0VgYN+8AfOGG72qiFxE4C6tdmY9jOgmM5RbDGVINH1 dVgUY2VmGV+3GTzOfyuL4OUiRjOnZIxiD2O8pqQ5y4H6GGb2MOc2X+tO6VHgvxRb5INQ 5OvA== X-Gm-Message-State: APjAAAUnsWJD/wEG1Z2pTzKcxK8UbUJzuW5xMLdWrMRqMQ0/m/oW4Jyu fQV0fDG9XyrAHzKJx84RL63qmMPiePI= X-Google-Smtp-Source: APXvYqymXfUJ2BFcQflKQ6e3nb9QaU5+qh98dUEn8vLcjzYFm5U+qw02mESPSsggGB4b/oOt+EmYWQ== X-Received: by 2002:a63:c013:: with SMTP id h19mr86679883pgg.108.1564627616574; Wed, 31 Jul 2019 19:46:56 -0700 (PDT) Received: from localhost.localdomain ([123.201.36.76]) by smtp.gmail.com with ESMTPSA id 185sm79857669pfa.170.2019.07.31.19.46.54 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Wed, 31 Jul 2019 19:46:56 -0700 (PDT) From: Atharva Lele To: buildroot@buildroot.org Date: Thu, 1 Aug 2019 08:16:15 +0530 Message-Id: <20190801024643.11024-3-itsatharva@gmail.com> X-Mailer: git-send-email 2.22.0 In-Reply-To: <20190801024643.11024-1-itsatharva@gmail.com> References: <20190801024643.11024-1-itsatharva@gmail.com> MIME-Version: 1.0 Subject: [Buildroot] [PATCH v4 02/30] autobuild-run: move instance variable from kwargs to Builder class X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Atharva Lele , yann.morin.1998@free.fr, thomas.petazzoni@bootlin.com Errors-To: buildroot-bounces@busybox.net Sender: "buildroot" As discussed in the previous patch, these common variables are needed in many functions, and it'll be better to have them as part of the class. This will make the code cleaner and make it easier to introduce variability for reproducibility testing. Succeeding patches will move all variables from kwargs to the Builder constructor. Signed-off-by: Atharva Lele Acked-by: Arnout Vandecappelle (Essensium/Mind) --- Changes v1 -> v2: - Explicitly state class constructor argument --- scripts/autobuild-run | 26 ++++++++++++++------------ 1 file changed, 14 insertions(+), 12 deletions(-) diff --git a/scripts/autobuild-run b/scripts/autobuild-run index 6bd6856..9946ddc 100755 --- a/scripts/autobuild-run +++ b/scripts/autobuild-run @@ -271,6 +271,9 @@ class SystemInfo: return not missing_requirements class Builder: + def __init__(self, instance): + self.instance = instance + def prepare_build(self, **kwargs): """Prepare for the next build of the specified instance @@ -279,7 +282,7 @@ class Builder: code, and cleaning up remaining stuff from previous builds. """ - idir = "instance-%d" % kwargs['instance'] + idir = "instance-%d" % self.instance log = kwargs['log'] log_write(log, "INFO: preparing a new build") @@ -353,7 +356,7 @@ class Builder: def gen_config(self, **kwargs): """Generate a new random configuration.""" - idir = "instance-%d" % kwargs['instance'] + idir = "instance-%d" % self.instance log = kwargs['log'] outputdir = os.path.abspath(os.path.join(idir, "output")) srcdir = os.path.join(idir, "buildroot") @@ -403,7 +406,7 @@ class Builder: """ log = kwargs['log'] - idir = "instance-%d" % kwargs['instance'] + idir = "instance-%d" % self.instance outputdir = os.path.join(idir, "output") srcdir = os.path.join(idir, "buildroot") reproducible_results = os.path.join(outputdir, "results", "reproducible_results") @@ -435,7 +438,7 @@ class Builder: def do_build(self, **kwargs): """Run the build itself""" - idir = "instance-%d" % kwargs['instance'] + idir = "instance-%d" % self.instance log = kwargs['log'] nice = kwargs['nice'] @@ -466,9 +469,9 @@ class Builder: build_monitor.daemon = True build_monitor.start() - kwargs['buildpid'][kwargs['instance']] = sub.pid + kwargs['buildpid'][self.instance] = sub.pid ret = sub.wait() - kwargs['buildpid'][kwargs['instance']] = 0 + kwargs['buildpid'][self.instance] = 0 # If build failed, monitor thread would have exited at this point if monitor_thread_hung_build_flag.is_set(): @@ -500,7 +503,7 @@ class Builder: perform the actual build. """ - idir = "instance-%d" % kwargs['instance'] + idir = "instance-%d" % self.instance outputdir = os.path.abspath(os.path.join(idir, "output")) srcdir = os.path.join(idir, "buildroot") log = kwargs['log'] @@ -538,7 +541,7 @@ class Builder: are available) or stores them locally as tarballs. """ - idir = "instance-%d" % kwargs['instance'] + idir = "instance-%d" % self.instance log = kwargs['log'] outputdir = os.path.abspath(os.path.join(idir, "output")) @@ -690,7 +693,7 @@ class Builder: # No http login/password, keep tarballs locally with open(os.path.join(outputdir, "results.tar.bz2"), 'rb') as f: sha1 = hashlib.sha1(f.read()).hexdigest() - resultfilename = "instance-%d-%s.tar.bz2" % (kwargs['instance'], sha1) + resultfilename = "instance-%d-%s.tar.bz2" % (self.instance, sha1) os.rename(os.path.join(outputdir, "results.tar.bz2"), resultfilename) log_write(log, "INFO: results saved as %s" % resultfilename) @@ -701,7 +704,7 @@ class Builder: results. """ - idir = "instance-%d" % kwargs['instance'] + idir = "instance-%d" % self.instance # If it doesn't exist, create the instance directory if not os.path.exists(idir): @@ -840,9 +843,8 @@ def main(): buildpid = multiprocessing.Array('i', int(args['--ninstances'])) processes = [] for i in range(0, int(args['--ninstances'])): - builder = Builder() + builder = Builder(instance = i) p = multiprocessing.Process(target=builder.run_instance, kwargs=dict( - instance = i, njobs = args['--njobs'], sysinfo = sysinfo, http_url = args['--http-url'],