From patchwork Fri Jun 21 08:47:13 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Atharva Lele X-Patchwork-Id: 1120024 Return-Path: X-Original-To: incoming-buildroot@patchwork.ozlabs.org Delivered-To: patchwork-incoming-buildroot@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=busybox.net (client-ip=140.211.166.133; helo=hemlock.osuosl.org; envelope-from=buildroot-bounces@busybox.net; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="PonSx+1q"; dkim-atps=neutral Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 45VXQp4NvSz9s5c for ; Fri, 21 Jun 2019 18:48:34 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id 826F687F48; Fri, 21 Jun 2019 08:48:23 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Toqokgle0+qF; Fri, 21 Jun 2019 08:48:20 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by hemlock.osuosl.org (Postfix) with ESMTP id 7C7AF87F52; Fri, 21 Jun 2019 08:48:18 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by ash.osuosl.org (Postfix) with ESMTP id 09D821BF2AA for ; Fri, 21 Jun 2019 08:47:57 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id 0755187E2E for ; Fri, 21 Jun 2019 08:47:57 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id RwA84IFr7qdI for ; Fri, 21 Jun 2019 08:47:55 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mail-pf1-f195.google.com (mail-pf1-f195.google.com [209.85.210.195]) by hemlock.osuosl.org (Postfix) with ESMTPS id 0B0AB87D5E for ; Fri, 21 Jun 2019 08:47:55 +0000 (UTC) Received: by mail-pf1-f195.google.com with SMTP id i189so3227214pfg.10 for ; Fri, 21 Jun 2019 01:47:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=XiM3ekyEVQ6sXtN5n4CK/xeODaNpO63OI3rxgiN73fQ=; b=PonSx+1qvvvBllIOhNl5mGW2otcgjXCgN0v9Iat+IyOaVancEMbGVvcgy+E8J7G4+C dWh43Vy7k8FelT4E8JFMDeBBKo8/u53PsCu37YKXRad09ghF3dmvr+lr3ePO1InfXX02 u3YwxKeq0cKioloGpRc3/Quwcx6nAG6H008q4DsKbdORdPIN/sLUChYJL0FhgiobhVgf YR+d8OenX7Ujhm1frLIpzbLY1n5BXklshTxJWHBonETjkxNrVbtNJFTAYPY1qIOffsFb hIwvQiVw8cBjnJ+yEeHCt/QjUswRCZJV90idIJhMinlxdvuetcM5T3miEYETw+tNewdf //jQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=XiM3ekyEVQ6sXtN5n4CK/xeODaNpO63OI3rxgiN73fQ=; b=LR14Gvm0hyuhxEMuQa6BkZc1/R+CB2EFbkE8/lbjJPJLMdAqYgZyBbDC6YvKS8eFsl NCT2DnzgrPyjg2UcFA9KPAp+pBqp8XHuiCnW7UAI6VOaEmO1urZXtiKC8jbLcuhRpvNY rLLdynqaRIzjLt1Wsb6SHDXYzDYcQRveFZ5JAIRwLUytoG7ClzU34y1dE9Evu6rH953I HvFQmHoM/w1dgdZuF3FZvFBfaxS6w0QbZhLZhxkqM6L6LxxunifvzvzLGJeGcnRLlAJH 38oeKRuzgG6ATSpeysenbpoX98Adj+VZyDUCsghhcNaeh9kPa4eLo8UcA3bZC5thnLmQ QUkQ== X-Gm-Message-State: APjAAAXXDvfX4jU669RjF0ZJjdX/0EXCAyd2H+xbJ97A/RRFI6v5xTyX cxzStsKFAfFUQFhKYkcl6rBVvR5aFUrAgg== X-Google-Smtp-Source: APXvYqyw3yd06tsxxusnxYIZaaD7DHdztNz+cl/qs08kuHm1Z4NWVGeHOt51QzBYKli5LX2qd9fO2A== X-Received: by 2002:a63:4415:: with SMTP id r21mr16901174pga.182.1561106874170; Fri, 21 Jun 2019 01:47:54 -0700 (PDT) Received: from localhost.localdomain ([123.201.54.58]) by smtp.gmail.com with ESMTPSA id z186sm1949883pfz.7.2019.06.21.01.47.51 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Fri, 21 Jun 2019 01:47:53 -0700 (PDT) From: Atharva Lele To: buildroot@buildroot.org Date: Fri, 21 Jun 2019 14:17:13 +0530 Message-Id: <20190621084730.16411-2-itsatharva@gmail.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190621084730.16411-1-itsatharva@gmail.com> References: <20190621084730.16411-1-itsatharva@gmail.com> MIME-Version: 1.0 Subject: [Buildroot] [PATCH 02/19] autobuild-run: move instance variable from kwargs to Builder class X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Atharva Lele , yann.morin.1998@free.fr, thomas.petazzoni@bootlin.com Errors-To: buildroot-bounces@busybox.net Sender: "buildroot" As discussed in the previous patch, these common variables are needed in many functions, and it'll be better to have them as part of the class. This will make the code cleaner and make it easier to introduce variability for reproducibility testing. Succeeding patches will move all variables from kwargs to the Builder constructor. Signed-off-by: Atharva Lele Acked-by: Arnout Vandecappelle (Essensium/Mind) --- scripts/autobuild-run | 26 ++++++++++++++------------ 1 file changed, 14 insertions(+), 12 deletions(-) diff --git a/scripts/autobuild-run b/scripts/autobuild-run index 254d1b6..291cd9f 100755 --- a/scripts/autobuild-run +++ b/scripts/autobuild-run @@ -271,6 +271,9 @@ class SystemInfo: return not missing_requirements class Builder: + def __init__(self, instance): + self.instance = instance + def prepare_build(self, **kwargs): """Prepare for the next build of the specified instance @@ -279,7 +282,7 @@ class Builder: code, and cleaning up remaining stuff from previous builds. """ - idir = "instance-%d" % kwargs['instance'] + idir = "instance-%d" % self.instance log = kwargs['log'] log_write(log, "INFO: preparing a new build") @@ -353,7 +356,7 @@ class Builder: def gen_config(self, **kwargs): """Generate a new random configuration.""" - idir = "instance-%d" % kwargs['instance'] + idir = "instance-%d" % self.instance log = kwargs['log'] outputdir = os.path.abspath(os.path.join(idir, "output")) srcdir = os.path.join(idir, "buildroot") @@ -403,7 +406,7 @@ class Builder: """ log = kwargs['log'] - idir = "instance-%d" % kwargs['instance'] + idir = "instance-%d" % self.instance outputdir = os.path.join(idir, "output") srcdir = os.path.join(idir, "buildroot") reproducible_results = os.path.join(outputdir, "results", "reproducible_results") @@ -435,7 +438,7 @@ class Builder: def do_build(self, **kwargs): """Run the build itself""" - idir = "instance-%d" % kwargs['instance'] + idir = "instance-%d" % self.instance log = kwargs['log'] nice = kwargs['nice'] @@ -466,9 +469,9 @@ class Builder: build_monitor.daemon = True build_monitor.start() - kwargs['buildpid'][kwargs['instance']] = sub.pid + kwargs['buildpid'][self.instance] = sub.pid ret = sub.wait() - kwargs['buildpid'][kwargs['instance']] = 0 + kwargs['buildpid'][self.instance] = 0 # If build failed, monitor thread would have exited at this point if monitor_thread_hung_build_flag.is_set(): @@ -500,7 +503,7 @@ class Builder: perform the actual build. """ - idir = "instance-%d" % kwargs['instance'] + idir = "instance-%d" % self.instance outputdir = os.path.abspath(os.path.join(idir, "output")) srcdir = os.path.join(idir, "buildroot") log = kwargs['log'] @@ -538,7 +541,7 @@ class Builder: are available) or stores them locally as tarballs. """ - idir = "instance-%d" % kwargs['instance'] + idir = "instance-%d" % self.instance log = kwargs['log'] outputdir = os.path.abspath(os.path.join(idir, "output")) @@ -690,7 +693,7 @@ class Builder: # No http login/password, keep tarballs locally with open(os.path.join(outputdir, "results.tar.bz2"), 'rb') as f: sha1 = hashlib.sha1(f.read()).hexdigest() - resultfilename = "instance-%d-%s.tar.bz2" % (kwargs['instance'], sha1) + resultfilename = "instance-%d-%s.tar.bz2" % (self.instance, sha1) os.rename(os.path.join(outputdir, "results.tar.bz2"), resultfilename) log_write(log, "INFO: results saved as %s" % resultfilename) @@ -701,7 +704,7 @@ class Builder: results. """ - idir = "instance-%d" % kwargs['instance'] + idir = "instance-%d" % self.instance # If it doesn't exist, create the instance directory if not os.path.exists(idir): @@ -840,9 +843,8 @@ def main(): buildpid = multiprocessing.Array('i', int(args['--ninstances'])) processes = [] for i in range(0, int(args['--ninstances'])): - builder = Builder() + builder = Builder(i) p = multiprocessing.Process(target=builder.run_instance, kwargs=dict( - instance = i, njobs = args['--njobs'], sysinfo = sysinfo, http_url = args['--http-url'],