From patchwork Tue Aug 6 18:12:46 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Atharva Lele X-Patchwork-Id: 1143009 Return-Path: X-Original-To: incoming-buildroot@patchwork.ozlabs.org Delivered-To: patchwork-incoming-buildroot@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=busybox.net (client-ip=140.211.166.133; helo=hemlock.osuosl.org; envelope-from=buildroot-bounces@busybox.net; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="i1NjipX0"; dkim-atps=neutral Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4632n05YsXz9sMr for ; Wed, 7 Aug 2019 04:13:08 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id E50ED87CDB; Tue, 6 Aug 2019 18:13:06 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id odPEJGVubnPO; Tue, 6 Aug 2019 18:13:06 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by hemlock.osuosl.org (Postfix) with ESMTP id 57390878BB; Tue, 6 Aug 2019 18:13:06 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by ash.osuosl.org (Postfix) with ESMTP id 21F2B1BF2F4 for ; Tue, 6 Aug 2019 18:13:05 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id 1CCC5878BB for ; Tue, 6 Aug 2019 18:13:05 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id lSfdCzLnVkhU for ; Tue, 6 Aug 2019 18:13:04 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mail-pg1-f193.google.com (mail-pg1-f193.google.com [209.85.215.193]) by hemlock.osuosl.org (Postfix) with ESMTPS id 037888780D for ; Tue, 6 Aug 2019 18:13:04 +0000 (UTC) Received: by mail-pg1-f193.google.com with SMTP id n4so277801pgv.2 for ; Tue, 06 Aug 2019 11:13:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=wdFFKar1h/E68wTdJv5hRh+itemOQNJ3tPPNcPwCvOI=; b=i1NjipX0PfMeJrDllGiFKeIIn08gSPJJIgYsQ7L5nUlwCrFT+6toOqHDhfYZbczeDs pYHgyv6RifkM3scDn0jdR+ANFj7BRUU+H1ymu8CCHNgDTWYYaqMp5fg1IQ+v1bNQTonq vao/veL0qWgOcLAwSyeV5tLEuOVmo0F44GEDQRXaS1U64+yoYcxOCCPLLbzZ7mB/qFvj V2xz4ITeCHNkhrrBycvfJxHNfWk+f5vnWBl/uHAscCjGhrziV8dci702BIsUuLtIb7/u wTNQ4+YHcA51/4XY8WUO/glTSR5hZcSQIzMLsdZdnHDon4ftcY4GAkvnik94FXJEv5hS as1g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=wdFFKar1h/E68wTdJv5hRh+itemOQNJ3tPPNcPwCvOI=; b=dFTZGNy4myqMr385KyEnHZuv2ZYAa+Uju8Z8fZO2zhffIy5bh9M78+ARGjqkmG1fmW iMhQyGvtqDyOlntxwgmUay+2VYq+ZTPZ0FWnnff0aqRFp7c2lFA351nqBf/b43GdpSKa EwTOLuNJl5fZRBTqQlGDtKfhbhsauxAm7m8b13KOQ9XEPwPPYU0wod4fyqpIX/yH/bCP cmbEJqw/7OtnKiu8XdY/ffrjCEyUdpfiWUm3e+xd0oeN4kGsVMX7y0H1PLT8j5d42zJw +rfjVW7AAu+vtzy0T9PbHlNgHs0aPAPmZHx0zBpdmOo+Z9V7Sall8/wJ4enuySAWqK0F C0sA== X-Gm-Message-State: APjAAAX1ddIV8ovpTXNvtmtUa0VAzVIuiGrFAff0HVH6krdW7enMlW59 d7LUFq+bCabbqDuDsFZ/+S4p1oxAQ/4= X-Google-Smtp-Source: APXvYqz++nZHuoPa2CsqmfNS/FUi6Ket8IrCs4vclK8x/DINEUa5HvDJykOkCQ/VlcUJMsxzBtWxew== X-Received: by 2002:a62:5c01:: with SMTP id q1mr5059444pfb.53.1565115183222; Tue, 06 Aug 2019 11:13:03 -0700 (PDT) Received: from localhost.localdomain ([123.201.52.51]) by smtp.gmail.com with ESMTPSA id f3sm144563777pfg.165.2019.08.06.11.13.00 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Tue, 06 Aug 2019 11:13:02 -0700 (PDT) From: Atharva Lele To: buildroot@buildroot.org Date: Tue, 6 Aug 2019 23:42:46 +0530 Message-Id: <20190806181251.21885-1-itsatharva@gmail.com> X-Mailer: git-send-email 2.22.0 MIME-Version: 1.0 Subject: [Buildroot] [PATCH 1/6] autobuild-run: use different output directories for reproducible builds testing X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Atharva Lele , yann.morin.1998@free.fr, thomas.petazzoni@bootlin.com Errors-To: buildroot-bounces@busybox.net Sender: "buildroot" Having different output directories increases variation in the build. This commit also changes the directory name of the single output directory to "output-1", as having varied length names for output directories introduces a lot of reproducibility issues. Official Debian efforts also currently use same length names for different directories: "/build/1st/$pkg-$ver" and "/build/$pkg-$ver/2nd" Cfr: https://tests.reproducible-builds.org/debian/index_variations.html Signed-off-by: Atharva Lele --- scripts/autobuild-run | 22 +++++++++++++--------- 1 file changed, 13 insertions(+), 9 deletions(-) diff --git a/scripts/autobuild-run b/scripts/autobuild-run index 2afa865..45b7d2b 100755 --- a/scripts/autobuild-run +++ b/scripts/autobuild-run @@ -275,7 +275,10 @@ class Builder: # path to the output directory here is not relative to the # Buildroot sources, but to the location of the autobuilder # script. - self.outputdir = os.path.abspath(os.path.join(self.idir, "output")) + self.outputdir = os.path.abspath(os.path.join(self.idir, "output-1")) + # We use a different output directory for more variance in reproducibility + # testing. + self.outputdir_2 = os.path.abspath(os.path.join(self.idir, "output-2")) self.resultdir = os.path.join(self.outputdir, "results") # If it doesn't exist, create the instance directory @@ -434,8 +437,8 @@ class Builder: reproducible_results = os.path.join(self.outputdir, "results", "reproducible_results") # Using only tar images for now - build_1_image = os.path.join(self.outputdir, "images-1", "rootfs.tar") - build_2_image = os.path.join(self.outputdir, "images", "rootfs.tar") + build_1_image = os.path.join(self.outputdir, "images", "rootfs.tar") + build_2_image = os.path.join(self.outputdir_2, "images", "rootfs.tar") with open(reproducible_results, 'w') as diff: if self.sysinfo.has("diffoscope"): @@ -522,16 +525,17 @@ class Builder: log_write(self.log, "INFO: build 1 failed, skipping build 2") return ret - # First build has been built, move files and start build 2 - os.rename(os.path.join(self.outputdir, "images"), os.path.join(self.outputdir, "images-1")) + # Create the second output directory if it does not exist + if not os.path.exists(self.outputdir_2): + os.mkdir(self.outputdir_2) - # Clean up build 1 - f = open(os.path.join(self.outputdir, "logfile"), "w+") - subprocess.call(["make", "O=%s" % self.outputdir, "-C", self.srcdir, "clean"], stdout=f, stderr=f) + # Copy .config to the other output directory + shutil.copyfile(os.path.join(self.outputdir, ".config"), + os.path.join(self.outputdir_2, ".config")) # Start the second build log_write(self.log, "INFO: Reproducible Build Test, starting build 2") - ret = self.do_build(self.outputdir) + ret = self.do_build(self.outputdir_2) if ret != 0: log_write(self.log, "INFO: build 2 failed") return ret From patchwork Tue Aug 6 18:12:47 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Atharva Lele X-Patchwork-Id: 1143010 Return-Path: X-Original-To: incoming-buildroot@patchwork.ozlabs.org Delivered-To: patchwork-incoming-buildroot@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=busybox.net (client-ip=140.211.166.133; helo=hemlock.osuosl.org; envelope-from=buildroot-bounces@busybox.net; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="KFYpb+Dj"; dkim-atps=neutral Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4632n45mqPz9sMr for ; Wed, 7 Aug 2019 04:13:12 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id 3A7008780D; Tue, 6 Aug 2019 18:13:09 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id VMaS4b5Li10k; Tue, 6 Aug 2019 18:13:08 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by hemlock.osuosl.org (Postfix) with ESMTP id A2DC887D09; Tue, 6 Aug 2019 18:13:08 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from fraxinus.osuosl.org (smtp4.osuosl.org [140.211.166.137]) by ash.osuosl.org (Postfix) with ESMTP id BC4A91BF2F4 for ; Tue, 6 Aug 2019 18:13:06 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by fraxinus.osuosl.org (Postfix) with ESMTP id B8DC185910 for ; Tue, 6 Aug 2019 18:13:06 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from fraxinus.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id nbvlVkXYw-Xp for ; Tue, 6 Aug 2019 18:13:06 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mail-pf1-f194.google.com (mail-pf1-f194.google.com [209.85.210.194]) by fraxinus.osuosl.org (Postfix) with ESMTPS id 4F89785691 for ; Tue, 6 Aug 2019 18:13:06 +0000 (UTC) Received: by mail-pf1-f194.google.com with SMTP id y15so41949395pfn.5 for ; Tue, 06 Aug 2019 11:13:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=emYxr4Y2WQ2GYLfO2AiYjuwNucyDq9j8Sg76bET+TE8=; b=KFYpb+DjoDL076L+2wfYZM93D3+EOP8JXs4wQMybWC7Ix0csdheLD1z5uyFvy3uJAx ho+27nutxIFO3VMNzPLJ+4L0RJbUPqMOBpF3qHnh2NgsSu8mbqiq5Ai+FjYBcQWq0Ayz Xw95ZDsp+I4UMGdXqU4GUB/ZbpnOXIovbJ963k8fp/l57z41xpZ/jgp2jey2L9WOhi4s bhpPYXNDXKCuKdmvh5npQO8XxmyO6+Or9DxJ5/KH6Pca00zKqVzfE8oYlnxISEyOv3jp lQT3dE0fdiJHbjTlsRKZMuwGDAC/1LLD6/yj0n7WkQxIbTTkaldfslJFOCueBTlL/3tR S77w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=emYxr4Y2WQ2GYLfO2AiYjuwNucyDq9j8Sg76bET+TE8=; b=fZeVfH3dQcT7MD+QscDfxmEIl4Q0rPElrV8wr+yrMV/uaRlB1Cnmlqsiy7V4yJMleM NWO/3X/JTbP7+2FCo7JMJFXKmaG/Ef1LgYBUmw2o+++o88XyUs2pHavKWslWgCJ17i3c 5T38MR4PeCTHYMdmsvTVqSYHHtvhvtF+tw4lw/gwupx9AVYo5dwHXJ0NOFTqasSzDIJ8 qfT38fkJeteYwN/TcKXMn4O7IVhhgwHvnRP8vonz/xUMfaSll6b9XE5pQm5bKnM+OOLe 3PcMi7Wh/OmMKQ40Jwtw8CMEwtK1X4lxA2MMZXwQjZXWVJMSv0Duxjpp2elb5a40LBjM cm3A== X-Gm-Message-State: APjAAAUkCmEvazA1QF3F9vik+dOeo1QGJLh+Bb77/VRMavt1HyUTB6W2 0ukNtAN0UzFrJKDM8t8j1iN+4PrpOP8= X-Google-Smtp-Source: APXvYqwxl3t8EGm9OhvhE9lvV6viln8Ejfl2Sv3AiwdRvmi4Vgn/W1QTbLOxBUHVErhQI6UZ22ffmw== X-Received: by 2002:a62:198d:: with SMTP id 135mr4963145pfz.169.1565115185502; Tue, 06 Aug 2019 11:13:05 -0700 (PDT) Received: from localhost.localdomain ([123.201.52.51]) by smtp.gmail.com with ESMTPSA id f3sm144563777pfg.165.2019.08.06.11.13.03 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Tue, 06 Aug 2019 11:13:05 -0700 (PDT) From: Atharva Lele To: buildroot@buildroot.org Date: Tue, 6 Aug 2019 23:42:47 +0530 Message-Id: <20190806181251.21885-2-itsatharva@gmail.com> X-Mailer: git-send-email 2.22.0 In-Reply-To: <20190806181251.21885-1-itsatharva@gmail.com> References: <20190806181251.21885-1-itsatharva@gmail.com> MIME-Version: 1.0 Subject: [Buildroot] [PATCH 2/6] autobuild-run: make prepare_build() clean the output directory used for reproducibility testing X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Atharva Lele , yann.morin.1998@free.fr, thomas.petazzoni@bootlin.com Errors-To: buildroot-bounces@busybox.net Sender: "buildroot" The previous patch introduced a new output directory for the second build in a reproducibile build test. This patch will make sure that it is cleaned before the subsequent build process begins. Signed-off-by: Atharva Lele --- scripts/autobuild-run | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/scripts/autobuild-run b/scripts/autobuild-run index 45b7d2b..ab49dcb 100755 --- a/scripts/autobuild-run +++ b/scripts/autobuild-run @@ -383,6 +383,10 @@ class Builder: # shutil.rmtree doesn't remove write-protected files subprocess.call(["rm", "-rf", self.outputdir]) os.mkdir(self.outputdir) + + # If it exists, remove the other output directory used for reproducibility testing + if os.path.exists(self.outputdir_2): + subprocess.call(["rm", "-rf", self.outputdir_2]) with open(os.path.join(self.outputdir, "branch"), "w") as branchf: branchf.write(branch) From patchwork Tue Aug 6 18:12:48 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Atharva Lele X-Patchwork-Id: 1143011 Return-Path: X-Original-To: incoming-buildroot@patchwork.ozlabs.org Delivered-To: patchwork-incoming-buildroot@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=busybox.net (client-ip=140.211.166.138; helo=whitealder.osuosl.org; envelope-from=buildroot-bounces@busybox.net; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="ie01P3Kz"; dkim-atps=neutral Received: from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4632n64yfBz9sMr for ; Wed, 7 Aug 2019 04:13:14 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by whitealder.osuosl.org (Postfix) with ESMTP id 7CB9686912; Tue, 6 Aug 2019 18:13:11 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from whitealder.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id CAQfqBm5elXr; Tue, 6 Aug 2019 18:13:10 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by whitealder.osuosl.org (Postfix) with ESMTP id 00A3D8602E; Tue, 6 Aug 2019 18:13:10 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by ash.osuosl.org (Postfix) with ESMTP id 457B81BF2F4 for ; Tue, 6 Aug 2019 18:13:09 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by whitealder.osuosl.org (Postfix) with ESMTP id 42E478602E for ; Tue, 6 Aug 2019 18:13:09 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from whitealder.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 4pJXwayImauk for ; Tue, 6 Aug 2019 18:13:08 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mail-pg1-f196.google.com (mail-pg1-f196.google.com [209.85.215.196]) by whitealder.osuosl.org (Postfix) with ESMTPS id A80CD8601E for ; Tue, 6 Aug 2019 18:13:08 +0000 (UTC) Received: by mail-pg1-f196.google.com with SMTP id r26so6052648pgl.10 for ; Tue, 06 Aug 2019 11:13:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=LYqkCxqF41XumlDL+1DWI+nw1Klgm4l2EXDEXiAYgZ0=; b=ie01P3KzAy9pLmk7R6GLdRoTP2hAejKh3051cjKRyC7/nVSHzxVfjP24QZ/2+6Qg0T UIqbBW4WtorjXrTX8F3SGUFAERiga6bSfYZGi9feKbGP8KXZMoS1Rl9uM3c8+DwipiYL i76QQyU8vf4KDwYjkNLH6PMigOlg/w9SoBT9kmwSMwzh/lCy5+UTE3gEfxZcpGuBnVe+ x8wbLlIdTT0EOqTndjEfbI4OxRCfs+vRrymats9BSJldgQB82W+l3nwUSdWmSxIDlu/t NoD1myjJsXK92Vr7u8et+iykQNpFmVMCdI6+2gNxeJe6iR2eYoN2VV8BYD+5Fz5Y1uQq vILw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=LYqkCxqF41XumlDL+1DWI+nw1Klgm4l2EXDEXiAYgZ0=; b=VJljA7RMtGBu1/cYw6iayPOLgVFh7PRL5yJKPVphDVeqHy9e97X7oYY0zhGg+MAFgq DeUr3POo9Fc3nGv88aAdORYJzvq5uHmPLtHVO6xezfK/GCKB4lCTVtWldAHYIgx7rjta 1HmvS9cb8bSukQiDON9O01vTwH9KfZEUklM02SzURE2ejEO+9+cq5zlDhHvTzC9poA3U 2PxcvYgaadhBkNwbfjkZD+ccHNsA9hAUrS9AjnqK1PXq5u8bx6tNIfEiNkW78PCK8rTG KSzWzYEEgY5dR3L+CHJIkqR7abooCYCHBiMkTktEmrTAsOCf7MHVA+8SGtB1T+2RTd/a q6mA== X-Gm-Message-State: APjAAAUi3zeAZ4pVgVoqKK6CcJ5R+c/ZcN5ms5uh8r5GiXA9VQcPlp+x Lk/k4F1JGRVouGEPk8QnghbKPyb9jVg= X-Google-Smtp-Source: APXvYqyD7Kh5gTuvTh0SfuLWdUI3zSDy4BXVeU2U6tn754xfP4tIpXLXqMfDEFTIvyxcltH0EHLTMw== X-Received: by 2002:a63:2c8:: with SMTP id 191mr4095241pgc.139.1565115187778; Tue, 06 Aug 2019 11:13:07 -0700 (PDT) Received: from localhost.localdomain ([123.201.52.51]) by smtp.gmail.com with ESMTPSA id f3sm144563777pfg.165.2019.08.06.11.13.05 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Tue, 06 Aug 2019 11:13:07 -0700 (PDT) From: Atharva Lele To: buildroot@buildroot.org Date: Tue, 6 Aug 2019 23:42:48 +0530 Message-Id: <20190806181251.21885-3-itsatharva@gmail.com> X-Mailer: git-send-email 2.22.0 In-Reply-To: <20190806181251.21885-1-itsatharva@gmail.com> References: <20190806181251.21885-1-itsatharva@gmail.com> MIME-Version: 1.0 Subject: [Buildroot] [PATCH 3/6] autobuild-run: fix cross tools prefix for diffoscope X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Atharva Lele , yann.morin.1998@free.fr, thomas.petazzoni@bootlin.com Errors-To: buildroot-bounces@busybox.net Sender: "buildroot" Make outputs "make: entering directory" when called from a different directory which is unnecessary and yields a wrong prefix. Adding this argument supresses unnecessary output and yields the right prefix. Before and after: https://gitlab.com/snippets/1877119 Signed-off-by: Atharva Lele --- scripts/autobuild-run | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/scripts/autobuild-run b/scripts/autobuild-run index ab49dcb..69766b2 100755 --- a/scripts/autobuild-run +++ b/scripts/autobuild-run @@ -447,7 +447,8 @@ class Builder: with open(reproducible_results, 'w') as diff: if self.sysinfo.has("diffoscope"): # Prefix to point diffoscope towards cross-tools - prefix = subprocess.check_output(["make", "O=%s" % self.outputdir, "-C", self.srcdir, "printvars", "VARS=TARGET_CROSS"]) + prefix = subprocess.check_output(["make", "--no-print-directory", "O=%s" % self.outputdir, + "-C", self.srcdir, "printvars", "VARS=TARGET_CROSS"]) # Remove TARGET_CROSS= and \n from the string prefix = prefix[13:-1] log_write(self.log, "INFO: running diffoscope on images") From patchwork Tue Aug 6 18:12:49 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Atharva Lele X-Patchwork-Id: 1143012 Return-Path: X-Original-To: incoming-buildroot@patchwork.ozlabs.org Delivered-To: patchwork-incoming-buildroot@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=busybox.net (client-ip=140.211.166.138; helo=whitealder.osuosl.org; envelope-from=buildroot-bounces@busybox.net; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="UEptQXBW"; dkim-atps=neutral Received: from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4632nC0Nbvz9sMr for ; Wed, 7 Aug 2019 04:13:19 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by whitealder.osuosl.org (Postfix) with ESMTP id D9D2386A16; Tue, 6 Aug 2019 18:13:15 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from whitealder.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id m8fcLs+m8Yd7; Tue, 6 Aug 2019 18:13:13 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by whitealder.osuosl.org (Postfix) with ESMTP id E27D58660D; Tue, 6 Aug 2019 18:13:12 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from fraxinus.osuosl.org (smtp4.osuosl.org [140.211.166.137]) by ash.osuosl.org (Postfix) with ESMTP id 768941BF2F4 for ; Tue, 6 Aug 2019 18:13:11 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by fraxinus.osuosl.org (Postfix) with ESMTP id 7403485910 for ; Tue, 6 Aug 2019 18:13:11 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from fraxinus.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id IKtn7HH07bEf for ; Tue, 6 Aug 2019 18:13:11 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mail-pf1-f193.google.com (mail-pf1-f193.google.com [209.85.210.193]) by fraxinus.osuosl.org (Postfix) with ESMTPS id 101F685691 for ; Tue, 6 Aug 2019 18:13:11 +0000 (UTC) Received: by mail-pf1-f193.google.com with SMTP id m30so41946146pff.8 for ; Tue, 06 Aug 2019 11:13:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=AcCERNO+Zg+OMCb+Jym5K4IUu8FzYFR90vmc4k9rcWI=; b=UEptQXBWdo0nXdUYp41J1yyQygGHAcf3MXR3WrBQu7Mu0iRh1B/+Op2PeIyyT+9oGN QmtYivCL7gdKpMxqNE2YlgArV6zBT/sOHBIfBm5zJ3ppZPmPQAtggS0Vecs8t/X8MKSu qh3PpxOJ/qV5RlszEFotPnqdTglJhO4r+3eCcUfZNersoUoGPLWOaXnUsnGFe7MJKXf1 Cex2LhjryJVf+9RMr5fUemspf528gV1dSK5iSmLVR+4OOhgVbr3NCotowpk23SahN2xD wschxIXIqNDFJs5smacJnMt8YyWkx1d/URx4Qo8mV13poAGHlCnpnqFbsGqht5Ctbotr pv7Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=AcCERNO+Zg+OMCb+Jym5K4IUu8FzYFR90vmc4k9rcWI=; b=gkd3jqxgSKOnXCbttkZTCK9OoxXSh0tK0w7ry3HGTXOm4VyGr5KHBRcH3Bpy9NFEe1 EJxxB9J+3hXW4QO3kTZDG3FyY5oS8Adq3Hf0xtzi9XeWyhrJZUmsJDLLAZzvM6n6puqU UlETcuP9VLdmTtI4jHMJfri3rh33MyHCGiyMg3JP7LftWNwc/jMnaBbt8o0MSZuURfKB 9XDZ4ZBXCEqfw41POHz1CkNsz6DIIqS4KvtQm1XKYWE/hVzNx9byIIKXJAXiad5VkS6a 9QigDUgcZq/y4FbdcjOM1d3gy79nF1b/pE8AaBH8aF8s+YhCK6Nt809CpQKv/646VgX3 vIPA== X-Gm-Message-State: APjAAAUXVEtBGlZp2vvtl6QBU8vGLK3MsLtTgmiA0T0pwvtmFhfmZHUd Kx/bsT+nbPYNqJfjm4W6LvWZ7JRxYo4= X-Google-Smtp-Source: APXvYqxrOa25QeAtnQVYm5g387SxCtr51lr1p/bCr6tPfaHXGbLxhSPRA6qLLoC467Ud9I4faTkpVQ== X-Received: by 2002:a63:c246:: with SMTP id l6mr4231326pgg.210.1565115190152; Tue, 06 Aug 2019 11:13:10 -0700 (PDT) Received: from localhost.localdomain ([123.201.52.51]) by smtp.gmail.com with ESMTPSA id f3sm144563777pfg.165.2019.08.06.11.13.08 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Tue, 06 Aug 2019 11:13:09 -0700 (PDT) From: Atharva Lele To: buildroot@buildroot.org Date: Tue, 6 Aug 2019 23:42:49 +0530 Message-Id: <20190806181251.21885-4-itsatharva@gmail.com> X-Mailer: git-send-email 2.22.0 In-Reply-To: <20190806181251.21885-1-itsatharva@gmail.com> References: <20190806181251.21885-1-itsatharva@gmail.com> MIME-Version: 1.0 Subject: [Buildroot] [PATCH 4/6] autobuild-run: make diffoscope output to a JSON-formatted file X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Atharva Lele , yann.morin.1998@free.fr, thomas.petazzoni@bootlin.com Errors-To: buildroot-bounces@busybox.net Sender: "buildroot" Normal diffoscope output is readable by humans but not really convenient when working with it in code. JSON can be easily read and extracted information from. Signed-off-by: Atharva Lele --- scripts/autobuild-run | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/autobuild-run b/scripts/autobuild-run index 69766b2..520cfe2 100755 --- a/scripts/autobuild-run +++ b/scripts/autobuild-run @@ -453,7 +453,7 @@ class Builder: prefix = prefix[13:-1] log_write(self.log, "INFO: running diffoscope on images") subprocess.call(["diffoscope", build_1_image, build_2_image, - "--tool-prefix-binutils", prefix], stdout=diff, stderr=self.log) + "--tool-prefix-binutils", prefix, "--json", "-"], stdout=diff, stderr=self.log) else: log_write(self.log, "INFO: diffoscope not installed, falling back to cmp") subprocess.call(["cmp", "-b", build_1_image, build_2_image], stdout=diff, stderr=self.log) From patchwork Tue Aug 6 18:12:50 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Atharva Lele X-Patchwork-Id: 1143013 Return-Path: X-Original-To: incoming-buildroot@patchwork.ozlabs.org Delivered-To: patchwork-incoming-buildroot@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=busybox.net (client-ip=140.211.166.137; helo=fraxinus.osuosl.org; envelope-from=buildroot-bounces@busybox.net; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="MMcB9WMr"; dkim-atps=neutral Received: from fraxinus.osuosl.org (smtp4.osuosl.org [140.211.166.137]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4632nK36kWz9sMr for ; Wed, 7 Aug 2019 04:13:24 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by fraxinus.osuosl.org (Postfix) with ESMTP id 7B79885E25; Tue, 6 Aug 2019 18:13:20 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from fraxinus.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id P5jY4djYciEt; Tue, 6 Aug 2019 18:13:17 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by fraxinus.osuosl.org (Postfix) with ESMTP id 180C485910; Tue, 6 Aug 2019 18:13:17 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from silver.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by ash.osuosl.org (Postfix) with ESMTP id 599E01BF2F4 for ; Tue, 6 Aug 2019 18:13:15 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by silver.osuosl.org (Postfix) with ESMTP id 5722F2154E for ; Tue, 6 Aug 2019 18:13:15 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from silver.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id j73nhkrxIVTE for ; Tue, 6 Aug 2019 18:13:13 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mail-pg1-f195.google.com (mail-pg1-f195.google.com [209.85.215.195]) by silver.osuosl.org (Postfix) with ESMTPS id 5B59A20115 for ; Tue, 6 Aug 2019 18:13:13 +0000 (UTC) Received: by mail-pg1-f195.google.com with SMTP id n190so1197044pgn.0 for ; Tue, 06 Aug 2019 11:13:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=24tFgtZ93lJIBukOMUVf0QeTnNZJ/KjEvP5WiT2pb+M=; b=MMcB9WMrWlKhoz0rkARtD7SrZ9Lpw5zlPe+QFPu6rMFr6EY59PUYnwMQycbfD3qtG+ cu0l5r+F6HyVrW/9+qut0BPWPwbtE+0LoZ7SPUBSwLkYBxVuePeDSz0fXVY1WQtJnsaW xO2uFx3iq+cGRArErOGjox09GZIsHiJUnzplmrpRfl0x2fZ8z96FDHk4G1rvqNjmNlBJ KIFIRNzVHNTgxRQGxJ6lTlkWtPBHghJpKhB3pzwqTR5qOHgrTELBfErgWsR14SF3hZdt S56+l8icASOiP/b0LOrzE+Bnh0eFLOGLnVvO+FjVZJCnLVm2SUj3CQMedqSStdoBcFuk uxdw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=24tFgtZ93lJIBukOMUVf0QeTnNZJ/KjEvP5WiT2pb+M=; b=FC/BNN0PtaS8Cdj700blv10onEj5jRwNtbsknahUiBXG81s+0NeMXvpR10rtQE+NWt 8Hu5RURG5SGrGe8HLS0E+brvU6YKzce5wq+yCoxXxYIyIDTinVm/gJMVbecvzgk9nYWP 6LlIixl7ZXQn4ACixqzKdi09WZlJtVrA3HBbfe62k6Ax24waytVQSadu9VacQtpRbq1o Frc8+UfQHyM+oFgoyVHhimlDLaYw3nROsnpEMHZ8ZUtL7L+VKVBRKKNsQFJdM/lM++tI 0bOVdl4FP4lZHNueUGzd4YZPxWtlnldkQK3Ldz9SlGRgOJwRzLuOBs/IJD89EwCWOA9X lHTw== X-Gm-Message-State: APjAAAWbhi0xLMAI0kf5mzCRu2ikRfUkxHbIvFbVM+sGzTB9HkISkveN iLPHnXOs+l2biHrh6C4YxxLvit5TJZU= X-Google-Smtp-Source: APXvYqzFFjKT074VXn3WTsPImxOW2txzEcwum2egt13aLA2TZ2ULEHlcXdsEhQDyZQgzLKsTOIX6rA== X-Received: by 2002:a63:221f:: with SMTP id i31mr4252061pgi.251.1565115192522; Tue, 06 Aug 2019 11:13:12 -0700 (PDT) Received: from localhost.localdomain ([123.201.52.51]) by smtp.gmail.com with ESMTPSA id f3sm144563777pfg.165.2019.08.06.11.13.10 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Tue, 06 Aug 2019 11:13:12 -0700 (PDT) From: Atharva Lele To: buildroot@buildroot.org Date: Tue, 6 Aug 2019 23:42:50 +0530 Message-Id: <20190806181251.21885-5-itsatharva@gmail.com> X-Mailer: git-send-email 2.22.0 In-Reply-To: <20190806181251.21885-1-itsatharva@gmail.com> References: <20190806181251.21885-1-itsatharva@gmail.com> MIME-Version: 1.0 Subject: [Buildroot] [PATCH 5/6] autobuild-run: initial implementation of get_reproducibility_failure_reason() X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Atharva Lele , yann.morin.1998@free.fr, thomas.petazzoni@bootlin.com Errors-To: buildroot-bounces@busybox.net Sender: "buildroot" Analyze the JSON formatted output from diffoscope and check if the differences are due to a filesystem reproducibility issue or a package reproducibility issue. Also, discard the deltas because they might take up too much space. Signed-off-by: Atharva Lele --- scripts/autobuild-run | 71 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 71 insertions(+) diff --git a/scripts/autobuild-run b/scripts/autobuild-run index 520cfe2..f2f6460 100755 --- a/scripts/autobuild-run +++ b/scripts/autobuild-run @@ -131,6 +131,7 @@ import csv import docopt import errno import hashlib +import json import mmap import multiprocessing import os @@ -596,6 +597,76 @@ class Builder: if reject_results(): return + def get_reproducibility_failure_reason(reproducible_results): + def clean_delta(delta): + added = [] + deleted = [] + for line in delta: + if line.startswith("+"): + added.append(line) + if line.startswith("-"): + deleted.append(line) + return added, deleted + + with open(reproducible_results, "r") as reproduciblef: + json_data = json.load(reproduciblef) + + packages_file_list = os.path.join(self.outputdir, "build", "packages-file-list.txt") + + if json_data["unified_diff"] == None: + if json_data["details"][0]["source1"] == "file list": + json_data["details"].pop(0) + + for i in range(0, len(json_data["details"])): + diff_source = json_data["details"][i]["source1"] + with open(packages_file_list, "r") as packagef: + for line in packagef: + if diff_source in line: + package = line.split(',')[0] + + # Get package version + package_info = json.loads(subprocess.check_output(["make", "--no-print-directory", + "O=%s" % self.outputdir, + "-C", self.srcdir, + "%s-show-info" % package])) + if "version" in package_info[package]: + version = package_info[package]["version"] + reason = [package, version] + else: + reason = [package] + json_data["details"][i]["package"] = reason + json_data["details"][i].pop("source2") + if json_data["details"][i]["unified_diff"] == None: + json_data["details"][i].pop("unified_diff") + for j in range(0, len(json_data["details"][i]["details"])): + delta = json_data["details"][i]["details"][j]["unified_diff"].split("\n") + deltas = clean_delta(delta) + json_data["details"][i]["details"][j]["added"] = deltas[0][:100] + json_data["details"][i]["details"][j]["deleted"] = deltas[1][:100] + try: + json_data["details"][i]["details"][j].pop("unified_diff") + json_data["details"][i]["details"][j].pop("source2") + except KeyError as e: + log_write(self.log, "KeyError: %s not found in JSON details[%d][%d]" % (e, i, j)) + else: + delta = json_data["details"][i]["unified_diff"] + deltas = clean_delta(delta) + json_data["details"][i]["added"] = deltas[0][:100] + json_data["details"][i]["deleted"] = deltas[1][:100] + try: + json_data["details"][i].pop("unified_diff") + json_data["details"][i].pop("source2") + except KeyError as e: + log_write(self.log, "KeyError: %s not found in JSON details[%d]" % (e, i)) + reason = json_data["details"][0]["package"] + else: + reason = "filesystem" + + with open(reproducible_results, "w") as reproduciblef: + json.dump(json_data, reproduciblef, sort_keys=True, indent=4) + + return reason + def get_failure_reason(): # Output is a tuple (package, version), or None. lastlines = decode_bytes(subprocess.Popen( From patchwork Tue Aug 6 18:12:51 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Atharva Lele X-Patchwork-Id: 1143014 Return-Path: X-Original-To: incoming-buildroot@patchwork.ozlabs.org Delivered-To: patchwork-incoming-buildroot@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=busybox.net (client-ip=140.211.166.136; helo=silver.osuosl.org; envelope-from=buildroot-bounces@busybox.net; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="Bwe90TPP"; dkim-atps=neutral Received: from silver.osuosl.org (smtp3.osuosl.org [140.211.166.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4632nL0Cp4z9sN1 for ; Wed, 7 Aug 2019 04:13:25 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by silver.osuosl.org (Postfix) with ESMTP id 3FFCF21FA8; Tue, 6 Aug 2019 18:13:22 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from silver.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ehEV6UiZxLKy; Tue, 6 Aug 2019 18:13:20 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by silver.osuosl.org (Postfix) with ESMTP id F1B082154E; Tue, 6 Aug 2019 18:13:19 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by ash.osuosl.org (Postfix) with ESMTP id EDA4F1BF2F4 for ; Tue, 6 Aug 2019 18:13:16 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by whitealder.osuosl.org (Postfix) with ESMTP id E9B9186044 for ; Tue, 6 Aug 2019 18:13:16 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from whitealder.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id XlR6wpYCYQ4e for ; Tue, 6 Aug 2019 18:13:15 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mail-pf1-f195.google.com (mail-pf1-f195.google.com [209.85.210.195]) by whitealder.osuosl.org (Postfix) with ESMTPS id BD6BE86030 for ; Tue, 6 Aug 2019 18:13:15 +0000 (UTC) Received: by mail-pf1-f195.google.com with SMTP id t16so41907302pfe.11 for ; Tue, 06 Aug 2019 11:13:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=g72rXIxCQ0mpJLanEnXpdgNsf0LgWshq+q12vhzPPdI=; b=Bwe90TPPOTO6Jm5tdmNOe37wI06OGD8Rg1vG52zsnosoLk2ygdTBBo5p+hd67z+xZS AwLRHPPYQ2u7Vm8MhepeYuve9vshMO2biaUX7VhRjnXygbz4u3uHoW0VDDAlKET7/lIS 5/n8g8/M0z3rsGgcCf63tY4exQj6wE3hy2Xr9KIGLOmngyoNKeHkrirranYvTOAA3JlA M5eJZlWaQLKY73wIAAFzPIvP7pICcvDpmdgMELKBizqeM/DcaSz7qgCZRMVbkKftafyP BKI5hkZTa3tDa49IeSQQRIFmoEK6yGDyhDfoR3HF1xAKQOopqKeGJ1gwhP7auIAX0ixu +b+g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=g72rXIxCQ0mpJLanEnXpdgNsf0LgWshq+q12vhzPPdI=; b=nETtMTaqyfo+8cHmKgSeFLFE3c2W63KDL6BiJscQBgap/Q6s+GXQvkvE+nFlUDoOpl I6NL4FupnZV5TL6YRhZA8Ny3LmMjb1B2pOpJ00rZO3nTlI//i97LwXAkTSWHwLP+iPrM o9neRfzQKVjigIJ9TC4c5p2xM63GvH8AG5HXnm9PndxHzY5e741JJ51HiyHUIjXip8oa V+X/MiUnP5S5QvH4zmhARygWHbiM7Ej158DxqweprWgZhVltX4ARqwo9eW+7rsY1Eqbk OxqtOcgbMJtEvjdAZQqpEYZ36i3p/X9QYwZ7jWZmHW14pkXLhSlwXaSUK2/x2czwCt99 kDLQ== X-Gm-Message-State: APjAAAWi7h9ZnGAVJ42vPFiL6e+7b1WPxyXOd9FhrJmfwbXVJhJ0lbId Ja1ZJsEupVgH2O3GSHFLbwv61Hwl5qg= X-Google-Smtp-Source: APXvYqxNwXlugBq2SH4X2KGSkVayR6nlLltHQUDl2ZTOMwYfdONFBE2dwCkMD3sB+14gv+asSjAdQQ== X-Received: by 2002:a62:ae02:: with SMTP id q2mr4914637pff.1.1565115194993; Tue, 06 Aug 2019 11:13:14 -0700 (PDT) Received: from localhost.localdomain ([123.201.52.51]) by smtp.gmail.com with ESMTPSA id f3sm144563777pfg.165.2019.08.06.11.13.12 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Tue, 06 Aug 2019 11:13:14 -0700 (PDT) From: Atharva Lele To: buildroot@buildroot.org Date: Tue, 6 Aug 2019 23:42:51 +0530 Message-Id: <20190806181251.21885-6-itsatharva@gmail.com> X-Mailer: git-send-email 2.22.0 In-Reply-To: <20190806181251.21885-1-itsatharva@gmail.com> References: <20190806181251.21885-1-itsatharva@gmail.com> MIME-Version: 1.0 Subject: [Buildroot] [PATCH 6/6] autobuild-run: account for reproducibility failures in get_failure_reason() X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Atharva Lele , yann.morin.1998@free.fr, thomas.petazzoni@bootlin.com Errors-To: buildroot-bounces@busybox.net Sender: "buildroot" Signed-off-by: Atharva Lele --- scripts/autobuild-run | 29 ++++++++++++++++++++--------- 1 file changed, 20 insertions(+), 9 deletions(-) diff --git a/scripts/autobuild-run b/scripts/autobuild-run index f2f6460..1f12355 100755 --- a/scripts/autobuild-run +++ b/scripts/autobuild-run @@ -669,15 +669,26 @@ class Builder: def get_failure_reason(): # Output is a tuple (package, version), or None. - lastlines = decode_bytes(subprocess.Popen( - ["tail", "-n", "3", os.path.join(self.outputdir, "logfile")], - stdout=subprocess.PIPE).communicate()[0]).splitlines() - - regexp = re.compile(r'make: \*\*\* .*/(?:build|toolchain)/([^/]*)/') - for line in lastlines: - m = regexp.search(line) - if m: - return m.group(1).rsplit('-', 1) + # Output is "package-reproducible" in case of reproducibility failure. + + reproducible_results = os.path.join(self.resultdir, "reproducible_results") + if os.path.exists(reproducible_results) and os.stat(reproducible_results).st_size > 0: + if self.sysinfo.has("diffoscope"): + reason = get_reproducibility_failure_reason(reproducible_results) + reason.append("nonreproducible") + return reason + else: + return ["nonreproducible"] + else: + lastlines = decode_bytes(subprocess.Popen( + ["tail", "-n", "3", os.path.join(self.outputdir, "logfile")], + stdout=subprocess.PIPE).communicate()[0]).splitlines() + + regexp = re.compile(r'make: \*\*\* .*/(?:build|toolchain)/([^/]*)/') + for line in lastlines: + m = regexp.search(line) + if m: + return m.group(1).rsplit('-', 1) # not found return None