From patchwork Tue Aug 13 04:12:05 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Atharva Lele X-Patchwork-Id: 1146011 Return-Path: X-Original-To: incoming-buildroot@patchwork.ozlabs.org Delivered-To: patchwork-incoming-buildroot@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=busybox.net (client-ip=140.211.166.137; helo=fraxinus.osuosl.org; envelope-from=buildroot-bounces@busybox.net; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="iJdkSJ9R"; dkim-atps=neutral Received: from fraxinus.osuosl.org (smtp4.osuosl.org [140.211.166.137]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 466zng3X2mz9sNk for ; Tue, 13 Aug 2019 14:12:22 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by fraxinus.osuosl.org (Postfix) with ESMTP id 010B286166; Tue, 13 Aug 2019 04:12:20 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from fraxinus.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id rE7OBL1BCDCf; Tue, 13 Aug 2019 04:12:18 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by fraxinus.osuosl.org (Postfix) with ESMTP id 19B9084BB2; Tue, 13 Aug 2019 04:12:18 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by ash.osuosl.org (Postfix) with ESMTP id 6308C1BF369 for ; Tue, 13 Aug 2019 04:12:16 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id 5FD1187D73 for ; Tue, 13 Aug 2019 04:12:16 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id EaGGp7F2NMYz for ; Tue, 13 Aug 2019 04:12:15 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mail-pf1-f194.google.com (mail-pf1-f194.google.com [209.85.210.194]) by hemlock.osuosl.org (Postfix) with ESMTPS id B9AB787D6A for ; Tue, 13 Aug 2019 04:12:15 +0000 (UTC) Received: by mail-pf1-f194.google.com with SMTP id c81so1808154pfc.11 for ; Mon, 12 Aug 2019 21:12:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=QTPqGR+kDo8mxx2qLKfL+yamqdl53C6onWd3zDpIIEg=; b=iJdkSJ9RTpFzc3pDFyItrmSrKCjcRtBJrqG1qSAHUni90opruGVwET8drPiuxKcCo6 GBU4LWCXLcLsSHkXs4unMxoO/fAgod3wM/CfoV4cMB8nFzcUxDkzh7xmYwtQQqWnY+Uk 7/aHU3zRTTrglzYTMyYFHuaLjol1FmMJL0Q2uwIO3uCYIqpfsXO+dAQbyriyjtE3tbp+ uiMwGpifRuJdsZfH08nM9ewN7gvEBaE0Iv11DJ9E/C9N+/cH3UZBuhLUnvkLfoOo4N1o Ww/3eGNKV9iCZoxQXUr92IZ0+dVPxVRpf3EqQWa4nbQ0YSSxDbkiX1pPNLq+E0ns6h08 +l+Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=QTPqGR+kDo8mxx2qLKfL+yamqdl53C6onWd3zDpIIEg=; b=gYrOdT/OG6uXpizI9rHgDxsnw8EIORMxnbfd0ynt1ek4x1zn/HoPeVTX54yXI2I7Z8 7DGvq5Gz5e34+KpoGmzEchWAWSFWQjOUKZASbq6KnZqI2H2WJ5gv9ffIsnnUqW+w2pUD 8pXrr2DBZP+tUy69QftD0g71goy3kifWc+QS+NDzl3m22ChHfFb7GOp67A6wuTxqogFD nSfoYGylPikj4824ywufa5qtPcvC3Xr6P+k4jTLxfsQMKZlhmIpPkQHlDFliusizuteo Jid4EekW0ozyFtF3tsgXG2qcNMuwS3wkq4tmyfoNpq+++D1UsFqcnEVvt2FbwEIYVwHL VcrA== X-Gm-Message-State: APjAAAXNernREN/ruWZiHHGd9P6kzBEzsBehlJT4JoHEqdAlcIYAhnUy yJK3sYHUxUldYCdL/kMTDPRU7EwnzVs= X-Google-Smtp-Source: APXvYqw4RTQBSXqebiuypZ9wr1EVTb/ZLZsTJtNXgO7Uc/fyl8PLoEaku5qeCCYnYg79yS62IFmYCw== X-Received: by 2002:a63:4823:: with SMTP id v35mr32918796pga.138.1565669534796; Mon, 12 Aug 2019 21:12:14 -0700 (PDT) Received: from localhost.localdomain ([123.201.52.15]) by smtp.gmail.com with ESMTPSA id s5sm91946132pfm.97.2019.08.12.21.12.12 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Mon, 12 Aug 2019 21:12:14 -0700 (PDT) From: Atharva Lele To: buildroot@buildroot.org Date: Tue, 13 Aug 2019 09:42:05 +0530 Message-Id: <20190813041206.20715-2-itsatharva@gmail.com> X-Mailer: git-send-email 2.22.0 In-Reply-To: <20190813041206.20715-1-itsatharva@gmail.com> References: <20190813041206.20715-1-itsatharva@gmail.com> MIME-Version: 1.0 Subject: [Buildroot] [PATCH v3 2/3] autobuild-run: initial implementation of get_reproducibility_failure_reason() X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Atharva Lele , yann.morin.1998@free.fr, thomas.petazzoni@bootlin.com Errors-To: buildroot-bounces@busybox.net Sender: "buildroot" Analyze the JSON formatted output from diffoscope and check if the differences are due to a filesystem reproducibility issue or a package reproducibility issue. Also, discard the deltas because they might take up too much space. Signed-off-by: Atharva Lele --- Changes v1 -> v2: - Refactor using subfunctions and local variables (suggested by Thomas) - Added comments (suggested by Thomas) - Use more pythonic loops (suggested by Thomas) --- scripts/autobuild-run | 89 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 89 insertions(+) diff --git a/scripts/autobuild-run b/scripts/autobuild-run index ead81a0..9b8983f 100755 --- a/scripts/autobuild-run +++ b/scripts/autobuild-run @@ -131,6 +131,7 @@ import csv import docopt import errno import hashlib +import json import mmap import multiprocessing import os @@ -599,6 +600,94 @@ class Builder: if reject_results(): return + def get_reproducibility_failure_reason(reproducible_results): + def split_delta(delta): + # Take a delta and split it into added, deleted lines. + added = [] + deleted = [] + for line in delta: + if line.startswith("+"): + added.append(line) + if line.startswith("-"): + deleted.append(line) + return added, deleted + + def get_package(sourcef): + # Returns which package the source file belongs to. + with open(packages_file_list, "r") as packagef: + for line in packagef: + if sourcef in line: + package = line.split(',')[0] + + if package: + # Get package version + package_info = json.loads(subprocess.check_output(["make", "--no-print-directory", + "O=%s" % self.outputdir, + "-C", self.srcdir, + "%s-show-info" % package])) + if "version" in package_info[package]: + version = package_info[package]["version"] + return [package, version] + else: + return [package] + else: + return ["not found"] + + def cleanup(l): + # Takes a list and removes data which is redundant (source2) or data + # that might take up too much space (like huge diffs). + if "unified_diff" in l: + l.pop("unified_diff") + if "source2" in l: + l.pop("source2") + + + packages_file_list = os.path.join(self.outputdir, "build", "packages-file-list.txt") + + with open(reproducible_results, "r") as reproduciblef: + json_data = json.load(reproduciblef) + + if json_data["unified_diff"] == None: + # Remove the file list because it is not useful, i.e. it only shows + # which files vary, and nothing more. + if json_data["details"][0]["source1"] == "file list": + json_data["details"].pop(0) + + # Iterate over details in the diffoscope output. + for item in json_data["details"]: + diff_src = item["source1"] + item["package"] = get_package(diff_src) + + # In some cases, diffoscope uses multiple commands to get various + # diffs. Due to this, it generates a "details" key for those files + # instead of just storing the diff in the "unified_diff" key. + if item["unified_diff"] == None: + for item_details in item["details"]: + diff = item_details["unified_diff"].split("\n") + split_deltas = split_delta(diff) + item_details["added"] = split_deltas[0][:100] + item_details["deleted"] = split_deltas[1][:100] + cleanup(item_details) + else: + diff = item["unified_diff"].split("\n") + split_deltas = split_delta(diff) + item["added"] = split_deltas[0][:100] + item["deleted"] = split_deltas[1][:100] + cleanup(item) + # We currently just set the reason from first non-reproducible package in the + # dictionary. + reason = json_data["details"][0]["package"] + + # If there does exist a unified_diff directly for the .tar images, it is probably + # a filesystem reproducibility issue. + else: + reason = ["filesystem"] + + with open(reproducible_results, "w") as reproduciblef: + json.dump(json_data, reproduciblef, sort_keys=True, indent=4) + + return reason + def get_failure_reason(): # Output is a tuple (package, version), or None. lastlines = decode_bytes(subprocess.Popen(