From patchwork Mon Jun 17 09:34:54 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Atharva Lele X-Patchwork-Id: 1116920 Return-Path: X-Original-To: incoming-buildroot@patchwork.ozlabs.org Delivered-To: patchwork-incoming-buildroot@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=busybox.net (client-ip=140.211.166.136; helo=silver.osuosl.org; envelope-from=buildroot-bounces@busybox.net; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="RwTcpB3C"; dkim-atps=neutral Received: from silver.osuosl.org (smtp3.osuosl.org [140.211.166.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 45S5fW0Csgz9s4Y for ; Mon, 17 Jun 2019 19:35:14 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by silver.osuosl.org (Postfix) with ESMTP id 5C3B520020; Mon, 17 Jun 2019 09:35:12 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from silver.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id nuxJCeFNue0g; Mon, 17 Jun 2019 09:35:10 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by silver.osuosl.org (Postfix) with ESMTP id 7E74720034; Mon, 17 Jun 2019 09:35:10 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from silver.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by ash.osuosl.org (Postfix) with ESMTP id 131831BF3D5 for ; Mon, 17 Jun 2019 09:35:09 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by silver.osuosl.org (Postfix) with ESMTP id 0EA6D20034 for ; Mon, 17 Jun 2019 09:35:09 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from silver.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id lQEMYXLjDM6o for ; Mon, 17 Jun 2019 09:35:08 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mail-pg1-f196.google.com (mail-pg1-f196.google.com [209.85.215.196]) by silver.osuosl.org (Postfix) with ESMTPS id 3226220020 for ; Mon, 17 Jun 2019 09:35:08 +0000 (UTC) Received: by mail-pg1-f196.google.com with SMTP id l19so5476309pgh.9 for ; Mon, 17 Jun 2019 02:35:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=k9zk4/1Nl4BA8sJa8AcCW3pKGKc3wdvc9xtku5qc5rM=; b=RwTcpB3C//MPpsdzTbaZRobI3//efsNw4PY6hDOHPgCCMRXMSZW2ZRsniOd8CS93zk nEVSz2Lpn/M1jrUlri1n9C+frkldOkgULqOrf/GGFBDoMNlSUf6StRgQs1QEHi4vawuK 3UWWd5LP+jYeqBww2ge/Ta0cwCx7PZy8MC9johwl9DATm8O1YURnuUkveepcOpa9X2TX IeCU6wRq6vK6j/b53DJGjI/vw0uKQOHkGM4Uz1L2DYjO/Xh3y2PrxjzfscHSjLelhOXr /Jsfw4Nx1abs01m7QWUihWRiKr9yXJmsj/Y0vNEgguuq30FwN9USHzwESvkmc0Dd9c/8 9/eQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=k9zk4/1Nl4BA8sJa8AcCW3pKGKc3wdvc9xtku5qc5rM=; b=kcRNHpCbz0Lj4DxNX8taQF1xd+zh9i2VPLpdYh+zpmHSxz60cU+DZrcY3x/UCi5g3i fZwh2K9afOhVA9vB5ca/nteIwfSVab2Tq3hXfcSB4hnViUZ9M9GAHa18mGVF+3BVeNiW Xguo/5XByWqNB0KpCjkVsTuV20PKWqXTDpphwvy3lJ9vxH8i3wwuwzOV4XxHWm8QRsCE hUuHd70XWTRNZfn5yJVWXNAnlc8UpP/gvgWYRD73ccGBmbH7ciu3ZvwSQ7Fy2DkjgVo2 YS3Nq1KptJWofNjP4+KSF9ElMQVYv59n3eRQlrpxVL+qqB4cYNS5gk7HlTdfYkjyva5U QbHA== X-Gm-Message-State: APjAAAWxYleA0sJtZKgGVRbGbFalKUiP3KiJVtS9NLebaS0hr507OpHE zskQWy+Nz3njjW0JC0s3GVgEUDEd8zxAKw== X-Google-Smtp-Source: APXvYqzz2wteWNnPKmrHUNd0UZVSNfOlVr4VFBaOZxuWsB3i79jLysw5ZY3KIgudWOyR2yJRvNLbSg== X-Received: by 2002:a63:e24c:: with SMTP id y12mr48016059pgj.276.1560764107319; Mon, 17 Jun 2019 02:35:07 -0700 (PDT) Received: from localhost.localdomain ([123.201.194.75]) by smtp.gmail.com with ESMTPSA id f3sm4538515pfg.165.2019.06.17.02.35.04 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Mon, 17 Jun 2019 02:35:06 -0700 (PDT) From: Atharva Lele To: buildroot@buildroot.org Date: Mon, 17 Jun 2019 15:04:54 +0530 Message-Id: <20190617093456.6110-2-itsatharva@gmail.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190617093456.6110-1-itsatharva@gmail.com> References: <20190617093456.6110-1-itsatharva@gmail.com> MIME-Version: 1.0 Subject: [Buildroot] [PATCH v5 2/4] autobuild-run: initial implementation of check_reproducibility() X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Atharva Lele , yann.morin.1998@free.fr, thomas.petazzoni@bootlin.com Errors-To: buildroot-bounces@busybox.net Sender: "buildroot" For reproducible builds, we want to find out if there are any differences between two builds having the same configuration, and to find the reason behind them. The diffoscope tool looks inside different types of files to see where the differences lie. check_reproducibility() runs diffoscope on two output directories which are expected in output/images and output/images-1. Since it uses objdump, it needs to be provided the cross-compile prefix which is derived from the TARGET_CROSS variable. Since diffoscope may not be installed, we fall back to cmp for byte-by-byte comparison. We add diffoscope to list of optional programs to avoid repeated checking of its presence. Signed-off-by: Atharva Lele Reviewed-by: Yann E. MORIN --- Changes v4 -> v5: - Fix cmp fallback command Changes v3 -> v4: - Removed spaces on empty lines (suggested by Yann) Changes v2 -> v3: - Use file size of reproducible_results to check reproducibility status, rather than exit status of cmp or diff (suggested by arnout) - Rename results file (diffoscope_output.txt -> reproducible_results) to avoid confusion - Change handling of diffoscope output text file using with open() - Changed commit message to have all necessary info (suggested by arnout) - Removed leftover code from when I was using an exception to handle diffoscope presense (thanks to arnout) Changes v1 -> v2: - move diffoscope output to results dir (suggested by arnout) - fix make printvars call - Add diffoscope to DEFAULT_OPTIONAL_PROGS --- scripts/autobuild-run | 39 ++++++++++++++++++++++++++++++++++++++- 1 file changed, 38 insertions(+), 1 deletion(-) diff --git a/scripts/autobuild-run b/scripts/autobuild-run index 190a254..3f5af65 100755 --- a/scripts/autobuild-run +++ b/scripts/autobuild-run @@ -204,7 +204,7 @@ def get_branch(): class SystemInfo: DEFAULT_NEEDED_PROGS = ["make", "git", "gcc"] - DEFAULT_OPTIONAL_PROGS = ["bzr", "java", "javac", "jar"] + DEFAULT_OPTIONAL_PROGS = ["bzr", "diffoscope", "java", "javac", "jar"] def __init__(self): self.needed_progs = list(self.__class__.DEFAULT_NEEDED_PROGS) @@ -394,6 +394,43 @@ def stop_on_build_hang(monitor_thread_hung_build_flag, break monitor_thread_stop_flag.wait(30) +def check_reproducibility(**kwargs): + """Check reproducibility of builds + + Use diffoscope on the built images, if diffoscope is not + installed, fallback to cmp + """ + + log = kwargs['log'] + idir = "instance-%d" % kwargs['instance'] + outputdir = os.path.join(idir, "output") + srcdir = os.path.join(idir, "buildroot") + reproducible_results = os.path.join(outputdir, "results", "reproducible_results") + # Using only tar images for now + build_1_image = os.path.join(outputdir, "images-1", "rootfs.tar") + build_2_image = os.path.join(outputdir, "images", "rootfs.tar") + + with open(reproducible_results, 'w') as diff: + if kwargs['sysinfo'].has("diffoscope"): + # Prefix to point diffoscope towards cross-tools + prefix = subprocess.check_output(["make", "O=%s" % outputdir, "-C", srcdir, "printvars", "VARS=TARGET_CROSS"]) + # Remove TARGET_CROSS= and \n from the string + prefix = prefix[13:-1] + log_write(log, "INFO: running diffoscope on images") + subprocess.call(["diffoscope", build_1_image, build_2_image, + "--tool-prefix-binutils", prefix], stdout=diff, stderr=log) + else: + log_write(log, "INFO: diffoscope not installed, falling back to cmp") + subprocess.call(["cmp", "-b", build_1_image, build_2_image], stdout=diff, stderr=log) + + if os.stat(reproducible_results).st_size > 0: + log_write(log, "INFO: Build is non-reproducible.") + return -1 + + # rootfs images match byte-for-byte -> reproducible image + log_write(log, "INFO: Build is reproducible!") + return 0 + def do_build(**kwargs): """Run the build itself"""