From patchwork Fri Nov 18 09:10:20 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?SsOpcsO0bWUgUG91aWxsZXI=?= X-Patchwork-Id: 696506 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3tKsgT01MFz9t0X for ; Fri, 18 Nov 2016 20:11:32 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by whitealder.osuosl.org (Postfix) with ESMTP id CDEE587949; Fri, 18 Nov 2016 09:11:30 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from whitealder.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 76uaq4CEXF4V; Fri, 18 Nov 2016 09:11:27 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by whitealder.osuosl.org (Postfix) with ESMTP id ED85386E3F; Fri, 18 Nov 2016 09:11:26 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by ash.osuosl.org (Postfix) with ESMTP id 7998B1C067B for ; Fri, 18 Nov 2016 09:10:32 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id 76CE28A38A for ; Fri, 18 Nov 2016 09:10:32 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ge0GRGf7EPGl for ; Fri, 18 Nov 2016 09:10:30 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.7.6 Received: from lupi.sysmic.org (sysmic.org [62.210.89.17]) by hemlock.osuosl.org (Postfix) with ESMTPS id 2C2C38A390 for ; Fri, 18 Nov 2016 09:10:30 +0000 (UTC) Received: from lupi.online.net (sysmic.org [62.210.89.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA256 (128/128 bits)) (No client certificate requested) (Authenticated sender: jezz) by lupi.sysmic.org (Postfix) with ESMTPSA id 53690426D3; Fri, 18 Nov 2016 10:10:27 +0100 (CET) From: =?UTF-8?q?J=C3=A9r=C3=B4me=20Pouiller?= To: buildroot@busybox.net Date: Fri, 18 Nov 2016 10:10:20 +0100 Message-Id: <1479460224-6119-12-git-send-email-jezz@sysmic.org> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1479460224-6119-1-git-send-email-jezz@sysmic.org> References: <1479460224-6119-1-git-send-email-jezz@sysmic.org> MIME-Version: 1.0 Cc: Thomas Petazzoni , =?UTF-8?q?J=C3=A9r=C3=B4me=20Pouiller?= Subject: [Buildroot] [PATCH v2 11/15] python2: generate reproducible .pyc X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: buildroot-bounces@busybox.net Sender: "buildroot" .pyc files contain modification time of .py source. In order to make build reproducible, we fix modification time of all .py before to compile .pyc files. In order to guarantee .pyc are regenerated regardless their modification time, we remove .pyc before to compile. However, I wonder if it wouldn't be simpler to always call compile_all with 'force' flag. This work was sponsored by `BA Robotic Systems'. Signed-off-by: Jérôme Pouiller --- package/python/python.mk | 30 ++++++++++++++++++++---------- 1 file changed, 20 insertions(+), 10 deletions(-) diff --git a/package/python/python.mk b/package/python/python.mk index cc65376..b0ff1fd 100644 --- a/package/python/python.mk +++ b/package/python/python.mk @@ -226,6 +226,26 @@ PYTHON_PATH = $(TARGET_DIR)/usr/lib/python$(PYTHON_VERSION_MAJOR)/sysconfigdata/ $(eval $(autotools-package)) $(eval $(host-autotools-package)) +# Normally, *.pyc files should not have been compiled, but just in +# case, we make sure we remove all of them. +# However, do not remove .pyc if source .py is not present. +ifneq ($(BR2_PACKAGE_PYTHON_PY_ONLY)$(BR2_REPRODUCIBLE)),) +define PYTHON_REMOVE_PYC_FILES + find $(TARGET_DIR)/usr/lib/python$(PYTHON_VERSION_MAJOR) -name '*.py' -print0 | \ + sed -z -e s/py$$/pyc/ | \ + xargs -0 --no-run-if-empty rm -f +endef +PYTHON_TARGET_FINALIZE_HOOKS += PYTHON_REMOVE_PYC_FILES +endif + +ifeq ($(BR2_REPRODUCIBLE),y) +define PYTHON_FIX_TIME +find $(TARGET_DIR)/usr/lib/python$(PYTHON_VERSION_MAJOR) -name '*.py' -print0 | \ + xargs -0 --no-run-if-empty touch -d @$(SOURCE_DATE_EPOCH) +endef +PYTHON_TARGET_FINALIZE_HOOKS += PYTHON_FIX_TIME +endif + define PYTHON_CREATE_PYC_FILES PYTHONPATH="$(PYTHON_PATH)" \ $(HOST_DIR)/usr/bin/python$(PYTHON_VERSION_MAJOR) \ @@ -245,16 +265,6 @@ endef PYTHON_TARGET_FINALIZE_HOOKS += PYTHON_REMOVE_PY_FILES endif -# Normally, *.pyc files should not have been compiled, but just in -# case, we make sure we remove all of them. -ifeq ($(BR2_PACKAGE_PYTHON_PY_ONLY),y) -define PYTHON_REMOVE_PYC_FILES - find $(TARGET_DIR)/usr/lib/python$(PYTHON_VERSION_MAJOR) -name '*.pyc' -print0 | \ - xargs -0 --no-run-if-empty rm -f -endef -PYTHON_TARGET_FINALIZE_HOOKS += PYTHON_REMOVE_PYC_FILES -endif - # In all cases, we don't want to keep the optimized .pyo files define PYTHON_REMOVE_PYO_FILES find $(TARGET_DIR)/usr/lib/python$(PYTHON_VERSION_MAJOR) -name '*.pyo' -print0 | \