From patchwork Thu Apr 30 09:52:45 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Petazzoni X-Patchwork-Id: 1280107 Return-Path: X-Original-To: incoming-buildroot@patchwork.ozlabs.org Delivered-To: patchwork-incoming-buildroot@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=busybox.net (client-ip=140.211.166.137; helo=fraxinus.osuosl.org; envelope-from=buildroot-bounces@busybox.net; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=bootlin.com Received: from fraxinus.osuosl.org (smtp4.osuosl.org [140.211.166.137]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 49CW0c4ZQTz9sPF for ; Thu, 30 Apr 2020 19:53:20 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by fraxinus.osuosl.org (Postfix) with ESMTP id 24CDD86D28; Thu, 30 Apr 2020 09:53:19 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from fraxinus.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id oX-MZHYUU4RN; Thu, 30 Apr 2020 09:53:18 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by fraxinus.osuosl.org (Postfix) with ESMTP id 7D31086CC5; Thu, 30 Apr 2020 09:53:18 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from silver.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by ash.osuosl.org (Postfix) with ESMTP id 7A4991BF86C for ; Thu, 30 Apr 2020 09:53:08 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by silver.osuosl.org (Postfix) with ESMTP id 76165204E4 for ; Thu, 30 Apr 2020 09:53:08 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from silver.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ZC-LcS8qNGjp for ; Thu, 30 Apr 2020 09:53:07 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from relay12.mail.gandi.net (relay12.mail.gandi.net [217.70.178.232]) by silver.osuosl.org (Postfix) with ESMTPS id 22DBC204D3 for ; Thu, 30 Apr 2020 09:53:06 +0000 (UTC) Received: from localhost (lfbn-tou-1-915-109.w86-210.abo.wanadoo.fr [86.210.146.109]) (Authenticated sender: thomas.petazzoni@bootlin.com) by relay12.mail.gandi.net (Postfix) with ESMTPSA id 3BFBF200008; Thu, 30 Apr 2020 09:53:05 +0000 (UTC) From: Thomas Petazzoni To: Buildroot List Date: Thu, 30 Apr 2020 11:52:45 +0200 Message-Id: <20200430095249.782597-9-thomas.petazzoni@bootlin.com> X-Mailer: git-send-email 2.25.4 In-Reply-To: <20200430095249.782597-1-thomas.petazzoni@bootlin.com> References: <20200430095249.782597-1-thomas.petazzoni@bootlin.com> MIME-Version: 1.0 Subject: [Buildroot] [PATCH 08/11] package/pkg-generic.mk: detect files overwritten in TARGET_DIR and HOST_DIR X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Peter Korsgaard , "Yann E. MORIN" , Thomas Petazzoni Errors-To: buildroot-bounces@busybox.net Sender: "buildroot" With per-package directory support, it is absolutely critical that a given package does not overwrite files already installed by another package. However, we currently don't have anything in Buildroot that detects this situation. We used to have check-uniq-files, but it was dropped in commit 2496189a4207173e4cd5bbab90256f911175ee57. This commit is a new implementation of such a detection, which is based on calculating and verifying MD5 hashes of installed files: the calculation is done at the beginning of the configure step, the verification during the newly introduced "install" step that takes place after all installation steps. Due to this calculation and verification having some overhead, it is only enabled when BR2_PER_PACKAGE_DIRECTORIES=y. Note that having BR2_PER_PACKAGE_DIRECTORIES=y also means that on average the HOST_DIR and TARGET_DIR contain less files than on non-per-package build: we only have in HOST_DIR and TARGET_DIR the dependencies of the current package being built. This helps a bit in mitigating the additional cost of this verification. Note that we are not handling separately HOST_DIR and STAGING_DIR, like we're doing with the pkg_size_{before,after} functions. Instead, the verification on HOST_DIR walks down into the STAGING_DIR. Signed-off-by: Thomas Petazzoni --- package/pkg-generic.mk | 25 +++++++++++++++++++++++++ 1 file changed, 25 insertions(+) diff --git a/package/pkg-generic.mk b/package/pkg-generic.mk index 6e06d735ad..82af4afaee 100644 --- a/package/pkg-generic.mk +++ b/package/pkg-generic.mk @@ -102,6 +102,27 @@ define fixup-libtool-files endef endif +# Functions to detect overwritten files + +ifeq ($(BR2_PER_PACKAGE_DIRECTORIES),y) +# $(1): base directory to search in +# $(2): suffix of file (optional) +define pkg_detect_overwrite_before + cd $(1); \ + LC_ALL=C find . -type f -exec md5sum {} \; > $($(PKG)_DIR)/.files$(2).md5 +endef + +# $(1): base directory to search in +# $(2): suffix of file (optional) +define pkg_detect_overwrite_after + cd $(1); \ + if test -s $($(PKG)_DIR)/.files$(2).md5 ; then \ + LC_ALL=C md5sum --quiet -c $($(PKG)_DIR)/.files$(2).md5 || \ + { echo "ERROR: package $($(PKG)_NAME) has overwritten files installed by a previous package, aborting."; exit 1; } ; \ + fi +endef +endif + # Functions to collect statistics about installed files # $(1): base directory to search in @@ -235,6 +256,8 @@ $(BUILD_DIR)/%/.stamp_configured: @$(call pkg_size_before,$(TARGET_DIR)) @$(call pkg_size_before,$(STAGING_DIR),-staging) @$(call pkg_size_before,$(HOST_DIR),-host) + @$(call pkg_detect_overwrite_before,$(TARGET_DIR)) + @$(call pkg_detect_overwrite_before,$(HOST_DIR),-host) $(call fixup-libtool-files,$(NAME),$(STAGING_DIR)) $(foreach hook,$($(PKG)_PRE_CONFIGURE_HOOKS),$(call $(hook))$(sep)) $($(PKG)_CONFIGURE_CMDS) @@ -360,6 +383,8 @@ $(BUILD_DIR)/%/.stamp_installed: @$(call pkg_size_after,$(STAGING_DIR),-staging) @$(call pkg_size_after,$(HOST_DIR),-host) @$(call check_bin_arch) + @$(call pkg_detect_overwrite_after,$(TARGET_DIR)) + @$(call pkg_detect_overwrite_after,$(HOST_DIR),-host) $(Q)touch $@ # Remove package sources