From patchwork Wed Nov 14 10:55:52 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Petazzoni X-Patchwork-Id: 997611 Return-Path: X-Original-To: incoming-buildroot@patchwork.ozlabs.org Delivered-To: patchwork-incoming-buildroot@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=busybox.net (client-ip=140.211.166.137; helo=fraxinus.osuosl.org; envelope-from=buildroot-bounces@busybox.net; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=bootlin.com Received: from fraxinus.osuosl.org (smtp4.osuosl.org [140.211.166.137]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 42w1d917nCz9s7W for ; Wed, 14 Nov 2018 21:56:13 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by fraxinus.osuosl.org (Postfix) with ESMTP id D11D386F33; Wed, 14 Nov 2018 10:56:10 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from fraxinus.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id smr4N_ELXeRn; Wed, 14 Nov 2018 10:56:09 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by fraxinus.osuosl.org (Postfix) with ESMTP id 546D58629C; Wed, 14 Nov 2018 10:56:09 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by ash.osuosl.org (Postfix) with ESMTP id C5A1F1C22DE for ; Wed, 14 Nov 2018 10:56:06 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id C363E87E5B for ; Wed, 14 Nov 2018 10:56:06 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 0dMrKXErgbh8 for ; Wed, 14 Nov 2018 10:56:04 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mail.bootlin.com (mail.bootlin.com [62.4.15.54]) by hemlock.osuosl.org (Postfix) with ESMTP id 6E42B87E0A for ; Wed, 14 Nov 2018 10:56:04 +0000 (UTC) Received: by mail.bootlin.com (Postfix, from userid 110) id BE39520791; Wed, 14 Nov 2018 11:56:03 +0100 (CET) Received: from localhost (aaubervilliers-681-1-30-49.w90-88.abo.wanadoo.fr [90.88.15.49]) by mail.bootlin.com (Postfix) with ESMTPSA id 9555B206A1; Wed, 14 Nov 2018 11:56:03 +0100 (CET) From: Thomas Petazzoni To: Buildroot List , "Arnout Vandecappelle (Essensium/Mind)" , "Yann E. MORIN" , Peter Korsgaard , Andreas Naumann Date: Wed, 14 Nov 2018 11:55:52 +0100 Message-Id: <20181114105557.12599-2-thomas.petazzoni@bootlin.com> X-Mailer: git-send-email 2.19.1 In-Reply-To: <20181114105557.12599-1-thomas.petazzoni@bootlin.com> References: <20181114105557.12599-1-thomas.petazzoni@bootlin.com> MIME-Version: 1.0 Subject: [Buildroot] [PATCH next v4 1/6] Makefile: evaluate CCACHE and HOST{CC, CXX} at time of use X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Thomas Petazzoni Errors-To: buildroot-bounces@busybox.net Sender: "buildroot" As we are going to move to per-package SDK, the location of CCACHE and therefore the definitions of HOSTCC and HOSTCXX need to be evaluated at the time of use and not at the time of assignment. Indeed, the value of HOST_DIR changes from one package to the other. Therefore, we need to change from := to =. In addition, while doing A := $(something) $(A) is possible, doing A = $(something) $(A) is not legal. So, instead of defining HOSTCC in terms of the current HOSTCC variable, we re-use HOSTCC_NOCCACHE instead. Signed-off-by: Thomas Petazzoni Acked-by: "Yann E. MORIN" --- Changes since v2: - New patch --- Makefile | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/Makefile b/Makefile index 94adff3406..2819d44124 100644 --- a/Makefile +++ b/Makefile @@ -471,11 +471,11 @@ BR_PATH = "$(HOST_DIR)/bin:$(HOST_DIR)/sbin:$(PATH)" TARGET_DIR_WARNING_FILE = $(BASE_TARGET_DIR)/THIS_IS_NOT_YOUR_ROOT_FILESYSTEM ifeq ($(BR2_CCACHE),y) -CCACHE := $(HOST_DIR)/bin/ccache +CCACHE = $(HOST_DIR)/bin/ccache BR_CACHE_DIR ?= $(call qstrip,$(BR2_CCACHE_DIR)) export BR_CACHE_DIR -HOSTCC := $(CCACHE) $(HOSTCC) -HOSTCXX := $(CCACHE) $(HOSTCXX) +HOSTCC = $(CCACHE) $(HOSTCC_NOCCACHE) +HOSTCXX = $(CCACHE) $(HOSTCXX_NOCCACHE) else export BR_NO_CCACHE endif From patchwork Wed Nov 14 10:55:53 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Petazzoni X-Patchwork-Id: 997610 Return-Path: X-Original-To: incoming-buildroot@patchwork.ozlabs.org Delivered-To: patchwork-incoming-buildroot@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=busybox.net (client-ip=140.211.166.136; helo=silver.osuosl.org; envelope-from=buildroot-bounces@busybox.net; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=bootlin.com Received: from silver.osuosl.org (smtp3.osuosl.org [140.211.166.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 42w1d84P5sz9s5c for ; Wed, 14 Nov 2018 21:56:12 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by silver.osuosl.org (Postfix) with ESMTP id 5733730007; Wed, 14 Nov 2018 10:56:08 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from silver.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 94GlP3q80rW2; Wed, 14 Nov 2018 10:56:06 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by silver.osuosl.org (Postfix) with ESMTP id 967DF2FFFE; Wed, 14 Nov 2018 10:56:06 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from silver.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by ash.osuosl.org (Postfix) with ESMTP id D4E741C22DE for ; Wed, 14 Nov 2018 10:56:05 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by silver.osuosl.org (Postfix) with ESMTP id D1A9030007 for ; Wed, 14 Nov 2018 10:56:05 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from silver.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id K5qOmSWplxzl for ; Wed, 14 Nov 2018 10:56:05 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mail.bootlin.com (mail.bootlin.com [62.4.15.54]) by silver.osuosl.org (Postfix) with ESMTP id 0237F30021 for ; Wed, 14 Nov 2018 10:56:04 +0000 (UTC) Received: by mail.bootlin.com (Postfix, from userid 110) id 19D3D20D72; Wed, 14 Nov 2018 11:56:04 +0100 (CET) Received: from localhost (aaubervilliers-681-1-30-49.w90-88.abo.wanadoo.fr [90.88.15.49]) by mail.bootlin.com (Postfix) with ESMTPSA id E5564206A1; Wed, 14 Nov 2018 11:56:03 +0100 (CET) From: Thomas Petazzoni To: Buildroot List , "Arnout Vandecappelle (Essensium/Mind)" , "Yann E. MORIN" , Peter Korsgaard , Andreas Naumann Date: Wed, 14 Nov 2018 11:55:53 +0100 Message-Id: <20181114105557.12599-3-thomas.petazzoni@bootlin.com> X-Mailer: git-send-email 2.19.1 In-Reply-To: <20181114105557.12599-1-thomas.petazzoni@bootlin.com> References: <20181114105557.12599-1-thomas.petazzoni@bootlin.com> MIME-Version: 1.0 Subject: [Buildroot] [PATCH next v4 2/6] support/scripts/check-host-rpath: split condition on two statements X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Thomas Petazzoni Errors-To: buildroot-bounces@busybox.net Sender: "buildroot" Inside the check_elf_has_rpath(), we check if the host binary has a correct RPATH, which should be either an absolute path to $(HOST_DIR)/lib, or a relative path using $ORIGIN. Those two conditions are checked in a single statements, but as we are going to add a third condition, let's split this up a bit: - If we have a RPATH to $(HOST_DIR)/lib -> we're good, return 0 - If we have a RPATH to $ORIGIN/../lib -> we're good, return 0 - Otherwise, we will exit the loop, and return 1 Signed-off-by: Thomas Petazzoni Acked-by: "Yann E. MORIN" --- support/scripts/check-host-rpath | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/support/scripts/check-host-rpath b/support/scripts/check-host-rpath index 169628decb..6c5767da05 100755 --- a/support/scripts/check-host-rpath +++ b/support/scripts/check-host-rpath @@ -63,7 +63,8 @@ check_elf_has_rpath() { for dir in ${rpath//:/ }; do # Remove duplicate and trailing '/' for proper match dir="$( sed -r -e 's:/+:/:g; s:/$::;' <<<"${dir}" )" - [ "${dir}" = "${hostdir}/lib" -o "${dir}" = "\$ORIGIN/../lib" ] && return 0 + [ "${dir}" = "${hostdir}/lib" ] && return 0 + [ "${dir}" = "\$ORIGIN/../lib" ] && return 0 done done < <( readelf -d "${file}" \ |sed -r -e '/.* \(R(UN)?PATH\) +Library r(un)?path: \[(.+)\]$/!d' \ From patchwork Wed Nov 14 10:55:54 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Petazzoni X-Patchwork-Id: 997612 Return-Path: X-Original-To: incoming-buildroot@patchwork.ozlabs.org Delivered-To: patchwork-incoming-buildroot@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=busybox.net (client-ip=140.211.166.133; helo=hemlock.osuosl.org; envelope-from=buildroot-bounces@busybox.net; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=bootlin.com Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 42w1dB5PhMz9s5c for ; Wed, 14 Nov 2018 21:56:14 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id 92A82884BF; Wed, 14 Nov 2018 10:56:12 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Dj9D4acY+KVh; Wed, 14 Nov 2018 10:56:11 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by hemlock.osuosl.org (Postfix) with ESMTP id BC4A187E3D; Wed, 14 Nov 2018 10:56:11 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by ash.osuosl.org (Postfix) with ESMTP id 638A11C22DE for ; Wed, 14 Nov 2018 10:56:08 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id 6122C87E0A for ; Wed, 14 Nov 2018 10:56:08 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id IItWjJT7E8YJ for ; Wed, 14 Nov 2018 10:56:05 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mail.bootlin.com (mail.bootlin.com [62.4.15.54]) by hemlock.osuosl.org (Postfix) with ESMTP id 179A887E3D for ; Wed, 14 Nov 2018 10:56:05 +0000 (UTC) Received: by mail.bootlin.com (Postfix, from userid 110) id 6B03220D83; Wed, 14 Nov 2018 11:56:04 +0100 (CET) Received: from localhost (aaubervilliers-681-1-30-49.w90-88.abo.wanadoo.fr [90.88.15.49]) by mail.bootlin.com (Postfix) with ESMTPSA id 3AB6C206A1; Wed, 14 Nov 2018 11:56:04 +0100 (CET) From: Thomas Petazzoni To: Buildroot List , "Arnout Vandecappelle (Essensium/Mind)" , "Yann E. MORIN" , Peter Korsgaard , Andreas Naumann Date: Wed, 14 Nov 2018 11:55:54 +0100 Message-Id: <20181114105557.12599-4-thomas.petazzoni@bootlin.com> X-Mailer: git-send-email 2.19.1 In-Reply-To: <20181114105557.12599-1-thomas.petazzoni@bootlin.com> References: <20181114105557.12599-1-thomas.petazzoni@bootlin.com> MIME-Version: 1.0 Subject: [Buildroot] [PATCH next v4 3/6] Makefile: rework main directory creation logic X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Thomas Petazzoni Errors-To: buildroot-bounces@busybox.net Sender: "buildroot" In the current code, the creation of the main output folders (BUILD_DIR, STAGING_DIR, HOST_DIR, TARGET_DIR, etc.) is done by a global "dirs" target. While this works fine in the current situation, it doesn't work well in a context where per-package host and target directories are used. For example, with the current code and per-package host directories, the output/staging symbolic link ends up being created as a link to the per-package package sysroot directory of the first package being built, instead of the global sysroot. This commit reworks the creation of those directories by having the package/pkg-generic.mk code ensure that the build directory, target directory, host directory, staging directory and binaries directory exist before they are needed. Two new targets, host-finalize and staging-finalize are added in the main Makefile to create the compatibility symlinks for host and staging directories. They will be extended later with additional logic for per-package directories. Thanks to those changes, the global "dirs" target is entirely removed. Signed-off-by: Thomas Petazzoni --- Makefile | 21 +++++++++------------ fs/common.mk | 1 + package/pkg-generic.mk | 7 ++++++- 3 files changed, 16 insertions(+), 13 deletions(-) diff --git a/Makefile b/Makefile index 2819d44124..e675ac26aa 100644 --- a/Makefile +++ b/Makefile @@ -572,10 +572,6 @@ $(foreach pkg,$(call UPPERCASE,$(PACKAGES)),\ endif -.PHONY: dirs -dirs: $(BUILD_DIR) $(STAGING_DIR) $(BASE_TARGET_DIR) \ - $(HOST_DIR) $(HOST_DIR_SYMLINK) $(BINARIES_DIR) - $(BUILD_DIR)/buildroot-config/auto.conf: $(BR2_CONFIG) $(MAKE1) $(EXTRAMAKEARGS) HOSTCC="$(HOSTCC_NOCCACHE)" HOSTCXX="$(HOSTCXX_NOCCACHE)" syncconfig @@ -605,11 +601,6 @@ sdk: prepare-sdk $(BR2_TAR_HOST_DEPENDENCY) --transform='s#^\.#$(BR2_SDK_PREFIX)#' \ -C $(HOST_DIR) "." -# Populating the staging with the base directories is handled by the skeleton package -$(STAGING_DIR): - @mkdir -p $(STAGING_DIR) - @ln -snf $(STAGING_DIR) $(BASE_DIR)/staging - RSYNC_VCS_EXCLUSIONS = \ --exclude .svn --exclude .git --exclude .hg --exclude .bzr \ --exclude CVS @@ -710,8 +701,14 @@ $(TARGETS_ROOTFS): target-finalize # Avoid the rootfs name leaking down the dependency chain target-finalize: ROOTFS= +host-finalize: $(HOST_DIR_SYMLINK) + +.PHONY: staging-finalize +staging-finalize: + @ln -snf $(STAGING_DIR) $(BASE_DIR)/staging + .PHONY: target-finalize -target-finalize: $(PACKAGES) +target-finalize: $(PACKAGES) host-finalize @$(call MESSAGE,"Finalizing target directory") # Check files that are touched by more than one package ./support/scripts/check-uniq-files -t target $(BUILD_DIR)/packages-file-list.txt @@ -782,7 +779,7 @@ endif touch $(TARGET_DIR)/usr .PHONY: target-post-image -target-post-image: $(TARGETS_ROOTFS) target-finalize +target-post-image: $(TARGETS_ROOTFS) target-finalize staging-finalize @rm -f $(ROOTFS_COMMON_TAR) @$(foreach s, $(call qstrip,$(BR2_ROOTFS_POST_IMAGE_SCRIPT)), \ $(call MESSAGE,"Executing post-image script $(s)"); \ @@ -811,7 +808,7 @@ legal-info-prepare: $(LEGAL_INFO_DIR) @cp $(BR2_CONFIG) $(LEGAL_INFO_DIR)/buildroot.config .PHONY: legal-info -legal-info: dirs legal-info-clean legal-info-prepare $(foreach p,$(PACKAGES),$(p)-all-legal-info) \ +legal-info: legal-info-clean legal-info-prepare $(foreach p,$(PACKAGES),$(p)-all-legal-info) \ $(REDIST_SOURCES_DIR_TARGET) $(REDIST_SOURCES_DIR_HOST) @cat support/legal-info/README.header >>$(LEGAL_REPORT) @if [ -r $(LEGAL_WARNINGS) ]; then \ diff --git a/fs/common.mk b/fs/common.mk index 2a5a202a89..96658428ba 100644 --- a/fs/common.mk +++ b/fs/common.mk @@ -145,6 +145,7 @@ $$(BINARIES_DIR)/rootfs.$(1): ROOTFS=$(2) $$(BINARIES_DIR)/rootfs.$(1): FAKEROOT_SCRIPT=$$(ROOTFS_$(2)_DIR)/fakeroot $$(BINARIES_DIR)/rootfs.$(1): $$(ROOTFS_$(2)_DEPENDENCIES) @$$(call MESSAGE,"Generating root filesystem image rootfs.$(1)") + mkdir -p $$(@D) rm -rf $$(ROOTFS_$(2)_DIR) mkdir -p $$(ROOTFS_$(2)_DIR) echo '#!/bin/sh' > $$(FAKEROOT_SCRIPT) diff --git a/package/pkg-generic.mk b/package/pkg-generic.mk index f34f46afc8..309fd8cd48 100644 --- a/package/pkg-generic.mk +++ b/package/pkg-generic.mk @@ -173,6 +173,7 @@ $(BUILD_DIR)/%/.stamp_extracted: $(BUILD_DIR)/%/.stamp_rsynced: @$(call step_start,rsync) @$(call MESSAGE,"Syncing from source dir $(SRCDIR)") + @mkdir -p $(@D) $(foreach hook,$($(PKG)_PRE_RSYNC_HOOKS),$(call $(hook))$(sep)) @test -d $(SRCDIR) || (echo "ERROR: $(SRCDIR) does not exist" ; exit 1) rsync -au --chmod=u=rwX,go=rX $(RSYNC_VCS_EXCLUSIONS) $($(PKG)_OVERRIDE_SRCDIR_RSYNC_EXCLUSIONS) $(call qstrip,$(SRCDIR))/ $(@D) @@ -238,6 +239,7 @@ $(BUILD_DIR)/%/.stamp_built:: $(BUILD_DIR)/%/.stamp_host_installed: @$(call step_start,install-host) @$(call MESSAGE,"Installing to host directory") + @mkdir -p $(HOST_DIR) $(foreach hook,$($(PKG)_PRE_INSTALL_HOOKS),$(call $(hook))$(sep)) +$($(PKG)_INSTALL_CMDS) $(foreach hook,$($(PKG)_POST_INSTALL_HOOKS),$(call $(hook))$(sep)) @@ -267,6 +269,7 @@ $(BUILD_DIR)/%/.stamp_host_installed: $(BUILD_DIR)/%/.stamp_staging_installed: @$(call step_start,install-staging) @$(call MESSAGE,"Installing to staging directory") + @mkdir -p $(STAGING_DIR) $(foreach hook,$($(PKG)_PRE_INSTALL_STAGING_HOOKS),$(call $(hook))$(sep)) +$($(PKG)_INSTALL_STAGING_CMDS) $(foreach hook,$($(PKG)_POST_INSTALL_STAGING_HOOKS),$(call $(hook))$(sep)) @@ -298,6 +301,7 @@ $(BUILD_DIR)/%/.stamp_staging_installed: # Install to images dir $(BUILD_DIR)/%/.stamp_images_installed: @$(call step_start,install-image) + @mkdir -p $(BINARIES_DIR) $(foreach hook,$($(PKG)_PRE_INSTALL_IMAGES_HOOKS),$(call $(hook))$(sep)) @$(call MESSAGE,"Installing to images directory") +$($(PKG)_INSTALL_IMAGES_CMDS) @@ -309,6 +313,7 @@ $(BUILD_DIR)/%/.stamp_images_installed: $(BUILD_DIR)/%/.stamp_target_installed: @$(call step_start,install-target) @$(call MESSAGE,"Installing to target") + @mkdir -p $(TARGET_DIR) $(foreach hook,$($(PKG)_PRE_INSTALL_TARGET_HOOKS),$(call $(hook))$(sep)) +$($(PKG)_INSTALL_TARGET_CMDS) $(if $(BR2_INIT_SYSTEMD),\ @@ -735,7 +740,7 @@ $$($(2)_TARGET_BUILD): $$($(2)_TARGET_CONFIGURE) $(1)-configure: $$($(2)_TARGET_CONFIGURE) $$($(2)_TARGET_CONFIGURE): | $$($(2)_FINAL_DEPENDENCIES) -$$($(2)_TARGET_SOURCE) $$($(2)_TARGET_RSYNC): | dirs prepare +$$($(2)_TARGET_SOURCE) $$($(2)_TARGET_RSYNC): | prepare $$($(2)_TARGET_SOURCE) $$($(2)_TARGET_RSYNC): | dependencies ifeq ($$($(2)_OVERRIDE_SRCDIR),) From patchwork Wed Nov 14 10:55:55 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Petazzoni X-Patchwork-Id: 997613 Return-Path: X-Original-To: incoming-buildroot@patchwork.ozlabs.org Delivered-To: patchwork-incoming-buildroot@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=busybox.net (client-ip=140.211.166.136; helo=silver.osuosl.org; envelope-from=buildroot-bounces@busybox.net; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=bootlin.com Received: from silver.osuosl.org (smtp3.osuosl.org [140.211.166.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 42w1dC2RZbz9s7W for ; Wed, 14 Nov 2018 21:56:15 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by silver.osuosl.org (Postfix) with ESMTP id 9275A300B3; Wed, 14 Nov 2018 10:56:13 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from silver.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id JOznHF+pA50e; Wed, 14 Nov 2018 10:56:10 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by silver.osuosl.org (Postfix) with ESMTP id D1E9430032; Wed, 14 Nov 2018 10:56:10 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by ash.osuosl.org (Postfix) with ESMTP id DA4531C22DE for ; Wed, 14 Nov 2018 10:56:07 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id D815187E0A for ; Wed, 14 Nov 2018 10:56:07 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id RWSXRWMyh4vq for ; Wed, 14 Nov 2018 10:56:05 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mail.bootlin.com (mail.bootlin.com [62.4.15.54]) by hemlock.osuosl.org (Postfix) with ESMTP id 5E78487E45 for ; Wed, 14 Nov 2018 10:56:05 +0000 (UTC) Received: by mail.bootlin.com (Postfix, from userid 110) id B402820D8A; Wed, 14 Nov 2018 11:56:04 +0100 (CET) Received: from localhost (aaubervilliers-681-1-30-49.w90-88.abo.wanadoo.fr [90.88.15.49]) by mail.bootlin.com (Postfix) with ESMTPSA id 883C8206A1; Wed, 14 Nov 2018 11:56:04 +0100 (CET) From: Thomas Petazzoni To: Buildroot List , "Arnout Vandecappelle (Essensium/Mind)" , "Yann E. MORIN" , Peter Korsgaard , Andreas Naumann Date: Wed, 14 Nov 2018 11:55:55 +0100 Message-Id: <20181114105557.12599-5-thomas.petazzoni@bootlin.com> X-Mailer: git-send-email 2.19.1 In-Reply-To: <20181114105557.12599-1-thomas.petazzoni@bootlin.com> References: <20181114105557.12599-1-thomas.petazzoni@bootlin.com> MIME-Version: 1.0 Subject: [Buildroot] [PATCH next v4 4/6] Makefile: move .NOTPARALLEL statement after including .config file X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Thomas Petazzoni Errors-To: buildroot-bounces@busybox.net Sender: "buildroot" In a follow-up commit, we will make the .NOTPARALLEL statement conditional on a Config.in option, so we need to move it further down. Signed-off-by: Thomas Petazzoni Reviewed-by: "Yann E. MORIN" --- Makefile | 32 ++++++++++++++++---------------- 1 file changed, 16 insertions(+), 16 deletions(-) diff --git a/Makefile b/Makefile index e675ac26aa..24c803872d 100644 --- a/Makefile +++ b/Makefile @@ -105,22 +105,6 @@ ifneq ($(firstword $(sort $(RUNNING_MAKE_VERSION) $(MIN_MAKE_VERSION))),$(MIN_MA $(error You have make '$(RUNNING_MAKE_VERSION)' installed. GNU make >= $(MIN_MAKE_VERSION) is required) endif -# Parallel execution of this Makefile is disabled because it changes -# the packages building order, that can be a problem for two reasons: -# - If a package has an unspecified optional dependency and that -# dependency is present when the package is built, it is used, -# otherwise it isn't (but compilation happily proceeds) so the end -# result will differ if the order is swapped due to parallel -# building. -# - Also changing the building order can be a problem if two packages -# manipulate the same file in the target directory. -# -# Taking into account the above considerations, if you still want to execute -# this top-level Makefile in parallel comment the ".NOTPARALLEL" line and -# use the -j option when building, e.g: -# make -j$((`getconf _NPROCESSORS_ONLN`+1)) -.NOTPARALLEL: - # absolute path TOPDIR := $(CURDIR) CONFIG_CONFIG_IN = Config.in @@ -246,6 +230,22 @@ ifeq ($(filter $(noconfig_targets),$(MAKECMDGOALS)),) -include $(BR2_CONFIG) endif +# Parallel execution of this Makefile is disabled because it changes +# the packages building order, that can be a problem for two reasons: +# - If a package has an unspecified optional dependency and that +# dependency is present when the package is built, it is used, +# otherwise it isn't (but compilation happily proceeds) so the end +# result will differ if the order is swapped due to parallel +# building. +# - Also changing the building order can be a problem if two packages +# manipulate the same file in the target directory. +# +# Taking into account the above considerations, if you still want to execute +# this top-level Makefile in parallel comment the ".NOTPARALLEL" line and +# use the -j option when building, e.g: +# make -j$((`getconf _NPROCESSORS_ONLN`+1)) +.NOTPARALLEL: + # timezone and locale may affect build output ifeq ($(BR2_REPRODUCIBLE),y) export TZ = UTC From patchwork Wed Nov 14 10:55:56 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Petazzoni X-Patchwork-Id: 997615 Return-Path: X-Original-To: incoming-buildroot@patchwork.ozlabs.org Delivered-To: patchwork-incoming-buildroot@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=busybox.net (client-ip=140.211.166.136; helo=silver.osuosl.org; envelope-from=buildroot-bounces@busybox.net; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=bootlin.com Received: from silver.osuosl.org (smtp3.osuosl.org [140.211.166.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 42w1dF4Wh6z9s5c for ; Wed, 14 Nov 2018 21:56:17 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by silver.osuosl.org (Postfix) with ESMTP id C9B553001C; Wed, 14 Nov 2018 10:56:15 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from silver.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 53XcuiQtIZuH; Wed, 14 Nov 2018 10:56:12 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by silver.osuosl.org (Postfix) with ESMTP id 70E373006A; Wed, 14 Nov 2018 10:56:12 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by ash.osuosl.org (Postfix) with ESMTP id 0062F1C22DE for ; Wed, 14 Nov 2018 10:56:09 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id F20A787E0A for ; Wed, 14 Nov 2018 10:56:08 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id y-fC4BUnLVXf for ; Wed, 14 Nov 2018 10:56:05 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mail.bootlin.com (mail.bootlin.com [62.4.15.54]) by hemlock.osuosl.org (Postfix) with ESMTP id A6F5688411 for ; Wed, 14 Nov 2018 10:56:05 +0000 (UTC) Received: by mail.bootlin.com (Postfix, from userid 110) id 09A0720D92; Wed, 14 Nov 2018 11:56:05 +0100 (CET) Received: from localhost (aaubervilliers-681-1-30-49.w90-88.abo.wanadoo.fr [90.88.15.49]) by mail.bootlin.com (Postfix) with ESMTPSA id D3724206A1; Wed, 14 Nov 2018 11:56:04 +0100 (CET) From: Thomas Petazzoni To: Buildroot List , "Arnout Vandecappelle (Essensium/Mind)" , "Yann E. MORIN" , Peter Korsgaard , Andreas Naumann Date: Wed, 14 Nov 2018 11:55:56 +0100 Message-Id: <20181114105557.12599-6-thomas.petazzoni@bootlin.com> X-Mailer: git-send-email 2.19.1 In-Reply-To: <20181114105557.12599-1-thomas.petazzoni@bootlin.com> References: <20181114105557.12599-1-thomas.petazzoni@bootlin.com> MIME-Version: 1.0 Subject: [Buildroot] [PATCH next v4 5/6] Makefile: define TARGET_DIR_WARNING_FILE relative to TARGET_DIR X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Thomas Petazzoni Errors-To: buildroot-bounces@busybox.net Sender: "buildroot" In commit 7e9870ce32d6329d9e3d602247fbe1709a2275a4 ("core: introduce intermediate BASE_TARGET_DIR variable"), the definition of TARGET_DIR_WARNING_FILE was changed to use $(BASE_TARGET_DIR) instead of $(TARGET_DIR). However, this change is incompatible with per-package directories, and is in fact not needed. With per-package directories, using $(BASE_TARGET_DIR) means that TARGET_DIR_WARNING_FILE is output/target/THIS_IS_NOT_YOUR_ROOT_FILESYSTEM. Due to this, when skeleton-init-common or skeleton-custom attempt to install it, it fails, because it should be installed to their package per-package target directory, and not the global output/target folder that doesn't exist yet. The failure looks like this: /usr/bin/install -m 0644 support/misc/target-dir-warning.txt /home/thomas/projets/buildroot/output/target/THIS_IS_NOT_YOUR_ROOT_FILESYSTEM /usr/bin/install: cannot create regular file '/home/thomas/projets/buildroot/output/target/THIS_IS_NOT_YOUR_ROOT_FILESYSTEM': No such file or directory make[1]: *** [package/pkg-generic.mk:336: /home/thomas/projets/buildroot/output/build/skeleton-init-common/.stamp_target_installed] Error 1 TARGET_DIR_WARNING_FILE is used in three places: - In skeleton-custom.mk and skeleton-init-common.mk, where as explained above, using $(TARGET_DIR) fixes the use of $(TARGET_DIR_WARNING_FILE) in the context of per-package target directories. - In fs/common.mk, where it is used as argument to $(notdir ...) to retrieve just the name of the warning file. So in this case, we really don't care about the path of the file, just its name. Signed-off-by: Thomas Petazzoni --- Makefile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Makefile b/Makefile index 24c803872d..23032988a5 100644 --- a/Makefile +++ b/Makefile @@ -468,7 +468,7 @@ BR_PATH = "$(HOST_DIR)/bin:$(HOST_DIR)/sbin:$(PATH)" # Location of a file giving a big fat warning that output/target # should not be used as the root filesystem. -TARGET_DIR_WARNING_FILE = $(BASE_TARGET_DIR)/THIS_IS_NOT_YOUR_ROOT_FILESYSTEM +TARGET_DIR_WARNING_FILE = $(TARGET_DIR)/THIS_IS_NOT_YOUR_ROOT_FILESYSTEM ifeq ($(BR2_CCACHE),y) CCACHE = $(HOST_DIR)/bin/ccache From patchwork Wed Nov 14 10:55:57 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Petazzoni X-Patchwork-Id: 997614 Return-Path: X-Original-To: incoming-buildroot@patchwork.ozlabs.org Delivered-To: patchwork-incoming-buildroot@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=busybox.net (client-ip=140.211.166.138; helo=whitealder.osuosl.org; envelope-from=buildroot-bounces@busybox.net; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=bootlin.com Received: from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 42w1dC6c66z9s5c for ; Wed, 14 Nov 2018 21:56:15 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by whitealder.osuosl.org (Postfix) with ESMTP id CF90387554; Wed, 14 Nov 2018 10:56:13 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from whitealder.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id SEpfmQcebWCE; Wed, 14 Nov 2018 10:56:10 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by whitealder.osuosl.org (Postfix) with ESMTP id 1521387C16; Wed, 14 Nov 2018 10:56:10 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by ash.osuosl.org (Postfix) with ESMTP id 820A51C22DE for ; Wed, 14 Nov 2018 10:56:07 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id 7FA4C87E0A for ; Wed, 14 Nov 2018 10:56:07 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id YiCGU9F-IA4O for ; Wed, 14 Nov 2018 10:56:06 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mail.bootlin.com (mail.bootlin.com [62.4.15.54]) by hemlock.osuosl.org (Postfix) with ESMTP id 0CAEC884C5 for ; Wed, 14 Nov 2018 10:56:06 +0000 (UTC) Received: by mail.bootlin.com (Postfix, from userid 110) id 63D5220D98; Wed, 14 Nov 2018 11:56:05 +0100 (CET) Received: from localhost (aaubervilliers-681-1-30-49.w90-88.abo.wanadoo.fr [90.88.15.49]) by mail.bootlin.com (Postfix) with ESMTPSA id 2D47A206A1; Wed, 14 Nov 2018 11:56:05 +0100 (CET) From: Thomas Petazzoni To: Buildroot List , "Arnout Vandecappelle (Essensium/Mind)" , "Yann E. MORIN" , Peter Korsgaard , Andreas Naumann Date: Wed, 14 Nov 2018 11:55:57 +0100 Message-Id: <20181114105557.12599-7-thomas.petazzoni@bootlin.com> X-Mailer: git-send-email 2.19.1 In-Reply-To: <20181114105557.12599-1-thomas.petazzoni@bootlin.com> References: <20181114105557.12599-1-thomas.petazzoni@bootlin.com> MIME-Version: 1.0 Subject: [Buildroot] [PATCH next v4 6/6] core: implement per-package SDK and target X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Thomas Petazzoni Errors-To: buildroot-bounces@busybox.net Sender: "buildroot" This commit implemnts the core of the move to per-package SDK and target directories. The main idea is that instead of having a global output/host and output/target in which all packages install files, we switch to per-package host and target folders, that only contain their explicit dependencies. There are two main benefits: - Packages will no longer discover dependencies that they do not explicitly indicate in their _DEPENDENCIES variable. - We can support top-level parallel build properly, because a package only "sees" its own host directory and target directory, isolated from the build of other packages that can happen in parallel. It works as follows: - A new output/per-package/ folder is created, which will contain one sub-folder per package, and inside it, a "host" folder and a "target" folder: output/per-package/busybox/target output/per-package/busybox/host output/per-package/host-fakeroot/target output/per-package/host-fakeroot/host This output/per-package/ folder is PER_PACKAGE_DIR. - The global TARGET_DIR and HOST_DIR variable now automatically point to the per-package directory when PKG is defined. So whenever a package references $(HOST_DIR) or $(TARGET_DIR) in its build process, it effectively references the per-package host/target directories. Note that STAGING_DIR is a sub-dir of HOST_DIR, so it is handled as well. - Of course, packages have dependencies, so those dependencies must be installed in the per-package host and target folders. To do so, we simply rsync (using hard links to save space and time) the host and target folders of the direct dependencies of the package to the current package host and target folders. We only need to take care of direct dependencies (and not recursively all dependencies), because we accumulate into those per-package host and target folders the files installed by the dependencies. Note that this only works because we make the assumption that one package does *not* overwrite files installed by another package. This is done for "extract dependencies" at the beginning of the extract step, and for "normal dependencies" at the beginning of the configure step. This is basically enough to make per-package SDK and target work. The only gotcha is that at the end of the build, output/target and output/host are empty, which means that: - The filesystem image creation code cannot work. - We don't have a SDK to build code outside of Buildroot. In order to fix this, this commit extends the target-finalize step so that it starts by populating output/target and output/host by rsync-ing into them the target and host directories of all packages listed in the $(PACKAGES) variable. It is necessary to do this sequentially in the target-finalize step and not in each package. Doing it in package installation means that it can be done in parallel. In that case, there is a chance that two rsyncs are creating the same hardlink or directory at the same time, which makes one of them fail. Signed-off-by: Thomas Petazzoni Signed-off-by: Andreas Naumann --- Changes since v2: - Account for _EXTRACT_DEPENDENCIES in the extract step of the package, by rsync'ing their host and target directories to the current package host/target directory. Changes since v1: - Remove the LD_LIBRARY_PATH change since we're now longer relying on LD_LIBRARY_PATH to allow host programs to find their libraries. - Improve commit log according to Arnout suggestions - Remove -u option from rsync calls in the main Makefile, suggested by Arnout - Drop entirely the definitions of _TARGET_DIR, _STAGING_DIR and _HOST_DIR, and instead make the global TARGET_DIR, HOST_DIR variables point to the per-package directories when PKG is defined. Suggested by Arnout. --- Config.in | 18 ++++++++++++++++ Makefile | 37 +++++++++++++++++++++++++++----- fs/common.mk | 2 +- package/pkg-generic.mk | 20 ++++++++++++++++- support/scripts/check-host-rpath | 5 ++++- 5 files changed, 74 insertions(+), 8 deletions(-) diff --git a/Config.in b/Config.in index 03e4eb3928..69e34ec256 100644 --- a/Config.in +++ b/Config.in @@ -700,6 +700,24 @@ config BR2_REPRODUCIBLE This is labeled as an experimental feature, as not all packages behave properly to ensure reproducibility. +config BR2_PER_PACKAGE_FOLDERS + bool "Use per-package folders (experimental)" + help + This option will change the build process of Buildroot + package to use per-package target and host directories. + + This is useful for two related purposes: + + - Cleanly isolate the build of each package, so that a + given package only "sees" the dependencies it has + explicitly expressed, and not other packages that may + have by chance been built before. + + - Enable top-level parallel build. + + This is labeled as an experimental feature, as not all + packages behave properly with per-package folders. + endmenu comment "Security Hardening Options" diff --git a/Makefile b/Makefile index 23032988a5..f3c0e2326e 100644 --- a/Makefile +++ b/Makefile @@ -207,7 +207,8 @@ BINARIES_DIR := $(BASE_DIR)/images # The target directory is common to all packages, # but there is one that is specific to each filesystem. BASE_TARGET_DIR := $(BASE_DIR)/target -TARGET_DIR = $(if $(ROOTFS),$(ROOTFS_$(ROOTFS)_TARGET_DIR),$(BASE_TARGET_DIR)) +PER_PACKAGE_DIR := $(BASE_DIR)/per-package + # initial definition so that 'make clean' works for most users, even without # .config. HOST_DIR will be overwritten later when .config is included. HOST_DIR := $(BASE_DIR)/host @@ -230,6 +231,7 @@ ifeq ($(filter $(noconfig_targets),$(MAKECMDGOALS)),) -include $(BR2_CONFIG) endif +ifeq ($(BR2_PER_PACKAGE_FOLDERS),) # Parallel execution of this Makefile is disabled because it changes # the packages building order, that can be a problem for two reasons: # - If a package has an unspecified optional dependency and that @@ -245,6 +247,13 @@ endif # use the -j option when building, e.g: # make -j$((`getconf _NPROCESSORS_ONLN`+1)) .NOTPARALLEL: +endif + +ifeq ($(BR2_PER_PACKAGE_FOLDERS),y) +TARGET_DIR = $(if $(ROOTFS),$(ROOTFS_$(ROOTFS)_TARGET_DIR),$(if $(PKG),$(PER_PACKAGE_DIR)/$($(PKG)_NAME)/target,$(BASE_TARGET_DIR))) +else +TARGET_DIR = $(if $(ROOTFS),$(ROOTFS_$(ROOTFS)_TARGET_DIR),$(BASE_TARGET_DIR)) +endif # timezone and locale may affect build output ifeq ($(BR2_REPRODUCIBLE),y) @@ -455,7 +464,11 @@ LZCAT := $(call qstrip,$(BR2_LZCAT)) TAR_OPTIONS = $(call qstrip,$(BR2_TAR_OPTIONS)) -xf # packages compiled for the host go here +ifeq ($(BR2_PER_PACKAGE_FOLDERS),y) +HOST_DIR = $(if $(PKG),$(PER_PACKAGE_DIR)/$($(PKG)_NAME)/host,$(call qstrip,$(BR2_HOST_DIR))) +else HOST_DIR := $(call qstrip,$(BR2_HOST_DIR)) +endif ifneq ($(HOST_DIR),$(BASE_DIR)/host) HOST_DIR_SYMLINK = $(BASE_DIR)/host @@ -701,14 +714,28 @@ $(TARGETS_ROOTFS): target-finalize # Avoid the rootfs name leaking down the dependency chain target-finalize: ROOTFS= -host-finalize: $(HOST_DIR_SYMLINK) +host-finalize: $(PACKAGES) $(HOST_DIR) $(HOST_DIR_SYMLINK) +ifeq ($(BR2_PER_PACKAGE_FOLDERS),y) + @$(call MESSAGE,"Creating global host directory") + $(foreach pkg,$(PACKAGES),\ + rsync -a --link-dest=$(PER_PACKAGE_DIR)/$(pkg)/host/ \ + $(PER_PACKAGE_DIR)/$(pkg)/host/ \ + $(HOST_DIR)$(sep)) +endif .PHONY: staging-finalize staging-finalize: @ln -snf $(STAGING_DIR) $(BASE_DIR)/staging .PHONY: target-finalize -target-finalize: $(PACKAGES) host-finalize +target-finalize: $(PACKAGES) $(TARGET_DIR) host-finalize +ifeq ($(BR2_PER_PACKAGE_FOLDERS),y) + @$(call MESSAGE,"Creating global target directory") + $(foreach pkg,$(PACKAGES),\ + rsync -a --link-dest=$(PER_PACKAGE_DIR)/$(pkg)/target/ \ + $(PER_PACKAGE_DIR)/$(pkg)/target/ \ + $(TARGET_DIR)$(sep)) +endif @$(call MESSAGE,"Finalizing target directory") # Check files that are touched by more than one package ./support/scripts/check-uniq-files -t target $(BUILD_DIR)/packages-file-list.txt @@ -972,7 +999,7 @@ savedefconfig: $(BUILD_DIR)/buildroot-config/conf prepare-kconfig # staging and target directories do NOT list these as # dependencies anywhere else -$(BUILD_DIR) $(BASE_TARGET_DIR) $(HOST_DIR) $(BINARIES_DIR) $(LEGAL_INFO_DIR) $(REDIST_SOURCES_DIR_TARGET) $(REDIST_SOURCES_DIR_HOST): +$(BUILD_DIR) $(BASE_TARGET_DIR) $(HOST_DIR) $(BINARIES_DIR) $(LEGAL_INFO_DIR) $(REDIST_SOURCES_DIR_TARGET) $(REDIST_SOURCES_DIR_HOST) $(PER_PACKAGE_DIR): @mkdir -p $@ # outputmakefile generates a Makefile in the output directory, if using a @@ -1011,7 +1038,7 @@ printvars: clean: rm -rf $(BASE_TARGET_DIR) $(BINARIES_DIR) $(HOST_DIR) $(HOST_DIR_SYMLINK) \ $(BUILD_DIR) $(BASE_DIR)/staging \ - $(LEGAL_INFO_DIR) $(GRAPHS_DIR) + $(LEGAL_INFO_DIR) $(GRAPHS_DIR) $(PER_PACKAGE_DIR) .PHONY: distclean distclean: clean diff --git a/fs/common.mk b/fs/common.mk index 96658428ba..fc4be3cc05 100644 --- a/fs/common.mk +++ b/fs/common.mk @@ -172,7 +172,7 @@ rootfs-$(1): $$(BINARIES_DIR)/rootfs.$(1) ifeq ($$(BR2_TARGET_ROOTFS_$(2)),y) TARGETS_ROOTFS += rootfs-$(1) -PACKAGES += $$(filter-out rootfs-%,$$(ROOTFS_$(2)_DEPENDENCIES)) +PACKAGES += $$(filter-out rootfs-%,$$(ROOTFS_$(2)_DEPENDENCIES)) $(ROOTFS_COMMON_DEPENDENCIES) endif # Check for legacy POST_TARGETS rules diff --git a/package/pkg-generic.mk b/package/pkg-generic.mk index 309fd8cd48..8ec5e8db73 100644 --- a/package/pkg-generic.mk +++ b/package/pkg-generic.mk @@ -98,7 +98,7 @@ GLOBAL_INSTRUMENTATION_HOOKS += check_bin_arch # have a proper DT_RPATH or DT_RUNPATH tag define check_host_rpath $(if $(filter install-host,$(2)),\ - $(if $(filter end,$(1)),support/scripts/check-host-rpath $(3) $(HOST_DIR))) + $(if $(filter end,$(1)),support/scripts/check-host-rpath $(3) $(HOST_DIR) $(PER_PACKAGE_DIR))) endef GLOBAL_INSTRUMENTATION_HOOKS += check_host_rpath @@ -126,6 +126,21 @@ endef GLOBAL_INSTRUMENTATION_HOOKS += step_user endif +# $1: deps +ifeq ($(BR2_PER_PACKAGE_FOLDERS),y) +define prepare-per-package-folder + mkdir -p $(HOST_DIR) $(TARGET_DIR) + $(foreach pkg,$(1),\ + rsync -a --link-dest=$(PER_PACKAGE_DIR)/$(pkg)/host/ \ + $(PER_PACKAGE_DIR)/$(pkg)/host/ \ + $(HOST_DIR)$(sep)) + $(foreach pkg,$(1),\ + rsync -a --link-dest=$(PER_PACKAGE_DIR)/$(pkg)/target/ \ + $(PER_PACKAGE_DIR)/$(pkg)/target/ \ + $(TARGET_DIR)$(sep)) +endef +endif + ################################################################################ # Implicit targets -- produce a stamp file for each step of a package build ################################################################################ @@ -133,6 +148,7 @@ endif # Retrieve the archive $(BUILD_DIR)/%/.stamp_downloaded: @$(call step_start,download) + $(call prepare-per-package-folder,$($(PKG)_FINAL_DOWNLOAD_DEPENDENCIES)) $(foreach hook,$($(PKG)_PRE_DOWNLOAD_HOOKS),$(call $(hook))$(sep)) # Only show the download message if it isn't already downloaded $(Q)for p in $($(PKG)_ALL_DOWNLOADS); do \ @@ -159,6 +175,7 @@ $(BUILD_DIR)/%/.stamp_actual_downloaded: $(BUILD_DIR)/%/.stamp_extracted: @$(call step_start,extract) @$(call MESSAGE,"Extracting") + $(call prepare-per-package-folder,$($(PKG)_FINAL_EXTRACT_DEPENDENCIES)) $(foreach hook,$($(PKG)_PRE_EXTRACT_HOOKS),$(call $(hook))$(sep)) $(Q)mkdir -p $(@D) $($(PKG)_EXTRACT_CMDS) @@ -219,6 +236,7 @@ $(foreach dir,$(call qstrip,$(BR2_GLOBAL_PATCH_DIR)),\ $(BUILD_DIR)/%/.stamp_configured: @$(call step_start,configure) @$(call MESSAGE,"Configuring") + $(call prepare-per-package-folder,$($(PKG)_FINAL_DEPENDENCIES)) $(foreach hook,$($(PKG)_PRE_CONFIGURE_HOOKS),$(call $(hook))$(sep)) $($(PKG)_CONFIGURE_CMDS) $(foreach hook,$($(PKG)_POST_CONFIGURE_HOOKS),$(call $(hook))$(sep)) diff --git a/support/scripts/check-host-rpath b/support/scripts/check-host-rpath index 6c5767da05..787a1763b0 100755 --- a/support/scripts/check-host-rpath +++ b/support/scripts/check-host-rpath @@ -11,6 +11,7 @@ export LC_ALL=C main() { local pkg="${1}" local hostdir="${2}" + local perpackagedir="${3}" local file ret # Remove duplicate and trailing '/' for proper match @@ -20,7 +21,7 @@ main() { while read file; do is_elf "${file}" || continue elf_needs_rpath "${file}" "${hostdir}" || continue - check_elf_has_rpath "${file}" "${hostdir}" && continue + check_elf_has_rpath "${file}" "${hostdir}" "${perpackagedir}" && continue if [ ${ret} -eq 0 ]; then ret=1 printf "***\n" @@ -57,6 +58,7 @@ elf_needs_rpath() { check_elf_has_rpath() { local file="${1}" local hostdir="${2}" + local perpackagedir="${3}" local rpath dir while read rpath; do @@ -65,6 +67,7 @@ check_elf_has_rpath() { dir="$( sed -r -e 's:/+:/:g; s:/$::;' <<<"${dir}" )" [ "${dir}" = "${hostdir}/lib" ] && return 0 [ "${dir}" = "\$ORIGIN/../lib" ] && return 0 + [[ ${dir} =~ ${perpackagedir}/[^/]*/host/lib ]] && return 0 done done < <( readelf -d "${file}" \ |sed -r -e '/.* \(R(UN)?PATH\) +Library r(un)?path: \[(.+)\]$/!d' \