From patchwork Thu Mar 12 09:15:29 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Petazzoni X-Patchwork-Id: 1253431 Return-Path: X-Original-To: incoming-buildroot@patchwork.ozlabs.org Delivered-To: patchwork-incoming-buildroot@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=busybox.net (client-ip=140.211.166.137; helo=fraxinus.osuosl.org; envelope-from=buildroot-bounces@busybox.net; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=bootlin.com Received: from fraxinus.osuosl.org (smtp4.osuosl.org [140.211.166.137]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 48dNTw5b6Fz9sP7 for ; Thu, 12 Mar 2020 20:15:48 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by fraxinus.osuosl.org (Postfix) with ESMTP id 88BC286AFD; Thu, 12 Mar 2020 09:15:44 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from fraxinus.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Vd_sPqH0pInu; Thu, 12 Mar 2020 09:15:42 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by fraxinus.osuosl.org (Postfix) with ESMTP id 20F0F862AC; Thu, 12 Mar 2020 09:15:41 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from silver.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by ash.osuosl.org (Postfix) with ESMTP id BFB401BF9B6 for ; Thu, 12 Mar 2020 09:15:38 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by silver.osuosl.org (Postfix) with ESMTP id BAD1A20373 for ; Thu, 12 Mar 2020 09:15:38 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from silver.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Yb+g9MYz+3RB for ; Thu, 12 Mar 2020 09:15:36 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from relay8-d.mail.gandi.net (relay8-d.mail.gandi.net [217.70.183.201]) by silver.osuosl.org (Postfix) with ESMTPS id 6D21B20361 for ; Thu, 12 Mar 2020 09:15:36 +0000 (UTC) X-Originating-IP: 90.89.41.158 Received: from localhost (lfbn-tou-1-1473-158.w90-89.abo.wanadoo.fr [90.89.41.158]) (Authenticated sender: thomas.petazzoni@bootlin.com) by relay8-d.mail.gandi.net (Postfix) with ESMTPSA id 47B481BF210; Thu, 12 Mar 2020 09:15:33 +0000 (UTC) From: Thomas Petazzoni To: buildroot@buildroot.org Date: Thu, 12 Mar 2020 10:15:29 +0100 Message-Id: <20200312091530.287681-1-thomas.petazzoni@bootlin.com> X-Mailer: git-send-email 2.24.1 MIME-Version: 1.0 Subject: [Buildroot] [PATCH 1/2] package/pkg-generic.mk: create folders before calling hooks X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Thomas Petazzoni Errors-To: buildroot-bounces@busybox.net Sender: "buildroot" In commit 0e2be4db8ab01d479177a3a187c22525752195ae ("package/pkg-generic: make file list logic parallel build compatible"), the logic to create the list of files installed by a particular package was significantly reworked to be compatible with top-level parallel build. Before this commit, there was only a after-install step of listing the files in HOST_DIR/TARGET_DIR/STAGING_DIR. But after this commit, we now have a before-install logic and an after-install logic. It turns out that when the before-install logic is called for the very first host package, $(HOST_DIR) doesn't exist yet, and therefore the cd $(2) fails, with an error message: /bin/sh: line 0: cd: /home/thomas/buildroot/output/host: No such file or directory In fact, $(HOST_DIR), $(STAGING_DIR), $(TARGET_DIR) and $(BINARIES_DIR) are created by the make rules for host installation, staging installation, target installation and images installation, but *after* calling the step_start hooks. So, we simply fix this problem by creating the directories *before* calling the step_start hooks. Signed-off-by: Thomas Petazzoni --- This is different solution than the one initially proposed at http://patchwork.ozlabs.org/patch/1252046/, following some discussion with Yann E. Morin on IRC. Since the creation of the directories is already done somewhere, it makes sense to rely on that rather than adding more code to create them in a difference place. This patch needs to be backported to 2020.02.x. --- package/pkg-generic.mk | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/package/pkg-generic.mk b/package/pkg-generic.mk index c1b9fe2e59..47238f2649 100644 --- a/package/pkg-generic.mk +++ b/package/pkg-generic.mk @@ -273,9 +273,9 @@ $(BUILD_DIR)/%/.stamp_built:: # Install to host dir $(BUILD_DIR)/%/.stamp_host_installed: + @mkdir -p $(HOST_DIR) @$(call step_start,install-host) @$(call MESSAGE,"Installing to host directory") - @mkdir -p $(HOST_DIR) $(foreach hook,$($(PKG)_PRE_INSTALL_HOOKS),$(call $(hook))$(sep)) +$($(PKG)_INSTALL_CMDS) $(foreach hook,$($(PKG)_POST_INSTALL_HOOKS),$(call $(hook))$(sep)) @@ -303,9 +303,9 @@ $(BUILD_DIR)/%/.stamp_host_installed: # empty when we use an internal toolchain. # $(BUILD_DIR)/%/.stamp_staging_installed: + @mkdir -p $(STAGING_DIR) @$(call step_start,install-staging) @$(call MESSAGE,"Installing to staging directory") - @mkdir -p $(STAGING_DIR) $(foreach hook,$($(PKG)_PRE_INSTALL_STAGING_HOOKS),$(call $(hook))$(sep)) +$($(PKG)_INSTALL_STAGING_CMDS) $(foreach hook,$($(PKG)_POST_INSTALL_STAGING_HOOKS),$(call $(hook))$(sep)) @@ -345,8 +345,8 @@ $(BUILD_DIR)/%/.stamp_staging_installed: # Install to images dir $(BUILD_DIR)/%/.stamp_images_installed: - @$(call step_start,install-image) @mkdir -p $(BINARIES_DIR) + @$(call step_start,install-image) @$(call MESSAGE,"Installing to images directory") $(foreach hook,$($(PKG)_PRE_INSTALL_IMAGES_HOOKS),$(call $(hook))$(sep)) +$($(PKG)_INSTALL_IMAGES_CMDS) @@ -356,9 +356,9 @@ $(BUILD_DIR)/%/.stamp_images_installed: # Install to target dir $(BUILD_DIR)/%/.stamp_target_installed: + @mkdir -p $(TARGET_DIR) @$(call step_start,install-target) @$(call MESSAGE,"Installing to target") - @mkdir -p $(TARGET_DIR) $(foreach hook,$($(PKG)_PRE_INSTALL_TARGET_HOOKS),$(call $(hook))$(sep)) +$($(PKG)_INSTALL_TARGET_CMDS) $(if $(BR2_INIT_SYSTEMD),\ From patchwork Thu Mar 12 09:15:30 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Petazzoni X-Patchwork-Id: 1253430 Return-Path: X-Original-To: incoming-buildroot@patchwork.ozlabs.org Delivered-To: patchwork-incoming-buildroot@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=busybox.net (client-ip=140.211.166.137; helo=fraxinus.osuosl.org; envelope-from=buildroot-bounces@busybox.net; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=bootlin.com Received: from fraxinus.osuosl.org (smtp4.osuosl.org [140.211.166.137]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 48dNTp1d6kz9sNg for ; Thu, 12 Mar 2020 20:15:42 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by fraxinus.osuosl.org (Postfix) with ESMTP id B44FF86121; Thu, 12 Mar 2020 09:15:40 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from fraxinus.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id pCfZFKz20qNh; Thu, 12 Mar 2020 09:15:40 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by fraxinus.osuosl.org (Postfix) with ESMTP id 2BD7E86153; Thu, 12 Mar 2020 09:15:40 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by ash.osuosl.org (Postfix) with ESMTP id A73251BF4E6 for ; Thu, 12 Mar 2020 09:15:38 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id A386F892A3 for ; Thu, 12 Mar 2020 09:15:38 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id b9KcxzVuUd0O for ; Thu, 12 Mar 2020 09:15:37 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from relay5-d.mail.gandi.net (relay5-d.mail.gandi.net [217.70.183.197]) by hemlock.osuosl.org (Postfix) with ESMTPS id 3B36A89274 for ; Thu, 12 Mar 2020 09:15:37 +0000 (UTC) X-Originating-IP: 90.89.41.158 Received: from localhost (lfbn-tou-1-1473-158.w90-89.abo.wanadoo.fr [90.89.41.158]) (Authenticated sender: thomas.petazzoni@bootlin.com) by relay5-d.mail.gandi.net (Postfix) with ESMTPSA id 3EB6D1C0009; Thu, 12 Mar 2020 09:15:34 +0000 (UTC) From: Thomas Petazzoni To: buildroot@buildroot.org Date: Thu, 12 Mar 2020 10:15:30 +0100 Message-Id: <20200312091530.287681-2-thomas.petazzoni@bootlin.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200312091530.287681-1-thomas.petazzoni@bootlin.com> References: <20200312091530.287681-1-thomas.petazzoni@bootlin.com> MIME-Version: 1.0 Subject: [Buildroot] [PATCH 2/2] Makefile: don't hang the build if there are no file lists X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Thomas Petazzoni Errors-To: buildroot-bounces@busybox.net Sender: "buildroot" In very limited configurations, it is possible to have a case where no .files-list-staging.txt files are created. In this case: cat $(sort $(wildcard $(BUILD_DIR)/*/.files-list-staging.txt)) > \ $(BUILD_DIR)/packages-file-list-staging.txt becomes: cat > \ $(BUILD_DIR)/packages-file-list-staging.txt which of course makes the build hang.. forever. So we fix this by checking the list is not empty. To keep the code readable, we introduce an intermediate variable to store the list of these files. Signed-off-by: Thomas Petazzoni --- This is a fix, it should be backported to 2020.02.x --- Makefile | 16 ++++++++++------ 1 file changed, 10 insertions(+), 6 deletions(-) diff --git a/Makefile b/Makefile index 074d08c04a..3fdd52c677 100644 --- a/Makefile +++ b/Makefile @@ -727,6 +727,10 @@ $(TARGETS_ROOTFS): target-finalize # Avoid the rootfs name leaking down the dependency chain target-finalize: ROOTFS= +TARGET_DIR_FILES_LISTS = $(sort $(wildcard $(BUILD_DIR)/*/.files-list.txt)) +HOST_DIR_FILES_LISTS = $(sort $(wildcard $(BUILD_DIR)/*/.files-list-host.txt)) +STAGING_DIR_FILES_LISTS = $(sort $(wildcard $(BUILD_DIR)/*/.files-list-staging.txt)) + .PHONY: host-finalize host-finalize: $(PACKAGES) $(HOST_DIR) $(HOST_DIR_SYMLINK) @$(call MESSAGE,"Finalizing host directory") @@ -807,12 +811,12 @@ endif # merged /usr touch $(TARGET_DIR)/usr - cat $(sort $(wildcard $(BUILD_DIR)/*/.files-list.txt)) > \ - $(BUILD_DIR)/packages-file-list.txt - cat $(sort $(wildcard $(BUILD_DIR)/*/.files-list-host.txt)) > \ - $(BUILD_DIR)/packages-file-list-host.txt - cat $(sort $(wildcard $(BUILD_DIR)/*/.files-list-staging.txt)) > \ - $(BUILD_DIR)/packages-file-list-staging.txt + $(if $(TARGET_DIR_FILES_LISTS), \ + cat $(TARGET_DIR_FILES_LISTS) > $(BUILD_DIR)/packages-file-list.txt) + $(if $(HOST_DIR_FILES_LISTS), \ + cat $(HOST_DIR_FILES_LISTS) > $(BUILD_DIR)/packages-file-list-host.txt) + $(if $(STAGING_DIR_FILES_LISTS), \ + cat $(STAGING_DIR_FILES_LISTS) > $(BUILD_DIR)/packages-file-list-staging.txt) .PHONY: target-post-image target-post-image: $(TARGETS_ROOTFS) target-finalize staging-finalize