From patchwork Tue Nov 5 16:46:44 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Petazzoni X-Patchwork-Id: 1189847 Return-Path: X-Original-To: incoming-buildroot@patchwork.ozlabs.org Delivered-To: patchwork-incoming-buildroot@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=busybox.net (client-ip=140.211.166.136; helo=silver.osuosl.org; envelope-from=buildroot-bounces@busybox.net; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=bootlin.com Received: from silver.osuosl.org (smtp3.osuosl.org [140.211.166.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 476wYq4lKLz9s4Y for ; Wed, 6 Nov 2019 03:47:11 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by silver.osuosl.org (Postfix) with ESMTP id EA2CD221F0; Tue, 5 Nov 2019 16:47:06 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from silver.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id D7Wyf1zLqgXK; Tue, 5 Nov 2019 16:47:04 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by silver.osuosl.org (Postfix) with ESMTP id 6AB8820456; Tue, 5 Nov 2019 16:47:04 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from fraxinus.osuosl.org (smtp4.osuosl.org [140.211.166.137]) by ash.osuosl.org (Postfix) with ESMTP id 54C591BF5B4 for ; Tue, 5 Nov 2019 16:47:01 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by fraxinus.osuosl.org (Postfix) with ESMTP id 4ADF988EF9 for ; Tue, 5 Nov 2019 16:47:01 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from fraxinus.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id wV8jZygImoxq for ; Tue, 5 Nov 2019 16:46:59 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from relay7-d.mail.gandi.net (relay7-d.mail.gandi.net [217.70.183.200]) by fraxinus.osuosl.org (Postfix) with ESMTPS id 6153488ED6 for ; Tue, 5 Nov 2019 16:46:58 +0000 (UTC) X-Originating-IP: 86.250.200.211 Received: from localhost (lfbn-1-17395-211.w86-250.abo.wanadoo.fr [86.250.200.211]) (Authenticated sender: thomas.petazzoni@bootlin.com) by relay7-d.mail.gandi.net (Postfix) with ESMTPSA id ADFD72000E; Tue, 5 Nov 2019 16:46:55 +0000 (UTC) From: Thomas Petazzoni To: Buildroot List Date: Tue, 5 Nov 2019 17:46:44 +0100 Message-Id: <20191105164646.23820-6-thomas.petazzoni@bootlin.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20191105164646.23820-1-thomas.petazzoni@bootlin.com> References: <20191105164646.23820-1-thomas.petazzoni@bootlin.com> MIME-Version: 1.0 Subject: [Buildroot] [PATCH next v8 5/6] docs/manual: add details about top-level parallel build support X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Thomas De Schampheleire , Thomas Petazzoni Errors-To: buildroot-bounces@busybox.net Sender: "buildroot" Signed-off-by: Thomas Petazzoni --- docs/manual/common-usage.txt | 47 +++++++++++++++++++++++++++++ docs/manual/faq-troubleshooting.txt | 3 ++ docs/manual/quickstart.txt | 8 ++--- 3 files changed, 54 insertions(+), 4 deletions(-) diff --git a/docs/manual/common-usage.txt b/docs/manual/common-usage.txt index a87980d4a2..2d315de440 100644 --- a/docs/manual/common-usage.txt +++ b/docs/manual/common-usage.txt @@ -347,6 +347,53 @@ Refer to the help text of this script for more details: utils/size-stats-compare -h ---------------- +[[top-level-parallel-build]] +=== Top-level parallel build + +.Note +This section deals with a very experimental feature, which is known to +break even in some non-unusual situations. Use at your own risk. + +Buildroot has always been capable of using parallel build on a per +package basis: each package is built by Buildroot using +make -jN+ (or +the equivalent invocation for non-make-based build systems). The level +of parallelism is by default number of CPUs + 1, but it can be +adjusted using the +BR2_JLEVEL+ configuration option. + +Until 2019.02, Buildroot was however building packages in a serial +fashion: each package was built one after the other, without +parallelization of the build between packages. As of 2019.02, +Buildroot has experimental support for *top-level parallel build*, +which allows some signicant build time savings by building packages +that have no dependency relationship in parallel. This feature is +however marked as experimental and is known not to work in some cases. + +In order to use top-level parallel build, one must: + +. Enable the option +BR2_PER_PACKAGE_DIRECTORIES+ in the Buildroot +configuration + +. Use +make -jN+ when starting the Buildroot build + +Internally, the +BR2_PER_PACKAGE_DIRECTORIES+ will enable a mechanism +called *per-package directories*, which will have the following +effects: + +* Instead of a global _target_ directory and a global _host_ directory + common to all packages, per-package _target_ and _host_ directories + will be used, in +$(O)/per-package//target/+ and + +$(O)/per-package//host/+ respectively. Those folders will be + populated from the corresponding folders of the package dependencies + at the beginning of ++ build. The compiler and all other tools + will therefore only be able to see and access files installed by + dependencies explicitly listed by ++. + +* At the end of the build, the global _target_ and _host_ directories + will be populated, located in +$(O)/target+ and +$(O)/host+ + respectively. This means that during the build, those folders will + be empty and it's only at the very end of the build that they will + be populated. + include::eclipse-integration.txt[] include::advanced.txt[] diff --git a/docs/manual/faq-troubleshooting.txt b/docs/manual/faq-troubleshooting.txt index b144c9e7f0..5adf3fa6ce 100644 --- a/docs/manual/faq-troubleshooting.txt +++ b/docs/manual/faq-troubleshooting.txt @@ -239,3 +239,6 @@ help reduce the build time: * Buy new hardware. SSDs and lots of RAM are key to speeding up the builds. + + * Experiment with top-level parallel build, see + xref:top-level-parallel-build[]. diff --git a/docs/manual/quickstart.txt b/docs/manual/quickstart.txt index 74158ae249..77b73ef116 100644 --- a/docs/manual/quickstart.txt +++ b/docs/manual/quickstart.txt @@ -60,10 +60,10 @@ To start the build process, simply run: $ make -------------------- -You *should never* use +make -jN+ with Buildroot: top-level parallel -make is currently not supported. Instead, use the +BR2_JLEVEL+ option -to tell Buildroot to run the compilation of each individual package -with +make -jN+. +By default, Buildroot does not support top-level parallel build, so +running +make -jN+ is not necessary. There is however experimental +support for top-level parallel build, see +xref:top-level-parallel-build[]. The `make` command will generally perform the following steps: