From patchwork Fri Dec 28 10:43:34 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Petazzoni X-Patchwork-Id: 1019154 Return-Path: X-Original-To: incoming-buildroot@patchwork.ozlabs.org Delivered-To: patchwork-incoming-buildroot@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=busybox.net (client-ip=140.211.166.137; helo=fraxinus.osuosl.org; envelope-from=buildroot-bounces@busybox.net; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=bootlin.com Received: from fraxinus.osuosl.org (smtp4.osuosl.org [140.211.166.137]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 43R3HB1s10z9s1c for ; Fri, 28 Dec 2018 21:44:22 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by fraxinus.osuosl.org (Postfix) with ESMTP id A335E8586A; Fri, 28 Dec 2018 10:44:19 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from fraxinus.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 9a8jAh2fp_ZS; Fri, 28 Dec 2018 10:44:17 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by fraxinus.osuosl.org (Postfix) with ESMTP id 11D7684D56; Fri, 28 Dec 2018 10:44:17 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from silver.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by ash.osuosl.org (Postfix) with ESMTP id C79041C31E2 for ; Fri, 28 Dec 2018 10:44:07 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by silver.osuosl.org (Postfix) with ESMTP id BE24F21FEF for ; Fri, 28 Dec 2018 10:44:07 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from silver.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id vDAwh8fnFkVN for ; Fri, 28 Dec 2018 10:44:07 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mail.bootlin.com (mail.bootlin.com [62.4.15.54]) by silver.osuosl.org (Postfix) with ESMTP id E72D922881 for ; Fri, 28 Dec 2018 10:44:06 +0000 (UTC) Received: by mail.bootlin.com (Postfix, from userid 110) id 2138720A2E; Fri, 28 Dec 2018 11:44:06 +0100 (CET) Received: from localhost (mat33-2-88-189-187-82.fbx.proxad.net [88.189.187.82]) by mail.bootlin.com (Postfix) with ESMTPSA id E98E020A4A; Fri, 28 Dec 2018 11:43:51 +0100 (CET) From: Thomas Petazzoni To: "Arnout Vandecappelle (Essensium/Mind)" , "Yann E. MORIN" , Andreas Naumann , Buildroot List Date: Fri, 28 Dec 2018 11:43:34 +0100 Message-Id: <20181228104335.22379-8-thomas.petazzoni@bootlin.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20181228104335.22379-1-thomas.petazzoni@bootlin.com> References: <20181228104335.22379-1-thomas.petazzoni@bootlin.com> MIME-Version: 1.0 Subject: [Buildroot] [PATCH v7 7/8] docs/manual: add details about top-level parallel build support X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Thomas De Schampheleire , Thomas Petazzoni Errors-To: buildroot-bounces@busybox.net Sender: "buildroot" Signed-off-by: Thomas Petazzoni --- docs/manual/common-usage.txt | 44 +++++++++++++++++++++++++++++ docs/manual/faq-troubleshooting.txt | 3 ++ docs/manual/quickstart.txt | 8 +++--- 3 files changed, 51 insertions(+), 4 deletions(-) diff --git a/docs/manual/common-usage.txt b/docs/manual/common-usage.txt index e3d7578c85..ebe3238e38 100644 --- a/docs/manual/common-usage.txt +++ b/docs/manual/common-usage.txt @@ -329,6 +329,50 @@ Refer to the help text of this script for more details: utils/size-stats-compare -h ---------------- +[[top-level-parallel-build]] +=== Top-level parallel build + +Buildroot has always been capable of using parallel build on a per +package basis: each package is built by Buildroot using +make -jN+ (or +the equivalent invocation for non-make-based build systems). The level +of parallelism is by default number of CPUs + 1, but it can be +adjusted using the +BR2_JLEVEL+ configuration option. + +Until 2019.02, Buildroot was however building packages in a serial +fashion: each package was built one after the other, without +parallelization of the build between packages. As of 2019.02, +Buildroot has experimental support for *top-level parallel build*, +which allows some signicant build time savings by building packages +that have no dependency relationship in parallel. This feature is +however marked as experimental and is known to not work in all +situations. + +In order to use top-level parallel build, one must: + +. Enable the option +BR2_PER_PACKAGE_DIRECTORIES+ in the Buildroot +configuration + +. Use +make -jN+ when starting the Buildroot build + +Internally, the +BR2_PER_PACKAGE_DIRECTORIES+ will enable a mechanism +called *per-package directories*, which will have the following +effects: + +* Instead of a global _target_ directory and a global _host_ directory + common to all packages, per-package _target_ and _host_ directories + will be used, in +$(O)/per-package//target/+ and + +$(O)/per-package//host/+ respectively. Those folders will be + populated from the corresponding folders of the package dependencies + at the beginning of ++ build. The compiler and all other tools + will therefore only be able to see and access files installed by + dependencies explicitly listed by ++. + +* At the end of the build, the global _target_ and _host_ directories + will be populated, located in +$(O)/target+ and +$(O)/host+ + respectively. This means that during the build, those folders will + be empty and it's only at the very end of the build that they will + be populated. + include::eclipse-integration.txt[] include::advanced.txt[] diff --git a/docs/manual/faq-troubleshooting.txt b/docs/manual/faq-troubleshooting.txt index b144c9e7f0..5adf3fa6ce 100644 --- a/docs/manual/faq-troubleshooting.txt +++ b/docs/manual/faq-troubleshooting.txt @@ -239,3 +239,6 @@ help reduce the build time: * Buy new hardware. SSDs and lots of RAM are key to speeding up the builds. + + * Experiment with top-level parallel build, see + xref:top-level-parallel-build[]. diff --git a/docs/manual/quickstart.txt b/docs/manual/quickstart.txt index 74158ae249..77b73ef116 100644 --- a/docs/manual/quickstart.txt +++ b/docs/manual/quickstart.txt @@ -60,10 +60,10 @@ To start the build process, simply run: $ make -------------------- -You *should never* use +make -jN+ with Buildroot: top-level parallel -make is currently not supported. Instead, use the +BR2_JLEVEL+ option -to tell Buildroot to run the compilation of each individual package -with +make -jN+. +By default, Buildroot does not support top-level parallel build, so +running +make -jN+ is not necessary. There is however experimental +support for top-level parallel build, see +xref:top-level-parallel-build[]. The `make` command will generally perform the following steps: