From patchwork Fri Mar 23 21:48:13 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Petazzoni X-Patchwork-Id: 890249 Return-Path: X-Original-To: incoming-buildroot@patchwork.ozlabs.org Delivered-To: patchwork-incoming-buildroot@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=busybox.net (client-ip=140.211.166.133; helo=hemlock.osuosl.org; envelope-from=buildroot-bounces@busybox.net; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=bootlin.com Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 407HGf1SDJz9s0p for ; Sat, 24 Mar 2018 08:48:26 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id 832278A4F3; Fri, 23 Mar 2018 21:48:24 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id XtxSyVSs+epJ; Fri, 23 Mar 2018 21:48:23 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by hemlock.osuosl.org (Postfix) with ESMTP id 910848A4EB; Fri, 23 Mar 2018 21:48:23 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by ash.osuosl.org (Postfix) with ESMTP id 66EB11C2039 for ; Fri, 23 Mar 2018 21:48:20 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by whitealder.osuosl.org (Postfix) with ESMTP id 6428082785 for ; Fri, 23 Mar 2018 21:48:20 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from whitealder.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id XLqUwwq1rmYQ for ; Fri, 23 Mar 2018 21:48:19 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mail.bootlin.com (mail.bootlin.com [62.4.15.54]) by whitealder.osuosl.org (Postfix) with ESMTP id 6C184827E4 for ; Fri, 23 Mar 2018 21:48:19 +0000 (UTC) Received: by mail.bootlin.com (Postfix, from userid 110) id 25BB52072D; Fri, 23 Mar 2018 22:48:17 +0100 (CET) Received: from localhost (LFbn-TOU-1-408-85.w86-206.abo.wanadoo.fr [86.206.234.85]) by mail.bootlin.com (Postfix) with ESMTPSA id BD1B320728; Fri, 23 Mar 2018 22:48:16 +0100 (CET) From: Thomas Petazzoni To: Buildroot List , "Yann E. MORIN" Date: Fri, 23 Mar 2018 22:48:13 +0100 Message-Id: <20180323214815.19831-2-thomas.petazzoni@bootlin.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180323214815.19831-1-thomas.petazzoni@bootlin.com> References: <20180323214815.19831-1-thomas.petazzoni@bootlin.com> Subject: [Buildroot] [PATCH v2 1/3] toolchain-configs.csv: re-organize for test-pkg X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.24 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Thomas Petazzoni , Thomas Petazzoni MIME-Version: 1.0 Errors-To: buildroot-bounces@busybox.net Sender: "buildroot" From: Thomas Petazzoni This commit reorganizes the toolchain-configs.csv so that the first toolchains are a subset of "useful" toolchains to be tested by contributors to validate a package. This subset is the one that will be used by default by test-pkg. Signed-off-by: Thomas Petazzoni Reviewed-by: "Yann E. MORIN" --- Changes since v1: - Add better comments to explain the two "sections" of the toolchain list, as suggested by Yann E. Morin. --- .../autobuild/toolchain-configs.csv | 32 ++++++++++++++++++---- 1 file changed, 26 insertions(+), 6 deletions(-) diff --git a/support/config-fragments/autobuild/toolchain-configs.csv b/support/config-fragments/autobuild/toolchain-configs.csv index 2010113f44..1bce7ac577 100644 --- a/support/config-fragments/autobuild/toolchain-configs.csv +++ b/support/config-fragments/autobuild/toolchain-configs.csv @@ -1,20 +1,41 @@ +# This file is sorted by "importance" of toolchains, so that by +# default test-pkg tests a useful subset of toolchains + +# Toolchains used by default by test-pkg: + +# Test a regular uClibc toolchain +support/config-fragments/autobuild/br-arm-full.config,x86_64 + +# Test a toolchain with glibc and a very recent gcc version +support/config-fragments/autobuild/br-arm-cortex-a9-glibc.config,x86_64 + +# Test a noMMU toolchain with no dynamic library support +support/config-fragments/autobuild/br-arm-cortex-m4-full.config,x86_64 + +# Test a musl toolchain +support/config-fragments/autobuild/br-x86-64-musl.config,x86_64 + +# Test a noMMU toolchain with dynamic library support +support/config-fragments/autobuild/br-bfin-full.config,x86_64 + +# Test a MMU toolchain without dynamic library support +support/config-fragments/autobuild/br-arm-full-static.config,x86_64 + +# Test a toolchain with an old gcc version (gcc 4.8) support/config-fragments/autobuild/armv5-ctng-linux-gnueabi.config,x86 + +# Toolchains used by test-pkg only when the '-a' option is passed: support/config-fragments/autobuild/armv7-ctng-linux-gnueabihf.config,x86 support/config-fragments/autobuild/br-aarch64-glibc.config,x86_64 support/config-fragments/autobuild/br-arc-full-internal.config,any support/config-fragments/autobuild/br-arc-internal-glibc.config,any support/config-fragments/autobuild/br-arcle-hs38.config,x86_64 support/config-fragments/autobuild/br-arm-basic.config,x86_64 -support/config-fragments/autobuild/br-arm-cortex-a9-glibc.config,x86_64 support/config-fragments/autobuild/br-arm-cortex-a9-musl.config,x86_64 -support/config-fragments/autobuild/br-arm-cortex-m4-full.config,x86_64 -support/config-fragments/autobuild/br-arm-full.config,x86_64 support/config-fragments/autobuild/br-arm-full-nothread.config,x86_64 -support/config-fragments/autobuild/br-arm-full-static.config,x86_64 support/config-fragments/autobuild/br-arm-internal-full.config,any support/config-fragments/autobuild/br-arm-internal-glibc.config,any support/config-fragments/autobuild/br-arm-internal-musl.config,any -support/config-fragments/autobuild/br-bfin-full.config,x86_64 support/config-fragments/autobuild/br-i386-pentium4-full.config,x86_64 support/config-fragments/autobuild/br-i386-pentium-mmx-musl.config,x86_64 support/config-fragments/autobuild/br-m68k-5208-full.config,x86_64 @@ -36,7 +57,6 @@ support/config-fragments/autobuild/br-sh4-full.config,x86_64 support/config-fragments/autobuild/br-sparc-uclibc.config,x86_64 support/config-fragments/autobuild/br-sparc64-glibc.config,x86_64 support/config-fragments/autobuild/br-x86-64-core2-full.config,x86_64 -support/config-fragments/autobuild/br-x86-64-musl.config,x86_64 support/config-fragments/autobuild/br-xtensa-full.config,x86_64 support/config-fragments/autobuild/br-xtensa-full-internal.config,any support/config-fragments/autobuild/i686-ctng-linux-gnu.config,x86 From patchwork Fri Mar 23 21:48:14 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Petazzoni X-Patchwork-Id: 890252 Return-Path: X-Original-To: incoming-buildroot@patchwork.ozlabs.org Delivered-To: patchwork-incoming-buildroot@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=busybox.net (client-ip=140.211.166.136; helo=silver.osuosl.org; envelope-from=buildroot-bounces@busybox.net; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=bootlin.com Received: from silver.osuosl.org (smtp3.osuosl.org [140.211.166.136]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 407HGh361gz9s0p for ; Sat, 24 Mar 2018 08:48:28 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by silver.osuosl.org (Postfix) with ESMTP id 73556249E5; Fri, 23 Mar 2018 21:48:25 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from silver.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id sEIW0nQ1k2QI; Fri, 23 Mar 2018 21:48:24 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by silver.osuosl.org (Postfix) with ESMTP id 36E9B22891; Fri, 23 Mar 2018 21:48:24 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by ash.osuosl.org (Postfix) with ESMTP id 79B131CF07A for ; Fri, 23 Mar 2018 21:48:20 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id 766B38A4EB for ; Fri, 23 Mar 2018 21:48:20 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id QY-RizfNhj1I for ; Fri, 23 Mar 2018 21:48:19 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mail.bootlin.com (mail.bootlin.com [62.4.15.54]) by hemlock.osuosl.org (Postfix) with ESMTP id 7BDA08A4E8 for ; Fri, 23 Mar 2018 21:48:19 +0000 (UTC) Received: by mail.bootlin.com (Postfix, from userid 110) id 76BA32082C; Fri, 23 Mar 2018 22:48:17 +0100 (CET) Received: from localhost (LFbn-TOU-1-408-85.w86-206.abo.wanadoo.fr [86.206.234.85]) by mail.bootlin.com (Postfix) with ESMTPSA id 157BC203B8; Fri, 23 Mar 2018 22:48:17 +0100 (CET) From: Thomas Petazzoni To: Buildroot List , "Yann E. MORIN" Date: Fri, 23 Mar 2018 22:48:14 +0100 Message-Id: <20180323214815.19831-3-thomas.petazzoni@bootlin.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180323214815.19831-1-thomas.petazzoni@bootlin.com> References: <20180323214815.19831-1-thomas.petazzoni@bootlin.com> Subject: [Buildroot] [PATCH v2 2/3] test-pkg: test a subset of toolchains by default, add -a and -n options X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.24 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Thomas Petazzoni MIME-Version: 1.0 Errors-To: buildroot-bounces@busybox.net Sender: "buildroot" From: Thomas Petazzoni During the latest Buildroot Developers meeting, we discussed that test-pkg would perhaps be more widely used if it tested a smaller subset of toolchains. Indeed, it currently tests 47 toolchains, which takes very long to build. Several of the toolchain configurations are quite similar, and it is perhaps not necessary for contributors to test them all before submitting a package. Therefore, this commit changes the test-pkg script to only test a subset of the toolchain configurations by default. The N first configurations of the CSV files are tested, where N is hard-coded in the script. The CSV file has therefore been re-organized to have the first N toolchains be the most important ones. A -a/--all option is added to test with all toolchains, while a -n/--number option is added to test with the first N toolchains, N being passed on the command line. Note that the list of toolchains (built in the "toolchains" shell variable) is no longer sorted. Indeed, when the first N toolchains are tested, we want them to be tested in the same order as they are listed in the CSV file, as we are careful to order them in an interesting order. We only sort when all toolchains are tested. Signed-off-by: Thomas Petazzoni Tested-by: Matt Weber Reviewed-by: "Yann E. MORIN" --- Changes since v1: - Keep alphabetic sorting in the definition of the 'o' variable that contains the list of short options. Suggested by Yann E. Morin. 'h' for help remains a special case and therefore remains at the first position. - Keep alphabetic sorting when handling options. Here as well, --help remains first, as it already was. Suggested by Yann E. Morin. - Use spaces and not tabs for indentation. Suggested by Yann E. Morin. --- utils/test-pkg | 56 +++++++++++++++++++++++++++++++++++++++++++++++++------- 1 file changed, 49 insertions(+), 7 deletions(-) diff --git a/utils/test-pkg b/utils/test-pkg index cd7b9dad7b..2e069b6005 100755 --- a/utils/test-pkg +++ b/utils/test-pkg @@ -5,28 +5,37 @@ TOOLCHAINS_CSV='support/config-fragments/autobuild/toolchain-configs.csv' main() { local o O opts - local cfg dir pkg random toolchains_dir toolchain + local cfg dir pkg random toolchains_dir toolchain all number mode local ret nb nb_skip nb_fail nb_legal nb_tc build_dir local -a toolchains - o='hc:d:p:r:t:' + o='hac:d:n:p:r:t:' O='help,config-snippet:build-dir:package:,random:,toolchains-dir:' opts="$(getopt -n "${my_name}" -o "${o}" -l "${O}" -- "${@}")" eval set -- "${opts}" random=0 + all=0 + number=0 + mode=0 toolchains_csv="${TOOLCHAINS_CSV}" while [ ${#} -gt 0 ]; do case "${1}" in (-h|--help) help; exit 0 ;; + (-a|--all) + all=1; shift 1 + ;; (-c|--config-snippet) cfg="${2}"; shift 2 ;; (-d|--build-dir) dir="${2}"; shift 2 ;; + (-n|--number) + number="${2}"; shift 2 + ;; (-p|--package) pkg="${2}"; shift 2 ;; @@ -51,15 +60,37 @@ main() { dir="${HOME}/br-test-pkg" fi + if [ ${random} -gt 0 ]; then + mode=$((mode+1)) + fi + + if [ ${number} -gt 0 ]; then + mode=$((mode+1)) + fi + + if [ ${all} -eq 1 ]; then + mode=$((mode+1)) + fi + + # Default mode is to test the N first toolchains, which have been + # chosen to be a good selection of toolchains. + if [ ${mode} -eq 0 ] ; then + number=7 + elif [ ${mode} -gt 1 ] ; then + printf "error: --all, --number and --random are mutually exclusive\n" >&2; exit 1 + fi + # Extract the URLs of the toolchains; drop internal toolchains # E.g.: http://server/path/to/name.config,arch,libc # --> http://server/path/to/name.config toolchains=($(sed -r -e 's/,.*//; /internal/d; /^#/d; /^$/d;' "${toolchains_csv}" \ |if [ ${random} -gt 0 ]; then \ sort -R |head -n ${random} - else - cat - fi |sort + elif [ ${number} -gt 0 ]; then \ + head -n ${number} + else + sort + fi ) ) @@ -154,6 +185,10 @@ toolchain config fragment and the required host architecture, separated by a comma. The config fragments should contain only the toolchain and architecture settings. +By default, a useful subset of toolchains is tested. If needed, all +toolchains can be tested (-a), an arbitrary number of toolchains (-n +in order, -r for random). + Options: -h, --help @@ -170,9 +205,16 @@ Options: Test-build the package PKG, by running 'make PKG'; if not specified, just runs 'make'. + -a, --all + Test all toolchains, instead of the default subset defined by + Buildroot developers. + + -n N, --number N + Test N toolchains, in the order defined in the toolchain CSV + file. + -r N, --random N - Limit the tests to the N randomly selected toolchains, instead of - building with all toolchains. + Limit the tests to the N randomly selected toolchains. -t CSVFILE, --toolchains-csv CSVFILE CSV file containing the paths to config fragments of toolchains to From patchwork Fri Mar 23 21:48:15 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Petazzoni X-Patchwork-Id: 890250 Return-Path: X-Original-To: incoming-buildroot@patchwork.ozlabs.org Delivered-To: patchwork-incoming-buildroot@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=busybox.net (client-ip=140.211.166.138; helo=whitealder.osuosl.org; envelope-from=buildroot-bounces@busybox.net; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=bootlin.com Received: from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 407HGg1vXCz9s0t for ; Sat, 24 Mar 2018 08:48:27 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by whitealder.osuosl.org (Postfix) with ESMTP id 68C5482EE0; Fri, 23 Mar 2018 21:48:24 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from whitealder.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ct6v10MMS2FG; Fri, 23 Mar 2018 21:48:22 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by whitealder.osuosl.org (Postfix) with ESMTP id C323A827E4; Fri, 23 Mar 2018 21:48:22 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by ash.osuosl.org (Postfix) with ESMTP id 3D86D1CF07A for ; Fri, 23 Mar 2018 21:48:20 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by whitealder.osuosl.org (Postfix) with ESMTP id 397878284F for ; Fri, 23 Mar 2018 21:48:20 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from whitealder.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id klBnVtSE33Be for ; Fri, 23 Mar 2018 21:48:19 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mail.bootlin.com (mail.bootlin.com [62.4.15.54]) by whitealder.osuosl.org (Postfix) with ESMTP id 6B36B82785 for ; Fri, 23 Mar 2018 21:48:19 +0000 (UTC) Received: by mail.bootlin.com (Postfix, from userid 110) id AC68320728; Fri, 23 Mar 2018 22:48:17 +0100 (CET) Received: from localhost (LFbn-TOU-1-408-85.w86-206.abo.wanadoo.fr [86.206.234.85]) by mail.bootlin.com (Postfix) with ESMTPSA id 6515320715; Fri, 23 Mar 2018 22:48:17 +0100 (CET) From: Thomas Petazzoni To: Buildroot List , "Yann E. MORIN" Date: Fri, 23 Mar 2018 22:48:15 +0100 Message-Id: <20180323214815.19831-4-thomas.petazzoni@bootlin.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180323214815.19831-1-thomas.petazzoni@bootlin.com> References: <20180323214815.19831-1-thomas.petazzoni@bootlin.com> Subject: [Buildroot] [PATCH v2 3/3] docs/manual: update the documentation about test-pkg X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.24 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Thomas Petazzoni MIME-Version: 1.0 Errors-To: buildroot-bounces@busybox.net Sender: "buildroot" From: Thomas Petazzoni Signed-off-by: Thomas Petazzoni Acked-by: "Yann E. MORIN" --- docs/manual/adding-packages-tips.txt | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/docs/manual/adding-packages-tips.txt b/docs/manual/adding-packages-tips.txt index 19577fa821..f37f9874e7 100644 --- a/docs/manual/adding-packages-tips.txt +++ b/docs/manual/adding-packages-tips.txt @@ -77,9 +77,14 @@ and what package to test: $ ./utils/test-pkg -c libcurl.config -p libcurl ---- -This will try to build your package against all the toolchains used -by the autobuilders (except for the internal toolchains, because it takes -too long to do so). The output lists all toolchains and the corresponding +By default, +test-pkg+ will build your package against a subset of the +toolchains used by the autobuilders, which has been selected by the +Buildroot developers as being the most useful and representative +subset. If you want to test all toolchains, pass the +-a+ option. Note +that in any case, internal toolchains are excluded as they take too +long to build. + +The output lists all toolchains that are tested and the corresponding result (excerpt, results are fake): ----