From patchwork Mon Jun 28 20:15:15 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Yann E. MORIN" X-Patchwork-Id: 1498122 Return-Path: X-Original-To: incoming-buildroot@patchwork.ozlabs.org Delivered-To: patchwork-incoming-buildroot@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=busybox.net (client-ip=140.211.166.136; helo=smtp3.osuosl.org; envelope-from=buildroot-bounces@busybox.net; receiver=) Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=A+v1AzBT; dkim-atps=neutral Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4GDJld6qSDz9sX3 for ; Tue, 29 Jun 2021 06:16:13 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 08F2560800; Mon, 28 Jun 2021 20:16:12 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id HXpoVcKG2M3q; Mon, 28 Jun 2021 20:16:11 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by smtp3.osuosl.org (Postfix) with ESMTP id 2973C607E6; Mon, 28 Jun 2021 20:16:10 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by ash.osuosl.org (Postfix) with ESMTP id 3AB1C1BF2B0 for ; Mon, 28 Jun 2021 20:15:22 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 37C2E40265 for ; Mon, 28 Jun 2021 20:15:22 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Authentication-Results: smtp2.osuosl.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id eyH0didUmy0d for ; Mon, 28 Jun 2021 20:15:21 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.8.0 Received: from mail-wm1-x330.google.com (mail-wm1-x330.google.com [IPv6:2a00:1450:4864:20::330]) by smtp2.osuosl.org (Postfix) with ESMTPS id 26E0A40251 for ; Mon, 28 Jun 2021 20:15:21 +0000 (UTC) Received: by mail-wm1-x330.google.com with SMTP id l18-20020a1ced120000b029014c1adff1edso260342wmh.4 for ; Mon, 28 Jun 2021 13:15:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ZXm1uOzJaF1LEQqGwu5yy6hCOoKdYj6T/0yH6C8Jz3s=; b=A+v1AzBTrtUzX2PJ3SOhWN4eoz5fa8FVOag4Nna4jO4GvHejeRXLM9UdO0OOCXAv3N JqPzuseXrZqnc1YRdjIzuALF9RG6/P69ZbykTtb5ls3Qky0FfXsuElCHf8nMAC7NAkQQ GHEoBnR3OXzAMYa2tY/WiYJgU3brmrrPavsz+mqye7uiaM5eFwx+BwbjwjInILiTRMXm FhQm25UZvWHcuMaEdWFYCGHwWkpYsWiyethdkWiOvokH2W02zgLLDIhseqJyBb5d5Gw9 H9N/lFBrqLNfwt2seYok0jwIk7ioBylR28L43EwbB9jf5rFmvk17UKsvwWZTHjbwm0Y9 b/bA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references:mime-version:content-transfer-encoding; bh=ZXm1uOzJaF1LEQqGwu5yy6hCOoKdYj6T/0yH6C8Jz3s=; b=j2AdHGtyZwzkTy3+sJ/aQuvhK6Rxv4Vuyr8F986UNsGctgcCBvV5l3rkxBT9ElchhP lvljyGLiC0yVNrQhCdTFm5XLaNZk1VD4CToDiFxWgqs7SLZWID8Q1MNkTS8+0QR70/nO J4CqjYvfn64fNUgHUf6YOuD1DNxzIFslqsndnGfGSclkLQ8gjF5mqU63ACTnP1YumOTe KinvlJ2ixWvGbizQPPsHCYKtphFMgmZGqFlS88iZ5bMP27aH71GVblZGnIYGA5ZAK0se RjSpoHvXWxxUC4EpPcpfzpfg04L06jIvhbQqyasApC5EYVdp/JburdNV8S+20yyL8E/Q sbpw== X-Gm-Message-State: AOAM530mU4P1BDtsha8wOqDEA72reYMVHXIreRaIU2LrqA4yczsuVkMf CdK8pHagPe/lHjjqT18Hfrlff4HeXvw= X-Google-Smtp-Source: ABdhPJzPrT+CnI36nD2k5iuFkZFemi2YKYqOZeWNZ5ml/d7mE4bVAvQKsE0pY/2LQSHrqh4i9/Vbrw== X-Received: by 2002:a05:600c:358e:: with SMTP id p14mr14448630wmq.136.1624911319406; Mon, 28 Jun 2021 13:15:19 -0700 (PDT) Received: from scaer.home ([2a01:cb19:8b51:cb00:4096:d846:2edb:69d9]) by smtp.gmail.com with ESMTPSA id y7sm482505wma.22.2021.06.28.13.15.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 28 Jun 2021 13:15:19 -0700 (PDT) From: "Yann E. MORIN" To: buildroot@buildroot.org Date: Mon, 28 Jun 2021 22:15:15 +0200 Message-Id: <354f4577a7b6d3b1e5570a7c678e2d60bcf866e7.1624911306.git.yann.morin.1998@free.fr> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: MIME-Version: 1.0 Subject: [Buildroot] [PATCH 4/4 v5] utils/test-pkg: add gitlab-ci support X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Romain Naour , "Yann E . MORIN" Errors-To: buildroot-bounces@busybox.net Sender: "buildroot" From: Romain Naour The gitlab-ci support in test-pkg allows to parallelize the test-pkg work into several gitlab jobs. It's much faster than local serialized testing. To triger this, a developper will have to add, to the latest commit of their branch, a token on its own line, followd by a configuration fragment, e.g.: test-pkg config: SOME_OPTION=y # OTHER_OPTION is not set SOME_VARIABLE="some value" This configuration fragment is used as input to test-pkg. To be able to generate one job per test to run, we need the list of tests in the parent pipeline, and the individual .config files (one per test) in the child pipeline. We use the newly-introduce --prepare-only mode to test-pkg, and collect all the generated .config files as artefacts; those are inherited in the child pipeline via the "needs::pipeline" and "needs::job" directives. This is a bit tricky, and is best described by the Gitlab-CI documentation [0]. We also list those .config files to generate the actual list of jobs to run in the child pipeline. Notes: - if the user provides an empty fragment, this is considered an error: indeed, without a fragment (and the package name), there is no way to know what to test; - if that fragment yields an empty list of tests, then there is nothing to test either, so that is also considered an error. [0] https://docs.gitlab.com/ee/ci/yaml/README.html#artifact-downloads-to-child-pipelines Signed-off-by: Romain Naour Cc: Arnout Vandecappelle (Essensium/Mind) [yann.morin.1998@free.fr: - split the change to test-pkg to its own patch - generate the actual yml snippet in support/scripts/generate-gitlab-ci-yml, listing the .config files created by test-pkg - some code-style-candies... ] Signed-off-by: Yann E. MORIN --- v5: Split the change to test-pkg to its own patch List the .config files Also store the missing.config files as artefacts v4: reworked by Yann: tst-pkg only lists, does not generate the yml code The yml code is generated by support/scripts/generate-gitlab-ci-yml Empty fragment is an error Fragment that yierlds no test is also an error v3: Implement Arnout's review: http://lists.busybox.net/pipermail/buildroot/2021-May/310656.html Enable artifacts download from child-pipeline v2: Rework this patch following Arnout review use CI_COMMIT_DESCRIPTION remove .config from artifacts but keep images directory since it can be useful for further issue investigation use the "br-test-pkg" prefix for test-pkg jobs --- .gitlab-ci.yml | 5 +++++ support/misc/gitlab-ci.yml.in | 22 +++++++++++++++++++++ support/scripts/generate-gitlab-ci-yml | 27 +++++++++++++++++++++++++- 3 files changed, 53 insertions(+), 1 deletion(-) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index e85ac32033..bf9f2dca6c 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -11,8 +11,11 @@ generate-gitlab-ci-yml: stage: generate-gitlab-ci script: ./support/scripts/generate-gitlab-ci-yml support/misc/gitlab-ci.yml.in > generated-gitlab-ci.yml artifacts: + when: always paths: - generated-gitlab-ci.yml + - br-test-pkg/*/.config + - br-test-pkg/*/missing.config buildroot-pipeline: stage: build @@ -21,3 +24,5 @@ buildroot-pipeline: - artifact: generated-gitlab-ci.yml job: generate-gitlab-ci-yml strategy: depend + variables: + PARENT_PIPELINE_ID: $CI_PIPELINE_ID diff --git a/support/misc/gitlab-ci.yml.in b/support/misc/gitlab-ci.yml.in index 1ee3772154..be7951b3d2 100644 --- a/support/misc/gitlab-ci.yml.in +++ b/support/misc/gitlab-ci.yml.in @@ -80,3 +80,25 @@ - test-output/*/.config - test-output/*/images/* +.test_pkg: + stage: build + before_script: + - OUTPUT_DIR=${CI_JOB_NAME} + script: + - echo "Configure Buildroot for ${OUTPUT_DIR}" + - make O=${OUTPUT_DIR} syncconfig + - make O=${OUTPUT_DIR} savedefconfig + - echo 'Build buildroot' + - *run_make + needs: + - pipeline: $PARENT_PIPELINE_ID + job: generate-gitlab-ci-yml + artifacts: + when: always + expire_in: 2 weeks + paths: + - build.log + - br-test-pkg/*/.config + - br-test-pkg/*/defconfig + - br-test-pkg/*/build/build-time.log + - br-test-pkg/*/build/packages-file-list*.txt diff --git a/support/scripts/generate-gitlab-ci-yml b/support/scripts/generate-gitlab-ci-yml index 3f498e08fd..063c5081da 100755 --- a/support/scripts/generate-gitlab-ci-yml +++ b/support/scripts/generate-gitlab-ci-yml @@ -23,7 +23,7 @@ _EOF_ gen_tests() { local -a basics defconfigs runtimes - local do_basics do_defconfigs do_runtime + local do_basics do_defconfigs do_runtime do_testpkg local defconfigs_ext cfg tst basics=( DEVELOPERS flake8 package ) @@ -77,9 +77,30 @@ gen_tests() { esac fi + # Retrieve defconfig for test-pkg from the git commit message (if any) + if grep -q -E '^test-pkg config:$' <<<"${CI_COMMIT_DESCRIPTION}"; then + sed -r -n -e '/^test-pkg config:$/{:a;n;p;ba;}' \ + <<<"${CI_COMMIT_DESCRIPTION}" \ + >defconfig.frag + if [ ! -s defconfig.frag ]; then + printf "Empty configuration fragment.\n" >&2; exit 1 + fi + # Use --all since we expect the user having already pre-tested the + # new package with the default subset of toolchains. + ./utils/test-pkg \ + --all --prepare-only \ + --config-snippet defconfig.frag \ + --build-dir br-test-pkg >&2 + do_testpkg=( $(ls -1 br-test-pkg/*/.config 2>/dev/null |xargs -r dirname ) ) + if [ "${#do_testpkg[@]}" -eq 0 ]; then + printf "Configuration fragment enables no test.\n" >&2; exit 1 + fi + fi + # If nothing else, at least do the basics to generate a valid pipeline if [ -z "${do_defconfigs}" \ -a -z "${do_runtime}" \ + -a -z "${do_testpkg}" \ ] then do_basics=true @@ -101,6 +122,10 @@ gen_tests() { if ${do_runtime:-false}; then printf '%s: { extends: .runtime_test_base }\n' "${runtimes[@]}" fi + + if [ -n "${do_testpkg}" ]; then + printf '%s: { extends: .test_pkg }\n' "${do_testpkg[@]}" + fi } main "${@}"