From patchwork Sun Apr 8 07:20:29 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ricardo Martincoski X-Patchwork-Id: 896012 Return-Path: X-Original-To: incoming-buildroot@patchwork.ozlabs.org Delivered-To: patchwork-incoming-buildroot@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=busybox.net (client-ip=140.211.166.137; helo=fraxinus.osuosl.org; envelope-from=buildroot-bounces@busybox.net; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="RjdFWyxt"; dkim-atps=neutral Received: from fraxinus.osuosl.org (smtp4.osuosl.org [140.211.166.137]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 40JlGm3kPfz9s1l for ; Sun, 8 Apr 2018 17:21:18 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by fraxinus.osuosl.org (Postfix) with ESMTP id B6C8A86ED7; Sun, 8 Apr 2018 07:21:14 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from fraxinus.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id P49ydSwJol8Z; Sun, 8 Apr 2018 07:21:12 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by fraxinus.osuosl.org (Postfix) with ESMTP id EE1BD86E54; Sun, 8 Apr 2018 07:21:11 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by ash.osuosl.org (Postfix) with ESMTP id 765F61C27B5 for ; Sun, 8 Apr 2018 07:21:10 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by whitealder.osuosl.org (Postfix) with ESMTP id 7293C88257 for ; Sun, 8 Apr 2018 07:21:10 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from whitealder.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id AnASxEzjI35R for ; Sun, 8 Apr 2018 07:21:09 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mail-qt0-f172.google.com (mail-qt0-f172.google.com [209.85.216.172]) by whitealder.osuosl.org (Postfix) with ESMTPS id EA2BD88241 for ; Sun, 8 Apr 2018 07:21:08 +0000 (UTC) Received: by mail-qt0-f172.google.com with SMTP id j26so5831944qtl.11 for ; Sun, 08 Apr 2018 00:21:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id; bh=+9MGHp75yGBKM8R6Th6vl/7VKOjzOjuRKso/15OAIl0=; b=RjdFWyxtRVniFtsC49BgTCEUWn3KBqMLvKyXrg+21D5CUvj64B2b+G0IwuNqCj+VEt t+e6musHeRLXN4w1xWh6AwT+j5G00j2qsoTLtMfzVlZaMoYW6ILNK9WiYWfUNo9JLaVu y+zjirp5Ahvx78ITYhXmZH6qTyT0IQ//fgqTZJGNG+w1KigCY+UmJK3IMZVfCFgmWkUJ 26DviG30yRMerA4bf5N5hbRa/bR2Ow62KBKgMpYiZQeen013uQyMRO9gpuB8YqFJXNTl CwiexNXsBKaW0IFY9CNl56VutgMsADwzKa7rp25NE45n6B8W3BooyBYiVzEQRZcfbHL4 849Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=+9MGHp75yGBKM8R6Th6vl/7VKOjzOjuRKso/15OAIl0=; b=ATTojnMC2D55QwT89srUskPbYbKPOJYYJ0YYgjbv0txWpiTHWi1Hh4ufm36UoKhCtp IPzYM0VwNq1cfGOSPI5FuF7LdvmqW+5M+loLKumQyneugPL8p0zReG1vTuZdGZdaAMVi LqZu4ypN3U6RcjWR6kibltKMH1AWJDbMg8pz7n2BUpmqKx9QKVPxvfEnG3jLPkLoCej4 fe+J1f6eXiZO+804KtGSms3fPljnxom03HS2qHtwfCpEKfyK6Rf2gdENxUwcIkM25KPe 5d55Z1z1LwNPN533tt3qpahC4ZISGmCzm8c622A9JW6dnIPvdQ4cSwD5Z0jCQGjBxdJc qQvg== X-Gm-Message-State: ALQs6tCebPh37+mhNZu3eKtDoaNN77jxbka10sd8hflgU+ewpsctpRu6 G982P2+9CFiXJe3I71ifGlFY3Q== X-Google-Smtp-Source: AIpwx4/6ayWJYml0uQSYQHmICbVOpcpYHKvZ8dJt54+aUXFEf4aIaX1mB684CPJo8UY7quu9HlHb3A== X-Received: by 10.237.54.65 with SMTP id e59mr47278699qtb.28.1523172066470; Sun, 08 Apr 2018 00:21:06 -0700 (PDT) Received: from ultri4.home ([187.113.201.194]) by smtp.gmail.com with ESMTPSA id v1sm4016042qtg.18.2018.04.08.00.21.02 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 08 Apr 2018 00:21:05 -0700 (PDT) From: Ricardo Martincoski To: buildroot@buildroot.org Date: Sun, 8 Apr 2018 04:20:29 -0300 Message-Id: <20180408072029.24888-1-ricardo.martincoski@gmail.com> X-Mailer: git-send-email 2.14.1 Subject: [Buildroot] [RFC PATCH] Test download of all git packages without backup site X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.24 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: "Yann E . MORIN" , Ricardo Martincoski MIME-Version: 1.0 Errors-To: buildroot-bounces@busybox.net Sender: "buildroot" To test a given sha1, checkout it in a clean tree (no .config file), cherry-pick this test, re-run ./make.gitlab-ci.yml and commit .gitlab-ci.yml before pushing it to Gitlab CI. The script is ugly, but it is not meant to be applied anyway. Signed-off-by: Ricardo Martincoski Cc: Yann E. MORIN --- NOTE: As soon as I send this patch I will change it to Not Applicable as it is not meant to be applied. This is still far from an exhaustive test but it seems a good subset: - all packages with site method = git - all boot, linux... that have custom site url in the intree defconfigs 2018.02: https://gitlab.com/RicardoMartincoski/buildroot/pipelines/20146633 master (d3a8177e): https://gitlab.com/RicardoMartincoski/buildroot/pipelines/20146667 master + patchwork 895967: https://gitlab.com/RicardoMartincoski/buildroot/pipelines/20146714 All tests that fail at master + patchwork 895967 also fail for 2018.02. 18xx-ti-utils and linux-firmware fail on master but are ok on master + patchwork 895967. Inpecting the tarball I could see 18xx-ti-utils also has a symlink to a destination starting with ../ Some tests fail in both master and 2018.02: - at91bootstrap3 v3.8.6: hash mismatch - gummiboot: not found, moved to https://anongit.freedesktop.org/git/gummiboot.git ? - kvm-unit-tests: not found - kvmtool: not found - mcelog: not found - trace-cmd: not found - uemacs: not found These 5 use https://cdn.kernel.org/pub/scm/ which is right now empty, maybe a temporary issue? This alternative url works: https://git.kernel.org/pub/scm/linux/kernel/git/will/kvmtool.git - xloader: unable to connect to gitorious.org --- make.gitlab-ci.yml | 71 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 71 insertions(+) create mode 100755 make.gitlab-ci.yml diff --git a/make.gitlab-ci.yml b/make.gitlab-ci.yml new file mode 100755 index 0000000000..3719bbc469 --- /dev/null +++ b/make.gitlab-ci.yml @@ -0,0 +1,71 @@ +#!/bin/bash +ONLY_ONE=0 +if [ "$1" == "1" ]; then + ONLY_ONE=1 +fi +FILE=".gitlab-ci.yml" + +echo "# $0" > ${FILE} + +grep 'image:' .gitlab-ci.yml.in >> ${FILE} + +( + echo + echo '.download: &download' + echo ' script:' + echo ' - make defconfig' + echo ' - ./utils/config --set-str BR2_BACKUP_SITE ""' + # shellcheck disable=SC2016 + echo ' - make ${CI_JOB_NAME}' + echo +) >> ${FILE} + +# HACKS: fix package names with '_', remove the make target for the target version for host-only packages +packages="$(make BR2_HAVE_DOT_CONFIG=y -s printvars VARS=%_SITE_METHOD \ + | grep '=git' \ + | sed -e 's/_SITE_METHOD=git$//g' \ + | tr "A-Z_" "a-z-" \ + | sed -e 's/xdriver-xf86/xdriver_xf86/g' \ + | grep -v '^netsurf-buildsystem' \ + | grep -v '^opkg-utils' \ + | grep -v '^pseudo' \ + | grep -v '^vboot-utils')" + +for package in ${packages}; do + echo "${package}-source: *download" + if [ "${ONLY_ONE}" -eq 1 ]; then + break + fi +done >> $FILE + +targets="$(grep -h CUSTOM_REPO_URL configs/* \ + | sed -e 's/^BR2_TARGET_//g' -e 's/^BR2_//g' -e 's/_CUSTOM_REPO_URL=.*//g' \ + | sort -u)" + +function print_script { target="$1" + ( + echo + echo ".defconfig-${target}: &defconfig-download-${target}" + echo ' script:' + # shellcheck disable=SC2016 + echo ' - make ${CI_JOB_NAME%-'"${target}-source}" + echo ' - ./utils/config --set-str BR2_BACKUP_SITE ""' + echo " - make ${target}-source" + echo + ) >> ${FILE} +} + +for target_upcase in ${targets}; do + target="$(echo "${target_upcase}" | sed -e 's/LINUX_KERNEL/LINUX/g' | tr "A-Z_" "a-z-")" + print_script "${target}" + + defconfigs="$(grep -l "${target_upcase}_CUSTOM_REPO_URL" configs/* \ + | sed -e 's,^configs/,,g')" + + for defconfig in ${defconfigs}; do + echo "${defconfig}-${target}-source: *defconfig-download-${target}" + if [ "${ONLY_ONE}" -eq 1 ]; then + break 1 + fi + done >> ${FILE} +done