From patchwork Tue May 1 08:44:12 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Yann E. MORIN" X-Patchwork-Id: 906979 Return-Path: X-Original-To: incoming-buildroot@patchwork.ozlabs.org Delivered-To: patchwork-incoming-buildroot@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=busybox.net (client-ip=140.211.166.138; helo=whitealder.osuosl.org; envelope-from=buildroot-bounces@busybox.net; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=free.fr Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="kOx9JE7m"; dkim-atps=neutral Received: from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 40Zw2407Vqz9s0W for ; Tue, 1 May 2018 18:44:27 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by whitealder.osuosl.org (Postfix) with ESMTP id 4FBAB8758B; Tue, 1 May 2018 08:44:26 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from whitealder.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id f2NDhmTikTMf; Tue, 1 May 2018 08:44:25 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by whitealder.osuosl.org (Postfix) with ESMTP id 414BC8689B; Tue, 1 May 2018 08:44:25 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from silver.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by ash.osuosl.org (Postfix) with ESMTP id CE8081C0DEF for ; Tue, 1 May 2018 08:44:21 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by silver.osuosl.org (Postfix) with ESMTP id CC2AA22158 for ; Tue, 1 May 2018 08:44:21 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from silver.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id o+d85sXlcHxO for ; Tue, 1 May 2018 08:44:21 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mail-wm0-f65.google.com (mail-wm0-f65.google.com [74.125.82.65]) by silver.osuosl.org (Postfix) with ESMTPS id 2571522052 for ; Tue, 1 May 2018 08:44:21 +0000 (UTC) Received: by mail-wm0-f65.google.com with SMTP id m70so16477347wma.2 for ; Tue, 01 May 2018 01:44:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references; bh=aqz9PS+qv5gTfwoxPE6wbJodbXQn+MG47H29k6zWVkI=; b=kOx9JE7mEnI5QrcX56DtA8yGkV76oA8HvgSH2kl0Jn+BQimuPkkVfITw0sAlCgVa91 xAUHvDQmBYumtIm1PTzoCYHkeqT5qvMclejAI/eT/qaq/SemOZxXrifXFrgqeEX3qctO gvSQEL+BypS9l6XVyeza1vbNW0C7Xz4Cbj18v3EusEkC6kVFyXRSACpgTzeTXY3JXctS bwd8er9HQhcs/GqN+qU6yiMppgHOhVHc4rZVFVl/H/D+IHVadTWQVXCmEso2VIOighxm izJ+GgzVfXblL0ubMkps/fzfwxzSWhoI3Rh6Ycx6BhAcP9mKlemYQJluQGnedBvH9k0C D4MQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references; bh=aqz9PS+qv5gTfwoxPE6wbJodbXQn+MG47H29k6zWVkI=; b=Z0Lb+BXMW/gMUlyIZGjoIJtg1Lqu8ezQOVRwWh2EC1RJNk/iKmzbbqIIqLwyvkGONk LDsKMcimtrELn2tZwsN4/S44uVN8/KZDTtHneWMTAx9IURXmYPsBPzgHEMuYRtpJGSWJ 8bXtSqiZ5BuLlF7DD9nOKpe/IY7Lk10C4KeC1TxuDDAeCUHYP+ut7K61Sh73UXdwYf8S RP3KOw+aIlY01ymqHIwhFuuY1zwhvbZSW3XO/umy5dOt9aLDgWRzb28wWQ5ldaRgofM5 zCe8SVHe4jOHqAqaAcMFcLPosXtVs69aPDwV+d5p0pEW/I9LRIj3BuwUFGM+CcnSObbh IBFw== X-Gm-Message-State: ALQs6tCqMKcgFu/q0id1rtJ2mWdHLFBzBVKEEIpLG/Z6XQT7pB6KvNNb NqKaoOPQD0eH3kRS/vznysqQYg== X-Google-Smtp-Source: AB8JxZqpZPNt4JnvnyXfNSoEcHWA3VAyMgWgp0yZ+pQ5eQ/gP74aWFYtd3GsFysA1aC95QHFpYrr4Q== X-Received: by 10.28.126.141 with SMTP id z135mr8952646wmc.6.1525164259472; Tue, 01 May 2018 01:44:19 -0700 (PDT) Received: from scaer.bzh.lan (2a01cb088610730058f24ca3e844065b.ipv6.abo.wanadoo.fr. [2a01:cb08:8610:7300:58f2:4ca3:e844:65b]) by smtp.gmail.com with ESMTPSA id t189sm11019720wmf.22.2018.05.01.01.44.18 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 01 May 2018 01:44:18 -0700 (PDT) From: "Yann E. MORIN" To: buildroot@buildroot.org Date: Tue, 1 May 2018 10:44:12 +0200 Message-Id: X-Mailer: git-send-email 2.14.1 In-Reply-To: References: Subject: [Buildroot] [PATCH 4/7 v5] download/git: try to recover from utterly-broken repositories X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.24 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Ricardo Martincoski , "Yann E. MORIN" , Thomas Petazzoni , Maxime Hadjinlian MIME-Version: 1.0 Errors-To: buildroot-bounces@busybox.net Sender: "buildroot" In some cases, the repository may be in a state we can't automatically recover from, especially since we must still support oldish git versions that do not provide the necessary commands or options thereof. As a last-ditch recovery, delete the repository and recreate the cache from scratch. Signed-off-by: "Yann E. MORIN" Cc: Maxime Hadjinlian Cc: Thomas Petazzoni Cc: Ricardo Martincoski Cc: Arnout Vandecappelle --- support/download/git | 37 ++++++++++++++++++++++++++++++++++--- 1 file changed, 34 insertions(+), 3 deletions(-) diff --git a/support/download/git b/support/download/git index bd37a0a8d9..39e57aac34 100755 --- a/support/download/git +++ b/support/download/git @@ -1,7 +1,7 @@ #!/usr/bin/env bash # We want to catch any unexpected failure, and exit immediately -set -e +set -E # Download helper for git, to be called from the download wrapper script # @@ -16,6 +16,33 @@ set -e # Environment: # GIT : the git command to call +# Save our path and options in case we need to call ourselves again +myname="${0}" +declare -a OPTS=("${@}") + +# This function is called when an error occurs. Its job is to attempt a +# clone from scratch (only once!) in case the git tree is borked, or in +# case an unexpected and unsupported situation arises with submodules +# or uncommitted stuff (e.g. if the user manually mucked around in the +# git cache). +_on_error() { + local ret=${?} + + printf "Detected a corrupted git cache.\n" >&2 + if ${BR_GIT_BACKEND_FIRST_FAULT:-false}; then + printf "This is the second time in a row; bailing out\n" >&2 + exit ${ret} + fi + export BR_GIT_BACKEND_FIRST_FAULT=true + + printf "Removing it and starting afresh.\n" >&2 + + popd >/dev/null + rm -rf "${git_cache}" + + exec "${myname}" "${OPTS[@]}" || exit ${ret} +} + verbose= recurse=0 while getopts "${BR_BACKEND_DL_GETOPTS}" OPT; do @@ -39,6 +66,9 @@ git_cache="${dl_dir}/git" mkdir -p "${git_cache}" pushd "${git_cache}" >/dev/null +# Any error now should try to recover +trap _on_error ERR + # Caller needs to single-quote its arguments to prevent them from # being expanded a second time (in case there are spaces in them) _git() { @@ -114,8 +144,9 @@ if ! _git fetch origin "'${cset}:${cset}'" >/dev/null 2>&1; then printf "Could not fetch special ref '%s'; assuming it is not special.\n" "${cset}" fi -# Check that the changeset does exist. If it does not, no reason to go -# on, we can fast-track to the exit path. +# Check that the changeset does exist. If it does not, re-cloning from +# scratch won't help, so we don't want to trash the repository for a +# missing commit. We just exit without going through the ERR trap. if ! _git rev-parse --quiet --verify "'${cset}^{commit}'" >/dev/null 2>&1; then printf "Commit '%s' does not exist in this repository\n." "${cset}" exit 1