From patchwork Fri Apr 1 20:25:28 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Yann E. MORIN" X-Patchwork-Id: 605009 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from fraxinus.osuosl.org (smtp4.osuosl.org [140.211.166.137]) by ozlabs.org (Postfix) with ESMTP id 3qcCYw2Vmqz9sdm for ; Sat, 2 Apr 2016 07:25:40 +1100 (AEDT) Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b=ivSHVizp; dkim-atps=neutral Received: from localhost (localhost [127.0.0.1]) by fraxinus.osuosl.org (Postfix) with ESMTP id 9D775A638C; Fri, 1 Apr 2016 20:25:39 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from fraxinus.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id K-kC91uB0ZT1; Fri, 1 Apr 2016 20:25:37 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by fraxinus.osuosl.org (Postfix) with ESMTP id 27C6DA6375; Fri, 1 Apr 2016 20:25:37 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by ash.osuosl.org (Postfix) with ESMTP id 2AF511C0B41 for ; Fri, 1 Apr 2016 20:25:36 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id 271C095947 for ; Fri, 1 Apr 2016 20:25:36 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id EACAzJDo3cIy for ; Fri, 1 Apr 2016 20:25:35 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mail-wm0-f68.google.com (mail-wm0-f68.google.com [74.125.82.68]) by hemlock.osuosl.org (Postfix) with ESMTPS id 1F50C9593F for ; Fri, 1 Apr 2016 20:25:35 +0000 (UTC) Received: by mail-wm0-f68.google.com with SMTP id i204so965647wmd.0 for ; Fri, 01 Apr 2016 13:25:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references; bh=/1V3Klwbuq5ubjAM5C4kSEPC87jxhvE5UvMqgRTNXYY=; b=ivSHVizpwpUoVaJAWl1bFufxLwTvdo7c5kfnkK9prK7XLsQBZkbALcByL4CEXx9kDo k7sgurpGADRu+X7Tr9z/k66yTJmU18Xb5rxa995pAvl5LC8RodPXqmXc12jqhHpUnRfT rYZpiIpIXMb2+nOQILbKUpHc6GI23U3hTBqIHLNpki+R1TRFV4h6Ipxlm/SkeGqDByE1 kz8StyIL0hPZoGo1BgETye/WgPc5GcIr4lcdWN9muwRUNKqouvhpsdRKeIbrEiBQhjBk mVUh67RA0eikPRvKwfENmGKeqQteG9o5kncvKscuSp6UDxNwYlxYEAZiDKSII1NzIq5g ruKA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references; bh=/1V3Klwbuq5ubjAM5C4kSEPC87jxhvE5UvMqgRTNXYY=; b=dzHtNv/XVP7edSl1BZKTLVqZAq/krSZuKZaOZk8zUUFojJ4mtqgbakgJN2rz2iP7B+ N+r82ktubbqnSO5oabY+HHSzEPwHnUbFS2pTqvLvrS5gc5sW5L50wq0NMNvZMqMakejL 8sA0zYtb5pHdYfbO/lbqYvo2kcZTWPLeRq58pOkhT+WKLzXXh7zbSyuxA6AcroPDz5Yk odsebDmTRqyrGzAx7Njk+1asJFwdomYhTyQXiIkqy1JNbn0EqFKmFViE2NgeXZmlEBwA LBDU9/Ln+1oHvB6ECi8GSMCfRwB/FdvkPw9IZ4FMLYf9VEzwTgdrrkhU0JnKNchZuCtN Vh8g== X-Gm-Message-State: AD7BkJLbNHO34ELZXGe4gnF9UpsiiG3TW50eWLmUgcyLGU6czhJz80mcM+srDPZQdBsAZg== X-Received: by 10.28.4.195 with SMTP id 186mr498914wme.45.1459542333760; Fri, 01 Apr 2016 13:25:33 -0700 (PDT) Received: from gourin.bzh.lan (ns304657.ip-46-105-103.eu. [46.105.103.66]) by smtp.gmail.com with ESMTPSA id w203sm559814wmg.14.2016.04.01.13.25.32 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Fri, 01 Apr 2016 13:25:32 -0700 (PDT) From: "Yann E. MORIN" To: buildroot@buildroot.org Date: Fri, 1 Apr 2016 22:25:28 +0200 Message-Id: <46bd01446c4d29cc9420380986b1a498a8606375.1459541702.git.yann.morin.1998@free.fr> X-Mailer: git-send-email 1.9.1 In-Reply-To: References: Cc: "Yann E. MORIN" Subject: [Buildroot] [PATCH 1/4 v2] support/download/git: do not use bare clones X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: buildroot-bounces@busybox.net Sender: "buildroot" Currently, we are using bare clones, so as to minimise the disk usage, most notably for largeish repositories such as the one for the Linux kernel, which can go beyond the 1GiB barrier. However, this precludes updating (and thus using) the submodules, if any, of the repositories, as a working copy is required to use submodules (becaue we need to know the list of submodules, where to find them, where to clone them, what cset to checkout, and all those is dependent upon the checked out cset of the father repository). Switch to using /plain/ clones with a working copy. This means that the extra refs used by some forges (like pull-requests for Github, or changes for gerrit...) are no longer fetched as part of the clone, because git does not offer to do a mirror clone when there is a working copy. Instead, we have to fetch those special refs by hand. Since there is no easy solution to know whether the cset the user asked for is such a special ref or not, we just try to always fetch the cset requested by the user; if this fails, we assume that this is not a special ref (most probably, it is a sha1) and we defer the check to the archive creation, which would fail if the requested cset is missing anyway. Signed-off-by: "Yann E. MORIN" Tested-by: Matt Weber Reviewed-by: Matt Weber --- support/download/git | 21 ++++++++++++++++++--- 1 file changed, 18 insertions(+), 3 deletions(-) diff --git a/support/download/git b/support/download/git index 314b388..20b436e 100755 --- a/support/download/git +++ b/support/download/git @@ -41,7 +41,7 @@ _git() { git_done=0 if [ -n "$(_git ls-remote "'${repo}'" "'${cset}'" 2>&1)" ]; then printf "Doing shallow clone\n" - if _git clone ${verbose} --depth 1 -b "'${cset}'" --bare "'${repo}'" "'${basename}'"; then + if _git clone ${verbose} --depth 1 -b "'${cset}'" "'${repo}'" "'${basename}'"; then git_done=1 else printf "Shallow clone failed, falling back to doing a full clone\n" @@ -49,10 +49,25 @@ if [ -n "$(_git ls-remote "'${repo}'" "'${cset}'" 2>&1)" ]; then fi if [ ${git_done} -eq 0 ]; then printf "Doing full clone\n" - _git clone ${verbose} --mirror "'${repo}'" "'${basename}'" + _git clone ${verbose} "'${repo}'" "'${basename}'" +fi + +pushd "${basename}" >/dev/null + +# Try to get the special refs exposed by some forges (pull-requests for +# github, changes for gerrit...). There is no easy way to know whether +# the cset the user passed us is such a special ref or a tag or a sha1 +# or whatever else. We'll eventually fail at checking out that cset, +# below, if there is an issue anyway. Since most of the cset we're gonna +# have to clone are not such special refs, consign the output to oblivion +# so as not to alarm unsuspecting users, but still trace it as a warning. +if ! _git fetch "'${cset}:${cset}'" >/dev/null 2>&1; then + printf "Could not fetch special ref '%s'; assuming it is not special.\n" "${cset}" fi -GIT_DIR="${basename}" \ _git archive --prefix="'${basename}/'" -o "'${output}.tmp'" --format=tar "'${cset}'" +# Not really required, but here for consistency +popd >/dev/null + gzip -n <"${output}.tmp" >"${output}"