From patchwork Thu Nov 19 10:36:03 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?SsOpcsO0bWUgUG91aWxsZXI=?= X-Patchwork-Id: 546382 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by ozlabs.org (Postfix) with ESMTP id 2F6F214149B for ; Thu, 19 Nov 2015 21:36:39 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id 4A01E87A52; Thu, 19 Nov 2015 10:36:33 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id F0xlHZc9BfHO; Thu, 19 Nov 2015 10:36:32 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by hemlock.osuosl.org (Postfix) with ESMTP id 753F594BA1; Thu, 19 Nov 2015 10:36:32 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by ash.osuosl.org (Postfix) with ESMTP id D02DF1CF337 for ; Thu, 19 Nov 2015 10:36:24 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by whitealder.osuosl.org (Postfix) with ESMTP id CCEA28BF5A for ; Thu, 19 Nov 2015 10:36:24 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from whitealder.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id gNX3Pz5VqCcM for ; Thu, 19 Nov 2015 10:36:23 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.7.6 Received: from lupi.sysmic.org (sysmic.org [62.210.89.17]) by whitealder.osuosl.org (Postfix) with ESMTPS id 6692486BDF for ; Thu, 19 Nov 2015 10:36:23 +0000 (UTC) Received: from localhost.localdomain (unknown [212.234.226.241]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA256 (128/128 bits)) (No client certificate requested) (Authenticated sender: jezz) by lupi.sysmic.org (Postfix) with ESMTPSA id 0164542F48; Thu, 19 Nov 2015 11:36:20 +0100 (CET) From: =?UTF-8?q?J=C3=A9r=C3=B4me=20Pouiller?= To: buildroot@busybox.net Date: Thu, 19 Nov 2015 11:36:03 +0100 Message-Id: <1447929366-8972-3-git-send-email-jezz@sysmic.org> X-Mailer: git-send-email 2.1.4 In-Reply-To: <1447929366-8972-1-git-send-email-jezz@sysmic.org> References: <1447929366-8972-1-git-send-email-jezz@sysmic.org> MIME-Version: 1.0 Cc: =?UTF-8?q?J=C3=A9r=C3=B4me=20Pouiller?= Subject: [Buildroot] [PATCH 2/5] download/git: allow to create archives containing shallowed git repos X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: buildroot-bounces@busybox.net Sender: "buildroot" We introduce a new tarball format for projects managed with git. This new format is a shallowed version of upstream repository. This format is a little larger (20 to 50% in my tests) than standard one and contains information about upstream git repository. That simplify workflow with upstream. It is now possible to reconnect a build directory with upstream repository using: git fetch --unshallow If you want to also get remote tags and branch (as in a standard clone), run: git config remote.origin.fetch refs/heads/*:refs/remotes/origin/* git fetch For large repository, you can use a local repository as reference in order to speedup unshallow process (read warnings about '--reference' option in git-clone manual page): echo "repo_path/.git/objects" > .git/objects/info/alternates git fetch --unshallow Note 1: some repository does not support unshallowing (especially anonscm.debian.org) Note 2: I have tested this patch with git 2.1 (from Debian Jessie). Signed-off-by: Jérôme Pouiller --- support/download/git | 30 +++++++++++++++++++++++++++--- 1 file changed, 27 insertions(+), 3 deletions(-) diff --git a/support/download/git b/support/download/git index 0e6103b..6755f65 100755 --- a/support/download/git +++ b/support/download/git @@ -46,7 +46,31 @@ if [ ${git_done} -eq 0 ]; then ${GIT} clone ${verbose} --bare "${repo}" "${basename}" fi -GIT_DIR="${basename}" \ -${GIT} archive --prefix="${basename}/" -o "${output}.tmp" --format=tar "${cset}" +if [[ ${output} != *.git.tar.gz* ]]; then + GIT_DIR="${basename}" \ + ${GIT} archive --prefix="${basename}/" -o "${output}.tmp" --format=tar "${cset}" -gzip <"${output}.tmp" >"${output}" + gzip <"${output}.tmp" >"${output}" +fi +if [[ ${output} = *.git.tar.gz* ]]; then + GIT_DIR="${basename}" ${GIT} reset --soft ${cset} + + # Git cannot shallow a repository on an arbitrary commit-id. It must be a + # symbolic reference of a remote repository (HEAD, tag or branch) (problem + # seems to be resolved in git 2.5). + # However, we want to be able to shallow on any commit-id. + # We didn't find any other way than use an intermediate repository in order + # to work around this limitation. + # (In add, this method make easier pruning of packed objects) + # + # Note 1: --depth is ignored in local clones. Git suggest to use file:// + # in order to force copy of objects. We have to use an absolute path with + # these URIs + # + # Note 2: ${basename}.git will be removed by dl-wrapper (since it is in + # same directory than ${basename}) + ${GIT} clone "file://$(readlink -f ${basename})" "${basename}.git" --bare --depth 1 + GIT_DIR="${basename}.git" ${GIT} remote set-url origin "${repo}" + + tar czf "${output}" "${basename}.git" +fi