From patchwork Mon Apr 2 08:14:22 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxime Hadjinlian X-Patchwork-Id: 894117 Return-Path: X-Original-To: incoming-buildroot@patchwork.ozlabs.org Delivered-To: patchwork-incoming-buildroot@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=busybox.net (client-ip=140.211.166.133; helo=hemlock.osuosl.org; envelope-from=buildroot-bounces@busybox.net; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="nb6C3mK7"; dkim-atps=neutral Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 40F4lB4PzLz9s28 for ; Mon, 2 Apr 2018 18:14:46 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id E6E4D89EEB; Mon, 2 Apr 2018 08:14:43 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id FJW-csN7yUGW; Mon, 2 Apr 2018 08:14:42 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by hemlock.osuosl.org (Postfix) with ESMTP id 5620289F2A; Mon, 2 Apr 2018 08:14:42 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by ash.osuosl.org (Postfix) with ESMTP id 3178D1C0B66 for ; Mon, 2 Apr 2018 08:14:41 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by whitealder.osuosl.org (Postfix) with ESMTP id 2DD7988E16 for ; Mon, 2 Apr 2018 08:14:41 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from whitealder.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id pqWKountYbrv for ; Mon, 2 Apr 2018 08:14:39 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mail-wr0-f172.google.com (mail-wr0-f172.google.com [209.85.128.172]) by whitealder.osuosl.org (Postfix) with ESMTPS id 776A588DD4 for ; Mon, 2 Apr 2018 08:14:39 +0000 (UTC) Received: by mail-wr0-f172.google.com with SMTP id p53so12655111wrc.10 for ; Mon, 02 Apr 2018 01:14:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id; bh=5LO1BxfcdBdC1wxV3okTYdy33FbQSOe9XWeZbRI2MGQ=; b=nb6C3mK7Xd8s2+0VXgmRrq0GflT93E9Kw/dvYLSy+2azSsK6AKvOfx2i/UPfwJ6Fg1 OzepyA7Qdzp/hY02z6I4YqNaRg7fYTD9Yvydq/L5AcG2F3ZUA0GJAjIa0Ee+ZZbJYglW 3AmWwU654aQDv/1QAnd36n5YnGgn1z6SqtgAVI1ar/HkNrLGTopJx7N4FUPOJALlNSSY pSXFfwPnrBfJNuKCQFgK3R7oxdRcrJM6WZzoVz1bdYPC8nKuYQE8LyEnyVIYspb/GRo6 fEFtTMcPYi0q9/4EPt/K82srb35myU/XBXkrqu0XyFvXVN07Emqki6b1gMCXanOlet+5 yyag== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=5LO1BxfcdBdC1wxV3okTYdy33FbQSOe9XWeZbRI2MGQ=; b=rGt4LFe0zF3oxA8+Afc97l8wOZ/IM5ggRn7wEhPYMbyoeBRHWENOro8ycvHt1Qc4cJ 7w5Vp4hLGQOTWHhXg38GwJ/j1nH12wM7RzV5FyScO1B4eJk42byLmY+kKAmzzidkPn78 TbUOZKhxFJn6cyzr0gVvZRqF+IWhBhZdLwZnAwWoZxnWi6J6HdVF7ac0psgYKeA0FUzE nuy/ce//WLwbsvWhx/kAu9MIJwVOH6G3kQ6FfzaQHUd1egpmBd0xKu4+Ek5M8MIufr7S 4SfvRB7cWEHOnsP8KXiwkMX6j2DICuOUg/XwoACKCXxW7Q+xySD6k+CjMNMiqQ2R4jvA PCZA== X-Gm-Message-State: AElRT7HQOUEILwUNIg9aD8BgeYmcXEO3MxDfrPvPOwFW3IX3ifNdjD2T fdv/p0EL7Aipbt2dTRGTxpXFKWfZ X-Google-Smtp-Source: AIpwx499dvtHo1BeDFu/kzsdF1QE3jc5xQkzEjYEsJs0FQoooJ5sBVrEwiveL5jZKiDgasxtK17DVQ== X-Received: by 10.223.182.16 with SMTP id f16mr6195472wre.23.1522656876990; Mon, 02 Apr 2018 01:14:36 -0700 (PDT) Received: from cerise.online.net ([195.154.228.195]) by smtp.gmail.com with ESMTPSA id a205sm51981wmf.18.2018.04.02.01.14.36 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 02 Apr 2018 01:14:36 -0700 (PDT) From: Maxime Hadjinlian To: buildroot@buildroot.org Date: Mon, 2 Apr 2018 10:14:22 +0200 Message-Id: <20180402081434.4411-1-maxime.hadjinlian@gmail.com> X-Mailer: git-send-email 2.16.2 Subject: [Buildroot] [v4 01/13] core/pkg-download: change all helpers to use common options X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.24 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Thomas Petazzoni , "Yann E. MORIN" , Maxime Hadjinlian MIME-Version: 1.0 Errors-To: buildroot-bounces@busybox.net Sender: "buildroot" From: "Yann E. MORIN" Currently all download helpers accepts the local output file, the remote locations, the changesets and so on... as positional arguments. This was well and nice when that's was all we needed. But then we added an option to quiesce their verbosity, and that was shoehorned with a trivial getopts, still keeping all the existing positional arguments as... positional arguments. Adding yet more options while keeping positional arguments will not be very easy, even if we do not envision any new option in the foreseeable future (but 640K ought to be enough for everyone, remember? ;-) ). Change all helpers to accept a set of generic options (-q for quiet and -o for the output file) as well as helper-specific options (like -r for the repository, -c for a changeset...). Maxime: Changed -R to -r for recurse (only for the git backend) Changed -r to -u for URI (for all backend) Change -R to -c for cset (for CVS and SVN backend) Add the export of the BR_BACKEND_DL_GETOPTS so all the backend wrapper can use the same option easily Now all the backends use the same common options. Signed-off-by: "Yann E. MORIN" Signed-off-by: Maxime Hadjinlian Cc: Thomas Petazzoni Reviewed-by: "Yann E. MORIN" Reviewed-by: Thomas Petazzoni --- v1 -> v2: - Fix comment in bzr backend - Remove 'R' from BR_BACKEND_DL_GETOPTS - check-hash shouldn't have had its options changed since it's not a download backend (Arnout) --- package/pkg-download.mk | 38 +++++++++++++++++++------------------- support/download/bzr | 25 ++++++++++++++----------- support/download/cp | 17 +++++++++-------- support/download/cvs | 34 +++++++++++++++++++--------------- support/download/dl-wrapper | 7 ++++++- support/download/git | 33 +++++++++++++++++---------------- support/download/hg | 25 ++++++++++++++----------- support/download/scp | 19 ++++++++++--------- support/download/svn | 25 ++++++++++++++----------- support/download/wget | 17 +++++++++-------- 10 files changed, 131 insertions(+), 109 deletions(-) diff --git a/package/pkg-download.mk b/package/pkg-download.mk index 6f15deba28..a410dce1ee 100644 --- a/package/pkg-download.mk +++ b/package/pkg-download.mk @@ -77,9 +77,9 @@ define DOWNLOAD_GIT -H $(PKGDIR)/$($(PKG)_RAWNAME).hash \ $(QUIET) \ -- \ - $($(PKG)_SITE) \ - $($(PKG)_DL_VERSION) \ - $($(PKG)_BASENAME_RAW) \ + -u $($(PKG)_SITE) \ + -c $($(PKG)_DL_VERSION) \ + -n $($(PKG)_BASENAME_RAW) \ $($(PKG)_DL_OPTS) endef @@ -88,9 +88,9 @@ define DOWNLOAD_BZR -o $(DL_DIR)/$($(PKG)_SOURCE) \ $(QUIET) \ -- \ - $($(PKG)_SITE) \ - $($(PKG)_DL_VERSION) \ - $($(PKG)_BASENAME_RAW) \ + -u $($(PKG)_SITE) \ + -c $($(PKG)_DL_VERSION) \ + -n $($(PKG)_BASENAME_RAW) \ $($(PKG)_DL_OPTS) endef @@ -99,10 +99,10 @@ define DOWNLOAD_CVS -o $(DL_DIR)/$($(PKG)_SOURCE) \ $(QUIET) \ -- \ - $(call stripurischeme,$(call qstrip,$($(PKG)_SITE))) \ - $($(PKG)_DL_VERSION) \ - $($(PKG)_RAWNAME) \ - $($(PKG)_BASENAME_RAW) \ + -u $(call stripurischeme,$(call qstrip,$($(PKG)_SITE))) \ + -c $($(PKG)_DL_VERSION) \ + -N $($(PKG)_RAWNAME) \ + -n $($(PKG)_BASENAME_RAW) \ $($(PKG)_DL_OPTS) endef @@ -111,9 +111,9 @@ define DOWNLOAD_SVN -o $(DL_DIR)/$($(PKG)_SOURCE) \ $(QUIET) \ -- \ - $($(PKG)_SITE) \ - $($(PKG)_DL_VERSION) \ - $($(PKG)_BASENAME_RAW) \ + -u $($(PKG)_SITE) \ + -c $($(PKG)_DL_VERSION) \ + -n $($(PKG)_BASENAME_RAW) \ $($(PKG)_DL_OPTS) endef @@ -126,7 +126,7 @@ define DOWNLOAD_SCP -H $(PKGDIR)/$($(PKG)_RAWNAME).hash \ $(QUIET) \ -- \ - '$(call stripurischeme,$(call qstrip,$(1)))' \ + -u '$(call stripurischeme,$(call qstrip,$(1)))' \ $($(PKG)_DL_OPTS) endef @@ -135,9 +135,9 @@ define DOWNLOAD_HG -o $(DL_DIR)/$($(PKG)_SOURCE) \ $(QUIET) \ -- \ - $($(PKG)_SITE) \ - $($(PKG)_DL_VERSION) \ - $($(PKG)_BASENAME_RAW) \ + -u $($(PKG)_SITE) \ + -c $($(PKG)_DL_VERSION) \ + -n $($(PKG)_BASENAME_RAW) \ $($(PKG)_DL_OPTS) endef @@ -147,7 +147,7 @@ define DOWNLOAD_WGET -H $(PKGDIR)/$($(PKG)_RAWNAME).hash \ $(QUIET) \ -- \ - '$(call qstrip,$(1))' \ + -u '$(call qstrip,$(1))' \ $($(PKG)_DL_OPTS) endef @@ -157,7 +157,7 @@ define DOWNLOAD_LOCALFILES -H $(PKGDIR)/$($(PKG)_RAWNAME).hash \ $(QUIET) \ -- \ - $(call stripurischeme,$(call qstrip,$(1))) \ + -u $(call stripurischeme,$(call qstrip,$(1))) \ $($(PKG)_DL_OPTS) endef diff --git a/support/download/bzr b/support/download/bzr index 75b7b415c1..5289a421cd 100755 --- a/support/download/bzr +++ b/support/download/bzr @@ -5,28 +5,31 @@ set -e # Download helper for bzr, to be called from the download wrapper script # -# Call it as: -# .../bzr [-q] OUT_FILE REPO_URL REV BASENAME +# Options: +# -q Be quiet +# -o FILE Generate archive in FILE. +# -u URI Clone from repository at URI. +# -c CSET Use changeset (or revision) CSET. +# -n NAME Use basename NAME. # # Environment: # BZR : the bzr command to call verbose= -while getopts :q OPT; do +while getopts "${BR_BACKEND_DL_GETOPTS}" OPT; do case "${OPT}" in q) verbose=-q;; + o) output="${OPTARG}";; + u) uri="${OPTARG}";; + c) cset="${OPTARG}";; + n) basename="${OPTARG}";; + :) printf "option '%s' expects a mandatory argument\n" "${OPTARG}"; exit 1;; \?) printf "unknown option '%s'\n" "${OPTARG}" >&2; exit 1;; esac done -shift $((OPTIND-1)) -output="${1}" -repo="${2}" -rev="${3}" -basename="${4}" - -shift 4 # Get rid of our options +shift $((OPTIND-1)) # Get rid of our options # Caller needs to single-quote its arguments to prevent them from # being expanded a second time (in case there are spaces in them) @@ -51,5 +54,5 @@ if [ ${bzr_version} -ge ${bzr_min_version} ]; then fi _bzr export ${verbose} --root="'${basename}/'" --format=tgz \ - ${timestamp_opt} - "${@}" "'${repo}'" -r "'${rev}'" \ + ${timestamp_opt} - "${@}" "'${uri}'" -r "'${cset}'" \ >"${output}" diff --git a/support/download/cp b/support/download/cp index 0ee1f3ba82..52fe2de83d 100755 --- a/support/download/cp +++ b/support/download/cp @@ -5,8 +5,10 @@ set -e # Download helper for cp, to be called from the download wrapper script # -# Call it as: -# .../cp [-q] OUT_FILE SRC_FILE +# Options: +# -q Be quiet. +# -o FILE Copy to file FILE. +# -u FILE Copy from file FILE. # # Environment: # LOCALFILES: the cp command to call @@ -17,18 +19,17 @@ set -e # Make 'cp' verbose by default, so it behaves a bit like the others. verbose=-v -while getopts :q OPT; do +while getopts "${BR_BACKEND_DL_GETOPTS}" OPT; do case "${OPT}" in q) verbose=;; + o) output="${OPTARG}";; + u) source="${OPTARG}";; + :) printf "option '%s' expects a mandatory argument\n" "${OPTARG}"; exit 1;; \?) printf "unknown option '%s'\n" "${OPTARG}" >&2; exit 1;; esac done -shift $((OPTIND-1)) -output="${1}" -source="${2}" - -shift 2 # Get rid of our options +shift $((OPTIND-1)) # Get rid of our options # Caller needs to single-quote its arguments to prevent them from # being expanded a second time (in case there are spaces in them) diff --git a/support/download/cvs b/support/download/cvs index 50050ab1c9..69d5c71f28 100755 --- a/support/download/cvs +++ b/support/download/cvs @@ -5,28 +5,32 @@ set -e # Download helper for cvs, to be called from the download wrapper script # -# Call it as: -# .../cvs [-q] OUT_FILE CVS_URL REV PKG_NAME BASENAME +# Options: +# -q Be quiet +# -o FILE Generate archive in FILE. +# -u URI Checkout from repository at URI. +# -c REV Use revision REV. +# -N RAWNAME Use rawname (aka module name) RAWNAME. +# -n NAME Use basename NAME. # # Environment: # CVS : the cvs command to call verbose= -while getopts :q OPT; do +while getopts "${BR_BACKEND_DL_GETOPTS}" OPT; do case "${OPT}" in q) verbose=-Q;; + o) output="${OPTARG}";; + u) uri="${OPTARG}";; + c) rev="${OPTARG}";; + N) rawname="${OPTARG}";; + n) basename="${OPTARG}";; + :) printf "option '%s' expects a mandatory argument\n" "${OPTARG}"; exit 1;; \?) printf "unknown option '%s'\n" "${OPTARG}" >&2; exit 1;; esac done -shift $((OPTIND-1)) -output="${1}" -repo="${2}" -rev="${3}" -rawname="${4}" -basename="${5}" - -shift 5 # Get rid of our options +shift $((OPTIND-1)) # Get rid of our options # Caller needs to single-quote its arguments to prevent them from # being expanded a second time (in case there are spaces in them) @@ -42,14 +46,14 @@ else select="-r" fi -# The absence of an initial : on ${repo} means access method undefined -if [[ ! "${repo}" =~ ^: ]]; then +# The absence of an initial : on ${uri} means access method undefined +if [[ ! "${uri}" =~ ^: ]]; then # defaults to anonymous pserver - repo=":pserver:anonymous@${repo}" + uri=":pserver:anonymous@${uri}" fi export TZ=UTC -_cvs ${verbose} -z3 -d"'${repo}'" \ +_cvs ${verbose} -z3 -d"'${uri}'" \ co "${@}" -d "'${basename}'" ${select} "'${rev}'" -P "'${rawname}'" tar czf "${output}" "${basename}" diff --git a/support/download/dl-wrapper b/support/download/dl-wrapper index b018819833..abc51f637a 100755 --- a/support/download/dl-wrapper +++ b/support/download/dl-wrapper @@ -19,6 +19,8 @@ # We want to catch any unexpected failure, and exit immediately. set -e +export BR_BACKEND_DL_GETOPTS=":hc:o:n:N:H:ru:q" + main() { local OPT OPTARG local backend output hfile recurse quiet rc @@ -83,7 +85,10 @@ main() { # If the backend fails, we can just remove the temporary directory to # remove all the cruft it may have left behind. Then we just exit in # error too. - if ! "${OLDPWD}/support/download/${backend}" ${quiet} ${recurse} "${tmpf}" "${@}"; then + if ! "${OLDPWD}/support/download/${backend}" \ + ${quiet} ${recurse} \ + -o "${tmpf}" "${@}" + then rm -rf "${tmpd}" exit 1 fi diff --git a/support/download/git b/support/download/git index f590ff6494..58a2c6ad9d 100755 --- a/support/download/git +++ b/support/download/git @@ -5,32 +5,33 @@ set -e # Download helper for git, to be called from the download wrapper script # -# Call it as: -# .../git [-q] [-r] OUT_FILE REPO_URL CSET BASENAME -# -# -q Be quiet. -# -r Clone and archive sub-modules. +# Options: +# -q Be quiet. +# -r Clone and archive sub-modules. +# -o FILE Generate archive in FILE. +# -u URI Clone from repository at URI. +# -c CSET Use changeset CSET. +# -n NAME Use basename NAME. # # Environment: # GIT : the git command to call verbose= recurse=0 -while getopts :qr OPT; do +while getopts "${BR_BACKEND_DL_GETOPTS}" OPT; do case "${OPT}" in q) verbose=-q; exec >/dev/null;; r) recurse=1;; + o) output="${OPTARG}";; + u) uri="${OPTARG}";; + c) cset="${OPTARG}";; + n) basename="${OPTARG}";; + :) printf "option '%s' expects a mandatory argument\n" "${OPTARG}"; exit 1;; \?) printf "unknown option '%s'\n" "${OPTARG}" >&2; exit 1;; esac done -shift $((OPTIND-1)) -output="${1}" -repo="${2}" -cset="${3}" -basename="${4}" - -shift 4 # Get rid of our options +shift $((OPTIND-1)) # Get rid of our options # Caller needs to single-quote its arguments to prevent them from # being expanded a second time (in case there are spaces in them) @@ -46,9 +47,9 @@ _git() { # Messages for the type of clone used are provided to ease debugging in case of # problems git_done=0 -if [ -n "$(_git ls-remote "'${repo}'" "'${cset}'" 2>&1)" ]; then +if [ -n "$(_git ls-remote "'${uri}'" "'${cset}'" 2>&1)" ]; then printf "Doing shallow clone\n" - if _git clone ${verbose} "${@}" --depth 1 -b "'${cset}'" "'${repo}'" "'${basename}'"; then + if _git clone ${verbose} "${@}" --depth 1 -b "'${cset}'" "'${uri}'" "'${basename}'"; then git_done=1 else printf "Shallow clone failed, falling back to doing a full clone\n" @@ -56,7 +57,7 @@ if [ -n "$(_git ls-remote "'${repo}'" "'${cset}'" 2>&1)" ]; then fi if [ ${git_done} -eq 0 ]; then printf "Doing full clone\n" - _git clone ${verbose} "${@}" "'${repo}'" "'${basename}'" + _git clone ${verbose} "${@}" "'${uri}'" "'${basename}'" fi pushd "${basename}" >/dev/null diff --git a/support/download/hg b/support/download/hg index 3af01690b3..efb515fca5 100755 --- a/support/download/hg +++ b/support/download/hg @@ -5,27 +5,30 @@ set -e # Download helper for hg, to be called from the download wrapper script # -# Call it as: -# .../hg [-q] OUT_FILE REPO_URL CSET BASENAME +# Options: +# -q Be quiet. +# -o FILE Generate archive in FILE. +# -u URI Clone from repository at URI. +# -c CSET Use changeset (or revision) CSET. +# -n NAME Use basename NAME. # # Environment: # HG : the hg command to call verbose= -while getopts :q OPT; do +while getopts "${BR_BACKEND_DL_GETOPTS}" OPT; do case "${OPT}" in q) verbose=-q;; + o) output="${OPTARG}";; + u) uri="${OPTARG}";; + c) cset="${OPTARG}";; + n) basename="${OPTARG}";; + :) printf "option '%s' expects a mandatory argument\n" "${OPTARG}"; exit 1;; \?) printf "unknown option '%s'\n" "${OPTARG}" >&2; exit 1;; esac done -shift $((OPTIND-1)) -output="${1}" -repo="${2}" -cset="${3}" -basename="${4}" - -shift 4 # Get rid of our options +shift $((OPTIND-1)) # Get rid of our options # Caller needs to single-quote its arguments to prevent them from # being expanded a second time (in case there are spaces in them) @@ -33,7 +36,7 @@ _hg() { eval ${HG} "${@}" } -_hg clone ${verbose} "${@}" --noupdate "'${repo}'" "'${basename}'" +_hg clone ${verbose} "${@}" --noupdate "'${uri}'" "'${basename}'" _hg archive ${verbose} --repository "'${basename}'" --type tgz \ --prefix "'${basename}'" --rev "'${cset}'" \ diff --git a/support/download/scp b/support/download/scp index 825fd41c64..8ecf2f4b22 100755 --- a/support/download/scp +++ b/support/download/scp @@ -5,25 +5,26 @@ set -e # Download helper for scp, to be called from the download wrapper script # -# Call it as: -# .../scp [-q] OUT_FILE SRC_URL +# Options: +# -q Be quiet. +# -o FILE Copy to local file FILE. +# -u FILE Copy from remote file FILE. # # Environment: # SCP : the scp command to call verbose= -while getopts :q OPT; do +while getopts "${BR_BACKEND_DL_GETOPTS}" OPT; do case "${OPT}" in q) verbose=-q;; + o) output="${OPTARG}";; + u) uri="${OPTARG}";; + :) printf "option '%s' expects a mandatory argument\n" "${OPTARG}"; exit 1;; \?) printf "unknown option '%s'\n" "${OPTARG}" >&2; exit 1;; esac done -shift $((OPTIND-1)) -output="${1}" -url="${2}" - -shift 2 # Get rid of our options +shift $((OPTIND-1)) # Get rid of our options # Caller needs to single-quote its arguments to prevent them from # being expanded a second time (in case there are spaces in them) @@ -31,4 +32,4 @@ _scp() { eval ${SCP} "${@}" } -_scp ${verbose} "${@}" "'${url}'" "'${output}'" +_scp ${verbose} "${@}" "'${uri}'" "'${output}'" diff --git a/support/download/svn b/support/download/svn index 77abf3d02d..542b25c0a2 100755 --- a/support/download/svn +++ b/support/download/svn @@ -5,27 +5,30 @@ set -e # Download helper for svn, to be called from the download wrapper script # -# Call it as: -# .../svn [-q] OUT_FILE REPO_URL REV BASNAME +# Options: +# -q Be quiet. +# -o FILE Generate archive in FILE. +# -u URI Checkout from repository at URI. +# -c REV Use revision REV. +# -n NAME Use basename NAME. # # Environment: # SVN : the svn command to call verbose= -while getopts :q OPT; do +while getopts "${BR_BACKEND_DL_GETOPTS}" OPT; do case "${OPT}" in q) verbose=-q;; + o) output="${OPTARG}";; + u) uri="${OPTARG}";; + c) rev="${OPTARG}";; + n) basename="${OPTARG}";; + :) printf "option '%s' expects a mandatory argument\n" "${OPTARG}"; exit 1;; \?) printf "unknown option '%s'\n" "${OPTARG}" >&2; exit 1;; esac done -shift $((OPTIND-1)) -output="${1}" -repo="${2}" -rev="${3}" -basename="${4}" - -shift 4 # Get rid of our options +shift $((OPTIND-1)) # Get rid of our options # Caller needs to single-quote its arguments to prevent them from # being expanded a second time (in case there are spaces in them) @@ -33,6 +36,6 @@ _svn() { eval ${SVN} "${@}" } -_svn export ${verbose} "${@}" "'${repo}@${rev}'" "'${basename}'" +_svn export ${verbose} "${@}" "'${uri}@${rev}'" "'${basename}'" tar czf "${output}" "${basename}" diff --git a/support/download/wget b/support/download/wget index 768de904c3..fece6663ca 100755 --- a/support/download/wget +++ b/support/download/wget @@ -5,25 +5,26 @@ set -e # Download helper for wget, to be called from the download wrapper script # -# Call it as: -# .../wget [-q] OUT_FILE URL +# Options: +# -q Be quiet. +# -o FILE Save into file FILE. +# -u URL Download file at URL. # # Environment: # WGET : the wget command to call verbose= -while getopts :q OPT; do +while getopts "${BR_BACKEND_DL_GETOPTS}" OPT; do case "${OPT}" in q) verbose=-q;; + o) output="${OPTARG}";; + u) url="${OPTARG}";; + :) printf "option '%s' expects a mandatory argument\n" "${OPTARG}"; exit 1;; \?) printf "unknown option '%s'\n" "${OPTARG}" >&2; exit 1;; esac done -shift $((OPTIND-1)) -output="${1}" -url="${2}" - -shift 2 # Get rid of our options +shift $((OPTIND-1)) # Get rid of our options # Caller needs to single-quote its arguments to prevent them from # being expanded a second time (in case there are spaces in them) From patchwork Mon Apr 2 08:14:23 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxime Hadjinlian X-Patchwork-Id: 894116 Return-Path: X-Original-To: incoming-buildroot@patchwork.ozlabs.org Delivered-To: patchwork-incoming-buildroot@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=busybox.net (client-ip=140.211.166.137; helo=fraxinus.osuosl.org; envelope-from=buildroot-bounces@busybox.net; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="eXNCzhDA"; dkim-atps=neutral Received: from fraxinus.osuosl.org (smtp4.osuosl.org [140.211.166.137]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 40F4lB4Q7qz9s29 for ; Mon, 2 Apr 2018 18:14:46 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by fraxinus.osuosl.org (Postfix) with ESMTP id 095DB87C49; Mon, 2 Apr 2018 08:14:45 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from fraxinus.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Qu_Fobdrj_xF; Mon, 2 Apr 2018 08:14:43 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by fraxinus.osuosl.org (Postfix) with ESMTP id 077C787F0E; Mon, 2 Apr 2018 08:14:43 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by ash.osuosl.org (Postfix) with ESMTP id 9967D1C0B66 for ; Mon, 2 Apr 2018 08:14:41 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by whitealder.osuosl.org (Postfix) with ESMTP id 9643F88E6A for ; Mon, 2 Apr 2018 08:14:41 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from whitealder.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id jc2G3Os3LYtY for ; Mon, 2 Apr 2018 08:14:40 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mail-wr0-f179.google.com (mail-wr0-f179.google.com [209.85.128.179]) by whitealder.osuosl.org (Postfix) with ESMTPS id EE79488DF1 for ; Mon, 2 Apr 2018 08:14:39 +0000 (UTC) Received: by mail-wr0-f179.google.com with SMTP id y55so12661519wry.3 for ; Mon, 02 Apr 2018 01:14:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=yHPyBPk9Mc94PFkJrSiKw5IfoG8yE5TPG8d4UhgM2xE=; b=eXNCzhDAF4+rEqDtW6BtAGchmioRs2z/3Sn3wmJg1k4Vy1pLAUuk0WaPuoRxKQ4nhb bgtdx25xtJlYTlYU3PURdcZG3Yo9dL3kzBc9YgVOLtQcWZznuGTtzvhZ79X13aenn7LX mwkcSvBPztwJRUtAaX82yasx9hHe0HGG+M2GcP1zoAOFEmeLHB6WTjHtAIF5P3oF6Pn0 lcqi8gzs6IPK9v4gC8qkGnt6uMKbF2Xt/sJpwehSBXIN8WVbWzvH4NGXUm+7tGBrqv80 /1mVq4jwvzKgNu6i0lNn4K52U9zqZVpUD/vKXskCW4uCwOt1qIuGfYOcNbCkHY9N8Sth ziSA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=yHPyBPk9Mc94PFkJrSiKw5IfoG8yE5TPG8d4UhgM2xE=; b=W60dOe9zhyrdQPC/VbAjXEqF1gYwDyiz45etd2zkHel9hrJ2eCn5Be1oIcZ4inqUPw VYtnBonZuUKQlnyGAfSQwCFb7aO4/tAFJKLNzmuggEreWYLyzgUK8oE2mP9YJAHwCfZo WJmeWS/CuGD/7oA6ufRaGbYcUdc6ozbuQ0CoLECs0iZv4MxseDjZl5Wp7zpYD0V5GFlm L2CrlK4oYIQ++XrQBmO+N+naLgqXP0hpTvx6AKtylR1tXNdxFzfftNDwG5naW9zcduq5 NjFsZZ80EH8Mn+g+8z9gvB0qzB9HhL8GkfoX9RtPhhy93hDT5Bhwln+vaJlu5Rs611R7 Zg1g== X-Gm-Message-State: AElRT7G+fZZMUvGBCL6GrJ26c6Vz24QU+dERGzUkDc1pkHJX3giM1WWz TrN5PIWBsAnshqInds7EvHXH1UZH X-Google-Smtp-Source: AIpwx4/CkgToYsLzB589NoPxg7KHlFSDsKOc/vGsbf8J+3t6debdXAzJrwzZdoPcwAREMP5f5zUrGA== X-Received: by 10.223.220.70 with SMTP id m6mr5810863wrj.244.1522656877827; Mon, 02 Apr 2018 01:14:37 -0700 (PDT) Received: from cerise.online.net ([195.154.228.195]) by smtp.gmail.com with ESMTPSA id a205sm51981wmf.18.2018.04.02.01.14.37 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 02 Apr 2018 01:14:37 -0700 (PDT) From: Maxime Hadjinlian To: buildroot@buildroot.org Date: Mon, 2 Apr 2018 10:14:23 +0200 Message-Id: <20180402081434.4411-2-maxime.hadjinlian@gmail.com> X-Mailer: git-send-email 2.16.2 In-Reply-To: <20180402081434.4411-1-maxime.hadjinlian@gmail.com> References: <20180402081434.4411-1-maxime.hadjinlian@gmail.com> Subject: [Buildroot] [v4 02/13] download: put most of the infra in dl-wrapper X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.24 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Maxime Hadjinlian MIME-Version: 1.0 Errors-To: buildroot-bounces@busybox.net Sender: "buildroot" The goal here is to simplify the infrastructure by putting most of the code in the dl-wrapper as it's easier to implement and to read. Most of the functions were common already, this patch finalizes it by making the pkg-download.mk pass all the parameters needed to the dl-wrapper which in turns will pass everything to every backend. The backend will then cherry-pick what it needs from these arguments and act accordingly. It eases the transition to the addition of a sub directory per package in the DL_DIR, and later on, a git cache. Signed-off-by: Maxime Hadjinlian Tested-by: Luca Ceresoli Reviewed-by: Luca Ceresoli Reviewed-by: "Yann E. MORIN" --- v1 -> v2: - Rename cp backend to file (Arnout) - Don't use BR_BACKEND_DL_GETOPTS for dl-wrapper (Arnout) - Add "urlencode" to scheme passed to the dl-wrapper to support the fact that we need to urlencode the filename when using PRIMARY and BACKUP mirror (some files are named toto.c?v=1.0) (Arnout) - Fix uristripscheme replaced by bash'ism (Arnout) - Add the check hash into the loop, exit with error only if all the download+check failed. (Arnout) --- package/pkg-download.mk | 170 ++++++++------------------------------------ support/download/cvs | 2 +- support/download/dl-wrapper | 139 +++++++++++++++++++++++------------- support/download/wget | 10 ++- 4 files changed, 129 insertions(+), 192 deletions(-) diff --git a/package/pkg-download.mk b/package/pkg-download.mk index a410dce1ee..2ee745fddd 100644 --- a/package/pkg-download.mk +++ b/package/pkg-download.mk @@ -42,6 +42,8 @@ DL_DIR := $(shell mkdir -p $(DL_DIR) && cd $(DL_DIR) >/dev/null && pwd) # # geturischeme: http geturischeme = $(firstword $(subst ://, ,$(call qstrip,$(1)))) +# getschemeplusuri: git|parameter+http://example.com +getschemeplusuri = $(call geturischeme,$(1))$(if $(2),\|$(2))+$(1) # stripurischeme: www.example.com/dir/file stripurischeme = $(lastword $(subst ://, ,$(call qstrip,$(1)))) # domain: www.example.com @@ -61,152 +63,42 @@ github = https://github.com/$(1)/$(2)/archive/$(3) export BR_NO_CHECK_HASH_FOR = ################################################################################ -# The DOWNLOAD_* helpers are in charge of getting a working copy -# of the source repository for their corresponding SCM, -# checking out the requested version / commit / tag, and create an -# archive out of it. DOWNLOAD_SCP uses scp to obtain a remote file with -# ssh authentication. DOWNLOAD_WGET is the normal wget-based download -# mechanism. -# -################################################################################ - -define DOWNLOAD_GIT - $(EXTRA_ENV) $(DL_WRAPPER) -b git \ - -o $(DL_DIR)/$($(PKG)_SOURCE) \ - $(if $($(PKG)_GIT_SUBMODULES),-r) \ - -H $(PKGDIR)/$($(PKG)_RAWNAME).hash \ - $(QUIET) \ - -- \ - -u $($(PKG)_SITE) \ - -c $($(PKG)_DL_VERSION) \ - -n $($(PKG)_BASENAME_RAW) \ - $($(PKG)_DL_OPTS) -endef - -define DOWNLOAD_BZR - $(EXTRA_ENV) $(DL_WRAPPER) -b bzr \ - -o $(DL_DIR)/$($(PKG)_SOURCE) \ - $(QUIET) \ - -- \ - -u $($(PKG)_SITE) \ - -c $($(PKG)_DL_VERSION) \ - -n $($(PKG)_BASENAME_RAW) \ - $($(PKG)_DL_OPTS) -endef - -define DOWNLOAD_CVS - $(EXTRA_ENV) $(DL_WRAPPER) -b cvs \ - -o $(DL_DIR)/$($(PKG)_SOURCE) \ - $(QUIET) \ - -- \ - -u $(call stripurischeme,$(call qstrip,$($(PKG)_SITE))) \ - -c $($(PKG)_DL_VERSION) \ - -N $($(PKG)_RAWNAME) \ - -n $($(PKG)_BASENAME_RAW) \ - $($(PKG)_DL_OPTS) -endef - -define DOWNLOAD_SVN - $(EXTRA_ENV) $(DL_WRAPPER) -b svn \ - -o $(DL_DIR)/$($(PKG)_SOURCE) \ - $(QUIET) \ - -- \ - -u $($(PKG)_SITE) \ - -c $($(PKG)_DL_VERSION) \ - -n $($(PKG)_BASENAME_RAW) \ - $($(PKG)_DL_OPTS) -endef - -# SCP URIs should be of the form scp://[user@]host:filepath -# Note that filepath is relative to the user's home directory, so you may want -# to prepend the path with a slash: scp://[user@]host:/absolutepath -define DOWNLOAD_SCP - $(EXTRA_ENV) $(DL_WRAPPER) -b scp \ - -o $(DL_DIR)/$(2) \ - -H $(PKGDIR)/$($(PKG)_RAWNAME).hash \ - $(QUIET) \ - -- \ - -u '$(call stripurischeme,$(call qstrip,$(1)))' \ - $($(PKG)_DL_OPTS) -endef - -define DOWNLOAD_HG - $(EXTRA_ENV) $(DL_WRAPPER) -b hg \ - -o $(DL_DIR)/$($(PKG)_SOURCE) \ - $(QUIET) \ - -- \ - -u $($(PKG)_SITE) \ - -c $($(PKG)_DL_VERSION) \ - -n $($(PKG)_BASENAME_RAW) \ - $($(PKG)_DL_OPTS) -endef - -define DOWNLOAD_WGET - $(EXTRA_ENV) $(DL_WRAPPER) -b wget \ - -o $(DL_DIR)/$(2) \ - -H $(PKGDIR)/$($(PKG)_RAWNAME).hash \ - $(QUIET) \ - -- \ - -u '$(call qstrip,$(1))' \ - $($(PKG)_DL_OPTS) -endef - -define DOWNLOAD_LOCALFILES - $(EXTRA_ENV) $(DL_WRAPPER) -b cp \ - -o $(DL_DIR)/$(2) \ - -H $(PKGDIR)/$($(PKG)_RAWNAME).hash \ - $(QUIET) \ - -- \ - -u $(call stripurischeme,$(call qstrip,$(1))) \ - $($(PKG)_DL_OPTS) -endef - -################################################################################ -# DOWNLOAD -- Download helper. Will try to download source from: +# DOWNLOAD -- Download helper. Will call DL_WRAPPER which will try to download +# source from: # 1) BR2_PRIMARY_SITE if enabled # 2) Download site, unless BR2_PRIMARY_SITE_ONLY is set # 3) BR2_BACKUP_SITE if enabled, unless BR2_PRIMARY_SITE_ONLY is set # # Argument 1 is the source location # -# E.G. use like this: -# $(call DOWNLOAD,$(FOO_SITE)) -# -# For PRIMARY and BACKUP site, any ? in the URL is replaced by %3F. A ? in -# the URL is used to separate query arguments, but the PRIMARY and BACKUP -# sites serve just plain files. ################################################################################ +ifneq ($(call qstrip,$(BR2_PRIMARY_SITE)),) +DOWNLOAD_URIS += \ + -u $(call getschemeplusuri,$(BR2_PRIMARY_SITE),urlencode) +endif + +ifeq ($(BR2_PRIMARY_SITE_ONLY),) +DOWNLOAD_URIS += \ + -u $($(PKG)_SITE_METHOD)+$(dir $(1)) +ifneq ($(call qstrip,$(BR2_BACKUP_SITE)),) +DOWNLOAD_URIS += \ + -u $(call getschemeplusuri,$(BR2_BACKUP_SITE),urlencode) +endif +endif + define DOWNLOAD - $(call DOWNLOAD_INNER,$(1),$(notdir $(1)),DOWNLOAD) -endef - -define DOWNLOAD_INNER - $(Q)$(if $(filter bzr cvs hg svn,$($(PKG)_SITE_METHOD)),export BR_NO_CHECK_HASH_FOR=$(2);) \ - if test -n "$(call qstrip,$(BR2_PRIMARY_SITE))" ; then \ - case "$(call geturischeme,$(BR2_PRIMARY_SITE))" in \ - file) $(call $(3)_LOCALFILES,$(BR2_PRIMARY_SITE)/$(2),$(2)) && exit ;; \ - scp) $(call $(3)_SCP,$(BR2_PRIMARY_SITE)/$(2),$(2)) && exit ;; \ - *) $(call $(3)_WGET,$(BR2_PRIMARY_SITE)/$(subst ?,%3F,$(2)),$(2)) && exit ;; \ - esac ; \ - fi ; \ - if test "$(BR2_PRIMARY_SITE_ONLY)" = "y" ; then \ - exit 1 ; \ - fi ; \ - if test -n "$(1)" ; then \ - case "$($(PKG)_SITE_METHOD)" in \ - git) $($(3)_GIT) && exit ;; \ - svn) $($(3)_SVN) && exit ;; \ - cvs) $($(3)_CVS) && exit ;; \ - bzr) $($(3)_BZR) && exit ;; \ - file) $($(3)_LOCALFILES) && exit ;; \ - scp) $($(3)_SCP) && exit ;; \ - hg) $($(3)_HG) && exit ;; \ - *) $(call $(3)_WGET,$(1),$(2)) && exit ;; \ - esac ; \ - fi ; \ - if test -n "$(call qstrip,$(BR2_BACKUP_SITE))" ; then \ - $(call $(3)_WGET,$(BR2_BACKUP_SITE)/$(subst ?,%3F,$(2)),$(2)) && exit ; \ - fi ; \ - exit 1 + $(Q)$(if $(filter bzr cvs hg svn,$($(PKG)_SITE_METHOD)),BR_NO_CHECK_HASH_FOR=$(notdir $(1));) \ + $(EXTRA_ENV) $(DL_WRAPPER) \ + -c $($(PKG)_DL_VERSION) \ + -f $(notdir $(1)) \ + -H $(PKGDIR)/$($(PKG)_RAWNAME).hash \ + -n $($(PKG)_BASENAME_RAW) \ + -N $($(PKG)_RAWNAME) \ + -o $(DL_DIR)/$(notdir $(1)) \ + $(if $($(PKG)_GIT_SUBMODULES),-r) \ + $(DOWNLOAD_URIS) \ + $(QUIET) \ + -- \ + $($(PKG)_DL_OPTS) endef diff --git a/support/download/cvs b/support/download/cvs index 69d5c71f28..3f77b849e4 100755 --- a/support/download/cvs +++ b/support/download/cvs @@ -21,7 +21,7 @@ while getopts "${BR_BACKEND_DL_GETOPTS}" OPT; do case "${OPT}" in q) verbose=-Q;; o) output="${OPTARG}";; - u) uri="${OPTARG}";; + u) uri="${OPTARG#*://}";; c) rev="${OPTARG}";; N) rawname="${OPTARG}";; n) basename="${OPTARG}";; diff --git a/support/download/dl-wrapper b/support/download/dl-wrapper index abc51f637a..58cc04b4c4 100755 --- a/support/download/dl-wrapper +++ b/support/download/dl-wrapper @@ -19,31 +19,34 @@ # We want to catch any unexpected failure, and exit immediately. set -e -export BR_BACKEND_DL_GETOPTS=":hc:o:n:N:H:ru:q" +export BR_BACKEND_DL_GETOPTS=":hc:o:n:N:H:ru:qf:e" main() { local OPT OPTARG local backend output hfile recurse quiet rc + local -a uris # Parse our options; anything after '--' is for the backend - while getopts :hb:o:H:rq OPT; do + while getopts ":hc:o:n:N:H:rf:u:q" OPT; do case "${OPT}" in h) help; exit 0;; - b) backend="${OPTARG}";; + c) cset="${OPTARG}";; o) output="${OPTARG}";; + n) raw_base_name="${OPTARG}";; + N) base_name="${OPTARG}";; H) hfile="${OPTARG}";; r) recurse="-r";; + f) filename="${OPTARG}";; + u) uris+=( "${OPTARG}" );; q) quiet="-q";; :) error "option '%s' expects a mandatory argument\n" "${OPTARG}";; \?) error "unknown option '%s'\n" "${OPTARG}";; esac done + # Forget our options, and keep only those for the backend shift $((OPTIND-1)) - if [ -z "${backend}" ]; then - error "no backend specified, use -b\n" - fi if [ -z "${output}" ]; then error "no output specified, use -o\n" fi @@ -66,48 +69,85 @@ main() { warn "Re-downloading '%s'...\n" "${output##*/}" fi - # tmpd is a temporary directory in which backends may store intermediate - # by-products of the download. - # tmpf is the file in which the backends should put the downloaded content. - # tmpd is located in $(BUILD_DIR), so as not to clutter the (precious) - # $(BR2_DL_DIR) - # We let the backends create tmpf, so they are able to set whatever - # permission bits they want (although we're only really interested in - # the executable bit.) - tmpd="$(mktemp -d "${BUILD_DIR}/.${output##*/}.XXXXXX")" - tmpf="${tmpd}/output" + # Look through all the uris that we were given to downoad the package + # source + download_and_check=0 + rc=1 + for uri in "${uris[@]}"; do + backend=${uri%+*} + case "${backend}" in + git|svn|cvs|bzr|file|scp|hg) ;; + *) backend="wget" ;; + esac + uri=${uri#*+} - # Helpers expect to run in a directory that is *really* trashable, so - # they are free to create whatever files and/or sub-dirs they might need. - # Doing the 'cd' here rather than in all backends is easier. - cd "${tmpd}" + urlencode=${backend#*|} + # urlencode must be "urlencode" + [ "${urlencode}" != "urlencode" ] && urlencode="" - # If the backend fails, we can just remove the temporary directory to - # remove all the cruft it may have left behind. Then we just exit in - # error too. - if ! "${OLDPWD}/support/download/${backend}" \ - ${quiet} ${recurse} \ - -o "${tmpf}" "${@}" - then - rm -rf "${tmpd}" - exit 1 - fi + # tmpd is a temporary directory in which backends may store + # intermediate by-products of the download. + # tmpf is the file in which the backends should put the downloaded + # content. + # tmpd is located in $(BUILD_DIR), so as not to clutter the (precious) + # $(BR2_DL_DIR) + # We let the backends create tmpf, so they are able to set whatever + # permission bits they want (although we're only really interested in + # the executable bit.) + tmpd="$(mktemp -d "${BUILD_DIR}/.${output##*/}.XXXXXX")" + tmpf="${tmpd}/output" - # cd back to free the temp-dir, so we can remove it later - cd "${OLDPWD}" + # Helpers expect to run in a directory that is *really* trashable, so + # they are free to create whatever files and/or sub-dirs they might need. + # Doing the 'cd' here rather than in all backends is easier. + cd "${tmpd}" - # Check if the downloaded file is sane, and matches the stored hashes - # for that file - if support/download/check-hash ${quiet} "${hfile}" "${tmpf}" "${output##*/}"; then - rc=0 - else - if [ ${?} -ne 3 ]; then + # If the backend fails, we can just remove the content of the temporary + # directory to remove all the cruft it may have left behind, and tries + # the next URI until it succeeds. Once out of URI to tries, we need to + # cleanup and exit. + if ! "${OLDPWD}/support/download/${backend}" \ + $([ -n "${urlencode}" ] && printf %s '-e') \ + -c "${cset}" \ + -n "${raw_base_name}" \ + -N "${raw_name}" \ + -f "${filename}" \ + -u "${uri}" \ + -o "${tmpf}" \ + ${quiet} ${recurse} "${@}" + then rm -rf "${tmpd}" - exit 1 + # cd back to keep path coherence + cd "${OLDPWD}" + continue fi - # the hash file exists and there was no hash to check the file against - rc=1 + # cd back to free the temp-dir, so we can remove it later + cd "${OLDPWD}" + + # Check if the downloaded file is sane, and matches the stored hashes + # for that file + if support/download/check-hash ${quiet} "${hfile}" "${tmpf}" "${output##*/}"; then + rc=0 + else + if [ ${?} -ne 3 ]; then + rm -rf "${tmpd}" + continue + fi + + # the hash file exists and there was no hash to check the file + # against + rc=1 + fi + download_and_check=1 + break + done + + # We tried every URI possible, none seems to work or to check against the + # available hash. *ABORT MISSION* + if [ "${download_and_check}" -eq 0 ]; then + rm -rf "${tmpd}" + exit 1 fi # tmp_output is in the same directory as the final output, so we can @@ -173,16 +213,13 @@ DESCRIPTION -h This help text. - -b BACKEND - Wrap the specified BACKEND. Known backends are: - bzr Bazaar - cp Local files - cvs Concurrent Versions System - git Git - hg Mercurial - scp Secure copy - svn Subversion - wget HTTP download + -u URIs + The URI to get the file from, the URI must respect the format given in + the example. + You may give as many '-u URI' as you want, the script will stop at the + frist successful download. + + Example: backend+URI; git+http://example.com or http+http://example.com -o FILE Store the downloaded archive in FILE. diff --git a/support/download/wget b/support/download/wget index fece6663ca..c69e6071aa 100755 --- a/support/download/wget +++ b/support/download/wget @@ -8,7 +8,9 @@ set -e # Options: # -q Be quiet. # -o FILE Save into file FILE. +# -f FILENAME The filename of the tarball to get at URL # -u URL Download file at URL. +# -e ENCODE Tell wget to urlencode the filename passed to it # # Environment: # WGET : the wget command to call @@ -18,7 +20,9 @@ while getopts "${BR_BACKEND_DL_GETOPTS}" OPT; do case "${OPT}" in q) verbose=-q;; o) output="${OPTARG}";; + f) filename="${OPTARG}";; u) url="${OPTARG}";; + e) encode="-e";; :) printf "option '%s' expects a mandatory argument\n" "${OPTARG}"; exit 1;; \?) printf "unknown option '%s'\n" "${OPTARG}" >&2; exit 1;; esac @@ -32,4 +36,8 @@ _wget() { eval ${WGET} "${@}" } -_wget ${verbose} "${@}" -O "'${output}'" "'${url}'" +# Replace every '?' with '%3F' in the filename; only for the PRIMARY and BACKUP +# mirror +[ -n "${encode}" ] && filename=${filename//\?/%3F} + +_wget ${verbose} "${@}" -O "'${output}'" "'${url}/${filename}'" From patchwork Mon Apr 2 08:14:24 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxime Hadjinlian X-Patchwork-Id: 894125 Return-Path: X-Original-To: incoming-buildroot@patchwork.ozlabs.org Delivered-To: patchwork-incoming-buildroot@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=busybox.net (client-ip=140.211.166.133; helo=hemlock.osuosl.org; envelope-from=buildroot-bounces@busybox.net; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="c5YBJStV"; dkim-atps=neutral Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 40F4lR4rNHz9s28 for ; Mon, 2 Apr 2018 18:14:59 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id E21B08A303; Mon, 2 Apr 2018 08:14:57 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id zHGdI+O-PifY; Mon, 2 Apr 2018 08:14:47 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by hemlock.osuosl.org (Postfix) with ESMTP id E5E048A328; Mon, 2 Apr 2018 08:14:46 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by ash.osuosl.org (Postfix) with ESMTP id 8AB1B1CF0CA for ; Mon, 2 Apr 2018 08:14:44 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by whitealder.osuosl.org (Postfix) with ESMTP id 872D488E16 for ; Mon, 2 Apr 2018 08:14:44 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from whitealder.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 7T-kQLZiunoC for ; Mon, 2 Apr 2018 08:14:41 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mail-wr0-f193.google.com (mail-wr0-f193.google.com [209.85.128.193]) by whitealder.osuosl.org (Postfix) with ESMTPS id 5555B88DD4 for ; Mon, 2 Apr 2018 08:14:41 +0000 (UTC) Received: by mail-wr0-f193.google.com with SMTP id 80so12690431wrb.2 for ; Mon, 02 Apr 2018 01:14:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=11pav01LPXI9/0/4+WjGO5nuEO8eWdY59PQf8PP9BZs=; b=c5YBJStVT1V/xGiCcr2r+T4CCegR9rATB95tEOCiqEWp9cN6btBMaysbQPD98xHaSu sGB6jLMXPIWquOVRhdrCeS7SvtWagRYA5ISZ5GkCk0Rm0iey8d0oPQHspZPXQ0t52LvO yrBZWocsjLT5FUEYDpAYDsZSK68DV18LIfpctZkYlPvJfaeknTwhp+RAWnG1u3S4g91S 8PGBJfseslwL/XA0cYYK1nuaZ/V25/GWxy/+q0cGxOdcfYZAovX4WO1VOgiXSLdSGV5z Bcuw0n5Ym3sVmUwcYs7xBxkmW6WHJI4kK4ch4dsxR/WpLJ7kfFa4Hl0I6xSDwjh615el WuMA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=11pav01LPXI9/0/4+WjGO5nuEO8eWdY59PQf8PP9BZs=; b=gZ9P/Y1G1K4utJEyzFIHoit2i28Wgrxyoqbyp3fomvuHllxujT+fZB1EGZny/Dx0Iw IY/wbxXy7IewGmBDDWHnVkHjK9T+fcdAuE9XfgHPUjuWCu0U0/MuWK1X5WF7zLQAJPOW SzkppclcsuERBzisbPy7y3pumLKADRLuH+Tmqe22WXAMSoA5NzgzDYFHdSRaVxATRqcq jueyz9Or5HhoeqnhqIv+BAQVUK+RaMSYExt8FxYI5iYQ6Pte/lGLQdMM5+hIdFHL53zO TThFwcko59A2PFM5WaJ5b9ySYoIXaOC5Kq5VizIaVCO5Adqjf1rckhmlNFI5dBTEXRr/ QRFg== X-Gm-Message-State: AElRT7ETSn8tD/pOiBbhzG6RfGyHO9fw6ca72bgW1lS+J2cvqCxTRvsi 9Q07X6A0zcAmtm/50uNOeP/WykFo X-Google-Smtp-Source: AIpwx4/N33BEFG0daeLPSGywAxdYsjTPhFtEbTkQ70NWWhhZ53AKANzo7d2nTwDBk7eavBdQGqN3Mw== X-Received: by 10.223.136.146 with SMTP id f18mr6157536wrf.57.1522656878971; Mon, 02 Apr 2018 01:14:38 -0700 (PDT) Received: from cerise.online.net ([195.154.228.195]) by smtp.gmail.com with ESMTPSA id a205sm51981wmf.18.2018.04.02.01.14.37 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 02 Apr 2018 01:14:38 -0700 (PDT) From: Maxime Hadjinlian To: buildroot@buildroot.org Date: Mon, 2 Apr 2018 10:14:24 +0200 Message-Id: <20180402081434.4411-3-maxime.hadjinlian@gmail.com> X-Mailer: git-send-email 2.16.2 In-Reply-To: <20180402081434.4411-1-maxime.hadjinlian@gmail.com> References: <20180402081434.4411-1-maxime.hadjinlian@gmail.com> Subject: [Buildroot] [v4 03/13] packages: use new $($PKG)_DL_DIR) variable X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.24 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Maxime Hadjinlian MIME-Version: 1.0 Errors-To: buildroot-bounces@busybox.net Sender: "buildroot" Instead of DL_DIR, the package should now use $(PKG)_DL_DIR to ease the transition into a new directory structure for DL_DIR. This commit has been generated with the following scripts: for i in $(find . -iname "*.mk"); do if ! grep -q "\$(DL_DIR)" ${i}; then continue fi pkg_name="$(basename $(dirname ${i}))" [ "${pkg_name}" = "package" ] && continue raw_pkg_name=$(echo ${pkg_name} | tr [a-z] [A-Z] | tr '-' '_') pkg_dl_dir="${raw_pkg_name}_DL_DIR" sed -i "s/\$(DL_DIR)/\$($pkg_dl_dir)/" ${i} done Signed-off-by: Maxime Hadjinlian --- boot/at91bootstrap/at91bootstrap.mk | 2 +- package/amd-catalyst/amd-catalyst.mk | 2 +- package/android-tools/android-tools.mk | 2 +- package/angularjs/angularjs.mk | 2 +- package/bootstrap/bootstrap.mk | 4 +++- package/cache-calibrator/cache-calibrator.mk | 2 +- package/cargo/cargo.mk | 4 ++-- package/cracklib/cracklib.mk | 2 +- package/cryptopp/cryptopp.mk | 2 +- package/devmem2/devmem2.mk | 2 +- package/dhrystone/dhrystone.mk | 2 +- package/doom-wad/doom-wad.mk | 2 +- package/espeak/espeak.mk | 2 +- package/fan-ctrl/fan-ctrl.mk | 2 +- package/freescale-imx/firmware-imx/firmware-imx.mk | 2 +- package/freescale-imx/gpu-amd-bin-mx51/gpu-amd-bin-mx51.mk | 2 +- package/freescale-imx/imx-codec/imx-codec.mk | 2 +- package/freescale-imx/imx-gpu-g2d/imx-gpu-g2d.mk | 2 +- package/freescale-imx/imx-gpu-viv/imx-gpu-viv.mk | 2 +- package/freescale-imx/imx-parser/imx-parser.mk | 2 +- package/freescale-imx/imx-vpu/imx-vpu.mk | 2 +- package/freescale-imx/imx-vpuwrap/imx-vpuwrap.mk | 2 +- package/freescale-imx/libz160/libz160.mk | 2 +- package/gcc/gcc.mk | 2 +- package/irrlicht/irrlicht.mk | 2 +- package/jquery-mobile/jquery-mobile.mk | 2 +- package/jquery-sparkline/jquery-sparkline.mk | 2 +- package/jquery-ui-themes/jquery-ui-themes.mk | 2 +- package/jquery-ui/jquery-ui.mk | 2 +- package/jquery-validation/jquery-validation.mk | 2 +- package/jquery/jquery.mk | 2 +- package/kodi/kodi.mk | 6 +++--- package/libb64/libb64.mk | 2 +- package/libfreeimage/libfreeimage.mk | 2 +- package/libjson/libjson.mk | 2 +- package/libsoil/libsoil.mk | 2 +- package/lsof/lsof.mk | 2 +- package/musl-compat-headers/musl-compat-headers.mk | 2 +- package/nmon/nmon.mk | 2 +- package/nvidia-driver/nvidia-driver.mk | 2 +- .../nvidia-tegra23/nvidia-tegra23-codecs/nvidia-tegra23-codecs.mk | 2 +- package/openobex/openobex.mk | 6 ++++++ package/opentyrian-data/opentyrian-data.mk | 2 +- package/perl/perl.mk | 2 +- package/python-keyring/python-keyring.mk | 1 - package/python-pytz/python-pytz.mk | 2 +- package/python-simplegeneric/python-simplegeneric.mk | 2 +- package/rapidxml/rapidxml.mk | 2 +- package/rpi-wifi-firmware/rpi-wifi-firmware.mk | 2 +- package/rust-bin/rust-bin.mk | 2 +- package/sam-ba/sam-ba.mk | 2 +- package/spidev_test/spidev_test.mk | 2 +- package/tar/tar.mk | 2 +- package/tesseract-ocr/tesseract-ocr.mk | 2 +- package/ti-cgt-pru/ti-cgt-pru.mk | 4 ++-- package/ti-gfx/ti-gfx.mk | 4 ++-- package/ts4900-fpga/ts4900-fpga.mk | 2 +- package/unscd/unscd.mk | 2 +- package/urg/urg.mk | 4 +++- package/waf/waf.mk | 2 +- package/whetstone/whetstone.mk | 2 +- package/wilc1000-firmware/wilc1000-firmware.mk | 2 +- package/zynq-boot-bin/zynq-boot-bin.mk | 2 +- 63 files changed, 76 insertions(+), 67 deletions(-) diff --git a/boot/at91bootstrap/at91bootstrap.mk b/boot/at91bootstrap/at91bootstrap.mk index f6550588d3..c68b56b6ec 100644 --- a/boot/at91bootstrap/at91bootstrap.mk +++ b/boot/at91bootstrap/at91bootstrap.mk @@ -17,7 +17,7 @@ AT91BOOTSTRAP_INSTALL_IMAGES = YES AT91BOOTSTRAP_INSTALL_TARGET = NO define AT91BOOTSTRAP_EXTRACT_CMDS - $(UNZIP) -d $(BUILD_DIR) $(DL_DIR)/$(AT91BOOTSTRAP_SOURCE) + $(UNZIP) -d $(BUILD_DIR) $(AT91BOOTSTRAP_DL_DIR)/$(AT91BOOTSTRAP_SOURCE) mv $(BUILD_DIR)/Bootstrap-v$(AT91BOOTSTRAP_VERSION)/* $(@D) rmdir $(BUILD_DIR)/Bootstrap-v$(AT91BOOTSTRAP_VERSION) endef diff --git a/package/amd-catalyst/amd-catalyst.mk b/package/amd-catalyst/amd-catalyst.mk index b9396e11fa..d864095c31 100644 --- a/package/amd-catalyst/amd-catalyst.mk +++ b/package/amd-catalyst/amd-catalyst.mk @@ -17,7 +17,7 @@ AMD_CATALYST_ARCH_DIR = $(@D)/arch/x86$(AMD_CATALYST_SUFFIX) AMD_CATALYST_LIB_SUFFIX = $(if $(BR2_x86_64),64) define AMD_CATALYST_EXTRACT_CMDS - unzip -q $(DL_DIR)/$(AMD_CATALYST_SOURCE) -d $(@D) + unzip -q $(AMD_CATALYST_DL_DIR)/$(AMD_CATALYST_SOURCE) -d $(@D) $(SHELL) $(@D)/AMD-Catalyst-$(AMD_CATALYST_VERSION)-Linux-installer-$(AMD_CATALYST_VERBOSE_VER)-x86.x86_64.run --extract $(@D) endef diff --git a/package/android-tools/android-tools.mk b/package/android-tools/android-tools.mk index f6c6913a68..11e0a15a7f 100644 --- a/package/android-tools/android-tools.mk +++ b/package/android-tools/android-tools.mk @@ -15,7 +15,7 @@ ANDROID_TOOLS_LICENSE_FILES = debian/copyright # Extract the Debian tarball inside the sources define ANDROID_TOOLS_DEBIAN_EXTRACT $(call suitable-extractor,$(notdir $(ANDROID_TOOLS_EXTRA_DOWNLOADS))) \ - $(DL_DIR)/$(notdir $(ANDROID_TOOLS_EXTRA_DOWNLOADS)) | \ + $(ANDROID_TOOLS_DL_DIR)/$(notdir $(ANDROID_TOOLS_EXTRA_DOWNLOADS)) | \ $(TAR) -C $(@D) $(TAR_OPTIONS) - endef diff --git a/package/angularjs/angularjs.mk b/package/angularjs/angularjs.mk index 60702a26d2..a941bc3828 100644 --- a/package/angularjs/angularjs.mk +++ b/package/angularjs/angularjs.mk @@ -12,7 +12,7 @@ ANGULARJS_LICENSE = MIT ANGULARJS_LICENSE_FILES = angular.js define ANGULARJS_EXTRACT_CMDS - unzip $(DL_DIR)/$(ANGULARJS_SOURCE) -d $(@D) + unzip $(ANGULARJS_DL_DIR)/$(ANGULARJS_SOURCE) -d $(@D) mv $(@D)/angular-$(ANGULARJS_VERSION)/* $(@D) rmdir $(@D)/angular-$(ANGULARJS_VERSION) endef diff --git a/package/bootstrap/bootstrap.mk b/package/bootstrap/bootstrap.mk index 61d6c826fc..50cdc322e4 100644 --- a/package/bootstrap/bootstrap.mk +++ b/package/bootstrap/bootstrap.mk @@ -11,7 +11,9 @@ BOOTSTRAP_LICENSE = MIT BOOTSTRAP_LICENSE_FILES = css/bootstrap.css define BOOTSTRAP_EXTRACT_CMDS - $(UNZIP) $(DL_DIR)/$(BOOTSTRAP_SOURCE) -d $(@D) + $(UNZIP) $(BOOTSTRAP_DL_DIR)/$(BOOTSTRAP_SOURCE) -d $(@D) + mv $(@D)/bootstrap-$(BOOTSTRAP_VERSION)-dist/* $(@D)/ + rmdir $(@D)/bootstrap-$(BOOTSTRAP_VERSION)-dist endef define BOOTSTRAP_INSTALL_TARGET_CMDS diff --git a/package/cache-calibrator/cache-calibrator.mk b/package/cache-calibrator/cache-calibrator.mk index 6f12d119ef..203732e4f7 100644 --- a/package/cache-calibrator/cache-calibrator.mk +++ b/package/cache-calibrator/cache-calibrator.mk @@ -10,7 +10,7 @@ CACHE_CALIBRATOR_LICENSE = Cache calibrator license CACHE_CALIBRATOR_LICENSE_FILES = calibrator.c.license define CACHE_CALIBRATOR_EXTRACT_CMDS - cp $(DL_DIR)/$(CACHE_CALIBRATOR_SOURCE) $(@D) + cp $(CACHE_CALIBRATOR_DL_DIR)/$(CACHE_CALIBRATOR_SOURCE) $(@D) endef define CACHE_CALIBRATOR_EXTRACT_LICENSE diff --git a/package/cargo/cargo.mk b/package/cargo/cargo.mk index 3fd088d727..36b2718e90 100644 --- a/package/cargo/cargo.mk +++ b/package/cargo/cargo.mk @@ -37,7 +37,7 @@ HOST_CARGO_HOME = $(HOST_DIR)/share/cargo define HOST_CARGO_EXTRACT_DEPS @mkdir -p $(@D)/vendor $(call suitable-extractor,$(CARGO_DEPS_SOURCE)) \ - $(DL_DIR)/$(CARGO_DEPS_SOURCE) | \ + $(CARGO_DL_DIR)/$(CARGO_DEPS_SOURCE) | \ $(TAR) --strip-components=1 -C $(@D)/vendor $(TAR_OPTIONS) - endef @@ -46,7 +46,7 @@ HOST_CARGO_POST_EXTRACT_HOOKS += HOST_CARGO_EXTRACT_DEPS define HOST_CARGO_EXTRACT_INSTALLER @mkdir -p $(@D)/src/rust-installer $(call suitable-extractor,$(CARGO_INSTALLER_SOURCE)) \ - $(DL_DIR)/$(CARGO_INSTALLER_SOURCE) | \ + $(CARGO_DL_DIR)/$(CARGO_INSTALLER_SOURCE) | \ $(TAR) --strip-components=1 -C $(@D)/src/rust-installer $(TAR_OPTIONS) - endef diff --git a/package/cracklib/cracklib.mk b/package/cracklib/cracklib.mk index 2fa3d3b4ea..aeee60d1e7 100644 --- a/package/cracklib/cracklib.mk +++ b/package/cracklib/cracklib.mk @@ -29,7 +29,7 @@ HOST_CRACKLIB_CONF_OPTS += --without-python --without-zlib ifeq ($(BR2_PACKAGE_CRACKLIB_FULL_DICT),y) CRACKLIB_EXTRA_DOWNLOADS = cracklib-words-$(CRACKLIB_VERSION).gz -CRACKLIB_DICT_SOURCE = $(DL_DIR)/cracklib-words-$(CRACKLIB_VERSION).gz +CRACKLIB_DICT_SOURCE = $(CRACKLIB_DL_DIR)/cracklib-words-$(CRACKLIB_VERSION).gz else CRACKLIB_DICT_SOURCE = $(@D)/dicts/cracklib-small endif diff --git a/package/cryptopp/cryptopp.mk b/package/cryptopp/cryptopp.mk index d7aabf04b8..6da39b09f2 100644 --- a/package/cryptopp/cryptopp.mk +++ b/package/cryptopp/cryptopp.mk @@ -12,7 +12,7 @@ CRYPTOPP_LICENSE_FILES = License.txt CRYPTOPP_INSTALL_STAGING = YES define HOST_CRYPTOPP_EXTRACT_CMDS - $(UNZIP) $(DL_DIR)/$(CRYPTOPP_SOURCE) -d $(@D) + $(UNZIP) $(CRYPTOPP_DL_DIR)/$(CRYPTOPP_SOURCE) -d $(@D) endef HOST_CRYPTOPP_MAKE_OPTS = \ diff --git a/package/devmem2/devmem2.mk b/package/devmem2/devmem2.mk index eb92db6098..044e6fa596 100644 --- a/package/devmem2/devmem2.mk +++ b/package/devmem2/devmem2.mk @@ -11,7 +11,7 @@ DEVMEM2_LICENSE = GPL-2.0+ DEVMEM2_LICENSE_FILES = devmem2.c.license define DEVMEM2_EXTRACT_CMDS - cp $(DL_DIR)/$($(PKG)_SOURCE) $(@D)/ + cp $(DEVMEM2_DL_DIR)/$($(PKG)_SOURCE) $(@D)/ endef define DEVMEM2_EXTRACT_LICENSE diff --git a/package/dhrystone/dhrystone.mk b/package/dhrystone/dhrystone.mk index c0bca9895d..d54c8d94e8 100644 --- a/package/dhrystone/dhrystone.mk +++ b/package/dhrystone/dhrystone.mk @@ -9,7 +9,7 @@ DHRYSTONE_SOURCE = dhry-c DHRYSTONE_SITE = http://www.netlib.org/benchmark define DHRYSTONE_EXTRACT_CMDS - (cd $(@D) && $(SHELL) $(DL_DIR)/$($(PKG)_SOURCE)) + (cd $(@D) && $(SHELL) $(DHRYSTONE_DL_DIR)/$($(PKG)_SOURCE)) $(Q)cp package/dhrystone/Makefile $(@D)/ endef diff --git a/package/doom-wad/doom-wad.mk b/package/doom-wad/doom-wad.mk index d3ac731bd5..f348fc2207 100644 --- a/package/doom-wad/doom-wad.mk +++ b/package/doom-wad/doom-wad.mk @@ -11,7 +11,7 @@ DOOM_WAD_SOURCE = doom$(subst .,,$(DOOM_WAD_VERSION))s.zip DOOM_WAD_SITE = http://www.jbserver.com/downloads/games/doom/misc/shareware define DOOM_WAD_EXTRACT_CMDS - $(UNZIP) -p $(DL_DIR)/$($(PKG)_SOURCE) 'DOOMS_19.[12]' > \ + $(UNZIP) -p $(DOOM_WAD_DL_DIR)/$($(PKG)_SOURCE) 'DOOMS_19.[12]' > \ $(@D)/doom-$(DOOM_WAD_VERSION).zip $(UNZIP) -d $(@D) $(@D)/doom-$(DOOM_WAD_VERSION).zip DOOM1.WAD endef diff --git a/package/espeak/espeak.mk b/package/espeak/espeak.mk index 226f17fc05..aa0b5b0716 100644 --- a/package/espeak/espeak.mk +++ b/package/espeak/espeak.mk @@ -21,7 +21,7 @@ ESPEAK_DEPENDENCIES = pulseaudio endif define ESPEAK_EXTRACT_CMDS - $(UNZIP) -d $(@D) $(DL_DIR)/$(ESPEAK_SOURCE) + $(UNZIP) -d $(@D) $(ESPEAK_DL_DIR)/$(ESPEAK_SOURCE) mv $(@D)/espeak-$(ESPEAK_VERSION)-source/* $(@D) $(RM) -r $(@D)/espeak-$(ESPEAK_VERSION)-source endef diff --git a/package/fan-ctrl/fan-ctrl.mk b/package/fan-ctrl/fan-ctrl.mk index 53533a8a17..81a52f1e9f 100644 --- a/package/fan-ctrl/fan-ctrl.mk +++ b/package/fan-ctrl/fan-ctrl.mk @@ -12,7 +12,7 @@ FAN_CTRL_LICENSE = GPL-2.0+ FAN_CTRL_LICENSE_FILES = fan-ctrl.c define FAN_CTRL_EXTRACT_CMDS - cp $(DL_DIR)/$(FAN_CTRL_SOURCE) $(@D)/fan-ctrl.c + cp $(FAN_CTRL_DL_DIR)/$(FAN_CTRL_SOURCE) $(@D)/fan-ctrl.c endef define FAN_CTRL_BUILD_CMDS diff --git a/package/freescale-imx/firmware-imx/firmware-imx.mk b/package/freescale-imx/firmware-imx/firmware-imx.mk index 630afcaa37..d5387b38da 100644 --- a/package/freescale-imx/firmware-imx/firmware-imx.mk +++ b/package/freescale-imx/firmware-imx/firmware-imx.mk @@ -15,7 +15,7 @@ FIRMWARE_IMX_REDISTRIBUTE = NO FIRMWARE_IMX_BLOBS = sdma vpu define FIRMWARE_IMX_EXTRACT_CMDS - $(call FREESCALE_IMX_EXTRACT_HELPER,$(DL_DIR)/$(FIRMWARE_IMX_SOURCE)) + $(call FREESCALE_IMX_EXTRACT_HELPER,$(FIRMWARE_IMX_DL_DIR)/$(FIRMWARE_IMX_SOURCE)) endef define FIRMWARE_IMX_INSTALL_TARGET_CMDS diff --git a/package/freescale-imx/gpu-amd-bin-mx51/gpu-amd-bin-mx51.mk b/package/freescale-imx/gpu-amd-bin-mx51/gpu-amd-bin-mx51.mk index 44a6b4c7d2..5586073712 100644 --- a/package/freescale-imx/gpu-amd-bin-mx51/gpu-amd-bin-mx51.mk +++ b/package/freescale-imx/gpu-amd-bin-mx51/gpu-amd-bin-mx51.mk @@ -23,7 +23,7 @@ GPU_AMD_BIN_MX51_LICENSE_FILES = EULA GPU_AMD_BIN_MX51_REDISTRIBUTE = NO define GPU_AMD_BIN_MX51_EXTRACT_CMDS - $(call FREESCALE_IMX_EXTRACT_HELPER,$(DL_DIR)/$(GPU_AMD_BIN_MX51_SOURCE)) + $(call FREESCALE_IMX_EXTRACT_HELPER,$(GPU_AMD_BIN_MX51_DL_DIR)/$(GPU_AMD_BIN_MX51_SOURCE)) endef # Upstream headers need to be compiled with -D_LINUX. It is more convenient diff --git a/package/freescale-imx/imx-codec/imx-codec.mk b/package/freescale-imx/imx-codec/imx-codec.mk index ea4d90110f..784c1fa4a9 100644 --- a/package/freescale-imx/imx-codec/imx-codec.mk +++ b/package/freescale-imx/imx-codec/imx-codec.mk @@ -26,7 +26,7 @@ IMX_CODEC_CONF_OPTS += --enable-vpu endif define IMX_CODEC_EXTRACT_CMDS - $(call FREESCALE_IMX_EXTRACT_HELPER,$(DL_DIR)/$(IMX_CODEC_SOURCE)) + $(call FREESCALE_IMX_EXTRACT_HELPER,$(IMX_CODEC_DL_DIR)/$(IMX_CODEC_SOURCE)) endef # FIXME The Makefile installs both the arm9 and arm11 versions of the diff --git a/package/freescale-imx/imx-gpu-g2d/imx-gpu-g2d.mk b/package/freescale-imx/imx-gpu-g2d/imx-gpu-g2d.mk index 0c92b826c4..aafd70dc9d 100644 --- a/package/freescale-imx/imx-gpu-g2d/imx-gpu-g2d.mk +++ b/package/freescale-imx/imx-gpu-g2d/imx-gpu-g2d.mk @@ -15,7 +15,7 @@ IMX_GPU_G2D_LICENSE_FILES = EULA COPYING IMX_GPU_G2D_REDISTRIBUTE = NO define IMX_GPU_G2D_EXTRACT_CMDS - $(call FREESCALE_IMX_EXTRACT_HELPER,$(DL_DIR)/$(IMX_GPU_G2D_SOURCE)) + $(call FREESCALE_IMX_EXTRACT_HELPER,$(IMX_GPU_G2D_DL_DIR)/$(IMX_GPU_G2D_SOURCE)) endef define IMX_GPU_G2D_INSTALL_STAGING_CMDS diff --git a/package/freescale-imx/imx-gpu-viv/imx-gpu-viv.mk b/package/freescale-imx/imx-gpu-viv/imx-gpu-viv.mk index a9f8b9d9bd..0dc2072984 100644 --- a/package/freescale-imx/imx-gpu-viv/imx-gpu-viv.mk +++ b/package/freescale-imx/imx-gpu-viv/imx-gpu-viv.mk @@ -26,7 +26,7 @@ IMX_GPU_VIV_DEPENDENCIES += xlib_libXdamage xlib_libXext xlib_libXfixes endif define IMX_GPU_VIV_EXTRACT_CMDS - $(call FREESCALE_IMX_EXTRACT_HELPER,$(DL_DIR)/$(IMX_GPU_VIV_SOURCE)) + $(call FREESCALE_IMX_EXTRACT_HELPER,$(IMX_GPU_VIV_DL_DIR)/$(IMX_GPU_VIV_SOURCE)) endef # Instead of building, we fix up the inconsistencies that exist diff --git a/package/freescale-imx/imx-parser/imx-parser.mk b/package/freescale-imx/imx-parser/imx-parser.mk index d79a4444c8..11536e5024 100644 --- a/package/freescale-imx/imx-parser/imx-parser.mk +++ b/package/freescale-imx/imx-parser/imx-parser.mk @@ -24,7 +24,7 @@ IMX_PARSER_CONF_OPTS += --enable-fsw endif define IMX_PARSER_EXTRACT_CMDS - $(call FREESCALE_IMX_EXTRACT_HELPER,$(DL_DIR)/$(IMX_PARSER_SOURCE)) + $(call FREESCALE_IMX_EXTRACT_HELPER,$(IMX_PARSER_DL_DIR)/$(IMX_PARSER_SOURCE)) endef # The Makefile installs several versions of the libraries, but we only diff --git a/package/freescale-imx/imx-vpu/imx-vpu.mk b/package/freescale-imx/imx-vpu/imx-vpu.mk index e3a1ee661d..2bcfe53eba 100644 --- a/package/freescale-imx/imx-vpu/imx-vpu.mk +++ b/package/freescale-imx/imx-vpu/imx-vpu.mk @@ -21,7 +21,7 @@ IMX_VPU_LICENSE_FILES = EULA COPYING IMX_VPU_REDISTRIBUTE = NO define IMX_VPU_EXTRACT_CMDS - $(call FREESCALE_IMX_EXTRACT_HELPER,$(DL_DIR)/$(IMX_VPU_SOURCE)) + $(call FREESCALE_IMX_EXTRACT_HELPER,$(IMX_VPU_DL_DIR)/$(IMX_VPU_SOURCE)) endef define IMX_VPU_BUILD_CMDS diff --git a/package/freescale-imx/imx-vpuwrap/imx-vpuwrap.mk b/package/freescale-imx/imx-vpuwrap/imx-vpuwrap.mk index 7cbf7784c9..edba87279f 100644 --- a/package/freescale-imx/imx-vpuwrap/imx-vpuwrap.mk +++ b/package/freescale-imx/imx-vpuwrap/imx-vpuwrap.mk @@ -15,7 +15,7 @@ IMX_VPUWRAP_LICENSE_FILES = EULA COPYING IMX_VPUWRAP_REDISTRIBUTE = NO define IMX_VPUWRAP_EXTRACT_CMDS - $(call FREESCALE_IMX_EXTRACT_HELPER,$(DL_DIR)/$(IMX_VPUWRAP_SOURCE)) + $(call FREESCALE_IMX_EXTRACT_HELPER,$(IMX_VPUWRAP_DL_DIR)/$(IMX_VPUWRAP_SOURCE)) endef $(eval $(autotools-package)) diff --git a/package/freescale-imx/libz160/libz160.mk b/package/freescale-imx/libz160/libz160.mk index af43b1859d..5b1feeb441 100644 --- a/package/freescale-imx/libz160/libz160.mk +++ b/package/freescale-imx/libz160/libz160.mk @@ -15,7 +15,7 @@ LIBZ160_LICENSE_FILES = EULA LIBZ160_REDISTRIBUTE = NO define LIBZ160_EXTRACT_CMDS - $(call FREESCALE_IMX_EXTRACT_HELPER,$(DL_DIR)/$(LIBZ160_SOURCE)) + $(call FREESCALE_IMX_EXTRACT_HELPER,$(LIBZ160_DL_DIR)/$(LIBZ160_SOURCE)) endef define LIBZ160_INSTALL_STAGING_CMDS diff --git a/package/gcc/gcc.mk b/package/gcc/gcc.mk index 27fc1e987c..f42f36a967 100644 --- a/package/gcc/gcc.mk +++ b/package/gcc/gcc.mk @@ -316,7 +316,7 @@ HOST_GCC_COMMON_MAKE_OPTS = \ gcc_cv_libc_provides_ssp=$(if $(BR2_TOOLCHAIN_HAS_SSP),yes,no) ifeq ($(BR2_CCACHE),y) -HOST_GCC_COMMON_CCACHE_HASH_FILES += $(DL_DIR)/$(GCC_SOURCE) +HOST_GCC_COMMON_CCACHE_HASH_FILES += $(GCC_DL_DIR)/$(GCC_SOURCE) # Cfr. PATCH_BASE_DIRS in .stamp_patched, but we catch both versioned # and unversioned patches unconditionally. Moreover, to facilitate the diff --git a/package/irrlicht/irrlicht.mk b/package/irrlicht/irrlicht.mk index ccd1045ca0..a1e190d98d 100644 --- a/package/irrlicht/irrlicht.mk +++ b/package/irrlicht/irrlicht.mk @@ -25,7 +25,7 @@ IRRLICHT_SUBDIR = source/Irrlicht IRRLICHT_DEPENDENCIES = libgl xlib_libXxf86vm define IRRLICHT_EXTRACT_CMDS - $(UNZIP) -d $(@D) $(DL_DIR)/$(IRRLICHT_SOURCE) + $(UNZIP) -d $(@D) $(IRRLICHT_DL_DIR)/$(IRRLICHT_SOURCE) mv $(@D)/irrlicht-$(IRRLICHT_VERSION)/* $(@D) $(RM) -r $(@D)/irrlicht-$(IRRLICHT_VERSION) endef diff --git a/package/jquery-mobile/jquery-mobile.mk b/package/jquery-mobile/jquery-mobile.mk index 1067952904..1171e60d0a 100644 --- a/package/jquery-mobile/jquery-mobile.mk +++ b/package/jquery-mobile/jquery-mobile.mk @@ -10,7 +10,7 @@ JQUERY_MOBILE_SOURCE = jquery.mobile-$(JQUERY_MOBILE_VERSION).zip JQUERY_MOBILE_LICENSE = MIT define JQUERY_MOBILE_EXTRACT_CMDS - $(UNZIP) -d $(@D) $(DL_DIR)/$(JQUERY_MOBILE_SOURCE) + $(UNZIP) -d $(@D) $(JQUERY_MOBILE_DL_DIR)/$(JQUERY_MOBILE_SOURCE) endef JQUERY_MOBILE_INSTALLED_FILES = \ diff --git a/package/jquery-sparkline/jquery-sparkline.mk b/package/jquery-sparkline/jquery-sparkline.mk index 165c2fa99c..683d84f4aa 100644 --- a/package/jquery-sparkline/jquery-sparkline.mk +++ b/package/jquery-sparkline/jquery-sparkline.mk @@ -10,7 +10,7 @@ JQUERY_SPARKLINE_SOURCE = jquery.sparkline.min.js JQUERY_SPARKLINE_LICENSE = BSD-3-Clause define JQUERY_SPARKLINE_EXTRACT_CMDS - cp $(DL_DIR)/$(JQUERY_SPARKLINE_SOURCE) $(@D) + cp $(JQUERY_SPARKLINE_DL_DIR)/$(JQUERY_SPARKLINE_SOURCE) $(@D) endef define JQUERY_SPARKLINE_INSTALL_TARGET_CMDS diff --git a/package/jquery-ui-themes/jquery-ui-themes.mk b/package/jquery-ui-themes/jquery-ui-themes.mk index cd3dab6d87..5108389a77 100644 --- a/package/jquery-ui-themes/jquery-ui-themes.mk +++ b/package/jquery-ui-themes/jquery-ui-themes.mk @@ -12,7 +12,7 @@ JQUERY_UI_THEMES_LICENSE_FILES = MIT-LICENSE.txt JQUERY_UI_THEMES_DEPENDENCIES = jquery-ui define JQUERY_UI_THEMES_EXTRACT_CMDS - $(UNZIP) -d $(@D) $(DL_DIR)/$(JQUERY_UI_THEMES_SOURCE) + $(UNZIP) -d $(@D) $(JQUERY_UI_THEMES_DL_DIR)/$(JQUERY_UI_THEMES_SOURCE) mv $(@D)/jquery-ui-themes-$(JQUERY_UI_THEMES_VERSION)/* $(@D) $(RM) -r $(@D)/jquery-ui-themes-$(JQUERY_UI_THEMES_VERSION) endef diff --git a/package/jquery-ui/jquery-ui.mk b/package/jquery-ui/jquery-ui.mk index d829d6241b..9284e683f1 100644 --- a/package/jquery-ui/jquery-ui.mk +++ b/package/jquery-ui/jquery-ui.mk @@ -14,7 +14,7 @@ JQUERY_UI_LICENSE = MIT JQUERY_UI_LICENSE_FILES = MIT-LICENSE.txt define JQUERY_UI_EXTRACT_CMDS - $(UNZIP) -d $(@D) $(DL_DIR)/$(JQUERY_UI_SOURCE) + $(UNZIP) -d $(@D) $(JQUERY_UI_DL_DIR)/$(JQUERY_UI_SOURCE) mv $(@D)/jquery-ui-$(JQUERY_UI_VERSION)/* $(@D) $(RM) -r $(@D)/jquery-ui-$(JQUERY_UI_VERSION) endef diff --git a/package/jquery-validation/jquery-validation.mk b/package/jquery-validation/jquery-validation.mk index 10d2a92afc..a8c2897983 100644 --- a/package/jquery-validation/jquery-validation.mk +++ b/package/jquery-validation/jquery-validation.mk @@ -11,7 +11,7 @@ JQUERY_VALIDATION_LICENSE = MIT JQUERY_VALIDATION_LICENSE_FILES = README.md define JQUERY_VALIDATION_EXTRACT_CMDS - $(UNZIP) -d $(@D) $(DL_DIR)/$(JQUERY_VALIDATION_SOURCE) + $(UNZIP) -d $(@D) $(JQUERY_VALIDATION_DL_DIR)/$(JQUERY_VALIDATION_SOURCE) endef define JQUERY_VALIDATION_INSTALL_TARGET_CMDS diff --git a/package/jquery/jquery.mk b/package/jquery/jquery.mk index 363ee09633..f75eded8b3 100644 --- a/package/jquery/jquery.mk +++ b/package/jquery/jquery.mk @@ -10,7 +10,7 @@ JQUERY_SOURCE = jquery-$(JQUERY_VERSION).min.js JQUERY_LICENSE = MIT define JQUERY_EXTRACT_CMDS - cp $(DL_DIR)/$(JQUERY_SOURCE) $(@D) + cp $(JQUERY_DL_DIR)/$(JQUERY_SOURCE) $(@D) endef define JQUERY_INSTALL_TARGET_CMDS diff --git a/package/kodi/kodi.mk b/package/kodi/kodi.mk index 9d3b65ae91..f3fa0b55b0 100644 --- a/package/kodi/kodi.mk +++ b/package/kodi/kodi.mk @@ -69,9 +69,9 @@ KODI_CONF_OPTS += \ -DDEPENDS_PATH=$(@D) \ -DWITH_FFMPEG=$(STAGING_DIR)/usr \ -DWITH_TEXTUREPACKER=$(HOST_DIR)/bin/TexturePacker \ - -DLIBDVDCSS_URL=$(DL_DIR)/$(KODI_LIBDVDCSS_VERSION).tar.gz \ - -DLIBDVDNAV_URL=$(DL_DIR)/$(KODI_LIBDVDNAV_VERSION).tar.gz \ - -DLIBDVDREAD_URL=$(DL_DIR)/$(KODI_LIBDVDREAD_VERSION).tar.gz \ + -DLIBDVDCSS_URL=$(KODI_DL_DIR)/$(KODI_LIBDVDCSS_VERSION).tar.gz \ + -DLIBDVDNAV_URL=$(KODI_DL_DIR)/$(KODI_LIBDVDNAV_VERSION).tar.gz \ + -DLIBDVDREAD_URL=$(KODI_DL_DIR)/$(KODI_LIBDVDREAD_VERSION).tar.gz \ -DENABLE_IMX=OFF ifeq ($(BR2_ENABLE_LOCALE),) diff --git a/package/libb64/libb64.mk b/package/libb64/libb64.mk index 4dea9593a0..ed6d3cf4b4 100644 --- a/package/libb64/libb64.mk +++ b/package/libb64/libb64.mk @@ -14,7 +14,7 @@ LIBB64_INSTALL_STAGING = YES LIBB64_INSTALL_TARGET = NO define LIBB64_EXTRACT_CMDS - unzip $(DL_DIR)/$(LIBB64_SOURCE) -d $(BUILD_DIR) + unzip $(LIBB64_DL_DIR)/$(LIBB64_SOURCE) -d $(BUILD_DIR) endef define LIBB64_BUILD_CMDS diff --git a/package/libfreeimage/libfreeimage.mk b/package/libfreeimage/libfreeimage.mk index 0ca23933a6..e0aa1f0ae8 100644 --- a/package/libfreeimage/libfreeimage.mk +++ b/package/libfreeimage/libfreeimage.mk @@ -12,7 +12,7 @@ LIBFREEIMAGE_LICENSE_FILES = license-gplv2.txt license-gplv3.txt license-fi.txt LIBFREEIMAGE_INSTALL_STAGING = YES define LIBFREEIMAGE_EXTRACT_CMDS - $(UNZIP) $(DL_DIR)/$(LIBFREEIMAGE_SOURCE) -d $(@D) + $(UNZIP) $(LIBFREEIMAGE_DL_DIR)/$(LIBFREEIMAGE_SOURCE) -d $(@D) mv $(@D)/FreeImage/* $(@D) rmdir $(@D)/FreeImage endef diff --git a/package/libjson/libjson.mk b/package/libjson/libjson.mk index 74224ba657..d04ddc40f3 100644 --- a/package/libjson/libjson.mk +++ b/package/libjson/libjson.mk @@ -23,7 +23,7 @@ endif LIBJSON_MAKE_OPTS += BUILD_TYPE= CXXFLAGS="$(LIBJSON_CXXFLAGS)" define LIBJSON_EXTRACT_CMDS - $(UNZIP) -d $(@D) $(DL_DIR)/$(LIBJSON_SOURCE) + $(UNZIP) -d $(@D) $(LIBJSON_DL_DIR)/$(LIBJSON_SOURCE) mv $(@D)/libjson/* $(@D) $(RM) -r $(@D)/libjson $(SED) '/ldconfig/d' $(@D)/makefile diff --git a/package/libsoil/libsoil.mk b/package/libsoil/libsoil.mk index efa67d9eee..2945edd8dd 100644 --- a/package/libsoil/libsoil.mk +++ b/package/libsoil/libsoil.mk @@ -14,7 +14,7 @@ LIBSOIL_LICENSE_FILES = src/stb_image_aug.c src/image_helper.c LIBSOIL_MAKEFILE = ../projects/makefile/alternate_Makefile.txt define LIBSOIL_EXTRACT_CMDS - $(UNZIP) -d $(@D) $(DL_DIR)/$(LIBSOIL_SOURCE) + $(UNZIP) -d $(@D) $(LIBSOIL_DL_DIR)/$(LIBSOIL_SOURCE) mv $(@D)/Simple\ OpenGL\ Image\ Library/* $(@D) endef diff --git a/package/lsof/lsof.mk b/package/lsof/lsof.mk index e5cd4bce6e..0dc8e2de2f 100644 --- a/package/lsof/lsof.mk +++ b/package/lsof/lsof.mk @@ -32,7 +32,7 @@ endif # The .tar.bz2 contains another .tar, which contains the source code. define LSOF_EXTRACT_CMDS - $(call suitable-extractor,$(LSOF_SOURCE)) $(DL_DIR)/$(LSOF_SOURCE) | \ + $(call suitable-extractor,$(LSOF_SOURCE)) $(LSOF_DL_DIR)/$(LSOF_SOURCE) | \ $(TAR) -O $(TAR_OPTIONS) - lsof_$(LSOF_VERSION)/lsof_$(LSOF_VERSION)_src.tar | \ $(TAR) --strip-components=1 -C $(LSOF_DIR) $(TAR_OPTIONS) - endef diff --git a/package/musl-compat-headers/musl-compat-headers.mk b/package/musl-compat-headers/musl-compat-headers.mk index 91f5074ef6..1cfa894879 100644 --- a/package/musl-compat-headers/musl-compat-headers.mk +++ b/package/musl-compat-headers/musl-compat-headers.mk @@ -20,7 +20,7 @@ MUSL_COMPAT_HEADERS_INSTALL_STAGING = YES # Copying both headers so legal-info finds them (they are _LICENSE_FILES) define MUSL_COMPAT_HEADERS_EXTRACT_CMDS - $(INSTALL) -m 0644 -D $(DL_DIR)/$(notdir $(MUSL_COMPAT_HEADERS_QUEUE_H)) $(@D)/queue.h + $(INSTALL) -m 0644 -D $(MUSL_COMPAT_HEADERS_DL_DIR)/$(notdir $(MUSL_COMPAT_HEADERS_QUEUE_H)) $(@D)/queue.h $(INSTALL) -m 0644 -D $(MUSL_COMPAT_HEADERS_PKGDIR)/cdefs.h $(@D)/cdefs.h endef diff --git a/package/nmon/nmon.mk b/package/nmon/nmon.mk index f561d6dcc0..f7edd66a26 100644 --- a/package/nmon/nmon.mk +++ b/package/nmon/nmon.mk @@ -13,7 +13,7 @@ NMON_DEPENDENCIES = ncurses NMON_CFLAGS = $(TARGET_CFLAGS) -D JFS -D GETUSER -D LARGEMEM -D DEBIAN define NMON_EXTRACT_CMDS - cp $(DL_DIR)/$(NMON_SOURCE) $(@D) + cp $(NMON_DL_DIR)/$(NMON_SOURCE) $(@D) endef define NMON_BUILD_CMDS diff --git a/package/nvidia-driver/nvidia-driver.mk b/package/nvidia-driver/nvidia-driver.mk index e56661059f..404fa881ec 100644 --- a/package/nvidia-driver/nvidia-driver.mk +++ b/package/nvidia-driver/nvidia-driver.mk @@ -144,7 +144,7 @@ endif # BR2_PACKAGE_NVIDIA_DRIVER_MODULE == y # virtually everywhere, and it is fine enough to provide useful options. # Except it can't extract into an existing (even empty) directory. define NVIDIA_DRIVER_EXTRACT_CMDS - $(SHELL) $(DL_DIR)/$(NVIDIA_DRIVER_SOURCE) --extract-only --target \ + $(SHELL) $(NVIDIA_DRIVER_DL_DIR)/$(NVIDIA_DRIVER_SOURCE) --extract-only --target \ $(@D)/tmp-extract chmod u+w -R $(@D) mv $(@D)/tmp-extract/* $(@D)/tmp-extract/.manifest $(@D) diff --git a/package/nvidia-tegra23/nvidia-tegra23-codecs/nvidia-tegra23-codecs.mk b/package/nvidia-tegra23/nvidia-tegra23-codecs/nvidia-tegra23-codecs.mk index 5514643416..2885021c58 100644 --- a/package/nvidia-tegra23/nvidia-tegra23-codecs/nvidia-tegra23-codecs.mk +++ b/package/nvidia-tegra23/nvidia-tegra23-codecs/nvidia-tegra23-codecs.mk @@ -15,7 +15,7 @@ NVIDIA_TEGRA23_CODECS_REDISTRIBUTE = NO define NVIDIA_TEGRA23_CODECS_EXTRACT_CMDS $(INSTALL) -d $(@D) $(call suitable-extractor,$(NVIDIA_TEGRA23_CODECS_SOURCE)) \ - $(DL_DIR)/$(NVIDIA_TEGRA23_CODECS_SOURCE) | \ + $(NVIDIA_TEGRA23_CODECS_DL_DIR)/$(NVIDIA_TEGRA23_CODECS_SOURCE) | \ $(TAR) --strip-components=0 -C $(@D) $(TAR_OPTIONS) - $(INSTALL) -d $(@D)/restricted_codecs $(call suitable-extractor,$(@D)/restricted_codecs.tbz2) \ diff --git a/package/openobex/openobex.mk b/package/openobex/openobex.mk index ea41f65a80..c19780110c 100644 --- a/package/openobex/openobex.mk +++ b/package/openobex/openobex.mk @@ -19,6 +19,12 @@ ifeq ($(BR2_PACKAGE_BLUEZ_UTILS),y) OPENOBEX_DEPENDENCIES += bluez_utils endif +define OPENOBEX_EXTRACT_CMDS + $(UNZIP) -d $(@D) $(OPENOBEX_DL_DIR)/$(OPENOBEX_SOURCE) + mv $(@D)/openobex-$(OPENOBEX_VERSION)-Source/* $(@D) + $(RM) -r $(@D)/openobex-$(OPENOBEX_VERSION)-Source +endef + ifeq ($(BR2_PACKAGE_BLUEZ5_UTILS),y) OPENOBEX_DEPENDENCIES += bluez5_utils endif diff --git a/package/opentyrian-data/opentyrian-data.mk b/package/opentyrian-data/opentyrian-data.mk index 4b879df1c3..9e38b61bfa 100644 --- a/package/opentyrian-data/opentyrian-data.mk +++ b/package/opentyrian-data/opentyrian-data.mk @@ -10,7 +10,7 @@ OPENTYRIAN_DATA_SOURCE = tyrian21.zip OPENTYRIAN_DATA_LICENSE = Freeware define OPENTYRIAN_DATA_EXTRACT_CMDS - $(UNZIP) -d $(@D) $(DL_DIR)/$(OPENTYRIAN_DATA_SOURCE) + $(UNZIP) -d $(@D) $(OPENTYRIAN_DATA_DL_DIR)/$(OPENTYRIAN_DATA_SOURCE) endef define OPENTYRIAN_DATA_INSTALL_TARGET_CMDS diff --git a/package/perl/perl.mk b/package/perl/perl.mk index 58bf3eb6ae..c47367b9da 100644 --- a/package/perl/perl.mk +++ b/package/perl/perl.mk @@ -24,7 +24,7 @@ PERL_EXTRA_DOWNLOADS = $(PERL_CROSS_SITE)/$(PERL_CROSS_SOURCE) # as a separate package. Instead, it is downloaded and extracted # together with perl define PERL_CROSS_EXTRACT - $(call suitable-extractor,$(PERL_CROSS_SOURCE)) $(DL_DIR)/$(PERL_CROSS_SOURCE) | \ + $(call suitable-extractor,$(PERL_CROSS_SOURCE)) $(PERL_DL_DIR)/$(PERL_CROSS_SOURCE) | \ $(TAR) --strip-components=1 -C $(@D) $(TAR_OPTIONS) - endef PERL_POST_EXTRACT_HOOKS += PERL_CROSS_EXTRACT diff --git a/package/python-keyring/python-keyring.mk b/package/python-keyring/python-keyring.mk index 27db2802ed..8ac18ae052 100644 --- a/package/python-keyring/python-keyring.mk +++ b/package/python-keyring/python-keyring.mk @@ -6,7 +6,6 @@ PYTHON_KEYRING_VERSION = 10.5.0 PYTHON_KEYRING_SOURCE = keyring-$(PYTHON_KEYRING_VERSION).tar.gz -PYTHON_KEYRING_SITE = https://pypi.python.org/packages/42/2e/51bd1739fe335095a2174db3f2f230346762e7e572471059540146a521f6 PYTHON_KEYRING_SETUP_TYPE = setuptools PYTHON_KEYRING_LICENSE = MIT PYTHON_KEYRING_LICENSE_FILES = LICENSE diff --git a/package/python-pytz/python-pytz.mk b/package/python-pytz/python-pytz.mk index 6e130a6b2f..1d3603389b 100644 --- a/package/python-pytz/python-pytz.mk +++ b/package/python-pytz/python-pytz.mk @@ -12,7 +12,7 @@ PYTHON_PYTZ_LICENSE = MIT PYTHON_PYTZ_LICENSE_FILES = LICENSE.txt define PYTHON_PYTZ_EXTRACT_CMDS - unzip $(DL_DIR)/$(PYTHON_PYTZ_SOURCE) -d $(@D) + unzip $(PYTHON_PYTZ_DL_DIR)/$(PYTHON_PYTZ_SOURCE) -d $(@D) mv $(@D)/pytz-$(PYTHON_PYTZ_VERSION)/* $(@D) rmdir $(@D)/pytz-$(PYTHON_PYTZ_VERSION) endef diff --git a/package/python-simplegeneric/python-simplegeneric.mk b/package/python-simplegeneric/python-simplegeneric.mk index cc84320428..d3bfa69a7e 100644 --- a/package/python-simplegeneric/python-simplegeneric.mk +++ b/package/python-simplegeneric/python-simplegeneric.mk @@ -16,7 +16,7 @@ PYTHON_SIMPLEGENERIC_LICENSE = ZPL-2.1 PYTHON_SIMPLEGENERIC_SETUP_TYPE = setuptools define PYTHON_SIMPLEGENERIC_EXTRACT_CMDS - unzip $(DL_DIR)/$(PYTHON_SIMPLEGENERIC_SOURCE) -d $(@D) + unzip $(PYTHON_SIMPLEGENERIC_DL_DIR)/$(PYTHON_SIMPLEGENERIC_SOURCE) -d $(@D) mv $(@D)/simplegeneric-$(PYTHON_SIMPLEGENERIC_VERSION)/* $(@D) rmdir $(@D)/simplegeneric-$(PYTHON_SIMPLEGENERIC_VERSION) endef diff --git a/package/rapidxml/rapidxml.mk b/package/rapidxml/rapidxml.mk index 2bec8fe1f2..9d034d8205 100644 --- a/package/rapidxml/rapidxml.mk +++ b/package/rapidxml/rapidxml.mk @@ -15,7 +15,7 @@ RAPIDXML_INSTALL_TARGET = NO RAPIDXML_INSTALL_STAGING = YES define RAPIDXML_EXTRACT_CMDS - $(UNZIP) -d $(@D) $(DL_DIR)/$(RAPIDXML_SOURCE) + $(UNZIP) -d $(@D) $(RAPIDXML_DL_DIR)/$(RAPIDXML_SOURCE) mv $(@D)/rapidxml-$(RAPIDXML_VERSION)/* $(@D)/ rmdir $(@D)/rapidxml-$(RAPIDXML_VERSION) endef diff --git a/package/rpi-wifi-firmware/rpi-wifi-firmware.mk b/package/rpi-wifi-firmware/rpi-wifi-firmware.mk index 6c855a8e14..83473d5bb9 100644 --- a/package/rpi-wifi-firmware/rpi-wifi-firmware.mk +++ b/package/rpi-wifi-firmware/rpi-wifi-firmware.mk @@ -12,7 +12,7 @@ RPI_WIFI_FIRMWARE_SITE = https://raw.githubusercontent.com/RPi-Distro/firmware-n RPI_WIFI_FIRMWARE_LICENSE = PROPRIETARY define RPI_WIFI_FIRMWARE_EXTRACT_CMDS - cp $(DL_DIR)/$($(PKG)_SOURCE) $(@D)/ + cp $(RPI_WIFI_FIRMWARE_DL_DIR)/$($(PKG)_SOURCE) $(@D)/ endef define RPI_WIFI_FIRMWARE_INSTALL_TARGET_CMDS diff --git a/package/rust-bin/rust-bin.mk b/package/rust-bin/rust-bin.mk index cd5844b115..759d468304 100644 --- a/package/rust-bin/rust-bin.mk +++ b/package/rust-bin/rust-bin.mk @@ -22,7 +22,7 @@ HOST_RUST_BIN_LIBSTD_HOST_PREFIX = rust-std-$(RUST_BIN_VERSION)-$(RUSTC_HOST_NAM define HOST_RUST_BIN_LIBSTD_EXTRACT mkdir -p $(@D)/std $(foreach f,$(HOST_RUST_BIN_EXTRA_DOWNLOADS), \ - $(call suitable-extractor,$(f)) $(DL_DIR)/$(f) | \ + $(call suitable-extractor,$(f)) $(RUST_BIN_DL_DIR)/$(f) | \ $(TAR) -C $(@D)/std $(TAR_OPTIONS) - ) cd $(@D)/rustc/lib/rustlib; \ diff --git a/package/sam-ba/sam-ba.mk b/package/sam-ba/sam-ba.mk index a7941459c7..acc03a3b6b 100644 --- a/package/sam-ba/sam-ba.mk +++ b/package/sam-ba/sam-ba.mk @@ -13,7 +13,7 @@ SAM_BA_LICENSE_FILES = doc/license.txt tcl_lib/boards.tcl \ applets/sam4c/libraries/libchip_sam4c/include/sam4c/sam4c32e-1.h define HOST_SAM_BA_EXTRACT_CMDS - $(UNZIP) -d $(BUILD_DIR) $(DL_DIR)/$(SAM_BA_SOURCE) + $(UNZIP) -d $(BUILD_DIR) $(SAM_BA_DL_DIR)/$(SAM_BA_SOURCE) mv $(BUILD_DIR)/sam-ba_cdc_linux/* $(@D) rmdir $(BUILD_DIR)/sam-ba_cdc_linux/ endef diff --git a/package/spidev_test/spidev_test.mk b/package/spidev_test/spidev_test.mk index 1d657803b4..bf8170cd2c 100644 --- a/package/spidev_test/spidev_test.mk +++ b/package/spidev_test/spidev_test.mk @@ -32,7 +32,7 @@ endef SPIDEV_TEST_POST_PATCH_HOOKS += SPIDEV_ADD_LINUX_IOCTL define SPIDEV_TEST_EXTRACT_CMDS - cp $(DL_DIR)/$(SPIDEV_TEST_SOURCE) $(@D)/spidev_test.c + cp $(SPIDEV_TEST_DL_DIR)/$(SPIDEV_TEST_SOURCE) $(@D)/spidev_test.c endef define SPIDEV_TEST_BUILD_CMDS diff --git a/package/tar/tar.mk b/package/tar/tar.mk index 92b6e9eaa5..9942e77737 100644 --- a/package/tar/tar.mk +++ b/package/tar/tar.mk @@ -40,7 +40,7 @@ HOST_TAR_SOURCE = tar-$(TAR_VERSION).cpio.gz define HOST_TAR_EXTRACT_CMDS mkdir -p $(@D) cd $(@D) && \ - $(call suitable-extractor,$(HOST_TAR_SOURCE)) $(DL_DIR)/$(HOST_TAR_SOURCE) | cpio -i --preserve-modification-time + $(call suitable-extractor,$(HOST_TAR_SOURCE)) $(TAR_DL_DIR)/$(HOST_TAR_SOURCE) | cpio -i --preserve-modification-time mv $(@D)/tar-$(TAR_VERSION)/* $(@D) rmdir $(@D)/tar-$(TAR_VERSION) endef diff --git a/package/tesseract-ocr/tesseract-ocr.mk b/package/tesseract-ocr/tesseract-ocr.mk index 9e315b239a..919a4360ed 100644 --- a/package/tesseract-ocr/tesseract-ocr.mk +++ b/package/tesseract-ocr/tesseract-ocr.mk @@ -62,7 +62,7 @@ TESSERACT_OCR_PRE_CONFIGURE_HOOKS += TESSERACT_OCR_PRECONFIGURE # Language data files installation define TESSERACT_OCR_INSTALL_LANG_DATA $(foreach langfile,$(TESSERACT_OCR_DATA_FILES), \ - $(INSTALL) -D -m 0644 $(DL_DIR)/$(langfile) \ + $(INSTALL) -D -m 0644 $(TESSERACT_OCR_DL_DIR)/$(langfile) \ $(TARGET_DIR)/usr/share/tessdata/$(langfile) ) endef diff --git a/package/ti-cgt-pru/ti-cgt-pru.mk b/package/ti-cgt-pru/ti-cgt-pru.mk index 0b9a64b3c9..df3b97bef5 100644 --- a/package/ti-cgt-pru/ti-cgt-pru.mk +++ b/package/ti-cgt-pru/ti-cgt-pru.mk @@ -13,8 +13,8 @@ TI_CGT_PRU_LICENSE_FILES = PRU_Code_Generation_Tools_2.2.x_manifest.html \ pru_rts_2_2_0_82167478-F8C9-49b2-82BD-12F8550770F9.spdx define HOST_TI_CGT_PRU_EXTRACT_CMDS - chmod +x $(DL_DIR)/$(TI_CGT_PRU_SOURCE) - $(DL_DIR)/$(TI_CGT_PRU_SOURCE) --prefix $(@D) --mode unattended + chmod +x $(TI_CGT_PRU_DL_DIR)/$(TI_CGT_PRU_SOURCE) + $(TI_CGT_PRU_DL_DIR)/$(TI_CGT_PRU_SOURCE) --prefix $(@D) --mode unattended mv $(@D)/ti-cgt-pru_$(TI_CGT_PRU_VERSION)/* $(@D) rmdir $(@D)/ti-cgt-pru_$(TI_CGT_PRU_VERSION)/ endef diff --git a/package/ti-gfx/ti-gfx.mk b/package/ti-gfx/ti-gfx.mk index 428878a08b..9fad553780 100644 --- a/package/ti-gfx/ti-gfx.mk +++ b/package/ti-gfx/ti-gfx.mk @@ -94,8 +94,8 @@ TI_GFX_HDR_DIRS = OGLES2/EGL OGLES2/EWS OGLES2/GLES2 OGLES2/KHR \ OGLES/GLES bufferclass_ti/ pvr2d/ wsegl/ define TI_GFX_EXTRACT_CMDS - chmod +x $(DL_DIR)/$(TI_GFX_SOURCE) - printf "Y\nY\n qY\n\n" | $(DL_DIR)/$(TI_GFX_SOURCE) \ + chmod +x $(TI_GFX_DL_DIR)/$(TI_GFX_SOURCE) + printf "Y\nY\n qY\n\n" | $(TI_GFX_DL_DIR)/$(TI_GFX_SOURCE) \ --prefix $(@D) \ --mode console endef diff --git a/package/ts4900-fpga/ts4900-fpga.mk b/package/ts4900-fpga/ts4900-fpga.mk index ed951b8c8f..7bb62a3984 100644 --- a/package/ts4900-fpga/ts4900-fpga.mk +++ b/package/ts4900-fpga/ts4900-fpga.mk @@ -11,7 +11,7 @@ TS4900_FPGA_SITE = ftp://ftp.embeddedarm.com/ts-socket-macrocontrollers/ts-4900- # https://github.com/embeddedarm/meta-ts/blob/f31860f1204b64f765a5380d3b93a2cf18234f90/recipes-extras/ts4900-fpga/ts4900-fpga.bb#L6 define TS4900_FPGA_EXTRACT_CMDS - cp $(DL_DIR)/$(TS4900_FPGA_SOURCE) $(@D) + cp $(TS4900_FPGA_DL_DIR)/$(TS4900_FPGA_SOURCE) $(@D) endef define TS4900_FPGA_INSTALL_TARGET_CMDS diff --git a/package/unscd/unscd.mk b/package/unscd/unscd.mk index f0eb5d6ad6..36f2de271b 100644 --- a/package/unscd/unscd.mk +++ b/package/unscd/unscd.mk @@ -11,7 +11,7 @@ UNSCD_LICENSE = GPL-2.0 UNSCD_LICENSE_FILES = $(UNSCD_SOURCE) define UNSCD_EXTRACT_CMDS - cp $(DL_DIR)/$($(PKG)_SOURCE) $(@D)/ + cp $(UNSCD_DL_DIR)/$($(PKG)_SOURCE) $(@D)/ endef define UNSCD_BUILD_CMDS diff --git a/package/urg/urg.mk b/package/urg/urg.mk index 966627fe36..1346079893 100644 --- a/package/urg/urg.mk +++ b/package/urg/urg.mk @@ -25,7 +25,9 @@ endif URG_CONFIG_SCRIPTS = c_urg-config urg-config define URG_EXTRACT_CMDS - $(UNZIP) -d $(BUILD_DIR) $(DL_DIR)/$(URG_SOURCE) + $(UNZIP) -d $(BUILD_DIR) $(URG_DL_DIR)/$(URG_SOURCE) + test -d $(URG_DIR) || \ + mv $(BUILD_DIR)/$(subst .zip,,$(URG_SOURCE)) $(URG_DIR) endef $(eval $(autotools-package)) diff --git a/package/waf/waf.mk b/package/waf/waf.mk index e7ac891b39..639c26a3b0 100644 --- a/package/waf/waf.mk +++ b/package/waf/waf.mk @@ -9,7 +9,7 @@ WAF_SOURCE = waf-$(WAF_VERSION) WAF_SITE = https://waf.io/ define HOST_WAF_EXTRACT_CMDS - $(INSTALL) -D -m 0755 $(DL_DIR)/waf-$(WAF_VERSION) $(@D)/waf + $(INSTALL) -D -m 0755 $(WAF_DL_DIR)/waf-$(WAF_VERSION) $(@D)/waf endef define HOST_WAF_INSTALL_CMDS diff --git a/package/whetstone/whetstone.mk b/package/whetstone/whetstone.mk index d9b45638c5..3b6ec3419b 100644 --- a/package/whetstone/whetstone.mk +++ b/package/whetstone/whetstone.mk @@ -9,7 +9,7 @@ WHETSTONE_SOURCE = whetstone.c WHETSTONE_SITE = http://www.netlib.org/benchmark define WHETSTONE_EXTRACT_CMDS - cp $(DL_DIR)/$($(PKG)_SOURCE) $(@D)/ + cp $(WHETSTONE_DL_DIR)/$($(PKG)_SOURCE) $(@D)/ endef define WHETSTONE_BUILD_CMDS diff --git a/package/wilc1000-firmware/wilc1000-firmware.mk b/package/wilc1000-firmware/wilc1000-firmware.mk index 5dcf19045b..6f504d67a0 100644 --- a/package/wilc1000-firmware/wilc1000-firmware.mk +++ b/package/wilc1000-firmware/wilc1000-firmware.mk @@ -11,7 +11,7 @@ WILC1000_FIRMWARE_SOURCE = v$(WILC1000_FIRMWARE_VERSION)_Firmware.zip WILC1000_FIRMWARE_LICENSE = PROPRIETARY define WILC1000_FIRMWARE_EXTRACT_CMDS - $(UNZIP) -d $(BUILD_DIR) $(DL_DIR)/$(WILC1000_FIRMWARE_SOURCE) + $(UNZIP) -d $(BUILD_DIR) $(WILC1000_FIRMWARE_DL_DIR)/$(WILC1000_FIRMWARE_SOURCE) mv $(BUILD_DIR)/wireless-firmware-$(WILC1000_FIRMWARE_VERSION)_Firmware/* $(@D) rmdir $(BUILD_DIR)/wireless-firmware-$(WILC1000_FIRMWARE_VERSION)_Firmware endef diff --git a/package/zynq-boot-bin/zynq-boot-bin.mk b/package/zynq-boot-bin/zynq-boot-bin.mk index 3648c63654..c523e04f45 100644 --- a/package/zynq-boot-bin/zynq-boot-bin.mk +++ b/package/zynq-boot-bin/zynq-boot-bin.mk @@ -12,7 +12,7 @@ ZYNQ_BOOT_BIN_LICENSE = GPL-3.0+ HOST_ZYNQ_BOOT_BIN_DEPENDENCIES = host-python define HOST_ZYNQ_BOOT_BIN_EXTRACT_CMDS - cp $(DL_DIR)/$(ZYNQ_BOOT_BIN_SOURCE) $(@D) + cp $(ZYNQ_BOOT_BIN_DL_DIR)/$(ZYNQ_BOOT_BIN_SOURCE) $(@D) endef define HOST_ZYNQ_BOOT_BIN_INSTALL_CMDS From patchwork Mon Apr 2 08:14:25 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxime Hadjinlian X-Patchwork-Id: 894119 Return-Path: X-Original-To: incoming-buildroot@patchwork.ozlabs.org Delivered-To: patchwork-incoming-buildroot@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=busybox.net (client-ip=140.211.166.133; helo=hemlock.osuosl.org; envelope-from=buildroot-bounces@busybox.net; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="tVYgrfOI"; dkim-atps=neutral Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 40F4lH6WcSz9s28 for ; Mon, 2 Apr 2018 18:14:51 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id 6F6888A317; Mon, 2 Apr 2018 08:14:46 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id qEcWTEpQl+z5; Mon, 2 Apr 2018 08:14:44 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by hemlock.osuosl.org (Postfix) with ESMTP id 9DBE48A045; Mon, 2 Apr 2018 08:14:43 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by ash.osuosl.org (Postfix) with ESMTP id 3A4181C0B66 for ; Mon, 2 Apr 2018 08:14:42 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by whitealder.osuosl.org (Postfix) with ESMTP id 37A5788DF1 for ; Mon, 2 Apr 2018 08:14:42 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from whitealder.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id oRv7vgPDCrVd for ; Mon, 2 Apr 2018 08:14:41 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mail-wr0-f195.google.com (mail-wr0-f195.google.com [209.85.128.195]) by whitealder.osuosl.org (Postfix) with ESMTPS id 7B31688E16 for ; Mon, 2 Apr 2018 08:14:41 +0000 (UTC) Received: by mail-wr0-f195.google.com with SMTP id c24so12666815wrc.6 for ; Mon, 02 Apr 2018 01:14:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=eg+z8tnjcXCctzsOL+y0poPn2y3Of8Cs01InuZxmUaI=; b=tVYgrfOIliHsCRqOUPxnnmBmpl+KFH5aitpo8rHsVSon7Y7HsNlyTY4K/Pm2mIxErr +FYj7u2db35QU+lEnlKqEcB9yIjkWXCXprQViMhhc371EouYVWnie0+02KKJ+VspthUf JsWWurFNmm0zcKFM1IB1rKN8zlSOwMrvZK4A3wGs2JcFTWtBf3WNlUHRbIuh+gGLYjUF HE43NBw9XSNuacqzx5lU1vGvX1RS74Y9Ov+0ONWY7/5EOwtXJCfi0+ahNKt8/CibDHlv THrVuX4+F99jOAecwWzwDX8xhIujRUjFBiS/BAGfnuxdcjH1xWmWrzl3NbW7qqVmlGhM 3uTQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=eg+z8tnjcXCctzsOL+y0poPn2y3Of8Cs01InuZxmUaI=; b=Y2CiHTwnaJzcWl6YgtTx3AgUv+x2xX6/WecpTMTSISKSRVXhS+U1V/FYPO5MtZVHAw XgWEpYKHdeVCrPLSB//tk4Uxjj1r5Zib4YOW7NFSrhXV+2Bg3wMoUFvbAos+qrr0NDXZ CULODUDoE/eJNpSq6bk01EZV2jUx24kn5cMRkClQw4gcHYcNqvseZPQSBf413RY7ML/2 IQK5SIeSn8wiwFkyolQEH4l8XH6X2/M92VwM61gRg8X3aDrTIZjgM+ve+vJOLBqGLgaq 25E4rTWCABXJAZZLihZvLKR826Rqf9p1V3GEbl3V0CQPcDYQQUPXCUVr8EFjqc5L9WfR Hwew== X-Gm-Message-State: AElRT7HTt0V9N5cNxSnKNIDaCLK/l6LBfMBwzHYhdoYZoZgBwJjnd/Cy 5Flr0/Wivz3B+1pq1G50dMKbdyhr X-Google-Smtp-Source: AIpwx49Mo+Sk40RbDhHUieTCum5xZORyHwaonAHydQP4ARdU59N65amd3eC7WMFp4c6jymxWGvSygQ== X-Received: by 10.223.154.47 with SMTP id z44mr5808544wrb.239.1522656879746; Mon, 02 Apr 2018 01:14:39 -0700 (PDT) Received: from cerise.online.net ([195.154.228.195]) by smtp.gmail.com with ESMTPSA id a205sm51981wmf.18.2018.04.02.01.14.38 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 02 Apr 2018 01:14:39 -0700 (PDT) From: Maxime Hadjinlian To: buildroot@buildroot.org Date: Mon, 2 Apr 2018 10:14:25 +0200 Message-Id: <20180402081434.4411-4-maxime.hadjinlian@gmail.com> X-Mailer: git-send-email 2.16.2 In-Reply-To: <20180402081434.4411-1-maxime.hadjinlian@gmail.com> References: <20180402081434.4411-1-maxime.hadjinlian@gmail.com> Subject: [Buildroot] [v4 04/13] arc/xtensa: store the eXtensa overlay in the per-package DL_DIR X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.24 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: "Yann E. MORIN" , Maxime Hadjinlian MIME-Version: 1.0 Errors-To: buildroot-bounces@busybox.net Sender: "buildroot" From: "Yann E. MORIN" Signed-off-by: "Yann E. MORIN" Cc: Maxime Hadjinlian --- arch/arch.mk.xtensa | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/arch.mk.xtensa b/arch/arch.mk.xtensa index 2b6cd26d03..fd410f6bfa 100644 --- a/arch/arch.mk.xtensa +++ b/arch/arch.mk.xtensa @@ -12,7 +12,7 @@ BR_ARCH_XTENSA_OVERLAY_FILE = $(call qstrip,$(BR2_XTENSA_OVERLAY_FILE)) ifneq ($(filter http://% https://% ftp://% scp://%,$(BR_ARCH_XTENSA_OVERLAY_FILE)),) ARCH_XTENSA_OVERLAY_URL = $(BR_ARCH_XTENSA_OVERLAY_FILE) -ARCH_XTENSA_OVERLAY_FILE = $(DL_DIR)/$(notdir $(BR_ARCH_XTENSA_OVERLAY_FILE)) +ARCH_XTENSA_OVERLAY_FILE = $($(PKG)_DL_DIR)/$(notdir $(BR_ARCH_XTENSA_OVERLAY_FILE)) # Do not check that file, we can't know its hash BR_NO_CHECK_HASH_FOR += $(notdir $(ARCH_XTENSA_OVERLAY_URL)) else From patchwork Mon Apr 2 08:14:26 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxime Hadjinlian X-Patchwork-Id: 894118 Return-Path: X-Original-To: incoming-buildroot@patchwork.ozlabs.org Delivered-To: patchwork-incoming-buildroot@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=busybox.net (client-ip=140.211.166.137; helo=fraxinus.osuosl.org; envelope-from=buildroot-bounces@busybox.net; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="s1WjSESf"; dkim-atps=neutral Received: from fraxinus.osuosl.org (smtp4.osuosl.org [140.211.166.137]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 40F4lF2vSVz9s28 for ; Mon, 2 Apr 2018 18:14:49 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by fraxinus.osuosl.org (Postfix) with ESMTP id CE914883AD; Mon, 2 Apr 2018 08:14:47 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from fraxinus.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id WWnPK1kzE1QG; Mon, 2 Apr 2018 08:14:46 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by fraxinus.osuosl.org (Postfix) with ESMTP id 4B97B883B3; Mon, 2 Apr 2018 08:14:46 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from fraxinus.osuosl.org (smtp4.osuosl.org [140.211.166.137]) by ash.osuosl.org (Postfix) with ESMTP id 7AEFC1C0B66 for ; Mon, 2 Apr 2018 08:14:44 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by fraxinus.osuosl.org (Postfix) with ESMTP id 77FB487C49 for ; Mon, 2 Apr 2018 08:14:44 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from fraxinus.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id IpNfC2Hac8OA for ; Mon, 2 Apr 2018 08:14:43 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mail-wr0-f196.google.com (mail-wr0-f196.google.com [209.85.128.196]) by fraxinus.osuosl.org (Postfix) with ESMTPS id E70D787C23 for ; Mon, 2 Apr 2018 08:14:42 +0000 (UTC) Received: by mail-wr0-f196.google.com with SMTP id 80so12690500wrb.2 for ; Mon, 02 Apr 2018 01:14:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=69m6NLLN08qB2SlBhkkYdmS2k5gJzhMwNGxixdvgP84=; b=s1WjSESfidL+ZcGoJllp+xbdii3gy9a6oI2eXoCBqHTZL00RLKZkEHU3Cdce7KFBEj P1knNZMvJitLONTldN98bDbjdcbDExjsj4ebd6//kDfPjRe6ukNeOeK8BmiFuQQ3YPAV 3tIUguoEIhBwOXoeZR3KQYdcpR4bZmGvS2mVtiI4Oy3zSZ8PWqAYwa6j7x9U58p2OSo8 vVEqnR1eg+BehTZrYTAC0MoCQSBqnkJUIEBnZ5lSQ1AzjhXgo8SB0n6VIkgPdpsxsSXK 9VT47ptLwN4XrnuwYE0PvE4CLVr4gYy4oMY0FRqkZGv2YIrUAquGR5Lb+zen17hdvBGT +F0Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=69m6NLLN08qB2SlBhkkYdmS2k5gJzhMwNGxixdvgP84=; b=HR1C5O6JD5LEbmgDrg9ib0D0l7QlrclAShahcdMNI70FtpA+14FtCo/Lm+HQ6Ew2G2 5v5fnvlygmS4NS4FoAJuPLsgI7OOz43xEa3xEvQJ2GJGHh56rnH34WWZF9GeUZts1exU 0dygrd7IYivxvj8NqWyte6sTbZ6RRG8uWsWNbjmeTCvDcP1oZycDfxPLOwE5dTsZcWSN Wudy1nL7tvi8gLBwvatgaMd6isqIIseedWYpqcqysEyz2Txweq/4mBnC8G1JhcPEVYJR 6d/QK0J6tcP0mfuSjQbIeCKU7X/4x92SXZQLaQ+Gh1sXTAbqatMLgLmag3Ot+KlwkqiH YIlw== X-Gm-Message-State: AElRT7FlMX/vP4+c3v8BLvjjTK945oiCpXz4KYQ8CHZ5otUcxK4vW9SY +EM9W4AF/3j3CwCozCG9A4/1Wb8F X-Google-Smtp-Source: AIpwx487Q2MQ2WhCZLMADW7BsTqu2mEHVwRE6UEkRDx2WeLobIpNgCj1LODjf9ob0eomPbGbPtgU7A== X-Received: by 10.223.177.213 with SMTP id r21mr6232855wra.89.1522656880959; Mon, 02 Apr 2018 01:14:40 -0700 (PDT) Received: from cerise.online.net ([195.154.228.195]) by smtp.gmail.com with ESMTPSA id a205sm51981wmf.18.2018.04.02.01.14.39 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 02 Apr 2018 01:14:39 -0700 (PDT) From: Maxime Hadjinlian To: buildroot@buildroot.org Date: Mon, 2 Apr 2018 10:14:26 +0200 Message-Id: <20180402081434.4411-5-maxime.hadjinlian@gmail.com> X-Mailer: git-send-email 2.16.2 In-Reply-To: <20180402081434.4411-1-maxime.hadjinlian@gmail.com> References: <20180402081434.4411-1-maxime.hadjinlian@gmail.com> Subject: [Buildroot] [v4 05/13] pkg-{download, generic}: use new $($(PKG)_DL_DIR) X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.24 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Maxime Hadjinlian MIME-Version: 1.0 Errors-To: buildroot-bounces@busybox.net Sender: "buildroot" Let the infrastructure use the new variable $(PKG)_DL_DIR Signed-off-by: Maxime Hadjinlian Reviewed-by: "Yann E. MORIN" --- package/pkg-download.mk | 2 +- package/pkg-generic.mk | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/package/pkg-download.mk b/package/pkg-download.mk index 2ee745fddd..151b2e5423 100644 --- a/package/pkg-download.mk +++ b/package/pkg-download.mk @@ -95,7 +95,7 @@ define DOWNLOAD -H $(PKGDIR)/$($(PKG)_RAWNAME).hash \ -n $($(PKG)_BASENAME_RAW) \ -N $($(PKG)_RAWNAME) \ - -o $(DL_DIR)/$(notdir $(1)) \ + -o $($(PKG)_DL_DIR)/$(notdir $(1)) \ $(if $($(PKG)_GIT_SUBMODULES),-r) \ $(DOWNLOAD_URIS) \ $(QUIET) \ diff --git a/package/pkg-generic.mk b/package/pkg-generic.mk index 25a34e6ca2..e7f4a0402d 100644 --- a/package/pkg-generic.mk +++ b/package/pkg-generic.mk @@ -613,7 +613,7 @@ $(2)_TARGET_DIRCLEAN = $$($(2)_DIR)/.stamp_dircleaned # default extract command $(2)_EXTRACT_CMDS ?= \ - $$(if $$($(2)_SOURCE),$$(INFLATE$$(suffix $$($(2)_SOURCE))) $$(DL_DIR)/$$($(2)_SOURCE) | \ + $$(if $$($(2)_SOURCE),$$(INFLATE$$(suffix $$($(2)_SOURCE))) $$($(2)_DL_DIR)/$$($(2)_SOURCE) | \ $$(TAR) --strip-components=$$($(2)_STRIP_COMPONENTS) \ -C $$($(2)_DIR) \ $$(foreach x,$$($(2)_EXCLUDES),--exclude='$$(x)' ) \ From patchwork Mon Apr 2 08:14:27 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxime Hadjinlian X-Patchwork-Id: 894120 Return-Path: X-Original-To: incoming-buildroot@patchwork.ozlabs.org Delivered-To: patchwork-incoming-buildroot@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=busybox.net (client-ip=140.211.166.137; helo=fraxinus.osuosl.org; envelope-from=buildroot-bounces@busybox.net; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="MnQLoPw8"; dkim-atps=neutral Received: from fraxinus.osuosl.org (smtp4.osuosl.org [140.211.166.137]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 40F4lL0wQ9z9s28 for ; Mon, 2 Apr 2018 18:14:54 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by fraxinus.osuosl.org (Postfix) with ESMTP id 0580A881F5; Mon, 2 Apr 2018 08:14:50 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from fraxinus.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Dhg5deOugdG2; Mon, 2 Apr 2018 08:14:49 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by fraxinus.osuosl.org (Postfix) with ESMTP id 86088883B3; Mon, 2 Apr 2018 08:14:49 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by ash.osuosl.org (Postfix) with ESMTP id EDBAB1C0B66 for ; Mon, 2 Apr 2018 08:14:44 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by whitealder.osuosl.org (Postfix) with ESMTP id EA77688DD4 for ; Mon, 2 Apr 2018 08:14:44 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from whitealder.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 4zQbgjvUVNWq for ; Mon, 2 Apr 2018 08:14:43 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mail-wr0-f196.google.com (mail-wr0-f196.google.com [209.85.128.196]) by whitealder.osuosl.org (Postfix) with ESMTPS id 5F03A88DF1 for ; Mon, 2 Apr 2018 08:14:43 +0000 (UTC) Received: by mail-wr0-f196.google.com with SMTP id y55so12661692wry.3 for ; Mon, 02 Apr 2018 01:14:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=3M7Ejc5RK01w/EAj9H/XAOghdF1+3FrNuPjMMFqv8Yo=; b=MnQLoPw8GNZfVOq28LB+3DexFVA9jf4bJFanVjPqfxqT8/4FP9Yp3X5oPeYj2DQGSt itb+F6NPgqSKqfes5C4vRoF/BG93ZRKcbnvpaqk07Fp2luMSxZn4jfBmsAQJnIrOio/a qwv4OqhsfLG90jBRBDkdR4Cxczw0h1heW2pvBLHI13t7/2mUftXFht5ahpMxKefj6vFG Ycfc42cQmcP24NWHTJFks8H+ZgijUJLyVbhXLCMLZHUd+PtsGZLuTj641uDm8QDS4ip1 DoWRhx66Du+HyBYFywkh6E7AoA3reePBDBXM0vy5RsoYe89nCQwzHF9tiVz/jl+asdxH yceA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=3M7Ejc5RK01w/EAj9H/XAOghdF1+3FrNuPjMMFqv8Yo=; b=C9eVdf2I51mIPYl77SaV6nnRGkLBV45IfUYFhkEiyS8vDMfftPqbJ1bhbbNC7q1Aje ue3ZxzbS2l06ICKFD2BIhUetW1M9NrvFOO9avFLuiwuF6IAG50GTqiOE69x3yct2ARmi iy5zbtoe5iikgErB9NadNYJCVlUcPX42jCPpMEBEAUY483Rb4kix7bNKiPjSREu2WfO3 FldjTOEmP8U6DU79HUiXm1wHi5QqPDiv4VlFZ5Q1SVNm8oBAWNLTdc3+h1A0CergU0jD Ye9/EE/kcUQLvxgAtfUZ+b4pP9XlTN/mu2U/UH6pc9cbooFClSYBNDIoDsl1YzvHqjOv jw0Q== X-Gm-Message-State: AElRT7FzC8PIAS5t94aHELt6BF7RFJe99zGhmiWQQ997xuPv/hWTZeC3 qXoL8O6DuW+BPtVsm0kSUtUux0d/ X-Google-Smtp-Source: AIpwx48vUByp+Dz/DaaldIbQN5tSriq0ViT5uqUEPm6J9KwyDW6wW4vpOuKrM+IW5vgIRg1BJH5Eug== X-Received: by 10.223.182.13 with SMTP id f13mr5746340wre.262.1522656881595; Mon, 02 Apr 2018 01:14:41 -0700 (PDT) Received: from cerise.online.net ([195.154.228.195]) by smtp.gmail.com with ESMTPSA id a205sm51981wmf.18.2018.04.02.01.14.40 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 02 Apr 2018 01:14:41 -0700 (PDT) From: Maxime Hadjinlian To: buildroot@buildroot.org Date: Mon, 2 Apr 2018 10:14:27 +0200 Message-Id: <20180402081434.4411-6-maxime.hadjinlian@gmail.com> X-Mailer: git-send-email 2.16.2 In-Reply-To: <20180402081434.4411-1-maxime.hadjinlian@gmail.com> References: <20180402081434.4411-1-maxime.hadjinlian@gmail.com> Subject: [Buildroot] [v4 06/13] support/download: make sure the download folder is created X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.24 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Maxime Hadjinlian MIME-Version: 1.0 Errors-To: buildroot-bounces@busybox.net Sender: "buildroot" At the moment, it means that we make sure that BR2_DL_DIR is created, in the future, it will make sure that BR2_DL_DIR/PKG_NAME/ is created. Signed-off-by: Maxime Hadjinlian --- package/pkg-download.mk | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/package/pkg-download.mk b/package/pkg-download.mk index 151b2e5423..e88d3f9fda 100644 --- a/package/pkg-download.mk +++ b/package/pkg-download.mk @@ -88,7 +88,8 @@ endif endif define DOWNLOAD - $(Q)$(if $(filter bzr cvs hg svn,$($(PKG)_SITE_METHOD)),BR_NO_CHECK_HASH_FOR=$(notdir $(1));) \ + $(Q)mkdir -p $($(PKG)_DL_DIR)/ + $(Q)$(if $(filter bzr cvs hg svn,$($(PKG)_SITE_METHOD)),BR_NO_CHECK_HASH_FOR=$(notdir $(1))) \ $(EXTRA_ENV) $(DL_WRAPPER) \ -c $($(PKG)_DL_VERSION) \ -f $(notdir $(1)) \ From patchwork Mon Apr 2 08:14:28 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxime Hadjinlian X-Patchwork-Id: 894123 Return-Path: X-Original-To: incoming-buildroot@patchwork.ozlabs.org Delivered-To: patchwork-incoming-buildroot@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=busybox.net (client-ip=140.211.166.137; helo=fraxinus.osuosl.org; envelope-from=buildroot-bounces@busybox.net; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="OkOTsYpj"; dkim-atps=neutral Received: from fraxinus.osuosl.org (smtp4.osuosl.org [140.211.166.137]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 40F4lP5tVHz9s28 for ; Mon, 2 Apr 2018 18:14:57 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by fraxinus.osuosl.org (Postfix) with ESMTP id 566A888283; Mon, 2 Apr 2018 08:14:52 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from fraxinus.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id mVMT2QSvCk-y; Mon, 2 Apr 2018 08:14:51 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by fraxinus.osuosl.org (Postfix) with ESMTP id A845B8844A; Mon, 2 Apr 2018 08:14:51 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by ash.osuosl.org (Postfix) with ESMTP id 782C21C0B66 for ; Mon, 2 Apr 2018 08:14:47 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id 71FB48A2D5 for ; Mon, 2 Apr 2018 08:14:47 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id xdAEFYNzjv0W for ; Mon, 2 Apr 2018 08:14:45 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mail-wm0-f67.google.com (mail-wm0-f67.google.com [74.125.82.67]) by hemlock.osuosl.org (Postfix) with ESMTPS id 76C608A2DC for ; Mon, 2 Apr 2018 08:14:44 +0000 (UTC) Received: by mail-wm0-f67.google.com with SMTP id x82so25399709wmg.1 for ; Mon, 02 Apr 2018 01:14:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=3kAC1mXBCZ2L7pjWwqhVxntwN74AKmjv+S4+VNWG1r0=; b=OkOTsYpjJdcmLHJczAKKDlCHAvns+vJCZveKAZN9dRBjHRTRqxZE/Dl/kApEF+GrQn aL98rYdGbDScbr9bvG30cEwMbEMWNw6WJj6DnvGd7Orb+Uqfr/TfmmkFqQ4wY5Oe5Sd+ ffUckXhsc2blvE4N53WDJS+WUWSKmGt4s8cREuEPschQWjM5yaUTCyXThnS0Ixtgy1pu z8wDRT6SkEeZgR5wjmhZo34cavfT/cWMclKEl0JM9Sfxk9mg6jhR5+IgtRGB5/pjMS4O T5isUZ/ZWxmKZbL4MyvvTiAPFh0XRQ4yHlafAWTYkl613tpqfdtwW+Jl53HKWD7ZcWwF Pm4A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=3kAC1mXBCZ2L7pjWwqhVxntwN74AKmjv+S4+VNWG1r0=; b=ikhxRKRBzZuh0R+kqPtq4iUn7esf/epqXhWF4V2nQEh/UmJVJCUaQmm8NRtWEGY+xk GXr4oeN4Re3KcetXkELmgnUHgq/HzASE8yoVfcXsX4sTa0ISzEqg4jOOMzvK/lyacSYL VrUM4/H54pLsPyp75vIma4SqWEtziqpP2jsBE3zm8qt0Z2bhsoyWPjDaZcAsQjtecPP7 81VfPDKv+2KkiN2zHY5Cz5NUkGjzIat/4Kk9kzzNQFAEFASkqYUHPmebgkUjs1Bu22kv E1FY8xIBsZBzZ2jYk4Ycf775WRkXZkmaxLB2uNEvm+vQ4EFfYY8nIzcH3KiAKXswBI7y iy5w== X-Gm-Message-State: ALQs6tDD+gufH8A3kCHnK+QKVqpDp68X/3EN5G3e6G9dHJXyisMvnTeJ 4zQKZ0AuMd+v0JX2iRxT5n/v+Rwz X-Google-Smtp-Source: AIpwx49zQxlT43I66l1SnRA85Axy3dNL7bEMTX+Ns8Kmhzns/pIxh42yHdiVtYi1nfj202VSrzk0HQ== X-Received: by 10.28.112.11 with SMTP id l11mr128083wmc.31.1522656882501; Mon, 02 Apr 2018 01:14:42 -0700 (PDT) Received: from cerise.online.net ([195.154.228.195]) by smtp.gmail.com with ESMTPSA id a205sm51981wmf.18.2018.04.02.01.14.41 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 02 Apr 2018 01:14:41 -0700 (PDT) From: Maxime Hadjinlian To: buildroot@buildroot.org Date: Mon, 2 Apr 2018 10:14:28 +0200 Message-Id: <20180402081434.4411-7-maxime.hadjinlian@gmail.com> X-Mailer: git-send-email 2.16.2 In-Reply-To: <20180402081434.4411-1-maxime.hadjinlian@gmail.com> References: <20180402081434.4411-1-maxime.hadjinlian@gmail.com> Subject: [Buildroot] [v4 07/13] pkg-generic: add a subdirectory to the DL_DIR X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.24 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Maxime Hadjinlian MIME-Version: 1.0 Errors-To: buildroot-bounces@busybox.net Sender: "buildroot" With all the previous changes, we are now ready to add a subdirectory to the DL_DIR. The structure will now be DL_DIR/PKG_NAME/{FILE1,FILE2} This is needed for multiple reasons: - Avoid patches with name like SHA1.patch laying flat in DL_DIR, which makes it hard to know to which packages they apply - Avoid possible collisions if two releases have the same name (e.g: v01.tar) - Allow the possibility to handle a git cache per package in the newly created subdirectory. Signed-off-by: Maxime Hadjinlian Reviewed-by: "Yann E. MORIN" --- package/pkg-generic.mk | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/package/pkg-generic.mk b/package/pkg-generic.mk index e7f4a0402d..c84b1eb4a2 100644 --- a/package/pkg-generic.mk +++ b/package/pkg-generic.mk @@ -432,7 +432,7 @@ endif $(2)_BASENAME = $$(if $$($(2)_VERSION),$(1)-$$($(2)_VERSION),$(1)) $(2)_BASENAME_RAW = $$(if $$($(2)_VERSION),$$($(2)_RAWNAME)-$$($(2)_VERSION),$$($(2)_RAWNAME)) -$(2)_DL_DIR = $$(DL_DIR) +$(2)_DL_DIR = $$(DL_DIR)/$$($(2)_RAWNAME) $(2)_DIR = $$(BUILD_DIR)/$$($(2)_BASENAME) ifndef $(2)_SUBDIR From patchwork Mon Apr 2 08:14:29 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxime Hadjinlian X-Patchwork-Id: 894122 Return-Path: X-Original-To: incoming-buildroot@patchwork.ozlabs.org Delivered-To: patchwork-incoming-buildroot@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=busybox.net (client-ip=140.211.166.138; helo=whitealder.osuosl.org; envelope-from=buildroot-bounces@busybox.net; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="Y2nkFo4e"; dkim-atps=neutral Received: from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 40F4lL4vHRz9s2B for ; Mon, 2 Apr 2018 18:14:54 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by whitealder.osuosl.org (Postfix) with ESMTP id 7AD1289016; Mon, 2 Apr 2018 08:14:51 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from whitealder.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ZHCi690IwGOX; Mon, 2 Apr 2018 08:14:50 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by whitealder.osuosl.org (Postfix) with ESMTP id 41CA288E16; Mon, 2 Apr 2018 08:14:50 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from fraxinus.osuosl.org (smtp4.osuosl.org [140.211.166.137]) by ash.osuosl.org (Postfix) with ESMTP id 9F9191C0B66 for ; Mon, 2 Apr 2018 08:14:46 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by fraxinus.osuosl.org (Postfix) with ESMTP id 9BE4E87F26 for ; Mon, 2 Apr 2018 08:14:46 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from fraxinus.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id SdBOXSL3E1AI for ; Mon, 2 Apr 2018 08:14:45 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mail-wm0-f65.google.com (mail-wm0-f65.google.com [74.125.82.65]) by fraxinus.osuosl.org (Postfix) with ESMTPS id 545DD8838C for ; Mon, 2 Apr 2018 08:14:45 +0000 (UTC) Received: by mail-wm0-f65.google.com with SMTP id v21so23846360wmc.1 for ; Mon, 02 Apr 2018 01:14:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=a0o6fH7VixaQ7L+J7u3bN/UihqG8QcyjKErYmFi2mUE=; b=Y2nkFo4euMr4lXPW1BNPfu5iiVEaoQgCUGNlkFnIjp0KzimW2W0zyh9Yz8CZ4t6dvH xV5WISjYJtIzyHixt95SRaG0b/Vl7Us6MrZGZlrJBb1Pfv/Jo/qxj+06zkFFP86AKkFX fdWePJ0htJQ3caVt1QNsao1ixhFNwv4zvE4SEPVOUTm6J8J6SPK0Ld1ct+4tfDK63Mda vO5Ps8a1k+xfyXRkGGnUl8WAX8HTGstgIhBK1a48MKKNaHFepWtjte8mlndl4LWna5vf VexEsl5mY/2KKhLX7N0DfamHbtnjmcjOzYU0+2JOOrrJVK4CJoHkltXvazsrGqFgJVoL U7pg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=a0o6fH7VixaQ7L+J7u3bN/UihqG8QcyjKErYmFi2mUE=; b=Dr7WT2umO1u/FGcmQGtnn13k6AoeplP72EIEBDJsjMXz8GPKkZAgxQXXl2mq4+FCbZ Wvx27Dow6UW2s1thOTe+cydG6tFVUqto/Z/RsvFf0GNU2wCdKPSY9JgC8E913r/kWr3U ypRA++38UUm1bMqPjhZV3fZim6jX1/AJN+t3Ya6wqXos143DnzmfO3Bcmhxc0210zHfk yXwkGXYdpLW8sO3a2MEYl6D46Pjop2RUIpQDGqFuE0mGuTVGyt5Onch70vC3dOpzNP8o EFSMUi92t6i2n176A2IVlLlm3kzbOrK2oBxXUObZ512I3skEZtus8uLMmnBAwH/eJm3O c3XA== X-Gm-Message-State: AElRT7EeBVK/YSOB5Jf1W+2DDupg+WbzzN9TfGJT1jPthor0WHnDeAUF cF1oqsRRI65udXcebIlipzENMa1T X-Google-Smtp-Source: AIpwx48ETcXu0WViZrtPs8Ds6Tt97J+B3xXpUskfmutSbQmaLf9LaatAQrt038MvpbVZQLJyLDFaeg== X-Received: by 10.28.125.194 with SMTP id y185mr153806wmc.160.1522656883444; Mon, 02 Apr 2018 01:14:43 -0700 (PDT) Received: from cerise.online.net ([195.154.228.195]) by smtp.gmail.com with ESMTPSA id a205sm51981wmf.18.2018.04.02.01.14.42 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 02 Apr 2018 01:14:42 -0700 (PDT) From: Maxime Hadjinlian To: buildroot@buildroot.org Date: Mon, 2 Apr 2018 10:14:29 +0200 Message-Id: <20180402081434.4411-8-maxime.hadjinlian@gmail.com> X-Mailer: git-send-email 2.16.2 In-Reply-To: <20180402081434.4411-1-maxime.hadjinlian@gmail.com> References: <20180402081434.4411-1-maxime.hadjinlian@gmail.com> Subject: [Buildroot] [v4 08/13] pkg-download: support new subdir for mirrors X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.24 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Maxime Hadjinlian MIME-Version: 1.0 Errors-To: buildroot-bounces@busybox.net Sender: "buildroot" Since we introduced subdirectory to the DL_DIR, we need to support them in the PRIMARY and BACKUP mirror as they evolve to the new tree structure. We check first the new URI (with the subdir), and in case of failure, we go check without. By checking both URIs, we ensure that old mirror are usable. Signed-off-by: Maxime Hadjinlian Reviewed-by: "Yann E. MORIN" --- package/pkg-download.mk | 2 ++ 1 file changed, 2 insertions(+) diff --git a/package/pkg-download.mk b/package/pkg-download.mk index e88d3f9fda..54edbae29a 100644 --- a/package/pkg-download.mk +++ b/package/pkg-download.mk @@ -75,6 +75,7 @@ export BR_NO_CHECK_HASH_FOR = ifneq ($(call qstrip,$(BR2_PRIMARY_SITE)),) DOWNLOAD_URIS += \ + -u $(call getschemeplusuri,$(BR2_PRIMARY_SITE)/$(notdir $($(PKG)_DL_DIR)),urlencode) \ -u $(call getschemeplusuri,$(BR2_PRIMARY_SITE),urlencode) endif @@ -83,6 +84,7 @@ DOWNLOAD_URIS += \ -u $($(PKG)_SITE_METHOD)+$(dir $(1)) ifneq ($(call qstrip,$(BR2_BACKUP_SITE)),) DOWNLOAD_URIS += \ + -u $(call getschemeplusuri,$(BR2_BACKUP_SITE)/$(notdir $($(PKG)_DL_DIR)),urlencode) \ -u $(call getschemeplusuri,$(BR2_BACKUP_SITE),urlencode) endif endif From patchwork Mon Apr 2 08:14:30 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxime Hadjinlian X-Patchwork-Id: 894121 Return-Path: X-Original-To: incoming-buildroot@patchwork.ozlabs.org Delivered-To: patchwork-incoming-buildroot@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=busybox.net (client-ip=140.211.166.136; helo=silver.osuosl.org; envelope-from=buildroot-bounces@busybox.net; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="Ao3ccASk"; dkim-atps=neutral Received: from silver.osuosl.org (smtp3.osuosl.org [140.211.166.136]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 40F4lL2R6Fz9s29 for ; Mon, 2 Apr 2018 18:14:54 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by silver.osuosl.org (Postfix) with ESMTP id F0C5B227E9; Mon, 2 Apr 2018 08:14:51 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from silver.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id iEIX0eZbbfWk; Mon, 2 Apr 2018 08:14:51 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by silver.osuosl.org (Postfix) with ESMTP id 00D5129DD7; Mon, 2 Apr 2018 08:14:51 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by ash.osuosl.org (Postfix) with ESMTP id F12961C0B66 for ; Mon, 2 Apr 2018 08:14:46 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by whitealder.osuosl.org (Postfix) with ESMTP id ED6A888DF1 for ; Mon, 2 Apr 2018 08:14:46 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from whitealder.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id TO89H96w8TjE for ; Mon, 2 Apr 2018 08:14:46 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mail-wm0-f67.google.com (mail-wm0-f67.google.com [74.125.82.67]) by whitealder.osuosl.org (Postfix) with ESMTPS id 3256088DD4 for ; Mon, 2 Apr 2018 08:14:46 +0000 (UTC) Received: by mail-wm0-f67.google.com with SMTP id t67so3907322wmt.0 for ; Mon, 02 Apr 2018 01:14:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=T4JoTW1ufh8VmfBVHfZrE1XVYcAe9P47+URixlrurHE=; b=Ao3ccASk4D+ebpHA9+k97tCkzZnRdrFq2o8RmkyAmJylYYP0sRFV4kLss9HE/dl5sm k73G9k0i174xICWsimIf9h4Z2KF3tQO2FGtPTy1JekHF5t0lA3rHBA+DAdTUIrKXgwor ZKr6fSInINfhSyzLUC1UPpbekUgxjmNjP0nijBF9NuRSEkCT9zLeSEHW7RFuUSEc8sBg Ibx47IistT8onf3GaE784yGMKw323zeRtX4Twsf0ojiQoJuwf6rlHJmmSOYowpqvy81Q +zT2GjgEmAiCPZOrIQmw1D0nB4+THcgbuK93N6MQT1pgY/1enFxg0Cq8FUnxCoHkgsEI 0aYg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=T4JoTW1ufh8VmfBVHfZrE1XVYcAe9P47+URixlrurHE=; b=k1bSEqBNfp+aFWSKPkWqa/8v4qZnLUGNxdhbgQpZohxCusIcZGzD3mP2PY7FJBemer AEKgRdUwDhBHPLJFeHqOyp49tL9WgYX0YmrOk2uNm56p1fOmCfU5efZeeLUkEJXrVhe7 U7cG9tXOWs44iXm/93v/A3e2odgC9ghjAlm9Ue6XvKBIljLtiCxwKLYTiBsO7e1CDvsI OS/5FZJeywfU5TjubGF7MZ4quHrgwzdueOX98T+hAOyaXMJfily9Eya5791/+M3mNP5l lzQNNUEykkNJgyBECey6DPEnYH/bevpVEr8xXkAp5yGdoJDnVRuhDmGkQ9IOhfREBXjL Fzaw== X-Gm-Message-State: ALQs6tA9nps3eUMsuOUQqeVW6/mgjfYlKFtJlcX3ZnSkQMUFbxWU8L2J WrvMHwrsh8b/m4IJMgP7lNLh0EUG X-Google-Smtp-Source: AIpwx49GL/ho5pO5o+gA9+GQADKDT+bj9FzTJdW8Ao8LM+NUZQ56Ie3/d968Hd6uAJr8vQqNEFg8xw== X-Received: by 10.28.174.79 with SMTP id x76mr144171wme.41.1522656884374; Mon, 02 Apr 2018 01:14:44 -0700 (PDT) Received: from cerise.online.net ([195.154.228.195]) by smtp.gmail.com with ESMTPSA id a205sm51981wmf.18.2018.04.02.01.14.43 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 02 Apr 2018 01:14:43 -0700 (PDT) From: Maxime Hadjinlian To: buildroot@buildroot.org Date: Mon, 2 Apr 2018 10:14:30 +0200 Message-Id: <20180402081434.4411-9-maxime.hadjinlian@gmail.com> X-Mailer: git-send-email 2.16.2 In-Reply-To: <20180402081434.4411-1-maxime.hadjinlian@gmail.com> References: <20180402081434.4411-1-maxime.hadjinlian@gmail.com> Subject: [Buildroot] [v4 09/13] pkg-generic: introduce _SAME_SOURCE_AS X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.24 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Maxime Hadjinlian MIME-Version: 1.0 Errors-To: buildroot-bounces@busybox.net Sender: "buildroot" This per package variable can be used to specify that a package shares the same sources as another package. The use case here is for example linux-headers and linux, which share the same sources (because they are the same upstream project), so we don't want to download twice the kernel, nor store it multiple times either. Make will automatically try to help by introducing leading and trailing spaces when replacing a line-continuation '\', so we need to call $(strip) on the variable. Signed-off-by: Maxime Hadjinlian Reviewed-by: "Yann E. MORIN" --- package/pkg-generic.mk | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/package/pkg-generic.mk b/package/pkg-generic.mk index c84b1eb4a2..a034c7ea02 100644 --- a/package/pkg-generic.mk +++ b/package/pkg-generic.mk @@ -432,7 +432,10 @@ endif $(2)_BASENAME = $$(if $$($(2)_VERSION),$(1)-$$($(2)_VERSION),$(1)) $(2)_BASENAME_RAW = $$(if $$($(2)_VERSION),$$($(2)_RAWNAME)-$$($(2)_VERSION),$$($(2)_RAWNAME)) -$(2)_DL_DIR = $$(DL_DIR)/$$($(2)_RAWNAME) +$(2)_DL_DIR = $$(strip $$(if $$($(2)_SAME_SOURCE_AS), \ + $$($$(call UPPERCASE,$$($(2)_SAME_SOURCE_AS))_DL_DIR), \ + $$(DL_DIR)/$$($(2)_RAWNAME)) \ + ) $(2)_DIR = $$(BUILD_DIR)/$$($(2)_BASENAME) ifndef $(2)_SUBDIR From patchwork Mon Apr 2 08:14:31 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxime Hadjinlian X-Patchwork-Id: 894124 Return-Path: X-Original-To: incoming-buildroot@patchwork.ozlabs.org Delivered-To: patchwork-incoming-buildroot@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=busybox.net (client-ip=140.211.166.136; helo=silver.osuosl.org; envelope-from=buildroot-bounces@busybox.net; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="BwGTD06+"; dkim-atps=neutral Received: from silver.osuosl.org (smtp3.osuosl.org [140.211.166.136]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 40F4lQ4wt6z9s29 for ; Mon, 2 Apr 2018 18:14:58 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by silver.osuosl.org (Postfix) with ESMTP id E7AF12DCA3; Mon, 2 Apr 2018 08:14:56 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from silver.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id hkwQVs90hvYA; Mon, 2 Apr 2018 08:14:52 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by silver.osuosl.org (Postfix) with ESMTP id 5636E2DCD3; Mon, 2 Apr 2018 08:14:52 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from silver.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by ash.osuosl.org (Postfix) with ESMTP id EFF7E1C0B66 for ; Mon, 2 Apr 2018 08:14:47 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by silver.osuosl.org (Postfix) with ESMTP id EC45229DD7 for ; Mon, 2 Apr 2018 08:14:47 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from silver.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id jfFbn2jey65W for ; Mon, 2 Apr 2018 08:14:47 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mail-wm0-f67.google.com (mail-wm0-f67.google.com [74.125.82.67]) by silver.osuosl.org (Postfix) with ESMTPS id 1E2CD227E9 for ; Mon, 2 Apr 2018 08:14:47 +0000 (UTC) Received: by mail-wm0-f67.google.com with SMTP id r82so25573361wme.0 for ; Mon, 02 Apr 2018 01:14:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=PCnZPZWBy+U2jMxIzfzRKBMBsGRCEfNFPCBtpBDwgA0=; b=BwGTD06+NOo7kJZ6rOrKX25MXofp+Lkr+34fqDqdBuIxvr9FQ0CIvb5r/lqgaDH3ka KIJo8my5Mney3/xdpHs1rPUQeDV3Lwpwzf6BkGv/CNeJXOBTN46GvWAgq1hRTZnFwRyD 43pUBng3uEr3oCBMRloXeVyi17Zla4sIfojF70zmVJoetgK7awdARP+mbqhR1wlWHSlI d34Xw4N6zJZ9cYYeoGvTN+xtnbND7uPVSoZgpqq2XZMIDR29ocTveQiS8rj9Obyxkeeb z5WoEdH6UXVLJFAOt0r80mdiJZbwx3nXKl7nsvvV4yS4jQCVtkTPk66sR2oB39fIbM/B mBAw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=PCnZPZWBy+U2jMxIzfzRKBMBsGRCEfNFPCBtpBDwgA0=; b=W04abZlw/tvvAIWPaK2KtZu+Fpuxo138U/AYQnijo10QgXFTYVV2NbQecwwMzGAU4h KFQ+9r84MyRIkVcMMiiHMBKXCyJmuj8OuU8mr7DjGpqhXGlW615XIXeYNTrJgObbNFGt dHJcTA3XNZNR+7yLFKwTYO4PkSiFo4a7Fa388ZZX7o3Aq4OxsoL5x5VcAejiLv03Lt9W yIrixJ+lfIeWlvE1xbi7qLqsDXXY1ByduPyCfV1AqeDdgTq6UILfOddoEO71iZEOqlUd 9xUEd1f4Jh90wXZmGpLd5bBgXcmO4adWsdBa4fkXUVUOYfNXsl3TfwWgLfaNBJCeB/C3 azRw== X-Gm-Message-State: AElRT7HzmBWM5ektQdWjtDn1WWx6W5QRJH7CbbH5HX0luaOInBeBssNV sd32DdvB7WtXZFfUR3hhauCMrVo+ X-Google-Smtp-Source: AIpwx4/hZeQwRD9QJw4AZSLuZAjpY8BQf2P/aJZT7+ZpA02YM2+IjRAVM7u9IMZTaPbxVf0GAdrzVw== X-Received: by 10.28.228.131 with SMTP id b125mr113433wmh.153.1522656885284; Mon, 02 Apr 2018 01:14:45 -0700 (PDT) Received: from cerise.online.net ([195.154.228.195]) by smtp.gmail.com with ESMTPSA id a205sm51981wmf.18.2018.04.02.01.14.44 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 02 Apr 2018 01:14:44 -0700 (PDT) From: Maxime Hadjinlian To: buildroot@buildroot.org Date: Mon, 2 Apr 2018 10:14:31 +0200 Message-Id: <20180402081434.4411-10-maxime.hadjinlian@gmail.com> X-Mailer: git-send-email 2.16.2 In-Reply-To: <20180402081434.4411-1-maxime.hadjinlian@gmail.com> References: <20180402081434.4411-1-maxime.hadjinlian@gmail.com> Subject: [Buildroot] [v4 10/13] package: share downloaded files for big packages X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.24 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Thomas Petazzoni , "Yann E. MORIN" , Maxime Hadjinlian MIME-Version: 1.0 Errors-To: buildroot-bounces@busybox.net Sender: "buildroot" From: "Yann E. MORIN" Theses packages are given as an example of the use of the *_SAME_SOURCE_AS feature. There maybe other packages that would benefit from that feature, they would need to be added on a case by case basis. Signed-off-by: "Yann E. MORIN" Signed-off-by: Maxime Hadjinlian Cc: Maxime Hadjinlian Cc: Thomas Petazzoni --- package/gcc/gcc-initial/gcc-initial.mk | 5 +++++ package/linux-headers/linux-headers.mk | 3 +++ package/mesa3d-headers/mesa3d-headers.mk | 1 + 3 files changed, 9 insertions(+) diff --git a/package/gcc/gcc-initial/gcc-initial.mk b/package/gcc/gcc-initial/gcc-initial.mk index c476b2faeb..07da50afa8 100644 --- a/package/gcc/gcc-initial/gcc-initial.mk +++ b/package/gcc/gcc-initial/gcc-initial.mk @@ -8,6 +8,11 @@ GCC_INITIAL_VERSION = $(GCC_VERSION) GCC_INITIAL_SITE = $(GCC_SITE) GCC_INITIAL_SOURCE = $(GCC_SOURCE) +# We do not have a 'gcc' package per-se; we only have two incarnations, +# gcc-initial and gcc-final. gcc-initial is just am internal step that +# users should not care about, while gcc-final is the one they shall see. +HOST_GCC_INITIAL_SAME_SOURCE_AS = host-gcc-final + HOST_GCC_INITIAL_DEPENDENCIES = $(HOST_GCC_COMMON_DEPENDENCIES) HOST_GCC_INITIAL_EXCLUDES = $(HOST_GCC_EXCLUDES) diff --git a/package/linux-headers/linux-headers.mk b/package/linux-headers/linux-headers.mk index f1e3790608..6f4071c61a 100644 --- a/package/linux-headers/linux-headers.mk +++ b/package/linux-headers/linux-headers.mk @@ -82,6 +82,9 @@ endif endif # ! BR2_KERNEL_HEADERS_AS_KERNEL +# linux-headers really is the same as the linux package +LINUX_HEADERS_SAME_SOURCE_AS = linux + LINUX_HEADERS_LICENSE = GPL-2.0 LINUX_HEADERS_LICENSE_FILES = COPYING diff --git a/package/mesa3d-headers/mesa3d-headers.mk b/package/mesa3d-headers/mesa3d-headers.mk index b48d965d8a..39a25270db 100644 --- a/package/mesa3d-headers/mesa3d-headers.mk +++ b/package/mesa3d-headers/mesa3d-headers.mk @@ -15,6 +15,7 @@ endif MESA3D_HEADERS_VERSION = 18.0.0 MESA3D_HEADERS_SOURCE = mesa-$(MESA3D_HEADERS_VERSION).tar.xz MESA3D_HEADERS_SITE = https://mesa.freedesktop.org/archive +MESA3D_HEADERS_SAME_SOURCE_AS = mesa3d MESA3D_HEADERS_LICENSE = MIT, SGI, Khronos MESA3D_HEADERS_LICENSE_FILES = docs/license.html From patchwork Mon Apr 2 08:14:32 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxime Hadjinlian X-Patchwork-Id: 894126 Return-Path: X-Original-To: incoming-buildroot@patchwork.ozlabs.org Delivered-To: patchwork-incoming-buildroot@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=busybox.net (client-ip=140.211.166.137; helo=fraxinus.osuosl.org; envelope-from=buildroot-bounces@busybox.net; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="B9TUvj9/"; dkim-atps=neutral Received: from fraxinus.osuosl.org (smtp4.osuosl.org [140.211.166.137]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 40F4lX0WhHz9s28 for ; Mon, 2 Apr 2018 18:15:03 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by fraxinus.osuosl.org (Postfix) with ESMTP id 38E0788496; Mon, 2 Apr 2018 08:14:55 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from fraxinus.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id rxOhOSwBtGDV; Mon, 2 Apr 2018 08:14:53 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by fraxinus.osuosl.org (Postfix) with ESMTP id 94F9B8844A; Mon, 2 Apr 2018 08:14:53 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by ash.osuosl.org (Postfix) with ESMTP id 735651C0B66 for ; Mon, 2 Apr 2018 08:14:49 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by whitealder.osuosl.org (Postfix) with ESMTP id 70EE488E16 for ; Mon, 2 Apr 2018 08:14:49 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from whitealder.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id wSP0jR6FDlyy for ; Mon, 2 Apr 2018 08:14:48 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mail-wm0-f65.google.com (mail-wm0-f65.google.com [74.125.82.65]) by whitealder.osuosl.org (Postfix) with ESMTPS id 0AF8888DD4 for ; Mon, 2 Apr 2018 08:14:48 +0000 (UTC) Received: by mail-wm0-f65.google.com with SMTP id v21so23846572wmc.1 for ; Mon, 02 Apr 2018 01:14:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=MSHN+G3a8UQQJBwAYb3t2B46nW7Ox95DmsaSUYjNJ0o=; b=B9TUvj9/3LpedBqwvbjlFv7ITj7BYMUEUlJKId45phDr2I5ZTeBYe2BceYo2c2yJQd ijLnGo0iTMUNLOgxoVo1lM2vi3JmXcBSj+VFm+joCPHcOVO0bTv+kkLOor403nM6w+lS pSS6O67EQhpc8gCBdtk2U8PgiofGpH9V7XIO3Q00GpKEgJDLAil+JDzmp0NOTrXlTfCK umDEaq01rqrfvFUESN00QrKYoPUk0SvAcpThc9LfAWZMp/o+qIzHol+aNH0MnytllclF GjrXmjwFw1Sz5U4cgq0L+DYwReHsESSfjcyscUdYIZO9AJ4/E6rWgpY8bpTMFluhfQ4u 6caw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=MSHN+G3a8UQQJBwAYb3t2B46nW7Ox95DmsaSUYjNJ0o=; b=D1njKfY8F4CZNUdzXulOPUABCJsFwnYkUW+f2TyyeTyCq19mmRlMewF5HGXoVzmmmL tkhTMxaXTbftBR/F21EQZ6flcgr1ot7pvKS/HTshWaMP5U8woDseOUZHk5PIsS/pSYr4 6WahRb6dQpXGuub2Qam6ZYRSkz+oEII1tr3uxFZEqrJzyOeozSCfzSb2VhrsrZlU9M1F g6fzuceWkrzqPKbJVuOPvQmeRLDbDISTuU7SqBuQccOfqTxx/DemuEH+9FLrdWlSPEhy Ju6G0DOZgpg1ErQ7D/QDXMC1G93A5BpVEqEHS/82a/wO7CXdCT7D6CM0ycJ61ylVbWR0 +8xQ== X-Gm-Message-State: AElRT7FbI8FDS8IdAI0fzQRc0rfRLYCN0N/K0QZgDFBpWPco87UafhpG fkPwVOz4dA9upH7SuThMY4qKmNGi X-Google-Smtp-Source: AIpwx4+kYoLdle1L4PZyOucpG76HS13dVpG9W4wjEmPKUwxNReiz/ibhlq2pi9wsI4DaP0BCtIZG2w== X-Received: by 10.28.54.154 with SMTP id y26mr142963wmh.102.1522656886145; Mon, 02 Apr 2018 01:14:46 -0700 (PDT) Received: from cerise.online.net ([195.154.228.195]) by smtp.gmail.com with ESMTPSA id a205sm51981wmf.18.2018.04.02.01.14.45 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 02 Apr 2018 01:14:45 -0700 (PDT) From: Maxime Hadjinlian To: buildroot@buildroot.org Date: Mon, 2 Apr 2018 10:14:32 +0200 Message-Id: <20180402081434.4411-11-maxime.hadjinlian@gmail.com> X-Mailer: git-send-email 2.16.2 In-Reply-To: <20180402081434.4411-1-maxime.hadjinlian@gmail.com> References: <20180402081434.4411-1-maxime.hadjinlian@gmail.com> Subject: [Buildroot] [v4 11/13] help/manual: update help about the new $(LIBFOO_DL_DIR) X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.24 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Maxime Hadjinlian MIME-Version: 1.0 Errors-To: buildroot-bounces@busybox.net Sender: "buildroot" Signed-off-by: Maxime Hadjinlian Reviewed-by: "Yann E. MORIN" --- Config.in | 3 +++ docs/manual/adding-packages-generic.txt | 6 +++--- 2 files changed, 6 insertions(+), 3 deletions(-) diff --git a/Config.in b/Config.in index 71b9f60e5b..c2a1925088 100644 --- a/Config.in +++ b/Config.in @@ -200,6 +200,9 @@ config BR2_DL_DIR If the Linux shell environment has defined the BR2_DL_DIR environment variable, then this overrides this configuration item. + The directory is organized with a subdirectory for each package. + Each package has its own $(LIBFOO_DL_DIR) variable that can be used + to find the correct path. The default is $(TOPDIR)/dl diff --git a/docs/manual/adding-packages-generic.txt b/docs/manual/adding-packages-generic.txt index 9d1428ad40..62906d92bb 100644 --- a/docs/manual/adding-packages-generic.txt +++ b/docs/manual/adding-packages-generic.txt @@ -263,7 +263,7 @@ information is (assuming the package name is +libfoo+) : the file using this URL. Otherwise, Buildroot will assume the file to be downloaded is located at +LIBFOO_SITE+. Buildroot will not do anything with those additional files, except download them: it will - be up to the package recipe to use them from +$(DL_DIR)+. + be up to the package recipe to use them from +$(LIBFOO_DL_DIR)+. * +LIBFOO_SITE_METHOD+ determines the method used to fetch or copy the package source code. In many cases, Buildroot guesses the method @@ -554,8 +554,8 @@ In the action definitions, you can use the following variables: * +$(@D)+, which contains the directory in which the package source code has been uncompressed. -* +$(DL_DIR)+ contains the path to the directory where all the downloads made - by Buildroot are stored. +* +$(LIBFOO_DL_DIR)+ contains the path to the directory where all the downloads + made by Buildroot for +libfoo+ are stored in. * +$(TARGET_CC)+, +$(TARGET_LD)+, etc. to get the target cross-compilation utilities From patchwork Mon Apr 2 08:14:33 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxime Hadjinlian X-Patchwork-Id: 894127 Return-Path: X-Original-To: incoming-buildroot@patchwork.ozlabs.org Delivered-To: patchwork-incoming-buildroot@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=busybox.net (client-ip=140.211.166.136; helo=silver.osuosl.org; envelope-from=buildroot-bounces@busybox.net; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="oh+G2nNh"; dkim-atps=neutral Received: from silver.osuosl.org (smtp3.osuosl.org [140.211.166.136]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 40F4lX2WQsz9s29 for ; Mon, 2 Apr 2018 18:15:04 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by silver.osuosl.org (Postfix) with ESMTP id 4ABDC2DF60; Mon, 2 Apr 2018 08:14:59 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from silver.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Jd190Flm0DBb; Mon, 2 Apr 2018 08:14:54 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by silver.osuosl.org (Postfix) with ESMTP id 56E972DB15; Mon, 2 Apr 2018 08:14:54 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by ash.osuosl.org (Postfix) with ESMTP id 6DB6D1C0B66 for ; Mon, 2 Apr 2018 08:14:52 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by whitealder.osuosl.org (Postfix) with ESMTP id 6866D89033 for ; Mon, 2 Apr 2018 08:14:52 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from whitealder.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Obq16O09-MM7 for ; Mon, 2 Apr 2018 08:14:49 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mail-wr0-f179.google.com (mail-wr0-f179.google.com [209.85.128.179]) by whitealder.osuosl.org (Postfix) with ESMTPS id F3E2188DF1 for ; Mon, 2 Apr 2018 08:14:48 +0000 (UTC) Received: by mail-wr0-f179.google.com with SMTP id l49so12680201wrl.4 for ; Mon, 02 Apr 2018 01:14:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=3XBtUHv6ghcqHXSQVWe1ka9hmxHLGjt5TNBy/TKGAtk=; b=oh+G2nNh3yJkJ9Fi7FNNoOtAw045s9JL4mpjrMhgePEGt87SGOkrzYbiRmY1EK4MHD L2SA0+DbvpcMVaoJ9PifgXlFDs0o06isZjdU+ySlS8th+M5uLwqba2I5VSADdmaqpFQe eRyX369KN8TuYXd+7/n7fzO7kl91YQIGpl/ep8LVVCJyqLakg/v323RlQn56I3lkv7gS KcavLmaeSkG1YLpVe9WSC3msp1W4nM/qi99ZVuZWqj+UAT/udXjRFJ34hNlWvuU7l6x/ xSxYC55SyBJGaFMdeFeX6ht2EE9Qs1oBR7BqlZ2qnSykh0oDHOGaYI2ATz1bRM9HRqQO esTw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=3XBtUHv6ghcqHXSQVWe1ka9hmxHLGjt5TNBy/TKGAtk=; b=gUcU8YKOMLq7y7UPMuZWigxvDW+q5RDNEPICX7PEF3uUqykj14HEj015FDRDON3lp3 8ai5fDHigVctdfD/kSJNqSa7e5hosnNCmcLMAipqgxQ/Z5omhd5nIv5OVBQXTtJGtuxG HTQVXMFOO2uX6gCFq+6Xf5D+elYau23SP1mIjE/zlLd57z+V4l6Ytmg2GvTZYsNwrbQ9 /NNDi4jyoHfOzwOOV8LXYC5ILebOQopZH4syvcw7WP1Km/1rV4zRL0BcqJqR2sQeQlFo uAmEjYI2W4A9ZfPx8I3ZBlFhwnBF2fViTzaWu3XzptF4U7386SKL8CpD83FT9YVZt860 cxAA== X-Gm-Message-State: AElRT7HzHM//OR3jPOQazSMkl3bKxDlKUAddWBW0/4jepR3rXumBb1zp 0Pf3ng4nUPiTeD8cU1lubI6aW/ex X-Google-Smtp-Source: AIpwx48yyRCpQYvC4O3nsAKNC043kUQoFptRd1USp2MVwPNP6FN8npu2+BUWfqTtpnw1diF4Xsm/AQ== X-Received: by 10.223.209.132 with SMTP id h4mr6127208wri.12.1522656887247; Mon, 02 Apr 2018 01:14:47 -0700 (PDT) Received: from cerise.online.net ([195.154.228.195]) by smtp.gmail.com with ESMTPSA id a205sm51981wmf.18.2018.04.02.01.14.46 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 02 Apr 2018 01:14:46 -0700 (PDT) From: Maxime Hadjinlian To: buildroot@buildroot.org Date: Mon, 2 Apr 2018 10:14:33 +0200 Message-Id: <20180402081434.4411-12-maxime.hadjinlian@gmail.com> X-Mailer: git-send-email 2.16.2 In-Reply-To: <20180402081434.4411-1-maxime.hadjinlian@gmail.com> References: <20180402081434.4411-1-maxime.hadjinlian@gmail.com> Subject: [Buildroot] [v4 12/13] download: add flock call before dl-wrapper X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.24 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Maxime Hadjinlian MIME-Version: 1.0 Errors-To: buildroot-bounces@busybox.net Sender: "buildroot" In order to introduce the cache mechanisms, we need to have a lock on the $(LIBFOO_DL_DIR), otherwise it would be impossible to do parallel download (a shared DL_DIR for two buildroot instances). To make sure the directory exists, the mkdir call has been removed from the dl-wrapper and put in the infrastructure. Signed-off-by: Maxime Hadjinlian Reviewed-by: "Yann E. MORIN" --- package/pkg-download.mk | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/package/pkg-download.mk b/package/pkg-download.mk index 54edbae29a..05842bd430 100644 --- a/package/pkg-download.mk +++ b/package/pkg-download.mk @@ -19,6 +19,7 @@ SSH := $(call qstrip,$(BR2_SSH)) export LOCALFILES := $(call qstrip,$(BR2_LOCALFILES)) DL_WRAPPER = support/download/dl-wrapper +FLOCK = flock $($(PKG)_DL_DIR)/ # DL_DIR may have been set already from the environment ifeq ($(origin DL_DIR),undefined) @@ -92,7 +93,7 @@ endif define DOWNLOAD $(Q)mkdir -p $($(PKG)_DL_DIR)/ $(Q)$(if $(filter bzr cvs hg svn,$($(PKG)_SITE_METHOD)),BR_NO_CHECK_HASH_FOR=$(notdir $(1))) \ - $(EXTRA_ENV) $(DL_WRAPPER) \ + $(EXTRA_ENV) $(FLOCK) $(DL_WRAPPER) \ -c $($(PKG)_DL_VERSION) \ -f $(notdir $(1)) \ -H $(PKGDIR)/$($(PKG)_RAWNAME).hash \ From patchwork Mon Apr 2 08:14:34 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxime Hadjinlian X-Patchwork-Id: 894128 Return-Path: X-Original-To: incoming-buildroot@patchwork.ozlabs.org Delivered-To: patchwork-incoming-buildroot@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=busybox.net (client-ip=140.211.166.137; helo=fraxinus.osuosl.org; envelope-from=buildroot-bounces@busybox.net; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="SkKGw9aT"; dkim-atps=neutral Received: from fraxinus.osuosl.org (smtp4.osuosl.org [140.211.166.137]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 40F4lZ42Nvz9s28 for ; Mon, 2 Apr 2018 18:15:06 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by fraxinus.osuosl.org (Postfix) with ESMTP id EFF138842C; Mon, 2 Apr 2018 08:15:03 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from fraxinus.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id D5dM4Bm8qdBe; Mon, 2 Apr 2018 08:15:02 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by fraxinus.osuosl.org (Postfix) with ESMTP id 897C1883F3; Mon, 2 Apr 2018 08:15:02 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by ash.osuosl.org (Postfix) with ESMTP id 470B41C0B66 for ; Mon, 2 Apr 2018 08:15:01 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id 41DAD8A2D5 for ; Mon, 2 Apr 2018 08:15:01 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 8TpyRCgWq6Pe for ; Mon, 2 Apr 2018 08:14:59 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mail-wr0-f195.google.com (mail-wr0-f195.google.com [209.85.128.195]) by hemlock.osuosl.org (Postfix) with ESMTPS id 52C6689F69 for ; Mon, 2 Apr 2018 08:14:50 +0000 (UTC) Received: by mail-wr0-f195.google.com with SMTP id s12so2054894wrc.8 for ; Mon, 02 Apr 2018 01:14:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=uEh5LE0RhwjZHWMtSJPau3fNhtd9kyqGgayxcS8rh2s=; b=SkKGw9aTV6KGiRd+IAh9Gr9kaaMVk9JFZ2JCL/8P4dkkfyPXqm1EhC/ptunXp1EmJ7 8FqaDIpMaVJRzxMHZiHZr+vv0WxPMC4mr9AcdxGlrEkqy3P+G5f6FeBifNrMu8wUAE4p MW0hhvdFn3QBiAynU3l9Nx24Y0oBKsojg3Na9ASFJvAKGJFmujGSxmekzwmzCRLCebzp PZf7ZWzK0erTX0Jxu0DavwIv0+p9GUa9Wvn/P/uarP/5HPoz35QNxO8YR7ufgPmyFzk2 LY98o/Nt3BjIH0iS+yT/CfW5LVN1XWcJmLVJ7+A6SYyZxYFphMKzLevIG8Jf1/Jr4awv da5w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=uEh5LE0RhwjZHWMtSJPau3fNhtd9kyqGgayxcS8rh2s=; b=jMvjK9fYF/pj6cs4TCCxA6GnhUnQjmjXlv/3YkY+hxYdNsZc/yaVi9+cFsFeZTthKq 8keTKbrAo3TXyw94naRrHUr3z110EScOSFcwluUspRDvPcMiPxbqMzzxxJkW6zU0vJuJ sRQbMziFiLGzl57r64Shjt96PTCYFIHiW7TsR0MuWf+lv4RrRrxYUkSu02B9CL/O4VMk 2JYamuClTr1efH1Z4umiu/C6b9ANu0QVTJ+QpYavVi9LW7hCIG4rXztNzfpNy4KF3Lje +2KcxxDCdpYkGpkjQ+GTiWhgx5vuWZcObBHDv3aXniRUY5UWpSldyAQA1IDoUJh6c1uY PWkg== X-Gm-Message-State: AElRT7E4I+izkAQ1OfvfnJffUvotceIbbT/yqlmp8E2CSYDzQI/qluAh BlhmRz2HxMOYMu5SQu4OZgFL4HOA X-Google-Smtp-Source: AIpwx4+m7r8dNrtYN3EIb7j2t3HQmZheQ7CNs8CZoww9YmebTdWYFV/ObRxn6GaZA0WReF+IQL2slQ== X-Received: by 10.223.224.200 with SMTP id e8mr5807487wri.149.1522656888419; Mon, 02 Apr 2018 01:14:48 -0700 (PDT) Received: from cerise.online.net ([195.154.228.195]) by smtp.gmail.com with ESMTPSA id a205sm51981wmf.18.2018.04.02.01.14.47 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 02 Apr 2018 01:14:47 -0700 (PDT) From: Maxime Hadjinlian To: buildroot@buildroot.org Date: Mon, 2 Apr 2018 10:14:34 +0200 Message-Id: <20180402081434.4411-13-maxime.hadjinlian@gmail.com> X-Mailer: git-send-email 2.16.2 In-Reply-To: <20180402081434.4411-1-maxime.hadjinlian@gmail.com> References: <20180402081434.4411-1-maxime.hadjinlian@gmail.com> Subject: [Buildroot] [v4 13/13] download: git: introduce cache feature X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.24 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Maxime Hadjinlian MIME-Version: 1.0 Errors-To: buildroot-bounces@busybox.net Sender: "buildroot" Now we keep the git clone that we download and generates our tarball from there. The main goal here is that if you change the version of a package (say Linux), instead of cloning all over again, you will simply 'git fetch' from the repo the missing objects, then generates the tarball again. This should speed the 'source' part of the build significantly. The drawback is that the DL_DIR will grow much larger; but time is more important than disk space nowadays. Signed-off-by: Maxime Hadjinlian --- support/download/git | 63 ++++++++++++++++++++++++++++++++-------------------- 1 file changed, 39 insertions(+), 24 deletions(-) diff --git a/support/download/git b/support/download/git index 58a2c6ad9d..c6b0f81f13 100755 --- a/support/download/git +++ b/support/download/git @@ -39,29 +39,40 @@ _git() { eval ${GIT} "${@}" } -# Try a shallow clone, since it is faster than a full clone - but that only -# works if the version is a ref (tag or branch). Before trying to do a shallow -# clone we check if ${cset} is in the list provided by git ls-remote. If not -# we fall back on a full clone. +# We want to check if a cache of the git clone of this repo already exists. +git_cache="${BR2_DL_DIR}/${basename%%-*}/git" + +# If the cache directory already exists, don't try to clone. +if [ ! -d "${git_cache}" ]; then + _git init "'${git_cache}'" + _git -C "'${git_cache}'" remote add origin "'${uri}'" +fi + +pushd "${git_cache}" >/dev/null + +_git remote set-url origin "'${uri}'" + +# Try to fetch with limited depth, since it is faster than a full clone - but +# that only works if the version is a ref (tag or branch). Before trying to do +# a shallow clone we check if ${cset} is in the list provided by git ls-remote. +# If not we fall back on a full fetch. # -# Messages for the type of clone used are provided to ease debugging in case of -# problems +# Messages for the type of clone used are provided to ease debugging in +# case of problems git_done=0 -if [ -n "$(_git ls-remote "'${uri}'" "'${cset}'" 2>&1)" ]; then - printf "Doing shallow clone\n" - if _git clone ${verbose} "${@}" --depth 1 -b "'${cset}'" "'${uri}'" "'${basename}'"; then +if [ -n "$(_git ls-remote origin "'${cset}'" 2>&1)" ]; then + printf "Doing fetch with limited depth\n" + if _git fetch "${@}" --depth 1 origin "'${cset}'"; then git_done=1 else - printf "Shallow clone failed, falling back to doing a full clone\n" + printf "Fetching ref failed, falling back to fetching all refs\n" fi fi if [ ${git_done} -eq 0 ]; then - printf "Doing full clone\n" - _git clone ${verbose} "${@}" "'${uri}'" "'${basename}'" + printf "Fetching all references\n" + _git fetch origin -t fi -pushd "${basename}" >/dev/null - # Try to get the special refs exposed by some forges (pull-requests for # github, changes for gerrit...). There is no easy way to know whether # the cset the user passed us is such a special ref or a tag or a sha1 @@ -86,20 +97,24 @@ if [ ${recurse} -eq 1 ]; then _git submodule update --init --recursive fi -# We do not want the .git dir; we keep other .git files, in case they -# are the only files in their directory. +# Generate the archive, sort with the C locale so that it is reproducible +# We do not want the .git dir; we keep other .git +# files, in case they are the only files in their directory. # The .git dir would generate non reproducible tarballs as it depends on # the state of the remote server. It also would generate large tarballs # (gigabytes for some linux trees) when a full clone took place. -rm -rf .git +find . -not -type d \ + -and -not -path "./.git/*" >"${output}.list" +LC_ALL=C sort <"${output}.list" >"${output}.list.sorted" -popd >/dev/null - -# Generate the archive, sort with the C locale so that it is reproducible -find "${basename}" -not -type d >"${basename}.list" -LC_ALL=C sort <"${basename}.list" >"${basename}.list.sorted" # Create GNU-format tarballs, since that's the format of the tarballs on # sources.buildroot.org and used in the *.hash files -tar cf - --numeric-owner --owner=0 --group=0 --mtime="${date}" --format=gnu \ - -T "${basename}.list.sorted" >"${output}.tar" +tar cf - --transform="s/^\.$/${basename}/" \ + --numeric-owner --owner=0 --group=0 --mtime="${date}" --format=gnu \ + -T "${output}.list.sorted" >"${output}.tar" gzip -6 -n <"${output}.tar" >"${output}" + +rm -f "${output}.list" +rm -f "${output}.list.sorted" + +popd >/dev/null