[PATCHv3] core/sdk: generate the SDK tarball ourselves

Message ID 20180804082741.10783-1-yann.morin.1998@free.fr
State Accepted
Headers show
Series
  • [PATCHv3] core/sdk: generate the SDK tarball ourselves
Related show

Commit Message

Yann E. MORIN Aug. 4, 2018, 8:27 a.m.
Currently, the wording in the manual instructs the user to generate a
tarball from "the contents of the +output/host+ directory".

This is pretty confusing, because taken literally, this would amount to
running a command like:

    tar cf my-sdk.tar -C output/host/ .

This creates a tarbomb [0], which is very bad practice, because when
extracted, it creates multiple files in the current directory.

What one really wants to do, is create a tarball of the host/ directory,
with something like:

    tar cf my-sdk.tar -C output host/

However, this is not much better, because the top-most directory would
have a very common name, host/, which is pretty easy to get conflict
with when it gets extracted.

So, we fix that mess by giving the top-most directory a recognisable
name, based on the target tuple, which we also use as the name of the
archive (suffixed with the usual +.tar.gz+.) We offer the user the
possibility to override that default by specifying the +BR2_SDK_PREFIX+
variable on the command line.

Since this is an output file, we place it in the images/ directory.

As some users expressed a very strong feeling that they do not want to
generate a tarball at all, and that doing so would badly hurt their
workflows [1], we actually prepare the SDK as was previously done, but
under the new, intermediate rule 'prepare-sdk'. The existing 'sdk' rule
obviously depend on that before generating the tarball.

We choose to make the existing rule to generate the tarball, and
introduce a new rule to just prepare the SDK, rather than keep the
existing rule as-is and introduce a new one to generate the tarball,
because it makes sense to have the simplest rule do the correct thing,
leaving advanced, power users use the longest command. If someone
already had a wrapper that called 'sdk' and expected just the host
directory to be prepared, then this is not broken; it just takes a bit
longer (gzip is pretty fast).

Update the manual accordingly.

[0] https://en.wikipedia.org/wiki/Tar_(computing)#Tarbomb
[1] http://lists.busybox.net/pipermail/buildroot/2018-June/thread.html#223377
    and some messages in the ensuing thread...

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Wolfgang Grandegger <wg@grandegger.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Stefan Becker <chemobejk@gmail.com>
Cc: Trent Piepho <tpiepho@impinj.com>

---
Changes v2 -> v3:
  - drop the version from the prefix  (Stefan, Thomas)
  - actually have 'sdk' depend on 'prepare-sdk'
  - anonymise the generated tarball

Changes v1 -> v2:
  - drop the part of the manual stating this can't be re-used as a
    pre-built toolchain  (Arnout)
  - make the top-level directory and tarball name configurable
    (Trent, Thomas)
  - change the default name  (Arnout)
  - allow just preparing the SDK (Stefan, Trent)
  - typoes  (Thomas)
---
 Makefile                                  | 15 +++++++++++++--
 docs/manual/using-buildroot-toolchain.txt | 26 +++++++++++++++++---------
 2 files changed, 30 insertions(+), 11 deletions(-)

Comments

Stefan Becker Aug. 5, 2018, 1:04 p.m. | #1
Hi Yann, All,

looks OK to me.

On Sat, Aug 4, 2018 at 11:27 AM Yann E. MORIN <yann.morin.1998@free.fr>
wrote:

> Currently, the wording in the manual instructs the user to generate a
> tarball from "the contents of the +output/host+ directory".
>
> This is pretty confusing, because taken literally, this would amount to
> running a command like:
>
>     tar cf my-sdk.tar -C output/host/ .
>
> This creates a tarbomb [0], which is very bad practice, because when
> extracted, it creates multiple files in the current directory.
>
> What one really wants to do, is create a tarball of the host/ directory,
> with something like:
>
>     tar cf my-sdk.tar -C output host/
>
> However, this is not much better, because the top-most directory would
> have a very common name, host/, which is pretty easy to get conflict
> with when it gets extracted.
>
> So, we fix that mess by giving the top-most directory a recognisable
> name, based on the target tuple, which we also use as the name of the
> archive (suffixed with the usual +.tar.gz+.) We offer the user the
> possibility to override that default by specifying the +BR2_SDK_PREFIX+
> variable on the command line.
>
> Since this is an output file, we place it in the images/ directory.
>
> As some users expressed a very strong feeling that they do not want to
> generate a tarball at all, and that doing so would badly hurt their
> workflows [1], we actually prepare the SDK as was previously done, but
> under the new, intermediate rule 'prepare-sdk'. The existing 'sdk' rule
> obviously depend on that before generating the tarball.
>
> We choose to make the existing rule to generate the tarball, and
> introduce a new rule to just prepare the SDK, rather than keep the
> existing rule as-is and introduce a new one to generate the tarball,
> because it makes sense to have the simplest rule do the correct thing,
> leaving advanced, power users use the longest command. If someone
> already had a wrapper that called 'sdk' and expected just the host
> directory to be prepared, then this is not broken; it just takes a bit
> longer (gzip is pretty fast).
>
> Update the manual accordingly.
>
> [0] https://en.wikipedia.org/wiki/Tar_(computing)#Tarbomb
> [1]
> http://lists.busybox.net/pipermail/buildroot/2018-June/thread.html#223377
>     and some messages in the ensuing thread...
>
> Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
> Cc: Wolfgang Grandegger <wg@grandegger.com>
> Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
> Cc: Arnout Vandecappelle <arnout@mind.be>
> Cc: Stefan Becker <chemobejk@gmail.com>
> Cc: Trent Piepho <tpiepho@impinj.com>
>
> ---
> Changes v2 -> v3:
>   - drop the version from the prefix  (Stefan, Thomas)
>   - actually have 'sdk' depend on 'prepare-sdk'
>   - anonymise the generated tarball
>
> Changes v1 -> v2:
>   - drop the part of the manual stating this can't be re-used as a
>     pre-built toolchain  (Arnout)
>   - make the top-level directory and tarball name configurable
>     (Trent, Thomas)
>   - change the default name  (Arnout)
>   - allow just preparing the SDK (Stefan, Trent)
>   - typoes  (Thomas)
> ---
>  Makefile                                  | 15 +++++++++++++--
>  docs/manual/using-buildroot-toolchain.txt | 26 +++++++++++++++++---------
>  2 files changed, 30 insertions(+), 11 deletions(-)
>
> diff --git a/Makefile b/Makefile
> index 37c7072c7c..e13e5eaefa 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -573,8 +573,8 @@ prepare: $(BUILD_DIR)/buildroot-config/auto.conf
>  .PHONY: world
>  world: target-post-image
>
> -.PHONY: sdk
> -sdk: world
> +.PHONY: prepare-sdk
> +prepare-sdk: world
>         @$(call MESSAGE,"Rendering the SDK relocatable")
>         $(TOPDIR)/support/scripts/fix-rpath host
>         $(TOPDIR)/support/scripts/fix-rpath staging
> @@ -582,6 +582,17 @@ sdk: world
>         mkdir -p $(HOST_DIR)/share/buildroot
>         echo $(HOST_DIR) > $(HOST_DIR)/share/buildroot/sdk-location
>
> +BR2_SDK_PREFIX ?= $(GNU_TARGET_NAME)_sdk-buildroot
> +.PHONY: sdk
> +sdk: prepare-sdk $(BR2_TAR_HOST_DEPENDENCY)
> +       @$(call MESSAGE,"Generating SDK tarball")
> +       $(if $(BR2_SDK_PREFIX),,$(error BR2_SDK_PREFIX can not be empty))
> +       $(Q)mkdir -p $(BINARIES_DIR)
> +       $(TAR) czf "$(BINARIES_DIR)/$(BR2_SDK_PREFIX).tar.gz" \
> +               --owner=0 --group=0 --numeric-owner \
> +               --transform='s#^\.#$(BR2_SDK_PREFIX)#' \
> +               -C $(HOST_DIR) "."
> +
>  # Populating the staging with the base directories is handled by the
> skeleton package
>  $(STAGING_DIR):
>         @mkdir -p $(STAGING_DIR)
> diff --git a/docs/manual/using-buildroot-toolchain.txt
> b/docs/manual/using-buildroot-toolchain.txt
> index 3246dc2411..0c0c35fced 100644
> --- a/docs/manual/using-buildroot-toolchain.txt
> +++ b/docs/manual/using-buildroot-toolchain.txt
> @@ -12,15 +12,23 @@ The toolchain generated by Buildroot is located by
> default in
>  +output/host/bin/+ to your PATH environment variable and then to
>  use +ARCH-linux-gcc+, +ARCH-linux-objdump+, +ARCH-linux-ld+, etc.
>
> -It is possible to relocate the toolchain, this allows to distribute
> -the toolchain to other developers to build applications for your
> -target. To achieve this:
> +Alternatively, Buildroot can also export the toolchain and the development
> +files of all selected packages, as an SDK, by running the command
> ++make sdk+. This generates a tarball of the content of the host directory
> ++output/host/+, named +<TARGET-TUPLE>_sdk-buildroot.tar.gz+ (which can be
> +overriden by setting the environment variable +BR2_SDK_PREFIX+) and
> +located in the output directory +output/images/+.
>
> -* run +make sdk+, which prepares the toolchain to be relocatable;
> -* tarball the contents of the +output/host+ directory;
> -* distribute the resulting tarball.
> +This tarball can then be distributed to application developers, when
> +they want to develop their applications that are not (yet) packaged as
> +a Buildroot package.
>
> -Once the toolchain is installed to the new location, the user must run
> -the +relocate-sdk.sh+ script to make sure all paths are updated with
> -the new location.
> +Upon extracting the SDK tarball, the user must run the script
> ++relocate-sdk.sh+ (located at the top directory of the SDK), to make
> +sure all paths are updated with the new location.
>
> +Alternatively, if you just want to prepare the SDK without generating
> +the tarball (e.g. because you will just be moving the +host+ directory,
> +or will be generating the tarball on your own), Buildroot also allows
> +you to just prepare the SDK with +make prepare-sdk+ without actually
> +generating a tarball.
> --
> 2.14.1
>
>
<div dir="ltr"><div>Hi Yann, All,</div><div><br></div><div>looks OK to me.<br></div></div><br><div class="gmail_quote"><div dir="ltr">On Sat, Aug 4, 2018 at 11:27 AM Yann E. MORIN &lt;<a href="mailto:yann.morin.1998@free.fr">yann.morin.1998@free.fr</a>&gt; wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Currently, the wording in the manual instructs the user to generate a<br>
tarball from &quot;the contents of the +output/host+ directory&quot;.<br>
<br>
This is pretty confusing, because taken literally, this would amount to<br>
running a command like:<br>
<br>
    tar cf my-sdk.tar -C output/host/ .<br>
<br>
This creates a tarbomb [0], which is very bad practice, because when<br>
extracted, it creates multiple files in the current directory.<br>
<br>
What one really wants to do, is create a tarball of the host/ directory,<br>
with something like:<br>
<br>
    tar cf my-sdk.tar -C output host/<br>
<br>
However, this is not much better, because the top-most directory would<br>
have a very common name, host/, which is pretty easy to get conflict<br>
with when it gets extracted.<br>
<br>
So, we fix that mess by giving the top-most directory a recognisable<br>
name, based on the target tuple, which we also use as the name of the<br>
archive (suffixed with the usual +.tar.gz+.) We offer the user the<br>
possibility to override that default by specifying the +BR2_SDK_PREFIX+<br>
variable on the command line.<br>
<br>
Since this is an output file, we place it in the images/ directory.<br>
<br>
As some users expressed a very strong feeling that they do not want to<br>
generate a tarball at all, and that doing so would badly hurt their<br>
workflows [1], we actually prepare the SDK as was previously done, but<br>
under the new, intermediate rule &#39;prepare-sdk&#39;. The existing &#39;sdk&#39; rule<br>
obviously depend on that before generating the tarball.<br>
<br>
We choose to make the existing rule to generate the tarball, and<br>
introduce a new rule to just prepare the SDK, rather than keep the<br>
existing rule as-is and introduce a new one to generate the tarball,<br>
because it makes sense to have the simplest rule do the correct thing,<br>
leaving advanced, power users use the longest command. If someone<br>
already had a wrapper that called &#39;sdk&#39; and expected just the host<br>
directory to be prepared, then this is not broken; it just takes a bit<br>
longer (gzip is pretty fast).<br>
<br>
Update the manual accordingly.<br>
<br>
[0] <a href="https://en.wikipedia.org/wiki/Tar_(computing)#Tarbomb" rel="noreferrer" target="_blank">https://en.wikipedia.org/wiki/Tar_(computing)#Tarbomb</a><br>
[1] <a href="http://lists.busybox.net/pipermail/buildroot/2018-June/thread.html#223377" rel="noreferrer" target="_blank">http://lists.busybox.net/pipermail/buildroot/2018-June/thread.html#223377</a><br>
    and some messages in the ensuing thread...<br>
<br>
Signed-off-by: &quot;Yann E. MORIN&quot; &lt;<a href="mailto:yann.morin.1998@free.fr" target="_blank">yann.morin.1998@free.fr</a>&gt;<br>
Cc: Wolfgang Grandegger &lt;<a href="mailto:wg@grandegger.com" target="_blank">wg@grandegger.com</a>&gt;<br>
Cc: Thomas Petazzoni &lt;<a href="mailto:thomas.petazzoni@bootlin.com" target="_blank">thomas.petazzoni@bootlin.com</a>&gt;<br>
Cc: Arnout Vandecappelle &lt;<a href="mailto:arnout@mind.be" target="_blank">arnout@mind.be</a>&gt;<br>
Cc: Stefan Becker &lt;<a href="mailto:chemobejk@gmail.com" target="_blank">chemobejk@gmail.com</a>&gt;<br>
Cc: Trent Piepho &lt;<a href="mailto:tpiepho@impinj.com" target="_blank">tpiepho@impinj.com</a>&gt;<br>
<br>
---<br>
Changes v2 -&gt; v3:<br>
  - drop the version from the prefix  (Stefan, Thomas)<br>
  - actually have &#39;sdk&#39; depend on &#39;prepare-sdk&#39;<br>
  - anonymise the generated tarball<br>
<br>
Changes v1 -&gt; v2:<br>
  - drop the part of the manual stating this can&#39;t be re-used as a<br>
    pre-built toolchain  (Arnout)<br>
  - make the top-level directory and tarball name configurable<br>
    (Trent, Thomas)<br>
  - change the default name  (Arnout)<br>
  - allow just preparing the SDK (Stefan, Trent)<br>
  - typoes  (Thomas)<br>
---<br>
 Makefile                                  | 15 +++++++++++++--<br>
 docs/manual/using-buildroot-toolchain.txt | 26 +++++++++++++++++---------<br>
 2 files changed, 30 insertions(+), 11 deletions(-)<br>
<br>
diff --git a/Makefile b/Makefile<br>
index 37c7072c7c..e13e5eaefa 100644<br>
--- a/Makefile<br>
+++ b/Makefile<br>
@@ -573,8 +573,8 @@ prepare: $(BUILD_DIR)/buildroot-config/auto.conf<br>
 .PHONY: world<br>
 world: target-post-image<br>
<br>
-.PHONY: sdk<br>
-sdk: world<br>
+.PHONY: prepare-sdk<br>
+prepare-sdk: world<br>
        @$(call MESSAGE,&quot;Rendering the SDK relocatable&quot;)<br>
        $(TOPDIR)/support/scripts/fix-rpath host<br>
        $(TOPDIR)/support/scripts/fix-rpath staging<br>
@@ -582,6 +582,17 @@ sdk: world<br>
        mkdir -p $(HOST_DIR)/share/buildroot<br>
        echo $(HOST_DIR) &gt; $(HOST_DIR)/share/buildroot/sdk-location<br>
<br>
+BR2_SDK_PREFIX ?= $(GNU_TARGET_NAME)_sdk-buildroot<br>
+.PHONY: sdk<br>
+sdk: prepare-sdk $(BR2_TAR_HOST_DEPENDENCY)<br>
+       @$(call MESSAGE,&quot;Generating SDK tarball&quot;)<br>
+       $(if $(BR2_SDK_PREFIX),,$(error BR2_SDK_PREFIX can not be empty))<br>
+       $(Q)mkdir -p $(BINARIES_DIR)<br>
+       $(TAR) czf &quot;$(BINARIES_DIR)/$(BR2_SDK_PREFIX).tar.gz&quot; \<br>
+               --owner=0 --group=0 --numeric-owner \<br>
+               --transform=&#39;s#^\.#$(BR2_SDK_PREFIX)#&#39; \<br>
+               -C $(HOST_DIR) &quot;.&quot;<br>
+<br>
 # Populating the staging with the base directories is handled by the skeleton package<br>
 $(STAGING_DIR):<br>
        @mkdir -p $(STAGING_DIR)<br>
diff --git a/docs/manual/using-buildroot-toolchain.txt b/docs/manual/using-buildroot-toolchain.txt<br>
index 3246dc2411..0c0c35fced 100644<br>
--- a/docs/manual/using-buildroot-toolchain.txt<br>
+++ b/docs/manual/using-buildroot-toolchain.txt<br>
@@ -12,15 +12,23 @@ The toolchain generated by Buildroot is located by default in<br>
 +output/host/bin/+ to your PATH environment variable and then to<br>
 use +ARCH-linux-gcc+, +ARCH-linux-objdump+, +ARCH-linux-ld+, etc.<br>
<br>
-It is possible to relocate the toolchain, this allows to distribute<br>
-the toolchain to other developers to build applications for your<br>
-target. To achieve this:<br>
+Alternatively, Buildroot can also export the toolchain and the development<br>
+files of all selected packages, as an SDK, by running the command<br>
++make sdk+. This generates a tarball of the content of the host directory<br>
++output/host/+, named +&lt;TARGET-TUPLE&gt;_sdk-buildroot.tar.gz+ (which can be<br>
+overriden by setting the environment variable +BR2_SDK_PREFIX+) and<br>
+located in the output directory +output/images/+.<br>
<br>
-* run +make sdk+, which prepares the toolchain to be relocatable;<br>
-* tarball the contents of the +output/host+ directory;<br>
-* distribute the resulting tarball.<br>
+This tarball can then be distributed to application developers, when<br>
+they want to develop their applications that are not (yet) packaged as<br>
+a Buildroot package.<br>
<br>
-Once the toolchain is installed to the new location, the user must run<br>
-the +relocate-sdk.sh+ script to make sure all paths are updated with<br>
-the new location.<br>
+Upon extracting the SDK tarball, the user must run the script<br>
++relocate-sdk.sh+ (located at the top directory of the SDK), to make<br>
+sure all paths are updated with the new location.<br>
<br>
+Alternatively, if you just want to prepare the SDK without generating<br>
+the tarball (e.g. because you will just be moving the +host+ directory,<br>
+or will be generating the tarball on your own), Buildroot also allows<br>
+you to just prepare the SDK with +make prepare-sdk+ without actually<br>
+generating a tarball.<br>
-- <br>
2.14.1<br>
<br>
</blockquote></div>
Yann E. MORIN Aug. 5, 2018, 1:07 p.m. | #2
Stefan, All,

On 2018-08-05 16:04 +0300, Stefan Becker spake thusly:
> Hi Yann, All,
> looks OK to me.

Thanks! :-)

Is that an equivalent to a formal 'Reviewed-by' tag? ;-)
https://buildroot.org/downloads/manual/manual.html#_reviewing_and_testing_patches

Regards,
Yann E. MORIN.

> On Sat, Aug 4, 2018 at 11:27 AM Yann E. MORIN < [1]yann.morin.1998@free.fr> wrote:
> 
>   Currently, the wording in the manual instructs the user to generate a
>   tarball from "the contents of the +output/host+ directory".
> 
>   This is pretty confusing, because taken literally, this would amount to
>   running a command like:
> 
>       tar cf my-sdk.tar -C output/host/ .
> 
>   This creates a tarbomb [0], which is very bad practice, because when
>   extracted, it creates multiple files in the current directory.
> 
>   What one really wants to do, is create a tarball of the host/ directory,
>   with something like:
> 
>       tar cf my-sdk.tar -C output host/
> 
>   However, this is not much better, because the top-most directory would
>   have a very common name, host/, which is pretty easy to get conflict
>   with when it gets extracted.
> 
>   So, we fix that mess by giving the top-most directory a recognisable
>   name, based on the target tuple, which we also use as the name of the
>   archive (suffixed with the usual +.tar.gz+.) We offer the user the
>   possibility to override that default by specifying the +BR2_SDK_PREFIX+
>   variable on the command line.
> 
>   Since this is an output file, we place it in the images/ directory.
> 
>   As some users expressed a very strong feeling that they do not want to
>   generate a tarball at all, and that doing so would badly hurt their
>   workflows [1], we actually prepare the SDK as was previously done, but
>   under the new, intermediate rule 'prepare-sdk'. The existing 'sdk' rule
>   obviously depend on that before generating the tarball.
> 
>   We choose to make the existing rule to generate the tarball, and
>   introduce a new rule to just prepare the SDK, rather than keep the
>   existing rule as-is and introduce a new one to generate the tarball,
>   because it makes sense to have the simplest rule do the correct thing,
>   leaving advanced, power users use the longest command. If someone
>   already had a wrapper that called 'sdk' and expected just the host
>   directory to be prepared, then this is not broken; it just takes a bit
>   longer (gzip is pretty fast).
> 
>   Update the manual accordingly.
> 
>   [0] [2]https://en.wikipedia.org/wiki/Tar_(computing)#Tarbomb
>   [1] [3]http://lists.busybox.net/pipermail/buildroot/2018-June/thread.html#223377
>       and some messages in the ensuing thread...
> 
>   Signed-off-by: "Yann E. MORIN" < [4]yann.morin.1998@free.fr>
>   Cc: Wolfgang Grandegger < [5]wg@grandegger.com>
>   Cc: Thomas Petazzoni < [6]thomas.petazzoni@bootlin.com>
>   Cc: Arnout Vandecappelle < [7]arnout@mind.be>
>   Cc: Stefan Becker < [8]chemobejk@gmail.com>
>   Cc: Trent Piepho < [9]tpiepho@impinj.com>
> 
>   ---
>   Changes v2 -> v3:
>     - drop the version from the prefix  (Stefan, Thomas)
>     - actually have 'sdk' depend on 'prepare-sdk'
>     - anonymise the generated tarball
> 
>   Changes v1 -> v2:
>     - drop the part of the manual stating this can't be re-used as a
>       pre-built toolchain  (Arnout)
>     - make the top-level directory and tarball name configurable
>       (Trent, Thomas)
>     - change the default name  (Arnout)
>     - allow just preparing the SDK (Stefan, Trent)
>     - typoes  (Thomas)
>   ---
>    Makefile                               
>     | 15 +++++++++++++--
>    docs/manual/using-buildroot-toolchain.txt | 26 +++++++++++++++++---------
>    2 files changed, 30 insertions(+), 11 deletions(-)
> 
>   diff --git a/Makefile b/Makefile
>   index 37c7072c7c..e13e5eaefa 100644
>   --- a/Makefile
>   +++ b/Makefile
>   @@ -573,8 +573,8 @@ prepare: $(BUILD_DIR)/buildroot-config/auto.conf
>    .PHONY: world
>    world: target-post-image
> 
>   -.PHONY: sdk
>   -sdk: world
>   +.PHONY: prepare-sdk
>   +prepare-sdk: world
>           @$(call MESSAGE,"Rendering the SDK relocatable")
>           $(TOPDIR)/support/scripts/fix-rpath host
>           $(TOPDIR)/support/scripts/fix-rpath staging
>   @@ -582,6 +582,17 @@ sdk: world
>           mkdir -p $(HOST_DIR)/share/buildroot
>           echo $(HOST_DIR) > $(HOST_DIR)/share/buildroot/sdk-location
> 
>   +BR2_SDK_PREFIX ?= $(GNU_TARGET_NAME)_sdk-buildroot
>   +.PHONY: sdk
>   +sdk: prepare-sdk $(BR2_TAR_HOST_DEPENDENCY)
>   +       @$(call MESSAGE,"Generating SDK tarball")
>   +       $(if $(BR2_SDK_PREFIX),,$(error BR2_SDK_PREFIX can not be empty))
>   +       $(Q)mkdir -p $(BINARIES_DIR)
>   +       $(TAR) czf "$(BINARIES_DIR)/$(BR2_SDK_PREFIX).tar.gz" \
>   +               --owner=0 --group=0 --numeric-owner \
>   +               --transform='s#^\.#$(BR2_SDK_PREFIX)#' \
>   +               -C $(HOST_DIR) "."
>   +
>    # Populating the staging with the base directories is handled by the skeleton package
>    $(STAGING_DIR):
>           @mkdir -p $(STAGING_DIR)
>   diff --git a/docs/manual/using-buildroot-toolchain.txt b/docs/manual/using-buildroot-toolchain.txt
>   index 3246dc2411..0c0c35fced 100644
>   --- a/docs/manual/using-buildroot-toolchain.txt
>   +++ b/docs/manual/using-buildroot-toolchain.txt
>   @@ -12,15 +12,23 @@ The toolchain generated by Buildroot is located by default in
>    +output/host/bin/+ to your PATH environment variable and then to
>    use +ARCH-linux-gcc+, +ARCH-linux-objdump+, +ARCH-linux-ld+, etc.
> 
>   -It is possible to relocate the toolchain, this allows to distribute
>   -the toolchain to other developers to build applications for your
>   -target. To achieve this:
>   +Alternatively, Buildroot can also export the toolchain and the development
>   +files of all selected packages, as an SDK, by running the command
>   ++make sdk+. This generates a tarball of the content of the host directory
>   ++output/host/+, named +<TARGET-TUPLE>_sdk-buildroot.tar.gz+ (which can be
>   +overriden by setting the environment variable +BR2_SDK_PREFIX+) and
>   +located in the output directory +output/images/+.
> 
>   -* run +make sdk+, which prepares the toolchain to be relocatable;
>   -* tarball the contents of the +output/host+ directory;
>   -* distribute the resulting tarball.
>   +This tarball can then be distributed to application developers, when
>   +they want to develop their applications that are not (yet) packaged as
>   +a Buildroot package.
> 
>   -Once the toolchain is installed to the new location, the user must run
>   -the +relocate-sdk.sh+ script to make sure all paths are updated with
>   -the new location.
>   +Upon extracting the SDK tarball, the user must run the script
>   ++relocate-sdk.sh+ (located at the top directory of the SDK), to make
>   +sure all paths are updated with the new location.
> 
>   +Alternatively, if you just want to prepare the SDK without generating
>   +the tarball (e.g. because you will just be moving the +host+ directory,
>   +or will be generating the tarball on your own), Buildroot also allows
>   +you to just prepare the SDK with +make prepare-sdk+ without actually
>   +generating a tarball.
>   --
>   2.14.1
> 
> Links:
> 1. mailto:yann.morin.1998@free.fr
> 2. https://en.wikipedia.org/wiki/Tar_(computing)#Tarbomb
> 3. http://lists.busybox.net/pipermail/buildroot/2018-June/thread.html#223377
> 4. mailto:yann.morin.1998@free.fr
> 5. mailto:wg@grandegger.com
> 6. mailto:thomas.petazzoni@bootlin.com
> 7. mailto:arnout@mind.be
> 8. mailto:chemobejk@gmail.com
> 9. mailto:tpiepho@impinj.com
Stefan Becker Aug. 5, 2018, 1:20 p.m. | #3
Reviewed-by: Stefan Becker <chemobejk@gmail.com>

No time to test this though...

On Sat, Aug 4, 2018 at 11:27 AM Yann E. MORIN <yann.morin.1998@free.fr>
wrote:

> Currently, the wording in the manual instructs the user to generate a
> tarball from "the contents of the +output/host+ directory".
>
> This is pretty confusing, because taken literally, this would amount to
> running a command like:
>
>     tar cf my-sdk.tar -C output/host/ .
>
> This creates a tarbomb [0], which is very bad practice, because when
> extracted, it creates multiple files in the current directory.
>
> What one really wants to do, is create a tarball of the host/ directory,
> with something like:
>
>     tar cf my-sdk.tar -C output host/
>
> However, this is not much better, because the top-most directory would
> have a very common name, host/, which is pretty easy to get conflict
> with when it gets extracted.
>
> So, we fix that mess by giving the top-most directory a recognisable
> name, based on the target tuple, which we also use as the name of the
> archive (suffixed with the usual +.tar.gz+.) We offer the user the
> possibility to override that default by specifying the +BR2_SDK_PREFIX+
> variable on the command line.
>
> Since this is an output file, we place it in the images/ directory.
>
> As some users expressed a very strong feeling that they do not want to
> generate a tarball at all, and that doing so would badly hurt their
> workflows [1], we actually prepare the SDK as was previously done, but
> under the new, intermediate rule 'prepare-sdk'. The existing 'sdk' rule
> obviously depend on that before generating the tarball.
>
> We choose to make the existing rule to generate the tarball, and
> introduce a new rule to just prepare the SDK, rather than keep the
> existing rule as-is and introduce a new one to generate the tarball,
> because it makes sense to have the simplest rule do the correct thing,
> leaving advanced, power users use the longest command. If someone
> already had a wrapper that called 'sdk' and expected just the host
> directory to be prepared, then this is not broken; it just takes a bit
> longer (gzip is pretty fast).
>
> Update the manual accordingly.
>
> [0] https://en.wikipedia.org/wiki/Tar_(computing)#Tarbomb
> [1]
> http://lists.busybox.net/pipermail/buildroot/2018-June/thread.html#223377
>     and some messages in the ensuing thread...
>
> Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
> Cc: Wolfgang Grandegger <wg@grandegger.com>
> Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
> Cc: Arnout Vandecappelle <arnout@mind.be>
> Cc: Stefan Becker <chemobejk@gmail.com>
> Cc: Trent Piepho <tpiepho@impinj.com>
>
> ---
> Changes v2 -> v3:
>   - drop the version from the prefix  (Stefan, Thomas)
>   - actually have 'sdk' depend on 'prepare-sdk'
>   - anonymise the generated tarball
>
> Changes v1 -> v2:
>   - drop the part of the manual stating this can't be re-used as a
>     pre-built toolchain  (Arnout)
>   - make the top-level directory and tarball name configurable
>     (Trent, Thomas)
>   - change the default name  (Arnout)
>   - allow just preparing the SDK (Stefan, Trent)
>   - typoes  (Thomas)
> ---
>  Makefile                                  | 15 +++++++++++++--
>  docs/manual/using-buildroot-toolchain.txt | 26 +++++++++++++++++---------
>  2 files changed, 30 insertions(+), 11 deletions(-)
>
> diff --git a/Makefile b/Makefile
> index 37c7072c7c..e13e5eaefa 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -573,8 +573,8 @@ prepare: $(BUILD_DIR)/buildroot-config/auto.conf
>  .PHONY: world
>  world: target-post-image
>
> -.PHONY: sdk
> -sdk: world
> +.PHONY: prepare-sdk
> +prepare-sdk: world
>         @$(call MESSAGE,"Rendering the SDK relocatable")
>         $(TOPDIR)/support/scripts/fix-rpath host
>         $(TOPDIR)/support/scripts/fix-rpath staging
> @@ -582,6 +582,17 @@ sdk: world
>         mkdir -p $(HOST_DIR)/share/buildroot
>         echo $(HOST_DIR) > $(HOST_DIR)/share/buildroot/sdk-location
>
> +BR2_SDK_PREFIX ?= $(GNU_TARGET_NAME)_sdk-buildroot
> +.PHONY: sdk
> +sdk: prepare-sdk $(BR2_TAR_HOST_DEPENDENCY)
> +       @$(call MESSAGE,"Generating SDK tarball")
> +       $(if $(BR2_SDK_PREFIX),,$(error BR2_SDK_PREFIX can not be empty))
> +       $(Q)mkdir -p $(BINARIES_DIR)
> +       $(TAR) czf "$(BINARIES_DIR)/$(BR2_SDK_PREFIX).tar.gz" \
> +               --owner=0 --group=0 --numeric-owner \
> +               --transform='s#^\.#$(BR2_SDK_PREFIX)#' \
> +               -C $(HOST_DIR) "."
> +
>  # Populating the staging with the base directories is handled by the
> skeleton package
>  $(STAGING_DIR):
>         @mkdir -p $(STAGING_DIR)
> diff --git a/docs/manual/using-buildroot-toolchain.txt
> b/docs/manual/using-buildroot-toolchain.txt
> index 3246dc2411..0c0c35fced 100644
> --- a/docs/manual/using-buildroot-toolchain.txt
> +++ b/docs/manual/using-buildroot-toolchain.txt
> @@ -12,15 +12,23 @@ The toolchain generated by Buildroot is located by
> default in
>  +output/host/bin/+ to your PATH environment variable and then to
>  use +ARCH-linux-gcc+, +ARCH-linux-objdump+, +ARCH-linux-ld+, etc.
>
> -It is possible to relocate the toolchain, this allows to distribute
> -the toolchain to other developers to build applications for your
> -target. To achieve this:
> +Alternatively, Buildroot can also export the toolchain and the development
> +files of all selected packages, as an SDK, by running the command
> ++make sdk+. This generates a tarball of the content of the host directory
> ++output/host/+, named +<TARGET-TUPLE>_sdk-buildroot.tar.gz+ (which can be
> +overriden by setting the environment variable +BR2_SDK_PREFIX+) and
> +located in the output directory +output/images/+.
>
> -* run +make sdk+, which prepares the toolchain to be relocatable;
> -* tarball the contents of the +output/host+ directory;
> -* distribute the resulting tarball.
> +This tarball can then be distributed to application developers, when
> +they want to develop their applications that are not (yet) packaged as
> +a Buildroot package.
>
> -Once the toolchain is installed to the new location, the user must run
> -the +relocate-sdk.sh+ script to make sure all paths are updated with
> -the new location.
> +Upon extracting the SDK tarball, the user must run the script
> ++relocate-sdk.sh+ (located at the top directory of the SDK), to make
> +sure all paths are updated with the new location.
>
> +Alternatively, if you just want to prepare the SDK without generating
> +the tarball (e.g. because you will just be moving the +host+ directory,
> +or will be generating the tarball on your own), Buildroot also allows
> +you to just prepare the SDK with +make prepare-sdk+ without actually
> +generating a tarball.
> --
> 2.14.1
>
>
<div dir="ltr"><div>Reviewed-by: Stefan Becker &lt;<a href="mailto:chemobejk@gmail.com">chemobejk@gmail.com</a>&gt;</div><div><br></div><div>No time to test this though...<br></div></div><br><div class="gmail_quote"><div dir="ltr">On Sat, Aug 4, 2018 at 11:27 AM Yann E. MORIN &lt;<a href="mailto:yann.morin.1998@free.fr">yann.morin.1998@free.fr</a>&gt; wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Currently, the wording in the manual instructs the user to generate a<br>
tarball from &quot;the contents of the +output/host+ directory&quot;.<br>
<br>
This is pretty confusing, because taken literally, this would amount to<br>
running a command like:<br>
<br>
    tar cf my-sdk.tar -C output/host/ .<br>
<br>
This creates a tarbomb [0], which is very bad practice, because when<br>
extracted, it creates multiple files in the current directory.<br>
<br>
What one really wants to do, is create a tarball of the host/ directory,<br>
with something like:<br>
<br>
    tar cf my-sdk.tar -C output host/<br>
<br>
However, this is not much better, because the top-most directory would<br>
have a very common name, host/, which is pretty easy to get conflict<br>
with when it gets extracted.<br>
<br>
So, we fix that mess by giving the top-most directory a recognisable<br>
name, based on the target tuple, which we also use as the name of the<br>
archive (suffixed with the usual +.tar.gz+.) We offer the user the<br>
possibility to override that default by specifying the +BR2_SDK_PREFIX+<br>
variable on the command line.<br>
<br>
Since this is an output file, we place it in the images/ directory.<br>
<br>
As some users expressed a very strong feeling that they do not want to<br>
generate a tarball at all, and that doing so would badly hurt their<br>
workflows [1], we actually prepare the SDK as was previously done, but<br>
under the new, intermediate rule &#39;prepare-sdk&#39;. The existing &#39;sdk&#39; rule<br>
obviously depend on that before generating the tarball.<br>
<br>
We choose to make the existing rule to generate the tarball, and<br>
introduce a new rule to just prepare the SDK, rather than keep the<br>
existing rule as-is and introduce a new one to generate the tarball,<br>
because it makes sense to have the simplest rule do the correct thing,<br>
leaving advanced, power users use the longest command. If someone<br>
already had a wrapper that called &#39;sdk&#39; and expected just the host<br>
directory to be prepared, then this is not broken; it just takes a bit<br>
longer (gzip is pretty fast).<br>
<br>
Update the manual accordingly.<br>
<br>
[0] <a href="https://en.wikipedia.org/wiki/Tar_(computing)#Tarbomb" rel="noreferrer" target="_blank">https://en.wikipedia.org/wiki/Tar_(computing)#Tarbomb</a><br>
[1] <a href="http://lists.busybox.net/pipermail/buildroot/2018-June/thread.html#223377" rel="noreferrer" target="_blank">http://lists.busybox.net/pipermail/buildroot/2018-June/thread.html#223377</a><br>
    and some messages in the ensuing thread...<br>
<br>
Signed-off-by: &quot;Yann E. MORIN&quot; &lt;<a href="mailto:yann.morin.1998@free.fr" target="_blank">yann.morin.1998@free.fr</a>&gt;<br>
Cc: Wolfgang Grandegger &lt;<a href="mailto:wg@grandegger.com" target="_blank">wg@grandegger.com</a>&gt;<br>
Cc: Thomas Petazzoni &lt;<a href="mailto:thomas.petazzoni@bootlin.com" target="_blank">thomas.petazzoni@bootlin.com</a>&gt;<br>
Cc: Arnout Vandecappelle &lt;<a href="mailto:arnout@mind.be" target="_blank">arnout@mind.be</a>&gt;<br>
Cc: Stefan Becker &lt;<a href="mailto:chemobejk@gmail.com" target="_blank">chemobejk@gmail.com</a>&gt;<br>
Cc: Trent Piepho &lt;<a href="mailto:tpiepho@impinj.com" target="_blank">tpiepho@impinj.com</a>&gt;<br>
<br>
---<br>
Changes v2 -&gt; v3:<br>
  - drop the version from the prefix  (Stefan, Thomas)<br>
  - actually have &#39;sdk&#39; depend on &#39;prepare-sdk&#39;<br>
  - anonymise the generated tarball<br>
<br>
Changes v1 -&gt; v2:<br>
  - drop the part of the manual stating this can&#39;t be re-used as a<br>
    pre-built toolchain  (Arnout)<br>
  - make the top-level directory and tarball name configurable<br>
    (Trent, Thomas)<br>
  - change the default name  (Arnout)<br>
  - allow just preparing the SDK (Stefan, Trent)<br>
  - typoes  (Thomas)<br>
---<br>
 Makefile                                  | 15 +++++++++++++--<br>
 docs/manual/using-buildroot-toolchain.txt | 26 +++++++++++++++++---------<br>
 2 files changed, 30 insertions(+), 11 deletions(-)<br>
<br>
diff --git a/Makefile b/Makefile<br>
index 37c7072c7c..e13e5eaefa 100644<br>
--- a/Makefile<br>
+++ b/Makefile<br>
@@ -573,8 +573,8 @@ prepare: $(BUILD_DIR)/buildroot-config/auto.conf<br>
 .PHONY: world<br>
 world: target-post-image<br>
<br>
-.PHONY: sdk<br>
-sdk: world<br>
+.PHONY: prepare-sdk<br>
+prepare-sdk: world<br>
        @$(call MESSAGE,&quot;Rendering the SDK relocatable&quot;)<br>
        $(TOPDIR)/support/scripts/fix-rpath host<br>
        $(TOPDIR)/support/scripts/fix-rpath staging<br>
@@ -582,6 +582,17 @@ sdk: world<br>
        mkdir -p $(HOST_DIR)/share/buildroot<br>
        echo $(HOST_DIR) &gt; $(HOST_DIR)/share/buildroot/sdk-location<br>
<br>
+BR2_SDK_PREFIX ?= $(GNU_TARGET_NAME)_sdk-buildroot<br>
+.PHONY: sdk<br>
+sdk: prepare-sdk $(BR2_TAR_HOST_DEPENDENCY)<br>
+       @$(call MESSAGE,&quot;Generating SDK tarball&quot;)<br>
+       $(if $(BR2_SDK_PREFIX),,$(error BR2_SDK_PREFIX can not be empty))<br>
+       $(Q)mkdir -p $(BINARIES_DIR)<br>
+       $(TAR) czf &quot;$(BINARIES_DIR)/$(BR2_SDK_PREFIX).tar.gz&quot; \<br>
+               --owner=0 --group=0 --numeric-owner \<br>
+               --transform=&#39;s#^\.#$(BR2_SDK_PREFIX)#&#39; \<br>
+               -C $(HOST_DIR) &quot;.&quot;<br>
+<br>
 # Populating the staging with the base directories is handled by the skeleton package<br>
 $(STAGING_DIR):<br>
        @mkdir -p $(STAGING_DIR)<br>
diff --git a/docs/manual/using-buildroot-toolchain.txt b/docs/manual/using-buildroot-toolchain.txt<br>
index 3246dc2411..0c0c35fced 100644<br>
--- a/docs/manual/using-buildroot-toolchain.txt<br>
+++ b/docs/manual/using-buildroot-toolchain.txt<br>
@@ -12,15 +12,23 @@ The toolchain generated by Buildroot is located by default in<br>
 +output/host/bin/+ to your PATH environment variable and then to<br>
 use +ARCH-linux-gcc+, +ARCH-linux-objdump+, +ARCH-linux-ld+, etc.<br>
<br>
-It is possible to relocate the toolchain, this allows to distribute<br>
-the toolchain to other developers to build applications for your<br>
-target. To achieve this:<br>
+Alternatively, Buildroot can also export the toolchain and the development<br>
+files of all selected packages, as an SDK, by running the command<br>
++make sdk+. This generates a tarball of the content of the host directory<br>
++output/host/+, named +&lt;TARGET-TUPLE&gt;_sdk-buildroot.tar.gz+ (which can be<br>
+overriden by setting the environment variable +BR2_SDK_PREFIX+) and<br>
+located in the output directory +output/images/+.<br>
<br>
-* run +make sdk+, which prepares the toolchain to be relocatable;<br>
-* tarball the contents of the +output/host+ directory;<br>
-* distribute the resulting tarball.<br>
+This tarball can then be distributed to application developers, when<br>
+they want to develop their applications that are not (yet) packaged as<br>
+a Buildroot package.<br>
<br>
-Once the toolchain is installed to the new location, the user must run<br>
-the +relocate-sdk.sh+ script to make sure all paths are updated with<br>
-the new location.<br>
+Upon extracting the SDK tarball, the user must run the script<br>
++relocate-sdk.sh+ (located at the top directory of the SDK), to make<br>
+sure all paths are updated with the new location.<br>
<br>
+Alternatively, if you just want to prepare the SDK without generating<br>
+the tarball (e.g. because you will just be moving the +host+ directory,<br>
+or will be generating the tarball on your own), Buildroot also allows<br>
+you to just prepare the SDK with +make prepare-sdk+ without actually<br>
+generating a tarball.<br>
-- <br>
2.14.1<br>
<br>
</blockquote></div>
Yann E. MORIN Aug. 5, 2018, 1:29 p.m. | #4
Stefan All,

On 2018-08-05 16:20 +0300, Stefan Becker spake thusly:
> Reviewed-by: Stefan Becker < [1]chemobejk@gmail.com>
> No time to test this though...

No problem. Thanks! :-)

Regards,
Yann E. MORIN.

> On Sat, Aug 4, 2018 at 11:27 AM Yann E. MORIN < [2]yann.morin.1998@free.fr> wrote:
> 
>   Currently, the wording in the manual instructs the user to generate a
>   tarball from "the contents of the +output/host+ directory".
> 
>   This is pretty confusing, because taken literally, this would amount to
>   running a command like:
> 
>       tar cf my-sdk.tar -C output/host/ .
> 
>   This creates a tarbomb [0], which is very bad practice, because when
>   extracted, it creates multiple files in the current directory.
> 
>   What one really wants to do, is create a tarball of the host/ directory,
>   with something like:
> 
>       tar cf my-sdk.tar -C output host/
> 
>   However, this is not much better, because the top-most directory would
>   have a very common name, host/, which is pretty easy to get conflict
>   with when it gets extracted.
> 
>   So, we fix that mess by giving the top-most directory a recognisable
>   name, based on the target tuple, which we also use as the name of the
>   archive (suffixed with the usual +.tar.gz+.) We offer the user the
>   possibility to override that default by specifying the +BR2_SDK_PREFIX+
>   variable on the command line.
> 
>   Since this is an output file, we place it in the images/ directory.
> 
>   As some users expressed a very strong feeling that they do not want to
>   generate a tarball at all, and that doing so would badly hurt their
>   workflows [1], we actually prepare the SDK as was previously done, but
>   under the new, intermediate rule 'prepare-sdk'. The existing 'sdk' rule
>   obviously depend on that before generating the tarball.
> 
>   We choose to make the existing rule to generate the tarball, and
>   introduce a new rule to just prepare the SDK, rather than keep the
>   existing rule as-is and introduce a new one to generate the tarball,
>   because it makes sense to have the simplest rule do the correct thing,
>   leaving advanced, power users use the longest command. If someone
>   already had a wrapper that called 'sdk' and expected just the host
>   directory to be prepared, then this is not broken; it just takes a bit
>   longer (gzip is pretty fast).
> 
>   Update the manual accordingly.
> 
>   [0] [3]https://en.wikipedia.org/wiki/Tar_(computing)#Tarbomb
>   [1] [4]http://lists.busybox.net/pipermail/buildroot/2018-June/thread.html#223377
>       and some messages in the ensuing thread...
> 
>   Signed-off-by: "Yann E. MORIN" < [5]yann.morin.1998@free.fr>
>   Cc: Wolfgang Grandegger < [6]wg@grandegger.com>
>   Cc: Thomas Petazzoni < [7]thomas.petazzoni@bootlin.com>
>   Cc: Arnout Vandecappelle < [8]arnout@mind.be>
>   Cc: Stefan Becker < [9]chemobejk@gmail.com>
>   Cc: Trent Piepho < [10]tpiepho@impinj.com>
> 
>   ---
>   Changes v2 -> v3:
>     - drop the version from the prefix  (Stefan, Thomas)
>     - actually have 'sdk' depend on 'prepare-sdk'
>     - anonymise the generated tarball
> 
>   Changes v1 -> v2:
>     - drop the part of the manual stating this can't be re-used as a
>       pre-built toolchain  (Arnout)
>     - make the top-level directory and tarball name configurable
>       (Trent, Thomas)
>     - change the default name  (Arnout)
>     - allow just preparing the SDK (Stefan, Trent)
>     - typoes  (Thomas)
>   ---
>    Makefile                               
>     | 15 +++++++++++++--
>    docs/manual/using-buildroot-toolchain.txt | 26 +++++++++++++++++---------
>    2 files changed, 30 insertions(+), 11 deletions(-)
> 
>   diff --git a/Makefile b/Makefile
>   index 37c7072c7c..e13e5eaefa 100644
>   --- a/Makefile
>   +++ b/Makefile
>   @@ -573,8 +573,8 @@ prepare: $(BUILD_DIR)/buildroot-config/auto.conf
>    .PHONY: world
>    world: target-post-image
> 
>   -.PHONY: sdk
>   -sdk: world
>   +.PHONY: prepare-sdk
>   +prepare-sdk: world
>           @$(call MESSAGE,"Rendering the SDK relocatable")
>           $(TOPDIR)/support/scripts/fix-rpath host
>           $(TOPDIR)/support/scripts/fix-rpath staging
>   @@ -582,6 +582,17 @@ sdk: world
>           mkdir -p $(HOST_DIR)/share/buildroot
>           echo $(HOST_DIR) > $(HOST_DIR)/share/buildroot/sdk-location
> 
>   +BR2_SDK_PREFIX ?= $(GNU_TARGET_NAME)_sdk-buildroot
>   +.PHONY: sdk
>   +sdk: prepare-sdk $(BR2_TAR_HOST_DEPENDENCY)
>   +       @$(call MESSAGE,"Generating SDK tarball")
>   +       $(if $(BR2_SDK_PREFIX),,$(error BR2_SDK_PREFIX can not be empty))
>   +       $(Q)mkdir -p $(BINARIES_DIR)
>   +       $(TAR) czf "$(BINARIES_DIR)/$(BR2_SDK_PREFIX).tar.gz" \
>   +               --owner=0 --group=0 --numeric-owner \
>   +               --transform='s#^\.#$(BR2_SDK_PREFIX)#' \
>   +               -C $(HOST_DIR) "."
>   +
>    # Populating the staging with the base directories is handled by the skeleton package
>    $(STAGING_DIR):
>           @mkdir -p $(STAGING_DIR)
>   diff --git a/docs/manual/using-buildroot-toolchain.txt b/docs/manual/using-buildroot-toolchain.txt
>   index 3246dc2411..0c0c35fced 100644
>   --- a/docs/manual/using-buildroot-toolchain.txt
>   +++ b/docs/manual/using-buildroot-toolchain.txt
>   @@ -12,15 +12,23 @@ The toolchain generated by Buildroot is located by default in
>    +output/host/bin/+ to your PATH environment variable and then to
>    use +ARCH-linux-gcc+, +ARCH-linux-objdump+, +ARCH-linux-ld+, etc.
> 
>   -It is possible to relocate the toolchain, this allows to distribute
>   -the toolchain to other developers to build applications for your
>   -target. To achieve this:
>   +Alternatively, Buildroot can also export the toolchain and the development
>   +files of all selected packages, as an SDK, by running the command
>   ++make sdk+. This generates a tarball of the content of the host directory
>   ++output/host/+, named +<TARGET-TUPLE>_sdk-buildroot.tar.gz+ (which can be
>   +overriden by setting the environment variable +BR2_SDK_PREFIX+) and
>   +located in the output directory +output/images/+.
> 
>   -* run +make sdk+, which prepares the toolchain to be relocatable;
>   -* tarball the contents of the +output/host+ directory;
>   -* distribute the resulting tarball.
>   +This tarball can then be distributed to application developers, when
>   +they want to develop their applications that are not (yet) packaged as
>   +a Buildroot package.
> 
>   -Once the toolchain is installed to the new location, the user must run
>   -the +relocate-sdk.sh+ script to make sure all paths are updated with
>   -the new location.
>   +Upon extracting the SDK tarball, the user must run the script
>   ++relocate-sdk.sh+ (located at the top directory of the SDK), to make
>   +sure all paths are updated with the new location.
> 
>   +Alternatively, if you just want to prepare the SDK without generating
>   +the tarball (e.g. because you will just be moving the +host+ directory,
>   +or will be generating the tarball on your own), Buildroot also allows
>   +you to just prepare the SDK with +make prepare-sdk+ without actually
>   +generating a tarball.
>   --
>   2.14.1
> 
> Links:
> 1. mailto:chemobejk@gmail.com
> 2. mailto:yann.morin.1998@free.fr
> 3. https://en.wikipedia.org/wiki/Tar_(computing)#Tarbomb
> 4. http://lists.busybox.net/pipermail/buildroot/2018-June/thread.html#223377
> 5. mailto:yann.morin.1998@free.fr
> 6. mailto:wg@grandegger.com
> 7. mailto:thomas.petazzoni@bootlin.com
> 8. mailto:arnout@mind.be
> 9. mailto:chemobejk@gmail.com
> 10. mailto:tpiepho@impinj.com

> _______________________________________________
> buildroot mailing list
> buildroot@busybox.net
> http://lists.busybox.net/mailman/listinfo/buildroot
Thomas Petazzoni Aug. 14, 2018, 2:08 p.m. | #5
Hello,

On Sat,  4 Aug 2018 10:27:41 +0200, Yann E. MORIN wrote:
> Currently, the wording in the manual instructs the user to generate a
> tarball from "the contents of the +output/host+ directory".
> 
> This is pretty confusing, because taken literally, this would amount to
> running a command like:
> 
>     tar cf my-sdk.tar -C output/host/ .
> 
> This creates a tarbomb [0], which is very bad practice, because when
> extracted, it creates multiple files in the current directory.
> 
> What one really wants to do, is create a tarball of the host/ directory,
> with something like:
> 
>     tar cf my-sdk.tar -C output host/
> 
> However, this is not much better, because the top-most directory would
> have a very common name, host/, which is pretty easy to get conflict
> with when it gets extracted.
> 
> So, we fix that mess by giving the top-most directory a recognisable
> name, based on the target tuple, which we also use as the name of the
> archive (suffixed with the usual +.tar.gz+.) We offer the user the
> possibility to override that default by specifying the +BR2_SDK_PREFIX+
> variable on the command line.
> 
> Since this is an output file, we place it in the images/ directory.
> 
> As some users expressed a very strong feeling that they do not want to
> generate a tarball at all, and that doing so would badly hurt their
> workflows [1], we actually prepare the SDK as was previously done, but
> under the new, intermediate rule 'prepare-sdk'. The existing 'sdk' rule
> obviously depend on that before generating the tarball.
> 
> We choose to make the existing rule to generate the tarball, and
> introduce a new rule to just prepare the SDK, rather than keep the
> existing rule as-is and introduce a new one to generate the tarball,
> because it makes sense to have the simplest rule do the correct thing,
> leaving advanced, power users use the longest command. If someone
> already had a wrapper that called 'sdk' and expected just the host
> directory to be prepared, then this is not broken; it just takes a bit
> longer (gzip is pretty fast).
> 
> Update the manual accordingly.
> 
> [0] https://en.wikipedia.org/wiki/Tar_(computing)#Tarbomb
> [1] http://lists.busybox.net/pipermail/buildroot/2018-June/thread.html#223377
>     and some messages in the ensuing thread...
> 
> Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
> Cc: Wolfgang Grandegger <wg@grandegger.com>
> Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
> Cc: Arnout Vandecappelle <arnout@mind.be>
> Cc: Stefan Becker <chemobejk@gmail.com>
> Cc: Trent Piepho <tpiepho@impinj.com>
> 
> ---
> Changes v2 -> v3:
>   - drop the version from the prefix  (Stefan, Thomas)
>   - actually have 'sdk' depend on 'prepare-sdk'
>   - anonymise the generated tarball

Applied to next, thanks!

Thomas

Patch

diff --git a/Makefile b/Makefile
index 37c7072c7c..e13e5eaefa 100644
--- a/Makefile
+++ b/Makefile
@@ -573,8 +573,8 @@  prepare: $(BUILD_DIR)/buildroot-config/auto.conf
 .PHONY: world
 world: target-post-image
 
-.PHONY: sdk
-sdk: world
+.PHONY: prepare-sdk
+prepare-sdk: world
 	@$(call MESSAGE,"Rendering the SDK relocatable")
 	$(TOPDIR)/support/scripts/fix-rpath host
 	$(TOPDIR)/support/scripts/fix-rpath staging
@@ -582,6 +582,17 @@  sdk: world
 	mkdir -p $(HOST_DIR)/share/buildroot
 	echo $(HOST_DIR) > $(HOST_DIR)/share/buildroot/sdk-location
 
+BR2_SDK_PREFIX ?= $(GNU_TARGET_NAME)_sdk-buildroot
+.PHONY: sdk
+sdk: prepare-sdk $(BR2_TAR_HOST_DEPENDENCY)
+	@$(call MESSAGE,"Generating SDK tarball")
+	$(if $(BR2_SDK_PREFIX),,$(error BR2_SDK_PREFIX can not be empty))
+	$(Q)mkdir -p $(BINARIES_DIR)
+	$(TAR) czf "$(BINARIES_DIR)/$(BR2_SDK_PREFIX).tar.gz" \
+		--owner=0 --group=0 --numeric-owner \
+		--transform='s#^\.#$(BR2_SDK_PREFIX)#' \
+		-C $(HOST_DIR) "."
+
 # Populating the staging with the base directories is handled by the skeleton package
 $(STAGING_DIR):
 	@mkdir -p $(STAGING_DIR)
diff --git a/docs/manual/using-buildroot-toolchain.txt b/docs/manual/using-buildroot-toolchain.txt
index 3246dc2411..0c0c35fced 100644
--- a/docs/manual/using-buildroot-toolchain.txt
+++ b/docs/manual/using-buildroot-toolchain.txt
@@ -12,15 +12,23 @@  The toolchain generated by Buildroot is located by default in
 +output/host/bin/+ to your PATH environment variable and then to
 use +ARCH-linux-gcc+, +ARCH-linux-objdump+, +ARCH-linux-ld+, etc.
 
-It is possible to relocate the toolchain, this allows to distribute
-the toolchain to other developers to build applications for your
-target. To achieve this:
+Alternatively, Buildroot can also export the toolchain and the development
+files of all selected packages, as an SDK, by running the command
++make sdk+. This generates a tarball of the content of the host directory
++output/host/+, named +<TARGET-TUPLE>_sdk-buildroot.tar.gz+ (which can be
+overriden by setting the environment variable +BR2_SDK_PREFIX+) and
+located in the output directory +output/images/+.
 
-* run +make sdk+, which prepares the toolchain to be relocatable;
-* tarball the contents of the +output/host+ directory;
-* distribute the resulting tarball.
+This tarball can then be distributed to application developers, when
+they want to develop their applications that are not (yet) packaged as
+a Buildroot package.
 
-Once the toolchain is installed to the new location, the user must run
-the +relocate-sdk.sh+ script to make sure all paths are updated with
-the new location.
+Upon extracting the SDK tarball, the user must run the script
++relocate-sdk.sh+ (located at the top directory of the SDK), to make
+sure all paths are updated with the new location.
 
+Alternatively, if you just want to prepare the SDK without generating
+the tarball (e.g. because you will just be moving the +host+ directory,
+or will be generating the tarball on your own), Buildroot also allows
+you to just prepare the SDK with +make prepare-sdk+ without actually
+generating a tarball.