diff mbox

[1/7] Add <PKG>_STRIP_COMPONENTS for packages with non-standard tarballs

Message ID 1436624110-20428-1-git-send-email-luca@lucaceresoli.net
State Accepted
Headers show

Commit Message

Luca Ceresoli July 11, 2015, 2:15 p.m. UTC
Usually package tarballs contain only a directory named
"<pkg-name>-<pkg-version>" which contains the actual source tree. To properly
extract the source files in $(BUILD_DIR), Buildroot passes the
--strip-components=1 option to tar.

Unfortunately a few packages ship in a non-standard way, with no root
directory or with more than one. In Buildroot these packages must be handled by
overriding the <PKG>_DOWNLOAD_CMDS.

As the number of such packages is growing, introduce a <PKG>_STRIP_COMPONENTS
variable for packages to request a number of components to strip different
from 1.

Signed-off-by: Luca Ceresoli <luca@lucaceresoli.net>
Cc: Arnout Vandecappelle <arnout@mind.be>
---
 package/pkg-generic.mk | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)

Comments

Arnout Vandecappelle July 11, 2015, 10:28 p.m. UTC | #1
On 07/11/15 16:15, Luca Ceresoli wrote:
> Usually package tarballs contain only a directory named
> "<pkg-name>-<pkg-version>" which contains the actual source tree. To properly
> extract the source files in $(BUILD_DIR), Buildroot passes the
> --strip-components=1 option to tar.
> 
> Unfortunately a few packages ship in a non-standard way, with no root
> directory or with more than one. In Buildroot these packages must be handled by
> overriding the <PKG>_DOWNLOAD_CMDS.
> 
> As the number of such packages is growing, introduce a <PKG>_STRIP_COMPONENTS
> variable for packages to request a number of components to strip different
> from 1.
> 
> Signed-off-by: Luca Ceresoli <luca@lucaceresoli.net>
> Cc: Arnout Vandecappelle <arnout@mind.be>

Acked-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>


 Regards,
 Arnout

> ---
>  package/pkg-generic.mk | 10 +++++++++-
>  1 file changed, 9 insertions(+), 1 deletion(-)
> 
> diff --git a/package/pkg-generic.mk b/package/pkg-generic.mk
> index 9fe01b8d4d12..8aad1b4acc23 100644
> --- a/package/pkg-generic.mk
> +++ b/package/pkg-generic.mk
> @@ -322,6 +322,14 @@ ifndef $(2)_SUBDIR
>   endif
>  endif
>  
> +ifndef $(2)_STRIP_COMPONENTS
> + ifdef $(3)_STRIP_COMPONENTS
> +  $(2)_STRIP_COMPONENTS = $$($(3)_STRIP_COMPONENTS)
> + else
> +  $(2)_STRIP_COMPONENTS ?= 1
> + endif
> +endif
> +
>  $(2)_SRCDIR		       = $$($(2)_DIR)/$$($(2)_SUBDIR)
>  $(2)_BUILDDIR		       ?= $$($(2)_SRCDIR)
>  
> @@ -431,7 +439,7 @@ $(2)_TARGET_DIRCLEAN =		$$($(2)_DIR)/.stamp_dircleaned
>  # default extract command
>  $(2)_EXTRACT_CMDS ?= \
>  	$$(if $$($(2)_SOURCE),$$(INFLATE$$(suffix $$($(2)_SOURCE))) $$(DL_DIR)/$$($(2)_SOURCE) | \
> -	$$(TAR) $$(TAR_STRIP_COMPONENTS)=1 -C $$($(2)_DIR) $$(TAR_OPTIONS) -)
> +	$$(TAR) $$(TAR_STRIP_COMPONENTS)=$$($(2)_STRIP_COMPONENTS) -C $$($(2)_DIR) $$(TAR_OPTIONS) -)
>  
>  # pre/post-steps hooks
>  $(2)_PRE_DOWNLOAD_HOOKS         ?=
>
Thomas Petazzoni July 11, 2015, 10:52 p.m. UTC | #2
Dear Luca Ceresoli,

On Sat, 11 Jul 2015 16:15:04 +0200, Luca Ceresoli wrote:
> Usually package tarballs contain only a directory named
> "<pkg-name>-<pkg-version>" which contains the actual source tree. To properly
> extract the source files in $(BUILD_DIR), Buildroot passes the
> --strip-components=1 option to tar.
> 
> Unfortunately a few packages ship in a non-standard way, with no root
> directory or with more than one. In Buildroot these packages must be handled by
> overriding the <PKG>_DOWNLOAD_CMDS.
> 
> As the number of such packages is growing, introduce a <PKG>_STRIP_COMPONENTS
> variable for packages to request a number of components to strip different
> from 1.
> 
> Signed-off-by: Luca Ceresoli <luca@lucaceresoli.net>
> Cc: Arnout Vandecappelle <arnout@mind.be>
> ---
>  package/pkg-generic.mk | 10 +++++++++-
>  1 file changed, 9 insertions(+), 1 deletion(-)

Patches 1 to 6 applied.

Best regards,

Thomas
Arnout Vandecappelle July 11, 2015, 10:56 p.m. UTC | #3
Since the cover letter is missing, I'll reply to this one...

On 07/11/15 16:15, Luca Ceresoli wrote:
> Usually package tarballs contain only a directory named
> "<pkg-name>-<pkg-version>" which contains the actual source tree. To properly
> extract the source files in $(BUILD_DIR), Buildroot passes the
> --strip-components=1 option to tar.

 Great work.

 I think the following packages could also be converted:

flite
nvidia-tegra23-codecs

but admittedly, they look a little more complicated.

 Regards,
 Arnout
Thomas Petazzoni July 12, 2015, 11:44 a.m. UTC | #4
Dear Luca Ceresoli,

On Sat, 11 Jul 2015 16:15:04 +0200, Luca Ceresoli wrote:
> Usually package tarballs contain only a directory named
> "<pkg-name>-<pkg-version>" which contains the actual source tree. To properly
> extract the source files in $(BUILD_DIR), Buildroot passes the
> --strip-components=1 option to tar.
> 
> Unfortunately a few packages ship in a non-standard way, with no root
> directory or with more than one. In Buildroot these packages must be handled by
> overriding the <PKG>_DOWNLOAD_CMDS.
> 
> As the number of such packages is growing, introduce a <PKG>_STRIP_COMPONENTS
> variable for packages to request a number of components to strip different
> from 1.
> 
> Signed-off-by: Luca Ceresoli <luca@lucaceresoli.net>
> Cc: Arnout Vandecappelle <arnout@mind.be>
> ---
>  package/pkg-generic.mk | 10 +++++++++-
>  1 file changed, 9 insertions(+), 1 deletion(-)

This actually triggered a failure in the build of the tar package:

   http://autobuild.buildroot.org/results/ae2/ae20df67f99f75b1ba5d5b7316ad265d66f3aa66/build-end.log

We have already fixed it, it was quite interesting:

   http://git.buildroot.net/buildroot/commit/?id=ef7cc99c7a432aa3929015183f7c7476b7cd1b68

Thomas
Luca Ceresoli July 12, 2015, 1:30 p.m. UTC | #5
Dear Thomas,

Thomas Petazzoni wrote:
> Dear Luca Ceresoli,
>
> On Sat, 11 Jul 2015 16:15:04 +0200, Luca Ceresoli wrote:
>> Usually package tarballs contain only a directory named
>> "<pkg-name>-<pkg-version>" which contains the actual source tree. To properly
>> extract the source files in $(BUILD_DIR), Buildroot passes the
>> --strip-components=1 option to tar.
>>
>> Unfortunately a few packages ship in a non-standard way, with no root
>> directory or with more than one. In Buildroot these packages must be handled by
>> overriding the <PKG>_DOWNLOAD_CMDS.
>>
>> As the number of such packages is growing, introduce a <PKG>_STRIP_COMPONENTS
>> variable for packages to request a number of components to strip different
>> from 1.
>>
>> Signed-off-by: Luca Ceresoli <luca@lucaceresoli.net>
>> Cc: Arnout Vandecappelle <arnout@mind.be>
>> ---
>>   package/pkg-generic.mk | 10 +++++++++-
>>   1 file changed, 9 insertions(+), 1 deletion(-)
>
> This actually triggered a failure in the build of the tar package:
>
>     http://autobuild.buildroot.org/results/ae2/ae20df67f99f75b1ba5d5b7316ad265d66f3aa66/build-end.log
>
> We have already fixed it, it was quite interesting:
>
>     http://git.buildroot.net/buildroot/commit/?id=ef7cc99c7a432aa3929015183f7c7476b7cd1b68

Ouch. O:-)

Thanks for fixing it so quickly!
Luca Ceresoli July 12, 2015, 2:26 p.m. UTC | #6
Dear Arnout,

Arnout Vandecappelle wrote:
>   Since the cover letter is missing, I'll reply to this one...
>
> On 07/11/15 16:15, Luca Ceresoli wrote:
>> Usually package tarballs contain only a directory named
>> "<pkg-name>-<pkg-version>" which contains the actual source tree. To properly
>> extract the source files in $(BUILD_DIR), Buildroot passes the
>> --strip-components=1 option to tar.
>
>   Great work.
>
>   I think the following packages could also be converted:
>
> flite

I gave it a try. Here it is, http://patchwork.ozlabs.org/patch/494116/
but I'm not sure it's exactly a lovely patch...
diff mbox

Patch

diff --git a/package/pkg-generic.mk b/package/pkg-generic.mk
index 9fe01b8d4d12..8aad1b4acc23 100644
--- a/package/pkg-generic.mk
+++ b/package/pkg-generic.mk
@@ -322,6 +322,14 @@  ifndef $(2)_SUBDIR
  endif
 endif
 
+ifndef $(2)_STRIP_COMPONENTS
+ ifdef $(3)_STRIP_COMPONENTS
+  $(2)_STRIP_COMPONENTS = $$($(3)_STRIP_COMPONENTS)
+ else
+  $(2)_STRIP_COMPONENTS ?= 1
+ endif
+endif
+
 $(2)_SRCDIR		       = $$($(2)_DIR)/$$($(2)_SUBDIR)
 $(2)_BUILDDIR		       ?= $$($(2)_SRCDIR)
 
@@ -431,7 +439,7 @@  $(2)_TARGET_DIRCLEAN =		$$($(2)_DIR)/.stamp_dircleaned
 # default extract command
 $(2)_EXTRACT_CMDS ?= \
 	$$(if $$($(2)_SOURCE),$$(INFLATE$$(suffix $$($(2)_SOURCE))) $$(DL_DIR)/$$($(2)_SOURCE) | \
-	$$(TAR) $$(TAR_STRIP_COMPONENTS)=1 -C $$($(2)_DIR) $$(TAR_OPTIONS) -)
+	$$(TAR) $$(TAR_STRIP_COMPONENTS)=$$($(2)_STRIP_COMPONENTS) -C $$($(2)_DIR) $$(TAR_OPTIONS) -)
 
 # pre/post-steps hooks
 $(2)_PRE_DOWNLOAD_HOOKS         ?=