[LEDE-DEV] build: Fix unpacking of overlaid packages

Submitted by Michal Sojka on March 17, 2017, 2:50 p.m.

Details

Message ID 20170317145051.24279-1-sojkam1@fel.cvut.cz
State Rejected
Headers show

Commit Message

Michal Sojka March 17, 2017, 2:50 p.m.
Since commit 5af113eb7c0f6e4b2d09e536e1a2adc4e2d6071d, LEDE's build
system contains overlay functionality, which allows to locally amend
properties of packages. When one wants to overlay certain properties
such as PKG_VERSION, the unpack command (UNPACK_CMD) uses the original
value rather than the overlaid one and unpacking fails.

Fix that problem by replacing simply expanded variable with
recursively expanded variable. This ensures that the unpack command
uses the overlaid values, because the variable expansion is deferred
after the overlay is loaded.

Signed-off-by: Michal Sojka <sojkam1@fel.cvut.cz>
Signed-off-by: Michal Vokáč <michal.vokac@cvut.cz>
---
 include/unpack.mk | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

Comments

Michal Sojka March 17, 2017, 10:54 p.m.
On Fri, Mar 17 2017, Michal Sojka wrote:
> Since commit 5af113eb7c0f6e4b2d09e536e1a2adc4e2d6071d, LEDE's build
> system contains overlay functionality, which allows to locally amend
> properties of packages. When one wants to overlay certain properties
> such as PKG_VERSION, the unpack command (UNPACK_CMD) uses the original
> value rather than the overlaid one and unpacking fails.

Hmm, there are more places where is this overlay functionality broken.
I'll send another patch that makes it work for me, but there are places,
which cannot be fixed that easy. One of them is the -iremap CFLAG, which
is appended to TARGET_CFLAGS with "+=" operator, which expands to the
non-overlaid path.

-Michal
Yousong Zhou March 23, 2017, 5:23 a.m.
On 18 March 2017 at 06:54, Michal Sojka <sojkam1@fel.cvut.cz> wrote:
> On Fri, Mar 17 2017, Michal Sojka wrote:
>> Since commit 5af113eb7c0f6e4b2d09e536e1a2adc4e2d6071d, LEDE's build
>> system contains overlay functionality, which allows to locally amend
>> properties of packages. When one wants to overlay certain properties
>> such as PKG_VERSION, the unpack command (UNPACK_CMD) uses the original
>> value rather than the overlaid one and unpacking fails.
>
> Hmm, there are more places where is this overlay functionality broken.
> I'll send another patch that makes it work for me, but there are places,
> which cannot be fixed that easy. One of them is the -iremap CFLAG, which
> is appended to TARGET_CFLAGS with "+=" operator, which expands to the
> non-overlaid path.
>
> -Michal

The change will incur too much implementation details and is very
likely hard to maintain later.

In your case, I would recommend either maintaining your own branch, or
maintain a package feed for your needs and prefer it at feed install
time.

Regards,
                yousong
Michal Sojka March 23, 2017, 7:40 a.m.
Hi Yousong,

On Thu, Mar 23 2017, Yousong Zhou wrote:
> On 18 March 2017 at 06:54, Michal Sojka <sojkam1@fel.cvut.cz> wrote:
>> On Fri, Mar 17 2017, Michal Sojka wrote:
>>> Since commit 5af113eb7c0f6e4b2d09e536e1a2adc4e2d6071d, LEDE's build
>>> system contains overlay functionality, which allows to locally amend
>>> properties of packages. When one wants to overlay certain properties
>>> such as PKG_VERSION, the unpack command (UNPACK_CMD) uses the original
>>> value rather than the overlaid one and unpacking fails.
>>
>> Hmm, there are more places where is this overlay functionality broken.
>> I'll send another patch that makes it work for me, but there are places,
>> which cannot be fixed that easy. One of them is the -iremap CFLAG, which
>> is appended to TARGET_CFLAGS with "+=" operator, which expands to the
>> non-overlaid path.
>>
>> -Michal
>
> The change will incur too much implementation details and is very
> likely hard to maintain later.

Yes, I think the same. IMHO, the best thing to do would be to remove the
package overlay code, because it either does not work at all or it works
only in a few very limited (and undocumented) cases.

> In your case, I would recommend either maintaining your own branch, or
> maintain a package feed for your needs and prefer it at feed install
> time.

What do you mean by "prefer at feed install time"?

Thanks
-Michal
Yousong Zhou March 23, 2017, 8:04 a.m.
On 23 March 2017 at 15:40, Michal Sojka <sojkam1@fel.cvut.cz> wrote:
> Hi Yousong,
>
> On Thu, Mar 23 2017, Yousong Zhou wrote:
>> On 18 March 2017 at 06:54, Michal Sojka <sojkam1@fel.cvut.cz> wrote:
>>> On Fri, Mar 17 2017, Michal Sojka wrote:
>>>> Since commit 5af113eb7c0f6e4b2d09e536e1a2adc4e2d6071d, LEDE's build
>>>> system contains overlay functionality, which allows to locally amend
>>>> properties of packages. When one wants to overlay certain properties
>>>> such as PKG_VERSION, the unpack command (UNPACK_CMD) uses the original
>>>> value rather than the overlaid one and unpacking fails.
>>>
>>> Hmm, there are more places where is this overlay functionality broken.
>>> I'll send another patch that makes it work for me, but there are places,
>>> which cannot be fixed that easy. One of them is the -iremap CFLAG, which
>>> is appended to TARGET_CFLAGS with "+=" operator, which expands to the
>>> non-overlaid path.
>>>
>>> -Michal
>>
>> The change will incur too much implementation details and is very
>> likely hard to maintain later.
>
> Yes, I think the same. IMHO, the best thing to do would be to remove the
> package overlay code, because it either does not work at all or it works
> only in a few very limited (and undocumented) cases.

I am not yet a user of it yet but the past experience [1] gave me give
me the impression that doing it right was not easy and required a fair
amount of trial and errors

 [1] https://gist.github.com/yousong/1df4fcee324dd6b6095e6b551e2806a9

>
>> In your case, I would recommend either maintaining your own branch, or
>> maintain a package feed for your needs and prefer it at feed install
>> time.
>
> What do you mean by "prefer at feed install time"?
>

I mean "-p <feedname>: Prefer this feed when installing packages." of
scripts/feeds

                yousong
Felix Fietkau April 3, 2017, 10:37 a.m.
On 2017-03-23 08:40, Michal Sojka wrote:
> Hi Yousong,
> 
> On Thu, Mar 23 2017, Yousong Zhou wrote:
>> On 18 March 2017 at 06:54, Michal Sojka <sojkam1@fel.cvut.cz> wrote:
>>> On Fri, Mar 17 2017, Michal Sojka wrote:
>>>> Since commit 5af113eb7c0f6e4b2d09e536e1a2adc4e2d6071d, LEDE's build
>>>> system contains overlay functionality, which allows to locally amend
>>>> properties of packages. When one wants to overlay certain properties
>>>> such as PKG_VERSION, the unpack command (UNPACK_CMD) uses the original
>>>> value rather than the overlaid one and unpacking fails.
>>>
>>> Hmm, there are more places where is this overlay functionality broken.
>>> I'll send another patch that makes it work for me, but there are places,
>>> which cannot be fixed that easy. One of them is the -iremap CFLAG, which
>>> is appended to TARGET_CFLAGS with "+=" operator, which expands to the
>>> non-overlaid path.
>>>
>>> -Michal
>>
>> The change will incur too much implementation details and is very
>> likely hard to maintain later.
> 
> Yes, I think the same. IMHO, the best thing to do would be to remove the
> package overlay code, because it either does not work at all or it works
> only in a few very limited (and undocumented) cases.
I agree. I've pushed a commit to my staging tree that removes package
overlay support.

- Felix

Patch hide | download patch | download mbox

diff --git a/include/unpack.mk b/include/unpack.mk
index a139827490..6e4de7ec8e 100644
--- a/include/unpack.mk
+++ b/include/unpack.mk
@@ -21,15 +21,15 @@  ifeq ($(strip $(UNPACK_CMD)),)
 
     ifeq ($(filter gz tgz,$(EXT)),$(EXT))
       EXT:=$(call ext,$(PKG_SOURCE:.$(EXT)=))
-      DECOMPRESS_CMD:=gzip -dc $(DL_DIR)/$(PKG_SOURCE) |
+      DECOMPRESS_CMD=gzip -dc $(DL_DIR)/$(PKG_SOURCE) |
     endif
     ifeq ($(filter bzip2 bz2 bz tbz2 tbz,$(EXT)),$(EXT))
       EXT:=$(call ext,$(PKG_SOURCE:.$(EXT)=))
-      DECOMPRESS_CMD:=bzcat $(DL_DIR)/$(PKG_SOURCE) |
+      DECOMPRESS_CMD=bzcat $(DL_DIR)/$(PKG_SOURCE) |
     endif
     ifeq ($(filter xz txz,$(EXT)),$(EXT))
       EXT:=$(call ext,$(PKG_SOURCE:.$(EXT)=))
-      DECOMPRESS_CMD:=xzcat $(DL_DIR)/$(PKG_SOURCE) |
+      DECOMPRESS_CMD=xzcat $(DL_DIR)/$(PKG_SOURCE) |
     endif
     ifeq ($(filter tgz tbz tbz2 txz,$(EXT1)),$(EXT1))
       EXT:=tar