[LEDE-DEV] build: Fix unpacking of overlaid packages

Since commit 5af113eb7c0f6e4b2d09e536e1a2adc4e2d6071d, LEDE's build
system contains overlay functionality, which allows to locally amend
properties of packages. When one wants to overlay certain properties
such as PKG_VERSION, the unpack command (UNPACK_CMD) uses the original
value rather than the overlaid one and unpacking fails.

Fix that problem by replacing simply expanded variable with
recursively expanded variable. This ensures that the unpack command
uses the overlaid values, because the variable expansion is deferred
after the overlay is loaded.

Signed-off-by: Michal Sojka <sojkam1@fel.cvut.cz>
Signed-off-by: Michal Vokáč <michal.vokac@cvut.cz>
---
 include/unpack.mk | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

On Fri, Mar 17 2017, Michal Sojka wrote:
> Since commit 5af113eb7c0f6e4b2d09e536e1a2adc4e2d6071d, LEDE's build
> system contains overlay functionality, which allows to locally amend
> properties of packages. When one wants to overlay certain properties
> such as PKG_VERSION, the unpack command (UNPACK_CMD) uses the original
> value rather than the overlaid one and unpacking fails.

Hmm, there are more places where is this overlay functionality broken.
I'll send another patch that makes it work for me, but there are places,
which cannot be fixed that easy. One of them is the -iremap CFLAG, which
is appended to TARGET_CFLAGS with "+=" operator, which expands to the
non-overlaid path.

-Michal

On 18 March 2017 at 06:54, Michal Sojka <sojkam1@fel.cvut.cz> wrote:
> On Fri, Mar 17 2017, Michal Sojka wrote:
>> Since commit 5af113eb7c0f6e4b2d09e536e1a2adc4e2d6071d, LEDE's build
>> system contains overlay functionality, which allows to locally amend
>> properties of packages. When one wants to overlay certain properties
>> such as PKG_VERSION, the unpack command (UNPACK_CMD) uses the original
>> value rather than the overlaid one and unpacking fails.
>
> Hmm, there are more places where is this overlay functionality broken.
> I'll send another patch that makes it work for me, but there are places,
> which cannot be fixed that easy. One of them is the -iremap CFLAG, which
> is appended to TARGET_CFLAGS with "+=" operator, which expands to the
> non-overlaid path.
>
> -Michal

The change will incur too much implementation details and is very
likely hard to maintain later.

In your case, I would recommend either maintaining your own branch, or
maintain a package feed for your needs and prefer it at feed install
time.

Regards,
                yousong

Hi Yousong,

On Thu, Mar 23 2017, Yousong Zhou wrote:
> On 18 March 2017 at 06:54, Michal Sojka <sojkam1@fel.cvut.cz> wrote:
>> On Fri, Mar 17 2017, Michal Sojka wrote:
>>> Since commit 5af113eb7c0f6e4b2d09e536e1a2adc4e2d6071d, LEDE's build
>>> system contains overlay functionality, which allows to locally amend
>>> properties of packages. When one wants to overlay certain properties
>>> such as PKG_VERSION, the unpack command (UNPACK_CMD) uses the original
>>> value rather than the overlaid one and unpacking fails.
>>
>> Hmm, there are more places where is this overlay functionality broken.
>> I'll send another patch that makes it work for me, but there are places,
>> which cannot be fixed that easy. One of them is the -iremap CFLAG, which
>> is appended to TARGET_CFLAGS with "+=" operator, which expands to the
>> non-overlaid path.
>>
>> -Michal
>
> The change will incur too much implementation details and is very
> likely hard to maintain later.

Yes, I think the same. IMHO, the best thing to do would be to remove the
package overlay code, because it either does not work at all or it works
only in a few very limited (and undocumented) cases.

> In your case, I would recommend either maintaining your own branch, or
> maintain a package feed for your needs and prefer it at feed install
> time.

What do you mean by "prefer at feed install time"?

Thanks
-Michal

On 23 March 2017 at 15:40, Michal Sojka <sojkam1@fel.cvut.cz> wrote:
> Hi Yousong,
>
> On Thu, Mar 23 2017, Yousong Zhou wrote:
>> On 18 March 2017 at 06:54, Michal Sojka <sojkam1@fel.cvut.cz> wrote:
>>> On Fri, Mar 17 2017, Michal Sojka wrote:
>>>> Since commit 5af113eb7c0f6e4b2d09e536e1a2adc4e2d6071d, LEDE's build
>>>> system contains overlay functionality, which allows to locally amend
>>>> properties of packages. When one wants to overlay certain properties
>>>> such as PKG_VERSION, the unpack command (UNPACK_CMD) uses the original
>>>> value rather than the overlaid one and unpacking fails.
>>>
>>> Hmm, there are more places where is this overlay functionality broken.
>>> I'll send another patch that makes it work for me, but there are places,
>>> which cannot be fixed that easy. One of them is the -iremap CFLAG, which
>>> is appended to TARGET_CFLAGS with "+=" operator, which expands to the
>>> non-overlaid path.
>>>
>>> -Michal
>>
>> The change will incur too much implementation details and is very
>> likely hard to maintain later.
>
> Yes, I think the same. IMHO, the best thing to do would be to remove the
> package overlay code, because it either does not work at all or it works
> only in a few very limited (and undocumented) cases.

I am not yet a user of it yet but the past experience [1] gave me give
me the impression that doing it right was not easy and required a fair
amount of trial and errors

 [1] https://gist.github.com/yousong/1df4fcee324dd6b6095e6b551e2806a9

>
>> In your case, I would recommend either maintaining your own branch, or
>> maintain a package feed for your needs and prefer it at feed install
>> time.
>
> What do you mean by "prefer at feed install time"?
>

I mean "-p <feedname>: Prefer this feed when installing packages." of
scripts/feeds

                yousong

On 2017-03-23 08:40, Michal Sojka wrote:
> Hi Yousong,
> 
> On Thu, Mar 23 2017, Yousong Zhou wrote:
>> On 18 March 2017 at 06:54, Michal Sojka <sojkam1@fel.cvut.cz> wrote:
>>> On Fri, Mar 17 2017, Michal Sojka wrote:
>>>> Since commit 5af113eb7c0f6e4b2d09e536e1a2adc4e2d6071d, LEDE's build
>>>> system contains overlay functionality, which allows to locally amend
>>>> properties of packages. When one wants to overlay certain properties
>>>> such as PKG_VERSION, the unpack command (UNPACK_CMD) uses the original
>>>> value rather than the overlaid one and unpacking fails.
>>>
>>> Hmm, there are more places where is this overlay functionality broken.
>>> I'll send another patch that makes it work for me, but there are places,
>>> which cannot be fixed that easy. One of them is the -iremap CFLAG, which
>>> is appended to TARGET_CFLAGS with "+=" operator, which expands to the
>>> non-overlaid path.
>>>
>>> -Michal
>>
>> The change will incur too much implementation details and is very
>> likely hard to maintain later.
> 
> Yes, I think the same. IMHO, the best thing to do would be to remove the
> package overlay code, because it either does not work at all or it works
> only in a few very limited (and undocumented) cases.
I agree. I've pushed a commit to my staging tree that removes package
overlay support.

- Felix