[04/14,v4] package/skeleton: split out into skeleton-custom

For the custom skeleton, we practicaly do nothing, except ensure it
contains the basic, required directories, and that those are properly
setup wrt. merged /usr.

Furthermore, our current skeleton is not fit for systemd, and we'll
have to split things out into various skeletons.

So, off-load the custom skeleton into its own package.

Thus, the existing skeleton package is now limited to:

  - when using our default skeleton, install and tweak it properly;

  - when using a custom skeleton, do nothing except for depending on
    the skeleton-custom package.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>

---
Note: this calls for making skeleton a virtual package. This will come
in follwup patches, after skeleton has lost all its code to the various
skeleton packages.

---
Changes v3 -> v4:
  - lib{32.64} are also tweaked in the skeleton  (Arnout)
---
 package/Config.in                          |   1 +
 package/skeleton-custom/Config.in          |   3 +
 package/skeleton-custom/skeleton-custom.mk | 105 +++++++++++++++++++++++++++++
 package/skeleton/skeleton.mk               |  51 +-------------
 system/Config.in                           |   2 +-
 5 files changed, 112 insertions(+), 50 deletions(-)
 create mode 100644 package/skeleton-custom/Config.in
 create mode 100644 package/skeleton-custom/skeleton-custom.mk

Hi Yann,

On 25-07-17 23:14, Yann E. MORIN wrote:
> For the custom skeleton, we practicaly do nothing, except ensure it
                              do practically

> contains the basic, required directories, and that those are properly
> setup wrt. merged /usr.
[snip]

 I don't like this: you are doing *a whole lot* more than just moving stuff from
skeleton to skeleton-custom (and renaming the variables):

> +SKELETON_CUSTOM_LIB_INODE = $(shell stat -c '%i' $(SKELETON_CUSTOM_PATH)/lib/. 2>/dev/null)
> -SKELETON_LIB_INODE = $(shell stat -c '%i' $(SKELETON_PATH)/lib/.)

> +# Ensure that the custom skeleton has /lib, /bin and /sbin, and their
> +# /usr counterparts
> +ifeq ($(SKELETON_CUSTOM_LIB_INODE),)
> +SKELETON_CUSTOM_MISSING_DIRS += /lib
> +endif

> +# For a merged /usr, ensure that /lib, /bin and /sbin and their /usr
> +# counterparts are appropriately setup as symlinks ones to the others.

> +ifneq ($(SKELETON_CUSTOM_LIB_INODE),$(SKELETON_CUSTOM_USR_LIB_INODE))
> +SKELETON_CUSTOM_NOT_MERGED_USR_DIRS += /lib
> +endif
> -ifneq ($(SKELETON_LIB_INODE),$(SKELETON_USR_LIB_INODE))
> -SKELETON_CUSTOM_NOT_MERGED_USR += /lib
> -endif

> +ifeq ($(BR2_PACKAGE_SKELETON_CUSTOM)$(BR_BUILDING),yy)
> +ifneq ($(SKELETON_CUSTOM_MISSING_DIRS),)
> +$(error The custom skeleton in $(SKELETON_CUSTOM_PATH) is \
> +	missing those directories or symlinks: \
> +	$(SKELETON_CUSTOM_MISSING_DIRS))
> +endif

 Now, I understand that it's tricky to rebase things correctly, and splitting
this up is anyway mainly relevant for review, so I've just updated the commit
message to reflect all these changes.

> +$(error The custom skeleton in $(SKELETON_PATH) is not \
> +       using a merged /usr for the following directories: \
> +       $(SKELETON_CUSTOM_NOT_MERGED_USR_DIRS))

 Spaces should be tab.

> +# Provided by the 'skeleton' package:
> +# - SKELETON_RSYNC
> +# - SKELETON_LIB_SYMLINK

 As mentioned in an earlier review, this is in system now and it's quite clear
that SYSTEM_RSYNC comes from system.mk, so I've removed this.


 I've fixed all that and was going to apply, but then I noticed that patch 1
breaks our tests, so I refrained from pushing it. You can fetch it from
https://gitlab.com/arnout/buildroot (master branch; will be rebased away in a
couple of days).

 Regards,
 Arnout

On 26-07-17 02:16, Arnout Vandecappelle wrote:
>  I've fixed all that and was going to apply, but then I noticed that patch 1
> breaks our tests, so I refrained from pushing it. You can fetch it from
> https://gitlab.com/arnout/buildroot (master branch; will be rebased away in a
> couple of days).

 So I pushed this commit with the runtime testing patch 11 on top of to gitlab
to see what would happen, and the systemd test fails. It seems the getty on
ttyAMA0 can't be started. Is that to be expected, due to the /var/log->/tmp symlink?

 You can inspect the output on [1], but it's not saying much.

 Regards,
 Arnout

[1]
https://gitlab.com/arnout/buildroot/-/jobs/24489865/artifacts/raw/test-output/TestInitSystemSystemdRwFull-run.log

Hello,

On Wed, Jul 26, 2017 at 08:49 AM, Arnout Vandecappelle wrote:

> On 26-07-17 02:16, Arnout Vandecappelle wrote:
>>  I've fixed all that and was going to apply, but then I noticed that patch 1
>> breaks our tests, so I refrained from pushing it. You can fetch it from
>> https://gitlab.com/arnout/buildroot (master branch; will be rebased away in a
>> couple of days).
> 
>  So I pushed this commit with the runtime testing patch 11 on top of to gitlab
> to see what would happen, and the systemd test fails. It seems the getty on
> ttyAMA0 can't be started. Is that to be expected, due to the /var/log->/tmp symlink?
> 
>  You can inspect the output on [1], but it's not saying much.

It seems a timeout while waiting for the login prompt.

> [1]
> https://gitlab.com/arnout/buildroot/-/jobs/24489865/artifacts/raw/test-output/TestInitSystemSystemdRwFull-run.log

This test you triggered ran on the runner e11ae361.

I borrowed your commit (e8e38e9e) and did some tests.

I got lucky and the test I triggered ran on the shared runner 4e4528ca and
passed [2].
Then I retriggered it to run on my private runner, while I was also running
a resource intensive task (actually I was running the same test in the same
commit locally with the default -j 0) and the test in the runner failed [3]
(while the local test passed).

[2] https://gitlab.com/RicardoMartincoski/buildroot/-/jobs/25090126
[3] https://gitlab.com/RicardoMartincoski/buildroot/-/jobs/25101360


TestIPythonPy3 for instance seems to be have tidy timeout for math_floor_test.
Its first execution [4] ran in the (free, as in free beer!) elastic runner from
gitlab and failed. The second execution [5] ran in my private runner (a docker
in my personal computer) and passed.

[4] https://gitlab.com/RicardoMartincoski/buildroot/-/jobs/25090131
[5] https://gitlab.com/RicardoMartincoski/buildroot/-/jobs/25099593


Concerning the test infra (I say that including gitlab) we should choose a path
to follow:
1) Use private runners
2) Adapt timeouts when a test sporadically fails and keep using the gitlab
runners ([6] shows that we already use them)

[6] https://gitlab.com/buildroot.org/buildroot/-/jobs/24184768

I would go with option 2. We are not doing CI for now, so sporadic failures are
not a big concern IMO.

Regards,
Ricardo

Hello,

On Fri, 28 Jul 2017 00:02:34 -0300, Ricardo Martincoski wrote:

> This test you triggered ran on the runner e11ae361.
> 
> I borrowed your commit (e8e38e9e) and did some tests.
> 
> I got lucky and the test I triggered ran on the shared runner 4e4528ca and
> passed [2].
> Then I retriggered it to run on my private runner, while I was also running
> a resource intensive task (actually I was running the same test in the same
> commit locally with the default -j 0) and the test in the runner failed [3]
> (while the local test passed).
> 
> [2] https://gitlab.com/RicardoMartincoski/buildroot/-/jobs/25090126
> [3] https://gitlab.com/RicardoMartincoski/buildroot/-/jobs/25101360
> 
> 
> TestIPythonPy3 for instance seems to be have tidy timeout for math_floor_test.
> Its first execution [4] ran in the (free, as in free beer!) elastic runner from
> gitlab and failed. The second execution [5] ran in my private runner (a docker
> in my personal computer) and passed.
> 
> [4] https://gitlab.com/RicardoMartincoski/buildroot/-/jobs/25090131
> [5] https://gitlab.com/RicardoMartincoski/buildroot/-/jobs/25099593
> 
> Concerning the test infra (I say that including gitlab) we should choose a path
> to follow:
> 1) Use private runners

So it is possible to "contribute" as a private runner for your own CI
projects hosted on the public Gitlab ?

> 2) Adapt timeouts when a test sporadically fails and keep using the gitlab
> runners ([6] shows that we already use them)
> 
> [6] https://gitlab.com/buildroot.org/buildroot/-/jobs/24184768
> 
> I would go with option 2. We are not doing CI for now, so sporadic failures are
> not a big concern IMO.

The idea of using Gitlab is (amongst other things) to leverage the
public runners. That being said, sporadic failures are really not nice,
because they always cause confusion.

Why don't we significantly increase the timeout, like to several
minutes? In most cases, the test will be successful, and the expected
output will arrive in a few seconds. It's only when there is a real
problem in the test that we will hit the timeout, but in this case,
having the test take a few more minutes is not a big deal.

Best regards,

Thomas

On 28-07-17 08:18, Thomas Petazzoni wrote:
> Hello,
> 
> On Fri, 28 Jul 2017 00:02:34 -0300, Ricardo Martincoski wrote:
[snip]
>> Concerning the test infra (I say that including gitlab) we should choose a path
>> to follow:
>> 1) Use private runners
> 
> So it is possible to "contribute" as a private runner for your own CI
> projects hosted on the public Gitlab ?

 Absolutely! It is in fact more or less intended to be used that way, i.e. as a
company you let gitlab.com handle your repository, but you provide your own
machines to run tests.

 That is certainly one way to go. It would also allow us to do some additional
things that are currently impossible, e.g. generating the buildroot-base docker
image.


>> 2) Adapt timeouts when a test sporadically fails and keep using the gitlab
>> runners ([6] shows that we already use them)
>>
>> [6] https://gitlab.com/buildroot.org/buildroot/-/jobs/24184768
>>
>> I would go with option 2. We are not doing CI for now, so sporadic failures are
>> not a big concern IMO.
> 
> The idea of using Gitlab is (amongst other things) to leverage the
> public runners. That being said, sporadic failures are really not nice,
> because they always cause confusion.

 Agreed.


> Why don't we significantly increase the timeout, like to several
> minutes? In most cases, the test will be successful, and the expected
> output will arrive in a few seconds. It's only when there is a real
> problem in the test that we will hit the timeout, but in this case,
> having the test take a few more minutes is not a big deal.

 That's actually a very good idea.

 However, I do wonder if that will really help. Looking at the log of [1] again,
it really looks like for 15 seconds, the *only* thing going on was starting
getty. That makes me think that there must be something else going on than just
a timeout (increasing the timeout was my first thought as well).

 As it stands, things are rather hard to debug. Perhaps we should do something
like store the modified rootfs as an artefact so it can be downloaded and you
can extract the journal from it?

 Regards,
 Arnout


[1]
https://gitlab.com/arnout/buildroot/-/jobs/24489865/artifacts/raw/test-output/TestInitSystemSystemdRwFull-run.log