[2/2] gitlab-ci: fix pipelines

When gitlab prepares a job to run, it checks out the repository with a
non-root user, and spawns a container that runs as root, with some UID
mapping that makes the files be owned by root in the container. However,
our pipelines run as a nont-root user.

Commit bde165f7ad (.gitlab-ci.yml: update Docker image to use) updated
the docker image that is used to run in our pipelines.

That new image includes a git version that is stricter about the
ownership of the git tree it is acting in: git aborts in error when the
user running it does not own the repository.

We use `git ls-tree` quite a lot in our check-{flake8,package,symbols}
rules, so they all fail (in various ways).

To fix this, we either need to fix the ownership or tell git to ignore
the situation.

It is most probably impossible to change the ownership of the files: we
run as non-root,and the files belong to root (in the container). So
we're stuck.

The alternative, is to do as git suggest, and tell it to ignore the
situation. In a local setup, this woujld be very insecure, but in the
pipelines, this is in a throw-away container, wehre a single user exists
and is running, so we don't care much (if at all).

Add a global before_script that registers the git config to ignore
ownership issues in the buildroot repository; see [0] for the definition
of the CI_PROJECT_DIR variable. Note: unlike what is said in there, and
in [1], the value actually seen in CI_PROJECT_DIR is already prefixed
with CI_BUILDS_DIR (the documentation is unclear about that point).

[0] https://docs.gitlab.com/ee/ci/variables/predefined_variables.html
[1] https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-runners-section

Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Cc: Romain Naour <romain.naour@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
---
 support/misc/gitlab-ci.yml.in | 3 +++
 1 file changed, 3 insertions(+)

Hello Yann,

Is "fix pipelines" your WIP patch subject ?
"gitlab-ci: ignore ownership of the git tree" seems better?

Le 07/02/2023 à 22:33, Yann E. MORIN a écrit :
> When gitlab prepares a job to run, it checks out the repository with a
> non-root user, and spawns a container that runs as root, with some UID
> mapping that makes the files be owned by root in the container. However,
> our pipelines run as a nont-root user.
> 
> Commit bde165f7ad (.gitlab-ci.yml: update Docker image to use) updated
> the docker image that is used to run in our pipelines.
> 
> That new image includes a git version that is stricter about the
> ownership of the git tree it is acting in: git aborts in error when the
> user running it does not own the repository.
> 
> We use `git ls-tree` quite a lot in our check-{flake8,package,symbols}
> rules, so they all fail (in various ways).
> 
> To fix this, we either need to fix the ownership or tell git to ignore
> the situation.
> 
> It is most probably impossible to change the ownership of the files: we
> run as non-root,and the files belong to root (in the container). So
> we're stuck.
> 
> The alternative, is to do as git suggest, and tell it to ignore the
> situation. In a local setup, this woujld be very insecure, but in the
> pipelines, this is in a throw-away container, wehre a single user exists
> and is running, so we don't care much (if at all).
> 
> Add a global before_script that registers the git config to ignore
> ownership issues in the buildroot repository; see [0] for the definition
> of the CI_PROJECT_DIR variable. Note: unlike what is said in there, and
> in [1], the value actually seen in CI_PROJECT_DIR is already prefixed
> with CI_BUILDS_DIR (the documentation is unclear about that point).
> 
> [0] https://docs.gitlab.com/ee/ci/variables/predefined_variables.html
> [1] https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-runners-section

Reviewed-by: Romain Naour <romain.naour@smile.fr>


> 
> Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
> Cc: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
> Cc: Romain Naour <romain.naour@gmail.com>
> Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
> ---
>  support/misc/gitlab-ci.yml.in | 3 +++
>  1 file changed, 3 insertions(+)
> 
> diff --git a/support/misc/gitlab-ci.yml.in b/support/misc/gitlab-ci.yml.in
> index 9c1faf0d5f..38aca31fb5 100644
> --- a/support/misc/gitlab-ci.yml.in
> +++ b/support/misc/gitlab-ci.yml.in
> @@ -1,3 +1,6 @@
> +before_script:
> +  - git config --global --add safe.directory ${CI_PROJECT_DIR}

Note: I checked if we have the issue using the utils/docker-run directly,
enabling safe.directory is not necessary.

Best regards,
Romain


> +
>  .check-check-package_base:
>      script:
>          - python3 -m pytest -v utils/checkpackagelib/

Romain, All,

On 2023-02-07 22:57 +0100, Romain Naour spake thusly:
> Is "fix pipelines" your WIP patch subject ?
> "gitlab-ci: ignore ownership of the git tree" seems better?

No, I really intended to write "fix pipeline".

Indeed, the subject of a commit log should be a summary of the semantic
change, not of how it is done.

Thanks for the review! :-)

Regards,
Yann E. MORIN.

> Le 07/02/2023 à 22:33, Yann E. MORIN a écrit :
> > When gitlab prepares a job to run, it checks out the repository with a
> > non-root user, and spawns a container that runs as root, with some UID
> > mapping that makes the files be owned by root in the container. However,
> > our pipelines run as a nont-root user.
> > 
> > Commit bde165f7ad (.gitlab-ci.yml: update Docker image to use) updated
> > the docker image that is used to run in our pipelines.
> > 
> > That new image includes a git version that is stricter about the
> > ownership of the git tree it is acting in: git aborts in error when the
> > user running it does not own the repository.
> > 
> > We use `git ls-tree` quite a lot in our check-{flake8,package,symbols}
> > rules, so they all fail (in various ways).
> > 
> > To fix this, we either need to fix the ownership or tell git to ignore
> > the situation.
> > 
> > It is most probably impossible to change the ownership of the files: we
> > run as non-root,and the files belong to root (in the container). So
> > we're stuck.
> > 
> > The alternative, is to do as git suggest, and tell it to ignore the
> > situation. In a local setup, this woujld be very insecure, but in the
> > pipelines, this is in a throw-away container, wehre a single user exists
> > and is running, so we don't care much (if at all).
> > 
> > Add a global before_script that registers the git config to ignore
> > ownership issues in the buildroot repository; see [0] for the definition
> > of the CI_PROJECT_DIR variable. Note: unlike what is said in there, and
> > in [1], the value actually seen in CI_PROJECT_DIR is already prefixed
> > with CI_BUILDS_DIR (the documentation is unclear about that point).
> > 
> > [0] https://docs.gitlab.com/ee/ci/variables/predefined_variables.html
> > [1] https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-runners-section
> 
> Reviewed-by: Romain Naour <romain.naour@smile.fr>
> 
> 
> > 
> > Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
> > Cc: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
> > Cc: Romain Naour <romain.naour@gmail.com>
> > Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
> > ---
> >  support/misc/gitlab-ci.yml.in | 3 +++
> >  1 file changed, 3 insertions(+)
> > 
> > diff --git a/support/misc/gitlab-ci.yml.in b/support/misc/gitlab-ci.yml.in
> > index 9c1faf0d5f..38aca31fb5 100644
> > --- a/support/misc/gitlab-ci.yml.in
> > +++ b/support/misc/gitlab-ci.yml.in
> > @@ -1,3 +1,6 @@
> > +before_script:
> > +  - git config --global --add safe.directory ${CI_PROJECT_DIR}
> 
> Note: I checked if we have the issue using the utils/docker-run directly,
> enabling safe.directory is not necessary.
> 
> Best regards,
> Romain
> 
> 
> > +
> >  .check-check-package_base:
> >      script:
> >          - python3 -m pytest -v utils/checkpackagelib/
> 
> _______________________________________________
> buildroot mailing list
> buildroot@buildroot.org
> https://lists.buildroot.org/mailman/listinfo/buildroot