diff mbox series

[RFC] ci: Make GitLab CI build quicker and versatile

Message ID 20210502181406.168056-1-toertel@gmail.com
State New
Headers show
Series [RFC] ci: Make GitLab CI build quicker and versatile | expand

Commit Message

Mark Jonas May 2, 2021, 6:14 p.m. UTC
This patch replaces the current GitLab CI with one offering the
following additional features:

- Use pre-built container image to reduce setup time
- Perform linting using REUSE, ShellCheck, and shfmt
- Use matrix build to speed up tests

The 'check' stage shall be considered as a demonstrator to convey the
idea of adding linting to the CI pipeline.

The matrix build of the 'build' stage only performs well if the pool of
available build machines contains two or more runners. When using the
shared runners of GitLab.com this is the case.

When using GitLab.com the time needed for a CI run is reduced from about
14 minutes to less than 6 minutes.

Signed-off-by: Mark Jonas <toertel@gmail.com>
---
 .gitlab-ci.yml | 95 +++++++++++++++++++++++++++++++++++++++++++++-----
 1 file changed, 87 insertions(+), 8 deletions(-)

Comments

Michael Adler May 3, 2021, 7:59 a.m. UTC | #1
Hi Mark,

thanks for your RFC and including me.

> - Perform linting using REUSE, ShellCheck, and shfmt

I'm a huge proponent of linting, so definitely ACK from my side.

> - Use matrix build to speed up tests

In principle fine, but:

> - Use pre-built container image to reduce setup time

I'm against this proposition though. Supply chain attacks are a real threat nowadays (e.g. [1]), so I'm not a fan of
blindly executing the container image from an untrusted source such as `toertel/swupdate-contribute` (both locally and
on privately hosted Gitlab instances). (Also, in restricted environments, it might not be possible at all to pull these
images for the aforementioned security reasons.)
So my suggestion is to use the official Alpine Linux image (which receives regular security updates) and install the
necessary packages via an appropriate channel (preferably, through the package manager). In my opinion, that's usually
a good compromise between security and performance. While this should work for the linting stage, it will probably not
work that well for the build stage. In that case, it could be an option to host and maintain an *official* SWUpdate
container image which can be verified and trusted by the users. This is an additional maintenance burden though.

Some additional notes:

1) The primary reason I put the testing logic into shell scripts was to make it easily re-usable among different CI
	 providers (you have inlined the code again, making it unusable in Travis CI, for instance). Further, having the code
	 inside shell scripts makes it fairly trivial to test on different distributions (e.g. Debian Bullseye works out the
	 box) and operating systems (e.g. FreeBSD).

2) I think the Gitlab CI config should be a subset of (or equal to) the Travis CI configuration from a testing point of
	 view, since I think that Travis CI is currently the primary CI platform?

Kind Regards,
  Michael

[1] https://web.archive.org/web/20210424202549/https://www.reuters.com/technology/codecov-hackers-breached-hundreds-restricted-customer-sites-sources-2021-04-19/
Mark Jonas May 3, 2021, 6:28 p.m. UTC | #2
HI Michael,

> thanks for your RFC and including me.

You are welcome. :)

> > - Perform linting using REUSE, ShellCheck, and shfmt
>
> I'm a huge proponent of linting, so definitely ACK from my side.
>
> > - Use matrix build to speed up tests
>
> In principle fine, but:
>
> > - Use pre-built container image to reduce setup time
>
> I'm against this proposition though. Supply chain attacks are a real threat nowadays (e.g. [1]), so I'm not a fan of
> blindly executing the container image from an untrusted source such as `toertel/swupdate-contribute` (both locally and
> on privately hosted Gitlab instances). (Also, in restricted environments, it might not be possible at all to pull these
> images for the aforementioned security reasons.)
> So my suggestion is to use the official Alpine Linux image (which receives regular security updates) and install the
> necessary packages via an appropriate channel (preferably, through the package manager). In my opinion, that's usually
> a good compromise between security and performance. While this should work for the linting stage, it will probably not
> work that well for the build stage. In that case, it could be an option to host and maintain an *official* SWUpdate
> container image which can be verified and trusted by the users. This is an additional maintenance burden though.

I understand your arguments regarding the vulnerability of hardly
maintained containers. So yes, we should plan to improve the
robustness by replacing the pre-built image with on-demand configured
Alpine containers. Alternatively we could also use containers from the
original tool authors which do not contain other user-space
applications and libraries at all. For example, mvdan/shftm and
koalaman/shellcheck also offer images which only contain a statically
linked binary.

If you look up the maintainer of toertel/swupdate-contribute you will
understand why I would not call it untrusted. :)

https://gitlab.com/toertel/docker-image-swupdate-contribute

For toertel/swupdate-contribute I would not have a problem with a
friendly takeover by the SWUpdate project. For example, U-Boot also
builds its own trini/u-boot-gitlab-ci-runner image.

> Some additional notes:
>
> 1) The primary reason I put the testing logic into shell scripts was to make it easily re-usable among different CI
>          providers (you have inlined the code again, making it unusable in Travis CI, for instance). Further, having the code
>          inside shell scripts makes it fairly trivial to test on different distributions (e.g. Debian Bullseye works out the
>          box) and operating systems (e.g. FreeBSD).

That's fine. I also do that for my projects even if there is only a
GitLab pipeline. This way it is a lot easier to write, debug, and
maintain the pipeline. IMHO, so far my RFC does not contain anything
worth a script.

What I still need to learn (or work around) is how to derive the
matrix build from a scan of the configs directory. Alternatively I was
thinking about creating a test script (!) which would implement a kind
of a divide and conquer strategy. You would call it with two
parameters; one is the total number of partitions and the other would
be the partition which should be worked on.

+    script:
+        - test_split.sh
+        - make -j $(nproc)
+        - make tests
+    parallel:
+        matrix:
+            - TOTAL: 4
+              FRAG:
+                - 1
+                - 2
+                - 3
+                - 4

>
> 2) I think the Gitlab CI config should be a subset of (or equal to) the Travis CI configuration from a testing point of
>          view, since I think that Travis CI is currently the primary CI platform?

Correct. Personal tooling of Stefano like Coverity should only be on
Travis CI. I have not really used Travis CI before but I am rather
sure it can do the same things as GitLab CI. I would have to learn how
to properly use Travis first if it has to happen at the same time.

To conclude, it more or less boils down to the question whether the
SWUpdate project would be fine using toertel/swupdate-contribute or
alternatively create its own CI (and development support) image.

Cheers,
Mark

On Mon, May 3, 2021 at 10:03 AM Michael Adler <michael.adler@siemens.com> wrote:
>
> Hi Mark,
>
> thanks for your RFC and including me.
>
> > - Perform linting using REUSE, ShellCheck, and shfmt
>
> I'm a huge proponent of linting, so definitely ACK from my side.
>
> > - Use matrix build to speed up tests
>
> In principle fine, but:
>
> > - Use pre-built container image to reduce setup time
>
> I'm against this proposition though. Supply chain attacks are a real threat nowadays (e.g. [1]), so I'm not a fan of
> blindly executing the container image from an untrusted source such as `toertel/swupdate-contribute` (both locally and
> on privately hosted Gitlab instances). (Also, in restricted environments, it might not be possible at all to pull these
> images for the aforementioned security reasons.)
> So my suggestion is to use the official Alpine Linux image (which receives regular security updates) and install the
> necessary packages via an appropriate channel (preferably, through the package manager). In my opinion, that's usually
> a good compromise between security and performance. While this should work for the linting stage, it will probably not
> work that well for the build stage. In that case, it could be an option to host and maintain an *official* SWUpdate
> container image which can be verified and trusted by the users. This is an additional maintenance burden though.
>
> Some additional notes:
>
> 1) The primary reason I put the testing logic into shell scripts was to make it easily re-usable among different CI
>          providers (you have inlined the code again, making it unusable in Travis CI, for instance). Further, having the code
>          inside shell scripts makes it fairly trivial to test on different distributions (e.g. Debian Bullseye works out the
>          box) and operating systems (e.g. FreeBSD).
>
> 2) I think the Gitlab CI config should be a subset of (or equal to) the Travis CI configuration from a testing point of
>          view, since I think that Travis CI is currently the primary CI platform?
>
> Kind Regards,
>   Michael
>
> [1] https://web.archive.org/web/20210424202549/https://www.reuters.com/technology/codecov-hackers-breached-hundreds-restricted-customer-sites-sources-2021-04-19/
>
> --
> Michael Adler
>
> Siemens AG
> T RDA IOT SES-DE
> Otto-Hahn-Ring 6
> 81739 München, Deutschland
>
> Siemens Aktiengesellschaft: Vorsitzender des Aufsichtsrats: Jim Hagemann Snabe;
> Vorstand: Joe Kaeser, Vorsitzender; Roland Busch, Klaus Helmrich, Cedrik Neike,
> Matthias Rebellius, Ralf P. Thomas, Judith Wiese; Sitz der Gesellschaft: Berlin
> und München, Deutschland; Registergericht: Berlin-Charlottenburg, HRB 12300,
> München, HRB 6684; WEEE-Reg.-Nr. DE 23691322
Michael Adler May 4, 2021, 7:26 a.m. UTC | #3
Hi Mark,

> Alternatively we could also use containers from the original tool authors which do not contain other user-space
> applications and libraries at all.

to be honest, if I had to choose between the Alpine Linux package of such a tool and the container built by its author,
I would go for Alpine without a doubt. It's not that I do not trust the author, but I have more confidence in the
security (and QA!) of a major Linux distribution such as Alpine.

> If you look up the maintainer of toertel/swupdate-contribute you will understand why I would not call it untrusted. :)

Yes but my point from above still stands :) It's not just you that needs to be trusted, it's the *whole* supply chain
(which is long!).

> For toertel/swupdate-contribute I would not have a problem with a friendly takeover by the SWUpdate project. For
> example, U-Boot also builds its own trini/u-boot-gitlab-ci-runner image.

It's a trade off between maintenance cost and performance. If we maintain our own image, we need to update it regularly
to detect regressions which could be caused by library updates (unlikely due to LTS distro though).

> IMHO, so far my RFC does not contain anything worth a script.

I typically use a Git hook to run the tests locally before I push something. If the convention is that all linting logic
is inside a `lint.sh` script, it's set-and-forget. If the logic resides in the yaml file, I need to diff for changes
on every pull (in theory).

> I have not really used Travis CI before but I am rather sure it can do the same things as GitLab CI.

IMO, Travis CI is not what it's used to be anymore. Before additional effort is put into Travis CI, I would like to
throw Cirrus CI in the ring, since it would allow us to test on FreeBSD as well [1].

> What I still need to learn (or work around) is how to derive the matrix build from a scan of the configs directory.

Maybe [2] is an option. I haven't looked into it yet, but it's on my TODO list.

> To conclude, it more or less boils down to the question whether the SWUpdate project would be fine using
> toertel/swupdate-contribute or alternatively create its own CI (and development support) image.

... or continue using the upstream Ubuntu image and accepting the 'penalty' of a 7(?) minutes increased feedback cycle? :)

Kind Regards,
  Michael

[1] https://cirrus-ci.org/features/#comparison-with-popular-ciaas
[2] https://gitlab.com/dbsystel/gitlab-ci-python-library
Stefano Babic May 4, 2021, 9:16 a.m. UTC | #4
Hi Michael, Mark,

On 03.05.21 20:28, Mark Jonas wrote:
> HI Michael,
> 
>> thanks for your RFC and including me.
> 
> You are welcome. :)
> 
>>> - Perform linting using REUSE, ShellCheck, and shfmt
>>
>> I'm a huge proponent of linting, so definitely ACK from my side.
>>
>>> - Use matrix build to speed up tests
>>
>> In principle fine, but:
>>
>>> - Use pre-built container image to reduce setup time
>>
>> I'm against this proposition though. Supply chain attacks are a real threat nowadays (e.g. [1]), so I'm not a fan of
>> blindly executing the container image from an untrusted source such as `toertel/swupdate-contribute` (both locally and
>> on privately hosted Gitlab instances). (Also, in restricted environments, it might not be possible at all to pull these
>> images for the aforementioned security reasons.)
>> So my suggestion is to use the official Alpine Linux image (which receives regular security updates) and install the
>> necessary packages via an appropriate channel (preferably, through the package manager). In my opinion, that's usually
>> a good compromise between security and performance. While this should work for the linting stage, it will probably not
>> work that well for the build stage. In that case, it could be an option to host and maintain an *official* SWUpdate
>> container image which can be verified and trusted by the users. This is an additional maintenance burden though.
> 
> I understand your arguments regarding the vulnerability of hardly
> maintained containers. So yes, we should plan to improve the
> robustness by replacing the pre-built image with on-demand configured
> Alpine containers. Alternatively we could also use containers from the
> original tool authors which do not contain other user-space
> applications and libraries at all. For example, mvdan/shftm and
> koalaman/shellcheck also offer images which only contain a statically
> linked binary.
> 
> If you look up the maintainer of toertel/swupdate-contribute you will
> understand why I would not call it untrusted. :)
> 
> https://gitlab.com/toertel/docker-image-swupdate-contribute
> 
> For toertel/swupdate-contribute I would not have a problem with a
> friendly takeover by the SWUpdate project. For example, U-Boot also
> builds its own trini/u-boot-gitlab-ci-runner image.

Tom has pushed the image, and the Dockerfile is part of U-Boot's project 
(in tools/Docker). We can do in the same way, and the Dockerfile can be 
merged into SWUpdate and then maintained. I suggest the same place as in 
U-Boot, that is tools/docker and documentation in doc/.

> 
>> Some additional notes:
>>
>> 1) The primary reason I put the testing logic into shell scripts was to make it easily re-usable among different CI
>>           providers (you have inlined the code again, making it unusable in Travis CI, for instance). Further, having the code
>>           inside shell scripts makes it fairly trivial to test on different distributions (e.g. Debian Bullseye works out the
>>           box) and operating systems (e.g. FreeBSD).
> 
> That's fine. I also do that for my projects even if there is only a
> GitLab pipeline. This way it is a lot easier to write, debug, and
> maintain the pipeline. IMHO, so far my RFC does not contain anything
> worth a script.
> 
> What I still need to learn (or work around) is how to derive the
> matrix build from a scan of the configs directory. Alternatively I was
> thinking about creating a test script (!) which would implement a kind
> of a divide and conquer strategy. You would call it with two
> parameters; one is the total number of partitions and the other would
> be the partition which should be worked on.
> 
> +    script:
> +        - test_split.sh
> +        - make -j $(nproc)
> +        - make tests
> +    parallel:
> +        matrix:
> +            - TOTAL: 4
> +              FRAG:
> +                - 1
> +                - 2
> +                - 3
> +                - 4
> 
>>
>> 2) I think the Gitlab CI config should be a subset of (or equal to) the Travis CI configuration from a testing point of
>>           view, since I think that Travis CI is currently the primary CI platform?
> 
> Correct. Personal tooling of Stefano like Coverity should only be on
> Travis CI.

This was a way to try to reduce maintanance effort on my side. Request 
to run coverity came from community, and running in Travis avoided to me 
to run unknown code (the coverity proprietary tools) on my host. Anyway, 
this can move into the container, too. Coverity can run locally (that is 
in container) and push the result to coverity website.

Travis was used by many FOSS projects before, due to changes in usage is 
less interessant today. Anyway, as the number of daily builds is not 
comparable to other big projects (like U-Boot), I have not moved away 
from travis as CI, but this can be also be done. As I suggested before, 
I can move it on the same server used by U-Boot for CI.

> I have not really used Travis CI before but I am rather
> sure it can do the same things as GitLab CI.

It is. U-Boot moved completely from Travis and drop support for it. 
There are only gitlab CI runners now.

> I would have to learn how
> to properly use Travis first if it has to happen at the same time.
> 
> To conclude, it more or less boils down to the question whether the
> SWUpdate project would be fine using toertel/swupdate-contribute or
> alternatively create its own CI (and development support) image.

I will suggest to follow U-Boot's example. We can have a maintained 
Dockerfile in repo, and image is just based on this file and pushed to 
hub. And we can add Gitlab's CI runners, too, like U-Boot is doing.

Best regards,
Stefano

> 
> Cheers,
> Mark
> 
> On Mon, May 3, 2021 at 10:03 AM Michael Adler <michael.adler@siemens.com> wrote:
>>
>> Hi Mark,
>>
>> thanks for your RFC and including me.
>>
>>> - Perform linting using REUSE, ShellCheck, and shfmt
>>
>> I'm a huge proponent of linting, so definitely ACK from my side.
>>
>>> - Use matrix build to speed up tests
>>
>> In principle fine, but:
>>
>>> - Use pre-built container image to reduce setup time
>>
>> I'm against this proposition though. Supply chain attacks are a real threat nowadays (e.g. [1]), so I'm not a fan of
>> blindly executing the container image from an untrusted source such as `toertel/swupdate-contribute` (both locally and
>> on privately hosted Gitlab instances). (Also, in restricted environments, it might not be possible at all to pull these
>> images for the aforementioned security reasons.)
>> So my suggestion is to use the official Alpine Linux image (which receives regular security updates) and install the
>> necessary packages via an appropriate channel (preferably, through the package manager). In my opinion, that's usually
>> a good compromise between security and performance. While this should work for the linting stage, it will probably not
>> work that well for the build stage. In that case, it could be an option to host and maintain an *official* SWUpdate
>> container image which can be verified and trusted by the users. This is an additional maintenance burden though.
>>
>> Some additional notes:
>>
>> 1) The primary reason I put the testing logic into shell scripts was to make it easily re-usable among different CI
>>           providers (you have inlined the code again, making it unusable in Travis CI, for instance). Further, having the code
>>           inside shell scripts makes it fairly trivial to test on different distributions (e.g. Debian Bullseye works out the
>>           box) and operating systems (e.g. FreeBSD).
>>
>> 2) I think the Gitlab CI config should be a subset of (or equal to) the Travis CI configuration from a testing point of
>>           view, since I think that Travis CI is currently the primary CI platform?
>>
>> Kind Regards,
>>    Michael
>>
>> [1] https://web.archive.org/web/20210424202549/https://www.reuters.com/technology/codecov-hackers-breached-hundreds-restricted-customer-sites-sources-2021-04-19/
>>
>> --
>> Michael Adler
>>
>> Siemens AG
>> T RDA IOT SES-DE
>> Otto-Hahn-Ring 6
>> 81739 München, Deutschland
>>
>> Siemens Aktiengesellschaft: Vorsitzender des Aufsichtsrats: Jim Hagemann Snabe;
>> Vorstand: Joe Kaeser, Vorsitzender; Roland Busch, Klaus Helmrich, Cedrik Neike,
>> Matthias Rebellius, Ralf P. Thomas, Judith Wiese; Sitz der Gesellschaft: Berlin
>> und München, Deutschland; Registergericht: Berlin-Charlottenburg, HRB 12300,
>> München, HRB 6684; WEEE-Reg.-Nr. DE 23691322
>
Stefano Babic May 4, 2021, 9:34 a.m. UTC | #5
Hi Michael,

On 04.05.21 09:26, Michael Adler wrote:
> Hi Mark,
> 
>> Alternatively we could also use containers from the original tool authors which do not contain other user-space
>> applications and libraries at all.
> 
> to be honest, if I had to choose between the Alpine Linux package of such a tool and the container built by its author,
> I would go for Alpine without a doubt. It's not that I do not trust the author, but I have more confidence in the
> security (and QA!) of a major Linux distribution such as Alpine.
> 

Frankly speaking, the security issues in this case are not so hard as in 
many other cases. The container is just used to test a build - however, 
build is also tested locally by me and by the contributors. Nothing 
related to the container flow on the devices. It is not that we deliver 
SWUpdate in a container, we just test the build using a container.

>> If you look up the maintainer of toertel/swupdate-contribute you will understand why I would not call it untrusted. :)
> 
> Yes but my point from above still stands :) It's not just you that needs to be trusted, it's the *whole* supply chain
> (which is long!).
> 
>> For toertel/swupdate-contribute I would not have a problem with a friendly takeover by the SWUpdate project. For
>> example, U-Boot also builds its own trini/u-boot-gitlab-ci-runner image.
> 
> It's a trade off between maintenance cost and performance. If we maintain our own image, we need to update it regularly
> to detect regressions which could be caused by library updates (unlikely due to LTS distro though).
> 
>> IMHO, so far my RFC does not contain anything worth a script.
> 
> I typically use a Git hook to run the tests locally before I push something. If the convention is that all linting logic
> is inside a `lint.sh` script, it's set-and-forget. If the logic resides in the yaml file, I need to diff for changes
> on every pull (in theory).
> 
>> I have not really used Travis CI before but I am rather sure it can do the same things as GitLab CI.
> 
> IMO, Travis CI is not what it's used to be anymore. Before additional effort is put into Travis CI,

That's right - so I guess that SWUpdate will move from Travis in some time.

> I would like to
> throw Cirrus CI in the ring, since it would allow us to test on FreeBSD as well [1].
> 
>> What I still need to learn (or work around) is how to derive the matrix build from a scan of the configs directory.
> 
> Maybe [2] is an option. I haven't looked into it yet, but it's on my TODO list.
> 
>> To conclude, it more or less boils down to the question whether the SWUpdate project would be fine using
>> toertel/swupdate-contribute or alternatively create its own CI (and development support) image.

I will propose to have a community maintained file (like U-Boot is 
doing) to be used with docker-compose / podman to create the image, and 
the image is just create using it. We can add an "official" swupdate 
docker-id to increase user's confiance.

> 
> ... or continue using the upstream Ubuntu image and accepting the 'penalty' of a 7(?) minutes increased feedback cycle? :)

I test and build mainly on my host, so a penalty for build time is not 
an issue on my case, but it could be on your side. Let me know what ou 
think.

Best regards,
Stefano Babic
Mark Jonas May 4, 2021, 5:53 p.m. UTC | #6
Hi Michael,

> > Alternatively we could also use containers from the original tool authors which do not contain other user-space
> > applications and libraries at all.
>
> to be honest, if I had to choose between the Alpine Linux package of such a tool and the container built by its author,
> I would go for Alpine without a doubt. It's not that I do not trust the author, but I have more confidence in the
> security (and QA!) of a major Linux distribution such as Alpine.

OK, technically understood. Though I still have doubt whether the
actual risk concerning the images we are talking about is worth the
effort.

> > If you look up the maintainer of toertel/swupdate-contribute you will understand why I would not call it untrusted. :)
>
> Yes but my point from above still stands :) It's not just you that needs to be trusted, it's the *whole* supply chain
> (which is long!).

Why is the supply chain long? The build step uses the official
Docker-in-Docker image and the Dockerfile is based on ubuntu:focal
with latest security fixes. What else do you consider as part of the
supply chain?

> > For toertel/swupdate-contribute I would not have a problem with a friendly takeover by the SWUpdate project. For
> > example, U-Boot also builds its own trini/u-boot-gitlab-ci-runner image.
>
> It's a trade off between maintenance cost and performance. If we maintain our own image, we need to update it regularly
> to detect regressions which could be caused by library updates (unlikely due to LTS distro though).

toertel/swupdate-contribute:latest is updated once a week. So we can
check off "regularly". It is an automatic build, test, and deploy
process so we can also check off "maintenance cost". So far I only had
build / test failures because either something changed in the SWUpdate
build process which triggered the test stage, or the GitLab
infrastructure fumbled. For the latter so far I just had to click on
the retry button.

> > IMHO, so far my RFC does not contain anything worth a script.
>
> I typically use a Git hook to run the tests locally before I push something. If the convention is that all linting logic
> is inside a `lint.sh` script, it's set-and-forget. If the logic resides in the yaml file, I need to diff for changes
> on every pull (in theory).

OK. Yet for me in CI parallelization and  a simple pipeline script is
more desirable.

> > I have not really used Travis CI before but I am rather sure it can do the same things as GitLab CI.
>
> IMO, Travis CI is not what it's used to be anymore. Before additional effort is put into Travis CI, I would like to
> throw Cirrus CI in the ring, since it would allow us to test on FreeBSD as well [1].

GitHub Actions could also be a viable alternative to Travis CI.

> > What I still need to learn (or work around) is how to derive the matrix build from a scan of the configs directory.
>
> Maybe [2] is an option. I haven't looked into it yet, but it's on my TODO list.

Yes, technically it should be possible to create a GitLab pipeline
using GitLab pipeline.

> > To conclude, it more or less boils down to the question whether the SWUpdate project would be fine using
> > toertel/swupdate-contribute or alternatively create its own CI (and development support) image.
>
> ... or continue using the upstream Ubuntu image and accepting the 'penalty' of a 7(?) minutes increased feedback cycle? :)

Maybe your pain with a slow pipeline is less than mine because you are
using Git hooks. I prefer a CI pipeline for the same purpose.

I also prefer to not install all dependencies on my development
machine and instead encapsulate the environment in a container. This
is an additional lever for my investment in maintaining
toertel/swupdate-contribute. My hope was it would also help others.

Cheers,
Mark
Michael Adler May 5, 2021, 8:46 a.m. UTC | #7
Hi Mark, hi Stefano,

> Why is the supply chain long? The build step uses the official Docker-in-Docker image and the Dockerfile is based on
> ubuntu:focal with latest security fixes.

ok, I hadn't looked into the specifics and assumed you were building the image on your machine, which would have made
your hardware and operating system security part of the supply chain.

> It is an automatic build, test, and deploy process so we can also check off "maintenance cost".

Nice job!

> OK. Yet for me in CI parallelization and  a simple pipeline script is more desirable.

I don't see a contradiction here, e.g. call your scripts 01-shfmt.sh, 02-shellcheck.sh and I could just run all (or some
of) the scripts in a for-loop. No vendor lock-in this way and no restriction regarding parallelization. Win-win :)

> GitHub Actions could also be a viable alternative to Travis CI.

True, although I think - unlike Cirrus CI - it does not support FreeBSD (which SWUpdate supports but is mostly
untested). However, since the project is hosted on Github, I would argue it's a better choice than Gitlab CI (even
though I prefer Gitlab CI personally ;).

> I also prefer to not install all dependencies on my development machine and instead encapsulate the environment in
> a container.

Me too, which is why I use nix-shell [1] for this, at least for basic testing, but we're getting off-topic ;)

> I will propose to have a community maintained file (like U-Boot is doing) to
> be used with docker-compose / podman to create the image, and the image is
> just create using it. We can add an "official" swupdate docker-id to
> increase user's confiance.

Sounds good to me, especially the SWUpdate branding to make it official.

Kind Regards,
  Michael

[1] https://nixos.org/manual/nix/unstable/command-ref/nix-shell.html
Stefano Babic May 5, 2021, 9:33 a.m. UTC | #8
Hi Mark, Michael,

On 05.05.21 10:46, Michael Adler wrote:
> Hi Mark, hi Stefano,
> 
>> Why is the supply chain long? The build step uses the official Docker-in-Docker image and the Dockerfile is based on
>> ubuntu:focal with latest security fixes.
> 
> ok, I hadn't looked into the specifics and assumed you were building the image on your machine, which would have made
> your hardware and operating system security part of the supply chain.
> 
>> It is an automatic build, test, and deploy process so we can also check off "maintenance cost".
> 
> Nice job!
> 

+1

>> OK. Yet for me in CI parallelization and  a simple pipeline script is more desirable.
> 
> I don't see a contradiction here, e.g. call your scripts 01-shfmt.sh, 02-shellcheck.sh and I could just run all (or some
> of) the scripts in a for-loop. No vendor lock-in this way and no restriction regarding parallelization. Win-win :)
> 

+1

>> GitHub Actions could also be a viable alternative to Travis CI.
> 
> True, although I think - unlike Cirrus CI - it does not support FreeBSD (which SWUpdate supports but is mostly
> untested). However, since the project is hosted on Github, I would argue it's a better choice than Gitlab CI (even
> though I prefer Gitlab CI personally ;).

This is why I can offer something. We have at DENX a public Gitlab 
server for FOSS projects:

	https://source.denx.de

We can set any runner we want. Currently, it hosts U-Boot (for all 
maintainers) and Xenomai, and it mirrors SWUpdate.

> 
>> I also prefer to not install all dependencies on my development machine and instead encapsulate the environment in
>> a container.

+1

> 
> Me too, which is why I use nix-shell [1] for this, at least for basic testing, but we're getting off-topic ;)
> 
>> I will propose to have a community maintained file (like U-Boot is doing) to
>> be used with docker-compose / podman to create the image, and the image is
>> just create using it. We can add an "official" swupdate docker-id to
>> increase user's confiance.
> 
> Sounds good to me, especially the SWUpdate branding to make it official.

Right.


Best regards,
Stefano
Michael Adler May 5, 2021, 2:37 p.m. UTC | #9
Hi Stefano,

> We can set any runner we want.

you mean, for example, DENX could provide us a FreeBSD runner?

Kind Regards,
  Michael
Stefano Babic May 5, 2021, 2:43 p.m. UTC | #10
Hi Michael,

On 05.05.21 16:37, Michael Adler wrote:
> Hi Stefano,
> 
>> We can set any runner we want.
> 
> you mean, for example, DENX could provide us a FreeBSD runner?

Right. Gitlab supports a Virtualbox machine (maybe can be done easier 
with KVM), and this can be used for the CI. We could set up a runner then.

Regards,
Stefano
Michael Adler May 5, 2021, 2:49 p.m. UTC | #11
> Right. Gitlab supports a Virtualbox machine (maybe can be done easier with
> KVM), and this can be used for the CI. We could set up a runner then.

Sounds great! I can tell from experience that adding a custom Gitlab runner is fairly easy. The "hard" part is getting
the machine (or VM) up and running. So to me this sounds like the way to proceed then? (Adding custom runners etc. can
be done later, it's just good to know that this perspective is available).
Stefano Babic May 5, 2021, 3 p.m. UTC | #12
Hi Michael,

On 05.05.21 16:49, Michael Adler wrote:
>> Right. Gitlab supports a Virtualbox machine (maybe can be done easier with
>> KVM), and this can be used for the CI. We could set up a runner then.
> 
> Sounds great! I can tell from experience that adding a custom Gitlab runner is fairly easy. The "hard" part is getting
> the machine (or VM) up and running. So to me this sounds like the way to proceed then?

Yes. I discussed yesterday inside DENX, this server has currently just 
runners for U-Boot (maybe also for Xenomai), and we will add runners for 
SWUpdate, too. And we can add a FreePBX VM.

> (Adding custom runners etc. can
> be done later, it's just good to know that this perspective is available).

Regards,
Stefano
diff mbox series

Patch

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index b0cdd10..39bc95e 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -1,10 +1,89 @@ 
+# SPDX-FileCopyrightText: 2021 Mark Jonas <toertel@gmail.com>
+#
+# SPDX-License-Identifier: MIT
 
-ubuntu-focal:
-  image: ubuntu:focal
-  stage: build
-  before_script:
-    - ./ci/setup.sh
-    - ./ci/install-src-deps.sh
-  script:
-    - ./ci/test-configs.sh
+stages:
+    - check
+    - build
 
+
+########## job templates ##########
+
+.check_job:
+    stage: check
+    tags:
+        - docker
+
+.build_job:
+    stage: build
+    tags:
+        - docker
+
+
+########## stage check ##########
+
+reuse:
+    extends: .check_job
+    image:
+        name: "fsfe/reuse:0.12.1"
+        entrypoint: [""]
+    script:
+        - reuse lint
+    allow_failure: true
+
+shellcheck:
+    extends: .check_job
+    image: "koalaman/shellcheck-alpine:v0.7.2"
+    script:
+        - shellcheck --version
+        - find . -name '*.sh' -print0 | xargs -0 -n1 -I @@ sh -c '{ echo Checking "@@" ...; shellcheck "@@"; }'
+    allow_failure: true
+
+shfmt:
+    extends: .check_job
+    image:
+        name: "mvdan/shfmt:v3.2.4-alpine"
+        entrypoint: [""]
+    script:
+        - /bin/shfmt -version
+        - /bin/shfmt -f .
+        - /bin/shfmt -d .
+    allow_failure: true
+
+
+########## stage build ##########
+
+build:
+    extends: .build_job
+    image:
+        name: "toertel/swupdate-contribute:latest"
+    script:
+        - make "${DEFCONFIG}"
+        - make -j $(nproc)
+        - make tests
+    parallel:
+        matrix:
+            - DEFCONFIG:
+                - all_handlers_defconfig
+                - bootloader_ebg_defconfig
+                - bootloader_grub_defconfig
+                - bootloader_uboot_defconfig
+                - cms1_defconfig
+                - cms_defconfig
+                - hawkbit_defconfig
+                - mbedtls_defconfig
+                - no_ssl_defconfig
+                - nodwl_defconfig
+                - raspi_defconfig
+                - sha256_defconfig
+                - suricatta_http_defconfig
+                - swuforwarder_defconfig
+                - test_defconfig
+                - with_ebg_defconfig
+                - with_lua_handlers_defconfig
+                - with_lua_nohandlers_defconfig
+                - with_rdiff_defconfig
+                - with_systemd_defconfig
+                - with_ucfw_defconfig
+                - without_libconfig_defconfig
+                - without_lua_defconfig