| Message ID | 20230310133247.39268-1-philmd@linaro.org |
|---|---|
| State | New |
| Headers | show |
| Series | [PATCH-for-8.0] gitlab-ci: Remove job building EDK2 firmware binaries | expand |
On Fri, 10 Mar 2023 at 13:33, Philippe Mathieu-Daudé <philmd@linaro.org> wrote: > > When we introduced this Gitlab-CI job in commit 71920809ce > ("gitlab-ci.yml: Add jobs to build EDK2 firmware binaries"), > the naive plan was to have reproducible binaries by downloading > what this job would build, testing it and eventually committing > it. With retrospective, nothing happened 3 years later and this > job is just bitrotting: > > Step 1/3 : FROM ubuntu:18.04 > 18.04: Pulling from library/ubuntu > mediaType in manifest should be > 'application/vnd.docker.distribution.manifest.v2+json' not > 'application/vnd.oci.image.manifest.v1+json' > > Remove this job to avoid wasting maintenance and CI ressources. Does the same thing hold for the opensbi job ? thanks -- PMM
On 10/3/23 14:38, Peter Maydell wrote: > On Fri, 10 Mar 2023 at 13:33, Philippe Mathieu-Daudé <philmd@linaro.org> wrote: >> >> When we introduced this Gitlab-CI job in commit 71920809ce >> ("gitlab-ci.yml: Add jobs to build EDK2 firmware binaries"), >> the naive plan was to have reproducible binaries by downloading >> what this job would build, testing it and eventually committing >> it. With retrospective, nothing happened 3 years later and this >> job is just bitrotting: >> >> Step 1/3 : FROM ubuntu:18.04 >> 18.04: Pulling from library/ubuntu >> mediaType in manifest should be >> 'application/vnd.docker.distribution.manifest.v2+json' not >> 'application/vnd.oci.image.manifest.v1+json' >> >> Remove this job to avoid wasting maintenance and CI ressources. > > Does the same thing hold for the opensbi job ? Cc'ing Bin, I have no idea.
On Fri, Mar 10, 2023 at 9:50 PM Philippe Mathieu-Daudé <philmd@linaro.org> wrote: > > On 10/3/23 14:38, Peter Maydell wrote: > > On Fri, 10 Mar 2023 at 13:33, Philippe Mathieu-Daudé <philmd@linaro.org> wrote: > >> > >> When we introduced this Gitlab-CI job in commit 71920809ce > >> ("gitlab-ci.yml: Add jobs to build EDK2 firmware binaries"), > >> the naive plan was to have reproducible binaries by downloading > >> what this job would build, testing it and eventually committing > >> it. With retrospective, nothing happened 3 years later and this > >> job is just bitrotting: > >> > >> Step 1/3 : FROM ubuntu:18.04 > >> 18.04: Pulling from library/ubuntu > >> mediaType in manifest should be > >> 'application/vnd.docker.distribution.manifest.v2+json' not > >> 'application/vnd.oci.image.manifest.v1+json' > >> > >> Remove this job to avoid wasting maintenance and CI ressources. > > > > Does the same thing hold for the opensbi job ? > > Cc'ing Bin, I have no idea. > The OpenSBI job now builds fine. I have no preference on keeping vs. removing it. I remember our previous goal was to create CI jobs for every pc-bios image but apparently that never happened. Regards, Bin
On 13/3/23 03:09, Bin Meng wrote: > On Fri, Mar 10, 2023 at 9:50 PM Philippe Mathieu-Daudé > <philmd@linaro.org> wrote: >> >> On 10/3/23 14:38, Peter Maydell wrote: >>> On Fri, 10 Mar 2023 at 13:33, Philippe Mathieu-Daudé <philmd@linaro.org> wrote: >>>> >>>> When we introduced this Gitlab-CI job in commit 71920809ce >>>> ("gitlab-ci.yml: Add jobs to build EDK2 firmware binaries"), >>>> the naive plan was to have reproducible binaries by downloading >>>> what this job would build, testing it and eventually committing >>>> it. With retrospective, nothing happened 3 years later and this >>>> job is just bitrotting: >>>> >>>> Step 1/3 : FROM ubuntu:18.04 >>>> 18.04: Pulling from library/ubuntu >>>> mediaType in manifest should be >>>> 'application/vnd.docker.distribution.manifest.v2+json' not >>>> 'application/vnd.oci.image.manifest.v1+json' >>>> >>>> Remove this job to avoid wasting maintenance and CI ressources. >>> >>> Does the same thing hold for the opensbi job ? >> >> Cc'ing Bin, I have no idea. >> > > The OpenSBI job now builds fine. I have no preference on keeping vs. > removing it. > > I remember our previous goal was to create CI jobs for every pc-bios > image but apparently that never happened. Yes, and I don't see interest in the community (neither worry that pc-bios/ images committed are built on each maintainer workstations). If it isn't consumed by QEMU, then better remove that job and save precious CI minutes. I presume OpenSBI itself is already tested by its mainstream project.
Hi Philippe, On Mon, Mar 13, 2023 at 4:51 PM Philippe Mathieu-Daudé <philmd@linaro.org> wrote: > > On 13/3/23 03:09, Bin Meng wrote: > > On Fri, Mar 10, 2023 at 9:50 PM Philippe Mathieu-Daudé > > <philmd@linaro.org> wrote: > >> > >> On 10/3/23 14:38, Peter Maydell wrote: > >>> On Fri, 10 Mar 2023 at 13:33, Philippe Mathieu-Daudé <philmd@linaro.org> wrote: > >>>> > >>>> When we introduced this Gitlab-CI job in commit 71920809ce > >>>> ("gitlab-ci.yml: Add jobs to build EDK2 firmware binaries"), > >>>> the naive plan was to have reproducible binaries by downloading > >>>> what this job would build, testing it and eventually committing > >>>> it. With retrospective, nothing happened 3 years later and this > >>>> job is just bitrotting: > >>>> > >>>> Step 1/3 : FROM ubuntu:18.04 > >>>> 18.04: Pulling from library/ubuntu > >>>> mediaType in manifest should be > >>>> 'application/vnd.docker.distribution.manifest.v2+json' not > >>>> 'application/vnd.oci.image.manifest.v1+json' > >>>> > >>>> Remove this job to avoid wasting maintenance and CI ressources. > >>> > >>> Does the same thing hold for the opensbi job ? > >> > >> Cc'ing Bin, I have no idea. > >> > > > > The OpenSBI job now builds fine. I have no preference on keeping vs. > > removing it. > > > > I remember our previous goal was to create CI jobs for every pc-bios > > image but apparently that never happened. > > Yes, and I don't see interest in the community (neither worry that > pc-bios/ images committed are built on each maintainer workstations). > > If it isn't consumed by QEMU, then better remove that job and save > precious CI minutes. I presume OpenSBI itself is already tested > by its mainstream project. Not sure what does "consumed" here mean? QEMU uses OpenSBI images on RISC-V machines by default. Regards, Bin
On 13/3/23 10:35, Bin Meng wrote: > Hi Philippe, > > On Mon, Mar 13, 2023 at 4:51 PM Philippe Mathieu-Daudé > <philmd@linaro.org> wrote: >> >> On 13/3/23 03:09, Bin Meng wrote: >>> On Fri, Mar 10, 2023 at 9:50 PM Philippe Mathieu-Daudé >>> <philmd@linaro.org> wrote: >>>> >>>> On 10/3/23 14:38, Peter Maydell wrote: >>>>> On Fri, 10 Mar 2023 at 13:33, Philippe Mathieu-Daudé <philmd@linaro.org> wrote: >>>>>> >>>>>> When we introduced this Gitlab-CI job in commit 71920809ce >>>>>> ("gitlab-ci.yml: Add jobs to build EDK2 firmware binaries"), >>>>>> the naive plan was to have reproducible binaries by downloading >>>>>> what this job would build, testing it and eventually committing >>>>>> it. With retrospective, nothing happened 3 years later and this >>>>>> job is just bitrotting: >>>>>> >>>>>> Step 1/3 : FROM ubuntu:18.04 >>>>>> 18.04: Pulling from library/ubuntu >>>>>> mediaType in manifest should be >>>>>> 'application/vnd.docker.distribution.manifest.v2+json' not >>>>>> 'application/vnd.oci.image.manifest.v1+json' >>>>>> >>>>>> Remove this job to avoid wasting maintenance and CI ressources. >>>>> >>>>> Does the same thing hold for the opensbi job ? >>>> >>>> Cc'ing Bin, I have no idea. >>>> >>> >>> The OpenSBI job now builds fine. I have no preference on keeping vs. >>> removing it. >>> >>> I remember our previous goal was to create CI jobs for every pc-bios >>> image but apparently that never happened. >> >> Yes, and I don't see interest in the community (neither worry that >> pc-bios/ images committed are built on each maintainer workstations). >> >> If it isn't consumed by QEMU, then better remove that job and save >> precious CI minutes. I presume OpenSBI itself is already tested >> by its mainstream project. > > Not sure what does "consumed" here mean? > > QEMU uses OpenSBI images on RISC-V machines by default. QEMU repository allows building QEMU system binaries which 'consume' the following (committed) files: - pc-bios/opensbi-riscv32-generic-fw_dynamic.bin - pc-bios/opensbi-riscv64-generic-fw_dynamic.bin We don't need to run the build-opensbi job to run QEMU: we use the prebuilt images.
On Mon, Mar 13, 2023 at 6:01 PM Philippe Mathieu-Daudé <philmd@linaro.org> wrote: > > On 13/3/23 10:35, Bin Meng wrote: > > Hi Philippe, > > > > On Mon, Mar 13, 2023 at 4:51 PM Philippe Mathieu-Daudé > > <philmd@linaro.org> wrote: > >> > >> On 13/3/23 03:09, Bin Meng wrote: > >>> On Fri, Mar 10, 2023 at 9:50 PM Philippe Mathieu-Daudé > >>> <philmd@linaro.org> wrote: > >>>> > >>>> On 10/3/23 14:38, Peter Maydell wrote: > >>>>> On Fri, 10 Mar 2023 at 13:33, Philippe Mathieu-Daudé <philmd@linaro.org> wrote: > >>>>>> > >>>>>> When we introduced this Gitlab-CI job in commit 71920809ce > >>>>>> ("gitlab-ci.yml: Add jobs to build EDK2 firmware binaries"), > >>>>>> the naive plan was to have reproducible binaries by downloading > >>>>>> what this job would build, testing it and eventually committing > >>>>>> it. With retrospective, nothing happened 3 years later and this > >>>>>> job is just bitrotting: > >>>>>> > >>>>>> Step 1/3 : FROM ubuntu:18.04 > >>>>>> 18.04: Pulling from library/ubuntu > >>>>>> mediaType in manifest should be > >>>>>> 'application/vnd.docker.distribution.manifest.v2+json' not > >>>>>> 'application/vnd.oci.image.manifest.v1+json' > >>>>>> > >>>>>> Remove this job to avoid wasting maintenance and CI ressources. > >>>>> > >>>>> Does the same thing hold for the opensbi job ? > >>>> > >>>> Cc'ing Bin, I have no idea. > >>>> > >>> > >>> The OpenSBI job now builds fine. I have no preference on keeping vs. > >>> removing it. > >>> > >>> I remember our previous goal was to create CI jobs for every pc-bios > >>> image but apparently that never happened. > >> > >> Yes, and I don't see interest in the community (neither worry that > >> pc-bios/ images committed are built on each maintainer workstations). > >> > >> If it isn't consumed by QEMU, then better remove that job and save > >> precious CI minutes. I presume OpenSBI itself is already tested > >> by its mainstream project. > > > > Not sure what does "consumed" here mean? > > > > QEMU uses OpenSBI images on RISC-V machines by default. > > QEMU repository allows building QEMU system binaries which 'consume' > the following (committed) files: > - pc-bios/opensbi-riscv32-generic-fw_dynamic.bin > - pc-bios/opensbi-riscv64-generic-fw_dynamic.bin > > We don't need to run the build-opensbi job to run QEMU: we use the > prebuilt images. Okay, thanks for the clarification. I believe that's the case for every pc-bios image? If yes, we don't need to build these pc-bios images in QEMU CI. Regards, Bin
diff --git a/.gitlab-ci.d/edk2.yml b/.gitlab-ci.d/edk2.yml deleted file mode 100644 index 314e101745..0000000000 --- a/.gitlab-ci.d/edk2.yml +++ /dev/null @@ -1,85 +0,0 @@ -# All jobs needing docker-edk2 must use the same rules it uses. -.edk2_job_rules: - rules: - # Forks don't get pipelines unless QEMU_CI=1 or QEMU_CI=2 is set - - if: '$QEMU_CI != "1" && $QEMU_CI != "2" && $CI_PROJECT_NAMESPACE != "qemu-project"' - when: never - - # In forks, if QEMU_CI=1 is set, then create manual job - # if any of the files affecting the build are touched - - if: '$QEMU_CI == "1" && $CI_PROJECT_NAMESPACE != "qemu-project"' - changes: - - .gitlab-ci.d/edk2.yml - - .gitlab-ci.d/edk2/Dockerfile - - roms/edk2/* - when: manual - - # In forks, if QEMU_CI=1 is set, then create manual job - # if the branch/tag starts with 'edk2' - - if: '$QEMU_CI == "1" && $CI_PROJECT_NAMESPACE != "qemu-project" && $CI_COMMIT_REF_NAME =~ /^edk2/' - when: manual - - # In forks, if QEMU_CI=1 is set, then create manual job - # if last commit msg contains 'EDK2' (case insensitive) - - if: '$QEMU_CI == "1" && $CI_PROJECT_NAMESPACE != "qemu-project" && $CI_COMMIT_MESSAGE =~ /edk2/i' - when: manual - - # Run if any files affecting the build output are touched - - changes: - - .gitlab-ci.d/edk2.yml - - .gitlab-ci.d/edk2/Dockerfile - - roms/edk2/* - when: on_success - - # Run if the branch/tag starts with 'edk2' - - if: '$CI_COMMIT_REF_NAME =~ /^edk2/' - when: on_success - - # Run if last commit msg contains 'EDK2' (case insensitive) - - if: '$CI_COMMIT_MESSAGE =~ /edk2/i' - when: on_success - -docker-edk2: - extends: .edk2_job_rules - stage: containers - image: docker:19.03.1 - services: - - docker:19.03.1-dind - variables: - GIT_DEPTH: 3 - IMAGE_TAG: $CI_REGISTRY_IMAGE:edk2-cross-build - # We don't use TLS - DOCKER_HOST: tcp://docker:2375 - DOCKER_TLS_CERTDIR: "" - before_script: - - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY - script: - - docker pull $IMAGE_TAG || true - - docker build --cache-from $IMAGE_TAG --tag $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA - --tag $IMAGE_TAG .gitlab-ci.d/edk2 - - docker push $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA - - docker push $IMAGE_TAG - -build-edk2: - extends: .edk2_job_rules - stage: build - needs: ['docker-edk2'] - artifacts: - paths: # 'artifacts.zip' will contains the following files: - - pc-bios/edk2*bz2 - - pc-bios/edk2-licenses.txt - - edk2-stdout.log - - edk2-stderr.log - image: $CI_REGISTRY_IMAGE:edk2-cross-build - variables: - GIT_DEPTH: 3 - script: # Clone the required submodules and build EDK2 - - git submodule update --init roms/edk2 - - git -C roms/edk2 submodule update --init -- - ArmPkg/Library/ArmSoftFloatLib/berkeley-softfloat-3 - BaseTools/Source/C/BrotliCompress/brotli - CryptoPkg/Library/OpensslLib/openssl - MdeModulePkg/Library/BrotliCustomDecompressLib/brotli - - export JOBS=$(($(getconf _NPROCESSORS_ONLN) + 1)) - - echo "=== Using ${JOBS} simultaneous jobs ===" - - make -j${JOBS} -C roms efi 2>&1 1>edk2-stdout.log | tee -a edk2-stderr.log >&2 diff --git a/.gitlab-ci.d/edk2/Dockerfile b/.gitlab-ci.d/edk2/Dockerfile deleted file mode 100644 index bbe50ff832..0000000000 --- a/.gitlab-ci.d/edk2/Dockerfile +++ /dev/null @@ -1,27 +0,0 @@ -# -# Docker image to cross-compile EDK2 firmware binaries -# -FROM ubuntu:18.04 - -MAINTAINER Philippe Mathieu-Daudé <f4bug@amsat.org> - -# Install packages required to build EDK2 -RUN apt update \ - && \ - \ - DEBIAN_FRONTEND=noninteractive \ - apt install --assume-yes --no-install-recommends \ - build-essential \ - ca-certificates \ - dos2unix \ - gcc-aarch64-linux-gnu \ - gcc-arm-linux-gnueabi \ - git \ - iasl \ - make \ - nasm \ - python3 \ - uuid-dev \ - && \ - \ - rm -rf /var/lib/apt/lists/* diff --git a/.gitlab-ci.d/qemu-project.yml b/.gitlab-ci.d/qemu-project.yml index 691d9bf5dc..a7ed447fe4 100644 --- a/.gitlab-ci.d/qemu-project.yml +++ b/.gitlab-ci.d/qemu-project.yml @@ -4,7 +4,6 @@ include: - local: '/.gitlab-ci.d/base.yml' - local: '/.gitlab-ci.d/stages.yml' - - local: '/.gitlab-ci.d/edk2.yml' - local: '/.gitlab-ci.d/opensbi.yml' - local: '/.gitlab-ci.d/containers.yml' - local: '/.gitlab-ci.d/crossbuilds.yml' diff --git a/MAINTAINERS b/MAINTAINERS index 3f24bc2787..b45b540fbd 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -3329,8 +3329,6 @@ F: roms/edk2 F: roms/edk2-* F: tests/data/uefi-boot-images/ F: tests/uefi-test-tools/ -F: .gitlab-ci.d/edk2.yml -F: .gitlab-ci.d/edk2/ VT-d Emulation M: Michael S. Tsirkin <mst@redhat.com>
When we introduced this Gitlab-CI job in commit 71920809ce ("gitlab-ci.yml: Add jobs to build EDK2 firmware binaries"), the naive plan was to have reproducible binaries by downloading what this job would build, testing it and eventually committing it. With retrospective, nothing happened 3 years later and this job is just bitrotting: Step 1/3 : FROM ubuntu:18.04 18.04: Pulling from library/ubuntu mediaType in manifest should be 'application/vnd.docker.distribution.manifest.v2+json' not 'application/vnd.oci.image.manifest.v1+json' Remove this job to avoid wasting maintenance and CI ressources. Reported-by: Palmer Dabbelt <palmer@rivosinc.com> Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org> --- Supersedes: <20230222155341.10127-1-palmer@rivosinc.com> --- .gitlab-ci.d/edk2.yml | 85 ----------------------------------- .gitlab-ci.d/edk2/Dockerfile | 27 ----------- .gitlab-ci.d/qemu-project.yml | 1 - MAINTAINERS | 2 - 4 files changed, 115 deletions(-) delete mode 100644 .gitlab-ci.d/edk2.yml delete mode 100644 .gitlab-ci.d/edk2/Dockerfile