| Message ID | 1582551584-20093-5-git-send-email-bmeng.cn@gmail.com |
|---|---|
| State | New |
| Headers | show |
| Series | riscv: Upgrade OpenSBI to v0.6 and add 32-bit sifive_u bios image | expand |
On Mon, Feb 24, 2020 at 5:40 AM Bin Meng <bmeng.cn@gmail.com> wrote: > > Add two GitLab jobs to build the OpenSBI firmware binaries. > > The first job builds a Docker image with the packages requisite > to build OpenSBI, and stores this image in the GitLab registry. > The second job pulls the image from the registry and builds the > OpenSBI firmware binaries. > > The docker image is only rebuilt if the GitLab YAML or the > Dockerfile is updated. The second job is only built when the > roms/opensbi/ submodule is updated, when a git-ref starts with > 'opensbi' or when the last commit contains 'OpenSBI'. The files > generated are archived in the artifacts.zip file. > > With OpenSBI v0.6, it took 2 minutes 56 seconds to build > the docker image, and 1 minute 24 seconds to generate the > artifacts.zip with the firmware binaries (filesize: 111KiB). > > See: https://gitlab.com/lbmeng/qemu/pipelines/120520138 > > Suggested-by: Philippe Mathieu-Daudé <philmd@redhat.com> > Signed-off-by: Bin Meng <bmeng.cn@gmail.com> Acked-by: Alistair Francis <alistair.francis@wdc.com> Alistair > > --- > > Changes in v2: > - new patch: Add GitLab jobs to build OpenSBI firmware binaries > > .gitlab-ci-opensbi.yml | 63 +++++++++++++++++++++++++++++++++++++++++ > .gitlab-ci.d/opensbi/Dockerfile | 33 +++++++++++++++++++++ > .gitlab-ci.yml | 1 + > 3 files changed, 97 insertions(+) > create mode 100644 .gitlab-ci-opensbi.yml > create mode 100644 .gitlab-ci.d/opensbi/Dockerfile > > diff --git a/.gitlab-ci-opensbi.yml b/.gitlab-ci-opensbi.yml > new file mode 100644 > index 0000000..dd051c0 > --- /dev/null > +++ b/.gitlab-ci-opensbi.yml > @@ -0,0 +1,63 @@ > +docker-opensbi: > + stage: build > + rules: # Only run this job when the Dockerfile is modified > + - changes: > + - .gitlab-ci-opensbi.yml > + - .gitlab-ci.d/opensbi/Dockerfile > + when: always > + image: docker:19.03.1 > + services: > + - docker:19.03.1-dind > + variables: > + GIT_DEPTH: 3 > + IMAGE_TAG: $CI_REGISTRY_IMAGE:opensbi-cross-build > + # We don't use TLS > + DOCKER_HOST: tcp://docker:2375 > + DOCKER_TLS_CERTDIR: "" > + before_script: > + - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY > + script: > + - docker pull $IMAGE_TAG || true > + - docker build --cache-from $IMAGE_TAG --tag $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA > + --tag $IMAGE_TAG .gitlab-ci.d/opensbi > + - docker push $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA > + - docker push $IMAGE_TAG > + > +build-opensbi: > + rules: # Only run this job when ... > + - changes: # ... roms/opensbi/ is modified (submodule updated) > + - roms/opensbi/* > + when: always > + - if: '$CI_COMMIT_REF_NAME =~ /^opensbi/' # or the branch/tag starts with 'opensbi' > + when: always > + - if: '$CI_COMMIT_MESSAGE =~ /opensbi/i' # or last commit description contains 'OpenSBI' > + when: always > + artifacts: > + paths: # 'artifacts.zip' will contains the following files: > + - pc-bios/opensbi-riscv32-sifive_u-fw_jump.bin > + - pc-bios/opensbi-riscv32-virt-fw_jump.bin > + - pc-bios/opensbi-riscv64-sifive_u-fw_jump.bin > + - pc-bios/opensbi-riscv64-virt-fw_jump.bin > + - opensbi32-virt-stdout.log > + - opensbi32-virt-stderr.log > + - opensbi64-virt-stdout.log > + - opensbi64-virt-stderr.log > + - opensbi32-sifive_u-stdout.log > + - opensbi32-sifive_u-stderr.log > + - opensbi64-sifive_u-stdout.log > + - opensbi64-sifive_u-stderr.log > + image: $CI_REGISTRY_IMAGE:opensbi-cross-build > + variables: > + GIT_DEPTH: 3 > + script: # Clone the required submodules and build OpenSBI > + - git submodule update --init roms/opensbi > + - export JOBS=$(($(getconf _NPROCESSORS_ONLN) + 1)) > + - echo "=== Using ${JOBS} simultaneous jobs ===" > + - make -j${JOBS} -C roms/opensbi clean > + - make -j${JOBS} -C roms opensbi32-virt 2>&1 1>opensbi32-virt-stdout.log | tee -a opensbi32-virt-stderr.log >&2 > + - make -j${JOBS} -C roms/opensbi clean > + - make -j${JOBS} -C roms opensbi64-virt 2>&1 1>opensbi64-virt-stdout.log | tee -a opensbi64-virt-stderr.log >&2 > + - make -j${JOBS} -C roms/opensbi clean > + - make -j${JOBS} -C roms opensbi32-sifive_u 2>&1 1>opensbi32-sifive_u-stdout.log | tee -a opensbi32-sifive_u-stderr.log >&2 > + - make -j${JOBS} -C roms/opensbi clean > + - make -j${JOBS} -C roms opensbi64-sifive_u 2>&1 1>opensbi64-sifive_u-stdout.log | tee -a opensbi64-sifive_u-stderr.log >&2 > diff --git a/.gitlab-ci.d/opensbi/Dockerfile b/.gitlab-ci.d/opensbi/Dockerfile > new file mode 100644 > index 0000000..4ba8a4d > --- /dev/null > +++ b/.gitlab-ci.d/opensbi/Dockerfile > @@ -0,0 +1,33 @@ > +# > +# Docker image to cross-compile OpenSBI firmware binaries > +# > +FROM ubuntu:18.04 > + > +MAINTAINER Bin Meng <bmeng.cn@gmail.com> > + > +# Install packages required to build OpenSBI > +RUN apt update \ > + && \ > + \ > + DEBIAN_FRONTEND=noninteractive \ > + apt install --assume-yes --no-install-recommends \ > + build-essential \ > + ca-certificates \ > + git \ > + make \ > + wget \ > + && \ > + \ > + rm -rf /var/lib/apt/lists/* > + > +# Manually install the kernel.org "Crosstool" based toolchains for gcc-8.3 > +RUN wget -O - \ > + https://mirrors.edge.kernel.org/pub/tools/crosstool/files/bin/x86_64/8.3.0/x86_64-gcc-8.3.0-nolibc-riscv32-linux.tar.xz \ > + | tar -C /opt -xJ > +RUN wget -O - \ > + https://mirrors.edge.kernel.org/pub/tools/crosstool/files/bin/x86_64/8.3.0/x86_64-gcc-8.3.0-nolibc-riscv64-linux.tar.xz \ > + | tar -C /opt -xJ > + > +# Export the toolchains to the system path > +ENV PATH="/opt/gcc-8.3.0-nolibc/riscv32-linux/bin:${PATH}" > +ENV PATH="/opt/gcc-8.3.0-nolibc/riscv64-linux/bin:${PATH}" > diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml > index c15e394..4c3a72b 100644 > --- a/.gitlab-ci.yml > +++ b/.gitlab-ci.yml > @@ -1,5 +1,6 @@ > include: > - local: '/.gitlab-ci-edk2.yml' > + - local: '/.gitlab-ci-opensbi.yml' > > before_script: > - apt-get update -qq > -- > 2.7.4 > >
On Mon, 24 Feb 2020 05:39:44 PST (-0800), bmeng.cn@gmail.com wrote: > Add two GitLab jobs to build the OpenSBI firmware binaries. > > The first job builds a Docker image with the packages requisite > to build OpenSBI, and stores this image in the GitLab registry. > The second job pulls the image from the registry and builds the > OpenSBI firmware binaries. > > The docker image is only rebuilt if the GitLab YAML or the > Dockerfile is updated. The second job is only built when the > roms/opensbi/ submodule is updated, when a git-ref starts with > 'opensbi' or when the last commit contains 'OpenSBI'. The files > generated are archived in the artifacts.zip file. > > With OpenSBI v0.6, it took 2 minutes 56 seconds to build > the docker image, and 1 minute 24 seconds to generate the > artifacts.zip with the firmware binaries (filesize: 111KiB). > > See: https://gitlab.com/lbmeng/qemu/pipelines/120520138 > > Suggested-by: Philippe Mathieu-Daudé <philmd@redhat.com> > Signed-off-by: Bin Meng <bmeng.cn@gmail.com> I don't really understand how this works well enough to be comfortable putting my Reviewed-by on it -- I'm certainly not opposed to it, it's just not really my thing. I've put it in my queue without a review, but my understanding is that we need one to send it up (and I usually avoid putting things in my queue until they have a review). If nobody else has time to review it then I'll do so before sending the PR, but that may delay things. It'd be great if someone could save me the time... :) Thanks for doing this, though -- wve're sorely lacking in CI! > > --- > > Changes in v2: > - new patch: Add GitLab jobs to build OpenSBI firmware binaries > > .gitlab-ci-opensbi.yml | 63 +++++++++++++++++++++++++++++++++++++++++ > .gitlab-ci.d/opensbi/Dockerfile | 33 +++++++++++++++++++++ > .gitlab-ci.yml | 1 + > 3 files changed, 97 insertions(+) > create mode 100644 .gitlab-ci-opensbi.yml > create mode 100644 .gitlab-ci.d/opensbi/Dockerfile > > diff --git a/.gitlab-ci-opensbi.yml b/.gitlab-ci-opensbi.yml > new file mode 100644 > index 0000000..dd051c0 > --- /dev/null > +++ b/.gitlab-ci-opensbi.yml > @@ -0,0 +1,63 @@ > +docker-opensbi: > + stage: build > + rules: # Only run this job when the Dockerfile is modified > + - changes: > + - .gitlab-ci-opensbi.yml > + - .gitlab-ci.d/opensbi/Dockerfile > + when: always > + image: docker:19.03.1 > + services: > + - docker:19.03.1-dind > + variables: > + GIT_DEPTH: 3 > + IMAGE_TAG: $CI_REGISTRY_IMAGE:opensbi-cross-build > + # We don't use TLS > + DOCKER_HOST: tcp://docker:2375 > + DOCKER_TLS_CERTDIR: "" > + before_script: > + - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY > + script: > + - docker pull $IMAGE_TAG || true > + - docker build --cache-from $IMAGE_TAG --tag $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA > + --tag $IMAGE_TAG .gitlab-ci.d/opensbi > + - docker push $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA > + - docker push $IMAGE_TAG > + > +build-opensbi: > + rules: # Only run this job when ... > + - changes: # ... roms/opensbi/ is modified (submodule updated) > + - roms/opensbi/* > + when: always > + - if: '$CI_COMMIT_REF_NAME =~ /^opensbi/' # or the branch/tag starts with 'opensbi' > + when: always > + - if: '$CI_COMMIT_MESSAGE =~ /opensbi/i' # or last commit description contains 'OpenSBI' > + when: always > + artifacts: > + paths: # 'artifacts.zip' will contains the following files: > + - pc-bios/opensbi-riscv32-sifive_u-fw_jump.bin > + - pc-bios/opensbi-riscv32-virt-fw_jump.bin > + - pc-bios/opensbi-riscv64-sifive_u-fw_jump.bin > + - pc-bios/opensbi-riscv64-virt-fw_jump.bin > + - opensbi32-virt-stdout.log > + - opensbi32-virt-stderr.log > + - opensbi64-virt-stdout.log > + - opensbi64-virt-stderr.log > + - opensbi32-sifive_u-stdout.log > + - opensbi32-sifive_u-stderr.log > + - opensbi64-sifive_u-stdout.log > + - opensbi64-sifive_u-stderr.log > + image: $CI_REGISTRY_IMAGE:opensbi-cross-build > + variables: > + GIT_DEPTH: 3 > + script: # Clone the required submodules and build OpenSBI > + - git submodule update --init roms/opensbi > + - export JOBS=$(($(getconf _NPROCESSORS_ONLN) + 1)) > + - echo "=== Using ${JOBS} simultaneous jobs ===" > + - make -j${JOBS} -C roms/opensbi clean > + - make -j${JOBS} -C roms opensbi32-virt 2>&1 1>opensbi32-virt-stdout.log | tee -a opensbi32-virt-stderr.log >&2 > + - make -j${JOBS} -C roms/opensbi clean > + - make -j${JOBS} -C roms opensbi64-virt 2>&1 1>opensbi64-virt-stdout.log | tee -a opensbi64-virt-stderr.log >&2 > + - make -j${JOBS} -C roms/opensbi clean > + - make -j${JOBS} -C roms opensbi32-sifive_u 2>&1 1>opensbi32-sifive_u-stdout.log | tee -a opensbi32-sifive_u-stderr.log >&2 > + - make -j${JOBS} -C roms/opensbi clean > + - make -j${JOBS} -C roms opensbi64-sifive_u 2>&1 1>opensbi64-sifive_u-stdout.log | tee -a opensbi64-sifive_u-stderr.log >&2 > diff --git a/.gitlab-ci.d/opensbi/Dockerfile b/.gitlab-ci.d/opensbi/Dockerfile > new file mode 100644 > index 0000000..4ba8a4d > --- /dev/null > +++ b/.gitlab-ci.d/opensbi/Dockerfile > @@ -0,0 +1,33 @@ > +# > +# Docker image to cross-compile OpenSBI firmware binaries > +# > +FROM ubuntu:18.04 > + > +MAINTAINER Bin Meng <bmeng.cn@gmail.com> > + > +# Install packages required to build OpenSBI > +RUN apt update \ > + && \ > + \ > + DEBIAN_FRONTEND=noninteractive \ > + apt install --assume-yes --no-install-recommends \ > + build-essential \ > + ca-certificates \ > + git \ > + make \ > + wget \ > + && \ > + \ > + rm -rf /var/lib/apt/lists/* > + > +# Manually install the kernel.org "Crosstool" based toolchains for gcc-8.3 > +RUN wget -O - \ > + https://mirrors.edge.kernel.org/pub/tools/crosstool/files/bin/x86_64/8.3.0/x86_64-gcc-8.3.0-nolibc-riscv32-linux.tar.xz \ > + | tar -C /opt -xJ > +RUN wget -O - \ > + https://mirrors.edge.kernel.org/pub/tools/crosstool/files/bin/x86_64/8.3.0/x86_64-gcc-8.3.0-nolibc-riscv64-linux.tar.xz \ > + | tar -C /opt -xJ > + > +# Export the toolchains to the system path > +ENV PATH="/opt/gcc-8.3.0-nolibc/riscv32-linux/bin:${PATH}" > +ENV PATH="/opt/gcc-8.3.0-nolibc/riscv64-linux/bin:${PATH}" > diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml > index c15e394..4c3a72b 100644 > --- a/.gitlab-ci.yml > +++ b/.gitlab-ci.yml > @@ -1,5 +1,6 @@ > include: > - local: '/.gitlab-ci-edk2.yml' > + - local: '/.gitlab-ci-opensbi.yml' > > before_script: > - apt-get update -qq
diff --git a/.gitlab-ci-opensbi.yml b/.gitlab-ci-opensbi.yml new file mode 100644 index 0000000..dd051c0 --- /dev/null +++ b/.gitlab-ci-opensbi.yml @@ -0,0 +1,63 @@ +docker-opensbi: + stage: build + rules: # Only run this job when the Dockerfile is modified + - changes: + - .gitlab-ci-opensbi.yml + - .gitlab-ci.d/opensbi/Dockerfile + when: always + image: docker:19.03.1 + services: + - docker:19.03.1-dind + variables: + GIT_DEPTH: 3 + IMAGE_TAG: $CI_REGISTRY_IMAGE:opensbi-cross-build + # We don't use TLS + DOCKER_HOST: tcp://docker:2375 + DOCKER_TLS_CERTDIR: "" + before_script: + - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY + script: + - docker pull $IMAGE_TAG || true + - docker build --cache-from $IMAGE_TAG --tag $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA + --tag $IMAGE_TAG .gitlab-ci.d/opensbi + - docker push $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA + - docker push $IMAGE_TAG + +build-opensbi: + rules: # Only run this job when ... + - changes: # ... roms/opensbi/ is modified (submodule updated) + - roms/opensbi/* + when: always + - if: '$CI_COMMIT_REF_NAME =~ /^opensbi/' # or the branch/tag starts with 'opensbi' + when: always + - if: '$CI_COMMIT_MESSAGE =~ /opensbi/i' # or last commit description contains 'OpenSBI' + when: always + artifacts: + paths: # 'artifacts.zip' will contains the following files: + - pc-bios/opensbi-riscv32-sifive_u-fw_jump.bin + - pc-bios/opensbi-riscv32-virt-fw_jump.bin + - pc-bios/opensbi-riscv64-sifive_u-fw_jump.bin + - pc-bios/opensbi-riscv64-virt-fw_jump.bin + - opensbi32-virt-stdout.log + - opensbi32-virt-stderr.log + - opensbi64-virt-stdout.log + - opensbi64-virt-stderr.log + - opensbi32-sifive_u-stdout.log + - opensbi32-sifive_u-stderr.log + - opensbi64-sifive_u-stdout.log + - opensbi64-sifive_u-stderr.log + image: $CI_REGISTRY_IMAGE:opensbi-cross-build + variables: + GIT_DEPTH: 3 + script: # Clone the required submodules and build OpenSBI + - git submodule update --init roms/opensbi + - export JOBS=$(($(getconf _NPROCESSORS_ONLN) + 1)) + - echo "=== Using ${JOBS} simultaneous jobs ===" + - make -j${JOBS} -C roms/opensbi clean + - make -j${JOBS} -C roms opensbi32-virt 2>&1 1>opensbi32-virt-stdout.log | tee -a opensbi32-virt-stderr.log >&2 + - make -j${JOBS} -C roms/opensbi clean + - make -j${JOBS} -C roms opensbi64-virt 2>&1 1>opensbi64-virt-stdout.log | tee -a opensbi64-virt-stderr.log >&2 + - make -j${JOBS} -C roms/opensbi clean + - make -j${JOBS} -C roms opensbi32-sifive_u 2>&1 1>opensbi32-sifive_u-stdout.log | tee -a opensbi32-sifive_u-stderr.log >&2 + - make -j${JOBS} -C roms/opensbi clean + - make -j${JOBS} -C roms opensbi64-sifive_u 2>&1 1>opensbi64-sifive_u-stdout.log | tee -a opensbi64-sifive_u-stderr.log >&2 diff --git a/.gitlab-ci.d/opensbi/Dockerfile b/.gitlab-ci.d/opensbi/Dockerfile new file mode 100644 index 0000000..4ba8a4d --- /dev/null +++ b/.gitlab-ci.d/opensbi/Dockerfile @@ -0,0 +1,33 @@ +# +# Docker image to cross-compile OpenSBI firmware binaries +# +FROM ubuntu:18.04 + +MAINTAINER Bin Meng <bmeng.cn@gmail.com> + +# Install packages required to build OpenSBI +RUN apt update \ + && \ + \ + DEBIAN_FRONTEND=noninteractive \ + apt install --assume-yes --no-install-recommends \ + build-essential \ + ca-certificates \ + git \ + make \ + wget \ + && \ + \ + rm -rf /var/lib/apt/lists/* + +# Manually install the kernel.org "Crosstool" based toolchains for gcc-8.3 +RUN wget -O - \ + https://mirrors.edge.kernel.org/pub/tools/crosstool/files/bin/x86_64/8.3.0/x86_64-gcc-8.3.0-nolibc-riscv32-linux.tar.xz \ + | tar -C /opt -xJ +RUN wget -O - \ + https://mirrors.edge.kernel.org/pub/tools/crosstool/files/bin/x86_64/8.3.0/x86_64-gcc-8.3.0-nolibc-riscv64-linux.tar.xz \ + | tar -C /opt -xJ + +# Export the toolchains to the system path +ENV PATH="/opt/gcc-8.3.0-nolibc/riscv32-linux/bin:${PATH}" +ENV PATH="/opt/gcc-8.3.0-nolibc/riscv64-linux/bin:${PATH}" diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index c15e394..4c3a72b 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -1,5 +1,6 @@ include: - local: '/.gitlab-ci-edk2.yml' + - local: '/.gitlab-ci-opensbi.yml' before_script: - apt-get update -qq
Add two GitLab jobs to build the OpenSBI firmware binaries. The first job builds a Docker image with the packages requisite to build OpenSBI, and stores this image in the GitLab registry. The second job pulls the image from the registry and builds the OpenSBI firmware binaries. The docker image is only rebuilt if the GitLab YAML or the Dockerfile is updated. The second job is only built when the roms/opensbi/ submodule is updated, when a git-ref starts with 'opensbi' or when the last commit contains 'OpenSBI'. The files generated are archived in the artifacts.zip file. With OpenSBI v0.6, it took 2 minutes 56 seconds to build the docker image, and 1 minute 24 seconds to generate the artifacts.zip with the firmware binaries (filesize: 111KiB). See: https://gitlab.com/lbmeng/qemu/pipelines/120520138 Suggested-by: Philippe Mathieu-Daudé <philmd@redhat.com> Signed-off-by: Bin Meng <bmeng.cn@gmail.com> --- Changes in v2: - new patch: Add GitLab jobs to build OpenSBI firmware binaries .gitlab-ci-opensbi.yml | 63 +++++++++++++++++++++++++++++++++++++++++ .gitlab-ci.d/opensbi/Dockerfile | 33 +++++++++++++++++++++ .gitlab-ci.yml | 1 + 3 files changed, 97 insertions(+) create mode 100644 .gitlab-ci-opensbi.yml create mode 100644 .gitlab-ci.d/opensbi/Dockerfile