| Message ID | 20230531150824.32349-6-cconte@redhat.com |
|---|---|
| State | New |
| Headers | show |
| Series | [1/5] gitlab-ci: Remove unused Python package | expand |
On Wed, May 31, 2023 at 04:08:24PM +0100, Camilla Conte wrote: > Previous attempt by Alex Bennée, for the records: > https://lore.kernel.org/qemu-devel/20230330101141.30199-12-alex.bennee@linaro.org/ > > Does not enable caching, as suggested in a previous review: > https://lore.kernel.org/qemu-devel/ZCVpMDLPJcdnkAJz@redhat.com/ In this mail I suggested we entirely skip running kaniko if we determine the dockerfile content is unchanged since last build... > diff --git a/.gitlab-ci.d/container-template.yml b/.gitlab-ci.d/container-template.yml > index d146d1d197..0e93f5f13e 100644 > --- a/.gitlab-ci.d/container-template.yml > +++ b/.gitlab-ci.d/container-template.yml > + - /kaniko/executor > + --dockerfile "$DOCKERFILE" > + --destination "$TAG" ....this is running kaniko unconditionally on every pipeline. I fear this is going to increase the CI credits we are burning on every pipeline. While QEMU has added its private k8s runner, our contributors forks are still burning limited CI credits. Do you have pipelines showing the CI credit usage of a pipeline before and after this change ? With regards, Daniel
diff --git a/.gitlab-ci.d/container-template.yml b/.gitlab-ci.d/container-template.yml index d146d1d197..0e93f5f13e 100644 --- a/.gitlab-ci.d/container-template.yml +++ b/.gitlab-ci.d/container-template.yml @@ -1,27 +1,14 @@ .container_job_template: extends: .base_job_template stage: containers - image: docker:stable - services: - - docker:stable-dind + image: + name: gcr.io/kaniko-project/executor:debug + entrypoint: [""] variables: TAG: "$CI_REGISTRY_IMAGE/qemu/$NAME:latest" - COMMON_TAG: "$CI_REGISTRY/qemu-project/qemu/qemu/$NAME:latest" DOCKERFILE: "$CI_PROJECT_DIR/tests/docker/dockerfiles/$NAME.docker" - BUILD_CONTEXT: "$CI_PROJECT_DIR" - before_script: - - docker login $CI_REGISTRY -u "$CI_REGISTRY_USER" -p "$CI_REGISTRY_PASSWORD" - - until docker info; do sleep 1; done script: - echo "TAG:$TAG" - - echo "COMMON_TAG:$COMMON_TAG" - - > - docker build - --tag "$TAG" - --cache-from "$TAG" - --cache-from "$COMMON_TAG" - --build-arg BUILDKIT_INLINE_CACHE=1 - -f "$DOCKERFILE" "$BUILD_CONTEXT" - - docker push "$TAG" - after_script: - - docker logout + - /kaniko/executor + --dockerfile "$DOCKERFILE" + --destination "$TAG" diff --git a/.gitlab-ci.d/opensbi.yml b/.gitlab-ci.d/opensbi.yml index bb90c56bcf..e531ce2728 100644 --- a/.gitlab-ci.d/opensbi.yml +++ b/.gitlab-ci.d/opensbi.yml @@ -50,7 +50,6 @@ docker-opensbi: GIT_DEPTH: 3 NAME: opensbi-cross-build DOCKERFILE: "$CI_PROJECT_DIR/.gitlab-ci.d/opensbi/Dockerfile" - BUILD_CONTEXT: "$CI_PROJECT_DIR/.gitlab-ci.d/opensbi" build-opensbi: rules: diff --git a/scripts/ci/gitlab-kubernetes-runners/values.yaml b/scripts/ci/gitlab-kubernetes-runners/values.yaml index 204a96a842..ed6c5894bc 100644 --- a/scripts/ci/gitlab-kubernetes-runners/values.yaml +++ b/scripts/ci/gitlab-kubernetes-runners/values.yaml @@ -4,27 +4,15 @@ rbac: create: true concurrent: 200 runners: - privileged: true config: | [[runners]] limit = 100 - environment = [ - "DOCKER_HOST=tcp://docker:2376", - "DOCKER_TLS_CERTDIR=/certs", - "DOCKER_TLS_VERIFY=1", - "DOCKER_CERT_PATH=/certs/client" - ] [runners.kubernetes] poll_timeout = 1200 - image = "ubuntu:20.04" cpu_request = "0.5" service_cpu_request = "0.5" helper_cpu_request = "0.25" cpu_request_overwrite_max_allowed = "7" memory_request_overwrite_max_allowed = "30Gi" - [[runners.kubernetes.volumes.empty_dir]] - name = "docker-certs" - mount_path = "/certs/client" - medium = "Memory" [runners.kubernetes.node_selector] agentpool = "jobs"
Previous attempt by Alex Bennée, for the records: https://lore.kernel.org/qemu-devel/20230330101141.30199-12-alex.bennee@linaro.org/ Does not enable caching, as suggested in a previous review: https://lore.kernel.org/qemu-devel/ZCVpMDLPJcdnkAJz@redhat.com/ Does not specify a context since no Dockerfile is using COPY or ADD instructions. Does not enable reproducible builds as that results in builds failing with an out of memory error. See issue "Using --reproducible loads entire image into memory": https://github.com/GoogleContainerTools/kaniko/issues/862 Signed-off-by: Camilla Conte <cconte@redhat.com> --- .gitlab-ci.d/container-template.yml | 25 +++++-------------- .gitlab-ci.d/opensbi.yml | 1 - .../ci/gitlab-kubernetes-runners/values.yaml | 12 --------- 3 files changed, 6 insertions(+), 32 deletions(-)