diff mbox series

[1/2] gitlab: prevent CI jobs running pushes to default branch or tags

Message ID 20210719121215.2356409-2-berrange@redhat.com
State New
Headers show
Series gitlab: limit jobs executed in most pipelines by default | expand

Commit Message

Daniel P. Berrangé July 19, 2021, 12:12 p.m. UTC
When patch series are accepted they are first pushed to the 'staging'
branch, which triggers a CI pipeline. If-and-only-if the pipeline
succeeds, the code is then pushed to the default branch. The latter
step triggers a second pipeline that duplicates what was just validated
in the 'staging' branch. This wastes CI resources and can delay jobs
needed by a subsequently pushed 'staging' branch.

This is particularly evident with jobs delegated to Cirrus CI where
there is very limited concurrency of jobs. The duplicate jobs in
'master' often delay the next 'staging' jobs enough to cause 1 hour
timeout to be hit.

There is similar wasted CI resource when release tags are pushed to
the repo, since the branch associated with the release will have
already passed CI.

For user forks is is reasonable to skip CI on the default branch since
that's either usually unused or follows the main repo default branch.
Meanwhile tags in user forks are typically used to track postings of
specific patch series versions. It is expected that a contributor
will have tested before sending the patch series. Thus pipelines for
the default branch and tag pushes in forks are redundant too.

This patch makes use of the 'workflow:' configuration to setup rules
that apply to the entire pipeline. The rules allow pipelines to be
created ONLY in response to:

 - Push events to a non-default branch
 - Scheduled pipelines
 - REST API pipeline triggers
 - Web UI pipeline triggers

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
---
 .gitlab-ci.d/qemu-project.yml |  1 +
 .gitlab-ci.d/workflow.yml     | 56 +++++++++++++++++++++++++++++++++++
 2 files changed, 57 insertions(+)
 create mode 100644 .gitlab-ci.d/workflow.yml
diff mbox series

Patch

diff --git a/.gitlab-ci.d/qemu-project.yml b/.gitlab-ci.d/qemu-project.yml
index b3d79bc429..13202e31e9 100644
--- a/.gitlab-ci.d/qemu-project.yml
+++ b/.gitlab-ci.d/qemu-project.yml
@@ -2,6 +2,7 @@ 
 # https://gitlab.com/qemu-project/qemu/-/pipelines
 
 include:
+  - local: '/.gitlab-ci.d/workflow.yml'
   - local: '/.gitlab-ci.d/stages.yml'
   - local: '/.gitlab-ci.d/edk2.yml'
   - local: '/.gitlab-ci.d/opensbi.yml'
diff --git a/.gitlab-ci.d/workflow.yml b/.gitlab-ci.d/workflow.yml
new file mode 100644
index 0000000000..5c877bae11
--- /dev/null
+++ b/.gitlab-ci.d/workflow.yml
@@ -0,0 +1,56 @@ 
+
+# This defines global scenarios for controlling execution of the
+# entire pipeline. The pipelines are set to run when
+#
+#  - Pushing to a non-default branch
+#  - Triggered via REST API
+#  - Triggered via Web UI
+#  - Regular scheduled run
+#
+workflow:
+  rules:
+    # For the main repo, prior to merging a patch series, CI is run
+    # on the "staging" branch. The result gates whether the series
+    # is then pushed to the default branch.
+    #
+    # For user forks, the default branch is typically unused or
+    # otherwise just tracks the main repo default branch in an
+    # adhoc manner. Work is typically done on feature branches
+    # instead.
+    #
+    # In neither case do we wish to run CI pipelines for the
+    # default branch
+    - if: '$CI_PIPELINE_SOURCE == "push" && $CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH'
+      when: never
+
+    # For the main repo, tags represent official releases.
+    # The branch associated with the release will have passed
+    # a CI pipeline already
+    #
+    # For user forks, tags are often added by tools like
+    # git-publish at the same time as pushing the branch.
+    #
+    # In neither case do we wish to run CI pipelines for tags
+    - if: '$CI_PIPELINE_SOURCE == "push" && $CI_COMMIT_TAG'
+      when: never
+
+    # Run CI on any git pushes to remaining non-dfault branches
+    - if: '$CI_PIPELINE_SOURCE == "push" && $CI_COMMIT_BRANCH'
+      when: always
+
+    # Run CI on any regular scheduled pipelines (ie nightly builds)
+    - if: '$CI_PIPELINE_SOURCE == "schedule"'
+      when: always
+
+    # Run CI if user invokes a pipeline using explicit REST API call
+    - if: '$CI_PIPELINE_SOURCE == "api"'
+      when: always
+
+    # Run CI if user invokes a pipeline using Web UI
+    - if: '$CI_PIPELINE_SOURCE == "web"'
+      when: always
+
+    # Don't run CI in any other scenario ($CI_PIPELINE_SOURCE values
+    # covering: external, chat, webide, merge_request_event,
+    # external_pull_request_event, parent_pipeline, trigger, or pipeline)
+    - when: never