| Message ID | 20210415215141.1865467-1-crosa@redhat.com |
|---|---|
| Headers | show |
| Series | Tests: introduce custom jobs | expand |
On 15/04/21 23:51, Cleber Rosa wrote: > Different users (or even companies) have different interests, and > may want to run a reduced set of tests during development, or a > larger set of tests during QE. > > To cover these use cases, this introduces some example (but > functional) jobs. > > It's expected that some common jobs will come up from common > requirements for different users (and maybe be added to a common > location such as tests/jobs), and that very specific jobs will be > added to directories specific to certain groups, say > "contrib/com.redhat/jobs" or the like. > > This series does*not* add new jobs to GitLab CI pipeline, but this is > expected to be done later on custom runners. That is, custom runners > could be used for custom jobs. Anyway, a GitLab CI pipeline can be > seen here: > > https://gitlab.com/cleber.gnu/qemu/-/pipelines/287210066 > > This is based on the Avocado version bump patch: > > https://lists.gnu.org/archive/html/qemu-devel/2021-04/msg02391.html I admit I haven't even having read the patches (only the diffstat), but still: documentation please. Paolo
On Fri, Apr 16, 2021 at 06:22:12PM +0200, Paolo Bonzini wrote: > On 15/04/21 23:51, Cleber Rosa wrote: > > Different users (or even companies) have different interests, and > > may want to run a reduced set of tests during development, or a > > larger set of tests during QE. > > > > To cover these use cases, this introduces some example (but > > functional) jobs. > > > > It's expected that some common jobs will come up from common > > requirements for different users (and maybe be added to a common > > location such as tests/jobs), and that very specific jobs will be > > added to directories specific to certain groups, say > > "contrib/com.redhat/jobs" or the like. > > > > This series does*not* add new jobs to GitLab CI pipeline, but this is > > expected to be done later on custom runners. That is, custom runners > > could be used for custom jobs. Anyway, a GitLab CI pipeline can be > > seen here: > > > > https://gitlab.com/cleber.gnu/qemu/-/pipelines/287210066 > > > > This is based on the Avocado version bump patch: > > > > https://lists.gnu.org/archive/html/qemu-devel/2021-04/msg02391.html > > I admit I haven't even having read the patches (only the diffstat), but > still: documentation please. > > Paolo > Hi Paolo, Absolutely, formal docs are very much needed and will be provided. For now, please refer to patch 8's commit message. It contains basic usage information for these jobs, and pointers to external documentation. PS: I'm very much interested in knowing what are user's first questions or use cases, so that I can tune the documentation accordingly. Questions and ideas like the one from Phil (about a YAML config file) would definitely help me to write a more relevant set of docs. Thanks, - Cleber.
Different users (or even companies) have different interests, and may want to run a reduced set of tests during development, or a larger set of tests during QE. To cover these use cases, this introduces some example (but functional) jobs. It's expected that some common jobs will come up from common requirements for different users (and maybe be added to a common location such as tests/jobs), and that very specific jobs will be added to directories specific to certain groups, say "contrib/com.redhat/jobs" or the like. This series does *not* add new jobs to GitLab CI pipeline, but this is expected to be done later on custom runners. That is, custom runners could be used for custom jobs. Anyway, a GitLab CI pipeline can be seen here: https://gitlab.com/cleber.gnu/qemu/-/pipelines/287210066 This is based on the Avocado version bump patch: https://lists.gnu.org/archive/html/qemu-devel/2021-04/msg02391.html Based-On: <20210414161144.1598980-1-crosa@redhat.com> Cleber Rosa (8): Acceptance Jobs: preserve the cache for pip on GitLab CI Acceptance tests: do not try to reuse packages from the system tests/acceptance/linux_ssh_mips_malta.py: drop identical setUp tests/acceptance/migration.py: cancel test if migration is not supported tests/acceptance/cpu_queries.py: use the proper logging channels Acceptance tests: prevent shutdown on non-specific target tests tests/acceptance/migration.py: cancel test on s390x Tests: add custom test jobs .gitlab-ci.yml | 1 + configure | 2 +- tests/Makefile.include | 10 +++- tests/acceptance/cpu_queries.py | 4 +- tests/acceptance/linux_ssh_mips_malta.py | 7 +-- tests/acceptance/migration.py | 16 ++++-- tests/acceptance/version.py | 2 +- tests/jobs/acceptance-all-targets.py | 67 ++++++++++++++++++++++++ tests/jobs/acceptance-kvm-only.py | 35 +++++++++++++ tests/jobs/qtest-unit-acceptance.py | 31 +++++++++++ tests/jobs/qtest-unit.py | 24 +++++++++ tests/jobs/utils.py | 22 ++++++++ 12 files changed, 207 insertions(+), 14 deletions(-) create mode 100644 tests/jobs/acceptance-all-targets.py create mode 100644 tests/jobs/acceptance-kvm-only.py create mode 100644 tests/jobs/qtest-unit-acceptance.py create mode 100644 tests/jobs/qtest-unit.py create mode 100644 tests/jobs/utils.py