| Message ID | 20190117185628.21862-7-crosa@redhat.com |
|---|---|
| State | New |
| Headers | show |
| Series | Acceptance Tests: target architecture support | expand |
On Thu, 17 Jan 2019 13:56:16 -0500 Cleber Rosa <crosa@redhat.com> wrote: > Currently, the only test that contains some target architecture > information is "boot_linux_console.py" which test contains a "x86_64" > tag. But that tag is not respected in the default execution, that is, > "make check-acceptance" doesn't do anything with it. > > That said, even the target architecture handling currently present in > the "avocado_qemu.Test" class, class is pretty limited. For instance, > by default, it chooses a target based on the host architecture. > > Because the original implementation of the tags feature in Avocado did > not include any time of namespace or "key:val" mechanism, no tag has > relation to another tag. The new implementation of the tags feature > from version 67.0 onwards, allows "key:val" tags, and because of that, > a test can be classified with a tag in a given key. For instance, the > new proposed version of the "boot_linux_console.py" test, which > downloads and attempts to run a x86_64 kernel, is now tagged as: > > :avocado: tags=arch:x86_64 > > This means that it can be filtered (out) when no x86_64 target is > available. At the same time, tests that don't have a "arch:" tag, > will not be filtered out. > > Signed-off-by: Cleber Rosa <crosa@redhat.com> > --- > tests/Makefile.include | 3 +++ > tests/acceptance/boot_linux_console.py | 2 +- > tests/requirements.txt | 2 +- > 3 files changed, 5 insertions(+), 2 deletions(-) > > diff --git a/tests/acceptance/boot_linux_console.py b/tests/acceptance/boot_linux_console.py > index 98324f7591..46b20bdfe2 100644 > --- a/tests/acceptance/boot_linux_console.py > +++ b/tests/acceptance/boot_linux_console.py > @@ -19,7 +19,7 @@ class BootLinuxConsole(Test): > and the kernel command line is properly passed from QEMU to the kernel > > :avocado: enable > - :avocado: tags=x86_64 > + :avocado: tags=arch:x86_64 > """ > > timeout = 60 You probably want to do the same change in virtio_version.py; otherwise, if I run the acceptance tests on s390x, it will run into timeouts (it looks like that test is intended to be run with x86 machines anyway.)
On Thu, Jan 17, 2019 at 01:56:16PM -0500, Cleber Rosa wrote: > Currently, the only test that contains some target architecture > information is "boot_linux_console.py" which test contains a "x86_64" > tag. But that tag is not respected in the default execution, that is, > "make check-acceptance" doesn't do anything with it. > > That said, even the target architecture handling currently present in > the "avocado_qemu.Test" class, class is pretty limited. For instance, > by default, it chooses a target based on the host architecture. > > Because the original implementation of the tags feature in Avocado did > not include any time of namespace or "key:val" mechanism, no tag has > relation to another tag. The new implementation of the tags feature > from version 67.0 onwards, allows "key:val" tags, and because of that, > a test can be classified with a tag in a given key. For instance, the > new proposed version of the "boot_linux_console.py" test, which > downloads and attempts to run a x86_64 kernel, is now tagged as: > > :avocado: tags=arch:x86_64 > > This means that it can be filtered (out) when no x86_64 target is > available. At the same time, tests that don't have a "arch:" tag, > will not be filtered out. > > Signed-off-by: Cleber Rosa <crosa@redhat.com> Reviewed-by: Caio Carrara <ccarrara@redhat.com> > --- > tests/Makefile.include | 3 +++ > tests/acceptance/boot_linux_console.py | 2 +- > tests/requirements.txt | 2 +- > 3 files changed, 5 insertions(+), 2 deletions(-) > {...} >
On 1/18/19 5:38 AM, Cornelia Huck wrote: > On Thu, 17 Jan 2019 13:56:16 -0500 > Cleber Rosa <crosa@redhat.com> wrote: > >> Currently, the only test that contains some target architecture >> information is "boot_linux_console.py" which test contains a "x86_64" >> tag. But that tag is not respected in the default execution, that is, >> "make check-acceptance" doesn't do anything with it. >> >> That said, even the target architecture handling currently present in >> the "avocado_qemu.Test" class, class is pretty limited. For instance, >> by default, it chooses a target based on the host architecture. >> >> Because the original implementation of the tags feature in Avocado did >> not include any time of namespace or "key:val" mechanism, no tag has >> relation to another tag. The new implementation of the tags feature >> from version 67.0 onwards, allows "key:val" tags, and because of that, >> a test can be classified with a tag in a given key. For instance, the >> new proposed version of the "boot_linux_console.py" test, which >> downloads and attempts to run a x86_64 kernel, is now tagged as: >> >> :avocado: tags=arch:x86_64 >> >> This means that it can be filtered (out) when no x86_64 target is >> available. At the same time, tests that don't have a "arch:" tag, >> will not be filtered out. >> >> Signed-off-by: Cleber Rosa <crosa@redhat.com> >> --- >> tests/Makefile.include | 3 +++ >> tests/acceptance/boot_linux_console.py | 2 +- >> tests/requirements.txt | 2 +- >> 3 files changed, 5 insertions(+), 2 deletions(-) >> > >> diff --git a/tests/acceptance/boot_linux_console.py b/tests/acceptance/boot_linux_console.py >> index 98324f7591..46b20bdfe2 100644 >> --- a/tests/acceptance/boot_linux_console.py >> +++ b/tests/acceptance/boot_linux_console.py >> @@ -19,7 +19,7 @@ class BootLinuxConsole(Test): >> and the kernel command line is properly passed from QEMU to the kernel >> >> :avocado: enable >> - :avocado: tags=x86_64 >> + :avocado: tags=arch:x86_64 >> """ >> >> timeout = 60 > > You probably want to do the same change in virtio_version.py; > otherwise, if I run the acceptance tests on s390x, it will run into > timeouts (it looks like that test is intended to be run with x86 > machines anyway.) > Right, good catch. Also, the same applies to the "linux_initrd.py" test, merged after this version. - Cleber.
diff --git a/tests/Makefile.include b/tests/Makefile.include index c73298740d..af00c35ff5 100644 --- a/tests/Makefile.include +++ b/tests/Makefile.include @@ -957,6 +957,7 @@ TESTS_RESULTS_DIR=$(BUILD_DIR)/tests/results # Any number of command separated loggers are accepted. For more # information please refer to "avocado --help". AVOCADO_SHOW=app +AVOCADO_TAGS=$(patsubst %-softmmu,-t arch:%, $(filter %-softmmu,$(TARGET_DIRS))) PYTHON3 = $(shell $(PYTHON) -c 'import sys; print(1 if sys.version_info >= (3, 0) else 0)') ifeq ($(PYTHON3), 1) @@ -983,6 +984,8 @@ check-acceptance: check-venv $(TESTS_RESULTS_DIR) $(call quiet-command, \ $(TESTS_VENV_DIR)/bin/python -m avocado \ --show=$(AVOCADO_SHOW) run --job-results-dir=$(TESTS_RESULTS_DIR) \ + --filter-by-tags-include-empty --filter-by-tags-include-empty-key \ + $(AVOCADO_TAGS) \ --failfast=on $(SRC_PATH)/tests/acceptance, \ "AVOCADO", "tests/acceptance") diff --git a/tests/acceptance/boot_linux_console.py b/tests/acceptance/boot_linux_console.py index 98324f7591..46b20bdfe2 100644 --- a/tests/acceptance/boot_linux_console.py +++ b/tests/acceptance/boot_linux_console.py @@ -19,7 +19,7 @@ class BootLinuxConsole(Test): and the kernel command line is properly passed from QEMU to the kernel :avocado: enable - :avocado: tags=x86_64 + :avocado: tags=arch:x86_64 """ timeout = 60 diff --git a/tests/requirements.txt b/tests/requirements.txt index 64c6e27a94..426a9e8a9a 100644 --- a/tests/requirements.txt +++ b/tests/requirements.txt @@ -1,4 +1,4 @@ # Add Python module requirements, one per line, to be installed # in the tests/venv Python virtual environment. For more info, # refer to: https://pip.pypa.io/en/stable/user_guide/#id1 -avocado-framework==65.0 +avocado-framework==67.0
Currently, the only test that contains some target architecture information is "boot_linux_console.py" which test contains a "x86_64" tag. But that tag is not respected in the default execution, that is, "make check-acceptance" doesn't do anything with it. That said, even the target architecture handling currently present in the "avocado_qemu.Test" class, class is pretty limited. For instance, by default, it chooses a target based on the host architecture. Because the original implementation of the tags feature in Avocado did not include any time of namespace or "key:val" mechanism, no tag has relation to another tag. The new implementation of the tags feature from version 67.0 onwards, allows "key:val" tags, and because of that, a test can be classified with a tag in a given key. For instance, the new proposed version of the "boot_linux_console.py" test, which downloads and attempts to run a x86_64 kernel, is now tagged as: :avocado: tags=arch:x86_64 This means that it can be filtered (out) when no x86_64 target is available. At the same time, tests that don't have a "arch:" tag, will not be filtered out. Signed-off-by: Cleber Rosa <crosa@redhat.com> --- tests/Makefile.include | 3 +++ tests/acceptance/boot_linux_console.py | 2 +- tests/requirements.txt | 2 +- 3 files changed, 5 insertions(+), 2 deletions(-)