| Message ID | 20180530184156.15634-1-crosa@redhat.com |
|---|---|
| Headers | show |
| Series | Acceptance/functional tests | expand |
Hi Eduardo, On 05/30/2018 03:41 PM, Cleber Rosa wrote: > TL;DR > ===== > > Another version, with a minimalist approach, of the acceptance tests > infrastructure for QEMU, based on the Avocado Testing Framework. What do you think of this series? Are you OK to take it via your tree, or should it goes via Fam's "Build and test automation"? Thanks, Phil.
On Tue, Jun 05, 2018 at 11:49:07AM -0300, Philippe Mathieu-Daudé wrote: > Hi Eduardo, > > On 05/30/2018 03:41 PM, Cleber Rosa wrote: > > TL;DR > > ===== > > > > Another version, with a minimalist approach, of the acceptance tests > > infrastructure for QEMU, based on the Avocado Testing Framework. > > What do you think of this series? > > Are you OK to take it via your tree, or should it goes via Fam's "Build > and test automation"? I can take it via my tree, I just couldn't reserve the time to check the existing review comments and finish reviewing it. But if Fam or other maintainer already thinks the series look good and can be merged, please don't delay this because of me. Acked-by: Eduardo Habkost <ehabkost@redhat.com>
On Wed, May 30, 2018 at 02:41:51PM -0400, Cleber Rosa wrote: > TL;DR > ===== > > Another version, with a minimalist approach, of the acceptance tests > infrastructure for QEMU, based on the Avocado Testing Framework. Queued on python-next. Thank you very much, and sorry for taking so long to queue this.
TL;DR ===== Another version, with a minimalist approach, of the acceptance tests infrastructure for QEMU, based on the Avocado Testing Framework. Background ========== The previous version, still considered an RFC, sent to the list by Eduardo[1] was based on the work held in Amador's branch[2]. After reviewing it under a different light, including the experiences done and reported by Philippe[3], it was clear to me that a different approach would be better. Differences from previous versions ================================== The main difference is that this series include only the minimal changes deemed necessary to have a starting point. I like to think that it's better connected to the QEMU community and project needs, and will hopefully allow for a more organic growth. Since this version has less features than the previous versions, provided it's accepted, these are the next probable development tasks: * Provide a simple variants mechanism to allow the same tests to be run under different targets, machine models and devices (present on the previous versions as a "YAML to Mux" file with architecture definitions) * Implement QEMUMachine migration support (present on the previous version in the "avocado_qemu.test._VM" class) * Implement Guest OS image selection and download (mostly an Avocado feature, paired with a parameter convention and cloud-init support code) * Implement interactive support for Guest OS sessions (present on the previous versions, supported by the aexpect Python module) * Move the Python modules that are used on tests to a more proper "module" organization. This should allow for a proper place to hold tests for the Python modules themselves. Even though this version shares very little (if any) code with the previous versions, the following is a list of noteworthy changes: * Tests directory is now "tests/acceptance" (was "tests/avocado") * Base test class is now "avocado_qemu.Test" (was "avocado_qemu.test.QemuTest") * Base test class is now hosted in "avocado_qemu/__init__.py" (was "avocado_qemu/test.py") * Direct use of "qemu.QEMUMachine", that is, the avocado_qemu.test._VM class is gone * avocado_qemu.Test won't search for QEMU binaries on $PATH. To use QEMU binary on a custom system location it's necessary to use the "qemu_bin" parameter * Example test in README.rst is distributed as a real test ("test_version.py") * A new "Linux boot console" test, loosely modeled after Phillipe's use case Changes ======= From v3: -------- * Gave more information on the meaning/intended use of acceptance tests * Added a clearer example on running tests with tags * Added more information on what a QEMUMachine is From v2 ------- * Renamed acceptance test files, dropping "test_" prefix. * Dropped "scripts/test_qemu.py" until we have a proper Python module location on which to put their own tests. From v1 ------- * Moved documentation to docs/devel/testing.rst * Changed code markers from "::" to ".. code::" in the documentation for consistency * Added copyright notices * Removed useless "qemu_bin=None" attribution on avocado_qemu.Test.setUp() * Removed reference about setting arguments, instead of appending them, on "Acceptance tests: add quick VNC tests" commit message * Fixed typo (s/inteligence/intelligence/) on "scripts/qemu.py: introduce set_console() method" commit message * Removed useless "return" in add_args() Commit summary ============== Cleber Rosa (5): Add functional/acceptance tests infrastructure scripts/qemu.py: allow adding to the list of extra arguments Acceptance tests: add quick VNC tests scripts/qemu.py: introduce set_console() method Acceptance tests: add Linux kernel boot and console checking test docs/devel/testing.rst | 192 +++++++++++++++++++++++++++++ scripts/qemu.py | 103 +++++++++++++++- tests/acceptance/README.rst | 10 ++ tests/acceptance/avocado_qemu/__init__.py | 54 ++++++++ tests/acceptance/boot_linux_console.py | 47 +++++++ tests/acceptance/version.py | 24 ++++ tests/acceptance/vnc.py | 60 +++++++++ 7 files changed, 489 insertions(+), 1 deletion(-) create mode 100644 tests/acceptance/README.rst create mode 100644 tests/acceptance/avocado_qemu/__init__.py create mode 100644 tests/acceptance/boot_linux_console.py create mode 100644 tests/acceptance/version.py create mode 100644 tests/acceptance/vnc.py --- [1] https://lists.gnu.org/archive/html/qemu-devel/2018-04/msg03443.html [2] https://github.com/apahim/qemu/commits/avocado_qemu [3] https://lists.gnu.org/archive/html/qemu-devel/2018-04/msg03076.html