mbox series

[v4,0/9] KVM: selftests: Create KVM selftests runner

Message ID 20260331194202.1722082-1-vipinsh@google.com
Headers show
Series KVM: selftests: Create KVM selftests runner | expand

Message

Vipin Sharma March 31, 2026, 7:41 p.m. UTC 
Hello,

This is v4 of KVM selftest runner.

To recap, KVM Selftest Runner allows running KVM selftests with added
features not present in default kselftest infrastructure like
parallelism, user friendly output, saving output hierarchically, etc.
Check README file in the patch 9.

This series is also available on github at:

https://github.com/shvipin/linux kvm/selftests/runner-v4

v4:
- Copy runner code and default test case files to out-of-tree build
  directory.
- Added comparison with kernel selftest framework.
- Added python minimum version (3.6) check for the runner.
- Bug fix where runner was not handling the child process output if returned in
  bytes instead of string.

v3: https://lore.kernel.org/kvm/20250930163635.4035866-1-vipinsh@google.com/
- Created "tests_install" rule in Makefile.kvm to auto generate default
  testcases, which will be ignored in .gitignore.
- Changed command line option names to pass testcase files, directories,
  executable paths, print based on test status, and what to print.
  Removed certain other options based on feedback in v2.
- Merged command.py into selftest.py
- Fixed issue where timed out test's stdout and stderr were not printed.
- Reduced python version from 3.7 to 3.6.
- Fixed issue where test status numerical value was printed instead of
  text like PASSED, FAILED, SKIPPED, etc.
- Added README.rst.

v2: https://lore.kernel.org/kvm/20250606235619.1841595-1-vipinsh@google.com/
- Automatic default test generation.
- Command line flag to provide executables location
- Dump output to filesystem with timestamp
- Accept absolute path of *.test files/directory location
- Sticky status at bottom for the current state of runner.
- Knobs to control output verbosity
- Colored output for terminals.

v1: https://lore.kernel.org/kvm/20250222005943.3348627-1-vipinsh@google.com/
- Parallel test execution.
- Dumping separate output for each test.
- Timeout for test execution
- Specify single test or a test directory.

RFC: https://lore.kernel.org/kvm/20240821223012.3757828-1-vipinsh@google.com/


Vipin Sharma (9):
  KVM: selftest: Create KVM selftest runner
  KVM: selftests: Provide executables path option to the KVM selftest
    runner
  KVM: selftests: Add timeout option in selftests runner
  KVM: selftests: Add option to save selftest runner output to a
    directory
  KVM: selftests: Run tests concurrently in KVM selftests runner
  KVM: selftests: Add various print flags to KVM selftest runner
  KVM: selftests: Print sticky KVM selftests runner status at bottom
  KVM: selftests: Add rule to generate default tests for KVM selftests
    runner
  KVM: selftests: Provide README.rst for KVM selftests runner

 tools/testing/selftests/kvm/.gitignore        |   6 +-
 tools/testing/selftests/kvm/Makefile.kvm      |  26 ++-
 tools/testing/selftests/kvm/runner/README.rst |  95 +++++++++
 .../testing/selftests/kvm/runner/__main__.py  | 189 ++++++++++++++++++
 .../testing/selftests/kvm/runner/selftest.py  |  99 +++++++++
 .../selftests/kvm/runner/test_runner.py       |  83 ++++++++
 .../2slot_5vcpu_10iter.test                   |   1 +
 .../no_dirty_log_protect.test                 |   1 +
 8 files changed, 498 insertions(+), 2 deletions(-)
 create mode 100644 tools/testing/selftests/kvm/runner/README.rst
 create mode 100644 tools/testing/selftests/kvm/runner/__main__.py
 create mode 100644 tools/testing/selftests/kvm/runner/selftest.py
 create mode 100644 tools/testing/selftests/kvm/runner/test_runner.py
 create mode 100644 tools/testing/selftests/kvm/tests/dirty_log_perf_test/2slot_5vcpu_10iter.test
 create mode 100644 tools/testing/selftests/kvm/tests/dirty_log_perf_test/no_dirty_log_protect.test


base-commit: df83746075778958954aa0460cca55f4b3fc9c02