| Message ID | 20230303160727.3977246-7-berrange@redhat.com |
|---|---|
| State | New |
| Headers | show |
| Series | iotests: make meson aware of individual I/O tests | expand |
On 03/03/2023 17.07, Daniel P. Berrangé wrote: > The current test runner is only safe against parallel execution within > a single instance of the 'check' process, and only if -j is given a > value greater than 2. This prevents running multiple copies of the > 'check' process for different test scenarios. > > This change switches the output / socket directories to always include > the test name, image format and image protocol. This should allow full > parallelism of all distinct test scenarios. eg running both qcow2 and > raw tests at the same time, or both file and nbd tests at the same > time. > > It would be possible to allow for parallelism of the same test scenario > by including the pid, but that would potentially let many directories > accumulate over time on failures, so is not done. > > Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> > --- > tests/qemu-iotests/testrunner.py | 20 ++++++++++---------- > 1 file changed, 10 insertions(+), 10 deletions(-) Reviewed-by: Thomas Huth <thuth@redhat.com>
On Fri, Mar 03, 2023 at 04:07:25PM +0000, Daniel P. Berrangé wrote: > The current test runner is only safe against parallel execution within > a single instance of the 'check' process, and only if -j is given a > value greater than 2. This prevents running multiple copies of the > 'check' process for different test scenarios. > > This change switches the output / socket directories to always include > the test name, image format and image protocol. This should allow full > parallelism of all distinct test scenarios. eg running both qcow2 and > raw tests at the same time, or both file and nbd tests at the same > time. > > It would be possible to allow for parallelism of the same test scenario > by including the pid, but that would potentially let many directories > accumulate over time on failures, so is not done. > > Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> > --- > tests/qemu-iotests/testrunner.py | 20 ++++++++++---------- > 1 file changed, 10 insertions(+), 10 deletions(-) git bisect points to this commit as being the reason behind the following regression in iotests: $ ./check -nbd 104 ... --- /home/eblake/qemu/tests/qemu-iotests/104.out +++ /home/eblake/qemu/build/tests/qemu-iotests/scratch/raw-nbd-104/104.out.bad @@ -2,11 +2,11 @@ === Check qemu-img info output === Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1024 -image: TEST_DIR/t.IMGFMT +image: nbd+unix://?socket=/tmp/tmpqathrkpn/IMGFMT-nbd-104/nbd file format: IMGFMT virtual size: 1 KiB (1024 bytes) Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1234 -image: TEST_DIR/t.IMGFMT +image: nbd+unix://?socket=/tmp/tmpqathrkpn/IMGFMT-nbd-104/nbd file format: IMGFMT virtual size: 1.5 KiB (1536 bytes) *** done Failures: 104 Back in 2015, commit a231cb2726, we added a hack that turned nbd://... into TEST_DIR/t.IMGFMT to satisfy the output matching in the various iotests, but with our new per-test directories, that filter is no longer firing.
diff --git a/tests/qemu-iotests/testrunner.py b/tests/qemu-iotests/testrunner.py index 81519ed6e2..7b322272e9 100644 --- a/tests/qemu-iotests/testrunner.py +++ b/tests/qemu-iotests/testrunner.py @@ -228,13 +228,11 @@ def find_reference(self, test: str) -> str: return f'{test}.out' - def do_run_test(self, test: str, mp: bool) -> TestResult: + def do_run_test(self, test: str) -> TestResult: """ Run one test :param test: test file path - :param mp: if true, we are in a multiprocessing environment, use - personal subdirectories for test run Note: this method may be called from subprocess, so it does not change ``self`` object in any way! @@ -257,12 +255,14 @@ def do_run_test(self, test: str, mp: bool) -> TestResult: args = [str(f_test.resolve())] env = self.env.prepare_subprocess(args) - if mp: - # Split test directories, so that tests running in parallel don't - # break each other. - for d in ['TEST_DIR', 'SOCK_DIR']: - env[d] = os.path.join(env[d], f_test.name) - Path(env[d]).mkdir(parents=True, exist_ok=True) + + # Split test directories, so that tests running in parallel don't + # break each other. + for d in ['TEST_DIR', 'SOCK_DIR']: + env[d] = os.path.join( + env[d], + f"{self.env.imgfmt}-{self.env.imgproto}-{f_test.name}") + Path(env[d]).mkdir(parents=True, exist_ok=True) test_dir = env['TEST_DIR'] f_bad = Path(test_dir, f_test.name + '.out.bad') @@ -347,7 +347,7 @@ def run_test(self, test: str, testname = os.path.basename(test) print(f'# running {self.env.imgfmt} {testname}') - res = self.do_run_test(test, mp) + res = self.do_run_test(test) end = datetime.datetime.now().strftime('%H:%M:%S') self.test_print_one_line(test=test,
The current test runner is only safe against parallel execution within a single instance of the 'check' process, and only if -j is given a value greater than 2. This prevents running multiple copies of the 'check' process for different test scenarios. This change switches the output / socket directories to always include the test name, image format and image protocol. This should allow full parallelism of all distinct test scenarios. eg running both qcow2 and raw tests at the same time, or both file and nbd tests at the same time. It would be possible to allow for parallelism of the same test scenario by including the pid, but that would potentially let many directories accumulate over time on failures, so is not done. Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> --- tests/qemu-iotests/testrunner.py | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-)