iotests: Add more "skip_if_unsupported" statements to the python tests

The python code already contains a possibility to skip tests if the
corresponding driver is not available in the qemu binary - use it
in more spots to avoid that the tests are failing if the driver has
been disabled.

Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 tests/qemu-iotests/030 |  3 +++
 tests/qemu-iotests/040 |  2 ++
 tests/qemu-iotests/041 | 14 +++++++++++++-
 tests/qemu-iotests/245 |  2 ++
 4 files changed, 20 insertions(+), 1 deletion(-)

On 19.08.19 11:21, Thomas Huth wrote:
> The python code already contains a possibility to skip tests if the
> corresponding driver is not available in the qemu binary - use it
> in more spots to avoid that the tests are failing if the driver has
> been disabled.
> 
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
>  tests/qemu-iotests/030 |  3 +++
>  tests/qemu-iotests/040 |  2 ++
>  tests/qemu-iotests/041 | 14 +++++++++++++-
>  tests/qemu-iotests/245 |  2 ++
>  4 files changed, 20 insertions(+), 1 deletion(-)

[...]

> diff --git a/tests/qemu-iotests/041 b/tests/qemu-iotests/041
> index 26bf1701eb..f45d20fbe0 100755
> --- a/tests/qemu-iotests/041
> +++ b/tests/qemu-iotests/041
> @@ -817,6 +817,7 @@ class TestRepairQuorum(iotests.QMPTestCase):
>      image_len = 1 * 1024 * 1024 # MB
>      IMAGES = [ quorum_img1, quorum_img2, quorum_img3 ]
>  
> +    @iotests.skip_if_unsupported(['quorum'])
>      def setUp(self):
>          self.vm = iotests.VM()

It’s clear that none of these tests can run if there is no quorum
support, because setUp() creates a quorum node.  I think it would be
nice if it would suffice to just skip everything automatically if
setUp() is skipped and not have to bother about each of the test cases.

Coincidentally (:-)), I have a patch to do that, namely “iotests: Allow
skipping test cases” in my “iotests: Selfish patches” series:

https://lists.nongnu.org/archive/html/qemu-block/2019-06/msg01106.html

Yes, that means you cannot use an annotation because it needs @self to
be able to skip the test.  Hm... But I think I can make that work by
simply s/case_notrun/args[0].case_skip/ in skip_if_unsupported()?

Max

On 19.08.19 21:13, Max Reitz wrote:
> On 19.08.19 11:21, Thomas Huth wrote:
>> The python code already contains a possibility to skip tests if the
>> corresponding driver is not available in the qemu binary - use it
>> in more spots to avoid that the tests are failing if the driver has
>> been disabled.
>>
>> Signed-off-by: Thomas Huth <thuth@redhat.com>
>> ---
>>  tests/qemu-iotests/030 |  3 +++
>>  tests/qemu-iotests/040 |  2 ++
>>  tests/qemu-iotests/041 | 14 +++++++++++++-
>>  tests/qemu-iotests/245 |  2 ++
>>  4 files changed, 20 insertions(+), 1 deletion(-)
> 
> [...]
> 
>> diff --git a/tests/qemu-iotests/041 b/tests/qemu-iotests/041
>> index 26bf1701eb..f45d20fbe0 100755
>> --- a/tests/qemu-iotests/041
>> +++ b/tests/qemu-iotests/041
>> @@ -817,6 +817,7 @@ class TestRepairQuorum(iotests.QMPTestCase):
>>      image_len = 1 * 1024 * 1024 # MB
>>      IMAGES = [ quorum_img1, quorum_img2, quorum_img3 ]
>>  
>> +    @iotests.skip_if_unsupported(['quorum'])
>>      def setUp(self):
>>          self.vm = iotests.VM()
> 
> It’s clear that none of these tests can run if there is no quorum
> support, because setUp() creates a quorum node.  I think it would be
> nice if it would suffice to just skip everything automatically if
> setUp() is skipped and not have to bother about each of the test cases.
> 
> Coincidentally (:-)), I have a patch to do that, namely “iotests: Allow
> skipping test cases” in my “iotests: Selfish patches” series:
> 
> https://lists.nongnu.org/archive/html/qemu-block/2019-06/msg01106.html
> 
> Yes, that means you cannot use an annotation because it needs @self to
> be able to skip the test.  Hm... But I think I can make that work by
> simply s/case_notrun/args[0].case_skip/ in skip_if_unsupported()?

(Sent a v3 now that does that.)

(Max)

On 8/19/19 9:13 PM, Max Reitz wrote:
> On 19.08.19 11:21, Thomas Huth wrote:
>> The python code already contains a possibility to skip tests if the
>> corresponding driver is not available in the qemu binary - use it
>> in more spots to avoid that the tests are failing if the driver has
>> been disabled.
>>
>> Signed-off-by: Thomas Huth <thuth@redhat.com>
>> ---
>>  tests/qemu-iotests/030 |  3 +++
>>  tests/qemu-iotests/040 |  2 ++
>>  tests/qemu-iotests/041 | 14 +++++++++++++-
>>  tests/qemu-iotests/245 |  2 ++
>>  4 files changed, 20 insertions(+), 1 deletion(-)
> 
> [...]
> 
>> diff --git a/tests/qemu-iotests/041 b/tests/qemu-iotests/041
>> index 26bf1701eb..f45d20fbe0 100755
>> --- a/tests/qemu-iotests/041
>> +++ b/tests/qemu-iotests/041
>> @@ -817,6 +817,7 @@ class TestRepairQuorum(iotests.QMPTestCase):
>>      image_len = 1 * 1024 * 1024 # MB
>>      IMAGES = [ quorum_img1, quorum_img2, quorum_img3 ]
>>  
>> +    @iotests.skip_if_unsupported(['quorum'])
>>      def setUp(self):
>>          self.vm = iotests.VM()
> 
> It’s clear that none of these tests can run if there is no quorum
> support, because setUp() creates a quorum node.  I think it would be
> nice if it would suffice to just skip everything automatically if
> setUp() is skipped and not have to bother about each of the test cases.
> 
> Coincidentally (:-)), I have a patch to do that, namely “iotests: Allow
> skipping test cases” in my “iotests: Selfish patches” series:
> 
> https://lists.nongnu.org/archive/html/qemu-block/2019-06/msg01106.html
> 
> Yes, that means you cannot use an annotation because it needs @self to
> be able to skip the test.  Hm... But I think I can make that work by
> simply s/case_notrun/args[0].case_skip/ in skip_if_unsupported()?

Sure, feel free to ignore my patch here or to modify it according to
your reworks. As long as we finally get the iotests into a shape where
they are a little bit more flexible wrt the enabled/disabled drivers,
I'm happy.

 Thomas

On 20.08.19 09:14, Thomas Huth wrote:
> On 8/19/19 9:13 PM, Max Reitz wrote:
>> On 19.08.19 11:21, Thomas Huth wrote:
>>> The python code already contains a possibility to skip tests if the
>>> corresponding driver is not available in the qemu binary - use it
>>> in more spots to avoid that the tests are failing if the driver has
>>> been disabled.
>>>
>>> Signed-off-by: Thomas Huth <thuth@redhat.com>
>>> ---
>>>  tests/qemu-iotests/030 |  3 +++
>>>  tests/qemu-iotests/040 |  2 ++
>>>  tests/qemu-iotests/041 | 14 +++++++++++++-
>>>  tests/qemu-iotests/245 |  2 ++
>>>  4 files changed, 20 insertions(+), 1 deletion(-)
>>
>> [...]
>>
>>> diff --git a/tests/qemu-iotests/041 b/tests/qemu-iotests/041
>>> index 26bf1701eb..f45d20fbe0 100755
>>> --- a/tests/qemu-iotests/041
>>> +++ b/tests/qemu-iotests/041
>>> @@ -817,6 +817,7 @@ class TestRepairQuorum(iotests.QMPTestCase):
>>>      image_len = 1 * 1024 * 1024 # MB
>>>      IMAGES = [ quorum_img1, quorum_img2, quorum_img3 ]
>>>  
>>> +    @iotests.skip_if_unsupported(['quorum'])
>>>      def setUp(self):
>>>          self.vm = iotests.VM()
>>
>> It’s clear that none of these tests can run if there is no quorum
>> support, because setUp() creates a quorum node.  I think it would be
>> nice if it would suffice to just skip everything automatically if
>> setUp() is skipped and not have to bother about each of the test cases.
>>
>> Coincidentally (:-)), I have a patch to do that, namely “iotests: Allow
>> skipping test cases” in my “iotests: Selfish patches” series:
>>
>> https://lists.nongnu.org/archive/html/qemu-block/2019-06/msg01106.html
>>
>> Yes, that means you cannot use an annotation because it needs @self to
>> be able to skip the test.  Hm... But I think I can make that work by
>> simply s/case_notrun/args[0].case_skip/ in skip_if_unsupported()?
> 
> Sure, feel free to ignore my patch here or to modify it according to
> your reworks. As long as we finally get the iotests into a shape where
> they are a little bit more flexible wrt the enabled/disabled drivers,
> I'm happy.

Oh, no, I don’t mean to ignore the patch.  I just think that a
skip_if_unsupported() annotation above setUp() should be enough to skip
all tests.

Max

On 19/08/2019 12:21, Thomas Huth wrote:
> The python code already contains a possibility to skip tests if the
> corresponding driver is not available in the qemu binary - use it
> in more spots to avoid that the tests are failing if the driver has
> been disabled.
> 
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
>   tests/qemu-iotests/030 |  3 +++
>   tests/qemu-iotests/040 |  2 ++
>   tests/qemu-iotests/041 | 14 +++++++++++++-
>   tests/qemu-iotests/245 |  2 ++
>   4 files changed, 20 insertions(+), 1 deletion(-)
> 
> diff --git a/tests/qemu-iotests/030 b/tests/qemu-iotests/030
> index 1b69f318c6..18ad24ccdf 100755
> --- a/tests/qemu-iotests/030
> +++ b/tests/qemu-iotests/030
> @@ -530,6 +530,7 @@ class TestQuorum(iotests.QMPTestCase):
>       children = []
>       backing = []
>   
> +    @iotests.skip_if_unsupported(['quorum'])
>       def setUp(self):

Based on the Max's series "iotests: Selfish patches",
this '+' suffice for the whole methods of the class.
So, I can confirm after testing it.

>           opts = ['driver=quorum', 'vote-threshold=2']
>   
> @@ -552,6 +553,7 @@ class TestQuorum(iotests.QMPTestCase):
>           self.vm.add_drive(path = None, opts = ','.join(opts))
>           self.vm.launch()
>   
> +    @iotests.skip_if_unsupported(['quorum'])
>       def tearDown(self):
>           self.vm.shutdown()
>           for img in self.children:
> @@ -559,6 +561,7 @@ class TestQuorum(iotests.QMPTestCase):
>           for img in self.backing:
>               os.remove(img)
>   
> +    @iotests.skip_if_unsupported(['quorum'])
>       def test_stream_quorum(self):
>           if not iotests.supports_quorum():
>               return
> diff --git a/tests/qemu-iotests/040 b/tests/qemu-iotests/040
> index aa0b1847e3..0ec4fb71a9 100755
> --- a/tests/qemu-iotests/040
> +++ b/tests/qemu-iotests/040
> @@ -110,6 +110,7 @@ class TestSingleDrive(ImageCommitTestCase):
>           self.assertEqual(-1, qemu_io('-f', 'raw', '-c', 'read -P 0xab 0 524288', backing_img).find("verification failed"))
>           self.assertEqual(-1, qemu_io('-f', 'raw', '-c', 'read -P 0xef 524288 524288', backing_img).find("verification failed"))
>   
> +    @iotests.skip_if_unsupported(['throttle'])
>       def test_commit_with_filter_and_quit(self):
>           result = self.vm.qmp('object-add', qom_type='throttle-group', id='tg')
>           self.assert_qmp(result, 'return', {})
> @@ -129,6 +130,7 @@ class TestSingleDrive(ImageCommitTestCase):
>           self.has_quit = True
>   
>       # Same as above, but this time we add the filter after starting the job
> +    @iotests.skip_if_unsupported(['throttle'])
>       def test_commit_plus_filter_and_quit(self):
>           result = self.vm.qmp('object-add', qom_type='throttle-group', id='tg')
>           self.assert_qmp(result, 'return', {})
> diff --git a/tests/qemu-iotests/041 b/tests/qemu-iotests/041
> index 26bf1701eb..f45d20fbe0 100755
> --- a/tests/qemu-iotests/041
> +++ b/tests/qemu-iotests/041
> @@ -817,6 +817,7 @@ class TestRepairQuorum(iotests.QMPTestCase):
>       image_len = 1 * 1024 * 1024 # MB
>       IMAGES = [ quorum_img1, quorum_img2, quorum_img3 ]
>   
> +    @iotests.skip_if_unsupported(['quorum'])
>       def setUp(self):

Also here, as Max wrote. So, this series can be simplified and go after 
the "iotests: Selfish patches".

Andrey

>           self.vm = iotests.VM()
>   
> @@ -841,7 +842,7 @@ class TestRepairQuorum(iotests.QMPTestCase):
>               result = self.vm.qmp("blockdev-add", **args)
>               self.assert_qmp(result, 'return', {})
>   
> -
> +    @iotests.skip_if_unsupported(['quorum'])
>       def tearDown(self):
>           self.vm.shutdown()
>           for i in self.IMAGES + [ quorum_repair_img, quorum_snapshot_file ]:
> @@ -851,6 +852,7 @@ class TestRepairQuorum(iotests.QMPTestCase):
>               except OSError:
>                   pass
>   
> +    @iotests.skip_if_unsupported(['quorum'])
>       def test_complete(self):
>           if not iotests.supports_quorum():
>               return
> @@ -870,6 +872,7 @@ class TestRepairQuorum(iotests.QMPTestCase):
>           self.assertTrue(iotests.compare_images(quorum_img2, quorum_repair_img),
>                           'target image does not match source after mirroring')
>   
> +    @iotests.skip_if_unsupported(['quorum'])
>       def test_cancel(self):
>           if not iotests.supports_quorum():
>               return
> @@ -887,6 +890,7 @@ class TestRepairQuorum(iotests.QMPTestCase):
>           self.assert_has_block_node(None, quorum_img3)
>           self.vm.shutdown()
>   
> +    @iotests.skip_if_unsupported(['quorum'])
>       def test_cancel_after_ready(self):
>           if not iotests.supports_quorum():
>               return
> @@ -906,6 +910,7 @@ class TestRepairQuorum(iotests.QMPTestCase):
>           self.assertTrue(iotests.compare_images(quorum_img2, quorum_repair_img),
>                           'target image does not match source after mirroring')
>   
> +    @iotests.skip_if_unsupported(['quorum'])
>       def test_pause(self):
>           if not iotests.supports_quorum():
>               return
> @@ -934,6 +939,7 @@ class TestRepairQuorum(iotests.QMPTestCase):
>           self.assertTrue(iotests.compare_images(quorum_img2, quorum_repair_img),
>                           'target image does not match source after mirroring')
>   
> +    @iotests.skip_if_unsupported(['quorum'])
>       def test_medium_not_found(self):
>           if not iotests.supports_quorum():
>               return
> @@ -948,6 +954,7 @@ class TestRepairQuorum(iotests.QMPTestCase):
>                                target=quorum_repair_img, format=iotests.imgfmt)
>           self.assert_qmp(result, 'error/class', 'GenericError')
>   
> +    @iotests.skip_if_unsupported(['quorum'])
>       def test_image_not_found(self):
>           if not iotests.supports_quorum():
>               return
> @@ -958,6 +965,7 @@ class TestRepairQuorum(iotests.QMPTestCase):
>                                format=iotests.imgfmt)
>           self.assert_qmp(result, 'error/class', 'GenericError')
>   
> +    @iotests.skip_if_unsupported(['quorum'])
>       def test_device_not_found(self):
>           if not iotests.supports_quorum():
>               return
> @@ -969,6 +977,7 @@ class TestRepairQuorum(iotests.QMPTestCase):
>                                target=quorum_repair_img, format=iotests.imgfmt)
>           self.assert_qmp(result, 'error/class', 'GenericError')
>   
> +    @iotests.skip_if_unsupported(['quorum'])
>       def test_wrong_sync_mode(self):
>           if not iotests.supports_quorum():
>               return
> @@ -979,6 +988,7 @@ class TestRepairQuorum(iotests.QMPTestCase):
>                                target=quorum_repair_img, format=iotests.imgfmt)
>           self.assert_qmp(result, 'error/class', 'GenericError')
>   
> +    @iotests.skip_if_unsupported(['quorum'])
>       def test_no_node_name(self):
>           if not iotests.supports_quorum():
>               return
> @@ -988,6 +998,7 @@ class TestRepairQuorum(iotests.QMPTestCase):
>                                target=quorum_repair_img, format=iotests.imgfmt)
>           self.assert_qmp(result, 'error/class', 'GenericError')
>   
> +    @iotests.skip_if_unsupported(['quorum'])
>       def test_nonexistent_replaces(self):
>           if not iotests.supports_quorum():
>               return
> @@ -997,6 +1008,7 @@ class TestRepairQuorum(iotests.QMPTestCase):
>                                target=quorum_repair_img, format=iotests.imgfmt)
>           self.assert_qmp(result, 'error/class', 'GenericError')
>   
> +    @iotests.skip_if_unsupported(['quorum'])
>       def test_after_a_quorum_snapshot(self):
>           if not iotests.supports_quorum():
>               return
> diff --git a/tests/qemu-iotests/245 b/tests/qemu-iotests/245
> index bc1ceb9792..1bdc659ecd 100644
> --- a/tests/qemu-iotests/245
> +++ b/tests/qemu-iotests/245
> @@ -478,6 +478,7 @@ class TestBlockdevReopen(iotests.QMPTestCase):
>       # This test verifies that we can't change the children of a block
>       # device during a reopen operation in a way that would create
>       # cycles in the node graph
> +    @iotests.skip_if_unsupported(['blkverify'])
>       def test_graph_cycles(self):
>           opts = []
>   
> @@ -534,6 +535,7 @@ class TestBlockdevReopen(iotests.QMPTestCase):
>           self.assert_qmp(result, 'return', {})
>   
>       # Misc reopen tests with different block drivers
> +    @iotests.skip_if_unsupported(['quorum'])
>       def test_misc_drivers(self):
>           ####################
>           ###### quorum ######
>