diff mbox

[2/2] make check: Add qemu-iotests subset

Message ID 1331206372-10405-3-git-send-email-kwolf@redhat.com
State New
Headers show

Commit Message

Kevin Wolf March 8, 2012, 11:32 a.m. UTC 
Run the 'quick' group from qemu-iotests during 'make check'.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
---
 tests/Makefile        |    1 +
 tests/qemu-iotests.sh |   18 ++++++++++++++++++
 2 files changed, 19 insertions(+), 0 deletions(-)
 create mode 100755 tests/qemu-iotests.sh

Comments

Stefan Hajnoczi March 8, 2012, 3:51 p.m. UTC | #1 
On Thu, Mar 8, 2012 at 11:32 AM, Kevin Wolf <kwolf@redhat.com> wrote:
> Run the 'quick' group from qemu-iotests during 'make check'.
>
> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
> ---
>  tests/Makefile        |    1 +
>  tests/qemu-iotests.sh |   18 ++++++++++++++++++

I think tests/qemu-iotests-quick.sh would be a clearer name since this
is not a general-purpose qemu-iotests wrapper - it only works when
called from QEMU's root directory and only invokes the 'quick' group.

> +./check -T -nocache -raw -g quick || ret=1
> +./check -T -nocache -qcow2 -g quick || ret=1

Some love for qed?  It adds 10s on my box but we're already up at 26s.

Stefan
Kevin Wolf March 8, 2012, 4:10 p.m. UTC | #2 
Am 08.03.2012 16:51, schrieb Stefan Hajnoczi:
> On Thu, Mar 8, 2012 at 11:32 AM, Kevin Wolf <kwolf@redhat.com> wrote:
>> Run the 'quick' group from qemu-iotests during 'make check'.
>>
>> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
>> ---
>>  tests/Makefile        |    1 +
>>  tests/qemu-iotests.sh |   18 ++++++++++++++++++
> 
> I think tests/qemu-iotests-quick.sh would be a clearer name since this
> is not a general-purpose qemu-iotests wrapper - it only works when
> called from QEMU's root directory and only invokes the 'quick' group.

Ok, I'll rename it.

>> +./check -T -nocache -raw -g quick || ret=1
>> +./check -T -nocache -qcow2 -g quick || ret=1
> 
> Some love for qed?  It adds 10s on my box but we're already up at 26s.

I'm used to high expectations, but loving QED is a bit too much... ;-)

Not sure what to do about all the formats. Ideally we would test all of
them (at least those with better implementations, VMDK, VHD, VDI), but
that would definitely take too long. I hope that in the not too distant
future, QED will have a similar position to qcow1, but we may consider
adding it for now.

However, 'make check' is really the quick test that you run when you
don't change anything in the image formats. If you do, you should do a
full qemu-iotests run. So the important question is whether QED is
likely to reveal any breakage outside block/* that the qcow2 tests
wouldn't find.

With the same reason we can probably drop the raw tests: qcow2 is the
most featureful format, so any breakage should be visible there. Most of
raw-posix.c should be part of the qcow2 tests already.

Kevin
Stefan Hajnoczi March 8, 2012, 4:52 p.m. UTC | #3 
On Thu, Mar 8, 2012 at 4:10 PM, Kevin Wolf <kwolf@redhat.com> wrote:
> Am 08.03.2012 16:51, schrieb Stefan Hajnoczi:
>> On Thu, Mar 8, 2012 at 11:32 AM, Kevin Wolf <kwolf@redhat.com> wrote:
>>> Run the 'quick' group from qemu-iotests during 'make check'.
>>>
>>> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
>>> ---
>>>  tests/Makefile        |    1 +
>>>  tests/qemu-iotests.sh |   18 ++++++++++++++++++
>>
>> I think tests/qemu-iotests-quick.sh would be a clearer name since this
>> is not a general-purpose qemu-iotests wrapper - it only works when
>> called from QEMU's root directory and only invokes the 'quick' group.
>
> Ok, I'll rename it.
>
>>> +./check -T -nocache -raw -g quick || ret=1
>>> +./check -T -nocache -qcow2 -g quick || ret=1
>>
>> Some love for qed?  It adds 10s on my box but we're already up at 26s.
>
> I'm used to high expectations, but loving QED is a bit too much... ;-)
>
> Not sure what to do about all the formats. Ideally we would test all of
> them (at least those with better implementations, VMDK, VHD, VDI), but
> that would definitely take too long. I hope that in the not too distant
> future, QED will have a similar position to qcow1, but we may consider
> adding it for now.
>
> However, 'make check' is really the quick test that you run when you
> don't change anything in the image formats. If you do, you should do a
> full qemu-iotests run. So the important question is whether QED is
> likely to reveal any breakage outside block/* that the qcow2 tests
> wouldn't find.
>
> With the same reason we can probably drop the raw tests: qcow2 is the
> most featureful format, so any breakage should be visible there. Most of
> raw-posix.c should be part of the qcow2 tests already.

In another thread I think Anthony suggested check-block.  So if we
just test qcow2 here to make sure the block layer works, then we can
do a full run in check-block with raw, qed, and friends.

Stefan
Kevin Wolf March 8, 2012, 5:01 p.m. UTC | #4 
Am 08.03.2012 17:52, schrieb Stefan Hajnoczi:
> On Thu, Mar 8, 2012 at 4:10 PM, Kevin Wolf <kwolf@redhat.com> wrote:
>> Am 08.03.2012 16:51, schrieb Stefan Hajnoczi:
>>> On Thu, Mar 8, 2012 at 11:32 AM, Kevin Wolf <kwolf@redhat.com> wrote:
>>>> Run the 'quick' group from qemu-iotests during 'make check'.
>>>>
>>>> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
>>>> ---
>>>>  tests/Makefile        |    1 +
>>>>  tests/qemu-iotests.sh |   18 ++++++++++++++++++
>>>
>>> I think tests/qemu-iotests-quick.sh would be a clearer name since this
>>> is not a general-purpose qemu-iotests wrapper - it only works when
>>> called from QEMU's root directory and only invokes the 'quick' group.
>>
>> Ok, I'll rename it.
>>
>>>> +./check -T -nocache -raw -g quick || ret=1
>>>> +./check -T -nocache -qcow2 -g quick || ret=1
>>>
>>> Some love for qed?  It adds 10s on my box but we're already up at 26s.
>>
>> I'm used to high expectations, but loving QED is a bit too much... ;-)
>>
>> Not sure what to do about all the formats. Ideally we would test all of
>> them (at least those with better implementations, VMDK, VHD, VDI), but
>> that would definitely take too long. I hope that in the not too distant
>> future, QED will have a similar position to qcow1, but we may consider
>> adding it for now.
>>
>> However, 'make check' is really the quick test that you run when you
>> don't change anything in the image formats. If you do, you should do a
>> full qemu-iotests run. So the important question is whether QED is
>> likely to reveal any breakage outside block/* that the qcow2 tests
>> wouldn't find.
>>
>> With the same reason we can probably drop the raw tests: qcow2 is the
>> most featureful format, so any breakage should be visible there. Most of
>> raw-posix.c should be part of the qcow2 tests already.
> 
> In another thread I think Anthony suggested check-block.  So if we
> just test qcow2 here to make sure the block layer works, then we can
> do a full run in check-block with raw, qed, and friends.

Yes, I think that would make a lot of sense. Do you want me to include a
check-block in v2 (that would be similar to qemu-iotests-quick.sh, just
without -g quick and for more formats) or should we introduce something
more sophisticated later?

Kevin
Stefan Hajnoczi March 9, 2012, 10:10 a.m. UTC | #5 
On Thu, Mar 8, 2012 at 5:01 PM, Kevin Wolf <kwolf@redhat.com> wrote:
> Am 08.03.2012 17:52, schrieb Stefan Hajnoczi:
>> On Thu, Mar 8, 2012 at 4:10 PM, Kevin Wolf <kwolf@redhat.com> wrote:
>>> Am 08.03.2012 16:51, schrieb Stefan Hajnoczi:
>>>> On Thu, Mar 8, 2012 at 11:32 AM, Kevin Wolf <kwolf@redhat.com> wrote:
>>>>> Run the 'quick' group from qemu-iotests during 'make check'.
>>>>>
>>>>> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
>>>>> ---
>>>>>  tests/Makefile        |    1 +
>>>>>  tests/qemu-iotests.sh |   18 ++++++++++++++++++
>>>>
>>>> I think tests/qemu-iotests-quick.sh would be a clearer name since this
>>>> is not a general-purpose qemu-iotests wrapper - it only works when
>>>> called from QEMU's root directory and only invokes the 'quick' group.
>>>
>>> Ok, I'll rename it.
>>>
>>>>> +./check -T -nocache -raw -g quick || ret=1
>>>>> +./check -T -nocache -qcow2 -g quick || ret=1
>>>>
>>>> Some love for qed?  It adds 10s on my box but we're already up at 26s.
>>>
>>> I'm used to high expectations, but loving QED is a bit too much... ;-)
>>>
>>> Not sure what to do about all the formats. Ideally we would test all of
>>> them (at least those with better implementations, VMDK, VHD, VDI), but
>>> that would definitely take too long. I hope that in the not too distant
>>> future, QED will have a similar position to qcow1, but we may consider
>>> adding it for now.
>>>
>>> However, 'make check' is really the quick test that you run when you
>>> don't change anything in the image formats. If you do, you should do a
>>> full qemu-iotests run. So the important question is whether QED is
>>> likely to reveal any breakage outside block/* that the qcow2 tests
>>> wouldn't find.
>>>
>>> With the same reason we can probably drop the raw tests: qcow2 is the
>>> most featureful format, so any breakage should be visible there. Most of
>>> raw-posix.c should be part of the qcow2 tests already.
>>
>> In another thread I think Anthony suggested check-block.  So if we
>> just test qcow2 here to make sure the block layer works, then we can
>> do a full run in check-block with raw, qed, and friends.
>
> Yes, I think that would make a lot of sense. Do you want me to include a
> check-block in v2 (that would be similar to qemu-iotests-quick.sh, just
> without -g quick and for more formats) or should we introduce something
> more sophisticated later?

If you can add it right away that would be convenient - it will save
us from having to run qemu-iotests ./check manually.

Stefan
diff mbox

Patch

diff --git a/tests/Makefile b/tests/Makefile
index 74b29dc..155ecbd 100644
--- a/tests/Makefile
+++ b/tests/Makefile
@@ -1,6 +1,7 @@ 
 CHECKS = check-qdict check-qfloat check-qint check-qstring check-qlist
 CHECKS += check-qjson test-qmp-output-visitor test-qmp-input-visitor
 CHECKS += test-string-input-visitor test-string-output-visitor test-coroutine
+CHECKS += tests/qemu-iotests.sh
 
 check-qint.o check-qstring.o check-qdict.o check-qlist.o check-qfloat.o check-qjson.o test-coroutine.o: $(GENERATED_HEADERS)
 
diff --git a/tests/qemu-iotests.sh b/tests/qemu-iotests.sh
new file mode 100755
index 0000000..af99635
--- /dev/null
+++ b/tests/qemu-iotests.sh
@@ -0,0 +1,18 @@ 
+#!/bin/sh
+
+# We don't know which of the system emulator binaries there is (or if there is
+# any at all), so the 'quick' group doesn't contain any tests that require
+# running qemu proper. Assign a fake binary name so that qemu-iotests doesn't
+# complain about the missing binary.
+export QEMU_PROG="this_should_be_unused"
+
+export QEMU_IMG_PROG="$(pwd)/qemu-img"
+export QEMU_IO_PROG="$(pwd)/qemu-io"
+
+cd tests/qemu-iotests
+
+ret=0
+./check -T -nocache -raw -g quick || ret=1
+./check -T -nocache -qcow2 -g quick || ret=1
+
+exit $ret