diff mbox

[3/3] qemu-iotests: Some more write_zeroes tests

Message ID 1463503863-19009-4-git-send-email-kwolf@redhat.com
State New
Headers show

Commit Message

Kevin Wolf May 17, 2016, 4:51 p.m. UTC 
This covers some more write_zeroes cases which are relevant for the
recent qcow2 optimisations that check the allocation status of the
backing file for partial cluster write_zeroes requests.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
---
 tests/qemu-iotests/154     | 100 +++++++++++++++++++++++++++++++++++++++++++++
 tests/qemu-iotests/154.out |  64 +++++++++++++++++++++++++++++
 tests/qemu-iotests/group   |   1 +
 3 files changed, 165 insertions(+)
 create mode 100755 tests/qemu-iotests/154
 create mode 100644 tests/qemu-iotests/154.out

Comments

Eric Blake May 17, 2016, 8:35 p.m. UTC | #1 
On 05/17/2016 10:51 AM, Kevin Wolf wrote:
> This covers some more write_zeroes cases which are relevant for the
> recent qcow2 optimisations that check the allocation status of the
> backing file for partial cluster write_zeroes requests.
> 
> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
> ---

> +# qcow2 specific bdrv_write_zeroes tests with backing files (complements 034)

> +_supported_fmt qcow2

I guess test 034 covered cases of larger-than-sector write_zero, and
that the reason you needed a new test id is because this one is qcow2
specific (the general test covers multiple formats, but the behavior on
sub-cluster zeroing is specific to qcow2). Makes sense so far.

> +
> +CLUSTER_SIZE=4k
> +size=128M
> +
> +echo
> +echo == backing file contains zeros ==
> +
> +# Make sure that the whole cluster is allocated even for partial write_zeroes
> +# when the backing file contains zeros
> +
> +CLUSTER_SIZE=512 TEST_IMG="$TEST_IMG.base" _make_test_img $size
> +_make_test_img -b "$TEST_IMG.base"
> +
> +$QEMU_IO -c "write -z 0k 2k" "$TEST_IMG" | _filter_qemu_io

head aligned, tail is unaligned

> +$QEMU_IO -c "write -z 10k 2k" "$TEST_IMG" | _filter_qemu_io

head is unaligned, tail is unaligned

> +$QEMU_IO -c "write -z 17k 2k" "$TEST_IMG" | _filter_qemu_io

both unaligned, and completely contained within a single cluster

No test of crossing a cluster boundary at this point?  I would probably do:

$QEMU_IO -c "write -z 27k 2k" "$TEST_IMG" | _filter_qemu_io

then check that the map shows an allocation starting at 24k for 8192 bytes

> +$QEMU_IMG map --output=json "$TEST_IMG" | _filter_qemu_img_map
> +
> +echo
> +echo == backing file contains non-zero data before write_zeros ==
> +
> +CLUSTER_SIZE=512 TEST_IMG="$TEST_IMG.base" _make_test_img $size
> +_make_test_img -b "$TEST_IMG.base"
> +
> +$QEMU_IO -c "write -P 0x11 32k 1k" "$TEST_IMG.base" | _filter_qemu_io
> +$QEMU_IO -c "write -z 34k 1k" "$TEST_IMG" | _filter_qemu_io

cluster contains partial non-zero data, which does not overlap with the
write zero request. Good - you're testing that we don't nuke the rest of
the cluster (that is, we cannot convert an entire cluster to 0 unless
all sectors read as 0 before the conversion), but instead do proper
copy-on-write.

> +$QEMU_IO -c "read -P 0x11 32k 1k" "$TEST_IMG" | _filter_qemu_io
> +$QEMU_IO -c "read -P 0 33k 3k" "$TEST_IMG" | _filter_qemu_io
> +$QEMU_IMG map --output=json "$TEST_IMG" | _filter_qemu_img_map
> +
> +echo
> +echo == backing file contains non-zero data after write_zeros ==
> +
> +CLUSTER_SIZE=512 TEST_IMG="$TEST_IMG.base" _make_test_img $size
> +_make_test_img -b "$TEST_IMG.base"
> +
> +$QEMU_IO -c "write -P 0x11 43k 1k" "$TEST_IMG.base" | _filter_qemu_io
> +$QEMU_IO -c "write -z 41k 1k" "$TEST_IMG" | _filter_qemu_io
> +$QEMU_IO -c "read -P 0x11 43k 1k" "$TEST_IMG" | _filter_qemu_io
> +$QEMU_IO -c "read -P 0 40k 3k" "$TEST_IMG" | _filter_qemu_io
> +$QEMU_IMG map --output=json "$TEST_IMG" | _filter_qemu_img_map
> +
> +echo
> +echo == spanning multiple clusters ==
> +
> +CLUSTER_SIZE=512 TEST_IMG="$TEST_IMG.base" _make_test_img $size
> +_make_test_img -b "$TEST_IMG.base"
> +
> +$QEMU_IO -c "write -P 0x11 74k 2k" "$TEST_IMG.base" | _filter_qemu_io
> +$QEMU_IO -c "write -z 66k 7k" "$TEST_IMG" | _filter_qemu_io

the tail from 72k to 73k overlaps the sector that has to be
copy-on-write to pick up the data at 74k; while the earlier portion
picks up the clusters at 64k and 68k.  Looks good.

What you have looks fine, but again, I'd consider also testing a
sub-cluster size 'write -z' that spans a cluster boundary, including the
case where it is overwriting data from the backing file.  Something like:

$QEMU_IO -c "write -P 0x11 2k 4k" "$TEST_IMG.base" | _filter_qemu_io
$QEMU_IO -c "write -z 3k 2k" "$TEST_IMG" | _filter_qemu_io
$QEMU_IO -c "read -P 0 0k 2k" "$TEST_IMG" | _filter_qemu_io
$QEMU_IO -c "read -P 0x11 2k 1k" "$TEST_IMG" | _filter_qemu_io
$QEMU_IO -c "read -P 0 3k 2k" "$TEST_IMG" | _filter_qemu_io
$QEMU_IO -c "read -P 0x11 5k 1k" "$TEST_IMG" | _filter_qemu_io
$QEMU_IO -c "read -P 0 6k 2k" "$TEST_IMG" | _filter_qemu_io
Kevin Wolf May 18, 2016, 10:09 a.m. UTC | #2 
Am 17.05.2016 um 22:35 hat Eric Blake geschrieben:
> No test of crossing a cluster boundary at this point?  I would probably do:
> 
> $QEMU_IO -c "write -z 27k 2k" "$TEST_IMG" | _filter_qemu_io
> 
> then check that the map shows an allocation starting at 24k for 8192 bytes

> What you have looks fine, but again, I'd consider also testing a
> sub-cluster size 'write -z' that spans a cluster boundary, including the
> case where it is overwriting data from the backing file.

Okay, I'll put some more time into this test case. What we're testing
here is a source for corruption bugs, so it's probably worth doing it
right.

Kevin
diff mbox

Patch

diff --git a/tests/qemu-iotests/154 b/tests/qemu-iotests/154
new file mode 100755
index 0000000..e9b5a8f
--- /dev/null
+++ b/tests/qemu-iotests/154
@@ -0,0 +1,100 @@ 
+#!/bin/bash
+#
+# qcow2 specific bdrv_write_zeroes tests with backing files (complements 034)
+#
+# Copyright (C) 2016 Red Hat, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+
+# creator
+owner=kwolf@redhat.com
+
+seq=`basename $0`
+echo "QA output created by $seq"
+
+here=`pwd`
+status=1	# failure is the default!
+
+_cleanup()
+{
+	_cleanup_test_img
+}
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+# get standard environment, filters and checks
+. ./common.rc
+. ./common.filter
+
+_supported_fmt qcow2
+_supported_proto file
+_supported_os Linux
+
+CLUSTER_SIZE=4k
+size=128M
+
+echo
+echo == backing file contains zeros ==
+
+# Make sure that the whole cluster is allocated even for partial write_zeroes
+# when the backing file contains zeros
+
+CLUSTER_SIZE=512 TEST_IMG="$TEST_IMG.base" _make_test_img $size
+_make_test_img -b "$TEST_IMG.base"
+
+$QEMU_IO -c "write -z 0k 2k" "$TEST_IMG" | _filter_qemu_io
+$QEMU_IO -c "write -z 10k 2k" "$TEST_IMG" | _filter_qemu_io
+$QEMU_IO -c "write -z 17k 2k" "$TEST_IMG" | _filter_qemu_io
+$QEMU_IMG map --output=json "$TEST_IMG" | _filter_qemu_img_map
+
+echo
+echo == backing file contains non-zero data before write_zeros ==
+
+CLUSTER_SIZE=512 TEST_IMG="$TEST_IMG.base" _make_test_img $size
+_make_test_img -b "$TEST_IMG.base"
+
+$QEMU_IO -c "write -P 0x11 32k 1k" "$TEST_IMG.base" | _filter_qemu_io
+$QEMU_IO -c "write -z 34k 1k" "$TEST_IMG" | _filter_qemu_io
+$QEMU_IO -c "read -P 0x11 32k 1k" "$TEST_IMG" | _filter_qemu_io
+$QEMU_IO -c "read -P 0 33k 3k" "$TEST_IMG" | _filter_qemu_io
+$QEMU_IMG map --output=json "$TEST_IMG" | _filter_qemu_img_map
+
+echo
+echo == backing file contains non-zero data after write_zeros ==
+
+CLUSTER_SIZE=512 TEST_IMG="$TEST_IMG.base" _make_test_img $size
+_make_test_img -b "$TEST_IMG.base"
+
+$QEMU_IO -c "write -P 0x11 43k 1k" "$TEST_IMG.base" | _filter_qemu_io
+$QEMU_IO -c "write -z 41k 1k" "$TEST_IMG" | _filter_qemu_io
+$QEMU_IO -c "read -P 0x11 43k 1k" "$TEST_IMG" | _filter_qemu_io
+$QEMU_IO -c "read -P 0 40k 3k" "$TEST_IMG" | _filter_qemu_io
+$QEMU_IMG map --output=json "$TEST_IMG" | _filter_qemu_img_map
+
+echo
+echo == spanning multiple clusters ==
+
+CLUSTER_SIZE=512 TEST_IMG="$TEST_IMG.base" _make_test_img $size
+_make_test_img -b "$TEST_IMG.base"
+
+$QEMU_IO -c "write -P 0x11 74k 2k" "$TEST_IMG.base" | _filter_qemu_io
+$QEMU_IO -c "write -z 66k 7k" "$TEST_IMG" | _filter_qemu_io
+$QEMU_IO -c "read -P 0x11 74k 2k" "$TEST_IMG" | _filter_qemu_io
+$QEMU_IO -c "read -P 0 64k 10k" "$TEST_IMG" | _filter_qemu_io
+$QEMU_IMG map --output=json "$TEST_IMG" | _filter_qemu_img_map
+
+# success, all done
+echo "*** done"
+rm -f $seq.full
+status=0
diff --git a/tests/qemu-iotests/154.out b/tests/qemu-iotests/154.out
new file mode 100644
index 0000000..daf3f6d
--- /dev/null
+++ b/tests/qemu-iotests/154.out
@@ -0,0 +1,64 @@ 
+QA output created by 154
+
+== backing file contains zeros ==
+Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=134217728
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728 backing_file=TEST_DIR/t.IMGFMT.base
+wrote 2048/2048 bytes at offset 0
+2 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+wrote 2048/2048 bytes at offset 10240
+2 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+wrote 2048/2048 bytes at offset 17408
+2 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+[{ "start": 0, "length": 4096, "depth": 0, "zero": true, "data": false},
+{ "start": 4096, "length": 4096, "depth": 1, "zero": true, "data": false},
+{ "start": 8192, "length": 4096, "depth": 0, "zero": true, "data": false},
+{ "start": 12288, "length": 4096, "depth": 1, "zero": true, "data": false},
+{ "start": 16384, "length": 4096, "depth": 0, "zero": true, "data": false},
+{ "start": 20480, "length": 134197248, "depth": 1, "zero": true, "data": false}]
+
+== backing file contains non-zero data before write_zeros ==
+Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=134217728
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728 backing_file=TEST_DIR/t.IMGFMT.base
+wrote 1024/1024 bytes at offset 32768
+1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+wrote 1024/1024 bytes at offset 34816
+1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+read 1024/1024 bytes at offset 32768
+1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+read 3072/3072 bytes at offset 33792
+3 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+[{ "start": 0, "length": 32768, "depth": 1, "zero": true, "data": false},
+{ "start": 32768, "length": 4096, "depth": 0, "zero": false, "data": true, "offset": 20480},
+{ "start": 36864, "length": 134180864, "depth": 1, "zero": true, "data": false}]
+
+== backing file contains non-zero data after write_zeros ==
+Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=134217728
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728 backing_file=TEST_DIR/t.IMGFMT.base
+wrote 1024/1024 bytes at offset 44032
+1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+wrote 1024/1024 bytes at offset 41984
+1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+read 1024/1024 bytes at offset 44032
+1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+read 3072/3072 bytes at offset 40960
+3 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+[{ "start": 0, "length": 40960, "depth": 1, "zero": true, "data": false},
+{ "start": 40960, "length": 4096, "depth": 0, "zero": false, "data": true, "offset": 20480},
+{ "start": 45056, "length": 134172672, "depth": 1, "zero": true, "data": false}]
+
+== spanning multiple clusters ==
+Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=134217728
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728 backing_file=TEST_DIR/t.IMGFMT.base
+wrote 2048/2048 bytes at offset 75776
+2 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+wrote 7168/7168 bytes at offset 67584
+7 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+read 2048/2048 bytes at offset 75776
+2 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+read 10240/10240 bytes at offset 65536
+10 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+[{ "start": 0, "length": 65536, "depth": 1, "zero": true, "data": false},
+{ "start": 65536, "length": 8192, "depth": 0, "zero": true, "data": false},
+{ "start": 73728, "length": 4096, "depth": 0, "zero": false, "data": true, "offset": 20480},
+{ "start": 77824, "length": 134139904, "depth": 1, "zero": true, "data": false}]
+*** done
diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group
index 6067673..ab1d76e 100644
--- a/tests/qemu-iotests/group
+++ b/tests/qemu-iotests/group
@@ -153,3 +153,4 @@ 
 149 rw auto sudo
 150 rw auto quick
 152 rw auto quick
+154 rw auto backing quick