Message ID | 1413815733-22829-12-git-send-email-mreitz@redhat.com |
---|---|
State | New |
Headers | show |
Am 20.10.2014 um 16:35 hat Max Reitz geschrieben: > There are certain cases where repairing a qcow2 image might actually > damage it further (or rather, where repairing it has in fact damaged it > further with the old qcow2 check implementation). This should not > happen, so add a test for these cases. > > Furthermore, the repair function now repairs refblocks beyond the image > end by resizing the image accordingly. Add several tests for this as > well. > > Signed-off-by: Max Reitz <mreitz@redhat.com> > Reviewed-by: Eric Blake <eblake@redhat.com> In case you didn't know: qemu-img handles hex offsets just fine, so there's no need to comment the hex value and then convert it to decimal for the real command. > +--- Refblock is unallocated --- > + > +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 > +Repairing refcount block 1 is outside image > +ERROR cluster 16 refcount=0 reference=1 > +Rebuilding refcount structure > +Repairing cluster 1 refcount=1 reference=0 > +Repairing cluster 2 refcount=1 reference=0 > +Repairing cluster 16 refcount=1 reference=0 > +The following inconsistencies were found and repaired: > + > + 0 leaked clusters > + 2 corruptions > + > +Double checking the fixed image now... > +No errors were found on the image. > + > +--- Signed overflow after the refblock --- > + > +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 > +Repairing refcount block 1 is outside image > +ERROR could not resize image: Invalid argument > +Rebuilding refcount structure > +Repairing cluster 1 refcount=1 reference=0 > +Repairing cluster 2 refcount=1 reference=0 > +The following inconsistencies were found and repaired: > + > + 0 leaked clusters > + 1 corruptions > + > +Double checking the fixed image now... > +No errors were found on the image. This looks fishy. Compare this to the output of the previous case. We're now missing the corruption for the refblock because *nb_clusters wasn't increased. Don't we actually run the risk of allocating a clusters during the refcount rebuild that was outside the image, but couldn't be repaired? Perhaps a resize failure needs to stop the repair. Kevin
On 2014-10-21 at 16:12, Kevin Wolf wrote: > Am 20.10.2014 um 16:35 hat Max Reitz geschrieben: >> There are certain cases where repairing a qcow2 image might actually >> damage it further (or rather, where repairing it has in fact damaged it >> further with the old qcow2 check implementation). This should not >> happen, so add a test for these cases. >> >> Furthermore, the repair function now repairs refblocks beyond the image >> end by resizing the image accordingly. Add several tests for this as >> well. >> >> Signed-off-by: Max Reitz <mreitz@redhat.com> >> Reviewed-by: Eric Blake <eblake@redhat.com> > In case you didn't know: qemu-img handles hex offsets just fine, so > there's no need to comment the hex value and then convert it to decimal > for the real command. Aha *g* I did it that way in 060 and since then I just copied from there... >> +--- Refblock is unallocated --- >> + >> +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 >> +Repairing refcount block 1 is outside image >> +ERROR cluster 16 refcount=0 reference=1 >> +Rebuilding refcount structure >> +Repairing cluster 1 refcount=1 reference=0 >> +Repairing cluster 2 refcount=1 reference=0 >> +Repairing cluster 16 refcount=1 reference=0 >> +The following inconsistencies were found and repaired: >> + >> + 0 leaked clusters >> + 2 corruptions >> + >> +Double checking the fixed image now... >> +No errors were found on the image. >> + >> +--- Signed overflow after the refblock --- >> + >> +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 >> +Repairing refcount block 1 is outside image >> +ERROR could not resize image: Invalid argument >> +Rebuilding refcount structure >> +Repairing cluster 1 refcount=1 reference=0 >> +Repairing cluster 2 refcount=1 reference=0 >> +The following inconsistencies were found and repaired: >> + >> + 0 leaked clusters >> + 1 corruptions >> + >> +Double checking the fixed image now... >> +No errors were found on the image. > This looks fishy. Compare this to the output of the previous case. We're > now missing the corruption for the refblock because *nb_clusters wasn't > increased. I think we're rather missing the corruption for cluster 16 refcount=0. And I'd find that completely fine. > Don't we actually run the risk of allocating a clusters during the > refcount rebuild that was outside the image, but couldn't be repaired? > Perhaps a resize failure needs to stop the repair. The other way would be to unconditionally call inc_refcounts(). But I think it does not matter either way. If the refcount structure is rebuilt, all current refblocks are leaked anyway, so overwriting them is not an issue, I think. Max
Am 21.10.2014 um 16:20 hat Max Reitz geschrieben: > On 2014-10-21 at 16:12, Kevin Wolf wrote: > >Am 20.10.2014 um 16:35 hat Max Reitz geschrieben: > >>There are certain cases where repairing a qcow2 image might actually > >>damage it further (or rather, where repairing it has in fact damaged it > >>further with the old qcow2 check implementation). This should not > >>happen, so add a test for these cases. > >> > >>Furthermore, the repair function now repairs refblocks beyond the image > >>end by resizing the image accordingly. Add several tests for this as > >>well. > >> > >>Signed-off-by: Max Reitz <mreitz@redhat.com> > >>Reviewed-by: Eric Blake <eblake@redhat.com> > >In case you didn't know: qemu-img handles hex offsets just fine, so > >there's no need to comment the hex value and then convert it to decimal > >for the real command. > > Aha *g* > > I did it that way in 060 and since then I just copied from there... > > >>+--- Refblock is unallocated --- > >>+ > >>+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 > >>+Repairing refcount block 1 is outside image > >>+ERROR cluster 16 refcount=0 reference=1 > >>+Rebuilding refcount structure > >>+Repairing cluster 1 refcount=1 reference=0 > >>+Repairing cluster 2 refcount=1 reference=0 > >>+Repairing cluster 16 refcount=1 reference=0 > >>+The following inconsistencies were found and repaired: > >>+ > >>+ 0 leaked clusters > >>+ 2 corruptions > >>+ > >>+Double checking the fixed image now... > >>+No errors were found on the image. > >>+ > >>+--- Signed overflow after the refblock --- > >>+ > >>+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 > >>+Repairing refcount block 1 is outside image > >>+ERROR could not resize image: Invalid argument > >>+Rebuilding refcount structure > >>+Repairing cluster 1 refcount=1 reference=0 > >>+Repairing cluster 2 refcount=1 reference=0 > >>+The following inconsistencies were found and repaired: > >>+ > >>+ 0 leaked clusters > >>+ 1 corruptions > >>+ > >>+Double checking the fixed image now... > >>+No errors were found on the image. > >This looks fishy. Compare this to the output of the previous case. We're > >now missing the corruption for the refblock because *nb_clusters wasn't > >increased. > > I think we're rather missing the corruption for cluster 16 > refcount=0. And I'd find that completely fine. Wasn't cluster 16 the additional refblock? Technically, it should now be a corruption for cluster $HUGE_NUMBER, though I'll accept your excuse below (for refblocks at least). > >Don't we actually run the risk of allocating a clusters during the > >refcount rebuild that was outside the image, but couldn't be repaired? > >Perhaps a resize failure needs to stop the repair. > > The other way would be to unconditionally call inc_refcounts(). > > But I think it does not matter either way. If the refcount structure > is rebuilt, all current refblocks are leaked anyway, so overwriting > them is not an issue, I think. True. My concern was more about data blocks, but these are handled in a different place. Nevertheless, I think inc_refcounts() could ignore them with just a warning and a refcount rebuild would potentially overwrite them. Kevin
diff --git a/tests/qemu-iotests/108 b/tests/qemu-iotests/108 new file mode 100755 index 0000000..a2458be --- /dev/null +++ b/tests/qemu-iotests/108 @@ -0,0 +1,141 @@ +#!/bin/bash +# +# Test case for repairing qcow2 images which cannot be repaired using +# the on-disk refcount structures +# +# Copyright (C) 2014 Red Hat, Inc. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see <http://www.gnu.org/licenses/>. +# + +# creator +owner=mreitz@redhat.com + +seq="$(basename $0)" +echo "QA output created by $seq" + +here="$PWD" +tmp=/tmp/$$ +status=1 # failure is the default! + +_cleanup() +{ + _cleanup_test_img +} +trap "_cleanup; exit \$status" 0 1 2 3 15 + +# get standard environment, filters and checks +. ./common.rc +. ./common.filter + +# This tests qocw2-specific low-level functionality +_supported_fmt qcow2 +_supported_proto file +_supported_os Linux + +echo +echo '=== Repairing an image without any refcount table ===' +echo + +_make_test_img 64M +# just write some data +$QEMU_IO -c 'write -P 42 0 64k' "$TEST_IMG" | _filter_qemu_io + +# refcount_table_offset +poke_file "$TEST_IMG" 48 "\x00\x00\x00\x00\x00\x00\x00\x00" +# refcount_table_clusters +poke_file "$TEST_IMG" 56 "\x00\x00\x00\x00" + +_check_test_img -r all + +$QEMU_IO -c 'read -P 42 0 64k' "$TEST_IMG" | _filter_qemu_io + +echo +echo '=== Repairing unreferenced data cluster in new refblock area ===' +echo + +IMGOPTS='cluster_size=512' _make_test_img 64M +# Allocate the first 128 kB in the image (first refblock) +$QEMU_IO -c 'write 0 111104' "$TEST_IMG" | _filter_qemu_io +# should be 131072 +stat -c '%s' "$TEST_IMG" + +# Enter a cluster at 128 kB (0x20000) +# XXX: This (0x1ccc8) should be the first free entry in the last L2 table, but +# we cannot be sure +poke_file "$TEST_IMG" 117960 "\x80\x00\x00\x00\x00\x02\x00\x00" + +# Fill the cluster +truncate -s 131584 "$TEST_IMG" +$QEMU_IO -c "open -o driver=raw $TEST_IMG" -c 'write -P 42 128k 512' \ + | _filter_qemu_io + +# The data should now appear at this guest offset +$QEMU_IO -c 'read -P 42 111104 512' "$TEST_IMG" | _filter_qemu_io + +# This cluster is unallocated; fix it +_check_test_img -r all + +# This repair operation must have allocated a new refblock; and that refblock +# should not overlap with the unallocated data cluster. If it does, the data +# will be damaged, so check it. +$QEMU_IO -c 'read -P 42 111104 512' "$TEST_IMG" | _filter_qemu_io + +echo +echo '=== Repairing refblock beyond the image end ===' +echo + +echo +echo '--- Otherwise clean ---' +echo + +_make_test_img 64M +# Normally, qemu doesn't create empty refblocks, so we just have to do it by +# hand +# XXX: This (0x10008) should be the entry for the second refblock +poke_file "$TEST_IMG" 65544 "\x00\x00\x00\x00\x00\x10\x00\x00" +# Mark that refblock as used +# XXX: This (0x20020) should be the 17th entry (cluster 16) of the first +# refblock +poke_file "$TEST_IMG" 131104 "\x00\x01" +_check_test_img -r all + +echo +echo '--- Refblock is unallocated ---' +echo + +_make_test_img 64M +poke_file "$TEST_IMG" 65544 "\x00\x00\x00\x00\x00\x10\x00\x00" +_check_test_img -r all + +echo +echo '--- Signed overflow after the refblock ---' +echo + +_make_test_img 64M +poke_file "$TEST_IMG" 65544 "\x7f\xff\xff\xff\xff\xff\x00\x00" +_check_test_img -r all + +echo +echo '--- Unsigned overflow after the refblock ---' +echo + +_make_test_img 64M +poke_file "$TEST_IMG" 65544 "\xff\xff\xff\xff\xff\xff\x00\x00" +_check_test_img -r all + +# success, all done +echo '*** done' +rm -f $seq.full +status=0 diff --git a/tests/qemu-iotests/108.out b/tests/qemu-iotests/108.out new file mode 100644 index 0000000..824d5cf --- /dev/null +++ b/tests/qemu-iotests/108.out @@ -0,0 +1,110 @@ +QA output created by 108 + +=== Repairing an image without any refcount table === + +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 +wrote 65536/65536 bytes at offset 0 +64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +ERROR cluster 0 refcount=0 reference=1 +ERROR cluster 3 refcount=0 reference=1 +ERROR cluster 4 refcount=0 reference=1 +ERROR cluster 5 refcount=0 reference=1 +Rebuilding refcount structure +The following inconsistencies were found and repaired: + + 0 leaked clusters + 4 corruptions + +Double checking the fixed image now... +No errors were found on the image. +read 65536/65536 bytes at offset 0 +64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +=== Repairing unreferenced data cluster in new refblock area === + +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 +wrote 111104/111104 bytes at offset 0 +108.500 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +131072 +wrote 512/512 bytes at offset 131072 +512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 512/512 bytes at offset 111104 +512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +ERROR cluster 256 refcount=0 reference=1 +Rebuilding refcount structure +Repairing cluster 1 refcount=1 reference=0 +Repairing cluster 2 refcount=1 reference=0 +The following inconsistencies were found and repaired: + + 0 leaked clusters + 1 corruptions + +Double checking the fixed image now... +No errors were found on the image. +read 512/512 bytes at offset 111104 +512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +=== Repairing refblock beyond the image end === + + +--- Otherwise clean --- + +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 +Repairing refcount block 1 is outside image +The following inconsistencies were found and repaired: + + 0 leaked clusters + 1 corruptions + +Double checking the fixed image now... +No errors were found on the image. + +--- Refblock is unallocated --- + +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 +Repairing refcount block 1 is outside image +ERROR cluster 16 refcount=0 reference=1 +Rebuilding refcount structure +Repairing cluster 1 refcount=1 reference=0 +Repairing cluster 2 refcount=1 reference=0 +Repairing cluster 16 refcount=1 reference=0 +The following inconsistencies were found and repaired: + + 0 leaked clusters + 2 corruptions + +Double checking the fixed image now... +No errors were found on the image. + +--- Signed overflow after the refblock --- + +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 +Repairing refcount block 1 is outside image +ERROR could not resize image: Invalid argument +Rebuilding refcount structure +Repairing cluster 1 refcount=1 reference=0 +Repairing cluster 2 refcount=1 reference=0 +The following inconsistencies were found and repaired: + + 0 leaked clusters + 1 corruptions + +Double checking the fixed image now... +No errors were found on the image. + +--- Unsigned overflow after the refblock --- + +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 +Repairing refcount block 1 is outside image +ERROR could not resize image: Invalid argument +Rebuilding refcount structure +Repairing cluster 1 refcount=1 reference=0 +Repairing cluster 2 refcount=1 reference=0 +The following inconsistencies were found and repaired: + + 0 leaked clusters + 1 corruptions + +Double checking the fixed image now... +No errors were found on the image. +*** done diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group index b230996..be2054f 100644 --- a/tests/qemu-iotests/group +++ b/tests/qemu-iotests/group @@ -106,3 +106,4 @@ 103 rw auto quick 104 rw auto 105 rw auto quick +108 rw auto quick