Message ID | 20190927122355.7344-1-vsementsov@virtuozzo.com |
---|---|
Headers | show |
Series | qcow2-bitmaps: rewrite reopening logic | expand |
On 9/27/19 8:23 AM, Vladimir Sementsov-Ogievskiy wrote: > Hi all! > > Bitmaps reopening is buggy, reopening-rw just not working at all and > reopening-ro may lead to producing broken incremental > backup if we do temporary snapshot in a meantime. > > v5: > 01: - add Max's r-b > - fix s/QSIMPLE_INIT/QTAILQ_INIT/ in a comment > 02: - add Max's and John's a-b > 03: - improve test to check bitmap hashes in more safe way > 07: - drop wrong statement from commit message > - log events by hand > 08: - drop 'the' from comment > - add Corruption in case of existent IN_USE on RO->RW reopen > 09: - add Max's a-b and John's r-b > > v4: Drop complicated solution around reopening logic [Kevin], fix > the existing bug in a simplest way > > Vladimir Sementsov-Ogievskiy (9): > block: switch reopen queue from QSIMPLEQ to QTAILQ > block: reverse order for reopen commits > iotests: add test-case to 165 to test reopening qcow2 bitmaps to RW > block/qcow2-bitmap: get rid of bdrv_has_changed_persistent_bitmaps > block/qcow2-bitmap: drop qcow2_reopen_bitmaps_rw_hint() > block/qcow2-bitmap: do not remove bitmaps on reopen-ro > iotests: add test 260 to check bitmap life after snapshot + commit > block/qcow2-bitmap: fix and improve qcow2_reopen_bitmaps_rw > qcow2-bitmap: move bitmap reopen-rw code to qcow2_reopen_commit > > block/qcow2.h | 5 +- > include/block/block.h | 2 +- > include/block/block_int.h | 6 -- > include/block/dirty-bitmap.h | 1 - > block.c | 53 +++++------ > block/dirty-bitmap.c | 12 --- > block/qcow2-bitmap.c | 164 ++++++++++++++++++++++------------- > block/qcow2.c | 17 +++- > tests/qemu-iotests/165 | 57 +++++++++++- > tests/qemu-iotests/165.out | 4 +- > tests/qemu-iotests/260 | 89 +++++++++++++++++++ > tests/qemu-iotests/260.out | 52 +++++++++++ > tests/qemu-iotests/group | 1 + > 13 files changed, 343 insertions(+), 120 deletions(-) > create mode 100755 tests/qemu-iotests/260 > create mode 100644 tests/qemu-iotests/260.out > With the understanding that there are caveats that make this unsafe, but less unsafe and is a strict improvement: Reviewed-by: John Snow <jsnow@redhat.com>
On 9/27/19 8:23 AM, Vladimir Sementsov-Ogievskiy wrote: > Hi all! > > Bitmaps reopening is buggy, reopening-rw just not working at all and > reopening-ro may lead to producing broken incremental > backup if we do temporary snapshot in a meantime. > > v5: > 01: - add Max's r-b > - fix s/QSIMPLE_INIT/QTAILQ_INIT/ in a comment > 02: - add Max's and John's a-b > 03: - improve test to check bitmap hashes in more safe way > 07: - drop wrong statement from commit message > - log events by hand > 08: - drop 'the' from comment > - add Corruption in case of existent IN_USE on RO->RW reopen > 09: - add Max's a-b and John's r-b > > v4: Drop complicated solution around reopening logic [Kevin], fix > the existing bug in a simplest way > > Vladimir Sementsov-Ogievskiy (9): > block: switch reopen queue from QSIMPLEQ to QTAILQ > block: reverse order for reopen commits > iotests: add test-case to 165 to test reopening qcow2 bitmaps to RW > block/qcow2-bitmap: get rid of bdrv_has_changed_persistent_bitmaps > block/qcow2-bitmap: drop qcow2_reopen_bitmaps_rw_hint() > block/qcow2-bitmap: do not remove bitmaps on reopen-ro > iotests: add test 260 to check bitmap life after snapshot + commit > block/qcow2-bitmap: fix and improve qcow2_reopen_bitmaps_rw > qcow2-bitmap: move bitmap reopen-rw code to qcow2_reopen_commit > > block/qcow2.h | 5 +- > include/block/block.h | 2 +- > include/block/block_int.h | 6 -- > include/block/dirty-bitmap.h | 1 - > block.c | 53 +++++------ > block/dirty-bitmap.c | 12 --- > block/qcow2-bitmap.c | 164 ++++++++++++++++++++++------------- > block/qcow2.c | 17 +++- > tests/qemu-iotests/165 | 57 +++++++++++- > tests/qemu-iotests/165.out | 4 +- > tests/qemu-iotests/260 | 89 +++++++++++++++++++ > tests/qemu-iotests/260.out | 52 +++++++++++ > tests/qemu-iotests/group | 1 + > 13 files changed, 343 insertions(+), 120 deletions(-) > create mode 100755 tests/qemu-iotests/260 > create mode 100644 tests/qemu-iotests/260.out > Provisionally staged, pending feedback from Kevin. Some minor rebase conflicts against the current bitmaps branch: 012/17:[----] [-C] 'block/qcow2-bitmap: get rid of bdrv_has_changed_persistent_bitmaps' 014/17:[0004] [FC] 'block/qcow2-bitmap: do not remove bitmaps on reopen-ro' 017/17:[----] [-C] 'qcow2-bitmap: move bitmap reopen-rw code to qcow2_reopen_commit' 12: just context against changed _first and _next prototypes. 14: the signature of bdrv_release_dirty_bitmap has changed, which incurs a change in the refactor. 17: bdrv_can_store... has changed to bdrv_co_can_store... Same for bdrv_remove_persistent. Thanks, applied to my bitmaps tree: https://github.com/jnsnow/qemu/commits/bitmaps https://github.com/jnsnow/qemu.git --js
28.09.2019 2:36, John Snow wrote: > > > On 9/27/19 8:23 AM, Vladimir Sementsov-Ogievskiy wrote: >> Hi all! >> >> Bitmaps reopening is buggy, reopening-rw just not working at all and >> reopening-ro may lead to producing broken incremental >> backup if we do temporary snapshot in a meantime. >> >> v5: >> 01: - add Max's r-b >> - fix s/QSIMPLE_INIT/QTAILQ_INIT/ in a comment >> 02: - add Max's and John's a-b >> 03: - improve test to check bitmap hashes in more safe way >> 07: - drop wrong statement from commit message >> - log events by hand >> 08: - drop 'the' from comment >> - add Corruption in case of existent IN_USE on RO->RW reopen >> 09: - add Max's a-b and John's r-b >> >> v4: Drop complicated solution around reopening logic [Kevin], fix >> the existing bug in a simplest way >> >> Vladimir Sementsov-Ogievskiy (9): >> block: switch reopen queue from QSIMPLEQ to QTAILQ >> block: reverse order for reopen commits >> iotests: add test-case to 165 to test reopening qcow2 bitmaps to RW >> block/qcow2-bitmap: get rid of bdrv_has_changed_persistent_bitmaps >> block/qcow2-bitmap: drop qcow2_reopen_bitmaps_rw_hint() >> block/qcow2-bitmap: do not remove bitmaps on reopen-ro >> iotests: add test 260 to check bitmap life after snapshot + commit >> block/qcow2-bitmap: fix and improve qcow2_reopen_bitmaps_rw >> qcow2-bitmap: move bitmap reopen-rw code to qcow2_reopen_commit >> >> block/qcow2.h | 5 +- >> include/block/block.h | 2 +- >> include/block/block_int.h | 6 -- >> include/block/dirty-bitmap.h | 1 - >> block.c | 53 +++++------ >> block/dirty-bitmap.c | 12 --- >> block/qcow2-bitmap.c | 164 ++++++++++++++++++++++------------- >> block/qcow2.c | 17 +++- >> tests/qemu-iotests/165 | 57 +++++++++++- >> tests/qemu-iotests/165.out | 4 +- >> tests/qemu-iotests/260 | 89 +++++++++++++++++++ >> tests/qemu-iotests/260.out | 52 +++++++++++ >> tests/qemu-iotests/group | 1 + >> 13 files changed, 343 insertions(+), 120 deletions(-) >> create mode 100755 tests/qemu-iotests/260 >> create mode 100644 tests/qemu-iotests/260.out >> > > Provisionally staged, pending feedback from Kevin. > > Some minor rebase conflicts against the current bitmaps branch: > > 012/17:[----] [-C] 'block/qcow2-bitmap: get rid of > bdrv_has_changed_persistent_bitmaps' > 014/17:[0004] [FC] 'block/qcow2-bitmap: do not remove bitmaps on reopen-ro' > 017/17:[----] [-C] 'qcow2-bitmap: move bitmap reopen-rw code to > qcow2_reopen_commit' > > > 12: just context against changed _first and _next prototypes. > 14: the signature of bdrv_release_dirty_bitmap has changed, which incurs > a change in the refactor. > 17: bdrv_can_store... has changed to bdrv_co_can_store... > Same for bdrv_remove_persistent. > > > Thanks, applied to my bitmaps tree: > > https://github.com/jnsnow/qemu/commits/bitmaps > https://github.com/jnsnow/qemu.git > Thanks a lot!