| Message ID | 20200521192137.1120211-5-eblake@redhat.com |
|---|---|
| State | New |
| Headers | show |
| Series | qemu-img: Add convert --bitmaps | expand |
21.05.2020 22:21, Eric Blake wrote: > Make it easier to copy all the persistent bitmaps of (the top layer > of) a source image along with its guest-visible contents, by adding a > boolean flag for use with qemu-img convert. This is basically > shorthand, as the same effect could be accomplished with a series of > 'qemu-img bitmap --add' and 'qemu-img bitmap --merge -b source' > commands, or by their corresponding QMP commands. > > Note that this command will fail in the same scenarios where 'qemu-img > measure' omits a 'bitmaps size:' line, namely, when either the source > or the destination lacks persistent bitmap support altogether. > > See also https://bugzilla.redhat.com/show_bug.cgi?id=1779893 > > While touching this, clean up a couple coding issues spotted in the > same function: an extra blank line, and merging back-to-back 'if > (!skip_create)' blocks. > > Signed-off-by: Eric Blake <eblake@redhat.com> > --- [..] > diff --git a/qemu-img.c b/qemu-img.c > index 0778d8f56614..8ecebe178890 100644 > --- a/qemu-img.c > +++ b/qemu-img.c > @@ -78,6 +78,7 @@ enum { > OPTION_ENABLE = 272, > OPTION_DISABLE = 273, > OPTION_MERGE = 274, [..] > > static int img_convert(int argc, char **argv) > @@ -2160,6 +2195,8 @@ static int img_convert(int argc, char **argv) > int64_t ret = -EINVAL; > bool force_share = false; > bool explict_min_sparse = false; > + bool bitmaps = false; > + size_t nbitmaps = 0; > > ImgConvertState s = (ImgConvertState) { > /* Need at least 4k of zeros for sparse detection */ > @@ -2179,6 +2216,7 @@ static int img_convert(int argc, char **argv) > {"target-image-opts", no_argument, 0, OPTION_TARGET_IMAGE_OPTS}, > {"salvage", no_argument, 0, OPTION_SALVAGE}, > {"target-is-zero", no_argument, 0, OPTION_TARGET_IS_ZERO}, > + {"bitmaps", no_argument, 0, OPTION_BITMAPS}, > {0, 0, 0, 0} > }; > c = getopt_long(argc, argv, ":hf:O:B:Cco:l:S:pt:T:qnm:WU", > @@ -2304,6 +2342,9 @@ static int img_convert(int argc, char **argv) > */ > s.has_zero_init = true; > break; > + case OPTION_BITMAPS: > + bitmaps = true; > + break; > } > } > > @@ -2365,7 +2406,6 @@ static int img_convert(int argc, char **argv) > goto fail_getopt; > } > > - > /* ret is still -EINVAL until here */ > ret = bdrv_parse_cache_mode(src_cache, &src_flags, &src_writethrough); > if (ret < 0) { > @@ -2525,6 +2565,27 @@ static int img_convert(int argc, char **argv) > } > } > > + /* Determine how many bitmaps need copying */ > + if (bitmaps) { > + BdrvDirtyBitmap *bm; > + > + if (s.src_num > 1) { > + error_report("Copying bitmaps only possible with single source"); > + ret = -1; > + goto out; > + } > + if (!bdrv_supports_persistent_dirty_bitmap(blk_bs(s.src[0]))) { > + error_report("Source lacks bitmap support"); > + ret = -1; > + goto out; > + } > + FOR_EACH_DIRTY_BITMAP(blk_bs(s.src[0]), bm) { > + if (bdrv_dirty_bitmap_get_persistence(bm)) { > + nbitmaps++; > + } > + } > + } > + > /* > * The later open call will need any decryption secrets, and > * bdrv_create() will purge "opts", so extract them now before > @@ -2533,9 +2594,7 @@ static int img_convert(int argc, char **argv) > if (!skip_create) { > open_opts = qdict_new(); > qemu_opt_foreach(opts, img_add_key_secrets, open_opts, &error_abort); > - } > > - if (!skip_create) { > /* Create the new image */ > ret = bdrv_create(drv, out_filename, opts, &local_err); > if (ret < 0) { > @@ -2573,6 +2632,13 @@ static int img_convert(int argc, char **argv) > } > out_bs = blk_bs(s.target); > > + if (nbitmaps > 0 && !bdrv_supports_persistent_dirty_bitmap(out_bs)) { We will not fail, if target doesn't support bitmaps, source supports them but has no bitmaps? Doesn't seem to be a problem, but a bit less strict than you write in commit message. So, maybe, s/nbitmaps > 0/bitmaps/ > + error_report("Format driver '%s' does not support bitmaps", > + out_fmt); Hmm seems, out_fmt may be NULL at this point, consider the path: const char *out_fmt = NULL ... [no -O option] --target-image-opts, so out_fmt doesn't default to "raw" but remains NULL ... So, with s/out_fmt/out_bs->drv->format_name/ (and with or without s/nbitmaps > 0/bitmaps/): Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
On 5/25/20 2:51 AM, Vladimir Sementsov-Ogievskiy wrote: > 21.05.2020 22:21, Eric Blake wrote: >> Make it easier to copy all the persistent bitmaps of (the top layer >> of) a source image along with its guest-visible contents, by adding a >> boolean flag for use with qemu-img convert. This is basically >> shorthand, as the same effect could be accomplished with a series of >> 'qemu-img bitmap --add' and 'qemu-img bitmap --merge -b source' >> commands, or by their corresponding QMP commands. >> >> Note that this command will fail in the same scenarios where 'qemu-img >> measure' omits a 'bitmaps size:' line, namely, when either the source >> or the destination lacks persistent bitmap support altogether. >> >> See also https://bugzilla.redhat.com/show_bug.cgi?id=1779893 >> >> While touching this, clean up a couple coding issues spotted in the >> same function: an extra blank line, and merging back-to-back 'if >> (!skip_create)' blocks. >> >> Signed-off-by: Eric Blake <eblake@redhat.com> >> --- > >> @@ -2573,6 +2632,13 @@ static int img_convert(int argc, char **argv) >> } >> out_bs = blk_bs(s.target); >> >> + if (nbitmaps > 0 && >> !bdrv_supports_persistent_dirty_bitmap(out_bs)) { > > We will not fail, if target doesn't support bitmaps, source supports > them but has no bitmaps? Doesn't seem to be a problem, but a bit less > strict than you write in commit message. > > So, maybe, s/nbitmaps > 0/bitmaps/ In fact, nbitmaps is not needed at all (it was useful in earlier iterations, but as the series has morphed, it is no longer buying me anything useful). > >> + error_report("Format driver '%s' does not support bitmaps", >> + out_fmt); > > Hmm seems, out_fmt may be NULL at this point, consider the path: > const char *out_fmt = NULL > ... > [no -O option] > --target-image-opts, so out_fmt doesn't default to "raw" but remains NULL > ... > > So, with s/out_fmt/out_bs->drv->format_name/ (and with or without > s/nbitmaps > 0/bitmaps/): > > > Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Okay, I'm squashing this in, and adding your R-b. Pull request coming shortly. diff --git i/qemu-img.c w/qemu-img.c index 8ecebe178890..d7e846e60742 100644 --- i/qemu-img.c +++ w/qemu-img.c @@ -2196,7 +2196,6 @@ static int img_convert(int argc, char **argv) bool force_share = false; bool explict_min_sparse = false; bool bitmaps = false; - size_t nbitmaps = 0; ImgConvertState s = (ImgConvertState) { /* Need at least 4k of zeros for sparse detection */ @@ -2565,10 +2564,8 @@ static int img_convert(int argc, char **argv) } } - /* Determine how many bitmaps need copying */ + /* Determine if bitmaps need copying */ if (bitmaps) { - BdrvDirtyBitmap *bm; - if (s.src_num > 1) { error_report("Copying bitmaps only possible with single source"); ret = -1; @@ -2579,11 +2576,6 @@ static int img_convert(int argc, char **argv) ret = -1; goto out; } - FOR_EACH_DIRTY_BITMAP(blk_bs(s.src[0]), bm) { - if (bdrv_dirty_bitmap_get_persistence(bm)) { - nbitmaps++; - } - } } /* @@ -2632,9 +2624,9 @@ static int img_convert(int argc, char **argv) } out_bs = blk_bs(s.target); - if (nbitmaps > 0 && !bdrv_supports_persistent_dirty_bitmap(out_bs)) { + if (bitmaps && !bdrv_supports_persistent_dirty_bitmap(out_bs)) { error_report("Format driver '%s' does not support bitmaps", - out_fmt); + out_bs->drv->format_name); ret = -1; goto out; } @@ -2700,7 +2692,7 @@ static int img_convert(int argc, char **argv) ret = convert_do_copy(&s); /* Now copy the bitmaps */ - if (nbitmaps > 0 && ret == 0) { + if (bitmaps && ret == 0) { ret = convert_copy_bitmaps(blk_bs(s.src[0]), out_bs); }
diff --git a/docs/tools/qemu-img.rst b/docs/tools/qemu-img.rst index 320cb52b9f61..69cd9a30373a 100644 --- a/docs/tools/qemu-img.rst +++ b/docs/tools/qemu-img.rst @@ -162,6 +162,10 @@ Parameters to convert subcommand: .. program:: qemu-img-convert +.. option:: --bitmaps + + Additionally copy all persistent bitmaps from the top layer of the source + .. option:: -n Skip the creation of the target volume @@ -397,7 +401,7 @@ Command description: 4 Error on reading data -.. option:: convert [--object OBJECTDEF] [--image-opts] [--target-image-opts] [--target-is-zero] [-U] [-C] [-c] [-p] [-q] [-n] [-f FMT] [-t CACHE] [-T SRC_CACHE] [-O OUTPUT_FMT] [-B BACKING_FILE] [-o OPTIONS] [-l SNAPSHOT_PARAM] [-S SPARSE_SIZE] [-m NUM_COROUTINES] [-W] FILENAME [FILENAME2 [...]] OUTPUT_FILENAME +.. option:: convert [--object OBJECTDEF] [--image-opts] [--target-image-opts] [--target-is-zero] [--bitmaps] [-U] [-C] [-c] [-p] [-q] [-n] [-f FMT] [-t CACHE] [-T SRC_CACHE] [-O OUTPUT_FMT] [-B BACKING_FILE] [-o OPTIONS] [-l SNAPSHOT_PARAM] [-S SPARSE_SIZE] [-m NUM_COROUTINES] [-W] FILENAME [FILENAME2 [...]] OUTPUT_FILENAME Convert the disk image *FILENAME* or a snapshot *SNAPSHOT_PARAM* to disk image *OUTPUT_FILENAME* using format *OUTPUT_FMT*. It can diff --git a/qemu-img.c b/qemu-img.c index 0778d8f56614..8ecebe178890 100644 --- a/qemu-img.c +++ b/qemu-img.c @@ -78,6 +78,7 @@ enum { OPTION_ENABLE = 272, OPTION_DISABLE = 273, OPTION_MERGE = 274, + OPTION_BITMAPS = 275, }; typedef enum OutputFormat { @@ -191,6 +192,7 @@ static void QEMU_NORETURN help(void) " hiding corruption that has already occurred.\n" "\n" "Parameters to convert subcommand:\n" + " '--bitmaps' copies all top-level persistent bitmaps to destination\n" " '-m' specifies how many coroutines work in parallel during the convert\n" " process (defaults to 8)\n" " '-W' allow to write to the target out of order rather than sequential\n" @@ -2139,6 +2141,39 @@ static int convert_do_copy(ImgConvertState *s) return s->ret; } +static int convert_copy_bitmaps(BlockDriverState *src, BlockDriverState *dst) +{ + BdrvDirtyBitmap *bm; + Error *err = NULL; + + FOR_EACH_DIRTY_BITMAP(src, bm) { + const char *name; + + if (!bdrv_dirty_bitmap_get_persistence(bm)) { + continue; + } + name = bdrv_dirty_bitmap_name(bm); + qmp_block_dirty_bitmap_add(dst->node_name, name, + true, bdrv_dirty_bitmap_granularity(bm), + true, true, + true, !bdrv_dirty_bitmap_enabled(bm), + &err); + if (err) { + error_reportf_err(err, "Failed to create bitmap %s: ", name); + return -1; + } + + do_dirty_bitmap_merge(dst->node_name, name, src->node_name, name, + &err); + if (err) { + error_reportf_err(err, "Failed to populate bitmap %s: ", name); + return -1; + } + } + + return 0; +} + #define MAX_BUF_SECTORS 32768 static int img_convert(int argc, char **argv) @@ -2160,6 +2195,8 @@ static int img_convert(int argc, char **argv) int64_t ret = -EINVAL; bool force_share = false; bool explict_min_sparse = false; + bool bitmaps = false; + size_t nbitmaps = 0; ImgConvertState s = (ImgConvertState) { /* Need at least 4k of zeros for sparse detection */ @@ -2179,6 +2216,7 @@ static int img_convert(int argc, char **argv) {"target-image-opts", no_argument, 0, OPTION_TARGET_IMAGE_OPTS}, {"salvage", no_argument, 0, OPTION_SALVAGE}, {"target-is-zero", no_argument, 0, OPTION_TARGET_IS_ZERO}, + {"bitmaps", no_argument, 0, OPTION_BITMAPS}, {0, 0, 0, 0} }; c = getopt_long(argc, argv, ":hf:O:B:Cco:l:S:pt:T:qnm:WU", @@ -2304,6 +2342,9 @@ static int img_convert(int argc, char **argv) */ s.has_zero_init = true; break; + case OPTION_BITMAPS: + bitmaps = true; + break; } } @@ -2365,7 +2406,6 @@ static int img_convert(int argc, char **argv) goto fail_getopt; } - /* ret is still -EINVAL until here */ ret = bdrv_parse_cache_mode(src_cache, &src_flags, &src_writethrough); if (ret < 0) { @@ -2525,6 +2565,27 @@ static int img_convert(int argc, char **argv) } } + /* Determine how many bitmaps need copying */ + if (bitmaps) { + BdrvDirtyBitmap *bm; + + if (s.src_num > 1) { + error_report("Copying bitmaps only possible with single source"); + ret = -1; + goto out; + } + if (!bdrv_supports_persistent_dirty_bitmap(blk_bs(s.src[0]))) { + error_report("Source lacks bitmap support"); + ret = -1; + goto out; + } + FOR_EACH_DIRTY_BITMAP(blk_bs(s.src[0]), bm) { + if (bdrv_dirty_bitmap_get_persistence(bm)) { + nbitmaps++; + } + } + } + /* * The later open call will need any decryption secrets, and * bdrv_create() will purge "opts", so extract them now before @@ -2533,9 +2594,7 @@ static int img_convert(int argc, char **argv) if (!skip_create) { open_opts = qdict_new(); qemu_opt_foreach(opts, img_add_key_secrets, open_opts, &error_abort); - } - if (!skip_create) { /* Create the new image */ ret = bdrv_create(drv, out_filename, opts, &local_err); if (ret < 0) { @@ -2573,6 +2632,13 @@ static int img_convert(int argc, char **argv) } out_bs = blk_bs(s.target); + if (nbitmaps > 0 && !bdrv_supports_persistent_dirty_bitmap(out_bs)) { + error_report("Format driver '%s' does not support bitmaps", + out_fmt); + ret = -1; + goto out; + } + if (s.compressed && !block_driver_can_compress(out_bs->drv)) { error_report("Compression not supported for this file format"); ret = -1; @@ -2632,6 +2698,12 @@ static int img_convert(int argc, char **argv) } ret = convert_do_copy(&s); + + /* Now copy the bitmaps */ + if (nbitmaps > 0 && ret == 0) { + ret = convert_copy_bitmaps(blk_bs(s.src[0]), out_bs); + } + out: if (!ret) { qemu_progress_print(100, 0); diff --git a/qemu-img-cmds.hx b/qemu-img-cmds.hx index a87d3cb264ce..10b910b67cf8 100644 --- a/qemu-img-cmds.hx +++ b/qemu-img-cmds.hx @@ -46,9 +46,9 @@ SRST ERST DEF("convert", img_convert, - "convert [--object objectdef] [--image-opts] [--target-image-opts] [--target-is-zero] [-U] [-C] [-c] [-p] [-q] [-n] [-f fmt] [-t cache] [-T src_cache] [-O output_fmt] [-B backing_file] [-o options] [-l snapshot_param] [-S sparse_size] [-m num_coroutines] [-W] [--salvage] filename [filename2 [...]] output_filename") + "convert [--object objectdef] [--image-opts] [--target-image-opts] [--target-is-zero] [--bitmaps] [-U] [-C] [-c] [-p] [-q] [-n] [-f fmt] [-t cache] [-T src_cache] [-O output_fmt] [-B backing_file] [-o options] [-l snapshot_param] [-S sparse_size] [-m num_coroutines] [-W] [--salvage] filename [filename2 [...]] output_filename") SRST -.. option:: convert [--object OBJECTDEF] [--image-opts] [--target-image-opts] [--target-is-zero] [-U] [-C] [-c] [-p] [-q] [-n] [-f FMT] [-t CACHE] [-T SRC_CACHE] [-O OUTPUT_FMT] [-B BACKING_FILE] [-o OPTIONS] [-l SNAPSHOT_PARAM] [-S SPARSE_SIZE] [-m NUM_COROUTINES] [-W] [--salvage] FILENAME [FILENAME2 [...]] OUTPUT_FILENAME +.. option:: convert [--object OBJECTDEF] [--image-opts] [--target-image-opts] [--target-is-zero] [--bitmaps] [-U] [-C] [-c] [-p] [-q] [-n] [-f FMT] [-t CACHE] [-T SRC_CACHE] [-O OUTPUT_FMT] [-B BACKING_FILE] [-o OPTIONS] [-l SNAPSHOT_PARAM] [-S SPARSE_SIZE] [-m NUM_COROUTINES] [-W] [--salvage] FILENAME [FILENAME2 [...]] OUTPUT_FILENAME ERST DEF("create", img_create,
Make it easier to copy all the persistent bitmaps of (the top layer of) a source image along with its guest-visible contents, by adding a boolean flag for use with qemu-img convert. This is basically shorthand, as the same effect could be accomplished with a series of 'qemu-img bitmap --add' and 'qemu-img bitmap --merge -b source' commands, or by their corresponding QMP commands. Note that this command will fail in the same scenarios where 'qemu-img measure' omits a 'bitmaps size:' line, namely, when either the source or the destination lacks persistent bitmap support altogether. See also https://bugzilla.redhat.com/show_bug.cgi?id=1779893 While touching this, clean up a couple coding issues spotted in the same function: an extra blank line, and merging back-to-back 'if (!skip_create)' blocks. Signed-off-by: Eric Blake <eblake@redhat.com> --- docs/tools/qemu-img.rst | 6 +++- qemu-img.c | 78 +++++++++++++++++++++++++++++++++++++++-- qemu-img-cmds.hx | 4 +-- 3 files changed, 82 insertions(+), 6 deletions(-)