[RFC,4/8] block: add block_backup QMP command

@block-backup

Start a point-in-time copy of a block device to a new destination.

@device:  the name of the device whose writes should be mirrored.

@target: the target of the new image. If the file exists, or if it
         is a device, the existing file/device will be used as the new
         destination.  If it does not exist, a new file will be created.

@format: #optional the format of the new destination, default is to
         probe if @mode is 'existing', else the format of the source

@mode: #optional whether and how QEMU should create a new image, default is
       'absolute-paths'.

@speed:  #optional the maximum speed, in bytes per second

Returns: nothing on success
         If @device is not a valid block device, DeviceNotFound

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 blockdev.c       | 92 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 qapi-schema.json | 28 +++++++++++++++++
 qmp-commands.hx  |  6 ++++
 3 files changed, 126 insertions(+)

On 03/09/2013 03:22 PM, Stefan Hajnoczi wrote:
> @block-backup
> 
> Start a point-in-time copy of a block device to a new destination.
> 

Is a BLOCK_JOB_COMPLETED event emitted when the copy is completed?  If
not, it should be.

> +# Since 1.5
> +##
> +{ 'command': 'block-backup',
> +  'data': { 'device': 'str', 'target': 'str', '*format': 'str',
> +            '*mode': 'NewImageMode', '*speed': 'int' } }

This creates a new job type - do you need to update other locations
(such as BlockJobInfo's @type, and in QMP/qmp-events.txt under
BLOCK_JOB_CANCELLED/BLOCK_JOB_COMPLETED @type) to call out what string
is used for the new job type?  For that matter, is it finally time to
introduce a new enum type for all valid block job types, and use that
enum type instead of 'str' anywhere QMP data structures distinguish
based on job type?

On 03/09/2013 03:22 PM, Stefan Hajnoczi wrote:
> @block-backup
> 
> Start a point-in-time copy of a block device to a new destination.
> 

I'm trying to figure out how this is different from drive-mirror.  If I
understand correctly:

After starting drive-mirror, a write to the block device is also written
to the mirror, so that the destination sees the new data

After starting block-backup, a write to the block device flushes the old
data to the destination, so that the destination sees the old data

Timing-wise, I can accomplish a backup through either command, with the
following differences:

With drive-mirror, I start a job, wait for it to hit sync'd state, then
cancel the job. The copy is tied to the point where I cancel, and the
moment I cancel, I no longer have to worry about keeping the destination
writable (that is, the bulk of the copying is done prior to the point in
time).

With block-backup, I start a job, then wait for it to complete.  The
copy is tied to the point where I started the job, but as that may take
some time, I have to keep the destination writable until the job
completes (that is, the bulk of the work is done after the point in time).

The concept is indeed useful; more so if we can wire this into
'transaction' to capture multiple disks at the same point in time.

On Thu, Mar 14, 2013 at 03:52:16PM -0600, Eric Blake wrote:
> On 03/09/2013 03:22 PM, Stefan Hajnoczi wrote:
> > @block-backup
> > 
> > Start a point-in-time copy of a block device to a new destination.
> > 
> 
> I'm trying to figure out how this is different from drive-mirror.  If I
> understand correctly:
> 
> After starting drive-mirror, a write to the block device is also written
> to the mirror, so that the destination sees the new data
> 
> After starting block-backup, a write to the block device flushes the old
> data to the destination, so that the destination sees the old data
> 
> Timing-wise, I can accomplish a backup through either command, with the
> following differences:
> 
> With drive-mirror, I start a job, wait for it to hit sync'd state, then
> cancel the job. The copy is tied to the point where I cancel, and the
> moment I cancel, I no longer have to worry about keeping the destination
> writable (that is, the bulk of the copying is done prior to the point in
> time).
> 
> With block-backup, I start a job, then wait for it to complete.  The
> copy is tied to the point where I started the job, but as that may take
> some time, I have to keep the destination writable until the job
> completes (that is, the bulk of the work is done after the point in time).
> 
> The concept is indeed useful; more so if we can wire this into
> 'transaction' to capture multiple disks at the same point in time.

Although drive-mirror can be used in a similar way, the problem is that
you have to reach sync state before you can make the "point-in-time"
copy.  In other words, you need to plan ahead.

With block-backup the point-in-time copy is made the instant you issue
the command - this is much more practical.  Nevertheless, at one point I
think I asked Paolo if the block-backup code could be a sub-mode of
drive-mirror.  It seems that keeping them as separate block jobs is
reasonable since they work differently.

Regarding 'transaction', I agree.  It is handy to support 'block-backup'
as a 'transaction' action.  It can simplify QMP client error handling
since it is not necessary to manually cancel block jobs that have been
started when the last one fails to start.  Another consideration is that
'transaction' eliminates QMP round-trips and can therefore reduce guest
downtime when 'block-backup' is used after 'migrate' (guest is paused).

Stefan

Il 09/03/2013 23:22, Stefan Hajnoczi ha scritto:
> @block-backup
> 
> Start a point-in-time copy of a block device to a new destination.
> 
> @device:  the name of the device whose writes should be mirrored.
> 
> @target: the target of the new image. If the file exists, or if it
>          is a device, the existing file/device will be used as the new
>          destination.  If it does not exist, a new file will be created.
> 
> @format: #optional the format of the new destination, default is to
>          probe if @mode is 'existing', else the format of the source
> 
> @mode: #optional whether and how QEMU should create a new image, default is
>        'absolute-paths'.
> 
> @speed:  #optional the maximum speed, in bytes per second
> 
> Returns: nothing on success
>          If @device is not a valid block device, DeviceNotFound

Two features that would be nice to have:

1) a sync mode (full/top/none) like drive-mirror;

2) transaction support to start multiple backups live without stopping
the VM (for use cases that do not include migration)

Thanks,

Paolo

> Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
> ---
>  blockdev.c       | 92 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>  qapi-schema.json | 28 +++++++++++++++++
>  qmp-commands.hx  |  6 ++++
>  3 files changed, 126 insertions(+)
> 
> diff --git a/blockdev.c b/blockdev.c
> index 0e67d06..95a72de 100644
> --- a/blockdev.c
> +++ b/blockdev.c
> @@ -1208,6 +1208,98 @@ void qmp_block_commit(const char *device,
>      drive_get_ref(drive_get_by_blockdev(bs));
>  }
>  
> +void qmp_block_backup(const char *device, const char *target,
> +                      bool has_format, const char *format,
> +                      bool has_mode, enum NewImageMode mode,
> +                      bool has_speed, int64_t speed,
> +                      Error **errp)
> +{
> +    BlockDriverState *bs;
> +    BlockDriverState *target_bs;
> +    BlockDriver *proto_drv;
> +    BlockDriver *drv = NULL;
> +    Error *local_err = NULL;
> +    int flags;
> +    uint64_t size;
> +    int ret;
> +
> +    if (!has_speed) {
> +        speed = 0;
> +    }
> +    if (!has_mode) {
> +        mode = NEW_IMAGE_MODE_ABSOLUTE_PATHS;
> +    }
> +
> +    bs = bdrv_find(device);
> +    if (!bs) {
> +        error_set(errp, QERR_DEVICE_NOT_FOUND, device);
> +        return;
> +    }
> +
> +    if (!bdrv_is_inserted(bs)) {
> +        error_set(errp, QERR_DEVICE_HAS_NO_MEDIUM, device);
> +        return;
> +    }
> +
> +    if (!has_format) {
> +        format = mode == NEW_IMAGE_MODE_EXISTING ? NULL : bs->drv->format_name;
> +    }
> +    if (format) {
> +        drv = bdrv_find_format(format);
> +        if (!drv) {
> +            error_set(errp, QERR_INVALID_BLOCK_FORMAT, format);
> +            return;
> +        }
> +    }
> +
> +    if (bdrv_in_use(bs)) {
> +        error_set(errp, QERR_DEVICE_IN_USE, device);
> +        return;
> +    }
> +
> +    flags = bs->open_flags | BDRV_O_RDWR;
> +
> +    proto_drv = bdrv_find_protocol(target);
> +    if (!proto_drv) {
> +        error_set(errp, QERR_INVALID_BLOCK_FORMAT, format);
> +        return;
> +    }
> +
> +    bdrv_get_geometry(bs, &size);
> +    size *= 512;
> +    if (mode != NEW_IMAGE_MODE_EXISTING) {
> +        assert(format && drv);
> +        bdrv_img_create(target, format,
> +                        NULL, NULL, NULL, size, flags, &local_err, false);
> +    }
> +
> +    if (error_is_set(&local_err)) {
> +        error_propagate(errp, local_err);
> +        return;
> +    }
> +
> +    target_bs = bdrv_new("");
> +    ret = bdrv_open(target_bs, target, flags, drv);
> +
> +    if (ret < 0) {
> +        bdrv_delete(target_bs);
> +        error_set(errp, QERR_OPEN_FILE_FAILED, target);
> +        return;
> +    }
> +
> +    backup_start(bs, target_bs, speed, block_job_cb, bs, &local_err);
> +    if (local_err != NULL) {
> +        bdrv_delete(target_bs);
> +        error_propagate(errp, local_err);
> +        return;
> +    }
> +
> +    /* Grab a reference so hotplug does not delete the BlockDriverState from
> +     * underneath us.
> +     */
> +    drive_get_ref(drive_get_by_blockdev(bs));
> +}
> +
>  #define DEFAULT_MIRROR_BUF_SIZE   (10 << 20)
>  
>  void qmp_drive_mirror(const char *device, const char *target,
> diff --git a/qapi-schema.json b/qapi-schema.json
> index 6b64aec..1dbf7b5 100644
> --- a/qapi-schema.json
> +++ b/qapi-schema.json
> @@ -1716,6 +1716,34 @@
>              '*speed': 'int' } }
>  
>  ##
> +# @block-backup
> +#
> +# Start a point-in-time copy of a block device to a new destination.
> +#
> +# @device:  the name of the device whose writes should be mirrored.
> +#
> +# @target: the target of the new image. If the file exists, or if it
> +#          is a device, the existing file/device will be used as the new
> +#          destination.  If it does not exist, a new file will be created.
> +#
> +# @format: #optional the format of the new destination, default is to
> +#          probe if @mode is 'existing', else the format of the source
> +#
> +# @mode: #optional whether and how QEMU should create a new image, default is
> +#        'absolute-paths'.
> +#
> +# @speed:  #optional the maximum speed, in bytes per second
> +#
> +# Returns: nothing on success
> +#          If @device is not a valid block device, DeviceNotFound
> +#
> +# Since 1.5
> +##
> +{ 'command': 'block-backup',
> +  'data': { 'device': 'str', 'target': 'str', '*format': 'str',
> +            '*mode': 'NewImageMode', '*speed': 'int' } }
> +
> +##
>  # @drive-mirror
>  #
>  # Start mirroring a block device's writes to a new destination.
> diff --git a/qmp-commands.hx b/qmp-commands.hx
> index 95022e2..bda8eb4 100644
> --- a/qmp-commands.hx
> +++ b/qmp-commands.hx
> @@ -889,6 +889,12 @@ EQMP
>      },
>  
>      {
> +        .name       = "block-backup",
> +        .args_type  = "device:B,target:s,speed:i?,mode:s?,format:s?",
> +        .mhandler.cmd_new = qmp_marshal_input_block_backup,
> +    },
> +
> +    {
>          .name       = "block-job-set-speed",
>          .args_type  = "device:B,speed:o",
>          .mhandler.cmd_new = qmp_marshal_input_block_job_set_speed,
>