[v7,00/10] qemu-img convert with copy offloading
mbox series

Message ID 20180529055959.32002-1-famz@redhat.com
Headers show
Series
  • qemu-img convert with copy offloading
Related show

Message

Fam Zheng May 29, 2018, 5:59 a.m. UTC
v7: Fix qcow2.

v6: Pick up rev-by from Stefan and Eric.
    Tweak patch 2 commit message.

v5: - Fix raw offset/bytes check for read. [Eric]
    - Fix qcow2_handle_l2meta. [Stefan]
    - Add coroutine_fn whereever appropriate. [Stefan]

v4: - Fix raw offset and size. [Eric]
    - iscsi: Drop unnecessary return values and variables in favor of
      constants. [Stefan]
    - qcow2: Handle small backing case. [Stefan]
    - file-posix: Translate ENOSYS to ENOTSUP. [Stefan]
    - API documentation and commit message. [Stefan]
    - Add rev-by to patches 3, 5 - 10. [Stefan, Eric]

This series introduces block layer API for copy offloading and makes use of it
in qemu-img convert.

For now we implemented the operation in local file protocol with
copy_file_range(2).  Besides that it's possible to add similar to iscsi, nfs
and potentially more.

As far as its usage goes, in addition to qemu-img convert, we can emulate
offloading in scsi-disk (handle EXTENDED COPY command), and use the API in
block jobs too.

Fam Zheng (10):
  block: Introduce API for copy offloading
  raw: Check byte range uniformly
  raw: Implement copy offloading
  qcow2: Implement copy offloading
  file-posix: Implement bdrv_co_copy_range
  iscsi: Query and save device designator when opening
  iscsi: Create and use iscsi_co_wait_for_task
  iscsi: Implement copy offloading
  block-backend: Add blk_co_copy_range
  qemu-img: Convert with copy offloading

 block/block-backend.c          |  18 +++
 block/file-posix.c             |  96 ++++++++++++-
 block/io.c                     |  97 +++++++++++++
 block/iscsi.c                  | 314 ++++++++++++++++++++++++++++++++++++-----
 block/qcow2.c                  | 229 ++++++++++++++++++++++++++----
 block/raw-format.c             |  96 +++++++++----
 include/block/block.h          |  32 +++++
 include/block/block_int.h      |  38 +++++
 include/block/raw-aio.h        |  10 +-
 include/scsi/constants.h       |   5 +
 include/sysemu/block-backend.h |   4 +
 qemu-img.c                     |  50 ++++++-
 12 files changed, 890 insertions(+), 99 deletions(-)

Comments

Stefan Hajnoczi May 30, 2018, 4:06 p.m. UTC | #1
On Tue, May 29, 2018 at 01:59:49PM +0800, Fam Zheng wrote:
> v7: Fix qcow2.
> 
> v6: Pick up rev-by from Stefan and Eric.
>     Tweak patch 2 commit message.
> 
> v5: - Fix raw offset/bytes check for read. [Eric]
>     - Fix qcow2_handle_l2meta. [Stefan]
>     - Add coroutine_fn whereever appropriate. [Stefan]
> 
> v4: - Fix raw offset and size. [Eric]
>     - iscsi: Drop unnecessary return values and variables in favor of
>       constants. [Stefan]
>     - qcow2: Handle small backing case. [Stefan]
>     - file-posix: Translate ENOSYS to ENOTSUP. [Stefan]
>     - API documentation and commit message. [Stefan]
>     - Add rev-by to patches 3, 5 - 10. [Stefan, Eric]
> 
> This series introduces block layer API for copy offloading and makes use of it
> in qemu-img convert.
> 
> For now we implemented the operation in local file protocol with
> copy_file_range(2).  Besides that it's possible to add similar to iscsi, nfs
> and potentially more.
> 
> As far as its usage goes, in addition to qemu-img convert, we can emulate
> offloading in scsi-disk (handle EXTENDED COPY command), and use the API in
> block jobs too.

Fails to compile on Fedora 28.

Oops, I guess my glibc is too new.  This type of bug doesn't happen very
often :D :D :D.

block/file-posix.c:1452:14: error: static declaration of ‘copy_file_range’ follows non-static declaration
 static off_t copy_file_range(int in_fd, off_t *in_off, int out_fd,
              ^~~~~~~~~~~~~~~
In file included from /home/stefanha/qemu/include/qemu/osdep.h:75,
                 from block/file-posix.c:25:
/usr/include/unistd.h:1107:9: note: previous declaration of ‘copy_file_range’ was here
 ssize_t copy_file_range (int __infd, __off64_t *__pinoff,
         ^~~~~~~~~~~~~~~
Fam Zheng June 1, 2018, 2:35 a.m. UTC | #2
On Wed, 05/30 17:06, Stefan Hajnoczi wrote:
> On Tue, May 29, 2018 at 01:59:49PM +0800, Fam Zheng wrote:
> > v7: Fix qcow2.
> > 
> > v6: Pick up rev-by from Stefan and Eric.
> >     Tweak patch 2 commit message.
> > 
> > v5: - Fix raw offset/bytes check for read. [Eric]
> >     - Fix qcow2_handle_l2meta. [Stefan]
> >     - Add coroutine_fn whereever appropriate. [Stefan]
> > 
> > v4: - Fix raw offset and size. [Eric]
> >     - iscsi: Drop unnecessary return values and variables in favor of
> >       constants. [Stefan]
> >     - qcow2: Handle small backing case. [Stefan]
> >     - file-posix: Translate ENOSYS to ENOTSUP. [Stefan]
> >     - API documentation and commit message. [Stefan]
> >     - Add rev-by to patches 3, 5 - 10. [Stefan, Eric]
> > 
> > This series introduces block layer API for copy offloading and makes use of it
> > in qemu-img convert.
> > 
> > For now we implemented the operation in local file protocol with
> > copy_file_range(2).  Besides that it's possible to add similar to iscsi, nfs
> > and potentially more.
> > 
> > As far as its usage goes, in addition to qemu-img convert, we can emulate
> > offloading in scsi-disk (handle EXTENDED COPY command), and use the API in
> > block jobs too.
> 
> Fails to compile on Fedora 28.
> 
> Oops, I guess my glibc is too new.  This type of bug doesn't happen very
> often :D :D :D.
> 
> block/file-posix.c:1452:14: error: static declaration of ‘copy_file_range’ follows non-static declaration
>  static off_t copy_file_range(int in_fd, off_t *in_off, int out_fd,
>               ^~~~~~~~~~~~~~~
> In file included from /home/stefanha/qemu/include/qemu/osdep.h:75,
>                  from block/file-posix.c:25:
> /usr/include/unistd.h:1107:9: note: previous declaration of ‘copy_file_range’ was here
>  ssize_t copy_file_range (int __infd, __off64_t *__pinoff,
>          ^~~~~~~~~~~~~~~

It means it's time to update our Fedora docker image! Will fix this by adding a
configure test.

Fam