From patchwork Wed May 9 14:58:06 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fam Zheng X-Patchwork-Id: 910911 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 40h02S0kKRz9s3q for ; Thu, 10 May 2018 01:02:24 +1000 (AEST) Received: from localhost ([::1]:56980 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fGQbl-00024v-Nh for incoming@patchwork.ozlabs.org; Wed, 09 May 2018 11:02:21 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:56647) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fGQYF-0006pt-MI for qemu-devel@nongnu.org; Wed, 09 May 2018 10:58:44 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fGQYE-0006Bk-AR for qemu-devel@nongnu.org; Wed, 09 May 2018 10:58:43 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:45888 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fGQY3-0005qC-2p; Wed, 09 May 2018 10:58:31 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 4BD3A4059FEB; Wed, 9 May 2018 14:58:30 +0000 (UTC) Received: from lemon.usersys.redhat.com (ovpn-12-170.pek2.redhat.com [10.72.12.170]) by smtp.corp.redhat.com (Postfix) with ESMTP id 1BE9ED74BA; Wed, 9 May 2018 14:58:21 +0000 (UTC) From: Fam Zheng To: qemu-devel@nongnu.org Date: Wed, 9 May 2018 22:58:06 +0800 Message-Id: <20180509145815.3330-1-famz@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.11.54.5 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.5]); Wed, 09 May 2018 14:58:30 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.5]); Wed, 09 May 2018 14:58:30 +0000 (UTC) for IP:'10.11.54.5' DOMAIN:'int-mx05.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'famz@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PATCH v3 0/9] qemu-img convert with copy offloading X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Fam Zheng , Stefan Hajnoczi , qemu-block@nongnu.org, Peter Lieven , Max Reitz , Ronnie Sahlberg , Paolo Bonzini Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" v3: Drop RFC. Update iotests reference output. Address Stefan's comments to v2: - Added a TODO for qcow2 copy with refcnt within the same image. - Include sys/syscall.h only on Linux. - Provide a dummy implementation returning -ENOSYS to reduce #ifdefs. - Use aio_ prefix for RawPosixAIOData fields. - Iscsi: split patches for code refactoring and device designator query. - Iscsi: improve parameter verification, types. don't use meaningless offset calculation. - Iscsi: set next pointer to NULL. - Iscsi: don't free uninitialized pointers. - Block backend: check request byte range. - Add reviewed-by lines. v2: - Add iscsi EXTENDED COPY. - Design change: bdrv_co_copy_range_{from,to}. [Stefan] - Retry upon EINTR. [Stefan] - Drop the bounce buffer fallback. It is inefficient to attempt the offloaded copy over and over again if the error is returned from the host rather than checking our internal state. E.g. trying copy_file_range between two filesystems results in EXDEV, in which case qemu-img.c should switch back to the copy path for the remaining sectors. This series introduces block layer API for copy offloading and makes use of it in qemu-img convert. For now we implemented the operation in local file protocol with copy_file_range(2). Besides that it's possible to add similar to iscsi, nfs and potentially more. As far as its usage goes, in addition to qemu-img convert, we can emulate offloading in scsi-disk (handle EXTENDED COPY command), and use the API in block jobs too. Fam Zheng (9): block: Introduce API for copy offloading raw: Implement copy offloading qcow2: Implement copy offloading file-posix: Implement bdrv_co_copy_range iscsi: Query and save device designator when opening iscsi: Create and use iscsi_co_wait_for_task iscsi: Implement copy offloading block-backend: Add blk_co_copy_range qemu-img: Convert with copy offloading block/block-backend.c | 18 +++ block/file-posix.c | 94 +++++++++++- block/io.c | 91 +++++++++++ block/iscsi.c | 323 ++++++++++++++++++++++++++++++++++----- block/qcow2.c | 228 +++++++++++++++++++++++---- block/raw-format.c | 20 +++ include/block/block.h | 4 + include/block/block_int.h | 30 ++++ include/block/raw-aio.h | 10 +- include/scsi/constants.h | 5 + include/sysemu/block-backend.h | 4 + qemu-img.c | 50 +++++- tests/qemu-iotests/178.out.qcow2 | 8 +- 13 files changed, 807 insertions(+), 78 deletions(-)