From patchwork Fri Aug 9 15:32:01 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vladimir Sementsov-Ogievskiy X-Patchwork-Id: 1144760 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=virtuozzo.com Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 464q6F2RmGz9sP9 for ; Sat, 10 Aug 2019 01:34:13 +1000 (AEST) Received: from localhost ([::1]:60149 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hw6uB-0005QD-ED for incoming@patchwork.ozlabs.org; Fri, 09 Aug 2019 11:34:11 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49087) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hw6sN-00041j-IJ for qemu-devel@nongnu.org; Fri, 09 Aug 2019 11:32:21 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hw6sK-0002p3-Rv for qemu-devel@nongnu.org; Fri, 09 Aug 2019 11:32:19 -0400 Received: from relay.sw.ru ([185.231.240.75]:57586) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hw6sI-0002ij-1i; Fri, 09 Aug 2019 11:32:14 -0400 Received: from [10.94.3.0] (helo=kvm.qa.sw.ru) by relay.sw.ru with esmtp (Exim 4.92) (envelope-from ) id 1hw6sC-0004Cp-7N; Fri, 09 Aug 2019 18:32:08 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org Date: Fri, 9 Aug 2019 18:32:01 +0300 Message-Id: <20190809153207.49288-2-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20190809153207.49288-1-vsementsov@virtuozzo.com> References: <20190809153207.49288-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH v2 1/7] block/backup: deal with zero detection X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: fam@euphon.net, kwolf@redhat.com, vsementsov@virtuozzo.com, armbru@redhat.com, qemu-devel@nongnu.org, mreitz@redhat.com, stefanha@redhat.com, den@openvz.org, jsnow@redhat.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" We have detect_zeroes option, so at least for blockdev-backup user should define it if zero-detection is needed. For drive-backup leave detection enabled by default but do it through existing option instead of open-coding. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz --- block/backup.c | 15 ++++++--------- blockdev.c | 8 ++++---- 2 files changed, 10 insertions(+), 13 deletions(-) diff --git a/block/backup.c b/block/backup.c index adc4d44244..d815436455 100644 --- a/block/backup.c +++ b/block/backup.c @@ -113,7 +113,10 @@ static int coroutine_fn backup_cow_with_bounce_buffer(BackupBlockJob *job, BlockBackend *blk = job->common.blk; int nbytes; int read_flags = is_write_notifier ? BDRV_REQ_NO_SERIALISING : 0; - int write_flags = job->serialize_target_writes ? BDRV_REQ_SERIALISING : 0; + int write_flags = + (job->serialize_target_writes ? BDRV_REQ_SERIALISING : 0) | + (job->compress ? BDRV_REQ_WRITE_COMPRESSED : 0); + assert(QEMU_IS_ALIGNED(start, job->cluster_size)); bdrv_reset_dirty_bitmap(job->copy_bitmap, start, job->cluster_size); @@ -131,14 +134,8 @@ static int coroutine_fn backup_cow_with_bounce_buffer(BackupBlockJob *job, goto fail; } - if (buffer_is_zero(*bounce_buffer, nbytes)) { - ret = blk_co_pwrite_zeroes(job->target, start, - nbytes, write_flags | BDRV_REQ_MAY_UNMAP); - } else { - ret = blk_co_pwrite(job->target, start, - nbytes, *bounce_buffer, write_flags | - (job->compress ? BDRV_REQ_WRITE_COMPRESSED : 0)); - } + ret = blk_co_pwrite(job->target, start, nbytes, *bounce_buffer, + write_flags); if (ret < 0) { trace_backup_do_cow_write_fail(job, start, ret); if (error_is_read) { diff --git a/blockdev.c b/blockdev.c index 29c6c6044a..2d7e7be538 100644 --- a/blockdev.c +++ b/blockdev.c @@ -3613,7 +3613,7 @@ static BlockJob *do_drive_backup(DriveBackup *backup, JobTxn *txn, BlockDriverState *source = NULL; BlockJob *job = NULL; AioContext *aio_context; - QDict *options = NULL; + QDict *options; Error *local_err = NULL; int flags; int64_t size; @@ -3686,10 +3686,10 @@ static BlockJob *do_drive_backup(DriveBackup *backup, JobTxn *txn, goto out; } + options = qdict_new(); + qdict_put_str(options, "discard", "unmap"); + qdict_put_str(options, "detect-zeroes", "unmap"); if (backup->format) { - if (!options) { - options = qdict_new(); - } qdict_put_str(options, "driver", backup->format); } From patchwork Fri Aug 9 15:32:02 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vladimir Sementsov-Ogievskiy X-Patchwork-Id: 1144761 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=virtuozzo.com Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 464q6l3PK4z9sP3 for ; Sat, 10 Aug 2019 01:34:39 +1000 (AEST) Received: from localhost ([::1]:60156 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hw6ub-0006if-IO for incoming@patchwork.ozlabs.org; Fri, 09 Aug 2019 11:34:37 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49086) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hw6sN-00041h-IE for qemu-devel@nongnu.org; Fri, 09 Aug 2019 11:32:21 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hw6sL-0002pQ-5k for qemu-devel@nongnu.org; Fri, 09 Aug 2019 11:32:19 -0400 Received: from relay.sw.ru ([185.231.240.75]:57596) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hw6sI-0002io-1r; Fri, 09 Aug 2019 11:32:14 -0400 Received: from [10.94.3.0] (helo=kvm.qa.sw.ru) by relay.sw.ru with esmtp (Exim 4.92) (envelope-from ) id 1hw6sC-0004Cp-FL; Fri, 09 Aug 2019 18:32:08 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org Date: Fri, 9 Aug 2019 18:32:02 +0300 Message-Id: <20190809153207.49288-3-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20190809153207.49288-1-vsementsov@virtuozzo.com> References: <20190809153207.49288-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH v2 2/7] block/backup: refactor write_flags X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: fam@euphon.net, kwolf@redhat.com, vsementsov@virtuozzo.com, armbru@redhat.com, qemu-devel@nongnu.org, mreitz@redhat.com, stefanha@redhat.com, den@openvz.org, jsnow@redhat.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" write flags are constant, let's store it in BackupBlockJob instead of recalculating. It also makes two boolean fields to be unused, so, drop them. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: John Snow Reviewed-by: Max Reitz --- block/backup.c | 24 ++++++++++++------------ 1 file changed, 12 insertions(+), 12 deletions(-) diff --git a/block/backup.c b/block/backup.c index d815436455..c6a3b2b7bb 100644 --- a/block/backup.c +++ b/block/backup.c @@ -50,14 +50,13 @@ typedef struct BackupBlockJob { uint64_t len; uint64_t bytes_read; int64_t cluster_size; - bool compress; NotifierWithReturn before_write; QLIST_HEAD(, CowRequest) inflight_reqs; bool use_copy_range; int64_t copy_range_size; - bool serialize_target_writes; + BdrvRequestFlags write_flags; bool initializing_bitmap; } BackupBlockJob; @@ -113,10 +112,6 @@ static int coroutine_fn backup_cow_with_bounce_buffer(BackupBlockJob *job, BlockBackend *blk = job->common.blk; int nbytes; int read_flags = is_write_notifier ? BDRV_REQ_NO_SERIALISING : 0; - int write_flags = - (job->serialize_target_writes ? BDRV_REQ_SERIALISING : 0) | - (job->compress ? BDRV_REQ_WRITE_COMPRESSED : 0); - assert(QEMU_IS_ALIGNED(start, job->cluster_size)); bdrv_reset_dirty_bitmap(job->copy_bitmap, start, job->cluster_size); @@ -135,7 +130,7 @@ static int coroutine_fn backup_cow_with_bounce_buffer(BackupBlockJob *job, } ret = blk_co_pwrite(job->target, start, nbytes, *bounce_buffer, - write_flags); + job->write_flags); if (ret < 0) { trace_backup_do_cow_write_fail(job, start, ret); if (error_is_read) { @@ -163,7 +158,6 @@ static int coroutine_fn backup_cow_with_offload(BackupBlockJob *job, BlockBackend *blk = job->common.blk; int nbytes; int read_flags = is_write_notifier ? BDRV_REQ_NO_SERIALISING : 0; - int write_flags = job->serialize_target_writes ? BDRV_REQ_SERIALISING : 0; assert(QEMU_IS_ALIGNED(job->copy_range_size, job->cluster_size)); assert(QEMU_IS_ALIGNED(start, job->cluster_size)); @@ -172,7 +166,7 @@ static int coroutine_fn backup_cow_with_offload(BackupBlockJob *job, bdrv_reset_dirty_bitmap(job->copy_bitmap, start, job->cluster_size * nr_clusters); ret = blk_co_copy_range(blk, start, job->target, start, nbytes, - read_flags, write_flags); + read_flags, job->write_flags); if (ret < 0) { trace_backup_do_cow_copy_range_fail(job, start, ret); bdrv_set_dirty_bitmap(job->copy_bitmap, start, @@ -748,10 +742,16 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs, job->sync_mode = sync_mode; job->sync_bitmap = sync_bitmap; job->bitmap_mode = bitmap_mode; - job->compress = compress; - /* Detect image-fleecing (and similar) schemes */ - job->serialize_target_writes = bdrv_chain_contains(target, bs); + /* + * Set write flags: + * 1. Detect image-fleecing (and similar) schemes + * 2. Handle compression + */ + job->write_flags = + (bdrv_chain_contains(target, bs) ? BDRV_REQ_SERIALISING : 0) | + (compress ? BDRV_REQ_WRITE_COMPRESSED : 0); + job->cluster_size = cluster_size; job->copy_bitmap = copy_bitmap; copy_bitmap = NULL; From patchwork Fri Aug 9 15:32:03 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vladimir Sementsov-Ogievskiy X-Patchwork-Id: 1144765 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=virtuozzo.com Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 464q8L2rNqz9sBF for ; Sat, 10 Aug 2019 01:36:01 +1000 (AEST) Received: from localhost ([::1]:60174 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hw6vv-0000wb-QG for incoming@patchwork.ozlabs.org; Fri, 09 Aug 2019 11:35:59 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49081) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hw6sN-00041c-I3 for qemu-devel@nongnu.org; Fri, 09 Aug 2019 11:32:21 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hw6sL-0002pb-7k for qemu-devel@nongnu.org; Fri, 09 Aug 2019 11:32:19 -0400 Received: from relay.sw.ru ([185.231.240.75]:57598) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hw6sI-0002ik-10; Fri, 09 Aug 2019 11:32:14 -0400 Received: from [10.94.3.0] (helo=kvm.qa.sw.ru) by relay.sw.ru with esmtp (Exim 4.92) (envelope-from ) id 1hw6sC-0004Cp-NW; Fri, 09 Aug 2019 18:32:08 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org Date: Fri, 9 Aug 2019 18:32:03 +0300 Message-Id: <20190809153207.49288-4-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20190809153207.49288-1-vsementsov@virtuozzo.com> References: <20190809153207.49288-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH v2 3/7] block/io: handle alignment and max_transfer for copy_range X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: fam@euphon.net, kwolf@redhat.com, vsementsov@virtuozzo.com, armbru@redhat.com, qemu-devel@nongnu.org, mreitz@redhat.com, stefanha@redhat.com, den@openvz.org, jsnow@redhat.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" copy_range ignores these limitations, let's improve it. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz --- block/io.c | 48 ++++++++++++++++++++++++++++++++++++++++-------- 1 file changed, 40 insertions(+), 8 deletions(-) diff --git a/block/io.c b/block/io.c index 06305c6ea6..a5efb2200f 100644 --- a/block/io.c +++ b/block/io.c @@ -3005,11 +3005,24 @@ static int coroutine_fn bdrv_co_copy_range_internal( { BdrvTrackedRequest req; int ret; + uint32_t align = MAX(src->bs->bl.request_alignment, + dst->bs->bl.request_alignment); + uint32_t max_transfer = + QEMU_ALIGN_DOWN(MIN_NON_ZERO(MIN_NON_ZERO(src->bs->bl.max_transfer, + dst->bs->bl.max_transfer), + INT_MAX), align); /* TODO We can support BDRV_REQ_NO_FALLBACK here */ assert(!(read_flags & BDRV_REQ_NO_FALLBACK)); assert(!(write_flags & BDRV_REQ_NO_FALLBACK)); + if (max_transfer == 0 && bytes > 0) { + /* + * For example, if source max_transfer is smaller than target alignment. + */ + return -ENOTSUP; + } + if (!dst || !dst->bs) { return -ENOMEDIUM; } @@ -3031,7 +3044,10 @@ static int coroutine_fn bdrv_co_copy_range_internal( if (!src->bs->drv->bdrv_co_copy_range_from || !dst->bs->drv->bdrv_co_copy_range_to - || src->bs->encrypted || dst->bs->encrypted) { + || src->bs->encrypted || dst->bs->encrypted || + !QEMU_IS_ALIGNED(src_offset, src->bs->bl.request_alignment) || + !QEMU_IS_ALIGNED(dst_offset, dst->bs->bl.request_alignment) || + !QEMU_IS_ALIGNED(bytes, align)) { return -ENOTSUP; } @@ -3046,11 +3062,22 @@ static int coroutine_fn bdrv_co_copy_range_internal( wait_serialising_requests(&req); } - ret = src->bs->drv->bdrv_co_copy_range_from(src->bs, - src, src_offset, - dst, dst_offset, - bytes, - read_flags, write_flags); + while (bytes) { + int num = MIN(bytes, max_transfer); + + ret = src->bs->drv->bdrv_co_copy_range_from(src->bs, + src, src_offset, + dst, dst_offset, + num, + read_flags, + write_flags); + if (ret < 0) { + break; + } + bytes -= num; + src_offset += num; + dst_offset += num; + } tracked_request_end(&req); bdrv_dec_in_flight(src->bs); @@ -3060,12 +3087,17 @@ static int coroutine_fn bdrv_co_copy_range_internal( BDRV_TRACKED_WRITE); ret = bdrv_co_write_req_prepare(dst, dst_offset, bytes, &req, write_flags); - if (!ret) { + while (!ret && bytes) { + int num = MIN(bytes, max_transfer); + ret = dst->bs->drv->bdrv_co_copy_range_to(dst->bs, src, src_offset, dst, dst_offset, - bytes, + num, read_flags, write_flags); + bytes -= num; + src_offset += num; + dst_offset += num; } bdrv_co_write_req_finish(dst, dst_offset, bytes, &req, ret); tracked_request_end(&req); From patchwork Fri Aug 9 15:32:04 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vladimir Sementsov-Ogievskiy X-Patchwork-Id: 1144764 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=virtuozzo.com Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 464q7F2CzFz9sP3 for ; Sat, 10 Aug 2019 01:35:05 +1000 (AEST) Received: from localhost ([::1]:60166 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hw6v1-0007zj-D7 for incoming@patchwork.ozlabs.org; Fri, 09 Aug 2019 11:35:03 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49083) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hw6sN-00041e-I3 for qemu-devel@nongnu.org; Fri, 09 Aug 2019 11:32:21 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hw6sL-0002pd-7l for qemu-devel@nongnu.org; Fri, 09 Aug 2019 11:32:19 -0400 Received: from relay.sw.ru ([185.231.240.75]:57610) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hw6sI-0002im-2q; Fri, 09 Aug 2019 11:32:14 -0400 Received: from [10.94.3.0] (helo=kvm.qa.sw.ru) by relay.sw.ru with esmtp (Exim 4.92) (envelope-from ) id 1hw6sD-0004Cp-01; Fri, 09 Aug 2019 18:32:09 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org Date: Fri, 9 Aug 2019 18:32:04 +0300 Message-Id: <20190809153207.49288-5-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20190809153207.49288-1-vsementsov@virtuozzo.com> References: <20190809153207.49288-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH v2 4/7] block/backup: drop handling of max_transfer for copy_range X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: fam@euphon.net, kwolf@redhat.com, vsementsov@virtuozzo.com, armbru@redhat.com, qemu-devel@nongnu.org, mreitz@redhat.com, stefanha@redhat.com, den@openvz.org, jsnow@redhat.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Since previous commit, copy_range supports max_transfer, so we don't need to handle it by hand. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz --- block/backup.c | 11 ++--------- 1 file changed, 2 insertions(+), 9 deletions(-) diff --git a/block/backup.c b/block/backup.c index c6a3b2b7bb..228ba9423c 100644 --- a/block/backup.c +++ b/block/backup.c @@ -54,7 +54,6 @@ typedef struct BackupBlockJob { QLIST_HEAD(, CowRequest) inflight_reqs; bool use_copy_range; - int64_t copy_range_size; BdrvRequestFlags write_flags; bool initializing_bitmap; @@ -156,12 +155,11 @@ static int coroutine_fn backup_cow_with_offload(BackupBlockJob *job, int ret; int nr_clusters; BlockBackend *blk = job->common.blk; - int nbytes; + int nbytes = end - start; int read_flags = is_write_notifier ? BDRV_REQ_NO_SERIALISING : 0; - assert(QEMU_IS_ALIGNED(job->copy_range_size, job->cluster_size)); + assert(end - start < INT_MAX); assert(QEMU_IS_ALIGNED(start, job->cluster_size)); - nbytes = MIN(job->copy_range_size, end - start); nr_clusters = DIV_ROUND_UP(nbytes, job->cluster_size); bdrv_reset_dirty_bitmap(job->copy_bitmap, start, job->cluster_size * nr_clusters); @@ -756,11 +754,6 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs, job->copy_bitmap = copy_bitmap; copy_bitmap = NULL; job->use_copy_range = !compress; /* compression isn't supported for it */ - job->copy_range_size = MIN_NON_ZERO(blk_get_max_transfer(job->common.blk), - blk_get_max_transfer(job->target)); - job->copy_range_size = MAX(job->cluster_size, - QEMU_ALIGN_UP(job->copy_range_size, - job->cluster_size)); /* Required permissions are already taken with target's blk_new() */ block_job_add_bdrv(&job->common, "target", target, 0, BLK_PERM_ALL, From patchwork Fri Aug 9 15:32:05 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vladimir Sementsov-Ogievskiy X-Patchwork-Id: 1144759 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=virtuozzo.com Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 464q5J67kyz9sBF for ; Sat, 10 Aug 2019 01:33:24 +1000 (AEST) Received: from localhost ([::1]:60134 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hw6tO-0004U7-Uc for incoming@patchwork.ozlabs.org; Fri, 09 Aug 2019 11:33:22 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49052) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hw6sL-0003v2-IN for qemu-devel@nongnu.org; Fri, 09 Aug 2019 11:32:18 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hw6sK-0002oh-CT for qemu-devel@nongnu.org; Fri, 09 Aug 2019 11:32:17 -0400 Received: from relay.sw.ru ([185.231.240.75]:57606) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hw6sI-0002il-0r; Fri, 09 Aug 2019 11:32:14 -0400 Received: from [10.94.3.0] (helo=kvm.qa.sw.ru) by relay.sw.ru with esmtp (Exim 4.92) (envelope-from ) id 1hw6sD-0004Cp-7J; Fri, 09 Aug 2019 18:32:09 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org Date: Fri, 9 Aug 2019 18:32:05 +0300 Message-Id: <20190809153207.49288-6-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20190809153207.49288-1-vsementsov@virtuozzo.com> References: <20190809153207.49288-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH v2 5/7] block/backup: fix backup_cow_with_offload for last cluster X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: fam@euphon.net, kwolf@redhat.com, vsementsov@virtuozzo.com, armbru@redhat.com, qemu-devel@nongnu.org, mreitz@redhat.com, stefanha@redhat.com, den@openvz.org, jsnow@redhat.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" We shouldn't try to copy bytes beyond EOF. Fix it. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz --- block/backup.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/block/backup.c b/block/backup.c index 228ba9423c..d482d93458 100644 --- a/block/backup.c +++ b/block/backup.c @@ -155,7 +155,7 @@ static int coroutine_fn backup_cow_with_offload(BackupBlockJob *job, int ret; int nr_clusters; BlockBackend *blk = job->common.blk; - int nbytes = end - start; + int nbytes = MIN(end - start, job->len - start); int read_flags = is_write_notifier ? BDRV_REQ_NO_SERIALISING : 0; assert(end - start < INT_MAX); From patchwork Fri Aug 9 15:32:06 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vladimir Sementsov-Ogievskiy X-Patchwork-Id: 1144762 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=virtuozzo.com Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 464q6t5lymz9sBF for ; Sat, 10 Aug 2019 01:34:46 +1000 (AEST) Received: from localhost ([::1]:60160 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hw6ui-00075D-T8 for incoming@patchwork.ozlabs.org; Fri, 09 Aug 2019 11:34:44 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49089) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hw6sN-00041o-Ih for qemu-devel@nongnu.org; Fri, 09 Aug 2019 11:32:21 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hw6sL-0002pS-5o for qemu-devel@nongnu.org; Fri, 09 Aug 2019 11:32:19 -0400 Received: from relay.sw.ru ([185.231.240.75]:57612) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hw6sI-0002ih-1Z; Fri, 09 Aug 2019 11:32:14 -0400 Received: from [10.94.3.0] (helo=kvm.qa.sw.ru) by relay.sw.ru with esmtp (Exim 4.92) (envelope-from ) id 1hw6sD-0004Cp-Hj; Fri, 09 Aug 2019 18:32:09 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org Date: Fri, 9 Aug 2019 18:32:06 +0300 Message-Id: <20190809153207.49288-7-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20190809153207.49288-1-vsementsov@virtuozzo.com> References: <20190809153207.49288-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH v2 6/7] block/backup: teach backup_cow_with_bounce_buffer to copy more at once X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: fam@euphon.net, kwolf@redhat.com, vsementsov@virtuozzo.com, armbru@redhat.com, qemu-devel@nongnu.org, mreitz@redhat.com, stefanha@redhat.com, den@openvz.org, jsnow@redhat.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" backup_cow_with_offload can transfer more than on cluster. Let backup_cow_with_bounce_buffer behave similarly. It reduces number of IO and there are no needs to copy cluster by cluster. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/backup.c | 29 +++++++++++++++-------------- 1 file changed, 15 insertions(+), 14 deletions(-) diff --git a/block/backup.c b/block/backup.c index d482d93458..155e21d0a3 100644 --- a/block/backup.c +++ b/block/backup.c @@ -104,22 +104,25 @@ static int coroutine_fn backup_cow_with_bounce_buffer(BackupBlockJob *job, int64_t start, int64_t end, bool is_write_notifier, - bool *error_is_read, - void **bounce_buffer) + bool *error_is_read) { int ret; BlockBackend *blk = job->common.blk; int nbytes; int read_flags = is_write_notifier ? BDRV_REQ_NO_SERIALISING : 0; + void *bounce_buffer; assert(QEMU_IS_ALIGNED(start, job->cluster_size)); - bdrv_reset_dirty_bitmap(job->copy_bitmap, start, job->cluster_size); - nbytes = MIN(job->cluster_size, job->len - start); - if (!*bounce_buffer) { - *bounce_buffer = blk_blockalign(blk, job->cluster_size); + + nbytes = MIN(end - start, job->len - start); + bounce_buffer = blk_try_blockalign(blk, nbytes); + if (!bounce_buffer) { + return -ENOMEM; } - ret = blk_co_pread(blk, start, nbytes, *bounce_buffer, read_flags); + bdrv_reset_dirty_bitmap(job->copy_bitmap, start, end - start); + + ret = blk_co_pread(blk, start, nbytes, bounce_buffer, read_flags); if (ret < 0) { trace_backup_do_cow_read_fail(job, start, ret); if (error_is_read) { @@ -128,7 +131,7 @@ static int coroutine_fn backup_cow_with_bounce_buffer(BackupBlockJob *job, goto fail; } - ret = blk_co_pwrite(job->target, start, nbytes, *bounce_buffer, + ret = blk_co_pwrite(job->target, start, nbytes, bounce_buffer, job->write_flags); if (ret < 0) { trace_backup_do_cow_write_fail(job, start, ret); @@ -138,9 +141,12 @@ static int coroutine_fn backup_cow_with_bounce_buffer(BackupBlockJob *job, goto fail; } + qemu_vfree(bounce_buffer); return nbytes; + fail: bdrv_set_dirty_bitmap(job->copy_bitmap, start, job->cluster_size); + qemu_vfree(bounce_buffer); return ret; } @@ -254,7 +260,6 @@ static int coroutine_fn backup_do_cow(BackupBlockJob *job, CowRequest cow_request; int ret = 0; int64_t start, end; /* bytes */ - void *bounce_buffer = NULL; int64_t skip_bytes; qemu_co_rwlock_rdlock(&job->flush_rwlock); @@ -303,7 +308,7 @@ static int coroutine_fn backup_do_cow(BackupBlockJob *job, if (!job->use_copy_range) { ret = backup_cow_with_bounce_buffer(job, start, dirty_end, is_write_notifier, - error_is_read, &bounce_buffer); + error_is_read); } if (ret < 0) { break; @@ -318,10 +323,6 @@ static int coroutine_fn backup_do_cow(BackupBlockJob *job, ret = 0; } - if (bounce_buffer) { - qemu_vfree(bounce_buffer); - } - cow_request_end(&cow_request); trace_backup_do_cow_return(job, offset, bytes, ret); From patchwork Fri Aug 9 15:32:07 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vladimir Sementsov-Ogievskiy X-Patchwork-Id: 1144766 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=virtuozzo.com Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 464q9D50X5z9sNF for ; Sat, 10 Aug 2019 01:36:48 +1000 (AEST) Received: from localhost ([::1]:60192 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hw6wg-000337-Ps for incoming@patchwork.ozlabs.org; Fri, 09 Aug 2019 11:36:46 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49127) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hw6sR-000431-Am for qemu-devel@nongnu.org; Fri, 09 Aug 2019 11:32:25 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hw6sN-0002qz-Gn for qemu-devel@nongnu.org; Fri, 09 Aug 2019 11:32:21 -0400 Received: from relay.sw.ru ([185.231.240.75]:57614) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hw6sI-0002ii-1a; Fri, 09 Aug 2019 11:32:14 -0400 Received: from [10.94.3.0] (helo=kvm.qa.sw.ru) by relay.sw.ru with esmtp (Exim 4.92) (envelope-from ) id 1hw6sD-0004Cp-To; Fri, 09 Aug 2019 18:32:10 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org Date: Fri, 9 Aug 2019 18:32:07 +0300 Message-Id: <20190809153207.49288-8-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20190809153207.49288-1-vsementsov@virtuozzo.com> References: <20190809153207.49288-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH v2 7/7] block/backup: merge duplicated logic into backup_do_cow X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: fam@euphon.net, kwolf@redhat.com, vsementsov@virtuozzo.com, armbru@redhat.com, qemu-devel@nongnu.org, mreitz@redhat.com, stefanha@redhat.com, den@openvz.org, jsnow@redhat.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" backup_cow_with_offload and backup_cow_with_bounce_buffer contains a lot of duplicated logic. Move it into backup_do_cow. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz --- block/backup.c | 84 +++++++++++++++++++------------------------------- 1 file changed, 31 insertions(+), 53 deletions(-) diff --git a/block/backup.c b/block/backup.c index 155e21d0a3..ae780e1260 100644 --- a/block/backup.c +++ b/block/backup.c @@ -100,85 +100,60 @@ static void cow_request_end(CowRequest *req) /* Copy range to target with a bounce buffer and return the bytes copied. If * error occurred, return a negative error number */ -static int coroutine_fn backup_cow_with_bounce_buffer(BackupBlockJob *job, - int64_t start, - int64_t end, - bool is_write_notifier, - bool *error_is_read) +static int coroutine_fn backup_cow_with_bounce_buffer( + BackupBlockJob *job, int64_t offset, int64_t bytes, + BdrvRequestFlags read_flags, bool *error_is_read) { - int ret; + int ret = 0; BlockBackend *blk = job->common.blk; - int nbytes; - int read_flags = is_write_notifier ? BDRV_REQ_NO_SERIALISING : 0; - void *bounce_buffer; - - assert(QEMU_IS_ALIGNED(start, job->cluster_size)); + void *bounce_buffer = blk_try_blockalign(blk, bytes); - nbytes = MIN(end - start, job->len - start); - bounce_buffer = blk_try_blockalign(blk, nbytes); if (!bounce_buffer) { return -ENOMEM; } - bdrv_reset_dirty_bitmap(job->copy_bitmap, start, end - start); - - ret = blk_co_pread(blk, start, nbytes, bounce_buffer, read_flags); + ret = blk_co_pread(blk, offset, bytes, bounce_buffer, read_flags); if (ret < 0) { - trace_backup_do_cow_read_fail(job, start, ret); + trace_backup_do_cow_read_fail(job, offset, ret); if (error_is_read) { *error_is_read = true; } - goto fail; + goto out; } - ret = blk_co_pwrite(job->target, start, nbytes, bounce_buffer, + ret = blk_co_pwrite(job->target, offset, bytes, bounce_buffer, job->write_flags); if (ret < 0) { - trace_backup_do_cow_write_fail(job, start, ret); + trace_backup_do_cow_write_fail(job, offset, ret); if (error_is_read) { *error_is_read = false; } - goto fail; + goto out; } +out: qemu_vfree(bounce_buffer); - return nbytes; -fail: - bdrv_set_dirty_bitmap(job->copy_bitmap, start, job->cluster_size); - qemu_vfree(bounce_buffer); return ret; - } /* Copy range to target and return the bytes copied. If error occurred, return a * negative error number. */ static int coroutine_fn backup_cow_with_offload(BackupBlockJob *job, - int64_t start, - int64_t end, - bool is_write_notifier) + int64_t offset, + int64_t bytes, + BdrvRequestFlags read_flags) { int ret; - int nr_clusters; BlockBackend *blk = job->common.blk; - int nbytes = MIN(end - start, job->len - start); - int read_flags = is_write_notifier ? BDRV_REQ_NO_SERIALISING : 0; - - assert(end - start < INT_MAX); - assert(QEMU_IS_ALIGNED(start, job->cluster_size)); - nr_clusters = DIV_ROUND_UP(nbytes, job->cluster_size); - bdrv_reset_dirty_bitmap(job->copy_bitmap, start, - job->cluster_size * nr_clusters); - ret = blk_co_copy_range(blk, start, job->target, start, nbytes, + + ret = blk_co_copy_range(blk, offset, job->target, offset, bytes, read_flags, job->write_flags); if (ret < 0) { - trace_backup_do_cow_copy_range_fail(job, start, ret); - bdrv_set_dirty_bitmap(job->copy_bitmap, start, - job->cluster_size * nr_clusters); - return ret; + trace_backup_do_cow_copy_range_fail(job, offset, ret); } - return nbytes; + return ret; } /* @@ -261,6 +236,8 @@ static int coroutine_fn backup_do_cow(BackupBlockJob *job, int ret = 0; int64_t start, end; /* bytes */ int64_t skip_bytes; + BdrvRequestFlags read_flags = + is_write_notifier ? BDRV_REQ_NO_SERIALISING : 0; qemu_co_rwlock_rdlock(&job->flush_rwlock); @@ -274,6 +251,7 @@ static int coroutine_fn backup_do_cow(BackupBlockJob *job, while (start < end) { int64_t dirty_end; + int64_t cur_bytes; if (!bdrv_dirty_bitmap_get(job->copy_bitmap, start)) { trace_backup_do_cow_skip(job, start); @@ -297,30 +275,30 @@ static int coroutine_fn backup_do_cow(BackupBlockJob *job, } trace_backup_do_cow_process(job, start); + cur_bytes = MIN(dirty_end - start, job->len - start); + bdrv_reset_dirty_bitmap(job->copy_bitmap, start, dirty_end - start); if (job->use_copy_range) { - ret = backup_cow_with_offload(job, start, dirty_end, - is_write_notifier); + ret = backup_cow_with_offload(job, start, cur_bytes, read_flags); if (ret < 0) { job->use_copy_range = false; } } if (!job->use_copy_range) { - ret = backup_cow_with_bounce_buffer(job, start, dirty_end, - is_write_notifier, - error_is_read); + ret = backup_cow_with_bounce_buffer(job, start, cur_bytes, + read_flags, error_is_read); } if (ret < 0) { + bdrv_set_dirty_bitmap(job->copy_bitmap, start, dirty_end - start); break; } /* Publish progress, guest I/O counts as progress too. Note that the * offset field is an opaque progress value, it is not a disk offset. */ - start += ret; - job->bytes_read += ret; - job_progress_update(&job->common.job, ret); - ret = 0; + start += cur_bytes; + job->bytes_read += cur_bytes; + job_progress_update(&job->common.job, cur_bytes); } cow_request_end(&cow_request);