From patchwork Fri Nov 22 16:05:04 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 1199530 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="KH4qJO3P"; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 47KLs51glWz9sPK for ; Sat, 23 Nov 2019 03:06:33 +1100 (AEDT) Received: from localhost ([::1]:52472 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iYBS2-0000KK-HL for incoming@patchwork.ozlabs.org; Fri, 22 Nov 2019 11:06:30 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:47760) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iYBR2-0000CJ-DU for qemu-devel@nongnu.org; Fri, 22 Nov 2019 11:05:29 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iYBR0-0002iS-0I for qemu-devel@nongnu.org; Fri, 22 Nov 2019 11:05:27 -0500 Received: from us-smtp-delivery-1.mimecast.com ([205.139.110.120]:25155 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iYBQz-0002hJ-Ri for qemu-devel@nongnu.org; Fri, 22 Nov 2019 11:05:25 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1574438724; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=D3OK9Ld0gd/B0T+lw61N3zerjG3eHIwgOUwt+QI4d6s=; b=KH4qJO3PwslvpVTHtF8llpYa9fTX2wHbRKpf5U2gAgxfcO5qULmcyUYwAqbYfCTH46lNmU UuUFID18o9jjrobZIM+GkoDUBKvMe+SgWy+kyBxf54/as4n7w7JFeGNu2g3QcYvzaJw/RQ D6HOcwqJgf/7AQ8R5OFrt9icDbECtBw= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-86-LODmYaM-POGhBk-Qs3u2VA-1; Fri, 22 Nov 2019 11:05:23 -0500 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 47723188352F; Fri, 22 Nov 2019 16:05:22 +0000 (UTC) Received: from linux.fritz.box.com (ovpn-117-252.ams2.redhat.com [10.36.117.252]) by smtp.corp.redhat.com (Postfix) with ESMTP id 9C5CD100032E; Fri, 22 Nov 2019 16:05:20 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Subject: [PATCH v3 1/8] block: bdrv_co_do_pwrite_zeroes: 64 bit 'bytes' parameter Date: Fri, 22 Nov 2019 17:05:04 +0100 Message-Id: <20191122160511.8377-2-kwolf@redhat.com> In-Reply-To: <20191122160511.8377-1-kwolf@redhat.com> References: <20191122160511.8377-1-kwolf@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-MC-Unique: LODmYaM-POGhBk-Qs3u2VA-1 X-Mimecast-Spam-Score: 0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 205.139.110.120 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, qemu-devel@nongnu.org, mreitz@redhat.com, stefanha@redhat.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" bdrv_co_do_pwrite_zeroes() can already cope with maximum request sizes by calling the driver in a loop until everything is done. Make the small remaining change that is necessary to let it accept a 64 bit byte count. Signed-off-by: Kevin Wolf Reviewed-by: Eric Blake Reviewed-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Alberto Garcia Reviewed-by: Stefan Hajnoczi --- block/io.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/block/io.c b/block/io.c index f75777f5ea..003f4ea38c 100644 --- a/block/io.c +++ b/block/io.c @@ -42,7 +42,7 @@ static void bdrv_parent_cb_resize(BlockDriverState *bs); static int coroutine_fn bdrv_co_do_pwrite_zeroes(BlockDriverState *bs, - int64_t offset, int bytes, BdrvRequestFlags flags); + int64_t offset, int64_t bytes, BdrvRequestFlags flags); static void bdrv_parent_drained_begin(BlockDriverState *bs, BdrvChild *ignore, bool ignore_bds_parents) @@ -1730,7 +1730,7 @@ int coroutine_fn bdrv_co_preadv_part(BdrvChild *child, } static int coroutine_fn bdrv_co_do_pwrite_zeroes(BlockDriverState *bs, - int64_t offset, int bytes, BdrvRequestFlags flags) + int64_t offset, int64_t bytes, BdrvRequestFlags flags) { BlockDriver *drv = bs->drv; QEMUIOVector qiov; @@ -1760,7 +1760,7 @@ static int coroutine_fn bdrv_co_do_pwrite_zeroes(BlockDriverState *bs, assert(max_write_zeroes >= bs->bl.request_alignment); while (bytes > 0 && !ret) { - int num = bytes; + int num = MIN(bytes, BDRV_REQUEST_MAX_BYTES); /* Align request. Block drivers can expect the "bulk" of the request * to be aligned, and that unaligned requests do not cross cluster From patchwork Fri Nov 22 16:05:05 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 1199532 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="KaQln+Zf"; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 47KLvz1PJWz9sPj for ; Sat, 23 Nov 2019 03:09:03 +1100 (AEDT) Received: from localhost ([::1]:52548 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iYBUT-0003ph-1f for incoming@patchwork.ozlabs.org; Fri, 22 Nov 2019 11:09:01 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:47830) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iYBR6-0000LU-PF for qemu-devel@nongnu.org; Fri, 22 Nov 2019 11:05:35 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iYBR3-0002kv-MW for qemu-devel@nongnu.org; Fri, 22 Nov 2019 11:05:32 -0500 Received: from us-smtp-delivery-1.mimecast.com ([205.139.110.120]:46149 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iYBR3-0002ka-HS for qemu-devel@nongnu.org; Fri, 22 Nov 2019 11:05:29 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1574438729; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=F8ok8XK7Tfda8KpjFnC6PElx66iJN+3u/JTZmrrKNXY=; b=KaQln+Zf5lRGU9npnm4/aC2gkVbUCce9dqYVArp0L//ZQ52yUw45MXH90rwASrVgRgBZtA LTqvJkLccZfxlFJcLthScXJcS+5RLYYkj6bt4ZLdhEXCBgL1j3hVtfhjAm3uMMKBJS0cYH aSEl9X+UKt/7QHP6aGQNpisbk7XgLqY= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-212-q2ysPz5BPN2PbfOSXcO2BQ-1; Fri, 22 Nov 2019 11:05:25 -0500 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 6E3541883522; Fri, 22 Nov 2019 16:05:24 +0000 (UTC) Received: from linux.fritz.box.com (ovpn-117-252.ams2.redhat.com [10.36.117.252]) by smtp.corp.redhat.com (Postfix) with ESMTP id 956E7100032E; Fri, 22 Nov 2019 16:05:22 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Subject: [PATCH v3 2/8] block: Add no_fallback parameter to bdrv_co_truncate() Date: Fri, 22 Nov 2019 17:05:05 +0100 Message-Id: <20191122160511.8377-3-kwolf@redhat.com> In-Reply-To: <20191122160511.8377-1-kwolf@redhat.com> References: <20191122160511.8377-1-kwolf@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-MC-Unique: q2ysPz5BPN2PbfOSXcO2BQ-1 X-Mimecast-Spam-Score: 0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 205.139.110.120 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, qemu-devel@nongnu.org, mreitz@redhat.com, stefanha@redhat.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" This adds a no_fallback parameter to bdrv_co_truncate(), bdrv_truncate() and blk_truncate() in preparation for a fix that potentially needs to zero-write the new area. no_fallback will use BDRV_REQ_NO_FALLBACK for this operation and lets the truncate fail if an efficient zero write isn't possible. Only qmp_block_resize() passes true for this parameter because it is a blocking monitor command, so we don't want to add more potentially slow I/O operations to it than we already have. All other users will accept even a slow fallback to avoid failure. Signed-off-by: Kevin Wolf Reviewed-by: Eric Blake Reviewed-by: Max Reitz --- include/block/block.h | 5 +++-- include/sysemu/block-backend.h | 2 +- block/block-backend.c | 4 ++-- block/commit.c | 4 ++-- block/crypto.c | 4 ++-- block/io.c | 16 ++++++++++++---- block/mirror.c | 2 +- block/parallels.c | 6 +++--- block/qcow.c | 4 ++-- block/qcow2-refcount.c | 2 +- block/qcow2.c | 19 +++++++++++-------- block/qed.c | 2 +- block/raw-format.c | 2 +- block/vdi.c | 2 +- block/vhdx-log.c | 2 +- block/vhdx.c | 6 +++--- block/vmdk.c | 10 ++++++---- block/vpc.c | 2 +- blockdev.c | 2 +- qemu-img.c | 2 +- qemu-io-cmds.c | 2 +- tests/test-block-iothread.c | 6 +++--- 22 files changed, 60 insertions(+), 46 deletions(-) diff --git a/include/block/block.h b/include/block/block.h index 1df9848e74..3e44677905 100644 --- a/include/block/block.h +++ b/include/block/block.h @@ -347,9 +347,10 @@ BlockDriverState *bdrv_find_backing_image(BlockDriverState *bs, void bdrv_refresh_filename(BlockDriverState *bs); int coroutine_fn bdrv_co_truncate(BdrvChild *child, int64_t offset, bool exact, - PreallocMode prealloc, Error **errp); + PreallocMode prealloc, bool no_fallback, + Error **errp); int bdrv_truncate(BdrvChild *child, int64_t offset, bool exact, - PreallocMode prealloc, Error **errp); + PreallocMode prealloc, bool no_fallback, Error **errp); int64_t bdrv_nb_sectors(BlockDriverState *bs); int64_t bdrv_getlength(BlockDriverState *bs); diff --git a/include/sysemu/block-backend.h b/include/sysemu/block-backend.h index b198deca0b..487b29d13e 100644 --- a/include/sysemu/block-backend.h +++ b/include/sysemu/block-backend.h @@ -238,7 +238,7 @@ int coroutine_fn blk_co_pwrite_zeroes(BlockBackend *blk, int64_t offset, int blk_pwrite_compressed(BlockBackend *blk, int64_t offset, const void *buf, int bytes); int blk_truncate(BlockBackend *blk, int64_t offset, bool exact, - PreallocMode prealloc, Error **errp); + PreallocMode prealloc, bool no_fallback, Error **errp); int blk_pdiscard(BlockBackend *blk, int64_t offset, int bytes); int blk_save_vmstate(BlockBackend *blk, const uint8_t *buf, int64_t pos, int size); diff --git a/block/block-backend.c b/block/block-backend.c index 8b8f2a80a0..fcc9d60cdb 100644 --- a/block/block-backend.c +++ b/block/block-backend.c @@ -2073,14 +2073,14 @@ int blk_pwrite_compressed(BlockBackend *blk, int64_t offset, const void *buf, } int blk_truncate(BlockBackend *blk, int64_t offset, bool exact, - PreallocMode prealloc, Error **errp) + PreallocMode prealloc, bool no_fallback, Error **errp) { if (!blk_is_available(blk)) { error_setg(errp, "No medium inserted"); return -ENOMEDIUM; } - return bdrv_truncate(blk->root, offset, exact, prealloc, errp); + return bdrv_truncate(blk->root, offset, exact, prealloc, no_fallback, errp); } static void blk_pdiscard_entry(void *opaque) diff --git a/block/commit.c b/block/commit.c index 23c90b3b91..f074181d83 100644 --- a/block/commit.c +++ b/block/commit.c @@ -155,7 +155,7 @@ static int coroutine_fn commit_run(Job *job, Error **errp) } if (base_len < len) { - ret = blk_truncate(s->base, len, false, PREALLOC_MODE_OFF, NULL); + ret = blk_truncate(s->base, len, false, PREALLOC_MODE_OFF, false, NULL); if (ret) { goto out; } @@ -472,7 +472,7 @@ int bdrv_commit(BlockDriverState *bs) * we must return an error */ if (length > backing_length) { ret = blk_truncate(backing, length, false, PREALLOC_MODE_OFF, - &local_err); + false, &local_err); if (ret < 0) { error_report_err(local_err); goto ro_cleanup; diff --git a/block/crypto.c b/block/crypto.c index 24823835c1..0f28e8b4e1 100644 --- a/block/crypto.c +++ b/block/crypto.c @@ -114,7 +114,7 @@ static ssize_t block_crypto_init_func(QCryptoBlock *block, * which will be used by the crypto header */ return blk_truncate(data->blk, data->size + headerlen, false, - data->prealloc, errp); + data->prealloc, false, errp); } @@ -311,7 +311,7 @@ block_crypto_co_truncate(BlockDriverState *bs, int64_t offset, bool exact, offset += payload_offset; - return bdrv_co_truncate(bs->file, offset, exact, prealloc, errp); + return bdrv_co_truncate(bs->file, offset, exact, prealloc, false, errp); } static void block_crypto_close(BlockDriverState *bs) diff --git a/block/io.c b/block/io.c index 003f4ea38c..42e7558954 100644 --- a/block/io.c +++ b/block/io.c @@ -3313,9 +3313,15 @@ static void bdrv_parent_cb_resize(BlockDriverState *bs) * If 'exact' is true, the file must be resized to exactly the given * 'offset'. Otherwise, it is sufficient for the node to be at least * 'offset' bytes in length. + * + * If 'no_fallback' is true, a possibly needed writte_zeroes operation to avoid + * making a longer backing file visible will use BDRV_REQ_NO_FALLBACK. If the + * zero write is necessary and this flag is set, bdrv_co_truncate() will fail + * if efficient zero writes cannot be provided. */ int coroutine_fn bdrv_co_truncate(BdrvChild *child, int64_t offset, bool exact, - PreallocMode prealloc, Error **errp) + PreallocMode prealloc, bool no_fallback, + Error **errp) { BlockDriverState *bs = child->bs; BlockDriver *drv = bs->drv; @@ -3372,7 +3378,8 @@ int coroutine_fn bdrv_co_truncate(BdrvChild *child, int64_t offset, bool exact, if (drv->bdrv_co_truncate) { ret = drv->bdrv_co_truncate(bs, offset, exact, prealloc, errp); } else if (bs->file && drv->is_filter) { - ret = bdrv_co_truncate(bs->file, offset, exact, prealloc, errp); + ret = bdrv_co_truncate(bs->file, offset, exact, prealloc, no_fallback, + errp); } else { error_setg(errp, "Image format driver does not support resize"); ret = -ENOTSUP; @@ -3405,6 +3412,7 @@ typedef struct TruncateCo { int64_t offset; bool exact; PreallocMode prealloc; + bool no_fallback; Error **errp; int ret; } TruncateCo; @@ -3413,12 +3421,12 @@ static void coroutine_fn bdrv_truncate_co_entry(void *opaque) { TruncateCo *tco = opaque; tco->ret = bdrv_co_truncate(tco->child, tco->offset, tco->exact, - tco->prealloc, tco->errp); + tco->prealloc, tco->no_fallback, tco->errp); aio_wait_kick(); } int bdrv_truncate(BdrvChild *child, int64_t offset, bool exact, - PreallocMode prealloc, Error **errp) + PreallocMode prealloc, bool no_fallback, Error **errp) { Coroutine *co; TruncateCo tco = { diff --git a/block/mirror.c b/block/mirror.c index f0f2d9dff1..a333533a38 100644 --- a/block/mirror.c +++ b/block/mirror.c @@ -879,7 +879,7 @@ static int coroutine_fn mirror_run(Job *job, Error **errp) if (s->bdev_length > base_length) { ret = blk_truncate(s->target, s->bdev_length, false, - PREALLOC_MODE_OFF, NULL); + PREALLOC_MODE_OFF, false, NULL); if (ret < 0) { goto immediate_exit; } diff --git a/block/parallels.c b/block/parallels.c index 7a01997659..859f3f4904 100644 --- a/block/parallels.c +++ b/block/parallels.c @@ -203,7 +203,7 @@ static int64_t allocate_clusters(BlockDriverState *bs, int64_t sector_num, } else { ret = bdrv_truncate(bs->file, (s->data_end + space) << BDRV_SECTOR_BITS, - false, PREALLOC_MODE_OFF, NULL); + false, PREALLOC_MODE_OFF, false, NULL); } if (ret < 0) { return ret; @@ -493,7 +493,7 @@ static int coroutine_fn parallels_co_check(BlockDriverState *bs, * That means we have to pass exact=true. */ ret = bdrv_truncate(bs->file, res->image_end_offset, true, - PREALLOC_MODE_OFF, &local_err); + PREALLOC_MODE_OFF, false, &local_err); if (ret < 0) { error_report_err(local_err); res->check_errors++; @@ -888,7 +888,7 @@ static void parallels_close(BlockDriverState *bs) /* errors are ignored, so we might as well pass exact=true */ bdrv_truncate(bs->file, s->data_end << BDRV_SECTOR_BITS, true, - PREALLOC_MODE_OFF, NULL); + PREALLOC_MODE_OFF, false, NULL); } g_free(s->bat_dirty_bmap); diff --git a/block/qcow.c b/block/qcow.c index fce8989868..f43cb59cc0 100644 --- a/block/qcow.c +++ b/block/qcow.c @@ -480,7 +480,7 @@ static int get_cluster_offset(BlockDriverState *bs, return -E2BIG; } ret = bdrv_truncate(bs->file, cluster_offset + s->cluster_size, - false, PREALLOC_MODE_OFF, NULL); + false, PREALLOC_MODE_OFF, false, NULL); if (ret < 0) { return ret; } @@ -1034,7 +1034,7 @@ static int qcow_make_empty(BlockDriverState *bs) l1_length) < 0) return -1; ret = bdrv_truncate(bs->file, s->l1_table_offset + l1_length, false, - PREALLOC_MODE_OFF, NULL); + PREALLOC_MODE_OFF, false, NULL); if (ret < 0) return ret; diff --git a/block/qcow2-refcount.c b/block/qcow2-refcount.c index f67ac6b2d8..fdfdec336d 100644 --- a/block/qcow2-refcount.c +++ b/block/qcow2-refcount.c @@ -2017,7 +2017,7 @@ static int check_refblocks(BlockDriverState *bs, BdrvCheckResult *res, } ret = bdrv_truncate(bs->file, offset + s->cluster_size, false, - PREALLOC_MODE_OFF, &local_err); + PREALLOC_MODE_OFF, false, &local_err); if (ret < 0) { error_report_err(local_err); goto resize_fail; diff --git a/block/qcow2.c b/block/qcow2.c index 7c18721741..b201383c3d 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -3075,7 +3075,7 @@ static int coroutine_fn preallocate_co(BlockDriverState *bs, uint64_t offset, mode = PREALLOC_MODE_OFF; } ret = bdrv_co_truncate(s->data_file, host_offset + cur_bytes, false, - mode, errp); + mode, false, errp); if (ret < 0) { return ret; } @@ -3490,7 +3490,7 @@ qcow2_co_create(BlockdevCreateOptions *create_options, Error **errp) /* Okay, now that we have a valid image, let's give it the right size */ ret = blk_truncate(blk, qcow2_opts->size, false, qcow2_opts->preallocation, - errp); + false, errp); if (ret < 0) { error_prepend(errp, "Could not resize image: "); goto out; @@ -4035,7 +4035,7 @@ static int coroutine_fn qcow2_co_truncate(BlockDriverState *bs, int64_t offset, * always fulfilled, so there is no need to pass it on.) */ bdrv_co_truncate(bs->file, (last_cluster + 1) * s->cluster_size, - false, PREALLOC_MODE_OFF, &local_err); + false, PREALLOC_MODE_OFF, false, &local_err); if (local_err) { warn_reportf_err(local_err, "Failed to truncate the tail of the image: "); @@ -4057,7 +4057,8 @@ static int coroutine_fn qcow2_co_truncate(BlockDriverState *bs, int64_t offset, * file should be resized to the exact target size, too, * so we pass @exact here. */ - ret = bdrv_co_truncate(s->data_file, offset, exact, prealloc, errp); + ret = bdrv_co_truncate(s->data_file, offset, exact, prealloc, false, + errp); if (ret < 0) { goto fail; } @@ -4143,7 +4144,8 @@ static int coroutine_fn qcow2_co_truncate(BlockDriverState *bs, int64_t offset, new_file_size = allocation_start + nb_new_data_clusters * s->cluster_size; /* Image file grows, so @exact does not matter */ - ret = bdrv_co_truncate(bs->file, new_file_size, false, prealloc, errp); + ret = bdrv_co_truncate(bs->file, new_file_size, false, prealloc, false, + errp); if (ret < 0) { error_prepend(errp, "Failed to resize underlying file: "); qcow2_free_clusters(bs, allocation_start, @@ -4246,7 +4248,8 @@ qcow2_co_pwritev_compressed_part(BlockDriverState *bs, if (len < 0) { return len; } - return bdrv_co_truncate(bs->file, len, false, PREALLOC_MODE_OFF, NULL); + return bdrv_co_truncate(bs->file, len, false, PREALLOC_MODE_OFF, false, + NULL); } if (offset_into_cluster(s, offset)) { @@ -4484,7 +4487,7 @@ static int make_completely_empty(BlockDriverState *bs) } ret = bdrv_truncate(bs->file, (3 + l1_clusters) * s->cluster_size, false, - PREALLOC_MODE_OFF, &local_err); + PREALLOC_MODE_OFF, false, &local_err); if (ret < 0) { error_report_err(local_err); goto fail; @@ -5327,7 +5330,7 @@ static int qcow2_amend_options(BlockDriverState *bs, QemuOpts *opts, * Amending image options should ensure that the image has * exactly the given new values, so pass exact=true here. */ - ret = blk_truncate(blk, new_size, true, PREALLOC_MODE_OFF, errp); + ret = blk_truncate(blk, new_size, true, PREALLOC_MODE_OFF, false, errp); blk_unref(blk); if (ret < 0) { return ret; diff --git a/block/qed.c b/block/qed.c index d8c4e5fb1e..11e5cb37d9 100644 --- a/block/qed.c +++ b/block/qed.c @@ -677,7 +677,7 @@ static int coroutine_fn bdrv_qed_co_create(BlockdevCreateOptions *opts, * The QED format associates file length with allocation status, * so a new file (which is empty) must have a length of 0. */ - ret = blk_truncate(blk, 0, true, PREALLOC_MODE_OFF, errp); + ret = blk_truncate(blk, 0, true, PREALLOC_MODE_OFF, false, errp); if (ret < 0) { goto out; } diff --git a/block/raw-format.c b/block/raw-format.c index 3a76ec7dd2..950334cdb4 100644 --- a/block/raw-format.c +++ b/block/raw-format.c @@ -387,7 +387,7 @@ static int coroutine_fn raw_co_truncate(BlockDriverState *bs, int64_t offset, s->size = offset; offset += s->offset; - return bdrv_co_truncate(bs->file, offset, exact, prealloc, errp); + return bdrv_co_truncate(bs->file, offset, exact, prealloc, false, errp); } static void raw_eject(BlockDriverState *bs, bool eject_flag) diff --git a/block/vdi.c b/block/vdi.c index 0142da7233..2deb600614 100644 --- a/block/vdi.c +++ b/block/vdi.c @@ -875,7 +875,7 @@ static int coroutine_fn vdi_co_do_create(BlockdevCreateOptions *create_options, if (image_type == VDI_TYPE_STATIC) { ret = blk_truncate(blk, offset + blocks * block_size, false, - PREALLOC_MODE_OFF, errp); + PREALLOC_MODE_OFF, false, errp); if (ret < 0) { error_prepend(errp, "Failed to statically allocate file"); goto exit; diff --git a/block/vhdx-log.c b/block/vhdx-log.c index 13a49c2a33..4cd32a1231 100644 --- a/block/vhdx-log.c +++ b/block/vhdx-log.c @@ -558,7 +558,7 @@ static int vhdx_log_flush(BlockDriverState *bs, BDRVVHDXState *s, goto exit; } ret = bdrv_truncate(bs->file, new_file_size, false, - PREALLOC_MODE_OFF, NULL); + PREALLOC_MODE_OFF, false, NULL); if (ret < 0) { goto exit; } diff --git a/block/vhdx.c b/block/vhdx.c index f02d2611be..a58b2e2768 100644 --- a/block/vhdx.c +++ b/block/vhdx.c @@ -1264,7 +1264,7 @@ static int vhdx_allocate_block(BlockDriverState *bs, BDRVVHDXState *s, } return bdrv_truncate(bs->file, *new_offset + s->block_size, false, - PREALLOC_MODE_OFF, NULL); + PREALLOC_MODE_OFF, false, NULL); } /* @@ -1703,13 +1703,13 @@ static int vhdx_create_bat(BlockBackend *blk, BDRVVHDXState *s, /* All zeroes, so we can just extend the file - the end of the BAT * is the furthest thing we have written yet */ ret = blk_truncate(blk, data_file_offset, false, PREALLOC_MODE_OFF, - errp); + false, errp); if (ret < 0) { goto exit; } } else if (type == VHDX_TYPE_FIXED) { ret = blk_truncate(blk, data_file_offset + image_size, false, - PREALLOC_MODE_OFF, errp); + PREALLOC_MODE_OFF, false, errp); if (ret < 0) { goto exit; } diff --git a/block/vmdk.c b/block/vmdk.c index 20e909d997..1fbfed45ce 100644 --- a/block/vmdk.c +++ b/block/vmdk.c @@ -2077,7 +2077,7 @@ vmdk_co_pwritev_compressed(BlockDriverState *bs, uint64_t offset, } length = QEMU_ALIGN_UP(length, BDRV_SECTOR_SIZE); ret = bdrv_truncate(s->extents[i].file, length, false, - PREALLOC_MODE_OFF, NULL); + PREALLOC_MODE_OFF, false, NULL); if (ret < 0) { return ret; } @@ -2118,7 +2118,8 @@ static int vmdk_init_extent(BlockBackend *blk, int gd_buf_size; if (flat) { - ret = blk_truncate(blk, filesize, false, PREALLOC_MODE_OFF, errp); + ret = blk_truncate(blk, filesize, false, PREALLOC_MODE_OFF, false, + errp); goto exit; } magic = cpu_to_be32(VMDK4_MAGIC); @@ -2182,7 +2183,7 @@ static int vmdk_init_extent(BlockBackend *blk, } ret = blk_truncate(blk, le64_to_cpu(header.grain_offset) << 9, false, - PREALLOC_MODE_OFF, errp); + PREALLOC_MODE_OFF, false, errp); if (ret < 0) { goto exit; } @@ -2523,7 +2524,8 @@ static int coroutine_fn vmdk_co_do_create(int64_t size, /* bdrv_pwrite write padding zeros to align to sector, we don't need that * for description file */ if (desc_offset == 0) { - ret = blk_truncate(blk, desc_len, false, PREALLOC_MODE_OFF, errp); + ret = blk_truncate(blk, desc_len, false, PREALLOC_MODE_OFF, false, + errp); if (ret < 0) { goto exit; } diff --git a/block/vpc.c b/block/vpc.c index a65550298e..9202a517e1 100644 --- a/block/vpc.c +++ b/block/vpc.c @@ -898,7 +898,7 @@ static int create_fixed_disk(BlockBackend *blk, uint8_t *buf, /* Add footer to total size */ total_size += HEADER_SIZE; - ret = blk_truncate(blk, total_size, false, PREALLOC_MODE_OFF, errp); + ret = blk_truncate(blk, total_size, false, PREALLOC_MODE_OFF, false, errp); if (ret < 0) { return ret; } diff --git a/blockdev.c b/blockdev.c index 8e029e9c01..d7d3c250f6 100644 --- a/blockdev.c +++ b/blockdev.c @@ -3204,7 +3204,7 @@ void qmp_block_resize(bool has_device, const char *device, } bdrv_drained_begin(bs); - ret = blk_truncate(blk, size, false, PREALLOC_MODE_OFF, errp); + ret = blk_truncate(blk, size, false, PREALLOC_MODE_OFF, true, errp); bdrv_drained_end(bs); out: diff --git a/qemu-img.c b/qemu-img.c index 95a24b9762..7ee450bc18 100644 --- a/qemu-img.c +++ b/qemu-img.c @@ -3836,7 +3836,7 @@ static int img_resize(int argc, char **argv) * resizing, so pass @exact=true. It is of no use to report * success when the image has not actually been resized. */ - ret = blk_truncate(blk, total_size, true, prealloc, &err); + ret = blk_truncate(blk, total_size, true, prealloc, false, &err); if (!ret) { qprintf(quiet, "Image resized.\n"); } else { diff --git a/qemu-io-cmds.c b/qemu-io-cmds.c index 1b7e700020..08cce9d4cf 100644 --- a/qemu-io-cmds.c +++ b/qemu-io-cmds.c @@ -1715,7 +1715,7 @@ static int truncate_f(BlockBackend *blk, int argc, char **argv) * exact=true. It is better to err on the "emit more errors" side * than to be overly permissive. */ - ret = blk_truncate(blk, offset, true, PREALLOC_MODE_OFF, &local_err); + ret = blk_truncate(blk, offset, true, PREALLOC_MODE_OFF, false, &local_err); if (ret < 0) { error_report_err(local_err); return ret; diff --git a/tests/test-block-iothread.c b/tests/test-block-iothread.c index 0c861809f0..339dc2c9bd 100644 --- a/tests/test-block-iothread.c +++ b/tests/test-block-iothread.c @@ -185,18 +185,18 @@ static void test_sync_op_truncate(BdrvChild *c) int ret; /* Normal success path */ - ret = bdrv_truncate(c, 65536, false, PREALLOC_MODE_OFF, NULL); + ret = bdrv_truncate(c, 65536, false, PREALLOC_MODE_OFF, false, NULL); g_assert_cmpint(ret, ==, 0); /* Early error: Negative offset */ - ret = bdrv_truncate(c, -2, false, PREALLOC_MODE_OFF, NULL); + ret = bdrv_truncate(c, -2, false, PREALLOC_MODE_OFF, false, NULL); g_assert_cmpint(ret, ==, -EINVAL); /* Error: Read-only image */ c->bs->read_only = true; c->bs->open_flags &= ~BDRV_O_RDWR; - ret = bdrv_truncate(c, 65536, false, PREALLOC_MODE_OFF, NULL); + ret = bdrv_truncate(c, 65536, false, PREALLOC_MODE_OFF, false, NULL); g_assert_cmpint(ret, ==, -EACCES); c->bs->read_only = false; From patchwork Fri Nov 22 16:05:06 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 1199533 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="cKCFQCku"; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 47KLwZ3hHqz9sPK for ; Sat, 23 Nov 2019 03:09:34 +1100 (AEDT) Received: from localhost ([::1]:52584 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iYBUy-0004d0-5E for incoming@patchwork.ozlabs.org; Fri, 22 Nov 2019 11:09:32 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:47800) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iYBR4-0000I3-Rr for qemu-devel@nongnu.org; Fri, 22 Nov 2019 11:05:31 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iYBR3-0002l1-NZ for qemu-devel@nongnu.org; Fri, 22 Nov 2019 11:05:30 -0500 Received: from us-smtp-1.mimecast.com ([205.139.110.61]:32604 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iYBR3-0002kj-JL for qemu-devel@nongnu.org; Fri, 22 Nov 2019 11:05:29 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1574438729; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=pw2Brk0D+0cTmO+ThyXqjBZwmszkPvwLmjKihJY9kPE=; b=cKCFQCkujQabQSuDUGoWW35e29iyMs67Z3k2ENqq9w/jS5/heCkjrAWJyYNdloeFpV2ETj hfqiUwjgjs09DF0CAG2O6N+n5iWtKzWNNnaR/lvxvXvYj7kckbzZWt/zRnn2R8LBVmDPp3 kcghpUAiGIrAjbXpyswMBrbHpt3K8To= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-422-Sn5ci1yxPPazAWuIFj5waQ-1; Fri, 22 Nov 2019 11:05:27 -0500 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 6479A1883521; Fri, 22 Nov 2019 16:05:26 +0000 (UTC) Received: from linux.fritz.box.com (ovpn-117-252.ams2.redhat.com [10.36.117.252]) by smtp.corp.redhat.com (Postfix) with ESMTP id BCF1D10372C3; Fri, 22 Nov 2019 16:05:24 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Subject: [PATCH v3 3/8] qcow2: Declare BDRV_REQ_NO_FALLBACK supported Date: Fri, 22 Nov 2019 17:05:06 +0100 Message-Id: <20191122160511.8377-4-kwolf@redhat.com> In-Reply-To: <20191122160511.8377-1-kwolf@redhat.com> References: <20191122160511.8377-1-kwolf@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-MC-Unique: Sn5ci1yxPPazAWuIFj5waQ-1 X-Mimecast-Spam-Score: 0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 205.139.110.61 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, qemu-devel@nongnu.org, mreitz@redhat.com, stefanha@redhat.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" In the common case, qcow2_co_pwrite_zeroes() already only modifies metadata case, so we're fine with or without BDRV_REQ_NO_FALLBACK set. The only exception is when using an external data file, where the request is passed down to the block driver of the external data file. We are forwarding the BDRV_REQ_NO_FALLBACK flag there, though, so this is fine, too. Declare the flag supported therefore. Signed-off-by: Kevin Wolf Reviewed-by: Eric Blake Reviewed-by: Max Reitz Reviewed-by: Alberto Garcia --- block/qcow2.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/block/qcow2.c b/block/qcow2.c index b201383c3d..3fa10bf807 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -1722,7 +1722,8 @@ static int coroutine_fn qcow2_do_open(BlockDriverState *bs, QDict *options, } } - bs->supported_zero_flags = header.version >= 3 ? BDRV_REQ_MAY_UNMAP : 0; + bs->supported_zero_flags = header.version >= 3 ? + BDRV_REQ_MAY_UNMAP | BDRV_REQ_NO_FALLBACK : 0; /* Repair image if dirty */ if (!(flags & (BDRV_O_CHECK | BDRV_O_INACTIVE)) && !bs->read_only && From patchwork Fri Nov 22 16:05:07 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 1199537 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="eESFS+uB"; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 47KM0P6cBVz9sPK for ; Sat, 23 Nov 2019 03:12:53 +1100 (AEDT) Received: from localhost ([::1]:52612 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iYBYA-0007sg-Fb for incoming@patchwork.ozlabs.org; Fri, 22 Nov 2019 11:12:50 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:47854) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iYBR8-0000Ou-Gc for qemu-devel@nongnu.org; Fri, 22 Nov 2019 11:05:35 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iYBR7-0002r4-A2 for qemu-devel@nongnu.org; Fri, 22 Nov 2019 11:05:34 -0500 Received: from us-smtp-delivery-1.mimecast.com ([205.139.110.120]:41609 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iYBR7-0002op-3e for qemu-devel@nongnu.org; Fri, 22 Nov 2019 11:05:33 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1574438732; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=2/6lgTvhUffQT9Ya7xc06OJxjRNaeStSgj7a61gvGUo=; b=eESFS+uBXQV3Met9XOZrGNoIpOY1MbQlbXKdB5t+lwJpEUTwZzusMzmMAD/nhXsYC60v7R KeSb0jk5ZpAu5B2WubqJTh/he3zqQWFukkCylz6hO8yXK7Ug+5hl1DQXGVs1OtflF4ByWX v6AhA4TjmnoIXRHIw1SSOrlOqXcLGW8= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-378-pc2zqUFHO6un948GWoxDVw-1; Fri, 22 Nov 2019 11:05:29 -0500 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 67691107ACC5; Fri, 22 Nov 2019 16:05:28 +0000 (UTC) Received: from linux.fritz.box.com (ovpn-117-252.ams2.redhat.com [10.36.117.252]) by smtp.corp.redhat.com (Postfix) with ESMTP id B2B0810372C3; Fri, 22 Nov 2019 16:05:26 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Subject: [PATCH v3 4/8] block: truncate: Don't make backing file data visible Date: Fri, 22 Nov 2019 17:05:07 +0100 Message-Id: <20191122160511.8377-5-kwolf@redhat.com> In-Reply-To: <20191122160511.8377-1-kwolf@redhat.com> References: <20191122160511.8377-1-kwolf@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-MC-Unique: pc2zqUFHO6un948GWoxDVw-1 X-Mimecast-Spam-Score: 0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 205.139.110.120 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, qemu-devel@nongnu.org, mreitz@redhat.com, stefanha@redhat.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" When extending the size of an image that has a backing file larger than its old size, make sure that the backing file data doesn't become visible in the guest, but the added area is properly zeroed out. Consider the following scenario where the overlay is shorter than its backing file: base.qcow2: AAAAAAAA overlay.qcow2: BBBB When resizing (extending) overlay.qcow2, the new blocks should not stay unallocated and make the additional As from base.qcow2 visible like before this patch, but zeros should be read. A similar case happens with the various variants of a commit job when an intermediate file is short (- for unallocated): base.qcow2: A-A-AAAA mid.qcow2: BB-B top.qcow2: C--C--C- After commit top.qcow2 to mid.qcow2, the following happens: mid.qcow2: CB-C00C0 (correct result) mid.qcow2: CB-C--C- (before this fix) Without the fix, blocks that previously read as zeros on top.qcow2 suddenly turn into A. Signed-off-by: Kevin Wolf Reviewed-by: Eric Blake Reviewed-by: Max Reitz --- block/io.c | 33 +++++++++++++++++++++++++++++++++ 1 file changed, 33 insertions(+) diff --git a/block/io.c b/block/io.c index 42e7558954..61a63d9dc2 100644 --- a/block/io.c +++ b/block/io.c @@ -3392,12 +3392,45 @@ int coroutine_fn bdrv_co_truncate(BdrvChild *child, int64_t offset, bool exact, ret = refresh_total_sectors(bs, offset >> BDRV_SECTOR_BITS); if (ret < 0) { error_setg_errno(errp, -ret, "Could not refresh total sector count"); + goto fail_refresh_total_sectors; } else { offset = bs->total_sectors * BDRV_SECTOR_SIZE; } + + /* + * If the image has a backing file that is large enough that it would + * provide data for the new area, we cannot leave it unallocated because + * then the backing file content would become visible. Instead, zero-fill + * the area where backing file and new area overlap. + * + * Note that if the image has a backing file, but was opened without the + * backing file, taking care of keeping things consistent with that backing + * file is the user's responsibility. + */ + if (new_bytes && bs->backing && prealloc == PREALLOC_MODE_OFF) { + int64_t backing_len; + + backing_len = bdrv_getlength(backing_bs(bs)); + if (backing_len < 0) { + ret = backing_len; + goto fail_refresh_total_sectors; + } + + if (backing_len > old_size) { + ret = bdrv_co_do_pwrite_zeroes( + bs, old_size, MIN(new_bytes, backing_len - old_size), + BDRV_REQ_ZERO_WRITE | BDRV_REQ_MAY_UNMAP | + (no_fallback ? BDRV_REQ_NO_FALLBACK : 0)); + if (ret < 0) { + goto fail_refresh_total_sectors; + } + } + } + /* It's possible that truncation succeeded but refresh_total_sectors * failed, but the latter doesn't affect how we should finish the request. * Pass 0 as the last parameter so that dirty bitmaps etc. are handled. */ +fail_refresh_total_sectors: bdrv_co_write_req_finish(child, offset - new_bytes, new_bytes, &req, 0); out: From patchwork Fri Nov 22 16:05:08 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 1199545 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="XgaymrD3"; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 47KM3k3CXYz9sPj for ; Sat, 23 Nov 2019 03:15:46 +1100 (AEDT) Received: from localhost ([::1]:52652 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iYBay-0002cf-4w for incoming@patchwork.ozlabs.org; Fri, 22 Nov 2019 11:15:44 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:47892) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iYBRA-0000SX-WE for qemu-devel@nongnu.org; Fri, 22 Nov 2019 11:05:38 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iYBR9-0002vb-Fy for qemu-devel@nongnu.org; Fri, 22 Nov 2019 11:05:36 -0500 Received: from us-smtp-2.mimecast.com ([207.211.31.81]:47893 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iYBR9-0002u1-BT for qemu-devel@nongnu.org; Fri, 22 Nov 2019 11:05:35 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1574438735; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=XrtsKI6UtYWJu+YSlWZgfDftptplBLsPijshH5RaEuE=; b=XgaymrD3Z64h8BEmso8fL8O9+tH3KZJ4QQoTcLbvIbtcBsRZKRgzeefE7DuyZgDkFu+yfi wdvyul3AjHixgz++DZYSazur3n2fmDCYYE66G5cKXEQsp8V30vMflSKOmOuxhBpjIk/HGg nl7yd5MKkL4YFYX31bSczMTCYnmWcuE= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-371-RjzgS99zNYytX2rf00FBcA-1; Fri, 22 Nov 2019 11:05:31 -0500 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 54D5ADB20; Fri, 22 Nov 2019 16:05:30 +0000 (UTC) Received: from linux.fritz.box.com (ovpn-117-252.ams2.redhat.com [10.36.117.252]) by smtp.corp.redhat.com (Postfix) with ESMTP id AC6B110372C3; Fri, 22 Nov 2019 16:05:28 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Subject: [PATCH v3 5/8] iotests: Add qemu_io_log() Date: Fri, 22 Nov 2019 17:05:08 +0100 Message-Id: <20191122160511.8377-6-kwolf@redhat.com> In-Reply-To: <20191122160511.8377-1-kwolf@redhat.com> References: <20191122160511.8377-1-kwolf@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-MC-Unique: RjzgS99zNYytX2rf00FBcA-1 X-Mimecast-Spam-Score: 0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 207.211.31.81 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, qemu-devel@nongnu.org, mreitz@redhat.com, stefanha@redhat.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Add a function that runs qemu-io and logs the output with the appropriate filters applied. Signed-off-by: Kevin Wolf Reviewed-by: Eric Blake Reviewed-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Alberto Garcia Reviewed-by: Stefan Hajnoczi --- tests/qemu-iotests/iotests.py | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index df0708923d..fc78852ae5 100644 --- a/tests/qemu-iotests/iotests.py +++ b/tests/qemu-iotests/iotests.py @@ -162,6 +162,11 @@ def qemu_io(*args): sys.stderr.write('qemu-io received signal %i: %s\n' % (-exitcode, ' '.join(args))) return subp.communicate()[0] +def qemu_io_log(*args): + result = qemu_io(*args) + log(result, filters=[filter_testfiles, filter_qemu_io]) + return result + def qemu_io_silent(*args): '''Run qemu-io and return the exit code, suppressing stdout''' args = qemu_io_args + list(args) From patchwork Fri Nov 22 16:05:09 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 1199536 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="enU9LrE+"; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 47KLzd17k6z9sPK for ; Sat, 23 Nov 2019 03:12:11 +1100 (AEDT) Received: from localhost ([::1]:52602 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iYBXV-00075A-9d for incoming@patchwork.ozlabs.org; Fri, 22 Nov 2019 11:12:09 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:47894) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iYBRA-0000Si-OC for qemu-devel@nongnu.org; Fri, 22 Nov 2019 11:05:38 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iYBR9-0002v1-9O for qemu-devel@nongnu.org; Fri, 22 Nov 2019 11:05:36 -0500 Received: from us-smtp-delivery-1.mimecast.com ([205.139.110.120]:34450 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iYBR9-0002t4-4A for qemu-devel@nongnu.org; Fri, 22 Nov 2019 11:05:35 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1574438734; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=qzOCGyGUkB6MqvpabN7+rCH0SE/ivBCj0/o+iivJo0c=; b=enU9LrE+9dME1ClHYy3XNRbygYJIERaaFWIRWjl5TMN8StxTBedjkdqiXvcSpMg14fnglH 95SOVH9BDD7BiHPEh/6qAfKaZM9N6DLzoAdJ4O2xVi42U+79jZ7vsC3ywThvJgg3+F6YBi WSFNak4//mTC29enGCfPNSUjCvk05WU= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-96-4jnCmucqPe22hJo8rnc-KQ-1; Fri, 22 Nov 2019 11:05:33 -0500 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 495B7801E74; Fri, 22 Nov 2019 16:05:32 +0000 (UTC) Received: from linux.fritz.box.com (ovpn-117-252.ams2.redhat.com [10.36.117.252]) by smtp.corp.redhat.com (Postfix) with ESMTP id A0E6E100032E; Fri, 22 Nov 2019 16:05:30 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Subject: [PATCH v3 6/8] iotests: Fix timeout in run_job() Date: Fri, 22 Nov 2019 17:05:09 +0100 Message-Id: <20191122160511.8377-7-kwolf@redhat.com> In-Reply-To: <20191122160511.8377-1-kwolf@redhat.com> References: <20191122160511.8377-1-kwolf@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-MC-Unique: 4jnCmucqPe22hJo8rnc-KQ-1 X-Mimecast-Spam-Score: 0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 205.139.110.120 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, qemu-devel@nongnu.org, mreitz@redhat.com, stefanha@redhat.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" run_job() accepts a wait parameter for a timeout, but it doesn't actually use it. The only thing that is missing is passing it to events_wait(), so do that now. Signed-off-by: Kevin Wolf Reviewed-by: Eric Blake Reviewed-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Alberto Garcia Reviewed-by: Stefan Hajnoczi --- tests/qemu-iotests/iotests.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index fc78852ae5..0ac3ad4b04 100644 --- a/tests/qemu-iotests/iotests.py +++ b/tests/qemu-iotests/iotests.py @@ -609,7 +609,7 @@ class VM(qtest.QEMUQtestMachine): ] error = None while True: - ev = filter_qmp_event(self.events_wait(events)) + ev = filter_qmp_event(self.events_wait(events, timeout=wait)) if ev['event'] != 'JOB_STATUS_CHANGE': if use_log: log(ev) From patchwork Fri Nov 22 16:05:10 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 1199544 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="hgzqoMom"; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 47KM2q6Gmwz9sPj for ; Sat, 23 Nov 2019 03:14:59 +1100 (AEDT) Received: from localhost ([::1]:52648 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iYBaD-0001it-OI for incoming@patchwork.ozlabs.org; Fri, 22 Nov 2019 11:14:57 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:47923) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iYBRC-0000WO-Bx for qemu-devel@nongnu.org; Fri, 22 Nov 2019 11:05:39 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iYBRB-0002yx-Ce for qemu-devel@nongnu.org; Fri, 22 Nov 2019 11:05:38 -0500 Received: from us-smtp-1.mimecast.com ([205.139.110.61]:53609 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iYBRB-0002y1-7y for qemu-devel@nongnu.org; Fri, 22 Nov 2019 11:05:37 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1574438736; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=csk/bOJL8RWRBNiVg8ETj1cbdqwbCfGTJZj0fyj26G4=; b=hgzqoMomDhjQN4cwD2CmIDIRa4FNPCAa4NYXrBKorHaDr8639V7qpExSvXpVo7Gkv1vine T6wb2Qwpzp7OYH8polTdGFJuFpRG7BTMv3DdPEynh6gg5TcbkcARXJyBPsEQvsC4MAz8wi kTqPB1XKXrSxug5DET1Ksdgv5kQ3oK0= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-72-bca1ptYpO66UqX1Mjuu-CA-1; Fri, 22 Nov 2019 11:05:35 -0500 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 43A888018A3; Fri, 22 Nov 2019 16:05:34 +0000 (UTC) Received: from linux.fritz.box.com (ovpn-117-252.ams2.redhat.com [10.36.117.252]) by smtp.corp.redhat.com (Postfix) with ESMTP id 98620100032E; Fri, 22 Nov 2019 16:05:32 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Subject: [PATCH v3 7/8] iotests: Support job-complete in run_job() Date: Fri, 22 Nov 2019 17:05:10 +0100 Message-Id: <20191122160511.8377-8-kwolf@redhat.com> In-Reply-To: <20191122160511.8377-1-kwolf@redhat.com> References: <20191122160511.8377-1-kwolf@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-MC-Unique: bca1ptYpO66UqX1Mjuu-CA-1 X-Mimecast-Spam-Score: 0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 205.139.110.61 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, qemu-devel@nongnu.org, mreitz@redhat.com, stefanha@redhat.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Automatically complete jobs that have a 'ready' state and need an explicit job-complete. Without this, run_job() would hang for such jobs. Signed-off-by: Kevin Wolf Reviewed-by: Eric Blake Reviewed-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Alberto Garcia Reviewed-by: Stefan Hajnoczi --- tests/qemu-iotests/iotests.py | 2 ++ 1 file changed, 2 insertions(+) diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index 0ac3ad4b04..b46d298766 100644 --- a/tests/qemu-iotests/iotests.py +++ b/tests/qemu-iotests/iotests.py @@ -622,6 +622,8 @@ class VM(qtest.QEMUQtestMachine): error = j['error'] if use_log: log('Job failed: %s' % (j['error'])) + elif status == 'ready': + self.qmp_log('job-complete', id=job) elif status == 'pending' and not auto_finalize: if pre_finalize: pre_finalize() From patchwork Fri Nov 22 16:05:11 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 1199534 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="hWPi8Xqf"; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 47KLxp39x2z9sPK for ; Sat, 23 Nov 2019 03:10:38 +1100 (AEDT) Received: from localhost ([::1]:52594 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iYBW0-00062f-5G for incoming@patchwork.ozlabs.org; Fri, 22 Nov 2019 11:10:36 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:47984) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iYBRK-0000i7-OZ for qemu-devel@nongnu.org; Fri, 22 Nov 2019 11:05:49 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iYBRI-00033y-1t for qemu-devel@nongnu.org; Fri, 22 Nov 2019 11:05:46 -0500 Received: from us-smtp-delivery-1.mimecast.com ([207.211.31.120]:50360 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iYBRH-00033k-Ss for qemu-devel@nongnu.org; Fri, 22 Nov 2019 11:05:44 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1574438743; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Ga8B00g8JNq3CSgfdmfkCSpMQ1QT4hQbdnfMRvbR3yQ=; b=hWPi8XqfEhg8DdZDJDzYZfVrGYP2dmMRcldNT3JlP90XbqPBKKMqVGWweyV7mpkJnCz+8+ qSCwC5RFRRdtRrL7EsH3Int9CUU/CXxCgO8P+q6ADNTPoC4iLD5H3Gqmlb4RWvcXcavA1f iNA4jAIwE0ILtOHcUX1U8J1e6yIyqvg= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-181-QGbqLggoN7ur92S3ViHB_w-1; Fri, 22 Nov 2019 11:05:40 -0500 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 0B1FA8024C0; Fri, 22 Nov 2019 16:05:39 +0000 (UTC) Received: from linux.fritz.box.com (ovpn-117-252.ams2.redhat.com [10.36.117.252]) by smtp.corp.redhat.com (Postfix) with ESMTP id 8F602100032E; Fri, 22 Nov 2019 16:05:34 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Subject: [PATCH v3 8/8] iotests: Test committing to short backing file Date: Fri, 22 Nov 2019 17:05:11 +0100 Message-Id: <20191122160511.8377-9-kwolf@redhat.com> In-Reply-To: <20191122160511.8377-1-kwolf@redhat.com> References: <20191122160511.8377-1-kwolf@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-MC-Unique: QGbqLggoN7ur92S3ViHB_w-1 X-Mimecast-Spam-Score: 0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 207.211.31.120 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, qemu-devel@nongnu.org, mreitz@redhat.com, stefanha@redhat.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Kevin Wolf Reviewed-by: Eric Blake Reviewed-by: Vladimir Sementsov-Ogievskiy Tested-by: Vladimir Sementsov-Ogievskiy --- tests/qemu-iotests/274 | 152 +++++++++++++++++++++++++ tests/qemu-iotests/274.out | 203 ++++++++++++++++++++++++++++++++++ tests/qemu-iotests/group | 1 + tests/qemu-iotests/iotests.py | 2 +- 4 files changed, 357 insertions(+), 1 deletion(-) create mode 100755 tests/qemu-iotests/274 create mode 100644 tests/qemu-iotests/274.out diff --git a/tests/qemu-iotests/274 b/tests/qemu-iotests/274 new file mode 100755 index 0000000000..7b238f41da --- /dev/null +++ b/tests/qemu-iotests/274 @@ -0,0 +1,152 @@ +#!/usr/bin/env python +# +# Copyright (C) 2019 Red Hat, Inc. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . +# +# Creator/Owner: Kevin Wolf +# +# Some tests for short backing files and short overlays + +import iotests +import os + +iotests.verify_image_format(supported_fmts=['qcow2']) +iotests.verify_platform(['linux']) + +size_short = 1 * 1024 * 1024 +size_long = 2 * 1024 * 1024 +size_diff = size_long - size_short + +def create_chain(): + iotests.qemu_img_log('create', '-f', iotests.imgfmt, base, + str(size_long)) + iotests.qemu_img_log('create', '-f', iotests.imgfmt, '-b', base, mid, + str(size_short)) + iotests.qemu_img_log('create', '-f', iotests.imgfmt, '-b', mid, top, + str(size_long)) + + iotests.qemu_io_log('-c', 'write -P 1 0 %d' % size_long, base) + +def create_vm(): + vm = iotests.VM() + vm.add_blockdev('file,filename=%s,node-name=base-file' % (base)) + vm.add_blockdev('%s,file=base-file,node-name=base' % (iotests.imgfmt)) + vm.add_blockdev('file,filename=%s,node-name=mid-file' % (mid)) + vm.add_blockdev('%s,file=mid-file,node-name=mid,backing=base' % (iotests.imgfmt)) + vm.add_drive(top, 'backing=mid,node-name=top') + return vm + +with iotests.FilePath('base') as base, \ + iotests.FilePath('mid') as mid, \ + iotests.FilePath('top') as top: + + iotests.log('== Commit tests ==') + + create_chain() + + iotests.log('=== Check visible data ===') + + iotests.qemu_io_log('-c', 'read -P 1 0 %d' % size_short, top) + iotests.qemu_io_log('-c', 'read -P 0 %d %d' % (size_short, size_diff), top) + + iotests.log('=== Checking allocation status ===') + + iotests.qemu_io_log('-c', 'alloc 0 %d' % size_short, + '-c', 'alloc %d %d' % (size_short, size_diff), + base) + + iotests.qemu_io_log('-c', 'alloc 0 %d' % size_short, + '-c', 'alloc %d %d' % (size_short, size_diff), + mid) + + iotests.qemu_io_log('-c', 'alloc 0 %d' % size_short, + '-c', 'alloc %d %d' % (size_short, size_diff), + top) + + iotests.log('=== Checking map ===') + + iotests.qemu_img_log('map', '--output=json', base) + iotests.qemu_img_log('map', '--output=human', base) + iotests.qemu_img_log('map', '--output=json', mid) + iotests.qemu_img_log('map', '--output=human', mid) + iotests.qemu_img_log('map', '--output=json', top) + iotests.qemu_img_log('map', '--output=human', top) + + iotests.log('=== Testing qemu-img commit (top -> mid) ===') + + iotests.qemu_img_log('commit', top) + iotests.img_info_log(mid) + iotests.qemu_io_log('-c', 'read -P 1 0 %d' % size_short, mid) + iotests.qemu_io_log('-c', 'read -P 0 %d %d' % (size_short, size_diff), mid) + + iotests.log('=== Testing HMP commit (top -> mid) ===') + + create_chain() + with create_vm() as vm: + vm.launch() + vm.qmp_log('human-monitor-command', command_line='commit drive0') + + iotests.img_info_log(mid) + iotests.qemu_io_log('-c', 'read -P 1 0 %d' % size_short, mid) + iotests.qemu_io_log('-c', 'read -P 0 %d %d' % (size_short, size_diff), mid) + + iotests.log('=== Testing QMP active commit (top -> mid) ===') + + create_chain() + with create_vm() as vm: + vm.launch() + vm.qmp_log('block-commit', device='top', base_node='mid', + job_id='job0', auto_dismiss=False) + vm.run_job('job0', wait=5) + + iotests.img_info_log(mid) + iotests.qemu_io_log('-c', 'read -P 1 0 %d' % size_short, mid) + iotests.qemu_io_log('-c', 'read -P 0 %d %d' % (size_short, size_diff), mid) + + + iotests.log('== Resize tests ==') + + # Use different sizes for different allocation modes: + # + # We want to have at least one test where 32 bit truncation in the size of + # the overlapping area becomes visible. This is covered by the + # prealloc='off' case (1G to 6G is an overlap of 5G). + # + # However, we can only do this for modes that don't preallocate data + # because otherwise we might run out of space on the test host. + for (prealloc, base_size, top_size_old, top_size_new, off) in [ + ('off', '6G', '1G', '8G', '5G'), + ('metadata', '32G', '30G', '33G', '31G'), + ('falloc', '10M', '5M', '15M', '9M'), + ('full', '16M', '8M', '12M', '11M')]: + + iotests.log('=== preallocation=%s ===' % prealloc) + iotests.qemu_img_log('create', '-f', iotests.imgfmt, base, base_size) + iotests.qemu_img_log('create', '-f', iotests.imgfmt, '-b', base, top, + top_size_old) + iotests.qemu_io_log('-c', 'write -P 1 %s 64k' % off, base) + + # After this, 0 to base_size should be allocated/zeroed + # base_size to top_size_new should be unallocated with + # preallocation=off and allocated with preallocation enabled + iotests.qemu_img_log('resize', '-f', iotests.imgfmt, + '--preallocation', prealloc, top, top_size_new) + iotests.qemu_io_log('-c', 'read -P 0 %s 64k' % off, top) + + # Metadata preallocation doesn't have a defined result on the file + # system level with respect to holes, so skip it here + iotests.qemu_io_log('-c', 'map', top) + if prealloc != 'metadata': + iotests.qemu_img_log('map', '--output=json', top) diff --git a/tests/qemu-iotests/274.out b/tests/qemu-iotests/274.out new file mode 100644 index 0000000000..b18a69fd42 --- /dev/null +++ b/tests/qemu-iotests/274.out @@ -0,0 +1,203 @@ +== Commit tests == +Formatting 'TEST_DIR/PID-base', fmt=qcow2 size=2097152 cluster_size=65536 lazy_refcounts=off refcount_bits=16 + +Formatting 'TEST_DIR/PID-mid', fmt=qcow2 size=1048576 backing_file=TEST_DIR/PID-base cluster_size=65536 lazy_refcounts=off refcount_bits=16 + +Formatting 'TEST_DIR/PID-top', fmt=qcow2 size=2097152 backing_file=TEST_DIR/PID-mid cluster_size=65536 lazy_refcounts=off refcount_bits=16 + +wrote 2097152/2097152 bytes at offset 0 +2 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +=== Check visible data === +read 1048576/1048576 bytes at offset 0 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +read 1048576/1048576 bytes at offset 1048576 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +=== Checking allocation status === +1048576/1048576 bytes allocated at offset 0 bytes +1048576/1048576 bytes allocated at offset 1 MiB + +0/1048576 bytes allocated at offset 0 bytes +0/0 bytes allocated at offset 1 MiB + +0/1048576 bytes allocated at offset 0 bytes +0/1048576 bytes allocated at offset 1 MiB + +=== Checking map === +[{ "start": 0, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": 327680}] + +Offset Length Mapped to File +0 0x200000 0x50000 TEST_DIR/PID-base + +[{ "start": 0, "length": 1048576, "depth": 1, "zero": false, "data": true, "offset": 327680}] + +Offset Length Mapped to File +0 0x100000 0x50000 TEST_DIR/PID-base + +[{ "start": 0, "length": 1048576, "depth": 2, "zero": false, "data": true, "offset": 327680}, +{ "start": 1048576, "length": 1048576, "depth": 0, "zero": true, "data": false}] + +Offset Length Mapped to File +0 0x100000 0x50000 TEST_DIR/PID-base + +=== Testing qemu-img commit (top -> mid) === +Image committed. + +image: TEST_IMG +file format: IMGFMT +virtual size: 2 MiB (2097152 bytes) +cluster_size: 65536 +backing file: TEST_DIR/PID-base +Format specific information: + compat: 1.1 + lazy refcounts: false + refcount bits: 16 + corrupt: false + +read 1048576/1048576 bytes at offset 0 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +read 1048576/1048576 bytes at offset 1048576 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +=== Testing HMP commit (top -> mid) === +Formatting 'TEST_DIR/PID-base', fmt=qcow2 size=2097152 cluster_size=65536 lazy_refcounts=off refcount_bits=16 + +Formatting 'TEST_DIR/PID-mid', fmt=qcow2 size=1048576 backing_file=TEST_DIR/PID-base cluster_size=65536 lazy_refcounts=off refcount_bits=16 + +Formatting 'TEST_DIR/PID-top', fmt=qcow2 size=2097152 backing_file=TEST_DIR/PID-mid cluster_size=65536 lazy_refcounts=off refcount_bits=16 + +wrote 2097152/2097152 bytes at offset 0 +2 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +{"execute": "human-monitor-command", "arguments": {"command-line": "commit drive0"}} +{"return": ""} +image: TEST_IMG +file format: IMGFMT +virtual size: 2 MiB (2097152 bytes) +cluster_size: 65536 +backing file: TEST_DIR/PID-base +Format specific information: + compat: 1.1 + lazy refcounts: false + refcount bits: 16 + corrupt: false + +read 1048576/1048576 bytes at offset 0 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +read 1048576/1048576 bytes at offset 1048576 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +=== Testing QMP active commit (top -> mid) === +Formatting 'TEST_DIR/PID-base', fmt=qcow2 size=2097152 cluster_size=65536 lazy_refcounts=off refcount_bits=16 + +Formatting 'TEST_DIR/PID-mid', fmt=qcow2 size=1048576 backing_file=TEST_DIR/PID-base cluster_size=65536 lazy_refcounts=off refcount_bits=16 + +Formatting 'TEST_DIR/PID-top', fmt=qcow2 size=2097152 backing_file=TEST_DIR/PID-mid cluster_size=65536 lazy_refcounts=off refcount_bits=16 + +wrote 2097152/2097152 bytes at offset 0 +2 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +{"execute": "block-commit", "arguments": {"auto-dismiss": false, "base-node": "mid", "device": "top", "job-id": "job0"}} +{"return": {}} +{"execute": "job-complete", "arguments": {"id": "job0"}} +{"return": {}} +{"data": {"device": "job0", "len": 0, "offset": 0, "speed": 0, "type": "commit"}, "event": "BLOCK_JOB_READY", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}} +{"data": {"device": "job0", "len": 0, "offset": 0, "speed": 0, "type": "commit"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}} +{"execute": "job-dismiss", "arguments": {"id": "job0"}} +{"return": {}} +image: TEST_IMG +file format: IMGFMT +virtual size: 2 MiB (2097152 bytes) +cluster_size: 65536 +backing file: TEST_DIR/PID-base +Format specific information: + compat: 1.1 + lazy refcounts: false + refcount bits: 16 + corrupt: false + +read 1048576/1048576 bytes at offset 0 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +read 1048576/1048576 bytes at offset 1048576 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +== Resize tests == +=== preallocation=off === +Formatting 'TEST_DIR/PID-base', fmt=qcow2 size=6442450944 cluster_size=65536 lazy_refcounts=off refcount_bits=16 + +Formatting 'TEST_DIR/PID-top', fmt=qcow2 size=1073741824 backing_file=TEST_DIR/PID-base cluster_size=65536 lazy_refcounts=off refcount_bits=16 + +wrote 65536/65536 bytes at offset 5368709120 +64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +Image resized. + +read 65536/65536 bytes at offset 5368709120 +64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +1 GiB (0x40000000) bytes not allocated at offset 0 bytes (0x0) +5 GiB (0x140000000) bytes allocated at offset 1 GiB (0x40000000) +2 GiB (0x80000000) bytes not allocated at offset 6 GiB (0x180000000) + +[{ "start": 0, "length": 1073741824, "depth": 1, "zero": true, "data": false}, +{ "start": 1073741824, "length": 7516192768, "depth": 0, "zero": true, "data": false}] + +=== preallocation=metadata === +Formatting 'TEST_DIR/PID-base', fmt=qcow2 size=34359738368 cluster_size=65536 lazy_refcounts=off refcount_bits=16 + +Formatting 'TEST_DIR/PID-top', fmt=qcow2 size=32212254720 backing_file=TEST_DIR/PID-base cluster_size=65536 lazy_refcounts=off refcount_bits=16 + +wrote 65536/65536 bytes at offset 33285996544 +64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +Image resized. + +read 65536/65536 bytes at offset 33285996544 +64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +30 GiB (0x780000000) bytes not allocated at offset 0 bytes (0x0) +3 GiB (0xc0000000) bytes allocated at offset 30 GiB (0x780000000) + +=== preallocation=falloc === +Formatting 'TEST_DIR/PID-base', fmt=qcow2 size=10485760 cluster_size=65536 lazy_refcounts=off refcount_bits=16 + +Formatting 'TEST_DIR/PID-top', fmt=qcow2 size=5242880 backing_file=TEST_DIR/PID-base cluster_size=65536 lazy_refcounts=off refcount_bits=16 + +wrote 65536/65536 bytes at offset 9437184 +64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +Image resized. + +read 65536/65536 bytes at offset 9437184 +64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +5 MiB (0x500000) bytes not allocated at offset 0 bytes (0x0) +10 MiB (0xa00000) bytes allocated at offset 5 MiB (0x500000) + +[{ "start": 0, "length": 5242880, "depth": 1, "zero": true, "data": false}, +{ "start": 5242880, "length": 10485760, "depth": 0, "zero": false, "data": true, "offset": 327680}] + +=== preallocation=full === +Formatting 'TEST_DIR/PID-base', fmt=qcow2 size=16777216 cluster_size=65536 lazy_refcounts=off refcount_bits=16 + +Formatting 'TEST_DIR/PID-top', fmt=qcow2 size=8388608 backing_file=TEST_DIR/PID-base cluster_size=65536 lazy_refcounts=off refcount_bits=16 + +wrote 65536/65536 bytes at offset 11534336 +64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +Image resized. + +read 65536/65536 bytes at offset 11534336 +64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +8 MiB (0x800000) bytes not allocated at offset 0 bytes (0x0) +4 MiB (0x400000) bytes allocated at offset 8 MiB (0x800000) + +[{ "start": 0, "length": 8388608, "depth": 1, "zero": true, "data": false}, +{ "start": 8388608, "length": 4194304, "depth": 0, "zero": false, "data": true, "offset": 327680}] + diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group index 6b10a6a762..b377a17e78 100644 --- a/tests/qemu-iotests/group +++ b/tests/qemu-iotests/group @@ -285,4 +285,5 @@ 270 rw backing quick 272 rw 273 backing quick +274 rw backing 277 rw quick diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index b46d298766..9135dd52b6 100644 --- a/tests/qemu-iotests/iotests.py +++ b/tests/qemu-iotests/iotests.py @@ -149,7 +149,7 @@ def img_info_log(filename, filter_path=None, imgopts=False, extra_args=[]): output = qemu_img_pipe(*args) if not filter_path: filter_path = filename - log(filter_img_info(output, filter_path)) + log(filter_img_info(output, filter_path), filters=[filter_testfiles]) def qemu_io(*args): '''Run qemu-io and return the stdout data'''