From patchwork Wed Feb 27 17:22:37 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 1049027 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 448jGS0Wz2z9sBF for ; Thu, 28 Feb 2019 04:24:14 +1100 (AEDT) Received: from localhost ([127.0.0.1]:47882 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gz2wG-0002xa-Mi for incoming@patchwork.ozlabs.org; Wed, 27 Feb 2019 12:24:12 -0500 Received: from eggs.gnu.org ([209.51.188.92]:51189) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gz2vY-0002wS-0Y for qemu-devel@nongnu.org; Wed, 27 Feb 2019 12:23:28 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gz2vM-0002x1-2E for qemu-devel@nongnu.org; Wed, 27 Feb 2019 12:23:19 -0500 Received: from mx1.redhat.com ([209.132.183.28]:59204) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1gz2vG-0002tQ-Cl; Wed, 27 Feb 2019 12:23:10 -0500 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 55A2C78EA4; Wed, 27 Feb 2019 17:23:09 +0000 (UTC) Received: from linux.fritz.box.com (ovpn-117-19.ams2.redhat.com [10.36.117.19]) by smtp.corp.redhat.com (Postfix) with ESMTP id 85DF3171A8; Wed, 27 Feb 2019 17:23:05 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Date: Wed, 27 Feb 2019 18:22:37 +0100 Message-Id: <20190227172256.30368-2-kwolf@redhat.com> In-Reply-To: <20190227172256.30368-1-kwolf@redhat.com> References: <20190227172256.30368-1-kwolf@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.27]); Wed, 27 Feb 2019 17:23:09 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH 01/20] qemu-iotests: Test qcow2 preallocation modes X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org, mreitz@redhat.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Kevin Wolf Reviewed-by: Eric Blake --- tests/qemu-iotests/243 | 63 ++++++++++++++++++++++++++++++++++++++ tests/qemu-iotests/243.out | 26 ++++++++++++++++ tests/qemu-iotests/group | 1 + 3 files changed, 90 insertions(+) create mode 100755 tests/qemu-iotests/243 create mode 100644 tests/qemu-iotests/243.out diff --git a/tests/qemu-iotests/243 b/tests/qemu-iotests/243 new file mode 100755 index 0000000000..5ab1b7863d --- /dev/null +++ b/tests/qemu-iotests/243 @@ -0,0 +1,63 @@ +#!/bin/bash +# +# Test qcow2 preallocation +# +# Copyright (C) 2019 Red Hat, Inc. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . +# + +# creator +owner=kwolf@redhat.com + +seq=$(basename $0) +echo "QA output created by $seq" + +status=1 # failure is the default! + +_cleanup() +{ + _cleanup_test_img +} +trap "_cleanup; exit \$status" 0 1 2 3 15 + +# get standard environment, filters and checks +. ./common.rc +. ./common.filter + +_supported_fmt qcow2 +_supported_proto file +_supported_os Linux + +for mode in off metadata falloc full; do + + echo + echo "=== preallocation=$mode ===" + echo + + IMGOPTS="preallocation=$mode" _make_test_img 64M + + echo -n "File size: " + du -b $TEST_IMG | cut -f1 + + # Can't use precise numbers here because they differ between filesystems + echo -n "Disk usage: " + [ $(du -B1 $TEST_IMG | cut -f1) -lt 1048576 ] && echo "low" || echo "high" + +done + +# success, all done +echo "*** done" +rm -f $seq.full +status=0 diff --git a/tests/qemu-iotests/243.out b/tests/qemu-iotests/243.out new file mode 100644 index 0000000000..e531753ad1 --- /dev/null +++ b/tests/qemu-iotests/243.out @@ -0,0 +1,26 @@ +QA output created by 243 + +=== preallocation=off === + +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 preallocation=off +File size: 196616 +Disk usage: low + +=== preallocation=metadata === + +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 preallocation=metadata +File size: 67436544 +Disk usage: low + +=== preallocation=falloc === + +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 preallocation=falloc +File size: 67436544 +Disk usage: high + +=== preallocation=full === + +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 preallocation=full +File size: 67436544 +Disk usage: high +*** done diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group index b5ca63cf72..03aa93dbf5 100644 --- a/tests/qemu-iotests/group +++ b/tests/qemu-iotests/group @@ -241,3 +241,4 @@ 239 rw auto quick 240 auto quick 242 rw auto quick +243 rw auto quick From patchwork Wed Feb 27 17:22:38 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 1049040 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 448jQR6mgfz9s70 for ; Thu, 28 Feb 2019 04:31:11 +1100 (AEDT) Received: from localhost ([127.0.0.1]:48023 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gz32z-0000Dp-PP for incoming@patchwork.ozlabs.org; Wed, 27 Feb 2019 12:31:09 -0500 Received: from eggs.gnu.org ([209.51.188.92]:51501) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gz2wT-0003fX-PQ for qemu-devel@nongnu.org; Wed, 27 Feb 2019 12:24:29 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gz2wO-0003et-6z for qemu-devel@nongnu.org; Wed, 27 Feb 2019 12:24:25 -0500 Received: from mx1.redhat.com ([209.132.183.28]:55282) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1gz2wC-0002vh-4A; Wed, 27 Feb 2019 12:24:08 -0500 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 8788C30224BE; Wed, 27 Feb 2019 17:23:12 +0000 (UTC) Received: from linux.fritz.box.com (ovpn-117-19.ams2.redhat.com [10.36.117.19]) by smtp.corp.redhat.com (Postfix) with ESMTP id A2ACC171A8; Wed, 27 Feb 2019 17:23:09 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Date: Wed, 27 Feb 2019 18:22:38 +0100 Message-Id: <20190227172256.30368-3-kwolf@redhat.com> In-Reply-To: <20190227172256.30368-1-kwolf@redhat.com> References: <20190227172256.30368-1-kwolf@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.49]); Wed, 27 Feb 2019 17:23:12 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH 02/20] qcow2: Simplify preallocation code X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org, mreitz@redhat.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Image creation already involves a bdrv_co_truncate() call, which allows to specify a preallocation mode. Just pass the right mode there and remove the code that is made redundant by this. Signed-off-by: Kevin Wolf --- block/qcow2.c | 28 +--------------------------- 1 file changed, 1 insertion(+), 27 deletions(-) diff --git a/block/qcow2.c b/block/qcow2.c index 1de5e24613..e38041e817 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -2940,19 +2940,6 @@ qcow2_co_create(BlockdevCreateOptions *create_options, Error **errp) goto out; } - if (qcow2_opts->preallocation == PREALLOC_MODE_FULL || - qcow2_opts->preallocation == PREALLOC_MODE_FALLOC) - { - int64_t prealloc_size = - qcow2_calc_prealloc_size(qcow2_opts->size, cluster_size, - refcount_order); - - ret = blk_truncate(blk, prealloc_size, qcow2_opts->preallocation, errp); - if (ret < 0) { - goto out; - } - } - /* Write the header */ QEMU_BUILD_BUG_ON((1 << MIN_CLUSTER_BITS) < sizeof(*header)); header = g_malloc0(cluster_size); @@ -3034,7 +3021,7 @@ qcow2_co_create(BlockdevCreateOptions *create_options, Error **errp) } /* Okay, now that we have a valid image, let's give it the right size */ - ret = blk_truncate(blk, qcow2_opts->size, PREALLOC_MODE_OFF, errp); + ret = blk_truncate(blk, qcow2_opts->size, qcow2_opts->preallocation, errp); if (ret < 0) { error_prepend(errp, "Could not resize image: "); goto out; @@ -3066,19 +3053,6 @@ qcow2_co_create(BlockdevCreateOptions *create_options, Error **errp) } } - /* And if we're supposed to preallocate metadata, do that now */ - if (qcow2_opts->preallocation != PREALLOC_MODE_OFF) { - BDRVQcow2State *s = blk_bs(blk)->opaque; - qemu_co_mutex_lock(&s->lock); - ret = preallocate_co(blk_bs(blk), 0, qcow2_opts->size); - qemu_co_mutex_unlock(&s->lock); - - if (ret < 0) { - error_setg_errno(errp, -ret, "Could not preallocate metadata"); - goto out; - } - } - blk_unref(blk); blk = NULL; From patchwork Wed Feb 27 17:22:39 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 1049029 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 448jHG4ZTHz9sBF for ; Thu, 28 Feb 2019 04:24:58 +1100 (AEDT) Received: from localhost ([127.0.0.1]:47888 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gz2wy-0003I7-DV for incoming@patchwork.ozlabs.org; Wed, 27 Feb 2019 12:24:56 -0500 Received: from eggs.gnu.org ([209.51.188.92]:51249) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gz2vq-00034K-4V for qemu-devel@nongnu.org; Wed, 27 Feb 2019 12:23:46 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gz2vp-0003Ir-6W for qemu-devel@nongnu.org; Wed, 27 Feb 2019 12:23:46 -0500 Received: from mx1.redhat.com ([209.132.183.28]:45572) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1gz2vN-0002wB-T2; Wed, 27 Feb 2019 12:23:23 -0500 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 0FB2430BCE63; Wed, 27 Feb 2019 17:23:14 +0000 (UTC) Received: from linux.fritz.box.com (ovpn-117-19.ams2.redhat.com [10.36.117.19]) by smtp.corp.redhat.com (Postfix) with ESMTP id CB05B171A0; Wed, 27 Feb 2019 17:23:12 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Date: Wed, 27 Feb 2019 18:22:39 +0100 Message-Id: <20190227172256.30368-4-kwolf@redhat.com> In-Reply-To: <20190227172256.30368-1-kwolf@redhat.com> References: <20190227172256.30368-1-kwolf@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.43]); Wed, 27 Feb 2019 17:23:14 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH 03/20] qcow2: Extend spec for external data files X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org, mreitz@redhat.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" This adds external data file to the qcow2 spec as a new incompatible feature. Signed-off-by: Kevin Wolf --- docs/interop/qcow2.txt | 38 ++++++++++++++++++++++++++++++++++---- 1 file changed, 34 insertions(+), 4 deletions(-) diff --git a/docs/interop/qcow2.txt b/docs/interop/qcow2.txt index fb5cb47245..1c1849dc26 100644 --- a/docs/interop/qcow2.txt +++ b/docs/interop/qcow2.txt @@ -97,7 +97,19 @@ in the description of a field. be written to (unless for regaining consistency). - Bits 2-63: Reserved (set to 0) + Bit 2: External data file bit. If this bit is set, an + external data file is used. Guest clusters are + then stored in the external data file. For such + images, clusters in the external data file are + not refcounted. The offset field in the + Standard Cluster Descriptor must match the + guest offset and neither compressed clusters + nor internal snapshots are supported. + + An external data file name header extension may + be present if this bit is set. + + Bits 3-63: Reserved (set to 0) 80 - 87: compatible_features Bitmask of compatible features. An implementation can @@ -126,7 +138,17 @@ in the description of a field. bit is unset, the bitmaps extension data must be considered inconsistent. - Bits 1-63: Reserved (set to 0) + Bit 1: If this bit is set, the external data file can + be read as a consistent standalone raw image + without looking at the qcow2 metadata. + + Setting this bit has a performance impact for + some operations on the image (e.g. writing + zeros requires writing to the data file instead + of only setting the zero flag in the L2 table + entry) and conflicts with backing files. + + Bits 2-63: Reserved (set to 0) 96 - 99: refcount_order Describes the width of a reference count block entry (width @@ -148,6 +170,7 @@ be stored. Each extension has a structure like the following: 0x6803f857 - Feature name table 0x23852875 - Bitmaps extension 0x0537be77 - Full disk encryption header pointer + 0x44415441 - External data file name other - Unknown header extension, can be safely ignored @@ -437,6 +460,11 @@ L2 table entry: This information is only accurate in L2 tables that are reachable from the active L1 table. + With external data files, all guest clusters have an + implicit refcount of 1 (because of the fixed host = guest + mapping for guest cluster offsets), so this bit should be 1 + for all allocated clusters. + Standard Cluster Descriptor: Bit 0: If set to 1, the cluster reads as all zeros. The host @@ -450,8 +478,10 @@ Standard Cluster Descriptor: 1 - 8: Reserved (set to 0) 9 - 55: Bits 9-55 of host cluster offset. Must be aligned to a - cluster boundary. If the offset is 0, the cluster is - unallocated. + cluster boundary. If the offset is 0 and bit 63 is clear, + the cluster is unallocated. The offset may only be 0 with + bit 63 set (indicating a host cluster offset of 0) when an + external data file is used. 56 - 61: Reserved (set to 0) From patchwork Wed Feb 27 17:22:40 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 1049074 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 448jj00mn5z9s4Y for ; Thu, 28 Feb 2019 04:43:46 +1100 (AEDT) Received: from localhost ([127.0.0.1]:48222 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gz3FA-000157-A7 for incoming@patchwork.ozlabs.org; Wed, 27 Feb 2019 12:43:44 -0500 Received: from eggs.gnu.org ([209.51.188.92]:51786) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gz2wf-0003s9-CQ for qemu-devel@nongnu.org; Wed, 27 Feb 2019 12:24:39 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gz2wa-0003uu-UY for qemu-devel@nongnu.org; Wed, 27 Feb 2019 12:24:36 -0500 Received: from mx1.redhat.com ([209.132.183.28]:35866) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1gz2wL-0002yA-UK; Wed, 27 Feb 2019 12:24:23 -0500 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 0058AC00735B; Wed, 27 Feb 2019 17:23:17 +0000 (UTC) Received: from linux.fritz.box.com (ovpn-117-19.ams2.redhat.com [10.36.117.19]) by smtp.corp.redhat.com (Postfix) with ESMTP id 5C9C517153; Wed, 27 Feb 2019 17:23:14 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Date: Wed, 27 Feb 2019 18:22:40 +0100 Message-Id: <20190227172256.30368-5-kwolf@redhat.com> In-Reply-To: <20190227172256.30368-1-kwolf@redhat.com> References: <20190227172256.30368-1-kwolf@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.32]); Wed, 27 Feb 2019 17:23:17 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH 04/20] qcow2: Basic definitions for external data files X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org, mreitz@redhat.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" This adds basic constants, struct fields and helper function for external data file support to the implementation. QCOW2_INCOMPAT_MASK and QCOW2_AUTOCLEAR_MASK are not updated yet so that opening images with an external data file still fails (we don't handle them correctly yet). Signed-off-by: Kevin Wolf --- block/qcow2.h | 32 ++++++++++++++++++++++---------- block/qcow2.c | 9 +++++++++ tests/qemu-iotests/031.out | 8 ++++---- tests/qemu-iotests/036.out | 4 ++-- tests/qemu-iotests/061.out | 14 +++++++------- 5 files changed, 44 insertions(+), 23 deletions(-) diff --git a/block/qcow2.h b/block/qcow2.h index 9dd02df831..c63c3959f7 100644 --- a/block/qcow2.h +++ b/block/qcow2.h @@ -197,13 +197,15 @@ enum { /* Incompatible feature bits */ enum { - QCOW2_INCOMPAT_DIRTY_BITNR = 0, - QCOW2_INCOMPAT_CORRUPT_BITNR = 1, - QCOW2_INCOMPAT_DIRTY = 1 << QCOW2_INCOMPAT_DIRTY_BITNR, - QCOW2_INCOMPAT_CORRUPT = 1 << QCOW2_INCOMPAT_CORRUPT_BITNR, - - QCOW2_INCOMPAT_MASK = QCOW2_INCOMPAT_DIRTY - | QCOW2_INCOMPAT_CORRUPT, + QCOW2_INCOMPAT_DIRTY_BITNR = 0, + QCOW2_INCOMPAT_CORRUPT_BITNR = 1, + QCOW2_INCOMPAT_DATA_FILE_BITNR = 2, + QCOW2_INCOMPAT_DIRTY = 1 << QCOW2_INCOMPAT_DIRTY_BITNR, + QCOW2_INCOMPAT_CORRUPT = 1 << QCOW2_INCOMPAT_CORRUPT_BITNR, + QCOW2_INCOMPAT_DATA_FILE = 1 << QCOW2_INCOMPAT_DATA_FILE_BITNR, + + QCOW2_INCOMPAT_MASK = QCOW2_INCOMPAT_DIRTY + | QCOW2_INCOMPAT_CORRUPT, }; /* Compatible feature bits */ @@ -216,10 +218,12 @@ enum { /* Autoclear feature bits */ enum { - QCOW2_AUTOCLEAR_BITMAPS_BITNR = 0, - QCOW2_AUTOCLEAR_BITMAPS = 1 << QCOW2_AUTOCLEAR_BITMAPS_BITNR, + QCOW2_AUTOCLEAR_BITMAPS_BITNR = 0, + QCOW2_AUTOCLEAR_DATA_FILE_RAW_BITNR = 1, + QCOW2_AUTOCLEAR_BITMAPS = 1 << QCOW2_AUTOCLEAR_BITMAPS_BITNR, + QCOW2_AUTOCLEAR_DATA_FILE_RAW = 1 << QCOW2_AUTOCLEAR_DATA_FILE_RAW_BITNR, - QCOW2_AUTOCLEAR_MASK = QCOW2_AUTOCLEAR_BITMAPS, + QCOW2_AUTOCLEAR_MASK = QCOW2_AUTOCLEAR_BITMAPS, }; enum qcow2_discard_type { @@ -340,6 +344,8 @@ typedef struct BDRVQcow2State { CoQueue compress_wait_queue; int nb_compress_threads; + + BdrvChild *data_file; } BDRVQcow2State; typedef struct Qcow2COWRegion { @@ -457,6 +463,12 @@ typedef enum QCow2MetadataOverlap { #define REFT_OFFSET_MASK 0xfffffffffffffe00ULL +static inline bool has_data_file(BlockDriverState *bs) +{ + BDRVQcow2State *s = bs->opaque; + return (s->data_file != bs->file); +} + static inline int64_t start_of_cluster(BDRVQcow2State *s, int64_t offset) { return offset & ~(s->cluster_size - 1); diff --git a/block/qcow2.c b/block/qcow2.c index e38041e817..c2e3a31d1d 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -73,6 +73,7 @@ typedef struct { #define QCOW2_EXT_MAGIC_FEATURE_TABLE 0x6803f857 #define QCOW2_EXT_MAGIC_CRYPTO_HEADER 0x0537be77 #define QCOW2_EXT_MAGIC_BITMAPS 0x23852875 +#define QCOW2_EXT_MAGIC_DATA_FILE 0x44415441 static int coroutine_fn qcow2_co_preadv_compressed(BlockDriverState *bs, @@ -1440,6 +1441,9 @@ static int coroutine_fn qcow2_do_open(BlockDriverState *bs, QDict *options, goto fail; } + /* TODO Open external data file */ + s->data_file = bs->file; + /* qcow2_read_extension may have set up the crypto context * if the crypt method needs a header region, some methods * don't need header extensions, so must check here @@ -2428,6 +2432,11 @@ int qcow2_update_header(BlockDriverState *bs) .bit = QCOW2_INCOMPAT_CORRUPT_BITNR, .name = "corrupt bit", }, + { + .type = QCOW2_FEAT_TYPE_INCOMPATIBLE, + .bit = QCOW2_INCOMPAT_DATA_FILE_BITNR, + .name = "external data file", + }, { .type = QCOW2_FEAT_TYPE_COMPATIBLE, .bit = QCOW2_COMPAT_LAZY_REFCOUNTS_BITNR, diff --git a/tests/qemu-iotests/031.out b/tests/qemu-iotests/031.out index 7f5050b816..68a74d03b9 100644 --- a/tests/qemu-iotests/031.out +++ b/tests/qemu-iotests/031.out @@ -117,7 +117,7 @@ header_length 104 Header extension: magic 0x6803f857 -length 144 +length 192 data Header extension: @@ -150,7 +150,7 @@ header_length 104 Header extension: magic 0x6803f857 -length 144 +length 192 data Header extension: @@ -164,7 +164,7 @@ No errors were found on the image. magic 0x514649fb version 3 -backing_file_offset 0x148 +backing_file_offset 0x178 backing_file_size 0x17 cluster_bits 16 size 67108864 @@ -188,7 +188,7 @@ data 'host_device' Header extension: magic 0x6803f857 -length 144 +length 192 data Header extension: diff --git a/tests/qemu-iotests/036.out b/tests/qemu-iotests/036.out index 9b009b8c15..e489b44386 100644 --- a/tests/qemu-iotests/036.out +++ b/tests/qemu-iotests/036.out @@ -58,7 +58,7 @@ header_length 104 Header extension: magic 0x6803f857 -length 144 +length 192 data @@ -86,7 +86,7 @@ header_length 104 Header extension: magic 0x6803f857 -length 144 +length 192 data *** done diff --git a/tests/qemu-iotests/061.out b/tests/qemu-iotests/061.out index 183f7dd690..758284011b 100644 --- a/tests/qemu-iotests/061.out +++ b/tests/qemu-iotests/061.out @@ -26,7 +26,7 @@ header_length 104 Header extension: magic 0x6803f857 -length 144 +length 192 data magic 0x514649fb @@ -84,7 +84,7 @@ header_length 104 Header extension: magic 0x6803f857 -length 144 +length 192 data magic 0x514649fb @@ -144,7 +144,7 @@ header_length 104 Header extension: magic 0x6803f857 -length 144 +length 192 data ERROR cluster 5 refcount=0 reference=1 @@ -199,7 +199,7 @@ header_length 104 Header extension: magic 0x6803f857 -length 144 +length 192 data magic 0x514649fb @@ -268,7 +268,7 @@ header_length 104 Header extension: magic 0x6803f857 -length 144 +length 192 data read 65536/65536 bytes at offset 44040192 @@ -306,7 +306,7 @@ header_length 104 Header extension: magic 0x6803f857 -length 144 +length 192 data ERROR cluster 5 refcount=0 reference=1 @@ -335,7 +335,7 @@ header_length 104 Header extension: magic 0x6803f857 -length 144 +length 192 data read 131072/131072 bytes at offset 0 From patchwork Wed Feb 27 17:22:41 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 1049034 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 448jL62xFHz9sBF for ; Thu, 28 Feb 2019 04:27:23 +1100 (AEDT) Received: from localhost ([127.0.0.1]:47944 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gz2zI-0005QF-U5 for incoming@patchwork.ozlabs.org; Wed, 27 Feb 2019 12:27:20 -0500 Received: from eggs.gnu.org ([209.51.188.92]:51264) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gz2vs-000367-OW for qemu-devel@nongnu.org; Wed, 27 Feb 2019 12:23:50 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gz2vp-0003J4-7f for qemu-devel@nongnu.org; Wed, 27 Feb 2019 12:23:48 -0500 Received: from mx1.redhat.com ([209.132.183.28]:57012) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1gz2vV-0002zL-Ga; Wed, 27 Feb 2019 12:23:28 -0500 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 9444DC0ABF4D; Wed, 27 Feb 2019 17:23:20 +0000 (UTC) Received: from linux.fritz.box.com (ovpn-117-19.ams2.redhat.com [10.36.117.19]) by smtp.corp.redhat.com (Postfix) with ESMTP id 4C75B171A0; Wed, 27 Feb 2019 17:23:17 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Date: Wed, 27 Feb 2019 18:22:41 +0100 Message-Id: <20190227172256.30368-6-kwolf@redhat.com> In-Reply-To: <20190227172256.30368-1-kwolf@redhat.com> References: <20190227172256.30368-1-kwolf@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.31]); Wed, 27 Feb 2019 17:23:20 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH 05/20] qcow2: Pass bs to qcow2_get_cluster_type() X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org, mreitz@redhat.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Kevin Wolf --- block/qcow2.h | 3 ++- block/qcow2-cluster.c | 37 +++++++++++++++++++------------------ block/qcow2-refcount.c | 10 +++++----- 3 files changed, 26 insertions(+), 24 deletions(-) diff --git a/block/qcow2.h b/block/qcow2.h index c63c3959f7..7a34bd0c53 100644 --- a/block/qcow2.h +++ b/block/qcow2.h @@ -510,7 +510,8 @@ static inline int64_t qcow2_vm_state_offset(BDRVQcow2State *s) return (int64_t)s->l1_vm_state_index << (s->cluster_bits + s->l2_bits); } -static inline QCow2ClusterType qcow2_get_cluster_type(uint64_t l2_entry) +static inline QCow2ClusterType qcow2_get_cluster_type(BlockDriverState *bs, + uint64_t l2_entry) { if (l2_entry & QCOW_OFLAG_COMPRESSED) { return QCOW2_CLUSTER_COMPRESSED; diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c index 179aa2c728..9cc8f0f3e4 100644 --- a/block/qcow2-cluster.c +++ b/block/qcow2-cluster.c @@ -380,8 +380,8 @@ fail: * as contiguous. (This allows it, for example, to stop at the first compressed * cluster which may require a different handling) */ -static int count_contiguous_clusters(int nb_clusters, int cluster_size, - uint64_t *l2_slice, uint64_t stop_flags) +static int count_contiguous_clusters(BlockDriverState *bs, int nb_clusters, + int cluster_size, uint64_t *l2_slice, uint64_t stop_flags) { int i; QCow2ClusterType first_cluster_type; @@ -394,7 +394,7 @@ static int count_contiguous_clusters(int nb_clusters, int cluster_size, } /* must be allocated */ - first_cluster_type = qcow2_get_cluster_type(first_entry); + first_cluster_type = qcow2_get_cluster_type(bs, first_entry); assert(first_cluster_type == QCOW2_CLUSTER_NORMAL || first_cluster_type == QCOW2_CLUSTER_ZERO_ALLOC); @@ -412,7 +412,8 @@ static int count_contiguous_clusters(int nb_clusters, int cluster_size, * Checks how many consecutive unallocated clusters in a given L2 * slice have the same cluster type. */ -static int count_contiguous_clusters_unallocated(int nb_clusters, +static int count_contiguous_clusters_unallocated(BlockDriverState *bs, + int nb_clusters, uint64_t *l2_slice, QCow2ClusterType wanted_type) { @@ -422,7 +423,7 @@ static int count_contiguous_clusters_unallocated(int nb_clusters, wanted_type == QCOW2_CLUSTER_UNALLOCATED); for (i = 0; i < nb_clusters; i++) { uint64_t entry = be64_to_cpu(l2_slice[i]); - QCow2ClusterType type = qcow2_get_cluster_type(entry); + QCow2ClusterType type = qcow2_get_cluster_type(bs, entry); if (type != wanted_type) { break; @@ -595,7 +596,7 @@ int qcow2_get_cluster_offset(BlockDriverState *bs, uint64_t offset, * true */ assert(nb_clusters <= INT_MAX); - type = qcow2_get_cluster_type(*cluster_offset); + type = qcow2_get_cluster_type(bs, *cluster_offset); if (s->qcow_version < 3 && (type == QCOW2_CLUSTER_ZERO_PLAIN || type == QCOW2_CLUSTER_ZERO_ALLOC)) { qcow2_signal_corruption(bs, true, -1, -1, "Zero cluster entry found" @@ -613,14 +614,14 @@ int qcow2_get_cluster_offset(BlockDriverState *bs, uint64_t offset, case QCOW2_CLUSTER_ZERO_PLAIN: case QCOW2_CLUSTER_UNALLOCATED: /* how many empty clusters ? */ - c = count_contiguous_clusters_unallocated(nb_clusters, + c = count_contiguous_clusters_unallocated(bs, nb_clusters, &l2_slice[l2_index], type); *cluster_offset = 0; break; case QCOW2_CLUSTER_ZERO_ALLOC: case QCOW2_CLUSTER_NORMAL: /* how many allocated clusters ? */ - c = count_contiguous_clusters(nb_clusters, s->cluster_size, + c = count_contiguous_clusters(bs, nb_clusters, s->cluster_size, &l2_slice[l2_index], QCOW_OFLAG_ZERO); *cluster_offset &= L2E_OFFSET_MASK; if (offset_into_cluster(s, *cluster_offset)) { @@ -1013,14 +1014,14 @@ void qcow2_alloc_cluster_abort(BlockDriverState *bs, QCowL2Meta *m) * write, but require COW to be performed (this includes yet unallocated space, * which must copy from the backing file) */ -static int count_cow_clusters(BDRVQcow2State *s, int nb_clusters, +static int count_cow_clusters(BlockDriverState *bs, int nb_clusters, uint64_t *l2_slice, int l2_index) { int i; for (i = 0; i < nb_clusters; i++) { uint64_t l2_entry = be64_to_cpu(l2_slice[l2_index + i]); - QCow2ClusterType cluster_type = qcow2_get_cluster_type(l2_entry); + QCow2ClusterType cluster_type = qcow2_get_cluster_type(bs, l2_entry); switch(cluster_type) { case QCOW2_CLUSTER_NORMAL: @@ -1165,7 +1166,7 @@ static int handle_copied(BlockDriverState *bs, uint64_t guest_offset, cluster_offset = be64_to_cpu(l2_slice[l2_index]); /* Check how many clusters are already allocated and don't need COW */ - if (qcow2_get_cluster_type(cluster_offset) == QCOW2_CLUSTER_NORMAL + if (qcow2_get_cluster_type(bs, cluster_offset) == QCOW2_CLUSTER_NORMAL && (cluster_offset & QCOW_OFLAG_COPIED)) { /* If a specific host_offset is required, check it */ @@ -1189,7 +1190,7 @@ static int handle_copied(BlockDriverState *bs, uint64_t guest_offset, /* We keep all QCOW_OFLAG_COPIED clusters */ keep_clusters = - count_contiguous_clusters(nb_clusters, s->cluster_size, + count_contiguous_clusters(bs, nb_clusters, s->cluster_size, &l2_slice[l2_index], QCOW_OFLAG_COPIED | QCOW_OFLAG_ZERO); assert(keep_clusters <= nb_clusters); @@ -1324,7 +1325,7 @@ static int handle_alloc(BlockDriverState *bs, uint64_t guest_offset, if (entry & QCOW_OFLAG_COMPRESSED) { nb_clusters = 1; } else { - nb_clusters = count_cow_clusters(s, nb_clusters, l2_slice, l2_index); + nb_clusters = count_cow_clusters(bs, nb_clusters, l2_slice, l2_index); } /* This function is only called when there were no non-COW clusters, so if @@ -1332,7 +1333,7 @@ static int handle_alloc(BlockDriverState *bs, uint64_t guest_offset, * wrong with our code. */ assert(nb_clusters > 0); - if (qcow2_get_cluster_type(entry) == QCOW2_CLUSTER_ZERO_ALLOC && + if (qcow2_get_cluster_type(bs, entry) == QCOW2_CLUSTER_ZERO_ALLOC && (entry & QCOW_OFLAG_COPIED) && (!*host_offset || start_of_cluster(s, *host_offset) == (entry & L2E_OFFSET_MASK))) @@ -1352,7 +1353,7 @@ static int handle_alloc(BlockDriverState *bs, uint64_t guest_offset, * would be fine, too, but count_cow_clusters() above has limited * nb_clusters already to a range of COW clusters */ preallocated_nb_clusters = - count_contiguous_clusters(nb_clusters, s->cluster_size, + count_contiguous_clusters(bs, nb_clusters, s->cluster_size, &l2_slice[l2_index], QCOW_OFLAG_COPIED); assert(preallocated_nb_clusters > 0); @@ -1616,7 +1617,7 @@ static int discard_in_l2_slice(BlockDriverState *bs, uint64_t offset, * If full_discard is true, the sector should not read back as zeroes, * but rather fall through to the backing file. */ - switch (qcow2_get_cluster_type(old_l2_entry)) { + switch (qcow2_get_cluster_type(bs, old_l2_entry)) { case QCOW2_CLUSTER_UNALLOCATED: if (full_discard || !bs->backing) { continue; @@ -1729,7 +1730,7 @@ static int zero_in_l2_slice(BlockDriverState *bs, uint64_t offset, * Minimize L2 changes if the cluster already reads back as * zeroes with correct allocation. */ - cluster_type = qcow2_get_cluster_type(old_offset); + cluster_type = qcow2_get_cluster_type(bs, old_offset); if (cluster_type == QCOW2_CLUSTER_ZERO_PLAIN || (cluster_type == QCOW2_CLUSTER_ZERO_ALLOC && !unmap)) { continue; @@ -1871,7 +1872,7 @@ static int expand_zero_clusters_in_l1(BlockDriverState *bs, uint64_t *l1_table, uint64_t l2_entry = be64_to_cpu(l2_slice[j]); int64_t offset = l2_entry & L2E_OFFSET_MASK; QCow2ClusterType cluster_type = - qcow2_get_cluster_type(l2_entry); + qcow2_get_cluster_type(bs, l2_entry); if (cluster_type != QCOW2_CLUSTER_ZERO_PLAIN && cluster_type != QCOW2_CLUSTER_ZERO_ALLOC) { diff --git a/block/qcow2-refcount.c b/block/qcow2-refcount.c index 6f13d470d3..05e7974d7e 100644 --- a/block/qcow2-refcount.c +++ b/block/qcow2-refcount.c @@ -1157,7 +1157,7 @@ void qcow2_free_any_clusters(BlockDriverState *bs, uint64_t l2_entry, { BDRVQcow2State *s = bs->opaque; - switch (qcow2_get_cluster_type(l2_entry)) { + switch (qcow2_get_cluster_type(bs, l2_entry)) { case QCOW2_CLUSTER_COMPRESSED: { int nb_csectors; @@ -1300,7 +1300,7 @@ int qcow2_update_snapshot_refcount(BlockDriverState *bs, entry &= ~QCOW_OFLAG_COPIED; offset = entry & L2E_OFFSET_MASK; - switch (qcow2_get_cluster_type(entry)) { + switch (qcow2_get_cluster_type(bs, entry)) { case QCOW2_CLUSTER_COMPRESSED: nb_csectors = ((entry >> s->csize_shift) & s->csize_mask) + 1; @@ -1582,7 +1582,7 @@ static int check_refcounts_l2(BlockDriverState *bs, BdrvCheckResult *res, for(i = 0; i < s->l2_size; i++) { l2_entry = be64_to_cpu(l2_table[i]); - switch (qcow2_get_cluster_type(l2_entry)) { + switch (qcow2_get_cluster_type(bs, l2_entry)) { case QCOW2_CLUSTER_COMPRESSED: /* Compressed clusters don't have QCOW_OFLAG_COPIED */ if (l2_entry & QCOW_OFLAG_COPIED) { @@ -1633,7 +1633,7 @@ static int check_refcounts_l2(BlockDriverState *bs, BdrvCheckResult *res, /* Correct offsets are cluster aligned */ if (offset_into_cluster(s, offset)) { - if (qcow2_get_cluster_type(l2_entry) == + if (qcow2_get_cluster_type(bs, l2_entry) == QCOW2_CLUSTER_ZERO_ALLOC) { fprintf(stderr, "%s offset=%" PRIx64 ": Preallocated zero " @@ -1868,7 +1868,7 @@ static int check_oflag_copied(BlockDriverState *bs, BdrvCheckResult *res, for (j = 0; j < s->l2_size; j++) { uint64_t l2_entry = be64_to_cpu(l2_table[j]); uint64_t data_offset = l2_entry & L2E_OFFSET_MASK; - QCow2ClusterType cluster_type = qcow2_get_cluster_type(l2_entry); + QCow2ClusterType cluster_type = qcow2_get_cluster_type(bs, l2_entry); if (cluster_type == QCOW2_CLUSTER_NORMAL || cluster_type == QCOW2_CLUSTER_ZERO_ALLOC) { From patchwork Wed Feb 27 17:22:42 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 1049038 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 448jPR1r23z9s70 for ; Thu, 28 Feb 2019 04:30:19 +1100 (AEDT) Received: from localhost ([127.0.0.1]:47986 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gz329-0007mc-4L for incoming@patchwork.ozlabs.org; Wed, 27 Feb 2019 12:30:17 -0500 Received: from eggs.gnu.org ([209.51.188.92]:51473) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gz2wT-0003f1-6n for qemu-devel@nongnu.org; Wed, 27 Feb 2019 12:24:25 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gz2wE-0003ZT-Gv for qemu-devel@nongnu.org; Wed, 27 Feb 2019 12:24:13 -0500 Received: from mx1.redhat.com ([209.132.183.28]:37522) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1gz2wB-00030Z-Fu; Wed, 27 Feb 2019 12:24:07 -0500 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 84F4D316BCCD; Wed, 27 Feb 2019 17:23:23 +0000 (UTC) Received: from linux.fritz.box.com (ovpn-117-19.ams2.redhat.com [10.36.117.19]) by smtp.corp.redhat.com (Postfix) with ESMTP id DD5DE61469; Wed, 27 Feb 2019 17:23:20 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Date: Wed, 27 Feb 2019 18:22:42 +0100 Message-Id: <20190227172256.30368-7-kwolf@redhat.com> In-Reply-To: <20190227172256.30368-1-kwolf@redhat.com> References: <20190227172256.30368-1-kwolf@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.41]); Wed, 27 Feb 2019 17:23:23 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH 06/20] qcow2: Prepare qcow2_get_cluster_type() for external data file X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org, mreitz@redhat.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Kevin Wolf --- block/qcow2.h | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/block/qcow2.h b/block/qcow2.h index 7a34bd0c53..8fe2d55005 100644 --- a/block/qcow2.h +++ b/block/qcow2.h @@ -521,7 +521,15 @@ static inline QCow2ClusterType qcow2_get_cluster_type(BlockDriverState *bs, } return QCOW2_CLUSTER_ZERO_PLAIN; } else if (!(l2_entry & L2E_OFFSET_MASK)) { - return QCOW2_CLUSTER_UNALLOCATED; + /* Offset 0 generally means unallocated, but it is ambiguous with + * external data files because 0 is a valid offset there. However, all + * clusters in external data files always have refcount 1, so we can + * rely on QCOW_OFLAG_COPIED to disambiguate. */ + if (has_data_file(bs) && (l2_entry & QCOW_OFLAG_COPIED)) { + return QCOW2_CLUSTER_NORMAL; + } else { + return QCOW2_CLUSTER_UNALLOCATED; + } } else { return QCOW2_CLUSTER_NORMAL; } From patchwork Wed Feb 27 17:22:43 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 1049044 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 448jVC5k2Dz9s70 for ; Thu, 28 Feb 2019 04:34:26 +1100 (AEDT) Received: from localhost ([127.0.0.1]:48058 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gz367-0002bh-6l for incoming@patchwork.ozlabs.org; Wed, 27 Feb 2019 12:34:23 -0500 Received: from eggs.gnu.org ([209.51.188.92]:51634) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gz2wZ-0003kq-Cc for qemu-devel@nongnu.org; Wed, 27 Feb 2019 12:24:32 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gz2wY-0003pD-7p for qemu-devel@nongnu.org; Wed, 27 Feb 2019 12:24:31 -0500 Received: from mx1.redhat.com ([209.132.183.28]:50784) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1gz2wE-00031q-74; Wed, 27 Feb 2019 12:24:10 -0500 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 89B3430BAD87; Wed, 27 Feb 2019 17:23:26 +0000 (UTC) Received: from linux.fritz.box.com (ovpn-117-19.ams2.redhat.com [10.36.117.19]) by smtp.corp.redhat.com (Postfix) with ESMTP id CF29917153; Wed, 27 Feb 2019 17:23:23 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Date: Wed, 27 Feb 2019 18:22:43 +0100 Message-Id: <20190227172256.30368-8-kwolf@redhat.com> In-Reply-To: <20190227172256.30368-1-kwolf@redhat.com> References: <20190227172256.30368-1-kwolf@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.47]); Wed, 27 Feb 2019 17:23:26 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH 07/20] qcow2: Prepare count_contiguous_clusters() for external data file X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org, mreitz@redhat.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Offset 0 can be valid for normal (allocated) clusters now, so use qcow2_get_cluster_type() instead. Signed-off-by: Kevin Wolf --- block/qcow2-cluster.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c index 9cc8f0f3e4..660161bf00 100644 --- a/block/qcow2-cluster.c +++ b/block/qcow2-cluster.c @@ -389,12 +389,12 @@ static int count_contiguous_clusters(BlockDriverState *bs, int nb_clusters, uint64_t first_entry = be64_to_cpu(l2_slice[0]); uint64_t offset = first_entry & mask; - if (!offset) { + first_cluster_type = qcow2_get_cluster_type(bs, first_entry); + if (first_cluster_type == QCOW2_CLUSTER_UNALLOCATED) { return 0; } /* must be allocated */ - first_cluster_type = qcow2_get_cluster_type(bs, first_entry); assert(first_cluster_type == QCOW2_CLUSTER_NORMAL || first_cluster_type == QCOW2_CLUSTER_ZERO_ALLOC); From patchwork Wed Feb 27 17:22:44 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 1049053 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 448jYm2pqVz9s1b for ; Thu, 28 Feb 2019 04:37:32 +1100 (AEDT) Received: from localhost ([127.0.0.1]:48120 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gz397-0004ro-Rx for incoming@patchwork.ozlabs.org; Wed, 27 Feb 2019 12:37:29 -0500 Received: from eggs.gnu.org ([209.51.188.92]:51696) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gz2wa-0003mS-Lo for qemu-devel@nongnu.org; Wed, 27 Feb 2019 12:24:36 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gz2wY-0003p0-8i for qemu-devel@nongnu.org; Wed, 27 Feb 2019 12:24:32 -0500 Received: from mx1.redhat.com ([209.132.183.28]:36110) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1gz2wD-00037v-FV; Wed, 27 Feb 2019 12:24:09 -0500 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 85A44C06586E; Wed, 27 Feb 2019 17:23:30 +0000 (UTC) Received: from linux.fritz.box.com (ovpn-117-19.ams2.redhat.com [10.36.117.19]) by smtp.corp.redhat.com (Postfix) with ESMTP id D78A417153; Wed, 27 Feb 2019 17:23:26 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Date: Wed, 27 Feb 2019 18:22:44 +0100 Message-Id: <20190227172256.30368-9-kwolf@redhat.com> In-Reply-To: <20190227172256.30368-1-kwolf@redhat.com> References: <20190227172256.30368-1-kwolf@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.32]); Wed, 27 Feb 2019 17:23:30 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH 08/20] qcow2: Don't assume 0 is an invalid cluster offset X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org, mreitz@redhat.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" The cluster allocation code uses 0 as an invalid offset that is used in case of errors or as "offset not yet determined". With external data files, a host cluster offset of 0 becomes valid, though. Define a constant INV_OFFSET (which is not cluster aligned and will therefore never be a valid offset) that can be used for such purposes. This removes the additional host_offset == 0 check that commit ff52aab2df5 introduced; the confusion between an invalid offset and (erroneous) allocation at offset 0 is removed with this change. Signed-off-by: Kevin Wolf --- block/qcow2.h | 2 ++ block/qcow2-cluster.c | 59 ++++++++++++++++++++----------------------- 2 files changed, 29 insertions(+), 32 deletions(-) diff --git a/block/qcow2.h b/block/qcow2.h index 8fe2d55005..e3bf322be1 100644 --- a/block/qcow2.h +++ b/block/qcow2.h @@ -463,6 +463,8 @@ typedef enum QCow2MetadataOverlap { #define REFT_OFFSET_MASK 0xfffffffffffffe00ULL +#define INV_OFFSET (-1ULL) + static inline bool has_data_file(BlockDriverState *bs) { BDRVQcow2State *s = bs->opaque; diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c index 660161bf00..1d09f5454e 100644 --- a/block/qcow2-cluster.c +++ b/block/qcow2-cluster.c @@ -1109,9 +1109,9 @@ static int handle_dependencies(BlockDriverState *bs, uint64_t guest_offset, /* * Checks how many already allocated clusters that don't require a copy on - * write there are at the given guest_offset (up to *bytes). If - * *host_offset is not zero, only physically contiguous clusters beginning at - * this host offset are counted. + * write there are at the given guest_offset (up to *bytes). If *host_offset is + * not INV_OFFSET, only physically contiguous clusters beginning at this host + * offset are counted. * * Note that guest_offset may not be cluster aligned. In this case, the * returned *host_offset points to exact byte referenced by guest_offset and @@ -1143,8 +1143,8 @@ static int handle_copied(BlockDriverState *bs, uint64_t guest_offset, trace_qcow2_handle_copied(qemu_coroutine_self(), guest_offset, *host_offset, *bytes); - assert(*host_offset == 0 || offset_into_cluster(s, guest_offset) - == offset_into_cluster(s, *host_offset)); + assert(*host_offset == INV_OFFSET || offset_into_cluster(s, guest_offset) + == offset_into_cluster(s, *host_offset)); /* * Calculate the number of clusters to look for. We stop at L2 slice @@ -1182,7 +1182,7 @@ static int handle_copied(BlockDriverState *bs, uint64_t guest_offset, goto out; } - if (*host_offset != 0 && !offset_matches) { + if (*host_offset != INV_OFFSET && !offset_matches) { *bytes = 0; ret = 0; goto out; @@ -1225,10 +1225,10 @@ out: * contain the number of clusters that have been allocated and are contiguous * in the image file. * - * If *host_offset is non-zero, it specifies the offset in the image file at - * which the new clusters must start. *nb_clusters can be 0 on return in this - * case if the cluster at host_offset is already in use. If *host_offset is - * zero, the clusters can be allocated anywhere in the image file. + * If *host_offset is not INV_OFFSET, it specifies the offset in the image file + * at which the new clusters must start. *nb_clusters can be 0 on return in + * this case if the cluster at host_offset is already in use. If *host_offset + * is INV_OFFSET, the clusters can be allocated anywhere in the image file. * * *host_offset is updated to contain the offset into the image file at which * the first allocated cluster starts. @@ -1247,7 +1247,7 @@ static int do_alloc_cluster_offset(BlockDriverState *bs, uint64_t guest_offset, /* Allocate new clusters */ trace_qcow2_cluster_alloc_phys(qemu_coroutine_self()); - if (*host_offset == 0) { + if (*host_offset == INV_OFFSET) { int64_t cluster_offset = qcow2_alloc_clusters(bs, *nb_clusters * s->cluster_size); if (cluster_offset < 0) { @@ -1267,8 +1267,8 @@ static int do_alloc_cluster_offset(BlockDriverState *bs, uint64_t guest_offset, /* * Allocates new clusters for an area that either is yet unallocated or needs a - * copy on write. If *host_offset is non-zero, clusters are only allocated if - * the new allocation can match the specified host offset. + * copy on write. If *host_offset is not INV_OFFSET, clusters are only + * allocated if the new allocation can match the specified host offset. * * Note that guest_offset may not be cluster aligned. In this case, the * returned *host_offset points to exact byte referenced by guest_offset and @@ -1296,7 +1296,7 @@ static int handle_alloc(BlockDriverState *bs, uint64_t guest_offset, int ret; bool keep_old_clusters = false; - uint64_t alloc_cluster_offset = 0; + uint64_t alloc_cluster_offset = INV_OFFSET; trace_qcow2_handle_alloc(qemu_coroutine_self(), guest_offset, *host_offset, *bytes); @@ -1335,7 +1335,7 @@ static int handle_alloc(BlockDriverState *bs, uint64_t guest_offset, if (qcow2_get_cluster_type(bs, entry) == QCOW2_CLUSTER_ZERO_ALLOC && (entry & QCOW_OFLAG_COPIED) && - (!*host_offset || + (*host_offset == INV_OFFSET || start_of_cluster(s, *host_offset) == (entry & L2E_OFFSET_MASK))) { int preallocated_nb_clusters; @@ -1367,9 +1367,10 @@ static int handle_alloc(BlockDriverState *bs, uint64_t guest_offset, qcow2_cache_put(s->l2_table_cache, (void **) &l2_slice); - if (!alloc_cluster_offset) { + if (alloc_cluster_offset == INV_OFFSET) { /* Allocate, if necessary at a given offset in the image file */ - alloc_cluster_offset = start_of_cluster(s, *host_offset); + alloc_cluster_offset = *host_offset == INV_OFFSET ? INV_OFFSET : + start_of_cluster(s, *host_offset); ret = do_alloc_cluster_offset(bs, guest_offset, &alloc_cluster_offset, &nb_clusters); if (ret < 0) { @@ -1382,16 +1383,7 @@ static int handle_alloc(BlockDriverState *bs, uint64_t guest_offset, return 0; } - /* !*host_offset would overwrite the image header and is reserved for - * "no host offset preferred". If 0 was a valid host offset, it'd - * trigger the following overlap check; do that now to avoid having an - * invalid value in *host_offset. */ - if (!alloc_cluster_offset) { - ret = qcow2_pre_write_overlap_check(bs, 0, alloc_cluster_offset, - nb_clusters * s->cluster_size); - assert(ret < 0); - goto fail; - } + assert(alloc_cluster_offset != INV_OFFSET); } /* @@ -1483,14 +1475,14 @@ int qcow2_alloc_cluster_offset(BlockDriverState *bs, uint64_t offset, again: start = offset; remaining = *bytes; - cluster_offset = 0; - *host_offset = 0; + cluster_offset = INV_OFFSET; + *host_offset = INV_OFFSET; cur_bytes = 0; *m = NULL; while (true) { - if (!*host_offset) { + if (*host_offset == INV_OFFSET && cluster_offset != INV_OFFSET) { *host_offset = start_of_cluster(s, cluster_offset); } @@ -1498,7 +1490,10 @@ again: start += cur_bytes; remaining -= cur_bytes; - cluster_offset += cur_bytes; + + if (cluster_offset != INV_OFFSET) { + cluster_offset += cur_bytes; + } if (remaining == 0) { break; @@ -1570,7 +1565,7 @@ again: *bytes -= remaining; assert(*bytes > 0); - assert(*host_offset != 0); + assert(*host_offset != INV_OFFSET); return 0; } From patchwork Wed Feb 27 17:22:45 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 1049032 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 448jJM6hGJz9sBR for ; Thu, 28 Feb 2019 04:25:55 +1100 (AEDT) Received: from localhost ([127.0.0.1]:47900 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gz2xt-0004Ag-NE for incoming@patchwork.ozlabs.org; Wed, 27 Feb 2019 12:25:53 -0500 Received: from eggs.gnu.org ([209.51.188.92]:51496) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gz2wT-0003fS-Oi for qemu-devel@nongnu.org; Wed, 27 Feb 2019 12:24:29 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gz2wO-0003er-6M for qemu-devel@nongnu.org; Wed, 27 Feb 2019 12:24:25 -0500 Received: from mx1.redhat.com ([209.132.183.28]:37598) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1gz2wB-0003Be-Q7; Wed, 27 Feb 2019 12:24:08 -0500 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 142E5316E90F; Wed, 27 Feb 2019 17:23:32 +0000 (UTC) Received: from linux.fritz.box.com (ovpn-117-19.ams2.redhat.com [10.36.117.19]) by smtp.corp.redhat.com (Postfix) with ESMTP id D3DAF61469; Wed, 27 Feb 2019 17:23:30 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Date: Wed, 27 Feb 2019 18:22:45 +0100 Message-Id: <20190227172256.30368-10-kwolf@redhat.com> In-Reply-To: <20190227172256.30368-1-kwolf@redhat.com> References: <20190227172256.30368-1-kwolf@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.41]); Wed, 27 Feb 2019 17:23:32 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH 09/20] qcow2: Return 0/-errno in qcow2_alloc_compressed_cluster_offset() X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org, mreitz@redhat.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" qcow2_alloc_compressed_cluster_offset() used to return the cluster offset for success and 0 for error. This doesn't only conflict with 0 as a valid host offset, but also loses the error code. Similar to the change made to qcow2_alloc_cluster_offset() for uncompressed clusters in commit 148da7ea9d6, make the function return 0/-errno and return the allocated cluster offset in a by-reference parameter. Signed-off-by: Kevin Wolf --- block/qcow2.h | 7 ++++--- block/qcow2-cluster.c | 28 +++++++++++++--------------- block/qcow2.c | 19 ++++++++----------- tests/qemu-iotests/220.out | 2 +- 4 files changed, 26 insertions(+), 30 deletions(-) diff --git a/block/qcow2.h b/block/qcow2.h index e3bf322be1..fad6abf602 100644 --- a/block/qcow2.h +++ b/block/qcow2.h @@ -647,9 +647,10 @@ int qcow2_get_cluster_offset(BlockDriverState *bs, uint64_t offset, int qcow2_alloc_cluster_offset(BlockDriverState *bs, uint64_t offset, unsigned int *bytes, uint64_t *host_offset, QCowL2Meta **m); -uint64_t qcow2_alloc_compressed_cluster_offset(BlockDriverState *bs, - uint64_t offset, - int compressed_size); +int qcow2_alloc_compressed_cluster_offset(BlockDriverState *bs, + uint64_t offset, + int compressed_size, + uint64_t *host_offset); int qcow2_alloc_cluster_link_l2(BlockDriverState *bs, QCowL2Meta *m); void qcow2_alloc_cluster_abort(BlockDriverState *bs, QCowL2Meta *m); diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c index 1d09f5454e..8c4b4005ff 100644 --- a/block/qcow2-cluster.c +++ b/block/qcow2-cluster.c @@ -736,19 +736,16 @@ static int get_cluster_table(BlockDriverState *bs, uint64_t offset, /* * alloc_compressed_cluster_offset * - * For a given offset of the disk image, return cluster offset in - * qcow2 file. - * - * If the offset is not found, allocate a new compressed cluster. - * - * Return the cluster offset if successful, - * Return 0, otherwise. + * For a given offset on the virtual disk, allocate a new compressed cluster + * and put the host offset of the cluster into *host_offset. If a cluster is + * already allocated at the offset, return an error. * + * Return 0 on success and -errno in error cases */ - -uint64_t qcow2_alloc_compressed_cluster_offset(BlockDriverState *bs, - uint64_t offset, - int compressed_size) +int qcow2_alloc_compressed_cluster_offset(BlockDriverState *bs, + uint64_t offset, + int compressed_size, + uint64_t *host_offset) { BDRVQcow2State *s = bs->opaque; int l2_index, ret; @@ -758,7 +755,7 @@ uint64_t qcow2_alloc_compressed_cluster_offset(BlockDriverState *bs, ret = get_cluster_table(bs, offset, &l2_slice, &l2_index); if (ret < 0) { - return 0; + return ret; } /* Compression can't overwrite anything. Fail if the cluster was already @@ -766,13 +763,13 @@ uint64_t qcow2_alloc_compressed_cluster_offset(BlockDriverState *bs, cluster_offset = be64_to_cpu(l2_slice[l2_index]); if (cluster_offset & L2E_OFFSET_MASK) { qcow2_cache_put(s->l2_table_cache, (void **) &l2_slice); - return 0; + return -EIO; } cluster_offset = qcow2_alloc_bytes(bs, compressed_size); if (cluster_offset < 0) { qcow2_cache_put(s->l2_table_cache, (void **) &l2_slice); - return 0; + return cluster_offset; } nb_csectors = ((cluster_offset + compressed_size - 1) >> 9) - @@ -790,7 +787,8 @@ uint64_t qcow2_alloc_compressed_cluster_offset(BlockDriverState *bs, l2_slice[l2_index] = cpu_to_be64(cluster_offset); qcow2_cache_put(s->l2_table_cache, (void **) &l2_slice); - return cluster_offset; + *host_offset = cluster_offset & s->cluster_offset_mask; + return 0; } static int perform_cow(BlockDriverState *bs, QCowL2Meta *m) diff --git a/block/qcow2.c b/block/qcow2.c index c2e3a31d1d..11ff5e0434 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -3885,17 +3885,16 @@ qcow2_co_pwritev_compressed(BlockDriverState *bs, uint64_t offset, int ret; size_t out_len; uint8_t *buf, *out_buf; - int64_t cluster_offset; + uint64_t cluster_offset; if (bytes == 0) { /* align end of file to a sector boundary to ease reading with sector based I/Os */ - cluster_offset = bdrv_getlength(bs->file->bs); - if (cluster_offset < 0) { - return cluster_offset; + int64_t len = bdrv_getlength(bs->file->bs); + if (len < 0) { + return len; } - return bdrv_co_truncate(bs->file, cluster_offset, PREALLOC_MODE_OFF, - NULL); + return bdrv_co_truncate(bs->file, len, PREALLOC_MODE_OFF, NULL); } if (offset_into_cluster(s, offset)) { @@ -3932,14 +3931,12 @@ qcow2_co_pwritev_compressed(BlockDriverState *bs, uint64_t offset, } qemu_co_mutex_lock(&s->lock); - cluster_offset = - qcow2_alloc_compressed_cluster_offset(bs, offset, out_len); - if (!cluster_offset) { + ret = qcow2_alloc_compressed_cluster_offset(bs, offset, out_len, + &cluster_offset); + if (ret < 0) { qemu_co_mutex_unlock(&s->lock); - ret = -EIO; goto fail; } - cluster_offset &= s->cluster_offset_mask; ret = qcow2_pre_write_overlap_check(bs, 0, cluster_offset, out_len); qemu_co_mutex_unlock(&s->lock); diff --git a/tests/qemu-iotests/220.out b/tests/qemu-iotests/220.out index af3021fd88..33b994b8a1 100644 --- a/tests/qemu-iotests/220.out +++ b/tests/qemu-iotests/220.out @@ -38,7 +38,7 @@ wrote 2097152/2097152 bytes at offset 37748736 No errors were found on the image. image size 39845888 == Trying to write compressed cluster == -write failed: Input/output error +write failed: File too large image size 562949957615616 == Writing normal cluster == wrote 2097152/2097152 bytes at offset 0 From patchwork Wed Feb 27 17:22:46 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 1049052 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 448jYB5c6hz9s2R for ; Thu, 28 Feb 2019 04:37:02 +1100 (AEDT) Received: from localhost ([127.0.0.1]:48116 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gz38e-0004XD-M7 for incoming@patchwork.ozlabs.org; Wed, 27 Feb 2019 12:37:00 -0500 Received: from eggs.gnu.org ([209.51.188.92]:51644) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gz2wZ-0003kx-J0 for qemu-devel@nongnu.org; Wed, 27 Feb 2019 12:24:33 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gz2wY-0003ov-7D for qemu-devel@nongnu.org; Wed, 27 Feb 2019 12:24:31 -0500 Received: from mx1.redhat.com ([209.132.183.28]:50246) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1gz2wD-0003D4-VX; Wed, 27 Feb 2019 12:24:10 -0500 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 0A94B3E2B0; Wed, 27 Feb 2019 17:23:36 +0000 (UTC) Received: from linux.fritz.box.com (ovpn-117-19.ams2.redhat.com [10.36.117.19]) by smtp.corp.redhat.com (Postfix) with ESMTP id 605AC17153; Wed, 27 Feb 2019 17:23:32 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Date: Wed, 27 Feb 2019 18:22:46 +0100 Message-Id: <20190227172256.30368-11-kwolf@redhat.com> In-Reply-To: <20190227172256.30368-1-kwolf@redhat.com> References: <20190227172256.30368-1-kwolf@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.30]); Wed, 27 Feb 2019 17:23:36 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH 10/20] qcow2: Prepare qcow2_co_block_status() for data file X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org, mreitz@redhat.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Offset 0 cannot be assumed to mean an unallocated cluster any more. Instead, the cluster type needs to be checked. *file must refer to the data file instead of the image file if a valid offset is returned from qcow2_co_block_status(). Signed-off-by: Kevin Wolf --- block/qcow2.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/block/qcow2.c b/block/qcow2.c index 11ff5e0434..8dc6f21047 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -1817,11 +1817,11 @@ static int coroutine_fn qcow2_co_block_status(BlockDriverState *bs, *pnum = bytes; - if (cluster_offset != 0 && ret != QCOW2_CLUSTER_COMPRESSED && + if ((ret == QCOW2_CLUSTER_NORMAL || ret == QCOW2_CLUSTER_ZERO_ALLOC) && !s->crypto) { index_in_cluster = offset & (s->cluster_size - 1); *map = cluster_offset | index_in_cluster; - *file = bs->file->bs; + *file = s->data_file->bs; status |= BDRV_BLOCK_OFFSET_VALID; } if (ret == QCOW2_CLUSTER_ZERO_PLAIN || ret == QCOW2_CLUSTER_ZERO_ALLOC) { From patchwork Wed Feb 27 17:22:47 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 1049055 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 448jck4ghhz9s1b for ; Thu, 28 Feb 2019 04:40:06 +1100 (AEDT) Received: from localhost ([127.0.0.1]:48142 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gz3Ba-0006bc-Ot for incoming@patchwork.ozlabs.org; Wed, 27 Feb 2019 12:40:02 -0500 Received: from eggs.gnu.org ([209.51.188.92]:51914) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gz2wm-000413-TV for qemu-devel@nongnu.org; Wed, 27 Feb 2019 12:24:47 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gz2wi-00042o-EQ for qemu-devel@nongnu.org; Wed, 27 Feb 2019 12:24:44 -0500 Received: from mx1.redhat.com ([209.132.183.28]:52134) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1gz2wU-0003FB-5T; Wed, 27 Feb 2019 12:24:28 -0500 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id B77D230C3A62; Wed, 27 Feb 2019 17:23:37 +0000 (UTC) Received: from linux.fritz.box.com (ovpn-117-19.ams2.redhat.com [10.36.117.19]) by smtp.corp.redhat.com (Postfix) with ESMTP id 5820117153; Wed, 27 Feb 2019 17:23:36 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Date: Wed, 27 Feb 2019 18:22:47 +0100 Message-Id: <20190227172256.30368-12-kwolf@redhat.com> In-Reply-To: <20190227172256.30368-1-kwolf@redhat.com> References: <20190227172256.30368-1-kwolf@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.44]); Wed, 27 Feb 2019 17:23:37 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH 11/20] qcow2: External file I/O X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org, mreitz@redhat.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" This changes the qcow2 implementation to direct all guest data I/O to s->data_file rather than bs->file, while metadata I/O still uses bs->file. At the moment, this is still always the same, but soon we'll add options to set s->data_file to an external data file. Signed-off-by: Kevin Wolf --- block/qcow2.h | 2 +- block/qcow2-bitmap.c | 7 +++--- block/qcow2-cache.c | 6 ++--- block/qcow2-cluster.c | 46 +++++++++++++++++++++++++++++++------ block/qcow2-refcount.c | 39 +++++++++++++++++++++++-------- block/qcow2-snapshot.c | 7 +++--- block/qcow2.c | 52 +++++++++++++++++++++++++++++++++--------- 7 files changed, 122 insertions(+), 37 deletions(-) diff --git a/block/qcow2.h b/block/qcow2.h index fad6abf602..aac7fc4348 100644 --- a/block/qcow2.h +++ b/block/qcow2.h @@ -622,7 +622,7 @@ void qcow2_process_discards(BlockDriverState *bs, int ret); int qcow2_check_metadata_overlap(BlockDriverState *bs, int ign, int64_t offset, int64_t size); int qcow2_pre_write_overlap_check(BlockDriverState *bs, int ign, int64_t offset, - int64_t size); + int64_t size, bool data_file); int qcow2_inc_refcounts_imrt(BlockDriverState *bs, BdrvCheckResult *res, void **refcount_table, int64_t *refcount_table_size, diff --git a/block/qcow2-bitmap.c b/block/qcow2-bitmap.c index 3ee524da4b..9d968bdcda 100644 --- a/block/qcow2-bitmap.c +++ b/block/qcow2-bitmap.c @@ -778,7 +778,8 @@ static int bitmap_list_store(BlockDriverState *bs, Qcow2BitmapList *bm_list, * directory in-place (actually, turn-off the extension), which is checked * in qcow2_check_metadata_overlap() */ ret = qcow2_pre_write_overlap_check( - bs, in_place ? QCOW2_OL_BITMAP_DIRECTORY : 0, dir_offset, dir_size); + bs, in_place ? QCOW2_OL_BITMAP_DIRECTORY : 0, dir_offset, dir_size, + false); if (ret < 0) { goto fail; } @@ -1224,7 +1225,7 @@ static uint64_t *store_bitmap_data(BlockDriverState *bs, memset(buf + write_size, 0, s->cluster_size - write_size); } - ret = qcow2_pre_write_overlap_check(bs, 0, off, s->cluster_size); + ret = qcow2_pre_write_overlap_check(bs, 0, off, s->cluster_size, false); if (ret < 0) { error_setg_errno(errp, -ret, "Qcow2 overlap check failed"); goto fail; @@ -1292,7 +1293,7 @@ static int store_bitmap(BlockDriverState *bs, Qcow2Bitmap *bm, Error **errp) } ret = qcow2_pre_write_overlap_check(bs, 0, tb_offset, - tb_size * sizeof(tb[0])); + tb_size * sizeof(tb[0]), false); if (ret < 0) { error_setg_errno(errp, -ret, "Qcow2 overlap check failed"); goto fail; diff --git a/block/qcow2-cache.c b/block/qcow2-cache.c index d9dafa31e5..df02e7b20a 100644 --- a/block/qcow2-cache.c +++ b/block/qcow2-cache.c @@ -205,13 +205,13 @@ static int qcow2_cache_entry_flush(BlockDriverState *bs, Qcow2Cache *c, int i) if (c == s->refcount_block_cache) { ret = qcow2_pre_write_overlap_check(bs, QCOW2_OL_REFCOUNT_BLOCK, - c->entries[i].offset, c->table_size); + c->entries[i].offset, c->table_size, false); } else if (c == s->l2_table_cache) { ret = qcow2_pre_write_overlap_check(bs, QCOW2_OL_ACTIVE_L2, - c->entries[i].offset, c->table_size); + c->entries[i].offset, c->table_size, false); } else { ret = qcow2_pre_write_overlap_check(bs, 0, - c->entries[i].offset, c->table_size); + c->entries[i].offset, c->table_size, false); } if (ret < 0) { diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c index 8c4b4005ff..7579f5a5ae 100644 --- a/block/qcow2-cluster.c +++ b/block/qcow2-cluster.c @@ -153,7 +153,7 @@ int qcow2_grow_l1_table(BlockDriverState *bs, uint64_t min_size, /* the L1 position has not yet been updated, so these clusters must * indeed be completely free */ ret = qcow2_pre_write_overlap_check(bs, 0, new_l1_table_offset, - new_l1_size2); + new_l1_size2, false); if (ret < 0) { goto fail; } @@ -238,7 +238,7 @@ int qcow2_write_l1_entry(BlockDriverState *bs, int l1_index) } ret = qcow2_pre_write_overlap_check(bs, QCOW2_OL_ACTIVE_L1, - s->l1_table_offset + 8 * l1_start_index, sizeof(buf)); + s->l1_table_offset + 8 * l1_start_index, sizeof(buf), false); if (ret < 0) { return ret; } @@ -490,6 +490,7 @@ static int coroutine_fn do_perform_cow_write(BlockDriverState *bs, unsigned offset_in_cluster, QEMUIOVector *qiov) { + BDRVQcow2State *s = bs->opaque; int ret; if (qiov->size == 0) { @@ -497,13 +498,13 @@ static int coroutine_fn do_perform_cow_write(BlockDriverState *bs, } ret = qcow2_pre_write_overlap_check(bs, 0, - cluster_offset + offset_in_cluster, qiov->size); + cluster_offset + offset_in_cluster, qiov->size, true); if (ret < 0) { return ret; } BLKDBG_EVENT(bs->file, BLKDBG_COW_WRITE); - ret = bdrv_co_pwritev(bs->file, cluster_offset + offset_in_cluster, + ret = bdrv_co_pwritev(s->data_file, cluster_offset + offset_in_cluster, qiov->size, qiov, 0); if (ret < 0) { return ret; @@ -607,6 +608,14 @@ int qcow2_get_cluster_offset(BlockDriverState *bs, uint64_t offset, } switch (type) { case QCOW2_CLUSTER_COMPRESSED: + if (has_data_file(bs)) { + qcow2_signal_corruption(bs, true, -1, -1, "Compressed cluster " + "entry found in image with external data " + "file (L2 offset: %#" PRIx64 ", L2 index: " + "%#x)", l2_offset, l2_index); + ret = -EIO; + goto fail; + } /* Compressed clusters can only be processed one by one */ c = 1; *cluster_offset &= L2E_COMPRESSED_OFFSET_SIZE_MASK; @@ -633,6 +642,17 @@ int qcow2_get_cluster_offset(BlockDriverState *bs, uint64_t offset, ret = -EIO; goto fail; } + if (has_data_file(bs) && *cluster_offset != offset - offset_in_cluster) + { + qcow2_signal_corruption(bs, true, -1, -1, + "External data file host cluster offset %#" + PRIx64 " does not match guest cluster " + "offset: %#" PRIx64 + ", L2 index: %#x)", *cluster_offset, + offset - offset_in_cluster, l2_index); + ret = -EIO; + goto fail; + } break; default: abort(); @@ -753,6 +773,10 @@ int qcow2_alloc_compressed_cluster_offset(BlockDriverState *bs, int64_t cluster_offset; int nb_csectors; + if (has_data_file(bs)) { + return 0; + } + ret = get_cluster_table(bs, offset, &l2_slice, &l2_index); if (ret < 0) { return ret; @@ -1243,6 +1267,13 @@ static int do_alloc_cluster_offset(BlockDriverState *bs, uint64_t guest_offset, trace_qcow2_do_alloc_clusters_offset(qemu_coroutine_self(), guest_offset, *host_offset, *nb_clusters); + if (has_data_file(bs)) { + assert(*host_offset == INV_OFFSET || + *host_offset == start_of_cluster(s, guest_offset)); + *host_offset = start_of_cluster(s, guest_offset); + return 0; + } + /* Allocate new clusters */ trace_qcow2_cluster_alloc_phys(qemu_coroutine_self()); if (*host_offset == INV_OFFSET) { @@ -1919,7 +1950,7 @@ static int expand_zero_clusters_in_l1(BlockDriverState *bs, uint64_t *l1_table, } ret = qcow2_pre_write_overlap_check(bs, 0, offset, - s->cluster_size); + s->cluster_size, true); if (ret < 0) { if (cluster_type == QCOW2_CLUSTER_ZERO_PLAIN) { qcow2_free_clusters(bs, offset, s->cluster_size, @@ -1928,7 +1959,8 @@ static int expand_zero_clusters_in_l1(BlockDriverState *bs, uint64_t *l1_table, goto fail; } - ret = bdrv_pwrite_zeroes(bs->file, offset, s->cluster_size, 0); + ret = bdrv_pwrite_zeroes(s->data_file, offset, + s->cluster_size, 0); if (ret < 0) { if (cluster_type == QCOW2_CLUSTER_ZERO_PLAIN) { qcow2_free_clusters(bs, offset, s->cluster_size, @@ -1955,7 +1987,7 @@ static int expand_zero_clusters_in_l1(BlockDriverState *bs, uint64_t *l1_table, if (l2_dirty) { ret = qcow2_pre_write_overlap_check( bs, QCOW2_OL_INACTIVE_L2 | QCOW2_OL_ACTIVE_L2, - slice_offset, slice_size2); + slice_offset, slice_size2, false); if (ret < 0) { goto fail; } diff --git a/block/qcow2-refcount.c b/block/qcow2-refcount.c index 05e7974d7e..df73580e5d 100644 --- a/block/qcow2-refcount.c +++ b/block/qcow2-refcount.c @@ -1156,8 +1156,20 @@ void qcow2_free_any_clusters(BlockDriverState *bs, uint64_t l2_entry, int nb_clusters, enum qcow2_discard_type type) { BDRVQcow2State *s = bs->opaque; + QCow2ClusterType ctype = qcow2_get_cluster_type(bs, l2_entry); - switch (qcow2_get_cluster_type(bs, l2_entry)) { + if (has_data_file(bs)) { + if (s->discard_passthrough[type] && + (ctype == QCOW2_CLUSTER_NORMAL || + ctype == QCOW2_CLUSTER_ZERO_ALLOC)) + { + bdrv_pdiscard(s->data_file, l2_entry & L2E_OFFSET_MASK, + nb_clusters << s->cluster_bits); + } + return; + } + + switch (ctype) { case QCOW2_CLUSTER_COMPRESSED: { int nb_csectors; @@ -1649,7 +1661,7 @@ static int check_refcounts_l2(BlockDriverState *bs, BdrvCheckResult *res, l2_table[i] = cpu_to_be64(l2_entry); ret = qcow2_pre_write_overlap_check(bs, QCOW2_OL_ACTIVE_L2 | QCOW2_OL_INACTIVE_L2, - l2e_offset, sizeof(uint64_t)); + l2e_offset, sizeof(uint64_t), false); if (ret < 0) { fprintf(stderr, "ERROR: Overlap check failed\n"); res->check_errors++; @@ -1898,7 +1910,8 @@ static int check_oflag_copied(BlockDriverState *bs, BdrvCheckResult *res, if (l2_dirty) { ret = qcow2_pre_write_overlap_check(bs, QCOW2_OL_ACTIVE_L2, - l2_offset, s->cluster_size); + l2_offset, s->cluster_size, + false); if (ret < 0) { fprintf(stderr, "ERROR: Could not write L2 table; metadata " "overlap check failed: %s\n", strerror(-ret)); @@ -2366,7 +2379,7 @@ write_refblocks: } ret = qcow2_pre_write_overlap_check(bs, 0, refblock_offset, - s->cluster_size); + s->cluster_size, false); if (ret < 0) { fprintf(stderr, "ERROR writing refblock: %s\n", strerror(-ret)); goto fail; @@ -2417,7 +2430,8 @@ write_refblocks: } ret = qcow2_pre_write_overlap_check(bs, 0, reftable_offset, - reftable_size * sizeof(uint64_t)); + reftable_size * sizeof(uint64_t), + false); if (ret < 0) { fprintf(stderr, "ERROR writing reftable: %s\n", strerror(-ret)); goto fail; @@ -2751,10 +2765,15 @@ QEMU_BUILD_BUG_ON(QCOW2_OL_MAX_BITNR != ARRAY_SIZE(metadata_ol_names)); * overlaps; or a negative value (-errno) on error. */ int qcow2_pre_write_overlap_check(BlockDriverState *bs, int ign, int64_t offset, - int64_t size) + int64_t size, bool data_file) { - int ret = qcow2_check_metadata_overlap(bs, ign, offset, size); + int ret; + + if (data_file && has_data_file(bs)) { + return 0; + } + ret = qcow2_check_metadata_overlap(bs, ign, offset, size); if (ret < 0) { return ret; } else if (ret > 0) { @@ -2855,7 +2874,8 @@ static int flush_refblock(BlockDriverState *bs, uint64_t **reftable, if (reftable_index < *reftable_size && (*reftable)[reftable_index]) { offset = (*reftable)[reftable_index]; - ret = qcow2_pre_write_overlap_check(bs, 0, offset, s->cluster_size); + ret = qcow2_pre_write_overlap_check(bs, 0, offset, s->cluster_size, + false); if (ret < 0) { error_setg_errno(errp, -ret, "Overlap check failed"); return ret; @@ -3121,7 +3141,8 @@ int qcow2_change_refcount_order(BlockDriverState *bs, int refcount_order, /* Write the new reftable */ ret = qcow2_pre_write_overlap_check(bs, 0, new_reftable_offset, - new_reftable_size * sizeof(uint64_t)); + new_reftable_size * sizeof(uint64_t), + false); if (ret < 0) { error_setg_errno(errp, -ret, "Overlap check failed"); goto done; diff --git a/block/qcow2-snapshot.c b/block/qcow2-snapshot.c index 20e8472191..5ae3407f68 100644 --- a/block/qcow2-snapshot.c +++ b/block/qcow2-snapshot.c @@ -184,7 +184,7 @@ static int qcow2_write_snapshots(BlockDriverState *bs) /* The snapshot list position has not yet been updated, so these clusters * must indeed be completely free */ - ret = qcow2_pre_write_overlap_check(bs, 0, offset, snapshots_size); + ret = qcow2_pre_write_overlap_check(bs, 0, offset, snapshots_size, false); if (ret < 0) { goto fail; } @@ -389,7 +389,7 @@ int qcow2_snapshot_create(BlockDriverState *bs, QEMUSnapshotInfo *sn_info) } ret = qcow2_pre_write_overlap_check(bs, 0, sn->l1_table_offset, - s->l1_size * sizeof(uint64_t)); + s->l1_size * sizeof(uint64_t), false); if (ret < 0) { goto fail; } @@ -528,7 +528,8 @@ int qcow2_snapshot_goto(BlockDriverState *bs, const char *snapshot_id) } ret = qcow2_pre_write_overlap_check(bs, QCOW2_OL_ACTIVE_L1, - s->l1_table_offset, cur_l1_bytes); + s->l1_table_offset, cur_l1_bytes, + false); if (ret < 0) { goto fail; } diff --git a/block/qcow2.c b/block/qcow2.c index 8dc6f21047..b77d856007 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -140,7 +140,7 @@ static ssize_t qcow2_crypto_hdr_init_func(QCryptoBlock *block, size_t headerlen, /* Zero fill remaining space in cluster so it has predictable * content in case of future spec changes */ clusterlen = size_to_clusters(s, headerlen) * s->cluster_size; - assert(qcow2_pre_write_overlap_check(bs, 0, ret, clusterlen) == 0); + assert(qcow2_pre_write_overlap_check(bs, 0, ret, clusterlen, false) == 0); ret = bdrv_pwrite_zeroes(bs->file, ret + headerlen, clusterlen - headerlen, 0); @@ -1953,7 +1953,7 @@ static coroutine_fn int qcow2_co_preadv(BlockDriverState *bs, uint64_t offset, */ if (!cluster_data) { cluster_data = - qemu_try_blockalign(bs->file->bs, + qemu_try_blockalign(s->data_file->bs, QCOW_MAX_CRYPT_CLUSTERS * s->cluster_size); if (cluster_data == NULL) { @@ -1969,7 +1969,7 @@ static coroutine_fn int qcow2_co_preadv(BlockDriverState *bs, uint64_t offset, BLKDBG_EVENT(bs->file, BLKDBG_READ_AIO); qemu_co_mutex_unlock(&s->lock); - ret = bdrv_co_preadv(bs->file, + ret = bdrv_co_preadv(s->data_file, cluster_offset + offset_in_cluster, cur_bytes, &hd_qiov, 0); qemu_co_mutex_lock(&s->lock); @@ -2128,7 +2128,7 @@ static coroutine_fn int qcow2_co_pwritev(BlockDriverState *bs, uint64_t offset, } ret = qcow2_pre_write_overlap_check(bs, 0, - cluster_offset + offset_in_cluster, cur_bytes); + cluster_offset + offset_in_cluster, cur_bytes, true); if (ret < 0) { goto fail; } @@ -2142,7 +2142,7 @@ static coroutine_fn int qcow2_co_pwritev(BlockDriverState *bs, uint64_t offset, BLKDBG_EVENT(bs->file, BLKDBG_WRITE_AIO); trace_qcow2_writev_data(qemu_coroutine_self(), cluster_offset + offset_in_cluster); - ret = bdrv_co_pwritev(bs->file, + ret = bdrv_co_pwritev(s->data_file, cluster_offset + offset_in_cluster, cur_bytes, &hd_qiov, 0); qemu_co_mutex_lock(&s->lock); @@ -3344,7 +3344,7 @@ qcow2_co_copy_range_from(BlockDriverState *bs, goto out; case QCOW2_CLUSTER_NORMAL: - child = bs->file; + child = s->data_file; copy_offset += offset_into_cluster(s, src_offset); if ((copy_offset & 511) != 0) { ret = -EIO; @@ -3414,14 +3414,14 @@ qcow2_co_copy_range_to(BlockDriverState *bs, assert((cluster_offset & 511) == 0); ret = qcow2_pre_write_overlap_check(bs, 0, - cluster_offset + offset_in_cluster, cur_bytes); + cluster_offset + offset_in_cluster, cur_bytes, true); if (ret < 0) { goto fail; } qemu_co_mutex_unlock(&s->lock); ret = bdrv_co_copy_range_to(src, src_offset, - bs->file, + s->data_file, cluster_offset + offset_in_cluster, cur_bytes, read_flags, write_flags); qemu_co_mutex_lock(&s->lock); @@ -3576,6 +3576,17 @@ static int coroutine_fn qcow2_co_truncate(BlockDriverState *bs, int64_t offset, int64_t old_file_size, new_file_size; uint64_t nb_new_data_clusters, nb_new_l2_tables; + /* With a data file, preallocation means just allocating the metadata + * and forwarding the truncate request to the data file */ + if (has_data_file(bs)) { + ret = preallocate_co(bs, old_length, offset); + if (ret < 0) { + error_setg_errno(errp, -ret, "Preallocation failed"); + goto fail; + } + break; + } + old_file_size = bdrv_getlength(bs->file->bs); if (old_file_size < 0) { error_setg_errno(errp, -old_file_size, @@ -3684,6 +3695,16 @@ static int coroutine_fn qcow2_co_truncate(BlockDriverState *bs, int64_t offset, bs->total_sectors = offset / BDRV_SECTOR_SIZE; + if (has_data_file(bs)) { + if (prealloc == PREALLOC_MODE_METADATA) { + prealloc = PREALLOC_MODE_OFF; + } + ret = bdrv_co_truncate(s->data_file, offset, prealloc, errp); + if (ret < 0) { + goto fail; + } + } + /* write updated header.size */ offset = cpu_to_be64(offset); ret = bdrv_pwrite_sync(bs->file, offsetof(QCowHeader, size), @@ -3887,6 +3908,10 @@ qcow2_co_pwritev_compressed(BlockDriverState *bs, uint64_t offset, uint8_t *buf, *out_buf; uint64_t cluster_offset; + if (has_data_file(bs)) { + return -ENOTSUP; + } + if (bytes == 0) { /* align end of file to a sector boundary to ease reading with sector based I/Os */ @@ -3938,7 +3963,7 @@ qcow2_co_pwritev_compressed(BlockDriverState *bs, uint64_t offset, goto fail; } - ret = qcow2_pre_write_overlap_check(bs, 0, cluster_offset, out_len); + ret = qcow2_pre_write_overlap_check(bs, 0, cluster_offset, out_len, true); qemu_co_mutex_unlock(&s->lock); if (ret < 0) { goto fail; @@ -3950,8 +3975,8 @@ qcow2_co_pwritev_compressed(BlockDriverState *bs, uint64_t offset, }; qemu_iovec_init_external(&hd_qiov, &iov, 1); - BLKDBG_EVENT(bs->file, BLKDBG_WRITE_COMPRESSED); - ret = bdrv_co_pwritev(bs->file, cluster_offset, out_len, &hd_qiov, 0); + BLKDBG_EVENT(s->data_file, BLKDBG_WRITE_COMPRESSED); + ret = bdrv_co_pwritev(s->data_file, cluster_offset, out_len, &hd_qiov, 0); if (ret < 0) { goto fail; } @@ -4543,6 +4568,11 @@ static int qcow2_downgrade(BlockDriverState *bs, int target_version, return -ENOTSUP; } + if (has_data_file(bs)) { + error_setg(errp, "Cannot downgrade an image with a data file"); + return -ENOTSUP; + } + /* clear incompatible features */ if (s->incompatible_features & QCOW2_INCOMPAT_DIRTY) { ret = qcow2_mark_clean(bs); From patchwork Wed Feb 27 17:22:48 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 1049056 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 448jdM62nsz9s1b for ; Thu, 28 Feb 2019 04:40:39 +1100 (AEDT) Received: from localhost ([127.0.0.1]:48148 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gz3C9-0006yU-Mg for incoming@patchwork.ozlabs.org; Wed, 27 Feb 2019 12:40:37 -0500 Received: from eggs.gnu.org ([209.51.188.92]:51780) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gz2wf-0003s3-Ak for qemu-devel@nongnu.org; Wed, 27 Feb 2019 12:24:38 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gz2wc-0003wS-V2 for qemu-devel@nongnu.org; Wed, 27 Feb 2019 12:24:36 -0500 Received: from mx1.redhat.com ([209.132.183.28]:42302) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1gz2wY-0003Gb-M8; Wed, 27 Feb 2019 12:24:30 -0500 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 3DC8D30C23DE; Wed, 27 Feb 2019 17:23:41 +0000 (UTC) Received: from linux.fritz.box.com (ovpn-117-19.ams2.redhat.com [10.36.117.19]) by smtp.corp.redhat.com (Postfix) with ESMTP id 0EFD417153; Wed, 27 Feb 2019 17:23:37 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Date: Wed, 27 Feb 2019 18:22:48 +0100 Message-Id: <20190227172256.30368-13-kwolf@redhat.com> In-Reply-To: <20190227172256.30368-1-kwolf@redhat.com> References: <20190227172256.30368-1-kwolf@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.44]); Wed, 27 Feb 2019 17:23:41 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH 12/20] qcow2: Return error for snapshot operation with data file X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org, mreitz@redhat.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Internal snapshots and an external data file are incompatible because snapshots require refcounting and non-linear mapping. Return an error for all of the snapshot operations if an external data file is in use. Signed-off-by: Kevin Wolf --- block/qcow2-snapshot.c | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/block/qcow2-snapshot.c b/block/qcow2-snapshot.c index 5ae3407f68..a6ffae89a6 100644 --- a/block/qcow2-snapshot.c +++ b/block/qcow2-snapshot.c @@ -353,6 +353,10 @@ int qcow2_snapshot_create(BlockDriverState *bs, QEMUSnapshotInfo *sn_info) return -EFBIG; } + if (has_data_file(bs)) { + return -ENOTSUP; + } + memset(sn, 0, sizeof(*sn)); /* Generate an ID */ @@ -466,6 +470,10 @@ int qcow2_snapshot_goto(BlockDriverState *bs, const char *snapshot_id) int ret; uint64_t *sn_l1_table = NULL; + if (has_data_file(bs)) { + return -ENOTSUP; + } + /* Search the snapshot */ snapshot_index = find_snapshot_by_id_or_name(bs, snapshot_id); if (snapshot_index < 0) { @@ -599,6 +607,10 @@ int qcow2_snapshot_delete(BlockDriverState *bs, QCowSnapshot sn; int snapshot_index, ret; + if (has_data_file(bs)) { + return -ENOTSUP; + } + /* Search the snapshot */ snapshot_index = find_snapshot_by_id_and_name(bs, snapshot_id, name); if (snapshot_index < 0) { @@ -670,6 +682,9 @@ int qcow2_snapshot_list(BlockDriverState *bs, QEMUSnapshotInfo **psn_tab) QCowSnapshot *sn; int i; + if (has_data_file(bs)) { + return -ENOTSUP; + } if (!s->nb_snapshots) { *psn_tab = NULL; return s->nb_snapshots; From patchwork Wed Feb 27 17:22:49 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 1049042 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 448jTM4CQHz9s70 for ; Thu, 28 Feb 2019 04:33:43 +1100 (AEDT) Received: from localhost ([127.0.0.1]:48046 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gz35R-00023g-GT for incoming@patchwork.ozlabs.org; Wed, 27 Feb 2019 12:33:41 -0500 Received: from eggs.gnu.org ([209.51.188.92]:51492) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gz2wT-0003fN-NO for qemu-devel@nongnu.org; Wed, 27 Feb 2019 12:24:29 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gz2wL-0003dM-UN for qemu-devel@nongnu.org; Wed, 27 Feb 2019 12:24:23 -0500 Received: from mx1.redhat.com ([209.132.183.28]:57262) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1gz2wB-0003H8-I4; Wed, 27 Feb 2019 12:24:07 -0500 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id BFD26C0AC902; Wed, 27 Feb 2019 17:23:42 +0000 (UTC) Received: from linux.fritz.box.com (ovpn-117-19.ams2.redhat.com [10.36.117.19]) by smtp.corp.redhat.com (Postfix) with ESMTP id 89BAB17153; Wed, 27 Feb 2019 17:23:41 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Date: Wed, 27 Feb 2019 18:22:49 +0100 Message-Id: <20190227172256.30368-14-kwolf@redhat.com> In-Reply-To: <20190227172256.30368-1-kwolf@redhat.com> References: <20190227172256.30368-1-kwolf@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.31]); Wed, 27 Feb 2019 17:23:42 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH 13/20] qcow2: Support external data file in qemu-img check X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org, mreitz@redhat.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" For external data files, data clusters must be excluded from the refcount calculations. Instead, an implicit refcount of 1 is assumed for the COPIED flag. Compressed clusters and internal snapshots are incompatible with external data files, so print an error if they are in use for images with an external data file. Signed-off-by: Kevin Wolf --- block/qcow2-refcount.c | 41 ++++++++++++++++++++++++++++++----------- 1 file changed, 30 insertions(+), 11 deletions(-) diff --git a/block/qcow2-refcount.c b/block/qcow2-refcount.c index df73580e5d..e0fe322500 100644 --- a/block/qcow2-refcount.c +++ b/block/qcow2-refcount.c @@ -1605,6 +1605,13 @@ static int check_refcounts_l2(BlockDriverState *bs, BdrvCheckResult *res, res->corruptions++; } + if (has_data_file(bs)) { + fprintf(stderr, "ERROR compressed cluster %d with data file, " + "entry=0x%" PRIx64 "\n", i, l2_entry); + res->corruptions++; + break; + } + /* Mark cluster as used */ nb_csectors = ((l2_entry >> s->csize_shift) & s->csize_mask) + 1; @@ -1695,11 +1702,13 @@ static int check_refcounts_l2(BlockDriverState *bs, BdrvCheckResult *res, } /* Mark cluster as used */ - ret = qcow2_inc_refcounts_imrt(bs, res, - refcount_table, refcount_table_size, - offset, s->cluster_size); - if (ret < 0) { - goto fail; + if (!has_data_file(bs)) { + ret = qcow2_inc_refcounts_imrt(bs, res, refcount_table, + refcount_table_size, + offset, s->cluster_size); + if (ret < 0) { + goto fail; + } } break; } @@ -1884,12 +1893,16 @@ static int check_oflag_copied(BlockDriverState *bs, BdrvCheckResult *res, if (cluster_type == QCOW2_CLUSTER_NORMAL || cluster_type == QCOW2_CLUSTER_ZERO_ALLOC) { - ret = qcow2_get_refcount(bs, - data_offset >> s->cluster_bits, - &refcount); - if (ret < 0) { - /* don't print message nor increment check_errors */ - continue; + if (has_data_file(bs)) { + refcount = 1; + } else { + ret = qcow2_get_refcount(bs, + data_offset >> s->cluster_bits, + &refcount); + if (ret < 0) { + /* don't print message nor increment check_errors */ + continue; + } } if ((refcount == 1) != ((l2_entry & QCOW_OFLAG_COPIED) != 0)) { fprintf(stderr, "%s OFLAG_COPIED data cluster: " @@ -2083,6 +2096,12 @@ static int calculate_refcounts(BlockDriverState *bs, BdrvCheckResult *res, } /* snapshots */ + if (has_data_file(bs) && s->nb_snapshots) { + fprintf(stderr, "ERROR %d snapshots in image with data file\n", + s->nb_snapshots); + res->corruptions++; + } + for (i = 0; i < s->nb_snapshots; i++) { sn = s->snapshots + i; if (offset_into_cluster(s, sn->l1_table_offset)) { From patchwork Wed Feb 27 17:22:50 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 1049035 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 448jLW6W6Fz9s70 for ; Thu, 28 Feb 2019 04:27:47 +1100 (AEDT) Received: from localhost ([127.0.0.1]:47948 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gz2zh-0005iU-OS for incoming@patchwork.ozlabs.org; Wed, 27 Feb 2019 12:27:45 -0500 Received: from eggs.gnu.org ([209.51.188.92]:51504) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gz2wT-0003fY-PS for qemu-devel@nongnu.org; Wed, 27 Feb 2019 12:24:29 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gz2wO-0003em-69 for qemu-devel@nongnu.org; Wed, 27 Feb 2019 12:24:25 -0500 Received: from mx1.redhat.com ([209.132.183.28]:9319) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1gz2wB-0003Hg-Kb; Wed, 27 Feb 2019 12:24:07 -0500 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 4FF0E30BDA10; Wed, 27 Feb 2019 17:23:44 +0000 (UTC) Received: from linux.fritz.box.com (ovpn-117-19.ams2.redhat.com [10.36.117.19]) by smtp.corp.redhat.com (Postfix) with ESMTP id 19A4517153; Wed, 27 Feb 2019 17:23:42 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Date: Wed, 27 Feb 2019 18:22:50 +0100 Message-Id: <20190227172256.30368-15-kwolf@redhat.com> In-Reply-To: <20190227172256.30368-1-kwolf@redhat.com> References: <20190227172256.30368-1-kwolf@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.46]); Wed, 27 Feb 2019 17:23:44 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH 14/20] qcow2: Add basic data-file infrastructure X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org, mreitz@redhat.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" This adds a .bdrv_open option to specify the external data file node. Signed-off-by: Kevin Wolf --- qapi/block-core.json | 7 ++++++- block/qcow2.h | 4 +++- block/qcow2.c | 34 ++++++++++++++++++++++++++++++++-- 3 files changed, 41 insertions(+), 4 deletions(-) diff --git a/qapi/block-core.json b/qapi/block-core.json index 2b8afbb924..de4d4fd0e4 100644 --- a/qapi/block-core.json +++ b/qapi/block-core.json @@ -3080,6 +3080,10 @@ # encrypted images, except when doing a metadata-only # probe of the image. (since 2.10) # +# @data-file: reference to or definition of the external data file. +# This may only be specified for images that require an +# external data file. (since 4.0) +# # Since: 2.9 ## { 'struct': 'BlockdevOptionsQcow2', @@ -3094,7 +3098,8 @@ '*l2-cache-entry-size': 'int', '*refcount-cache-size': 'int', '*cache-clean-interval': 'int', - '*encrypt': 'BlockdevQcow2Encryption' } } + '*encrypt': 'BlockdevQcow2Encryption', + '*data-file': 'BlockdevRef' } } ## # @SshHostKeyCheckMode: diff --git a/block/qcow2.h b/block/qcow2.h index aac7fc4348..f23c003a46 100644 --- a/block/qcow2.h +++ b/block/qcow2.h @@ -91,6 +91,7 @@ #define DEFAULT_CLUSTER_SIZE 65536 +#define QCOW2_OPT_DATA_FILE "data-file" #define QCOW2_OPT_LAZY_REFCOUNTS "lazy-refcounts" #define QCOW2_OPT_DISCARD_REQUEST "pass-discard-request" #define QCOW2_OPT_DISCARD_SNAPSHOT "pass-discard-snapshot" @@ -205,7 +206,8 @@ enum { QCOW2_INCOMPAT_DATA_FILE = 1 << QCOW2_INCOMPAT_DATA_FILE_BITNR, QCOW2_INCOMPAT_MASK = QCOW2_INCOMPAT_DIRTY - | QCOW2_INCOMPAT_CORRUPT, + | QCOW2_INCOMPAT_CORRUPT + | QCOW2_INCOMPAT_DATA_FILE, }; /* Compatible feature bits */ diff --git a/block/qcow2.c b/block/qcow2.c index b77d856007..37ab645e7b 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -1441,8 +1441,31 @@ static int coroutine_fn qcow2_do_open(BlockDriverState *bs, QDict *options, goto fail; } - /* TODO Open external data file */ - s->data_file = bs->file; + /* Open external data file */ + s->data_file = bdrv_open_child(NULL, options, "data-file", bs, &child_file, + true, &local_err); + if (local_err) { + error_propagate(errp, local_err); + ret = -EINVAL; + goto fail; + } + + if (s->incompatible_features & QCOW2_INCOMPAT_DATA_FILE) { + if (!s->data_file) { + error_setg(errp, "'data-file' is required for this image"); + ret = -EINVAL; + goto fail; + } + } else { + if (s->data_file) { + error_setg(errp, "'data-file' can only be set for images with an " + "external data file"); + ret = -EINVAL; + goto fail; + } else { + s->data_file = bs->file; + } + } /* qcow2_read_extension may have set up the crypto context * if the crypt method needs a header region, some methods @@ -1615,6 +1638,9 @@ static int coroutine_fn qcow2_do_open(BlockDriverState *bs, QDict *options, return ret; fail: + if (has_data_file(bs)) { + bdrv_unref_child(bs, s->data_file); + } g_free(s->unknown_header_fields); cleanup_unknown_header_ext(bs); qcow2_free_snapshots(bs); @@ -2234,6 +2260,10 @@ static void qcow2_close(BlockDriverState *bs) g_free(s->image_backing_file); g_free(s->image_backing_format); + if (has_data_file(bs)) { + bdrv_unref_child(bs, s->data_file); + } + qcow2_refcount_close(bs); qcow2_free_snapshots(bs); } From patchwork Wed Feb 27 17:22:51 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 1049037 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 448jNg5jJkz9s70 for ; Thu, 28 Feb 2019 04:29:39 +1100 (AEDT) Received: from localhost ([127.0.0.1]:47978 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gz31V-0007Gb-LW for incoming@patchwork.ozlabs.org; Wed, 27 Feb 2019 12:29:37 -0500 Received: from eggs.gnu.org ([209.51.188.92]:51840) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gz2wi-0003w0-FO for qemu-devel@nongnu.org; Wed, 27 Feb 2019 12:24:41 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gz2wf-0003yP-BF for qemu-devel@nongnu.org; Wed, 27 Feb 2019 12:24:40 -0500 Received: from mx1.redhat.com ([209.132.183.28]:45673) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1gz2wY-0003JM-JE; Wed, 27 Feb 2019 12:24:30 -0500 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id CD7AC7DCE1; Wed, 27 Feb 2019 17:23:45 +0000 (UTC) Received: from linux.fritz.box.com (ovpn-117-19.ams2.redhat.com [10.36.117.19]) by smtp.corp.redhat.com (Postfix) with ESMTP id 9B2C717153; Wed, 27 Feb 2019 17:23:44 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Date: Wed, 27 Feb 2019 18:22:51 +0100 Message-Id: <20190227172256.30368-16-kwolf@redhat.com> In-Reply-To: <20190227172256.30368-1-kwolf@redhat.com> References: <20190227172256.30368-1-kwolf@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.27]); Wed, 27 Feb 2019 17:23:45 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH 15/20] qcow2: Creating images with external data file X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org, mreitz@redhat.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" This adds a .bdrv_create option to use an external data file. Signed-off-by: Kevin Wolf --- qapi/block-core.json | 4 ++++ include/block/block_int.h | 1 + block/qcow2.c | 26 ++++++++++++++++++++++++++ 3 files changed, 31 insertions(+) diff --git a/qapi/block-core.json b/qapi/block-core.json index de4d4fd0e4..2303266bc4 100644 --- a/qapi/block-core.json +++ b/qapi/block-core.json @@ -4135,6 +4135,9 @@ # Driver specific image creation options for qcow2. # # @file Node to create the image format on +# @data-file Node to use as an external data file in which all guest +# data is stored so that only metadata remains in the qcow2 +# file (since: 4.0) # @size Size of the virtual disk in bytes # @version Compatibility level (default: v3) # @backing-file File name of the backing file if a backing file @@ -4150,6 +4153,7 @@ ## { 'struct': 'BlockdevCreateOptionsQcow2', 'data': { 'file': 'BlockdevRef', + '*data-file': 'BlockdevRef', 'size': 'size', '*version': 'BlockdevQcow2Version', '*backing-file': 'str', diff --git a/include/block/block_int.h b/include/block/block_int.h index 836d67c1ae..acd29ce521 100644 --- a/include/block/block_int.h +++ b/include/block/block_int.h @@ -56,6 +56,7 @@ #define BLOCK_OPT_NOCOW "nocow" #define BLOCK_OPT_OBJECT_SIZE "object_size" #define BLOCK_OPT_REFCOUNT_BITS "refcount_bits" +#define BLOCK_OPT_DATA_FILE "data_file" #define BLOCK_PROBE_BUF_SIZE 512 diff --git a/block/qcow2.c b/block/qcow2.c index 37ab645e7b..a6144689ea 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -2868,6 +2868,7 @@ qcow2_co_create(BlockdevCreateOptions *create_options, Error **errp) */ BlockBackend *blk = NULL; BlockDriverState *bs = NULL; + BlockDriverState *data_bs = NULL; QCowHeader *header; size_t cluster_size; int version; @@ -2964,6 +2965,20 @@ qcow2_co_create(BlockdevCreateOptions *create_options, Error **errp) } refcount_order = ctz32(qcow2_opts->refcount_bits); + if (qcow2_opts->data_file) { + if (version < 3) { + error_setg(errp, "External data files are only supported with " + "compatibility level 1.1 and above (use version=v3 or " + "greater)"); + ret = -EINVAL; + goto out; + } + data_bs = bdrv_open_blockdev_ref(qcow2_opts->data_file, errp); + if (bs == NULL) { + ret = -EIO; + goto out; + } + } /* Create BlockBackend to write to the image */ blk = blk_new(BLK_PERM_WRITE | BLK_PERM_RESIZE, BLK_PERM_ALL); @@ -3002,6 +3017,10 @@ qcow2_co_create(BlockdevCreateOptions *create_options, Error **errp) header->compatible_features |= cpu_to_be64(QCOW2_COMPAT_LAZY_REFCOUNTS); } + if (data_bs) { + header->incompatible_features |= + cpu_to_be64(QCOW2_INCOMPAT_DATA_FILE); + } ret = blk_pwrite(blk, 0, header, cluster_size, 0); g_free(header); @@ -3032,6 +3051,9 @@ qcow2_co_create(BlockdevCreateOptions *create_options, Error **errp) options = qdict_new(); qdict_put_str(options, "driver", "qcow2"); qdict_put_str(options, "file", bs->node_name); + if (data_bs) { + qdict_put_str(options, "data-file", data_bs->node_name); + } blk = blk_new_open(NULL, NULL, options, BDRV_O_RDWR | BDRV_O_RESIZE | BDRV_O_NO_FLUSH, &local_err); @@ -3104,6 +3126,9 @@ qcow2_co_create(BlockdevCreateOptions *create_options, Error **errp) options = qdict_new(); qdict_put_str(options, "driver", "qcow2"); qdict_put_str(options, "file", bs->node_name); + if (data_bs) { + qdict_put_str(options, "data-file", data_bs->node_name); + } blk = blk_new_open(NULL, NULL, options, BDRV_O_RDWR | BDRV_O_NO_BACKING | BDRV_O_NO_IO, &local_err); @@ -3117,6 +3142,7 @@ qcow2_co_create(BlockdevCreateOptions *create_options, Error **errp) out: blk_unref(blk); bdrv_unref(bs); + bdrv_unref(data_bs); return ret; } From patchwork Wed Feb 27 17:22:52 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 1049054 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 448jYx641wz9sBF for ; Thu, 28 Feb 2019 04:37:41 +1100 (AEDT) Received: from localhost ([127.0.0.1]:48124 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gz39H-0004wn-Op for incoming@patchwork.ozlabs.org; Wed, 27 Feb 2019 12:37:39 -0500 Received: from eggs.gnu.org ([209.51.188.92]:51910) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gz2wm-00040z-T2 for qemu-devel@nongnu.org; Wed, 27 Feb 2019 12:24:47 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gz2wi-00042j-EA for qemu-devel@nongnu.org; Wed, 27 Feb 2019 12:24:44 -0500 Received: from mx1.redhat.com ([209.132.183.28]:15402) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1gz2wU-0003KA-5C; Wed, 27 Feb 2019 12:24:28 -0500 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 63676300561F; Wed, 27 Feb 2019 17:23:47 +0000 (UTC) Received: from linux.fritz.box.com (ovpn-117-19.ams2.redhat.com [10.36.117.19]) by smtp.corp.redhat.com (Postfix) with ESMTP id 2A3E6171A0; Wed, 27 Feb 2019 17:23:46 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Date: Wed, 27 Feb 2019 18:22:52 +0100 Message-Id: <20190227172256.30368-17-kwolf@redhat.com> In-Reply-To: <20190227172256.30368-1-kwolf@redhat.com> References: <20190227172256.30368-1-kwolf@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.44]); Wed, 27 Feb 2019 17:23:47 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH 16/20] qcow2: Store data file name in the image X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org, mreitz@redhat.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Rather than requiring that the external data file node is passed explicitly when creating the qcow2 node, store the filename in the designated header extension during .bdrv_create and read it from there as a default during .bdrv_open. Signed-off-by: Kevin Wolf --- qapi/block-core.json | 8 +++- block/qcow2.h | 1 + block/qcow2.c | 94 +++++++++++++++++++++++++++++++++++++- tests/qemu-iotests/082.out | 27 +++++++++++ 4 files changed, 128 insertions(+), 2 deletions(-) diff --git a/qapi/block-core.json b/qapi/block-core.json index 2303266bc4..e6faa94fa2 100644 --- a/qapi/block-core.json +++ b/qapi/block-core.json @@ -59,6 +59,9 @@ # # @compat: compatibility level # +# @data-file: the filename of the external data file that is stored in the +# image and used as a default for opening the image (since: 4.0) +# # @lazy-refcounts: on or off; only valid for compat >= 1.1 # # @corrupt: true if the image has been marked corrupt; only valid for @@ -76,6 +79,7 @@ { 'struct': 'ImageInfoSpecificQCow2', 'data': { 'compat': 'str', + '*data-file': 'str', '*lazy-refcounts': 'bool', '*corrupt': 'bool', 'refcount-bits': 'int', @@ -3082,7 +3086,9 @@ # # @data-file: reference to or definition of the external data file. # This may only be specified for images that require an -# external data file. (since 4.0) +# external data file. If it is not specified for such +# an image, the data file name is loaded from the image +# file. (since 4.0) # # Since: 2.9 ## diff --git a/block/qcow2.h b/block/qcow2.h index f23c003a46..a9c9cb4a26 100644 --- a/block/qcow2.h +++ b/block/qcow2.h @@ -343,6 +343,7 @@ typedef struct BDRVQcow2State { * override) */ char *image_backing_file; char *image_backing_format; + char *image_data_file; CoQueue compress_wait_queue; int nb_compress_threads; diff --git a/block/qcow2.c b/block/qcow2.c index a6144689ea..24c023e13d 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -398,6 +398,21 @@ static int qcow2_read_extensions(BlockDriverState *bs, uint64_t start_offset, #endif break; + case QCOW2_EXT_MAGIC_DATA_FILE: + { + s->image_data_file = g_malloc0(ext.len + 1); + ret = bdrv_pread(bs->file, offset, s->image_data_file, ext.len); + if (ret < 0) { + error_setg_errno(errp, -ret, + "ERROR: Could not read data file name"); + return ret; + } +#ifdef DEBUG_EXT + printf("Qcow2: Got external data file %s\n", s->image_data_file); +#endif + break; + } + default: /* unknown magic - save it in case we need to rewrite the header */ /* If you add a new feature, make sure to also update the fast @@ -1451,6 +1466,15 @@ static int coroutine_fn qcow2_do_open(BlockDriverState *bs, QDict *options, } if (s->incompatible_features & QCOW2_INCOMPAT_DATA_FILE) { + if (!s->data_file && s->image_data_file) { + s->data_file = bdrv_open_child(s->image_data_file, options, + "data-file", bs, &child_file, + false, errp); + if (!s->data_file) { + ret = -EINVAL; + goto fail; + } + } if (!s->data_file) { error_setg(errp, "'data-file' is required for this image"); ret = -EINVAL; @@ -1638,6 +1662,7 @@ static int coroutine_fn qcow2_do_open(BlockDriverState *bs, QDict *options, return ret; fail: + g_free(s->image_data_file); if (has_data_file(bs)) { bdrv_unref_child(bs, s->data_file); } @@ -2257,6 +2282,7 @@ static void qcow2_close(BlockDriverState *bs) g_free(s->unknown_header_fields); cleanup_unknown_header_ext(bs); + g_free(s->image_data_file); g_free(s->image_backing_file); g_free(s->image_backing_format); @@ -2433,6 +2459,19 @@ int qcow2_update_header(BlockDriverState *bs) buflen -= ret; } + /* External data file header extension */ + if (has_data_file(bs) && s->image_data_file) { + ret = header_ext_add(buf, QCOW2_EXT_MAGIC_DATA_FILE, + s->image_data_file, strlen(s->image_data_file), + buflen); + if (ret < 0) { + goto fail; + } + + buf += ret; + buflen -= ret; + } + /* Full disk encryption header pointer extension */ if (s->crypto_header.offset != 0) { s->crypto_header.offset = cpu_to_be64(s->crypto_header.offset); @@ -3074,6 +3113,12 @@ qcow2_co_create(BlockdevCreateOptions *create_options, Error **errp) abort(); } + /* Set the external data file if necessary */ + if (data_bs) { + BDRVQcow2State *s = blk_bs(blk)->opaque; + s->image_data_file = g_strdup(data_bs->filename); + } + /* Create a full header (including things like feature table) */ ret = qcow2_update_header(blk_bs(blk)); if (ret < 0) { @@ -3153,6 +3198,7 @@ static int coroutine_fn qcow2_co_create_opts(const char *filename, QemuOpts *opt QDict *qdict; Visitor *v; BlockDriverState *bs = NULL; + BlockDriverState *data_bs = NULL; Error *local_err = NULL; const char *val; int ret; @@ -3216,6 +3262,26 @@ static int coroutine_fn qcow2_co_create_opts(const char *filename, QemuOpts *opt goto finish; } + /* Create and open an external data file (protocol layer) */ + val = qdict_get_try_str(qdict, BLOCK_OPT_DATA_FILE); + if (val) { + ret = bdrv_create_file(val, opts, errp); + if (ret < 0) { + goto finish; + } + + data_bs = bdrv_open(val, NULL, NULL, + BDRV_O_RDWR | BDRV_O_RESIZE | BDRV_O_PROTOCOL, + errp); + if (data_bs == NULL) { + ret = -EIO; + goto finish; + } + + qdict_del(qdict, BLOCK_OPT_DATA_FILE); + qdict_put_str(qdict, "data-file", data_bs->node_name); + } + /* Set 'driver' and 'node' options */ qdict_put_str(qdict, "driver", "qcow2"); qdict_put_str(qdict, "file", bs->node_name); @@ -3250,6 +3316,7 @@ static int coroutine_fn qcow2_co_create_opts(const char *filename, QemuOpts *opt finish: qobject_unref(qdict); bdrv_unref(bs); + bdrv_unref(data_bs); qapi_free_BlockdevCreateOptions(create_options); return ret; } @@ -4548,6 +4615,8 @@ static ImageInfoSpecific *qcow2_get_specific_info(BlockDriverState *bs, .refcount_bits = s->refcount_bits, .has_bitmaps = !!bitmaps, .bitmaps = bitmaps, + .has_data_file = !!s->image_data_file, + .data_file = g_strdup(s->image_data_file), }; } else { /* if this assertion fails, this probably means a new version was @@ -4750,7 +4819,7 @@ static int qcow2_amend_options(BlockDriverState *bs, QemuOpts *opts, BDRVQcow2State *s = bs->opaque; int old_version = s->qcow_version, new_version = old_version; uint64_t new_size = 0; - const char *backing_file = NULL, *backing_format = NULL; + const char *backing_file = NULL, *backing_format = NULL, *data_file = NULL; bool lazy_refcounts = s->use_lazy_refcounts; const char *compat = NULL; uint64_t cluster_size = s->cluster_size; @@ -4832,6 +4901,13 @@ static int qcow2_amend_options(BlockDriverState *bs, QemuOpts *opts, "may not exceed 64 bits"); return -EINVAL; } + } else if (!strcmp(desc->name, BLOCK_OPT_DATA_FILE)) { + data_file = qemu_opt_get(opts, BLOCK_OPT_DATA_FILE); + if (data_file && !has_data_file(bs)) { + error_setg(errp, "data-file can only be set for images that " + "use an external data file"); + return -EINVAL; + } } else { /* if this point is reached, this probably means a new option was * added without having it covered here */ @@ -4878,6 +4954,17 @@ static int qcow2_amend_options(BlockDriverState *bs, QemuOpts *opts, } } + if (data_file) { + g_free(s->image_data_file); + s->image_data_file = *data_file ? g_strdup(data_file) : NULL; + } + + ret = qcow2_update_header(bs); + if (ret < 0) { + error_setg_errno(errp, -ret, "Failed to update the image header"); + return ret; + } + if (backing_file || backing_format) { ret = qcow2_change_backing_file(bs, backing_file ?: s->image_backing_file, @@ -5025,6 +5112,11 @@ static QemuOptsList qcow2_create_opts = { .type = QEMU_OPT_STRING, .help = "Image format of the base image" }, + { + .name = BLOCK_OPT_DATA_FILE, + .type = QEMU_OPT_STRING, + .help = "File name of an external data file" + }, { .name = BLOCK_OPT_ENCRYPT, .type = QEMU_OPT_BOOL, diff --git a/tests/qemu-iotests/082.out b/tests/qemu-iotests/082.out index 0ce18c075b..7dc59f6075 100644 --- a/tests/qemu-iotests/082.out +++ b/tests/qemu-iotests/082.out @@ -48,6 +48,7 @@ Supported options: backing_fmt= - Image format of the base image cluster_size= - qcow2 cluster size compat= - Compatibility level (0.10 or 1.1) + data_file= - File name of an external data file encrypt.cipher-alg= - Name of encryption cipher algorithm encrypt.cipher-mode= - Name of encryption cipher mode encrypt.format= - Encrypt the image, format choices: 'aes', 'luks' @@ -69,6 +70,7 @@ Supported options: backing_fmt= - Image format of the base image cluster_size= - qcow2 cluster size compat= - Compatibility level (0.10 or 1.1) + data_file= - File name of an external data file encrypt.cipher-alg= - Name of encryption cipher algorithm encrypt.cipher-mode= - Name of encryption cipher mode encrypt.format= - Encrypt the image, format choices: 'aes', 'luks' @@ -90,6 +92,7 @@ Supported options: backing_fmt= - Image format of the base image cluster_size= - qcow2 cluster size compat= - Compatibility level (0.10 or 1.1) + data_file= - File name of an external data file encrypt.cipher-alg= - Name of encryption cipher algorithm encrypt.cipher-mode= - Name of encryption cipher mode encrypt.format= - Encrypt the image, format choices: 'aes', 'luks' @@ -111,6 +114,7 @@ Supported options: backing_fmt= - Image format of the base image cluster_size= - qcow2 cluster size compat= - Compatibility level (0.10 or 1.1) + data_file= - File name of an external data file encrypt.cipher-alg= - Name of encryption cipher algorithm encrypt.cipher-mode= - Name of encryption cipher mode encrypt.format= - Encrypt the image, format choices: 'aes', 'luks' @@ -132,6 +136,7 @@ Supported options: backing_fmt= - Image format of the base image cluster_size= - qcow2 cluster size compat= - Compatibility level (0.10 or 1.1) + data_file= - File name of an external data file encrypt.cipher-alg= - Name of encryption cipher algorithm encrypt.cipher-mode= - Name of encryption cipher mode encrypt.format= - Encrypt the image, format choices: 'aes', 'luks' @@ -153,6 +158,7 @@ Supported options: backing_fmt= - Image format of the base image cluster_size= - qcow2 cluster size compat= - Compatibility level (0.10 or 1.1) + data_file= - File name of an external data file encrypt.cipher-alg= - Name of encryption cipher algorithm encrypt.cipher-mode= - Name of encryption cipher mode encrypt.format= - Encrypt the image, format choices: 'aes', 'luks' @@ -174,6 +180,7 @@ Supported options: backing_fmt= - Image format of the base image cluster_size= - qcow2 cluster size compat= - Compatibility level (0.10 or 1.1) + data_file= - File name of an external data file encrypt.cipher-alg= - Name of encryption cipher algorithm encrypt.cipher-mode= - Name of encryption cipher mode encrypt.format= - Encrypt the image, format choices: 'aes', 'luks' @@ -195,6 +202,7 @@ Supported options: backing_fmt= - Image format of the base image cluster_size= - qcow2 cluster size compat= - Compatibility level (0.10 or 1.1) + data_file= - File name of an external data file encrypt.cipher-alg= - Name of encryption cipher algorithm encrypt.cipher-mode= - Name of encryption cipher mode encrypt.format= - Encrypt the image, format choices: 'aes', 'luks' @@ -231,6 +239,7 @@ Supported options: backing_fmt= - Image format of the base image cluster_size= - qcow2 cluster size compat= - Compatibility level (0.10 or 1.1) + data_file= - File name of an external data file encrypt.cipher-alg= - Name of encryption cipher algorithm encrypt.cipher-mode= - Name of encryption cipher mode encrypt.format= - Encrypt the image, format choices: 'aes', 'luks' @@ -304,6 +313,7 @@ Supported options: backing_fmt= - Image format of the base image cluster_size= - qcow2 cluster size compat= - Compatibility level (0.10 or 1.1) + data_file= - File name of an external data file encrypt.cipher-alg= - Name of encryption cipher algorithm encrypt.cipher-mode= - Name of encryption cipher mode encrypt.format= - Encrypt the image, format choices: 'aes', 'luks' @@ -325,6 +335,7 @@ Supported options: backing_fmt= - Image format of the base image cluster_size= - qcow2 cluster size compat= - Compatibility level (0.10 or 1.1) + data_file= - File name of an external data file encrypt.cipher-alg= - Name of encryption cipher algorithm encrypt.cipher-mode= - Name of encryption cipher mode encrypt.format= - Encrypt the image, format choices: 'aes', 'luks' @@ -346,6 +357,7 @@ Supported options: backing_fmt= - Image format of the base image cluster_size= - qcow2 cluster size compat= - Compatibility level (0.10 or 1.1) + data_file= - File name of an external data file encrypt.cipher-alg= - Name of encryption cipher algorithm encrypt.cipher-mode= - Name of encryption cipher mode encrypt.format= - Encrypt the image, format choices: 'aes', 'luks' @@ -367,6 +379,7 @@ Supported options: backing_fmt= - Image format of the base image cluster_size= - qcow2 cluster size compat= - Compatibility level (0.10 or 1.1) + data_file= - File name of an external data file encrypt.cipher-alg= - Name of encryption cipher algorithm encrypt.cipher-mode= - Name of encryption cipher mode encrypt.format= - Encrypt the image, format choices: 'aes', 'luks' @@ -388,6 +401,7 @@ Supported options: backing_fmt= - Image format of the base image cluster_size= - qcow2 cluster size compat= - Compatibility level (0.10 or 1.1) + data_file= - File name of an external data file encrypt.cipher-alg= - Name of encryption cipher algorithm encrypt.cipher-mode= - Name of encryption cipher mode encrypt.format= - Encrypt the image, format choices: 'aes', 'luks' @@ -409,6 +423,7 @@ Supported options: backing_fmt= - Image format of the base image cluster_size= - qcow2 cluster size compat= - Compatibility level (0.10 or 1.1) + data_file= - File name of an external data file encrypt.cipher-alg= - Name of encryption cipher algorithm encrypt.cipher-mode= - Name of encryption cipher mode encrypt.format= - Encrypt the image, format choices: 'aes', 'luks' @@ -430,6 +445,7 @@ Supported options: backing_fmt= - Image format of the base image cluster_size= - qcow2 cluster size compat= - Compatibility level (0.10 or 1.1) + data_file= - File name of an external data file encrypt.cipher-alg= - Name of encryption cipher algorithm encrypt.cipher-mode= - Name of encryption cipher mode encrypt.format= - Encrypt the image, format choices: 'aes', 'luks' @@ -451,6 +467,7 @@ Supported options: backing_fmt= - Image format of the base image cluster_size= - qcow2 cluster size compat= - Compatibility level (0.10 or 1.1) + data_file= - File name of an external data file encrypt.cipher-alg= - Name of encryption cipher algorithm encrypt.cipher-mode= - Name of encryption cipher mode encrypt.format= - Encrypt the image, format choices: 'aes', 'luks' @@ -487,6 +504,7 @@ Supported options: backing_fmt= - Image format of the base image cluster_size= - qcow2 cluster size compat= - Compatibility level (0.10 or 1.1) + data_file= - File name of an external data file encrypt.cipher-alg= - Name of encryption cipher algorithm encrypt.cipher-mode= - Name of encryption cipher mode encrypt.format= - Encrypt the image, format choices: 'aes', 'luks' @@ -568,6 +586,7 @@ Creation options for 'qcow2': backing_fmt= - Image format of the base image cluster_size= - qcow2 cluster size compat= - Compatibility level (0.10 or 1.1) + data_file= - File name of an external data file encrypt.cipher-alg= - Name of encryption cipher algorithm encrypt.cipher-mode= - Name of encryption cipher mode encrypt.format= - Encrypt the image, format choices: 'aes', 'luks' @@ -590,6 +609,7 @@ Creation options for 'qcow2': backing_fmt= - Image format of the base image cluster_size= - qcow2 cluster size compat= - Compatibility level (0.10 or 1.1) + data_file= - File name of an external data file encrypt.cipher-alg= - Name of encryption cipher algorithm encrypt.cipher-mode= - Name of encryption cipher mode encrypt.format= - Encrypt the image, format choices: 'aes', 'luks' @@ -612,6 +632,7 @@ Creation options for 'qcow2': backing_fmt= - Image format of the base image cluster_size= - qcow2 cluster size compat= - Compatibility level (0.10 or 1.1) + data_file= - File name of an external data file encrypt.cipher-alg= - Name of encryption cipher algorithm encrypt.cipher-mode= - Name of encryption cipher mode encrypt.format= - Encrypt the image, format choices: 'aes', 'luks' @@ -634,6 +655,7 @@ Creation options for 'qcow2': backing_fmt= - Image format of the base image cluster_size= - qcow2 cluster size compat= - Compatibility level (0.10 or 1.1) + data_file= - File name of an external data file encrypt.cipher-alg= - Name of encryption cipher algorithm encrypt.cipher-mode= - Name of encryption cipher mode encrypt.format= - Encrypt the image, format choices: 'aes', 'luks' @@ -656,6 +678,7 @@ Creation options for 'qcow2': backing_fmt= - Image format of the base image cluster_size= - qcow2 cluster size compat= - Compatibility level (0.10 or 1.1) + data_file= - File name of an external data file encrypt.cipher-alg= - Name of encryption cipher algorithm encrypt.cipher-mode= - Name of encryption cipher mode encrypt.format= - Encrypt the image, format choices: 'aes', 'luks' @@ -678,6 +701,7 @@ Creation options for 'qcow2': backing_fmt= - Image format of the base image cluster_size= - qcow2 cluster size compat= - Compatibility level (0.10 or 1.1) + data_file= - File name of an external data file encrypt.cipher-alg= - Name of encryption cipher algorithm encrypt.cipher-mode= - Name of encryption cipher mode encrypt.format= - Encrypt the image, format choices: 'aes', 'luks' @@ -700,6 +724,7 @@ Creation options for 'qcow2': backing_fmt= - Image format of the base image cluster_size= - qcow2 cluster size compat= - Compatibility level (0.10 or 1.1) + data_file= - File name of an external data file encrypt.cipher-alg= - Name of encryption cipher algorithm encrypt.cipher-mode= - Name of encryption cipher mode encrypt.format= - Encrypt the image, format choices: 'aes', 'luks' @@ -722,6 +747,7 @@ Creation options for 'qcow2': backing_fmt= - Image format of the base image cluster_size= - qcow2 cluster size compat= - Compatibility level (0.10 or 1.1) + data_file= - File name of an external data file encrypt.cipher-alg= - Name of encryption cipher algorithm encrypt.cipher-mode= - Name of encryption cipher mode encrypt.format= - Encrypt the image, format choices: 'aes', 'luks' @@ -761,6 +787,7 @@ Creation options for 'qcow2': backing_fmt= - Image format of the base image cluster_size= - qcow2 cluster size compat= - Compatibility level (0.10 or 1.1) + data_file= - File name of an external data file encrypt.cipher-alg= - Name of encryption cipher algorithm encrypt.cipher-mode= - Name of encryption cipher mode encrypt.format= - Encrypt the image, format choices: 'aes', 'luks' From patchwork Wed Feb 27 17:22:53 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 1049041 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 448jSM2GjHz9s70 for ; Thu, 28 Feb 2019 04:32:50 +1100 (AEDT) Received: from localhost ([127.0.0.1]:48038 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gz34a-0001KT-Lw for incoming@patchwork.ozlabs.org; Wed, 27 Feb 2019 12:32:48 -0500 Received: from eggs.gnu.org ([209.51.188.92]:51912) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gz2wm-000410-TB for qemu-devel@nongnu.org; Wed, 27 Feb 2019 12:24:47 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gz2wi-00042b-D8 for qemu-devel@nongnu.org; Wed, 27 Feb 2019 12:24:44 -0500 Received: from mx1.redhat.com ([209.132.183.28]:43666) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1gz2wU-0003L5-3b; Wed, 27 Feb 2019 12:24:26 -0500 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 1E81C309CC04; Wed, 27 Feb 2019 17:23:49 +0000 (UTC) Received: from linux.fritz.box.com (ovpn-117-19.ams2.redhat.com [10.36.117.19]) by smtp.corp.redhat.com (Postfix) with ESMTP id B1F3C17153; Wed, 27 Feb 2019 17:23:47 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Date: Wed, 27 Feb 2019 18:22:53 +0100 Message-Id: <20190227172256.30368-18-kwolf@redhat.com> In-Reply-To: <20190227172256.30368-1-kwolf@redhat.com> References: <20190227172256.30368-1-kwolf@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.48]); Wed, 27 Feb 2019 17:23:49 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH 17/20] qcow2: Implement data-file-raw create option X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org, mreitz@redhat.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Provide an option to force QEMU to always keep the external data file consistent as a standalone read-only raw image. At the moment, this means making sure that write_zeroes requests are forwarded to the data file instead of just updating the metadata, and checking that no backing file is used. Signed-off-by: Kevin Wolf --- qapi/block-core.json | 9 ++++++ block/qcow2.h | 9 +++++- include/block/block_int.h | 1 + block/qcow2-cluster.c | 10 +++++++ block/qcow2.c | 56 ++++++++++++++++++++++++++++++++++++-- tests/qemu-iotests/082.out | 27 ++++++++++++++++++ 6 files changed, 109 insertions(+), 3 deletions(-) diff --git a/qapi/block-core.json b/qapi/block-core.json index e6faa94fa2..919d0530b2 100644 --- a/qapi/block-core.json +++ b/qapi/block-core.json @@ -62,6 +62,10 @@ # @data-file: the filename of the external data file that is stored in the # image and used as a default for opening the image (since: 4.0) # +# @data-file-raw: True if the external data file must stay valid as a +# standalone (read-only) raw image without looking at qcow2 +# metadata (since: 4.0) +# # @lazy-refcounts: on or off; only valid for compat >= 1.1 # # @corrupt: true if the image has been marked corrupt; only valid for @@ -80,6 +84,7 @@ 'data': { 'compat': 'str', '*data-file': 'str', + '*data-file-raw': 'bool', '*lazy-refcounts': 'bool', '*corrupt': 'bool', 'refcount-bits': 'int', @@ -4144,6 +4149,9 @@ # @data-file Node to use as an external data file in which all guest # data is stored so that only metadata remains in the qcow2 # file (since: 4.0) +# @data-file-raw True if the external data file must stay valid as a +# standalone (read-only) raw image without looking at qcow2 +# metadata (default: false; since: 4.0) # @size Size of the virtual disk in bytes # @version Compatibility level (default: v3) # @backing-file File name of the backing file if a backing file @@ -4160,6 +4168,7 @@ { 'struct': 'BlockdevCreateOptionsQcow2', 'data': { 'file': 'BlockdevRef', '*data-file': 'BlockdevRef', + '*data-file-raw': 'bool', 'size': 'size', '*version': 'BlockdevQcow2Version', '*backing-file': 'str', diff --git a/block/qcow2.h b/block/qcow2.h index a9c9cb4a26..de2a3bdfc5 100644 --- a/block/qcow2.h +++ b/block/qcow2.h @@ -225,7 +225,8 @@ enum { QCOW2_AUTOCLEAR_BITMAPS = 1 << QCOW2_AUTOCLEAR_BITMAPS_BITNR, QCOW2_AUTOCLEAR_DATA_FILE_RAW = 1 << QCOW2_AUTOCLEAR_DATA_FILE_RAW_BITNR, - QCOW2_AUTOCLEAR_MASK = QCOW2_AUTOCLEAR_BITMAPS, + QCOW2_AUTOCLEAR_MASK = QCOW2_AUTOCLEAR_BITMAPS + | QCOW2_AUTOCLEAR_DATA_FILE_RAW, }; enum qcow2_discard_type { @@ -474,6 +475,12 @@ static inline bool has_data_file(BlockDriverState *bs) return (s->data_file != bs->file); } +static inline bool data_file_is_raw(BlockDriverState *bs) +{ + BDRVQcow2State *s = bs->opaque; + return !!(s->autoclear_features & QCOW2_AUTOCLEAR_DATA_FILE_RAW); +} + static inline int64_t start_of_cluster(BDRVQcow2State *s, int64_t offset) { return offset & ~(s->cluster_size - 1); diff --git a/include/block/block_int.h b/include/block/block_int.h index acd29ce521..a23cabaddd 100644 --- a/include/block/block_int.h +++ b/include/block/block_int.h @@ -57,6 +57,7 @@ #define BLOCK_OPT_OBJECT_SIZE "object_size" #define BLOCK_OPT_REFCOUNT_BITS "refcount_bits" #define BLOCK_OPT_DATA_FILE "data_file" +#define BLOCK_OPT_DATA_FILE_RAW "data_file_raw" #define BLOCK_PROBE_BUF_SIZE 512 diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c index 7579f5a5ae..974a4e8656 100644 --- a/block/qcow2-cluster.c +++ b/block/qcow2-cluster.c @@ -1783,6 +1783,16 @@ int qcow2_cluster_zeroize(BlockDriverState *bs, uint64_t offset, int64_t cleared; int ret; + /* If we have to stay in sync with an external data file, zero out + * s->data_file first. */ + if (data_file_is_raw(bs)) { + assert(has_data_file(bs)); + ret = bdrv_co_pwrite_zeroes(s->data_file, offset, bytes, flags); + if (ret < 0) { + return ret; + } + } + /* Caller must pass aligned values, except at image end */ assert(QEMU_IS_ALIGNED(offset, s->cluster_size)); assert(QEMU_IS_ALIGNED(end_offset, s->cluster_size) || diff --git a/block/qcow2.c b/block/qcow2.c index 24c023e13d..503239d865 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -1486,8 +1486,14 @@ static int coroutine_fn qcow2_do_open(BlockDriverState *bs, QDict *options, "external data file"); ret = -EINVAL; goto fail; - } else { - s->data_file = bs->file; + } + + s->data_file = bs->file; + + if (data_file_is_raw(bs)) { + error_setg(errp, "data-file-raw requires a data file"); + ret = -EINVAL; + goto fail; } } @@ -2594,6 +2600,12 @@ static int qcow2_change_backing_file(BlockDriverState *bs, { BDRVQcow2State *s = bs->opaque; + /* Adding a backing file means that the external data file alone won't be + * enough to make sense of the content */ + if (backing_file && data_file_is_raw(bs)) { + return -EINVAL; + } + if (backing_file && strlen(backing_file) > 1023) { return -EINVAL; } @@ -3004,6 +3016,18 @@ qcow2_co_create(BlockdevCreateOptions *create_options, Error **errp) } refcount_order = ctz32(qcow2_opts->refcount_bits); + if (qcow2_opts->data_file_raw && !qcow2_opts->data_file) { + error_setg(errp, "data-file-raw requires data-file"); + ret = -EINVAL; + goto out; + } + if (qcow2_opts->data_file_raw && qcow2_opts->has_backing_file) { + error_setg(errp, "Backing file and data-file-raw cannot be used at " + "the same time"); + ret = -EINVAL; + goto out; + } + if (qcow2_opts->data_file) { if (version < 3) { error_setg(errp, "External data files are only supported with " @@ -3060,6 +3084,10 @@ qcow2_co_create(BlockdevCreateOptions *create_options, Error **errp) header->incompatible_features |= cpu_to_be64(QCOW2_INCOMPAT_DATA_FILE); } + if (qcow2_opts->data_file_raw) { + header->autoclear_features |= + cpu_to_be64(QCOW2_AUTOCLEAR_DATA_FILE_RAW); + } ret = blk_pwrite(blk, 0, header, cluster_size, 0); g_free(header); @@ -3241,6 +3269,7 @@ static int coroutine_fn qcow2_co_create_opts(const char *filename, QemuOpts *opt { BLOCK_OPT_REFCOUNT_BITS, "refcount-bits" }, { BLOCK_OPT_ENCRYPT, BLOCK_OPT_ENCRYPT_FORMAT }, { BLOCK_OPT_COMPAT_LEVEL, "version" }, + { BLOCK_OPT_DATA_FILE_RAW, "data-file-raw" }, { NULL, NULL }, }; @@ -4617,6 +4646,8 @@ static ImageInfoSpecific *qcow2_get_specific_info(BlockDriverState *bs, .bitmaps = bitmaps, .has_data_file = !!s->image_data_file, .data_file = g_strdup(s->image_data_file), + .has_data_file_raw = has_data_file(bs), + .data_file_raw = data_file_is_raw(bs), }; } else { /* if this assertion fails, this probably means a new version was @@ -4821,6 +4852,7 @@ static int qcow2_amend_options(BlockDriverState *bs, QemuOpts *opts, uint64_t new_size = 0; const char *backing_file = NULL, *backing_format = NULL, *data_file = NULL; bool lazy_refcounts = s->use_lazy_refcounts; + bool data_file_raw = data_file_is_raw(bs); const char *compat = NULL; uint64_t cluster_size = s->cluster_size; bool encrypt; @@ -4908,6 +4940,14 @@ static int qcow2_amend_options(BlockDriverState *bs, QemuOpts *opts, "use an external data file"); return -EINVAL; } + } else if (!strcmp(desc->name, BLOCK_OPT_DATA_FILE_RAW)) { + data_file_raw = qemu_opt_get_bool(opts, BLOCK_OPT_DATA_FILE_RAW, + data_file_raw); + if (data_file_raw && !data_file_is_raw(bs)) { + error_setg(errp, "data-file-raw cannot be set on existing " + "images"); + return -EINVAL; + } } else { /* if this point is reached, this probably means a new option was * added without having it covered here */ @@ -4954,6 +4994,13 @@ static int qcow2_amend_options(BlockDriverState *bs, QemuOpts *opts, } } + /* data-file-raw blocks backing files, so clear it first if requested */ + if (data_file_raw) { + s->autoclear_features |= QCOW2_AUTOCLEAR_DATA_FILE_RAW; + } else { + s->autoclear_features &= ~QCOW2_AUTOCLEAR_DATA_FILE_RAW; + } + if (data_file) { g_free(s->image_data_file); s->image_data_file = *data_file ? g_strdup(data_file) : NULL; @@ -5117,6 +5164,11 @@ static QemuOptsList qcow2_create_opts = { .type = QEMU_OPT_STRING, .help = "File name of an external data file" }, + { + .name = BLOCK_OPT_DATA_FILE_RAW, + .type = QEMU_OPT_BOOL, + .help = "The external data file must stay valid as a raw image" + }, { .name = BLOCK_OPT_ENCRYPT, .type = QEMU_OPT_BOOL, diff --git a/tests/qemu-iotests/082.out b/tests/qemu-iotests/082.out index 7dc59f6075..915640613f 100644 --- a/tests/qemu-iotests/082.out +++ b/tests/qemu-iotests/082.out @@ -49,6 +49,7 @@ Supported options: cluster_size= - qcow2 cluster size compat= - Compatibility level (0.10 or 1.1) data_file= - File name of an external data file + data_file_raw= - The external data file must stay valid as a raw image encrypt.cipher-alg= - Name of encryption cipher algorithm encrypt.cipher-mode= - Name of encryption cipher mode encrypt.format= - Encrypt the image, format choices: 'aes', 'luks' @@ -71,6 +72,7 @@ Supported options: cluster_size= - qcow2 cluster size compat= - Compatibility level (0.10 or 1.1) data_file= - File name of an external data file + data_file_raw= - The external data file must stay valid as a raw image encrypt.cipher-alg= - Name of encryption cipher algorithm encrypt.cipher-mode= - Name of encryption cipher mode encrypt.format= - Encrypt the image, format choices: 'aes', 'luks' @@ -93,6 +95,7 @@ Supported options: cluster_size= - qcow2 cluster size compat= - Compatibility level (0.10 or 1.1) data_file= - File name of an external data file + data_file_raw= - The external data file must stay valid as a raw image encrypt.cipher-alg= - Name of encryption cipher algorithm encrypt.cipher-mode= - Name of encryption cipher mode encrypt.format= - Encrypt the image, format choices: 'aes', 'luks' @@ -115,6 +118,7 @@ Supported options: cluster_size= - qcow2 cluster size compat= - Compatibility level (0.10 or 1.1) data_file= - File name of an external data file + data_file_raw= - The external data file must stay valid as a raw image encrypt.cipher-alg= - Name of encryption cipher algorithm encrypt.cipher-mode= - Name of encryption cipher mode encrypt.format= - Encrypt the image, format choices: 'aes', 'luks' @@ -137,6 +141,7 @@ Supported options: cluster_size= - qcow2 cluster size compat= - Compatibility level (0.10 or 1.1) data_file= - File name of an external data file + data_file_raw= - The external data file must stay valid as a raw image encrypt.cipher-alg= - Name of encryption cipher algorithm encrypt.cipher-mode= - Name of encryption cipher mode encrypt.format= - Encrypt the image, format choices: 'aes', 'luks' @@ -159,6 +164,7 @@ Supported options: cluster_size= - qcow2 cluster size compat= - Compatibility level (0.10 or 1.1) data_file= - File name of an external data file + data_file_raw= - The external data file must stay valid as a raw image encrypt.cipher-alg= - Name of encryption cipher algorithm encrypt.cipher-mode= - Name of encryption cipher mode encrypt.format= - Encrypt the image, format choices: 'aes', 'luks' @@ -181,6 +187,7 @@ Supported options: cluster_size= - qcow2 cluster size compat= - Compatibility level (0.10 or 1.1) data_file= - File name of an external data file + data_file_raw= - The external data file must stay valid as a raw image encrypt.cipher-alg= - Name of encryption cipher algorithm encrypt.cipher-mode= - Name of encryption cipher mode encrypt.format= - Encrypt the image, format choices: 'aes', 'luks' @@ -203,6 +210,7 @@ Supported options: cluster_size= - qcow2 cluster size compat= - Compatibility level (0.10 or 1.1) data_file= - File name of an external data file + data_file_raw= - The external data file must stay valid as a raw image encrypt.cipher-alg= - Name of encryption cipher algorithm encrypt.cipher-mode= - Name of encryption cipher mode encrypt.format= - Encrypt the image, format choices: 'aes', 'luks' @@ -240,6 +248,7 @@ Supported options: cluster_size= - qcow2 cluster size compat= - Compatibility level (0.10 or 1.1) data_file= - File name of an external data file + data_file_raw= - The external data file must stay valid as a raw image encrypt.cipher-alg= - Name of encryption cipher algorithm encrypt.cipher-mode= - Name of encryption cipher mode encrypt.format= - Encrypt the image, format choices: 'aes', 'luks' @@ -314,6 +323,7 @@ Supported options: cluster_size= - qcow2 cluster size compat= - Compatibility level (0.10 or 1.1) data_file= - File name of an external data file + data_file_raw= - The external data file must stay valid as a raw image encrypt.cipher-alg= - Name of encryption cipher algorithm encrypt.cipher-mode= - Name of encryption cipher mode encrypt.format= - Encrypt the image, format choices: 'aes', 'luks' @@ -336,6 +346,7 @@ Supported options: cluster_size= - qcow2 cluster size compat= - Compatibility level (0.10 or 1.1) data_file= - File name of an external data file + data_file_raw= - The external data file must stay valid as a raw image encrypt.cipher-alg= - Name of encryption cipher algorithm encrypt.cipher-mode= - Name of encryption cipher mode encrypt.format= - Encrypt the image, format choices: 'aes', 'luks' @@ -358,6 +369,7 @@ Supported options: cluster_size= - qcow2 cluster size compat= - Compatibility level (0.10 or 1.1) data_file= - File name of an external data file + data_file_raw= - The external data file must stay valid as a raw image encrypt.cipher-alg= - Name of encryption cipher algorithm encrypt.cipher-mode= - Name of encryption cipher mode encrypt.format= - Encrypt the image, format choices: 'aes', 'luks' @@ -380,6 +392,7 @@ Supported options: cluster_size= - qcow2 cluster size compat= - Compatibility level (0.10 or 1.1) data_file= - File name of an external data file + data_file_raw= - The external data file must stay valid as a raw image encrypt.cipher-alg= - Name of encryption cipher algorithm encrypt.cipher-mode= - Name of encryption cipher mode encrypt.format= - Encrypt the image, format choices: 'aes', 'luks' @@ -402,6 +415,7 @@ Supported options: cluster_size= - qcow2 cluster size compat= - Compatibility level (0.10 or 1.1) data_file= - File name of an external data file + data_file_raw= - The external data file must stay valid as a raw image encrypt.cipher-alg= - Name of encryption cipher algorithm encrypt.cipher-mode= - Name of encryption cipher mode encrypt.format= - Encrypt the image, format choices: 'aes', 'luks' @@ -424,6 +438,7 @@ Supported options: cluster_size= - qcow2 cluster size compat= - Compatibility level (0.10 or 1.1) data_file= - File name of an external data file + data_file_raw= - The external data file must stay valid as a raw image encrypt.cipher-alg= - Name of encryption cipher algorithm encrypt.cipher-mode= - Name of encryption cipher mode encrypt.format= - Encrypt the image, format choices: 'aes', 'luks' @@ -446,6 +461,7 @@ Supported options: cluster_size= - qcow2 cluster size compat= - Compatibility level (0.10 or 1.1) data_file= - File name of an external data file + data_file_raw= - The external data file must stay valid as a raw image encrypt.cipher-alg= - Name of encryption cipher algorithm encrypt.cipher-mode= - Name of encryption cipher mode encrypt.format= - Encrypt the image, format choices: 'aes', 'luks' @@ -468,6 +484,7 @@ Supported options: cluster_size= - qcow2 cluster size compat= - Compatibility level (0.10 or 1.1) data_file= - File name of an external data file + data_file_raw= - The external data file must stay valid as a raw image encrypt.cipher-alg= - Name of encryption cipher algorithm encrypt.cipher-mode= - Name of encryption cipher mode encrypt.format= - Encrypt the image, format choices: 'aes', 'luks' @@ -505,6 +522,7 @@ Supported options: cluster_size= - qcow2 cluster size compat= - Compatibility level (0.10 or 1.1) data_file= - File name of an external data file + data_file_raw= - The external data file must stay valid as a raw image encrypt.cipher-alg= - Name of encryption cipher algorithm encrypt.cipher-mode= - Name of encryption cipher mode encrypt.format= - Encrypt the image, format choices: 'aes', 'luks' @@ -587,6 +605,7 @@ Creation options for 'qcow2': cluster_size= - qcow2 cluster size compat= - Compatibility level (0.10 or 1.1) data_file= - File name of an external data file + data_file_raw= - The external data file must stay valid as a raw image encrypt.cipher-alg= - Name of encryption cipher algorithm encrypt.cipher-mode= - Name of encryption cipher mode encrypt.format= - Encrypt the image, format choices: 'aes', 'luks' @@ -610,6 +629,7 @@ Creation options for 'qcow2': cluster_size= - qcow2 cluster size compat= - Compatibility level (0.10 or 1.1) data_file= - File name of an external data file + data_file_raw= - The external data file must stay valid as a raw image encrypt.cipher-alg= - Name of encryption cipher algorithm encrypt.cipher-mode= - Name of encryption cipher mode encrypt.format= - Encrypt the image, format choices: 'aes', 'luks' @@ -633,6 +653,7 @@ Creation options for 'qcow2': cluster_size= - qcow2 cluster size compat= - Compatibility level (0.10 or 1.1) data_file= - File name of an external data file + data_file_raw= - The external data file must stay valid as a raw image encrypt.cipher-alg= - Name of encryption cipher algorithm encrypt.cipher-mode= - Name of encryption cipher mode encrypt.format= - Encrypt the image, format choices: 'aes', 'luks' @@ -656,6 +677,7 @@ Creation options for 'qcow2': cluster_size= - qcow2 cluster size compat= - Compatibility level (0.10 or 1.1) data_file= - File name of an external data file + data_file_raw= - The external data file must stay valid as a raw image encrypt.cipher-alg= - Name of encryption cipher algorithm encrypt.cipher-mode= - Name of encryption cipher mode encrypt.format= - Encrypt the image, format choices: 'aes', 'luks' @@ -679,6 +701,7 @@ Creation options for 'qcow2': cluster_size= - qcow2 cluster size compat= - Compatibility level (0.10 or 1.1) data_file= - File name of an external data file + data_file_raw= - The external data file must stay valid as a raw image encrypt.cipher-alg= - Name of encryption cipher algorithm encrypt.cipher-mode= - Name of encryption cipher mode encrypt.format= - Encrypt the image, format choices: 'aes', 'luks' @@ -702,6 +725,7 @@ Creation options for 'qcow2': cluster_size= - qcow2 cluster size compat= - Compatibility level (0.10 or 1.1) data_file= - File name of an external data file + data_file_raw= - The external data file must stay valid as a raw image encrypt.cipher-alg= - Name of encryption cipher algorithm encrypt.cipher-mode= - Name of encryption cipher mode encrypt.format= - Encrypt the image, format choices: 'aes', 'luks' @@ -725,6 +749,7 @@ Creation options for 'qcow2': cluster_size= - qcow2 cluster size compat= - Compatibility level (0.10 or 1.1) data_file= - File name of an external data file + data_file_raw= - The external data file must stay valid as a raw image encrypt.cipher-alg= - Name of encryption cipher algorithm encrypt.cipher-mode= - Name of encryption cipher mode encrypt.format= - Encrypt the image, format choices: 'aes', 'luks' @@ -748,6 +773,7 @@ Creation options for 'qcow2': cluster_size= - qcow2 cluster size compat= - Compatibility level (0.10 or 1.1) data_file= - File name of an external data file + data_file_raw= - The external data file must stay valid as a raw image encrypt.cipher-alg= - Name of encryption cipher algorithm encrypt.cipher-mode= - Name of encryption cipher mode encrypt.format= - Encrypt the image, format choices: 'aes', 'luks' @@ -788,6 +814,7 @@ Creation options for 'qcow2': cluster_size= - qcow2 cluster size compat= - Compatibility level (0.10 or 1.1) data_file= - File name of an external data file + data_file_raw= - The external data file must stay valid as a raw image encrypt.cipher-alg= - Name of encryption cipher algorithm encrypt.cipher-mode= - Name of encryption cipher mode encrypt.format= - Encrypt the image, format choices: 'aes', 'luks' From patchwork Wed Feb 27 17:22:54 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 1049036 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 448jMB30bxz9s70 for ; Thu, 28 Feb 2019 04:28:22 +1100 (AEDT) Received: from localhost ([127.0.0.1]:47956 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gz30G-0006DS-9Y for incoming@patchwork.ozlabs.org; Wed, 27 Feb 2019 12:28:20 -0500 Received: from eggs.gnu.org ([209.51.188.92]:51495) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gz2wT-0003fR-Og for qemu-devel@nongnu.org; Wed, 27 Feb 2019 12:24:29 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gz2wO-0003eo-6g for qemu-devel@nongnu.org; Wed, 27 Feb 2019 12:24:25 -0500 Received: from mx1.redhat.com ([209.132.183.28]:53638) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1gz2wB-0003Lg-Vo; Wed, 27 Feb 2019 12:24:08 -0500 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id A0C5F9FDDA; Wed, 27 Feb 2019 17:23:50 +0000 (UTC) Received: from linux.fritz.box.com (ovpn-117-19.ams2.redhat.com [10.36.117.19]) by smtp.corp.redhat.com (Postfix) with ESMTP id 6B3B761469; Wed, 27 Feb 2019 17:23:49 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Date: Wed, 27 Feb 2019 18:22:54 +0100 Message-Id: <20190227172256.30368-19-kwolf@redhat.com> In-Reply-To: <20190227172256.30368-1-kwolf@redhat.com> References: <20190227172256.30368-1-kwolf@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.39]); Wed, 27 Feb 2019 17:23:50 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH 18/20] qemu-iotests: Preallocation with external data file X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org, mreitz@redhat.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Test that preallocating metadata results in a somewhat larger qcow2 file, but preallocating data only affects the disk usage of the data file and the qcow2 file stays small. Signed-off-by: Kevin Wolf --- tests/qemu-iotests/243 | 22 ++++++++++++++++++++++ tests/qemu-iotests/243.out | 32 ++++++++++++++++++++++++++++++++ 2 files changed, 54 insertions(+) diff --git a/tests/qemu-iotests/243 b/tests/qemu-iotests/243 index 5ab1b7863d..0e807dabf6 100755 --- a/tests/qemu-iotests/243 +++ b/tests/qemu-iotests/243 @@ -29,6 +29,7 @@ status=1 # failure is the default! _cleanup() { _cleanup_test_img + rm -f $TEST_IMG.data } trap "_cleanup; exit \$status" 0 1 2 3 15 @@ -57,6 +58,27 @@ for mode in off metadata falloc full; do done +for mode in off metadata falloc full; do + + echo + echo "=== External data file: preallocation=$mode ===" + echo + + IMGOPTS="data_file=$TEST_IMG.data,preallocation=$mode" _make_test_img 64M + + echo -n "qcow2 file size: " + du -b $TEST_IMG | cut -f1 + echo -n "data file size: " + du -b $TEST_IMG.data | cut -f1 + + # Can't use precise numbers here because they differ between filesystems + echo -n "qcow2 disk usage: " + [ $(du -B1 $TEST_IMG | cut -f1) -lt 1048576 ] && echo "low" || echo "high" + echo -n "data disk usage: " + [ $(du -B1 $TEST_IMG.data | cut -f1) -lt 1048576 ] && echo "low" || echo "high" + +done + # success, all done echo "*** done" rm -f $seq.full diff --git a/tests/qemu-iotests/243.out b/tests/qemu-iotests/243.out index e531753ad1..dcb33fac32 100644 --- a/tests/qemu-iotests/243.out +++ b/tests/qemu-iotests/243.out @@ -23,4 +23,36 @@ Disk usage: high Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 preallocation=full File size: 67436544 Disk usage: high + +=== External data file: preallocation=off === + +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 data_file=TEST_DIR/t.IMGFMT.data preallocation=off +qcow2 file size: 196616 +data file size: 67108864 +qcow2 disk usage: low +data disk usage: low + +=== External data file: preallocation=metadata === + +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 data_file=TEST_DIR/t.IMGFMT.data preallocation=metadata +qcow2 file size: 327680 +data file size: 67108864 +qcow2 disk usage: low +data disk usage: low + +=== External data file: preallocation=falloc === + +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 data_file=TEST_DIR/t.IMGFMT.data preallocation=falloc +qcow2 file size: 327680 +data file size: 67108864 +qcow2 disk usage: low +data disk usage: high + +=== External data file: preallocation=full === + +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 data_file=TEST_DIR/t.IMGFMT.data preallocation=full +qcow2 file size: 327680 +data file size: 67108864 +qcow2 disk usage: low +data disk usage: high *** done From patchwork Wed Feb 27 17:22:55 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 1049045 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 448jVH31hrz9sBL for ; Thu, 28 Feb 2019 04:34:31 +1100 (AEDT) Received: from localhost ([127.0.0.1]:48060 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gz36D-0002g8-9l for incoming@patchwork.ozlabs.org; Wed, 27 Feb 2019 12:34:29 -0500 Received: from eggs.gnu.org ([209.51.188.92]:51906) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gz2wm-00040w-RQ for qemu-devel@nongnu.org; Wed, 27 Feb 2019 12:24:47 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gz2wi-00042y-Ef for qemu-devel@nongnu.org; Wed, 27 Feb 2019 12:24:44 -0500 Received: from mx1.redhat.com ([209.132.183.28]:55101) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1gz2wY-0003MM-7F; Wed, 27 Feb 2019 12:24:30 -0500 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 2ED1A30B4786; Wed, 27 Feb 2019 17:23:52 +0000 (UTC) Received: from linux.fritz.box.com (ovpn-117-19.ams2.redhat.com [10.36.117.19]) by smtp.corp.redhat.com (Postfix) with ESMTP id EE21317153; Wed, 27 Feb 2019 17:23:50 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Date: Wed, 27 Feb 2019 18:22:55 +0100 Message-Id: <20190227172256.30368-20-kwolf@redhat.com> In-Reply-To: <20190227172256.30368-1-kwolf@redhat.com> References: <20190227172256.30368-1-kwolf@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.46]); Wed, 27 Feb 2019 17:23:52 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH 19/20] qemu-iotests: General tests for qcow2 with external data file X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org, mreitz@redhat.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Kevin Wolf --- tests/qemu-iotests/244 | 200 +++++++++++++++++++++++++++++++++++++ tests/qemu-iotests/244.out | 125 +++++++++++++++++++++++ tests/qemu-iotests/group | 1 + 3 files changed, 326 insertions(+) create mode 100755 tests/qemu-iotests/244 create mode 100644 tests/qemu-iotests/244.out diff --git a/tests/qemu-iotests/244 b/tests/qemu-iotests/244 new file mode 100755 index 0000000000..d8e7122305 --- /dev/null +++ b/tests/qemu-iotests/244 @@ -0,0 +1,200 @@ +#!/bin/bash +# +# Test qcow2 with external data files +# +# Copyright (C) 2019 Red Hat, Inc. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . +# + +# creator +owner=kwolf@redhat.com + +seq=$(basename $0) +echo "QA output created by $seq" + +status=1 # failure is the default! + +_cleanup() +{ + _cleanup_test_img + rm -f $TEST_IMG.data + rm -f $TEST_IMG.src +} +trap "_cleanup; exit \$status" 0 1 2 3 15 + +# get standard environment, filters and checks +. ./common.rc +. ./common.filter + +_supported_fmt qcow2 +_supported_proto file +_supported_os Linux + +echo +echo "=== Create and open image with external data file ===" +echo + +echo "With data file name in the image:" +IMGOPTS="data_file=$TEST_IMG.data" _make_test_img 64M +_check_test_img + +$QEMU_IO -c "open $TEST_IMG" -c "read -P 0 0 64k" 2>&1 | _filter_qemu_io | _filter_testdir +$QEMU_IO -c "open -odata-file.filename=$TEST_IMG.data $TEST_IMG" -c "read -P 0 0 64k" 2>&1 | _filter_qemu_io | _filter_testdir +$QEMU_IO -c "open -odata-file.filename=inexistent $TEST_IMG" -c "read -P 0 0 64k" 2>&1 | _filter_qemu_io | _filter_testdir + +echo +echo "Data file required, but without data file name in the image:" +$QEMU_IMG amend -odata_file= $TEST_IMG + +$QEMU_IO -c "open $TEST_IMG" -c "read -P 0 0 64k" 2>&1 | _filter_qemu_io | _filter_testdir +$QEMU_IO -c "open -odata-file.filename=$TEST_IMG.data $TEST_IMG" -c "read -P 0 0 64k" 2>&1 | _filter_qemu_io | _filter_testdir +$QEMU_IO -c "open -odata-file.filename=inexistent $TEST_IMG" -c "read -P 0 0 64k" 2>&1 | _filter_qemu_io | _filter_testdir + +echo +echo "Setting data-file for an image with internal data:" +_make_test_img 64M + +$QEMU_IO -c "open -odata-file.filename=$TEST_IMG.data $TEST_IMG" -c "read -P 0 0 64k" 2>&1 | _filter_qemu_io | _filter_testdir +$QEMU_IO -c "open -odata-file.filename=inexistent $TEST_IMG" -c "read -P 0 0 64k" 2>&1 | _filter_qemu_io | _filter_testdir + +echo +echo "=== Conflicting features ===" +echo + +echo "Convert to compressed target with data file:" +TEST_IMG="$TEST_IMG.src" _make_test_img 64M + +$QEMU_IO -c 'write -P 0x11 0 1M' \ + -f $IMGFMT "$TEST_IMG.src" | + _filter_qemu_io + +$QEMU_IMG convert -f $IMGFMT -O $IMGFMT -c -odata_file="$TEST_IMG.data" \ + "$TEST_IMG.src" "$TEST_IMG" + +echo +echo "Convert uncompressed, then write compressed data manually:" +$QEMU_IMG convert -f $IMGFMT -O $IMGFMT -odata_file="$TEST_IMG.data" \ + "$TEST_IMG.src" "$TEST_IMG" +$QEMU_IMG compare "$TEST_IMG.src" "$TEST_IMG" + +$QEMU_IO -c 'write -c -P 0x22 0 1M' \ + -f $IMGFMT "$TEST_IMG" | + _filter_qemu_io +_check_test_img + +echo +echo "Take an internal snapshot:" + +$QEMU_IMG snapshot -c test "$TEST_IMG" +_check_test_img + +echo +echo "=== Standalone image with external data file (efficient) ===" +echo + +IMGOPTS="data_file=$TEST_IMG.data" _make_test_img 64M + +echo -n "qcow2 file size before I/O: " +du -b $TEST_IMG | cut -f1 + +# Create image with the following layout +# 0-1 MB: Unallocated +# 1-2 MB: Written (pattern 0x11) +# 2-3 MB: Discarded +# 3-4 MB: Zero write over discarded space +# 4-5 MB: Zero write over written space +# 5-6 MB: Zero write over unallocated space + +echo +$QEMU_IO -c 'write -P 0x11 1M 4M' \ + -c 'discard 2M 2M' \ + -c 'write -z 3M 3M' \ + -f $IMGFMT "$TEST_IMG" | + _filter_qemu_io +_check_test_img + +echo +$QEMU_IMG map --output=json "$TEST_IMG" + +echo +$QEMU_IO -c 'read -P 0 0 1M' \ + -c 'read -P 0x11 1M 1M' \ + -c 'read -P 0 2M 4M' \ + -f $IMGFMT "$TEST_IMG" | + _filter_qemu_io + +# Zero clusters are only marked as such in the qcow2 metadata, but contain +# stale data in the external data file +echo +$QEMU_IO -c 'read -P 0 0 1M' \ + -c 'read -P 0x11 1M 1M' \ + -c 'read -P 0 2M 2M' \ + -c 'read -P 0x11 4M 1M' \ + -c 'read -P 0 5M 1M' \ + -f raw "$TEST_IMG.data" | + _filter_qemu_io + + +echo -n "qcow2 file size after I/O: " +du -b $TEST_IMG | cut -f1 + +echo +echo "=== Standalone image with external data file (valid raw) ===" +echo + +IMGOPTS="data_file=$TEST_IMG.data,data_file_raw=on" _make_test_img 64M + +echo -n "qcow2 file size before I/O: " +du -b $TEST_IMG | cut -f1 + +echo +$QEMU_IO -c 'write -P 0x11 1M 4M' \ + -c 'discard 2M 2M' \ + -c 'write -z 3M 3M' \ + -f $IMGFMT "$TEST_IMG" | + _filter_qemu_io +_check_test_img + +echo +$QEMU_IMG map --output=json "$TEST_IMG" + +echo +$QEMU_IO -c 'read -P 0 0 1M' \ + -c 'read -P 0x11 1M 1M' \ + -c 'read -P 0 2M 4M' \ + -f $IMGFMT "$TEST_IMG" | + _filter_qemu_io + +echo +$QEMU_IMG compare "$TEST_IMG" "$TEST_IMG.data" + +echo -n "qcow2 file size after I/O: " +du -b $TEST_IMG | cut -f1 + +echo +echo "=== bdrv_co_block_status test for file and offset=0 ===" +echo + +IMGOPTS="data_file=$TEST_IMG.data" _make_test_img 64M + +$QEMU_IO -c 'write -P 0x11 0 1M' -f $IMGFMT "$TEST_IMG" | _filter_qemu_io +$QEMU_IO -c 'read -P 0x11 0 1M' -f $IMGFMT "$TEST_IMG" | _filter_qemu_io +$QEMU_IMG map --output=human "$TEST_IMG" | _filter_testdir +$QEMU_IMG map --output=json "$TEST_IMG" + +# success, all done +echo "*** done" +rm -f $seq.full +status=0 diff --git a/tests/qemu-iotests/244.out b/tests/qemu-iotests/244.out new file mode 100644 index 0000000000..98e5946976 --- /dev/null +++ b/tests/qemu-iotests/244.out @@ -0,0 +1,125 @@ +QA output created by 244 + +=== Create and open image with external data file === + +With data file name in the image: +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 data_file=TEST_DIR/t.IMGFMT.data +No errors were found on the image. +read 65536/65536 bytes at offset 0 +64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 65536/65536 bytes at offset 0 +64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +can't open device TEST_DIR/t.qcow2: Could not open 'inexistent': No such file or directory +no file open, try 'help open' + +Data file required, but without data file name in the image: +can't open device TEST_DIR/t.qcow2: 'data-file' is required for this image +no file open, try 'help open' +read 65536/65536 bytes at offset 0 +64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +can't open device TEST_DIR/t.qcow2: Could not open 'inexistent': No such file or directory +no file open, try 'help open' + +Setting data-file for an image with internal data: +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 +can't open device TEST_DIR/t.qcow2: 'data-file' can only be set for images with an external data file +no file open, try 'help open' +can't open device TEST_DIR/t.qcow2: Could not open 'inexistent': No such file or directory +no file open, try 'help open' + +=== Conflicting features === + +Convert to compressed target with data file: +Formatting 'TEST_DIR/t.IMGFMT.src', fmt=IMGFMT size=67108864 +wrote 1048576/1048576 bytes at offset 0 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +qemu-img: error while writing sector 0: Operation not supported + +Convert uncompressed, then write compressed data manually: +Images are identical. +write failed: Operation not supported +No errors were found on the image. + +Take an internal snapshot: +qemu-img: Could not create snapshot 'test': -95 (Operation not supported) +No errors were found on the image. + +=== Standalone image with external data file (efficient) === + +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 data_file=TEST_DIR/t.IMGFMT.data +qcow2 file size before I/O: 196616 + +wrote 4194304/4194304 bytes at offset 1048576 +4 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +discard 2097152/2097152 bytes at offset 2097152 +2 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 3145728/3145728 bytes at offset 3145728 +3 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +No errors were found on the image. + +[{ "start": 0, "length": 1048576, "depth": 0, "zero": true, "data": false}, +{ "start": 1048576, "length": 1048576, "depth": 0, "zero": false, "data": true, "offset": 1048576}, +{ "start": 2097152, "length": 2097152, "depth": 0, "zero": true, "data": false}, +{ "start": 4194304, "length": 1048576, "depth": 0, "zero": true, "data": false, "offset": 4194304}, +{ "start": 5242880, "length": 61865984, "depth": 0, "zero": true, "data": false}] + +read 1048576/1048576 bytes at offset 0 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 1048576/1048576 bytes at offset 1048576 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 4194304/4194304 bytes at offset 2097152 +4 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +read 1048576/1048576 bytes at offset 0 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 1048576/1048576 bytes at offset 1048576 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 2097152/2097152 bytes at offset 2097152 +2 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 1048576/1048576 bytes at offset 4194304 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 1048576/1048576 bytes at offset 5242880 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +qcow2 file size after I/O: 327680 + +=== Standalone image with external data file (valid raw) === + +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 data_file=TEST_DIR/t.IMGFMT.data data_file_raw=on +qcow2 file size before I/O: 196616 + +wrote 4194304/4194304 bytes at offset 1048576 +4 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +discard 2097152/2097152 bytes at offset 2097152 +2 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 3145728/3145728 bytes at offset 3145728 +3 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +No errors were found on the image. + +[{ "start": 0, "length": 1048576, "depth": 0, "zero": true, "data": false}, +{ "start": 1048576, "length": 1048576, "depth": 0, "zero": false, "data": true, "offset": 1048576}, +{ "start": 2097152, "length": 2097152, "depth": 0, "zero": true, "data": false}, +{ "start": 4194304, "length": 1048576, "depth": 0, "zero": true, "data": false, "offset": 4194304}, +{ "start": 5242880, "length": 61865984, "depth": 0, "zero": true, "data": false}] + +read 1048576/1048576 bytes at offset 0 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 1048576/1048576 bytes at offset 1048576 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 4194304/4194304 bytes at offset 2097152 +4 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +Images are identical. +qcow2 file size after I/O: 327680 + +=== bdrv_co_block_status test for file and offset=0 === + +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 data_file=TEST_DIR/t.IMGFMT.data +wrote 1048576/1048576 bytes at offset 0 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 1048576/1048576 bytes at offset 0 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +Offset Length Mapped to File +0 0x100000 0 TEST_DIR/t.qcow2.data +[{ "start": 0, "length": 1048576, "depth": 0, "zero": false, "data": true, "offset": 0}, +{ "start": 1048576, "length": 66060288, "depth": 0, "zero": true, "data": false}] +*** done diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group index 03aa93dbf5..36100b803c 100644 --- a/tests/qemu-iotests/group +++ b/tests/qemu-iotests/group @@ -242,3 +242,4 @@ 240 auto quick 242 rw auto quick 243 rw auto quick +244 rw auto quick From patchwork Wed Feb 27 17:22:56 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 1049039 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 448jQB6Nkrz9sBL for ; Thu, 28 Feb 2019 04:30:58 +1100 (AEDT) Received: from localhost ([127.0.0.1]:48002 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gz32m-0008Os-Nj for incoming@patchwork.ozlabs.org; Wed, 27 Feb 2019 12:30:56 -0500 Received: from eggs.gnu.org ([209.51.188.92]:51665) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gz2wa-0003lg-1U for qemu-devel@nongnu.org; Wed, 27 Feb 2019 12:24:35 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gz2wY-0003p4-7e for qemu-devel@nongnu.org; Wed, 27 Feb 2019 12:24:31 -0500 Received: from mx1.redhat.com ([209.132.183.28]:55744) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1gz2wE-0003Mq-Dn; Wed, 27 Feb 2019 12:24:12 -0500 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id AFF4130FD84B; Wed, 27 Feb 2019 17:23:53 +0000 (UTC) Received: from linux.fritz.box.com (ovpn-117-19.ams2.redhat.com [10.36.117.19]) by smtp.corp.redhat.com (Postfix) with ESMTP id 7CC3517153; Wed, 27 Feb 2019 17:23:52 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Date: Wed, 27 Feb 2019 18:22:56 +0100 Message-Id: <20190227172256.30368-21-kwolf@redhat.com> In-Reply-To: <20190227172256.30368-1-kwolf@redhat.com> References: <20190227172256.30368-1-kwolf@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.49]); Wed, 27 Feb 2019 17:23:53 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH 20/20] qemu-iotests: amend with external data file X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org, mreitz@redhat.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Kevin Wolf --- tests/qemu-iotests/061 | 45 ++++++++++++++++++- tests/qemu-iotests/061.out | 89 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 133 insertions(+), 1 deletion(-) diff --git a/tests/qemu-iotests/061 b/tests/qemu-iotests/061 index 1a50163419..838e0854f1 100755 --- a/tests/qemu-iotests/061 +++ b/tests/qemu-iotests/061 @@ -28,7 +28,8 @@ status=1 # failure is the default! _cleanup() { - _cleanup_test_img + _cleanup_test_img + rm -f $TEST_IMG.data } trap "_cleanup; exit \$status" 0 1 2 3 15 @@ -250,6 +251,48 @@ $QEMU_IMG snapshot -c foo "$TEST_IMG" $QEMU_IMG amend -p -o "compat=0.10" "$TEST_IMG" _check_test_img +echo +echo "=== Testing version downgrade with external data file ===" +echo +IMGOPTS="compat=1.1,data_file=$TEST_IMG.data" _make_test_img 64M +$QEMU_IMG amend -o "compat=0.10" "$TEST_IMG" +_img_info --format-specific +_check_test_img + +echo +echo "=== Try changing the external data file ===" +echo +IMGOPTS="compat=1.1" _make_test_img 64M +$QEMU_IMG amend -o "data_file=foo" "$TEST_IMG" + +echo +IMGOPTS="compat=1.1,data_file=$TEST_IMG.data" _make_test_img 64M +$QEMU_IMG amend -o "data_file=foo" "$TEST_IMG" +_img_info --format-specific +TEST_IMG="data-file.filename=$TEST_IMG.data,file.filename=$TEST_IMG" _img_info --format-specific --image-opts + +echo +$QEMU_IMG amend -o "data_file=" --image-opts "data-file.filename=$TEST_IMG.data,file.filename=$TEST_IMG" +_img_info --format-specific +TEST_IMG="data-file.filename=$TEST_IMG.data,file.filename=$TEST_IMG" _img_info --format-specific --image-opts + +echo +echo "=== Clearing and setting data-file-raw ===" +echo +IMGOPTS="compat=1.1,data_file=$TEST_IMG.data,data_file_raw=on" _make_test_img 64M +$QEMU_IMG amend -o "data_file_raw=on" "$TEST_IMG" +_img_info --format-specific +_check_test_img + +$QEMU_IMG amend -o "data_file_raw=off" "$TEST_IMG" +_img_info --format-specific +_check_test_img + +$QEMU_IMG amend -o "data_file_raw=on" "$TEST_IMG" +_img_info --format-specific +_check_test_img + + # success, all done echo "*** done" rm -f $seq.full diff --git a/tests/qemu-iotests/061.out b/tests/qemu-iotests/061.out index 758284011b..9fe1ec702f 100644 --- a/tests/qemu-iotests/061.out +++ b/tests/qemu-iotests/061.out @@ -488,4 +488,93 @@ wrote 65536/65536 bytes at offset 3221225472 64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) (0.00/100%) (6.25/100%) (12.50/100%) (18.75/100%) (25.00/100%) (31.25/100%) (37.50/100%) (43.75/100%) (50.00/100%) (56.25/100%) (62.50/100%) (68.75/100%) (75.00/100%) (81.25/100%) (87.50/100%) (93.75/100%) (100.00/100%) (100.00/100%) No errors were found on the image. + +=== Testing version downgrade with external data file === + +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 data_file=TEST_DIR/t.IMGFMT.data +qemu-img: Cannot downgrade an image with a data file +image: TEST_DIR/t.IMGFMT +file format: IMGFMT +virtual size: 64M (67108864 bytes) +cluster_size: 65536 +Format specific information: + compat: 1.1 + lazy refcounts: false + refcount bits: 16 + data file: TEST_DIR/t.IMGFMT.data + data file raw: false + corrupt: false +No errors were found on the image. + +=== Try changing the external data file === + +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 +qemu-img: data-file can only be set for images that use an external data file + +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 data_file=TEST_DIR/t.IMGFMT.data +qemu-img: Could not open 'TEST_DIR/t.IMGFMT': Could not open 'foo': No such file or directory +image: TEST_DIR/t.IMGFMT +file format: IMGFMT +virtual size: 64M (67108864 bytes) +cluster_size: 65536 +Format specific information: + compat: 1.1 + lazy refcounts: false + refcount bits: 16 + data file: foo + data file raw: false + corrupt: false + +qemu-img: Could not open 'TEST_DIR/t.IMGFMT': 'data-file' is required for this image +image: TEST_DIR/t.IMGFMT +file format: IMGFMT +virtual size: 64M (67108864 bytes) +cluster_size: 65536 +Format specific information: + compat: 1.1 + lazy refcounts: false + refcount bits: 16 + data file raw: false + corrupt: false + +=== Clearing and setting data-file-raw === + +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 data_file=TEST_DIR/t.IMGFMT.data data_file_raw=on +image: TEST_DIR/t.IMGFMT +file format: IMGFMT +virtual size: 64M (67108864 bytes) +cluster_size: 65536 +Format specific information: + compat: 1.1 + lazy refcounts: false + refcount bits: 16 + data file: TEST_DIR/t.IMGFMT.data + data file raw: true + corrupt: false +No errors were found on the image. +image: TEST_DIR/t.IMGFMT +file format: IMGFMT +virtual size: 64M (67108864 bytes) +cluster_size: 65536 +Format specific information: + compat: 1.1 + lazy refcounts: false + refcount bits: 16 + data file: TEST_DIR/t.IMGFMT.data + data file raw: false + corrupt: false +No errors were found on the image. +qemu-img: data-file-raw cannot be set on existing images +image: TEST_DIR/t.IMGFMT +file format: IMGFMT +virtual size: 64M (67108864 bytes) +cluster_size: 65536 +Format specific information: + compat: 1.1 + lazy refcounts: false + refcount bits: 16 + data file: TEST_DIR/t.IMGFMT.data + data file raw: false + corrupt: false +No errors were found on the image. *** done