[{"id":1772273,"web_url":"http://patchwork.ozlabs.org/comment/1772273/","msgid":"","list_archive_url":null,"date":"2017-09-20T21:38:34","subject":"Re: [Qemu-devel] [Qemu-block] [PATCH 2/2] qcow2: truncate the tail\n\tof the image file after shrinking the image","submitter":{"id":64343,"url":"http://patchwork.ozlabs.org/api/people/64343/","name":"John Snow","email":"jsnow@redhat.com"},"content":"On 09/20/2017 09:58 AM, Pavel Butsykin wrote:\n> Now after shrinking the image, at the end of the image file, there might be a\n> tail that probably will never be used. So we can find the last used cluster and\n> cut the tail.\n> \n> Signed-off-by: Pavel Butsykin \n> ---\n> block/qcow2-refcount.c | 21 +++++++++++++++++++++\n> block/qcow2.c | 19 +++++++++++++++++++\n> block/qcow2.h | 1 +\n> 3 files changed, 41 insertions(+)\n> \n> diff --git a/block/qcow2-refcount.c b/block/qcow2-refcount.c\n> index 88d5a3f1ad..5e221a166c 100644\n> --- a/block/qcow2-refcount.c\n> +++ b/block/qcow2-refcount.c\n> @@ -3181,3 +3181,24 @@ out:\n> g_free(reftable_tmp);\n> return ret;\n> }\n> +\n> +int64_t qcow2_get_last_cluster(BlockDriverState *bs, int64_t size)\n> +{\n> + BDRVQcow2State *s = bs->opaque;\n> + int64_t i, last_cluster, nb_clusters = size_to_clusters(s, size);\n> + uint64_t refcount;\n> +\n> + for (i = 0, last_cluster = 0; i < nb_clusters; i++) {\n> + int ret = qcow2_get_refcount(bs, i, &refcount);\n> + if (ret < 0) {\n> + fprintf(stderr, \"Can't get refcount for cluster %\" PRId64 \": %s\\n\",\n> + i, strerror(-ret));\n> + continue;\n> + }\n> +\n> + if (refcount > 0) {\n> + last_cluster = i;\n> + }\n> + }\n> + return last_cluster;\n> +}> diff --git a/block/qcow2.c b/block/qcow2.c\n> index 8a4311d338..c3b6dd44c4 100644\n> --- a/block/qcow2.c\n> +++ b/block/qcow2.c\n> @@ -3106,6 +3106,7 @@ static int qcow2_truncate(BlockDriverState *bs, int64_t offset,\n> new_l1_size = size_to_l1(s, offset);\n> \n> if (offset < old_length) {\n> + int64_t image_end_offset, old_file_size;\n> if (prealloc != PREALLOC_MODE_OFF) {\n> error_setg(errp,\n> \"Preallocation can't be used for shrinking an image\");\n> @@ -3134,6 +3135,24 @@ static int qcow2_truncate(BlockDriverState *bs, int64_t offset,\n> \"Failed to discard unused refblocks\");\n> return ret;\n> }\n> +\n> + old_file_size = bdrv_getlength(bs->file->bs);\n> + if (old_file_size < 0) {\n> + error_setg_errno(errp, -old_file_size,\n> + \"Failed to inquire current file length\");\n> + return old_file_size;\n> + }\n> + image_end_offset = (qcow2_get_last_cluster(bs, old_file_size) + 1) *\n> + s->cluster_size;\n> + if (image_end_offset < old_file_size) {\n> + ret = bdrv_truncate(bs->file, image_end_offset,\n> + PREALLOC_MODE_OFF, NULL);\n> + if (ret < 0) {\n> + error_setg_errno(errp, -ret,\n> + \"Failed to truncate the tail of the image\");\n\nI've recently become skeptical of what partial resize successes look\nlike, but that's an issue for another day entirely.\n\n> + return ret;\n> + }\n> + }\n> } else {\n> ret = qcow2_grow_l1_table(bs, new_l1_size, true);\n> if (ret < 0) {\n> diff --git a/block/qcow2.h b/block/qcow2.h\n> index 5a289a81e2..782a206ecb 100644\n> --- a/block/qcow2.h\n> +++ b/block/qcow2.h\n> @@ -597,6 +597,7 @@ int qcow2_change_refcount_order(BlockDriverState *bs, int refcount_order,\n> BlockDriverAmendStatusCB *status_cb,\n> void *cb_opaque, Error **errp);\n> int qcow2_shrink_reftable(BlockDriverState *bs);\n> +int64_t qcow2_get_last_cluster(BlockDriverState *bs, int64_t size);\n> \n> /* qcow2-cluster.c functions */\n> int qcow2_grow_l1_table(BlockDriverState *bs, uint64_t min_size,\n> \n\nReviewed-by: John Snow \n\nLooks sane to me, but under which circumstances might we grow such a\ntail? I assume the actual truncate call aligns to cluster boundaries as\nappropriate, so is this a bit of a \"quick fix\" to cull unused clusters\nthat happened to be near the truncate boundary?\n\nIt might be worth documenting the circumstances that produces this\nunused space that will never get used. My hunch is that such unused\nspace should likely be getting reclaimed elsewhere and not here, but\nperhaps I'm misunderstanding the causal factors.\n\n--js","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","ext-mx01.extmail.prod.ext.phx2.redhat.com;\n\tdmarc=none (p=none dis=none) header.from=redhat.com","ext-mx01.extmail.prod.ext.phx2.redhat.com;\n\tspf=fail smtp.mailfrom=jsnow@redhat.com"],"Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xyCmv0Jhvz9sBd\n\tfor ;\n\tThu, 21 Sep 2017 07:39:09 +1000 (AEST)","from localhost ([::1]:50776 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1dumi1-0003pD-GM\n\tfor incoming@patchwork.ozlabs.org; Wed, 20 Sep 2017 17:39:05 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:36326)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1dumhf-0003me-6I\n\tfor qemu-devel@nongnu.org; Wed, 20 Sep 2017 17:38:44 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1dumhe-00007G-38\n\tfor qemu-devel@nongnu.org; Wed, 20 Sep 2017 17:38:43 -0400","from mx1.redhat.com ([209.132.183.28]:49514)\n\tby eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)\n\t(Exim 4.71) (envelope-from )\n\tid 1dumhZ-0008Ud-6g; Wed, 20 Sep 2017 17:38:37 -0400","from smtp.corp.redhat.com\n\t(int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12])\n\t(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby mx1.redhat.com (Postfix) with ESMTPS id 215E081DF7;\n\tWed, 20 Sep 2017 21:38:36 +0000 (UTC)","from [10.18.17.130] (dhcp-17-130.bos.redhat.com [10.18.17.130])\n\tby smtp.corp.redhat.com (Postfix) with ESMTP id 2C7CB610B0;\n\tWed, 20 Sep 2017 21:38:35 +0000 (UTC)"],"DMARC-Filter":"OpenDMARC Filter v1.3.2 mx1.redhat.com 215E081DF7","To":"Pavel Butsykin , qemu-block@nongnu.org,\n\tqemu-devel@nongnu.org","References":"<20170920135833.20472-1-pbutsykin@virtuozzo.com>\n\t<20170920135833.20472-3-pbutsykin@virtuozzo.com>","From":"John Snow ","Message-ID":"","Date":"Wed, 20 Sep 2017 17:38:34 -0400","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101\n\tThunderbird/52.3.0","MIME-Version":"1.0","In-Reply-To":"<20170920135833.20472-3-pbutsykin@virtuozzo.com>","Content-Type":"text/plain; charset=utf-8","Content-Language":"en-US","Content-Transfer-Encoding":"7bit","X-Scanned-By":"MIMEDefang 2.79 on 10.5.11.12","X-Greylist":"Sender IP whitelisted, not delayed by milter-greylist-4.5.16\n\t(mx1.redhat.com [10.5.110.25]);\n\tWed, 20 Sep 2017 21:38:36 +0000 (UTC)","X-detected-operating-system":"by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]\n\t[fuzzy]","X-Received-From":"209.132.183.28","Subject":"Re: [Qemu-devel] [Qemu-block] [PATCH 2/2] qcow2: truncate the tail\n\tof the image file after shrinking the image","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"kwolf@redhat.com, den@openvz.org, mreitz@redhat.com","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}},{"id":1772628,"web_url":"http://patchwork.ozlabs.org/comment/1772628/","msgid":"<79eb6628-130e-af9d-cd39-56bf71194ae3@virtuozzo.com>","list_archive_url":null,"date":"2017-09-21T09:49:55","subject":"Re: [Qemu-devel] [Qemu-block] [PATCH 2/2] qcow2: truncate the tail\n\tof the image file after shrinking the image","submitter":{"id":67271,"url":"http://patchwork.ozlabs.org/api/people/67271/","name":"Pavel Butsykin","email":"pbutsykin@virtuozzo.com"},"content":"On 21.09.2017 00:38, John Snow wrote:\n> \n> \n> On 09/20/2017 09:58 AM, Pavel Butsykin wrote:\n>> Now after shrinking the image, at the end of the image file, there might be a\n>> tail that probably will never be used. So we can find the last used cluster and\n>> cut the tail.\n>>\n>> Signed-off-by: Pavel Butsykin \n>> ---\n>> block/qcow2-refcount.c | 21 +++++++++++++++++++++\n>> block/qcow2.c | 19 +++++++++++++++++++\n>> block/qcow2.h | 1 +\n>> 3 files changed, 41 insertions(+)\n>>\n>> diff --git a/block/qcow2-refcount.c b/block/qcow2-refcount.c\n>> index 88d5a3f1ad..5e221a166c 100644\n>> --- a/block/qcow2-refcount.c\n>> +++ b/block/qcow2-refcount.c\n>> @@ -3181,3 +3181,24 @@ out:\n>> g_free(reftable_tmp);\n>> return ret;\n>> }\n>> +\n>> +int64_t qcow2_get_last_cluster(BlockDriverState *bs, int64_t size)\n>> +{\n>> + BDRVQcow2State *s = bs->opaque;\n>> + int64_t i, last_cluster, nb_clusters = size_to_clusters(s, size);\n>> + uint64_t refcount;\n>> +\n>> + for (i = 0, last_cluster = 0; i < nb_clusters; i++) {\n>> + int ret = qcow2_get_refcount(bs, i, &refcount);\n>> + if (ret < 0) {\n>> + fprintf(stderr, \"Can't get refcount for cluster %\" PRId64 \": %s\\n\",\n>> + i, strerror(-ret));\n>> + continue;\n>> + }\n>> +\n>> + if (refcount > 0) {\n>> + last_cluster = i;\n>> + }\n>> + }\n>> + return last_cluster;\n>> +}> diff --git a/block/qcow2.c b/block/qcow2.c\n>> index 8a4311d338..c3b6dd44c4 100644\n>> --- a/block/qcow2.c\n>> +++ b/block/qcow2.c\n>> @@ -3106,6 +3106,7 @@ static int qcow2_truncate(BlockDriverState *bs, int64_t offset,\n>> new_l1_size = size_to_l1(s, offset);\n>> \n>> if (offset < old_length) {\n>> + int64_t image_end_offset, old_file_size;\n>> if (prealloc != PREALLOC_MODE_OFF) {\n>> error_setg(errp,\n>> \"Preallocation can't be used for shrinking an image\");\n>> @@ -3134,6 +3135,24 @@ static int qcow2_truncate(BlockDriverState *bs, int64_t offset,\n>> \"Failed to discard unused refblocks\");\n>> return ret;\n>> }\n>> +\n>> + old_file_size = bdrv_getlength(bs->file->bs);\n>> + if (old_file_size < 0) {\n>> + error_setg_errno(errp, -old_file_size,\n>> + \"Failed to inquire current file length\");\n>> + return old_file_size;\n>> + }\n>> + image_end_offset = (qcow2_get_last_cluster(bs, old_file_size) + 1) *\n>> + s->cluster_size;\n>> + if (image_end_offset < old_file_size) {\n>> + ret = bdrv_truncate(bs->file, image_end_offset,\n>> + PREALLOC_MODE_OFF, NULL);\n>> + if (ret < 0) {\n>> + error_setg_errno(errp, -ret,\n>> + \"Failed to truncate the tail of the image\");\n> \n> I've recently become skeptical of what partial resize successes look\n> like, but that's an issue for another day entirely.\n> \n>> + return ret;\n>> + }\n>> + }\n>> } else {\n>> ret = qcow2_grow_l1_table(bs, new_l1_size, true);\n>> if (ret < 0) {\n>> diff --git a/block/qcow2.h b/block/qcow2.h\n>> index 5a289a81e2..782a206ecb 100644\n>> --- a/block/qcow2.h\n>> +++ b/block/qcow2.h\n>> @@ -597,6 +597,7 @@ int qcow2_change_refcount_order(BlockDriverState *bs, int refcount_order,\n>> BlockDriverAmendStatusCB *status_cb,\n>> void *cb_opaque, Error **errp);\n>> int qcow2_shrink_reftable(BlockDriverState *bs);\n>> +int64_t qcow2_get_last_cluster(BlockDriverState *bs, int64_t size);\n>> \n>> /* qcow2-cluster.c functions */\n>> int qcow2_grow_l1_table(BlockDriverState *bs, uint64_t min_size,\n>>\n> \n> Reviewed-by: John Snow \n> \n> Looks sane to me, but under which circumstances might we grow such a\n> tail? I assume the actual truncate call aligns to cluster boundaries as\n> appropriate, so is this a bit of a \"quick fix\" to cull unused clusters\n> that happened to be near the truncate boundary?\n> \n> It might be worth documenting the circumstances that produces this\n> unused space that will never get used. My hunch is that such unused\n> space should likely be getting reclaimed elsewhere and not here, but\n> perhaps I'm misunderstanding the causal factors.\n>\n\nThis is a consequence of how we implemented shrinking the qcow2 image.\n(https://lists.gnu.org/archive/html/qemu-devel/2017-09/msg04580.html)\nBut on the other hand, if we need to shrink the qcow2 image without\ncopying the data, this is the only way. The same guest offset can be\nconverted to almost any host offset in the file i.e. the first guest\ncluster may be located somewhere at the end or the middle of the image\nfile. So we can't just take and truncate the image file on the border of\nthe truncation, therefore to shrink the image we just discard the\nclusters that corresponds to the truncated area. The result is a\nsparse image file where the apparent file size differs from actual size.\nAnd the tail in this case is the difference between the actual size and\nlast used cluster in the image, so in fact the cutting of the tail does\nnot change the apparent file size.\n\n> --js\n>","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","ozlabs.org;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=virtuozzo.com header.i=@virtuozzo.com\n\theader.b=\"ZbzQG9Rg\"; dkim-atps=neutral","spf=none (sender IP is )\n\tsmtp.mailfrom=pbutsykin@virtuozzo.com; "],"Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xyX0t1Dzcz9t4B\n\tfor ;\n\tThu, 21 Sep 2017 19:50:37 +1000 (AEST)","from localhost ([::1]:52657 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1duy7w-000530-4C\n\tfor incoming@patchwork.ozlabs.org; Thu, 21 Sep 2017 05:50:36 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:35877)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1duy7U-00050T-PH\n\tfor qemu-devel@nongnu.org; Thu, 21 Sep 2017 05:50:10 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1duy7Q-00046W-Qm\n\tfor qemu-devel@nongnu.org; Thu, 21 Sep 2017 05:50:08 -0400","from mail-he1eur01on0135.outbound.protection.outlook.com\n\t([104.47.0.135]:4923\n\thelo=EUR01-HE1-obe.outbound.protection.outlook.com)\n\tby eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32)\n\t(Exim 4.71) (envelope-from )\n\tid 1duy7P-00042v-RK; Thu, 21 Sep 2017 05:50:04 -0400","from [172.16.25.50] (195.214.232.6) by\n\tVI1PR0802MB2559.eurprd08.prod.outlook.com (2603:10a6:800:ae::9) with\n\tMicrosoft SMTP Server (version=TLS1_2,\n\tcipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.77.7;\n\tThu, 21 Sep 2017 09:49:59 +0000"],"DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed; d=virtuozzo.com;\n\ts=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;\n\tbh=fpUG/wZqs2InWlhRTJ+aKFX5JVlWpu+qvTJJzf+hbho=;\n\tb=ZbzQG9RgEeCsN7BVUT8bm3/kDqJL2fdOAsYhxQpd2pX1GRQlTtzg5+qAofFrKaJ7AXbMmk7IzUlosE9uagu/ZTRKiZycQgF09mENFVMheutuxpQfpUBzV8Nizf0aRWZhiRkAYjPlSXdUmpErmbbnxglDZKOYQHvbwhGMvKDmwUA=","To":"John Snow , qemu-block@nongnu.org,\n\tqemu-devel@nongnu.org","References":"<20170920135833.20472-1-pbutsykin@virtuozzo.com>\n\t<20170920135833.20472-3-pbutsykin@virtuozzo.com>\n\t","From":"Pavel Butsykin ","Message-ID":"<79eb6628-130e-af9d-cd39-56bf71194ae3@virtuozzo.com>","Date":"Thu, 21 Sep 2017 12:49:55 +0300","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101\n\tThunderbird/52.2.1","MIME-Version":"1.0","In-Reply-To":"","Content-Type":"text/plain; charset=utf-8; format=flowed","Content-Language":"en-US","Content-Transfer-Encoding":"7bit","X-Originating-IP":"[195.214.232.6]","X-ClientProxiedBy":"HE1PR0402CA0053.eurprd04.prod.outlook.com\n\t(2603:10a6:7:7c::42) To VI1PR0802MB2559.eurprd08.prod.outlook.com\n\t(2603:10a6:800:ae::9)","X-MS-PublicTrafficType":"Email","X-MS-Office365-Filtering-Correlation-Id":"ee94f623-5570-4344-c3f2-08d500d62032","X-Microsoft-Antispam":"UriScan:; BCL:0; PCL:0;\n\tRULEID:(300000500095)(300135000095)(300000501095)(300135300095)(22001)(300000502095)(300135100095)(2017030254152)(300000503095)(300135400095)(2017052603199)(201703131423075)(201703031133081)(201702281549075)(300000504095)(300135200095)(300000505095)(300135600095)(300000506095)(300135500095);\n\tSRVR:VI1PR0802MB2559; ","X-Microsoft-Exchange-Diagnostics":["1; VI1PR0802MB2559;\n\t3:5bPHeJjcZkLVv5rykLOclJnn32Zk6ETAoou3M4SG4xZGGNKFfzUBfqG9JPE87txpG5hY/LaTkicHk7TXnsUcq5fMCqMUpwE8J6e17C9Y3jWoQOudqzbmmuNdGewmPp3LY2GEYXHJ5+m2/or+YWrT+nBeIdX9ua2bmxOTR2stMtW3T1e0T+sg696gIkPi4wouO/4ApAZkrzq1sgaWJZzAr0a/vMBHwwSSz19ITe887pdFTZF/28r0dL54oszrM0/s;\n\t25:r7q0qfjrYg1UpDf85DsvhsV03LtPmZD2o91peIn/7aZ+AunNRYYRnqvFwTK6hCiEVH8Ltlu+GyHFrF46LWT4T4tlBOyFEgK3mQRhDVBVifwxeGDBq2LuYvYRFJzb5uwwXyBcjXamXyNshcXu5YTaHu6/xL6mpA2y37KrmG2N39k2GOpW2D2SIqbPIcu+Aqamy03varS/0Ij1JSifkkO8iQpovfeqaDqgaW6pLPl56b2akOZrhaGcBbyVOfcsxNPPDwqBXF/7Xfz+jjS8hys3gt8ezMVCD3YO93ySnlCWBh55IhqlGzEDBkgLGRTNy5HoNELkP7knHl0SI3ds/vBJVw==;\n\t31:UG6epVkarRJ9lIqko/wRC8B10CZNKpOgBg6wyVEFcBC7S6fqSCMhRGAYxB3IaA+Q0MAbgJVuRvtvB71suYqFzY3CHsAYfxQy3gUw1hKZeG4Gg+Cm4yFsnzR+EiqKCY+KYe0WFxWBrRhdpoTo16/nysyOxrftsdASjcf3v4shsFwFim3i+u5dTcie3KxW4f5J9aQdx7EsZgBGtxpYG6lvQo+skdXnX9YL9O0ur2sYkZc=","1; VI1PR0802MB2559;\n\t20:kBXq3Syps/99GyU85KjoToGl1DxpgUCPKRUzrCm2EQn/dWHtk60jbhNiYJwaieTbLVuMpz5aTUyKeIZH6BMkG7KIlRRju+zxmeLMg77ATFRFKZRKced8td20drIuzJxaDJZkrzLomcyCyL1CY42Fts/oyudc3rK6vUwMq6hzWNXdonExXmeJpin0C/SDmEv0bM32IHBNGGFzdmjuSWXGJemGJ8n9YSA/2xEd5Q7yYn67/SVRoH4mvJ5FtKnbBCKeyQ1u1Z/Z907qilsou98Nx7VmajV09ntzWK9O9ynyrj1Q2lMcLUtg6tMsCgmouYZ2Rs3mzu8BehfkDh1tTRTZBoEJnTreRNV05FiqzbpUy/PsWPw5TwvwAz6XeRGgBHJrJVY98ShA+Y4VHesheHsH4EKD/eWm0xFv4SV1zL1Kmbo=;\n\t4:uw2d9T54Ju8nccYz2eaCF2RpOl79w1KMGW9AcJ0PrAy0oYP5s7bcjkNcu7IbNg728WAPHsBGwXu5B0CV6XvPBiEKIbM96BPlI9a0+bgeY8wVa7TrUZyJWbUDlFVx+98oZun5LJVsWpSRjx0oawXPwQAb3C+BfCprJevdwvcCa0joQCm9QrGRiiEjWCJ77K/HaWbhQUbq+36WpqM0yLLT/D0HYe3Ctu+MIxBTpBIQBYt7Fm9C3VcjfQ99KrvcvZso0HQsbM/LUeazbz/BuVMNcMBBlM9kGufYdtZhe9g0TX1L/nz/6RTijGTZnlKHMX7l8Prtvzr1oguJyL+V6lSUMw==","=?utf-8?q?1=3BVI1PR0802MB2559=3B23=3A6N?=\n\t=?utf-8?q?Ps6/u1/qkRXCJq1OlIeFywl08pfwoJvL03PrH/XRzfkkUQSEdtS6Cg9v?=\n\t=?utf-8?q?XCTQ1sIKBIdPFAQ+gpW5NePWD3CaTTzzV44QVnm9F3thxSTRCVdJm1nC?=\n\t=?utf-8?q?dduZYsZN1pdZF2rPF6EHdQDOSEytC1kGZ0qzY4X0MtehoegUayTrq2yA?=\n\t=?utf-8?q?M5I2j9zOYSvcI6PwFiWRno87Lf5oVCG2c0aKqO1FMPMdCa+adfWsQXT9?=\n\t=?utf-8?q?m3gzQGVKTy/BHv5dOfuTs/xLIavpRSjI6ZIx+Ykp69tYC8nfHepaJ3y2?=\n\t=?utf-8?q?4eTX40FdnlO2tWD4il79+DROA6MY2vsq9ioJoKU/pgQbvwc4Oo4vg3qD?=\n\t=?utf-8?q?oaOloJapNI0+CD8Cbvhwyml4Bp0r65pODMbV8vMhcaqlHBT7X5OI4Mhn?=\n\t=?utf-8?q?paM5jVTs0+nXtT+5Nj2z9q4DnnMOuAfKZ2FkdJTG5DL/zXRkckyNz/UM?=\n\t=?utf-8?q?huedhNwqKoAox55KuDPnr9SMwDReI67CaUyYFLDMWysf3oX9nurNwMmq?=\n\t=?utf-8?q?m6mQ2aHRBIPOfVnxl5j3SfFnT5xwHsMcVETjb9jjF4Q/gErIxuzPOuRQ?=\n\t=?utf-8?q?8xsGpX9zxiDgcCvtSNmRDBIRAYe+ju71MdV3La2oM7C9I7DYTDzZ4MBj?=\n\t=?utf-8?q?7IMX5Tf4A7gjeuUQL/xRYanvTQF7VdaE9YeeeP7+cEV9BSAtyqRJ8xnL?=\n\t=?utf-8?q?UUCJUYFjoaNiX+yZl1K8VXW7z3GSOfxZC8qzcmGmwDjgNLdxkmPTFTPR?=\n\t=?utf-8?q?HuOyVOD7Cp5Ke33kqr6q3Hma7xJay5FjLHYnYweqBU8E8ecFrNcfdOlt?=\n\t=?utf-8?q?uTdpv8i+veYiKqKjwFOA0Q25YfX5m1Yzm6/JHiA8fSMjUI/KeNtjeUI9?=\n\t=?utf-8?q?O2jjf1GQhA3UXjPVqB7NNlR8tzWCiUYYETF6vSZPaJr39vBm1kvg2t8F?=\n\t=?utf-8?q?DKn9ycYRgDpKuiliKwsMQS3R3AWEfTIzi0gTI4Qr4WiqHcXDyoj4ecP8?=\n\t=?utf-8?q?OOnO0f8w80JwTGrkDBMnNGSVGZ6lA9K2zc359yqaW0xBWQkdgCWY95yh?=\n\t=?utf-8?q?GY6RowgXUzVBOO+x5iBT2efV+MS9Yy3N6bo/jI9WrvSGaxOwtKYDRjAs?=\n\t=?utf-8?q?TVlnVSFgSABalWfofxphP57s/Ypnic7/zNALc14Nlv3hsgoi9lvtm8To?=\n\t=?utf-8?q?8fkBc0A8bCN6+8ZW0RKgNQOJrUW5dOQIID2eYPgqmE47xx+kNmrbJ9rk?=\n\t=?utf-8?q?Qoo4ik0KM6RbwDQZACuJ0rkRTlaILe8GEJBu7Qx+IY/ArPjkWXhY4vgk?=\n\t=?utf-8?q?BFbo3NWkdJs2MTV8SM2GKVbVj1FDIsZ8RjeOCiDffvXpsw1bIMkhGxmY?=\n\t=?utf-8?q?GvYT9aKq5uaKm9DfxvHCrbdkHgFVCRa5EEdU6zCaOycAdJlYGHltzR?=","1; VI1PR0802MB2559;\n\t6:94h4zArK6X5sf8+zVUzPMfc4XFw+wfZtQi9YE1G/4OE0HA96i3KuCOx9ClycoB0DeDCZKELoJWuBU/HUX59/UB5ioQ3w0+S6ms3k5fBdZjTwq+ZYGWoi/FD8gqBQg7SjFdXBLvisc9TlaWD7XRVsQpBvYgV+vwLKdc1cYPVqBDyWhLVQrGvp3yKeZB0fNgYGxm1m5P+YbSJvy80UFb7xmw3BIgYSWxvu+jur8Z5ZGoHfKEo4qM4CPjC3IBdEUgpjJgKE/ii4BMoY2rPmj8wDjLn8wW2BM1gLezBxumKou/c1FkzQ6dEoYrfwCu+20ianz73eqCdavRdWSoR+Ndq6HA==;\n\t5:bCP10Bgiew1KCxgwbhia+y/K89/rDu1hx2+IX7g7ZTZ1BDQD1/BRU7d6pNXDjZm8vv9na3gLOyB1uQIirdA4duOHHYSMZR8LQqtC7wKjpPq3FrN0/VCU/XN+oqEGcX5Oq422IZ3lq2eKkY+VCeLibQ==;\n\t24:T/o7+WykzL1OS1l+IhT14Pkwdtmgn176SEjUC6Np+/TcjpWJtaqXwRywWFLSkCtcxwOX5yuJMvJoXYDdiz4gWoQTcUVMswSisIFhZ+ZVFU4=;\n\t7:Ub1/uIaBFSdWCHcfO3ETme54yqvbT0X1XZhF3/KGE4bkpnuRC577IiJNqfrReypvnev8ah6/PdCMIFW8CM/AHp8hgBZoYJCpfalmVoOx90V3OeCfG6JTTNEX9FuoVE1sqNxx2NGz45Fx5k5+BQ+87ea6el3IpUFMt9C5rjRdG15TG9Qshrrzp2YTNAt97nXmaSSK7Xg5Ccxrf272E5bfx8mjd2eS87E67GZ7W173y4Y=","1; VI1PR0802MB2559;\n\t20:oWoUfs9+sbnKs+hkSVotO9Z6qWdeFgw6sYt6q4yjImmpbGrtobpex1xKrwZhSLqNKWN/d8ocbXU86tSNBR2b59ZtsG3qu5sgBWesB51AyvoFtQzZAzfbOyxvz6DTPcKgXGIuej2Aguw44JJ8nAnSVO032VDY4lCxmmNejpTgnk0="],"X-MS-TrafficTypeDiagnostic":"VI1PR0802MB2559:","X-Exchange-Antispam-Report-Test":"UriScan:(226959686518644)(22074186197030);","X-Microsoft-Antispam-PRVS":"","X-Exchange-Antispam-Report-CFA-Test":"BCL:0; PCL:0;\n\tRULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(2401047)(8121501046)(5005006)(93006095)(93001095)(100000703101)(100105400095)(3002001)(10201501046)(6041248)(20161123564025)(20161123562025)(20161123555025)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123560025)(20161123558100)(6072148)(201708071742011)(100000704101)(100105200095)(100000705101)(100105500095);\n\tSRVR:VI1PR0802MB2559; BCL:0; PCL:0;\n\tRULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095);\n\tSRVR:VI1PR0802MB2559; ","X-Forefront-PRVS":"04371797A5","X-Forefront-Antispam-Report":"SFV:NSPM;\n\tSFS:(10019020)(6009001)(6049001)(346002)(376002)(199003)(189002)(24454002)(377454003)(58126008)(2906002)(6306002)(36756003)(31686004)(65826007)(50986999)(54356999)(76176999)(305945005)(97736004)(229853002)(478600001)(53546010)(6666003)(53936002)(101416001)(23676002)(6486002)(33646002)(77096006)(64126003)(68736007)(47776003)(316002)(230700001)(16576012)(16526017)(2950100002)(5660300001)(66066001)(65956001)(8936002)(65806001)(6116002)(105586002)(3846002)(106356001)(86362001)(25786009)(81166006)(81156014)(31696002)(189998001)(50466002)(8676002)(4326008)(7736002)(83506001)(6246003);\n\tDIR:OUT; SFP:1102; SCL:1; SRVR:VI1PR0802MB2559; H:[172.16.25.50];\n\tFPR:; SPF:None; PTR:InfoNoRecords; A:1; MX:1; LANG:en; ","Received-SPF":"None (protection.outlook.com: virtuozzo.com does not designate\n\tpermitted sender hosts)","SpamDiagnosticOutput":"1:99","SpamDiagnosticMetadata":"NSPM","X-OriginatorOrg":"virtuozzo.com","X-MS-Exchange-CrossTenant-OriginalArrivalTime":"21 Sep 2017 09:49:59.8648\n\t(UTC)","X-MS-Exchange-CrossTenant-FromEntityHeader":"Hosted","X-MS-Exchange-CrossTenant-Id":"0bc7f26d-0264-416e-a6fc-8352af79c58f","X-MS-Exchange-Transport-CrossTenantHeadersStamped":"VI1PR0802MB2559","X-detected-operating-system":"by eggs.gnu.org: Windows 7 or 8 [fuzzy]","X-Received-From":"104.47.0.135","Subject":"Re: [Qemu-devel] [Qemu-block] [PATCH 2/2] qcow2: truncate the tail\n\tof the image file after shrinking the image","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"kwolf@redhat.com, den@openvz.org, mreitz@redhat.com","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}},{"id":1772927,"web_url":"http://patchwork.ozlabs.org/comment/1772927/","msgid":"","list_archive_url":null,"date":"2017-09-21T15:28:19","subject":"Re: [Qemu-devel] [PATCH 2/2] qcow2: truncate the tail of the image\n\tfile after shrinking the image","submitter":{"id":36836,"url":"http://patchwork.ozlabs.org/api/people/36836/","name":"Max Reitz","email":"mreitz@redhat.com"},"content":"On 2017-09-20 15:58, Pavel Butsykin wrote:\n> Now after shrinking the image, at the end of the image file, there might be a\n> tail that probably will never be used. So we can find the last used cluster and\n> cut the tail.\n> \n> Signed-off-by: Pavel Butsykin \n> ---\n> block/qcow2-refcount.c | 21 +++++++++++++++++++++\n> block/qcow2.c | 19 +++++++++++++++++++\n> block/qcow2.h | 1 +\n> 3 files changed, 41 insertions(+)\n> \n> diff --git a/block/qcow2-refcount.c b/block/qcow2-refcount.c\n> index 88d5a3f1ad..5e221a166c 100644\n> --- a/block/qcow2-refcount.c\n> +++ b/block/qcow2-refcount.c\n> @@ -3181,3 +3181,24 @@ out:\n> g_free(reftable_tmp);\n> return ret;\n> }\n> +\n> +int64_t qcow2_get_last_cluster(BlockDriverState *bs, int64_t size)\n> +{\n> + BDRVQcow2State *s = bs->opaque;\n> + int64_t i, last_cluster, nb_clusters = size_to_clusters(s, size);\n> + uint64_t refcount;\n> +\n> + for (i = 0, last_cluster = 0; i < nb_clusters; i++) {\n> + int ret = qcow2_get_refcount(bs, i, &refcount);\n> + if (ret < 0) {\n> + fprintf(stderr, \"Can't get refcount for cluster %\" PRId64 \": %s\\n\",\n> + i, strerror(-ret));\n> + continue;\n> + }\n> +\n> + if (refcount > 0) {\n> + last_cluster = i;\n> + }\n> + }\n> + return last_cluster;\n> +}\n\nWouldn't it make more sense to start from the end of the image?\n\nMax","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","ext-mx05.extmail.prod.ext.phx2.redhat.com;\n\tdmarc=none (p=none dis=none) header.from=redhat.com","ext-mx05.extmail.prod.ext.phx2.redhat.com;\n\tspf=fail smtp.mailfrom=mreitz@redhat.com"],"Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xygWB2snvz9t49\n\tfor ;\n\tFri, 22 Sep 2017 01:28:54 +1000 (AEST)","from localhost ([::1]:54202 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1dv3PI-0005Pb-IK\n\tfor incoming@patchwork.ozlabs.org; Thu, 21 Sep 2017 11:28:52 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:35934)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1dv3Ow-0005OQ-8t\n\tfor qemu-devel@nongnu.org; Thu, 21 Sep 2017 11:28:31 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1dv3Ov-0004ah-9P\n\tfor qemu-devel@nongnu.org; Thu, 21 Sep 2017 11:28:30 -0400","from mx1.redhat.com ([209.132.183.28]:49668)\n\tby eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)\n\t(Exim 4.71) (envelope-from )\n\tid 1dv3Op-0004YI-TE; Thu, 21 Sep 2017 11:28:24 -0400","from smtp.corp.redhat.com\n\t(int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15])\n\t(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby mx1.redhat.com (Postfix) with ESMTPS id CC79590902;\n\tThu, 21 Sep 2017 15:28:22 +0000 (UTC)","from dresden.str.redhat.com (ovpn-204-199.brq.redhat.com\n\t[10.40.204.199])\n\tby smtp.corp.redhat.com (Postfix) with ESMTPS id 37B765EDE1;\n\tThu, 21 Sep 2017 15:28:21 +0000 (UTC)"],"DMARC-Filter":"OpenDMARC Filter v1.3.2 mx1.redhat.com CC79590902","To":"Pavel Butsykin , qemu-block@nongnu.org,\n\tqemu-devel@nongnu.org","References":"<20170920135833.20472-1-pbutsykin@virtuozzo.com>\n\t<20170920135833.20472-3-pbutsykin@virtuozzo.com>","From":"Max Reitz ","Message-ID":"","Date":"Thu, 21 Sep 2017 17:28:19 +0200","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101\n\tThunderbird/52.3.0","MIME-Version":"1.0","In-Reply-To":"<20170920135833.20472-3-pbutsykin@virtuozzo.com>","Content-Type":"multipart/signed; micalg=pgp-sha256;\n\tprotocol=\"application/pgp-signature\";\n\tboundary=\"gfrM87qRwCsCj946QDBplaDW8TbQfn0NC\"","X-Scanned-By":"MIMEDefang 2.79 on 10.5.11.15","X-Greylist":"Sender IP whitelisted, not delayed by milter-greylist-4.5.16\n\t(mx1.redhat.com [10.5.110.29]);\n\tThu, 21 Sep 2017 15:28:23 +0000 (UTC)","X-detected-operating-system":"by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]\n\t[fuzzy]","X-Received-From":"209.132.183.28","X-Content-Filtered-By":"Mailman/MimeDel 2.1.21","Subject":"Re: [Qemu-devel] [PATCH 2/2] qcow2: truncate the tail of the image\n\tfile after shrinking the image","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"kwolf@redhat.com, den@openvz.org","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}},{"id":1772931,"web_url":"http://patchwork.ozlabs.org/comment/1772931/","msgid":"<129077ca-af54-fe7c-40ee-850547f63cff@redhat.com>","list_archive_url":null,"date":"2017-09-21T15:30:38","subject":"Re: [Qemu-devel] [PATCH 2/2] qcow2: truncate the tail of the image\n\tfile after shrinking the image","submitter":{"id":36836,"url":"http://patchwork.ozlabs.org/api/people/36836/","name":"Max Reitz","email":"mreitz@redhat.com"},"content":"On 2017-09-20 15:58, Pavel Butsykin wrote:\n> Now after shrinking the image, at the end of the image file, there might be a\n> tail that probably will never be used. So we can find the last used cluster and\n> cut the tail.\n> \n> Signed-off-by: Pavel Butsykin \n> ---\n> block/qcow2-refcount.c | 21 +++++++++++++++++++++\n> block/qcow2.c | 19 +++++++++++++++++++\n> block/qcow2.h | 1 +\n> 3 files changed, 41 insertions(+)\n> \n> diff --git a/block/qcow2-refcount.c b/block/qcow2-refcount.c\n> index 88d5a3f1ad..5e221a166c 100644\n> --- a/block/qcow2-refcount.c\n> +++ b/block/qcow2-refcount.c\n> @@ -3181,3 +3181,24 @@ out:\n> g_free(reftable_tmp);\n> return ret;\n> }\n> +\n> +int64_t qcow2_get_last_cluster(BlockDriverState *bs, int64_t size)\n> +{\n> + BDRVQcow2State *s = bs->opaque;\n> + int64_t i, last_cluster, nb_clusters = size_to_clusters(s, size);\n> + uint64_t refcount;\n> +\n> + for (i = 0, last_cluster = 0; i < nb_clusters; i++) {\n> + int ret = qcow2_get_refcount(bs, i, &refcount);\n> + if (ret < 0) {\n> + fprintf(stderr, \"Can't get refcount for cluster %\" PRId64 \": %s\\n\",\n> + i, strerror(-ret));\n> + continue;\n\nOh, and another thing: If you decide to ignore errors here, I'd at least\nconsider the cluster allocated.\n\nOf course it would also be possible not to ignore errors, and instead\nreturn them to the caller which would then just not truncate the file.\n\nMax\n\n> + }\n> +\n> + if (refcount > 0) {\n> + last_cluster = i;\n> + }\n> + }\n> + return last_cluster;\n> +}","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","ext-mx07.extmail.prod.ext.phx2.redhat.com;\n\tdmarc=none (p=none dis=none) header.from=redhat.com","ext-mx07.extmail.prod.ext.phx2.redhat.com;\n\tspf=fail smtp.mailfrom=mreitz@redhat.com"],"Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xygZZ3pJZz9t4B\n\tfor ;\n\tFri, 22 Sep 2017 01:31:50 +1000 (AEST)","from localhost ([::1]:54217 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1dv3S8-00077Y-Li\n\tfor incoming@patchwork.ozlabs.org; Thu, 21 Sep 2017 11:31:48 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:38315)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1dv3Rk-00075q-RP\n\tfor qemu-devel@nongnu.org; Thu, 21 Sep 2017 11:31:29 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1dv3Re-0007t8-NY\n\tfor qemu-devel@nongnu.org; Thu, 21 Sep 2017 11:31:24 -0400","from mx1.redhat.com ([209.132.183.28]:56974)\n\tby eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)\n\t(Exim 4.71) (envelope-from )\n\tid 1dv3R4-0007DU-AO; Thu, 21 Sep 2017 11:30:42 -0400","from smtp.corp.redhat.com\n\t(int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12])\n\t(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby mx1.redhat.com (Postfix) with ESMTPS id 4AC8FC071886;\n\tThu, 21 Sep 2017 15:30:41 +0000 (UTC)","from dresden.str.redhat.com (ovpn-204-199.brq.redhat.com\n\t[10.40.204.199])\n\tby smtp.corp.redhat.com (Postfix) with ESMTPS id ACF7D610B0;\n\tThu, 21 Sep 2017 15:30:39 +0000 (UTC)"],"DMARC-Filter":"OpenDMARC Filter v1.3.2 mx1.redhat.com 4AC8FC071886","To":"Pavel Butsykin , qemu-block@nongnu.org,\n\tqemu-devel@nongnu.org","References":"<20170920135833.20472-1-pbutsykin@virtuozzo.com>\n\t<20170920135833.20472-3-pbutsykin@virtuozzo.com>","From":"Max Reitz ","Message-ID":"<129077ca-af54-fe7c-40ee-850547f63cff@redhat.com>","Date":"Thu, 21 Sep 2017 17:30:38 +0200","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101\n\tThunderbird/52.3.0","MIME-Version":"1.0","In-Reply-To":"<20170920135833.20472-3-pbutsykin@virtuozzo.com>","Content-Type":"multipart/signed; micalg=pgp-sha256;\n\tprotocol=\"application/pgp-signature\";\n\tboundary=\"CiwGHoVTMa0hdqTqjokl2ooBqdH8x5L5t\"","X-Scanned-By":"MIMEDefang 2.79 on 10.5.11.12","X-Greylist":"Sender IP whitelisted, not delayed by milter-greylist-4.5.16\n\t(mx1.redhat.com [10.5.110.31]);\n\tThu, 21 Sep 2017 15:30:41 +0000 (UTC)","X-detected-operating-system":"by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]\n\t[fuzzy]","X-Received-From":"209.132.183.28","X-Content-Filtered-By":"Mailman/MimeDel 2.1.21","Subject":"Re: [Qemu-devel] [PATCH 2/2] qcow2: truncate the tail of the image\n\tfile after shrinking the image","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"kwolf@redhat.com, den@openvz.org","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}},{"id":1772951,"web_url":"http://patchwork.ozlabs.org/comment/1772951/","msgid":"<45a6e398-ebbf-fb7c-8d42-535376ae4234@virtuozzo.com>","list_archive_url":null,"date":"2017-09-21T15:52:29","subject":"Re: [Qemu-devel] [PATCH 2/2] qcow2: truncate the tail of the image\n\tfile after shrinking the image","submitter":{"id":67271,"url":"http://patchwork.ozlabs.org/api/people/67271/","name":"Pavel Butsykin","email":"pbutsykin@virtuozzo.com"},"content":"On 21.09.2017 18:30, Max Reitz wrote:\n> On 2017-09-20 15:58, Pavel Butsykin wrote:\n>> Now after shrinking the image, at the end of the image file, there might be a\n>> tail that probably will never be used. So we can find the last used cluster and\n>> cut the tail.\n>>\n>> Signed-off-by: Pavel Butsykin \n>> ---\n>> block/qcow2-refcount.c | 21 +++++++++++++++++++++\n>> block/qcow2.c | 19 +++++++++++++++++++\n>> block/qcow2.h | 1 +\n>> 3 files changed, 41 insertions(+)\n>>\n>> diff --git a/block/qcow2-refcount.c b/block/qcow2-refcount.c\n>> index 88d5a3f1ad..5e221a166c 100644\n>> --- a/block/qcow2-refcount.c\n>> +++ b/block/qcow2-refcount.c\n>> @@ -3181,3 +3181,24 @@ out:\n>> g_free(reftable_tmp);\n>> return ret;\n>> }\n>> +\n>> +int64_t qcow2_get_last_cluster(BlockDriverState *bs, int64_t size)\n>> +{\n>> + BDRVQcow2State *s = bs->opaque;\n>> + int64_t i, last_cluster, nb_clusters = size_to_clusters(s, size);\n>> + uint64_t refcount;\n>> +\n>> + for (i = 0, last_cluster = 0; i < nb_clusters; i++) {\n>> + int ret = qcow2_get_refcount(bs, i, &refcount);\n>> + if (ret < 0) {\n>> + fprintf(stderr, \"Can't get refcount for cluster %\" PRId64 \": %s\\n\",\n>> + i, strerror(-ret));\n>> + continue;\n> \n> Oh, and another thing: If you decide to ignore errors here, I'd at least\n> consider the cluster allocated.\n> \n> Of course it would also be possible not to ignore errors, and instead\n> return them to the caller which would then just not truncate the file.\n\nYes, it seems so safer.\n\n> Max\n> \n>> + }\n>> +\n>> + if (refcount > 0) {\n>> + last_cluster = i;\n>> + }\n>> + }\n>> + return last_cluster;\n>> +}\n>","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","ozlabs.org;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=virtuozzo.com header.i=@virtuozzo.com\n\theader.b=\"IteC2jsl\"; dkim-atps=neutral","spf=none (sender IP is )\n\tsmtp.mailfrom=pbutsykin@virtuozzo.com; "],"Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xyh3W07Xjz9t4B\n\tfor ;\n\tFri, 22 Sep 2017 01:53:24 +1000 (AEST)","from localhost ([::1]:54290 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1dv3n0-0004Zt-2k\n\tfor incoming@patchwork.ozlabs.org; Thu, 21 Sep 2017 11:53:22 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:46779)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1dv3mL-0004YX-Io\n\tfor qemu-devel@nongnu.org; Thu, 21 Sep 2017 11:52:42 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1dv3mI-0006D8-9S\n\tfor qemu-devel@nongnu.org; Thu, 21 Sep 2017 11:52:41 -0400","from mail-db5eur01on0129.outbound.protection.outlook.com\n\t([104.47.2.129]:18752\n\thelo=EUR01-DB5-obe.outbound.protection.outlook.com)\n\tby eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32)\n\t(Exim 4.71) (envelope-from )\n\tid 1dv3mH-0006CK-O6; Thu, 21 Sep 2017 11:52:38 -0400","from [172.16.25.50] (195.214.232.6) by\n\tHE1PR0802MB2554.eurprd08.prod.outlook.com (2603:10a6:3:e1::19) with\n\tMicrosoft SMTP Server (version=TLS1_2,\n\tcipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.56.11;\n\tThu, 21 Sep 2017 15:52:33 +0000"],"DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed; d=virtuozzo.com;\n\ts=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;\n\tbh=up+3NSFEtGctTQ01kOO8ET9UR9oEfRB08Wzdm1R7pok=;\n\tb=IteC2jslW8/T+KztAhp2LvC6qBqKUXd3WYVYMCNoSk66dmyOdS4Gs4myKNX2m/q74RkRm/hd7+/euf7bux8d5//aYjkrt8RER9jvKqNoSED4BMfoOWZp+DhWq11GetBhUNIoxs71IRjM+bP19w+DgEi+lmgf3cCWMZC3FYHPU2c=","To":"Max Reitz , qemu-block@nongnu.org,\n\tqemu-devel@nongnu.org","References":"<20170920135833.20472-1-pbutsykin@virtuozzo.com>\n\t<20170920135833.20472-3-pbutsykin@virtuozzo.com>\n\t<129077ca-af54-fe7c-40ee-850547f63cff@redhat.com>","From":"Pavel Butsykin ","Message-ID":"<45a6e398-ebbf-fb7c-8d42-535376ae4234@virtuozzo.com>","Date":"Thu, 21 Sep 2017 18:52:29 +0300","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101\n\tThunderbird/52.2.1","MIME-Version":"1.0","In-Reply-To":"<129077ca-af54-fe7c-40ee-850547f63cff@redhat.com>","Content-Type":"text/plain; charset=utf-8; format=flowed","Content-Language":"en-US","Content-Transfer-Encoding":"7bit","X-Originating-IP":"[195.214.232.6]","X-ClientProxiedBy":"AM5PR0202CA0015.eurprd02.prod.outlook.com\n\t(2603:10a6:203:69::25) To HE1PR0802MB2554.eurprd08.prod.outlook.com\n\t(2603:10a6:3:e1::19)","X-MS-PublicTrafficType":"Email","X-MS-Office365-Filtering-Correlation-Id":"a6b45ca5-0a68-4973-a715-08d50108c667","X-Microsoft-Antispam":"UriScan:; BCL:0; PCL:0;\n\tRULEID:(300000500095)(300135000095)(300000501095)(300135300095)(22001)(300000502095)(300135100095)(2017030254152)(300000503095)(300135400095)(201703131423075)(201703031133081)(201702281549075)(300000504095)(300135200095)(300000505095)(300135600095)(300000506095)(300135500095);\n\tSRVR:HE1PR0802MB2554; ","X-Microsoft-Exchange-Diagnostics":["1; HE1PR0802MB2554;\n\t3:N5VpmbRXtLSWe8vgYiYVAgQS+uh670IuVISKgu03U8x4NqjP21dyusPby7QCY+oaPVuCRJDxNMXRKS1/xBlhuqIMZPMjlYSiBf8OLjAHlDrcL1DtZOxZaDFnpPVcacHMFs3u0ICjvVYa68dn1UwjqNp0nVtIZtP7n9lmD3OSi6yPaxjueh1HkK94BmHB8uMUAN2Qt3hnBGGT8H2zJTlB8yjF8B9OhR7j/hwDoZ4p4wechBVv7LtcnLz5Pbb74qN6;\n\t25:MQVlB3Qs76BElWuLu+JQmmvMC5rPrpOcYpZWQXrSP86FHhZHAQRk7cthNbP+YV/d1f+0BWCh65T4fJgjfcnSOo0F9hCahd9zvUbeAXekcMMNBE0ST5dU0ZF/gN4fhFWoU0X+LJXOgOyNYetkvpeE/CBGN+8tmxbh/jzG+ForuZqrxF0/v40sUwQd+5hvN6F6Sf53DCTbFGnNhD4/0VSyje7ghVyY414JLPxwcJ8F0eat8yfN8G6tj4iOSKBOjoydq1iZPrja62luAER1NILso5oLFKzbauvCpahQHz+6Dlp8lJt6Ki+Qn/M7ZX3oBu2j6Ut5ERa3hEPhQJNtLg51Pw==;\n\t31:QDPxwR+hRgoMEldGgsZkOBt0vLtgZK6a/c0R4fT0Yehl7Njo8Z5Mo9mVoUG+oSg5ysOb4ktP73P4GuOkr0PkNgtn02RwHuwFlQBmgceR9K34bhdDDg50E+uO3pHh+3rIJp1AjTuIVfXixvvhDz14e0b7uOv8WuBHe57eGIy21hk8u1NS7XR07Oj51Yy/GBE2SmIEUcA/V9/Che1v+WQt9oJNL3GMJ1TJsCXOHbG5NnI=","1; HE1PR0802MB2554;\n\t20:8h8BKFzvM4FPiPRlxJoQTPkyhhcIaExj2W7Shglb/H3muRTQjm7+qMeTlNZrstbWxNjd5+AurNCftByks1NZYLO4e5XVH8VwYktYnBzbr3Fv/5c7lgtwVu8jdxTus7mU8LpgoHv8AskXWxDpflxec5kAVhX8IsU/O+vy6pywCpDUdj1Xk82+CEQfnwpioRZa2FUMPslULj1vMJChsSKltUjVkgnyW8qmlOaRcmDttNwzP9deop7SSPt2OiTF14LLVhzZxp0fwtN0HLejCaYGXF/q0A5fP+HCk90rDhkMullJNtQNwsuEpB2FQzjOQvxU9JBS9aKkz5cO9HOwjh25buvFUtbx6ydUX/f6rNOnPMZSes4Dnr3xVLmSME7x/0z/1W5GhRX0uKwBWUS2aAJM9+wTEqEBA/Cb4S8XXNJoJZ0=;\n\t4:JXGVKkFxRlFqG3sIajZ70XIM5COqrYk/NwOGiRdAxRhIZIKALxhoBGiulOl+XAziJdrPPST8aMqqHogqNCuh9r80xm4gJiLQ7Hc35WwsaZA+FrZw0xpHlxG3apwvjuJPKVi5ns3bIw1aYL0cQnjLnOu7KYCYisax0Kfkj3yr/RJgpBjmvDbK9yqVSZmlS07F6kQInj7zk+oljfZDdXuT3VMiKOXDfWG5eub1iX2gIUugvwF6BY6E5WKIHyT4tPWr","=?utf-8?q?1=3BHE1PR0802MB2554=3B23=3ACT?=\n\t=?utf-8?q?AK57KfNyBDlFmmQp2q6UfXMlA6g+WsYDm5LIplWuxLNZU2cc6YlSnQZ2?=\n\t=?utf-8?q?9faTW7GW82kSAjY91dzV0faWo/mzq75CTYv/1/5YrkKEF+hP9V4T7a/b?=\n\t=?utf-8?q?ksGkVQLWWC28pM0KHiBW4ym58RGHPPVC4xF5jIbcu1TTJSUTOJ8/b84v?=\n\t=?utf-8?q?9Hki1DZRK4XJKgpOCOBMgE0s08PdumD5MkUHuf56jEnKtfmxGX4Y0d0B?=\n\t=?utf-8?q?5ikrO684+Uq2IGRM+lovWHrTvneeh48Zscu6Ggn1Ie1Ph+B97NsUo548?=\n\t=?utf-8?q?eDmbIrQCrI7jc3muM0ir1yBB91+/b1LBHZJ4/MwCkiqD3/LzYtRmiz9f?=\n\t=?utf-8?q?oECG4XzH/UEMCQ/OzcM1wrp49JlwQz5buqZoEjVgCb+SX05dB30Fi43i?=\n\t=?utf-8?q?mTALrtvCaPK3JV75j7N1QZT03yEFcs8Az7AhQt5xi22qQYZEe1qt/IXo?=\n\t=?utf-8?q?VDvoqEDDrIHjBUMDjXQ1opY+5sZH+ochru7CWIAsE5id9ErluaQbmBzm?=\n\t=?utf-8?q?IckfJOeDsSvqIb7FzQhqdNFKQ4hlO1k2nXSTR42A46Sn6Fgqcg5+uKvj?=\n\t=?utf-8?q?kni8TX8ju8529iC0bSLsYOCHjwlkENJkUdRVjhtyiRfaodN9Q5Z/B+UY?=\n\t=?utf-8?q?iZe8XdgE/npi/z4DhM4+wu3gntGTimJp9LzVy7c1LhB8gSQTysPo/doz?=\n\t=?utf-8?q?7Ngho62ubcOJ7zXebJXYu2VI6QaHXud4eFECs7h809PixJp0HBVoL4Ah?=\n\t=?utf-8?q?dlEr+MwnWEQQf0DYmrp24gdQCOQopuGpkRBgJa6ttWLJg7XFWpIWU3OO?=\n\t=?utf-8?q?sBG1RrovW9rBnowDad41TsJGgCGs0tX0V/WlkK31jW8YR7Da/WgezIPF?=\n\t=?utf-8?q?++M60LRLXP2B2+eYBhjjMGycwZbfqJ0WK1KBK9SMqCYqDto2BnyKWRJ8?=\n\t=?utf-8?q?oJylTiaLPRg18lO9UyAAt+D43eGgbj8qNH3Eawq2U+bxlXvY3SEozreF?=\n\t=?utf-8?q?bYoHbezyhK3InvA24slsi3O2Fz3AObtSeC1mAbFLvnVerHKJek4Xk0nm?=\n\t=?utf-8?q?6MDS6M9ew4QnVangO5BcxYYpxiSjOqN5mpfO6u3AJwKXPeRMouxkXfvv?=\n\t=?utf-8?q?7PgRI6Z6PtU2Qm762rzAQF09WSQl2pO0VyQyIJI1zfdrmLpjHnb/TIKg?=\n\t=?utf-8?q?F9G82kcXpczak/NOAj/8ZBbAMitCxzm4BnjbPsTSirVuBwK8Bm1lv8SQ?=\n\t=?utf-8?q?9WMzsaKpQKZO0RaC8DOOHoQQROC96z83nqq39Hl3MhYP9N1OwZ8KfF+2?=\n\t=?utf-8?q?oy3aAPTjdcjhny583Q1kOuY9hV3YxX1IX35WoCVP3eQbBDd/FKRwnPgP?=\n\t=?utf-8?q?eGSGEuiHZYBFpwteLvOV6gCPz7LgElxIDt7/GxdPVmN6B8WcBKqUCszF?=\n\t=?utf-8?q?ESflMBXje+mKyCUM55f50ZsXr1OC/gVAuStZboFDZUBCoN1HpskWO7HI?=\n\t=?utf-8?q?ldiySI?=","1; HE1PR0802MB2554;\n\t6:ECmfxNMzaLP5uNmCdfAlN9j6Vus/JlOu5YRWI0ytRUwAA0QWr4+bTvRd4tBybsrQxuIZBc9XniwCZToGKNx+zwD3s3oZKv55bYSHQZ4uzNcVyAp8mqXQn0JvZnl5fX4XgGo+BQBP+Y3sx412VXZCqGNhwjdgAje9gbNZG7geKV9OOWGE7qNdI7G5c30/ZFf/9po04i+1dnGn5UfMIxRZO5ycme6MJ8ykkIn0T5dUdALBMFOxq/TwhtP4PSX9ODWsBtuHst796hmYKMRHKnnHjUZ8wXBMq8aAZ2eahpy7D1c24IfqspTCiIkIPb8H/ifoG3cnWyZHtOPYMkkLfGMthw==;\n\t5:7oMLxB3eoTkGRsGcph6beBaDrNm7gXnBt9+S2/eu7QYBr8W4iLLi4JrzIudW4JPxbV/6XIorl1rRmVh+vs4rUIafZ24h0RxyX2Nwe5VfQKwdHrB4oFQCO9ZelSXxfbOfPd7qXpAcnB+8A0JkHe0T8g==;\n\t24:rzKAeSTub/SwHABDjyVhnB/T+pKstI9z0ObKwg3YcdYrW65q4dgOU2Qw2QepLXGu8n6OuRHhTWV7ZbQIVET+QETBEcvmgNYVBE2WCWxi0u4=;\n\t7:laEmNq8mU/YG14BPiYPr0Kmnaqxf85FZG8XWP68UGUqD7Sc/x8/qZf3tE1ppOKe5ZQpyTlcm5hBmkKdwl79MUInlTn7GjQPnsq5UFA4DNBL/stglgAgJ7r07/XCoCbiHWCuM0QbyRZPAy4Y7JPlsSMIIW8T/WliU0pK9xG8UD/BlCcJJHzz7tdFppcINwMQ2ABT1pBh7rMv1nt9r5AvECj7zall+DL4tLPtgO5pEtyc=","1; HE1PR0802MB2554;\n\t20:YfCFHWElv1Qu9GhD/PjlOCYxM82KxbGYlAIaqm0es470+XySjZqWhVbk2f9Zhy+FTRiY7HnYQB4aeJUSEY0igzujy8XeEC780HKbMG/W+wwvf9tyXv7YCgLraheQcPF06dMifyNYJWRoY9mDYWFoWBIHbQNDXMKP7pViCGH0rKo="],"X-MS-TrafficTypeDiagnostic":"HE1PR0802MB2554:","X-Exchange-Antispam-Report-Test":"UriScan:;","X-Microsoft-Antispam-PRVS":"","X-Exchange-Antispam-Report-CFA-Test":"BCL:0; PCL:0;\n\tRULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(2401047)(5005006)(8121501046)(93006095)(93001095)(10201501046)(100000703101)(100105400095)(3002001)(6041248)(20161123562025)(20161123564025)(20161123555025)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123558100)(20161123560025)(6072148)(201708071742011)(100000704101)(100105200095)(100000705101)(100105500095);\n\tSRVR:HE1PR0802MB2554; BCL:0; PCL:0;\n\tRULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095);\n\tSRVR:HE1PR0802MB2554; ","X-Forefront-PRVS":"04371797A5","X-Forefront-Antispam-Report":"SFV:NSPM;\n\tSFS:(10019020)(7370300001)(4630300001)(6009001)(6049001)(376002)(346002)(199003)(24454002)(377424004)(189002)(3846002)(33646002)(4326008)(31686004)(86362001)(101416001)(81156014)(230700001)(54356999)(16576012)(31696002)(50986999)(8676002)(76176999)(81166006)(16526017)(68736007)(189998001)(83506001)(305945005)(2906002)(7736002)(478600001)(107886003)(97736004)(58126008)(6246003)(53936002)(23676002)(47776003)(25786009)(5660300001)(36756003)(53546010)(50466002)(65806001)(316002)(65956001)(65826007)(66066001)(77096006)(6116002)(7350300001)(64126003)(6666003)(105586002)(2950100002)(106356001)(229853002)(6486002);\n\tDIR:OUT; SFP:1102; SCL:1; SRVR:HE1PR0802MB2554; H:[172.16.25.50];\n\tFPR:; SPF:None; PTR:InfoNoRecords; A:1; MX:1; LANG:en; ","Received-SPF":"None (protection.outlook.com: virtuozzo.com does not designate\n\tpermitted sender hosts)","SpamDiagnosticOutput":"1:99","SpamDiagnosticMetadata":"NSPM","X-OriginatorOrg":"virtuozzo.com","X-MS-Exchange-CrossTenant-OriginalArrivalTime":"21 Sep 2017 15:52:33.6591\n\t(UTC)","X-MS-Exchange-CrossTenant-FromEntityHeader":"Hosted","X-MS-Exchange-Transport-CrossTenantHeadersStamped":"HE1PR0802MB2554","X-detected-operating-system":"by eggs.gnu.org: Windows 7 or 8 [fuzzy]","X-Received-From":"104.47.2.129","Subject":"Re: [Qemu-devel] [PATCH 2/2] qcow2: truncate the tail of the image\n\tfile after shrinking the image","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"kwolf@redhat.com, den@openvz.org","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}},{"id":1772962,"web_url":"http://patchwork.ozlabs.org/comment/1772962/","msgid":"<175657ad-e6b4-0323-4f32-db9217be6bac@virtuozzo.com>","list_archive_url":null,"date":"2017-09-21T16:16:56","subject":"Re: [Qemu-devel] [PATCH 2/2] qcow2: truncate the tail of the image\n\tfile after shrinking the image","submitter":{"id":67271,"url":"http://patchwork.ozlabs.org/api/people/67271/","name":"Pavel Butsykin","email":"pbutsykin@virtuozzo.com"},"content":"On 21.09.2017 18:28, Max Reitz wrote:\n> On 2017-09-20 15:58, Pavel Butsykin wrote:\n>> Now after shrinking the image, at the end of the image file, there might be a\n>> tail that probably will never be used. So we can find the last used cluster and\n>> cut the tail.\n>>\n>> Signed-off-by: Pavel Butsykin \n>> ---\n>> block/qcow2-refcount.c | 21 +++++++++++++++++++++\n>> block/qcow2.c | 19 +++++++++++++++++++\n>> block/qcow2.h | 1 +\n>> 3 files changed, 41 insertions(+)\n>>\n>> diff --git a/block/qcow2-refcount.c b/block/qcow2-refcount.c\n>> index 88d5a3f1ad..5e221a166c 100644\n>> --- a/block/qcow2-refcount.c\n>> +++ b/block/qcow2-refcount.c\n>> @@ -3181,3 +3181,24 @@ out:\n>> g_free(reftable_tmp);\n>> return ret;\n>> }\n>> +\n>> +int64_t qcow2_get_last_cluster(BlockDriverState *bs, int64_t size)\n>> +{\n>> + BDRVQcow2State *s = bs->opaque;\n>> + int64_t i, last_cluster, nb_clusters = size_to_clusters(s, size);\n>> + uint64_t refcount;\n>> +\n>> + for (i = 0, last_cluster = 0; i < nb_clusters; i++) {\n>> + int ret = qcow2_get_refcount(bs, i, &refcount);\n>> + if (ret < 0) {\n>> + fprintf(stderr, \"Can't get refcount for cluster %\" PRId64 \": %s\\n\",\n>> + i, strerror(-ret));\n>> + continue;\n>> + }\n>> +\n>> + if (refcount > 0) {\n>> + last_cluster = i;\n>> + }\n>> + }\n>> + return last_cluster;\n>> +}\n> \n> Wouldn't it make more sense to start from the end of the image?\n\nIf this will reduce the iterations, then yes. But it will depend on the\nsituation. If you truncate the image more than half, it can increase the\nnumber of iterations. But intuitively it seems that to start from the\nend would be better :)\n> Max\n>","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","ozlabs.org;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=virtuozzo.com header.i=@virtuozzo.com\n\theader.b=\"Ur3ivWv0\"; dkim-atps=neutral","spf=none (sender IP is )\n\tsmtp.mailfrom=pbutsykin@virtuozzo.com; "],"Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xyhbQ6C3xz9t4B\n\tfor ;\n\tFri, 22 Sep 2017 02:17:38 +1000 (AEST)","from localhost ([::1]:54543 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1dv4AS-0001Qn-Mr\n\tfor incoming@patchwork.ozlabs.org; Thu, 21 Sep 2017 12:17:36 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:53717)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1dv4A0-0001Op-61\n\tfor qemu-devel@nongnu.org; Thu, 21 Sep 2017 12:17:09 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1dv49v-00009h-UQ\n\tfor qemu-devel@nongnu.org; Thu, 21 Sep 2017 12:17:08 -0400","from mail-eopbgr50105.outbound.protection.outlook.com\n\t([40.107.5.105]:1377\n\thelo=EUR03-VE1-obe.outbound.protection.outlook.com)\n\tby eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32)\n\t(Exim 4.71) (envelope-from )\n\tid 1dv49v-00008x-81; Thu, 21 Sep 2017 12:17:03 -0400","from [172.16.25.50] (195.214.232.6) by\n\tHE1PR0802MB2553.eurprd08.prod.outlook.com (2603:10a6:3:e1::18) with\n\tMicrosoft SMTP Server (version=TLS1_2,\n\tcipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.56.11;\n\tThu, 21 Sep 2017 16:17:00 +0000"],"DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed; d=virtuozzo.com;\n\ts=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;\n\tbh=EqpKvOTdKvY8/7meQgZQiDE6EE1WWwxS+t/g7MLql1w=;\n\tb=Ur3ivWv0PMQj1wHUYv9RA32ecXlvIny3pMuUV0jUM95B7808LQjf+h1vBALcb6MTnLNTFXKJk9szQogEoRKhrG9cCluE1NTw+ykb2kKJNBoj26dlO7s/E440c6GpWNDo3gdxvrzJu+R3Hj3b48p4F5CLzgZiYLC+zLxH44DRcss=","To":"Max Reitz , qemu-block@nongnu.org,\n\tqemu-devel@nongnu.org","References":"<20170920135833.20472-1-pbutsykin@virtuozzo.com>\n\t<20170920135833.20472-3-pbutsykin@virtuozzo.com>\n\t","From":"Pavel Butsykin ","Message-ID":"<175657ad-e6b4-0323-4f32-db9217be6bac@virtuozzo.com>","Date":"Thu, 21 Sep 2017 19:16:56 +0300","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101\n\tThunderbird/52.2.1","MIME-Version":"1.0","In-Reply-To":"","Content-Type":"text/plain; charset=utf-8; format=flowed","Content-Language":"en-US","Content-Transfer-Encoding":"7bit","X-Originating-IP":"[195.214.232.6]","X-ClientProxiedBy":"HE1PR0102CA0016.eurprd01.prod.exchangelabs.com\n\t(2603:10a6:7:14::29) To HE1PR0802MB2553.eurprd08.prod.outlook.com\n\t(2603:10a6:3:e1::18)","X-MS-PublicTrafficType":"Email","X-MS-Office365-Filtering-Correlation-Id":"d5c6d758-7afb-4093-2613-08d5010c30b8","X-Microsoft-Antispam":"UriScan:; BCL:0; PCL:0;\n\tRULEID:(300000500095)(300135000095)(300000501095)(300135300095)(22001)(300000502095)(300135100095)(2017030254152)(300000503095)(300135400095)(201703131423075)(201703031133081)(201702281549075)(300000504095)(300135200095)(300000505095)(300135600095)(300000506095)(300135500095);\n\tSRVR:HE1PR0802MB2553; ","X-Microsoft-Exchange-Diagnostics":["1; HE1PR0802MB2553;\n\t3:XrXKfAp60Cp1jw0GTFRtTRzkXoe5B0xF/MTF9EWgksPhebXASaAGgdiutYMcdmSxpW1Wx4Nc7oBPjIpm43oUfbmiZtJKuBKar/XOMlzgfn263uuodnrm8R7ZRYcV/1ETo2Fc45bp/GacdaVOYQRxtM2/wgXk1UODM0qa7qvEa3BEF3uiOK7GZtwKLwSelQPky52K9W8jgOIG/EuTYG46RwfSBFb85ln1+lmybRR+/D7FCJqo5k2gAACCPv3NO9sr;\n\t25:6FMeh51vwSqm78dPsGStkpfPzEC4Cri42Cejclkg+z7+J6Pdn6y2LyM3EQUSbCBQRXJd45BzMnDfa3U7Cu4qEzNB//cOblrPWcO7oPVwLfv/mRGZSNYY3GDhwWvCrCQdY6NCpf9oIh+3P45F3aAiuk/Qelt8U6/ZK7XTQOoDlBZi6CVR82suDNx/CsDsKMRk1HIsH6KeZtrVUjIq56cZboN9z2g676YcozYia1yz5TO4MOqU2fVicYONda5ovAgVUi4+nw95PbAhXPbcuQQ9RHrIJUVX6PEGxg3SQb5s8FBh1QzLNWVyrWb78nSl4v/lM+4WOu/qxjxjhP0kkM/Qmw==;\n\t31:WrY+H1zA+LcfAdWXJKOaEqqcLH17f0W79WJEbVKRSbTMb5hqDfQc+7nRReb32GiGcl37ALEI5bHKokfnsVbAOJ3ftWe5cePVF2UI03nB50S+soVVydlnZb03q/cv2uKwutFocooWSO56ceAnQw12obbmbFxwnyBhpGn4tARzVKKGNQL7paqp1f8fD8dhYrreWVfVhg4Nv4J6fLeZn7EfoLxOiBrCrGASSd0ak1K7mKM=","1; HE1PR0802MB2553;\n\t20:Wfi7KHSfwY4s0CNMoiI22v/65eAexok/iiVDxAgN7D3LdkWOfQp34vSEeovnc8HZhj/TP42mgTgRHYatFFpSO9E1EX/pfcMVvSnfU3+1AMeaVDSwrYfmuz1Hqv2t+UqPWoxT1FQUR/hl/q3JAuD5czypkDO+6sr8ufo6K9odul7jaP7kx0lzFmXEvQhFSq3SXEgLOghTmeGSoLS5HgITKadNmDdS4PwWlWA5sPSjK5YWt+cpM7JHSBRWzSkNpxl9xoKKyplbbw4hH+IkZawQ/duqZDnOqi+2AC2J1LD72Q5+cwaGa1QWTnprGyeE85Krq/MeNm5ZTWZrdU3DC6FOl2YJ15EEWlywUsGoaE/n0Gts2nny6TsmL/XLMAezb412zBf44D/5NvvB00bw9WdoljsU5IsBzfOkx0BKRJE85aw=;\n\t4:h2hVQERbXdbR7NMWLJN1kLy8AHV7wmyHuy3q3U5v98NKxw9P+0yfL8Kwp4W5bbYCcvtbw2Mrh4NfyEDYptNfIpoODrAhgof2Mtn2r5r5kkQbfO0/ZkkPEWQ2oHCqNIc5VBz83cQnx4UYeTABghDbiGPNSDF2x8Ppb/Z4KbVb3rZbBAK0NhLKc7CCpUoJ+BO+HVkA7jh32c5PoZBVRZZgwaXJINcRgh+kthLGmp8jJURftVVe4vClx/Hme2NDQFgT","=?utf-8?q?1=3BHE1PR0802MB2553=3B23=3A9P?=\n\t=?utf-8?q?itEQHcepTkIeVduQEW6BNBeBWvXtPct7iyVc53uNZyrG4HuNTt4dFBfD?=\n\t=?utf-8?q?K6kGMzlNHgXRuilHrXFdziqFzmINH+gg/jwKW3yx201df7UcbKz5+ih7?=\n\t=?utf-8?q?F39sCr5+mGKG4X4C+bTJe76579QGnLqBrDlyaU4wCfqUa6fNeWRHFZ1v?=\n\t=?utf-8?q?m8B7+VKJs0nVCKXg2DmbgVp0vxYoMin2mlQOoLSlQICtGSG9gDwqn02J?=\n\t=?utf-8?q?8nX/jnstTxxB8bEIkSY7vt3VSDBF1xXSyXh4RVpBh/FMBq4L9wE88TK5?=\n\t=?utf-8?q?RaK6YbwAaqDSivmePnylIjJMqURrMJdygrniEOHdiU8z/E6CWjqNii48?=\n\t=?utf-8?q?D4TrOYpTPucYRiHuV00nWqrj29UyPBsdQUuUzgNrid4x5AzJPw46jbgz?=\n\t=?utf-8?q?FHZf7KEXQ1TEJLSkxlVnkzjH5sK+keZ0zVbjZvtiw22fxe0gQytvzsQI?=\n\t=?utf-8?q?TSTL/26dNSwwWKvfz8MjziCRDX4LPOJu78xxJOXwj4DbRBCiFnqMbvcr?=\n\t=?utf-8?q?fC5Wg7erELBgmCkwmXmx9uzcAsdAs/nUHPktgx9MNevabTzR79X96w6f?=\n\t=?utf-8?q?5/2k2P1HGyh/VScnVA7ygZAUUcYBcIcyuU1UWe6ZmhIXtitYAPQYk0/L?=\n\t=?utf-8?q?a0lZGw5wTqvHQ6q0jnFqU1Vec3OdboP2nbA4GaQmTFH57GDm3PZl6+LB?=\n\t=?utf-8?q?CJNNIt96jl5ZkoB681SDypyH9Cny3TP6qWgm/hm4/jhPe8Hoa7EmUkdf?=\n\t=?utf-8?q?ysi4Ho9/L+vSylk8dqGZjtu3sNs+Lg8D7JYdzTGc3kRUZ+dzR0BYrsuK?=\n\t=?utf-8?q?7Fh12CnW6O/EW5lDXS0C8GxM48ZheM1kd3TIx5J7xoMfjFv5JnW3gtTa?=\n\t=?utf-8?q?WZg7NLywHZjS/3hfnG7iawD3HvT6RBCPeOSDBWorBZCM9IYIgEFu9UGv?=\n\t=?utf-8?q?v2+KOuLjdSC8Cn89uqryXX1OIh44DRLhjGzWIrY4+QD8caQFFjXRGBrR?=\n\t=?utf-8?q?iOp0kKMjd03kHjHp8NNppCOGVL7Dip6ezsE5l8UhGkvTCE96/Hho4zxf?=\n\t=?utf-8?q?Vw0t8mALmLZk4891/Hp9iWGpEqur6yRFrXwMIxUgKtyPgjq0Cqvquu4M?=\n\t=?utf-8?q?+I/brarRboUrEmKDPS8avF0zp3aRTVAmiLxyR7THakXnMpRHNpoKMp0X?=\n\t=?utf-8?q?wfXxH15Aot9kO0Yym1IOO9KD1jw5XX+QTsqomQl6noFkqy6VzNkPen4+?=\n\t=?utf-8?q?4rD8RRt3fgTb1+/DE+ugw/dE3DFV+EBk0iwVxpWbb0jry42nyW3DW14T?=\n\t=?utf-8?q?tVmiP+CvVXhKwG/H5IaiCy37DlU/HC5c9HWLYFb6AIh7dN7krqmRvqEX?=\n\t=?utf-8?q?fQ4zWovLeglK3kxafbCFVdp6PdCf6NLve/WH973PCr6iW0KydBdUMTjY?=\n\t=?utf-8?q?G6zjQ3JNoM4lr9/3qP/QzvPrGua5oPoSJUl5TP8HxZ6KBIX8QfbP1rbN?=\n\t=?utf-8?q?/Maekl?=","1; HE1PR0802MB2553;\n\t6:Y0DmrrUxJYR90xwDmkvbzjVCtRCdzGiS5bxonW2YgF886WWu/jCM2eRjei8zJg3ziXIcx1sUtsd3pGMh4/fNu7SDYLcijSTNYuwae9rUSjv6/47124Ni6v7GRhexr9DiQAqB57HYUqIrVC9Y4eHUhRTsq8RAumu6ZG+42En/OdShUjE3Agv6l6dSfTTxPD/+PwYFDBxUMFZBr2tb5cRjAG6UOLfnZNYyUK7tKx2WImin8FGVt7DST0Se7GM5sKe3gtrjO/ePiL6krFEA/QuAaPEEYhQd2k+nnevmiK+j+PEJbgZ21UHT2k1F6gSyrqxHImVGTzvjclc023TI9pJ8EQ==;\n\t5:VY1NqkcF23dKav0l1MB8tWD+tMQdMMuLD8gMs3bKNKX6w9iq7LwA+4HPkpE+Z4FRLrUS5qIRsdRcnZwKAoEvi0HGRRaPUaHJpKVGTUjnj7+lOPWX4/v7frGPUOXY//8Q2xirJ7/j+sJ+zk1O3Aco/A==;\n\t24:a3HXiArMy9RiN7aLr0mLxUh2Vk1/QTEmjzZFjtihohet+01mvcJ4a7UiH7Q3WeVk4OSCmgNZrqfRj8OX02fwNMuaVtxYqMQR9PbXfancjgE=;\n\t7:dBjwP7F+vrimYDMjhZL2Ul7A9s0eg3KxbpoKWzmVBn6t8Uijhasn1ojqM7pq2VCzwz12BRdgOXi/2tUtWiiXIv8Ap05yYcxxycqSp/J5EJIFWCRwJRA4V6viefveyAEwP1nKWKMwGWYx0PK0WtBaSm2IqashYgi1/eXxzoWPdPx1eX4wF7x2yAGMnt7FUqcHgu/8E0ef7uBPWB0acDykIZVOQ8f99XSmabXH7UmXLsw=","1; HE1PR0802MB2553;\n\t20:upWkaHme6o+ciMY9ivU565w3VycqvXygZV/GBVzxiUN7SLFsaMZ6woyn+IicBbQ7KID3+ZEoAbDeWyE9xz7hnU/v/9Suke12Bny9spyheEZysmBq8G4x4tLbfhz+Pj295Zw/Gh/2Ac0XpFNrga7XkRHzEf31DH5Ea6veqafrXUQ="],"X-MS-TrafficTypeDiagnostic":"HE1PR0802MB2553:","X-Exchange-Antispam-Report-Test":"UriScan:;","X-Microsoft-Antispam-PRVS":"","X-Exchange-Antispam-Report-CFA-Test":"BCL:0; PCL:0;\n\tRULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(2401047)(5005006)(8121501046)(3002001)(93006095)(93001095)(10201501046)(100000703101)(100105400095)(6041248)(20161123558100)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123562025)(20161123555025)(20161123564025)(20161123560025)(6072148)(201708071742011)(100000704101)(100105200095)(100000705101)(100105500095);\n\tSRVR:HE1PR0802MB2553; BCL:0; PCL:0;\n\tRULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095);\n\tSRVR:HE1PR0802MB2553; ","X-Forefront-PRVS":"04371797A5","X-Forefront-Antispam-Report":"SFV:NSPM;\n\tSFS:(10019020)(7370300001)(4630300001)(6009001)(6049001)(376002)(346002)(189002)(377424004)(24454002)(199003)(33646002)(305945005)(4326008)(7736002)(189998001)(107886003)(8676002)(31696002)(81156014)(81166006)(6246003)(86362001)(53936002)(97736004)(16526017)(16576012)(77096006)(2950100002)(6486002)(58126008)(23676002)(65826007)(316002)(229853002)(6666003)(66066001)(65806001)(65956001)(68736007)(3846002)(2906002)(5660300001)(31686004)(478600001)(25786009)(47776003)(6116002)(50986999)(50466002)(7350300001)(230700001)(64126003)(76176999)(54356999)(83506001)(36756003)(106356001)(105586002)(101416001)(53546010);\n\tDIR:OUT; SFP:1102; SCL:1; SRVR:HE1PR0802MB2553; H:[172.16.25.50];\n\tFPR:; SPF:None; PTR:InfoNoRecords; MX:1; A:1; LANG:en; ","Received-SPF":"None (protection.outlook.com: virtuozzo.com does not designate\n\tpermitted sender hosts)","SpamDiagnosticOutput":"1:99","SpamDiagnosticMetadata":"NSPM","X-OriginatorOrg":"virtuozzo.com","X-MS-Exchange-CrossTenant-OriginalArrivalTime":"21 Sep 2017 16:17:00.7184\n\t(UTC)","X-MS-Exchange-CrossTenant-FromEntityHeader":"Hosted","X-MS-Exchange-Transport-CrossTenantHeadersStamped":"HE1PR0802MB2553","X-detected-operating-system":"by eggs.gnu.org: Windows 7 or 8 [fuzzy]","X-Received-From":"40.107.5.105","Subject":"Re: [Qemu-devel] [PATCH 2/2] qcow2: truncate the tail of the image\n\tfile after shrinking the image","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"kwolf@redhat.com, den@openvz.org","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}},{"id":1772966,"web_url":"http://patchwork.ozlabs.org/comment/1772966/","msgid":"","list_archive_url":null,"date":"2017-09-21T16:20:23","subject":"Re: [Qemu-devel] [PATCH 2/2] qcow2: truncate the tail of the image\n\tfile after shrinking the image","submitter":{"id":36836,"url":"http://patchwork.ozlabs.org/api/people/36836/","name":"Max Reitz","email":"mreitz@redhat.com"},"content":"On 2017-09-21 18:16, Pavel Butsykin wrote:\n> On 21.09.2017 18:28, Max Reitz wrote:\n>> On 2017-09-20 15:58, Pavel Butsykin wrote:\n>>> Now after shrinking the image, at the end of the image file, there\n>>> might be a\n>>> tail that probably will never be used. So we can find the last used\n>>> cluster and\n>>> cut the tail.\n>>>\n>>> Signed-off-by: Pavel Butsykin \n>>> ---\n>>>   block/qcow2-refcount.c | 21 +++++++++++++++++++++\n>>>   block/qcow2.c          | 19 +++++++++++++++++++\n>>>   block/qcow2.h          |  1 +\n>>>   3 files changed, 41 insertions(+)\n>>>\n>>> diff --git a/block/qcow2-refcount.c b/block/qcow2-refcount.c\n>>> index 88d5a3f1ad..5e221a166c 100644\n>>> --- a/block/qcow2-refcount.c\n>>> +++ b/block/qcow2-refcount.c\n>>> @@ -3181,3 +3181,24 @@ out:\n>>>       g_free(reftable_tmp);\n>>>       return ret;\n>>>   }\n>>> +\n>>> +int64_t qcow2_get_last_cluster(BlockDriverState *bs, int64_t size)\n>>> +{\n>>> +    BDRVQcow2State *s = bs->opaque;\n>>> +    int64_t i, last_cluster, nb_clusters = size_to_clusters(s, size);\n>>> +    uint64_t refcount;\n>>> +\n>>> +    for (i = 0, last_cluster = 0; i < nb_clusters; i++) {\n>>> +        int ret = qcow2_get_refcount(bs, i, &refcount);\n>>> +        if (ret < 0) {\n>>> +            fprintf(stderr, \"Can't get refcount for cluster %\"\n>>> PRId64 \": %s\\n\",\n>>> +                    i, strerror(-ret));\n>>> +            continue;\n>>> +        }\n>>> +\n>>> +        if (refcount > 0) {\n>>> +            last_cluster = i;\n>>> +        }\n>>> +    }\n>>> +    return last_cluster;\n>>> +}\n>>\n>> Wouldn't it make more sense to start from the end of the image?\n> \n> If this will reduce the iterations, then yes. But it will depend on the\n> situation. If you truncate the image more than half, it can increase the\n> number of iterations. But intuitively it seems that to start from the\n> end would be better :)\n\nThat's one thing (also, I think usually the end should coincide with\nsome allocated cluster, but yes, that's just intuition :-)); but also,\nthis would simplify things a bit because we would no longer need the\nlast_cluster variable (the loop would just stop at the first allocated\ncluster).\n\nMax","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","ext-mx06.extmail.prod.ext.phx2.redhat.com;\n\tdmarc=none (p=none dis=none) header.from=redhat.com","ext-mx06.extmail.prod.ext.phx2.redhat.com;\n\tspf=fail smtp.mailfrom=mreitz@redhat.com"],"Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xyhgl6GTWz9t42\n\tfor ;\n\tFri, 22 Sep 2017 02:21:23 +1000 (AEST)","from localhost ([::1]:54559 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1dv4E5-0002uj-VJ\n\tfor incoming@patchwork.ozlabs.org; Thu, 21 Sep 2017 12:21:21 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:54491)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1dv4DR-0002pp-Os\n\tfor qemu-devel@nongnu.org; Thu, 21 Sep 2017 12:20:43 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1dv4DQ-0002DA-FT\n\tfor qemu-devel@nongnu.org; Thu, 21 Sep 2017 12:20:41 -0400","from mx1.redhat.com ([209.132.183.28]:50126)\n\tby eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)\n\t(Exim 4.71) (envelope-from )\n\tid 1dv4DK-0002BL-Jw; Thu, 21 Sep 2017 12:20:34 -0400","from smtp.corp.redhat.com\n\t(int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16])\n\t(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby mx1.redhat.com (Postfix) with ESMTPS id 71E3C1E31D;\n\tThu, 21 Sep 2017 16:20:33 +0000 (UTC)","from dresden.str.redhat.com (ovpn-204-199.brq.redhat.com\n\t[10.40.204.199])\n\tby smtp.corp.redhat.com (Postfix) with ESMTPS id 09CBE65E85;\n\tThu, 21 Sep 2017 16:20:24 +0000 (UTC)"],"DMARC-Filter":"OpenDMARC Filter v1.3.2 mx1.redhat.com 71E3C1E31D","To":"Pavel Butsykin , qemu-block@nongnu.org,\n\tqemu-devel@nongnu.org","References":"<20170920135833.20472-1-pbutsykin@virtuozzo.com>\n\t<20170920135833.20472-3-pbutsykin@virtuozzo.com>\n\t\n\t<175657ad-e6b4-0323-4f32-db9217be6bac@virtuozzo.com>","From":"Max Reitz ","Message-ID":"","Date":"Thu, 21 Sep 2017 18:20:23 +0200","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101\n\tThunderbird/52.3.0","MIME-Version":"1.0","In-Reply-To":"<175657ad-e6b4-0323-4f32-db9217be6bac@virtuozzo.com>","Content-Type":"multipart/signed; micalg=pgp-sha256;\n\tprotocol=\"application/pgp-signature\";\n\tboundary=\"GjHv9St3LhmJLht028Sl9gk0pL9L85DmB\"","X-Scanned-By":"MIMEDefang 2.79 on 10.5.11.16","X-Greylist":"Sender IP whitelisted, not delayed by milter-greylist-4.5.16\n\t(mx1.redhat.com [10.5.110.30]);\n\tThu, 21 Sep 2017 16:20:33 +0000 (UTC)","X-detected-operating-system":"by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]\n\t[fuzzy]","X-Received-From":"209.132.183.28","X-Content-Filtered-By":"Mailman/MimeDel 2.1.21","Subject":"Re: [Qemu-devel] [PATCH 2/2] qcow2: truncate the tail of the image\n\tfile after shrinking the image","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"kwolf@redhat.com, den@openvz.org","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}},{"id":1772998,"web_url":"http://patchwork.ozlabs.org/comment/1772998/","msgid":"","list_archive_url":null,"date":"2017-09-21T16:48:30","subject":"Re: [Qemu-devel] [Qemu-block] [PATCH 2/2] qcow2: truncate the tail\n\tof the image file after shrinking the image","submitter":{"id":64343,"url":"http://patchwork.ozlabs.org/api/people/64343/","name":"John Snow","email":"jsnow@redhat.com"},"content":"On 09/21/2017 05:49 AM, Pavel Butsykin wrote:\n> On 21.09.2017 00:38, John Snow wrote:\n>>\n>>\n>> On 09/20/2017 09:58 AM, Pavel Butsykin wrote:\n>>> Now after shrinking the image, at the end of the image file, there\n>>> might be a\n>>> tail that probably will never be used. So we can find the last used\n>>> cluster and\n>>> cut the tail.\n>>>\n>>> Signed-off-by: Pavel Butsykin \n>>> ---\n>>>   block/qcow2-refcount.c | 21 +++++++++++++++++++++\n>>>   block/qcow2.c          | 19 +++++++++++++++++++\n>>>   block/qcow2.h          |  1 +\n>>>   3 files changed, 41 insertions(+)\n>>>\n>>> diff --git a/block/qcow2-refcount.c b/block/qcow2-refcount.c\n>>> index 88d5a3f1ad..5e221a166c 100644\n>>> --- a/block/qcow2-refcount.c\n>>> +++ b/block/qcow2-refcount.c\n>>> @@ -3181,3 +3181,24 @@ out:\n>>>       g_free(reftable_tmp);\n>>>       return ret;\n>>>   }\n>>> +\n>>> +int64_t qcow2_get_last_cluster(BlockDriverState *bs, int64_t size)\n>>> +{\n>>> +    BDRVQcow2State *s = bs->opaque;\n>>> +    int64_t i, last_cluster, nb_clusters = size_to_clusters(s, size);\n>>> +    uint64_t refcount;\n>>> +\n>>> +    for (i = 0, last_cluster = 0; i < nb_clusters; i++) {\n>>> +        int ret = qcow2_get_refcount(bs, i, &refcount);\n>>> +        if (ret < 0) {\n>>> +            fprintf(stderr, \"Can't get refcount for cluster %\"\n>>> PRId64 \": %s\\n\",\n>>> +                    i, strerror(-ret));\n>>> +            continue;\n>>> +        }\n>>> +\n>>> +        if (refcount > 0) {\n>>> +            last_cluster = i;\n>>> +        }\n>>> +    }\n>>> +    return last_cluster;\n>>> +}> diff --git a/block/qcow2.c b/block/qcow2.c\n>>> index 8a4311d338..c3b6dd44c4 100644\n>>> --- a/block/qcow2.c\n>>> +++ b/block/qcow2.c\n>>> @@ -3106,6 +3106,7 @@ static int qcow2_truncate(BlockDriverState *bs,\n>>> int64_t offset,\n>>>       new_l1_size = size_to_l1(s, offset);\n>>>         if (offset < old_length) {\n>>> +        int64_t image_end_offset, old_file_size;\n>>>           if (prealloc != PREALLOC_MODE_OFF) {\n>>>               error_setg(errp,\n>>>                          \"Preallocation can't be used for shrinking\n>>> an image\");\n>>> @@ -3134,6 +3135,24 @@ static int qcow2_truncate(BlockDriverState\n>>> *bs, int64_t offset,\n>>>                                \"Failed to discard unused refblocks\");\n>>>               return ret;\n>>>           }\n>>> +\n>>> +        old_file_size = bdrv_getlength(bs->file->bs);\n>>> +        if (old_file_size < 0) {\n>>> +            error_setg_errno(errp, -old_file_size,\n>>> +                             \"Failed to inquire current file length\");\n>>> +            return old_file_size;\n>>> +        }\n>>> +        image_end_offset = (qcow2_get_last_cluster(bs,\n>>> old_file_size) + 1) *\n>>> +                           s->cluster_size;\n>>> +        if (image_end_offset < old_file_size) {\n>>> +            ret = bdrv_truncate(bs->file, image_end_offset,\n>>> +                                PREALLOC_MODE_OFF, NULL);\n>>> +            if (ret < 0) {\n>>> +                error_setg_errno(errp, -ret,\n>>> +                                 \"Failed to truncate the tail of the\n>>> image\");\n>>\n>> I've recently become skeptical of what partial resize successes look\n>> like, but that's an issue for another day entirely.\n>>\n>>> +                return ret;\n>>> +            }\n>>> +        }\n>>>       } else {\n>>>           ret = qcow2_grow_l1_table(bs, new_l1_size, true);\n>>>           if (ret < 0) {\n>>> diff --git a/block/qcow2.h b/block/qcow2.h\n>>> index 5a289a81e2..782a206ecb 100644\n>>> --- a/block/qcow2.h\n>>> +++ b/block/qcow2.h\n>>> @@ -597,6 +597,7 @@ int qcow2_change_refcount_order(BlockDriverState\n>>> *bs, int refcount_order,\n>>>                                   BlockDriverAmendStatusCB *status_cb,\n>>>                                   void *cb_opaque, Error **errp);\n>>>   int qcow2_shrink_reftable(BlockDriverState *bs);\n>>> +int64_t qcow2_get_last_cluster(BlockDriverState *bs, int64_t size);\n>>>     /* qcow2-cluster.c functions */\n>>>   int qcow2_grow_l1_table(BlockDriverState *bs, uint64_t min_size,\n>>>\n>>\n>> Reviewed-by: John Snow \n>>\n>> Looks sane to me, but under which circumstances might we grow such a\n>> tail? I assume the actual truncate call aligns to cluster boundaries as\n>> appropriate, so is this a bit of a \"quick fix\" to cull unused clusters\n>> that happened to be near the truncate boundary?\n>>\n>> It might be worth documenting the circumstances that produces this\n>> unused space that will never get used. My hunch is that such unused\n>> space should likely be getting reclaimed elsewhere and not here, but\n>> perhaps I'm misunderstanding the causal factors.\n>>\n> \n> This is a consequence of how we implemented shrinking the qcow2 image.\n> (https://lists.gnu.org/archive/html/qemu-devel/2017-09/msg04580.html)\n> But on the other hand, if we need to shrink the qcow2 image without\n> copying the data, this is the only way. The same guest offset can be\n> converted to almost any host offset in the file i.e. the first guest\n> cluster may be located somewhere at the end or the middle of the image\n> file. So we can't just take and truncate the image file on the border of\n> the truncation, therefore to shrink the image we just discard the\n> clusters that corresponds to the truncated area. The result is a\n> sparse image file where the apparent file size differs from actual size.\n> And the tail in this case is the difference between the actual size and\n> last used cluster in the image, so in fact the cutting of the tail does\n> not change the apparent file size.\n> \n\nOh, duh, I get it. The truncation itself creates a lot of sparseness.\n\nThanks for the explanation.","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","ext-mx03.extmail.prod.ext.phx2.redhat.com;\n\tdmarc=none (p=none dis=none) header.from=redhat.com","ext-mx03.extmail.prod.ext.phx2.redhat.com;\n\tspf=fail smtp.mailfrom=jsnow@redhat.com"],"Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xyjhN4b6hz9t4c\n\tfor ;\n\tFri, 22 Sep 2017 03:07:00 +1000 (AEST)","from localhost ([::1]:54748 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1dv4wE-0007eB-0N\n\tfor incoming@patchwork.ozlabs.org; Thu, 21 Sep 2017 13:06:58 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:34783)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1dv4eW-0001G0-FR\n\tfor qemu-devel@nongnu.org; Thu, 21 Sep 2017 12:48:42 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1dv4eU-000174-Nk\n\tfor qemu-devel@nongnu.org; Thu, 21 Sep 2017 12:48:40 -0400","from mx1.redhat.com ([209.132.183.28]:34348)\n\tby eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)\n\t(Exim 4.71) (envelope-from )\n\tid 1dv4eP-00012U-9k; Thu, 21 Sep 2017 12:48:33 -0400","from smtp.corp.redhat.com\n\t(int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14])\n\t(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby mx1.redhat.com (Postfix) with ESMTPS id 4DA717E428;\n\tThu, 21 Sep 2017 16:48:32 +0000 (UTC)","from [10.10.121.182] (ovpn-121-182.rdu2.redhat.com [10.10.121.182])\n\tby smtp.corp.redhat.com (Postfix) with ESMTP id C40675D9CC;\n\tThu, 21 Sep 2017 16:48:30 +0000 (UTC)"],"DMARC-Filter":"OpenDMARC Filter v1.3.2 mx1.redhat.com 4DA717E428","To":"Pavel Butsykin , qemu-block@nongnu.org,\n\tqemu-devel@nongnu.org","References":"<20170920135833.20472-1-pbutsykin@virtuozzo.com>\n\t<20170920135833.20472-3-pbutsykin@virtuozzo.com>\n\t\n\t<79eb6628-130e-af9d-cd39-56bf71194ae3@virtuozzo.com>","From":"John Snow ","Message-ID":"","Date":"Thu, 21 Sep 2017 12:48:30 -0400","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101\n\tThunderbird/52.3.0","MIME-Version":"1.0","In-Reply-To":"<79eb6628-130e-af9d-cd39-56bf71194ae3@virtuozzo.com>","Content-Type":"text/plain; charset=utf-8","Content-Language":"en-US","X-Scanned-By":"MIMEDefang 2.79 on 10.5.11.14","X-Greylist":"Sender IP whitelisted, not delayed by milter-greylist-4.5.16\n\t(mx1.redhat.com [10.5.110.27]);\n\tThu, 21 Sep 2017 16:48:32 +0000 (UTC)","Content-Transfer-Encoding":"quoted-printable","X-detected-operating-system":"by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]\n\t[fuzzy]","X-Received-From":"209.132.183.28","Subject":"Re: [Qemu-devel] [Qemu-block] [PATCH 2/2] qcow2: truncate the tail\n\tof the image file after shrinking the image","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"kwolf@redhat.com, den@openvz.org, mreitz@redhat.com","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}},{"id":1773046,"web_url":"http://patchwork.ozlabs.org/comment/1773046/","msgid":"<4393a82e-de82-ed8d-15ae-8defe59413ac@redhat.com>","list_archive_url":null,"date":"2017-09-21T18:23:32","subject":"Re: [Qemu-devel] [Qemu-block] [PATCH 2/2] qcow2: truncate the tail\n\tof the image file after shrinking the image","submitter":{"id":6591,"url":"http://patchwork.ozlabs.org/api/people/6591/","name":"Eric Blake","email":"eblake@redhat.com"},"content":"On 09/21/2017 11:48 AM, John Snow wrote:\n>>> Looks sane to me, but under which circumstances might we grow such a\n>>> tail? I assume the actual truncate call aligns to cluster boundaries as\n>>> appropriate, so is this a bit of a \"quick fix\" to cull unused clusters\n>>> that happened to be near the truncate boundary?\n>>>\n>>> It might be worth documenting the circumstances that produces this\n>>> unused space that will never get used. My hunch is that such unused\n>>> space should likely be getting reclaimed elsewhere and not here, but\n>>> perhaps I'm misunderstanding the causal factors.\n>>>\n>>\n>> This is a consequence of how we implemented shrinking the qcow2 image.\n>> (https://lists.gnu.org/archive/html/qemu-devel/2017-09/msg04580.html)\n>> But on the other hand, if we need to shrink the qcow2 image without\n>> copying the data, this is the only way. The same guest offset can be\n>> converted to almost any host offset in the file i.e. the first guest\n>> cluster may be located somewhere at the end or the middle of the image\n>> file. So we can't just take and truncate the image file on the border of\n>> the truncation, therefore to shrink the image we just discard the\n>> clusters that corresponds to the truncated area. The result is a\n>> sparse image file where the apparent file size differs from actual size.\n>> And the tail in this case is the difference between the actual size and\n>> last used cluster in the image, so in fact the cutting of the tail does\n>> not change the apparent file size.\n>>\n> \n> Oh, duh, I get it. The truncation itself creates a lot of sparseness.\n\nIt is also interesting to think about whether we should someday\nimplement a qcow2 defragmenting operation (either a simple one: pull all\nlater clusters into earlier holes, but the end result is not necessarily\ncontiguous; or a complex one: shuffle things so that all clusters are in\norder, even though it may require moving some clusters around twice), so\nthat you can remove all holes from the file (perhaps useful if the file\nis stored on a system that does not support holes). But not for this\nseries :)","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","ext-mx06.extmail.prod.ext.phx2.redhat.com;\n\tdmarc=none (p=none dis=none) header.from=redhat.com","ext-mx06.extmail.prod.ext.phx2.redhat.com;\n\tspf=fail smtp.mailfrom=eblake@redhat.com"],"Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xylPW67CSz9sP1\n\tfor ;\n\tFri, 22 Sep 2017 04:24:14 +1000 (AEST)","from localhost ([::1]:55023 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1dv68x-0002nr-UZ\n\tfor incoming@patchwork.ozlabs.org; Thu, 21 Sep 2017 14:24:11 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:60802)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1dv68Z-0002mh-I3\n\tfor qemu-devel@nongnu.org; Thu, 21 Sep 2017 14:23:48 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1dv68Y-0003c1-AN\n\tfor qemu-devel@nongnu.org; Thu, 21 Sep 2017 14:23:47 -0400","from mx1.redhat.com ([209.132.183.28]:52512)\n\tby eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)\n\t(Exim 4.71) (envelope-from )\n\tid 1dv68T-0003ZD-Rg; Thu, 21 Sep 2017 14:23:42 -0400","from smtp.corp.redhat.com\n\t(int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15])\n\t(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby mx1.redhat.com (Postfix) with ESMTPS id 2FDEB267D4;\n\tThu, 21 Sep 2017 18:23:40 +0000 (UTC)","from [10.10.124.97] (ovpn-124-97.rdu2.redhat.com [10.10.124.97])\n\tby smtp.corp.redhat.com (Postfix) with ESMTP id 3830E5EDE1;\n\tThu, 21 Sep 2017 18:23:35 +0000 (UTC)"],"DMARC-Filter":"OpenDMARC Filter v1.3.2 mx1.redhat.com 2FDEB267D4","To":"John Snow , Pavel Butsykin , \n\tqemu-block@nongnu.org, qemu-devel@nongnu.org","References":"<20170920135833.20472-1-pbutsykin@virtuozzo.com>\n\t<20170920135833.20472-3-pbutsykin@virtuozzo.com>\n\t\n\t<79eb6628-130e-af9d-cd39-56bf71194ae3@virtuozzo.com>\n\t","From":"Eric Blake ","Openpgp":"url=http://people.redhat.com/eblake/eblake.gpg","Organization":"Red Hat, Inc.","Message-ID":"<4393a82e-de82-ed8d-15ae-8defe59413ac@redhat.com>","Date":"Thu, 21 Sep 2017 13:23:32 -0500","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101\n\tThunderbird/52.3.0","MIME-Version":"1.0","In-Reply-To":"","Content-Type":"multipart/signed; micalg=pgp-sha256;\n\tprotocol=\"application/pgp-signature\";\n\tboundary=\"HMupOISu5dlbE4vKPVNvg4dbMEIfXquAP\"","X-Scanned-By":"MIMEDefang 2.79 on 10.5.11.15","X-Greylist":"Sender IP whitelisted, not delayed by milter-greylist-4.5.16\n\t(mx1.redhat.com [10.5.110.30]);\n\tThu, 21 Sep 2017 18:23:40 +0000 (UTC)","X-detected-operating-system":"by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]\n\t[fuzzy]","X-Received-From":"209.132.183.28","X-Content-Filtered-By":"Mailman/MimeDel 2.1.21","Subject":"Re: [Qemu-devel] [Qemu-block] [PATCH 2/2] qcow2: truncate the tail\n\tof the image file after shrinking the image","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"kwolf@redhat.com, den@openvz.org, mreitz@redhat.com","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}}]