{"id":813516,"url":"http://patchwork.ozlabs.org/api/patches/813516/?format=json","web_url":"http://patchwork.ozlabs.org/project/qemu-devel/patch/20170913160333.23622-15-eblake@redhat.com/","project":{"id":14,"url":"http://patchwork.ozlabs.org/api/projects/14/?format=json","name":"QEMU Development","link_name":"qemu-devel","list_id":"qemu-devel.nongnu.org","list_email":"qemu-devel@nongnu.org","web_url":"","scm_url":"","webscm_url":"","list_archive_url":"","list_archive_url_format":"","commit_url_format":""},"msgid":"<20170913160333.23622-15-eblake@redhat.com>","list_archive_url":null,"date":"2017-09-13T16:03:24","name":"[v4,14/23] qemu-img: Speed up compare on pre-allocated larger file","commit_ref":null,"pull_url":null,"state":"new","archived":false,"hash":"d2bae517fe1304be4f6e8dffb1cc761787697271","submitter":{"id":6591,"url":"http://patchwork.ozlabs.org/api/people/6591/?format=json","name":"Eric Blake","email":"eblake@redhat.com"},"delegate":null,"mbox":"http://patchwork.ozlabs.org/project/qemu-devel/patch/20170913160333.23622-15-eblake@redhat.com/mbox/","series":[{"id":2944,"url":"http://patchwork.ozlabs.org/api/series/2944/?format=json","web_url":"http://patchwork.ozlabs.org/project/qemu-devel/list/?series=2944","date":"2017-09-13T16:03:10","name":"make bdrv_get_block_status byte-based","version":4,"mbox":"http://patchwork.ozlabs.org/series/2944/mbox/"}],"comments":"http://patchwork.ozlabs.org/api/patches/813516/comments/","check":"pending","checks":"http://patchwork.ozlabs.org/api/patches/813516/checks/","tags":{},"related":[],"headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=208.118.235.17; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","ext-mx07.extmail.prod.ext.phx2.redhat.com;\n\tdmarc=none (p=none dis=none) header.from=redhat.com","ext-mx07.extmail.prod.ext.phx2.redhat.com;\n\tspf=fail smtp.mailfrom=eblake@redhat.com"],"Received":["from lists.gnu.org (lists.gnu.org [208.118.235.17])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xsmty6w9Qz9sPk\n\tfor ;\n\tThu, 14 Sep 2017 02:14:02 +1000 (AEST)","from localhost ([::1]:43400 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1dsAIa-00048T-Qy\n\tfor incoming@patchwork.ozlabs.org; Wed, 13 Sep 2017 12:14:00 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:45961)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1dsA9b-0001tt-9i\n\tfor qemu-devel@nongnu.org; Wed, 13 Sep 2017 12:04:44 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1dsA9a-0006KD-Ap\n\tfor qemu-devel@nongnu.org; Wed, 13 Sep 2017 12:04:43 -0400","from mx1.redhat.com ([209.132.183.28]:37280)\n\tby eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)\n\t(Exim 4.71) (envelope-from )\n\tid 1dsA9V-0006GV-Nd; Wed, 13 Sep 2017 12:04:37 -0400","from smtp.corp.redhat.com\n\t(int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11])\n\t(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby mx1.redhat.com (Postfix) with ESMTPS id BC7D0C04B92E;\n\tWed, 13 Sep 2017 16:04:36 +0000 (UTC)","from red.redhat.com (ovpn-120-201.rdu2.redhat.com [10.10.120.201])\n\tby smtp.corp.redhat.com (Postfix) with ESMTP id F1BBD19E72;\n\tWed, 13 Sep 2017 16:04:31 +0000 (UTC)"],"DMARC-Filter":"OpenDMARC Filter v1.3.2 mx1.redhat.com BC7D0C04B92E","From":"Eric Blake ","To":"qemu-devel@nongnu.org","Date":"Wed, 13 Sep 2017 11:03:24 -0500","Message-Id":"<20170913160333.23622-15-eblake@redhat.com>","In-Reply-To":"<20170913160333.23622-1-eblake@redhat.com>","References":"<20170913160333.23622-1-eblake@redhat.com>","X-Scanned-By":"MIMEDefang 2.79 on 10.5.11.11","X-Greylist":"Sender IP whitelisted, not delayed by milter-greylist-4.5.16\n\t(mx1.redhat.com [10.5.110.31]);\n\tWed, 13 Sep 2017 16:04:36 +0000 (UTC)","X-detected-operating-system":"by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]\n\t[fuzzy]","X-Received-From":"209.132.183.28","Subject":"[Qemu-devel] [PATCH v4 14/23] qemu-img: Speed up compare on\n\tpre-allocated larger file","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"kwolf@redhat.com, jsnow@redhat.com, famz@redhat.com,\n\tqemu-block@nongnu.org, Max Reitz ","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"},"content":"Compare the following images with all-zero contents:\n$ truncate --size 1M A\n$ qemu-img create -f qcow2 -o preallocation=off B 1G\n$ qemu-img create -f qcow2 -o preallocation=metadata C 1G\n\nOn my machine, the difference is noticeable for pre-patch speeds,\nwith more than an order of magnitude in difference caused by the\nchoice of preallocation in the qcow2 file:\n\n$ time ./qemu-img compare -f raw -F qcow2 A B\nWarning: Image size mismatch!\nImages are identical.\n\nreal\t0m0.014s\nuser\t0m0.007s\nsys\t0m0.007s\n\n$ time ./qemu-img compare -f raw -F qcow2 A C\nWarning: Image size mismatch!\nImages are identical.\n\nreal\t0m0.341s\nuser\t0m0.144s\nsys\t0m0.188s\n\nWhy? Because bdrv_is_allocated() returns false for image B but\ntrue for image C, throwing away the fact that both images know\nvia lseek(SEEK_HOLE) that the entire image still reads as zero.\nFrom there, qemu-img ends up calling bdrv_pread() for every byte\nof the tail, instead of quickly looking for the next allocation.\nThe solution: use block_status instead of is_allocated, giving:\n\n$ time ./qemu-img compare -f raw -F qcow2 A C\nWarning: Image size mismatch!\nImages are identical.\n\nreal\t0m0.014s\nuser\t0m0.011s\nsys\t0m0.003s\n\nwhich is on par with the speeds for no pre-allocation.\n\nSigned-off-by: Eric Blake \n\n---\nv3: new patch\n---\n qemu-img.c | 8 ++++----\n 1 file changed, 4 insertions(+), 4 deletions(-)","diff":"diff --git a/qemu-img.c b/qemu-img.c\nindex f8423e9b3f..f5ab29d176 100644\n--- a/qemu-img.c\n+++ b/qemu-img.c\n@@ -1477,11 +1477,11 @@ static int img_compare(int argc, char **argv)\n while (sector_num < progress_base) {\n int64_t count;\n\n- ret = bdrv_is_allocated_above(blk_bs(blk_over), NULL,\n+ ret = bdrv_block_status_above(blk_bs(blk_over), NULL,\n sector_num * BDRV_SECTOR_SIZE,\n (progress_base - sector_num) *\n BDRV_SECTOR_SIZE,\n- &count);\n+ &count, NULL);\n if (ret < 0) {\n ret = 3;\n error_report(\"Sector allocation test failed for %s\",\n@@ -1489,11 +1489,11 @@ static int img_compare(int argc, char **argv)\n goto out;\n\n }\n- /* TODO relax this once bdrv_is_allocated_above does not enforce\n+ /* TODO relax this once bdrv_block_status_above does not enforce\n * sector alignment */\n assert(QEMU_IS_ALIGNED(count, BDRV_SECTOR_SIZE));\n nb_sectors = count >> BDRV_SECTOR_BITS;\n- if (ret) {\n+ if (ret & BDRV_BLOCK_ALLOCATED && !(ret & BDRV_BLOCK_ZERO)) {\n nb_sectors = MIN(nb_sectors, IO_BUF_SIZE >> BDRV_SECTOR_BITS);\n ret = check_empty_sectors(blk_over, sector_num, nb_sectors,\n filename_over, buf1, quiet);\n","prefixes":["v4","14/23"]}