[{"id":1775815,"web_url":"http://patchwork.ozlabs.org/comment/1775815/","msgid":"<00c44c8d-479b-b954-e605-56a3ed00651f@redhat.com>","list_archive_url":null,"date":"2017-09-26T19:39:08","subject":"Re: [Qemu-devel] [PATCH v4 07/23] block: Convert\n\tbdrv_get_block_status() to bytes","submitter":{"id":64343,"url":"http://patchwork.ozlabs.org/api/people/64343/","name":"John Snow","email":"jsnow@redhat.com"},"content":"On 09/13/2017 12:03 PM, Eric Blake wrote:\n> We are gradually moving away from sector-based interfaces, towards\n> byte-based. In the common case, allocation is unlikely to ever use\n> values that are not naturally sector-aligned, but it is possible\n> that byte-based values will let us be more precise about allocation\n> at the end of an unaligned file that can do byte-based access.\n> \n> Changing the name of the function from bdrv_get_block_status() to\n> bdrv_block_status() ensures that the compiler enforces that all\n> callers are updated. For now, the io.c layer still assert()s that\n> all callers are sector-aligned, but that can be relaxed when a later\n> patch implements byte-based block status in the drivers.\n> \n> Note that we have an inherent limitation in the BDRV_BLOCK_* return\n> values: BDRV_BLOCK_OFFSET_VALID can only return the start of a\n> sector, even if we later relax the interface to query for the status\n> starting at an intermediate byte; document the obvious interpretation\n> that valid offsets are always sector-relative.\n> \n> Therefore, for the most part this patch is just the addition of scaling\n> at the callers followed by inverse scaling at bdrv_block_status(). But\n> some code, particularly bdrv_is_allocated(), gets a lot simpler because\n> it no longer has to mess with sectors.\n> \n> For ease of review, bdrv_get_block_status_above() will be tackled\n> separately.\n> \n> Signed-off-by: Eric Blake \n> \n> ---\n> v3: clamp bytes to 32-bits, rather than asserting\n> v2: rebase to earlier changes\n> ---\n> include/block/block.h | 12 +++++++-----\n> block/io.c | 31 +++++++++++++++++++------------\n> block/qcow2-cluster.c | 2 +-\n> qemu-img.c | 20 +++++++++++---------\n> 4 files changed, 38 insertions(+), 27 deletions(-)\n> \n> diff --git a/include/block/block.h b/include/block/block.h\n> index bb3b95d491..7a9a8db588 100644\n> --- a/include/block/block.h\n> +++ b/include/block/block.h\n> @@ -138,8 +138,10 @@ typedef struct HDGeometry {\n> *\n> * If BDRV_BLOCK_OFFSET_VALID is set, bits 9-62 (BDRV_BLOCK_OFFSET_MASK)\n> * represent the offset in the returned BDS that is allocated for the\n> - * corresponding raw data; however, whether that offset actually contains\n> - * data also depends on BDRV_BLOCK_DATA and BDRV_BLOCK_ZERO, as follows:\n> + * corresponding raw data. Individual bytes are at the same sector-relative\n> + * locations (and thus, this bit cannot be set for mappings which are\n> + * not equivalent modulo 512). However, whether that offset actually\n> + * contains data also depends on BDRV_BLOCK_DATA, as follows:\n> *\n> * DATA ZERO OFFSET_VALID\n> * t t t sectors read as zero, returned file is zero at offset\n> @@ -421,9 +423,9 @@ int bdrv_has_zero_init_1(BlockDriverState *bs);\n> int bdrv_has_zero_init(BlockDriverState *bs);\n> bool bdrv_unallocated_blocks_are_zero(BlockDriverState *bs);\n> bool bdrv_can_write_zeroes_with_unmap(BlockDriverState *bs);\n> -int64_t bdrv_get_block_status(BlockDriverState *bs, int64_t sector_num,\n> - int nb_sectors, int *pnum,\n> - BlockDriverState **file);\n> +int64_t bdrv_block_status(BlockDriverState *bs, int64_t offset,\n> + int64_t bytes, int64_t *pnum,\n> + BlockDriverState **file);\n> int64_t bdrv_get_block_status_above(BlockDriverState *bs,\n> BlockDriverState *base,\n> int64_t sector_num,\n> diff --git a/block/io.c b/block/io.c\n> index 638b3890b7..1ed46bcece 100644\n> --- a/block/io.c\n> +++ b/block/io.c\n> @@ -695,7 +695,6 @@ int bdrv_make_zero(BdrvChild *child, BdrvRequestFlags flags)\n> {\n> int64_t target_size, ret, bytes, offset = 0;\n> BlockDriverState *bs = child->bs;\n> - int n; /* sectors */\n> \n> target_size = bdrv_getlength(bs);\n> if (target_size < 0) {\n> @@ -707,24 +706,23 @@ int bdrv_make_zero(BdrvChild *child, BdrvRequestFlags flags)\n> if (bytes <= 0) {\n> return 0;\n> }\n> - ret = bdrv_get_block_status(bs, offset >> BDRV_SECTOR_BITS,\n> - bytes >> BDRV_SECTOR_BITS, &n, NULL);\n> + ret = bdrv_block_status(bs, offset, bytes, &bytes, NULL);\n> if (ret < 0) {\n> error_report(\"error getting block status at offset %\" PRId64 \": %s\",\n> offset, strerror(-ret));\n> return ret;\n> }\n> if (ret & BDRV_BLOCK_ZERO) {\n> - offset += n * BDRV_SECTOR_BITS;\n> + offset += bytes;\n> continue;\n> }\n> - ret = bdrv_pwrite_zeroes(child, offset, n * BDRV_SECTOR_SIZE, flags);\n> + ret = bdrv_pwrite_zeroes(child, offset, bytes, flags);\n> if (ret < 0) {\n> error_report(\"error writing zeroes at offset %\" PRId64 \": %s\",\n> offset, strerror(-ret));\n> return ret;\n> }\n> - offset += n * BDRV_SECTOR_SIZE;\n> + offset += bytes;\n> }\n> }\n> \n> @@ -1983,13 +1981,22 @@ int64_t bdrv_get_block_status_above(BlockDriverState *bs,\n> nb_sectors, pnum, file);\n> }\n> \n> -int64_t bdrv_get_block_status(BlockDriverState *bs,\n> - int64_t sector_num,\n> - int nb_sectors, int *pnum,\n> - BlockDriverState **file)\n> +int64_t bdrv_block_status(BlockDriverState *bs,\n> + int64_t offset, int64_t bytes, int64_t *pnum,\n> + BlockDriverState **file)\n> {\n> - return bdrv_get_block_status_above(bs, backing_bs(bs),\n> - sector_num, nb_sectors, pnum, file);\n> + int64_t ret;\n> + int n;\n> +\n> + assert(QEMU_IS_ALIGNED(offset | bytes, BDRV_SECTOR_SIZE));\n> + bytes = MIN(bytes, BDRV_REQUEST_MAX_BYTES);\n> + ret = bdrv_get_block_status_above(bs, backing_bs(bs),\n> + offset >> BDRV_SECTOR_BITS,\n> + bytes >> BDRV_SECTOR_BITS, &n, file);\n> + if (pnum) {\n> + *pnum = n * BDRV_SECTOR_SIZE;\n> + }\n\nIs it safe to truncate the request in the event that the caller did not\nprovide a pnum target? that is, how will they know for what range we are\nanswering?\n\n> + return ret;\n> }\n> \n> int coroutine_fn bdrv_is_allocated(BlockDriverState *bs, int64_t offset,\n> diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c\n> index 0d4824993c..d837b3980d 100644\n> --- a/block/qcow2-cluster.c\n> +++ b/block/qcow2-cluster.c\n> @@ -1584,7 +1584,7 @@ static int discard_single_l2(BlockDriverState *bs, uint64_t offset,\n> * cluster is already marked as zero, or if it's unallocated and we\n> * don't have a backing file.\n> *\n> - * TODO We might want to use bdrv_get_block_status(bs) here, but we're\n> + * TODO We might want to use bdrv_block_status(bs) here, but we're\n\nthanks for updating comments too :)\n\n> * holding s->lock, so that doesn't work today.\n> *\n> * If full_discard is true, the sector should not read back as zeroes,\n> diff --git a/qemu-img.c b/qemu-img.c\n> index 54f7682069..897f80abb3 100644\n> --- a/qemu-img.c\n> +++ b/qemu-img.c\n> @@ -1598,9 +1598,14 @@ static int convert_iteration_sectors(ImgConvertState *s, int64_t sector_num)\n> \n> if (s->sector_next_status <= sector_num) {\n> if (s->target_has_backing) {\n> - ret = bdrv_get_block_status(blk_bs(s->src[src_cur]),\n> - sector_num - src_cur_offset,\n> - n, &n, NULL);\n> + int64_t count = n * BDRV_SECTOR_SIZE;\n> +\n> + ret = bdrv_block_status(blk_bs(s->src[src_cur]),\n> + (sector_num - src_cur_offset) *\n> + BDRV_SECTOR_SIZE,\n> + count, &count, NULL);\n> + assert(ret < 0 || QEMU_IS_ALIGNED(count, BDRV_SECTOR_SIZE));\n> + n = count >> BDRV_SECTOR_BITS;\n> } else {\n> ret = bdrv_get_block_status_above(blk_bs(s->src[src_cur]), NULL,\n> sector_num - src_cur_offset,\n> @@ -2677,9 +2682,7 @@ static int get_block_status(BlockDriverState *bs, int64_t offset,\n> int depth;\n> BlockDriverState *file;\n> bool has_offset;\n> - int nb_sectors = bytes >> BDRV_SECTOR_BITS;\n> \n> - assert(bytes < INT_MAX);\n> /* As an optimization, we could cache the current range of unallocated\n> * clusters in each file of the chain, and avoid querying the same\n> * range repeatedly.\n> @@ -2687,12 +2690,11 @@ static int get_block_status(BlockDriverState *bs, int64_t offset,\n> \n> depth = 0;\n> for (;;) {\n> - ret = bdrv_get_block_status(bs, offset >> BDRV_SECTOR_BITS, nb_sectors,\n> - &nb_sectors, &file);\n> + ret = bdrv_block_status(bs, offset, bytes, &bytes, &file);\n> if (ret < 0) {\n> return ret;\n> }\n> - assert(nb_sectors);\n> + assert(bytes);\n> if (ret & (BDRV_BLOCK_ZERO|BDRV_BLOCK_DATA)) {\n> break;\n> }\n> @@ -2709,7 +2711,7 @@ static int get_block_status(BlockDriverState *bs, int64_t offset,\n> \n> *e = (MapEntry) {\n> .start = offset,\n> - .length = nb_sectors * BDRV_SECTOR_SIZE,\n> + .length = bytes,\n> .data = !!(ret & BDRV_BLOCK_DATA),\n> .zero = !!(ret & BDRV_BLOCK_ZERO),\n> .offset = ret & BDRV_BLOCK_OFFSET_MASK,\n> \n\nRest appears obviously correct.","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","ext-mx06.extmail.prod.ext.phx2.redhat.com;\n\tdmarc=none (p=none dis=none) header.from=redhat.com","ext-mx06.extmail.prod.ext.phx2.redhat.com;\n\tspf=fail smtp.mailfrom=jsnow@redhat.com"],"Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3y1rrc5l6bz9sPr\n\tfor ;\n\tWed, 27 Sep 2017 05:39:50 +1000 (AEST)","from localhost ([::1]:50903 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1dwvhs-0005JS-U3\n\tfor incoming@patchwork.ozlabs.org; Tue, 26 Sep 2017 15:39:48 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:36306)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1dwvhS-0005FF-IJ\n\tfor qemu-devel@nongnu.org; Tue, 26 Sep 2017 15:39:24 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1dwvhR-0000HQ-2C\n\tfor qemu-devel@nongnu.org; Tue, 26 Sep 2017 15:39:22 -0400","from mx1.redhat.com ([209.132.183.28]:33042)\n\tby eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)\n\t(Exim 4.71) (envelope-from )\n\tid 1dwvhL-0000Dm-MW; Tue, 26 Sep 2017 15:39:15 -0400","from smtp.corp.redhat.com\n\t(int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14])\n\t(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby mx1.redhat.com (Postfix) with ESMTPS id ACC5FC6552;\n\tTue, 26 Sep 2017 19:39:14 +0000 (UTC)","from [10.18.17.130] (dhcp-17-130.bos.redhat.com [10.18.17.130])\n\tby smtp.corp.redhat.com (Postfix) with ESMTP id EA53C68705;\n\tTue, 26 Sep 2017 19:39:08 +0000 (UTC)"],"DMARC-Filter":"OpenDMARC Filter v1.3.2 mx1.redhat.com ACC5FC6552","To":"Eric Blake , qemu-devel@nongnu.org","References":"<20170913160333.23622-1-eblake@redhat.com>\n\t<20170913160333.23622-8-eblake@redhat.com>","From":"John Snow ","Message-ID":"<00c44c8d-479b-b954-e605-56a3ed00651f@redhat.com>","Date":"Tue, 26 Sep 2017 15:39:08 -0400","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101\n\tThunderbird/52.3.0","MIME-Version":"1.0","In-Reply-To":"<20170913160333.23622-8-eblake@redhat.com>","Content-Type":"text/plain; charset=utf-8","Content-Language":"en-US","Content-Transfer-Encoding":"7bit","X-Scanned-By":"MIMEDefang 2.79 on 10.5.11.14","X-Greylist":"Sender IP whitelisted, not delayed by milter-greylist-4.5.16\n\t(mx1.redhat.com [10.5.110.30]);\n\tTue, 26 Sep 2017 19:39:14 +0000 (UTC)","X-detected-operating-system":"by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]\n\t[fuzzy]","X-Received-From":"209.132.183.28","Subject":"Re: [Qemu-devel] [PATCH v4 07/23] block: Convert\n\tbdrv_get_block_status() to bytes","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"kwolf@redhat.com, famz@redhat.com, Stefan Hajnoczi ,\n\tqemu-block@nongnu.org, Max Reitz ","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}},{"id":1775821,"web_url":"http://patchwork.ozlabs.org/comment/1775821/","msgid":"<9dd34363-1f30-8c38-979b-32cc53b5d252@redhat.com>","list_archive_url":null,"date":"2017-09-26T19:57:21","subject":"Re: [Qemu-devel] [PATCH v4 07/23] block: Convert\n\tbdrv_get_block_status() to bytes","submitter":{"id":6591,"url":"http://patchwork.ozlabs.org/api/people/6591/","name":"Eric Blake","email":"eblake@redhat.com"},"content":"On 09/26/2017 02:39 PM, John Snow wrote:\n>> -int64_t bdrv_get_block_status(BlockDriverState *bs,\n>> - int64_t sector_num,\n>> - int nb_sectors, int *pnum,\n>> - BlockDriverState **file)\n>> +int64_t bdrv_block_status(BlockDriverState *bs,\n>> + int64_t offset, int64_t bytes, int64_t *pnum,\n>> + BlockDriverState **file)\n>> {\n>> - return bdrv_get_block_status_above(bs, backing_bs(bs),\n>> - sector_num, nb_sectors, pnum, file);\n>> + int64_t ret;\n>> + int n;\n>> +\n>> + assert(QEMU_IS_ALIGNED(offset | bytes, BDRV_SECTOR_SIZE));\n>> + bytes = MIN(bytes, BDRV_REQUEST_MAX_BYTES);\n>> + ret = bdrv_get_block_status_above(bs, backing_bs(bs),\n>> + offset >> BDRV_SECTOR_BITS,\n>> + bytes >> BDRV_SECTOR_BITS, &n, file);\n>> + if (pnum) {\n>> + *pnum = n * BDRV_SECTOR_SIZE;\n>> + }\n> \n> Is it safe to truncate the request in the event that the caller did not\n> provide a pnum target? that is, how will they know for what range we are\n> answering?\n\nHmm. I think I have some rebase cruft here. At one point, I was playing\nwith the idea of allowing pnum == NULL for ALL get_status() callers,\nsimilar to the existing block/vvfat.c:cluster_was_modified():\n\nblock/vvfat.c: res = bdrv_is_allocated(s->qcow->bs,\nblock/vvfat.c- (offset + i) *\nBDRV_SECTOR_SIZE,\nblock/vvfat.c-\nBDRV_SECTOR_SIZE, NULL);\n\nbut looking further, only bdrv_is_allocated() (and NOT\nbdrv_[get_]block_status) is ever used in that manner. Or, in terms of\nthe 'mapping' variable, a NULL pnum only makes sense when mapping ==\nfalse. So the conditional on 'if (pnum)' should be dropped here.","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","ext-mx07.extmail.prod.ext.phx2.redhat.com;\n\tdmarc=none (p=none dis=none) header.from=redhat.com","ext-mx07.extmail.prod.ext.phx2.redhat.com;\n\tspf=fail smtp.mailfrom=eblake@redhat.com"],"Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3y1sFb624Tz9t3F\n\tfor ;\n\tWed, 27 Sep 2017 05:58:10 +1000 (AEST)","from localhost ([::1]:50938 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1dwvzb-0000Fy-0P\n\tfor incoming@patchwork.ozlabs.org; Tue, 26 Sep 2017 15:58:07 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:38880)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1dwvz9-0000Eh-Qp\n\tfor qemu-devel@nongnu.org; Tue, 26 Sep 2017 15:57:40 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1dwvz8-00028G-Vv\n\tfor qemu-devel@nongnu.org; Tue, 26 Sep 2017 15:57:39 -0400","from mx1.redhat.com ([209.132.183.28]:37536)\n\tby eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)\n\t(Exim 4.71) (envelope-from )\n\tid 1dwvz2-00025T-OZ; Tue, 26 Sep 2017 15:57:32 -0400","from smtp.corp.redhat.com\n\t(int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12])\n\t(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby mx1.redhat.com (Postfix) with ESMTPS id 11F12C04B946;\n\tTue, 26 Sep 2017 19:57:31 +0000 (UTC)","from [10.10.124.97] (ovpn-124-97.rdu2.redhat.com [10.10.124.97])\n\tby smtp.corp.redhat.com (Postfix) with ESMTP id CFFC917A40;\n\tTue, 26 Sep 2017 19:57:22 +0000 (UTC)"],"DMARC-Filter":"OpenDMARC Filter v1.3.2 mx1.redhat.com 11F12C04B946","To":"John Snow , qemu-devel@nongnu.org","References":"<20170913160333.23622-1-eblake@redhat.com>\n\t<20170913160333.23622-8-eblake@redhat.com>\n\t<00c44c8d-479b-b954-e605-56a3ed00651f@redhat.com>","From":"Eric Blake ","Openpgp":"url=http://people.redhat.com/eblake/eblake.gpg","Organization":"Red Hat, Inc.","Message-ID":"<9dd34363-1f30-8c38-979b-32cc53b5d252@redhat.com>","Date":"Tue, 26 Sep 2017 14:57:21 -0500","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101\n\tThunderbird/52.3.0","MIME-Version":"1.0","In-Reply-To":"<00c44c8d-479b-b954-e605-56a3ed00651f@redhat.com>","Content-Type":"multipart/signed; micalg=pgp-sha256;\n\tprotocol=\"application/pgp-signature\";\n\tboundary=\"6UKSnInKonnJUT1fxshi5cHd2Q9ApTnVa\"","X-Scanned-By":"MIMEDefang 2.79 on 10.5.11.12","X-Greylist":"Sender IP whitelisted, not delayed by milter-greylist-4.5.16\n\t(mx1.redhat.com [10.5.110.31]);\n\tTue, 26 Sep 2017 19:57:31 +0000 (UTC)","X-detected-operating-system":"by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]\n\t[fuzzy]","X-Received-From":"209.132.183.28","X-Content-Filtered-By":"Mailman/MimeDel 2.1.21","Subject":"Re: [Qemu-devel] [PATCH v4 07/23] block: Convert\n\tbdrv_get_block_status() to bytes","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"kwolf@redhat.com, famz@redhat.com, Stefan Hajnoczi ,\n\tqemu-block@nongnu.org, Max Reitz ","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}}]