{"id":807892,"url":"http://patchwork.ozlabs.org/api/1.0/patches/807892/?format=json","project":{"id":14,"url":"http://patchwork.ozlabs.org/api/1.0/projects/14/?format=json","name":"QEMU Development","link_name":"qemu-devel","list_id":"qemu-devel.nongnu.org","list_email":"qemu-devel@nongnu.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20170830210542.2153-6-eblake@redhat.com>","date":"2017-08-30T21:05:29","name":"[v6,05/18] dirty-bitmap: Change bdrv_dirty_bitmap_size() to report bytes","commit_ref":null,"pull_url":null,"state":"new","archived":false,"hash":"0c976d69f9ca55f9ce7a8c13cae3bd58f71957d3","submitter":{"id":6591,"url":"http://patchwork.ozlabs.org/api/1.0/people/6591/?format=json","name":"Eric Blake","email":"eblake@redhat.com"},"delegate":null,"mbox":"http://patchwork.ozlabs.org/project/qemu-devel/patch/20170830210542.2153-6-eblake@redhat.com/mbox/","series":[{"id":700,"url":"http://patchwork.ozlabs.org/api/1.0/series/700/?format=json","date":"2017-08-30T21:05:25","name":"make dirty-bitmap byte-based","version":6,"mbox":"http://patchwork.ozlabs.org/series/700/mbox/"}],"check":"pending","checks":"http://patchwork.ozlabs.org/api/patches/807892/checks/","tags":{},"headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","ext-mx09.extmail.prod.ext.phx2.redhat.com;\n\tdmarc=none (p=none dis=none) header.from=redhat.com","ext-mx09.extmail.prod.ext.phx2.redhat.com;\n\tspf=fail smtp.mailfrom=eblake@redhat.com"],"Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xjJ3n1T25z9s8P\n\tfor ;\n\tThu, 31 Aug 2017 07:07:17 +1000 (AEST)","from localhost ([::1]:52690 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1dnACh-00044H-4s\n\tfor incoming@patchwork.ozlabs.org; Wed, 30 Aug 2017 17:07:15 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:36907)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1dnABX-0003jA-Ug\n\tfor qemu-devel@nongnu.org; Wed, 30 Aug 2017 17:06:05 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1dnABW-0007TG-MY\n\tfor qemu-devel@nongnu.org; Wed, 30 Aug 2017 17:06:03 -0400","from mx1.redhat.com ([209.132.183.28]:34820)\n\tby eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)\n\t(Exim 4.71) (envelope-from )\n\tid 1dnABS-0007Mi-Fz; Wed, 30 Aug 2017 17:05:58 -0400","from smtp.corp.redhat.com\n\t(int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16])\n\t(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby mx1.redhat.com (Postfix) with ESMTPS id 8A39D4E33D;\n\tWed, 30 Aug 2017 21:05:57 +0000 (UTC)","from red.redhat.com (ovpn-122-186.rdu2.redhat.com [10.10.122.186])\n\tby smtp.corp.redhat.com (Postfix) with ESMTP id 629AC77557;\n\tWed, 30 Aug 2017 21:05:56 +0000 (UTC)"],"DMARC-Filter":"OpenDMARC Filter v1.3.2 mx1.redhat.com 8A39D4E33D","From":"Eric Blake ","To":"qemu-devel@nongnu.org","Date":"Wed, 30 Aug 2017 16:05:29 -0500","Message-Id":"<20170830210542.2153-6-eblake@redhat.com>","In-Reply-To":"<20170830210542.2153-1-eblake@redhat.com>","References":"<20170830210542.2153-1-eblake@redhat.com>","X-Scanned-By":"MIMEDefang 2.79 on 10.5.11.16","X-Greylist":"Sender IP whitelisted, not delayed by milter-greylist-4.5.16\n\t(mx1.redhat.com [10.5.110.38]);\n\tWed, 30 Aug 2017 21:05:57 +0000 (UTC)","X-detected-operating-system":"by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]\n\t[fuzzy]","X-Received-From":"209.132.183.28","Subject":"[Qemu-devel] [PATCH v6 05/18] dirty-bitmap: Change\n\tbdrv_dirty_bitmap_size() to report bytes","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"kwolf@redhat.com, vsementsov@virtuozzo.com, Fam Zheng , \n\tqemu-block@nongnu.org, Max Reitz , jsnow@redhat.com","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"},"content":"We are still using an internal hbitmap that tracks a size in sectors,\nwith the granularity scaled down accordingly, because it lets us\nuse a shortcut for our iterators which are currently sector-based.\nBut there's no reason we can't track the dirty bitmap size in bytes,\nsince it is (mostly) an internal-only variable (remember, the size\nis how many bytes are covered by the bitmap, not how many bytes the\nbitmap occupies). Furthermore, we're already reporting bytes for\nbdrv_dirty_bitmap_granularity(); mixing bytes and sectors in our\nreturn values is a recipe for confusion. A later cleanup will\nconvert dirty bitmap internals to be entirely byte-based,\neliminating the intermediate sector rounding added here; and\ntechnically, since bdrv_getlength() already rounds up to sectors,\nour use of DIV_ROUND_UP is more for theoretical completeness than\nfor any actual rounding.\n\nThe only external caller in qcow2-bitmap.c is temporarily more verbose\n(because it is still using sector-based math), but will later be\nswitched to track progress by bytes instead of sectors.\n\nUse is_power_of_2() while at it, instead of open-coding that, and\nadd an assertion where bdrv_getlength() should not fail.\n\nSigned-off-by: Eric Blake \nReviewed-by: John Snow \n\n---\nv6: no change\nv5: fix bdrv_dirty_bitmap_truncate [John], drop R-b\nv4: retitle from \"Track size in bytes\", rebase to persistent bitmaps,\nround up when converting bytes to sectors\nv3: no change\nv2: tweak commit message, no code change\n---\n block/dirty-bitmap.c | 26 +++++++++++++++-----------\n block/qcow2-bitmap.c | 14 ++++++++------\n 2 files changed, 23 insertions(+), 17 deletions(-)","diff":"diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c\nindex 42a55e4a4b..e65ec4f7ec 100644\n--- a/block/dirty-bitmap.c\n+++ b/block/dirty-bitmap.c\n@@ -1,7 +1,7 @@\n /*\n * Block Dirty Bitmap\n *\n- * Copyright (c) 2016 Red Hat. Inc\n+ * Copyright (c) 2016-2017 Red Hat. Inc\n *\n * Permission is hereby granted, free of charge, to any person obtaining a copy\n * of this software and associated documentation files (the \"Software\"), to deal\n@@ -42,7 +42,7 @@ struct BdrvDirtyBitmap {\n HBitmap *meta; /* Meta dirty bitmap */\n BdrvDirtyBitmap *successor; /* Anonymous child; implies frozen status */\n char *name; /* Optional non-empty unique ID */\n- int64_t size; /* Size of the bitmap (Number of sectors) */\n+ int64_t size; /* Size of the bitmap, in bytes */\n bool disabled; /* Bitmap is disabled. It ignores all writes to\n the device */\n int active_iterators; /* How many iterators are active */\n@@ -115,17 +115,14 @@ BdrvDirtyBitmap *bdrv_create_dirty_bitmap(BlockDriverState *bs,\n {\n int64_t bitmap_size;\n BdrvDirtyBitmap *bitmap;\n- uint32_t sector_granularity;\n\n- assert((granularity & (granularity - 1)) == 0);\n+ assert(is_power_of_2(granularity) && granularity >= BDRV_SECTOR_SIZE);\n\n if (name && bdrv_find_dirty_bitmap(bs, name)) {\n error_setg(errp, \"Bitmap already exists: %s\", name);\n return NULL;\n }\n- sector_granularity = granularity >> BDRV_SECTOR_BITS;\n- assert(sector_granularity);\n- bitmap_size = bdrv_nb_sectors(bs);\n+ bitmap_size = bdrv_getlength(bs);\n if (bitmap_size < 0) {\n error_setg_errno(errp, -bitmap_size, \"could not get length of device\");\n errno = -bitmap_size;\n@@ -133,7 +130,12 @@ BdrvDirtyBitmap *bdrv_create_dirty_bitmap(BlockDriverState *bs,\n }\n bitmap = g_new0(BdrvDirtyBitmap, 1);\n bitmap->mutex = &bs->dirty_bitmap_mutex;\n- bitmap->bitmap = hbitmap_alloc(bitmap_size, ctz32(sector_granularity));\n+ /*\n+ * TODO - let hbitmap track full granularity. For now, it is tracking\n+ * only sector granularity, as a shortcut for our iterators.\n+ */\n+ bitmap->bitmap = hbitmap_alloc(DIV_ROUND_UP(bitmap_size, BDRV_SECTOR_SIZE),\n+ ctz32(granularity) - BDRV_SECTOR_BITS);\n bitmap->size = bitmap_size;\n bitmap->name = g_strdup(name);\n bitmap->disabled = false;\n@@ -305,13 +307,14 @@ BdrvDirtyBitmap *bdrv_reclaim_dirty_bitmap(BlockDriverState *bs,\n void bdrv_dirty_bitmap_truncate(BlockDriverState *bs)\n {\n BdrvDirtyBitmap *bitmap;\n- uint64_t size = bdrv_nb_sectors(bs);\n+ int64_t size = bdrv_getlength(bs);\n\n+ assert(size >= 0);\n bdrv_dirty_bitmaps_lock(bs);\n QLIST_FOREACH(bitmap, &bs->dirty_bitmaps, list) {\n assert(!bdrv_dirty_bitmap_frozen(bitmap));\n assert(!bitmap->active_iterators);\n- hbitmap_truncate(bitmap->bitmap, size);\n+ hbitmap_truncate(bitmap->bitmap, DIV_ROUND_UP(size, BDRV_SECTOR_SIZE));\n bitmap->size = size;\n }\n bdrv_dirty_bitmaps_unlock(bs);\n@@ -549,7 +552,8 @@ void bdrv_clear_dirty_bitmap(BdrvDirtyBitmap *bitmap, HBitmap **out)\n hbitmap_reset_all(bitmap->bitmap);\n } else {\n HBitmap *backup = bitmap->bitmap;\n- bitmap->bitmap = hbitmap_alloc(bitmap->size,\n+ bitmap->bitmap = hbitmap_alloc(DIV_ROUND_UP(bitmap->size,\n+ BDRV_SECTOR_SIZE),\n hbitmap_granularity(backup));\n *out = backup;\n }\ndiff --git a/block/qcow2-bitmap.c b/block/qcow2-bitmap.c\nindex b3ee4c794a..65122e9ae1 100644\n--- a/block/qcow2-bitmap.c\n+++ b/block/qcow2-bitmap.c\n@@ -295,10 +295,11 @@ static int load_bitmap_data(BlockDriverState *bs,\n BDRVQcow2State *s = bs->opaque;\n uint64_t sector, sbc;\n uint64_t bm_size = bdrv_dirty_bitmap_size(bitmap);\n+ uint64_t bm_sectors = DIV_ROUND_UP(bm_size, BDRV_SECTOR_SIZE);\n uint8_t *buf = NULL;\n uint64_t i, tab_size =\n size_to_clusters(s,\n- bdrv_dirty_bitmap_serialization_size(bitmap, 0, bm_size));\n+ bdrv_dirty_bitmap_serialization_size(bitmap, 0, bm_sectors));\n\n if (tab_size != bitmap_table_size || tab_size > BME_MAX_TABLE_SIZE) {\n return -EINVAL;\n@@ -307,7 +308,7 @@ static int load_bitmap_data(BlockDriverState *bs,\n buf = g_malloc(s->cluster_size);\n sbc = sectors_covered_by_bitmap_cluster(s, bitmap);\n for (i = 0, sector = 0; i < tab_size; ++i, sector += sbc) {\n- uint64_t count = MIN(bm_size - sector, sbc);\n+ uint64_t count = MIN(bm_sectors - sector, sbc);\n uint64_t entry = bitmap_table[i];\n uint64_t offset = entry & BME_TABLE_ENTRY_OFFSET_MASK;\n\n@@ -1077,13 +1078,14 @@ static uint64_t *store_bitmap_data(BlockDriverState *bs,\n int64_t sector;\n uint64_t sbc;\n uint64_t bm_size = bdrv_dirty_bitmap_size(bitmap);\n+ uint64_t bm_sectors = DIV_ROUND_UP(bm_size, BDRV_SECTOR_SIZE);\n const char *bm_name = bdrv_dirty_bitmap_name(bitmap);\n uint8_t *buf = NULL;\n BdrvDirtyBitmapIter *dbi;\n uint64_t *tb;\n uint64_t tb_size =\n size_to_clusters(s,\n- bdrv_dirty_bitmap_serialization_size(bitmap, 0, bm_size));\n+ bdrv_dirty_bitmap_serialization_size(bitmap, 0, bm_sectors));\n\n if (tb_size > BME_MAX_TABLE_SIZE ||\n tb_size * s->cluster_size > BME_MAX_PHYS_SIZE)\n@@ -1101,7 +1103,7 @@ static uint64_t *store_bitmap_data(BlockDriverState *bs,\n dbi = bdrv_dirty_iter_new(bitmap, 0);\n buf = g_malloc(s->cluster_size);\n sbc = sectors_covered_by_bitmap_cluster(s, bitmap);\n- assert(DIV_ROUND_UP(bm_size, sbc) == tb_size);\n+ assert(DIV_ROUND_UP(bm_sectors, sbc) == tb_size);\n\n while ((sector = bdrv_dirty_iter_next(dbi)) != -1) {\n uint64_t cluster = sector / sbc;\n@@ -1109,7 +1111,7 @@ static uint64_t *store_bitmap_data(BlockDriverState *bs,\n int64_t off;\n\n sector = cluster * sbc;\n- end = MIN(bm_size, sector + sbc);\n+ end = MIN(bm_sectors, sector + sbc);\n write_size =\n bdrv_dirty_bitmap_serialization_size(bitmap, sector, end - sector);\n assert(write_size <= s->cluster_size);\n@@ -1141,7 +1143,7 @@ static uint64_t *store_bitmap_data(BlockDriverState *bs,\n goto fail;\n }\n\n- if (end >= bm_size) {\n+ if (end >= bm_sectors) {\n break;\n }\n\n","prefixes":["v6","05/18"]}