From patchwork Tue Sep 19 20:18:50 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Blake X-Patchwork-Id: 815831 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3xxZJz5TgLz9s81 for ; Wed, 20 Sep 2017 06:31:15 +1000 (AEST) Received: from localhost ([::1]:45261 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1duPAn-0006mU-R5 for incoming@patchwork.ozlabs.org; Tue, 19 Sep 2017 16:31:13 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:42283) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1duOzV-0005Fm-5w for qemu-devel@nongnu.org; Tue, 19 Sep 2017 16:19:37 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1duOzT-0004fX-KE for qemu-devel@nongnu.org; Tue, 19 Sep 2017 16:19:33 -0400 Received: from mx1.redhat.com ([209.132.183.28]:33288) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1duOzJ-0004XB-NT; Tue, 19 Sep 2017 16:19:21 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id C25947E42A; Tue, 19 Sep 2017 20:19:20 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com C25947E42A Authentication-Results: ext-mx03.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx03.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=eblake@redhat.com Received: from red.redhat.com (ovpn-124-97.rdu2.redhat.com [10.10.124.97]) by smtp.corp.redhat.com (Postfix) with ESMTP id D49025C891; Tue, 19 Sep 2017 20:19:12 +0000 (UTC) From: Eric Blake To: qemu-devel@nongnu.org Date: Tue, 19 Sep 2017 15:18:50 -0500 Message-Id: <20170919201910.25656-1-eblake@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.27]); Tue, 19 Sep 2017 20:19:20 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v9 00/20] make dirty-bitmap byte-based X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, famz@redhat.com, vsementsov@virtuozzo.com, jsnow@redhat.com, qemu-block@nongnu.org Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" There are patches floating around to add NBD_CMD_BLOCK_STATUS, but NBD wants to report status on byte granularity (even if the reporting will probably be naturally aligned to sectors or even much higher levels). I've therefore started the task of converting our block status code to report at a byte granularity rather than sectors. Now that 2.11 is open, I'm rebasing/reposting the remaining patches. The overall conversion currently looks like: part 1: bdrv_is_allocated (merged in 2.10, commit 51b0a488) part 2: dirty-bitmap (this series, v8 was here [1]) part 3: bdrv_get_block_status (v4 is posted [2] and is mostly reviewed) part 4: .bdrv_co_block_status (v3 is posted [3], but needs review) Available as a tag at: git fetch git://repo.or.cz/qemu/ericb.git nbd-byte-dirty-v8 [1] https://lists.gnu.org/archive/html/qemu-devel/2017-09/msg04733.html [2] https://lists.gnu.org/archive/html/qemu-devel/2017-09/msg03543.html [3] https://lists.gnu.org/archive/html/qemu-devel/2017-09/msg03812.html Since v8: - add R-b where appropriate - patch 5 simplified again [Fam] - patch 18: add a comment, and enhance iotests 165 to test that aspect [Kevin] 001/20:[----] [--] 'block: Make bdrv_img_create() size selection easier to read' 002/20:[----] [--] 'hbitmap: Rename serialization_granularity to serialization_align' 003/20:[----] [--] 'qcow2: Ensure bitmap serialization is aligned' 004/20:[----] [--] 'dirty-bitmap: Drop unused functions' 005/20:[0004] [FC] 'dirty-bitmap: Avoid size query failure during truncate' 006/20:[----] [--] 'dirty-bitmap: Change bdrv_dirty_bitmap_size() to report bytes' 007/20:[----] [--] 'dirty-bitmap: Track bitmap size by bytes' 008/20:[----] [--] 'dirty-bitmap: Change bdrv_dirty_bitmap_*serialize*() to take bytes' 009/20:[----] [--] 'qcow2: Switch sectors_covered_by_bitmap_cluster() to byte-based' 010/20:[----] [--] 'dirty-bitmap: Set iterator start by offset, not sector' 011/20:[----] [--] 'dirty-bitmap: Change bdrv_dirty_iter_next() to report byte offset' 012/20:[----] [--] 'dirty-bitmap: Change bdrv_get_dirty_count() to report bytes' 013/20:[----] [--] 'dirty-bitmap: Change bdrv_get_dirty_locked() to take bytes' 014/20:[----] [--] 'dirty-bitmap: Change bdrv_[re]set_dirty_bitmap() to use bytes' 015/20:[----] [--] 'mirror: Switch mirror_dirty_init() to byte-based iteration' 016/20:[----] [--] 'qcow2: Switch qcow2_measure() to byte-based iteration' 017/20:[----] [--] 'qcow2: Switch load_bitmap_data() to byte-based iteration' 018/20:[0007] [FC] 'qcow2: Switch store_bitmap_data() to byte-based iteration' 019/20:[----] [--] 'dirty-bitmap: Switch bdrv_set_dirty() to bytes' 020/20:[----] [--] 'dirty-bitmap: Convert internal hbitmap size/granularity' Eric Blake (20): block: Make bdrv_img_create() size selection easier to read hbitmap: Rename serialization_granularity to serialization_align qcow2: Ensure bitmap serialization is aligned dirty-bitmap: Drop unused functions dirty-bitmap: Avoid size query failure during truncate dirty-bitmap: Change bdrv_dirty_bitmap_size() to report bytes dirty-bitmap: Track bitmap size by bytes dirty-bitmap: Change bdrv_dirty_bitmap_*serialize*() to take bytes qcow2: Switch sectors_covered_by_bitmap_cluster() to byte-based dirty-bitmap: Set iterator start by offset, not sector dirty-bitmap: Change bdrv_dirty_iter_next() to report byte offset dirty-bitmap: Change bdrv_get_dirty_count() to report bytes dirty-bitmap: Change bdrv_get_dirty_locked() to take bytes dirty-bitmap: Change bdrv_[re]set_dirty_bitmap() to use bytes mirror: Switch mirror_dirty_init() to byte-based iteration qcow2: Switch qcow2_measure() to byte-based iteration qcow2: Switch load_bitmap_data() to byte-based iteration qcow2: Switch store_bitmap_data() to byte-based iteration dirty-bitmap: Switch bdrv_set_dirty() to bytes dirty-bitmap: Convert internal hbitmap size/granularity include/block/block_int.h | 2 +- include/block/dirty-bitmap.h | 43 ++++++-------- include/qemu/hbitmap.h | 8 +-- block/io.c | 6 +- block.c | 17 ++++-- block/backup.c | 7 +-- block/dirty-bitmap.c | 134 ++++++++++++++----------------------------- block/mirror.c | 79 +++++++++++-------------- block/qcow2-bitmap.c | 62 +++++++++++--------- block/qcow2.c | 22 ++++--- migration/block.c | 12 ++-- tests/test-hbitmap.c | 10 ++-- util/hbitmap.c | 8 +-- tests/qemu-iotests/165 | 2 +- 14 files changed, 173 insertions(+), 239 deletions(-)