From patchwork Fri Oct 4 15:46:57 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vladimir Sementsov-Ogievskiy X-Patchwork-Id: 1171922 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=virtuozzo.com Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 46lDzS6Qnvz9s4Y for ; Sat, 5 Oct 2019 01:57:40 +1000 (AEST) Received: from localhost ([::1]:50034 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iGPxE-0007hQ-LW for incoming@patchwork.ozlabs.org; Fri, 04 Oct 2019 11:57:16 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:41147) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iGPnj-00037z-8D for qemu-devel@nongnu.org; Fri, 04 Oct 2019 11:47:29 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iGPnh-0005gc-Tl for qemu-devel@nongnu.org; Fri, 04 Oct 2019 11:47:27 -0400 Received: from relay.sw.ru ([185.231.240.75]:36352) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iGPnR-0005Ed-Hz; Fri, 04 Oct 2019 11:47:21 -0400 Received: from [10.94.3.0] (helo=kvm.qa.sw.ru) by relay.sw.ru with esmtp (Exim 4.92.2) (envelope-from ) id 1iGPnK-00019e-RA; Fri, 04 Oct 2019 18:47:02 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org Subject: [PATCH 1/5] block: Mark commit and mirror as filter drivers Date: Fri, 4 Oct 2019 18:46:57 +0300 Message-Id: <20191004154701.3202-2-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20191004154701.3202-1-vsementsov@virtuozzo.com> References: <20191004154701.3202-1-vsementsov@virtuozzo.com> MIME-Version: 1.0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 185.231.240.75 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: fam@euphon.net, kwolf@redhat.com, vsementsov@virtuozzo.com, quintela@redhat.com, qemu-devel@nongnu.org, mreitz@redhat.com, stefanha@redhat.com, den@openvz.org, jsnow@redhat.com, dgilbert@redhat.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" From: Max Reitz The commit and mirror block nodes are filters, so they should be marked as such. Signed-off-by: Max Reitz Signed-off-by: Vladimir Sementsov-Ogievskiy [squash comment fix from another Max's patch and adjust commit msg] --- include/block/block_int.h | 8 +++++--- block/commit.c | 2 ++ block/mirror.c | 2 ++ 3 files changed, 9 insertions(+), 3 deletions(-) diff --git a/include/block/block_int.h b/include/block/block_int.h index 0422acdf1c..4af0fa8fe4 100644 --- a/include/block/block_int.h +++ b/include/block/block_int.h @@ -89,9 +89,11 @@ struct BlockDriver { int instance_size; /* set to true if the BlockDriver is a block filter. Block filters pass - * certain callbacks that refer to data (see block.c) to their bs->file if - * the driver doesn't implement them. Drivers that do not wish to forward - * must implement them and return -ENOTSUP. + * certain callbacks that refer to data (see block.c) to their bs->file + * or bs->backing (whichever one exists) if the driver doesn't implement + * them. Drivers that do not wish to forward must implement them and return + * -ENOTSUP. + * Note that filters are not allowed to modify data. */ bool is_filter; /* for snapshots block filter like Quorum can implement the diff --git a/block/commit.c b/block/commit.c index bc8454463d..07e7a3cd0a 100644 --- a/block/commit.c +++ b/block/commit.c @@ -253,6 +253,8 @@ static BlockDriver bdrv_commit_top = { .bdrv_co_block_status = bdrv_co_block_status_from_backing, .bdrv_refresh_filename = bdrv_commit_top_refresh_filename, .bdrv_child_perm = bdrv_commit_top_child_perm, + + .is_filter = true, }; void commit_start(const char *job_id, BlockDriverState *bs, diff --git a/block/mirror.c b/block/mirror.c index fe984efb90..838781dfaa 100644 --- a/block/mirror.c +++ b/block/mirror.c @@ -1487,6 +1487,8 @@ static BlockDriver bdrv_mirror_top = { .bdrv_refresh_filename = bdrv_mirror_top_refresh_filename, .bdrv_child_perm = bdrv_mirror_top_child_perm, .bdrv_refresh_limits = bdrv_mirror_top_refresh_limits, + + .is_filter = true, }; static BlockJob *mirror_start_job( From patchwork Fri Oct 4 15:46:58 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vladimir Sementsov-Ogievskiy X-Patchwork-Id: 1171911 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=virtuozzo.com Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 46lDr047L3z9s4Y for ; Sat, 5 Oct 2019 01:51:10 +1000 (AEST) Received: from localhost ([::1]:49972 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iGPrF-0003kg-LQ for incoming@patchwork.ozlabs.org; Fri, 04 Oct 2019 11:51:06 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:41127) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iGPni-00037F-GU for qemu-devel@nongnu.org; Fri, 04 Oct 2019 11:47:28 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iGPnh-0005e2-2u for qemu-devel@nongnu.org; Fri, 04 Oct 2019 11:47:26 -0400 Received: from relay.sw.ru ([185.231.240.75]:36332) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iGPnR-0005Eb-Hf; Fri, 04 Oct 2019 11:47:21 -0400 Received: from [10.94.3.0] (helo=kvm.qa.sw.ru) by relay.sw.ru with esmtp (Exim 4.92.2) (envelope-from ) id 1iGPnK-00019e-VT; Fri, 04 Oct 2019 18:47:03 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org Subject: [PATCH 2/5] migretion/block-dirty-bitmap: refactor init_dirty_bitmap_migration Date: Fri, 4 Oct 2019 18:46:58 +0300 Message-Id: <20191004154701.3202-3-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20191004154701.3202-1-vsementsov@virtuozzo.com> References: <20191004154701.3202-1-vsementsov@virtuozzo.com> MIME-Version: 1.0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 185.231.240.75 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: fam@euphon.net, kwolf@redhat.com, vsementsov@virtuozzo.com, quintela@redhat.com, qemu-devel@nongnu.org, mreitz@redhat.com, stefanha@redhat.com, den@openvz.org, jsnow@redhat.com, dgilbert@redhat.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Split out handling one bs, it is needed for the following commit, which will handle BlockBackends in separate. Signed-off-by: Vladimir Sementsov-Ogievskiy --- migration/block-dirty-bitmap.c | 93 +++++++++++++++++++--------------- 1 file changed, 51 insertions(+), 42 deletions(-) diff --git a/migration/block-dirty-bitmap.c b/migration/block-dirty-bitmap.c index 5121f86d73..46641b7861 100644 --- a/migration/block-dirty-bitmap.c +++ b/migration/block-dirty-bitmap.c @@ -268,59 +268,68 @@ static void dirty_bitmap_mig_cleanup(void) } /* Called with iothread lock taken. */ -static int init_dirty_bitmap_migration(void) +static int add_bitmaps_to_list(BlockDriverState *bs, const char *bs_name) { - BlockDriverState *bs; BdrvDirtyBitmap *bitmap; DirtyBitmapMigBitmapState *dbms; Error *local_err = NULL; - dirty_bitmap_mig_state.bulk_completed = false; - dirty_bitmap_mig_state.prev_bs = NULL; - dirty_bitmap_mig_state.prev_bitmap = NULL; - dirty_bitmap_mig_state.no_bitmaps = false; + for (bitmap = bdrv_dirty_bitmap_next(bs, NULL); bitmap; + bitmap = bdrv_dirty_bitmap_next(bs, bitmap)) + { + if (!bdrv_dirty_bitmap_name(bitmap)) { + continue; + } - for (bs = bdrv_next_all_states(NULL); bs; bs = bdrv_next_all_states(bs)) { - const char *name = bdrv_get_device_or_node_name(bs); + if (!bs_name || strcmp(bs_name, "") == 0) { + error_report("Found bitmap '%s' in unnamed node %p. It can't " + "be migrated", bdrv_dirty_bitmap_name(bitmap), bs); + return -1; + } - for (bitmap = bdrv_dirty_bitmap_next(bs, NULL); bitmap; - bitmap = bdrv_dirty_bitmap_next(bs, bitmap)) - { - if (!bdrv_dirty_bitmap_name(bitmap)) { - continue; - } + if (bdrv_dirty_bitmap_check(bitmap, BDRV_BITMAP_DEFAULT, &local_err)) { + error_report_err(local_err); + return -1; + } - if (!name || strcmp(name, "") == 0) { - error_report("Found bitmap '%s' in unnamed node %p. It can't " - "be migrated", bdrv_dirty_bitmap_name(bitmap), bs); - goto fail; - } + bdrv_ref(bs); + bdrv_dirty_bitmap_set_busy(bitmap, true); + + dbms = g_new0(DirtyBitmapMigBitmapState, 1); + dbms->bs = bs; + dbms->node_name = bs_name; + dbms->bitmap = bitmap; + dbms->total_sectors = bdrv_nb_sectors(bs); + dbms->sectors_per_chunk = CHUNK_SIZE * 8 * + bdrv_dirty_bitmap_granularity(bitmap) >> BDRV_SECTOR_BITS; + if (bdrv_dirty_bitmap_enabled(bitmap)) { + dbms->flags |= DIRTY_BITMAP_MIG_START_FLAG_ENABLED; + } + if (bdrv_dirty_bitmap_get_persistence(bitmap)) { + dbms->flags |= DIRTY_BITMAP_MIG_START_FLAG_PERSISTENT; + } - if (bdrv_dirty_bitmap_check(bitmap, BDRV_BITMAP_DEFAULT, - &local_err)) { - error_report_err(local_err); - goto fail; - } + QSIMPLEQ_INSERT_TAIL(&dirty_bitmap_mig_state.dbms_list, + dbms, entry); + } - bdrv_ref(bs); - bdrv_dirty_bitmap_set_busy(bitmap, true); - - dbms = g_new0(DirtyBitmapMigBitmapState, 1); - dbms->bs = bs; - dbms->node_name = name; - dbms->bitmap = bitmap; - dbms->total_sectors = bdrv_nb_sectors(bs); - dbms->sectors_per_chunk = CHUNK_SIZE * 8 * - bdrv_dirty_bitmap_granularity(bitmap) >> BDRV_SECTOR_BITS; - if (bdrv_dirty_bitmap_enabled(bitmap)) { - dbms->flags |= DIRTY_BITMAP_MIG_START_FLAG_ENABLED; - } - if (bdrv_dirty_bitmap_get_persistence(bitmap)) { - dbms->flags |= DIRTY_BITMAP_MIG_START_FLAG_PERSISTENT; - } + return 0; +} + +/* Called with iothread lock taken. */ +static int init_dirty_bitmap_migration(void) +{ + BlockDriverState *bs; + DirtyBitmapMigBitmapState *dbms; + + dirty_bitmap_mig_state.bulk_completed = false; + dirty_bitmap_mig_state.prev_bs = NULL; + dirty_bitmap_mig_state.prev_bitmap = NULL; + dirty_bitmap_mig_state.no_bitmaps = false; - QSIMPLEQ_INSERT_TAIL(&dirty_bitmap_mig_state.dbms_list, - dbms, entry); + for (bs = bdrv_next_all_states(NULL); bs; bs = bdrv_next_all_states(bs)) { + if (add_bitmaps_to_list(bs, bdrv_get_device_or_node_name(bs))) { + goto fail; } } From patchwork Fri Oct 4 15:46:59 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vladimir Sementsov-Ogievskiy X-Patchwork-Id: 1171912 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=virtuozzo.com Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 46lDrd39RFz9s4Y for ; Sat, 5 Oct 2019 01:51:45 +1000 (AEST) Received: from localhost ([::1]:49974 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iGPrq-0003xw-SP for incoming@patchwork.ozlabs.org; Fri, 04 Oct 2019 11:51:42 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:41111) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iGPnh-000371-UL for qemu-devel@nongnu.org; Fri, 04 Oct 2019 11:47:26 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iGPng-0005du-UR for qemu-devel@nongnu.org; Fri, 04 Oct 2019 11:47:25 -0400 Received: from relay.sw.ru ([185.231.240.75]:36340) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iGPnR-0005Ec-Hz; Fri, 04 Oct 2019 11:47:20 -0400 Received: from [10.94.3.0] (helo=kvm.qa.sw.ru) by relay.sw.ru with esmtp (Exim 4.92.2) (envelope-from ) id 1iGPnL-00019e-2t; Fri, 04 Oct 2019 18:47:03 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org Subject: [PATCH 3/5] block/dirty-bitmap: add bdrv_has_named_bitmaps helper Date: Fri, 4 Oct 2019 18:46:59 +0300 Message-Id: <20191004154701.3202-4-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20191004154701.3202-1-vsementsov@virtuozzo.com> References: <20191004154701.3202-1-vsementsov@virtuozzo.com> MIME-Version: 1.0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 185.231.240.75 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: fam@euphon.net, kwolf@redhat.com, vsementsov@virtuozzo.com, quintela@redhat.com, qemu-devel@nongnu.org, mreitz@redhat.com, stefanha@redhat.com, den@openvz.org, jsnow@redhat.com, dgilbert@redhat.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" To be used for bitmap migration in further commit. Signed-off-by: Vladimir Sementsov-Ogievskiy --- include/block/dirty-bitmap.h | 1 + block/dirty-bitmap.c | 13 +++++++++++++ 2 files changed, 14 insertions(+) diff --git a/include/block/dirty-bitmap.h b/include/block/dirty-bitmap.h index 4b4b731b46..4c0ebe5c2a 100644 --- a/include/block/dirty-bitmap.h +++ b/include/block/dirty-bitmap.h @@ -101,6 +101,7 @@ int64_t bdrv_get_meta_dirty_count(BdrvDirtyBitmap *bitmap); void bdrv_dirty_bitmap_truncate(BlockDriverState *bs, int64_t bytes); bool bdrv_dirty_bitmap_readonly(const BdrvDirtyBitmap *bitmap); bool bdrv_has_readonly_bitmaps(BlockDriverState *bs); +bool bdrv_has_named_bitmaps(BlockDriverState *bs); bool bdrv_dirty_bitmap_get_autoload(const BdrvDirtyBitmap *bitmap); bool bdrv_dirty_bitmap_get_persistence(BdrvDirtyBitmap *bitmap); bool bdrv_dirty_bitmap_inconsistent(const BdrvDirtyBitmap *bitmap); diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c index 134e0c9a0c..cfc957ebc2 100644 --- a/block/dirty-bitmap.c +++ b/block/dirty-bitmap.c @@ -742,6 +742,19 @@ bool bdrv_has_readonly_bitmaps(BlockDriverState *bs) return false; } +bool bdrv_has_named_bitmaps(BlockDriverState *bs) +{ + BdrvDirtyBitmap *bm; + + QLIST_FOREACH(bm, &bs->dirty_bitmaps, list) { + if (bdrv_dirty_bitmap_name(bm)) { + return true; + } + } + + return false; +} + /* Called with BQL taken. */ void bdrv_dirty_bitmap_set_persistence(BdrvDirtyBitmap *bitmap, bool persistent) { From patchwork Fri Oct 4 15:47:00 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vladimir Sementsov-Ogievskiy X-Patchwork-Id: 1171919 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=virtuozzo.com Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 46lDyG1bqjz9sPv for ; Sat, 5 Oct 2019 01:56:37 +1000 (AEST) Received: from localhost ([::1]:50018 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iGPwL-0006dF-W9 for incoming@patchwork.ozlabs.org; Fri, 04 Oct 2019 11:56:22 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:41154) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iGPnj-00038G-GV for qemu-devel@nongnu.org; Fri, 04 Oct 2019 11:47:29 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iGPnh-0005gf-U9 for qemu-devel@nongnu.org; Fri, 04 Oct 2019 11:47:27 -0400 Received: from relay.sw.ru ([185.231.240.75]:36348) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iGPnR-0005EY-J5; Fri, 04 Oct 2019 11:47:20 -0400 Received: from [10.94.3.0] (helo=kvm.qa.sw.ru) by relay.sw.ru with esmtp (Exim 4.92.2) (envelope-from ) id 1iGPnL-00019e-G6; Fri, 04 Oct 2019 18:47:03 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org Subject: [PATCH 4/5] migration/block-dirty-bitmap: fix bitmaps migration during mirror job Date: Fri, 4 Oct 2019 18:47:00 +0300 Message-Id: <20191004154701.3202-5-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20191004154701.3202-1-vsementsov@virtuozzo.com> References: <20191004154701.3202-1-vsementsov@virtuozzo.com> MIME-Version: 1.0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 185.231.240.75 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: fam@euphon.net, kwolf@redhat.com, vsementsov@virtuozzo.com, quintela@redhat.com, qemu-devel@nongnu.org, mreitz@redhat.com, stefanha@redhat.com, den@openvz.org, jsnow@redhat.com, dgilbert@redhat.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Important thing for bitmap migration is to select destination block node to obtain the migrated bitmap. Prepatch, on source we use bdrv_get_device_or_node_name() to identify the node, and on target we do bdrv_lookup_bs. bdrv_get_device_or_node_name() returns blk name only for direct children of blk. So, bitmaps of direct children of blks are migrated by blk name and others - by node name. Libvirt currently is unprepared to bitmap migration by node-name, node-names are mostly auto-generated. So actually only migration by blk name works. Now, consider classic libvirt migrations assisted by mirror block job: mirror block job inserts filter, so our source is not a direct child of blk, and bitmaps are migrated by node-names. And this just don't work. Let's fix it by allowing use blk-name even if some implicit filters are inserted. Note, that we possibly want to allow explicit filters skipping too, but this is another story. Note2: we, of course, can't skip filters and use blk name to migrate bitmaps in filtered node by blk name for this blk if these filters have named bitmaps which should be migrated. Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1652424 Signed-off-by: Vladimir Sementsov-Ogievskiy --- migration/block-dirty-bitmap.c | 39 +++++++++++++++++++++++++++++++++- 1 file changed, 38 insertions(+), 1 deletion(-) diff --git a/migration/block-dirty-bitmap.c b/migration/block-dirty-bitmap.c index 46641b7861..3105479c50 100644 --- a/migration/block-dirty-bitmap.c +++ b/migration/block-dirty-bitmap.c @@ -321,14 +321,48 @@ static int init_dirty_bitmap_migration(void) { BlockDriverState *bs; DirtyBitmapMigBitmapState *dbms; + GHashTable *handled_by_blk = g_hash_table_new(NULL, NULL); + BlockBackend *blk; dirty_bitmap_mig_state.bulk_completed = false; dirty_bitmap_mig_state.prev_bs = NULL; dirty_bitmap_mig_state.prev_bitmap = NULL; dirty_bitmap_mig_state.no_bitmaps = false; + /* + * Use blockdevice name for direct (or filtered) children of named block + * backends. + */ + for (blk = blk_next(NULL); blk; blk = blk_next(blk)) { + const char *name = blk_name(blk); + + if (!name || strcmp(name, "") == 0) { + continue; + } + + bs = blk_bs(blk); + + /* Skip filters without bitmaos */ + while (bs && bs->drv && bs->drv->is_filter && + !bdrv_has_named_bitmaps(bs)) + { + bs = bs->backing->bs ?: bs->file->bs; + } + + if (bs && bs->drv && !bs->drv->is_filter) { + if (add_bitmaps_to_list(bs, name)) { + goto fail; + } + g_hash_table_add(handled_by_blk, bs); + } + } + for (bs = bdrv_next_all_states(NULL); bs; bs = bdrv_next_all_states(bs)) { - if (add_bitmaps_to_list(bs, bdrv_get_device_or_node_name(bs))) { + if (g_hash_table_contains(handled_by_blk, bs)) { + continue; + } + + if (add_bitmaps_to_list(bs, bdrv_get_node_name(bs))) { goto fail; } } @@ -342,9 +376,12 @@ static int init_dirty_bitmap_migration(void) dirty_bitmap_mig_state.no_bitmaps = true; } + g_hash_table_destroy(handled_by_blk); + return 0; fail: + g_hash_table_destroy(handled_by_blk); dirty_bitmap_mig_cleanup(); return -1; From patchwork Fri Oct 4 15:47:01 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vladimir Sementsov-Ogievskiy X-Patchwork-Id: 1171910 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=virtuozzo.com Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 46lDpZ4y3Cz9s4Y for ; Sat, 5 Oct 2019 01:49:58 +1000 (AEST) Received: from localhost ([::1]:49958 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iGPq6-00039H-4h for incoming@patchwork.ozlabs.org; Fri, 04 Oct 2019 11:49:56 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:41118) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iGPni-00037D-5z for qemu-devel@nongnu.org; Fri, 04 Oct 2019 11:47:27 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iGPng-0005dW-QF for qemu-devel@nongnu.org; Fri, 04 Oct 2019 11:47:26 -0400 Received: from relay.sw.ru ([185.231.240.75]:36354) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iGPnR-0005EZ-J6; Fri, 04 Oct 2019 11:47:20 -0400 Received: from [10.94.3.0] (helo=kvm.qa.sw.ru) by relay.sw.ru with esmtp (Exim 4.92.2) (envelope-from ) id 1iGPnL-00019e-IW; Fri, 04 Oct 2019 18:47:03 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org Subject: [PATCH 5/5] iotests: 194: test also migration of dirty bitmap Date: Fri, 4 Oct 2019 18:47:01 +0300 Message-Id: <20191004154701.3202-6-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20191004154701.3202-1-vsementsov@virtuozzo.com> References: <20191004154701.3202-1-vsementsov@virtuozzo.com> MIME-Version: 1.0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 185.231.240.75 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: fam@euphon.net, kwolf@redhat.com, vsementsov@virtuozzo.com, quintela@redhat.com, qemu-devel@nongnu.org, mreitz@redhat.com, stefanha@redhat.com, den@openvz.org, jsnow@redhat.com, dgilbert@redhat.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Test that dirty bitmap migration works when we deal with mirror. Signed-off-by: Vladimir Sementsov-Ogievskiy --- tests/qemu-iotests/194 | 14 ++++++++++---- tests/qemu-iotests/194.out | 6 ++++++ 2 files changed, 16 insertions(+), 4 deletions(-) diff --git a/tests/qemu-iotests/194 b/tests/qemu-iotests/194 index d746ab1e21..bcf55a03e1 100755 --- a/tests/qemu-iotests/194 +++ b/tests/qemu-iotests/194 @@ -42,6 +42,8 @@ with iotests.FilePath('source.img') as source_img_path, \ .add_incoming('unix:{0}'.format(migration_sock_path)) .launch()) + source_vm.qmp_log('block-dirty-bitmap-add', node='drive0', name='bitmap0') + iotests.log('Launching NBD server on destination...') iotests.log(dest_vm.qmp('nbd-server-start', addr={'type': 'unix', 'data': {'path': nbd_sock_path}})) iotests.log(dest_vm.qmp('nbd-server-add', device='drive0', writable=True)) @@ -61,12 +63,14 @@ with iotests.FilePath('source.img') as source_img_path, \ filters=[iotests.filter_qmp_event]) iotests.log('Starting migration...') - source_vm.qmp('migrate-set-capabilities', - capabilities=[{'capability': 'events', 'state': True}]) - dest_vm.qmp('migrate-set-capabilities', - capabilities=[{'capability': 'events', 'state': True}]) + capabilities = [{'capability': 'events', 'state': True}, + {'capability': 'dirty-bitmaps', 'state': True}] + source_vm.qmp('migrate-set-capabilities', capabilities=capabilities) + dest_vm.qmp('migrate-set-capabilities', capabilities=capabilities) iotests.log(source_vm.qmp('migrate', uri='unix:{0}'.format(migration_sock_path))) + source_vm.qmp_log('migrate-start-postcopy') + while True: event1 = source_vm.event_wait('MIGRATION') iotests.log(event1, filters=[iotests.filter_qmp_event]) @@ -82,3 +86,5 @@ with iotests.FilePath('source.img') as source_img_path, \ iotests.log('Stopping the NBD server on destination...') iotests.log(dest_vm.qmp('nbd-server-stop')) break + + iotests.log(source_vm.qmp('query-block')['return'][0]['dirty-bitmaps']) diff --git a/tests/qemu-iotests/194.out b/tests/qemu-iotests/194.out index 71857853fb..dd60dcc14f 100644 --- a/tests/qemu-iotests/194.out +++ b/tests/qemu-iotests/194.out @@ -1,4 +1,6 @@ Launching VMs... +{"execute": "block-dirty-bitmap-add", "arguments": {"name": "bitmap0", "node": "drive0"}} +{"return": {}} Launching NBD server on destination... {"return": {}} {"return": {}} @@ -8,11 +10,15 @@ Waiting for `drive-mirror` to complete... {"data": {"device": "mirror-job0", "len": 1073741824, "offset": 1073741824, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_READY", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}} Starting migration... {"return": {}} +{"execute": "migrate-start-postcopy", "arguments": {}} +{"return": {}} {"data": {"status": "setup"}, "event": "MIGRATION", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}} {"data": {"status": "active"}, "event": "MIGRATION", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}} +{"data": {"status": "postcopy-active"}, "event": "MIGRATION", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}} {"data": {"status": "completed"}, "event": "MIGRATION", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}} Gracefully ending the `drive-mirror` job on source... {"return": {}} {"data": {"device": "mirror-job0", "len": 1073741824, "offset": 1073741824, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}} Stopping the NBD server on destination... {"return": {}} +[{"busy": false, "count": 0, "granularity": 65536, "name": "bitmap0", "persistent": false, "recording": true, "status": "active"}]