{"id":2233225,"url":"http://patchwork.ozlabs.org/api/1.1/patches/2233225/?format=json","web_url":"http://patchwork.ozlabs.org/project/qemu-devel/patch/20260505202640.1011006-12-peterx@redhat.com/","project":{"id":14,"url":"http://patchwork.ozlabs.org/api/1.1/projects/14/?format=json","name":"QEMU Development","link_name":"qemu-devel","list_id":"qemu-devel.nongnu.org","list_email":"qemu-devel@nongnu.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20260505202640.1011006-12-peterx@redhat.com>","date":"2026-05-05T20:26:28","name":"[PULL,11/23] migration: Move iteration counter out of RAM","commit_ref":null,"pull_url":null,"state":"new","archived":false,"hash":"646333047d58b3edc74a56ff0985d9eb69f9be1b","submitter":{"id":67717,"url":"http://patchwork.ozlabs.org/api/1.1/people/67717/?format=json","name":"Peter Xu","email":"peterx@redhat.com"},"delegate":null,"mbox":"http://patchwork.ozlabs.org/project/qemu-devel/patch/20260505202640.1011006-12-peterx@redhat.com/mbox/","series":[{"id":502897,"url":"http://patchwork.ozlabs.org/api/1.1/series/502897/?format=json","web_url":"http://patchwork.ozlabs.org/project/qemu-devel/list/?series=502897","date":"2026-05-05T20:26:17","name":"[PULL,01/23] migration: Fix blocking in POSTCOPY_DEVICE during package load","version":1,"mbox":"http://patchwork.ozlabs.org/series/502897/mbox/"}],"comments":"http://patchwork.ozlabs.org/api/patches/2233225/comments/","check":"pending","checks":"http://patchwork.ozlabs.org/api/patches/2233225/checks/","tags":{},"headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@legolas.ozlabs.org","Authentication-Results":["legolas.ozlabs.org;\n\tdkim=pass (1024-bit key;\n unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256\n header.s=mimecast20190719 header.b=W2OJLdqC;\n\tdkim=pass (2048-bit key;\n unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256\n header.s=google header.b=iFQtbsIv;\n\tdkim-atps=neutral","legolas.ozlabs.org;\n spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org\n (client-ip=209.51.188.17; helo=lists1p.gnu.org;\n envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n receiver=patchwork.ozlabs.org)"],"Received":["from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17])\n\t(using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits))\n\t(No client certificate requested)\n\tby legolas.ozlabs.org (Postfix) with ESMTPS id 4g999Y0V6Pz1yK2\n\tfor ; Wed, 06 May 2026 06:29:53 +1000 (AEST)","from localhost ([::1] helo=lists1p.gnu.org)\n\tby lists1p.gnu.org with esmtp (Exim 4.90_1)\n\t(envelope-from )\n\tid 1wKMMN-0004bC-2G; Tue, 05 May 2026 16:27:15 -0400","from eggs.gnu.org ([2001:470:142:3::10])\n by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)\n (Exim 4.90_1) (envelope-from ) id 1wKMMB-0004Sa-Mb\n for qemu-devel@nongnu.org; Tue, 05 May 2026 16:27:05 -0400","from us-smtp-delivery-124.mimecast.com ([170.10.129.124])\n by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)\n (Exim 4.90_1) (envelope-from ) id 1wKMM8-0002co-KO\n for qemu-devel@nongnu.org; Tue, 05 May 2026 16:27:03 -0400","from mail-qv1-f69.google.com (mail-qv1-f69.google.com\n [209.85.219.69]) by relay.mimecast.com with ESMTP with STARTTLS\n (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id\n us-mta-493-UTfb5-BHNqGE6y2Syc4JWA-1; Tue, 05 May 2026 16:26:58 -0400","by mail-qv1-f69.google.com with SMTP id\n 6a1803df08f44-8acaea1ffe7so142717196d6.2\n for ; Tue, 05 May 2026 13:26:58 -0700 (PDT)","from x1.com ([142.189.10.167]) by smtp.gmail.com with ESMTPSA id\n 6a1803df08f44-8b53c6b8123sm155283806d6.35.2026.05.05.13.26.56\n (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);\n Tue, 05 May 2026 13:26:56 -0700 (PDT)"],"DKIM-Signature":["v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com;\n s=mimecast20190719; t=1778012819;\n h=from:from:reply-to:subject:subject:date:date:message-id:message-id:\n to:to:cc:cc:mime-version:mime-version:\n content-transfer-encoding:content-transfer-encoding:\n in-reply-to:in-reply-to:references:references;\n bh=MloABw5GE3WIOEB46lkCJz/DCq9G15tQHSXMQreAC9I=;\n b=W2OJLdqCd1+8gEaUf3X59eumjz3vK8+ZFgvaravse+lwtQGw32+4K8fKubStQXIWJOfiXr\n l+gsCizO1GQDbOCcqTiwi3d3VaZQmWfMh+nRiYWwQcftc2gJI91XjWE8oYcTN55Eu/vtiU\n 0lhpnGYN9pp9zHmaKjebLkOU1xJ2pHc=","v=1; a=rsa-sha256; c=relaxed/relaxed;\n d=redhat.com; s=google; t=1778012818; x=1778617618; darn=nongnu.org;\n h=content-transfer-encoding:mime-version:references:in-reply-to\n :message-id:date:subject:cc:to:from:from:to:cc:subject:date\n :message-id:reply-to;\n bh=MloABw5GE3WIOEB46lkCJz/DCq9G15tQHSXMQreAC9I=;\n b=iFQtbsIv3pLopwKjSYa1ri9MA9S8wfvFDe5Vjd2RGHxLFFtnuJ5KilyRN6td5yvWoT\n SEhessTL6fRDEZfjVHc1mpLPa2YBPL8FXVMnxFAdTu4ngYX3/MNWkeSsn0dykwc1Kd74\n QXnuq11UIQe9vfLe3uzAQnFn/t0q1EIxkjQ7GkScoCh38THOxzQ/BcV3mUZ5zjpxDMaV\n xMP12P3kotfAb/bmk/t2Sq/zSwwwN2j8iA8z5u6ncNesTNX5gjrmwIJ463G1/d9GqVxE\n EYjSn75j77y4t6OmNyDQRR5WBxLtBSyvmRjetn4XWyDNp02fl4sm36HUX419AtYWrqUG\n VIiw=="],"X-MC-Unique":"UTfb5-BHNqGE6y2Syc4JWA-1","X-Mimecast-MFC-AGG-ID":"UTfb5-BHNqGE6y2Syc4JWA_1778012818","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n d=1e100.net; s=20251104; t=1778012818; x=1778617618;\n h=content-transfer-encoding:mime-version:references:in-reply-to\n :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from\n :to:cc:subject:date:message-id:reply-to;\n bh=MloABw5GE3WIOEB46lkCJz/DCq9G15tQHSXMQreAC9I=;\n b=CrHMeYbpKU21t6hFAa8W+SzHlCtbnbjJqGbOE2cyCa5zdpvC9pNVCjbIMBTPIVzqUD\n xstzZFO8XwojiAjM/KNsNrnCeJox5Smo1S6XVeKps4oqBKRQnVOJ4363K1ReDBV/a6la\n XxklWY2rhnZO5Z2oOfGsOIWAUkc4VMzC18YbK5ydMJCKNIumwWnF7NVe4dnJc0MDlvSz\n eR9pu7w4ZNxUUim+LjZasN1Qhpyoc3nui1fXU5rzAabpS+PuRsOTbg1J0MFvUyHlxHQb\n gV6XRdEFJZUD8fhJ+vOoxGuzGtY3eTZg6dsJ2hbkPJOaRQCKXZpkVzgI4P8K7NsaAj09\n qIdQ==","X-Gm-Message-State":"AOJu0YyBtG1bes8j5IFUTWGeA0Y5QSHwJ02W/xWXJXA8PL2qAglemh6A\n Zi+I6pQEQVvH1pvwPDAsK+EmgJb+gL1aq/06vIDP/rgt7Ikv1Fqj94iU0Yt0sTQGCa62jNv4ryC\n 1aqUPdWO9/NWRKpea66jPBf/6xCn0XkZ+4W2R17q9QtGAQ+XG5VGRfDV9MVnSy/OMwtGR6HGVX0\n cUldua0u9aAejemk/5mh0fBBPk5lIwY6Krs01Z4Q==","X-Gm-Gg":"AeBDievyq8iAG4oMsT//1pjYNsZ4MZlEmmz4tVn4qokqjGoyD5WvYbaikcesNeenqTR\n /DIHJawzHzPWmPSdDi9yN5HiOcyskMVK1w20dcZR2FLCsLZJF57ZtWiVxsaJdsPd+ijcatRAxJn\n Y6VSLcw0buKf8yYOovU61y701YuECO+bz1QY9jbLWTIXSzOmG4vJWhPPy48NiZlt96h2C//vP5E\n 8eTOI/n/kK2xVn2BAvvS6OMZg3/uCWOb4Fy18cKJCNgW+QGUtZE5OlY8HiJMp9olVURWPChqsz/\n 9c2JQ8wc9ZNlwtxhK4h8WLVwT6Ofj9kXAR7Giwq4sn0sMIr/eX6L6ep1tECP1McCTizTi+hgPBT\n z5tR0IgArUgrHCZwECrcubchAIvyxySB6xQHBW6Agg9yL01L3/U7or3Q=","X-Received":["by 2002:a0c:e089:0:b0:8ac:acd6:c988 with SMTP id\n 6a1803df08f44-8bc422a67f3mr5318746d6.1.1778012817573;\n Tue, 05 May 2026 13:26:57 -0700 (PDT)","by 2002:a0c:e089:0:b0:8ac:acd6:c988 with SMTP id\n 6a1803df08f44-8bc422a67f3mr5318116d6.1.1778012816962;\n Tue, 05 May 2026 13:26:56 -0700 (PDT)"],"From":"Peter Xu ","To":"qemu-devel@nongnu.org","Cc":"Fabiano Rosas , Paolo Bonzini ,\n Peter Xu , Hyman Huang ,\n Prasad Pandit , Juraj Marcin ","Subject":"[PULL 11/23] migration: Move iteration counter out of RAM","Date":"Tue, 5 May 2026 16:26:28 -0400","Message-ID":"<20260505202640.1011006-12-peterx@redhat.com>","X-Mailer":"git-send-email 2.53.0","In-Reply-To":"<20260505202640.1011006-1-peterx@redhat.com>","References":"<20260505202640.1011006-1-peterx@redhat.com>","MIME-Version":"1.0","Content-Transfer-Encoding":"8bit","Received-SPF":"pass client-ip=170.10.129.124; envelope-from=peterx@redhat.com;\n helo=us-smtp-delivery-124.mimecast.com","X-Spam_score_int":"-24","X-Spam_score":"-2.5","X-Spam_bar":"--","X-Spam_report":"(-2.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.443,\n DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1,\n RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001,\n SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no","X-Spam_action":"no action","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"qemu development ","List-Unsubscribe":",\n ","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n ","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org"},"content":"It used to hide in RAM dirty sync path. Now with more modules being able\nto slow sync on dirty information, keeping it there may not be good anymore\nbecause it's not RAM's own concept for iterations: all modules should\nfollow.\n\nMore importantly, mgmt may try to query dirty info (to make policy\ndecisions like adjusting downtime) by listening to iteration count changes\nvia QMP events. So we must make sure the boost of iterations only happens\n_after_ the dirty sync operations with whatever form (RAM's dirty bitmap\nsync, or VFIO's different ioctls to fetch latest dirty info from kernel).\n\nMove this to core migration path to manage, together with the event\ngeneration, so that it can be well ordered with the sync operations for all\nmodules.\n\nThis brings a good side effect that we should have an old issue regarding\nto cpu_throttle_dirty_sync_timer_tick() which can randomly boost iteration\ncounts (because it invokes sync ops). Now it won't, which is actually the\nright behavior.\n\nSaid that, we have code (not only QEMU, but likely mgmt too) assuming the\n1st iteration will always shows dirty count to 1. Make it initialized with\n1 this time, because we'll miss the dirty sync for setup() on boosting this\ncounter now.\n\nReviewed-by: Hyman Huang \nReviewed-by: Prasad Pandit \nReviewed-by: Juraj Marcin \nLink: https://lore.kernel.org/r/20260421202110.306051-10-peterx@redhat.com\nSigned-off-by: Peter Xu \n---\n migration/migration-stats.h | 3 ++-\n migration/migration.c | 29 ++++++++++++++++++++++++++---\n migration/ram.c | 6 ------\n 3 files changed, 28 insertions(+), 10 deletions(-)","diff":"diff --git a/migration/migration-stats.h b/migration/migration-stats.h\nindex 1153520f7a..326ddb0088 100644\n--- a/migration/migration-stats.h\n+++ b/migration/migration-stats.h\n@@ -43,7 +43,8 @@ typedef struct {\n */\n uint64_t dirty_pages_rate;\n /*\n- * Number of times we have synchronized guest bitmaps.\n+ * Number of times we have synchronized guest bitmaps. This always\n+ * starts from 1 for the 1st iteration.\n */\n uint64_t dirty_sync_count;\n /*\ndiff --git a/migration/migration.c b/migration/migration.c\nindex 049b69fbe7..8abc7e0327 100644\n--- a/migration/migration.c\n+++ b/migration/migration.c\n@@ -1654,10 +1654,15 @@ int migrate_init(MigrationState *s, Error **errp)\n s->threshold_size = 0;\n s->switchover_acked = false;\n s->rdma_migration = false;\n+\n /*\n- * set mig_stats memory to zero for a new migration\n+ * set mig_stats memory to zero for a new migration.. except the\n+ * iteration counter, which we want to make sure it returns 1 for the\n+ * first iteration.\n */\n memset(&mig_stats, 0, sizeof(mig_stats));\n+ mig_stats.dirty_sync_count = 1;\n+\n migration_reset_vfio_bytes_transferred();\n \n s->postcopy_package_loaded = false;\n@@ -3234,10 +3239,28 @@ static bool migration_iteration_next_ready(MigrationState *s,\n static void migration_iteration_go_next(MigPendingData *pending)\n {\n /*\n- * Do a slow sync will achieve this. TODO: move RAM iteration code\n- * into the core layer.\n+ * Do a slow sync first before boosting the iteration count.\n */\n qemu_savevm_query_pending(pending, true);\n+\n+ /*\n+ * Boost dirty sync count to reflect we finished one iteration.\n+ *\n+ * NOTE: we need to make sure when this happens (together with the\n+ * event sent below) all modules have slow-synced the pending data\n+ * above. That means a write mem barrier, but qatomic_add() should be\n+ * enough.\n+ *\n+ * It's because a mgmt could wait on the iteration event to query again\n+ * on pending data for policy changes (e.g. downtime adjustments). The\n+ * ordering will make sure the query will fetch the latest results from\n+ * all the modules.\n+ */\n+ qatomic_add(&mig_stats.dirty_sync_count, 1);\n+\n+ if (migrate_events()) {\n+ qapi_event_send_migration_pass(mig_stats.dirty_sync_count);\n+ }\n }\n \n static bool postcopy_should_start(MigrationState *s, MigPendingData *pending)\ndiff --git a/migration/ram.c b/migration/ram.c\nindex 44503bf3f7..ecd4b6165c 100644\n--- a/migration/ram.c\n+++ b/migration/ram.c\n@@ -1136,8 +1136,6 @@ static void migration_bitmap_sync(RAMState *rs, bool last_stage)\n RAMBlock *block;\n int64_t end_time;\n \n- qatomic_add(&mig_stats.dirty_sync_count, 1);\n-\n if (!rs->time_last_bitmap_sync) {\n rs->time_last_bitmap_sync = qemu_clock_get_ms(QEMU_CLOCK_REALTIME);\n }\n@@ -1172,10 +1170,6 @@ static void migration_bitmap_sync(RAMState *rs, bool last_stage)\n rs->num_dirty_pages_period = 0;\n rs->bytes_xfer_prev = migration_transferred_bytes();\n }\n- if (migrate_events()) {\n- uint64_t generation = qatomic_read(&mig_stats.dirty_sync_count);\n- qapi_event_send_migration_pass(generation);\n- }\n }\n \n void migration_bitmap_sync_precopy(bool last_stage)\n","prefixes":["PULL","11/23"]}