[{"id":3675425,"web_url":"http://patchwork.ozlabs.org/comment/3675425/","msgid":"","list_archive_url":null,"date":"2026-04-09T17:41:21","subject":"Re: [PATCH 13/14] migration/qapi: Introduce system-wise \"remaining\"\n reports","submitter":{"id":89058,"url":"http://patchwork.ozlabs.org/api/people/89058/","name":"Juraj Marcin","email":"jmarcin@redhat.com"},"content":"On 2026-04-08 12:55, Peter Xu wrote:\n> Currently, mgmt can only query for remaining RAM, not system-wise remaining\n> data. It was not a problem before, because for a very long time RAM was\n> the only part that matters.\n> \n> After VFIO migrations landed upstream, it may not be true anymore\n> especially considering that there can be GPU devices that contain GBs of\n> device states.\n> \n> Add a new \"remaining\" field in query-migrate results, reflecting\n> system-wise remaining data, which will include everything (e.g. VFIO).\n> \n> This information will be useful for mgmt to implement generic way of stall\n> detection that covers all system resources. Say, when system remaining\n> data does not decrease anymore for a relatively long period of time, then\n> it may mean that there is a challenge of converging, so mgmt can act based\n> on how this value changes over time (especially if sampled after each\n> migration iteration).\n> \n> Before this patch, \"expected_downtime\" almost played this role. For\n> example, by monitoring \"expected_downtime\" at the beginning of each\n> iteration can in most cases also reflect the progress of migration\n> system-wise. Said that, \"expected_downtime\" was always calculated based on\n> a bandwidth value that can fluctuate a lot if avail-switchover-bandwidth is\n> not used. This new \"remaining\" field will remove that part of uncertainty\n> for mgmt.\n> \n> With the new field, HMP \"info migrate\" now reports this:\n> \n> (qemu) info migrate\n> Status: active\n> Time (ms): total=12080, setup=14, exp_down=300\n> Remaining (bytes): 1.36 GiB <------------------- newline\n> RAM info:\n> Throughput (Mbps): 840.50\n> Sizes: pagesize=4 KiB, total=4.02 GiB\n> Transfers: transferred=1.18 GiB, remain=1.36 GiB\n> Channels: precopy=1.18 GiB, multifd=0 B, postcopy=0 B\n> Page Types: normal=307923, zero=388148\n> Page Rates (pps): transfer=25660\n> Others: dirty_syncs=1\n> \n> It should be the same value as RAM's remaining report when VFIO is not\n> involved, and it should report more than that when VFIO is involved.\n> \n> Cc: Markus Armbruster \n> Cc: Dr. David Alan Gilbert \n> Signed-off-by: Peter Xu \n> ---\n> qapi/migration.json | 4 ++++\n> migration/migration-hmp-cmds.c | 5 +++++\n> migration/migration.c | 11 +++++++++++\n> 3 files changed, 20 insertions(+)\n> \n\nReviewed-by: Juraj Marcin ","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@legolas.ozlabs.org","Authentication-Results":["legolas.ozlabs.org;\n\tdkim=pass (1024-bit key;\n unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256\n header.s=mimecast20190719 header.b=I/Ut1jS4;\n\tdkim=pass (2048-bit key;\n unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256\n header.s=google header.b=c/WmC2fd;\n\tdkim-atps=neutral","legolas.ozlabs.org;\n spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org\n (client-ip=209.51.188.17; helo=lists.gnu.org;\n envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n receiver=patchwork.ozlabs.org)"],"Received":["from lists.gnu.org (lists1p.gnu.org [209.51.188.17])\n\t(using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits))\n\t(No client certificate requested)\n\tby legolas.ozlabs.org (Postfix) with ESMTPS id 4fs6gv6k8mz1yCv\n\tfor ; Fri, 10 Apr 2026 03:42:03 +1000 (AEST)","from localhost ([::1] helo=lists1p.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.90_1)\n\t(envelope-from )\n\tid 1wAtNu-00060y-KR; Thu, 09 Apr 2026 13:41:44 -0400","from eggs.gnu.org ([2001:470:142:3::10])\n by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)\n (Exim 4.90_1) (envelope-from )\n id 1wAtNm-0005yw-5C\n for qemu-devel@nongnu.org; Thu, 09 Apr 2026 13:41:36 -0400","from us-smtp-delivery-124.mimecast.com ([170.10.129.124])\n by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)\n (Exim 4.90_1) (envelope-from )\n id 1wAtNh-0000wy-SG\n for qemu-devel@nongnu.org; Thu, 09 Apr 2026 13:41:31 -0400","from mail-wr1-f69.google.com (mail-wr1-f69.google.com\n [209.85.221.69]) by relay.mimecast.com with ESMTP with STARTTLS\n (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id\n us-mta-587-cqpDIFKFNhS35YEM7O9YLw-1; Thu, 09 Apr 2026 13:41:26 -0400","by mail-wr1-f69.google.com with SMTP id\n ffacd0b85a97d-43cf5b4dac8so1169309f8f.0\n for ; Thu, 09 Apr 2026 10:41:26 -0700 (PDT)","from fedora (nat-88-212-17-233.antik.sk. [88.212.17.233])\n by smtp.gmail.com with ESMTPSA id\n 5b1f17b1804b1-488d532ed4dsm11300115e9.4.2026.04.09.10.41.23\n (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);\n Thu, 09 Apr 2026 10:41:23 -0700 (PDT)"],"DKIM-Signature":["v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com;\n s=mimecast20190719; t=1775756489;\n h=from:from:reply-to:subject:subject:date:date:message-id:message-id:\n to:to:cc:cc:mime-version:mime-version:content-type:content-type:\n in-reply-to:in-reply-to:references:references;\n bh=0p8aMjfA/pkJTXNKooNCnVbelA3KrsJqJyjArbsCDHg=;\n b=I/Ut1jS4uN/LaSQeUgkUjpTEmPqbRy5+0JEdvBiVkR408BSJG8JnzzMXMG06zugsCrQN3I\n HDLILRarhQ2G+QhZiip1zm4XQCOdMqUUZvChKqdLB1hk7RKJg0HSX+6lga/axJrAUPDaFJ\n 8tsUM6X5yeBhM+WFsp6STgNw08ymGWw=","v=1; a=rsa-sha256; c=relaxed/relaxed;\n d=redhat.com; s=google; t=1775756485; x=1776361285; darn=nongnu.org;\n h=in-reply-to:content-disposition:mime-version:references:message-id\n :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;\n bh=0p8aMjfA/pkJTXNKooNCnVbelA3KrsJqJyjArbsCDHg=;\n b=c/WmC2fdzom3qGoh6s1pi2S3CQGrBLzbEkxpzOh429lE18JPwChKOKeTGcjMc37det\n U3NfVVz5y78sAUoyq1MJ+76NiaGvkbJ0IyOMKmcC70IXtrQgl+F+SJXKXZoVBeRJFUJD\n jl6eiLA/C2k/ZHo7iMktzqxrjkzps1SgzBbUdiMWl1f9LPtJRJnVMtRQl0d5uVMJzMWQ\n dV5ExLp6yFCne3cEal2P5q09RKWDP2a5cE+q+3xxNylt04Jtg72BEyGs9ivTmSRswMxU\n A35g2qbL8D6guagXs+Gb5edgmf6U/PE3fyB1LFMrzSOd2Yxyk/sW1zxmWbJqPoQQGrwM\n uGLg=="],"X-MC-Unique":"cqpDIFKFNhS35YEM7O9YLw-1","X-Mimecast-MFC-AGG-ID":"cqpDIFKFNhS35YEM7O9YLw_1775756485","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n d=1e100.net; s=20251104; t=1775756485; x=1776361285;\n h=in-reply-to:content-disposition:mime-version:references:message-id\n :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc\n :subject:date:message-id:reply-to;\n bh=0p8aMjfA/pkJTXNKooNCnVbelA3KrsJqJyjArbsCDHg=;\n b=b0W0gRAzQ1IBcSxsibrYnC/OxJWoIMuI/fHTzFWNjbMhcrYZcj9XdP9JVVLd5MbFL4\n Vomgz3oPaNpwgnw13ecxfOpsexSrEDhZdY+rFmXiGNrbQzsJvijmcHfYGhRnMGwdgrCP\n GI2gonfhCieNkPu0kW3efzJNjCDqt4EzOQBZYPzBRI7dX2QaYZd//7oZuwIEalbtU0Y0\n IgEOoxjQBmIp9+Va9Lhpb99raq3Df/SflwNVY27Hz+y0azE337q7GOK1WEJGBYuzGyZK\n BbEe2hPjIRiPZ/TybEzfr+PQ4chyRRluvh/P5HtT6TjSLcrT7m0T1oJeGqoXkQjDnPRj\n FbBA==","X-Gm-Message-State":"AOJu0YxFTNy3hgcZhTq+IWxkxGMpFUo/VRakDYZ4vEs+9RHPWuUYH5jS\n zyrXU+W0jFAskANNwPwMgiLn1XG3UBukhj8bbSMtAVyR6I+7NIbEacIK/DsjEhcE2I6wgaLOoxd\n uSDKI6KgjZ/PXQAoOo7rxM6CjjrdnwyxbzEX39oeJHZFU/T1ycVX+TwCO","X-Gm-Gg":"AeBDiesVaDIC5s0MZGJdK9tKKzkEAlLiiQqITn9iXUYNUx6VK7rfNJmpIdSppHDvM6z\n /MdByY5Rrj/gtgVmJdnrzadLq3Xa2U7cgSu+MXopipQa/mZUn2JNOdlxc4emhPbKW7turlqkVkv\n C5CCg5rYfN+UKhJyTpPN0klvXiRc2VKmcg9T1NwrxOou/B+IX+VuDE13JxzT8SiQe0WOEQ0LVwd\n +XBq+eTRONcBfxrUdEvZ+CongYLaxjTPkPNOLtzZRS/JcaTYXtVgkw7drx/4lTU6MJcxXN/ObId\n 4a/n4KKGSvS25ru6wJW23Txry440jm8fCBRrvqNRC+Y8GXflwSHcqHbeicFLO6pyE38y+tws7Zl\n cgtX353mbrkPuzUXRT0SHZK3BVMlZrl1VbjWvLhM=","X-Received":["by 2002:a05:600c:42d1:b0:488:a62d:76d7 with SMTP id\n 5b1f17b1804b1-488a62d7745mr142449155e9.1.1775756485284;\n Thu, 09 Apr 2026 10:41:25 -0700 (PDT)","by 2002:a05:600c:42d1:b0:488:a62d:76d7 with SMTP id\n 5b1f17b1804b1-488a62d7745mr142448955e9.1.1775756484789;\n Thu, 09 Apr 2026 10:41:24 -0700 (PDT)"],"Date":"Thu, 9 Apr 2026 19:41:21 +0200","From":"Juraj Marcin ","To":"Peter Xu ","Cc":"qemu-devel@nongnu.org,\n \"Maciej S . Szmigiero\" , Daniel P\n\t=?utf-8?b?LiBCZXJyYW5nw6k=?= ,\n Zhiyi Guo , Prasad Pandit ,\n Avihai Horon , Kirti Wankhede ,\n\t=?utf-8?q?C=C3=A9dric?= Le Goater ,\n Fabiano Rosas , Joao Martins ,\n Markus Armbruster , Alex Williamson ,\n \"Dr. David Alan Gilbert\" ","Subject":"Re: [PATCH 13/14] migration/qapi: Introduce system-wise \"remaining\"\n reports","Message-ID":"","References":"<20260408165559.157108-1-peterx@redhat.com>\n <20260408165559.157108-14-peterx@redhat.com>","MIME-Version":"1.0","Content-Type":"text/plain; charset=us-ascii","Content-Disposition":"inline","In-Reply-To":"<20260408165559.157108-14-peterx@redhat.com>","Received-SPF":"pass client-ip=170.10.129.124; envelope-from=jmarcin@redhat.com;\n helo=us-smtp-delivery-124.mimecast.com","X-Spam_score_int":"-25","X-Spam_score":"-2.6","X-Spam_bar":"--","X-Spam_report":"(-2.6 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.54,\n DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1,\n RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001,\n RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001,\n SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no","X-Spam_action":"no action","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"qemu development ","List-Unsubscribe":",\n ","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n ","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org"}},{"id":3675525,"web_url":"http://patchwork.ozlabs.org/comment/3675525/","msgid":"","list_archive_url":null,"date":"2026-04-09T21:48:57","subject":"Re: [PATCH 13/14] migration/qapi: Introduce system-wise \"remaining\"\n reports","submitter":{"id":86099,"url":"http://patchwork.ozlabs.org/api/people/86099/","name":"Dr. David Alan Gilbert","email":"dave@treblig.org"},"content":"* Peter Xu (peterx@redhat.com) wrote:\n> Currently, mgmt can only query for remaining RAM, not system-wise remaining\n> data. It was not a problem before, because for a very long time RAM was\n> the only part that matters.\n> \n> After VFIO migrations landed upstream, it may not be true anymore\n> especially considering that there can be GPU devices that contain GBs of\n> device states.\n> \n> Add a new \"remaining\" field in query-migrate results, reflecting\n> system-wise remaining data, which will include everything (e.g. VFIO).\n\nOf course you realise the next thing people will ask for is being able\nto ask *which* vfio device is the one that's busy.\n\nReviewed-by: Dr. David Alan Gilbert \n\n> This information will be useful for mgmt to implement generic way of stall\n> detection that covers all system resources. Say, when system remaining\n> data does not decrease anymore for a relatively long period of time, then\n> it may mean that there is a challenge of converging, so mgmt can act based\n> on how this value changes over time (especially if sampled after each\n> migration iteration).\n> \n> Before this patch, \"expected_downtime\" almost played this role. For\n> example, by monitoring \"expected_downtime\" at the beginning of each\n> iteration can in most cases also reflect the progress of migration\n> system-wise. Said that, \"expected_downtime\" was always calculated based on\n> a bandwidth value that can fluctuate a lot if avail-switchover-bandwidth is\n> not used. This new \"remaining\" field will remove that part of uncertainty\n> for mgmt.\n> \n> With the new field, HMP \"info migrate\" now reports this:\n> \n> (qemu) info migrate\n> Status: active\n> Time (ms): total=12080, setup=14, exp_down=300\n> Remaining (bytes): 1.36 GiB <------------------- newline\n> RAM info:\n> Throughput (Mbps): 840.50\n> Sizes: pagesize=4 KiB, total=4.02 GiB\n> Transfers: transferred=1.18 GiB, remain=1.36 GiB\n> Channels: precopy=1.18 GiB, multifd=0 B, postcopy=0 B\n> Page Types: normal=307923, zero=388148\n> Page Rates (pps): transfer=25660\n> Others: dirty_syncs=1\n> \n> It should be the same value as RAM's remaining report when VFIO is not\n> involved, and it should report more than that when VFIO is involved.\n> \n> Cc: Markus Armbruster \n> Cc: Dr. David Alan Gilbert \n> Signed-off-by: Peter Xu \n> ---\n> qapi/migration.json | 4 ++++\n> migration/migration-hmp-cmds.c | 5 +++++\n> migration/migration.c | 11 +++++++++++\n> 3 files changed, 20 insertions(+)\n> \n> diff --git a/qapi/migration.json b/qapi/migration.json\n> index e3ad3f0604..a6e24b5685 100644\n> --- a/qapi/migration.json\n> +++ b/qapi/migration.json\n> @@ -300,6 +300,9 @@\n> # average memory load of the virtual CPU indirectly. Note that\n> # zero means guest doesn't dirty memory. (Since 8.1)\n> #\n> +# @remaining: amount of bytes remaining to be migrated system-wise,\n> +# includes both RAM and all devices (like VFIO). (Since 11.1)\n> +#\n> # Features:\n> #\n> # @unstable: Members @postcopy-latency, @postcopy-vcpu-latency,\n> @@ -310,6 +313,7 @@\n> ##\n> { 'struct': 'MigrationInfo',\n> 'data': {'*status': 'MigrationStatus', '*ram': 'MigrationRAMStats',\n> + '*remaining': 'uint64',\n> '*vfio': 'VfioStats',\n> '*xbzrle-cache': 'XBZRLECacheStats',\n> '*total-time': 'int',\n> diff --git a/migration/migration-hmp-cmds.c b/migration/migration-hmp-cmds.c\n> index 0a193b8f54..721c211086 100644\n> --- a/migration/migration-hmp-cmds.c\n> +++ b/migration/migration-hmp-cmds.c\n> @@ -178,6 +178,11 @@ void hmp_info_migrate(Monitor *mon, const QDict *qdict)\n> }\n> }\n> \n> + if (info->has_remaining) {\n> + g_autofree char *remaining = size_to_str(info->remaining);\n> + monitor_printf(mon, \"Remaining (bytes): \\t%s\\n\", remaining);\n> + }\n> +\n> if (info->has_socket_address) {\n> SocketAddressList *addr;\n> \n> diff --git a/migration/migration.c b/migration/migration.c\n> index 4010e5dcf5..c2aa145106 100644\n> --- a/migration/migration.c\n> +++ b/migration/migration.c\n> @@ -1076,6 +1076,16 @@ static void populate_time_info(MigrationInfo *info, MigrationState *s)\n> }\n> }\n> \n> +static void populate_global_info(MigrationInfo *info, MigrationState *s)\n> +{\n> + MigPendingData data = { };\n> +\n> + qemu_savevm_query_pending(&data, false);\n> +\n> + info->has_remaining = true;\n> + info->remaining = data.total_bytes;\n> +}\n> +\n> static void populate_ram_info(MigrationInfo *info, MigrationState *s)\n> {\n> size_t page_size = qemu_target_page_size();\n> @@ -1177,6 +1187,7 @@ static void fill_source_migration_info(MigrationInfo *info)\n> /* TODO add some postcopy stats */\n> populate_time_info(info, s);\n> populate_ram_info(info, s);\n> + populate_global_info(info, s);\n> migration_populate_vfio_info(info);\n> break;\n> case MIGRATION_STATUS_COLO:\n> -- \n> 2.53.0\n>","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@legolas.ozlabs.org","Authentication-Results":["legolas.ozlabs.org;\n\tdkim=pass (2048-bit key;\n unprotected) header.d=treblig.org header.i=@treblig.org header.a=rsa-sha256\n header.s=bytemarkmx header.b=IKPOm9jv;\n\tdkim-atps=neutral","legolas.ozlabs.org;\n spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org\n (client-ip=209.51.188.17; helo=lists.gnu.org;\n envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n receiver=patchwork.ozlabs.org)"],"Received":["from lists.gnu.org (lists1p.gnu.org [209.51.188.17])\n\t(using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits))\n\t(No client certificate requested)\n\tby legolas.ozlabs.org (Postfix) with ESMTPS id 4fsD9t6cN1z1y05\n\tfor ; Fri, 10 Apr 2026 07:49:53 +1000 (AEST)","from localhost ([::1] helo=lists1p.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.90_1)\n\t(envelope-from )\n\tid 1wAxFL-0003GU-4s; Thu, 09 Apr 2026 17:49:07 -0400","from eggs.gnu.org ([2001:470:142:3::10])\n by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)\n (Exim 4.90_1) (envelope-from ) id 1wAxFJ-0003GE-DA\n for qemu-devel@nongnu.org; Thu, 09 Apr 2026 17:49:05 -0400","from mx.treblig.org ([2a00:1098:5b::1])\n by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)\n (Exim 4.90_1) (envelope-from ) id 1wAxFG-0001ZF-Ve\n for qemu-devel@nongnu.org; Thu, 09 Apr 2026 17:49:05 -0400","from dg by mx.treblig.org with local (Exim 4.98.2)\n (envelope-from ) id 1wAxFB-00000007HfO-3lz4;\n Thu, 09 Apr 2026 21:48:57 +0000"],"DKIM-Signature":"v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=treblig.org\n ; s=bytemarkmx;\n h=Content-Type:MIME-Version:Message-ID:Subject:From:Date:From\n :Subject; bh=I3+zygPDdrYt1oWBgZV98MSYzKehFQGf3kqAyk4J2Kw=; b=IKPOm9jvQ4NvrCkt\n 0aFMPpQbnaDJ/IgKjwY2/w3+y7zBmG/lMokwer3Zh50rD5ZuKxr3lXHljyqTE4ytWevDFjvk1kdBt\n BOGqyK9Hdhldc9OIciU8Khty6iuAjq34bsHX35279EyJ1iTt+DgXhk0nuoNTaW3Zjo4OuZQg80pQr\n 3z/OrGX6QCYKF7DnE/jz1FAd7x4DXfUlxZ4hW/zq001/POY33rloygk6VeBntgugVDMVhfA/l1OOi\n hvGhbGvlldup57SRSjDSA3+e3q5kecV2F+AQmTLXJVr4Lf2Bm8c0BrpYyBu3oOwgOPAAL+IoRKGFp\n gQoy1L0IJzuwefd5HQ==;","Date":"Thu, 9 Apr 2026 21:48:57 +0000","From":"\"Dr. David Alan Gilbert\" ","To":"Peter Xu ","Cc":"qemu-devel@nongnu.org,\n \"Maciej S . Szmigiero\" , Daniel P\n\t=?iso-8859-1?q?=2E_Berrang=E9?= ,\n Zhiyi Guo , Juraj Marcin ,\n Prasad Pandit , Avihai Horon ,\n Kirti Wankhede ,\n =?iso-8859-1?q?C=E9dric?= Le Goater ,\n Fabiano Rosas , Joao Martins ,\n Markus Armbruster , Alex Williamson ","Subject":"Re: [PATCH 13/14] migration/qapi: Introduce system-wise \"remaining\"\n reports","Message-ID":"","References":"<20260408165559.157108-1-peterx@redhat.com>\n <20260408165559.157108-14-peterx@redhat.com>","MIME-Version":"1.0","Content-Type":"text/plain; charset=iso-8859-1","Content-Disposition":"inline","In-Reply-To":"<20260408165559.157108-14-peterx@redhat.com>","X-Chocolate":"70 percent or better cocoa solids preferably","X-Operating-System":"Linux/6.12.74+deb13+1-amd64 (x86_64)","X-Uptime":"21:48:18 up 27 days, 23:01, 2 users, load average: 0.02, 0.03,\n 0.00","User-Agent":"Mutt/2.2.13 (2024-03-09)","Received-SPF":"pass client-ip=2a00:1098:5b::1; envelope-from=dg@treblig.org;\n helo=mx.treblig.org","X-Spam_score_int":"-20","X-Spam_score":"-2.1","X-Spam_bar":"--","X-Spam_report":"(-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1,\n DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001,\n SPF_PASS=-0.001 autolearn=ham autolearn_force=no","X-Spam_action":"no action","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"qemu development ","List-Unsubscribe":",\n ","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n ","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org"}},{"id":3675535,"web_url":"http://patchwork.ozlabs.org/comment/3675535/","msgid":"<87v7dz4vx9.fsf@suse.de>","list_archive_url":null,"date":"2026-04-09T22:21:22","subject":"Re: [PATCH 13/14] migration/qapi: Introduce system-wise \"remaining\"\n reports","submitter":{"id":85343,"url":"http://patchwork.ozlabs.org/api/people/85343/","name":"Fabiano Rosas","email":"farosas@suse.de"},"content":"Peter Xu writes:\n\n> Currently, mgmt can only query for remaining RAM, not system-wise remaining\n> data. It was not a problem before, because for a very long time RAM was\n> the only part that matters.\n>\n> After VFIO migrations landed upstream, it may not be true anymore\n> especially considering that there can be GPU devices that contain GBs of\n> device states.\n>\n> Add a new \"remaining\" field in query-migrate results, reflecting\n> system-wise remaining data, which will include everything (e.g. VFIO).\n>\n> This information will be useful for mgmt to implement generic way of stall\n> detection that covers all system resources. Say, when system remaining\n> data does not decrease anymore for a relatively long period of time, then\n> it may mean that there is a challenge of converging, so mgmt can act based\n> on how this value changes over time (especially if sampled after each\n> migration iteration).\n>\n> Before this patch, \"expected_downtime\" almost played this role. For\n> example, by monitoring \"expected_downtime\" at the beginning of each\n> iteration can in most cases also reflect the progress of migration\n> system-wise. Said that, \"expected_downtime\" was always calculated based on\n> a bandwidth value that can fluctuate a lot if avail-switchover-bandwidth is\n> not used. This new \"remaining\" field will remove that part of uncertainty\n> for mgmt.\n>\n> With the new field, HMP \"info migrate\" now reports this:\n>\n> (qemu) info migrate\n> Status: active\n> Time (ms): total=12080, setup=14, exp_down=300\n> Remaining (bytes): 1.36 GiB <------------------- newline\n\nEither bytes or GiB. Better to simply remove the \"(bytes)\" string.\n\n> RAM info:\n> Throughput (Mbps): 840.50\n> Sizes: pagesize=4 KiB, total=4.02 GiB\n> Transfers: transferred=1.18 GiB, remain=1.36 GiB\n> Channels: precopy=1.18 GiB, multifd=0 B, postcopy=0 B\n> Page Types: normal=307923, zero=388148\n> Page Rates (pps): transfer=25660\n> Others: dirty_syncs=1\n>\n> It should be the same value as RAM's remaining report when VFIO is not\n> involved, and it should report more than that when VFIO is involved.\n>\n> Cc: Markus Armbruster \n> Cc: Dr. David Alan Gilbert \n> Signed-off-by: Peter Xu \n> ---\n> qapi/migration.json | 4 ++++\n> migration/migration-hmp-cmds.c | 5 +++++\n> migration/migration.c | 11 +++++++++++\n> 3 files changed, 20 insertions(+)\n>\n> diff --git a/qapi/migration.json b/qapi/migration.json\n> index e3ad3f0604..a6e24b5685 100644\n> --- a/qapi/migration.json\n> +++ b/qapi/migration.json\n> @@ -300,6 +300,9 @@\n> # average memory load of the virtual CPU indirectly. Note that\n> # zero means guest doesn't dirty memory. (Since 8.1)\n> #\n> +# @remaining: amount of bytes remaining to be migrated system-wise,\n> +# includes both RAM and all devices (like VFIO). (Since 11.1)\n> +#\n> # Features:\n> #\n> # @unstable: Members @postcopy-latency, @postcopy-vcpu-latency,\n> @@ -310,6 +313,7 @@\n> ##\n> { 'struct': 'MigrationInfo',\n> 'data': {'*status': 'MigrationStatus', '*ram': 'MigrationRAMStats',\n> + '*remaining': 'uint64',\n> '*vfio': 'VfioStats',\n> '*xbzrle-cache': 'XBZRLECacheStats',\n> '*total-time': 'int',\n> diff --git a/migration/migration-hmp-cmds.c b/migration/migration-hmp-cmds.c\n> index 0a193b8f54..721c211086 100644\n> --- a/migration/migration-hmp-cmds.c\n> +++ b/migration/migration-hmp-cmds.c\n> @@ -178,6 +178,11 @@ void hmp_info_migrate(Monitor *mon, const QDict *qdict)\n> }\n> }\n> \n> + if (info->has_remaining) {\n> + g_autofree char *remaining = size_to_str(info->remaining);\n> + monitor_printf(mon, \"Remaining (bytes): \\t%s\\n\", remaining);\n> + }\n> +\n> if (info->has_socket_address) {\n> SocketAddressList *addr;\n> \n> diff --git a/migration/migration.c b/migration/migration.c\n> index 4010e5dcf5..c2aa145106 100644\n> --- a/migration/migration.c\n> +++ b/migration/migration.c\n> @@ -1076,6 +1076,16 @@ static void populate_time_info(MigrationInfo *info, MigrationState *s)\n> }\n> }\n> \n> +static void populate_global_info(MigrationInfo *info, MigrationState *s)\n> +{\n> + MigPendingData data = { };\n> +\n> + qemu_savevm_query_pending(&data, false);\n> +\n> + info->has_remaining = true;\n> + info->remaining = data.total_bytes;\n> +}\n> +\n> static void populate_ram_info(MigrationInfo *info, MigrationState *s)\n> {\n> size_t page_size = qemu_target_page_size();\n> @@ -1177,6 +1187,7 @@ static void fill_source_migration_info(MigrationInfo *info)\n> /* TODO add some postcopy stats */\n> populate_time_info(info, s);\n> populate_ram_info(info, s);\n> + populate_global_info(info, s);\n> migration_populate_vfio_info(info);\n> break;\n> case MIGRATION_STATUS_COLO:","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@legolas.ozlabs.org","Authentication-Results":["legolas.ozlabs.org;\n\tdkim=pass (1024-bit key;\n unprotected) header.d=suse.de header.i=@suse.de header.a=rsa-sha256\n header.s=susede2_rsa header.b=EX3c+BmG;\n\tdkim=pass header.d=suse.de header.i=@suse.de header.a=ed25519-sha256\n header.s=susede2_ed25519 header.b=oPr2uhUP;\n\tdkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de\n header.a=rsa-sha256 header.s=susede2_rsa header.b=EX3c+BmG;\n\tdkim=neutral header.d=suse.de header.i=@suse.de header.a=ed25519-sha256\n header.s=susede2_ed25519 header.b=oPr2uhUP;\n\tdkim-atps=neutral","legolas.ozlabs.org;\n spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org\n (client-ip=209.51.188.17; helo=lists.gnu.org;\n envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n receiver=patchwork.ozlabs.org)","smtp-out2.suse.de;\n\tnone"],"Received":["from lists.gnu.org (lists1p.gnu.org [209.51.188.17])\n\t(using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits))\n\t(No client certificate requested)\n\tby legolas.ozlabs.org (Postfix) with ESMTPS id 4fsDtq0KS0z1yCv\n\tfor ; Fri, 10 Apr 2026 08:21:53 +1000 (AEST)","from localhost ([::1] helo=lists1p.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.90_1)\n\t(envelope-from )\n\tid 1wAxkh-0008PF-E5; Thu, 09 Apr 2026 18:21:31 -0400","from eggs.gnu.org ([2001:470:142:3::10])\n by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)\n (Exim 4.90_1) (envelope-from ) id 1wAxke-0008Oo-Ij\n for qemu-devel@nongnu.org; Thu, 09 Apr 2026 18:21:28 -0400","from smtp-out2.suse.de ([2a07:de40:b251:101:10:150:64:2])\n by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128)\n (Exim 4.90_1) (envelope-from ) id 1wAxkc-0003ns-LD\n for qemu-devel@nongnu.org; Thu, 09 Apr 2026 18:21:28 -0400","from imap1.dmz-prg2.suse.org (unknown [10.150.64.97])\n (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)\n key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest\n SHA256)\n (No client certificate requested)\n by smtp-out2.suse.de (Postfix) with ESMTPS id 3781F5BCF6;\n Thu, 9 Apr 2026 22:21:25 +0000 (UTC)","from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])\n (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)\n key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest\n SHA256)\n (No client certificate requested)\n by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id B93ED4A0B3;\n Thu, 9 Apr 2026 22:21:24 +0000 (UTC)","from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])\n by imap1.dmz-prg2.suse.org with ESMTPSA id 2q5+IGQm2Gk4QgAAD6G6ig\n (envelope-from ); Thu, 09 Apr 2026 22:21:24 +0000"],"DKIM-Signature":["v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de;\n s=susede2_rsa;\n t=1775773285;\n h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:\n mime-version:mime-version:content-type:content-type:\n in-reply-to:in-reply-to:references:references;\n bh=LY3aMi32xfMGySyi7sa6G9Bibaq89EH0Jw9iWgTQ/WY=;\n b=EX3c+BmGSLR0t+FuW9TlPHNgiM8wI22k73LNuSnZrXcXnTQTsc1n+9bNT4wwKNJKLCGWK6\n j17QOQVkN/lTPGY8+CE9hoDMdgJnM+Od1cnEFqcoJn5fX7okYJ/7FhL2wtJBtzo4G/1hlT\n URR+DdZp2/wJ+qO//SZmdaFOaqOACTg=","v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de;\n s=susede2_ed25519; t=1775773285;\n h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:\n mime-version:mime-version:content-type:content-type:\n in-reply-to:in-reply-to:references:references;\n bh=LY3aMi32xfMGySyi7sa6G9Bibaq89EH0Jw9iWgTQ/WY=;\n b=oPr2uhUPXi/b8vVaKNRCFtDNoNJKvY/8fiu3vCw6fc3QYpzA5igNzDN4dljyAzdML7cSKZ\n S5H9YrlnWC95ZNCw==","v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de;\n s=susede2_rsa;\n t=1775773285;\n h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:\n mime-version:mime-version:content-type:content-type:\n in-reply-to:in-reply-to:references:references;\n bh=LY3aMi32xfMGySyi7sa6G9Bibaq89EH0Jw9iWgTQ/WY=;\n b=EX3c+BmGSLR0t+FuW9TlPHNgiM8wI22k73LNuSnZrXcXnTQTsc1n+9bNT4wwKNJKLCGWK6\n j17QOQVkN/lTPGY8+CE9hoDMdgJnM+Od1cnEFqcoJn5fX7okYJ/7FhL2wtJBtzo4G/1hlT\n URR+DdZp2/wJ+qO//SZmdaFOaqOACTg=","v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de;\n s=susede2_ed25519; t=1775773285;\n h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:\n mime-version:mime-version:content-type:content-type:\n in-reply-to:in-reply-to:references:references;\n bh=LY3aMi32xfMGySyi7sa6G9Bibaq89EH0Jw9iWgTQ/WY=;\n b=oPr2uhUPXi/b8vVaKNRCFtDNoNJKvY/8fiu3vCw6fc3QYpzA5igNzDN4dljyAzdML7cSKZ\n S5H9YrlnWC95ZNCw=="],"From":"Fabiano Rosas ","To":"Peter Xu , qemu-devel@nongnu.org","Cc":"\"Maciej S . Szmigiero\" , =?utf-8?q?Daniel_P_?=\n\t=?utf-8?q?=2E_Berrang=C3=A9?= ,\n Zhiyi Guo , Juraj Marcin ,\n Peter Xu , Prasad Pandit ,\n Avihai Horon , Kirti Wankhede ,\n\t=?utf-8?q?C=C3=A9dric?= Le Goater ,\n Joao Martins ,\n Markus Armbruster , Alex Williamson ,\n \"Dr. David Alan Gilbert\" ","Subject":"Re: [PATCH 13/14] migration/qapi: Introduce system-wise \"remaining\"\n reports","In-Reply-To":"<20260408165559.157108-14-peterx@redhat.com>","References":"<20260408165559.157108-1-peterx@redhat.com>\n <20260408165559.157108-14-peterx@redhat.com>","Date":"Thu, 09 Apr 2026 19:21:22 -0300","Message-ID":"<87v7dz4vx9.fsf@suse.de>","MIME-Version":"1.0","Content-Type":"text/plain","X-Spamd-Result":"default: False [-4.30 / 50.00]; BAYES_HAM(-3.00)[100.00%];\n NEURAL_HAM_LONG(-1.00)[-1.000];\n NEURAL_HAM_SHORT(-0.20)[-1.000]; MIME_GOOD(-0.10)[text/plain];\n DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519];\n FUZZY_RATELIMITED(0.00)[rspamd.com]; ARC_NA(0.00)[];\n MIME_TRACE(0.00)[0:+]; TO_MATCH_ENVRCPT_ALL(0.00)[];\n RCPT_COUNT_TWELVE(0.00)[15]; FROM_HAS_DN(0.00)[];\n RCVD_TLS_ALL(0.00)[]; MISSING_XM_UA(0.00)[];\n FROM_EQ_ENVFROM(0.00)[]; TO_DN_SOME(0.00)[];\n RCVD_COUNT_TWO(0.00)[2]; RCVD_VIA_SMTP_AUTH(0.00)[];\n MID_RHS_MATCH_FROM(0.00)[];\n DBL_BLOCKED_OPENRESOLVER(0.00)[suse.de:mid]","X-Spam-Score":"-4.30","Received-SPF":"pass client-ip=2a07:de40:b251:101:10:150:64:2;\n envelope-from=farosas@suse.de; helo=smtp-out2.suse.de","X-Spam_score_int":"-20","X-Spam_score":"-2.1","X-Spam_bar":"--","X-Spam_report":"(-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1,\n DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001,\n SPF_PASS=-0.001 autolearn=ham autolearn_force=no","X-Spam_action":"no action","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"qemu development ","List-Unsubscribe":",\n ","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n ","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org"}}]