{"id":2221045,"url":"http://patchwork.ozlabs.org/api/1.0/patches/2221045/?format=json","project":{"id":14,"url":"http://patchwork.ozlabs.org/api/1.0/projects/14/?format=json","name":"QEMU Development","link_name":"qemu-devel","list_id":"qemu-devel.nongnu.org","list_email":"qemu-devel@nongnu.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20260408165559.157108-14-peterx@redhat.com>","date":"2026-04-08T16:55:57","name":"[13/14] migration/qapi: Introduce system-wise \"remaining\" reports","commit_ref":null,"pull_url":null,"state":"new","archived":false,"hash":"00f9cd3cc9f0f3378ea583e7dc615a7dc02497e3","submitter":{"id":67717,"url":"http://patchwork.ozlabs.org/api/1.0/people/67717/?format=json","name":"Peter Xu","email":"peterx@redhat.com"},"delegate":null,"mbox":"http://patchwork.ozlabs.org/project/qemu-devel/patch/20260408165559.157108-14-peterx@redhat.com/mbox/","series":[{"id":499176,"url":"http://patchwork.ozlabs.org/api/1.0/series/499176/?format=json","date":"2026-04-08T16:55:55","name":"migration/vfio: Fix a few issues on API misuse or statistic reports","version":1,"mbox":"http://patchwork.ozlabs.org/series/499176/mbox/"}],"check":"pending","checks":"http://patchwork.ozlabs.org/api/patches/2221045/checks/","tags":{},"headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@legolas.ozlabs.org","Authentication-Results":["legolas.ozlabs.org;\n\tdkim=pass (1024-bit key;\n unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256\n header.s=mimecast20190719 header.b=CZVWN+iK;\n\tdkim=pass (2048-bit key;\n unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256\n header.s=google header.b=tXmLDhgU;\n\tdkim-atps=neutral","legolas.ozlabs.org;\n spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org\n (client-ip=209.51.188.17; helo=lists.gnu.org;\n envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n receiver=patchwork.ozlabs.org)"],"Received":["from lists.gnu.org (unknown [209.51.188.17])\n\t(using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits))\n\t(No client certificate requested)\n\tby legolas.ozlabs.org (Postfix) with ESMTPS id 4frWrH2tjRz1xy1\n\tfor ; Thu, 09 Apr 2026 04:32:15 +1000 (AEST)","from localhost ([::1] helo=lists1p.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.90_1)\n\t(envelope-from )\n\tid 1wAXh4-0007Jm-F0; Wed, 08 Apr 2026 14:32:02 -0400","from eggs.gnu.org ([2001:470:142:3::10])\n by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)\n (Exim 4.90_1) (envelope-from ) id 1wAXgq-00075k-CE\n for qemu-devel@nongnu.org; Wed, 08 Apr 2026 14:31:51 -0400","from us-smtp-delivery-124.mimecast.com ([170.10.133.124])\n by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)\n (Exim 4.90_1) (envelope-from ) id 1wAWCW-00029k-LG\n for qemu-devel@nongnu.org; Wed, 08 Apr 2026 12:56:26 -0400","from mail-qt1-f199.google.com (mail-qt1-f199.google.com\n [209.85.160.199]) by relay.mimecast.com with ESMTP with STARTTLS\n (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id\n us-mta-168-o24luCgIMiCxOCtsOdYteA-1; Wed, 08 Apr 2026 12:56:22 -0400","by mail-qt1-f199.google.com with SMTP id\n d75a77b69052e-50da31af14cso2649191cf.1\n for ; Wed, 08 Apr 2026 09:56:22 -0700 (PDT)","from x1.com ([142.189.10.167]) by smtp.gmail.com with ESMTPSA id\n d75a77b69052e-50d712c2617sm130491901cf.31.2026.04.08.09.56.19\n (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);\n Wed, 08 Apr 2026 09:56:20 -0700 (PDT)"],"DKIM-Signature":["v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com;\n s=mimecast20190719; t=1775667384;\n h=from:from:reply-to:subject:subject:date:date:message-id:message-id:\n to:to:cc:cc:mime-version:mime-version:\n content-transfer-encoding:content-transfer-encoding:\n in-reply-to:in-reply-to:references:references;\n bh=gbrwSMWX65NN2qFtsFxD677N8GbRpenQYWl5sMfBjrg=;\n b=CZVWN+iKXM0zMljnBtIJpacfAY9v9kMfqe8t3Viww9hrEf8DPbFHz24fTlUIOiifjWEePK\n /85QzvZVe714kcwPcll8232Ok2pMUQndoaiKMiN61ZrxVhBO1d7k5p62of4Y8amrPkQtdA\n TTcBvpR1mkfRPOR0Er8uY+VVLKUSlOU=","v=1; a=rsa-sha256; c=relaxed/relaxed;\n d=redhat.com; s=google; t=1775667382; x=1776272182; darn=nongnu.org;\n h=content-transfer-encoding:mime-version:references:in-reply-to\n :message-id:date:subject:cc:to:from:from:to:cc:subject:date\n :message-id:reply-to;\n bh=gbrwSMWX65NN2qFtsFxD677N8GbRpenQYWl5sMfBjrg=;\n b=tXmLDhgU5LT3gMH0iZPzlpfuteFlDjKEzBHSSlX9z380p7pvZuyoyRlkFoadOXZULl\n B6xxKIFTnWN9T+svvDqUrmc6L7rl/mcHhwaEm4bHT6HBscaYd9x5zyXSWhqJG/duM45S\n If45oDHPHV/jbcsmxRK05xZVituxUXR1al0UwN/Kcs9R1AT/nUcL9G3tgJfc6IFXuJKY\n DgkU6VP3Shenl/IdsnVZvJs4qmhI4LaECWbkfkQSpeVdBikzgMgHVHdBvdxgGBjPCTV2\n MNEHkqq548q9a/afF3bL+p/hz823veBiou39nOAFe4yC6J7MLXnQVGy4Q5WE76bxxny5\n hzgw=="],"X-MC-Unique":"o24luCgIMiCxOCtsOdYteA-1","X-Mimecast-MFC-AGG-ID":"o24luCgIMiCxOCtsOdYteA_1775667382","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n d=1e100.net; s=20251104; t=1775667382; x=1776272182;\n h=content-transfer-encoding:mime-version:references:in-reply-to\n :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from\n :to:cc:subject:date:message-id:reply-to;\n bh=gbrwSMWX65NN2qFtsFxD677N8GbRpenQYWl5sMfBjrg=;\n b=DUVnBAkL6CLVDURKEjJoU17EzQoh8LXzzaxRNlZYqtJ1yvKxDC0yGfO751cYs7om6N\n q2A/PQKBplAOXuIlViPg2E6GkuxOl8NsHUi8wIVgn3AZbZ+bsjdAsNymQlNPTCU3mvaH\n 9VGzub+tCAdqrFRm3q2jIBShtD04gmx6WPpcFVs8nS+iN3ErETlfI6yrvALaO2uJd0lS\n 4irgked5JaqxyG8bRmqj7J8NjfVeoStHMSKieA7/k3J/NiqI+rjbcWKrU4+EHUh691VG\n NxRGrXnm0JRPDpj3b/4+MSGS6kLhv+edHYDl82NnKfSkfHUY1dBvt1EK8ZI7ac50sRzH\n 80DA==","X-Gm-Message-State":"AOJu0YywdTP4axNgJgrqmC0gsW3n2gvqQXeEyq5dk6LQLHy8ihBVCA1H\n 7MsoQImfKHiPDpNqeU4V1cMBaaxWUCNfF31wwWfigttC0KzRbXQllgjS8xfw6GrGMbX9V1qnc+2\n gnlaVpibZg+hDG93r6NIAI5D9bnrGB9nXTAbe7A/dtKfbmdnFaVtgwqcCmfIeSt0rH0eAzP3JcK\n yh4twNKAAiakXH5CH2RjmQybrYLqkdOXRHkUmRfg==","X-Gm-Gg":"AeBDieuS8PiDxi/ozdD51Sz7tb3XYj6A99OvO4yK7ozm0E/xzHfpEfmimtM389BNyD8\n Vd4jXSlteec1FN1oKKellQifHbwXmI/HbDPbVz26P3lsBEIIkx1I3/F/jOZArbw343m6dw1jUhI\n hzC9THVxVA2e7dxCEQwEFLMiwYzAGydV3qtauEa8kmTKXxSpTYWQsond7odWFMAbrfCTmCgfwVK\n fK6Ntz8QhXYljOV/hTWqzPrGio3x7Nx4dVMp9EwffFmZjTvKMHZ5uGxG4uXqrTxQM+RsBvLjvMu\n 8FPeGS1FWtIIRJvc9km0Pji20OFWYXfYoqv1RHaoSdU2zJRpWIeFLmqgKJWV5lWfKJFYd0ks2sR\n +e2Tzw6W0yWsZ5GX7U5zQ/11U7EzTOR7c0VdS1rc/5Vzm","X-Received":["by 2002:a05:622a:124b:b0:50d:83de:4933 with SMTP id\n d75a77b69052e-50d83de50cfmr229016051cf.20.1775667381673;\n Wed, 08 Apr 2026 09:56:21 -0700 (PDT)","by 2002:a05:622a:124b:b0:50d:83de:4933 with SMTP id\n d75a77b69052e-50d83de50cfmr229015211cf.20.1775667380973;\n Wed, 08 Apr 2026 09:56:20 -0700 (PDT)"],"From":"Peter Xu ","To":"qemu-devel@nongnu.org","Cc":"\"Maciej S . Szmigiero\" , =?utf-8?q?Daniel_P_?=\n\t=?utf-8?q?=2E_Berrang=C3=A9?= ,\n Zhiyi Guo , Juraj Marcin ,\n Peter Xu , Prasad Pandit ,\n Avihai Horon , Kirti Wankhede ,\n\t=?utf-8?q?C=C3=A9dric_Le_Goater?= ,\n Fabiano Rosas , Joao Martins ,\n Markus Armbruster , Alex Williamson ,\n \"Dr. David Alan Gilbert\" ","Subject":"[PATCH 13/14] migration/qapi: Introduce system-wise \"remaining\"\n reports","Date":"Wed, 8 Apr 2026 12:55:57 -0400","Message-ID":"<20260408165559.157108-14-peterx@redhat.com>","X-Mailer":"git-send-email 2.53.0","In-Reply-To":"<20260408165559.157108-1-peterx@redhat.com>","References":"<20260408165559.157108-1-peterx@redhat.com>","MIME-Version":"1.0","Content-Transfer-Encoding":"8bit","Received-SPF":"pass client-ip=170.10.133.124; envelope-from=peterx@redhat.com;\n helo=us-smtp-delivery-124.mimecast.com","X-Spam_score_int":"-25","X-Spam_score":"-2.6","X-Spam_bar":"--","X-Spam_report":"(-2.6 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.54,\n DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1,\n RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=0.001,\n RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001,\n SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no","X-Spam_action":"no action","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"qemu development ","List-Unsubscribe":",\n ","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n ","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org"},"content":"Currently, mgmt can only query for remaining RAM, not system-wise remaining\ndata. It was not a problem before, because for a very long time RAM was\nthe only part that matters.\n\nAfter VFIO migrations landed upstream, it may not be true anymore\nespecially considering that there can be GPU devices that contain GBs of\ndevice states.\n\nAdd a new \"remaining\" field in query-migrate results, reflecting\nsystem-wise remaining data, which will include everything (e.g. VFIO).\n\nThis information will be useful for mgmt to implement generic way of stall\ndetection that covers all system resources. Say, when system remaining\ndata does not decrease anymore for a relatively long period of time, then\nit may mean that there is a challenge of converging, so mgmt can act based\non how this value changes over time (especially if sampled after each\nmigration iteration).\n\nBefore this patch, \"expected_downtime\" almost played this role. For\nexample, by monitoring \"expected_downtime\" at the beginning of each\niteration can in most cases also reflect the progress of migration\nsystem-wise. Said that, \"expected_downtime\" was always calculated based on\na bandwidth value that can fluctuate a lot if avail-switchover-bandwidth is\nnot used. This new \"remaining\" field will remove that part of uncertainty\nfor mgmt.\n\nWith the new field, HMP \"info migrate\" now reports this:\n\n(qemu) info migrate\nStatus: active\nTime (ms): total=12080, setup=14, exp_down=300\nRemaining (bytes): 1.36 GiB <------------------- newline\nRAM info:\n Throughput (Mbps): 840.50\n Sizes: pagesize=4 KiB, total=4.02 GiB\n Transfers: transferred=1.18 GiB, remain=1.36 GiB\n Channels: precopy=1.18 GiB, multifd=0 B, postcopy=0 B\n Page Types: normal=307923, zero=388148\n Page Rates (pps): transfer=25660\n Others: dirty_syncs=1\n\nIt should be the same value as RAM's remaining report when VFIO is not\ninvolved, and it should report more than that when VFIO is involved.\n\nCc: Markus Armbruster \nCc: Dr. David Alan Gilbert \nSigned-off-by: Peter Xu \n---\n qapi/migration.json | 4 ++++\n migration/migration-hmp-cmds.c | 5 +++++\n migration/migration.c | 11 +++++++++++\n 3 files changed, 20 insertions(+)","diff":"diff --git a/qapi/migration.json b/qapi/migration.json\nindex e3ad3f0604..a6e24b5685 100644\n--- a/qapi/migration.json\n+++ b/qapi/migration.json\n@@ -300,6 +300,9 @@\n # average memory load of the virtual CPU indirectly. Note that\n # zero means guest doesn't dirty memory. (Since 8.1)\n #\n+# @remaining: amount of bytes remaining to be migrated system-wise,\n+# includes both RAM and all devices (like VFIO). (Since 11.1)\n+#\n # Features:\n #\n # @unstable: Members @postcopy-latency, @postcopy-vcpu-latency,\n@@ -310,6 +313,7 @@\n ##\n { 'struct': 'MigrationInfo',\n 'data': {'*status': 'MigrationStatus', '*ram': 'MigrationRAMStats',\n+ '*remaining': 'uint64',\n '*vfio': 'VfioStats',\n '*xbzrle-cache': 'XBZRLECacheStats',\n '*total-time': 'int',\ndiff --git a/migration/migration-hmp-cmds.c b/migration/migration-hmp-cmds.c\nindex 0a193b8f54..721c211086 100644\n--- a/migration/migration-hmp-cmds.c\n+++ b/migration/migration-hmp-cmds.c\n@@ -178,6 +178,11 @@ void hmp_info_migrate(Monitor *mon, const QDict *qdict)\n }\n }\n \n+ if (info->has_remaining) {\n+ g_autofree char *remaining = size_to_str(info->remaining);\n+ monitor_printf(mon, \"Remaining (bytes): \\t%s\\n\", remaining);\n+ }\n+\n if (info->has_socket_address) {\n SocketAddressList *addr;\n \ndiff --git a/migration/migration.c b/migration/migration.c\nindex 4010e5dcf5..c2aa145106 100644\n--- a/migration/migration.c\n+++ b/migration/migration.c\n@@ -1076,6 +1076,16 @@ static void populate_time_info(MigrationInfo *info, MigrationState *s)\n }\n }\n \n+static void populate_global_info(MigrationInfo *info, MigrationState *s)\n+{\n+ MigPendingData data = { };\n+\n+ qemu_savevm_query_pending(&data, false);\n+\n+ info->has_remaining = true;\n+ info->remaining = data.total_bytes;\n+}\n+\n static void populate_ram_info(MigrationInfo *info, MigrationState *s)\n {\n size_t page_size = qemu_target_page_size();\n@@ -1177,6 +1187,7 @@ static void fill_source_migration_info(MigrationInfo *info)\n /* TODO add some postcopy stats */\n populate_time_info(info, s);\n populate_ram_info(info, s);\n+ populate_global_info(info, s);\n migration_populate_vfio_info(info);\n break;\n case MIGRATION_STATUS_COLO:\n","prefixes":["13/14"]}