Patch Detail
get:
Show a patch.
patch:
Update a patch.
put:
Update a patch.
GET /api/patches/2224051/?format=api
{ "id": 2224051, "url": "http://patchwork.ozlabs.org/api/patches/2224051/?format=api", "web_url": "http://patchwork.ozlabs.org/project/qemu-devel/patch/20260416175659.41215-1-viking4@gmail.com/", "project": { "id": 14, "url": "http://patchwork.ozlabs.org/api/projects/14/?format=api", "name": "QEMU Development", "link_name": "qemu-devel", "list_id": "qemu-devel.nongnu.org", "list_email": "qemu-devel@nongnu.org", "web_url": "", "scm_url": "", "webscm_url": "", "list_archive_url": "", "list_archive_url_format": "", "commit_url_format": "" }, "msgid": "<20260416175659.41215-1-viking4@gmail.com>", "list_archive_url": null, "date": "2026-04-16T17:56:59", "name": "migration: expose per-device state save times via query-migrate", "commit_ref": null, "pull_url": null, "state": "new", "archived": false, "hash": "95854084ece0208c28e360f5b02aae786b14181f", "submitter": { "id": 92831, "url": "http://patchwork.ozlabs.org/api/people/92831/?format=api", "name": "Trieu Huynh", "email": "vikingtc4@gmail.com" }, "delegate": null, "mbox": "http://patchwork.ozlabs.org/project/qemu-devel/patch/20260416175659.41215-1-viking4@gmail.com/mbox/", "series": [ { "id": 500192, "url": "http://patchwork.ozlabs.org/api/series/500192/?format=api", "web_url": "http://patchwork.ozlabs.org/project/qemu-devel/list/?series=500192", "date": "2026-04-16T17:56:59", "name": "migration: expose per-device state save times via query-migrate", "version": 1, "mbox": "http://patchwork.ozlabs.org/series/500192/mbox/" } ], "comments": "http://patchwork.ozlabs.org/api/patches/2224051/comments/", "check": "pending", "checks": "http://patchwork.ozlabs.org/api/patches/2224051/checks/", "tags": {}, "related": [], "headers": { "Return-Path": "<qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org>", "X-Original-To": "incoming@patchwork.ozlabs.org", "Delivered-To": "patchwork-incoming@legolas.ozlabs.org", "Authentication-Results": [ "legolas.ozlabs.org;\n\tdkim=pass (2048-bit key;\n unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256\n header.s=20251104 header.b=RfUadpYw;\n\tdkim-atps=neutral", "legolas.ozlabs.org;\n spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org\n (client-ip=209.51.188.17; helo=lists1p.gnu.org;\n envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n receiver=patchwork.ozlabs.org)" ], "Received": [ "from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17])\n\t(using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits))\n\t(No client certificate requested)\n\tby legolas.ozlabs.org (Postfix) with ESMTPS id 4fxQhd2wkcz1yG9\n\tfor <incoming@patchwork.ozlabs.org>; Fri, 17 Apr 2026 03:57:37 +1000 (AEST)", "from localhost ([::1] helo=lists1p.gnu.org)\n\tby lists1p.gnu.org with esmtp (Exim 4.90_1)\n\t(envelope-from <qemu-devel-bounces@nongnu.org>)\n\tid 1wDQxl-0004XC-KK; Thu, 16 Apr 2026 13:57:13 -0400", "from eggs.gnu.org ([2001:470:142:3::10])\n by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)\n (Exim 4.90_1) (envelope-from <vikingtc4@gmail.com>)\n id 1wDQxj-0004Wb-FZ\n for qemu-devel@nongnu.org; Thu, 16 Apr 2026 13:57:11 -0400", "from mail-pl1-x629.google.com ([2607:f8b0:4864:20::629])\n by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128)\n (Exim 4.90_1) (envelope-from <vikingtc4@gmail.com>)\n id 1wDQxh-0006hq-Ai\n for qemu-devel@nongnu.org; Thu, 16 Apr 2026 13:57:11 -0400", "by mail-pl1-x629.google.com with SMTP id\n d9443c01a7336-2b25cf1b5f0so55095015ad.3\n for <qemu-devel@nongnu.org>; Thu, 16 Apr 2026 10:57:08 -0700 (PDT)", "from localhost.localdomain ([42.114.219.141])\n by smtp.gmail.com with ESMTPSA id\n d9443c01a7336-2b47810b898sm59435405ad.25.2026.04.16.10.57.05\n (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);\n Thu, 16 Apr 2026 10:57:07 -0700 (PDT)" ], "DKIM-Signature": "v=1; a=rsa-sha256; c=relaxed/relaxed;\n d=gmail.com; s=20251104; t=1776362228; x=1776967028; darn=nongnu.org;\n h=content-transfer-encoding:mime-version:message-id:date:subject:cc\n :to:from:from:to:cc:subject:date:message-id:reply-to;\n bh=kBFInjrJF2j/zE2Lb7WPQPedUT1UUv3ujUW7DIa2cWA=;\n b=RfUadpYwuqpBFOaOJdy7vVjs+YdtOX1DhU2vvbXlt87r0ZyBbmv4fXVoA052JxRGxT\n dWQHwq2g/az1Fjv4JTr+5pL5HooEwhwe7fVGVFzI/O4jrqXauTkO/N+HH+oj+cg171g7\n QvMdCKihciSdQmGoYn/KJZ5AB/LWjSD21EBgANiwamtkCS8TwCutBK5UCuyJu3jOU4oS\n 7calf5mfBr5pZotJGfm9u2c2XZipBB2x9doz+MO6mLC8qHtu0kHDZOmVm/bkj+EGmUFE\n ncgyS3umEB38YYoUj9C/Tx8oEmhzvHPthREmSbEeAdSXM+GdAtv9wgm2QXiJHXqVvnxA\n M0Qw==", "X-Google-DKIM-Signature": "v=1; a=rsa-sha256; c=relaxed/relaxed;\n d=1e100.net; s=20251104; t=1776362228; x=1776967028;\n h=content-transfer-encoding:mime-version:message-id:date:subject:cc\n :to:from:x-gm-gg:x-gm-message-state:from:to:cc:subject:date\n :message-id:reply-to;\n bh=kBFInjrJF2j/zE2Lb7WPQPedUT1UUv3ujUW7DIa2cWA=;\n b=R0DIu6fscuiwHL64ncmhp1m3HI0uEKhGIE0cnHGJruvHb0ueLlsYtC974aYqb/7rND\n iE8fo7aSgPOcUBK3K34rGm61JTz5tq+7VVITSQHE8zS//bLAPQHm0fUrm+l5NCblrhzi\n f6rq2oOiwmrBwN8CgMKN8D8ztYPvlu2d+lvcC6c8MohRIBvCwJ94fb0YMXU6cpM+LJy/\n m5TaBLcQiSlg8alElW4L/0JKaXshAYoRTPfpxnwE2O78xzbghBphMdC8ZGXZKDNgU/d2\n MuMJ/Q+szXzLrOjQeNF1GJHMRowq/5LPPMYvDlxn/nXXn4fQrWFMaOC49RRt2k347dfP\n iMbw==", "X-Gm-Message-State": "AOJu0YwCjNPc5+e2ZFmwl/fxnHX84dw925PpuRgaOwV6aQX0n2OtRLHE\n mTkx1ZJO6mEtrwsxg6Db02oaeHxpxeiiq129B5wGBnoEM1aFcjyRD2M47Y/Ue53J", "X-Gm-Gg": "AeBDiev6QnIclR7hTkd+3oMVGbpAYjzA06Lpxpuy4dgjLI8iOwXWEe+2CDWQ2QzoQqz\n NVbqeEBdPlIz1nLsoWhjRzDoPUzFp3FFoiOviaJg9ugEtawYqoHlv7DxVntIB2JSVYZo0gjaEtJ\n 9Ju/xJCGUHiqAcLRc1Z43LefAynCg3vWt/WxX0RK3vvuuYxrEFulPCKtVd4v9I4jrxhdA8BCjHY\n NZgORr9Wi3tntiOL/E+7pnHFDgCt0xaBGI34VmTu0on/TO6kAiZwH/53vNevXwRgbBY6LPxzLjW\n 9qXWH9BuFW6d46FxrFKsmSIaSAHl8aRRhgKymUfFTfIbNjByqHcN1PhVhdzBpZeIMW4T2GdsTLV\n iiHh2yIFBJyOh2GLj8tSplKbBdYKoSQlVHTypafYQwQanKYTAe3YynbuY10ONAhRS5pKcoDgASi\n ftcfLd3QRLQcI3rOQJj7gYMjXKqoOnQ/5YiL/scmipGPZlE+ydB1DVE2E58w==", "X-Received": "by 2002:a17:903:f84:b0:2b2:596d:594 with SMTP id\n d9443c01a7336-2b2d59619a5mr259362895ad.13.1776362227395;\n Thu, 16 Apr 2026 10:57:07 -0700 (PDT)", "From": "Trieu Huynh <vikingtc4@gmail.com>", "X-Google-Original-From": "Trieu Huynh <viking4@gmail.com>", "To": "qemu-devel@nongnu.org", "Cc": "Trieu Huynh <vikingtc4@gmail.com>, Peter Xu <peterx@redhat.com>,\n Fabiano Rosas <farosas@suse.de>, Eric Blake <eblake@redhat.com>,\n Markus Armbruster <armbru@redhat.com>", "Subject": "[PATCH] migration: expose per-device state save times via\n query-migrate", "Date": "Fri, 17 Apr 2026 00:56:59 +0700", "Message-ID": "<20260416175659.41215-1-viking4@gmail.com>", "X-Mailer": "git-send-email 2.43.0", "MIME-Version": "1.0", "Content-Transfer-Encoding": "8bit", "Received-SPF": "pass client-ip=2607:f8b0:4864:20::629;\n envelope-from=vikingtc4@gmail.com; helo=mail-pl1-x629.google.com", "X-Spam_score_int": "-17", "X-Spam_score": "-1.8", "X-Spam_bar": "-", "X-Spam_report": "(-1.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1,\n DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1,\n FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001,\n RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001,\n SPF_PASS=-0.001 autolearn=ham autolearn_force=no", "X-Spam_action": "no action", "X-BeenThere": "qemu-devel@nongnu.org", "X-Mailman-Version": "2.1.29", "Precedence": "list", "List-Id": "qemu development <qemu-devel.nongnu.org>", "List-Unsubscribe": "<https://lists.nongnu.org/mailman/options/qemu-devel>,\n <mailto:qemu-devel-request@nongnu.org?subject=unsubscribe>", "List-Archive": "<https://lists.nongnu.org/archive/html/qemu-devel>", "List-Post": "<mailto:qemu-devel@nongnu.org>", "List-Help": "<mailto:qemu-devel-request@nongnu.org?subject=help>", "List-Subscribe": "<https://lists.nongnu.org/mailman/listinfo/qemu-devel>,\n <mailto:qemu-devel-request@nongnu.org?subject=subscribe>", "Errors-To": "qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org", "Sender": "qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org" }, "content": "From: Trieu Huynh <vikingtc4@gmail.com>\n\nThe stop-and-copy phase pauses the VM and saves all non-iterable device\nstates. qemu_savevm_state_non_iterable() already measures per-device\nelapsed time for tracing (trace_vmstate_downtime_save, added in\ncommit 3c80f14272), but this information is never stored or surfaced\nto somewhere.\n\nExpose the result through a new 'device-state-times' list in\nMigrationInfo, filled by qemu_savevm_get_device_state_times() helper\nand returned by query-migrate when status is completed.\n\nA new QAPI type is introduced:\nDeviceSaveStateTime { 'name': 'str', 'instance-id': 'int', 'save-time': 'int' }\nwhere 'save-time' is the elapsed time in microseconds.\n\nThis allows operators and tooling to identify which device(s) are\ncontributing most to migration downtime.\n\nRelated prior work: Joao Martins posted a phase-level downtime\nbreakdown series in 2023 [1] that stalled during review. This patch\ntakes a complementary approach with per-device rather than per-phase\ngranularity, building on the tracepoint infrastructure that was merged\nfrom that discussion.\n\n* As-is:\n{\"execute\":\"query-migrate\"}\n{\n \"return\": {\n \"expected-downtime\": 300,\n \"status\": \"active\",\n \"setup-time\": 1,\n \"total-time\": 25501,\n \"ram\": {\n \"total\": 8607571968,\n \"postcopy-requests\": 0,\n \"dirty-sync-count\": 1,\n \"multifd-bytes\": 0,\n \"pages-per-second\": 32960,\n \"downtime-bytes\": 0,\n \"page-size\": 4096,\n \"remaining\": 2027167744,\n \"postcopy-bytes\": 0,\n \"mbps\": 1082.1441599999998,\n \"transferred\": 3409746788,\n \"dirty-sync-missed-zero-copy\": 0,\n \"precopy-bytes\": 3409871158,\n \"duplicate\": 777383,\n \"dirty-pages-rate\": 0,\n \"normal-bytes\": 3396239360,\n \"normal\": 829160\n }\n }\n}\n\n* To-be:\n{\"execute\":\"query-migrate\"}\n{\n \"return\": {\n \"device-state-times\": [\n {\n \"name\": \"apic\",\n \"instance-id\": 0,\n \"save-time\": 35\n },\n {\n \"name\": \"cpu_common\",\n \"instance-id\": 0,\n \"save-time\": 4\n },\n {\n \"name\": \"cpu\",\n \"instance-id\": 0,\n \"save-time\": 145\n },\n\n ...\n\n {\n \"name\": \"kvm-tpr-opt\",\n \"instance-id\": 0,\n \"save-time\": 23\n },\n {\n \"name\": \"0000:00:00.0/I440FX\",\n \"instance-id\": 0,\n \"save-time\": 9\n },\n {\n \"name\": \"PCIHost\",\n \"instance-id\": 0,\n \"save-time\": 3\n },\n {\n \"name\": \"PCIBUS\",\n \"instance-id\": 0,\n \"save-time\": 4\n },\n ],\n \"status\": \"completed\",\n \"setup-time\": 1,\n \"downtime\": 42,\n \"total-time\": 36049,\n \"ram\": {\n \"total\": 8607571968,\n \"postcopy-requests\": 0,\n \"dirty-sync-count\": 5,\n \"multifd-bytes\": 0,\n \"pages-per-second\": 32980,\n \"downtime-bytes\": 39224358,\n \"page-size\": 4096,\n \"remaining\": 0,\n \"postcopy-bytes\": 0,\n \"mbps\": 1080.48687394585,\n \"transferred\": 4868673854,\n \"dirty-sync-missed-zero-copy\": 0,\n \"precopy-bytes\": 4829122945,\n \"duplicate\": 1032714,\n \"dirty-pages-rate\": 0,\n \"normal-bytes\": 4849577984,\n \"normal\": 1183979\n }\n }\n}\n\n[1] https://lore.kernel.org/qemu-devel/20230926161841.98464-1-joao.m.martins@oracle.com/\nRelated-to: https://wiki.qemu.org/ToDo/LiveMigration#Device_state_downtime_analysis_and_accountings\n\nSigned-off-by: Trieu Huynh <vikingtc4@gmail.com>\n---\n migration/migration.c | 1 +\n migration/savevm.c | 29 +++++++++++++++++++++++++++++\n migration/savevm.h | 3 +++\n qapi/migration.json | 26 +++++++++++++++++++++++++-\n 4 files changed, 58 insertions(+), 1 deletion(-)", "diff": "diff --git a/migration/migration.c b/migration/migration.c\nindex 5c9aaa6e58..61bd98dc84 100644\n--- a/migration/migration.c\n+++ b/migration/migration.c\n@@ -1161,6 +1161,7 @@ static void fill_source_migration_info(MigrationInfo *info)\n populate_time_info(info, s);\n populate_ram_info(info, s);\n migration_populate_vfio_info(info);\n+ qemu_savevm_get_device_state_times(info);\n break;\n case MIGRATION_STATUS_FAILED:\n info->has_status = true;\ndiff --git a/migration/savevm.c b/migration/savevm.c\nindex dd58f2a705..3b1c4ff4a7 100644\n--- a/migration/savevm.c\n+++ b/migration/savevm.c\n@@ -46,6 +46,7 @@\n #include \"qapi/qapi-commands-migration.h\"\n #include \"qapi/clone-visitor.h\"\n #include \"qapi/qapi-builtin-visit.h\"\n+#include \"qapi/qapi-visit-migration.h\"\n #include \"qemu/error-report.h\"\n #include \"system/cpus.h\"\n #include \"system/memory.h\"\n@@ -263,6 +264,9 @@ typedef struct SaveState {\n QemuUUID uuid;\n } SaveState;\n \n+/* Per-device save times recorded */\n+static DeviceSaveStateTimeList *savevm_device_state_times;\n+\n static SaveState savevm_state = {\n .handlers = QTAILQ_HEAD_INITIALIZER(savevm_state.handlers),\n .handler_pri_head = { [0 ... MIG_PRI_MAX] = NULL },\n@@ -1710,11 +1714,18 @@ int qemu_savevm_state_non_iterable(QEMUFile *f, Error **errp)\n int64_t start_ts_each, end_ts_each;\n JSONWriter *vmdesc = ms->vmdesc;\n SaveStateEntry *se;\n+ DeviceSaveStateTimeList **tail;\n+ int64_t save_time;\n int ret;\n \n /* Making sure cpu states are synchronized before saving non-iterable */\n cpu_synchronize_all_states();\n \n+ /* Reset list from any previous migration run */\n+ qapi_free_DeviceSaveStateTimeList(savevm_device_state_times);\n+ savevm_device_state_times = NULL;\n+ tail = &savevm_device_state_times;\n+\n QTAILQ_FOREACH(se, &savevm_state.handlers, entry) {\n if (se->vmsd && se->vmsd->early_setup) {\n /* Already saved during qemu_savevm_state_setup(). */\n@@ -1731,6 +1742,14 @@ int qemu_savevm_state_non_iterable(QEMUFile *f, Error **errp)\n end_ts_each = qemu_clock_get_us(QEMU_CLOCK_REALTIME);\n trace_vmstate_downtime_save(\"non-iterable\", se->idstr, se->instance_id,\n end_ts_each - start_ts_each);\n+ save_time = end_ts_each - start_ts_each;\n+ if (se->vmsd && save_time > 0) {\n+ DeviceSaveStateTime *dsst = g_new0(DeviceSaveStateTime, 1);\n+ dsst->name = g_strdup(se->idstr);\n+ dsst->instance_id = se->instance_id;\n+ dsst->save_time = save_time;\n+ QAPI_LIST_APPEND(tail, dsst);\n+ }\n }\n \n trace_vmstate_downtime_checkpoint(\"src-non-iterable-saved\");\n@@ -3177,6 +3196,16 @@ bool qemu_loadvm_load_state_buffer(const char *idstr, uint32_t instance_id,\n return se->ops->load_state_buffer(se->opaque, buf, len, errp);\n }\n \n+void qemu_savevm_get_device_state_times(MigrationInfo *info)\n+{\n+ if (!savevm_device_state_times) {\n+ return;\n+ }\n+ info->device_state_times = QAPI_CLONE(DeviceSaveStateTimeList,\n+ savevm_device_state_times);\n+ info->has_device_state_times = true;\n+}\n+\n bool save_snapshot(const char *name, bool overwrite, const char *vmstate,\n bool has_devices, strList *devices, Error **errp)\n {\ndiff --git a/migration/savevm.h b/migration/savevm.h\nindex b3d1e8a13c..a18af7eaed 100644\n--- a/migration/savevm.h\n+++ b/migration/savevm.h\n@@ -14,6 +14,8 @@\n #ifndef MIGRATION_SAVEVM_H\n #define MIGRATION_SAVEVM_H\n \n+#include \"qapi/qapi-types-migration.h\"\n+\n #define QEMU_VM_FILE_MAGIC 0x5145564d\n #define QEMU_VM_FILE_VERSION_COMPAT 0x00000002\n #define QEMU_VM_FILE_VERSION 0x00000003\n@@ -78,5 +80,6 @@ int qemu_savevm_state_non_iterable_early(QEMUFile *f,\n Error **errp);\n bool qemu_loadvm_load_state_buffer(const char *idstr, uint32_t instance_id,\n char *buf, size_t len, Error **errp);\n+void qemu_savevm_get_device_state_times(MigrationInfo *info);\n \n #endif\ndiff --git a/qapi/migration.json b/qapi/migration.json\nindex 7134d4ce47..17c7876392 100644\n--- a/qapi/migration.json\n+++ b/qapi/migration.json\n@@ -200,6 +200,23 @@\n { 'struct': 'VfioStats',\n 'data': {'transferred': 'int' } }\n \n+##\n+# @DeviceSaveStateTime:\n+#\n+# Save time information for a single device's state during migration.\n+#\n+# @name: device name as registered in the migration stream\n+#\n+# @instance-id: device instance ID\n+#\n+# @save-time: time in microseconds spent saving this device's state\n+# during the stop-and-copy phase\n+#\n+# Since: 10.2\n+##\n+{ 'struct': 'DeviceSaveStateTime',\n+ 'data': { 'name': 'str', 'instance-id': 'uint32', 'save-time': 'uint64' } }\n+\n ##\n # @MigrationInfo:\n #\n@@ -300,6 +317,12 @@\n # average memory load of the virtual CPU indirectly. Note that\n # zero means guest doesn't dirty memory. (Since 8.1)\n #\n+# @device-state-times: list of per-device state save times recorded\n+# during the stop-and-copy phase. Only present after a completed\n+# migration. Entries with zero save time are omitted. Useful for\n+# diagnosing which devices contribute most to migration downtime.\n+# (Since 10.2)\n+#\n # Features:\n #\n # @unstable: Members @postcopy-latency, @postcopy-vcpu-latency,\n@@ -331,7 +354,8 @@\n 'type': 'uint64', 'features': [ 'unstable' ] },\n '*socket-address': ['SocketAddress'],\n '*dirty-limit-throttle-time-per-round': 'uint64',\n- '*dirty-limit-ring-full-time': 'uint64'} }\n+ '*dirty-limit-ring-full-time': 'uint64',\n+ '*device-state-times': ['DeviceSaveStateTime']} }\n \n ##\n # @query-migrate:\n", "prefixes": [] }