Cover Letter Detail
Show a cover letter.
GET /api/covers/2225929/?format=api
{ "id": 2225929, "url": "http://patchwork.ozlabs.org/api/covers/2225929/?format=api", "web_url": "http://patchwork.ozlabs.org/project/qemu-devel/cover/20260421202110.306051-1-peterx@redhat.com/", "project": { "id": 14, "url": "http://patchwork.ozlabs.org/api/projects/14/?format=api", "name": "QEMU Development", "link_name": "qemu-devel", "list_id": "qemu-devel.nongnu.org", "list_email": "qemu-devel@nongnu.org", "web_url": "", "scm_url": "", "webscm_url": "", "list_archive_url": "", "list_archive_url_format": "", "commit_url_format": "" }, "msgid": "<20260421202110.306051-1-peterx@redhat.com>", "list_archive_url": null, "date": "2026-04-21T20:20:54", "name": "[v2,00/16] migration/vfio: Fix a few issues on API misuse or statistic reports", "submitter": { "id": 67717, "url": "http://patchwork.ozlabs.org/api/people/67717/?format=api", "name": "Peter Xu", "email": "peterx@redhat.com" }, "mbox": "http://patchwork.ozlabs.org/project/qemu-devel/cover/20260421202110.306051-1-peterx@redhat.com/mbox/", "series": [ { "id": 500889, "url": "http://patchwork.ozlabs.org/api/series/500889/?format=api", "web_url": "http://patchwork.ozlabs.org/project/qemu-devel/list/?series=500889", "date": "2026-04-21T20:20:55", "name": "migration/vfio: Fix a few issues on API misuse or statistic reports", "version": 2, "mbox": "http://patchwork.ozlabs.org/series/500889/mbox/" } ], "comments": "http://patchwork.ozlabs.org/api/covers/2225929/comments/", "headers": { "Return-Path": "<qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org>", "X-Original-To": "incoming@patchwork.ozlabs.org", "Delivered-To": "patchwork-incoming@legolas.ozlabs.org", "Authentication-Results": [ "legolas.ozlabs.org;\n\tdkim=pass (1024-bit key;\n unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256\n header.s=mimecast20190719 header.b=ajerbpk+;\n\tdkim=pass (2048-bit key;\n unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256\n header.s=google header.b=oCugs/Pp;\n\tdkim-atps=neutral", "legolas.ozlabs.org;\n spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org\n (client-ip=209.51.188.17; helo=lists1p.gnu.org;\n envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n receiver=patchwork.ozlabs.org)" ], "Received": [ "from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17])\n\t(using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits))\n\t(No client certificate requested)\n\tby legolas.ozlabs.org (Postfix) with ESMTPS id 4g0YgM5xj7z1yGs\n\tfor <incoming@patchwork.ozlabs.org>; Wed, 22 Apr 2026 06:22:23 +1000 (AEST)", "from localhost ([::1] helo=lists1p.gnu.org)\n\tby lists1p.gnu.org with esmtp (Exim 4.90_1)\n\t(envelope-from <qemu-devel-bounces@nongnu.org>)\n\tid 1wFHay-0000uw-2K; Tue, 21 Apr 2026 16:21:20 -0400", "from eggs.gnu.org ([2001:470:142:3::10])\n by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)\n (Exim 4.90_1) (envelope-from <peterx@redhat.com>) id 1wFHaw-0000uH-Qm\n for qemu-devel@nongnu.org; Tue, 21 Apr 2026 16:21:18 -0400", "from us-smtp-delivery-124.mimecast.com ([170.10.129.124])\n by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)\n (Exim 4.90_1) (envelope-from <peterx@redhat.com>) id 1wFHau-0008Bk-KW\n for qemu-devel@nongnu.org; Tue, 21 Apr 2026 16:21:18 -0400", "from mail-qv1-f70.google.com (mail-qv1-f70.google.com\n [209.85.219.70]) by relay.mimecast.com with ESMTP with STARTTLS\n (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id\n us-mta-25-zaZ-73_uNuWBmYQu99TxNw-1; Tue, 21 Apr 2026 16:21:14 -0400", "by mail-qv1-f70.google.com with SMTP id\n 6a1803df08f44-8a22dbeeb96so94192016d6.3\n for <qemu-devel@nongnu.org>; Tue, 21 Apr 2026 13:21:14 -0700 (PDT)", "from x1.local ([142.189.10.167]) by smtp.gmail.com with ESMTPSA id\n 6a1803df08f44-8b02ae5c26csm114343556d6.31.2026.04.21.13.21.11\n (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);\n Tue, 21 Apr 2026 13:21:11 -0700 (PDT)" ], "DKIM-Signature": [ "v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com;\n s=mimecast20190719; t=1776802875;\n h=from:from:reply-to:subject:subject:date:date:message-id:message-id:\n to:to:cc:cc:mime-version:mime-version:content-type:content-type:\n content-transfer-encoding:content-transfer-encoding;\n bh=9adj77LoG/K3NSf2FVkJM97xr60l2Lnk6pP8u4l8F+o=;\n b=ajerbpk+wXqBeUfT5fFWdrq54aYjQPhdJSFd44zmPn6uB4/8yc7+1NzEjjiJiP8A1Z4Zvf\n 3bVwpoKS2/tlV/pwpIdImyciywwt/utFwTE0p6bT/basxpOz0NJm3EdMstIxmFfLlYXmhP\n wulYWp3IFfQzAEW5uMjA1+JF/fhCfZc=", "v=1; a=rsa-sha256; c=relaxed/relaxed;\n d=redhat.com; s=google; t=1776802873; x=1777407673; darn=nongnu.org;\n h=content-transfer-encoding:mime-version:message-id:date:subject:cc\n :to:from:from:to:cc:subject:date:message-id:reply-to;\n bh=9adj77LoG/K3NSf2FVkJM97xr60l2Lnk6pP8u4l8F+o=;\n b=oCugs/PpGUoUZO5aVJBGJXeqBO83VxDwbQjRpVM2aIGu0GK8cwlKzaFPbEv+zTgPpz\n slVlWsBYQP9vzGXr6OnlU+/M1Gkz8V/jXnA5UycJM7Xx0imPhNE/5PG6WB1amWbfRE/M\n 5lSidaU5RKtMA1k4LdoyfQArXrFfro0GcSehP3sHgx7X8ALV/ChTLcaxykNy0z8lj6Jt\n lbHNxhlClHaK8BtMEH0PAwMZq2MS96vA60sC+xcIPZt+7oblEPAPCgaf6q/DVr163lm8\n xHlGsKj9afgIXnGlJvMuRqWRZsb1xJAgDRzcR62Pn1KIQQ5iDNHyDgjplU9mWLfdeBh+\n gXhg==" ], "X-MC-Unique": "zaZ-73_uNuWBmYQu99TxNw-1", "X-Mimecast-MFC-AGG-ID": "zaZ-73_uNuWBmYQu99TxNw_1776802874", "X-Google-DKIM-Signature": "v=1; a=rsa-sha256; c=relaxed/relaxed;\n d=1e100.net; s=20251104; t=1776802873; x=1777407673;\n h=content-transfer-encoding:mime-version:message-id:date:subject:cc\n :to:from:x-gm-gg:x-gm-message-state:from:to:cc:subject:date\n :message-id:reply-to;\n bh=9adj77LoG/K3NSf2FVkJM97xr60l2Lnk6pP8u4l8F+o=;\n b=T0rcYrml3Htn2oeOHCtkWjVvtVjFNSKO4cU1tX3CxDGrAS0GrdEE5e7B3yataqbpGp\n CsI87AtjXFnTKWo7r7S/9IHqaECNIRGtLuoZQ+GLzAZUP3nOXkUXkHTOsT0Af6DqYVfh\n Adn7+8w5tUtxehAOOe/7nOHSapsSdZPq1gh3lDaBpN/YiRYocihmkMRVO8A+8hxDSNpt\n nraeteV6xh/KyJvzd23L7FX64/i8j6tBbUPxojCU5cfLA0fPlh6foLjm1glgUA+glphM\n Vla1JiQ0/gnYGLuIDan6nWEgSljVb8SD2DggInClWc/aOD8EBv3pNpGYvDM18OG32/ex\n vnfQ==", "X-Gm-Message-State": "AOJu0YxRIco+02xBUEzVD7LHcOstfV9UT0tkbtDRNfpy9xMjaBSnUKjg\n gFcuXsmu6CB9ul+E7t/yWRYRQ+x3ueMoZgQoZxnLxXe+P5afGnhNZVMoJKmYboqoeltwpmKaFSy\n 42otyb/0fzWY8SBml9vnxe6yh9w5MgvgqWcn2q9OSb6v4DUZlsh6LY6yZToPqmKILSeKth/gd3p\n Db1fDTRsRuiU+l4M+KgCK6SoYbrUIqzwJWNaeY3g==", "X-Gm-Gg": "AeBDieuz0Z+2okqEpgWOWRO4jjVHtGsPsmE6KU0zMxzmXQOkObiBiFidSdloqoUcuRV\n Q2YIeN8yFcJ0gqU4Vym1bSp1RnlgnGtzI6bNtS2gunuAfgqMmceQP6hijEQyfZ8eWkkltYuyX72\n I14TN3hG4yuCSlU6rwgNim0aWj5zc9FVwj8RtEp4Lq1Ymjh/tDPsSG8NwNOIUwdLiOmMziFEcX0\n XTU2a5nEA3tBAir+ekMk7scubexAOLNXioCkDiu2G3N2p86pFCmKzWB5ilq4xKF13qX4/Kkwlt8\n RB5QA1AaavPKc1VpKR9OzXenorhBE3Kb2v+RWA4gVeUKV31za4WtxYBgTgzC6c35nVUig6HyJEg\n TDJYilMPHN2EK4efMDdu+qXiXVtKd73owqVtKzuKXdNzv3RA44gpodiHGPw==", "X-Received": [ "by 2002:a05:6214:33ca:b0:89c:d808:c255 with SMTP id\n 6a1803df08f44-8b0281d57b4mr338200676d6.37.1776802873169;\n Tue, 21 Apr 2026 13:21:13 -0700 (PDT)", "by 2002:a05:6214:33ca:b0:89c:d808:c255 with SMTP id\n 6a1803df08f44-8b0281d57b4mr338199506d6.37.1776802872432;\n Tue, 21 Apr 2026 13:21:12 -0700 (PDT)" ], "From": "Peter Xu <peterx@redhat.com>", "To": "qemu-devel@nongnu.org", "Cc": "Joao Martins <joao.m.martins@oracle.com>,\n Markus Armbruster <armbru@redhat.com>,\n =?utf-8?q?C=C3=A9dric_Le_Goater?= <clg@redhat.com>,\n Avihai Horon <avihaih@nvidia.com>,\n =?utf-8?q?Daniel_P_=2E_Berrang=C3=A9?= <berrange@redhat.com>,\n Fabiano Rosas <farosas@suse.de>, Prasad Pandit <ppandit@redhat.com>,\n Alex Williamson <alex@shazbot.org>, Kirti Wankhede <kwankhede@nvidia.com>,\n Zhiyi Guo <zhguo@redhat.com>, Peter Xu <peterx@redhat.com>,\n \"Maciej S . Szmigiero\" <mail@maciej.szmigiero.name>,\n Juraj Marcin <jmarcin@redhat.com>", "Subject": "[PATCH v2 00/16] migration/vfio: Fix a few issues on API misuse or\n statistic reports", "Date": "Tue, 21 Apr 2026 16:20:54 -0400", "Message-ID": "<20260421202110.306051-1-peterx@redhat.com>", "X-Mailer": "git-send-email 2.53.0", "MIME-Version": "1.0", "Content-Type": "text/plain; charset=\"utf-8\"", "Content-Transfer-Encoding": "8bit", "Received-SPF": "pass client-ip=170.10.129.124; envelope-from=peterx@redhat.com;\n helo=us-smtp-delivery-124.mimecast.com", "X-Spam_score_int": "-20", "X-Spam_score": "-2.1", "X-Spam_bar": "--", "X-Spam_report": "(-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001,\n DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1,\n RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001,\n SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no", "X-Spam_action": "no action", "X-BeenThere": "qemu-devel@nongnu.org", "X-Mailman-Version": "2.1.29", "Precedence": "list", "List-Id": "qemu development <qemu-devel.nongnu.org>", "List-Unsubscribe": "<https://lists.nongnu.org/mailman/options/qemu-devel>,\n <mailto:qemu-devel-request@nongnu.org?subject=unsubscribe>", "List-Archive": "<https://lists.nongnu.org/archive/html/qemu-devel>", "List-Post": "<mailto:qemu-devel@nongnu.org>", "List-Help": "<mailto:qemu-devel-request@nongnu.org?subject=help>", "List-Subscribe": "<https://lists.nongnu.org/mailman/listinfo/qemu-devel>,\n <mailto:qemu-devel-request@nongnu.org?subject=subscribe>", "Errors-To": "qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org", "Sender": "qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org" }, "content": "CI: https://gitlab.com/peterx/qemu/-/pipelines/2469074018\nrfc: https://lore.kernel.org/r/20260319231302.123135-1-peterx@redhat.com\nv1: https://lore.kernel.org/r/20260408165559.157108-1-peterx@redhat.com\n\nv2:\n- Added tags\n- Patch 4\n - Fix and rework doc for @save_query_pending [Juraj]\n - Trace \"exact\" in trace_vfio_state_pending() [Avihai]\n - Avoid mentioning \"pre-copy\" in vfio.rst doc for query [Avihai]\n- Patch 12\n - English errors [Fabiano]\n- Patch 13\n - Remove \" (bytes)\" in HMP line [Fabiano]\n- Added patch \"qemu-iotests: Add query-migrate test for dirty-bitmap\"\n - This covers a bug that I found when testing v1\n- Added patch \"vfio/migration: Add tracepoints for precopy/stopcopy query\n ioctls\" to be able to dump the raw results from the two VFIO ioctls\n- Replace patch \"migration: Make qemu_savevm_query_pending() available\n anytime\" with patch \"migration: Remember total dirty bytes in mig_stats\"\n - I fell back to \"cache the total dirty bytes\" idea on this one to avoid\n complication of save_query_pending() invoked anywhere.\n\nOverview\n========\n\nVFIO migration was merged quite a while, but we do still see things off\nhere and there. This series tries to address some of them, but only based\non my limited understandings.\n\nTwo major issues I wanted to resolve:\n\n(1) VFIO reports state_pending_{exact|estimate}() differently\n\nIt reports stop-only sizes in exact() only (which includes both precopy and\nstopcopy data), while in estimate() it only reports precopy data. This is\nviolating the API. It was done like it to trigger proper sync on the VFIO\nioctls only but it was only a workaround. This series should fix it by\nintroducing stopcopy size reporting facility for vmstate handlers.\n\n(2) expected_downtime / remaining doesn't take VFIO devices into account\n\nWhen query migration, QEMU reports one field called \"expected-downtime\".\nThe document was phrasing this almost from RAM perspective, but ideally it\nshould be about an estimated blackout window (in milliseconds) if we\nswitchover anytime, based on known information.\n\nThis didn't yet took VFIO into account, especially in the case of VFIO\ndevices that may contain a large amount of device states (like GPUs).\n\nFor problem (2), the use case should be that an mgmt app when migrating a\nVFIO GPU device needs to always adjust downtime for migration to converge,\nbecause when it's involved normal downtime like 300ms will normally not\nsuffice.\n\nNow the issue with that is the mgmt doesn't have a good way to know exactly\nhow well the precopy goes with the whole system and the GPU device.\n\nThe hope is fixed expected_downtime will provide one way for the mgmt app\nto have a reasonable hint for downtime to setup to converge a migration.\n\nMeanwhile, with a system-wise \"remaining\" field introduced, mgmt can query\nthis results at beginning of each iteration to know if a stall is\nhappening, IOW, if it's likely that this migration will not converge at\nall. When detected, mgmt can start to consider the expected_downtime value\nreported above for converging this migration. See more on testing below.\n\nTests\n=====\n\nThanks to Cédric on help testing v2. One thing to mention is we did\nencounter one case where we observed reported dirty size overflowed for\nuint64_t (on both expected_downtime and system remaining data).\n\nQuotes from test results from Cédric, migrating a RHEL9 VM with a vGPU\n(NVIDIA L4-2B) and an MLX5 VF, from a RHEL9 host (vGPU mdev) to a RHEL10\nhost (vGPU VF), with the vGPU under load (glxgears):\n\n(qemu) info migrate\nStatus: active\nTime (ms): total=21140, setup=86, exp_down=152455434886355 <---- !?!\nRemaining: 16 EiB <---- !?!\nRAM info:\n Throughput (Mbps): 967.98\n Sizes: pagesize=4 KiB, total=4 GiB\n Transfers: transferred=2.29 GiB, remain=4.7 MiB\n Channels: precopy=1.91 GiB, multifd=0 B, postcopy=0 B, vfio=387 MiB\n Page Types: normal=499427, zero=559708\n Page Rates (pps): transfer=0, dirty=1892\n Others: dirty_syncs=3\n\nIt got fixed itself after a few more rounds of iterations, so it also\ndidn't affects migration ultimately. Further attempts didn't reproduce it\nafter I added the tracepoint patch. It would be good if someone knows if it\nwas a known driver issue.\n\nFor detailed testing steps, please refer to v1's cover letter.\n\nPeter Xu (16):\n qemu-iotests: Add query-migrate test for dirty-bitmap\n migration: Fix low possibility downtime violation\n migration/qapi: Rename MigrationStats to MigrationRAMStats\n vfio/migration: Cache stop size in VFIOMigration\n migration/treewide: Merge @state_pending_{exact|estimate} APIs\n migration: Use the new save_query_pending() API directly\n migration: Introduce stopcopy_bytes in save_query_pending()\n vfio/migration: Fix incorrect reporting for VFIO pending data\n migration: Move iteration counter out of RAM\n migration: Introduce a helper to return switchover bw estimate\n migration: Calculate expected downtime on demand\n migration: Fix calculation of expected_downtime to take VFIO info\n migration: Remember total dirty bytes in mig_stats\n migration/qapi: Introduce system-wise \"remaining\" reports\n migration/qapi: Update unit for avail-switchover-bandwidth\n vfio/migration: Add tracepoints for precopy/stopcopy query ioctls\n\n docs/about/removed-features.rst | 2 +-\n docs/devel/migration/main.rst | 9 +-\n docs/devel/migration/vfio.rst | 9 +-\n qapi/migration.json | 32 ++--\n hw/vfio/vfio-migration-internal.h | 8 +\n include/migration/register.h | 59 +++---\n migration/migration-stats.h | 20 +-\n migration/migration.h | 2 +-\n migration/savevm.h | 7 +-\n hw/s390x/s390-stattrib.c | 9 +-\n hw/vfio/migration.c | 123 +++++++-----\n migration/block-dirty-bitmap.c | 10 +-\n migration/migration-hmp-cmds.c | 5 +\n migration/migration.c | 177 +++++++++++++-----\n migration/ram.c | 40 +---\n migration/savevm.c | 42 ++---\n hw/vfio/trace-events | 5 +-\n migration/trace-events | 3 +-\n .../tests/migrate-bitmaps-postcopy-test | 6 +\n 19 files changed, 322 insertions(+), 246 deletions(-)" }