{"id":2175432,"url":"http://patchwork.ozlabs.org/api/1.0/patches/2175432/?format=json","project":{"id":14,"url":"http://patchwork.ozlabs.org/api/1.0/projects/14/?format=json","name":"QEMU Development","link_name":"qemu-devel","list_id":"qemu-devel.nongnu.org","list_email":"qemu-devel@nongnu.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20251218062643.624796-9-zhenzhong.duan@intel.com>","date":"2025-12-18T06:26:29","name":"[v6,8/9] vfio/migration: Add migration blocker if VM memory is too large to cause unmap_bitmap failure","commit_ref":null,"pull_url":null,"state":"new","archived":false,"hash":"7ae1e99970674be787b5ef521f5556772013a6e9","submitter":{"id":81636,"url":"http://patchwork.ozlabs.org/api/1.0/people/81636/?format=json","name":"Duan, Zhenzhong","email":"zhenzhong.duan@intel.com"},"delegate":null,"mbox":"http://patchwork.ozlabs.org/project/qemu-devel/patch/20251218062643.624796-9-zhenzhong.duan@intel.com/mbox/","series":[{"id":485789,"url":"http://patchwork.ozlabs.org/api/1.0/series/485789/?format=json","date":"2025-12-18T06:26:21","name":"vfio: relax the vIOMMU check","version":6,"mbox":"http://patchwork.ozlabs.org/series/485789/mbox/"}],"check":"pending","checks":"http://patchwork.ozlabs.org/api/patches/2175432/checks/","tags":{},"headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@legolas.ozlabs.org","Authentication-Results":["legolas.ozlabs.org;\n\tdkim=pass (2048-bit key;\n unprotected) header.d=intel.com header.i=@intel.com header.a=rsa-sha256\n header.s=Intel header.b=jviC8cpd;\n\tdkim-atps=neutral","legolas.ozlabs.org;\n spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org\n (client-ip=209.51.188.17; helo=lists.gnu.org;\n envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n receiver=patchwork.ozlabs.org)"],"Received":["from lists.gnu.org (lists.gnu.org [209.51.188.17])\n\t(using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits))\n\t(No client certificate requested)\n\tby legolas.ozlabs.org (Postfix) with ESMTPS id 4dX12c0nrkz1y2F\n\tfor ; Thu, 18 Dec 2025 17:28:40 +1100 (AEDT)","from localhost ([::1] helo=lists1p.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.90_1)\n\t(envelope-from )\n\tid 1vW7UU-0003Qs-Tf; Thu, 18 Dec 2025 01:27:58 -0500","from eggs.gnu.org ([2001:470:142:3::10])\n by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)\n (Exim 4.90_1) (envelope-from )\n id 1vW7UM-0003OP-BH\n for qemu-devel@nongnu.org; Thu, 18 Dec 2025 01:27:51 -0500","from mgamail.intel.com ([198.175.65.18])\n by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)\n (Exim 4.90_1) (envelope-from )\n id 1vW7UK-00019A-2L\n for qemu-devel@nongnu.org; Thu, 18 Dec 2025 01:27:49 -0500","from orviesa005.jf.intel.com ([10.64.159.145])\n by orvoesa110.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384;\n 17 Dec 2025 22:27:47 -0800","from unknown (HELO gnr-sp-2s-612.sh.intel.com) ([10.112.230.229])\n by orviesa005-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384;\n 17 Dec 2025 22:27:44 -0800"],"DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple;\n d=intel.com; i=@intel.com; q=dns/txt; s=Intel;\n t=1766039268; x=1797575268;\n h=from:to:cc:subject:date:message-id:in-reply-to:\n references:mime-version:content-transfer-encoding;\n bh=TlYJ1Hm3YmVmGZqWoDjzErOnh3Q4DaVDlQ3gQNxQfKk=;\n b=jviC8cpdSxBVHtv7hxxhclqY35GmnCaJDdzKI099U3B2A4MwodRt+nAg\n nGENH0axJV2uVhjZfT2S/SHp2hy2GoXt3z4zuHD1SNgI0aRvH+b1boS1v\n 8yvEENM4EeqDtLHihlLEIbMP7kbWbwwZQKHQV8dVGSmLCTjzgookL/a1o\n mnsS5mD8lcPuUAw2yw3/t7irckQs/jtT/d1utYuv/lxyi2xmuJgk/AYcC\n 6RSnOWhhvMCTpHoBIhQjMq+EPkO0ZiPyO9YM6ojMFAq3+vipyAIUsQlJr\n slvv5dH6fzDQdGaoYMTb8amEkYfVOl3cN1LhLHnG05N7rOawu+4RhAMC4 w==;","X-CSE-ConnectionGUID":["+253xDH6QDezdvGKIRM9xg==","37X6eWGSTNC/oUfSISmIlg=="],"X-CSE-MsgGUID":["mm9wlPUCRhGuKtt1osZjbw==","mU/88zmpSouADmtMJEZVCQ=="],"X-IronPort-AV":["E=McAfee;i=\"6800,10657,11645\"; a=\"68028572\"","E=Sophos;i=\"6.21,156,1763452800\"; d=\"scan'208\";a=\"68028572\"","E=Sophos;i=\"6.21,156,1763452800\"; d=\"scan'208\";a=\"203569907\""],"X-ExtLoop1":"1","From":"Zhenzhong Duan ","To":"qemu-devel@nongnu.org","Cc":"alex@shazbot.org, clg@redhat.com, mst@redhat.com, jasowang@redhat.com,\n yi.l.liu@intel.com, clement.mathieu--drif@eviden.com,\n eric.auger@redhat.com, joao.m.martins@oracle.com, avihaih@nvidia.com,\n xudong.hao@intel.com, giovanni.cabiddu@intel.com, rohith.s.r@intel.com,\n mark.gross@intel.com, arjan.van.de.ven@intel.com,\n Zhenzhong Duan ","Subject":"[PATCH v6 8/9] vfio/migration: Add migration blocker if VM memory is\n too large to cause unmap_bitmap failure","Date":"Thu, 18 Dec 2025 01:26:29 -0500","Message-ID":"<20251218062643.624796-9-zhenzhong.duan@intel.com>","X-Mailer":"git-send-email 2.47.1","In-Reply-To":"<20251218062643.624796-1-zhenzhong.duan@intel.com>","References":"<20251218062643.624796-1-zhenzhong.duan@intel.com>","MIME-Version":"1.0","Content-Transfer-Encoding":"8bit","Received-SPF":"pass client-ip=198.175.65.18;\n envelope-from=zhenzhong.duan@intel.com; helo=mgamail.intel.com","X-Spam_score_int":"-43","X-Spam_score":"-4.4","X-Spam_bar":"----","X-Spam_report":"(-4.4 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001,\n DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1,\n RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001,\n RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001,\n SPF_PASS=-0.001 autolearn=ham autolearn_force=no","X-Spam_action":"no action","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n ","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n ","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org"},"content":"With default config, kernel VFIO IOMMU type1 driver limits dirty bitmap to\n256MB for unmap_bitmap ioctl so the maximum guest memory region is no more\nthan 8TB size for the ioctl to succeed.\n\nBe conservative here to limit total guest memory to max value supported\nby unmap_bitmap ioctl or else add a migration blocker. IOMMUFD backend\ndoesn't have such limit, one can use it if there is a need to migrate such\nlarge VM.\n\nSuggested-by: Yi Liu \nSigned-off-by: Zhenzhong Duan \nReviewed-by: Yi Liu \n---\n hw/vfio/migration.c | 34 ++++++++++++++++++++++++++++++++++\n 1 file changed, 34 insertions(+)","diff":"diff --git a/hw/vfio/migration.c b/hw/vfio/migration.c\nindex 4c06e3db93..86e5b7ab55 100644\n--- a/hw/vfio/migration.c\n+++ b/hw/vfio/migration.c\n@@ -16,6 +16,7 @@\n #include \n \n #include \"system/runstate.h\"\n+#include \"hw/boards.h\"\n #include \"hw/vfio/vfio-device.h\"\n #include \"hw/vfio/vfio-migration.h\"\n #include \"migration/misc.h\"\n@@ -1152,6 +1153,32 @@ static bool vfio_viommu_preset(VFIODevice *vbasedev)\n return vbasedev->bcontainer->space->as != &address_space_memory;\n }\n \n+static bool vfio_dirty_tracking_exceed_limit(VFIODevice *vbasedev)\n+{\n+ VFIOContainer *bcontainer = vbasedev->bcontainer;\n+ uint64_t max_size, page_size;\n+\n+ if (!bcontainer->dirty_pages_supported) {\n+ return false;\n+ }\n+\n+ /*\n+ * VFIO IOMMU type1 driver has limitation of bitmap size on unmap_bitmap\n+ * ioctl(), calculate the limit and compare with guest memory size to\n+ * catch dirty tracking failure early.\n+ *\n+ * This limit is 8TB with default kernel and QEMU config, we are a bit\n+ * conservative here as VM memory layout may be nonconsecutive or VM\n+ * can run with vIOMMU enabled so the limitation could be relaxed. One\n+ * can also switch to use IOMMUFD backend if there is a need to migrate\n+ * large VM.\n+ */\n+ page_size = 1 << ctz64(bcontainer->dirty_pgsizes);\n+ max_size = bcontainer->max_dirty_bitmap_size * BITS_PER_BYTE * page_size;\n+\n+ return current_machine->ram_size > max_size;\n+}\n+\n /*\n * Return true when either migration initialized or blocker registered.\n * Currently only return false when adding blocker fails which will\n@@ -1193,6 +1220,13 @@ bool vfio_migration_realize(VFIODevice *vbasedev, Error **errp)\n goto add_blocker;\n }\n \n+ if (vfio_dirty_tracking_exceed_limit(vbasedev)) {\n+ error_setg(&err, \"%s: Migration is currently not supported with \"\n+ \"large memory VM due to dirty tracking limitation in \"\n+ \"backend\", vbasedev->name);\n+ goto add_blocker;\n+ }\n+\n warn_report(\"%s: VFIO device doesn't support device and \"\n \"IOMMU dirty tracking\", vbasedev->name);\n }\n","prefixes":["v6","8/9"]}