[{"id":3681692,"web_url":"http://patchwork.ozlabs.org/comment/3681692/","msgid":"","list_archive_url":null,"date":"2026-04-23T22:10:55","subject":"Re: [PATCH v4 08/11] PCI: liveupdate: Require preserved devices are\n in immutable singleton IOMMU groups","submitter":{"id":69449,"url":"http://patchwork.ozlabs.org/api/people/69449/","name":"David Matlack","email":"dmatlack@google.com"},"content":"On Thu, Apr 23, 2026 at 2:23 PM David Matlack wrote:\n>\n> Restrict support for preserving PCI devices across Live Update to\n> devices in immutable singleton IOMMU groups. A device's group is\n> considered immutable if all bridges upstream from the device up to the\n> root port have the required ACS features enabled.\n>\n> Since ACS flags are inherited across a Live Update for preserved devices\n> and all the way up to the root port, the preserved device should be in a\n> singleton IOMMU group after kexec in the new kernel.\n>\n> This change should still permit all the current use-cases for PCI device\n> preservation across Live Update, since it is intended to be used in\n> Cloud enviroments which should have the required ACS features enabled\n> for virtualization purposes.\n>\n> If a device is part of a multi-device IOMMU group, preserving it will\n> now fail with an error. This restriction may be lifted in the future if\n> support for preserving multi-device groups is desired.\n>\n> Signed-off-by: David Matlack \n\nJason, do you think requiring singleton iommu groups is still\nnecessary/useful now that this series preserves ACS flags on preserved\ndevices and upstream bridges?","headers":{"Return-Path":"\n ","X-Original-To":["incoming@patchwork.ozlabs.org","linux-pci@vger.kernel.org"],"Delivered-To":"patchwork-incoming@legolas.ozlabs.org","Authentication-Results":["legolas.ozlabs.org;\n\tdkim=pass (2048-bit key;\n unprotected) header.d=google.com header.i=@google.com header.a=rsa-sha256\n header.s=20251104 header.b=RCSNOXLc;\n\tdkim-atps=neutral","legolas.ozlabs.org;\n spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org\n (client-ip=2600:3c04:e001:36c::12fc:5321; helo=tor.lore.kernel.org;\n envelope-from=linux-pci+bounces-53080-incoming=patchwork.ozlabs.org@vger.kernel.org;\n receiver=patchwork.ozlabs.org)","smtp.subspace.kernel.org;\n\tdkim=pass (2048-bit key) header.d=google.com header.i=@google.com\n header.b=\"RCSNOXLc\"","smtp.subspace.kernel.org;\n arc=pass smtp.client-ip=209.85.208.176","smtp.subspace.kernel.org;\n dmarc=pass (p=reject dis=none) header.from=google.com","smtp.subspace.kernel.org;\n spf=pass smtp.mailfrom=google.com"],"Received":["from tor.lore.kernel.org (tor.lore.kernel.org\n [IPv6:2600:3c04:e001:36c::12fc:5321])\n\t(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)\n\t key-exchange x25519)\n\t(No client certificate requested)\n\tby legolas.ozlabs.org (Postfix) with ESMTPS id 4g1r0T2ydCz1xvV\n\tfor ; Fri, 24 Apr 2026 08:11:37 +1000 (AEST)","from smtp.subspace.kernel.org (conduit.subspace.kernel.org\n [100.90.174.1])\n\tby tor.lore.kernel.org (Postfix) with ESMTP id 0275E30185BB\n\tfor ; Thu, 23 Apr 2026 22:11:30 +0000 (UTC)","from localhost.localdomain (localhost.localdomain [127.0.0.1])\n\tby smtp.subspace.kernel.org (Postfix) with ESMTP id 1D6DE3A759B;\n\tThu, 23 Apr 2026 22:11:27 +0000 (UTC)","from mail-lj1-f176.google.com (mail-lj1-f176.google.com\n [209.85.208.176])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits))\n\t(No client certificate requested)\n\tby smtp.subspace.kernel.org (Postfix) with ESMTPS id 9ABFA1E5724\n\tfor ; Thu, 23 Apr 2026 22:11:25 +0000 (UTC)","by mail-lj1-f176.google.com with SMTP id\n 38308e7fff4ca-38be5e86918so82596121fa.3\n for ; Thu, 23 Apr 2026 15:11:25 -0700 (PDT)"],"ARC-Seal":["i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116;\n\tt=1776982287; cv=pass;\n b=s6eHayHut6rPn2IvmqOSSFHymNf9h5bZy62JRiR4udbIZm0NgI2tluDmkfCpCcjdcxmWhz5/CEADEYA2qEu37ZUKkcOx0wLVhsCBl9oa+JA8PlyF2eh2iggs95UcfdoujjiOjjka3LDTzJkpf4+Lh6OJ8qriUl4TG1loSGZ+rjw=","i=1; a=rsa-sha256; t=1776982284; cv=none;\n d=google.com; s=arc-20240605;\n b=H4phw1oYfdFfg9IcSlz3CJmjaer8UN4do2mUXcOHlm523ctz0lKg+vJtm4xzjusyPs\n so2fYIzB1r9xtBlhcpEqvbrsoienr99R4QraUwi9nQ7PyX/NhgOGY+L2P0CcH0curu4B\n 5A+y90RreRKM7lP/TeWUsTF7dFBoWJ/0rrLz9VqamKnWF1KlrgT8t45frpVgBKthb2qQ\n Bjo5KzxQL9dUvvN4pSO82oEoh20Ik0wX9qTjymCQH5GwO+3j/EGelrI3u/SGEECgsvSs\n BM/MlZ5epVYsFdIkvPft/c/4L654QmbIKfssCq/WLGcrqfTcveTvjX62q4ZcCrPcs4yg\n yL8Q=="],"ARC-Message-Signature":["i=2; a=rsa-sha256; d=subspace.kernel.org;\n\ts=arc-20240116; t=1776982287; c=relaxed/simple;\n\tbh=1LE8KttVtGM+9CpiUlwllHY6AYyzUrNOnEbtRD51iDc=;\n\th=MIME-Version:References:In-Reply-To:From:Date:Message-ID:Subject:\n\t To:Cc:Content-Type;\n b=F7lGIAl3O16E4QaSD4ePRdfk/sIj7DohnkXu2cw165JiWmX2UuLfEmupw8B7dCjgbsrf6yklp1v8Jqe9zA/ZEA9kbZIikdgvIZUhNCWDjWOuEiLyFR3avgPF9zAq0NwDNOyiBqASAa/fpojuP32bOqQsJCS2BAFSQLv1LDawLCw=","i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com;\n s=arc-20240605;\n h=content-transfer-encoding:cc:to:subject:message-id:date:from\n :in-reply-to:references:mime-version:dkim-signature;\n bh=1LE8KttVtGM+9CpiUlwllHY6AYyzUrNOnEbtRD51iDc=;\n fh=Xes6DeTKMFbJzVeq4kaGFSzw0PxLC93maqFzZtbiabc=;\n b=YllcFiSpgHco864if4ttalN7ygIQu7oHjt20ldaneCn4SEzIYikWNM24fYqyKVncPm\n 6Kt6M7S0uZeUILdVFDHA17Fec+pUipo5KP2RIxDrxbY+bZIRK1G/4u9rqlLY98EUbqel\n eEBPjYs/8UkyXCS8/X4V24bdGSNSvSGrFXmEwPgY8n/zvifc/GXTlzB1VD8+4xGrZWPa\n JLiT4dylMySJady4xazkPt4nvKNLfVeEcxeRSdTUbJII8p5mlmuJWtX64O42uOOUKirv\n 05WJ88Gw8wU4Ep4S4pWG8Ee6IzlEO2pI2dhTLvdkOjj4fE/p2XrLjaosyelEYGhGITLE\n DiUg==;\n darn=vger.kernel.org"],"ARC-Authentication-Results":["i=2; smtp.subspace.kernel.org;\n dmarc=pass (p=reject dis=none) header.from=google.com;\n spf=pass smtp.mailfrom=google.com;\n dkim=pass (2048-bit key) header.d=google.com header.i=@google.com\n header.b=RCSNOXLc; arc=pass smtp.client-ip=209.85.208.176","i=1; mx.google.com; arc=none"],"DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n d=google.com; s=20251104; t=1776982284; x=1777587084;\n darn=vger.kernel.org;\n h=content-transfer-encoding:cc:to:subject:message-id:date:from\n :in-reply-to:references:mime-version:from:to:cc:subject:date\n :message-id:reply-to;\n bh=1LE8KttVtGM+9CpiUlwllHY6AYyzUrNOnEbtRD51iDc=;\n b=RCSNOXLc2RDedMpLLcSB07VXN1TcWxVckcnL+0r0qTQkGkuKmxa6Pe/0LfyQgxCWVW\n vNQBZwvY1uaaYFeB+fSgwtmyj4ienMzkRenCQenC8idV3GjbKmOk6mSXSimI7SRzo3cz\n T3Ho028BCBb2k1zqzwpXLqDiD+uZMONBNm23qJHHN+l4bgy26AOFaoY4EH/sCzcckY8e\n xmM9z4bNMDGKuvioRZPyI2fAe2qtEHbzZndIcKFq2Iw9MrWPdKIRaTWgMZfJmgN27EGa\n 5clB2hQ2DN2VDdmcrWBwe6EqpiPV3YUSOgWd4waYbjZH/B/pKgVxIwJz6cS8ttLniK94\n 6JbA==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n d=1e100.net; s=20251104; t=1776982284; x=1777587084;\n h=content-transfer-encoding:cc:to:subject:message-id:date:from\n :in-reply-to:references:mime-version:x-gm-gg:x-gm-message-state:from\n :to:cc:subject:date:message-id:reply-to;\n bh=1LE8KttVtGM+9CpiUlwllHY6AYyzUrNOnEbtRD51iDc=;\n b=RTfzNUEHbl8S9HUS/DJDYi9/zqMpdxT/SFoIfy7tmTLjACittmZEAVoih+BGj7cur8\n ETUxkWQh6HQFofl4g2z0238f87t0qo2aDfWOJIVTZwLP0H0ARDPl9mt+7JJ7vBFsWQ1q\n ieTU2UobyX4r5cTC0k9ssQr4oFUTrGRHvZa3frsjSMszWmh0O9RCEGlTAnXUhIVF3iDF\n c0ncFyzZV6epw/6UZwC4/iUwmhpvVayIgYMoka9E5usxmpy3DVS4hRXiEFQCIHdVqwtq\n CEFxaBHlBpbJvdwkAv8jFSeYhsEX/BOvjF1uvN5nOP2pqse/v7+dPRHO/XiXGwHh5clL\n +7KQ==","X-Forwarded-Encrypted":"i=1;\n AFNElJ9E0J4io6UBCRmgFcJ2NLxRu8n2NsX3Kx8b30jHFwsImaKAcRk01CCsvD//7EZNR+rDKUkj7JQm4IM=@vger.kernel.org","X-Gm-Message-State":"AOJu0YyqXEPSiMLvList46aKTWoJN42QmGt7klx4THRErmXL5S3vh/Q3\n\tn77Dz2t530rhSLQpNISk9iyPrUbNvNBzbtYbtHnQgvzoomP7WhigKpGmKk2opFoycWzzD74NrMf\n\thlqquXMjL0agsgWRfRh2tlL8M3lFe660JgNgAnSwt","X-Gm-Gg":"AeBDiev4P9L9h0fFzobsVNz1dyGLtmoyAaaqhohsizbp2+OCJgPt3PF4PsJKehNp71B\n\tLPI+K7AVVR6v0Q5gIifHHNBcgIkESfd2KCU8uC74SUMqSjikBOSJUIdYHGfhqbZ1AP1FfN0pgIa\n\t2kkfE35jH67lWHINGhInZl+9lP8Zuaf9qh2cbmcXPwzFRzlAp9lbAvNpnBkCyjSBE5v/GIRrXe7\n\tQKhr6/yTRpBKXL0vyoWT20jYRnx8blzwEXDzggIrYsFXGUg4uCHMLK/TSv22lRhfCekkd1gY6ho\n\tc4rWLQxYbbMa37qA/JeBajP8HicGCg==","X-Received":"by 2002:a2e:bc83:0:b0:38c:6b7:ad39 with SMTP id\n 38308e7fff4ca-38ec7818193mr99821181fa.13.1776982283419; Thu, 23 Apr 2026\n 15:11:23 -0700 (PDT)","Precedence":"bulk","X-Mailing-List":"linux-pci@vger.kernel.org","List-Id":"","List-Subscribe":"","List-Unsubscribe":"","MIME-Version":"1.0","References":"<20260423212316.3431746-1-dmatlack@google.com>\n <20260423212316.3431746-9-dmatlack@google.com>","In-Reply-To":"<20260423212316.3431746-9-dmatlack@google.com>","From":"David Matlack ","Date":"Thu, 23 Apr 2026 15:10:55 -0700","X-Gm-Features":"AQROBzCzQHNRAyujF9tGZltx4kVtGjkMoj2uVw4QUXf8lLhcB6v0kyr5BiRYiDc","Message-ID":"\n ","Subject":"Re: [PATCH v4 08/11] PCI: liveupdate: Require preserved devices are\n in immutable singleton IOMMU groups","To":"iommu@lists.linux.dev, kexec@lists.infradead.org,\n\tlinux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org,\n\tlinux-pci@vger.kernel.org","Cc":"Adithya Jayachandran ,\n Alexander Graf ,\n\tAlex Williamson , Bjorn Helgaas ,\n Chris Li ,\n\tDavid Rientjes ,\n Jacob Pan ,\n\tJason Gunthorpe , Joerg Roedel ,\n Jonathan Corbet ,\n\tJosh Hilke , Leon Romanovsky ,\n Lukas Wunner ,\n\tMike Rapoport , Parav Pandit ,\n\tPasha Tatashin ,\n Pranjal Shrivastava ,\n\tPratyush Yadav , Robin Murphy ,\n\tSaeed Mahameed , Samiullah Khawaja ,\n\tShuah Khan , Will Deacon ,\n\tWilliam Tu , Yi Liu ","Content-Type":"text/plain; charset=\"UTF-8\"","Content-Transfer-Encoding":"quoted-printable"}},{"id":3681708,"web_url":"http://patchwork.ozlabs.org/comment/3681708/","msgid":"<20260423225253.GA3444440@nvidia.com>","list_archive_url":null,"date":"2026-04-23T22:52:53","subject":"Re: [PATCH v4 08/11] PCI: liveupdate: Require preserved devices are\n in immutable singleton IOMMU groups","submitter":{"id":79424,"url":"http://patchwork.ozlabs.org/api/people/79424/","name":"Jason Gunthorpe","email":"jgg@nvidia.com"},"content":"On Thu, Apr 23, 2026 at 03:10:55PM -0700, David Matlack wrote:\n> On Thu, Apr 23, 2026 at 2:23 PM David Matlack wrote:\n> >\n> > Restrict support for preserving PCI devices across Live Update to\n> > devices in immutable singleton IOMMU groups. A device's group is\n> > considered immutable if all bridges upstream from the device up to the\n> > root port have the required ACS features enabled.\n> >\n> > Since ACS flags are inherited across a Live Update for preserved devices\n> > and all the way up to the root port, the preserved device should be in a\n> > singleton IOMMU group after kexec in the new kernel.\n> >\n> > This change should still permit all the current use-cases for PCI device\n> > preservation across Live Update, since it is intended to be used in\n> > Cloud enviroments which should have the required ACS features enabled\n> > for virtualization purposes.\n> >\n> > If a device is part of a multi-device IOMMU group, preserving it will\n> > now fail with an error. This restriction may be lifted in the future if\n> > support for preserving multi-device groups is desired.\n> >\n> > Signed-off-by: David Matlack \n> \n> Jason, do you think requiring singleton iommu groups is still\n> necessary/useful now that this series preserves ACS flags on preserved\n> devices and upstream bridges?\n\nI have forgotten why we introduced that? There are alot of funky\nthings about iommu groups that might be important upon restoration..\n\nLike if you preserve one group member but not the other what do you ?\n\nEven if you have ACS flags there are cases where groups are still\naliasing DMA..\n\nFrankly, multi-device iommu groups don't even work fully last time we\ntried to use them in a VMM. So I think I would not expect them to ever\nintersect with live update. Blocking something tricky you can't test\ndoes seem like a reasonable thing.\n\nJason","headers":{"Return-Path":"\n ","X-Original-To":["incoming@patchwork.ozlabs.org","linux-pci@vger.kernel.org"],"Delivered-To":"patchwork-incoming@legolas.ozlabs.org","Authentication-Results":["legolas.ozlabs.org;\n\tdkim=pass (2048-bit key;\n unprotected) header.d=Nvidia.com header.i=@Nvidia.com header.a=rsa-sha256\n header.s=selector2 header.b=R1qsp5YU;\n\tdkim-atps=neutral","legolas.ozlabs.org;\n spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org\n (client-ip=2600:3c04:e001:36c::12fc:5321; helo=tor.lore.kernel.org;\n envelope-from=linux-pci+bounces-53083-incoming=patchwork.ozlabs.org@vger.kernel.org;\n receiver=patchwork.ozlabs.org)","smtp.subspace.kernel.org;\n\tdkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com\n header.b=\"R1qsp5YU\"","smtp.subspace.kernel.org;\n arc=fail smtp.client-ip=40.107.208.38","smtp.subspace.kernel.org;\n dmarc=pass (p=reject dis=none) header.from=nvidia.com","smtp.subspace.kernel.org;\n spf=fail smtp.mailfrom=nvidia.com","dkim=none (message not signed)\n header.d=none;dmarc=none action=none header.from=nvidia.com;"],"Received":["from tor.lore.kernel.org (tor.lore.kernel.org\n [IPv6:2600:3c04:e001:36c::12fc:5321])\n\t(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)\n\t key-exchange x25519 server-signature ECDSA (secp384r1) server-digest SHA384)\n\t(No client certificate requested)\n\tby legolas.ozlabs.org (Postfix) with ESMTPS id 4g1rwY5Zgmz1y2d\n\tfor ; Fri, 24 Apr 2026 08:53:17 +1000 (AEST)","from smtp.subspace.kernel.org (conduit.subspace.kernel.org\n [100.90.174.1])\n\tby tor.lore.kernel.org (Postfix) with ESMTP id 4DC1B300D173\n\tfor ; Thu, 23 Apr 2026 22:53:15 +0000 (UTC)","from localhost.localdomain (localhost.localdomain [127.0.0.1])\n\tby smtp.subspace.kernel.org (Postfix) with ESMTP id C1C1C397E9E;\n\tThu, 23 Apr 2026 22:53:13 +0000 (UTC)","from PH0PR06CU001.outbound.protection.outlook.com\n (mail-westus3azon11011038.outbound.protection.outlook.com [40.107.208.38])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))\n\t(No client certificate requested)\n\tby smtp.subspace.kernel.org (Postfix) with ESMTPS id 19D923976BB;\n\tThu, 23 Apr 2026 22:53:04 +0000 (UTC)","from LV8PR12MB9620.namprd12.prod.outlook.com (2603:10b6:408:2a1::19)\n by SJ0PR12MB6757.namprd12.prod.outlook.com (2603:10b6:a03:449::21) with\n Microsoft SMTP Server (version=TLS1_2,\n cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9846.21; Thu, 23 Apr\n 2026 22:52:56 +0000","from LV8PR12MB9620.namprd12.prod.outlook.com\n ([fe80::299d:f5e0:3550:1528]) by LV8PR12MB9620.namprd12.prod.outlook.com\n ([fe80::299d:f5e0:3550:1528%5]) with mapi id 15.20.9846.019; Thu, 23 Apr 2026\n 22:52:56 +0000"],"ARC-Seal":["i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116;\n\tt=1776984791; cv=fail;\n b=unp2vaqHaLMVsd3PVP10IajXYFibSOgVBSyE4r9lYlsIsBI5sIP/dDcEOKAgzkC6zZGOuHTfKLl9cWSl4mMqDNgTkh2dAdiXpshyuzxZffBAs2mDBXtdq66DVESXbD/1I0gHvBvSuBntHV8D9YpBMT3L39y3qANln2qzalNNiC8=","i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none;\n b=nDBpk65ms4Q9Khnx6EqwzhLQdHWRfy94uaAh3R+jxcCz0/SUimYpfll96JdMO20lGJO5c30BF5w/nUq5b5SHwpKbBkPT2iFV1SwdvgJbBGa5wXCNqq8kW4iplY0oGj4y9Xm56ld5W3+4gVwrUKgFbq4OdqA0e8g3TtH1+AAq+iEbZGHeqZuWQ2TzAFNVuI/E4G2GQhzmwFqrkg7L+8sVGAyXSdiKj5P3iZ+HGbGAbR7E40MqMgFhCATDL+ORotBBySNMf3j1cc5kET/OP5CXrEAgO5MGrN9v2TeqrY9RENdKStdvAbvgMzVFuToi7iXbxjSEvYDji+285BIrHDTEnA=="],"ARC-Message-Signature":["i=2; a=rsa-sha256; d=subspace.kernel.org;\n\ts=arc-20240116; t=1776984791; c=relaxed/simple;\n\tbh=YMZnuZeXx+ni1tAqhSVRI83gzW4mXGmeBMX3HixJTlE=;\n\th=Date:From:To:Cc:Subject:Message-ID:References:Content-Type:\n\t Content-Disposition:In-Reply-To:MIME-Version;\n b=BCRMAywJYfOsBBBXBw18tbLC9YChLNVJdBRgrDr07IW68p1RG5nvm0/TFt5MAyymVto/eFnz01SdEjkjciCDRs0tUKG8KtDIspKnC1Zg6DOBDFHgS0ou9mCYZQ6nhWEK9WE8SUlYJunASFhlSLbeFsxCPnTBcDbH8vUQ0yaOcI0=","i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;\n s=arcselector10001;\n h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;\n bh=nGllgZSTj86q79vl+Z0gA8Li+LPlxtnlpDQ+5QFNJI8=;\n b=sIqOiXT/1YjwBRF4VpAN3Ovi0uNIPQzqwzYWRbJUGEBOOtKeNIQ585jVZKhlYkuBEcehTMQtz/2Gwa72iVlL5NaDDJ8jNA2vipDfZyp2ijaIT4+JRB2MPCxIFXXjYTs5VWp/c2rRJf1jwXS7/T9p406/nsNgRhqu/jJEigBL1/piC7Kq4Lt71tF40k1tt/L/U3v2E/JYFyO08phHUJ6Y6QQgyAwbCw8AnTiTIjte1/eRL1CZL0VeOBixQXKwU01AKm+29xiS0hWarZAuzmTt/BtmjEjO383pE5Yha8izNY1w3pCxTrBclmGgI1oAwpVqA2gSv5AEnIYPZ50HFpJbpw=="],"ARC-Authentication-Results":["i=2; smtp.subspace.kernel.org;\n dmarc=pass (p=reject dis=none) header.from=nvidia.com;\n spf=fail smtp.mailfrom=nvidia.com;\n dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com\n header.b=R1qsp5YU; arc=fail smtp.client-ip=40.107.208.38","i=1; mx.microsoft.com 1; spf=pass\n smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com;\n dkim=pass header.d=nvidia.com; arc=none"],"DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com;\n s=selector2;\n h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;\n bh=nGllgZSTj86q79vl+Z0gA8Li+LPlxtnlpDQ+5QFNJI8=;\n b=R1qsp5YUH6TuCNEvpATGs3AxFDj5GKYZt4k1ycnu1wlhZReH5r1vcatUQf3hbu1ASrJ0ZTSbmARmeBocPQwodeTRP8vZU8/myo9e0SzEd7JJLI4k1ygWU5KLnLlNEAKyvz5wc3SUzgB9PgQBbsJpVilZWUBlhNdWUvUY0c6Nwnl4i2RyPglQl/LKFRumV79GJz1hiJwstwTnW1ecQPptqFHM+sjf+uetYJH3nmHQ/GIZ2JsAWxy3tOpWxieYJsrn2DspeVqH8LXeQgCquusv5UD8VFiv9PY76jfKkLtdUPjCpVqBQ+sr9qYt7JnK2xGKE+9+OfxYS2J1p0sC2WUcCg==","Date":"Thu, 23 Apr 2026 19:52:53 -0300","From":"Jason Gunthorpe ","To":"David Matlack ","Cc":"iommu@lists.linux.dev, kexec@lists.infradead.org,\n\tlinux-doc@vger.kernel.org, linux-kernel@vger.kernel.org,\n\tlinux-mm@kvack.org, linux-pci@vger.kernel.org,\n\tAdithya Jayachandran ,\n\tAlexander Graf ,\n\tAlex Williamson ,\n\tBjorn Helgaas , Chris Li ,\n\tDavid Rientjes ,\n\tJacob Pan ,\n\tJoerg Roedel , Jonathan Corbet ,\n\tJosh Hilke ,\n\tLeon Romanovsky , Lukas Wunner ,\n\tMike Rapoport , Parav Pandit ,\n\tPasha Tatashin ,\n\tPranjal Shrivastava ,\n\tPratyush Yadav ,\n\tRobin Murphy ,\n\tSaeed Mahameed ,\n\tSamiullah Khawaja ,\n\tShuah Khan ,\n\tWill Deacon , William Tu ,\n\tYi Liu ","Subject":"Re: [PATCH v4 08/11] PCI: liveupdate: Require preserved devices are\n in immutable singleton IOMMU groups","Message-ID":"<20260423225253.GA3444440@nvidia.com>","References":"<20260423212316.3431746-1-dmatlack@google.com>\n <20260423212316.3431746-9-dmatlack@google.com>\n ","Content-Type":"text/plain; charset=utf-8","Content-Disposition":"inline","Content-Transfer-Encoding":"8bit","In-Reply-To":"\n ","X-ClientProxiedBy":"YT3PR01CA0037.CANPRD01.PROD.OUTLOOK.COM\n (2603:10b6:b01:82::14) To LV8PR12MB9620.namprd12.prod.outlook.com\n (2603:10b6:408:2a1::19)","Precedence":"bulk","X-Mailing-List":"linux-pci@vger.kernel.org","List-Id":"","List-Subscribe":"","List-Unsubscribe":"","MIME-Version":"1.0","X-MS-PublicTrafficType":"Email","X-MS-TrafficTypeDiagnostic":"LV8PR12MB9620:EE_|SJ0PR12MB6757:EE_","X-MS-Office365-Filtering-Correlation-Id":"e777414c-6e64-4055-c1f4-08dea18b0ea6","X-MS-Exchange-SenderADCheck":"1","X-MS-Exchange-AntiSpam-Relay":"0","X-Microsoft-Antispam":"\n\tBCL:0;ARA:13230040|366016|1800799024|7416014|376014|18002099003|22082099003|56012099003;","X-Microsoft-Antispam-Message-Info":"\n\t9E5nrqZ+WQxtFDiNe88eFWNuzkdZ1mqgMO4ovmVqy+2meNBKuhOKB/Z6bTyJ8w8INXHUtjVPxrQVMbFTfC6OyKl5x1WibZQYqs7SHNLtvEryunURao7rLwYTVnnvXJe3gHIiqF31OwquBta3RpnrbPDV24mSp5wSemT90y+wLZeo0N5ZG3oj7U/3MHWwXbyy7H7rZ9gLS9wah+5Coujfi+ZRj2K2itYjcQkkqRgmmeOZiB54r6bat1h3Vu8FvptY8QDFp7pifZKqNo0nHif5ikXys26HiCZOaRZke0RIikyyRxo+tBBg3avPHZuR82VhsDKTAhI7nudmIIaKM1DooDm9qtSjfnWT1IuhuUhxow8MBvesjGUQZv+9Orurduf3URLbDVlaNMCv/oJczOqygcLrM1krHKTdviPZkNT3o6CQ+oTQCkZfwLKdTtWG11tlNkd/deX9ttdPQhzXqt19P0iPgWG2kkgXupOOyxdoKhnvh7prtc+r0v5BpyVFQYHEj9qILqQX6wR2NDJQgpheS28hyUN03seQ+9id1H5+Hx7Ko1A8l/YCru+WxOBvkOwfi84XUVmekRbFPqcoekg30R7iUCtybzbLJQ2ZrHbzSWdivvxLQl/aH5oOpf2kL239lB/eX87P+w9oowKoxTl2kXKY7ypplhZH4b+pjBufPxjK/KM2vv1KvCrGHUUNP2uuXuzkTlKG+Xye69xbrqd6+FFzNK49kWot3HPlVM0ZFYI=","X-Forefront-Antispam-Report":"\n\tCIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:LV8PR12MB9620.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(366016)(1800799024)(7416014)(376014)(18002099003)(22082099003)(56012099003);DIR:OUT;SFP:1101;","X-MS-Exchange-AntiSpam-MessageData-ChunkCount":"1","X-MS-Exchange-AntiSpam-MessageData-0":"=?utf-8?q?tVk/lDJ1tmml9Yh0t7k9g66jVC8Z?=\n\t=?utf-8?q?AMDJVDYgZpixlSXv/2sMtHooEQu+PpyyckMWM4YvD8/rxjkuGPGh1ob2k+Rgj3kgS?=\n\t=?utf-8?q?hX7R19y3seaZuIvxXq7/fHQ7rLh6V4NZ4Vspbwoj03AiGUOAXr0pHq9RXPOF05Uah?=\n\t=?utf-8?q?A+lnsMWOkgCqHq5NX7o8B5PztAatcuMSboaJPrPUnAt08xamKOaRHcYyfzrZ330JN?=\n\t=?utf-8?q?cLb0Yr4B1AAVe5Jhbok04eliy6C1xYEVmfl7b1dl1dpq8ZVwR0kfNpDZ/R9edi0AO?=\n\t=?utf-8?q?5UYVBRru/mXzzF8J3HnXMbZMrBbFUKshTfkjJhhd74G8rO8WQJOMksIeyHyW6NebV?=\n\t=?utf-8?q?1EXpceUUQcLLdarJuk8s76VyioKS25IuroPfdDS4rVbnIe6wPmFS2CWE4uuF6H5zj?=\n\t=?utf-8?q?q0C3yGDzSE+RStD9hYP1hPPx/AzxXPB1rw21trDzx2Ot9jl1N1fNoZucB3gtuhjHM?=\n\t=?utf-8?q?ZJZ96uCNWmS2mg6nC5B5hfmCYr/NKGzDqwK3Dxff3IvZTAmgCWMJSgZSyCATMj+sl?=\n\t=?utf-8?q?ifKF0nK3HZwaRlwpXwSeutgYgv38e/Wj0sZxUVzy0qYE2Y9LNjVL1b+BjJTZ73cn0?=\n\t=?utf-8?q?Nk6983wkPdTojdRzv25p4sClz5OLsM0p+MwHSsyKY/xIHyh3ZLXv8REi3lPsYdN5s?=\n\t=?utf-8?q?pVHo1XGJoWh1H70AvJAOIhltA1jAKhsIbEjg7StFfbKgm0+HnvZPlaZ2lIu6wtluC?=\n\t=?utf-8?q?Jyt4FbisSX0oAL6K5mXtHACRQf8l1Wg4HdmRryEQOZZODmIvkzCqfakYFoqets1TJ?=\n\t=?utf-8?q?SDYCIvdCXYco7b2i0ZDlMI8IML4jNnI9QSpEZnrhSK4UwGQmIoOCm6a04B3FtSoaQ?=\n\t=?utf-8?q?SC5EKL6kSpGIRNe+qMR/b0RhZYSYtUtHGimZtd1mn3g9iqORGhf5RmhMVyoSzRZbV?=\n\t=?utf-8?q?j9kG9tgTLf3oEiThTyYW7PUppcCZDOPmQnZuyfaKCg5EvgjyK9lz0XEX5V9u92uca?=\n\t=?utf-8?q?BMKb/tw46BK8O+n5Q/j6oamdN2oYea4e+pv3S5X3s2sPfZx1xzL2Sby3h8P2QVcdt?=\n\t=?utf-8?q?9Ax3OUoxNrtpp9ItTIsdsDUjOSyJL4XV5K/BznFAtQhqCNKt9OF7mpzf8H0lNvxSE?=\n\t=?utf-8?q?Aq7FvEzt28c7PenKAB+sqcFMtMaPePxl0cAIigzbguClhLRbGgxd1znnClaO6O13Y?=\n\t=?utf-8?q?gjTidPqE7imE49gLsgS8/Lh/1WA4l95u7z+qslTM55dcuqaE9/nVPPL66074q4YoT?=\n\t=?utf-8?q?DTCLDFvEjekuCeJ9bNl4IR9Q+K4N6VGIIDHkopjaMT6aIyE3xUWgKWgGGLHLEV54u?=\n\t=?utf-8?q?prvwnDZ9rjIYsfQ5Xxu4enl6u9AaMn3ffOtAW/SsVtYocEk0Pc6M6/pUF61EvaI05?=\n\t=?utf-8?q?kfT2qLlUzzm4Hr2qjWFvMGttBk7GWDMIoVhpFoEtBUrKo5b31Cf9UAJFPmwMs7EsT?=\n\t=?utf-8?q?ZdYuTUJEw0idPC3ZM0gHHUNZb9uZcpa/QA6ngn3dZX0h69Cp2OLvyeDgx/QLfllMH?=\n\t=?utf-8?q?mWOL7gRyz2OLfaFmGABm30PJjcEb8rT4o6Q7U/kda/6Vf7g6za/QAFFfEQmmYsTcf?=\n\t=?utf-8?q?SjWOP0ydtf99Bgw275WM8kxXzrnTvbGCL0kt6+9TT53B0aBFJxTLyOB0fSB5zA9A4?=\n\t=?utf-8?q?WeKQWmK3p7eZ8hTGpHVQ56+FzvmZo01Tks+xEHLUjnfUpEyKIboR73vN4rXcCq8Sz?=\n\t=?utf-8?q?a/RW4unMVV?=","X-OriginatorOrg":"Nvidia.com","X-MS-Exchange-CrossTenant-Network-Message-Id":"\n e777414c-6e64-4055-c1f4-08dea18b0ea6","X-MS-Exchange-CrossTenant-AuthSource":"LV8PR12MB9620.namprd12.prod.outlook.com","X-MS-Exchange-CrossTenant-AuthAs":"Internal","X-MS-Exchange-CrossTenant-OriginalArrivalTime":"23 Apr 2026 22:52:55.4662\n (UTC)","X-MS-Exchange-CrossTenant-FromEntityHeader":"Hosted","X-MS-Exchange-CrossTenant-Id":"43083d15-7273-40c1-b7db-39efd9ccc17a","X-MS-Exchange-CrossTenant-MailboxType":"HOSTED","X-MS-Exchange-CrossTenant-UserPrincipalName":"\n fKFRlQfbUZ3pXp0uvb6C62YtwriAvp4rKVzDPKSoFpdrCyNfnSQ6ljRrRPtO8iyB","X-MS-Exchange-Transport-CrossTenantHeadersStamped":"SJ0PR12MB6757"}},{"id":3681709,"web_url":"http://patchwork.ozlabs.org/comment/3681709/","msgid":"","list_archive_url":null,"date":"2026-04-23T23:09:01","subject":"Re: [PATCH v4 08/11] PCI: liveupdate: Require preserved devices are\n in immutable singleton IOMMU groups","submitter":{"id":69449,"url":"http://patchwork.ozlabs.org/api/people/69449/","name":"David Matlack","email":"dmatlack@google.com"},"content":"On Thu, Apr 23, 2026 at 3:53 PM Jason Gunthorpe wrote:\n>\n> On Thu, Apr 23, 2026 at 03:10:55PM -0700, David Matlack wrote:\n> > On Thu, Apr 23, 2026 at 2:23 PM David Matlack wrote:\n> > >\n> > > Restrict support for preserving PCI devices across Live Update to\n> > > devices in immutable singleton IOMMU groups. A device's group is\n> > > considered immutable if all bridges upstream from the device up to the\n> > > root port have the required ACS features enabled.\n> > >\n> > > Since ACS flags are inherited across a Live Update for preserved devices\n> > > and all the way up to the root port, the preserved device should be in a\n> > > singleton IOMMU group after kexec in the new kernel.\n> > >\n> > > This change should still permit all the current use-cases for PCI device\n> > > preservation across Live Update, since it is intended to be used in\n> > > Cloud enviroments which should have the required ACS features enabled\n> > > for virtualization purposes.\n> > >\n> > > If a device is part of a multi-device IOMMU group, preserving it will\n> > > now fail with an error. This restriction may be lifted in the future if\n> > > support for preserving multi-device groups is desired.\n> > >\n> > > Signed-off-by: David Matlack \n> >\n> > Jason, do you think requiring singleton iommu groups is still\n> > necessary/useful now that this series preserves ACS flags on preserved\n> > devices and upstream bridges?\n>\n> I have forgotten why we introduced that? There are alot of funky\n> things about iommu groups that might be important upon restoration..\n\nYou had originally suggested it in this thread:\n\n https://lore.kernel.org/kvm/20260301192236.GQ5933@nvidia.com/\n\n> Like if you preserve one group member but not the other what do you ?\n\nYeah I imagine there could be some tricky cases there...\n\nI wonder if PCI core is the right layer to enforce this. Maybe this\nfits better into Sami's IOMMU core series since that is where all\nthose tricky cases will be (I imagine?).\n\n> Even if you have ACS flags there are cases where groups are still\n> aliasing DMA..\n\nHm, if a DMA alias can be created after boot time enumeration even\nwith the REQ_ACS_FLAGS check, then\npci_device_group_immutable_singleton() is not really immutable.\n\n\n\n> Frankly, multi-device iommu groups don't even work fully last time we\n> tried to use them in a VMM. So I think I would not expect them to ever\n> intersect with live update. Blocking something tricky you can't test\n> does seem like a reasonable thing.","headers":{"Return-Path":"\n ","X-Original-To":["incoming@patchwork.ozlabs.org","linux-pci@vger.kernel.org"],"Delivered-To":"patchwork-incoming@legolas.ozlabs.org","Authentication-Results":["legolas.ozlabs.org;\n\tdkim=pass (2048-bit key;\n unprotected) header.d=google.com header.i=@google.com header.a=rsa-sha256\n header.s=20251104 header.b=VqmA597z;\n\tdkim-atps=neutral","legolas.ozlabs.org;\n spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org\n (client-ip=2600:3c04:e001:36c::12fc:5321; helo=tor.lore.kernel.org;\n envelope-from=linux-pci+bounces-53084-incoming=patchwork.ozlabs.org@vger.kernel.org;\n receiver=patchwork.ozlabs.org)","smtp.subspace.kernel.org;\n\tdkim=pass (2048-bit key) header.d=google.com header.i=@google.com\n header.b=\"VqmA597z\"","smtp.subspace.kernel.org;\n arc=pass smtp.client-ip=209.85.208.173","smtp.subspace.kernel.org;\n dmarc=pass (p=reject dis=none) header.from=google.com","smtp.subspace.kernel.org;\n spf=pass smtp.mailfrom=google.com"],"Received":["from tor.lore.kernel.org (tor.lore.kernel.org\n [IPv6:2600:3c04:e001:36c::12fc:5321])\n\t(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)\n\t key-exchange x25519 server-signature ECDSA (secp384r1) server-digest SHA384)\n\t(No client certificate requested)\n\tby legolas.ozlabs.org (Postfix) with ESMTPS id 4g1sHg3TWKz1y2d\n\tfor ; Fri, 24 Apr 2026 09:09:51 +1000 (AEST)","from smtp.subspace.kernel.org (conduit.subspace.kernel.org\n [100.90.174.1])\n\tby tor.lore.kernel.org (Postfix) with ESMTP id DC66E300D170\n\tfor ; Thu, 23 Apr 2026 23:09:33 +0000 (UTC)","from localhost.localdomain (localhost.localdomain [127.0.0.1])\n\tby smtp.subspace.kernel.org (Postfix) with ESMTP id 1FF9A3D8118;\n\tThu, 23 Apr 2026 23:09:33 +0000 (UTC)","from mail-lj1-f173.google.com (mail-lj1-f173.google.com\n [209.85.208.173])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits))\n\t(No client certificate requested)\n\tby smtp.subspace.kernel.org (Postfix) with ESMTPS id 8F0BE3D7D9F\n\tfor ; Thu, 23 Apr 2026 23:09:31 +0000 (UTC)","by mail-lj1-f173.google.com with SMTP id\n 38308e7fff4ca-38e8292423fso63454581fa.0\n for ; Thu, 23 Apr 2026 16:09:31 -0700 (PDT)"],"ARC-Seal":["i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116;\n\tt=1776985773; cv=pass;\n b=YPzSPV6almp06npFguqw0pTbIAVesjJ2yJeuNME5jOxcaNpS1Ea6MDeFA4z8YGlo+BZjJD/lBGzVvMvj/ZTCK+cOnGCEhlcmVW9kK8hInpPmmrOf2cmfooWaWNHRat9EfvIKUHfTR76bgtnOTTB7M1TnYDpCcV5rfJIEOSyh0Lw=","i=1; a=rsa-sha256; t=1776985770; cv=none;\n d=google.com; s=arc-20240605;\n b=e/0kspVBuYa50oQlPFRgKEiPiiWSOs6QTQB6hCAI/sndPbls1V1XBJGpE4LFLRpQMl\n GBpBiArak6FRnZR35v9SoN4L1hl20ZRWHhRhZImF5mAeyP3aTPw1F11XjMS4QbV7PnFO\n vJoyhEelYbrF3GvwFpo0x6JFcuMzR5fhT+oebtF3Hb0FRDJlOGRPMIVyW8n2vd9dAKvZ\n lwBl45ppwf5Gxx8EUvgprfyzer+MAjhmtm1HnJKXLPcvp/fROPLony++OfeRo0gjnu0y\n MkiVFjBZYOOci9rgttj4ClBUu9VcXerOFzAGdm8vnOZc9sJlYxTDvGSzuHsVVoivCVcA\n r/QA=="],"ARC-Message-Signature":["i=2; a=rsa-sha256; d=subspace.kernel.org;\n\ts=arc-20240116; t=1776985773; c=relaxed/simple;\n\tbh=VSDvtpQyodvkH89hy45oW/Z42kZp4O88fgmSS7BMvzs=;\n\th=MIME-Version:References:In-Reply-To:From:Date:Message-ID:Subject:\n\t To:Cc:Content-Type;\n b=uOHjsqje0uIIyg3ynMyJix+Rk8lnGSOtxPfJEyhirIo0tzgi3Zr+NmagMUeWWHHf2I2MpSw4UEYR/DLjpmeT/WyMOHrqFZ6TRgBvyyNPlg8SBJBWU4+z8DZQv/YRtttQKe7Ir338i0OZ5KUBDL7v2RyqB70eDqRlzBKrGgXCg5I=","i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com;\n s=arc-20240605;\n h=content-transfer-encoding:cc:to:subject:message-id:date:from\n :in-reply-to:references:mime-version:dkim-signature;\n bh=du3muYvsQpTSBNV17AfC1+OtrIQr2PLFz51edOhTt/Q=;\n fh=BZbvJ0t8vqyXp3VhFh7/tjl/cbEMmmDk8QQaLCP1dys=;\n b=lNQr6dAsVlCBKDxNBH9PO2Sn555ZzJeCZ+9azojz2mV7wBFzoTjNpqrnEYAkbwk0ic\n +0RJyJjHhj/cdBTN2UKXT2ZNfEGwquxkuqxjmUo57wDBxnpP4CM6L9QNSGLKOC1a+6DZ\n E2BZnd8CuMmrCZbGoya2SOxLE6MF36kkEzajaz0mdYEBDz1sxx6AL9D10YwQcaCmC7AW\n Rk1zn08g45SBeBDFF9iBtBJZfDEQxLmk22qzRrJsd6/DZVBzCpn+P5Bdb8hCEFyiohpi\n BDEre6hQZKQzcP41ip1Hs7hAkiWEjoEpfB4ECr27FMjcxtrJlzSYDXgNgYZnfdzCEpSv\n QHSg==;\n darn=vger.kernel.org"],"ARC-Authentication-Results":["i=2; smtp.subspace.kernel.org;\n dmarc=pass (p=reject dis=none) header.from=google.com;\n spf=pass smtp.mailfrom=google.com;\n dkim=pass (2048-bit key) header.d=google.com header.i=@google.com\n header.b=VqmA597z; arc=pass smtp.client-ip=209.85.208.173","i=1; mx.google.com; arc=none"],"DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n d=google.com; s=20251104; t=1776985770; x=1777590570;\n darn=vger.kernel.org;\n h=content-transfer-encoding:cc:to:subject:message-id:date:from\n :in-reply-to:references:mime-version:from:to:cc:subject:date\n :message-id:reply-to;\n bh=du3muYvsQpTSBNV17AfC1+OtrIQr2PLFz51edOhTt/Q=;\n b=VqmA597zpWPLJykZ+qVtHxQMIF57cL7+JJDjRu7DOn15qon3vfMNKNm21wFO0ihqQr\n a+DYoC1AesbkgDijJF4s9J5H/xrQBR1u8s9yOO6g5YfrCK+lsegit/Hol6g9DMw8wQLV\n xYMJvMUP0JqsYvWAmnvV0ipZgELNNx/fs5E6rFOg3lekil5oKXntEvDynEcjnVaMPxDn\n 3DQAatSeUa8bCmpNzqNw7dxfdAX1lNp5gbU8/rQ42J0ZuFEGsrpaNRx+R8J98e4x5Ahz\n xu+5ds4dWTs2kaboCtwWxxbgYL+QwxJ2FolBvn2of0RruQY0ktQ9nwwBlSO5a6dV9VOF\n b2sw==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n d=1e100.net; s=20251104; t=1776985770; x=1777590570;\n h=content-transfer-encoding:cc:to:subject:message-id:date:from\n :in-reply-to:references:mime-version:x-gm-gg:x-gm-message-state:from\n :to:cc:subject:date:message-id:reply-to;\n bh=du3muYvsQpTSBNV17AfC1+OtrIQr2PLFz51edOhTt/Q=;\n b=rgKeCO8Pp1wYZp/aezOnDVinMTMwz5D/8WDCfL1hb7XURuruJuDGDOgtM3lildNqSC\n GEx76G2K8cLEAWVmc06n6HEhWyC+SkN0Smp+3U9v1dAK01AQXKYreHXXXP+amIyanDeP\n vKab7yNdp7GSTsj6XIEncIrgy7DNGeobR+SGV74GZhhtMSd1d5Snjyruu5n0/kJWrnmy\n vIIIdJ7OzqMyZzokffiUxcyFvlhPPCNB++0xLILkl1xqKjS2hR1PuXFtVJAZQJ2vQdAu\n tnoCBpqRus0vXxJs4XfsIk9SvYG7ihczQ+8uO5WPEDiQ9oGhPrpELH3SBldJDODOaWuH\n OOwA==","X-Forwarded-Encrypted":"i=1;\n AFNElJ/fLwHvh6qi3drbB0IgNrOzc7oqZc9aLzmiLBJ9q9qP2twug70128s3+GJ7M+UI5BuGux9SELKqrh8=@vger.kernel.org","X-Gm-Message-State":"AOJu0YyHPbVjsuSeN7vYTV2EHWumIm7Bdi9f/vD0Kw9y7wj1ww8CP9UK\n\t0iz5KUx/0Vy4i7QKVmy/nB/ck7+2GslFxCAiD4j2J/uYdILSfTj7Stdwjgt82vsmSGjasoSKY2A\n\tj6V1t+ExX7GDVgnDGnm/wpUE+iqvOD24Z6CjPgJVG","X-Gm-Gg":"AeBDietfHOkf6VmCAAHS3OtwumZHJeU5ZjdG/W+yg4YGAJCPRzMv2xkSBfLh7bv3sy2\n\tTgRtZbq3Ivx+sDnlARiejdVVqtMEds8Gj3ax+70RueXQLDCmIDA6echXiwMi0I8q+qtWEIgLuWF\n\tD0U8ut399WvVIGWDnVAIO1A5LM5MGWZXr2d/0H9R1nduFBfJqgvAGM3g92ESWvjGVHHt6lMUMSq\n\tO7YQ5oB/RQ+PzL4wQizj7b6HalAJ32zEFSPAbGpN0XQ8pBKuVnODcNItqL+SToe4YUOjcue3gO7\n\t20sjcWFkhCpK3eOati1FSIXbKO/AyA==","X-Received":"by 2002:a05:651c:418e:b0:38a:965d:79ac with SMTP id\n 38308e7fff4ca-38ec7845ffdmr82161421fa.12.1776985769266; Thu, 23 Apr 2026\n 16:09:29 -0700 (PDT)","Precedence":"bulk","X-Mailing-List":"linux-pci@vger.kernel.org","List-Id":"","List-Subscribe":"","List-Unsubscribe":"","MIME-Version":"1.0","References":"<20260423212316.3431746-1-dmatlack@google.com>\n <20260423212316.3431746-9-dmatlack@google.com>\n \n <20260423225253.GA3444440@nvidia.com>","In-Reply-To":"<20260423225253.GA3444440@nvidia.com>","From":"David Matlack ","Date":"Thu, 23 Apr 2026 16:09:01 -0700","X-Gm-Features":"AQROBzCagtk2g3nQiz1XNHTS_Y7Vpcsrr0LE0644u_NVwvd3XuR0UjLC-oW_vCY","Message-ID":"\n ","Subject":"Re: [PATCH v4 08/11] PCI: liveupdate: Require preserved devices are\n in immutable singleton IOMMU groups","To":"Jason Gunthorpe ","Cc":"iommu@lists.linux.dev, kexec@lists.infradead.org,\n\tlinux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org,\n\tlinux-pci@vger.kernel.org, Adithya Jayachandran ,\n\tAlexander Graf , Alex Williamson ,\n Bjorn Helgaas ,\n\tChris Li , David Rientjes ,\n\tJacob Pan , Joerg Roedel ,\n\tJonathan Corbet , Josh Hilke ,\n Leon Romanovsky ,\n\tLukas Wunner , Mike Rapoport ,\n Parav Pandit ,\n\tPasha Tatashin ,\n Pranjal Shrivastava ,\n\tPratyush Yadav , Robin Murphy ,\n\tSaeed Mahameed , Samiullah Khawaja ,\n\tShuah Khan , Will Deacon ,\n\tWilliam Tu , Yi Liu ","Content-Type":"text/plain; charset=\"UTF-8\"","Content-Transfer-Encoding":"quoted-printable"}},{"id":3681711,"web_url":"http://patchwork.ozlabs.org/comment/3681711/","msgid":"","list_archive_url":null,"date":"2026-04-23T23:27:46","subject":"Re: [PATCH v4 08/11] PCI: liveupdate: Require preserved devices are\n in immutable singleton IOMMU groups","submitter":{"id":91295,"url":"http://patchwork.ozlabs.org/api/people/91295/","name":"Samiullah Khawaja","email":"skhawaja@google.com"},"content":"On Thu, Apr 23, 2026 at 04:09:01PM -0700, David Matlack wrote:\n>On Thu, Apr 23, 2026 at 3:53 PM Jason Gunthorpe wrote:\n>>\n>> On Thu, Apr 23, 2026 at 03:10:55PM -0700, David Matlack wrote:\n>> > On Thu, Apr 23, 2026 at 2:23 PM David Matlack wrote:\n>> > >\n>> > > Restrict support for preserving PCI devices across Live Update to\n>> > > devices in immutable singleton IOMMU groups. A device's group is\n>> > > considered immutable if all bridges upstream from the device up to the\n>> > > root port have the required ACS features enabled.\n>> > >\n>> > > Since ACS flags are inherited across a Live Update for preserved devices\n>> > > and all the way up to the root port, the preserved device should be in a\n>> > > singleton IOMMU group after kexec in the new kernel.\n>> > >\n>> > > This change should still permit all the current use-cases for PCI device\n>> > > preservation across Live Update, since it is intended to be used in\n>> > > Cloud enviroments which should have the required ACS features enabled\n>> > > for virtualization purposes.\n>> > >\n>> > > If a device is part of a multi-device IOMMU group, preserving it will\n>> > > now fail with an error. This restriction may be lifted in the future if\n>> > > support for preserving multi-device groups is desired.\n>> > >\n>> > > Signed-off-by: David Matlack \n>> >\n>> > Jason, do you think requiring singleton iommu groups is still\n>> > necessary/useful now that this series preserves ACS flags on preserved\n>> > devices and upstream bridges?\n>>\n>> I have forgotten why we introduced that? There are alot of funky\n>> things about iommu groups that might be important upon restoration..\n>\n>You had originally suggested it in this thread:\n>\n> https://lore.kernel.org/kvm/20260301192236.GQ5933@nvidia.com/\n>\n>> Like if you preserve one group member but not the other what do you ?\n>\n>Yeah I imagine there could be some tricky cases there...\n>\n>I wonder if PCI core is the right layer to enforce this. Maybe this\n>fits better into Sami's IOMMU core series since that is where all\n>those tricky cases will be (I imagine?).\n\n+1\n\nAlso I think this should probably be checked by iommufd and invoked\nthrough vfio cdev. Basically when vfio cdev calls into iommufd to\npreserve IOMMU specific aspects of device (PASID table etc), iommufd can\ncheck this and return error.\n>\n>> Even if you have ACS flags there are cases where groups are still\n>> aliasing DMA..\n>\n>Hm, if a DMA alias can be created after boot time enumeration even\n>with the REQ_ACS_FLAGS check, then\n>pci_device_group_immutable_singleton() is not really immutable.\n>\n>\n>\n>> Frankly, multi-device iommu groups don't even work fully last time we\n>> tried to use them in a VMM. So I think I would not expect them to ever\n>> intersect with live update. Blocking something tricky you can't test\n>> does seem like a reasonable thing.","headers":{"Return-Path":"\n ","X-Original-To":["incoming@patchwork.ozlabs.org","linux-pci@vger.kernel.org"],"Delivered-To":"patchwork-incoming@legolas.ozlabs.org","Authentication-Results":["legolas.ozlabs.org;\n\tdkim=pass (2048-bit key;\n unprotected) header.d=google.com header.i=@google.com header.a=rsa-sha256\n header.s=20251104 header.b=Y8DjPBHe;\n\tdkim-atps=neutral","legolas.ozlabs.org;\n spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org\n (client-ip=172.105.105.114; helo=tor.lore.kernel.org;\n envelope-from=linux-pci+bounces-53085-incoming=patchwork.ozlabs.org@vger.kernel.org;\n receiver=patchwork.ozlabs.org)","smtp.subspace.kernel.org;\n\tdkim=pass (2048-bit key) header.d=google.com header.i=@google.com\n header.b=\"Y8DjPBHe\"","smtp.subspace.kernel.org;\n arc=none smtp.client-ip=209.85.214.170","smtp.subspace.kernel.org;\n dmarc=pass (p=reject dis=none) header.from=google.com","smtp.subspace.kernel.org;\n spf=pass smtp.mailfrom=google.com"],"Received":["from tor.lore.kernel.org (tor.lore.kernel.org [172.105.105.114])\n\t(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)\n\t key-exchange x25519 server-signature ECDSA (secp384r1) server-digest SHA384)\n\t(No client certificate requested)\n\tby legolas.ozlabs.org (Postfix) with ESMTPS id 4g1shX0rcjz1xvV\n\tfor ; Fri, 24 Apr 2026 09:27:56 +1000 (AEST)","from smtp.subspace.kernel.org (conduit.subspace.kernel.org\n [100.90.174.1])\n\tby tor.lore.kernel.org (Postfix) with ESMTP id DC89F3013A41\n\tfor ; Thu, 23 Apr 2026 23:27:52 +0000 (UTC)","from localhost.localdomain (localhost.localdomain [127.0.0.1])\n\tby smtp.subspace.kernel.org (Postfix) with ESMTP id DEBE33D88E7;\n\tThu, 23 Apr 2026 23:27:51 +0000 (UTC)","from mail-pl1-f170.google.com (mail-pl1-f170.google.com\n [209.85.214.170])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits))\n\t(No client certificate requested)\n\tby smtp.subspace.kernel.org (Postfix) with ESMTPS id 870403D8119\n\tfor ; Thu, 23 Apr 2026 23:27:50 +0000 (UTC)","by mail-pl1-f170.google.com with SMTP id\n d9443c01a7336-2b243198058so187515ad.1\n for ; Thu, 23 Apr 2026 16:27:50 -0700 (PDT)","from google.com (195.236.83.34.bc.googleusercontent.com.\n [34.83.236.195])\n by smtp.gmail.com with ESMTPSA id\n 41be03b00d2f7-c7976f9cabfsm18153636a12.9.2026.04.23.16.27.48\n (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);\n Thu, 23 Apr 2026 16:27:48 -0700 (PDT)"],"ARC-Seal":"i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116;\n\tt=1776986871; cv=none;\n b=H3ryIPyx9lK/EIATsiw9XBwlcO7U9uzztkDGL/kHOiol8JBGHaPQGwqmg8Hv1UGoC0gdC3jOlEQmU1jEpi4k2puPD/3WlDlfxYONQJ5tzWE7HGy9n3MhAKeJOUE1G1MeC2QrPSJZJ0vRTFuF1FFhG0/wWFxPlJ5a1k8mcOJ5PW4=","ARC-Message-Signature":"i=1; a=rsa-sha256; d=subspace.kernel.org;\n\ts=arc-20240116; t=1776986871; c=relaxed/simple;\n\tbh=gCwmX1OtHkLJhZqpnRYlpSdpxrI4jzQ0Eg7EJB7J8GE=;\n\th=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version:\n\t Content-Type:Content-Disposition:In-Reply-To;\n b=ifTEaU7uxz541FKA7xYd07tzzb8l+Ld30cl8s2d85VAsGmm6W9x69LY7PSux+PNI3nRY4MuKEX5BfPGOLdL1y/uZXfUjT1NbRQw3aBvoPaSCWdD33nAUP4vf/xgzpLJSGSrmDKEjzwjSUbjO4U2PAgnY+KxMSVuaJiQeiNTbRSU=","ARC-Authentication-Results":"i=1; smtp.subspace.kernel.org;\n dmarc=pass (p=reject dis=none) header.from=google.com;\n spf=pass smtp.mailfrom=google.com;\n dkim=pass (2048-bit key) header.d=google.com header.i=@google.com\n header.b=Y8DjPBHe; arc=none smtp.client-ip=209.85.214.170","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n d=google.com; s=20251104; t=1776986870; x=1777591670;\n darn=vger.kernel.org;\n h=in-reply-to:content-transfer-encoding:content-disposition\n :mime-version:references:message-id:subject:cc:to:from:date:from:to\n :cc:subject:date:message-id:reply-to;\n bh=lt0LRXP7f5oacTu2NcOl/7iPlzf9+apNICsUQAZ1DMY=;\n b=Y8DjPBHeWa3hHxMbSz88wXYEAoUayTTessiivRl6o617Gm8uDsSALcGpeLvpUoQDeR\n xCzj4i84K3eQylbZn9c027XRRiswacYDUth72A2Qx78P9e14JNwbPNkfEAjn1o2D7e/J\n IJF7z4Ez6mYTdgsf10Cem+Pqz+nS23kTGskCOAgC/EKF/ttcdzXkDm0UQiXTfdN24e2L\n aXV2cUNStVMAJEahW+0uyAZo6klmdfdw46TNN0+DoZJ4HIZBfsj68lW4c9Rd5yaDjo4Y\n fb0nhO0qUmuQ6m0hPMDxhdSUyYN0SRB4M/+Qc98Lz1ahIJV5eLMScqt0pXJOCRFUOIk2\n 1NyA==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n d=1e100.net; s=20251104; t=1776986870; x=1777591670;\n h=in-reply-to:content-transfer-encoding:content-disposition\n :mime-version:references:message-id:subject:cc:to:from:date:x-gm-gg\n :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;\n bh=lt0LRXP7f5oacTu2NcOl/7iPlzf9+apNICsUQAZ1DMY=;\n b=e1/29PDqLne3hLCPQNsGUT7z9bRn9ZLDQ5yWuZwxnFiaiItqRAzTcmGqNCRXQTUnrh\n U6gX92YNRylflPK6ArxEbgFqAG4rkRo2Pj8YUMQY7avk2JfWolU+JLNE4UPMigTvNZ7R\n nfPBohIUHweVSED0rL+0u/0mxMvTui2xqC6pQR4iZJXa9akDuEX1U9LHiy9g+Fou27ND\n F4EayNVp398bUr3sV38DXIrSIx9t0Ft2AqJ0rP1xgO2+/HnVaBsU8AgIJTo0ByNKeQh5\n k0j04lv8DXopHOsCQcPXqc76TPrxfq7pW5aiRP/r8OzHla0BtLUIIlBHjpRYM11lq05C\n q0Ew==","X-Forwarded-Encrypted":"i=1;\n AFNElJ8QxbHk2H0idOcl3eC5f18G3CWNyx1l4wtqYjdgQeqSjhQGY6saOedxhqHOinaYwFapF76pU7pltqE=@vger.kernel.org","X-Gm-Message-State":"AOJu0Yxe/Sixg48AjIv77VI0giOo2vh3cPQlJ7ZphKEErkz8+pWpzzgx\n\tZDMY73qjCC9B6gxUBTWEv6sAJoRRp5VvTxbN2Qu/Sd2GZ5BRjUjuANcUYcP4vzZZIQ==","X-Gm-Gg":"AeBDievkbh6uSTOBw7hAK7gUoW+utZLC00KUvFKJlNAveTQuRm++zhRHI99NDnnZjuf\n\thNdTEffZdo/a/HOrZczyOR1vxzXOhwg39KkMkID71FOms3iJUQZcunf9pYZ4PjbLvBQjIKP8dmQ\n\ttDlZ1WJQ4+Dq64xk4x2fAplMVxNmWzMSNPST+A9SDX0jX2LCBPIr0aswPdHC4tNwq+F8GG5lBBI\n\t/9sEMI0dQlnkbpqALNoQNVvD08W7SL6G6VD+1cbSBMc7ZTmp7jFtYlW/HplIzeJgdLuWfwfsyds\n\tcsFPQEkdGcv4Vpx0XhCzLtVYT5Qy8qKfhwTZf1CEdrHlMgKifG6YJhmeE96pAI79jAiSFW2e0qO\n\tu1L1Fb2tBUDn+b8voUN9FVAoGfB9E33lYOZTdhh7pgTL5zYR26ok7m58oENIoUzamfs5wvUF/z4\n\tQR8Eja59Rek1qiPOoSNwEHyZJN0UEK5tx0eXz9WtSUZqlpbt9bNnsQ1MJmQNReeAFHcB9/PLEy0\n\tAClP4PsFC4=","X-Received":"by 2002:a17:902:cf01:b0:2b0:5e19:1862 with SMTP id\n d9443c01a7336-2b603f4969amr20900125ad.5.1776986869309;\n Thu, 23 Apr 2026 16:27:49 -0700 (PDT)","Date":"Thu, 23 Apr 2026 23:27:46 +0000","From":"Samiullah Khawaja ","To":"David Matlack ","Cc":"Jason Gunthorpe , iommu@lists.linux.dev,\n\tkexec@lists.infradead.org, linux-doc@vger.kernel.org,\n linux-kernel@vger.kernel.org,\n\tlinux-mm@kvack.org, linux-pci@vger.kernel.org,\n\tAdithya Jayachandran ,\n Alexander Graf ,\n\tAlex Williamson , Bjorn Helgaas ,\n\tChris Li , David Rientjes ,\n\tJacob Pan , Joerg Roedel ,\n Jonathan Corbet ,\n\tJosh Hilke , Leon Romanovsky ,\n\tLukas Wunner , Mike Rapoport ,\n Parav Pandit ,\n\tPasha Tatashin ,\n Pranjal Shrivastava ,\n\tPratyush Yadav , Robin Murphy ,\n\tSaeed Mahameed , Shuah Khan ,\n\tWill Deacon , William Tu ,\n Yi Liu ","Subject":"Re: [PATCH v4 08/11] PCI: liveupdate: Require preserved devices are\n in immutable singleton IOMMU groups","Message-ID":"","References":"<20260423212316.3431746-1-dmatlack@google.com>\n <20260423212316.3431746-9-dmatlack@google.com>\n \n <20260423225253.GA3444440@nvidia.com>\n ","Precedence":"bulk","X-Mailing-List":"linux-pci@vger.kernel.org","List-Id":"","List-Subscribe":"","List-Unsubscribe":"","MIME-Version":"1.0","Content-Type":"text/plain; charset=utf-8; format=flowed","Content-Disposition":"inline","Content-Transfer-Encoding":"8bit","In-Reply-To":"\n "}}]