[{"id":3678291,"web_url":"http://patchwork.ozlabs.org/comment/3678291/","msgid":"<20260416173100.GA13378@bhelgaas>","list_archive_url":null,"date":"2026-04-16T17:31:00","subject":"Re: [PATCH] PCI: Avoid FLR for NVIDIA 5090 GPU","submitter":{"id":67298,"url":"http://patchwork.ozlabs.org/api/people/67298/","name":"Bjorn Helgaas","email":"helgaas@kernel.org"},"content":"[+cc Alex, Jason]\n\nOn Thu, Apr 16, 2026 at 03:07:06PM +0800, yuan.gao wrote:\n> When passing through the NVIDIA 5090 GPU to a vm, there is a certain\n> probability of encountering an flr timeout during vm shutdown, which\n> subsequently leads to a soft lock of the host cpu.\n\nIf possible, would like confirmation of device erratum from Nvidia.\nIf there's no known erratum, there might be something wrong in the\nLinux FLR and wait.\n\n> As described in this post\n> (https://forum.level1techs.com/t/do-your-rtx-5090-or-general-rtx-50-series-has-reset-bug-in-vm-passthrough/228549).\n> \n> And in dmesg:\n> \n> [401106.011979] vfio-pci 0000:d8:00.0: not ready 1023ms after FLR; waiting\n> [401108.700074] vfio-pci 0000:d8:00.0: not ready 2047ms after FLR; waiting\n> [401112.412204] vfio-pci 0000:d8:00.0: not ready 4095ms after FLR; waiting\n> [401118.620399] vfio-pci 0000:d8:00.0: not ready 8191ms after FLR; waiting\n> [401128.860788] vfio-pci 0000:d8:00.0: not ready 16383ms after FLR; waiting\n> [401147.293518] vfio-pci 0000:d8:00.0: not ready 32767ms after FLR; waiting\n> [401185.694859] vfio-pci 0000:d8:00.0: not ready 65535ms after FLR; giving up\n> [401195.372583] vfio-pci 0000:38:00.2: Relaying device request to user (#0)\n> \n> [401208.274941] watchdog: BUG: soft lockup - CPU#11 stuck for 21s! [CPU 22/KVM:30337]\n> \n> [401209.887848] CPU: 11 PID: 30337 Comm: CPU 22/KVM Kdump: loaded Not tainted\n> [401209.887854] RIP: 0010:pci_mmcfg_read+0xaa/0xd0\n> \n> [401209.887866] Call Trace:\n> [401209.887872] pci_bus_read_config_dword+0x43/0x70\n> [401209.b887876] pci_find_next_ext_capability.part.20+0x65/0xc0\n> [401209.887879] pci_restore_state.part.39+0x6d/0x3f0\n> [401209.887883] vfio_pci_disable+0x22b/0x4d0 [vfio_pci]\n> [401209.887886] ? __dentry_kill+0x118/0x160\n> [401209.887888] vfio_pci_release+0x5a/0xb0 [vfio_pci]\n> [401209.887891] vfio_device_fops_release+0x18/0x30 [vfio]\n> [401209.887894] __fput+0x98/0x240\n> [401209.887897] task_work_run+0x6a/0xa0\n> [401209.887899] do_exit+0x375/0xb10\n> [401209.887900] do_group_exit+0x3a/0xa0\n> [401209.887902] get_signal+0x140/0x7d0\n> [401209.887906] arch_do_signal+0x2c/0x260\n> [401209.887909] exit_to_user_mode_prepare+0xc0/0x120\n> [401209.887912] syscall_exit_to_user_mode+0x27/0x180\n> [401209.887915] entry_SYSCALL_64_after_hwframe+0x44/0xa9\n> \n> The flr seems to have some issues on the NVIDIA 5090 GPU,\n> so I’ve added flr-related quirks for these devices.\n> \n> And with this patch in place, the host kernel doesn't exhibit these\n> problems. The vm starts up and works as expected with the passed-through\n> NVIDIA 5090 GPU.\n> \n> Signed-off-by: yuan.gao \n> ---\n> drivers/pci/quirks.c | 3 +++\n> 1 file changed, 3 insertions(+)\n> \n> diff --git a/drivers/pci/quirks.c b/drivers/pci/quirks.c\n> index 48946cca4be72..71f833f3e2d84 100644\n> --- a/drivers/pci/quirks.c\n> +++ b/drivers/pci/quirks.c\n> @@ -5618,6 +5618,9 @@ DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_AMD, 0x7901, quirk_no_flr);\n> DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_INTEL, 0x1502, quirk_no_flr);\n> DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_INTEL, 0x1503, quirk_no_flr);\n> DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_MEDIATEK, 0x0616, quirk_no_flr);\n> +DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_NVIDIA, 0x2b85, quirk_no_flr);\n> +DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_NVIDIA, 0x2b87, quirk_no_flr);\n> +DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_NVIDIA, 0x2b8c, quirk_no_flr);\n> \n> /* FLR may cause the SolidRun SNET DPU (rev 0x1) to hang */\n> static void quirk_no_flr_snet(struct pci_dev *dev)\n> -- \n> 2.32.0\n>","headers":{"Return-Path":"\n ","X-Original-To":["incoming@patchwork.ozlabs.org","linux-pci@vger.kernel.org"],"Delivered-To":"patchwork-incoming@legolas.ozlabs.org","Authentication-Results":["legolas.ozlabs.org;\n\tdkim=pass (2048-bit key;\n unprotected) header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256\n header.s=k20201202 header.b=tgXm2Nc0;\n\tdkim-atps=neutral","legolas.ozlabs.org;\n spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org\n (client-ip=2600:3c09:e001:a7::12fc:5321; helo=sto.lore.kernel.org;\n envelope-from=linux-pci+bounces-52623-incoming=patchwork.ozlabs.org@vger.kernel.org;\n receiver=patchwork.ozlabs.org)","smtp.subspace.kernel.org;\n\tdkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org\n header.b=\"tgXm2Nc0\"","smtp.subspace.kernel.org;\n arc=none smtp.client-ip=10.30.226.201"],"Received":["from sto.lore.kernel.org (sto.lore.kernel.org\n [IPv6:2600:3c09:e001:a7::12fc:5321])\n\t(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)\n\t key-exchange x25519 server-signature ECDSA (secp384r1) server-digest SHA384)\n\t(No client certificate requested)\n\tby legolas.ozlabs.org (Postfix) with ESMTPS id 4fxQ625qmnz1yD3\n\tfor ; Fri, 17 Apr 2026 03:31:06 +1000 (AEST)","from smtp.subspace.kernel.org (conduit.subspace.kernel.org\n [100.90.174.1])\n\tby sto.lore.kernel.org (Postfix) with ESMTP id F024730157B3\n\tfor ; Thu, 16 Apr 2026 17:31:02 +0000 (UTC)","from localhost.localdomain (localhost.localdomain [127.0.0.1])\n\tby smtp.subspace.kernel.org (Postfix) with ESMTP id CD9693246ED;\n\tThu, 16 Apr 2026 17:31:01 +0000 (UTC)","from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org\n [10.30.226.201])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))\n\t(No client certificate requested)\n\tby smtp.subspace.kernel.org (Postfix) with ESMTPS id A95C22E424F;\n\tThu, 16 Apr 2026 17:31:01 +0000 (UTC)","by smtp.kernel.org (Postfix) with ESMTPSA id 6DE8BC2BCAF;\n\tThu, 16 Apr 2026 17:31:01 +0000 (UTC)"],"ARC-Seal":"i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116;\n\tt=1776360661; cv=none;\n b=n5K10PE/CiLqmxYAZmVn/mad8V9asjX03HSzpV7eoUxhemccOEnUdG1eDlTGWNTznzynFrJ3a5mr4rfe0jP39ait3/fJdxUk0tgM2IZqRPJ0iHdYdYxbbtbp5PUYVUKPCenp9F0mdjMYkM+6Dru2Jb11e9FWrQuQAjAwhSDCk9c=","ARC-Message-Signature":"i=1; a=rsa-sha256; d=subspace.kernel.org;\n\ts=arc-20240116; t=1776360661; c=relaxed/simple;\n\tbh=vGNw0uxuy2/Dir37UhfQwUgLTRlb/4/Oq7Jd0D0d6lo=;\n\th=Date:From:To:Cc:Subject:Message-ID:MIME-Version:Content-Type:\n\t Content-Disposition:In-Reply-To;\n b=gQP0BawMQTAaVIFXQQTnKQKy4AWooXJBt2JKHPZClA6sqrcmmOqfnKuYTLNIFDIHjxxgMpfBPeUpfNmrri1K/6v8FthQnnzyU0W9WljqXoHZ6dGUhPCmysgUpqqG6fGVvvLCKIhiVFfYwAZDqIRXpe/S51DFcu96Zmrdfq2JrGs=","ARC-Authentication-Results":"i=1; smtp.subspace.kernel.org;\n dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org\n header.b=tgXm2Nc0; arc=none smtp.client-ip=10.30.226.201","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;\n\ts=k20201202; t=1776360661;\n\tbh=vGNw0uxuy2/Dir37UhfQwUgLTRlb/4/Oq7Jd0D0d6lo=;\n\th=Date:From:To:Cc:Subject:In-Reply-To:From;\n\tb=tgXm2Nc0oTKwPuU96QTpoy5h6bdCnb3EqRE/aG2JxxfL+EP/0rTRkZIqrvyMb8/eX\n\t 34k9XCYKXuABtjW5tCDx/2j0ICV33OJjGDnzcR+GyHjBVoklegYQinm0K5UUXvoutt\n\t o+OLfPym8f2yqXZkAnLfrvaz44LWGZgB5Wb8Lr9VCZeTdlTQCY6rCwKkTyeCWwK5SZ\n\t AOuc/OoZ7JyLXuUURB4rIarcEiMaw7aB9iqFwSy2zfkL5FRlXHvbteXzvHzEa7OOO9\n\t gIMIPU6m8XtExNPmXp7EQfrW/RjI/S+fuh4jXtvSSJ5BdBS+XahtOL9tvk4uZMCnJv\n\t 4CXPi9gSF8aSA==","Date":"Thu, 16 Apr 2026 12:31:00 -0500","From":"Bjorn Helgaas ","To":"\"yuan.gao\" ","Cc":"Bjorn Helgaas , linux-pci@vger.kernel.org,\n\tlinux-kernel@vger.kernel.org, Alex Williamson ,\n\tJason Gunthorpe ","Subject":"Re: [PATCH] PCI: Avoid FLR for NVIDIA 5090 GPU","Message-ID":"<20260416173100.GA13378@bhelgaas>","Precedence":"bulk","X-Mailing-List":"linux-pci@vger.kernel.org","List-Id":"","List-Subscribe":"","List-Unsubscribe":"","MIME-Version":"1.0","Content-Type":"text/plain; charset=utf-8","Content-Disposition":"inline","Content-Transfer-Encoding":"8bit","In-Reply-To":"<20260416070707.3242381-1-yuan.gao@ucloud.cn>"}},{"id":3678412,"web_url":"http://patchwork.ozlabs.org/comment/3678412/","msgid":"<20260416223754.GC761338@nvidia.com>","list_archive_url":null,"date":"2026-04-16T22:37:54","subject":"Re: [PATCH] PCI: Avoid FLR for NVIDIA 5090 GPU","submitter":{"id":79424,"url":"http://patchwork.ozlabs.org/api/people/79424/","name":"Jason Gunthorpe","email":"jgg@nvidia.com"},"content":"On Thu, Apr 16, 2026 at 12:31:00PM -0500, Bjorn Helgaas wrote:\n> On Thu, Apr 16, 2026 at 03:07:06PM +0800, yuan.gao wrote:\n> > When passing through the NVIDIA 5090 GPU to a vm, there is a certain\n> > probability of encountering an flr timeout during vm shutdown, which\n> > subsequently leads to a soft lock of the host cpu.\n> \n> If possible, would like confirmation of device erratum from Nvidia.\n> If there's no known erratum, there might be something wrong in the\n> Linux FLR and wait.\n\nI asked and was told there is a known device firmware defect that\ncauses this.\n\nSo blanket disabling FLR without detecting good and bad FW is not a\ngood idea.\n\nI suggest Yuan try to use an NVIDIA support channel to try to resolve\nthe issue with their card..\n\nJason","headers":{"Return-Path":"\n ","X-Original-To":["incoming@patchwork.ozlabs.org","linux-pci@vger.kernel.org"],"Delivered-To":"patchwork-incoming@legolas.ozlabs.org","Authentication-Results":["legolas.ozlabs.org;\n\tdkim=pass (2048-bit key;\n unprotected) header.d=Nvidia.com header.i=@Nvidia.com header.a=rsa-sha256\n header.s=selector2 header.b=QtLuCwAx;\n\tdkim-atps=neutral","legolas.ozlabs.org;\n spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org\n (client-ip=172.105.105.114; helo=tor.lore.kernel.org;\n envelope-from=linux-pci+bounces-52657-incoming=patchwork.ozlabs.org@vger.kernel.org;\n receiver=patchwork.ozlabs.org)","smtp.subspace.kernel.org;\n\tdkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com\n header.b=\"QtLuCwAx\"","smtp.subspace.kernel.org;\n arc=fail smtp.client-ip=52.101.53.33","smtp.subspace.kernel.org;\n dmarc=pass (p=reject dis=none) header.from=nvidia.com","smtp.subspace.kernel.org;\n spf=fail smtp.mailfrom=nvidia.com","dkim=none (message not signed)\n header.d=none;dmarc=none action=none header.from=nvidia.com;"],"Received":["from tor.lore.kernel.org (tor.lore.kernel.org [172.105.105.114])\n\t(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)\n\t key-exchange x25519 server-signature ECDSA (secp384r1) server-digest SHA384)\n\t(No client certificate requested)\n\tby legolas.ozlabs.org (Postfix) with ESMTPS id 4fxXwG1WRhz1yCv\n\tfor ; Fri, 17 Apr 2026 08:38:06 +1000 (AEST)","from smtp.subspace.kernel.org (conduit.subspace.kernel.org\n [100.90.174.1])\n\tby tor.lore.kernel.org (Postfix) with ESMTP id 81B033045649\n\tfor ; Thu, 16 Apr 2026 22:38:03 +0000 (UTC)","from localhost.localdomain (localhost.localdomain [127.0.0.1])\n\tby smtp.subspace.kernel.org (Postfix) with ESMTP id E6ABC3314D2;\n\tThu, 16 Apr 2026 22:38:00 +0000 (UTC)","from BL0PR03CU003.outbound.protection.outlook.com\n (mail-eastusazon11012033.outbound.protection.outlook.com [52.101.53.33])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))\n\t(No client certificate requested)\n\tby smtp.subspace.kernel.org (Postfix) with ESMTPS id 93EC52C08DC;\n\tThu, 16 Apr 2026 22:37:59 +0000 (UTC)","from LV8PR12MB9620.namprd12.prod.outlook.com (2603:10b6:408:2a1::19)\n by DS2PR12MB9751.namprd12.prod.outlook.com (2603:10b6:8:2ad::17) with\n Microsoft SMTP Server (version=TLS1_2,\n cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9818.25; Thu, 16 Apr\n 2026 22:37:56 +0000","from LV8PR12MB9620.namprd12.prod.outlook.com\n ([fe80::299d:f5e0:3550:1528]) by LV8PR12MB9620.namprd12.prod.outlook.com\n ([fe80::299d:f5e0:3550:1528%5]) with mapi id 15.20.9818.017; Thu, 16 Apr 2026\n 22:37:56 +0000"],"ARC-Seal":["i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116;\n\tt=1776379080; cv=fail;\n b=XHMIW8ihS9/jAivNzNRmutuRpoKBzORaxkfIXRlr3H/adY7morjbkIT2NueF8jsuGOnUQWecJOPjbQ1xawRIQulxCg7FdO7UoSI8CJErzvyh5JL8pGMiaVeHBjvH3dNc7fmN2jguBJMpQceNBpmE35i05/9opimc9eX+oQve+l0=","i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none;\n b=RSkhj0+ETSCWeXLiKM0NBTlSo54M8TsQyvOlzSUYg9Ns1X4t+5DmyM4mHWi7dtNdDlCHG52Tl76ZeKk14FuKa6ci3ldnmd6mfTpBN3U2n2K9beTFXDAuxeirBNPxSc1Vljcz1WkiR2Jh8ps0PuI7Eq9si+UhpVpJPiEu1sf9JB6p21wG55cnVyCat7w0KYOXasOaob4p8ULFFHfYc6D+Wo3bFxxe4qg0WHFOdu3oHX382v+0qeuseCFJcfHcDzj5rDL7bxe+bKAy4Tuz4roAyVLimYrhcCoONbbVZhZUuCQbewaU66lkvWGAeaZfb8uZxDbaQDpzyb8RMhztOJ9gAQ=="],"ARC-Message-Signature":["i=2; a=rsa-sha256; d=subspace.kernel.org;\n\ts=arc-20240116; t=1776379080; c=relaxed/simple;\n\tbh=i7LYvunWbrSaOBJl7N8w7dSO3/Wa0XQcUazZRXb2Jf0=;\n\th=Date:From:To:Cc:Subject:Message-ID:References:Content-Type:\n\t Content-Disposition:In-Reply-To:MIME-Version;\n b=m+71pzvPwZ1LPoUF5tlksx7fsI//1NkpgT0IQA1Q5DQ/wyiT6m6jjy0LB80dfqHmE5oaHfqFpQzj10b25PJ4EKlmoxd/Dr1q9TGRvzT3zZXTXIJ+cp1vLbJgZV/zZpcfcqapFWXpXm6ZNytcMTuC1DkuktP0krLGpUe6pNlZ5Vc=","i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;\n s=arcselector10001;\n h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;\n bh=4EeWwIBeQmzmpu5xrtUX5rDd/fCiAHZ5GpFIWOnmoic=;\n b=Xr2HA32mXW2zbO8fV0PirGJgGWLYDXsd+PnG0wme9PF0/6BhxvTMCNZht+hE3CzJVQ3aVeyZy/VewyYCPZ/lG4IBJWO7dihJTkJkf+YfW9x0dJRux0fTWZfEWlB1VDUHNAnvWfNMuCfTOD0M/uCuqYNfP/WUr3BbwY2BGxDkJJr0UINwxXTLlHYlq+oUe1b1PbbCYzG4qcbe/REY5dItuLIaz8zmhpotSNoWsuwPhS8ncHInreZ5yYJCmqz1Rmdl7qsFlb6l0hcAwDybezreOt3Q07WZ+RKlxi4zJPCJ4/I8g13s7k0i5l0FvJFr3MqwBQo1Vbw8ISZ6KHt59FZ00g=="],"ARC-Authentication-Results":["i=2; smtp.subspace.kernel.org;\n dmarc=pass (p=reject dis=none) header.from=nvidia.com;\n spf=fail smtp.mailfrom=nvidia.com;\n dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com\n header.b=QtLuCwAx; arc=fail smtp.client-ip=52.101.53.33","i=1; mx.microsoft.com 1; spf=pass\n smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com;\n dkim=pass header.d=nvidia.com; arc=none"],"DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com;\n s=selector2;\n h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;\n bh=4EeWwIBeQmzmpu5xrtUX5rDd/fCiAHZ5GpFIWOnmoic=;\n b=QtLuCwAxZo4Mx0x9OQKlqhMW5zaxaDhrxMmNRTZkdBlu2zQ20FcXuAM+pFdjPp+fpBlL7/tBFxL371/0gMtJmGbom1Y264iIPzFnkUt4x6JiRWe/qHCa5R4msaojvuBN5R6ApJTwENmGl7Ws0FZNHKndHLMBAy72h8iWF6EAH4VRhkZQ9Qzmm3qR+gvqKOrgcPphb1hKZkzPlEDbQ3n6FLkUkDVD7bMFdaYnm1MTj/I1K4KsmIc5sz2schaE19rh/+gZ7hE7W3xkDDRpwgyQkDvXpqOPMjHM3P7Je2umxRhmY7zoeMEA7ToamS/HPOj4z/zGpAIoIE17YUMg6EOx+g==","Date":"Thu, 16 Apr 2026 19:37:54 -0300","From":"Jason Gunthorpe ","To":"Bjorn Helgaas ","Cc":"\"yuan.gao\" , Bjorn Helgaas ,\n\tlinux-pci@vger.kernel.org, linux-kernel@vger.kernel.org,\n\tAlex Williamson ","Subject":"Re: [PATCH] PCI: Avoid FLR for NVIDIA 5090 GPU","Message-ID":"<20260416223754.GC761338@nvidia.com>","References":"<20260416070707.3242381-1-yuan.gao@ucloud.cn>\n <20260416173100.GA13378@bhelgaas>","Content-Type":"text/plain; charset=us-ascii","Content-Disposition":"inline","In-Reply-To":"<20260416173100.GA13378@bhelgaas>","X-ClientProxiedBy":"BL1PR13CA0374.namprd13.prod.outlook.com\n (2603:10b6:208:2c0::19) To LV8PR12MB9620.namprd12.prod.outlook.com\n (2603:10b6:408:2a1::19)","Precedence":"bulk","X-Mailing-List":"linux-pci@vger.kernel.org","List-Id":"","List-Subscribe":"","List-Unsubscribe":"","MIME-Version":"1.0","X-MS-PublicTrafficType":"Email","X-MS-TrafficTypeDiagnostic":"LV8PR12MB9620:EE_|DS2PR12MB9751:EE_","X-MS-Office365-Filtering-Correlation-Id":"901b2aa4-5a46-41b7-a0be-08de9c08ce02","X-MS-Exchange-SenderADCheck":"1","X-MS-Exchange-AntiSpam-Relay":"0","X-Microsoft-Antispam":"\n\tBCL:0;ARA:13230040|376014|366016|1800799024|22082099003|56012099003|18002099003;","X-Microsoft-Antispam-Message-Info":"\n\txRbwF3jukWMr+Mn24mVehjdQ7K4WTApJ6lmuccnPqTjVt91M5tM0hKiXDz7LWUXqvDM7mlTuqngZvFmcSlFhV1deM61gwKLKxFEUzHLrfUfjTBZ6Upp17RYCKIuV4HP2EDz4atXP+KV4A2zVTo0nKOz89Li9aE9kRhnVjd+mLS9I+mMxQtfnMEmzGMvkERyum1UsIFD284TP9K6o7FY3rvt+A2+tfHFeB/KgY8TNQZe3vKW/YcZu9AcU2qPhAJmnC6x2AY/VNt3uLQoZz81dH6ZraH2Ex43sq1kbmNLjOjYE6lKyhlfMfS/zbBFWDp70DZaPMcjQKg4+ES8bPL6HDVxc9j0ERUwEADWwjtzCqPH1WYgPJAp8p3EEWQqd/bVmI+P5tesLZzVkidYVrfo6aMaOID98xh5HTJwD79hkiCkQKegEB1IZ4BVVfi8/QsrrzwyVCOD6e0xnXofGrcsuZIecfnniWks7l0mkT4O7Fb5VnO5RpLQNXgMMWcsVJ5LLC9rGnXRCBPe/QNK+u8tpEdJKc40V5WMKC8pXLqGOjwxmrSGoY+ylwVCj8DEETDWFoak+aMf7qKI4MI6FKGSVKZ6GLFhzbWt2LupUwTP2fD3ye0B1RSeNtsJvFefLtu1/useYWTvibfXhdzwygXOEbDeXv2LiuKWji3/9uPTKQA9TWjJ47RXtZsHphwC1t2qxWlcrRModjwuh2pWaxpbWXGckbXRkZ6G3FKbPM69Q6RE=","X-Forefront-Antispam-Report":"\n\tCIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:LV8PR12MB9620.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(376014)(366016)(1800799024)(22082099003)(56012099003)(18002099003);DIR:OUT;SFP:1101;","X-MS-Exchange-AntiSpam-MessageData-ChunkCount":"1","X-MS-Exchange-AntiSpam-MessageData-0":"\n VTfUYFW6tZ9x3/p+m4ROSZ/N+LhWUYLDiT5222nBllZqBgc02axQuYz8PfrOMj0HfrVtMAE+Q5LaerloOxyGkpgsK9IPxhajqQHoCOImnBymbhXeg8ZjD2ASib7BncfO+UVj7gcM8jAl75WppJq5v48Tq0fzJAjRTAUsc9a84lKXWu+VzGZ/Xn3UpKRTGfEoemXElteePR45PVX9joDZ1G2Atbti7jWevRldEdBluMXL4X/H/WMGlGdDBH47n+QbmWudofvd5rWtPmhsI8t/oSUmJQG32mi4mFi1p4YbNJ9RnLcSkdN7zVJEK679ak/NRCTq3HVXbCX4FNNttr29H91BCyqK11jZvIkOM4bSVJRyJ5LY9RJA9rKndu+O/jR4eRCdmDHETKqeQ84wXGULXOTr0keG2g6XTKmXzG3MsY8shnn0dDvujnf7po9rF49RqjuWShGi8smMZAxW4AZ7YPeeofqUL3D1eb8MCFD0ik3yLdqKaZaNYctXEIO/wEV3ooIHSKtIyQldnKKEBsYUvrWp7Bhka3J/kw7Dszj2OqypICX191g3LG1qXFFUbdue3kKms+13CNOdiS+SGNNZ8U7N8gtQpnk+iUA4nDVxhvSW7LzFrPDyOtygVNFBgHpeyiMCGQuGZ0/qJLr2BbXbVGsVXXSqhljCS9MbII8QB5M5KDPwJ2rXX35x+ZA6d6Cue7i1oJ73vb62LZxX4sbU76xHmovOrupJCLY9J0+OBIUUZfYIFtjRR7sGbC+NMydP7YJEeGDTFExFwZb/y31TiFyTA1V6qcdfKoySg23YJ99FwBY58FWKKVuLfXKU6fITdnde0aUUg/KwxzT7eSdqygwtLkRZciQ2AEvJiUnezv4bppM8YcF1JdFd5gWBt9FOqDv5aUn/JNezTWlsfVKrKQXAZUq+XeYuKXjBnOt9ADCj48wy43yyvqrcRbC4+rHJ9IDaYAt1q1GJQWq3docw8HODDUmXHMYPZF6p7S0fj1B9h+nzrGmbRa9kEZoQseAtnHJ3r5Th6eQTMAh5CiLAtUOJ2DuUawF0hiWJfFi8Vu9qnt5PDgejLpLj9+BYVlvqDKReu/ouDic2eFTkiv7GJ0FklDl/LPKunNIqQGZfsTgJUMrR9Fr0I6oul3V1J9Xv0g6vtEluGVDPv+g0XsKX91RgOKncTgImd/QnHfrhldFhAO6ROkjuxW+nx2p1tBftcKSpP4tMXhTDtRwC+6kC96PSimNvovtoQUEbcMRp9I4QKrlxUjN8NHB9vmlZ0ipJRTc59VoajW3mrugZ2SCQ26XLmrGEZ+rS/jmhzIoGxJ/VAmfEq2nEh6P+eqkdTapoFMRDext3dDIdEmMKbsy+5ia2EkKr+IbxSXUwltdkCFfk+t47GG6WBkYFA3yDxJQitw8tRI4F7gXm1jMYtL6x7YXcbsYWhiFlOsmcKscK66FX1poJlqVW0IhKp54Vh6+6jSrVT2Qk+iwhuw6WC40A6VIm13uakXwUKKFSiuE+UI2y6VqSW5nps1ObdSQdS5DfT+rwGRPbjjxBlBZpuUtwZj4x/JdE6oP3uJ70Z3aIgtg6L6eWmmqbPzcRonxUo4dgFkqGtfChFfARDGaG2Gbh2ZHDcBA39F2icvl6/1dZ06LzF+m8kOIJyn7NRtyNR+Y83dnA+a2spD8bTNZzuwKRjerWdQaENR79Ag8q6oRodxLCx0tlvT/PA9MxM2aEhvW6","X-OriginatorOrg":"Nvidia.com","X-MS-Exchange-CrossTenant-Network-Message-Id":"\n 901b2aa4-5a46-41b7-a0be-08de9c08ce02","X-MS-Exchange-CrossTenant-AuthSource":"LV8PR12MB9620.namprd12.prod.outlook.com","X-MS-Exchange-CrossTenant-AuthAs":"Internal","X-MS-Exchange-CrossTenant-OriginalArrivalTime":"16 Apr 2026 22:37:56.4806\n (UTC)","X-MS-Exchange-CrossTenant-FromEntityHeader":"Hosted","X-MS-Exchange-CrossTenant-Id":"43083d15-7273-40c1-b7db-39efd9ccc17a","X-MS-Exchange-CrossTenant-MailboxType":"HOSTED","X-MS-Exchange-CrossTenant-UserPrincipalName":"\n IOBwenCloqT4+F2xnVaYuWrCGmg/0ylO8uHDzvOqaactn9f1oPlxnu5O/7pr3DpF","X-MS-Exchange-Transport-CrossTenantHeadersStamped":"DS2PR12MB9751"}},{"id":3678542,"web_url":"http://patchwork.ozlabs.org/comment/3678542/","msgid":"<20260417020655.zun4qjiu3qsn4nnd@yuangap>","list_archive_url":null,"date":"2026-04-17T02:06:55","subject":"Re: [PATCH] PCI: Avoid FLR for NVIDIA 5090 GPU","submitter":{"id":93167,"url":"http://patchwork.ozlabs.org/api/people/93167/","name":"yuan.gao","email":"yuan.gao@ucloud.cn"},"content":"On Thu, Apr 16, 2026 at 07:37:54PM -0300, Jason Gunthorpe wrote:\n> On Thu, Apr 16, 2026 at 12:31:00PM -0500, Bjorn Helgaas wrote:\n> > On Thu, Apr 16, 2026 at 03:07:06PM +0800, yuan.gao wrote:\n> > > When passing through the NVIDIA 5090 GPU to a vm, there is a certain\n> > > probability of encountering an flr timeout during vm shutdown, which\n> > > subsequently leads to a soft lock of the host cpu.\n> > \n> > If possible, would like confirmation of device erratum from Nvidia.\n> > If there's no known erratum, there might be something wrong in the\n> > Linux FLR and wait.\n> \n> I asked and was told there is a known device firmware defect that\n> causes this.\n> \n> So blanket disabling FLR without detecting good and bad FW is not a\n> good idea.\n> \n> I suggest Yuan try to use an NVIDIA support channel to try to resolve\n> the issue with their card..\n> \n> Jason\nGot it, thanks.\n\nCheers,\nYuan Gao","headers":{"Return-Path":"\n ","X-Original-To":["incoming@patchwork.ozlabs.org","linux-pci@vger.kernel.org"],"Delivered-To":"patchwork-incoming@legolas.ozlabs.org","Authentication-Results":["legolas.ozlabs.org;\n\tdkim=pass (1024-bit key;\n unprotected) header.d=ucloud.cn header.i=@ucloud.cn header.a=rsa-sha256\n header.s=default header.b=i1kQuzSh;\n\tdkim-atps=neutral","legolas.ozlabs.org;\n spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org\n (client-ip=2600:3c0a:e001:db::12fc:5321; helo=sea.lore.kernel.org;\n envelope-from=linux-pci+bounces-52691-incoming=patchwork.ozlabs.org@vger.kernel.org;\n receiver=patchwork.ozlabs.org)","smtp.subspace.kernel.org;\n\tdkim=pass (1024-bit key) header.d=ucloud.cn header.i=@ucloud.cn\n header.b=\"i1kQuzSh\"","smtp.subspace.kernel.org;\n arc=none smtp.client-ip=220.197.31.72","smtp.subspace.kernel.org;\n dmarc=pass (p=quarantine dis=none) header.from=ucloud.cn","smtp.subspace.kernel.org;\n spf=pass smtp.mailfrom=ucloud.cn"],"Received":["from sea.lore.kernel.org (sea.lore.kernel.org\n [IPv6:2600:3c0a:e001:db::12fc:5321])\n\t(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)\n\t key-exchange x25519 server-signature ECDSA (secp384r1) server-digest SHA384)\n\t(No client certificate requested)\n\tby legolas.ozlabs.org (Postfix) with ESMTPS id 4fxnK63bgcz1yGt\n\tfor ; Fri, 17 Apr 2026 17:56:58 +1000 (AEST)","from smtp.subspace.kernel.org (conduit.subspace.kernel.org\n [100.90.174.1])\n\tby sea.lore.kernel.org (Postfix) with ESMTP id BB4573211E59\n\tfor ; Fri, 17 Apr 2026 07:50:12 +0000 (UTC)","from localhost.localdomain (localhost.localdomain [127.0.0.1])\n\tby smtp.subspace.kernel.org (Postfix) with ESMTP id 9E7103815E2;\n\tFri, 17 Apr 2026 07:50:06 +0000 (UTC)","from mail-m1973172.qiye.163.com (mail-m1973172.qiye.163.com\n [220.197.31.72])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))\n\t(No client certificate requested)\n\tby smtp.subspace.kernel.org (Postfix) with ESMTPS id 20A7F37F729;\n\tFri, 17 Apr 2026 07:49:58 +0000 (UTC)","from localhost (unknown [106.75.220.2])\n\tby smtp.qiye.163.com (Hmail) with ESMTP id 1905520db;\n\tFri, 17 Apr 2026 10:06:56 +0800 (GMT+08:00)"],"ARC-Seal":"i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116;\n\tt=1776412204; cv=none;\n b=TZb7j97ozhkQ/sq39WAtVu3zRRMMy+zfR7P2xpE6BRoMLmy9yBe84IJ6bERuoSwU0aUa9ujPRrc+px7jfEakJsCQpcJEZxBCrcXLFE2ISi96qtt2cva+Hvole3/NnBdTXTKDbuG8GqABMIbFmfobrnvAahpGSYGT+mrJ78S01dU=","ARC-Message-Signature":"i=1; a=rsa-sha256; d=subspace.kernel.org;\n\ts=arc-20240116; t=1776412204; c=relaxed/simple;\n\tbh=oVwJ37WsLfBvBVidoPjvHj+PbzMcx0J+WOYOZma3nPs=;\n\th=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version:\n\t Content-Type:Content-Disposition:In-Reply-To;\n b=HooKnul9zEpaqrtvHHUp0xC83kR4C9DsNbY9ZoUBfTpB4EZQDAf24GJgArJkK9OvaZ3PFGlSv5OaleNJQoCiC4iZh9WmXXWEX2FfsGeA2/JuQn9wV2+jgydjZc28eaNtZrh1JGvnC7I6F3fAuhF5m5i2hWkh7tpT3zTJNrp8TUI=","ARC-Authentication-Results":"i=1; smtp.subspace.kernel.org;\n dmarc=pass (p=quarantine dis=none) header.from=ucloud.cn;\n spf=pass smtp.mailfrom=ucloud.cn;\n dkim=pass (1024-bit key) header.d=ucloud.cn header.i=@ucloud.cn\n header.b=i1kQuzSh; arc=none smtp.client-ip=220.197.31.72","Date":"Fri, 17 Apr 2026 10:06:55 +0800","From":"\"yuan.gao\" ","To":"Jason Gunthorpe ","Cc":"Bjorn Helgaas , Bjorn Helgaas ,\n\tlinux-pci@vger.kernel.org, linux-kernel@vger.kernel.org,\n\tAlex Williamson ","Subject":"Re: [PATCH] PCI: Avoid FLR for NVIDIA 5090 GPU","Message-ID":"<20260417020655.zun4qjiu3qsn4nnd@yuangap>","References":"<20260416070707.3242381-1-yuan.gao@ucloud.cn>\n <20260416173100.GA13378@bhelgaas>\n <20260416223754.GC761338@nvidia.com>","Precedence":"bulk","X-Mailing-List":"linux-pci@vger.kernel.org","List-Id":"","List-Subscribe":"","List-Unsubscribe":"","MIME-Version":"1.0","Content-Type":"text/plain; charset=us-ascii","Content-Disposition":"inline","In-Reply-To":"<20260416223754.GC761338@nvidia.com>","X-HM-Tid":"0a9d9930f5d70229kunm9e0eb86b13c871","X-HM-MType":"1","X-HM-Spam-Status":"e1kfGhgUHx5ZQUpXWQgPGg8OCBgUHx5ZQUlOS1dZFg8aDwILHllBWSg2Ly\n\ttZV1koWUFJQjdXWS1ZQUlXWQ8JGhUIEh9ZQVlDS0seVh8fTEhJSU5MH0tOQlYVFAkWGhdVGRETFh\n\toSFyQUDg9ZV1kYEgtZQVlKS01VTE5VSUlLVUlZV1kWGg8SFR0UWUFZT0tIVUpLSU9PT0hVSktLVU\n\tpCS0tZBg++","DKIM-Signature":"a=rsa-sha256;\n\tb=i1kQuzShgsCW2r3wrSR5EZa+ugiQX1MFNPFq9TaKwZgV3MSinf6P242O2/XhmaYDl9FRN7oVhvo1xAjFvcjZ/lMe6TsPSsqSMQkredhQjffklisHMS47iAV6YQiTyZSX8lXyTbjqbyc9O85SkbTJbFgJQNdg3hMtNEYi+nQdTa4=;\n c=relaxed/relaxed; s=default; d=ucloud.cn; v=1;\n\tbh=bfqXgrnYB5A/hDRBS5OQqI7LAV26xC87aPYM7fgT1Uk=;\n\th=date:mime-version:subject:message-id:from;"}}]