[{"id":3682822,"web_url":"http://patchwork.ozlabs.org/comment/3682822/","msgid":"","list_archive_url":null,"date":"2026-04-27T16:32:41","subject":"Re: [PATCH v4 2/3] PCI: Allow ATS to be always on for pre-CXL devices","submitter":{"id":13225,"url":"http://patchwork.ozlabs.org/api/people/13225/","name":"Dave Jiang","email":"dave.jiang@intel.com"},"content":"On 4/26/26 10:54 PM, Nicolin Chen wrote:\n> Some NVIDIA GPU/NIC devices, though they don't implement CXL config space,\n> have many CXL-like properties. Call this kind \"pre-CXL\".\n> \n> Similar to CXL.cache capability, these pre-CXL devices also require the ATS\n> function even when their RIDs are IOMMU bypassed, i.e. keep ATS \"always on\"\n> v.s. \"on demand\" when a non-zero PASID line gets enabled in SVA use cases.\n> \n> Introduce pci_dev_specific_ats_always_on() quirk function to scan a list of\n> IDs for these devices. Then, include it in pci_ats_always_on().\n> \n> Suggested-by: Jason Gunthorpe \n> Reviewed-by: Nirmoy Das \n> Tested-by: Nirmoy Das \n> Reviewed-by: Jonathan Cameron \n> Reviewed-by: Jason Gunthorpe \n> Reviewed-by: Kevin Tian \n> Signed-off-by: Nicolin Chen \n\nReviewed-by: Dave Jiang \n\n\n> ---\n> drivers/pci/pci.h | 9 +++++++++\n> drivers/pci/ats.c | 3 ++-\n> drivers/pci/quirks.c | 38 ++++++++++++++++++++++++++++++++++++++\n> 3 files changed, 49 insertions(+), 1 deletion(-)\n> \n> diff --git a/drivers/pci/pci.h b/drivers/pci/pci.h\n> index 4a14f88e543a2..4e0077478cd7a 100644\n> --- a/drivers/pci/pci.h\n> +++ b/drivers/pci/pci.h\n> @@ -1155,6 +1155,15 @@ static inline int pci_dev_specific_reset(struct pci_dev *dev, bool probe)\n> }\n> #endif\n> \n> +#if defined(CONFIG_PCI_QUIRKS) && defined(CONFIG_PCI_ATS)\n> +bool pci_dev_specific_ats_always_on(struct pci_dev *dev);\n> +#else\n> +static inline bool pci_dev_specific_ats_always_on(struct pci_dev *dev)\n> +{\n> +\treturn false;\n> +}\n> +#endif\n> +\n> #if defined(CONFIG_PCI_QUIRKS) && defined(CONFIG_ARM64)\n> int acpi_get_rc_resources(struct device *dev, const char *hid, u16 segment,\n> \t\t\t struct resource *res);\n> diff --git a/drivers/pci/ats.c b/drivers/pci/ats.c\n> index fc871858b65bc..3846447ea322f 100644\n> --- a/drivers/pci/ats.c\n> +++ b/drivers/pci/ats.c\n> @@ -244,7 +244,8 @@ bool pci_ats_always_on(struct pci_dev *pdev)\n> \tif (pdev->is_virtfn)\n> \t\tpdev = pci_physfn(pdev);\n> \n> -\treturn pci_cxl_ats_always_on(pdev);\n> +\treturn pci_cxl_ats_always_on(pdev) ||\n> +\t pci_dev_specific_ats_always_on(pdev);\n> }\n> EXPORT_SYMBOL_GPL(pci_ats_always_on);\n> \n> diff --git a/drivers/pci/quirks.c b/drivers/pci/quirks.c\n> index caaed1a01dc02..887babba97cc7 100644\n> --- a/drivers/pci/quirks.c\n> +++ b/drivers/pci/quirks.c\n> @@ -5715,6 +5715,44 @@ DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_INTEL, 0x1457, quirk_intel_e2000_no_ats);\n> DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_INTEL, 0x1459, quirk_intel_e2000_no_ats);\n> DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_INTEL, 0x145a, quirk_intel_e2000_no_ats);\n> DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_INTEL, 0x145c, quirk_intel_e2000_no_ats);\n> +\n> +static bool quirk_nvidia_gpu_ats_always_on(struct pci_dev *pdev)\n> +{\n> +\tswitch (pdev->device) {\n> +\tcase 0x2e00 ... 0x2e3f: /* GB20B */\n> +\t\treturn true;\n> +\t}\n> +\treturn false;\n> +}\n> +\n> +static const struct pci_dev_ats_always_on {\n> +\tu16 vendor;\n> +\tu16 device;\n> +\tbool (*ats_always_on)(struct pci_dev *dev);\n> +} pci_dev_ats_always_on[] = {\n> +\t/* NVIDIA GPUs */\n> +\t{ PCI_VENDOR_ID_NVIDIA, PCI_ANY_ID, quirk_nvidia_gpu_ats_always_on },\n> +\t/* NVIDIA CX10 Family NVlink-C2C */\n> +\t{ PCI_VENDOR_ID_MELLANOX, 0x2101, NULL },\n> +\t{ 0 }\n> +};\n> +\n> +/* Some pre-CXL devices require ATS when it is IOMMU-bypassed */\n> +bool pci_dev_specific_ats_always_on(struct pci_dev *pdev)\n> +{\n> +\tconst struct pci_dev_ats_always_on *i;\n> +\n> +\tfor (i = pci_dev_ats_always_on; i->vendor; i++) {\n> +\t\tif (i->vendor != pdev->vendor)\n> +\t\t\tcontinue;\n> +\t\tif (i->ats_always_on && i->ats_always_on(pdev))\n> +\t\t\treturn true;\n> +\t\tif (!i->ats_always_on && i->device == pdev->device)\n> +\t\t\treturn true;\n> +\t}\n> +\n> +\treturn false;\n> +}\n> #endif /* CONFIG_PCI_ATS */\n> \n> /* Freescale PCIe doesn't support MSI in RC mode */","headers":{"Return-Path":"\n ","X-Original-To":["incoming@patchwork.ozlabs.org","linux-pci@vger.kernel.org"],"Delivered-To":"patchwork-incoming@legolas.ozlabs.org","Authentication-Results":["legolas.ozlabs.org;\n\tdkim=pass (2048-bit key;\n unprotected) header.d=intel.com header.i=@intel.com header.a=rsa-sha256\n header.s=Intel header.b=SLV7Dpht;\n\tdkim-atps=neutral","legolas.ozlabs.org;\n spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org\n (client-ip=2600:3c04:e001:36c::12fc:5321; helo=tor.lore.kernel.org;\n envelope-from=linux-pci+bounces-53233-incoming=patchwork.ozlabs.org@vger.kernel.org;\n receiver=patchwork.ozlabs.org)","smtp.subspace.kernel.org;\n\tdkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com\n header.b=\"SLV7Dpht\"","smtp.subspace.kernel.org;\n arc=none smtp.client-ip=198.175.65.21","smtp.subspace.kernel.org;\n dmarc=pass (p=none dis=none) header.from=intel.com","smtp.subspace.kernel.org;\n spf=pass smtp.mailfrom=intel.com"],"Received":["from tor.lore.kernel.org (tor.lore.kernel.org\n [IPv6:2600:3c04:e001:36c::12fc:5321])\n\t(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)\n\t key-exchange x25519)\n\t(No client certificate requested)\n\tby legolas.ozlabs.org (Postfix) with ESMTPS id 4g48JM58S3z1xvV\n\tfor ; Tue, 28 Apr 2026 02:33:23 +1000 (AEST)","from smtp.subspace.kernel.org (conduit.subspace.kernel.org\n [100.90.174.1])\n\tby tor.lore.kernel.org (Postfix) with ESMTP id 8BB5F3017258\n\tfor ; Mon, 27 Apr 2026 16:32:46 +0000 (UTC)","from localhost.localdomain (localhost.localdomain [127.0.0.1])\n\tby smtp.subspace.kernel.org (Postfix) with ESMTP id BAAEF3CF042;\n\tMon, 27 Apr 2026 16:32:45 +0000 (UTC)","from mgamail.intel.com (mgamail.intel.com [198.175.65.21])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))\n\t(No client certificate requested)\n\tby smtp.subspace.kernel.org (Postfix) with ESMTPS id 780B3359A65;\n\tMon, 27 Apr 2026 16:32:43 +0000 (UTC)","from orviesa001.jf.intel.com ([10.64.159.141])\n by orvoesa113.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384;\n 27 Apr 2026 09:32:44 -0700","from msatwood-mobl.amr.corp.intel.com (HELO [10.125.108.172])\n ([10.125.108.172])\n by smtpauth.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384;\n 27 Apr 2026 09:32:42 -0700"],"ARC-Seal":"i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116;\n\tt=1777307565; cv=none;\n b=FCIdU0hUI4KGH//Rx/Ma5BViO4qqIJsDxgSkHtk/6Tl5trfcs2qj+eduZ0oB67GLh6Oavjwjfnj6rQ4VCfc0ICRam0nE+0Swcglv0aGY9GJ8lORixWYnkUGWQ7hZmC50n4pG581vja0l6S4h+BZUs7Bxcw+g7DfnWzkFHcxK/io=","ARC-Message-Signature":"i=1; a=rsa-sha256; d=subspace.kernel.org;\n\ts=arc-20240116; t=1777307565; c=relaxed/simple;\n\tbh=Y0roxTXaVRo9dgpe+WBRbf1u7DnmhCufjiiekE3A1aw=;\n\th=Message-ID:Date:MIME-Version:Subject:To:Cc:References:From:\n\t In-Reply-To:Content-Type;\n b=hk6kNZj69uAPwn0kznyyZ0Uop2Kc1qXwv7SWi/pPWxnEYR9EAXv6fTBbxkNit3O9upeEmWyHo+gdGRyiOy3Ce+vIJxtdoZuQ1A90Qpa6yOcfW7lnUj2HkBN/aj9ezPyT1qZIeND5QfIgjGmcxaP2qrKtGrxDFCFrpMiTl0jfqUw=","ARC-Authentication-Results":"i=1; smtp.subspace.kernel.org;\n dmarc=pass (p=none dis=none) header.from=intel.com;\n spf=pass smtp.mailfrom=intel.com;\n dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com\n header.b=SLV7Dpht; arc=none smtp.client-ip=198.175.65.21","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple;\n d=intel.com; i=@intel.com; q=dns/txt; s=Intel;\n t=1777307564; x=1808843564;\n h=message-id:date:mime-version:subject:to:cc:references:\n from:in-reply-to:content-transfer-encoding;\n bh=Y0roxTXaVRo9dgpe+WBRbf1u7DnmhCufjiiekE3A1aw=;\n b=SLV7DphtTtGQYhRq7XWALzd68nBG3sz6HrCNNVGe0CSwASMnZLNU5ZGN\n H6+gdtdNRv+wJTthNLMk7Xfo6H2+UqCInKG3gY+/fukvhBQ7daWeTnvIW\n /WTaw/Movrk/7Hy9tmG+pBzRJ7D0ewrhcOhyU43+KHksLWc1qp05zAGgx\n IHVmja+vGabMieqA/o5zdh5RsE71OYK2SyFftbjt3WGxDK22VVy5wVE3I\n AOkj2jKVZgoy5lrwD2ApzhgvAmot8EFNCc7K54P0xB0chm2sFUHxWZM5D\n czkoGFE9uZY0xvI6xMFBuUi8bGas6gLIo86teYFLx8bG0xzpZPLQxZv5I\n g==;","X-CSE-ConnectionGUID":["uBT0RpetTsGzzS4WXzwoRQ==","1RUd+HJWR5uLHpUSBZOvmg=="],"X-CSE-MsgGUID":["QEmxxlv1RYSTlzoRyTDCuA==","cCHic7IMRuiseDsi8VPeig=="],"X-IronPort-AV":["E=McAfee;i=\"6800,10657,11769\"; a=\"78093720\"","E=Sophos;i=\"6.23,202,1770624000\";\n d=\"scan'208\";a=\"78093720\"","E=Sophos;i=\"6.23,202,1770624000\";\n d=\"scan'208\";a=\"271816409\""],"X-ExtLoop1":"1","Message-ID":"","Date":"Mon, 27 Apr 2026 09:32:41 -0700","Precedence":"bulk","X-Mailing-List":"linux-pci@vger.kernel.org","List-Id":"","List-Subscribe":"","List-Unsubscribe":"","MIME-Version":"1.0","User-Agent":"Mozilla Thunderbird","Subject":"Re: [PATCH v4 2/3] PCI: Allow ATS to be always on for pre-CXL devices","To":"Nicolin Chen , jgg@nvidia.com, will@kernel.org,\n robin.murphy@arm.com, bhelgaas@google.com","Cc":"joro@8bytes.org, praan@google.com, baolu.lu@linux.intel.com,\n kevin.tian@intel.com, miko.lenczewski@arm.com,\n linux-arm-kernel@lists.infradead.org, iommu@lists.linux.dev,\n linux-kernel@vger.kernel.org, linux-pci@vger.kernel.org,\n dan.j.williams@intel.com, jonathan.cameron@huawei.com, vsethi@nvidia.com,\n linux-cxl@vger.kernel.org, nirmoyd@nvidia.com","References":"\n <1a8cf5e88051ab5c10417edb94df598ecbc810cf.1777269009.git.nicolinc@nvidia.com>","Content-Language":"en-US","From":"Dave Jiang ","In-Reply-To":"\n <1a8cf5e88051ab5c10417edb94df598ecbc810cf.1777269009.git.nicolinc@nvidia.com>","Content-Type":"text/plain; charset=UTF-8","Content-Transfer-Encoding":"7bit"}}]