[{"id":3669304,"web_url":"http://patchwork.ozlabs.org/comment/3669304/","msgid":"<20260325200835.GA1290451@bhelgaas>","list_archive_url":null,"date":"2026-03-25T20:08:35","subject":"Re: [PATCH v6 1/2] PCI: AtomicOps: Do not enable without support in\n root complex","submitter":{"id":67298,"url":"http://patchwork.ozlabs.org/api/people/67298/","name":"Bjorn Helgaas","email":"helgaas@kernel.org"},"content":"On Wed, Mar 25, 2026 at 04:16:17PM +0100, Gerd Bayer wrote:\n> When inspecting the config space of a Connect-X physical function in an\n> s390 system after it was initialized by the mlx5_core device driver, we\n> found the function to be enabled to request AtomicOps despite the\n> system's root-complex lacking support for completing them:\n> \n> 1ed0:00:00.1 Ethernet controller: Mellanox Technologies MT2894 Family [ConnectX-6 Lx]\n> \tSubsystem: Mellanox Technologies Device 0002\n> [...]\n> \tDevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-\n> \t\t AtomicOpsCtl: ReqEn+\n> \t\t IDOReq- IDOCompl- LTR- EmergencyPowerReductionReq-\n> \t\t 10BitTagReq- OBFF Disabled, EETLPPrefixBlk-\n> \n> Turns out the device driver calls pci_enable_atomic_ops_to_root() which\n> defaulted to enable AtomicOps requests even if it had no information\n> about the root-port that the PCIe device is attached to. Similarly,\n> AtomicOps requests are enabled for root complex integrated endpoints\n> (RCiEPs) unconditionally.\n> \n> Change the logic of pci_enable_atomic_ops_to_root() to fully traverse the\n> PCIe tree upwards, check that the bridge devices support delivering\n> AtomicOps transactions, and finally check that there is a root port at\n> the end that does support completing AtomicOps - or that the support for\n> completing AtomicOps at the root complex is announced through some other\n> arch specific way.\n> \n> Introduce a new pcibios_connects_to_atomicops_capable_rc() function to\n> implement the check - and default to always \"true\". This leaves the\n> semantics for today's RCiEPs intact. Pass in the device in question and\n> the requested capabilities for future expansions.\n> For s390, override pcibios_connects_to_atomicops_capable_rc() to\n> always return \"false\".\n> \n> Do not change the enablement of AtomicOps requests if there is no\n> positive confirmation that the root complex can complete PCIe AtomicOps.\n> \n> Reported-by: Alexander Schmidt \n> Cc: stable@vger.kernel.org\n> Fixes: 430a23689dea (\"PCI: Add pci_enable_atomic_ops_to_root()\")\n> Signed-off-by: Gerd Bayer \n> ---\n> arch/s390/pci/pci.c | 5 +++++\n> drivers/pci/pci.c | 48 +++++++++++++++++++++++++++++++-----------------\n> include/linux/pci.h | 1 +\n> 3 files changed, 37 insertions(+), 17 deletions(-)\n> \n> diff --git a/arch/s390/pci/pci.c b/arch/s390/pci/pci.c\n> index 2a430722cbe415dd56c92fed2e513e524f46481a..a0bef77082a153a258fbe4abb1070b22e020888e 100644\n> --- a/arch/s390/pci/pci.c\n> +++ b/arch/s390/pci/pci.c\n> @@ -265,6 +265,11 @@ static int zpci_cfg_store(struct zpci_dev *zdev, int offset, u32 val, u8 len)\n> \treturn rc;\n> }\n> \n> +bool pcibios_connects_to_atomicops_capable_rc(struct pci_dev *dev, u32 cap_mask)\n> +{\n> +\treturn false;\n> +}\n> +\n> resource_size_t pcibios_align_resource(void *data, const struct resource *res,\n> \t\t\t\t resource_size_t size,\n> \t\t\t\t resource_size_t align)\n> diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c\n> index 8479c2e1f74f1044416281aba11bf071ea89488a..006aa589926cb290de43f152100ddaf9961407d1 100644\n> --- a/drivers/pci/pci.c\n> +++ b/drivers/pci/pci.c\n> @@ -3660,6 +3660,19 @@ void pci_acs_init(struct pci_dev *dev)\n> \tpci_disable_broken_acs_cap(dev);\n> }\n> \n> +static bool pci_is_atomicops_capable_rp(struct pci_dev *dev, u32 cap, u32 cap_mask)\n> +{\n> +\tif (!dev || !(pci_pcie_type(dev) == PCI_EXP_TYPE_ROOT_PORT))\n> +\t\treturn false;\n> +\n> +\treturn (cap & cap_mask) == cap_mask;\n> +}\n> +\n> +bool __weak pcibios_connects_to_atomicops_capable_rc(struct pci_dev *dev, u32 cap_mask)\n> +{\n> +\treturn true;\n> +}\n> +\n> /**\n> * pci_enable_atomic_ops_to_root - enable AtomicOp requests to root port\n> * @dev: the PCI device\n> @@ -3676,8 +3689,9 @@ void pci_acs_init(struct pci_dev *dev)\n> int pci_enable_atomic_ops_to_root(struct pci_dev *dev, u32 cap_mask)\n> {\n> \tstruct pci_bus *bus = dev->bus;\n> -\tstruct pci_dev *bridge;\n> -\tu32 cap, ctl2;\n> +\tstruct pci_dev *bridge = NULL;\n> +\tu32 cap = 0;\n> +\tu32 ctl2;\n> \n> \t/*\n> \t * Per PCIe r5.0, sec 9.3.5.10, the AtomicOp Requester Enable bit\n> @@ -3714,29 +3728,29 @@ int pci_enable_atomic_ops_to_root(struct pci_dev *dev, u32 cap_mask)\n> \t\tswitch (pci_pcie_type(bridge)) {\n> \t\t/* Ensure switch ports support AtomicOp routing */\n> \t\tcase PCI_EXP_TYPE_UPSTREAM:\n> -\t\tcase PCI_EXP_TYPE_DOWNSTREAM:\n> -\t\t\tif (!(cap & PCI_EXP_DEVCAP2_ATOMIC_ROUTE))\n> -\t\t\t\treturn -EINVAL;\n> -\t\t\tbreak;\n> -\n> -\t\t/* Ensure root port supports all the sizes we care about */\n> -\t\tcase PCI_EXP_TYPE_ROOT_PORT:\n> -\t\t\tif ((cap & cap_mask) != cap_mask)\n> -\t\t\t\treturn -EINVAL;\n> -\t\t\tbreak;\n> -\t\t}\n> -\n> -\t\t/* Ensure upstream ports don't block AtomicOps on egress */\n> -\t\tif (pci_pcie_type(bridge) == PCI_EXP_TYPE_UPSTREAM) {\n> +\t\t\t/* Upstream ports must not block AtomicOps on egress */\n> \t\t\tpcie_capability_read_dword(bridge, PCI_EXP_DEVCTL2,\n> \t\t\t\t\t\t &ctl2);\n> \t\t\tif (ctl2 & PCI_EXP_DEVCTL2_ATOMIC_EGRESS_BLOCK)\n> \t\t\t\treturn -EINVAL;\n> +\t\t\tfallthrough;\n> +\t\t/* All switch ports need to route AtomicOps */\n> +\t\tcase PCI_EXP_TYPE_DOWNSTREAM:\n> +\t\t\tif (!(cap & PCI_EXP_DEVCAP2_ATOMIC_ROUTE))\n> +\t\t\t\treturn -EINVAL;\n> +\t\t\tbreak;\n> \t\t}\n> -\n> \t\tbus = bus->parent;\n> \t}\n> \n> +\t/*\n> +\t * Finally, last bridge must be root port and support requested sizes\n> +\t * or firmware asserts support\n> +\t */\n> +\tif (!(pci_is_atomicops_capable_rp(bridge, cap, cap_mask) ||\n> +\t pcibios_connects_to_atomicops_capable_rc(dev, cap_mask)))\n> +\t\treturn -EINVAL;\n\nSashiko says:\n\n Since the generic weak implementation of\n pcibios_connects_to_atomicops_capable_rc() unconditionally returns\n true, the logical OR expression pci_is_atomicops_capable_rp(...) ||\n true will always evaluate to true. This makes the entire if\n condition evaluate to false.\n\n Because of this, it appears -EINVAL is never returned here, and any\n standard endpoint behind a Root Port will successfully be granted\n AtomicOps even if the Root Port lacks the capability in its\n PCI_EXP_DEVCAP2 register.\n\n> +\n> \tpcie_capability_set_word(dev, PCI_EXP_DEVCTL2,\n> \t\t\t\t PCI_EXP_DEVCTL2_ATOMIC_REQ);\n> \treturn 0;\n> diff --git a/include/linux/pci.h b/include/linux/pci.h\n> index 1c270f1d512301de4d462fe7e5097c32af5c6f8d..ef90604c39859ea8e61e5392d0bdaa1b0e43874b 100644\n> --- a/include/linux/pci.h\n> +++ b/include/linux/pci.h\n> @@ -692,6 +692,7 @@ void pci_set_host_bridge_release(struct pci_host_bridge *bridge,\n> \t\t\t\t void *release_data);\n> \n> int pcibios_root_bridge_prepare(struct pci_host_bridge *bridge);\n> +bool pcibios_connects_to_atomicops_capable_rc(struct pci_dev *dev, u32 cap_mask);\n> \n> #define PCI_REGION_FLAG_MASK\t0x0fU\t/* These bits of resource flags tell us the PCI region flags */\n> \n> \n> -- \n> 2.51.0\n>","headers":{"Return-Path":"\n ","X-Original-To":["incoming@patchwork.ozlabs.org","linux-pci@vger.kernel.org"],"Delivered-To":"patchwork-incoming@legolas.ozlabs.org","Authentication-Results":["legolas.ozlabs.org;\n\tdkim=pass (2048-bit key;\n unprotected) header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256\n header.s=k20201202 header.b=Z9Kn5rjL;\n\tdkim-atps=neutral","legolas.ozlabs.org;\n spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org\n (client-ip=2600:3c04:e001:36c::12fc:5321; helo=tor.lore.kernel.org;\n envelope-from=linux-pci+bounces-51127-incoming=patchwork.ozlabs.org@vger.kernel.org;\n receiver=patchwork.ozlabs.org)","smtp.subspace.kernel.org;\n\tdkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org\n header.b=\"Z9Kn5rjL\"","smtp.subspace.kernel.org;\n arc=none smtp.client-ip=10.30.226.201"],"Received":["from tor.lore.kernel.org (tor.lore.kernel.org\n [IPv6:2600:3c04:e001:36c::12fc:5321])\n\t(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)\n\t key-exchange x25519 server-signature ECDSA (secp384r1) server-digest SHA384)\n\t(No client certificate requested)\n\tby legolas.ozlabs.org (Postfix) with ESMTPS id 4fgyfF1GP4z1xy1\n\tfor ; Thu, 26 Mar 2026 07:08:53 +1100 (AEDT)","from smtp.subspace.kernel.org (conduit.subspace.kernel.org\n [100.90.174.1])\n\tby tor.lore.kernel.org (Postfix) with ESMTP id 044C23040017\n\tfor ; Wed, 25 Mar 2026 20:08:41 +0000 (UTC)","from localhost.localdomain (localhost.localdomain [127.0.0.1])\n\tby smtp.subspace.kernel.org (Postfix) with ESMTP id C33D63537C1;\n\tWed, 25 Mar 2026 20:08:37 +0000 (UTC)","from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org\n [10.30.226.201])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))\n\t(No client certificate requested)\n\tby smtp.subspace.kernel.org (Postfix) with ESMTPS id 9D7C4F4F1;\n\tWed, 25 Mar 2026 20:08:37 +0000 (UTC)","by smtp.kernel.org (Postfix) with ESMTPSA id 06011C4CEF7;\n\tWed, 25 Mar 2026 20:08:36 +0000 (UTC)"],"ARC-Seal":"i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116;\n\tt=1774469317; cv=none;\n b=Utloc6mUignDphi2XTgUiPBJhnRaSo6d6GMq8RX3wyRWoFnvtzGltqZIW6PFfMj+ArU9K+lwKdANLnCslBQ7tO2QN0wETW4hv0kqepWGjAi1COU6hoSSOt71P6CGl43qWvqMghWDk7sSB2QXfanZUGwVF10GO/paL3/KLIHEW04=","ARC-Message-Signature":"i=1; a=rsa-sha256; d=subspace.kernel.org;\n\ts=arc-20240116; t=1774469317; c=relaxed/simple;\n\tbh=u0Cd5omrwW3YhsWw9m4ecHl9huofQib83nNWmHHNvaM=;\n\th=Date:From:To:Cc:Subject:Message-ID:MIME-Version:Content-Type:\n\t Content-Disposition:In-Reply-To;\n b=g/ZEWvVUvtDougtFY+CmzeCM7/05Ly55P1SWq/v52HNPdueXyWrNL42CbJbIsPmZGUrwzLUg4LE03FoWbrrS3Ob3Ep/HIj7yuS86ZY4Dn3dd75w2LjkQV+LVFPr+AQvsw0EZjmi9H5lyFA/YHFTMPwhauj+xuFtUI26i92YtHnI=","ARC-Authentication-Results":"i=1; smtp.subspace.kernel.org;\n dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org\n header.b=Z9Kn5rjL; arc=none smtp.client-ip=10.30.226.201","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;\n\ts=k20201202; t=1774469317;\n\tbh=u0Cd5omrwW3YhsWw9m4ecHl9huofQib83nNWmHHNvaM=;\n\th=Date:From:To:Cc:Subject:In-Reply-To:From;\n\tb=Z9Kn5rjLrhwK1kdqyaXdq9wyvg4ZxB1tPZ9jrqFS7RwJK2jJUBL+K3GypmqN49WUc\n\t 7DF+RGwWI5JuMyhsC0VEXPeh93v1wJO7JvfeAJO0ldnRt6wJfPjLvSBUQai2E9Z94y\n\t rYO+BbiXWGd1o+13OdefEL0nuHWpvX+F0iemDZ2+CKOk6mHrz7Le9Q+mA03UxWfvFO\n\t VB5JM9Ml5U+22RVNF76VLqtK+gMk2Oz4adIOgGsd3tIKIV+b0RbOP1CEW5SPVZZMvh\n\t FGpia0iUKWLj+azTSqC1objNUboZrH/JXpm/l94LrGqBGhna07ebvTkNnVrNiN6EjE\n\t Vh7QS5w2CLHOg==","Date":"Wed, 25 Mar 2026 15:08:35 -0500","From":"Bjorn Helgaas ","To":"Gerd Bayer ","Cc":"Bjorn Helgaas ,\n\tJay Cornwall ,\n\tFelix Kuehling ,\n\tChristian Borntraeger ,\n\tNiklas Schnelle ,\n\tGerald Schaefer ,\n\tHeiko Carstens ,\n\tVasily Gorbik ,\n\tAlexander Gordeev ,\n\tSven Schnelle ,\n\tLeon Romanovsky ,\n\tAlexander Schmidt , linux-s390@vger.kernel.org,\n\tlinux-pci@vger.kernel.org, linux-kernel@vger.kernel.org,\n\tnetdev@vger.kernel.org, linux-rdma@vger.kernel.org,\n\tstable@vger.kernel.org","Subject":"Re: [PATCH v6 1/2] PCI: AtomicOps: Do not enable without support in\n root complex","Message-ID":"<20260325200835.GA1290451@bhelgaas>","Precedence":"bulk","X-Mailing-List":"linux-pci@vger.kernel.org","List-Id":"","List-Subscribe":"","List-Unsubscribe":"","MIME-Version":"1.0","Content-Type":"text/plain; charset=us-ascii","Content-Disposition":"inline","In-Reply-To":"<20260325-fix_pciatops-v6-1-10bf19d76dd1@linux.ibm.com>"}},{"id":3669592,"web_url":"http://patchwork.ozlabs.org/comment/3669592/","msgid":"<4183b471777efa949ce6f7b860c81282e91666ef.camel@linux.ibm.com>","list_archive_url":null,"date":"2026-03-26T09:51:19","subject":"Re: [PATCH v6 1/2] PCI: AtomicOps: Do not enable without support in\n root complex","submitter":{"id":85804,"url":"http://patchwork.ozlabs.org/api/people/85804/","name":"Gerd Bayer","email":"gbayer@linux.ibm.com"},"content":"On Wed, 2026-03-25 at 15:08 -0500, Bjorn Helgaas wrote:\n> On Wed, Mar 25, 2026 at 04:16:17PM +0100, Gerd Bayer wrote:\n> > When inspecting the config space of a Connect-X physical function in an\n> > s390 system after it was initialized by the mlx5_core device driver, we\n> > found the function to be enabled to request AtomicOps despite the\n> > system's root-complex lacking support for completing them:\n> > \n> > 1ed0:00:00.1 Ethernet controller: Mellanox Technologies MT2894 Family [ConnectX-6 Lx]\n> > \tSubsystem: Mellanox Technologies Device 0002\n> > [...]\n> > \tDevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-\n> > \t\t AtomicOpsCtl: ReqEn+\n> > \t\t IDOReq- IDOCompl- LTR- EmergencyPowerReductionReq-\n> > \t\t 10BitTagReq- OBFF Disabled, EETLPPrefixBlk-\n> > \n> > Turns out the device driver calls pci_enable_atomic_ops_to_root() which\n> > defaulted to enable AtomicOps requests even if it had no information\n> > about the root-port that the PCIe device is attached to. Similarly,\n> > AtomicOps requests are enabled for root complex integrated endpoints\n> > (RCiEPs) unconditionally.\n> > \n> > Change the logic of pci_enable_atomic_ops_to_root() to fully traverse the\n> > PCIe tree upwards, check that the bridge devices support delivering\n> > AtomicOps transactions, and finally check that there is a root port at\n> > the end that does support completing AtomicOps - or that the support for\n> > completing AtomicOps at the root complex is announced through some other\n> > arch specific way.\n> > \n> > Introduce a new pcibios_connects_to_atomicops_capable_rc() function to\n> > implement the check - and default to always \"true\". This leaves the\n> > semantics for today's RCiEPs intact. Pass in the device in question and\n> > the requested capabilities for future expansions.\n> > For s390, override pcibios_connects_to_atomicops_capable_rc() to\n> > always return \"false\".\n> > \n> > Do not change the enablement of AtomicOps requests if there is no\n> > positive confirmation that the root complex can complete PCIe AtomicOps.\n> > \n> > Reported-by: Alexander Schmidt \n> > Cc: stable@vger.kernel.org\n> > Fixes: 430a23689dea (\"PCI: Add pci_enable_atomic_ops_to_root()\")\n> > Signed-off-by: Gerd Bayer \n> > ---\n> > arch/s390/pci/pci.c | 5 +++++\n> > drivers/pci/pci.c | 48 +++++++++++++++++++++++++++++++-----------------\n> > include/linux/pci.h | 1 +\n> > 3 files changed, 37 insertions(+), 17 deletions(-)\n> > \n> > diff --git a/arch/s390/pci/pci.c b/arch/s390/pci/pci.c\n> > index 2a430722cbe415dd56c92fed2e513e524f46481a..a0bef77082a153a258fbe4abb1070b22e020888e 100644\n> > --- a/arch/s390/pci/pci.c\n> > +++ b/arch/s390/pci/pci.c\n> > @@ -265,6 +265,11 @@ static int zpci_cfg_store(struct zpci_dev *zdev, int offset, u32 val, u8 len)\n> > \treturn rc;\n> > }\n> > \n> > +bool pcibios_connects_to_atomicops_capable_rc(struct pci_dev *dev, u32 cap_mask)\n> > +{\n> > +\treturn false;\n> > +}\n> > +\n> > resource_size_t pcibios_align_resource(void *data, const struct resource *res,\n> > \t\t\t\t resource_size_t size,\n> > \t\t\t\t resource_size_t align)\n> > diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c\n> > index 8479c2e1f74f1044416281aba11bf071ea89488a..006aa589926cb290de43f152100ddaf9961407d1 100644\n> > --- a/drivers/pci/pci.c\n> > +++ b/drivers/pci/pci.c\n> > @@ -3660,6 +3660,19 @@ void pci_acs_init(struct pci_dev *dev)\n> > \tpci_disable_broken_acs_cap(dev);\n> > }\n> > \n> > +static bool pci_is_atomicops_capable_rp(struct pci_dev *dev, u32 cap, u32 cap_mask)\n> > +{\n> > +\tif (!dev || !(pci_pcie_type(dev) == PCI_EXP_TYPE_ROOT_PORT))\n> > +\t\treturn false;\n> > +\n> > +\treturn (cap & cap_mask) == cap_mask;\n> > +}\n> > +\n> > +bool __weak pcibios_connects_to_atomicops_capable_rc(struct pci_dev *dev, u32 cap_mask)\n> > +{\n> > +\treturn true;\n> > +}\n> > +\n> > /**\n> > * pci_enable_atomic_ops_to_root - enable AtomicOp requests to root port\n> > * @dev: the PCI device\n> > @@ -3676,8 +3689,9 @@ void pci_acs_init(struct pci_dev *dev)\n> > int pci_enable_atomic_ops_to_root(struct pci_dev *dev, u32 cap_mask)\n> > {\n> > \tstruct pci_bus *bus = dev->bus;\n> > -\tstruct pci_dev *bridge;\n> > -\tu32 cap, ctl2;\n> > +\tstruct pci_dev *bridge = NULL;\n> > +\tu32 cap = 0;\n> > +\tu32 ctl2;\n> > \n> > \t/*\n> > \t * Per PCIe r5.0, sec 9.3.5.10, the AtomicOp Requester Enable bit\n> > @@ -3714,29 +3728,29 @@ int pci_enable_atomic_ops_to_root(struct pci_dev *dev, u32 cap_mask)\n> > \t\tswitch (pci_pcie_type(bridge)) {\n> > \t\t/* Ensure switch ports support AtomicOp routing */\n> > \t\tcase PCI_EXP_TYPE_UPSTREAM:\n> > -\t\tcase PCI_EXP_TYPE_DOWNSTREAM:\n> > -\t\t\tif (!(cap & PCI_EXP_DEVCAP2_ATOMIC_ROUTE))\n> > -\t\t\t\treturn -EINVAL;\n> > -\t\t\tbreak;\n> > -\n> > -\t\t/* Ensure root port supports all the sizes we care about */\n> > -\t\tcase PCI_EXP_TYPE_ROOT_PORT:\n> > -\t\t\tif ((cap & cap_mask) != cap_mask)\n> > -\t\t\t\treturn -EINVAL;\n> > -\t\t\tbreak;\n> > -\t\t}\n> > -\n> > -\t\t/* Ensure upstream ports don't block AtomicOps on egress */\n> > -\t\tif (pci_pcie_type(bridge) == PCI_EXP_TYPE_UPSTREAM) {\n> > +\t\t\t/* Upstream ports must not block AtomicOps on egress */\n> > \t\t\tpcie_capability_read_dword(bridge, PCI_EXP_DEVCTL2,\n> > \t\t\t\t\t\t &ctl2);\n> > \t\t\tif (ctl2 & PCI_EXP_DEVCTL2_ATOMIC_EGRESS_BLOCK)\n> > \t\t\t\treturn -EINVAL;\n> > +\t\t\tfallthrough;\n> > +\t\t/* All switch ports need to route AtomicOps */\n> > +\t\tcase PCI_EXP_TYPE_DOWNSTREAM:\n> > +\t\t\tif (!(cap & PCI_EXP_DEVCAP2_ATOMIC_ROUTE))\n> > +\t\t\t\treturn -EINVAL;\n> > +\t\t\tbreak;\n> > \t\t}\n> > -\n> > \t\tbus = bus->parent;\n> > \t}\n> > \n> > +\t/*\n> > +\t * Finally, last bridge must be root port and support requested sizes\n> > +\t * or firmware asserts support\n> > +\t */\n> > +\tif (!(pci_is_atomicops_capable_rp(bridge, cap, cap_mask) ||\n> > +\t pcibios_connects_to_atomicops_capable_rc(dev, cap_mask)))\n> > +\t\treturn -EINVAL;\n> \n> Sashiko says:\n> \n> Since the generic weak implementation of\n> pcibios_connects_to_atomicops_capable_rc() unconditionally returns\n> true, the logical OR expression pci_is_atomicops_capable_rp(...) ||\n> true will always evaluate to true. This makes the entire if\n> condition evaluate to false.\n> \n> Because of this, it appears -EINVAL is never returned here, and any\n> standard endpoint behind a Root Port will successfully be granted\n> AtomicOps even if the Root Port lacks the capability in its\n> PCI_EXP_DEVCAP2 register.\n\nI've made the generic implementation of\npcibios_connects_to_atomicops_capable_rc() default to return \"true\" to\npreserve the current code's handling of RCiEPs: Since they are not\nattached to a root port, their dev->bus->parent is NULL and the entire\nwhile-loop is bypassed - before this patch and after. (Sashiko was\npointing at that being regressed with v4.)\n\nThe whole point of pcibios_connects_to_atomicops_capable_rc() is to\nallow different architectures to implement a discriminator outside of\nPCIe's structure - potentially depending on CPU model or more.\n\nThe only point I wonder about: Should\npcibios_connects_to_atomicops_capable_rc() default to return \"false\"\nand deliberately change the behavior for today's RCiEP's (if there are\nany...)?\n\n> \n> > +\n> > \tpcie_capability_set_word(dev, PCI_EXP_DEVCTL2,\n> > \t\t\t\t PCI_EXP_DEVCTL2_ATOMIC_REQ);\n> > \treturn 0;\n> > diff --git a/include/linux/pci.h b/include/linux/pci.h\n> > index 1c270f1d512301de4d462fe7e5097c32af5c6f8d..ef90604c39859ea8e61e5392d0bdaa1b0e43874b 100644\n> > --- a/include/linux/pci.h\n> > +++ b/include/linux/pci.h\n> > @@ -692,6 +692,7 @@ void pci_set_host_bridge_release(struct pci_host_bridge *bridge,\n> > \t\t\t\t void *release_data);\n> > \n> > int pcibios_root_bridge_prepare(struct pci_host_bridge *bridge);\n> > +bool pcibios_connects_to_atomicops_capable_rc(struct pci_dev *dev, u32 cap_mask);\n> > \n> > #define PCI_REGION_FLAG_MASK\t0x0fU\t/* These bits of resource flags tell us the PCI region flags */\n> > \n> > \n> > -- \n> > 2.51.0\n> > \n\nThanks,\nGerd","headers":{"Return-Path":"\n ","X-Original-To":["incoming@patchwork.ozlabs.org","linux-pci@vger.kernel.org"],"Delivered-To":"patchwork-incoming@legolas.ozlabs.org","Authentication-Results":["legolas.ozlabs.org;\n\tdkim=pass (2048-bit key;\n unprotected) header.d=ibm.com header.i=@ibm.com header.a=rsa-sha256\n header.s=pp1 header.b=Ve8fYCXH;\n\tdkim-atps=neutral","legolas.ozlabs.org;\n spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org\n (client-ip=2600:3c0a:e001:db::12fc:5321; helo=sea.lore.kernel.org;\n envelope-from=linux-pci+bounces-51166-incoming=patchwork.ozlabs.org@vger.kernel.org;\n receiver=patchwork.ozlabs.org)","smtp.subspace.kernel.org;\n\tdkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com\n header.b=\"Ve8fYCXH\"","smtp.subspace.kernel.org;\n arc=none smtp.client-ip=148.163.158.5","smtp.subspace.kernel.org;\n dmarc=pass (p=none dis=none) header.from=linux.ibm.com","smtp.subspace.kernel.org;\n spf=pass smtp.mailfrom=linux.ibm.com"],"Received":["from sea.lore.kernel.org (sea.lore.kernel.org\n [IPv6:2600:3c0a:e001:db::12fc:5321])\n\t(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)\n\t key-exchange x25519 server-signature ECDSA (secp384r1) server-digest SHA384)\n\t(No client certificate requested)\n\tby legolas.ozlabs.org (Postfix) with ESMTPS id 4fhK283QXnz1y1G\n\tfor ; Thu, 26 Mar 2026 20:57:20 +1100 (AEDT)","from smtp.subspace.kernel.org (conduit.subspace.kernel.org\n [100.90.174.1])\n\tby sea.lore.kernel.org (Postfix) with ESMTP id 3FDFC3050217\n\tfor ; Thu, 26 Mar 2026 09:51:35 +0000 (UTC)","from localhost.localdomain (localhost.localdomain [127.0.0.1])\n\tby smtp.subspace.kernel.org (Postfix) with ESMTP id 9D2F93B4EB2;\n\tThu, 26 Mar 2026 09:51:33 +0000 (UTC)","from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com\n [148.163.158.5])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))\n\t(No client certificate requested)\n\tby smtp.subspace.kernel.org (Postfix) with ESMTPS id 919803B2FD5;\n\tThu, 26 Mar 2026 09:51:29 +0000 (UTC)","from pps.filterd (m0356516.ppops.net [127.0.0.1])\n\tby mx0a-001b2d01.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id\n 62Q7cxGX2322759;\n\tThu, 26 Mar 2026 09:51:25 GMT","from ppma22.wdc07v.mail.ibm.com\n (5c.69.3da9.ip4.static.sl-reverse.com [169.61.105.92])\n\tby mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4d1ktv3kuf-1\n\t(version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);\n\tThu, 26 Mar 2026 09:51:24 +0000 (GMT)","from pps.filterd (ppma22.wdc07v.mail.ibm.com [127.0.0.1])\n\tby ppma22.wdc07v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id\n 62Q5m0sN005972;\n\tThu, 26 Mar 2026 09:51:23 GMT","from smtprelay06.fra02v.mail.ibm.com ([9.218.2.230])\n\tby ppma22.wdc07v.mail.ibm.com (PPS) with ESMTPS id 4d261ytg5y-1\n\t(version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);\n\tThu, 26 Mar 2026 09:51:23 +0000","from smtpav05.fra02v.mail.ibm.com (smtpav05.fra02v.mail.ibm.com\n [10.20.54.104])\n\tby smtprelay06.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id\n 62Q9pJlu20185348\n\t(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK);\n\tThu, 26 Mar 2026 09:51:19 GMT","from smtpav05.fra02v.mail.ibm.com (unknown [127.0.0.1])\n\tby IMSVA (Postfix) with ESMTP id 9850E2004D;\n\tThu, 26 Mar 2026 09:51:19 +0000 (GMT)","from smtpav05.fra02v.mail.ibm.com (unknown [127.0.0.1])\n\tby IMSVA (Postfix) with ESMTP id 493D12004B;\n\tThu, 26 Mar 2026 09:51:19 +0000 (GMT)","from [9.52.210.163] (unknown [9.52.210.163])\n\tby smtpav05.fra02v.mail.ibm.com (Postfix) with ESMTP;\n\tThu, 26 Mar 2026 09:51:19 +0000 (GMT)"],"ARC-Seal":"i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116;\n\tt=1774518693; cv=none;\n b=LHIw++A3s5x78E/hM1fJavw8yxPz1hDEjlX2ADFWKkgynzIWSCMVbQM7dStqcJLU1brxz79Lp90f9xajfCKmh0m3hFurEEIph4jMbAAMGVgmXMBSP6MPUPkmWNM1DLb5tIZ0wKvyDt9+mzRDyIqCPbM1HWS0IZ6umrbKp4NAUxY=","ARC-Message-Signature":"i=1; a=rsa-sha256; d=subspace.kernel.org;\n\ts=arc-20240116; t=1774518693; c=relaxed/simple;\n\tbh=1F1e+PD6WQJUH7+46B7/HIOsr2UDXoO60eIIHWfuy9Q=;\n\th=Message-ID:Subject:From:To:Cc:Date:In-Reply-To:References:\n\t Content-Type:MIME-Version;\n b=krwZZv/Fl0ypadXYvGMp9GmEUArYqTr9x0MytQbS5jqLjQtyX6Xpyz2dSq4dzHkVRx++96NEdxQR4E8ldHIRhcrb1YuznCGFrQtrcVUzk8VdrDJuRTqEwx4fzipnRDsFeAwwtz/EefJ8bWlWJggi8ZNNWKTM95uhzw7cGNepRoc=","ARC-Authentication-Results":"i=1; smtp.subspace.kernel.org;\n dmarc=pass (p=none dis=none) header.from=linux.ibm.com;\n spf=pass smtp.mailfrom=linux.ibm.com;\n dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com\n header.b=Ve8fYCXH; arc=none smtp.client-ip=148.163.158.5","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc\n\t:content-transfer-encoding:content-type:date:from:in-reply-to\n\t:message-id:mime-version:references:subject:to; s=pp1; bh=1kgxzl\n\tLd5zBWO+ZzZO1oRCKJxbowZoD8EBRF0PyTh0Q=; b=Ve8fYCXHgY0pU4QFhgtTEO\n\tkvTn0vaOcu1Spqi3bMtkmL/2foTGoRdpxKhIyQt/5dt/CFx/MHirAD9ncdk6X+rt\n\tGAm6/jqCWe8qqjHVwgsAw2fFmENVllJjdMtAFtVdVgLXxQS5uV/2hzY9gg69HuLg\n\tcpdOuWvnfrFw5+X7rHggb6yJLI9/uVqNvZBnhF6LxGCDGYt4WAFUot/Lflr3w8Gb\n\t5bf9fQ+GgsLgSwS9HitCb4tlhkf+E3fpLOIpcFwxhBjlsnRW30ATFCNFH/MeoLo6\n\tk16HrVLa9gURaKkF0uMIQ/2tX7V22hnq85jusXLFvPkG0dFt1MI1XRXXyh9na8nw\n\t==","Message-ID":"<4183b471777efa949ce6f7b860c81282e91666ef.camel@linux.ibm.com>","Subject":"Re: [PATCH v6 1/2] PCI: AtomicOps: Do not enable without support in\n root complex","From":"Gerd Bayer ","To":"Bjorn Helgaas ","Cc":"Bjorn Helgaas , Jay Cornwall\n ,\n Felix Kuehling ,\n Christian\n Borntraeger ,\n Niklas Schnelle\t\n ,\n Gerald Schaefer ,\n Heiko Carstens , Vasily Gorbik ,\n Alexander Gordeev ,\n Sven Schnelle\n ,\n Leon Romanovsky ,\n Alexander Schmidt\n , linux-s390@vger.kernel.org,\n linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org,\n netdev@vger.kernel.org, linux-rdma@vger.kernel.org,\n stable@vger.kernel.org, Gerd Bayer ","Date":"Thu, 26 Mar 2026 10:51:19 +0100","In-Reply-To":"<20260325200835.GA1290451@bhelgaas>","References":"<20260325200835.GA1290451@bhelgaas>","Content-Type":"text/plain; charset=\"UTF-8\"","Content-Transfer-Encoding":"quoted-printable","User-Agent":"Evolution 3.58.3 (3.58.3-1.fc43) ","Precedence":"bulk","X-Mailing-List":"linux-pci@vger.kernel.org","List-Id":"","List-Subscribe":"","List-Unsubscribe":"","MIME-Version":"1.0","X-TM-AS-GCONF":"00","X-Proofpoint-Reinject":"loops=2 maxloops=12","X-Proofpoint-ORIG-GUID":"OSqgECE0RlHxvqNr9gv6e4eF2D_OyIL8","X-Authority-Analysis":"v=2.4 cv=aMr9aL9m c=1 sm=1 tr=0 ts=69c5019c cx=c_pps\n a=5BHTudwdYE3Te8bg5FgnPg==:117 a=5BHTudwdYE3Te8bg5FgnPg==:17\n a=IkcTkHD0fZMA:10 a=Yq5XynenixoA:10 a=VkNPw1HP01LnGYTKEx00:22\n a=RnoormkPH1_aCDwRdu11:22 a=Y2IxJ9c9Rs8Kov3niI8_:22 a=VnNF1IyMAAAA:8\n a=VwQbUJbxAAAA:8 a=x6K-tPQhctwWebVqKp8A:9 a=QEXdDO2ut3YA:10","X-Proofpoint-Spam-Details-Enc":"AW1haW4tMjYwMzI2MDA2OSBTYWx0ZWRfXwU8JLL36ZvE9\n u3BIiHMeyg4IJAMCMHH9FdAbLenEj91xljHWC3fmBBtHg72UmTKkmQSleVmY2loLQOSyKlnoixz\n 4aVLM7QBLlp9vNplTa0xS9aywiIju+ZzHk0gYcvdvjzFD7mIn8Jaz6V6TtVWvSdoMTK0qRi0dOA\n aPpy+OZNO+g10mYiiPeL7KZstIjaIyiIq0A8zqcxAcwity2UiXqFZsNZQTlXRhmj4Q0TYKX6moh\n +AWwWCZpPzLe9i7KhupA119dJL0/qF4rNzFytiOjhHnctimgSe++RETHKGRaJH0HmUs6MPJyeNX\n 5UEkTF7J2ZKdYhsslEHj+GqtmJCJ3HcsCd7wla6/x+NavBW3KV2XKn1eePEid4gCOJt/LZNDLQ0\n hYVTKuSlPpH/lsas0zuEaWIYti9RvldqMzNk4TbbHN5Z2LGhtpKfsC7ZU6JwBMd3j+enNFcR5wM\n 4qrwBLn4WZmNH1ME0Zg==","X-Proofpoint-GUID":"1hiv0VvXh2pfW19u80XK0ds2_UlMWuiJ","X-Proofpoint-Virus-Version":"vendor=baseguard\n engine=ICAP:2.0.293,Aquarius:18.0.1143,Hydra:6.1.51,FMLib:17.12.100.49\n definitions=2026-03-26_02,2026-03-24_01,2025-10-01_01","X-Proofpoint-Spam-Details":"rule=outbound_notspam policy=outbound score=0\n spamscore=0 lowpriorityscore=0 adultscore=0 impostorscore=0 malwarescore=0\n suspectscore=0 phishscore=0 priorityscore=1501 bulkscore=0 clxscore=1015\n classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0\n reason=mlx scancount=1 engine=8.22.0-2603050001 definitions=main-2603260069"}},{"id":3669839,"web_url":"http://patchwork.ozlabs.org/comment/3669839/","msgid":"<20260326164002.GA1325368@bhelgaas>","list_archive_url":null,"date":"2026-03-26T16:40:02","subject":"Re: [PATCH v6 1/2] PCI: AtomicOps: Do not enable without support in\n root complex","submitter":{"id":67298,"url":"http://patchwork.ozlabs.org/api/people/67298/","name":"Bjorn Helgaas","email":"helgaas@kernel.org"},"content":"On Thu, Mar 26, 2026 at 10:51:19AM +0100, Gerd Bayer wrote:\n> On Wed, 2026-03-25 at 15:08 -0500, Bjorn Helgaas wrote:\n> > On Wed, Mar 25, 2026 at 04:16:17PM +0100, Gerd Bayer wrote:\n> > > When inspecting the config space of a Connect-X physical function in an\n> > > s390 system after it was initialized by the mlx5_core device driver, we\n> > > found the function to be enabled to request AtomicOps despite the\n> > > system's root-complex lacking support for completing them:\n> > > \n> > > 1ed0:00:00.1 Ethernet controller: Mellanox Technologies MT2894 Family [ConnectX-6 Lx]\n> > > \tSubsystem: Mellanox Technologies Device 0002\n> > > [...]\n> > > \tDevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-\n> > > \t\t AtomicOpsCtl: ReqEn+\n> > > \t\t IDOReq- IDOCompl- LTR- EmergencyPowerReductionReq-\n> > > \t\t 10BitTagReq- OBFF Disabled, EETLPPrefixBlk-\n> > > \n> > > Turns out the device driver calls pci_enable_atomic_ops_to_root() which\n> > > defaulted to enable AtomicOps requests even if it had no information\n> > > about the root-port that the PCIe device is attached to. Similarly,\n> > > AtomicOps requests are enabled for root complex integrated endpoints\n> > > (RCiEPs) unconditionally.\n> > > \n> > > Change the logic of pci_enable_atomic_ops_to_root() to fully traverse the\n> > > PCIe tree upwards, check that the bridge devices support delivering\n> > > AtomicOps transactions, and finally check that there is a root port at\n> > > the end that does support completing AtomicOps - or that the support for\n> > > completing AtomicOps at the root complex is announced through some other\n> > > arch specific way.\n> > > \n> > > Introduce a new pcibios_connects_to_atomicops_capable_rc() function to\n> > > implement the check - and default to always \"true\". This leaves the\n> > > semantics for today's RCiEPs intact. Pass in the device in question and\n> > > the requested capabilities for future expansions.\n> > > For s390, override pcibios_connects_to_atomicops_capable_rc() to\n> > > always return \"false\".\n> > > \n> > > Do not change the enablement of AtomicOps requests if there is no\n> > > positive confirmation that the root complex can complete PCIe AtomicOps.\n> > > \n> > > Reported-by: Alexander Schmidt \n> > > Cc: stable@vger.kernel.org\n> > > Fixes: 430a23689dea (\"PCI: Add pci_enable_atomic_ops_to_root()\")\n> > > Signed-off-by: Gerd Bayer \n> > > ---\n> > > arch/s390/pci/pci.c | 5 +++++\n> > > drivers/pci/pci.c | 48 +++++++++++++++++++++++++++++++-----------------\n> > > include/linux/pci.h | 1 +\n> > > 3 files changed, 37 insertions(+), 17 deletions(-)\n> > > \n> > > diff --git a/arch/s390/pci/pci.c b/arch/s390/pci/pci.c\n> > > index 2a430722cbe415dd56c92fed2e513e524f46481a..a0bef77082a153a258fbe4abb1070b22e020888e 100644\n> > > --- a/arch/s390/pci/pci.c\n> > > +++ b/arch/s390/pci/pci.c\n> > > @@ -265,6 +265,11 @@ static int zpci_cfg_store(struct zpci_dev *zdev, int offset, u32 val, u8 len)\n> > > \treturn rc;\n> > > }\n> > > \n> > > +bool pcibios_connects_to_atomicops_capable_rc(struct pci_dev *dev, u32 cap_mask)\n> > > +{\n> > > +\treturn false;\n> > > +}\n> > > +\n> > > resource_size_t pcibios_align_resource(void *data, const struct resource *res,\n> > > \t\t\t\t resource_size_t size,\n> > > \t\t\t\t resource_size_t align)\n> > > diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c\n> > > index 8479c2e1f74f1044416281aba11bf071ea89488a..006aa589926cb290de43f152100ddaf9961407d1 100644\n> > > --- a/drivers/pci/pci.c\n> > > +++ b/drivers/pci/pci.c\n> > > @@ -3660,6 +3660,19 @@ void pci_acs_init(struct pci_dev *dev)\n> > > \tpci_disable_broken_acs_cap(dev);\n> > > }\n> > > \n> > > +static bool pci_is_atomicops_capable_rp(struct pci_dev *dev, u32 cap, u32 cap_mask)\n> > > +{\n> > > +\tif (!dev || !(pci_pcie_type(dev) == PCI_EXP_TYPE_ROOT_PORT))\n> > > +\t\treturn false;\n> > > +\n> > > +\treturn (cap & cap_mask) == cap_mask;\n> > > +}\n> > > +\n> > > +bool __weak pcibios_connects_to_atomicops_capable_rc(struct pci_dev *dev, u32 cap_mask)\n> > > +{\n> > > +\treturn true;\n> > > +}\n> > > +\n> > > /**\n> > > * pci_enable_atomic_ops_to_root - enable AtomicOp requests to root port\n> > > * @dev: the PCI device\n> > > @@ -3676,8 +3689,9 @@ void pci_acs_init(struct pci_dev *dev)\n> > > int pci_enable_atomic_ops_to_root(struct pci_dev *dev, u32 cap_mask)\n> > > {\n> > > \tstruct pci_bus *bus = dev->bus;\n> > > -\tstruct pci_dev *bridge;\n> > > -\tu32 cap, ctl2;\n> > > +\tstruct pci_dev *bridge = NULL;\n> > > +\tu32 cap = 0;\n> > > +\tu32 ctl2;\n> > > \n> > > \t/*\n> > > \t * Per PCIe r5.0, sec 9.3.5.10, the AtomicOp Requester Enable bit\n> > > @@ -3714,29 +3728,29 @@ int pci_enable_atomic_ops_to_root(struct pci_dev *dev, u32 cap_mask)\n> > > \t\tswitch (pci_pcie_type(bridge)) {\n> > > \t\t/* Ensure switch ports support AtomicOp routing */\n> > > \t\tcase PCI_EXP_TYPE_UPSTREAM:\n> > > -\t\tcase PCI_EXP_TYPE_DOWNSTREAM:\n> > > -\t\t\tif (!(cap & PCI_EXP_DEVCAP2_ATOMIC_ROUTE))\n> > > -\t\t\t\treturn -EINVAL;\n> > > -\t\t\tbreak;\n> > > -\n> > > -\t\t/* Ensure root port supports all the sizes we care about */\n> > > -\t\tcase PCI_EXP_TYPE_ROOT_PORT:\n> > > -\t\t\tif ((cap & cap_mask) != cap_mask)\n> > > -\t\t\t\treturn -EINVAL;\n> > > -\t\t\tbreak;\n> > > -\t\t}\n> > > -\n> > > -\t\t/* Ensure upstream ports don't block AtomicOps on egress */\n> > > -\t\tif (pci_pcie_type(bridge) == PCI_EXP_TYPE_UPSTREAM) {\n> > > +\t\t\t/* Upstream ports must not block AtomicOps on egress */\n> > > \t\t\tpcie_capability_read_dword(bridge, PCI_EXP_DEVCTL2,\n> > > \t\t\t\t\t\t &ctl2);\n> > > \t\t\tif (ctl2 & PCI_EXP_DEVCTL2_ATOMIC_EGRESS_BLOCK)\n> > > \t\t\t\treturn -EINVAL;\n> > > +\t\t\tfallthrough;\n> > > +\t\t/* All switch ports need to route AtomicOps */\n> > > +\t\tcase PCI_EXP_TYPE_DOWNSTREAM:\n> > > +\t\t\tif (!(cap & PCI_EXP_DEVCAP2_ATOMIC_ROUTE))\n> > > +\t\t\t\treturn -EINVAL;\n> > > +\t\t\tbreak;\n> > > \t\t}\n> > > -\n> > > \t\tbus = bus->parent;\n> > > \t}\n> > > \n> > > +\t/*\n> > > +\t * Finally, last bridge must be root port and support requested sizes\n> > > +\t * or firmware asserts support\n> > > +\t */\n> > > +\tif (!(pci_is_atomicops_capable_rp(bridge, cap, cap_mask) ||\n> > > +\t pcibios_connects_to_atomicops_capable_rc(dev, cap_mask)))\n> > > +\t\treturn -EINVAL;\n> > \n> > Sashiko says:\n> > \n> > Since the generic weak implementation of\n> > pcibios_connects_to_atomicops_capable_rc() unconditionally returns\n> > true, the logical OR expression pci_is_atomicops_capable_rp(...) ||\n> > true will always evaluate to true. This makes the entire if\n> > condition evaluate to false.\n> > \n> > Because of this, it appears -EINVAL is never returned here, and any\n> > standard endpoint behind a Root Port will successfully be granted\n> > AtomicOps even if the Root Port lacks the capability in its\n> > PCI_EXP_DEVCAP2 register.\n> \n> I've made the generic implementation of\n> pcibios_connects_to_atomicops_capable_rc() default to return \"true\" to\n> preserve the current code's handling of RCiEPs: Since they are not\n> attached to a root port, their dev->bus->parent is NULL and the entire\n> while-loop is bypassed - before this patch and after. (Sashiko was\n> pointing at that being regressed with v4.)\n\nThe v4 patch definitely changed the behavior for RCiEPs: the current\nv7.0-rc1 code always enables AtomicOps for RCiEPs, and the v4 patch\nnever enables AtomicOps for RCiEPs. But I'm not sure this is a\nregression. It definitely *could* break an RCiEP, but AFAIK we have\nno information about whether the RC supports AtomicOps, so enabling\nthem and telling the driver that AtomicOps work might be a lie.\n\nIIUC, the motivation for this series was to avoid enabling AtomicOps\non s390 where there is no visible Root Port, and you have platform\nknowledg that whatever is upstream from the endpoint in fact does not\nsupport them.\n\nI think we should avoid enabling AtomicOps unless we know for certain\nthat the completer (Root Port or RC) supports them. To me that sounds\nlike:\n\n 1) Never enable AtomicOps for RCiEPs.\n\n 2) Only enable AtomicOps for endpoints below a Root Port that\n supports AtomicOps.\n\nThis could be two separate patches, where the second would fix the\ns390 issue reported by Alexander.\n\nIf we come across RCiEPs that need AtomicOps and we somehow know that\nthe RC supports them, we can add a quirk or something to take\nadvantage of it.\n\nWe are still hand-waving about peer-to-peer transactions; we don't\neven try to account for that because we don't know what peer might be\nthe completer.\n\n> The whole point of pcibios_connects_to_atomicops_capable_rc() is to\n> allow different architectures to implement a discriminator outside of\n> PCIe's structure - potentially depending on CPU model or more.\n> \n> The only point I wonder about: Should\n> pcibios_connects_to_atomicops_capable_rc() default to return \"false\"\n> and deliberately change the behavior for today's RCiEP's (if there are\n> any...)?\n> \n> > \n> > > +\n> > > \tpcie_capability_set_word(dev, PCI_EXP_DEVCTL2,\n> > > \t\t\t\t PCI_EXP_DEVCTL2_ATOMIC_REQ);\n> > > \treturn 0;\n> > > diff --git a/include/linux/pci.h b/include/linux/pci.h\n> > > index 1c270f1d512301de4d462fe7e5097c32af5c6f8d..ef90604c39859ea8e61e5392d0bdaa1b0e43874b 100644\n> > > --- a/include/linux/pci.h\n> > > +++ b/include/linux/pci.h\n> > > @@ -692,6 +692,7 @@ void pci_set_host_bridge_release(struct pci_host_bridge *bridge,\n> > > \t\t\t\t void *release_data);\n> > > \n> > > int pcibios_root_bridge_prepare(struct pci_host_bridge *bridge);\n> > > +bool pcibios_connects_to_atomicops_capable_rc(struct pci_dev *dev, u32 cap_mask);\n> > > \n> > > #define PCI_REGION_FLAG_MASK\t0x0fU\t/* These bits of resource flags tell us the PCI region flags */\n> > > \n> > > \n> > > -- \n> > > 2.51.0\n> > > \n> \n> Thanks,\n> Gerd","headers":{"Return-Path":"\n ","X-Original-To":["incoming@patchwork.ozlabs.org","linux-pci@vger.kernel.org"],"Delivered-To":"patchwork-incoming@legolas.ozlabs.org","Authentication-Results":["legolas.ozlabs.org;\n\tdkim=pass (2048-bit key;\n unprotected) header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256\n header.s=k20201202 header.b=pisLqagj;\n\tdkim-atps=neutral","legolas.ozlabs.org;\n spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org\n (client-ip=172.232.135.74; helo=sto.lore.kernel.org;\n envelope-from=linux-pci+bounces-51205-incoming=patchwork.ozlabs.org@vger.kernel.org;\n receiver=patchwork.ozlabs.org)","smtp.subspace.kernel.org;\n\tdkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org\n header.b=\"pisLqagj\"","smtp.subspace.kernel.org;\n arc=none smtp.client-ip=10.30.226.201"],"Received":["from sto.lore.kernel.org (sto.lore.kernel.org [172.232.135.74])\n\t(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)\n\t key-exchange x25519 server-signature ECDSA (secp384r1) server-digest SHA384)\n\t(No client certificate requested)\n\tby legolas.ozlabs.org (Postfix) with ESMTPS id 4fhV9C1cG9z1yGD\n\tfor ; Fri, 27 Mar 2026 03:49:03 +1100 (AEDT)","from smtp.subspace.kernel.org (conduit.subspace.kernel.org\n [100.90.174.1])\n\tby sto.lore.kernel.org (Postfix) with ESMTP id A5768308ED46\n\tfor ; Thu, 26 Mar 2026 16:40:39 +0000 (UTC)","from localhost.localdomain (localhost.localdomain [127.0.0.1])\n\tby smtp.subspace.kernel.org (Postfix) with ESMTP id 627A240F8F6;\n\tThu, 26 Mar 2026 16:40:04 +0000 (UTC)","from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org\n [10.30.226.201])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))\n\t(No client certificate requested)\n\tby smtp.subspace.kernel.org (Postfix) with ESMTPS id 3A91F4035CD;\n\tThu, 26 Mar 2026 16:40:03 +0000 (UTC)","by smtp.kernel.org (Postfix) with ESMTPSA id 9BD1BC116C6;\n\tThu, 26 Mar 2026 16:40:03 +0000 (UTC)"],"ARC-Seal":"i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116;\n\tt=1774543204; cv=none;\n b=mJPOPdO8twPDzSqtwwy74+gCJnqE1Up6ToPA6vLThG99qYeQRjctwX+She/+zcil7neKm6ayEBqniWqNH9YhV1sY9TLJ7gyvcwJTemBJULYuAgFRitudfLW7e5j2cdcfvFHqElQPbB2Sd0KD4cZyRHbmjHbRjVRUB36FW1zV1hU=","ARC-Message-Signature":"i=1; a=rsa-sha256; d=subspace.kernel.org;\n\ts=arc-20240116; t=1774543204; c=relaxed/simple;\n\tbh=v9OxL3z6qKsfDWe7vgJygT5T/5mg3FeLB6gOdPjSyDk=;\n\th=Date:From:To:Cc:Subject:Message-ID:MIME-Version:Content-Type:\n\t Content-Disposition:In-Reply-To;\n b=AzwpW9huwKCjMnbMx3SFlL/M+7ZOzAVIYiZIp77Ap2h3AnY5WOw8Nd21+fB8oNIbiYvZDI3fyUfmtALHBhNFXAhdw/keQyHSTjceuEgewNng0cLINE/XjNKoZGEWYLal2r9dt95wh02nEQdVXfMFx5QpDoASVkvyDcz33meQqXs=","ARC-Authentication-Results":"i=1; smtp.subspace.kernel.org;\n dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org\n header.b=pisLqagj; arc=none smtp.client-ip=10.30.226.201","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;\n\ts=k20201202; t=1774543203;\n\tbh=v9OxL3z6qKsfDWe7vgJygT5T/5mg3FeLB6gOdPjSyDk=;\n\th=Date:From:To:Cc:Subject:In-Reply-To:From;\n\tb=pisLqagj3vB7EHVWtrYeETBb1Gl8iQLkOtglrdUeLgotRTEpoTq6BAXEW845d4eTN\n\t DbK7B6mnUv8MXwuZy+l8Ewm1Bp4/wszn0Itp3iNaoURdrX4Vzb+EMTyANgbZJpqO3+\n\t Y641c7/ZpyRQV9qxCKNZmtH/wN1HWyA7PAFtaH/hUtCqlIJKajhEIgUGJlQoOvZKJ3\n\t rP/VpkLPjhOnP6NMR2/N4iMdpN4DFyZxnqRQnK/c8/XChTyXH8pv6P0BGsONKLOFSU\n\t QhDYiEU+YTg6LoechbtMk8kIA8WLWnwdnAeRqyKMIvrDYLDp35X3k32Bmf3LoSi0Bf\n\t EE4YkDDj60FvA==","Date":"Thu, 26 Mar 2026 11:40:02 -0500","From":"Bjorn Helgaas ","To":"Gerd Bayer ","Cc":"Bjorn Helgaas ,\n\tJay Cornwall ,\n\tFelix Kuehling ,\n\tChristian Borntraeger ,\n\tNiklas Schnelle ,\n\tGerald Schaefer ,\n\tHeiko Carstens ,\n\tVasily Gorbik ,\n\tAlexander Gordeev ,\n\tSven Schnelle ,\n\tLeon Romanovsky ,\n\tAlexander Schmidt , linux-s390@vger.kernel.org,\n\tlinux-pci@vger.kernel.org, linux-kernel@vger.kernel.org,\n\tnetdev@vger.kernel.org, linux-rdma@vger.kernel.org,\n\tstable@vger.kernel.org, Gerd Bayer ","Subject":"Re: [PATCH v6 1/2] PCI: AtomicOps: Do not enable without support in\n root complex","Message-ID":"<20260326164002.GA1325368@bhelgaas>","Precedence":"bulk","X-Mailing-List":"linux-pci@vger.kernel.org","List-Id":"","List-Subscribe":"","List-Unsubscribe":"","MIME-Version":"1.0","Content-Type":"text/plain; charset=us-ascii","Content-Disposition":"inline","In-Reply-To":"<4183b471777efa949ce6f7b860c81282e91666ef.camel@linux.ibm.com>"}},{"id":3670965,"web_url":"http://patchwork.ozlabs.org/comment/3670965/","msgid":"","list_archive_url":null,"date":"2026-03-30T08:59:28","subject":"Re: [PATCH v6 1/2] PCI: AtomicOps: Do not enable without support in\n root complex","submitter":{"id":85804,"url":"http://patchwork.ozlabs.org/api/people/85804/","name":"Gerd Bayer","email":"gbayer@linux.ibm.com"},"content":"On Thu, 2026-03-26 at 11:40 -0500, Bjorn Helgaas wrote:\n> On Thu, Mar 26, 2026 at 10:51:19AM +0100, Gerd Bayer wrote:\n> > On Wed, 2026-03-25 at 15:08 -0500, Bjorn Helgaas wrote:\n> > > On Wed, Mar 25, 2026 at 04:16:17PM +0100, Gerd Bayer wrote:\n> > > > When inspecting the config space of a Connect-X physical function in an\n> > > > s390 system after it was initialized by the mlx5_core device driver, we\n> > > > found the function to be enabled to request AtomicOps despite the\n> > > > system's root-complex lacking support for completing them:\n> > > > \n> > > > 1ed0:00:00.1 Ethernet controller: Mellanox Technologies MT2894 Family [ConnectX-6 Lx]\n> > > > \tSubsystem: Mellanox Technologies Device 0002\n> > > > [...]\n> > > > \tDevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-\n> > > > \t\t AtomicOpsCtl: ReqEn+\n> > > > \t\t IDOReq- IDOCompl- LTR- EmergencyPowerReductionReq-\n> > > > \t\t 10BitTagReq- OBFF Disabled, EETLPPrefixBlk-\n> > > > \n> > > > Turns out the device driver calls pci_enable_atomic_ops_to_root() which\n> > > > defaulted to enable AtomicOps requests even if it had no information\n> > > > about the root-port that the PCIe device is attached to. Similarly,\n> > > > AtomicOps requests are enabled for root complex integrated endpoints\n> > > > (RCiEPs) unconditionally.\n> > > > \n> > > > Change the logic of pci_enable_atomic_ops_to_root() to fully traverse the\n> > > > PCIe tree upwards, check that the bridge devices support delivering\n> > > > AtomicOps transactions, and finally check that there is a root port at\n> > > > the end that does support completing AtomicOps - or that the support for\n> > > > completing AtomicOps at the root complex is announced through some other\n> > > > arch specific way.\n> > > > \n> > > > Introduce a new pcibios_connects_to_atomicops_capable_rc() function to\n> > > > implement the check - and default to always \"true\". This leaves the\n> > > > semantics for today's RCiEPs intact. Pass in the device in question and\n> > > > the requested capabilities for future expansions.\n> > > > For s390, override pcibios_connects_to_atomicops_capable_rc() to\n> > > > always return \"false\".\n> > > > \n> > > > Do not change the enablement of AtomicOps requests if there is no\n> > > > positive confirmation that the root complex can complete PCIe AtomicOps.\n> > > > \n> > > > Reported-by: Alexander Schmidt \n> > > > Cc: stable@vger.kernel.org\n> > > > Fixes: 430a23689dea (\"PCI: Add pci_enable_atomic_ops_to_root()\")\n> > > > Signed-off-by: Gerd Bayer \n> > > > ---\n> > > > arch/s390/pci/pci.c | 5 +++++\n> > > > drivers/pci/pci.c | 48 +++++++++++++++++++++++++++++++-----------------\n> > > > include/linux/pci.h | 1 +\n> > > > 3 files changed, 37 insertions(+), 17 deletions(-)\n> > > > \n> > > > diff --git a/arch/s390/pci/pci.c b/arch/s390/pci/pci.c\n> > > > index 2a430722cbe415dd56c92fed2e513e524f46481a..a0bef77082a153a258fbe4abb1070b22e020888e 100644\n> > > > --- a/arch/s390/pci/pci.c\n> > > > +++ b/arch/s390/pci/pci.c\n> > > > @@ -265,6 +265,11 @@ static int zpci_cfg_store(struct zpci_dev *zdev, int offset, u32 val, u8 len)\n> > > > \treturn rc;\n> > > > }\n> > > > \n> > > > +bool pcibios_connects_to_atomicops_capable_rc(struct pci_dev *dev, u32 cap_mask)\n> > > > +{\n> > > > +\treturn false;\n> > > > +}\n> > > > +\n> > > > resource_size_t pcibios_align_resource(void *data, const struct resource *res,\n> > > > \t\t\t\t resource_size_t size,\n> > > > \t\t\t\t resource_size_t align)\n> > > > diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c\n> > > > index 8479c2e1f74f1044416281aba11bf071ea89488a..006aa589926cb290de43f152100ddaf9961407d1 100644\n> > > > --- a/drivers/pci/pci.c\n> > > > +++ b/drivers/pci/pci.c\n> > > > @@ -3660,6 +3660,19 @@ void pci_acs_init(struct pci_dev *dev)\n> > > > \tpci_disable_broken_acs_cap(dev);\n> > > > }\n> > > > \n> > > > +static bool pci_is_atomicops_capable_rp(struct pci_dev *dev, u32 cap, u32 cap_mask)\n> > > > +{\n> > > > +\tif (!dev || !(pci_pcie_type(dev) == PCI_EXP_TYPE_ROOT_PORT))\n> > > > +\t\treturn false;\n> > > > +\n> > > > +\treturn (cap & cap_mask) == cap_mask;\n> > > > +}\n> > > > +\n> > > > +bool __weak pcibios_connects_to_atomicops_capable_rc(struct pci_dev *dev, u32 cap_mask)\n> > > > +{\n> > > > +\treturn true;\n> > > > +}\n> > > > +\n> > > > /**\n> > > > * pci_enable_atomic_ops_to_root - enable AtomicOp requests to root port\n> > > > * @dev: the PCI device\n> > > > @@ -3676,8 +3689,9 @@ void pci_acs_init(struct pci_dev *dev)\n> > > > int pci_enable_atomic_ops_to_root(struct pci_dev *dev, u32 cap_mask)\n> > > > {\n> > > > \tstruct pci_bus *bus = dev->bus;\n> > > > -\tstruct pci_dev *bridge;\n> > > > -\tu32 cap, ctl2;\n> > > > +\tstruct pci_dev *bridge = NULL;\n> > > > +\tu32 cap = 0;\n> > > > +\tu32 ctl2;\n> > > > \n> > > > \t/*\n> > > > \t * Per PCIe r5.0, sec 9.3.5.10, the AtomicOp Requester Enable bit\n> > > > @@ -3714,29 +3728,29 @@ int pci_enable_atomic_ops_to_root(struct pci_dev *dev, u32 cap_mask)\n> > > > \t\tswitch (pci_pcie_type(bridge)) {\n> > > > \t\t/* Ensure switch ports support AtomicOp routing */\n> > > > \t\tcase PCI_EXP_TYPE_UPSTREAM:\n> > > > -\t\tcase PCI_EXP_TYPE_DOWNSTREAM:\n> > > > -\t\t\tif (!(cap & PCI_EXP_DEVCAP2_ATOMIC_ROUTE))\n> > > > -\t\t\t\treturn -EINVAL;\n> > > > -\t\t\tbreak;\n> > > > -\n> > > > -\t\t/* Ensure root port supports all the sizes we care about */\n> > > > -\t\tcase PCI_EXP_TYPE_ROOT_PORT:\n> > > > -\t\t\tif ((cap & cap_mask) != cap_mask)\n> > > > -\t\t\t\treturn -EINVAL;\n> > > > -\t\t\tbreak;\n> > > > -\t\t}\n> > > > -\n> > > > -\t\t/* Ensure upstream ports don't block AtomicOps on egress */\n> > > > -\t\tif (pci_pcie_type(bridge) == PCI_EXP_TYPE_UPSTREAM) {\n> > > > +\t\t\t/* Upstream ports must not block AtomicOps on egress */\n> > > > \t\t\tpcie_capability_read_dword(bridge, PCI_EXP_DEVCTL2,\n> > > > \t\t\t\t\t\t &ctl2);\n> > > > \t\t\tif (ctl2 & PCI_EXP_DEVCTL2_ATOMIC_EGRESS_BLOCK)\n> > > > \t\t\t\treturn -EINVAL;\n> > > > +\t\t\tfallthrough;\n> > > > +\t\t/* All switch ports need to route AtomicOps */\n> > > > +\t\tcase PCI_EXP_TYPE_DOWNSTREAM:\n> > > > +\t\t\tif (!(cap & PCI_EXP_DEVCAP2_ATOMIC_ROUTE))\n> > > > +\t\t\t\treturn -EINVAL;\n> > > > +\t\t\tbreak;\n> > > > \t\t}\n> > > > -\n> > > > \t\tbus = bus->parent;\n> > > > \t}\n> > > > \n> > > > +\t/*\n> > > > +\t * Finally, last bridge must be root port and support requested sizes\n> > > > +\t * or firmware asserts support\n> > > > +\t */\n> > > > +\tif (!(pci_is_atomicops_capable_rp(bridge, cap, cap_mask) ||\n> > > > +\t pcibios_connects_to_atomicops_capable_rc(dev, cap_mask)))\n> > > > +\t\treturn -EINVAL;\n> > > \n> > > Sashiko says:\n> > > \n> > > Since the generic weak implementation of\n> > > pcibios_connects_to_atomicops_capable_rc() unconditionally returns\n> > > true, the logical OR expression pci_is_atomicops_capable_rp(...) ||\n> > > true will always evaluate to true. This makes the entire if\n> > > condition evaluate to false.\n> > > \n> > > Because of this, it appears -EINVAL is never returned here, and any\n> > > standard endpoint behind a Root Port will successfully be granted\n> > > AtomicOps even if the Root Port lacks the capability in its\n> > > PCI_EXP_DEVCAP2 register.\n> > \n> > I've made the generic implementation of\n> > pcibios_connects_to_atomicops_capable_rc() default to return \"true\" to\n> > preserve the current code's handling of RCiEPs: Since they are not\n> > attached to a root port, their dev->bus->parent is NULL and the entire\n> > while-loop is bypassed - before this patch and after. (Sashiko was\n> > pointing at that being regressed with v4.)\n> \n> The v4 patch definitely changed the behavior for RCiEPs: the current\n> v7.0-rc1 code always enables AtomicOps for RCiEPs, and the v4 patch\n> never enables AtomicOps for RCiEPs. But I'm not sure this is a\n> regression. It definitely *could* break an RCiEP, but AFAIK we have\n> no information about whether the RC supports AtomicOps, so enabling\n> them and telling the driver that AtomicOps work might be a lie.\n> \n> IIUC, the motivation for this series was to avoid enabling AtomicOps\n> on s390 where there is no visible Root Port, and you have platform\n> knowledg that whatever is upstream from the endpoint in fact does not\n> support them.\n> \n> I think we should avoid enabling AtomicOps unless we know for certain\n> that the completer (Root Port or RC) supports them. To me that sounds\n> like:\n> \n> 1) Never enable AtomicOps for RCiEPs.\n> \n> 2) Only enable AtomicOps for endpoints below a Root Port that\n> supports AtomicOps.\n> \n> This could be two separate patches, where the second would fix the\n> s390 issue reported by Alexander.\n\nThat sounds like a plan. Working on a v7 - keeping the reference\nupdates as a third patch of the series.\n\n> If we come across RCiEPs that need AtomicOps and we somehow know that\n> the RC supports them, we can add a quirk or something to take\n> advantage of it.\n\nLeave that as an execise for later.\n\n> We are still hand-waving about peer-to-peer transactions; we don't\n> even try to account for that because we don't know what peer might be\n> the completer.\n\nAnyway, I'm wondering what the adoption of AtomicOps looks like. Not\nmuch turned up in my searches.\n\n> > The whole point of pcibios_connects_to_atomicops_capable_rc() is to\n> > allow different architectures to implement a discriminator outside of\n> > PCIe's structure - potentially depending on CPU model or more.\n> > \n> > The only point I wonder about: Should\n> > pcibios_connects_to_atomicops_capable_rc() default to return \"false\"\n> > and deliberately change the behavior for today's RCiEP's (if there are\n> > any...)?\n> > \n> > > \n> > > > +\n> > > > \tpcie_capability_set_word(dev, PCI_EXP_DEVCTL2,\n> > > > \t\t\t\t PCI_EXP_DEVCTL2_ATOMIC_REQ);\n> > > > \treturn 0;\n> > > > diff --git a/include/linux/pci.h b/include/linux/pci.h\n> > > > index 1c270f1d512301de4d462fe7e5097c32af5c6f8d..ef90604c39859ea8e61e5392d0bdaa1b0e43874b 100644\n> > > > --- a/include/linux/pci.h\n> > > > +++ b/include/linux/pci.h\n> > > > @@ -692,6 +692,7 @@ void pci_set_host_bridge_release(struct pci_host_bridge *bridge,\n> > > > \t\t\t\t void *release_data);\n> > > > \n> > > > int pcibios_root_bridge_prepare(struct pci_host_bridge *bridge);\n> > > > +bool pcibios_connects_to_atomicops_capable_rc(struct pci_dev *dev, u32 cap_mask);\n> > > > \n> > > > #define PCI_REGION_FLAG_MASK\t0x0fU\t/* These bits of resource flags tell us the PCI region flags */\n> > > > \n> > > > \n> > > > -- \n> > > > 2.51.0\n> > > > \n> > \n> > Thanks,\n> > Gerd\n\nThanks,\nGerd","headers":{"Return-Path":"\n ","X-Original-To":["incoming@patchwork.ozlabs.org","linux-pci@vger.kernel.org"],"Delivered-To":"patchwork-incoming@legolas.ozlabs.org","Authentication-Results":["legolas.ozlabs.org;\n\tdkim=pass (2048-bit key;\n unprotected) header.d=ibm.com header.i=@ibm.com header.a=rsa-sha256\n header.s=pp1 header.b=GcKciWWr;\n\tdkim-atps=neutral","legolas.ozlabs.org;\n spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org\n (client-ip=2600:3c15:e001:75::12fc:5321; helo=sin.lore.kernel.org;\n envelope-from=linux-pci+bounces-51421-incoming=patchwork.ozlabs.org@vger.kernel.org;\n receiver=patchwork.ozlabs.org)","smtp.subspace.kernel.org;\n\tdkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com\n header.b=\"GcKciWWr\"","smtp.subspace.kernel.org;\n arc=none smtp.client-ip=148.163.156.1","smtp.subspace.kernel.org;\n dmarc=pass (p=none dis=none) header.from=linux.ibm.com","smtp.subspace.kernel.org;\n spf=pass smtp.mailfrom=linux.ibm.com"],"Received":["from sin.lore.kernel.org (sin.lore.kernel.org\n [IPv6:2600:3c15:e001:75::12fc:5321])\n\t(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)\n\t key-exchange x25519)\n\t(No client certificate requested)\n\tby legolas.ozlabs.org (Postfix) with ESMTPS id 4fklqF1PGwz1xrn\n\tfor ; Mon, 30 Mar 2026 20:11:21 +1100 (AEDT)","from smtp.subspace.kernel.org (conduit.subspace.kernel.org\n [100.90.174.1])\n\tby sin.lore.kernel.org (Postfix) with ESMTP id 4A0203015171\n\tfor ; Mon, 30 Mar 2026 08:59:50 +0000 (UTC)","from localhost.localdomain (localhost.localdomain [127.0.0.1])\n\tby smtp.subspace.kernel.org (Postfix) with ESMTP id 503493ACF16;\n\tMon, 30 Mar 2026 08:59:46 +0000 (UTC)","from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com\n [148.163.156.1])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))\n\t(No client certificate requested)\n\tby smtp.subspace.kernel.org (Postfix) with ESMTPS id B3956158DCF;\n\tMon, 30 Mar 2026 08:59:44 +0000 (UTC)","from pps.filterd (m0356517.ppops.net [127.0.0.1])\n\tby mx0a-001b2d01.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id\n 62TJtdcu4101669;\n\tMon, 30 Mar 2026 08:59:34 GMT","from ppma11.dal12v.mail.ibm.com\n (db.9e.1632.ip4.static.sl-reverse.com [50.22.158.219])\n\tby mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4d66q2x1hd-1\n\t(version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);\n\tMon, 30 Mar 2026 08:59:33 +0000 (GMT)","from pps.filterd (ppma11.dal12v.mail.ibm.com [127.0.0.1])\n\tby ppma11.dal12v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id\n 62U6R8jA008732;\n\tMon, 30 Mar 2026 08:59:32 GMT","from smtprelay06.fra02v.mail.ibm.com ([9.218.2.230])\n\tby ppma11.dal12v.mail.ibm.com (PPS) with ESMTPS id 4d6v11bwxf-1\n\t(version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);\n\tMon, 30 Mar 2026 08:59:32 +0000","from smtpav01.fra02v.mail.ibm.com (smtpav01.fra02v.mail.ibm.com\n [10.20.54.100])\n\tby smtprelay06.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id\n 62U8xT5x29360496\n\t(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK);\n\tMon, 30 Mar 2026 08:59:29 GMT","from smtpav01.fra02v.mail.ibm.com (unknown [127.0.0.1])\n\tby IMSVA (Postfix) with ESMTP id 42B012004D;\n\tMon, 30 Mar 2026 08:59:29 +0000 (GMT)","from smtpav01.fra02v.mail.ibm.com (unknown [127.0.0.1])\n\tby IMSVA (Postfix) with ESMTP id E7D012004B;\n\tMon, 30 Mar 2026 08:59:28 +0000 (GMT)","from [9.52.210.163] (unknown [9.52.210.163])\n\tby smtpav01.fra02v.mail.ibm.com (Postfix) with ESMTP;\n\tMon, 30 Mar 2026 08:59:28 +0000 (GMT)"],"ARC-Seal":"i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116;\n\tt=1774861186; cv=none;\n b=oKxdUtdozbYSEK6R45byQA9tUCrjhdXN6nftZtqI+aAYshAgnJeatW53wd5rWd2LGd7ur3vqxblWIl6zF6pTLUC4feN2TeJB7WMUreXkzItzGC1fV2JmeC1+CtRvkbQZitZqx6/D8mxESIrvw//u5O4z29CLVKbKyZ84HsHXO+A=","ARC-Message-Signature":"i=1; a=rsa-sha256; d=subspace.kernel.org;\n\ts=arc-20240116; t=1774861186; c=relaxed/simple;\n\tbh=giilVXXGkIuHlzuop8wJXfCCBPgtzG3AIY5OihAqbFc=;\n\th=Message-ID:Subject:From:To:Cc:Date:In-Reply-To:References:\n\t Content-Type:MIME-Version;\n b=NhIUJgdwKzHbPl0Mfa570tDuWntt2FzcFx3Ck2/zNavoZCYCUnfSROuTxjECw7R3zeUJA+GhlGJLlG78tlWx1s7i9Z0KMTFoEfeLt5vhXOYKAx6fHWoTpsNWXwY621yN2mRfKEZN5JXV3BgGbBme01qbUAizfvMnOVC8ZHuwiyQ=","ARC-Authentication-Results":"i=1; smtp.subspace.kernel.org;\n dmarc=pass (p=none dis=none) header.from=linux.ibm.com;\n spf=pass smtp.mailfrom=linux.ibm.com;\n dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com\n header.b=GcKciWWr; arc=none smtp.client-ip=148.163.156.1","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc\n\t:content-transfer-encoding:content-type:date:from:in-reply-to\n\t:message-id:mime-version:references:subject:to; s=pp1; bh=k792O7\n\tqTJqaxyo1o2gXXm95mAwWzi3w6ipuEO0MpZVI=; b=GcKciWWrKASWoFyyiBf1vG\n\tZLVV4lCObeEwbt2t4qaTQ8qDUQ/cNs17gLvoz+LpxLpdwT4WyuxCHHgtGElvuXg/\n\tDUx/SV4txfjVsSmMvkyy961i+bIFUSeZEa4md2Tt53rHFO5h+sLqXbT8up1W4jat\n\tk0Koawmm8TJNXtiMQAG7FVuPxDU8szoPT8dD6JmVtGCrahynqkIkrIksLJgJcVyb\n\tFGC13pv448RUnCWHherSEMTR47lY34X+cHDGUEns2H5mSxs0r+zJXVyDrTFhlWdK\n\tNIHJHoHA2hMSj/G0i1zb1c/A8/k+kEtUxqlikhf+OFPMjdFMFVcVnDfVDEzDYz5g\n\t==","Message-ID":"","Subject":"Re: [PATCH v6 1/2] PCI: AtomicOps: Do not enable without support in\n root complex","From":"Gerd Bayer ","To":"Bjorn Helgaas ","Cc":"Bjorn Helgaas , Jay Cornwall\n ,\n Felix Kuehling ,\n Christian\n Borntraeger ,\n Niklas Schnelle\t\n ,\n Gerald Schaefer ,\n Heiko Carstens , Vasily Gorbik ,\n Alexander Gordeev ,\n Sven Schnelle\n ,\n Leon Romanovsky ,\n Alexander Schmidt\n , linux-s390@vger.kernel.org,\n linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org,\n netdev@vger.kernel.org, linux-rdma@vger.kernel.org,\n stable@vger.kernel.org, Gerd Bayer ","Date":"Mon, 30 Mar 2026 10:59:28 +0200","In-Reply-To":"<20260326164002.GA1325368@bhelgaas>","References":"<20260326164002.GA1325368@bhelgaas>","Content-Type":"text/plain; charset=\"UTF-8\"","Content-Transfer-Encoding":"quoted-printable","User-Agent":"Evolution 3.58.3 (3.58.3-1.fc43) ","Precedence":"bulk","X-Mailing-List":"linux-pci@vger.kernel.org","List-Id":"","List-Subscribe":"","List-Unsubscribe":"","MIME-Version":"1.0","X-TM-AS-GCONF":"00","X-Proofpoint-Reinject":"loops=2 maxloops=12","X-Proofpoint-GUID":"8801-oVXRFUo1z80j49SlCDZHkOHnxqw","X-Authority-Analysis":"v=2.4 cv=frzRpV4f c=1 sm=1 tr=0 ts=69ca3b76 cx=c_pps\n a=aDMHemPKRhS1OARIsFnwRA==:117 a=aDMHemPKRhS1OARIsFnwRA==:17\n a=IkcTkHD0fZMA:10 a=Yq5XynenixoA:10 a=VkNPw1HP01LnGYTKEx00:22\n a=RnoormkPH1_aCDwRdu11:22 a=U7nrCbtTmkRpXpFmAIza:22 a=VnNF1IyMAAAA:8\n a=VwQbUJbxAAAA:8 a=DF3YyuCdIVBqkp6MlwEA:9 a=QEXdDO2ut3YA:10","X-Proofpoint-ORIG-GUID":"gkqxAUfufMUhyx0qPWi-92SNYRfuzxim","X-Proofpoint-Spam-Details-Enc":"AW1haW4tMjYwMzMwMDA2OCBTYWx0ZWRfX7s8l1gJSp1/d\n wKD2SeRvbLJA3NYu+8uicsGBpO03ZIqK/sP65UVFLBy6o9X0jmGnsBh+4JwcmrUbSKRTbmcfJLy\n iKS9kzAdb/qUmkITQlCVIbPaVF10PiOaTrOdMdFF7PhNCfpvnpO2WOTPSh+Jg6zaMS90VJf/U8c\n trVsrQ4op0VfZ3wItplJR50DaNxqX1Y/6H4OpMDLw+YTHf71qiXgrmoKOKoEKHmO5XL2JH1pa7r\n k4TDu64XsR49l7wz/SlaImrWUmQG7FMJ2AWo4OA9117W3oPIadzCsCWEryt4M6GRhNeqj1LEtDn\n RqweZRFUwNhRpQrnY6I8tBKYA9nZAjdvXGajh1CRmrPlHNxCsGzB+ZsFjkuEVQ85YxoM/sM2Cig\n 8ngEs8YihFZ+Nt0UQv+HTpm7NPNLm5fSW05EGTIpmlTMs/ZgFJ9yYs8Hr/6Epmy7Nd52x7nqbpp\n zjGvnBhKKaa58zdHpjg==","X-Proofpoint-Virus-Version":"vendor=baseguard\n engine=ICAP:2.0.293,Aquarius:18.0.1143,Hydra:6.1.51,FMLib:17.12.100.49\n definitions=2026-03-29_05,2026-03-28_01,2025-10-01_01","X-Proofpoint-Spam-Details":"rule=outbound_notspam policy=outbound score=0\n impostorscore=0 spamscore=0 priorityscore=1501 malwarescore=0 clxscore=1015\n lowpriorityscore=0 bulkscore=0 adultscore=0 suspectscore=0 phishscore=0\n classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0\n reason=mlx scancount=1 engine=8.22.0-2603050001 definitions=main-2603300068"}}]