| Message ID | 20261001-fix-p2p-acs-v4-0-v9-10-1a8e0f50ddd9@nvidia.com |
|---|---|
| State | New |
| Headers | show |
| Series | PCI/P2PDMA: Route peer-to-peer DMA by TLP class | expand |
On Thu, Oct 01, 2026 at 02:55:18PM +0300, Leon Romanovsky wrote: > From: Leon Romanovsky <leonro@nvidia.com> > > A Downstream Port with ACS Direct Translated P2P enabled routes a Request > whose Address Type is Translated "to the peer Egress Port without > redirection, regardless of ACS P2P Request Redirect and ACS P2P Egress > Control", per PCIe r7.0 sec 6.12.3. P2PDMA assumes every Request carries an > Untranslated address, so it sends an ATS client through the host bridge > even where the fabric would route it straight to the peer. "sends an ATS client through the host bridge" -- I assume this really means "we told the caller that Requests would be routed through the host bridge" when in reality they wouldn't? I don't think this actually changes any routing in the fabric, does it? So essentially we told the caller that P2P between A and B was, e.g., 5 hops when it was really only 2? > Add PCI_P2PDMA_TLP_TRANSLATED and consult Direct Translated P2P for the > Requests it describes. > > Reviewed-by: Logan Gunthorpe <logang@deltatee.com> > Tested-by: Tushar Dave <tdave@nvidia.com> > Signed-off-by: Leon Romanovsky <leonro@nvidia.com> > --- > drivers/pci/p2pdma.c | 9 +++++++++ > 1 file changed, 9 insertions(+) > > diff --git a/drivers/pci/p2pdma.c b/drivers/pci/p2pdma.c > index 569a74de3b3a..3fd2cb8d16f0 100644 > --- a/drivers/pci/p2pdma.c > +++ b/drivers/pci/p2pdma.c > @@ -548,6 +548,15 @@ pci_acs_p2pdma_request(u16 ctrl, unsigned int tlp_flags) > */ > if (ctrl & PCI_ACS_TB) > return PCI_ACS_P2PDMA_BLOCKED; > + > + /* > + * PCIe r7.0 sec 6.12.3: ACS Direct Translated P2P routes a > + * Request carrying a Translated address to the peer "without > + * redirection, regardless of ACS P2P Request Redirect and ACS > + * P2P Egress Control settings". > + */ > + if (ctrl & PCI_ACS_DT) > + return PCI_ACS_P2PDMA_DIRECT; > } > > return ctrl & (PCI_ACS_RR | PCI_ACS_EC) ? > > -- > 2.55.0 >
On Tue, Oct 06, 2026 at 05:28:45PM -0500, Bjorn Helgaas wrote: > On Thu, Oct 01, 2026 at 02:55:18PM +0300, Leon Romanovsky wrote: > > From: Leon Romanovsky <leonro@nvidia.com> > > > > A Downstream Port with ACS Direct Translated P2P enabled routes a Request > > whose Address Type is Translated "to the peer Egress Port without > > redirection, regardless of ACS P2P Request Redirect and ACS P2P Egress > > Control", per PCIe r7.0 sec 6.12.3. P2PDMA assumes every Request carries an > > Untranslated address, so it sends an ATS client through the host bridge > > even where the fabric would route it straight to the peer. > > "sends an ATS client through the host bridge" -- I assume this really > means "we told the caller that Requests would be routed through the > host bridge" when in reality they wouldn't? I don't think this > actually changes any routing in the fabric, does it? I'd say it does. Before this patch, we didn't return `bus_address` here nor took the P2P path. Thanks > > So essentially we told the caller that P2P between A and B was, e.g., > 5 hops when it was really only 2? > > > Add PCI_P2PDMA_TLP_TRANSLATED and consult Direct Translated P2P for the > > Requests it describes. > > > > Reviewed-by: Logan Gunthorpe <logang@deltatee.com> > > Tested-by: Tushar Dave <tdave@nvidia.com> > > Signed-off-by: Leon Romanovsky <leonro@nvidia.com> > > --- > > drivers/pci/p2pdma.c | 9 +++++++++ > > 1 file changed, 9 insertions(+) > > > > diff --git a/drivers/pci/p2pdma.c b/drivers/pci/p2pdma.c > > index 569a74de3b3a..3fd2cb8d16f0 100644 > > --- a/drivers/pci/p2pdma.c > > +++ b/drivers/pci/p2pdma.c > > @@ -548,6 +548,15 @@ pci_acs_p2pdma_request(u16 ctrl, unsigned int tlp_flags) > > */ > > if (ctrl & PCI_ACS_TB) > > return PCI_ACS_P2PDMA_BLOCKED; > > + > > + /* > > + * PCIe r7.0 sec 6.12.3: ACS Direct Translated P2P routes a > > + * Request carrying a Translated address to the peer "without > > + * redirection, regardless of ACS P2P Request Redirect and ACS > > + * P2P Egress Control settings". > > + */ > > + if (ctrl & PCI_ACS_DT) > > + return PCI_ACS_P2PDMA_DIRECT; > > } > > > > return ctrl & (PCI_ACS_RR | PCI_ACS_EC) ? > > > > -- > > 2.55.0 > >
diff --git a/drivers/pci/p2pdma.c b/drivers/pci/p2pdma.c index 569a74de3b3a..3fd2cb8d16f0 100644 --- a/drivers/pci/p2pdma.c +++ b/drivers/pci/p2pdma.c @@ -548,6 +548,15 @@ pci_acs_p2pdma_request(u16 ctrl, unsigned int tlp_flags) */ if (ctrl & PCI_ACS_TB) return PCI_ACS_P2PDMA_BLOCKED; + + /* + * PCIe r7.0 sec 6.12.3: ACS Direct Translated P2P routes a + * Request carrying a Translated address to the peer "without + * redirection, regardless of ACS P2P Request Redirect and ACS + * P2P Egress Control settings". + */ + if (ctrl & PCI_ACS_DT) + return PCI_ACS_P2PDMA_DIRECT; } return ctrl & (PCI_ACS_RR | PCI_ACS_EC) ?