| Message ID | 20220830155224.103907-1-helgaas@kernel.org |
|---|---|
| State | New |
| Headers | show |
| Series | PCI: Disable PTM on Upstream Ports during suspend | expand |
On 8/30/22 8:52 AM, Bjorn Helgaas wrote: > From: Bjorn Helgaas <bhelgaas@google.com> > > During suspend, we want to disable PTM on Root Ports because that allows > some chips, e.g., Intel mobile chips since Coffee Lake, to enter a > lower-power PM state. > > Previously we only disabled PTM for Root Ports, but PCIe r6.0, sec 2.2.8, > strongly recommends that functions support generation of Messages in non-D0 > states, so we must assume that Switch Upstream Ports or Endpoints may send > PTM Request Messages while in D1, D2, and D3hot. > > A PTM Message received by a Downstream Port, e.g., a Root Port, with PTM > disabled must be treated as an Unsupported Request (sec 6.21.3). > > With this topology: > > 0000:00:1d.0 Root Port to [bus 08-71] > 0000:08:00.0 Switch Upstream Port to [bus 09-71] > > Kai-Heng reported errors like this: > > pcieport 0000:00:1d.0: AER: Uncorrected (Non-Fatal) error received: 0000:00:1d.0 > pcieport 0000:00:1d.0: PCIe Bus Error: severity=Uncorrected (Non-Fatal), type=Transaction Layer, (Requester ID) > pcieport 0000:00:1d.0: device [8086:7ab0] error status/mask=00100000/00004000 > pcieport 0000:00:1d.0: [20] UnsupReq (First) > pcieport 0000:00:1d.0: AER: TLP Header: 34000000 08000052 00000000 00000000 > > Decoding this (from PCIe r6.0, sec 2.2.1.1, 2.2.8.10) shows that 00:1d.0 > logged an Unsupported Request error when it received a PTM Request with > Requester ID 08:00.0. > > To prevent this error, disable PTM when suspending Upstream Ports > (including those on Endpoints), not just Root Ports. > > Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=215453 > Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=216210 > Based-on: https://lore.kernel.org/r/20220706123244.18056-1-kai.heng.feng@canonical.com > Based-on-patch-by: Kai-Heng Feng <kai.heng.feng@canonical.com> > Reported-by: Kai-Heng Feng <kai.heng.feng@canonical.com> > Tested-by: Kai-Heng Feng <kai.heng.feng@canonical.com> > Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> > Cc: David E. Box <david.e.box@linux.intel.com> > Cc: Sathyanarayanan Kuppuswamy <sathyanarayanan.kuppuswamy@linux.intel.com> > --- Looks good to me Reviewed-by: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com> > drivers/pci/pci.c | 30 ++++++++++++++---------------- > drivers/pci/pcie/ptm.c | 8 +++++++- > 2 files changed, 21 insertions(+), 17 deletions(-) > > diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c > index 95bc329e74c0..96487a9ce5bf 100644 > --- a/drivers/pci/pci.c > +++ b/drivers/pci/pci.c > @@ -2707,14 +2707,19 @@ int pci_prepare_to_sleep(struct pci_dev *dev) > return -EIO; > > /* > - * There are systems (for example, Intel mobile chips since Coffee > - * Lake) where the power drawn while suspended can be significantly > - * reduced by disabling PTM on PCIe root ports as this allows the > - * port to enter a lower-power PM state and the SoC to reach a > - * lower-power idle state as a whole. > + * We want to disable PTM on Root Ports because that allows some > + * chips, e.g., Intel mobile chips since Coffee Lake, to enter a > + * lower-power PM state. > + * > + * PCIe r6.0, sec 2.2.8, strongly recommends that functions support > + * generation of messages in non-D0 states, so we assume Switch > + * Upstream Ports or Endpoints may send PTM Requests while in D1, > + * D2, and D3hot. A PTM message received by a Downstream Port > + * (including a Root Port) with PTM disabled must be treated as an > + * Unsupported Request (sec 6.21.3). To prevent this error, > + * disable PTM in *all* devices, not just Root Ports. > */ > - if (pci_pcie_type(dev) == PCI_EXP_TYPE_ROOT_PORT) > - pci_disable_ptm(dev); > + pci_disable_ptm(dev); > > pci_enable_wake(dev, target_state, wakeup); > > @@ -2764,15 +2769,8 @@ int pci_finish_runtime_suspend(struct pci_dev *dev) > if (target_state == PCI_POWER_ERROR) > return -EIO; > > - /* > - * There are systems (for example, Intel mobile chips since Coffee > - * Lake) where the power drawn while suspended can be significantly > - * reduced by disabling PTM on PCIe root ports as this allows the > - * port to enter a lower-power PM state and the SoC to reach a > - * lower-power idle state as a whole. > - */ > - if (pci_pcie_type(dev) == PCI_EXP_TYPE_ROOT_PORT) > - pci_disable_ptm(dev); > + /* See rationale above for disabling PTM */ > + pci_disable_ptm(dev); > > __pci_enable_wake(dev, target_state, pci_dev_run_wake(dev)); > > diff --git a/drivers/pci/pcie/ptm.c b/drivers/pci/pcie/ptm.c > index 368a254e3124..ec338470d13f 100644 > --- a/drivers/pci/pcie/ptm.c > +++ b/drivers/pci/pcie/ptm.c > @@ -31,12 +31,18 @@ static void pci_ptm_info(struct pci_dev *dev) > > void pci_disable_ptm(struct pci_dev *dev) > { > - int ptm; > + int type, ptm; > u16 ctrl; > > if (!pci_is_pcie(dev)) > return; > > + type = pci_pcie_type(dev); > + if (!(type == PCI_EXP_TYPE_ROOT_PORT || > + type == PCI_EXP_TYPE_UPSTREAM || > + type == PCI_EXP_TYPE_ENDPOINT)) > + return; > + > ptm = pci_find_ext_capability(dev, PCI_EXT_CAP_ID_PTM); > if (!ptm) > return;
Hi Bjorn, On Tue, Aug 30, 2022 at 10:52:24AM -0500, Bjorn Helgaas wrote: > + type = pci_pcie_type(dev); > + if (!(type == PCI_EXP_TYPE_ROOT_PORT || > + type == PCI_EXP_TYPE_UPSTREAM || > + type == PCI_EXP_TYPE_ENDPOINT)) > + return; Perhaps switch () instead? switch (pci_pcie_type(dev)) { case PCI_EXP_TYPE_ROOT_PORT: case PCI_EXP_TYPE_UPSTREAM: case PCI_EXP_TYPE_ENDPOINT: break; default: return; } Either way, Reviewed-by: Mika Westerberg <mika.westerberg@linux.intel.com>
[+cc Rafael, Rajvi, linux-pm] On Wed, Aug 31, 2022 at 12:53:05PM +0300, Mika Westerberg wrote: > On Tue, Aug 30, 2022 at 10:52:24AM -0500, Bjorn Helgaas wrote: > > + type = pci_pcie_type(dev); > > + if (!(type == PCI_EXP_TYPE_ROOT_PORT || > > + type == PCI_EXP_TYPE_UPSTREAM || > > + type == PCI_EXP_TYPE_ENDPOINT)) > > + return; > > Perhaps switch () instead? > > switch (pci_pcie_type(dev)) { > case PCI_EXP_TYPE_ROOT_PORT: > case PCI_EXP_TYPE_UPSTREAM: > case PCI_EXP_TYPE_ENDPOINT: > break; > default: > return; > } > > Either way, > > Reviewed-by: Mika Westerberg <mika.westerberg@linux.intel.com> Thanks! I dropped this patch for now. I think Rajvi's similar patch [1] is much better because we only need this kind of check in one place -- where we enable PTM. Also, we need to tighten this up so we *always* disable PTM [2]. The patch I posted still calls pci_disable_ptm() from pci_prepare_to_sleep(), which means we only do it when !pdev->state_saved, so we leave PTM enabled if the driver saves its own state. Bjorn [1] https://lore.kernel.org/r/20220830104913.1620539-2-rajvi.jingar@linux.intel.com [2] https://lore.kernel.org/r/CAJZ5v0iHckqia4OywKzSNWFCaq7eOkJcm5yXJdT2_sNdd36gDw@mail.gmail.com
diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c index 95bc329e74c0..96487a9ce5bf 100644 --- a/drivers/pci/pci.c +++ b/drivers/pci/pci.c @@ -2707,14 +2707,19 @@ int pci_prepare_to_sleep(struct pci_dev *dev) return -EIO; /* - * There are systems (for example, Intel mobile chips since Coffee - * Lake) where the power drawn while suspended can be significantly - * reduced by disabling PTM on PCIe root ports as this allows the - * port to enter a lower-power PM state and the SoC to reach a - * lower-power idle state as a whole. + * We want to disable PTM on Root Ports because that allows some + * chips, e.g., Intel mobile chips since Coffee Lake, to enter a + * lower-power PM state. + * + * PCIe r6.0, sec 2.2.8, strongly recommends that functions support + * generation of messages in non-D0 states, so we assume Switch + * Upstream Ports or Endpoints may send PTM Requests while in D1, + * D2, and D3hot. A PTM message received by a Downstream Port + * (including a Root Port) with PTM disabled must be treated as an + * Unsupported Request (sec 6.21.3). To prevent this error, + * disable PTM in *all* devices, not just Root Ports. */ - if (pci_pcie_type(dev) == PCI_EXP_TYPE_ROOT_PORT) - pci_disable_ptm(dev); + pci_disable_ptm(dev); pci_enable_wake(dev, target_state, wakeup); @@ -2764,15 +2769,8 @@ int pci_finish_runtime_suspend(struct pci_dev *dev) if (target_state == PCI_POWER_ERROR) return -EIO; - /* - * There are systems (for example, Intel mobile chips since Coffee - * Lake) where the power drawn while suspended can be significantly - * reduced by disabling PTM on PCIe root ports as this allows the - * port to enter a lower-power PM state and the SoC to reach a - * lower-power idle state as a whole. - */ - if (pci_pcie_type(dev) == PCI_EXP_TYPE_ROOT_PORT) - pci_disable_ptm(dev); + /* See rationale above for disabling PTM */ + pci_disable_ptm(dev); __pci_enable_wake(dev, target_state, pci_dev_run_wake(dev)); diff --git a/drivers/pci/pcie/ptm.c b/drivers/pci/pcie/ptm.c index 368a254e3124..ec338470d13f 100644 --- a/drivers/pci/pcie/ptm.c +++ b/drivers/pci/pcie/ptm.c @@ -31,12 +31,18 @@ static void pci_ptm_info(struct pci_dev *dev) void pci_disable_ptm(struct pci_dev *dev) { - int ptm; + int type, ptm; u16 ctrl; if (!pci_is_pcie(dev)) return; + type = pci_pcie_type(dev); + if (!(type == PCI_EXP_TYPE_ROOT_PORT || + type == PCI_EXP_TYPE_UPSTREAM || + type == PCI_EXP_TYPE_ENDPOINT)) + return; + ptm = pci_find_ext_capability(dev, PCI_EXT_CAP_ID_PTM); if (!ptm) return;