| Message ID | 1511763628-11856-4-git-send-email-okaya@codeaurora.org |
|---|---|
| State | Not Applicable |
| Delegated to: | Bjorn Helgaas |
| Headers | show |
| Series | [V2,1/7] PCI: protect restore with device lock to be consistent | expand |
On Mon, Nov 27, 2017 at 01:20:24AM -0500, Sinan Kaya wrote: > Rev 3.1 Sec 2.3.1 Request Handling Rules: > Valid reset conditions after which a device is permitted to return CRS > are: > * Cold, Warm, and Hot Resets, > * FLR > * A reset initiated in response to a D3hot to D0 uninitialized > > Try to reuse FLR implementation towards other reset types. Should we keep a helper for the FLR wait? If not I guess the reference for AF_FLR should be the PCI spec, not the PCIE spec.
On 11/29/2017 12:38 PM, Christoph Hellwig wrote: > On Mon, Nov 27, 2017 at 01:20:24AM -0500, Sinan Kaya wrote: >> Rev 3.1 Sec 2.3.1 Request Handling Rules: >> Valid reset conditions after which a device is permitted to return CRS >> are: >> * Cold, Warm, and Hot Resets, >> * FLR >> * A reset initiated in response to a D3hot to D0 uninitialized >> >> Try to reuse FLR implementation towards other reset types. > > Should we keep a helper for the FLR wait? If not I guess the reference > for AF_FLR should be the PCI spec, not the PCIE spec. > Bjorn, what is your opinion? Sinan
On Mon, Nov 27, 2017 at 01:20:24AM -0500, Sinan Kaya wrote: > Rev 3.1 Sec 2.3.1 Request Handling Rules: > Valid reset conditions after which a device is permitted to return CRS > are: > * Cold, Warm, and Hot Resets, > * FLR > * A reset initiated in response to a D3hot to D0 uninitialized > > Try to reuse FLR implementation towards other reset types. > > Signed-off-by: Sinan Kaya <okaya@codeaurora.org> > --- > drivers/pci/pci.c | 47 ++++++++++++++++++++++++++++++----------------- > 1 file changed, 30 insertions(+), 17 deletions(-) > > diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c > index 27ec45d..ed3c3bc 100644 > --- a/drivers/pci/pci.c > +++ b/drivers/pci/pci.c > @@ -125,6 +125,9 @@ static int __init pcie_port_pm_setup(char *str) > } > __setup("pcie_port_pm=", pcie_port_pm_setup); > > +/* time to wait after a reset for device to become responsive */ > +#define PCIE_RESET_READY_POLL_MS 60000 > + > /** > * pci_bus_max_busnr - returns maximum PCI bus number of given bus' children > * @bus: pointer to PCI bus structure to search > @@ -3820,20 +3823,13 @@ int pci_wait_for_pending_transaction(struct pci_dev *dev) > } > EXPORT_SYMBOL(pci_wait_for_pending_transaction); > > -static int pci_flr_wait(struct pci_dev *dev) > +static int pci_dev_wait(struct pci_dev *dev, char *reset_type, int timeout) > { > - int delay = 1, timeout = 60000; > + int delay = 1; > u32 id; > > /* > - * Per PCIe r3.1, sec 6.6.2, a device must complete an FLR within > - * 100ms, but may silently discard requests while the FLR is in > - * progress. Wait 100ms before trying to access the device. > - */ > - msleep(100); > - > - /* > - * After 100ms, the device should not silently discard config > + * After reset, the device should not silently discard config > * requests, but it may still indicate that it needs more time by > * responding to them with CRS completions. The Root Port will > * generally synthesize ~0 data to complete the read (except when > @@ -3847,14 +3843,14 @@ static int pci_flr_wait(struct pci_dev *dev) > pci_read_config_dword(dev, PCI_COMMAND, &id); > while (id == ~0) { > if (delay > timeout) { > - dev_warn(&dev->dev, "not ready %dms after FLR; giving up\n", > - 100 + delay - 1); > + dev_warn(&dev->dev, "not ready %dms after %s; giving up\n", > + delay - 1, reset_type); > return -ENOTTY; > } > > if (delay > 1000) > - dev_info(&dev->dev, "not ready %dms after FLR; waiting\n", > - 100 + delay - 1); > + dev_info(&dev->dev, "not ready %dms after %s; waiting\n", > + delay - 1, reset_type); > > msleep(delay); > delay *= 2; > @@ -3862,7 +3858,8 @@ static int pci_flr_wait(struct pci_dev *dev) > } > > if (delay > 1000) > - dev_info(&dev->dev, "ready %dms after FLR\n", 100 + delay - 1); > + dev_info(&dev->dev, "ready %dms after %s\n", delay - 1, > + reset_type); > > return 0; > } > @@ -3899,7 +3896,15 @@ int pcie_flr(struct pci_dev *dev) > dev_err(&dev->dev, "timed out waiting for pending transaction; performing function level reset anyway\n"); > > pcie_capability_set_word(dev, PCI_EXP_DEVCTL, PCI_EXP_DEVCTL_BCR_FLR); > - return pci_flr_wait(dev); > + > + /* > + * Per PCIe r3.1, sec 6.6.2, a device must complete an FLR within > + * 100ms, but may silently discard requests while the FLR is in > + * progress. Wait 100ms before trying to access the device. > + */ > + msleep(100); > + > + return pci_dev_wait(dev, "FLR", PCIE_RESET_READY_POLL_MS); > } > EXPORT_SYMBOL_GPL(pcie_flr); > > @@ -3932,7 +3937,15 @@ static int pci_af_flr(struct pci_dev *dev, int probe) > dev_err(&dev->dev, "timed out waiting for pending transaction; performing AF function level reset anyway\n"); > > pci_write_config_byte(dev, pos + PCI_AF_CTRL, PCI_AF_CTRL_FLR); > - return pci_flr_wait(dev); > + > + /* > + * Per PCIe r3.1, sec 6.6.2, a device must complete an FLR within I think this should reference the "Advanced Capabilities for Conventional PCI" ECN, shouldn't it? The one I see is dated 13 April 2006, updated 27 July 2006, and I don't see a PCI spec that includes it. > + * 100ms, but may silently discard requests while the FLR is in > + * progress. Wait 100ms before trying to access the device. > + */ > + msleep(100); > + > + return pci_dev_wait(dev, "AF_FLR", PCIE_RESET_READY_POLL_MS); CRS is not applicable to conventional PCI. The ECN mentions waiting 100ms. I don't see anything about polling after that, but I guess it probably doesn't hurt anything. > } > > /** > -- > 1.9.1 > > > _______________________________________________ > linux-arm-kernel mailing list > linux-arm-kernel@lists.infradead.org > http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
On 12/13/2017 5:43 PM, Bjorn Helgaas wrote: >> + >> + /* >> + * Per PCIe r3.1, sec 6.6.2, a device must complete an FLR within > I think this should reference the "Advanced Capabilities for > Conventional PCI" ECN, shouldn't it? The one I see is dated 13 April > 2006, updated 27 July 2006, and I don't see a PCI spec that includes > it. I'll add reference to PCI spec as well. Any other comments on the series? > >> + * 100ms, but may silently discard requests while the FLR is in >> + * progress. Wait 100ms before trying to access the device. >> + */ >> + msleep(100); >> + >> + return pci_dev_wait(dev, "AF_FLR", PCIE_RESET_READY_POLL_MS); > CRS is not applicable to conventional PCI. The ECN mentions waiting > 100ms. I don't see anything about polling after that, but I guess it > probably doesn't hurt anything. ok, I'll keep it as it is. > >> }
diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c index 27ec45d..ed3c3bc 100644 --- a/drivers/pci/pci.c +++ b/drivers/pci/pci.c @@ -125,6 +125,9 @@ static int __init pcie_port_pm_setup(char *str) } __setup("pcie_port_pm=", pcie_port_pm_setup); +/* time to wait after a reset for device to become responsive */ +#define PCIE_RESET_READY_POLL_MS 60000 + /** * pci_bus_max_busnr - returns maximum PCI bus number of given bus' children * @bus: pointer to PCI bus structure to search @@ -3820,20 +3823,13 @@ int pci_wait_for_pending_transaction(struct pci_dev *dev) } EXPORT_SYMBOL(pci_wait_for_pending_transaction); -static int pci_flr_wait(struct pci_dev *dev) +static int pci_dev_wait(struct pci_dev *dev, char *reset_type, int timeout) { - int delay = 1, timeout = 60000; + int delay = 1; u32 id; /* - * Per PCIe r3.1, sec 6.6.2, a device must complete an FLR within - * 100ms, but may silently discard requests while the FLR is in - * progress. Wait 100ms before trying to access the device. - */ - msleep(100); - - /* - * After 100ms, the device should not silently discard config + * After reset, the device should not silently discard config * requests, but it may still indicate that it needs more time by * responding to them with CRS completions. The Root Port will * generally synthesize ~0 data to complete the read (except when @@ -3847,14 +3843,14 @@ static int pci_flr_wait(struct pci_dev *dev) pci_read_config_dword(dev, PCI_COMMAND, &id); while (id == ~0) { if (delay > timeout) { - dev_warn(&dev->dev, "not ready %dms after FLR; giving up\n", - 100 + delay - 1); + dev_warn(&dev->dev, "not ready %dms after %s; giving up\n", + delay - 1, reset_type); return -ENOTTY; } if (delay > 1000) - dev_info(&dev->dev, "not ready %dms after FLR; waiting\n", - 100 + delay - 1); + dev_info(&dev->dev, "not ready %dms after %s; waiting\n", + delay - 1, reset_type); msleep(delay); delay *= 2; @@ -3862,7 +3858,8 @@ static int pci_flr_wait(struct pci_dev *dev) } if (delay > 1000) - dev_info(&dev->dev, "ready %dms after FLR\n", 100 + delay - 1); + dev_info(&dev->dev, "ready %dms after %s\n", delay - 1, + reset_type); return 0; } @@ -3899,7 +3896,15 @@ int pcie_flr(struct pci_dev *dev) dev_err(&dev->dev, "timed out waiting for pending transaction; performing function level reset anyway\n"); pcie_capability_set_word(dev, PCI_EXP_DEVCTL, PCI_EXP_DEVCTL_BCR_FLR); - return pci_flr_wait(dev); + + /* + * Per PCIe r3.1, sec 6.6.2, a device must complete an FLR within + * 100ms, but may silently discard requests while the FLR is in + * progress. Wait 100ms before trying to access the device. + */ + msleep(100); + + return pci_dev_wait(dev, "FLR", PCIE_RESET_READY_POLL_MS); } EXPORT_SYMBOL_GPL(pcie_flr); @@ -3932,7 +3937,15 @@ static int pci_af_flr(struct pci_dev *dev, int probe) dev_err(&dev->dev, "timed out waiting for pending transaction; performing AF function level reset anyway\n"); pci_write_config_byte(dev, pos + PCI_AF_CTRL, PCI_AF_CTRL_FLR); - return pci_flr_wait(dev); + + /* + * Per PCIe r3.1, sec 6.6.2, a device must complete an FLR within + * 100ms, but may silently discard requests while the FLR is in + * progress. Wait 100ms before trying to access the device. + */ + msleep(100); + + return pci_dev_wait(dev, "AF_FLR", PCIE_RESET_READY_POLL_MS); } /**
Rev 3.1 Sec 2.3.1 Request Handling Rules: Valid reset conditions after which a device is permitted to return CRS are: * Cold, Warm, and Hot Resets, * FLR * A reset initiated in response to a D3hot to D0 uninitialized Try to reuse FLR implementation towards other reset types. Signed-off-by: Sinan Kaya <okaya@codeaurora.org> --- drivers/pci/pci.c | 47 ++++++++++++++++++++++++++++++----------------- 1 file changed, 30 insertions(+), 17 deletions(-)