[V2,3/7] PCI: make pci_flr_wait() generic and rename to pci_dev_wait()

Message ID 1511763628-11856-4-git-send-email-okaya@codeaurora.org
State New
Delegated to: Bjorn Helgaas
Headers show
Series
  • [V2,1/7] PCI: protect restore with device lock to be consistent
Related show

Commit Message

Sinan Kaya Nov. 27, 2017, 6:20 a.m.
Rev 3.1 Sec 2.3.1 Request Handling Rules:
Valid reset conditions after which a device is permitted to return CRS
are:
* Cold, Warm, and Hot Resets,
* FLR
* A reset initiated in response to a D3hot to D0 uninitialized

Try to reuse FLR implementation towards other reset types.

Signed-off-by: Sinan Kaya <okaya@codeaurora.org>
---
 drivers/pci/pci.c | 47 ++++++++++++++++++++++++++++++-----------------
 1 file changed, 30 insertions(+), 17 deletions(-)

Comments

Christoph Hellwig Nov. 29, 2017, 5:38 p.m. | #1
On Mon, Nov 27, 2017 at 01:20:24AM -0500, Sinan Kaya wrote:
> Rev 3.1 Sec 2.3.1 Request Handling Rules:
> Valid reset conditions after which a device is permitted to return CRS
> are:
> * Cold, Warm, and Hot Resets,
> * FLR
> * A reset initiated in response to a D3hot to D0 uninitialized
> 
> Try to reuse FLR implementation towards other reset types.

Should we keep a helper for the FLR wait?  If not I guess the reference
for AF_FLR should be the PCI spec, not the PCIE spec.
Sinan Kaya Dec. 4, 2017, 1:17 p.m. | #2
On 11/29/2017 12:38 PM, Christoph Hellwig wrote:
> On Mon, Nov 27, 2017 at 01:20:24AM -0500, Sinan Kaya wrote:
>> Rev 3.1 Sec 2.3.1 Request Handling Rules:
>> Valid reset conditions after which a device is permitted to return CRS
>> are:
>> * Cold, Warm, and Hot Resets,
>> * FLR
>> * A reset initiated in response to a D3hot to D0 uninitialized
>>
>> Try to reuse FLR implementation towards other reset types.
> 
> Should we keep a helper for the FLR wait?  If not I guess the reference
> for AF_FLR should be the PCI spec, not the PCIE spec.
> 

Bjorn,

what is your opinion?

Sinan
Bjorn Helgaas Dec. 13, 2017, 10:43 p.m. | #3
On Mon, Nov 27, 2017 at 01:20:24AM -0500, Sinan Kaya wrote:
> Rev 3.1 Sec 2.3.1 Request Handling Rules:
> Valid reset conditions after which a device is permitted to return CRS
> are:
> * Cold, Warm, and Hot Resets,
> * FLR
> * A reset initiated in response to a D3hot to D0 uninitialized
> 
> Try to reuse FLR implementation towards other reset types.
> 
> Signed-off-by: Sinan Kaya <okaya@codeaurora.org>
> ---
>  drivers/pci/pci.c | 47 ++++++++++++++++++++++++++++++-----------------
>  1 file changed, 30 insertions(+), 17 deletions(-)
> 
> diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
> index 27ec45d..ed3c3bc 100644
> --- a/drivers/pci/pci.c
> +++ b/drivers/pci/pci.c
> @@ -125,6 +125,9 @@ static int __init pcie_port_pm_setup(char *str)
>  }
>  __setup("pcie_port_pm=", pcie_port_pm_setup);
>  
> +/* time to wait after a reset for device to become responsive */
> +#define PCIE_RESET_READY_POLL_MS 60000
> +
>  /**
>   * pci_bus_max_busnr - returns maximum PCI bus number of given bus' children
>   * @bus: pointer to PCI bus structure to search
> @@ -3820,20 +3823,13 @@ int pci_wait_for_pending_transaction(struct pci_dev *dev)
>  }
>  EXPORT_SYMBOL(pci_wait_for_pending_transaction);
>  
> -static int pci_flr_wait(struct pci_dev *dev)
> +static int pci_dev_wait(struct pci_dev *dev, char *reset_type, int timeout)
>  {
> -	int delay = 1, timeout = 60000;
> +	int delay = 1;
>  	u32 id;
>  
>  	/*
> -	 * Per PCIe r3.1, sec 6.6.2, a device must complete an FLR within
> -	 * 100ms, but may silently discard requests while the FLR is in
> -	 * progress.  Wait 100ms before trying to access the device.
> -	 */
> -	msleep(100);
> -
> -	/*
> -	 * After 100ms, the device should not silently discard config
> +	 * After reset, the device should not silently discard config
>  	 * requests, but it may still indicate that it needs more time by
>  	 * responding to them with CRS completions.  The Root Port will
>  	 * generally synthesize ~0 data to complete the read (except when
> @@ -3847,14 +3843,14 @@ static int pci_flr_wait(struct pci_dev *dev)
>  	pci_read_config_dword(dev, PCI_COMMAND, &id);
>  	while (id == ~0) {
>  		if (delay > timeout) {
> -			dev_warn(&dev->dev, "not ready %dms after FLR; giving up\n",
> -				 100 + delay - 1);
> +			dev_warn(&dev->dev, "not ready %dms after %s; giving up\n",
> +				 delay - 1, reset_type);
>  			return -ENOTTY;
>  		}
>  
>  		if (delay > 1000)
> -			dev_info(&dev->dev, "not ready %dms after FLR; waiting\n",
> -				 100 + delay - 1);
> +			dev_info(&dev->dev, "not ready %dms after %s; waiting\n",
> +				 delay - 1, reset_type);
>  
>  		msleep(delay);
>  		delay *= 2;
> @@ -3862,7 +3858,8 @@ static int pci_flr_wait(struct pci_dev *dev)
>  	}
>  
>  	if (delay > 1000)
> -		dev_info(&dev->dev, "ready %dms after FLR\n", 100 + delay - 1);
> +		dev_info(&dev->dev, "ready %dms after %s\n", delay - 1,
> +			 reset_type);
>  
>  	return 0;
>  }
> @@ -3899,7 +3896,15 @@ int pcie_flr(struct pci_dev *dev)
>  		dev_err(&dev->dev, "timed out waiting for pending transaction; performing function level reset anyway\n");
>  
>  	pcie_capability_set_word(dev, PCI_EXP_DEVCTL, PCI_EXP_DEVCTL_BCR_FLR);
> -	return pci_flr_wait(dev);
> +
> +	/*
> +	 * Per PCIe r3.1, sec 6.6.2, a device must complete an FLR within
> +	 * 100ms, but may silently discard requests while the FLR is in
> +	 * progress.  Wait 100ms before trying to access the device.
> +	 */
> +	msleep(100);
> +
> +	return pci_dev_wait(dev, "FLR", PCIE_RESET_READY_POLL_MS);
>  }
>  EXPORT_SYMBOL_GPL(pcie_flr);
>  
> @@ -3932,7 +3937,15 @@ static int pci_af_flr(struct pci_dev *dev, int probe)
>  		dev_err(&dev->dev, "timed out waiting for pending transaction; performing AF function level reset anyway\n");
>  
>  	pci_write_config_byte(dev, pos + PCI_AF_CTRL, PCI_AF_CTRL_FLR);
> -	return pci_flr_wait(dev);
> +
> +	/*
> +	 * Per PCIe r3.1, sec 6.6.2, a device must complete an FLR within

I think this should reference the "Advanced Capabilities for
Conventional PCI" ECN, shouldn't it?  The one I see is dated 13 April
2006, updated 27 July 2006, and I don't see a PCI spec that includes
it.

> +	 * 100ms, but may silently discard requests while the FLR is in
> +	 * progress.  Wait 100ms before trying to access the device.
> +	 */
> +	msleep(100);
> +
> +	return pci_dev_wait(dev, "AF_FLR", PCIE_RESET_READY_POLL_MS);

CRS is not applicable to conventional PCI.  The ECN mentions waiting
100ms.  I don't see anything about polling after that, but I guess it
probably doesn't hurt anything.

>  }
>  
>  /**
> -- 
> 1.9.1
> 
> 
> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Patch

diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
index 27ec45d..ed3c3bc 100644
--- a/drivers/pci/pci.c
+++ b/drivers/pci/pci.c
@@ -125,6 +125,9 @@  static int __init pcie_port_pm_setup(char *str)
 }
 __setup("pcie_port_pm=", pcie_port_pm_setup);
 
+/* time to wait after a reset for device to become responsive */
+#define PCIE_RESET_READY_POLL_MS 60000
+
 /**
  * pci_bus_max_busnr - returns maximum PCI bus number of given bus' children
  * @bus: pointer to PCI bus structure to search
@@ -3820,20 +3823,13 @@  int pci_wait_for_pending_transaction(struct pci_dev *dev)
 }
 EXPORT_SYMBOL(pci_wait_for_pending_transaction);
 
-static int pci_flr_wait(struct pci_dev *dev)
+static int pci_dev_wait(struct pci_dev *dev, char *reset_type, int timeout)
 {
-	int delay = 1, timeout = 60000;
+	int delay = 1;
 	u32 id;
 
 	/*
-	 * Per PCIe r3.1, sec 6.6.2, a device must complete an FLR within
-	 * 100ms, but may silently discard requests while the FLR is in
-	 * progress.  Wait 100ms before trying to access the device.
-	 */
-	msleep(100);
-
-	/*
-	 * After 100ms, the device should not silently discard config
+	 * After reset, the device should not silently discard config
 	 * requests, but it may still indicate that it needs more time by
 	 * responding to them with CRS completions.  The Root Port will
 	 * generally synthesize ~0 data to complete the read (except when
@@ -3847,14 +3843,14 @@  static int pci_flr_wait(struct pci_dev *dev)
 	pci_read_config_dword(dev, PCI_COMMAND, &id);
 	while (id == ~0) {
 		if (delay > timeout) {
-			dev_warn(&dev->dev, "not ready %dms after FLR; giving up\n",
-				 100 + delay - 1);
+			dev_warn(&dev->dev, "not ready %dms after %s; giving up\n",
+				 delay - 1, reset_type);
 			return -ENOTTY;
 		}
 
 		if (delay > 1000)
-			dev_info(&dev->dev, "not ready %dms after FLR; waiting\n",
-				 100 + delay - 1);
+			dev_info(&dev->dev, "not ready %dms after %s; waiting\n",
+				 delay - 1, reset_type);
 
 		msleep(delay);
 		delay *= 2;
@@ -3862,7 +3858,8 @@  static int pci_flr_wait(struct pci_dev *dev)
 	}
 
 	if (delay > 1000)
-		dev_info(&dev->dev, "ready %dms after FLR\n", 100 + delay - 1);
+		dev_info(&dev->dev, "ready %dms after %s\n", delay - 1,
+			 reset_type);
 
 	return 0;
 }
@@ -3899,7 +3896,15 @@  int pcie_flr(struct pci_dev *dev)
 		dev_err(&dev->dev, "timed out waiting for pending transaction; performing function level reset anyway\n");
 
 	pcie_capability_set_word(dev, PCI_EXP_DEVCTL, PCI_EXP_DEVCTL_BCR_FLR);
-	return pci_flr_wait(dev);
+
+	/*
+	 * Per PCIe r3.1, sec 6.6.2, a device must complete an FLR within
+	 * 100ms, but may silently discard requests while the FLR is in
+	 * progress.  Wait 100ms before trying to access the device.
+	 */
+	msleep(100);
+
+	return pci_dev_wait(dev, "FLR", PCIE_RESET_READY_POLL_MS);
 }
 EXPORT_SYMBOL_GPL(pcie_flr);
 
@@ -3932,7 +3937,15 @@  static int pci_af_flr(struct pci_dev *dev, int probe)
 		dev_err(&dev->dev, "timed out waiting for pending transaction; performing AF function level reset anyway\n");
 
 	pci_write_config_byte(dev, pos + PCI_AF_CTRL, PCI_AF_CTRL_FLR);
-	return pci_flr_wait(dev);
+
+	/*
+	 * Per PCIe r3.1, sec 6.6.2, a device must complete an FLR within
+	 * 100ms, but may silently discard requests while the FLR is in
+	 * progress.  Wait 100ms before trying to access the device.
+	 */
+	msleep(100);
+
+	return pci_dev_wait(dev, "AF_FLR", PCIE_RESET_READY_POLL_MS);
 }
 
 /**