diff mbox series

[1/1] vfio-pci/nvlink2: Allow fallback to ibm,mmio-atsd[0]

Message ID 426f75e09ac1a6879a6d51f592bf683c698b4bda.1580959044.git.sbobroff@linux.ibm.com
State New
Headers show
Series [1/1] vfio-pci/nvlink2: Allow fallback to ibm,mmio-atsd[0] | expand

Checks

Context Check Description
snowpatch_ozlabs/needsstable success Patch has no Fixes tags
snowpatch_ozlabs/checkpatch warning total: 0 errors, 0 warnings, 1 checks, 19 lines checked
snowpatch_ozlabs/build-pmac32 success Build succeeded
snowpatch_ozlabs/build-ppc64e success Build succeeded
snowpatch_ozlabs/build-ppc64be success Build succeeded
snowpatch_ozlabs/build-ppc64le success Build succeeded
snowpatch_ozlabs/apply_patch success Successfully applied on branch powerpc/merge (530a1cfd52af0aba1af4b1c9a7bc66a202a459b1)

Commit Message

Sam Bobroff Feb. 6, 2020, 3:17 a.m. UTC
Older versions of skiboot only provide a single value in the device
tree property "ibm,mmio-atsd", even when multiple Address Translation
Shoot Down (ATSD) registers are present. This prevents NVLink2 devices
(other than the first) from being used with vfio-pci because vfio-pci
expects to be able to assign a dedicated ATSD register to each NVLink2
device.

However, ATSD registers can be shared among devices. This change
allows vfio-pci to fall back to sharing the register at index 0 if
necessary.

Signed-off-by: Sam Bobroff <sbobroff@linux.ibm.com>
---
 drivers/vfio/pci/vfio_pci_nvlink2.c | 13 +++++++++++--
 1 file changed, 11 insertions(+), 2 deletions(-)

Comments

Alexey Kardashevskiy Feb. 6, 2020, 4:23 a.m. UTC | #1
On 06/02/2020 14:17, Sam Bobroff wrote:
> Older versions of skiboot only provide a single value in the device	
> tree property "ibm,mmio-atsd", even when multiple Address Translation
> Shoot Down (ATSD) registers are present. This prevents NVLink2 devices
> (other than the first) from being used with vfio-pci because vfio-pci
> expects to be able to assign a dedicated ATSD register to each NVLink2
> device.
> 
> However, ATSD registers can be shared among devices. This change
> allows vfio-pci to fall back to sharing the register at index 0 if
> necessary.
> 
> Signed-off-by: Sam Bobroff <sbobroff@linux.ibm.com>
> ---
>  drivers/vfio/pci/vfio_pci_nvlink2.c | 13 +++++++++++--
>  1 file changed, 11 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/vfio/pci/vfio_pci_nvlink2.c b/drivers/vfio/pci/vfio_pci_nvlink2.c
> index f2983f0f84be..851ba673882b 100644
> --- a/drivers/vfio/pci/vfio_pci_nvlink2.c
> +++ b/drivers/vfio/pci/vfio_pci_nvlink2.c
> @@ -420,8 +420,17 @@ int vfio_pci_ibm_npu2_init(struct vfio_pci_device *vdev)
>  
>  	if (of_property_read_u64_index(hose->dn, "ibm,mmio-atsd", nvlink_index,
>  			&mmio_atsd)) {
> -		dev_warn(&vdev->pdev->dev, "No available ATSD found\n");
> -		mmio_atsd = 0;
> +		dev_warn(&vdev->pdev->dev,
> +			 "No ibm,mmio-atsd[%d] found: trying ibm,mmio-atsd[0]\n",
> +			 nvlink_index);


We do not really need this warning (nvlink_index doesn't matter that
much, we can work out from the device tree what happened), warnings
below are enough (if you really want, you can print nvlink_index there).

Either way,

Reviewed-by: Alexey Kardashevskiy <aik@ozlabs.ru>




> +		if (of_property_read_u64_index(hose->dn, "ibm,mmio-atsd", 0,
> +				&mmio_atsd)) {
> +			dev_warn(&vdev->pdev->dev, "No available ATSD found\n");
> +			mmio_atsd = 0;
> +		} else {
> +			dev_warn(&vdev->pdev->dev,
> +				 "Using fallback ibm,mmio-atsd[0] for ATSD.\n");
> +		}
>  	}
>  
>  	if (of_property_read_u64(npu_node, "ibm,device-tgt-addr", &tgt)) {
>
Sam Bobroff Feb. 7, 2020, 2:39 a.m. UTC | #2
On Thu, Feb 06, 2020 at 03:23:03PM +1100, Alexey Kardashevskiy wrote:
> 
> 
> On 06/02/2020 14:17, Sam Bobroff wrote:
> > Older versions of skiboot only provide a single value in the device	
> > tree property "ibm,mmio-atsd", even when multiple Address Translation
> > Shoot Down (ATSD) registers are present. This prevents NVLink2 devices
> > (other than the first) from being used with vfio-pci because vfio-pci
> > expects to be able to assign a dedicated ATSD register to each NVLink2
> > device.
> > 
> > However, ATSD registers can be shared among devices. This change
> > allows vfio-pci to fall back to sharing the register at index 0 if
> > necessary.
> > 
> > Signed-off-by: Sam Bobroff <sbobroff@linux.ibm.com>
> > ---
> >  drivers/vfio/pci/vfio_pci_nvlink2.c | 13 +++++++++++--
> >  1 file changed, 11 insertions(+), 2 deletions(-)
> > 
> > diff --git a/drivers/vfio/pci/vfio_pci_nvlink2.c b/drivers/vfio/pci/vfio_pci_nvlink2.c
> > index f2983f0f84be..851ba673882b 100644
> > --- a/drivers/vfio/pci/vfio_pci_nvlink2.c
> > +++ b/drivers/vfio/pci/vfio_pci_nvlink2.c
> > @@ -420,8 +420,17 @@ int vfio_pci_ibm_npu2_init(struct vfio_pci_device *vdev)
> >  
> >  	if (of_property_read_u64_index(hose->dn, "ibm,mmio-atsd", nvlink_index,
> >  			&mmio_atsd)) {
> > -		dev_warn(&vdev->pdev->dev, "No available ATSD found\n");
> > -		mmio_atsd = 0;
> > +		dev_warn(&vdev->pdev->dev,
> > +			 "No ibm,mmio-atsd[%d] found: trying ibm,mmio-atsd[0]\n",
> > +			 nvlink_index);
> 
> 
> We do not really need this warning (nvlink_index doesn't matter that
> much, we can work out from the device tree what happened), warnings
> below are enough (if you really want, you can print nvlink_index there).
> 
> Either way,
> 
> Reviewed-by: Alexey Kardashevskiy <aik@ozlabs.ru>
> 

Thanks,

I'll change it if there's some reason to do another version but
otherwise leave it as is.

Sam.
> 
> 
> 
> > +		if (of_property_read_u64_index(hose->dn, "ibm,mmio-atsd", 0,
> > +				&mmio_atsd)) {
> > +			dev_warn(&vdev->pdev->dev, "No available ATSD found\n");
> > +			mmio_atsd = 0;
> > +		} else {
> > +			dev_warn(&vdev->pdev->dev,
> > +				 "Using fallback ibm,mmio-atsd[0] for ATSD.\n");
> > +		}
> >  	}
> >  
> >  	if (of_property_read_u64(npu_node, "ibm,device-tgt-addr", &tgt)) {
> > 
> 
> -- 
> Alexey
Sam Bobroff Feb. 7, 2020, 4:43 a.m. UTC | #3
On Fri, Feb 07, 2020 at 01:39:14PM +1100, Sam Bobroff wrote:
> On Thu, Feb 06, 2020 at 03:23:03PM +1100, Alexey Kardashevskiy wrote:
> > 
> > 
> > On 06/02/2020 14:17, Sam Bobroff wrote:
> > > Older versions of skiboot only provide a single value in the device	
> > > tree property "ibm,mmio-atsd", even when multiple Address Translation
> > > Shoot Down (ATSD) registers are present. This prevents NVLink2 devices
> > > (other than the first) from being used with vfio-pci because vfio-pci
> > > expects to be able to assign a dedicated ATSD register to each NVLink2
> > > device.
> > > 
> > > However, ATSD registers can be shared among devices. This change
> > > allows vfio-pci to fall back to sharing the register at index 0 if
> > > necessary.
> > > 
> > > Signed-off-by: Sam Bobroff <sbobroff@linux.ibm.com>
> > > ---
> > >  drivers/vfio/pci/vfio_pci_nvlink2.c | 13 +++++++++++--
> > >  1 file changed, 11 insertions(+), 2 deletions(-)
> > > 
> > > diff --git a/drivers/vfio/pci/vfio_pci_nvlink2.c b/drivers/vfio/pci/vfio_pci_nvlink2.c
> > > index f2983f0f84be..851ba673882b 100644
> > > --- a/drivers/vfio/pci/vfio_pci_nvlink2.c
> > > +++ b/drivers/vfio/pci/vfio_pci_nvlink2.c
> > > @@ -420,8 +420,17 @@ int vfio_pci_ibm_npu2_init(struct vfio_pci_device *vdev)
> > >  
> > >  	if (of_property_read_u64_index(hose->dn, "ibm,mmio-atsd", nvlink_index,
> > >  			&mmio_atsd)) {
> > > -		dev_warn(&vdev->pdev->dev, "No available ATSD found\n");
> > > -		mmio_atsd = 0;
> > > +		dev_warn(&vdev->pdev->dev,
> > > +			 "No ibm,mmio-atsd[%d] found: trying ibm,mmio-atsd[0]\n",
> > > +			 nvlink_index);
> > 
> > 
> > We do not really need this warning (nvlink_index doesn't matter that
> > much, we can work out from the device tree what happened), warnings
> > below are enough (if you really want, you can print nvlink_index there).
> > 
> > Either way,
> > 
> > Reviewed-by: Alexey Kardashevskiy <aik@ozlabs.ru>
> > 
> 
> Thanks,
> 
> I'll change it if there's some reason to do another version but
> otherwise leave it as is.
> 
> Sam.

Oh! I almost forgot: maybe this should carry a fixes tag?
Fixes: 7f92891778df ("vfio_pci: Add NVIDIA GV100GL [Tesla V100 SXM2] subdriver")

> > 
> > 
> > 
> > > +		if (of_property_read_u64_index(hose->dn, "ibm,mmio-atsd", 0,
> > > +				&mmio_atsd)) {
> > > +			dev_warn(&vdev->pdev->dev, "No available ATSD found\n");
> > > +			mmio_atsd = 0;
> > > +		} else {
> > > +			dev_warn(&vdev->pdev->dev,
> > > +				 "Using fallback ibm,mmio-atsd[0] for ATSD.\n");
> > > +		}
> > >  	}
> > >  
> > >  	if (of_property_read_u64(npu_node, "ibm,device-tgt-addr", &tgt)) {
> > > 
> > 
> > -- 
> > Alexey
diff mbox series

Patch

diff --git a/drivers/vfio/pci/vfio_pci_nvlink2.c b/drivers/vfio/pci/vfio_pci_nvlink2.c
index f2983f0f84be..851ba673882b 100644
--- a/drivers/vfio/pci/vfio_pci_nvlink2.c
+++ b/drivers/vfio/pci/vfio_pci_nvlink2.c
@@ -420,8 +420,17 @@  int vfio_pci_ibm_npu2_init(struct vfio_pci_device *vdev)
 
 	if (of_property_read_u64_index(hose->dn, "ibm,mmio-atsd", nvlink_index,
 			&mmio_atsd)) {
-		dev_warn(&vdev->pdev->dev, "No available ATSD found\n");
-		mmio_atsd = 0;
+		dev_warn(&vdev->pdev->dev,
+			 "No ibm,mmio-atsd[%d] found: trying ibm,mmio-atsd[0]\n",
+			 nvlink_index);
+		if (of_property_read_u64_index(hose->dn, "ibm,mmio-atsd", 0,
+				&mmio_atsd)) {
+			dev_warn(&vdev->pdev->dev, "No available ATSD found\n");
+			mmio_atsd = 0;
+		} else {
+			dev_warn(&vdev->pdev->dev,
+				 "Using fallback ibm,mmio-atsd[0] for ATSD.\n");
+		}
 	}
 
 	if (of_property_read_u64(npu_node, "ibm,device-tgt-addr", &tgt)) {