| Message ID | 1559923710-11923-2-git-send-email-arbab@linux.ibm.com |
|---|---|
| State | Accepted |
| Headers | show |
| Series | None | expand |
On 08/06/2019 02:08, Reza Arbab wrote: > After putting all a GPU's links in reset, do a cache purge in case we > have CPU cache lines belonging to the now-unaccessible GPU memory. > > Fixes: 68d11e4460ec ("npu2: Reset NVLinks when resetting a GPU") > Cc: skiboot-stable@lists.ozlabs.org > Signed-off-by: Reza Arbab <arbab@linux.ibm.com> > --- > hw/npu2.c | 6 ++++++ > 1 file changed, 6 insertions(+) > > diff --git a/hw/npu2.c b/hw/npu2.c > index 774592911110..3a2808d7133c 100644 > --- a/hw/npu2.c > +++ b/hw/npu2.c > @@ -561,6 +561,8 @@ static int64_t npu2_gpu_bridge_sec_bus_reset(void *dev, > > gpu = list_top(&pd->children, struct pci_device, link); > if (gpu && (*data & PCI_CFG_BRCTL_SECONDARY_RESET)) { > + int64_t rc; > + > dt_for_each_compatible(dt_root, np, "ibm,power9-npu-pciex") { > npphb = pci_get_phb(dt_prop_get_cell(np, > "ibm,opal-phbid", 1)); > @@ -574,6 +576,10 @@ static int64_t npu2_gpu_bridge_sec_bus_reset(void *dev, > npu2_dev_procedure_reset(ndev); > } > } > + > + rc = purge_l2_l3_caches(); > + if (rc) > + return rc; Ah, missed that :( Probably would make more sense to have this in npu2_dev_procedure_reset() as the other caller also purges caches. Either way Reviewed-by: Alexey Kardashevskiy <aik@ozlabs.ru> > } > > return OPAL_PARTIAL; >
On Tue, Jun 11, 2019 at 12:55:48PM +1000, Alexey Kardashevskiy wrote: >On 08/06/2019 02:08, Reza Arbab wrote: >> After putting all a GPU's links in reset, do a cache purge in case we >> have CPU cache lines belonging to the now-unaccessible GPU memory. >> >> Fixes: 68d11e4460ec ("npu2: Reset NVLinks when resetting a GPU") >> Cc: skiboot-stable@lists.ozlabs.org >> Signed-off-by: Reza Arbab <arbab@linux.ibm.com> >> --- >> hw/npu2.c | 6 ++++++ >> 1 file changed, 6 insertions(+) >> >> diff --git a/hw/npu2.c b/hw/npu2.c >> index 774592911110..3a2808d7133c 100644 >> --- a/hw/npu2.c >> +++ b/hw/npu2.c >> @@ -561,6 +561,8 @@ static int64_t npu2_gpu_bridge_sec_bus_reset(void *dev, >> >> gpu = list_top(&pd->children, struct pci_device, link); >> if (gpu && (*data & PCI_CFG_BRCTL_SECONDARY_RESET)) { >> + int64_t rc; >> + >> dt_for_each_compatible(dt_root, np, "ibm,power9-npu-pciex") { >> npphb = pci_get_phb(dt_prop_get_cell(np, >> "ibm,opal-phbid", 1)); >> @@ -574,6 +576,10 @@ static int64_t npu2_gpu_bridge_sec_bus_reset(void *dev, >> npu2_dev_procedure_reset(ndev); >> } >> } >> + >> + rc = purge_l2_l3_caches(); >> + if (rc) >> + return rc; > > >Ah, missed that :( Probably would make more sense to have this in >npu2_dev_procedure_reset() as the other caller also purges caches. That is tempting but we'd end up purging the cache a bunch of times, once for each link. Probably better to reset each link, and then just do one purge afterwards.
Reza Arbab <arbab@linux.ibm.com> writes: > After putting all a GPU's links in reset, do a cache purge in case we > have CPU cache lines belonging to the now-unaccessible GPU memory. > > Fixes: 68d11e4460ec ("npu2: Reset NVLinks when resetting a GPU") > Cc: skiboot-stable@lists.ozlabs.org > Signed-off-by: Reza Arbab <arbab@linux.ibm.com> Thanks, merged to master as of d4f2f77377dab27eaf792aa089090bcdd953a5cc
On 06/07/2019 09:38 PM, Reza Arbab wrote: > After putting all a GPU's links in reset, do a cache purge in case we > have CPU cache lines belonging to the now-unaccessible GPU memory. > > Fixes: 68d11e4460ec ("npu2: Reset NVLinks when resetting a GPU") > Cc: skiboot-stable@lists.ozlabs.org It will be part of v6.3.2. Thanks! -Vasant
diff --git a/hw/npu2.c b/hw/npu2.c index 774592911110..3a2808d7133c 100644 --- a/hw/npu2.c +++ b/hw/npu2.c @@ -561,6 +561,8 @@ static int64_t npu2_gpu_bridge_sec_bus_reset(void *dev, gpu = list_top(&pd->children, struct pci_device, link); if (gpu && (*data & PCI_CFG_BRCTL_SECONDARY_RESET)) { + int64_t rc; + dt_for_each_compatible(dt_root, np, "ibm,power9-npu-pciex") { npphb = pci_get_phb(dt_prop_get_cell(np, "ibm,opal-phbid", 1)); @@ -574,6 +576,10 @@ static int64_t npu2_gpu_bridge_sec_bus_reset(void *dev, npu2_dev_procedure_reset(ndev); } } + + rc = purge_l2_l3_caches(); + if (rc) + return rc; } return OPAL_PARTIAL;
After putting all a GPU's links in reset, do a cache purge in case we have CPU cache lines belonging to the now-unaccessible GPU memory. Fixes: 68d11e4460ec ("npu2: Reset NVLinks when resetting a GPU") Cc: skiboot-stable@lists.ozlabs.org Signed-off-by: Reza Arbab <arbab@linux.ibm.com> --- hw/npu2.c | 6 ++++++ 1 file changed, 6 insertions(+)