| Message ID | 20181113082823.2440-17-aik@ozlabs.ru (mailing list archive) |
|---|---|
| State | Superseded |
| Headers | show |
| Series | powerpc/powernv/npu, vfio: NVIDIA V100 + P9 passthrough | expand |
| Context | Check | Description |
|---|---|---|
| snowpatch_ozlabs/apply_patch | warning | next/apply_patch Patch failed to apply |
| snowpatch_ozlabs/apply_patch | fail | Failed to apply to any branch |
On Tue, Nov 13, 2018 at 07:28:17PM +1100, Alexey Kardashevskiy wrote: > Flushing caches using the dcbf instruction takes quite some time if we > need to flush gigabytes (16GB takes more than 15s); OPAL just added > a big hammer to flush all caches. > > This adds opal_purge_cache() which will be used later to flush caches > for coherent GPU memory which might suddenly become unavailable if a GPU > is reset and NVLink is not (re)trained. > > Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru> Reviewed-by: David Gibson <david@gibson.dropbear.id.au> > --- > arch/powerpc/include/asm/opal-api.h | 3 ++- > arch/powerpc/include/asm/opal.h | 1 + > arch/powerpc/platforms/powernv/opal.c | 1 + > arch/powerpc/platforms/powernv/opal-wrappers.S | 1 + > 4 files changed, 5 insertions(+), 1 deletion(-) > > diff --git a/arch/powerpc/include/asm/opal-api.h b/arch/powerpc/include/asm/opal-api.h > index 870fb7b..55bc640 100644 > --- a/arch/powerpc/include/asm/opal-api.h > +++ b/arch/powerpc/include/asm/opal-api.h > @@ -210,7 +210,8 @@ > #define OPAL_PCI_GET_PBCQ_TUNNEL_BAR 164 > #define OPAL_PCI_SET_PBCQ_TUNNEL_BAR 165 > #define OPAL_NX_COPROC_INIT 167 > -#define OPAL_LAST 167 > +#define OPAL_CLEAR_CACHE 170 > +#define OPAL_LAST 170 > > #define QUIESCE_HOLD 1 /* Spin all calls at entry */ > #define QUIESCE_REJECT 2 /* Fail all calls with OPAL_BUSY */ > diff --git a/arch/powerpc/include/asm/opal.h b/arch/powerpc/include/asm/opal.h > index ff38664..7db576e 100644 > --- a/arch/powerpc/include/asm/opal.h > +++ b/arch/powerpc/include/asm/opal.h > @@ -294,6 +294,7 @@ int opal_set_power_shift_ratio(u32 handle, int token, u32 psr); > int opal_sensor_group_clear(u32 group_hndl, int token); > int opal_sensor_group_enable(u32 group_hndl, int token, bool enable); > int opal_nx_coproc_init(uint32_t chip_id, uint32_t ct); > +int opal_purge_cache(void); > > s64 opal_signal_system_reset(s32 cpu); > s64 opal_quiesce(u64 shutdown_type, s32 cpu); > diff --git a/arch/powerpc/platforms/powernv/opal.c b/arch/powerpc/platforms/powernv/opal.c > index beed86f..44ce824 100644 > --- a/arch/powerpc/platforms/powernv/opal.c > +++ b/arch/powerpc/platforms/powernv/opal.c > @@ -1113,3 +1113,4 @@ EXPORT_SYMBOL_GPL(opal_int_eoi); > EXPORT_SYMBOL_GPL(opal_error_code); > /* Export the below symbol for NX compression */ > EXPORT_SYMBOL(opal_nx_coproc_init); > +EXPORT_SYMBOL(opal_purge_cache); > diff --git a/arch/powerpc/platforms/powernv/opal-wrappers.S b/arch/powerpc/platforms/powernv/opal-wrappers.S > index 2515282..5b886a6 100644 > --- a/arch/powerpc/platforms/powernv/opal-wrappers.S > +++ b/arch/powerpc/platforms/powernv/opal-wrappers.S > @@ -331,3 +331,4 @@ OPAL_CALL(opal_pci_set_pbcq_tunnel_bar, OPAL_PCI_SET_PBCQ_TUNNEL_BAR); > OPAL_CALL(opal_sensor_read_u64, OPAL_SENSOR_READ_U64); > OPAL_CALL(opal_sensor_group_enable, OPAL_SENSOR_GROUP_ENABLE); > OPAL_CALL(opal_nx_coproc_init, OPAL_NX_COPROC_INIT); > +OPAL_CALL(opal_purge_cache, OPAL_CLEAR_CACHE);
diff --git a/arch/powerpc/include/asm/opal-api.h b/arch/powerpc/include/asm/opal-api.h index 870fb7b..55bc640 100644 --- a/arch/powerpc/include/asm/opal-api.h +++ b/arch/powerpc/include/asm/opal-api.h @@ -210,7 +210,8 @@ #define OPAL_PCI_GET_PBCQ_TUNNEL_BAR 164 #define OPAL_PCI_SET_PBCQ_TUNNEL_BAR 165 #define OPAL_NX_COPROC_INIT 167 -#define OPAL_LAST 167 +#define OPAL_CLEAR_CACHE 170 +#define OPAL_LAST 170 #define QUIESCE_HOLD 1 /* Spin all calls at entry */ #define QUIESCE_REJECT 2 /* Fail all calls with OPAL_BUSY */ diff --git a/arch/powerpc/include/asm/opal.h b/arch/powerpc/include/asm/opal.h index ff38664..7db576e 100644 --- a/arch/powerpc/include/asm/opal.h +++ b/arch/powerpc/include/asm/opal.h @@ -294,6 +294,7 @@ int opal_set_power_shift_ratio(u32 handle, int token, u32 psr); int opal_sensor_group_clear(u32 group_hndl, int token); int opal_sensor_group_enable(u32 group_hndl, int token, bool enable); int opal_nx_coproc_init(uint32_t chip_id, uint32_t ct); +int opal_purge_cache(void); s64 opal_signal_system_reset(s32 cpu); s64 opal_quiesce(u64 shutdown_type, s32 cpu); diff --git a/arch/powerpc/platforms/powernv/opal.c b/arch/powerpc/platforms/powernv/opal.c index beed86f..44ce824 100644 --- a/arch/powerpc/platforms/powernv/opal.c +++ b/arch/powerpc/platforms/powernv/opal.c @@ -1113,3 +1113,4 @@ EXPORT_SYMBOL_GPL(opal_int_eoi); EXPORT_SYMBOL_GPL(opal_error_code); /* Export the below symbol for NX compression */ EXPORT_SYMBOL(opal_nx_coproc_init); +EXPORT_SYMBOL(opal_purge_cache); diff --git a/arch/powerpc/platforms/powernv/opal-wrappers.S b/arch/powerpc/platforms/powernv/opal-wrappers.S index 2515282..5b886a6 100644 --- a/arch/powerpc/platforms/powernv/opal-wrappers.S +++ b/arch/powerpc/platforms/powernv/opal-wrappers.S @@ -331,3 +331,4 @@ OPAL_CALL(opal_pci_set_pbcq_tunnel_bar, OPAL_PCI_SET_PBCQ_TUNNEL_BAR); OPAL_CALL(opal_sensor_read_u64, OPAL_SENSOR_READ_U64); OPAL_CALL(opal_sensor_group_enable, OPAL_SENSOR_GROUP_ENABLE); OPAL_CALL(opal_nx_coproc_init, OPAL_NX_COPROC_INIT); +OPAL_CALL(opal_purge_cache, OPAL_CLEAR_CACHE);
Flushing caches using the dcbf instruction takes quite some time if we need to flush gigabytes (16GB takes more than 15s); OPAL just added a big hammer to flush all caches. This adds opal_purge_cache() which will be used later to flush caches for coherent GPU memory which might suddenly become unavailable if a GPU is reset and NVLink is not (re)trained. Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru> --- arch/powerpc/include/asm/opal-api.h | 3 ++- arch/powerpc/include/asm/opal.h | 1 + arch/powerpc/platforms/powernv/opal.c | 1 + arch/powerpc/platforms/powernv/opal-wrappers.S | 1 + 4 files changed, 5 insertions(+), 1 deletion(-)