| Message ID | 20220404101536.104794-1-ruscur@russell.cc (mailing list archive) |
|---|---|
| State | Accepted |
| Headers | show |
| Series | [v2,1/2] powerpc/powernv: Get L1D flush requirements from device-tree | expand |
On Mon, 4 Apr 2022 at 10:15, Russell Currey <ruscur@russell.cc> wrote: > > The device-tree properties no-need-l1d-flush-msr-pr-1-to-0 and > no-need-l1d-flush-kernel-on-user-access are the equivalents of > H_CPU_BEHAV_NO_L1D_FLUSH_ENTRY and H_CPU_BEHAV_NO_L1D_FLUSH_UACCESS > from the H_GET_CPU_CHARACTERISTICS hcall on pseries respectively. > > In commit d02fa40d759f ("powerpc/powernv: Remove POWER9 PVR version > check for entry and uaccess flushes") the condition for disabling the > L1D flush on kernel entry and user access was changed from any non-P9 > CPU to only checking P7 and P8. Without the appropriate device-tree > checks for newer processors on powernv, these flushes are unnecessarily > enabled on those systems. This patch corrects this. > > Fixes: d02fa40d759f ("powerpc/powernv: Remove POWER9 PVR version check for entry and uaccess flushes") > Reported-by: Joel Stanley <joel@jms.id.au> > Signed-off-by: Russell Currey <ruscur@russell.cc> I booted both patches in this series on a power10 powernv machine, applied on top of v5.18-rc1: $ dmesg |grep -i flush [ 0.000000] rfi-flush: fallback displacement flush available [ 0.000000] rfi-flush: patched 12 locations (no flush) [ 0.000000] count-cache-flush: flush disabled. [ 0.000000] link-stack-flush: flush disabled. $ grep . /sys/devices/system/cpu/vulnerabilities/* /sys/devices/system/cpu/vulnerabilities/itlb_multihit:Not affected /sys/devices/system/cpu/vulnerabilities/l1tf:Not affected /sys/devices/system/cpu/vulnerabilities/mds:Not affected /sys/devices/system/cpu/vulnerabilities/meltdown:Not affected /sys/devices/system/cpu/vulnerabilities/spec_store_bypass:Not affected /sys/devices/system/cpu/vulnerabilities/spectre_v1:Mitigation: __user pointer sanitization, ori31 speculation barrier enabled /sys/devices/system/cpu/vulnerabilities/spectre_v2:Mitigation: Software count cache flush (hardware accelerated), Software link stack flush /sys/devices/system/cpu/vulnerabilities/srbds:Not affected /sys/devices/system/cpu/vulnerabilities/tsx_async_abort:Not affected Does that match what we expect? Cheers, Joel > --- > arch/powerpc/platforms/powernv/setup.c | 6 ++++++ > 1 file changed, 6 insertions(+) > > diff --git a/arch/powerpc/platforms/powernv/setup.c b/arch/powerpc/platforms/powernv/setup.c > index 105d889abd51..378f7e5f18d2 100644 > --- a/arch/powerpc/platforms/powernv/setup.c > +++ b/arch/powerpc/platforms/powernv/setup.c > @@ -96,6 +96,12 @@ static void __init init_fw_feat_flags(struct device_node *np) > > if (fw_feature_is("disabled", "needs-spec-barrier-for-bound-checks", np)) > security_ftr_clear(SEC_FTR_BNDS_CHK_SPEC_BAR); > + > + if (fw_feature_is("enabled", "no-need-l1d-flush-msr-pr-1-to-0", np)) > + security_ftr_clear(SEC_FTR_L1D_FLUSH_ENTRY); > + > + if (fw_feature_is("enabled", "no-need-l1d-flush-kernel-on-user-access", np)) > + security_ftr_clear(SEC_FTR_L1D_FLUSH_UACCESS); > } > > static void __init pnv_setup_security_mitigations(void) > -- > 2.35.1 >
Joel Stanley <joel@jms.id.au> writes: > On Mon, 4 Apr 2022 at 10:15, Russell Currey <ruscur@russell.cc> wrote: >> >> The device-tree properties no-need-l1d-flush-msr-pr-1-to-0 and >> no-need-l1d-flush-kernel-on-user-access are the equivalents of >> H_CPU_BEHAV_NO_L1D_FLUSH_ENTRY and H_CPU_BEHAV_NO_L1D_FLUSH_UACCESS >> from the H_GET_CPU_CHARACTERISTICS hcall on pseries respectively. >> >> In commit d02fa40d759f ("powerpc/powernv: Remove POWER9 PVR version >> check for entry and uaccess flushes") the condition for disabling the >> L1D flush on kernel entry and user access was changed from any non-P9 >> CPU to only checking P7 and P8. Without the appropriate device-tree >> checks for newer processors on powernv, these flushes are unnecessarily >> enabled on those systems. This patch corrects this. >> >> Fixes: d02fa40d759f ("powerpc/powernv: Remove POWER9 PVR version check for entry and uaccess flushes") >> Reported-by: Joel Stanley <joel@jms.id.au> >> Signed-off-by: Russell Currey <ruscur@russell.cc> > > I booted both patches in this series on a power10 powernv machine, > applied on top of v5.18-rc1: > > $ dmesg |grep -i flush > [ 0.000000] rfi-flush: fallback displacement flush available > [ 0.000000] rfi-flush: patched 12 locations (no flush) > [ 0.000000] count-cache-flush: flush disabled. > [ 0.000000] link-stack-flush: flush disabled. > > $ grep . /sys/devices/system/cpu/vulnerabilities/* > /sys/devices/system/cpu/vulnerabilities/itlb_multihit:Not affected > /sys/devices/system/cpu/vulnerabilities/l1tf:Not affected > /sys/devices/system/cpu/vulnerabilities/mds:Not affected > /sys/devices/system/cpu/vulnerabilities/meltdown:Not affected > /sys/devices/system/cpu/vulnerabilities/spec_store_bypass:Not affected > /sys/devices/system/cpu/vulnerabilities/spectre_v1:Mitigation: __user > pointer sanitization, ori31 speculation barrier enabled > /sys/devices/system/cpu/vulnerabilities/spectre_v2:Mitigation: > Software count cache flush (hardware accelerated), Software link stack > flush > /sys/devices/system/cpu/vulnerabilities/srbds:Not affected > /sys/devices/system/cpu/vulnerabilities/tsx_async_abort:Not affected > > Does that match what we expect? AFAIK yes. Happy for ruscur to correct me though. Can you also try running the kernel selftests under tools/testing/selftests/powerpc/security/ ? I suspect some of them might fail, because they have specific knowledge of things and might need an update. cheers
On Tue, 5 Apr 2022 at 06:13, Michael Ellerman <mpe@ellerman.id.au> wrote: > > Joel Stanley <joel@jms.id.au> writes: > > On Mon, 4 Apr 2022 at 10:15, Russell Currey <ruscur@russell.cc> wrote: > >> > >> The device-tree properties no-need-l1d-flush-msr-pr-1-to-0 and > >> no-need-l1d-flush-kernel-on-user-access are the equivalents of > >> H_CPU_BEHAV_NO_L1D_FLUSH_ENTRY and H_CPU_BEHAV_NO_L1D_FLUSH_UACCESS > >> from the H_GET_CPU_CHARACTERISTICS hcall on pseries respectively. > >> > >> In commit d02fa40d759f ("powerpc/powernv: Remove POWER9 PVR version > >> check for entry and uaccess flushes") the condition for disabling the > >> L1D flush on kernel entry and user access was changed from any non-P9 > >> CPU to only checking P7 and P8. Without the appropriate device-tree > >> checks for newer processors on powernv, these flushes are unnecessarily > >> enabled on those systems. This patch corrects this. > >> > >> Fixes: d02fa40d759f ("powerpc/powernv: Remove POWER9 PVR version check for entry and uaccess flushes") > >> Reported-by: Joel Stanley <joel@jms.id.au> > >> Signed-off-by: Russell Currey <ruscur@russell.cc> > > > > I booted both patches in this series on a power10 powernv machine, > > applied on top of v5.18-rc1: > > > > $ dmesg |grep -i flush > > [ 0.000000] rfi-flush: fallback displacement flush available > > [ 0.000000] rfi-flush: patched 12 locations (no flush) > > [ 0.000000] count-cache-flush: flush disabled. > > [ 0.000000] link-stack-flush: flush disabled. > > > > $ grep . /sys/devices/system/cpu/vulnerabilities/* > > /sys/devices/system/cpu/vulnerabilities/itlb_multihit:Not affected > > /sys/devices/system/cpu/vulnerabilities/l1tf:Not affected > > /sys/devices/system/cpu/vulnerabilities/mds:Not affected > > /sys/devices/system/cpu/vulnerabilities/meltdown:Not affected > > /sys/devices/system/cpu/vulnerabilities/spec_store_bypass:Not affected > > /sys/devices/system/cpu/vulnerabilities/spectre_v1:Mitigation: __user > > pointer sanitization, ori31 speculation barrier enabled > > /sys/devices/system/cpu/vulnerabilities/spectre_v2:Mitigation: > > Software count cache flush (hardware accelerated), Software link stack > > flush > > /sys/devices/system/cpu/vulnerabilities/srbds:Not affected > > /sys/devices/system/cpu/vulnerabilities/tsx_async_abort:Not affected > > > > Does that match what we expect? > > AFAIK yes. Happy for ruscur to correct me though. > > Can you also try running the kernel selftests under > tools/testing/selftests/powerpc/security/ ? Here's the results when booted with no_spectrev2 (which I keep on doing by accident, as this machine has it in it's nvram): make[1]: Entering directory '/home/joel/dev/kernels/linus/tools/testing/selftests/powerpc/security' TAP version 13 1..5 # selftests: powerpc/security: rfi_flush # test: rfi_flush_test # tags: git_version:v5.18-rc1-0-g312310928417 # PASS (L1D misses with rfi_flush=0: 63101900 < 95000000) [10/10 pass] # PASS (L1D misses with rfi_flush=1: 196001003 > 190000000) [10/10 pass] # success: rfi_flush_test ok 1 selftests: powerpc/security: rfi_flush # selftests: powerpc/security: entry_flush # test: entry_flush_test # tags: git_version:v5.18-rc1-0-g312310928417 # PASS (L1D misses with entry_flush=0: 52766044 < 95000000) [10/10 pass] # PASS (L1D misses with entry_flush=1: 196082833 > 190000000) [10/10 pass] # success: entry_flush_test ok 2 selftests: powerpc/security: entry_flush # selftests: powerpc/security: uaccess_flush # test: uaccess_flush_test # tags: git_version:v5.18-rc1-0-g312310928417 # PASS (L1D misses with uaccess_flush=0: 68646267 < 95000000) [10/10 pass] # PASS (L1D misses with uaccess_flush=1: 194177355 > 190000000) [10/10 pass] # success: uaccess_flush_test ok 3 selftests: powerpc/security: uaccess_flush # selftests: powerpc/security: spectre_v2 # test: spectre_v2 # tags: git_version:v5.18-rc1-0-g312310928417 # sysfs reports: 'Vulnerable' # PM_BR_PRED_CCACHE: result 0 running/enabled 2090650862 # PM_BR_MPRED_CCACHE: result 1 running/enabled 2090648016 # Miss percent 0 % # OK - Measured branch prediction rates match reported spectre v2 mitigation. # success: spectre_v2 ok 4 selftests: powerpc/security: spectre_v2 # selftests: powerpc/security: mitigation-patching.sh # Spawned threads enabling/disabling mitigations ... # Waiting for timeout ... # OK ok 5 selftests: powerpc/security: mitigation-patching.sh Here's the same thing without the command line option set: make[1]: Entering directory '/home/joel/dev/kernels/linus/tools/testing/selftests/powerpc/security' TAP version 13 1..5 # selftests: powerpc/security: rfi_flush # test: rfi_flush_test # tags: git_version:v5.18-rc1-0-g312310928417 # PASS (L1D misses with rfi_flush=0: 47289780 < 95000000) [10/10 pass] # PASS (L1D misses with rfi_flush=1: 195001301 > 190000000) [10/10 pass] # success: rfi_flush_test ok 1 selftests: powerpc/security: rfi_flush # selftests: powerpc/security: entry_flush # test: entry_flush_test # tags: git_version:v5.18-rc1-0-g312310928417 # PASS (L1D misses with entry_flush=0: 63510274 < 95000000) [10/10 pass] # PASS (L1D misses with entry_flush=1: 195093006 > 190000000) [10/10 pass] # success: entry_flush_test ok 2 selftests: powerpc/security: entry_flush # selftests: powerpc/security: uaccess_flush # test: uaccess_flush_test # tags: git_version:v5.18-rc1-0-g312310928417 # PASS (L1D misses with uaccess_flush=0: 62184912 < 95000000) [10/10 pass] # PASS (L1D misses with uaccess_flush=1: 196018163 > 190000000) [10/10 pass] # success: uaccess_flush_test ok 3 selftests: powerpc/security: uaccess_flush # selftests: powerpc/security: spectre_v2 # test: spectre_v2 # tags: git_version:v5.18-rc1-0-g312310928417 # sysfs reports: 'Mitigation: Software count cache flush (hardware accelerated), Software link stack flush' # PM_BR_PRED_CCACHE: result 0 running/enabled 2016985490 # PM_BR_MPRED_CCACHE: result 1 running/enabled 2016981520 # Miss percent 0 % # OK - Measured branch prediction rates match reported spectre v2 mitigation. # success: spectre_v2 ok 4 selftests: powerpc/security: spectre_v2 # selftests: powerpc/security: mitigation-patching.sh # Spawned threads enabling/disabling mitigations ... # Waiting for timeout ... # OK ok 5 selftests: powerpc/security: mitigation-patching.sh > > I suspect some of them might fail, because they have specific knowledge > of things and might need an update. > > cheers
On Tue, 2022-04-05 at 02:49 +0000, Joel Stanley wrote: > I booted both patches in this series on a power10 powernv machine, > applied on top of v5.18-rc1: > > $ dmesg |grep -i flush > [ 0.000000] rfi-flush: fallback displacement flush available > [ 0.000000] rfi-flush: patched 12 locations (no flush) > [ 0.000000] count-cache-flush: flush disabled. > [ 0.000000] link-stack-flush: flush disabled. In this case you'd be looking for stf-barrier, uaccess-flush and entry- flush so this doesn't tell us anything. This must have been from a no_spectrev2 boot with count cache and link stack flushes disabled. > > $ grep . /sys/devices/system/cpu/vulnerabilities/* > /sys/devices/system/cpu/vulnerabilities/itlb_multihit:Not affected > /sys/devices/system/cpu/vulnerabilities/l1tf:Not affected > /sys/devices/system/cpu/vulnerabilities/mds:Not affected > /sys/devices/system/cpu/vulnerabilities/meltdown:Not affected > /sys/devices/system/cpu/vulnerabilities/spec_store_bypass:Not > affected > /sys/devices/system/cpu/vulnerabilities/spectre_v1:Mitigation: __user > pointer sanitization, ori31 speculation barrier enabled > /sys/devices/system/cpu/vulnerabilities/spectre_v2:Mitigation: > Software count cache flush (hardware accelerated), Software link > stack > flush > /sys/devices/system/cpu/vulnerabilities/srbds:Not affected > /sys/devices/system/cpu/vulnerabilities/tsx_async_abort:Not affected > > Does that match what we expect? This is as expected for P10, though clearly from a different boot to the above :) - Russell > > Cheers, > > Joel
Joel Stanley <joel@jms.id.au> writes: > On Tue, 5 Apr 2022 at 06:13, Michael Ellerman <mpe@ellerman.id.au> wrote: >> >> Joel Stanley <joel@jms.id.au> writes: >> > On Mon, 4 Apr 2022 at 10:15, Russell Currey <ruscur@russell.cc> wrote: >> >> >> >> The device-tree properties no-need-l1d-flush-msr-pr-1-to-0 and >> >> no-need-l1d-flush-kernel-on-user-access are the equivalents of >> >> H_CPU_BEHAV_NO_L1D_FLUSH_ENTRY and H_CPU_BEHAV_NO_L1D_FLUSH_UACCESS >> >> from the H_GET_CPU_CHARACTERISTICS hcall on pseries respectively. >> >> >> >> In commit d02fa40d759f ("powerpc/powernv: Remove POWER9 PVR version >> >> check for entry and uaccess flushes") the condition for disabling the >> >> L1D flush on kernel entry and user access was changed from any non-P9 >> >> CPU to only checking P7 and P8. Without the appropriate device-tree >> >> checks for newer processors on powernv, these flushes are unnecessarily >> >> enabled on those systems. This patch corrects this. >> >> >> >> Fixes: d02fa40d759f ("powerpc/powernv: Remove POWER9 PVR version check for entry and uaccess flushes") >> >> Reported-by: Joel Stanley <joel@jms.id.au> >> >> Signed-off-by: Russell Currey <ruscur@russell.cc> >> > >> > I booted both patches in this series on a power10 powernv machine, >> > applied on top of v5.18-rc1: >> > >> > $ dmesg |grep -i flush >> > [ 0.000000] rfi-flush: fallback displacement flush available >> > [ 0.000000] rfi-flush: patched 12 locations (no flush) >> > [ 0.000000] count-cache-flush: flush disabled. >> > [ 0.000000] link-stack-flush: flush disabled. >> > >> > $ grep . /sys/devices/system/cpu/vulnerabilities/* >> > /sys/devices/system/cpu/vulnerabilities/itlb_multihit:Not affected >> > /sys/devices/system/cpu/vulnerabilities/l1tf:Not affected >> > /sys/devices/system/cpu/vulnerabilities/mds:Not affected >> > /sys/devices/system/cpu/vulnerabilities/meltdown:Not affected >> > /sys/devices/system/cpu/vulnerabilities/spec_store_bypass:Not affected >> > /sys/devices/system/cpu/vulnerabilities/spectre_v1:Mitigation: __user >> > pointer sanitization, ori31 speculation barrier enabled >> > /sys/devices/system/cpu/vulnerabilities/spectre_v2:Mitigation: >> > Software count cache flush (hardware accelerated), Software link stack >> > flush >> > /sys/devices/system/cpu/vulnerabilities/srbds:Not affected >> > /sys/devices/system/cpu/vulnerabilities/tsx_async_abort:Not affected >> > >> > Does that match what we expect? >> >> AFAIK yes. Happy for ruscur to correct me though. >> >> Can you also try running the kernel selftests under >> tools/testing/selftests/powerpc/security/ ? > > Here's the results when booted with no_spectrev2 (which I keep on > doing by accident, as this machine has it in it's nvram): > > make[1]: Entering directory > '/home/joel/dev/kernels/linus/tools/testing/selftests/powerpc/security' > TAP version 13 > 1..5 > # selftests: powerpc/security: rfi_flush > # test: rfi_flush_test > # tags: git_version:v5.18-rc1-0-g312310928417 > # PASS (L1D misses with rfi_flush=0: 63101900 < 95000000) [10/10 pass] > # PASS (L1D misses with rfi_flush=1: 196001003 > 190000000) [10/10 pass] > # success: rfi_flush_test > ok 1 selftests: powerpc/security: rfi_flush > # selftests: powerpc/security: entry_flush > # test: entry_flush_test > # tags: git_version:v5.18-rc1-0-g312310928417 > # PASS (L1D misses with entry_flush=0: 52766044 < 95000000) [10/10 pass] > # PASS (L1D misses with entry_flush=1: 196082833 > 190000000) [10/10 pass] > # success: entry_flush_test > ok 2 selftests: powerpc/security: entry_flush > # selftests: powerpc/security: uaccess_flush > # test: uaccess_flush_test > # tags: git_version:v5.18-rc1-0-g312310928417 > # PASS (L1D misses with uaccess_flush=0: 68646267 < 95000000) [10/10 pass] > # PASS (L1D misses with uaccess_flush=1: 194177355 > 190000000) [10/10 pass] > # success: uaccess_flush_test > ok 3 selftests: powerpc/security: uaccess_flush > # selftests: powerpc/security: spectre_v2 > # test: spectre_v2 > # tags: git_version:v5.18-rc1-0-g312310928417 > # sysfs reports: 'Vulnerable' > # PM_BR_PRED_CCACHE: result 0 running/enabled 2090650862 > # PM_BR_MPRED_CCACHE: result 1 running/enabled 2090648016 > # Miss percent 0 % > # OK - Measured branch prediction rates match reported spectre v2 mitigation. > # success: spectre_v2 > ok 4 selftests: powerpc/security: spectre_v2 > # selftests: powerpc/security: mitigation-patching.sh > # Spawned threads enabling/disabling mitigations ... > # Waiting for timeout ... > # OK > ok 5 selftests: powerpc/security: mitigation-patching.sh > > > Here's the same thing without the command line option set: Thanks. > # test: spectre_v2 > # tags: git_version:v5.18-rc1-0-g312310928417 > # sysfs reports: 'Mitigation: Software count cache flush (hardware > accelerated), Software link stack flush' > # PM_BR_PRED_CCACHE: result 0 running/enabled 2016985490 > # PM_BR_MPRED_CCACHE: result 1 running/enabled 2016981520 > # Miss percent 0 % > # OK - Measured branch prediction rates match reported spectre v2 mitigation. > # success: spectre_v2 This passed, but looks wrong, it says there were zero branches correctly predicted. I think the PMU events were are using are wrong for P10, the test will need updating to use the right events for P10. cheers
On Mon, 4 Apr 2022 20:15:35 +1000, Russell Currey wrote: > The device-tree properties no-need-l1d-flush-msr-pr-1-to-0 and > no-need-l1d-flush-kernel-on-user-access are the equivalents of > H_CPU_BEHAV_NO_L1D_FLUSH_ENTRY and H_CPU_BEHAV_NO_L1D_FLUSH_UACCESS > from the H_GET_CPU_CHARACTERISTICS hcall on pseries respectively. > > In commit d02fa40d759f ("powerpc/powernv: Remove POWER9 PVR version > check for entry and uaccess flushes") the condition for disabling the > L1D flush on kernel entry and user access was changed from any non-P9 > CPU to only checking P7 and P8. Without the appropriate device-tree > checks for newer processors on powernv, these flushes are unnecessarily > enabled on those systems. This patch corrects this. > > [...] Applied to powerpc/next. [1/2] powerpc/powernv: Get L1D flush requirements from device-tree https://git.kernel.org/powerpc/c/2efee6adb56159288bce9d1ab51fc9056d7007d4 [2/2] powerpc/powernv: Get STF barrier requirements from device-tree https://git.kernel.org/powerpc/c/d2a3c131981d4498571908df95c3c9393a00adf5 cheers
diff --git a/arch/powerpc/platforms/powernv/setup.c b/arch/powerpc/platforms/powernv/setup.c index 105d889abd51..378f7e5f18d2 100644 --- a/arch/powerpc/platforms/powernv/setup.c +++ b/arch/powerpc/platforms/powernv/setup.c @@ -96,6 +96,12 @@ static void __init init_fw_feat_flags(struct device_node *np) if (fw_feature_is("disabled", "needs-spec-barrier-for-bound-checks", np)) security_ftr_clear(SEC_FTR_BNDS_CHK_SPEC_BAR); + + if (fw_feature_is("enabled", "no-need-l1d-flush-msr-pr-1-to-0", np)) + security_ftr_clear(SEC_FTR_L1D_FLUSH_ENTRY); + + if (fw_feature_is("enabled", "no-need-l1d-flush-kernel-on-user-access", np)) + security_ftr_clear(SEC_FTR_L1D_FLUSH_UACCESS); } static void __init pnv_setup_security_mitigations(void)
The device-tree properties no-need-l1d-flush-msr-pr-1-to-0 and no-need-l1d-flush-kernel-on-user-access are the equivalents of H_CPU_BEHAV_NO_L1D_FLUSH_ENTRY and H_CPU_BEHAV_NO_L1D_FLUSH_UACCESS from the H_GET_CPU_CHARACTERISTICS hcall on pseries respectively. In commit d02fa40d759f ("powerpc/powernv: Remove POWER9 PVR version check for entry and uaccess flushes") the condition for disabling the L1D flush on kernel entry and user access was changed from any non-P9 CPU to only checking P7 and P8. Without the appropriate device-tree checks for newer processors on powernv, these flushes are unnecessarily enabled on those systems. This patch corrects this. Fixes: d02fa40d759f ("powerpc/powernv: Remove POWER9 PVR version check for entry and uaccess flushes") Reported-by: Joel Stanley <joel@jms.id.au> Signed-off-by: Russell Currey <ruscur@russell.cc> --- arch/powerpc/platforms/powernv/setup.c | 6 ++++++ 1 file changed, 6 insertions(+)