Message ID | 1614061909-1734-1-git-send-email-atrajeev@linux.vnet.ibm.com (mailing list archive) |
---|---|
State | Rejected |
Headers | show |
Series | powerpc/perf: Fix handling of privilege level checks in perf interrupt context | expand |
Context | Check | Description |
---|---|---|
snowpatch_ozlabs/apply_patch | success | Successfully applied on branch powerpc/merge (626a6c3d2e20da80aaa710104f34ea6037b28b33) |
snowpatch_ozlabs/build-ppc64le | success | Build succeeded |
snowpatch_ozlabs/build-ppc64be | success | Build succeeded |
snowpatch_ozlabs/build-ppc64e | success | Build succeeded |
snowpatch_ozlabs/build-pmac32 | success | Build succeeded |
snowpatch_ozlabs/checkpatch | warning | total: 0 errors, 1 warnings, 0 checks, 40 lines checked |
snowpatch_ozlabs/needsstable | success | Patch has no Fixes tags |
(CC'ing LSM and SELinux lists; the initial message can be found here: https://lore.kernel.org/linuxppc-dev/1614061909-1734-1-git-send-email-atrajeev@linux.vnet.ibm.com/T/) On Tue, Feb 23, 2021 at 7:32 AM Athira Rajeev <atrajeev@linux.vnet.ibm.com> wrote: > > Running "perf mem record" in powerpc platforms with selinux enabled > resulted in soft lockup's. Below call-trace was seen in the logs: > > CPU: 58 PID: 3751 Comm: sssd_nss Not tainted 5.11.0-rc7+ #2 > NIP: c000000000dff3d4 LR: c000000000dff3d0 CTR: 0000000000000000 > REGS: c000007fffab7d60 TRAP: 0100 Not tainted (5.11.0-rc7+) > <<>> > NIP [c000000000dff3d4] _raw_spin_lock_irqsave+0x94/0x120 > LR [c000000000dff3d0] _raw_spin_lock_irqsave+0x90/0x120 > Call Trace: > [c00000000fd471a0] [c00000000fd47260] 0xc00000000fd47260 (unreliable) > [c00000000fd471e0] [c000000000b5fbbc] skb_queue_tail+0x3c/0x90 > [c00000000fd47220] [c000000000296edc] audit_log_end+0x6c/0x180 > [c00000000fd47260] [c0000000006a3f20] common_lsm_audit+0xb0/0xe0 > [c00000000fd472a0] [c00000000066c664] slow_avc_audit+0xa4/0x110 > [c00000000fd47320] [c00000000066cff4] avc_has_perm+0x1c4/0x260 > [c00000000fd47430] [c00000000066e064] selinux_perf_event_open+0x74/0xd0 > [c00000000fd47450] [c000000000669888] security_perf_event_open+0x68/0xc0 > [c00000000fd47490] [c00000000013d788] record_and_restart+0x6e8/0x7f0 > [c00000000fd476c0] [c00000000013dabc] perf_event_interrupt+0x22c/0x560 > [c00000000fd477d0] [c00000000002d0fc] performance_monitor_exception+0x4c/0x60 > [c00000000fd477f0] [c00000000000b378] performance_monitor_common_virt+0x1c8/0x1d0 > interrupt: f00 at _raw_spin_lock_irqsave+0x38/0x120 > NIP: c000000000dff378 LR: c000000000b5fbbc CTR: c0000000007d47f0 > REGS: c00000000fd47860 TRAP: 0f00 Not tainted (5.11.0-rc7+) > <<>> > NIP [c000000000dff378] _raw_spin_lock_irqsave+0x38/0x120 > LR [c000000000b5fbbc] skb_queue_tail+0x3c/0x90 > interrupt: f00 > [c00000000fd47b00] [0000000000000038] 0x38 (unreliable) > [c00000000fd47b40] [c00000000aae6200] 0xc00000000aae6200 > [c00000000fd47b80] [c000000000296edc] audit_log_end+0x6c/0x180 > [c00000000fd47bc0] [c00000000029f494] audit_log_exit+0x344/0xf80 > [c00000000fd47d10] [c0000000002a2b00] __audit_syscall_exit+0x2c0/0x320 > [c00000000fd47d60] [c000000000032878] do_syscall_trace_leave+0x148/0x200 > [c00000000fd47da0] [c00000000003d5b4] syscall_exit_prepare+0x324/0x390 > [c00000000fd47e10] [c00000000000d76c] system_call_common+0xfc/0x27c > > The above trace shows that while the CPU was handling a performance > monitor exception, there was a call to "security_perf_event_open" > function. In powerpc core-book3s, this function is called from > 'perf_allow_kernel' check during recording of data address in the sample > via perf_get_data_addr(). > > Commit da97e18458fb ("perf_event: Add support for LSM and SELinux checks") > introduced security enhancements to perf. As part of this commit, the new > security hook for perf_event_open was added in all places where perf > paranoid check was previously used. In powerpc core-book3s code, originally > had paranoid checks in 'perf_get_data_addr' and 'power_pmu_bhrb_read'. So > 'perf_paranoid_kernel' checks were replaced with 'perf_allow_kernel' in > these pmu helper functions as well. > > The intention of paranoid checks in core-book3s is to verify privilege > access before capturing some of the sample data. Along with paranoid > checks, 'perf_allow_kernel' also does a 'security_perf_event_open'. Since > these functions are accessed while recording sample, we end up in calling > selinux_perf_event_open in PMI context. Some of the security functions > use spinlock like sidtab_sid2str_put(). If a perf interrupt hits under > a spin lock and if we end up in calling selinux hook functions in PMI > handler, this could cause a dead lock. > > Since the purpose of this security hook is to control access to > perf_event_open, it is not right to call this in interrupt context. > But in case of powerpc PMU, we need the privilege checks for specific > samples from branch history ring buffer and sampling register values. > Reference commits: > Commit cd1231d7035f ("powerpc/perf: Prevent kernel address leak via > perf_get_data_addr()") > Commit bb19af816025 ("powerpc/perf: Prevent kernel address leak to > userspace via BHRB buffer") > > As a fix, patch caches 'perf_allow_kernel' value in event_init in > 'pmu_private' field of perf_event. The cached value is used in the > PMI code path. > > Suggested-by: Michael Ellerman <mpe@ellerman.id.au> > Signed-off-by: Athira Rajeev <atrajeev@linux.vnet.ibm.com> > --- > arch/powerpc/perf/core-book3s.c | 16 ++++++++++++++-- > 1 file changed, 14 insertions(+), 2 deletions(-) > > diff --git a/arch/powerpc/perf/core-book3s.c b/arch/powerpc/perf/core-book3s.c > index 4b4319d8..9e9f67f 100644 > --- a/arch/powerpc/perf/core-book3s.c > +++ b/arch/powerpc/perf/core-book3s.c > @@ -189,6 +189,11 @@ static inline unsigned long perf_ip_adjust(struct pt_regs *regs) > return 0; > } > > +static bool event_allow_kernel(struct perf_event *event) > +{ > + return (bool)event->pmu_private; > +} > + > /* > * The user wants a data address recorded. > * If we're not doing instruction sampling, give them the SDAR > @@ -222,7 +227,7 @@ static inline void perf_get_data_addr(struct perf_event *event, struct pt_regs * > if (!(mmcra & MMCRA_SAMPLE_ENABLE) || sdar_valid) > *addrp = mfspr(SPRN_SDAR); > > - if (is_kernel_addr(mfspr(SPRN_SDAR)) && perf_allow_kernel(&event->attr) != 0) > + if (is_kernel_addr(mfspr(SPRN_SDAR)) && !event_allow_kernel(event)) > *addrp = 0; > } > > @@ -507,7 +512,7 @@ static void power_pmu_bhrb_read(struct perf_event *event, struct cpu_hw_events * > * addresses, hence include a check before filtering code > */ > if (!(ppmu->flags & PPMU_ARCH_31) && > - is_kernel_addr(addr) && perf_allow_kernel(&event->attr) != 0) > + is_kernel_addr(addr) && !event_allow_kernel(event)) > continue; > > /* Branches are read most recent first (ie. mfbhrb 0 is > @@ -2049,6 +2054,13 @@ static int power_pmu_event_init(struct perf_event *event) > if (err) > return -EINVAL; > > + /* > + * We (ab)use pmu_private to cache the result of perf_allow_kernel(). We > + * need access to that result at interrupt time, but can't call > + * perf_allow_kernel() directly from interrupt context. > + */ > + event->pmu_private = (void *)(long)(perf_allow_kernel(&event->attr) == 0); I don't think you need this. Unless I'm missing something, you can simply use "event->attr.exclude_kernel" in place of "!event_allow_kernel(event)". If it is set, then there must have been a successful perf_allow_kernel() check in perf_event_open(2) before the event was created. power_pmu_event_init() would be called shortly after via perf_event_alloc() -> perf_init_event(), so I don't think this additional check would add much value. > + > event->hw.config = events[n]; > event->hw.event_base = cflags[n]; > event->hw.last_period = event->hw.sample_period; > -- > 1.8.3.1 >
On Tue, Feb 23, 2021 at 01:31:49AM -0500, Athira Rajeev wrote: > Running "perf mem record" in powerpc platforms with selinux enabled > resulted in soft lockup's. Below call-trace was seen in the logs: > > CPU: 58 PID: 3751 Comm: sssd_nss Not tainted 5.11.0-rc7+ #2 > NIP: c000000000dff3d4 LR: c000000000dff3d0 CTR: 0000000000000000 > REGS: c000007fffab7d60 TRAP: 0100 Not tainted (5.11.0-rc7+) > <<>> > NIP [c000000000dff3d4] _raw_spin_lock_irqsave+0x94/0x120 > LR [c000000000dff3d0] _raw_spin_lock_irqsave+0x90/0x120 > Call Trace: > [c00000000fd471a0] [c00000000fd47260] 0xc00000000fd47260 (unreliable) > [c00000000fd471e0] [c000000000b5fbbc] skb_queue_tail+0x3c/0x90 > [c00000000fd47220] [c000000000296edc] audit_log_end+0x6c/0x180 > [c00000000fd47260] [c0000000006a3f20] common_lsm_audit+0xb0/0xe0 > [c00000000fd472a0] [c00000000066c664] slow_avc_audit+0xa4/0x110 > [c00000000fd47320] [c00000000066cff4] avc_has_perm+0x1c4/0x260 > [c00000000fd47430] [c00000000066e064] selinux_perf_event_open+0x74/0xd0 > [c00000000fd47450] [c000000000669888] security_perf_event_open+0x68/0xc0 > [c00000000fd47490] [c00000000013d788] record_and_restart+0x6e8/0x7f0 > [c00000000fd476c0] [c00000000013dabc] perf_event_interrupt+0x22c/0x560 > [c00000000fd477d0] [c00000000002d0fc] performance_monitor_exception+0x4c/0x60 > [c00000000fd477f0] [c00000000000b378] performance_monitor_common_virt+0x1c8/0x1d0 > interrupt: f00 at _raw_spin_lock_irqsave+0x38/0x120 > NIP: c000000000dff378 LR: c000000000b5fbbc CTR: c0000000007d47f0 > REGS: c00000000fd47860 TRAP: 0f00 Not tainted (5.11.0-rc7+) > <<>> > NIP [c000000000dff378] _raw_spin_lock_irqsave+0x38/0x120 > LR [c000000000b5fbbc] skb_queue_tail+0x3c/0x90 > interrupt: f00 > [c00000000fd47b00] [0000000000000038] 0x38 (unreliable) > [c00000000fd47b40] [c00000000aae6200] 0xc00000000aae6200 > [c00000000fd47b80] [c000000000296edc] audit_log_end+0x6c/0x180 > [c00000000fd47bc0] [c00000000029f494] audit_log_exit+0x344/0xf80 > [c00000000fd47d10] [c0000000002a2b00] __audit_syscall_exit+0x2c0/0x320 > [c00000000fd47d60] [c000000000032878] do_syscall_trace_leave+0x148/0x200 > [c00000000fd47da0] [c00000000003d5b4] syscall_exit_prepare+0x324/0x390 > [c00000000fd47e10] [c00000000000d76c] system_call_common+0xfc/0x27c > > The above trace shows that while the CPU was handling a performance > monitor exception, there was a call to "security_perf_event_open" > function. In powerpc core-book3s, this function is called from > 'perf_allow_kernel' check during recording of data address in the sample > via perf_get_data_addr(). > > Commit da97e18458fb ("perf_event: Add support for LSM and SELinux checks") > introduced security enhancements to perf. As part of this commit, the new > security hook for perf_event_open was added in all places where perf > paranoid check was previously used. In powerpc core-book3s code, originally > had paranoid checks in 'perf_get_data_addr' and 'power_pmu_bhrb_read'. So > 'perf_paranoid_kernel' checks were replaced with 'perf_allow_kernel' in > these pmu helper functions as well. > > The intention of paranoid checks in core-book3s is to verify privilege > access before capturing some of the sample data. Along with paranoid > checks, 'perf_allow_kernel' also does a 'security_perf_event_open'. Since > these functions are accessed while recording sample, we end up in calling > selinux_perf_event_open in PMI context. Some of the security functions > use spinlock like sidtab_sid2str_put(). If a perf interrupt hits under > a spin lock and if we end up in calling selinux hook functions in PMI > handler, this could cause a dead lock. > > Since the purpose of this security hook is to control access to > perf_event_open, it is not right to call this in interrupt context. > But in case of powerpc PMU, we need the privilege checks for specific > samples from branch history ring buffer and sampling register values. I'm confused... why would you need those checks at event time? Either the event has perf_event_attr::exclude_kernel and it then isn't allowed to expose kernel addresses, or it doesn't and it is. There should never be an event-time question of permission like this. If you allow creation of an event, you're allowing the data it generates.
Peter Zijlstra <peterz@infradead.org> writes: > On Tue, Feb 23, 2021 at 01:31:49AM -0500, Athira Rajeev wrote: >> Running "perf mem record" in powerpc platforms with selinux enabled >> resulted in soft lockup's. Below call-trace was seen in the logs: ... >> >> Since the purpose of this security hook is to control access to >> perf_event_open, it is not right to call this in interrupt context. >> But in case of powerpc PMU, we need the privilege checks for specific >> samples from branch history ring buffer and sampling register values. > > I'm confused... why would you need those checks at event time? Either > the event has perf_event_attr::exclude_kernel and it then isn't allowed > to expose kernel addresses, or it doesn't and it is. Well one of us is confused that's for sure ^_^ I missed/forgot that we had that logic in open. I think the reason we got here is that in the past we didn't have the event in the low-level routines where we want to check, power_pmu_bhrb_read() and perf_get_data_addr(), so we hacked in a perf_paranoid_kernel() check. Which was wrong. Then Joel's patch plumbed the event through and switched those paranoid checks to perf_allow_kernel(). Anyway, we'll just switch those to exclude_kernel checks. > There should never be an event-time question of permission like this. If > you allow creation of an event, you're allowing the data it generates. Ack. cheers
> On 23-Feb-2021, at 6:24 PM, Michael Ellerman <mpe@ellerman.id.au> wrote: > > Peter Zijlstra <peterz@infradead.org> writes: >> On Tue, Feb 23, 2021 at 01:31:49AM -0500, Athira Rajeev wrote: >>> Running "perf mem record" in powerpc platforms with selinux enabled >>> resulted in soft lockup's. Below call-trace was seen in the logs: > ... >>> >>> Since the purpose of this security hook is to control access to >>> perf_event_open, it is not right to call this in interrupt context. >>> But in case of powerpc PMU, we need the privilege checks for specific >>> samples from branch history ring buffer and sampling register values. >> >> I'm confused... why would you need those checks at event time? Either >> the event has perf_event_attr::exclude_kernel and it then isn't allowed >> to expose kernel addresses, or it doesn't and it is. > > Well one of us is confused that's for sure ^_^ > > I missed/forgot that we had that logic in open. > > I think the reason we got here is that in the past we didn't have the > event in the low-level routines where we want to check, > power_pmu_bhrb_read() and perf_get_data_addr(), so we hacked in a > perf_paranoid_kernel() check. Which was wrong. > > Then Joel's patch plumbed the event through and switched those paranoid > checks to perf_allow_kernel(). > > Anyway, we'll just switch those to exclude_kernel checks. > >> There should never be an event-time question of permission like this. If >> you allow creation of an event, you're allowing the data it generates. > > Ack. Thanks for all the reviews. I will send a V2 with using 'event->attr.exclude_kernel' in the checks. Athira > > cheers
diff --git a/arch/powerpc/perf/core-book3s.c b/arch/powerpc/perf/core-book3s.c index 4b4319d8..9e9f67f 100644 --- a/arch/powerpc/perf/core-book3s.c +++ b/arch/powerpc/perf/core-book3s.c @@ -189,6 +189,11 @@ static inline unsigned long perf_ip_adjust(struct pt_regs *regs) return 0; } +static bool event_allow_kernel(struct perf_event *event) +{ + return (bool)event->pmu_private; +} + /* * The user wants a data address recorded. * If we're not doing instruction sampling, give them the SDAR @@ -222,7 +227,7 @@ static inline void perf_get_data_addr(struct perf_event *event, struct pt_regs * if (!(mmcra & MMCRA_SAMPLE_ENABLE) || sdar_valid) *addrp = mfspr(SPRN_SDAR); - if (is_kernel_addr(mfspr(SPRN_SDAR)) && perf_allow_kernel(&event->attr) != 0) + if (is_kernel_addr(mfspr(SPRN_SDAR)) && !event_allow_kernel(event)) *addrp = 0; } @@ -507,7 +512,7 @@ static void power_pmu_bhrb_read(struct perf_event *event, struct cpu_hw_events * * addresses, hence include a check before filtering code */ if (!(ppmu->flags & PPMU_ARCH_31) && - is_kernel_addr(addr) && perf_allow_kernel(&event->attr) != 0) + is_kernel_addr(addr) && !event_allow_kernel(event)) continue; /* Branches are read most recent first (ie. mfbhrb 0 is @@ -2049,6 +2054,13 @@ static int power_pmu_event_init(struct perf_event *event) if (err) return -EINVAL; + /* + * We (ab)use pmu_private to cache the result of perf_allow_kernel(). We + * need access to that result at interrupt time, but can't call + * perf_allow_kernel() directly from interrupt context. + */ + event->pmu_private = (void *)(long)(perf_allow_kernel(&event->attr) == 0); + event->hw.config = events[n]; event->hw.event_base = cflags[n]; event->hw.last_period = event->hw.sample_period;
Running "perf mem record" in powerpc platforms with selinux enabled resulted in soft lockup's. Below call-trace was seen in the logs: CPU: 58 PID: 3751 Comm: sssd_nss Not tainted 5.11.0-rc7+ #2 NIP: c000000000dff3d4 LR: c000000000dff3d0 CTR: 0000000000000000 REGS: c000007fffab7d60 TRAP: 0100 Not tainted (5.11.0-rc7+) <<>> NIP [c000000000dff3d4] _raw_spin_lock_irqsave+0x94/0x120 LR [c000000000dff3d0] _raw_spin_lock_irqsave+0x90/0x120 Call Trace: [c00000000fd471a0] [c00000000fd47260] 0xc00000000fd47260 (unreliable) [c00000000fd471e0] [c000000000b5fbbc] skb_queue_tail+0x3c/0x90 [c00000000fd47220] [c000000000296edc] audit_log_end+0x6c/0x180 [c00000000fd47260] [c0000000006a3f20] common_lsm_audit+0xb0/0xe0 [c00000000fd472a0] [c00000000066c664] slow_avc_audit+0xa4/0x110 [c00000000fd47320] [c00000000066cff4] avc_has_perm+0x1c4/0x260 [c00000000fd47430] [c00000000066e064] selinux_perf_event_open+0x74/0xd0 [c00000000fd47450] [c000000000669888] security_perf_event_open+0x68/0xc0 [c00000000fd47490] [c00000000013d788] record_and_restart+0x6e8/0x7f0 [c00000000fd476c0] [c00000000013dabc] perf_event_interrupt+0x22c/0x560 [c00000000fd477d0] [c00000000002d0fc] performance_monitor_exception+0x4c/0x60 [c00000000fd477f0] [c00000000000b378] performance_monitor_common_virt+0x1c8/0x1d0 interrupt: f00 at _raw_spin_lock_irqsave+0x38/0x120 NIP: c000000000dff378 LR: c000000000b5fbbc CTR: c0000000007d47f0 REGS: c00000000fd47860 TRAP: 0f00 Not tainted (5.11.0-rc7+) <<>> NIP [c000000000dff378] _raw_spin_lock_irqsave+0x38/0x120 LR [c000000000b5fbbc] skb_queue_tail+0x3c/0x90 interrupt: f00 [c00000000fd47b00] [0000000000000038] 0x38 (unreliable) [c00000000fd47b40] [c00000000aae6200] 0xc00000000aae6200 [c00000000fd47b80] [c000000000296edc] audit_log_end+0x6c/0x180 [c00000000fd47bc0] [c00000000029f494] audit_log_exit+0x344/0xf80 [c00000000fd47d10] [c0000000002a2b00] __audit_syscall_exit+0x2c0/0x320 [c00000000fd47d60] [c000000000032878] do_syscall_trace_leave+0x148/0x200 [c00000000fd47da0] [c00000000003d5b4] syscall_exit_prepare+0x324/0x390 [c00000000fd47e10] [c00000000000d76c] system_call_common+0xfc/0x27c The above trace shows that while the CPU was handling a performance monitor exception, there was a call to "security_perf_event_open" function. In powerpc core-book3s, this function is called from 'perf_allow_kernel' check during recording of data address in the sample via perf_get_data_addr(). Commit da97e18458fb ("perf_event: Add support for LSM and SELinux checks") introduced security enhancements to perf. As part of this commit, the new security hook for perf_event_open was added in all places where perf paranoid check was previously used. In powerpc core-book3s code, originally had paranoid checks in 'perf_get_data_addr' and 'power_pmu_bhrb_read'. So 'perf_paranoid_kernel' checks were replaced with 'perf_allow_kernel' in these pmu helper functions as well. The intention of paranoid checks in core-book3s is to verify privilege access before capturing some of the sample data. Along with paranoid checks, 'perf_allow_kernel' also does a 'security_perf_event_open'. Since these functions are accessed while recording sample, we end up in calling selinux_perf_event_open in PMI context. Some of the security functions use spinlock like sidtab_sid2str_put(). If a perf interrupt hits under a spin lock and if we end up in calling selinux hook functions in PMI handler, this could cause a dead lock. Since the purpose of this security hook is to control access to perf_event_open, it is not right to call this in interrupt context. But in case of powerpc PMU, we need the privilege checks for specific samples from branch history ring buffer and sampling register values. Reference commits: Commit cd1231d7035f ("powerpc/perf: Prevent kernel address leak via perf_get_data_addr()") Commit bb19af816025 ("powerpc/perf: Prevent kernel address leak to userspace via BHRB buffer") As a fix, patch caches 'perf_allow_kernel' value in event_init in 'pmu_private' field of perf_event. The cached value is used in the PMI code path. Suggested-by: Michael Ellerman <mpe@ellerman.id.au> Signed-off-by: Athira Rajeev <atrajeev@linux.vnet.ibm.com> --- arch/powerpc/perf/core-book3s.c | 16 ++++++++++++++-- 1 file changed, 14 insertions(+), 2 deletions(-)