| Message ID | 10c86c909f870d90b3094f76b692b44ebe9caeac.1787976185.git.ritesh.list@gmail.com (mailing list archive) |
|---|---|
| State | Under Review |
| Headers | show |
| Series | powerpc: Don't drop _TIF_RESTOREALL on syscall restart | expand |
| Context | Check | Description |
|---|---|---|
| snowpatch_ozlabs/github-powerpc_ppctests | success | Successfully ran 10 jobs. |
| snowpatch_ozlabs/github-powerpc_selftests | success | Successfully ran 10 jobs. |
| snowpatch_ozlabs/github-powerpc_sparse | success | Successfully ran 4 jobs. |
| snowpatch_ozlabs/github-powerpc_kernel_qemu | success | Successfully ran 22 jobs. |
| snowpatch_ozlabs/github-powerpc_clang | success | Successfully ran 5 jobs. |
On 29/08/26 9:49 am, Ritesh Harjani (IBM) wrote: > So the syscall return sequence is as follows: > A syscall return to userspace is prepared and then a short asm sequence > that actually does the RFI. Note that this asm range is restartable i.e. > EE is still on, so an interrupt (e.g. decrementer or external interrupt) > can hit while SRR/GPRs are being loaded. This is defined via: > > RESTART_TABLE(.Lsyscall_rst_start, .Lsyscall_rst_end, syscall_restart) > > This restart table then sends us to syscall_restart rather than resuming > in the middle of the RFI. The same stub is also used if irq_happened > already has a pending bit (soft-masked irq that has not been replayed > yet (PowerPC special case of local_irq_disable())). > > Here is a bit of a flow of sequence of code to visualize: > syscall_exit_prepare > decide full-GPR restore (_TIF_RESTOREALL) for signal, > rt_sigreturn or syscall trace > save that in regs->exit_result and return it in r3 > | > v > .Lsyscall_rst_start .. _end EE still on > irq_happened set or interrupt in this range? > | no | yes > v v > cmpdi r3,0 syscall_exit_restart > restore all / zero replay irq, try exit again > volatiles; RFI must return flags in r3 > again for the same cmpdi > > Now r3 after prepare is the flags word, not the actual syscall return. A nested > interrupt clobbers it, so the restart stub reloads RESULT into r3 and the > C handler (syscall_exit_restart()) should put the flags back (because later asm > checks whether r3 returned from C has _TIF_RESTOREALL set or not): > cmpdi r3, 0 > bne .Lsyscall_restore_regs > > Note that syscall_exit_restart() already ORs any new _TIF_RESTOREALL into > exit_result, but then it only returns the new sample and not the full > regs->exit_result. > > That sample could be often 0 even when restore-all is still required: > > - rt_sigreturn / syscall trace set the bit in prepare's local > ret and in exit_result. They never set exit_flags, which is > what restart samples. > > - a signal does set exit_flags but restart clears it. A > second pass through the stub then returns 0 while > exit_result still has the bit. > > The asm as mentioned earlier then treats r3==0 as the fast path and > zeros r0/r4-r12. That means the userspace that needed the full register > set could SIGSEGVs, (which could happen often in ld64.so.2 like while > doing a parallel kernel build as reported by Venkat). > > So we should instead return the accumulated exit_result, like how we do > in interrupt_exit_user_restart(). Note that prior to this commit > 263e5159e00a ("powerpc: Fix exit_flags field placement in pt_regs for ptrace") > we were returning regs->exit_result from syscall_exit_restart(), but > this commit changed that behaviour. > > Fixes: 263e5159e00a ("powerpc: Fix exit_flags field placement in pt_regs for ptrace") > Reported-by: Venkat Rao Bagalkote <venkat88@linux.ibm.com> > Closes: https://lore.kernel.org/all/75419f88-eab9-444b-bf97-28a9765819ad@linux.ibm.com/ > Signed-off-by: Ritesh Harjani (IBM) <ritesh.list@gmail.com> > --- This patch fixes reported issue. Tested-by: Venkat Rao Bagalkote <venkat88@linux.ibm.com> Regards, Venkat. > Sorry about the long commit msg. It took sometime for me to fully understand > that complex path, so I thought I may as well document that properly. > > arch/powerpc/kernel/interrupt.c | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/arch/powerpc/kernel/interrupt.c b/arch/powerpc/kernel/interrupt.c > index 5b88bf72786c..55f9c0c9922a 100644 > --- a/arch/powerpc/kernel/interrupt.c > +++ b/arch/powerpc/kernel/interrupt.c > @@ -175,7 +175,7 @@ notrace unsigned long syscall_exit_restart(unsigned long r3, struct pt_regs *reg > current_thread_info()->exit_flags &= ~_TIF_RESTOREALL; > regs->exit_result |= ret; > > - return ret; > + return regs->exit_result; > } > #endif > > -- > 2.39.5 > >
On Sat, Aug 29, 2026 at 09:49:00AM +0530, Ritesh Harjani (IBM) wrote: > So the syscall return sequence is as follows: > A syscall return to userspace is prepared and then a short asm sequence > that actually does the RFI. Note that this asm range is restartable i.e. > EE is still on, so an interrupt (e.g. decrementer or external interrupt) > can hit while SRR/GPRs are being loaded. This is defined via: > > RESTART_TABLE(.Lsyscall_rst_start, .Lsyscall_rst_end, syscall_restart) > > This restart table then sends us to syscall_restart rather than resuming > in the middle of the RFI. The same stub is also used if irq_happened > already has a pending bit (soft-masked irq that has not been replayed > yet (PowerPC special case of local_irq_disable())). > > Here is a bit of a flow of sequence of code to visualize: > syscall_exit_prepare > decide full-GPR restore (_TIF_RESTOREALL) for signal, > rt_sigreturn or syscall trace > save that in regs->exit_result and return it in r3 > | > v > .Lsyscall_rst_start .. _end EE still on > irq_happened set or interrupt in this range? > | no | yes > v v > cmpdi r3,0 syscall_exit_restart > restore all / zero replay irq, try exit again > volatiles; RFI must return flags in r3 > again for the same cmpdi > > Now r3 after prepare is the flags word, not the actual syscall return. A nested > interrupt clobbers it, so the restart stub reloads RESULT into r3 and the > C handler (syscall_exit_restart()) should put the flags back (because later asm > checks whether r3 returned from C has _TIF_RESTOREALL set or not): > cmpdi r3, 0 > bne .Lsyscall_restore_regs > > Note that syscall_exit_restart() already ORs any new _TIF_RESTOREALL into > exit_result, but then it only returns the new sample and not the full > regs->exit_result. > > That sample could be often 0 even when restore-all is still required: > > - rt_sigreturn / syscall trace set the bit in prepare's local > ret and in exit_result. They never set exit_flags, which is > what restart samples. > > - a signal does set exit_flags but restart clears it. A > second pass through the stub then returns 0 while > exit_result still has the bit. > > The asm as mentioned earlier then treats r3==0 as the fast path and > zeros r0/r4-r12. That means the userspace that needed the full register > set could SIGSEGVs, (which could happen often in ld64.so.2 like while > doing a parallel kernel build as reported by Venkat). > > So we should instead return the accumulated exit_result, like how we do > in interrupt_exit_user_restart(). Note that prior to this commit > 263e5159e00a ("powerpc: Fix exit_flags field placement in pt_regs for ptrace") > we were returning regs->exit_result from syscall_exit_restart(), but > this commit changed that behaviour. > > Fixes: 263e5159e00a ("powerpc: Fix exit_flags field placement in pt_regs for ptrace") > Reported-by: Venkat Rao Bagalkote <venkat88@linux.ibm.com> > Closes: https://lore.kernel.org/all/75419f88-eab9-444b-bf97-28a9765819ad@linux.ibm.com/ > Signed-off-by: Ritesh Harjani (IBM) <ritesh.list@gmail.com> > --- > Sorry about the long commit msg. It took sometime for me to fully understand > that complex path, so I thought I may as well document that properly. > > arch/powerpc/kernel/interrupt.c | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/arch/powerpc/kernel/interrupt.c b/arch/powerpc/kernel/interrupt.c > index 5b88bf72786c..55f9c0c9922a 100644 > --- a/arch/powerpc/kernel/interrupt.c > +++ b/arch/powerpc/kernel/interrupt.c > @@ -175,7 +175,7 @@ notrace unsigned long syscall_exit_restart(unsigned long r3, struct pt_regs *reg > current_thread_info()->exit_flags &= ~_TIF_RESTOREALL; > regs->exit_result |= ret; > > - return ret; > + return regs->exit_result; > } > #endif > > -- > 2.39.5 > > LGTM Reviewed-by: Mukesh Kumar Chaurasiya (IBM) <mkchauras@gmail.com>
On 8/29/26 9:49 AM, Ritesh Harjani (IBM) wrote: > So the syscall return sequence is as follows: > A syscall return to userspace is prepared and then a short asm sequence > that actually does the RFI. Note that this asm range is restartable i.e. > EE is still on, so an interrupt (e.g. decrementer or external interrupt) > can hit while SRR/GPRs are being loaded. This is defined via: > > RESTART_TABLE(.Lsyscall_rst_start, .Lsyscall_rst_end, syscall_restart) > > This restart table then sends us to syscall_restart rather than resuming > in the middle of the RFI. The same stub is also used if irq_happened > already has a pending bit (soft-masked irq that has not been replayed > yet (PowerPC special case of local_irq_disable())). > > Here is a bit of a flow of sequence of code to visualize: > syscall_exit_prepare > decide full-GPR restore (_TIF_RESTOREALL) for signal, > rt_sigreturn or syscall trace > save that in regs->exit_result and return it in r3 > | > v > .Lsyscall_rst_start .. _end EE still on > irq_happened set or interrupt in this range? > | no | yes > v v > cmpdi r3,0 syscall_exit_restart > restore all / zero replay irq, try exit again > volatiles; RFI must return flags in r3 > again for the same cmpdi > > Now r3 after prepare is the flags word, not the actual syscall return. A nested > interrupt clobbers it, so the restart stub reloads RESULT into r3 and the > C handler (syscall_exit_restart()) should put the flags back (because later asm > checks whether r3 returned from C has _TIF_RESTOREALL set or not): > cmpdi r3, 0 > bne .Lsyscall_restore_regs > > Note that syscall_exit_restart() already ORs any new _TIF_RESTOREALL into > exit_result, but then it only returns the new sample and not the full > regs->exit_result. > > That sample could be often 0 even when restore-all is still required: > > - rt_sigreturn / syscall trace set the bit in prepare's local > ret and in exit_result. They never set exit_flags, which is > what restart samples. > > - a signal does set exit_flags but restart clears it. A > second pass through the stub then returns 0 while > exit_result still has the bit. > > The asm as mentioned earlier then treats r3==0 as the fast path and > zeros r0/r4-r12. That means the userspace that needed the full register > set could SIGSEGVs, (which could happen often in ld64.so.2 like while > doing a parallel kernel build as reported by Venkat). > > So we should instead return the accumulated exit_result, like how we do > in interrupt_exit_user_restart(). Note that prior to this commit > 263e5159e00a ("powerpc: Fix exit_flags field placement in pt_regs for ptrace") > we were returning regs->exit_result from syscall_exit_restart(), but > this commit changed that behaviour. > Thanks for the fix. segfaults in ld64.so.2 are no longer seen with kernel compilation. Tested-by: Shrikanth Hegde <sshegde@linux.ibm.com> Reviewed-by: Shrikanth Hegde <sshegde@linux.ibm.com> > Fixes: 263e5159e00a ("powerpc: Fix exit_flags field placement in pt_regs for ptrace") > Reported-by: Venkat Rao Bagalkote <venkat88@linux.ibm.com> > Closes: https://lore.kernel.org/all/75419f88-eab9-444b-bf97-28a9765819ad@linux.ibm.com/ > Signed-off-by: Ritesh Harjani (IBM) <ritesh.list@gmail.com> > --- > Sorry about the long commit msg. It took sometime for me to fully understand > that complex path, so I thought I may as well document that properly. > > arch/powerpc/kernel/interrupt.c | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/arch/powerpc/kernel/interrupt.c b/arch/powerpc/kernel/interrupt.c > index 5b88bf72786c..55f9c0c9922a 100644 > --- a/arch/powerpc/kernel/interrupt.c > +++ b/arch/powerpc/kernel/interrupt.c > @@ -175,7 +175,7 @@ notrace unsigned long syscall_exit_restart(unsigned long r3, struct pt_regs *reg > current_thread_info()->exit_flags &= ~_TIF_RESTOREALL; > regs->exit_result |= ret; > > - return ret; > + return regs->exit_result; > } > #endif > > -- > 2.39.5 >
On 2026/08/29 09:49 AM, Ritesh Harjani (IBM) wrote: > So the syscall return sequence is as follows: > A syscall return to userspace is prepared and then a short asm sequence > that actually does the RFI. Note that this asm range is restartable i.e. > EE is still on, so an interrupt (e.g. decrementer or external interrupt) > can hit while SRR/GPRs are being loaded. This is defined via: > > RESTART_TABLE(.Lsyscall_rst_start, .Lsyscall_rst_end, syscall_restart) > > This restart table then sends us to syscall_restart rather than resuming > in the middle of the RFI. The same stub is also used if irq_happened > already has a pending bit (soft-masked irq that has not been replayed > yet (PowerPC special case of local_irq_disable())). > > Here is a bit of a flow of sequence of code to visualize: > syscall_exit_prepare > decide full-GPR restore (_TIF_RESTOREALL) for signal, > rt_sigreturn or syscall trace > save that in regs->exit_result and return it in r3 > | > v > .Lsyscall_rst_start .. _end EE still on > irq_happened set or interrupt in this range? > | no | yes > v v > cmpdi r3,0 syscall_exit_restart > restore all / zero replay irq, try exit again > volatiles; RFI must return flags in r3 > again for the same cmpdi > > Now r3 after prepare is the flags word, not the actual syscall return. A nested > interrupt clobbers it, so the restart stub reloads RESULT into r3 and the > C handler (syscall_exit_restart()) should put the flags back (because later asm > checks whether r3 returned from C has _TIF_RESTOREALL set or not): > cmpdi r3, 0 > bne .Lsyscall_restore_regs > > Note that syscall_exit_restart() already ORs any new _TIF_RESTOREALL into > exit_result, but then it only returns the new sample and not the full > regs->exit_result. > > That sample could be often 0 even when restore-all is still required: > > - rt_sigreturn / syscall trace set the bit in prepare's local > ret and in exit_result. They never set exit_flags, which is > what restart samples. > > - a signal does set exit_flags but restart clears it. A > second pass through the stub then returns 0 while > exit_result still has the bit. > > The asm as mentioned earlier then treats r3==0 as the fast path and > zeros r0/r4-r12. That means the userspace that needed the full register > set could SIGSEGVs, (which could happen often in ld64.so.2 like while > doing a parallel kernel build as reported by Venkat). > > So we should instead return the accumulated exit_result, like how we do > in interrupt_exit_user_restart(). Note that prior to this commit > 263e5159e00a ("powerpc: Fix exit_flags field placement in pt_regs for ptrace") > we were returning regs->exit_result from syscall_exit_restart(), but > this commit changed that behaviour. Thanks for the detailed commit message — it really helps understand this subtle path. > > Fixes: 263e5159e00a ("powerpc: Fix exit_flags field placement in pt_regs for ptrace") > Reported-by: Venkat Rao Bagalkote <venkat88@linux.ibm.com> > Closes: https://lore.kernel.org/all/75419f88-eab9-444b-bf97-28a9765819ad@linux.ibm.com/ > Signed-off-by: Ritesh Harjani (IBM) <ritesh.list@gmail.com> > --- > Sorry about the long commit msg. It took sometime for me to fully understand > that complex path, so I thought I may as well document that properly. > > arch/powerpc/kernel/interrupt.c | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/arch/powerpc/kernel/interrupt.c b/arch/powerpc/kernel/interrupt.c > index 5b88bf72786c..55f9c0c9922a 100644 > --- a/arch/powerpc/kernel/interrupt.c > +++ b/arch/powerpc/kernel/interrupt.c > @@ -175,7 +175,7 @@ notrace unsigned long syscall_exit_restart(unsigned long r3, struct pt_regs *reg > current_thread_info()->exit_flags &= ~_TIF_RESTOREALL; > regs->exit_result |= ret; > > - return ret; > + return regs->exit_result; This change fixes the userspace crashes which I observed while compiling kernel on an LPAR booted with Linux 7.3-rc1. Also, the change look good to me. Hence, Tested-by: Amit Machhiwal <amachhiw@linux.ibm.com> Reviewed-by: Amit Machhiwal <amachhiw@linux.ibm.com> Thanks, Amit
On Sat, 29 Aug, 2026, 9:49 am Ritesh Harjani (IBM), <ritesh.list@gmail.com> wrote: > So the syscall return sequence is as follows: > A syscall return to userspace is prepared and then a short asm sequence > that actually does the RFI. Note that this asm range is restartable i.e. > EE is still on, so an interrupt (e.g. decrementer or external interrupt) > can hit while SRR/GPRs are being loaded. This is defined via: > > RESTART_TABLE(.Lsyscall_rst_start, .Lsyscall_rst_end, syscall_restart) > > This restart table then sends us to syscall_restart rather than resuming > in the middle of the RFI. The same stub is also used if irq_happened > already has a pending bit (soft-masked irq that has not been replayed > yet (PowerPC special case of local_irq_disable())). > > Here is a bit of a flow of sequence of code to visualize: > syscall_exit_prepare > decide full-GPR restore (_TIF_RESTOREALL) for signal, > rt_sigreturn or syscall trace > save that in regs->exit_result and return it in r3 > | > v > .Lsyscall_rst_start .. _end EE still on > irq_happened set or interrupt in this range? > | no | yes > v v > cmpdi r3,0 syscall_exit_restart > restore all / zero replay irq, try exit again > volatiles; RFI must return flags in r3 > again for the same cmpdi > > Now r3 after prepare is the flags word, not the actual syscall return. A > nested > interrupt clobbers it, so the restart stub reloads RESULT into r3 and the > C handler (syscall_exit_restart()) should put the flags back (because > later asm > checks whether r3 returned from C has _TIF_RESTOREALL set or not): > cmpdi r3, 0 > bne .Lsyscall_restore_regs > > Note that syscall_exit_restart() already ORs any new _TIF_RESTOREALL into > exit_result, but then it only returns the new sample and not the full > regs->exit_result. > > That sample could be often 0 even when restore-all is still required: > > - rt_sigreturn / syscall trace set the bit in prepare's local > ret and in exit_result. They never set exit_flags, which is > what restart samples. > > - a signal does set exit_flags but restart clears it. A > second pass through the stub then returns 0 while > exit_result still has the bit. > > The asm as mentioned earlier then treats r3==0 as the fast path and > zeros r0/r4-r12. That means the userspace that needed the full register > set could SIGSEGVs, (which could happen often in ld64.so.2 like while > doing a parallel kernel build as reported by Venkat). > > So we should instead return the accumulated exit_result, like how we do > in interrupt_exit_user_restart(). Note that prior to this commit > 263e5159e00a ("powerpc: Fix exit_flags field placement in pt_regs for > ptrace") > we were returning regs->exit_result from syscall_exit_restart(), but > this commit changed that behaviour. > > Fixes: 263e5159e00a ("powerpc: Fix exit_flags field placement in pt_regs > for ptrace") > Reported-by: Venkat Rao Bagalkote <venkat88@linux.ibm.com> > Closes: > https://lore.kernel.org/all/75419f88-eab9-444b-bf97-28a9765819ad@linux.ibm.com/ > Signed-off-by: Ritesh Harjani (IBM) <ritesh.list@gmail.com> > --- > Sorry about the long commit msg. It took sometime for me to fully > understand > that complex path, so I thought I may as well document that properly. > > arch/powerpc/kernel/interrupt.c | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/arch/powerpc/kernel/interrupt.c > b/arch/powerpc/kernel/interrupt.c > index 5b88bf72786c..55f9c0c9922a 100644 > --- a/arch/powerpc/kernel/interrupt.c > +++ b/arch/powerpc/kernel/interrupt.c > @@ -175,7 +175,7 @@ notrace unsigned long syscall_exit_restart(unsigned > long r3, struct pt_regs *reg > current_thread_info()->exit_flags &= ~_TIF_RESTOREALL; > regs->exit_result |= ret; > > - return ret; > + return regs->exit_result; > Thanks for the fix, Ritesh! Are we leaving ret unused as well? Otherwise, Reviewed-by: Harsh Prateek Bora <harsh.prateek.bora@gmail.com> } > #endif > > -- > 2.39.5 > > >
Harsh Prateek Bora <harsh.prateek.bora@gmail.com> writes: > > Thanks for the fix, Ritesh! > Are we leaving ret unused as well? > No, it's not unused. It is getting used to sample whether exit_flags has _TIF_RESTOREALL set or not. Though maybe it can be optimized away if needed in a separate patch later. But for now we are good and this fix is needed as many of us have reported ld64 crash on v7.2. > Otherwise, > Reviewed-by: Harsh Prateek Bora <harsh.prateek.bora@gmail.com> > Thanks! -ritesh
On Sat, 2026-08-29 at 09:49 +0530, Ritesh Harjani (IBM) wrote: > So the syscall return sequence is as follows: > A syscall return to userspace is prepared and then a short asm sequence > that actually does the RFI. Note that this asm range is restartable i.e. > EE is still on, so an interrupt (e.g. decrementer or external interrupt) > can hit while SRR/GPRs are being loaded. This is defined via: > > RESTART_TABLE(.Lsyscall_rst_start, .Lsyscall_rst_end, syscall_restart) > > This restart table then sends us to syscall_restart rather than resuming > in the middle of the RFI. The same stub is also used if irq_happened > already has a pending bit (soft-masked irq that has not been replayed > yet (PowerPC special case of local_irq_disable())). > > Here is a bit of a flow of sequence of code to visualize: > syscall_exit_prepare > decide full-GPR restore (_TIF_RESTOREALL) for signal, > rt_sigreturn or syscall trace > save that in regs->exit_result and return it in r3 > | > v > .Lsyscall_rst_start .. _end EE still on > irq_happened set or interrupt in this range? > | no | yes > v v > cmpdi r3,0 syscall_exit_restart > restore all / zero replay irq, try exit again > volatiles; RFI must return flags in r3 > again for the same cmpdi > > Now r3 after prepare is the flags word, not the actual syscall return. A nested > interrupt clobbers it, so the restart stub reloads RESULT into r3 and the > C handler (syscall_exit_restart()) should put the flags back (because later asm > checks whether r3 returned from C has _TIF_RESTOREALL set or not): > cmpdi r3, 0 > bne .Lsyscall_restore_regs > > Note that syscall_exit_restart() already ORs any new _TIF_RESTOREALL into > exit_result, but then it only returns the new sample and not the full > regs->exit_result. > > That sample could be often 0 even when restore-all is still required: > > - rt_sigreturn / syscall trace set the bit in prepare's local > ret and in exit_result. They never set exit_flags, which is > what restart samples. > > - a signal does set exit_flags but restart clears it. A > second pass through the stub then returns 0 while > exit_result still has the bit. > > The asm as mentioned earlier then treats r3==0 as the fast path and > zeros r0/r4-r12. That means the userspace that needed the full register > set could SIGSEGVs, (which could happen often in ld64.so.2 like while > doing a parallel kernel build as reported by Venkat). > > So we should instead return the accumulated exit_result, like how we do > in interrupt_exit_user_restart(). Note that prior to this commit > 263e5159e00a ("powerpc: Fix exit_flags field placement in pt_regs for ptrace") > we were returning regs->exit_result from syscall_exit_restart(), but > this commit changed that behaviour. > > Fixes: 263e5159e00a ("powerpc: Fix exit_flags field placement in pt_regs for ptrace") > Reported-by: Venkat Rao Bagalkote <venkat88@linux.ibm.com> > Closes: https://lore.kernel.org/all/75419f88-eab9-444b-bf97-28a9765819ad@linux.ibm.com/ > Signed-off-by: Ritesh Harjani (IBM) <ritesh.list@gmail.com> > --- > Sorry about the long commit msg. It took sometime for me to fully understand > that complex path, so I thought I may as well document that properly. > > arch/powerpc/kernel/interrupt.c | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/arch/powerpc/kernel/interrupt.c b/arch/powerpc/kernel/interrupt.c > index 5b88bf72786c..55f9c0c9922a 100644 > --- a/arch/powerpc/kernel/interrupt.c > +++ b/arch/powerpc/kernel/interrupt.c > @@ -175,7 +175,7 @@ notrace unsigned long syscall_exit_restart(unsigned long r3, struct pt_regs *reg > current_thread_info()->exit_flags &= ~_TIF_RESTOREALL; > regs->exit_result |= ret; > > - return ret; > + return regs->exit_result; > } > #endif > > -- > 2.39.5 > Hi Ritesh, Thanks for the fix, I observed several processes consistently segfaulting in ld64.so.2 especially during parallel kernel builds. I have hit this issue multiple times while running 7.3-rc1: [13619.338946] [ T75988] grep[75988]: segfault (11) at 2a720 nip 7fffa1276694 lr 7fffa12757f8 code 1 in ld64.so.2[36694,7fffa1240000+50000] [13619.339095] [ T75988] grep[75988]: code: 7ce903a6 60000000 60420000 f9490008 f9490010 39290020 f949fff8 f9490000 [13619.339120] [ T75988] grep[75988]: code: 4200ffec 89210376 3d42ffff 394a7ea0 <e9028000> 7d081050 f9410030 f9010020 [13620.062652] [ T76441] sh[76441]: segfault (11) at 2a720 nip 7fff9c026694 lr 7fff9c0257f8 code 1 in ld64.so.2[36694,7fff9bff0000+50000] [13620.062807] [ T76441] sh[76441]: code: 7ce903a6 60000000 60420000 f9490008 f9490010 39290020 f949fff8 f9490000 [13620.062923] [ T76441] sh[76441]: code: 4200ffec 89210376 3d42ffff 394a7ea0 <e9028000> 7d081050 f9410030 f9010020 [13622.951229] [ T78978] as[78978]: segfault (11) at 2a720 nip 7fff99806694 lr 7fff998057f8 code 1 in ld64.so.2[36694,7fff997d0000+50000] [13622.951378] [ T78978] as[78978]: code: 7ce903a6 60000000 60420000 f9490008 f9490010 39290020 f949fff8 f9490000 [13622.951403] [ T78978] as[78978]: code: 4200ffec 89210376 3d42ffff 394a7ea0 <e9028000> 7d081050 f9410030 f9010020 [13624.287972] [ T80382] sh[80382]: segfault (11) at 2a720 nip 7fff7ed76694 lr 7fff7ed757f8 code 1 in ld64.so.2[36694,7fff7ed40000+50000] [13624.288112] [ T80382] sh[80382]: code: 7ce903a6 60000000 60420000 f9490008 f9490010 39290020 f949fff8 f9490000 [13624.288137] [ T80382] sh[80382]: code: 4200ffec 89210376 3d42ffff 394a7ea0 <e9028000> 7d081050 f9410030 f9010020 [13631.123076] [ T87355] rm[87355]: segfault (11) at 2a720 nip 7fffa9476694 lr 7fffa94757f8 code 1 in ld64.so.2[36694,7fffa9440000+50000] [13631.123222] [ T87355] rm[87355]: code: 7ce903a6 60000000 60420000 f9490008 f9490010 39290020 f949fff8 f9490000 [13631.123249] [ T87355] rm[87355]: code: 4200ffec 89210376 3d42ffff 394a7ea0 <e9028000> 7d081050 f9410030 f9010020 With this patch, I have not observed any segfaults in ld64.so.2. This patch fixes the issue for me. Tested-by: Aboorva Devarajan <aboorvad@linux.ibm.com>
diff --git a/arch/powerpc/kernel/interrupt.c b/arch/powerpc/kernel/interrupt.c index 5b88bf72786c..55f9c0c9922a 100644 --- a/arch/powerpc/kernel/interrupt.c +++ b/arch/powerpc/kernel/interrupt.c @@ -175,7 +175,7 @@ notrace unsigned long syscall_exit_restart(unsigned long r3, struct pt_regs *reg current_thread_info()->exit_flags &= ~_TIF_RESTOREALL; regs->exit_result |= ret; - return ret; + return regs->exit_result; } #endif
So the syscall return sequence is as follows: A syscall return to userspace is prepared and then a short asm sequence that actually does the RFI. Note that this asm range is restartable i.e. EE is still on, so an interrupt (e.g. decrementer or external interrupt) can hit while SRR/GPRs are being loaded. This is defined via: RESTART_TABLE(.Lsyscall_rst_start, .Lsyscall_rst_end, syscall_restart) This restart table then sends us to syscall_restart rather than resuming in the middle of the RFI. The same stub is also used if irq_happened already has a pending bit (soft-masked irq that has not been replayed yet (PowerPC special case of local_irq_disable())). Here is a bit of a flow of sequence of code to visualize: syscall_exit_prepare decide full-GPR restore (_TIF_RESTOREALL) for signal, rt_sigreturn or syscall trace save that in regs->exit_result and return it in r3 | v .Lsyscall_rst_start .. _end EE still on irq_happened set or interrupt in this range? | no | yes v v cmpdi r3,0 syscall_exit_restart restore all / zero replay irq, try exit again volatiles; RFI must return flags in r3 again for the same cmpdi Now r3 after prepare is the flags word, not the actual syscall return. A nested interrupt clobbers it, so the restart stub reloads RESULT into r3 and the C handler (syscall_exit_restart()) should put the flags back (because later asm checks whether r3 returned from C has _TIF_RESTOREALL set or not): cmpdi r3, 0 bne .Lsyscall_restore_regs Note that syscall_exit_restart() already ORs any new _TIF_RESTOREALL into exit_result, but then it only returns the new sample and not the full regs->exit_result. That sample could be often 0 even when restore-all is still required: - rt_sigreturn / syscall trace set the bit in prepare's local ret and in exit_result. They never set exit_flags, which is what restart samples. - a signal does set exit_flags but restart clears it. A second pass through the stub then returns 0 while exit_result still has the bit. The asm as mentioned earlier then treats r3==0 as the fast path and zeros r0/r4-r12. That means the userspace that needed the full register set could SIGSEGVs, (which could happen often in ld64.so.2 like while doing a parallel kernel build as reported by Venkat). So we should instead return the accumulated exit_result, like how we do in interrupt_exit_user_restart(). Note that prior to this commit 263e5159e00a ("powerpc: Fix exit_flags field placement in pt_regs for ptrace") we were returning regs->exit_result from syscall_exit_restart(), but this commit changed that behaviour. Fixes: 263e5159e00a ("powerpc: Fix exit_flags field placement in pt_regs for ptrace") Reported-by: Venkat Rao Bagalkote <venkat88@linux.ibm.com> Closes: https://lore.kernel.org/all/75419f88-eab9-444b-bf97-28a9765819ad@linux.ibm.com/ Signed-off-by: Ritesh Harjani (IBM) <ritesh.list@gmail.com> --- Sorry about the long commit msg. It took sometime for me to fully understand that complex path, so I thought I may as well document that properly. arch/powerpc/kernel/interrupt.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) -- 2.39.5