| Message ID | 68fe24ea-7795-24d8-211b-9d8a50affe9f@akamai.com (mailing list archive) |
|---|---|
| State | Not Applicable |
| Headers | show |
On 02/27/2017 02:50 PM, Jason Baron wrote: > > > On 02/27/2017 05:45 PM, David Daney wrote: >> On 02/27/2017 02:36 PM, Steven Rostedt wrote: >>> On Mon, 27 Feb 2017 14:21:21 -0800 >>> David Daney <ddaney@caviumnetworks.com> wrote: >>> >>>> See attached for mips. It seems to do the right thing. >>>> >>>> I leave it as an exercise to the reader to fix the other architectures. >>>> >>>> Consult your own binutils experts to verify that what I say is true. >>> >>> It may still just be safer to do the pointers instead. That way we >>> don't need to worry about some strange arch or off by one binutils >>> messing it up. >> >> Obviously it is your choice, but this is bog standard ELF linking. In >> theory even the arrays of power-of-2 sized objects should also supply an >> entity size. Think __ex_table and its ilk. >> >> >> The benefit of supplying an entsize is that you don't have to change the >> structure of the existing code and risk breaking something in the >> process. >> >> David Daney >> >> > > Thanks for the suggestion! I would like to see if this resolves the ppc > issue we had. I'm attaching a powerpc patch based on your suggestion. > Hopefully, Sachin can try it. > If there are problems, you could try something like: $ find . -name \*\.o | xargs mips64-octeon-linux-gnu-readelf -eW | grep 'File:\| __jump_table' File: ./drivers/firmware/built-in.o File: ./drivers/built-in.o [3249] __jump_table PROGBITS 0000000000000000 1838c8 0022c8 18 WAM 0 0 8 File: ./drivers/spi/built-in.o [82] __jump_table PROGBITS 0000000000000000 008cb0 000048 18 WAM 0 0 8 File: ./drivers/spi/spi-cavium-octeon.o File: ./drivers/spi/spi-cavium.o File: ./drivers/spi/spi.o . . . Look for files where the size of the __jump_table section is not a integer multiple of the entsize. > Thanks, > > -Jason
> Thanks for the suggestion! I would like to see if this resolves the ppc issue we had. I'm attaching a powerpc patch based on your suggestion. Hopefully, Sachin can try it. > > Thanks, > I tried this patch. It does not fix the warning. [ 11.709071] mount (2956) used greatest stack depth: 10176 bytes left [ 11.731883] ------------[ cut here ]------------ [ 11.731911] WARNING: CPU: 3 PID: 2972 at kernel/jump_label.c:287 static_key_set_entries.isra.10+0x3c/0x50 [ 11.731915] Modules linked in: nfsd(+) ip_tables x_tables autofs4 [ 11.731925] CPU: 3 PID: 2972 Comm: modprobe Not tainted 4.10.0-next-20170227 #4 [ 11.731930] task: c00000077b284a00 task.stack: c00000077b8b8000 [ 11.731933] NIP: c0000000017bf84c LR: c0000000017bfcbc CTR: 0000000000000000 [ 11.731937] REGS: c00000077b8bb800 TRAP: 0700 Not tainted (4.10.0-next-20170227) [ 11.731940] MSR: 800000000282b033 <SF,VEC,VSX,EE,FP,ME,IR,DR,RI,LE> [ 11.731948] CR: 48248282 XER: 00000001 [ 11.731953] CFAR: c0000000017bf81c SOFTE: 1 GPR00: c0000000017bfc7c c00000077b8bba80 c00000000266c300 d0000000063e28f8 GPR04: d0000000063e5b57 0000000100000017 c0000000017bf5a0 0000000000000000 GPR08: 0000000000052eb3 0000000000000001 c00000000258c300 0000000000000001 GPR12: c000000001b5b460 c00000000ea80c00 0000000000000020 d000000006380bb0 GPR16: c00000077b8bbda0 c00000077b8bbdec 0000000000000000 0000000000008580 GPR20: d000000006410000 d0000000063e7ea8 c00000000256db90 0000000000000001 GPR24: c00000000258ca14 0000000000000000 c0000000025737f8 d0000000063e5c17 GPR28: 0000000000000000 d0000000063e6780 d0000000063e28f0 d0000000063e5b57 [ 11.732000] NIP [c0000000017bf84c] static_key_set_entries.isra.10+0x3c/0x50 [ 11.732004] LR [c0000000017bfcbc] jump_label_module_notify+0x20c/0x420 [ 11.732007] Call Trace: [ 11.732011] [c00000077b8bba80] [c0000000017bfc7c] jump_label_module_notify+0x1cc/0x420 (unreliable) [ 11.732019] [c00000077b8bbb40] [c0000000016b69b0] notifier_call_chain+0x90/0x100 [ 11.732024] [c00000077b8bbb90] [c0000000016b6e80] __blocking_notifier_call_chain+0x60/0x90 [ 11.732029] [c00000077b8bbbe0] [c0000000017380ec] load_module+0x1c2c/0x2760 [ 11.732034] [c00000077b8bbd70] [c000000001738e80] SyS_finit_module+0xc0/0xf0 [ 11.732040] [c00000077b8bbe30] [c0000000015cb8e0] system_call+0x38/0xfc [ 11.732043] Instruction dump: [ 11.732046] 40c20018 e9230000 792907a0 7c844b78 f8830000 4e800020 3d42fff2 892a0714 [ 11.732053] 2f890000 40feffe0 39200001 992a0714 <0fe00000> 4bffffd0 60000000 60000000 [ 11.732061] ---[ end trace 13c67d418143453c ]--- [ 11.732319] Installing knfsd (copyright (C) 1996 okir@monad.swb.de). I have collected the o/p of the command suggested by David. Here is a snippet from the run File: ./arch/powerpc/kernel/built-in.o [383] __jump_table PROGBITS 0000000000000000 068020 000c78 18 WAM 0 0 1 File: ./arch/powerpc/kernel/rtasd.o File: ./arch/powerpc/kernel/of_platform.o File: ./arch/powerpc/kernel/eeh_event.o File: ./arch/powerpc/kernel/setup_64.o [18] __jump_table PROGBITS 0000000000000000 001240 000048 18 WAM 0 0 1 File: ./arch/powerpc/kernel/rtas-proc.o File: ./arch/powerpc/kernel/signal_64.o [13] __jump_table PROGBITS 0000000000000000 001c68 000060 18 WAM 0 0 1 Have attached the complete o/p here for reference. Thanks -Sachin
On Tue, 28 Feb 2017 10:25:46 +0530 Sachin Sant <sachinp@linux.vnet.ibm.com> wrote: > File: ./net/ipv4/xfrm4_input.o > [12] __jump_table PROGBITS 0000000000000000 000639 000018 18 WAM 0 0 1 > File: ./net/ipv4/udplite.o > File: ./net/ipv4/xfrm4_output.o > [ 9] __jump_table PROGBITS 0000000000000000 000481 000018 18 WAM 0 0 1 Looks like there's some issues right there. -- Steve
On 02/28/2017 08:21 AM, Steven Rostedt wrote: > On Tue, 28 Feb 2017 10:25:46 +0530 > Sachin Sant <sachinp@linux.vnet.ibm.com> wrote: > >> File: ./net/ipv4/xfrm4_input.o >> [12] __jump_table PROGBITS 0000000000000000 000639 000018 18 WAM 0 0 1 >> File: ./net/ipv4/udplite.o >> File: ./net/ipv4/xfrm4_output.o >> [ 9] __jump_table PROGBITS 0000000000000000 000481 000018 18 WAM 0 0 1 > > Looks like there's some issues right there. Those look good to me 18/18 = 1 with no remainder. The odd numbers are the offset of the section in the ELF file. If you look at the stack trace, it seems that it is during module loading. Are the primitives for generating the tables doing something different for the module case? I am not familiar enough with the powerpc ABIs to know. Try this: $ perl -n -e 's/\[ /\[/; my @f = split " "; print hex($f[5]) % 0x18 if $#f > 5; print $_' <~/jump_table.log There are no entries with size that is not a multiple of 0x18. I think my patch to add the ENTSIZE is not doing anything here. I suspect that the alignment of the __jump_table section in the .ko files is not correct, and you are seeing some sort of problem due to that. > > -- Steve >
On 02/28/2017 01:16 PM, David Daney wrote: > On 02/28/2017 08:21 AM, Steven Rostedt wrote: >> On Tue, 28 Feb 2017 10:25:46 +0530 >> Sachin Sant <sachinp@linux.vnet.ibm.com> wrote: >> >>> File: ./net/ipv4/xfrm4_input.o >>> [12] __jump_table PROGBITS 0000000000000000 000639 >>> 000018 18 WAM 0 0 1 >>> File: ./net/ipv4/udplite.o >>> File: ./net/ipv4/xfrm4_output.o >>> [ 9] __jump_table PROGBITS 0000000000000000 000481 >>> 000018 18 WAM 0 0 1 >> >> Looks like there's some issues right there. > > Those look good to me 18/18 = 1 with no remainder. The odd numbers are > the offset of the section in the ELF file. > > If you look at the stack trace, it seems that it is during module loading. > > Are the primitives for generating the tables doing something different > for the module case? I am not familiar enough with the powerpc ABIs to > know. > > Try this: > > $ perl -n -e 's/\[ /\[/; my @f = split " "; print hex($f[5]) % 0x18 if > $#f > 5; print $_' <~/jump_table.log > > > There are no entries with size that is not a multiple of 0x18. > > I think my patch to add the ENTSIZE is not doing anything here. > > I suspect that the alignment of the __jump_table section in the .ko > files is not correct, and you are seeing some sort of problem due to that. > > Hi, Yes, if you look at the trace that Sachin sent the module being loaded that does the WARN_ON() is nfsd.ko. That module from Sachin's trace has: [31] __jump_table PROGBITS 0000000000000000 03fd77 0000c0 18 WAM 0 0 1 So its not the size but rather the start offset '03fd77', that is the problem here. That is what the WARN_ON triggers on, that the start of the table is not 4-byte aligned. Using a ppc cross-compiler and the ENTSIZE patch that line does not change, however if I use the initial patch posted in this thread, the start does align to 4-bytes and thus the warning goes away, as Sachin verified. In fact, without the patch I found several modules that don't start at the proper alignment, however with the patch that started this thread they were all properly aligned. In terms of the '.balign' causing holes, we originally added the '_ASM_ALIGN' to x86 for precisely this reason. See commit: ef64789 jump label: Add _ASM_ALIGN for x86 and x86_64 and discussion. In addition, we have a lot of runtime with the .balign in the tree and I'm not aware of any holes in the table. I think the code would blow up pretty badly if there were. A number of arches were already using the '.balign', and the patch I proposed simply added it to remaining ones, now that we added a WARN_ON() to catch this condition. Thanks, -Jason
On 02/28/2017 10:39 AM, Jason Baron wrote: > > > On 02/28/2017 01:16 PM, David Daney wrote: >> On 02/28/2017 08:21 AM, Steven Rostedt wrote: >>> On Tue, 28 Feb 2017 10:25:46 +0530 >>> Sachin Sant <sachinp@linux.vnet.ibm.com> wrote: >>> >>>> File: ./net/ipv4/xfrm4_input.o >>>> [12] __jump_table PROGBITS 0000000000000000 000639 >>>> 000018 18 WAM 0 0 1 >>>> File: ./net/ipv4/udplite.o >>>> File: ./net/ipv4/xfrm4_output.o >>>> [ 9] __jump_table PROGBITS 0000000000000000 000481 >>>> 000018 18 WAM 0 0 1 >>> >>> Looks like there's some issues right there. >> >> Those look good to me 18/18 = 1 with no remainder. The odd numbers are >> the offset of the section in the ELF file. >> >> If you look at the stack trace, it seems that it is during module >> loading. >> >> Are the primitives for generating the tables doing something different >> for the module case? I am not familiar enough with the powerpc ABIs to >> know. >> >> Try this: >> >> $ perl -n -e 's/\[ /\[/; my @f = split " "; print hex($f[5]) % 0x18 if >> $#f > 5; print $_' <~/jump_table.log >> >> >> There are no entries with size that is not a multiple of 0x18. >> >> I think my patch to add the ENTSIZE is not doing anything here. >> >> I suspect that the alignment of the __jump_table section in the .ko >> files is not correct, and you are seeing some sort of problem due to >> that. >> >> > > Hi, > > Yes, if you look at the trace that Sachin sent the module being loaded > that does the WARN_ON() is nfsd.ko. > > That module from Sachin's trace has: > > [31] __jump_table PROGBITS 0000000000000000 03fd77 0000c0 > 18 WAM 0 0 1 The problem is then the section alignment (last column) for power. On mips with no patches applied, we get: [17] __jump_table PROGBITS 0000000000000000 00d2c0 000048 00 WA 0 0 8 Look, proper alignment! The question I have is why do the power ".llong" and ".long" assembler directives not force section alignment? Is there an alternative that could be used that would result in the proper alignment? Would ".word" work? If not, then I would say patch only power with your balign thing. 8-byte alignment for 64-bit kernel, 4-byte alignment for 32-bit kernel > > So its not the size but rather the start offset '03fd77', that is the > problem here. That is what the WARN_ON triggers on, that the start of > the table is not 4-byte aligned. > > Using a ppc cross-compiler and the ENTSIZE patch that line does not > change, however if I use the initial patch posted in this thread, the > start does align to 4-bytes and thus the warning goes away, as Sachin > verified. In fact, without the patch I found several modules that don't > start at the proper alignment, however with the patch that started this > thread they were all properly aligned. > > In terms of the '.balign' causing holes, we originally added the > '_ASM_ALIGN' to x86 for precisely this reason. See commit: > ef64789 jump label: Add _ASM_ALIGN for x86 and x86_64 and discussion. > > In addition, we have a lot of runtime with the .balign in the tree and > I'm not aware of any holes in the table. I think the code would blow up > pretty badly if there were. > > A number of arches were already using the '.balign', and the patch I > proposed simply added it to remaining ones, now that we added a > WARN_ON() to catch this condition. > > Thanks, > > -Jason > > > >
On 02/28/2017 11:05 AM, David Daney wrote: > On 02/28/2017 10:39 AM, Jason Baron wrote: >> [...] >>> I suspect that the alignment of the __jump_table section in the .ko >>> files is not correct, and you are seeing some sort of problem due to >>> that. >>> >>> >> >> Hi, >> >> Yes, if you look at the trace that Sachin sent the module being loaded >> that does the WARN_ON() is nfsd.ko. >> >> That module from Sachin's trace has: >> >> [31] __jump_table PROGBITS 0000000000000000 03fd77 0000c0 >> 18 WAM 0 0 1 > > The problem is then the section alignment (last column) for power. > > On mips with no patches applied, we get: > > [17] __jump_table PROGBITS 0000000000000000 00d2c0 000048 > 00 WA 0 0 8 > > Look, proper alignment! > > The question I have is why do the power ".llong" and ".long" assembler > directives not force section alignment? Is there an alternative that > could be used that would result in the proper alignment? Would ".word" > work? > > If not, then I would say patch only power with your balign thing. 8-byte > alignment for 64-bit kernel, 4-byte alignment for 32-bit kernel > I think the proper fix is either: A) Modify scripts/module-common.lds to force __jump_table alignment for all architectures. B) Add arch/powerpc/kernel/module.lds to force __jump_table alignment for powerpc only. David. > >> >> So its not the size but rather the start offset '03fd77', that is the >> problem here. That is what the WARN_ON triggers on, that the start of >> the table is not 4-byte aligned. >> >> Using a ppc cross-compiler and the ENTSIZE patch that line does not >> change, however if I use the initial patch posted in this thread, the >> start does align to 4-bytes and thus the warning goes away, as Sachin >> verified. In fact, without the patch I found several modules that don't >> start at the proper alignment, however with the patch that started this >> thread they were all properly aligned. >> >> In terms of the '.balign' causing holes, we originally added the >> '_ASM_ALIGN' to x86 for precisely this reason. See commit: >> ef64789 jump label: Add _ASM_ALIGN for x86 and x86_64 and discussion. >> >> In addition, we have a lot of runtime with the .balign in the tree and >> I'm not aware of any holes in the table. I think the code would blow up >> pretty badly if there were. >> >> A number of arches were already using the '.balign', and the patch I >> proposed simply added it to remaining ones, now that we added a >> WARN_ON() to catch this condition. >> >> Thanks, >> >> -Jason >> >> >> >> > > > _______________________________________________ > linux-arm-kernel mailing list > linux-arm-kernel@lists.infradead.org > http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
On 02/28/2017 02:22 PM, David Daney wrote: > On 02/28/2017 11:05 AM, David Daney wrote: >> On 02/28/2017 10:39 AM, Jason Baron wrote: >>> > [...] >>>> I suspect that the alignment of the __jump_table section in the .ko >>>> files is not correct, and you are seeing some sort of problem due to >>>> that. >>>> >>>> >>> >>> Hi, >>> >>> Yes, if you look at the trace that Sachin sent the module being loaded >>> that does the WARN_ON() is nfsd.ko. >>> >>> That module from Sachin's trace has: >>> >>> [31] __jump_table PROGBITS 0000000000000000 03fd77 0000c0 >>> 18 WAM 0 0 1 >> >> The problem is then the section alignment (last column) for power. >> >> On mips with no patches applied, we get: >> >> [17] __jump_table PROGBITS 0000000000000000 00d2c0 000048 >> 00 WA 0 0 8 >> >> Look, proper alignment! >> >> The question I have is why do the power ".llong" and ".long" assembler >> directives not force section alignment? Is there an alternative that >> could be used that would result in the proper alignment? Would ".word" >> work? >> >> If not, then I would say patch only power with your balign thing. 8-byte >> alignment for 64-bit kernel, 4-byte alignment for 32-bit kernel >> > > I think the proper fix is either: > > A) Modify scripts/module-common.lds to force __jump_table alignment for > all architectures. > > B) Add arch/powerpc/kernel/module.lds to force __jump_table alignment > for powerpc only. > > David. > > Ok, I can try adding it to the linger script. FWIW, here is my before and after with the .balign thing for the nfsd.ko module on powperc (using a cross-compiler): before: [31] __jump_table PROGBITS 0000000000000000 03ee3e 0000f0 00 WA 0 0 1 after: [31] __jump_table PROGBITS 0000000000000000 03ee40 0000f0 00 WA 0 0 4 Thanks, -Jason > >> >>> >>> So its not the size but rather the start offset '03fd77', that is the >>> problem here. That is what the WARN_ON triggers on, that the start of >>> the table is not 4-byte aligned. >>> >>> Using a ppc cross-compiler and the ENTSIZE patch that line does not >>> change, however if I use the initial patch posted in this thread, the >>> start does align to 4-bytes and thus the warning goes away, as Sachin >>> verified. In fact, without the patch I found several modules that don't >>> start at the proper alignment, however with the patch that started this >>> thread they were all properly aligned. >>> >>> In terms of the '.balign' causing holes, we originally added the >>> '_ASM_ALIGN' to x86 for precisely this reason. See commit: >>> ef64789 jump label: Add _ASM_ALIGN for x86 and x86_64 and discussion. >>> >>> In addition, we have a lot of runtime with the .balign in the tree and >>> I'm not aware of any holes in the table. I think the code would blow up >>> pretty badly if there were. >>> >>> A number of arches were already using the '.balign', and the patch I >>> proposed simply added it to remaining ones, now that we added a >>> WARN_ON() to catch this condition. >>> >>> Thanks, >>> >>> -Jason >>> >>> >>> >>> >> >> >> _______________________________________________ >> linux-arm-kernel mailing list >> linux-arm-kernel@lists.infradead.org >> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel >
diff --git a/arch/powerpc/include/asm/jump_label.h b/arch/powerpc/include/asm/jump_label.h index 9a287e0ac8b1..3c5660e50f9a 100644 --- a/arch/powerpc/include/asm/jump_label.h +++ b/arch/powerpc/include/asm/jump_label.h @@ -19,14 +19,26 @@ #define JUMP_ENTRY_TYPE stringify_in_c(FTR_ENTRY_LONG) #define JUMP_LABEL_NOP_SIZE 4 +#ifdef CONFIG_PPC64 +typedef u64 jump_label_t; +#else +typedef u32 jump_label_t; +#endif + +struct jump_entry { + jump_label_t code; + jump_label_t target; + jump_label_t key; +}; + static __always_inline bool arch_static_branch(struct static_key *key, bool branch) { asm_volatile_goto("1:\n\t" "nop # arch_static_branch\n\t" - ".pushsection __jump_table, \"aw\"\n\t" + ".pushsection __jump_table, \"awM\",@progbits, %1\n\t" JUMP_ENTRY_TYPE "1b, %l[l_yes], %c0\n\t" ".popsection \n\t" - : : "i" (&((char *)key)[branch]) : : l_yes); + : : "i" (&((char *)key)[branch]), "i" (sizeof(struct jump_entry)) : : l_yes); return false; l_yes: @@ -37,32 +49,24 @@ static __always_inline bool arch_static_branch_jump(struct static_key *key, bool { asm_volatile_goto("1:\n\t" "b %l[l_yes] # arch_static_branch_jump\n\t" - ".pushsection __jump_table, \"aw\"\n\t" + ".pushsection __jump_table, \"awM\",@progbits, %1\n\t" JUMP_ENTRY_TYPE "1b, %l[l_yes], %c0\n\t" ".popsection \n\t" - : : "i" (&((char *)key)[branch]) : : l_yes); + : : "i" (&((char *)key)[branch]), "i" (sizeof(struct jump_entry)) : : l_yes); return false; l_yes: return true; } -#ifdef CONFIG_PPC64 -typedef u64 jump_label_t; + #else -typedef u32 jump_label_t; -#endif -struct jump_entry { - jump_label_t code; - jump_label_t target; - jump_label_t key; -}; +#define ENTRY_SIZE (ULONG_SIZE * 3) -#else #define ARCH_STATIC_BRANCH(LABEL, KEY) \ 1098: nop; \ - .pushsection __jump_table, "aw"; \ + .pushsection __jump_table, "awM",@progbits,ENTRY_SIZE; \ FTR_ENTRY_LONG 1098b, LABEL, KEY; \ .popsection #endif