Message ID | 1409132041-11890-1-git-send-email-zhong@linux.vnet.ibm.com (mailing list archive) |
---|---|
State | Accepted |
Delegated to: | Michael Ellerman |
Headers | show |
On 27.08.2014 [17:33:59 +0800], Li Zhong wrote: > With commit 2fabf084b6ad ("powerpc: reorder per-cpu NUMA information's > initialization"), during boottime, cpu_numa_callback() is called > earlier(before their online) for each cpu, and verify_cpu_node_mapping() > uses cpu_to_node() to check whether siblings are in the same node. > > It skips the checking for siblings that are not online yet. So the only > check done here is for the bootcpu, which is online at that time. But > the per-cpu numa_node cpu_to_node() uses hasn't been set up yet (which > will be set up in smp_prepare_cpus()). > > So I saw something like following reported: > [ 0.000000] CPU thread siblings 1/2/3 and 0 don't belong to the same > node! > > As we don't actually do the checking during this early stage, so maybe > we could directly call numa_setup_cpu() in do_init_bootmem(). > > Cc: Nishanth Aravamudan <nacc@linux.vnet.ibm.com> > Cc: Nathan Fontenot <nfont@linux.vnet.ibm.com> > Signed-off-by: Li Zhong <zhong@linux.vnet.ibm.com> Acked-by: Nishanth Aravamudan <nacc@linux.vnet.ibm.com> > --- > arch/powerpc/mm/numa.c | 3 +-- > 1 file changed, 1 insertion(+), 2 deletions(-) > > diff --git a/arch/powerpc/mm/numa.c b/arch/powerpc/mm/numa.c > index d7737a5..9918c02 100644 > --- a/arch/powerpc/mm/numa.c > +++ b/arch/powerpc/mm/numa.c > @@ -1128,8 +1128,7 @@ void __init do_init_bootmem(void) > * early in boot, cf. smp_prepare_cpus(). > */ > for_each_possible_cpu(cpu) { > - cpu_numa_callback(&ppc64_numa_nb, CPU_UP_PREPARE, > - (void *)(unsigned long)cpu); > + numa_setup_cpu((unsigned long)cpu); > } > } > > -- > 1.9.1 >
Ben & Michael, What's the status of these patches? Thanks, Nish On 27.08.2014 [17:33:59 +0800], Li Zhong wrote: > With commit 2fabf084b6ad ("powerpc: reorder per-cpu NUMA information's > initialization"), during boottime, cpu_numa_callback() is called > earlier(before their online) for each cpu, and verify_cpu_node_mapping() > uses cpu_to_node() to check whether siblings are in the same node. > > It skips the checking for siblings that are not online yet. So the only > check done here is for the bootcpu, which is online at that time. But > the per-cpu numa_node cpu_to_node() uses hasn't been set up yet (which > will be set up in smp_prepare_cpus()). > > So I saw something like following reported: > [ 0.000000] CPU thread siblings 1/2/3 and 0 don't belong to the same > node! > > As we don't actually do the checking during this early stage, so maybe > we could directly call numa_setup_cpu() in do_init_bootmem(). > > Cc: Nishanth Aravamudan <nacc@linux.vnet.ibm.com> > Cc: Nathan Fontenot <nfont@linux.vnet.ibm.com> > Signed-off-by: Li Zhong <zhong@linux.vnet.ibm.com> > --- > arch/powerpc/mm/numa.c | 3 +-- > 1 file changed, 1 insertion(+), 2 deletions(-) > > diff --git a/arch/powerpc/mm/numa.c b/arch/powerpc/mm/numa.c > index d7737a5..9918c02 100644 > --- a/arch/powerpc/mm/numa.c > +++ b/arch/powerpc/mm/numa.c > @@ -1128,8 +1128,7 @@ void __init do_init_bootmem(void) > * early in boot, cf. smp_prepare_cpus(). > */ > for_each_possible_cpu(cpu) { > - cpu_numa_callback(&ppc64_numa_nb, CPU_UP_PREPARE, > - (void *)(unsigned long)cpu); > + numa_setup_cpu((unsigned long)cpu); > } > } > > -- > 1.9.1 >
On Thu, 2014-10-02 at 14:13 -0700, Nishanth Aravamudan wrote: > Ben & Michael, > > What's the status of these patches? Waiting for somebody to review them ? :-) Cheers, Ben. > Thanks, > Nish > > On 27.08.2014 [17:33:59 +0800], Li Zhong wrote: > > With commit 2fabf084b6ad ("powerpc: reorder per-cpu NUMA information's > > initialization"), during boottime, cpu_numa_callback() is called > > earlier(before their online) for each cpu, and verify_cpu_node_mapping() > > uses cpu_to_node() to check whether siblings are in the same node. > > > > It skips the checking for siblings that are not online yet. So the only > > check done here is for the bootcpu, which is online at that time. But > > the per-cpu numa_node cpu_to_node() uses hasn't been set up yet (which > > will be set up in smp_prepare_cpus()). > > > > So I saw something like following reported: > > [ 0.000000] CPU thread siblings 1/2/3 and 0 don't belong to the same > > node! > > > > As we don't actually do the checking during this early stage, so maybe > > we could directly call numa_setup_cpu() in do_init_bootmem(). > > > > Cc: Nishanth Aravamudan <nacc@linux.vnet.ibm.com> > > Cc: Nathan Fontenot <nfont@linux.vnet.ibm.com> > > Signed-off-by: Li Zhong <zhong@linux.vnet.ibm.com> > > --- > > arch/powerpc/mm/numa.c | 3 +-- > > 1 file changed, 1 insertion(+), 2 deletions(-) > > > > diff --git a/arch/powerpc/mm/numa.c b/arch/powerpc/mm/numa.c > > index d7737a5..9918c02 100644 > > --- a/arch/powerpc/mm/numa.c > > +++ b/arch/powerpc/mm/numa.c > > @@ -1128,8 +1128,7 @@ void __init do_init_bootmem(void) > > * early in boot, cf. smp_prepare_cpus(). > > */ > > for_each_possible_cpu(cpu) { > > - cpu_numa_callback(&ppc64_numa_nb, CPU_UP_PREPARE, > > - (void *)(unsigned long)cpu); > > + numa_setup_cpu((unsigned long)cpu); > > } > > } > > > > -- > > 1.9.1 > >
On 03.10.2014 [07:28:19 +1000], Benjamin Herrenschmidt wrote: > On Thu, 2014-10-02 at 14:13 -0700, Nishanth Aravamudan wrote: > > Ben & Michael, > > > > What's the status of these patches? > > Waiting for somebody to review them ? :-) Heh, I acked them about a month ago, I can give a stronger Reviewed-by, if you prefer? -Nish
On Thu, 2014-10-02 at 14:53 -0700, Nishanth Aravamudan wrote: > On 03.10.2014 [07:28:19 +1000], Benjamin Herrenschmidt wrote: > > On Thu, 2014-10-02 at 14:13 -0700, Nishanth Aravamudan wrote: > > > Ben & Michael, > > > > > > What's the status of these patches? > > > > Waiting for somebody to review them ? :-) > > Heh, I acked them about a month ago, I can give a stronger Reviewed-by, > if you prefer? Nah, nothing other than having me or mpe pick them up. I've been letting Michael handle this merge window, so maybe he should take it :-) Cheers, Ben.
On Thu, 2014-10-02 at 14:13 -0700, Nishanth Aravamudan wrote: > Ben & Michael, > > What's the status of these patches? Been in my next for a week :) https://git.kernel.org/cgit/linux/kernel/git/mpe/linux.git/log/?h=next cheers
On 03.10.2014 [10:50:20 +1000], Michael Ellerman wrote: > On Thu, 2014-10-02 at 14:13 -0700, Nishanth Aravamudan wrote: > > Ben & Michael, > > > > What's the status of these patches? > > Been in my next for a week :) > > https://git.kernel.org/cgit/linux/kernel/git/mpe/linux.git/log/?h=next Ah ok, thanks -- I wasn't following your tree, my fault. Do we want these to go back to 3.17-stable, as they fix some annoying splats during boot (non-fatal afaict, though)? -Nish
On Fri, 2014-10-03 at 16:26 -0700, Nishanth Aravamudan wrote: > On 03.10.2014 [10:50:20 +1000], Michael Ellerman wrote: > > On Thu, 2014-10-02 at 14:13 -0700, Nishanth Aravamudan wrote: > > > Ben & Michael, > > > > > > What's the status of these patches? > > > > Been in my next for a week :) > > > > https://git.kernel.org/cgit/linux/kernel/git/mpe/linux.git/log/?h=next > > Ah ok, thanks -- I wasn't following your tree, my fault. Not really your fault, I hadn't announced my trees existence :) > Do we want these to go back to 3.17-stable, as they fix some annoying splats > during boot (non-fatal afaict, though)? Up to you really, I don't know how often/bad they were. I haven't added CC stable tags to the commits, so if you want them in stable you should send them explicitly. cheers
On 07.10.2014 [17:28:38 +1100], Michael Ellerman wrote: > On Fri, 2014-10-03 at 16:26 -0700, Nishanth Aravamudan wrote: > > On 03.10.2014 [10:50:20 +1000], Michael Ellerman wrote: > > > On Thu, 2014-10-02 at 14:13 -0700, Nishanth Aravamudan wrote: > > > > Ben & Michael, > > > > > > > > What's the status of these patches? > > > > > > Been in my next for a week :) > > > > > > https://git.kernel.org/cgit/linux/kernel/git/mpe/linux.git/log/?h=next > > > > Ah ok, thanks -- I wasn't following your tree, my fault. > > Not really your fault, I hadn't announced my trees existence :) > > > Do we want these to go back to 3.17-stable, as they fix some annoying splats > > during boot (non-fatal afaict, though)? > > Up to you really, I don't know how often/bad they were. I haven't added CC > stable tags to the commits, so if you want them in stable you should send them > explicitly. I think they occur every boot, unconditionally, on pseries. Doesn't prevent boot, just really noisy. I think it'd be good to get them into -stable. Li Zhong, can you push them once they get sent upstream? Thanks, Nish
On 二, 2014-10-07 at 08:33 -0700, Nishanth Aravamudan wrote: > On 07.10.2014 [17:28:38 +1100], Michael Ellerman wrote: > > On Fri, 2014-10-03 at 16:26 -0700, Nishanth Aravamudan wrote: > > > On 03.10.2014 [10:50:20 +1000], Michael Ellerman wrote: > > > > On Thu, 2014-10-02 at 14:13 -0700, Nishanth Aravamudan wrote: > > > > > Ben & Michael, > > > > > > > > > > What's the status of these patches? > > > > > > > > Been in my next for a week :) > > > > > > > > https://git.kernel.org/cgit/linux/kernel/git/mpe/linux.git/log/?h=next > > > > > > Ah ok, thanks -- I wasn't following your tree, my fault. > > > > Not really your fault, I hadn't announced my trees existence :) > > > > > Do we want these to go back to 3.17-stable, as they fix some annoying splats > > > during boot (non-fatal afaict, though)? > > > > Up to you really, I don't know how often/bad they were. I haven't added CC > > stable tags to the commits, so if you want them in stable you should send them > > explicitly. > > I think they occur every boot, unconditionally, on pseries. Doesn't > prevent boot, just really noisy. I think it'd be good to get them into > -stable. > > Li Zhong, can you push them once they get sent upstream? Ok, I will send these patches to stable mailing list after it is merged. Thanks, Zhong > > Thanks, > Nish
On 二, 2014-10-07 at 08:33 -0700, Nishanth Aravamudan wrote: > On 07.10.2014 [17:28:38 +1100], Michael Ellerman wrote: > > On Fri, 2014-10-03 at 16:26 -0700, Nishanth Aravamudan wrote: > > > On 03.10.2014 [10:50:20 +1000], Michael Ellerman wrote: > > > > On Thu, 2014-10-02 at 14:13 -0700, Nishanth Aravamudan wrote: > > > > > Ben & Michael, > > > > > > > > > > What's the status of these patches? > > > > > > > > Been in my next for a week :) > > > > > > > > https://git.kernel.org/cgit/linux/kernel/git/mpe/linux.git/log/?h=next > > > > > > Ah ok, thanks -- I wasn't following your tree, my fault. > > > > Not really your fault, I hadn't announced my trees existence :) > > > > > Do we want these to go back to 3.17-stable, as they fix some annoying splats > > > during boot (non-fatal afaict, though)? > > > > Up to you really, I don't know how often/bad they were. I haven't added CC > > stable tags to the commits, so if you want them in stable you should send them > > explicitly. > > I think they occur every boot, unconditionally, on pseries. Doesn't > prevent boot, just really noisy. I think it'd be good to get them into > -stable. > > Li Zhong, can you push them once they get sent upstream? I guess I only need to send the first two patches to stable? Thanks, Zhong > > Thanks, > Nish
On Tue, 2014-10-14 at 10:39 +0800, Li Zhong wrote: > On 二, 2014-10-07 at 08:33 -0700, Nishanth Aravamudan wrote: > > On 07.10.2014 [17:28:38 +1100], Michael Ellerman wrote: > > > On Fri, 2014-10-03 at 16:26 -0700, Nishanth Aravamudan wrote: > > > > On 03.10.2014 [10:50:20 +1000], Michael Ellerman wrote: > > > > > On Thu, 2014-10-02 at 14:13 -0700, Nishanth Aravamudan wrote: > > > > > > Ben & Michael, > > > > > > > > > > > > What's the status of these patches? > > > > > > > > > > Been in my next for a week :) > > > > > > > > > > https://git.kernel.org/cgit/linux/kernel/git/mpe/linux.git/log/?h=next > > > > > > > > Ah ok, thanks -- I wasn't following your tree, my fault. > > > > > > Not really your fault, I hadn't announced my trees existence :) > > > > > > > Do we want these to go back to 3.17-stable, as they fix some annoying splats > > > > during boot (non-fatal afaict, though)? > > > > > > Up to you really, I don't know how often/bad they were. I haven't added CC > > > stable tags to the commits, so if you want them in stable you should send them > > > explicitly. > > > > I think they occur every boot, unconditionally, on pseries. Doesn't > > prevent boot, just really noisy. I think it'd be good to get them into > > -stable. > > > > Li Zhong, can you push them once they get sent upstream? > > I guess I only need to send the first two patches to stable? Probably. It's not clear from the changelog how serious a problem it fixes. See Documentation/stable_kernel_rules.txt cheers
On 二, 2014-10-14 at 15:35 +1100, Michael Ellerman wrote: > On Tue, 2014-10-14 at 10:39 +0800, Li Zhong wrote: > > On 二, 2014-10-07 at 08:33 -0700, Nishanth Aravamudan wrote: > > > On 07.10.2014 [17:28:38 +1100], Michael Ellerman wrote: > > > > On Fri, 2014-10-03 at 16:26 -0700, Nishanth Aravamudan wrote: > > > > > On 03.10.2014 [10:50:20 +1000], Michael Ellerman wrote: > > > > > > On Thu, 2014-10-02 at 14:13 -0700, Nishanth Aravamudan wrote: > > > > > > > Ben & Michael, > > > > > > > > > > > > > > What's the status of these patches? > > > > > > > > > > > > Been in my next for a week :) > > > > > > > > > > > > https://git.kernel.org/cgit/linux/kernel/git/mpe/linux.git/log/?h=next > > > > > > > > > > Ah ok, thanks -- I wasn't following your tree, my fault. > > > > > > > > Not really your fault, I hadn't announced my trees existence :) > > > > > > > > > Do we want these to go back to 3.17-stable, as they fix some annoying splats > > > > > during boot (non-fatal afaict, though)? > > > > > > > > Up to you really, I don't know how often/bad they were. I haven't added CC > > > > stable tags to the commits, so if you want them in stable you should send them > > > > explicitly. > > > > > > I think they occur every boot, unconditionally, on pseries. Doesn't > > > prevent boot, just really noisy. I think it'd be good to get them into > > > -stable. > > > > > > Li Zhong, can you push them once they get sent upstream? > > > > I guess I only need to send the first two patches to stable? > > Probably. It's not clear from the changelog how serious a problem it fixes. > > See Documentation/stable_kernel_rules.txt OK, I'll send the first two to stable. Thanks, Zhong > > cheers > > >
diff --git a/arch/powerpc/mm/numa.c b/arch/powerpc/mm/numa.c index d7737a5..9918c02 100644 --- a/arch/powerpc/mm/numa.c +++ b/arch/powerpc/mm/numa.c @@ -1128,8 +1128,7 @@ void __init do_init_bootmem(void) * early in boot, cf. smp_prepare_cpus(). */ for_each_possible_cpu(cpu) { - cpu_numa_callback(&ppc64_numa_nb, CPU_UP_PREPARE, - (void *)(unsigned long)cpu); + numa_setup_cpu((unsigned long)cpu); } }
With commit 2fabf084b6ad ("powerpc: reorder per-cpu NUMA information's initialization"), during boottime, cpu_numa_callback() is called earlier(before their online) for each cpu, and verify_cpu_node_mapping() uses cpu_to_node() to check whether siblings are in the same node. It skips the checking for siblings that are not online yet. So the only check done here is for the bootcpu, which is online at that time. But the per-cpu numa_node cpu_to_node() uses hasn't been set up yet (which will be set up in smp_prepare_cpus()). So I saw something like following reported: [ 0.000000] CPU thread siblings 1/2/3 and 0 don't belong to the same node! As we don't actually do the checking during this early stage, so maybe we could directly call numa_setup_cpu() in do_init_bootmem(). Cc: Nishanth Aravamudan <nacc@linux.vnet.ibm.com> Cc: Nathan Fontenot <nfont@linux.vnet.ibm.com> Signed-off-by: Li Zhong <zhong@linux.vnet.ibm.com> --- arch/powerpc/mm/numa.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-)