Message ID | 20201028182334.13466-1-cai@redhat.com (mailing list archive) |
---|---|
State | Accepted |
Commit | 99f070b62322a4b8c1252952735806d09eb44b68 |
Headers | show |
Series | powerpc/smp: Move rcu_cpu_starting() earlier | expand |
Context | Check | Description |
---|---|---|
snowpatch_ozlabs/apply_patch | success | Successfully applied on branch powerpc/merge (8cb17737940b156329cb5210669b9c9b23f4dd56) |
snowpatch_ozlabs/build-ppc64le | success | Build succeeded |
snowpatch_ozlabs/build-ppc64be | success | Build succeeded |
snowpatch_ozlabs/build-ppc64e | success | Build succeeded |
snowpatch_ozlabs/build-pmac32 | success | Build succeeded |
snowpatch_ozlabs/checkpatch | success | |
snowpatch_ozlabs/needsstable | success | Patch has no Fixes tags |
On Wed, Oct 28, 2020 at 02:23:34PM -0400, Qian Cai wrote: > The call to rcu_cpu_starting() in start_secondary() is not early enough > in the CPU-hotplug onlining process, which results in lockdep splats as > follows: > > WARNING: suspicious RCU usage > ----------------------------- > kernel/locking/lockdep.c:3497 RCU-list traversed in non-reader section!! > > other info that might help us debug this: > > RCU used illegally from offline CPU! > rcu_scheduler_active = 1, debug_locks = 1 > no locks held by swapper/1/0. > > Call Trace: > dump_stack+0xec/0x144 (unreliable) > lockdep_rcu_suspicious+0x128/0x14c > __lock_acquire+0x1060/0x1c60 > lock_acquire+0x140/0x5f0 > _raw_spin_lock_irqsave+0x64/0xb0 > clockevents_register_device+0x74/0x270 > register_decrementer_clockevent+0x94/0x110 > start_secondary+0x134/0x800 > start_secondary_prolog+0x10/0x14 > > This is avoided by moving the call to rcu_cpu_starting up near the > beginning of the start_secondary() function. Note that the > raw_smp_processor_id() is required in order to avoid calling into > lockdep before RCU has declared the CPU to be watched for readers. > > Link: https://lore.kernel.org/lkml/160223032121.7002.1269740091547117869.tip-bot2@tip-bot2/ > Signed-off-by: Qian Cai <cai@redhat.com> Acked-by: Paul E. McKenney <paulmck@kernel.org> > --- > arch/powerpc/kernel/smp.c | 3 ++- > 1 file changed, 2 insertions(+), 1 deletion(-) > > diff --git a/arch/powerpc/kernel/smp.c b/arch/powerpc/kernel/smp.c > index 3c6b9822f978..8c2857cbd960 100644 > --- a/arch/powerpc/kernel/smp.c > +++ b/arch/powerpc/kernel/smp.c > @@ -1393,13 +1393,14 @@ static void add_cpu_to_masks(int cpu) > /* Activate a secondary processor. */ > void start_secondary(void *unused) > { > - unsigned int cpu = smp_processor_id(); > + unsigned int cpu = raw_smp_processor_id(); > > mmgrab(&init_mm); > current->active_mm = &init_mm; > > smp_store_cpu_info(cpu); > set_dec(tb_ticks_per_jiffy); > + rcu_cpu_starting(cpu); > preempt_disable(); > cpu_callin_map[cpu] = 1; > > -- > 2.28.0 >
Qian Cai <cai@redhat.com> writes: > The call to rcu_cpu_starting() in start_secondary() is not early enough > in the CPU-hotplug onlining process, which results in lockdep splats as > follows: Since when? What kernel version? I haven't seen this running CPU hotplug tests with PROVE_LOCKING=y on v5.10-rc1. Am I missing a CONFIG? cheers > WARNING: suspicious RCU usage > ----------------------------- > kernel/locking/lockdep.c:3497 RCU-list traversed in non-reader section!! > > other info that might help us debug this: > > RCU used illegally from offline CPU! > rcu_scheduler_active = 1, debug_locks = 1 > no locks held by swapper/1/0. > > Call Trace: > dump_stack+0xec/0x144 (unreliable) > lockdep_rcu_suspicious+0x128/0x14c > __lock_acquire+0x1060/0x1c60 > lock_acquire+0x140/0x5f0 > _raw_spin_lock_irqsave+0x64/0xb0 > clockevents_register_device+0x74/0x270 > register_decrementer_clockevent+0x94/0x110 > start_secondary+0x134/0x800 > start_secondary_prolog+0x10/0x14 > > This is avoided by moving the call to rcu_cpu_starting up near the > beginning of the start_secondary() function. Note that the > raw_smp_processor_id() is required in order to avoid calling into > lockdep before RCU has declared the CPU to be watched for readers. > > Link: https://lore.kernel.org/lkml/160223032121.7002.1269740091547117869.tip-bot2@tip-bot2/ > Signed-off-by: Qian Cai <cai@redhat.com> > --- > arch/powerpc/kernel/smp.c | 3 ++- > 1 file changed, 2 insertions(+), 1 deletion(-) > > diff --git a/arch/powerpc/kernel/smp.c b/arch/powerpc/kernel/smp.c > index 3c6b9822f978..8c2857cbd960 100644 > --- a/arch/powerpc/kernel/smp.c > +++ b/arch/powerpc/kernel/smp.c > @@ -1393,13 +1393,14 @@ static void add_cpu_to_masks(int cpu) > /* Activate a secondary processor. */ > void start_secondary(void *unused) > { > - unsigned int cpu = smp_processor_id(); > + unsigned int cpu = raw_smp_processor_id(); > > mmgrab(&init_mm); > current->active_mm = &init_mm; > > smp_store_cpu_info(cpu); > set_dec(tb_ticks_per_jiffy); > + rcu_cpu_starting(cpu); > preempt_disable(); > cpu_callin_map[cpu] = 1; > > -- > 2.28.0
On Thu, Oct 29, 2020 at 11:09:07AM +1100, Michael Ellerman wrote: > Qian Cai <cai@redhat.com> writes: > > The call to rcu_cpu_starting() in start_secondary() is not early enough > > in the CPU-hotplug onlining process, which results in lockdep splats as > > follows: > > Since when? > What kernel version? > > I haven't seen this running CPU hotplug tests with PROVE_LOCKING=y on > v5.10-rc1. Am I missing a CONFIG? My guess would be that adding CONFIG_PROVE_RAW_LOCK_NESTING=y will get you some splats. Thanx, Paul > cheers > > > > WARNING: suspicious RCU usage > > ----------------------------- > > kernel/locking/lockdep.c:3497 RCU-list traversed in non-reader section!! > > > > other info that might help us debug this: > > > > RCU used illegally from offline CPU! > > rcu_scheduler_active = 1, debug_locks = 1 > > no locks held by swapper/1/0. > > > > Call Trace: > > dump_stack+0xec/0x144 (unreliable) > > lockdep_rcu_suspicious+0x128/0x14c > > __lock_acquire+0x1060/0x1c60 > > lock_acquire+0x140/0x5f0 > > _raw_spin_lock_irqsave+0x64/0xb0 > > clockevents_register_device+0x74/0x270 > > register_decrementer_clockevent+0x94/0x110 > > start_secondary+0x134/0x800 > > start_secondary_prolog+0x10/0x14 > > > > This is avoided by moving the call to rcu_cpu_starting up near the > > beginning of the start_secondary() function. Note that the > > raw_smp_processor_id() is required in order to avoid calling into > > lockdep before RCU has declared the CPU to be watched for readers. > > > > Link: https://lore.kernel.org/lkml/160223032121.7002.1269740091547117869.tip-bot2@tip-bot2/ > > Signed-off-by: Qian Cai <cai@redhat.com> > > --- > > arch/powerpc/kernel/smp.c | 3 ++- > > 1 file changed, 2 insertions(+), 1 deletion(-) > > > > diff --git a/arch/powerpc/kernel/smp.c b/arch/powerpc/kernel/smp.c > > index 3c6b9822f978..8c2857cbd960 100644 > > --- a/arch/powerpc/kernel/smp.c > > +++ b/arch/powerpc/kernel/smp.c > > @@ -1393,13 +1393,14 @@ static void add_cpu_to_masks(int cpu) > > /* Activate a secondary processor. */ > > void start_secondary(void *unused) > > { > > - unsigned int cpu = smp_processor_id(); > > + unsigned int cpu = raw_smp_processor_id(); > > > > mmgrab(&init_mm); > > current->active_mm = &init_mm; > > > > smp_store_cpu_info(cpu); > > set_dec(tb_ticks_per_jiffy); > > + rcu_cpu_starting(cpu); > > preempt_disable(); > > cpu_callin_map[cpu] = 1; > > > > -- > > 2.28.0
On Thu, 2020-10-29 at 11:09 +1100, Michael Ellerman wrote: > Qian Cai <cai@redhat.com> writes: > > The call to rcu_cpu_starting() in start_secondary() is not early enough > > in the CPU-hotplug onlining process, which results in lockdep splats as > > follows: > > Since when? For me, it is since the commit in the link which looks now merged into v5.10-rc1. Then, it needs CONFIG_PROVE_RCU_LIST=y. > What kernel version? > > I haven't seen this running CPU hotplug tests with PROVE_LOCKING=y on > v5.10-rc1. Am I missing a CONFIG? > > cheers > > > > WARNING: suspicious RCU usage > > ----------------------------- > > kernel/locking/lockdep.c:3497 RCU-list traversed in non-reader section!! > > > > other info that might help us debug this: > > > > RCU used illegally from offline CPU! > > rcu_scheduler_active = 1, debug_locks = 1 > > no locks held by swapper/1/0. > > > > Call Trace: > > dump_stack+0xec/0x144 (unreliable) > > lockdep_rcu_suspicious+0x128/0x14c > > __lock_acquire+0x1060/0x1c60 > > lock_acquire+0x140/0x5f0 > > _raw_spin_lock_irqsave+0x64/0xb0 > > clockevents_register_device+0x74/0x270 > > register_decrementer_clockevent+0x94/0x110 > > start_secondary+0x134/0x800 > > start_secondary_prolog+0x10/0x14 > > > > This is avoided by moving the call to rcu_cpu_starting up near the > > beginning of the start_secondary() function. Note that the > > raw_smp_processor_id() is required in order to avoid calling into > > lockdep before RCU has declared the CPU to be watched for readers. > > > > Link: > > https://lore.kernel.org/lkml/160223032121.7002.1269740091547117869.tip-bot2@tip-bot2/ > > Signed-off-by: Qian Cai <cai@redhat.com> > > --- > > arch/powerpc/kernel/smp.c | 3 ++- > > 1 file changed, 2 insertions(+), 1 deletion(-) > > > > diff --git a/arch/powerpc/kernel/smp.c b/arch/powerpc/kernel/smp.c > > index 3c6b9822f978..8c2857cbd960 100644 > > --- a/arch/powerpc/kernel/smp.c > > +++ b/arch/powerpc/kernel/smp.c > > @@ -1393,13 +1393,14 @@ static void add_cpu_to_masks(int cpu) > > /* Activate a secondary processor. */ > > void start_secondary(void *unused) > > { > > - unsigned int cpu = smp_processor_id(); > > + unsigned int cpu = raw_smp_processor_id(); > > > > mmgrab(&init_mm); > > current->active_mm = &init_mm; > > > > smp_store_cpu_info(cpu); > > set_dec(tb_ticks_per_jiffy); > > + rcu_cpu_starting(cpu); > > preempt_disable(); > > cpu_callin_map[cpu] = 1; > > > > -- > > 2.28.0
On Wed, 2020-10-28 at 17:31 -0700, Paul E. McKenney wrote: > On Thu, Oct 29, 2020 at 11:09:07AM +1100, Michael Ellerman wrote: > > Qian Cai <cai@redhat.com> writes: > > > The call to rcu_cpu_starting() in start_secondary() is not early enough > > > in the CPU-hotplug onlining process, which results in lockdep splats as > > > follows: > > > > Since when? > > What kernel version? > > > > I haven't seen this running CPU hotplug tests with PROVE_LOCKING=y on > > v5.10-rc1. Am I missing a CONFIG? > > My guess would be that adding CONFIG_PROVE_RAW_LOCK_NESTING=y will > get you some splats. Well, I don't have that set, so it should be CONFIG_PROVE_RCU_LIST=y. Anyway, this is .config to reproduce on Power9 NV: https://cailca.coding.net/public/linux/mm/git/files/master/powerpc.config
On Wed, 28 Oct 2020 14:23:34 -0400, Qian Cai wrote: > The call to rcu_cpu_starting() in start_secondary() is not early enough > in the CPU-hotplug onlining process, which results in lockdep splats as > follows: > > WARNING: suspicious RCU usage > ----------------------------- > kernel/locking/lockdep.c:3497 RCU-list traversed in non-reader section!! > > [...] Applied to powerpc/fixes. [1/1] powerpc/smp: Call rcu_cpu_starting() earlier https://git.kernel.org/powerpc/c/99f070b62322a4b8c1252952735806d09eb44b68 cheers
diff --git a/arch/powerpc/kernel/smp.c b/arch/powerpc/kernel/smp.c index 3c6b9822f978..8c2857cbd960 100644 --- a/arch/powerpc/kernel/smp.c +++ b/arch/powerpc/kernel/smp.c @@ -1393,13 +1393,14 @@ static void add_cpu_to_masks(int cpu) /* Activate a secondary processor. */ void start_secondary(void *unused) { - unsigned int cpu = smp_processor_id(); + unsigned int cpu = raw_smp_processor_id(); mmgrab(&init_mm); current->active_mm = &init_mm; smp_store_cpu_info(cpu); set_dec(tb_ticks_per_jiffy); + rcu_cpu_starting(cpu); preempt_disable(); cpu_callin_map[cpu] = 1;
The call to rcu_cpu_starting() in start_secondary() is not early enough in the CPU-hotplug onlining process, which results in lockdep splats as follows: WARNING: suspicious RCU usage ----------------------------- kernel/locking/lockdep.c:3497 RCU-list traversed in non-reader section!! other info that might help us debug this: RCU used illegally from offline CPU! rcu_scheduler_active = 1, debug_locks = 1 no locks held by swapper/1/0. Call Trace: dump_stack+0xec/0x144 (unreliable) lockdep_rcu_suspicious+0x128/0x14c __lock_acquire+0x1060/0x1c60 lock_acquire+0x140/0x5f0 _raw_spin_lock_irqsave+0x64/0xb0 clockevents_register_device+0x74/0x270 register_decrementer_clockevent+0x94/0x110 start_secondary+0x134/0x800 start_secondary_prolog+0x10/0x14 This is avoided by moving the call to rcu_cpu_starting up near the beginning of the start_secondary() function. Note that the raw_smp_processor_id() is required in order to avoid calling into lockdep before RCU has declared the CPU to be watched for readers. Link: https://lore.kernel.org/lkml/160223032121.7002.1269740091547117869.tip-bot2@tip-bot2/ Signed-off-by: Qian Cai <cai@redhat.com> --- arch/powerpc/kernel/smp.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)