Patchwork mtd: Fix bug using smp_processor_id() in preemptible ubi_bgt1d kthread

login
register
mail settings
Submitter Philby John
Date June 14, 2010, 11:04 a.m.
Message ID <1276513457.16642.3.camel@localhost.localdomain>
Download mbox | patch
Permalink /patch/55519/
State New
Headers show

Comments

Philby John - June 14, 2010, 11:04 a.m.
mtd: Fix bug using smp_processor_id() in preemptible ubi_bgt1d kthread

On a MIPS Cavium Octeon CN5020 when trying to create a UBI volume,
on the NOR flash, the kernel thread ubi_bgt1d calls
cfi_amdstd_write_buffers() --> do_write_buffer() -->
INVALIDATE_CACHE_UDELAY --> __udelay(). Its __udelay() that calls
smp_processor_id() in preemptible code, which you are not supposed to.
Fix the problem by disabling preemption.

The kernel error messages seen when trying to create UBI volume is
BUG: using smp_processor_id() in preemptible [00000000] code: ubi_bgt1d/843
caller is __udelay+0x14/0x70
Call Trace:
[<ffffffff8110b0d4>] dump_stack+0x8/0x34
[<ffffffff812ee1ac>] debug_smp_processor_id+0x114/0x130
[<ffffffff812e9274>] __udelay+0x14/0x70
[<ffffffff81337c0c>] cfi_amdstd_write_buffers+0xa9c/0xd70
[<ffffffff8134cab0>] ubi_io_sync_erase+0x248/0x390
[<ffffffff8134d714>] erase_worker+0x6c/0x4f8
[<ffffffff8134e4fc>] do_work+0xac/0x138
[<ffffffff8134e6a0>] ubi_thread+0x118/0x1a8
[<ffffffff8115ebe0>] kthread+0x88/0x90
[<ffffffff81115650>] kernel_thread_helper+0x10/0x18

Signed-off-by: Philby John <pjohn@mvista.com>
---
 include/linux/mtd/cfi.h |    2 ++
 1 files changed, 2 insertions(+), 0 deletions(-)
Jamie Lokier - June 14, 2010, 3:04 p.m.
Philby John wrote:
> mtd: Fix bug using smp_processor_id() in preemptible ubi_bgt1d kthread
> 
> On a MIPS Cavium Octeon CN5020 when trying to create a UBI volume,
> on the NOR flash, the kernel thread ubi_bgt1d calls
> cfi_amdstd_write_buffers() --> do_write_buffer() -->
> INVALIDATE_CACHE_UDELAY --> __udelay(). Its __udelay() that calls
> smp_processor_id() in preemptible code, which you are not supposed to.
> Fix the problem by disabling preemption.

The MTD code just calls udelay().
Are you sure it isn't permitted to call udelay() from preemptible code?
I think it is fine.

Perhaps MIPS udelay() should be disabling preemption itself, or
(as x86 does) using raw_smp_processor_id() instead?  Or perhaps the x86
version is a bug because the current CPU might change during the delay loop?

See git commit 5c1ea08215f1f830dfaf4819a5f22efca41c3832
"x86: enable preemption in delay"

I don't think it makes sense to disable preemption in all udelay()
calls in drivers, so my NAK to this MTD patch.  To workaround,
consider putting the preempt_disable in MIPS udelay(), or using
raw_smp_processor_id() in it, after reading the above git commit's
message.  A proper fix would accept a context switch during the delay
and rescale the remaining count, but even on x86 they haven't done
that yet :-)

Regards,
-- Jamie
Philby John - June 14, 2010, 3:37 p.m.
On Mon, 2010-06-14 at 16:04 +0100, Jamie Lokier wrote:
> Philby John wrote:
> > mtd: Fix bug using smp_processor_id() in preemptible ubi_bgt1d kthread
> > 
> > On a MIPS Cavium Octeon CN5020 when trying to create a UBI volume,
> > on the NOR flash, the kernel thread ubi_bgt1d calls
> > cfi_amdstd_write_buffers() --> do_write_buffer() -->
> > INVALIDATE_CACHE_UDELAY --> __udelay(). Its __udelay() that calls
> > smp_processor_id() in preemptible code, which you are not supposed to.
> > Fix the problem by disabling preemption.
> 
> The MTD code just calls udelay().
> Are you sure it isn't permitted to call udelay() from preemptible code?
> I think it is fine.


The mips code uses __udelay() where the macro current_cpu_data returns
the actual data structure on a per CPU basis by calling
smp_processor_id(). Since I have enabled CONFIG_DEBUG_PREEMPT, this
would call debug_smp_processor_id(). This function would check

a)if the thread is preemptiable. If preemption is disabled, normal flow.
b)If irqs are disabled, if yes normal flow.
c)if the thread is bound to a single cpu, if yes normal flow
d)or if its an early bootup

None of these condition get satisfied and hence the kernel error
messages are seen. So I think yes for MIPS, udelay() shouldn't be called
in preemptiable code.

> 
> Perhaps MIPS udelay() should be disabling preemption itself,

I will need to investigate this. Will follow up soon.

>  or
> (as x86 does) using raw_smp_processor_id() instead?

I have enabled CONFIG_DEBUG_PREEMPT so this would call
debug_smp_processor_id() instead of raw_smp_processor_id().

>   Or perhaps the x86
> version is a bug because the current CPU might change during the delay loop?
> 

Yes, isn't this a possibility? In that case shouldn't we be using
spin_lock_irqsave() ?

> See git commit 5c1ea08215f1f830dfaf4819a5f22efca41c3832
> "x86: enable preemption in delay"
> 
> I don't think it makes sense to disable preemption in all udelay()
> calls in drivers, so my NAK to this MTD patch.  To workaround,
> consider putting the preempt_disable in MIPS udelay(),

This would definitely work.

>  or using
> raw_smp_processor_id() in it, after reading the above git commit's
> message.

Will look into this.

Thanks
Philby
Philby John - June 14, 2010, 4:40 p.m.
On Mon, 2010-06-14 at 16:04 +0100, Jamie Lokier wrote:
> Philby John wrote:
> > mtd: Fix bug using smp_processor_id() in preemptible ubi_bgt1d kthread
> > 
> > On a MIPS Cavium Octeon CN5020 when trying to create a UBI volume,
> > on the NOR flash, the kernel thread ubi_bgt1d calls
> > cfi_amdstd_write_buffers() --> do_write_buffer() -->
> > INVALIDATE_CACHE_UDELAY --> __udelay(). Its __udelay() that calls
> > smp_processor_id() in preemptible code, which you are not supposed to.
> > Fix the problem by disabling preemption.
> 
> The MTD code just calls udelay().
> Are you sure it isn't permitted to call udelay() from preemptible code?
> I think it is fine.

It isn't really udelay() but smp_processor_id() that you are not to call
from a preemptible thread. Now I also see Ed Swierk has done a similar
thing https://patchwork.kernel.org/patch/4049/ and he comments "..which
calls smp_processor_id(), which is not supposed to be called from a
preemptible thread."


So perhaps I can use preempt_disable() around just this call in function
__udelay()?

Regards,
Philby
Philby John - June 15, 2010, 12:26 p.m.
Hello Jamie,

On Mon, 2010-06-14 at 16:04 +0100, Jamie Lokier wrote:
> Philby John wrote:
> > mtd: Fix bug using smp_processor_id() in preemptible ubi_bgt1d kthread
> > 
> > On a MIPS Cavium Octeon CN5020 when trying to create a UBI volume,
> > on the NOR flash, the kernel thread ubi_bgt1d calls
> > cfi_amdstd_write_buffers() --> do_write_buffer() -->
> > INVALIDATE_CACHE_UDELAY --> __udelay(). Its __udelay() that calls
> > smp_processor_id() in preemptible code, which you are not supposed to.
> > Fix the problem by disabling preemption.
> 
> The MTD code just calls udelay().
> Are you sure it isn't permitted to call udelay() from preemptible code?
> I think it is fine.
> 
> Perhaps MIPS udelay() should be disabling preemption itself, or
> (as x86 does) using raw_smp_processor_id() instead?

Sorry for the noise. I now find that raw_smp_processor_id() has been
implemented specific to MIPS in the latest kernel, I was using 2.6.32.

Thanks and regards,
Philby

Patch

diff --git a/include/linux/mtd/cfi.h b/include/linux/mtd/cfi.h
index 574d9ee..9673213 100644
--- a/include/linux/mtd/cfi.h
+++ b/include/linux/mtd/cfi.h
@@ -495,7 +495,9 @@  static inline void cfi_udelay(int us)
 	if (us >= 1000) {
 		msleep((us+999)/1000);
 	} else {
+		preempt_disable();
 		udelay(us);
+		preempt_enable();
 		cond_resched();
 	}
 }