| Submitter | Daniel Hellstrom |
|---|---|
| Date | Feb. 2, 2011, 7:40 p.m. |
| Message ID | <1296675656-19165-1-git-send-email-daniel@gaisler.com> |
| Download | mbox | patch |
| Permalink | /patch/81524/ |
| State | Changes Requested |
| Delegated to: | David Miller |
| Headers | show |
Comments
From: Daniel Hellstrom <daniel@gaisler.com> Date: Wed, 2 Feb 2011 20:40:55 +0100 > The current sparc32 SMP IPI generation is implemented the > cross call function. The cross call function uses IRQ15 the > NMI, this is has the effect that IPIs will interrupt IRQ > critical areas and hang the system. Typically on/after > spin_lock_irqsave calls can be aborted. > > The cross call functionality must still exist to flush > cache/TLBS. > > This patch provides CPU models a custom way to implement > generation of IPIs on the generic code's request. The > typical approach is to generate an IRQ for each IPI case. > > After this patch each sparc32 SMP CPU model needs to > implement IPIs in order to function properly. > > Signed-off-by: Daniel Hellstrom <daniel@gaisler.com> Ok, this looks mostly fine to me. Once we have the sun4m/sun4d versions written I can think about applying this. We might want to abstract out the common code of the dispatch and service stuff you have for LEON, otherwise we'll have 3 copies of that code that sets and clears the mask state. Perhaps something using BTFIXUP or similar. -- To unsubscribe from this list: send the line "unsubscribe sparclinux" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
David Miller wrote: >From: Daniel Hellstrom <daniel@gaisler.com> >Date: Wed, 2 Feb 2011 20:40:55 +0100 > > > >>The current sparc32 SMP IPI generation is implemented the >>cross call function. The cross call function uses IRQ15 the >>NMI, this is has the effect that IPIs will interrupt IRQ >>critical areas and hang the system. Typically on/after >>spin_lock_irqsave calls can be aborted. >> >>The cross call functionality must still exist to flush >>cache/TLBS. >> >>This patch provides CPU models a custom way to implement >>generation of IPIs on the generic code's request. The >>typical approach is to generate an IRQ for each IPI case. >> >>After this patch each sparc32 SMP CPU model needs to >>implement IPIs in order to function properly. >> >>Signed-off-by: Daniel Hellstrom <daniel@gaisler.com> >> >> > >Ok, this looks mostly fine to me. > >Once we have the sun4m/sun4d versions written I can think about >applying this. > >We might want to abstract out the common code of the dispatch and >service stuff you have for LEON, otherwise we'll have 3 copies of >that code that sets and clears the mask state. > >Perhaps something using BTFIXUP or similar. > > I was thinking so too, BTFIXUPs for CPU model specific "IPI operations" to avoid the switch-statement in the arch_ functions. Daniel -- To unsubscribe from this list: send the line "unsubscribe sparclinux" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
From: Daniel Hellstrom <daniel@gaisler.com> Date: Thu, 03 Feb 2011 09:50:05 +0100 > David Miller wrote: > >>Ok, this looks mostly fine to me. >> >>Once we have the sun4m/sun4d versions written I can think about >>applying this. >> >>We might want to abstract out the common code of the dispatch and >>service stuff you have for LEON, otherwise we'll have 3 copies of >>that code that sets and clears the mask state. >> >>Perhaps something using BTFIXUP or similar. >> > I was thinking so too, BTFIXUPs for CPU model specific "IPI > operations" to avoid the switch-statement in the arch_ functions. Exactly. -- To unsubscribe from this list: send the line "unsubscribe sparclinux" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
David Miller wrote: >From: Daniel Hellstrom <daniel@gaisler.com> >Date: Thu, 03 Feb 2011 09:50:05 +0100 > > > >>David Miller wrote: >> >> >> >>>Ok, this looks mostly fine to me. >>> >>>Once we have the sun4m/sun4d versions written I can think about >>>applying this. >>> >>>We might want to abstract out the common code of the dispatch and >>>service stuff you have for LEON, otherwise we'll have 3 copies of >>>that code that sets and clears the mask state. >>> >>>Perhaps something using BTFIXUP or similar. >>> >>> >>> >>I was thinking so too, BTFIXUPs for CPU model specific "IPI >>operations" to avoid the switch-statement in the arch_ functions. >> >> > >Exactly. > > I have made new IPI patches that uses BTFIXUPs as we discussed above. From another discussion we had: David Miller wrote: >I think we can make this work on sun4m/sun4c/sun4d, which have several >software interrupt vectors available. > >sun4m has 15 soft interrupts, in the sun4m_irq_percpu->{pending,clear,set} >registers, these live sequentially starting at bit 16, as per the definition >of the SUN4M_SOFT_INT() macro. > >The only catch is that we'll need to peek at the ->pending register(s) to >determine if we have a hardware or software interrupt pending at a given >PIL level (or both). > > I have made an implementation for sun4m which I think will work, I added code to the IRQ traphandler to check for the IPIs as you suggested above and clearing the pending register. The previous sun4m code had some IPI checks already which I beleive was old unused code that was never called, so I removed that.. Do you have any ideas what was going on in the old code (smp_reschedule_irq)? Please see patch and commit message. I don't have the sun4d-SMP hardware so it would be great if someone could test the patches? >I'm not sure how the software interrupt triggering works on sun4c, but that >doesn't matter since we only need this on SMP. > > I skip sun4c in this patch. >The sun4d code has a sun4d_send_ipi() interface from which the necessary >code can be constructed. > > I started making a sun4d patch however I don't know the hardware enough to make a guess. My problem is that I think I can generate an IPI but don't know how to ACK them, detect them, which IRQs to use etc. Is there a way to distinguish between soft and hard IRQs on sun4d? If I had a piece of document about sun4d it could be possible, but guessing from looking at unimplemented code... I hope someone else can take over here, probably it is very similar to the sun4m implementation. Also, this time I added some IPI IRQ statistics to the genirq arch_show_interrupts() function, I hope that it may help someone trying to implement IPIs on sun4d. After booting on a LEON4 dualcore board: # cat /proc/interrupts CPU0 CPU1 2: 2808 0 leon-edge timer 3: 2809 2741 leon-per-cpu ticker 5: 198 0 leon-edge apbuart 13: 0 0 leon-pcilvl GRPCI2_JUMP 20: 0 0 grpci2-pcilvl GRPCI2_ERR RES: 29 6 IPI rescheduling interrupts CAL: 35 32 IPI function call interrupts NMI: 1 1 Non-maskable interrupts Daniel -- To unsubscribe from this list: send the line "unsubscribe sparclinux" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Patch
diff --git a/arch/sparc/Kconfig b/arch/sparc/Kconfig index e48f471..cef291e 100644 --- a/arch/sparc/Kconfig +++ b/arch/sparc/Kconfig @@ -25,6 +25,7 @@ config SPARC select HAVE_DMA_ATTRS select HAVE_DMA_API_DEBUG select HAVE_ARCH_JUMP_LABEL + select USE_GENERIC_SMP_HELPERS if SMP config SPARC32 def_bool !64BIT @@ -43,7 +44,6 @@ config SPARC64 select HAVE_DYNAMIC_FTRACE select HAVE_FTRACE_MCOUNT_RECORD select HAVE_SYSCALL_TRACEPOINTS - select USE_GENERIC_SMP_HELPERS if SMP select RTC_DRV_CMOS select RTC_DRV_BQ4802 select RTC_DRV_SUN4V diff --git a/arch/sparc/include/asm/smp_32.h b/arch/sparc/include/asm/smp_32.h index d82d7f4..8390ef6 100644 --- a/arch/sparc/include/asm/smp_32.h +++ b/arch/sparc/include/asm/smp_32.h @@ -73,19 +73,8 @@ static inline void xc4(smpfunc_t func, unsigned long arg1, unsigned long arg2, unsigned long arg3, unsigned long arg4) { smp_cross_call(func, cpu_online_map, arg1, arg2, arg3, arg4); } -static inline int smp_call_function(void (*func)(void *info), void *info, int wait) -{ - xc1((smpfunc_t)func, (unsigned long)info); - return 0; -} - -static inline int smp_call_function_single(int cpuid, void (*func) (void *info), - void *info, int wait) -{ - smp_cross_call((smpfunc_t)func, cpumask_of_cpu(cpuid), - (unsigned long) info, 0, 0, 0); - return 0; -} +extern void arch_send_call_function_single_ipi(int cpu); +extern void arch_send_call_function_ipi_mask(const struct cpumask *mask); static inline int cpu_logical_map(int cpu) { diff --git a/arch/sparc/kernel/smp_32.c b/arch/sparc/kernel/smp_32.c index 91c10fb..a1bb3a8 100644 --- a/arch/sparc/kernel/smp_32.c +++ b/arch/sparc/kernel/smp_32.c @@ -125,13 +125,45 @@ struct linux_prom_registers smp_penguin_ctable __cpuinitdata = { 0 }; void smp_send_reschedule(int cpu) { - /* See sparc64 */ + /* + * CPU model dependent way of implementing IPI generation targeting + * a single CPU. The trap handler needs only to do trap entry/return + * to call schedule. + */ + switch (sparc_cpu_model) { + default: + BUG(); + } } void smp_send_stop(void) { } +void arch_send_call_function_single_ipi(int cpu) +{ + /* + * CPU model dependent way of implementing IPI generation targeting + * a single CPU + */ + switch (sparc_cpu_model) { + default: + BUG(); + } +} + +void arch_send_call_function_ipi_mask(const struct cpumask *mask) +{ + /* + * CPU model dependent way of implementing IPI generation targeting + * a set of CPUs + */ + switch (sparc_cpu_model) { + default: + BUG(); + } +} + void smp_flush_cache_all(void) { xc0((smpfunc_t) BTFIXUP_CALL(local_flush_cache_all));
The current sparc32 SMP IPI generation is implemented the cross call function. The cross call function uses IRQ15 the NMI, this is has the effect that IPIs will interrupt IRQ critical areas and hang the system. Typically on/after spin_lock_irqsave calls can be aborted. The cross call functionality must still exist to flush cache/TLBS. This patch provides CPU models a custom way to implement generation of IPIs on the generic code's request. The typical approach is to generate an IRQ for each IPI case. After this patch each sparc32 SMP CPU model needs to implement IPIs in order to function properly. Signed-off-by: Daniel Hellstrom <daniel@gaisler.com> --- arch/sparc/Kconfig | 2 +- arch/sparc/include/asm/smp_32.h | 15 ++------------- arch/sparc/kernel/smp_32.c | 34 +++++++++++++++++++++++++++++++++- 3 files changed, 36 insertions(+), 15 deletions(-)