| Message ID | 20160114145007.GC17776@linutronix.de |
|---|---|
| State | Not Applicable, archived |
| Delegated to: | David Miller |
| Headers | show |
On 14.01.2016 15:50, Sebastian Andrzej Siewior wrote: > * Thomas Gleixner | 2016-01-13 18:31:46 [+0100]: > >> On Wed, 13 Jan 2016, Sebastian Andrzej Siewior wrote: >>> +#ifdef CONFIG_PREEMPT_RT_FULL >>> +static inline int dev_recursion_level(void) >>> +{ >>> + return atomic_read(¤t->xmit_recursion); >> >> Why would you need an atomic here. current does hardly race against itself. > > right. We are just adding a second recursion limit solely to openvswitch which has the same problem: https://patchwork.ozlabs.org/patch/566769/ This time also we depend on rcu_read_lock marking the section being nonpreemptible. Nice would be a more generic solution here which doesn't need to always add something to *current. Thanks, Hannes
On Thu, 2016-01-14 at 23:02 +0100, Hannes Frederic Sowa wrote: > We are just adding a second recursion limit solely to openvswitch which > has the same problem: > > https://patchwork.ozlabs.org/patch/566769/ > > This time also we depend on rcu_read_lock marking the section being > nonpreemptible. Nice would be a more generic solution here which doesn't > need to always add something to *current. Note that rcu_read_lock() does not imply that preemption is disabled.
On 14.01.2016 23:20, Eric Dumazet wrote: > On Thu, 2016-01-14 at 23:02 +0100, Hannes Frederic Sowa wrote: > >> We are just adding a second recursion limit solely to openvswitch which >> has the same problem: >> >> https://patchwork.ozlabs.org/patch/566769/ >> >> This time also we depend on rcu_read_lock marking the section being >> nonpreemptible. Nice would be a more generic solution here which doesn't >> need to always add something to *current. > > > Note that rcu_read_lock() does not imply that preemption is disabled. Exactly, it is conditional on CONFIG_PREEMPT_CPU/CONFIG_PREMPT_COUNT but haven't thought about exactly that in this moment. I will resend this patch with better protection. Thanks Eric!
On Fri, 15 Jan 2016, Hannes Frederic Sowa wrote: > On 14.01.2016 23:20, Eric Dumazet wrote: > > On Thu, 2016-01-14 at 23:02 +0100, Hannes Frederic Sowa wrote: > > > > > We are just adding a second recursion limit solely to openvswitch which > > > has the same problem: > > > > > > https://patchwork.ozlabs.org/patch/566769/ > > > > > > This time also we depend on rcu_read_lock marking the section being > > > nonpreemptible. Nice would be a more generic solution here which doesn't > > > need to always add something to *current. > > > > > > Note that rcu_read_lock() does not imply that preemption is disabled. > > Exactly, it is conditional on CONFIG_PREEMPT_CPU/CONFIG_PREMPT_COUNT but > haven't thought about exactly that in this moment. Wrong. CONFIG_PREEMPT_RCU makes RCU preemptible. If that is not set then it fiddles with preempt_count when CONFIG_PREEMPT_COUNT=y. If CONFIG_PREEMPT_COUNT=n then you have a non preemptible system anyway. So you cannot assume that rcu_read_lock() disables preemption. Thanks, tglx
On 15.01.2016 09:21, Thomas Gleixner wrote: > On Fri, 15 Jan 2016, Hannes Frederic Sowa wrote: >> On 14.01.2016 23:20, Eric Dumazet wrote: >>> On Thu, 2016-01-14 at 23:02 +0100, Hannes Frederic Sowa wrote: >>> >>>> We are just adding a second recursion limit solely to openvswitch which >>>> has the same problem: >>>> >>>> https://patchwork.ozlabs.org/patch/566769/ >>>> >>>> This time also we depend on rcu_read_lock marking the section being >>>> nonpreemptible. Nice would be a more generic solution here which doesn't >>>> need to always add something to *current. >>> >>> >>> Note that rcu_read_lock() does not imply that preemption is disabled. >> >> Exactly, it is conditional on CONFIG_PREEMPT_CPU/CONFIG_PREMPT_COUNT but >> haven't thought about exactly that in this moment. > > Wrong. CONFIG_PREEMPT_RCU makes RCU preemptible. > > If that is not set then it fiddles with preempt_count when > CONFIG_PREEMPT_COUNT=y. If CONFIG_PREEMPT_COUNT=n then you have a non > preemptible system anyway. > > So you cannot assume that rcu_read_lock() disables preemption. Sorry for maybe writing it misleading but that is exactly what I wanted to say here. Yes, I agree, I didn't really check because of _bh and rcu_read_lock. This was a mistake. ;) I already send out an updated patch with added preemption guards. Thanks, Hannes
--- a/include/linux/netdevice.h +++ b/include/linux/netdevice.h @@ -2249,11 +2249,20 @@ void netdev_freemem(struct net_device *d void synchronize_net(void); int init_dummy_netdev(struct net_device *dev); +#ifdef CONFIG_PREEMPT_RT_FULL +static inline int dev_recursion_level(void) +{ + return current->xmit_recursion; +} + +#else + DECLARE_PER_CPU(int, xmit_recursion); static inline int dev_recursion_level(void) { return this_cpu_read(xmit_recursion); } +#endif struct net_device *dev_get_by_index(struct net *net, int ifindex); struct net_device *__dev_get_by_index(struct net *net, int ifindex); --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -1851,6 +1851,9 @@ struct task_struct { #ifdef CONFIG_DEBUG_ATOMIC_SLEEP unsigned long task_state_change; #endif +#ifdef CONFIG_PREEMPT_RT_FULL + int xmit_recursion; +#endif int pagefault_disabled; /* CPU-specific state of this task */ struct thread_struct thread; --- a/net/core/dev.c +++ b/net/core/dev.c @@ -2940,9 +2940,44 @@ static void skb_update_prio(struct sk_bu #define skb_update_prio(skb) #endif +#ifdef CONFIG_PREEMPT_RT_FULL + +static inline int xmit_rec_read(void) +{ + return current->xmit_recursion; +} + +static inline void xmit_rec_inc(void) +{ + current->xmit_recursion++; +} + +static inline void xmit_rec_dec(void) +{ + current->xmit_recursion--; +} + +#else + DEFINE_PER_CPU(int, xmit_recursion); EXPORT_SYMBOL(xmit_recursion); +static inline int xmit_rec_read(void) +{ + return __this_cpu_read(xmit_recursion); +} + +static inline void xmit_rec_inc(void) +{ + __this_cpu_inc(xmit_recursion); +} + +static inline int xmit_rec_dec(void) +{ + __this_cpu_dec(xmit_recursion); +} +#endif + #define RECURSION_LIMIT 10 /** @@ -3135,7 +3170,7 @@ static int __dev_queue_xmit(struct sk_bu if (txq->xmit_lock_owner != cpu) { - if (__this_cpu_read(xmit_recursion) > RECURSION_LIMIT) + if (xmit_rec_read() > RECURSION_LIMIT) goto recursion_alert; skb = validate_xmit_skb(skb, dev); @@ -3145,9 +3180,9 @@ static int __dev_queue_xmit(struct sk_bu HARD_TX_LOCK(dev, txq, cpu); if (!netif_xmit_stopped(txq)) { - __this_cpu_inc(xmit_recursion); + xmit_rec_inc(); skb = dev_hard_start_xmit(skb, dev, txq, &rc); - __this_cpu_dec(xmit_recursion); + xmit_rec_dec(); if (dev_xmit_complete(rc)) { HARD_TX_UNLOCK(dev, txq); goto out;