| Message ID | 20110601163628.GA2418@osiris.boeblingen.de.ibm.com |
|---|---|
| State | RFC, archived |
| Delegated to: | David Miller |
| Headers | show |
Le mercredi 01 juin 2011 à 18:36 +0200, Heiko Carstens a écrit : > On Wed, Jun 01, 2011 at 02:13:19PM +0200, Eric Dumazet wrote: > > Le mercredi 01 juin 2011 à 12:33 +0200, Frank Blaschka a écrit : > > > Hi Dave, Eric, > > > > > > during heavy network load we turn off/on cpus. > > > Sometimes this causes a stall on the network device. > > > Digging into the dump I found out following: > > > > > > napi is scheduled but does not run. From the I/O buffers > > > and the napi state I see napi/rx_softirq processing has stopped > > > because the budget was reached. napi stays in the > > > softnet_data poll_list and the rx_softirq was raised again. > > > > > > I assume at this time the cpu offline comes in. > > > the rx softirq is raised/moved to another cpu but napi stays in the poll_list > > > of the softnet_data of the now offline cpu. > > > > > > reviewing dev_cpu_callback (net/core/dev.c) I did not find the poll_list > > > is transfered to the new cpu. Do you think this could cause the stall or > > > did I miss something? > > > > > > Thx for your help. > > > > Hi Frank > > > > I believe you are right, I cant see where the poll_list transfert from > > dead cpu to online cpu is done. > > > > Do you want to prepare a patch ? > > Frank will be offline until next week. I assume the patch below would fix > the problem, however its untested. I doubt we can verify that it really > fixes the problem also until next week (public holiday tomorrow). > > diff --git a/net/core/dev.c b/net/core/dev.c > index 6561021..6d6a7cf 100644 > --- a/net/core/dev.c > +++ b/net/core/dev.c > @@ -5981,6 +5981,8 @@ static int dev_cpu_callback(struct notifier_block *nfb, > oldsd->output_queue = NULL; > oldsd->output_queue_tailp = &oldsd->output_queue; > } > + /* Append NAPI poll list from offline CPU. */ > + list_splice_init(&oldsd->poll_list, &sd->poll_list); > > raise_softirq_irqoff(NET_TX_SOFTIRQ); > local_irq_enable(); Same here, I'll be offline for the next 4 days ;) Please make sure we raise NET_RX_SOFTIRQ on new cpu if necessary. -- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Wed, Jun 01, 2011 at 06:55:21PM +0200, Eric Dumazet wrote: > > + /* Append NAPI poll list from offline CPU. */ > > + list_splice_init(&oldsd->poll_list, &sd->poll_list); > > > > raise_softirq_irqoff(NET_TX_SOFTIRQ); > > local_irq_enable(); > > Please make sure we raise NET_RX_SOFTIRQ on new cpu if necessary. Well, see two lines below the list_splice_init() call ;) -- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Le mercredi 01 juin 2011 à 20:12 +0200, Heiko Carstens a écrit : > On Wed, Jun 01, 2011 at 06:55:21PM +0200, Eric Dumazet wrote: > > > + /* Append NAPI poll list from offline CPU. */ > > > + list_splice_init(&oldsd->poll_list, &sd->poll_list); > > > > > > raise_softirq_irqoff(NET_TX_SOFTIRQ); > > > local_irq_enable(); > > > > Please make sure we raise NET_RX_SOFTIRQ on new cpu if necessary. > > Well, see two lines below the list_splice_init() call ;) I see nothing... NET_TX_SOFTIRQ and NET_RX_SOFTIRQ are not the same -- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
diff --git a/net/core/dev.c b/net/core/dev.c index 6561021..6d6a7cf 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -5981,6 +5981,8 @@ static int dev_cpu_callback(struct notifier_block *nfb, oldsd->output_queue = NULL; oldsd->output_queue_tailp = &oldsd->output_queue; } + /* Append NAPI poll list from offline CPU. */ + list_splice_init(&oldsd->poll_list, &sd->poll_list); raise_softirq_irqoff(NET_TX_SOFTIRQ); local_irq_enable();