diff mbox

net: rps: fix data stall after hotplug

Message ID b3ecb218932daa656a796cfa6e9e62b9.squirrel@www.codeaurora.org
State Not Applicable, archived
Delegated to: David Miller
Headers show

Commit Message

Subash Abhinov Kasiviswanathan March 19, 2015, 7:54 p.m. UTC
When RPS is enabled, IPI is triggered to enqueue the
backlog NAPI to the poll list. If the CPU is hotplugged
after the NAPI_STATE_SCHED bit is set on
enqueue_to_backlog but before the IPI is delivered
successfully, the poll list does not have the backlog
NAPI queued. As a consequence of this, dev_cpu_callback
does not clear the NAPI_STATE_SCHED bit on hotplug.
Since NAPI_STATE_SCHED is set even after the cpu comes
back up, packets get enqueued onto the input packet queue
but are never processed since the IPI will not be triggered.

This patch handles this race by unconditionally resetting
the NAPI state for the backlog NAPI on the offline CPU.

Signed-off-by: Subash Abhinov Kasiviswanathan <subashab@codeaurora.org>
---
 net/core/dev.c | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)


--
The Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
 a Linux Foundation Collaborative Project

--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Comments

Eric Dumazet March 19, 2015, 9:50 p.m. UTC | #1
On Thu, 2015-03-19 at 19:54 +0000, subashab@codeaurora.org wrote:
> When RPS is enabled, IPI is triggered to enqueue the
> backlog NAPI to the poll list. If the CPU is hotplugged
> after the NAPI_STATE_SCHED bit is set on
> enqueue_to_backlog but before the IPI is delivered
> successfully, the poll list does not have the backlog
> NAPI queued. As a consequence of this, dev_cpu_callback
> does not clear the NAPI_STATE_SCHED bit on hotplug.
> Since NAPI_STATE_SCHED is set even after the cpu comes
> back up, packets get enqueued onto the input packet queue
> but are never processed since the IPI will not be triggered.
> 
> This patch handles this race by unconditionally resetting
> the NAPI state for the backlog NAPI on the offline CPU.
> 
> Signed-off-by: Subash Abhinov Kasiviswanathan <subashab@codeaurora.org>
> ---
>  net/core/dev.c | 5 ++---
>  1 file changed, 2 insertions(+), 3 deletions(-)
> 
> diff --git a/net/core/dev.c b/net/core/dev.c
> index 6f561de..61d9579 100644
> --- a/net/core/dev.c
> +++ b/net/core/dev.c
> @@ -7119,12 +7119,11 @@ static int dev_cpu_callback(struct notifier_block
> *nfb,
>  							    poll_list);
> 
>  		list_del_init(&napi->poll_list);
> -		if (napi->poll == process_backlog)
> -			napi->state = 0;
> -		else
> +		if (napi->poll != process_backlog)
>  			____napi_schedule(sd, napi);
>  	}
> 
> +	oldsd->backlog.state = 0;
>  	raise_softirq_irqoff(NET_TX_SOFTIRQ);
>  	local_irq_enable();

Are you seeing this race on x86 ?

If IPI are not reliable on your arch, I am guessing you should fix them.

Otherwise, even without hotplug you'll have hangs.


--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
diff mbox

Patch

diff --git a/net/core/dev.c b/net/core/dev.c
index 6f561de..61d9579 100644
--- a/net/core/dev.c
+++ b/net/core/dev.c
@@ -7119,12 +7119,11 @@  static int dev_cpu_callback(struct notifier_block
*nfb,
 							    poll_list);

 		list_del_init(&napi->poll_list);
-		if (napi->poll == process_backlog)
-			napi->state = 0;
-		else
+		if (napi->poll != process_backlog)
 			____napi_schedule(sd, napi);
 	}

+	oldsd->backlog.state = 0;
 	raise_softirq_irqoff(NET_TX_SOFTIRQ);
 	local_irq_enable();