| Message ID | PU1P153MB01692432D92B3A726FBDDC68BF300@PU1P153MB0169.APCP153.PROD.OUTLOOK.COM |
|---|---|
| State | Changes Requested, archived |
| Delegated to: | David Miller |
| Headers | show |
| Series | hv_netvsc: Fix a deadlock by getting rtnl_lock earlier in netvsc_probe() | expand |
From: Dexuan Cui <decui@microsoft.com> Date: Wed, 22 Aug 2018 21:20:03 +0000 > --- > drivers/net/hyperv/netvsc_drv.c | 11 ++++++++++- > 1 file changed, 10 insertions(+), 1 deletion(-) > > > FYI: these are the related 3 paths which show the deadlock: This incredibly useful information belongs in the commit log message, and therefore before the --- and signoffs.
> From: David Miller <davem@davemloft.net> > Sent: Wednesday, August 29, 2018 17:49 > > From: Dexuan Cui <decui@microsoft.com> > Date: Wed, 22 Aug 2018 21:20:03 +0000 > > > --- > > drivers/net/hyperv/netvsc_drv.c | 11 ++++++++++- > > 1 file changed, 10 insertions(+), 1 deletion(-) > > > > > > FYI: these are the related 3 paths which show the deadlock: > > This incredibly useful information belongs in the commit log > message, and therefore before the --- and signoffs. Hi David, I was afraid the call-traces are too detailed. :-) Can you please move the info to before the --- line? Or, should I resend the patch with the commit log updated? Thanks, -- Dexuan
From: Dexuan Cui <decui@microsoft.com> Date: Thu, 30 Aug 2018 00:58:48 +0000 >> From: David Miller <davem@davemloft.net> >> Sent: Wednesday, August 29, 2018 17:49 >> >> From: Dexuan Cui <decui@microsoft.com> >> Date: Wed, 22 Aug 2018 21:20:03 +0000 >> >> > --- >> > drivers/net/hyperv/netvsc_drv.c | 11 ++++++++++- >> > 1 file changed, 10 insertions(+), 1 deletion(-) >> > >> > >> > FYI: these are the related 3 paths which show the deadlock: >> >> This incredibly useful information belongs in the commit log >> message, and therefore before the --- and signoffs. > > Hi David, > I was afraid the call-traces are too detailed. :-) > > Can you please move the info to before the --- line? > > Or, should I resend the patch with the commit log updated? Please resend.
> From: David Miller <davem@davemloft.net> > Sent: Wednesday, August 29, 2018 19:09 > > Hi David, > > I was afraid the call-traces are too detailed. :-) > > > > Can you please move the info to before the --- line? > > > > Or, should I resend the patch with the commit log updated? > > Please resend. OK. Will do. -- Dexuan
diff --git a/drivers/net/hyperv/netvsc_drv.c b/drivers/net/hyperv/netvsc_drv.c index 1121a1ec..4fd14a0 100644 --- a/drivers/net/hyperv/netvsc_drv.c +++ b/drivers/net/hyperv/netvsc_drv.c @@ -2206,6 +2206,16 @@ static int netvsc_probe(struct hv_device *dev, memcpy(net->dev_addr, device_info.mac_adr, ETH_ALEN); + /* We must get rtnl_lock before scheduling nvdev->subchan_work, + * otherwise netvsc_subchan_work() can get rtnl_lock first and wait + * all subchannels to show up, but that may not happen because + * netvsc_probe() can't get rtnl_lock and as a result vmbus_onoffer() + * -> ... -> device_add() -> ... -> __device_attach() can't get + * the device lock, so all the subchannels can't be processed -- + * finally netvsc_subchan_work() hangs for ever. + */ + rtnl_lock(); + if (nvdev->num_chn > 1) schedule_work(&nvdev->subchan_work); @@ -2224,7 +2234,6 @@ static int netvsc_probe(struct hv_device *dev, else net->max_mtu = ETH_DATA_LEN; - rtnl_lock(); ret = register_netdevice(net); if (ret != 0) { pr_err("Unable to register netdev.\n");
This patch fixes the race between netvsc_probe() and rndis_set_subchannel(), which can cause a deadlock. Fixes: 8195b1396ec8 ("hv_netvsc: fix deadlock on hotplug") Signed-off-by: Dexuan Cui <decui@microsoft.com> Cc: Stephen Hemminger <sthemmin@microsoft.com> Cc: K. Y. Srinivasan <kys@microsoft.com> Cc: Haiyang Zhang <haiyangz@microsoft.com> --- drivers/net/hyperv/netvsc_drv.c | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) FYI: these are the related 3 paths which show the deadlock: #1: Workqueue: hv_vmbus_con vmbus_onmessage_work [hv_vmbus] Call Trace: schedule schedule_preempt_disabled __mutex_lock __device_attach bus_probe_device device_add vmbus_device_register vmbus_onoffer vmbus_onmessage_work process_one_work worker_thread kthread ret_from_fork #2: schedule schedule_preempt_disabled __mutex_lock netvsc_probe vmbus_probe really_probe __driver_attach bus_for_each_dev driver_attach_async async_run_entry_fn process_one_work worker_thread kthread ret_from_fork #3: Workqueue: events netvsc_subchan_work [hv_netvsc] Call Trace: schedule rndis_set_subchannel netvsc_subchan_work process_one_work worker_thread kthread ret_from_fork Before path #1 finishes, path #2 can start to run, because just before the "bus_probe_device(dev);" in device_add() in path #1, there is a line "object_uevent(&dev->kobj, KOBJ_ADD);", so systemd-udevd can immediately try to load hv_netvsc and hence path #2 can start to run.