Message ID | 20171016142828.2742-1-jiri@resnulli.us |
---|---|
State | Accepted, archived |
Delegated to: | David Miller |
Headers | show |
Series | [net] mlxsw: core: Fix possible deadlock | expand |
From: Jiri Pirko <jiri@resnulli.us> Date: Mon, 16 Oct 2017 16:28:28 +0200 > From: Ido Schimmel <idosch@mellanox.com> > > When an EMAD is transmitted, a timeout work item is scheduled with a > delay of 200ms, so that another EMAD will be retried until a maximum of > five retries. > > In certain situations, it's possible for the function waiting on the > EMAD to be associated with a work item that is queued on the same > workqueue (`mlxsw_core`) as the timeout work item. This results in > flushing a work item on the same workqueue. > > According to commit e159489baa71 ("workqueue: relax lockdep annotation > on flush_work()") the above may lead to a deadlock in case the workqueue > has only one worker active or if the system in under memory pressure and > the rescue worker is in use. The latter explains the very rare and > random nature of the lockdep splats we have been seeing: ... > Fix this by creating another workqueue for EMAD timeouts, thereby > preventing the situation of a work item trying to flush a work item > queued on the same workqueue. > > Fixes: caf7297e7ab5f ("mlxsw: core: Introduce support for asynchronous EMAD register access") > Signed-off-by: Ido Schimmel <idosch@mellanox.com> > Reported-by: Jiri Pirko <jiri@mellanox.com> > Signed-off-by: Jiri Pirko <jiri@mellanox.com> Applied.
diff --git a/drivers/net/ethernet/mellanox/mlxsw/core.c b/drivers/net/ethernet/mellanox/mlxsw/core.c index 9d5e7cf..f3315bc 100644 --- a/drivers/net/ethernet/mellanox/mlxsw/core.c +++ b/drivers/net/ethernet/mellanox/mlxsw/core.c @@ -96,6 +96,7 @@ struct mlxsw_core { const struct mlxsw_bus *bus; void *bus_priv; const struct mlxsw_bus_info *bus_info; + struct workqueue_struct *emad_wq; struct list_head rx_listener_list; struct list_head event_listener_list; struct { @@ -465,7 +466,7 @@ static void mlxsw_emad_trans_timeout_schedule(struct mlxsw_reg_trans *trans) { unsigned long timeout = msecs_to_jiffies(MLXSW_EMAD_TIMEOUT_MS); - mlxsw_core_schedule_dw(&trans->timeout_dw, timeout); + queue_delayed_work(trans->core->emad_wq, &trans->timeout_dw, timeout); } static int mlxsw_emad_transmit(struct mlxsw_core *mlxsw_core, @@ -587,12 +588,18 @@ static const struct mlxsw_listener mlxsw_emad_rx_listener = static int mlxsw_emad_init(struct mlxsw_core *mlxsw_core) { + struct workqueue_struct *emad_wq; u64 tid; int err; if (!(mlxsw_core->bus->features & MLXSW_BUS_F_TXRX)) return 0; + emad_wq = alloc_workqueue("mlxsw_core_emad", WQ_MEM_RECLAIM, 0); + if (!emad_wq) + return -ENOMEM; + mlxsw_core->emad_wq = emad_wq; + /* Set the upper 32 bits of the transaction ID field to a random * number. This allows us to discard EMADs addressed to other * devices. @@ -619,6 +626,7 @@ static int mlxsw_emad_init(struct mlxsw_core *mlxsw_core) err_emad_trap_set: mlxsw_core_trap_unregister(mlxsw_core, &mlxsw_emad_rx_listener, mlxsw_core); + destroy_workqueue(mlxsw_core->emad_wq); return err; } @@ -631,6 +639,7 @@ static void mlxsw_emad_fini(struct mlxsw_core *mlxsw_core) mlxsw_core->emad.use_emad = false; mlxsw_core_trap_unregister(mlxsw_core, &mlxsw_emad_rx_listener, mlxsw_core); + destroy_workqueue(mlxsw_core->emad_wq); } static struct sk_buff *mlxsw_emad_alloc(const struct mlxsw_core *mlxsw_core,