Patchwork [011/102] IPoIB: Fix send lockup due to missed TX completion

mail settings
Submitter Luis Henriques
Date April 8, 2013, 9:49 a.m.
Message ID <>
Download mbox | patch
Permalink /patch/234627/
State New
Headers show


Luis Henriques - April 8, 2013, 9:49 a.m. -stable review patch.  If anyone has any objections, please let me know.


From: Mike Marciniszyn <>

commit 1ee9e2aa7b31427303466776f455d43e5e3c9275 upstream.

Commit f0dc117abdfa ("IPoIB: Fix TX queue lockup with mixed UD/CM
traffic") attempts to solve an issue where unprocessed UD send
completions can deadlock the netdev.

The patch doesn't fully resolve the issue because if more than half
the tx_outstanding's were UD and all of the destinations are RC
reachable, arming the CQ doesn't solve the issue.

This patch uses the IB_CQ_REPORT_MISSED_EVENTS on the
ib_req_notify_cq().  If the rc is above 0, the UD send cq completion
callback is called directly to re-arm the send completion timer.

This issue is seen in very large parallel filesystem deployments
and the patch has been shown to correct the issue.

Reviewed-by: Dean Luick <>
Signed-off-by: Mike Marciniszyn <>
Signed-off-by: Roland Dreier <>
Signed-off-by: Luis Henriques <>
 drivers/infiniband/ulp/ipoib/ipoib_cm.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)


diff --git a/drivers/infiniband/ulp/ipoib/ipoib_cm.c b/drivers/infiniband/ulp/ipoib/ipoib_cm.c
index 014504d..3767853 100644
--- a/drivers/infiniband/ulp/ipoib/ipoib_cm.c
+++ b/drivers/infiniband/ulp/ipoib/ipoib_cm.c
@@ -755,9 +755,13 @@  void ipoib_cm_send(struct net_device *dev, struct sk_buff *skb, struct ipoib_cm_
 		if (++priv->tx_outstanding == ipoib_sendq_size) {
 			ipoib_dbg(priv, "TX ring 0x%x full, stopping kernel net queue\n",
-			if (ib_req_notify_cq(priv->send_cq, IB_CQ_NEXT_COMP))
-				ipoib_warn(priv, "request notify on send CQ failed\n");
+			rc = ib_req_notify_cq(priv->send_cq,
+			if (rc < 0)
+				ipoib_warn(priv, "request notify on send CQ failed\n");
+			else if (rc)
+				ipoib_send_comp_handler(priv->send_cq, dev);