Patch Detail
get:
Show a patch.
patch:
Update a patch.
put:
Update a patch.
GET /api/patches/850296/?format=api
{ "id": 850296, "url": "http://patchwork.ozlabs.org/api/patches/850296/?format=api", "web_url": "http://patchwork.ozlabs.org/project/intel-wired-lan/patch/20171218101725.75870-1-alice.michael@intel.com/", "project": { "id": 46, "url": "http://patchwork.ozlabs.org/api/projects/46/?format=api", "name": "Intel Wired Ethernet development", "link_name": "intel-wired-lan", "list_id": "intel-wired-lan.osuosl.org", "list_email": "intel-wired-lan@osuosl.org", "web_url": "", "scm_url": "", "webscm_url": "", "list_archive_url": "", "list_archive_url_format": "", "commit_url_format": "" }, "msgid": "<20171218101725.75870-1-alice.michael@intel.com>", "list_archive_url": null, "date": "2017-12-18T10:17:25", "name": "[next,S83-V5,6/9] i40e/i40evf: Detect and recover hung queue scenario", "commit_ref": null, "pull_url": null, "state": "accepted", "archived": false, "hash": "44f69b489a2ccf2fdb4fc7fd895ce3f6884a5768", "submitter": { "id": 71123, "url": "http://patchwork.ozlabs.org/api/people/71123/?format=api", "name": "Michael, Alice", "email": "alice.michael@intel.com" }, "delegate": { "id": 68, "url": "http://patchwork.ozlabs.org/api/users/68/?format=api", "username": "jtkirshe", "first_name": "Jeff", "last_name": "Kirsher", "email": "jeffrey.t.kirsher@intel.com" }, "mbox": "http://patchwork.ozlabs.org/project/intel-wired-lan/patch/20171218101725.75870-1-alice.michael@intel.com/mbox/", "series": [ { "id": 19211, "url": "http://patchwork.ozlabs.org/api/series/19211/?format=api", "web_url": "http://patchwork.ozlabs.org/project/intel-wired-lan/list/?series=19211", "date": "2017-12-18T10:14:44", "name": "[next,S83-V5,1/9] i40e/i40evf: Enable NVMUpdate to retrieve AdminQ and add preservation flags for NVM update", "version": 1, "mbox": "http://patchwork.ozlabs.org/series/19211/mbox/" } ], "comments": "http://patchwork.ozlabs.org/api/patches/850296/comments/", "check": "pending", "checks": "http://patchwork.ozlabs.org/api/patches/850296/checks/", "tags": {}, "related": [], "headers": { "Return-Path": "<intel-wired-lan-bounces@osuosl.org>", "X-Original-To": [ "incoming@patchwork.ozlabs.org", "intel-wired-lan@lists.osuosl.org" ], "Delivered-To": [ "patchwork-incoming@bilbo.ozlabs.org", "intel-wired-lan@lists.osuosl.org" ], "Authentication-Results": "ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=osuosl.org\n\t(client-ip=140.211.166.138; helo=whitealder.osuosl.org;\n\tenvelope-from=intel-wired-lan-bounces@osuosl.org;\n\treceiver=<UNKNOWN>)", "Received": [ "from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138])\n\t(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3z0qDv1V4Pz9t3F\n\tfor <incoming@patchwork.ozlabs.org>;\n\tTue, 19 Dec 2017 05:24:15 +1100 (AEDT)", "from localhost (localhost [127.0.0.1])\n\tby whitealder.osuosl.org (Postfix) with ESMTP id 82D518849B;\n\tMon, 18 Dec 2017 18:24:13 +0000 (UTC)", "from whitealder.osuosl.org ([127.0.0.1])\n\tby localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024)\n\twith ESMTP id U6Vso+5CkTGS; Mon, 18 Dec 2017 18:24:10 +0000 (UTC)", "from ash.osuosl.org (ash.osuosl.org [140.211.166.34])\n\tby whitealder.osuosl.org (Postfix) with ESMTP id 62AA588493;\n\tMon, 18 Dec 2017 18:24:10 +0000 (UTC)", "from silver.osuosl.org (smtp3.osuosl.org [140.211.166.136])\n\tby ash.osuosl.org (Postfix) with ESMTP id ACC0F1C2092\n\tfor <intel-wired-lan@lists.osuosl.org>;\n\tMon, 18 Dec 2017 18:24:09 +0000 (UTC)", "from localhost (localhost [127.0.0.1])\n\tby silver.osuosl.org (Postfix) with ESMTP id A8FAE2FE3B\n\tfor <intel-wired-lan@lists.osuosl.org>;\n\tMon, 18 Dec 2017 18:24:09 +0000 (UTC)", "from silver.osuosl.org ([127.0.0.1])\n\tby localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024)\n\twith ESMTP id PnyAgfNC5TRi for <intel-wired-lan@lists.osuosl.org>;\n\tMon, 18 Dec 2017 18:24:08 +0000 (UTC)", "from mga02.intel.com (mga02.intel.com [134.134.136.20])\n\tby silver.osuosl.org (Postfix) with ESMTPS id CC7BB2FDFC\n\tfor <intel-wired-lan@lists.osuosl.org>;\n\tMon, 18 Dec 2017 18:24:08 +0000 (UTC)", "from orsmga004.jf.intel.com ([10.7.209.38])\n\tby orsmga101.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384;\n\t18 Dec 2017 10:24:08 -0800", "from unknown (HELO localhost.jf.intel.com) ([10.166.16.121])\n\tby orsmga004.jf.intel.com with ESMTP; 18 Dec 2017 10:24:08 -0800" ], "X-Virus-Scanned": [ "amavisd-new at osuosl.org", "amavisd-new at osuosl.org" ], "X-Greylist": "domain auto-whitelisted by SQLgrey-1.7.6", "X-Amp-Result": "SKIPPED(no attachment in message)", "X-Amp-File-Uploaded": "False", "X-ExtLoop1": "1", "X-IronPort-AV": "E=Sophos;i=\"5.45,422,1508828400\"; d=\"scan'208\";a=\"159838858\"", "From": "Alice Michael <alice.michael@intel.com>", "To": "alice.michael@intel.com,\n\tintel-wired-lan@lists.osuosl.org", "Date": "Mon, 18 Dec 2017 05:17:25 -0500", "Message-Id": "<20171218101725.75870-1-alice.michael@intel.com>", "X-Mailer": "git-send-email 2.9.5", "Subject": "[Intel-wired-lan] [next PATCH S83-V5 6/9] i40e/i40evf: Detect and\n\trecover hung queue scenario", "X-BeenThere": "intel-wired-lan@osuosl.org", "X-Mailman-Version": "2.1.24", "Precedence": "list", "List-Id": "Intel Wired Ethernet Linux Kernel Driver Development\n\t<intel-wired-lan.osuosl.org>", "List-Unsubscribe": "<https://lists.osuosl.org/mailman/options/intel-wired-lan>, \n\t<mailto:intel-wired-lan-request@osuosl.org?subject=unsubscribe>", "List-Archive": "<http://lists.osuosl.org/pipermail/intel-wired-lan/>", "List-Post": "<mailto:intel-wired-lan@osuosl.org>", "List-Help": "<mailto:intel-wired-lan-request@osuosl.org?subject=help>", "List-Subscribe": "<https://lists.osuosl.org/mailman/listinfo/intel-wired-lan>, \n\t<mailto:intel-wired-lan-request@osuosl.org?subject=subscribe>", "MIME-Version": "1.0", "Content-Type": "text/plain; charset=\"us-ascii\"", "Content-Transfer-Encoding": "7bit", "Errors-To": "intel-wired-lan-bounces@osuosl.org", "Sender": "\"Intel-wired-lan\" <intel-wired-lan-bounces@osuosl.org>" }, "content": "From: Sudheer Mogilappagari <sudheer.mogilappagari@intel.com>\n\nIn VFs, there is a known issue which can cause writebacks\nto not occur when interrupts are disabled and there are\nless than 4 descriptors resulting in TX timeout. Timeout\ncan also occur due to lost interrupt.\n\nThe current implementation for detecting and recovering\nfrom hung queues in the PF is problematic because it actually\nactively encourages lost interrupts. By triggering a SW\ninterrupt, interrupts are forced on. If we are already in\nnapi_poll and an interrupt fires, napi_poll will not be\nrescheduled and the interrupt is effectively lost; thereby\npotentially *causing* hung queues.\n\nThis patch checks whether packets are being processed between\nevery watchdog cycle and determine potential hung queue and\nfires triggers SW interrupt only for that particular queue.\n\nSigned-off-by: Sudheer Mogilappagari <sudheer.mogilappagari@intel.com>\n---\n drivers/net/ethernet/intel/i40e/i40e_main.c | 100 +-----------------------\n drivers/net/ethernet/intel/i40e/i40e_txrx.c | 54 +++++++++++++\n drivers/net/ethernet/intel/i40e/i40e_txrx.h | 2 +\n drivers/net/ethernet/intel/i40evf/i40e_txrx.c | 54 +++++++++++++\n drivers/net/ethernet/intel/i40evf/i40e_txrx.h | 2 +\n drivers/net/ethernet/intel/i40evf/i40evf_main.c | 2 +\n 6 files changed, 115 insertions(+), 99 deletions(-)", "diff": "diff --git a/drivers/net/ethernet/intel/i40e/i40e_main.c b/drivers/net/ethernet/intel/i40e/i40e_main.c\nindex 7a882d7..a53f76f 100644\n--- a/drivers/net/ethernet/intel/i40e/i40e_main.c\n+++ b/drivers/net/ethernet/intel/i40e/i40e_main.c\n@@ -4862,104 +4862,6 @@ static int i40e_pf_wait_queues_disabled(struct i40e_pf *pf)\n #endif\n \n /**\n- * i40e_detect_recover_hung_queue - Function to detect and recover hung_queue\n- * @q_idx: TX queue number\n- * @vsi: Pointer to VSI struct\n- *\n- * This function checks specified queue for given VSI. Detects hung condition.\n- * We proactively detect hung TX queues by checking if interrupts are disabled\n- * but there are pending descriptors. If it appears hung, attempt to recover\n- * by triggering a SW interrupt.\n- **/\n-static void i40e_detect_recover_hung_queue(int q_idx, struct i40e_vsi *vsi)\n-{\n-\tstruct i40e_ring *tx_ring = NULL;\n-\tstruct i40e_pf\t*pf;\n-\tu32 val, tx_pending;\n-\tint i;\n-\n-\tpf = vsi->back;\n-\n-\t/* now that we have an index, find the tx_ring struct */\n-\tfor (i = 0; i < vsi->num_queue_pairs; i++) {\n-\t\tif (vsi->tx_rings[i] && vsi->tx_rings[i]->desc) {\n-\t\t\tif (q_idx == vsi->tx_rings[i]->queue_index) {\n-\t\t\t\ttx_ring = vsi->tx_rings[i];\n-\t\t\t\tbreak;\n-\t\t\t}\n-\t\t}\n-\t}\n-\n-\tif (!tx_ring)\n-\t\treturn;\n-\n-\t/* Read interrupt register */\n-\tif (pf->flags & I40E_FLAG_MSIX_ENABLED)\n-\t\tval = rd32(&pf->hw,\n-\t\t\t I40E_PFINT_DYN_CTLN(tx_ring->q_vector->v_idx +\n-\t\t\t\t\t tx_ring->vsi->base_vector - 1));\n-\telse\n-\t\tval = rd32(&pf->hw, I40E_PFINT_DYN_CTL0);\n-\n-\ttx_pending = i40e_get_tx_pending(tx_ring);\n-\n-\t/* Interrupts are disabled and TX pending is non-zero,\n-\t * trigger the SW interrupt (don't wait). Worst case\n-\t * there will be one extra interrupt which may result\n-\t * into not cleaning any queues because queues are cleaned.\n-\t */\n-\tif (tx_pending && (!(val & I40E_PFINT_DYN_CTLN_INTENA_MASK)))\n-\t\ti40e_force_wb(vsi, tx_ring->q_vector);\n-}\n-\n-/**\n- * i40e_detect_recover_hung - Function to detect and recover hung_queues\n- * @pf: pointer to PF struct\n- *\n- * LAN VSI has netdev and netdev has TX queues. This function is to check\n- * each of those TX queues if they are hung, trigger recovery by issuing\n- * SW interrupt.\n- **/\n-static void i40e_detect_recover_hung(struct i40e_pf *pf)\n-{\n-\tstruct net_device *netdev;\n-\tstruct i40e_vsi *vsi;\n-\tunsigned int i;\n-\n-\t/* Only for LAN VSI */\n-\tvsi = pf->vsi[pf->lan_vsi];\n-\n-\tif (!vsi)\n-\t\treturn;\n-\n-\t/* Make sure, VSI state is not DOWN/RECOVERY_PENDING */\n-\tif (test_bit(__I40E_VSI_DOWN, vsi->back->state) ||\n-\t test_bit(__I40E_RESET_RECOVERY_PENDING, vsi->back->state))\n-\t\treturn;\n-\n-\t/* Make sure type is MAIN VSI */\n-\tif (vsi->type != I40E_VSI_MAIN)\n-\t\treturn;\n-\n-\tnetdev = vsi->netdev;\n-\tif (!netdev)\n-\t\treturn;\n-\n-\t/* Bail out if netif_carrier is not OK */\n-\tif (!netif_carrier_ok(netdev))\n-\t\treturn;\n-\n-\t/* Go thru' TX queues for netdev */\n-\tfor (i = 0; i < netdev->num_tx_queues; i++) {\n-\t\tstruct netdev_queue *q;\n-\n-\t\tq = netdev_get_tx_queue(netdev, i);\n-\t\tif (q)\n-\t\t\ti40e_detect_recover_hung_queue(i, vsi);\n-\t}\n-}\n-\n-/**\n * i40e_get_iscsi_tc_map - Return TC map for iSCSI APP\n * @pf: pointer to PF\n *\n@@ -9751,7 +9653,7 @@ static void i40e_service_task(struct work_struct *work)\n \tif (test_and_set_bit(__I40E_SERVICE_SCHED, pf->state))\n \t\treturn;\n \n-\ti40e_detect_recover_hung(pf);\n+\ti40e_detect_recover_hung(pf->vsi[pf->lan_vsi]);\n \ti40e_sync_filters_subtask(pf);\n \ti40e_reset_subtask(pf);\n \ti40e_handle_mdd_event(pf);\ndiff --git a/drivers/net/ethernet/intel/i40e/i40e_txrx.c b/drivers/net/ethernet/intel/i40e/i40e_txrx.c\nindex 5bc2748..2a492ec 100644\n--- a/drivers/net/ethernet/intel/i40e/i40e_txrx.c\n+++ b/drivers/net/ethernet/intel/i40e/i40e_txrx.c\n@@ -725,6 +725,59 @@ u32 i40e_get_tx_pending(struct i40e_ring *ring)\n \treturn 0;\n }\n \n+/**\n+ * i40e_detect_recover_hung - Function to detect and recover hung_queues\n+ * @vsi: pointer to vsi struct with tx queues\n+ *\n+ * VSI has netdev and netdev has TX queues. This function is to check each of\n+ * those TX queues if they are hung, trigger recovery by issuing SW interrupt.\n+ **/\n+void i40e_detect_recover_hung(struct i40e_vsi *vsi)\n+{\n+\tstruct i40e_ring *tx_ring = NULL;\n+\tstruct net_device *netdev;\n+\tunsigned int i;\n+\tint packets;\n+\n+\tif (!vsi)\n+\t\treturn;\n+\n+\tif (test_bit(__I40E_VSI_DOWN, vsi->state))\n+\t\treturn;\n+\n+\tnetdev = vsi->netdev;\n+\tif (!netdev)\n+\t\treturn;\n+\n+\tif (!netif_carrier_ok(netdev))\n+\t\treturn;\n+\n+\tfor (i = 0; i < vsi->num_queue_pairs; i++) {\n+\t\ttx_ring = vsi->tx_rings[i];\n+\t\tif (tx_ring && tx_ring->desc) {\n+\t\t\t/* If packet counter has not changed the queue is\n+\t\t\t * likely stalled, so force an interrupt for this\n+\t\t\t * queue.\n+\t\t\t *\n+\t\t\t * prev_pkt_ctr would be negative if there was no\n+\t\t\t * pending work.\n+\t\t\t */\n+\t\t\tpackets = tx_ring->stats.packets & INT_MAX;\n+\t\t\tif (tx_ring->tx_stats.prev_pkt_ctr == packets) {\n+\t\t\t\ti40e_force_wb(vsi, tx_ring->q_vector);\n+\t\t\t\tcontinue;\n+\t\t\t}\n+\n+\t\t\t/* Memory barrier between read of packet count and call\n+\t\t\t * to i40e_get_tx_pending()\n+\t\t\t */\n+\t\t\tsmp_rmb();\n+\t\t\ttx_ring->tx_stats.prev_pkt_ctr =\n+\t\t\t i40e_get_tx_pending(tx_ring) ? packets : -1;\n+\t\t}\n+\t}\n+}\n+\n #define WB_STRIDE 4\n \n /**\n@@ -1162,6 +1215,7 @@ int i40e_setup_tx_descriptors(struct i40e_ring *tx_ring)\n \n \ttx_ring->next_to_use = 0;\n \ttx_ring->next_to_clean = 0;\n+\ttx_ring->tx_stats.prev_pkt_ctr = -1;\n \treturn 0;\n \n err:\ndiff --git a/drivers/net/ethernet/intel/i40e/i40e_txrx.h b/drivers/net/ethernet/intel/i40e/i40e_txrx.h\nindex fbae118..f1c13be 100644\n--- a/drivers/net/ethernet/intel/i40e/i40e_txrx.h\n+++ b/drivers/net/ethernet/intel/i40e/i40e_txrx.h\n@@ -331,6 +331,7 @@ struct i40e_tx_queue_stats {\n \tu64 tx_done_old;\n \tu64 tx_linearize;\n \tu64 tx_force_wb;\n+\tint prev_pkt_ctr;\n };\n \n struct i40e_rx_queue_stats {\n@@ -498,6 +499,7 @@ void i40e_free_rx_resources(struct i40e_ring *rx_ring);\n int i40e_napi_poll(struct napi_struct *napi, int budget);\n void i40e_force_wb(struct i40e_vsi *vsi, struct i40e_q_vector *q_vector);\n u32 i40e_get_tx_pending(struct i40e_ring *ring);\n+void i40e_detect_recover_hung(struct i40e_vsi *vsi);\n int __i40e_maybe_stop_tx(struct i40e_ring *tx_ring, int size);\n bool __i40e_chk_linearize(struct sk_buff *skb);\n \ndiff --git a/drivers/net/ethernet/intel/i40evf/i40e_txrx.c b/drivers/net/ethernet/intel/i40evf/i40e_txrx.c\nindex 1ba29bb..c7831f7 100644\n--- a/drivers/net/ethernet/intel/i40evf/i40e_txrx.c\n+++ b/drivers/net/ethernet/intel/i40evf/i40e_txrx.c\n@@ -148,6 +148,59 @@ u32 i40evf_get_tx_pending(struct i40e_ring *ring, bool in_sw)\n \treturn 0;\n }\n \n+/**\n+ * i40evf_detect_recover_hung - Function to detect and recover hung_queues\n+ * @vsi: pointer to vsi struct with tx queues\n+ *\n+ * VSI has netdev and netdev has TX queues. This function is to check each of\n+ * those TX queues if they are hung, trigger recovery by issuing SW interrupt.\n+ **/\n+void i40evf_detect_recover_hung(struct i40e_vsi *vsi)\n+{\n+\tstruct i40e_ring *tx_ring = NULL;\n+\tstruct net_device *netdev;\n+\tunsigned int i;\n+\tint packets;\n+\n+\tif (!vsi)\n+\t\treturn;\n+\n+\tif (test_bit(__I40E_VSI_DOWN, vsi->state))\n+\t\treturn;\n+\n+\tnetdev = vsi->netdev;\n+\tif (!netdev)\n+\t\treturn;\n+\n+\tif (!netif_carrier_ok(netdev))\n+\t\treturn;\n+\n+\tfor (i = 0; i < vsi->back->num_active_queues; i++) {\n+\t\ttx_ring = &vsi->back->tx_rings[i];\n+\t\tif (tx_ring && tx_ring->desc) {\n+\t\t\t/* If packet counter has not changed the queue is\n+\t\t\t * likely stalled, so force an interrupt for this\n+\t\t\t * queue.\n+\t\t\t *\n+\t\t\t * prev_pkt_ctr would be negative if there was no\n+\t\t\t * pending work.\n+\t\t\t */\n+\t\t\tpackets = tx_ring->stats.packets & INT_MAX;\n+\t\t\tif (tx_ring->tx_stats.prev_pkt_ctr == packets) {\n+\t\t\t\ti40evf_force_wb(vsi, tx_ring->q_vector);\n+\t\t\t\tcontinue;\n+\t\t\t}\n+\n+\t\t\t/* Memory barrier between read of packet count and call\n+\t\t\t * to i40evf_get_tx_pending()\n+\t\t\t */\n+\t\t\tsmp_rmb();\n+\t\t\ttx_ring->tx_stats.prev_pkt_ctr =\n+\t\t\t i40evf_get_tx_pending(tx_ring, false) ? packets : -1;\n+\t\t}\n+\t}\n+}\n+\n #define WB_STRIDE 4\n \n /**\n@@ -469,6 +522,7 @@ int i40evf_setup_tx_descriptors(struct i40e_ring *tx_ring)\n \n \ttx_ring->next_to_use = 0;\n \ttx_ring->next_to_clean = 0;\n+\ttx_ring->tx_stats.prev_pkt_ctr = -1;\n \treturn 0;\n \n err:\ndiff --git a/drivers/net/ethernet/intel/i40evf/i40e_txrx.h b/drivers/net/ethernet/intel/i40evf/i40e_txrx.h\nindex 8d26c85..e72f16b 100644\n--- a/drivers/net/ethernet/intel/i40evf/i40e_txrx.h\n+++ b/drivers/net/ethernet/intel/i40evf/i40e_txrx.h\n@@ -313,6 +313,7 @@ struct i40e_tx_queue_stats {\n \tu64 tx_done_old;\n \tu64 tx_linearize;\n \tu64 tx_force_wb;\n+\tint prev_pkt_ctr;\n \tu64 tx_lost_interrupt;\n };\n \n@@ -467,6 +468,7 @@ void i40evf_free_rx_resources(struct i40e_ring *rx_ring);\n int i40evf_napi_poll(struct napi_struct *napi, int budget);\n void i40evf_force_wb(struct i40e_vsi *vsi, struct i40e_q_vector *q_vector);\n u32 i40evf_get_tx_pending(struct i40e_ring *ring, bool in_sw);\n+void i40evf_detect_recover_hung(struct i40e_vsi *vsi);\n int __i40evf_maybe_stop_tx(struct i40e_ring *tx_ring, int size);\n bool __i40evf_chk_linearize(struct sk_buff *skb);\n \ndiff --git a/drivers/net/ethernet/intel/i40evf/i40evf_main.c b/drivers/net/ethernet/intel/i40evf/i40evf_main.c\nindex 0e5df19..8934f78 100644\n--- a/drivers/net/ethernet/intel/i40evf/i40evf_main.c\n+++ b/drivers/net/ethernet/intel/i40evf/i40evf_main.c\n@@ -1716,6 +1716,8 @@ static void i40evf_watchdog_task(struct work_struct *work)\n \tif (adapter->state == __I40EVF_RUNNING)\n \t\ti40evf_request_stats(adapter);\n watchdog_done:\n+\tif (adapter->state == __I40EVF_RUNNING)\n+\t\ti40evf_detect_recover_hung(&adapter->vsi);\n \tclear_bit(__I40EVF_IN_CRITICAL_TASK, &adapter->crit_section);\n restart_watchdog:\n \tif (adapter->state == __I40EVF_REMOVE)\n", "prefixes": [ "next", "S83-V5", "6/9" ] }