Patch Detail
get:
Show a patch.
patch:
Update a patch.
put:
Update a patch.
GET /api/patches/766318/?format=api
{ "id": 766318, "url": "http://patchwork.ozlabs.org/api/patches/766318/?format=api", "web_url": "http://patchwork.ozlabs.org/project/intel-wired-lan/patch/20170524055535.14155-3-bjorn.topel@gmail.com/", "project": { "id": 46, "url": "http://patchwork.ozlabs.org/api/projects/46/?format=api", "name": "Intel Wired Ethernet development", "link_name": "intel-wired-lan", "list_id": "intel-wired-lan.osuosl.org", "list_email": "intel-wired-lan@osuosl.org", "web_url": "", "scm_url": "", "webscm_url": "", "list_archive_url": "", "list_archive_url_format": "", "commit_url_format": "" }, "msgid": "<20170524055535.14155-3-bjorn.topel@gmail.com>", "list_archive_url": null, "date": "2017-05-24T05:55:35", "name": "[v7,2/2] i40e: add support for XDP_TX action", "commit_ref": null, "pull_url": null, "state": "accepted", "archived": false, "hash": "bb048482ad6720303d2dcfac0ad0e2ea53512d17", "submitter": { "id": 70569, "url": "http://patchwork.ozlabs.org/api/people/70569/?format=api", "name": "Björn Töpel", "email": "bjorn.topel@gmail.com" }, "delegate": { "id": 68, "url": "http://patchwork.ozlabs.org/api/users/68/?format=api", "username": "jtkirshe", "first_name": "Jeff", "last_name": "Kirsher", "email": "jeffrey.t.kirsher@intel.com" }, "mbox": "http://patchwork.ozlabs.org/project/intel-wired-lan/patch/20170524055535.14155-3-bjorn.topel@gmail.com/mbox/", "series": [], "comments": "http://patchwork.ozlabs.org/api/patches/766318/comments/", "check": "pending", "checks": "http://patchwork.ozlabs.org/api/patches/766318/checks/", "tags": {}, "related": [], "headers": { "Return-Path": "<intel-wired-lan-bounces@osuosl.org>", "X-Original-To": [ "incoming@patchwork.ozlabs.org", "intel-wired-lan@lists.osuosl.org" ], "Delivered-To": [ "patchwork-incoming@bilbo.ozlabs.org", "intel-wired-lan@lists.osuosl.org" ], "Received": [ "from silver.osuosl.org (smtp3.osuosl.org [140.211.166.136])\n\t(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3wXhTY3MsVz9sP9\n\tfor <incoming@patchwork.ozlabs.org>;\n\tWed, 24 May 2017 15:56:01 +1000 (AEST)", "from localhost (localhost [127.0.0.1])\n\tby silver.osuosl.org (Postfix) with ESMTP id 74D0030324;\n\tWed, 24 May 2017 05:55:59 +0000 (UTC)", "from silver.osuosl.org ([127.0.0.1])\n\tby localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024)\n\twith ESMTP id N+dA1Nvx0a9Y; Wed, 24 May 2017 05:55:52 +0000 (UTC)", "from ash.osuosl.org (ash.osuosl.org [140.211.166.34])\n\tby silver.osuosl.org (Postfix) with ESMTP id 13F6530335;\n\tWed, 24 May 2017 05:55:52 +0000 (UTC)", "from fraxinus.osuosl.org (smtp4.osuosl.org [140.211.166.137])\n\tby ash.osuosl.org (Postfix) with ESMTP id B01E61C05C2\n\tfor <intel-wired-lan@lists.osuosl.org>;\n\tWed, 24 May 2017 05:55:50 +0000 (UTC)", "from localhost (localhost [127.0.0.1])\n\tby fraxinus.osuosl.org (Postfix) with ESMTP id AAC4086EB1\n\tfor <intel-wired-lan@lists.osuosl.org>;\n\tWed, 24 May 2017 05:55:50 +0000 (UTC)", "from fraxinus.osuosl.org ([127.0.0.1])\n\tby localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024)\n\twith ESMTP id 9nkyosW_LR2m for <intel-wired-lan@lists.osuosl.org>;\n\tWed, 24 May 2017 05:55:49 +0000 (UTC)", "from mga06.intel.com (mga06.intel.com [134.134.136.31])\n\tby fraxinus.osuosl.org (Postfix) with ESMTPS id E664986E81\n\tfor <intel-wired-lan@lists.osuosl.org>;\n\tWed, 24 May 2017 05:55:48 +0000 (UTC)", "from fmsmga006.fm.intel.com ([10.253.24.20])\n\tby orsmga104.jf.intel.com with ESMTP; 23 May 2017 22:55:48 -0700", "from btopel-mobl1.isw.intel.com (HELO localhost.localdomain)\n\t([10.103.211.137])\n\tby fmsmga006.fm.intel.com with ESMTP; 23 May 2017 22:55:45 -0700" ], "X-Virus-Scanned": [ "amavisd-new at osuosl.org", "amavisd-new at osuosl.org" ], "X-Greylist": "from auto-whitelisted by SQLgrey-1.7.6", "X-ExtLoop1": "1", "X-IronPort-AV": "E=Sophos;i=\"5.38,384,1491289200\"; d=\"scan'208\";a=\"106238934\"", "From": "=?UTF-8?q?Bj=C3=B6rn=20T=C3=B6pel?= <bjorn.topel@gmail.com>", "To": "jeffrey.t.kirsher@intel.com, intel-wired-lan@lists.osuosl.org,\n\talexander.duyck@gmail.com", "Date": "Wed, 24 May 2017 07:55:35 +0200", "Message-Id": "<20170524055535.14155-3-bjorn.topel@gmail.com>", "X-Mailer": "git-send-email 2.11.0", "In-Reply-To": "<20170524055535.14155-1-bjorn.topel@gmail.com>", "References": "<20170524055535.14155-1-bjorn.topel@gmail.com>", "MIME-Version": "1.0", "Cc": "daniel@iogearbox.net,\n\t=?UTF-8?q?Bj=C3=B6rn=20T=C3=B6pel?= <bjorn.topel@intel.com>,\n\tmagnus.karlsson@intel.com", "Subject": "[Intel-wired-lan] [PATCH v7 2/2] i40e: add support for XDP_TX action", "X-BeenThere": "intel-wired-lan@osuosl.org", "X-Mailman-Version": "2.1.18-1", "Precedence": "list", "List-Id": "Intel Wired Ethernet Linux Kernel Driver Development\n\t<intel-wired-lan.osuosl.org>", "List-Unsubscribe": "<https://lists.osuosl.org/mailman/options/intel-wired-lan>, \n\t<mailto:intel-wired-lan-request@osuosl.org?subject=unsubscribe>", "List-Archive": "<http://lists.osuosl.org/pipermail/intel-wired-lan/>", "List-Post": "<mailto:intel-wired-lan@osuosl.org>", "List-Help": "<mailto:intel-wired-lan-request@osuosl.org?subject=help>", "List-Subscribe": "<https://lists.osuosl.org/mailman/listinfo/intel-wired-lan>, \n\t<mailto:intel-wired-lan-request@osuosl.org?subject=subscribe>", "Content-Type": "text/plain; charset=\"utf-8\"", "Content-Transfer-Encoding": "base64", "Errors-To": "intel-wired-lan-bounces@osuosl.org", "Sender": "\"Intel-wired-lan\" <intel-wired-lan-bounces@osuosl.org>" }, "content": "From: Björn Töpel <bjorn.topel@intel.com>\n\nThis patch adds proper XDP_TX action support. For each Tx ring, an\nadditional XDP Tx ring is allocated and setup. This version does the\nDMA mapping in the fast-path, which will penalize performance for\nIOMMU enabled systems. Further, debugfs support is not wired up for\nthe XDP Tx rings.\n\nSigned-off-by: Björn Töpel <bjorn.topel@intel.com>\n---\n drivers/net/ethernet/intel/i40e/i40e.h | 1 +\n drivers/net/ethernet/intel/i40e/i40e_ethtool.c | 42 +++-\n drivers/net/ethernet/intel/i40e/i40e_main.c | 277 +++++++++++++++++++------\n drivers/net/ethernet/intel/i40e/i40e_txrx.c | 118 ++++++++++-\n drivers/net/ethernet/intel/i40e/i40e_txrx.h | 11 +\n 5 files changed, 373 insertions(+), 76 deletions(-)", "diff": "diff --git a/drivers/net/ethernet/intel/i40e/i40e.h b/drivers/net/ethernet/intel/i40e/i40e.h\nindex d3195b29d53c..4250ab55a9f1 100644\n--- a/drivers/net/ethernet/intel/i40e/i40e.h\n+++ b/drivers/net/ethernet/intel/i40e/i40e.h\n@@ -629,6 +629,7 @@ struct i40e_vsi {\n \t/* These are containers of ring pointers, allocated at run-time */\n \tstruct i40e_ring **rx_rings;\n \tstruct i40e_ring **tx_rings;\n+\tstruct i40e_ring **xdp_rings; /* XDP Tx rings */\n \n \tu32 active_filters;\n \tu32 promisc_threshold;\ndiff --git a/drivers/net/ethernet/intel/i40e/i40e_ethtool.c b/drivers/net/ethernet/intel/i40e/i40e_ethtool.c\nindex 3d58762efbc0..9d3233c2c9cd 100644\n--- a/drivers/net/ethernet/intel/i40e/i40e_ethtool.c\n+++ b/drivers/net/ethernet/intel/i40e/i40e_ethtool.c\n@@ -1299,6 +1299,17 @@ static void i40e_get_ringparam(struct net_device *netdev,\n \tring->rx_jumbo_pending = 0;\n }\n \n+static bool i40e_active_tx_ring_index(struct i40e_vsi *vsi, u16 index)\n+{\n+\tif (i40e_enabled_xdp_vsi(vsi)) {\n+\t\treturn index < vsi->num_queue_pairs ||\n+\t\t\t(index >= vsi->alloc_queue_pairs &&\n+\t\t\t index < vsi->alloc_queue_pairs + vsi->num_queue_pairs);\n+\t}\n+\n+\treturn index < vsi->num_queue_pairs;\n+}\n+\n static int i40e_set_ringparam(struct net_device *netdev,\n \t\t\t struct ethtool_ringparam *ring)\n {\n@@ -1308,6 +1319,7 @@ static int i40e_set_ringparam(struct net_device *netdev,\n \tstruct i40e_vsi *vsi = np->vsi;\n \tstruct i40e_pf *pf = vsi->back;\n \tu32 new_rx_count, new_tx_count;\n+\tu16 tx_alloc_queue_pairs;\n \tint timeout = 50;\n \tint i, err = 0;\n \n@@ -1345,6 +1357,8 @@ static int i40e_set_ringparam(struct net_device *netdev,\n \t\tfor (i = 0; i < vsi->num_queue_pairs; i++) {\n \t\t\tvsi->tx_rings[i]->count = new_tx_count;\n \t\t\tvsi->rx_rings[i]->count = new_rx_count;\n+\t\t\tif (i40e_enabled_xdp_vsi(vsi))\n+\t\t\t\tvsi->xdp_rings[i]->count = new_tx_count;\n \t\t}\n \t\tgoto done;\n \t}\n@@ -1354,20 +1368,24 @@ static int i40e_set_ringparam(struct net_device *netdev,\n \t * to the Tx and Rx ring structs.\n \t */\n \n-\t/* alloc updated Tx resources */\n+\t/* alloc updated Tx and XDP Tx resources */\n+\ttx_alloc_queue_pairs = vsi->alloc_queue_pairs *\n+\t\t\t (i40e_enabled_xdp_vsi(vsi) ? 2 : 1);\n \tif (new_tx_count != vsi->tx_rings[0]->count) {\n \t\tnetdev_info(netdev,\n \t\t\t \"Changing Tx descriptor count from %d to %d.\\n\",\n \t\t\t vsi->tx_rings[0]->count, new_tx_count);\n-\t\ttx_rings = kcalloc(vsi->alloc_queue_pairs,\n+\t\ttx_rings = kcalloc(tx_alloc_queue_pairs,\n \t\t\t\t sizeof(struct i40e_ring), GFP_KERNEL);\n \t\tif (!tx_rings) {\n \t\t\terr = -ENOMEM;\n \t\t\tgoto done;\n \t\t}\n \n-\t\tfor (i = 0; i < vsi->num_queue_pairs; i++) {\n-\t\t\t/* clone ring and setup updated count */\n+\t\tfor (i = 0; i < tx_alloc_queue_pairs; i++) {\n+\t\t\tif (!i40e_active_tx_ring_index(vsi, i))\n+\t\t\t\tcontinue;\n+\n \t\t\ttx_rings[i] = *vsi->tx_rings[i];\n \t\t\ttx_rings[i].count = new_tx_count;\n \t\t\t/* the desc and bi pointers will be reallocated in the\n@@ -1379,6 +1397,8 @@ static int i40e_set_ringparam(struct net_device *netdev,\n \t\t\tif (err) {\n \t\t\t\twhile (i) {\n \t\t\t\t\ti--;\n+\t\t\t\t\tif (!i40e_active_tx_ring_index(vsi, i))\n+\t\t\t\t\t\tcontinue;\n \t\t\t\t\ti40e_free_tx_resources(&tx_rings[i]);\n \t\t\t\t}\n \t\t\t\tkfree(tx_rings);\n@@ -1446,9 +1466,11 @@ static int i40e_set_ringparam(struct net_device *netdev,\n \ti40e_down(vsi);\n \n \tif (tx_rings) {\n-\t\tfor (i = 0; i < vsi->num_queue_pairs; i++) {\n-\t\t\ti40e_free_tx_resources(vsi->tx_rings[i]);\n-\t\t\t*vsi->tx_rings[i] = tx_rings[i];\n+\t\tfor (i = 0; i < tx_alloc_queue_pairs; i++) {\n+\t\t\tif (i40e_active_tx_ring_index(vsi, i)) {\n+\t\t\t\ti40e_free_tx_resources(vsi->tx_rings[i]);\n+\t\t\t\t*vsi->tx_rings[i] = tx_rings[i];\n+\t\t\t}\n \t\t}\n \t\tkfree(tx_rings);\n \t\ttx_rings = NULL;\n@@ -1479,8 +1501,10 @@ static int i40e_set_ringparam(struct net_device *netdev,\n free_tx:\n \t/* error cleanup if the Rx allocations failed after getting Tx */\n \tif (tx_rings) {\n-\t\tfor (i = 0; i < vsi->num_queue_pairs; i++)\n-\t\t\ti40e_free_tx_resources(&tx_rings[i]);\n+\t\tfor (i = 0; i < tx_alloc_queue_pairs; i++) {\n+\t\t\tif (i40e_active_tx_ring_index(vsi, i))\n+\t\t\t\ti40e_free_tx_resources(vsi->tx_rings[i]);\n+\t\t}\n \t\tkfree(tx_rings);\n \t\ttx_rings = NULL;\n \t}\ndiff --git a/drivers/net/ethernet/intel/i40e/i40e_main.c b/drivers/net/ethernet/intel/i40e/i40e_main.c\nindex 1ef2ba840f86..9f4763db9204 100644\n--- a/drivers/net/ethernet/intel/i40e/i40e_main.c\n+++ b/drivers/net/ethernet/intel/i40e/i40e_main.c\n@@ -408,6 +408,27 @@ struct rtnl_link_stats64 *i40e_get_vsi_stats_struct(struct i40e_vsi *vsi)\n }\n \n /**\n+ * i40e_get_netdev_stats_struct_tx - populate stats from a Tx ring\n+ * @ring: Tx ring to get statistics from\n+ * @stats: statistics entry to be updated\n+ **/\n+static void i40e_get_netdev_stats_struct_tx(struct i40e_ring *ring,\n+\t\t\t\t\t struct rtnl_link_stats64 *stats)\n+{\n+\tu64 bytes, packets;\n+\tunsigned int start;\n+\n+\tdo {\n+\t\tstart = u64_stats_fetch_begin_irq(&ring->syncp);\n+\t\tpackets = ring->stats.packets;\n+\t\tbytes = ring->stats.bytes;\n+\t} while (u64_stats_fetch_retry_irq(&ring->syncp, start));\n+\n+\tstats->tx_packets += packets;\n+\tstats->tx_bytes += bytes;\n+}\n+\n+/**\n * i40e_get_netdev_stats_struct - Get statistics for netdev interface\n * @netdev: network interface device structure\n *\n@@ -437,15 +458,8 @@ static void i40e_get_netdev_stats_struct(struct net_device *netdev,\n \t\ttx_ring = ACCESS_ONCE(vsi->tx_rings[i]);\n \t\tif (!tx_ring)\n \t\t\tcontinue;\n+\t\ti40e_get_netdev_stats_struct_tx(tx_ring, stats);\n \n-\t\tdo {\n-\t\t\tstart = u64_stats_fetch_begin_irq(&tx_ring->syncp);\n-\t\t\tpackets = tx_ring->stats.packets;\n-\t\t\tbytes = tx_ring->stats.bytes;\n-\t\t} while (u64_stats_fetch_retry_irq(&tx_ring->syncp, start));\n-\n-\t\tstats->tx_packets += packets;\n-\t\tstats->tx_bytes += bytes;\n \t\trx_ring = &tx_ring[1];\n \n \t\tdo {\n@@ -456,6 +470,9 @@ static void i40e_get_netdev_stats_struct(struct net_device *netdev,\n \n \t\tstats->rx_packets += packets;\n \t\tstats->rx_bytes += bytes;\n+\n+\t\tif (i40e_enabled_xdp_vsi(vsi))\n+\t\t\ti40e_get_netdev_stats_struct_tx(&rx_ring[1], stats);\n \t}\n \trcu_read_unlock();\n \n@@ -2814,6 +2831,12 @@ static int i40e_vsi_setup_tx_resources(struct i40e_vsi *vsi)\n \tfor (i = 0; i < vsi->num_queue_pairs && !err; i++)\n \t\terr = i40e_setup_tx_descriptors(vsi->tx_rings[i]);\n \n+\tif (!i40e_enabled_xdp_vsi(vsi))\n+\t\treturn err;\n+\n+\tfor (i = 0; i < vsi->num_queue_pairs && !err; i++)\n+\t\terr = i40e_setup_tx_descriptors(vsi->xdp_rings[i]);\n+\n \treturn err;\n }\n \n@@ -2827,12 +2850,17 @@ static void i40e_vsi_free_tx_resources(struct i40e_vsi *vsi)\n {\n \tint i;\n \n-\tif (!vsi->tx_rings)\n-\t\treturn;\n+\tif (vsi->tx_rings) {\n+\t\tfor (i = 0; i < vsi->num_queue_pairs; i++)\n+\t\t\tif (vsi->tx_rings[i] && vsi->tx_rings[i]->desc)\n+\t\t\t\ti40e_free_tx_resources(vsi->tx_rings[i]);\n+\t}\n \n-\tfor (i = 0; i < vsi->num_queue_pairs; i++)\n-\t\tif (vsi->tx_rings[i] && vsi->tx_rings[i]->desc)\n-\t\t\ti40e_free_tx_resources(vsi->tx_rings[i]);\n+\tif (vsi->xdp_rings) {\n+\t\tfor (i = 0; i < vsi->num_queue_pairs; i++)\n+\t\t\tif (vsi->xdp_rings[i] && vsi->xdp_rings[i]->desc)\n+\t\t\t\ti40e_free_tx_resources(vsi->xdp_rings[i]);\n+\t}\n }\n \n /**\n@@ -3093,6 +3121,12 @@ static int i40e_vsi_configure_tx(struct i40e_vsi *vsi)\n \tfor (i = 0; (i < vsi->num_queue_pairs) && !err; i++)\n \t\terr = i40e_configure_tx_ring(vsi->tx_rings[i]);\n \n+\tif (!i40e_enabled_xdp_vsi(vsi))\n+\t\treturn err;\n+\n+\tfor (i = 0; (i < vsi->num_queue_pairs) && !err; i++)\n+\t\terr = i40e_configure_tx_ring(vsi->xdp_rings[i]);\n+\n \treturn err;\n }\n \n@@ -3237,6 +3271,7 @@ static int i40e_vsi_configure(struct i40e_vsi *vsi)\n **/\n static void i40e_vsi_configure_msix(struct i40e_vsi *vsi)\n {\n+\tbool has_xdp = i40e_enabled_xdp_vsi(vsi);\n \tstruct i40e_pf *pf = vsi->back;\n \tstruct i40e_hw *hw = &pf->hw;\n \tu16 vector;\n@@ -3267,17 +3302,29 @@ static void i40e_vsi_configure_msix(struct i40e_vsi *vsi)\n \t\t/* Linked list for the queuepairs assigned to this vector */\n \t\twr32(hw, I40E_PFINT_LNKLSTN(vector - 1), qp);\n \t\tfor (q = 0; q < q_vector->num_ringpairs; q++) {\n+\t\t\tu32 nextqp = has_xdp ? qp + vsi->alloc_queue_pairs : qp;\n \t\t\tu32 val;\n \n \t\t\tval = I40E_QINT_RQCTL_CAUSE_ENA_MASK |\n \t\t\t (I40E_RX_ITR << I40E_QINT_RQCTL_ITR_INDX_SHIFT) |\n \t\t\t (vector << I40E_QINT_RQCTL_MSIX_INDX_SHIFT) |\n-\t\t\t (qp << I40E_QINT_RQCTL_NEXTQ_INDX_SHIFT)|\n+\t\t\t (nextqp << I40E_QINT_RQCTL_NEXTQ_INDX_SHIFT)|\n \t\t\t (I40E_QUEUE_TYPE_TX\n \t\t\t\t << I40E_QINT_RQCTL_NEXTQ_TYPE_SHIFT);\n \n \t\t\twr32(hw, I40E_QINT_RQCTL(qp), val);\n \n+\t\t\tif (has_xdp) {\n+\t\t\t\tval = I40E_QINT_TQCTL_CAUSE_ENA_MASK |\n+\t\t\t\t (I40E_TX_ITR << I40E_QINT_TQCTL_ITR_INDX_SHIFT) |\n+\t\t\t\t (vector << I40E_QINT_TQCTL_MSIX_INDX_SHIFT) |\n+\t\t\t\t (qp << I40E_QINT_TQCTL_NEXTQ_INDX_SHIFT)|\n+\t\t\t\t (I40E_QUEUE_TYPE_TX\n+\t\t\t\t << I40E_QINT_TQCTL_NEXTQ_TYPE_SHIFT);\n+\n+\t\t\t\twr32(hw, I40E_QINT_TQCTL(nextqp), val);\n+\t\t\t}\n+\n \t\t\tval = I40E_QINT_TQCTL_CAUSE_ENA_MASK |\n \t\t\t (I40E_TX_ITR << I40E_QINT_TQCTL_ITR_INDX_SHIFT) |\n \t\t\t (vector << I40E_QINT_TQCTL_MSIX_INDX_SHIFT) |\n@@ -3342,6 +3389,7 @@ static void i40e_enable_misc_int_causes(struct i40e_pf *pf)\n **/\n static void i40e_configure_msi_and_legacy(struct i40e_vsi *vsi)\n {\n+\tu32 nextqp = i40e_enabled_xdp_vsi(vsi) ? vsi->alloc_queue_pairs : 0;\n \tstruct i40e_q_vector *q_vector = vsi->q_vectors[0];\n \tstruct i40e_pf *pf = vsi->back;\n \tstruct i40e_hw *hw = &pf->hw;\n@@ -3362,12 +3410,22 @@ static void i40e_configure_msi_and_legacy(struct i40e_vsi *vsi)\n \twr32(hw, I40E_PFINT_LNKLST0, 0);\n \n \t/* Associate the queue pair to the vector and enable the queue int */\n-\tval = I40E_QINT_RQCTL_CAUSE_ENA_MASK\t\t |\n-\t (I40E_RX_ITR << I40E_QINT_RQCTL_ITR_INDX_SHIFT) |\n+\tval = I40E_QINT_RQCTL_CAUSE_ENA_MASK\t\t |\n+\t (I40E_RX_ITR << I40E_QINT_RQCTL_ITR_INDX_SHIFT) |\n+\t (nextqp\t << I40E_QINT_RQCTL_NEXTQ_INDX_SHIFT)|\n \t (I40E_QUEUE_TYPE_TX << I40E_QINT_TQCTL_NEXTQ_TYPE_SHIFT);\n \n \twr32(hw, I40E_QINT_RQCTL(0), val);\n \n+\tif (i40e_enabled_xdp_vsi(vsi)) {\n+\t\tval = I40E_QINT_TQCTL_CAUSE_ENA_MASK\t\t |\n+\t\t (I40E_TX_ITR << I40E_QINT_TQCTL_ITR_INDX_SHIFT)|\n+\t\t (I40E_QUEUE_TYPE_TX\n+\t\t << I40E_QINT_TQCTL_NEXTQ_TYPE_SHIFT);\n+\n+\t wr32(hw, I40E_QINT_TQCTL(nextqp), val);\n+\t}\n+\n \tval = I40E_QINT_TQCTL_CAUSE_ENA_MASK\t\t |\n \t (I40E_TX_ITR << I40E_QINT_TQCTL_ITR_INDX_SHIFT) |\n \t (I40E_QUEUE_END_OF_LIST << I40E_QINT_TQCTL_NEXTQ_INDX_SHIFT);\n@@ -3534,6 +3592,9 @@ static void i40e_vsi_disable_irq(struct i40e_vsi *vsi)\n \tfor (i = 0; i < vsi->num_queue_pairs; i++) {\n \t\twr32(hw, I40E_QINT_TQCTL(vsi->tx_rings[i]->reg_idx), 0);\n \t\twr32(hw, I40E_QINT_RQCTL(vsi->rx_rings[i]->reg_idx), 0);\n+\t\tif (!i40e_enabled_xdp_vsi(vsi))\n+\t\t\tcontinue;\n+\t\twr32(hw, I40E_QINT_TQCTL(vsi->xdp_rings[i]->reg_idx), 0);\n \t}\n \n \tif (pf->flags & I40E_FLAG_MSIX_ENABLED) {\n@@ -3836,6 +3897,16 @@ static void i40e_map_vector_to_qp(struct i40e_vsi *vsi, int v_idx, int qp_idx)\n \tq_vector->tx.ring = tx_ring;\n \tq_vector->tx.count++;\n \n+\t/* Place XDP Tx ring in the same q_vector ring list as regular Tx */\n+\tif (i40e_enabled_xdp_vsi(vsi)) {\n+\t\tstruct i40e_ring *xdp_ring = vsi->xdp_rings[qp_idx];\n+\n+\t\txdp_ring->q_vector = q_vector;\n+\t\txdp_ring->next = q_vector->tx.ring;\n+\t\tq_vector->tx.ring = xdp_ring;\n+\t\tq_vector->tx.count++;\n+\t}\n+\n \trx_ring->q_vector = q_vector;\n \trx_ring->next = q_vector->rx.ring;\n \tq_vector->rx.ring = rx_ring;\n@@ -4015,6 +4086,33 @@ static void i40e_control_tx_q(struct i40e_pf *pf, int pf_q, bool enable)\n }\n \n /**\n+ * i40e_control_wait_tx_q - Start/stop Tx queue and wait for completion\n+ * @seid: VSI SEID\n+ * @pf: the PF structure\n+ * @pf_q: the PF queue to configure\n+ * @is_xdp: true if the queue is used for XDP\n+ * @enable: start or stop the queue\n+ **/\n+static int i40e_control_wait_tx_q(int seid, struct i40e_pf *pf, int pf_q,\n+\t\t\t\t bool is_xdp, bool enable)\n+{\n+\tint ret;\n+\n+\ti40e_control_tx_q(pf, pf_q, enable);\n+\n+\t/* wait for the change to finish */\n+\tret = i40e_pf_txq_wait(pf, pf_q, enable);\n+\tif (ret) {\n+\t\tdev_info(&pf->pdev->dev,\n+\t\t\t \"VSI seid %d %sTx ring %d %sable timeout\\n\",\n+\t\t\t seid, (is_xdp ? \"XDP \" : \"\"), pf_q,\n+\t\t\t (enable ? \"en\" : \"dis\"));\n+\t}\n+\n+\treturn ret;\n+}\n+\n+/**\n * i40e_vsi_control_tx - Start or stop a VSI's rings\n * @vsi: the VSI being configured\n * @enable: start or stop the rings\n@@ -4026,16 +4124,20 @@ static int i40e_vsi_control_tx(struct i40e_vsi *vsi, bool enable)\n \n \tpf_q = vsi->base_queue;\n \tfor (i = 0; i < vsi->num_queue_pairs; i++, pf_q++) {\n-\t\ti40e_control_tx_q(pf, pf_q, enable);\n+\t\tret = i40e_control_wait_tx_q(vsi->seid, pf,\n+\t\t\t\t\t pf_q,\n+\t\t\t\t\t false /*is xdp*/, enable);\n+\t\tif (ret)\n+\t\t\tbreak;\n \n-\t\t/* wait for the change to finish */\n-\t\tret = i40e_pf_txq_wait(pf, pf_q, enable);\n-\t\tif (ret) {\n-\t\t\tdev_info(&pf->pdev->dev,\n-\t\t\t\t \"VSI seid %d Tx ring %d %sable timeout\\n\",\n-\t\t\t\t vsi->seid, pf_q, (enable ? \"en\" : \"dis\"));\n+\t\tif (!i40e_enabled_xdp_vsi(vsi))\n+\t\t\tcontinue;\n+\n+\t\tret = i40e_control_wait_tx_q(vsi->seid, pf,\n+\t\t\t\t\t pf_q + vsi->alloc_queue_pairs,\n+\t\t\t\t\t true /*is xdp*/, enable);\n+\t\tif (ret)\n \t\t\tbreak;\n-\t\t}\n \t}\n \n \treturn ret;\n@@ -4547,7 +4649,21 @@ int i40e_vsi_wait_queues_disabled(struct i40e_vsi *vsi)\n \t\t\t\t vsi->seid, pf_q);\n \t\t\treturn ret;\n \t\t}\n-\t\t/* Check and wait for the Tx queue */\n+\n+\t\tif (!i40e_enabled_xdp_vsi(vsi))\n+\t\t\tgoto wait_rx;\n+\n+\t\t/* Check and wait for the XDP Tx queue */\n+\t\tret = i40e_pf_txq_wait(pf, pf_q + vsi->alloc_queue_pairs,\n+\t\t\t\t false);\n+\t\tif (ret) {\n+\t\t\tdev_info(&pf->pdev->dev,\n+\t\t\t\t \"VSI seid %d XDP Tx ring %d disable timeout\\n\",\n+\t\t\t\t vsi->seid, pf_q);\n+\t\t\treturn ret;\n+\t\t}\n+wait_rx:\n+\t\t/* Check and wait for the Rx queue */\n \t\tret = i40e_pf_rxq_wait(pf, pf_q, false);\n \t\tif (ret) {\n \t\t\tdev_info(&pf->pdev->dev,\n@@ -5466,6 +5582,8 @@ void i40e_down(struct i40e_vsi *vsi)\n \n \tfor (i = 0; i < vsi->num_queue_pairs; i++) {\n \t\ti40e_clean_tx_ring(vsi->tx_rings[i]);\n+\t\tif (i40e_enabled_xdp_vsi(vsi))\n+\t\t\ti40e_clean_tx_ring(vsi->xdp_rings[i]);\n \t\ti40e_clean_rx_ring(vsi->rx_rings[i]);\n \t}\n \n@@ -7534,15 +7652,22 @@ static int i40e_set_num_rings_in_vsi(struct i40e_vsi *vsi)\n **/\n static int i40e_vsi_alloc_arrays(struct i40e_vsi *vsi, bool alloc_qvectors)\n {\n+\tstruct i40e_ring **next_rings;\n \tint size;\n \tint ret = 0;\n \n-\t/* allocate memory for both Tx and Rx ring pointers */\n-\tsize = sizeof(struct i40e_ring *) * vsi->alloc_queue_pairs * 2;\n+\t/* allocate memory for both Tx, XDP Tx and Rx ring pointers */\n+\tsize = sizeof(struct i40e_ring *) * vsi->alloc_queue_pairs *\n+\t (i40e_enabled_xdp_vsi(vsi) ? 3 : 2);\n \tvsi->tx_rings = kzalloc(size, GFP_KERNEL);\n \tif (!vsi->tx_rings)\n \t\treturn -ENOMEM;\n-\tvsi->rx_rings = &vsi->tx_rings[vsi->alloc_queue_pairs];\n+\tnext_rings = vsi->tx_rings + vsi->alloc_queue_pairs;\n+\tif (i40e_enabled_xdp_vsi(vsi)) {\n+\t\tvsi->xdp_rings = next_rings;\n+\t\tnext_rings += vsi->alloc_queue_pairs;\n+\t}\n+\tvsi->rx_rings = next_rings;\n \n \tif (alloc_qvectors) {\n \t\t/* allocate memory for q_vector pointers */\n@@ -7662,6 +7787,7 @@ static void i40e_vsi_free_arrays(struct i40e_vsi *vsi, bool free_qvectors)\n \tkfree(vsi->tx_rings);\n \tvsi->tx_rings = NULL;\n \tvsi->rx_rings = NULL;\n+\tvsi->xdp_rings = NULL;\n }\n \n /**\n@@ -7745,6 +7871,8 @@ static void i40e_vsi_clear_rings(struct i40e_vsi *vsi)\n \t\t\tkfree_rcu(vsi->tx_rings[i], rcu);\n \t\t\tvsi->tx_rings[i] = NULL;\n \t\t\tvsi->rx_rings[i] = NULL;\n+\t\t\tif (vsi->xdp_rings)\n+\t\t\t\tvsi->xdp_rings[i] = NULL;\n \t\t}\n \t}\n }\n@@ -7755,43 +7883,61 @@ static void i40e_vsi_clear_rings(struct i40e_vsi *vsi)\n **/\n static int i40e_alloc_rings(struct i40e_vsi *vsi)\n {\n-\tstruct i40e_ring *tx_ring, *rx_ring;\n+\tint i, qpv = i40e_enabled_xdp_vsi(vsi) ? 3 : 2;\n \tstruct i40e_pf *pf = vsi->back;\n-\tint i;\n+\tstruct i40e_ring *ring;\n \n \t/* Set basic values in the rings to be used later during open() */\n \tfor (i = 0; i < vsi->alloc_queue_pairs; i++) {\n \t\t/* allocate space for both Tx and Rx in one shot */\n-\t\ttx_ring = kzalloc(sizeof(struct i40e_ring) * 2, GFP_KERNEL);\n-\t\tif (!tx_ring)\n+\t\tring = kcalloc(qpv, sizeof(struct i40e_ring), GFP_KERNEL);\n+\t\tif (!ring)\n \t\t\tgoto err_out;\n \n-\t\ttx_ring->queue_index = i;\n-\t\ttx_ring->reg_idx = vsi->base_queue + i;\n-\t\ttx_ring->ring_active = false;\n-\t\ttx_ring->vsi = vsi;\n-\t\ttx_ring->netdev = vsi->netdev;\n-\t\ttx_ring->dev = &pf->pdev->dev;\n-\t\ttx_ring->count = vsi->num_desc;\n-\t\ttx_ring->size = 0;\n-\t\ttx_ring->dcb_tc = 0;\n+\t\tring->queue_index = i;\n+\t\tring->reg_idx = vsi->base_queue + i;\n+\t\tring->ring_active = false;\n+\t\tring->vsi = vsi;\n+\t\tring->netdev = vsi->netdev;\n+\t\tring->dev = &pf->pdev->dev;\n+\t\tring->count = vsi->num_desc;\n+\t\tring->size = 0;\n+\t\tring->dcb_tc = 0;\n \t\tif (vsi->back->flags & I40E_FLAG_WB_ON_ITR_CAPABLE)\n-\t\t\ttx_ring->flags = I40E_TXR_FLAGS_WB_ON_ITR;\n-\t\ttx_ring->tx_itr_setting = pf->tx_itr_default;\n-\t\tvsi->tx_rings[i] = tx_ring;\n-\n-\t\trx_ring = &tx_ring[1];\n-\t\trx_ring->queue_index = i;\n-\t\trx_ring->reg_idx = vsi->base_queue + i;\n-\t\trx_ring->ring_active = false;\n-\t\trx_ring->vsi = vsi;\n-\t\trx_ring->netdev = vsi->netdev;\n-\t\trx_ring->dev = &pf->pdev->dev;\n-\t\trx_ring->count = vsi->num_desc;\n-\t\trx_ring->size = 0;\n-\t\trx_ring->dcb_tc = 0;\n-\t\trx_ring->rx_itr_setting = pf->rx_itr_default;\n-\t\tvsi->rx_rings[i] = rx_ring;\n+\t\t\tring->flags = I40E_TXR_FLAGS_WB_ON_ITR;\n+\t\tring->tx_itr_setting = pf->tx_itr_default;\n+\t\tvsi->tx_rings[i] = ring++;\n+\n+\t\tif (!i40e_enabled_xdp_vsi(vsi))\n+\t\t\tgoto setup_rx;\n+\n+\t\tring->queue_index = vsi->alloc_queue_pairs + i;\n+\t\tring->reg_idx = vsi->base_queue + ring->queue_index;\n+\t\tring->ring_active = false;\n+\t\tring->vsi = vsi;\n+\t\tring->netdev = NULL;\n+\t\tring->dev = &pf->pdev->dev;\n+\t\tring->count = vsi->num_desc;\n+\t\tring->size = 0;\n+\t\tring->dcb_tc = 0;\n+\t\tif (vsi->back->flags & I40E_FLAG_WB_ON_ITR_CAPABLE)\n+\t\t\tring->flags = I40E_TXR_FLAGS_WB_ON_ITR;\n+\t\tset_ring_xdp(ring);\n+\t\tring->tx_itr_setting = pf->tx_itr_default;\n+\t\tvsi->xdp_rings[i] = ring++;\n+\n+setup_rx:\n+\t\tring->queue_index = i;\n+\t\tring->reg_idx = vsi->base_queue + i;\n+\t\tring->ring_active = false;\n+\t\tring->vsi = vsi;\n+\t\tring->netdev = vsi->netdev;\n+\t\tring->dev = &pf->pdev->dev;\n+\t\tring->count = vsi->num_desc;\n+\t\tring->size = 0;\n+\t\tring->dcb_tc = 0;\n+\t\tring->rx_itr_setting = pf->rx_itr_default;\n+\t\tvsi->rx_rings[i] = ring;\n \t}\n \n \treturn 0;\n@@ -9997,6 +10143,7 @@ static int i40e_vsi_setup_vectors(struct i40e_vsi *vsi)\n **/\n static struct i40e_vsi *i40e_vsi_reinit_setup(struct i40e_vsi *vsi)\n {\n+\tu16 alloc_queue_pairs;\n \tstruct i40e_pf *pf;\n \tu8 enabled_tc;\n \tint ret;\n@@ -10015,11 +10162,14 @@ static struct i40e_vsi *i40e_vsi_reinit_setup(struct i40e_vsi *vsi)\n \tif (ret)\n \t\tgoto err_vsi;\n \n-\tret = i40e_get_lump(pf, pf->qp_pile, vsi->alloc_queue_pairs, vsi->idx);\n+\talloc_queue_pairs = vsi->alloc_queue_pairs *\n+\t\t\t (i40e_enabled_xdp_vsi(vsi) ? 2 : 1);\n+\n+\tret = i40e_get_lump(pf, pf->qp_pile, alloc_queue_pairs, vsi->idx);\n \tif (ret < 0) {\n \t\tdev_info(&pf->pdev->dev,\n \t\t\t \"failed to get tracking for %d queues for VSI %d err %d\\n\",\n-\t\t\t vsi->alloc_queue_pairs, vsi->seid, ret);\n+\t\t\t alloc_queue_pairs, vsi->seid, ret);\n \t\tgoto err_vsi;\n \t}\n \tvsi->base_queue = ret;\n@@ -10075,6 +10225,7 @@ struct i40e_vsi *i40e_vsi_setup(struct i40e_pf *pf, u8 type,\n {\n \tstruct i40e_vsi *vsi = NULL;\n \tstruct i40e_veb *veb = NULL;\n+\tu16 alloc_queue_pairs;\n \tint ret, i;\n \tint v_idx;\n \n@@ -10162,12 +10313,14 @@ struct i40e_vsi *i40e_vsi_setup(struct i40e_pf *pf, u8 type,\n \telse if (type == I40E_VSI_SRIOV)\n \t\tvsi->vf_id = param1;\n \t/* assign it some queues */\n-\tret = i40e_get_lump(pf, pf->qp_pile, vsi->alloc_queue_pairs,\n-\t\t\t\tvsi->idx);\n+\talloc_queue_pairs = vsi->alloc_queue_pairs *\n+\t\t\t (i40e_enabled_xdp_vsi(vsi) ? 2 : 1);\n+\n+\tret = i40e_get_lump(pf, pf->qp_pile, alloc_queue_pairs, vsi->idx);\n \tif (ret < 0) {\n \t\tdev_info(&pf->pdev->dev,\n \t\t\t \"failed to get tracking for %d queues for VSI %d err=%d\\n\",\n-\t\t\t vsi->alloc_queue_pairs, vsi->seid, ret);\n+\t\t\t alloc_queue_pairs, vsi->seid, ret);\n \t\tgoto err_vsi;\n \t}\n \tvsi->base_queue = ret;\ndiff --git a/drivers/net/ethernet/intel/i40e/i40e_txrx.c b/drivers/net/ethernet/intel/i40e/i40e_txrx.c\nindex f744f843bc72..b936febc315a 100644\n--- a/drivers/net/ethernet/intel/i40e/i40e_txrx.c\n+++ b/drivers/net/ethernet/intel/i40e/i40e_txrx.c\n@@ -630,6 +630,8 @@ static void i40e_unmap_and_free_tx_resource(struct i40e_ring *ring,\n \tif (tx_buffer->skb) {\n \t\tif (tx_buffer->tx_flags & I40E_TX_FLAGS_FD_SB)\n \t\t\tkfree(tx_buffer->raw_buf);\n+\t\telse if (ring_is_xdp(ring))\n+\t\t\tpage_frag_free(tx_buffer->raw_buf);\n \t\telse\n \t\t\tdev_kfree_skb_any(tx_buffer->skb);\n \t\tif (dma_unmap_len(tx_buffer, len))\n@@ -771,8 +773,11 @@ static bool i40e_clean_tx_irq(struct i40e_vsi *vsi,\n \t\ttotal_bytes += tx_buf->bytecount;\n \t\ttotal_packets += tx_buf->gso_segs;\n \n-\t\t/* free the skb */\n-\t\tnapi_consume_skb(tx_buf->skb, napi_budget);\n+\t\t/* free the skb/XDP data */\n+\t\tif (ring_is_xdp(tx_ring))\n+\t\t\tpage_frag_free(tx_buf->raw_buf);\n+\t\telse\n+\t\t\tnapi_consume_skb(tx_buf->skb, napi_budget);\n \n \t\t/* unmap skb header data */\n \t\tdma_unmap_single(tx_ring->dev,\n@@ -848,6 +853,9 @@ static bool i40e_clean_tx_irq(struct i40e_vsi *vsi,\n \t\t\ttx_ring->arm_wb = true;\n \t}\n \n+\tif (ring_is_xdp(tx_ring))\n+\t\treturn !!budget;\n+\n \t/* notify netdev of completed buffers */\n \tnetdev_tx_completed_queue(txring_txq(tx_ring),\n \t\t\t\t total_packets, total_bytes);\n@@ -1969,6 +1977,10 @@ static bool i40e_is_non_eop(struct i40e_ring *rx_ring,\n \n #define I40E_XDP_PASS 0\n #define I40E_XDP_CONSUMED 1\n+#define I40E_XDP_TX 2\n+\n+static int i40e_xmit_xdp_ring(struct xdp_buff *xdp,\n+\t\t\t struct i40e_ring *xdp_ring);\n \n /**\n * i40e_run_xdp - run an XDP program\n@@ -1979,6 +1991,7 @@ static struct sk_buff *i40e_run_xdp(struct i40e_ring *rx_ring,\n \t\t\t\t struct xdp_buff *xdp)\n {\n \tint result = I40E_XDP_PASS;\n+\tstruct i40e_ring *xdp_ring;\n \tstruct bpf_prog *xdp_prog;\n \tu32 act;\n \n@@ -1992,9 +2005,12 @@ static struct sk_buff *i40e_run_xdp(struct i40e_ring *rx_ring,\n \tswitch (act) {\n \tcase XDP_PASS:\n \t\tbreak;\n+\tcase XDP_TX:\n+\t\txdp_ring = rx_ring->vsi->xdp_rings[rx_ring->queue_index];\n+\t\tresult = i40e_xmit_xdp_ring(xdp, xdp_ring);\n+\t\tbreak;\n \tdefault:\n \t\tbpf_warn_invalid_xdp_action(act);\n-\tcase XDP_TX:\n \tcase XDP_ABORTED:\n \t\ttrace_xdp_exception(rx_ring->netdev, xdp_prog, act);\n \t\t/* fallthrough -- handle aborts by dropping packet */\n@@ -2008,6 +2024,27 @@ static struct sk_buff *i40e_run_xdp(struct i40e_ring *rx_ring,\n }\n \n /**\n+ * i40e_rx_buffer_flip - adjusted rx_buffer to point to an unused region\n+ * @rx_ring: Rx ring\n+ * @rx_buffer: Rx buffer to adjust\n+ * @size: Size of adjustment\n+ **/\n+static void i40e_rx_buffer_flip(struct i40e_ring *rx_ring,\n+\t\t\t\tstruct i40e_rx_buffer *rx_buffer,\n+\t\t\t\tunsigned int size)\n+{\n+#if (PAGE_SIZE < 8192)\n+\tunsigned int truesize = i40e_rx_pg_size(rx_ring) / 2;\n+\n+\trx_buffer->page_offset ^= truesize;\n+#else\n+\tunsigned int truesize = SKB_DATA_ALIGN(i40e_rx_offset(rx_ring) + size);\n+\n+\trx_buffer->page_offset += truesize;\n+#endif\n+}\n+\n+/**\n * i40e_clean_rx_irq - Clean completed descriptors from Rx ring - bounce buf\n * @rx_ring: rx descriptor ring to transact packets on\n * @budget: Total limit on number of packets to process\n@@ -2024,7 +2061,7 @@ static int i40e_clean_rx_irq(struct i40e_ring *rx_ring, int budget)\n \tunsigned int total_rx_bytes = 0, total_rx_packets = 0;\n \tstruct sk_buff *skb = rx_ring->skb;\n \tu16 cleaned_count = I40E_DESC_UNUSED(rx_ring);\n-\tbool failure = false;\n+\tbool failure = false, xdp_xmit = false;\n \n \twhile (likely(total_rx_packets < budget)) {\n \t\tstruct i40e_rx_buffer *rx_buffer;\n@@ -2081,9 +2118,14 @@ static int i40e_clean_rx_irq(struct i40e_ring *rx_ring, int budget)\n \t\t}\n \n \t\tif (IS_ERR(skb)) {\n+\t\t\tif (PTR_ERR(skb) == -I40E_XDP_TX) {\n+\t\t\t\txdp_xmit = true;\n+\t\t\t\ti40e_rx_buffer_flip(rx_ring, rx_buffer, size);\n+\t\t\t} else {\n+\t\t\t\trx_buffer->pagecnt_bias++;\n+\t\t\t}\n \t\t\ttotal_rx_bytes += size;\n \t\t\ttotal_rx_packets++;\n-\t\t\trx_buffer->pagecnt_bias++;\n \t\t} else if (skb) {\n \t\t\ti40e_add_rx_frag(rx_ring, rx_buffer, skb, size);\n \t\t} else if (ring_uses_build_skb(rx_ring)) {\n@@ -2131,6 +2173,19 @@ static int i40e_clean_rx_irq(struct i40e_ring *rx_ring, int budget)\n \t\ttotal_rx_packets++;\n \t}\n \n+\tif (xdp_xmit) {\n+\t\tstruct i40e_ring *xdp_ring;\n+\n+\t\txdp_ring = rx_ring->vsi->xdp_rings[rx_ring->queue_index];\n+\n+\t\t/* Force memory writes to complete before letting h/w\n+\t\t * know there are new descriptors to fetch.\n+\t\t */\n+\t\twmb();\n+\n+\t\twritel(xdp_ring->next_to_use, xdp_ring->tail);\n+\t}\n+\n \trx_ring->skb = skb;\n \n \tu64_stats_update_begin(&rx_ring->syncp);\n@@ -3188,6 +3243,59 @@ static inline int i40e_tx_map(struct i40e_ring *tx_ring, struct sk_buff *skb,\n }\n \n /**\n+ * i40e_xmit_xdp_ring - transmits an XDP buffer to an XDP Tx ring\n+ * @xdp: data to transmit\n+ * @xdp_ring: XDP Tx ring\n+ **/\n+static int i40e_xmit_xdp_ring(struct xdp_buff *xdp,\n+\t\t\t struct i40e_ring *xdp_ring)\n+{\n+\tu32 size = xdp->data_end - xdp->data;\n+\tu16 i = xdp_ring->next_to_use;\n+\tstruct i40e_tx_buffer *tx_bi;\n+\tstruct i40e_tx_desc *tx_desc;\n+\tdma_addr_t dma;\n+\n+\tif (!unlikely(I40E_DESC_UNUSED(xdp_ring))) {\n+\t\txdp_ring->tx_stats.tx_busy++;\n+\t\treturn I40E_XDP_CONSUMED;\n+\t}\n+\n+\tdma = dma_map_single(xdp_ring->dev, xdp->data, size, DMA_TO_DEVICE);\n+\tif (dma_mapping_error(xdp_ring->dev, dma))\n+\t\treturn I40E_XDP_CONSUMED;\n+\n+\ttx_bi = &xdp_ring->tx_bi[i];\n+\ttx_bi->bytecount = size;\n+\ttx_bi->gso_segs = 1;\n+\ttx_bi->raw_buf = xdp->data;\n+\n+\t/* record length, and DMA address */\n+\tdma_unmap_len_set(tx_bi, len, size);\n+\tdma_unmap_addr_set(tx_bi, dma, dma);\n+\n+\ttx_desc = I40E_TX_DESC(xdp_ring, i);\n+\ttx_desc->buffer_addr = cpu_to_le64(dma);\n+\ttx_desc->cmd_type_offset_bsz = build_ctob(I40E_TX_DESC_CMD_ICRC\n+\t\t\t\t\t\t | I40E_TXD_CMD,\n+\t\t\t\t\t\t 0, size, 0);\n+\n+\t/* Make certain all of the status bits have been updated\n+\t * before next_to_watch is written.\n+\t */\n+\tsmp_wmb();\n+\n+\ti++;\n+\tif (i == xdp_ring->count)\n+\t\ti = 0;\n+\n+\ttx_bi->next_to_watch = tx_desc;\n+\txdp_ring->next_to_use = i;\n+\n+\treturn I40E_XDP_TX;\n+}\n+\n+/**\n * i40e_xmit_frame_ring - Sends buffer on Tx ring\n * @skb: send buffer\n * @tx_ring: ring to send buffer on\ndiff --git a/drivers/net/ethernet/intel/i40e/i40e_txrx.h b/drivers/net/ethernet/intel/i40e/i40e_txrx.h\nindex 31f0b162996f..b288d58313a6 100644\n--- a/drivers/net/ethernet/intel/i40e/i40e_txrx.h\n+++ b/drivers/net/ethernet/intel/i40e/i40e_txrx.h\n@@ -396,6 +396,7 @@ struct i40e_ring {\n \tu16 flags;\n #define I40E_TXR_FLAGS_WB_ON_ITR\t\tBIT(0)\n #define I40E_RXR_FLAGS_BUILD_SKB_ENABLED\tBIT(1)\n+#define I40E_TXR_FLAGS_XDP\t\t\tBIT(2)\n \n \t/* stats structs */\n \tstruct i40e_queue_stats\tstats;\n@@ -438,6 +439,16 @@ static inline void clear_ring_build_skb_enabled(struct i40e_ring *ring)\n \tring->flags &= ~I40E_RXR_FLAGS_BUILD_SKB_ENABLED;\n }\n \n+static inline bool ring_is_xdp(struct i40e_ring *ring)\n+{\n+\treturn !!(ring->flags & I40E_TXR_FLAGS_XDP);\n+}\n+\n+static inline void set_ring_xdp(struct i40e_ring *ring)\n+{\n+\tring->flags |= I40E_TXR_FLAGS_XDP;\n+}\n+\n enum i40e_latency_range {\n \tI40E_LOWEST_LATENCY = 0,\n \tI40E_LOW_LATENCY = 1,\n", "prefixes": [ "v7", "2/2" ] }