Patch Detail
get:
Show a patch.
patch:
Update a patch.
put:
Update a patch.
GET /api/patches/1178241/?format=api
{ "id": 1178241, "url": "http://patchwork.ozlabs.org/api/patches/1178241/?format=api", "web_url": "http://patchwork.ozlabs.org/project/intel-wired-lan/patch/20191016150201.41597-3-anthony.l.nguyen@intel.com/", "project": { "id": 46, "url": "http://patchwork.ozlabs.org/api/projects/46/?format=api", "name": "Intel Wired Ethernet development", "link_name": "intel-wired-lan", "list_id": "intel-wired-lan.osuosl.org", "list_email": "intel-wired-lan@osuosl.org", "web_url": "", "scm_url": "", "webscm_url": "", "list_archive_url": "", "list_archive_url_format": "", "commit_url_format": "" }, "msgid": "<20191016150201.41597-3-anthony.l.nguyen@intel.com>", "list_archive_url": null, "date": "2019-10-16T15:01:55", "name": "[S30,v2,3/9] ice: Add support for XDP", "commit_ref": null, "pull_url": null, "state": "superseded", "archived": false, "hash": "c72d88351715b1a9ab074153046326eb3ca7f3f8", "submitter": { "id": 68875, "url": "http://patchwork.ozlabs.org/api/people/68875/?format=api", "name": "Tony Nguyen", "email": "anthony.l.nguyen@intel.com" }, "delegate": { "id": 68, "url": "http://patchwork.ozlabs.org/api/users/68/?format=api", "username": "jtkirshe", "first_name": "Jeff", "last_name": "Kirsher", "email": "jeffrey.t.kirsher@intel.com" }, "mbox": "http://patchwork.ozlabs.org/project/intel-wired-lan/patch/20191016150201.41597-3-anthony.l.nguyen@intel.com/mbox/", "series": [ { "id": 136670, "url": "http://patchwork.ozlabs.org/api/series/136670/?format=api", "web_url": "http://patchwork.ozlabs.org/project/intel-wired-lan/list/?series=136670", "date": "2019-10-16T15:01:54", "name": "[S30,v2,1/9] ice: Introduce ice_base.c", "version": 2, "mbox": "http://patchwork.ozlabs.org/series/136670/mbox/" } ], "comments": "http://patchwork.ozlabs.org/api/patches/1178241/comments/", "check": "pending", "checks": "http://patchwork.ozlabs.org/api/patches/1178241/checks/", "tags": {}, "related": [], "headers": { "Return-Path": "<intel-wired-lan-bounces@osuosl.org>", "X-Original-To": [ "incoming@patchwork.ozlabs.org", "intel-wired-lan@lists.osuosl.org" ], "Delivered-To": [ "patchwork-incoming@bilbo.ozlabs.org", "intel-wired-lan@lists.osuosl.org" ], "Authentication-Results": [ "ozlabs.org; spf=pass (sender SPF authorized)\n\tsmtp.mailfrom=osuosl.org (client-ip=140.211.166.133;\n\thelo=hemlock.osuosl.org;\n\tenvelope-from=intel-wired-lan-bounces@osuosl.org;\n\treceiver=<UNKNOWN>)", "ozlabs.org;\n\tdmarc=fail (p=none dis=none) header.from=intel.com" ], "Received": [ "from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 46tpVd2SdVz9sNx\n\tfor <incoming@patchwork.ozlabs.org>;\n\tThu, 17 Oct 2019 10:32:25 +1100 (AEDT)", "from localhost (localhost [127.0.0.1])\n\tby hemlock.osuosl.org (Postfix) with ESMTP id DEAEA88524;\n\tWed, 16 Oct 2019 23:32:23 +0000 (UTC)", "from hemlock.osuosl.org ([127.0.0.1])\n\tby localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024)\n\twith ESMTP id 4anlUTkUST9N; Wed, 16 Oct 2019 23:32:19 +0000 (UTC)", "from ash.osuosl.org (ash.osuosl.org [140.211.166.34])\n\tby hemlock.osuosl.org (Postfix) with ESMTP id 702C288543;\n\tWed, 16 Oct 2019 23:32:19 +0000 (UTC)", "from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133])\n\tby ash.osuosl.org (Postfix) with ESMTP id 346EA1BF5A1\n\tfor <intel-wired-lan@lists.osuosl.org>;\n\tWed, 16 Oct 2019 23:32:14 +0000 (UTC)", "from localhost (localhost [127.0.0.1])\n\tby hemlock.osuosl.org (Postfix) with ESMTP id 20370877E1\n\tfor <intel-wired-lan@lists.osuosl.org>;\n\tWed, 16 Oct 2019 23:32:14 +0000 (UTC)", "from hemlock.osuosl.org ([127.0.0.1])\n\tby localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024)\n\twith ESMTP id nADztKny7ZcQ for <intel-wired-lan@lists.osuosl.org>;\n\tWed, 16 Oct 2019 23:32:08 +0000 (UTC)", "from mga04.intel.com (mga04.intel.com [192.55.52.120])\n\tby hemlock.osuosl.org (Postfix) with ESMTPS id 02F4A88524\n\tfor <intel-wired-lan@lists.osuosl.org>;\n\tWed, 16 Oct 2019 23:32:07 +0000 (UTC)", "from orsmga007.jf.intel.com ([10.7.209.58])\n\tby fmsmga104.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384;\n\t16 Oct 2019 16:32:07 -0700", "from unknown (HELO localhost.jf.intel.com) ([10.166.244.174])\n\tby orsmga007.jf.intel.com with ESMTP; 16 Oct 2019 16:32:06 -0700" ], "X-Virus-Scanned": [ "amavisd-new at osuosl.org", "amavisd-new at osuosl.org" ], "X-Greylist": "domain auto-whitelisted by SQLgrey-1.7.6", "X-Amp-Result": "SKIPPED(no attachment in message)", "X-Amp-File-Uploaded": "False", "X-ExtLoop1": "1", "X-IronPort-AV": "E=Sophos;i=\"5.67,305,1566889200\"; d=\"scan'208\";a=\"186310791\"", "From": "Tony Nguyen <anthony.l.nguyen@intel.com>", "To": "intel-wired-lan@lists.osuosl.org", "Date": "Wed, 16 Oct 2019 08:01:55 -0700", "Message-Id": "<20191016150201.41597-3-anthony.l.nguyen@intel.com>", "X-Mailer": "git-send-email 2.20.1", "In-Reply-To": "<20191016150201.41597-1-anthony.l.nguyen@intel.com>", "References": "<20191016150201.41597-1-anthony.l.nguyen@intel.com>", "MIME-Version": "1.0", "Subject": "[Intel-wired-lan] [PATCH S30 v2 3/9] ice: Add support for XDP", "X-BeenThere": "intel-wired-lan@osuosl.org", "X-Mailman-Version": "2.1.29", "Precedence": "list", "List-Id": "Intel Wired Ethernet Linux Kernel Driver Development\n\t<intel-wired-lan.osuosl.org>", "List-Unsubscribe": "<https://lists.osuosl.org/mailman/options/intel-wired-lan>, \n\t<mailto:intel-wired-lan-request@osuosl.org?subject=unsubscribe>", "List-Archive": "<http://lists.osuosl.org/pipermail/intel-wired-lan/>", "List-Post": "<mailto:intel-wired-lan@osuosl.org>", "List-Help": "<mailto:intel-wired-lan-request@osuosl.org?subject=help>", "List-Subscribe": "<https://lists.osuosl.org/mailman/listinfo/intel-wired-lan>, \n\t<mailto:intel-wired-lan-request@osuosl.org?subject=subscribe>", "Content-Type": "text/plain; charset=\"us-ascii\"", "Content-Transfer-Encoding": "7bit", "Errors-To": "intel-wired-lan-bounces@osuosl.org", "Sender": "\"Intel-wired-lan\" <intel-wired-lan-bounces@osuosl.org>" }, "content": "From: Maciej Fijalkowski <maciej.fijalkowski@intel.com>\n\nAdd support for XDP. Implement ndo_bpf and ndo_xdp_xmit. Upon load of\nan XDP program, allocate additional Tx rings for dedicated XDP use.\nThe following actions are supported: XDP_TX, XDP_DROP, XDP_REDIRECT,\nXDP_PASS, and XDP_ABORTED.\n\nSigned-off-by: Maciej Fijalkowski <maciej.fijalkowski@intel.com>\nSigned-off-by: Tony Nguyen <anthony.l.nguyen@intel.com>\n---\n drivers/net/ethernet/intel/ice/ice.h | 24 +-\n drivers/net/ethernet/intel/ice/ice_base.c | 28 +-\n drivers/net/ethernet/intel/ice/ice_ethtool.c | 50 ++-\n drivers/net/ethernet/intel/ice/ice_lib.c | 69 +++-\n drivers/net/ethernet/intel/ice/ice_lib.h | 6 +\n drivers/net/ethernet/intel/ice/ice_main.c | 319 +++++++++++++++++\n drivers/net/ethernet/intel/ice/ice_txrx.c | 351 ++++++++++++++++---\n drivers/net/ethernet/intel/ice/ice_txrx.h | 34 +-\n 8 files changed, 822 insertions(+), 59 deletions(-)", "diff": "diff --git a/drivers/net/ethernet/intel/ice/ice.h b/drivers/net/ethernet/intel/ice/ice.h\nindex 42eaae2f356d..c9eceb796861 100644\n--- a/drivers/net/ethernet/intel/ice/ice.h\n+++ b/drivers/net/ethernet/intel/ice/ice.h\n@@ -29,8 +29,10 @@\n #include <linux/ip.h>\n #include <linux/sctp.h>\n #include <linux/ipv6.h>\n+#include <linux/pkt_sched.h>\n #include <linux/if_bridge.h>\n #include <linux/ctype.h>\n+#include <linux/bpf.h>\n #include <linux/avf/virtchnl.h>\n #include <linux/mfd/core.h>\n #include <net/ipv6.h>\n@@ -81,8 +83,7 @@ extern const char ice_drv_ver[];\n \n #define ICE_DFLT_NETIF_M (NETIF_MSG_DRV | NETIF_MSG_PROBE | NETIF_MSG_LINK)\n \n-#define ICE_MAX_MTU\t(ICE_AQ_SET_MAC_FRAME_SIZE_MAX - \\\n-\t\t\t(ETH_HLEN + ETH_FCS_LEN + (VLAN_HLEN * 2)))\n+#define ICE_MAX_MTU\t(ICE_AQ_SET_MAC_FRAME_SIZE_MAX - ICE_ETH_PKT_HDR_PAD)\n \n #define ICE_UP_TABLE_TRANSLATE(val, i) \\\n \t\t(((val) << ICE_AQ_VSI_UP_TABLE_UP##i##_S) & \\\n@@ -286,6 +287,10 @@ struct ice_vsi {\n \tu16 num_tx_desc;\n \tu16 qset_handle[ICE_MAX_TRAFFIC_CLASS];\n \tstruct ice_tc_cfg tc_cfg;\n+\tstruct bpf_prog *xdp_prog;\n+\tstruct ice_ring **xdp_rings;\t /* XDP ring array */\n+\tu16 num_xdp_txq;\t\t /* Used XDP queues */\n+\tu8 xdp_mapping_mode;\t\t /* ICE_MAP_MODE_[CONTIG|SCATTER] */\n } ____cacheline_internodealigned_in_smp;\n \n /* struct that defines an interrupt vector */\n@@ -438,6 +443,16 @@ static inline struct ice_pf *ice_netdev_to_pf(struct net_device *netdev)\n \treturn np->vsi->back;\n }\n \n+static inline bool ice_is_xdp_ena_vsi(struct ice_vsi *vsi)\n+{\n+\treturn !!vsi->xdp_prog;\n+}\n+\n+static inline void ice_set_ring_xdp(struct ice_ring *ring)\n+{\n+\tring->flags |= ICE_TX_FLAGS_RING_XDP;\n+}\n+\n /**\n * ice_get_main_vsi - Get the PF VSI\n * @pf: PF instance\n@@ -464,6 +479,11 @@ int ice_up(struct ice_vsi *vsi);\n int ice_down(struct ice_vsi *vsi);\n int ice_vsi_cfg(struct ice_vsi *vsi);\n struct ice_vsi *ice_lb_vsi_setup(struct ice_pf *pf, struct ice_port_info *pi);\n+int ice_prepare_xdp_rings(struct ice_vsi *vsi);\n+int ice_destroy_xdp_rings(struct ice_vsi *vsi);\n+int\n+ice_xdp_xmit(struct net_device *dev, int n, struct xdp_frame **frames,\n+\t u32 flags);\n int ice_set_rss(struct ice_vsi *vsi, u8 *seed, u8 *lut, u16 lut_size);\n int ice_get_rss(struct ice_vsi *vsi, u8 *seed, u8 *lut, u16 lut_size);\n void ice_fill_rss_lut(u8 *lut, u16 rss_table_size, u16 rss_size);\ndiff --git a/drivers/net/ethernet/intel/ice/ice_base.c b/drivers/net/ethernet/intel/ice/ice_base.c\nindex 1dd9c18ecd8c..e335256d483d 100644\n--- a/drivers/net/ethernet/intel/ice/ice_base.c\n+++ b/drivers/net/ethernet/intel/ice/ice_base.c\n@@ -198,6 +198,9 @@ static void ice_cfg_itr_gran(struct ice_hw *hw)\n */\n static u16 ice_calc_q_handle(struct ice_vsi *vsi, struct ice_ring *ring, u8 tc)\n {\n+\tWARN_ONCE(ice_ring_is_xdp(ring) && tc,\n+\t\t \"XDP ring can't belong to TC other than 0\");\n+\n \t/* idea here for calculation is that we subtract the number of queue\n \t * count from TC that ring belongs to from it's absolute queue index\n \t * and as a result we get the queue's index within TC.\n@@ -287,6 +290,22 @@ int ice_setup_rx_ctx(struct ice_ring *ring)\n \t/* clear the context structure first */\n \tmemset(&rlan_ctx, 0, sizeof(rlan_ctx));\n \n+\tring->rx_buf_len = vsi->rx_buf_len;\n+\n+\tif (ring->vsi->type == ICE_VSI_PF) {\n+\t\tif (!xdp_rxq_info_is_reg(&ring->xdp_rxq))\n+\t\t\txdp_rxq_info_reg(&ring->xdp_rxq, ring->netdev,\n+\t\t\t\t\t ring->q_index);\n+\n+\t\terr = xdp_rxq_info_reg_mem_model(&ring->xdp_rxq,\n+\t\t\t\t\t\t MEM_TYPE_PAGE_SHARED, NULL);\n+\t\tif (err)\n+\t\t\treturn err;\n+\t}\n+\t/* Receive Queue Base Address.\n+\t * Indicates the starting address of the descriptor queue defined in\n+\t * 128 Byte units.\n+\t */\n \trlan_ctx.base = ring->dma >> 7;\n \n \trlan_ctx.qlen = ring->count;\n@@ -294,7 +313,7 @@ int ice_setup_rx_ctx(struct ice_ring *ring)\n \t/* Receive Packet Data Buffer Size.\n \t * The Packet Data Buffer Size is defined in 128 byte units.\n \t */\n-\trlan_ctx.dbuf = vsi->rx_buf_len >> ICE_RLAN_CTX_DBUF_S;\n+\trlan_ctx.dbuf = ring->rx_buf_len >> ICE_RLAN_CTX_DBUF_S;\n \n \t/* use 32 byte descriptors */\n \trlan_ctx.dsize = 1;\n@@ -657,6 +676,13 @@ ice_cfg_txq_interrupt(struct ice_vsi *vsi, u16 txq, u16 msix_idx, u16 itr_idx)\n \t ((msix_idx << QINT_TQCTL_MSIX_INDX_S) & QINT_TQCTL_MSIX_INDX_M);\n \n \twr32(hw, QINT_TQCTL(vsi->txq_map[txq]), val);\n+\tif (ice_is_xdp_ena_vsi(vsi)) {\n+\t\tu32 xdp_txq = txq + vsi->num_xdp_txq;\n+\n+\t\twr32(hw, QINT_TQCTL(vsi->txq_map[xdp_txq]),\n+\t\t val);\n+\t}\n+\tice_flush(hw);\n }\n \n /**\ndiff --git a/drivers/net/ethernet/intel/ice/ice_ethtool.c b/drivers/net/ethernet/intel/ice/ice_ethtool.c\nindex 7e23034df955..6cee99b5865b 100644\n--- a/drivers/net/ethernet/intel/ice/ice_ethtool.c\n+++ b/drivers/net/ethernet/intel/ice/ice_ethtool.c\n@@ -2577,6 +2577,7 @@ ice_set_ringparam(struct net_device *netdev, struct ethtool_ringparam *ring)\n {\n \tstruct ice_ring *tx_rings = NULL, *rx_rings = NULL;\n \tstruct ice_netdev_priv *np = netdev_priv(netdev);\n+\tstruct ice_ring *xdp_rings = NULL;\n \tstruct ice_vsi *vsi = np->vsi;\n \tstruct ice_pf *pf = vsi->back;\n \tint i, timeout = 50, err = 0;\n@@ -2624,6 +2625,11 @@ ice_set_ringparam(struct net_device *netdev, struct ethtool_ringparam *ring)\n \t\t\tvsi->tx_rings[i]->count = new_tx_cnt;\n \t\tfor (i = 0; i < vsi->alloc_rxq; i++)\n \t\t\tvsi->rx_rings[i]->count = new_rx_cnt;\n+\t\tif (ice_is_xdp_ena_vsi(vsi))\n+\t\t\tfor (i = 0; i < vsi->num_xdp_txq; i++)\n+\t\t\t\tvsi->xdp_rings[i]->count = new_tx_cnt;\n+\t\tvsi->num_tx_desc = new_tx_cnt;\n+\t\tvsi->num_rx_desc = new_rx_cnt;\n \t\tnetdev_dbg(netdev, \"Link is down, descriptor count change happens when link is brought up\\n\");\n \t\tgoto done;\n \t}\n@@ -2650,15 +2656,43 @@ ice_set_ringparam(struct net_device *netdev, struct ethtool_ringparam *ring)\n \t\ttx_rings[i].tx_buf = NULL;\n \t\terr = ice_setup_tx_ring(&tx_rings[i]);\n \t\tif (err) {\n-\t\t\twhile (i) {\n-\t\t\t\ti--;\n+\t\t\twhile (i--)\n \t\t\t\tice_clean_tx_ring(&tx_rings[i]);\n-\t\t\t}\n \t\t\tdevm_kfree(&pf->pdev->dev, tx_rings);\n \t\t\tgoto done;\n \t\t}\n \t}\n \n+\tif (!ice_is_xdp_ena_vsi(vsi))\n+\t\tgoto process_rx;\n+\n+\t/* alloc updated XDP resources */\n+\tnetdev_info(netdev, \"Changing XDP descriptor count from %d to %d\\n\",\n+\t\t vsi->xdp_rings[0]->count, new_tx_cnt);\n+\n+\txdp_rings = devm_kcalloc(&pf->pdev->dev, vsi->num_xdp_txq,\n+\t\t\t\t sizeof(*xdp_rings), GFP_KERNEL);\n+\tif (!xdp_rings) {\n+\t\terr = -ENOMEM;\n+\t\tgoto free_tx;\n+\t}\n+\n+\tfor (i = 0; i < vsi->num_xdp_txq; i++) {\n+\t\t/* clone ring and setup updated count */\n+\t\txdp_rings[i] = *vsi->xdp_rings[i];\n+\t\txdp_rings[i].count = new_tx_cnt;\n+\t\txdp_rings[i].desc = NULL;\n+\t\txdp_rings[i].tx_buf = NULL;\n+\t\terr = ice_setup_tx_ring(&xdp_rings[i]);\n+\t\tif (err) {\n+\t\t\twhile (i--)\n+\t\t\t\tice_clean_tx_ring(&xdp_rings[i]);\n+\t\t\tdevm_kfree(&pf->pdev->dev, xdp_rings);\n+\t\t\tgoto free_tx;\n+\t\t}\n+\t\tice_set_ring_xdp(&xdp_rings[i]);\n+\t}\n+\n process_rx:\n \tif (new_rx_cnt == vsi->rx_rings[0]->count)\n \t\tgoto process_link;\n@@ -2737,6 +2771,16 @@ ice_set_ringparam(struct net_device *netdev, struct ethtool_ringparam *ring)\n \t\t\tdevm_kfree(&pf->pdev->dev, rx_rings);\n \t\t}\n \n+\t\tif (xdp_rings) {\n+\t\t\tfor (i = 0; i < vsi->num_xdp_txq; i++) {\n+\t\t\t\tice_free_tx_ring(vsi->xdp_rings[i]);\n+\t\t\t\t*vsi->xdp_rings[i] = xdp_rings[i];\n+\t\t\t}\n+\t\t\tdevm_kfree(&pf->pdev->dev, xdp_rings);\n+\t\t}\n+\n+\t\tvsi->num_tx_desc = new_tx_cnt;\n+\t\tvsi->num_rx_desc = new_rx_cnt;\n \t\tice_up(vsi);\n \t}\n \tgoto done;\ndiff --git a/drivers/net/ethernet/intel/ice/ice_lib.c b/drivers/net/ethernet/intel/ice/ice_lib.c\nindex 74c3bd191671..d918dbd071db 100644\n--- a/drivers/net/ethernet/intel/ice/ice_lib.c\n+++ b/drivers/net/ethernet/intel/ice/ice_lib.c\n@@ -46,7 +46,8 @@ static int ice_vsi_alloc_arrays(struct ice_vsi *vsi)\n \tif (!vsi->rx_rings)\n \t\tgoto err_rings;\n \n-\tvsi->txq_map = devm_kcalloc(&pf->pdev->dev, vsi->alloc_txq,\n+\t/* XDP will have vsi->alloc_txq Tx queues as well, so double the size */\n+\tvsi->txq_map = devm_kcalloc(&pf->pdev->dev, (2 * vsi->alloc_txq),\n \t\t\t\t sizeof(*vsi->txq_map), GFP_KERNEL);\n \n \tif (!vsi->txq_map)\n@@ -1218,6 +1219,20 @@ int ice_vsi_kill_vlan(struct ice_vsi *vsi, u16 vid)\n \treturn err;\n }\n \n+/**\n+ * ice_vsi_cfg_frame_size - setup max frame size and Rx buffer length\n+ * @vsi: VSI\n+ */\n+void ice_vsi_cfg_frame_size(struct ice_vsi *vsi)\n+{\n+\tif (vsi->netdev && vsi->netdev->mtu > ETH_DATA_LEN)\n+\t\tvsi->max_frame = vsi->netdev->mtu + ICE_ETH_PKT_HDR_PAD;\n+\telse\n+\t\tvsi->max_frame = ICE_RXBUF_2048;\n+\n+\tvsi->rx_buf_len = ICE_RXBUF_2048;\n+}\n+\n /**\n * ice_vsi_cfg_rxqs - Configure the VSI for Rx\n * @vsi: the VSI being configured\n@@ -1232,13 +1247,7 @@ int ice_vsi_cfg_rxqs(struct ice_vsi *vsi)\n \tif (vsi->type == ICE_VSI_VF)\n \t\tgoto setup_rings;\n \n-\tif (vsi->netdev && vsi->netdev->mtu > ETH_DATA_LEN)\n-\t\tvsi->max_frame = vsi->netdev->mtu +\n-\t\t\tETH_HLEN + ETH_FCS_LEN + VLAN_HLEN;\n-\telse\n-\t\tvsi->max_frame = ICE_RXBUF_2048;\n-\n-\tvsi->rx_buf_len = ICE_RXBUF_2048;\n+\tice_vsi_cfg_frame_size(vsi);\n setup_rings:\n \t/* set up individual rings */\n \tfor (i = 0; i < vsi->num_rxq; i++) {\n@@ -1300,6 +1309,18 @@ int ice_vsi_cfg_lan_txqs(struct ice_vsi *vsi)\n \treturn ice_vsi_cfg_txqs(vsi, vsi->tx_rings);\n }\n \n+/**\n+ * ice_vsi_cfg_xdp_txqs - Configure Tx queues dedicated for XDP in given VSI\n+ * @vsi: the VSI being configured\n+ *\n+ * Return 0 on success and a negative value on error\n+ * Configure the Tx queues dedicated for XDP in given VSI for operation.\n+ */\n+int ice_vsi_cfg_xdp_txqs(struct ice_vsi *vsi)\n+{\n+\treturn ice_vsi_cfg_txqs(vsi, vsi->xdp_rings);\n+}\n+\n /**\n * ice_intrl_usec_to_reg - convert interrupt rate limit to register value\n * @intrl: interrupt rate limit in usecs\n@@ -1523,6 +1544,15 @@ ice_vsi_stop_lan_tx_rings(struct ice_vsi *vsi, enum ice_disq_rst_src rst_src,\n \treturn ice_vsi_stop_tx_rings(vsi, rst_src, rel_vmvf_num, vsi->tx_rings);\n }\n \n+/**\n+ * ice_vsi_stop_xdp_tx_rings - Disable XDP Tx rings\n+ * @vsi: the VSI being configured\n+ */\n+int ice_vsi_stop_xdp_tx_rings(struct ice_vsi *vsi)\n+{\n+\treturn ice_vsi_stop_tx_rings(vsi, ICE_NO_RESET, 0, vsi->xdp_rings);\n+}\n+\n /**\n * ice_cfg_vlan_pruning - enable or disable VLAN pruning on the VSI\n * @vsi: VSI to enable or disable VLAN pruning on\n@@ -1920,6 +1950,11 @@ static void ice_vsi_release_msix(struct ice_vsi *vsi)\n \t\twr32(hw, GLINT_ITR(ICE_IDX_ITR1, reg_idx), 0);\n \t\tfor (q = 0; q < q_vector->num_ring_tx; q++) {\n \t\t\twr32(hw, QINT_TQCTL(vsi->txq_map[txq]), 0);\n+\t\t\tif (ice_is_xdp_ena_vsi(vsi)) {\n+\t\t\t\tu32 xdp_txq = txq + vsi->num_xdp_txq;\n+\n+\t\t\t\twr32(hw, QINT_TQCTL(vsi->txq_map[xdp_txq]), 0);\n+\t\t\t}\n \t\t\ttxq++;\n \t\t}\n \n@@ -2302,6 +2337,11 @@ int ice_vsi_rebuild(struct ice_vsi *vsi)\n \t\tvsi->base_vector = 0;\n \t}\n \n+\tif (ice_is_xdp_ena_vsi(vsi))\n+\t\t/* return value check can be skipped here, it always returns\n+\t\t * 0 if reset is in progress\n+\t\t */\n+\t\tice_destroy_xdp_rings(vsi);\n \tice_vsi_put_qs(vsi);\n \tice_vsi_clear_rings(vsi);\n \tice_vsi_free_arrays(vsi);\n@@ -2342,6 +2382,13 @@ int ice_vsi_rebuild(struct ice_vsi *vsi)\n \t\t\tgoto err_vectors;\n \n \t\tice_vsi_map_rings_to_vectors(vsi);\n+\t\tif (ice_is_xdp_ena_vsi(vsi)) {\n+\t\t\tvsi->num_xdp_txq = vsi->alloc_txq;\n+\t\t\tvsi->xdp_mapping_mode = ICE_VSI_MAP_CONTIG;\n+\t\t\tret = ice_prepare_xdp_rings(vsi);\n+\t\t\tif (ret)\n+\t\t\t\tgoto err_vectors;\n+\t\t}\n \t\t/* Do not exit if configuring RSS had an issue, at least\n \t\t * receive traffic on first queue. Hence no need to capture\n \t\t * return value\n@@ -2368,9 +2415,13 @@ int ice_vsi_rebuild(struct ice_vsi *vsi)\n \t}\n \n \t/* configure VSI nodes based on number of queues and TC's */\n-\tfor (i = 0; i < vsi->tc_cfg.numtc; i++)\n+\tfor (i = 0; i < vsi->tc_cfg.numtc; i++) {\n \t\tmax_txqs[i] = vsi->alloc_txq;\n \n+\t\tif (ice_is_xdp_ena_vsi(vsi))\n+\t\t\tmax_txqs[i] += vsi->num_xdp_txq;\n+\t}\n+\n \tstatus = ice_cfg_vsi_lan(vsi->port_info, vsi->idx, vsi->tc_cfg.ena_tc,\n \t\t\t\t max_txqs);\n \tif (status) {\ndiff --git a/drivers/net/ethernet/intel/ice/ice_lib.h b/drivers/net/ethernet/intel/ice/ice_lib.h\nindex 9919d3f810e3..53758ec71d3a 100644\n--- a/drivers/net/ethernet/intel/ice/ice_lib.h\n+++ b/drivers/net/ethernet/intel/ice/ice_lib.h\n@@ -38,6 +38,10 @@ int\n ice_vsi_stop_lan_tx_rings(struct ice_vsi *vsi, enum ice_disq_rst_src rst_src,\n \t\t\t u16 rel_vmvf_num);\n \n+int ice_vsi_cfg_xdp_txqs(struct ice_vsi *vsi);\n+\n+int ice_vsi_stop_xdp_tx_rings(struct ice_vsi *vsi);\n+\n int ice_cfg_vlan_pruning(struct ice_vsi *vsi, bool ena, bool vlan_promisc);\n \n void ice_cfg_sw_lldp(struct ice_vsi *vsi, bool tx, bool create);\n@@ -81,6 +85,8 @@ void ice_vsi_free_tx_rings(struct ice_vsi *vsi);\n \n int ice_vsi_manage_rss_lut(struct ice_vsi *vsi, bool ena);\n \n+void ice_vsi_cfg_frame_size(struct ice_vsi *vsi);\n+\n u32 ice_intrl_usec_to_reg(u8 intrl, u8 gran);\n \n char *ice_nvm_version_str(struct ice_hw *hw);\ndiff --git a/drivers/net/ethernet/intel/ice/ice_main.c b/drivers/net/ethernet/intel/ice/ice_main.c\nindex d2abffe827ef..605f28e0f4e3 100644\n--- a/drivers/net/ethernet/intel/ice/ice_main.c\n+++ b/drivers/net/ethernet/intel/ice/ice_main.c\n@@ -1703,6 +1703,302 @@ static int ice_vsi_req_irq_msix(struct ice_vsi *vsi, char *basename)\n \treturn err;\n }\n \n+/**\n+ * ice_xdp_alloc_setup_rings - Allocate and setup Tx rings for XDP\n+ * @vsi: VSI to setup Tx rings used by XDP\n+ *\n+ * Return 0 on success and negative value on error\n+ */\n+static int ice_xdp_alloc_setup_rings(struct ice_vsi *vsi)\n+{\n+\tstruct device *dev = &vsi->back->pdev->dev;\n+\tint i;\n+\n+\tfor (i = 0; i < vsi->num_xdp_txq; i++) {\n+\t\tu16 xdp_q_idx = vsi->alloc_txq + i;\n+\t\tstruct ice_ring *xdp_ring;\n+\n+\t\txdp_ring = kzalloc(sizeof(*xdp_ring), GFP_KERNEL);\n+\n+\t\tif (!xdp_ring)\n+\t\t\tgoto free_xdp_rings;\n+\n+\t\txdp_ring->q_index = xdp_q_idx;\n+\t\txdp_ring->reg_idx = vsi->txq_map[xdp_q_idx];\n+\t\txdp_ring->ring_active = false;\n+\t\txdp_ring->vsi = vsi;\n+\t\txdp_ring->netdev = NULL;\n+\t\txdp_ring->dev = dev;\n+\t\txdp_ring->count = vsi->num_tx_desc;\n+\t\tvsi->xdp_rings[i] = xdp_ring;\n+\t\tif (ice_setup_tx_ring(xdp_ring))\n+\t\t\tgoto free_xdp_rings;\n+\t\tice_set_ring_xdp(xdp_ring);\n+\t}\n+\n+\treturn 0;\n+\n+free_xdp_rings:\n+\tfor (; i >= 0; i--)\n+\t\tif (vsi->xdp_rings[i] && vsi->xdp_rings[i]->desc)\n+\t\t\tice_free_tx_ring(vsi->xdp_rings[i]);\n+\treturn -ENOMEM;\n+}\n+\n+/**\n+ * ice_prepare_xdp_rings - Allocate, configure and setup Tx rings for XDP\n+ * @vsi: VSI to bring up Tx rings used by XDP\n+ *\n+ * Return 0 on success and negative value on error\n+ */\n+int ice_prepare_xdp_rings(struct ice_vsi *vsi)\n+{\n+\tu16 max_txqs[ICE_MAX_TRAFFIC_CLASS] = { 0 };\n+\tint xdp_rings_rem = vsi->num_xdp_txq;\n+\tstruct ice_pf *pf = vsi->back;\n+\tstruct ice_qs_cfg xdp_qs_cfg = {\n+\t\t.qs_mutex = &pf->avail_q_mutex,\n+\t\t.pf_map = pf->avail_txqs,\n+\t\t.pf_map_size = pf->max_pf_txqs,\n+\t\t.q_count = vsi->num_xdp_txq,\n+\t\t.scatter_count = ICE_MAX_SCATTER_TXQS,\n+\t\t.vsi_map = vsi->txq_map,\n+\t\t.vsi_map_offset = vsi->alloc_txq,\n+\t\t.mapping_mode = vsi->xdp_mapping_mode\n+\t};\n+\tenum ice_status status;\n+\tint i, v_idx;\n+\n+\tvsi->xdp_rings = devm_kcalloc(&pf->pdev->dev, vsi->num_xdp_txq,\n+\t\t\t\t sizeof(*vsi->xdp_rings), GFP_KERNEL);\n+\tif (!vsi->xdp_rings)\n+\t\treturn -ENOMEM;\n+\n+\tif (__ice_vsi_get_qs(&xdp_qs_cfg))\n+\t\tgoto err_map_xdp;\n+\n+\tif (ice_xdp_alloc_setup_rings(vsi))\n+\t\tgoto clear_xdp_rings;\n+\n+\t/* follow the logic from ice_vsi_map_rings_to_vectors */\n+\tice_for_each_q_vector(vsi, v_idx) {\n+\t\tstruct ice_q_vector *q_vector = vsi->q_vectors[v_idx];\n+\t\tint xdp_rings_per_v, q_id, q_base;\n+\n+\t\txdp_rings_per_v = DIV_ROUND_UP(xdp_rings_rem,\n+\t\t\t\t\t vsi->num_q_vectors - v_idx);\n+\t\tq_base = vsi->num_xdp_txq - xdp_rings_rem;\n+\n+\t\tfor (q_id = q_base; q_id < (q_base + xdp_rings_per_v); q_id++) {\n+\t\t\tstruct ice_ring *xdp_ring = vsi->xdp_rings[q_id];\n+\n+\t\t\txdp_ring->q_vector = q_vector;\n+\t\t\txdp_ring->next = q_vector->tx.ring;\n+\t\t\tq_vector->tx.ring = xdp_ring;\n+\t\t}\n+\t\txdp_rings_rem -= xdp_rings_per_v;\n+\t}\n+\n+\t/* omit the scheduler update if in reset path; XDP queues will be\n+\t * taken into account at the end of ice_vsi_rebuild, where\n+\t * ice_cfg_vsi_lan is being called\n+\t */\n+\tif (ice_is_reset_in_progress(pf->state))\n+\t\treturn 0;\n+\n+\t/* tell the Tx scheduler that right now we have\n+\t * additional queues\n+\t */\n+\tfor (i = 0; i < vsi->tc_cfg.numtc; i++)\n+\t\tmax_txqs[i] = vsi->num_txq + vsi->num_xdp_txq;\n+\n+\tstatus = ice_cfg_vsi_lan(vsi->port_info, vsi->idx, vsi->tc_cfg.ena_tc,\n+\t\t\t\t max_txqs);\n+\tif (status) {\n+\t\tdev_err(&pf->pdev->dev,\n+\t\t\t\"Failed VSI LAN queue config for XDP, error:%d\\n\",\n+\t\t\tstatus);\n+\t\tgoto clear_xdp_rings;\n+\t}\n+\n+\treturn 0;\n+clear_xdp_rings:\n+\tfor (i = 0; i < vsi->num_xdp_txq; i++)\n+\t\tif (vsi->xdp_rings[i]) {\n+\t\t\tkfree_rcu(vsi->xdp_rings[i], rcu);\n+\t\t\tvsi->xdp_rings[i] = NULL;\n+\t\t}\n+\n+err_map_xdp:\n+\tmutex_lock(&pf->avail_q_mutex);\n+\tfor (i = 0; i < vsi->num_xdp_txq; i++) {\n+\t\tclear_bit(vsi->txq_map[i + vsi->alloc_txq], pf->avail_txqs);\n+\t\tvsi->txq_map[i + vsi->alloc_txq] = ICE_INVAL_Q_INDEX;\n+\t}\n+\tmutex_unlock(&pf->avail_q_mutex);\n+\n+\tdevm_kfree(&pf->pdev->dev, vsi->xdp_rings);\n+\treturn -ENOMEM;\n+}\n+\n+/**\n+ * ice_destroy_xdp_rings - undo the configuration made by ice_prepare_xdp_rings\n+ * @vsi: VSI to remove XDP rings\n+ *\n+ * Detach XDP rings from irq vectors, clean up the PF bitmap and free\n+ * resources\n+ */\n+int ice_destroy_xdp_rings(struct ice_vsi *vsi)\n+{\n+\tu16 max_txqs[ICE_MAX_TRAFFIC_CLASS] = { 0 };\n+\tstruct ice_pf *pf = vsi->back;\n+\tint i, v_idx;\n+\n+\t/* q_vectors are freed in reset path so there's no point in detaching\n+\t * rings; in case of rebuild being triggered not from reset reset bits\n+\t * in pf->state won't be set, so additionally check first q_vector\n+\t * against NULL\n+\t */\n+\tif (ice_is_reset_in_progress(pf->state) || !vsi->q_vectors[0])\n+\t\tgoto free_qmap;\n+\n+\tice_for_each_q_vector(vsi, v_idx) {\n+\t\tstruct ice_q_vector *q_vector = vsi->q_vectors[v_idx];\n+\t\tstruct ice_ring *ring;\n+\n+\t\tice_for_each_ring(ring, q_vector->tx)\n+\t\t\tif (!ring->tx_buf || !ice_ring_is_xdp(ring))\n+\t\t\t\tbreak;\n+\n+\t\t/* restore the value of last node prior to XDP setup */\n+\t\tq_vector->tx.ring = ring;\n+\t}\n+\n+free_qmap:\n+\tmutex_lock(&pf->avail_q_mutex);\n+\tfor (i = 0; i < vsi->num_xdp_txq; i++) {\n+\t\tclear_bit(vsi->txq_map[i + vsi->alloc_txq], pf->avail_txqs);\n+\t\tvsi->txq_map[i + vsi->alloc_txq] = ICE_INVAL_Q_INDEX;\n+\t}\n+\tmutex_unlock(&pf->avail_q_mutex);\n+\n+\tfor (i = 0; i < vsi->num_xdp_txq; i++)\n+\t\tif (vsi->xdp_rings[i]) {\n+\t\t\tif (vsi->xdp_rings[i]->desc)\n+\t\t\t\tice_free_tx_ring(vsi->xdp_rings[i]);\n+\t\t\tkfree_rcu(vsi->xdp_rings[i], rcu);\n+\t\t\tvsi->xdp_rings[i] = NULL;\n+\t\t}\n+\n+\tdevm_kfree(&pf->pdev->dev, vsi->xdp_rings);\n+\tvsi->xdp_rings = NULL;\n+\n+\tif (ice_is_reset_in_progress(pf->state) || !vsi->q_vectors[0])\n+\t\treturn 0;\n+\n+\t/* notify Tx scheduler that we destroyed XDP queues and bring\n+\t * back the old number of child nodes\n+\t */\n+\tfor (i = 0; i < vsi->tc_cfg.numtc; i++)\n+\t\tmax_txqs[i] = vsi->num_txq;\n+\n+\treturn ice_cfg_vsi_lan(vsi->port_info, vsi->idx, vsi->tc_cfg.ena_tc,\n+\t\t\t max_txqs);\n+}\n+\n+/**\n+ * ice_xdp_setup_prog - Add or remove XDP eBPF program\n+ * @vsi: VSI to setup XDP for\n+ * @prog: XDP program\n+ * @extack: netlink extended ack\n+ */\n+static int\n+ice_xdp_setup_prog(struct ice_vsi *vsi, struct bpf_prog *prog,\n+\t\t struct netlink_ext_ack *extack)\n+{\n+\tint frame_size = vsi->netdev->mtu + ICE_ETH_PKT_HDR_PAD;\n+\tbool if_running = netif_running(vsi->netdev);\n+\tstruct bpf_prog *old_prog;\n+\tint i, ret = 0;\n+\n+\tif (frame_size > vsi->rx_buf_len) {\n+\t\tNL_SET_ERR_MSG_MOD(extack, \"MTU too large for loading XDP\");\n+\t\treturn -ENOTSUPP;\n+\t}\n+\n+\tif (!ice_is_xdp_ena_vsi(vsi) && !prog)\n+\t\treturn 0;\n+\n+\t/* need to stop netdev while setting up the program for Rx rings */\n+\tif (if_running && !test_and_set_bit(__ICE_DOWN, vsi->state)) {\n+\t\tret = ice_down(vsi);\n+\t\tif (ret) {\n+\t\t\tNL_SET_ERR_MSG_MOD(extack,\n+\t\t\t\t\t \"Preparing device for XDP attach failed\");\n+\t\t\tgoto skip_setting_prog;\n+\t\t}\n+\t}\n+\n+\tif (!ice_is_xdp_ena_vsi(vsi) && prog) {\n+\t\tvsi->num_xdp_txq = vsi->alloc_txq;\n+\t\tvsi->xdp_mapping_mode = ICE_VSI_MAP_CONTIG;\n+\t\tif (ice_prepare_xdp_rings(vsi)) {\n+\t\t\tNL_SET_ERR_MSG_MOD(extack,\n+\t\t\t\t\t \"Setting up XDP Tx resources failed\");\n+\t\t\tret = -ENOMEM;\n+\t\t\tgoto skip_setting_prog;\n+\t\t}\n+\t} else if (ice_is_xdp_ena_vsi(vsi) && !prog) {\n+\t\tif (ice_destroy_xdp_rings(vsi)) {\n+\t\t\tNL_SET_ERR_MSG_MOD(extack,\n+\t\t\t\t\t \"Freeing XDP Tx resources failed\");\n+\t\t\tret = -ENOMEM;\n+\t\t\tgoto skip_setting_prog;\n+\t\t}\n+\t}\n+\n+\told_prog = xchg(&vsi->xdp_prog, prog);\n+\tif (old_prog)\n+\t\tbpf_prog_put(old_prog);\n+\n+\tice_for_each_rxq(vsi, i)\n+\t\tWRITE_ONCE(vsi->rx_rings[i]->xdp_prog, vsi->xdp_prog);\n+\n+\tif (if_running)\n+\t\tret = ice_up(vsi);\n+\n+skip_setting_prog:\n+\treturn ret;\n+}\n+\n+/**\n+ * ice_xdp - implements XDP handler\n+ * @dev: netdevice\n+ * @xdp: XDP command\n+ */\n+static int ice_xdp(struct net_device *dev, struct netdev_bpf *xdp)\n+{\n+\tstruct ice_netdev_priv *np = netdev_priv(dev);\n+\tstruct ice_vsi *vsi = np->vsi;\n+\n+\tif (vsi->type != ICE_VSI_PF) {\n+\t\tNL_SET_ERR_MSG_MOD(xdp->extack,\n+\t\t\t\t \"XDP can be loaded only on PF VSI\");\n+\t\treturn -EINVAL;\n+\t}\n+\n+\tswitch (xdp->command) {\n+\tcase XDP_SETUP_PROG:\n+\t\treturn ice_xdp_setup_prog(vsi, xdp->prog, xdp->extack);\n+\tcase XDP_QUERY_PROG:\n+\t\txdp->prog_id = vsi->xdp_prog ? vsi->xdp_prog->aux->id : 0;\n+\t\treturn 0;\n+\tdefault:\n+\t\tNL_SET_ERR_MSG_MOD(xdp->extack, \"Unknown XDP command\");\n+\t\treturn -EINVAL;\n+\t}\n+}\n+\n /**\n * ice_ena_misc_vector - enable the non-queue interrupts\n * @pf: board private structure\n@@ -2262,6 +2558,8 @@ static int ice_setup_pf_sw(struct ice_pf *pf)\n \t\tstatus = -ENODEV;\n \t\tgoto unroll_vsi_setup;\n \t}\n+\t/* netdev has to be configured before setting frame size */\n+\tice_vsi_cfg_frame_size(vsi);\n \n \t/* registering the NAPI handler requires both the queues and\n \t * netdev to be created, which are done in ice_pf_vsi_setup()\n@@ -3613,6 +3911,8 @@ int ice_vsi_cfg(struct ice_vsi *vsi)\n \tice_vsi_cfg_dcb_rings(vsi);\n \n \terr = ice_vsi_cfg_lan_txqs(vsi);\n+\tif (!err && ice_is_xdp_ena_vsi(vsi))\n+\t\terr = ice_vsi_cfg_xdp_txqs(vsi);\n \tif (!err)\n \t\terr = ice_vsi_cfg_rxqs(vsi);\n \n@@ -4028,6 +4328,13 @@ int ice_down(struct ice_vsi *vsi)\n \t\tnetdev_err(vsi->netdev,\n \t\t\t \"Failed stop Tx rings, VSI %d error %d\\n\",\n \t\t\t vsi->vsi_num, tx_err);\n+\tif (!tx_err && ice_is_xdp_ena_vsi(vsi)) {\n+\t\ttx_err = ice_vsi_stop_xdp_tx_rings(vsi);\n+\t\tif (tx_err)\n+\t\t\tnetdev_err(vsi->netdev,\n+\t\t\t\t \"Failed stop XDP rings, VSI %d error %d\\n\",\n+\t\t\t\t vsi->vsi_num, tx_err);\n+\t}\n \n \trx_err = ice_vsi_stop_rx_rings(vsi);\n \tif (rx_err)\n@@ -4466,6 +4773,16 @@ static int ice_change_mtu(struct net_device *netdev, int new_mtu)\n \t\treturn 0;\n \t}\n \n+\tif (ice_is_xdp_ena_vsi(vsi)) {\n+\t\tint frame_size = ICE_RXBUF_2048 - XDP_PACKET_HEADROOM;\n+\n+\t\tif (new_mtu + ICE_ETH_PKT_HDR_PAD > frame_size) {\n+\t\t\tnetdev_err(netdev, \"max MTU for XDP usage is %d\\n\",\n+\t\t\t\t frame_size);\n+\t\t\treturn -EINVAL;\n+\t\t}\n+\t}\n+\n \tif (new_mtu < netdev->min_mtu) {\n \t\tnetdev_err(netdev, \"new MTU invalid. min_mtu is %d\\n\",\n \t\t\t netdev->min_mtu);\n@@ -5004,4 +5321,6 @@ static const struct net_device_ops ice_netdev_ops = {\n \t.ndo_fdb_add = ice_fdb_add,\n \t.ndo_fdb_del = ice_fdb_del,\n \t.ndo_tx_timeout = ice_tx_timeout,\n+\t.ndo_bpf = ice_xdp,\n+\t.ndo_xdp_xmit = ice_xdp_xmit,\n };\ndiff --git a/drivers/net/ethernet/intel/ice/ice_txrx.c b/drivers/net/ethernet/intel/ice/ice_txrx.c\nindex 33dd103035dc..69181b01df91 100644\n--- a/drivers/net/ethernet/intel/ice/ice_txrx.c\n+++ b/drivers/net/ethernet/intel/ice/ice_txrx.c\n@@ -5,6 +5,9 @@\n \n #include <linux/prefetch.h>\n #include <linux/mm.h>\n+#include <linux/bpf_trace.h>\n+#include <net/xdp.h>\n+#include \"ice_lib.h\"\n #include \"ice.h\"\n #include \"ice_dcb_lib.h\"\n \n@@ -19,7 +22,10 @@ static void\n ice_unmap_and_free_tx_buf(struct ice_ring *ring, struct ice_tx_buf *tx_buf)\n {\n \tif (tx_buf->skb) {\n-\t\tdev_kfree_skb_any(tx_buf->skb);\n+\t\tif (ice_ring_is_xdp(ring))\n+\t\t\tpage_frag_free(tx_buf->raw_buf);\n+\t\telse\n+\t\t\tdev_kfree_skb_any(tx_buf->skb);\n \t\tif (dma_unmap_len(tx_buf, len))\n \t\t\tdma_unmap_single(ring->dev,\n \t\t\t\t\t dma_unmap_addr(tx_buf, dma),\n@@ -136,8 +142,11 @@ static bool ice_clean_tx_irq(struct ice_ring *tx_ring, int napi_budget)\n \t\ttotal_bytes += tx_buf->bytecount;\n \t\ttotal_pkts += tx_buf->gso_segs;\n \n-\t\t/* free the skb */\n-\t\tnapi_consume_skb(tx_buf->skb, napi_budget);\n+\t\tif (ice_ring_is_xdp(tx_ring))\n+\t\t\tpage_frag_free(tx_buf->raw_buf);\n+\t\telse\n+\t\t\t/* free the skb */\n+\t\t\tnapi_consume_skb(tx_buf->skb, napi_budget);\n \n \t\t/* unmap skb header data */\n \t\tdma_unmap_single(tx_ring->dev,\n@@ -195,6 +204,9 @@ static bool ice_clean_tx_irq(struct ice_ring *tx_ring, int napi_budget)\n \ttx_ring->q_vector->tx.total_bytes += total_bytes;\n \ttx_ring->q_vector->tx.total_pkts += total_pkts;\n \n+\tif (ice_ring_is_xdp(tx_ring))\n+\t\treturn !!budget;\n+\n \tnetdev_tx_completed_queue(txring_txq(tx_ring), total_pkts,\n \t\t\t\t total_bytes);\n \n@@ -319,6 +331,10 @@ void ice_clean_rx_ring(struct ice_ring *rx_ring)\n void ice_free_rx_ring(struct ice_ring *rx_ring)\n {\n \tice_clean_rx_ring(rx_ring);\n+\tif (rx_ring->vsi->type == ICE_VSI_PF)\n+\t\tif (xdp_rxq_info_is_reg(&rx_ring->xdp_rxq))\n+\t\t\txdp_rxq_info_unreg(&rx_ring->xdp_rxq);\n+\trx_ring->xdp_prog = NULL;\n \tdevm_kfree(rx_ring->dev, rx_ring->rx_buf);\n \trx_ring->rx_buf = NULL;\n \n@@ -363,6 +379,15 @@ int ice_setup_rx_ring(struct ice_ring *rx_ring)\n \n \trx_ring->next_to_use = 0;\n \trx_ring->next_to_clean = 0;\n+\n+\tif (ice_is_xdp_ena_vsi(rx_ring->vsi))\n+\t\tWRITE_ONCE(rx_ring->xdp_prog, rx_ring->vsi->xdp_prog);\n+\n+\tif (rx_ring->vsi->type == ICE_VSI_PF &&\n+\t !xdp_rxq_info_is_reg(&rx_ring->xdp_rxq))\n+\t\tif (xdp_rxq_info_reg(&rx_ring->xdp_rxq, rx_ring->netdev,\n+\t\t\t\t rx_ring->q_index))\n+\t\t\tgoto err;\n \treturn 0;\n \n err:\n@@ -402,6 +427,214 @@ static void ice_release_rx_desc(struct ice_ring *rx_ring, u32 val)\n \t}\n }\n \n+/**\n+ * ice_rx_offset - Return expected offset into page to access data\n+ * @rx_ring: Ring we are requesting offset of\n+ *\n+ * Returns the offset value for ring into the data buffer.\n+ */\n+static unsigned int ice_rx_offset(struct ice_ring *rx_ring)\n+{\n+\treturn ice_is_xdp_ena_vsi(rx_ring->vsi) ? XDP_PACKET_HEADROOM : 0;\n+}\n+\n+/**\n+ * ice_xdp_ring_update_tail - Updates the XDP Tx ring tail register\n+ * @xdp_ring: XDP Tx ring\n+ *\n+ * This function updates the XDP Tx ring tail register.\n+ */\n+static void ice_xdp_ring_update_tail(struct ice_ring *xdp_ring)\n+{\n+\t/* Force memory writes to complete before letting h/w\n+\t * know there are new descriptors to fetch.\n+\t */\n+\twmb();\n+\twritel_relaxed(xdp_ring->next_to_use, xdp_ring->tail);\n+}\n+\n+/**\n+ * ice_xmit_xdp_ring - submit single packet to XDP ring for transmission\n+ * @data: packet data pointer\n+ * @size: packet data size\n+ * @xdp_ring: XDP ring for transmission\n+ */\n+static int ice_xmit_xdp_ring(void *data, u16 size, struct ice_ring *xdp_ring)\n+{\n+\tu16 i = xdp_ring->next_to_use;\n+\tstruct ice_tx_desc *tx_desc;\n+\tstruct ice_tx_buf *tx_buf;\n+\tdma_addr_t dma;\n+\n+\tif (!unlikely(ICE_DESC_UNUSED(xdp_ring))) {\n+\t\txdp_ring->tx_stats.tx_busy++;\n+\t\treturn ICE_XDP_CONSUMED;\n+\t}\n+\n+\tdma = dma_map_single(xdp_ring->dev, data, size, DMA_TO_DEVICE);\n+\tif (dma_mapping_error(xdp_ring->dev, dma))\n+\t\treturn ICE_XDP_CONSUMED;\n+\n+\ttx_buf = &xdp_ring->tx_buf[i];\n+\ttx_buf->bytecount = size;\n+\ttx_buf->gso_segs = 1;\n+\ttx_buf->raw_buf = data;\n+\n+\t/* record length, and DMA address */\n+\tdma_unmap_len_set(tx_buf, len, size);\n+\tdma_unmap_addr_set(tx_buf, dma, dma);\n+\n+\ttx_desc = ICE_TX_DESC(xdp_ring, i);\n+\ttx_desc->buf_addr = cpu_to_le64(dma);\n+\ttx_desc->cmd_type_offset_bsz = ice_build_ctob(ICE_TXD_LAST_DESC_CMD, 0,\n+\t\t\t\t\t\t size, 0);\n+\n+\t/* Make certain all of the status bits have been updated\n+\t * before next_to_watch is written.\n+\t */\n+\tsmp_wmb();\n+\n+\ti++;\n+\tif (i == xdp_ring->count)\n+\t\ti = 0;\n+\n+\ttx_buf->next_to_watch = tx_desc;\n+\txdp_ring->next_to_use = i;\n+\n+\treturn ICE_XDP_TX;\n+}\n+\n+/**\n+ * ice_xmit_xdp_buff - convert an XDP buffer to an XDP frame and send it\n+ * @xdp: XDP buffer\n+ * @xdp_ring: XDP Tx ring\n+ *\n+ * Returns negative on failure, 0 on success.\n+ */\n+static int ice_xmit_xdp_buff(struct xdp_buff *xdp, struct ice_ring *xdp_ring)\n+{\n+\tstruct xdp_frame *xdpf = convert_to_xdp_frame(xdp);\n+\n+\tif (unlikely(!xdpf))\n+\t\treturn ICE_XDP_CONSUMED;\n+\n+\treturn ice_xmit_xdp_ring(xdpf->data, xdpf->len, xdp_ring);\n+}\n+\n+/**\n+ * ice_run_xdp - Executes an XDP program on initialized xdp_buff\n+ * @rx_ring: Rx ring\n+ * @xdp: xdp_buff used as input to the XDP program\n+ * @xdp_prog: XDP program to run\n+ *\n+ * Returns any of ICE_XDP_{PASS, CONSUMED, TX, REDIR}\n+ */\n+static int\n+ice_run_xdp(struct ice_ring *rx_ring, struct xdp_buff *xdp,\n+\t struct bpf_prog *xdp_prog)\n+{\n+\tint err, result = ICE_XDP_PASS;\n+\tstruct ice_ring *xdp_ring;\n+\tu32 act;\n+\n+\tact = bpf_prog_run_xdp(xdp_prog, xdp);\n+\tswitch (act) {\n+\tcase XDP_PASS:\n+\t\tbreak;\n+\tcase XDP_TX:\n+\t\txdp_ring = rx_ring->vsi->xdp_rings[rx_ring->q_index];\n+\t\tresult = ice_xmit_xdp_buff(xdp, xdp_ring);\n+\t\tbreak;\n+\tcase XDP_REDIRECT:\n+\t\terr = xdp_do_redirect(rx_ring->netdev, xdp, xdp_prog);\n+\t\tresult = !err ? ICE_XDP_REDIR : ICE_XDP_CONSUMED;\n+\t\tbreak;\n+\tdefault:\n+\t\tbpf_warn_invalid_xdp_action(act);\n+\t\t/* fallthrough -- not supported action */\n+\tcase XDP_ABORTED:\n+\t\ttrace_xdp_exception(rx_ring->netdev, xdp_prog, act);\n+\t\t/* fallthrough -- handle aborts by dropping frame */\n+\tcase XDP_DROP:\n+\t\tresult = ICE_XDP_CONSUMED;\n+\t\tbreak;\n+\t}\n+\n+\treturn result;\n+}\n+\n+/**\n+ * ice_xdp_xmit - submit packets to XDP ring for transmission\n+ * @dev: netdev\n+ * @n: number of XDP frames to be transmitted\n+ * @frames: XDP frames to be transmitted\n+ * @flags: transmit flags\n+ *\n+ * Returns number of frames successfully sent. Frames that fail are\n+ * free'ed via XDP return API.\n+ * For error cases, a negative errno code is returned and no-frames\n+ * are transmitted (caller must handle freeing frames).\n+ */\n+int\n+ice_xdp_xmit(struct net_device *dev, int n, struct xdp_frame **frames,\n+\t u32 flags)\n+{\n+\tstruct ice_netdev_priv *np = netdev_priv(dev);\n+\tunsigned int queue_index = smp_processor_id();\n+\tstruct ice_vsi *vsi = np->vsi;\n+\tstruct ice_ring *xdp_ring;\n+\tint drops = 0, i;\n+\n+\tif (test_bit(__ICE_DOWN, vsi->state))\n+\t\treturn -ENETDOWN;\n+\n+\tif (!ice_is_xdp_ena_vsi(vsi) || queue_index >= vsi->num_xdp_txq)\n+\t\treturn -ENXIO;\n+\n+\tif (unlikely(flags & ~XDP_XMIT_FLAGS_MASK))\n+\t\treturn -EINVAL;\n+\n+\txdp_ring = vsi->xdp_rings[queue_index];\n+\tfor (i = 0; i < n; i++) {\n+\t\tstruct xdp_frame *xdpf = frames[i];\n+\t\tint err;\n+\n+\t\terr = ice_xmit_xdp_ring(xdpf->data, xdpf->len, xdp_ring);\n+\t\tif (err != ICE_XDP_TX) {\n+\t\t\txdp_return_frame_rx_napi(xdpf);\n+\t\t\tdrops++;\n+\t\t}\n+\t}\n+\n+\tif (unlikely(flags & XDP_XMIT_FLUSH))\n+\t\tice_xdp_ring_update_tail(xdp_ring);\n+\n+\treturn n - drops;\n+}\n+\n+/**\n+ * ice_finalize_xdp_rx - Bump XDP Tx tail and/or flush redirect map\n+ * @rx_ring: Rx ring\n+ * @xdp_res: Result of the receive batch\n+ *\n+ * This function bumps XDP Tx tail and/or flush redirect map, and\n+ * should be called when a batch of packets has been processed in the\n+ * napi loop.\n+ */\n+static void\n+ice_finalize_xdp_rx(struct ice_ring *rx_ring, unsigned int xdp_res)\n+{\n+\tif (xdp_res & ICE_XDP_REDIR)\n+\t\txdp_do_flush_map();\n+\n+\tif (xdp_res & ICE_XDP_TX) {\n+\t\tstruct ice_ring *xdp_ring =\n+\t\t\trx_ring->vsi->xdp_rings[rx_ring->q_index];\n+\n+\t\tice_xdp_ring_update_tail(xdp_ring);\n+\t}\n+}\n+\n /**\n * ice_alloc_mapped_page - recycle or make a new page\n * @rx_ring: ring to use\n@@ -444,7 +677,7 @@ ice_alloc_mapped_page(struct ice_ring *rx_ring, struct ice_rx_buf *bi)\n \n \tbi->dma = dma;\n \tbi->page = page;\n-\tbi->page_offset = 0;\n+\tbi->page_offset = ice_rx_offset(rx_ring);\n \tpage_ref_add(page, USHRT_MAX - 1);\n \tbi->pagecnt_bias = USHRT_MAX;\n \n@@ -682,7 +915,7 @@ ice_get_rx_buf(struct ice_ring *rx_ring, struct sk_buff **skb,\n * ice_construct_skb - Allocate skb and populate it\n * @rx_ring: Rx descriptor ring to transact packets on\n * @rx_buf: Rx buffer to pull data from\n- * @size: the length of the packet\n+ * @xdp: xdp_buff pointing to the data\n *\n * This function allocates an skb. It then populates it with the page\n * data from the current receive descriptor, taking care to set up the\n@@ -690,16 +923,16 @@ ice_get_rx_buf(struct ice_ring *rx_ring, struct sk_buff **skb,\n */\n static struct sk_buff *\n ice_construct_skb(struct ice_ring *rx_ring, struct ice_rx_buf *rx_buf,\n-\t\t unsigned int size)\n+\t\t struct xdp_buff *xdp)\n {\n-\tvoid *va = page_address(rx_buf->page) + rx_buf->page_offset;\n+\tunsigned int size = xdp->data_end - xdp->data;\n \tunsigned int headlen;\n \tstruct sk_buff *skb;\n \n \t/* prefetch first cache line of first page */\n-\tprefetch(va);\n+\tprefetch(xdp->data);\n #if L1_CACHE_BYTES < 128\n-\tprefetch((u8 *)va + L1_CACHE_BYTES);\n+\tprefetch((void *)(xdp->data + L1_CACHE_BYTES));\n #endif /* L1_CACHE_BYTES */\n \n \t/* allocate a skb to store the frags */\n@@ -712,10 +945,11 @@ ice_construct_skb(struct ice_ring *rx_ring, struct ice_rx_buf *rx_buf,\n \t/* Determine available headroom for copy */\n \theadlen = size;\n \tif (headlen > ICE_RX_HDR_SIZE)\n-\t\theadlen = eth_get_headlen(skb->dev, va, ICE_RX_HDR_SIZE);\n+\t\theadlen = eth_get_headlen(skb->dev, xdp->data, ICE_RX_HDR_SIZE);\n \n \t/* align pull length to size of long to optimize memcpy performance */\n-\tmemcpy(__skb_put(skb, headlen), va, ALIGN(headlen, sizeof(long)));\n+\tmemcpy(__skb_put(skb, headlen), xdp->data, ALIGN(headlen,\n+\t\t\t\t\t\t\t sizeof(long)));\n \n \t/* if we exhaust the linear part then add what is left as a frag */\n \tsize -= headlen;\n@@ -745,11 +979,18 @@ ice_construct_skb(struct ice_ring *rx_ring, struct ice_rx_buf *rx_buf,\n * @rx_ring: Rx descriptor ring to transact packets on\n * @rx_buf: Rx buffer to pull data from\n *\n- * This function will clean up the contents of the rx_buf. It will\n- * either recycle the buffer or unmap it and free the associated resources.\n+ * This function will update next_to_clean and then clean up the contents\n+ * of the rx_buf. It will either recycle the buffer or unmap it and free\n+ * the associated resources.\n */\n static void ice_put_rx_buf(struct ice_ring *rx_ring, struct ice_rx_buf *rx_buf)\n {\n+\tu32 ntc = rx_ring->next_to_clean + 1;\n+\n+\t/* fetch, update, and store next to clean */\n+\tntc = (ntc < rx_ring->count) ? ntc : 0;\n+\trx_ring->next_to_clean = ntc;\n+\n \tif (!rx_buf)\n \t\treturn;\n \n@@ -813,30 +1054,20 @@ ice_test_staterr(union ice_32b_rx_flex_desc *rx_desc, const u16 stat_err_bits)\n * @rx_desc: Rx descriptor for current buffer\n * @skb: Current socket buffer containing buffer in progress\n *\n- * This function updates next to clean. If the buffer is an EOP buffer\n- * this function exits returning false, otherwise it will place the\n- * sk_buff in the next buffer to be chained and return true indicating\n- * that this is in fact a non-EOP buffer.\n+ * If the buffer is an EOP buffer, this function exits returning false,\n+ * otherwise return true indicating that this is in fact a non-EOP buffer.\n */\n static bool\n ice_is_non_eop(struct ice_ring *rx_ring, union ice_32b_rx_flex_desc *rx_desc,\n \t struct sk_buff *skb)\n {\n-\tu32 ntc = rx_ring->next_to_clean + 1;\n-\n-\t/* fetch, update, and store next to clean */\n-\tntc = (ntc < rx_ring->count) ? ntc : 0;\n-\trx_ring->next_to_clean = ntc;\n-\n-\tprefetch(ICE_RX_DESC(rx_ring, ntc));\n-\n \t/* if we are the last buffer then there is nothing else to do */\n #define ICE_RXD_EOF BIT(ICE_RX_FLEX_DESC_STATUS0_EOF_S)\n \tif (likely(ice_test_staterr(rx_desc, ICE_RXD_EOF)))\n \t\treturn false;\n \n \t/* place skb in next buffer to be received */\n-\trx_ring->rx_buf[ntc].skb = skb;\n+\trx_ring->rx_buf[rx_ring->next_to_clean].skb = skb;\n \trx_ring->rx_stats.non_eop_descs++;\n \n \treturn true;\n@@ -1006,8 +1237,13 @@ static int ice_clean_rx_irq(struct ice_ring *rx_ring, int budget)\n {\n \tunsigned int total_rx_bytes = 0, total_rx_pkts = 0;\n \tu16 cleaned_count = ICE_DESC_UNUSED(rx_ring);\n+\tunsigned int xdp_res, xdp_xmit = 0;\n+\tstruct bpf_prog *xdp_prog = NULL;\n+\tstruct xdp_buff xdp;\n \tbool failure;\n \n+\txdp.rxq = &rx_ring->xdp_rxq;\n+\n \t/* start the loop to process Rx packets bounded by 'budget' */\n \twhile (likely(total_rx_pkts < (unsigned int)budget)) {\n \t\tunion ice_32b_rx_flex_desc *rx_desc;\n@@ -1042,10 +1278,46 @@ static int ice_clean_rx_irq(struct ice_ring *rx_ring, int budget)\n \t\t/* retrieve a buffer from the ring */\n \t\trx_buf = ice_get_rx_buf(rx_ring, &skb, size);\n \n+\t\tif (!size) {\n+\t\t\txdp.data = NULL;\n+\t\t\txdp.data_end = NULL;\n+\t\t\tgoto construct_skb;\n+\t\t}\n+\n+\t\txdp.data = page_address(rx_buf->page) + rx_buf->page_offset;\n+\t\txdp.data_hard_start = xdp.data - ice_rx_offset(rx_ring);\n+\t\txdp_set_data_meta_invalid(&xdp);\n+\t\txdp.data_end = xdp.data + size;\n+\n+\t\trcu_read_lock();\n+\t\txdp_prog = READ_ONCE(rx_ring->xdp_prog);\n+\t\tif (!xdp_prog) {\n+\t\t\trcu_read_unlock();\n+\t\t\tgoto construct_skb;\n+\t\t}\n+\n+\t\txdp_res = ice_run_xdp(rx_ring, &xdp, xdp_prog);\n+\t\trcu_read_unlock();\n+\t\tif (xdp_res) {\n+\t\t\tif (xdp_res & (ICE_XDP_TX | ICE_XDP_REDIR)) {\n+\t\t\t\txdp_xmit |= xdp_res;\n+\t\t\t\tice_rx_buf_adjust_pg_offset(rx_buf,\n+\t\t\t\t\t\t\t ICE_RXBUF_2048);\n+\t\t\t} else {\n+\t\t\t\trx_buf->pagecnt_bias++;\n+\t\t\t}\n+\t\t\ttotal_rx_bytes += size;\n+\t\t\ttotal_rx_pkts++;\n+\n+\t\t\tcleaned_count++;\n+\t\t\tice_put_rx_buf(rx_ring, rx_buf);\n+\t\t\tcontinue;\n+\t\t}\n+construct_skb:\n \t\tif (skb)\n \t\t\tice_add_rx_frag(rx_buf, skb, size);\n \t\telse\n-\t\t\tskb = ice_construct_skb(rx_ring, rx_buf, size);\n+\t\t\tskb = ice_construct_skb(rx_ring, rx_buf, &xdp);\n \n \t\t/* exit if we failed to retrieve a buffer */\n \t\tif (!skb) {\n@@ -1099,6 +1371,9 @@ static int ice_clean_rx_irq(struct ice_ring *rx_ring, int budget)\n \t/* return up to cleaned_count buffers to hardware */\n \tfailure = ice_alloc_rx_bufs(rx_ring, cleaned_count);\n \n+\tif (xdp_prog)\n+\t\tice_finalize_xdp_rx(rx_ring, xdp_xmit);\n+\n \t/* update queue and vector specific stats */\n \tu64_stats_update_begin(&rx_ring->syncp);\n \trx_ring->stats.pkts += total_rx_pkts;\n@@ -1527,17 +1802,6 @@ int ice_napi_poll(struct napi_struct *napi, int budget)\n \treturn min_t(int, work_done, budget - 1);\n }\n \n-/* helper function for building cmd/type/offset */\n-static __le64\n-build_ctob(u64 td_cmd, u64 td_offset, unsigned int size, u64 td_tag)\n-{\n-\treturn cpu_to_le64(ICE_TX_DESC_DTYPE_DATA |\n-\t\t\t (td_cmd << ICE_TXD_QW1_CMD_S) |\n-\t\t\t (td_offset << ICE_TXD_QW1_OFFSET_S) |\n-\t\t\t ((u64)size << ICE_TXD_QW1_TX_BUF_SZ_S) |\n-\t\t\t (td_tag << ICE_TXD_QW1_L2TAG1_S));\n-}\n-\n /**\n * __ice_maybe_stop_tx - 2nd level check for Tx stop conditions\n * @tx_ring: the ring to be checked\n@@ -1638,7 +1902,8 @@ ice_tx_map(struct ice_ring *tx_ring, struct ice_tx_buf *first,\n \t\t */\n \t\twhile (unlikely(size > ICE_MAX_DATA_PER_TXD)) {\n \t\t\ttx_desc->cmd_type_offset_bsz =\n-\t\t\t\tbuild_ctob(td_cmd, td_offset, max_data, td_tag);\n+\t\t\t\tice_build_ctob(td_cmd, td_offset, max_data,\n+\t\t\t\t\t td_tag);\n \n \t\t\ttx_desc++;\n \t\t\ti++;\n@@ -1658,8 +1923,8 @@ ice_tx_map(struct ice_ring *tx_ring, struct ice_tx_buf *first,\n \t\tif (likely(!data_len))\n \t\t\tbreak;\n \n-\t\ttx_desc->cmd_type_offset_bsz = build_ctob(td_cmd, td_offset,\n-\t\t\t\t\t\t\t size, td_tag);\n+\t\ttx_desc->cmd_type_offset_bsz = ice_build_ctob(td_cmd, td_offset,\n+\t\t\t\t\t\t\t size, td_tag);\n \n \t\ttx_desc++;\n \t\ti++;\n@@ -1689,9 +1954,9 @@ ice_tx_map(struct ice_ring *tx_ring, struct ice_tx_buf *first,\n \t\ti = 0;\n \n \t/* write last descriptor with RS and EOP bits */\n-\ttd_cmd |= (u64)(ICE_TX_DESC_CMD_EOP | ICE_TX_DESC_CMD_RS);\n+\ttd_cmd |= (u64)ICE_TXD_LAST_DESC_CMD;\n \ttx_desc->cmd_type_offset_bsz =\n-\t\t\tbuild_ctob(td_cmd, td_offset, size, td_tag);\n+\t\t\tice_build_ctob(td_cmd, td_offset, size, td_tag);\n \n \t/* Force memory writes to complete before letting h/w know there\n \t * are new descriptors to fetch.\ndiff --git a/drivers/net/ethernet/intel/ice/ice_txrx.h b/drivers/net/ethernet/intel/ice/ice_txrx.h\nindex a914e603b2ed..f74ec83faa55 100644\n--- a/drivers/net/ethernet/intel/ice/ice_txrx.h\n+++ b/drivers/net/ethernet/intel/ice/ice_txrx.h\n@@ -22,6 +22,16 @@\n #define ICE_RX_BUF_WRITE\t16\t/* Must be power of 2 */\n #define ICE_MAX_TXQ_PER_TXQG\t128\n \n+static inline __le64\n+ice_build_ctob(u64 td_cmd, u64 td_offset, unsigned int size, u64 td_tag)\n+{\n+\treturn cpu_to_le64(ICE_TX_DESC_DTYPE_DATA |\n+\t\t\t (td_cmd << ICE_TXD_QW1_CMD_S) |\n+\t\t\t (td_offset << ICE_TXD_QW1_OFFSET_S) |\n+\t\t\t ((u64)size << ICE_TXD_QW1_TX_BUF_SZ_S) |\n+\t\t\t (td_tag << ICE_TXD_QW1_L2TAG1_S));\n+}\n+\n /* We are assuming that the cache line is always 64 Bytes here for ice.\n * In order to make sure that is a correct assumption there is a check in probe\n * to print a warning if the read from GLPCI_CNF2 tells us that the cache line\n@@ -49,12 +59,24 @@\n #define ICE_TX_FLAGS_VLAN_PR_S\t29\n #define ICE_TX_FLAGS_VLAN_S\t16\n \n+#define ICE_XDP_PASS\t\t0\n+#define ICE_XDP_CONSUMED\tBIT(0)\n+#define ICE_XDP_TX\t\tBIT(1)\n+#define ICE_XDP_REDIR\t\tBIT(2)\n+\n #define ICE_RX_DMA_ATTR \\\n \t(DMA_ATTR_SKIP_CPU_SYNC | DMA_ATTR_WEAK_ORDERING)\n \n+#define ICE_ETH_PKT_HDR_PAD\t(ETH_HLEN + ETH_FCS_LEN + (VLAN_HLEN * 2))\n+\n+#define ICE_TXD_LAST_DESC_CMD (ICE_TX_DESC_CMD_EOP | ICE_TX_DESC_CMD_RS)\n+\n struct ice_tx_buf {\n \tstruct ice_tx_desc *next_to_watch;\n-\tstruct sk_buff *skb;\n+\tunion {\n+\t\tstruct sk_buff *skb;\n+\t\tvoid *raw_buf; /* used for XDP */\n+\t};\n \tunsigned int bytecount;\n \tunsigned short gso_segs;\n \tu32 tx_flags;\n@@ -198,9 +220,14 @@ struct ice_ring {\n \t};\n \n \tstruct rcu_head rcu;\t\t/* to avoid race on free */\n+\tstruct bpf_prog *xdp_prog;\n+\t/* CL3 - 3rd cacheline starts here */\n+\tstruct xdp_rxq_info xdp_rxq;\n \t/* CLX - the below items are only accessed infrequently and should be\n \t * in their own cache line if possible\n \t */\n+#define ICE_TX_FLAGS_RING_XDP\t\tBIT(0)\n+\tu8 flags;\n \tdma_addr_t dma;\t\t\t/* physical address of ring */\n \tunsigned int size;\t\t/* length of descriptor ring in bytes */\n \tu32 txq_teid;\t\t\t/* Added Tx queue TEID */\n@@ -208,6 +235,11 @@ struct ice_ring {\n \tu8 dcb_tc;\t\t\t/* Traffic class of ring */\n } ____cacheline_internodealigned_in_smp;\n \n+static inline bool ice_ring_is_xdp(struct ice_ring *ring)\n+{\n+\treturn !!(ring->flags & ICE_TX_FLAGS_RING_XDP);\n+}\n+\n struct ice_ring_container {\n \t/* head of linked-list of rings */\n \tstruct ice_ring *ring;\n", "prefixes": [ "S30", "v2", "3/9" ] }