From patchwork Wed Oct 28 20:19:34 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andre Guedes X-Patchwork-Id: 1389394 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=osuosl.org (client-ip=140.211.166.133; helo=hemlock.osuosl.org; envelope-from=intel-wired-lan-bounces@osuosl.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=intel.com Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4CM0LC1vJNz9sVk for ; Thu, 29 Oct 2020 07:20:02 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id 405E9872B2; Wed, 28 Oct 2020 20:19:59 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id kRvFiMAKV4DX; Wed, 28 Oct 2020 20:19:58 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by hemlock.osuosl.org (Postfix) with ESMTP id 0B7848729E; Wed, 28 Oct 2020 20:19:58 +0000 (UTC) X-Original-To: intel-wired-lan@lists.osuosl.org Delivered-To: intel-wired-lan@lists.osuosl.org Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by ash.osuosl.org (Postfix) with ESMTP id AFBA01BF958 for ; Wed, 28 Oct 2020 20:19:56 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id AC9CB8729E for ; Wed, 28 Oct 2020 20:19:56 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id zwkPMpRWtsjV for ; Wed, 28 Oct 2020 20:19:55 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by hemlock.osuosl.org (Postfix) with ESMTPS id C9C2682B51 for ; Wed, 28 Oct 2020 20:19:55 +0000 (UTC) IronPort-SDR: 2O/IzI12OWyOGpqVibN6q8cRiL87a7KQ1hG3xhyjB6xelPfNI0eHeR2le9QCAIsjG8XRpqD1QP snV4bnXz0I3w== X-IronPort-AV: E=McAfee;i="6000,8403,9788"; a="164833070" X-IronPort-AV: E=Sophos;i="5.77,427,1596524400"; d="scan'208";a="164833070" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Oct 2020 13:19:54 -0700 IronPort-SDR: ftZzK0gREVf0HwNnCy9yhOQ48EkhgG4gVivwhT9YPcolMPWb6NGl9nwFdCsj4IMNkWKP7fhhe9 u9Puwzg0TLqg== X-IronPort-AV: E=Sophos;i="5.77,427,1596524400"; d="scan'208";a="536374745" Received: from zcolon-mobl.amr.corp.intel.com ([10.209.108.9]) by orsmga005-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Oct 2020 13:19:54 -0700 From: Andre Guedes To: intel-wired-lan@lists.osuosl.org Date: Wed, 28 Oct 2020 13:19:34 -0700 Message-Id: <20201028201943.93147-2-andre.guedes@intel.com> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20201028201943.93147-1-andre.guedes@intel.com> References: <20201028201943.93147-1-andre.guedes@intel.com> MIME-Version: 1.0 Subject: [Intel-wired-lan] [PATCH v2 01/10] igc: Fix igc_ptp_rx_pktstamp() X-BeenThere: intel-wired-lan@osuosl.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel Wired Ethernet Linux Kernel Driver Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: intel-wired-lan-bounces@osuosl.org Sender: "Intel-wired-lan" The comment describing the timestamps layout in the packet buffer is wrong and the code is actually retrieving the timestamp in Timer 1 reference instead of Timer 0. This hasn't been a big issue so far because hardware is configured to report both timestamps using Timer 0 (see IGC_SRRCTL register configuration in igc_ptp_enable_rx_timestamp() helper). This patch fixes the comment and the code so we retrieve the timestamp in Timer 0 reference as expected. This patch also takes the opportunity to get rid of the hw.mac.type check since it is not required. Fixes: 81b055205e8ba ("igc: Add support for RX timestamping") Signed-off-by: Andre Guedes --- drivers/net/ethernet/intel/igc/igc.h | 2 +- drivers/net/ethernet/intel/igc/igc_ptp.c | 72 +++++++++++++----------- 2 files changed, 41 insertions(+), 33 deletions(-) diff --git a/drivers/net/ethernet/intel/igc/igc.h b/drivers/net/ethernet/intel/igc/igc.h index 83d59b08e883..b66dda992d32 100644 --- a/drivers/net/ethernet/intel/igc/igc.h +++ b/drivers/net/ethernet/intel/igc/igc.h @@ -552,7 +552,7 @@ void igc_ptp_init(struct igc_adapter *adapter); void igc_ptp_reset(struct igc_adapter *adapter); void igc_ptp_suspend(struct igc_adapter *adapter); void igc_ptp_stop(struct igc_adapter *adapter); -void igc_ptp_rx_pktstamp(struct igc_q_vector *q_vector, void *va, +void igc_ptp_rx_pktstamp(struct igc_q_vector *q_vector, u32 *va, struct sk_buff *skb); int igc_ptp_set_ts_config(struct net_device *netdev, struct ifreq *ifr); int igc_ptp_get_ts_config(struct net_device *netdev, struct ifreq *ifr); diff --git a/drivers/net/ethernet/intel/igc/igc_ptp.c b/drivers/net/ethernet/intel/igc/igc_ptp.c index d73c4aaac610..79873f6df335 100644 --- a/drivers/net/ethernet/intel/igc/igc_ptp.c +++ b/drivers/net/ethernet/intel/igc/igc_ptp.c @@ -154,46 +154,54 @@ static void igc_ptp_systim_to_hwtstamp(struct igc_adapter *adapter, } /** - * igc_ptp_rx_pktstamp - retrieve Rx per packet timestamp + * igc_ptp_rx_pktstamp - Retrieve timestamp from rx packet buffer * @q_vector: Pointer to interrupt specific structure * @va: Pointer to address containing Rx buffer * @skb: Buffer containing timestamp and packet * - * This function is meant to retrieve the first timestamp from the - * first buffer of an incoming frame. The value is stored in little - * endian format starting on byte 0. There's a second timestamp - * starting on byte 8. - **/ -void igc_ptp_rx_pktstamp(struct igc_q_vector *q_vector, void *va, + * This function retrieves the timestamp saved in the beginning of packet + * buffer. While two timestamps are available, one in timer0 reference and the + * other in timer1 reference, this function considers only the timestamp in + * timer0 reference. + */ +void igc_ptp_rx_pktstamp(struct igc_q_vector *q_vector, u32 *va, struct sk_buff *skb) { struct igc_adapter *adapter = q_vector->adapter; - __le64 *regval = (__le64 *)va; - int adjust = 0; - - /* The timestamp is recorded in little endian format. - * DWORD: | 0 | 1 | 2 | 3 - * Field: | Timer0 Low | Timer0 High | Timer1 Low | Timer1 High + u64 regval; + int adjust; + + /* Timestamps are saved in little endian at the beginning of the packet + * buffer following the layout: + * + * | 0 | 1 | 2 | 3 | + * | Timer1 SYSTIML | Timer1 SYSTIMH | Timer0 SYSTIML | Timer0 SYSTIMH | + * + * SYSTIML holds the nanoseconds part while SYSTIMH holds the seconds + * part of the timestamp. */ - igc_ptp_systim_to_hwtstamp(adapter, skb_hwtstamps(skb), - le64_to_cpu(regval[0])); - - /* adjust timestamp for the RX latency based on link speed */ - if (adapter->hw.mac.type == igc_i225) { - switch (adapter->link_speed) { - case SPEED_10: - adjust = IGC_I225_RX_LATENCY_10; - break; - case SPEED_100: - adjust = IGC_I225_RX_LATENCY_100; - break; - case SPEED_1000: - adjust = IGC_I225_RX_LATENCY_1000; - break; - case SPEED_2500: - adjust = IGC_I225_RX_LATENCY_2500; - break; - } + regval = le32_to_cpu(va[2]); + regval |= (u64)le32_to_cpu(va[3]) << 32; + igc_ptp_systim_to_hwtstamp(adapter, skb_hwtstamps(skb), regval); + + /* Adjust timestamp for the RX latency based on link speed */ + switch (adapter->link_speed) { + case SPEED_10: + adjust = IGC_I225_RX_LATENCY_10; + break; + case SPEED_100: + adjust = IGC_I225_RX_LATENCY_100; + break; + case SPEED_1000: + adjust = IGC_I225_RX_LATENCY_1000; + break; + case SPEED_2500: + adjust = IGC_I225_RX_LATENCY_2500; + break; + default: + adjust = 0; + netdev_warn_once(adapter->netdev, "Imprecise timestamp\n"); + break; } skb_hwtstamps(skb)->hwtstamp = ktime_sub_ns(skb_hwtstamps(skb)->hwtstamp, adjust); From patchwork Wed Oct 28 20:19:35 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andre Guedes X-Patchwork-Id: 1389393 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=osuosl.org (client-ip=140.211.166.137; helo=fraxinus.osuosl.org; envelope-from=intel-wired-lan-bounces@osuosl.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=intel.com Received: from fraxinus.osuosl.org (smtp4.osuosl.org [140.211.166.137]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4CM0LC0n8Wz9sVh for ; Thu, 29 Oct 2020 07:20:02 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by fraxinus.osuosl.org (Postfix) with ESMTP id 4E88885540; Wed, 28 Oct 2020 20:20:00 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from fraxinus.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id g1YF1DNc6K3Q; Wed, 28 Oct 2020 20:19:57 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by fraxinus.osuosl.org (Postfix) with ESMTP id A84B8854C2; Wed, 28 Oct 2020 20:19:57 +0000 (UTC) X-Original-To: intel-wired-lan@lists.osuosl.org Delivered-To: intel-wired-lan@lists.osuosl.org Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by ash.osuosl.org (Postfix) with ESMTP id 883DF1BF836 for ; Wed, 28 Oct 2020 20:19:56 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id 8409F872D1 for ; Wed, 28 Oct 2020 20:19:56 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id K3GJfKrwm1Pg for ; Wed, 28 Oct 2020 20:19:56 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by hemlock.osuosl.org (Postfix) with ESMTPS id F224B8729E for ; Wed, 28 Oct 2020 20:19:55 +0000 (UTC) IronPort-SDR: ydyOjNOVzYA2M4nfJcps21gyu5l5HFMoqKXqIU7ydnZCyYCI+td2YN8B5IID9fLMG3hlffJLQr MW3cTS7W/3Tg== X-IronPort-AV: E=McAfee;i="6000,8403,9788"; a="164833071" X-IronPort-AV: E=Sophos;i="5.77,427,1596524400"; d="scan'208";a="164833071" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Oct 2020 13:19:54 -0700 IronPort-SDR: E70u81iKkAXXzvAUFYVzfitk0UVqBJomNUs3ZI7lsaEMFt3CVdtxaUpu5g19Gwvgx4kFwluf7X rgvyeUeNhh5A== X-IronPort-AV: E=Sophos;i="5.77,427,1596524400"; d="scan'208";a="536374748" Received: from zcolon-mobl.amr.corp.intel.com ([10.209.108.9]) by orsmga005-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Oct 2020 13:19:54 -0700 From: Andre Guedes To: intel-wired-lan@lists.osuosl.org Date: Wed, 28 Oct 2020 13:19:35 -0700 Message-Id: <20201028201943.93147-3-andre.guedes@intel.com> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20201028201943.93147-1-andre.guedes@intel.com> References: <20201028201943.93147-1-andre.guedes@intel.com> MIME-Version: 1.0 Subject: [Intel-wired-lan] [PATCH v2 02/10] igc: Remove unused argument from igc_tx_cmd_type() X-BeenThere: intel-wired-lan@osuosl.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel Wired Ethernet Linux Kernel Driver Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: intel-wired-lan-bounces@osuosl.org Sender: "Intel-wired-lan" The 'skb' argument from igc_tx_cmd_type() is not used so this patch removes it. Signed-off-by: Andre Guedes --- drivers/net/ethernet/intel/igc/igc_main.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/drivers/net/ethernet/intel/igc/igc_main.c b/drivers/net/ethernet/intel/igc/igc_main.c index 240565aee2a3..5ce9253cca19 100644 --- a/drivers/net/ethernet/intel/igc/igc_main.c +++ b/drivers/net/ethernet/intel/igc/igc_main.c @@ -1042,7 +1042,7 @@ static inline int igc_maybe_stop_tx(struct igc_ring *tx_ring, const u16 size) ((u32)((_input) & (_flag)) * ((_result) / (_flag))) : \ ((u32)((_input) & (_flag)) / ((_flag) / (_result)))) -static u32 igc_tx_cmd_type(struct sk_buff *skb, u32 tx_flags) +static u32 igc_tx_cmd_type(u32 tx_flags) { /* set type for advanced descriptor with frame checksum insertion */ u32 cmd_type = IGC_ADVTXD_DTYP_DATA | @@ -1091,7 +1091,7 @@ static int igc_tx_map(struct igc_ring *tx_ring, u16 i = tx_ring->next_to_use; unsigned int data_len, size; dma_addr_t dma; - u32 cmd_type = igc_tx_cmd_type(skb, tx_flags); + u32 cmd_type = igc_tx_cmd_type(tx_flags); tx_desc = IGC_TX_DESC(tx_ring, i); From patchwork Wed Oct 28 20:19:36 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andre Guedes X-Patchwork-Id: 1389399 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=osuosl.org (client-ip=140.211.166.136; helo=silver.osuosl.org; envelope-from=intel-wired-lan-bounces@osuosl.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=intel.com Received: from silver.osuosl.org (smtp3.osuosl.org [140.211.166.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4CM0LK46FQz9sVh for ; Thu, 29 Oct 2020 07:20:09 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by silver.osuosl.org (Postfix) with ESMTP id A2DE220373; Wed, 28 Oct 2020 20:20:06 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from silver.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id du-AqKzA8YSP; Wed, 28 Oct 2020 20:19:59 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by silver.osuosl.org (Postfix) with ESMTP id 7DB1D2076B; Wed, 28 Oct 2020 20:19:58 +0000 (UTC) X-Original-To: intel-wired-lan@lists.osuosl.org Delivered-To: intel-wired-lan@lists.osuosl.org Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by ash.osuosl.org (Postfix) with ESMTP id CD5421BF836 for ; Wed, 28 Oct 2020 20:19:56 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id CA10182B51 for ; Wed, 28 Oct 2020 20:19:56 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id p95wGcD6tL-Z for ; Wed, 28 Oct 2020 20:19:56 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by hemlock.osuosl.org (Postfix) with ESMTPS id 3BA38872B7 for ; Wed, 28 Oct 2020 20:19:56 +0000 (UTC) IronPort-SDR: +egiSczJr81BylCVjQXzkFAhj8QxwWktGI2m273APNxgw+qi/Flad/2kuBICK5bkpS1R7c5vyZ Ht5OdWZF0iaw== X-IronPort-AV: E=McAfee;i="6000,8403,9788"; a="164833072" X-IronPort-AV: E=Sophos;i="5.77,427,1596524400"; d="scan'208";a="164833072" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Oct 2020 13:19:54 -0700 IronPort-SDR: Dh0Q7kilDaGm8cezAisT8/wK4jfYKWqq+emMU4Ypgtnrcz0aYPeEpwRhOizWKXTZpB3TJdZJrK ZB+eoWgxI9oA== X-IronPort-AV: E=Sophos;i="5.77,427,1596524400"; d="scan'208";a="536374749" Received: from zcolon-mobl.amr.corp.intel.com ([10.209.108.9]) by orsmga005-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Oct 2020 13:19:54 -0700 From: Andre Guedes To: intel-wired-lan@lists.osuosl.org Date: Wed, 28 Oct 2020 13:19:36 -0700 Message-Id: <20201028201943.93147-4-andre.guedes@intel.com> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20201028201943.93147-1-andre.guedes@intel.com> References: <20201028201943.93147-1-andre.guedes@intel.com> MIME-Version: 1.0 Subject: [Intel-wired-lan] [PATCH v2 03/10] igc: Introduce igc_rx_buffer_flip() helper X-BeenThere: intel-wired-lan@osuosl.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel Wired Ethernet Linux Kernel Driver Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: intel-wired-lan-bounces@osuosl.org Sender: "Intel-wired-lan" The igc driver implements the same page recycling scheme from other Intel drivers which reuses the page by flipping the buffer. The code to handle buffer flips is duplicated in many locations so this patch introduces the igc_rx_buffer_flip() helper and uses it where applicable. Signed-off-by: Andre Guedes --- drivers/net/ethernet/intel/igc/igc_main.c | 42 +++++++++++------------ 1 file changed, 20 insertions(+), 22 deletions(-) diff --git a/drivers/net/ethernet/intel/igc/igc_main.c b/drivers/net/ethernet/intel/igc/igc_main.c index 5ce9253cca19..31dc58a82cf3 100644 --- a/drivers/net/ethernet/intel/igc/igc_main.c +++ b/drivers/net/ethernet/intel/igc/igc_main.c @@ -1513,6 +1513,16 @@ static struct igc_rx_buffer *igc_get_rx_buffer(struct igc_ring *rx_ring, return rx_buffer; } +static void igc_rx_buffer_flip(struct igc_rx_buffer *buffer, + unsigned int truesize) +{ +#if (PAGE_SIZE < 8192) + buffer->page_offset ^= truesize; +#else + buffer->page_offset += truesize; +#endif +} + /** * igc_add_rx_frag - Add contents of Rx buffer to sk_buff * @rx_ring: rx descriptor ring to transact packets on @@ -1527,20 +1537,18 @@ static void igc_add_rx_frag(struct igc_ring *rx_ring, struct sk_buff *skb, unsigned int size) { + unsigned int truesize; #if (PAGE_SIZE < 8192) - unsigned int truesize = igc_rx_pg_size(rx_ring) / 2; - - skb_add_rx_frag(skb, skb_shinfo(skb)->nr_frags, rx_buffer->page, - rx_buffer->page_offset, size, truesize); - rx_buffer->page_offset ^= truesize; + truesize = igc_rx_pg_size(rx_ring) / 2; #else - unsigned int truesize = ring_uses_build_skb(rx_ring) ? - SKB_DATA_ALIGN(IGC_SKB_PAD + size) : - SKB_DATA_ALIGN(size); + truesize = ring_uses_build_skb(rx_ring) ? + SKB_DATA_ALIGN(IGC_SKB_PAD + size) : + SKB_DATA_ALIGN(size); +#endif skb_add_rx_frag(skb, skb_shinfo(skb)->nr_frags, rx_buffer->page, rx_buffer->page_offset, size, truesize); - rx_buffer->page_offset += truesize; -#endif + + igc_rx_buffer_flip(rx_buffer, truesize); } static struct sk_buff *igc_build_skb(struct igc_ring *rx_ring, @@ -1569,13 +1577,7 @@ static struct sk_buff *igc_build_skb(struct igc_ring *rx_ring, skb_reserve(skb, IGC_SKB_PAD); __skb_put(skb, size); - /* update buffer offset */ -#if (PAGE_SIZE < 8192) - rx_buffer->page_offset ^= truesize; -#else - rx_buffer->page_offset += truesize; -#endif - + igc_rx_buffer_flip(rx_buffer, truesize); return skb; } @@ -1621,11 +1623,7 @@ static struct sk_buff *igc_construct_skb(struct igc_ring *rx_ring, skb_add_rx_frag(skb, 0, rx_buffer->page, (va + headlen) - page_address(rx_buffer->page), size, truesize); -#if (PAGE_SIZE < 8192) - rx_buffer->page_offset ^= truesize; -#else - rx_buffer->page_offset += truesize; -#endif + igc_rx_buffer_flip(rx_buffer, truesize); } else { rx_buffer->pagecnt_bias++; } From patchwork Wed Oct 28 20:19:37 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andre Guedes X-Patchwork-Id: 1389400 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=osuosl.org (client-ip=140.211.166.136; helo=silver.osuosl.org; envelope-from=intel-wired-lan-bounces@osuosl.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=intel.com Received: from silver.osuosl.org (smtp3.osuosl.org [140.211.166.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4CM0LL3cphz9sVk for ; Thu, 29 Oct 2020 07:20:10 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by silver.osuosl.org (Postfix) with ESMTP id D7B93203DC; Wed, 28 Oct 2020 20:20:08 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from silver.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id cuu1bghrKY5M; Wed, 28 Oct 2020 20:20:02 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by silver.osuosl.org (Postfix) with ESMTP id 69BA820797; Wed, 28 Oct 2020 20:20:02 +0000 (UTC) X-Original-To: intel-wired-lan@lists.osuosl.org Delivered-To: intel-wired-lan@lists.osuosl.org Received: from fraxinus.osuosl.org (smtp4.osuosl.org [140.211.166.137]) by ash.osuosl.org (Postfix) with ESMTP id 2FF321BF836 for ; Wed, 28 Oct 2020 20:20:00 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by fraxinus.osuosl.org (Postfix) with ESMTP id 2C9FB85540 for ; Wed, 28 Oct 2020 20:20:00 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from fraxinus.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 54iH607LDOH3 for ; Wed, 28 Oct 2020 20:19:59 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by fraxinus.osuosl.org (Postfix) with ESMTPS id CC77C85D08 for ; Wed, 28 Oct 2020 20:19:58 +0000 (UTC) IronPort-SDR: bLcrWbZbeym9Fvzpndw9Yhi+UhQHqCZQS8RkbKYWmuzt4i7GOE6UPzPtIdbPO+VvtWs1cebJ+d rPhiF/TsCzOg== X-IronPort-AV: E=McAfee;i="6000,8403,9788"; a="253031523" X-IronPort-AV: E=Sophos;i="5.77,427,1596524400"; d="scan'208";a="253031523" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Oct 2020 13:19:55 -0700 IronPort-SDR: BdupcocdHWMrgQitm17xdjQFywSEWdRvhxbpZrKSPOL+Qb874qWS5sG7Z3RGTIe7VHG9/bD5nS rNJY6jZeE00Q== X-IronPort-AV: E=Sophos;i="5.77,427,1596524400"; d="scan'208";a="536374750" Received: from zcolon-mobl.amr.corp.intel.com ([10.209.108.9]) by orsmga005-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Oct 2020 13:19:54 -0700 From: Andre Guedes To: intel-wired-lan@lists.osuosl.org Date: Wed, 28 Oct 2020 13:19:37 -0700 Message-Id: <20201028201943.93147-5-andre.guedes@intel.com> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20201028201943.93147-1-andre.guedes@intel.com> References: <20201028201943.93147-1-andre.guedes@intel.com> MIME-Version: 1.0 Subject: [Intel-wired-lan] [PATCH v2 04/10] igc: Introduce igc_get_rx_frame_truesize() helper X-BeenThere: intel-wired-lan@osuosl.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel Wired Ethernet Linux Kernel Driver Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: intel-wired-lan-bounces@osuosl.org Sender: "Intel-wired-lan" The RX frame truesize calculation is scattered throughout the RX code. This patch creates the helper function igc_get_rx_frame_truesize() and uses it where applicable. Signed-off-by: Andre Guedes --- drivers/net/ethernet/intel/igc/igc_main.c | 29 ++++++++++++++--------- 1 file changed, 18 insertions(+), 11 deletions(-) diff --git a/drivers/net/ethernet/intel/igc/igc_main.c b/drivers/net/ethernet/intel/igc/igc_main.c index 31dc58a82cf3..15c67e5763d3 100644 --- a/drivers/net/ethernet/intel/igc/igc_main.c +++ b/drivers/net/ethernet/intel/igc/igc_main.c @@ -1523,6 +1523,22 @@ static void igc_rx_buffer_flip(struct igc_rx_buffer *buffer, #endif } +static unsigned int igc_get_rx_frame_truesize(struct igc_ring *ring, + unsigned int size) +{ + unsigned int truesize; + +#if (PAGE_SIZE < 8192) + truesize = igc_rx_pg_size(ring) / 2; +#else + truesize = ring_uses_build_skb(ring) ? + SKB_DATA_ALIGN(sizeof(struct skb_shared_info)) + + SKB_DATA_ALIGN(IGC_SKB_PAD + size) : + SKB_DATA_ALIGN(size); +#endif + return truesize; +} + /** * igc_add_rx_frag - Add contents of Rx buffer to sk_buff * @rx_ring: rx descriptor ring to transact packets on @@ -1557,12 +1573,7 @@ static struct sk_buff *igc_build_skb(struct igc_ring *rx_ring, unsigned int size) { void *va = page_address(rx_buffer->page) + rx_buffer->page_offset; -#if (PAGE_SIZE < 8192) - unsigned int truesize = igc_rx_pg_size(rx_ring) / 2; -#else - unsigned int truesize = SKB_DATA_ALIGN(sizeof(struct skb_shared_info)) + - SKB_DATA_ALIGN(IGC_SKB_PAD + size); -#endif + unsigned int truesize = igc_get_rx_frame_truesize(rx_ring, size); struct sk_buff *skb; /* prefetch first cache line of first page */ @@ -1587,11 +1598,7 @@ static struct sk_buff *igc_construct_skb(struct igc_ring *rx_ring, unsigned int size) { void *va = page_address(rx_buffer->page) + rx_buffer->page_offset; -#if (PAGE_SIZE < 8192) - unsigned int truesize = igc_rx_pg_size(rx_ring) / 2; -#else - unsigned int truesize = SKB_DATA_ALIGN(size); -#endif + unsigned int truesize = igc_get_rx_frame_truesize(rx_ring, size); unsigned int headlen; struct sk_buff *skb; From patchwork Wed Oct 28 20:19:38 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andre Guedes X-Patchwork-Id: 1389403 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=osuosl.org (client-ip=140.211.166.136; helo=silver.osuosl.org; envelope-from=intel-wired-lan-bounces@osuosl.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=intel.com Received: from silver.osuosl.org (smtp3.osuosl.org [140.211.166.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4CM0LS3N40z9sVh for ; Thu, 29 Oct 2020 07:20:16 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by silver.osuosl.org (Postfix) with ESMTP id 1166020373; Wed, 28 Oct 2020 20:20:15 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from silver.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id hO7a0EoKwAzy; Wed, 28 Oct 2020 20:20:06 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by silver.osuosl.org (Postfix) with ESMTP id BB93B21503; Wed, 28 Oct 2020 20:20:04 +0000 (UTC) X-Original-To: intel-wired-lan@lists.osuosl.org Delivered-To: intel-wired-lan@lists.osuosl.org Received: from silver.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by ash.osuosl.org (Postfix) with ESMTP id 31D081BF836 for ; Wed, 28 Oct 2020 20:20:02 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by silver.osuosl.org (Postfix) with ESMTP id 0DE7020797 for ; Wed, 28 Oct 2020 20:20:02 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from silver.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id SpLE2dzPmBTw for ; Wed, 28 Oct 2020 20:19:57 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by silver.osuosl.org (Postfix) with ESMTPS id 42A72203DC for ; Wed, 28 Oct 2020 20:19:57 +0000 (UTC) IronPort-SDR: Rqzw0N5B2C2dagOHhsJQWbwBDMz+YdWaerHiLOHNY1q+eQbp80yrcKCBJS577GzeH3J75fojya bFAVO5MK/CyA== X-IronPort-AV: E=McAfee;i="6000,8403,9788"; a="147614191" X-IronPort-AV: E=Sophos;i="5.77,427,1596524400"; d="scan'208";a="147614191" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Oct 2020 13:19:55 -0700 IronPort-SDR: kci0nNjtGtiIu4422KqI5OmV0K85rAb+GemVyy1CNlidJFaLNk3Nm2ZcME+ocMcSxkv7DJBf9q lKWaAI3EvGqQ== X-IronPort-AV: E=Sophos;i="5.77,427,1596524400"; d="scan'208";a="536374752" Received: from zcolon-mobl.amr.corp.intel.com ([10.209.108.9]) by orsmga005-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Oct 2020 13:19:54 -0700 From: Andre Guedes To: intel-wired-lan@lists.osuosl.org Date: Wed, 28 Oct 2020 13:19:38 -0700 Message-Id: <20201028201943.93147-6-andre.guedes@intel.com> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20201028201943.93147-1-andre.guedes@intel.com> References: <20201028201943.93147-1-andre.guedes@intel.com> MIME-Version: 1.0 Subject: [Intel-wired-lan] [PATCH v2 05/10] igc: Refactor rx timestamp handling X-BeenThere: intel-wired-lan@osuosl.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel Wired Ethernet Linux Kernel Driver Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: intel-wired-lan-bounces@osuosl.org Sender: "Intel-wired-lan" This patch refactors the rx timestamp handling in preparation to land XDP support. RX timestamps are put in the rx buffer by hardware, before the packet data. When creating the xdp buffer, we will need to check the rx descriptor to determine if the buffer contains timestamp information and consider the offset when setting xdp.data. The rx descriptor check is already done in igc_construct_skb(). To avoid code duplication, this patch moves the timestamp handling to igc_clean_rx_irq() so both skb and xdp paths can reuse it. Signed-off-by: Andre Guedes --- drivers/net/ethernet/intel/igc/igc.h | 3 +-- drivers/net/ethernet/intel/igc/igc_main.c | 30 +++++++++++++++-------- drivers/net/ethernet/intel/igc/igc_ptp.c | 25 ++++++++++--------- 3 files changed, 34 insertions(+), 24 deletions(-) diff --git a/drivers/net/ethernet/intel/igc/igc.h b/drivers/net/ethernet/intel/igc/igc.h index b66dda992d32..ae91d51073ca 100644 --- a/drivers/net/ethernet/intel/igc/igc.h +++ b/drivers/net/ethernet/intel/igc/igc.h @@ -552,8 +552,7 @@ void igc_ptp_init(struct igc_adapter *adapter); void igc_ptp_reset(struct igc_adapter *adapter); void igc_ptp_suspend(struct igc_adapter *adapter); void igc_ptp_stop(struct igc_adapter *adapter); -void igc_ptp_rx_pktstamp(struct igc_q_vector *q_vector, u32 *va, - struct sk_buff *skb); +ktime_t igc_ptp_rx_pktstamp(struct igc_adapter *adapter, u32 *buf); int igc_ptp_set_ts_config(struct net_device *netdev, struct ifreq *ifr); int igc_ptp_get_ts_config(struct net_device *netdev, struct ifreq *ifr); void igc_ptp_tx_hang(struct igc_adapter *adapter); diff --git a/drivers/net/ethernet/intel/igc/igc_main.c b/drivers/net/ethernet/intel/igc/igc_main.c index 15c67e5763d3..84ffde75e968 100644 --- a/drivers/net/ethernet/intel/igc/igc_main.c +++ b/drivers/net/ethernet/intel/igc/igc_main.c @@ -1594,10 +1594,11 @@ static struct sk_buff *igc_build_skb(struct igc_ring *rx_ring, static struct sk_buff *igc_construct_skb(struct igc_ring *rx_ring, struct igc_rx_buffer *rx_buffer, - union igc_adv_rx_desc *rx_desc, - unsigned int size) + unsigned int size, int pkt_offset, + ktime_t timestamp) { - void *va = page_address(rx_buffer->page) + rx_buffer->page_offset; + void *va = page_address(rx_buffer->page) + rx_buffer->page_offset + + pkt_offset; unsigned int truesize = igc_get_rx_frame_truesize(rx_ring, size); unsigned int headlen; struct sk_buff *skb; @@ -1610,11 +1611,8 @@ static struct sk_buff *igc_construct_skb(struct igc_ring *rx_ring, if (unlikely(!skb)) return NULL; - if (unlikely(igc_test_staterr(rx_desc, IGC_RXDADV_STAT_TSIP))) { - igc_ptp_rx_pktstamp(rx_ring->q_vector, va, skb); - va += IGC_TS_HDR_LEN; - size -= IGC_TS_HDR_LEN; - } + if (timestamp) + skb_hwtstamps(skb)->hwtstamp = timestamp; /* Determine available headroom for copy */ headlen = size; @@ -1913,6 +1911,8 @@ static int igc_clean_rx_irq(struct igc_q_vector *q_vector, const int budget) while (likely(total_packets < budget)) { union igc_adv_rx_desc *rx_desc; struct igc_rx_buffer *rx_buffer; + ktime_t timestamp = 0; + int pkt_offset = 0; unsigned int size; /* return some buffers to hardware, one at a time is too slow */ @@ -1934,14 +1934,24 @@ static int igc_clean_rx_irq(struct igc_q_vector *q_vector, const int budget) rx_buffer = igc_get_rx_buffer(rx_ring, size); + if (igc_test_staterr(rx_desc, IGC_RXDADV_STAT_TSIP)) { + void *pktbuf = page_address(rx_buffer->page) + + rx_buffer->page_offset; + + timestamp = igc_ptp_rx_pktstamp(q_vector->adapter, + pktbuf); + pkt_offset = IGC_TS_HDR_LEN; + size -= IGC_TS_HDR_LEN; + } + /* retrieve a buffer from the ring */ if (skb) igc_add_rx_frag(rx_ring, rx_buffer, skb, size); else if (ring_uses_build_skb(rx_ring)) skb = igc_build_skb(rx_ring, rx_buffer, rx_desc, size); else - skb = igc_construct_skb(rx_ring, rx_buffer, - rx_desc, size); + skb = igc_construct_skb(rx_ring, rx_buffer, size, + pkt_offset, timestamp); /* exit if we failed to retrieve a buffer */ if (!skb) { diff --git a/drivers/net/ethernet/intel/igc/igc_ptp.c b/drivers/net/ethernet/intel/igc/igc_ptp.c index 79873f6df335..4331c2dcffb2 100644 --- a/drivers/net/ethernet/intel/igc/igc_ptp.c +++ b/drivers/net/ethernet/intel/igc/igc_ptp.c @@ -155,20 +155,20 @@ static void igc_ptp_systim_to_hwtstamp(struct igc_adapter *adapter, /** * igc_ptp_rx_pktstamp - Retrieve timestamp from rx packet buffer - * @q_vector: Pointer to interrupt specific structure - * @va: Pointer to address containing Rx buffer - * @skb: Buffer containing timestamp and packet + * @adapter: Pointer to adapter the packet buffer belongs to + * @buf: Pointer to packet buffer * * This function retrieves the timestamp saved in the beginning of packet * buffer. While two timestamps are available, one in timer0 reference and the * other in timer1 reference, this function considers only the timestamp in * timer0 reference. + * + * Returns: Timestamp value. */ -void igc_ptp_rx_pktstamp(struct igc_q_vector *q_vector, u32 *va, - struct sk_buff *skb) +ktime_t igc_ptp_rx_pktstamp(struct igc_adapter *adapter, u32 *buf) { - struct igc_adapter *adapter = q_vector->adapter; - u64 regval; + ktime_t timestamp; + u32 secs, nsecs; int adjust; /* Timestamps are saved in little endian at the beginning of the packet @@ -180,9 +180,10 @@ void igc_ptp_rx_pktstamp(struct igc_q_vector *q_vector, u32 *va, * SYSTIML holds the nanoseconds part while SYSTIMH holds the seconds * part of the timestamp. */ - regval = le32_to_cpu(va[2]); - regval |= (u64)le32_to_cpu(va[3]) << 32; - igc_ptp_systim_to_hwtstamp(adapter, skb_hwtstamps(skb), regval); + nsecs = le32_to_cpu(buf[2]); + secs = le32_to_cpu(buf[3]); + + timestamp = ktime_set(secs, nsecs); /* Adjust timestamp for the RX latency based on link speed */ switch (adapter->link_speed) { @@ -203,8 +204,8 @@ void igc_ptp_rx_pktstamp(struct igc_q_vector *q_vector, u32 *va, netdev_warn_once(adapter->netdev, "Imprecise timestamp\n"); break; } - skb_hwtstamps(skb)->hwtstamp = - ktime_sub_ns(skb_hwtstamps(skb)->hwtstamp, adjust); + + return ktime_sub_ns(timestamp, adjust); } static void igc_ptp_disable_rx_timestamp(struct igc_adapter *adapter) From patchwork Wed Oct 28 20:19:39 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andre Guedes X-Patchwork-Id: 1389395 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=osuosl.org (client-ip=140.211.166.137; helo=fraxinus.osuosl.org; envelope-from=intel-wired-lan-bounces@osuosl.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=intel.com Received: from fraxinus.osuosl.org (smtp4.osuosl.org [140.211.166.137]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4CM0LC4nP0z9sVt for ; Thu, 29 Oct 2020 07:20:03 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by fraxinus.osuosl.org (Postfix) with ESMTP id 1194E854DB; Wed, 28 Oct 2020 20:20:02 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from fraxinus.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Ky851Vu-NZz1; Wed, 28 Oct 2020 20:20:01 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by fraxinus.osuosl.org (Postfix) with ESMTP id F261686529; Wed, 28 Oct 2020 20:20:00 +0000 (UTC) X-Original-To: intel-wired-lan@lists.osuosl.org Delivered-To: intel-wired-lan@lists.osuosl.org Received: from silver.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by ash.osuosl.org (Postfix) with ESMTP id 6A45E1BF836 for ; Wed, 28 Oct 2020 20:19:59 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by silver.osuosl.org (Postfix) with ESMTP id 46F7720797 for ; Wed, 28 Oct 2020 20:19:59 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from silver.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id a5+0dz0vd4Pu for ; Wed, 28 Oct 2020 20:19:57 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by silver.osuosl.org (Postfix) with ESMTPS id 8947A203BB for ; Wed, 28 Oct 2020 20:19:57 +0000 (UTC) IronPort-SDR: pluVctZRPk24+COj8xdrBprG0udbh43TNUHq1JHMVlj+j7Z26Nho3iF6v7V9i+SsDfillvPyy8 q4gb5v7mlYfA== X-IronPort-AV: E=McAfee;i="6000,8403,9788"; a="147614192" X-IronPort-AV: E=Sophos;i="5.77,427,1596524400"; d="scan'208";a="147614192" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Oct 2020 13:19:55 -0700 IronPort-SDR: zMf354C+eAfycwwGkm+RvIsHNZ64kNHWi7G1HhOOItK3P1KmTzodcyFHQo+7LIbooN4Jh7ppLM gmZRnFT4AQtQ== X-IronPort-AV: E=Sophos;i="5.77,427,1596524400"; d="scan'208";a="536374753" Received: from zcolon-mobl.amr.corp.intel.com ([10.209.108.9]) by orsmga005-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Oct 2020 13:19:55 -0700 From: Andre Guedes To: intel-wired-lan@lists.osuosl.org Date: Wed, 28 Oct 2020 13:19:39 -0700 Message-Id: <20201028201943.93147-7-andre.guedes@intel.com> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20201028201943.93147-1-andre.guedes@intel.com> References: <20201028201943.93147-1-andre.guedes@intel.com> MIME-Version: 1.0 Subject: [Intel-wired-lan] [PATCH v2 06/10] igc: Add set/clear large buffer helpers X-BeenThere: intel-wired-lan@osuosl.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel Wired Ethernet Linux Kernel Driver Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: intel-wired-lan-bounces@osuosl.org Sender: "Intel-wired-lan" While commit 13b5b7fd6a4a ("igc: Add support for Tx/Rx rings") introduced code to handle larger packet buffers, it missed the set/clear helpers which enable/disable that feature. This patch introduces the missing helpers which will be use in the next patch. Signed-off-by: Andre Guedes --- drivers/net/ethernet/intel/igc/igc.h | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/drivers/net/ethernet/intel/igc/igc.h b/drivers/net/ethernet/intel/igc/igc.h index ae91d51073ca..e72f1fc772aa 100644 --- a/drivers/net/ethernet/intel/igc/igc.h +++ b/drivers/net/ethernet/intel/igc/igc.h @@ -509,6 +509,10 @@ enum igc_ring_flags_t { #define ring_uses_large_buffer(ring) \ test_bit(IGC_RING_FLAG_RX_3K_BUFFER, &(ring)->flags) +#define set_ring_uses_large_buffer(ring) \ + set_bit(IGC_RING_FLAG_RX_3K_BUFFER, &(ring)->flags) +#define clear_ring_uses_large_buffer(ring) \ + clear_bit(IGC_RING_FLAG_RX_3K_BUFFER, &(ring)->flags) #define ring_uses_build_skb(ring) \ test_bit(IGC_RING_FLAG_RX_BUILD_SKB_ENABLED, &(ring)->flags) From patchwork Wed Oct 28 20:19:40 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andre Guedes X-Patchwork-Id: 1389398 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=osuosl.org (client-ip=140.211.166.133; helo=hemlock.osuosl.org; envelope-from=intel-wired-lan-bounces@osuosl.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=intel.com Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4CM0LJ082Zz9sVk for ; Thu, 29 Oct 2020 07:20:08 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id 9EB96874DA; Wed, 28 Oct 2020 20:20:06 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id YzWqjFpP5dlQ; Wed, 28 Oct 2020 20:20:05 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by hemlock.osuosl.org (Postfix) with ESMTP id 9A60987477; Wed, 28 Oct 2020 20:20:05 +0000 (UTC) X-Original-To: intel-wired-lan@lists.osuosl.org Delivered-To: intel-wired-lan@lists.osuosl.org Received: from silver.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by ash.osuosl.org (Postfix) with ESMTP id 95E4B1BF836 for ; Wed, 28 Oct 2020 20:20:03 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by silver.osuosl.org (Postfix) with ESMTP id 88F94214D2 for ; Wed, 28 Oct 2020 20:20:03 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from silver.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 3awwAnMdzGwV for ; Wed, 28 Oct 2020 20:19:57 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by silver.osuosl.org (Postfix) with ESMTPS id 6B1972010F for ; Wed, 28 Oct 2020 20:19:57 +0000 (UTC) IronPort-SDR: e2Ue+atGZYZgx5QMF90yJbbNrggatUn8CBOloxoGGz+4d56cumtB8rxEJAmhOfcSeHF8YI/8iy APAN9blq+Z6Q== X-IronPort-AV: E=McAfee;i="6000,8403,9788"; a="147614194" X-IronPort-AV: E=Sophos;i="5.77,427,1596524400"; d="scan'208";a="147614194" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Oct 2020 13:19:55 -0700 IronPort-SDR: UNfea66dXjW/3sjO1bMQ8Z9DicBpp6LDenbDfwMeYV65PFk5pwUU/1xmuFk6ImdPehWUdCO++y 54j2+oCykHYA== X-IronPort-AV: E=Sophos;i="5.77,427,1596524400"; d="scan'208";a="536374756" Received: from zcolon-mobl.amr.corp.intel.com ([10.209.108.9]) by orsmga005-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Oct 2020 13:19:55 -0700 From: Andre Guedes To: intel-wired-lan@lists.osuosl.org Date: Wed, 28 Oct 2020 13:19:40 -0700 Message-Id: <20201028201943.93147-8-andre.guedes@intel.com> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20201028201943.93147-1-andre.guedes@intel.com> References: <20201028201943.93147-1-andre.guedes@intel.com> MIME-Version: 1.0 Subject: [Intel-wired-lan] [PATCH v2 07/10] igc: Add initial XDP support X-BeenThere: intel-wired-lan@osuosl.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel Wired Ethernet Linux Kernel Driver Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: intel-wired-lan-bounces@osuosl.org Sender: "Intel-wired-lan" This patch adds the initial XDP support to the igc driver. For now, only XDP_PASS, XDP_DROP, XDP_ABORTED actions are supported. Upcoming patches will add support for the remaining XDP actions. XDP-specific helpers are defined in a new file, igc_xdp.c. These helpers are utilized in igc_main.c to implement the ndo_bpf callback and to handle XDP programs execution in igc_clean_rx_irq(). By default, the driver uses rx buffers with 2 KB size. When XDP is enabled, it uses larger buffers so we have enough space to accommodate the headroom and tailroom required by XDP infrastructure. Also, the driver doesn't support XDP functionality with frames that span over multiple buffers so jumbo frames are not allowed for now. The approach implemented by this patch follows the approach implemented in other Intel drivers as much as possible for the sake of consistency across the drivers. Quick comment regarding igc_build_skb(): this patch doesn't touch it because the function is never called. It seems its support is incomplete/in progress. The function was added by commit 0507ef8a0372b ("igc: Add transmit and receive fastpath and interrupt handlers") but ring_uses_build_skb() always return False since the IGC_RING_FLAG_RX_ BUILD_SKB_ENABLED isn't set anywhere in the driver code. This patch has been tested with the sample app "xdp1" located in samples/bpf/ dir. Signed-off-by: Andre Guedes --- drivers/net/ethernet/intel/igc/Makefile | 2 +- drivers/net/ethernet/intel/igc/igc.h | 2 + drivers/net/ethernet/intel/igc/igc_main.c | 73 ++++++++++++++++++---- drivers/net/ethernet/intel/igc/igc_xdp.c | 76 +++++++++++++++++++++++ drivers/net/ethernet/intel/igc/igc_xdp.h | 18 ++++++ 5 files changed, 159 insertions(+), 12 deletions(-) create mode 100644 drivers/net/ethernet/intel/igc/igc_xdp.c create mode 100644 drivers/net/ethernet/intel/igc/igc_xdp.h diff --git a/drivers/net/ethernet/intel/igc/Makefile b/drivers/net/ethernet/intel/igc/Makefile index 1c3051db9085..95d1e8c490a4 100644 --- a/drivers/net/ethernet/intel/igc/Makefile +++ b/drivers/net/ethernet/intel/igc/Makefile @@ -8,4 +8,4 @@ obj-$(CONFIG_IGC) += igc.o igc-objs := igc_main.o igc_mac.o igc_i225.o igc_base.o igc_nvm.o igc_phy.o \ -igc_diag.o igc_ethtool.o igc_ptp.o igc_dump.o igc_tsn.o +igc_diag.o igc_ethtool.o igc_ptp.o igc_dump.o igc_tsn.o igc_xdp.o diff --git a/drivers/net/ethernet/intel/igc/igc.h b/drivers/net/ethernet/intel/igc/igc.h index e72f1fc772aa..5c2f363106ae 100644 --- a/drivers/net/ethernet/intel/igc/igc.h +++ b/drivers/net/ethernet/intel/igc/igc.h @@ -224,6 +224,8 @@ struct igc_adapter { struct mutex ptm_time_lock; /* protects host and device timestamps */ ktime_t ptm_device_time; struct system_counterval_t ptm_host_time; + + struct bpf_prog *xdp_prog; }; void igc_up(struct igc_adapter *adapter); diff --git a/drivers/net/ethernet/intel/igc/igc_main.c b/drivers/net/ethernet/intel/igc/igc_main.c index 84ffde75e968..585ad575d9fd 100644 --- a/drivers/net/ethernet/intel/igc/igc_main.c +++ b/drivers/net/ethernet/intel/igc/igc_main.c @@ -17,6 +17,7 @@ #include "igc.h" #include "igc_hw.h" #include "igc_tsn.h" +#include "igc_xdp.h" #define DRV_SUMMARY "Intel(R) 2.5G Ethernet Linux Driver" @@ -346,6 +347,8 @@ static void igc_clean_rx_ring(struct igc_ring *rx_ring) { u16 i = rx_ring->next_to_clean; + clear_ring_uses_large_buffer(rx_ring); + dev_kfree_skb(rx_ring->skb); rx_ring->skb = NULL; @@ -514,6 +517,9 @@ static void igc_configure_rx_ring(struct igc_adapter *adapter, u32 srrctl = 0, rxdctl = 0; u64 rdba = ring->dma; + if (igc_xdp_is_enabled(adapter)) + set_ring_uses_large_buffer(ring); + /* disable the queue */ wr32(IGC_RXDCTL(reg_idx), 0); @@ -1594,12 +1600,12 @@ static struct sk_buff *igc_build_skb(struct igc_ring *rx_ring, static struct sk_buff *igc_construct_skb(struct igc_ring *rx_ring, struct igc_rx_buffer *rx_buffer, - unsigned int size, int pkt_offset, + struct xdp_buff *xdp, ktime_t timestamp) { - void *va = page_address(rx_buffer->page) + rx_buffer->page_offset + - pkt_offset; + unsigned int size = xdp->data_end - xdp->data; unsigned int truesize = igc_get_rx_frame_truesize(rx_ring, size); + void *va = xdp->data; unsigned int headlen; struct sk_buff *skb; @@ -1748,6 +1754,10 @@ static bool igc_cleanup_headers(struct igc_ring *rx_ring, union igc_adv_rx_desc *rx_desc, struct sk_buff *skb) { + /* XDP packets use error pointer so abort at this point */ + if (IS_ERR(skb)) + return true; + if (unlikely(igc_test_staterr(rx_desc, IGC_RXDEXT_STATERR_RXE))) { struct net_device *netdev = rx_ring->netdev; @@ -1787,7 +1797,14 @@ static void igc_put_rx_buffer(struct igc_ring *rx_ring, static inline unsigned int igc_rx_offset(struct igc_ring *rx_ring) { - return ring_uses_build_skb(rx_ring) ? IGC_SKB_PAD : 0; + struct igc_adapter *adapter = rx_ring->q_vector->adapter; + + if (ring_uses_build_skb(rx_ring)) + return IGC_SKB_PAD; + if (igc_xdp_is_enabled(adapter)) + return XDP_PACKET_HEADROOM; + + return 0; } static bool igc_alloc_mapped_page(struct igc_ring *rx_ring, @@ -1912,8 +1929,10 @@ static int igc_clean_rx_irq(struct igc_q_vector *q_vector, const int budget) union igc_adv_rx_desc *rx_desc; struct igc_rx_buffer *rx_buffer; ktime_t timestamp = 0; + struct xdp_buff xdp; int pkt_offset = 0; unsigned int size; + void *pktbuf; /* return some buffers to hardware, one at a time is too slow */ if (cleaned_count >= IGC_RX_BUFFER_WRITE) { @@ -1934,24 +1953,38 @@ static int igc_clean_rx_irq(struct igc_q_vector *q_vector, const int budget) rx_buffer = igc_get_rx_buffer(rx_ring, size); - if (igc_test_staterr(rx_desc, IGC_RXDADV_STAT_TSIP)) { - void *pktbuf = page_address(rx_buffer->page) + - rx_buffer->page_offset; + pktbuf = page_address(rx_buffer->page) + rx_buffer->page_offset; + if (igc_test_staterr(rx_desc, IGC_RXDADV_STAT_TSIP)) { timestamp = igc_ptp_rx_pktstamp(q_vector->adapter, pktbuf); pkt_offset = IGC_TS_HDR_LEN; size -= IGC_TS_HDR_LEN; } - /* retrieve a buffer from the ring */ - if (skb) + if (!skb) { + struct igc_adapter *adapter = q_vector->adapter; + + xdp.data = pktbuf + pkt_offset; + xdp.data_end = xdp.data + size; + xdp.data_hard_start = pktbuf - igc_rx_offset(rx_ring); + xdp_set_data_meta_invalid(&xdp); + xdp.frame_sz = igc_get_rx_frame_truesize(rx_ring, size); + + skb = igc_xdp_run_prog(adapter, &xdp); + } + + if (IS_ERR(skb)) { + rx_buffer->pagecnt_bias++; + total_packets++; + total_bytes += size; + } else if (skb) igc_add_rx_frag(rx_ring, rx_buffer, skb, size); else if (ring_uses_build_skb(rx_ring)) skb = igc_build_skb(rx_ring, rx_buffer, rx_desc, size); else - skb = igc_construct_skb(rx_ring, rx_buffer, size, - pkt_offset, timestamp); + skb = igc_construct_skb(rx_ring, rx_buffer, &xdp, + timestamp); /* exit if we failed to retrieve a buffer */ if (!skb) { @@ -3893,6 +3926,11 @@ static int igc_change_mtu(struct net_device *netdev, int new_mtu) int max_frame = new_mtu + ETH_HLEN + ETH_FCS_LEN + VLAN_HLEN; struct igc_adapter *adapter = netdev_priv(netdev); + if (igc_xdp_is_enabled(adapter) && new_mtu > ETH_DATA_LEN) { + netdev_dbg(netdev, "Jumbo frames not supported with XDP"); + return -EINVAL; + } + /* adjust max frame to be at least the size of a standard frame */ if (max_frame < (ETH_FRAME_LEN + ETH_FCS_LEN)) max_frame = ETH_FRAME_LEN + ETH_FCS_LEN; @@ -4881,6 +4919,18 @@ static int igc_setup_tc(struct net_device *dev, enum tc_setup_type type, } } +static int igc_bpf(struct net_device *dev, struct netdev_bpf *bpf) +{ + struct igc_adapter *adapter = netdev_priv(dev); + + switch (bpf->command) { + case XDP_SETUP_PROG: + return igc_xdp_set_prog(adapter, bpf->prog, bpf->extack); + default: + return -EOPNOTSUPP; + } +} + static const struct net_device_ops igc_netdev_ops = { .ndo_open = igc_open, .ndo_stop = igc_close, @@ -4894,6 +4944,7 @@ static const struct net_device_ops igc_netdev_ops = { .ndo_features_check = igc_features_check, .ndo_do_ioctl = igc_ioctl, .ndo_setup_tc = igc_setup_tc, + .ndo_bpf = igc_bpf, }; /* PCIe configuration access */ diff --git a/drivers/net/ethernet/intel/igc/igc_xdp.c b/drivers/net/ethernet/intel/igc/igc_xdp.c new file mode 100644 index 000000000000..faec0bedc550 --- /dev/null +++ b/drivers/net/ethernet/intel/igc/igc_xdp.c @@ -0,0 +1,76 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) 2020, Intel Corporation. */ + +#include + +#include "igc.h" +#include "igc_xdp.h" + +int igc_xdp_set_prog(struct igc_adapter *adapter, struct bpf_prog *prog, + struct netlink_ext_ack *extack) +{ + struct net_device *dev = adapter->netdev; + bool if_running = netif_running(dev); + struct bpf_prog *old_prog; + + if (dev->mtu > ETH_DATA_LEN) { + /* For now, the driver doesn't support XDP functionality with + * jumbo frames so we return error. + */ + NL_SET_ERR_MSG_MOD(extack, "Jumbo frames not supported"); + return -EOPNOTSUPP; + } + + if (if_running) + igc_close(dev); + + old_prog = xchg(&adapter->xdp_prog, prog); + if (old_prog) + bpf_prog_put(old_prog); + + if (if_running) + igc_open(dev); + + return 0; +} + +struct sk_buff *igc_xdp_run_prog(struct igc_adapter *adapter, + struct xdp_buff *xdp) +{ + struct bpf_prog *prog; + int res; + u32 act; + + rcu_read_lock(); + + prog = READ_ONCE(adapter->xdp_prog); + if (!prog) { + res = IGC_XDP_PASS; + goto unlock; + } + + act = bpf_prog_run_xdp(prog, xdp); + switch (act) { + case XDP_PASS: + res = IGC_XDP_PASS; + break; + default: + bpf_warn_invalid_xdp_action(act); + fallthrough; + case XDP_ABORTED: + trace_xdp_exception(adapter->netdev, prog, act); + fallthrough; + case XDP_DROP: + res = IGC_XDP_CONSUMED; + break; + } + +unlock: + rcu_read_unlock(); + return ERR_PTR(-res); +} + +bool igc_xdp_is_enabled(struct igc_adapter *adapter) +{ + return !!adapter->xdp_prog; +} diff --git a/drivers/net/ethernet/intel/igc/igc_xdp.h b/drivers/net/ethernet/intel/igc/igc_xdp.h new file mode 100644 index 000000000000..ab36ab4d48fd --- /dev/null +++ b/drivers/net/ethernet/intel/igc/igc_xdp.h @@ -0,0 +1,18 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Copyright (c) 2020, Intel Corporation. */ + +#ifndef _IGC_XDP_H_ +#define _IGC_XDP_H_ + +#define IGC_XDP_PASS 0 +#define IGC_XDP_CONSUMED BIT(0) + +int igc_xdp_set_prog(struct igc_adapter *adapter, struct bpf_prog *prog, + struct netlink_ext_ack *extack); + +struct sk_buff *igc_xdp_run_prog(struct igc_adapter *adapter, + struct xdp_buff *xdp); + +bool igc_xdp_is_enabled(struct igc_adapter *adapter); + +#endif /* _IGC_XDP_H_ */ From patchwork Wed Oct 28 20:19:41 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andre Guedes X-Patchwork-Id: 1389401 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=osuosl.org (client-ip=140.211.166.133; helo=hemlock.osuosl.org; envelope-from=intel-wired-lan-bounces@osuosl.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=intel.com Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4CM0LM5T1xz9sVh for ; Thu, 29 Oct 2020 07:20:11 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id 3749C874DB; Wed, 28 Oct 2020 20:20:10 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id LbHUl2i-DyC0; Wed, 28 Oct 2020 20:20:08 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by hemlock.osuosl.org (Postfix) with ESMTP id 336C3874D0; Wed, 28 Oct 2020 20:20:08 +0000 (UTC) X-Original-To: intel-wired-lan@lists.osuosl.org Delivered-To: intel-wired-lan@lists.osuosl.org Received: from silver.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by ash.osuosl.org (Postfix) with ESMTP id 4184B1BF836 for ; Wed, 28 Oct 2020 20:20:06 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by silver.osuosl.org (Postfix) with ESMTP id 2B714214EB for ; Wed, 28 Oct 2020 20:20:06 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from silver.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id jmYi0XllvlWW for ; Wed, 28 Oct 2020 20:19:57 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by silver.osuosl.org (Postfix) with ESMTPS id 0518520373 for ; Wed, 28 Oct 2020 20:19:56 +0000 (UTC) IronPort-SDR: 7753dMcSqsn2qKESPMQaxbZkz4gtq2ksiCMOJoVoGhEVR6Q6eWFcBwOuO3tM5hMw2qfwkNEfI/ ufYNUCX68W6Q== X-IronPort-AV: E=McAfee;i="6000,8403,9788"; a="147614197" X-IronPort-AV: E=Sophos;i="5.77,427,1596524400"; d="scan'208";a="147614197" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Oct 2020 13:19:56 -0700 IronPort-SDR: lzMFbdz+p0x26rO7wqjP5Cv0YFXGrzI3UL79sDyhfka7OO1y3eLAqQxoZCAI/xP+sBX3TnAt1s Np/4U4SCfB+w== X-IronPort-AV: E=Sophos;i="5.77,427,1596524400"; d="scan'208";a="536374759" Received: from zcolon-mobl.amr.corp.intel.com ([10.209.108.9]) by orsmga005-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Oct 2020 13:19:55 -0700 From: Andre Guedes To: intel-wired-lan@lists.osuosl.org Date: Wed, 28 Oct 2020 13:19:41 -0700 Message-Id: <20201028201943.93147-9-andre.guedes@intel.com> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20201028201943.93147-1-andre.guedes@intel.com> References: <20201028201943.93147-1-andre.guedes@intel.com> MIME-Version: 1.0 Subject: [Intel-wired-lan] [PATCH v2 08/10] igc: Add support for XDP_TX action X-BeenThere: intel-wired-lan@osuosl.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel Wired Ethernet Linux Kernel Driver Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: intel-wired-lan-bounces@osuosl.org Sender: "Intel-wired-lan" This patch adds support for XDP_TX action which enables XDP programs to transmit back receiving frames. The main changes in igc_xdp.c are related to the transmission of the packet (e.g. select the transmission queue, prepare buffer, prepare descriptor) while the changes in igc_main.c are related to tx_buffer management and XDP resources setup required to perform the XDP_TX action. I225 controller has only 4 tx hardware queues. Since XDP programs may not even issue an XDP_TX action, this patch doesn't reserve dedicated queues just for XDP like other Intel drivers do. Instead, the queues are shared between the network stack and XDP. The netdev queue lock is used to ensure mutual exclusion. Since frames can now be transmitted via XDP_TX, the igc_tx_buffer structure is modified so we are able to save a reference to the xdp frame for later clean up once the packet is transmitted. The tx_buffer is mapped to either a skb or a xdpf so we use a union to save the skb or xdpf pointer and have a bit in tx_flags to indicate which field to use. This patch has been tested with the sample app "xdp2" located in samples/bpf/ dir. Signed-off-by: Andre Guedes --- drivers/net/ethernet/intel/igc/igc.h | 9 +- drivers/net/ethernet/intel/igc/igc_main.c | 74 +++++++++++-- drivers/net/ethernet/intel/igc/igc_xdp.c | 124 ++++++++++++++++++++++ drivers/net/ethernet/intel/igc/igc_xdp.h | 6 ++ 4 files changed, 202 insertions(+), 11 deletions(-) diff --git a/drivers/net/ethernet/intel/igc/igc.h b/drivers/net/ethernet/intel/igc/igc.h index 5c2f363106ae..9c566d930ab0 100644 --- a/drivers/net/ethernet/intel/igc/igc.h +++ b/drivers/net/ethernet/intel/igc/igc.h @@ -112,6 +112,8 @@ struct igc_ring { struct sk_buff *skb; }; }; + + struct xdp_rxq_info xdp_rxq; } ____cacheline_internodealigned_in_smp; /* Board specific private data structure */ @@ -380,6 +382,8 @@ enum igc_tx_flags { /* olinfo flags */ IGC_TX_FLAGS_IPV4 = 0x10, IGC_TX_FLAGS_CSUM = 0x20, + + IGC_TX_FLAGS_XDP = 0x100, }; enum igc_boards { @@ -402,7 +406,10 @@ enum igc_boards { struct igc_tx_buffer { union igc_adv_tx_desc *next_to_watch; unsigned long time_stamp; - struct sk_buff *skb; + union { + struct sk_buff *skb; + struct xdp_frame *xdpf; + }; unsigned int bytecount; u16 gso_segs; __be16 protocol; diff --git a/drivers/net/ethernet/intel/igc/igc_main.c b/drivers/net/ethernet/intel/igc/igc_main.c index 585ad575d9fd..7bcfc72b00fc 100644 --- a/drivers/net/ethernet/intel/igc/igc_main.c +++ b/drivers/net/ethernet/intel/igc/igc_main.c @@ -178,8 +178,10 @@ static void igc_clean_tx_ring(struct igc_ring *tx_ring) while (i != tx_ring->next_to_use) { union igc_adv_tx_desc *eop_desc, *tx_desc; - /* Free all the Tx ring sk_buffs */ - dev_kfree_skb_any(tx_buffer->skb); + if (tx_buffer->tx_flags & IGC_TX_FLAGS_XDP) + xdp_return_frame(tx_buffer->xdpf); + else + dev_kfree_skb_any(tx_buffer->skb); /* unmap skb header data */ dma_unmap_single(tx_ring->dev, @@ -407,6 +409,8 @@ void igc_free_rx_resources(struct igc_ring *rx_ring) { igc_clean_rx_ring(rx_ring); + igc_xdp_unregister_rxq_info(rx_ring); + vfree(rx_ring->rx_buffer_info); rx_ring->rx_buffer_info = NULL; @@ -444,7 +448,11 @@ int igc_setup_rx_resources(struct igc_ring *rx_ring) { struct net_device *ndev = rx_ring->netdev; struct device *dev = rx_ring->dev; - int size, desc_len; + int size, desc_len, res; + + res = igc_xdp_register_rxq_info(rx_ring); + if (res < 0) + return res; size = sizeof(struct igc_rx_buffer) * rx_ring->count; rx_ring->rx_buffer_info = vzalloc(size); @@ -470,6 +478,7 @@ int igc_setup_rx_resources(struct igc_ring *rx_ring) return 0; err: + igc_xdp_unregister_rxq_info(rx_ring); vfree(rx_ring->rx_buffer_info); rx_ring->rx_buffer_info = NULL; netdev_err(ndev, "Unable to allocate memory for Rx descriptor ring\n"); @@ -1918,20 +1927,49 @@ static void igc_alloc_rx_buffers(struct igc_ring *rx_ring, u16 cleaned_count) } } +/* This function assumes __netif_tx_lock is held by the caller. */ +static void igc_flush_tx_descriptors(struct igc_ring *ring) +{ + /* Once tail pointer is updated, hardware can fetch the descriptors + * any time so we issue a write membar here to ensure all memory + * writes are complete before the tail pointer is updated. + */ + wmb(); + writel(ring->next_to_use, ring->tail); +} + +static void igc_finalize_xdp(struct igc_adapter *adapter, int status) +{ + int cpu = smp_processor_id(); + struct netdev_queue *nq; + struct igc_ring *ring; + + if (status & IGC_XDP_TX) { + ring = igc_xdp_get_tx_ring(adapter, cpu); + nq = txring_txq(ring); + + __netif_tx_lock(nq, cpu); + igc_flush_tx_descriptors(ring); + __netif_tx_unlock(nq); + } +} + static int igc_clean_rx_irq(struct igc_q_vector *q_vector, const int budget) { unsigned int total_bytes = 0, total_packets = 0; + struct igc_adapter *adapter = q_vector->adapter; struct igc_ring *rx_ring = q_vector->rx.ring; struct sk_buff *skb = rx_ring->skb; u16 cleaned_count = igc_desc_unused(rx_ring); + int xdp_status = 0; while (likely(total_packets < budget)) { union igc_adv_rx_desc *rx_desc; struct igc_rx_buffer *rx_buffer; + unsigned int size, truesize; ktime_t timestamp = 0; struct xdp_buff xdp; int pkt_offset = 0; - unsigned int size; void *pktbuf; /* return some buffers to hardware, one at a time is too slow */ @@ -1952,6 +1990,7 @@ static int igc_clean_rx_irq(struct igc_q_vector *q_vector, const int budget) dma_rmb(); rx_buffer = igc_get_rx_buffer(rx_ring, size); + truesize = igc_get_rx_frame_truesize(rx_ring, size); pktbuf = page_address(rx_buffer->page) + rx_buffer->page_offset; @@ -1963,19 +2002,29 @@ static int igc_clean_rx_irq(struct igc_q_vector *q_vector, const int budget) } if (!skb) { - struct igc_adapter *adapter = q_vector->adapter; - xdp.data = pktbuf + pkt_offset; xdp.data_end = xdp.data + size; xdp.data_hard_start = pktbuf - igc_rx_offset(rx_ring); xdp_set_data_meta_invalid(&xdp); - xdp.frame_sz = igc_get_rx_frame_truesize(rx_ring, size); + xdp.frame_sz = truesize; + xdp.rxq = &rx_ring->xdp_rxq; skb = igc_xdp_run_prog(adapter, &xdp); } if (IS_ERR(skb)) { - rx_buffer->pagecnt_bias++; + unsigned int xdp_res = -PTR_ERR(skb); + + switch (xdp_res) { + case IGC_XDP_CONSUMED: + rx_buffer->pagecnt_bias++; + break; + case IGC_XDP_TX: + igc_rx_buffer_flip(rx_buffer, truesize); + xdp_status |= xdp_res; + break; + } + total_packets++; total_bytes += size; } else if (skb) @@ -2021,6 +2070,9 @@ static int igc_clean_rx_irq(struct igc_q_vector *q_vector, const int budget) total_packets++; } + if (xdp_status) + igc_finalize_xdp(adapter, xdp_status); + /* place incomplete frames back on ring for completion */ rx_ring->skb = skb; @@ -2082,8 +2134,10 @@ static bool igc_clean_tx_irq(struct igc_q_vector *q_vector, int napi_budget) total_bytes += tx_buffer->bytecount; total_packets += tx_buffer->gso_segs; - /* free the skb */ - napi_consume_skb(tx_buffer->skb, napi_budget); + if (tx_buffer->tx_flags & IGC_TX_FLAGS_XDP) + xdp_return_frame(tx_buffer->xdpf); + else + napi_consume_skb(tx_buffer->skb, napi_budget); /* unmap skb header data */ dma_unmap_single(tx_ring->dev, diff --git a/drivers/net/ethernet/intel/igc/igc_xdp.c b/drivers/net/ethernet/intel/igc/igc_xdp.c index faec0bedc550..17852cc3a438 100644 --- a/drivers/net/ethernet/intel/igc/igc_xdp.c +++ b/drivers/net/ethernet/intel/igc/igc_xdp.c @@ -34,6 +34,97 @@ int igc_xdp_set_prog(struct igc_adapter *adapter, struct bpf_prog *prog, return 0; } +static int igc_xdp_init_tx_buffer(struct igc_tx_buffer *buffer, + struct xdp_frame *xdpf, + struct igc_ring *ring) +{ + dma_addr_t dma; + + dma = dma_map_single(ring->dev, xdpf->data, xdpf->len, DMA_TO_DEVICE); + if (dma_mapping_error(ring->dev, dma)) { + netdev_err_once(ring->netdev, "Failed to map DMA for TX\n"); + return -ENOMEM; + } + + buffer->xdpf = xdpf; + buffer->tx_flags = IGC_TX_FLAGS_XDP; + buffer->protocol = 0; + buffer->bytecount = xdpf->len; + buffer->gso_segs = 1; + buffer->time_stamp = jiffies; + dma_unmap_len_set(buffer, len, xdpf->len); + dma_unmap_addr_set(buffer, dma, dma); + return 0; +} + +/* This function requires __netif_tx_lock is held by the caller. */ +static int igc_xdp_init_tx_descriptor(struct igc_ring *ring, + struct xdp_frame *xdpf) +{ + struct igc_tx_buffer *buffer; + union igc_adv_tx_desc *desc; + u32 cmd_type, olinfo_status; + int err; + + if (!igc_desc_unused(ring)) + return -EBUSY; + + buffer = &ring->tx_buffer_info[ring->next_to_use]; + err = igc_xdp_init_tx_buffer(buffer, xdpf, ring); + if (err) + return err; + + cmd_type = IGC_ADVTXD_DTYP_DATA | IGC_ADVTXD_DCMD_DEXT | + IGC_ADVTXD_DCMD_IFCS | IGC_TXD_DCMD | + buffer->bytecount; + olinfo_status = buffer->bytecount << IGC_ADVTXD_PAYLEN_SHIFT; + + desc = IGC_TX_DESC(ring, ring->next_to_use); + desc->read.cmd_type_len = cpu_to_le32(cmd_type); + desc->read.olinfo_status = cpu_to_le32(olinfo_status); + desc->read.buffer_addr = cpu_to_le64(dma_unmap_addr(buffer, dma)); + + netdev_tx_sent_queue(txring_txq(ring), buffer->bytecount); + + buffer->next_to_watch = desc; + + ring->next_to_use++; + if (ring->next_to_use == ring->count) + ring->next_to_use = 0; + + return 0; +} + +struct igc_ring *igc_xdp_get_tx_ring(struct igc_adapter *adapter, int cpu) +{ + int index = cpu; + + if (index >= adapter->num_tx_queues) + index = index % adapter->num_tx_queues; + + return adapter->tx_ring[index]; +} + +static int igc_xdp_xmit_back(struct igc_adapter *adapter, struct xdp_buff *xdp) +{ + struct xdp_frame *xdpf = xdp_convert_buff_to_frame(xdp); + int cpu = smp_processor_id(); + struct netdev_queue *nq; + struct igc_ring *ring; + int res; + + if (unlikely(!xdpf)) + return -EFAULT; + + ring = igc_xdp_get_tx_ring(adapter, cpu); + nq = txring_txq(ring); + + __netif_tx_lock(nq, cpu); + res = igc_xdp_init_tx_descriptor(ring, xdpf); + __netif_tx_unlock(nq); + return res; +} + struct sk_buff *igc_xdp_run_prog(struct igc_adapter *adapter, struct xdp_buff *xdp) { @@ -54,6 +145,12 @@ struct sk_buff *igc_xdp_run_prog(struct igc_adapter *adapter, case XDP_PASS: res = IGC_XDP_PASS; break; + case XDP_TX: + if (igc_xdp_xmit_back(adapter, xdp) < 0) + res = IGC_XDP_CONSUMED; + else + res = IGC_XDP_TX; + break; default: bpf_warn_invalid_xdp_action(act); fallthrough; @@ -74,3 +171,30 @@ bool igc_xdp_is_enabled(struct igc_adapter *adapter) { return !!adapter->xdp_prog; } + +int igc_xdp_register_rxq_info(struct igc_ring *ring) +{ + struct net_device *dev = ring->netdev; + int err; + + err = xdp_rxq_info_reg(&ring->xdp_rxq, dev, ring->queue_index); + if (err) { + netdev_err(dev, "Failed to register xdp rxq info\n"); + return err; + } + + err = xdp_rxq_info_reg_mem_model(&ring->xdp_rxq, MEM_TYPE_PAGE_SHARED, + NULL); + if (err) { + netdev_err(dev, "Failed to register xdp rxq mem model\n"); + xdp_rxq_info_unreg(&ring->xdp_rxq); + return err; + } + + return 0; +} + +void igc_xdp_unregister_rxq_info(struct igc_ring *ring) +{ + xdp_rxq_info_unreg(&ring->xdp_rxq); +} diff --git a/drivers/net/ethernet/intel/igc/igc_xdp.h b/drivers/net/ethernet/intel/igc/igc_xdp.h index ab36ab4d48fd..fdb31f40dbe5 100644 --- a/drivers/net/ethernet/intel/igc/igc_xdp.h +++ b/drivers/net/ethernet/intel/igc/igc_xdp.h @@ -6,6 +6,7 @@ #define IGC_XDP_PASS 0 #define IGC_XDP_CONSUMED BIT(0) +#define IGC_XDP_TX BIT(1) int igc_xdp_set_prog(struct igc_adapter *adapter, struct bpf_prog *prog, struct netlink_ext_ack *extack); @@ -15,4 +16,9 @@ struct sk_buff *igc_xdp_run_prog(struct igc_adapter *adapter, bool igc_xdp_is_enabled(struct igc_adapter *adapter); +int igc_xdp_register_rxq_info(struct igc_ring *ring); +void igc_xdp_unregister_rxq_info(struct igc_ring *ring); + +struct igc_ring *igc_xdp_get_tx_ring(struct igc_adapter *adapter, int cpu); + #endif /* _IGC_XDP_H_ */ From patchwork Wed Oct 28 20:19:42 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andre Guedes X-Patchwork-Id: 1389402 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=osuosl.org (client-ip=140.211.166.136; helo=silver.osuosl.org; envelope-from=intel-wired-lan-bounces@osuosl.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=intel.com Received: from silver.osuosl.org (smtp3.osuosl.org [140.211.166.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4CM0LN5LNFz9sVk for ; Thu, 29 Oct 2020 07:20:12 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by silver.osuosl.org (Postfix) with ESMTP id 0C4A920373; Wed, 28 Oct 2020 20:20:11 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from silver.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id dhPb1c3e6P1K; Wed, 28 Oct 2020 20:20:03 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by silver.osuosl.org (Postfix) with ESMTP id A391A214D2; Wed, 28 Oct 2020 20:20:03 +0000 (UTC) X-Original-To: intel-wired-lan@lists.osuosl.org Delivered-To: intel-wired-lan@lists.osuosl.org Received: from silver.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by ash.osuosl.org (Postfix) with ESMTP id 771BD1BF836 for ; Wed, 28 Oct 2020 20:20:01 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by silver.osuosl.org (Postfix) with ESMTP id 6786320797 for ; Wed, 28 Oct 2020 20:20:01 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from silver.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id FLrMWn3kz8ah for ; Wed, 28 Oct 2020 20:19:57 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by silver.osuosl.org (Postfix) with ESMTPS id A5230204F2 for ; Wed, 28 Oct 2020 20:19:57 +0000 (UTC) IronPort-SDR: b+hpcWSQlF8Pnj0yxJZ5QDLVNUT0aRZpgVu7wFceHstSmCQuGFKDxPaoQ7aMlASQtsDD3G+tKG yBi6iL92rXlw== X-IronPort-AV: E=McAfee;i="6000,8403,9788"; a="147614198" X-IronPort-AV: E=Sophos;i="5.77,427,1596524400"; d="scan'208";a="147614198" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Oct 2020 13:19:56 -0700 IronPort-SDR: 0kJCclF40wG2VEwyGrzIC/SMs7f2VEAQuFFUROdmpCBXs5rT1XsCr6JHFEQOhzH51od7te9zmg dfo44XXy140Q== X-IronPort-AV: E=Sophos;i="5.77,427,1596524400"; d="scan'208";a="536374760" Received: from zcolon-mobl.amr.corp.intel.com ([10.209.108.9]) by orsmga005-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Oct 2020 13:19:56 -0700 From: Andre Guedes To: intel-wired-lan@lists.osuosl.org Date: Wed, 28 Oct 2020 13:19:42 -0700 Message-Id: <20201028201943.93147-10-andre.guedes@intel.com> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20201028201943.93147-1-andre.guedes@intel.com> References: <20201028201943.93147-1-andre.guedes@intel.com> MIME-Version: 1.0 Subject: [Intel-wired-lan] [PATCH v2 09/10] igc: Add support for XDP_REDIRECT action X-BeenThere: intel-wired-lan@osuosl.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel Wired Ethernet Linux Kernel Driver Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: intel-wired-lan-bounces@osuosl.org Sender: "Intel-wired-lan" This patch adds support for the XDP_REDIRECT action which enables XDP programs to redirect packets arriving at I225 NIC. The patch also tweaks the driver's page counting scheme (as described in '8ce29c679a6e i40e: tweak page counting for XDP_REDIRECT' and implemented by other Intel drivers) in order to properly support XDP_REDIRECT action. This patch has been tested with the sample apps "xdp_redirect_cpu" and "xdp_redirect_map" located in samples/bpf/. Signed-off-by: Andre Guedes --- drivers/net/ethernet/intel/igc/igc_main.c | 11 ++++++++--- drivers/net/ethernet/intel/igc/igc_xdp.c | 6 ++++++ drivers/net/ethernet/intel/igc/igc_xdp.h | 1 + 3 files changed, 15 insertions(+), 3 deletions(-) diff --git a/drivers/net/ethernet/intel/igc/igc_main.c b/drivers/net/ethernet/intel/igc/igc_main.c index 7bcfc72b00fc..f9c7bbc1347e 100644 --- a/drivers/net/ethernet/intel/igc/igc_main.c +++ b/drivers/net/ethernet/intel/igc/igc_main.c @@ -1710,8 +1710,8 @@ static bool igc_can_reuse_rx_page(struct igc_rx_buffer *rx_buffer) * the pagecnt_bias and page count so that we fully restock the * number of references the driver holds. */ - if (unlikely(!pagecnt_bias)) { - page_ref_add(page, USHRT_MAX); + if (unlikely(pagecnt_bias == 1)) { + page_ref_add(page, USHRT_MAX - 1); rx_buffer->pagecnt_bias = USHRT_MAX; } @@ -1852,7 +1852,8 @@ static bool igc_alloc_mapped_page(struct igc_ring *rx_ring, bi->dma = dma; bi->page = page; bi->page_offset = igc_rx_offset(rx_ring); - bi->pagecnt_bias = 1; + page_ref_add(page, USHRT_MAX - 1); + bi->pagecnt_bias = USHRT_MAX; return true; } @@ -1952,6 +1953,9 @@ static void igc_finalize_xdp(struct igc_adapter *adapter, int status) igc_flush_tx_descriptors(ring); __netif_tx_unlock(nq); } + + if (status & IGC_XDP_REDIRECT) + xdp_do_flush(); } static int igc_clean_rx_irq(struct igc_q_vector *q_vector, const int budget) @@ -2020,6 +2024,7 @@ static int igc_clean_rx_irq(struct igc_q_vector *q_vector, const int budget) rx_buffer->pagecnt_bias++; break; case IGC_XDP_TX: + case IGC_XDP_REDIRECT: igc_rx_buffer_flip(rx_buffer, truesize); xdp_status |= xdp_res; break; diff --git a/drivers/net/ethernet/intel/igc/igc_xdp.c b/drivers/net/ethernet/intel/igc/igc_xdp.c index 17852cc3a438..39b8310dfba1 100644 --- a/drivers/net/ethernet/intel/igc/igc_xdp.c +++ b/drivers/net/ethernet/intel/igc/igc_xdp.c @@ -151,6 +151,12 @@ struct sk_buff *igc_xdp_run_prog(struct igc_adapter *adapter, else res = IGC_XDP_TX; break; + case XDP_REDIRECT: + if (xdp_do_redirect(adapter->netdev, xdp, prog) < 0) + res = IGC_XDP_CONSUMED; + else + res = IGC_XDP_REDIRECT; + break; default: bpf_warn_invalid_xdp_action(act); fallthrough; diff --git a/drivers/net/ethernet/intel/igc/igc_xdp.h b/drivers/net/ethernet/intel/igc/igc_xdp.h index fdb31f40dbe5..1c38a80c3aa0 100644 --- a/drivers/net/ethernet/intel/igc/igc_xdp.h +++ b/drivers/net/ethernet/intel/igc/igc_xdp.h @@ -7,6 +7,7 @@ #define IGC_XDP_PASS 0 #define IGC_XDP_CONSUMED BIT(0) #define IGC_XDP_TX BIT(1) +#define IGC_XDP_REDIRECT BIT(2) int igc_xdp_set_prog(struct igc_adapter *adapter, struct bpf_prog *prog, struct netlink_ext_ack *extack); From patchwork Wed Oct 28 20:19:43 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andre Guedes X-Patchwork-Id: 1389397 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=osuosl.org (client-ip=140.211.166.133; helo=hemlock.osuosl.org; envelope-from=intel-wired-lan-bounces@osuosl.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=intel.com Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4CM0LG0kQLz9sVh for ; Thu, 29 Oct 2020 07:20:06 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id 7B34C874B2; Wed, 28 Oct 2020 20:20:04 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id AWY8UbcnrbQT; Wed, 28 Oct 2020 20:20:03 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by hemlock.osuosl.org (Postfix) with ESMTP id 42B32873CE; Wed, 28 Oct 2020 20:20:03 +0000 (UTC) X-Original-To: intel-wired-lan@lists.osuosl.org Delivered-To: intel-wired-lan@lists.osuosl.org Received: from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by ash.osuosl.org (Postfix) with ESMTP id C2C4C1BF836 for ; Wed, 28 Oct 2020 20:20:00 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by whitealder.osuosl.org (Postfix) with ESMTP id BFABB869F1 for ; Wed, 28 Oct 2020 20:20:00 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from whitealder.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id d0bm7qt4+7dK for ; Wed, 28 Oct 2020 20:19:59 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mga17.intel.com (mga17.intel.com [192.55.52.151]) by whitealder.osuosl.org (Postfix) with ESMTPS id D7AF0869C4 for ; Wed, 28 Oct 2020 20:19:59 +0000 (UTC) IronPort-SDR: IdpHN8PUIlKxYSDVcKqo1MFPwLooKsTtsztxjKr9kpFe+aMja3B4NDUtbbmDVWvhBLcdcQAIU/ dlQMtNUuOc2g== X-IronPort-AV: E=McAfee;i="6000,8403,9788"; a="148187214" X-IronPort-AV: E=Sophos;i="5.77,427,1596524400"; d="scan'208";a="148187214" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Oct 2020 13:19:56 -0700 IronPort-SDR: pxbIMzuTs7R5Yjly3UKZML98KBkMT1ou8iuTfoEG2gbDQ9OU+vLBhONpLjle6v5q20cUcI7tEF NRY6hGL1wZOg== X-IronPort-AV: E=Sophos;i="5.77,427,1596524400"; d="scan'208";a="536374763" Received: from zcolon-mobl.amr.corp.intel.com ([10.209.108.9]) by orsmga005-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Oct 2020 13:19:56 -0700 From: Andre Guedes To: intel-wired-lan@lists.osuosl.org Date: Wed, 28 Oct 2020 13:19:43 -0700 Message-Id: <20201028201943.93147-11-andre.guedes@intel.com> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20201028201943.93147-1-andre.guedes@intel.com> References: <20201028201943.93147-1-andre.guedes@intel.com> MIME-Version: 1.0 Subject: [Intel-wired-lan] [PATCH v2 10/10] igc: Implement ndo_xdp_xmit callback X-BeenThere: intel-wired-lan@osuosl.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel Wired Ethernet Linux Kernel Driver Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: intel-wired-lan-bounces@osuosl.org Sender: "Intel-wired-lan" This patch introduces the helper igc_xdp_xmit() which implements the ndo_xdp_xmit ops, enabling the igc driver to transmit packets forwarded to it by xdp programs running on other interfaces. This patch has been tested with the sample app "xdp_redirect_map" located in samples/bpf/. Signed-off-by: Andre Guedes --- drivers/net/ethernet/intel/igc/igc_main.c | 41 +++++++++++++++++++++++ drivers/net/ethernet/intel/igc/igc_xdp.c | 3 +- drivers/net/ethernet/intel/igc/igc_xdp.h | 2 ++ 3 files changed, 44 insertions(+), 2 deletions(-) diff --git a/drivers/net/ethernet/intel/igc/igc_main.c b/drivers/net/ethernet/intel/igc/igc_main.c index f9c7bbc1347e..4d821f04e3de 100644 --- a/drivers/net/ethernet/intel/igc/igc_main.c +++ b/drivers/net/ethernet/intel/igc/igc_main.c @@ -4990,6 +4990,46 @@ static int igc_bpf(struct net_device *dev, struct netdev_bpf *bpf) } } +static int igc_xdp_xmit(struct net_device *dev, int num_frames, + struct xdp_frame **frames, u32 flags) +{ + struct igc_adapter *adapter = netdev_priv(dev); + int cpu = smp_processor_id(); + struct netdev_queue *nq; + struct igc_ring *ring; + int i, drops; + + if (unlikely(test_bit(__IGC_DOWN, &adapter->state))) + return -ENETDOWN; + + if (unlikely(flags & ~XDP_XMIT_FLAGS_MASK)) + return -EINVAL; + + ring = igc_xdp_get_tx_ring(adapter, cpu); + nq = txring_txq(ring); + + __netif_tx_lock(nq, cpu); + + drops = 0; + for (i = 0; i < num_frames; i++) { + int err; + struct xdp_frame *xdpf = frames[i]; + + err = igc_xdp_init_tx_descriptor(ring, xdpf); + if (err) { + xdp_return_frame_rx_napi(xdpf); + drops++; + } + } + + if (flags & XDP_XMIT_FLUSH) + igc_flush_tx_descriptors(ring); + + __netif_tx_unlock(nq); + + return num_frames - drops; +} + static const struct net_device_ops igc_netdev_ops = { .ndo_open = igc_open, .ndo_stop = igc_close, @@ -5004,6 +5044,7 @@ static const struct net_device_ops igc_netdev_ops = { .ndo_do_ioctl = igc_ioctl, .ndo_setup_tc = igc_setup_tc, .ndo_bpf = igc_bpf, + .ndo_xdp_xmit = igc_xdp_xmit, }; /* PCIe configuration access */ diff --git a/drivers/net/ethernet/intel/igc/igc_xdp.c b/drivers/net/ethernet/intel/igc/igc_xdp.c index 39b8310dfba1..2fe6fac38d0b 100644 --- a/drivers/net/ethernet/intel/igc/igc_xdp.c +++ b/drivers/net/ethernet/intel/igc/igc_xdp.c @@ -58,8 +58,7 @@ static int igc_xdp_init_tx_buffer(struct igc_tx_buffer *buffer, } /* This function requires __netif_tx_lock is held by the caller. */ -static int igc_xdp_init_tx_descriptor(struct igc_ring *ring, - struct xdp_frame *xdpf) +int igc_xdp_init_tx_descriptor(struct igc_ring *ring, struct xdp_frame *xdpf) { struct igc_tx_buffer *buffer; union igc_adv_tx_desc *desc; diff --git a/drivers/net/ethernet/intel/igc/igc_xdp.h b/drivers/net/ethernet/intel/igc/igc_xdp.h index 1c38a80c3aa0..801fb27fbb85 100644 --- a/drivers/net/ethernet/intel/igc/igc_xdp.h +++ b/drivers/net/ethernet/intel/igc/igc_xdp.h @@ -22,4 +22,6 @@ void igc_xdp_unregister_rxq_info(struct igc_ring *ring); struct igc_ring *igc_xdp_get_tx_ring(struct igc_adapter *adapter, int cpu); +int igc_xdp_init_tx_descriptor(struct igc_ring *ring, struct xdp_frame *xdpf); + #endif /* _IGC_XDP_H_ */