From patchwork Tue Jan 17 16:37:13 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander H Duyck X-Patchwork-Id: 716300 X-Patchwork-Delegate: jeffrey.t.kirsher@intel.com Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from fraxinus.osuosl.org (smtp4.osuosl.org [140.211.166.137]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3v2wk828D1z9ryk for ; Wed, 18 Jan 2017 03:37:20 +1100 (AEDT) Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="CEmnWFU1"; dkim-atps=neutral Received: from localhost (localhost [127.0.0.1]) by fraxinus.osuosl.org (Postfix) with ESMTP id BEFD088B0C; Tue, 17 Jan 2017 16:37:18 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from fraxinus.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id GQ6gXMbMFgqB; Tue, 17 Jan 2017 16:37:16 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by fraxinus.osuosl.org (Postfix) with ESMTP id CA3E288A34; Tue, 17 Jan 2017 16:37:16 +0000 (UTC) X-Original-To: intel-wired-lan@lists.osuosl.org Delivered-To: intel-wired-lan@lists.osuosl.org Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by ash.osuosl.org (Postfix) with ESMTP id 169BF1C0306 for ; Tue, 17 Jan 2017 16:37:16 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id 11F8E8A8A3 for ; Tue, 17 Jan 2017 16:37:16 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id UpMyH4xAHjHj for ; Tue, 17 Jan 2017 16:37:15 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mail-pg0-f67.google.com (mail-pg0-f67.google.com [74.125.83.67]) by hemlock.osuosl.org (Postfix) with ESMTPS id 28BFB87E06 for ; Tue, 17 Jan 2017 16:37:15 +0000 (UTC) Received: by mail-pg0-f67.google.com with SMTP id 204so8406248pge.2 for ; Tue, 17 Jan 2017 08:37:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=subject:from:to:date:message-id:in-reply-to:references:user-agent :mime-version:content-transfer-encoding; bh=bM2lEfamsxuVR5CLC9OYQ72+SMrfnf4BJfgDQQf07aE=; b=CEmnWFU1tjtNYHovuWPVZEWBTkSSiGtGGexadPNXRckk5ITsNoqSaNBnGUDxFIzpJ1 N1jsS83+eDxSra67CUCB6PISVZPKcFWzncZxfxaSDt7+3OCKWHEaBROO2+zTG/6N7GDL O09jfeWTy3usVeNFXc2c+jQ5W7+0cOO3GCkHaPfxE8/7Ve+WIIUWWEz6pHeBmJsRtTnU +w8k+8XCZQ9A0a1hkUijkDY2bGlOwmIrgrq5NoN7ltyBOOqUH0Bwv8qKESUlL1ftx84Y GZfaq9rotCkE5lwIeLA/bjSeOIK51cx0FCe8lQMeIt6Mc+NDY5TVhVR3CnLreQT6TpcQ 5dBg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:from:to:date:message-id:in-reply-to :references:user-agent:mime-version:content-transfer-encoding; bh=bM2lEfamsxuVR5CLC9OYQ72+SMrfnf4BJfgDQQf07aE=; b=q3IsX7ln3LnX44fzAPMlT7aomvbeSFFSW9epsM5kCsTRmgxOCZ99U8ttyhwW0L5Y6u P62lMLL232SkJM0ItaHs4XKFuiokyz0NDR8EasCUkt+NvYLQSckAum89gduJsqo9cViq CVsW4Ia9edEtze4LrMlNjAf9WJwHron54MW1EJswuMqgneGUjX13MCe+a89dwWExjDCc ji4AZlDtNgJNPkHZg5GrWh6OrkGVobJ2MLtdH/WZi9Ecc5Xt1UYmLxczSyBsziaDBE4c RsgxoVbQTT/YRBBNYnN7upcLhAsxHxBObwOatpVKkvcfbA4LfIAhFK1D2DEp1tP/aLWa 2c9Q== X-Gm-Message-State: AIkVDXJ+z6abDcSfM0l0f5d1IEKhFkKul83e5tlg8DoDrvo5o0GOngJlPJiYqreI1kj4Kw== X-Received: by 10.98.13.203 with SMTP id 72mr25896316pfn.64.1484671034782; Tue, 17 Jan 2017 08:37:14 -0800 (PST) Received: from localhost.localdomain ([2001:470:b:9c3:9e5c:8eff:fe4f:f2d0]) by smtp.gmail.com with ESMTPSA id 18sm57386814pgf.28.2017.01.17.08.37.14 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 17 Jan 2017 08:37:14 -0800 (PST) From: Alexander Duyck To: intel-wired-lan@lists.osuosl.org, jeffrey.t.kirsher@intel.com Date: Tue, 17 Jan 2017 08:37:13 -0800 Message-ID: <20170117163709.5423.61819.stgit@localhost.localdomain> In-Reply-To: <20170117163401.5423.37993.stgit@localhost.localdomain> References: <20170117163401.5423.37993.stgit@localhost.localdomain> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 Subject: [Intel-wired-lan] [next PATCH v2 10/11] ixgbe: Add support for build_skb X-BeenThere: intel-wired-lan@lists.osuosl.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Intel Wired Ethernet Linux Kernel Driver Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: intel-wired-lan-bounces@lists.osuosl.org Sender: "Intel-wired-lan" From: Alexander Duyck This patch adds build_skb support to the Rx path. There are several advantages to this change. 1. It avoids the memcpy and skb->head allocation for small packets which improves performance by about 5% in my tests. 2. It avoids the memcpy, skb->head allocation, and eth_get_headlen for larger packets improving performance by about 10% in my tests. 3. For VXLAN packets it allows the full header to be in skb->data which improves the performance by as much as 30% in some of my tests. Signed-off-by: Alexander Duyck Tested-by: Andrew Bowers --- drivers/net/ethernet/intel/ixgbe/ixgbe_main.c | 49 ++++++++++++++++++++++++- 1 file changed, 48 insertions(+), 1 deletion(-) diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c index 8529eafb9717..dd04b76ae0bd 100644 --- a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c @@ -1913,7 +1913,7 @@ static bool ixgbe_cleanup_headers(struct ixgbe_ring *rx_ring, } /* place header in linear portion of buffer */ - if (skb_is_nonlinear(skb)) + if (!skb_headlen(skb)) ixgbe_pull_tail(rx_ring, skb); #ifdef IXGBE_FCOE @@ -2142,6 +2142,49 @@ static struct sk_buff *ixgbe_construct_skb(struct ixgbe_ring *rx_ring, return skb; } +static struct sk_buff *ixgbe_build_skb(struct ixgbe_ring *rx_ring, + struct ixgbe_rx_buffer *rx_buffer, + union ixgbe_adv_rx_desc *rx_desc, + unsigned int size) +{ + void *va = page_address(rx_buffer->page) + rx_buffer->page_offset; +#if (PAGE_SIZE < 8192) + unsigned int truesize = ixgbe_rx_pg_size(rx_ring) / 2; +#else + unsigned int truesize = SKB_DATA_ALIGN(sizeof(struct skb_shared_info)) + + SKB_DATA_ALIGN(IXGBE_SKB_PAD + size); +#endif + struct sk_buff *skb; + + /* prefetch first cache line of first page */ + prefetch(va); +#if L1_CACHE_BYTES < 128 + prefetch(va + L1_CACHE_BYTES); +#endif + + /* build an skb to around the page buffer */ + skb = build_skb(va - IXGBE_SKB_PAD, truesize); + if (unlikely(!skb)) + return NULL; + + /* update pointers within the skb to store the data */ + skb_reserve(skb, IXGBE_SKB_PAD); + __skb_put(skb, size); + + /* record DMA address if this is the start of a chain of buffers */ + if (!ixgbe_test_staterr(rx_desc, IXGBE_RXD_STAT_EOP)) + IXGBE_CB(skb)->dma = rx_buffer->dma; + + /* update buffer offset */ +#if (PAGE_SIZE < 8192) + rx_buffer->page_offset ^= truesize; +#else + rx_buffer->page_offset += truesize; +#endif + + return skb; +} + /** * ixgbe_clean_rx_irq - Clean completed descriptors from Rx ring - bounce buf * @q_vector: structure containing interrupt and ring information @@ -2195,6 +2238,9 @@ static int ixgbe_clean_rx_irq(struct ixgbe_q_vector *q_vector, /* retrieve a buffer from the ring */ if (skb) ixgbe_add_rx_frag(rx_ring, rx_buffer, skb, size); + else if (ring_uses_build_skb(rx_ring)) + skb = ixgbe_build_skb(rx_ring, rx_buffer, + rx_desc, size); else skb = ixgbe_construct_skb(rx_ring, rx_buffer, rx_desc, size); @@ -3970,6 +4016,7 @@ static void ixgbe_set_rx_buffer_len(struct ixgbe_adapter *adapter) if (test_bit(__IXGBE_RX_FCOE, &rx_ring->state)) set_bit(__IXGBE_RX_3K_BUFFER, &rx_ring->state); + clear_bit(__IXGBE_RX_BUILD_SKB_ENABLED, &rx_ring->state); if (adapter->flags2 & IXGBE_FLAG2_RX_LEGACY) continue;