From patchwork Sat Feb 25 17:32:29 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Fastabend X-Patchwork-Id: 732421 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from fraxinus.osuosl.org (smtp4.osuosl.org [140.211.166.137]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3vVw6M06VTz9s7M for ; Sun, 26 Feb 2017 04:32:58 +1100 (AEDT) Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="jQwkuPud"; dkim-atps=neutral Received: from localhost (localhost [127.0.0.1]) by fraxinus.osuosl.org (Postfix) with ESMTP id 4EFDC868EE; Sat, 25 Feb 2017 17:32:57 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from fraxinus.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id CIF316oxpig5; Sat, 25 Feb 2017 17:32:55 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by fraxinus.osuosl.org (Postfix) with ESMTP id BDAFA868B6; Sat, 25 Feb 2017 17:32:55 +0000 (UTC) X-Original-To: intel-wired-lan@lists.osuosl.org Delivered-To: intel-wired-lan@lists.osuosl.org Received: from fraxinus.osuosl.org (smtp4.osuosl.org [140.211.166.137]) by ash.osuosl.org (Postfix) with ESMTP id 903EF1BFCC3 for ; Sat, 25 Feb 2017 17:32:54 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by fraxinus.osuosl.org (Postfix) with ESMTP id 8931B868B6 for ; Sat, 25 Feb 2017 17:32:54 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from fraxinus.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id sX7F8IwyZwd3 for ; Sat, 25 Feb 2017 17:32:53 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mail-pg0-f65.google.com (mail-pg0-f65.google.com [74.125.83.65]) by fraxinus.osuosl.org (Postfix) with ESMTPS id ACAEE868A8 for ; Sat, 25 Feb 2017 17:32:53 +0000 (UTC) Received: by mail-pg0-f65.google.com with SMTP id 5so7252995pgj.0 for ; Sat, 25 Feb 2017 09:32:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:subject:to:cc:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=XfgutZLdGw9ZA7LsGzZ3oENJdoFRPq49fAnNUlTbDR4=; b=jQwkuPudECe4ksUR3YRyBn+dHttBFbdhvg9gmWXdhcKgIhn7abUnNZVuQe7EBxu+xh hFvRP1BtNNAzgqXDWoWJ06wBXzzGpZQHQ2Fp2cmClvAYN3/eO3rTfMAHjDamirSSzK6I QjLJ3lV+zKTt1K5f9H1HzW7kTbndmHvk0pDLePtHTkxef3nPSSSpn8UKrIDpV5OKvRg0 kxwEXKitYqHuqxUfD/nTm79uImP/r6oLR8TIwxSj2vBpxBu2DEPZmj0ll36f/LPXAgDM +gZFAlfe5pzJ1EvoCk99fC4VS1C9c1U3dzDJU83+WBupEleu35uk5iNkvbVnj9lIxyAc LoBg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:subject:to:cc:date:message-id:in-reply-to :references:user-agent:mime-version:content-transfer-encoding; bh=XfgutZLdGw9ZA7LsGzZ3oENJdoFRPq49fAnNUlTbDR4=; b=Ch/WeuWG8GDTUE28sYJhMl5gcevjYpMLv8YQFDcvvOwo6R97K3/Z7xJGCnwAfOA5ns o/BBVEfSHzX0nkpm3WdEsenYX+Rdx7Jl0na31oRUieb+M+Zea9bqh5eSXE5+xsKQgmKz fAds3o8jc+bntr/blPy9yilMqny8m/w4qKKepUHz5RTijefbuMw9kL1ZnrcOH8wyfG2h CwBsYyAEahHvckvV2gCYsD9qoqSrLIzrtyx64SS/SShCiCh5pooJJLRmPo5JKDR8reFE PwLy3fJzD2ovowxYgThr4zy1uzYs+MjQwl8+BLRb2HIXy1R6ND2dKy/Np97wiMLoB4eU tTdA== X-Gm-Message-State: AMke39nxZCGHDevOGrnSakj2PTD3kPZHB+/LcA/PqhJsKy3Z20SOTpon0H+0Rua3lrd5AA== X-Received: by 10.84.136.75 with SMTP id 69mr12738898plk.172.1488043973269; Sat, 25 Feb 2017 09:32:53 -0800 (PST) Received: from [127.0.1.1] ([72.168.144.251]) by smtp.gmail.com with ESMTPSA id s8sm21659759pfj.30.2017.02.25.09.32.41 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 25 Feb 2017 09:32:52 -0800 (PST) From: John Fastabend X-Google-Original-From: John Fastabend To: alexander.h.duyck@intel.com, daniel@iogearbox.net, alexander.duyck@gmail.com, john.r.fastabend@intel.com, bjorn.topel@intel.com, alexei.starovoitov@gmail.com, john.fastabend@intel.com Date: Sat, 25 Feb 2017 09:32:29 -0800 Message-ID: <20170225173229.32741.12321.stgit@john-Precision-Tower-5810> In-Reply-To: <20170225172422.32741.67877.stgit@john-Precision-Tower-5810> References: <20170225172422.32741.67877.stgit@john-Precision-Tower-5810> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 Cc: intel-wired-lan@lists.osuosl.org, magnus.karlsson@intel.com Subject: [Intel-wired-lan] [net-next PATCH 1/3] ixgbe: add XDP support for pass and drop actions X-BeenThere: intel-wired-lan@lists.osuosl.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Intel Wired Ethernet Linux Kernel Driver Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: intel-wired-lan-bounces@lists.osuosl.org Sender: "Intel-wired-lan" Basic XDP drop support for ixgbe. Uses READ_ONCE/xchg semantics on XDP programs instead of rcu primitives as suggested by Daniel Borkmann and Alex Duyck. Signed-off-by: John Fastabend --- drivers/net/ethernet/intel/ixgbe/ixgbe.h | 2 drivers/net/ethernet/intel/ixgbe/ixgbe_main.c | 108 +++++++++++++++++++++++++ 2 files changed, 108 insertions(+), 2 deletions(-) diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe.h b/drivers/net/ethernet/intel/ixgbe/ixgbe.h index b812913..2d12c24 100644 --- a/drivers/net/ethernet/intel/ixgbe/ixgbe.h +++ b/drivers/net/ethernet/intel/ixgbe/ixgbe.h @@ -273,6 +273,7 @@ struct ixgbe_ring { struct ixgbe_ring *next; /* pointer to next ring in q_vector */ struct ixgbe_q_vector *q_vector; /* backpointer to host q_vector */ struct net_device *netdev; /* netdev ring belongs to */ + struct bpf_prog *xdp_prog; struct device *dev; /* device for DMA mapping */ struct ixgbe_fwd_adapter *l2_accel_priv; void *desc; /* descriptor ring memory */ @@ -510,6 +511,7 @@ struct ixgbe_adapter { unsigned long active_vlans[BITS_TO_LONGS(VLAN_N_VID)]; /* OS defined structs */ struct net_device *netdev; + struct bpf_prog *xdp_prog; struct pci_dev *pdev; unsigned long state; diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c index e3da397..ec2c38f 100644 --- a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c @@ -49,6 +49,9 @@ #include #include #include +#include +#include +#include #include #include #include @@ -2051,7 +2054,7 @@ static void ixgbe_put_rx_buffer(struct ixgbe_ring *rx_ring, /* hand second half of page back to the ring */ ixgbe_reuse_rx_page(rx_ring, rx_buffer); } else { - if (IXGBE_CB(skb)->dma == rx_buffer->dma) { + if (skb && IXGBE_CB(skb)->dma == rx_buffer->dma) { /* the page has been released from the ring */ IXGBE_CB(skb)->page_released = true; } else { @@ -2157,6 +2160,42 @@ static struct sk_buff *ixgbe_build_skb(struct ixgbe_ring *rx_ring, return skb; } +static int ixgbe_run_xdp(struct ixgbe_ring *rx_ring, + struct ixgbe_rx_buffer *rx_buffer, + unsigned int size) +{ + struct bpf_prog *xdp_prog; + struct xdp_buff xdp; + void *addr; + u32 act; + + xdp_prog = READ_ONCE(rx_ring->xdp_prog); + + if (!xdp_prog) + return 0; + + addr = page_address(rx_buffer->page) + rx_buffer->page_offset; + xdp.data_hard_start = addr; + xdp.data = addr; + xdp.data_end = addr + size; + + act = bpf_prog_run_xdp(xdp_prog, &xdp); + switch (act) { + case XDP_PASS: + return 0; + default: + bpf_warn_invalid_xdp_action(act); + case XDP_TX: + case XDP_ABORTED: + trace_xdp_exception(rx_ring->netdev, xdp_prog, act); + /* fallthrough -- handle aborts by dropping packet */ + case XDP_DROP: + rx_buffer->pagecnt_bias++; /* give page back */ + break; + } + return size; +} + /** * ixgbe_clean_rx_irq - Clean completed descriptors from Rx ring - bounce buf * @q_vector: structure containing interrupt and ring information @@ -2187,6 +2226,7 @@ static int ixgbe_clean_rx_irq(struct ixgbe_q_vector *q_vector, struct ixgbe_rx_buffer *rx_buffer; struct sk_buff *skb; unsigned int size; + int consumed; /* return some buffers to hardware, one at a time is too slow */ if (cleaned_count >= IXGBE_RX_BUFFER_WRITE) { @@ -2207,6 +2247,19 @@ static int ixgbe_clean_rx_irq(struct ixgbe_q_vector *q_vector, rx_buffer = ixgbe_get_rx_buffer(rx_ring, rx_desc, &skb, size); + rcu_read_lock(); + consumed = ixgbe_run_xdp(rx_ring, rx_buffer, size); + rcu_read_unlock(); + + if (consumed) { + ixgbe_put_rx_buffer(rx_ring, rx_buffer, skb); + cleaned_count++; + ixgbe_is_non_eop(rx_ring, rx_desc, skb); + total_rx_packets++; + total_rx_bytes += size; + continue; + } + /* retrieve a buffer from the ring */ if (skb) ixgbe_add_rx_frag(rx_ring, rx_buffer, skb, size); @@ -6106,6 +6159,12 @@ int ixgbe_setup_rx_resources(struct ixgbe_ring *rx_ring) return -ENOMEM; } +static void ixgbe_setup_xdp_resource(struct ixgbe_adapter *adapter, + struct ixgbe_ring *ring) +{ + xchg(&ring->xdp_prog, adapter->xdp_prog); +} + /** * ixgbe_setup_all_rx_resources - allocate all queues Rx resources * @adapter: board private structure @@ -6122,8 +6181,10 @@ static int ixgbe_setup_all_rx_resources(struct ixgbe_adapter *adapter) for (i = 0; i < adapter->num_rx_queues; i++) { err = ixgbe_setup_rx_resources(adapter->rx_ring[i]); - if (!err) + if (!err) { + ixgbe_setup_xdp_resource(adapter, adapter->rx_ring[i]); continue; + } e_err(probe, "Allocation for Rx Queue %u failed\n", i); goto err_setup_rx; @@ -9455,6 +9516,48 @@ static void ixgbe_fwd_del(struct net_device *pdev, void *priv) return features; } +static int ixgbe_xdp_setup(struct net_device *dev, struct bpf_prog *prog) +{ + int i, frame_size = dev->mtu + ETH_HLEN + ETH_FCS_LEN + VLAN_HLEN; + struct ixgbe_adapter *adapter = netdev_priv(dev); + struct bpf_prog *old_adapter_prog; + + /* verify ixgbe ring attributes are sufficient for XDP */ + for (i = 0; i < adapter->num_rx_queues; i++) { + struct ixgbe_ring *ring = adapter->rx_ring[i]; + + if (ring_is_rsc_enabled(ring)) + return -EINVAL; + + if (frame_size > ixgbe_rx_bufsz(ring)) + return -EINVAL; + } + + old_adapter_prog = xchg(&adapter->xdp_prog, prog); + for (i = 0; i < adapter->num_rx_queues; i++) + ixgbe_setup_xdp_resource(adapter, adapter->rx_ring[i]); + + if (old_adapter_prog) + bpf_prog_put(old_adapter_prog); + + return 0; +} + +static int ixgbe_xdp(struct net_device *dev, struct netdev_xdp *xdp) +{ + struct ixgbe_adapter *adapter = netdev_priv(dev); + + switch (xdp->command) { + case XDP_SETUP_PROG: + return ixgbe_xdp_setup(dev, xdp->prog); + case XDP_QUERY_PROG: + xdp->prog_attached = !!(adapter->rx_ring[0]->xdp_prog); + return 0; + default: + return -EINVAL; + } +} + static const struct net_device_ops ixgbe_netdev_ops = { .ndo_open = ixgbe_open, .ndo_stop = ixgbe_close, @@ -9500,6 +9603,7 @@ static void ixgbe_fwd_del(struct net_device *pdev, void *priv) .ndo_udp_tunnel_add = ixgbe_add_udp_tunnel_port, .ndo_udp_tunnel_del = ixgbe_del_udp_tunnel_port, .ndo_features_check = ixgbe_features_check, + .ndo_xdp = ixgbe_xdp, }; /**