From patchwork Sun Jul 22 15:13:04 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Toshiaki Makita X-Patchwork-Id: 947477 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="AZdXtBPm"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 41YSn82MBjz9s3N for ; Mon, 23 Jul 2018 01:13:32 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729837AbeGVQK3 (ORCPT ); Sun, 22 Jul 2018 12:10:29 -0400 Received: from mail-pg1-f193.google.com ([209.85.215.193]:39538 "EHLO mail-pg1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728394AbeGVQK2 (ORCPT ); Sun, 22 Jul 2018 12:10:28 -0400 Received: by mail-pg1-f193.google.com with SMTP id g2-v6so10423887pgs.6 for ; Sun, 22 Jul 2018 08:13:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=HVYT0cjsAEwe3aB9bFEs36Am7aGr36kf3dkF2f1syuw=; b=AZdXtBPmyeY3KnAlryfAYh8Ie22xBo3EekXSxFm89cPDS7xxddswSSjv9urJOUx+tC VzN8kA0wEn/p7V8oxOBW/6CZoBY5s9q5tWQjSv3aUPWwGHuB44Gw6GiuAKXL15pxPFof t7CPz7hblBzA/1vx9zaK2g/RCwUBWR/RhuPojeh5izKe8OMXWV1ECR5YjjcVhm57wK0e ve6/Ud4Hik2ymak4bNdnzI5Q2DH8uDDPxA21BjlbxYL2HCWPlRZIbzSko4BBuIF2NZqQ 6NJClbgOPENomeInr7Euvwp3FUEKKAFvdahtlcj3y7z3Pjpm659ugmANku3zeJJMFtBL kYbA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=HVYT0cjsAEwe3aB9bFEs36Am7aGr36kf3dkF2f1syuw=; b=hqmBctF4fw+IHH3Ns3MvGjrP2Ti5menDiiAB+Cs+SnPPWUjqqfnFHJ4Y1TWADtc93l vSKxsPAdM2vhXyOu+S5aPw0Cbm9+YOfY8AyEGKvoYTJZep5IPil8v+6t5wBZqZeTl0Vq OZgL70HUctJLeiZscOLuy3U16R9JxgWVfIz/ZPyGTZYCxBeLnGR0VsR9PuoUy0Q61sL9 KP/mWhKO0dr6AThY2nX6JLfuaUstjJBWwaLe+H1vH2turm63olwo3Z1nlJ/IHM3OLm4L Km1HOWbDgYreIVAjoaHuTTPjqlk7uiV8wQP+Sug0Q6MpKPtM8l+9uvino1+d/keubpGX 1Yew== X-Gm-Message-State: AOUpUlEXt7B3FTzwyowLFSxcuCWzS5e4P9LIEPsev0e6KNQ/YqVsTZys ZT8adZt/kMNxu8DvXao9sm5a26uH X-Google-Smtp-Source: AAOMgpfsMv0k02v1lNk0w/GBESNOWDFOb05V5laK/zCgeQRC5s+kkwdzhZ9nv2K3XKkwsRQa1SXFRA== X-Received: by 2002:a63:6243:: with SMTP id w64-v6mr8903457pgb.179.1532272408493; Sun, 22 Jul 2018 08:13:28 -0700 (PDT) Received: from localhost.localdomain (i153-145-22-9.s42.a013.ap.plala.or.jp. [153.145.22.9]) by smtp.gmail.com with ESMTPSA id v6-v6sm12092940pfa.28.2018.07.22.08.13.26 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Sun, 22 Jul 2018 08:13:27 -0700 (PDT) From: Toshiaki Makita To: netdev@vger.kernel.org, Alexei Starovoitov , Daniel Borkmann Cc: Toshiaki Makita , Jesper Dangaard Brouer Subject: [PATCH v3 bpf-next 4/8] veth: Handle xdp_frames in xdp napi ring Date: Mon, 23 Jul 2018 00:13:04 +0900 Message-Id: <20180722151308.5480-5-toshiaki.makita1@gmail.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180722151308.5480-1-toshiaki.makita1@gmail.com> References: <20180722151308.5480-1-toshiaki.makita1@gmail.com> Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Toshiaki Makita This is preparation for XDP TX and ndo_xdp_xmit. This allows napi handler to handle xdp_frames through xdp ring as well as sk_buff. v3: - Revert v2 change around rings and use a flag to differentiate skb and xdp_frame, since bulk skb xmit makes little performance difference for now. v2: - Use another ring instead of using flag to differentiate skb and xdp_frame. This approach makes bulk skb transmit possible in veth_xmit later. - Clear xdp_frame feilds in skb->head. - Implement adjust_tail. Signed-off-by: Toshiaki Makita --- drivers/net/veth.c | 87 ++++++++++++++++++++++++++++++++++++++++++++++++++---- 1 file changed, 82 insertions(+), 5 deletions(-) diff --git a/drivers/net/veth.c b/drivers/net/veth.c index f5b72e937d9d..4be75c58bc6a 100644 --- a/drivers/net/veth.c +++ b/drivers/net/veth.c @@ -22,12 +22,12 @@ #include #include #include -#include #include #define DRV_NAME "veth" #define DRV_VERSION "1.0" +#define VETH_XDP_FLAG BIT(0) #define VETH_RING_SIZE 256 #define VETH_XDP_HEADROOM (XDP_PACKET_HEADROOM + NET_IP_ALIGN) @@ -115,6 +115,24 @@ static const struct ethtool_ops veth_ethtool_ops = { /* general routines */ +static bool veth_is_xdp_frame(void *ptr) +{ + return (unsigned long)ptr & VETH_XDP_FLAG; +} + +static void *veth_ptr_to_xdp(void *ptr) +{ + return (void *)((unsigned long)ptr & ~VETH_XDP_FLAG); +} + +static void veth_ptr_free(void *ptr) +{ + if (veth_is_xdp_frame(ptr)) + xdp_return_frame(veth_ptr_to_xdp(ptr)); + else + kfree_skb(ptr); +} + static void __veth_xdp_flush(struct veth_priv *priv) { /* Write ptr_ring before reading rx_notify_masked */ @@ -249,6 +267,61 @@ static struct sk_buff *veth_build_skb(void *head, int headroom, int len, return skb; } +static struct sk_buff *veth_xdp_rcv_one(struct veth_priv *priv, + struct xdp_frame *frame) +{ + int len = frame->len, delta = 0; + struct bpf_prog *xdp_prog; + unsigned int headroom; + struct sk_buff *skb; + + rcu_read_lock(); + xdp_prog = rcu_dereference(priv->xdp_prog); + if (likely(xdp_prog)) { + struct xdp_buff xdp; + u32 act; + + xdp.data_hard_start = frame->data - frame->headroom; + xdp.data = frame->data; + xdp.data_end = frame->data + frame->len; + xdp.data_meta = frame->data - frame->metasize; + xdp.rxq = &priv->xdp_rxq; + + act = bpf_prog_run_xdp(xdp_prog, &xdp); + + switch (act) { + case XDP_PASS: + delta = frame->data - xdp.data; + len = xdp.data_end - xdp.data; + break; + default: + bpf_warn_invalid_xdp_action(act); + case XDP_ABORTED: + trace_xdp_exception(priv->dev, xdp_prog, act); + case XDP_DROP: + goto err_xdp; + } + } + rcu_read_unlock(); + + headroom = frame->data - delta - (void *)frame; + skb = veth_build_skb(frame, headroom, len, 0); + if (!skb) { + xdp_return_frame(frame); + goto err; + } + + memset(frame, 0, sizeof(*frame)); + skb->protocol = eth_type_trans(skb, priv->dev); +err: + return skb; +err_xdp: + rcu_read_unlock(); + xdp_return_frame(frame); + + return NULL; +} + static struct sk_buff *veth_xdp_rcv_skb(struct veth_priv *priv, struct sk_buff *skb) { @@ -358,12 +431,16 @@ static int veth_xdp_rcv(struct veth_priv *priv, int budget) int i, done = 0; for (i = 0; i < budget; i++) { - struct sk_buff *skb = __ptr_ring_consume(&priv->xdp_ring); + void *ptr = __ptr_ring_consume(&priv->xdp_ring); + struct sk_buff *skb; - if (!skb) + if (!ptr) break; - skb = veth_xdp_rcv_skb(priv, skb); + if (veth_is_xdp_frame(ptr)) + skb = veth_xdp_rcv_one(priv, veth_ptr_to_xdp(ptr)); + else + skb = veth_xdp_rcv_skb(priv, ptr); if (skb) napi_gro_receive(&priv->xdp_napi, skb); @@ -416,7 +493,7 @@ static void veth_napi_del(struct net_device *dev) napi_disable(&priv->xdp_napi); netif_napi_del(&priv->xdp_napi); priv->rx_notify_masked = false; - ptr_ring_cleanup(&priv->xdp_ring, __skb_array_destroy_skb); + ptr_ring_cleanup(&priv->xdp_ring, veth_ptr_free); } static int veth_enable_xdp(struct net_device *dev)