From patchwork Wed May 13 01:45:57 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Ahern X-Patchwork-Id: 1288896 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=none dis=none) header.from=kernel.org Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=default header.b=QRSr1XV2; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 49MHZZ1pxWz9sSf for ; Wed, 13 May 2020 11:46:14 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732004AbgEMBqM (ORCPT ); Tue, 12 May 2020 21:46:12 -0400 Received: from mail.kernel.org ([198.145.29.99]:40404 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731910AbgEMBqL (ORCPT ); Tue, 12 May 2020 21:46:11 -0400 Received: from C02YQ0RWLVCF.internal.digitalocean.com (c-73-181-34-237.hsd1.co.comcast.net [73.181.34.237]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 94E9C207BC; Wed, 13 May 2020 01:46:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1589334371; bh=QmSY0DTkfhy13FYDnKDgFH45jP01Djrm+1DRtCEGZA0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=QRSr1XV26s6jg0oSzOW35Gv/0+yl+BoHAs1BRqNShFswRig1O4Z18gJW4rbANd9Mt EB62ZHbz7PkDjVmo+A4VydOUEuyxuLgsBk7SgbiwbvV2QivLvvmJGmbKKbBOiXLeoC 1UTuWiYGzD6Ul76MIEba5i+SEUESxdj9T4A7aq5c= From: David Ahern To: netdev@vger.kernel.org Cc: davem@davemloft.net, kuba@kernel.org, prashantbhole.linux@gmail.com, brouer@redhat.com, toke@redhat.com, daniel@iogearbox.net, john.fastabend@gmail.com, ast@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, andriin@fb.com, dsahern@gmail.com, David Ahern Subject: [PATCH v5 bpf-next 01/11] net: Refactor convert_to_xdp_frame Date: Tue, 12 May 2020 19:45:57 -0600 Message-Id: <20200513014607.40418-2-dsahern@kernel.org> X-Mailer: git-send-email 2.21.1 (Apple Git-122.3) In-Reply-To: <20200513014607.40418-1-dsahern@kernel.org> References: <20200513014607.40418-1-dsahern@kernel.org> MIME-Version: 1.0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: David Ahern Move the guts of convert_to_xdp_frame to a new helper, update_xdp_frame so it can be reused in a later patch. Suggested-by: Jesper Dangaard Brouer Signed-off-by: David Ahern Acked-by: Jesper Dangaard Brouer Acked-by: John Fastabend --- include/net/xdp.h | 30 ++++++++++++++++++++---------- 1 file changed, 20 insertions(+), 10 deletions(-) diff --git a/include/net/xdp.h b/include/net/xdp.h index 3cc6d5d84aa4..3264fa882de3 100644 --- a/include/net/xdp.h +++ b/include/net/xdp.h @@ -93,32 +93,42 @@ static inline void xdp_scrub_frame(struct xdp_frame *frame) struct xdp_frame *xdp_convert_zc_to_xdp_frame(struct xdp_buff *xdp); -/* Convert xdp_buff to xdp_frame */ static inline -struct xdp_frame *convert_to_xdp_frame(struct xdp_buff *xdp) +bool update_xdp_frame(struct xdp_buff *xdp, struct xdp_frame *xdp_frame) { - struct xdp_frame *xdp_frame; int metasize; int headroom; - if (xdp->rxq->mem.type == MEM_TYPE_ZERO_COPY) - return xdp_convert_zc_to_xdp_frame(xdp); - /* Assure headroom is available for storing info */ headroom = xdp->data - xdp->data_hard_start; metasize = xdp->data - xdp->data_meta; metasize = metasize > 0 ? metasize : 0; if (unlikely((headroom - metasize) < sizeof(*xdp_frame))) - return NULL; - - /* Store info in top of packet */ - xdp_frame = xdp->data_hard_start; + return false; xdp_frame->data = xdp->data; xdp_frame->len = xdp->data_end - xdp->data; xdp_frame->headroom = headroom - sizeof(*xdp_frame); xdp_frame->metasize = metasize; + return true; +} + +/* Convert xdp_buff to xdp_frame */ +static inline +struct xdp_frame *convert_to_xdp_frame(struct xdp_buff *xdp) +{ + struct xdp_frame *xdp_frame; + + if (xdp->rxq->mem.type == MEM_TYPE_ZERO_COPY) + return xdp_convert_zc_to_xdp_frame(xdp); + + /* Store info in top of packet */ + xdp_frame = xdp->data_hard_start; + + if (unlikely(!update_xdp_frame(xdp, xdp_frame))) + return NULL; + /* rxq only valid until napi_schedule ends, convert to xdp_mem_info */ xdp_frame->mem = xdp->rxq->mem; From patchwork Wed May 13 01:45:58 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Ahern X-Patchwork-Id: 1288897 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=none dis=none) header.from=kernel.org Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=default header.b=KeQC/ym3; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 49MHZc0GxDz9sPF for ; Wed, 13 May 2020 11:46:16 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732043AbgEMBqP (ORCPT ); Tue, 12 May 2020 21:46:15 -0400 Received: from mail.kernel.org ([198.145.29.99]:40422 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731910AbgEMBqN (ORCPT ); Tue, 12 May 2020 21:46:13 -0400 Received: from C02YQ0RWLVCF.internal.digitalocean.com (c-73-181-34-237.hsd1.co.comcast.net [73.181.34.237]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 7E2B223127; Wed, 13 May 2020 01:46:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1589334372; bh=c7FAwZ008IMM11Xea34xQoPuCX2npLLlhNRter2L0vA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=KeQC/ym3+pX1iEK2GHnRy2jvF17eQa7TcbPC6UGKqA/jrM8u/ON6GCb8d76h/gq9u cI2A39j8TWYxdhrgbdWZ2FTBz6L5N2ll4iz5QwbZwn1snQoFhB7nf/i9gTS0SZQgz3 6i7sF4gqPmj3mvLYrmhbu8s2CSRgi7MOZ9o0pL6M= From: David Ahern To: netdev@vger.kernel.org Cc: davem@davemloft.net, kuba@kernel.org, prashantbhole.linux@gmail.com, brouer@redhat.com, toke@redhat.com, daniel@iogearbox.net, john.fastabend@gmail.com, ast@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, andriin@fb.com, dsahern@gmail.com, David Ahern Subject: [PATCH v5 bpf-next 02/11] net: uapi for XDP programs in the egress path Date: Tue, 12 May 2020 19:45:58 -0600 Message-Id: <20200513014607.40418-3-dsahern@kernel.org> X-Mailer: git-send-email 2.21.1 (Apple Git-122.3) In-Reply-To: <20200513014607.40418-1-dsahern@kernel.org> References: <20200513014607.40418-1-dsahern@kernel.org> MIME-Version: 1.0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: David Ahern Running programs in the egress path, on skbs or xdp_frames, does not require driver specific resources like Rx path. Accordingly, the programs can be run in core code, so add xdp_egress_prog to net_device to hold a reference to an attached program. For UAPI, add XDP_FLAGS_EGRESS_MODE to specify attach is at egress, add a new attach flag, XDP_ATTACHED_EGRESS_CORE, for reporting the attach point at the core, egress level and add IFLA_XDP_EGRESS_PROG_ID for reporting the program id. Add rtnl_xdp_prog_egress to fill in link message with egress data. Signed-off-by: David Ahern --- include/linux/netdevice.h | 1 + include/uapi/linux/if_link.h | 6 +++++- net/core/rtnetlink.c | 23 +++++++++++++++++++++-- tools/include/uapi/linux/if_link.h | 6 +++++- 4 files changed, 32 insertions(+), 4 deletions(-) diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h index 5a8d40f1ffe2..594c13d4cd00 100644 --- a/include/linux/netdevice.h +++ b/include/linux/netdevice.h @@ -1995,6 +1995,7 @@ struct net_device { unsigned int real_num_rx_queues; struct bpf_prog __rcu *xdp_prog; + struct bpf_prog __rcu *xdp_egress_prog; unsigned long gro_flush_timeout; int napi_defer_hard_irqs; rx_handler_func_t __rcu *rx_handler; diff --git a/include/uapi/linux/if_link.h b/include/uapi/linux/if_link.h index a009365ad67b..de32a9e94d74 100644 --- a/include/uapi/linux/if_link.h +++ b/include/uapi/linux/if_link.h @@ -976,9 +976,11 @@ enum { #define XDP_FLAGS_DRV_MODE (1U << 2) #define XDP_FLAGS_HW_MODE (1U << 3) #define XDP_FLAGS_REPLACE (1U << 4) +#define XDP_FLAGS_EGRESS_MODE (1U << 5) #define XDP_FLAGS_MODES (XDP_FLAGS_SKB_MODE | \ XDP_FLAGS_DRV_MODE | \ - XDP_FLAGS_HW_MODE) + XDP_FLAGS_HW_MODE | \ + XDP_FLAGS_EGRESS_MODE) #define XDP_FLAGS_MASK (XDP_FLAGS_UPDATE_IF_NOEXIST | \ XDP_FLAGS_MODES | XDP_FLAGS_REPLACE) @@ -989,6 +991,7 @@ enum { XDP_ATTACHED_SKB, XDP_ATTACHED_HW, XDP_ATTACHED_MULTI, + XDP_ATTACHED_EGRESS_CORE, }; enum { @@ -1001,6 +1004,7 @@ enum { IFLA_XDP_SKB_PROG_ID, IFLA_XDP_HW_PROG_ID, IFLA_XDP_EXPECTED_FD, + IFLA_XDP_EGRESS_PROG_ID, __IFLA_XDP_MAX, }; diff --git a/net/core/rtnetlink.c b/net/core/rtnetlink.c index 2269199c5891..9084dfebc699 100644 --- a/net/core/rtnetlink.c +++ b/net/core/rtnetlink.c @@ -982,7 +982,7 @@ static size_t rtnl_xdp_size(void) size_t xdp_size = nla_total_size(0) + /* nest IFLA_XDP */ nla_total_size(1) + /* XDP_ATTACHED */ nla_total_size(4) + /* XDP_PROG_ID (or 1st mode) */ - nla_total_size(4); /* XDP__PROG_ID */ + nla_total_size(4) * 2; /* XDP__PROG_ID */ return xdp_size; } @@ -1402,6 +1402,18 @@ static int rtnl_fill_link_ifmap(struct sk_buff *skb, struct net_device *dev) return 0; } +static u32 rtnl_xdp_prog_egress(struct net_device *dev) +{ + const struct bpf_prog *prog; + + ASSERT_RTNL(); + + prog = rtnl_dereference(dev->xdp_egress_prog); + if (!prog) + return 0; + return prog->aux->id; +} + static u32 rtnl_xdp_prog_skb(struct net_device *dev) { const struct bpf_prog *generic_xdp_prog; @@ -1474,6 +1486,12 @@ static int rtnl_xdp_fill(struct sk_buff *skb, struct net_device *dev) IFLA_XDP_HW_PROG_ID, rtnl_xdp_prog_hw); if (err) goto err_cancel; + err = rtnl_xdp_report_one(skb, dev, &prog_id, &mode, + XDP_ATTACHED_EGRESS_CORE, + IFLA_XDP_EGRESS_PROG_ID, + rtnl_xdp_prog_egress); + if (err) + goto err_cancel; err = nla_put_u8(skb, IFLA_XDP_ATTACHED, mode); if (err) @@ -2790,7 +2808,8 @@ static int do_setlink(const struct sk_buff *skb, if (err < 0) goto errout; - if (xdp[IFLA_XDP_ATTACHED] || xdp[IFLA_XDP_PROG_ID]) { + if (xdp[IFLA_XDP_ATTACHED] || xdp[IFLA_XDP_PROG_ID] || + xdp[IFLA_XDP_EGRESS_PROG_ID]) { err = -EINVAL; goto errout; } diff --git a/tools/include/uapi/linux/if_link.h b/tools/include/uapi/linux/if_link.h index cafedbbfefbe..1d61cb46a77c 100644 --- a/tools/include/uapi/linux/if_link.h +++ b/tools/include/uapi/linux/if_link.h @@ -964,9 +964,11 @@ enum { #define XDP_FLAGS_DRV_MODE (1U << 2) #define XDP_FLAGS_HW_MODE (1U << 3) #define XDP_FLAGS_REPLACE (1U << 4) +#define XDP_FLAGS_EGRESS_MODE (1U << 5) #define XDP_FLAGS_MODES (XDP_FLAGS_SKB_MODE | \ XDP_FLAGS_DRV_MODE | \ - XDP_FLAGS_HW_MODE) + XDP_FLAGS_HW_MODE | \ + XDP_FLAGS_EGRESS_MODE) #define XDP_FLAGS_MASK (XDP_FLAGS_UPDATE_IF_NOEXIST | \ XDP_FLAGS_MODES | XDP_FLAGS_REPLACE) @@ -977,6 +979,7 @@ enum { XDP_ATTACHED_SKB, XDP_ATTACHED_HW, XDP_ATTACHED_MULTI, + XDP_ATTACHED_EGRESS_CORE, }; enum { @@ -989,6 +992,7 @@ enum { IFLA_XDP_SKB_PROG_ID, IFLA_XDP_HW_PROG_ID, IFLA_XDP_EXPECTED_FD, + IFLA_XDP_EGRESS_PROG_ID, __IFLA_XDP_MAX, }; From patchwork Wed May 13 01:45:59 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Ahern X-Patchwork-Id: 1288900 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=none dis=none) header.from=kernel.org Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=default header.b=fQSYe7e0; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 49MHZh00NZz9sPF for ; Wed, 13 May 2020 11:46:19 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732108AbgEMBqS (ORCPT ); Tue, 12 May 2020 21:46:18 -0400 Received: from mail.kernel.org ([198.145.29.99]:40472 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1732015AbgEMBqO (ORCPT ); Tue, 12 May 2020 21:46:14 -0400 Received: from C02YQ0RWLVCF.internal.digitalocean.com (c-73-181-34-237.hsd1.co.comcast.net [73.181.34.237]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 71A402492A; Wed, 13 May 2020 01:46:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1589334373; bh=8b4+XTz2mLq34mnoxFs6xIkw/tB+P66T3si0vrPxh1E=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=fQSYe7e0At+NmfSyd9ldQ3Wh8Po34XAxz4ZH/5CbLt84nSFiMNdBnBnxLF/EBaqQH TN30XSN4/4jxsYedZ26A8HZMzEj0ngpSUYeW4VpnidjBGl1txfDnaUFyAJTxgbB28R oJ515NMeGrcjjQJeCMltFqN054IROe9fWZq/mPc4= From: David Ahern To: netdev@vger.kernel.org Cc: davem@davemloft.net, kuba@kernel.org, prashantbhole.linux@gmail.com, brouer@redhat.com, toke@redhat.com, daniel@iogearbox.net, john.fastabend@gmail.com, ast@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, andriin@fb.com, dsahern@gmail.com, David Ahern Subject: [PATCH v5 bpf-next 03/11] net: Add XDP setup and query commands for Tx programs Date: Tue, 12 May 2020 19:45:59 -0600 Message-Id: <20200513014607.40418-4-dsahern@kernel.org> X-Mailer: git-send-email 2.21.1 (Apple Git-122.3) In-Reply-To: <20200513014607.40418-1-dsahern@kernel.org> References: <20200513014607.40418-1-dsahern@kernel.org> MIME-Version: 1.0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: David Ahern Add new netdev commands, XDP_SETUP_PROG_EGRESS and XDP_QUERY_PROG_EGRESS, to query and setup egress programs. Update dev_change_xdp_fd and dev_xdp_install to check for egress mode via XDP_FLAGS_EGRESS_MODE in the flags. If egress bool is set, then use XDP_SETUP_PROG_EGRESS in dev_xdp_install and XDP_QUERY_PROG_EGRESS in dev_change_xdp_fd. Signed-off-by: David Ahern Co-developed-by: Prashant Bhole Signed-off-by: Prashant Bhole --- include/linux/netdevice.h | 2 ++ net/core/dev.c | 20 +++++++++++++++----- 2 files changed, 17 insertions(+), 5 deletions(-) diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h index 594c13d4cd00..ee0cb73ca18a 100644 --- a/include/linux/netdevice.h +++ b/include/linux/netdevice.h @@ -873,8 +873,10 @@ enum bpf_netdev_command { */ XDP_SETUP_PROG, XDP_SETUP_PROG_HW, + XDP_SETUP_PROG_EGRESS, XDP_QUERY_PROG, XDP_QUERY_PROG_HW, + XDP_QUERY_PROG_EGRESS, /* BPF program for offload callbacks, invoked at program load time. */ BPF_OFFLOAD_MAP_ALLOC, BPF_OFFLOAD_MAP_FREE, diff --git a/net/core/dev.c b/net/core/dev.c index afff16849c26..c0455e764f97 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -8600,13 +8600,16 @@ static int dev_xdp_install(struct net_device *dev, bpf_op_t bpf_op, struct bpf_prog *prog) { bool non_hw = !(flags & XDP_FLAGS_HW_MODE); + bool egress = flags & XDP_FLAGS_EGRESS_MODE; struct bpf_prog *prev_prog = NULL; struct netdev_bpf xdp; int err; if (non_hw) { - prev_prog = bpf_prog_by_id(__dev_xdp_query(dev, bpf_op, - XDP_QUERY_PROG)); + enum bpf_netdev_command cmd; + + cmd = egress ? XDP_QUERY_PROG_EGRESS : XDP_QUERY_PROG; + prev_prog = bpf_prog_by_id(__dev_xdp_query(dev, bpf_op, cmd)); if (IS_ERR(prev_prog)) prev_prog = NULL; } @@ -8615,7 +8618,7 @@ static int dev_xdp_install(struct net_device *dev, bpf_op_t bpf_op, if (flags & XDP_FLAGS_HW_MODE) xdp.command = XDP_SETUP_PROG_HW; else - xdp.command = XDP_SETUP_PROG; + xdp.command = egress ? XDP_SETUP_PROG_EGRESS : XDP_SETUP_PROG; xdp.extack = extack; xdp.flags = flags; xdp.prog = prog; @@ -8677,12 +8680,18 @@ int dev_change_xdp_fd(struct net_device *dev, struct netlink_ext_ack *extack, bpf_op_t bpf_op, bpf_chk; struct bpf_prog *prog; bool offload; + bool egress; int err; ASSERT_RTNL(); offload = flags & XDP_FLAGS_HW_MODE; - query = offload ? XDP_QUERY_PROG_HW : XDP_QUERY_PROG; + egress = flags & XDP_FLAGS_EGRESS_MODE; + if (egress) + query = XDP_QUERY_PROG_EGRESS; + else + query = offload ? XDP_QUERY_PROG_HW : XDP_QUERY_PROG; + bpf_op = bpf_chk = ops->ndo_bpf; if (!bpf_op && (flags & (XDP_FLAGS_DRV_MODE | XDP_FLAGS_HW_MODE))) { @@ -8712,7 +8721,8 @@ int dev_change_xdp_fd(struct net_device *dev, struct netlink_ext_ack *extack, } } if (fd >= 0) { - if (!offload && __dev_xdp_query(dev, bpf_chk, XDP_QUERY_PROG)) { + if (!offload && !egress && + __dev_xdp_query(dev, bpf_chk, XDP_QUERY_PROG)) { NL_SET_ERR_MSG(extack, "native and generic XDP can't be active at the same time"); return -EEXIST; } From patchwork Wed May 13 01:46:00 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Ahern X-Patchwork-Id: 1288899 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=none dis=none) header.from=kernel.org Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=default header.b=TkCcVuX4; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 49MHZf5Tykz9sPF for ; Wed, 13 May 2020 11:46:18 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732095AbgEMBqS (ORCPT ); Tue, 12 May 2020 21:46:18 -0400 Received: from mail.kernel.org ([198.145.29.99]:40476 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1732025AbgEMBqO (ORCPT ); Tue, 12 May 2020 21:46:14 -0400 Received: from C02YQ0RWLVCF.internal.digitalocean.com (c-73-181-34-237.hsd1.co.comcast.net [73.181.34.237]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 5B2A62176D; Wed, 13 May 2020 01:46:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1589334374; bh=lEBLyFT15HCByb+IUiNLsBXb2A5sDPMavaUD/jBFk7w=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=TkCcVuX49Wm8/1N4l/O9JdqnRzbAVNWF8y37hQAtAfXdu8q87gnZ6ZRQuGe2VEluV x7xObLjWRpyyZ9GQqC/zse1J4hmJedQwwmGylP4YHP9vEc2BMbIxAV/W1o/dKwzQBq Oeh2QqK530ZpViOgQVqYbikAFleXIn08wOcfS17k= From: David Ahern To: netdev@vger.kernel.org Cc: davem@davemloft.net, kuba@kernel.org, prashantbhole.linux@gmail.com, brouer@redhat.com, toke@redhat.com, daniel@iogearbox.net, john.fastabend@gmail.com, ast@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, andriin@fb.com, dsahern@gmail.com, David Ahern Subject: [PATCH v5 bpf-next 04/11] net: Add BPF_XDP_EGRESS as a bpf_attach_type Date: Tue, 12 May 2020 19:46:00 -0600 Message-Id: <20200513014607.40418-5-dsahern@kernel.org> X-Mailer: git-send-email 2.21.1 (Apple Git-122.3) In-Reply-To: <20200513014607.40418-1-dsahern@kernel.org> References: <20200513014607.40418-1-dsahern@kernel.org> MIME-Version: 1.0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: David Ahern Add new bpf_attach_type, BPF_XDP_EGRESS, for BPF programs attached at the XDP layer, but the egress path. Since egress path will not have ingress_ifindex and rx_queue_index set, update xdp_is_valid_access to block access to these entries in the xdp context when a program is attached with expected_attach_type set. Update dev_change_xdp_fd to verify expected_attach_type for a program is BPF_XDP_EGRESS if egress argument is set. The next patch adds support for the egress ifindex. Signed-off-by: Prashant Bhole Co-developed-by: David Ahern Signed-off-by: David Ahern --- include/uapi/linux/bpf.h | 1 + net/core/dev.c | 11 +++++++++++ net/core/filter.c | 11 +++++++++++ tools/include/uapi/linux/bpf.h | 1 + 4 files changed, 24 insertions(+) diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h index bfb31c1be219..05accb95bb4a 100644 --- a/include/uapi/linux/bpf.h +++ b/include/uapi/linux/bpf.h @@ -220,6 +220,7 @@ enum bpf_attach_type { BPF_MODIFY_RETURN, BPF_LSM_MAC, BPF_TRACE_ITER, + BPF_XDP_EGRESS, __MAX_BPF_ATTACH_TYPE }; diff --git a/net/core/dev.c b/net/core/dev.c index c0455e764f97..88672ea4fc80 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -8737,6 +8737,17 @@ int dev_change_xdp_fd(struct net_device *dev, struct netlink_ext_ack *extack, if (IS_ERR(prog)) return PTR_ERR(prog); + if (egress && prog->expected_attach_type != BPF_XDP_EGRESS) { + NL_SET_ERR_MSG(extack, "XDP program in Tx path must use BPF_XDP_EGRESS attach type"); + bpf_prog_put(prog); + return -EINVAL; + } + if (!egress && prog->expected_attach_type == BPF_XDP_EGRESS) { + NL_SET_ERR_MSG(extack, "XDP program in Rx path can not use BPF_XDP_EGRESS attach type"); + bpf_prog_put(prog); + return -EINVAL; + } + if (!offload && bpf_prog_is_dev_bound(prog->aux)) { NL_SET_ERR_MSG(extack, "using device-bound program without HW_MODE flag is not supported"); bpf_prog_put(prog); diff --git a/net/core/filter.c b/net/core/filter.c index da0634979f53..19272eb7bb8f 100644 --- a/net/core/filter.c +++ b/net/core/filter.c @@ -6931,6 +6931,17 @@ static bool xdp_is_valid_access(int off, int size, const struct bpf_prog *prog, struct bpf_insn_access_aux *info) { + /* Rx data is only accessible from original XDP where + * expected_attach_type is not set + */ + if (prog->expected_attach_type) { + switch (off) { + case offsetof(struct xdp_md, ingress_ifindex): + case offsetof(struct xdp_md, rx_queue_index): + return false; + } + } + if (type == BPF_WRITE) { if (bpf_prog_is_dev_bound(prog->aux)) { switch (off) { diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h index bfb31c1be219..05accb95bb4a 100644 --- a/tools/include/uapi/linux/bpf.h +++ b/tools/include/uapi/linux/bpf.h @@ -220,6 +220,7 @@ enum bpf_attach_type { BPF_MODIFY_RETURN, BPF_LSM_MAC, BPF_TRACE_ITER, + BPF_XDP_EGRESS, __MAX_BPF_ATTACH_TYPE }; From patchwork Wed May 13 01:46:01 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Ahern X-Patchwork-Id: 1288898 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=none dis=none) header.from=kernel.org Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=default header.b=hX4cq7h9; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 49MHZd5Lqyz9sPF for ; Wed, 13 May 2020 11:46:17 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732080AbgEMBqR (ORCPT ); Tue, 12 May 2020 21:46:17 -0400 Received: from mail.kernel.org ([198.145.29.99]:40494 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731910AbgEMBqP (ORCPT ); Tue, 12 May 2020 21:46:15 -0400 Received: from C02YQ0RWLVCF.internal.digitalocean.com (c-73-181-34-237.hsd1.co.comcast.net [73.181.34.237]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 569DF24929; Wed, 13 May 2020 01:46:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1589334375; bh=QZyMYN62IGo4Lj8GBMLJZh8eRIrTfLlpw8fLjAqAhfo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=hX4cq7h9g8bajnGibNDGgsUfrEGh0Rk5RKd4Et4cPIxykv8mf7XU6+4JR48GUmSIa HXDs0hz+fz3QuLdzDE84MeihfAHpZEJmSuAZeJLuS3xCHR4CMBMN7MSSoCY4kZFl1B sxTtLaNpNfUwSXnxDbMHhe+K3W6SM/zZwdztw5EU= From: David Ahern To: netdev@vger.kernel.org Cc: davem@davemloft.net, kuba@kernel.org, prashantbhole.linux@gmail.com, brouer@redhat.com, toke@redhat.com, daniel@iogearbox.net, john.fastabend@gmail.com, ast@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, andriin@fb.com, dsahern@gmail.com, David Ahern Subject: [PATCH v5 bpf-next 05/11] xdp: Add xdp_txq_info to xdp_buff Date: Tue, 12 May 2020 19:46:01 -0600 Message-Id: <20200513014607.40418-6-dsahern@kernel.org> X-Mailer: git-send-email 2.21.1 (Apple Git-122.3) In-Reply-To: <20200513014607.40418-1-dsahern@kernel.org> References: <20200513014607.40418-1-dsahern@kernel.org> MIME-Version: 1.0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: David Ahern Add xdp_txq_info as the Tx counterpart to xdp_rxq_info. At the moment only the device is added. Other fields (queue_index) can be added as use cases arise. From a UAPI perspective, add egress_ifindex to xdp context. Update the verifier to reject accesses to egress_ifindex by rx programs. Signed-off-by: David Ahern --- include/net/xdp.h | 5 +++++ include/uapi/linux/bpf.h | 2 ++ net/core/filter.c | 15 +++++++++++++++ tools/include/uapi/linux/bpf.h | 2 ++ 4 files changed, 24 insertions(+) diff --git a/include/net/xdp.h b/include/net/xdp.h index 3264fa882de3..2b85b8649201 100644 --- a/include/net/xdp.h +++ b/include/net/xdp.h @@ -63,6 +63,10 @@ struct xdp_rxq_info { struct xdp_mem_info mem; } ____cacheline_aligned; /* perf critical, avoid false-sharing */ +struct xdp_txq_info { + struct net_device *dev; +}; + struct xdp_buff { void *data; void *data_end; @@ -70,6 +74,7 @@ struct xdp_buff { void *data_hard_start; unsigned long handle; struct xdp_rxq_info *rxq; + struct xdp_txq_info *txq; }; struct xdp_frame { diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h index 05accb95bb4a..54d50dd62498 100644 --- a/include/uapi/linux/bpf.h +++ b/include/uapi/linux/bpf.h @@ -3586,6 +3586,8 @@ struct xdp_md { /* Below access go through struct xdp_rxq_info */ __u32 ingress_ifindex; /* rxq->dev->ifindex */ __u32 rx_queue_index; /* rxq->queue_index */ + + __u32 egress_ifindex; /* txq->dev->ifindex */ }; enum sk_action { diff --git a/net/core/filter.c b/net/core/filter.c index 19272eb7bb8f..c1edd800b4f1 100644 --- a/net/core/filter.c +++ b/net/core/filter.c @@ -6940,6 +6940,11 @@ static bool xdp_is_valid_access(int off, int size, case offsetof(struct xdp_md, rx_queue_index): return false; } + } else if (prog->expected_attach_type != BPF_XDP_EGRESS) { + switch (off) { + case offsetof(struct xdp_md, egress_ifindex): + return false; + } } if (type == BPF_WRITE) { @@ -7889,6 +7894,16 @@ static u32 xdp_convert_ctx_access(enum bpf_access_type type, offsetof(struct xdp_rxq_info, queue_index)); break; + case offsetof(struct xdp_md, egress_ifindex): + *insn++ = BPF_LDX_MEM(BPF_FIELD_SIZEOF(struct xdp_buff, txq), + si->dst_reg, si->src_reg, + offsetof(struct xdp_buff, txq)); + *insn++ = BPF_LDX_MEM(BPF_FIELD_SIZEOF(struct xdp_txq_info, dev), + si->dst_reg, si->dst_reg, + offsetof(struct xdp_txq_info, dev)); + *insn++ = BPF_LDX_MEM(BPF_W, si->dst_reg, si->dst_reg, + offsetof(struct net_device, ifindex)); + break; } return insn - insn_buf; diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h index 05accb95bb4a..54d50dd62498 100644 --- a/tools/include/uapi/linux/bpf.h +++ b/tools/include/uapi/linux/bpf.h @@ -3586,6 +3586,8 @@ struct xdp_md { /* Below access go through struct xdp_rxq_info */ __u32 ingress_ifindex; /* rxq->dev->ifindex */ __u32 rx_queue_index; /* rxq->queue_index */ + + __u32 egress_ifindex; /* txq->dev->ifindex */ }; enum sk_action { From patchwork Wed May 13 01:46:02 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Ahern X-Patchwork-Id: 1288906 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=none dis=none) header.from=kernel.org Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=default header.b=an63eFQP; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 49MHZw70hqz9sPF for ; Wed, 13 May 2020 11:46:32 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732152AbgEMBqc (ORCPT ); Tue, 12 May 2020 21:46:32 -0400 Received: from mail.kernel.org ([198.145.29.99]:40514 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728228AbgEMBqQ (ORCPT ); Tue, 12 May 2020 21:46:16 -0400 Received: from C02YQ0RWLVCF.internal.digitalocean.com (c-73-181-34-237.hsd1.co.comcast.net [73.181.34.237]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 454592492B; Wed, 13 May 2020 01:46:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1589334376; bh=SSWd8S1f95DfGyhP5HFRg/GjvOR9Vy7boGaZY5sUiAg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=an63eFQPgcYmojWLv+siqnO7au/oI8Qt71T+fvmGclCN7lQ2gqx9UG6s0KGvR5p5d bxjKmjONiakUQcVrXG8Bo05gaLvSXtbJoN8fS6Ovryg6ANpI4HHP/nemhQM2DjRbtM 0JkNbuk6kU81R8qTQixdqmZxn4LFJ8cz/BdruD/c= From: David Ahern To: netdev@vger.kernel.org Cc: davem@davemloft.net, kuba@kernel.org, prashantbhole.linux@gmail.com, brouer@redhat.com, toke@redhat.com, daniel@iogearbox.net, john.fastabend@gmail.com, ast@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, andriin@fb.com, dsahern@gmail.com, David Ahern Subject: [PATCH v5 bpf-next 06/11] net: set XDP egress program on netdevice Date: Tue, 12 May 2020 19:46:02 -0600 Message-Id: <20200513014607.40418-7-dsahern@kernel.org> X-Mailer: git-send-email 2.21.1 (Apple Git-122.3) In-Reply-To: <20200513014607.40418-1-dsahern@kernel.org> References: <20200513014607.40418-1-dsahern@kernel.org> MIME-Version: 1.0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: David Ahern This patch handles the plumbing for installing an XDP egress program on a net_device by handling XDP_SETUP_PROG_EGRESS and XDP_QUERY_PROG_EGRESS in generic_xdp_install handler. New static key is added to signal when an egress program has been installed. Update dev_xdp_uninstall to remove egress programs. Signed-off-by: David Ahern --- include/linux/netdevice.h | 2 ++ net/core/dev.c | 48 +++++++++++++++++++++++++++------------ 2 files changed, 36 insertions(+), 14 deletions(-) diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h index ee0cb73ca18a..651baeb36729 100644 --- a/include/linux/netdevice.h +++ b/include/linux/netdevice.h @@ -752,6 +752,8 @@ struct netdev_rx_queue { #endif } ____cacheline_aligned_in_smp; +extern struct static_key_false xdp_egress_needed_key; + /* * RX queue sysfs structures and functions. */ diff --git a/net/core/dev.c b/net/core/dev.c index 88672ea4fc80..97954f835ceb 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -4641,6 +4641,7 @@ void generic_xdp_tx(struct sk_buff *skb, struct bpf_prog *xdp_prog) } static DEFINE_STATIC_KEY_FALSE(generic_xdp_needed_key); +DEFINE_STATIC_KEY_FALSE(xdp_egress_needed_key); int do_xdp_generic(struct bpf_prog *xdp_prog, struct sk_buff *skb) { @@ -5336,12 +5337,12 @@ static void __netif_receive_skb_list(struct list_head *head) static int generic_xdp_install(struct net_device *dev, struct netdev_bpf *xdp) { - struct bpf_prog *old = rtnl_dereference(dev->xdp_prog); - struct bpf_prog *new = xdp->prog; + struct bpf_prog *old, *new = xdp->prog; int ret = 0; switch (xdp->command) { case XDP_SETUP_PROG: + old = rtnl_dereference(dev->xdp_prog); rcu_assign_pointer(dev->xdp_prog, new); if (old) bpf_prog_put(old); @@ -5354,11 +5355,25 @@ static int generic_xdp_install(struct net_device *dev, struct netdev_bpf *xdp) dev_disable_gro_hw(dev); } break; + case XDP_SETUP_PROG_EGRESS: + old = rtnl_dereference(dev->xdp_egress_prog); + rcu_assign_pointer(dev->xdp_egress_prog, new); + if (old) + bpf_prog_put(old); + if (old && !new) + static_branch_dec(&xdp_egress_needed_key); + else if (new && !old) + static_branch_inc(&xdp_egress_needed_key); + break; case XDP_QUERY_PROG: + old = rtnl_dereference(dev->xdp_prog); + xdp->prog_id = old ? old->aux->id : 0; + break; + case XDP_QUERY_PROG_EGRESS: + old = rtnl_dereference(dev->xdp_egress_prog); xdp->prog_id = old ? old->aux->id : 0; break; - default: ret = -EINVAL; break; @@ -8641,6 +8656,10 @@ static void dev_xdp_uninstall(struct net_device *dev) /* Remove generic XDP */ WARN_ON(dev_xdp_install(dev, generic_xdp_install, NULL, 0, NULL)); + /* Remove XDP egress */ + WARN_ON(dev_xdp_install(dev, generic_xdp_install, NULL, + XDP_FLAGS_EGRESS_MODE, NULL)); + /* Remove from the driver */ ndo_bpf = dev->netdev_ops->ndo_bpf; if (!ndo_bpf) @@ -8687,21 +8706,22 @@ int dev_change_xdp_fd(struct net_device *dev, struct netlink_ext_ack *extack, offload = flags & XDP_FLAGS_HW_MODE; egress = flags & XDP_FLAGS_EGRESS_MODE; - if (egress) + if (egress) { query = XDP_QUERY_PROG_EGRESS; - else + bpf_op = bpf_chk = generic_xdp_install; + } else { query = offload ? XDP_QUERY_PROG_HW : XDP_QUERY_PROG; - - bpf_op = bpf_chk = ops->ndo_bpf; - if (!bpf_op && (flags & (XDP_FLAGS_DRV_MODE | XDP_FLAGS_HW_MODE))) { - NL_SET_ERR_MSG(extack, "underlying driver does not support XDP in native mode"); - return -EOPNOTSUPP; + bpf_op = bpf_chk = ops->ndo_bpf; + if (!bpf_op && (flags & (XDP_FLAGS_DRV_MODE | XDP_FLAGS_HW_MODE))) { + NL_SET_ERR_MSG(extack, "underlying driver does not support XDP in native mode"); + return -EOPNOTSUPP; + } + if (!bpf_op || (flags & XDP_FLAGS_SKB_MODE)) + bpf_op = generic_xdp_install; + if (bpf_op == bpf_chk) + bpf_chk = generic_xdp_install; } - if (!bpf_op || (flags & XDP_FLAGS_SKB_MODE)) - bpf_op = generic_xdp_install; - if (bpf_op == bpf_chk) - bpf_chk = generic_xdp_install; prog_id = __dev_xdp_query(dev, bpf_op, query); if (flags & XDP_FLAGS_REPLACE) { From patchwork Wed May 13 01:46:03 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Ahern X-Patchwork-Id: 1288901 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=none dis=none) header.from=kernel.org Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=default header.b=dqylJf96; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 49MHZj4Bxhz9sPF for ; Wed, 13 May 2020 11:46:21 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732120AbgEMBqU (ORCPT ); Tue, 12 May 2020 21:46:20 -0400 Received: from mail.kernel.org ([198.145.29.99]:40530 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1732066AbgEMBqR (ORCPT ); Tue, 12 May 2020 21:46:17 -0400 Received: from C02YQ0RWLVCF.internal.digitalocean.com (c-73-181-34-237.hsd1.co.comcast.net [73.181.34.237]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 371122492D; Wed, 13 May 2020 01:46:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1589334377; bh=qIGnmqx/IfZtLlSpPlOo4jKzm4xSYh6U82yhLyybz/c=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=dqylJf96jgDM54ECOZXhXF0nxdK6gtGvkkMwaB0z0zgxjQ60PK9SFVgIosvWIaEvw cfzS5jrHh0s8eP9c1FQ1P6WFDDeYWePWjE/mmS6mEyAipet5T9LR+Sp2QGbh1FBD9u Zc8kcDlcQkBPwQwqpUtNduR1MdZxhLnFDyS+NBoU= From: David Ahern To: netdev@vger.kernel.org Cc: davem@davemloft.net, kuba@kernel.org, prashantbhole.linux@gmail.com, brouer@redhat.com, toke@redhat.com, daniel@iogearbox.net, john.fastabend@gmail.com, ast@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, andriin@fb.com, dsahern@gmail.com, David Ahern Subject: [PATCH v5 bpf-next 07/11] net: Support xdp in the Tx path for xdp_frames Date: Tue, 12 May 2020 19:46:03 -0600 Message-Id: <20200513014607.40418-8-dsahern@kernel.org> X-Mailer: git-send-email 2.21.1 (Apple Git-122.3) In-Reply-To: <20200513014607.40418-1-dsahern@kernel.org> References: <20200513014607.40418-1-dsahern@kernel.org> MIME-Version: 1.0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: David Ahern Add support to run Tx path program on xdp_frames by adding a hook to bq_xmit_all before xdp_frames are passed to ndo_xdp_xmit for the device. If an xdp_frame is dropped by the program, it is removed from the xdp_frames array with subsequent entries moved up. Signed-off-by: David Ahern --- include/linux/netdevice.h | 2 ++ kernel/bpf/devmap.c | 19 +++++++--- net/core/dev.c | 74 +++++++++++++++++++++++++++++++++++++++ 3 files changed, 90 insertions(+), 5 deletions(-) diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h index 651baeb36729..042190df23d5 100644 --- a/include/linux/netdevice.h +++ b/include/linux/netdevice.h @@ -3717,6 +3717,8 @@ static inline void dev_consume_skb_any(struct sk_buff *skb) void generic_xdp_tx(struct sk_buff *skb, struct bpf_prog *xdp_prog); int do_xdp_generic(struct bpf_prog *xdp_prog, struct sk_buff *skb); +unsigned int do_xdp_egress(struct net_device *dev, struct xdp_frame **frames, + unsigned int count); int netif_rx(struct sk_buff *skb); int netif_rx_ni(struct sk_buff *skb); int netif_receive_skb(struct sk_buff *skb); diff --git a/kernel/bpf/devmap.c b/kernel/bpf/devmap.c index a51d9fb7a359..513dec5f67b9 100644 --- a/kernel/bpf/devmap.c +++ b/kernel/bpf/devmap.c @@ -321,24 +321,33 @@ static int bq_xmit_all(struct xdp_dev_bulk_queue *bq, u32 flags) { struct net_device *dev = bq->dev; int sent = 0, drops = 0, err = 0; + unsigned int count = bq->count; int i; - if (unlikely(!bq->count)) + if (unlikely(!count)) return 0; - for (i = 0; i < bq->count; i++) { + for (i = 0; i < count; i++) { struct xdp_frame *xdpf = bq->q[i]; prefetch(xdpf); } - sent = dev->netdev_ops->ndo_xdp_xmit(dev, bq->count, bq->q, flags); + if (static_branch_unlikely(&xdp_egress_needed_key)) { + count = do_xdp_egress(dev, bq->q, count); + drops += bq->count - count; + /* all frames consumed by the xdp program? */ + if (!count) + goto out; + } + + sent = dev->netdev_ops->ndo_xdp_xmit(dev, count, bq->q, flags); if (sent < 0) { err = sent; sent = 0; goto error; } - drops = bq->count - sent; + drops += count - sent; out: bq->count = 0; @@ -350,7 +359,7 @@ static int bq_xmit_all(struct xdp_dev_bulk_queue *bq, u32 flags) /* If ndo_xdp_xmit fails with an errno, no frames have been * xmit'ed and it's our responsibility to them free all. */ - for (i = 0; i < bq->count; i++) { + for (i = 0; i < count; i++) { struct xdp_frame *xdpf = bq->q[i]; xdp_return_frame_rx_napi(xdpf); diff --git a/net/core/dev.c b/net/core/dev.c index 97954f835ceb..e6a737b84768 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -4673,6 +4673,80 @@ int do_xdp_generic(struct bpf_prog *xdp_prog, struct sk_buff *skb) } EXPORT_SYMBOL_GPL(do_xdp_generic); +static u32 __xdp_egress_frame(struct net_device *dev, + struct bpf_prog *xdp_prog, + struct xdp_frame *xdp_frame, + struct xdp_txq_info *txq) +{ + struct xdp_buff xdp; + u32 act; + + xdp.data_hard_start = xdp_frame->data - xdp_frame->headroom + - sizeof(*xdp_frame); + xdp.data = xdp_frame->data; + xdp.data_end = xdp.data + xdp_frame->len; + xdp.data_meta = xdp.data - xdp_frame->metasize; + xdp.txq = txq; + + act = bpf_prog_run_xdp(xdp_prog, &xdp); + switch (act) { + case XDP_DROP: + fallthrough; + case XDP_PASS: + break; + case XDP_TX: + fallthrough; + case XDP_REDIRECT: + fallthrough; + default: + bpf_warn_invalid_xdp_action(act); + fallthrough; + case XDP_ABORTED: + trace_xdp_exception(dev, xdp_prog, act); + act = XDP_DROP; + break; + } + + /* if not dropping frame, readjust pointers in case + * program made changes to the buffer + */ + if (act != XDP_DROP) { + if (unlikely(!update_xdp_frame(&xdp, xdp_frame))) + return XDP_DROP; + } + + return act; +} + +unsigned int do_xdp_egress(struct net_device *dev, struct xdp_frame **frames, + unsigned int count) +{ + struct bpf_prog *xdp_prog; + + xdp_prog = rcu_dereference(dev->xdp_egress_prog); + if (xdp_prog) { + struct xdp_txq_info txq = { .dev = dev }; + unsigned int i, j; + u32 act; + + for (i = 0, j = 0; i < count; i++) { + struct xdp_frame *frame = frames[i]; + + act = __xdp_egress_frame(dev, xdp_prog, frame, &txq); + if (act == XDP_DROP) { + xdp_return_frame_rx_napi(frame); + continue; + } + + frames[j] = frame; + j++; + } + count = j; + } + + return count; +} + static int netif_rx_internal(struct sk_buff *skb) { int ret; From patchwork Wed May 13 01:46:04 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Ahern X-Patchwork-Id: 1288905 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=none dis=none) header.from=kernel.org Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=default header.b=IrokJwTH; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 49MHZt5Z1Vz9sPF for ; Wed, 13 May 2020 11:46:30 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732148AbgEMBq3 (ORCPT ); Tue, 12 May 2020 21:46:29 -0400 Received: from mail.kernel.org ([198.145.29.99]:40550 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1732097AbgEMBqS (ORCPT ); Tue, 12 May 2020 21:46:18 -0400 Received: from C02YQ0RWLVCF.internal.digitalocean.com (c-73-181-34-237.hsd1.co.comcast.net [73.181.34.237]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 23CF32492C; Wed, 13 May 2020 01:46:17 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1589334377; bh=06hekma2Y+FUxTbYVja9RUDu/KKq2ZJVX+3fqheaWzQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=IrokJwTH4R556l4NlmBh5x9r36eTDxuoXOhYTRoxt117WpD68c7RO7y/rqWKmrVuj fwk5HIsFnIGOyTBAYUB/s8sXbaDUsSUT6j7B79Fah8bwLTFywZDpmTkWzVbKm287RS M0FFnhiR1kaSl7uNd0jSx49XxFztOjgW8MHVK500= From: David Ahern To: netdev@vger.kernel.org Cc: davem@davemloft.net, kuba@kernel.org, prashantbhole.linux@gmail.com, brouer@redhat.com, toke@redhat.com, daniel@iogearbox.net, john.fastabend@gmail.com, ast@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, andriin@fb.com, dsahern@gmail.com, David Ahern Subject: [PATCH v5 bpf-next 08/11] libbpf: Add egress XDP support Date: Tue, 12 May 2020 19:46:04 -0600 Message-Id: <20200513014607.40418-9-dsahern@kernel.org> X-Mailer: git-send-email 2.21.1 (Apple Git-122.3) In-Reply-To: <20200513014607.40418-1-dsahern@kernel.org> References: <20200513014607.40418-1-dsahern@kernel.org> MIME-Version: 1.0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: David Ahern New section name hint, xdp_egress, is added to set expected attach type at program load. Programs can use xdp_egress as the prefix in the SEC statement to load the program with the BPF_XDP_EGRESS attach type set. egress_prog_id is added to xdp_link_info to report the program id. Signed-off-by: David Ahern --- tools/lib/bpf/libbpf.c | 2 ++ tools/lib/bpf/libbpf.h | 1 + tools/lib/bpf/netlink.c | 6 ++++++ 3 files changed, 9 insertions(+) diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c index 3da66540b54b..5d1d513d9958 100644 --- a/tools/lib/bpf/libbpf.c +++ b/tools/lib/bpf/libbpf.c @@ -6635,6 +6635,8 @@ static const struct bpf_sec_def section_defs[] = { .expected_attach_type = BPF_TRACE_ITER, .is_attach_btf = true, .attach_fn = attach_iter), + BPF_EAPROG_SEC("xdp_egress", BPF_PROG_TYPE_XDP, + BPF_XDP_EGRESS), BPF_PROG_SEC("xdp", BPF_PROG_TYPE_XDP), BPF_PROG_SEC("perf_event", BPF_PROG_TYPE_PERF_EVENT), BPF_PROG_SEC("lwt_in", BPF_PROG_TYPE_LWT_IN), diff --git a/tools/lib/bpf/libbpf.h b/tools/lib/bpf/libbpf.h index 8ea69558f0a8..d3ded4b2da02 100644 --- a/tools/lib/bpf/libbpf.h +++ b/tools/lib/bpf/libbpf.h @@ -463,6 +463,7 @@ struct xdp_link_info { __u32 hw_prog_id; __u32 skb_prog_id; __u8 attach_mode; + __u32 egress_prog_id; }; struct bpf_xdp_set_link_opts { diff --git a/tools/lib/bpf/netlink.c b/tools/lib/bpf/netlink.c index 312f887570b2..da0b383dbd5d 100644 --- a/tools/lib/bpf/netlink.c +++ b/tools/lib/bpf/netlink.c @@ -280,6 +280,10 @@ static int get_xdp_info(void *cookie, void *msg, struct nlattr **tb) xdp_id->info.hw_prog_id = libbpf_nla_getattr_u32( xdp_tb[IFLA_XDP_HW_PROG_ID]); + if (xdp_tb[IFLA_XDP_EGRESS_PROG_ID]) + xdp_id->info.egress_prog_id = libbpf_nla_getattr_u32( + xdp_tb[IFLA_XDP_EGRESS_PROG_ID]); + return 0; } @@ -331,6 +335,8 @@ static __u32 get_xdp_id(struct xdp_link_info *info, __u32 flags) return info->hw_prog_id; if (flags & XDP_FLAGS_SKB_MODE) return info->skb_prog_id; + if (flags & XDP_FLAGS_EGRESS_MODE) + return info->egress_prog_id; return 0; } From patchwork Wed May 13 01:46:05 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Ahern X-Patchwork-Id: 1288902 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=none dis=none) header.from=kernel.org Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=default header.b=wy0DlDyc; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 49MHZk6x20z9sPF for ; Wed, 13 May 2020 11:46:22 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732131AbgEMBqV (ORCPT ); Tue, 12 May 2020 21:46:21 -0400 Received: from mail.kernel.org ([198.145.29.99]:40570 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1732110AbgEMBqT (ORCPT ); Tue, 12 May 2020 21:46:19 -0400 Received: from C02YQ0RWLVCF.internal.digitalocean.com (c-73-181-34-237.hsd1.co.comcast.net [73.181.34.237]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 1AFDB2492F; Wed, 13 May 2020 01:46:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1589334378; bh=ZoOs7c6CweO6U3Pi9sQQi0yOGPA9qLIJHCdxcUKLzOo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=wy0DlDycw2A0CO6+kAKtzrbKzjqvwvgCVa0z2jVBbGghW9ib/181eZTRkrSqZ9+Lf 0aL6BBRlIOGkjSbZ8Xen3QqUAEOGP7TiQLwrYdG4Y4lp4krDlrVquj/La90JrvWbI7 n3IALgwNZYXGdFLQ5q9/SSozxiO8fkGK7hsjfLJk= From: David Ahern To: netdev@vger.kernel.org Cc: davem@davemloft.net, kuba@kernel.org, prashantbhole.linux@gmail.com, brouer@redhat.com, toke@redhat.com, daniel@iogearbox.net, john.fastabend@gmail.com, ast@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, andriin@fb.com, dsahern@gmail.com, David Ahern Subject: [PATCH v5 bpf-next 09/11] bpftool: Add support for XDP egress Date: Tue, 12 May 2020 19:46:05 -0600 Message-Id: <20200513014607.40418-10-dsahern@kernel.org> X-Mailer: git-send-email 2.21.1 (Apple Git-122.3) In-Reply-To: <20200513014607.40418-1-dsahern@kernel.org> References: <20200513014607.40418-1-dsahern@kernel.org> MIME-Version: 1.0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: David Ahern Add xdp_egress as a program type since it requires a new attach type. This follows suit with other program type + attach type combintations and leverages the SEC name in libbpf. Add NET_ATTACH_TYPE_XDP_EGRESS and update attach_type_strings to allow a user to specify 'xdp_egress' as the attach or detach point. Update do_attach_detach_xdp to set XDP_FLAGS_EGRESS_MODE if egress is selected. Update do_xdp_dump_one to show egress program ids. Update the documentation and help output. Signed-off-by: David Ahern --- tools/bpf/bpftool/Documentation/bpftool-net.rst | 4 +++- tools/bpf/bpftool/Documentation/bpftool-prog.rst | 2 +- tools/bpf/bpftool/bash-completion/bpftool | 4 ++-- tools/bpf/bpftool/net.c | 6 +++++- tools/bpf/bpftool/netlink_dumper.c | 5 +++++ tools/bpf/bpftool/prog.c | 2 +- 6 files changed, 17 insertions(+), 6 deletions(-) diff --git a/tools/bpf/bpftool/Documentation/bpftool-net.rst b/tools/bpf/bpftool/Documentation/bpftool-net.rst index aa7450736179..e1c83188a2b5 100644 --- a/tools/bpf/bpftool/Documentation/bpftool-net.rst +++ b/tools/bpf/bpftool/Documentation/bpftool-net.rst @@ -26,7 +26,8 @@ NET COMMANDS | **bpftool** **net help** | | *PROG* := { **id** *PROG_ID* | **pinned** *FILE* | **tag** *PROG_TAG* } -| *ATTACH_TYPE* := { **xdp** | **xdpgeneric** | **xdpdrv** | **xdpoffload** } +| *ATTACH_TYPE* := +| { **xdp** | **xdpgeneric** | **xdpdrv** | **xdpoffload** | **xdp_egress** } DESCRIPTION =========== @@ -63,6 +64,7 @@ DESCRIPTION **xdpgeneric** - Generic XDP. runs at generic XDP hook when packet already enters receive path as skb; **xdpdrv** - Native XDP. runs earliest point in driver's receive path; **xdpoffload** - Offload XDP. runs directly on NIC on each packet reception; + **xdp_egress** - XDP in egress path. runs at core networking level; **bpftool** **net detach** *ATTACH_TYPE* **dev** *NAME* Detach bpf program attached to network interface *NAME* with diff --git a/tools/bpf/bpftool/Documentation/bpftool-prog.rst b/tools/bpf/bpftool/Documentation/bpftool-prog.rst index 5948e9d89c8d..739dee23c610 100644 --- a/tools/bpf/bpftool/Documentation/bpftool-prog.rst +++ b/tools/bpf/bpftool/Documentation/bpftool-prog.rst @@ -44,7 +44,7 @@ PROG COMMANDS | **cgroup/connect4** | **cgroup/connect6** | **cgroup/sendmsg4** | **cgroup/sendmsg6** | | **cgroup/recvmsg4** | **cgroup/recvmsg6** | **cgroup/sysctl** | | **cgroup/getsockopt** | **cgroup/setsockopt** | -| **struct_ops** | **fentry** | **fexit** | **freplace** +| **struct_ops** | **fentry** | **fexit** | **freplace** | **xdp_egress** | } | *ATTACH_TYPE* := { | **msg_verdict** | **stream_verdict** | **stream_parser** | **flow_dissector** diff --git a/tools/bpf/bpftool/bash-completion/bpftool b/tools/bpf/bpftool/bash-completion/bpftool index 9f0f20e73b87..92cc24a0a3cb 100644 --- a/tools/bpf/bpftool/bash-completion/bpftool +++ b/tools/bpf/bpftool/bash-completion/bpftool @@ -477,7 +477,7 @@ _bpftool() cgroup/post_bind4 cgroup/post_bind6 \ cgroup/sysctl cgroup/getsockopt \ cgroup/setsockopt struct_ops \ - fentry fexit freplace" -- \ + fentry fexit freplace xdp_egress" -- \ "$cur" ) ) return 0 ;; @@ -1022,7 +1022,7 @@ _bpftool() ;; net) local PROG_TYPE='id pinned tag name' - local ATTACH_TYPES='xdp xdpgeneric xdpdrv xdpoffload' + local ATTACH_TYPES='xdp xdpgeneric xdpdrv xdpoffload xdp_egress' case $command in show|list) [[ $prev != "$command" ]] && return 0 diff --git a/tools/bpf/bpftool/net.c b/tools/bpf/bpftool/net.c index c5e3895b7c8b..714fa075521b 100644 --- a/tools/bpf/bpftool/net.c +++ b/tools/bpf/bpftool/net.c @@ -61,6 +61,7 @@ enum net_attach_type { NET_ATTACH_TYPE_XDP_GENERIC, NET_ATTACH_TYPE_XDP_DRIVER, NET_ATTACH_TYPE_XDP_OFFLOAD, + NET_ATTACH_TYPE_XDP_EGRESS, }; static const char * const attach_type_strings[] = { @@ -68,6 +69,7 @@ static const char * const attach_type_strings[] = { [NET_ATTACH_TYPE_XDP_GENERIC] = "xdpgeneric", [NET_ATTACH_TYPE_XDP_DRIVER] = "xdpdrv", [NET_ATTACH_TYPE_XDP_OFFLOAD] = "xdpoffload", + [NET_ATTACH_TYPE_XDP_EGRESS] = "xdp_egress", }; const size_t net_attach_type_size = ARRAY_SIZE(attach_type_strings); @@ -286,6 +288,8 @@ static int do_attach_detach_xdp(int progfd, enum net_attach_type attach_type, flags |= XDP_FLAGS_DRV_MODE; if (attach_type == NET_ATTACH_TYPE_XDP_OFFLOAD) flags |= XDP_FLAGS_HW_MODE; + if (attach_type == NET_ATTACH_TYPE_XDP_EGRESS) + flags |= XDP_FLAGS_EGRESS_MODE; return bpf_set_link_xdp_fd(ifindex, progfd, flags); } @@ -464,7 +468,7 @@ static int do_help(int argc, char **argv) " %s %s help\n" "\n" " " HELP_SPEC_PROGRAM "\n" - " ATTACH_TYPE := { xdp | xdpgeneric | xdpdrv | xdpoffload }\n" + " ATTACH_TYPE := { xdp | xdpgeneric | xdpdrv | xdpoffload | xdp_egress }\n" "\n" "Note: Only xdp and tc attachments are supported now.\n" " For progs attached to cgroups, use \"bpftool cgroup\"\n" diff --git a/tools/bpf/bpftool/netlink_dumper.c b/tools/bpf/bpftool/netlink_dumper.c index 5f65140b003b..68e4909b6073 100644 --- a/tools/bpf/bpftool/netlink_dumper.c +++ b/tools/bpf/bpftool/netlink_dumper.c @@ -55,6 +55,8 @@ static int do_xdp_dump_one(struct nlattr *attr, unsigned int ifindex, xdp_dump_prog_id(tb, IFLA_XDP_SKB_PROG_ID, "generic", true); xdp_dump_prog_id(tb, IFLA_XDP_DRV_PROG_ID, "driver", true); xdp_dump_prog_id(tb, IFLA_XDP_HW_PROG_ID, "offload", true); + xdp_dump_prog_id(tb, IFLA_XDP_EGRESS_PROG_ID, + "egress", true); if (json_output) jsonw_end_array(json_wtr); } else if (mode == XDP_ATTACHED_DRV) { @@ -63,6 +65,9 @@ static int do_xdp_dump_one(struct nlattr *attr, unsigned int ifindex, xdp_dump_prog_id(tb, IFLA_XDP_PROG_ID, "generic", false); } else if (mode == XDP_ATTACHED_HW) { xdp_dump_prog_id(tb, IFLA_XDP_PROG_ID, "offload", false); + } else if (mode == XDP_ATTACHED_EGRESS_CORE) { + xdp_dump_prog_id(tb, IFLA_XDP_EGRESS_PROG_ID, + "egress", false); } NET_END_OBJECT_FINAL; diff --git a/tools/bpf/bpftool/prog.c b/tools/bpf/bpftool/prog.c index b6e5ba568f98..86e19357afc8 100644 --- a/tools/bpf/bpftool/prog.c +++ b/tools/bpf/bpftool/prog.c @@ -2014,7 +2014,7 @@ static int do_help(int argc, char **argv) " cgroup/post_bind6 | cgroup/connect4 | cgroup/connect6 |\n" " cgroup/sendmsg4 | cgroup/sendmsg6 | cgroup/recvmsg4 |\n" " cgroup/recvmsg6 | cgroup/getsockopt | cgroup/setsockopt |\n" - " struct_ops | fentry | fexit | freplace }\n" + " struct_ops | fentry | fexit | freplace | xdp_egress }\n" " ATTACH_TYPE := { msg_verdict | stream_verdict | stream_parser |\n" " flow_dissector }\n" " METRIC := { cycles | instructions | l1d_loads | llc_misses }\n" From patchwork Wed May 13 01:46:06 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Ahern X-Patchwork-Id: 1288903 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=none dis=none) header.from=kernel.org Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=default header.b=Kur1NyVN; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 49MHZm69yrz9sPF for ; Wed, 13 May 2020 11:46:24 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732135AbgEMBqX (ORCPT ); Tue, 12 May 2020 21:46:23 -0400 Received: from mail.kernel.org ([198.145.29.99]:40586 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1732114AbgEMBqU (ORCPT ); Tue, 12 May 2020 21:46:20 -0400 Received: from C02YQ0RWLVCF.internal.digitalocean.com (c-73-181-34-237.hsd1.co.comcast.net [73.181.34.237]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 10D7820714; Wed, 13 May 2020 01:46:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1589334379; bh=Vw2FCOouxN7bROtFES0en98jMRaoYmLfRNQ/ko0YO0M=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Kur1NyVN0m1RWD1Fonn8EkO3f6QZba9Ui9IqMJcZ0g7iJqjMBSmopVzXILjyzpkiJ s72ofWi+sKdB00o9LYIaQBqRHK6JmbM7h6dD0AAaBgTbs2Nh2JsL84nj6gcmXU1eYP 6Zfz+jKsDiuPsRiBHIlN49ePdEsz/ah4Yd9/CAFQ= From: David Ahern To: netdev@vger.kernel.org Cc: davem@davemloft.net, kuba@kernel.org, prashantbhole.linux@gmail.com, brouer@redhat.com, toke@redhat.com, daniel@iogearbox.net, john.fastabend@gmail.com, ast@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, andriin@fb.com, dsahern@gmail.com, David Ahern Subject: [PATCH v5 bpf-next 10/11] selftest: Add xdp_egress attach tests Date: Tue, 12 May 2020 19:46:06 -0600 Message-Id: <20200513014607.40418-11-dsahern@kernel.org> X-Mailer: git-send-email 2.21.1 (Apple Git-122.3) In-Reply-To: <20200513014607.40418-1-dsahern@kernel.org> References: <20200513014607.40418-1-dsahern@kernel.org> MIME-Version: 1.0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: David Ahern Add xdp_egress attach tests: 1. verify egress programs cannot access ingress entries in xdp context 2. verify ability to load, attach, and detach xdp egress to a device. Signed-off-by: David Ahern --- .../bpf/prog_tests/xdp_egress_attach.c | 56 +++++++++++++++++++ .../selftests/bpf/progs/test_xdp_egress.c | 12 ++++ .../bpf/progs/test_xdp_egress_fail.c | 16 ++++++ 3 files changed, 84 insertions(+) create mode 100644 tools/testing/selftests/bpf/prog_tests/xdp_egress_attach.c create mode 100644 tools/testing/selftests/bpf/progs/test_xdp_egress.c create mode 100644 tools/testing/selftests/bpf/progs/test_xdp_egress_fail.c diff --git a/tools/testing/selftests/bpf/prog_tests/xdp_egress_attach.c b/tools/testing/selftests/bpf/prog_tests/xdp_egress_attach.c new file mode 100644 index 000000000000..5253754b27de --- /dev/null +++ b/tools/testing/selftests/bpf/prog_tests/xdp_egress_attach.c @@ -0,0 +1,56 @@ +// SPDX-License-Identifier: GPL-2.0 +#include +#include + +#define IFINDEX_LO 1 + +void test_xdp_egress_attach(void) +{ + struct bpf_prog_load_attr attr = { + .prog_type = BPF_PROG_TYPE_XDP, + .expected_attach_type = BPF_XDP_EGRESS, + }; + struct bpf_prog_info info = {}; + __u32 id, len = sizeof(info); + struct bpf_object *obj; + __u32 duration = 0; + int err, fd = -1; + + /* should fail - accesses rx queue info */ + attr.file = "./test_xdp_egress_fail.o", + err = bpf_prog_load_xattr(&attr, &obj, &fd); + if (CHECK(err == 0 && fd >= 0, "xdp_egress with rx failed to load", + "load of xdp_egress with rx succeeded instead of failed")) + return; + + attr.file = "./test_xdp_egress.o", + err = bpf_prog_load_xattr(&attr, &obj, &fd); + if (CHECK_FAIL(err)) + return; + + err = bpf_obj_get_info_by_fd(fd, &info, &len); + if (CHECK_FAIL(err)) + goto out_close; + + err = bpf_set_link_xdp_fd(IFINDEX_LO, fd, XDP_FLAGS_EGRESS_MODE); + if (CHECK(err, "xdp attach", "xdp attach failed")) + goto out_close; + + err = bpf_get_link_xdp_id(IFINDEX_LO, &id, XDP_FLAGS_EGRESS_MODE); + if (CHECK(err || id != info.id, "id_check", + "loaded prog id %u != id %u, err %d", info.id, id, err)) + goto out; + +out: + err = bpf_set_link_xdp_fd(IFINDEX_LO, -1, XDP_FLAGS_EGRESS_MODE); + if (CHECK(err, "xdp detach", "xdp detach failed")) + goto out_close; + + err = bpf_get_link_xdp_id(IFINDEX_LO, &id, XDP_FLAGS_EGRESS_MODE); + if (CHECK(err || id, "id_check", + "failed to detach program %u", id)) + goto out; + +out_close: + bpf_object__close(obj); +} diff --git a/tools/testing/selftests/bpf/progs/test_xdp_egress.c b/tools/testing/selftests/bpf/progs/test_xdp_egress.c new file mode 100644 index 000000000000..0477e8537b7f --- /dev/null +++ b/tools/testing/selftests/bpf/progs/test_xdp_egress.c @@ -0,0 +1,12 @@ +// SPDX-License-Identifier: GPL-2.0 + +#include +#include + +SEC("xdp_egress") +int xdp_egress_good(struct xdp_md *ctx) +{ + __u32 idx = ctx->egress_ifindex; + + return idx == 1 ? XDP_DROP : XDP_PASS; +} diff --git a/tools/testing/selftests/bpf/progs/test_xdp_egress_fail.c b/tools/testing/selftests/bpf/progs/test_xdp_egress_fail.c new file mode 100644 index 000000000000..76b47b1d3bc3 --- /dev/null +++ b/tools/testing/selftests/bpf/progs/test_xdp_egress_fail.c @@ -0,0 +1,16 @@ +// SPDX-License-Identifier: GPL-2.0 + +#include +#include + +SEC("xdp_egress") +int xdp_egress_fail(struct xdp_md *ctx) +{ + __u32 rxq = ctx->rx_queue_index; + __u32 idx = ctx->ingress_ifindex; + + if (idx == 1) + return XDP_DROP; + + return rxq ? XDP_DROP : XDP_PASS; +} From patchwork Wed May 13 01:46:07 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Ahern X-Patchwork-Id: 1288904 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=none dis=none) header.from=kernel.org Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=default header.b=pEhJYhEA; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 49MHZp1XHJz9sPF for ; Wed, 13 May 2020 11:46:26 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732141AbgEMBqY (ORCPT ); Tue, 12 May 2020 21:46:24 -0400 Received: from mail.kernel.org ([198.145.29.99]:40606 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1732066AbgEMBqV (ORCPT ); Tue, 12 May 2020 21:46:21 -0400 Received: from C02YQ0RWLVCF.internal.digitalocean.com (c-73-181-34-237.hsd1.co.comcast.net [73.181.34.237]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 058B724931; Wed, 13 May 2020 01:46:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1589334380; bh=y6cYodOnqiQ7E1VdSwSuqVLLoLlzcN3HGsrUIcZG4y0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=pEhJYhEAC+TjiUaipMkOecJH5QhcwVWY669QBCHevKOi5I/xVSp27z2kbh/sX3faz Zr0tGCQRreVW5BBp4F2X06HbmDWxNbUj28JhhKMdY0/xEai9B6Qsp/i5b4F4hwTUjk kLRQqPZeFS46gE9aCT3VrTK1fpxXK91cFGOnhMtM= From: David Ahern To: netdev@vger.kernel.org Cc: davem@davemloft.net, kuba@kernel.org, prashantbhole.linux@gmail.com, brouer@redhat.com, toke@redhat.com, daniel@iogearbox.net, john.fastabend@gmail.com, ast@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, andriin@fb.com, dsahern@gmail.com, David Ahern Subject: [PATCH v5 bpf-next 11/11] samples/bpf: add XDP egress support to xdp1 Date: Tue, 12 May 2020 19:46:07 -0600 Message-Id: <20200513014607.40418-12-dsahern@kernel.org> X-Mailer: git-send-email 2.21.1 (Apple Git-122.3) In-Reply-To: <20200513014607.40418-1-dsahern@kernel.org> References: <20200513014607.40418-1-dsahern@kernel.org> MIME-Version: 1.0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: David Ahern xdp1 and xdp2 now accept -E flag to set XDP program in the egress path. Signed-off-by: Prashant Bhole Signed-off-by: David Ahern --- samples/bpf/xdp1_user.c | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/samples/bpf/xdp1_user.c b/samples/bpf/xdp1_user.c index c447ad9e3a1d..bb104f4d8c5e 100644 --- a/samples/bpf/xdp1_user.c +++ b/samples/bpf/xdp1_user.c @@ -73,7 +73,8 @@ static void usage(const char *prog) "OPTS:\n" " -S use skb-mode\n" " -N enforce native mode\n" - " -F force loading prog\n", + " -F force loading prog\n" + " -E egress path program\n", prog); } @@ -85,7 +86,7 @@ int main(int argc, char **argv) }; struct bpf_prog_info info = {}; __u32 info_len = sizeof(info); - const char *optstr = "FSN"; + const char *optstr = "FSNE"; int prog_fd, map_fd, opt; struct bpf_object *obj; struct bpf_map *map; @@ -103,13 +104,17 @@ int main(int argc, char **argv) case 'F': xdp_flags &= ~XDP_FLAGS_UPDATE_IF_NOEXIST; break; + case 'E': + xdp_flags |= XDP_FLAGS_EGRESS_MODE; + prog_load_attr.expected_attach_type = BPF_XDP_EGRESS; + break; default: usage(basename(argv[0])); return 1; } } - if (!(xdp_flags & XDP_FLAGS_SKB_MODE)) + if (!(xdp_flags & (XDP_FLAGS_SKB_MODE | XDP_FLAGS_EGRESS_MODE))) xdp_flags |= XDP_FLAGS_DRV_MODE; if (optind == argc) {