From patchwork Wed Jun 27 02:46:11 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Saeed Mahameed X-Patchwork-Id: 935240 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=dev.mellanox.co.il Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=dev-mellanox-co-il.20150623.gappssmtp.com header.i=@dev-mellanox-co-il.20150623.gappssmtp.com header.b="bTHeE0BW"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 41FnPR0CwWz9s0n for ; Wed, 27 Jun 2018 12:47:07 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S932234AbeF0CrE (ORCPT ); Tue, 26 Jun 2018 22:47:04 -0400 Received: from mail-pg0-f66.google.com ([74.125.83.66]:34804 "EHLO mail-pg0-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753798AbeF0Cqi (ORCPT ); Tue, 26 Jun 2018 22:46:38 -0400 Received: by mail-pg0-f66.google.com with SMTP id y1-v6so264513pgv.1 for ; Tue, 26 Jun 2018 19:46:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=dev-mellanox-co-il.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=vCbp+rgmKext4hvkU6mjPzQuIm/WD87/o8TkxiNWP8s=; b=bTHeE0BWbM7zIUO+i+RfVmYIA6E6H6Bc+H17+r04fvX6pX+1Pxs16rUvMMqbpaYu/p +PPadwJBDxO+TNFs6gKQ+LZ8fO/6IFrWovm3EtCC/5YqTlXodt/dG6SMvKYu+c4xVSdY 3Baryop8DfBFHZM+IS1DczPInUdapgWIXfFLXRS/goS47Iu/SJSg+TVt14amwkA6wjRY vZt8ndpJ/56wcFIB/7ebIEXTvXbSst+FtSgBd/5DJmtpu1lFY1nkaYGHy7ndPYmrATs8 smpNgRmGCpfxT415MhEKZy0Nzgfxl9kBg+S0OZ7GABuSnR/vIxpUlxZlMdCLNDz9fF7B tbwQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=vCbp+rgmKext4hvkU6mjPzQuIm/WD87/o8TkxiNWP8s=; b=FGksWFCrRFI4Dp1Pv9CTrrc2UhTpOt3ePbIQWJtBzFuSrwlCRB5G1jq5IsJekY+A4V U0r3wBt336B0Vcw8ojNmsTKaWSS+aIhdUWtLbo+I20LmZmisxXFNIOnYtBPIKqA8KYnA i0khVnjB2PCuNmf107xfxu2PPYfXsM4jGvN1dsfsS3R41L96lx+Ts/6HUcscdfOxlZ33 a0QyXtO+5D6SE2g8PCiztYm2hyJpAGtOKC7DD1nSD9In/FdsXrTZCrZ5dTiuW/Wym7pf E//nuBt+lWEqZpZGER6QfEmn/4XHRgfj01aniu+kFTY6N6FiIRzCQOV++9LqAXi7Fsdf LZNQ== X-Gm-Message-State: APt69E0jBZKXV4rrBcC/t24bZf+nMvxf2OjHLRzu5zD0V+qQomZYTZWh /HskeHLYCvttfHIuU9/5AcYEpA== X-Google-Smtp-Source: AAOMgpdXEcmJrHvhpGiyq6Z0fmqhxgqCo3AMbGgjK1j4sOLeKTH/ffmyuEFeJZJnCBCd/f17Wfyqig== X-Received: by 2002:a62:509a:: with SMTP id g26-v6mr4006602pfj.157.1530067597830; Tue, 26 Jun 2018 19:46:37 -0700 (PDT) Received: from stpd.hsd1.ca.comcast.net ([2601:647:4000:4586::4088]) by smtp.gmail.com with ESMTPSA id j21-v6sm3950904pfn.121.2018.06.26.19.46.35 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Tue, 26 Jun 2018 19:46:36 -0700 (PDT) From: Saeed Mahameed X-Google-Original-From: Saeed Mahameed To: Jesper Dangaard Brouer , Alexei Starovoitov , Daniel Borkmann Cc: neerav.parikh@intel.com, pjwaskiewicz@gmail.com, ttoukan.linux@gmail.com, Tariq Toukan , alexander.h.duyck@intel.com, peter.waskiewicz.jr@intel.com, Opher Reviv , Rony Efraim , netdev@vger.kernel.org, Saeed Mahameed Subject: [RFC bpf-next 2/6] net: xdp: RX meta data infrastructure Date: Tue, 26 Jun 2018 19:46:11 -0700 Message-Id: <20180627024615.17856-3-saeedm@mellanox.com> X-Mailer: git-send-email 2.17.0 In-Reply-To: <20180627024615.17856-1-saeedm@mellanox.com> References: <20180627024615.17856-1-saeedm@mellanox.com> Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org The idea from this patch is to define a well known structure for XDP meta data fields format and offset placement inside the xdp data meta buffer. For that driver will need some static information to know what to provide and where, enters struct xdp_md_info and xdp_md_info_arr: struct xdp_md_info { __u16 present:1; __u16 offset:15; /* offset from data_meta in xdp_md buffer */ }; /* XDP meta data offsets info array * present bit describes if a meta data is or will be present in xdp_md buff * offset describes where a meta data is or should be placed in xdp_md buff * * Kernel builds this array using xdp_md_info_build helper on demand. * User space builds it statically in the xdp program. */ typedef struct xdp_md_info xdp_md_info_arr[XDP_DATA_META_MAX]; Offsets in xdp_md_info_arr are always in ascending order and only for requested meta data: example : flags = XDP_FLAGS_META_HASH | XDP_FLAGS_META_VLAN; xdp_md_info_arr mdi = { [XDP_DATA_META_HASH] = {.offset = 0, .present = 1}, [XDP_DATA_META_MARK] = {.offset = 0, .present = 0}, [XDP_DATA_META_VLAN] = {.offset = sizeof(struct xdp_md_hash), .present = 1}, [XDP_DATA_META_CSUM] = {.offset = 0, .present = 0}, } i.e: hash fields will always appear first then the vlan for every xdp_md. Once requested to provide xdp meta data, device driver will use a pre-built xdp_md_info_arr which was built via xdp_md_info_build on xdp setup, xdp_md_info_arr will tell the driver what is the offset of each meta data. This patch also provides helper functions for the device drivers to store meta data into xdp_buff, and helper function for XDP programs to fetch specific xdp meta data from xdp_md buffer. Signed-off-by: Saeed Mahameed --- include/linux/netdevice.h | 1 + include/net/xdp.h | 6 ++ include/uapi/linux/bpf.h | 114 +++++++++++++++++++++++++++++++++++ include/uapi/linux/if_link.h | 16 +++-- net/core/dev.c | 21 +++++++ 5 files changed, 152 insertions(+), 6 deletions(-) diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h index fc8b6ce48a0f..172363310ade 100644 --- a/include/linux/netdevice.h +++ b/include/linux/netdevice.h @@ -838,6 +838,7 @@ struct netdev_bpf { u32 flags; struct bpf_prog *prog; struct netlink_ext_ack *extack; + xdp_md_info_arr md_info; }; /* XDP_QUERY_PROG */ struct { diff --git a/include/net/xdp.h b/include/net/xdp.h index 2deea7166a34..afe302613ae1 100644 --- a/include/net/xdp.h +++ b/include/net/xdp.h @@ -138,6 +138,12 @@ xdp_set_data_meta_invalid(struct xdp_buff *xdp) xdp->data_meta = xdp->data + 1; } +static __always_inline void +xdp_reset_data_meta(struct xdp_buff *xdp) +{ + xdp->data_meta = xdp->data_hard_start; +} + static __always_inline bool xdp_data_meta_unsupported(const struct xdp_buff *xdp) { diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h index 59b19b6a40d7..e320e7421565 100644 --- a/include/uapi/linux/bpf.h +++ b/include/uapi/linux/bpf.h @@ -2353,6 +2353,120 @@ struct xdp_md { __u32 rx_queue_index; /* rxq->queue_index */ }; +enum { + XDP_DATA_META_HASH = 0, + XDP_DATA_META_MARK, + XDP_DATA_META_VLAN, + XDP_DATA_META_CSUM_COMPLETE, + + XDP_DATA_META_MAX, +}; + +struct xdp_md_hash { + __u32 hash; + __u8 type; +} __attribute__((aligned(4))); + +struct xdp_md_mark { + __u32 mark; +} __attribute__((aligned(4))); + +struct xdp_md_vlan { + __u16 vlan; +} __attribute__((aligned(4))); + +struct xdp_md_csum { + __u16 csum; +} __attribute__((aligned(4))); + +static const __u16 xdp_md_size[XDP_DATA_META_MAX] = { + sizeof(struct xdp_md_hash), /* XDP_DATA_META_HASH */ + sizeof(struct xdp_md_mark), /* XDP_DATA_META_MARK */ + sizeof(struct xdp_md_vlan), /* XDP_DATA_META_VLAN */ + sizeof(struct xdp_md_csum), /* XDP_DATA_META_CSUM_COMPLETE */ +}; + +struct xdp_md_info { + __u16 present:1; + __u16 offset:15; /* offset from data_meta in xdp_md buffer */ +}; + +/* XDP meta data offsets info array + * present bit describes if a meta data is or will be present in xdp_md buff + * offset describes where a meta data is or should be placed in xdp_md buff + * + * Kernel builds this array using xdp_md_info_build helper on demand. + * User space builds it statically in the xdp program. + */ +typedef struct xdp_md_info xdp_md_info_arr[XDP_DATA_META_MAX]; + +static __always_inline __u8 +xdp_data_meta_present(xdp_md_info_arr mdi, int meta_data) +{ + return mdi[meta_data].present; +} + +static __always_inline void +xdp_data_meta_set_hash(xdp_md_info_arr mdi, void *data_meta, __u32 hash, __u8 htype) +{ + struct xdp_md_hash *hash_md; + + hash_md = (struct xdp_md_hash *)((char*)data_meta + mdi[XDP_DATA_META_HASH].offset); + hash_md->hash = hash; + hash_md->type = htype; +} + +static __always_inline struct xdp_md_hash * +xdp_data_meta_get_hash(xdp_md_info_arr mdi, void *data_meta) +{ + return (struct xdp_md_hash *)((char*)data_meta + mdi[XDP_DATA_META_HASH].offset); +} + +static __always_inline void +xdp_data_meta_set_mark(xdp_md_info_arr mdi, void *data_meta, __u32 mark) +{ + struct xdp_md_mark *mark_md; + + mark_md = (struct xdp_md_mark *)((char*)data_meta + mdi[XDP_DATA_META_MARK].offset); + mark_md->mark = mark; +} + +static __always_inline struct xdp_md_mark * +xdp_data_meta_get_mark(xdp_md_info_arr mdi, void *data_meta) +{ + return (struct xdp_md_mark *)((char*)data_meta + mdi[XDP_DATA_META_MARK].offset); +} + +static __always_inline void +xdp_data_meta_set_vlan(xdp_md_info_arr mdi, void *data_meta, __u16 vlan_tci) +{ + struct xdp_md_vlan *vlan_md; + + vlan_md = (struct xdp_md_vlan *)((char*)data_meta + mdi[XDP_DATA_META_VLAN].offset); + vlan_md->vlan = vlan_tci; +} + +static __always_inline struct xdp_md_vlan * +xdp_data_meta_get_vlan(xdp_md_info_arr mdi, void *data_meta) +{ + return (struct xdp_md_vlan *)((char*)data_meta + mdi[XDP_DATA_META_VLAN].offset); +} + +static __always_inline void +xdp_data_meta_set_csum_complete(xdp_md_info_arr mdi, void *data_meta, __u16 csum) +{ + struct xdp_md_csum *csum_md; + + csum_md = (struct xdp_md_csum *)((char*)data_meta + mdi[XDP_DATA_META_CSUM_COMPLETE].offset); + csum_md->csum = csum; +} + +static __always_inline struct xdp_md_csum * +xdp_data_meta_get_csum_complete(xdp_md_info_arr mdi, void *data_meta) +{ + return (struct xdp_md_csum *)((char*)data_meta + mdi[XDP_DATA_META_CSUM_COMPLETE].offset); +} + enum sk_action { SK_DROP = 0, SK_PASS, diff --git a/include/uapi/linux/if_link.h b/include/uapi/linux/if_link.h index dfb1e26bacef..5bdc07bfad35 100644 --- a/include/uapi/linux/if_link.h +++ b/include/uapi/linux/if_link.h @@ -4,6 +4,7 @@ #include #include +#include /* This struct should be in sync with struct rtnl_link_stats64 */ struct rtnl_link_stats { @@ -913,13 +914,16 @@ enum { XDP_FLAGS_HW_MODE) /* TODO : add new netlink xdp u64 meta_flags - * for meta data only + * for meta data only & build according XDP_DATA_META_* enums */ -#define XDP_FLAGS_META_HASH (1U << 16) -#define XDP_FLAGS_META_MARK (1U << 17) -#define XDP_FLAGS_META_VLAN (1U << 18) -#define XDP_FLAGS_META_CSUM_COMPLETE (1U << 19) -#define XDP_FLAGS_META_ALL (XDP_FLAGS_META_HASH | \ + +#define XDP_FLAGS_META_SHIFT (16) +#define XDP_FLAG_META(FLAG) ((1U << XDP_DATA_META_##FLAG) << XDP_FLAGS_META_SHIFT) +#define XDP_FLAGS_META_HASH XDP_FLAG_META(HASH) +#define XDP_FLAGS_META_MARK XDP_FLAG_META(MARK) +#define XDP_FLAGS_META_VLAN XDP_FLAG_META(VLAN) +#define XDP_FLAGS_META_CSUM_COMPLETE XDP_FLAG_META(CSUM_COMPLETE) +#define XDP_FLAGS_META_ALL (XDP_FLAGS_META_HASH | \ XDP_FLAGS_META_MARK | \ XDP_FLAGS_META_VLAN | \ XDP_FLAGS_META_CSUM_COMPLETE) diff --git a/net/core/dev.c b/net/core/dev.c index 8a5cc2c731ec..141dd6632b00 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -7310,6 +7310,25 @@ static bool __dev_xdp_meta_supported(struct net_device *dev, return ((xdp.meta_flags & meta_flags) == meta_flags); } +static void xdp_md_info_build(xdp_md_info_arr mdi, u32 xdp_flags_meta) +{ + u16 offset = 0; + int i; + + for (i = 0; i < XDP_DATA_META_MAX; i++) { + u32 meta_data_flag = BIT(i) << XDP_FLAGS_META_SHIFT; + + if (!(meta_data_flag & xdp_flags_meta)) { + mdi[i].present = 0; + continue; + } + + mdi[i].present = 1; + mdi[i].offset = offset; + offset += xdp_md_size[i]; + } +} + static int dev_xdp_install(struct net_device *dev, bpf_op_t bpf_op, struct netlink_ext_ack *extack, u32 flags, struct bpf_prog *prog) @@ -7325,6 +7344,8 @@ static int dev_xdp_install(struct net_device *dev, bpf_op_t bpf_op, xdp.flags = flags; xdp.prog = prog; + xdp_md_info_build(xdp.md_info, flags & XDP_FLAGS_META_ALL); + return bpf_op(dev, &xdp); }