From patchwork Wed Jun 27 02:46:10 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Saeed Mahameed X-Patchwork-Id: 935235 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=dev.mellanox.co.il Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=dev-mellanox-co-il.20150623.gappssmtp.com header.i=@dev-mellanox-co-il.20150623.gappssmtp.com header.b="vGk1vSbt"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 41FnNx6RGkz9s0n for ; Wed, 27 Jun 2018 12:46:41 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S932135AbeF0Cqk (ORCPT ); Tue, 26 Jun 2018 22:46:40 -0400 Received: from mail-pf0-f194.google.com ([209.85.192.194]:42842 "EHLO mail-pf0-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751924AbeF0Cqg (ORCPT ); Tue, 26 Jun 2018 22:46:36 -0400 Received: by mail-pf0-f194.google.com with SMTP id w7-v6so269362pfn.9 for ; Tue, 26 Jun 2018 19:46:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=dev-mellanox-co-il.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=BBxusQg1kVf0RA0T9WSUGDy/DwQ/jjg97slBT18O4Ic=; b=vGk1vSbtBWWZoroPfws46sXzhmp1HFzSnAXFD4HcfuS+FcoXz94hxoZb9IoJTOvT0B EEszdZrjv2LhGt+oVf6A3LP1DKlOlDXhuzxww9dLFhBCEINkp2V8HuufZTrbN9z3qk7H JeyChD/2B/MC7TiO5cptcEaWGhbxQosYpNey1i1Ck1KlGjSfp1pievPX41yIHT6IB3BF 7eGF1uzos04mcBWdFl+Hbv8Ir3MaHsvqDSb0wrAGNjNsMxuuQhoCsEpAjY35Xehlyrew GKeK3n1WsPPFvoqnd9ykBPVv48pzlqgoQ6D3vAR71EXYm4RR3NM6bfN8Ui51MfuqA+rw soTw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=BBxusQg1kVf0RA0T9WSUGDy/DwQ/jjg97slBT18O4Ic=; b=Dy8hFPVMO9CuP4TifTgfnekypWOVz4Zt7nEPGKD2A6YpLakn+bU7FXQtRYHhHdHp/1 LUhrU291P7jE9yXleO5hR8Olhepq6P4yJRIOtN+ZhenOJe6urmCQhinVadUmcV9n7TvY NP2eUih0mleqz7AsWrRp8L1voWcsx7hnR4sk5RAnGgBlBGXDyn8AZ6CIPH0rmHapHkcb uV+paOb0jWj/m8gChOHlkcjhJw3LM02ksSIprS+QEfA/athyebbZz/A6xDEZAvZJn+oU F84uqPcG6e5kcpotQ/mVVAgyXL+jxxJeWT7w4DeaYl+2CI74M1f3cDJKV1hpmX0PI37p dx9w== X-Gm-Message-State: APt69E3RkQYtEU4ulApvfv70rGo/3ZL7uL2Yuar96YXoiWz1HGaxKvMk J6RYTlFfgWjiUlAiXIQ2ntKQWw== X-Google-Smtp-Source: AAOMgpcha81ZflJwFBkaAFv6UAJR9woip3HBXT/yphhe7NHNG18tkHdlAuGBeiY3xzeI1cQWfM0FtA== X-Received: by 2002:a65:62d0:: with SMTP id m16-v6mr459602pgv.281.1530067595915; Tue, 26 Jun 2018 19:46:35 -0700 (PDT) Received: from stpd.hsd1.ca.comcast.net ([2601:647:4000:4586::4088]) by smtp.gmail.com with ESMTPSA id j21-v6sm3950904pfn.121.2018.06.26.19.46.33 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Tue, 26 Jun 2018 19:46:34 -0700 (PDT) From: Saeed Mahameed X-Google-Original-From: Saeed Mahameed To: Jesper Dangaard Brouer , Alexei Starovoitov , Daniel Borkmann Cc: neerav.parikh@intel.com, pjwaskiewicz@gmail.com, ttoukan.linux@gmail.com, Tariq Toukan , alexander.h.duyck@intel.com, peter.waskiewicz.jr@intel.com, Opher Reviv , Rony Efraim , netdev@vger.kernel.org, Saeed Mahameed Subject: [RFC bpf-next 1/6] net: xdp: Add support for meta data flags requests Date: Tue, 26 Jun 2018 19:46:10 -0700 Message-Id: <20180627024615.17856-2-saeedm@mellanox.com> X-Mailer: git-send-email 2.17.0 In-Reply-To: <20180627024615.17856-1-saeedm@mellanox.com> References: <20180627024615.17856-1-saeedm@mellanox.com> Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org A user space application can request to enable a specific set of meta data to be reported in every xdp buffer provided to the xdp program. When meta_data flags are required, XDP devices must respond to XDP_QUERY_META_FLAGS command with all the meta data flags the device actually supports, and the kernel will cross check them with the requested flags, in case of discrepancy the xdp install will fail. If the flags are supported, the device must guarantee to deliver all RX packets with only the meta data requested in meta data_flags on xdp_install operation. The following flags are added, and can be provided by the netlink xdp flags field. +#define XDP_FLAGS_META_HASH (1U << 16) +#define XDP_FLAGS_META_FLOW_MARK (1U << 17) +#define XDP_FLAGS_META_VLAN (1U << 18) +#define XDP_FLAGS_META_CSUM_COMPLETE (1U << 19) The format, device delivery methods and XDP program access to such meta data is discussed in a later patch. TODO: use a different flags field for XDP meta data, to make sure we have more free bits. Signed-off-by: Saeed Mahameed --- include/linux/netdevice.h | 9 +++++++++ include/uapi/linux/if_link.h | 16 +++++++++++++++- net/core/dev.c | 20 ++++++++++++++++++++ 3 files changed, 44 insertions(+), 1 deletion(-) diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h index 3ec9850c7936..fc8b6ce48a0f 100644 --- a/include/linux/netdevice.h +++ b/include/linux/netdevice.h @@ -812,6 +812,10 @@ enum bpf_netdev_command { * is equivalent to XDP_ATTACHED_DRV. */ XDP_QUERY_PROG, + /* Query Supported XDP_FLAGS_META_*, Called before new XDP program setup + * and only if meta_flags were requested by the user to validate if the + * device supports the requested flags, if not program setup will fail */ + XDP_QUERY_META_FLAGS, /* BPF program for offload callbacks, invoked at program load time. */ BPF_OFFLOAD_VERIFIER_PREP, BPF_OFFLOAD_TRANSLATE, @@ -842,6 +846,11 @@ struct netdev_bpf { /* flags with which program was installed */ u32 prog_flags; }; + /* XDP_QUERY_META_FLAGS */ + struct { + /* TODO u64 */ + u32 meta_flags; + }; /* BPF_OFFLOAD_VERIFIER_PREP */ struct { struct bpf_prog *prog; diff --git a/include/uapi/linux/if_link.h b/include/uapi/linux/if_link.h index cf01b6824244..dfb1e26bacef 100644 --- a/include/uapi/linux/if_link.h +++ b/include/uapi/linux/if_link.h @@ -911,8 +911,22 @@ enum { #define XDP_FLAGS_MODES (XDP_FLAGS_SKB_MODE | \ XDP_FLAGS_DRV_MODE | \ XDP_FLAGS_HW_MODE) + +/* TODO : add new netlink xdp u64 meta_flags + * for meta data only + */ +#define XDP_FLAGS_META_HASH (1U << 16) +#define XDP_FLAGS_META_MARK (1U << 17) +#define XDP_FLAGS_META_VLAN (1U << 18) +#define XDP_FLAGS_META_CSUM_COMPLETE (1U << 19) +#define XDP_FLAGS_META_ALL (XDP_FLAGS_META_HASH | \ + XDP_FLAGS_META_MARK | \ + XDP_FLAGS_META_VLAN | \ + XDP_FLAGS_META_CSUM_COMPLETE) + #define XDP_FLAGS_MASK (XDP_FLAGS_UPDATE_IF_NOEXIST | \ - XDP_FLAGS_MODES) + XDP_FLAGS_MODES | \ + XDP_FLAGS_META_ALL) /* These are stored into IFLA_XDP_ATTACHED on dump. */ enum { diff --git a/net/core/dev.c b/net/core/dev.c index a5aa1c7444e6..8a5cc2c731ec 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -7295,6 +7295,21 @@ static u8 __dev_xdp_attached(struct net_device *dev, bpf_op_t bpf_op) return xdp.prog_attached; } +static bool __dev_xdp_meta_supported(struct net_device *dev, + bpf_op_t bpf_op, u32 meta_flags) +{ + struct netdev_bpf xdp = {}; + + /* Backward compatible, all devices support no meta_flags */ + if (!meta_flags) + return true; + + xdp.command = XDP_QUERY_META_FLAGS; + bpf_op(dev, &xdp); + + return ((xdp.meta_flags & meta_flags) == meta_flags); +} + static int dev_xdp_install(struct net_device *dev, bpf_op_t bpf_op, struct netlink_ext_ack *extack, u32 flags, struct bpf_prog *prog) @@ -7362,12 +7377,17 @@ int dev_change_xdp_fd(struct net_device *dev, struct netlink_ext_ack *extack, bpf_chk = generic_xdp_install; if (fd >= 0) { + u32 meta_flags = (flags & XDP_FLAGS_META_ALL); + if (bpf_chk && __dev_xdp_attached(dev, bpf_chk)) return -EEXIST; if ((flags & XDP_FLAGS_UPDATE_IF_NOEXIST) && __dev_xdp_attached(dev, bpf_op)) return -EBUSY; + if (!__dev_xdp_meta_supported(dev, bpf_op, meta_flags)) + return -EINVAL; + prog = bpf_prog_get_type_dev(fd, BPF_PROG_TYPE_XDP, bpf_op == ops->ndo_bpf); if (IS_ERR(prog)) From patchwork Wed Jun 27 02:46:11 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Saeed Mahameed X-Patchwork-Id: 935240 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=dev.mellanox.co.il Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=dev-mellanox-co-il.20150623.gappssmtp.com header.i=@dev-mellanox-co-il.20150623.gappssmtp.com header.b="bTHeE0BW"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 41FnPR0CwWz9s0n for ; Wed, 27 Jun 2018 12:47:07 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S932234AbeF0CrE (ORCPT ); Tue, 26 Jun 2018 22:47:04 -0400 Received: from mail-pg0-f66.google.com ([74.125.83.66]:34804 "EHLO mail-pg0-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753798AbeF0Cqi (ORCPT ); Tue, 26 Jun 2018 22:46:38 -0400 Received: by mail-pg0-f66.google.com with SMTP id y1-v6so264513pgv.1 for ; Tue, 26 Jun 2018 19:46:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=dev-mellanox-co-il.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=vCbp+rgmKext4hvkU6mjPzQuIm/WD87/o8TkxiNWP8s=; b=bTHeE0BWbM7zIUO+i+RfVmYIA6E6H6Bc+H17+r04fvX6pX+1Pxs16rUvMMqbpaYu/p +PPadwJBDxO+TNFs6gKQ+LZ8fO/6IFrWovm3EtCC/5YqTlXodt/dG6SMvKYu+c4xVSdY 3Baryop8DfBFHZM+IS1DczPInUdapgWIXfFLXRS/goS47Iu/SJSg+TVt14amwkA6wjRY vZt8ndpJ/56wcFIB/7ebIEXTvXbSst+FtSgBd/5DJmtpu1lFY1nkaYGHy7ndPYmrATs8 smpNgRmGCpfxT415MhEKZy0Nzgfxl9kBg+S0OZ7GABuSnR/vIxpUlxZlMdCLNDz9fF7B tbwQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=vCbp+rgmKext4hvkU6mjPzQuIm/WD87/o8TkxiNWP8s=; b=FGksWFCrRFI4Dp1Pv9CTrrc2UhTpOt3ePbIQWJtBzFuSrwlCRB5G1jq5IsJekY+A4V U0r3wBt336B0Vcw8ojNmsTKaWSS+aIhdUWtLbo+I20LmZmisxXFNIOnYtBPIKqA8KYnA i0khVnjB2PCuNmf107xfxu2PPYfXsM4jGvN1dsfsS3R41L96lx+Ts/6HUcscdfOxlZ33 a0QyXtO+5D6SE2g8PCiztYm2hyJpAGtOKC7DD1nSD9In/FdsXrTZCrZ5dTiuW/Wym7pf E//nuBt+lWEqZpZGER6QfEmn/4XHRgfj01aniu+kFTY6N6FiIRzCQOV++9LqAXi7Fsdf LZNQ== X-Gm-Message-State: APt69E0jBZKXV4rrBcC/t24bZf+nMvxf2OjHLRzu5zD0V+qQomZYTZWh /HskeHLYCvttfHIuU9/5AcYEpA== X-Google-Smtp-Source: AAOMgpdXEcmJrHvhpGiyq6Z0fmqhxgqCo3AMbGgjK1j4sOLeKTH/ffmyuEFeJZJnCBCd/f17Wfyqig== X-Received: by 2002:a62:509a:: with SMTP id g26-v6mr4006602pfj.157.1530067597830; Tue, 26 Jun 2018 19:46:37 -0700 (PDT) Received: from stpd.hsd1.ca.comcast.net ([2601:647:4000:4586::4088]) by smtp.gmail.com with ESMTPSA id j21-v6sm3950904pfn.121.2018.06.26.19.46.35 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Tue, 26 Jun 2018 19:46:36 -0700 (PDT) From: Saeed Mahameed X-Google-Original-From: Saeed Mahameed To: Jesper Dangaard Brouer , Alexei Starovoitov , Daniel Borkmann Cc: neerav.parikh@intel.com, pjwaskiewicz@gmail.com, ttoukan.linux@gmail.com, Tariq Toukan , alexander.h.duyck@intel.com, peter.waskiewicz.jr@intel.com, Opher Reviv , Rony Efraim , netdev@vger.kernel.org, Saeed Mahameed Subject: [RFC bpf-next 2/6] net: xdp: RX meta data infrastructure Date: Tue, 26 Jun 2018 19:46:11 -0700 Message-Id: <20180627024615.17856-3-saeedm@mellanox.com> X-Mailer: git-send-email 2.17.0 In-Reply-To: <20180627024615.17856-1-saeedm@mellanox.com> References: <20180627024615.17856-1-saeedm@mellanox.com> Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org The idea from this patch is to define a well known structure for XDP meta data fields format and offset placement inside the xdp data meta buffer. For that driver will need some static information to know what to provide and where, enters struct xdp_md_info and xdp_md_info_arr: struct xdp_md_info { __u16 present:1; __u16 offset:15; /* offset from data_meta in xdp_md buffer */ }; /* XDP meta data offsets info array * present bit describes if a meta data is or will be present in xdp_md buff * offset describes where a meta data is or should be placed in xdp_md buff * * Kernel builds this array using xdp_md_info_build helper on demand. * User space builds it statically in the xdp program. */ typedef struct xdp_md_info xdp_md_info_arr[XDP_DATA_META_MAX]; Offsets in xdp_md_info_arr are always in ascending order and only for requested meta data: example : flags = XDP_FLAGS_META_HASH | XDP_FLAGS_META_VLAN; xdp_md_info_arr mdi = { [XDP_DATA_META_HASH] = {.offset = 0, .present = 1}, [XDP_DATA_META_MARK] = {.offset = 0, .present = 0}, [XDP_DATA_META_VLAN] = {.offset = sizeof(struct xdp_md_hash), .present = 1}, [XDP_DATA_META_CSUM] = {.offset = 0, .present = 0}, } i.e: hash fields will always appear first then the vlan for every xdp_md. Once requested to provide xdp meta data, device driver will use a pre-built xdp_md_info_arr which was built via xdp_md_info_build on xdp setup, xdp_md_info_arr will tell the driver what is the offset of each meta data. This patch also provides helper functions for the device drivers to store meta data into xdp_buff, and helper function for XDP programs to fetch specific xdp meta data from xdp_md buffer. Signed-off-by: Saeed Mahameed --- include/linux/netdevice.h | 1 + include/net/xdp.h | 6 ++ include/uapi/linux/bpf.h | 114 +++++++++++++++++++++++++++++++++++ include/uapi/linux/if_link.h | 16 +++-- net/core/dev.c | 21 +++++++ 5 files changed, 152 insertions(+), 6 deletions(-) diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h index fc8b6ce48a0f..172363310ade 100644 --- a/include/linux/netdevice.h +++ b/include/linux/netdevice.h @@ -838,6 +838,7 @@ struct netdev_bpf { u32 flags; struct bpf_prog *prog; struct netlink_ext_ack *extack; + xdp_md_info_arr md_info; }; /* XDP_QUERY_PROG */ struct { diff --git a/include/net/xdp.h b/include/net/xdp.h index 2deea7166a34..afe302613ae1 100644 --- a/include/net/xdp.h +++ b/include/net/xdp.h @@ -138,6 +138,12 @@ xdp_set_data_meta_invalid(struct xdp_buff *xdp) xdp->data_meta = xdp->data + 1; } +static __always_inline void +xdp_reset_data_meta(struct xdp_buff *xdp) +{ + xdp->data_meta = xdp->data_hard_start; +} + static __always_inline bool xdp_data_meta_unsupported(const struct xdp_buff *xdp) { diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h index 59b19b6a40d7..e320e7421565 100644 --- a/include/uapi/linux/bpf.h +++ b/include/uapi/linux/bpf.h @@ -2353,6 +2353,120 @@ struct xdp_md { __u32 rx_queue_index; /* rxq->queue_index */ }; +enum { + XDP_DATA_META_HASH = 0, + XDP_DATA_META_MARK, + XDP_DATA_META_VLAN, + XDP_DATA_META_CSUM_COMPLETE, + + XDP_DATA_META_MAX, +}; + +struct xdp_md_hash { + __u32 hash; + __u8 type; +} __attribute__((aligned(4))); + +struct xdp_md_mark { + __u32 mark; +} __attribute__((aligned(4))); + +struct xdp_md_vlan { + __u16 vlan; +} __attribute__((aligned(4))); + +struct xdp_md_csum { + __u16 csum; +} __attribute__((aligned(4))); + +static const __u16 xdp_md_size[XDP_DATA_META_MAX] = { + sizeof(struct xdp_md_hash), /* XDP_DATA_META_HASH */ + sizeof(struct xdp_md_mark), /* XDP_DATA_META_MARK */ + sizeof(struct xdp_md_vlan), /* XDP_DATA_META_VLAN */ + sizeof(struct xdp_md_csum), /* XDP_DATA_META_CSUM_COMPLETE */ +}; + +struct xdp_md_info { + __u16 present:1; + __u16 offset:15; /* offset from data_meta in xdp_md buffer */ +}; + +/* XDP meta data offsets info array + * present bit describes if a meta data is or will be present in xdp_md buff + * offset describes where a meta data is or should be placed in xdp_md buff + * + * Kernel builds this array using xdp_md_info_build helper on demand. + * User space builds it statically in the xdp program. + */ +typedef struct xdp_md_info xdp_md_info_arr[XDP_DATA_META_MAX]; + +static __always_inline __u8 +xdp_data_meta_present(xdp_md_info_arr mdi, int meta_data) +{ + return mdi[meta_data].present; +} + +static __always_inline void +xdp_data_meta_set_hash(xdp_md_info_arr mdi, void *data_meta, __u32 hash, __u8 htype) +{ + struct xdp_md_hash *hash_md; + + hash_md = (struct xdp_md_hash *)((char*)data_meta + mdi[XDP_DATA_META_HASH].offset); + hash_md->hash = hash; + hash_md->type = htype; +} + +static __always_inline struct xdp_md_hash * +xdp_data_meta_get_hash(xdp_md_info_arr mdi, void *data_meta) +{ + return (struct xdp_md_hash *)((char*)data_meta + mdi[XDP_DATA_META_HASH].offset); +} + +static __always_inline void +xdp_data_meta_set_mark(xdp_md_info_arr mdi, void *data_meta, __u32 mark) +{ + struct xdp_md_mark *mark_md; + + mark_md = (struct xdp_md_mark *)((char*)data_meta + mdi[XDP_DATA_META_MARK].offset); + mark_md->mark = mark; +} + +static __always_inline struct xdp_md_mark * +xdp_data_meta_get_mark(xdp_md_info_arr mdi, void *data_meta) +{ + return (struct xdp_md_mark *)((char*)data_meta + mdi[XDP_DATA_META_MARK].offset); +} + +static __always_inline void +xdp_data_meta_set_vlan(xdp_md_info_arr mdi, void *data_meta, __u16 vlan_tci) +{ + struct xdp_md_vlan *vlan_md; + + vlan_md = (struct xdp_md_vlan *)((char*)data_meta + mdi[XDP_DATA_META_VLAN].offset); + vlan_md->vlan = vlan_tci; +} + +static __always_inline struct xdp_md_vlan * +xdp_data_meta_get_vlan(xdp_md_info_arr mdi, void *data_meta) +{ + return (struct xdp_md_vlan *)((char*)data_meta + mdi[XDP_DATA_META_VLAN].offset); +} + +static __always_inline void +xdp_data_meta_set_csum_complete(xdp_md_info_arr mdi, void *data_meta, __u16 csum) +{ + struct xdp_md_csum *csum_md; + + csum_md = (struct xdp_md_csum *)((char*)data_meta + mdi[XDP_DATA_META_CSUM_COMPLETE].offset); + csum_md->csum = csum; +} + +static __always_inline struct xdp_md_csum * +xdp_data_meta_get_csum_complete(xdp_md_info_arr mdi, void *data_meta) +{ + return (struct xdp_md_csum *)((char*)data_meta + mdi[XDP_DATA_META_CSUM_COMPLETE].offset); +} + enum sk_action { SK_DROP = 0, SK_PASS, diff --git a/include/uapi/linux/if_link.h b/include/uapi/linux/if_link.h index dfb1e26bacef..5bdc07bfad35 100644 --- a/include/uapi/linux/if_link.h +++ b/include/uapi/linux/if_link.h @@ -4,6 +4,7 @@ #include #include +#include /* This struct should be in sync with struct rtnl_link_stats64 */ struct rtnl_link_stats { @@ -913,13 +914,16 @@ enum { XDP_FLAGS_HW_MODE) /* TODO : add new netlink xdp u64 meta_flags - * for meta data only + * for meta data only & build according XDP_DATA_META_* enums */ -#define XDP_FLAGS_META_HASH (1U << 16) -#define XDP_FLAGS_META_MARK (1U << 17) -#define XDP_FLAGS_META_VLAN (1U << 18) -#define XDP_FLAGS_META_CSUM_COMPLETE (1U << 19) -#define XDP_FLAGS_META_ALL (XDP_FLAGS_META_HASH | \ + +#define XDP_FLAGS_META_SHIFT (16) +#define XDP_FLAG_META(FLAG) ((1U << XDP_DATA_META_##FLAG) << XDP_FLAGS_META_SHIFT) +#define XDP_FLAGS_META_HASH XDP_FLAG_META(HASH) +#define XDP_FLAGS_META_MARK XDP_FLAG_META(MARK) +#define XDP_FLAGS_META_VLAN XDP_FLAG_META(VLAN) +#define XDP_FLAGS_META_CSUM_COMPLETE XDP_FLAG_META(CSUM_COMPLETE) +#define XDP_FLAGS_META_ALL (XDP_FLAGS_META_HASH | \ XDP_FLAGS_META_MARK | \ XDP_FLAGS_META_VLAN | \ XDP_FLAGS_META_CSUM_COMPLETE) diff --git a/net/core/dev.c b/net/core/dev.c index 8a5cc2c731ec..141dd6632b00 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -7310,6 +7310,25 @@ static bool __dev_xdp_meta_supported(struct net_device *dev, return ((xdp.meta_flags & meta_flags) == meta_flags); } +static void xdp_md_info_build(xdp_md_info_arr mdi, u32 xdp_flags_meta) +{ + u16 offset = 0; + int i; + + for (i = 0; i < XDP_DATA_META_MAX; i++) { + u32 meta_data_flag = BIT(i) << XDP_FLAGS_META_SHIFT; + + if (!(meta_data_flag & xdp_flags_meta)) { + mdi[i].present = 0; + continue; + } + + mdi[i].present = 1; + mdi[i].offset = offset; + offset += xdp_md_size[i]; + } +} + static int dev_xdp_install(struct net_device *dev, bpf_op_t bpf_op, struct netlink_ext_ack *extack, u32 flags, struct bpf_prog *prog) @@ -7325,6 +7344,8 @@ static int dev_xdp_install(struct net_device *dev, bpf_op_t bpf_op, xdp.flags = flags; xdp.prog = prog; + xdp_md_info_build(xdp.md_info, flags & XDP_FLAGS_META_ALL); + return bpf_op(dev, &xdp); } From patchwork Wed Jun 27 02:46:12 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Saeed Mahameed X-Patchwork-Id: 935236 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=dev.mellanox.co.il Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=dev-mellanox-co-il.20150623.gappssmtp.com header.i=@dev-mellanox-co-il.20150623.gappssmtp.com header.b="M4iqgXI7"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 41FnP16NRRz9s0n for ; Wed, 27 Jun 2018 12:46:45 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S932182AbeF0Cqn (ORCPT ); Tue, 26 Jun 2018 22:46:43 -0400 Received: from mail-pf0-f193.google.com ([209.85.192.193]:33504 "EHLO mail-pf0-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S932147AbeF0Cqk (ORCPT ); Tue, 26 Jun 2018 22:46:40 -0400 Received: by mail-pf0-f193.google.com with SMTP id b17-v6so277888pfi.0 for ; Tue, 26 Jun 2018 19:46:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=dev-mellanox-co-il.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=qe4tY7xllpKH8ufoMybtXAfLGU/2Rk9saspbN/RcKEo=; b=M4iqgXI7937RJQwvxINY8JC9QdMzSHMlIeHMAmWUdy5byiQ333Cxl3lcv7jY4hdsX2 1jcjU4bflnrnbAg2M4zhxAIlGV4ho8/8NhxH4hIh33Th6XppNAhXnc8DjIHAYBPlNTq1 Myxk5vspOi+bowe3NeIbTHAc5qGxcaKHhXG9l1ns7tnTuhzLnUupHx8S/T8lggUAvdSE QHU2WpaFyHZ+KuFfIZ7u8RxZ0VjnOwLG9UsIWEnITTSdqPF7mm5r9EAL6QtQUUFy2t1e R7edcc1/Ykl2oDvKRn2k0U/JxohxzYU84lgDnS6++qM6NoquA9bR7hkW0Q+pmFykUMro i4Tg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=qe4tY7xllpKH8ufoMybtXAfLGU/2Rk9saspbN/RcKEo=; b=LnY8+UEbL3hd5FKRsdfstqiH2La4oDnl0/Ya87tsc/YOq8elLeNsa2DOTupAse8PlF gw1kpxGPcy8dcDllzwa7+RvFQJDMmdl8mvGwGJ3GeMkZL9prZmEMaADgBa41iikx1sWv odgZyFerr5RY40YBajdzkAAxxHH7Mg9n4CZwYy7NziGwlvfjoLrbsOpdVsrfGRdhWepf JwJYiuB8sxL1x92wbk0fOhEM4FyrwolP+oTs1sdT4XSFeu8JcWE/+3r5VVw33kYWBfy3 2am3ZqJZTIZiXGNTtxlh+LL8Jd2OQLi5bxq0AyVre22lN3CcVh7GHUSw5j+NbrumeCIF I/5w== X-Gm-Message-State: APt69E3OXXXXw6QwF5hefm9XlqSxwvh3SAvCN7cFhWk5CKwPR4L8RD3N jie1F5sWMGangI5unxAogBZEbA== X-Google-Smtp-Source: AAOMgpd1m8/Rjy5CjbfM2AuEMnhOgpp3XKX95/NAnBbpxZPkjgiKxL0ZIvkAzR//xVOlpxigSmPvHw== X-Received: by 2002:a62:d09:: with SMTP id v9-v6mr3904239pfi.163.1530067599810; Tue, 26 Jun 2018 19:46:39 -0700 (PDT) Received: from stpd.hsd1.ca.comcast.net ([2601:647:4000:4586::4088]) by smtp.gmail.com with ESMTPSA id j21-v6sm3950904pfn.121.2018.06.26.19.46.38 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Tue, 26 Jun 2018 19:46:38 -0700 (PDT) From: Saeed Mahameed X-Google-Original-From: Saeed Mahameed To: Jesper Dangaard Brouer , Alexei Starovoitov , Daniel Borkmann Cc: neerav.parikh@intel.com, pjwaskiewicz@gmail.com, ttoukan.linux@gmail.com, Tariq Toukan , alexander.h.duyck@intel.com, peter.waskiewicz.jr@intel.com, Opher Reviv , Rony Efraim , netdev@vger.kernel.org, Saeed Mahameed Subject: [RFC bpf-next 3/6] net/mlx5e: Store xdp flags and meta data info Date: Tue, 26 Jun 2018 19:46:12 -0700 Message-Id: <20180627024615.17856-4-saeedm@mellanox.com> X-Mailer: git-send-email 2.17.0 In-Reply-To: <20180627024615.17856-1-saeedm@mellanox.com> References: <20180627024615.17856-1-saeedm@mellanox.com> Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Make xdp flags and meta data info avaiable to RQs data path, to enable xdp meta data offloads in next two patches. Signed-off-by: Saeed Mahameed --- drivers/net/ethernet/mellanox/mlx5/core/en.h | 10 +++- .../net/ethernet/mellanox/mlx5/core/en_main.c | 52 +++++++++++-------- .../net/ethernet/mellanox/mlx5/core/en_rx.c | 2 +- 3 files changed, 40 insertions(+), 24 deletions(-) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en.h b/drivers/net/ethernet/mellanox/mlx5/core/en.h index eb9eb7aa953a..5893acfae307 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/en.h @@ -236,6 +236,12 @@ enum mlx5e_priv_flag { #define MLX5E_MAX_BW_ALLOC 100 /* Max percentage of BW allocation */ #endif +struct mlx5e_xdp_info { + struct bpf_prog *prog; + u32 flags; + xdp_md_info_arr md_info; +}; + struct mlx5e_params { u8 log_sq_size; u8 rq_wq_type; @@ -257,7 +263,7 @@ struct mlx5e_params { bool tx_dim_enabled; u32 lro_timeout; u32 pflags; - struct bpf_prog *xdp_prog; + struct mlx5e_xdp_info xdp; unsigned int sw_mtu; int hard_mtu; }; @@ -566,7 +572,7 @@ struct mlx5e_rq { struct net_dim dim; /* Dynamic Interrupt Moderation */ /* XDP */ - struct bpf_prog *xdp_prog; + struct mlx5e_xdp_info xdp; unsigned int hw_mtu; struct mlx5e_xdpsq xdpsq; DECLARE_BITMAP(flags, 8); diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c index 56c1b6f5593e..8debae6b9cab 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c @@ -96,7 +96,7 @@ bool mlx5e_check_fragmented_striding_rq_cap(struct mlx5_core_dev *mdev) static u32 mlx5e_rx_get_linear_frag_sz(struct mlx5e_params *params) { - if (!params->xdp_prog) { + if (!params->xdp.prog) { u16 hw_mtu = MLX5E_SW2HW_MTU(params, params->sw_mtu); u16 rq_headroom = MLX5_RX_HEADROOM + NET_IP_ALIGN; @@ -170,7 +170,7 @@ static u8 mlx5e_mpwqe_get_log_num_strides(struct mlx5_core_dev *mdev, static u16 mlx5e_get_rq_headroom(struct mlx5_core_dev *mdev, struct mlx5e_params *params) { - u16 linear_rq_headroom = params->xdp_prog ? + u16 linear_rq_headroom = params->xdp.prog ? XDP_PACKET_HEADROOM : MLX5_RX_HEADROOM; bool is_linear_skb; @@ -205,7 +205,7 @@ bool mlx5e_striding_rq_possible(struct mlx5_core_dev *mdev, { return mlx5e_check_fragmented_striding_rq_cap(mdev) && !MLX5_IPSEC_DEV(mdev) && - !(params->xdp_prog && !mlx5e_rx_mpwqe_is_linear_skb(mdev, params)); + !(params->xdp.prog && !mlx5e_rx_mpwqe_is_linear_skb(mdev, params)); } void mlx5e_set_rq_type(struct mlx5_core_dev *mdev, struct mlx5e_params *params) @@ -489,11 +489,12 @@ static int mlx5e_alloc_rq(struct mlx5e_channel *c, rq->mdev = mdev; rq->hw_mtu = MLX5E_SW2HW_MTU(params, params->sw_mtu); rq->stats = &c->priv->channel_stats[c->ix].rq; + rq->xdp = params->xdp; - rq->xdp_prog = params->xdp_prog ? bpf_prog_inc(params->xdp_prog) : NULL; - if (IS_ERR(rq->xdp_prog)) { - err = PTR_ERR(rq->xdp_prog); - rq->xdp_prog = NULL; + rq->xdp.prog = params->xdp.prog ? bpf_prog_inc(params->xdp.prog) : NULL; + if (IS_ERR(rq->xdp.prog)) { + err = PTR_ERR(rq->xdp.prog); + rq->xdp.prog = NULL; goto err_rq_wq_destroy; } @@ -501,7 +502,7 @@ static int mlx5e_alloc_rq(struct mlx5e_channel *c, if (err < 0) goto err_rq_wq_destroy; - rq->buff.map_dir = rq->xdp_prog ? DMA_BIDIRECTIONAL : DMA_FROM_DEVICE; + rq->buff.map_dir = rq->xdp.prog ? DMA_BIDIRECTIONAL : DMA_FROM_DEVICE; rq->buff.headroom = mlx5e_get_rq_headroom(mdev, params); pool_size = 1 << params->log_rq_mtu_frames; @@ -679,8 +680,8 @@ static int mlx5e_alloc_rq(struct mlx5e_channel *c, } err_rq_wq_destroy: - if (rq->xdp_prog) - bpf_prog_put(rq->xdp_prog); + if (rq->xdp.prog) + bpf_prog_put(rq->xdp.prog); xdp_rxq_info_unreg(&rq->xdp_rxq); if (rq->page_pool) page_pool_destroy(rq->page_pool); @@ -693,8 +694,8 @@ static void mlx5e_free_rq(struct mlx5e_rq *rq) { int i; - if (rq->xdp_prog) - bpf_prog_put(rq->xdp_prog); + if (rq->xdp.prog) + bpf_prog_put(rq->xdp.prog); xdp_rxq_info_unreg(&rq->xdp_rxq); if (rq->page_pool) @@ -1906,7 +1907,7 @@ static int mlx5e_open_channel(struct mlx5e_priv *priv, int ix, c->netdev = priv->netdev; c->mkey_be = cpu_to_be32(priv->mdev->mlx5e_res.mkey.key); c->num_tc = params->num_tc; - c->xdp = !!params->xdp_prog; + c->xdp = !!params->xdp.prog; c->stats = &priv->channel_stats[ix].ch; mlx5_vector2eqn(priv->mdev, ix, &eqn, &irq); @@ -4090,12 +4091,12 @@ static void mlx5e_tx_timeout(struct net_device *dev) queue_work(priv->wq, &priv->tx_timeout_work); } -static int mlx5e_xdp_set(struct net_device *netdev, struct bpf_prog *prog) +static int mlx5e_xdp_set(struct net_device *netdev, struct mlx5e_xdp_info *xdp) { struct mlx5e_priv *priv = netdev_priv(netdev); - struct bpf_prog *old_prog; - int err = 0; + struct bpf_prog *old_prog, *prog = xdp->prog; bool reset, was_opened; + int err = 0; int i; mutex_lock(&priv->state_lock); @@ -4114,7 +4115,7 @@ static int mlx5e_xdp_set(struct net_device *netdev, struct bpf_prog *prog) was_opened = test_bit(MLX5E_STATE_OPENED, &priv->state); /* no need for full reset when exchanging programs */ - reset = (!priv->channels.params.xdp_prog || !prog); + reset = (!priv->channels.params.xdp.prog || !prog); if (was_opened && reset) mlx5e_close_locked(netdev); @@ -4132,10 +4133,12 @@ static int mlx5e_xdp_set(struct net_device *netdev, struct bpf_prog *prog) /* exchange programs, extra prog reference we got from caller * as long as we don't fail from this point onwards. */ - old_prog = xchg(&priv->channels.params.xdp_prog, prog); + old_prog = xchg(&priv->channels.params.xdp.prog, prog); if (old_prog) bpf_prog_put(old_prog); + priv->channels.params.xdp = *xdp; + if (reset) /* change RQ type according to priv->xdp_prog */ mlx5e_set_rq_type(priv->mdev, &priv->channels.params); @@ -4155,7 +4158,8 @@ static int mlx5e_xdp_set(struct net_device *netdev, struct bpf_prog *prog) napi_synchronize(&c->napi); /* prevent mlx5e_poll_rx_cq from accessing rq->xdp_prog */ - old_prog = xchg(&c->rq.xdp_prog, prog); + old_prog = xchg(&c->rq.xdp.prog, prog); + c->rq.xdp = *xdp; set_bit(MLX5E_RQ_STATE_ENABLED, &c->rq.state); /* napi_schedule in case we have missed anything */ @@ -4177,7 +4181,7 @@ static u32 mlx5e_xdp_query(struct net_device *dev) u32 prog_id = 0; mutex_lock(&priv->state_lock); - xdp_prog = priv->channels.params.xdp_prog; + xdp_prog = priv->channels.params.xdp.prog; if (xdp_prog) prog_id = xdp_prog->aux->id; mutex_unlock(&priv->state_lock); @@ -4187,9 +4191,15 @@ static u32 mlx5e_xdp_query(struct net_device *dev) static int mlx5e_xdp(struct net_device *dev, struct netdev_bpf *xdp) { + struct mlx5e_xdp_info xdp_info; + switch (xdp->command) { case XDP_SETUP_PROG: - return mlx5e_xdp_set(dev, xdp->prog); + xdp_info.prog = xdp->prog; + xdp_info.flags = xdp->flags; + memcpy(xdp_info.md_info, xdp->md_info, sizeof(xdp->md_info)); + + return mlx5e_xdp_set(dev, &xdp_info); case XDP_QUERY_PROG: xdp->prog_id = mlx5e_xdp_query(dev); xdp->prog_attached = !!xdp->prog_id; diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c b/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c index d3a1dd20e41d..d12577c17011 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c @@ -925,7 +925,7 @@ static inline bool mlx5e_xdp_handle(struct mlx5e_rq *rq, struct mlx5e_dma_info *di, void *va, u16 *rx_headroom, u32 *len) { - struct bpf_prog *prog = READ_ONCE(rq->xdp_prog); + struct bpf_prog *prog = READ_ONCE(rq->xdp.prog); struct xdp_buff xdp; u32 act; int err; From patchwork Wed Jun 27 02:46:13 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Saeed Mahameed X-Patchwork-Id: 935238 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=dev.mellanox.co.il Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=dev-mellanox-co-il.20150623.gappssmtp.com header.i=@dev-mellanox-co-il.20150623.gappssmtp.com header.b="SzA5HJlq"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 41FnPD4NmGz9s0n for ; Wed, 27 Jun 2018 12:46:56 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S932216AbeF0Cqz (ORCPT ); Tue, 26 Jun 2018 22:46:55 -0400 Received: from mail-pg0-f65.google.com ([74.125.83.65]:37606 "EHLO mail-pg0-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S932150AbeF0Cqm (ORCPT ); Tue, 26 Jun 2018 22:46:42 -0400 Received: by mail-pg0-f65.google.com with SMTP id o11-v6so259314pgv.4 for ; Tue, 26 Jun 2018 19:46:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=dev-mellanox-co-il.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=gpgcdf3GumARXnbXdcpVBbNO+CVebKsA+RWgJfCxS+M=; b=SzA5HJlqqq+4ZT6z10ulLLuJUsctbotsPznkGkEyNttyQvwsOOQUFA4bwI7XIrjsTK uBzO6+UPQsDOb2u7vCHse3FnFf18Ss1FkjJsmBbi8+dA4SOOjQf4k4tV70vVxAkuU4Dc PDVLGfD2gQo/8lI2orwWYsrrOEV4XZ9piWZxgJw7pCPS/dGYQuKk58k9etgbs9fGllzQ pY9yCpYP39W63TLspAVWWnfkkA2GIPFK4OtoKOR9fSdS/8P0bQlnUXfWhqpS+wN2X6Wy 8eC/Zdi3GAMkvN41f/kMgJVKPlebbstmq0D7SF/7pphHJWMDZQDprwve8gHfYh8jM0PU oFqw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=gpgcdf3GumARXnbXdcpVBbNO+CVebKsA+RWgJfCxS+M=; b=hU0UD8v5THpVbWA1kCY79F2DcpbD7zY16fmIvZ3cPvjBf9YBDBspMTirzCDgVEZwK3 nXyVUR4GnDsoq6Jvnuvtqf7lITYrbaXNQkxVR+nIu5PdD/87PKgOSDAwVKfHf/YkHuLI 1IOk1YgKgLVQLyd6zkfczKVeHaOX/BbqH9zFBEcwdpsJ2WpIZmpSY0xvA2ciEfUsxdvE D3uWWkiaYt8hfIn4H24L9W/Qm3+av3IpxlKXD91QzmwFM0EeReYcD1+T0qb+NhpSqKu4 nEWukQ1rOfsELlzFEIGbEvmJ5z9H8OGxXAsvrMVa+nyp1vd6bFblom6LjXHFT0XuU/VW wMsA== X-Gm-Message-State: APt69E0zR7ZIXd9Ep4XKDeu4Lok5pue7uzugPv0rTJzVqL4cC5FgkHo1 37Bc/XqDELFafyaKCduTfdKN6sd3 X-Google-Smtp-Source: AAOMgpc+ta8N5EfPJRNdNk2yeoPr4j9m2QvIP4uR8kHy2i1rrNafYNObl+QxHhX6h8sSrH6TPfXOjQ== X-Received: by 2002:a62:8917:: with SMTP id v23-v6mr4014505pfd.127.1530067601967; Tue, 26 Jun 2018 19:46:41 -0700 (PDT) Received: from stpd.hsd1.ca.comcast.net ([2601:647:4000:4586::4088]) by smtp.gmail.com with ESMTPSA id j21-v6sm3950904pfn.121.2018.06.26.19.46.40 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Tue, 26 Jun 2018 19:46:40 -0700 (PDT) From: Saeed Mahameed X-Google-Original-From: Saeed Mahameed To: Jesper Dangaard Brouer , Alexei Starovoitov , Daniel Borkmann Cc: neerav.parikh@intel.com, pjwaskiewicz@gmail.com, ttoukan.linux@gmail.com, Tariq Toukan , alexander.h.duyck@intel.com, peter.waskiewicz.jr@intel.com, Opher Reviv , Rony Efraim , netdev@vger.kernel.org, Saeed Mahameed Subject: [RFC bpf-next 4/6] net/mlx5e: Pass CQE to RX handlers Date: Tue, 26 Jun 2018 19:46:13 -0700 Message-Id: <20180627024615.17856-5-saeedm@mellanox.com> X-Mailer: git-send-email 2.17.0 In-Reply-To: <20180627024615.17856-1-saeedm@mellanox.com> References: <20180627024615.17856-1-saeedm@mellanox.com> Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org CQE has all the meta data information from HW. Make it available to the driver xdp handlers. Signed-off-by: Saeed Mahameed --- drivers/net/ethernet/mellanox/mlx5/core/en.h | 9 ++++++--- drivers/net/ethernet/mellanox/mlx5/core/en_rx.c | 15 +++++++++------ 2 files changed, 15 insertions(+), 9 deletions(-) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en.h b/drivers/net/ethernet/mellanox/mlx5/core/en.h index 5893acfae307..98bb315fc8a8 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/en.h @@ -505,7 +505,8 @@ struct mlx5e_rq; typedef void (*mlx5e_fp_handle_rx_cqe)(struct mlx5e_rq*, struct mlx5_cqe64*); typedef struct sk_buff * (*mlx5e_fp_skb_from_cqe_mpwrq)(struct mlx5e_rq *rq, struct mlx5e_mpw_info *wi, - u16 cqe_bcnt, u32 head_offset, u32 page_idx); + u16 cqe_bcnt, u32 head_offset, u32 page_idx, + struct mlx5_cqe64 *cqe); typedef struct sk_buff * (*mlx5e_fp_skb_from_cqe)(struct mlx5e_rq *rq, struct mlx5_cqe64 *cqe, struct mlx5e_wqe_frag_info *wi, u32 cqe_bcnt); @@ -901,10 +902,12 @@ void mlx5e_dealloc_rx_mpwqe(struct mlx5e_rq *rq, u16 ix); void mlx5e_free_rx_mpwqe(struct mlx5e_rq *rq, struct mlx5e_mpw_info *wi); struct sk_buff * mlx5e_skb_from_cqe_mpwrq_linear(struct mlx5e_rq *rq, struct mlx5e_mpw_info *wi, - u16 cqe_bcnt, u32 head_offset, u32 page_idx); + u16 cqe_bcnt, u32 head_offset, u32 page_idx, + struct mlx5_cqe64 *cqe); struct sk_buff * mlx5e_skb_from_cqe_mpwrq_nonlinear(struct mlx5e_rq *rq, struct mlx5e_mpw_info *wi, - u16 cqe_bcnt, u32 head_offset, u32 page_idx); + u16 cqe_bcnt, u32 head_offset, u32 page_idx, + struct mlx5_cqe64 *cqe); struct sk_buff * mlx5e_skb_from_cqe_linear(struct mlx5e_rq *rq, struct mlx5_cqe64 *cqe, struct mlx5e_wqe_frag_info *wi, u32 cqe_bcnt); diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c b/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c index d12577c17011..e37f9747a0e3 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c @@ -923,7 +923,8 @@ static inline bool mlx5e_xmit_xdp_frame(struct mlx5e_rq *rq, /* returns true if packet was consumed by xdp */ static inline bool mlx5e_xdp_handle(struct mlx5e_rq *rq, struct mlx5e_dma_info *di, - void *va, u16 *rx_headroom, u32 *len) + void *va, u16 *rx_headroom, + u32 *len, struct mlx5_cqe64 *cqe) { struct bpf_prog *prog = READ_ONCE(rq->xdp.prog); struct xdp_buff xdp; @@ -1012,7 +1013,7 @@ mlx5e_skb_from_cqe_linear(struct mlx5e_rq *rq, struct mlx5_cqe64 *cqe, } rcu_read_lock(); - consumed = mlx5e_xdp_handle(rq, di, va, &rx_headroom, &cqe_bcnt); + consumed = mlx5e_xdp_handle(rq, di, va, &rx_headroom, &cqe_bcnt, cqe); rcu_read_unlock(); if (consumed) return NULL; /* page/packet was consumed by XDP */ @@ -1155,7 +1156,8 @@ void mlx5e_handle_rx_cqe_rep(struct mlx5e_rq *rq, struct mlx5_cqe64 *cqe) struct sk_buff * mlx5e_skb_from_cqe_mpwrq_nonlinear(struct mlx5e_rq *rq, struct mlx5e_mpw_info *wi, - u16 cqe_bcnt, u32 head_offset, u32 page_idx) + u16 cqe_bcnt, u32 head_offset, u32 page_idx, + struct mlx5_cqe64 *cqe) { u16 headlen = min_t(u16, MLX5E_RX_MAX_HEAD, cqe_bcnt); struct mlx5e_dma_info *di = &wi->umr.dma_info[page_idx]; @@ -1202,7 +1204,8 @@ mlx5e_skb_from_cqe_mpwrq_nonlinear(struct mlx5e_rq *rq, struct mlx5e_mpw_info *w struct sk_buff * mlx5e_skb_from_cqe_mpwrq_linear(struct mlx5e_rq *rq, struct mlx5e_mpw_info *wi, - u16 cqe_bcnt, u32 head_offset, u32 page_idx) + u16 cqe_bcnt, u32 head_offset, u32 page_idx, + struct mlx5_cqe64 *cqe) { struct mlx5e_dma_info *di = &wi->umr.dma_info[page_idx]; u16 rx_headroom = rq->buff.headroom; @@ -1221,7 +1224,7 @@ mlx5e_skb_from_cqe_mpwrq_linear(struct mlx5e_rq *rq, struct mlx5e_mpw_info *wi, prefetch(data); rcu_read_lock(); - consumed = mlx5e_xdp_handle(rq, di, va, &rx_headroom, &cqe_bcnt32); + consumed = mlx5e_xdp_handle(rq, di, va, &rx_headroom, &cqe_bcnt32, cqe); rcu_read_unlock(); if (consumed) { if (__test_and_clear_bit(MLX5E_RQ_FLAG_XDP_XMIT, rq->flags)) @@ -1268,7 +1271,7 @@ void mlx5e_handle_rx_cqe_mpwrq(struct mlx5e_rq *rq, struct mlx5_cqe64 *cqe) cqe_bcnt = mpwrq_get_cqe_byte_cnt(cqe); skb = rq->mpwqe.skb_from_cqe_mpwrq(rq, wi, cqe_bcnt, head_offset, - page_idx); + page_idx, cqe); if (!skb) goto mpwrq_cqe_out; From patchwork Wed Jun 27 02:46:14 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Saeed Mahameed X-Patchwork-Id: 935237 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=dev.mellanox.co.il Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=dev-mellanox-co-il.20150623.gappssmtp.com header.i=@dev-mellanox-co-il.20150623.gappssmtp.com header.b="lOGFWu0H"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 41FnP55FnSz9s0n for ; Wed, 27 Jun 2018 12:46:49 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S932214AbeF0Cqr (ORCPT ); Tue, 26 Jun 2018 22:46:47 -0400 Received: from mail-pl0-f65.google.com ([209.85.160.65]:46195 "EHLO mail-pl0-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S932147AbeF0Cqo (ORCPT ); Tue, 26 Jun 2018 22:46:44 -0400 Received: by mail-pl0-f65.google.com with SMTP id 30-v6so282617pld.13 for ; Tue, 26 Jun 2018 19:46:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=dev-mellanox-co-il.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=bo6NWNbu0Z77UtlSRH84WMQTHNsz2Hj8haWbGNNE8+g=; b=lOGFWu0H5av4Smb5y+gkL1z+Xvh5Q+kDoGIk/W+3Yux5UUtF25UvJRDK+SWFcOPpgf Q8Ydb3toseWtN60xpHlvA6WX9r6E7d1vwdFIck92hUoGdX77UnA9LoxsGhRaX4yTnuWB KVfvZx6vrqaDXA1wExqlNMtoV50vRawFKWBlweE1L6pVgolyPliCtOyTRlSpaWR/DbUw O73IdizjWW9i18FfExSpw0p5Z97gw3N0we8l0EXlOTd0YPhARivmqPj5OBODV9+SWlOl O2BEZn6SW001aQ/IrpX53ruAX9qS4g0Exny7VYHh6AkFbFbtFO4NwIApULsnmH3j4X1g oeww== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=bo6NWNbu0Z77UtlSRH84WMQTHNsz2Hj8haWbGNNE8+g=; b=DVgP4W8X8chV4PCDHNnijkg47KuzTA7T+xTj7bqa7WhhlNJlVA8iyVziaTXQmOjRdV qDVHBe1mNwpRxIFgCIwY1PRBGW1WZfoVPDP45sOn4p8zzZLDq+vklMjiZAN6dVeHK511 whaJwPwbWEyxMhbmiqBtr2KaM3fi+8z7WJ1ySFrHLD51ydcK5HeIj+7JhPeZ9C0or2Sh CBJMHUddK79SHkrd5QVaGsg/DBQz7GGLg160R5avbpklKwC+oAXUZIbB7nxsE8HnOjlI o+4BzcxkOP1vp8/Bif5jBOIHBfSoWXFLFCUXeDaiyGvNcGYQ27LwUdcwWqclW1hJh4ta KAYw== X-Gm-Message-State: APt69E3SbrUhTBmM28k2eNUICH5hkhXCuuGJC1LdnPbgTXPQ+WSCFL1l mpdO/uvxuu/cbHaqetCg+igGZA== X-Google-Smtp-Source: ADUXVKJ8+Vd5QFP0UTU38w5F0kEZsc8U0LHi1f4IJ5Q3vywpap9XBOfD2DXROHDN3PQ0IsJBLDOmyw== X-Received: by 2002:a17:902:b81:: with SMTP id 1-v6mr4218608plr.164.1530067603748; Tue, 26 Jun 2018 19:46:43 -0700 (PDT) Received: from stpd.hsd1.ca.comcast.net ([2601:647:4000:4586::4088]) by smtp.gmail.com with ESMTPSA id j21-v6sm3950904pfn.121.2018.06.26.19.46.42 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Tue, 26 Jun 2018 19:46:42 -0700 (PDT) From: Saeed Mahameed X-Google-Original-From: Saeed Mahameed To: Jesper Dangaard Brouer , Alexei Starovoitov , Daniel Borkmann Cc: neerav.parikh@intel.com, pjwaskiewicz@gmail.com, ttoukan.linux@gmail.com, Tariq Toukan , alexander.h.duyck@intel.com, peter.waskiewicz.jr@intel.com, Opher Reviv , Rony Efraim , netdev@vger.kernel.org, Saeed Mahameed Subject: [RFC bpf-next 5/6] net/mlx5e: Add XDP RX meta data support Date: Tue, 26 Jun 2018 19:46:14 -0700 Message-Id: <20180627024615.17856-6-saeedm@mellanox.com> X-Mailer: git-send-email 2.17.0 In-Reply-To: <20180627024615.17856-1-saeedm@mellanox.com> References: <20180627024615.17856-1-saeedm@mellanox.com> Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Implement XDP meta data hash and vlan support. 1. on xdp setup ndo: add support for XDP_QUERY_META_FLAGS and return the two supported flags 2. use xdp_data_meta_set_hash and xdp_data_meta_set_vlan helpers to fill in the meta data fileds. Signed-off-by: Saeed Mahameed --- .../net/ethernet/mellanox/mlx5/core/en_main.c | 6 ++++ .../net/ethernet/mellanox/mlx5/core/en_rx.c | 30 ++++++++++++++++++- 2 files changed, 35 insertions(+), 1 deletion(-) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c index 8debae6b9cab..3d1066a953cb 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c @@ -4189,6 +4189,9 @@ static u32 mlx5e_xdp_query(struct net_device *dev) return prog_id; } +#define MLX5E_SUPPORTED_XDP_META_FLAGS \ + (XDP_FLAGS_META_HASH | XDP_FLAGS_META_VLAN) + static int mlx5e_xdp(struct net_device *dev, struct netdev_bpf *xdp) { struct mlx5e_xdp_info xdp_info; @@ -4204,6 +4207,9 @@ static int mlx5e_xdp(struct net_device *dev, struct netdev_bpf *xdp) xdp->prog_id = mlx5e_xdp_query(dev); xdp->prog_attached = !!xdp->prog_id; return 0; + case XDP_QUERY_META_FLAGS: + xdp->meta_flags = MLX5E_SUPPORTED_XDP_META_FLAGS; + return 0; default: return -EINVAL; } diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c b/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c index e37f9747a0e3..1f3e934d0dd8 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c @@ -920,6 +920,29 @@ static inline bool mlx5e_xmit_xdp_frame(struct mlx5e_rq *rq, return true; } +static void +mlx5e_xdp_fill_data_meta(xdp_md_info_arr mdi, void *data_meta, struct mlx5_cqe64 *cqe) +{ + if (xdp_data_meta_present(mdi, XDP_DATA_META_HASH)) + { + u8 cht = cqe->rss_hash_type; + int ht = (cht & CQE_RSS_HTYPE_L4) ? PKT_HASH_TYPE_L4 : + (cht & CQE_RSS_HTYPE_IP) ? PKT_HASH_TYPE_L3 : + PKT_HASH_TYPE_NONE; + u32 hash = be32_to_cpu(cqe->rss_hash_result); + + xdp_data_meta_set_hash(mdi, data_meta, hash, ht); + } + + if (xdp_data_meta_present(mdi, XDP_DATA_META_VLAN)) + { + u16 vlan = (!!cqe_has_vlan(cqe) * VLAN_TAG_PRESENT) | + be16_to_cpu(cqe->vlan_info); + + xdp_data_meta_set_vlan(mdi, data_meta, vlan); + } +} + /* returns true if packet was consumed by xdp */ static inline bool mlx5e_xdp_handle(struct mlx5e_rq *rq, struct mlx5e_dma_info *di, @@ -935,11 +958,16 @@ static inline bool mlx5e_xdp_handle(struct mlx5e_rq *rq, return false; xdp.data = va + *rx_headroom; - xdp_set_data_meta_invalid(&xdp); xdp.data_end = xdp.data + *len; xdp.data_hard_start = va; xdp.rxq = &rq->xdp_rxq; + if (rq->xdp.flags & XDP_FLAGS_META_ALL) { + xdp_reset_data_meta(&xdp); + mlx5e_xdp_fill_data_meta(rq->xdp.md_info, xdp.data_meta, cqe); + } else + xdp_set_data_meta_invalid(&xdp); + act = bpf_prog_run_xdp(prog, &xdp); switch (act) { case XDP_PASS: From patchwork Wed Jun 27 02:46:15 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Saeed Mahameed X-Patchwork-Id: 935239 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=dev.mellanox.co.il Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=dev-mellanox-co-il.20150623.gappssmtp.com header.i=@dev-mellanox-co-il.20150623.gappssmtp.com header.b="OaJUzL6y"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 41FnPJ4hpBz9s0n for ; Wed, 27 Jun 2018 12:47:00 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754079AbeF0Cqx (ORCPT ); Tue, 26 Jun 2018 22:46:53 -0400 Received: from mail-pf0-f196.google.com ([209.85.192.196]:43559 "EHLO mail-pf0-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S932186AbeF0Cqq (ORCPT ); Tue, 26 Jun 2018 22:46:46 -0400 Received: by mail-pf0-f196.google.com with SMTP id y8-v6so269013pfm.10 for ; Tue, 26 Jun 2018 19:46:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=dev-mellanox-co-il.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=DJ7cGyRbugq9xhJ2UW7uRaQumNTMGTIpGMv/rVkSO8s=; b=OaJUzL6yj2zN31UMS85E6QaIkQNqOWP2VLmomzDWLKfv6LIzbbMPAexS0cmEHKrE4O sgVV5QjziGBf0lKmfXaub8v2GALbl/Qra3AZNP8J+QT1koneYxFdC1Jz0wadsn92iU3a ldKuk7+kfb24mf4IZ61kOYrtDZQ374pxF+EVdmgdfBKl9vJWfIlGHSGfXUMKY1xplqf9 TfOR+Z4SdsbtGA/v5fjim6Qq9xNqsaLQZLRzOTb3eLnhC9vPWjdtzZ38UUMI2zQ3p63S J6JbWnLHVYx/fZQ0AxbqBaoddjA3Z3tGAiC+FxVJXgGJw/OPs8J3dy2MrOWSo+E2kE7k bifg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=DJ7cGyRbugq9xhJ2UW7uRaQumNTMGTIpGMv/rVkSO8s=; b=dsym/kKMprVQnvDJHHeqYmEDM7HgJccpyhpt4gxUHNA2eIxn8xiEQNLme8SP+UEWMt cOVs9tBE9+O5mR5eIpyeTQQCua9agTB8M8YPI8JymhmcYT2bzM6DGQeLSuVaMbrS446E TeT9UnHE/MuixASolbeu3ka1kNTpf61XxqmBE2xqYsvzl2+fjlae7QNTIEnFnGxxYWfW FP5J2nhFYWnbooaPVhuOC43Tul5XB/hAamzBgnclySNi9VYQGiD9lZYSlPoyTx025jKd QkIVXqErHrwVQCOz+NxCJxOD8DNRL3eylVeG8kSbG84zXItyBivt0s4tXiyOtb74I2mJ /n9g== X-Gm-Message-State: APt69E2HoiknBLme8YR9UxZC0BuL4GOnKW9OiUd6Rn5JxrI1kLaOvB4g OThY23ebXMPe8ylG3/MQnfkR5Q== X-Google-Smtp-Source: ADUXVKI40+Uqnmbqh59e+9fj77LWRPSOnFlFtLR647i6k+iOVSzmScZfi10vVTYt0RB1Ps2AKs2BBQ== X-Received: by 2002:a65:6612:: with SMTP id w18-v6mr3457813pgv.38.1530067605587; Tue, 26 Jun 2018 19:46:45 -0700 (PDT) Received: from stpd.hsd1.ca.comcast.net ([2601:647:4000:4586::4088]) by smtp.gmail.com with ESMTPSA id j21-v6sm3950904pfn.121.2018.06.26.19.46.43 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Tue, 26 Jun 2018 19:46:44 -0700 (PDT) From: Saeed Mahameed X-Google-Original-From: Saeed Mahameed To: Jesper Dangaard Brouer , Alexei Starovoitov , Daniel Borkmann Cc: neerav.parikh@intel.com, pjwaskiewicz@gmail.com, ttoukan.linux@gmail.com, Tariq Toukan , alexander.h.duyck@intel.com, peter.waskiewicz.jr@intel.com, Opher Reviv , Rony Efraim , netdev@vger.kernel.org, Saeed Mahameed Subject: [RFC bpf-next 6/6] samples/bpf: Add meta data hash example to xdp_redirect_cpu Date: Tue, 26 Jun 2018 19:46:15 -0700 Message-Id: <20180627024615.17856-7-saeedm@mellanox.com> X-Mailer: git-send-email 2.17.0 In-Reply-To: <20180627024615.17856-1-saeedm@mellanox.com> References: <20180627024615.17856-1-saeedm@mellanox.com> Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Add a new program (prog_num = 4) that will not parse packets and will use the meta data hash to spread/redirect traffic into different cpus. For the new program we set on bpf_set_link_xdp_fd: xdp_flags |= XDP_FLAGS_META_HASH | XDP_FLAGS_META_VLAN; On mlx5 it will succeed since mlx5 already supports these flags. The new program will read the value of the hash from the data_meta pointer from the xdp_md and will use it to compute the destination cpu. Note: I didn't test this patch to show redirect works with the hash! I only used it to see that the hash and vlan values are set correctly by the driver and can be seen by the xdp program. * I faced some difficulties to read the hash value using the helper functions defined in the previous patches, but once i used the same logic with out these functions it worked ! Will have to figure this out later. Signed-off-by: Saeed Mahameed --- samples/bpf/xdp_redirect_cpu_kern.c | 67 +++++++++++++++++++++++++++++ samples/bpf/xdp_redirect_cpu_user.c | 7 +++ 2 files changed, 74 insertions(+) diff --git a/samples/bpf/xdp_redirect_cpu_kern.c b/samples/bpf/xdp_redirect_cpu_kern.c index 303e9e7161f3..d6b3f55f342a 100644 --- a/samples/bpf/xdp_redirect_cpu_kern.c +++ b/samples/bpf/xdp_redirect_cpu_kern.c @@ -376,6 +376,73 @@ int xdp_prognum3_proto_separate(struct xdp_md *ctx) return bpf_redirect_map(&cpu_map, cpu_dest, 0); } +#if 0 +xdp_md_info_arr mdi = { + [XDP_DATA_META_HASH] = {.offset = 0, .present = 1}, + [XDP_DATA_META_VLAN] = {.offset = sizeof(struct xdp_md_hash), .present = 1}, +}; +#endif + +SEC("xdp_cpu_map4_hash_separate") +int xdp_prognum4_hash_separate(struct xdp_md *ctx) +{ + void *data_meta = (void *)(long)ctx->data_meta; + void *data_end = (void *)(long)ctx->data_end; + void *data = (void *)(long)ctx->data; + struct xdp_md_hash *hash; + struct xdp_md_vlan *vlan; + struct datarec *rec; + u32 cpu_dest = 0; + u32 cpu_idx = 0; + u32 *cpu_lookup; + u32 key = 0; + + /* Count RX packet in map */ + rec = bpf_map_lookup_elem(&rx_cnt, &key); + if (!rec) + return XDP_ABORTED; + rec->processed++; + + /* for some reason this code fails to be verified */ +#if 0 + hash = xdp_data_meta_get_hash(mdi, data_meta); + if (hash + 1 > data) + return XDP_ABORTED; + + vlan = xdp_data_meta_get_vlan(mdi, data_meta); + if (vlan + 1 > data) + return XDP_ABORTED; +#endif + + /* Work around for the above code */ + hash = data_meta; /* since we know hash will appear first */ + if (hash + 1 > data) + return XDP_ABORTED; + +#if 0 + // Just for testing + /* We know that vlan will appear after the hash */ + vlan = (void *)((char *)data_meta + sizeof(*hash)); + if (vlan + 1 > data) { + return XDP_ABORTED; + } +#endif + + cpu_idx = reciprocal_scale(hash->hash, MAX_CPUS); + + cpu_lookup = bpf_map_lookup_elem(&cpus_available, &cpu_idx); + if (!cpu_lookup) + return XDP_ABORTED; + cpu_dest = *cpu_lookup; + + if (cpu_dest >= MAX_CPUS) { + rec->issue++; + return XDP_ABORTED; + } + + return bpf_redirect_map(&cpu_map, cpu_dest, 0); +} + SEC("xdp_cpu_map4_ddos_filter_pktgen") int xdp_prognum4_ddos_filter_pktgen(struct xdp_md *ctx) { diff --git a/samples/bpf/xdp_redirect_cpu_user.c b/samples/bpf/xdp_redirect_cpu_user.c index f6efaefd485b..3429215d5a7b 100644 --- a/samples/bpf/xdp_redirect_cpu_user.c +++ b/samples/bpf/xdp_redirect_cpu_user.c @@ -679,6 +679,13 @@ int main(int argc, char **argv) return EXIT_FAIL_OPTION; } + /* + * prog_num 4 requires xdp meta data hash + * Vlan is not required but added just for testing.. + */ + if (prog_num == 4) + xdp_flags |= XDP_FLAGS_META_HASH | XDP_FLAGS_META_VLAN; + /* Remove XDP program when program is interrupted */ signal(SIGINT, int_exit);