From patchwork Mon Dec 31 19:45:53 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Ophir Munk X-Patchwork-Id: 1019699 X-Patchwork-Delegate: ian.stokes@intel.com Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=openvswitch.org (client-ip=140.211.169.12; helo=mail.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=mellanox.com Received: from mail.linuxfoundation.org (mail.linuxfoundation.org [140.211.169.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 43T79V1yJZz9sDP for ; Tue, 1 Jan 2019 06:46:38 +1100 (AEDT) Received: from mail.linux-foundation.org (localhost [127.0.0.1]) by mail.linuxfoundation.org (Postfix) with ESMTP id E3852A7A; Mon, 31 Dec 2018 19:46:24 +0000 (UTC) X-Original-To: ovs-dev@openvswitch.org Delivered-To: ovs-dev@mail.linuxfoundation.org Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 63D9DA67 for ; Mon, 31 Dec 2018 19:46:23 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by smtp1.linuxfoundation.org (Postfix) with ESMTP id E5E927D2 for ; Mon, 31 Dec 2018 19:46:20 +0000 (UTC) Received: from Internal Mail-Server by MTLPINE1 (envelope-from ophirmu@mellanox.com) with ESMTPS (AES256-SHA encrypted); 31 Dec 2018 21:46:19 +0200 Received: from localhost.localdomain (pegasus05.mtr.labs.mlnx [10.210.16.100]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id wBVJkJ5w022011; Mon, 31 Dec 2018 21:46:19 +0200 Received: from pegasus05.mtr.labs.mlnx (localhost [127.0.0.1]) by localhost.localdomain (8.14.7/8.14.7) with ESMTP id wBVJkJ7U001675; Mon, 31 Dec 2018 19:46:19 GMT Received: (from root@localhost) by pegasus05.mtr.labs.mlnx (8.14.7/8.14.7/Submit) id wBVJkJLj001674; Mon, 31 Dec 2018 19:46:19 GMT From: Ophir Munk To: ovs-dev@openvswitch.org Date: Mon, 31 Dec 2018 19:45:53 +0000 Message-Id: <1546285557-1617-2-git-send-email-ophirmu@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1546285557-1617-1-git-send-email-ophirmu@mellanox.com> References: <1546285557-1617-1-git-send-email-ophirmu@mellanox.com> MIME-Version: 1.0 X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_NONE, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Cc: Shahaf Shuler , Simon Horman , Ilya Maximets , Thomas Monjalon Subject: [ovs-dev] [hwol RFC v1 1/5] netdev-dpdk: Add port output hw offload action X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: ovs-dev-bounces@openvswitch.org Errors-To: ovs-dev-bounces@openvswitch.org The DPDK commit in [1] adds a "transfer" flag to flow attributes. When enabled (transfer=1) the underlaying device is requested to perform the flow match and the flow actions in HW. It is relevant to DPDK ports in SRIOV mode. This commit adds a HW "port output" action request. If the HW rejects this request a fall-back plan is to request the HW to perform the flow match with a flow MARK action. It is considered a partial offload as the flow match is still performed in HW but the actual actions continue in SW using the MARK ID. In this case we set transfer=0. If the HW rejects this request the final fall-back plan is to continue in legacy OVS mode where both the flow match and the flow actions are performed in SW. HW offload must be enabled by executing: ovs-vsctl set Open_vSwitch . other_config:hw-offload=true [1[ DPDK commit 76e9a55b5b ("ethdev: add transfer attribute to flow API") Signed-off-by: Ophir Munk --- lib/dpif-netdev.c | 5 ++- lib/netdev-dpdk.c | 110 ++++++++++++++++++++++++++++++++++++++++++++++++------ 2 files changed, 101 insertions(+), 14 deletions(-) diff --git a/lib/dpif-netdev.c b/lib/dpif-netdev.c index 1564db9..5d06036 100644 --- a/lib/dpif-netdev.c +++ b/lib/dpif-netdev.c @@ -2352,6 +2352,7 @@ dp_netdev_flow_offload_put(struct dp_flow_offload_item *offload) } } info.flow_mark = mark; + info.dpif_class = pmd->dp->class; ovs_mutex_lock(&pmd->dp->port_mutex); port = dp_netdev_lookup_port(pmd->dp, in_port); @@ -6312,12 +6313,12 @@ dp_netdev_input__(struct dp_netdev_pmd_thread *pmd, /* All the flow batches need to be reset before any call to * packet_batch_per_flow_execute() as it could potentially trigger - * recirculation. When a packet matching flow ‘j’ happens to be + * recirculation. When a packet matching flow ???j??? happens to be * recirculated, the nested call to dp_netdev_input__() could potentially * classify the packet as matching another flow - say 'k'. It could happen * that in the previous call to dp_netdev_input__() that same flow 'k' had * already its own batches[k] still waiting to be served. So if its - * ‘batch’ member is not reset, the recirculated packet would be wrongly + * ???batch??? member is not reset, the recirculated packet would be wrongly * appended to batches[k] of the 1st call to dp_netdev_input__(). */ for (i = 0; i < n_batches; i++) { batches[i].flow->batch = NULL; diff --git a/lib/netdev-dpdk.c b/lib/netdev-dpdk.c index 18c6e25..04413eb 100644 --- a/lib/netdev-dpdk.c +++ b/lib/netdev-dpdk.c @@ -4260,6 +4260,14 @@ struct flow_actions { }; static void +free_flow_patterns(struct flow_patterns *patterns) +{ + free(patterns->items); + patterns->items = NULL; + patterns->cnt = 0; +} + +static void dump_flow_pattern(struct rte_flow_item *item) { struct ds s; @@ -4476,6 +4484,14 @@ add_flow_pattern(struct flow_patterns *patterns, enum rte_flow_item_type type, } static void +free_flow_actions(struct flow_actions *actions) +{ + free(actions->actions); + actions->actions = NULL; + actions->cnt = 0; +} + +static void add_flow_action(struct flow_actions *actions, enum rte_flow_action_type type, const void *conf) { @@ -4534,12 +4550,12 @@ add_flow_rss_action(struct flow_actions *actions, static int netdev_dpdk_add_rte_flow_offload(struct netdev *netdev, const struct match *match, - struct nlattr *nl_actions OVS_UNUSED, - size_t actions_len OVS_UNUSED, + struct nlattr *nl_actions, + size_t actions_len, const ovs_u128 *ufid, struct offload_info *info) { struct netdev_dpdk *dev = netdev_dpdk_cast(netdev); - const struct rte_flow_attr flow_attr = { + struct rte_flow_attr flow_attr = { .group = 0, .priority = 0, .ingress = 1, @@ -4550,6 +4566,7 @@ netdev_dpdk_add_rte_flow_offload(struct netdev *netdev, struct rte_flow *flow; struct rte_flow_error error; uint8_t *ipv4_next_proto_mask = NULL; + struct action_rss_data *rss = NULL; int ret = 0; /* Eth */ @@ -4737,20 +4754,88 @@ end_proto_check: add_flow_pattern(&patterns, RTE_FLOW_ITEM_TYPE_END, NULL, NULL); - struct rte_flow_action_mark mark; - struct action_rss_data *rss; + const struct nlattr *a; + unsigned int left; + if (!actions_len || !nl_actions) { + VLOG_DBG("%s: skip flow offload without actions\n", + netdev_get_name(netdev)); + ret = -1; + goto out; + } + struct rte_flow_action_port_id port_id; + NL_ATTR_FOR_EACH_UNSAFE (a, left, nl_actions, actions_len) { + int type = nl_attr_type(a); + switch ((enum ovs_action_attr) type) { + case OVS_ACTION_ATTR_OUTPUT: { + odp_port_t odp_port = nl_attr_get_odp_port(a); + /* Output port should be hardware port number. */ + struct netdev *dst_netdev = netdev_ports_get(odp_port, + info->dpif_class); + if (!dst_netdev) { + VLOG_WARN("Cannot find dpdk port %u", odp_to_u32(odp_port)); + continue; + } + struct netdev_dpdk *dst_dev = netdev_dpdk_cast(dst_netdev); + port_id.id = dst_dev->port_id; + port_id.original = 0; + add_flow_action(&actions, RTE_FLOW_ACTION_TYPE_PORT_ID, &port_id); + break; + } + case OVS_ACTION_ATTR_PUSH_VLAN: + case OVS_ACTION_ATTR_POP_VLAN: + case OVS_ACTION_ATTR_UNSPEC: + case OVS_ACTION_ATTR_USERSPACE: + case OVS_ACTION_ATTR_SET: + case OVS_ACTION_ATTR_SAMPLE: + case OVS_ACTION_ATTR_RECIRC: + case OVS_ACTION_ATTR_HASH: + case OVS_ACTION_ATTR_PUSH_MPLS: + case OVS_ACTION_ATTR_POP_MPLS: + case OVS_ACTION_ATTR_SET_MASKED: + case OVS_ACTION_ATTR_CT: + case OVS_ACTION_ATTR_TRUNC: + case OVS_ACTION_ATTR_PUSH_ETH: + case OVS_ACTION_ATTR_POP_ETH: + case OVS_ACTION_ATTR_CT_CLEAR: + case OVS_ACTION_ATTR_PUSH_NSH: + case OVS_ACTION_ATTR_POP_NSH: + case OVS_ACTION_ATTR_METER: + case OVS_ACTION_ATTR_TUNNEL_PUSH: + case OVS_ACTION_ATTR_TUNNEL_POP: + case OVS_ACTION_ATTR_CLONE: + case __OVS_ACTION_ATTR_MAX: + goto mark_action; + break; + } + } + add_flow_action(&actions, RTE_FLOW_ACTION_TYPE_END, NULL); + ovs_mutex_lock(&dev->mutex); + flow_attr.transfer = 1; + flow = rte_flow_create(dev->port_id, &flow_attr, patterns.items, + actions.actions, &error); + ovs_mutex_unlock(&dev->mutex); + if (!flow) { + VLOG_ERR("%s: rte flow creat offload error: %u : message : %s\n", + netdev_get_name(netdev), error.type, error.message); + } else { + goto install_flow; + } + + /* + * If flow actions failed to be offloaded - try offloading the mark action + */ + struct rte_flow_action_mark mark; +mark_action: + free_flow_actions(&actions); mark.id = info->flow_mark; add_flow_action(&actions, RTE_FLOW_ACTION_TYPE_MARK, &mark); - ovs_mutex_lock(&dev->mutex); - rss = add_flow_rss_action(&actions, netdev); add_flow_action(&actions, RTE_FLOW_ACTION_TYPE_END, NULL); - + flow_attr.transfer = 0; flow = rte_flow_create(dev->port_id, &flow_attr, patterns.items, actions.actions, &error); - ovs_mutex_unlock(&dev->mutex); free(rss); @@ -4760,13 +4845,14 @@ end_proto_check: ret = -1; goto out; } + +install_flow: ufid_to_rte_flow_associate(ufid, flow); VLOG_DBG("%s: installed flow %p by ufid "UUID_FMT"\n", netdev_get_name(netdev), flow, UUID_ARGS((struct uuid *)ufid)); - out: - free(patterns.items); - free(actions.actions); + free_flow_patterns(&patterns); + free_flow_actions(&actions); return ret; }