From patchwork Sun Dec 9 01:14:49 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ophir Munk X-Patchwork-Id: 1009907 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=openvswitch.org (client-ip=140.211.169.12; helo=mail.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=mellanox.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=Mellanox.com header.i=@Mellanox.com header.b="Kjy+y+gb"; dkim-atps=neutral Received: from mail.linuxfoundation.org (mail.linuxfoundation.org [140.211.169.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 43C7Y02N8mz9s3Z for ; Sun, 9 Dec 2018 12:14:59 +1100 (AEDT) Received: from mail.linux-foundation.org (localhost [127.0.0.1]) by mail.linuxfoundation.org (Postfix) with ESMTP id B25C141C; Sun, 9 Dec 2018 01:14:55 +0000 (UTC) X-Original-To: ovs-dev@openvswitch.org Delivered-To: ovs-dev@mail.linuxfoundation.org Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 1CE2B74 for ; Sun, 9 Dec 2018 01:14:54 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.7.6 Received: from EUR03-DB5-obe.outbound.protection.outlook.com (mail-eopbgr40064.outbound.protection.outlook.com [40.107.4.64]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 62D6C82D for ; Sun, 9 Dec 2018 01:14:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=K5B3dVLvLJilGo6r2o+XVWJpsBs++xuxDwpMr8Q5gaQ=; b=Kjy+y+gbj+zdhEgXHbO5Wey2AG7pqeMtriveFV53kpf+Ib/7QVW4R+RFTNUzPagWWky7cN1V/F4B6rnAR/ft1JY3dqn0MH6kpBNX1swf9HvV4bIfNrwOQuGexuAO4X1rpOEYfRvs0Y7tEjv3tIursidXNtYN/SiaRnOsrzWRNas= Received: from VI1PR0502MB3743.eurprd05.prod.outlook.com (52.134.8.154) by VI1PR0502MB4062.eurprd05.prod.outlook.com (52.134.18.151) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1404.17; Sun, 9 Dec 2018 01:14:49 +0000 Received: from VI1PR0502MB3743.eurprd05.prod.outlook.com ([fe80::c53e:420a:9ec0:d4cd]) by VI1PR0502MB3743.eurprd05.prod.outlook.com ([fe80::c53e:420a:9ec0:d4cd%4]) with mapi id 15.20.1404.025; Sun, 9 Dec 2018 01:14:49 +0000 From: Ophir Munk To: "ovs-dev@openvswitch.org" Thread-Topic: [dpdk-hwol RFC v1] netdev-dpdk: add HW-offload port output action Thread-Index: AQHUj1yU7G8LUq7QO0ml4mnw9Xfl1w== Date: Sun, 9 Dec 2018 01:14:49 +0000 Message-ID: <1544318073-10985-1-git-send-email-ophirmu@mellanox.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: LO2P123CA0013.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:a6::25) To VI1PR0502MB3743.eurprd05.prod.outlook.com (2603:10a6:803:11::26) authentication-results: spf=none (sender IP is ) smtp.mailfrom=ophirmu@mellanox.com; x-ms-exchange-messagesentrepresentingtype: 1 x-originating-ip: [37.142.13.130] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; VI1PR0502MB4062; 6:gr1cQfHm9BjwsTqH0Fo/CnbmI9OPaLR9+0Xgdy5Iqclt/Os8xLctvCKXdNKJnsVnjl4XOhZEVSCa5Nu/IKWQPq+q6BCPinQaxYBSI8Gqj7m2+Q9eGE4wnOG50igYwOqo559CXgtgjDX2j6pJn/k10mR5KAJkFt4o020mYTA15F35ErU4QeUUgDUq7bkfCpDAl3DchoVXZrr8sRWIMdOKNwxOaeL2gQx1gj5qGU4D5OwAoRfghNeIXrh0OLveNEuOpiLxkNihGqsp/ub3Hvoo5pls+Y5A7LFxkCqOymj+At7ciumTsRJ8L6G23ceuKs1mQk6S/B5wVJLarSMl5Xqm+XsPdbYVS81vkikgV6GKREG12dyVl4BQmW+E0h/w7XtUcl8n/EwkZuGADScFX31fWiW1ifIvkdpq1G9yUV1qkpSuSMfXCJzxZqRPs7VZMu3fKBLi80TmeUfJM8pRYMu9kA==; 5:SXqSsdgKgE7KX0Eh6U5dEXWG4Ip7PVbuMGAa4trDZOOzjnI9k5u5tMpFXEz/hxAHXrz/Udp4qih7XOT4Lv/AalACTm50c0GrX5oYjXszRN75Q08B531Cdk9XrfUqJYGKAArfkTmYIQ5oXh+NcvaJQONfnyvwUnxJe/zCpEi68ug=; 7:CPBg3v+4MBeq83gP6dM2Ks8m8GwqyABnkdFDfZt5tBYyCswmQtmeOKDehqeRI5P2F1aNedeaFFp5EVtJt8h44r17bfeOhsTEkBZj+GIZimTYsL0dbW6DOZcvrlpclQEOjQVq13IF3srlJqAgzT968Q== x-ms-office365-filtering-correlation-id: 724aaac6-af4d-4343-ca7b-08d65d73b6f7 x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390098)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600074)(711020)(4618075)(2017052603328)(7153060)(7193020); SRVR:VI1PR0502MB4062; x-ms-traffictypediagnostic: VI1PR0502MB4062: x-ld-processed: a652971c-7d2e-4d9b-a6a4-d149256f461b,ExtAddr x-microsoft-antispam-prvs: x-ms-exchange-senderadcheck: 1 x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(8211001083)(6040522)(2401047)(8121501046)(5005006)(10201501046)(3002001)(3231455)(999002)(944501520)(52105112)(93006095)(93001095)(6055026)(148016)(149066)(150057)(6041310)(20161123558120)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123560045)(20161123564045)(20161123562045)(201708071742011)(7699051)(76991095); SRVR:VI1PR0502MB4062; BCL:0; PCL:0; RULEID:; SRVR:VI1PR0502MB4062; x-forefront-prvs: 0881A7A935 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(376002)(346002)(366004)(39850400004)(396003)(136003)(189003)(199004)(476003)(6116002)(478600001)(2616005)(6506007)(71190400001)(486006)(3846002)(26005)(71200400001)(66066001)(186003)(5660300001)(8936002)(6916009)(81166006)(52116002)(2906002)(575784001)(86362001)(386003)(102836004)(99286004)(14444005)(256004)(6512007)(5640700003)(7736002)(305945005)(54906003)(105586002)(97736004)(81156014)(8676002)(2501003)(4326008)(68736007)(316002)(36756003)(14454004)(6436002)(53936002)(6486002)(106356001)(39060400002)(2351001)(25786009); DIR:OUT; SFP:1101; SCL:1; SRVR:VI1PR0502MB4062; H:VI1PR0502MB3743.eurprd05.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; received-spf: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) x-microsoft-antispam-message-info: EWjRq9wzxdnocNC6EIeFtn1JYzk/TNq2TJin0DSyTrdnL3eRVwiGoPpoDTTEQpgT0/V8IGC4C+twY0/pmGlvuG61e2qVfTSwsqPQ9Kh094c10e+iFFrOC3sBd7roas0QKlfeb5AgDR3e2Lrx3x1NOX4eFMFwe7r3oZJba7+YdwGDkLmepk8WuMkm9IsrrIHJ45NHpXzc/gB1oUET0A3JlYzD5f4o2J5z/y/SoV3L2XR3Rxz7kq5ZHXW6BfZSZYSj+jZrCWKCfCKr0IIPG2L6R+rdfyi6gDra9p9US7oVLhGyW8bkrXuxc8WD/5DsQLnacC1+RJvwzqfEbEbDvqr/ZaXt+k/Cy/JpP+ay3oro1A0= spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM MIME-Version: 1.0 X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-Network-Message-Id: 724aaac6-af4d-4343-ca7b-08d65d73b6f7 X-MS-Exchange-CrossTenant-originalarrivaltime: 09 Dec 2018 01:14:49.7983 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0502MB4062 X-Spam-Status: No, score=-2.0 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Cc: Shahaf Shuler , Simon Horman , Thomas Monjalon , Ilya Maximets Subject: [ovs-dev] [dpdk-hwol RFC v1] netdev-dpdk: add HW-offload port output action X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: ovs-dev-bounces@openvswitch.org Errors-To: ovs-dev-bounces@openvswitch.org The DPDK commit in [1] adds a "transfer" flag to flow attributes. When enabled (transfer=1) the underlaying device is requested to perform the flow match and the flow actions in HW. It is relevant to DPDK ports in SRIOV mode. This commit adds a HW "port output" action request. If the HW rejects this request a fall-back plan is to request the HW to perform the flow match with a flow MARK action. It is considered a partial offload as the flow match is still performed in HW but the actual actions continue in SW using the MARK ID. In this case we set transfer=0. If the HW rejects this request the final fall-back plan is to continue in legacy OVS mode where both the flow match and the flow actions are performed in SW. HW offload must be enabled by executing: ovs-vsctl set Open_vSwitch . other_config:hw-offload=true [1[ DPDK commit 76e9a55b5b ("ethdev: add transfer attribute to flow API") Signed-off-by: Ophir Munk --- v1: Initial RFC lib/dpif-netdev.c | 5 ++- lib/netdev-dpdk.c | 110 ++++++++++++++++++++++++++++++++++++++++++++++++------ 2 files changed, 101 insertions(+), 14 deletions(-) diff --git a/lib/dpif-netdev.c b/lib/dpif-netdev.c index 1564db9..5d06036 100644 --- a/lib/dpif-netdev.c +++ b/lib/dpif-netdev.c @@ -2352,6 +2352,7 @@ dp_netdev_flow_offload_put(struct dp_flow_offload_item *offload) } } info.flow_mark = mark; + info.dpif_class = pmd->dp->class; ovs_mutex_lock(&pmd->dp->port_mutex); port = dp_netdev_lookup_port(pmd->dp, in_port); diff --git a/lib/netdev-dpdk.c b/lib/netdev-dpdk.c index 9dffa0e..d310f23 100644 --- a/lib/netdev-dpdk.c +++ b/lib/netdev-dpdk.c @@ -4246,6 +4246,14 @@ struct flow_actions { }; static void +free_flow_patterns(struct flow_patterns *patterns) +{ + free(patterns->items); + patterns->items = NULL; + patterns->cnt = 0; +} + +static void dump_flow_pattern(struct rte_flow_item *item) { struct ds s; @@ -4462,6 +4470,14 @@ add_flow_pattern(struct flow_patterns *patterns, enum rte_flow_item_type type, } static void +free_flow_actions(struct flow_actions *actions) +{ + free(actions->actions); + actions->actions = NULL; + actions->cnt = 0; +} + +static void add_flow_action(struct flow_actions *actions, enum rte_flow_action_type type, const void *conf) { @@ -4520,12 +4536,12 @@ add_flow_rss_action(struct flow_actions *actions, static int netdev_dpdk_add_rte_flow_offload(struct netdev *netdev, const struct match *match, - struct nlattr *nl_actions OVS_UNUSED, - size_t actions_len OVS_UNUSED, + struct nlattr *nl_actions, + size_t actions_len, const ovs_u128 *ufid, struct offload_info *info) { struct netdev_dpdk *dev = netdev_dpdk_cast(netdev); - const struct rte_flow_attr flow_attr = { + struct rte_flow_attr flow_attr = { .group = 0, .priority = 0, .ingress = 1, @@ -4536,6 +4552,7 @@ netdev_dpdk_add_rte_flow_offload(struct netdev *netdev, struct rte_flow *flow; struct rte_flow_error error; uint8_t *ipv4_next_proto_mask = NULL; + struct action_rss_data *rss = NULL; int ret = 0; /* Eth */ @@ -4723,20 +4740,88 @@ end_proto_check: add_flow_pattern(&patterns, RTE_FLOW_ITEM_TYPE_END, NULL, NULL); - struct rte_flow_action_mark mark; - struct action_rss_data *rss; + const struct nlattr *a; + unsigned int left; + if (!actions_len || !nl_actions) { + VLOG_DBG("%s: skip flow offload without actions\n", + netdev_get_name(netdev)); + ret = -1; + goto out; + } + NL_ATTR_FOR_EACH_UNSAFE (a, left, nl_actions, actions_len) { + int type = nl_attr_type(a); + switch ((enum ovs_action_attr) type) { + case OVS_ACTION_ATTR_OUTPUT: { + struct rte_flow_action_port_id port_id; + odp_port_t odp_port = nl_attr_get_odp_port(a); + /* Output port should be hardware port number. */ + struct netdev *dst_netdev = netdev_ports_get(odp_port, + info->dpif_class); + if (!dst_netdev) { + VLOG_WARN("Cannot find dpdk port %u", odp_to_u32(odp_port)); + continue; + } + struct netdev_dpdk *dst_dev = netdev_dpdk_cast(dst_netdev); + port_id.id = dst_dev->port_id; + port_id.original = 0; + add_flow_action(&actions, RTE_FLOW_ACTION_TYPE_PORT_ID, &port_id); + break; + } + case OVS_ACTION_ATTR_PUSH_VLAN: + case OVS_ACTION_ATTR_POP_VLAN: + case OVS_ACTION_ATTR_UNSPEC: + case OVS_ACTION_ATTR_USERSPACE: + case OVS_ACTION_ATTR_SET: + case OVS_ACTION_ATTR_SAMPLE: + case OVS_ACTION_ATTR_RECIRC: + case OVS_ACTION_ATTR_HASH: + case OVS_ACTION_ATTR_PUSH_MPLS: + case OVS_ACTION_ATTR_POP_MPLS: + case OVS_ACTION_ATTR_SET_MASKED: + case OVS_ACTION_ATTR_CT: + case OVS_ACTION_ATTR_TRUNC: + case OVS_ACTION_ATTR_PUSH_ETH: + case OVS_ACTION_ATTR_POP_ETH: + case OVS_ACTION_ATTR_CT_CLEAR: + case OVS_ACTION_ATTR_PUSH_NSH: + case OVS_ACTION_ATTR_POP_NSH: + case OVS_ACTION_ATTR_METER: + case OVS_ACTION_ATTR_TUNNEL_PUSH: + case OVS_ACTION_ATTR_TUNNEL_POP: + case OVS_ACTION_ATTR_CLONE: + case __OVS_ACTION_ATTR_MAX: + goto mark_action; + break; + } + } + add_flow_action(&actions, RTE_FLOW_ACTION_TYPE_END, NULL); + ovs_mutex_lock(&dev->mutex); + flow_attr.transfer = 1; + flow = rte_flow_create(dev->port_id, &flow_attr, patterns.items, + actions.actions, &error); + ovs_mutex_unlock(&dev->mutex); + if (!flow) { + VLOG_ERR("%s: rte flow creat offload error: %u : message : %s\n", + netdev_get_name(netdev), error.type, error.message); + } else { + goto install_flow; + } + + /* + * If flow actions failed to be offloaded - try offloading the mark action + */ + struct rte_flow_action_mark mark; +mark_action: + free_flow_actions(&actions); mark.id = info->flow_mark; add_flow_action(&actions, RTE_FLOW_ACTION_TYPE_MARK, &mark); - ovs_mutex_lock(&dev->mutex); - rss = add_flow_rss_action(&actions, netdev); add_flow_action(&actions, RTE_FLOW_ACTION_TYPE_END, NULL); - + flow_attr.transfer = 0; flow = rte_flow_create(dev->port_id, &flow_attr, patterns.items, actions.actions, &error); - ovs_mutex_unlock(&dev->mutex); free(rss); @@ -4746,13 +4831,14 @@ end_proto_check: ret = -1; goto out; } + +install_flow: ufid_to_rte_flow_associate(ufid, flow); VLOG_DBG("%s: installed flow %p by ufid "UUID_FMT"\n", netdev_get_name(netdev), flow, UUID_ARGS((struct uuid *)ufid)); - out: - free(patterns.items); - free(actions.actions); + free_flow_patterns(&patterns); + free_flow_actions(&actions); return ret; }