From patchwork Tue Feb 21 00:41:09 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ivan Malov X-Patchwork-Id: 1745355 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::138; helo=smtp1.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp1.osuosl.org (smtp1.osuosl.org [IPv6:2605:bc80:3010::138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4PLLKT5w54z23yV for ; Tue, 21 Feb 2023 11:49:49 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 25467834DE; Tue, 21 Feb 2023 00:49:48 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 25467834DE X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id zq47shqOeCxE; Tue, 21 Feb 2023 00:49:47 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp1.osuosl.org (Postfix) with ESMTPS id 59D128238D; Tue, 21 Feb 2023 00:49:46 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 59D128238D Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 336FBC0032; Tue, 21 Feb 2023 00:49:46 +0000 (UTC) X-Original-To: ovs-dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 1D6BEC002B for ; Tue, 21 Feb 2023 00:49:45 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id ED44D60F92 for ; Tue, 21 Feb 2023 00:49:44 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org ED44D60F92 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 1OjHL3pNc3xg for ; Tue, 21 Feb 2023 00:49:44 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 3EF1760AD1 Received: from agw.arknetworks.am (agw.arknetworks.am [79.141.165.80]) by smtp3.osuosl.org (Postfix) with ESMTPS id 3EF1760AD1 for ; Tue, 21 Feb 2023 00:49:44 +0000 (UTC) Received: from localhost.localdomain (unknown [78.109.74.189]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by agw.arknetworks.am (Postfix) with ESMTPSA id 5FA79E12F8; Tue, 21 Feb 2023 04:40:53 +0400 (+04) To: ovs-dev@openvswitch.org Date: Tue, 21 Feb 2023 04:41:09 +0400 Message-Id: <20230221004111.7945-2-ivan.malov@arknetworks.am> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20230221004111.7945-1-ivan.malov@arknetworks.am> References: <20230221004111.7945-1-ivan.malov@arknetworks.am> MIME-Version: 1.0 Cc: Eli Britstein , Ori Kam , David Marchand , Ilya Maximets Subject: [ovs-dev] [PATCH v4 1/3] netdev-dpdk: negotiate delivery of per-packet Rx metadata X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Ivan Malov via dev From: Ivan Malov Reply-To: Ivan Malov Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" This may be required by some PMDs in offload scenarios. Signed-off-by: Ivan Malov --- lib/netdev-dpdk.c | 40 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 40 insertions(+) diff --git a/lib/netdev-dpdk.c b/lib/netdev-dpdk.c index fb0dd43f7..2cebc3cca 100644 --- a/lib/netdev-dpdk.c +++ b/lib/netdev-dpdk.c @@ -1140,6 +1140,36 @@ dpdk_eth_flow_ctrl_setup(struct netdev_dpdk *dev) OVS_REQUIRES(dev->mutex) } } +static void +dpdk_eth_dev_init_rx_metadata(struct netdev_dpdk *dev) +{ + uint64_t rx_metadata = 0; + int ret; + + /* For the fallback offload (non-"transfer" rules) */ + rx_metadata |= RTE_ETH_RX_METADATA_USER_MARK; + /* For the full offload ("transfer" rules) */ + rx_metadata |= RTE_ETH_RX_METADATA_TUNNEL_ID; + + ret = rte_eth_rx_metadata_negotiate(dev->port_id, &rx_metadata); + if (ret == 0) { + if (!(rx_metadata & RTE_ETH_RX_METADATA_USER_MARK)) { + VLOG_DBG("The NIC will not provide per-packet USER_MARK on port " + DPDK_PORT_ID_FMT, dev->port_id); + } + if (!(rx_metadata & RTE_ETH_RX_METADATA_TUNNEL_ID)) { + VLOG_DBG("The NIC will not provide per-packet TUNNEL_ID on port " + DPDK_PORT_ID_FMT, dev->port_id); + } + } else if (ret == -ENOTSUP) { + VLOG_DBG("Rx metadata negotiate procedure is not supported on port " + DPDK_PORT_ID_FMT, dev->port_id); + } else { + VLOG_WARN("Cannot negotiate Rx metadata on port " + DPDK_PORT_ID_FMT, dev->port_id); + } +} + static int dpdk_eth_dev_init(struct netdev_dpdk *dev) OVS_REQUIRES(dev->mutex) @@ -1154,6 +1184,16 @@ dpdk_eth_dev_init(struct netdev_dpdk *dev) RTE_ETH_RX_OFFLOAD_TCP_CKSUM | RTE_ETH_RX_OFFLOAD_IPV4_CKSUM; + /* + * Full tunnel offload requires that tunnel ID metadata be + * delivered with "miss" packets from the hardware to the + * PMD. The same goes for megaflow mark metadata which is + * used in MARK + RSS offload scenario. + * + * Request delivery of such metadata. + */ + dpdk_eth_dev_init_rx_metadata(dev); + rte_eth_dev_info_get(dev->port_id, &info); if (strstr(info.driver_name, "vf") != NULL) { From patchwork Tue Feb 21 00:41:10 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ivan Malov X-Patchwork-Id: 1745357 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::133; helo=smtp2.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp2.osuosl.org (smtp2.osuosl.org [IPv6:2605:bc80:3010::133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4PLLKc5ZRcz23yV for ; Tue, 21 Feb 2023 11:49:56 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id BF1BE4158F; Tue, 21 Feb 2023 00:49:52 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org BF1BE4158F X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id SQlrP5Ig0k7M; Tue, 21 Feb 2023 00:49:50 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp2.osuosl.org (Postfix) with ESMTPS id 16C5140C47; Tue, 21 Feb 2023 00:49:49 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 16C5140C47 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 187C6C007F; Tue, 21 Feb 2023 00:49:47 +0000 (UTC) X-Original-To: ovs-dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp2.osuosl.org (smtp2.osuosl.org [IPv6:2605:bc80:3010::133]) by lists.linuxfoundation.org (Postfix) with ESMTP id 224C8C0032 for ; Tue, 21 Feb 2023 00:49:45 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id E3E45409D6 for ; Tue, 21 Feb 2023 00:49:44 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org E3E45409D6 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id enAprjsWSQPC for ; Tue, 21 Feb 2023 00:49:44 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 3554A4012A Received: from agw.arknetworks.am (agw.arknetworks.am [79.141.165.80]) by smtp2.osuosl.org (Postfix) with ESMTPS id 3554A4012A for ; Tue, 21 Feb 2023 00:49:44 +0000 (UTC) Received: from localhost.localdomain (unknown [78.109.74.189]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by agw.arknetworks.am (Postfix) with ESMTPSA id EA182E12FA; Tue, 21 Feb 2023 04:40:53 +0400 (+04) To: ovs-dev@openvswitch.org Date: Tue, 21 Feb 2023 04:41:10 +0400 Message-Id: <20230221004111.7945-3-ivan.malov@arknetworks.am> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20230221004111.7945-1-ivan.malov@arknetworks.am> References: <20230221004111.7945-1-ivan.malov@arknetworks.am> MIME-Version: 1.0 Cc: Eli Britstein , Ori Kam , David Marchand , Ilya Maximets Subject: [ovs-dev] [PATCH v4 2/3] netdev-offload-dpdk: replace action PORT_ID with REPRESENTED_PORT X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Ivan Malov via dev From: Ivan Malov Reply-To: Ivan Malov Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" Action PORT_ID has been deprecated. Use REPRESENTED_PORT instead. Signed-off-by: Ivan Malov --- lib/netdev-offload-dpdk.c | 31 +++++++++++++++++-------------- 1 file changed, 17 insertions(+), 14 deletions(-) diff --git a/lib/netdev-offload-dpdk.c b/lib/netdev-offload-dpdk.c index b3421c099..7f2598a53 100644 --- a/lib/netdev-offload-dpdk.c +++ b/lib/netdev-offload-dpdk.c @@ -735,14 +735,14 @@ dump_flow_action(struct ds *s, struct ds *s_extra, ds_put_cstr(s, "rss / "); } else if (actions->type == RTE_FLOW_ACTION_TYPE_COUNT) { ds_put_cstr(s, "count / "); - } else if (actions->type == RTE_FLOW_ACTION_TYPE_PORT_ID) { - const struct rte_flow_action_port_id *port_id = actions->conf; + } else if (actions->type == RTE_FLOW_ACTION_TYPE_REPRESENTED_PORT) { + const struct rte_flow_action_ethdev *ethdev = actions->conf; + + ds_put_cstr(s, "represented_port "); + + if (ethdev) + ds_put_format(s, "ethdev_port_id %d ", ethdev->port_id); - ds_put_cstr(s, "port_id "); - if (port_id) { - ds_put_format(s, "original %d id %d ", - port_id->original, port_id->id); - } ds_put_cstr(s, "/ "); } else if (actions->type == RTE_FLOW_ACTION_TYPE_DROP) { ds_put_cstr(s, "drop / "); @@ -1776,19 +1776,22 @@ add_count_action(struct flow_actions *actions) } static int -add_port_id_action(struct flow_actions *actions, - struct netdev *outdev) +add_represented_port_action(struct flow_actions *actions, + struct netdev *outdev) { - struct rte_flow_action_port_id *port_id; + struct rte_flow_action_ethdev *ethdev; int outdev_id; outdev_id = netdev_dpdk_get_port_id(outdev); if (outdev_id < 0) { return -1; } - port_id = xzalloc(sizeof *port_id); - port_id->id = outdev_id; - add_flow_action(actions, RTE_FLOW_ACTION_TYPE_PORT_ID, port_id); + + ethdev = xzalloc(sizeof *ethdev); + ethdev->port_id = outdev_id; + + add_flow_action(actions, RTE_FLOW_ACTION_TYPE_REPRESENTED_PORT, ethdev); + return 0; } @@ -1808,7 +1811,7 @@ add_output_action(struct netdev *netdev, return -1; } if (!netdev_flow_api_equals(netdev, outdev) || - add_port_id_action(actions, outdev)) { + add_represented_port_action(actions, outdev)) { VLOG_DBG_RL(&rl, "%s: Output to port \'%s\' cannot be offloaded.", netdev_get_name(netdev), netdev_get_name(outdev)); ret = -1; From patchwork Tue Feb 21 00:41:11 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ivan Malov X-Patchwork-Id: 1745356 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::136; helo=smtp3.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4PLLKb6VBnz23yV for ; Tue, 21 Feb 2023 11:49:55 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 1F4CB6109A; Tue, 21 Feb 2023 00:49:54 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 1F4CB6109A X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id QqvJtFcrLTYZ; Tue, 21 Feb 2023 00:49:51 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp3.osuosl.org (Postfix) with ESMTPS id 9034561020; Tue, 21 Feb 2023 00:49:50 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 9034561020 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id B2A4AC0083; Tue, 21 Feb 2023 00:49:47 +0000 (UTC) X-Original-To: ovs-dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by lists.linuxfoundation.org (Postfix) with ESMTP id 6C437C002B for ; Tue, 21 Feb 2023 00:49:45 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 54AC6828C5 for ; Tue, 21 Feb 2023 00:49:45 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 54AC6828C5 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id pE6va6fR2cR8 for ; Tue, 21 Feb 2023 00:49:44 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 018798238D Received: from agw.arknetworks.am (agw.arknetworks.am [79.141.165.80]) by smtp1.osuosl.org (Postfix) with ESMTPS id 018798238D for ; Tue, 21 Feb 2023 00:49:43 +0000 (UTC) Received: from localhost.localdomain (unknown [78.109.74.189]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by agw.arknetworks.am (Postfix) with ESMTPSA id 8088FE1341; Tue, 21 Feb 2023 04:40:54 +0400 (+04) To: ovs-dev@openvswitch.org Date: Tue, 21 Feb 2023 04:41:11 +0400 Message-Id: <20230221004111.7945-4-ivan.malov@arknetworks.am> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20230221004111.7945-1-ivan.malov@arknetworks.am> References: <20230221004111.7945-1-ivan.malov@arknetworks.am> MIME-Version: 1.0 Cc: Eli Britstein , Ori Kam , David Marchand , Ilya Maximets Subject: [ovs-dev] [PATCH v4 3/3] netdev-offload-dpdk: use flow transfer proxy mechanism X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Ivan Malov via dev From: Ivan Malov Reply-To: Ivan Malov Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" Manage "transfer" flows via the corresponding mechanism. Doing so requires that the traffic source be specified explicitly, via the corresponding pattern item. Signed-off-by: Ivan Malov --- lib/netdev-dpdk.c | 88 +++++++++++++++++++++++++++++++-------- lib/netdev-dpdk.h | 4 +- lib/netdev-offload-dpdk.c | 55 +++++++++++++++++++----- 3 files changed, 117 insertions(+), 30 deletions(-) diff --git a/lib/netdev-dpdk.c b/lib/netdev-dpdk.c index 2cebc3cca..3a9c9d9a0 100644 --- a/lib/netdev-dpdk.c +++ b/lib/netdev-dpdk.c @@ -434,6 +434,7 @@ enum dpdk_hw_ol_features { struct netdev_dpdk { PADDED_MEMBERS_CACHELINE_MARKER(CACHE_LINE_SIZE, cacheline0, + dpdk_port_t flow_transfer_proxy_port_id; dpdk_port_t port_id; /* If true, device was attached by rte_eth_dev_attach(). */ @@ -1183,6 +1184,7 @@ dpdk_eth_dev_init(struct netdev_dpdk *dev) uint32_t rx_chksm_offload_capa = RTE_ETH_RX_OFFLOAD_UDP_CKSUM | RTE_ETH_RX_OFFLOAD_TCP_CKSUM | RTE_ETH_RX_OFFLOAD_IPV4_CKSUM; + int ret; /* * Full tunnel offload requires that tunnel ID metadata be @@ -1194,6 +1196,24 @@ dpdk_eth_dev_init(struct netdev_dpdk *dev) */ dpdk_eth_dev_init_rx_metadata(dev); + /* + * Managing "transfer" flows requires that the user communicate them + * via a port which has the privilege to control the embedded switch. + * For some vendors, all ports in a given switching domain have + * this privilege. For other vendors, it's only one port. + * + * Get the proxy port ID and remember it for later use. + */ + ret = rte_flow_pick_transfer_proxy(dev->port_id, + &dev->flow_transfer_proxy_port_id, NULL); + if (ret != 0) { + /* + * The PMD does not indicate the proxy port. + * Assume the proxy is unneeded. + */ + dev->flow_transfer_proxy_port_id = dev->port_id; + } + rte_eth_dev_info_get(dev->port_id, &info); if (strstr(info.driver_name, "vf") != NULL) { @@ -3981,8 +4001,10 @@ netdev_dpdk_detach(struct unixctl_conn *conn, int argc OVS_UNUSED, const char *argv[], void *aux OVS_UNUSED) { struct ds used_interfaces = DS_EMPTY_INITIALIZER; + struct netdev_dpdk *dev_self = NULL; struct rte_eth_dev_info dev_info; dpdk_port_t sibling_port_id; + struct netdev_dpdk *dev; dpdk_port_t port_id; bool used = false; char *response; @@ -4000,8 +4022,6 @@ netdev_dpdk_detach(struct unixctl_conn *conn, int argc OVS_UNUSED, argv[1]); RTE_ETH_FOREACH_DEV_SIBLING (sibling_port_id, port_id) { - struct netdev_dpdk *dev; - LIST_FOR_EACH (dev, list_node, &dpdk_list) { if (dev->port_id != sibling_port_id) { continue; @@ -4021,6 +4041,25 @@ netdev_dpdk_detach(struct unixctl_conn *conn, int argc OVS_UNUSED, } ds_destroy(&used_interfaces); + /* + * The device being detached may happen to be a flow proxy port + * for another device (still attached). If so, do not allow to + * detach. Devices dependent on this one must be detached first. + */ + LIST_FOR_EACH (dev, list_node, &dpdk_list) { + if (dev->port_id == port_id) { + dev_self = dev; + } else if (dev->flow_transfer_proxy_port_id == port_id) { + response = xasprintf("Device '%s' can not be detached (flow proxy)", + argv[1]); + goto error; + } + } + + /* Indicate that the device being detached no longer needs a flow proxy. */ + if (dev_self != NULL) + dev_self->flow_transfer_proxy_port_id = port_id; + rte_eth_dev_info_get(port_id, &dev_info); rte_eth_dev_close(port_id); if (rte_dev_remove(dev_info.device) < 0) { @@ -5470,7 +5509,8 @@ unlock: } int -netdev_dpdk_get_port_id(struct netdev *netdev) +netdev_dpdk_get_port_id(struct netdev *netdev, + bool flow_transfer_proxy) { struct netdev_dpdk *dev; int ret = -1; @@ -5481,7 +5521,7 @@ netdev_dpdk_get_port_id(struct netdev *netdev) dev = netdev_dpdk_cast(netdev); ovs_mutex_lock(&dev->mutex); - ret = dev->port_id; + ret = flow_transfer_proxy ? dev->flow_transfer_proxy_port_id : dev->port_id; ovs_mutex_unlock(&dev->mutex); out: return ret; @@ -5517,13 +5557,15 @@ out: int netdev_dpdk_rte_flow_destroy(struct netdev *netdev, - struct rte_flow *rte_flow, + bool transfer, struct rte_flow *rte_flow, struct rte_flow_error *error) { struct netdev_dpdk *dev = netdev_dpdk_cast(netdev); int ret; - ret = rte_flow_destroy(dev->port_id, rte_flow, error); + ret = rte_flow_destroy(transfer ? + dev->flow_transfer_proxy_port_id : dev->port_id, + rte_flow, error); return ret; } @@ -5537,7 +5579,17 @@ netdev_dpdk_rte_flow_create(struct netdev *netdev, struct rte_flow *flow; struct netdev_dpdk *dev = netdev_dpdk_cast(netdev); - flow = rte_flow_create(dev->port_id, attr, items, actions, error); + if (!attr->transfer) { + /* + * The 1st item in any pattern is a traffic source one. + * It is unnecessary in the case of non-transfer rules. + */ + ++(items); + } + + flow = rte_flow_create(attr->transfer ? + dev->flow_transfer_proxy_port_id : dev->port_id, + attr, items, actions, error); return flow; } @@ -5565,7 +5617,8 @@ netdev_dpdk_rte_flow_query_count(struct netdev *netdev, } dev = netdev_dpdk_cast(netdev); - ret = rte_flow_query(dev->port_id, rte_flow, actions, query, error); + ret = rte_flow_query(dev->flow_transfer_proxy_port_id, rte_flow, + actions, query, error); return ret; } @@ -5587,8 +5640,8 @@ netdev_dpdk_rte_flow_tunnel_decap_set(struct netdev *netdev, dev = netdev_dpdk_cast(netdev); ovs_mutex_lock(&dev->mutex); - ret = rte_flow_tunnel_decap_set(dev->port_id, tunnel, actions, - num_of_actions, error); + ret = rte_flow_tunnel_decap_set(dev->flow_transfer_proxy_port_id, tunnel, + actions, num_of_actions, error); ovs_mutex_unlock(&dev->mutex); return ret; } @@ -5609,8 +5662,8 @@ netdev_dpdk_rte_flow_tunnel_match(struct netdev *netdev, dev = netdev_dpdk_cast(netdev); ovs_mutex_lock(&dev->mutex); - ret = rte_flow_tunnel_match(dev->port_id, tunnel, items, num_of_items, - error); + ret = rte_flow_tunnel_match(dev->flow_transfer_proxy_port_id, tunnel, + items, num_of_items, error); ovs_mutex_unlock(&dev->mutex); return ret; } @@ -5631,7 +5684,8 @@ netdev_dpdk_rte_flow_get_restore_info(struct netdev *netdev, dev = netdev_dpdk_cast(netdev); ovs_mutex_lock(&dev->mutex); - ret = rte_flow_get_restore_info(dev->port_id, m, info, error); + ret = rte_flow_get_restore_info(dev->flow_transfer_proxy_port_id, + m, info, error); ovs_mutex_unlock(&dev->mutex); return ret; } @@ -5652,8 +5706,8 @@ netdev_dpdk_rte_flow_tunnel_action_decap_release( dev = netdev_dpdk_cast(netdev); ovs_mutex_lock(&dev->mutex); - ret = rte_flow_tunnel_action_decap_release(dev->port_id, actions, - num_of_actions, error); + ret = rte_flow_tunnel_action_decap_release(dev->flow_transfer_proxy_port_id, + actions, num_of_actions, error); ovs_mutex_unlock(&dev->mutex); return ret; } @@ -5673,8 +5727,8 @@ netdev_dpdk_rte_flow_tunnel_item_release(struct netdev *netdev, dev = netdev_dpdk_cast(netdev); ovs_mutex_lock(&dev->mutex); - ret = rte_flow_tunnel_item_release(dev->port_id, items, num_of_items, - error); + ret = rte_flow_tunnel_item_release(dev->flow_transfer_proxy_port_id, + items, num_of_items, error); ovs_mutex_unlock(&dev->mutex); return ret; } diff --git a/lib/netdev-dpdk.h b/lib/netdev-dpdk.h index 5cd95d00f..80b616dae 100644 --- a/lib/netdev-dpdk.h +++ b/lib/netdev-dpdk.h @@ -36,7 +36,7 @@ bool netdev_dpdk_flow_api_supported(struct netdev *); int netdev_dpdk_rte_flow_destroy(struct netdev *netdev, - struct rte_flow *rte_flow, + bool transfer, struct rte_flow *rte_flow, struct rte_flow_error *error); struct rte_flow * netdev_dpdk_rte_flow_create(struct netdev *netdev, @@ -50,7 +50,7 @@ netdev_dpdk_rte_flow_query_count(struct netdev *netdev, struct rte_flow_query_count *query, struct rte_flow_error *error); int -netdev_dpdk_get_port_id(struct netdev *netdev); +netdev_dpdk_get_port_id(struct netdev *netdev, bool flow_transfer_proxy); #ifdef ALLOW_EXPERIMENTAL_API diff --git a/lib/netdev-offload-dpdk.c b/lib/netdev-offload-dpdk.c index 7f2598a53..890c0ada6 100644 --- a/lib/netdev-offload-dpdk.c +++ b/lib/netdev-offload-dpdk.c @@ -355,8 +355,18 @@ dump_flow_pattern(struct ds *s, if (item->type == RTE_FLOW_ITEM_TYPE_END) { ds_put_cstr(s, "end "); + } else if (item->type == RTE_FLOW_ITEM_TYPE_REPRESENTED_PORT) { + const struct rte_flow_item_ethdev *ethdev_spec = item->spec; + const struct rte_flow_item_ethdev *ethdev_mask = item->mask; + + ds_put_cstr(s, "represented_port "); + + DUMP_PATTERN_ITEM(ethdev_mask->port_id, false, "ethdev_port_id", + "%"PRIu16, ethdev_spec->port_id, + ethdev_mask->port_id, 0); } else if (flow_patterns->tnl_pmd_items_cnt && - pattern_index < flow_patterns->tnl_pmd_items_cnt) { + pattern_index < 1 /* REPRESENTED_PORT */ + + flow_patterns->tnl_pmd_items_cnt) { return; } else if (item->type == RTE_FLOW_ITEM_TYPE_ETH) { const struct rte_flow_item_eth *eth_spec = item->spec; @@ -882,7 +892,12 @@ dump_flow(struct ds *s, struct ds *s_extra, dump_flow_attr(s, s_extra, attr, flow_patterns, flow_actions); } ds_put_cstr(s, "pattern "); - for (i = 0; i < flow_patterns->cnt; i++) { + /* + * The 1st item in any pattern is a traffic source one. + * It is skipped in the case of non-transfer rules and + * must not be indicated when printing such rules, too. + */ + for (i = 1 - attr->transfer; i < flow_patterns->cnt; i++) { dump_flow_pattern(s, flow_patterns, i); } ds_put_cstr(s, "actions "); @@ -920,7 +935,8 @@ netdev_offload_dpdk_flow_create(struct netdev *netdev, extra_str = ds_cstr(&s_extra); VLOG_DBG_RL(&rl, "%s: rte_flow 0x%"PRIxPTR" %s flow create %d %s", netdev_get_name(netdev), (intptr_t) flow, extra_str, - netdev_dpdk_get_port_id(netdev), ds_cstr(&s)); + netdev_dpdk_get_port_id(netdev, attr->transfer), + ds_cstr(&s)); } } else { enum vlog_level level = VLL_WARN; @@ -935,7 +951,8 @@ netdev_offload_dpdk_flow_create(struct netdev *netdev, extra_str = ds_cstr(&s_extra); VLOG_RL(&rl, level, "%s: Failed flow: %s flow create %d %s", netdev_get_name(netdev), extra_str, - netdev_dpdk_get_port_id(netdev), ds_cstr(&s)); + netdev_dpdk_get_port_id(netdev, attr->transfer), + ds_cstr(&s)); } } ds_destroy(&s); @@ -1028,6 +1045,10 @@ free_flow_patterns(struct flow_patterns *patterns) struct rte_flow_error error; int i; + /* REPRESENTED_PORT */ + free(CONST_CAST(void *, patterns->items[0].spec)); + free(CONST_CAST(void *, patterns->items[0].mask)); + if (patterns->tnl_pmd_items) { struct rte_flow_item *tnl_pmd_items = patterns->tnl_pmd_items; uint32_t tnl_pmd_items_cnt = patterns->tnl_pmd_items_cnt; @@ -1042,7 +1063,8 @@ free_flow_patterns(struct flow_patterns *patterns) } } - for (i = patterns->tnl_pmd_items_cnt; i < patterns->cnt; i++) { + for (i = 1 /* REPRESENTED_PORT */ + patterns->tnl_pmd_items_cnt; + i < patterns->cnt; i++) { if (patterns->items[i].spec) { free(CONST_CAST(void *, patterns->items[i].spec)); } @@ -1114,13 +1136,13 @@ vport_to_rte_tunnel(struct netdev *vport, tunnel->tp_dst = tnl_cfg->dst_port; if (!VLOG_DROP_DBG(&rl)) { ds_put_format(s_tnl, "flow tunnel create %d type vxlan; ", - netdev_dpdk_get_port_id(netdev)); + netdev_dpdk_get_port_id(netdev, true)); } } else if (!strcmp(netdev_get_type(vport), "gre")) { tunnel->type = RTE_FLOW_ITEM_TYPE_GRE; if (!VLOG_DROP_DBG(&rl)) { ds_put_format(s_tnl, "flow tunnel create %d type gre; ", - netdev_dpdk_get_port_id(netdev)); + netdev_dpdk_get_port_id(netdev, true)); } } else { VLOG_DBG_RL(&rl, "vport type '%s' is not supported", @@ -1382,10 +1404,19 @@ parse_flow_match(struct netdev *netdev, struct flow_patterns *patterns, struct match *match) { + struct netdev *physdev = netdev_ports_get(orig_in_port, netdev->dpif_type); + struct rte_flow_item_ethdev *ethdev_spec = xzalloc(sizeof *ethdev_spec); + struct rte_flow_item_ethdev *ethdev_mask = xzalloc(sizeof *ethdev_mask); struct rte_flow_item_eth *eth_spec = NULL, *eth_mask = NULL; struct flow *consumed_masks; uint8_t proto = 0; + /* Add an explicit traffic source item to the beginning of the pattern. */ + ethdev_spec->port_id = netdev_dpdk_get_port_id(physdev, false); + *ethdev_mask = rte_flow_item_ethdev_mask; + add_flow_pattern(patterns, RTE_FLOW_ITEM_TYPE_REPRESENTED_PORT, + ethdev_spec, ethdev_mask, NULL); + consumed_masks = &match->wc.masks; if (!flow_tnl_dst_is_set(&match->flow.tunnel)) { @@ -1782,7 +1813,7 @@ add_represented_port_action(struct flow_actions *actions, struct rte_flow_action_ethdev *ethdev; int outdev_id; - outdev_id = netdev_dpdk_get_port_id(outdev); + outdev_id = netdev_dpdk_get_port_id(outdev, false); if (outdev_id < 0) { return -1; } @@ -2322,6 +2353,7 @@ netdev_offload_dpdk_flow_destroy(struct ufid_to_rte_flow_data *rte_flow_data) struct netdev *physdev; struct netdev *netdev; ovs_u128 *ufid; + bool transfer; int ret; ovs_mutex_lock(&rte_flow_data->lock); @@ -2333,12 +2365,13 @@ netdev_offload_dpdk_flow_destroy(struct ufid_to_rte_flow_data *rte_flow_data) rte_flow_data->dead = true; + transfer = rte_flow_data->actions_offloaded; rte_flow = rte_flow_data->rte_flow; physdev = rte_flow_data->physdev; netdev = rte_flow_data->netdev; ufid = &rte_flow_data->ufid; - ret = netdev_dpdk_rte_flow_destroy(physdev, rte_flow, &error); + ret = netdev_dpdk_rte_flow_destroy(physdev, transfer, rte_flow, &error); if (ret == 0) { struct netdev_offload_dpdk_data *data; @@ -2353,12 +2386,12 @@ netdev_offload_dpdk_flow_destroy(struct ufid_to_rte_flow_data *rte_flow_data) " flow destroy %d ufid " UUID_FMT, netdev_get_name(netdev), netdev_get_name(physdev), (intptr_t) rte_flow, - netdev_dpdk_get_port_id(physdev), + netdev_dpdk_get_port_id(physdev, transfer), UUID_ARGS((struct uuid *) ufid)); } else { VLOG_ERR("Failed flow: %s/%s: flow destroy %d ufid " UUID_FMT, netdev_get_name(netdev), netdev_get_name(physdev), - netdev_dpdk_get_port_id(physdev), + netdev_dpdk_get_port_id(physdev, transfer), UUID_ARGS((struct uuid *) ufid)); }