From patchwork Mon Jul 12 15:07:08 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eli Britstein X-Patchwork-Id: 1504040 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::133; helo=smtp2.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp2.osuosl.org (smtp2.osuosl.org [IPv6:2605:bc80:3010::133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4GNnDy2d0lz9sWc for ; Tue, 13 Jul 2021 01:07:29 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 0468D40385; Mon, 12 Jul 2021 15:07:27 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 2zzrHg9BrZvF; Mon, 12 Jul 2021 15:07:26 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp2.osuosl.org (Postfix) with ESMTPS id 2B93D4002B; Mon, 12 Jul 2021 15:07:25 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id BDB67C001A; Mon, 12 Jul 2021 15:07:24 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp1.osuosl.org (smtp1.osuosl.org [IPv6:2605:bc80:3010::138]) by lists.linuxfoundation.org (Postfix) with ESMTP id 52692C000E for ; Mon, 12 Jul 2021 15:07:23 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 342FD83227 for ; Mon, 12 Jul 2021 15:07:23 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id g3zPRWCupIKl for ; Mon, 12 Jul 2021 15:07:21 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by smtp1.osuosl.org (Postfix) with ESMTP id 3662A831E3 for ; Mon, 12 Jul 2021 15:07:20 +0000 (UTC) Received: from Internal Mail-Server by MTLPINE1 (envelope-from elibr@nvidia.com) with SMTP; 12 Jul 2021 18:07:18 +0300 Received: from nvidia.com (dev-r-vrt-214.mtr.labs.mlnx [10.212.214.1]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 16CF7IU5022570; Mon, 12 Jul 2021 18:07:18 +0300 From: Eli Britstein To: dev@openvswitch.org, Ilya Maximets Date: Mon, 12 Jul 2021 15:07:08 +0000 Message-Id: <20210712150710.19197-2-elibr@nvidia.com> X-Mailer: git-send-email 2.28.0.2311.g225365fb51 In-Reply-To: <20210712150710.19197-1-elibr@nvidia.com> References: <20210712150710.19197-1-elibr@nvidia.com> MIME-Version: 1.0 Cc: Eli Britstein , David Marchand , Majd Dibbiny Subject: [ovs-dev] [PATCH V2 1/3] dpif-netdev: Do not flush PMD offloads on reload X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" Before flushing offloads of a removed port was supported by [1], it was necessary to flush the 'marks'. In doing so, all offloads of the PMD are removed, include the ones that are not related to the removed port and that are not modified following this removal. As a result such flows are evicted from being offloaded, and won't resume offloading. As PMD offload flush is not necessary, avoid it. [1] 62d1c28e9ce0 ("dpif-netdev: Flush offload rules upon port deletion.") Signed-off-by: Eli Britstein Reviewed-by: Gaetan Rivet Reviewed-by: David Marchand --- lib/dpif-netdev.c | 13 ------------- 1 file changed, 13 deletions(-) diff --git a/lib/dpif-netdev.c b/lib/dpif-netdev.c index 610949f36..21b0e025d 100644 --- a/lib/dpif-netdev.c +++ b/lib/dpif-netdev.c @@ -2232,18 +2232,6 @@ mark_to_flow_disassociate(struct dp_netdev_pmd_thread *pmd, return ret; } -static void -flow_mark_flush(struct dp_netdev_pmd_thread *pmd) -{ - struct dp_netdev_flow *flow; - - CMAP_FOR_EACH (flow, mark_node, &flow_mark.mark_to_flow) { - if (flow->pmd_id == pmd->core_id) { - queue_netdev_flow_del(pmd, flow); - } - } -} - static struct dp_netdev_flow * mark_to_flow_find(const struct dp_netdev_pmd_thread *pmd, const uint32_t mark) @@ -4811,7 +4799,6 @@ reload_affected_pmds(struct dp_netdev *dp) CMAP_FOR_EACH (pmd, node, &dp->poll_threads) { if (pmd->need_reload) { - flow_mark_flush(pmd); dp_netdev_reload_pmd__(pmd); } } From patchwork Mon Jul 12 15:07:09 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eli Britstein X-Patchwork-Id: 1504043 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.133; helo=smtp2.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4GNnF70KPZz9sWc for ; Tue, 13 Jul 2021 01:07:39 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 6A22D403D6; Mon, 12 Jul 2021 15:07:32 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id bMuIwDIRXYng; Mon, 12 Jul 2021 15:07:31 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp2.osuosl.org (Postfix) with ESMTPS id 4933F403CB; Mon, 12 Jul 2021 15:07:30 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 845F9C002C; Mon, 12 Jul 2021 15:07:27 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp2.osuosl.org (smtp2.osuosl.org [IPv6:2605:bc80:3010::133]) by lists.linuxfoundation.org (Postfix) with ESMTP id DF11DC0026 for ; Mon, 12 Jul 2021 15:07:25 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id CE249400E9 for ; Mon, 12 Jul 2021 15:07:24 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id BoC-xUsbe5hh for ; Mon, 12 Jul 2021 15:07:21 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by smtp2.osuosl.org (Postfix) with ESMTP id 367C44002B for ; Mon, 12 Jul 2021 15:07:20 +0000 (UTC) Received: from Internal Mail-Server by MTLPINE1 (envelope-from elibr@nvidia.com) with SMTP; 12 Jul 2021 18:07:18 +0300 Received: from nvidia.com (dev-r-vrt-214.mtr.labs.mlnx [10.212.214.1]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 16CF7IU6022570; Mon, 12 Jul 2021 18:07:18 +0300 From: Eli Britstein To: dev@openvswitch.org, Ilya Maximets Date: Mon, 12 Jul 2021 15:07:09 +0000 Message-Id: <20210712150710.19197-3-elibr@nvidia.com> X-Mailer: git-send-email 2.28.0.2311.g225365fb51 In-Reply-To: <20210712150710.19197-1-elibr@nvidia.com> References: <20210712150710.19197-1-elibr@nvidia.com> MIME-Version: 1.0 Cc: Eli Britstein , David Marchand , Majd Dibbiny Subject: [ovs-dev] [PATCH V2 2/3] dpif-netdev: Fix offloads of modified flows X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" Association of a mark to a flow is done as part of its offload handling, in the offloading thread. However, the PMD thread specifies whether an offload request is an "add" or "modify" by the association of a mark to the flow. This is exposed to a race condition. A flow might be created with actions that cannot be fully offloaded, for example flooding (before MAC learning), and later modified to have actions that can be fully offloaded. If the two requests are queued before the offload thread handling, they are both marked as "add". When the offload thread handles them, the first request is partially offloaded, and the second one is ignored as the flow is already considered as offloaded. Fix it by specifying add/modify of an offload request by the actual flow state change, without relying on the mark. Fixes: 3c7330ebf036 ("netdev-offload-dpdk: Support offload of output action.") Signed-off-by: Eli Britstein Reviewed-by: Gaetan Rivet Reviewed-by: David Marchand --- lib/dpif-netdev.c | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/lib/dpif-netdev.c b/lib/dpif-netdev.c index 21b0e025d..9b2b8d6d9 100644 --- a/lib/dpif-netdev.c +++ b/lib/dpif-netdev.c @@ -2451,7 +2451,8 @@ static void queue_netdev_flow_put(struct dp_netdev_pmd_thread *pmd, struct dp_netdev_flow *flow, struct match *match, const struct nlattr *actions, size_t actions_len, - odp_port_t orig_in_port) + odp_port_t orig_in_port, + const struct dp_netdev_actions *old_actions) { struct dp_flow_offload_item *offload; int op; @@ -2467,11 +2468,9 @@ queue_netdev_flow_put(struct dp_netdev_pmd_thread *pmd, ovsthread_once_done(&offload_thread_once); } - if (flow->mark != INVALID_FLOW_MARK) { - op = DP_NETDEV_FLOW_OFFLOAD_OP_MOD; - } else { - op = DP_NETDEV_FLOW_OFFLOAD_OP_ADD; - } + op = old_actions + ? DP_NETDEV_FLOW_OFFLOAD_OP_MOD + : DP_NETDEV_FLOW_OFFLOAD_OP_ADD; offload = dp_netdev_alloc_flow_offload(pmd, flow, op); offload->match = *match; offload->actions = xmalloc(actions_len); @@ -3323,7 +3322,7 @@ dp_netdev_flow_add(struct dp_netdev_pmd_thread *pmd, dp_netdev_flow_hash(&flow->ufid)); queue_netdev_flow_put(pmd, flow, match, actions, actions_len, - orig_in_port); + orig_in_port, NULL); if (OVS_UNLIKELY(!VLOG_DROP_DBG((&upcall_rl)))) { struct ds ds = DS_EMPTY_INITIALIZER; @@ -3410,7 +3409,8 @@ flow_put_on_pmd(struct dp_netdev_pmd_thread *pmd, ovsrcu_set(&netdev_flow->actions, new_actions); queue_netdev_flow_put(pmd, netdev_flow, match, - put->actions, put->actions_len, ODPP_NONE); + put->actions, put->actions_len, ODPP_NONE, + old_actions); if (stats) { get_dpif_flow_status(pmd->dp, netdev_flow, stats, NULL); From patchwork Mon Jul 12 15:07:10 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eli Britstein X-Patchwork-Id: 1504041 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.133; helo=smtp2.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4GNnF05Q8Sz9sWc for ; Tue, 13 Jul 2021 01:07:32 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 505D340383; Mon, 12 Jul 2021 15:07:29 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id J3XNIqX5wOj8; Mon, 12 Jul 2021 15:07:28 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp2.osuosl.org (Postfix) with ESMTPS id 6FF0B4038A; Mon, 12 Jul 2021 15:07:27 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id BC606C0024; Mon, 12 Jul 2021 15:07:25 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp2.osuosl.org (smtp2.osuosl.org [IPv6:2605:bc80:3010::133]) by lists.linuxfoundation.org (Postfix) with ESMTP id 6104DC001A for ; Mon, 12 Jul 2021 15:07:23 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 43352400E9 for ; Mon, 12 Jul 2021 15:07:23 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id wDJ6w_vKmqwy for ; Mon, 12 Jul 2021 15:07:21 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by smtp2.osuosl.org (Postfix) with ESMTP id 3BAAB4018F for ; Mon, 12 Jul 2021 15:07:20 +0000 (UTC) Received: from Internal Mail-Server by MTLPINE1 (envelope-from elibr@nvidia.com) with SMTP; 12 Jul 2021 18:07:18 +0300 Received: from nvidia.com (dev-r-vrt-214.mtr.labs.mlnx [10.212.214.1]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 16CF7IU7022570; Mon, 12 Jul 2021 18:07:18 +0300 From: Eli Britstein To: dev@openvswitch.org, Ilya Maximets Date: Mon, 12 Jul 2021 15:07:10 +0000 Message-Id: <20210712150710.19197-4-elibr@nvidia.com> X-Mailer: git-send-email 2.28.0.2311.g225365fb51 In-Reply-To: <20210712150710.19197-1-elibr@nvidia.com> References: <20210712150710.19197-1-elibr@nvidia.com> MIME-Version: 1.0 Cc: Eli Britstein , David Marchand , Majd Dibbiny Subject: [ovs-dev] [PATCH V2 3/3] dpif-netdev: Log flow modification in debug level X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" Log flow modifications to help debugging. Signed-off-by: Eli Britstein Reviewed-by: Gaetan Rivet --- lib/dpif-netdev.c | 101 +++++++++++++++++++++++++--------------------- 1 file changed, 55 insertions(+), 46 deletions(-) diff --git a/lib/dpif-netdev.c b/lib/dpif-netdev.c index 9b2b8d6d9..caed3e7f2 100644 --- a/lib/dpif-netdev.c +++ b/lib/dpif-netdev.c @@ -2457,6 +2457,61 @@ queue_netdev_flow_put(struct dp_netdev_pmd_thread *pmd, struct dp_flow_offload_item *offload; int op; + if (OVS_UNLIKELY(!VLOG_DROP_DBG((&upcall_rl)))) { + struct ds ds = DS_EMPTY_INITIALIZER; + struct ofpbuf key_buf, mask_buf; + struct odp_flow_key_parms odp_parms = { + .flow = &match->flow, + .mask = &match->wc.masks, + .support = dp_netdev_support, + }; + + ofpbuf_init(&key_buf, 0); + ofpbuf_init(&mask_buf, 0); + + odp_flow_key_from_flow(&odp_parms, &key_buf); + odp_parms.key_buf = &key_buf; + odp_flow_key_from_mask(&odp_parms, &mask_buf); + + if (old_actions) { + ds_put_cstr(&ds, "flow_mod: "); + } else { + ds_put_cstr(&ds, "flow_add: "); + } + odp_format_ufid(&flow->ufid, &ds); + ds_put_cstr(&ds, " mega_"); + odp_format_ufid(&flow->mega_ufid, &ds); + ds_put_cstr(&ds, " "); + odp_flow_format(key_buf.data, key_buf.size, + mask_buf.data, mask_buf.size, + NULL, &ds, false); + if (old_actions) { + ds_put_cstr(&ds, ", old_actions:"); + format_odp_actions(&ds, old_actions->actions, old_actions->size, + NULL); + } + ds_put_cstr(&ds, ", actions:"); + format_odp_actions(&ds, actions, actions_len, NULL); + + VLOG_DBG("%s", ds_cstr(&ds)); + + ofpbuf_uninit(&key_buf); + ofpbuf_uninit(&mask_buf); + + /* Add a printout of the actual match installed. */ + struct match m; + ds_clear(&ds); + ds_put_cstr(&ds, "flow match: "); + miniflow_expand(&flow->cr.flow.mf, &m.flow); + miniflow_expand(&flow->cr.mask->mf, &m.wc.masks); + memset(&m.tun_md, 0, sizeof m.tun_md); + match_format(&m, NULL, &ds, OFP_DEFAULT_PRIORITY); + + VLOG_DBG("%s", ds_cstr(&ds)); + + ds_destroy(&ds); + } + if (!netdev_is_flow_api_enabled()) { return; } @@ -3324,52 +3379,6 @@ dp_netdev_flow_add(struct dp_netdev_pmd_thread *pmd, queue_netdev_flow_put(pmd, flow, match, actions, actions_len, orig_in_port, NULL); - if (OVS_UNLIKELY(!VLOG_DROP_DBG((&upcall_rl)))) { - struct ds ds = DS_EMPTY_INITIALIZER; - struct ofpbuf key_buf, mask_buf; - struct odp_flow_key_parms odp_parms = { - .flow = &match->flow, - .mask = &match->wc.masks, - .support = dp_netdev_support, - }; - - ofpbuf_init(&key_buf, 0); - ofpbuf_init(&mask_buf, 0); - - odp_flow_key_from_flow(&odp_parms, &key_buf); - odp_parms.key_buf = &key_buf; - odp_flow_key_from_mask(&odp_parms, &mask_buf); - - ds_put_cstr(&ds, "flow_add: "); - odp_format_ufid(ufid, &ds); - ds_put_cstr(&ds, " mega_"); - odp_format_ufid(&flow->mega_ufid, &ds); - ds_put_cstr(&ds, " "); - odp_flow_format(key_buf.data, key_buf.size, - mask_buf.data, mask_buf.size, - NULL, &ds, false); - ds_put_cstr(&ds, ", actions:"); - format_odp_actions(&ds, actions, actions_len, NULL); - - VLOG_DBG("%s", ds_cstr(&ds)); - - ofpbuf_uninit(&key_buf); - ofpbuf_uninit(&mask_buf); - - /* Add a printout of the actual match installed. */ - struct match m; - ds_clear(&ds); - ds_put_cstr(&ds, "flow match: "); - miniflow_expand(&flow->cr.flow.mf, &m.flow); - miniflow_expand(&flow->cr.mask->mf, &m.wc.masks); - memset(&m.tun_md, 0, sizeof m.tun_md); - match_format(&m, NULL, &ds, OFP_DEFAULT_PRIORITY); - - VLOG_DBG("%s", ds_cstr(&ds)); - - ds_destroy(&ds); - } - return flow; }