From patchwork Mon Mar 28 05:39:37 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Han Zhou X-Patchwork-Id: 1610005 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.138; helo=smtp1.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4KRhNk26Sbz9sFy for ; Mon, 28 Mar 2022 16:40:04 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 74EE782591; Mon, 28 Mar 2022 05:40:02 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id YU_rPfjZznj9; Mon, 28 Mar 2022 05:40:01 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp1.osuosl.org (Postfix) with ESMTPS id 8E8DB81A5C; Mon, 28 Mar 2022 05:40:00 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 4CE3DC0033; Mon, 28 Mar 2022 05:40:00 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp4.osuosl.org (smtp4.osuosl.org [140.211.166.137]) by lists.linuxfoundation.org (Postfix) with ESMTP id B6276C0012 for ; Mon, 28 Mar 2022 05:39:58 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 96B59408B2 for ; Mon, 28 Mar 2022 05:39:58 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Mp8G4ZyoLLtQ for ; Mon, 28 Mar 2022 05:39:57 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from relay3-d.mail.gandi.net (relay3-d.mail.gandi.net [217.70.183.195]) by smtp4.osuosl.org (Postfix) with ESMTPS id 19E74408A1 for ; Mon, 28 Mar 2022 05:39:56 +0000 (UTC) Received: (Authenticated sender: hzhou@ovn.org) by mail.gandi.net (Postfix) with ESMTPSA id 5138460008; Mon, 28 Mar 2022 05:39:52 +0000 (UTC) From: Han Zhou To: dev@openvswitch.org Date: Sun, 27 Mar 2022 22:39:37 -0700 Message-Id: <20220328053943.1606715-1-hzhou@ovn.org> X-Mailer: git-send-email 2.30.2 MIME-Version: 1.0 Subject: [ovs-dev] [PATCH ovn v4 0/6] Use ct_mark for masked access to make flows HW-offloading friendly. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" Some NICs support HW offloading for datapath flows, but masked access to the 128-bit ct_label field may prevent a flow being offloaded due to HW limitations. OVN's use of ct_label currently includes: - ct_label.blocked (1 bit) - ct_label.natted (1 bit) - ct_label.ecmp_reply_port (16 bits) - ct_label.ecmp_reply_eth (48 bits) - ct_label.label (32 bits) This patch moves the bits blocked, natted and ecmp_reply_port to use ct_mark (18 bits in total among the 32-bit ct_mark), and keep the rest of the fields in ct_label: - ct_mark.blocked (1 bit) - ct_mark.natted (1 bit) - ct_mark.ecmp_reply_port (16 bits) - ct_label.ecmp_reply_eth (48 bits) - ct_label.label (32 bits) This would allow HW offloading to work for most of the cases. For ct_label.ecmp_reply_eth, the flow matching it still uses masked access, but it doesn't matter because the flow is for new connections and requires ct_commit in its actions, so it wouldn't be offloaded anyway for those NICs. There is a flow for established connections that would access the masked field in the actions, while in this patch it avoids masked access by using a register xxreg1 to temporarily read the whole ct_label, and then use masked access to xxreg1 to read the actual value. The only exception is for ct_label.label, there is a flow that matches the masked field for ACL logging of reply direction. This patch cannot avoid the masked access to ct_label in this case. This flow is enabled only for the feature "log-related". So offloading may still not work for some NICs when an ACL is configured with a label and with "log-related" enabled. There are no other flows relying on masked ct_label match, but it's worth noting that the LB hairpin related flows using ct_label.natted which were hardcoded directly in ovn-controller are still kept to avoid traffic breaking during upgrading. It relies on the northd-internal-version to internally determine if it is currently upgrading from a version that requires the ct_label flows being kept, and automatically removes the flows when northd-internal-version is up-to-date. Reported-at: https://bugzilla.redhat.com/show_bug.cgi?id=1957786 v1 -> v2: Fixed two system test cases. v2 -> v3: - Addressed Numan's comment regarding hairpin flow upgrading problem by keeping ct_label flows together with ct_mark flows for hairpin, and provided an option to disable ct_label after upgrading. - Moved a misplaced chunk from patch2 to patch5 to fix related system tests. v3 -> v4: - Avoid the extra option to disable ct_label after upgrading for hairpin flows. Instead, reuse the north-internal-ver to determine automatically. For this to work, a new patch is added to handle SB_Global changes in I-P engine of ovn-controller. Han Zhou (6): ovn-sb.xml: Fix ct_lb documentation. actions: Add action ct_lb_mark. actions: Add stack push and pop actions. ovn-northd: Improve the doc and tests for ecmp-symmetric-reply. ovn-controller: Handle SB_Global:options:northd_internal_version in I-P engine. Use ct_mark for masked access to make flows HW-offloading friendly. NEWS | 2 + controller/lflow.c | 33 ++- controller/lflow.h | 1 + controller/ovn-controller.c | 79 ++++++ include/ovn/actions.h | 11 +- include/ovn/logical-fields.h | 3 + lib/actions.c | 128 ++++++++- lib/logical-fields.c | 17 +- lib/ovn-util.c | 25 +- lib/ovn-util.h | 4 + northd/northd.c | 107 ++++--- northd/ovn-northd.8.xml | 59 ++-- ovn-sb.xml | 54 +++- tests/ovn-controller.at | 46 +++ tests/ovn-northd.at | 526 ++++++++++++++++++----------------- tests/ovn.at | 208 +++++++------- tests/system-ovn.at | 178 ++++++------ utilities/ovn-trace.c | 72 ++++- 18 files changed, 1017 insertions(+), 536 deletions(-) Acked-by: Numan Siddique