From patchwork Thu Feb 9 18:01:03 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ilya Maximets X-Patchwork-Id: 1740079 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.137; helo=smtp4.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp4.osuosl.org (smtp4.osuosl.org [140.211.166.137]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4PCPnG5hnQz23jH for ; Fri, 10 Feb 2023 05:01:22 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id B00394185A; Thu, 9 Feb 2023 18:01:20 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org B00394185A X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id FLTVW8jTofNg; Thu, 9 Feb 2023 18:01:19 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp4.osuosl.org (Postfix) with ESMTPS id 90C87408E3; Thu, 9 Feb 2023 18:01:18 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 90C87408E3 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 6EF67C0033; Thu, 9 Feb 2023 18:01:18 +0000 (UTC) X-Original-To: ovs-dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp4.osuosl.org (smtp4.osuosl.org [140.211.166.137]) by lists.linuxfoundation.org (Postfix) with ESMTP id 9C36AC002B for ; Thu, 9 Feb 2023 18:01:17 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 7E32E408E6 for ; Thu, 9 Feb 2023 18:01:17 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 7E32E408E6 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id miaIqrtGk7_Q for ; Thu, 9 Feb 2023 18:01:16 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org B837E408CF Received: from relay11.mail.gandi.net (relay11.mail.gandi.net [IPv6:2001:4b98:dc4:8::231]) by smtp4.osuosl.org (Postfix) with ESMTPS id B837E408CF for ; Thu, 9 Feb 2023 18:01:15 +0000 (UTC) Received: (Authenticated sender: i.maximets@ovn.org) by mail.gandi.net (Postfix) with ESMTPSA id 24E94100002; Thu, 9 Feb 2023 18:01:10 +0000 (UTC) From: Ilya Maximets To: ovs-dev@openvswitch.org Date: Thu, 9 Feb 2023 19:01:03 +0100 Message-Id: <20230209180111.2214872-1-i.maximets@ovn.org> X-Mailer: git-send-email 2.39.1 MIME-Version: 1.0 Cc: Ilya Maximets , Dumitru Ceara Subject: [ovs-dev] [PATCH ovn v3 0/8] northd: Hash locks and lflow creation with dp groups. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" While running tests with ovn-heater it was observed that locking and unlocking dpg_lock can take more than 20% of CPU cycles in northd even without any contention. This series is trying to address that issue and add thread safety static analysis. While doing that, the code is re-worked to use and better utilize datapath group bitmaps. Why the version 2 has 6 more patches... ? Attempt to rebase v1 on top of a current main branch revealed a significant conflict with the previous commit: 7b94d212f694 ("northd: Refactor build_lrouter_nat_flows_for_lb function") This refactor made it hard to utilize hash locks in an efficient way for the LR NAT flows generation. So, in v2 the first commit doesn't attempt to optimize this part anymore and just locking and unlocking the hash lock each time. Patch #3 was introduced to re-work the aforementioned function in a way that it is possible to lock only one hash by pre-calculating datapath groups for each type of flows. However, that doesn't make a significant performance difference due to the added extra work. So, the code is further re-factored to better utilize datapath group bitmaps. It's hard to do that without touching a lot of code though. So, in order to keep the codebase uniform and elegant, bitmap usage is re-worked all the way up to generation of ls/lr lists for load balancers and introduction of a new wrapper that creates a new logical flow with a desired datapath group from the beginning. Patches are structured to make sense even without following ones. So, some parts of the code are changed multiple times throughout the set. This re-work also ended up with a nice code deduplication and shortening of many functions. Final result shows up to 37% performance improvement and up to 30% memory consumption reduction in ovn-heater's density-heavy 500 node scenario. Version 3: * Fixed the bitmap leak in case only first 3 patches are applied. * Changed the if condition in patch 7/8 to (!reject || build_non_meter) [Mark] * Added Ack from Mark to patches that didn't change (patch 4/8 didn't really change, but it needed a slight rebase after changes in 3/8). Ilya Maximets (8): northd: Use larger hash bucket locks instead of dp group ones. northd: Add thread safety analysis to lflow hash locks. northd: Optimize locking pattern for GW LR NAT flows for LB. northd: Use bitmaps for LB affinity flows. northd: Use bitmaps for LB lists of switches and routers. northd: Create logical flows with datapath groups. northd: Create metered flows with dp groups if CoPP is not configured. northd: Don't collect datapath groups for LB affinity if disabled. lib/lb.c | 26 +-- lib/lb.h | 9 +- northd/northd.c | 593 ++++++++++++++++++++++-------------------------- 3 files changed, 286 insertions(+), 342 deletions(-)