From patchwork Tue Sep 7 22:45:06 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ben Pfaff X-Patchwork-Id: 1525566 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::136; helo=smtp3.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4H40jL59Zyz9t1Q for ; Wed, 8 Sep 2021 08:45:41 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id DD9CA607B0; Tue, 7 Sep 2021 22:45:35 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id H11NtBiUPL1M; Tue, 7 Sep 2021 22:45:35 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp3.osuosl.org (Postfix) with ESMTPS id 3749D60602; Tue, 7 Sep 2021 22:45:34 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 094B6C0011; Tue, 7 Sep 2021 22:45:34 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by lists.linuxfoundation.org (Postfix) with ESMTP id 0D346C000D for ; Tue, 7 Sep 2021 22:45:32 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id DF57540243 for ; Tue, 7 Sep 2021 22:45:31 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 9f316YcsdeCu for ; Tue, 7 Sep 2021 22:45:28 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from relay8-d.mail.gandi.net (relay8-d.mail.gandi.net [217.70.183.201]) by smtp2.osuosl.org (Postfix) with ESMTPS id 392A740104 for ; Tue, 7 Sep 2021 22:45:27 +0000 (UTC) Received: (Authenticated sender: blp@ovn.org) by relay8-d.mail.gandi.net (Postfix) with ESMTPSA id 737A61BF203; Tue, 7 Sep 2021 22:45:23 +0000 (UTC) From: Ben Pfaff To: dev@openvswitch.org Date: Tue, 7 Sep 2021 15:45:06 -0700 Message-Id: <20210907224516.489604-1-blp@ovn.org> X-Mailer: git-send-email 2.31.1 MIME-Version: 1.0 Cc: Ben Pfaff Subject: [ovs-dev] [PATCH ovn v2 00/10] 3x performance improvement for ddlog with load balancer benchmark X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" With and without these patches, I see the following performance when I run the load-balancer heavy benchmark. The measurements include cold start with all the load balancers, then deleting the sctp load balancer and waiting for it to finish, then the same with the tcp load balancer, then the same with the udp load balancer. The measurements only include ddlog time; there is additional overhead talking to the database, but that's a constant factor. Without patches: 783 CPU seconds, 23.5 GB RAM, 13:06 elapsed time. With patches: 224 CPU seconds, 14.3 GB RAM, 3:45 elapsed time. I am continuing to work on performance. These patches are also available in: https://github.com/blp/ovs-reviews/tree/ovn-memory-8 v1->v2: - Add two new patches. - Rebase and rerun benchmarks (revisions reported above). Ben Pfaff (9): ovn-northd-ddlog: Make joins for ARP/ND flows slightly more efficient. ovn-northd-ddlog: Derive load balancer IP addresses in new LoadBalancer. ovn-northd-ddlog: Reverse order of joins for connection tracking flows. ovn-northd-ddlog: Avoid re-parsing LB IP addresses and ports. ovn-northd-ddlog: Simplify LBVIPWithStatus to include up_backends string. ovn-northd-ddlog: Avoid storing unused 'lbs' field in Router. ovn-northd-ddlog: Intern strings before joining when possible. ovn-northd-ddlog: Avoid map(ival) for ARP flows. ovn-northd-ddlog: Avoid unnecessary joins for SwitchPortARPForwards. Leonid Ryzhyk (1): ovn-northd-ddlog: Intern all strings in OVSDB tables. configure.ac | 2 +- manpages.mk | 1 - northd/copp.dl | 32 +- northd/helpers.dl | 14 +- northd/ipam.dl | 17 +- northd/lrouter.dl | 213 +++-- northd/lswitch.dl | 177 ++-- northd/multicast.dl | 44 +- northd/ovn-nb.dlopts | 1 + northd/ovn-sb.dlopts | 1 + northd/ovn.dl | 7 + northd/ovn_northd.dl | 2021 ++++++++++++++++++++---------------------- northd/ovsdb2ddlog2c | 6 +- tests/ovn-ic.at | 8 +- tests/ovn-northd.at | 20 +- tests/ovn.at | 6 +- 16 files changed, 1280 insertions(+), 1290 deletions(-) Acked-by: Mark Michelson