From patchwork Wed Aug 23 21:57:05 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ilya Maximets X-Patchwork-Id: 1824998 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.137; helo=smtp4.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp4.osuosl.org (smtp4.osuosl.org [140.211.166.137]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4RWKmp6dNJz1yNm for ; Thu, 24 Aug 2023 07:56:41 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 6824041872; Wed, 23 Aug 2023 21:56:38 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 6824041872 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id r5xtJy18-aXe; Wed, 23 Aug 2023 21:56:37 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp4.osuosl.org (Postfix) with ESMTPS id 24006410C3; Wed, 23 Aug 2023 21:56:36 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 24006410C3 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id ECC54C0039; Wed, 23 Aug 2023 21:56:35 +0000 (UTC) X-Original-To: ovs-dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) by lists.linuxfoundation.org (Postfix) with ESMTP id E868AC0032 for ; Wed, 23 Aug 2023 21:56:33 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id AFBE560C0C for ; Wed, 23 Aug 2023 21:56:33 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org AFBE560C0C X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id oyYlpdQ7ohkN for ; Wed, 23 Aug 2023 21:56:32 +0000 (UTC) Received: from relay1-d.mail.gandi.net (relay1-d.mail.gandi.net [IPv6:2001:4b98:dc4:8::221]) by smtp3.osuosl.org (Postfix) with ESMTPS id 7E0A560C03 for ; Wed, 23 Aug 2023 21:56:32 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 7E0A560C03 Received: by mail.gandi.net (Postfix) with ESMTPSA id 4DA9A240002; Wed, 23 Aug 2023 21:56:29 +0000 (UTC) From: Ilya Maximets To: ovs-dev@openvswitch.org Date: Wed, 23 Aug 2023 23:57:05 +0200 Message-Id: <20230823215705.1786348-1-i.maximets@ovn.org> X-Mailer: git-send-email 2.40.1 MIME-Version: 1.0 X-GND-Sasl: i.maximets@ovn.org Cc: Ilya Maximets Subject: [ovs-dev] [PATCH ovn] northd: Fix incorrect memory allocation for router group datapaths. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" We need to allocate memory for 'n_router_dps' pointers, but the current code allocates space for 'n_router_dps' datapaths, which is 762 byte structure. Having 5000 routers in the OpenStack setup, we allocate 762 * 5000 * 5000 ~= 17 GB of memory instead of 190 MB: 98.17% (19,962,746,247B) (heap allocation functions) ->90.67% (18,436,818,400B) xcalloc__ (util.c:124) ->90.67% (18,436,818,400B) xcalloc (util.c:161) ->90.67% (18,436,818,400B) build_lrouter_groups (northd.c:8707) ->90.67% (18,436,818,400B) ovnnb_db_run (northd.c:17611) ->90.67% (18,436,818,400B) en_northd_run (en-northd.c:137) ->90.67% (18,436,818,400B) engine_recompute (inc-proc-eng.c:415) ->90.67% (18,436,818,400B) engine_run_node (inc-proc-eng.c:477) ->90.67% (18,436,818,400B) engine_run (inc-proc-eng.c:528) ->90.67% (18,436,818,400B) inc_proc_northd_run (inc-proc-northd.c:316) ->90.67% (18,436,818,400B) main (ovn-northd.c:937) Also, even though this doesn't affect the allocation size, the order of arguments is incorrect. Fix that by using correct pointers and order. That saves a huge amount of memory as well as time on allocating and zeroing it out. In my testing northd node saved 10 seconds on these allocations. Fixes: b82d351976a1 ("ovn: Add generic HA chassis group") Signed-off-by: Ilya Maximets Acked-by: Mark Michelson --- northd/northd.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/northd/northd.c b/northd/northd.c index 0a749931e..c701d929e 100644 --- a/northd/northd.c +++ b/northd/northd.c @@ -8704,7 +8704,8 @@ build_lrouter_groups(struct hmap *lr_ports, struct ovs_list *lr_list) od->lr_group = xzalloc(sizeof *od->lr_group); /* Each logical router group can have max * 'n_router_dps'. So allocate enough memory. */ - od->lr_group->router_dps = xcalloc(sizeof *od, n_router_dps); + od->lr_group->router_dps = + xcalloc(n_router_dps, sizeof *od->lr_group->router_dps); od->lr_group->router_dps[0] = od; od->lr_group->n_router_dps = 1; sset_init(&od->lr_group->ha_chassis_groups);