From patchwork Fri Mar 25 15:08:45 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dumitru Ceara X-Patchwork-Id: 1609464 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=V/FJ9zNW; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::136; helo=smtp3.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4KQ58Y6CTVz9s07 for ; Sat, 26 Mar 2022 02:09:01 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id ECD6B60F1A; Fri, 25 Mar 2022 15:08:58 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id KRtZaJ9MHRex; Fri, 25 Mar 2022 15:08:58 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp3.osuosl.org (Postfix) with ESMTPS id 1DE2260BCF; Fri, 25 Mar 2022 15:08:57 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id E3B57C001D; Fri, 25 Mar 2022 15:08:56 +0000 (UTC) X-Original-To: ovs-dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 5DAFCC0012 for ; Fri, 25 Mar 2022 15:08:56 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 3F35B60BCF for ; Fri, 25 Mar 2022 15:08:56 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 3nBPjLNZIJIU for ; Fri, 25 Mar 2022 15:08:52 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by smtp3.osuosl.org (Postfix) with ESMTPS id 91D7860B00 for ; Fri, 25 Mar 2022 15:08:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1648220931; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=Z6z4BC9AQn4MoiH05QWvtTtau59avGyO/rN86gEdnVI=; b=V/FJ9zNWF2rhPAbkYsDNyZ+fykMbXpiTIsEajrksi8WVge2PP7bnWSmwiNnJJ5BPMnI4bK a3IwXTXxloFGsYikqRVrqPhLOup6ebwrsYJsuaFX/6z4CBSuSr0tPjsvU8Eb8XSGA2Xdsq HGy1XxXOjaCouQXrkcmODUdHzSyQnRw= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-652-ZZgpgSUPPbOAuHc8q12Sig-1; Fri, 25 Mar 2022 11:08:48 -0400 X-MC-Unique: ZZgpgSUPPbOAuHc8q12Sig-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 803AA1066559; Fri, 25 Mar 2022 15:08:48 +0000 (UTC) Received: from dceara.remote.csb (unknown [10.39.194.235]) by smtp.corp.redhat.com (Postfix) with ESMTP id C2CAA41136E0; Fri, 25 Mar 2022 15:08:47 +0000 (UTC) From: Dumitru Ceara To: ovs-dev@openvswitch.org Date: Fri, 25 Mar 2022 16:08:45 +0100 Message-Id: <20220325150845.4759-1-dceara@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.11.54.1 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=dceara@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [ovs-dev] [PATCH ovn] northd: Don't set SB.Load_Balancer columns if not needed. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" OVS commit 1cc618c32524 ("ovsdb-idl: Fix atomicity of writes that don't change a column's value.") [0] explains why writes that don't change a column's value cannot be optimized out early if the column is read/write. In northd, most tables have change tracking enabled, making all their columns read/write. That means that a write to a column (even if it doesn't change the value) will add the row to the current transaction. Validation is eventually performed before sending the transaction to the server but if there are lots of such records this becomes costly. Profiling what happens in northd when running with a NB database taken from an ovn-k8s-like scale test (16K load balancers applied to 120 logical switches and routers) we notice that ovn-northd was always writing to the SB.Load_Balancer columns even if nothing changed. This commit changes that behavior and only writes to the SB.Load_Balancer columns if needed. Without this change, in our test server, with ovn-northd running against NB database mentioned above, processing loop intervals were ~13 seconds. With this change applied, loop intervals go down to ~7 seconds. [0] https://github.com/openvswitch/ovs/commit/1cc618c32524 Signed-off-by: Dumitru Ceara --- northd/northd.c | 105 ++++++++++++++++++++++++++++++++++++++---------- 1 file changed, 83 insertions(+), 22 deletions(-) diff --git a/northd/northd.c b/northd/northd.c index a2cf8d6fc7..f16ca1da63 100644 --- a/northd/northd.c +++ b/northd/northd.c @@ -3951,6 +3951,56 @@ build_lb_port_related_data(struct hmap *datapaths, struct hmap *ports, build_lb_svcs(input_data, ovnsb_txn, ports, lbs); } +static bool +sb_lb_needs_update(const struct ovn_northd_lb *lb, + const struct sbrec_load_balancer *slb) +{ + if (strcmp(lb->nlb->name, slb->name)) { + return true; + } + + if (!smap_equal(&lb->nlb->vips, &slb->vips)) { + return true; + } + + if ((lb->nlb->protocol && !slb->protocol) + || (!lb->nlb->protocol && slb->protocol)) { + return true; + } + + if (strcmp(lb->nlb->protocol, slb->protocol)) { + return true; + } + + if (!smap_get_bool(&slb->options, "hairpin_orig_tuple", false)) { + return true; + } + + if (strcmp(smap_get_def(&lb->nlb->options, "hairpin_snat_ip", ""), + smap_get_def(&slb->options, "hairpin_snat_ip", ""))) { + return true; + } + + if (lb->n_nb_ls != slb->n_datapaths) { + return true; + } + + struct hmapx nb_datapaths = HMAPX_INITIALIZER(&nb_datapaths); + for (size_t i = 0; i < lb->n_nb_ls; i++) { + hmapx_add(&nb_datapaths, CONST_CAST(void *, lb->nb_ls[i]->sb)); + } + + bool stale = false; + for (size_t i = 0; i < slb->n_datapaths; i++) { + if (!hmapx_contains(&nb_datapaths, slb->datapaths[i])) { + stale = true; + break; + } + } + hmapx_destroy(&nb_datapaths); + return stale; +} + /* Syncs relevant load balancers (applied to logical switches) to the * Southbound database. */ @@ -3997,21 +4047,8 @@ sync_lbs(struct northd_input *input_data, struct ovsdb_idl_txn *ovnsb_txn, continue; } - /* Store the fact that northd provides the original (destination IP + - * transport port) tuple. - */ - struct smap options; - smap_clone(&options, &lb->nlb->options); - smap_replace(&options, "hairpin_orig_tuple", "true"); - - struct sbrec_datapath_binding **lb_dps = - xmalloc(lb->n_nb_ls * sizeof *lb_dps); - for (size_t i = 0; i < lb->n_nb_ls; i++) { - lb_dps[i] = CONST_CAST(struct sbrec_datapath_binding *, - lb->nb_ls[i]->sb); - } - - if (!lb->slb) { + sbrec_lb = lb->slb; + if (!sbrec_lb) { sbrec_lb = sbrec_load_balancer_insert(ovnsb_txn); lb->slb = sbrec_lb; char *lb_id = xasprintf( @@ -4021,13 +4058,37 @@ sync_lbs(struct northd_input *input_data, struct ovsdb_idl_txn *ovnsb_txn, sbrec_load_balancer_set_external_ids(sbrec_lb, &external_ids); free(lb_id); } - sbrec_load_balancer_set_name(lb->slb, lb->nlb->name); - sbrec_load_balancer_set_vips(lb->slb, &lb->nlb->vips); - sbrec_load_balancer_set_protocol(lb->slb, lb->nlb->protocol); - sbrec_load_balancer_set_datapaths(lb->slb, lb_dps, lb->n_nb_ls); - sbrec_load_balancer_set_options(lb->slb, &options); - smap_destroy(&options); - free(lb_dps); + + /* Only update SB.Load_Balancer columns if needed. */ + if (!lb->slb || sb_lb_needs_update(lb, sbrec_lb)) { + lb->slb = sbrec_lb; + sbrec_load_balancer_set_name(lb->slb, lb->nlb->name); + sbrec_load_balancer_set_vips(lb->slb, &lb->nlb->vips); + sbrec_load_balancer_set_protocol(lb->slb, lb->nlb->protocol); + + struct sbrec_datapath_binding **lb_dps = + xmalloc(lb->n_nb_ls * sizeof *lb_dps); + for (size_t i = 0; i < lb->n_nb_ls; i++) { + lb_dps[i] = CONST_CAST(struct sbrec_datapath_binding *, + lb->nb_ls[i]->sb); + } + sbrec_load_balancer_set_datapaths(lb->slb, lb_dps, lb->n_nb_ls); + free(lb_dps); + + /* Store the fact that northd provides the original (destination + * IP + transport port) tuple. + */ + struct smap options; + smap_clone(&options, &lb->nlb->options); + smap_add(&options, "hairpin_orig_tuple", "true"); + const char *hairpin_snat_ip = + smap_get(&lb->nlb->options, "hairpin_snat_ip"); + if (hairpin_snat_ip) { + smap_add(&options, "hairpin_snat_ip", hairpin_snat_ip); + } + sbrec_load_balancer_set_options(lb->slb, &options); + smap_destroy(&options); + } } /* Datapath_Binding.load_balancers is not used anymore, it's still in the