From patchwork Tue Nov 22 03:49:35 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Numan Siddique X-Patchwork-Id: 1707638 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::133; helo=smtp2.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp2.osuosl.org (smtp2.osuosl.org [IPv6:2605:bc80:3010::133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4NGVdG4xWSz23nR for ; Tue, 22 Nov 2022 14:49:54 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id B3867405F8; Tue, 22 Nov 2022 03:49:52 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org B3867405F8 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 3mvCdKsEuVVS; Tue, 22 Nov 2022 03:49:51 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp2.osuosl.org (Postfix) with ESMTPS id D80B3404F4; Tue, 22 Nov 2022 03:49:50 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org D80B3404F4 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id B13E7C0032; Tue, 22 Nov 2022 03:49:50 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by lists.linuxfoundation.org (Postfix) with ESMTP id 178E3C002D for ; Tue, 22 Nov 2022 03:49:50 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 68F0B81E11 for ; Tue, 22 Nov 2022 03:49:49 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 68F0B81E11 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id SexcCO2STCKd for ; Tue, 22 Nov 2022 03:49:48 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 61F518187F Received: from relay8-d.mail.gandi.net (relay8-d.mail.gandi.net [IPv6:2001:4b98:dc4:8::228]) by smtp1.osuosl.org (Postfix) with ESMTPS id 61F518187F for ; Tue, 22 Nov 2022 03:49:48 +0000 (UTC) Received: (Authenticated sender: numans@ovn.org) by mail.gandi.net (Postfix) with ESMTPSA id B79341BF204; Tue, 22 Nov 2022 03:49:45 +0000 (UTC) From: numans@ovn.org To: dev@openvswitch.org Date: Mon, 21 Nov 2022 22:49:35 -0500 Message-Id: <20221122034935.1973847-1-numans@ovn.org> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221122034908.1973726-1-numans@ovn.org> References: <20221122034908.1973726-1-numans@ovn.org> MIME-Version: 1.0 Subject: [ovs-dev] [PATCH ovn v4 1/3] Enhance the unixctl command inc-engine/show-stats X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Numan Siddique Enhance the command to specify the engine node name and engine stat type - recompute, compute or abort. Also updates the documentation. Signed-off-by: Numan Siddique Acked-by: Han Zhou --- controller/ovn-controller.8.xml | 10 +++++++ lib/inc-proc-eng.c | 49 +++++++++++++++++++++++++-------- northd/ovn-northd.8.xml | 34 +++++++++++++++++++++++ 3 files changed, 82 insertions(+), 11 deletions(-) diff --git a/controller/ovn-controller.8.xml b/controller/ovn-controller.8.xml index eb2950723..af70f9e36 100644 --- a/controller/ovn-controller.8.xml +++ b/controller/ovn-controller.8.xml @@ -716,6 +716,16 @@ +
inc-engine/show-stats engine_node_name counter_name
+
+

+ Display the ovn-controller engine counter(s) for the + specified engine_node_name. counter_name is + optional and can be one of recompute, + compute or abort. +

+
+
inc-engine/clear-stats
Reset ovn-controller engine counters. diff --git a/lib/inc-proc-eng.c b/lib/inc-proc-eng.c index 2e2b31033..21afcf92b 100644 --- a/lib/inc-proc-eng.c +++ b/lib/inc-proc-eng.c @@ -124,23 +124,50 @@ engine_clear_stats(struct unixctl_conn *conn, int argc OVS_UNUSED, } static void -engine_dump_stats(struct unixctl_conn *conn, int argc OVS_UNUSED, - const char *argv[] OVS_UNUSED, void *arg OVS_UNUSED) +engine_dump_stats(struct unixctl_conn *conn, int argc, + const char *argv[], void *arg OVS_UNUSED) { struct ds dump = DS_EMPTY_INITIALIZER; + const char *dump_eng_node_name = (argc > 1 ? argv[1] : NULL); + const char *dump_stat_type = (argc > 2 ? argv[2] : NULL); + bool success = true; for (size_t i = 0; i < engine_n_nodes; i++) { struct engine_node *node = engine_nodes[i]; - ds_put_format(&dump, - "Node: %s\n" - "- recompute: %12"PRIu64"\n" - "- compute: %12"PRIu64"\n" - "- abort: %12"PRIu64"\n", - node->name, node->stats.recompute, - node->stats.compute, node->stats.abort); + if (dump_eng_node_name && strcmp(node->name, dump_eng_node_name)) { + continue; + } + + if (!dump_stat_type) { + ds_put_format(&dump, + "Node: %s\n" + "- recompute: %12"PRIu64"\n" + "- compute: %12"PRIu64"\n" + "- abort: %12"PRIu64"\n", + node->name, node->stats.recompute, + node->stats.compute, node->stats.abort); + } else { + if (!strcmp(dump_stat_type, "recompute")) { + ds_put_format(&dump, "%"PRIu64, node->stats.recompute); + } else if (!strcmp(dump_stat_type, "compute")) { + ds_put_format(&dump, "%"PRIu64, node->stats.compute); + } else if (!strcmp(dump_stat_type, "abort")) { + ds_put_format(&dump, "%"PRIu64, node->stats.abort); + } else { + ds_put_format(&dump, "Invalid stat type : %s", dump_stat_type); + } + } + + if (dump_eng_node_name) { + break; + } + } + if (success) { + unixctl_command_reply(conn, ds_cstr(&dump)); + } else { + unixctl_command_reply_error(conn, "Invalid stat type"); } - unixctl_command_reply(conn, ds_cstr(&dump)); ds_destroy(&dump); } @@ -182,7 +209,7 @@ engine_init(struct engine_node *node, struct engine_arg *arg) } } - unixctl_command_register("inc-engine/show-stats", "", 0, 0, + unixctl_command_register("inc-engine/show-stats", "", 0, 2, engine_dump_stats, NULL); unixctl_command_register("inc-engine/clear-stats", "", 0, 0, engine_clear_stats, NULL); diff --git a/northd/ovn-northd.8.xml b/northd/ovn-northd.8.xml index 4b712cec4..e25993bdb 100644 --- a/northd/ovn-northd.8.xml +++ b/northd/ovn-northd.8.xml @@ -204,6 +204,40 @@

+
inc-engine/show-stats
+
+

+ Display ovn-northd engine counters. For each engine + node the following counters have been added: +

    +
  • + recompute +
    • +
  • + compute +
    • +
  • + abort +
    • +
+

+
+ +
inc-engine/show-stats engine_node_name counter_name
+
+

+ Display the ovn-northd engine counter(s) for the specified + engine_node_name. counter_name is optional and + can be one of recompute, compute or + abort. +

+
+ +
inc-engine/clear-stats
+
+

Reset ovn-northd engine counters.

+
+

From patchwork Tue Nov 22 03:49:54 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Numan Siddique X-Patchwork-Id: 1707639 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.137; helo=smtp4.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp4.osuosl.org (smtp4.osuosl.org [140.211.166.137]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4NGVdn3thHz23nj for ; Tue, 22 Nov 2022 14:50:21 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 8471241522; Tue, 22 Nov 2022 03:50:19 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 8471241522 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id DIhMkOXT6hlR; Tue, 22 Nov 2022 03:50:17 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp4.osuosl.org (Postfix) with ESMTPS id E6A3F41505; Tue, 22 Nov 2022 03:50:15 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org E6A3F41505 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id BE49AC002D; Tue, 22 Nov 2022 03:50:15 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by lists.linuxfoundation.org (Postfix) with ESMTP id CA032C002D for ; Tue, 22 Nov 2022 03:50:13 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 9F8E281E7E for ; Tue, 22 Nov 2022 03:50:13 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 9F8E281E7E X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 9LeFCo-t31ht for ; Tue, 22 Nov 2022 03:50:11 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 88CC781E9F Received: from relay5-d.mail.gandi.net (relay5-d.mail.gandi.net [IPv6:2001:4b98:dc4:8::225]) by smtp1.osuosl.org (Postfix) with ESMTPS id 88CC781E9F for ; Tue, 22 Nov 2022 03:50:10 +0000 (UTC) Received: (Authenticated sender: numans@ovn.org) by mail.gandi.net (Postfix) with ESMTPSA id D8E9E1C0002; Tue, 22 Nov 2022 03:50:05 +0000 (UTC) From: numans@ovn.org To: dev@openvswitch.org Date: Mon, 21 Nov 2022 22:49:54 -0500 Message-Id: <20221122034954.1973887-1-numans@ovn.org> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221122034908.1973726-1-numans@ovn.org> References: <20221122034908.1973726-1-numans@ovn.org> MIME-Version: 1.0 Subject: [ovs-dev] [PATCH ovn v4 2/3] northd IP: Add a new engine node 'en_sync_to_sb' to sync SB tables. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Numan Siddique A sub-engine node 'en_sync_to_sb_addr_set' is added with-in the 'en_sync_to_sb' node to sync the Address_Set table in the SB database. To start with, it falls back to full recompute all the time. Upcoming patch will add the incremental processing support to sync the SB Address_Set table. 'en_sync_to_sb' engine node can be enhanced further to sync other SB tables like - Port_Group, DHCP_Options, DNS etc. Acked-by: Mark Michelson Acked-by: Ales Musil Signed-off-by: Numan Siddique Acked-by: Han Zhou --- lib/ovn-util.c | 30 +++++ lib/ovn-util.h | 3 + northd/automake.mk | 4 + northd/en-northd-output.c | 58 ++++++++++ northd/en-northd-output.h | 17 +++ northd/en-northd.c | 4 - northd/en-sync-sb.c | 233 ++++++++++++++++++++++++++++++++++++++ northd/en-sync-sb.h | 14 +++ northd/inc-proc-northd.c | 24 +++- northd/northd.c | 173 ++-------------------------- northd/northd.h | 3 +- 11 files changed, 390 insertions(+), 173 deletions(-) create mode 100644 northd/en-northd-output.c create mode 100644 northd/en-northd-output.h create mode 100644 northd/en-sync-sb.c create mode 100644 northd/en-sync-sb.h diff --git a/lib/ovn-util.c b/lib/ovn-util.c index 597625a29..868472ace 100644 --- a/lib/ovn-util.c +++ b/lib/ovn-util.c @@ -938,3 +938,33 @@ daemon_started_recently(void) /* Ensure that at least an amount of time has passed. */ return time_wall_msec() - startup_ts <= DAEMON_STARTUP_DELAY_MS; } + +/* Builds a unique address set compatible name ([a-zA-Z_.][a-zA-Z_.0-9]*) + * for the router's load balancer VIP address set, combining the logical + * router's datapath tunnel key and address family. + * + * Also prefixes the name with 'prefix'. + */ +static char * +lr_lb_address_set_name_(uint32_t lr_tunnel_key, const char *prefix, + int addr_family) +{ + return xasprintf("%s_rtr_lb_%"PRIu32"_ip%s", prefix, lr_tunnel_key, + addr_family == AF_INET ? "4" : "6"); +} + +/* Builds the router's load balancer VIP address set name. */ +char * +lr_lb_address_set_name(uint32_t lr_tunnel_key, int addr_family) +{ + return lr_lb_address_set_name_(lr_tunnel_key, "", addr_family); +} + +/* Builds a string that refers to the the router's load balancer VIP address + * set name, that is: $. + */ +char * +lr_lb_address_set_ref(uint32_t lr_tunnel_key, int addr_family) +{ + return lr_lb_address_set_name_(lr_tunnel_key, "$", addr_family); +} diff --git a/lib/ovn-util.h b/lib/ovn-util.h index a1f1cf0ad..809ff1d36 100644 --- a/lib/ovn-util.h +++ b/lib/ovn-util.h @@ -315,4 +315,7 @@ void daemon_started_recently_ignore(void); bool daemon_started_recently(void); int64_t daemon_startup_ts(void); +char *lr_lb_address_set_name(uint32_t lr_tunnel_key, int addr_family); +char *lr_lb_address_set_ref(uint32_t lr_tunnel_key, int addr_family); + #endif /* OVN_UTIL_H */ diff --git a/northd/automake.mk b/northd/automake.mk index 81582867d..b7cfccf36 100644 --- a/northd/automake.mk +++ b/northd/automake.mk @@ -10,6 +10,10 @@ northd_ovn_northd_SOURCES = \ northd/en-northd.h \ northd/en-lflow.c \ northd/en-lflow.h \ + northd/en-northd-output.c \ + northd/en-northd-output.h \ + northd/en-sync-sb.c \ + northd/en-sync-sb.h \ northd/inc-proc-northd.c \ northd/inc-proc-northd.h \ northd/ipam.c \ diff --git a/northd/en-northd-output.c b/northd/en-northd-output.c new file mode 100644 index 000000000..cdb2f9558 --- /dev/null +++ b/northd/en-northd-output.c @@ -0,0 +1,58 @@ +/* + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at: + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#include + +#include +#include +#include + +#include "openvswitch/util.h" + +#include "en-northd-output.h" +#include "lib/inc-proc-eng.h" + +void * +en_northd_output_init(struct engine_node *node OVS_UNUSED, + struct engine_arg *arg OVS_UNUSED) +{ + return NULL; +} + +void +en_northd_output_run(struct engine_node *node, void *data OVS_UNUSED) +{ + engine_set_node_state(node, EN_UPDATED); +} + +void +en_northd_output_cleanup(void *data OVS_UNUSED) +{ + +} + +bool +northd_output_sync_to_sb_handler(struct engine_node *node, + void *data OVS_UNUSED) +{ + engine_set_node_state(node, EN_UPDATED); + return true; +} + +bool +northd_output_lflow_handler(struct engine_node *node, void *data OVS_UNUSED) +{ + engine_set_node_state(node, EN_UPDATED); + return true; +} diff --git a/northd/en-northd-output.h b/northd/en-northd-output.h new file mode 100644 index 000000000..d0079ea67 --- /dev/null +++ b/northd/en-northd-output.h @@ -0,0 +1,17 @@ +#ifndef EN_NORTHD_OUTPUT_H +#define EN_NORTHD_OUTPUT_H 1 + +#include "lib/inc-proc-eng.h" + +void *en_northd_output_init(struct engine_node *node OVS_UNUSED, + struct engine_arg *arg OVS_UNUSED); +void en_northd_output_run(struct engine_node *node OVS_UNUSED, + void *data OVS_UNUSED); + +void en_northd_output_cleanup(void *data); +bool northd_output_sync_to_sb_handler(struct engine_node *node, + void *data OVS_UNUSED); +bool northd_output_lflow_handler(struct engine_node *node, + void *data OVS_UNUSED); + +#endif diff --git a/northd/en-northd.c b/northd/en-northd.c index 7fe83db64..93891b0b7 100644 --- a/northd/en-northd.c +++ b/northd/en-northd.c @@ -72,8 +72,6 @@ void en_northd_run(struct engine_node *node, void *data) EN_OVSDB_GET(engine_get_input("NB_load_balancer_group", node)); input_data.nbrec_port_group_table = EN_OVSDB_GET(engine_get_input("NB_port_group", node)); - input_data.nbrec_address_set_table = - EN_OVSDB_GET(engine_get_input("NB_address_set", node)); input_data.nbrec_meter_table = EN_OVSDB_GET(engine_get_input("NB_meter", node)); input_data.nbrec_acl_table = @@ -99,8 +97,6 @@ void en_northd_run(struct engine_node *node, void *data) EN_OVSDB_GET(engine_get_input("SB_load_balancer", node)); input_data.sbrec_service_monitor_table = EN_OVSDB_GET(engine_get_input("SB_service_monitor", node)); - input_data.sbrec_address_set_table = - EN_OVSDB_GET(engine_get_input("SB_address_set", node)); input_data.sbrec_port_group_table = EN_OVSDB_GET(engine_get_input("SB_port_group", node)); input_data.sbrec_meter_table = diff --git a/northd/en-sync-sb.c b/northd/en-sync-sb.c new file mode 100644 index 000000000..6e8182f42 --- /dev/null +++ b/northd/en-sync-sb.c @@ -0,0 +1,233 @@ +/* + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at: + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#include + +#include +#include +#include + +#include "lib/svec.h" +#include "openvswitch/util.h" + +#include "en-sync-sb.h" +#include "lib/inc-proc-eng.h" +#include "lib/lb.h" +#include "lib/ovn-nb-idl.h" +#include "lib/ovn-sb-idl.h" +#include "lib/ovn-util.h" +#include "northd.h" + +#include "openvswitch/vlog.h" + +VLOG_DEFINE_THIS_MODULE(en_sync_to_sb); + +static void sync_addr_set(struct ovsdb_idl_txn *ovnsb_txn, const char *name, + const char **addrs, size_t n_addrs, + struct shash *sb_address_sets); +static void sync_addr_sets(const struct nbrec_address_set_table *, + const struct nbrec_port_group_table *, + const struct sbrec_address_set_table *, + struct ovsdb_idl_txn *ovnsb_txn, + struct hmap *datapaths); + +void * +en_sync_to_sb_init(struct engine_node *node OVS_UNUSED, + struct engine_arg *arg OVS_UNUSED) +{ + return NULL; +} + +void +en_sync_to_sb_run(struct engine_node *node, void *data OVS_UNUSED) +{ + engine_set_node_state(node, EN_UPDATED); +} + +void +en_sync_to_sb_cleanup(void *data OVS_UNUSED) +{ + +} + +void * +en_sync_to_sb_addr_set_init(struct engine_node *node OVS_UNUSED, + struct engine_arg *arg OVS_UNUSED) +{ + return NULL; +} + +void +en_sync_to_sb_addr_set_run(struct engine_node *node, void *data OVS_UNUSED) +{ + const struct nbrec_address_set_table *nb_address_set_table = + EN_OVSDB_GET(engine_get_input("NB_address_set", node)); + const struct nbrec_port_group_table *nb_port_group_table = + EN_OVSDB_GET(engine_get_input("NB_port_group", node)); + const struct sbrec_address_set_table *sb_address_set_table = + EN_OVSDB_GET(engine_get_input("SB_address_set", node)); + + const struct engine_context *eng_ctx = engine_get_context(); + struct northd_data *northd_data = engine_get_input_data("northd", node); + + sync_addr_sets(nb_address_set_table, nb_port_group_table, + sb_address_set_table, eng_ctx->ovnsb_idl_txn, + &northd_data->datapaths); + + engine_set_node_state(node, EN_UPDATED); +} + +void +en_sync_to_sb_addr_set_cleanup(void *data OVS_UNUSED) +{ + +} + +/* static functions. */ +static void +sync_addr_set(struct ovsdb_idl_txn *ovnsb_txn, const char *name, + const char **addrs, size_t n_addrs, + struct shash *sb_address_sets) +{ + const struct sbrec_address_set *sb_address_set; + sb_address_set = shash_find_and_delete(sb_address_sets, + name); + if (!sb_address_set) { + sb_address_set = sbrec_address_set_insert(ovnsb_txn); + sbrec_address_set_set_name(sb_address_set, name); + } + + sbrec_address_set_set_addresses(sb_address_set, + addrs, n_addrs); +} + +/* OVN_Southbound Address_Set table contains same records as in north + * bound, plus + * - the records generated from Port_Group table in north bound. + * + * There are 2 records generated from each port group, one for IPv4, and + * one for IPv6, named in the format: _ip4 and + * _ip6 respectively. MAC addresses are ignored. + * + * - the records generated for the load balancer VIP addresses which are + * routable from each logical router. + * + * We always update OVN_Southbound to match the Address_Set and Port_Group + * in OVN_Northbound, so that the address sets used in Logical_Flows in + * OVN_Southbound is checked against the proper set.*/ +static void +sync_addr_sets(const struct nbrec_address_set_table *nb_address_set_table, + const struct nbrec_port_group_table *nb_port_group_table, + const struct sbrec_address_set_table *sb_address_set_table, + struct ovsdb_idl_txn *ovnsb_txn, struct hmap *datapaths) +{ + struct shash sb_address_sets = SHASH_INITIALIZER(&sb_address_sets); + + const struct sbrec_address_set *sb_address_set; + SBREC_ADDRESS_SET_TABLE_FOR_EACH (sb_address_set, + sb_address_set_table) { + shash_add(&sb_address_sets, sb_address_set->name, sb_address_set); + } + + /* Service monitor MAC. */ + const char *svc_monitor_macp = northd_get_svc_monitor_mac(); + sync_addr_set(ovnsb_txn, "svc_monitor_mac", &svc_monitor_macp, 1, + &sb_address_sets); + + /* sync port group generated address sets first */ + const struct nbrec_port_group *nb_port_group; + NBREC_PORT_GROUP_TABLE_FOR_EACH (nb_port_group, + nb_port_group_table) { + struct svec ipv4_addrs = SVEC_EMPTY_INITIALIZER; + struct svec ipv6_addrs = SVEC_EMPTY_INITIALIZER; + for (size_t i = 0; i < nb_port_group->n_ports; i++) { + for (size_t j = 0; j < nb_port_group->ports[i]->n_addresses; j++) { + const char *addrs = nb_port_group->ports[i]->addresses[j]; + if (!is_dynamic_lsp_address(addrs)) { + split_addresses(addrs, &ipv4_addrs, &ipv6_addrs); + } + } + if (nb_port_group->ports[i]->dynamic_addresses) { + split_addresses(nb_port_group->ports[i]->dynamic_addresses, + &ipv4_addrs, &ipv6_addrs); + } + } + char *ipv4_addrs_name = xasprintf("%s_ip4", nb_port_group->name); + char *ipv6_addrs_name = xasprintf("%s_ip6", nb_port_group->name); + sync_addr_set(ovnsb_txn, ipv4_addrs_name, + /* "char **" is not compatible with "const char **" */ + (const char **) ipv4_addrs.names, + ipv4_addrs.n, &sb_address_sets); + sync_addr_set(ovnsb_txn, ipv6_addrs_name, + /* "char **" is not compatible with "const char **" */ + (const char **) ipv6_addrs.names, + ipv6_addrs.n, &sb_address_sets); + free(ipv4_addrs_name); + free(ipv6_addrs_name); + svec_destroy(&ipv4_addrs); + svec_destroy(&ipv6_addrs); + } + + /* Sync router load balancer VIP generated address sets. */ + struct ovn_datapath *od; + HMAP_FOR_EACH (od, key_node, datapaths) { + if (!od->nbr) { + continue; + } + + if (sset_count(&od->lb_ips->ips_v4_reachable)) { + char *ipv4_addrs_name = lr_lb_address_set_name(od->tunnel_key, + AF_INET); + const char **ipv4_addrs = + sset_array(&od->lb_ips->ips_v4_reachable); + + sync_addr_set(ovnsb_txn, ipv4_addrs_name, ipv4_addrs, + sset_count(&od->lb_ips->ips_v4_reachable), + &sb_address_sets); + free(ipv4_addrs_name); + free(ipv4_addrs); + } + + if (sset_count(&od->lb_ips->ips_v6_reachable)) { + char *ipv6_addrs_name = lr_lb_address_set_name(od->tunnel_key, + AF_INET6); + const char **ipv6_addrs = + sset_array(&od->lb_ips->ips_v6_reachable); + + sync_addr_set(ovnsb_txn, ipv6_addrs_name, ipv6_addrs, + sset_count(&od->lb_ips->ips_v6_reachable), + &sb_address_sets); + free(ipv6_addrs_name); + free(ipv6_addrs); + } + } + + /* sync user defined address sets, which may overwrite port group + * generated address sets if same name is used */ + const struct nbrec_address_set *nb_address_set; + NBREC_ADDRESS_SET_TABLE_FOR_EACH (nb_address_set, + nb_address_set_table) { + sync_addr_set(ovnsb_txn, nb_address_set->name, + /* "char **" is not compatible with "const char **" */ + (const char **) nb_address_set->addresses, + nb_address_set->n_addresses, &sb_address_sets); + } + + struct shash_node *node; + SHASH_FOR_EACH_SAFE (node, &sb_address_sets) { + sbrec_address_set_delete(node->data); + shash_delete(&sb_address_sets, node); + } + shash_destroy(&sb_address_sets); +} diff --git a/northd/en-sync-sb.h b/northd/en-sync-sb.h new file mode 100644 index 000000000..1b25ff90f --- /dev/null +++ b/northd/en-sync-sb.h @@ -0,0 +1,14 @@ +#ifndef EN_SYNC_SB_H +#define EN_SYNC_SB_H 1 + +#include "lib/inc-proc-eng.h" + +void *en_sync_to_sb_init(struct engine_node *, struct engine_arg *); +void en_sync_to_sb_run(struct engine_node *, void *data); +void en_sync_to_sb_cleanup(void *data); + +void *en_sync_to_sb_addr_set_init(struct engine_node *, struct engine_arg *); +void en_sync_to_sb_addr_set_run(struct engine_node *, void *data); +void en_sync_to_sb_addr_set_cleanup(void *data); + +#endif /* end of EN_SYNC_SB_H */ diff --git a/northd/inc-proc-northd.c b/northd/inc-proc-northd.c index 54e0ad3b0..dbba41127 100644 --- a/northd/inc-proc-northd.c +++ b/northd/inc-proc-northd.c @@ -32,6 +32,8 @@ #include "inc-proc-northd.h" #include "en-northd.h" #include "en-lflow.h" +#include "en-northd-output.h" +#include "en-sync-sb.h" #include "util.h" VLOG_DEFINE_THIS_MODULE(inc_proc_northd); @@ -153,6 +155,9 @@ static ENGINE_NODE(northd, "northd"); static ENGINE_NODE(lflow, "lflow"); static ENGINE_NODE(mac_binding_aging, "mac_binding_aging"); static ENGINE_NODE(mac_binding_aging_waker, "mac_binding_aging_waker"); +static ENGINE_NODE(northd_output, "northd_output"); +static ENGINE_NODE(sync_to_sb, "sync_to_sb"); +static ENGINE_NODE(sync_to_sb_addr_set, "sync_to_sb_addr_set"); void inc_proc_northd_init(struct ovsdb_idl_loop *nb, struct ovsdb_idl_loop *sb) @@ -164,7 +169,6 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, engine_add_input(&en_northd, &en_nb_logical_switch, NULL); engine_add_input(&en_northd, &en_nb_logical_switch_port, NULL); engine_add_input(&en_northd, &en_nb_forwarding_group, NULL); - engine_add_input(&en_northd, &en_nb_address_set, NULL); engine_add_input(&en_northd, &en_nb_port_group, NULL); engine_add_input(&en_northd, &en_nb_load_balancer, NULL); engine_add_input(&en_northd, &en_nb_load_balancer_group, NULL); @@ -191,7 +195,6 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, engine_add_input(&en_northd, &en_sb_chassis, NULL); engine_add_input(&en_northd, &en_sb_chassis_private, NULL); engine_add_input(&en_northd, &en_sb_encap, NULL); - engine_add_input(&en_northd, &en_sb_address_set, NULL); engine_add_input(&en_northd, &en_sb_port_group, NULL); engine_add_input(&en_northd, &en_sb_logical_dp_group, NULL); engine_add_input(&en_northd, &en_sb_meter, NULL); @@ -229,6 +232,21 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, * once I-P engine allows multiple root nodes. */ engine_add_input(&en_lflow, &en_mac_binding_aging, NULL); + engine_add_input(&en_sync_to_sb_addr_set, &en_nb_address_set, NULL); + engine_add_input(&en_sync_to_sb_addr_set, &en_nb_port_group, NULL); + engine_add_input(&en_sync_to_sb_addr_set, &en_northd, NULL); + engine_add_input(&en_sync_to_sb_addr_set, &en_sb_address_set, NULL); + + /* en_sync_to_sb engine node syncs the SB database tables from + * the NB database tables. + * Right now this engine only syncs the SB Address_Set table. + */ + engine_add_input(&en_sync_to_sb, &en_sync_to_sb_addr_set, NULL); + engine_add_input(&en_northd_output, &en_sync_to_sb, + northd_output_sync_to_sb_handler); + engine_add_input(&en_northd_output, &en_lflow, + northd_output_lflow_handler); + struct engine_arg engine_arg = { .nb_idl = nb->idl, .sb_idl = sb->idl, @@ -249,7 +267,7 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, struct ovsdb_idl_index *sbrec_mac_binding_by_datapath = mac_binding_by_datapath_index_create(sb->idl); - engine_init(&en_lflow, &engine_arg); + engine_init(&en_northd_output, &engine_arg); engine_ovsdb_node_add_index(&en_sb_chassis, "sbrec_chassis_by_name", diff --git a/northd/northd.c b/northd/northd.c index 00ff8f933..e5db55451 100644 --- a/northd/northd.c +++ b/northd/northd.c @@ -808,37 +808,6 @@ lr_has_lb_vip(struct ovn_datapath *od) return false; } -/* Builds a unique address set compatible name ([a-zA-Z_.][a-zA-Z_.0-9]*) - * for the router's load balancer VIP address set, combining the logical - * router's datapath tunnel key and address family. - * - * Also prefixes the name with 'prefix'. - */ -static char * -lr_lb_address_set_name_(const struct ovn_datapath *od, const char *prefix, - int addr_family) -{ - ovs_assert(od->nbr); - return xasprintf("%s_rtr_lb_%"PRIu32"_ip%s", prefix, od->tunnel_key, - addr_family == AF_INET ? "4" : "6"); -} - -/* Builds the router's load balancer VIP address set name. */ -static char * -lr_lb_address_set_name(const struct ovn_datapath *od, int addr_family) -{ - return lr_lb_address_set_name_(od, "", addr_family); -} - -/* Builds a string that refers to the the router's load balancer VIP address - * set name, that is: $. - */ -static char * -lr_lb_address_set_ref(const struct ovn_datapath *od, int addr_family) -{ - return lr_lb_address_set_name_(od, "$", addr_family); -} - static void init_lb_for_datapath(struct ovn_datapath *od) { @@ -13141,7 +13110,8 @@ build_lrouter_ipv4_ip_input(struct ovn_port *op, } /* Create a single ARP rule for all IPs that are used as VIPs. */ - char *lb_ips_v4_as = lr_lb_address_set_ref(op->od, AF_INET); + char *lb_ips_v4_as = lr_lb_address_set_ref(op->od->tunnel_key, + AF_INET); build_lrouter_arp_flow(op->od, op, lb_ips_v4_as, REG_INPORT_ETH_ADDR, match, false, 90, NULL, lflows); @@ -13157,7 +13127,8 @@ build_lrouter_ipv4_ip_input(struct ovn_port *op, } /* Create a single ND rule for all IPs that are used as VIPs. */ - char *lb_ips_v6_as = lr_lb_address_set_ref(op->od, AF_INET6); + char *lb_ips_v6_as = lr_lb_address_set_ref(op->od->tunnel_key, + AF_INET6); build_lrouter_nd_flow(op->od, op, "nd_na", lb_ips_v6_as, NULL, REG_INPORT_ETH_ADDR, match, false, 90, NULL, lflows, meter_groups); @@ -14918,136 +14889,6 @@ void build_lflows(struct lflow_input *input_data, hmap_destroy(&mcast_groups); } -static void -sync_address_set(struct ovsdb_idl_txn *ovnsb_txn, const char *name, - const char **addrs, size_t n_addrs, - struct shash *sb_address_sets) -{ - const struct sbrec_address_set *sb_address_set; - sb_address_set = shash_find_and_delete(sb_address_sets, - name); - if (!sb_address_set) { - sb_address_set = sbrec_address_set_insert(ovnsb_txn); - sbrec_address_set_set_name(sb_address_set, name); - } - - sbrec_address_set_set_addresses(sb_address_set, - addrs, n_addrs); -} - -/* OVN_Southbound Address_Set table contains same records as in north - * bound, plus the records generated from Port_Group table in north bound. - * - * There are 2 records generated from each port group, one for IPv4, and - * one for IPv6, named in the format: _ip4 and - * _ip6 respectively. MAC addresses are ignored. - * - * We always update OVN_Southbound to match the Address_Set and Port_Group - * in OVN_Northbound, so that the address sets used in Logical_Flows in - * OVN_Southbound is checked against the proper set.*/ -static void -sync_address_sets(struct northd_input *input_data, - struct ovsdb_idl_txn *ovnsb_txn, - struct hmap *datapaths) -{ - struct shash sb_address_sets = SHASH_INITIALIZER(&sb_address_sets); - - const struct sbrec_address_set *sb_address_set; - SBREC_ADDRESS_SET_TABLE_FOR_EACH (sb_address_set, - input_data->sbrec_address_set_table) { - shash_add(&sb_address_sets, sb_address_set->name, sb_address_set); - } - - /* Service monitor MAC. */ - const char *svc_monitor_macp = svc_monitor_mac; - sync_address_set(ovnsb_txn, "svc_monitor_mac", &svc_monitor_macp, 1, - &sb_address_sets); - - /* sync port group generated address sets first */ - const struct nbrec_port_group *nb_port_group; - NBREC_PORT_GROUP_TABLE_FOR_EACH (nb_port_group, - input_data->nbrec_port_group_table) { - struct svec ipv4_addrs = SVEC_EMPTY_INITIALIZER; - struct svec ipv6_addrs = SVEC_EMPTY_INITIALIZER; - for (size_t i = 0; i < nb_port_group->n_ports; i++) { - for (size_t j = 0; j < nb_port_group->ports[i]->n_addresses; j++) { - const char *addrs = nb_port_group->ports[i]->addresses[j]; - if (!is_dynamic_lsp_address(addrs)) { - split_addresses(addrs, &ipv4_addrs, &ipv6_addrs); - } - } - if (nb_port_group->ports[i]->dynamic_addresses) { - split_addresses(nb_port_group->ports[i]->dynamic_addresses, - &ipv4_addrs, &ipv6_addrs); - } - } - char *ipv4_addrs_name = xasprintf("%s_ip4", nb_port_group->name); - char *ipv6_addrs_name = xasprintf("%s_ip6", nb_port_group->name); - sync_address_set(ovnsb_txn, ipv4_addrs_name, - /* "char **" is not compatible with "const char **" */ - (const char **)ipv4_addrs.names, - ipv4_addrs.n, &sb_address_sets); - sync_address_set(ovnsb_txn, ipv6_addrs_name, - /* "char **" is not compatible with "const char **" */ - (const char **)ipv6_addrs.names, - ipv6_addrs.n, &sb_address_sets); - free(ipv4_addrs_name); - free(ipv6_addrs_name); - svec_destroy(&ipv4_addrs); - svec_destroy(&ipv6_addrs); - } - - /* Sync router load balancer VIP generated address sets. */ - struct ovn_datapath *od; - HMAP_FOR_EACH (od, key_node, datapaths) { - if (!od->nbr) { - continue; - } - - if (sset_count(&od->lb_ips->ips_v4_reachable)) { - char *ipv4_addrs_name = lr_lb_address_set_name(od, AF_INET); - const char **ipv4_addrs = - sset_array(&od->lb_ips->ips_v4_reachable); - - sync_address_set(ovnsb_txn, ipv4_addrs_name, ipv4_addrs, - sset_count(&od->lb_ips->ips_v4_reachable), - &sb_address_sets); - free(ipv4_addrs_name); - free(ipv4_addrs); - } - - if (sset_count(&od->lb_ips->ips_v6_reachable)) { - char *ipv6_addrs_name = lr_lb_address_set_name(od, AF_INET6); - const char **ipv6_addrs = - sset_array(&od->lb_ips->ips_v6_reachable); - - sync_address_set(ovnsb_txn, ipv6_addrs_name, ipv6_addrs, - sset_count(&od->lb_ips->ips_v6_reachable), - &sb_address_sets); - free(ipv6_addrs_name); - free(ipv6_addrs); - } - } - - /* sync user defined address sets, which may overwrite port group - * generated address sets if same name is used */ - const struct nbrec_address_set *nb_address_set; - NBREC_ADDRESS_SET_TABLE_FOR_EACH (nb_address_set, - input_data->nbrec_address_set_table) { - sync_address_set(ovnsb_txn, nb_address_set->name, - /* "char **" is not compatible with "const char **" */ - (const char **)nb_address_set->addresses, - nb_address_set->n_addresses, &sb_address_sets); - } - - struct shash_node *node; - SHASH_FOR_EACH_SAFE (node, &sb_address_sets) { - sbrec_address_set_delete(node->data); - shash_delete(&sb_address_sets, node); - } - shash_destroy(&sb_address_sets); -} - /* Each port group in Port_Group table in OVN_Northbound has a corresponding * entry in Port_Group table in OVN_Southbound. In OVN_Northbound the entries * contains lport uuids, while in OVN_Southbound we store the lport names. @@ -15918,7 +15759,6 @@ ovnnb_db_run(struct northd_input *input_data, ovn_update_ipv6_prefix(&data->ports); sync_lbs(input_data, ovnsb_txn, &data->datapaths, &data->lbs); - sync_address_sets(input_data, ovnsb_txn, &data->datapaths); sync_port_groups(input_data, ovnsb_txn, &data->port_groups); sync_meters(input_data, ovnsb_txn, &data->meter_groups); sync_dns_entries(input_data, ovnsb_txn, &data->datapaths); @@ -16195,3 +16035,8 @@ void northd_run(struct northd_input *input_data, stopwatch_stop(OVNSB_DB_RUN_STOPWATCH_NAME, time_msec()); } +const char * +northd_get_svc_monitor_mac(void) +{ + return svc_monitor_mac; +} diff --git a/northd/northd.h b/northd/northd.h index da90e2815..ea9bd5797 100644 --- a/northd/northd.h +++ b/northd/northd.h @@ -31,7 +31,6 @@ struct northd_input { const struct nbrec_load_balancer_group_table *nbrec_load_balancer_group_table; const struct nbrec_port_group_table *nbrec_port_group_table; - const struct nbrec_address_set_table *nbrec_address_set_table; const struct nbrec_meter_table *nbrec_meter_table; const struct nbrec_acl_table *nbrec_acl_table; const struct nbrec_static_mac_binding_table @@ -47,7 +46,6 @@ struct northd_input { const struct sbrec_fdb_table *sbrec_fdb_table; const struct sbrec_load_balancer_table *sbrec_load_balancer_table; const struct sbrec_service_monitor_table *sbrec_service_monitor_table; - const struct sbrec_address_set_table *sbrec_address_set_table; const struct sbrec_port_group_table *sbrec_port_group_table; const struct sbrec_meter_table *sbrec_meter_table; const struct sbrec_dns_table *sbrec_dns_table; @@ -281,4 +279,5 @@ void bfd_cleanup_connections(struct lflow_input *input_data, struct hmap *bfd_map); void run_update_worker_pool(int n_threads); +const char *northd_get_svc_monitor_mac(void); #endif /* NORTHD_H */ From patchwork Tue Nov 22 03:50:16 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Numan Siddique X-Patchwork-Id: 1707640 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::137; helo=smtp4.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp4.osuosl.org (smtp4.osuosl.org [IPv6:2605:bc80:3010::137]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4NGVfK6pvfz23nj for ; Tue, 22 Nov 2022 14:50:49 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id BC9254150E; Tue, 22 Nov 2022 03:50:47 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org BC9254150E X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id gBe9uI8yswOD; Tue, 22 Nov 2022 03:50:46 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp4.osuosl.org (Postfix) with ESMTPS id 1C5BF414E4; Tue, 22 Nov 2022 03:50:45 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 1C5BF414E4 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id D7EADC0032; Tue, 22 Nov 2022 03:50:44 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp4.osuosl.org (smtp4.osuosl.org [140.211.166.137]) by lists.linuxfoundation.org (Postfix) with ESMTP id D7EE2C002D for ; Tue, 22 Nov 2022 03:50:42 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 00117414E4 for ; Tue, 22 Nov 2022 03:50:34 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 00117414E4 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id JlDl43dFfJYl for ; Tue, 22 Nov 2022 03:50:33 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org B749B41503 Received: from relay11.mail.gandi.net (relay11.mail.gandi.net [217.70.178.231]) by smtp4.osuosl.org (Postfix) with ESMTPS id B749B41503 for ; Tue, 22 Nov 2022 03:50:32 +0000 (UTC) Received: (Authenticated sender: numans@ovn.org) by mail.gandi.net (Postfix) with ESMTPSA id 49C0F100003; Tue, 22 Nov 2022 03:50:27 +0000 (UTC) From: numans@ovn.org To: dev@openvswitch.org Date: Mon, 21 Nov 2022 22:50:16 -0500 Message-Id: <20221122035016.1973953-1-numans@ovn.org> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221122034908.1973726-1-numans@ovn.org> References: <20221122034908.1973726-1-numans@ovn.org> MIME-Version: 1.0 Subject: [ovs-dev] [PATCH ovn v4 3/3] northd: Add I-P for syncing SB address sets. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Numan Siddique Updates to NB address sets and NB port groups are handled incrementally for syncing the SB address sets. This patch doesn't support syncing the SB Address sets for the router load balancer vips incrementally, instead a full recompute is triggered for any changes to NB load balancers, NB load balancer groups and NB logical routers or if the 'en_northd' engine node changes (which is the case for all the Northbound changes except for the NB Address set for now). Acked-by: Mark Michelson Acked-by: Ales Musil Signed-off-by: Numan Siddique Acked-by: Han Zhou --- northd/en-sync-sb.c | 202 ++++++++++++++++++++++++++++++++++++--- northd/en-sync-sb.h | 5 + northd/inc-proc-northd.c | 12 ++- northd/ovn-northd.c | 2 + tests/ovn-northd.at | 102 ++++++++++++++++++++ 5 files changed, 306 insertions(+), 17 deletions(-) diff --git a/northd/en-sync-sb.c b/northd/en-sync-sb.c index 6e8182f42..6e33901a8 100644 --- a/northd/en-sync-sb.c +++ b/northd/en-sync-sb.c @@ -22,6 +22,7 @@ #include "openvswitch/util.h" #include "en-sync-sb.h" +#include "include/ovn/expr.h" #include "lib/inc-proc-eng.h" #include "lib/lb.h" #include "lib/ovn-nb-idl.h" @@ -41,6 +42,13 @@ static void sync_addr_sets(const struct nbrec_address_set_table *, const struct sbrec_address_set_table *, struct ovsdb_idl_txn *ovnsb_txn, struct hmap *datapaths); +static const struct sbrec_address_set *sb_address_set_lookup_by_name( + struct ovsdb_idl_index *, const char *name); +static void update_sb_addr_set(const char **nb_addresses, size_t n_addresses, + const struct sbrec_address_set *); +static void build_port_group_address_set(const struct nbrec_port_group *, + struct svec *ipv4_addrs, + struct svec *ipv6_addrs); void * en_sync_to_sb_init(struct engine_node *node OVS_UNUSED, @@ -94,6 +102,98 @@ en_sync_to_sb_addr_set_cleanup(void *data OVS_UNUSED) } +bool +sync_to_sb_addr_set_nb_address_set_handler(struct engine_node *node, + void *data OVS_UNUSED) +{ + const struct nbrec_address_set_table *nb_address_set_table = + EN_OVSDB_GET(engine_get_input("NB_address_set", node)); + + /* Return false if an address set is created or deleted. + * Handle I-P for only updated address sets. */ + const struct nbrec_address_set *nb_addr_set; + NBREC_ADDRESS_SET_TABLE_FOR_EACH_TRACKED (nb_addr_set, + nb_address_set_table) { + if (nbrec_address_set_is_new(nb_addr_set) || + nbrec_address_set_is_deleted(nb_addr_set)) { + return false; + } + } + + struct ovsdb_idl_index *sbrec_address_set_by_name = + engine_ovsdb_node_get_index( + engine_get_input("SB_address_set", node), + "sbrec_address_set_by_name"); + + NBREC_ADDRESS_SET_TABLE_FOR_EACH_TRACKED (nb_addr_set, + nb_address_set_table) { + const struct sbrec_address_set *sb_addr_set = + sb_address_set_lookup_by_name(sbrec_address_set_by_name, + nb_addr_set->name); + if (!sb_addr_set) { + return false; + } + update_sb_addr_set((const char **) nb_addr_set->addresses, + nb_addr_set->n_addresses, sb_addr_set); + } + + return true; +} + +bool +sync_to_sb_addr_set_nb_port_group_handler(struct engine_node *node, + void *data OVS_UNUSED) +{ + const struct nbrec_port_group *nb_pg; + const struct nbrec_port_group_table *nb_port_group_table = + EN_OVSDB_GET(engine_get_input("NB_port_group", node)); + NBREC_PORT_GROUP_TABLE_FOR_EACH_TRACKED (nb_pg, nb_port_group_table) { + if (nbrec_port_group_is_new(nb_pg) || + nbrec_port_group_is_deleted(nb_pg)) { + return false; + } + } + + struct ovsdb_idl_index *sbrec_address_set_by_name = + engine_ovsdb_node_get_index( + engine_get_input("SB_address_set", node), + "sbrec_address_set_by_name"); + NBREC_PORT_GROUP_TABLE_FOR_EACH_TRACKED (nb_pg, nb_port_group_table) { + char *ipv4_addrs_name = xasprintf("%s_ip4", nb_pg->name); + const struct sbrec_address_set *sb_addr_set_v4 = + sb_address_set_lookup_by_name(sbrec_address_set_by_name, + ipv4_addrs_name); + if (!sb_addr_set_v4) { + free(ipv4_addrs_name); + return false; + } + char *ipv6_addrs_name = xasprintf("%s_ip6", nb_pg->name); + const struct sbrec_address_set *sb_addr_set_v6 = + sb_address_set_lookup_by_name(sbrec_address_set_by_name, + ipv6_addrs_name); + if (!sb_addr_set_v6) { + free(ipv4_addrs_name); + free(ipv6_addrs_name); + return false; + } + + struct svec ipv4_addrs = SVEC_EMPTY_INITIALIZER; + struct svec ipv6_addrs = SVEC_EMPTY_INITIALIZER; + build_port_group_address_set(nb_pg, &ipv4_addrs, &ipv6_addrs); + update_sb_addr_set((const char **) ipv4_addrs.names, ipv4_addrs.n, + sb_addr_set_v4); + update_sb_addr_set((const char **) ipv6_addrs.names, ipv6_addrs.n, + sb_addr_set_v6); + + free(ipv4_addrs_name); + free(ipv6_addrs_name); + svec_destroy(&ipv4_addrs); + svec_destroy(&ipv6_addrs); + } + + return true; +} + /* static functions. */ static void sync_addr_set(struct ovsdb_idl_txn *ovnsb_txn, const char *name, @@ -106,10 +206,11 @@ sync_addr_set(struct ovsdb_idl_txn *ovnsb_txn, const char *name, if (!sb_address_set) { sb_address_set = sbrec_address_set_insert(ovnsb_txn); sbrec_address_set_set_name(sb_address_set, name); + sbrec_address_set_set_addresses(sb_address_set, + addrs, n_addrs); + } else { + update_sb_addr_set(addrs, n_addrs, sb_address_set); } - - sbrec_address_set_set_addresses(sb_address_set, - addrs, n_addrs); } /* OVN_Southbound Address_Set table contains same records as in north @@ -151,18 +252,7 @@ sync_addr_sets(const struct nbrec_address_set_table *nb_address_set_table, nb_port_group_table) { struct svec ipv4_addrs = SVEC_EMPTY_INITIALIZER; struct svec ipv6_addrs = SVEC_EMPTY_INITIALIZER; - for (size_t i = 0; i < nb_port_group->n_ports; i++) { - for (size_t j = 0; j < nb_port_group->ports[i]->n_addresses; j++) { - const char *addrs = nb_port_group->ports[i]->addresses[j]; - if (!is_dynamic_lsp_address(addrs)) { - split_addresses(addrs, &ipv4_addrs, &ipv6_addrs); - } - } - if (nb_port_group->ports[i]->dynamic_addresses) { - split_addresses(nb_port_group->ports[i]->dynamic_addresses, - &ipv4_addrs, &ipv6_addrs); - } - } + build_port_group_address_set(nb_port_group, &ipv4_addrs, &ipv6_addrs); char *ipv4_addrs_name = xasprintf("%s_ip4", nb_port_group->name); char *ipv6_addrs_name = xasprintf("%s_ip6", nb_port_group->name); sync_addr_set(ovnsb_txn, ipv4_addrs_name, @@ -231,3 +321,85 @@ sync_addr_sets(const struct nbrec_address_set_table *nb_address_set_table, } shash_destroy(&sb_address_sets); } + +static void +update_sb_addr_set(const char **nb_addresses, size_t n_addresses, + const struct sbrec_address_set *sb_as) +{ + struct expr_constant_set *cs_nb_as = + expr_constant_set_create_integers( + (const char *const *) nb_addresses, n_addresses); + struct expr_constant_set *cs_sb_as = + expr_constant_set_create_integers( + (const char *const *) sb_as->addresses, sb_as->n_addresses); + + struct expr_constant_set *addr_added = NULL; + struct expr_constant_set *addr_deleted = NULL; + expr_constant_set_integers_diff(cs_sb_as, cs_nb_as, &addr_added, + &addr_deleted); + + struct ds ds = DS_EMPTY_INITIALIZER; + if (addr_added && addr_added->n_values) { + for (size_t i = 0; i < addr_added->n_values; i++) { + ds_clear(&ds); + expr_constant_format(&addr_added->values[i], EXPR_C_INTEGER, &ds); + sbrec_address_set_update_addresses_addvalue(sb_as, ds_cstr(&ds)); + } + } + + if (addr_deleted && addr_deleted->n_values) { + for (size_t i = 0; i < addr_deleted->n_values; i++) { + ds_clear(&ds); + expr_constant_format(&addr_deleted->values[i], + EXPR_C_INTEGER, &ds); + sbrec_address_set_update_addresses_delvalue(sb_as, ds_cstr(&ds)); + } + } + + ds_destroy(&ds); + expr_constant_set_destroy(cs_nb_as); + free(cs_nb_as); + expr_constant_set_destroy(cs_sb_as); + free(cs_sb_as); + expr_constant_set_destroy(addr_added); + free(addr_added); + expr_constant_set_destroy(addr_deleted); + free(addr_deleted); +} + +static void +build_port_group_address_set(const struct nbrec_port_group *nb_port_group, + struct svec *ipv4_addrs, + struct svec *ipv6_addrs) +{ + for (size_t i = 0; i < nb_port_group->n_ports; i++) { + for (size_t j = 0; j < nb_port_group->ports[i]->n_addresses; j++) { + const char *addrs = nb_port_group->ports[i]->addresses[j]; + if (!is_dynamic_lsp_address(addrs)) { + split_addresses(addrs, ipv4_addrs, ipv6_addrs); + } + } + if (nb_port_group->ports[i]->dynamic_addresses) { + split_addresses(nb_port_group->ports[i]->dynamic_addresses, + ipv4_addrs, ipv6_addrs); + } + } +} + +/* Finds and returns the address set with the given 'name', or NULL if no such + * address set exists. */ +static const struct sbrec_address_set * +sb_address_set_lookup_by_name(struct ovsdb_idl_index *sbrec_addr_set_by_name, + const char *name) +{ + struct sbrec_address_set *target = sbrec_address_set_index_init_row( + sbrec_addr_set_by_name); + sbrec_address_set_index_set_name(target, name); + + struct sbrec_address_set *retval = sbrec_address_set_index_find( + sbrec_addr_set_by_name, target); + + sbrec_address_set_index_destroy_row(target); + + return retval; +} diff --git a/northd/en-sync-sb.h b/northd/en-sync-sb.h index 1b25ff90f..bcb9799d2 100644 --- a/northd/en-sync-sb.h +++ b/northd/en-sync-sb.h @@ -11,4 +11,9 @@ void *en_sync_to_sb_addr_set_init(struct engine_node *, struct engine_arg *); void en_sync_to_sb_addr_set_run(struct engine_node *, void *data); void en_sync_to_sb_addr_set_cleanup(void *data); +bool sync_to_sb_addr_set_nb_address_set_handler(struct engine_node *, + void *data); +bool sync_to_sb_addr_set_nb_port_group_handler(struct engine_node *, + void *data); + #endif /* end of EN_SYNC_SB_H */ diff --git a/northd/inc-proc-northd.c b/northd/inc-proc-northd.c index dbba41127..73f230b2c 100644 --- a/northd/inc-proc-northd.c +++ b/northd/inc-proc-northd.c @@ -232,8 +232,10 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, * once I-P engine allows multiple root nodes. */ engine_add_input(&en_lflow, &en_mac_binding_aging, NULL); - engine_add_input(&en_sync_to_sb_addr_set, &en_nb_address_set, NULL); - engine_add_input(&en_sync_to_sb_addr_set, &en_nb_port_group, NULL); + engine_add_input(&en_sync_to_sb_addr_set, &en_nb_address_set, + sync_to_sb_addr_set_nb_address_set_handler); + engine_add_input(&en_sync_to_sb_addr_set, &en_nb_port_group, + sync_to_sb_addr_set_nb_port_group_handler); engine_add_input(&en_sync_to_sb_addr_set, &en_northd, NULL); engine_add_input(&en_sync_to_sb_addr_set, &en_sb_address_set, NULL); @@ -290,6 +292,12 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, engine_ovsdb_node_add_index(&en_sb_mac_binding, "sbrec_mac_binding_by_datapath", sbrec_mac_binding_by_datapath); + + struct ovsdb_idl_index *sbrec_address_set_by_name + = ovsdb_idl_index_create1(sb->idl, &sbrec_address_set_col_name); + engine_ovsdb_node_add_index(&en_sb_address_set, + "sbrec_address_set_by_name", + sbrec_address_set_by_name); } void inc_proc_northd_run(struct ovsdb_idl_txn *ovnnb_txn, diff --git a/northd/ovn-northd.c b/northd/ovn-northd.c index faf7cbeb5..965353cd7 100644 --- a/northd/ovn-northd.c +++ b/northd/ovn-northd.c @@ -796,6 +796,8 @@ main(int argc, char *argv[]) /* Disable alerting for pure write-only columns. */ ovsdb_idl_omit_alert(ovnsb_idl_loop.idl, &sbrec_sb_global_col_nb_cfg); + ovsdb_idl_omit_alert(ovnsb_idl_loop.idl, &sbrec_address_set_col_name); + ovsdb_idl_omit_alert(ovnsb_idl_loop.idl, &sbrec_address_set_col_addresses); unixctl_command_register("sb-connection-status", "", 0, 0, ovn_conn_show, ovnsb_idl_loop.idl); diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at index 92b7f4be2..bc8334034 100644 --- a/tests/ovn-northd.at +++ b/tests/ovn-northd.at @@ -8007,3 +8007,105 @@ AT_CHECK([grep "lr_in_lb_aff_learn" R1flows | sort], [0], [dnl AT_CLEANUP ]) + +OVN_FOR_EACH_NORTHD_NO_HV([ +AT_SETUP([Address set incremental processing]) +ovn_start + +# Create a few resources. +check ovn-nbctl ls-add sw0 +check ovn-nbctl lsp-add sw0 sw0-p1 -- lsp-set-addresses sw0-p1 "02:ac:10:01:00:04 20.0.0.4" +check ovn-nbctl lsp-add sw0 sw0-p2 -- lsp-set-addresses sw0-p2 "02:ac:10:01:00:05 20.0.0.5" + +fetch_column NB:Logical_Switch_Port _uuid name=sw0-p1 + +ovn-nbctl list logical_switch_port name=sw0-p1 + +p1_uuid=$(fetch_column nb:Logical_Switch_Port _uuid name=sw0-p1) +p2_uuid=$(fetch_column nb:Logical_Switch_Port _uuid name=sw0-p2) + +echo "p1 uuid - $p1_uuid" +ovn-nbctl --wait=sb sync + +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +foo_as_uuid=$(ovn-nbctl create address_set name=foo addresses=\"1.1.1.1\",\"1.1.1.2\") +wait_column '1.1.1.1 1.1.1.2' Address_Set addresses name=foo +AT_CHECK([as northd ovn-appctl -t NORTHD_TYPE inc-engine/show-stats sync_to_sb_addr_set recompute], [0], [1 +]) + +rm -f northd/ovn-northd.log +check as northd ovn-appctl -t NORTHD_TYPE vlog/reopen +check as northd ovn-appctl -t NORTHD_TYPE vlog/set jsonrpc:dbg +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl add address_set $foo_as_uuid addresses 1.1.1.3 +wait_column '1.1.1.1 1.1.1.2 1.1.1.3' Address_Set addresses name=foo + +# There should be no recompute of the sync_to_sb_addr_set engine node . +AT_CHECK([as northd ovn-appctl -t NORTHD_TYPE inc-engine/show-stats sync_to_sb_addr_set recompute], [0], [0 +]) + +AT_CHECK([grep transact northd/ovn-northd.log | grep Address_Set | \ +grep -c mutate], [0], [1 +]) + +check ovn-nbctl add address_set $foo_as_uuid addresses \ +1.1.1.4 -- remove address_set $foo_as_uuid addresses 1.1.1.1 +wait_column '1.1.1.2 1.1.1.3 1.1.1.4' Address_Set addresses name=foo + +# There should be no recompute of the sync_to_sb_addr_set engine node . +AT_CHECK([as northd ovn-appctl -t NORTHD_TYPE inc-engine/show-stats sync_to_sb_addr_set recompute], [0], [0 +]) + +AT_CHECK([grep transact northd/ovn-northd.log | grep Address_Set | \ +grep -c mutate], [0], [2 +]) + +# Pause ovn-northd and add/remove few addresses. when it is resumed +# it should use mutate for updating the address sets. +check as northd ovn-appctl -t NORTHD_TYPE pause +check as northd-backup ovn-appctl -t NORTHD_TYPE pause + +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl add address_set $foo_as_uuid addresses 1.1.1.5 +check ovn-nbctl add address_set $foo_as_uuid addresses 1.1.1.6 +check ovn-nbctl remove address_set $foo_as_uuid addresses 1.1.1.2 + +check_column '1.1.1.2 1.1.1.3 1.1.1.4' Address_Set addresses name=foo + +# Resume northd now +check as northd ovn-appctl -t NORTHD_TYPE resume +wait_column '1.1.1.3 1.1.1.4 1.1.1.5 1.1.1.6' Address_Set addresses name=foo + +# There should be recompute of the sync_to_sb_addr_set engine node . +AT_CHECK([as northd ovn-appctl -t NORTHD_TYPE inc-engine/show-stats sync_to_sb_addr_set recompute], [0], [1 +]) + +AT_CHECK([grep transact northd/ovn-northd.log | grep Address_Set | \ +grep -c mutate], [0], [3 +]) + +# Create a port group. This should result in recompute of sb_to_sync_addr_set engine node. +check ovn-nbctl pg-add pg1 +wait_column '' Address_Set addresses name=pg1_ip4 + +AT_CHECK([as northd ovn-appctl -t NORTHD_TYPE inc-engine/show-stats sync_to_sb_addr_set recompute], [0], [2 +]) + +# Add sw0-p1 to port group pg1 +check ovn-nbctl add port_group pg1 ports ${p1_uuid} + +wait_column '20.0.0.4' Address_Set addresses name=pg1_ip4 + +# There should be recompute of the sync_to_sb_addr_set engine node since northd engine changes. +# There will be another recompute when the update message is received from the sb ovsdb-server. +# Once we add I-P for Port_Groups, there should be no recompute here. +AT_CHECK([as northd ovn-appctl -t NORTHD_TYPE inc-engine/show-stats sync_to_sb_addr_set recompute], [0], [4 +]) + +# Any change to northd engine node should result in full recompute of sync_to_sb_addr_set node. +check ovn-nbctl --wait=sb sync +AT_CHECK([as northd ovn-appctl -t NORTHD_TYPE inc-engine/show-stats sync_to_sb_addr_set recompute], [0], [5 +]) + +AT_CLEANUP +])