From patchwork Fri Aug 18 08:57:13 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Numan Siddique X-Patchwork-Id: 1822752 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::137; helo=smtp4.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp4.osuosl.org (smtp4.osuosl.org [IPv6:2605:bc80:3010::137]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4RRwk31Qq0z1ygH for ; Fri, 18 Aug 2023 18:57:30 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 4364440139; Fri, 18 Aug 2023 08:57:29 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 4364440139 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id lpjsXuwdP6cQ; Fri, 18 Aug 2023 08:57:27 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp4.osuosl.org (Postfix) with ESMTPS id D85C040224; Fri, 18 Aug 2023 08:57:26 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org D85C040224 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id B0E92C0039; Fri, 18 Aug 2023 08:57:26 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp1.osuosl.org (smtp1.osuosl.org [IPv6:2605:bc80:3010::138]) by lists.linuxfoundation.org (Postfix) with ESMTP id 1CCB3C0032 for ; Fri, 18 Aug 2023 08:57:26 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id DF96383C81 for ; Fri, 18 Aug 2023 08:57:25 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org DF96383C81 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id cIODdzUlmNbL for ; Fri, 18 Aug 2023 08:57:24 +0000 (UTC) Received: from relay9-d.mail.gandi.net (relay9-d.mail.gandi.net [IPv6:2001:4b98:dc4:8::229]) by smtp1.osuosl.org (Postfix) with ESMTPS id 5A1B783C86 for ; Fri, 18 Aug 2023 08:57:24 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 5A1B783C86 Received: by mail.gandi.net (Postfix) with ESMTPSA id 83C46FF809; Fri, 18 Aug 2023 08:57:20 +0000 (UTC) From: numans@ovn.org To: dev@openvswitch.org Date: Fri, 18 Aug 2023 14:27:13 +0530 Message-Id: <20230818085713.1030920-1-numans@ovn.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230818085606.1030792-1-numans@ovn.org> References: <20230818085606.1030792-1-numans@ovn.org> MIME-Version: 1.0 X-GND-Sasl: numans@ovn.org Subject: [ovs-dev] [PATCH ovn v6 01/16] northd I-P: Sync SB load balancers in a separate engine node. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Numan Siddique Similar to the commit [1], a new sub-engine node "sync_to_sb_lb" is added with-in the "sync_to_sb" to sync the SB load balancers. Its main input nodes are "northd" (to access the "lbs" hmap built by this node) and "sb_load_balancer" to access the SB load balancer. "sync_to_sb_lb" doesn't add any handlers and falls back to full recompute all the time. [1] - ccafcc2dc321("northd I-P: Add a new engine node 'en_sync_to_sb' to sync SB tables.") Signed-off-by: Numan Siddique Acked-by: Han Zhou Reviewed-by: Ales Musil --- northd/en-northd.c | 2 -- northd/en-sync-sb.c | 43 ++++++++++++++++++++++++++++++++++++++++ northd/en-sync-sb.h | 6 ++++++ northd/inc-proc-northd.c | 10 ++++++++-- northd/northd.c | 4 +--- northd/northd.h | 6 +++++- tests/ovn-northd.at | 23 +++++++++++++++++++++ 7 files changed, 86 insertions(+), 8 deletions(-) diff --git a/northd/en-northd.c b/northd/en-northd.c index 044fa70190..f9f2d04452 100644 --- a/northd/en-northd.c +++ b/northd/en-northd.c @@ -101,8 +101,6 @@ northd_get_input_data(struct engine_node *node, EN_OVSDB_GET(engine_get_input("SB_chassis", node)); input_data->sbrec_fdb_table = EN_OVSDB_GET(engine_get_input("SB_fdb", node)); - input_data->sbrec_load_balancer_table = - EN_OVSDB_GET(engine_get_input("SB_load_balancer", node)); input_data->sbrec_service_monitor_table = EN_OVSDB_GET(engine_get_input("SB_service_monitor", node)); input_data->sbrec_port_group_table = diff --git a/northd/en-sync-sb.c b/northd/en-sync-sb.c index d7fea981f2..dbe3c63752 100644 --- a/northd/en-sync-sb.c +++ b/northd/en-sync-sb.c @@ -211,6 +211,49 @@ sync_to_sb_addr_set_nb_port_group_handler(struct engine_node *node, return true; } +/* sync_to_sb_lb engine node functions. + * This engine node syncs the SB load balancers. + */ +void * +en_sync_to_sb_lb_init(struct engine_node *node OVS_UNUSED, + struct engine_arg *arg OVS_UNUSED) +{ + return NULL; +} + +void +en_sync_to_sb_lb_run(struct engine_node *node, void *data OVS_UNUSED) +{ + const struct sbrec_load_balancer_table *sb_load_balancer_table = + EN_OVSDB_GET(engine_get_input("SB_load_balancer", node)); + const struct engine_context *eng_ctx = engine_get_context(); + struct northd_data *northd_data = engine_get_input_data("northd", node); + + sync_lbs(eng_ctx->ovnsb_idl_txn, sb_load_balancer_table, + &northd_data->ls_datapaths, &northd_data->lbs); + engine_set_node_state(node, EN_UPDATED); +} + +void +en_sync_to_sb_lb_cleanup(void *data OVS_UNUSED) +{ + +} + +bool +sync_to_sb_lb_northd_handler(struct engine_node *node, void *data OVS_UNUSED) +{ + struct northd_data *nd = engine_get_input_data("northd", node); + if (nd->change_tracked) { + /* There are only NB LSP related changes and these can be safely + * ignore and returned true. However in case the northd engine + * tracking data includes other changes, we need to do additional + * checks before safely ignoring. */ + return true; + } + return false; +} + /* static functions. */ static void sync_addr_set(struct ovsdb_idl_txn *ovnsb_txn, const char *name, diff --git a/northd/en-sync-sb.h b/northd/en-sync-sb.h index bcb9799d24..06d2a57710 100644 --- a/northd/en-sync-sb.h +++ b/northd/en-sync-sb.h @@ -16,4 +16,10 @@ bool sync_to_sb_addr_set_nb_address_set_handler(struct engine_node *, bool sync_to_sb_addr_set_nb_port_group_handler(struct engine_node *, void *data); + +void *en_sync_to_sb_lb_init(struct engine_node *, struct engine_arg *); +void en_sync_to_sb_lb_run(struct engine_node *, void *data); +void en_sync_to_sb_lb_cleanup(void *data); +bool sync_to_sb_lb_northd_handler(struct engine_node *, void *data OVS_UNUSED); + #endif /* end of EN_SYNC_SB_H */ diff --git a/northd/inc-proc-northd.c b/northd/inc-proc-northd.c index d328deb222..c0874d5294 100644 --- a/northd/inc-proc-northd.c +++ b/northd/inc-proc-northd.c @@ -139,6 +139,7 @@ static ENGINE_NODE(sync_to_sb, "sync_to_sb"); static ENGINE_NODE(sync_to_sb_addr_set, "sync_to_sb_addr_set"); static ENGINE_NODE(fdb_aging, "fdb_aging"); static ENGINE_NODE(fdb_aging_waker, "fdb_aging_waker"); +static ENGINE_NODE(sync_to_sb_lb, "sync_to_sb_lb"); void inc_proc_northd_init(struct ovsdb_idl_loop *nb, struct ovsdb_idl_loop *sb) @@ -166,7 +167,6 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, engine_add_input(&en_northd, &en_sb_ha_chassis_group, NULL); engine_add_input(&en_northd, &en_sb_ip_multicast, NULL); engine_add_input(&en_northd, &en_sb_service_monitor, NULL); - engine_add_input(&en_northd, &en_sb_load_balancer, NULL); engine_add_input(&en_northd, &en_sb_fdb, NULL); engine_add_input(&en_northd, &en_sb_static_mac_binding, NULL); engine_add_input(&en_northd, &en_sb_chassis_template_var, NULL); @@ -202,11 +202,17 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, engine_add_input(&en_sync_to_sb_addr_set, &en_northd, NULL); engine_add_input(&en_sync_to_sb_addr_set, &en_sb_address_set, NULL); + engine_add_input(&en_sync_to_sb_lb, &en_northd, + sync_to_sb_lb_northd_handler); + engine_add_input(&en_sync_to_sb_lb, &en_sb_load_balancer, NULL); + /* en_sync_to_sb engine node syncs the SB database tables from * the NB database tables. - * Right now this engine only syncs the SB Address_Set table. + * Right now this engine syncs the SB Address_Set table and + * SB Load_Balancer table. */ engine_add_input(&en_sync_to_sb, &en_sync_to_sb_addr_set, NULL); + engine_add_input(&en_sync_to_sb, &en_sync_to_sb_lb, NULL); engine_add_input(&en_sync_from_sb, &en_northd, sync_from_sb_northd_handler); diff --git a/northd/northd.c b/northd/northd.c index 0a749931ee..a3bce551bc 100644 --- a/northd/northd.c +++ b/northd/northd.c @@ -4538,7 +4538,7 @@ ovn_dp_group_get_or_create(struct ovsdb_idl_txn *ovnsb_txn, /* Syncs relevant load balancers (applied to logical switches) to the * Southbound database. */ -static void +void sync_lbs(struct ovsdb_idl_txn *ovnsb_txn, const struct sbrec_load_balancer_table *sbrec_load_balancer_table, struct ovn_datapaths *ls_datapaths, struct hmap *lbs) @@ -17619,8 +17619,6 @@ ovnnb_db_run(struct northd_input *input_data, ovn_update_ipv6_options(&data->lr_ports); ovn_update_ipv6_prefix(&data->lr_ports); - sync_lbs(ovnsb_txn, input_data->sbrec_load_balancer_table, - &data->ls_datapaths, &data->lbs); sync_port_groups(ovnsb_txn, input_data->sbrec_port_group_table, &data->port_groups); sync_meters(ovnsb_txn, input_data->nbrec_meter_table, diff --git a/northd/northd.h b/northd/northd.h index f3e63b1e1a..48c282476a 100644 --- a/northd/northd.h +++ b/northd/northd.h @@ -16,6 +16,7 @@ #include "ovsdb-idl.h" +#include "lib/ovn-sb-idl.h" #include "lib/ovn-util.h" #include "lib/ovs-atomic.h" #include "lib/sset.h" @@ -47,7 +48,6 @@ struct northd_input { const struct sbrec_ha_chassis_group_table *sbrec_ha_chassis_group_table; const struct sbrec_chassis_table *sbrec_chassis_table; const struct sbrec_fdb_table *sbrec_fdb_table; - const struct sbrec_load_balancer_table *sbrec_load_balancer_table; const struct sbrec_service_monitor_table *sbrec_service_monitor_table; const struct sbrec_port_group_table *sbrec_port_group_table; const struct sbrec_meter_table *sbrec_meter_table; @@ -354,4 +354,8 @@ void bfd_cleanup_connections(const struct nbrec_bfd_table *, void run_update_worker_pool(int n_threads); const char *northd_get_svc_monitor_mac(void); + +void sync_lbs(struct ovsdb_idl_txn *, const struct sbrec_load_balancer_table *, + struct ovn_datapaths *ls_datapaths, struct hmap *lbs); + #endif /* NORTHD_H */ diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at index d5be3be75b..9e36f96182 100644 --- a/tests/ovn-northd.at +++ b/tests/ovn-northd.at @@ -9512,6 +9512,29 @@ AT_CHECK([ovn-sbctl lflow-list sw | grep ls_out_pre_lb | grep priority=110 | gre AT_CLEANUP ]) + +# This test checks the compute/recompute stats of sync_to_sb_lb engine node +OVN_FOR_EACH_NORTHD_NO_HV([ +AT_SETUP([sync_to_sb_lb incremental processing]) +ovn_start + +check ovn-nbctl ls-add sw +check ovn-nbctl lsp-add sw p1 + +check ovn-nbctl --wait=sb sync +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats + +check ovn-nbctl --wait=sb lsp-set-options p1 foo=bar +sb_lb_recomp=$(as northd ovn-appctl -t NORTHD_TYPE inc-engine/show-stats sync_to_sb_lb recompute) +AT_CHECK([test x$sb_lb_recomp = x0]) + +check ovn-nbctl --wait=sb lsp-set-type p1 external +sb_lb_recomp=$(as northd ovn-appctl -t NORTHD_TYPE inc-engine/show-stats sync_to_sb_lb recompute) +AT_CHECK([test x$sb_lb_recomp != x0]) + +AT_CLEANUP +]) + OVN_FOR_EACH_NORTHD_NO_HV([ AT_SETUP([LSP incremental processing]) ovn_start From patchwork Fri Aug 18 08:57:21 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Numan Siddique X-Patchwork-Id: 1822753 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.133; helo=smtp2.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4RRwkG6n46z1ygH for ; Fri, 18 Aug 2023 18:57:42 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 92A6841919; Fri, 18 Aug 2023 08:57:40 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 92A6841919 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id pxCIdrvrfD40; Fri, 18 Aug 2023 08:57:37 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp2.osuosl.org (Postfix) with ESMTPS id A1F72405A7; Fri, 18 Aug 2023 08:57:36 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org A1F72405A7 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 7A8B6C0072; Fri, 18 Aug 2023 08:57:36 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by lists.linuxfoundation.org (Postfix) with ESMTP id 4E42CC0032 for ; Fri, 18 Aug 2023 08:57:35 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 1A2BE405F9 for ; Fri, 18 Aug 2023 08:57:35 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 1A2BE405F9 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id EWorfZ7Pz6Qu for ; Fri, 18 Aug 2023 08:57:32 +0000 (UTC) Received: from relay6-d.mail.gandi.net (relay6-d.mail.gandi.net [IPv6:2001:4b98:dc4:8::226]) by smtp2.osuosl.org (Postfix) with ESMTPS id 812394169B for ; Fri, 18 Aug 2023 08:57:31 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 812394169B Received: by mail.gandi.net (Postfix) with ESMTPSA id B2624C0003; Fri, 18 Aug 2023 08:57:27 +0000 (UTC) From: numans@ovn.org To: dev@openvswitch.org Date: Fri, 18 Aug 2023 14:27:21 +0530 Message-Id: <20230818085721.1030934-1-numans@ovn.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230818085606.1030792-1-numans@ovn.org> References: <20230818085606.1030792-1-numans@ovn.org> MIME-Version: 1.0 X-GND-Sasl: numans@ovn.org Subject: [ovs-dev] [PATCH ovn v6 02/16] northd: Add a new engine node - lb_data. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Numan Siddique This patch separates out the 'lbs' and 'lb_groups' from the 'northd' engine node data into a new engine node 'lb_data'. This new node becomes an input to the 'northd' node. This makes handling the NB load balancer and load balancer group changes easier. Prior to this patch, 'struct ovn_northd_lb' and 'struct ovn_lb_group' were maintaing the logical switch and logical router association with a load balancer. This data is now maintained by 'northd' engine node. New structs 'struct ovn_lb_datapaths' and 'struct ovn_lb_group_datapaths' is added for this purpose. Acked-by: Han Zhou Signed-off-by: Numan Siddique Reviewed-by: Ales Musil --- lib/lb.c | 205 ++++++++++-- lib/lb.h | 86 +++-- northd/automake.mk | 2 + northd/en-lb-data.c | 125 ++++++++ northd/en-lb-data.h | 23 ++ northd/en-lflow.c | 3 +- northd/en-northd.c | 11 +- northd/en-sync-sb.c | 2 +- northd/inc-proc-northd.c | 8 +- northd/northd.c | 669 ++++++++++++++++++++++----------------- northd/northd.h | 15 +- 11 files changed, 781 insertions(+), 368 deletions(-) create mode 100644 northd/en-lb-data.c create mode 100644 northd/en-lb-data.h diff --git a/lib/lb.c b/lib/lb.c index 7afdaed65b..e874680a3f 100644 --- a/lib/lb.c +++ b/lib/lb.c @@ -26,6 +26,7 @@ #include "openvswitch/vlog.h" #include "lib/bitmap.h" #include "lib/smap.h" +#include "socket-util.h" VLOG_DEFINE_THIS_MODULE(lb); @@ -431,11 +432,62 @@ void ovn_northd_lb_vip_init(struct ovn_northd_lb_vip *lb_vip_nb, ovn_lb_get_health_check(nbrec_lb, vip_port_str, template); } +static void +ovn_lb_vip_backends_health_check_init(const struct ovn_northd_lb *lb, + const struct ovn_lb_vip *lb_vip, + struct ovn_northd_lb_vip *lb_vip_nb) +{ + struct ds key = DS_EMPTY_INITIALIZER; + + for (size_t j = 0; j < lb_vip->n_backends; j++) { + struct ovn_lb_backend *backend = &lb_vip->backends[j]; + ds_clear(&key); + ds_put_format(&key, IN6_IS_ADDR_V4MAPPED(&lb_vip->vip) + ? "%s" : "[%s]", backend->ip_str); + + const char *s = smap_get(&lb->nlb->ip_port_mappings, ds_cstr(&key)); + if (!s) { + continue; + } + + char *svc_mon_src_ip = NULL; + char *port_name = xstrdup(s); + char *p = strstr(port_name, ":"); + if (p) { + *p = 0; + p++; + struct sockaddr_storage svc_mon_src_addr; + if (!inet_parse_address(p, &svc_mon_src_addr)) { + static struct vlog_rate_limit rl = + VLOG_RATE_LIMIT_INIT(5, 1); + VLOG_WARN_RL(&rl, "Invalid svc mon src IP %s", p); + } else { + struct ds src_ip_s = DS_EMPTY_INITIALIZER; + ss_format_address_nobracks(&svc_mon_src_addr, + &src_ip_s); + svc_mon_src_ip = ds_steal_cstr(&src_ip_s); + } + } + + if (svc_mon_src_ip) { + struct ovn_northd_lb_backend *backend_nb = + &lb_vip_nb->backends_nb[j]; + backend_nb->health_check = true; + backend_nb->logical_port = xstrdup(port_name); + backend_nb->svc_mon_src_ip = svc_mon_src_ip; + } + free(port_name); + } + + ds_destroy(&key); +} + static void ovn_northd_lb_vip_destroy(struct ovn_northd_lb_vip *vip) { free(vip->backend_ips); for (size_t i = 0; i < vip->n_backends; i++) { + free(vip->backends_nb[i].logical_port); free(vip->backends_nb[i].svc_mon_src_ip); } free(vip->backends_nb); @@ -555,8 +607,7 @@ ovn_lb_get_health_check(const struct nbrec_load_balancer *nbrec_lb, } struct ovn_northd_lb * -ovn_northd_lb_create(const struct nbrec_load_balancer *nbrec_lb, - size_t n_ls_datapaths, size_t n_lr_datapaths) +ovn_northd_lb_create(const struct nbrec_load_balancer *nbrec_lb) { bool template = smap_get_bool(&nbrec_lb->options, "template", false); bool is_udp = nullable_string_is_equal(nbrec_lb->protocol, "udp"); @@ -595,9 +646,6 @@ ovn_northd_lb_create(const struct nbrec_load_balancer *nbrec_lb, } lb->affinity_timeout = affinity_timeout; - lb->nb_ls_map = bitmap_allocate(n_ls_datapaths); - lb->nb_lr_map = bitmap_allocate(n_lr_datapaths); - sset_init(&lb->ips_v4); sset_init(&lb->ips_v6); struct smap_node *node; @@ -632,6 +680,10 @@ ovn_northd_lb_create(const struct nbrec_load_balancer *nbrec_lb, xstrdup(node->value)); } n_vips++; + + if (lb_vip_nb->lb_health_check) { + ovn_lb_vip_backends_health_check_init(lb, lb_vip, lb_vip_nb); + } } /* It's possible that parsing VIPs fails. Update the lb->n_vips to the @@ -684,24 +736,6 @@ ovn_northd_lb_get_vips(const struct ovn_northd_lb *lb) return &lb->nlb->vips; } -void -ovn_northd_lb_add_lr(struct ovn_northd_lb *lb, size_t n, - struct ovn_datapath **ods) -{ - for (size_t i = 0; i < n; i++) { - bitmap_set1(lb->nb_lr_map, ods[i]->index); - } -} - -void -ovn_northd_lb_add_ls(struct ovn_northd_lb *lb, size_t n, - struct ovn_datapath **ods) -{ - for (size_t i = 0; i < n; i++) { - bitmap_set1(lb->nb_ls_map, ods[i]->index); - } -} - void ovn_northd_lb_destroy(struct ovn_northd_lb *lb) { @@ -715,20 +749,15 @@ ovn_northd_lb_destroy(struct ovn_northd_lb *lb) sset_destroy(&lb->ips_v4); sset_destroy(&lb->ips_v6); free(lb->selection_fields); - bitmap_free(lb->nb_lr_map); - bitmap_free(lb->nb_ls_map); free(lb); } /* Constructs a new 'struct ovn_lb_group' object from the Nb LB Group record - * and a hash map of all existing 'struct ovn_northd_lb' objects. Space will - * be allocated for 'max_ls_datapaths' logical switches and 'max_lr_datapaths' - * logical routers to which this LB Group is applied. Can be filled later - * with ovn_lb_group_add_ls() and ovn_lb_group_add_lr() respectively. */ + * and an array of 'struct ovn_northd_lb' objects for its associated + * load balancers. */ struct ovn_lb_group * ovn_lb_group_create(const struct nbrec_load_balancer_group *nbrec_lb_group, - const struct hmap *lbs, size_t max_ls_datapaths, - size_t max_lr_datapaths) + const struct hmap *lbs) { struct ovn_lb_group *lb_group; @@ -736,8 +765,6 @@ ovn_lb_group_create(const struct nbrec_load_balancer_group *nbrec_lb_group, lb_group->uuid = nbrec_lb_group->header_.uuid; lb_group->n_lbs = nbrec_lb_group->n_load_balancer; lb_group->lbs = xmalloc(lb_group->n_lbs * sizeof *lb_group->lbs); - lb_group->ls = xmalloc(max_ls_datapaths * sizeof *lb_group->ls); - lb_group->lr = xmalloc(max_lr_datapaths * sizeof *lb_group->lr); lb_group->lb_ips = ovn_lb_ip_set_create(); for (size_t i = 0; i < nbrec_lb_group->n_load_balancer; i++) { @@ -758,8 +785,6 @@ ovn_lb_group_destroy(struct ovn_lb_group *lb_group) ovn_lb_ip_set_destroy(lb_group->lb_ips); free(lb_group->lbs); - free(lb_group->ls); - free(lb_group->lr); free(lb_group); } @@ -943,3 +968,113 @@ ovn_lb_5tuples_destroy(struct hmap *tuples) hmap_destroy(tuples); } + +void +build_lrouter_lb_ips(struct ovn_lb_ip_set *lb_ips, + const struct ovn_northd_lb *lb) +{ + const char *ip_address; + + SSET_FOR_EACH (ip_address, &lb->ips_v4) { + sset_add(&lb_ips->ips_v4, ip_address); + if (lb->routable) { + sset_add(&lb_ips->ips_v4_routable, ip_address); + } + } + SSET_FOR_EACH (ip_address, &lb->ips_v6) { + sset_add(&lb_ips->ips_v6, ip_address); + if (lb->routable) { + sset_add(&lb_ips->ips_v6_routable, ip_address); + } + } +} + +/* lb datapaths functions */ +struct ovn_lb_datapaths * +ovn_lb_datapaths_create(const struct ovn_northd_lb *lb, size_t n_ls_datapaths, + size_t n_lr_datapaths) +{ + struct ovn_lb_datapaths *lb_dps = xzalloc(sizeof *lb_dps); + lb_dps->lb = lb; + lb_dps->nb_ls_map = bitmap_allocate(n_ls_datapaths); + lb_dps->nb_lr_map = bitmap_allocate(n_lr_datapaths); + + return lb_dps; +} + +struct ovn_lb_datapaths * +ovn_lb_datapaths_find(const struct hmap *lb_dps_map, + const struct uuid *lb_uuid) +{ + struct ovn_lb_datapaths *lb_dps; + size_t hash = uuid_hash(lb_uuid); + HMAP_FOR_EACH_WITH_HASH (lb_dps, hmap_node, hash, lb_dps_map) { + if (uuid_equals(&lb_dps->lb->nlb->header_.uuid, lb_uuid)) { + return lb_dps; + } + } + return NULL; +} + +void +ovn_lb_datapaths_destroy(struct ovn_lb_datapaths *lb_dps) +{ + bitmap_free(lb_dps->nb_lr_map); + bitmap_free(lb_dps->nb_ls_map); + free(lb_dps); +} + +void +ovn_lb_datapaths_add_lr(struct ovn_lb_datapaths *lb_dps, size_t n, + struct ovn_datapath **ods) +{ + for (size_t i = 0; i < n; i++) { + bitmap_set1(lb_dps->nb_lr_map, ods[i]->index); + } +} + +void +ovn_lb_datapaths_add_ls(struct ovn_lb_datapaths *lb_dps, size_t n, + struct ovn_datapath **ods) +{ + for (size_t i = 0; i < n; i++) { + bitmap_set1(lb_dps->nb_ls_map, ods[i]->index); + } +} + +struct ovn_lb_group_datapaths * +ovn_lb_group_datapaths_create(const struct ovn_lb_group *lb_group, + size_t max_ls_datapaths, + size_t max_lr_datapaths) +{ + struct ovn_lb_group_datapaths *lb_group_dps = + xzalloc(sizeof *lb_group_dps); + lb_group_dps->lb_group = lb_group; + lb_group_dps->ls = xmalloc(max_ls_datapaths * sizeof *lb_group_dps->ls); + lb_group_dps->lr = xmalloc(max_lr_datapaths * sizeof *lb_group_dps->lr); + + return lb_group_dps; +} + +void +ovn_lb_group_datapaths_destroy(struct ovn_lb_group_datapaths *lb_group_dps) +{ + free(lb_group_dps->ls); + free(lb_group_dps->lr); + free(lb_group_dps); +} + +struct ovn_lb_group_datapaths * +ovn_lb_group_datapaths_find(const struct hmap *lb_group_dps_map, + const struct uuid *lb_group_uuid) +{ + struct ovn_lb_group_datapaths *lb_group_dps; + size_t hash = uuid_hash(lb_group_uuid); + + HMAP_FOR_EACH_WITH_HASH (lb_group_dps, hmap_node, hash, lb_group_dps_map) { + if (uuid_equals(&lb_group_dps->lb_group->uuid, lb_group_uuid)) { + return lb_group_dps; + } + } + return NULL; +} diff --git a/lib/lb.h b/lib/lb.h index 23d8fc9e9b..0339050cba 100644 --- a/lib/lb.h +++ b/lib/lb.h @@ -59,7 +59,6 @@ struct ovn_northd_lb { struct hmap_node hmap_node; const struct nbrec_load_balancer *nlb; /* May be NULL. */ - const struct sbrec_load_balancer *slb; /* May be NULL. */ const char *proto; char *selection_fields; struct ovn_lb_vip *vips; @@ -78,14 +77,6 @@ struct ovn_northd_lb { struct sset ips_v4; struct sset ips_v6; - - size_t n_nb_ls; - unsigned long *nb_ls_map; - - size_t n_nb_lr; - unsigned long *nb_lr_map; - - struct ovn_dp_group *dpg; }; struct ovn_lb_vip { @@ -129,23 +120,19 @@ struct ovn_northd_lb_vip { }; struct ovn_northd_lb_backend { - struct ovn_port *op; /* Logical port to which the ip belong to. */ bool health_check; + char *logical_port; /* Logical port to which the ip belong to. */ char *svc_mon_src_ip; /* Source IP to use for monitoring. */ - const struct sbrec_service_monitor *sbrec_monitor; }; -struct ovn_northd_lb *ovn_northd_lb_create(const struct nbrec_load_balancer *, - size_t n_ls_datapaths, - size_t n_lr_datapaths); +struct ovn_northd_lb *ovn_northd_lb_create(const struct nbrec_load_balancer *); struct ovn_northd_lb *ovn_northd_lb_find(const struct hmap *, const struct uuid *); const struct smap *ovn_northd_lb_get_vips(const struct ovn_northd_lb *); void ovn_northd_lb_destroy(struct ovn_northd_lb *); -void ovn_northd_lb_add_lr(struct ovn_northd_lb *lb, size_t n, - struct ovn_datapath **ods); -void ovn_northd_lb_add_ls(struct ovn_northd_lb *lb, size_t n, - struct ovn_datapath **ods); + +void build_lrouter_lb_ips(struct ovn_lb_ip_set *, + const struct ovn_northd_lb *); struct ovn_lb_group { struct hmap_node hmap_node; @@ -153,35 +140,70 @@ struct ovn_lb_group { size_t n_lbs; struct ovn_northd_lb **lbs; struct ovn_lb_ip_set *lb_ips; +}; + +struct ovn_lb_group *ovn_lb_group_create( + const struct nbrec_load_balancer_group *, + const struct hmap *lbs); +void ovn_lb_group_destroy(struct ovn_lb_group *lb_group); +struct ovn_lb_group *ovn_lb_group_find(const struct hmap *lb_groups, + const struct uuid *); + +struct ovn_lb_datapaths { + struct hmap_node hmap_node; - /* Datapaths to which this LB group is applied. */ + const struct ovn_northd_lb *lb; + size_t n_nb_ls; + unsigned long *nb_ls_map; + + size_t n_nb_lr; + unsigned long *nb_lr_map; +}; + +struct ovn_lb_datapaths *ovn_lb_datapaths_create(const struct ovn_northd_lb *, + size_t n_ls_datapaths, + size_t n_lr_datapaths); +struct ovn_lb_datapaths *ovn_lb_datapaths_find(const struct hmap *, + const struct uuid *); +void ovn_lb_datapaths_destroy(struct ovn_lb_datapaths *); +void ovn_lb_datapaths_add_lr(struct ovn_lb_datapaths *, size_t n, + struct ovn_datapath **); +void ovn_lb_datapaths_add_ls(struct ovn_lb_datapaths *, size_t n, + struct ovn_datapath **); + +struct ovn_lb_group_datapaths { + struct hmap_node hmap_node; + + const struct ovn_lb_group *lb_group; + + /* Datapaths to which 'lb_group' is applied. */ size_t n_ls; struct ovn_datapath **ls; size_t n_lr; struct ovn_datapath **lr; }; -struct ovn_lb_group *ovn_lb_group_create( - const struct nbrec_load_balancer_group *, - const struct hmap *lbs, - size_t max_ls_datapaths, +struct ovn_lb_group_datapaths *ovn_lb_group_datapaths_create( + const struct ovn_lb_group *, size_t max_ls_datapaths, size_t max_lr_datapaths); -void ovn_lb_group_destroy(struct ovn_lb_group *lb_group); -struct ovn_lb_group *ovn_lb_group_find(const struct hmap *lb_groups, - const struct uuid *); + +void ovn_lb_group_datapaths_destroy(struct ovn_lb_group_datapaths *); +struct ovn_lb_group_datapaths *ovn_lb_group_datapaths_find( + const struct hmap *lb_group_dps, const struct uuid *); static inline void -ovn_lb_group_add_ls(struct ovn_lb_group *lb_group, size_t n, - struct ovn_datapath **ods) +ovn_lb_group_datapaths_add_ls(struct ovn_lb_group_datapaths *lbg_dps, size_t n, + struct ovn_datapath **ods) { - memcpy(&lb_group->ls[lb_group->n_ls], ods, n * sizeof *ods); - lb_group->n_ls += n; + memcpy(&lbg_dps->ls[lbg_dps->n_ls], ods, n * sizeof *ods); + lbg_dps->n_ls += n; } static inline void -ovn_lb_group_add_lr(struct ovn_lb_group *lb_group, struct ovn_datapath *lr) +ovn_lb_group_datapaths_add_lr(struct ovn_lb_group_datapaths *lbg_dps, + struct ovn_datapath *lr) { - lb_group->lr[lb_group->n_lr++] = lr; + lbg_dps->lr[lbg_dps->n_lr++] = lr; } struct ovn_controller_lb { diff --git a/northd/automake.mk b/northd/automake.mk index b17f1fdb54..11d44212f8 100644 --- a/northd/automake.mk +++ b/northd/automake.mk @@ -18,6 +18,8 @@ northd_ovn_northd_SOURCES = \ northd/en-sync-sb.h \ northd/en-sync-from-sb.c \ northd/en-sync-from-sb.h \ + northd/en-lb-data.c \ + northd/en-lb-data.h \ northd/inc-proc-northd.c \ northd/inc-proc-northd.h \ northd/ipam.c \ diff --git a/northd/en-lb-data.c b/northd/en-lb-data.c new file mode 100644 index 0000000000..04d8d3e5d7 --- /dev/null +++ b/northd/en-lb-data.c @@ -0,0 +1,125 @@ +/* + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at: + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#include + +#include +#include +#include + +/* OVS includes */ +#include "openvswitch/util.h" +#include "openvswitch/vlog.h" + +/* OVN includes */ +#include "en-lb-data.h" +#include "lib/inc-proc-eng.h" +#include "lib/lb.h" +#include "lib/ovn-nb-idl.h" +#include "lib/ovn-sb-idl.h" +#include "lib/ovn-util.h" +#include "northd.h" + +VLOG_DEFINE_THIS_MODULE(en_lb_data); + +static void lb_data_init(struct ed_type_lb_data *); +static void lb_data_destroy(struct ed_type_lb_data *); +static void build_lbs(const struct nbrec_load_balancer_table *, + const struct nbrec_load_balancer_group_table *, + struct hmap *lbs, struct hmap *lb_groups); + +void * +en_lb_data_init(struct engine_node *node OVS_UNUSED, + struct engine_arg *arg OVS_UNUSED) +{ + struct ed_type_lb_data *data = xzalloc(sizeof *data); + lb_data_init(data); + return data; +} + +void +en_lb_data_run(struct engine_node *node, void *data) +{ + struct ed_type_lb_data *lb_data = (struct ed_type_lb_data *) data; + lb_data_destroy(lb_data); + lb_data_init(lb_data); + + const struct nbrec_load_balancer_table *nb_lb_table = + EN_OVSDB_GET(engine_get_input("NB_load_balancer", node)); + const struct nbrec_load_balancer_group_table *nb_lbg_table = + EN_OVSDB_GET(engine_get_input("NB_load_balancer_group", node)); + + build_lbs(nb_lb_table, nb_lbg_table, &lb_data->lbs, &lb_data->lb_groups); + engine_set_node_state(node, EN_UPDATED); +} + +void +en_lb_data_cleanup(void *data) +{ + struct ed_type_lb_data *lb_data = (struct ed_type_lb_data *) data; + lb_data_destroy(lb_data); +} + +/* static functions. */ +static void +lb_data_init(struct ed_type_lb_data *lb_data) +{ + hmap_init(&lb_data->lbs); + hmap_init(&lb_data->lb_groups); +} + +static void +lb_data_destroy(struct ed_type_lb_data *lb_data) +{ + struct ovn_northd_lb *lb; + HMAP_FOR_EACH_POP (lb, hmap_node, &lb_data->lbs) { + ovn_northd_lb_destroy(lb); + } + hmap_destroy(&lb_data->lbs); + + struct ovn_lb_group *lb_group; + HMAP_FOR_EACH_POP (lb_group, hmap_node, &lb_data->lb_groups) { + ovn_lb_group_destroy(lb_group); + } + hmap_destroy(&lb_data->lb_groups); +} + +static void +build_lbs(const struct nbrec_load_balancer_table *nbrec_load_balancer_table, + const struct nbrec_load_balancer_group_table *nbrec_lb_group_table, + struct hmap *lbs, struct hmap *lb_groups) +{ + struct ovn_lb_group *lb_group; + struct ovn_northd_lb *lb_nb; + + const struct nbrec_load_balancer *nbrec_lb; + NBREC_LOAD_BALANCER_TABLE_FOR_EACH (nbrec_lb, nbrec_load_balancer_table) { + lb_nb = ovn_northd_lb_create(nbrec_lb); + hmap_insert(lbs, &lb_nb->hmap_node, + uuid_hash(&nbrec_lb->header_.uuid)); + } + + const struct nbrec_load_balancer_group *nbrec_lb_group; + NBREC_LOAD_BALANCER_GROUP_TABLE_FOR_EACH (nbrec_lb_group, + nbrec_lb_group_table) { + lb_group = ovn_lb_group_create(nbrec_lb_group, lbs); + + for (size_t i = 0; i < lb_group->n_lbs; i++) { + build_lrouter_lb_ips(lb_group->lb_ips, lb_group->lbs[i]); + } + + hmap_insert(lb_groups, &lb_group->hmap_node, + uuid_hash(&lb_group->uuid)); + } +} diff --git a/northd/en-lb-data.h b/northd/en-lb-data.h new file mode 100644 index 0000000000..96fb8c1f8d --- /dev/null +++ b/northd/en-lb-data.h @@ -0,0 +1,23 @@ +#ifndef EN_NORTHD_LB_DATA_H +#define EN_NORTHD_LB_DATA_H 1 + +#include + +#include "openvswitch/hmap.h" + +#include "lib/inc-proc-eng.h" + +/* struct which maintains the data of the engine node lb_data. */ +struct ed_type_lb_data { + /* hmap of load balancers. hmap node is 'struct ovn_northd_lb *' */ + struct hmap lbs; + + /* hmap of load balancer groups. hmap node is 'struct ovn_lb_group *' */ + struct hmap lb_groups; +}; + +void *en_lb_data_init(struct engine_node *, struct engine_arg *); +void en_lb_data_run(struct engine_node *, void *data); +void en_lb_data_cleanup(void *data); + +#endif /* end of EN_NORTHD_LB_DATA_H */ diff --git a/northd/en-lflow.c b/northd/en-lflow.c index 28ab1c67fb..db1bcbccd6 100644 --- a/northd/en-lflow.c +++ b/northd/en-lflow.c @@ -57,7 +57,8 @@ lflow_get_input_data(struct engine_node *node, lflow_input->lr_ports = &northd_data->lr_ports; lflow_input->port_groups = &northd_data->port_groups; lflow_input->meter_groups = &northd_data->meter_groups; - lflow_input->lbs = &northd_data->lbs; + lflow_input->lb_datapaths_map = &northd_data->lb_datapaths_map; + lflow_input->svc_monitor_map = &northd_data->svc_monitor_map; lflow_input->features = &northd_data->features; lflow_input->ovn_internal_version_changed = northd_data->ovn_internal_version_changed; diff --git a/northd/en-northd.c b/northd/en-northd.c index f9f2d04452..a0dae65752 100644 --- a/northd/en-northd.c +++ b/northd/en-northd.c @@ -20,6 +20,7 @@ #include "coverage.h" #include "en-northd.h" +#include "en-lb-data.h" #include "lib/inc-proc-eng.h" #include "lib/ovn-nb-idl.h" #include "openvswitch/list.h" /* TODO This is needed for ovn-parallel-hmap.h. @@ -70,10 +71,6 @@ northd_get_input_data(struct engine_node *node, EN_OVSDB_GET(engine_get_input("NB_logical_switch", node)); input_data->nbrec_logical_router_table = EN_OVSDB_GET(engine_get_input("NB_logical_router", node)); - input_data->nbrec_load_balancer_table = - EN_OVSDB_GET(engine_get_input("NB_load_balancer", node)); - input_data->nbrec_load_balancer_group_table = - EN_OVSDB_GET(engine_get_input("NB_load_balancer_group", node)); input_data->nbrec_port_group_table = EN_OVSDB_GET(engine_get_input("NB_port_group", node)); input_data->nbrec_meter_table = @@ -117,6 +114,11 @@ northd_get_input_data(struct engine_node *node, EN_OVSDB_GET(engine_get_input("SB_chassis_template_var", node)); input_data->sbrec_mirror_table = EN_OVSDB_GET(engine_get_input("SB_mirror", node)); + + struct ed_type_lb_data *lb_data = + engine_get_input_data("lb_data", node); + input_data->lbs = &lb_data->lbs; + input_data->lb_groups = &lb_data->lb_groups; } void @@ -130,6 +132,7 @@ en_northd_run(struct engine_node *node, void *data) northd_init(data); northd_get_input_data(node, &input_data); + COVERAGE_INC(northd_run); stopwatch_start(OVNNB_DB_RUN_STOPWATCH_NAME, time_msec()); ovnnb_db_run(&input_data, data, eng_ctx->ovnnb_idl_txn, diff --git a/northd/en-sync-sb.c b/northd/en-sync-sb.c index dbe3c63752..9832fce30a 100644 --- a/northd/en-sync-sb.c +++ b/northd/en-sync-sb.c @@ -230,7 +230,7 @@ en_sync_to_sb_lb_run(struct engine_node *node, void *data OVS_UNUSED) struct northd_data *northd_data = engine_get_input_data("northd", node); sync_lbs(eng_ctx->ovnsb_idl_txn, sb_load_balancer_table, - &northd_data->ls_datapaths, &northd_data->lbs); + &northd_data->ls_datapaths, &northd_data->lb_datapaths_map); engine_set_node_state(node, EN_UPDATED); } diff --git a/northd/inc-proc-northd.c b/northd/inc-proc-northd.c index c0874d5294..0a99e87bc2 100644 --- a/northd/inc-proc-northd.c +++ b/northd/inc-proc-northd.c @@ -30,6 +30,7 @@ #include "openvswitch/poll-loop.h" #include "openvswitch/vlog.h" #include "inc-proc-northd.h" +#include "en-lb-data.h" #include "en-northd.h" #include "en-lflow.h" #include "en-northd-output.h" @@ -140,6 +141,7 @@ static ENGINE_NODE(sync_to_sb_addr_set, "sync_to_sb_addr_set"); static ENGINE_NODE(fdb_aging, "fdb_aging"); static ENGINE_NODE(fdb_aging_waker, "fdb_aging_waker"); static ENGINE_NODE(sync_to_sb_lb, "sync_to_sb_lb"); +static ENGINE_NODE(lb_data, "lb_data"); void inc_proc_northd_init(struct ovsdb_idl_loop *nb, struct ovsdb_idl_loop *sb) @@ -147,8 +149,6 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, /* Define relationships between nodes where first argument is dependent * on the second argument */ engine_add_input(&en_northd, &en_nb_port_group, NULL); - engine_add_input(&en_northd, &en_nb_load_balancer, NULL); - engine_add_input(&en_northd, &en_nb_load_balancer_group, NULL); engine_add_input(&en_northd, &en_nb_acl, NULL); engine_add_input(&en_northd, &en_nb_logical_router, NULL); engine_add_input(&en_northd, &en_nb_mirror, NULL); @@ -178,6 +178,10 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, engine_add_input(&en_northd, &en_nb_logical_switch, northd_nb_logical_switch_handler); + engine_add_input(&en_lb_data, &en_nb_load_balancer, NULL); + engine_add_input(&en_lb_data, &en_nb_load_balancer_group, NULL); + engine_add_input(&en_northd, &en_lb_data, NULL); + engine_add_input(&en_mac_binding_aging, &en_nb_nb_global, NULL); engine_add_input(&en_mac_binding_aging, &en_sb_mac_binding, NULL); engine_add_input(&en_mac_binding_aging, &en_northd, NULL); diff --git a/northd/northd.c b/northd/northd.c index a3bce551bc..78b82950e4 100644 --- a/northd/northd.c +++ b/northd/northd.c @@ -3862,16 +3862,15 @@ struct service_monitor_info { static struct service_monitor_info * -create_or_get_service_mon(struct ovsdb_idl_txn *ovnsb_txn, - struct hmap *monitor_map, - const char *ip, const char *logical_port, - uint16_t service_port, const char *protocol) +get_service_mon(const struct hmap *monitor_map, + const char *ip, const char *logical_port, + uint16_t service_port, const char *protocol) { uint32_t hash = service_port; hash = hash_string(ip, hash); hash = hash_string(logical_port, hash); - struct service_monitor_info *mon_info; + struct service_monitor_info *mon_info; HMAP_FOR_EACH_WITH_HASH (mon_info, hmap_node, hash, monitor_map) { if (mon_info->sbrec_mon->port == service_port && !strcmp(mon_info->sbrec_mon->ip, ip) && @@ -3881,6 +3880,28 @@ create_or_get_service_mon(struct ovsdb_idl_txn *ovnsb_txn, } } + return NULL; +} + +static struct service_monitor_info * +create_or_get_service_mon(struct ovsdb_idl_txn *ovnsb_txn, + struct hmap *monitor_map, + const char *ip, const char *logical_port, + uint16_t service_port, const char *protocol) +{ + struct service_monitor_info *mon_info = + get_service_mon(monitor_map, ip, logical_port, service_port, + protocol); + + if (mon_info) { + return mon_info; + } + + /* get_service_mon() also calculates the hash the same way. */ + uint32_t hash = service_port; + hash = hash_string(ip, hash); + hash = hash_string(logical_port, hash); + struct sbrec_service_monitor *sbrec_mon = sbrec_service_monitor_insert(ovnsb_txn); sbrec_service_monitor_set_ip(sbrec_mon, ip); @@ -3894,7 +3915,8 @@ create_or_get_service_mon(struct ovsdb_idl_txn *ovnsb_txn, } static void -ovn_lb_svc_create(struct ovsdb_idl_txn *ovnsb_txn, struct ovn_northd_lb *lb, +ovn_lb_svc_create(struct ovsdb_idl_txn *ovnsb_txn, + const struct ovn_northd_lb *lb, struct hmap *monitor_map, struct hmap *ls_ports, struct sset *svc_monitor_lsps) { @@ -3911,58 +3933,27 @@ ovn_lb_svc_create(struct ovsdb_idl_txn *ovnsb_txn, struct ovn_northd_lb *lb, struct ovn_northd_lb_backend *backend_nb = &lb_vip_nb->backends_nb[j]; - struct ovn_port *op = NULL; - char *svc_mon_src_ip = NULL; - - struct ds key = DS_EMPTY_INITIALIZER; - ds_put_format(&key, - IN6_IS_ADDR_V4MAPPED(&lb_vip->vip) - ? "%s" : "[%s]", backend->ip_str); - - const char *s = smap_get(&lb->nlb->ip_port_mappings, - ds_cstr(&key)); - if (s) { - char *port_name = xstrdup(s); - char *p = strstr(port_name, ":"); - if (p) { - *p = 0; - p++; - sset_add(svc_monitor_lsps, port_name); - op = ovn_port_find(ls_ports, port_name); - struct sockaddr_storage svc_mon_src_addr; - if (!inet_parse_address(p, &svc_mon_src_addr)) { - static struct vlog_rate_limit rl = - VLOG_RATE_LIMIT_INIT(5, 1); - VLOG_WARN_RL(&rl, "Invalid svc mon src IP %s", p); - } else { - struct ds src_ip_s = DS_EMPTY_INITIALIZER; - ss_format_address_nobracks(&svc_mon_src_addr, - &src_ip_s); - svc_mon_src_ip = ds_steal_cstr(&src_ip_s); - } - } - free(port_name); + if (!backend_nb->health_check) { + continue; } - ds_destroy(&key); - if (!lb_vip_nb->lb_health_check || !op || !svc_mon_src_ip || - !lsp_is_enabled(op->nbsp)) { - free(svc_mon_src_ip); + sset_add(svc_monitor_lsps, backend_nb->logical_port); + struct ovn_port *op = ovn_port_find(ls_ports, + backend_nb->logical_port); + + if (!op || !lsp_is_enabled(op->nbsp)) { continue; } - backend_nb->op = op; - backend_nb->svc_mon_src_ip = svc_mon_src_ip; - const char *protocol = lb->nlb->protocol; if (!protocol || !protocol[0]) { protocol = "tcp"; } - backend_nb->health_check = true; + struct service_monitor_info *mon_info = create_or_get_service_mon(ovnsb_txn, monitor_map, backend->ip_str, - backend_nb->op->nbsp->name, + backend_nb->logical_port, backend->port, protocol); ovs_assert(mon_info); @@ -3991,18 +3982,20 @@ ovn_lb_svc_create(struct ovsdb_idl_txn *ovnsb_txn, struct ovn_northd_lb *lb, "offline"); } - backend_nb->sbrec_monitor = mon_info->sbrec_mon; mon_info->required = true; } } } static bool -build_lb_vip_actions(struct ovn_lb_vip *lb_vip, - struct ovn_northd_lb_vip *lb_vip_nb, +build_lb_vip_actions(const struct ovn_northd_lb *lb, + const struct ovn_lb_vip *lb_vip, + const struct ovn_northd_lb_vip *lb_vip_nb, struct ds *action, char *selection_fields, - struct ds *skip_snat_action, struct ds *force_snat_action, - bool ls_dp, const struct chassis_features *features) + struct ds *skip_snat_action, + struct ds *force_snat_action, + bool ls_dp, const struct chassis_features *features, + const struct hmap *svc_monitor_map) { const char *ct_lb_action = features->ct_no_masked_label ? "ct_lb_mark" : "ct_lb"; @@ -4017,10 +4010,27 @@ build_lb_vip_actions(struct ovn_lb_vip *lb_vip, struct ovn_lb_backend *backend = &lb_vip->backends[i]; struct ovn_northd_lb_backend *backend_nb = &lb_vip_nb->backends_nb[i]; - if (!backend_nb->health_check || - (backend_nb->health_check && backend_nb->sbrec_monitor && - backend_nb->sbrec_monitor->status && - strcmp(backend_nb->sbrec_monitor->status, "online"))) { + + if (!backend_nb->health_check) { + continue; + } + + const char *protocol = lb->nlb->protocol; + if (!protocol || !protocol[0]) { + protocol = "tcp"; + } + + struct service_monitor_info *mon_info = get_service_mon( + svc_monitor_map, backend->ip_str, backend_nb->logical_port, + backend->port, protocol); + + if (!mon_info) { + continue; + } + + ovs_assert(mon_info->sbrec_mon); + if (mon_info->sbrec_mon->status && + strcmp(mon_info->sbrec_mon->status, "online")) { continue; } @@ -4070,59 +4080,32 @@ build_lb_vip_actions(struct ovn_lb_vip *lb_vip, } static void -build_lrouter_lb_ips(struct ovn_lb_ip_set *lb_ips, - const struct ovn_northd_lb *lb) -{ - const char *ip_address; - - SSET_FOR_EACH (ip_address, &lb->ips_v4) { - sset_add(&lb_ips->ips_v4, ip_address); - if (lb->routable) { - sset_add(&lb_ips->ips_v4_routable, ip_address); - } - } - SSET_FOR_EACH (ip_address, &lb->ips_v6) { - sset_add(&lb_ips->ips_v6, ip_address); - if (lb->routable) { - sset_add(&lb_ips->ips_v6_routable, ip_address); - } - } -} - -static void -build_lbs(const struct nbrec_load_balancer_table *nbrec_load_balancer_table, - const struct nbrec_load_balancer_group_table *nbrec_lb_group_table, - struct ovn_datapaths *ls_datapaths, - struct ovn_datapaths *lr_datapaths, - struct hmap *lbs, struct hmap *lb_groups) +build_lb_datapaths(const struct hmap *lbs, const struct hmap *lb_groups, + struct ovn_datapaths *ls_datapaths, + struct ovn_datapaths *lr_datapaths, + struct hmap *lb_datapaths_map, + struct hmap *lb_group_datapaths_map) { const struct nbrec_load_balancer_group *nbrec_lb_group; - struct ovn_lb_group *lb_group; - struct ovn_northd_lb *lb; + struct ovn_lb_group_datapaths *lb_group_dps; + const struct ovn_lb_group *lb_group; + struct ovn_lb_datapaths *lb_dps; + const struct ovn_northd_lb *lb; - hmap_init(lbs); - hmap_init(lb_groups); + hmap_init(lb_datapaths_map); + hmap_init(lb_group_datapaths_map); - const struct nbrec_load_balancer *nbrec_lb; - NBREC_LOAD_BALANCER_TABLE_FOR_EACH (nbrec_lb, nbrec_load_balancer_table) { - struct ovn_northd_lb *lb_nb = ovn_northd_lb_create(nbrec_lb, - ods_size(ls_datapaths), - ods_size(lr_datapaths)); - hmap_insert(lbs, &lb_nb->hmap_node, - uuid_hash(&nbrec_lb->header_.uuid)); + HMAP_FOR_EACH (lb, hmap_node, lbs) { + lb_dps = ovn_lb_datapaths_create(lb, ods_size(ls_datapaths), + ods_size(lr_datapaths)); + hmap_insert(lb_datapaths_map, &lb_dps->hmap_node, + uuid_hash(&lb->nlb->header_.uuid)); } - NBREC_LOAD_BALANCER_GROUP_TABLE_FOR_EACH (nbrec_lb_group, - nbrec_lb_group_table) { - lb_group = ovn_lb_group_create(nbrec_lb_group, lbs, - ods_size(ls_datapaths), - ods_size(lr_datapaths)); - - for (size_t i = 0; i < lb_group->n_lbs; i++) { - build_lrouter_lb_ips(lb_group->lb_ips, lb_group->lbs[i]); - } - - hmap_insert(lb_groups, &lb_group->hmap_node, + HMAP_FOR_EACH (lb_group, hmap_node, lb_groups) { + lb_group_dps = ovn_lb_group_datapaths_create( + lb_group, ods_size(ls_datapaths), ods_size(lr_datapaths)); + hmap_insert(lb_group_datapaths_map, &lb_group_dps->hmap_node, uuid_hash(&lb_group->uuid)); } @@ -4135,22 +4118,19 @@ build_lbs(const struct nbrec_load_balancer_table *nbrec_load_balancer_table, for (size_t i = 0; i < od->nbs->n_load_balancer; i++) { const struct uuid *lb_uuid = &od->nbs->load_balancer[i]->header_.uuid; - lb = ovn_northd_lb_find(lbs, lb_uuid); - ovn_northd_lb_add_ls(lb, 1, &od); + lb_dps = ovn_lb_datapaths_find(lb_datapaths_map, lb_uuid); + ovs_assert(lb_dps); + ovn_lb_datapaths_add_ls(lb_dps, 1, &od); } for (size_t i = 0; i < od->nbs->n_load_balancer_group; i++) { nbrec_lb_group = od->nbs->load_balancer_group[i]; - lb_group = ovn_lb_group_find(lb_groups, - &nbrec_lb_group->header_.uuid); - ovn_lb_group_add_ls(lb_group, 1, &od); - } - } - - HMAP_FOR_EACH (lb_group, hmap_node, lb_groups) { - for (size_t j = 0; j < lb_group->n_lbs; j++) { - ovn_northd_lb_add_ls(lb_group->lbs[j], lb_group->n_ls, - lb_group->ls); + const struct uuid *lb_group_uuid = &nbrec_lb_group->header_.uuid; + lb_group_dps = + ovn_lb_group_datapaths_find(lb_group_datapaths_map, + lb_group_uuid); + ovs_assert(lb_group_dps); + ovn_lb_group_datapaths_add_ls(lb_group_dps, 1, &od); } } @@ -4172,15 +4152,21 @@ build_lbs(const struct nbrec_load_balancer_table *nbrec_load_balancer_table, size_t idx = (i + largest_group) % od->nbr->n_load_balancer_group; nbrec_lb_group = od->nbr->load_balancer_group[idx]; - lb_group = ovn_lb_group_find(lb_groups, - &nbrec_lb_group->header_.uuid); - ovn_lb_group_add_lr(lb_group, od); + const struct uuid *lb_group_uuid = &nbrec_lb_group->header_.uuid; + + lb_group_dps = + ovn_lb_group_datapaths_find(lb_group_datapaths_map, + lb_group_uuid); + ovs_assert(lb_group_dps); + ovn_lb_group_datapaths_add_lr(lb_group_dps, od); if (!od->lb_ips) { - od->lb_ips = ovn_lb_ip_set_clone(lb_group->lb_ips); + od->lb_ips = + ovn_lb_ip_set_clone(lb_group_dps->lb_group->lb_ips); } else { - for (size_t j = 0; j < lb_group->n_lbs; j++) { - build_lrouter_lb_ips(od->lb_ips, lb_group->lbs[j]); + for (size_t j = 0; j < lb_group_dps->lb_group->n_lbs; j++) { + build_lrouter_lb_ips(od->lb_ips, + lb_group_dps->lb_group->lbs[j]); } } } @@ -4192,16 +4178,23 @@ build_lbs(const struct nbrec_load_balancer_table *nbrec_load_balancer_table, for (size_t i = 0; i < od->nbr->n_load_balancer; i++) { const struct uuid *lb_uuid = &od->nbr->load_balancer[i]->header_.uuid; - lb = ovn_northd_lb_find(lbs, lb_uuid); - ovn_northd_lb_add_lr(lb, 1, &od); - build_lrouter_lb_ips(od->lb_ips, lb); + lb_dps = ovn_lb_datapaths_find(lb_datapaths_map, lb_uuid); + ovs_assert(lb_dps); + ovn_lb_datapaths_add_lr(lb_dps, 1, &od); + build_lrouter_lb_ips(od->lb_ips, lb_dps->lb); } } - HMAP_FOR_EACH (lb_group, hmap_node, lb_groups) { - for (size_t j = 0; j < lb_group->n_lbs; j++) { - ovn_northd_lb_add_lr(lb_group->lbs[j], lb_group->n_lr, - lb_group->lr); + HMAP_FOR_EACH (lb_group_dps, hmap_node, lb_group_datapaths_map) { + for (size_t j = 0; j < lb_group_dps->lb_group->n_lbs; j++) { + const struct uuid *lb_uuid = + &lb_group_dps->lb_group->lbs[j]->nlb->header_.uuid; + lb_dps = ovn_lb_datapaths_find(lb_datapaths_map, lb_uuid); + ovs_assert(lb_dps); + ovn_lb_datapaths_add_ls(lb_dps, lb_group_dps->n_ls, + lb_group_dps->ls); + ovn_lb_datapaths_add_lr(lb_dps, lb_group_dps->n_lr, + lb_group_dps->lr); } } } @@ -4210,10 +4203,10 @@ static void build_lb_svcs( struct ovsdb_idl_txn *ovnsb_txn, const struct sbrec_service_monitor_table *sbrec_service_monitor_table, - struct hmap *ls_ports, struct hmap *lbs, struct sset *svc_monitor_lsps) + struct hmap *ls_ports, struct hmap *lb_dps_map, + struct sset *svc_monitor_lsps, + struct hmap *svc_monitor_map) { - struct hmap monitor_map = HMAP_INITIALIZER(&monitor_map); - const struct sbrec_service_monitor *sbrec_mon; SBREC_SERVICE_MONITOR_TABLE_FOR_EACH (sbrec_mon, sbrec_service_monitor_table) { @@ -4223,24 +4216,23 @@ build_lb_svcs( struct service_monitor_info *mon_info = xzalloc(sizeof *mon_info); mon_info->sbrec_mon = sbrec_mon; mon_info->required = false; - hmap_insert(&monitor_map, &mon_info->hmap_node, hash); + hmap_insert(svc_monitor_map, &mon_info->hmap_node, hash); } - struct ovn_northd_lb *lb; - HMAP_FOR_EACH (lb, hmap_node, lbs) { - ovn_lb_svc_create(ovnsb_txn, lb, &monitor_map, ls_ports, + struct ovn_lb_datapaths *lb_dps; + HMAP_FOR_EACH (lb_dps, hmap_node, lb_dps_map) { + ovn_lb_svc_create(ovnsb_txn, lb_dps->lb, svc_monitor_map, ls_ports, svc_monitor_lsps); } struct service_monitor_info *mon_info; - HMAP_FOR_EACH_POP (mon_info, hmap_node, &monitor_map) { + HMAP_FOR_EACH_SAFE (mon_info, hmap_node, svc_monitor_map) { if (!mon_info->required) { sbrec_service_monitor_delete(mon_info->sbrec_mon); + hmap_remove(svc_monitor_map, &mon_info->hmap_node); + free(mon_info); } - - free(mon_info); } - hmap_destroy(&monitor_map); } static bool lrouter_port_ipv4_reachable(const struct ovn_port *op, @@ -4325,7 +4317,8 @@ build_lrouter_lbs_check(const struct ovn_datapaths *lr_datapaths) static void build_lrouter_lbs_reachable_ips(struct ovn_datapaths *lr_datapaths, - struct hmap *lbs, struct hmap *lb_groups) + struct hmap *lb_dps_map, + struct hmap *lb_group_dps_map) { struct ovn_datapath *od; @@ -4335,21 +4328,25 @@ build_lrouter_lbs_reachable_ips(struct ovn_datapaths *lr_datapaths, } for (size_t i = 0; i < od->nbr->n_load_balancer; i++) { - struct ovn_northd_lb *lb = - ovn_northd_lb_find(lbs, - &od->nbr->load_balancer[i]->header_.uuid); - build_lrouter_lb_reachable_ips(od, lb); + struct ovn_lb_datapaths *lb_dps = + ovn_lb_datapaths_find(lb_dps_map, + &od->nbr->load_balancer[i]->header_.uuid); + ovs_assert(lb_dps); + build_lrouter_lb_reachable_ips(od, lb_dps->lb); } for (size_t i = 0; i < od->nbr->n_load_balancer_group; i++) { const struct nbrec_load_balancer_group *nbrec_lb_group = od->nbr->load_balancer_group[i]; - struct ovn_lb_group *lb_group; - - lb_group = ovn_lb_group_find(lb_groups, - &nbrec_lb_group->header_.uuid); - for (size_t j = 0; j < lb_group->n_lbs; j++) { - build_lrouter_lb_reachable_ips(od, lb_group->lbs[j]); + struct ovn_lb_group_datapaths *lb_group_dps; + + lb_group_dps = + ovn_lb_group_datapaths_find(lb_group_dps_map, + &nbrec_lb_group->header_.uuid); + ovs_assert(lb_group_dps); + for (size_t j = 0; j < lb_group_dps->lb_group->n_lbs; j++) { + build_lrouter_lb_reachable_ips(od, + lb_group_dps->lb_group->lbs[j]); } } } @@ -4357,45 +4354,50 @@ build_lrouter_lbs_reachable_ips(struct ovn_datapaths *lr_datapaths, static void build_lswitch_lbs_from_lrouter(struct ovn_datapaths *lr_datapaths, - struct hmap *lbs, struct hmap *lb_groups) + struct hmap *lb_dps_map, + struct hmap *lb_group_dps_map) { if (!install_ls_lb_from_router) { return; } - struct ovn_northd_lb *lb; + struct ovn_lb_datapaths *lb_dps; size_t index; - HMAP_FOR_EACH (lb, hmap_node, lbs) { - BITMAP_FOR_EACH_1 (index, ods_size(lr_datapaths), lb->nb_lr_map) { + HMAP_FOR_EACH (lb_dps, hmap_node, lb_dps_map) { + BITMAP_FOR_EACH_1 (index, ods_size(lr_datapaths), lb_dps->nb_lr_map) { struct ovn_datapath *od = lr_datapaths->array[index]; - ovn_northd_lb_add_ls(lb, od->n_ls_peers, od->ls_peers); - } - } - - struct ovn_lb_group *lb_group; - HMAP_FOR_EACH (lb_group, hmap_node, lb_groups) { - for (size_t i = 0; i < lb_group->n_lr; i++) { - struct ovn_datapath *od = lb_group->lr[i]; - ovn_lb_group_add_ls(lb_group, od->n_ls_peers, od->ls_peers); - for (size_t j = 0; j < lb_group->n_lbs; j++) { - ovn_northd_lb_add_ls(lb_group->lbs[j], od->n_ls_peers, - od->ls_peers); + ovn_lb_datapaths_add_ls(lb_dps, od->n_ls_peers, od->ls_peers); + } + } + + struct ovn_lb_group_datapaths *lb_group_dps; + HMAP_FOR_EACH (lb_group_dps, hmap_node, lb_group_dps_map) { + for (size_t i = 0; i < lb_group_dps->n_lr; i++) { + struct ovn_datapath *od = lb_group_dps->lr[i]; + ovn_lb_group_datapaths_add_ls(lb_group_dps, od->n_ls_peers, + od->ls_peers); + for (size_t j = 0; j < lb_group_dps->lb_group->n_lbs; j++) { + const struct uuid *lb_uuid = + &lb_group_dps->lb_group->lbs[j]->nlb->header_.uuid; + lb_dps = ovn_lb_datapaths_find(lb_dps_map, lb_uuid); + ovs_assert(lb_dps); + ovn_lb_datapaths_add_ls(lb_dps, od->n_ls_peers, od->ls_peers); } } } } static void -build_lb_count_dps(struct hmap *lbs, +build_lb_count_dps(struct hmap *lb_dps_map, size_t n_ls_datapaths, size_t n_lr_datapaths) { - struct ovn_northd_lb *lb; + struct ovn_lb_datapaths *lb_dps; - HMAP_FOR_EACH (lb, hmap_node, lbs) { - lb->n_nb_lr = bitmap_count1(lb->nb_lr_map, n_lr_datapaths); - lb->n_nb_ls = bitmap_count1(lb->nb_ls_map, n_ls_datapaths); + HMAP_FOR_EACH (lb_dps, hmap_node, lb_dps_map) { + lb_dps->n_nb_lr = bitmap_count1(lb_dps->nb_lr_map, n_lr_datapaths); + lb_dps->n_nb_ls = bitmap_count1(lb_dps->nb_ls_map, n_ls_datapaths); } } @@ -4408,13 +4410,16 @@ build_lb_port_related_data( struct ovsdb_idl_txn *ovnsb_txn, const struct sbrec_service_monitor_table *sbrec_service_monitor_table, struct ovn_datapaths *lr_datapaths, struct hmap *ls_ports, - struct hmap *lbs, struct hmap *lb_groups, struct sset *svc_monitor_lsps) + struct hmap *lb_dps_map, struct hmap *lb_group_dps_map, + struct sset *svc_monitor_lsps, + struct hmap *svc_monitor_map) { build_lrouter_lbs_check(lr_datapaths); - build_lrouter_lbs_reachable_ips(lr_datapaths, lbs, lb_groups); - build_lb_svcs(ovnsb_txn, sbrec_service_monitor_table, ls_ports, lbs, - svc_monitor_lsps); - build_lswitch_lbs_from_lrouter(lr_datapaths, lbs, lb_groups); + build_lrouter_lbs_reachable_ips(lr_datapaths, lb_dps_map, + lb_group_dps_map); + build_lb_svcs(ovnsb_txn, sbrec_service_monitor_table, ls_ports, lb_dps_map, + svc_monitor_lsps, svc_monitor_map); + build_lswitch_lbs_from_lrouter(lr_datapaths, lb_dps_map, lb_group_dps_map); } @@ -4535,17 +4540,39 @@ ovn_dp_group_get_or_create(struct ovsdb_idl_txn *ovnsb_txn, return dpg; } +struct sb_lb { + struct hmap_node hmap_node; + + const struct sbrec_load_balancer *slb; + struct ovn_dp_group *dpg; + struct uuid lb_uuid; +}; + +static struct sb_lb * +find_slb_in_sb_lbs(struct hmap *sb_lbs, const struct uuid *lb_uuid) +{ + struct sb_lb *sb_lb; + HMAP_FOR_EACH_WITH_HASH (sb_lb, hmap_node, uuid_hash(lb_uuid), sb_lbs) { + if (uuid_equals(&sb_lb->lb_uuid, lb_uuid)) { + return sb_lb; + } + } + + return NULL; +} + /* Syncs relevant load balancers (applied to logical switches) to the * Southbound database. */ void sync_lbs(struct ovsdb_idl_txn *ovnsb_txn, const struct sbrec_load_balancer_table *sbrec_load_balancer_table, - struct ovn_datapaths *ls_datapaths, struct hmap *lbs) + struct ovn_datapaths *ls_datapaths, struct hmap *lb_dps_map) { struct hmap dp_groups = HMAP_INITIALIZER(&dp_groups); size_t bitmap_len = ods_size(ls_datapaths); - struct ovn_northd_lb *lb; + struct ovn_lb_datapaths *lb_dps; + struct hmap sb_lbs = HMAP_INITIALIZER(&sb_lbs); /* Delete any stale SB load balancer rows and create datapath * groups for existing ones. */ @@ -4568,28 +4595,32 @@ sync_lbs(struct ovsdb_idl_txn *ovnsb_txn, * "at-least-once" consistency for clustered database tables that * are not indexed in any way. */ - lb = ovn_northd_lb_find(lbs, &lb_uuid); - if (!lb || !lb->n_nb_ls || !hmapx_add(&existing_lbs, lb)) { + lb_dps = ovn_lb_datapaths_find(lb_dps_map, &lb_uuid); + if (!lb_dps || !lb_dps->n_nb_ls || !hmapx_add(&existing_lbs, lb_dps)) { sbrec_load_balancer_delete(sbrec_lb); continue; } - lb->slb = sbrec_lb; + struct sb_lb *sb_lb = xzalloc(sizeof *sb_lb); + sb_lb->lb_uuid = lb_uuid; + sb_lb->slb = sbrec_lb; + hmap_insert(&sb_lbs, &sb_lb->hmap_node, uuid_hash(&lb_uuid)); /* Find or create datapath group for this load balancer. */ - lb->dpg = ovn_dp_group_get_or_create(ovnsb_txn, &dp_groups, - lb->slb->datapath_group, - lb->n_nb_ls, lb->nb_ls_map, - bitmap_len, true, - ls_datapaths, NULL); + sb_lb->dpg = ovn_dp_group_get_or_create(ovnsb_txn, &dp_groups, + sb_lb->slb->datapath_group, + lb_dps->n_nb_ls, + lb_dps->nb_ls_map, + bitmap_len, true, + ls_datapaths, NULL); } hmapx_destroy(&existing_lbs); /* Create SB Load balancer records if not present and sync * the SB load balancer columns. */ - HMAP_FOR_EACH (lb, hmap_node, lbs) { + HMAP_FOR_EACH (lb_dps, hmap_node, lb_dps_map) { - if (!lb->n_nb_ls) { + if (!lb_dps->n_nb_ls) { continue; } @@ -4597,37 +4628,44 @@ sync_lbs(struct ovsdb_idl_txn *ovnsb_txn, * transport port) tuple. */ struct smap options; - smap_clone(&options, &lb->nlb->options); + smap_clone(&options, &lb_dps->lb->nlb->options); smap_replace(&options, "hairpin_orig_tuple", "true"); - if (!lb->slb) { + struct sb_lb *sb_lb = find_slb_in_sb_lbs(&sb_lbs, + &lb_dps->lb->nlb->header_.uuid); + ovs_assert(!sb_lb || (sb_lb->slb && sb_lb->dpg)); + struct ovn_dp_group *lb_dpg = NULL; + if (!sb_lb) { sbrec_lb = sbrec_load_balancer_insert(ovnsb_txn); - lb->slb = sbrec_lb; char *lb_id = xasprintf( - UUID_FMT, UUID_ARGS(&lb->nlb->header_.uuid)); + UUID_FMT, UUID_ARGS(&lb_dps->lb->nlb->header_.uuid)); const struct smap external_ids = SMAP_CONST1(&external_ids, "lb_id", lb_id); sbrec_load_balancer_set_external_ids(sbrec_lb, &external_ids); free(lb_id); + } else { + sbrec_lb = sb_lb->slb; + lb_dpg = sb_lb->dpg; } /* Find or create datapath group for this load balancer. */ - if (!lb->dpg) { - lb->dpg = ovn_dp_group_get_or_create(ovnsb_txn, &dp_groups, - lb->slb->datapath_group, - lb->n_nb_ls, lb->nb_ls_map, - bitmap_len, true, - ls_datapaths, NULL); + if (!lb_dpg) { + lb_dpg = ovn_dp_group_get_or_create(ovnsb_txn, &dp_groups, + sbrec_lb->datapath_group, + lb_dps->n_nb_ls, + lb_dps->nb_ls_map, bitmap_len, + true, ls_datapaths, NULL); } /* Update columns. */ - sbrec_load_balancer_set_name(lb->slb, lb->nlb->name); - sbrec_load_balancer_set_vips(lb->slb, ovn_northd_lb_get_vips(lb)); - sbrec_load_balancer_set_protocol(lb->slb, lb->nlb->protocol); - sbrec_load_balancer_set_datapath_group(lb->slb, lb->dpg->dp_group); - sbrec_load_balancer_set_options(lb->slb, &options); + sbrec_load_balancer_set_name(sbrec_lb, lb_dps->lb->nlb->name); + sbrec_load_balancer_set_vips(sbrec_lb, + ovn_northd_lb_get_vips(lb_dps->lb)); + sbrec_load_balancer_set_protocol(sbrec_lb, lb_dps->lb->nlb->protocol); + sbrec_load_balancer_set_datapath_group(sbrec_lb, lb_dpg->dp_group); + sbrec_load_balancer_set_options(sbrec_lb, &options); /* Clearing 'datapaths' column, since 'dp_group' is in use. */ - sbrec_load_balancer_set_datapaths(lb->slb, NULL, 0); + sbrec_load_balancer_set_datapaths(sbrec_lb, NULL, 0); smap_destroy(&options); } @@ -4638,6 +4676,12 @@ sync_lbs(struct ovsdb_idl_txn *ovnsb_txn, } hmap_destroy(&dp_groups); + struct sb_lb *sb_lb; + HMAP_FOR_EACH_POP (sb_lb, hmap_node, &sb_lbs) { + free(sb_lb); + } + hmap_destroy(&sb_lbs); + /* Datapath_Binding.load_balancers is not used anymore, it's still in the * schema for compatibility reasons. Reset it to empty, just in case. */ @@ -7871,15 +7915,17 @@ build_qos(struct ovn_datapath *od, struct hmap *lflows) { } static void -build_lb_rules_pre_stateful(struct hmap *lflows, struct ovn_northd_lb *lb, +build_lb_rules_pre_stateful(struct hmap *lflows, + struct ovn_lb_datapaths *lb_dps, bool ct_lb_mark, const struct ovn_datapaths *ls_datapaths, struct ds *match, struct ds *action) { - if (!lb->n_nb_ls) { + if (!lb_dps->n_nb_ls) { return; } + const struct ovn_northd_lb *lb = lb_dps->lb; for (size_t i = 0; i < lb->n_vips; i++) { struct ovn_lb_vip *lb_vip = &lb->vips[i]; ds_clear(action); @@ -7925,7 +7971,7 @@ build_lb_rules_pre_stateful(struct hmap *lflows, struct ovn_northd_lb *lb, } ovn_lflow_add_with_dp_group( - lflows, lb->nb_ls_map, ods_size(ls_datapaths), + lflows, lb_dps->nb_ls_map, ods_size(ls_datapaths), S_SWITCH_IN_PRE_STATEFUL, 120, ds_cstr(match), ds_cstr(action), &lb->nlb->header_); } @@ -7971,7 +8017,7 @@ build_lb_rules_pre_stateful(struct hmap *lflows, struct ovn_northd_lb *lb, * */ static void -build_lb_affinity_lr_flows(struct hmap *lflows, struct ovn_northd_lb *lb, +build_lb_affinity_lr_flows(struct hmap *lflows, const struct ovn_northd_lb *lb, struct ovn_lb_vip *lb_vip, char *new_lb_match, char *lb_action, const unsigned long *dp_bitmap, const struct ovn_datapaths *lr_datapaths) @@ -8157,14 +8203,16 @@ build_lb_affinity_lr_flows(struct hmap *lflows, struct ovn_northd_lb *lb, * */ static void -build_lb_affinity_ls_flows(struct hmap *lflows, struct ovn_northd_lb *lb, +build_lb_affinity_ls_flows(struct hmap *lflows, + struct ovn_lb_datapaths *lb_dps, struct ovn_lb_vip *lb_vip, const struct ovn_datapaths *ls_datapaths) { - if (!lb->affinity_timeout || !lb->n_nb_ls) { + if (!lb_dps->lb->affinity_timeout || !lb_dps->n_nb_ls) { return; } + const struct ovn_northd_lb *lb = lb_dps->lb; struct ds new_lb_match = DS_EMPTY_INITIALIZER; if (IN6_IS_ADDR_V4MAPPED(&lb_vip->vip)) { ds_put_format(&new_lb_match, @@ -8184,9 +8232,9 @@ build_lb_affinity_ls_flows(struct hmap *lflows, struct ovn_northd_lb *lb, static char *aff_check = REGBIT_KNOWN_LB_SESSION" = chk_lb_aff(); next;"; ovn_lflow_add_with_dp_group( - lflows, lb->nb_ls_map, ods_size(ls_datapaths), + lflows, lb_dps->nb_ls_map, ods_size(ls_datapaths), S_SWITCH_IN_LB_AFF_CHECK, 100, ds_cstr(&new_lb_match), aff_check, - &lb->nlb->header_); + &lb_dps->lb->nlb->header_); ds_destroy(&new_lb_match); struct ds aff_action = DS_EMPTY_INITIALIZER; @@ -8274,14 +8322,15 @@ build_lb_affinity_ls_flows(struct hmap *lflows, struct ovn_northd_lb *lb, /* Forward to OFTABLE_CHK_LB_AFFINITY table to store flow tuple. */ ovn_lflow_add_with_dp_group( - lflows, lb->nb_ls_map, ods_size(ls_datapaths), + lflows, lb_dps->nb_ls_map, ods_size(ls_datapaths), S_SWITCH_IN_LB_AFF_LEARN, 100, ds_cstr(&aff_match_learn), ds_cstr(&aff_action_learn), &lb->nlb->header_); /* Use already selected backend within affinity timeslot. */ ovn_lflow_add_with_dp_group( - lflows, lb->nb_ls_map, ods_size(ls_datapaths), S_SWITCH_IN_LB, 150, - ds_cstr(&aff_match), ds_cstr(&aff_action), &lb->nlb->header_); + lflows, lb_dps->nb_ls_map, ods_size(ls_datapaths), + S_SWITCH_IN_LB, 150, ds_cstr(&aff_match), ds_cstr(&aff_action), + &lb->nlb->header_); ds_truncate(&aff_action, aff_action_len); ds_truncate(&aff_action_learn, aff_action_learn_len); @@ -8314,11 +8363,13 @@ build_lrouter_lb_affinity_default_flows(struct ovn_datapath *od, } static void -build_lb_rules(struct hmap *lflows, struct ovn_northd_lb *lb, +build_lb_rules(struct hmap *lflows, struct ovn_lb_datapaths *lb_dps, const struct ovn_datapaths *ls_datapaths, const struct chassis_features *features, struct ds *match, - struct ds *action, const struct shash *meter_groups) + struct ds *action, const struct shash *meter_groups, + const struct hmap *svc_monitor_map) { + const struct ovn_northd_lb *lb = lb_dps->lb; for (size_t i = 0; i < lb->n_vips; i++) { struct ovn_lb_vip *lb_vip = &lb->vips[i]; struct ovn_northd_lb_vip *lb_vip_nb = &lb->vips_nb[i]; @@ -8339,9 +8390,10 @@ build_lb_rules(struct hmap *lflows, struct ovn_northd_lb *lb, /* New connections in Ingress table. */ const char *meter = NULL; - bool reject = build_lb_vip_actions(lb_vip, lb_vip_nb, action, - lb->selection_fields, NULL, - NULL, true, features); + bool reject = build_lb_vip_actions(lb, lb_vip, lb_vip_nb, action, + lb->selection_fields, + NULL, NULL, true, features, + svc_monitor_map); ds_put_format(match, "ct.new && %s.dst == %s", ip_match, lb_vip->vip_str); @@ -8352,15 +8404,17 @@ build_lb_rules(struct hmap *lflows, struct ovn_northd_lb *lb, priority = 120; } - build_lb_affinity_ls_flows(lflows, lb, lb_vip, ls_datapaths); + build_lb_affinity_ls_flows(lflows, lb_dps, lb_vip, ls_datapaths); unsigned long *dp_non_meter = NULL; bool build_non_meter = false; if (reject) { size_t index; - dp_non_meter = bitmap_clone(lb->nb_ls_map, ods_size(ls_datapaths)); - BITMAP_FOR_EACH_1 (index, ods_size(ls_datapaths), lb->nb_ls_map) { + dp_non_meter = bitmap_clone(lb_dps->nb_ls_map, + ods_size(ls_datapaths)); + BITMAP_FOR_EACH_1 (index, ods_size(ls_datapaths), + lb_dps->nb_ls_map) { struct ovn_datapath *od = ls_datapaths->array[index]; meter = copp_meter_get(COPP_REJECT, od->nbs->copp, @@ -8378,7 +8432,7 @@ build_lb_rules(struct hmap *lflows, struct ovn_northd_lb *lb, } if (!reject || build_non_meter) { ovn_lflow_add_with_dp_group( - lflows, dp_non_meter ? dp_non_meter : lb->nb_ls_map, + lflows, dp_non_meter ? dp_non_meter : lb_dps->nb_ls_map, ods_size(ls_datapaths), S_SWITCH_IN_LB, priority, ds_cstr(match), ds_cstr(action), &lb->nlb->header_); } @@ -9593,7 +9647,8 @@ build_lswitch_arp_nd_responder_default(struct ovn_datapath *od, /* Ingress table 19: ARP/ND responder for service monitor source ip. * (priority 110)*/ static void -build_lswitch_arp_nd_service_monitor(struct ovn_northd_lb *lb, +build_lswitch_arp_nd_service_monitor(const struct ovn_northd_lb *lb, + const struct hmap *ls_ports, struct hmap *lflows, struct ds *actions, struct ds *match) @@ -9608,7 +9663,14 @@ build_lswitch_arp_nd_service_monitor(struct ovn_northd_lb *lb, for (size_t j = 0; j < lb_vip_nb->n_backends; j++) { struct ovn_northd_lb_backend *backend_nb = &lb_vip_nb->backends_nb[j]; - if (!backend_nb->op || !backend_nb->svc_mon_src_ip) { + + if (!backend_nb->health_check) { + continue; + } + + struct ovn_port *op = ovn_port_find(ls_ports, + backend_nb->logical_port); + if (!op || !backend_nb->svc_mon_src_ip) { continue; } @@ -9650,7 +9712,7 @@ build_lswitch_arp_nd_service_monitor(struct ovn_northd_lb *lb, svc_monitor_mac); } ovn_lflow_add_with_hint(lflows, - backend_nb->op->od, + op->od, S_SWITCH_IN_ARP_ND_RSP, 110, ds_cstr(match), ds_cstr(actions), &lb->nlb->header_); @@ -11375,7 +11437,7 @@ struct lrouter_nat_lb_flows_ctx { struct ds *gw_redir_action; struct ovn_lb_vip *lb_vip; - struct ovn_northd_lb *lb; + const struct ovn_northd_lb *lb; bool reject; int prio; @@ -11507,14 +11569,16 @@ build_gw_lrouter_nat_flows_for_lb(struct lrouter_nat_lb_flows_ctx *ctx, static void build_lrouter_nat_flows_for_lb(struct ovn_lb_vip *lb_vip, - struct ovn_northd_lb *lb, + struct ovn_lb_datapaths *lb_dps, struct ovn_northd_lb_vip *vips_nb, const struct ovn_datapaths *lr_datapaths, struct hmap *lflows, struct ds *match, struct ds *action, const struct shash *meter_groups, - const struct chassis_features *features) + const struct chassis_features *features, + const struct hmap *svc_monitor_map) { + const struct ovn_northd_lb *lb = lb_dps->lb; bool ipv4 = lb_vip->address_family == AF_INET; const char *ip_match = ipv4 ? "ip4" : "ip6"; @@ -11529,9 +11593,10 @@ build_lrouter_nat_flows_for_lb(struct ovn_lb_vip *lb_vip, ds_clear(match); ds_clear(action); - bool reject = build_lb_vip_actions(lb_vip, vips_nb, action, + bool reject = build_lb_vip_actions(lb, lb_vip, vips_nb, action, lb->selection_fields, &skip_snat_act, - &force_snat_act, false, features); + &force_snat_act, false, features, + svc_monitor_map); /* Higher priority rules are added for load-balancing in DNAT * table. For every match (on a VIP[:port]), we add two flows. @@ -11606,7 +11671,7 @@ build_lrouter_nat_flows_for_lb(struct ovn_lb_vip *lb_vip, * lflow generation for them. */ size_t index; - BITMAP_FOR_EACH_1 (index, bitmap_len, lb->nb_lr_map) { + BITMAP_FOR_EACH_1 (index, bitmap_len, lb_dps->nb_lr_map) { struct ovn_datapath *od = lr_datapaths->array[index]; enum lrouter_nat_lb_flow_type type; @@ -11686,16 +11751,19 @@ build_lrouter_nat_flows_for_lb(struct ovn_lb_vip *lb_vip, } static void -build_lswitch_flows_for_lb(struct ovn_northd_lb *lb, struct hmap *lflows, +build_lswitch_flows_for_lb(struct ovn_lb_datapaths *lb_dps, + struct hmap *lflows, const struct shash *meter_groups, const struct ovn_datapaths *ls_datapaths, const struct chassis_features *features, + const struct hmap *svc_monitor_map, struct ds *match, struct ds *action) { - if (!lb->n_nb_ls) { + if (!lb_dps->n_nb_ls) { return; } + const struct ovn_northd_lb *lb = lb_dps->lb; for (size_t i = 0; i < lb->n_vips; i++) { struct ovn_lb_vip *lb_vip = &lb->vips[i]; @@ -11705,7 +11773,7 @@ build_lswitch_flows_for_lb(struct ovn_northd_lb *lb, struct hmap *lflows, } size_t index; - BITMAP_FOR_EACH_1 (index, ods_size(ls_datapaths), lb->nb_ls_map) { + BITMAP_FOR_EACH_1 (index, ods_size(ls_datapaths), lb_dps->nb_ls_map) { struct ovn_datapath *od = ls_datapaths->array[index]; ovn_lflow_add_with_hint__(lflows, od, @@ -11729,10 +11797,10 @@ build_lswitch_flows_for_lb(struct ovn_northd_lb *lb, struct hmap *lflows, * a higher priority rule for load balancing below also commits the * connection, so it is okay if we do not hit the above match on * REGBIT_CONNTRACK_COMMIT. */ - build_lb_rules_pre_stateful(lflows, lb, features->ct_no_masked_label, + build_lb_rules_pre_stateful(lflows, lb_dps, features->ct_no_masked_label, ls_datapaths, match, action); - build_lb_rules(lflows, lb, ls_datapaths, features, match, action, - meter_groups); + build_lb_rules(lflows, lb_dps, ls_datapaths, features, match, action, + meter_groups, svc_monitor_map); } /* If there are any load balancing rules, we should send the packet to @@ -11744,17 +11812,17 @@ build_lswitch_flows_for_lb(struct ovn_northd_lb *lb, struct hmap *lflows, * defragmentation to match on L4 ports. */ static void -build_lrouter_defrag_flows_for_lb(struct ovn_northd_lb *lb, +build_lrouter_defrag_flows_for_lb(struct ovn_lb_datapaths *lb_dps, struct hmap *lflows, const struct ovn_datapaths *lr_datapaths, struct ds *match) { - if (!lb->n_nb_lr) { + if (!lb_dps->n_nb_lr) { return; } - for (size_t i = 0; i < lb->n_vips; i++) { - struct ovn_lb_vip *lb_vip = &lb->vips[i]; + for (size_t i = 0; i < lb_dps->lb->n_vips; i++) { + struct ovn_lb_vip *lb_vip = &lb_dps->lb->vips[i]; bool ipv6 = lb_vip->address_family == AF_INET6; int prio = 100; @@ -11763,36 +11831,41 @@ build_lrouter_defrag_flows_for_lb(struct ovn_northd_lb *lb, lb_vip->vip_str); ovn_lflow_add_with_dp_group( - lflows, lb->nb_lr_map, ods_size(lr_datapaths), S_ROUTER_IN_DEFRAG, - prio, ds_cstr(match), "ct_dnat;", &lb->nlb->header_); + lflows, lb_dps->nb_lr_map, ods_size(lr_datapaths), + S_ROUTER_IN_DEFRAG, prio, ds_cstr(match), "ct_dnat;", + &lb_dps->lb->nlb->header_); } } static void -build_lrouter_flows_for_lb(struct ovn_northd_lb *lb, struct hmap *lflows, +build_lrouter_flows_for_lb(struct ovn_lb_datapaths *lb_dps, + struct hmap *lflows, const struct shash *meter_groups, const struct ovn_datapaths *lr_datapaths, const struct chassis_features *features, + const struct hmap *svc_monitor_map, struct ds *match, struct ds *action) { size_t index; - if (!lb->n_nb_lr) { + if (!lb_dps->n_nb_lr) { return; } + const struct ovn_northd_lb *lb = lb_dps->lb; for (size_t i = 0; i < lb->n_vips; i++) { struct ovn_lb_vip *lb_vip = &lb->vips[i]; - build_lrouter_nat_flows_for_lb(lb_vip, lb, &lb->vips_nb[i], + build_lrouter_nat_flows_for_lb(lb_vip, lb_dps, &lb->vips_nb[i], lr_datapaths, lflows, match, action, - meter_groups, features); + meter_groups, features, + svc_monitor_map); if (!build_empty_lb_event_flow(lb_vip, lb, match, action)) { continue; } - BITMAP_FOR_EACH_1 (index, ods_size(lr_datapaths), lb->nb_lr_map) { + BITMAP_FOR_EACH_1 (index, ods_size(lr_datapaths), lb_dps->nb_lr_map) { struct ovn_datapath *od = lr_datapaths->array[index]; ovn_lflow_add_with_hint__(lflows, od, S_ROUTER_IN_DNAT, @@ -11806,7 +11879,7 @@ build_lrouter_flows_for_lb(struct ovn_northd_lb *lb, struct hmap *lflows, } if (lb->skip_snat) { - BITMAP_FOR_EACH_1 (index, ods_size(lr_datapaths), lb->nb_lr_map) { + BITMAP_FOR_EACH_1 (index, ods_size(lr_datapaths), lb_dps->nb_lr_map) { struct ovn_datapath *od = lr_datapaths->array[index]; ovn_lflow_add(lflows, od, S_ROUTER_OUT_SNAT, 120, @@ -15523,7 +15596,8 @@ struct lswitch_flow_build_info { struct hmap *lflows; struct hmap *igmp_groups; const struct shash *meter_groups; - const struct hmap *lbs; + const struct hmap *lb_dps_map; + const struct hmap *svc_monitor_map; const struct hmap *bfd_connections; const struct chassis_features *features; char *svc_check_match; @@ -15667,7 +15741,7 @@ build_lflows_thread(void *arg) struct ovn_datapath *od; struct ovn_port *op; - struct ovn_northd_lb *lb; + struct ovn_lb_datapaths *lb_dps; struct ovn_igmp_group *igmp_group; int bnum; @@ -15734,28 +15808,33 @@ build_lflows_thread(void *arg) } } for (bnum = control->id; - bnum <= lsi->lbs->mask; + bnum <= lsi->lb_dps_map->mask; bnum += control->pool->size) { - HMAP_FOR_EACH_IN_PARALLEL (lb, hmap_node, bnum, lsi->lbs) { + HMAP_FOR_EACH_IN_PARALLEL (lb_dps, hmap_node, bnum, + lsi->lb_dps_map) { if (stop_parallel_processing()) { return NULL; } - build_lswitch_arp_nd_service_monitor(lb, lsi->lflows, + build_lswitch_arp_nd_service_monitor(lb_dps->lb, + lsi->ls_ports, + lsi->lflows, &lsi->match, &lsi->actions); - build_lrouter_defrag_flows_for_lb(lb, lsi->lflows, + build_lrouter_defrag_flows_for_lb(lb_dps, lsi->lflows, lsi->lr_datapaths, &lsi->match); - build_lrouter_flows_for_lb(lb, lsi->lflows, + build_lrouter_flows_for_lb(lb_dps, lsi->lflows, lsi->meter_groups, lsi->lr_datapaths, lsi->features, + lsi->svc_monitor_map, &lsi->match, &lsi->actions); - build_lswitch_flows_for_lb(lb, lsi->lflows, + build_lswitch_flows_for_lb(lb_dps, lsi->lflows, lsi->meter_groups, lsi->ls_datapaths, lsi->features, + lsi->svc_monitor_map, &lsi->match, &lsi->actions); } } @@ -15821,7 +15900,8 @@ build_lswitch_and_lrouter_flows(const struct ovn_datapaths *ls_datapaths, struct hmap *lflows, struct hmap *igmp_groups, const struct shash *meter_groups, - const struct hmap *lbs, + const struct hmap *lb_dps_map, + const struct hmap *svc_monitor_map, const struct hmap *bfd_connections, const struct chassis_features *features) { @@ -15848,7 +15928,8 @@ build_lswitch_and_lrouter_flows(const struct ovn_datapaths *ls_datapaths, lsiv[index].port_groups = port_groups; lsiv[index].igmp_groups = igmp_groups; lsiv[index].meter_groups = meter_groups; - lsiv[index].lbs = lbs; + lsiv[index].lb_dps_map = lb_dps_map; + lsiv[index].svc_monitor_map = svc_monitor_map; lsiv[index].bfd_connections = bfd_connections; lsiv[index].features = features; lsiv[index].svc_check_match = svc_check_match; @@ -15871,7 +15952,7 @@ build_lswitch_and_lrouter_flows(const struct ovn_datapaths *ls_datapaths, } else { struct ovn_datapath *od; struct ovn_port *op; - struct ovn_northd_lb *lb; + struct ovn_lb_datapaths *lb_dps; struct ovn_igmp_group *igmp_group; struct lswitch_flow_build_info lsi = { .ls_datapaths = ls_datapaths, @@ -15882,7 +15963,8 @@ build_lswitch_and_lrouter_flows(const struct ovn_datapaths *ls_datapaths, .lflows = lflows, .igmp_groups = igmp_groups, .meter_groups = meter_groups, - .lbs = lbs, + .lb_dps_map = lb_dps_map, + .svc_monitor_map = svc_monitor_map, .bfd_connections = bfd_connections, .features = features, .svc_check_match = svc_check_match, @@ -15914,17 +15996,19 @@ build_lswitch_and_lrouter_flows(const struct ovn_datapaths *ls_datapaths, } stopwatch_stop(LFLOWS_PORTS_STOPWATCH_NAME, time_msec()); stopwatch_start(LFLOWS_LBS_STOPWATCH_NAME, time_msec()); - HMAP_FOR_EACH (lb, hmap_node, lbs) { - build_lswitch_arp_nd_service_monitor(lb, lsi.lflows, - &lsi.actions, + HMAP_FOR_EACH (lb_dps, hmap_node, lb_dps_map) { + build_lswitch_arp_nd_service_monitor(lb_dps->lb, lsi.ls_ports, + lsi.lflows, &lsi.actions, &lsi.match); - build_lrouter_defrag_flows_for_lb(lb, lsi.lflows, lsi.lr_datapaths, - &lsi.match); - build_lrouter_flows_for_lb(lb, lsi.lflows, lsi.meter_groups, + build_lrouter_defrag_flows_for_lb(lb_dps, lsi.lflows, + lsi.lr_datapaths, &lsi.match); + build_lrouter_flows_for_lb(lb_dps, lsi.lflows, lsi.meter_groups, lsi.lr_datapaths, lsi.features, + lsi.svc_monitor_map, &lsi.match, &lsi.actions); - build_lswitch_flows_for_lb(lb, lsi.lflows, lsi.meter_groups, + build_lswitch_flows_for_lb(lb_dps, lsi.lflows, lsi.meter_groups, lsi.ls_datapaths, lsi.features, + lsi.svc_monitor_map, &lsi.match, &lsi.actions); } stopwatch_stop(LFLOWS_LBS_STOPWATCH_NAME, time_msec()); @@ -16024,7 +16108,9 @@ void build_lflows(struct ovsdb_idl_txn *ovnsb_txn, input_data->lr_ports, input_data->port_groups, lflows, &igmp_groups, - input_data->meter_groups, input_data->lbs, + input_data->meter_groups, + input_data->lb_datapaths_map, + input_data->svc_monitor_map, input_data->bfd_connections, input_data->features); @@ -17427,8 +17513,8 @@ northd_init(struct northd_data *data) hmap_init(&data->lr_ports); hmap_init(&data->port_groups); shash_init(&data->meter_groups); - hmap_init(&data->lbs); - hmap_init(&data->lb_groups); + hmap_init(&data->lb_datapaths_map); + hmap_init(&data->lb_group_datapaths_map); ovs_list_init(&data->lr_list); data->features = (struct chassis_features) { .ct_no_masked_label = true, @@ -17438,6 +17524,7 @@ northd_init(struct northd_data *data) }; data->ovn_internal_version_changed = false; sset_init(&data->svc_monitor_lsps); + hmap_init(&data->svc_monitor_map); data->change_tracked = false; ovs_list_init(&data->tracked_ls_changes.updated); } @@ -17445,17 +17532,18 @@ northd_init(struct northd_data *data) void northd_destroy(struct northd_data *data) { - struct ovn_northd_lb *lb; - HMAP_FOR_EACH_POP (lb, hmap_node, &data->lbs) { - ovn_northd_lb_destroy(lb); + struct ovn_lb_datapaths *lb_dps; + HMAP_FOR_EACH_POP (lb_dps, hmap_node, &data->lb_datapaths_map) { + ovn_lb_datapaths_destroy(lb_dps); } - hmap_destroy(&data->lbs); + hmap_destroy(&data->lb_datapaths_map); - struct ovn_lb_group *lb_group; - HMAP_FOR_EACH_POP (lb_group, hmap_node, &data->lb_groups) { - ovn_lb_group_destroy(lb_group); + struct ovn_lb_group_datapaths *lb_group_dps; + HMAP_FOR_EACH_POP (lb_group_dps, hmap_node, + &data->lb_group_datapaths_map) { + ovn_lb_group_datapaths_destroy(lb_group_dps); } - hmap_destroy(&data->lb_groups); + hmap_destroy(&data->lb_group_datapaths_map); struct ovn_port_group *pg; HMAP_FOR_EACH_SAFE (pg, key_node, &data->port_groups) { @@ -17470,6 +17558,12 @@ northd_destroy(struct northd_data *data) } shash_destroy(&data->meter_groups); + struct service_monitor_info *mon_info; + HMAP_FOR_EACH_POP (mon_info, hmap_node, &data->svc_monitor_map) { + free(mon_info); + } + hmap_destroy(&data->svc_monitor_map); + /* XXX Having to explicitly clean up macam here * is a bit strange. We don't explicitly initialize * macam in this module, but this is the logical place @@ -17578,10 +17672,9 @@ ovnnb_db_run(struct northd_input *input_data, input_data->sbrec_chassis_table, &data->ls_datapaths, &data->lr_datapaths, &data->lr_list); - build_lbs(input_data->nbrec_load_balancer_table, - input_data->nbrec_load_balancer_group_table, - &data->ls_datapaths, &data->lr_datapaths, &data->lbs, - &data->lb_groups); + build_lb_datapaths(input_data->lbs, input_data->lb_groups, + &data->ls_datapaths, &data->lr_datapaths, + &data->lb_datapaths_map, &data->lb_group_datapaths_map); build_ports(ovnsb_txn, input_data->sbrec_port_binding_table, input_data->sbrec_chassis_table, @@ -17596,9 +17689,11 @@ ovnnb_db_run(struct northd_input *input_data, build_lb_port_related_data(ovnsb_txn, input_data->sbrec_service_monitor_table, &data->lr_datapaths, &data->ls_ports, - &data->lbs, &data->lb_groups, - &data->svc_monitor_lsps); - build_lb_count_dps(&data->lbs, + &data->lb_datapaths_map, + &data->lb_group_datapaths_map, + &data->svc_monitor_lsps, + &data->svc_monitor_map); + build_lb_count_dps(&data->lb_datapaths_map, ods_size(&data->ls_datapaths), ods_size(&data->lr_datapaths)); build_ipam(&data->ls_datapaths.datapaths, &data->ls_ports); diff --git a/northd/northd.h b/northd/northd.h index 48c282476a..7d92028c7d 100644 --- a/northd/northd.h +++ b/northd/northd.h @@ -28,9 +28,6 @@ struct northd_input { const struct nbrec_nb_global_table *nbrec_nb_global_table; const struct nbrec_logical_switch_table *nbrec_logical_switch_table; const struct nbrec_logical_router_table *nbrec_logical_router_table; - const struct nbrec_load_balancer_table *nbrec_load_balancer_table; - const struct nbrec_load_balancer_group_table - *nbrec_load_balancer_group_table; const struct nbrec_port_group_table *nbrec_port_group_table; const struct nbrec_meter_table *nbrec_meter_table; const struct nbrec_acl_table *nbrec_acl_table; @@ -59,6 +56,10 @@ struct northd_input { *sbrec_chassis_template_var_table; const struct sbrec_mirror_table *sbrec_mirror_table; + /* Northd lb data node inputs*/ + const struct hmap *lbs; + const struct hmap *lb_groups; + /* Indexes */ struct ovsdb_idl_index *sbrec_chassis_by_name; struct ovsdb_idl_index *sbrec_chassis_by_hostname; @@ -110,12 +111,13 @@ struct northd_data { struct hmap lr_ports; struct hmap port_groups; struct shash meter_groups; - struct hmap lbs; - struct hmap lb_groups; + struct hmap lb_datapaths_map; + struct hmap lb_group_datapaths_map; struct ovs_list lr_list; bool ovn_internal_version_changed; struct chassis_features features; struct sset svc_monitor_lsps; + struct hmap svc_monitor_map; bool change_tracked; struct tracked_ls_changes tracked_ls_changes; }; @@ -146,9 +148,10 @@ struct lflow_input { const struct hmap *lr_ports; const struct hmap *port_groups; const struct shash *meter_groups; - const struct hmap *lbs; + const struct hmap *lb_datapaths_map; const struct hmap *bfd_connections; const struct chassis_features *features; + const struct hmap *svc_monitor_map; bool ovn_internal_version_changed; }; From patchwork Fri Aug 18 08:57:30 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Numan Siddique X-Patchwork-Id: 1822755 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.133; helo=smtp2.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4RRwkZ6Y6Jz1ygH for ; Fri, 18 Aug 2023 18:57:58 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 65BAE4190D; Fri, 18 Aug 2023 08:57:56 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 65BAE4190D X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id C_nvkAnM0h1d; Fri, 18 Aug 2023 08:57:47 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp2.osuosl.org (Postfix) with ESMTPS id 063B44193E; Fri, 18 Aug 2023 08:57:45 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 063B44193E Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id BD5F8C0032; Fri, 18 Aug 2023 08:57:44 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp2.osuosl.org (smtp2.osuosl.org [IPv6:2605:bc80:3010::133]) by lists.linuxfoundation.org (Postfix) with ESMTP id 9A501C0032 for ; Fri, 18 Aug 2023 08:57:43 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 559EF416BC for ; Fri, 18 Aug 2023 08:57:43 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 559EF416BC X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id kse8jJQPOC7B for ; Fri, 18 Aug 2023 08:57:39 +0000 (UTC) Received: from relay2-d.mail.gandi.net (relay2-d.mail.gandi.net [217.70.183.194]) by smtp2.osuosl.org (Postfix) with ESMTPS id 95AA34180C for ; Fri, 18 Aug 2023 08:57:38 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 95AA34180C Received: by mail.gandi.net (Postfix) with ESMTPSA id 3131C40004; Fri, 18 Aug 2023 08:57:33 +0000 (UTC) From: numans@ovn.org To: dev@openvswitch.org Date: Fri, 18 Aug 2023 14:27:30 +0530 Message-Id: <20230818085730.1030949-1-numans@ovn.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230818085606.1030792-1-numans@ovn.org> References: <20230818085606.1030792-1-numans@ovn.org> MIME-Version: 1.0 X-GND-Sasl: numans@ovn.org Subject: [ovs-dev] [PATCH ovn v6 03/16] northd: Add initial I-P for load balancer and load balancer groups X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Numan Siddique Any changes to load balancers and load balancer groups are handled incrementally in the newly added 'lb_data' engine node. 'lb_data' is input to 'northd' node and the handler - northd_lb_data_handler in 'northd' node handles the changes. If a load balancer or load balancer group is associated to a logical switch or router then 'northd' node falls back to full recompute. Upcoming patch will handle this scenario. Signed-off-by: Numan Siddique Acked-by: Han Zhou Reviewed-by: Ales Musil --- lib/lb.c | 82 +++++-- lib/lb.h | 9 + northd/en-lb-data.c | 273 ++++++++++++++++++++++- northd/en-lb-data.h | 50 +++++ northd/en-northd.c | 41 ++++ northd/en-northd.h | 1 + northd/inc-proc-northd.c | 12 +- northd/northd.c | 59 +++++ northd/northd.h | 7 + tests/ovn-northd.at | 457 +++++++++++++++++++++++++++++++++++++++ 10 files changed, 960 insertions(+), 31 deletions(-) diff --git a/lib/lb.c b/lib/lb.c index e874680a3f..6fd67e2218 100644 --- a/lib/lb.c +++ b/lib/lb.c @@ -606,13 +606,13 @@ ovn_lb_get_health_check(const struct nbrec_load_balancer *nbrec_lb, return NULL; } -struct ovn_northd_lb * -ovn_northd_lb_create(const struct nbrec_load_balancer *nbrec_lb) +static void +ovn_northd_lb_init(struct ovn_northd_lb *lb, + const struct nbrec_load_balancer *nbrec_lb) { bool template = smap_get_bool(&nbrec_lb->options, "template", false); bool is_udp = nullable_string_is_equal(nbrec_lb->protocol, "udp"); bool is_sctp = nullable_string_is_equal(nbrec_lb->protocol, "sctp"); - struct ovn_northd_lb *lb = xzalloc(sizeof *lb); int address_family = !strcmp(smap_get_def(&nbrec_lb->options, "address-family", "ipv4"), "ipv4") @@ -668,6 +668,10 @@ ovn_northd_lb_create(const struct nbrec_load_balancer *nbrec_lb) "reject", false); ovn_northd_lb_vip_init(lb_vip_nb, lb_vip, nbrec_lb, node->key, node->value, template); + if (lb_vip_nb->lb_health_check) { + lb->health_checks = true; + } + if (IN6_IS_ADDR_V4MAPPED(&lb_vip->vip)) { sset_add(&lb->ips_v4, lb_vip->vip_str); } else { @@ -711,6 +715,13 @@ ovn_northd_lb_create(const struct nbrec_load_balancer *nbrec_lb) ds_chomp(&sel_fields, ','); lb->selection_fields = ds_steal_cstr(&sel_fields); } +} + +struct ovn_northd_lb * +ovn_northd_lb_create(const struct nbrec_load_balancer *nbrec_lb) +{ + struct ovn_northd_lb *lb = xzalloc(sizeof *lb); + ovn_northd_lb_init(lb, nbrec_lb); return lb; } @@ -736,8 +747,8 @@ ovn_northd_lb_get_vips(const struct ovn_northd_lb *lb) return &lb->nlb->vips; } -void -ovn_northd_lb_destroy(struct ovn_northd_lb *lb) +static void +ovn_northd_lb_cleanup(struct ovn_northd_lb *lb) { for (size_t i = 0; i < lb->n_vips; i++) { ovn_lb_vip_destroy(&lb->vips[i]); @@ -745,24 +756,36 @@ ovn_northd_lb_destroy(struct ovn_northd_lb *lb) } free(lb->vips); free(lb->vips_nb); + lb->vips = NULL; + lb->vips_nb = NULL; smap_destroy(&lb->template_vips); sset_destroy(&lb->ips_v4); sset_destroy(&lb->ips_v6); free(lb->selection_fields); + lb->selection_fields = NULL; + lb->health_checks = false; +} + +void +ovn_northd_lb_destroy(struct ovn_northd_lb *lb) +{ + ovn_northd_lb_cleanup(lb); free(lb); } -/* Constructs a new 'struct ovn_lb_group' object from the Nb LB Group record - * and an array of 'struct ovn_northd_lb' objects for its associated - * load balancers. */ -struct ovn_lb_group * -ovn_lb_group_create(const struct nbrec_load_balancer_group *nbrec_lb_group, - const struct hmap *lbs) +void +ovn_northd_lb_reinit(struct ovn_northd_lb *lb, + const struct nbrec_load_balancer *nbrec_lb) { - struct ovn_lb_group *lb_group; + ovn_northd_lb_cleanup(lb); + ovn_northd_lb_init(lb, nbrec_lb); +} - lb_group = xzalloc(sizeof *lb_group); - lb_group->uuid = nbrec_lb_group->header_.uuid; +static void +ovn_lb_group_init(struct ovn_lb_group *lb_group, + const struct nbrec_load_balancer_group *nbrec_lb_group, + const struct hmap *lbs) +{ lb_group->n_lbs = nbrec_lb_group->n_load_balancer; lb_group->lbs = xmalloc(lb_group->n_lbs * sizeof *lb_group->lbs); lb_group->lb_ips = ovn_lb_ip_set_create(); @@ -772,10 +795,29 @@ ovn_lb_group_create(const struct nbrec_load_balancer_group *nbrec_lb_group, &nbrec_lb_group->load_balancer[i]->header_.uuid; lb_group->lbs[i] = ovn_northd_lb_find(lbs, lb_uuid); } +} +/* Constructs a new 'struct ovn_lb_group' object from the Nb LB Group record + * and an array of 'struct ovn_northd_lb' objects for its associated + * load balancers. */ +struct ovn_lb_group * +ovn_lb_group_create(const struct nbrec_load_balancer_group *nbrec_lb_group, + const struct hmap *lbs) +{ + struct ovn_lb_group *lb_group = xzalloc(sizeof *lb_group); + lb_group->uuid = nbrec_lb_group->header_.uuid; + ovn_lb_group_init(lb_group, nbrec_lb_group, lbs); return lb_group; } +static void +ovn_lb_group_cleanup(struct ovn_lb_group *lb_group) +{ + ovn_lb_ip_set_destroy(lb_group->lb_ips); + lb_group->lb_ips = NULL; + free(lb_group->lbs); +} + void ovn_lb_group_destroy(struct ovn_lb_group *lb_group) { @@ -783,11 +825,19 @@ ovn_lb_group_destroy(struct ovn_lb_group *lb_group) return; } - ovn_lb_ip_set_destroy(lb_group->lb_ips); - free(lb_group->lbs); + ovn_lb_group_cleanup(lb_group); free(lb_group); } +void +ovn_lb_group_reinit(struct ovn_lb_group *lb_group, + const struct nbrec_load_balancer_group *nbrec_lb_group, + const struct hmap *lbs) +{ + ovn_lb_group_cleanup(lb_group); + ovn_lb_group_init(lb_group, nbrec_lb_group, lbs); +} + struct ovn_lb_group * ovn_lb_group_find(const struct hmap *lb_groups, const struct uuid *uuid) { diff --git a/lib/lb.h b/lib/lb.h index 0339050cba..74905c73b7 100644 --- a/lib/lb.h +++ b/lib/lb.h @@ -77,6 +77,9 @@ struct ovn_northd_lb { struct sset ips_v4; struct sset ips_v6; + + /* Indicates if the load balancer has health checks configured. */ + bool health_checks; }; struct ovn_lb_vip { @@ -130,6 +133,8 @@ struct ovn_northd_lb *ovn_northd_lb_find(const struct hmap *, const struct uuid *); const struct smap *ovn_northd_lb_get_vips(const struct ovn_northd_lb *); void ovn_northd_lb_destroy(struct ovn_northd_lb *); +void ovn_northd_lb_reinit(struct ovn_northd_lb *, + const struct nbrec_load_balancer *); void build_lrouter_lb_ips(struct ovn_lb_ip_set *, const struct ovn_northd_lb *); @@ -148,6 +153,10 @@ struct ovn_lb_group *ovn_lb_group_create( void ovn_lb_group_destroy(struct ovn_lb_group *lb_group); struct ovn_lb_group *ovn_lb_group_find(const struct hmap *lb_groups, const struct uuid *); +void ovn_lb_group_reinit( + struct ovn_lb_group *lb_group, + const struct nbrec_load_balancer_group *, + const struct hmap *lbs); struct ovn_lb_datapaths { struct hmap_node hmap_node; diff --git a/northd/en-lb-data.c b/northd/en-lb-data.c index 04d8d3e5d7..328c003675 100644 --- a/northd/en-lb-data.c +++ b/northd/en-lb-data.c @@ -19,6 +19,7 @@ #include /* OVS includes */ +#include "include/openvswitch/hmap.h" #include "openvswitch/util.h" #include "openvswitch/vlog.h" @@ -38,7 +39,27 @@ static void lb_data_destroy(struct ed_type_lb_data *); static void build_lbs(const struct nbrec_load_balancer_table *, const struct nbrec_load_balancer_group_table *, struct hmap *lbs, struct hmap *lb_groups); +static struct ovn_lb_group *create_lb_group( + const struct nbrec_load_balancer_group *, struct hmap *lbs, + struct hmap *lb_groups); +static void destroy_tracked_data(struct ed_type_lb_data *); +static void add_crupdated_lb_to_tracked_data(struct ovn_northd_lb *, + struct tracked_lb_data *, + bool health_checks); +static void add_deleted_lb_to_tracked_data(struct ovn_northd_lb *, + struct tracked_lb_data *, + bool health_checks); +static struct crupdated_lb_group * + add_crupdated_lb_group_to_tracked_data(struct ovn_lb_group *, + struct tracked_lb_data *); +static void add_deleted_lb_group_to_tracked_data( + struct ovn_lb_group *, struct tracked_lb_data *); +/* 'lb_data' engine node manages the NB load balancers and load balancer + * groups. For each NB LB, it creates 'struct ovn_northd_lb' and + * for each NB LB group, it creates 'struct ovn_lb_group' and stores in + * the respective hmaps in it data (ed_type_lb_data). + */ void * en_lb_data_init(struct engine_node *node OVS_UNUSED, struct engine_arg *arg OVS_UNUSED) @@ -60,6 +81,7 @@ en_lb_data_run(struct engine_node *node, void *data) const struct nbrec_load_balancer_group_table *nb_lbg_table = EN_OVSDB_GET(engine_get_input("NB_load_balancer_group", node)); + lb_data->tracked = false; build_lbs(nb_lb_table, nb_lbg_table, &lb_data->lbs, &lb_data->lb_groups); engine_set_node_state(node, EN_UPDATED); } @@ -71,12 +93,155 @@ en_lb_data_cleanup(void *data) lb_data_destroy(lb_data); } +void +en_lb_data_clear_tracked_data(void *data) +{ + struct ed_type_lb_data *lb_data = (struct ed_type_lb_data *) data; + destroy_tracked_data(lb_data); +} + + +/* Handler functions. */ +bool +lb_data_load_balancer_handler(struct engine_node *node, void *data) +{ + const struct nbrec_load_balancer_table *nb_lb_table = + EN_OVSDB_GET(engine_get_input("NB_load_balancer", node)); + + struct ed_type_lb_data *lb_data = (struct ed_type_lb_data *) data; + + lb_data->tracked = true; + struct tracked_lb_data *trk_lb_data = &lb_data->tracked_lb_data; + + const struct nbrec_load_balancer *tracked_lb; + NBREC_LOAD_BALANCER_TABLE_FOR_EACH_TRACKED (tracked_lb, nb_lb_table) { + struct ovn_northd_lb *lb; + if (nbrec_load_balancer_is_new(tracked_lb)) { + /* New load balancer. */ + lb = ovn_northd_lb_create(tracked_lb); + hmap_insert(&lb_data->lbs, &lb->hmap_node, + uuid_hash(&tracked_lb->header_.uuid)); + add_crupdated_lb_to_tracked_data(lb, trk_lb_data, + lb->health_checks); + } else if (nbrec_load_balancer_is_deleted(tracked_lb)) { + lb = ovn_northd_lb_find(&lb_data->lbs, + &tracked_lb->header_.uuid); + ovs_assert(lb); + hmap_remove(&lb_data->lbs, &lb->hmap_node); + add_deleted_lb_to_tracked_data(lb, trk_lb_data, + lb->health_checks); + } else { + /* Load balancer updated. */ + lb = ovn_northd_lb_find(&lb_data->lbs, + &tracked_lb->header_.uuid); + ovs_assert(lb); + bool health_checks = lb->health_checks; + ovn_northd_lb_reinit(lb, tracked_lb); + health_checks |= lb->health_checks; + add_crupdated_lb_to_tracked_data(lb, trk_lb_data, health_checks); + } + } + + engine_set_node_state(node, EN_UPDATED); + return true; +} + +bool +lb_data_load_balancer_group_handler(struct engine_node *node, void *data) +{ + struct ed_type_lb_data *lb_data = (struct ed_type_lb_data *) data; + const struct nbrec_load_balancer_group_table *nb_lbg_table = + EN_OVSDB_GET(engine_get_input("NB_load_balancer_group", node)); + + lb_data->tracked = true; + struct tracked_lb_data *trk_lb_data = &lb_data->tracked_lb_data; + const struct nbrec_load_balancer_group *tracked_lb_group; + NBREC_LOAD_BALANCER_GROUP_TABLE_FOR_EACH_TRACKED (tracked_lb_group, + nb_lbg_table) { + if (nbrec_load_balancer_group_is_new(tracked_lb_group)) { + struct ovn_lb_group *lb_group = + create_lb_group(tracked_lb_group, &lb_data->lbs, + &lb_data->lb_groups); + struct crupdated_lb_group *clbg = + add_crupdated_lb_group_to_tracked_data(lb_group, trk_lb_data); + for (size_t i = 0; i < lb_group->n_lbs; i++) { + hmapx_add(&clbg->assoc_lbs, lb_group->lbs[i]); + } + } else if (nbrec_load_balancer_group_is_deleted(tracked_lb_group)) { + struct ovn_lb_group *lb_group; + lb_group = ovn_lb_group_find(&lb_data->lb_groups, + &tracked_lb_group->header_.uuid); + ovs_assert(lb_group); + hmap_remove(&lb_data->lb_groups, &lb_group->hmap_node); + add_deleted_lb_group_to_tracked_data(lb_group, trk_lb_data); + } else { + + struct ovn_lb_group *lb_group; + lb_group = ovn_lb_group_find(&lb_data->lb_groups, + &tracked_lb_group->header_.uuid); + ovs_assert(lb_group); + + /* Determine the lbs which are added or deleted for this + * lb group and add them to tracked data. + * Eg. If an lb group lbg1 before the update had [lb1, lb2, lb3] + * And in the update, lb2 was removed and lb4 got added, then + * add lb2 and lb4 to the trk_lb_data->crupdated_lbs. */ + struct hmapx pre_update_lbs = HMAPX_INITIALIZER(&pre_update_lbs); + for (size_t i = 0; i < lb_group->n_lbs; i++) { + hmapx_add(&pre_update_lbs, lb_group->lbs[i]); + } + ovn_lb_group_reinit(lb_group, tracked_lb_group, &lb_data->lbs); + for (size_t i = 0; i < lb_group->n_lbs; i++) { + build_lrouter_lb_ips(lb_group->lb_ips, lb_group->lbs[i]); + } + + struct crupdated_lb_group *clbg = + add_crupdated_lb_group_to_tracked_data(lb_group, trk_lb_data); + + for (size_t i = 0; i < lb_group->n_lbs; i++) { + struct ovn_northd_lb *lb = lb_group->lbs[i]; + struct hmapx_node *hmapx_node = hmapx_find(&pre_update_lbs, + lb); + if (!hmapx_node) { + hmapx_add(&clbg->assoc_lbs, lb); + } else { + hmapx_delete(&pre_update_lbs, hmapx_node); + } + } + + struct hmapx_node *hmapx_node; + HMAPX_FOR_EACH_SAFE (hmapx_node, &pre_update_lbs) { + struct ovn_northd_lb *lb = hmapx_node->data; + /* Check if the pre updated lb is actually deleted or + * just disassociated from the lb group. If it's just + * disassociated, then set 'has_dissassoc_lbs_from_lb_grops' to + * true. Later if required we can add this 'lb' to an hmapx of + * disassociated_lbs. */ + if (!hmapx_find(&trk_lb_data->deleted_lbs, lb)) { + trk_lb_data->has_dissassoc_lbs_from_lb_grops = true; + } + hmapx_delete(&pre_update_lbs, hmapx_node); + } + hmapx_destroy(&pre_update_lbs); + } + } + + engine_set_node_state(node, EN_UPDATED); + return true; +} + /* static functions. */ static void lb_data_init(struct ed_type_lb_data *lb_data) { hmap_init(&lb_data->lbs); hmap_init(&lb_data->lb_groups); + + struct tracked_lb_data *trk_lb_data = &lb_data->tracked_lb_data; + hmap_init(&trk_lb_data->crupdated_lbs); + hmapx_init(&trk_lb_data->deleted_lbs); + hmap_init(&trk_lb_data->crupdated_lb_groups); + hmapx_init(&trk_lb_data->deleted_lb_groups); } static void @@ -93,6 +258,12 @@ lb_data_destroy(struct ed_type_lb_data *lb_data) ovn_lb_group_destroy(lb_group); } hmap_destroy(&lb_data->lb_groups); + + destroy_tracked_data(lb_data); + hmap_destroy(&lb_data->tracked_lb_data.crupdated_lbs); + hmapx_destroy(&lb_data->tracked_lb_data.deleted_lbs); + hmapx_destroy(&lb_data->tracked_lb_data.deleted_lb_groups); + hmap_destroy(&lb_data->tracked_lb_data.crupdated_lb_groups); } static void @@ -100,12 +271,9 @@ build_lbs(const struct nbrec_load_balancer_table *nbrec_load_balancer_table, const struct nbrec_load_balancer_group_table *nbrec_lb_group_table, struct hmap *lbs, struct hmap *lb_groups) { - struct ovn_lb_group *lb_group; - struct ovn_northd_lb *lb_nb; - const struct nbrec_load_balancer *nbrec_lb; NBREC_LOAD_BALANCER_TABLE_FOR_EACH (nbrec_lb, nbrec_load_balancer_table) { - lb_nb = ovn_northd_lb_create(nbrec_lb); + struct ovn_northd_lb *lb_nb = ovn_northd_lb_create(nbrec_lb); hmap_insert(lbs, &lb_nb->hmap_node, uuid_hash(&nbrec_lb->header_.uuid)); } @@ -113,13 +281,98 @@ build_lbs(const struct nbrec_load_balancer_table *nbrec_load_balancer_table, const struct nbrec_load_balancer_group *nbrec_lb_group; NBREC_LOAD_BALANCER_GROUP_TABLE_FOR_EACH (nbrec_lb_group, nbrec_lb_group_table) { - lb_group = ovn_lb_group_create(nbrec_lb_group, lbs); + create_lb_group(nbrec_lb_group, lbs, lb_groups); + } +} - for (size_t i = 0; i < lb_group->n_lbs; i++) { - build_lrouter_lb_ips(lb_group->lb_ips, lb_group->lbs[i]); - } +static struct ovn_lb_group * +create_lb_group(const struct nbrec_load_balancer_group *nbrec_lb_group, + struct hmap *lbs, struct hmap *lb_groups) +{ + struct ovn_lb_group *lb_group = ovn_lb_group_create(nbrec_lb_group, lbs); - hmap_insert(lb_groups, &lb_group->hmap_node, - uuid_hash(&lb_group->uuid)); + for (size_t i = 0; i < lb_group->n_lbs; i++) { + build_lrouter_lb_ips(lb_group->lb_ips, lb_group->lbs[i]); } + + hmap_insert(lb_groups, &lb_group->hmap_node, + uuid_hash(&lb_group->uuid)); + + return lb_group; +} + +static void +destroy_tracked_data(struct ed_type_lb_data *lb_data) +{ + lb_data->tracked = false; + lb_data->tracked_lb_data.has_health_checks = false; + lb_data->tracked_lb_data.has_dissassoc_lbs_from_lb_grops = false; + + struct hmapx_node *node; + HMAPX_FOR_EACH_SAFE (node, &lb_data->tracked_lb_data.deleted_lbs) { + ovn_northd_lb_destroy(node->data); + hmapx_delete(&lb_data->tracked_lb_data.deleted_lbs, node); + } + + HMAPX_FOR_EACH_SAFE (node, &lb_data->tracked_lb_data.deleted_lb_groups) { + ovn_lb_group_destroy(node->data); + hmapx_delete(&lb_data->tracked_lb_data.deleted_lb_groups, node); + } + + struct crupdated_lb *clb; + HMAP_FOR_EACH_POP (clb, hmap_node, + &lb_data->tracked_lb_data.crupdated_lbs) { + free(clb); + } + + struct crupdated_lb_group *crupdated_lbg; + HMAP_FOR_EACH_POP (crupdated_lbg, hmap_node, + &lb_data->tracked_lb_data.crupdated_lb_groups) { + hmapx_destroy(&crupdated_lbg->assoc_lbs); + free(crupdated_lbg); + } +} + +static void +add_crupdated_lb_to_tracked_data(struct ovn_northd_lb *lb, + struct tracked_lb_data *tracked_lb_data, + bool health_checks) +{ + struct crupdated_lb *clb = xzalloc(sizeof *clb); + clb->lb = lb; + hmap_insert(&tracked_lb_data->crupdated_lbs, &clb->hmap_node, + uuid_hash(&lb->nlb->header_.uuid)); + if (health_checks) { + tracked_lb_data->has_health_checks = true; + } +} + +static void +add_deleted_lb_to_tracked_data(struct ovn_northd_lb *lb, + struct tracked_lb_data *tracked_lb_data, + bool health_checks) +{ + hmapx_add(&tracked_lb_data->deleted_lbs, lb); + if (health_checks) { + tracked_lb_data->has_health_checks = true; + } +} + +static struct crupdated_lb_group * +add_crupdated_lb_group_to_tracked_data(struct ovn_lb_group *lbg, + struct tracked_lb_data *tracked_lb_data) +{ + struct crupdated_lb_group *clbg = xzalloc(sizeof *clbg); + clbg->lbg = lbg; + hmapx_init(&clbg->assoc_lbs); + hmap_insert(&tracked_lb_data->crupdated_lb_groups, &clbg->hmap_node, + uuid_hash(&lbg->uuid)); + return clbg; +} + +static void +add_deleted_lb_group_to_tracked_data(struct ovn_lb_group *lbg, + struct tracked_lb_data *tracked_lb_data) +{ + hmapx_add(&tracked_lb_data->deleted_lb_groups, lbg); } diff --git a/northd/en-lb-data.h b/northd/en-lb-data.h index 96fb8c1f8d..2e9a024620 100644 --- a/northd/en-lb-data.h +++ b/northd/en-lb-data.h @@ -4,9 +4,51 @@ #include #include "openvswitch/hmap.h" +#include "include/openvswitch/list.h" +#include "lib/hmapx.h" #include "lib/inc-proc-eng.h" +struct ovn_northd_lb; +struct ovn_lb_group; + +struct crupdated_lb { + struct hmap_node hmap_node; + + struct ovn_northd_lb *lb; +}; + +struct crupdated_lb_group { + struct hmap_node hmap_node; + + struct ovn_lb_group *lbg; + /* hmapx of newly associated lbs to this lb group. + * hmapx node is 'struct ovn_northd_lb *' */ + struct hmapx assoc_lbs; +}; + +struct tracked_lb_data { + /* Both created and updated lbs. hmapx node is 'struct crupdated_lb *'. */ + struct hmap crupdated_lbs; + + /* Deleted lbs. */ + struct hmapx deleted_lbs; + + /* Both created and updated lb_groups. hmap node is + * 'struct crupdated_lb_group'. */ + struct hmap crupdated_lb_groups; + + /* Deleted lb_groups. hmapx node is 'struct ovn_lb_group *'. */ + struct hmapx deleted_lb_groups; + + /* Indicates if any of the tracked lb has health checks enabled. */ + bool has_health_checks; + + /* Indicates if any lb got disassociated from a lb group + * but not deleted. */ + bool has_dissassoc_lbs_from_lb_grops; +}; + /* struct which maintains the data of the engine node lb_data. */ struct ed_type_lb_data { /* hmap of load balancers. hmap node is 'struct ovn_northd_lb *' */ @@ -14,10 +56,18 @@ struct ed_type_lb_data { /* hmap of load balancer groups. hmap node is 'struct ovn_lb_group *' */ struct hmap lb_groups; + + /* tracked data*/ + bool tracked; + struct tracked_lb_data tracked_lb_data; }; void *en_lb_data_init(struct engine_node *, struct engine_arg *); void en_lb_data_run(struct engine_node *, void *data); void en_lb_data_cleanup(void *data); +void en_lb_data_clear_tracked_data(void *data); + +bool lb_data_load_balancer_handler(struct engine_node *, void *data); +bool lb_data_load_balancer_group_handler(struct engine_node *, void *data); #endif /* end of EN_NORTHD_LB_DATA_H */ diff --git a/northd/en-northd.c b/northd/en-northd.c index a0dae65752..d59ef062df 100644 --- a/northd/en-northd.c +++ b/northd/en-northd.c @@ -206,6 +206,47 @@ northd_sb_port_binding_handler(struct engine_node *node, return true; } +bool +northd_lb_data_handler(struct engine_node *node, void *data) +{ + struct ed_type_lb_data *lb_data = engine_get_input_data("lb_data", node); + + if (!lb_data->tracked) { + return false; + } + + if (lb_data->tracked_lb_data.has_health_checks) { + /* Fall back to recompute since a tracked load balancer + * has health checks configured as I-P is not yet supported + * for such load balancers. */ + return false; + } + + /* Fall back to recompute if any load balancer was dissociated from + * a load balancer group (but not deleted). */ + if (lb_data->tracked_lb_data.has_dissassoc_lbs_from_lb_grops) { + return false; + } + + /* Fall back to recompute if load balancer groups are deleted. */ + if (!hmapx_is_empty(&lb_data->tracked_lb_data.deleted_lb_groups)) { + return false; + } + + struct northd_data *nd = data; + + if (!northd_handle_lb_data_changes(&lb_data->tracked_lb_data, + &nd->ls_datapaths, + &nd->lr_datapaths, + &nd->lb_datapaths_map, + &nd->lb_group_datapaths_map)) { + return false; + } + + engine_set_node_state(node, EN_UPDATED); + return true; +} + void *en_northd_init(struct engine_node *node OVS_UNUSED, struct engine_arg *arg OVS_UNUSED) diff --git a/northd/en-northd.h b/northd/en-northd.h index 20cc77f108..3c77b64bb2 100644 --- a/northd/en-northd.h +++ b/northd/en-northd.h @@ -17,5 +17,6 @@ void en_northd_clear_tracked_data(void *data); bool northd_nb_nb_global_handler(struct engine_node *, void *data OVS_UNUSED); bool northd_nb_logical_switch_handler(struct engine_node *, void *data); bool northd_sb_port_binding_handler(struct engine_node *, void *data); +bool northd_lb_data_handler(struct engine_node *, void *data); #endif /* EN_NORTHD_H */ diff --git a/northd/inc-proc-northd.c b/northd/inc-proc-northd.c index 0a99e87bc2..75d059645f 100644 --- a/northd/inc-proc-northd.c +++ b/northd/inc-proc-northd.c @@ -141,13 +141,18 @@ static ENGINE_NODE(sync_to_sb_addr_set, "sync_to_sb_addr_set"); static ENGINE_NODE(fdb_aging, "fdb_aging"); static ENGINE_NODE(fdb_aging_waker, "fdb_aging_waker"); static ENGINE_NODE(sync_to_sb_lb, "sync_to_sb_lb"); -static ENGINE_NODE(lb_data, "lb_data"); +static ENGINE_NODE_WITH_CLEAR_TRACK_DATA(lb_data, "lb_data"); void inc_proc_northd_init(struct ovsdb_idl_loop *nb, struct ovsdb_idl_loop *sb) { /* Define relationships between nodes where first argument is dependent * on the second argument */ + engine_add_input(&en_lb_data, &en_nb_load_balancer, + lb_data_load_balancer_handler); + engine_add_input(&en_lb_data, &en_nb_load_balancer_group, + lb_data_load_balancer_group_handler); + engine_add_input(&en_northd, &en_nb_port_group, NULL); engine_add_input(&en_northd, &en_nb_acl, NULL); engine_add_input(&en_northd, &en_nb_logical_router, NULL); @@ -175,13 +180,10 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, northd_sb_port_binding_handler); engine_add_input(&en_northd, &en_nb_nb_global, northd_nb_nb_global_handler); + engine_add_input(&en_northd, &en_lb_data, northd_lb_data_handler); engine_add_input(&en_northd, &en_nb_logical_switch, northd_nb_logical_switch_handler); - engine_add_input(&en_lb_data, &en_nb_load_balancer, NULL); - engine_add_input(&en_lb_data, &en_nb_load_balancer_group, NULL); - engine_add_input(&en_northd, &en_lb_data, NULL); - engine_add_input(&en_mac_binding_aging, &en_nb_nb_global, NULL); engine_add_input(&en_mac_binding_aging, &en_sb_mac_binding, NULL); engine_add_input(&en_mac_binding_aging, &en_northd, NULL); diff --git a/northd/northd.c b/northd/northd.c index 78b82950e4..e9afa07177 100644 --- a/northd/northd.c +++ b/northd/northd.c @@ -40,6 +40,7 @@ #include "lib/lb.h" #include "memory.h" #include "northd.h" +#include "en-lb-data.h" #include "lib/ovn-parallel-hmap.h" #include "ovn/actions.h" #include "ovn/features.h" @@ -5362,6 +5363,64 @@ northd_handle_sb_port_binding_changes( return true; } +/* Handler for lb_data engine changes. For every tracked lb_data + * it creates or deletes the ovn_lb_datapaths/ovn_lb_group_datapaths + * from the lb_datapaths hmap and lb_group_datapaths hmap. */ +bool +northd_handle_lb_data_changes(struct tracked_lb_data *trk_lb_data, + struct ovn_datapaths *ls_datapaths, + struct ovn_datapaths *lr_datapaths, + struct hmap *lb_datapaths_map, + struct hmap *lb_group_datapaths_map) +{ + struct ovn_lb_datapaths *lb_dps; + struct ovn_northd_lb *lb; + struct hmapx_node *hmapx_node; + HMAPX_FOR_EACH (hmapx_node, &trk_lb_data->deleted_lbs) { + lb = hmapx_node->data; + const struct uuid *lb_uuid = &lb->nlb->header_.uuid; + + lb_dps = ovn_lb_datapaths_find(lb_datapaths_map, lb_uuid); + ovs_assert(lb_dps); + hmap_remove(lb_datapaths_map, &lb_dps->hmap_node); + ovn_lb_datapaths_destroy(lb_dps); + } + + struct crupdated_lb *clb; + HMAP_FOR_EACH (clb, hmap_node, &trk_lb_data->crupdated_lbs) { + lb = clb->lb; + const struct uuid *lb_uuid = &lb->nlb->header_.uuid; + + lb_dps = ovn_lb_datapaths_find(lb_datapaths_map, lb_uuid); + if (!lb_dps) { + lb_dps = ovn_lb_datapaths_create(lb, ods_size(ls_datapaths), + ods_size(lr_datapaths)); + hmap_insert(lb_datapaths_map, &lb_dps->hmap_node, + uuid_hash(lb_uuid)); + } + } + + struct ovn_lb_group_datapaths *lb_group_dps; + struct ovn_lb_group *lbg; + struct crupdated_lb_group *crupdated_lbg; + HMAP_FOR_EACH (crupdated_lbg, hmap_node, + &trk_lb_data->crupdated_lb_groups) { + lbg = crupdated_lbg->lbg; + const struct uuid *lb_uuid = &lbg->uuid; + + lb_group_dps = ovn_lb_group_datapaths_find(lb_group_datapaths_map, + lb_uuid); + if (!lb_group_dps) { + lb_group_dps = ovn_lb_group_datapaths_create( + lbg, ods_size(ls_datapaths), ods_size(lr_datapaths)); + hmap_insert(lb_group_datapaths_map, &lb_group_dps->hmap_node, + uuid_hash(lb_uuid)); + } + } + + return true; +} + struct multicast_group { const char *name; uint16_t key; /* OVN_MIN_MULTICAST...OVN_MAX_MULTICAST. */ diff --git a/northd/northd.h b/northd/northd.h index 7d92028c7d..7d17921fa2 100644 --- a/northd/northd.h +++ b/northd/northd.h @@ -347,6 +347,13 @@ bool lflow_handle_northd_ls_changes(struct ovsdb_idl_txn *ovnsb_txn, bool northd_handle_sb_port_binding_changes( const struct sbrec_port_binding_table *, struct hmap *ls_ports); +struct tracked_lb_data; +bool northd_handle_lb_data_changes(struct tracked_lb_data *, + struct ovn_datapaths *ls_datapaths, + struct ovn_datapaths *lr_datapaths, + struct hmap *lb_datapaths_map, + struct hmap *lb_group_datapaths_map); + void build_bfd_table(struct ovsdb_idl_txn *ovnsb_txn, const struct nbrec_bfd_table *, const struct sbrec_bfd_table *, diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at index 9e36f96182..213fd51d77 100644 --- a/tests/ovn-northd.at +++ b/tests/ovn-northd.at @@ -9751,3 +9751,460 @@ AT_CHECK([grep "lr_in_gw_redirect" R1flows |sed s'/table=../table=??/' |sort], [ AT_CLEANUP ]) + +OVN_FOR_EACH_NORTHD_NO_HV([ +AT_SETUP([Load balancer incremental processing]) +ovn_start + +check_engine_stats() { + node=$1 + recompute=$2 + compute=$3 + + echo "__file__:__line__: Checking engine stats for node $node : recompute - \ +$recompute : compute - $compute" + + node_stat=$(as northd ovn-appctl -t NORTHD_TYPE inc-engine/show-stats $node) + # node_stat will be of this format : + # - Node: lflow - recompute: 3 - compute: 0 - abort: 0 + node_recompute_ct=$(echo $node_stat | cut -d '-' -f2 | cut -d ':' -f2) + node_compute_ct=$(echo $node_stat | cut -d '-' -f3 | cut -d ':' -f2) + + if [[ "$recompute" == "norecompute" ]]; then + # node should not be recomputed + echo "Expecting $node recompute count - $node_recompute_ct to be 0" + check test "$node_recompute_ct" -eq "0" + else + echo "Expecting $node recompute count - $node_recompute_ct not to be 0" + check test "$node_recompute_ct" -ne "0" + fi + + if [[ "$compute" == "nocompute" ]]; then + # node should not be computed + echo "Expecting $node compute count - $node_compute_ct to be 0" + check test "$node_compute_ct" -eq "0" + else + echo "Expecting $node compute count - $node_compute_ct not to be 0" + check test "$node_compute_ct" -ne "0" + fi +} + +# Test I-P for load balancers. +# Presently ovn-northd handles I-P for NB LBs in northd_lb_data engine node +# only. +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb lb-add lb1 10.0.0.10:80 10.0.0.3:80 + +check_engine_stats lb_data norecompute compute +check_engine_stats northd norecompute compute +check_engine_stats lflow recompute nocompute + +CHECK_NO_CHANGE_AFTER_RECOMPUTE +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats + +check ovn-nbctl --wait=sb set load_balancer . ip_port_mappings:10.0.0.3=sw0-p1:10.0.0.2 +check_engine_stats lb_data norecompute compute +check_engine_stats northd norecompute compute +check_engine_stats lflow recompute nocompute + +check ovn-nbctl --wait=sb set load_balancer . options:foo=bar +check_engine_stats lb_data norecompute compute +check_engine_stats northd norecompute compute +check_engine_stats lflow recompute nocompute + +check ovn-nbctl --wait=sb -- lb-add lb2 20.0.0.10:80 20.0.0.20:80 -- lb-add lb3 30.0.0.10:80 30.0.0.20:80 +check_engine_stats lb_data norecompute compute +check_engine_stats northd norecompute compute +check_engine_stats lflow recompute nocompute + +CHECK_NO_CHANGE_AFTER_RECOMPUTE +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats + +check ovn-nbctl --wait=sb -- lb-del lb2 -- lb-del lb3 +check_engine_stats lb_data norecompute compute +check_engine_stats northd norecompute compute +check_engine_stats lflow recompute nocompute + +CHECK_NO_CHANGE_AFTER_RECOMPUTE +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats + +AT_CHECK([ovn-nbctl --wait=sb \ + -- --id=@hc create Load_Balancer_Health_Check vip="10.0.0.10\:80" \ + options:failure_count=100 \ + -- add Load_Balancer . health_check @hc | uuidfilt], [0], [<0> +]) +check_engine_stats lb_data norecompute compute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute + +# Any change to load balancer health check should also result in full recompute +# of northd node (but not northd_lb_data node) +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb set load_balancer_health_check . options:foo=bar1 +check_engine_stats lb_data norecompute compute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute + +# Delete the health check from the load balancer. northd engine node should do a full recompute. +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb clear Load_Balancer . health_check +check_engine_stats lb_data norecompute compute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute + +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl ls-add sw0 +check ovn-nbctl --wait=sb lr-add lr0 +ovn-nbctl lrp-add lr0 lr0-sw0 00:00:00:00:ff:01 10.0.0.1/24 +ovn-nbctl lsp-add sw0 sw0-lr0 +ovn-nbctl lsp-set-type sw0-lr0 router +ovn-nbctl lsp-set-addresses sw0-lr0 00:00:00:00:ff:01 +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +ovn-nbctl --wait=sb lsp-set-options sw0-lr0 router-port=lr0-sw0 +check_engine_stats lb_data norecompute nocompute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute + +# Associate lb1 to sw0. There should be a full recompute of northd engine node +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb ls-lb-add sw0 lb1 +check_engine_stats lb_data norecompute nocompute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute + +# Modify the backend of the lb1 vip +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb set load_balancer lb1 vips:'"10.0.0.10:80"'='"10.0.0.100:80"' +check_engine_stats lb_data norecompute compute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +# Cleanup the vip of lb1. +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb clear load_Balancer lb1 vips +check_engine_stats lb_data norecompute compute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +# Set the vips of lb1 back +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb lb-add lb1 10.0.0.10:80 10.0.0.3:80 +check_engine_stats lb_data norecompute compute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +# Add another vip to lb1 +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb lb-add lb1 10.0.0.20:80 10.0.0.30:8080 +check_engine_stats lb_data norecompute compute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +# Disassociate lb1 from sw0. There should be a full recompute of northd engine node. +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb ls-lb-del sw0 lb1 +check_engine_stats lb_data norecompute nocompute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +# Add lb1 to lr0 and then disassociate +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb lr-lb-add lr0 lb1 +check_engine_stats lb_data norecompute nocompute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +# Modify the backend of the lb1 vip +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb set load_balancer lb1 vips:'"10.0.0.10:80"'='"10.0.0.100:80"' +check_engine_stats lb_data norecompute compute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +# Cleanup the vip of lb1. +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb clear load_Balancer lb1 vips +check_engine_stats lb_data norecompute compute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +# Set the vips of lb1 back +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb lb-add lb1 10.0.0.10:80 10.0.0.3:80 +check_engine_stats lb_data norecompute compute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +# Add another vip to lb1 +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb lb-add lb1 10.0.0.20:80 10.0.0.30:8080 +check_engine_stats lb_data norecompute compute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb lr-lb-del lr0 lb1 +check_engine_stats lb_data norecompute nocompute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +# Test load balancer group now +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +lbg1_uuid=$(ovn-nbctl create load_balancer_group name=lbg1) +check_engine_stats lb_data norecompute compute +check_engine_stats northd norecompute compute +check_engine_stats lflow recompute nocompute + +CHECK_NO_CHANGE_AFTER_RECOMPUTE +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats + +lb1_uuid=$(fetch_column nb:Load_Balancer _uuid) + +# Add lb to the lbg1 group +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl add load_balancer_group . load_Balancer $lb1_uuid +check_engine_stats lb_data norecompute compute +check_engine_stats northd norecompute compute +check_engine_stats lflow recompute nocompute + +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl clear load_balancer_group . load_Balancer +check_engine_stats lb_data norecompute compute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute + +# Add back lb to the lbg1 group +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl add load_balancer_group . load_Balancer $lb1_uuid +check_engine_stats lb_data norecompute compute +check_engine_stats northd norecompute compute +check_engine_stats lflow recompute nocompute + +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl add logical_switch sw0 load_balancer_group $lbg1_uuid +check_engine_stats lb_data norecompute nocompute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute + +# Update lb and this should result in recompute +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb set load_balancer . options:bar=foo +check_engine_stats lb_data norecompute compute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute + +# Modify the backend of the lb1 vip +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb set load_balancer lb1 vips:'"10.0.0.10:80"'='"10.0.0.100:80"' +check_engine_stats lb_data norecompute compute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +# Cleanup the vip of lb1. +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb clear load_Balancer lb1 vips +check_engine_stats lb_data norecompute compute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +# Set the vips of lb1 back +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb lb-add lb1 10.0.0.10:80 10.0.0.3:80 +check_engine_stats lb_data norecompute compute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +# Add another vip to lb1 +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb lb-add lb1 10.0.0.20:80 10.0.0.30:8080 +check_engine_stats lb_data norecompute compute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl clear logical_switch sw0 load_balancer_group +check_engine_stats lb_data norecompute nocompute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute + +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl add logical_router lr0 load_balancer_group $lbg1_uuid +check_engine_stats lb_data norecompute nocompute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +# Modify the backend of the lb1 vip +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb set load_balancer lb1 vips:'"10.0.0.10:80"'='"10.0.0.100:80"' +check_engine_stats lb_data norecompute compute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +# Cleanup the vip of lb1. +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb clear load_Balancer lb1 vips +check_engine_stats lb_data norecompute compute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +# Set the vips of lb1 back +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb lb-add lb1 10.0.0.10:80 10.0.0.3:80 +check_engine_stats lb_data norecompute compute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +# Add another vip to lb1 +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb lb-add lb1 10.0.0.20:80 10.0.0.30:8080 +check_engine_stats lb_data norecompute compute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl clear logical_router lr0 load_balancer_group +check_engine_stats lb_data norecompute nocompute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute + +# Add back lb group to logical switch and then delete it. +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl add logical_switch sw0 load_balancer_group $lbg1_uuid +check_engine_stats lb_data norecompute nocompute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute + +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl clear logical_switch sw0 load_balancer_group -- \ + destroy load_balancer_group $lbg1_uuid +check_engine_stats lb_data norecompute compute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute + +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +# Test the scenario where a load balancer is associated to +# a logical switch sw0 and also to a lb group lbg1 and lbg1 +# is also associated to the logical switch sw0 and logical +# router lr1 + +check ovn-nbctl lr-add lr1 +check ovn-nbctl lb-add lb2 20.0.0.20:80 30.0.0.30:8080 +check ovn-nbctl lb-add lb3 30.0.0.20:80 30.0.0.30:8080 +check ovn-nbctl --wait=sb lb-add lb4 40.0.0.20:80 30.0.0.30:8080 + +lb2_uuid=$(fetch_column nb:Load_Balancer _uuid name=lb2) +lb3_uuid=$(fetch_column nb:Load_Balancer _uuid name=lb3) +lb4_uuid=$(fetch_column nb:Load_Balancer _uuid name=lb4) + +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +lbg1_uuid=$(ovn-nbctl create load_balancer_group name=lbg1) +check_engine_stats lb_data norecompute compute +check_engine_stats northd norecompute compute +check_engine_stats lflow recompute nocompute + +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl set load_balancer_group . load_balancer="$lb2_uuid,$lb3_uuid,$lb4_uuid" +check_engine_stats lb_data norecompute compute +check_engine_stats northd norecompute compute +check_engine_stats lflow recompute nocompute + +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl set logical_switch sw0 load_balancer_group=$lbg1_uuid +check_engine_stats lb_data norecompute nocompute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl set logical_router lr1 load_balancer_group=$lbg1_uuid +check_engine_stats lb_data norecompute nocompute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb ls-lb-add sw0 lb2 +check_engine_stats lb_data norecompute nocompute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb ls-lb-add sw0 lb3 +check_engine_stats lb_data norecompute nocompute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb lr-lb-add lr1 lb1 +check ovn-nbctl --wait=sb lr-lb-add lr1 lb2 +check_engine_stats lb_data norecompute nocompute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb ls-lb-del sw0 lb2 +check_engine_stats lb_data norecompute nocompute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb lr-lb-del lr1 lb2 +check_engine_stats lb_data norecompute nocompute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +# Deleting lb4 should not result in lflow recompute as it is +# only associated with logical switch sw0. +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb lb-del lb4 +check_engine_stats lb_data norecompute compute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +# Deleting lb2 should result in lflow recompute as it is +# associated with logical router lr1 through lb group. +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb lb-del lb2 +check_engine_stats lb_data norecompute compute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb remove load_balancer_group . load_balancer $lb3_uuid +check_engine_stats lb_data norecompute compute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +AT_CLEANUP +]) From patchwork Fri Aug 18 08:57:36 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Numan Siddique X-Patchwork-Id: 1822754 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::136; helo=smtp3.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4RRwkW1kfvz1ygH for ; Fri, 18 Aug 2023 18:57:55 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 7AD706158B; Fri, 18 Aug 2023 08:57:53 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 7AD706158B X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id CLjSHQWsIUrB; Fri, 18 Aug 2023 08:57:52 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp3.osuosl.org (Postfix) with ESMTPS id 287AA6158F; Fri, 18 Aug 2023 08:57:51 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 287AA6158F Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id E9EC8C0039; Fri, 18 Aug 2023 08:57:50 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp1.osuosl.org (smtp1.osuosl.org [IPv6:2605:bc80:3010::138]) by lists.linuxfoundation.org (Postfix) with ESMTP id 68DDFC0032 for ; Fri, 18 Aug 2023 08:57:49 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 2F20B83F23 for ; Fri, 18 Aug 2023 08:57:45 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 2F20B83F23 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id DtcTgPLADXxL for ; Fri, 18 Aug 2023 08:57:44 +0000 (UTC) Received: from relay6-d.mail.gandi.net (relay6-d.mail.gandi.net [IPv6:2001:4b98:dc4:8::226]) by smtp1.osuosl.org (Postfix) with ESMTPS id AD31D83EBF for ; Fri, 18 Aug 2023 08:57:43 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org AD31D83EBF Received: by mail.gandi.net (Postfix) with ESMTPSA id 88C6AC0002; Fri, 18 Aug 2023 08:57:40 +0000 (UTC) From: numans@ovn.org To: dev@openvswitch.org Date: Fri, 18 Aug 2023 14:27:36 +0530 Message-Id: <20230818085736.1030963-1-numans@ovn.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230818085606.1030792-1-numans@ovn.org> References: <20230818085606.1030792-1-numans@ovn.org> MIME-Version: 1.0 X-GND-Sasl: numans@ovn.org Subject: [ovs-dev] [PATCH ovn v6 04/16] northd: Refactor the 'northd' node code which handles logical switch changes. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Numan Siddique This will help in handling other column changes of a logical switch. Acked-by: Han Zhou Signed-off-by: Numan Siddique Reviewed-by: Ales Musil --- northd/en-lb-data.c | 2 +- northd/northd.c | 356 ++++++++++++++++++++++++++------------------ 2 files changed, 209 insertions(+), 149 deletions(-) diff --git a/northd/en-lb-data.c b/northd/en-lb-data.c index 328c003675..23f2cb1021 100644 --- a/northd/en-lb-data.c +++ b/northd/en-lb-data.c @@ -58,7 +58,7 @@ static void add_deleted_lb_group_to_tracked_data( /* 'lb_data' engine node manages the NB load balancers and load balancer * groups. For each NB LB, it creates 'struct ovn_northd_lb' and * for each NB LB group, it creates 'struct ovn_lb_group' and stores in - * the respective hmaps in it data (ed_type_lb_data). + * the respective hmaps in it's data (ed_type_lb_data). */ void * en_lb_data_init(struct engine_node *node OVS_UNUSED, diff --git a/northd/northd.c b/northd/northd.c index e9afa07177..4cc9ef8c8d 100644 --- a/northd/northd.c +++ b/northd/northd.c @@ -4891,23 +4891,29 @@ build_ports(struct ovsdb_idl_txn *ovnsb_txn, sset_destroy(&active_ha_chassis_grps); } +static void +destroy_tracked_ls_change(struct ls_change *ls_change) +{ + struct ovn_port *op; + LIST_FOR_EACH (op, list, &ls_change->added_ports) { + ovs_list_remove(&op->list); + } + LIST_FOR_EACH (op, list, &ls_change->updated_ports) { + ovs_list_remove(&op->list); + } + LIST_FOR_EACH_SAFE (op, list, &ls_change->deleted_ports) { + ovs_list_remove(&op->list); + ovn_port_destroy_orphan(op); + } +} + void destroy_northd_data_tracked_changes(struct northd_data *nd) { struct ls_change *ls_change; LIST_FOR_EACH_SAFE (ls_change, list_node, &nd->tracked_ls_changes.updated) { - struct ovn_port *op; - LIST_FOR_EACH (op, list, &ls_change->added_ports) { - ovs_list_remove(&op->list); - } - LIST_FOR_EACH (op, list, &ls_change->updated_ports) { - ovs_list_remove(&op->list); - } - LIST_FOR_EACH_SAFE (op, list, &ls_change->deleted_ports) { - ovs_list_remove(&op->list); - ovn_port_destroy_orphan(op); - } + destroy_tracked_ls_change(ls_change); ovs_list_remove(&ls_change->list_node); free(ls_change); } @@ -5024,15 +5030,21 @@ ls_port_create(struct ovsdb_idl_txn *ovnsb_txn, struct hmap *ls_ports, return op; } +/* Returns true if the logical switch has changes which can be + * incrementally handled. + * Presently supports i-p for the below changes: + * - logical switch ports. + */ static bool -check_ls_changes_other_than_lsp(const struct nbrec_logical_switch *ls) +ls_changes_can_be_handled( + const struct nbrec_logical_switch *ls) { /* Check if the columns are changed in this row. */ enum nbrec_logical_switch_column_id col; for (col = 0; col < NBREC_LOGICAL_SWITCH_N_COLUMNS; col++) { if (nbrec_logical_switch_is_updated(ls, col) && col != NBREC_LOGICAL_SWITCH_COL_PORTS) { - return true; + return false; } } @@ -5041,44 +5053,44 @@ check_ls_changes_other_than_lsp(const struct nbrec_logical_switch *ls) for (size_t i = 0; i < ls->n_acls; i++) { if (nbrec_acl_row_get_seqno(ls->acls[i], OVSDB_IDL_CHANGE_MODIFY) > 0) { - return true; + return false; } } if (ls->copp && nbrec_copp_row_get_seqno(ls->copp, OVSDB_IDL_CHANGE_MODIFY) > 0) { - return true; + return false; } for (size_t i = 0; i < ls->n_dns_records; i++) { if (nbrec_dns_row_get_seqno(ls->dns_records[i], OVSDB_IDL_CHANGE_MODIFY) > 0) { - return true; + return false; } } for (size_t i = 0; i < ls->n_forwarding_groups; i++) { if (nbrec_forwarding_group_row_get_seqno(ls->forwarding_groups[i], OVSDB_IDL_CHANGE_MODIFY) > 0) { - return true; + return false; } } for (size_t i = 0; i < ls->n_load_balancer; i++) { if (nbrec_load_balancer_row_get_seqno(ls->load_balancer[i], OVSDB_IDL_CHANGE_MODIFY) > 0) { - return true; + return false; } } for (size_t i = 0; i < ls->n_load_balancer_group; i++) { if (nbrec_load_balancer_group_row_get_seqno(ls->load_balancer_group[i], OVSDB_IDL_CHANGE_MODIFY) > 0) { - return true; + return false; } } for (size_t i = 0; i < ls->n_qos_rules; i++) { if (nbrec_qos_row_get_seqno(ls->qos_rules[i], OVSDB_IDL_CHANGE_MODIFY) > 0) { - return true; + return false; } } - return false; + return true; } static bool @@ -5119,6 +5131,172 @@ check_lsp_changes_other_than_up(const struct nbrec_logical_switch_port *nbsp) return false; } +/* Handles logical switch port changes of a changed logical switch. + * Returns false, if any logical port can't be incrementally handled. + */ +static bool +ls_handle_lsp_changes(struct ovsdb_idl_txn *ovnsb_idl_txn, + const struct nbrec_logical_switch *changed_ls, + const struct northd_input *ni, + struct northd_data *nd, + struct ovn_datapath *od, + struct ls_change *ls_change, + bool *updated) +{ + bool ls_ports_changed = false; + if (!nbrec_logical_switch_is_updated(changed_ls, + NBREC_LOGICAL_SWITCH_COL_PORTS)) { + + for (size_t i = 0; i < changed_ls->n_ports; i++) { + if (nbrec_logical_switch_port_row_get_seqno( + changed_ls->ports[i], OVSDB_IDL_CHANGE_MODIFY) > 0) { + ls_ports_changed = true; + break; + } + } + } else { + ls_ports_changed = true; + } + + if (!ls_ports_changed) { + return true; + } + + /* Check if the logical switch has only router ports before this change. + * If so, fall back to recompute. + * lflow engine node while building the lflows checks if the logical switch + * has any router ports and depending on that it adds different flows. + * See build_lswitch_rport_arp_req_flow() for more details. + * Note: We can definitely handle this scenario incrementally in the + * northd engine node and fall back to recompute in lflow engine node + * and even handle this incrementally in lflow node. Until we do that + * resort to full recompute of northd node. + */ + bool only_rports = (od->n_router_ports + && (od->n_router_ports == hmap_count(&od->ports))); + if (only_rports) { + return false; + } + + struct ovn_port *op; + HMAP_FOR_EACH (op, dp_node, &od->ports) { + op->visited = false; + } + + /* Compare the individual ports in the old and new Logical Switches */ + for (size_t j = 0; j < changed_ls->n_ports; ++j) { + struct nbrec_logical_switch_port *new_nbsp = changed_ls->ports[j]; + op = ovn_port_find_in_datapath(od, new_nbsp->name); + + if (!op) { + if (!lsp_can_be_inc_processed(new_nbsp)) { + goto fail; + } + op = ls_port_create(ovnsb_idl_txn, &nd->ls_ports, + new_nbsp->name, new_nbsp, od, NULL, NULL, + ni->sbrec_mirror_table, + ni->sbrec_chassis_table, + ni->sbrec_chassis_by_name, + ni->sbrec_chassis_by_hostname); + if (!op) { + goto fail; + } + ovs_list_push_back(&ls_change->added_ports, + &op->list); + } else if (ls_port_has_changed(op->nbsp, new_nbsp)) { + /* Existing port updated */ + bool temp = false; + if (lsp_is_type_changed(op->sb, new_nbsp, &temp) || + !op->lsp_can_be_inc_processed || + !lsp_can_be_inc_processed(new_nbsp)) { + goto fail; + } + const struct sbrec_port_binding *sb = op->sb; + if (sset_contains(&nd->svc_monitor_lsps, new_nbsp->name)) { + /* This port is used for svc monitor, which may be impacted + * by this change. Fallback to recompute. */ + goto fail; + } + if (!check_lsp_is_up && + !check_lsp_changes_other_than_up(new_nbsp)) { + /* If the only change is the "up" column while the + * "ignore_lsp_down" is set to true, just ignore this + * change. */ + op->visited = true; + continue; + } + struct ovs_list lflows = OVS_LIST_INITIALIZER(&lflows); + ovs_list_splice(&lflows, op->lflows.next, &op->lflows); + ovn_port_destroy(&nd->ls_ports, op); + op = ls_port_create(ovnsb_idl_txn, &nd->ls_ports, + new_nbsp->name, new_nbsp, od, sb, &lflows, + ni->sbrec_mirror_table, + ni->sbrec_chassis_table, + ni->sbrec_chassis_by_name, + ni->sbrec_chassis_by_hostname); + ovs_assert(ovs_list_is_empty(&lflows)); + if (!op) { + goto fail; + } + ovs_list_push_back(&ls_change->updated_ports, &op->list); + } + op->visited = true; + } + + /* Check for deleted ports */ + HMAP_FOR_EACH_SAFE (op, dp_node, &od->ports) { + if (!op->visited) { + if (!op->lsp_can_be_inc_processed) { + goto fail_clean_deleted; + } + if (sset_contains(&nd->svc_monitor_lsps, op->key)) { + /* This port was used for svc monitor, which may be + * impacted by this deletion. Fallback to recompute. */ + goto fail_clean_deleted; + } + ovs_list_push_back(&ls_change->deleted_ports, + &op->list); + hmap_remove(&nd->ls_ports, &op->key_node); + hmap_remove(&od->ports, &op->dp_node); + sbrec_port_binding_delete(op->sb); + delete_fdb_entry(ni->sbrec_fdb_by_dp_and_port, od->tunnel_key, + op->tunnel_key); + } + } + + /* Check if the logical switch has only router ports after this change. + * If so, fall back to recompute. + * lflow engine node while building the lflows checks if the logical switch + * has any router ports and depending on that it adds different flows. + * See build_lswitch_rport_arp_req_flow() for more details. + * Note: We can definitely handle this scenario incrementally in the + * northd engine node and fall back to recompute in lflow engine node + * and even handle this incrementally in lflow node. Until we do that + * resort to full recompute of northd node. + */ + only_rports = (od->n_router_ports + && (od->n_router_ports == hmap_count(&od->ports))); + if (only_rports) { + goto fail_clean_deleted; + } + + if (!ovs_list_is_empty(&ls_change->added_ports) || + !ovs_list_is_empty(&ls_change->updated_ports) || + !ovs_list_is_empty(&ls_change->deleted_ports)) { + *updated = true; + } + + return true; + +fail_clean_deleted: + LIST_FOR_EACH_POP (op, list, &ls_change->deleted_ports) { + ovn_port_destroy_orphan(op); + } + +fail: + return false; +} + /* Return true if changes are handled incrementally, false otherwise. * When there are any changes, try to track what's exactly changed and set * northd_data->change_tracked accordingly: change tracked - true, otherwise, @@ -5130,11 +5308,9 @@ northd_handle_ls_changes(struct ovsdb_idl_txn *ovnsb_idl_txn, { const struct nbrec_logical_switch *changed_ls; struct ls_change *ls_change = NULL; - struct ovn_port *op; NBREC_LOGICAL_SWITCH_TABLE_FOR_EACH_TRACKED (changed_ls, ni->nbrec_logical_switch_table) { - ls_change = NULL; if (nbrec_logical_switch_is_new(changed_ls) || nbrec_logical_switch_is_deleted(changed_ls)) { goto fail; @@ -5150,8 +5326,8 @@ northd_handle_ls_changes(struct ovsdb_idl_txn *ovnsb_idl_txn, goto fail; } - /* Now only able to handle lsp changes. */ - if (check_ls_changes_other_than_lsp(changed_ls)) { + /* Check if the ls changes can be handled or not. */ + if (!ls_changes_can_be_handled(changed_ls)) { goto fail; } @@ -5161,126 +5337,16 @@ northd_handle_ls_changes(struct ovsdb_idl_txn *ovnsb_idl_txn, ovs_list_init(&ls_change->deleted_ports); ovs_list_init(&ls_change->updated_ports); - HMAP_FOR_EACH (op, dp_node, &od->ports) { - op->visited = false; - } - - /* Check if the logical switch has only router ports before this change. - * If so, fall back to recompute. - * lflow engine node while building the lflows checks if the logical switch - * has any router ports and depending on that it adds different flows. - * See build_lswitch_rport_arp_req_flow() for more details. - * Note: We can definitely handle this scenario incrementally in the - * northd engine node and fall back to recompute in lflow engine node - * and even handle this incrementally in lflow node. Until we do that - * resort to full recompute of northd node. - */ - bool only_rports = (od->n_router_ports - && (od->n_router_ports == hmap_count(&od->ports))); - if (only_rports) { + bool updated = false; + if (!ls_handle_lsp_changes(ovnsb_idl_txn, changed_ls, + ni, nd, od, ls_change, + &updated)) { + destroy_tracked_ls_change(ls_change); + free(ls_change); goto fail; } - /* Compare the individual ports in the old and new Logical Switches */ - for (size_t j = 0; j < changed_ls->n_ports; ++j) { - struct nbrec_logical_switch_port *new_nbsp = changed_ls->ports[j]; - op = ovn_port_find_in_datapath(od, new_nbsp->name); - - if (!op) { - if (!lsp_can_be_inc_processed(new_nbsp)) { - goto fail; - } - op = ls_port_create(ovnsb_idl_txn, &nd->ls_ports, - new_nbsp->name, new_nbsp, od, NULL, NULL, - ni->sbrec_mirror_table, - ni->sbrec_chassis_table, - ni->sbrec_chassis_by_name, - ni->sbrec_chassis_by_hostname); - if (!op) { - goto fail; - } - ovs_list_push_back(&ls_change->added_ports, - &op->list); - } else if (ls_port_has_changed(op->nbsp, new_nbsp)) { - /* Existing port updated */ - bool temp = false; - if (lsp_is_type_changed(op->sb, new_nbsp, &temp) || - !op->lsp_can_be_inc_processed || - !lsp_can_be_inc_processed(new_nbsp)) { - goto fail; - } - const struct sbrec_port_binding *sb = op->sb; - if (sset_contains(&nd->svc_monitor_lsps, new_nbsp->name)) { - /* This port is used for svc monitor, which may be impacted - * by this change. Fallback to recompute. */ - goto fail; - } - if (!check_lsp_is_up && - !check_lsp_changes_other_than_up(new_nbsp)) { - /* If the only change is the "up" column while the - * "ignore_lsp_down" is set to true, just ignore this - * change. */ - op->visited = true; - continue; - } - struct ovs_list lflows = OVS_LIST_INITIALIZER(&lflows); - ovs_list_splice(&lflows, op->lflows.next, &op->lflows); - ovn_port_destroy(&nd->ls_ports, op); - op = ls_port_create(ovnsb_idl_txn, &nd->ls_ports, - new_nbsp->name, new_nbsp, od, sb, &lflows, - ni->sbrec_mirror_table, - ni->sbrec_chassis_table, - ni->sbrec_chassis_by_name, - ni->sbrec_chassis_by_hostname); - ovs_assert(ovs_list_is_empty(&lflows)); - if (!op) { - goto fail; - } - ovs_list_push_back(&ls_change->updated_ports, &op->list); - } - op->visited = true; - } - - /* Check for deleted ports */ - HMAP_FOR_EACH_SAFE (op, dp_node, &od->ports) { - if (!op->visited) { - if (!op->lsp_can_be_inc_processed) { - goto fail_clean_deleted; - } - if (sset_contains(&nd->svc_monitor_lsps, op->key)) { - /* This port was used for svc monitor, which may be - * impacted by this deletion. Fallback to recompute. */ - goto fail_clean_deleted; - } - ovs_list_push_back(&ls_change->deleted_ports, - &op->list); - hmap_remove(&nd->ls_ports, &op->key_node); - hmap_remove(&od->ports, &op->dp_node); - sbrec_port_binding_delete(op->sb); - delete_fdb_entry(ni->sbrec_fdb_by_dp_and_port, od->tunnel_key, - op->tunnel_key); - } - } - - /* Check if the logical switch has only router ports after this change. - * If so, fall back to recompute. - * lflow engine node while building the lflows checks if the logical switch - * has any router ports and depending on that it adds different flows. - * See build_lswitch_rport_arp_req_flow() for more details. - * Note: We can definitely handle this scenario incrementally in the - * northd engine node and fall back to recompute in lflow engine node - * and even handle this incrementally in lflow node. Until we do that - * resort to full recompute of northd node. - */ - only_rports = (od->n_router_ports - && (od->n_router_ports == hmap_count(&od->ports))); - if (only_rports) { - goto fail_clean_deleted; - } - - if (!ovs_list_is_empty(&ls_change->added_ports) || - !ovs_list_is_empty(&ls_change->updated_ports) || - !ovs_list_is_empty(&ls_change->deleted_ports)) { + if (updated) { ovs_list_push_back(&nd->tracked_ls_changes.updated, &ls_change->list_node); } else { @@ -5293,13 +5359,7 @@ northd_handle_ls_changes(struct ovsdb_idl_txn *ovnsb_idl_txn, } return true; -fail_clean_deleted: - LIST_FOR_EACH_POP (op, list, &ls_change->deleted_ports) { - ovn_port_destroy_orphan(op); - } - fail: - free(ls_change); destroy_northd_data_tracked_changes(nd); return false; } From patchwork Fri Aug 18 08:57:42 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Numan Siddique X-Patchwork-Id: 1822756 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::137; helo=smtp4.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp4.osuosl.org (smtp4.osuosl.org [IPv6:2605:bc80:3010::137]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4RRwl0390pz1ygH for ; Fri, 18 Aug 2023 18:58:20 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 9C50E42543; Fri, 18 Aug 2023 08:58:18 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 9C50E42543 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Dc5J5DiLYdpo; Fri, 18 Aug 2023 08:58:16 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp4.osuosl.org (Postfix) with ESMTPS id 5BC98426C6; Fri, 18 Aug 2023 08:58:15 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 5BC98426C6 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 12ACCC0072; Fri, 18 Aug 2023 08:58:15 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp1.osuosl.org (smtp1.osuosl.org [IPv6:2605:bc80:3010::138]) by lists.linuxfoundation.org (Postfix) with ESMTP id 20F26C0032 for ; Fri, 18 Aug 2023 08:58:14 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 5F4FF83F71 for ; Fri, 18 Aug 2023 08:57:58 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 5F4FF83F71 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id pjYlXtrbIWX0 for ; Fri, 18 Aug 2023 08:57:53 +0000 (UTC) Received: from relay9-d.mail.gandi.net (relay9-d.mail.gandi.net [217.70.183.199]) by smtp1.osuosl.org (Postfix) with ESMTPS id B774C83F57 for ; Fri, 18 Aug 2023 08:57:52 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org B774C83F57 Received: by mail.gandi.net (Postfix) with ESMTPSA id 67496FF802; Fri, 18 Aug 2023 08:57:49 +0000 (UTC) From: numans@ovn.org To: dev@openvswitch.org Date: Fri, 18 Aug 2023 14:27:42 +0530 Message-Id: <20230818085742.1030980-1-numans@ovn.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230818085606.1030792-1-numans@ovn.org> References: <20230818085606.1030792-1-numans@ovn.org> MIME-Version: 1.0 X-GND-Sasl: numans@ovn.org Subject: [ovs-dev] [PATCH ovn v6 05/16] northd: Handle load balancer changes for a logical switch. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Numan Siddique 'lb_data' engine node now also handles logical switch changes. Its data maintains ls to lb related information. i.e if a logical switch sw0 has lb1, lb2 and lb3 associated then it stores this info in its data. And when a new load balancer lb4 is associated to it, it stores this information in its tracked data so that 'northd' engine node can handle it accordingly. Tracked data will have information like: changed ls -> {sw0 : {associated_lbs: [lb4]} In the 'northd' engne node, an additional handler for 'lb_data' is added after the 'nbrec_logical_switch' changes. With this patch we now have 2 handlers for 'lb_data' in 'northd' engine node. ---- engine_add_input(&en_northd, &en_lb_data, northd_lb_data_handler_pre_od); engine_add_input(&en_northd, &en_nb_logical_switch, northd_nb_logical_switch_handler); engine_add_input(&en_northd, &en_lb_data, northd_lb_data_handler_post_od); ---- The first handler 'northd_lb_data_handler_pre_od' is called before the 'northd_nb_logical_switch_handler' handler and it just creates or deletes the lb_datapaths hmap for the tracked lbs. The second handler 'northd_lb_data_handler_post_od' is called after the 'northd_nb_logical_switch_handler' handler and it updates the ovn_lb_datapaths's 'nb_ls_map' bitmap. Eg. If the lb_data has the below tracked data: tracked_data = {'crupdated_lbs': [lb1, lb2], 'deleted_lbs': [lb3], 'crupdated_lb_groups': [lbg1, lbg2], 'crupdated_ls_lbs': [{ls: sw0, assoc_lbs: [lb1], {ls: sw1, assoc_lbs: [lb1, lb2]} then the first handler northd_lb_data_handler_pre_od(), creates the ovn_lb_datapaths object for lb1 and lb2 and deletes lb3 from the ovn_lb_datapaths hmap. Similarly for the created or updated lb groups lbg1 and lbg2. The second handler northd_lb_data_handler_post_od() updates the nb_ls_bitmap of lb1 for sw0 and sw1 and nb_ls_bitmap of lb2 for sw1. This second handler is added mainly so that the actual logical switch handler northd_nb_logical_switch_handler() has done modifying the 'od' struct. Signed-off-by: Numan Siddique Reviewed-by: Ales Musil --- lib/lb.c | 5 +- northd/en-lb-data.c | 176 +++++++++++++++++++++++++++++++++++++++ northd/en-lb-data.h | 17 ++++ northd/en-lflow.c | 6 ++ northd/en-northd.c | 40 +++++++-- northd/en-northd.h | 3 +- northd/inc-proc-northd.c | 5 +- northd/northd.c | 104 +++++++++++++++++++---- northd/northd.h | 15 ++-- tests/ovn-northd.at | 56 +++++++++---- 10 files changed, 380 insertions(+), 47 deletions(-) diff --git a/lib/lb.c b/lib/lb.c index 6fd67e2218..e6c9dc2be2 100644 --- a/lib/lb.c +++ b/lib/lb.c @@ -1088,7 +1088,10 @@ ovn_lb_datapaths_add_ls(struct ovn_lb_datapaths *lb_dps, size_t n, struct ovn_datapath **ods) { for (size_t i = 0; i < n; i++) { - bitmap_set1(lb_dps->nb_ls_map, ods[i]->index); + if (!bitmap_is_set(lb_dps->nb_ls_map, ods[i]->index)) { + bitmap_set1(lb_dps->nb_ls_map, ods[i]->index); + lb_dps->n_nb_ls++; + } } } diff --git a/northd/en-lb-data.c b/northd/en-lb-data.c index 23f2cb1021..e0c4db1422 100644 --- a/northd/en-lb-data.c +++ b/northd/en-lb-data.c @@ -39,6 +39,14 @@ static void lb_data_destroy(struct ed_type_lb_data *); static void build_lbs(const struct nbrec_load_balancer_table *, const struct nbrec_load_balancer_group_table *, struct hmap *lbs, struct hmap *lb_groups); +static void build_od_lb_map(const struct nbrec_logical_switch_table *, + struct hmap *od_lb_map); +static struct od_lb_data *find_od_lb_data(struct hmap *od_lb_map, + const struct uuid *od_uuid); +static void destroy_od_lb_data(struct od_lb_data *od_lb_data); +static struct od_lb_data *create_od_lb_data(struct hmap *od_lb_map, + const struct uuid *od_uuid); + static struct ovn_lb_group *create_lb_group( const struct nbrec_load_balancer_group *, struct hmap *lbs, struct hmap *lb_groups); @@ -54,6 +62,7 @@ static struct crupdated_lb_group * struct tracked_lb_data *); static void add_deleted_lb_group_to_tracked_data( struct ovn_lb_group *, struct tracked_lb_data *); +static bool is_ls_lbs_changed(const struct nbrec_logical_switch *nbs); /* 'lb_data' engine node manages the NB load balancers and load balancer * groups. For each NB LB, it creates 'struct ovn_northd_lb' and @@ -80,9 +89,13 @@ en_lb_data_run(struct engine_node *node, void *data) EN_OVSDB_GET(engine_get_input("NB_load_balancer", node)); const struct nbrec_load_balancer_group_table *nb_lbg_table = EN_OVSDB_GET(engine_get_input("NB_load_balancer_group", node)); + const struct nbrec_logical_switch_table *nb_ls_table = + EN_OVSDB_GET(engine_get_input("NB_logical_switch", node)); lb_data->tracked = false; build_lbs(nb_lb_table, nb_lbg_table, &lb_data->lbs, &lb_data->lb_groups); + build_od_lb_map(nb_ls_table, &lb_data->ls_lb_map); + engine_set_node_state(node, EN_UPDATED); } @@ -230,18 +243,104 @@ lb_data_load_balancer_group_handler(struct engine_node *node, void *data) return true; } +struct od_lb_data { + struct hmap_node hmap_node; + struct uuid od_uuid; + struct uuidset *lbs; + struct uuidset *lbgrps; +}; + +bool +lb_data_logical_switch_handler(struct engine_node *node, void *data) +{ + struct ed_type_lb_data *lb_data = (struct ed_type_lb_data *) data; + const struct nbrec_logical_switch_table *nb_ls_table = + EN_OVSDB_GET(engine_get_input("NB_logical_switch", node)); + + struct tracked_lb_data *trk_lb_data = &lb_data->tracked_lb_data; + lb_data->tracked = true; + + bool changed = false; + const struct nbrec_logical_switch *nbs; + NBREC_LOGICAL_SWITCH_TABLE_FOR_EACH_TRACKED (nbs, nb_ls_table) { + if (nbrec_logical_switch_is_deleted(nbs)) { + struct od_lb_data *od_lb_data = + find_od_lb_data(&lb_data->ls_lb_map, &nbs->header_.uuid); + if (od_lb_data) { + hmap_remove(&lb_data->ls_lb_map, &od_lb_data->hmap_node); + destroy_od_lb_data(od_lb_data); + } + } else { + if (!is_ls_lbs_changed(nbs)) { + continue; + } + struct crupdated_od_lb_data *codlb = xzalloc(sizeof *codlb); + codlb->od_uuid = nbs->header_.uuid; + uuidset_init(&codlb->assoc_lbs); + + struct od_lb_data *od_lb_data = + find_od_lb_data(&lb_data->ls_lb_map, &nbs->header_.uuid); + if (!od_lb_data) { + od_lb_data = create_od_lb_data(&lb_data->ls_lb_map, + &nbs->header_.uuid); + } + + struct uuidset *pre_lb_uuids = od_lb_data->lbs; + od_lb_data->lbs = xzalloc(sizeof *od_lb_data->lbs); + uuidset_init(od_lb_data->lbs); + + for (size_t i = 0; i < nbs->n_load_balancer; i++) { + const struct uuid *lb_uuid = + &nbs->load_balancer[i]->header_.uuid; + uuidset_insert(od_lb_data->lbs, lb_uuid); + + struct uuidset_node *unode = uuidset_find(pre_lb_uuids, + lb_uuid); + + if (!unode || (nbrec_load_balancer_row_get_seqno( + nbs->load_balancer[i], OVSDB_IDL_CHANGE_MODIFY) > 0)) { + /* Add this lb to the tracked data. */ + uuidset_insert(&codlb->assoc_lbs, lb_uuid); + changed = true; + } + + if (unode) { + uuidset_delete(pre_lb_uuids, unode); + } + } + + if (!uuidset_is_empty(pre_lb_uuids)) { + trk_lb_data->has_dissassoc_lbs_from_od = true; + changed = true; + } + + uuidset_destroy(pre_lb_uuids); + free(pre_lb_uuids); + + ovs_list_insert(&trk_lb_data->crupdated_ls_lbs, &codlb->list_node); + } + } + + if (changed) { + engine_set_node_state(node, EN_UPDATED); + } + return true; +} + /* static functions. */ static void lb_data_init(struct ed_type_lb_data *lb_data) { hmap_init(&lb_data->lbs); hmap_init(&lb_data->lb_groups); + hmap_init(&lb_data->ls_lb_map); struct tracked_lb_data *trk_lb_data = &lb_data->tracked_lb_data; hmap_init(&trk_lb_data->crupdated_lbs); hmapx_init(&trk_lb_data->deleted_lbs); hmap_init(&trk_lb_data->crupdated_lb_groups); hmapx_init(&trk_lb_data->deleted_lb_groups); + ovs_list_init(&trk_lb_data->crupdated_ls_lbs); } static void @@ -259,6 +358,12 @@ lb_data_destroy(struct ed_type_lb_data *lb_data) } hmap_destroy(&lb_data->lb_groups); + struct od_lb_data *od_lb_data; + HMAP_FOR_EACH_POP (od_lb_data, hmap_node, &lb_data->ls_lb_map) { + destroy_od_lb_data(od_lb_data); + } + hmap_destroy(&lb_data->ls_lb_map); + destroy_tracked_data(lb_data); hmap_destroy(&lb_data->tracked_lb_data.crupdated_lbs); hmapx_destroy(&lb_data->tracked_lb_data.deleted_lbs); @@ -301,12 +406,67 @@ create_lb_group(const struct nbrec_load_balancer_group *nbrec_lb_group, return lb_group; } +static void +build_od_lb_map(const struct nbrec_logical_switch_table *nbrec_ls_table, + struct hmap *od_lb_map) +{ + const struct nbrec_logical_switch *nbrec_ls; + NBREC_LOGICAL_SWITCH_TABLE_FOR_EACH (nbrec_ls, nbrec_ls_table) { + if (!nbrec_ls->n_load_balancer) { + continue; + } + + struct od_lb_data *od_lb_data = + create_od_lb_data(od_lb_map, &nbrec_ls->header_.uuid); + for (size_t i = 0; i < nbrec_ls->n_load_balancer; i++) { + uuidset_insert(od_lb_data->lbs, + &nbrec_ls->load_balancer[i]->header_.uuid); + } + } +} + +static struct od_lb_data * +create_od_lb_data(struct hmap *od_lb_map, const struct uuid *od_uuid) +{ + struct od_lb_data *od_lb_data = xzalloc(sizeof *od_lb_data); + od_lb_data->od_uuid = *od_uuid; + od_lb_data->lbs = xzalloc(sizeof *od_lb_data->lbs); + uuidset_init(od_lb_data->lbs); + + hmap_insert(od_lb_map, &od_lb_data->hmap_node, + uuid_hash(&od_lb_data->od_uuid)); + return od_lb_data; +} + +static struct od_lb_data * +find_od_lb_data(struct hmap *od_lb_map, const struct uuid *od_uuid) +{ + struct od_lb_data *od_lb_data; + HMAP_FOR_EACH_WITH_HASH (od_lb_data, hmap_node, uuid_hash(od_uuid), + od_lb_map) { + if (uuid_equals(&od_lb_data->od_uuid, od_uuid)) { + return od_lb_data; + } + } + + return NULL; +} + +static void +destroy_od_lb_data(struct od_lb_data *od_lb_data) +{ + uuidset_destroy(od_lb_data->lbs); + free(od_lb_data->lbs); + free(od_lb_data); +} + static void destroy_tracked_data(struct ed_type_lb_data *lb_data) { lb_data->tracked = false; lb_data->tracked_lb_data.has_health_checks = false; lb_data->tracked_lb_data.has_dissassoc_lbs_from_lb_grops = false; + lb_data->tracked_lb_data.has_dissassoc_lbs_from_od = false; struct hmapx_node *node; HMAPX_FOR_EACH_SAFE (node, &lb_data->tracked_lb_data.deleted_lbs) { @@ -331,6 +491,15 @@ destroy_tracked_data(struct ed_type_lb_data *lb_data) hmapx_destroy(&crupdated_lbg->assoc_lbs); free(crupdated_lbg); } + + struct crupdated_od_lb_data *codlb; + LIST_FOR_EACH_SAFE (codlb, list_node, + &lb_data->tracked_lb_data.crupdated_ls_lbs) { + ovs_list_remove(&codlb->list_node); + uuidset_destroy(&codlb->assoc_lbs); + + free(codlb); + } } static void @@ -376,3 +545,10 @@ add_deleted_lb_group_to_tracked_data(struct ovn_lb_group *lbg, { hmapx_add(&tracked_lb_data->deleted_lb_groups, lbg); } + +static bool +is_ls_lbs_changed(const struct nbrec_logical_switch *nbs) { + return ((nbrec_logical_switch_is_new(nbs) && nbs->n_load_balancer) + || nbrec_logical_switch_is_updated(nbs, + NBREC_LOGICAL_SWITCH_COL_LOAD_BALANCER)); +} diff --git a/northd/en-lb-data.h b/northd/en-lb-data.h index 2e9a024620..b2f86322a2 100644 --- a/northd/en-lb-data.h +++ b/northd/en-lb-data.h @@ -6,6 +6,7 @@ #include "openvswitch/hmap.h" #include "include/openvswitch/list.h" #include "lib/hmapx.h" +#include "lib/uuidset.h" #include "lib/inc-proc-eng.h" @@ -27,6 +28,13 @@ struct crupdated_lb_group { struct hmapx assoc_lbs; }; +struct crupdated_od_lb_data { + struct ovs_list list_node; + + struct uuid od_uuid; + struct uuidset assoc_lbs; +}; + struct tracked_lb_data { /* Both created and updated lbs. hmapx node is 'struct crupdated_lb *'. */ struct hmap crupdated_lbs; @@ -41,12 +49,19 @@ struct tracked_lb_data { /* Deleted lb_groups. hmapx node is 'struct ovn_lb_group *'. */ struct hmapx deleted_lb_groups; + /* List of logical switch <-> lb changes. List node is + * 'struct crupdated_od_lb_data' */ + struct ovs_list crupdated_ls_lbs; + /* Indicates if any of the tracked lb has health checks enabled. */ bool has_health_checks; /* Indicates if any lb got disassociated from a lb group * but not deleted. */ bool has_dissassoc_lbs_from_lb_grops; + + /* Indicates if a lb was disassociated from a logical switch. */ + bool has_dissassoc_lbs_from_od; }; /* struct which maintains the data of the engine node lb_data. */ @@ -56,6 +71,7 @@ struct ed_type_lb_data { /* hmap of load balancer groups. hmap node is 'struct ovn_lb_group *' */ struct hmap lb_groups; + struct hmap ls_lb_map; /* tracked data*/ bool tracked; @@ -69,5 +85,6 @@ void en_lb_data_clear_tracked_data(void *data); bool lb_data_load_balancer_handler(struct engine_node *, void *data); bool lb_data_load_balancer_group_handler(struct engine_node *, void *data); +bool lb_data_logical_switch_handler(struct engine_node *, void *data); #endif /* end of EN_NORTHD_LB_DATA_H */ diff --git a/northd/en-lflow.c b/northd/en-lflow.c index db1bcbccd6..77e2eff056 100644 --- a/northd/en-lflow.c +++ b/northd/en-lflow.c @@ -102,6 +102,12 @@ lflow_northd_handler(struct engine_node *node, if (!northd_data->change_tracked) { return false; } + + /* Fall back to recompute if lb related data has changed. */ + if (northd_data->lb_changed) { + return false; + } + const struct engine_context *eng_ctx = engine_get_context(); struct lflow_data *lflow_data = data; diff --git a/northd/en-northd.c b/northd/en-northd.c index d59ef062df..9d1838a1a4 100644 --- a/northd/en-northd.c +++ b/northd/en-northd.c @@ -207,7 +207,7 @@ northd_sb_port_binding_handler(struct engine_node *node, } bool -northd_lb_data_handler(struct engine_node *node, void *data) +northd_lb_data_handler_pre_od(struct engine_node *node, void *data) { struct ed_type_lb_data *lb_data = engine_get_input_data("lb_data", node); @@ -230,19 +230,47 @@ northd_lb_data_handler(struct engine_node *node, void *data) /* Fall back to recompute if load balancer groups are deleted. */ if (!hmapx_is_empty(&lb_data->tracked_lb_data.deleted_lb_groups)) { + } + + if (lb_data->tracked_lb_data.has_dissassoc_lbs_from_od) { return false; } struct northd_data *nd = data; - if (!northd_handle_lb_data_changes(&lb_data->tracked_lb_data, - &nd->ls_datapaths, - &nd->lr_datapaths, - &nd->lb_datapaths_map, - &nd->lb_group_datapaths_map)) { + if (!northd_handle_lb_data_changes_pre_od(&lb_data->tracked_lb_data, + &nd->ls_datapaths, + &nd->lr_datapaths, + &nd->lb_datapaths_map, + &nd->lb_group_datapaths_map)) { + return false; + } + + engine_set_node_state(node, EN_UPDATED); + return true; +} + +bool +northd_lb_data_handler_post_od(struct engine_node *node, void *data) +{ + struct ed_type_lb_data *lb_data = + engine_get_input_data("lb_data", node); + + ovs_assert(lb_data->tracked); + ovs_assert(!lb_data->tracked_lb_data.has_dissassoc_lbs_from_od); + + struct northd_data *nd = data; + + if (!northd_handle_lb_data_changes_post_od(&lb_data->tracked_lb_data, + &nd->ls_datapaths, + &nd->lb_datapaths_map)) { return false; } + /* Indicate the depedendant engine nodes that load balancer/group + * related data has changed (including association to logical + * switch/router). */ + nd->lb_changed = true; engine_set_node_state(node, EN_UPDATED); return true; } diff --git a/northd/en-northd.h b/northd/en-northd.h index 3c77b64bb2..5926e7a9d3 100644 --- a/northd/en-northd.h +++ b/northd/en-northd.h @@ -17,6 +17,7 @@ void en_northd_clear_tracked_data(void *data); bool northd_nb_nb_global_handler(struct engine_node *, void *data OVS_UNUSED); bool northd_nb_logical_switch_handler(struct engine_node *, void *data); bool northd_sb_port_binding_handler(struct engine_node *, void *data); -bool northd_lb_data_handler(struct engine_node *, void *data); +bool northd_lb_data_handler_pre_od(struct engine_node *, void *data); +bool northd_lb_data_handler_post_od(struct engine_node *, void *data); #endif /* EN_NORTHD_H */ diff --git a/northd/inc-proc-northd.c b/northd/inc-proc-northd.c index 75d059645f..402c94e88c 100644 --- a/northd/inc-proc-northd.c +++ b/northd/inc-proc-northd.c @@ -152,6 +152,8 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, lb_data_load_balancer_handler); engine_add_input(&en_lb_data, &en_nb_load_balancer_group, lb_data_load_balancer_group_handler); + engine_add_input(&en_lb_data, &en_nb_logical_switch, + lb_data_logical_switch_handler); engine_add_input(&en_northd, &en_nb_port_group, NULL); engine_add_input(&en_northd, &en_nb_acl, NULL); @@ -180,9 +182,10 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, northd_sb_port_binding_handler); engine_add_input(&en_northd, &en_nb_nb_global, northd_nb_nb_global_handler); - engine_add_input(&en_northd, &en_lb_data, northd_lb_data_handler); + engine_add_input(&en_northd, &en_lb_data, northd_lb_data_handler_pre_od); engine_add_input(&en_northd, &en_nb_logical_switch, northd_nb_logical_switch_handler); + engine_add_input(&en_northd, &en_lb_data, northd_lb_data_handler_post_od); engine_add_input(&en_mac_binding_aging, &en_nb_nb_global, NULL); engine_add_input(&en_mac_binding_aging, &en_sb_mac_binding, NULL); diff --git a/northd/northd.c b/northd/northd.c index 4cc9ef8c8d..6e8efbd496 100644 --- a/northd/northd.c +++ b/northd/northd.c @@ -853,7 +853,6 @@ ovn_datapath_destroy(struct hmap *datapaths, struct ovn_datapath *od) ovn_ls_port_group_destroy(&od->nb_pgs); destroy_mcast_info_for_datapath(od); destroy_ports_for_datapath(od); - free(od); } } @@ -4919,6 +4918,7 @@ destroy_northd_data_tracked_changes(struct northd_data *nd) } nd->change_tracked = false; + nd->lb_changed = false; } /* Check if a changed LSP can be handled incrementally within the I-P engine @@ -5034,6 +5034,7 @@ ls_port_create(struct ovsdb_idl_txn *ovnsb_txn, struct hmap *ls_ports, * incrementally handled. * Presently supports i-p for the below changes: * - logical switch ports. + * - load balancers. */ static bool ls_changes_can_be_handled( @@ -5042,8 +5043,11 @@ ls_changes_can_be_handled( /* Check if the columns are changed in this row. */ enum nbrec_logical_switch_column_id col; for (col = 0; col < NBREC_LOGICAL_SWITCH_N_COLUMNS; col++) { - if (nbrec_logical_switch_is_updated(ls, col) && - col != NBREC_LOGICAL_SWITCH_COL_PORTS) { + if (nbrec_logical_switch_is_updated(ls, col)) { + if (col == NBREC_LOGICAL_SWITCH_COL_PORTS || + col == NBREC_LOGICAL_SWITCH_COL_LOAD_BALANCER) { + continue; + } return false; } } @@ -5072,12 +5076,6 @@ ls_changes_can_be_handled( return false; } } - for (size_t i = 0; i < ls->n_load_balancer; i++) { - if (nbrec_load_balancer_row_get_seqno(ls->load_balancer[i], - OVSDB_IDL_CHANGE_MODIFY) > 0) { - return false; - } - } for (size_t i = 0; i < ls->n_load_balancer_group; i++) { if (nbrec_load_balancer_group_row_get_seqno(ls->load_balancer_group[i], OVSDB_IDL_CHANGE_MODIFY) > 0) { @@ -5357,6 +5355,7 @@ northd_handle_ls_changes(struct ovsdb_idl_txn *ovnsb_idl_txn, if (!ovs_list_is_empty(&nd->tracked_ls_changes.updated)) { nd->change_tracked = true; } + return true; fail: @@ -5425,16 +5424,21 @@ northd_handle_sb_port_binding_changes( /* Handler for lb_data engine changes. For every tracked lb_data * it creates or deletes the ovn_lb_datapaths/ovn_lb_group_datapaths - * from the lb_datapaths hmap and lb_group_datapaths hmap. */ + * from the lb_datapaths hmap and lb_group_datapaths hmap. + * + * This handler is called if there are any lb_data changes but before + * processing the logical switch changes. + * */ bool -northd_handle_lb_data_changes(struct tracked_lb_data *trk_lb_data, - struct ovn_datapaths *ls_datapaths, - struct ovn_datapaths *lr_datapaths, - struct hmap *lb_datapaths_map, - struct hmap *lb_group_datapaths_map) +northd_handle_lb_data_changes_pre_od(struct tracked_lb_data *trk_lb_data, + struct ovn_datapaths *ls_datapaths, + struct ovn_datapaths *lr_datapaths, + struct hmap *lb_datapaths_map, + struct hmap *lb_group_datapaths_map) { struct ovn_lb_datapaths *lb_dps; struct ovn_northd_lb *lb; + struct ovn_datapath *od; struct hmapx_node *hmapx_node; HMAPX_FOR_EACH (hmapx_node, &trk_lb_data->deleted_lbs) { lb = hmapx_node->data; @@ -5442,6 +5446,16 @@ northd_handle_lb_data_changes(struct tracked_lb_data *trk_lb_data, lb_dps = ovn_lb_datapaths_find(lb_datapaths_map, lb_uuid); ovs_assert(lb_dps); + + /* Re-evaluate 'od->has_lb_vip for od's associated with the + * deleted lb. */ + size_t index; + BITMAP_FOR_EACH_1 (index, ods_size(ls_datapaths), + lb_dps->nb_ls_map) { + od = ls_datapaths->array[index]; + init_lb_for_datapath(od); + } + hmap_remove(lb_datapaths_map, &lb_dps->hmap_node); ovn_lb_datapaths_destroy(lb_dps); } @@ -5481,6 +5495,65 @@ northd_handle_lb_data_changes(struct tracked_lb_data *trk_lb_data, return true; } +/* This handler is called if there are any lb_data changes but afer processing + * the logical switch changes. + * + * For every tracked data it does the following: + * For any changes to a created or updated logical switch due to + * association of a load balancer (eg. ovn-nbctl ls-lb-add sw0 lb1), + * the logical switch datapath is added to the load balancer (represented + * by 'struct ovn_lb_datapaths') by calling ovn_lb_datapaths_add_ls(). + * + * For every created or updated load balancer in the tracked data, + * it gets the associated logical switches and for each switch it + * re-evaluates 'od->has_lb_vip'. Since this handler is called after + * handling any logical switch changes. */ +bool +northd_handle_lb_data_changes_post_od(struct tracked_lb_data *trk_lb_data, + struct ovn_datapaths *ls_datapaths, + struct hmap *lb_datapaths_map) +{ + ovs_assert(!trk_lb_data->has_health_checks); + + struct ovn_northd_lb *lb; + struct ovn_lb_datapaths *lb_dps; + struct ovn_datapath *od; + struct crupdated_od_lb_data *codlb; + + LIST_FOR_EACH (codlb, list_node, &trk_lb_data->crupdated_ls_lbs) { + od = ovn_datapath_find(&ls_datapaths->datapaths, &codlb->od_uuid); + ovs_assert(od); + + struct uuidset_node *uuidnode; + UUIDSET_FOR_EACH (uuidnode, &codlb->assoc_lbs) { + lb_dps = ovn_lb_datapaths_find(lb_datapaths_map, &uuidnode->uuid); + ovs_assert(lb_dps); + ovn_lb_datapaths_add_ls(lb_dps, 1, &od); + } + + /* Re-evaluate 'od->has_lb_vip' */ + init_lb_for_datapath(od); + } + + struct crupdated_lb *clb; + HMAP_FOR_EACH (clb, hmap_node, &trk_lb_data->crupdated_lbs) { + lb = clb->lb; + const struct uuid *lb_uuid = &lb->nlb->header_.uuid; + + lb_dps = ovn_lb_datapaths_find(lb_datapaths_map, lb_uuid); + ovs_assert(lb_dps); + size_t index; + BITMAP_FOR_EACH_1 (index, ods_size(ls_datapaths), + lb_dps->nb_ls_map) { + od = ls_datapaths->array[index]; + /* Re-evaluate 'od->has_lb_vip' */ + init_lb_for_datapath(od); + } + } + + return true; +} + struct multicast_group { const char *name; uint16_t key; /* OVN_MIN_MULTICAST...OVN_MAX_MULTICAST. */ @@ -16667,6 +16740,7 @@ bool lflow_handle_northd_ls_changes(struct ovsdb_idl_txn *ovnsb_txn, struct hmap *lflows) { struct ls_change *ls_change; + LIST_FOR_EACH (ls_change, list_node, &ls_changes->updated) { const struct sbrec_multicast_group *sbmc_flood = mcast_group_lookup(lflow_input->sbrec_mcast_group_by_name_dp, diff --git a/northd/northd.h b/northd/northd.h index 7d17921fa2..0ed7215356 100644 --- a/northd/northd.h +++ b/northd/northd.h @@ -120,6 +120,8 @@ struct northd_data { struct hmap svc_monitor_map; bool change_tracked; struct tracked_ls_changes tracked_ls_changes; + bool lb_changed; /* Indicates if load balancers changed or association of + * load balancer to logical switch/router changed. */ }; struct lflow_data { @@ -348,11 +350,14 @@ bool northd_handle_sb_port_binding_changes( const struct sbrec_port_binding_table *, struct hmap *ls_ports); struct tracked_lb_data; -bool northd_handle_lb_data_changes(struct tracked_lb_data *, - struct ovn_datapaths *ls_datapaths, - struct ovn_datapaths *lr_datapaths, - struct hmap *lb_datapaths_map, - struct hmap *lb_group_datapaths_map); +bool northd_handle_lb_data_changes_pre_od(struct tracked_lb_data *, + struct ovn_datapaths *ls_datapaths, + struct ovn_datapaths *lr_datapaths, + struct hmap *lb_datapaths_map, + struct hmap *lb_group_datapaths_map); +bool northd_handle_lb_data_changes_post_od(struct tracked_lb_data *, + struct ovn_datapaths *ls_datapaths, + struct hmap *lb_datapaths_map); void build_bfd_table(struct ovsdb_idl_txn *ovnsb_txn, const struct nbrec_bfd_table *, diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at index 213fd51d77..9e9b26ce09 100644 --- a/tests/ovn-northd.at +++ b/tests/ovn-northd.at @@ -9861,22 +9861,23 @@ ovn-nbctl lsp-set-type sw0-lr0 router ovn-nbctl lsp-set-addresses sw0-lr0 00:00:00:00:ff:01 check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats ovn-nbctl --wait=sb lsp-set-options sw0-lr0 router-port=lr0-sw0 -check_engine_stats lb_data norecompute nocompute +check_engine_stats lb_data norecompute compute check_engine_stats northd recompute nocompute check_engine_stats lflow recompute nocompute -# Associate lb1 to sw0. There should be a full recompute of northd engine node +# Associate lb1 to sw0. There should be no recompute of northd engine node check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb ls-lb-add sw0 lb1 -check_engine_stats lb_data norecompute nocompute -check_engine_stats northd recompute nocompute +check_engine_stats lb_data norecompute compute +check_engine_stats northd norecompute compute check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE # Modify the backend of the lb1 vip check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb set load_balancer lb1 vips:'"10.0.0.10:80"'='"10.0.0.100:80"' check_engine_stats lb_data norecompute compute -check_engine_stats northd recompute nocompute +check_engine_stats northd norecompute compute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -9884,7 +9885,7 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb clear load_Balancer lb1 vips check_engine_stats lb_data norecompute compute -check_engine_stats northd recompute nocompute +check_engine_stats northd norecompute compute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -9892,7 +9893,7 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lb-add lb1 10.0.0.10:80 10.0.0.3:80 check_engine_stats lb_data norecompute compute -check_engine_stats northd recompute nocompute +check_engine_stats northd norecompute compute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -9900,14 +9901,33 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lb-add lb1 10.0.0.20:80 10.0.0.30:8080 check_engine_stats lb_data norecompute compute -check_engine_stats northd recompute nocompute +check_engine_stats northd norecompute compute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Disassociate lb1 from sw0. There should be a full recompute of northd engine node. check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb ls-lb-del sw0 lb1 -check_engine_stats lb_data norecompute nocompute +check_engine_stats lb_data norecompute compute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +# Associate lb1 to sw0 and also create a port sw0p1. This should result in +# full recompute of northd and lflow engine node. +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb ls-lb-add sw0 lb1 -- lsp-add sw0 sw0p1 +check_engine_stats lb_data norecompute compute +check_engine_stats northd recompute compute +check_engine_stats lflow recompute nocompute + +CHECK_NO_CHANGE_AFTER_RECOMPUTE +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats + +# Disassociate lb1 from sw0. There should be a recompute of northd engine node. +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb ls-lb-del sw0 lb1 +check_engine_stats lb_data norecompute compute check_engine_stats northd recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -9995,7 +10015,7 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl add logical_switch sw0 load_balancer_group $lbg1_uuid -check_engine_stats lb_data norecompute nocompute +check_engine_stats lb_data norecompute compute check_engine_stats northd recompute nocompute check_engine_stats lflow recompute nocompute @@ -10040,7 +10060,7 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl clear logical_switch sw0 load_balancer_group -check_engine_stats lb_data norecompute nocompute +check_engine_stats lb_data norecompute compute check_engine_stats northd recompute nocompute check_engine_stats lflow recompute nocompute @@ -10092,7 +10112,7 @@ check_engine_stats lflow recompute nocompute # Add back lb group to logical switch and then delete it. check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl add logical_switch sw0 load_balancer_group $lbg1_uuid -check_engine_stats lb_data norecompute nocompute +check_engine_stats lb_data norecompute compute check_engine_stats northd recompute nocompute check_engine_stats lflow recompute nocompute @@ -10133,7 +10153,7 @@ check_engine_stats lflow recompute nocompute check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl set logical_switch sw0 load_balancer_group=$lbg1_uuid -check_engine_stats lb_data norecompute nocompute +check_engine_stats lb_data norecompute compute check_engine_stats northd recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10147,15 +10167,15 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb ls-lb-add sw0 lb2 -check_engine_stats lb_data norecompute nocompute -check_engine_stats northd recompute nocompute +check_engine_stats lb_data norecompute compute +check_engine_stats northd norecompute compute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb ls-lb-add sw0 lb3 -check_engine_stats lb_data norecompute nocompute -check_engine_stats northd recompute nocompute +check_engine_stats lb_data norecompute compute +check_engine_stats northd norecompute compute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10169,7 +10189,7 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb ls-lb-del sw0 lb2 -check_engine_stats lb_data norecompute nocompute +check_engine_stats lb_data norecompute compute check_engine_stats northd recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE From patchwork Fri Aug 18 08:57:51 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Numan Siddique X-Patchwork-Id: 1822757 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::138; helo=smtp1.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp1.osuosl.org (smtp1.osuosl.org [IPv6:2605:bc80:3010::138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4RRwlD0Fdjz1ygH for ; Fri, 18 Aug 2023 18:58:31 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id E12B583FB8; Fri, 18 Aug 2023 08:58:29 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org E12B583FB8 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id KJrsRtcBT6_s; Fri, 18 Aug 2023 08:58:28 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp1.osuosl.org (Postfix) with ESMTPS id 668B783F49; Fri, 18 Aug 2023 08:58:27 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 668B783F49 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 439DBC0039; Fri, 18 Aug 2023 08:58:27 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp1.osuosl.org (smtp1.osuosl.org [IPv6:2605:bc80:3010::138]) by lists.linuxfoundation.org (Postfix) with ESMTP id 1E13BC0039 for ; Fri, 18 Aug 2023 08:58:26 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id E468183F49 for ; Fri, 18 Aug 2023 08:58:01 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org E468183F49 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 4l4hAXT7DtBz for ; Fri, 18 Aug 2023 08:57:58 +0000 (UTC) Received: from relay2-d.mail.gandi.net (relay2-d.mail.gandi.net [217.70.183.194]) by smtp1.osuosl.org (Postfix) with ESMTPS id 3C4FD83F7C for ; Fri, 18 Aug 2023 08:57:57 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 3C4FD83F7C Received: by mail.gandi.net (Postfix) with ESMTPSA id 5785940007; Fri, 18 Aug 2023 08:57:55 +0000 (UTC) From: numans@ovn.org To: dev@openvswitch.org Date: Fri, 18 Aug 2023 14:27:51 +0530 Message-Id: <20230818085751.1030995-1-numans@ovn.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230818085606.1030792-1-numans@ovn.org> References: <20230818085606.1030792-1-numans@ovn.org> MIME-Version: 1.0 X-GND-Sasl: numans@ovn.org Subject: [ovs-dev] [PATCH ovn v6 06/16] northd: Handle load balancer group changes for a logical switch. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Numan Siddique For every a given load balancer group 'A', northd engine data maintains a bitmap of datapaths associated to this lb group. So when lb group 'A' gets associated to a logical switch 's1', the bitmap index of 's1' is set in its bitmap. In order to handle the load balancer group changes incrementally for a logical switch, we need to set and clear the bitmap bits accordingly. And this patch does it. Signed-off-by: Numan Siddique Reviewed-by: Ales Musil --- northd/en-lb-data.c | 102 ++++++++++++++++++++++++++++++++------------ northd/en-lb-data.h | 4 ++ northd/en-northd.c | 9 +++- northd/northd.c | 70 ++++++++++++++++++++++++------ northd/northd.h | 5 ++- tests/ovn-northd.at | 30 +++++++++---- 6 files changed, 169 insertions(+), 51 deletions(-) diff --git a/northd/en-lb-data.c b/northd/en-lb-data.c index e0c4db1422..8619b4cc14 100644 --- a/northd/en-lb-data.c +++ b/northd/en-lb-data.c @@ -63,6 +63,7 @@ static struct crupdated_lb_group * static void add_deleted_lb_group_to_tracked_data( struct ovn_lb_group *, struct tracked_lb_data *); static bool is_ls_lbs_changed(const struct nbrec_logical_switch *nbs); +static bool is_ls_lbgrps_changed(const struct nbrec_logical_switch *nbs); /* 'lb_data' engine node manages the NB load balancers and load balancer * groups. For each NB LB, it creates 'struct ovn_northd_lb' and @@ -271,12 +272,15 @@ lb_data_logical_switch_handler(struct engine_node *node, void *data) destroy_od_lb_data(od_lb_data); } } else { - if (!is_ls_lbs_changed(nbs)) { + bool ls_lbs_changed = is_ls_lbs_changed(nbs); + bool ls_lbgrps_changed = is_ls_lbgrps_changed(nbs); + if (!ls_lbs_changed && !ls_lbgrps_changed) { continue; } struct crupdated_od_lb_data *codlb = xzalloc(sizeof *codlb); codlb->od_uuid = nbs->header_.uuid; uuidset_init(&codlb->assoc_lbs); + uuidset_init(&codlb->assoc_lbgrps); struct od_lb_data *od_lb_data = find_od_lb_data(&lb_data->ls_lb_map, &nbs->header_.uuid); @@ -285,38 +289,66 @@ lb_data_logical_switch_handler(struct engine_node *node, void *data) &nbs->header_.uuid); } - struct uuidset *pre_lb_uuids = od_lb_data->lbs; - od_lb_data->lbs = xzalloc(sizeof *od_lb_data->lbs); - uuidset_init(od_lb_data->lbs); - - for (size_t i = 0; i < nbs->n_load_balancer; i++) { - const struct uuid *lb_uuid = - &nbs->load_balancer[i]->header_.uuid; - uuidset_insert(od_lb_data->lbs, lb_uuid); + if (ls_lbs_changed) { + struct uuidset *pre_lb_uuids = od_lb_data->lbs; + od_lb_data->lbs = xzalloc(sizeof *od_lb_data->lbs); + uuidset_init(od_lb_data->lbs); + + for (size_t i = 0; i < nbs->n_load_balancer; i++) { + const struct uuid *lb_uuid = + &nbs->load_balancer[i]->header_.uuid; + uuidset_insert(od_lb_data->lbs, lb_uuid); + + struct uuidset_node *unode = uuidset_find(pre_lb_uuids, + lb_uuid); + + if (!unode || (nbrec_load_balancer_row_get_seqno( + nbs->load_balancer[i], + OVSDB_IDL_CHANGE_MODIFY) > 0)) { + /* Add this lb to the tracked data. */ + uuidset_insert(&codlb->assoc_lbs, lb_uuid); + changed = true; + } + + if (unode) { + uuidset_delete(pre_lb_uuids, unode); + } + } + if (!uuidset_is_empty(pre_lb_uuids)) { + trk_lb_data->has_dissassoc_lbs_from_od = true; + changed = true; + } - struct uuidset_node *unode = uuidset_find(pre_lb_uuids, - lb_uuid); + uuidset_destroy(pre_lb_uuids); + free(pre_lb_uuids); + } - if (!unode || (nbrec_load_balancer_row_get_seqno( - nbs->load_balancer[i], OVSDB_IDL_CHANGE_MODIFY) > 0)) { - /* Add this lb to the tracked data. */ - uuidset_insert(&codlb->assoc_lbs, lb_uuid); - changed = true; + if (ls_lbgrps_changed) { + struct uuidset *pre_lbgrp_uuids = od_lb_data->lbgrps; + od_lb_data->lbgrps = xzalloc(sizeof *od_lb_data->lbgrps); + uuidset_init(od_lb_data->lbgrps); + for (size_t i = 0; i < nbs->n_load_balancer_group; i++) { + const struct uuid *lbg_uuid = + &nbs->load_balancer_group[i]->header_.uuid; + uuidset_insert(od_lb_data->lbgrps, lbg_uuid); + + if (!uuidset_find_and_delete(pre_lbgrp_uuids, + lbg_uuid)) { + /* Add this lb group to the tracked data. */ + uuidset_insert(&codlb->assoc_lbgrps, lbg_uuid); + changed = true; + } } - if (unode) { - uuidset_delete(pre_lb_uuids, unode); + if (!uuidset_is_empty(pre_lbgrp_uuids)) { + trk_lb_data->has_dissassoc_lbgrps_from_od = true; + changed = true; } - } - if (!uuidset_is_empty(pre_lb_uuids)) { - trk_lb_data->has_dissassoc_lbs_from_od = true; - changed = true; + uuidset_destroy(pre_lbgrp_uuids); + free(pre_lbgrp_uuids); } - uuidset_destroy(pre_lb_uuids); - free(pre_lb_uuids); - ovs_list_insert(&trk_lb_data->crupdated_ls_lbs, &codlb->list_node); } } @@ -412,7 +444,7 @@ build_od_lb_map(const struct nbrec_logical_switch_table *nbrec_ls_table, { const struct nbrec_logical_switch *nbrec_ls; NBREC_LOGICAL_SWITCH_TABLE_FOR_EACH (nbrec_ls, nbrec_ls_table) { - if (!nbrec_ls->n_load_balancer) { + if (!nbrec_ls->n_load_balancer && !nbrec_ls->n_load_balancer_group) { continue; } @@ -422,6 +454,10 @@ build_od_lb_map(const struct nbrec_logical_switch_table *nbrec_ls_table, uuidset_insert(od_lb_data->lbs, &nbrec_ls->load_balancer[i]->header_.uuid); } + for (size_t i = 0; i < nbrec_ls->n_load_balancer_group; i++) { + uuidset_insert(od_lb_data->lbgrps, + &nbrec_ls->load_balancer_group[i]->header_.uuid); + } } } @@ -431,7 +467,9 @@ create_od_lb_data(struct hmap *od_lb_map, const struct uuid *od_uuid) struct od_lb_data *od_lb_data = xzalloc(sizeof *od_lb_data); od_lb_data->od_uuid = *od_uuid; od_lb_data->lbs = xzalloc(sizeof *od_lb_data->lbs); + od_lb_data->lbgrps = xzalloc(sizeof *od_lb_data->lbgrps); uuidset_init(od_lb_data->lbs); + uuidset_init(od_lb_data->lbgrps); hmap_insert(od_lb_map, &od_lb_data->hmap_node, uuid_hash(&od_lb_data->od_uuid)); @@ -456,7 +494,9 @@ static void destroy_od_lb_data(struct od_lb_data *od_lb_data) { uuidset_destroy(od_lb_data->lbs); + uuidset_destroy(od_lb_data->lbgrps); free(od_lb_data->lbs); + free(od_lb_data->lbgrps); free(od_lb_data); } @@ -467,6 +507,7 @@ destroy_tracked_data(struct ed_type_lb_data *lb_data) lb_data->tracked_lb_data.has_health_checks = false; lb_data->tracked_lb_data.has_dissassoc_lbs_from_lb_grops = false; lb_data->tracked_lb_data.has_dissassoc_lbs_from_od = false; + lb_data->tracked_lb_data.has_dissassoc_lbgrps_from_od = false; struct hmapx_node *node; HMAPX_FOR_EACH_SAFE (node, &lb_data->tracked_lb_data.deleted_lbs) { @@ -497,7 +538,7 @@ destroy_tracked_data(struct ed_type_lb_data *lb_data) &lb_data->tracked_lb_data.crupdated_ls_lbs) { ovs_list_remove(&codlb->list_node); uuidset_destroy(&codlb->assoc_lbs); - + uuidset_destroy(&codlb->assoc_lbgrps); free(codlb); } } @@ -552,3 +593,10 @@ is_ls_lbs_changed(const struct nbrec_logical_switch *nbs) { || nbrec_logical_switch_is_updated(nbs, NBREC_LOGICAL_SWITCH_COL_LOAD_BALANCER)); } + +static bool +is_ls_lbgrps_changed(const struct nbrec_logical_switch *nbs) { + return ((nbrec_logical_switch_is_new(nbs) && nbs->n_load_balancer_group) + || nbrec_logical_switch_is_updated(nbs, + NBREC_LOGICAL_SWITCH_COL_LOAD_BALANCER_GROUP)); +} diff --git a/northd/en-lb-data.h b/northd/en-lb-data.h index b2f86322a2..bc09ddb7eb 100644 --- a/northd/en-lb-data.h +++ b/northd/en-lb-data.h @@ -33,6 +33,7 @@ struct crupdated_od_lb_data { struct uuid od_uuid; struct uuidset assoc_lbs; + struct uuidset assoc_lbgrps; }; struct tracked_lb_data { @@ -62,6 +63,9 @@ struct tracked_lb_data { /* Indicates if a lb was disassociated from a logical switch. */ bool has_dissassoc_lbs_from_od; + + /* Indicates if a lb group was disassociated from a logical switch. */ + bool has_dissassoc_lbgrps_from_od; }; /* struct which maintains the data of the engine node lb_data. */ diff --git a/northd/en-northd.c b/northd/en-northd.c index 9d1838a1a4..545971f76f 100644 --- a/northd/en-northd.c +++ b/northd/en-northd.c @@ -236,6 +236,10 @@ northd_lb_data_handler_pre_od(struct engine_node *node, void *data) return false; } + if (lb_data->tracked_lb_data.has_dissassoc_lbgrps_from_od) { + return false; + } + struct northd_data *nd = data; if (!northd_handle_lb_data_changes_pre_od(&lb_data->tracked_lb_data, @@ -258,12 +262,15 @@ northd_lb_data_handler_post_od(struct engine_node *node, void *data) ovs_assert(lb_data->tracked); ovs_assert(!lb_data->tracked_lb_data.has_dissassoc_lbs_from_od); + ovs_assert(!lb_data->tracked_lb_data.has_dissassoc_lbgrps_from_od); + ovs_assert(!lb_data->tracked_lb_data.has_dissassoc_lbs_from_lb_grops); struct northd_data *nd = data; if (!northd_handle_lb_data_changes_post_od(&lb_data->tracked_lb_data, &nd->ls_datapaths, - &nd->lb_datapaths_map)) { + &nd->lb_datapaths_map, + &nd->lb_group_datapaths_map)) { return false; } diff --git a/northd/northd.c b/northd/northd.c index 6e8efbd496..1477b79331 100644 --- a/northd/northd.c +++ b/northd/northd.c @@ -5035,6 +5035,7 @@ ls_port_create(struct ovsdb_idl_txn *ovnsb_txn, struct hmap *ls_ports, * Presently supports i-p for the below changes: * - logical switch ports. * - load balancers. + * - load balancer groups. */ static bool ls_changes_can_be_handled( @@ -5045,7 +5046,8 @@ ls_changes_can_be_handled( for (col = 0; col < NBREC_LOGICAL_SWITCH_N_COLUMNS; col++) { if (nbrec_logical_switch_is_updated(ls, col)) { if (col == NBREC_LOGICAL_SWITCH_COL_PORTS || - col == NBREC_LOGICAL_SWITCH_COL_LOAD_BALANCER) { + col == NBREC_LOGICAL_SWITCH_COL_LOAD_BALANCER || + col == NBREC_LOGICAL_SWITCH_COL_LOAD_BALANCER_GROUP) { continue; } return false; @@ -5076,12 +5078,6 @@ ls_changes_can_be_handled( return false; } } - for (size_t i = 0; i < ls->n_load_balancer_group; i++) { - if (nbrec_load_balancer_group_row_get_seqno(ls->load_balancer_group[i], - OVSDB_IDL_CHANGE_MODIFY) > 0) { - return false; - } - } for (size_t i = 0; i < ls->n_qos_rules; i++) { if (nbrec_qos_row_get_seqno(ls->qos_rules[i], OVSDB_IDL_CHANGE_MODIFY) > 0) { @@ -5298,7 +5294,11 @@ fail: /* Return true if changes are handled incrementally, false otherwise. * When there are any changes, try to track what's exactly changed and set * northd_data->change_tracked accordingly: change tracked - true, otherwise, - * false. */ + * false. + * + * Note: Changes to load balancer and load balancer groups associated with + * the logical switches are handled separately in the lb_data change handlers. + * */ bool northd_handle_ls_changes(struct ovsdb_idl_txn *ovnsb_idl_txn, const struct northd_input *ni, @@ -5434,7 +5434,7 @@ northd_handle_lb_data_changes_pre_od(struct tracked_lb_data *trk_lb_data, struct ovn_datapaths *ls_datapaths, struct ovn_datapaths *lr_datapaths, struct hmap *lb_datapaths_map, - struct hmap *lb_group_datapaths_map) + struct hmap *lbgrp_datapaths_map) { struct ovn_lb_datapaths *lb_dps; struct ovn_northd_lb *lb; @@ -5482,12 +5482,12 @@ northd_handle_lb_data_changes_pre_od(struct tracked_lb_data *trk_lb_data, lbg = crupdated_lbg->lbg; const struct uuid *lb_uuid = &lbg->uuid; - lb_group_dps = ovn_lb_group_datapaths_find(lb_group_datapaths_map, + lb_group_dps = ovn_lb_group_datapaths_find(lbgrp_datapaths_map, lb_uuid); if (!lb_group_dps) { lb_group_dps = ovn_lb_group_datapaths_create( lbg, ods_size(ls_datapaths), ods_size(lr_datapaths)); - hmap_insert(lb_group_datapaths_map, &lb_group_dps->hmap_node, + hmap_insert(lbgrp_datapaths_map, &lb_group_dps->hmap_node, uuid_hash(lb_uuid)); } } @@ -5511,12 +5511,15 @@ northd_handle_lb_data_changes_pre_od(struct tracked_lb_data *trk_lb_data, bool northd_handle_lb_data_changes_post_od(struct tracked_lb_data *trk_lb_data, struct ovn_datapaths *ls_datapaths, - struct hmap *lb_datapaths_map) + struct hmap *lb_datapaths_map, + struct hmap *lbgrp_datapaths_map) { ovs_assert(!trk_lb_data->has_health_checks); + ovs_assert(!trk_lb_data->has_dissassoc_lbs_from_lb_grops); struct ovn_northd_lb *lb; struct ovn_lb_datapaths *lb_dps; + struct ovn_lb_group_datapaths *lbgrp_dps; struct ovn_datapath *od; struct crupdated_od_lb_data *codlb; @@ -5531,6 +5534,22 @@ northd_handle_lb_data_changes_post_od(struct tracked_lb_data *trk_lb_data, ovn_lb_datapaths_add_ls(lb_dps, 1, &od); } + UUIDSET_FOR_EACH (uuidnode, &codlb->assoc_lbgrps) { + lbgrp_dps = ovn_lb_group_datapaths_find(lbgrp_datapaths_map, + &uuidnode->uuid); + ovs_assert(lbgrp_dps); + ovn_lb_group_datapaths_add_ls(lbgrp_dps, 1, &od); + + /* Associate all the lbs of the lbgrp to the datapath 'od' */ + for (size_t j = 0; j < lbgrp_dps->lb_group->n_lbs; j++) { + const struct uuid *lb_uuid + = &lbgrp_dps->lb_group->lbs[j]->nlb->header_.uuid; + lb_dps = ovn_lb_datapaths_find(lb_datapaths_map, lb_uuid); + ovs_assert(lb_dps); + ovn_lb_datapaths_add_ls(lb_dps, 1, &od); + } + } + /* Re-evaluate 'od->has_lb_vip' */ init_lb_for_datapath(od); } @@ -5551,6 +5570,33 @@ northd_handle_lb_data_changes_post_od(struct tracked_lb_data *trk_lb_data, } } + struct ovn_lb_group *lbgrp; + struct crupdated_lb_group *crupdated_lbg; + HMAP_FOR_EACH (crupdated_lbg, hmap_node, + &trk_lb_data->crupdated_lb_groups) { + lbgrp = crupdated_lbg->lbg; + const struct uuid *lb_uuid = &lbgrp->uuid; + + lbgrp_dps = ovn_lb_group_datapaths_find(lbgrp_datapaths_map, + lb_uuid); + ovs_assert(lbgrp_dps); + + struct hmapx_node *hnode; + HMAPX_FOR_EACH (hnode, &crupdated_lbg->assoc_lbs) { + lb = hnode->data; + lb_uuid = &lb->nlb->header_.uuid; + lb_dps = ovn_lb_datapaths_find(lb_datapaths_map, lb_uuid); + ovs_assert(lb_dps); + for (size_t i = 0; i < lbgrp_dps->n_ls; i++) { + od = lbgrp_dps->ls[i]; + ovn_lb_datapaths_add_ls(lb_dps, 1, &od); + + /* Re-evaluate 'od->has_lb_vip' */ + init_lb_for_datapath(od); + } + } + } + return true; } diff --git a/northd/northd.h b/northd/northd.h index 0ed7215356..044d4ee0c0 100644 --- a/northd/northd.h +++ b/northd/northd.h @@ -354,10 +354,11 @@ bool northd_handle_lb_data_changes_pre_od(struct tracked_lb_data *, struct ovn_datapaths *ls_datapaths, struct ovn_datapaths *lr_datapaths, struct hmap *lb_datapaths_map, - struct hmap *lb_group_datapaths_map); + struct hmap *lbgrp_datapaths_map); bool northd_handle_lb_data_changes_post_od(struct tracked_lb_data *, struct ovn_datapaths *ls_datapaths, - struct hmap *lb_datapaths_map); + struct hmap *lb_datapaths_map, + struct hmap *lbgrp_datapaths_map); void build_bfd_table(struct ovsdb_idl_txn *ovnsb_txn, const struct nbrec_bfd_table *, diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at index 9e9b26ce09..b8b2ee390e 100644 --- a/tests/ovn-northd.at +++ b/tests/ovn-northd.at @@ -2141,6 +2141,18 @@ check ovn-nbctl --wait=sb sync AT_CHECK([ovn-sbctl lflow-list | grep "ls_out_pre_lb.*priority=100" | grep reg0 | sort], [0], [dnl ]) +# Now associate vip again to lb4 and then delete it. +check ovn-nbctl set load_balancer $lb4 vips:"10.0.0.13"="10.0.0.6" +check ovn-nbctl --wait=sb sync +AT_CHECK([ovn-sbctl lflow-list | grep "ls_out_pre_lb.*priority=100" | grep reg0 | sort], [0], [dnl + table=1 (ls_out_pre_lb ), priority=100 , match=(ip), action=(reg0[[2]] = 1; next;) +]) + +check ovn-nbctl lb-del $lb4 +check ovn-nbctl --wait=sb sync +AT_CHECK([ovn-sbctl lflow-list | grep "ls_out_pre_lb.*priority=100" | grep reg0 | sort], [0], [dnl +]) + AT_CLEANUP ]) @@ -10016,21 +10028,21 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl add logical_switch sw0 load_balancer_group $lbg1_uuid check_engine_stats lb_data norecompute compute -check_engine_stats northd recompute nocompute +check_engine_stats northd norecompute compute check_engine_stats lflow recompute nocompute -# Update lb and this should result in recompute +# Update lb and this should not result in northd recompute check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb set load_balancer . options:bar=foo check_engine_stats lb_data norecompute compute -check_engine_stats northd recompute nocompute +check_engine_stats northd norecompute compute check_engine_stats lflow recompute nocompute # Modify the backend of the lb1 vip check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb set load_balancer lb1 vips:'"10.0.0.10:80"'='"10.0.0.100:80"' check_engine_stats lb_data norecompute compute -check_engine_stats northd recompute nocompute +check_engine_stats northd norecompute compute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10038,7 +10050,7 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb clear load_Balancer lb1 vips check_engine_stats lb_data norecompute compute -check_engine_stats northd recompute nocompute +check_engine_stats northd norecompute compute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10046,7 +10058,7 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lb-add lb1 10.0.0.10:80 10.0.0.3:80 check_engine_stats lb_data norecompute compute -check_engine_stats northd recompute nocompute +check_engine_stats northd norecompute compute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10054,7 +10066,7 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lb-add lb1 10.0.0.20:80 10.0.0.30:8080 check_engine_stats lb_data norecompute compute -check_engine_stats northd recompute nocompute +check_engine_stats northd norecompute compute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10113,7 +10125,7 @@ check_engine_stats lflow recompute nocompute check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl add logical_switch sw0 load_balancer_group $lbg1_uuid check_engine_stats lb_data norecompute compute -check_engine_stats northd recompute nocompute +check_engine_stats northd norecompute compute check_engine_stats lflow recompute nocompute check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats @@ -10154,7 +10166,7 @@ check_engine_stats lflow recompute nocompute check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl set logical_switch sw0 load_balancer_group=$lbg1_uuid check_engine_stats lb_data norecompute compute -check_engine_stats northd recompute nocompute +check_engine_stats northd norecompute compute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE From patchwork Fri Aug 18 08:57:56 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Numan Siddique X-Patchwork-Id: 1822758 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::138; helo=smtp1.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp1.osuosl.org (smtp1.osuosl.org [IPv6:2605:bc80:3010::138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4RRwlZ6svXz1ygH for ; Fri, 18 Aug 2023 18:58:50 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 4046883FBF; Fri, 18 Aug 2023 08:58:49 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 4046883FBF X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id rzw1kUGbp2LP; Fri, 18 Aug 2023 08:58:47 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp1.osuosl.org (Postfix) with ESMTPS id 5419183F72; Fri, 18 Aug 2023 08:58:46 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 5419183F72 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 2C1B8C0072; Fri, 18 Aug 2023 08:58:46 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp1.osuosl.org (smtp1.osuosl.org [IPv6:2605:bc80:3010::138]) by lists.linuxfoundation.org (Postfix) with ESMTP id 1A05FC0032 for ; Fri, 18 Aug 2023 08:58:45 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 6AD8283EC4 for ; Fri, 18 Aug 2023 08:58:08 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 6AD8283EC4 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 63JQ7r66rySk for ; Fri, 18 Aug 2023 08:58:05 +0000 (UTC) Received: from relay3-d.mail.gandi.net (relay3-d.mail.gandi.net [IPv6:2001:4b98:dc4:8::223]) by smtp1.osuosl.org (Postfix) with ESMTPS id 02BC883F5B for ; Fri, 18 Aug 2023 08:58:04 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 02BC883F5B Received: by mail.gandi.net (Postfix) with ESMTPSA id EB0406000A; Fri, 18 Aug 2023 08:58:00 +0000 (UTC) From: numans@ovn.org To: dev@openvswitch.org Date: Fri, 18 Aug 2023 14:27:56 +0530 Message-Id: <20230818085756.1031010-1-numans@ovn.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230818085606.1030792-1-numans@ovn.org> References: <20230818085606.1030792-1-numans@ovn.org> MIME-Version: 1.0 X-GND-Sasl: numans@ovn.org Subject: [ovs-dev] [PATCH ovn v6 07/16] northd: Sync SB Port bindings NAT column in a separate engine node. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Numan Siddique A new engine node 'sync_to_sb_pb' is added within 'sync_to_sb' node to sync NAT column of Port bindings table. This separation is required in order to add load balancer group I-P handling in 'northd' engine node (which is handled in the next commit). 'sync_to_sb_pb' engine node can be later expanded to sync other Port binding columns if required. Signed-off-by: Numan Siddique Reviewed-by: Ales Musil --- northd/en-sync-sb.c | 31 +++++ northd/en-sync-sb.h | 4 + northd/inc-proc-northd.c | 8 +- northd/northd.c | 243 +++++++++++++++++++++------------------ northd/northd.h | 2 + 5 files changed, 174 insertions(+), 114 deletions(-) diff --git a/northd/en-sync-sb.c b/northd/en-sync-sb.c index 9832fce30a..552ed56452 100644 --- a/northd/en-sync-sb.c +++ b/northd/en-sync-sb.c @@ -254,6 +254,37 @@ sync_to_sb_lb_northd_handler(struct engine_node *node, void *data OVS_UNUSED) return false; } +/* sync_to_sb_pb engine node functions. + * This engine node syncs the SB Port Bindings (partly). + * en_northd engine create the SB Port binding rows and + * updates most of the columns. + * This engine node updates the port binding columns which + * needs to be updated after northd engine is run. + */ + +void * +en_sync_to_sb_pb_init(struct engine_node *node OVS_UNUSED, + struct engine_arg *arg OVS_UNUSED) +{ + return NULL; +} + +void +en_sync_to_sb_pb_run(struct engine_node *node, void *data OVS_UNUSED) +{ + const struct engine_context *eng_ctx = engine_get_context(); + struct northd_data *northd_data = engine_get_input_data("northd", node); + + sync_pbs(eng_ctx->ovnsb_idl_txn, &northd_data->ls_ports); + engine_set_node_state(node, EN_UPDATED); +} + +void +en_sync_to_sb_pb_cleanup(void *data OVS_UNUSED) +{ + +} + /* static functions. */ static void sync_addr_set(struct ovsdb_idl_txn *ovnsb_txn, const char *name, diff --git a/northd/en-sync-sb.h b/northd/en-sync-sb.h index 06d2a57710..700d3340e4 100644 --- a/northd/en-sync-sb.h +++ b/northd/en-sync-sb.h @@ -22,4 +22,8 @@ void en_sync_to_sb_lb_run(struct engine_node *, void *data); void en_sync_to_sb_lb_cleanup(void *data); bool sync_to_sb_lb_northd_handler(struct engine_node *, void *data OVS_UNUSED); +void *en_sync_to_sb_pb_init(struct engine_node *, struct engine_arg *); +void en_sync_to_sb_pb_run(struct engine_node *, void *data); +void en_sync_to_sb_pb_cleanup(void *data); + #endif /* end of EN_SYNC_SB_H */ diff --git a/northd/inc-proc-northd.c b/northd/inc-proc-northd.c index 402c94e88c..dc8b880fd8 100644 --- a/northd/inc-proc-northd.c +++ b/northd/inc-proc-northd.c @@ -141,6 +141,7 @@ static ENGINE_NODE(sync_to_sb_addr_set, "sync_to_sb_addr_set"); static ENGINE_NODE(fdb_aging, "fdb_aging"); static ENGINE_NODE(fdb_aging_waker, "fdb_aging_waker"); static ENGINE_NODE(sync_to_sb_lb, "sync_to_sb_lb"); +static ENGINE_NODE(sync_to_sb_pb, "sync_to_sb_pb"); static ENGINE_NODE_WITH_CLEAR_TRACK_DATA(lb_data, "lb_data"); void inc_proc_northd_init(struct ovsdb_idl_loop *nb, @@ -215,13 +216,16 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, sync_to_sb_lb_northd_handler); engine_add_input(&en_sync_to_sb_lb, &en_sb_load_balancer, NULL); + engine_add_input(&en_sync_to_sb_pb, &en_northd, NULL); + /* en_sync_to_sb engine node syncs the SB database tables from * the NB database tables. - * Right now this engine syncs the SB Address_Set table and - * SB Load_Balancer table. + * Right now this engine syncs the SB Address_Set table, + * SB Load_Balancer table and (partly) SB Port_Binding table. */ engine_add_input(&en_sync_to_sb, &en_sync_to_sb_addr_set, NULL); engine_add_input(&en_sync_to_sb, &en_sync_to_sb_lb, NULL); + engine_add_input(&en_sync_to_sb, &en_sync_to_sb_pb, NULL); engine_add_input(&en_sync_from_sb, &en_northd, sync_from_sb_northd_handler); diff --git a/northd/northd.c b/northd/northd.c index 1477b79331..a3bd21e0b4 100644 --- a/northd/northd.c +++ b/northd/northd.c @@ -3514,8 +3514,6 @@ ovn_port_update_sbrec(struct ovsdb_idl_txn *ovnsb_txn, ds_destroy(&s); sbrec_port_binding_set_external_ids(op->sb, &op->nbrp->external_ids); - - sbrec_port_binding_set_nat_addresses(op->sb, NULL, 0); } else { if (!lsp_is_router(op->nbsp)) { uint32_t queue_id = smap_get_int( @@ -3631,116 +3629,6 @@ ovn_port_update_sbrec(struct ovsdb_idl_txn *ovnsb_txn, } else { sbrec_port_binding_set_options(op->sb, NULL); } - const char *nat_addresses = smap_get(&op->nbsp->options, - "nat-addresses"); - size_t n_nats = 0; - char **nats = NULL; - bool l3dgw_ports = op->peer && op->peer->od && - op->peer->od->n_l3dgw_ports; - if (nat_addresses && !strcmp(nat_addresses, "router")) { - if (op->peer && op->peer->od - && (chassis || op->peer->od->n_l3dgw_ports)) { - bool exclude_lb_vips = smap_get_bool(&op->nbsp->options, - "exclude-lb-vips-from-garp", false); - nats = get_nat_addresses(op->peer, &n_nats, false, - !exclude_lb_vips); - } - } else if (nat_addresses && (chassis || l3dgw_ports)) { - struct lport_addresses laddrs; - if (!extract_lsp_addresses(nat_addresses, &laddrs)) { - static struct vlog_rate_limit rl = - VLOG_RATE_LIMIT_INIT(1, 1); - VLOG_WARN_RL(&rl, "Error extracting nat-addresses."); - } else { - destroy_lport_addresses(&laddrs); - n_nats = 1; - nats = xcalloc(1, sizeof *nats); - struct ds nat_addr = DS_EMPTY_INITIALIZER; - ds_put_format(&nat_addr, "%s", nat_addresses); - if (l3dgw_ports) { - const struct ovn_port *l3dgw_port = ( - is_l3dgw_port(op->peer) - ? op->peer - : op->peer->od->l3dgw_ports[0]); - ds_put_format(&nat_addr, " is_chassis_resident(%s)", - l3dgw_port->cr_port->json_key); - } - nats[0] = xstrdup(ds_cstr(&nat_addr)); - ds_destroy(&nat_addr); - } - } - - /* Add the router mac and IPv4 addresses to - * Port_Binding.nat_addresses so that GARP is sent for these - * IPs by the ovn-controller on which the distributed gateway - * router port resides if: - * - * - op->peer has 'reside-on-redirect-chassis' set and the - * the logical router datapath has distributed router port. - * - * - op->peer is distributed gateway router port. - * - * - op->peer's router is a gateway router and op has a localnet - * port. - * - * Note: Port_Binding.nat_addresses column is also used for - * sending the GARPs for the router port IPs. - * */ - bool add_router_port_garp = false; - if (op->peer && op->peer->nbrp && op->peer->od->n_l3dgw_ports) { - if (is_l3dgw_port(op->peer)) { - add_router_port_garp = true; - } else if (smap_get_bool(&op->peer->nbrp->options, - "reside-on-redirect-chassis", false)) { - if (op->peer->od->n_l3dgw_ports == 1) { - add_router_port_garp = true; - } else { - static struct vlog_rate_limit rl = - VLOG_RATE_LIMIT_INIT(1, 1); - VLOG_WARN_RL(&rl, "\"reside-on-redirect-chassis\" is " - "set on logical router port %s, which " - "is on logical router %s, which has %" - PRIuSIZE" distributed gateway ports. This" - "option can only be used when there is " - "a single distributed gateway port.", - op->peer->key, op->peer->od->nbr->name, - op->peer->od->n_l3dgw_ports); - } - } - } else if (chassis && op->od->n_localnet_ports) { - add_router_port_garp = true; - } - - if (add_router_port_garp) { - struct ds garp_info = DS_EMPTY_INITIALIZER; - ds_put_format(&garp_info, "%s", op->peer->lrp_networks.ea_s); - - for (size_t i = 0; i < op->peer->lrp_networks.n_ipv4_addrs; - i++) { - ds_put_format(&garp_info, " %s", - op->peer->lrp_networks.ipv4_addrs[i].addr_s); - } - - if (op->peer->od->n_l3dgw_ports) { - const struct ovn_port *l3dgw_port = ( - is_l3dgw_port(op->peer) - ? op->peer - : op->peer->od->l3dgw_ports[0]); - ds_put_format(&garp_info, " is_chassis_resident(%s)", - l3dgw_port->cr_port->json_key); - } - - n_nats++; - nats = xrealloc(nats, (n_nats * sizeof *nats)); - nats[n_nats - 1] = ds_steal_cstr(&garp_info); - ds_destroy(&garp_info); - } - sbrec_port_binding_set_nat_addresses(op->sb, - (const char **) nats, n_nats); - for (size_t i = 0; i < n_nats; i++) { - free(nats[i]); - } - free(nats); } sbrec_port_binding_set_parent_port(op->sb, op->nbsp->parent_name); @@ -4695,6 +4583,137 @@ sync_lbs(struct ovsdb_idl_txn *ovnsb_txn, } } +/* Sync the SB Port bindings which needs to be updated. + * Presently it syncs the nat column of port bindings corresponding to + * the logical switch ports. */ +void sync_pbs(struct ovsdb_idl_txn *ovnsb_idl_txn, struct hmap *ls_ports) +{ + ovs_assert(ovnsb_idl_txn); + + struct ovn_port *op; + HMAP_FOR_EACH (op, key_node, ls_ports) { + if (lsp_is_router(op->nbsp)) { + const char *chassis = NULL; + if (op->peer && op->peer->od && op->peer->od->nbr) { + chassis = smap_get(&op->peer->od->nbr->options, "chassis"); + } + + const char *nat_addresses = smap_get(&op->nbsp->options, + "nat-addresses"); + size_t n_nats = 0; + char **nats = NULL; + bool l3dgw_ports = op->peer && op->peer->od && + op->peer->od->n_l3dgw_ports; + if (nat_addresses && !strcmp(nat_addresses, "router")) { + if (op->peer && op->peer->od + && (chassis || op->peer->od->n_l3dgw_ports)) { + bool exclude_lb_vips = smap_get_bool(&op->nbsp->options, + "exclude-lb-vips-from-garp", false); + nats = get_nat_addresses(op->peer, &n_nats, false, + !exclude_lb_vips); + } + } else if (nat_addresses && (chassis || l3dgw_ports)) { + struct lport_addresses laddrs; + if (!extract_lsp_addresses(nat_addresses, &laddrs)) { + static struct vlog_rate_limit rl = + VLOG_RATE_LIMIT_INIT(1, 1); + VLOG_WARN_RL(&rl, "Error extracting nat-addresses."); + } else { + destroy_lport_addresses(&laddrs); + n_nats = 1; + nats = xcalloc(1, sizeof *nats); + struct ds nat_addr = DS_EMPTY_INITIALIZER; + ds_put_format(&nat_addr, "%s", nat_addresses); + if (l3dgw_ports) { + const struct ovn_port *l3dgw_port = ( + is_l3dgw_port(op->peer) + ? op->peer + : op->peer->od->l3dgw_ports[0]); + ds_put_format(&nat_addr, " is_chassis_resident(%s)", + l3dgw_port->cr_port->json_key); + } + nats[0] = xstrdup(ds_cstr(&nat_addr)); + ds_destroy(&nat_addr); + } + } + + /* Add the router mac and IPv4 addresses to + * Port_Binding.nat_addresses so that GARP is sent for these + * IPs by the ovn-controller on which the distributed gateway + * router port resides if: + * + * - op->peer has 'reside-on-redirect-chassis' set and the + * the logical router datapath has distributed router port. + * + * - op->peer is distributed gateway router port. + * + * - op->peer's router is a gateway router and op has a localnet + * port. + * + * Note: Port_Binding.nat_addresses column is also used for + * sending the GARPs for the router port IPs. + * */ + bool add_router_port_garp = false; + if (op->peer && op->peer->nbrp && op->peer->od->n_l3dgw_ports) { + if (is_l3dgw_port(op->peer)) { + add_router_port_garp = true; + } else if (smap_get_bool(&op->peer->nbrp->options, + "reside-on-redirect-chassis", false)) { + if (op->peer->od->n_l3dgw_ports == 1) { + add_router_port_garp = true; + } else { + static struct vlog_rate_limit rl = + VLOG_RATE_LIMIT_INIT(1, 1); + VLOG_WARN_RL(&rl, "\"reside-on-redirect-chassis\" is " + "set on logical router port %s, which " + "is on logical router %s, which has %" + PRIuSIZE" distributed gateway ports. This" + "option can only be used when there is " + "a single distributed gateway port.", + op->peer->key, op->peer->od->nbr->name, + op->peer->od->n_l3dgw_ports); + } + } + } else if (chassis && op->od->n_localnet_ports) { + add_router_port_garp = true; + } + + if (add_router_port_garp) { + struct ds garp_info = DS_EMPTY_INITIALIZER; + ds_put_format(&garp_info, "%s", op->peer->lrp_networks.ea_s); + + for (size_t i = 0; i < op->peer->lrp_networks.n_ipv4_addrs; + i++) { + ds_put_format(&garp_info, " %s", + op->peer->lrp_networks.ipv4_addrs[i].addr_s); + } + + if (op->peer->od->n_l3dgw_ports) { + const struct ovn_port *l3dgw_port = ( + is_l3dgw_port(op->peer) + ? op->peer + : op->peer->od->l3dgw_ports[0]); + ds_put_format(&garp_info, " is_chassis_resident(%s)", + l3dgw_port->cr_port->json_key); + } + + n_nats++; + nats = xrealloc(nats, (n_nats * sizeof *nats)); + nats[n_nats - 1] = ds_steal_cstr(&garp_info); + ds_destroy(&garp_info); + } + sbrec_port_binding_set_nat_addresses(op->sb, + (const char **) nats, n_nats); + for (size_t i = 0; i < n_nats; i++) { + free(nats[i]); + } + free(nats); + } else { + sbrec_port_binding_set_nat_addresses(op->sb, NULL, 0); + } + } +} + static bool ovn_port_add_tnlid(struct ovn_port *op, uint32_t tunnel_key) { diff --git a/northd/northd.h b/northd/northd.h index 044d4ee0c0..cd2e5394c2 100644 --- a/northd/northd.h +++ b/northd/northd.h @@ -374,4 +374,6 @@ const char *northd_get_svc_monitor_mac(void); void sync_lbs(struct ovsdb_idl_txn *, const struct sbrec_load_balancer_table *, struct ovn_datapaths *ls_datapaths, struct hmap *lbs); +void sync_pbs(struct ovsdb_idl_txn *, struct hmap *ls_ports); + #endif /* NORTHD_H */ From patchwork Fri Aug 18 08:58:02 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Numan Siddique X-Patchwork-Id: 1822759 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::137; helo=smtp4.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp4.osuosl.org (smtp4.osuosl.org [IPv6:2605:bc80:3010::137]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4RRwm00htSz1ygH for ; Fri, 18 Aug 2023 18:59:11 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id DACBF427E6; Fri, 18 Aug 2023 08:59:08 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org DACBF427E6 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id tvS44Bt-vZqA; Fri, 18 Aug 2023 08:59:05 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp4.osuosl.org (Postfix) with ESMTPS id 78D8D42553; Fri, 18 Aug 2023 08:59:04 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 78D8D42553 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 5D7A6C0072; Fri, 18 Aug 2023 08:59:04 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp1.osuosl.org (smtp1.osuosl.org [IPv6:2605:bc80:3010::138]) by lists.linuxfoundation.org (Postfix) with ESMTP id 23208C0072 for ; Fri, 18 Aug 2023 08:59:03 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id ED02083F9B for ; Fri, 18 Aug 2023 08:58:18 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org ED02083F9B X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 4nnPjK_2vYe7 for ; Fri, 18 Aug 2023 08:58:13 +0000 (UTC) Received: from relay8-d.mail.gandi.net (relay8-d.mail.gandi.net [217.70.183.201]) by smtp1.osuosl.org (Postfix) with ESMTPS id 5C61683F6D for ; Fri, 18 Aug 2023 08:58:11 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 5C61683F6D Received: by mail.gandi.net (Postfix) with ESMTPSA id C011B1BF209; Fri, 18 Aug 2023 08:58:06 +0000 (UTC) From: numans@ovn.org To: dev@openvswitch.org Date: Fri, 18 Aug 2023 14:28:02 +0530 Message-Id: <20230818085802.1031026-1-numans@ovn.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230818085606.1030792-1-numans@ovn.org> References: <20230818085606.1030792-1-numans@ovn.org> MIME-Version: 1.0 X-GND-Sasl: numans@ovn.org Subject: [ovs-dev] [PATCH ovn v6 08/16] northd: Handle load balancer/group changes for a logical router. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Numan Siddique When a logical router gets updated due to load balancer or load balancer groups changes, it is now incrementally handled first in 'lb_data' engine node similar to how logical switch changes are handled. The tracking data of 'lb_data' is updated similarly so that northd engine handler - northd_handle_lb_data_changes_post_od() handles it. A new handler northd_handle_lr_changes() is added in the 'northd' engine node for logical router changes. This handler returns true if only load balancer or load balancer group columns are changed. It returns false for any other changes. northd_handle_lb_data_changes_post_od() also sets the logical router od's lb_ips accordingly. Below are the scale testing results done with these patches applied using ovn-heater. The test ran the scenario - ocp-500-density-heavy.yml [1]. With these patches applied (with load balancer I-P handling in northd engine node) the resuts are: ------------------------------------------------------------------------------------------------------------------------------------------------------- Min (s) Median (s) 90%ile (s) 99%ile (s) Max (s) Mean (s) Total (s) Count Failed ------------------------------------------------------------------------------------------------------------------------------------------------------- Iteration Total 0.132929 2.157103 3.314847 3.331561 4.378626 1.581889 197.736147 125 0 Namespace.add_ports 0.005217 0.005760 0.006565 0.013348 0.021014 0.006106 0.763214 125 0 WorkerNode.bind_port 0.035205 0.045458 0.052278 0.059804 0.063941 0.045652 11.413122 250 0 WorkerNode.ping_port 0.005075 0.006814 3.088548 3.192577 4.242026 0.726453 181.613284 250 0 ------------------------------------------------------------------------------------------------------------------------------------------------------- The results with the present main are: ------------------------------------------------------------------------------------------------------------------------------------------------------- Min (s) Median (s) 90%ile (s) 99%ile (s) Max (s) Mean (s) Total (s) Count Failed ------------------------------------------------------------------------------------------------------------------------------------------------------- Iteration Total 4.377260 6.486962 7.502040 8.322587 8.334701 6.559002 819.875306 125 0 Namespace.add_ports 0.005112 0.005484 0.005953 0.009153 0.011452 0.005662 0.707752 125 0 WorkerNode.bind_port 0.035360 0.042732 0.049152 0.053698 0.056635 0.043215 10.803700 250 0 WorkerNode.ping_port 0.005338 1.599904 7.229649 7.798039 8.206537 3.209860 802.464911 250 0 ------------------------------------------------------------------------------------------------------------------------------------------------------- Few observations: - The total time taken has come down significantly from 819 seconds to 197. - 99%ile with these patches is 3.3 seconds compared to 8.3 seconds for the main. - 90%file with these patches is 3.3 seconds compared to 7.5 seconds for the main. - CPU utilization of northd during the test with these patches is between 100% to 300% which is almost the same as main. Main difference being that, with these patches the test duration is less and hence overall less CPU utilization. [1] - https://github.com/ovn-org/ovn-heater/blob/main/test-scenarios/ocp-500-density-heavy.yml Signed-off-by: Numan Siddique Reviewed-by: Ales Musil --- lib/lb.c | 46 +++++- lib/lb.h | 9 ++ northd/en-lb-data.c | 328 +++++++++++++++++++++++++++++++-------- northd/en-lb-data.h | 15 ++ northd/en-northd.c | 26 ++++ northd/en-northd.h | 1 + northd/inc-proc-northd.c | 5 +- northd/northd.c | 242 ++++++++++++++++++++++++++--- northd/northd.h | 3 + tests/ovn-northd.at | 42 ++--- 10 files changed, 604 insertions(+), 113 deletions(-) diff --git a/lib/lb.c b/lib/lb.c index e6c9dc2be2..d0d562b6fb 100644 --- a/lib/lb.c +++ b/lib/lb.c @@ -794,6 +794,7 @@ ovn_lb_group_init(struct ovn_lb_group *lb_group, const struct uuid *lb_uuid = &nbrec_lb_group->load_balancer[i]->header_.uuid; lb_group->lbs[i] = ovn_northd_lb_find(lbs, lb_uuid); + lb_group->has_routable_lb |= lb_group->lbs[i]->routable; } } @@ -815,6 +816,7 @@ ovn_lb_group_cleanup(struct ovn_lb_group *lb_group) { ovn_lb_ip_set_destroy(lb_group->lb_ips); lb_group->lb_ips = NULL; + lb_group->has_routable_lb = false; free(lb_group->lbs); } @@ -1022,23 +1024,54 @@ ovn_lb_5tuples_destroy(struct hmap *tuples) void build_lrouter_lb_ips(struct ovn_lb_ip_set *lb_ips, const struct ovn_northd_lb *lb) +{ + add_ips_to_lb_ip_set(lb_ips, lb->routable, &lb->ips_v4, &lb->ips_v6); +} + +void +add_ips_to_lb_ip_set(struct ovn_lb_ip_set *lb_ips, + bool is_routable, + const struct sset *lb_ips_v4, + const struct sset *lb_ips_v6) { const char *ip_address; - SSET_FOR_EACH (ip_address, &lb->ips_v4) { + SSET_FOR_EACH (ip_address, lb_ips_v4) { sset_add(&lb_ips->ips_v4, ip_address); - if (lb->routable) { + if (is_routable) { sset_add(&lb_ips->ips_v4_routable, ip_address); } } - SSET_FOR_EACH (ip_address, &lb->ips_v6) { + SSET_FOR_EACH (ip_address, lb_ips_v6) { sset_add(&lb_ips->ips_v6, ip_address); - if (lb->routable) { + if (is_routable) { sset_add(&lb_ips->ips_v6_routable, ip_address); } } } +void +remove_ips_from_lb_ip_set(struct ovn_lb_ip_set *lb_ips, + bool is_routable, + const struct sset *lb_ips_v4, + const struct sset *lb_ips_v6) +{ + const char *ip_address; + + SSET_FOR_EACH (ip_address, lb_ips_v4) { + sset_find_and_delete(&lb_ips->ips_v4, ip_address); + if (is_routable) { + sset_find_and_delete(&lb_ips->ips_v4_routable, ip_address); + } + } + SSET_FOR_EACH (ip_address, lb_ips_v6) { + sset_find_and_delete(&lb_ips->ips_v6, ip_address); + if (is_routable) { + sset_find_and_delete(&lb_ips->ips_v6_routable, ip_address); + } + } +} + /* lb datapaths functions */ struct ovn_lb_datapaths * ovn_lb_datapaths_create(const struct ovn_northd_lb *lb, size_t n_ls_datapaths, @@ -1079,7 +1112,10 @@ ovn_lb_datapaths_add_lr(struct ovn_lb_datapaths *lb_dps, size_t n, struct ovn_datapath **ods) { for (size_t i = 0; i < n; i++) { - bitmap_set1(lb_dps->nb_lr_map, ods[i]->index); + if (!bitmap_is_set(lb_dps->nb_lr_map, ods[i]->index)) { + bitmap_set1(lb_dps->nb_lr_map, ods[i]->index); + lb_dps->n_nb_lr++; + } } } diff --git a/lib/lb.h b/lib/lb.h index 74905c73b7..b8e3c1e8fb 100644 --- a/lib/lb.h +++ b/lib/lb.h @@ -138,6 +138,14 @@ void ovn_northd_lb_reinit(struct ovn_northd_lb *, void build_lrouter_lb_ips(struct ovn_lb_ip_set *, const struct ovn_northd_lb *); +void add_ips_to_lb_ip_set(struct ovn_lb_ip_set *lb_ips, + bool is_routable, + const struct sset *lb_ips_v4, + const struct sset *lb_ips_v6); +void remove_ips_from_lb_ip_set(struct ovn_lb_ip_set *lb_ips, + bool is_routable, + const struct sset *lb_ips_v4, + const struct sset *lb_ips_v6); struct ovn_lb_group { struct hmap_node hmap_node; @@ -145,6 +153,7 @@ struct ovn_lb_group { size_t n_lbs; struct ovn_northd_lb **lbs; struct ovn_lb_ip_set *lb_ips; + bool has_routable_lb; }; struct ovn_lb_group *ovn_lb_group_create( diff --git a/northd/en-lb-data.c b/northd/en-lb-data.c index 8619b4cc14..80cb959a47 100644 --- a/northd/en-lb-data.c +++ b/northd/en-lb-data.c @@ -40,20 +40,30 @@ static void build_lbs(const struct nbrec_load_balancer_table *, const struct nbrec_load_balancer_group_table *, struct hmap *lbs, struct hmap *lb_groups); static void build_od_lb_map(const struct nbrec_logical_switch_table *, - struct hmap *od_lb_map); + const struct nbrec_logical_router_table *, + struct hmap *ls_lb_map, struct hmap *lr_lb_map); static struct od_lb_data *find_od_lb_data(struct hmap *od_lb_map, const struct uuid *od_uuid); static void destroy_od_lb_data(struct od_lb_data *od_lb_data); static struct od_lb_data *create_od_lb_data(struct hmap *od_lb_map, const struct uuid *od_uuid); +static void handle_od_lb_changes(struct nbrec_load_balancer **, + size_t n_nbrec_lbs, + struct od_lb_data *od_lb_data, + struct ed_type_lb_data *lb_data, + struct crupdated_od_lb_data *); +static void handle_od_lbgrp_changes(struct nbrec_load_balancer_group **, + size_t n_nbrec_lbs, + struct od_lb_data *, + struct ed_type_lb_data *lb_data, + struct crupdated_od_lb_data *); static struct ovn_lb_group *create_lb_group( const struct nbrec_load_balancer_group *, struct hmap *lbs, struct hmap *lb_groups); static void destroy_tracked_data(struct ed_type_lb_data *); -static void add_crupdated_lb_to_tracked_data(struct ovn_northd_lb *, - struct tracked_lb_data *, - bool health_checks); +static struct crupdated_lb *add_crupdated_lb_to_tracked_data( + struct ovn_northd_lb *, struct tracked_lb_data *, bool health_checks); static void add_deleted_lb_to_tracked_data(struct ovn_northd_lb *, struct tracked_lb_data *, bool health_checks); @@ -64,6 +74,8 @@ static void add_deleted_lb_group_to_tracked_data( struct ovn_lb_group *, struct tracked_lb_data *); static bool is_ls_lbs_changed(const struct nbrec_logical_switch *nbs); static bool is_ls_lbgrps_changed(const struct nbrec_logical_switch *nbs); +static bool is_lr_lbs_changed(const struct nbrec_logical_router *); +static bool is_lr_lbgrps_changed(const struct nbrec_logical_router *); /* 'lb_data' engine node manages the NB load balancers and load balancer * groups. For each NB LB, it creates 'struct ovn_northd_lb' and @@ -92,10 +104,13 @@ en_lb_data_run(struct engine_node *node, void *data) EN_OVSDB_GET(engine_get_input("NB_load_balancer_group", node)); const struct nbrec_logical_switch_table *nb_ls_table = EN_OVSDB_GET(engine_get_input("NB_logical_switch", node)); + const struct nbrec_logical_router_table *nb_lr_table = + EN_OVSDB_GET(engine_get_input("NB_logical_router", node)); lb_data->tracked = false; build_lbs(nb_lb_table, nb_lbg_table, &lb_data->lbs, &lb_data->lb_groups); - build_od_lb_map(nb_ls_table, &lb_data->ls_lb_map); + build_od_lb_map(nb_ls_table, nb_lr_table, &lb_data->ls_lb_map, + &lb_data->lr_lb_map); engine_set_node_state(node, EN_UPDATED); } @@ -137,6 +152,7 @@ lb_data_load_balancer_handler(struct engine_node *node, void *data) uuid_hash(&tracked_lb->header_.uuid)); add_crupdated_lb_to_tracked_data(lb, trk_lb_data, lb->health_checks); + trk_lb_data->has_routable_lb |= lb->routable; } else if (nbrec_load_balancer_is_deleted(tracked_lb)) { lb = ovn_northd_lb_find(&lb_data->lbs, &tracked_lb->header_.uuid); @@ -144,15 +160,44 @@ lb_data_load_balancer_handler(struct engine_node *node, void *data) hmap_remove(&lb_data->lbs, &lb->hmap_node); add_deleted_lb_to_tracked_data(lb, trk_lb_data, lb->health_checks); + trk_lb_data->has_routable_lb |= lb->routable; } else { /* Load balancer updated. */ lb = ovn_northd_lb_find(&lb_data->lbs, &tracked_lb->header_.uuid); ovs_assert(lb); bool health_checks = lb->health_checks; + struct sset old_ips_v4 = SSET_INITIALIZER(&old_ips_v4); + struct sset old_ips_v6 = SSET_INITIALIZER(&old_ips_v6); + sset_swap(&lb->ips_v4, &old_ips_v4); + sset_swap(&lb->ips_v6, &old_ips_v6); ovn_northd_lb_reinit(lb, tracked_lb); health_checks |= lb->health_checks; - add_crupdated_lb_to_tracked_data(lb, trk_lb_data, health_checks); + struct crupdated_lb *clb = add_crupdated_lb_to_tracked_data( + lb, trk_lb_data, health_checks); + trk_lb_data->has_routable_lb |= lb->routable; + + /* Determine the inserted and deleted vips and store them in + * the tracked data. */ + const char *vip; + SSET_FOR_EACH (vip, &lb->ips_v4) { + if (!sset_find_and_delete(&old_ips_v4, vip)) { + sset_add(&clb->inserted_vips_v4, vip); + } + } + + sset_swap(&old_ips_v4, &clb->deleted_vips_v4); + + SSET_FOR_EACH (vip, &lb->ips_v6) { + if (!sset_find_and_delete(&old_ips_v6, vip)) { + sset_add(&clb->inserted_vips_v6, vip); + } + } + + sset_swap(&old_ips_v6, &clb->deleted_vips_v6); + + sset_destroy(&old_ips_v4); + sset_destroy(&old_ips_v6); } } @@ -181,6 +226,8 @@ lb_data_load_balancer_group_handler(struct engine_node *node, void *data) for (size_t i = 0; i < lb_group->n_lbs; i++) { hmapx_add(&clbg->assoc_lbs, lb_group->lbs[i]); } + + trk_lb_data->has_routable_lb |= lb_group->has_routable_lb; } else if (nbrec_load_balancer_group_is_deleted(tracked_lb_group)) { struct ovn_lb_group *lb_group; lb_group = ovn_lb_group_find(&lb_data->lb_groups, @@ -188,6 +235,7 @@ lb_data_load_balancer_group_handler(struct engine_node *node, void *data) ovs_assert(lb_group); hmap_remove(&lb_data->lb_groups, &lb_group->hmap_node); add_deleted_lb_group_to_tracked_data(lb_group, trk_lb_data); + trk_lb_data->has_routable_lb |= lb_group->has_routable_lb; } else { struct ovn_lb_group *lb_group; @@ -209,6 +257,8 @@ lb_data_load_balancer_group_handler(struct engine_node *node, void *data) build_lrouter_lb_ips(lb_group->lb_ips, lb_group->lbs[i]); } + trk_lb_data->has_routable_lb |= lb_group->has_routable_lb; + struct crupdated_lb_group *clbg = add_crupdated_lb_group_to_tracked_data(lb_group, trk_lb_data); @@ -277,6 +327,7 @@ lb_data_logical_switch_handler(struct engine_node *node, void *data) if (!ls_lbs_changed && !ls_lbgrps_changed) { continue; } + changed = true; struct crupdated_od_lb_data *codlb = xzalloc(sizeof *codlb); codlb->od_uuid = nbs->header_.uuid; uuidset_init(&codlb->assoc_lbs); @@ -290,66 +341,77 @@ lb_data_logical_switch_handler(struct engine_node *node, void *data) } if (ls_lbs_changed) { - struct uuidset *pre_lb_uuids = od_lb_data->lbs; - od_lb_data->lbs = xzalloc(sizeof *od_lb_data->lbs); - uuidset_init(od_lb_data->lbs); - - for (size_t i = 0; i < nbs->n_load_balancer; i++) { - const struct uuid *lb_uuid = - &nbs->load_balancer[i]->header_.uuid; - uuidset_insert(od_lb_data->lbs, lb_uuid); - - struct uuidset_node *unode = uuidset_find(pre_lb_uuids, - lb_uuid); - - if (!unode || (nbrec_load_balancer_row_get_seqno( - nbs->load_balancer[i], - OVSDB_IDL_CHANGE_MODIFY) > 0)) { - /* Add this lb to the tracked data. */ - uuidset_insert(&codlb->assoc_lbs, lb_uuid); - changed = true; - } - - if (unode) { - uuidset_delete(pre_lb_uuids, unode); - } - } - if (!uuidset_is_empty(pre_lb_uuids)) { - trk_lb_data->has_dissassoc_lbs_from_od = true; - changed = true; - } - - uuidset_destroy(pre_lb_uuids); - free(pre_lb_uuids); + handle_od_lb_changes(nbs->load_balancer, nbs->n_load_balancer, + od_lb_data, lb_data, codlb); } if (ls_lbgrps_changed) { - struct uuidset *pre_lbgrp_uuids = od_lb_data->lbgrps; - od_lb_data->lbgrps = xzalloc(sizeof *od_lb_data->lbgrps); - uuidset_init(od_lb_data->lbgrps); - for (size_t i = 0; i < nbs->n_load_balancer_group; i++) { - const struct uuid *lbg_uuid = - &nbs->load_balancer_group[i]->header_.uuid; - uuidset_insert(od_lb_data->lbgrps, lbg_uuid); - - if (!uuidset_find_and_delete(pre_lbgrp_uuids, - lbg_uuid)) { - /* Add this lb group to the tracked data. */ - uuidset_insert(&codlb->assoc_lbgrps, lbg_uuid); - changed = true; - } - } + handle_od_lbgrp_changes(nbs->load_balancer_group, + nbs->n_load_balancer_group, + od_lb_data, lb_data, codlb); + } - if (!uuidset_is_empty(pre_lbgrp_uuids)) { - trk_lb_data->has_dissassoc_lbgrps_from_od = true; - changed = true; - } + ovs_list_insert(&trk_lb_data->crupdated_ls_lbs, &codlb->list_node); + } + } + + if (changed) { + engine_set_node_state(node, EN_UPDATED); + } + return true; +} + +bool +lb_data_logical_router_handler(struct engine_node *node, void *data) +{ + struct ed_type_lb_data *lb_data = (struct ed_type_lb_data *) data; + const struct nbrec_logical_router_table *nbrec_lr_table = + EN_OVSDB_GET(engine_get_input("NB_logical_router", node)); - uuidset_destroy(pre_lbgrp_uuids); - free(pre_lbgrp_uuids); + struct tracked_lb_data *trk_lb_data = &lb_data->tracked_lb_data; + lb_data->tracked = true; + + bool changed = false; + const struct nbrec_logical_router *nbr; + NBREC_LOGICAL_ROUTER_TABLE_FOR_EACH_TRACKED (nbr, nbrec_lr_table) { + if (nbrec_logical_router_is_deleted(nbr)) { + struct od_lb_data *od_lb_data = + find_od_lb_data(&lb_data->lr_lb_map, &nbr->header_.uuid); + if (od_lb_data) { + hmap_remove(&lb_data->lr_lb_map, &od_lb_data->hmap_node); + destroy_od_lb_data(od_lb_data); } + } else { + bool lr_lbs_changed = is_lr_lbs_changed(nbr); + bool lr_lbgrps_changed = is_lr_lbgrps_changed(nbr); + if (!lr_lbs_changed && !lr_lbgrps_changed) { + continue; + } + changed = true; + struct crupdated_od_lb_data *codlb = xzalloc(sizeof *codlb); + codlb->od_uuid = nbr->header_.uuid; + uuidset_init(&codlb->assoc_lbs); + uuidset_init(&codlb->assoc_lbgrps); - ovs_list_insert(&trk_lb_data->crupdated_ls_lbs, &codlb->list_node); + struct od_lb_data *od_lb_data = + find_od_lb_data(&lb_data->lr_lb_map, &nbr->header_.uuid); + if (!od_lb_data) { + od_lb_data = create_od_lb_data(&lb_data->lr_lb_map, + &nbr->header_.uuid); + } + + if (lr_lbs_changed) { + handle_od_lb_changes(nbr->load_balancer, nbr->n_load_balancer, + od_lb_data, lb_data, codlb); + } + + if (lr_lbgrps_changed) { + handle_od_lbgrp_changes(nbr->load_balancer_group, + nbr->n_load_balancer_group, + od_lb_data, lb_data, codlb); + } + + ovs_list_insert(&trk_lb_data->crupdated_lr_lbs, &codlb->list_node); } } @@ -366,6 +428,7 @@ lb_data_init(struct ed_type_lb_data *lb_data) hmap_init(&lb_data->lbs); hmap_init(&lb_data->lb_groups); hmap_init(&lb_data->ls_lb_map); + hmap_init(&lb_data->lr_lb_map); struct tracked_lb_data *trk_lb_data = &lb_data->tracked_lb_data; hmap_init(&trk_lb_data->crupdated_lbs); @@ -373,6 +436,7 @@ lb_data_init(struct ed_type_lb_data *lb_data) hmap_init(&trk_lb_data->crupdated_lb_groups); hmapx_init(&trk_lb_data->deleted_lb_groups); ovs_list_init(&trk_lb_data->crupdated_ls_lbs); + ovs_list_init(&trk_lb_data->crupdated_lr_lbs); } static void @@ -396,6 +460,11 @@ lb_data_destroy(struct ed_type_lb_data *lb_data) } hmap_destroy(&lb_data->ls_lb_map); + HMAP_FOR_EACH_POP (od_lb_data, hmap_node, &lb_data->lr_lb_map) { + destroy_od_lb_data(od_lb_data); + } + hmap_destroy(&lb_data->lr_lb_map); + destroy_tracked_data(lb_data); hmap_destroy(&lb_data->tracked_lb_data.crupdated_lbs); hmapx_destroy(&lb_data->tracked_lb_data.deleted_lbs); @@ -440,7 +509,8 @@ create_lb_group(const struct nbrec_load_balancer_group *nbrec_lb_group, static void build_od_lb_map(const struct nbrec_logical_switch_table *nbrec_ls_table, - struct hmap *od_lb_map) + const struct nbrec_logical_router_table *nbrec_lr_table, + struct hmap *ls_lb_map, struct hmap *lr_lb_map) { const struct nbrec_logical_switch *nbrec_ls; NBREC_LOGICAL_SWITCH_TABLE_FOR_EACH (nbrec_ls, nbrec_ls_table) { @@ -448,17 +518,35 @@ build_od_lb_map(const struct nbrec_logical_switch_table *nbrec_ls_table, continue; } - struct od_lb_data *od_lb_data = - create_od_lb_data(od_lb_map, &nbrec_ls->header_.uuid); + struct od_lb_data *ls_lb_data = + create_od_lb_data(ls_lb_map, &nbrec_ls->header_.uuid); for (size_t i = 0; i < nbrec_ls->n_load_balancer; i++) { - uuidset_insert(od_lb_data->lbs, + uuidset_insert(ls_lb_data->lbs, &nbrec_ls->load_balancer[i]->header_.uuid); } for (size_t i = 0; i < nbrec_ls->n_load_balancer_group; i++) { - uuidset_insert(od_lb_data->lbgrps, + uuidset_insert(ls_lb_data->lbgrps, &nbrec_ls->load_balancer_group[i]->header_.uuid); } } + + const struct nbrec_logical_router *nbrec_lr; + NBREC_LOGICAL_ROUTER_TABLE_FOR_EACH (nbrec_lr, nbrec_lr_table) { + if (!nbrec_lr->n_load_balancer && !nbrec_lr->n_load_balancer_group) { + continue; + } + + struct od_lb_data *lr_lb_data = + create_od_lb_data(lr_lb_map, &nbrec_lr->header_.uuid); + for (size_t i = 0; i < nbrec_lr->n_load_balancer; i++) { + uuidset_insert(lr_lb_data->lbs, + &nbrec_lr->load_balancer[i]->header_.uuid); + } + for (size_t i = 0; i < nbrec_lr->n_load_balancer_group; i++) { + uuidset_insert(lr_lb_data->lbgrps, + &nbrec_lr->load_balancer_group[i]->header_.uuid); + } + } } static struct od_lb_data * @@ -500,6 +588,84 @@ destroy_od_lb_data(struct od_lb_data *od_lb_data) free(od_lb_data); } +static void +handle_od_lb_changes(struct nbrec_load_balancer **nbrec_lbs, + size_t n_nbrec_lbs, struct od_lb_data *od_lb_data, + struct ed_type_lb_data *lb_data, + struct crupdated_od_lb_data *codlb) +{ + struct tracked_lb_data *trk_lb_data = &lb_data->tracked_lb_data; + struct uuidset *pre_lb_uuids = od_lb_data->lbs; + od_lb_data->lbs = xzalloc(sizeof *od_lb_data->lbs); + uuidset_init(od_lb_data->lbs); + + for (size_t i = 0; i < n_nbrec_lbs; i++) { + const struct uuid *lb_uuid = &nbrec_lbs[i]->header_.uuid; + uuidset_insert(od_lb_data->lbs, lb_uuid); + + struct uuidset_node *unode = uuidset_find(pre_lb_uuids, lb_uuid); + + if (!unode || (nbrec_load_balancer_row_get_seqno( + nbrec_lbs[i], OVSDB_IDL_CHANGE_MODIFY) > 0)) { + /* Add this lb to the tracked data. */ + uuidset_insert(&codlb->assoc_lbs, lb_uuid); + + if (!trk_lb_data->has_routable_lb) { + struct ovn_northd_lb *lb = ovn_northd_lb_find(&lb_data->lbs, + lb_uuid); + ovs_assert(lb); + trk_lb_data->has_routable_lb |= lb->routable; + } + } + + if (unode) { + uuidset_delete(pre_lb_uuids, unode); + } + } + + if (!uuidset_is_empty(pre_lb_uuids)) { + trk_lb_data->has_dissassoc_lbs_from_od = true; + } + + uuidset_destroy(pre_lb_uuids); + free(pre_lb_uuids); +} + +static void +handle_od_lbgrp_changes(struct nbrec_load_balancer_group **nbrec_lbgrps, + size_t n_nbrec_lbgrps, struct od_lb_data *od_lb_data, + struct ed_type_lb_data *lb_data, + struct crupdated_od_lb_data *codlb) +{ + struct tracked_lb_data *trk_lb_data = &lb_data->tracked_lb_data; + struct uuidset *pre_lbgrp_uuids = od_lb_data->lbgrps; + od_lb_data->lbgrps = xzalloc(sizeof *od_lb_data->lbgrps); + uuidset_init(od_lb_data->lbgrps); + for (size_t i = 0; i < n_nbrec_lbgrps; i++) { + const struct uuid *lbgrp_uuid = &nbrec_lbgrps[i]->header_.uuid; + uuidset_insert(od_lb_data->lbgrps, lbgrp_uuid); + + if (!uuidset_find_and_delete(pre_lbgrp_uuids, lbgrp_uuid)) { + /* Add this lb group to the tracked data. */ + uuidset_insert(&codlb->assoc_lbgrps, lbgrp_uuid); + + if (!trk_lb_data->has_routable_lb) { + struct ovn_lb_group *lbgrp = + ovn_lb_group_find(&lb_data->lb_groups, lbgrp_uuid); + ovs_assert(lbgrp); + trk_lb_data->has_routable_lb |= lbgrp->has_routable_lb; + } + } + } + + if (!uuidset_is_empty(pre_lbgrp_uuids)) { + trk_lb_data->has_dissassoc_lbgrps_from_od = true; + } + + uuidset_destroy(pre_lbgrp_uuids); + free(pre_lbgrp_uuids); +} + static void destroy_tracked_data(struct ed_type_lb_data *lb_data) { @@ -508,6 +674,7 @@ destroy_tracked_data(struct ed_type_lb_data *lb_data) lb_data->tracked_lb_data.has_dissassoc_lbs_from_lb_grops = false; lb_data->tracked_lb_data.has_dissassoc_lbs_from_od = false; lb_data->tracked_lb_data.has_dissassoc_lbgrps_from_od = false; + lb_data->tracked_lb_data.has_routable_lb = false; struct hmapx_node *node; HMAPX_FOR_EACH_SAFE (node, &lb_data->tracked_lb_data.deleted_lbs) { @@ -523,6 +690,10 @@ destroy_tracked_data(struct ed_type_lb_data *lb_data) struct crupdated_lb *clb; HMAP_FOR_EACH_POP (clb, hmap_node, &lb_data->tracked_lb_data.crupdated_lbs) { + sset_destroy(&clb->inserted_vips_v4); + sset_destroy(&clb->inserted_vips_v6); + sset_destroy(&clb->deleted_vips_v4); + sset_destroy(&clb->deleted_vips_v6); free(clb); } @@ -541,9 +712,16 @@ destroy_tracked_data(struct ed_type_lb_data *lb_data) uuidset_destroy(&codlb->assoc_lbgrps); free(codlb); } + LIST_FOR_EACH_SAFE (codlb, list_node, + &lb_data->tracked_lb_data.crupdated_lr_lbs) { + ovs_list_remove(&codlb->list_node); + uuidset_destroy(&codlb->assoc_lbs); + uuidset_destroy(&codlb->assoc_lbgrps); + free(codlb); + } } -static void +static struct crupdated_lb * add_crupdated_lb_to_tracked_data(struct ovn_northd_lb *lb, struct tracked_lb_data *tracked_lb_data, bool health_checks) @@ -552,9 +730,15 @@ add_crupdated_lb_to_tracked_data(struct ovn_northd_lb *lb, clb->lb = lb; hmap_insert(&tracked_lb_data->crupdated_lbs, &clb->hmap_node, uuid_hash(&lb->nlb->header_.uuid)); + sset_init(&clb->inserted_vips_v4); + sset_init(&clb->inserted_vips_v6); + sset_init(&clb->deleted_vips_v4); + sset_init(&clb->deleted_vips_v6); if (health_checks) { tracked_lb_data->has_health_checks = true; } + + return clb; } static void @@ -600,3 +784,17 @@ is_ls_lbgrps_changed(const struct nbrec_logical_switch *nbs) { || nbrec_logical_switch_is_updated(nbs, NBREC_LOGICAL_SWITCH_COL_LOAD_BALANCER_GROUP)); } + +static bool +is_lr_lbs_changed(const struct nbrec_logical_router *nbr) { + return ((nbrec_logical_router_is_new(nbr) && nbr->n_load_balancer) + || nbrec_logical_router_is_updated(nbr, + NBREC_LOGICAL_ROUTER_COL_LOAD_BALANCER)); +} + +static bool +is_lr_lbgrps_changed(const struct nbrec_logical_router *nbr) { + return ((nbrec_logical_router_is_new(nbr) && nbr->n_load_balancer_group) + || nbrec_logical_router_is_updated(nbr, + NBREC_LOGICAL_ROUTER_COL_LOAD_BALANCER_GROUP)); +} diff --git a/northd/en-lb-data.h b/northd/en-lb-data.h index bc09ddb7eb..28b7367b51 100644 --- a/northd/en-lb-data.h +++ b/northd/en-lb-data.h @@ -6,6 +6,7 @@ #include "openvswitch/hmap.h" #include "include/openvswitch/list.h" #include "lib/hmapx.h" +#include "lib/sset.h" #include "lib/uuidset.h" #include "lib/inc-proc-eng.h" @@ -17,8 +18,13 @@ struct crupdated_lb { struct hmap_node hmap_node; struct ovn_northd_lb *lb; + struct sset inserted_vips_v4; + struct sset inserted_vips_v6; + struct sset deleted_vips_v4; + struct sset deleted_vips_v6; }; + struct crupdated_lb_group { struct hmap_node hmap_node; @@ -54,6 +60,10 @@ struct tracked_lb_data { * 'struct crupdated_od_lb_data' */ struct ovs_list crupdated_ls_lbs; + /* List of logical router <-> lb changes. List node is + * 'struct crupdated_od_lb_data' */ + struct ovs_list crupdated_lr_lbs; + /* Indicates if any of the tracked lb has health checks enabled. */ bool has_health_checks; @@ -66,6 +76,9 @@ struct tracked_lb_data { /* Indicates if a lb group was disassociated from a logical switch. */ bool has_dissassoc_lbgrps_from_od; + + /* Indicates if any lb (in the tracked data) has 'routable' flag set. */ + bool has_routable_lb; }; /* struct which maintains the data of the engine node lb_data. */ @@ -76,6 +89,7 @@ struct ed_type_lb_data { /* hmap of load balancer groups. hmap node is 'struct ovn_lb_group *' */ struct hmap lb_groups; struct hmap ls_lb_map; + struct hmap lr_lb_map; /* tracked data*/ bool tracked; @@ -90,5 +104,6 @@ void en_lb_data_clear_tracked_data(void *data); bool lb_data_load_balancer_handler(struct engine_node *, void *data); bool lb_data_load_balancer_group_handler(struct engine_node *, void *data); bool lb_data_logical_switch_handler(struct engine_node *, void *data); +bool lb_data_logical_router_handler(struct engine_node *, void *data); #endif /* end of EN_NORTHD_LB_DATA_H */ diff --git a/northd/en-northd.c b/northd/en-northd.c index 545971f76f..3be0f79e19 100644 --- a/northd/en-northd.c +++ b/northd/en-northd.c @@ -206,6 +206,26 @@ northd_sb_port_binding_handler(struct engine_node *node, return true; } +bool +northd_nb_logical_router_handler(struct engine_node *node, + void *data) +{ + struct northd_data *nd = data; + struct northd_input input_data; + + northd_get_input_data(node, &input_data); + + if (!northd_handle_lr_changes(&input_data, nd)) { + return false; + } + + if (nd->change_tracked) { + engine_set_node_state(node, EN_UPDATED); + } + + return true; +} + bool northd_lb_data_handler_pre_od(struct engine_node *node, void *data) { @@ -240,6 +260,10 @@ northd_lb_data_handler_pre_od(struct engine_node *node, void *data) return false; } + if (lb_data->tracked_lb_data.has_routable_lb) { + return false; + } + struct northd_data *nd = data; if (!northd_handle_lb_data_changes_pre_od(&lb_data->tracked_lb_data, @@ -264,11 +288,13 @@ northd_lb_data_handler_post_od(struct engine_node *node, void *data) ovs_assert(!lb_data->tracked_lb_data.has_dissassoc_lbs_from_od); ovs_assert(!lb_data->tracked_lb_data.has_dissassoc_lbgrps_from_od); ovs_assert(!lb_data->tracked_lb_data.has_dissassoc_lbs_from_lb_grops); + ovs_assert(!lb_data->tracked_lb_data.has_routable_lb); struct northd_data *nd = data; if (!northd_handle_lb_data_changes_post_od(&lb_data->tracked_lb_data, &nd->ls_datapaths, + &nd->lr_datapaths, &nd->lb_datapaths_map, &nd->lb_group_datapaths_map)) { return false; diff --git a/northd/en-northd.h b/northd/en-northd.h index 5926e7a9d3..84d8673e1b 100644 --- a/northd/en-northd.h +++ b/northd/en-northd.h @@ -16,6 +16,7 @@ void en_northd_cleanup(void *data); void en_northd_clear_tracked_data(void *data); bool northd_nb_nb_global_handler(struct engine_node *, void *data OVS_UNUSED); bool northd_nb_logical_switch_handler(struct engine_node *, void *data); +bool northd_nb_logical_router_handler(struct engine_node *, void *data); bool northd_sb_port_binding_handler(struct engine_node *, void *data); bool northd_lb_data_handler_pre_od(struct engine_node *, void *data); bool northd_lb_data_handler_post_od(struct engine_node *, void *data); diff --git a/northd/inc-proc-northd.c b/northd/inc-proc-northd.c index dc8b880fd8..9dbc2ec81a 100644 --- a/northd/inc-proc-northd.c +++ b/northd/inc-proc-northd.c @@ -155,10 +155,11 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, lb_data_load_balancer_group_handler); engine_add_input(&en_lb_data, &en_nb_logical_switch, lb_data_logical_switch_handler); + engine_add_input(&en_lb_data, &en_nb_logical_router, + lb_data_logical_router_handler); engine_add_input(&en_northd, &en_nb_port_group, NULL); engine_add_input(&en_northd, &en_nb_acl, NULL); - engine_add_input(&en_northd, &en_nb_logical_router, NULL); engine_add_input(&en_northd, &en_nb_mirror, NULL); engine_add_input(&en_northd, &en_nb_meter, NULL); engine_add_input(&en_northd, &en_nb_static_mac_binding, NULL); @@ -186,6 +187,8 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, engine_add_input(&en_northd, &en_lb_data, northd_lb_data_handler_pre_od); engine_add_input(&en_northd, &en_nb_logical_switch, northd_nb_logical_switch_handler); + engine_add_input(&en_northd, &en_nb_logical_router, + northd_nb_logical_router_handler); engine_add_input(&en_northd, &en_lb_data, northd_lb_data_handler_post_od); engine_add_input(&en_mac_binding_aging, &en_nb_nb_global, NULL); diff --git a/northd/northd.c b/northd/northd.c index a3bd21e0b4..41bca6a432 100644 --- a/northd/northd.c +++ b/northd/northd.c @@ -4127,54 +4127,63 @@ static bool lrouter_port_ipv4_reachable(const struct ovn_port *op, ovs_be32 addr); static bool lrouter_port_ipv6_reachable(const struct ovn_port *op, const struct in6_addr *addr); + static void -build_lrouter_lb_reachable_ips(struct ovn_datapath *od, - const struct ovn_northd_lb *lb) +add_neigh_ips_to_lrouter(struct ovn_datapath *od, + enum lb_neighbor_responder_mode neigh_mode, + const struct sset *lb_ips_v4, + const struct sset *lb_ips_v6) { /* If configured to not reply to any neighbor requests for all VIPs * return early. */ - if (lb->neigh_mode == LB_NEIGH_RESPOND_NONE) { + if (neigh_mode == LB_NEIGH_RESPOND_NONE) { return; } + const char *ip_address; + /* If configured to reply to neighbor requests for all VIPs force them * all to be considered "reachable". */ - if (lb->neigh_mode == LB_NEIGH_RESPOND_ALL) { - for (size_t i = 0; i < lb->n_vips; i++) { - if (lb->vips[i].address_family == AF_INET) { - sset_add(&od->lb_ips->ips_v4_reachable, lb->vips[i].vip_str); - } else { - sset_add(&od->lb_ips->ips_v6_reachable, lb->vips[i].vip_str); - } + if (neigh_mode == LB_NEIGH_RESPOND_ALL) { + SSET_FOR_EACH (ip_address, lb_ips_v4) { + sset_add(&od->lb_ips->ips_v4_reachable, ip_address); + } + SSET_FOR_EACH (ip_address, lb_ips_v6) { + sset_add(&od->lb_ips->ips_v6_reachable, ip_address); } + return; } /* Otherwise, a VIP is reachable if there's at least one router * subnet that includes it. */ - ovs_assert(lb->neigh_mode == LB_NEIGH_RESPOND_REACHABLE); - for (size_t i = 0; i < lb->n_vips; i++) { - if (lb->vips[i].address_family == AF_INET) { - ovs_be32 vip_ip4 = in6_addr_get_mapped_ipv4(&lb->vips[i].vip); - struct ovn_port *op; + ovs_assert(neigh_mode == LB_NEIGH_RESPOND_REACHABLE); + SSET_FOR_EACH (ip_address, lb_ips_v4) { + struct ovn_port *op; + ovs_be32 vip_ip4; + if (ip_parse(ip_address, &vip_ip4)) { HMAP_FOR_EACH (op, dp_node, &od->ports) { if (lrouter_port_ipv4_reachable(op, vip_ip4)) { sset_add(&od->lb_ips->ips_v4_reachable, - lb->vips[i].vip_str); + ip_address); break; } } - } else { - struct ovn_port *op; + } + } + SSET_FOR_EACH (ip_address, lb_ips_v6) { + struct ovn_port *op; + struct in6_addr vip; + if (ipv6_parse(ip_address, &vip)) { HMAP_FOR_EACH (op, dp_node, &od->ports) { - if (lrouter_port_ipv6_reachable(op, &lb->vips[i].vip)) { + if (lrouter_port_ipv6_reachable(op, &vip)) { sset_add(&od->lb_ips->ips_v6_reachable, - lb->vips[i].vip_str); + ip_address); break; } } @@ -4182,6 +4191,34 @@ build_lrouter_lb_reachable_ips(struct ovn_datapath *od, } } +static void +remove_lrouter_lb_reachable_ips(struct ovn_datapath *od, + enum lb_neighbor_responder_mode neigh_mode, + const struct sset *lb_ips_v4, + const struct sset *lb_ips_v6) +{ + if (neigh_mode == LB_NEIGH_RESPOND_NONE) { + return; + } + + const char *ip_address; + SSET_FOR_EACH (ip_address, lb_ips_v4) { + sset_find_and_delete(&od->lb_ips->ips_v4_reachable, ip_address); + } + SSET_FOR_EACH (ip_address, lb_ips_v6) { + sset_find_and_delete(&od->lb_ips->ips_v6_reachable, ip_address); + } +} + +static void +build_lrouter_lb_reachable_ips(struct ovn_datapath *od, + const struct ovn_northd_lb *lb) +{ + add_neigh_ips_to_lrouter(od, lb->neigh_mode, &lb->ips_v4, + &lb->ips_v6); +} + + static void build_lrouter_lbs_check(const struct ovn_datapaths *lr_datapaths) { @@ -5382,6 +5419,95 @@ fail: return false; } +/* Returns true if the logical router has changes which are not + * incrementally handled. + * Presently supports i-p for the below changes: + * - load balancers and load balancer groups. + */ +static bool +check_unsupported_inc_proc_for_lr_changes( + const struct nbrec_logical_router *lr) +{ + /* Check if the columns are changed in this row. */ + enum nbrec_logical_router_column_id col; + for (col = 0; col < NBREC_LOGICAL_ROUTER_N_COLUMNS; col++) { + if (nbrec_logical_router_is_updated(lr, col)) { + if (col == NBREC_LOGICAL_ROUTER_COL_LOAD_BALANCER || + col == NBREC_LOGICAL_ROUTER_COL_LOAD_BALANCER_GROUP) { + continue; + } + return true; + } + } + + /* Check if the referenced rows are changed. + XXX: Need a better OVSDB IDL interface for this check. */ + for (size_t i = 0; i < lr->n_ports; i++) { + if (nbrec_logical_router_port_row_get_seqno(lr->ports[i], + OVSDB_IDL_CHANGE_MODIFY) > 0) { + return true; + } + } + if (lr->copp && nbrec_copp_row_get_seqno(lr->copp, + OVSDB_IDL_CHANGE_MODIFY) > 0) { + return true; + } + for (size_t i = 0; i < lr->n_nat; i++) { + if (nbrec_nat_row_get_seqno(lr->nat[i], + OVSDB_IDL_CHANGE_MODIFY) > 0) { + return true; + } + } + for (size_t i = 0; i < lr->n_policies; i++) { + if (nbrec_logical_router_policy_row_get_seqno(lr->policies[i], + OVSDB_IDL_CHANGE_MODIFY) > 0) { + return true; + } + } + for (size_t i = 0; i < lr->n_static_routes; i++) { + if (nbrec_logical_router_static_route_row_get_seqno( + lr->static_routes[i], OVSDB_IDL_CHANGE_MODIFY) > 0) { + return true; + } + } + return false; +} + +/* Return true if changes are handled incrementally, false otherwise. + * When there are any changes, try to track what's exactly changed and set + * northd_data->change_tracked accordingly: change tracked - true, otherwise, + * false. + * Note: Changes to load balancer and load balancer groups associated with + * the logical routers are handled separately in the lb_data change + * handlers (northd_handle_lb_data_changes_pre_od and + * northd_handle_lb_data_changes_post_od). + * */ +bool +northd_handle_lr_changes(const struct northd_input *ni, + struct northd_data *nd) +{ + const struct nbrec_logical_router *changed_lr; + + NBREC_LOGICAL_ROUTER_TABLE_FOR_EACH_TRACKED (changed_lr, + ni->nbrec_logical_router_table) { + if (nbrec_logical_router_is_new(changed_lr) || + nbrec_logical_router_is_deleted(changed_lr)) { + goto fail; + } + + /* Presently only able to handle load balancer and + * load balancer group changes. */ + if (check_unsupported_inc_proc_for_lr_changes(changed_lr)) { + goto fail; + } + } + + return true; +fail: + destroy_northd_data_tracked_changes(nd); + return false; +} + bool northd_handle_sb_port_binding_changes( const struct sbrec_port_binding_table *sbrec_port_binding_table, @@ -5530,6 +5656,7 @@ northd_handle_lb_data_changes_pre_od(struct tracked_lb_data *trk_lb_data, bool northd_handle_lb_data_changes_post_od(struct tracked_lb_data *trk_lb_data, struct ovn_datapaths *ls_datapaths, + struct ovn_datapaths *lr_datapaths, struct hmap *lb_datapaths_map, struct hmap *lbgrp_datapaths_map) { @@ -5573,6 +5700,45 @@ northd_handle_lb_data_changes_post_od(struct tracked_lb_data *trk_lb_data, init_lb_for_datapath(od); } + LIST_FOR_EACH (codlb, list_node, &trk_lb_data->crupdated_lr_lbs) { + od = ovn_datapath_find(&lr_datapaths->datapaths, &codlb->od_uuid); + ovs_assert(od); + + struct uuidset_node *uuidnode; + UUIDSET_FOR_EACH (uuidnode, &codlb->assoc_lbs) { + lb_dps = ovn_lb_datapaths_find(lb_datapaths_map, &uuidnode->uuid); + ovs_assert(lb_dps); + ovn_lb_datapaths_add_lr(lb_dps, 1, &od); + + /* Add the lb_ips of lb_dps to the od. */ + build_lrouter_lb_ips(od->lb_ips, lb_dps->lb); + build_lrouter_lb_reachable_ips(od, lb_dps->lb); + } + + UUIDSET_FOR_EACH (uuidnode, &codlb->assoc_lbgrps) { + lbgrp_dps = ovn_lb_group_datapaths_find(lbgrp_datapaths_map, + &uuidnode->uuid); + ovs_assert(lbgrp_dps); + ovn_lb_group_datapaths_add_lr(lbgrp_dps, od); + + /* Associate all the lbs of the lbgrp to the datapath 'od' */ + for (size_t j = 0; j < lbgrp_dps->lb_group->n_lbs; j++) { + const struct uuid *lb_uuid + = &lbgrp_dps->lb_group->lbs[j]->nlb->header_.uuid; + lb_dps = ovn_lb_datapaths_find(lb_datapaths_map, lb_uuid); + ovs_assert(lb_dps); + ovn_lb_datapaths_add_lr(lb_dps, 1, &od); + + /* Add the lb_ips of lb_dps to the od. */ + build_lrouter_lb_ips(od->lb_ips, lb_dps->lb); + build_lrouter_lb_reachable_ips(od, lb_dps->lb); + } + } + + /* Re-evaluate 'od->has_lb_vip' */ + init_lb_for_datapath(od); + } + struct crupdated_lb *clb; HMAP_FOR_EACH (clb, hmap_node, &trk_lb_data->crupdated_lbs) { lb = clb->lb; @@ -5587,6 +5753,29 @@ northd_handle_lb_data_changes_post_od(struct tracked_lb_data *trk_lb_data, /* Re-evaluate 'od->has_lb_vip' */ init_lb_for_datapath(od); } + + BITMAP_FOR_EACH_1 (index, ods_size(lr_datapaths), + lb_dps->nb_lr_map) { + od = lr_datapaths->array[index]; + /* Re-evaluate 'od->has_lb_vip' */ + init_lb_for_datapath(od); + + /* Update the od->lb_ips with the deleted and inserted + * vips (if any). */ + remove_ips_from_lb_ip_set(od->lb_ips, lb->routable, + &clb->deleted_vips_v4, + &clb->deleted_vips_v6); + add_ips_to_lb_ip_set(od->lb_ips, lb->routable, + &clb->inserted_vips_v4, + &clb->inserted_vips_v6); + + remove_lrouter_lb_reachable_ips(od, lb->neigh_mode, + &clb->deleted_vips_v4, + &clb->deleted_vips_v6); + add_neigh_ips_to_lrouter(od, lb->neigh_mode, + &clb->inserted_vips_v4, + &clb->inserted_vips_v6); + } } struct ovn_lb_group *lbgrp; @@ -5606,8 +5795,19 @@ northd_handle_lb_data_changes_post_od(struct tracked_lb_data *trk_lb_data, lb_uuid = &lb->nlb->header_.uuid; lb_dps = ovn_lb_datapaths_find(lb_datapaths_map, lb_uuid); ovs_assert(lb_dps); + for (size_t i = 0; i < lbgrp_dps->n_lr; i++) { + od = lbgrp_dps->lr[i]; + ovn_lb_datapaths_add_lr(lb_dps, 1, &od); + + /* Re-evaluate 'od->has_lb_vip' */ + init_lb_for_datapath(od); + + /* Add the lb_ips of lb_dps to the od. */ + build_lrouter_lb_ips(od->lb_ips, lb_dps->lb); + } + for (size_t i = 0; i < lbgrp_dps->n_ls; i++) { - od = lbgrp_dps->ls[i]; + od = lbgrp_dps->ls[i]; ovn_lb_datapaths_add_ls(lb_dps, 1, &od); /* Re-evaluate 'od->has_lb_vip' */ diff --git a/northd/northd.h b/northd/northd.h index cd2e5394c2..1d344d57d6 100644 --- a/northd/northd.h +++ b/northd/northd.h @@ -335,6 +335,8 @@ void ovnsb_db_run(struct ovsdb_idl_txn *ovnnb_txn, bool northd_handle_ls_changes(struct ovsdb_idl_txn *, const struct northd_input *, struct northd_data *); +bool northd_handle_lr_changes(const struct northd_input *, + struct northd_data *); void destroy_northd_data_tracked_changes(struct northd_data *); void northd_destroy(struct northd_data *data); void northd_init(struct northd_data *data); @@ -357,6 +359,7 @@ bool northd_handle_lb_data_changes_pre_od(struct tracked_lb_data *, struct hmap *lbgrp_datapaths_map); bool northd_handle_lb_data_changes_post_od(struct tracked_lb_data *, struct ovn_datapaths *ls_datapaths, + struct ovn_datapaths *lr_datapaths, struct hmap *lb_datapaths_map, struct hmap *lbgrp_datapaths_map); diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at index b8b2ee390e..234d5a8bbd 100644 --- a/tests/ovn-northd.at +++ b/tests/ovn-northd.at @@ -9947,8 +9947,8 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE # Add lb1 to lr0 and then disassociate check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lr-lb-add lr0 lb1 -check_engine_stats lb_data norecompute nocompute -check_engine_stats northd recompute nocompute +check_engine_stats lb_data norecompute compute +check_engine_stats northd norecompute compute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -9956,7 +9956,7 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb set load_balancer lb1 vips:'"10.0.0.10:80"'='"10.0.0.100:80"' check_engine_stats lb_data norecompute compute -check_engine_stats northd recompute nocompute +check_engine_stats northd norecompute compute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -9964,7 +9964,7 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb clear load_Balancer lb1 vips check_engine_stats lb_data norecompute compute -check_engine_stats northd recompute nocompute +check_engine_stats northd norecompute compute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -9972,7 +9972,7 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lb-add lb1 10.0.0.10:80 10.0.0.3:80 check_engine_stats lb_data norecompute compute -check_engine_stats northd recompute nocompute +check_engine_stats northd norecompute compute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -9980,13 +9980,13 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lb-add lb1 10.0.0.20:80 10.0.0.30:8080 check_engine_stats lb_data norecompute compute -check_engine_stats northd recompute nocompute +check_engine_stats northd norecompute compute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lr-lb-del lr0 lb1 -check_engine_stats lb_data norecompute nocompute +check_engine_stats lb_data norecompute compute check_engine_stats northd recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10078,8 +10078,8 @@ check_engine_stats lflow recompute nocompute check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl add logical_router lr0 load_balancer_group $lbg1_uuid -check_engine_stats lb_data norecompute nocompute -check_engine_stats northd recompute nocompute +check_engine_stats lb_data norecompute compute +check_engine_stats northd norecompute compute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10087,7 +10087,7 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb set load_balancer lb1 vips:'"10.0.0.10:80"'='"10.0.0.100:80"' check_engine_stats lb_data norecompute compute -check_engine_stats northd recompute nocompute +check_engine_stats northd norecompute compute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10095,7 +10095,7 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb clear load_Balancer lb1 vips check_engine_stats lb_data norecompute compute -check_engine_stats northd recompute nocompute +check_engine_stats northd norecompute compute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10103,7 +10103,7 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lb-add lb1 10.0.0.10:80 10.0.0.3:80 check_engine_stats lb_data norecompute compute -check_engine_stats northd recompute nocompute +check_engine_stats northd norecompute compute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10111,13 +10111,13 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lb-add lb1 10.0.0.20:80 10.0.0.30:8080 check_engine_stats lb_data norecompute compute -check_engine_stats northd recompute nocompute +check_engine_stats northd norecompute compute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl clear logical_router lr0 load_balancer_group -check_engine_stats lb_data norecompute nocompute +check_engine_stats lb_data norecompute compute check_engine_stats northd recompute nocompute check_engine_stats lflow recompute nocompute @@ -10172,8 +10172,8 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl set logical_router lr1 load_balancer_group=$lbg1_uuid -check_engine_stats lb_data norecompute nocompute -check_engine_stats northd recompute nocompute +check_engine_stats lb_data norecompute compute +check_engine_stats northd norecompute compute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10194,8 +10194,8 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lr-lb-add lr1 lb1 check ovn-nbctl --wait=sb lr-lb-add lr1 lb2 -check_engine_stats lb_data norecompute nocompute -check_engine_stats northd recompute nocompute +check_engine_stats lb_data norecompute compute +check_engine_stats northd norecompute compute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10208,7 +10208,7 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lr-lb-del lr1 lb2 -check_engine_stats lb_data norecompute nocompute +check_engine_stats lb_data norecompute compute check_engine_stats northd recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10218,7 +10218,7 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lb-del lb4 check_engine_stats lb_data norecompute compute -check_engine_stats northd recompute nocompute +check_engine_stats northd norecompute compute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10227,7 +10227,7 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lb-del lb2 check_engine_stats lb_data norecompute compute -check_engine_stats northd recompute nocompute +check_engine_stats northd norecompute compute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE From patchwork Fri Aug 18 08:58:08 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Numan Siddique X-Patchwork-Id: 1822760 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::138; helo=smtp1.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp1.osuosl.org (smtp1.osuosl.org [IPv6:2605:bc80:3010::138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4RRwm23qW6z1ygH for ; Fri, 18 Aug 2023 18:59:14 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id C99CB83F78; Fri, 18 Aug 2023 08:59:12 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org C99CB83F78 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id YwW-yXXaLr6E; Fri, 18 Aug 2023 08:59:09 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp1.osuosl.org (Postfix) with ESMTPS id B450183F6D; Fri, 18 Aug 2023 08:59:07 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org B450183F6D Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 5A00AC0072; Fri, 18 Aug 2023 08:59:06 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp4.osuosl.org (smtp4.osuosl.org [IPv6:2605:bc80:3010::137]) by lists.linuxfoundation.org (Postfix) with ESMTP id 2C5ADC0032 for ; Fri, 18 Aug 2023 08:59:04 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id C18DD4266A for ; Fri, 18 Aug 2023 08:58:20 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org C18DD4266A X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 7lOWxSmO14Nv for ; Fri, 18 Aug 2023 08:58:18 +0000 (UTC) Received: from relay3-d.mail.gandi.net (relay3-d.mail.gandi.net [217.70.183.195]) by smtp4.osuosl.org (Postfix) with ESMTPS id 4D9E141E7F for ; Fri, 18 Aug 2023 08:58:17 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 4D9E141E7F Received: by mail.gandi.net (Postfix) with ESMTPSA id 2FB8D6000A; Fri, 18 Aug 2023 08:58:12 +0000 (UTC) From: numans@ovn.org To: dev@openvswitch.org Date: Fri, 18 Aug 2023 14:28:08 +0530 Message-Id: <20230818085808.1031041-1-numans@ovn.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230818085606.1030792-1-numans@ovn.org> References: <20230818085606.1030792-1-numans@ovn.org> MIME-Version: 1.0 X-GND-Sasl: numans@ovn.org Cc: Dumitru Ceara Subject: [ovs-dev] [PATCH ovn v6 09/16] northd: Use objdep mgr for lport to lflow references. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Numan Siddique Instead of maintaing the lport to lflow references using 'struct lflow_ref_list', this patch makes use of the existing objdep APIs. Since objdep_mgr is not thread safe, an instance of objdep_mgr is maintained for each 'ovn_port'. Once we add the thread safe support to objdep APIs we can move the objdep_mgr to 'lflow' engine date. Using objdep APIs does come with a cost in memory. We now maintain an additional hmap of ovn_lflow's to look up using uuid. But this will be useful to handle datapath and load balancer changes in the lflow engine node. This patch does few more changes which are significant: - Logical flows are now hashed without the logical datapaths. If a logical flow is referenced by just one datapath, we don't rehash it. - The synthetic 'hash' column of sbrec_logical_flow now doesn't use the logical datapath too. This means that when ovn-northd is updated/upgraded and has this commit, all the logical flows with 'logical_datapath' column set will get deleted and re-added causing some disruptions. - With the commit [1] which added I-P support for logical port changes, multiple logical flows with same match 'M' and actions 'A' are generated and stored without the dp groups, which was not the case prior to that patch. One example to generate these lflows is: ovn-nbctl lsp-set-addresses sw0p1 "MAC1 IP1" ovn-nbctl lsp-set-addresses sw1p1 "MAC1 IP1" ovn-nbctl lsp-set-addresses sw2p1 "MAC1 IP1" Now with this patch we go back to the earlier way. i.e one logical flow with logical_dp_groups set. - With this patch any updates to a logical port which doesn't result in new logical flows will not result in deletion and addition of same logical flows. Eg. ovn-nbctl set logical_switch_port sw0p1 external_ids:foo=bar will be a no-op to the SB logical flow table. [1] - 8bbd67891f68("northd: Incremental processing of VIF additions in 'lflow' node.") Suggested-by: Dumitru Ceara Signed-off-by: Numan Siddique --- lib/objdep.h | 2 + lib/ovn-util.c | 11 +- northd/en-lflow.c | 9 +- northd/inc-proc-northd.c | 5 +- northd/northd.c | 1690 ++++++++++++++++++++++---------------- northd/northd.h | 11 +- northd/ovn-northd.c | 4 + 7 files changed, 993 insertions(+), 739 deletions(-) diff --git a/lib/objdep.h b/lib/objdep.h index 1ea781947c..d599128ea3 100644 --- a/lib/objdep.h +++ b/lib/objdep.h @@ -27,6 +27,8 @@ enum objdep_type { OBJDEP_TYPE_PORTBINDING, OBJDEP_TYPE_MC_GROUP, OBJDEP_TYPE_TEMPLATE, + OBJDEP_TYPE_LPORT, + OBJDEP_TYPE_LFLOW_OD, OBJDEP_TYPE_MAX, }; diff --git a/lib/ovn-util.c b/lib/ovn-util.c index 3a237b7fe3..4f1b04d952 100644 --- a/lib/ovn-util.c +++ b/lib/ovn-util.c @@ -626,13 +626,10 @@ ovn_pipeline_from_name(const char *pipeline) uint32_t sbrec_logical_flow_hash(const struct sbrec_logical_flow *lf) { - const struct sbrec_datapath_binding *ld = lf->logical_datapath; - uint32_t hash = ovn_logical_flow_hash(lf->table_id, - ovn_pipeline_from_name(lf->pipeline), - lf->priority, lf->match, - lf->actions); - - return ld ? ovn_logical_flow_hash_datapath(&ld->header_.uuid, hash) : hash; + return ovn_logical_flow_hash(lf->table_id, + ovn_pipeline_from_name(lf->pipeline), + lf->priority, lf->match, + lf->actions); } uint32_t diff --git a/northd/en-lflow.c b/northd/en-lflow.c index 77e2eff056..b7538d6382 100644 --- a/northd/en-lflow.c +++ b/northd/en-lflow.c @@ -45,6 +45,8 @@ lflow_get_input_data(struct engine_node *node, EN_OVSDB_GET(engine_get_input("SB_multicast_group", node)); lflow_input->sbrec_igmp_group_table = EN_OVSDB_GET(engine_get_input("SB_igmp_group", node)); + lflow_input->sbrec_logical_dp_group_table = + EN_OVSDB_GET(engine_get_input("SB_logical_dp_group", node)); lflow_input->sbrec_mcast_group_by_name_dp = engine_ovsdb_node_get_index( @@ -85,7 +87,7 @@ void en_lflow_run(struct engine_node *node, void *data) lflow_input.sbrec_bfd_table, lflow_input.lr_ports, &bfd_connections); - build_lflows(eng_ctx->ovnsb_idl_txn, &lflow_input, &lflow_data->lflows); + build_lflows(eng_ctx->ovnsb_idl_txn, &lflow_input, lflow_data); bfd_cleanup_connections(lflow_input.nbrec_bfd_table, &bfd_connections); hmap_destroy(&bfd_connections); @@ -96,7 +98,7 @@ void en_lflow_run(struct engine_node *node, void *data) bool lflow_northd_handler(struct engine_node *node, - void *data) + void *data OVS_UNUSED) { struct northd_data *northd_data = engine_get_input_data("northd", node); if (!northd_data->change_tracked) { @@ -116,11 +118,12 @@ lflow_northd_handler(struct engine_node *node, if (!lflow_handle_northd_ls_changes(eng_ctx->ovnsb_idl_txn, &northd_data->tracked_ls_changes, - &lflow_input, &lflow_data->lflows)) { + &lflow_input, lflow_data)) { return false; } engine_set_node_state(node, EN_UPDATED); + return true; } diff --git a/northd/inc-proc-northd.c b/northd/inc-proc-northd.c index 9dbc2ec81a..e3bc2bda7b 100644 --- a/northd/inc-proc-northd.c +++ b/northd/inc-proc-northd.c @@ -95,7 +95,8 @@ static unixctl_cb_func chassis_features_list; SB_NODE(bfd, "bfd") \ SB_NODE(fdb, "fdb") \ SB_NODE(static_mac_binding, "static_mac_binding") \ - SB_NODE(chassis_template_var, "chassis_template_var") + SB_NODE(chassis_template_var, "chassis_template_var") \ + SB_NODE(logical_dp_group, "logical_dp_group") enum sb_engine_node { #define SB_NODE(NAME, NAME_STR) SB_##NAME, @@ -206,6 +207,8 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, engine_add_input(&en_lflow, &en_sb_logical_flow, NULL); engine_add_input(&en_lflow, &en_sb_multicast_group, NULL); engine_add_input(&en_lflow, &en_sb_igmp_group, NULL); + engine_add_input(&en_lflow, &en_sb_logical_dp_group, NULL); + engine_add_input(&en_lflow, &en_northd, lflow_northd_handler); engine_add_input(&en_sync_to_sb_addr_set, &en_nb_address_set, diff --git a/northd/northd.c b/northd/northd.c index 41bca6a432..9b9c6de826 100644 --- a/northd/northd.c +++ b/northd/northd.c @@ -1457,19 +1457,6 @@ struct ovn_port_routable_addresses { size_t n_addrs; }; -/* A node that maintains link between an object (such as an ovn_port) and - * a lflow. */ -struct lflow_ref_node { - /* This list follows different lflows referenced by the same object. List - * head is, for example, ovn_port->lflows. */ - struct ovs_list lflow_list_node; - /* This list follows different objects that reference the same lflow. List - * head is ovn_lflow->referenced_by. */ - struct ovs_list ref_list_node; - /* The lflow. */ - struct ovn_lflow *lflow; -}; - /* A logical switch port or logical router port. * * In steady state, an ovn_port points to a northbound Logical_Switch_Port @@ -1562,14 +1549,13 @@ struct ovn_port { /* Temporarily used for traversing a list (or hmap) of ports. */ bool visited; - /* List of struct lflow_ref_node that points to the lflows generated by - * this ovn_port. + /* objdep_mgr for the lflows generated by this ovn_port. * * This data is initialized and destroyed by the en_northd node, but * populated and used only by the en_lflow node. Ideally this data should - * be maintained as part of en_lflow's data (struct lflow_data): a hash - * index from ovn_port key to lflows. However, it would be less efficient - * and more complex: + * be maintained as part of en_lflow's data (struct lflow_data). + * However, it would require thread safe access support to objdep APIs + * and is more complex: * * 1. It would require an extra search (using the index) to find the * lflows. @@ -1578,11 +1564,11 @@ struct ovn_port { * lock which is obviously less efficient, or hash-based lock array which * is more complex. * - * Adding the list here is more straightforward. The drawback is that we - * need to keep in mind that this data belongs to en_lflow node, so never - * access it from any other nodes. + * Adding the lflow lflow_dep_mgr is more straightforward. The drawback + * is that we need to keep in mind that this data belongs to + * en_lflow node, so never access it from any other nodes. */ - struct ovs_list lflows; + struct objdep_mgr lflow_dep_mgr; }; static bool lsp_can_be_inc_processed(const struct nbrec_logical_switch_port *); @@ -1671,12 +1657,12 @@ ovn_port_create(struct hmap *ports, const char *key, op->l3dgw_port = op->cr_port = NULL; hmap_insert(ports, &op->key_node, hash_string(op->key, 0)); - ovs_list_init(&op->lflows); + objdep_mgr_init(&op->lflow_dep_mgr); return op; } static void -ovn_port_destroy_orphan(struct ovn_port *port) +ovn_port_cleanup(struct ovn_port *port) { if (port->tunnel_key) { ovs_assert(port->od); @@ -1686,6 +1672,8 @@ ovn_port_destroy_orphan(struct ovn_port *port) destroy_lport_addresses(&port->lsp_addrs[i]); } free(port->lsp_addrs); + port->n_lsp_addrs = 0; + port->lsp_addrs = NULL; if (port->peer) { port->peer->peer = NULL; @@ -1695,20 +1683,23 @@ ovn_port_destroy_orphan(struct ovn_port *port) destroy_lport_addresses(&port->ps_addrs[i]); } free(port->ps_addrs); + port->ps_addrs = NULL; + port->n_ps_addrs = 0; destroy_routable_addresses(&port->routables); destroy_lport_addresses(&port->lrp_networks); destroy_lport_addresses(&port->proxy_arp_addrs); +} + +static void +ovn_port_destroy_orphan(struct ovn_port *port) +{ + ovn_port_cleanup(port); free(port->json_key); free(port->key); + objdep_mgr_destroy(&port->lflow_dep_mgr); - struct lflow_ref_node *l; - LIST_FOR_EACH_SAFE (l, lflow_list_node, &port->lflows) { - ovs_list_remove(&l->lflow_list_node); - ovs_list_remove(&l->ref_list_node); - free(l); - } free(port); } @@ -4350,8 +4341,10 @@ build_lb_port_related_data( struct ovn_dp_group { unsigned long *bitmap; - struct sbrec_logical_dp_group *dp_group; + const struct sbrec_logical_dp_group *dp_group; + struct uuid dpg_uuid; struct hmap_node node; + size_t refcnt; }; static struct ovn_dp_group * @@ -4369,6 +4362,24 @@ ovn_dp_group_find(const struct hmap *dp_groups, return NULL; } +static inline void +inc_ovn_dp_group_ref(struct ovn_dp_group *dpg) +{ + dpg->refcnt++; +} + +static void +dec_ovn_dp_group_ref(struct hmap *dp_groups, struct ovn_dp_group *dpg) +{ + dpg->refcnt--; + + if (!dpg->refcnt) { + hmap_remove(dp_groups, &dpg->node); + free(dpg->bitmap); + free(dpg); + } +} + static struct sbrec_logical_dp_group * ovn_sb_insert_or_update_logical_dp_group( struct ovsdb_idl_txn *ovnsb_txn, @@ -4384,7 +4395,9 @@ ovn_sb_insert_or_update_logical_dp_group( sb[n++] = datapaths->array[index]->sb; } if (!dp_group) { - dp_group = sbrec_logical_dp_group_insert(ovnsb_txn); + struct uuid dpg_uuid = uuid_random(); + dp_group = sbrec_logical_dp_group_insert_persist_uuid( + ovnsb_txn, &dpg_uuid); } sbrec_logical_dp_group_set_datapaths( dp_group, (struct sbrec_datapath_binding **) sb, n); @@ -4393,12 +4406,23 @@ ovn_sb_insert_or_update_logical_dp_group( return dp_group; } -/* Given a desired bitmap, finds a datapath group in 'dp_groups'. If it - * doesn't exist, creates a new one and adds it to 'dp_groups'. +static struct ovn_dp_group * +ovn_dp_group_get(struct hmap *dp_groups, size_t desired_n, + const unsigned long *desired_bitmap, + size_t bitmap_len) +{ + uint32_t hash; + + hash = hash_int(desired_n, 0); + return ovn_dp_group_find(dp_groups, desired_bitmap, bitmap_len, hash); +} + +/* Creates a new datapath group and adds it to 'dp_groups'. * If 'sb_group' is provided, function will try to re-use this group by - * either taking it directly, or by modifying, if it's not already in use. */ + * either taking it directly, or by modifying, if it's not already in use. + * Caller should first call ovn_dp_group_get() before calling this function. */ static struct ovn_dp_group * -ovn_dp_group_get_or_create(struct ovsdb_idl_txn *ovnsb_txn, +ovn_dp_group_create(struct ovsdb_idl_txn *ovnsb_txn, struct hmap *dp_groups, struct sbrec_logical_dp_group *sb_group, size_t desired_n, @@ -4409,13 +4433,6 @@ ovn_dp_group_get_or_create(struct ovsdb_idl_txn *ovnsb_txn, const struct ovn_datapaths *lr_datapaths) { struct ovn_dp_group *dpg; - uint32_t hash; - - hash = hash_int(desired_n, 0); - dpg = ovn_dp_group_find(dp_groups, desired_bitmap, bitmap_len, hash); - if (dpg) { - return dpg; - } bool update_dp_group = false, can_modify = false; unsigned long *dpg_bitmap; @@ -4460,11 +4477,39 @@ ovn_dp_group_get_or_create(struct ovsdb_idl_txn *ovnsb_txn, desired_bitmap, is_switch ? ls_datapaths : lr_datapaths); } - hmap_insert(dp_groups, &dpg->node, hash); + dpg->dpg_uuid = dpg->dp_group->header_.uuid; + hmap_insert(dp_groups, &dpg->node, hash_int(desired_n, 0)); return dpg; } +/* Given a desired bitmap, finds a datapath group in 'dp_groups'. If it + * doesn't exist, creates a new one and adds it to 'dp_groups'. + * If 'sb_group' is provided, function will try to re-use this group by + * either taking it directly, or by modifying, if it's not already in use. */ +static struct ovn_dp_group * +ovn_dp_group_get_or_create(struct ovsdb_idl_txn *ovnsb_txn, + struct hmap *dp_groups, + struct sbrec_logical_dp_group *sb_group, + size_t desired_n, + const unsigned long *desired_bitmap, + size_t bitmap_len, + bool is_switch, + const struct ovn_datapaths *ls_datapaths, + const struct ovn_datapaths *lr_datapaths) +{ + struct ovn_dp_group *dpg; + + dpg = ovn_dp_group_get(dp_groups, desired_n, desired_bitmap, bitmap_len); + if (dpg) { + return dpg; + } + + return ovn_dp_group_create(ovnsb_txn, dp_groups, sb_group, desired_n, + desired_bitmap, bitmap_len, is_switch, + ls_datapaths, lr_datapaths); +} + struct sb_lb { struct hmap_node hmap_node; @@ -5040,28 +5085,20 @@ ovn_port_find_in_datapath(struct ovn_datapath *od, const char *name) return NULL; } -static struct ovn_port * -ls_port_create(struct ovsdb_idl_txn *ovnsb_txn, struct hmap *ls_ports, - const char *key, const struct nbrec_logical_switch_port *nbsp, - struct ovn_datapath *od, const struct sbrec_port_binding *sb, - struct ovs_list *lflows, - const struct sbrec_mirror_table *sbrec_mirror_table, - const struct sbrec_chassis_table *sbrec_chassis_table, - struct ovsdb_idl_index *sbrec_chassis_by_name, - struct ovsdb_idl_index *sbrec_chassis_by_hostname) +static bool +ls_port_init(struct ovn_port *op, struct ovsdb_idl_txn *ovnsb_txn, + struct hmap *ls_ports, struct ovn_datapath *od, + const struct sbrec_port_binding *sb, + const struct sbrec_mirror_table *sbrec_mirror_table, + const struct sbrec_chassis_table *sbrec_chassis_table, + struct ovsdb_idl_index *sbrec_chassis_by_name, + struct ovsdb_idl_index *sbrec_chassis_by_hostname) { - struct ovn_port *op = ovn_port_create(ls_ports, key, nbsp, NULL, - NULL); - parse_lsp_addrs(op); op->od = od; - hmap_insert(&od->ports, &op->dp_node, hmap_node_hash(&op->key_node)); - if (lflows) { - ovs_list_splice(&op->lflows, lflows->next, lflows); - } - + parse_lsp_addrs(op); /* Assign explicitly requested tunnel ids first. */ if (!ovn_port_assign_requested_tnl_id(sbrec_chassis_table, op)) { - return NULL; + return false; } if (sb) { op->sb = sb; @@ -5078,14 +5115,57 @@ ls_port_create(struct ovsdb_idl_txn *ovnsb_txn, struct hmap *ls_ports, } /* Assign new tunnel ids where needed. */ if (!ovn_port_allocate_key(sbrec_chassis_table, ls_ports, op)) { - return NULL; + return false; } ovn_port_update_sbrec(ovnsb_txn, sbrec_chassis_by_name, sbrec_chassis_by_hostname, NULL, sbrec_mirror_table, op, NULL, NULL); + return true; +} + +static struct ovn_port * +ls_port_create(struct ovsdb_idl_txn *ovnsb_txn, struct hmap *ls_ports, + const char *key, const struct nbrec_logical_switch_port *nbsp, + struct ovn_datapath *od, const struct sbrec_port_binding *sb, + const struct sbrec_mirror_table *sbrec_mirror_table, + const struct sbrec_chassis_table *sbrec_chassis_table, + struct ovsdb_idl_index *sbrec_chassis_by_name, + struct ovsdb_idl_index *sbrec_chassis_by_hostname) +{ + struct ovn_port *op = ovn_port_create(ls_ports, key, nbsp, NULL, + NULL); + hmap_insert(&od->ports, &op->dp_node, hmap_node_hash(&op->key_node)); + if (!ls_port_init(op, ovnsb_txn, ls_ports, od, sb, + sbrec_mirror_table, sbrec_chassis_table, + sbrec_chassis_by_name, sbrec_chassis_by_hostname)) { + ovn_port_destroy(ls_ports, op); + return NULL; + } + return op; } +static bool +ls_port_reinit(struct ovn_port *op, struct ovsdb_idl_txn *ovnsb_txn, + struct hmap *ls_ports, + const struct nbrec_logical_switch_port *nbsp, + const struct nbrec_logical_router_port *nbrp, + struct ovn_datapath *od, + const struct sbrec_port_binding *sb, + const struct sbrec_mirror_table *sbrec_mirror_table, + const struct sbrec_chassis_table *sbrec_chassis_table, + struct ovsdb_idl_index *sbrec_chassis_by_name, + struct ovsdb_idl_index *sbrec_chassis_by_hostname) +{ + ovn_port_cleanup(op); + op->sb = sb; + ovn_port_set_nb(op, nbsp, nbrp); + op->l3dgw_port = op->cr_port = NULL; + return ls_port_init(op, ovnsb_txn, ls_ports, od, sb, + sbrec_mirror_table, sbrec_chassis_table, + sbrec_chassis_by_name, sbrec_chassis_by_hostname); +} + /* Returns true if the logical switch has changes which can be * incrementally handled. * Presently supports i-p for the below changes: @@ -5243,7 +5323,7 @@ ls_handle_lsp_changes(struct ovsdb_idl_txn *ovnsb_idl_txn, goto fail; } op = ls_port_create(ovnsb_idl_txn, &nd->ls_ports, - new_nbsp->name, new_nbsp, od, NULL, NULL, + new_nbsp->name, new_nbsp, od, NULL, ni->sbrec_mirror_table, ni->sbrec_chassis_table, ni->sbrec_chassis_by_name, @@ -5275,17 +5355,12 @@ ls_handle_lsp_changes(struct ovsdb_idl_txn *ovnsb_idl_txn, op->visited = true; continue; } - struct ovs_list lflows = OVS_LIST_INITIALIZER(&lflows); - ovs_list_splice(&lflows, op->lflows.next, &op->lflows); - ovn_port_destroy(&nd->ls_ports, op); - op = ls_port_create(ovnsb_idl_txn, &nd->ls_ports, - new_nbsp->name, new_nbsp, od, sb, &lflows, - ni->sbrec_mirror_table, + if (!ls_port_reinit(op, ovnsb_idl_txn, &nd->ls_ports, + new_nbsp, NULL, + od, sb, ni->sbrec_mirror_table, ni->sbrec_chassis_table, ni->sbrec_chassis_by_name, - ni->sbrec_chassis_by_hostname); - ovs_assert(ovs_list_is_empty(&lflows)); - if (!op) { + ni->sbrec_chassis_by_hostname)) { goto fail; } ovs_list_push_back(&ls_change->updated_ports, &op->list); @@ -6173,6 +6248,7 @@ ovn_igmp_group_destroy(struct hmap *igmp_groups, struct ovn_lflow { struct hmap_node hmap_node; + struct hmap_node hash_node; struct ovs_list list_node; /* For temporary list of lflows. Don't remove at destroy. */ @@ -6189,15 +6265,14 @@ struct ovn_lflow { * 'dpg_bitmap'. */ struct ovn_dp_group *dpg; /* Link to unique Sb datapath group. */ - struct ovs_list referenced_by; /* List of struct lflow_ref_node. */ const char *where; struct uuid sb_uuid; /* SB DB row uuid, specified by northd. */ + struct uuid lflow_uuid; }; -static void ovn_lflow_destroy(struct hmap *lflows, struct ovn_lflow *lflow); +static void ovn_lflow_destroy(struct lflow_data *, struct ovn_lflow *lflow); static struct ovn_lflow *ovn_lflow_find(const struct hmap *lflows, - const struct ovn_datapath *od, enum ovn_stage stage, uint16_t priority, const char *match, const char *actions, @@ -6213,12 +6288,11 @@ ovn_lflow_hint(const struct ovsdb_idl_row *row) } static bool -ovn_lflow_equal(const struct ovn_lflow *a, const struct ovn_datapath *od, - enum ovn_stage stage, uint16_t priority, const char *match, +ovn_lflow_equal(const struct ovn_lflow *a, enum ovn_stage stage, + uint16_t priority, const char *match, const char *actions, const char *ctrl_meter) { - return (a->od == od - && a->stage == stage + return (a->stage == stage && a->priority == priority && !strcmp(a->match, match) && !strcmp(a->actions, actions) @@ -6236,10 +6310,9 @@ static void ovn_lflow_init(struct ovn_lflow *lflow, struct ovn_datapath *od, size_t dp_bitmap_len, enum ovn_stage stage, uint16_t priority, char *match, char *actions, char *io_port, char *ctrl_meter, - char *stage_hint, const char *where) + char *stage_hint, const char *where, uint32_t hash) { ovs_list_init(&lflow->list_node); - ovs_list_init(&lflow->referenced_by); lflow->dpg_bitmap = bitmap_allocate(dp_bitmap_len); lflow->od = od; lflow->stage = stage; @@ -6252,6 +6325,8 @@ ovn_lflow_init(struct ovn_lflow *lflow, struct ovn_datapath *od, lflow->dpg = NULL; lflow->where = where; lflow->sb_uuid = UUID_ZERO; + lflow->lflow_uuid = uuid_random(); + lflow->lflow_uuid.parts[0] = hash; } /* The lflow_hash_lock is a mutex array that protects updates to the shared @@ -6367,32 +6442,10 @@ ovn_dp_group_add_with_reference(struct ovn_lflow *lflow_ref, } } -/* This global variable collects the lflows generated by do_ovn_lflow_add(). - * start_collecting_lflows() will enable the lflow collection and the calls to - * do_ovn_lflow_add (or the macros ovn_lflow_add_...) will add generated lflows - * to the list. end_collecting_lflows() will disable it. */ -static thread_local struct ovs_list collected_lflows; -static thread_local bool collecting_lflows = false; - -static void -start_collecting_lflows(void) -{ - ovs_assert(!collecting_lflows); - ovs_list_init(&collected_lflows); - collecting_lflows = true; -} - -static void -end_collecting_lflows(void) -{ - ovs_assert(collecting_lflows); - collecting_lflows = false; -} - /* Adds a row with the specified contents to the Logical_Flow table. * Version to use when hash bucket locking is NOT required. */ -static void -do_ovn_lflow_add(struct hmap *lflow_map, const struct ovn_datapath *od, +static struct ovn_lflow * +do_ovn_lflow_add(struct lflow_data *lflow_data, const struct ovn_datapath *od, const unsigned long *dp_bitmap, size_t dp_bitmap_len, uint32_t hash, enum ovn_stage stage, uint16_t priority, const char *match, const char *actions, const char *io_port, @@ -6400,24 +6453,18 @@ do_ovn_lflow_add(struct hmap *lflow_map, const struct ovn_datapath *od, const char *where, const char *ctrl_meter) OVS_REQUIRES(fake_hash_mutex) { - struct ovn_lflow *old_lflow; struct ovn_lflow *lflow; size_t bitmap_len = od ? ods_size(od->datapaths) : dp_bitmap_len; ovs_assert(bitmap_len); - if (collecting_lflows) { - ovs_assert(od); - ovs_assert(!dp_bitmap); - } else { - old_lflow = ovn_lflow_find(lflow_map, NULL, stage, priority, match, - actions, ctrl_meter, hash); - if (old_lflow) { - ovn_dp_group_add_with_reference(old_lflow, od, dp_bitmap, - bitmap_len); - return; - } + old_lflow = ovn_lflow_find(&lflow_data->lflows_match_map, stage, + priority, match, actions, ctrl_meter, hash); + if (old_lflow) { + ovn_dp_group_add_with_reference(old_lflow, od, dp_bitmap, + bitmap_len); + return old_lflow; } lflow = xmalloc(sizeof *lflow); @@ -6428,25 +6475,27 @@ do_ovn_lflow_add(struct hmap *lflow_map, const struct ovn_datapath *od, xstrdup(match), xstrdup(actions), io_port ? xstrdup(io_port) : NULL, nullable_xstrdup(ctrl_meter), - ovn_lflow_hint(stage_hint), where); + ovn_lflow_hint(stage_hint), where, hash); ovn_dp_group_add_with_reference(lflow, od, dp_bitmap, bitmap_len); if (parallelization_state != STATE_USE_PARALLELIZATION) { - hmap_insert(lflow_map, &lflow->hmap_node, hash); + hmap_insert(&lflow_data->lflows_match_map, &lflow->hmap_node, hash); + hmap_insert(&lflow_data->lflows_hash_map, &lflow->hash_node, hash); } else { - hmap_insert_fast(lflow_map, &lflow->hmap_node, hash); + hmap_insert_fast(&lflow_data->lflows_match_map, &lflow->hmap_node, + hash); + hmap_insert_fast(&lflow_data->lflows_hash_map, &lflow->hash_node, + hash); thread_lflow_counter++; } - if (collecting_lflows) { - ovs_list_insert(&collected_lflows, &lflow->list_node); - } + return lflow; } /* Adds a row with the specified contents to the Logical_Flow table. */ static void -ovn_lflow_add_at(struct hmap *lflow_map, const struct ovn_datapath *od, +ovn_lflow_add_at(struct lflow_data *lflow_data, const struct ovn_datapath *od, const unsigned long *dp_bitmap, size_t dp_bitmap_len, enum ovn_stage stage, uint16_t priority, const char *match, const char *actions, const char *io_port, @@ -6465,45 +6514,91 @@ ovn_lflow_add_at(struct hmap *lflow_map, const struct ovn_datapath *od, priority, match, actions); - hash_lock = lflow_hash_lock(lflow_map, hash); - do_ovn_lflow_add(lflow_map, od, dp_bitmap, dp_bitmap_len, hash, stage, + hash_lock = lflow_hash_lock(&lflow_data->lflows_match_map, hash); + do_ovn_lflow_add(lflow_data, od, dp_bitmap, dp_bitmap_len, hash, stage, priority, match, actions, io_port, stage_hint, where, ctrl_meter); lflow_hash_unlock(hash_lock); } +/* Adds a row with the specified contents to the Logical_Flow table. */ +static void +ovn_lflow_add_objdep_ref(struct lflow_data *lflow_data, + const struct ovn_datapath *od, + const unsigned long *dp_bitmap, size_t dp_bitmap_len, + enum ovn_stage stage, uint16_t priority, + const char *match, const char *actions, + const char *io_port, const char *ctrl_meter, + const struct ovsdb_idl_row *stage_hint, + const char *where, struct objdep_mgr *lflow_dep_mgr, + enum objdep_type objdep_type, + const char *res_name, + const struct ovn_datapath *res_od) + OVS_EXCLUDED(fake_hash_mutex) +{ + struct ovs_mutex *hash_lock; + uint32_t hash; + + ovs_assert(lflow_dep_mgr); + ovs_assert(res_name); + ovs_assert(!od || + ovn_stage_to_datapath_type(stage) == ovn_datapath_get_type(od)); + + hash = ovn_logical_flow_hash(ovn_stage_get_table(stage), + ovn_stage_get_pipeline(stage), + priority, match, + actions); + + hash_lock = lflow_hash_lock(&lflow_data->lflows_match_map, hash); + struct ovn_lflow *lflow = + do_ovn_lflow_add(lflow_data, od, dp_bitmap, dp_bitmap_len, hash, stage, + priority, match, actions, io_port, stage_hint, where, + ctrl_meter); + + objdep_mgr_add(lflow_dep_mgr, objdep_type, res_name, &lflow->lflow_uuid); + + if (res_od && res_od != od) { + char uuid_s[UUID_LEN + 1]; + sprintf(uuid_s, UUID_FMT, UUID_ARGS(&lflow->lflow_uuid)); + + struct uuid u = UUID_ZERO; + u.parts[0] = od->index; + objdep_mgr_add(lflow_dep_mgr, OBJDEP_TYPE_LFLOW_OD, uuid_s, &u); + } + lflow_hash_unlock(hash_lock); +} + static void -__ovn_lflow_add_default_drop(struct hmap *lflow_map, +__ovn_lflow_add_default_drop(struct lflow_data *lflow_data, struct ovn_datapath *od, enum ovn_stage stage, const char *where) { - ovn_lflow_add_at(lflow_map, od, NULL, 0, stage, 0, "1", - debug_drop_action(), - NULL, NULL, NULL, where ); + ovn_lflow_add_at(lflow_data, od, NULL, 0, stage, 0, "1", + debug_drop_action(), NULL, NULL, NULL, where ); } /* Adds a row with the specified contents to the Logical_Flow table. */ -#define ovn_lflow_add_with_hint__(LFLOW_MAP, OD, STAGE, PRIORITY, MATCH, \ +#define ovn_lflow_add_with_hint__(LFLOW_DATA, OD, STAGE, PRIORITY, MATCH, \ ACTIONS, IN_OUT_PORT, CTRL_METER, \ STAGE_HINT) \ - ovn_lflow_add_at(LFLOW_MAP, OD, NULL, 0, STAGE, PRIORITY, MATCH, ACTIONS, \ + ovn_lflow_add_at(LFLOW_DATA, OD, NULL, 0, STAGE, PRIORITY, MATCH, ACTIONS,\ IN_OUT_PORT, CTRL_METER, STAGE_HINT, OVS_SOURCE_LOCATOR) -#define ovn_lflow_add_with_hint(LFLOW_MAP, OD, STAGE, PRIORITY, MATCH, \ +#define ovn_lflow_add_with_hint(LFLOW_DATA, OD, STAGE, PRIORITY, MATCH, \ ACTIONS, STAGE_HINT) \ - ovn_lflow_add_at(LFLOW_MAP, OD, NULL, 0, STAGE, PRIORITY, MATCH, ACTIONS, \ + ovn_lflow_add_at(LFLOW_DATA, OD, NULL, 0, STAGE, PRIORITY, MATCH, ACTIONS,\ NULL, NULL, STAGE_HINT, OVS_SOURCE_LOCATOR) -#define ovn_lflow_add_with_dp_group(LFLOW_MAP, DP_BITMAP, DP_BITMAP_LEN, \ +#define ovn_lflow_add_with_dp_group(LFLOW_DATA, DP_BITMAP, DP_BITMAP_LEN, \ STAGE, PRIORITY, MATCH, ACTIONS, \ STAGE_HINT) \ - ovn_lflow_add_at(LFLOW_MAP, NULL, DP_BITMAP, DP_BITMAP_LEN, STAGE, \ + ovn_lflow_add_at(LFLOW_DATA, NULL, DP_BITMAP, DP_BITMAP_LEN, STAGE, \ PRIORITY, MATCH, ACTIONS, NULL, NULL, STAGE_HINT, \ OVS_SOURCE_LOCATOR) -#define ovn_lflow_add_default_drop(LFLOW_MAP, OD, STAGE) \ - __ovn_lflow_add_default_drop(LFLOW_MAP, OD, STAGE, OVS_SOURCE_LOCATOR) +#define ovn_lflow_add_default_drop(LFLOW_DATA, OD, STAGE) \ + __ovn_lflow_add_default_drop(LFLOW_DATA, OD, STAGE, OVS_SOURCE_LOCATOR) /* This macro is similar to ovn_lflow_add_with_hint, except that it requires @@ -6516,30 +6611,40 @@ __ovn_lflow_add_default_drop(struct hmap *lflow_map, * - For egress pipeline, the lport that is used to match "outport". * * For now, only LS pipelines should use this macro. */ -#define ovn_lflow_add_with_lport_and_hint(LFLOW_MAP, OD, STAGE, PRIORITY, \ +#define ovn_lflow_add_with_lport_and_hint(LFLOW_DATA, OD, STAGE, PRIORITY, \ MATCH, ACTIONS, IN_OUT_PORT, \ STAGE_HINT) \ - ovn_lflow_add_at(LFLOW_MAP, OD, NULL, 0, STAGE, PRIORITY, MATCH, ACTIONS, \ + ovn_lflow_add_at(LFLOW_DATA, OD, NULL, 0, STAGE, PRIORITY, MATCH, ACTIONS,\ IN_OUT_PORT, NULL, STAGE_HINT, OVS_SOURCE_LOCATOR) -#define ovn_lflow_add(LFLOW_MAP, OD, STAGE, PRIORITY, MATCH, ACTIONS) \ - ovn_lflow_add_at(LFLOW_MAP, OD, NULL, 0, STAGE, PRIORITY, MATCH, ACTIONS, \ +#define ovn_lflow_add(LFLOW_DATA, OD, STAGE, PRIORITY, MATCH, ACTIONS) \ + ovn_lflow_add_at(LFLOW_DATA, OD, NULL, 0, STAGE, PRIORITY, MATCH, ACTIONS,\ NULL, NULL, NULL, OVS_SOURCE_LOCATOR) -#define ovn_lflow_metered(LFLOW_MAP, OD, STAGE, PRIORITY, MATCH, ACTIONS, \ +#define ovn_lflow_metered(LFLOW_DATA, OD, STAGE, PRIORITY, MATCH, ACTIONS, \ CTRL_METER) \ - ovn_lflow_add_with_hint__(LFLOW_MAP, OD, STAGE, PRIORITY, MATCH, \ + ovn_lflow_add_with_hint__(LFLOW_DATA, OD, STAGE, PRIORITY, MATCH, \ ACTIONS, NULL, CTRL_METER, NULL) +#define ovn_lflow_add_with_lport_lflow_ref(LFLOW_DATA, OD, STAGE, PRIORITY, \ + MATCH, ACTIONS, IN_OUT_PORT, \ + CTRL_METER, STAGE_HINT, \ + LFLOW_DEP_MGR, LPORT_NAME, \ + LPORT_OD) \ + ovn_lflow_add_objdep_ref(LFLOW_DATA, OD, NULL, 0, STAGE, PRIORITY, MATCH, \ + ACTIONS, IN_OUT_PORT, CTRL_METER, STAGE_HINT, \ + OVS_SOURCE_LOCATOR, LFLOW_DEP_MGR, \ + OBJDEP_TYPE_LPORT, LPORT_NAME, LPORT_OD) + static struct ovn_lflow * -ovn_lflow_find(const struct hmap *lflows, const struct ovn_datapath *od, +ovn_lflow_find(const struct hmap *lflows, enum ovn_stage stage, uint16_t priority, const char *match, const char *actions, const char *ctrl_meter, uint32_t hash) { struct ovn_lflow *lflow; HMAP_FOR_EACH_WITH_HASH (lflow, hmap_node, hash, lflows) { - if (ovn_lflow_equal(lflow, od, stage, priority, match, actions, + if (ovn_lflow_equal(lflow, stage, priority, match, actions, ctrl_meter)) { return lflow; } @@ -6547,12 +6652,27 @@ ovn_lflow_find(const struct hmap *lflows, const struct ovn_datapath *od, return NULL; } +static struct ovn_lflow * +ovn_lflow_uuid_find(const struct hmap *lflows_hash_map, + const struct uuid *lflow_uuid) +{ + uint32_t hash = lflow_uuid->parts[0]; + struct ovn_lflow *lflow; + HMAP_FOR_EACH_WITH_HASH (lflow, hash_node, hash, lflows_hash_map) { + if (uuid_equals(&lflow->lflow_uuid, lflow_uuid)) { + return lflow; + } + } + return NULL; +} + static void -ovn_lflow_destroy(struct hmap *lflows, struct ovn_lflow *lflow) +ovn_lflow_destroy(struct lflow_data *lflow_data, struct ovn_lflow *lflow) { if (lflow) { - if (lflows) { - hmap_remove(lflows, &lflow->hmap_node); + if (lflow_data) { + hmap_remove(&lflow_data->lflows_match_map, &lflow->hmap_node); + hmap_remove(&lflow_data->lflows_hash_map, &lflow->hash_node); } bitmap_free(lflow->dpg_bitmap); free(lflow->match); @@ -6560,28 +6680,10 @@ ovn_lflow_destroy(struct hmap *lflows, struct ovn_lflow *lflow) free(lflow->io_port); free(lflow->stage_hint); free(lflow->ctrl_meter); - struct lflow_ref_node *l; - LIST_FOR_EACH_SAFE (l, ref_list_node, &lflow->referenced_by) { - ovs_list_remove(&l->lflow_list_node); - ovs_list_remove(&l->ref_list_node); - free(l); - } free(lflow); } } -static void -link_ovn_port_to_lflows(struct ovn_port *op, struct ovs_list *lflows) -{ - struct ovn_lflow *f; - LIST_FOR_EACH (f, list_node, lflows) { - struct lflow_ref_node *lfrn = xmalloc(sizeof *lfrn); - lfrn->lflow = f; - ovs_list_insert(&op->lflows, &lfrn->lflow_list_node); - ovs_list_insert(&f->referenced_by, &lfrn->ref_list_node); - } -} - static bool build_dhcpv4_action(struct ovn_port *op, ovs_be32 offer_ip, struct ds *options_action, struct ds *response_action, @@ -6885,8 +6987,9 @@ ls_get_acl_flags(struct ovn_datapath *od) * build_lswitch_lflows_admission_control() handles the port security. */ static void -build_lswitch_port_sec_op(struct ovn_port *op, struct hmap *lflows, - struct ds *actions, struct ds *match) +build_lswitch_port_sec_op(struct ovn_port *op, struct lflow_data *lflows, + struct ds *actions, struct ds *match, + struct objdep_mgr *lflow_dep_mgr) { ovs_assert(op->nbsp); @@ -6899,16 +7002,18 @@ build_lswitch_port_sec_op(struct ovn_port *op, struct hmap *lflows, ds_put_format(match, "inport == %s", op->json_key); if (!lsp_is_enabled(op->nbsp)) { /* Drop packets from disabled logical ports. */ - ovn_lflow_add_with_lport_and_hint( + ovn_lflow_add_with_lport_lflow_ref( lflows, op->od, S_SWITCH_IN_CHECK_PORT_SEC, 100, ds_cstr(match), REGBIT_PORT_SEC_DROP" = 1; next;", - op->key, &op->nbsp->header_); + op->key, NULL, &op->nbsp->header_, lflow_dep_mgr, op->key, + op->od); ds_clear(match); ds_put_format(match, "outport == %s", op->json_key); - ovn_lflow_add_with_lport_and_hint( + ovn_lflow_add_with_lport_lflow_ref( lflows, op->od, S_SWITCH_IN_L2_UNKNOWN, 50, ds_cstr(match), - debug_drop_action(), op->key, &op->nbsp->header_); + debug_drop_action(), op->key, NULL, &op->nbsp->header_, + lflow_dep_mgr, op->key, op->od); return; } @@ -6921,17 +7026,19 @@ build_lswitch_port_sec_op(struct ovn_port *op, struct hmap *lflows, ds_put_format(actions, REGBIT_FROM_RAMP" = 1; "); ds_put_format(actions, "next(pipeline=ingress, table=%d);", ovn_stage_get_table(S_SWITCH_IN_HAIRPIN)); - ovn_lflow_add_with_lport_and_hint(lflows, op->od, - S_SWITCH_IN_CHECK_PORT_SEC, 70, - ds_cstr(match), ds_cstr(actions), - op->key, &op->nbsp->header_); + ovn_lflow_add_with_lport_lflow_ref(lflows, op->od, + S_SWITCH_IN_CHECK_PORT_SEC, 70, + ds_cstr(match), ds_cstr(actions), + op->key, NULL, &op->nbsp->header_, + lflow_dep_mgr, op->key, op->od); } else if (queue_id) { ds_put_cstr(actions, REGBIT_PORT_SEC_DROP" = check_in_port_sec(); next;"); - ovn_lflow_add_with_lport_and_hint(lflows, op->od, - S_SWITCH_IN_CHECK_PORT_SEC, 70, - ds_cstr(match), ds_cstr(actions), - op->key, &op->nbsp->header_); + ovn_lflow_add_with_lport_lflow_ref(lflows, op->od, + S_SWITCH_IN_CHECK_PORT_SEC, 70, + ds_cstr(match), ds_cstr(actions), + op->key, NULL, &op->nbsp->header_, + lflow_dep_mgr, op->key, op->od); if (!lsp_is_localnet(op->nbsp) && !op->od->n_localnet_ports) { return; @@ -6943,27 +7050,32 @@ build_lswitch_port_sec_op(struct ovn_port *op, struct hmap *lflows, ds_clear(match); if (lsp_is_localnet(op->nbsp)) { ds_put_format(match, "outport == %s", op->json_key); - ovn_lflow_add_with_lport_and_hint(lflows, op->od, - S_SWITCH_OUT_APPLY_PORT_SEC, 100, - ds_cstr(match), ds_cstr(actions), - op->key, &op->nbsp->header_); + ovn_lflow_add_with_lport_lflow_ref(lflows, op->od, + S_SWITCH_OUT_APPLY_PORT_SEC, + 100, ds_cstr(match), + ds_cstr(actions), + op->key, NULL, + &op->nbsp->header_, + lflow_dep_mgr, op->key, op->od); } else if (op->od->n_localnet_ports) { ds_put_format(match, "outport == %s && inport == %s", op->od->localnet_ports[0]->json_key, op->json_key); - ovn_lflow_add_with_lport_and_hint(lflows, op->od, + ovn_lflow_add_with_lport_lflow_ref(lflows, op->od, S_SWITCH_OUT_APPLY_PORT_SEC, 110, ds_cstr(match), ds_cstr(actions), - op->od->localnet_ports[0]->key, - &op->od->localnet_ports[0]->nbsp->header_); + op->od->localnet_ports[0]->key, NULL, + &op->od->localnet_ports[0]->nbsp->header_, + lflow_dep_mgr, op->key, op->od); } } } static void build_lswitch_learn_fdb_op( - struct ovn_port *op, struct hmap *lflows, - struct ds *actions, struct ds *match) + struct ovn_port *op, struct lflow_data *lflows, + struct ds *actions, struct ds *match, + struct objdep_mgr *lflow_dep_mgr) { ovs_assert(op->nbsp); @@ -6974,24 +7086,26 @@ build_lswitch_learn_fdb_op( ds_put_format(match, "inport == %s", op->json_key); ds_put_format(actions, REGBIT_LKUP_FDB " = lookup_fdb(inport, eth.src); next;"); - ovn_lflow_add_with_lport_and_hint(lflows, op->od, - S_SWITCH_IN_LOOKUP_FDB, 100, - ds_cstr(match), ds_cstr(actions), - op->key, &op->nbsp->header_); + ovn_lflow_add_with_lport_lflow_ref(lflows, op->od, + S_SWITCH_IN_LOOKUP_FDB, 100, + ds_cstr(match), ds_cstr(actions), + op->key, NULL, &op->nbsp->header_, + lflow_dep_mgr, op->key, op->od); ds_put_cstr(match, " && "REGBIT_LKUP_FDB" == 0"); ds_clear(actions); ds_put_cstr(actions, "put_fdb(inport, eth.src); next;"); - ovn_lflow_add_with_lport_and_hint(lflows, op->od, S_SWITCH_IN_PUT_FDB, - 100, ds_cstr(match), - ds_cstr(actions), op->key, - &op->nbsp->header_); + ovn_lflow_add_with_lport_lflow_ref(lflows, op->od, S_SWITCH_IN_PUT_FDB, + 100, ds_cstr(match), + ds_cstr(actions), op->key, NULL, + &op->nbsp->header_, + lflow_dep_mgr, op->key, op->od); } } static void build_lswitch_learn_fdb_od( - struct ovn_datapath *od, struct hmap *lflows) + struct ovn_datapath *od, struct lflow_data *lflows) { ovs_assert(od->nbs); ovn_lflow_add(lflows, od, S_SWITCH_IN_LOOKUP_FDB, 0, "1", "next;"); @@ -7005,7 +7119,7 @@ build_lswitch_learn_fdb_od( * (priority 100). */ static void build_lswitch_output_port_sec_od(struct ovn_datapath *od, - struct hmap *lflows) + struct lflow_data *lflows) { ovs_assert(od->nbs); ovn_lflow_add(lflows, od, S_SWITCH_OUT_CHECK_PORT_SEC, 100, @@ -7022,7 +7136,7 @@ build_lswitch_output_port_sec_od(struct ovn_datapath *od, static void skip_port_from_conntrack(struct ovn_datapath *od, struct ovn_port *op, enum ovn_stage in_stage, enum ovn_stage out_stage, - uint16_t priority, struct hmap *lflows) + uint16_t priority, struct lflow_data *lflows) { /* Can't use ct() for router ports. Consider the following configuration: * lp1(10.0.0.2) on hostA--ls1--lr0--ls2--lp2(10.0.1.2) on hostB, For a @@ -7056,7 +7170,7 @@ skip_port_from_conntrack(struct ovn_datapath *od, struct ovn_port *op, static void build_stateless_filter(struct ovn_datapath *od, const struct nbrec_acl *acl, - struct hmap *lflows) + struct lflow_data *lflows) { const char *action = REGBIT_ACL_STATELESS" = 1; next;"; if (!strcmp(acl->direction, "from-lport")) { @@ -7077,7 +7191,7 @@ build_stateless_filter(struct ovn_datapath *od, static void build_stateless_filters(struct ovn_datapath *od, const struct hmap *port_groups, - struct hmap *lflows) + struct lflow_data *lflows) { for (size_t i = 0; i < od->nbs->n_acls; i++) { const struct nbrec_acl *acl = od->nbs->acls[i]; @@ -7101,7 +7215,7 @@ build_stateless_filters(struct ovn_datapath *od, static void build_pre_acls(struct ovn_datapath *od, const struct hmap *port_groups, - struct hmap *lflows) + struct lflow_data *lflows) { /* Ingress and Egress Pre-ACL Table (Priority 0): Packets are * allowed by default. */ @@ -7229,7 +7343,7 @@ build_empty_lb_event_flow(struct ovn_lb_vip *lb_vip, static void build_interconn_mcast_snoop_flows(struct ovn_datapath *od, const struct shash *meter_groups, - struct hmap *lflows) + struct lflow_data *lflows) { struct mcast_switch_info *mcast_sw_info = &od->mcast_info.sw; if (!mcast_sw_info->enabled @@ -7263,7 +7377,7 @@ build_interconn_mcast_snoop_flows(struct ovn_datapath *od, static void build_pre_lb(struct ovn_datapath *od, const struct shash *meter_groups, - struct hmap *lflows) + struct lflow_data *lflows) { /* Handle IGMP/MLD packets crossing AZs. */ build_interconn_mcast_snoop_flows(od, meter_groups, lflows); @@ -7355,7 +7469,7 @@ build_pre_lb(struct ovn_datapath *od, const struct shash *meter_groups, static void build_pre_stateful(struct ovn_datapath *od, const struct chassis_features *features, - struct hmap *lflows) + struct lflow_data *lflows) { /* Ingress and Egress pre-stateful Table (Priority 0): Packets are * allowed by default. */ @@ -7387,7 +7501,7 @@ build_pre_stateful(struct ovn_datapath *od, static void build_acl_hints(struct ovn_datapath *od, const struct chassis_features *features, - struct hmap *lflows) + struct lflow_data *lflows) { /* This stage builds hints for the IN/OUT_ACL stage. Based on various * combinations of ct flags packets may hit only a subset of the logical @@ -7573,7 +7687,7 @@ build_acl_log(struct ds *actions, const struct nbrec_acl *acl, } static void -consider_acl(struct hmap *lflows, struct ovn_datapath *od, +consider_acl(struct lflow_data *lflows, struct ovn_datapath *od, const struct nbrec_acl *acl, bool has_stateful, bool ct_masked_mark, const struct shash *meter_groups, struct ds *match, struct ds *actions) @@ -7864,7 +7978,7 @@ build_port_group_lswitches( #define IPV6_CT_OMIT_MATCH "nd || nd_ra || nd_rs || mldv1 || mldv2" static void -build_acl_action_lflows(struct ovn_datapath *od, struct hmap *lflows, +build_acl_action_lflows(struct ovn_datapath *od, struct lflow_data *lflows, const char *default_acl_action, const struct shash *meter_groups, struct ds *match, @@ -7939,7 +8053,7 @@ build_acl_action_lflows(struct ovn_datapath *od, struct hmap *lflows, } static void -build_acl_log_related_flows(struct ovn_datapath *od, struct hmap *lflows, +build_acl_log_related_flows(struct ovn_datapath *od, struct lflow_data *lflows, const struct nbrec_acl *acl, bool has_stateful, bool ct_masked_mark, const struct shash *meter_groups, @@ -8013,7 +8127,7 @@ build_acl_log_related_flows(struct ovn_datapath *od, struct hmap *lflows, static void build_acls(struct ovn_datapath *od, const struct chassis_features *features, - struct hmap *lflows, const struct hmap *port_groups, + struct lflow_data *lflows, const struct hmap *port_groups, const struct shash *meter_groups) { const char *default_acl_action = default_acl_drop @@ -8311,7 +8425,7 @@ build_acls(struct ovn_datapath *od, const struct chassis_features *features, } static void -build_qos(struct ovn_datapath *od, struct hmap *lflows) { +build_qos(struct ovn_datapath *od, struct lflow_data *lflows) { struct ds action = DS_EMPTY_INITIALIZER; ovn_lflow_add(lflows, od, S_SWITCH_IN_QOS_MARK, 0, "1", "next;"); @@ -8372,7 +8486,7 @@ build_qos(struct ovn_datapath *od, struct hmap *lflows) { } static void -build_lb_rules_pre_stateful(struct hmap *lflows, +build_lb_rules_pre_stateful(struct lflow_data *lflows, struct ovn_lb_datapaths *lb_dps, bool ct_lb_mark, const struct ovn_datapaths *ls_datapaths, @@ -8474,7 +8588,8 @@ build_lb_rules_pre_stateful(struct hmap *lflows, * */ static void -build_lb_affinity_lr_flows(struct hmap *lflows, const struct ovn_northd_lb *lb, +build_lb_affinity_lr_flows(struct lflow_data *lflows, + const struct ovn_northd_lb *lb, struct ovn_lb_vip *lb_vip, char *new_lb_match, char *lb_action, const unsigned long *dp_bitmap, const struct ovn_datapaths *lr_datapaths) @@ -8660,7 +8775,7 @@ build_lb_affinity_lr_flows(struct hmap *lflows, const struct ovn_northd_lb *lb, * */ static void -build_lb_affinity_ls_flows(struct hmap *lflows, +build_lb_affinity_ls_flows(struct lflow_data *lflows, struct ovn_lb_datapaths *lb_dps, struct ovn_lb_vip *lb_vip, const struct ovn_datapaths *ls_datapaths) @@ -8803,7 +8918,7 @@ build_lb_affinity_ls_flows(struct hmap *lflows, static void build_lswitch_lb_affinity_default_flows(struct ovn_datapath *od, - struct hmap *lflows) + struct lflow_data *lflows) { ovs_assert(od->nbs); ovn_lflow_add(lflows, od, S_SWITCH_IN_LB_AFF_CHECK, 0, "1", "next;"); @@ -8812,7 +8927,7 @@ build_lswitch_lb_affinity_default_flows(struct ovn_datapath *od, static void build_lrouter_lb_affinity_default_flows(struct ovn_datapath *od, - struct hmap *lflows) + struct lflow_data *lflows) { ovs_assert(od->nbr); ovn_lflow_add(lflows, od, S_ROUTER_IN_LB_AFF_CHECK, 0, "1", "next;"); @@ -8820,7 +8935,7 @@ build_lrouter_lb_affinity_default_flows(struct ovn_datapath *od, } static void -build_lb_rules(struct hmap *lflows, struct ovn_lb_datapaths *lb_dps, +build_lb_rules(struct lflow_data *lflows, struct ovn_lb_datapaths *lb_dps, const struct ovn_datapaths *ls_datapaths, const struct chassis_features *features, struct ds *match, struct ds *action, const struct shash *meter_groups, @@ -8900,7 +9015,7 @@ build_lb_rules(struct hmap *lflows, struct ovn_lb_datapaths *lb_dps, static void build_stateful(struct ovn_datapath *od, const struct chassis_features *features, - struct hmap *lflows) + struct lflow_data *lflows) { const char *ct_block_action = features->ct_no_masked_label ? "ct_mark.blocked" @@ -8949,7 +9064,7 @@ build_stateful(struct ovn_datapath *od, } static void -build_lb_hairpin(struct ovn_datapath *od, struct hmap *lflows) +build_lb_hairpin(struct ovn_datapath *od, struct lflow_data *lflows) { /* Ingress Pre-Hairpin/Nat-Hairpin/Hairpin tabled (Priority 0). * Packets that don't need hairpinning should continue processing. @@ -9006,7 +9121,7 @@ build_lb_hairpin(struct ovn_datapath *od, struct hmap *lflows) } static void -build_vtep_hairpin(struct ovn_datapath *od, struct hmap *lflows) +build_vtep_hairpin(struct ovn_datapath *od, struct lflow_data *lflows) { if (!od->has_vtep_lports) { /* There is no need in these flows if datapath has no vtep lports. */ @@ -9054,7 +9169,7 @@ build_vtep_hairpin(struct ovn_datapath *od, struct hmap *lflows) /* Build logical flows for the forwarding groups */ static void -build_fwd_group_lflows(struct ovn_datapath *od, struct hmap *lflows) +build_fwd_group_lflows(struct ovn_datapath *od, struct lflow_data *lflows) { ovs_assert(od->nbs); if (!od->nbs->n_forwarding_groups) { @@ -9232,7 +9347,10 @@ static void build_lswitch_rport_arp_req_self_orig_flow(struct ovn_port *op, uint32_t priority, struct ovn_datapath *od, - struct hmap *lflows) + struct ovn_port *patch_op, + struct lflow_data *lflows, + const struct ovsdb_idl_row *hint, + struct objdep_mgr *lflow_dep_mgr) { struct sset all_eth_addrs = SSET_INITIALIZER(&all_eth_addrs); struct ds eth_src = DS_EMPTY_INITIALIZER; @@ -9277,8 +9395,11 @@ build_lswitch_rport_arp_req_self_orig_flow(struct ovn_port *op, ds_put_format(&match, "eth.src == %s && (arp.op == 1 || rarp.op == 3 || nd_ns)", ds_cstr(ð_src)); - ovn_lflow_add(lflows, od, S_SWITCH_IN_L2_LKUP, priority, ds_cstr(&match), - "outport = \""MC_FLOOD_L2"\"; output;"); + ovn_lflow_add_with_lport_lflow_ref(lflows, od, S_SWITCH_IN_L2_LKUP, + priority, ds_cstr(&match), + "outport = \""MC_FLOOD_L2"\"; output;", + NULL, NULL, hint, lflow_dep_mgr, + patch_op->key, patch_op->od); sset_destroy(&all_eth_addrs); ds_destroy(ð_src); @@ -9346,8 +9467,9 @@ lrouter_port_ipv6_reachable(const struct ovn_port *op, static void build_lswitch_rport_arp_req_flow(const char *ips, int addr_family, struct ovn_port *patch_op, struct ovn_datapath *od, - uint32_t priority, struct hmap *lflows, - const struct ovsdb_idl_row *stage_hint) + uint32_t priority, struct lflow_data *lflows, + const struct ovsdb_idl_row *stage_hint, + struct objdep_mgr *lflow_dep_mgr) { struct ds match = DS_EMPTY_INITIALIZER; struct ds actions = DS_EMPTY_INITIALIZER; @@ -9361,14 +9483,16 @@ build_lswitch_rport_arp_req_flow(const char *ips, ds_put_format(&actions, "clone {outport = %s; output; }; " "outport = \""MC_FLOOD_L2"\"; output;", patch_op->json_key); - ovn_lflow_add_with_hint(lflows, od, S_SWITCH_IN_L2_LKUP, + ovn_lflow_add_with_lport_lflow_ref(lflows, od, S_SWITCH_IN_L2_LKUP, priority, ds_cstr(&match), - ds_cstr(&actions), stage_hint); + ds_cstr(&actions), NULL, NULL, stage_hint, + lflow_dep_mgr, patch_op->key, patch_op->od); } else { ds_put_format(&actions, "outport = %s; output;", patch_op->json_key); - ovn_lflow_add_with_hint(lflows, od, S_SWITCH_IN_L2_LKUP, priority, - ds_cstr(&match), ds_cstr(&actions), - stage_hint); + ovn_lflow_add_with_lport_lflow_ref(lflows, od, S_SWITCH_IN_L2_LKUP, + priority, ds_cstr(&match), ds_cstr(&actions), + NULL, NULL, stage_hint, lflow_dep_mgr, + patch_op->key, patch_op->od); } ds_destroy(&match); @@ -9386,8 +9510,9 @@ static void build_lswitch_rport_arp_req_flows(struct ovn_port *op, struct ovn_datapath *sw_od, struct ovn_port *sw_op, - struct hmap *lflows, - const struct ovsdb_idl_row *stage_hint) + struct lflow_data *lflows, + const struct ovsdb_idl_row *stage_hint, + struct objdep_mgr *lflow_dep_mgr) { if (!op || !op->nbrp) { return; @@ -9413,7 +9538,7 @@ build_lswitch_rport_arp_req_flows(struct ovn_port *op, lrouter_port_ipv4_reachable(op, ipv4_addr)) { build_lswitch_rport_arp_req_flow( ip_addr, AF_INET, sw_op, sw_od, 80, lflows, - stage_hint); + stage_hint, lflow_dep_mgr); } } SSET_FOR_EACH (ip_addr, &op->od->lb_ips->ips_v6_reachable) { @@ -9426,7 +9551,7 @@ build_lswitch_rport_arp_req_flows(struct ovn_port *op, lrouter_port_ipv6_reachable(op, &ipv6_addr)) { build_lswitch_rport_arp_req_flow( ip_addr, AF_INET6, sw_op, sw_od, 80, lflows, - stage_hint); + stage_hint, lflow_dep_mgr); } } @@ -9449,13 +9574,13 @@ build_lswitch_rport_arp_req_flows(struct ovn_port *op, if (!sset_contains(&op->od->lb_ips->ips_v6, nat->external_ip)) { build_lswitch_rport_arp_req_flow( nat->external_ip, AF_INET6, sw_op, sw_od, 80, lflows, - stage_hint); + stage_hint, lflow_dep_mgr); } } else { if (!sset_contains(&op->od->lb_ips->ips_v4, nat->external_ip)) { build_lswitch_rport_arp_req_flow( nat->external_ip, AF_INET, sw_op, sw_od, 80, lflows, - stage_hint); + stage_hint, lflow_dep_mgr); } } } @@ -9463,12 +9588,12 @@ build_lswitch_rport_arp_req_flows(struct ovn_port *op, for (size_t i = 0; i < op->lrp_networks.n_ipv4_addrs; i++) { build_lswitch_rport_arp_req_flow( op->lrp_networks.ipv4_addrs[i].addr_s, AF_INET, sw_op, sw_od, 80, - lflows, stage_hint); + lflows, stage_hint, lflow_dep_mgr); } for (size_t i = 0; i < op->lrp_networks.n_ipv6_addrs; i++) { build_lswitch_rport_arp_req_flow( op->lrp_networks.ipv6_addrs[i].addr_s, AF_INET6, sw_op, sw_od, 80, - lflows, stage_hint); + lflows, stage_hint, lflow_dep_mgr); } /* Self originated ARP requests/RARP/ND need to be flooded as usual. @@ -9479,7 +9604,9 @@ build_lswitch_rport_arp_req_flows(struct ovn_port *op, * Priority: 75. */ if (sw_od->n_router_ports != sw_od->nbs->n_ports) { - build_lswitch_rport_arp_req_self_orig_flow(op, 75, sw_od, lflows); + build_lswitch_rport_arp_req_self_orig_flow(op, 75, sw_od, sw_op, + lflows, stage_hint, + lflow_dep_mgr); } } @@ -9488,7 +9615,8 @@ build_dhcpv4_options_flows(struct ovn_port *op, struct lport_addresses *lsp_addrs, struct ovn_port *inport, bool is_external, const struct shash *meter_groups, - struct hmap *lflows) + struct lflow_data *lflows, + struct objdep_mgr *lflow_dep_mgr) { struct ds match = DS_EMPTY_INITIALIZER; @@ -9511,7 +9639,7 @@ build_dhcpv4_options_flows(struct ovn_port *op, op->json_key); } - ovn_lflow_add_with_hint__(lflows, op->od, + ovn_lflow_add_with_lport_lflow_ref(lflows, op->od, S_SWITCH_IN_DHCP_OPTIONS, 100, ds_cstr(&match), ds_cstr(&options_action), @@ -9519,7 +9647,8 @@ build_dhcpv4_options_flows(struct ovn_port *op, copp_meter_get(COPP_DHCPV4_OPTS, op->od->nbs->copp, meter_groups), - &op->nbsp->dhcpv4_options->header_); + &op->nbsp->dhcpv4_options->header_, + lflow_dep_mgr, op->key, op->od); ds_clear(&match); /* Allow ip4.src = OFFER_IP and * ip4.dst = {SERVER_IP, 255.255.255.255} for the below @@ -9539,7 +9668,7 @@ build_dhcpv4_options_flows(struct ovn_port *op, op->json_key); } - ovn_lflow_add_with_hint__(lflows, op->od, + ovn_lflow_add_with_lport_lflow_ref(lflows, op->od, S_SWITCH_IN_DHCP_OPTIONS, 100, ds_cstr(&match), ds_cstr(&options_action), @@ -9547,7 +9676,8 @@ build_dhcpv4_options_flows(struct ovn_port *op, copp_meter_get(COPP_DHCPV4_OPTS, op->od->nbs->copp, meter_groups), - &op->nbsp->dhcpv4_options->header_); + &op->nbsp->dhcpv4_options->header_, + lflow_dep_mgr, op->key, op->od); ds_clear(&match); /* If REGBIT_DHCP_OPTS_RESULT is set, it means the @@ -9563,10 +9693,11 @@ build_dhcpv4_options_flows(struct ovn_port *op, op->json_key); } - ovn_lflow_add_with_lport_and_hint( + ovn_lflow_add_with_lport_lflow_ref( lflows, op->od, S_SWITCH_IN_DHCP_RESPONSE, 100, ds_cstr(&match), ds_cstr(&response_action), inport->key, - &op->nbsp->dhcpv4_options->header_); + NULL, &op->nbsp->dhcpv4_options->header_, + lflow_dep_mgr, op->key, op->od); ds_destroy(&options_action); ds_destroy(&response_action); ds_destroy(&ipv4_addr_match); @@ -9581,7 +9712,8 @@ build_dhcpv6_options_flows(struct ovn_port *op, struct lport_addresses *lsp_addrs, struct ovn_port *inport, bool is_external, const struct shash *meter_groups, - struct hmap *lflows) + struct lflow_data *lflows, + struct objdep_mgr *lflow_dep_mgr) { struct ds match = DS_EMPTY_INITIALIZER; @@ -9603,7 +9735,7 @@ build_dhcpv6_options_flows(struct ovn_port *op, op->json_key); } - ovn_lflow_add_with_hint__(lflows, op->od, + ovn_lflow_add_with_lport_lflow_ref(lflows, op->od, S_SWITCH_IN_DHCP_OPTIONS, 100, ds_cstr(&match), ds_cstr(&options_action), @@ -9611,15 +9743,17 @@ build_dhcpv6_options_flows(struct ovn_port *op, copp_meter_get(COPP_DHCPV6_OPTS, op->od->nbs->copp, meter_groups), - &op->nbsp->dhcpv6_options->header_); + &op->nbsp->dhcpv6_options->header_, + lflow_dep_mgr, op->key, op->od); /* If REGBIT_DHCP_OPTS_RESULT is set to 1, it means the * put_dhcpv6_opts action is successful */ ds_put_cstr(&match, " && "REGBIT_DHCP_OPTS_RESULT); - ovn_lflow_add_with_lport_and_hint( + ovn_lflow_add_with_lport_lflow_ref( lflows, op->od, S_SWITCH_IN_DHCP_RESPONSE, 100, ds_cstr(&match), ds_cstr(&response_action), inport->key, - &op->nbsp->dhcpv6_options->header_); + NULL, &op->nbsp->dhcpv6_options->header_, + lflow_dep_mgr, op->key, op->od); ds_destroy(&options_action); ds_destroy(&response_action); break; @@ -9631,7 +9765,8 @@ build_dhcpv6_options_flows(struct ovn_port *op, static void build_drop_arp_nd_flows_for_unbound_router_ports(struct ovn_port *op, const struct ovn_port *port, - struct hmap *lflows) + struct lflow_data *lflows, + struct objdep_mgr *dep_mgr) { struct ds match = DS_EMPTY_INITIALIZER; @@ -9648,10 +9783,10 @@ build_drop_arp_nd_flows_for_unbound_router_ports(struct ovn_port *op, port->json_key, op->lsp_addrs[i].ea_s, op->json_key, rp->lsp_addrs[k].ipv4_addrs[l].addr_s); - ovn_lflow_add_with_lport_and_hint( + ovn_lflow_add_with_lport_lflow_ref( lflows, op->od, S_SWITCH_IN_EXTERNAL_PORT, 100, ds_cstr(&match), debug_drop_action(), port->key, - &op->nbsp->header_); + NULL, &op->nbsp->header_, dep_mgr, op->key, op->od); } for (size_t l = 0; l < rp->lsp_addrs[k].n_ipv6_addrs; l++) { ds_clear(&match); @@ -9664,10 +9799,10 @@ build_drop_arp_nd_flows_for_unbound_router_ports(struct ovn_port *op, rp->lsp_addrs[k].ipv6_addrs[l].addr_s, rp->lsp_addrs[k].ipv6_addrs[l].sn_addr_s, rp->lsp_addrs[k].ipv6_addrs[l].addr_s); - ovn_lflow_add_with_lport_and_hint( + ovn_lflow_add_with_lport_lflow_ref( lflows, op->od, S_SWITCH_IN_EXTERNAL_PORT, 100, ds_cstr(&match), debug_drop_action(), port->key, - &op->nbsp->header_); + NULL, &op->nbsp->header_, dep_mgr, op->key, op->od); } ds_clear(&match); @@ -9678,12 +9813,13 @@ build_drop_arp_nd_flows_for_unbound_router_ports(struct ovn_port *op, port->json_key, op->lsp_addrs[i].ea_s, rp->lsp_addrs[k].ea_s, op->json_key); - ovn_lflow_add_with_lport_and_hint(lflows, op->od, + ovn_lflow_add_with_lport_lflow_ref(lflows, op->od, S_SWITCH_IN_EXTERNAL_PORT, 100, ds_cstr(&match), debug_drop_action(), - port->key, - &op->nbsp->header_); + port->key, NULL, + &op->nbsp->header_, + dep_mgr, op->key, op->od); } } } @@ -9698,7 +9834,7 @@ is_vlan_transparent(const struct ovn_datapath *od) static void build_lswitch_lflows_l2_unknown(struct ovn_datapath *od, - struct hmap *lflows) + struct lflow_data *lflows) { /* Ingress table 25/26: Destination lookup for unknown MACs. */ if (od->has_unknown) { @@ -9719,7 +9855,7 @@ static void build_lswitch_lflows_pre_acl_and_acl(struct ovn_datapath *od, const struct hmap *port_groups, const struct chassis_features *features, - struct hmap *lflows, + struct lflow_data *lflows, const struct shash *meter_groups) { ovs_assert(od->nbs); @@ -9740,7 +9876,7 @@ build_lswitch_lflows_pre_acl_and_acl(struct ovn_datapath *od, * 100). */ static void build_lswitch_lflows_admission_control(struct ovn_datapath *od, - struct hmap *lflows) + struct lflow_data *lflows) { ovs_assert(od->nbs); /* Logical VLANs not supported. */ @@ -9768,8 +9904,9 @@ build_lswitch_lflows_admission_control(struct ovn_datapath *od, static void build_lswitch_arp_nd_responder_skip_local(struct ovn_port *op, - struct hmap *lflows, - struct ds *match) + struct lflow_data *lflows, + struct ds *match, + struct objdep_mgr *lflow_dep_mgr) { ovs_assert(op->nbsp); if (!lsp_is_localnet(op->nbsp) || op->od->has_arp_proxy_port) { @@ -9777,21 +9914,23 @@ build_lswitch_arp_nd_responder_skip_local(struct ovn_port *op, } ds_clear(match); ds_put_format(match, "inport == %s", op->json_key); - ovn_lflow_add_with_lport_and_hint(lflows, op->od, - S_SWITCH_IN_ARP_ND_RSP, 100, - ds_cstr(match), "next;", op->key, - &op->nbsp->header_); + ovn_lflow_add_with_lport_lflow_ref(lflows, op->od, + S_SWITCH_IN_ARP_ND_RSP, 100, + ds_cstr(match), "next;", op->key, + NULL, &op->nbsp->header_, + lflow_dep_mgr, op->key, op->od); } /* Ingress table 19: ARP/ND responder, reply for known IPs. * (priority 50). */ static void build_lswitch_arp_nd_responder_known_ips(struct ovn_port *op, - struct hmap *lflows, + struct lflow_data *lflows, const struct hmap *ls_ports, const struct shash *meter_groups, struct ds *actions, - struct ds *match) + struct ds *match, + struct objdep_mgr *lflow_dep_mgr) { ovs_assert(op->nbsp); if (!strcmp(op->nbsp->type, "virtual")) { @@ -9868,11 +10007,12 @@ build_lswitch_arp_nd_responder_known_ips(struct ovn_port *op, "bind_vport(%s, inport); " "next;", op->json_key); - ovn_lflow_add_with_lport_and_hint(lflows, op->od, - S_SWITCH_IN_ARP_ND_RSP, 100, - ds_cstr(match), - ds_cstr(actions), vparent, - &vp->nbsp->header_); + ovn_lflow_add_with_lport_lflow_ref(lflows, op->od, + S_SWITCH_IN_ARP_ND_RSP, 100, + ds_cstr(match), + ds_cstr(actions), vparent, + NULL, &vp->nbsp->header_, + lflow_dep_mgr, op->key, op->od); } free(tokstr); @@ -9916,11 +10056,12 @@ build_lswitch_arp_nd_responder_known_ips(struct ovn_port *op, "output;", op->lsp_addrs[i].ea_s, op->lsp_addrs[i].ea_s, op->lsp_addrs[i].ipv4_addrs[j].addr_s); - ovn_lflow_add_with_hint(lflows, op->od, + ovn_lflow_add_with_lport_lflow_ref(lflows, op->od, S_SWITCH_IN_ARP_ND_RSP, 50, ds_cstr(match), - ds_cstr(actions), - &op->nbsp->header_); + ds_cstr(actions), NULL, NULL, + &op->nbsp->header_, + lflow_dep_mgr, op->key, op->od); /* Do not reply to an ARP request from the port that owns * the address (otherwise a DHCP client that ARPs to check @@ -9935,11 +10076,13 @@ build_lswitch_arp_nd_responder_known_ips(struct ovn_port *op, * network is not working as configured, so dropping the * request would frustrate that intent.) */ ds_put_format(match, " && inport == %s", op->json_key); - ovn_lflow_add_with_lport_and_hint(lflows, op->od, + ovn_lflow_add_with_lport_lflow_ref(lflows, op->od, S_SWITCH_IN_ARP_ND_RSP, 100, ds_cstr(match), - "next;", op->key, - &op->nbsp->header_); + "next;", op->key, NULL, + &op->nbsp->header_, + lflow_dep_mgr, op->key, + op->od); } /* For ND solicitations, we need to listen for both the @@ -9969,24 +10112,28 @@ build_lswitch_arp_nd_responder_known_ips(struct ovn_port *op, op->lsp_addrs[i].ipv6_addrs[j].addr_s, op->lsp_addrs[i].ipv6_addrs[j].addr_s, op->lsp_addrs[i].ea_s); - ovn_lflow_add_with_hint__(lflows, op->od, - S_SWITCH_IN_ARP_ND_RSP, 50, - ds_cstr(match), - ds_cstr(actions), - NULL, - copp_meter_get(COPP_ND_NA, - op->od->nbs->copp, - meter_groups), - &op->nbsp->header_); + ovn_lflow_add_with_lport_lflow_ref(lflows, op->od, + S_SWITCH_IN_ARP_ND_RSP, 50, + ds_cstr(match), + ds_cstr(actions), + NULL, + copp_meter_get(COPP_ND_NA, + op->od->nbs->copp, + meter_groups), + &op->nbsp->header_, + lflow_dep_mgr, op->key, + op->od); /* Do not reply to a solicitation from the port that owns * the address (otherwise DAD detection will fail). */ ds_put_format(match, " && inport == %s", op->json_key); - ovn_lflow_add_with_lport_and_hint(lflows, op->od, - S_SWITCH_IN_ARP_ND_RSP, - 100, ds_cstr(match), - "next;", op->key, - &op->nbsp->header_); + ovn_lflow_add_with_lport_lflow_ref(lflows, op->od, + S_SWITCH_IN_ARP_ND_RSP, + 100, ds_cstr(match), + "next;", op->key, NULL, + &op->nbsp->header_, + lflow_dep_mgr, op->key, + op->od); } } } @@ -10032,8 +10179,11 @@ build_lswitch_arp_nd_responder_known_ips(struct ovn_port *op, ea_s, ea_s); - ovn_lflow_add_with_hint(lflows, op->od, S_SWITCH_IN_ARP_ND_RSP, - 30, ds_cstr(match), ds_cstr(actions), &op->nbsp->header_); + ovn_lflow_add_with_lport_lflow_ref(lflows, op->od, + S_SWITCH_IN_ARP_ND_RSP, + 30, ds_cstr(match), ds_cstr(actions), + NULL, NULL, &op->nbsp->header_, + lflow_dep_mgr, op->key, op->od); } /* Add IPv6 NDP responses. @@ -10076,7 +10226,7 @@ build_lswitch_arp_nd_responder_known_ips(struct ovn_port *op, lsp_is_router(op->nbsp) ? "nd_na_router" : "nd_na", ea_s, ea_s); - ovn_lflow_add_with_hint__(lflows, op->od, + ovn_lflow_add_with_lport_lflow_ref(lflows, op->od, S_SWITCH_IN_ARP_ND_RSP, 30, ds_cstr(match), ds_cstr(actions), @@ -10084,7 +10234,8 @@ build_lswitch_arp_nd_responder_known_ips(struct ovn_port *op, copp_meter_get(COPP_ND_NA, op->od->nbs->copp, meter_groups), - &op->nbsp->header_); + &op->nbsp->header_, + lflow_dep_mgr, op->key, op->od); ds_destroy(&ip6_dst_match); ds_destroy(&nd_target_match); } @@ -10095,7 +10246,7 @@ build_lswitch_arp_nd_responder_known_ips(struct ovn_port *op, * (priority 0)*/ static void build_lswitch_arp_nd_responder_default(struct ovn_datapath *od, - struct hmap *lflows) + struct lflow_data *lflows) { ovs_assert(od->nbs); ovn_lflow_add(lflows, od, S_SWITCH_IN_ARP_ND_RSP, 0, "1", "next;"); @@ -10106,7 +10257,7 @@ build_lswitch_arp_nd_responder_default(struct ovn_datapath *od, static void build_lswitch_arp_nd_service_monitor(const struct ovn_northd_lb *lb, const struct hmap *ls_ports, - struct hmap *lflows, + struct lflow_data *lflows, struct ds *actions, struct ds *match) { @@ -10182,8 +10333,9 @@ build_lswitch_arp_nd_service_monitor(const struct ovn_northd_lb *lb, * priority 100 flows. */ static void build_lswitch_dhcp_options_and_response(struct ovn_port *op, - struct hmap *lflows, - const struct shash *meter_groups) + struct lflow_data *lflows, + const struct shash *meter_groups, + struct objdep_mgr *lflow_dep_mgr) { ovs_assert(op->nbsp); if (!lsp_is_enabled(op->nbsp) || lsp_is_router(op->nbsp)) { @@ -10212,19 +10364,19 @@ build_lswitch_dhcp_options_and_response(struct ovn_port *op, build_dhcpv4_options_flows( op, &op->lsp_addrs[i], op->od->localnet_ports[j], is_external, - meter_groups, lflows); + meter_groups, lflows, lflow_dep_mgr); build_dhcpv6_options_flows( op, &op->lsp_addrs[i], op->od->localnet_ports[j], is_external, - meter_groups, lflows); + meter_groups, lflows, lflow_dep_mgr); } } else { build_dhcpv4_options_flows(op, &op->lsp_addrs[i], op, is_external, meter_groups, - lflows); + lflows, lflow_dep_mgr); build_dhcpv6_options_flows(op, &op->lsp_addrs[i], op, is_external, meter_groups, - lflows); + lflows, lflow_dep_mgr); } } } @@ -10237,7 +10389,7 @@ build_lswitch_dhcp_options_and_response(struct ovn_port *op, * (priority 0). */ static void build_lswitch_dhcp_and_dns_defaults(struct ovn_datapath *od, - struct hmap *lflows) + struct lflow_data *lflows) { ovs_assert(od->nbs); ovn_lflow_add(lflows, od, S_SWITCH_IN_DHCP_OPTIONS, 0, "1", "next;"); @@ -10252,7 +10404,7 @@ build_lswitch_dhcp_and_dns_defaults(struct ovn_datapath *od, */ static void build_lswitch_dns_lookup_and_response(struct ovn_datapath *od, - struct hmap *lflows, + struct lflow_data *lflows, const struct shash *meter_groups) { ovs_assert(od->nbs); @@ -10283,7 +10435,8 @@ build_lswitch_dns_lookup_and_response(struct ovn_datapath *od, * binding the external ports. */ static void build_lswitch_external_port(struct ovn_port *op, - struct hmap *lflows) + struct lflow_data *lflows, + struct objdep_mgr *lflow_dep_mgr) { ovs_assert(op->nbsp); if (!lsp_is_external(op->nbsp)) { @@ -10291,7 +10444,7 @@ build_lswitch_external_port(struct ovn_port *op, } for (size_t i = 0; i < op->od->n_localnet_ports; i++) { build_drop_arp_nd_flows_for_unbound_router_ports( - op, op->od->localnet_ports[i], lflows); + op, op->od->localnet_ports[i], lflows, lflow_dep_mgr); } } @@ -10299,7 +10452,7 @@ build_lswitch_external_port(struct ovn_port *op, * (priority 70 - 100). */ static void build_lswitch_destination_lookup_bmcast(struct ovn_datapath *od, - struct hmap *lflows, + struct lflow_data *lflows, struct ds *actions, const struct shash *meter_groups) { @@ -10390,7 +10543,7 @@ build_lswitch_destination_lookup_bmcast(struct ovn_datapath *od, * (priority 90). */ static void build_lswitch_ip_mcast_igmp_mld(struct ovn_igmp_group *igmp_group, - struct hmap *lflows, + struct lflow_data *lflows, struct ds *actions, struct ds *match) { @@ -10471,9 +10624,10 @@ build_lswitch_ip_mcast_igmp_mld(struct ovn_igmp_group *igmp_group, /* Ingress table 25: Destination lookup, unicast handling (priority 50), */ static void build_lswitch_ip_unicast_lookup(struct ovn_port *op, - struct hmap *lflows, + struct lflow_data *lflows, struct ds *actions, - struct ds *match) + struct ds *match, + struct objdep_mgr *lflow_dep_mgr) { ovs_assert(op->nbsp); if (lsp_is_external(op->nbsp)) { @@ -10486,7 +10640,8 @@ build_lswitch_ip_unicast_lookup(struct ovn_port *op, */ if (lsp_is_router(op->nbsp)) { build_lswitch_rport_arp_req_flows(op->peer, op->od, op, lflows, - &op->nbsp->header_); + &op->nbsp->header_, + lflow_dep_mgr); } for (size_t i = 0; i < op->nbsp->n_addresses; i++) { @@ -10505,10 +10660,12 @@ build_lswitch_ip_unicast_lookup(struct ovn_port *op, ds_clear(actions); ds_put_format(actions, action, op->json_key); - ovn_lflow_add_with_hint(lflows, op->od, S_SWITCH_IN_L2_LKUP, + ovn_lflow_add_with_lport_lflow_ref(lflows, op->od, + S_SWITCH_IN_L2_LKUP, 50, ds_cstr(match), - ds_cstr(actions), - &op->nbsp->header_); + ds_cstr(actions), NULL, NULL, + &op->nbsp->header_, + lflow_dep_mgr, op->key, op->od); } else if (!strcmp(op->nbsp->addresses[i], "unknown")) { continue; } else if (is_dynamic_lsp_address(op->nbsp->addresses[i])) { @@ -10523,10 +10680,12 @@ build_lswitch_ip_unicast_lookup(struct ovn_port *op, ds_clear(actions); ds_put_format(actions, action, op->json_key); - ovn_lflow_add_with_hint(lflows, op->od, S_SWITCH_IN_L2_LKUP, + ovn_lflow_add_with_lport_lflow_ref(lflows, op->od, + S_SWITCH_IN_L2_LKUP, 50, ds_cstr(match), - ds_cstr(actions), - &op->nbsp->header_); + ds_cstr(actions), NULL, NULL, + &op->nbsp->header_, + lflow_dep_mgr, op->key, op->od); } else if (!strcmp(op->nbsp->addresses[i], "router")) { if (!op->peer || !op->peer->nbrp || !ovs_scan(op->peer->nbrp->mac, @@ -10578,10 +10737,11 @@ build_lswitch_ip_unicast_lookup(struct ovn_port *op, ds_clear(actions); ds_put_format(actions, action, op->json_key); - ovn_lflow_add_with_hint(lflows, op->od, + ovn_lflow_add_with_lport_lflow_ref(lflows, op->od, S_SWITCH_IN_L2_LKUP, 50, ds_cstr(match), ds_cstr(actions), - &op->nbsp->header_); + NULL, NULL, &op->nbsp->header_, + lflow_dep_mgr, op->key, op->od); /* Add ethernet addresses specified in NAT rules on * distributed logical routers. */ @@ -10601,11 +10761,13 @@ build_lswitch_ip_unicast_lookup(struct ovn_port *op, ds_clear(actions); ds_put_format(actions, action, op->json_key); - ovn_lflow_add_with_hint(lflows, op->od, + ovn_lflow_add_with_lport_lflow_ref(lflows, op->od, S_SWITCH_IN_L2_LKUP, 50, ds_cstr(match), ds_cstr(actions), - &op->nbsp->header_); + NULL, NULL, &op->nbsp->header_, + lflow_dep_mgr, op->key, + op->od); } } } @@ -10854,7 +11016,7 @@ get_outport_for_routing_policy_nexthop(struct ovn_datapath *od, } static void -build_routing_policy_flow(struct hmap *lflows, struct ovn_datapath *od, +build_routing_policy_flow(struct lflow_data *lflows, struct ovn_datapath *od, const struct hmap *lr_ports, const struct nbrec_logical_router_policy *rule, const struct ovsdb_idl_row *stage_hint) @@ -10919,7 +11081,8 @@ build_routing_policy_flow(struct hmap *lflows, struct ovn_datapath *od, } static void -build_ecmp_routing_policy_flows(struct hmap *lflows, struct ovn_datapath *od, +build_ecmp_routing_policy_flows(struct lflow_data *lflows, + struct ovn_datapath *od, const struct hmap *lr_ports, const struct nbrec_logical_router_policy *rule, uint16_t ecmp_group_id) @@ -11055,7 +11218,7 @@ get_route_table_id(struct simap *route_tables, const char *route_table_name) } static void -build_route_table_lflow(struct ovn_datapath *od, struct hmap *lflows, +build_route_table_lflow(struct ovn_datapath *od, struct lflow_data *lflows, struct nbrec_logical_router_port *lrp, struct simap *route_tables) { @@ -11466,7 +11629,7 @@ find_static_route_outport(struct ovn_datapath *od, const struct hmap *lr_ports, } static void -add_ecmp_symmetric_reply_flows(struct hmap *lflows, +add_ecmp_symmetric_reply_flows(struct lflow_data *lflows, struct ovn_datapath *od, bool ct_masked_mark, const char *port_ip, @@ -11639,7 +11802,7 @@ add_ecmp_symmetric_reply_flows(struct hmap *lflows, } static void -build_ecmp_route_flow(struct hmap *lflows, struct ovn_datapath *od, +build_ecmp_route_flow(struct lflow_data *lflows, struct ovn_datapath *od, bool ct_masked_mark, const struct hmap *lr_ports, struct ecmp_groups_node *eg) @@ -11726,7 +11889,7 @@ build_ecmp_route_flow(struct hmap *lflows, struct ovn_datapath *od, } static void -add_route(struct hmap *lflows, struct ovn_datapath *od, +add_route(struct lflow_data *lflows, struct ovn_datapath *od, const struct ovn_port *op, const char *lrp_addr_s, const char *network_s, int plen, const char *gateway, bool is_src_route, const uint32_t rtb_id, @@ -11789,7 +11952,7 @@ add_route(struct hmap *lflows, struct ovn_datapath *od, } static void -build_static_route_flow(struct hmap *lflows, struct ovn_datapath *od, +build_static_route_flow(struct lflow_data *lflows, struct ovn_datapath *od, const struct hmap *lr_ports, const struct parsed_route *route_) { @@ -11899,7 +12062,7 @@ struct lrouter_nat_lb_flows_ctx { int prio; - struct hmap *lflows; + struct lflow_data *lflows; const struct shash *meter_groups; }; @@ -12029,7 +12192,7 @@ build_lrouter_nat_flows_for_lb(struct ovn_lb_vip *lb_vip, struct ovn_lb_datapaths *lb_dps, struct ovn_northd_lb_vip *vips_nb, const struct ovn_datapaths *lr_datapaths, - struct hmap *lflows, + struct lflow_data *lflows, struct ds *match, struct ds *action, const struct shash *meter_groups, const struct chassis_features *features, @@ -12209,7 +12372,7 @@ build_lrouter_nat_flows_for_lb(struct ovn_lb_vip *lb_vip, static void build_lswitch_flows_for_lb(struct ovn_lb_datapaths *lb_dps, - struct hmap *lflows, + struct lflow_data *lflows, const struct shash *meter_groups, const struct ovn_datapaths *ls_datapaths, const struct chassis_features *features, @@ -12270,7 +12433,7 @@ build_lswitch_flows_for_lb(struct ovn_lb_datapaths *lb_dps, */ static void build_lrouter_defrag_flows_for_lb(struct ovn_lb_datapaths *lb_dps, - struct hmap *lflows, + struct lflow_data *lflows, const struct ovn_datapaths *lr_datapaths, struct ds *match) { @@ -12296,7 +12459,7 @@ build_lrouter_defrag_flows_for_lb(struct ovn_lb_datapaths *lb_dps, static void build_lrouter_flows_for_lb(struct ovn_lb_datapaths *lb_dps, - struct hmap *lflows, + struct lflow_data *lflows, const struct shash *meter_groups, const struct ovn_datapaths *lr_datapaths, const struct chassis_features *features, @@ -12453,7 +12616,7 @@ lrouter_dnat_and_snat_is_stateless(const struct nbrec_nat *nat) */ static inline void lrouter_nat_add_ext_ip_match(struct ovn_datapath *od, - struct hmap *lflows, struct ds *match, + struct lflow_data *lflows, struct ds *match, const struct nbrec_nat *nat, bool is_v6, bool is_src, int cidr_bits) { @@ -12520,7 +12683,7 @@ build_lrouter_arp_flow(struct ovn_datapath *od, struct ovn_port *op, const char *ip_address, const char *eth_addr, struct ds *extra_match, bool drop, uint16_t priority, const struct ovsdb_idl_row *hint, - struct hmap *lflows) + struct lflow_data *lflows) { struct ds match = DS_EMPTY_INITIALIZER; struct ds actions = DS_EMPTY_INITIALIZER; @@ -12570,7 +12733,8 @@ build_lrouter_nd_flow(struct ovn_datapath *od, struct ovn_port *op, const char *sn_ip_address, const char *eth_addr, struct ds *extra_match, bool drop, uint16_t priority, const struct ovsdb_idl_row *hint, - struct hmap *lflows, const struct shash *meter_groups) + struct lflow_data *lflows, + const struct shash *meter_groups) { struct ds match = DS_EMPTY_INITIALIZER; struct ds actions = DS_EMPTY_INITIALIZER; @@ -12621,7 +12785,7 @@ build_lrouter_nd_flow(struct ovn_datapath *od, struct ovn_port *op, static void build_lrouter_nat_arp_nd_flow(struct ovn_datapath *od, struct ovn_nat *nat_entry, - struct hmap *lflows, + struct lflow_data *lflows, const struct shash *meter_groups) { struct lport_addresses *ext_addrs = &nat_entry->ext_addrs; @@ -12644,7 +12808,7 @@ build_lrouter_nat_arp_nd_flow(struct ovn_datapath *od, static void build_lrouter_port_nat_arp_nd_flow(struct ovn_port *op, struct ovn_nat *nat_entry, - struct hmap *lflows, + struct lflow_data *lflows, const struct shash *meter_groups) { struct lport_addresses *ext_addrs = &nat_entry->ext_addrs; @@ -12716,7 +12880,7 @@ build_lrouter_port_nat_arp_nd_flow(struct ovn_port *op, static void build_lrouter_drop_own_dest(struct ovn_port *op, enum ovn_stage stage, uint16_t priority, bool drop_snat_ip, - struct hmap *lflows) + struct lflow_data *lflows) { struct ds match_ips = DS_EMPTY_INITIALIZER; @@ -12779,7 +12943,8 @@ build_lrouter_drop_own_dest(struct ovn_port *op, enum ovn_stage stage, } static void -build_lrouter_force_snat_flows(struct hmap *lflows, struct ovn_datapath *od, +build_lrouter_force_snat_flows(struct lflow_data *lflows, + struct ovn_datapath *od, const char *ip_version, const char *ip_addr, const char *context) { @@ -12806,7 +12971,7 @@ build_lrouter_force_snat_flows(struct hmap *lflows, struct ovn_datapath *od, static void build_lrouter_force_snat_flows_op(struct ovn_port *op, - struct hmap *lflows, + struct lflow_data *lflows, struct ds *match, struct ds *actions) { ovs_assert(op->nbrp); @@ -12878,7 +13043,7 @@ build_lrouter_force_snat_flows_op(struct ovn_port *op, } static void -build_lrouter_bfd_flows(struct hmap *lflows, struct ovn_port *op, +build_lrouter_bfd_flows(struct lflow_data *lflows, struct ovn_port *op, const struct shash *meter_groups) { if (!op->has_bfd) { @@ -12933,7 +13098,7 @@ build_lrouter_bfd_flows(struct hmap *lflows, struct ovn_port *op, */ static void build_adm_ctrl_flows_for_lrouter( - struct ovn_datapath *od, struct hmap *lflows) + struct ovn_datapath *od, struct lflow_data *lflows) { ovs_assert(od->nbr); /* Logical VLANs not supported. @@ -12971,7 +13136,7 @@ build_gateway_get_l2_hdr_size(struct ovn_port *op) * function. */ static void OVS_PRINTF_FORMAT(9, 10) -build_gateway_mtu_flow(struct hmap *lflows, struct ovn_port *op, +build_gateway_mtu_flow(struct lflow_data *lflows, struct ovn_port *op, enum ovn_stage stage, uint16_t prio_low, uint16_t prio_high, struct ds *match, struct ds *actions, const struct ovsdb_idl_row *hint, @@ -13032,7 +13197,7 @@ consider_l3dgw_port_is_centralized(struct ovn_port *op) */ static void build_adm_ctrl_flows_for_lrouter_port( - struct ovn_port *op, struct hmap *lflows, + struct ovn_port *op, struct lflow_data *lflows, struct ds *match, struct ds *actions) { ovs_assert(op->nbrp); @@ -13086,7 +13251,7 @@ build_adm_ctrl_flows_for_lrouter_port( * lflows for logical routers. */ static void build_neigh_learning_flows_for_lrouter( - struct ovn_datapath *od, struct hmap *lflows, + struct ovn_datapath *od, struct lflow_data *lflows, struct ds *match, struct ds *actions, const struct shash *meter_groups) { @@ -13217,7 +13382,7 @@ build_neigh_learning_flows_for_lrouter( * for logical router ports. */ static void build_neigh_learning_flows_for_lrouter_port( - struct ovn_port *op, struct hmap *lflows, + struct ovn_port *op, struct lflow_data *lflows, struct ds *match, struct ds *actions) { ovs_assert(op->nbrp); @@ -13279,7 +13444,7 @@ build_neigh_learning_flows_for_lrouter_port( * Adv (RA) options and response. */ static void build_ND_RA_flows_for_lrouter_port( - struct ovn_port *op, struct hmap *lflows, + struct ovn_port *op, struct lflow_data *lflows, struct ds *match, struct ds *actions, const struct shash *meter_groups) { @@ -13394,7 +13559,8 @@ build_ND_RA_flows_for_lrouter_port( /* Logical router ingress table ND_RA_OPTIONS & ND_RA_RESPONSE: RS * responder, by default goto next. (priority 0). */ static void -build_ND_RA_flows_for_lrouter(struct ovn_datapath *od, struct hmap *lflows) +build_ND_RA_flows_for_lrouter(struct ovn_datapath *od, + struct lflow_data *lflows) { ovs_assert(od->nbr); ovn_lflow_add(lflows, od, S_ROUTER_IN_ND_RA_OPTIONS, 0, "1", "next;"); @@ -13405,7 +13571,7 @@ build_ND_RA_flows_for_lrouter(struct ovn_datapath *od, struct hmap *lflows) * by default goto next. (priority 0). */ static void build_ip_routing_pre_flows_for_lrouter(struct ovn_datapath *od, - struct hmap *lflows) + struct lflow_data *lflows) { ovs_assert(od->nbr); ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_ROUTING_PRE, 0, "1", @@ -13433,7 +13599,7 @@ build_ip_routing_pre_flows_for_lrouter(struct ovn_datapath *od, */ static void build_ip_routing_flows_for_lrp( - struct ovn_port *op, struct hmap *lflows) + struct ovn_port *op, struct lflow_data *lflows) { ovs_assert(op->nbrp); for (int i = 0; i < op->lrp_networks.n_ipv4_addrs; i++) { @@ -13459,8 +13625,9 @@ build_ip_routing_flows_for_lrp( * ports, add routes to the LSP's peer router. */ static void -build_ip_routing_flows_for_router_type_lsp( - struct ovn_port *op, const struct hmap *lr_ports, struct hmap *lflows) +build_ip_routing_flows_for_router_type_lsp(struct ovn_port *op, + const struct hmap *lr_ports, + struct lflow_data *lflows) { ovs_assert(op->nbsp); if (!lsp_is_router(op->nbsp)) { @@ -13497,7 +13664,7 @@ build_ip_routing_flows_for_router_type_lsp( static void build_static_route_flows_for_lrouter( struct ovn_datapath *od, const struct chassis_features *features, - struct hmap *lflows, const struct hmap *lr_ports, + struct lflow_data *lflows, const struct hmap *lr_ports, const struct hmap *bfd_connections) { ovs_assert(od->nbr); @@ -13561,7 +13728,7 @@ build_static_route_flows_for_lrouter( */ static void build_mcast_lookup_flows_for_lrouter( - struct ovn_datapath *od, struct hmap *lflows, + struct ovn_datapath *od, struct lflow_data *lflows, struct ds *match, struct ds *actions) { ovs_assert(od->nbr); @@ -13662,7 +13829,7 @@ build_mcast_lookup_flows_for_lrouter( * advances to the next table for ARP/ND resolution. */ static void build_ingress_policy_flows_for_lrouter( - struct ovn_datapath *od, struct hmap *lflows, + struct ovn_datapath *od, struct lflow_data *lflows, const struct hmap *lr_ports) { ovs_assert(od->nbr); @@ -13696,7 +13863,7 @@ build_ingress_policy_flows_for_lrouter( /* Local router ingress table ARP_RESOLVE: ARP Resolution. */ static void build_arp_resolve_flows_for_lrouter( - struct ovn_datapath *od, struct hmap *lflows) + struct ovn_datapath *od, struct lflow_data *lflows) { ovs_assert(od->nbr); /* Multicast packets already have the outport set so just advance to @@ -13714,7 +13881,8 @@ build_arp_resolve_flows_for_lrouter( } static void -routable_addresses_to_lflows(struct hmap *lflows, struct ovn_port *router_port, +routable_addresses_to_lflows(struct lflow_data *lflows, + struct ovn_port *router_port, struct ovn_port *peer, struct ds *match, struct ds *actions) { @@ -13757,7 +13925,7 @@ routable_addresses_to_lflows(struct hmap *lflows, struct ovn_port *router_port, /* This function adds ARP resolve flows related to a LRP. */ static void build_arp_resolve_flows_for_lrp( - struct ovn_port *op, struct hmap *lflows, + struct ovn_port *op, struct lflow_data *lflows, struct ds *match, struct ds *actions) { ovs_assert(op->nbrp); @@ -13841,9 +14009,10 @@ build_arp_resolve_flows_for_lrp( /* This function adds ARP resolve flows related to a LSP. */ static void build_arp_resolve_flows_for_lsp( - struct ovn_port *op, struct hmap *lflows, + struct ovn_port *op, struct lflow_data *lflows, const struct hmap *lr_ports, - struct ds *match, struct ds *actions) + struct ds *match, struct ds *actions, + struct objdep_mgr *lflow_dep_mgr) { ovs_assert(op->nbsp); if (!lsp_is_enabled(op->nbsp)) { @@ -13883,11 +14052,13 @@ build_arp_resolve_flows_for_lsp( ds_clear(actions); ds_put_format(actions, "eth.dst = %s; next;", ea_s); - ovn_lflow_add_with_hint(lflows, peer->od, + ovn_lflow_add_with_lport_lflow_ref(lflows, peer->od, S_ROUTER_IN_ARP_RESOLVE, 100, ds_cstr(match), ds_cstr(actions), - &op->nbsp->header_); + NULL, NULL, + &op->nbsp->header_, + lflow_dep_mgr, op->key, op->od); } } @@ -13914,11 +14085,13 @@ build_arp_resolve_flows_for_lsp( ds_clear(actions); ds_put_format(actions, "eth.dst = %s; next;", ea_s); - ovn_lflow_add_with_hint(lflows, peer->od, + ovn_lflow_add_with_lport_lflow_ref(lflows, peer->od, S_ROUTER_IN_ARP_RESOLVE, 100, ds_cstr(match), ds_cstr(actions), - &op->nbsp->header_); + NULL, NULL, + &op->nbsp->header_, + lflow_dep_mgr, op->key, op->od); } } } @@ -13994,7 +14167,8 @@ build_arp_resolve_flows_for_lsp( } static void -build_icmperr_pkt_big_flows(struct ovn_port *op, int mtu, struct hmap *lflows, +build_icmperr_pkt_big_flows(struct ovn_port *op, int mtu, + struct lflow_data *lflows, const struct shash *meter_groups, struct ds *match, struct ds *actions, enum ovn_stage stage, struct ovn_port *outport) @@ -14073,7 +14247,7 @@ build_icmperr_pkt_big_flows(struct ovn_port *op, int mtu, struct hmap *lflows, static void build_check_pkt_len_flows_for_lrp(struct ovn_port *op, - struct hmap *lflows, + struct lflow_data *lflows, const struct hmap *lr_ports, const struct shash *meter_groups, struct ds *match, @@ -14123,7 +14297,7 @@ build_check_pkt_len_flows_for_lrp(struct ovn_port *op, * */ static void build_check_pkt_len_flows_for_lrouter( - struct ovn_datapath *od, struct hmap *lflows, + struct ovn_datapath *od, struct lflow_data *lflows, const struct hmap *lr_ports, struct ds *match, struct ds *actions, const struct shash *meter_groups) @@ -14150,7 +14324,7 @@ build_check_pkt_len_flows_for_lrouter( /* Logical router ingress table GW_REDIRECT: Gateway redirect. */ static void build_gateway_redirect_flows_for_lrouter( - struct ovn_datapath *od, struct hmap *lflows, + struct ovn_datapath *od, struct lflow_data *lflows, struct ds *match, struct ds *actions) { ovs_assert(od->nbr); @@ -14235,7 +14409,7 @@ build_gateway_redirect_flows_for_lrouter( * and sends an ARP/IPv6 NA request (priority 100). */ static void build_arp_request_flows_for_lrouter( - struct ovn_datapath *od, struct hmap *lflows, + struct ovn_datapath *od, struct lflow_data *lflows, struct ds *match, struct ds *actions, const struct shash *meter_groups) { @@ -14313,7 +14487,7 @@ build_arp_request_flows_for_lrouter( */ static void build_egress_delivery_flows_for_lrouter_port( - struct ovn_port *op, struct hmap *lflows, + struct ovn_port *op, struct lflow_data *lflows, struct ds *match, struct ds *actions) { ovs_assert(op->nbrp); @@ -14355,7 +14529,7 @@ build_egress_delivery_flows_for_lrouter_port( static void build_misc_local_traffic_drop_flows_for_lrouter( - struct ovn_datapath *od, struct hmap *lflows) + struct ovn_datapath *od, struct lflow_data *lflows) { ovs_assert(od->nbr); /* Allow IGMP and MLD packets (with TTL = 1) if the router is @@ -14437,7 +14611,7 @@ build_misc_local_traffic_drop_flows_for_lrouter( static void build_dhcpv6_reply_flows_for_lrouter_port( - struct ovn_port *op, struct hmap *lflows, + struct ovn_port *op, struct lflow_data *lflows, struct ds *match) { ovs_assert(op->nbrp); @@ -14457,7 +14631,7 @@ build_dhcpv6_reply_flows_for_lrouter_port( static void build_ipv6_input_flows_for_lrouter_port( - struct ovn_port *op, struct hmap *lflows, + struct ovn_port *op, struct lflow_data *lflows, struct ds *match, struct ds *actions, const struct shash *meter_groups) { @@ -14625,7 +14799,7 @@ build_ipv6_input_flows_for_lrouter_port( static void build_lrouter_arp_nd_for_datapath(struct ovn_datapath *od, - struct hmap *lflows, + struct lflow_data *lflows, const struct shash *meter_groups) { ovs_assert(od->nbr); @@ -14673,7 +14847,7 @@ build_lrouter_arp_nd_for_datapath(struct ovn_datapath *od, /* Logical router ingress table 3: IP Input for IPv4. */ static void build_lrouter_ipv4_ip_input(struct ovn_port *op, - struct hmap *lflows, + struct lflow_data *lflows, struct ds *match, struct ds *actions, const struct shash *meter_groups) { @@ -14994,7 +15168,7 @@ build_lrouter_in_unsnat_match(struct ovn_datapath *od, } static void -build_lrouter_in_unsnat_stateless_flow(struct hmap *lflows, +build_lrouter_in_unsnat_stateless_flow(struct lflow_data *lflows, struct ovn_datapath *od, const struct nbrec_nat *nat, struct ds *match, @@ -15016,7 +15190,7 @@ build_lrouter_in_unsnat_stateless_flow(struct hmap *lflows, } static void -build_lrouter_in_unsnat_in_czone_flow(struct hmap *lflows, +build_lrouter_in_unsnat_in_czone_flow(struct lflow_data *lflows, struct ovn_datapath *od, const struct nbrec_nat *nat, struct ds *match, bool distributed_nat, @@ -15050,7 +15224,8 @@ build_lrouter_in_unsnat_in_czone_flow(struct hmap *lflows, } static void -build_lrouter_in_unsnat_flow(struct hmap *lflows, struct ovn_datapath *od, +build_lrouter_in_unsnat_flow(struct lflow_data *lflows, + struct ovn_datapath *od, const struct nbrec_nat *nat, struct ds *match, bool distributed_nat, bool is_v6, struct ovn_port *l3dgw_port) @@ -15071,7 +15246,7 @@ build_lrouter_in_unsnat_flow(struct hmap *lflows, struct ovn_datapath *od, } static void -build_lrouter_in_dnat_flow(struct hmap *lflows, struct ovn_datapath *od, +build_lrouter_in_dnat_flow(struct lflow_data *lflows, struct ovn_datapath *od, const struct nbrec_nat *nat, struct ds *match, struct ds *actions, bool distributed_nat, int cidr_bits, bool is_v6, @@ -15141,7 +15316,8 @@ build_lrouter_in_dnat_flow(struct hmap *lflows, struct ovn_datapath *od, } static void -build_lrouter_out_undnat_flow(struct hmap *lflows, struct ovn_datapath *od, +build_lrouter_out_undnat_flow(struct lflow_data *lflows, + struct ovn_datapath *od, const struct nbrec_nat *nat, struct ds *match, struct ds *actions, bool distributed_nat, struct eth_addr mac, bool is_v6, @@ -15191,7 +15367,8 @@ build_lrouter_out_undnat_flow(struct hmap *lflows, struct ovn_datapath *od, } static void -build_lrouter_out_is_dnat_local(struct hmap *lflows, struct ovn_datapath *od, +build_lrouter_out_is_dnat_local(struct lflow_data *lflows, + struct ovn_datapath *od, const struct nbrec_nat *nat, struct ds *match, struct ds *actions, bool distributed_nat, bool is_v6, struct ovn_port *l3dgw_port) @@ -15221,7 +15398,8 @@ build_lrouter_out_is_dnat_local(struct hmap *lflows, struct ovn_datapath *od, } static void -build_lrouter_out_snat_match(struct hmap *lflows, struct ovn_datapath *od, +build_lrouter_out_snat_match(struct lflow_data *lflows, + struct ovn_datapath *od, const struct nbrec_nat *nat, struct ds *match, bool distributed_nat, int cidr_bits, bool is_v6, struct ovn_port *l3dgw_port) @@ -15249,7 +15427,7 @@ build_lrouter_out_snat_match(struct hmap *lflows, struct ovn_datapath *od, } static void -build_lrouter_out_snat_stateless_flow(struct hmap *lflows, +build_lrouter_out_snat_stateless_flow(struct lflow_data *lflows, struct ovn_datapath *od, const struct nbrec_nat *nat, struct ds *match, struct ds *actions, @@ -15292,7 +15470,7 @@ build_lrouter_out_snat_stateless_flow(struct hmap *lflows, } static void -build_lrouter_out_snat_in_czone_flow(struct hmap *lflows, +build_lrouter_out_snat_in_czone_flow(struct lflow_data *lflows, struct ovn_datapath *od, const struct nbrec_nat *nat, struct ds *match, @@ -15353,7 +15531,7 @@ build_lrouter_out_snat_in_czone_flow(struct hmap *lflows, } static void -build_lrouter_out_snat_flow(struct hmap *lflows, struct ovn_datapath *od, +build_lrouter_out_snat_flow(struct lflow_data *lflows, struct ovn_datapath *od, const struct nbrec_nat *nat, struct ds *match, struct ds *actions, bool distributed_nat, struct eth_addr mac, int cidr_bits, bool is_v6, @@ -15400,7 +15578,7 @@ build_lrouter_out_snat_flow(struct hmap *lflows, struct ovn_datapath *od, } static void -build_lrouter_ingress_nat_check_pkt_len(struct hmap *lflows, +build_lrouter_ingress_nat_check_pkt_len(struct lflow_data *lflows, const struct nbrec_nat *nat, struct ovn_datapath *od, bool is_v6, struct ds *match, struct ds *actions, @@ -15471,7 +15649,7 @@ build_lrouter_ingress_nat_check_pkt_len(struct hmap *lflows, } static void -build_lrouter_ingress_flow(struct hmap *lflows, struct ovn_datapath *od, +build_lrouter_ingress_flow(struct lflow_data *lflows, struct ovn_datapath *od, const struct nbrec_nat *nat, struct ds *match, struct ds *actions, struct eth_addr mac, bool distributed_nat, bool is_v6, @@ -15649,7 +15827,8 @@ lrouter_check_nat_entry(struct ovn_datapath *od, const struct nbrec_nat *nat, /* NAT, Defrag and load balancing. */ static void -build_lrouter_nat_defrag_and_lb(struct ovn_datapath *od, struct hmap *lflows, +build_lrouter_nat_defrag_and_lb(struct ovn_datapath *od, + struct lflow_data *lflows, const struct hmap *ls_ports, const struct hmap *lr_ports, struct ds *match, @@ -16050,7 +16229,7 @@ struct lswitch_flow_build_info { const struct hmap *ls_ports; const struct hmap *lr_ports; const struct hmap *port_groups; - struct hmap *lflows; + struct lflow_data *lflows; struct hmap *igmp_groups; const struct shash *meter_groups; const struct hmap *lb_dps_map; @@ -16138,27 +16317,28 @@ build_lswitch_and_lrouter_iterate_by_lsp(struct ovn_port *op, const struct shash *meter_groups, struct ds *match, struct ds *actions, - struct hmap *lflows) + struct lflow_data *lflows) { ovs_assert(op->nbsp); - start_collecting_lflows(); /* Build Logical Switch Flows. */ - build_lswitch_port_sec_op(op, lflows, actions, match); - build_lswitch_learn_fdb_op(op, lflows, actions, match); - build_lswitch_arp_nd_responder_skip_local(op, lflows, match); + build_lswitch_port_sec_op(op, lflows, actions, match, &op->lflow_dep_mgr); + build_lswitch_learn_fdb_op(op, lflows, actions, match, &op->lflow_dep_mgr); + build_lswitch_arp_nd_responder_skip_local(op, lflows, match, + &op->lflow_dep_mgr); build_lswitch_arp_nd_responder_known_ips(op, lflows, ls_ports, - meter_groups, actions, match); - build_lswitch_dhcp_options_and_response(op, lflows, meter_groups); - build_lswitch_external_port(op, lflows); - build_lswitch_ip_unicast_lookup(op, lflows, actions, match); + meter_groups, actions, match, + &op->lflow_dep_mgr); + build_lswitch_dhcp_options_and_response(op, lflows, meter_groups, + &op->lflow_dep_mgr); + build_lswitch_external_port(op, lflows, &op->lflow_dep_mgr); + build_lswitch_ip_unicast_lookup(op, lflows, actions, match, + &op->lflow_dep_mgr); /* Build Logical Router Flows. */ build_ip_routing_flows_for_router_type_lsp(op, lr_ports, lflows); - build_arp_resolve_flows_for_lsp(op, lflows, lr_ports, match, actions); - - link_ovn_port_to_lflows(op, &collected_lflows); - end_collecting_lflows(); + build_arp_resolve_flows_for_lsp(op, lflows, lr_ports, match, actions, + &op->lflow_dep_mgr); } /* Helper function to combine all lflow generation which is iterated by logical @@ -16354,7 +16534,7 @@ build_lswitch_and_lrouter_flows(const struct ovn_datapaths *ls_datapaths, const struct hmap *ls_ports, const struct hmap *lr_ports, const struct hmap *port_groups, - struct hmap *lflows, + struct lflow_data *lflows, struct hmap *igmp_groups, const struct shash *meter_groups, const struct hmap *lb_dps_map, @@ -16399,7 +16579,10 @@ build_lswitch_and_lrouter_flows(const struct ovn_datapaths *ls_datapaths, /* Run thread pool. */ run_pool_callback(build_lflows_pool, NULL, NULL, noop_callback); - fix_flow_map_size(lflows, lsiv, build_lflows_pool->size); + fix_flow_map_size(&lflows->lflows_match_map, lsiv, + build_lflows_pool->size); + fix_flow_map_size(&lflows->lflows_hash_map, lsiv, + build_lflows_pool->size); for (index = 0; index < build_lflows_pool->size; index++) { ds_destroy(&lsiv[index].match); @@ -16490,17 +16673,33 @@ static ssize_t max_seen_lflow_size = 128; void lflow_data_init(struct lflow_data *data) { - fast_hmap_size_for(&data->lflows, max_seen_lflow_size); + fast_hmap_size_for(&data->lflows_match_map, max_seen_lflow_size); + fast_hmap_size_for(&data->lflows_hash_map, max_seen_lflow_size); + hmap_init(&data->ls_dp_groups); + hmap_init(&data->lr_dp_groups); } void lflow_data_destroy(struct lflow_data *data) { struct ovn_lflow *lflow; - HMAP_FOR_EACH_SAFE (lflow, hmap_node, &data->lflows) { - ovn_lflow_destroy(&data->lflows, lflow); + HMAP_FOR_EACH_SAFE (lflow, hmap_node, &data->lflows_match_map) { + ovn_lflow_destroy(data, lflow); + } + hmap_destroy(&data->lflows_match_map); + hmap_destroy(&data->lflows_hash_map); + + struct ovn_dp_group *dpg; + HMAP_FOR_EACH_POP (dpg, node, &data->ls_dp_groups) { + bitmap_free(dpg->bitmap); + free(dpg); + } + hmap_destroy(&data->ls_dp_groups); + HMAP_FOR_EACH_POP (dpg, node, &data->lr_dp_groups) { + bitmap_free(dpg->bitmap); + free(dpg); } - hmap_destroy(&data->lflows); + hmap_destroy(&data->lr_dp_groups); } void run_update_worker_pool(int n_threads) @@ -16544,11 +16743,172 @@ create_sb_multicast_group(struct ovsdb_idl_txn *ovnsb_txn, return sbmc; } +static void +sync_lflow_to_sb(struct ovsdb_idl_txn *ovnsb_txn, + struct lflow_input *lflow_input, + struct lflow_data *lflow_data, + struct ovn_lflow *lflow, + const struct sbrec_logical_flow *sbflow) +{ + size_t n_datapaths; + struct ovn_datapath **datapaths_array; + struct hmap *dp_groups; + bool is_switch; + if (ovn_stage_to_datapath_type(lflow->stage) == DP_SWITCH) { + n_datapaths = ods_size(lflow_input->ls_datapaths); + datapaths_array = lflow_input->ls_datapaths->array; + dp_groups = &lflow_data->ls_dp_groups; + is_switch = true; + } else { + n_datapaths = ods_size(lflow_input->lr_datapaths); + datapaths_array = lflow_input->lr_datapaths->array; + dp_groups = &lflow_data->lr_dp_groups; + is_switch = false; + } + + lflow->n_ods = bitmap_count1(lflow->dpg_bitmap, n_datapaths); + ovs_assert(lflow->n_ods); + + struct ovn_dp_group *pre_sync_dpg = lflow->dpg; + if (lflow->n_ods == 1) { + /* There is only one datapath, so it should be moved out of the + * group to a single 'od'. */ + size_t index = bitmap_scan(lflow->dpg_bitmap, true, 0, + n_datapaths); + + lflow->od = datapaths_array[index]; + lflow->dpg = NULL; + } else { + lflow->od = NULL; + } + + struct sbrec_logical_dp_group *sbrec_dp_group = NULL; + + if (!sbflow) { + lflow->sb_uuid = uuid_random(); + sbflow = sbrec_logical_flow_insert_persist_uuid(ovnsb_txn, + &lflow->sb_uuid); + const char *pipeline = ovn_stage_get_pipeline_name(lflow->stage); + uint8_t table = ovn_stage_get_table(lflow->stage); + sbrec_logical_flow_set_pipeline(sbflow, pipeline); + sbrec_logical_flow_set_table_id(sbflow, table); + sbrec_logical_flow_set_priority(sbflow, lflow->priority); + sbrec_logical_flow_set_match(sbflow, lflow->match); + sbrec_logical_flow_set_actions(sbflow, lflow->actions); + if (lflow->io_port) { + struct smap tags = SMAP_INITIALIZER(&tags); + smap_add(&tags, "in_out_port", lflow->io_port); + sbrec_logical_flow_set_tags(sbflow, &tags); + smap_destroy(&tags); + } + sbrec_logical_flow_set_controller_meter(sbflow, lflow->ctrl_meter); + + /* Trim the source locator lflow->where, which looks something like + * "ovn/northd/northd.c:1234", down to just the part following the + * last slash, e.g. "northd.c:1234". */ + const char *slash = strrchr(lflow->where, '/'); +#if _WIN32 + const char *backslash = strrchr(lflow->where, '\\'); + if (!slash || backslash > slash) { + slash = backslash; + } +#endif + const char *where = slash ? slash + 1 : lflow->where; + + struct smap ids = SMAP_INITIALIZER(&ids); + smap_add(&ids, "stage-name", ovn_stage_to_str(lflow->stage)); + smap_add(&ids, "source", where); + if (lflow->stage_hint) { + smap_add(&ids, "stage-hint", lflow->stage_hint); + } + sbrec_logical_flow_set_external_ids(sbflow, &ids); + smap_destroy(&ids); + + } else { + lflow->sb_uuid = sbflow->header_.uuid; + sbrec_dp_group = sbflow->logical_dp_group; + + if (lflow_input->ovn_internal_version_changed) { + const char *stage_name = smap_get_def(&sbflow->external_ids, + "stage-name", ""); + const char *stage_hint = smap_get_def(&sbflow->external_ids, + "stage-hint", ""); + const char *source = smap_get_def(&sbflow->external_ids, + "source", ""); + + if (strcmp(stage_name, ovn_stage_to_str(lflow->stage))) { + sbrec_logical_flow_update_external_ids_setkey( + sbflow, "stage-name", ovn_stage_to_str(lflow->stage)); + } + if (lflow->stage_hint) { + if (strcmp(stage_hint, lflow->stage_hint)) { + sbrec_logical_flow_update_external_ids_setkey( + sbflow, "stage-hint", lflow->stage_hint); + } + } + if (lflow->where) { + + /* Trim the source locator lflow->where, which looks something + * like "ovn/northd/northd.c:1234", down to just the part + * following the last slash, e.g. "northd.c:1234". */ + const char *slash = strrchr(lflow->where, '/'); +#if _WIN32 + const char *backslash = strrchr(lflow->where, '\\'); + if (!slash || backslash > slash) { + slash = backslash; + } +#endif + const char *where = slash ? slash + 1 : lflow->where; + + if (strcmp(source, where)) { + sbrec_logical_flow_update_external_ids_setkey( + sbflow, "source", where); + } + } + } + } + + if (lflow->od) { + sbrec_logical_flow_set_logical_datapath(sbflow, lflow->od->sb); + sbrec_logical_flow_set_logical_dp_group(sbflow, NULL); + } else { + sbrec_logical_flow_set_logical_datapath(sbflow, NULL); + lflow->dpg = ovn_dp_group_get(dp_groups, lflow->n_ods, + lflow->dpg_bitmap, + n_datapaths); + if (lflow->dpg) { + /* Update the dpg's sb dp_group. */ + lflow->dpg->dp_group = sbrec_logical_dp_group_table_get_for_uuid( + lflow_input->sbrec_logical_dp_group_table, + &lflow->dpg->dpg_uuid); + ovs_assert(lflow->dpg->dp_group); + } else { + lflow->dpg = ovn_dp_group_create( + ovnsb_txn, dp_groups, sbrec_dp_group, + lflow->n_ods, lflow->dpg_bitmap, + n_datapaths, is_switch, + lflow_input->ls_datapaths, + lflow_input->lr_datapaths); + } + sbrec_logical_flow_set_logical_dp_group(sbflow, + lflow->dpg->dp_group); + } + + if (pre_sync_dpg != lflow->dpg) { + if (lflow->dpg) { + inc_ovn_dp_group_ref(lflow->dpg); + } + if (pre_sync_dpg) { + dec_ovn_dp_group_ref(dp_groups, pre_sync_dpg); + } + } +} + /* Updates the Logical_Flow and Multicast_Group tables in the OVN_SB database, * constructing their contents based on the OVN_NB database. */ void build_lflows(struct ovsdb_idl_txn *ovnsb_txn, struct lflow_input *input_data, - struct hmap *lflows) + struct lflow_data *lflow_data) { struct hmap mcast_groups; struct hmap igmp_groups; @@ -16563,7 +16923,7 @@ void build_lflows(struct ovsdb_idl_txn *ovnsb_txn, input_data->lr_datapaths, input_data->ls_ports, input_data->lr_ports, - input_data->port_groups, lflows, + input_data->port_groups, lflow_data, &igmp_groups, input_data->meter_groups, input_data->lb_datapaths_map, @@ -16578,72 +16938,17 @@ void build_lflows(struct ovsdb_idl_txn *ovnsb_txn, /* Parallel build may result in a suboptimal hash. Resize the * hash to a correct size before doing lookups */ + struct hmap *lflows = &lflow_data->lflows_match_map; hmap_expand(lflows); + hmap_expand(&lflow_data->lflows_hash_map); if (hmap_count(lflows) > max_seen_lflow_size) { max_seen_lflow_size = hmap_count(lflows); } - stopwatch_start(LFLOWS_DP_GROUPS_STOPWATCH_NAME, time_msec()); - /* Collecting all unique datapath groups. */ - struct hmap ls_dp_groups = HMAP_INITIALIZER(&ls_dp_groups); - struct hmap lr_dp_groups = HMAP_INITIALIZER(&lr_dp_groups); - struct hmap single_dp_lflows; - - /* Single dp_flows will never grow bigger than lflows, - * thus the two hmaps will remain the same size regardless - * of how many elements we remove from lflows and add to - * single_dp_lflows. - * Note - lflows is always sized for at least 128 flows. - */ - fast_hmap_size_for(&single_dp_lflows, max_seen_lflow_size); - - struct ovn_lflow *lflow; - HMAP_FOR_EACH_SAFE (lflow, hmap_node, lflows) { - struct ovn_datapath **datapaths_array; - size_t n_datapaths; - - if (ovn_stage_to_datapath_type(lflow->stage) == DP_SWITCH) { - n_datapaths = ods_size(input_data->ls_datapaths); - datapaths_array = input_data->ls_datapaths->array; - } else { - n_datapaths = ods_size(input_data->lr_datapaths); - datapaths_array = input_data->lr_datapaths->array; - } - - lflow->n_ods = bitmap_count1(lflow->dpg_bitmap, n_datapaths); - - ovs_assert(lflow->n_ods); - - if (lflow->n_ods == 1) { - /* There is only one datapath, so it should be moved out of the - * group to a single 'od'. */ - size_t index = bitmap_scan(lflow->dpg_bitmap, true, 0, - n_datapaths); - - bitmap_set0(lflow->dpg_bitmap, index); - lflow->od = datapaths_array[index]; - - /* Logical flow should be re-hashed to allow lookups. */ - uint32_t hash = hmap_node_hash(&lflow->hmap_node); - /* Remove from lflows. */ - hmap_remove(lflows, &lflow->hmap_node); - hash = ovn_logical_flow_hash_datapath(&lflow->od->sb->header_.uuid, - hash); - /* Add to single_dp_lflows. */ - hmap_insert_fast(&single_dp_lflows, &lflow->hmap_node, hash); - } - } - - /* Merge multiple and single dp hashes. */ - - fast_hmap_merge(lflows, &single_dp_lflows); - - hmap_destroy(&single_dp_lflows); - - stopwatch_stop(LFLOWS_DP_GROUPS_STOPWATCH_NAME, time_msec()); stopwatch_start(LFLOWS_TO_SB_STOPWATCH_NAME, time_msec()); + struct ovn_lflow *lflow; struct hmap lflows_temp = HMAP_INITIALIZER(&lflows_temp); /* Push changes to the Logical_Flow table to database. */ const struct sbrec_logical_flow *sbflow; @@ -16687,68 +16992,15 @@ void build_lflows(struct ovsdb_idl_txn *ovnsb_txn, = !strcmp(sbflow->pipeline, "ingress") ? P_IN : P_OUT; lflow = ovn_lflow_find( - lflows, dp_group ? NULL : logical_datapath_od, + lflows, ovn_stage_build(ovn_datapath_get_type(logical_datapath_od), pipeline, sbflow->table_id), sbflow->priority, sbflow->match, sbflow->actions, sbflow->controller_meter, sbflow->hash); if (lflow) { - struct hmap *dp_groups; - size_t n_datapaths; - bool is_switch; - - lflow->sb_uuid = sbflow->header_.uuid; - is_switch = ovn_stage_to_datapath_type(lflow->stage) == DP_SWITCH; - if (is_switch) { - n_datapaths = ods_size(input_data->ls_datapaths); - dp_groups = &ls_dp_groups; - } else { - n_datapaths = ods_size(input_data->lr_datapaths); - dp_groups = &lr_dp_groups; - } - if (input_data->ovn_internal_version_changed) { - const char *stage_name = smap_get_def(&sbflow->external_ids, - "stage-name", ""); - const char *stage_hint = smap_get_def(&sbflow->external_ids, - "stage-hint", ""); - const char *source = smap_get_def(&sbflow->external_ids, - "source", ""); - - if (strcmp(stage_name, ovn_stage_to_str(lflow->stage))) { - sbrec_logical_flow_update_external_ids_setkey(sbflow, - "stage-name", ovn_stage_to_str(lflow->stage)); - } - if (lflow->stage_hint) { - if (strcmp(stage_hint, lflow->stage_hint)) { - sbrec_logical_flow_update_external_ids_setkey(sbflow, - "stage-hint", lflow->stage_hint); - } - } - if (lflow->where) { - if (strcmp(source, lflow->where)) { - sbrec_logical_flow_update_external_ids_setkey(sbflow, - "source", lflow->where); - } - } - } - - if (lflow->od) { - sbrec_logical_flow_set_logical_datapath(sbflow, lflow->od->sb); - sbrec_logical_flow_set_logical_dp_group(sbflow, NULL); - } else { - lflow->dpg = ovn_dp_group_get_or_create( - ovnsb_txn, dp_groups, dp_group, - lflow->n_ods, lflow->dpg_bitmap, - n_datapaths, is_switch, - input_data->ls_datapaths, - input_data->lr_datapaths); - - sbrec_logical_flow_set_logical_datapath(sbflow, NULL); - sbrec_logical_flow_set_logical_dp_group(sbflow, - lflow->dpg->dp_group); - } + sync_lflow_to_sb(ovnsb_txn, input_data, lflow_data, + lflow, sbflow); - /* This lflow updated. Not needed anymore. */ hmap_remove(lflows, &lflow->hmap_node); hmap_insert(&lflows_temp, &lflow->hmap_node, hmap_node_hash(&lflow->hmap_node)); @@ -16758,71 +17010,8 @@ void build_lflows(struct ovsdb_idl_txn *ovnsb_txn, } HMAP_FOR_EACH_SAFE (lflow, hmap_node, lflows) { - const char *pipeline = ovn_stage_get_pipeline_name(lflow->stage); - uint8_t table = ovn_stage_get_table(lflow->stage); - struct hmap *dp_groups; - size_t n_datapaths; - bool is_switch; - - is_switch = ovn_stage_to_datapath_type(lflow->stage) == DP_SWITCH; - if (is_switch) { - n_datapaths = ods_size(input_data->ls_datapaths); - dp_groups = &ls_dp_groups; - } else { - n_datapaths = ods_size(input_data->lr_datapaths); - dp_groups = &lr_dp_groups; - } - - lflow->sb_uuid = uuid_random(); - sbflow = sbrec_logical_flow_insert_persist_uuid(ovnsb_txn, - &lflow->sb_uuid); - if (lflow->od) { - sbrec_logical_flow_set_logical_datapath(sbflow, lflow->od->sb); - } else { - lflow->dpg = ovn_dp_group_get_or_create( - ovnsb_txn, dp_groups, NULL, - lflow->n_ods, lflow->dpg_bitmap, - n_datapaths, is_switch, - input_data->ls_datapaths, - input_data->lr_datapaths); - - sbrec_logical_flow_set_logical_dp_group(sbflow, - lflow->dpg->dp_group); - } - - sbrec_logical_flow_set_pipeline(sbflow, pipeline); - sbrec_logical_flow_set_table_id(sbflow, table); - sbrec_logical_flow_set_priority(sbflow, lflow->priority); - sbrec_logical_flow_set_match(sbflow, lflow->match); - sbrec_logical_flow_set_actions(sbflow, lflow->actions); - if (lflow->io_port) { - struct smap tags = SMAP_INITIALIZER(&tags); - smap_add(&tags, "in_out_port", lflow->io_port); - sbrec_logical_flow_set_tags(sbflow, &tags); - smap_destroy(&tags); - } - sbrec_logical_flow_set_controller_meter(sbflow, lflow->ctrl_meter); - - /* Trim the source locator lflow->where, which looks something like - * "ovn/northd/northd.c:1234", down to just the part following the - * last slash, e.g. "northd.c:1234". */ - const char *slash = strrchr(lflow->where, '/'); -#if _WIN32 - const char *backslash = strrchr(lflow->where, '\\'); - if (!slash || backslash > slash) { - slash = backslash; - } -#endif - const char *where = slash ? slash + 1 : lflow->where; - - struct smap ids = SMAP_INITIALIZER(&ids); - smap_add(&ids, "stage-name", ovn_stage_to_str(lflow->stage)); - smap_add(&ids, "source", where); - if (lflow->stage_hint) { - smap_add(&ids, "stage-hint", lflow->stage_hint); - } - sbrec_logical_flow_set_external_ids(sbflow, &ids); - smap_destroy(&ids); + sync_lflow_to_sb(ovnsb_txn, input_data, lflow_data, lflow, NULL); + hmap_remove(lflows, &lflow->hmap_node); hmap_insert(&lflows_temp, &lflow->hmap_node, hmap_node_hash(&lflow->hmap_node)); @@ -16831,17 +17020,6 @@ void build_lflows(struct ovsdb_idl_txn *ovnsb_txn, hmap_destroy(&lflows_temp); stopwatch_stop(LFLOWS_TO_SB_STOPWATCH_NAME, time_msec()); - struct ovn_dp_group *dpg; - HMAP_FOR_EACH_POP (dpg, node, &ls_dp_groups) { - bitmap_free(dpg->bitmap); - free(dpg); - } - hmap_destroy(&ls_dp_groups); - HMAP_FOR_EACH_POP (dpg, node, &lr_dp_groups) { - bitmap_free(dpg->bitmap); - free(dpg); - } - hmap_destroy(&lr_dp_groups); /* Push changes to the Multicast_Group table to database. */ const struct sbrec_multicast_group *sbmc; @@ -16890,119 +17068,188 @@ void build_lflows(struct ovsdb_idl_txn *ovnsb_txn, hmap_destroy(&mcast_groups); } +static bool +is_lflow_and_od_type_match(struct ovn_datapath *od, + struct ovn_lflow *lflow) +{ + enum ovn_datapath_type type = od->nbs ? DP_SWITCH : DP_ROUTER; + return ovn_stage_to_datapath_type(lflow->stage) == type; +} + +/* Unlinks the lflows stored in the resource to object nodes for the + * datapath 'od' from the lflow dependecy manager. + * It basically clears the datapath id of the 'od' for the lflows + * in the 'res_node'. + */ static void -sync_lsp_lflows_to_sb(struct ovsdb_idl_txn *ovnsb_txn, - struct lflow_input *lflow_input, - struct hmap *lflows, - struct ovn_lflow *lflow) +unlink_objres_lflows(struct resource_to_objects_node *res_node, + struct ovn_datapath *od, + struct lflow_data *lflow_data, + struct objdep_mgr *lflowdep_mgr) { - size_t n_datapaths; - struct ovn_datapath **datapaths_array; - if (ovn_stage_to_datapath_type(lflow->stage) == DP_SWITCH) { - n_datapaths = ods_size(lflow_input->ls_datapaths); - datapaths_array = lflow_input->ls_datapaths->array; - } else { - n_datapaths = ods_size(lflow_input->lr_datapaths); - datapaths_array = lflow_input->lr_datapaths->array; - } - uint32_t n_ods = bitmap_count1(lflow->dpg_bitmap, n_datapaths); - ovs_assert(n_ods == 1); - /* There is only one datapath, so it should be moved out of the - * group to a single 'od'. */ - size_t index = bitmap_scan(lflow->dpg_bitmap, true, 0, - n_datapaths); - - bitmap_set0(lflow->dpg_bitmap, index); - lflow->od = datapaths_array[index]; - - /* Logical flow should be re-hashed to allow lookups. */ - uint32_t hash = hmap_node_hash(&lflow->hmap_node); - /* Remove from lflows. */ - hmap_remove(lflows, &lflow->hmap_node); - hash = ovn_logical_flow_hash_datapath(&lflow->od->sb->header_.uuid, - hash); - /* Add back. */ - hmap_insert(lflows, &lflow->hmap_node, hash); - - /* Sync to SB. */ - const struct sbrec_logical_flow *sbflow; - /* Note: uuid_random acquires a global mutex. If we parallelize the sync to - * SB this may become a bottleneck. */ - lflow->sb_uuid = uuid_random(); - sbflow = sbrec_logical_flow_insert_persist_uuid(ovnsb_txn, - &lflow->sb_uuid); - const char *pipeline = ovn_stage_get_pipeline_name(lflow->stage); - uint8_t table = ovn_stage_get_table(lflow->stage); - sbrec_logical_flow_set_logical_datapath(sbflow, lflow->od->sb); - sbrec_logical_flow_set_logical_dp_group(sbflow, NULL); - sbrec_logical_flow_set_pipeline(sbflow, pipeline); - sbrec_logical_flow_set_table_id(sbflow, table); - sbrec_logical_flow_set_priority(sbflow, lflow->priority); - sbrec_logical_flow_set_match(sbflow, lflow->match); - sbrec_logical_flow_set_actions(sbflow, lflow->actions); - if (lflow->io_port) { - struct smap tags = SMAP_INITIALIZER(&tags); - smap_add(&tags, "in_out_port", lflow->io_port); - sbrec_logical_flow_set_tags(sbflow, &tags); - smap_destroy(&tags); - } - sbrec_logical_flow_set_controller_meter(sbflow, lflow->ctrl_meter); - /* Trim the source locator lflow->where, which looks something like - * "ovn/northd/northd.c:1234", down to just the part following the - * last slash, e.g. "northd.c:1234". */ - const char *slash = strrchr(lflow->where, '/'); -#if _WIN32 - const char *backslash = strrchr(lflow->where, '\\'); - if (!slash || backslash > slash) { - slash = backslash; + if (!res_node) { + return; } -#endif - const char *where = slash ? slash + 1 : lflow->where; - struct smap ids = SMAP_INITIALIZER(&ids); - smap_add(&ids, "stage-name", ovn_stage_to_str(lflow->stage)); - smap_add(&ids, "source", where); - if (lflow->stage_hint) { - smap_add(&ids, "stage-hint", lflow->stage_hint); + struct object_to_resources_list_node *resource_list_node; + RESOURCE_FOR_EACH_OBJ (resource_list_node, res_node) { + const struct uuid *obj_uuid = &resource_list_node->obj_uuid; + struct ovn_lflow *lflow = ovn_lflow_uuid_find( + &lflow_data->lflows_hash_map, obj_uuid); + if (!lflow) { + continue; + } + + + /* Check if the lflow datapath is same the od datapath. */ + if (is_lflow_and_od_type_match(od, lflow)) { + bitmap_set0(lflow->dpg_bitmap, od->index); + } else { + /* The datapath type doesn't match. Which means this lflow was + * added due to a resource in the other type. + * Eg. For every logical switch port whose lswitch is connected + * to a router, an lflow is added in the lr_in_arp_resolve stage. + * Get the datapath index of this router and clear it. + * + * OBJDEP_TYPE_LFLOW_OD type is used to store this lflow object to + * logical router resource linking (logical router index is stored + * in the uuid.parts[0]). + */ + char uuid_s[UUID_LEN + 1]; + sprintf(uuid_s, UUID_FMT, UUID_ARGS(&lflow->lflow_uuid)); + + struct resource_to_objects_node *lflow_od_res = + objdep_mgr_find_objs(lflowdep_mgr, OBJDEP_TYPE_LFLOW_OD, + uuid_s); + if (lflow_od_res) { + struct object_to_resources_list_node *r_node; + RESOURCE_FOR_EACH_OBJ (r_node, lflow_od_res) { + size_t index = r_node->obj_uuid.parts[0]; + bitmap_set0(lflow->dpg_bitmap, index); + } + } + } } - sbrec_logical_flow_set_external_ids(sbflow, &ids); - smap_destroy(&ids); +} + +/* Gets all the lflows from the lflow dependency manager for the + * resource 'res_name' of objdep type 'objdep_type' and clears the + * datapath id (or unlinks the datapath from these lflows). */ +static void +unlink_lflows(struct ovn_datapath *od, enum objdep_type objdep_type, + const char *res_name, struct lflow_data *lflow_data, + struct objdep_mgr *lflowdep_mgr) +{ + struct resource_to_objects_node *res_node = objdep_mgr_find_objs( + lflowdep_mgr, objdep_type, res_name); + + unlink_objres_lflows(res_node, od, lflow_data, lflowdep_mgr); } static bool -delete_lflow_for_lsp(struct ovn_port *op, bool is_update, - const struct sbrec_logical_flow_table *sb_lflow_table, - struct hmap *lflows) +sync_lflows_from_objres(struct ovsdb_idl_txn *ovnsb_txn, + struct resource_to_objects_node *res_node, + struct lflow_input *lflow_input, + struct lflow_data *lflow_data, + struct objdep_mgr *lflowdep_mgr) { - struct lflow_ref_node *lfrn; - const char *operation = is_update ? "updated" : "deleted"; - LIST_FOR_EACH_SAFE (lfrn, lflow_list_node, &op->lflows) { - VLOG_DBG("Deleting SB lflow "UUID_FMT" for %s port %s", - UUID_ARGS(&lfrn->lflow->sb_uuid), operation, op->key); + if (!res_node) { + return true; + } + + struct uuidset lflow_uuidset = UUIDSET_INITIALIZER(&lflow_uuidset); + struct object_to_resources_list_node *resource_list_node; + RESOURCE_FOR_EACH_OBJ (resource_list_node, res_node) { + const struct uuid *obj_uuid = &resource_list_node->obj_uuid; + + struct ovn_lflow *lflow = ovn_lflow_uuid_find( + &lflow_data->lflows_hash_map, obj_uuid); + if (!lflow) { + continue; + } const struct sbrec_logical_flow *sblflow = - sbrec_logical_flow_table_get_for_uuid(sb_lflow_table, - &lfrn->lflow->sb_uuid); - if (sblflow) { - sbrec_logical_flow_delete(sblflow); + sbrec_logical_flow_table_get_for_uuid( + lflow_input->sbrec_logical_flow_table, &lflow->sb_uuid); + + size_t n_datapaths; + if (ovn_stage_to_datapath_type(lflow->stage) == DP_SWITCH) { + n_datapaths = ods_size(lflow_input->ls_datapaths); } else { - static struct vlog_rate_limit rl = - VLOG_RATE_LIMIT_INIT(1, 1); - VLOG_WARN_RL(&rl, "SB lflow "UUID_FMT" not found when handling " - "%s port %s. Recompute.", - UUID_ARGS(&lfrn->lflow->sb_uuid), operation, op->key); - return false; + n_datapaths = ods_size(lflow_input->lr_datapaths); + } + + size_t n_ods = bitmap_count1(lflow->dpg_bitmap, n_datapaths); + + if (n_ods) { + sync_lflow_to_sb(ovnsb_txn, lflow_input, lflow_data, + lflow, sblflow); + } else { + if (sblflow) { + sbrec_logical_flow_delete(sblflow); + ovn_lflow_destroy(lflow_data, lflow); + } else { + VLOG_ERR("SB lflow "UUID_FMT" not found when " + "deleting lflows for resource %s (type %d). " + "This should not happen. Asserting", + UUID_ARGS(&lflow->sb_uuid), + resource_list_node->resource_node->res_name, + resource_list_node->resource_node->type); + ovs_assert(sblflow); + } + uuidset_insert(&lflow_uuidset, obj_uuid); } + } - ovn_lflow_destroy(lflows, lfrn->lflow); + struct uuidset_node *unode; + UUIDSET_FOR_EACH (unode, &lflow_uuidset) { + objdep_mgr_remove_obj(lflowdep_mgr, &unode->uuid); } + uuidset_destroy(&lflow_uuidset); + return true; } +/* Gets all the lflows from the lflow dependency manager for the + * resource 'res_name' of objdep type 'objdep_type' and syncs the + * lflows to the SB DB. */ +static void +sync_lflows_for_res(struct ovsdb_idl_txn *ovnsb_txn, + enum objdep_type objdep_type, const char *res_name, + struct lflow_input *lflow_input, + struct lflow_data *lflow_data, + struct objdep_mgr *lflowdep_mgr) +{ + struct resource_to_objects_node *res_node = objdep_mgr_find_objs( + lflowdep_mgr, objdep_type, res_name); + + sync_lflows_from_objres(ovnsb_txn, res_node, lflow_input, + lflow_data, lflowdep_mgr); +} + +/* Gets all the lflows from the lflow dependency manager for the + * resource 'res_name' of objdep type 'objdep_type' and clears the + * datapath id (or unlinks the datapath from these lflows) and syncs + * the lflows to the SB DB. */ +static void +unlink_and_sync_lflows(struct ovsdb_idl_txn *sb_txn, struct ovn_datapath *od, + enum objdep_type objdep_type, const char *res_name, + struct lflow_input *lflow_input, + struct lflow_data *lflow_data, + struct objdep_mgr *lflowdep_mgr) +{ + struct resource_to_objects_node *res_node = objdep_mgr_find_objs( + lflowdep_mgr, objdep_type, res_name); + + unlink_objres_lflows(res_node, od, lflow_data, lflowdep_mgr); + sync_lflows_from_objres(sb_txn, res_node, lflow_input, + lflow_data, lflowdep_mgr); +} + bool lflow_handle_northd_ls_changes(struct ovsdb_idl_txn *ovnsb_txn, struct tracked_ls_changes *ls_changes, struct lflow_input *lflow_input, - struct hmap *lflows) + struct lflow_data *lflow_data) { struct ls_change *ls_change; @@ -17019,23 +17266,20 @@ bool lflow_handle_northd_ls_changes(struct ovsdb_idl_txn *ovnsb_txn, struct ovn_port *op; LIST_FOR_EACH (op, list, &ls_change->deleted_ports) { - if (!delete_lflow_for_lsp(op, false, - lflow_input->sbrec_logical_flow_table, - lflows)) { - return false; - } + /* unlink old lflows. */ + unlink_and_sync_lflows(ovnsb_txn, op->od, OBJDEP_TYPE_LPORT, + op->nbsp->name, lflow_input, lflow_data, + &op->lflow_dep_mgr); + objdep_mgr_clear(&op->lflow_dep_mgr); /* No need to update SB multicast groups, thanks to weak * references. */ } LIST_FOR_EACH (op, list, &ls_change->updated_ports) { - /* Delete old lflows. */ - if (!delete_lflow_for_lsp(op, true, - lflow_input->sbrec_logical_flow_table, - lflows)) { - return false; - } + /* unlink old lflows. */ + unlink_lflows(op->od, OBJDEP_TYPE_LPORT, op->nbsp->name, + lflow_data, &op->lflow_dep_mgr); /* Generate new lflows. */ struct ds match = DS_EMPTY_INITIALIZER; @@ -17044,7 +17288,7 @@ bool lflow_handle_northd_ls_changes(struct ovsdb_idl_txn *ovnsb_txn, lflow_input->lr_ports, lflow_input->meter_groups, &match, &actions, - lflows); + lflow_data); ds_destroy(&match); ds_destroy(&actions); @@ -17052,11 +17296,9 @@ bool lflow_handle_northd_ls_changes(struct ovsdb_idl_txn *ovnsb_txn, * groups. */ /* Sync the new flows to SB. */ - struct lflow_ref_node *lfrn; - LIST_FOR_EACH (lfrn, lflow_list_node, &op->lflows) { - sync_lsp_lflows_to_sb(ovnsb_txn, lflow_input, lflows, - lfrn->lflow); - } + sync_lflows_for_res(ovnsb_txn, OBJDEP_TYPE_LPORT, + op->nbsp->name, lflow_input, + lflow_data, &op->lflow_dep_mgr); } LIST_FOR_EACH (op, list, &ls_change->added_ports) { @@ -17066,7 +17308,7 @@ bool lflow_handle_northd_ls_changes(struct ovsdb_idl_txn *ovnsb_txn, lflow_input->lr_ports, lflow_input->meter_groups, &match, &actions, - lflows); + lflow_data); ds_destroy(&match); ds_destroy(&actions); @@ -17095,11 +17337,9 @@ bool lflow_handle_northd_ls_changes(struct ovsdb_idl_txn *ovnsb_txn, } /* Sync the newly added flows to SB. */ - struct lflow_ref_node *lfrn; - LIST_FOR_EACH (lfrn, lflow_list_node, &op->lflows) { - sync_lsp_lflows_to_sb(ovnsb_txn, lflow_input, lflows, - lfrn->lflow); - } + sync_lflows_for_res(ovnsb_txn, OBJDEP_TYPE_LPORT, + op->nbsp->name, lflow_input, + lflow_data, &op->lflow_dep_mgr); } } return true; diff --git a/northd/northd.h b/northd/northd.h index 1d344d57d6..4af6c4e9b0 100644 --- a/northd/northd.h +++ b/northd/northd.h @@ -19,6 +19,7 @@ #include "lib/ovn-sb-idl.h" #include "lib/ovn-util.h" #include "lib/ovs-atomic.h" +#include "lib/objdep.h" #include "lib/sset.h" #include "northd/ipam.h" #include "openvswitch/hmap.h" @@ -125,7 +126,10 @@ struct northd_data { }; struct lflow_data { - struct hmap lflows; + struct hmap lflows_match_map; + struct hmap lflows_hash_map; + struct hmap ls_dp_groups; + struct hmap lr_dp_groups; }; void lflow_data_init(struct lflow_data *); @@ -140,6 +144,7 @@ struct lflow_input { const struct sbrec_logical_flow_table *sbrec_logical_flow_table; const struct sbrec_multicast_group_table *sbrec_multicast_group_table; const struct sbrec_igmp_group_table *sbrec_igmp_group_table; + const struct sbrec_logical_dp_group_table *sbrec_logical_dp_group_table; /* Indexes */ struct ovsdb_idl_index *sbrec_mcast_group_by_name_dp; @@ -344,10 +349,10 @@ void northd_indices_create(struct northd_data *data, struct ovsdb_idl *ovnsb_idl); void build_lflows(struct ovsdb_idl_txn *ovnsb_txn, struct lflow_input *input_data, - struct hmap *lflows); + struct lflow_data *lflow_data); bool lflow_handle_northd_ls_changes(struct ovsdb_idl_txn *ovnsb_txn, struct tracked_ls_changes *, - struct lflow_input *, struct hmap *lflows); + struct lflow_input *, struct lflow_data *); bool northd_handle_sb_port_binding_changes( const struct sbrec_port_binding_table *, struct hmap *ls_ports); diff --git a/northd/ovn-northd.c b/northd/ovn-northd.c index 4fa1b039ea..edca0552c0 100644 --- a/northd/ovn-northd.c +++ b/northd/ovn-northd.c @@ -836,6 +836,10 @@ main(int argc, char *argv[]) ovsdb_idl_omit_alert(ovnsb_idl_loop.idl, &sbrec_multicast_group_columns[i]); } + for (size_t i = 0; i < SBREC_LOGICAL_DP_GROUP_N_COLUMNS; i++) { + ovsdb_idl_omit_alert(ovnsb_idl_loop.idl, + &sbrec_logical_dp_group_columns[i]); + } unixctl_command_register("sb-connection-status", "", 0, 0, ovn_conn_show, ovnsb_idl_loop.idl); From patchwork Fri Aug 18 08:58:15 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Numan Siddique X-Patchwork-Id: 1822761 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::133; helo=smtp2.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp2.osuosl.org (smtp2.osuosl.org [IPv6:2605:bc80:3010::133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4RRwm86KL2z1ygH for ; Fri, 18 Aug 2023 18:59:20 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 29622419B9; Fri, 18 Aug 2023 08:59:18 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 29622419B9 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id xk8SjmwP9is9; Fri, 18 Aug 2023 08:59:16 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp2.osuosl.org (Postfix) with ESMTPS id 2CAE641986; Fri, 18 Aug 2023 08:59:15 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 2CAE641986 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id DA600C0039; Fri, 18 Aug 2023 08:59:14 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp4.osuosl.org (smtp4.osuosl.org [IPv6:2605:bc80:3010::137]) by lists.linuxfoundation.org (Postfix) with ESMTP id 086DDC0039 for ; Fri, 18 Aug 2023 08:59:13 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id E9622426EE for ; Fri, 18 Aug 2023 08:58:24 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org E9622426EE X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 2DSSjKcOT00J for ; Fri, 18 Aug 2023 08:58:23 +0000 (UTC) Received: from relay7-d.mail.gandi.net (relay7-d.mail.gandi.net [217.70.183.200]) by smtp4.osuosl.org (Postfix) with ESMTPS id 62239426F2 for ; Fri, 18 Aug 2023 08:58:23 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 62239426F2 Received: by mail.gandi.net (Postfix) with ESMTPSA id C35AD20007; Fri, 18 Aug 2023 08:58:19 +0000 (UTC) From: numans@ovn.org To: dev@openvswitch.org Date: Fri, 18 Aug 2023 14:28:15 +0530 Message-Id: <20230818085815.1031063-1-numans@ovn.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230818085606.1030792-1-numans@ovn.org> References: <20230818085606.1030792-1-numans@ovn.org> MIME-Version: 1.0 X-GND-Sasl: numans@ovn.org Subject: [ovs-dev] [PATCH ovn v6 10/16] northd: Fix LSP incremental processing if dhcp options are set. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Numan Siddique The flows to allow DHCP response from ovn-controller were missing if a logical VIF port had dhcp v4/v6 options set and were handled incrementally. Fixes: 8bbd67891f68 ("northd: Incremental processing of VIF additions in 'lflow' node.") Signed-off-by: Numan Siddique --- northd/northd.c | 125 ++++++++++++++++++++++---------------------- tests/ovn-northd.at | 25 +++++++++ 2 files changed, 87 insertions(+), 63 deletions(-) diff --git a/northd/northd.c b/northd/northd.c index 9b9c6de826..6de3c11ac7 100644 --- a/northd/northd.c +++ b/northd/northd.c @@ -8328,68 +8328,6 @@ build_acls(struct ovn_datapath *od, const struct chassis_features *features, } } - /* Add 34000 priority flow to allow DHCP reply from ovn-controller to all - * logical ports of the datapath if the CMS has configured DHCPv4 options. - * */ - for (size_t i = 0; i < od->nbs->n_ports; i++) { - if (lsp_is_external(od->nbs->ports[i])) { - continue; - } - - if (od->nbs->ports[i]->dhcpv4_options) { - const char *server_id = smap_get( - &od->nbs->ports[i]->dhcpv4_options->options, "server_id"); - const char *server_mac = smap_get( - &od->nbs->ports[i]->dhcpv4_options->options, "server_mac"); - const char *lease_time = smap_get( - &od->nbs->ports[i]->dhcpv4_options->options, "lease_time"); - if (server_id && server_mac && lease_time) { - const char *dhcp_actions = - has_stateful ? REGBIT_ACL_VERDICT_ALLOW" = 1; " - "ct_commit; next;" - : REGBIT_ACL_VERDICT_ALLOW" = 1; next;"; - ds_clear(&match); - ds_put_format(&match, "outport == \"%s\" && eth.src == %s " - "&& ip4.src == %s && udp && udp.src == 67 " - "&& udp.dst == 68", od->nbs->ports[i]->name, - server_mac, server_id); - ovn_lflow_add_with_lport_and_hint( - lflows, od, S_SWITCH_OUT_ACL_EVAL, 34000, ds_cstr(&match), - dhcp_actions, od->nbs->ports[i]->name, - &od->nbs->ports[i]->dhcpv4_options->header_); - } - } - - if (od->nbs->ports[i]->dhcpv6_options) { - const char *server_mac = smap_get( - &od->nbs->ports[i]->dhcpv6_options->options, "server_id"); - struct eth_addr ea; - if (server_mac && eth_addr_from_string(server_mac, &ea)) { - /* Get the link local IP of the DHCPv6 server from the - * server MAC. */ - struct in6_addr lla; - in6_generate_lla(ea, &lla); - - char server_ip[INET6_ADDRSTRLEN + 1]; - ipv6_string_mapped(server_ip, &lla); - - const char *dhcp6_actions = - has_stateful ? REGBIT_ACL_VERDICT_ALLOW" = 1; " - "ct_commit; next;" - : REGBIT_ACL_VERDICT_ALLOW" = 1; next;"; - ds_clear(&match); - ds_put_format(&match, "outport == \"%s\" && eth.src == %s " - "&& ip6.src == %s && udp && udp.src == 547 " - "&& udp.dst == 546", od->nbs->ports[i]->name, - server_mac, server_ip); - ovn_lflow_add_with_lport_and_hint( - lflows, od, S_SWITCH_OUT_ACL_EVAL, 34000, ds_cstr(&match), - dhcp6_actions, od->nbs->ports[i]->name, - &od->nbs->ports[i]->dhcpv6_options->header_); - } - } - } - /* Add a 34000 priority flow to advance the DNS reply from ovn-controller, * if the CMS has configured DNS records for the datapath. */ @@ -9701,6 +9639,34 @@ build_dhcpv4_options_flows(struct ovn_port *op, ds_destroy(&options_action); ds_destroy(&response_action); ds_destroy(&ipv4_addr_match); + + /* Add 34000 priority flow to allow DHCP reply from ovn-controller + * to the ogical port of the datapath if the CMS has configured + * DHCPv4 options. + * */ + if (!is_external) { + const char *server_id = smap_get( + &op->nbsp->dhcpv4_options->options, "server_id"); + const char *server_mac = smap_get( + &op->nbsp->dhcpv4_options->options, "server_mac"); + const char *lease_time = smap_get( + &op->nbsp->dhcpv4_options->options, "lease_time"); + ovs_assert(server_id && server_mac && lease_time); + const char *dhcp_actions = + (op->od->has_stateful_acl || op->od->has_lb_vip) + ? REGBIT_ACL_VERDICT_ALLOW" = 1; ct_commit; next;" + : REGBIT_ACL_VERDICT_ALLOW" = 1; next;"; + ds_clear(&match); + ds_put_format(&match, "outport == %s && eth.src == %s " + "&& ip4.src == %s && udp && udp.src == 67 " + "&& udp.dst == 68",op->json_key, + server_mac, server_id); + ovn_lflow_add_with_lport_lflow_ref( + lflows, op->od, S_SWITCH_OUT_ACL_EVAL, 34000, + ds_cstr(&match),dhcp_actions, op->key, NULL, + &op->nbsp->dhcpv4_options->header_, + lflow_dep_mgr, op->key, op->od); + } break; } } @@ -9756,6 +9722,40 @@ build_dhcpv6_options_flows(struct ovn_port *op, lflow_dep_mgr, op->key, op->od); ds_destroy(&options_action); ds_destroy(&response_action); + + /* Add 34000 priority flow to allow DHCP reply from ovn-controller + * to the ogical port of the datapath if the CMS has configured + * DHCPv6 options. + * */ + if (!is_external) { + const char *server_mac = smap_get( + &op->nbsp->dhcpv6_options->options, "server_id"); + struct eth_addr ea; + ovs_assert(server_mac && + eth_addr_from_string(server_mac, &ea)); + /* Get the link local IP of the DHCPv6 server from the + * server MAC. */ + struct in6_addr lla; + in6_generate_lla(ea, &lla); + + char server_ip[INET6_ADDRSTRLEN + 1]; + ipv6_string_mapped(server_ip, &lla); + + const char *dhcp6_actions = + (op->od->has_stateful_acl || op->od->has_lb_vip) + ? REGBIT_ACL_VERDICT_ALLOW" = 1; ct_commit; next;" + : REGBIT_ACL_VERDICT_ALLOW" = 1; next;"; + ds_clear(&match); + ds_put_format(&match, "outport == %s && eth.src == %s " + "&& ip6.src == %s && udp && udp.src == 547 " + "&& udp.dst == 546", op->json_key, + server_mac, server_ip); + ovn_lflow_add_with_lport_lflow_ref( + lflows, op->od, S_SWITCH_OUT_ACL_EVAL, 34000, + ds_cstr(&match), dhcp6_actions, op->key, NULL, + &op->nbsp->dhcpv6_options->header_, + lflow_dep_mgr, op->key, op->od); + } break; } } @@ -16334,7 +16334,6 @@ build_lswitch_and_lrouter_iterate_by_lsp(struct ovn_port *op, build_lswitch_external_port(op, lflows, &op->lflow_dep_mgr); build_lswitch_ip_unicast_lookup(op, lflows, actions, match, &op->lflow_dep_mgr); - /* Build Logical Router Flows. */ build_ip_routing_flows_for_router_type_lsp(op, lr_ports, lflows); build_arp_resolve_flows_for_lsp(op, lflows, lr_ports, match, actions, diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at index 234d5a8bbd..a5f64ed5bd 100644 --- a/tests/ovn-northd.at +++ b/tests/ovn-northd.at @@ -9624,6 +9624,31 @@ check_recompute_counter 0 0 CHECK_NO_CHANGE_AFTER_RECOMPUTE +# Associate DHCP for lsp0-2 +ovn-nbctl dhcp-options-create 192.168.0.0/24 + +CIDR_UUID=$(ovn-nbctl --bare --columns=_uuid find dhcp_options cidr="192.168.0.0/24") +ovn-nbctl dhcp-options-set-options $CIDR_UUID lease_time=3600 router=192.168.0.1 server_id=192.168.0.1 server_mac=c0:ff:ee:00:00:01 hostname="\"foo\"" + +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +ovn-nbctl --wait=sb lsp-set-dhcpv4-options lsp0-2 $CIDR_UUID +check_recompute_counter 0 0 + +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +# Add IPv6 address and associate DHCPv6 for lsp0-2 +check ovn-nbctl lsp-set-addresses lsp0-2 "aa:aa:aa:00:00:01 192.168.0.11 aef0::4" +d1="$(ovn-nbctl create DHCP_Options cidr="aef0\:\:/64" \ +options="\"server_id\"=\"00:00:00:10:00:01\"")" + +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +ovn-nbctl --wait=sb lsp-set-dhcpv6-options lsp0-2 ${d1} +check_recompute_counter 0 0 + +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +check ovn-nbctl --wait=hv ls-del ls0 + OVN_CLEANUP([hv1]) AT_CLEANUP ]) From patchwork Fri Aug 18 08:58:21 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Numan Siddique X-Patchwork-Id: 1822762 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.138; helo=smtp1.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4RRwmJ4ctxz1ygH for ; Fri, 18 Aug 2023 18:59:28 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 0BA7E840BB; Fri, 18 Aug 2023 08:59:26 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 0BA7E840BB X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id UWfNkaSz3Z35; Fri, 18 Aug 2023 08:59:20 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp1.osuosl.org (Postfix) with ESMTPS id 0AFE184082; Fri, 18 Aug 2023 08:59:19 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 0AFE184082 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id AB165C0039; Fri, 18 Aug 2023 08:59:18 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp2.osuosl.org (smtp2.osuosl.org [IPv6:2605:bc80:3010::133]) by lists.linuxfoundation.org (Postfix) with ESMTP id 305BDC0DD6 for ; Fri, 18 Aug 2023 08:59:17 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id AFB4840543 for ; Fri, 18 Aug 2023 08:58:32 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org AFB4840543 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id hIxqoWhKXELN for ; Fri, 18 Aug 2023 08:58:29 +0000 (UTC) Received: from relay7-d.mail.gandi.net (relay7-d.mail.gandi.net [217.70.183.200]) by smtp2.osuosl.org (Postfix) with ESMTPS id BF9B041979 for ; Fri, 18 Aug 2023 08:58:28 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org BF9B041979 Received: by mail.gandi.net (Postfix) with ESMTPSA id CCF4320007; Fri, 18 Aug 2023 08:58:24 +0000 (UTC) From: numans@ovn.org To: dev@openvswitch.org Date: Fri, 18 Aug 2023 14:28:21 +0530 Message-Id: <20230818085821.1031079-1-numans@ovn.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230818085606.1030792-1-numans@ovn.org> References: <20230818085606.1030792-1-numans@ovn.org> MIME-Version: 1.0 X-GND-Sasl: numans@ovn.org Subject: [ovs-dev] [PATCH ovn v6 11/16] northd: Use objdep mgr for datapath/lb to lflow references. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Numan Siddique objdep_mgr is maintained in each 'ovn_datapath' and 'lb_datapaths' to store the references to the logical flows generated for each of these resources. The lflows related to load balancers for a datapath are stored in a separate objdep_type - OBJDEP_TYPE_LB and lflows which are common to both load balancers and conntrack related are stored in a separate objdep_type - OBJDEP_TYPE_CT_LB. This separation helps in incrementally processing the load balancer changes. Upcoming commit will make use of this to handle logical switch/router and load balancer changes incrementally in the lflow engine node. Signed-off-by: Numan Siddique --- lib/lb.c | 2 + lib/lb.h | 3 + lib/objdep.h | 4 + northd/northd.c | 1852 +++++++++++++++++++++++++++++++---------------- northd/northd.h | 2 + 5 files changed, 1219 insertions(+), 644 deletions(-) diff --git a/lib/lb.c b/lib/lb.c index d0d562b6fb..4a82222cfb 100644 --- a/lib/lb.c +++ b/lib/lb.c @@ -1082,6 +1082,7 @@ ovn_lb_datapaths_create(const struct ovn_northd_lb *lb, size_t n_ls_datapaths, lb_dps->nb_ls_map = bitmap_allocate(n_ls_datapaths); lb_dps->nb_lr_map = bitmap_allocate(n_lr_datapaths); + objdep_mgr_init(&lb_dps->lflow_dep_mgr); return lb_dps; } @@ -1104,6 +1105,7 @@ ovn_lb_datapaths_destroy(struct ovn_lb_datapaths *lb_dps) { bitmap_free(lb_dps->nb_lr_map); bitmap_free(lb_dps->nb_ls_map); + objdep_mgr_destroy(&lb_dps->lflow_dep_mgr); free(lb_dps); } diff --git a/lib/lb.h b/lib/lb.h index b8e3c1e8fb..e647cb02ca 100644 --- a/lib/lb.h +++ b/lib/lb.h @@ -22,6 +22,7 @@ #include "lib/smap.h" #include "openvswitch/hmap.h" #include "ovn-util.h" +#include "objdep.h" #include "sset.h" #include "uuid.h" @@ -176,6 +177,8 @@ struct ovn_lb_datapaths { size_t n_nb_lr; unsigned long *nb_lr_map; + + struct objdep_mgr lflow_dep_mgr; }; struct ovn_lb_datapaths *ovn_lb_datapaths_create(const struct ovn_northd_lb *, diff --git a/lib/objdep.h b/lib/objdep.h index d599128ea3..37f9bd82ea 100644 --- a/lib/objdep.h +++ b/lib/objdep.h @@ -28,6 +28,10 @@ enum objdep_type { OBJDEP_TYPE_MC_GROUP, OBJDEP_TYPE_TEMPLATE, OBJDEP_TYPE_LPORT, + OBJDEP_TYPE_OD, + OBJDEP_TYPE_LB, + OBJDEP_TYPE_CT_LB, + OBJDEP_TYPE_IGMP, OBJDEP_TYPE_LFLOW_OD, OBJDEP_TYPE_MAX, }; diff --git a/northd/northd.c b/northd/northd.c index 6de3c11ac7..8c0b732653 100644 --- a/northd/northd.c +++ b/northd/northd.c @@ -819,6 +819,7 @@ ovn_datapath_create(struct hmap *datapaths, const struct uuid *key, hmap_insert(datapaths, &od->key_node, uuid_hash(&od->key)); od->lr_group = NULL; hmap_init(&od->ports); + objdep_mgr_init(&od->lflow_dep_mgr); return od; } @@ -853,6 +854,7 @@ ovn_datapath_destroy(struct hmap *datapaths, struct ovn_datapath *od) ovn_ls_port_group_destroy(&od->nb_pgs); destroy_mcast_info_for_datapath(od); destroy_ports_for_datapath(od); + objdep_mgr_destroy(&od->lflow_dep_mgr); free(od); } } @@ -6047,6 +6049,8 @@ struct ovn_igmp_group { struct multicast_group mcgroup; struct ovs_list entries; /* List of SB entries for this group. */ + + struct objdep_mgr lflow_dep_mgr; }; static uint32_t @@ -6103,7 +6107,7 @@ ovn_igmp_group_add(struct ovsdb_idl_index *sbrec_mcast_group_by_name_dp, } igmp_group->mcgroup.name = address_s; ovs_list_init(&igmp_group->entries); - + objdep_mgr_init(&igmp_group->lflow_dep_mgr); hmap_insert(igmp_groups, &igmp_group->hmap_node, ovn_igmp_group_hash(datapath, address)); ovs_list_push_back(&datapath->mcast_info.groups, @@ -6236,6 +6240,7 @@ ovn_igmp_group_destroy(struct hmap *igmp_groups, } hmap_remove(igmp_groups, &igmp_group->hmap_node); ovs_list_remove(&igmp_group->list_node); + objdep_mgr_destroy(&igmp_group->lflow_dep_mgr); free(igmp_group); } } @@ -6445,7 +6450,8 @@ ovn_dp_group_add_with_reference(struct ovn_lflow *lflow_ref, /* Adds a row with the specified contents to the Logical_Flow table. * Version to use when hash bucket locking is NOT required. */ static struct ovn_lflow * -do_ovn_lflow_add(struct lflow_data *lflow_data, const struct ovn_datapath *od, +do_ovn_lflow_add_with_od_lflow_ref(struct lflow_data *lflow_data, + const struct ovn_datapath *od, const unsigned long *dp_bitmap, size_t dp_bitmap_len, uint32_t hash, enum ovn_stage stage, uint16_t priority, const char *match, const char *actions, const char *io_port, @@ -6495,45 +6501,17 @@ do_ovn_lflow_add(struct lflow_data *lflow_data, const struct ovn_datapath *od, /* Adds a row with the specified contents to the Logical_Flow table. */ static void -ovn_lflow_add_at(struct lflow_data *lflow_data, const struct ovn_datapath *od, +ovn_lflow_add_at(struct lflow_data *lflow_data, + const struct ovn_datapath *od, const unsigned long *dp_bitmap, size_t dp_bitmap_len, enum ovn_stage stage, uint16_t priority, - const char *match, const char *actions, const char *io_port, - const char *ctrl_meter, - const struct ovsdb_idl_row *stage_hint, const char *where) - OVS_EXCLUDED(fake_hash_mutex) -{ - struct ovs_mutex *hash_lock; - uint32_t hash; - - ovs_assert(!od || - ovn_stage_to_datapath_type(stage) == ovn_datapath_get_type(od)); - - hash = ovn_logical_flow_hash(ovn_stage_get_table(stage), - ovn_stage_get_pipeline(stage), - priority, match, - actions); - - hash_lock = lflow_hash_lock(&lflow_data->lflows_match_map, hash); - do_ovn_lflow_add(lflow_data, od, dp_bitmap, dp_bitmap_len, hash, stage, - priority, match, actions, io_port, stage_hint, where, - ctrl_meter); - lflow_hash_unlock(hash_lock); -} - -/* Adds a row with the specified contents to the Logical_Flow table. */ -static void -ovn_lflow_add_objdep_ref(struct lflow_data *lflow_data, - const struct ovn_datapath *od, - const unsigned long *dp_bitmap, size_t dp_bitmap_len, - enum ovn_stage stage, uint16_t priority, - const char *match, const char *actions, - const char *io_port, const char *ctrl_meter, - const struct ovsdb_idl_row *stage_hint, - const char *where, struct objdep_mgr *lflow_dep_mgr, - enum objdep_type objdep_type, - const char *res_name, - const struct ovn_datapath *res_od) + const char *match, const char *actions, + const char *io_port, const char *ctrl_meter, + const struct ovsdb_idl_row *stage_hint, + const char *where, struct objdep_mgr *lflow_dep_mgr, + enum objdep_type objdep_type, + const char *res_name, + const struct ovn_datapath *res_od) OVS_EXCLUDED(fake_hash_mutex) { struct ovs_mutex *hash_lock; @@ -6551,9 +6529,11 @@ ovn_lflow_add_objdep_ref(struct lflow_data *lflow_data, hash_lock = lflow_hash_lock(&lflow_data->lflows_match_map, hash); struct ovn_lflow *lflow = - do_ovn_lflow_add(lflow_data, od, dp_bitmap, dp_bitmap_len, hash, stage, - priority, match, actions, io_port, stage_hint, where, - ctrl_meter); + do_ovn_lflow_add_with_od_lflow_ref(lflow_data, od, dp_bitmap, + dp_bitmap_len, hash, stage, + priority, match, actions, + io_port, stage_hint, where, + ctrl_meter); objdep_mgr_add(lflow_dep_mgr, objdep_type, res_name, &lflow->lflow_uuid); @@ -6572,33 +6552,52 @@ static void __ovn_lflow_add_default_drop(struct lflow_data *lflow_data, struct ovn_datapath *od, enum ovn_stage stage, - const char *where) + const char *where, + struct objdep_mgr *lflow_dep_mgr, + enum objdep_type objdep_type, + const char *res_name, + const struct ovn_datapath *res_od) { ovn_lflow_add_at(lflow_data, od, NULL, 0, stage, 0, "1", - debug_drop_action(), NULL, NULL, NULL, where ); + debug_drop_action(), NULL, NULL, NULL, where, + lflow_dep_mgr, objdep_type, res_name, res_od); } /* Adds a row with the specified contents to the Logical_Flow table. */ #define ovn_lflow_add_with_hint__(LFLOW_DATA, OD, STAGE, PRIORITY, MATCH, \ ACTIONS, IN_OUT_PORT, CTRL_METER, \ - STAGE_HINT) \ + STAGE_HINT, LFLOW_DEP_MGR, OBJDEP_TYPE, \ + RES_NAME, RES_OD) \ ovn_lflow_add_at(LFLOW_DATA, OD, NULL, 0, STAGE, PRIORITY, MATCH, ACTIONS,\ - IN_OUT_PORT, CTRL_METER, STAGE_HINT, OVS_SOURCE_LOCATOR) + IN_OUT_PORT, CTRL_METER, STAGE_HINT, OVS_SOURCE_LOCATOR, \ + LFLOW_DEP_MGR, OBJDEP_TYPE, RES_NAME, RES_OD) #define ovn_lflow_add_with_hint(LFLOW_DATA, OD, STAGE, PRIORITY, MATCH, \ - ACTIONS, STAGE_HINT) \ + ACTIONS, STAGE_HINT, LFLOW_DEP_MGR, \ + OBJDEP_TYPE, RES_NAME, RES_OD) \ ovn_lflow_add_at(LFLOW_DATA, OD, NULL, 0, STAGE, PRIORITY, MATCH, ACTIONS,\ - NULL, NULL, STAGE_HINT, OVS_SOURCE_LOCATOR) + NULL, NULL, STAGE_HINT, OVS_SOURCE_LOCATOR, \ + LFLOW_DEP_MGR, OBJDEP_TYPE, RES_NAME, RES_OD) + +#define ovn_lflow_add(LFLOW_DATA, OD, STAGE, PRIORITY, MATCH, \ + ACTIONS, LFLOW_DEP_MGR, OBJDEP_TYPE, RES_NAME, RES_OD) \ + ovn_lflow_add_at(LFLOW_DATA, OD, NULL, 0, STAGE, PRIORITY, MATCH, ACTIONS,\ + NULL, NULL, NULL, OVS_SOURCE_LOCATOR, \ + LFLOW_DEP_MGR, OBJDEP_TYPE, RES_NAME, RES_OD) #define ovn_lflow_add_with_dp_group(LFLOW_DATA, DP_BITMAP, DP_BITMAP_LEN, \ STAGE, PRIORITY, MATCH, ACTIONS, \ - STAGE_HINT) \ + STAGE_HINT, LFLOW_DEP_MGR, OBJDEP_TYPE, \ + RES_NAME, RES_OD) \ ovn_lflow_add_at(LFLOW_DATA, NULL, DP_BITMAP, DP_BITMAP_LEN, STAGE, \ PRIORITY, MATCH, ACTIONS, NULL, NULL, STAGE_HINT, \ - OVS_SOURCE_LOCATOR) + OVS_SOURCE_LOCATOR, LFLOW_DEP_MGR, OBJDEP_TYPE, \ + RES_NAME, RES_OD) -#define ovn_lflow_add_default_drop(LFLOW_DATA, OD, STAGE) \ - __ovn_lflow_add_default_drop(LFLOW_DATA, OD, STAGE, OVS_SOURCE_LOCATOR) +#define ovn_lflow_add_default_drop(LFLOW_DATA, OD, STAGE, LFLOW_DEP_MGR, \ + OBJDEP_TYPE, RES_NAME, RES_OD) \ + __ovn_lflow_add_default_drop(LFLOW_DATA, OD, STAGE, OVS_SOURCE_LOCATOR, \ + LFLOW_DEP_MGR, OBJDEP_TYPE, RES_NAME, RES_OD) /* This macro is similar to ovn_lflow_add_with_hint, except that it requires @@ -6613,28 +6612,50 @@ __ovn_lflow_add_default_drop(struct lflow_data *lflow_data, * For now, only LS pipelines should use this macro. */ #define ovn_lflow_add_with_lport_and_hint(LFLOW_DATA, OD, STAGE, PRIORITY, \ MATCH, ACTIONS, IN_OUT_PORT, \ - STAGE_HINT) \ + STAGE_HINT, LFLOW_DEP_MGR, \ + OBJDEP_TYPE, RES_NAME, RES_OD) \ + ovn_lflow_add_at(LFLOW_DATA, OD, NULL, 0, STAGE, PRIORITY, MATCH, ACTIONS,\ + IN_OUT_PORT, NULL, STAGE_HINT, OVS_SOURCE_LOCATOR, \ + LFLOW_DEP_MGR, OBJDEP_TYPE, RES_NAME, RES_OD) + +#define ovn_lflow_add_with_od_lflow_ref(LFLOW_DATA, OD, STAGE, PRIORITY, \ + MATCH, ACTIONS, RES_NAME) \ ovn_lflow_add_at(LFLOW_DATA, OD, NULL, 0, STAGE, PRIORITY, MATCH, ACTIONS,\ - IN_OUT_PORT, NULL, STAGE_HINT, OVS_SOURCE_LOCATOR) + NULL, NULL, NULL, OVS_SOURCE_LOCATOR, \ + &(OD)->lflow_dep_mgr, OBJDEP_TYPE_OD, RES_NAME, OD) -#define ovn_lflow_add(LFLOW_DATA, OD, STAGE, PRIORITY, MATCH, ACTIONS) \ +/* Adds an lflow and stores it in the lfow dep mgr for the lflows which + * are related to LB. */ +#define ovn_lflow_add_with_lb_lflow_ref(LFLOW_DATA, OD, STAGE, PRIORITY, \ + MATCH, ACTIONS, RES_NAME) \ ovn_lflow_add_at(LFLOW_DATA, OD, NULL, 0, STAGE, PRIORITY, MATCH, ACTIONS,\ - NULL, NULL, NULL, OVS_SOURCE_LOCATOR) + NULL, NULL, NULL, OVS_SOURCE_LOCATOR, \ + &(OD)->lflow_dep_mgr, OBJDEP_TYPE_LB, RES_NAME, OD) + +/* Adds an lflow and stores it in the lfow dep mgr for the lflows which + * are related to both conntrack (ACLs) and LB. */ +#define ovn_lflow_add_with_ct_lb_lflow_ref(LFLOW_DATA, OD, STAGE, PRIORITY, \ + MATCH, ACTIONS, RES_NAME) \ + ovn_lflow_add_at(LFLOW_DATA, OD, NULL, 0, STAGE, PRIORITY, MATCH, ACTIONS,\ + NULL, NULL, NULL, OVS_SOURCE_LOCATOR, \ + &(OD)->lflow_dep_mgr, OBJDEP_TYPE_CT_LB, RES_NAME, OD) #define ovn_lflow_metered(LFLOW_DATA, OD, STAGE, PRIORITY, MATCH, ACTIONS, \ - CTRL_METER) \ + CTRL_METER, LFLOW_DEP_MGR, OBJDEP_TYPE, RES_NAME, \ + RES_OD) \ ovn_lflow_add_with_hint__(LFLOW_DATA, OD, STAGE, PRIORITY, MATCH, \ - ACTIONS, NULL, CTRL_METER, NULL) + ACTIONS, NULL, CTRL_METER, NULL, LFLOW_DEP_MGR, \ + OBJDEP_TYPE, RES_NAME, RES_OD) #define ovn_lflow_add_with_lport_lflow_ref(LFLOW_DATA, OD, STAGE, PRIORITY, \ MATCH, ACTIONS, IN_OUT_PORT, \ CTRL_METER, STAGE_HINT, \ LFLOW_DEP_MGR, LPORT_NAME, \ LPORT_OD) \ - ovn_lflow_add_objdep_ref(LFLOW_DATA, OD, NULL, 0, STAGE, PRIORITY, MATCH, \ - ACTIONS, IN_OUT_PORT, CTRL_METER, STAGE_HINT, \ - OVS_SOURCE_LOCATOR, LFLOW_DEP_MGR, \ - OBJDEP_TYPE_LPORT, LPORT_NAME, LPORT_OD) + ovn_lflow_add_at(LFLOW_DATA, OD, NULL, 0, STAGE, PRIORITY, MATCH, \ + ACTIONS, IN_OUT_PORT, CTRL_METER, STAGE_HINT, \ + OVS_SOURCE_LOCATOR, LFLOW_DEP_MGR, \ + OBJDEP_TYPE_LPORT, LPORT_NAME, LPORT_OD) static struct ovn_lflow * ovn_lflow_find(const struct hmap *lflows, @@ -7108,10 +7129,13 @@ build_lswitch_learn_fdb_od( struct ovn_datapath *od, struct lflow_data *lflows) { ovs_assert(od->nbs); - ovn_lflow_add(lflows, od, S_SWITCH_IN_LOOKUP_FDB, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_SWITCH_IN_PUT_FDB, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_SWITCH_IN_L2_LKUP, 0, "1", - "outport = get_fdb(eth.dst); next;"); + ovn_lflow_add_with_od_lflow_ref( + lflows, od, S_SWITCH_IN_LOOKUP_FDB, 0, "1", "next;", od->nbs->name); + ovn_lflow_add_with_od_lflow_ref( + lflows, od, S_SWITCH_IN_PUT_FDB, 0, "1", "next;", od->nbs->name); + ovn_lflow_add_with_od_lflow_ref( + lflows, od, S_SWITCH_IN_L2_LKUP, 0, "1", + "outport = get_fdb(eth.dst); next;", od->nbs->name); } /* Egress tables 8: Egress port security - IP (priority 0) @@ -7122,21 +7146,26 @@ build_lswitch_output_port_sec_od(struct ovn_datapath *od, struct lflow_data *lflows) { ovs_assert(od->nbs); - ovn_lflow_add(lflows, od, S_SWITCH_OUT_CHECK_PORT_SEC, 100, - "eth.mcast", REGBIT_PORT_SEC_DROP" = 0; next;"); - ovn_lflow_add(lflows, od, S_SWITCH_OUT_CHECK_PORT_SEC, 0, "1", - REGBIT_PORT_SEC_DROP" = check_out_port_sec(); next;"); + ovn_lflow_add_with_od_lflow_ref( + lflows, od, S_SWITCH_OUT_CHECK_PORT_SEC, 100, + "eth.mcast", REGBIT_PORT_SEC_DROP" = 0; next;", od->nbs->name); + ovn_lflow_add_with_od_lflow_ref( + lflows, od, S_SWITCH_OUT_CHECK_PORT_SEC, 0, "1", + REGBIT_PORT_SEC_DROP" = check_out_port_sec(); next;", od->nbs->name); - ovn_lflow_add(lflows, od, S_SWITCH_OUT_APPLY_PORT_SEC, 50, - REGBIT_PORT_SEC_DROP" == 1", debug_drop_action()); - ovn_lflow_add(lflows, od, S_SWITCH_OUT_APPLY_PORT_SEC, 0, - "1", "output;"); + ovn_lflow_add_with_od_lflow_ref( + lflows, od, S_SWITCH_OUT_APPLY_PORT_SEC, 50, + REGBIT_PORT_SEC_DROP" == 1", debug_drop_action(), od->nbs->name); + ovn_lflow_add_with_od_lflow_ref( + lflows, od, S_SWITCH_OUT_APPLY_PORT_SEC, 0, "1", "output;", + od->nbs->name); } static void skip_port_from_conntrack(struct ovn_datapath *od, struct ovn_port *op, enum ovn_stage in_stage, enum ovn_stage out_stage, - uint16_t priority, struct lflow_data *lflows) + uint16_t priority, struct lflow_data *lflows, + enum objdep_type objdep_type) { /* Can't use ct() for router ports. Consider the following configuration: * lp1(10.0.0.2) on hostA--ls1--lr0--ls2--lp2(10.0.1.2) on hostB, For a @@ -7158,10 +7187,14 @@ skip_port_from_conntrack(struct ovn_datapath *od, struct ovn_port *op, ovn_lflow_add_with_lport_and_hint(lflows, od, in_stage, priority, ingress_match, ingress_action, - op->key, &op->nbsp->header_); + op->key, &op->nbsp->header_, + &od->lflow_dep_mgr, objdep_type, + od->nbs->name, od); ovn_lflow_add_with_lport_and_hint(lflows, od, out_stage, priority, egress_match, egress_action, - op->key, &op->nbsp->header_); + op->key, &op->nbsp->header_, + &od->lflow_dep_mgr, objdep_type, + od->nbs->name, od); free(ingress_match); free(egress_match); @@ -7178,13 +7211,17 @@ build_stateless_filter(struct ovn_datapath *od, acl->priority + OVN_ACL_PRI_OFFSET, acl->match, action, - &acl->header_); + &acl->header_, + &od->lflow_dep_mgr, OBJDEP_TYPE_CT_LB, + od->nbs->name, od); } else { ovn_lflow_add_with_hint(lflows, od, S_SWITCH_OUT_PRE_ACL, acl->priority + OVN_ACL_PRI_OFFSET, acl->match, action, - &acl->header_); + &acl->header_, + &od->lflow_dep_mgr, OBJDEP_TYPE_CT_LB, + od->nbs->name, od); } } @@ -7219,14 +7256,18 @@ build_pre_acls(struct ovn_datapath *od, const struct hmap *port_groups, { /* Ingress and Egress Pre-ACL Table (Priority 0): Packets are * allowed by default. */ - ovn_lflow_add(lflows, od, S_SWITCH_IN_PRE_ACL, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_SWITCH_OUT_PRE_ACL, 0, "1", "next;"); + ovn_lflow_add_with_ct_lb_lflow_ref( + lflows, od, S_SWITCH_IN_PRE_ACL, 0, "1", "next;", od->nbs->name); + ovn_lflow_add_with_ct_lb_lflow_ref( + lflows, od, S_SWITCH_OUT_PRE_ACL, 0, "1", "next;", od->nbs->name); - ovn_lflow_add(lflows, od, S_SWITCH_IN_PRE_ACL, 110, - "eth.dst == $svc_monitor_mac", "next;"); + ovn_lflow_add_with_ct_lb_lflow_ref( + lflows, od, S_SWITCH_IN_PRE_ACL, 110, + "eth.dst == $svc_monitor_mac", "next;", od->nbs->name); - ovn_lflow_add(lflows, od, S_SWITCH_OUT_PRE_ACL, 110, - "eth.src == $svc_monitor_mac", "next;"); + ovn_lflow_add_with_ct_lb_lflow_ref( + lflows, od, S_SWITCH_OUT_PRE_ACL, 110, + "eth.src == $svc_monitor_mac", "next;", od->nbs->name); /* If there are any stateful ACL rules in this datapath, we may * send IP packets for some (allow) filters through the conntrack action, @@ -7235,13 +7276,13 @@ build_pre_acls(struct ovn_datapath *od, const struct hmap *port_groups, for (size_t i = 0; i < od->n_router_ports; i++) { skip_port_from_conntrack(od, od->router_ports[i], S_SWITCH_IN_PRE_ACL, S_SWITCH_OUT_PRE_ACL, - 110, lflows); + 110, lflows, OBJDEP_TYPE_CT_LB); } for (size_t i = 0; i < od->n_localnet_ports; i++) { skip_port_from_conntrack(od, od->localnet_ports[i], S_SWITCH_IN_PRE_ACL, S_SWITCH_OUT_PRE_ACL, - 110, lflows); + 110, lflows, OBJDEP_TYPE_CT_LB); } /* stateless filters always take precedence over stateful ACLs. */ @@ -7251,18 +7292,24 @@ build_pre_acls(struct ovn_datapath *od, const struct hmap *port_groups, * * Not to do conntrack on ND and ICMP destination * unreachable packets. */ - ovn_lflow_add(lflows, od, S_SWITCH_IN_PRE_ACL, 110, - "nd || nd_rs || nd_ra || mldv1 || mldv2 || " - "(udp && udp.src == 546 && udp.dst == 547)", "next;"); - ovn_lflow_add(lflows, od, S_SWITCH_OUT_PRE_ACL, 110, - "nd || nd_rs || nd_ra || mldv1 || mldv2 || " - "(udp && udp.src == 546 && udp.dst == 547)", "next;"); + ovn_lflow_add_with_ct_lb_lflow_ref( + lflows, od, S_SWITCH_IN_PRE_ACL, 110, + "nd || nd_rs || nd_ra || mldv1 || mldv2 || " + "(udp && udp.src == 546 && udp.dst == 547)", "next;", + od->nbs->name); + ovn_lflow_add_with_ct_lb_lflow_ref( + lflows, od, S_SWITCH_OUT_PRE_ACL, 110, + "nd || nd_rs || nd_ra || mldv1 || mldv2 || " + "(udp && udp.src == 546 && udp.dst == 547)", "next;", + od->nbs->name); /* Do not send multicast packets to conntrack. */ - ovn_lflow_add(lflows, od, S_SWITCH_IN_PRE_ACL, 110, "eth.mcast", - "next;"); - ovn_lflow_add(lflows, od, S_SWITCH_OUT_PRE_ACL, 110, "eth.mcast", - "next;"); + ovn_lflow_add_with_ct_lb_lflow_ref( + lflows, od, S_SWITCH_IN_PRE_ACL, 110, "eth.mcast", "next;", + od->nbs->name); + ovn_lflow_add_with_ct_lb_lflow_ref( + lflows, od, S_SWITCH_OUT_PRE_ACL, 110, "eth.mcast", "next;", + od->nbs->name); /* Ingress and Egress Pre-ACL Table (Priority 100). * @@ -7272,10 +7319,13 @@ build_pre_acls(struct ovn_datapath *od, const struct hmap *port_groups, * * 'REGBIT_CONNTRACK_DEFRAG' is set to let the pre-stateful table send * it to conntrack for tracking and defragmentation. */ - ovn_lflow_add(lflows, od, S_SWITCH_IN_PRE_ACL, 100, "ip", - REGBIT_CONNTRACK_DEFRAG" = 1; next;"); - ovn_lflow_add(lflows, od, S_SWITCH_OUT_PRE_ACL, 100, "ip", - REGBIT_CONNTRACK_DEFRAG" = 1; next;"); + ovn_lflow_add_with_ct_lb_lflow_ref( + lflows, od, S_SWITCH_IN_PRE_ACL, 100, "ip", + REGBIT_CONNTRACK_DEFRAG" = 1; next;", od->nbs->name); + ovn_lflow_add_with_ct_lb_lflow_ref(lflows, od, S_SWITCH_OUT_PRE_ACL, + 100, "ip", + REGBIT_CONNTRACK_DEFRAG" = 1; next;", + od->nbs->name); } else if (od->has_lb_vip) { /* We'll build stateless filters if there are LB rules so that * the stateless flows are not tracked in pre-lb. */ @@ -7362,7 +7412,9 @@ build_interconn_mcast_snoop_flows(struct ovn_datapath *od, ovn_lflow_metered(lflows, od, S_SWITCH_OUT_PRE_LB, 120, match, "clone { igmp; }; next;", copp_meter_get(COPP_IGMP, od->nbs->copp, - meter_groups)); + meter_groups), + &od->lflow_dep_mgr, OBJDEP_TYPE_OD, + od->nbs->name, od); free(match); /* Punt MLD traffic to controller. */ @@ -7370,11 +7422,15 @@ build_interconn_mcast_snoop_flows(struct ovn_datapath *od, ovn_lflow_metered(lflows, od, S_SWITCH_OUT_PRE_LB, 120, match, "clone { igmp; }; next;", copp_meter_get(COPP_IGMP, od->nbs->copp, - meter_groups)); + meter_groups), + &od->lflow_dep_mgr, OBJDEP_TYPE_OD, + od->nbs->name, od); free(match); } } +/* Builds pre_lb lflows which are generic to a logical switch datapath + * and do not change if any load balancer is associated or disassociated. */ static void build_pre_lb(struct ovn_datapath *od, const struct shash *meter_groups, struct lflow_data *lflows) @@ -7383,32 +7439,49 @@ build_pre_lb(struct ovn_datapath *od, const struct shash *meter_groups, build_interconn_mcast_snoop_flows(od, meter_groups, lflows); /* Do not send multicast packets to conntrack */ - ovn_lflow_add(lflows, od, S_SWITCH_IN_PRE_LB, 110, "eth.mcast", "next;"); - ovn_lflow_add(lflows, od, S_SWITCH_OUT_PRE_LB, 110, "eth.mcast", "next;"); + ovn_lflow_add_with_od_lflow_ref(lflows, od, S_SWITCH_IN_PRE_LB, 110, + "eth.mcast", "next;", od->nbs->name); + ovn_lflow_add_with_od_lflow_ref(lflows, od, S_SWITCH_OUT_PRE_LB, 110, + "eth.mcast", "next;", od->nbs->name); /* Do not send ND packets to conntrack */ - ovn_lflow_add(lflows, od, S_SWITCH_IN_PRE_LB, 110, + ovn_lflow_add_with_od_lflow_ref(lflows, od, S_SWITCH_IN_PRE_LB, 110, "nd || nd_rs || nd_ra || mldv1 || mldv2", - "next;"); - ovn_lflow_add(lflows, od, S_SWITCH_OUT_PRE_LB, 110, + "next;", od->nbs->name); + ovn_lflow_add_with_od_lflow_ref(lflows, od, S_SWITCH_OUT_PRE_LB, 110, "nd || nd_rs || nd_ra || mldv1 || mldv2", - "next;"); + "next;", od->nbs->name); /* Do not send service monitor packets to conntrack. */ - ovn_lflow_add(lflows, od, S_SWITCH_IN_PRE_LB, 110, - "eth.dst == $svc_monitor_mac", "next;"); - ovn_lflow_add(lflows, od, S_SWITCH_OUT_PRE_LB, 110, - "eth.src == $svc_monitor_mac", "next;"); + ovn_lflow_add_with_od_lflow_ref(lflows, od, S_SWITCH_IN_PRE_LB, 110, + "eth.dst == $svc_monitor_mac", "next;", od->nbs->name); + ovn_lflow_add_with_od_lflow_ref(lflows, od, S_SWITCH_OUT_PRE_LB, 110, + "eth.src == $svc_monitor_mac", "next;", od->nbs->name); /* Allow all packets to go to next tables by default. */ - ovn_lflow_add(lflows, od, S_SWITCH_IN_PRE_LB, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_SWITCH_OUT_PRE_LB, 0, "1", "next;"); + ovn_lflow_add_with_od_lflow_ref(lflows, od, S_SWITCH_IN_PRE_LB, 0, + "1", "next;", od->nbs->name); + ovn_lflow_add_with_od_lflow_ref(lflows, od, S_SWITCH_OUT_PRE_LB, 0, + "1", "next;", od->nbs->name); for (size_t i = 0; i < od->n_router_ports; i++) { skip_port_from_conntrack(od, od->router_ports[i], S_SWITCH_IN_PRE_LB, S_SWITCH_OUT_PRE_LB, - 110, lflows); + 110, lflows, OBJDEP_TYPE_OD); } + + /* Do not sent statless flows via conntrack */ + ovn_lflow_add_with_od_lflow_ref(lflows, od, S_SWITCH_IN_PRE_LB, 110, + REGBIT_ACL_STATELESS" == 1", "next;", od->nbs->name); + ovn_lflow_add_with_od_lflow_ref(lflows, od, S_SWITCH_OUT_PRE_LB, 110, + REGBIT_ACL_STATELESS" == 1", "next;", od->nbs->name); +} + +/* Builds pre_lb lflows which are load balancer related and may change + * if a load balancer is associated or disassociated. */ +static void +build_pre_lb_lb_related(struct ovn_datapath *od, struct lflow_data *lflows) +{ /* Localnet ports have no need for going through conntrack, unless * the logical switch has a load balancer. Then, conntrack is necessary * so that traffic arriving via the localnet port can be load @@ -7418,16 +7491,10 @@ build_pre_lb(struct ovn_datapath *od, const struct shash *meter_groups, for (size_t i = 0; i < od->n_localnet_ports; i++) { skip_port_from_conntrack(od, od->localnet_ports[i], S_SWITCH_IN_PRE_LB, S_SWITCH_OUT_PRE_LB, - 110, lflows); + 110, lflows, OBJDEP_TYPE_LB); } } - /* Do not sent statless flows via conntrack */ - ovn_lflow_add(lflows, od, S_SWITCH_IN_PRE_LB, 110, - REGBIT_ACL_STATELESS" == 1", "next;"); - ovn_lflow_add(lflows, od, S_SWITCH_OUT_PRE_LB, 110, - REGBIT_ACL_STATELESS" == 1", "next;"); - /* 'REGBIT_CONNTRACK_NAT' is set to let the pre-stateful table send * packet to conntrack for defragmentation and possibly for unNATting. * @@ -7459,10 +7526,12 @@ build_pre_lb(struct ovn_datapath *od, const struct shash *meter_groups, * add a lflow to drop ct.inv packets. */ if (od->has_lb_vip) { - ovn_lflow_add(lflows, od, S_SWITCH_IN_PRE_LB, - 100, "ip", REGBIT_CONNTRACK_NAT" = 1; next;"); - ovn_lflow_add(lflows, od, S_SWITCH_OUT_PRE_LB, - 100, "ip", REGBIT_CONNTRACK_NAT" = 1; next;"); + ovn_lflow_add_with_lb_lflow_ref( + lflows, od, S_SWITCH_IN_PRE_LB, 100, "ip", + REGBIT_CONNTRACK_NAT" = 1; next;", od->nbs->name); + ovn_lflow_add_with_lb_lflow_ref( + lflows, od, S_SWITCH_OUT_PRE_LB, 100, "ip", + REGBIT_CONNTRACK_NAT" = 1; next;", od->nbs->name); } } @@ -7473,8 +7542,10 @@ build_pre_stateful(struct ovn_datapath *od, { /* Ingress and Egress pre-stateful Table (Priority 0): Packets are * allowed by default. */ - ovn_lflow_add(lflows, od, S_SWITCH_IN_PRE_STATEFUL, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_SWITCH_OUT_PRE_STATEFUL, 0, "1", "next;"); + ovn_lflow_add_with_od_lflow_ref(lflows, od, S_SWITCH_IN_PRE_STATEFUL, 0, + "1", "next;", od->nbs->name); + ovn_lflow_add_with_od_lflow_ref(lflows, od, S_SWITCH_OUT_PRE_STATEFUL, 0, + "1", "next;", od->nbs->name); /* Note: priority-120 flows are added in build_lb_rules_pre_stateful(). */ @@ -7482,19 +7553,19 @@ build_pre_stateful(struct ovn_datapath *od, ? "ct_lb_mark;" : "ct_lb;"; - ovn_lflow_add(lflows, od, S_SWITCH_IN_PRE_STATEFUL, 110, - REGBIT_CONNTRACK_NAT" == 1", ct_lb_action); + ovn_lflow_add_with_od_lflow_ref(lflows, od, S_SWITCH_IN_PRE_STATEFUL, 110, + REGBIT_CONNTRACK_NAT" == 1", ct_lb_action, od->nbs->name); - ovn_lflow_add(lflows, od, S_SWITCH_OUT_PRE_STATEFUL, 110, - REGBIT_CONNTRACK_NAT" == 1", ct_lb_action); + ovn_lflow_add_with_od_lflow_ref(lflows, od, S_SWITCH_OUT_PRE_STATEFUL, 110, + REGBIT_CONNTRACK_NAT" == 1", ct_lb_action, od->nbs->name); /* If REGBIT_CONNTRACK_DEFRAG is set as 1, then the packets should be * sent to conntrack for tracking and defragmentation. */ - ovn_lflow_add(lflows, od, S_SWITCH_IN_PRE_STATEFUL, 100, - REGBIT_CONNTRACK_DEFRAG" == 1", "ct_next;"); + ovn_lflow_add_with_od_lflow_ref(lflows, od, S_SWITCH_IN_PRE_STATEFUL, 100, + REGBIT_CONNTRACK_DEFRAG" == 1", "ct_next;", od->nbs->name); - ovn_lflow_add(lflows, od, S_SWITCH_OUT_PRE_STATEFUL, 100, - REGBIT_CONNTRACK_DEFRAG" == 1", "ct_next;"); + ovn_lflow_add_with_od_lflow_ref(lflows, od, S_SWITCH_OUT_PRE_STATEFUL, 100, + REGBIT_CONNTRACK_DEFRAG" == 1", "ct_next;", od->nbs->name); } @@ -7527,9 +7598,11 @@ build_acl_hints(struct ovn_datapath *od, /* In any case, advance to the next stage. */ if (!od->has_acls && !od->has_lb_vip) { - ovn_lflow_add(lflows, od, stage, UINT16_MAX, "1", "next;"); + ovn_lflow_add_with_ct_lb_lflow_ref(lflows, od, stage, UINT16_MAX, + "1", "next;", od->nbs->name); } else { - ovn_lflow_add(lflows, od, stage, 0, "1", "next;"); + ovn_lflow_add_with_ct_lb_lflow_ref(lflows, od, stage, 0, "1", + "next;", od->nbs->name); } if (!od->has_stateful_acl && !od->has_lb_vip) { @@ -7540,10 +7613,11 @@ build_acl_hints(struct ovn_datapath *od, * or drop ACLs. For allow ACLs, the connection must also be committed * to conntrack so we set REGBIT_ACL_HINT_ALLOW_NEW. */ - ovn_lflow_add(lflows, od, stage, 7, "ct.new && !ct.est", - REGBIT_ACL_HINT_ALLOW_NEW " = 1; " - REGBIT_ACL_HINT_DROP " = 1; " - "next;"); + ovn_lflow_add_with_ct_lb_lflow_ref( + lflows, od, stage, 7, "ct.new && !ct.est", + REGBIT_ACL_HINT_ALLOW_NEW " = 1; " + REGBIT_ACL_HINT_DROP " = 1; " + "next;", od->nbs->name); /* Already established connections in the "request" direction that * are already marked as "blocked" may hit either: @@ -7556,16 +7630,18 @@ build_acl_hints(struct ovn_datapath *od, match = features->ct_no_masked_label ? "!ct.new && ct.est && !ct.rpl && ct_mark.blocked == 1" : "!ct.new && ct.est && !ct.rpl && ct_label.blocked == 1"; - ovn_lflow_add(lflows, od, stage, 6, match, - REGBIT_ACL_HINT_ALLOW_NEW " = 1; " - REGBIT_ACL_HINT_DROP " = 1; " - "next;"); + ovn_lflow_add_with_ct_lb_lflow_ref( + lflows, od, stage, 6, match, + REGBIT_ACL_HINT_ALLOW_NEW " = 1; " + REGBIT_ACL_HINT_DROP " = 1; " + "next;", od->nbs->name); /* Not tracked traffic can either be allowed or dropped. */ - ovn_lflow_add(lflows, od, stage, 5, "!ct.trk", - REGBIT_ACL_HINT_ALLOW " = 1; " - REGBIT_ACL_HINT_DROP " = 1; " - "next;"); + ovn_lflow_add_with_ct_lb_lflow_ref( + lflows, od, stage, 5, "!ct.trk", + REGBIT_ACL_HINT_ALLOW " = 1; " + REGBIT_ACL_HINT_DROP " = 1; " + "next;", od->nbs->name); /* Already established connections in the "request" direction may hit * either: @@ -7578,23 +7654,24 @@ build_acl_hints(struct ovn_datapath *od, match = features->ct_no_masked_label ? "!ct.new && ct.est && !ct.rpl && ct_mark.blocked == 0" : "!ct.new && ct.est && !ct.rpl && ct_label.blocked == 0"; - ovn_lflow_add(lflows, od, stage, 4, match, - REGBIT_ACL_HINT_ALLOW " = 1; " - REGBIT_ACL_HINT_BLOCK " = 1; " - "next;"); + ovn_lflow_add_with_ct_lb_lflow_ref( + lflows, od, stage, 4, match, + REGBIT_ACL_HINT_ALLOW " = 1; " + REGBIT_ACL_HINT_BLOCK " = 1; next;", + od->nbs->name); /* Not established or established and already blocked connections may * hit drop ACLs. */ - ovn_lflow_add(lflows, od, stage, 3, "!ct.est", - REGBIT_ACL_HINT_DROP " = 1; " - "next;"); + ovn_lflow_add_with_ct_lb_lflow_ref( + lflows, od, stage, 3, "!ct.est", + REGBIT_ACL_HINT_DROP " = 1; next;", od->nbs->name); match = features->ct_no_masked_label ? "ct.est && ct_mark.blocked == 1" : "ct.est && ct_label.blocked == 1"; - ovn_lflow_add(lflows, od, stage, 2, match, - REGBIT_ACL_HINT_DROP " = 1; " - "next;"); + ovn_lflow_add_with_ct_lb_lflow_ref( + lflows, od, stage, 2, match, + REGBIT_ACL_HINT_DROP " = 1; next;", od->nbs->name); /* Established connections that were previously allowed might hit * drop ACLs in which case the connection must be committed with @@ -7603,9 +7680,9 @@ build_acl_hints(struct ovn_datapath *od, match = features->ct_no_masked_label ? "ct.est && ct_mark.blocked == 0" : "ct.est && ct_label.blocked == 0"; - ovn_lflow_add(lflows, od, stage, 1, match, + ovn_lflow_add_with_ct_lb_lflow_ref(lflows, od, stage, 1, match, REGBIT_ACL_HINT_BLOCK " = 1; " - "next;"); + "next;", od->nbs->name); } } @@ -7739,7 +7816,8 @@ consider_acl(struct lflow_data *lflows, struct ovn_datapath *od, ds_put_format(match, "(%s)", acl->match); ovn_lflow_add_with_hint(lflows, od, stage, priority, ds_cstr(match), ds_cstr(actions), - &acl->header_); + &acl->header_, &od->lflow_dep_mgr, + OBJDEP_TYPE_CT_LB, od->nbs->name, od); return; } @@ -7776,7 +7854,8 @@ consider_acl(struct lflow_data *lflows, struct ovn_datapath *od, ds_put_cstr(actions, "next;"); ovn_lflow_add_with_hint(lflows, od, stage, priority, ds_cstr(match), ds_cstr(actions), - &acl->header_); + &acl->header_, &od->lflow_dep_mgr, + OBJDEP_TYPE_CT_LB, od->nbs->name, od); /* Match on traffic in the request direction for an established * connection tracking entry that has not been marked for @@ -7798,7 +7877,8 @@ consider_acl(struct lflow_data *lflows, struct ovn_datapath *od, ds_put_cstr(actions, "next;"); ovn_lflow_add_with_hint(lflows, od, stage, priority, ds_cstr(match), ds_cstr(actions), - &acl->header_); + &acl->header_, &od->lflow_dep_mgr, + OBJDEP_TYPE_CT_LB, od->nbs->name, od); } else if (!strcmp(acl->action, "drop") || !strcmp(acl->action, "reject")) { /* The implementation of "drop" differs if stateful ACLs are in @@ -7815,7 +7895,8 @@ consider_acl(struct lflow_data *lflows, struct ovn_datapath *od, ds_put_cstr(actions, "next;"); ovn_lflow_add_with_hint(lflows, od, stage, priority, ds_cstr(match), ds_cstr(actions), - &acl->header_); + &acl->header_, &od->lflow_dep_mgr, + OBJDEP_TYPE_CT_LB, od->nbs->name, od); /* For an existing connection without ct_mark.blocked set, we've * encountered a policy change. ACLs previously allowed * this connection and we committed the connection tracking @@ -7836,7 +7917,8 @@ consider_acl(struct lflow_data *lflows, struct ovn_datapath *od, ct_blocked_match); ovn_lflow_add_with_hint(lflows, od, stage, priority, ds_cstr(match), ds_cstr(actions), - &acl->header_); + &acl->header_, &od->lflow_dep_mgr, + OBJDEP_TYPE_CT_LB, od->nbs->name, od); } } @@ -8003,18 +8085,20 @@ build_acl_action_lflows(struct ovn_datapath *od, struct lflow_data *lflows, for (size_t i = 0; i < ARRAY_SIZE(stages); i++) { enum ovn_stage stage = stages[i]; if (!od->has_acls) { - ovn_lflow_add(lflows, od, stage, 0, "1", "next;"); + ovn_lflow_add_with_ct_lb_lflow_ref(lflows, od, stage, 0, "1", + "next;", od->nbs->name); continue; } ds_truncate(actions, verdict_len); ds_put_cstr(actions, "next;"); - ovn_lflow_add(lflows, od, stage, 1000, - REGBIT_ACL_VERDICT_ALLOW " == 1", ds_cstr(actions)); + ovn_lflow_add_with_ct_lb_lflow_ref(lflows, od, stage, 1000, + REGBIT_ACL_VERDICT_ALLOW " == 1", ds_cstr(actions), + od->nbs->name); ds_truncate(actions, verdict_len); ds_put_cstr(actions, debug_implicit_drop_action()); - ovn_lflow_add(lflows, od, stage, 1000, + ovn_lflow_add_with_ct_lb_lflow_ref(lflows, od, stage, 1000, REGBIT_ACL_VERDICT_DROP " == 1", - ds_cstr(actions)); + ds_cstr(actions), od->nbs->name); bool ingress = ovn_stage_get_pipeline(stage) == P_IN; ds_truncate(actions, verdict_len); @@ -8030,11 +8114,14 @@ build_acl_action_lflows(struct ovn_datapath *od, struct lflow_data *lflows, ovn_lflow_metered(lflows, od, stage, 1000, REGBIT_ACL_VERDICT_REJECT " == 1", ds_cstr(actions), copp_meter_get(COPP_REJECT, od->nbs->copp, - meter_groups)); + meter_groups), + &od->lflow_dep_mgr, OBJDEP_TYPE_CT_LB, + od->nbs->name, od); ds_truncate(actions, verdict_len); ds_put_cstr(actions, default_acl_action); - ovn_lflow_add(lflows, od, stage, 0, "1", ds_cstr(actions)); + ovn_lflow_add_with_ct_lb_lflow_ref(lflows, od, stage, 0, "1", + ds_cstr(actions), od->nbs->name); struct ds tier_actions = DS_EMPTY_INITIALIZER; for (size_t j = 0; j < od->max_acl_tier; j++) { @@ -8045,8 +8132,10 @@ build_acl_action_lflows(struct ovn_datapath *od, struct lflow_data *lflows, "next(pipeline=%s,table=%d);", j + 1, ingress ? "ingress" : "egress", ovn_stage_get_table(stage) - 1); - ovn_lflow_add(lflows, od, stage, 500, ds_cstr(match), - ds_cstr(&tier_actions)); + ovn_lflow_add_with_ct_lb_lflow_ref(lflows, od, stage, 500, + ds_cstr(match), + ds_cstr(&tier_actions), + od->nbs->name); } ds_destroy(&tier_actions); } @@ -8111,7 +8200,8 @@ build_acl_log_related_flows(struct ovn_datapath *od, struct lflow_data *lflows, ovn_lflow_add_with_hint(lflows, od, log_related_stage, UINT16_MAX - 2, ds_cstr(match), ds_cstr(actions), - &acl->header_); + &acl->header_, &od->lflow_dep_mgr, + OBJDEP_TYPE_CT_LB, od->nbs->name, od); ds_clear(match); ds_put_format(match, "!ct.est && ct.rel && !ct.new%s && " @@ -8122,7 +8212,8 @@ build_acl_log_related_flows(struct ovn_datapath *od, struct lflow_data *lflows, ovn_lflow_add_with_hint(lflows, od, log_related_stage, UINT16_MAX - 2, ds_cstr(match), ds_cstr(actions), - &acl->header_); + &acl->header_, &od->lflow_dep_mgr, + OBJDEP_TYPE_CT_LB, od->nbs->name, od); } static void @@ -8149,21 +8240,31 @@ build_acls(struct ovn_datapath *od, const struct chassis_features *features, * are any stateful ACLs in this datapath. */ if (!od->has_acls) { if (!od->has_lb_vip) { - ovn_lflow_add(lflows, od, S_SWITCH_IN_ACL_EVAL, UINT16_MAX, "1", - "next;"); - ovn_lflow_add(lflows, od, S_SWITCH_OUT_ACL_EVAL, UINT16_MAX, "1", - "next;"); + ovn_lflow_add_with_ct_lb_lflow_ref( + lflows, od, S_SWITCH_IN_ACL_EVAL, UINT16_MAX, "1", "next;", + od->nbs->name); + ovn_lflow_add_with_ct_lb_lflow_ref( + lflows, od, S_SWITCH_OUT_ACL_EVAL, UINT16_MAX, "1", "next;", + od->nbs->name); } else { - ovn_lflow_add(lflows, od, S_SWITCH_IN_ACL_EVAL, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_SWITCH_OUT_ACL_EVAL, 0, "1", "next;"); - } - ovn_lflow_add(lflows, od, S_SWITCH_IN_ACL_AFTER_LB_EVAL, 0, "1", - "next;"); + ovn_lflow_add_with_ct_lb_lflow_ref( + lflows, od, S_SWITCH_IN_ACL_EVAL, 0, "1", "next;", + od->nbs->name); + ovn_lflow_add_with_ct_lb_lflow_ref( + lflows, od, S_SWITCH_OUT_ACL_EVAL, 0, "1", "next;", + od->nbs->name); + } + ovn_lflow_add_with_ct_lb_lflow_ref( + lflows, od, S_SWITCH_IN_ACL_AFTER_LB_EVAL, 0, "1", "next;", + od->nbs->name); } else { - ovn_lflow_add(lflows, od, S_SWITCH_IN_ACL_EVAL, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_SWITCH_OUT_ACL_EVAL, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_SWITCH_IN_ACL_AFTER_LB_EVAL, 0, "1", - "next;"); + ovn_lflow_add_with_ct_lb_lflow_ref( + lflows, od, S_SWITCH_IN_ACL_EVAL, 0, "1", "next;", od->nbs->name); + ovn_lflow_add_with_ct_lb_lflow_ref( + lflows, od, S_SWITCH_OUT_ACL_EVAL, 0, "1", "next;", od->nbs->name); + ovn_lflow_add_with_ct_lb_lflow_ref( + lflows, od, S_SWITCH_IN_ACL_AFTER_LB_EVAL, 0, "1", "next;", + od->nbs->name); } @@ -8192,22 +8293,24 @@ build_acls(struct ovn_datapath *od, const struct chassis_features *features, * uses "next;". */ ds_clear(&match); ds_put_format(&match, "ip && ct.est && %s == 1", ct_blocked_match); - ovn_lflow_add(lflows, od, S_SWITCH_IN_ACL_EVAL, 1, + ovn_lflow_add_with_ct_lb_lflow_ref(lflows, od, S_SWITCH_IN_ACL_EVAL, 1, ds_cstr(&match), REGBIT_CONNTRACK_COMMIT" = 1; " - REGBIT_ACL_VERDICT_ALLOW" = 1; next;"); - ovn_lflow_add(lflows, od, S_SWITCH_OUT_ACL_EVAL, 1, - ds_cstr(&match), + REGBIT_ACL_VERDICT_ALLOW" = 1; next;", od->nbs->name); + ovn_lflow_add_with_ct_lb_lflow_ref(lflows, od, S_SWITCH_OUT_ACL_EVAL, + 1, ds_cstr(&match), REGBIT_CONNTRACK_COMMIT" = 1; " - REGBIT_ACL_VERDICT_ALLOW" = 1; next;"); + REGBIT_ACL_VERDICT_ALLOW" = 1; next;", od->nbs->name); const char *next_action = default_acl_drop ? "next;" : REGBIT_CONNTRACK_COMMIT" = 1; next;"; - ovn_lflow_add(lflows, od, S_SWITCH_IN_ACL_EVAL, 1, "ip && !ct.est", - next_action); - ovn_lflow_add(lflows, od, S_SWITCH_OUT_ACL_EVAL, 1, "ip && !ct.est", - next_action); + ovn_lflow_add_with_ct_lb_lflow_ref( + lflows, od, S_SWITCH_IN_ACL_EVAL, 1, "ip && !ct.est", next_action, + od->nbs->name); + ovn_lflow_add_with_ct_lb_lflow_ref( + lflows, od, S_SWITCH_OUT_ACL_EVAL, 1, "ip && !ct.est", + next_action, od->nbs->name); /* Ingress and Egress ACL Table (Priority 65532). * @@ -8220,10 +8323,14 @@ build_acls(struct ovn_datapath *od, const struct chassis_features *features, ds_put_format(&match, "%s(ct.est && ct.rpl && %s == 1)", use_ct_inv_match ? "ct.inv || " : "", ct_blocked_match); - ovn_lflow_add(lflows, od, S_SWITCH_IN_ACL_EVAL, UINT16_MAX - 3, - ds_cstr(&match), REGBIT_ACL_VERDICT_DROP " = 1; next;"); - ovn_lflow_add(lflows, od, S_SWITCH_OUT_ACL_EVAL, UINT16_MAX - 3, - ds_cstr(&match), REGBIT_ACL_VERDICT_DROP " = 1; next;"); + ovn_lflow_add_with_ct_lb_lflow_ref( + lflows, od, S_SWITCH_IN_ACL_EVAL, UINT16_MAX - 3, + ds_cstr(&match), REGBIT_ACL_VERDICT_DROP " = 1; next;", + od->nbs->name); + ovn_lflow_add_with_ct_lb_lflow_ref( + lflows, od, S_SWITCH_OUT_ACL_EVAL, UINT16_MAX - 3, + ds_cstr(&match), REGBIT_ACL_VERDICT_DROP " = 1; next;", + od->nbs->name); /* Ingress and Egress ACL Table (Priority 65535 - 3). * @@ -8239,14 +8346,16 @@ build_acls(struct ovn_datapath *od, const struct chassis_features *features, "ct.rpl && %s == 0", use_ct_inv_match ? " && !ct.inv" : "", ct_blocked_match); - ovn_lflow_add(lflows, od, S_SWITCH_IN_ACL_EVAL, UINT16_MAX - 3, - ds_cstr(&match), REGBIT_ACL_HINT_DROP" = 0; " - REGBIT_ACL_HINT_BLOCK" = 0; " - REGBIT_ACL_HINT_ALLOW_REL" = 1; " - REGBIT_ACL_VERDICT_ALLOW" = 1; next;"); - ovn_lflow_add(lflows, od, S_SWITCH_OUT_ACL_EVAL, UINT16_MAX - 3, - ds_cstr(&match), - REGBIT_ACL_VERDICT_ALLOW " = 1; next;"); + ovn_lflow_add_with_ct_lb_lflow_ref( + lflows, od, S_SWITCH_IN_ACL_EVAL, UINT16_MAX - 3, + ds_cstr(&match), REGBIT_ACL_HINT_DROP" = 0; " + REGBIT_ACL_HINT_BLOCK" = 0; " + REGBIT_ACL_HINT_ALLOW_REL" = 1; " + REGBIT_ACL_VERDICT_ALLOW" = 1; next;", od->nbs->name); + ovn_lflow_add_with_ct_lb_lflow_ref( + lflows, od, S_SWITCH_OUT_ACL_EVAL, UINT16_MAX - 3, + ds_cstr(&match), REGBIT_ACL_VERDICT_ALLOW " = 1; next;", + od->nbs->name); /* Ingress and Egress ACL Table (Priority 65535). * @@ -8274,16 +8383,20 @@ build_acls(struct ovn_datapath *od, const struct chassis_features *features, ds_put_format(&match, "!ct.est && ct.rel && !ct.new%s && %s == 0", use_ct_inv_match ? " && !ct.inv" : "", ct_blocked_match); - ovn_lflow_add(lflows, od, S_SWITCH_IN_ACL_EVAL, UINT16_MAX - 3, - ds_cstr(&match), ct_in_acl_action); - ovn_lflow_add(lflows, od, S_SWITCH_OUT_ACL_EVAL, UINT16_MAX - 3, - ds_cstr(&match), ct_out_acl_action); + ovn_lflow_add_with_ct_lb_lflow_ref( + lflows, od, S_SWITCH_IN_ACL_EVAL, UINT16_MAX - 3, + ds_cstr(&match), ct_in_acl_action, od->nbs->name); + ovn_lflow_add_with_ct_lb_lflow_ref(lflows, od, S_SWITCH_OUT_ACL_EVAL, + UINT16_MAX - 3, + ds_cstr(&match), ct_out_acl_action, + od->nbs->name); /* Reply and related traffic matched by an "allow-related" ACL * should be allowed in the ls_in_acl_after_lb stage too. */ - ovn_lflow_add(lflows, od, S_SWITCH_IN_ACL_AFTER_LB_EVAL, - UINT16_MAX - 3, - REGBIT_ACL_HINT_ALLOW_REL" == 1", - REGBIT_ACL_VERDICT_ALLOW " = 1; next;"); + ovn_lflow_add_with_ct_lb_lflow_ref( + lflows, od, S_SWITCH_IN_ACL_AFTER_LB_EVAL, + UINT16_MAX - 3, + REGBIT_ACL_HINT_ALLOW_REL" == 1", + REGBIT_ACL_VERDICT_ALLOW " = 1; next;", od->nbs->name); } /* Ingress and Egress ACL Table (Priority 65532). @@ -8293,15 +8406,18 @@ build_acls(struct ovn_datapath *od, const struct chassis_features *features, * Also, don't send them to conntrack because session tracking * for these protocols is not working properly: * https://bugzilla.kernel.org/show_bug.cgi?id=11797. */ - ovn_lflow_add(lflows, od, S_SWITCH_IN_ACL_EVAL, UINT16_MAX - 3, - IPV6_CT_OMIT_MATCH, - REGBIT_ACL_VERDICT_ALLOW " = 1; next;"); - ovn_lflow_add(lflows, od, S_SWITCH_OUT_ACL_EVAL, UINT16_MAX - 3, - IPV6_CT_OMIT_MATCH, - REGBIT_ACL_VERDICT_ALLOW " = 1; next;"); - ovn_lflow_add(lflows, od, S_SWITCH_IN_ACL_AFTER_LB_EVAL, UINT16_MAX - 3, - IPV6_CT_OMIT_MATCH, - REGBIT_ACL_VERDICT_ALLOW " = 1; next;"); + ovn_lflow_add_with_ct_lb_lflow_ref( + lflows, od, S_SWITCH_IN_ACL_EVAL, UINT16_MAX - 3, + IPV6_CT_OMIT_MATCH, REGBIT_ACL_VERDICT_ALLOW " = 1; next;", + od->nbs->name); + ovn_lflow_add_with_ct_lb_lflow_ref( + lflows, od, S_SWITCH_OUT_ACL_EVAL, UINT16_MAX - 3, + IPV6_CT_OMIT_MATCH, REGBIT_ACL_VERDICT_ALLOW " = 1; next;", + od->nbs->name); + ovn_lflow_add_with_ct_lb_lflow_ref( + lflows, od, S_SWITCH_IN_ACL_AFTER_LB_EVAL, UINT16_MAX - 3, + IPV6_CT_OMIT_MATCH, REGBIT_ACL_VERDICT_ALLOW " = 1; next;", + od->nbs->name); /* Ingress or Egress ACL Table (Various priorities). */ for (size_t i = 0; i < od->nbs->n_acls; i++) { @@ -8336,23 +8452,24 @@ build_acls(struct ovn_datapath *od, const struct chassis_features *features, has_stateful ? REGBIT_ACL_VERDICT_ALLOW" = 1; " "ct_commit; next;" : REGBIT_ACL_VERDICT_ALLOW" = 1; next;"; - ovn_lflow_add( + ovn_lflow_add_with_ct_lb_lflow_ref( lflows, od, S_SWITCH_OUT_ACL_EVAL, 34000, "udp.src == 53", - dns_actions); + dns_actions, od->nbs->name); } if (od->has_acls || od->has_lb_vip) { /* Add a 34000 priority flow to advance the service monitor reply * packets to skip applying ingress ACLs. */ - ovn_lflow_add(lflows, od, S_SWITCH_IN_ACL_EVAL, 34000, - "eth.dst == $svc_monitor_mac", - REGBIT_ACL_VERDICT_ALLOW" = 1; next;"); + ovn_lflow_add_with_ct_lb_lflow_ref(lflows, od, S_SWITCH_IN_ACL_EVAL, + 34000, "eth.dst == $svc_monitor_mac", + REGBIT_ACL_VERDICT_ALLOW" = 1; next;", od->nbs->name); /* Add a 34000 priority flow to advance the service monitor packets * generated by ovn-controller to skip applying egress ACLs. */ - ovn_lflow_add(lflows, od, S_SWITCH_OUT_ACL_EVAL, 34000, - "eth.src == $svc_monitor_mac", - REGBIT_ACL_VERDICT_ALLOW" = 1; next;"); + ovn_lflow_add_with_ct_lb_lflow_ref( + lflows, od, S_SWITCH_OUT_ACL_EVAL, 34000, + "eth.src == $svc_monitor_mac", + REGBIT_ACL_VERDICT_ALLOW" = 1; next;", od->nbs->name); } build_acl_action_lflows(od, lflows, default_acl_action, meter_groups, @@ -8366,10 +8483,14 @@ static void build_qos(struct ovn_datapath *od, struct lflow_data *lflows) { struct ds action = DS_EMPTY_INITIALIZER; - ovn_lflow_add(lflows, od, S_SWITCH_IN_QOS_MARK, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_SWITCH_OUT_QOS_MARK, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_SWITCH_IN_QOS_METER, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_SWITCH_OUT_QOS_METER, 0, "1", "next;"); + ovn_lflow_add_with_od_lflow_ref( + lflows, od, S_SWITCH_IN_QOS_MARK, 0, "1", "next;", od->nbs->name); + ovn_lflow_add_with_od_lflow_ref( + lflows, od, S_SWITCH_OUT_QOS_MARK, 0, "1", "next;", od->nbs->name); + ovn_lflow_add_with_od_lflow_ref( + lflows, od, S_SWITCH_IN_QOS_METER, 0, "1", "next;", od->nbs->name); + ovn_lflow_add_with_od_lflow_ref( + lflows, od, S_SWITCH_OUT_QOS_METER, 0, "1", "next;", od->nbs->name); for (size_t i = 0; i < od->nbs->n_qos_rules; i++) { struct nbrec_qos *qos = od->nbs->qos_rules[i]; @@ -8386,7 +8507,9 @@ build_qos(struct ovn_datapath *od, struct lflow_data *lflows) { ovn_lflow_add_with_hint(lflows, od, stage, qos->priority, qos->match, ds_cstr(&action), - &qos->header_); + &qos->header_, + &od->lflow_dep_mgr, + OBJDEP_TYPE_OD, od->nbs->name, od); } } @@ -8417,7 +8540,9 @@ build_qos(struct ovn_datapath *od, struct lflow_data *lflows) { ovn_lflow_add_with_hint(lflows, od, stage, qos->priority, qos->match, ds_cstr(&action), - &qos->header_); + &qos->header_, + &od->lflow_dep_mgr, + OBJDEP_TYPE_OD, od->nbs->name, od); } } ds_destroy(&action); @@ -8482,7 +8607,8 @@ build_lb_rules_pre_stateful(struct lflow_data *lflows, ovn_lflow_add_with_dp_group( lflows, lb_dps->nb_ls_map, ods_size(ls_datapaths), S_SWITCH_IN_PRE_STATEFUL, 120, ds_cstr(match), ds_cstr(action), - &lb->nlb->header_); + &lb->nlb->header_, &lb_dps->lflow_dep_mgr, + OBJDEP_TYPE_LB, lb->nlb->name, NULL); } } @@ -8530,7 +8656,8 @@ build_lb_affinity_lr_flows(struct lflow_data *lflows, const struct ovn_northd_lb *lb, struct ovn_lb_vip *lb_vip, char *new_lb_match, char *lb_action, const unsigned long *dp_bitmap, - const struct ovn_datapaths *lr_datapaths) + const struct ovn_datapaths *lr_datapaths, + struct objdep_mgr *lflow_dep_mgr) { if (!lb->affinity_timeout || bitmap_is_all_zeros(dp_bitmap, ods_size(lr_datapaths))) { @@ -8569,7 +8696,8 @@ build_lb_affinity_lr_flows(struct lflow_data *lflows, ovn_lflow_add_with_dp_group( lflows, dp_bitmap, ods_size(lr_datapaths), S_ROUTER_IN_LB_AFF_CHECK, - 100, new_lb_match, ds_cstr(&aff_check_action), &lb->nlb->header_); + 100, new_lb_match, ds_cstr(&aff_check_action), &lb->nlb->header_, + lflow_dep_mgr, OBJDEP_TYPE_LB, lb->nlb->name, NULL); /* Prepare common part of affinity LB and affinity learn action. */ ds_put_format(&aff_action, "%s = %s; ", reg_vip, lb_vip->vip_str); @@ -8651,12 +8779,14 @@ build_lb_affinity_lr_flows(struct lflow_data *lflows, ovn_lflow_add_with_dp_group( lflows, dp_bitmap, ods_size(lr_datapaths), S_ROUTER_IN_LB_AFF_LEARN, 100, ds_cstr(&aff_match_learn), - ds_cstr(&aff_action_learn), &lb->nlb->header_); + ds_cstr(&aff_action_learn), &lb->nlb->header_, + lflow_dep_mgr, OBJDEP_TYPE_LB, lb->nlb->name, NULL); /* Use already selected backend within affinity timeslot. */ ovn_lflow_add_with_dp_group( lflows, dp_bitmap, ods_size(lr_datapaths), S_ROUTER_IN_DNAT, 150, - ds_cstr(&aff_match), ds_cstr(&aff_action), &lb->nlb->header_); + ds_cstr(&aff_match), ds_cstr(&aff_action), &lb->nlb->header_, + lflow_dep_mgr, OBJDEP_TYPE_LB, lb->nlb->name, NULL); ds_truncate(&aff_action, aff_action_len); ds_truncate(&aff_action_learn, aff_action_learn_len); @@ -8744,7 +8874,8 @@ build_lb_affinity_ls_flows(struct lflow_data *lflows, ovn_lflow_add_with_dp_group( lflows, lb_dps->nb_ls_map, ods_size(ls_datapaths), S_SWITCH_IN_LB_AFF_CHECK, 100, ds_cstr(&new_lb_match), aff_check, - &lb_dps->lb->nlb->header_); + &lb_dps->lb->nlb->header_, + &lb_dps->lflow_dep_mgr, OBJDEP_TYPE_LB, lb->nlb->name, NULL); ds_destroy(&new_lb_match); struct ds aff_action = DS_EMPTY_INITIALIZER; @@ -8834,13 +8965,15 @@ build_lb_affinity_ls_flows(struct lflow_data *lflows, ovn_lflow_add_with_dp_group( lflows, lb_dps->nb_ls_map, ods_size(ls_datapaths), S_SWITCH_IN_LB_AFF_LEARN, 100, ds_cstr(&aff_match_learn), - ds_cstr(&aff_action_learn), &lb->nlb->header_); + ds_cstr(&aff_action_learn), &lb->nlb->header_, + &lb_dps->lflow_dep_mgr, OBJDEP_TYPE_LB, lb->nlb->name, NULL); /* Use already selected backend within affinity timeslot. */ ovn_lflow_add_with_dp_group( lflows, lb_dps->nb_ls_map, ods_size(ls_datapaths), S_SWITCH_IN_LB, 150, ds_cstr(&aff_match), ds_cstr(&aff_action), - &lb->nlb->header_); + &lb->nlb->header_, + &lb_dps->lflow_dep_mgr, OBJDEP_TYPE_LB, lb->nlb->name, NULL); ds_truncate(&aff_action, aff_action_len); ds_truncate(&aff_action_learn, aff_action_learn_len); @@ -8859,8 +8992,10 @@ build_lswitch_lb_affinity_default_flows(struct ovn_datapath *od, struct lflow_data *lflows) { ovs_assert(od->nbs); - ovn_lflow_add(lflows, od, S_SWITCH_IN_LB_AFF_CHECK, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_SWITCH_IN_LB_AFF_LEARN, 0, "1", "next;"); + ovn_lflow_add_with_od_lflow_ref(lflows, od, S_SWITCH_IN_LB_AFF_CHECK, 0, + "1", "next;", od->nbs->name); + ovn_lflow_add_with_od_lflow_ref(lflows, od, S_SWITCH_IN_LB_AFF_LEARN, 0, + "1", "next;", od->nbs->name); } static void @@ -8868,8 +9003,10 @@ build_lrouter_lb_affinity_default_flows(struct ovn_datapath *od, struct lflow_data *lflows) { ovs_assert(od->nbr); - ovn_lflow_add(lflows, od, S_ROUTER_IN_LB_AFF_CHECK, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_ROUTER_IN_LB_AFF_LEARN, 0, "1", "next;"); + ovn_lflow_add_with_od_lflow_ref(lflows, od, S_ROUTER_IN_LB_AFF_CHECK, 0, + "1", "next;", od->nbr->name); + ovn_lflow_add_with_od_lflow_ref(lflows, od, S_ROUTER_IN_LB_AFF_LEARN, 0, + "1", "next;", od->nbr->name); } static void @@ -8937,14 +9074,18 @@ build_lb_rules(struct lflow_data *lflows, struct ovn_lb_datapaths *lb_dps, ovn_lflow_add_with_hint__( lflows, od, S_SWITCH_IN_LB, priority, ds_cstr(match), ds_cstr(action), - NULL, meter, &lb->nlb->header_); + NULL, meter, &lb->nlb->header_, + &lb_dps->lflow_dep_mgr, OBJDEP_TYPE_LB, + lb->nlb->name, NULL); } } if (!reject || build_non_meter) { ovn_lflow_add_with_dp_group( lflows, dp_non_meter ? dp_non_meter : lb_dps->nb_ls_map, ods_size(ls_datapaths), S_SWITCH_IN_LB, priority, - ds_cstr(match), ds_cstr(action), &lb->nlb->header_); + ds_cstr(match), ds_cstr(action), &lb->nlb->header_, + &lb_dps->lflow_dep_mgr, OBJDEP_TYPE_LB, + lb->nlb->name, NULL); } bitmap_free(dp_non_meter); } @@ -8962,9 +9103,12 @@ build_stateful(struct ovn_datapath *od, /* Ingress LB, Ingress and Egress stateful Table (Priority 0): Packets are * allowed by default. */ - ovn_lflow_add(lflows, od, S_SWITCH_IN_LB, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_SWITCH_IN_STATEFUL, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_SWITCH_OUT_STATEFUL, 0, "1", "next;"); + ovn_lflow_add_with_od_lflow_ref(lflows, od, S_SWITCH_IN_LB, 0, "1", + "next;", od->nbs->name); + ovn_lflow_add_with_od_lflow_ref(lflows, od, S_SWITCH_IN_STATEFUL, 0, + "1", "next;", od->nbs->name); + ovn_lflow_add_with_od_lflow_ref(lflows, od, S_SWITCH_OUT_STATEFUL, 0, + "1", "next;", od->nbs->name); /* If REGBIT_CONNTRACK_COMMIT is set as 1 and * REGBIT_CONNTRACK_SET_LABEL is set to 1, then the packets should be @@ -8975,14 +9119,14 @@ build_stateful(struct ovn_datapath *od, ds_put_format(&actions, "ct_commit { %s = 0; " "ct_label.label = " REG_LABEL "; }; next;", ct_block_action); - ovn_lflow_add(lflows, od, S_SWITCH_IN_STATEFUL, 100, + ovn_lflow_add_with_od_lflow_ref(lflows, od, S_SWITCH_IN_STATEFUL, 100, REGBIT_CONNTRACK_COMMIT" == 1 && " REGBIT_ACL_LABEL" == 1", - ds_cstr(&actions)); - ovn_lflow_add(lflows, od, S_SWITCH_OUT_STATEFUL, 100, + ds_cstr(&actions), od->nbs->name); + ovn_lflow_add_with_od_lflow_ref(lflows, od, S_SWITCH_OUT_STATEFUL, 100, REGBIT_CONNTRACK_COMMIT" == 1 && " REGBIT_ACL_LABEL" == 1", - ds_cstr(&actions)); + ds_cstr(&actions), od->nbs->name); /* If REGBIT_CONNTRACK_COMMIT is set as 1, then the packets should be * committed to conntrack. We always set ct_mark.blocked to 0 here as @@ -8990,14 +9134,14 @@ build_stateful(struct ovn_datapath *od, * want to allow to continue. */ ds_clear(&actions); ds_put_format(&actions, "ct_commit { %s = 0; }; next;", ct_block_action); - ovn_lflow_add(lflows, od, S_SWITCH_IN_STATEFUL, 100, + ovn_lflow_add_with_od_lflow_ref(lflows, od, S_SWITCH_IN_STATEFUL, 100, REGBIT_CONNTRACK_COMMIT" == 1 && " REGBIT_ACL_LABEL" == 0", - ds_cstr(&actions)); - ovn_lflow_add(lflows, od, S_SWITCH_OUT_STATEFUL, 100, + ds_cstr(&actions), od->nbs->name); + ovn_lflow_add_with_od_lflow_ref(lflows, od, S_SWITCH_OUT_STATEFUL, 100, REGBIT_CONNTRACK_COMMIT" == 1 && " REGBIT_ACL_LABEL" == 0", - ds_cstr(&actions)); + ds_cstr(&actions), od->nbs->name); ds_destroy(&actions); } @@ -9007,9 +9151,12 @@ build_lb_hairpin(struct ovn_datapath *od, struct lflow_data *lflows) /* Ingress Pre-Hairpin/Nat-Hairpin/Hairpin tabled (Priority 0). * Packets that don't need hairpinning should continue processing. */ - ovn_lflow_add(lflows, od, S_SWITCH_IN_PRE_HAIRPIN, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_SWITCH_IN_NAT_HAIRPIN, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_SWITCH_IN_HAIRPIN, 0, "1", "next;"); + ovn_lflow_add_with_lb_lflow_ref(lflows, od, S_SWITCH_IN_PRE_HAIRPIN, 0, + "1", "next;", od->nbs->name); + ovn_lflow_add_with_lb_lflow_ref(lflows, od, S_SWITCH_IN_NAT_HAIRPIN, 0, + "1", "next;", od->nbs->name); + ovn_lflow_add_with_lb_lflow_ref(lflows, od, S_SWITCH_IN_HAIRPIN, 0, + "1", "next;", od->nbs->name); if (od->has_lb_vip) { /* Check if the packet needs to be hairpinned. @@ -9020,8 +9167,8 @@ build_lb_hairpin(struct ovn_datapath *od, struct lflow_data *lflows) lflows, od, S_SWITCH_IN_PRE_HAIRPIN, 100, "ip && ct.trk", REGBIT_HAIRPIN " = chk_lb_hairpin(); " REGBIT_HAIRPIN_REPLY " = chk_lb_hairpin_reply(); " - "next;", - &od->nbs->header_); + "next;", &od->nbs->header_, + &od->lflow_dep_mgr, OBJDEP_TYPE_LB, od->nbs->name, NULL); /* If packet needs to be hairpinned, snat the src ip with the VIP * for new sessions. */ @@ -9029,7 +9176,9 @@ build_lb_hairpin(struct ovn_datapath *od, struct lflow_data *lflows) "ip && ct.new && ct.trk" " && "REGBIT_HAIRPIN " == 1", "ct_snat_to_vip; next;", - &od->nbs->header_); + &od->nbs->header_, + &od->lflow_dep_mgr, OBJDEP_TYPE_LB, + od->nbs->name, od); /* If packet needs to be hairpinned, for established sessions there * should already be an SNAT conntrack entry. @@ -9038,23 +9187,27 @@ build_lb_hairpin(struct ovn_datapath *od, struct lflow_data *lflows) "ip && ct.est && ct.trk" " && "REGBIT_HAIRPIN " == 1", "ct_snat;", - &od->nbs->header_); + &od->nbs->header_, + &od->lflow_dep_mgr, OBJDEP_TYPE_LB, + od->nbs->name, od); /* For the reply of hairpinned traffic, snat the src ip to the VIP. */ ovn_lflow_add_with_hint(lflows, od, S_SWITCH_IN_NAT_HAIRPIN, 90, "ip && "REGBIT_HAIRPIN_REPLY " == 1", "ct_snat;", - &od->nbs->header_); + &od->nbs->header_, + &od->lflow_dep_mgr, OBJDEP_TYPE_LB, + od->nbs->name, od); /* Ingress Hairpin table. * - Priority 1: Packets that were SNAT-ed for hairpinning should be * looped back (i.e., swap ETH addresses and send back on inport). */ - ovn_lflow_add( + ovn_lflow_add_with_lb_lflow_ref( lflows, od, S_SWITCH_IN_HAIRPIN, 1, "("REGBIT_HAIRPIN " == 1 || " REGBIT_HAIRPIN_REPLY " == 1)", "eth.dst <-> eth.src; outport = inport; flags.loopback = 1; " - "output;"); + "output;", od->nbs->name); } } @@ -9071,8 +9224,9 @@ build_vtep_hairpin(struct ovn_datapath *od, struct lflow_data *lflows) */ char *action = xasprintf("next(pipeline=ingress, table=%d);", ovn_stage_get_table(S_SWITCH_IN_L2_LKUP)); - ovn_lflow_add(lflows, od, S_SWITCH_IN_HAIRPIN, 1000, - REGBIT_FROM_RAMP" == 1", action); + ovn_lflow_add_with_od_lflow_ref(lflows, od, S_SWITCH_IN_HAIRPIN, 1000, + REGBIT_FROM_RAMP" == 1", action, + od->nbs->name); free(action); /* Ingress pre-arp flow for traffic from VTEP (ramp) switch. @@ -9088,8 +9242,9 @@ build_vtep_hairpin(struct ovn_datapath *od, struct lflow_data *lflows) ds_put_format(&match, REGBIT_FROM_RAMP" == 1 && is_chassis_resident(%s)", op->cr_port->json_key); - ovn_lflow_add(lflows, od, S_SWITCH_IN_HAIRPIN, 2000, - ds_cstr(&match), "next;"); + ovn_lflow_add_with_od_lflow_ref(lflows, od, S_SWITCH_IN_HAIRPIN, + 2000, ds_cstr(&match), "next;", + od->nbs->name); } } @@ -9098,9 +9253,10 @@ build_vtep_hairpin(struct ovn_datapath *od, struct lflow_data *lflows) * Neighbor resolution for router ports is done in logical router ingress * pipeline. ARP resolution for vif lports is done directly by vif ports. */ - ovn_lflow_add(lflows, od, S_SWITCH_IN_ARP_ND_RSP, 65535, - REGBIT_FROM_RAMP" == 1 && (arp || nd_ns)", - "flags.loopback = 1; next;"); + ovn_lflow_add_with_od_lflow_ref(lflows, od, S_SWITCH_IN_ARP_ND_RSP, 65535, + REGBIT_FROM_RAMP" == 1 && (arp || nd_ns)", + "flags.loopback = 1; next;", + od->nbs->name); ds_destroy(&match); } @@ -9142,7 +9298,9 @@ build_fwd_group_lflows(struct ovn_datapath *od, struct lflow_data *lflows) ovn_lflow_add_with_hint(lflows, od, S_SWITCH_IN_ARP_ND_RSP, 50, ds_cstr(&match), ds_cstr(&actions), - &fwd_group->header_); + &fwd_group->header_, + &od->lflow_dep_mgr, OBJDEP_TYPE_OD, + od->nbs->name, NULL); /* L2 lookup for the forwarding group's virtual MAC */ ds_clear(&match); @@ -9165,7 +9323,9 @@ build_fwd_group_lflows(struct ovn_datapath *od, struct lflow_data *lflows) ds_put_format(&actions, "fwd_group(%s);", ds_cstr(&group_ports)); ovn_lflow_add_with_hint(lflows, od, S_SWITCH_IN_L2_LKUP, 50, ds_cstr(&match), ds_cstr(&actions), - &fwd_group->header_); + &fwd_group->header_, + &od->lflow_dep_mgr, OBJDEP_TYPE_OD, + od->nbs->name, NULL); } ds_destroy(&match); @@ -9838,15 +9998,17 @@ build_lswitch_lflows_l2_unknown(struct ovn_datapath *od, { /* Ingress table 25/26: Destination lookup for unknown MACs. */ if (od->has_unknown) { - ovn_lflow_add(lflows, od, S_SWITCH_IN_L2_UNKNOWN, 50, - "outport == \"none\"", - "outport = \""MC_UNKNOWN "\"; output;"); + ovn_lflow_add_with_od_lflow_ref(lflows, od, S_SWITCH_IN_L2_UNKNOWN, 50, + "outport == \"none\"", + "outport = \""MC_UNKNOWN "\"; output;", + od->nbs->name); } else { - ovn_lflow_add(lflows, od, S_SWITCH_IN_L2_UNKNOWN, 50, - "outport == \"none\"", debug_drop_action()); + ovn_lflow_add_with_od_lflow_ref(lflows, od, S_SWITCH_IN_L2_UNKNOWN, 50, + "outport == \"none\"", + debug_drop_action(), od->nbs->name); } - ovn_lflow_add(lflows, od, S_SWITCH_IN_L2_UNKNOWN, 0, "1", - "output;"); + ovn_lflow_add_with_od_lflow_ref(lflows, od, S_SWITCH_IN_L2_UNKNOWN, 0, "1", + "output;", od->nbs->name); } /* Build pre-ACL and ACL tables for both ingress and egress. @@ -9863,6 +10025,7 @@ build_lswitch_lflows_pre_acl_and_acl(struct ovn_datapath *od, build_pre_acls(od, port_groups, lflows); build_pre_lb(od, meter_groups, lflows); + build_pre_lb_lb_related(od, lflows); build_pre_stateful(od, features, lflows); build_acl_hints(od, features, lflows); build_acls(od, features, lflows, port_groups, meter_groups); @@ -9882,21 +10045,26 @@ build_lswitch_lflows_admission_control(struct ovn_datapath *od, /* Logical VLANs not supported. */ if (!is_vlan_transparent(od)) { /* Block logical VLANs. */ - ovn_lflow_add(lflows, od, S_SWITCH_IN_CHECK_PORT_SEC, 100, - "vlan.present", debug_drop_action()); + ovn_lflow_add_with_od_lflow_ref(lflows, od, S_SWITCH_IN_CHECK_PORT_SEC, + 100, "vlan.present", + debug_drop_action(), od->nbs->name); } /* Broadcast/multicast source address is invalid. */ - ovn_lflow_add(lflows, od, S_SWITCH_IN_CHECK_PORT_SEC, 100, - "eth.src[40]", debug_drop_action()); + ovn_lflow_add_with_od_lflow_ref(lflows, od, S_SWITCH_IN_CHECK_PORT_SEC, + 100, "eth.src[40]", debug_drop_action(), + od->nbs->name); - ovn_lflow_add(lflows, od, S_SWITCH_IN_CHECK_PORT_SEC, 50, "1", - REGBIT_PORT_SEC_DROP" = check_in_port_sec(); next;"); + ovn_lflow_add_with_od_lflow_ref( + lflows, od, S_SWITCH_IN_CHECK_PORT_SEC, 50, "1", + REGBIT_PORT_SEC_DROP" = check_in_port_sec(); next;", od->nbs->name); - ovn_lflow_add(lflows, od, S_SWITCH_IN_APPLY_PORT_SEC, 50, - REGBIT_PORT_SEC_DROP" == 1", debug_drop_action()); + ovn_lflow_add_with_od_lflow_ref(lflows, od, S_SWITCH_IN_APPLY_PORT_SEC, 50, + REGBIT_PORT_SEC_DROP" == 1", + debug_drop_action(), od->nbs->name); - ovn_lflow_add(lflows, od, S_SWITCH_IN_APPLY_PORT_SEC, 0, "1", "next;"); + ovn_lflow_add_with_od_lflow_ref(lflows, od, S_SWITCH_IN_APPLY_PORT_SEC, 0, + "1", "next;", od->nbs->name); } /* Ingress table 19: ARP/ND responder, skip requests coming from localnet @@ -10249,18 +10417,20 @@ build_lswitch_arp_nd_responder_default(struct ovn_datapath *od, struct lflow_data *lflows) { ovs_assert(od->nbs); - ovn_lflow_add(lflows, od, S_SWITCH_IN_ARP_ND_RSP, 0, "1", "next;"); + ovn_lflow_add_with_od_lflow_ref(lflows, od, S_SWITCH_IN_ARP_ND_RSP, 0, + "1", "next;", od->nbs->name); } /* Ingress table 19: ARP/ND responder for service monitor source ip. * (priority 110)*/ static void -build_lswitch_arp_nd_service_monitor(const struct ovn_northd_lb *lb, +build_lswitch_arp_nd_service_monitor(struct ovn_lb_datapaths *lb_dps, const struct hmap *ls_ports, struct lflow_data *lflows, struct ds *actions, struct ds *match) { + const struct ovn_northd_lb *lb = lb_dps->lb; for (size_t i = 0; i < lb->n_vips; i++) { struct ovn_northd_lb_vip *lb_vip_nb = &lb->vips_nb[i]; if (!lb_vip_nb->lb_health_check) { @@ -10323,7 +10493,9 @@ build_lswitch_arp_nd_service_monitor(const struct ovn_northd_lb *lb, op->od, S_SWITCH_IN_ARP_ND_RSP, 110, ds_cstr(match), ds_cstr(actions), - &lb->nlb->header_); + &lb->nlb->header_, + &lb_dps->lflow_dep_mgr, OBJDEP_TYPE_LB, + lb->nlb->name, NULL); } } } @@ -10392,11 +10564,16 @@ build_lswitch_dhcp_and_dns_defaults(struct ovn_datapath *od, struct lflow_data *lflows) { ovs_assert(od->nbs); - ovn_lflow_add(lflows, od, S_SWITCH_IN_DHCP_OPTIONS, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_SWITCH_IN_DHCP_RESPONSE, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_SWITCH_IN_DNS_LOOKUP, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_SWITCH_IN_DNS_RESPONSE, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_SWITCH_IN_EXTERNAL_PORT, 0, "1", "next;"); + ovn_lflow_add_with_od_lflow_ref(lflows, od, S_SWITCH_IN_DHCP_OPTIONS, 0, + "1", "next;", od->nbs->name); + ovn_lflow_add_with_od_lflow_ref(lflows, od, S_SWITCH_IN_DHCP_RESPONSE, 0, + "1", "next;", od->nbs->name); + ovn_lflow_add_with_od_lflow_ref(lflows, od, S_SWITCH_IN_DNS_LOOKUP, 0, + "1", "next;", od->nbs->name); + ovn_lflow_add_with_od_lflow_ref(lflows, od, S_SWITCH_IN_DNS_RESPONSE, 0, + "1", "next;", od->nbs->name); + ovn_lflow_add_with_od_lflow_ref(lflows, od, S_SWITCH_IN_EXTERNAL_PORT, 0, + "1", "next;", od->nbs->name); } /* Logical switch ingress table 22 and 23: DNS lookup and response @@ -10415,18 +10592,19 @@ build_lswitch_dns_lookup_and_response(struct ovn_datapath *od, "udp.dst == 53", REGBIT_DNS_LOOKUP_RESULT" = dns_lookup(); next;", copp_meter_get(COPP_DNS, od->nbs->copp, - meter_groups)); + meter_groups), + &od->lflow_dep_mgr, OBJDEP_TYPE_OD, od->nbs->name, od); const char *dns_action = "eth.dst <-> eth.src; ip4.src <-> ip4.dst; " "udp.dst = udp.src; udp.src = 53; outport = inport; " "flags.loopback = 1; output;"; const char *dns_match = "udp.dst == 53 && "REGBIT_DNS_LOOKUP_RESULT; - ovn_lflow_add(lflows, od, S_SWITCH_IN_DNS_RESPONSE, 100, - dns_match, dns_action); + ovn_lflow_add_with_od_lflow_ref(lflows, od, S_SWITCH_IN_DNS_RESPONSE, 100, + dns_match, dns_action, od->nbs->name); dns_action = "eth.dst <-> eth.src; ip6.src <-> ip6.dst; " "udp.dst = udp.src; udp.src = 53; outport = inport; " "flags.loopback = 1; output;"; - ovn_lflow_add(lflows, od, S_SWITCH_IN_DNS_RESPONSE, 100, - dns_match, dns_action); + ovn_lflow_add_with_od_lflow_ref(lflows, od, S_SWITCH_IN_DNS_RESPONSE, 100, + dns_match, dns_action, od->nbs->name); } /* Table 24: External port. Drop ARP request for router ips from @@ -10458,9 +10636,9 @@ build_lswitch_destination_lookup_bmcast(struct ovn_datapath *od, { ovs_assert(od->nbs); - ovn_lflow_add(lflows, od, S_SWITCH_IN_L2_LKUP, 110, + ovn_lflow_add_with_od_lflow_ref(lflows, od, S_SWITCH_IN_L2_LKUP, 110, "eth.dst == $svc_monitor_mac && (tcp || icmp || icmp6)", - "handle_svc_check(inport);"); + "handle_svc_check(inport);", od->nbs->name); struct mcast_switch_info *mcast_sw_info = &od->mcast_info.sw; @@ -10471,27 +10649,33 @@ build_lswitch_destination_lookup_bmcast(struct ovn_datapath *od, ovn_lflow_metered(lflows, od, S_SWITCH_IN_L2_LKUP, 100, "igmp", ds_cstr(actions), copp_meter_get(COPP_IGMP, od->nbs->copp, - meter_groups)); + meter_groups), + &od->lflow_dep_mgr, OBJDEP_TYPE_OD, + od->nbs->name, od); /* Punt MLD traffic to controller. */ ovn_lflow_metered(lflows, od, S_SWITCH_IN_L2_LKUP, 100, "mldv1 || mldv2", ds_cstr(actions), copp_meter_get(COPP_IGMP, od->nbs->copp, - meter_groups)); + meter_groups), + &od->lflow_dep_mgr, OBJDEP_TYPE_OD, + od->nbs->name, od); /* Flood all IP multicast traffic destined to 224.0.0.X to all * ports - RFC 4541, section 2.1.2, item 2. */ - ovn_lflow_add(lflows, od, S_SWITCH_IN_L2_LKUP, 85, - "ip4.mcast && ip4.dst == 224.0.0.0/24", - "outport = \""MC_FLOOD_L2"\"; output;"); + ovn_lflow_add_with_od_lflow_ref(lflows, od, S_SWITCH_IN_L2_LKUP, 85, + "ip4.mcast && ip4.dst == 224.0.0.0/24", + "outport = \""MC_FLOOD_L2"\"; output;", + od->nbs->name); /* Flood all IPv6 multicast traffic destined to reserved * multicast IPs (RFC 4291, 2.7.1). */ - ovn_lflow_add(lflows, od, S_SWITCH_IN_L2_LKUP, 85, - "ip6.mcast_flood", - "outport = \""MC_FLOOD"\"; output;"); + ovn_lflow_add_with_od_lflow_ref(lflows, od, S_SWITCH_IN_L2_LKUP, 85, + "ip6.mcast_flood", + "outport = \""MC_FLOOD"\"; output;", + od->nbs->name); /* Forward uregistered IP multicast to routers with relay enabled * and to any ports configured to flood IP multicast traffic. @@ -10521,21 +10705,24 @@ build_lswitch_destination_lookup_bmcast(struct ovn_datapath *od, ds_put_cstr(actions, debug_drop_action()); } - ovn_lflow_add(lflows, od, S_SWITCH_IN_L2_LKUP, 80, - "ip4.mcast || ip6.mcast", - ds_cstr(actions)); + ovn_lflow_add_with_od_lflow_ref(lflows, od, S_SWITCH_IN_L2_LKUP, + 80, "ip4.mcast || ip6.mcast", + ds_cstr(actions), od->nbs->name); } } if (!smap_get_bool(&od->nbs->other_config, "broadcast-arps-to-all-routers", true)) { - ovn_lflow_add(lflows, od, S_SWITCH_IN_L2_LKUP, 72, - "eth.mcast && (arp.op == 1 || nd_ns)", - "outport = \""MC_FLOOD_L2"\"; output;"); + ovn_lflow_add_with_od_lflow_ref(lflows, od, S_SWITCH_IN_L2_LKUP, 72, + "eth.mcast && (arp.op == 1 || nd_ns)", + "outport = \""MC_FLOOD_L2"\"; output;", + od->nbs->name); } - ovn_lflow_add(lflows, od, S_SWITCH_IN_L2_LKUP, 70, "eth.mcast", - "outport = \""MC_FLOOD"\"; output;"); + ovn_lflow_add_with_od_lflow_ref(lflows, od, S_SWITCH_IN_L2_LKUP, 70, + "eth.mcast", + "outport = \""MC_FLOOD"\"; output;", + od->nbs->name); } @@ -10616,8 +10803,10 @@ build_lswitch_ip_mcast_igmp_mld(struct ovn_igmp_group *igmp_group, ds_put_format(actions, "outport = \"%s\"; output; ", igmp_group->mcgroup.name); - ovn_lflow_add(lflows, igmp_group->datapath, S_SWITCH_IN_L2_LKUP, - 90, ds_cstr(match), ds_cstr(actions)); + ovn_lflow_add(lflows, igmp_group->datapath, S_SWITCH_IN_L2_LKUP, 90, + ds_cstr(match), ds_cstr(actions), + &igmp_group->lflow_dep_mgr, OBJDEP_TYPE_IGMP, + igmp_group->mcgroup.name, igmp_group->datapath); } } @@ -11075,7 +11264,9 @@ build_routing_policy_flow(struct lflow_data *lflows, struct ovn_datapath *od, ds_put_format(&match, "%s", rule->match); ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_POLICY, rule->priority, - ds_cstr(&match), ds_cstr(&actions), stage_hint); + ds_cstr(&match), ds_cstr(&actions), stage_hint, + &od->lflow_dep_mgr, OBJDEP_TYPE_OD, + od->nbr->name, od); ds_destroy(&match); ds_destroy(&actions); } @@ -11157,7 +11348,9 @@ build_ecmp_routing_policy_flows(struct lflow_data *lflows, ecmp_group_id, i + 1); ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_POLICY_ECMP, 100, ds_cstr(&match), - ds_cstr(&actions), &rule->header_); + ds_cstr(&actions), &rule->header_, + &od->lflow_dep_mgr, OBJDEP_TYPE_OD, + od->nbr->name, od); } ds_clear(&actions); @@ -11175,7 +11368,9 @@ build_ecmp_routing_policy_flows(struct lflow_data *lflows, ds_put_cstr(&actions, ");"); ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_POLICY, rule->priority, rule->match, - ds_cstr(&actions), &rule->header_); + ds_cstr(&actions), &rule->header_, + &od->lflow_dep_mgr, OBJDEP_TYPE_OD, + od->nbr->name, od); cleanup: ds_destroy(&match); @@ -11235,8 +11430,9 @@ build_route_table_lflow(struct ovn_datapath *od, struct lflow_data *lflows, ds_put_format(&actions, "%s = %d; next;", REG_ROUTE_TABLE_ID, rtb_id); - ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_ROUTING_PRE, 100, - ds_cstr(&match), ds_cstr(&actions)); + ovn_lflow_add_with_od_lflow_ref(lflows, od, S_ROUTER_IN_IP_ROUTING_PRE, + 100, ds_cstr(&match), ds_cstr(&actions), + od->nbr->name); ds_destroy(&match); ds_destroy(&actions); @@ -11659,13 +11855,15 @@ add_ecmp_symmetric_reply_flows(struct lflow_data *lflows, ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_DEFRAG, 100, ds_cstr(&base_match), REGBIT_KNOWN_ECMP_NH" = chk_ecmp_nh_mac(); ct_next;", - &st_route->header_); + &st_route->header_, &od->lflow_dep_mgr, OBJDEP_TYPE_OD, + od->nbr->name, od); /* And packets that go out over an ECMP route need conntrack */ ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_DEFRAG, 100, ds_cstr(route_match), REGBIT_KNOWN_ECMP_NH" = chk_ecmp_nh(); ct_next;", - &st_route->header_); + &st_route->header_, &od->lflow_dep_mgr, OBJDEP_TYPE_OD, + od->nbr->name, od); /* Save src eth and inport in ct_label for packets that arrive over * an ECMP route. @@ -11683,7 +11881,9 @@ add_ecmp_symmetric_reply_flows(struct lflow_data *lflows, IN6_IS_ADDR_V4MAPPED(&route->prefix) ? "false" : "true"); ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_ECMP_STATEFUL, 100, ds_cstr(&match), ds_cstr(&actions), - &st_route->header_); + &st_route->header_, + &od->lflow_dep_mgr, OBJDEP_TYPE_OD, + od->nbr->name, od); ds_clear(&match); ds_put_format(&match, "%s && (ct.new && !ct.est) && udp", ds_cstr(&base_match)); @@ -11696,7 +11896,9 @@ add_ecmp_symmetric_reply_flows(struct lflow_data *lflows, IN6_IS_ADDR_V4MAPPED(&route->prefix) ? "false" : "true"); ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_ECMP_STATEFUL, 100, ds_cstr(&match), ds_cstr(&actions), - &st_route->header_); + &st_route->header_, + &od->lflow_dep_mgr, OBJDEP_TYPE_OD, + od->nbr->name, od); ds_clear(&match); ds_put_format(&match, "%s && (ct.new && !ct.est) && sctp", ds_cstr(&base_match)); @@ -11709,7 +11911,9 @@ add_ecmp_symmetric_reply_flows(struct lflow_data *lflows, IN6_IS_ADDR_V4MAPPED(&route->prefix) ? "false" : "true"); ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_ECMP_STATEFUL, 100, ds_cstr(&match), ds_cstr(&actions), - &st_route->header_); + &st_route->header_, + &od->lflow_dep_mgr, OBJDEP_TYPE_OD, + od->nbr->name, od); ds_clear(&match); ds_put_format(&match, @@ -11724,7 +11928,9 @@ add_ecmp_symmetric_reply_flows(struct lflow_data *lflows, IN6_IS_ADDR_V4MAPPED(&route->prefix) ? "false" : "true"); ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_ECMP_STATEFUL, 100, ds_cstr(&match), ds_cstr(&actions), - &st_route->header_); + &st_route->header_, + &od->lflow_dep_mgr, OBJDEP_TYPE_OD, + od->nbr->name, od); ds_clear(&match); ds_put_format(&match, @@ -11739,7 +11945,9 @@ add_ecmp_symmetric_reply_flows(struct lflow_data *lflows, IN6_IS_ADDR_V4MAPPED(&route->prefix) ? "false" : "true"); ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_ECMP_STATEFUL, 100, ds_cstr(&match), ds_cstr(&actions), - &st_route->header_); + &st_route->header_, + &od->lflow_dep_mgr, OBJDEP_TYPE_OD, + od->nbr->name, od); ds_clear(&match); ds_put_format(&match, "%s && (!ct.rpl && ct.est) && sctp && "REGBIT_KNOWN_ECMP_NH" == 0", @@ -11753,7 +11961,9 @@ add_ecmp_symmetric_reply_flows(struct lflow_data *lflows, IN6_IS_ADDR_V4MAPPED(&route->prefix) ? "false" : "true"); ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_ECMP_STATEFUL, 100, ds_cstr(&match), ds_cstr(&actions), - &st_route->header_); + &st_route->header_, + &od->lflow_dep_mgr, OBJDEP_TYPE_OD, + od->nbr->name, od); /* Bypass ECMP selection if we already have ct_label information * for where to route the packet. @@ -11771,13 +11981,16 @@ add_ecmp_symmetric_reply_flows(struct lflow_data *lflows, IN6_IS_ADDR_V4MAPPED(&route->prefix) ? "" : "xx", port_ip, out_port->json_key); ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_IP_ROUTING, 10300, - ds_cstr(&match), ds_cstr(&actions), - &st_route->header_); + ds_cstr(&match), ds_cstr(&actions), + &st_route->header_, + &od->lflow_dep_mgr, OBJDEP_TYPE_OD, + od->nbr->name, od); /* Egress reply traffic for symmetric ECMP routes skips router policies. */ ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_POLICY, 65535, ds_cstr(&ecmp_reply), "next;", - &st_route->header_); + &st_route->header_, &od->lflow_dep_mgr, + OBJDEP_TYPE_OD, od->nbr->name, od); /* Use REG_ECMP_ETH_FULL to pass the eth field from ct_label to eth.dst to * avoid masked access to ct_label. Otherwise it may prevent OVS flow @@ -11793,7 +12006,9 @@ add_ecmp_symmetric_reply_flows(struct lflow_data *lflows, " pop(" REG_ECMP_ETH_FULL "); next;"; ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_ARP_RESOLVE, 200, ds_cstr(&ecmp_reply), - action, &st_route->header_); + action, &st_route->header_, + &od->lflow_dep_mgr, OBJDEP_TYPE_OD, + od->nbr->name, od); ds_destroy(&base_match); ds_destroy(&match); @@ -11836,8 +12051,9 @@ build_ecmp_route_flow(struct lflow_data *lflows, struct ovn_datapath *od, ds_put_cstr(&actions, ");"); - ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_ROUTING, priority, - ds_cstr(&route_match), ds_cstr(&actions)); + ovn_lflow_add_with_od_lflow_ref(lflows, od, S_ROUTER_IN_IP_ROUTING, + priority, ds_cstr(&route_match), + ds_cstr(&actions), od->nbr->name); /* Add per member flow */ struct ds match = DS_EMPTY_INITIALIZER; @@ -11880,7 +12096,9 @@ build_ecmp_route_flow(struct lflow_data *lflows, struct ovn_datapath *od, out_port->json_key); ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_IP_ROUTING_ECMP, 100, ds_cstr(&match), ds_cstr(&actions), - &route->header_); + &route->header_, + &od->lflow_dep_mgr, OBJDEP_TYPE_OD, + od->nbr->name, od); } sset_destroy(&visited_ports); ds_destroy(&match); @@ -11894,7 +12112,9 @@ add_route(struct lflow_data *lflows, struct ovn_datapath *od, const char *network_s, int plen, const char *gateway, bool is_src_route, const uint32_t rtb_id, const struct ovsdb_idl_row *stage_hint, bool is_discard_route, - int ofs) + int ofs, struct objdep_mgr *lflow_dep_mgr, + enum objdep_type res_type, const char *res_name, + struct ovn_datapath *res_od) { bool is_ipv4 = strchr(network_s, '.') ? true : false; struct ds match = DS_EMPTY_INITIALIZER; @@ -11939,12 +12159,16 @@ add_route(struct lflow_data *lflows, struct ovn_datapath *od, ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_IP_ROUTING, priority, ds_cstr(&match), ds_cstr(&actions), - stage_hint); + stage_hint, + lflow_dep_mgr, res_type, + res_name, res_od); if (op && op->has_bfd) { ds_put_format(&match, " && udp.dst == 3784"); ovn_lflow_add_with_hint(lflows, op->od, S_ROUTER_IN_IP_ROUTING, priority + 1, ds_cstr(&match), - ds_cstr(&common_actions), stage_hint); + ds_cstr(&common_actions), stage_hint, + lflow_dep_mgr, res_type, + res_name, res_od); } ds_destroy(&match); ds_destroy(&common_actions); @@ -11978,7 +12202,8 @@ build_static_route_flow(struct lflow_data *lflows, struct ovn_datapath *od, add_route(lflows, route_->is_discard_route ? od : out_port->od, out_port, lrp_addr_s, prefix_s, route_->plen, route->nexthop, route_->is_src_route, route_->route_table_id, &route->header_, - route_->is_discard_route, ofs); + route_->is_discard_route, ofs, + &od->lflow_dep_mgr, OBJDEP_TYPE_OD, od->nbr->name, od); free(prefix_s); } @@ -12075,7 +12300,10 @@ lrouter_use_common_zone(const struct ovn_datapath *od) static void build_distr_lrouter_nat_flows_for_lb(struct lrouter_nat_lb_flows_ctx *ctx, enum lrouter_nat_lb_flow_type type, - struct ovn_datapath *od) + struct ovn_datapath *od, + struct objdep_mgr *lflow_dep_mgr, + enum objdep_type res_type, + const char *res_name) { struct ovn_port *dgp = od->l3dgw_ports[0]; @@ -12114,7 +12342,8 @@ build_distr_lrouter_nat_flows_for_lb(struct lrouter_nat_lb_flows_ctx *ctx, ovn_lflow_add_with_hint__(ctx->lflows, od, S_ROUTER_IN_DNAT, ctx->prio, ds_cstr(ctx->new_match), ctx->new_action[type], - NULL, meter, &ctx->lb->nlb->header_); + NULL, meter, &ctx->lb->nlb->header_, + lflow_dep_mgr, res_type, res_name, NULL); ds_truncate(ctx->new_match, new_match_len); @@ -12133,7 +12362,8 @@ build_distr_lrouter_nat_flows_for_lb(struct lrouter_nat_lb_flows_ctx *ctx, ovn_lflow_add_with_hint(ctx->lflows, od, S_ROUTER_IN_GW_REDIRECT, 200, ds_cstr(ctx->undnat_match), ds_cstr(ctx->gw_redir_action), - &ctx->lb->nlb->header_); + &ctx->lb->nlb->header_, + lflow_dep_mgr, res_type, res_name, NULL); ds_truncate(ctx->undnat_match, undnat_match_len); ds_put_format(ctx->undnat_match, ") && (inport == %s || outport == %s)" @@ -12141,7 +12371,8 @@ build_distr_lrouter_nat_flows_for_lb(struct lrouter_nat_lb_flows_ctx *ctx, dgp->cr_port->json_key); ovn_lflow_add_with_hint(ctx->lflows, od, S_ROUTER_OUT_UNDNAT, 120, ds_cstr(ctx->undnat_match), undnat_action, - &ctx->lb->nlb->header_); + &ctx->lb->nlb->header_, + lflow_dep_mgr, res_type, res_name, NULL); ds_truncate(ctx->undnat_match, undnat_match_len); } @@ -12149,7 +12380,9 @@ static void build_gw_lrouter_nat_flows_for_lb(struct lrouter_nat_lb_flows_ctx *ctx, enum lrouter_nat_lb_flow_type type, const struct ovn_datapaths *lr_datapaths, - const unsigned long *dp_bitmap) + const unsigned long *dp_bitmap, + struct objdep_mgr *lflow_dep_mgr, + const char *lb_name) { unsigned long *dp_non_meter = NULL; bool build_non_meter = false; @@ -12175,14 +12408,16 @@ build_gw_lrouter_nat_flows_for_lb(struct lrouter_nat_lb_flows_ctx *ctx, bitmap_set0(dp_non_meter, index); ovn_lflow_add_with_hint__(ctx->lflows, od, S_ROUTER_IN_DNAT, ctx->prio, ds_cstr(ctx->new_match), ctx->new_action[type], - NULL, meter, &ctx->lb->nlb->header_); + NULL, meter, &ctx->lb->nlb->header_, + lflow_dep_mgr, OBJDEP_TYPE_LB, lb_name, NULL); } } if (!ctx->reject || build_non_meter) { ovn_lflow_add_with_dp_group(ctx->lflows, dp_non_meter ? dp_non_meter : dp_bitmap, ods_size(lr_datapaths), S_ROUTER_IN_DNAT, ctx->prio, ds_cstr(ctx->new_match), - ctx->new_action[type], &ctx->lb->nlb->header_); + ctx->new_action[type], &ctx->lb->nlb->header_, + lflow_dep_mgr, OBJDEP_TYPE_LB, lb_name, NULL); } bitmap_free(dp_non_meter); } @@ -12307,7 +12542,10 @@ build_lrouter_nat_flows_for_lb(struct ovn_lb_vip *lb_vip, if (!od->n_l3dgw_ports) { bitmap_set1(dp_bitmap[type], index); } else { - build_distr_lrouter_nat_flows_for_lb(&ctx, type, od); + build_distr_lrouter_nat_flows_for_lb(&ctx, type, od, + &lb_dps->lflow_dep_mgr, + OBJDEP_TYPE_LB, + lb_dps->lb->nlb->name); } if (lb->affinity_timeout) { @@ -12333,13 +12571,17 @@ build_lrouter_nat_flows_for_lb(struct ovn_lb_vip *lb_vip, * S_ROUTER_IN_DNAT stage. */ ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_UNSNAT, 120, ds_cstr(&unsnat_match), "next;", - &lb->nlb->header_); + &lb->nlb->header_, + &lb_dps->lflow_dep_mgr, OBJDEP_TYPE_LB, + lb_dps->lb->nlb->name, NULL); } } for (size_t type = 0; type < LROUTER_NAT_LB_FLOW_MAX; type++) { build_gw_lrouter_nat_flows_for_lb(&ctx, type, lr_datapaths, - dp_bitmap[type]); + dp_bitmap[type], + &lb_dps->lflow_dep_mgr, + lb_dps->lb->nlb->name); } /* LB affinity flows for datapaths where CMS has specified @@ -12348,7 +12590,7 @@ build_lrouter_nat_flows_for_lb(struct ovn_lb_vip *lb_vip, build_lb_affinity_lr_flows(lflows, lb, lb_vip, ds_cstr(match), "flags.force_snat_for_lb = 1; ", dp_bitmap[LROUTER_NAT_LB_AFF_FORCE_SNAT], - lr_datapaths); + lr_datapaths, &lb_dps->lflow_dep_mgr); /* LB affinity flows for datapaths where CMS has specified * skip_snat_for_lb floag option or regular datapaths. @@ -12357,7 +12599,7 @@ build_lrouter_nat_flows_for_lb(struct ovn_lb_vip *lb_vip, lb->skip_snat ? "flags.skip_snat_for_lb = 1; " : NULL; build_lb_affinity_lr_flows(lflows, lb, lb_vip, ds_cstr(match), lb_aff_action, dp_bitmap[LROUTER_NAT_LB_AFF], - lr_datapaths); + lr_datapaths, &lb_dps->lflow_dep_mgr); ds_destroy(&unsnat_match); ds_destroy(&undnat_match); @@ -12403,7 +12645,9 @@ build_lswitch_flows_for_lb(struct ovn_lb_datapaths *lb_dps, copp_meter_get(COPP_EVENT_ELB, od->nbs->copp, meter_groups), - &lb->nlb->header_); + &lb->nlb->header_, + &lb_dps->lflow_dep_mgr, OBJDEP_TYPE_LB, + lb->nlb->name, NULL); } /* Ignore L4 port information in the key because fragmented packets * may not have L4 information. The pre-stateful table will send @@ -12453,7 +12697,9 @@ build_lrouter_defrag_flows_for_lb(struct ovn_lb_datapaths *lb_dps, ovn_lflow_add_with_dp_group( lflows, lb_dps->nb_lr_map, ods_size(lr_datapaths), S_ROUTER_IN_DEFRAG, prio, ds_cstr(match), "ct_dnat;", - &lb_dps->lb->nlb->header_); + &lb_dps->lb->nlb->header_, + &lb_dps->lflow_dep_mgr, OBJDEP_TYPE_LB, + lb_dps->lb->nlb->name, NULL); } } @@ -12494,7 +12740,9 @@ build_lrouter_flows_for_lb(struct ovn_lb_datapaths *lb_dps, copp_meter_get(COPP_EVENT_ELB, od->nbr->copp, meter_groups), - &lb->nlb->header_); + &lb->nlb->header_, + &lb_dps->lflow_dep_mgr, OBJDEP_TYPE_LB, + lb->nlb->name, NULL); } } @@ -12503,7 +12751,9 @@ build_lrouter_flows_for_lb(struct ovn_lb_datapaths *lb_dps, struct ovn_datapath *od = lr_datapaths->array[index]; ovn_lflow_add(lflows, od, S_ROUTER_OUT_SNAT, 120, - "flags.skip_snat_for_lb == 1 && ip", "next;"); + "flags.skip_snat_for_lb == 1 && ip", "next;", + &lb_dps->lflow_dep_mgr, OBJDEP_TYPE_LB, + lb->nlb->name, NULL); } } } @@ -12669,7 +12919,8 @@ lrouter_nat_add_ext_ip_match(struct ovn_datapath *od, ovn_lflow_add_with_hint(lflows, od, stage, priority, ds_cstr(&match_exempt), "next;", - &nat->header_); + &nat->header_, &od->lflow_dep_mgr, + OBJDEP_TYPE_OD, od->nbr->name, od); ds_destroy(&match_exempt); } } @@ -12683,7 +12934,10 @@ build_lrouter_arp_flow(struct ovn_datapath *od, struct ovn_port *op, const char *ip_address, const char *eth_addr, struct ds *extra_match, bool drop, uint16_t priority, const struct ovsdb_idl_row *hint, - struct lflow_data *lflows) + struct lflow_data *lflows, + struct objdep_mgr *lflow_dep_mgr, + enum objdep_type res_type, const char *res_name, + struct ovn_datapath *res_od) { struct ds match = DS_EMPTY_INITIALIZER; struct ds actions = DS_EMPTY_INITIALIZER; @@ -12715,7 +12969,9 @@ build_lrouter_arp_flow(struct ovn_datapath *od, struct ovn_port *op, } ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_IP_INPUT, priority, - ds_cstr(&match), ds_cstr(&actions), hint); + ds_cstr(&match), ds_cstr(&actions), hint, + lflow_dep_mgr, res_type, + res_name, res_od); ds_destroy(&match); ds_destroy(&actions); @@ -12734,7 +12990,10 @@ build_lrouter_nd_flow(struct ovn_datapath *od, struct ovn_port *op, struct ds *extra_match, bool drop, uint16_t priority, const struct ovsdb_idl_row *hint, struct lflow_data *lflows, - const struct shash *meter_groups) + const struct shash *meter_groups, + struct objdep_mgr *lflow_dep_mgr, + enum objdep_type res_type, const char *res_name, + struct ovn_datapath *res_od) { struct ds match = DS_EMPTY_INITIALIZER; struct ds actions = DS_EMPTY_INITIALIZER; @@ -12757,7 +13016,9 @@ build_lrouter_nd_flow(struct ovn_datapath *od, struct ovn_port *op, if (drop) { ds_put_cstr(&actions, debug_drop_action()); ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_IP_INPUT, priority, - ds_cstr(&match), ds_cstr(&actions), hint); + ds_cstr(&match), ds_cstr(&actions), hint, + lflow_dep_mgr, res_type, + res_name, res_od); } else { ds_put_format(&actions, "%s { " @@ -12775,7 +13036,8 @@ build_lrouter_nd_flow(struct ovn_datapath *od, struct ovn_port *op, ds_cstr(&match), ds_cstr(&actions), NULL, copp_meter_get(COPP_ND_NA, od->nbr->copp, meter_groups), - hint); + hint, lflow_dep_mgr, res_type, + res_name, res_od); } ds_destroy(&match); @@ -12796,12 +13058,16 @@ build_lrouter_nat_arp_nd_flow(struct ovn_datapath *od, ext_addrs->ipv6_addrs[0].addr_s, ext_addrs->ipv6_addrs[0].sn_addr_s, REG_INPORT_ETH_ADDR, NULL, false, 90, - &nat->header_, lflows, meter_groups); + &nat->header_, lflows, meter_groups, + &od->lflow_dep_mgr, OBJDEP_TYPE_OD, + od->nbr->name, od); } else { build_lrouter_arp_flow(od, NULL, ext_addrs->ipv4_addrs[0].addr_s, REG_INPORT_ETH_ADDR, NULL, false, 90, - &nat->header_, lflows); + &nat->header_, lflows, + &od->lflow_dep_mgr, OBJDEP_TYPE_OD, + od->nbr->name, od); } } @@ -12857,21 +13123,29 @@ build_lrouter_port_nat_arp_nd_flow(struct ovn_port *op, ext_addrs->ipv6_addrs[0].addr_s, ext_addrs->ipv6_addrs[0].sn_addr_s, mac_s, &match, false, 92, - &nat->header_, lflows, meter_groups); + &nat->header_, lflows, meter_groups, + &op->lflow_dep_mgr, OBJDEP_TYPE_LPORT, + op->nbrp->name, op->od); build_lrouter_nd_flow(op->od, op, "nd_na", ext_addrs->ipv6_addrs[0].addr_s, ext_addrs->ipv6_addrs[0].sn_addr_s, mac_s, NULL, true, 91, - &nat->header_, lflows, meter_groups); + &nat->header_, lflows, meter_groups, + &op->lflow_dep_mgr, OBJDEP_TYPE_LPORT, + op->nbrp->name, op->od); } else { build_lrouter_arp_flow(op->od, op, ext_addrs->ipv4_addrs[0].addr_s, mac_s, &match, false, 92, - &nat->header_, lflows); + &nat->header_, lflows, + &op->lflow_dep_mgr, OBJDEP_TYPE_LPORT, + op->nbrp->name, op->od); build_lrouter_arp_flow(op->od, op, ext_addrs->ipv4_addrs[0].addr_s, mac_s, NULL, true, 91, - &nat->header_, lflows); + &nat->header_, lflows, + &op->lflow_dep_mgr, OBJDEP_TYPE_LPORT, + op->nbrp->name, op->od); } ds_destroy(&match); @@ -12906,7 +13180,11 @@ build_lrouter_drop_own_dest(struct ovn_port *op, enum ovn_stage stage, char *match = xasprintf("ip4.dst == {%s}", ds_cstr(&match_ips)); ovn_lflow_add_with_hint(lflows, op->od, stage, priority, match, debug_drop_action(), - &op->nbrp->header_); + &op->nbrp->header_, + &op->lflow_dep_mgr, + OBJDEP_TYPE_LPORT, + op->nbrp->name, + op->od); free(match); } } @@ -12935,7 +13213,11 @@ build_lrouter_drop_own_dest(struct ovn_port *op, enum ovn_stage stage, char *match = xasprintf("ip6.dst == {%s}", ds_cstr(&match_ips)); ovn_lflow_add_with_hint(lflows, op->od, stage, priority, match, debug_drop_action(), - &op->nbrp->header_); + &op->nbrp->header_, + &op->lflow_dep_mgr, + OBJDEP_TYPE_LPORT, + op->nbrp->name, + op->od); free(match); } } @@ -12946,14 +13228,16 @@ static void build_lrouter_force_snat_flows(struct lflow_data *lflows, struct ovn_datapath *od, const char *ip_version, const char *ip_addr, - const char *context) + const char *context, + enum objdep_type objdep_type) { struct ds match = DS_EMPTY_INITIALIZER; struct ds actions = DS_EMPTY_INITIALIZER; ds_put_format(&match, "ip%s && ip%s.dst == %s", ip_version, ip_version, ip_addr); - ovn_lflow_add(lflows, od, S_ROUTER_IN_UNSNAT, 110, - ds_cstr(&match), "ct_snat;"); + ovn_lflow_add(lflows, od, S_ROUTER_IN_UNSNAT, 110, ds_cstr(&match), + "ct_snat;", &od->lflow_dep_mgr, objdep_type, + od->nbr->name, od); /* Higher priority rules to force SNAT with the IP addresses * configured in the Gateway router. This only takes effect @@ -12962,8 +13246,9 @@ build_lrouter_force_snat_flows(struct lflow_data *lflows, ds_put_format(&match, "flags.force_snat_for_%s == 1 && ip%s", context, ip_version); ds_put_format(&actions, "ct_snat(%s);", ip_addr); - ovn_lflow_add(lflows, od, S_ROUTER_OUT_SNAT, 100, - ds_cstr(&match), ds_cstr(&actions)); + ovn_lflow_add(lflows, od, S_ROUTER_OUT_SNAT, 100, ds_cstr(&match), + ds_cstr(&actions), &od->lflow_dep_mgr, objdep_type, + od->nbr->name, od); ds_destroy(&match); ds_destroy(&actions); @@ -12985,8 +13270,10 @@ build_lrouter_force_snat_flows_op(struct ovn_port *op, ds_put_format(match, "inport == %s && ip4.dst == %s", op->json_key, op->lrp_networks.ipv4_addrs[0].addr_s); - ovn_lflow_add(lflows, op->od, S_ROUTER_IN_UNSNAT, 110, - ds_cstr(match), "ct_snat;"); + ovn_lflow_add(lflows, op->od, S_ROUTER_IN_UNSNAT, + 110, ds_cstr(match), "ct_snat;", + &op->lflow_dep_mgr, OBJDEP_TYPE_LPORT, + op->nbrp->name, op->od); ds_clear(match); @@ -12998,7 +13285,9 @@ build_lrouter_force_snat_flows_op(struct ovn_port *op, ds_put_format(actions, "ct_snat(%s);", op->lrp_networks.ipv4_addrs[0].addr_s); ovn_lflow_add(lflows, op->od, S_ROUTER_OUT_SNAT, 110, - ds_cstr(match), ds_cstr(actions)); + ds_cstr(match), ds_cstr(actions), + &op->lflow_dep_mgr, OBJDEP_TYPE_LPORT, + op->nbrp->name, op->od); if (op->lrp_networks.n_ipv4_addrs > 1) { static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 5); VLOG_WARN_RL(&rl, "Logical router port %s is configured with " @@ -13017,8 +13306,10 @@ build_lrouter_force_snat_flows_op(struct ovn_port *op, ds_put_format(match, "inport == %s && ip6.dst == %s", op->json_key, op->lrp_networks.ipv6_addrs[0].addr_s); - ovn_lflow_add(lflows, op->od, S_ROUTER_IN_UNSNAT, 110, - ds_cstr(match), "ct_snat;"); + ovn_lflow_add(lflows, op->od, S_ROUTER_IN_UNSNAT, + 110, ds_cstr(match), "ct_snat;", + &op->lflow_dep_mgr, OBJDEP_TYPE_LPORT, + op->nbrp->name, op->od); ds_clear(match); @@ -13030,7 +13321,9 @@ build_lrouter_force_snat_flows_op(struct ovn_port *op, ds_put_format(actions, "ct_snat(%s);", op->lrp_networks.ipv6_addrs[0].addr_s); ovn_lflow_add(lflows, op->od, S_ROUTER_OUT_SNAT, 110, - ds_cstr(match), ds_cstr(actions)); + ds_cstr(match), ds_cstr(actions), + &op->lflow_dep_mgr, OBJDEP_TYPE_LPORT, + op->nbrp->name, op->od); if (op->lrp_networks.n_ipv6_addrs > 2) { static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 5); VLOG_WARN_RL(&rl, "Logical router port %s is configured with " @@ -13059,7 +13352,9 @@ build_lrouter_bfd_flows(struct lflow_data *lflows, struct ovn_port *op, ds_cstr(&ip_list)); ovn_lflow_add_with_hint(lflows, op->od, S_ROUTER_IN_IP_INPUT, 110, ds_cstr(&match), "next; ", - &op->nbrp->header_); + &op->nbrp->header_, + &op->lflow_dep_mgr, OBJDEP_TYPE_LPORT, + op->nbrp->name, op->od); ds_clear(&match); ds_put_format(&match, "ip4.dst == %s && udp.dst == 3784", ds_cstr(&ip_list)); @@ -13067,7 +13362,9 @@ build_lrouter_bfd_flows(struct lflow_data *lflows, struct ovn_port *op, ds_cstr(&match), "handle_bfd_msg(); ", NULL, copp_meter_get(COPP_BFD, op->od->nbr->copp, meter_groups), - &op->nbrp->header_); + &op->nbrp->header_, + &op->lflow_dep_mgr, OBJDEP_TYPE_LPORT, + op->nbrp->name, op->od); } if (op->lrp_networks.n_ipv6_addrs) { ds_clear(&ip_list); @@ -13078,7 +13375,9 @@ build_lrouter_bfd_flows(struct lflow_data *lflows, struct ovn_port *op, ds_cstr(&ip_list)); ovn_lflow_add_with_hint(lflows, op->od, S_ROUTER_IN_IP_INPUT, 110, ds_cstr(&match), "next; ", - &op->nbrp->header_); + &op->nbrp->header_, + &op->lflow_dep_mgr, OBJDEP_TYPE_LPORT, + op->nbrp->name, op->od); ds_clear(&match); ds_put_format(&match, "ip6.dst == %s && udp.dst == 3784", ds_cstr(&ip_list)); @@ -13086,7 +13385,9 @@ build_lrouter_bfd_flows(struct lflow_data *lflows, struct ovn_port *op, ds_cstr(&match), "handle_bfd_msg(); ", NULL, copp_meter_get(COPP_BFD, op->od->nbr->copp, meter_groups), - &op->nbrp->header_); + &op->nbrp->header_, + &op->lflow_dep_mgr, OBJDEP_TYPE_LPORT, + op->nbrp->name, op->od); } ds_destroy(&ip_list); @@ -13103,11 +13404,14 @@ build_adm_ctrl_flows_for_lrouter( ovs_assert(od->nbr); /* Logical VLANs not supported. * Broadcast/multicast source address is invalid. */ - ovn_lflow_add(lflows, od, S_ROUTER_IN_ADMISSION, 100, - "vlan.present || eth.src[40]", debug_drop_action()); + ovn_lflow_add_with_od_lflow_ref(lflows, od, S_ROUTER_IN_ADMISSION, 100, + "vlan.present || eth.src[40]", + debug_drop_action(), od->nbr->name); /* Default action for L2 security is to drop. */ - ovn_lflow_add_default_drop(lflows, od, S_ROUTER_IN_ADMISSION); + ovn_lflow_add_default_drop(lflows, od, S_ROUTER_IN_ADMISSION, + &od->lflow_dep_mgr, OBJDEP_TYPE_OD, + od->nbr->name, od); } static int @@ -13135,11 +13439,14 @@ build_gateway_get_l2_hdr_size(struct ovn_port *op) /* All 'gateway_mtu' and 'gateway_mtu_bypass' flows should be built with this * function. */ -static void OVS_PRINTF_FORMAT(9, 10) +static void OVS_PRINTF_FORMAT(13, 14) build_gateway_mtu_flow(struct lflow_data *lflows, struct ovn_port *op, enum ovn_stage stage, uint16_t prio_low, uint16_t prio_high, struct ds *match, struct ds *actions, const struct ovsdb_idl_row *hint, + struct objdep_mgr *lflow_dep_mgr, + enum objdep_type res_type, const char *res_name, + struct ovn_datapath *res_od, const char *extra_actions_fmt, ...) { int gw_mtu = smap_get_int(&op->nbrp->options, "gateway_mtu", 0); @@ -13157,7 +13464,8 @@ build_gateway_mtu_flow(struct lflow_data *lflows, struct ovn_port *op, ds_put_format_valist(actions, extra_actions_fmt, extra_actions_args); ovn_lflow_add_with_hint(lflows, op->od, stage, prio_low, ds_cstr(match), ds_cstr(actions), - hint); + hint, lflow_dep_mgr, res_type, + res_name, res_od); if (gw_mtu > 0) { const char *gw_mtu_bypass = smap_get(&op->nbrp->options, @@ -13169,7 +13477,8 @@ build_gateway_mtu_flow(struct lflow_data *lflows, struct ovn_port *op, ds_put_format(match, " && (%s)", gw_mtu_bypass); ovn_lflow_add_with_hint(lflows, op->od, stage, prio_high, ds_cstr(match), ds_cstr(actions), - hint); + hint, lflow_dep_mgr, res_type, + res_name, res_od); } } va_end(extra_actions_args); @@ -13222,6 +13531,8 @@ build_adm_ctrl_flows_for_lrouter_port( ds_put_format(match, "eth.mcast && inport == %s", op->json_key); build_gateway_mtu_flow(lflows, op, S_ROUTER_IN_ADMISSION, 50, 55, match, actions, &op->nbrp->header_, + &op->lflow_dep_mgr, OBJDEP_TYPE_LPORT, + op->nbrp->name, op->od, REG_INPORT_ETH_ADDR " = %s; next;", op->lrp_networks.ea_s); @@ -13242,6 +13553,8 @@ build_adm_ctrl_flows_for_lrouter_port( } build_gateway_mtu_flow(lflows, op, S_ROUTER_IN_ADMISSION, 50, 55, match, actions, &op->nbrp->header_, + &op->lflow_dep_mgr, OBJDEP_TYPE_LPORT, + op->nbrp->name, op->od, REG_INPORT_ETH_ADDR " = %s; next;", op->lrp_networks.ea_s); } @@ -13299,16 +13612,18 @@ build_neigh_learning_flows_for_lrouter( " = lookup_arp(inport, arp.spa, arp.sha); %snext;", learn_from_arp_request ? "" : REGBIT_LOOKUP_NEIGHBOR_IP_RESULT" = 1; "); - ovn_lflow_add(lflows, od, S_ROUTER_IN_LOOKUP_NEIGHBOR, 100, - "arp.op == 2", ds_cstr(actions)); + ovn_lflow_add_with_od_lflow_ref(lflows, od, S_ROUTER_IN_LOOKUP_NEIGHBOR, + 100, "arp.op == 2", ds_cstr(actions), + od->nbr->name); ds_clear(actions); ds_put_format(actions, REGBIT_LOOKUP_NEIGHBOR_RESULT " = lookup_nd(inport, nd.target, nd.tll); %snext;", learn_from_arp_request ? "" : REGBIT_LOOKUP_NEIGHBOR_IP_RESULT" = 1; "); - ovn_lflow_add(lflows, od, S_ROUTER_IN_LOOKUP_NEIGHBOR, 100, "nd_na", - ds_cstr(actions)); + ovn_lflow_add_with_od_lflow_ref(lflows, od, S_ROUTER_IN_LOOKUP_NEIGHBOR, + 100, "nd_na", ds_cstr(actions), + od->nbr->name); if (!learn_from_arp_request) { /* Add flow to skip GARP LLA if we don't know it already. @@ -13323,9 +13638,10 @@ build_neigh_learning_flows_for_lrouter( " = lookup_nd(inport, ip6.src, nd.tll); " REGBIT_LOOKUP_NEIGHBOR_IP_RESULT " = lookup_nd_ip(inport, ip6.src); next;"); - ovn_lflow_add(lflows, od, S_ROUTER_IN_LOOKUP_NEIGHBOR, 110, - "nd_na && ip6.src == fe80::/10 && ip6.dst == ff00::/8", - ds_cstr(actions)); + ovn_lflow_add_with_od_lflow_ref( + lflows, od, S_ROUTER_IN_LOOKUP_NEIGHBOR, 110, + "nd_na && ip6.src == fe80::/10 && ip6.dst == ff00::/8", + ds_cstr(actions), od->nbr->name); } ds_clear(actions); @@ -13334,13 +13650,16 @@ build_neigh_learning_flows_for_lrouter( learn_from_arp_request ? "" : REGBIT_LOOKUP_NEIGHBOR_IP_RESULT " = lookup_nd_ip(inport, ip6.src); "); - ovn_lflow_add(lflows, od, S_ROUTER_IN_LOOKUP_NEIGHBOR, 100, "nd_ns", - ds_cstr(actions)); + ovn_lflow_add_with_od_lflow_ref(lflows, od, S_ROUTER_IN_LOOKUP_NEIGHBOR, + 100, "nd_ns", ds_cstr(actions), + od->nbr->name); /* For other packet types, we can skip neighbor learning. * So set REGBIT_LOOKUP_NEIGHBOR_RESULT to 1. */ - ovn_lflow_add(lflows, od, S_ROUTER_IN_LOOKUP_NEIGHBOR, 0, "1", - REGBIT_LOOKUP_NEIGHBOR_RESULT" = 1; next;"); + ovn_lflow_add_with_od_lflow_ref(lflows, od, S_ROUTER_IN_LOOKUP_NEIGHBOR, + 0, "1", + REGBIT_LOOKUP_NEIGHBOR_RESULT" = 1; next;", + od->nbr->name); /* Flows for LEARN_NEIGHBOR. */ /* Skip Neighbor learning if not required. */ @@ -13348,34 +13667,46 @@ build_neigh_learning_flows_for_lrouter( ds_put_format(match, REGBIT_LOOKUP_NEIGHBOR_RESULT" == 1%s", learn_from_arp_request ? "" : " || "REGBIT_LOOKUP_NEIGHBOR_IP_RESULT" == 0"); - ovn_lflow_add(lflows, od, S_ROUTER_IN_LEARN_NEIGHBOR, 100, - ds_cstr(match), "next;"); + ovn_lflow_add_with_od_lflow_ref(lflows, od, S_ROUTER_IN_LEARN_NEIGHBOR, + 100, ds_cstr(match), "next;", + od->nbr->name); ovn_lflow_metered(lflows, od, S_ROUTER_IN_LEARN_NEIGHBOR, 90, "arp", "put_arp(inport, arp.spa, arp.sha); next;", copp_meter_get(COPP_ARP, od->nbr->copp, - meter_groups)); + meter_groups), + &od->lflow_dep_mgr, OBJDEP_TYPE_OD, + od->nbr->name, od); - ovn_lflow_add(lflows, od, S_ROUTER_IN_LEARN_NEIGHBOR, 95, - "nd_ns && (ip6.src == 0 || nd.sll == 0)", "next;"); + ovn_lflow_add_with_od_lflow_ref(lflows, od, S_ROUTER_IN_LEARN_NEIGHBOR, 95, + "nd_ns && (ip6.src == 0 || nd.sll == 0)", + "next;", od->nbr->name); ovn_lflow_metered(lflows, od, S_ROUTER_IN_LEARN_NEIGHBOR, 95, "nd_na && nd.tll == 0", "put_nd(inport, nd.target, eth.src); next;", copp_meter_get(COPP_ND_NA, od->nbr->copp, - meter_groups)); + meter_groups), + &od->lflow_dep_mgr, OBJDEP_TYPE_OD, + od->nbr->name, od); ovn_lflow_metered(lflows, od, S_ROUTER_IN_LEARN_NEIGHBOR, 90, "nd_na", "put_nd(inport, nd.target, nd.tll); next;", copp_meter_get(COPP_ND_NA, od->nbr->copp, - meter_groups)); + meter_groups), + &od->lflow_dep_mgr, OBJDEP_TYPE_OD, + od->nbr->name, od); ovn_lflow_metered(lflows, od, S_ROUTER_IN_LEARN_NEIGHBOR, 90, "nd_ns", "put_nd(inport, ip6.src, nd.sll); next;", copp_meter_get(COPP_ND_NS, od->nbr->copp, - meter_groups)); + meter_groups), + &od->lflow_dep_mgr, OBJDEP_TYPE_OD, + od->nbr->name, od); - ovn_lflow_add_default_drop(lflows, od, S_ROUTER_IN_LEARN_NEIGHBOR); + ovn_lflow_add_default_drop(lflows, od, S_ROUTER_IN_LEARN_NEIGHBOR, + &od->lflow_dep_mgr, OBJDEP_TYPE_OD, + od->nbr->name, od); } /* Logical router ingress Table 1: Neighbor lookup lflows @@ -13415,7 +13746,9 @@ build_neigh_learning_flows_for_lrouter_port( ovn_lflow_add_with_hint(lflows, op->od, S_ROUTER_IN_LOOKUP_NEIGHBOR, 110, ds_cstr(match), actions_s, - &op->nbrp->header_); + &op->nbrp->header_, + &op->lflow_dep_mgr, OBJDEP_TYPE_LPORT, + op->nbrp->name, op->od); } ds_clear(match); ds_put_format(match, @@ -13436,7 +13769,9 @@ build_neigh_learning_flows_for_lrouter_port( ovn_lflow_add_with_hint(lflows, op->od, S_ROUTER_IN_LOOKUP_NEIGHBOR, 100, ds_cstr(match), ds_cstr(actions), - &op->nbrp->header_); + &op->nbrp->header_, + &op->lflow_dep_mgr, OBJDEP_TYPE_LPORT, + op->nbrp->name, op->od); } } @@ -13533,7 +13868,9 @@ build_ND_RA_flows_for_lrouter_port( copp_meter_get(COPP_ND_RA_OPTS, op->od->nbr->copp, meter_groups), - &op->nbrp->header_); + &op->nbrp->header_, + &op->lflow_dep_mgr, OBJDEP_TYPE_LPORT, + op->nbrp->name, op->od); ds_clear(actions); ds_clear(match); ds_put_format(match, "inport == %s && ip6.dst == ff02::2 && " @@ -13552,7 +13889,9 @@ build_ND_RA_flows_for_lrouter_port( ovn_lflow_add_with_hint(lflows, op->od, S_ROUTER_IN_ND_RA_RESPONSE, 50, ds_cstr(match), ds_cstr(actions), - &op->nbrp->header_); + &op->nbrp->header_, + &op->lflow_dep_mgr, OBJDEP_TYPE_LPORT, + op->nbrp->name, op->od); } } @@ -13563,8 +13902,10 @@ build_ND_RA_flows_for_lrouter(struct ovn_datapath *od, struct lflow_data *lflows) { ovs_assert(od->nbr); - ovn_lflow_add(lflows, od, S_ROUTER_IN_ND_RA_OPTIONS, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_ROUTER_IN_ND_RA_RESPONSE, 0, "1", "next;"); + ovn_lflow_add_with_od_lflow_ref(lflows, od, S_ROUTER_IN_ND_RA_OPTIONS, 0, + "1", "next;", od->nbr->name); + ovn_lflow_add_with_od_lflow_ref(lflows, od, S_ROUTER_IN_ND_RA_RESPONSE, 0, + "1", "next;", od->nbr->name); } /* Logical router ingress table IP_ROUTING_PRE: @@ -13574,8 +13915,9 @@ build_ip_routing_pre_flows_for_lrouter(struct ovn_datapath *od, struct lflow_data *lflows) { ovs_assert(od->nbr); - ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_ROUTING_PRE, 0, "1", - REG_ROUTE_TABLE_ID" = 0; next;"); + ovn_lflow_add_with_od_lflow_ref(lflows, od, S_ROUTER_IN_IP_ROUTING_PRE, 0, + "1", REG_ROUTE_TABLE_ID" = 0; next;", + od->nbr->name); } /* Logical router ingress table IP_ROUTING : IP Routing. @@ -13606,14 +13948,18 @@ build_ip_routing_flows_for_lrp( add_route(lflows, op->od, op, op->lrp_networks.ipv4_addrs[i].addr_s, op->lrp_networks.ipv4_addrs[i].network_s, op->lrp_networks.ipv4_addrs[i].plen, NULL, false, 0, - &op->nbrp->header_, false, ROUTE_PRIO_OFFSET_CONNECTED); + &op->nbrp->header_, false, ROUTE_PRIO_OFFSET_CONNECTED, + &op->lflow_dep_mgr, OBJDEP_TYPE_LPORT, + op->nbrp->name, op->od); } for (int i = 0; i < op->lrp_networks.n_ipv6_addrs; i++) { add_route(lflows, op->od, op, op->lrp_networks.ipv6_addrs[i].addr_s, op->lrp_networks.ipv6_addrs[i].network_s, op->lrp_networks.ipv6_addrs[i].plen, NULL, false, 0, - &op->nbrp->header_, false, ROUTE_PRIO_OFFSET_CONNECTED); + &op->nbrp->header_, false, ROUTE_PRIO_OFFSET_CONNECTED, + &op->lflow_dep_mgr, OBJDEP_TYPE_LPORT, + op->nbrp->name, op->od); } } @@ -13655,7 +14001,9 @@ build_ip_routing_flows_for_router_type_lsp(struct ovn_port *op, laddrs->ipv4_addrs[k].network_s, laddrs->ipv4_addrs[k].plen, NULL, false, 0, &peer->nbrp->header_, false, - ROUTE_PRIO_OFFSET_CONNECTED); + ROUTE_PRIO_OFFSET_CONNECTED, + &op->lflow_dep_mgr, OBJDEP_TYPE_LPORT, + op->nbsp->name, op->od); } } } @@ -13668,10 +14016,15 @@ build_static_route_flows_for_lrouter( const struct hmap *bfd_connections) { ovs_assert(od->nbr); - ovn_lflow_add_default_drop(lflows, od, S_ROUTER_IN_IP_ROUTING_ECMP); - ovn_lflow_add_default_drop(lflows, od, S_ROUTER_IN_IP_ROUTING); - ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_ROUTING_ECMP, 150, - REG_ECMP_GROUP_ID" == 0", "next;"); + ovn_lflow_add_default_drop(lflows, od, S_ROUTER_IN_IP_ROUTING_ECMP, + &od->lflow_dep_mgr, OBJDEP_TYPE_OD, + od->nbr->name, od); + ovn_lflow_add_default_drop(lflows, od, S_ROUTER_IN_IP_ROUTING, + &od->lflow_dep_mgr, OBJDEP_TYPE_OD, + od->nbr->name, od); + ovn_lflow_add_with_od_lflow_ref(lflows, od, S_ROUTER_IN_IP_ROUTING_ECMP, + 150, REG_ECMP_GROUP_ID" == 0", "next;", + od->nbr->name); struct hmap ecmp_groups = HMAP_INITIALIZER(&ecmp_groups); struct hmap unique_routes = HMAP_INITIALIZER(&unique_routes); @@ -13736,8 +14089,9 @@ build_mcast_lookup_flows_for_lrouter( /* Drop IPv6 multicast traffic that shouldn't be forwarded, * i.e., router solicitation and router advertisement. */ - ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_ROUTING, 10550, - "nd_rs || nd_ra", debug_drop_action()); + ovn_lflow_add_with_od_lflow_ref(lflows, od, S_ROUTER_IN_IP_ROUTING, 10550, + "nd_rs || nd_ra", debug_drop_action(), + od->nbr->name); if (!od->mcast_info.rtr.relay) { return; } @@ -13764,8 +14118,9 @@ build_mcast_lookup_flows_for_lrouter( } ds_put_format(actions, "outport = \"%s\"; ip.ttl--; next;", igmp_group->mcgroup.name); - ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_ROUTING, 10500, - ds_cstr(match), ds_cstr(actions)); + ovn_lflow_add_with_od_lflow_ref(lflows, od, S_ROUTER_IN_IP_ROUTING, + 10500, ds_cstr(match), + ds_cstr(actions), od->nbr->name); } /* If needed, flood unregistered multicast on statically configured @@ -13783,38 +14138,46 @@ build_mcast_lookup_flows_for_lrouter( ds_clear(match); ds_put_format(match, "eth.src == %s && igmp", op->lrp_networks.ea_s); - ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_ROUTING, 10550, - ds_cstr(match), debug_drop_action()); + ovn_lflow_add_with_od_lflow_ref(lflows, od, S_ROUTER_IN_IP_ROUTING, + 10550, ds_cstr(match), + debug_drop_action(), + od->nbr->name); ds_clear(match); ds_put_format(match, "eth.src == %s && (mldv1 || mldv2)", op->lrp_networks.ea_s); - ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_ROUTING, 10550, - ds_cstr(match), debug_drop_action()); + ovn_lflow_add_with_od_lflow_ref(lflows, od, S_ROUTER_IN_IP_ROUTING, + 10550, ds_cstr(match), + debug_drop_action(), + od->nbr->name); } - ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_ROUTING, 10460, - "igmp", + ovn_lflow_add_with_od_lflow_ref(lflows, od, S_ROUTER_IN_IP_ROUTING, + 10460, "igmp", "clone { " "outport = \""MC_STATIC"\"; " "next; " - "};"); - ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_ROUTING, 10460, - "mldv1 || mldv2", + "};", + od->nbr->name); + ovn_lflow_add_with_od_lflow_ref(lflows, od, S_ROUTER_IN_IP_ROUTING, + 10460, "mldv1 || mldv2", "clone { " "outport = \""MC_STATIC"\"; " "next; " - "};"); - ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_ROUTING, 10450, - "ip4.mcast || ip6.mcast", + "};", + od->nbr->name); + ovn_lflow_add_with_od_lflow_ref(lflows, od, S_ROUTER_IN_IP_ROUTING, + 10450, "ip4.mcast || ip6.mcast", "clone { " "outport = \""MC_STATIC"\"; " "ip.ttl--; " "next; " - "};"); + "};", + od->nbr->name); } else { - ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_ROUTING, 10450, - "ip4.mcast || ip6.mcast", debug_drop_action()); + ovn_lflow_add_with_od_lflow_ref(lflows, od, S_ROUTER_IN_IP_ROUTING, + 10450, "ip4.mcast || ip6.mcast", + debug_drop_action(), od->nbr->name); } } @@ -13835,11 +14198,15 @@ build_ingress_policy_flows_for_lrouter( ovs_assert(od->nbr); /* This is a catch-all rule. It has the lowest priority (0) * does a match-all("1") and pass-through (next) */ - ovn_lflow_add(lflows, od, S_ROUTER_IN_POLICY, 0, "1", - REG_ECMP_GROUP_ID" = 0; next;"); - ovn_lflow_add(lflows, od, S_ROUTER_IN_POLICY_ECMP, 150, - REG_ECMP_GROUP_ID" == 0", "next;"); - ovn_lflow_add_default_drop(lflows, od, S_ROUTER_IN_POLICY_ECMP); + ovn_lflow_add_with_od_lflow_ref(lflows, od, S_ROUTER_IN_POLICY, 0, "1", + REG_ECMP_GROUP_ID" = 0; next;", + od->nbr->name); + ovn_lflow_add_with_od_lflow_ref(lflows, od, S_ROUTER_IN_POLICY_ECMP, 150, + REG_ECMP_GROUP_ID" == 0", "next;", + od->nbr->name); + ovn_lflow_add_default_drop(lflows, od, S_ROUTER_IN_POLICY_ECMP, + &od->lflow_dep_mgr, OBJDEP_TYPE_OD, + od->nbr->name, od); /* Convert routing policies to flows. */ uint16_t ecmp_group_id = 1; @@ -13868,23 +14235,31 @@ build_arp_resolve_flows_for_lrouter( ovs_assert(od->nbr); /* Multicast packets already have the outport set so just advance to * next table (priority 500). */ - ovn_lflow_add(lflows, od, S_ROUTER_IN_ARP_RESOLVE, 500, - "ip4.mcast || ip6.mcast", "next;"); + ovn_lflow_add_with_od_lflow_ref(lflows, od, S_ROUTER_IN_ARP_RESOLVE, 500, + "ip4.mcast || ip6.mcast", "next;", + od->nbr->name); - ovn_lflow_add(lflows, od, S_ROUTER_IN_ARP_RESOLVE, 1, "ip4", - "get_arp(outport, " REG_NEXT_HOP_IPV4 "); next;"); + ovn_lflow_add_with_od_lflow_ref( + lflows, od, S_ROUTER_IN_ARP_RESOLVE, 1, "ip4", + "get_arp(outport, " REG_NEXT_HOP_IPV4 "); next;", od->nbr->name); - ovn_lflow_add(lflows, od, S_ROUTER_IN_ARP_RESOLVE, 1, "ip6", - "get_nd(outport, " REG_NEXT_HOP_IPV6 "); next;"); + ovn_lflow_add_with_od_lflow_ref( + lflows, od, S_ROUTER_IN_ARP_RESOLVE, 1, "ip6", + "get_nd(outport, " REG_NEXT_HOP_IPV6 "); next;", od->nbr->name); - ovn_lflow_add_default_drop(lflows, od, S_ROUTER_IN_ARP_RESOLVE); + ovn_lflow_add_default_drop(lflows, od, S_ROUTER_IN_ARP_RESOLVE, + &od->lflow_dep_mgr, OBJDEP_TYPE_OD, + od->nbr->name, od); } static void routable_addresses_to_lflows(struct lflow_data *lflows, struct ovn_port *router_port, struct ovn_port *peer, struct ds *match, - struct ds *actions) + struct ds *actions, + struct objdep_mgr *lflow_dep_mgr, + const char *res_name, + struct ovn_datapath *res_od) { struct ovn_port_routable_addresses *ra = &router_port->routables; if (!ra->n_addrs) { @@ -13908,7 +14283,8 @@ routable_addresses_to_lflows(struct lflow_data *lflows, ds_clear(actions); ds_put_format(actions, "eth.dst = %s; next;", ra->laddrs[i].ea_s); ovn_lflow_add(lflows, peer->od, S_ROUTER_IN_ARP_RESOLVE, 100, - ds_cstr(match), ds_cstr(actions)); + ds_cstr(match), ds_cstr(actions), lflow_dep_mgr, + OBJDEP_TYPE_LPORT, res_name, res_od); } } @@ -13948,10 +14324,15 @@ build_arp_resolve_flows_for_lrp( ds_clear(actions); ds_put_format(actions, "eth.dst = %s; next;", op->lrp_networks.ea_s); + /* Use the peer od's lflow dep mgr. */ ovn_lflow_add_with_hint(lflows, op->peer->od, S_ROUTER_IN_ARP_RESOLVE, 100, ds_cstr(match), ds_cstr(actions), - &op->nbrp->header_); + &op->nbrp->header_, + &op->lflow_dep_mgr, + OBJDEP_TYPE_LPORT, + op->nbrp->name, + op->od); } if (op->lrp_networks.n_ipv6_addrs) { @@ -13964,10 +14345,15 @@ build_arp_resolve_flows_for_lrp( ds_clear(actions); ds_put_format(actions, "eth.dst = %s; next;", op->lrp_networks.ea_s); + /* Use the peer od's lflow dep mgr. */ ovn_lflow_add_with_hint(lflows, op->peer->od, S_ROUTER_IN_ARP_RESOLVE, 100, ds_cstr(match), ds_cstr(actions), - &op->nbrp->header_); + &op->nbrp->header_, + &op->lflow_dep_mgr, + OBJDEP_TYPE_LPORT, + op->nbrp->name, + op->od); } } @@ -13992,7 +14378,11 @@ build_arp_resolve_flows_for_lrp( ovn_lflow_add_with_hint(lflows, op->od, S_ROUTER_IN_ARP_RESOLVE, 50, ds_cstr(match), ds_cstr(actions), - &op->nbrp->header_); + &op->nbrp->header_, + &op->lflow_dep_mgr, + OBJDEP_TYPE_LPORT, + op->nbrp->name, + op->od); } } @@ -14138,7 +14528,9 @@ build_arp_resolve_flows_for_lsp( ovn_lflow_add_with_hint(lflows, peer->od, S_ROUTER_IN_ARP_RESOLVE, 100, ds_cstr(match), ds_cstr(actions), - &op->nbsp->header_); + &op->nbsp->header_, + lflow_dep_mgr, OBJDEP_TYPE_LPORT, + op->key, op->od); } if (router_port->lrp_networks.n_ipv6_addrs) { @@ -14154,13 +14546,16 @@ build_arp_resolve_flows_for_lsp( ovn_lflow_add_with_hint(lflows, peer->od, S_ROUTER_IN_ARP_RESOLVE, 100, ds_cstr(match), ds_cstr(actions), - &op->nbsp->header_); + &op->nbsp->header_, + lflow_dep_mgr, OBJDEP_TYPE_LPORT, + op->key, op->od); } if (smap_get(&peer->od->nbr->options, "chassis") || peer->cr_port) { routable_addresses_to_lflows(lflows, router_port, peer, - match, actions); + match, actions, + lflow_dep_mgr, op->key, op->od); } } } @@ -14171,7 +14566,11 @@ build_icmperr_pkt_big_flows(struct ovn_port *op, int mtu, struct lflow_data *lflows, const struct shash *meter_groups, struct ds *match, struct ds *actions, enum ovn_stage stage, - struct ovn_port *outport) + struct ovn_port *outport, + struct objdep_mgr *lflow_dep_mgr, + enum objdep_type res_type, + const char *res_name, + struct ovn_datapath *res_od) { char *outport_match = outport ? xasprintf("outport == %s && ", outport->json_key) @@ -14207,7 +14606,10 @@ build_icmperr_pkt_big_flows(struct ovn_port *op, int mtu, COPP_ICMP4_ERR, op->od->nbr->copp, meter_groups), - &op->nbrp->header_); + &op->nbrp->header_, + lflow_dep_mgr, + res_type, + res_name, res_od); } if (op->lrp_networks.ipv6_addrs) { @@ -14240,7 +14642,10 @@ build_icmperr_pkt_big_flows(struct ovn_port *op, int mtu, COPP_ICMP6_ERR, op->od->nbr->copp, meter_groups), - &op->nbrp->header_); + &op->nbrp->header_, + lflow_dep_mgr, + res_type, + res_name, res_od); } free(outport_match); } @@ -14251,7 +14656,11 @@ build_check_pkt_len_flows_for_lrp(struct ovn_port *op, const struct hmap *lr_ports, const struct shash *meter_groups, struct ds *match, - struct ds *actions) + struct ds *actions, + struct objdep_mgr *lflow_dep_mgr, + enum objdep_type res_type, + const char *res_name, + struct ovn_datapath *res_od) { int gw_mtu = smap_get_int(&op->nbrp->options, "gateway_mtu", 0); if (gw_mtu <= 0) { @@ -14261,12 +14670,16 @@ build_check_pkt_len_flows_for_lrp(struct ovn_port *op, ds_clear(match); ds_put_format(match, "outport == %s", op->json_key); build_gateway_mtu_flow(lflows, op, S_ROUTER_IN_CHK_PKT_LEN, 50, 55, - match, actions, &op->nbrp->header_, "next;"); + match, actions, &op->nbrp->header_, + lflow_dep_mgr, res_type, + res_name, res_od, + "next;"); /* ingress traffic */ build_icmperr_pkt_big_flows(op, gw_mtu, lflows, meter_groups, match, actions, S_ROUTER_IN_IP_INPUT, - NULL); + NULL, lflow_dep_mgr, res_type, + res_name, res_od); for (size_t i = 0; i < op->od->nbr->n_ports; i++) { struct ovn_port *rp = ovn_port_find(lr_ports, @@ -14278,7 +14691,8 @@ build_check_pkt_len_flows_for_lrp(struct ovn_port *op, /* egress traffic */ build_icmperr_pkt_big_flows(rp, gw_mtu, lflows, meter_groups, match, actions, S_ROUTER_IN_LARGER_PKTS, - op); + op, lflow_dep_mgr, res_type, + res_name, res_od); } } @@ -14305,10 +14719,10 @@ build_check_pkt_len_flows_for_lrouter( ovs_assert(od->nbr); /* Packets are allowed by default. */ - ovn_lflow_add(lflows, od, S_ROUTER_IN_CHK_PKT_LEN, 0, "1", - "next;"); - ovn_lflow_add(lflows, od, S_ROUTER_IN_LARGER_PKTS, 0, "1", - "next;"); + ovn_lflow_add_with_od_lflow_ref(lflows, od, S_ROUTER_IN_CHK_PKT_LEN, 0, + "1", "next;", od->nbr->name); + ovn_lflow_add_with_od_lflow_ref(lflows, od, S_ROUTER_IN_LARGER_PKTS, 0, + "1", "next;", od->nbr->name); for (size_t i = 0; i < od->nbr->n_ports; i++) { struct ovn_port *rp = ovn_port_find(lr_ports, @@ -14317,7 +14731,9 @@ build_check_pkt_len_flows_for_lrouter( continue; } build_check_pkt_len_flows_for_lrp(rp, lflows, lr_ports, meter_groups, - match, actions); + match, actions, + &od->lflow_dep_mgr, OBJDEP_TYPE_OD, + od->nbr->name, od); } } @@ -14359,7 +14775,8 @@ build_gateway_redirect_flows_for_lrouter( od->l3dgw_ports[i]->cr_port->json_key); ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_GW_REDIRECT, 50, ds_cstr(match), ds_cstr(actions), - stage_hint); + stage_hint, &od->lflow_dep_mgr, + OBJDEP_TYPE_OD, od->nbr->name, od); for (int j = 0; j < od->n_nat_entries; j++) { const struct ovn_nat *nat = &od->nat_entries[j]; @@ -14378,28 +14795,34 @@ build_gateway_redirect_flows_for_lrouter( if (nat->nb->allowed_ext_ips) { ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_GW_REDIRECT, 75, ds_cstr(&match_ext), - ds_cstr(actions), stage_hint); + ds_cstr(actions), stage_hint, + &od->lflow_dep_mgr, + OBJDEP_TYPE_OD, od->nbr->name, od); if (add_def_flow) { ds_clear(&match_ext); ds_put_format(&match_ext, "ip && ip%s.dst == %s", nat_entry_is_v6(nat) ? "6" : "4", nat->nb->external_ip); - ovn_lflow_add(lflows, od, S_ROUTER_IN_GW_REDIRECT, 70, - ds_cstr(&match_ext), debug_drop_action()); + ovn_lflow_add_with_od_lflow_ref( + lflows, od, S_ROUTER_IN_GW_REDIRECT, 70, + ds_cstr(&match_ext), debug_drop_action(), + od->nbr->name); add_def_flow = false; } } else if (nat->nb->exempted_ext_ips) { ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_GW_REDIRECT, 75, ds_cstr(&match_ext), debug_drop_action(), - stage_hint); + stage_hint, &od->lflow_dep_mgr, + OBJDEP_TYPE_OD, od->nbr->name, od); } ds_destroy(&match_ext); } } /* Packets are allowed by default. */ - ovn_lflow_add(lflows, od, S_ROUTER_IN_GW_REDIRECT, 0, "1", "next;"); + ovn_lflow_add_with_od_lflow_ref(lflows, od, S_ROUTER_IN_GW_REDIRECT, 0, + "1", "next;", od->nbr->name); } /* Local router ingress table ARP_REQUEST: ARP request. @@ -14453,7 +14876,8 @@ build_arp_request_flows_for_lrouter( copp_meter_get(COPP_ND_NS_RESOLVE, od->nbr->copp, meter_groups), - &route->header_); + &route->header_, &od->lflow_dep_mgr, + OBJDEP_TYPE_OD, od->nbr->name, od); } ovn_lflow_metered(lflows, od, S_ROUTER_IN_ARP_REQUEST, 100, @@ -14466,7 +14890,8 @@ build_arp_request_flows_for_lrouter( "output; " "};", copp_meter_get(COPP_ARP_RESOLVE, od->nbr->copp, - meter_groups)); + meter_groups), + &od->lflow_dep_mgr, OBJDEP_TYPE_OD, od->nbr->name, od); ovn_lflow_metered(lflows, od, S_ROUTER_IN_ARP_REQUEST, 100, "eth.dst == 00:00:00:00:00:00 && ip6", "nd_ns { " @@ -14474,8 +14899,10 @@ build_arp_request_flows_for_lrouter( "output; " "};", copp_meter_get(COPP_ND_NS_RESOLVE, od->nbr->copp, - meter_groups)); - ovn_lflow_add(lflows, od, S_ROUTER_IN_ARP_REQUEST, 0, "1", "output;"); + meter_groups), + &od->lflow_dep_mgr, OBJDEP_TYPE_OD, od->nbr->name, od); + ovn_lflow_add_with_od_lflow_ref(lflows, od, S_ROUTER_IN_ARP_REQUEST, 0, + "1", "output;", od->nbr->name); } /* Logical router egress table DELIVERY: Delivery (priority 100-110). @@ -14515,16 +14942,22 @@ build_egress_delivery_flows_for_lrouter_port( op->json_key); ds_put_format(actions, "eth.src = %s; output;", op->lrp_networks.ea_s); - ovn_lflow_add(lflows, op->od, S_ROUTER_OUT_DELIVERY, 110, - ds_cstr(match), ds_cstr(actions)); + ovn_lflow_add(lflows, op->od, S_ROUTER_OUT_DELIVERY, + 110, ds_cstr(match), ds_cstr(actions), + &op->lflow_dep_mgr, OBJDEP_TYPE_LPORT, + op->nbrp->name, op->od); } ds_clear(match); ds_put_format(match, "outport == %s", op->json_key); ovn_lflow_add(lflows, op->od, S_ROUTER_OUT_DELIVERY, 100, - ds_cstr(match), "output;"); + ds_cstr(match), "output;", + &op->lflow_dep_mgr, OBJDEP_TYPE_LPORT, + op->nbrp->name, op->od); - ovn_lflow_add_default_drop(lflows, op->od, S_ROUTER_OUT_DELIVERY); + ovn_lflow_add_default_drop(lflows, op->od, S_ROUTER_OUT_DELIVERY, + &op->lflow_dep_mgr, OBJDEP_TYPE_LPORT, + op->nbrp->name, op->od); } static void @@ -14536,52 +14969,58 @@ build_misc_local_traffic_drop_flows_for_lrouter( * configured to flood them statically on some ports. */ if (od->mcast_info.rtr.flood_static) { - ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_INPUT, 120, - "igmp && ip.ttl == 1", "next;"); - ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_INPUT, 120, - "(mldv1 || mldv2) && ip.ttl == 1", "next;"); + ovn_lflow_add_with_od_lflow_ref(lflows, od, S_ROUTER_IN_IP_INPUT, 120, + "igmp && ip.ttl == 1", "next;", + od->nbr->name); + ovn_lflow_add_with_od_lflow_ref( + lflows, od, S_ROUTER_IN_IP_INPUT, 120, + "(mldv1 || mldv2) && ip.ttl == 1", "next;", od->nbr->name); } /* L3 admission control: drop multicast and broadcast source, localhost * source or destination, and zero network source or destination * (priority 100). */ - ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_INPUT, 100, + ovn_lflow_add_with_od_lflow_ref(lflows, od, S_ROUTER_IN_IP_INPUT, 100, "ip4.src_mcast ||" "ip4.src == 255.255.255.255 || " "ip4.src == 127.0.0.0/8 || " "ip4.dst == 127.0.0.0/8 || " "ip4.src == 0.0.0.0/8 || " "ip4.dst == 0.0.0.0/8", - debug_drop_action()); + debug_drop_action(), + od->nbr->name); /* Drop ARP packets (priority 85). ARP request packets for router's own * IPs are handled with priority-90 flows. * Drop IPv6 ND packets (priority 85). ND NA packets for router's own * IPs are handled with priority-90 flows. */ - ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_INPUT, 85, - "arp || nd", debug_drop_action()); + ovn_lflow_add_with_od_lflow_ref(lflows, od, S_ROUTER_IN_IP_INPUT, 85, + "arp || nd", debug_drop_action(), + od->nbr->name); /* Allow IPv6 multicast traffic that's supposed to reach the * router pipeline (e.g., router solicitations). */ - ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_INPUT, 84, "nd_rs || nd_ra", - "next;"); + ovn_lflow_add_with_od_lflow_ref(lflows, od, S_ROUTER_IN_IP_INPUT, 84, + "nd_rs || nd_ra", "next;", od->nbr->name); /* Drop other reserved multicast. */ - ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_INPUT, 83, - "ip6.mcast_rsvd", debug_drop_action()); + ovn_lflow_add_with_od_lflow_ref(lflows, od, S_ROUTER_IN_IP_INPUT, 83, + "ip6.mcast_rsvd", debug_drop_action(), + od->nbr->name); /* Allow other multicast if relay enabled (priority 82). */ - ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_INPUT, 82, + ovn_lflow_add_with_od_lflow_ref(lflows, od, S_ROUTER_IN_IP_INPUT, 82, "ip4.mcast || ip6.mcast", - (od->mcast_info.rtr.relay ? "next;" : - debug_drop_action())); + (od->mcast_info.rtr.relay ? "next;" : debug_drop_action()), + od->nbr->name); /* Drop Ethernet local broadcast. By definition this traffic should * not be forwarded.*/ - ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_INPUT, 50, - "eth.bcast", debug_drop_action()); + ovn_lflow_add_with_od_lflow_ref(lflows, od, S_ROUTER_IN_IP_INPUT, 50, + "eth.bcast", debug_drop_action(), + od->nbr->name); /* Avoid ICMP time exceeded for multicast, silent drop instead. * See RFC1812 section 5.3.1: @@ -14596,17 +15035,20 @@ build_misc_local_traffic_drop_flows_for_lrouter( * performance and functionality point of view. * * (priority-31 flows will send ICMP time exceeded) */ - ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_INPUT, 32, + ovn_lflow_add_with_od_lflow_ref(lflows, od, S_ROUTER_IN_IP_INPUT, 32, "ip.ttl == {0, 1} && !ip.later_frag && " - "(ip4.mcast || ip6.mcast)", debug_drop_action()); + "(ip4.mcast || ip6.mcast)", debug_drop_action(), + od->nbr->name); /* TTL discard */ - ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_INPUT, 30, - "ip.ttl == {0, 1}", debug_drop_action()); + ovn_lflow_add_with_od_lflow_ref(lflows, od, S_ROUTER_IN_IP_INPUT, 30, + "ip.ttl == {0, 1}", debug_drop_action(), + od->nbr->name); /* Pass other traffic not already handled to the next table for * routing. */ - ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_INPUT, 0, "1", "next;"); + ovn_lflow_add_with_od_lflow_ref(lflows, od, S_ROUTER_IN_IP_INPUT, 0, + "1", "next;", od->nbr->name); } static void @@ -14624,8 +15066,9 @@ build_dhcpv6_reply_flows_for_lrouter_port( " udp.dst == 546", op->lrp_networks.ipv6_addrs[i].addr_s); ovn_lflow_add(lflows, op->od, S_ROUTER_IN_IP_INPUT, 100, - ds_cstr(match), - "reg0 = 0; handle_dhcpv6_reply;"); + ds_cstr(match), "reg0 = 0; handle_dhcpv6_reply;", + &op->lflow_dep_mgr, OBJDEP_TYPE_LPORT, + op->nbrp->name, op->od); } } @@ -14657,7 +15100,9 @@ build_ipv6_input_flows_for_lrouter_port( "next; "; ovn_lflow_add_with_hint(lflows, op->od, S_ROUTER_IN_IP_INPUT, 90, ds_cstr(match), lrp_actions, - &op->nbrp->header_); + &op->nbrp->header_, + &op->lflow_dep_mgr, OBJDEP_TYPE_LPORT, + op->nbrp->name, op->od); } /* ND reply. These flows reply to ND solicitations for the @@ -14678,7 +15123,9 @@ build_ipv6_input_flows_for_lrouter_port( op->lrp_networks.ipv6_addrs[i].addr_s, op->lrp_networks.ipv6_addrs[i].sn_addr_s, REG_INPORT_ETH_ADDR, match, false, 90, - &op->nbrp->header_, lflows, meter_groups); + &op->nbrp->header_, lflows, meter_groups, + &op->lflow_dep_mgr, OBJDEP_TYPE_LPORT, + op->nbrp->name, op->od); } /* UDP/TCP/SCTP port unreachable */ @@ -14698,7 +15145,9 @@ build_ipv6_input_flows_for_lrouter_port( COPP_TCP_RESET, op->od->nbr->copp, meter_groups), - &op->nbrp->header_); + &op->nbrp->header_, + &op->lflow_dep_mgr, OBJDEP_TYPE_LPORT, + op->nbrp->name, op->od); ds_clear(match); ds_put_format(match, @@ -14714,7 +15163,9 @@ build_ipv6_input_flows_for_lrouter_port( COPP_TCP_RESET, op->od->nbr->copp, meter_groups), - &op->nbrp->header_); + &op->nbrp->header_, + &op->lflow_dep_mgr, OBJDEP_TYPE_LPORT, + op->nbrp->name, op->od); ds_clear(match); ds_put_format(match, @@ -14733,7 +15184,9 @@ build_ipv6_input_flows_for_lrouter_port( COPP_ICMP6_ERR, op->od->nbr->copp, meter_groups), - &op->nbrp->header_); + &op->nbrp->header_, + &op->lflow_dep_mgr, OBJDEP_TYPE_LPORT, + op->nbrp->name, op->od); ds_clear(match); ds_put_format(match, @@ -14752,7 +15205,9 @@ build_ipv6_input_flows_for_lrouter_port( COPP_ICMP6_ERR, op->od->nbr->copp, meter_groups), - &op->nbrp->header_); + &op->nbrp->header_, + &op->lflow_dep_mgr, OBJDEP_TYPE_LPORT, + op->nbrp->name, op->od); } } @@ -14792,7 +15247,9 @@ build_ipv6_input_flows_for_lrouter_port( 31, ds_cstr(match), ds_cstr(actions), NULL, copp_meter_get(COPP_ICMP6_ERR, op->od->nbr->copp, meter_groups), - &op->nbrp->header_); + &op->nbrp->header_, + &op->lflow_dep_mgr, OBJDEP_TYPE_LPORT, + op->nbrp->name, op->od); } ds_destroy(&ip_ds); } @@ -14867,7 +15324,9 @@ build_lrouter_ipv4_ip_input(struct ovn_port *op, ds_put_cstr(match, " && "REGBIT_EGRESS_LOOPBACK" == 0"); ovn_lflow_add_with_hint(lflows, op->od, S_ROUTER_IN_IP_INPUT, 100, ds_cstr(match), debug_drop_action(), - &op->nbrp->header_); + &op->nbrp->header_, + &op->lflow_dep_mgr, OBJDEP_TYPE_LPORT, + op->nbrp->name, op->od); /* ICMP echo reply. These flows reply to ICMP echo requests * received for the router's IP address. Since packets only @@ -14886,7 +15345,9 @@ build_lrouter_ipv4_ip_input(struct ovn_port *op, "next; "; ovn_lflow_add_with_hint(lflows, op->od, S_ROUTER_IN_IP_INPUT, 90, ds_cstr(match), icmp_actions, - &op->nbrp->header_); + &op->nbrp->header_, + &op->lflow_dep_mgr, OBJDEP_TYPE_LPORT, + op->nbrp->name, op->od); } /* BFD msg handling */ @@ -14919,7 +15380,9 @@ build_lrouter_ipv4_ip_input(struct ovn_port *op, 31, ds_cstr(match), ds_cstr(actions), NULL, copp_meter_get(COPP_ICMP4_ERR, op->od->nbr->copp, meter_groups), - &op->nbrp->header_); + &op->nbrp->header_, + &op->lflow_dep_mgr, OBJDEP_TYPE_LPORT, + op->nbrp->name, op->od); } ds_destroy(&ip_ds); @@ -14969,7 +15432,9 @@ build_lrouter_ipv4_ip_input(struct ovn_port *op, build_lrouter_arp_flow(op->od, op, op->lrp_networks.ipv4_addrs[i].addr_s, REG_INPORT_ETH_ADDR, match, false, 90, - &op->nbrp->header_, lflows); + &op->nbrp->header_, lflows, + &op->lflow_dep_mgr, OBJDEP_TYPE_LPORT, + op->nbrp->name, op->od); } if (sset_count(&op->od->lb_ips->ips_v4_reachable)) { @@ -14984,7 +15449,9 @@ build_lrouter_ipv4_ip_input(struct ovn_port *op, AF_INET); build_lrouter_arp_flow(op->od, op, lb_ips_v4_as, REG_INPORT_ETH_ADDR, - match, false, 90, NULL, lflows); + match, false, 90, NULL, lflows, + &op->lflow_dep_mgr, OBJDEP_TYPE_LPORT, + op->nbrp->name, op->od); free(lb_ips_v4_as); } @@ -15001,7 +15468,9 @@ build_lrouter_ipv4_ip_input(struct ovn_port *op, AF_INET6); build_lrouter_nd_flow(op->od, op, "nd_na", lb_ips_v6_as, NULL, REG_INPORT_ETH_ADDR, match, false, 90, - NULL, lflows, meter_groups); + NULL, lflows, meter_groups, + &op->lflow_dep_mgr, OBJDEP_TYPE_LPORT, + op->nbrp->name, op->od); free(lb_ips_v6_as); } @@ -15025,7 +15494,9 @@ build_lrouter_ipv4_ip_input(struct ovn_port *op, COPP_ICMP4_ERR, op->od->nbr->copp, meter_groups), - &op->nbrp->header_); + &op->nbrp->header_, + &op->lflow_dep_mgr, OBJDEP_TYPE_LPORT, + op->nbrp->name, op->od); ds_clear(match); ds_put_format(match, @@ -15041,7 +15512,9 @@ build_lrouter_ipv4_ip_input(struct ovn_port *op, COPP_TCP_RESET, op->od->nbr->copp, meter_groups), - &op->nbrp->header_); + &op->nbrp->header_, + &op->lflow_dep_mgr, OBJDEP_TYPE_LPORT, + op->nbrp->name, op->od); ds_clear(match); ds_put_format(match, @@ -15057,7 +15530,9 @@ build_lrouter_ipv4_ip_input(struct ovn_port *op, COPP_TCP_RESET, op->od->nbr->copp, meter_groups), - &op->nbrp->header_); + &op->nbrp->header_, + &op->lflow_dep_mgr, OBJDEP_TYPE_LPORT, + op->nbrp->name, op->od); ds_clear(match); ds_put_format(match, @@ -15076,7 +15551,9 @@ build_lrouter_ipv4_ip_input(struct ovn_port *op, COPP_ICMP4_ERR, op->od->nbr->copp, meter_groups), - &op->nbrp->header_); + &op->nbrp->header_, + &op->lflow_dep_mgr, OBJDEP_TYPE_LPORT, + op->nbrp->name, op->od); } } @@ -15186,7 +15663,9 @@ build_lrouter_in_unsnat_stateless_flow(struct lflow_data *lflows, ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_UNSNAT, priority, ds_cstr(match), "next;", - &nat->header_); + &nat->header_, + &od->lflow_dep_mgr, OBJDEP_TYPE_OD, + od->nbr->name, od); } static void @@ -15212,7 +15691,8 @@ build_lrouter_in_unsnat_in_czone_flow(struct lflow_data *lflows, ds_put_cstr(match, " && flags.loopback == 0"); ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_UNSNAT, 100, ds_cstr(match), "ct_snat_in_czone;", - &nat->header_); + &nat->header_, &od->lflow_dep_mgr, OBJDEP_TYPE_OD, + od->nbr->name, od); ds_truncate(match, common_match_len); /* Update common zone match for the hairpin traffic. */ @@ -15220,7 +15700,8 @@ build_lrouter_in_unsnat_in_czone_flow(struct lflow_data *lflows, ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_UNSNAT, 100, ds_cstr(match), "ct_snat;", - &nat->header_); + &nat->header_, &od->lflow_dep_mgr, OBJDEP_TYPE_OD, + od->nbr->name, od); } static void @@ -15242,7 +15723,8 @@ build_lrouter_in_unsnat_flow(struct lflow_data *lflows, ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_UNSNAT, priority, ds_cstr(match), "ct_snat;", - &nat->header_); + &nat->header_, &od->lflow_dep_mgr, OBJDEP_TYPE_OD, + od->nbr->name, od); } static void @@ -15312,7 +15794,9 @@ build_lrouter_in_dnat_flow(struct lflow_data *lflows, struct ovn_datapath *od, ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_DNAT, 100, ds_cstr(match), ds_cstr(actions), - &nat->header_); + &nat->header_, + &od->lflow_dep_mgr, OBJDEP_TYPE_OD, + od->nbr->name, od); } static void @@ -15363,7 +15847,9 @@ build_lrouter_out_undnat_flow(struct lflow_data *lflows, ovn_lflow_add_with_hint(lflows, od, S_ROUTER_OUT_UNDNAT, 100, ds_cstr(match), ds_cstr(actions), - &nat->header_); + &nat->header_, + &od->lflow_dep_mgr, OBJDEP_TYPE_OD, + od->nbr->name, od); } static void @@ -15394,7 +15880,9 @@ build_lrouter_out_is_dnat_local(struct lflow_data *lflows, ovn_lflow_add_with_hint(lflows, od, S_ROUTER_OUT_CHECK_DNAT_LOCAL, 50, ds_cstr(match), ds_cstr(actions), - &nat->header_); + &nat->header_, + &od->lflow_dep_mgr, OBJDEP_TYPE_OD, + od->nbr->name, od); } static void @@ -15466,7 +15954,9 @@ build_lrouter_out_snat_stateless_flow(struct lflow_data *lflows, ovn_lflow_add_with_hint(lflows, od, S_ROUTER_OUT_SNAT, priority, ds_cstr(match), - ds_cstr(actions), &nat->header_); + ds_cstr(actions), &nat->header_, + &od->lflow_dep_mgr, OBJDEP_TYPE_OD, + od->nbr->name, od); } static void @@ -15519,13 +16009,17 @@ build_lrouter_out_snat_in_czone_flow(struct lflow_data *lflows, ovn_lflow_add_with_hint(lflows, od, S_ROUTER_OUT_SNAT, priority, ds_cstr(match), - ds_cstr(actions), &nat->header_); + ds_cstr(actions), &nat->header_, + &od->lflow_dep_mgr, OBJDEP_TYPE_OD, + od->nbr->name, od); ds_put_cstr(match, " && "REGBIT_DST_NAT_IP_LOCAL" == 1"); ovn_lflow_add_with_hint(lflows, od, S_ROUTER_OUT_SNAT, priority + 1, ds_cstr(match), - ds_cstr(&zone_actions), &nat->header_); + ds_cstr(&zone_actions), &nat->header_, + &od->lflow_dep_mgr, OBJDEP_TYPE_OD, + od->nbr->name, od); ds_destroy(&zone_actions); } @@ -15574,7 +16068,9 @@ build_lrouter_out_snat_flow(struct lflow_data *lflows, struct ovn_datapath *od, ovn_lflow_add_with_hint(lflows, od, S_ROUTER_OUT_SNAT, priority, ds_cstr(match), - ds_cstr(actions), &nat->header_); + ds_cstr(actions), &nat->header_, + &od->lflow_dep_mgr, OBJDEP_TYPE_OD, + od->nbr->name, od); } static void @@ -15617,7 +16113,8 @@ build_lrouter_ingress_nat_check_pkt_len(struct lflow_data *lflows, COPP_ICMP4_ERR, od->nbr->copp, meter_groups), - &nat->header_); + &nat->header_, &od->lflow_dep_mgr, + OBJDEP_TYPE_OD, od->nbr->name, od); } else { ds_put_format(match, " && ip6 && ip6.dst == %s", nat->external_ip); /* Set icmp6.frag_mtu to gw_mtu */ @@ -15644,7 +16141,8 @@ build_lrouter_ingress_nat_check_pkt_len(struct lflow_data *lflows, COPP_ICMP6_ERR, od->nbr->copp, meter_groups), - &nat->header_); + &nat->header_, &od->lflow_dep_mgr, + OBJDEP_TYPE_OD, od->nbr->name, od); } } @@ -15664,7 +16162,9 @@ build_lrouter_ingress_flow(struct lflow_data *lflows, struct ovn_datapath *od, is_v6 ? "ip6.src" : "ip4.src", nat->external_ip); ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_IP_INPUT, 120, ds_cstr(match), "next;", - &nat->header_); + &nat->header_, + &od->lflow_dep_mgr, OBJDEP_TYPE_OD, + od->nbr->name, od); } /* Logical router ingress table 0: * For NAT on a distributed router, add rules allowing @@ -15688,6 +16188,8 @@ build_lrouter_ingress_flow(struct lflow_data *lflows, struct ovn_datapath *od, build_gateway_mtu_flow(lflows, l3dgw_port, S_ROUTER_IN_ADMISSION, 50, 55, match, actions, &nat->header_, + &od->lflow_dep_mgr, OBJDEP_TYPE_OD, + od->nbr->name, od, REG_INPORT_ETH_ADDR " = %s; next;", l3dgw_port->lrp_networks.ea_s); if (gw_mtu) { @@ -15827,29 +16329,14 @@ lrouter_check_nat_entry(struct ovn_datapath *od, const struct nbrec_nat *nat, /* NAT, Defrag and load balancing. */ static void -build_lrouter_nat_defrag_and_lb(struct ovn_datapath *od, - struct lflow_data *lflows, - const struct hmap *ls_ports, - const struct hmap *lr_ports, - struct ds *match, - struct ds *actions, - const struct shash *meter_groups, - const struct chassis_features *features) +build_lrouter_nat_flows_lb_related(struct ovn_datapath *od, + struct lflow_data *lflows, + struct ds *match, + const struct chassis_features *features) { - ovs_assert(od->nbr); - - /* Packets are allowed by default. */ - ovn_lflow_add(lflows, od, S_ROUTER_IN_DEFRAG, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_ROUTER_IN_UNSNAT, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_ROUTER_OUT_CHECK_DNAT_LOCAL, 0, "1", - REGBIT_DST_NAT_IP_LOCAL" = 0; next;"); - ovn_lflow_add(lflows, od, S_ROUTER_OUT_SNAT, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_ROUTER_IN_DNAT, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_ROUTER_OUT_UNDNAT, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_ROUTER_OUT_POST_UNDNAT, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_ROUTER_OUT_POST_SNAT, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_ROUTER_OUT_EGR_LOOP, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_ROUTER_IN_ECMP_STATEFUL, 0, "1", "next;"); + if (!od->has_lb_vip) { + return; + } const char *ct_flag_reg = features->ct_no_masked_label ? "ct_mark" @@ -15863,24 +16350,27 @@ build_lrouter_nat_defrag_and_lb(struct ovn_datapath *od, * a dynamically negotiated FTP data channel), but will allow * related traffic such as an ICMP Port Unreachable through * that's generated from a non-listening UDP port. */ - if (od->has_lb_vip && features->ct_lb_related) { + if (features->ct_lb_related) { ds_clear(match); ds_put_cstr(match, "ct.rel && !ct.est && !ct.new"); size_t match_len = match->length; ds_put_format(match, " && %s.skip_snat == 1", ct_flag_reg); - ovn_lflow_add(lflows, od, S_ROUTER_IN_DNAT, 70, ds_cstr(match), - "flags.skip_snat_for_lb = 1; ct_commit_nat;"); + ovn_lflow_add_with_lb_lflow_ref( + lflows, od, S_ROUTER_IN_DNAT, 70, ds_cstr(match), + "flags.skip_snat_for_lb = 1; ct_commit_nat;", od->nbr->name); ds_truncate(match, match_len); ds_put_format(match, " && %s.force_snat == 1", ct_flag_reg); - ovn_lflow_add(lflows, od, S_ROUTER_IN_DNAT, 70, ds_cstr(match), - "flags.force_snat_for_lb = 1; ct_commit_nat;"); + ovn_lflow_add_with_lb_lflow_ref( + lflows, od, S_ROUTER_IN_DNAT, 70, ds_cstr(match), + "flags.force_snat_for_lb = 1; ct_commit_nat;", od->nbr->name); ds_truncate(match, match_len); - ovn_lflow_add(lflows, od, S_ROUTER_IN_DNAT, 50, ds_cstr(match), - "ct_commit_nat;"); + ovn_lflow_add_with_lb_lflow_ref(lflows, od, S_ROUTER_IN_DNAT, 50, + ds_cstr(match), "ct_commit_nat;", + od->nbr->name); } /* Ingress DNAT (Priority 50/70). @@ -15888,25 +16378,53 @@ build_lrouter_nat_defrag_and_lb(struct ovn_datapath *od, * Pass the traffic that is already established to the next table with * proper flags set. */ - if (od->has_lb_vip) { - ds_clear(match); + ds_clear(match); - ds_put_format(match, "ct.est && !ct.rel && !ct.new && %s.natted", - ct_flag_reg); - size_t match_len = match->length; + ds_put_format(match, "ct.est && !ct.rel && !ct.new && %s.natted", + ct_flag_reg); + size_t match_len = match->length; - ds_put_format(match, " && %s.skip_snat == 1", ct_flag_reg); - ovn_lflow_add(lflows, od, S_ROUTER_IN_DNAT, 70, ds_cstr(match), - "flags.skip_snat_for_lb = 1; next;"); + ds_put_format(match, " && %s.skip_snat == 1", ct_flag_reg); + ovn_lflow_add_with_lb_lflow_ref( + lflows, od, S_ROUTER_IN_DNAT, 70, ds_cstr(match), + "flags.skip_snat_for_lb = 1; next;", od->nbr->name); - ds_truncate(match, match_len); - ds_put_format(match, " && %s.force_snat == 1", ct_flag_reg); - ovn_lflow_add(lflows, od, S_ROUTER_IN_DNAT, 70, ds_cstr(match), - "flags.force_snat_for_lb = 1; next;"); + ds_truncate(match, match_len); + ds_put_format(match, " && %s.force_snat == 1", ct_flag_reg); + ovn_lflow_add_with_lb_lflow_ref( + lflows, od, S_ROUTER_IN_DNAT, 70, ds_cstr(match), + "flags.force_snat_for_lb = 1; next;", od->nbr->name); - ds_truncate(match, match_len); - ovn_lflow_add(lflows, od, S_ROUTER_IN_DNAT, 50, ds_cstr(match), - "next;"); + ds_truncate(match, match_len); + ovn_lflow_add_with_lb_lflow_ref(lflows, od, S_ROUTER_IN_DNAT, 50, + ds_cstr(match), "next;", + od->nbr->name); + + /* Handle force SNAT options set in the gateway router. */ + if (od->is_gw_router) { + bool lb_force_snat_ip = + !lport_addresses_is_empty(&od->lb_force_snat_addrs); + if (lb_force_snat_ip) { + if (od->lb_force_snat_addrs.n_ipv4_addrs) { + build_lrouter_force_snat_flows(lflows, od, "4", + od->lb_force_snat_addrs.ipv4_addrs[0].addr_s, "lb", + OBJDEP_TYPE_LB); + } + if (od->lb_force_snat_addrs.n_ipv6_addrs) { + build_lrouter_force_snat_flows(lflows, od, "6", + od->lb_force_snat_addrs.ipv6_addrs[0].addr_s, "lb", + OBJDEP_TYPE_LB); + } + } + } +} + +static void +build_lrouter_nat_flows_ct_lb_related(struct ovn_datapath *od, + struct lflow_data *lflows) +{ + if (!(od->nbr->n_nat || od->has_lb_vip)) { + return; } /* If the router has load balancer or DNAT rules, re-circulate every packet @@ -15918,20 +16436,63 @@ build_lrouter_nat_defrag_and_lb(struct ovn_datapath *od, * not committed, it would produce ongoing datapath flows with the ct.new * flag set. Some NICs are unable to offload these flows. */ - if (od->is_gw_router && (od->nbr->n_nat || od->has_lb_vip)) { + if (od->is_gw_router) { /* Do not send ND or ICMP packets to connection tracking. */ - ovn_lflow_add(lflows, od, S_ROUTER_OUT_UNDNAT, 100, - "nd || nd_rs || nd_ra", "next;"); - ovn_lflow_add(lflows, od, S_ROUTER_OUT_UNDNAT, 50, - "ip", "flags.loopback = 1; ct_dnat;"); - ovn_lflow_add(lflows, od, S_ROUTER_OUT_POST_UNDNAT, 50, - "ip && ct.new", "ct_commit { } ; next; "); + ovn_lflow_add_with_ct_lb_lflow_ref(lflows, od, S_ROUTER_OUT_UNDNAT, + 100, "nd || nd_rs || nd_ra", + "next;", od->nbr->name); + ovn_lflow_add_with_ct_lb_lflow_ref(lflows, od, S_ROUTER_OUT_UNDNAT, 50, + "ip", "flags.loopback = 1; ct_dnat;", + od->nbr->name); + ovn_lflow_add_with_ct_lb_lflow_ref(lflows, od, + S_ROUTER_OUT_POST_UNDNAT, + 50, "ip && ct.new", + "ct_commit { } ; next; ", + od->nbr->name); } +} + +static void +build_lrouter_nat_defrag_and_lb(struct ovn_datapath *od, + struct lflow_data *lflows, + const struct hmap *ls_ports, + const struct hmap *lr_ports, + struct ds *match, + struct ds *actions, + const struct shash *meter_groups, + const struct chassis_features *features) +{ + ovs_assert(od->nbr); + + /* Packets are allowed by default. */ + ovn_lflow_add_with_od_lflow_ref(lflows, od, S_ROUTER_IN_DEFRAG, 0, + "1", "next;", od->nbr->name); + ovn_lflow_add_with_od_lflow_ref(lflows, od, S_ROUTER_IN_UNSNAT, 0, + "1", "next;", od->nbr->name); + ovn_lflow_add_with_od_lflow_ref(lflows, od, S_ROUTER_OUT_CHECK_DNAT_LOCAL, + 0, "1", + REGBIT_DST_NAT_IP_LOCAL" = 0; next;", + od->nbr->name); + ovn_lflow_add_with_od_lflow_ref(lflows, od, S_ROUTER_OUT_SNAT, 0, + "1", "next;", od->nbr->name); + ovn_lflow_add_with_od_lflow_ref(lflows, od, S_ROUTER_IN_DNAT, 0, + "1", "next;", od->nbr->name); + ovn_lflow_add_with_od_lflow_ref(lflows, od, S_ROUTER_OUT_UNDNAT, 0, + "1", "next;", od->nbr->name); + ovn_lflow_add_with_od_lflow_ref(lflows, od, S_ROUTER_OUT_POST_UNDNAT, + 0, "1", "next;", od->nbr->name); + ovn_lflow_add_with_od_lflow_ref(lflows, od, S_ROUTER_OUT_POST_SNAT, 0, + "1", "next;", od->nbr->name); + ovn_lflow_add_with_od_lflow_ref(lflows, od, S_ROUTER_OUT_EGR_LOOP, 0, + "1", "next;", od->nbr->name); + ovn_lflow_add_with_od_lflow_ref(lflows, od, S_ROUTER_IN_ECMP_STATEFUL, 0, + "1", "next;", od->nbr->name); /* Send the IPv6 NS packets to next table. When ovn-controller * generates IPv6 NS (for the action - nd_ns{}), the injected * packet would go through conntrack - which is not required. */ - ovn_lflow_add(lflows, od, S_ROUTER_OUT_SNAT, 120, "nd_ns", "next;"); + ovn_lflow_add_with_od_lflow_ref(lflows, od, S_ROUTER_OUT_SNAT, 120, + "nd_ns", "next;", od->nbr->name); /* NAT rules are only valid on Gateway routers and routers with * l3dgw_ports (router has port(s) with gateway chassis @@ -15944,8 +16505,6 @@ build_lrouter_nat_defrag_and_lb(struct ovn_datapath *od, bool dnat_force_snat_ip = !lport_addresses_is_empty(&od->dnat_force_snat_addrs); - bool lb_force_snat_ip = - !lport_addresses_is_empty(&od->lb_force_snat_addrs); for (int i = 0; i < od->nbr->n_nat; i++) { const struct nbrec_nat *nat = nat = od->nbr->nat[i]; @@ -16011,7 +16570,9 @@ build_lrouter_nat_defrag_and_lb(struct ovn_datapath *od, S_ROUTER_IN_ARP_RESOLVE, 150, ds_cstr(match), debug_drop_action(), - &nat->header_); + &nat->header_, + &od->lflow_dep_mgr, OBJDEP_TYPE_OD, + od->nbr->name, od); /* Now for packets coming from other (downlink) LRPs, allow ARP * resolve for the NAT IP, so that such packets can be * forwarded for E/W NAT. */ @@ -16030,7 +16591,9 @@ build_lrouter_nat_defrag_and_lb(struct ovn_datapath *od, S_ROUTER_IN_ARP_RESOLVE, 100, ds_cstr(match), ds_cstr(actions), - &nat->header_); + &nat->header_, + &od->lflow_dep_mgr, OBJDEP_TYPE_OD, + od->nbr->name, od); if (od->redirect_bridged && distributed_nat) { ds_clear(match); ds_put_format( @@ -16051,7 +16614,9 @@ build_lrouter_nat_defrag_and_lb(struct ovn_datapath *od, ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_ARP_RESOLVE, 90, ds_cstr(match), ds_cstr(actions), - &nat->header_); + &nat->header_, + &od->lflow_dep_mgr, OBJDEP_TYPE_OD, + od->nbr->name, od); } sset_add(&nat_entries, nat->external_ip); } @@ -16117,7 +16682,9 @@ build_lrouter_nat_defrag_and_lb(struct ovn_datapath *od, if (op && op->nbsp && !strcmp(op->nbsp->type, "virtual")) { ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_GW_REDIRECT, 80, ds_cstr(match), - debug_drop_action(), &nat->header_); + debug_drop_action(), &nat->header_, + &od->lflow_dep_mgr, OBJDEP_TYPE_OD, + od->nbr->name, od); } ds_put_format(match, " && is_chassis_resident(\"%s\")", nat->logical_port); @@ -16127,7 +16694,9 @@ build_lrouter_nat_defrag_and_lb(struct ovn_datapath *od, nat->external_ip); ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_GW_REDIRECT, 100, ds_cstr(match), - ds_cstr(actions), &nat->header_); + ds_cstr(actions), &nat->header_, + &od->lflow_dep_mgr, OBJDEP_TYPE_OD, + od->nbr->name, od); } /* Egress Loopback table: For NAT on a distributed router. @@ -16168,7 +16737,9 @@ build_lrouter_nat_defrag_and_lb(struct ovn_datapath *od, ovn_stage_get_table(S_ROUTER_IN_ADMISSION)); ovn_lflow_add_with_hint(lflows, od, S_ROUTER_OUT_EGR_LOOP, 100, ds_cstr(match), ds_cstr(actions), - &nat->header_); + &nat->header_, + &od->lflow_dep_mgr, OBJDEP_TYPE_OD, + od->nbr->name, od); } } @@ -16188,7 +16759,9 @@ build_lrouter_nat_defrag_and_lb(struct ovn_datapath *od, ds_put_cstr(actions, REGBIT_DST_NAT_IP_LOCAL" = 1; next;"); ovn_lflow_add_with_hint(lflows, od, S_ROUTER_OUT_CHECK_DNAT_LOCAL, 50, ds_cstr(match), ds_cstr(actions), - &od->nbr->header_); + &od->nbr->header_, + &od->lflow_dep_mgr, OBJDEP_TYPE_OD, + od->nbr->name, od); } @@ -16198,22 +16771,12 @@ build_lrouter_nat_defrag_and_lb(struct ovn_datapath *od, if (od->dnat_force_snat_addrs.n_ipv4_addrs) { build_lrouter_force_snat_flows(lflows, od, "4", od->dnat_force_snat_addrs.ipv4_addrs[0].addr_s, - "dnat"); + "dnat", OBJDEP_TYPE_OD); } if (od->dnat_force_snat_addrs.n_ipv6_addrs) { build_lrouter_force_snat_flows(lflows, od, "6", od->dnat_force_snat_addrs.ipv6_addrs[0].addr_s, - "dnat"); - } - } - if (lb_force_snat_ip) { - if (od->lb_force_snat_addrs.n_ipv4_addrs) { - build_lrouter_force_snat_flows(lflows, od, "4", - od->lb_force_snat_addrs.ipv4_addrs[0].addr_s, "lb"); - } - if (od->lb_force_snat_addrs.n_ipv6_addrs) { - build_lrouter_force_snat_flows(lflows, od, "6", - od->lb_force_snat_addrs.ipv6_addrs[0].addr_s, "lb"); + "dnat", OBJDEP_TYPE_OD); } } } @@ -16221,8 +16784,6 @@ build_lrouter_nat_defrag_and_lb(struct ovn_datapath *od, sset_destroy(&nat_entries); } - - struct lswitch_flow_build_info { const struct ovn_datapaths *ls_datapaths; const struct ovn_datapaths *lr_datapaths; @@ -16304,6 +16865,9 @@ build_lswitch_and_lrouter_iterate_by_lr(struct ovn_datapath *od, lsi->lr_ports, &lsi->match, &lsi->actions, lsi->meter_groups, lsi->features); + build_lrouter_nat_flows_lb_related(od, lsi->lflows, &lsi->match, + lsi->features); + build_lrouter_nat_flows_ct_lb_related(od, lsi->lflows); build_lrouter_lb_affinity_default_flows(od, lsi->lflows); } @@ -16452,7 +17016,7 @@ build_lflows_thread(void *arg) if (stop_parallel_processing()) { return NULL; } - build_lswitch_arp_nd_service_monitor(lb_dps->lb, + build_lswitch_arp_nd_service_monitor(lb_dps, lsi->ls_ports, lsi->lflows, &lsi->match, @@ -16636,7 +17200,7 @@ build_lswitch_and_lrouter_flows(const struct ovn_datapaths *ls_datapaths, stopwatch_stop(LFLOWS_PORTS_STOPWATCH_NAME, time_msec()); stopwatch_start(LFLOWS_LBS_STOPWATCH_NAME, time_msec()); HMAP_FOR_EACH (lb_dps, hmap_node, lb_dps_map) { - build_lswitch_arp_nd_service_monitor(lb_dps->lb, lsi.ls_ports, + build_lswitch_arp_nd_service_monitor(lb_dps, lsi.ls_ports, lsi.lflows, &lsi.actions, &lsi.match); build_lrouter_defrag_flows_for_lb(lb_dps, lsi.lflows, diff --git a/northd/northd.h b/northd/northd.h index 4af6c4e9b0..d0992f142e 100644 --- a/northd/northd.h +++ b/northd/northd.h @@ -325,6 +325,8 @@ struct ovn_datapath { /* Map of ovn_port objects belonging to this datapath. * This map doesn't include derived ports. */ struct hmap ports; + + struct objdep_mgr lflow_dep_mgr; }; void ovnnb_db_run(struct northd_input *input_data, From patchwork Fri Aug 18 08:58:27 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Numan Siddique X-Patchwork-Id: 1822763 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.138; helo=smtp1.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4RRwmd5TJ5z1ygH for ; Fri, 18 Aug 2023 18:59:45 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 986C283F7C; Fri, 18 Aug 2023 08:59:43 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 986C283F7C X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id YnsM0bGyUNAz; Fri, 18 Aug 2023 08:59:40 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp1.osuosl.org (Postfix) with ESMTPS id 9DD9583F4C; Fri, 18 Aug 2023 08:59:32 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 9DD9583F4C Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 5BC55C0039; Fri, 18 Aug 2023 08:59:32 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 24636C0039 for ; Fri, 18 Aug 2023 08:59:30 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id DC2F4615D5 for ; Fri, 18 Aug 2023 08:58:36 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org DC2F4615D5 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id mApD5QHBIYdO for ; Fri, 18 Aug 2023 08:58:35 +0000 (UTC) Received: from relay8-d.mail.gandi.net (relay8-d.mail.gandi.net [217.70.183.201]) by smtp3.osuosl.org (Postfix) with ESMTPS id EA575615D9 for ; Fri, 18 Aug 2023 08:58:34 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org EA575615D9 Received: by mail.gandi.net (Postfix) with ESMTPSA id E424D1BF206; Fri, 18 Aug 2023 08:58:31 +0000 (UTC) From: numans@ovn.org To: dev@openvswitch.org Date: Fri, 18 Aug 2023 14:28:27 +0530 Message-Id: <20230818085827.1031095-1-numans@ovn.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230818085606.1030792-1-numans@ovn.org> References: <20230818085606.1030792-1-numans@ovn.org> MIME-Version: 1.0 X-GND-Sasl: numans@ovn.org Subject: [ovs-dev] [PATCH ovn v6 12/16] Reference lb related lflows for lports in a separate objdep mgr type. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Numan Siddique This will help in handling the lflows for logical switch ports or router ports incrementally. Upcoming patch will make use of this separtion. Signed-off-by: Numan Siddique --- northd/northd.c | 333 ++++++++++++++++++++++++++++++++---------------- 1 file changed, 221 insertions(+), 112 deletions(-) diff --git a/northd/northd.c b/northd/northd.c index 8c0b732653..2ec768ab71 100644 --- a/northd/northd.c +++ b/northd/northd.c @@ -9567,7 +9567,8 @@ build_lswitch_rport_arp_req_flow(const char *ips, int addr_family, struct ovn_port *patch_op, struct ovn_datapath *od, uint32_t priority, struct lflow_data *lflows, const struct ovsdb_idl_row *stage_hint, - struct objdep_mgr *lflow_dep_mgr) + struct objdep_mgr *lflow_dep_mgr, + enum objdep_type objdep_type) { struct ds match = DS_EMPTY_INITIALIZER; struct ds actions = DS_EMPTY_INITIALIZER; @@ -9581,16 +9582,16 @@ build_lswitch_rport_arp_req_flow(const char *ips, ds_put_format(&actions, "clone {outport = %s; output; }; " "outport = \""MC_FLOOD_L2"\"; output;", patch_op->json_key); - ovn_lflow_add_with_lport_lflow_ref(lflows, od, S_SWITCH_IN_L2_LKUP, - priority, ds_cstr(&match), - ds_cstr(&actions), NULL, NULL, stage_hint, - lflow_dep_mgr, patch_op->key, patch_op->od); + ovn_lflow_add_with_hint(lflows, od, S_SWITCH_IN_L2_LKUP, priority, + ds_cstr(&match), ds_cstr(&actions), stage_hint, + lflow_dep_mgr, objdep_type, patch_op->key, + patch_op->od); } else { ds_put_format(&actions, "outport = %s; output;", patch_op->json_key); - ovn_lflow_add_with_lport_lflow_ref(lflows, od, S_SWITCH_IN_L2_LKUP, - priority, ds_cstr(&match), ds_cstr(&actions), - NULL, NULL, stage_hint, lflow_dep_mgr, - patch_op->key, patch_op->od); + ovn_lflow_add_with_hint(lflows, od, S_SWITCH_IN_L2_LKUP, priority, + ds_cstr(&match), ds_cstr(&actions), stage_hint, + lflow_dep_mgr, objdep_type, patch_op->key, + patch_op->od); } ds_destroy(&match); @@ -9620,39 +9621,6 @@ build_lswitch_rport_arp_req_flows(struct ovn_port *op, return; } - /* Forward ARP requests for owned IP addresses (L3, VIP, NAT) only to this - * router port. - * Priority: 80. - */ - - const char *ip_addr; - SSET_FOR_EACH (ip_addr, &op->od->lb_ips->ips_v4_reachable) { - ovs_be32 ipv4_addr; - - /* Check if the ovn port has a network configured on which we could - * expect ARP requests for the LB VIP. - */ - if (ip_parse(ip_addr, &ipv4_addr) && - lrouter_port_ipv4_reachable(op, ipv4_addr)) { - build_lswitch_rport_arp_req_flow( - ip_addr, AF_INET, sw_op, sw_od, 80, lflows, - stage_hint, lflow_dep_mgr); - } - } - SSET_FOR_EACH (ip_addr, &op->od->lb_ips->ips_v6_reachable) { - struct in6_addr ipv6_addr; - - /* Check if the ovn port has a network configured on which we could - * expect NS requests for the LB VIP. - */ - if (ipv6_parse(ip_addr, &ipv6_addr) && - lrouter_port_ipv6_reachable(op, &ipv6_addr)) { - build_lswitch_rport_arp_req_flow( - ip_addr, AF_INET6, sw_op, sw_od, 80, lflows, - stage_hint, lflow_dep_mgr); - } - } - for (size_t i = 0; i < op->od->nbr->n_nat; i++) { struct ovn_nat *nat_entry = &op->od->nat_entries[i]; const struct nbrec_nat *nat = nat_entry->nb; @@ -9672,13 +9640,13 @@ build_lswitch_rport_arp_req_flows(struct ovn_port *op, if (!sset_contains(&op->od->lb_ips->ips_v6, nat->external_ip)) { build_lswitch_rport_arp_req_flow( nat->external_ip, AF_INET6, sw_op, sw_od, 80, lflows, - stage_hint, lflow_dep_mgr); + stage_hint, lflow_dep_mgr, OBJDEP_TYPE_LPORT); } } else { if (!sset_contains(&op->od->lb_ips->ips_v4, nat->external_ip)) { build_lswitch_rport_arp_req_flow( nat->external_ip, AF_INET, sw_op, sw_od, 80, lflows, - stage_hint, lflow_dep_mgr); + stage_hint, lflow_dep_mgr, OBJDEP_TYPE_LPORT); } } } @@ -9686,12 +9654,12 @@ build_lswitch_rport_arp_req_flows(struct ovn_port *op, for (size_t i = 0; i < op->lrp_networks.n_ipv4_addrs; i++) { build_lswitch_rport_arp_req_flow( op->lrp_networks.ipv4_addrs[i].addr_s, AF_INET, sw_op, sw_od, 80, - lflows, stage_hint, lflow_dep_mgr); + lflows, stage_hint, lflow_dep_mgr, OBJDEP_TYPE_LPORT); } for (size_t i = 0; i < op->lrp_networks.n_ipv6_addrs; i++) { build_lswitch_rport_arp_req_flow( op->lrp_networks.ipv6_addrs[i].addr_s, AF_INET6, sw_op, sw_od, 80, - lflows, stage_hint, lflow_dep_mgr); + lflows, stage_hint, lflow_dep_mgr, OBJDEP_TYPE_LPORT); } /* Self originated ARP requests/RARP/ND need to be flooded as usual. @@ -10970,6 +10938,52 @@ build_lswitch_ip_unicast_lookup(struct ovn_port *op, } } +static void +build_lswitch_ip_unicast_lookup_lb_vips(struct ovn_port *op, + struct lflow_data *lflows, + struct objdep_mgr *lflow_dep_mgr) +{ + ovs_assert(op->nbsp); + + /* For ports connected to logical routers add flows to bypass the + * broadcast flooding of ARP/ND requests in table 19. We direct the + * requests only to the router port that owns the IP address. + */ + if (!lsp_is_router(op->nbsp) || !op->peer) { + return; + } + + struct ovn_port *peer = op->peer; + const char *ip_addr; + + SSET_FOR_EACH (ip_addr, &peer->od->lb_ips->ips_v4_reachable) { + ovs_be32 ipv4_addr; + + /* Check if the ovn port has a network configured on which we could + * expect ARP requests for the LB VIP. + */ + if (ip_parse(ip_addr, &ipv4_addr) && + lrouter_port_ipv4_reachable(peer, ipv4_addr)) { + build_lswitch_rport_arp_req_flow( + ip_addr, AF_INET, op, op->od, 80, lflows, + &op->nbsp->header_, lflow_dep_mgr, OBJDEP_TYPE_LB); + } + } + SSET_FOR_EACH (ip_addr, &peer->od->lb_ips->ips_v6_reachable) { + struct in6_addr ipv6_addr; + + /* Check if the ovn port has a network configured on which we could + * expect NS requests for the LB VIP. + */ + if (ipv6_parse(ip_addr, &ipv6_addr) && + lrouter_port_ipv6_reachable(peer, &ipv6_addr)) { + build_lswitch_rport_arp_req_flow( + ip_addr, AF_INET6, op, op->od, 80, lflows, + &op->nbsp->header_, lflow_dep_mgr, OBJDEP_TYPE_LB); + } + } +} + struct bfd_entry { struct hmap_node hmap_node; @@ -13154,7 +13168,8 @@ build_lrouter_port_nat_arp_nd_flow(struct ovn_port *op, static void build_lrouter_drop_own_dest(struct ovn_port *op, enum ovn_stage stage, uint16_t priority, bool drop_snat_ip, - struct lflow_data *lflows) + struct lflow_data *lflows, + enum objdep_type objdep_type) { struct ds match_ips = DS_EMPTY_INITIALIZER; @@ -13182,7 +13197,7 @@ build_lrouter_drop_own_dest(struct ovn_port *op, enum ovn_stage stage, match, debug_drop_action(), &op->nbrp->header_, &op->lflow_dep_mgr, - OBJDEP_TYPE_LPORT, + objdep_type, op->nbrp->name, op->od); free(match); @@ -13215,7 +13230,7 @@ build_lrouter_drop_own_dest(struct ovn_port *op, enum ovn_stage stage, match, debug_drop_action(), &op->nbrp->header_, &op->lflow_dep_mgr, - OBJDEP_TYPE_LPORT, + objdep_type, op->nbrp->name, op->od); free(match); @@ -13272,7 +13287,7 @@ build_lrouter_force_snat_flows_op(struct ovn_port *op, op->json_key, op->lrp_networks.ipv4_addrs[0].addr_s); ovn_lflow_add(lflows, op->od, S_ROUTER_IN_UNSNAT, 110, ds_cstr(match), "ct_snat;", - &op->lflow_dep_mgr, OBJDEP_TYPE_LPORT, + &op->lflow_dep_mgr, OBJDEP_TYPE_CT_LB, op->nbrp->name, op->od); ds_clear(match); @@ -13286,7 +13301,7 @@ build_lrouter_force_snat_flows_op(struct ovn_port *op, op->lrp_networks.ipv4_addrs[0].addr_s); ovn_lflow_add(lflows, op->od, S_ROUTER_OUT_SNAT, 110, ds_cstr(match), ds_cstr(actions), - &op->lflow_dep_mgr, OBJDEP_TYPE_LPORT, + &op->lflow_dep_mgr, OBJDEP_TYPE_CT_LB, op->nbrp->name, op->od); if (op->lrp_networks.n_ipv4_addrs > 1) { static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 5); @@ -13308,7 +13323,7 @@ build_lrouter_force_snat_flows_op(struct ovn_port *op, op->json_key, op->lrp_networks.ipv6_addrs[0].addr_s); ovn_lflow_add(lflows, op->od, S_ROUTER_IN_UNSNAT, 110, ds_cstr(match), "ct_snat;", - &op->lflow_dep_mgr, OBJDEP_TYPE_LPORT, + &op->lflow_dep_mgr, OBJDEP_TYPE_CT_LB, op->nbrp->name, op->od); ds_clear(match); @@ -13322,7 +13337,7 @@ build_lrouter_force_snat_flows_op(struct ovn_port *op, op->lrp_networks.ipv6_addrs[0].addr_s); ovn_lflow_add(lflows, op->od, S_ROUTER_OUT_SNAT, 110, ds_cstr(match), ds_cstr(actions), - &op->lflow_dep_mgr, OBJDEP_TYPE_LPORT, + &op->lflow_dep_mgr, OBJDEP_TYPE_CT_LB, op->nbrp->name, op->od); if (op->lrp_networks.n_ipv6_addrs > 2) { static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 5); @@ -14002,7 +14017,7 @@ build_ip_routing_flows_for_router_type_lsp(struct ovn_port *op, laddrs->ipv4_addrs[k].plen, NULL, false, 0, &peer->nbrp->header_, false, ROUTE_PRIO_OFFSET_CONNECTED, - &op->lflow_dep_mgr, OBJDEP_TYPE_LPORT, + &op->lflow_dep_mgr, OBJDEP_TYPE_CT_LB, op->nbsp->name, op->od); } } @@ -14284,7 +14299,7 @@ routable_addresses_to_lflows(struct lflow_data *lflows, ds_put_format(actions, "eth.dst = %s; next;", ra->laddrs[i].ea_s); ovn_lflow_add(lflows, peer->od, S_ROUTER_IN_ARP_RESOLVE, 100, ds_cstr(match), ds_cstr(actions), lflow_dep_mgr, - OBJDEP_TYPE_LPORT, res_name, res_od); + OBJDEP_TYPE_CT_LB, res_name, res_od); } } @@ -14393,7 +14408,7 @@ build_arp_resolve_flows_for_lrp( * Priority 2. */ build_lrouter_drop_own_dest(op, S_ROUTER_IN_ARP_RESOLVE, 2, true, - lflows); + lflows, OBJDEP_TYPE_LPORT); } /* This function adds ARP resolve flows related to a LSP. */ @@ -14550,15 +14565,63 @@ build_arp_resolve_flows_for_lsp( lflow_dep_mgr, OBJDEP_TYPE_LPORT, op->key, op->od); } + } + } +} - if (smap_get(&peer->od->nbr->options, "chassis") +static void +build_arp_resolve_flows_for_routable_addrs_throug_lsp( + struct ovn_port *op, struct lflow_data *lflows, + const struct hmap *lr_ports, + struct ds *match, struct ds *actions, + struct objdep_mgr *lflow_dep_mgr) +{ + ovs_assert(op->nbsp); + if (!lsp_is_enabled(op->nbsp)) { + return; + } + + if (!lsp_is_router(op->nbsp)) { + return; + } + + /* This is a logical switch port that connects to a router. */ + + /* The peer of this switch port is the router port for which + * we need to add logical flows such that it can resolve + * ARP entries for all the other router ports connected to + * the switch in question. */ + struct ovn_port *peer = ovn_port_get_peer(lr_ports, op); + if (!peer || !peer->nbrp) { + return; + } + + if (peer->od->nbr && + smap_get_bool(&peer->od->nbr->options, + "dynamic_neigh_routers", false)) { + return; + } + + for (size_t i = 0; i < op->od->n_router_ports; i++) { + struct ovn_port *router_port = + ovn_port_get_peer(lr_ports, op->od->router_ports[i]); + if (!router_port || !router_port->nbrp) { + continue; + } + + /* Skip the router port under consideration. */ + if (router_port == peer) { + continue; + } + + if (smap_get(&peer->od->nbr->options, "chassis") || peer->cr_port) { - routable_addresses_to_lflows(lflows, router_port, peer, - match, actions, - lflow_dep_mgr, op->key, op->od); - } + routable_addresses_to_lflows(lflows, router_port, peer, + match, actions, + lflow_dep_mgr, op->key, op->od); } } + } static void @@ -15437,43 +15500,6 @@ build_lrouter_ipv4_ip_input(struct ovn_port *op, op->nbrp->name, op->od); } - if (sset_count(&op->od->lb_ips->ips_v4_reachable)) { - ds_clear(match); - if (is_l3dgw_port(op)) { - ds_put_format(match, "is_chassis_resident(%s)", - op->cr_port->json_key); - } - - /* Create a single ARP rule for all IPs that are used as VIPs. */ - char *lb_ips_v4_as = lr_lb_address_set_ref(op->od->tunnel_key, - AF_INET); - build_lrouter_arp_flow(op->od, op, lb_ips_v4_as, - REG_INPORT_ETH_ADDR, - match, false, 90, NULL, lflows, - &op->lflow_dep_mgr, OBJDEP_TYPE_LPORT, - op->nbrp->name, op->od); - free(lb_ips_v4_as); - } - - if (sset_count(&op->od->lb_ips->ips_v6_reachable)) { - ds_clear(match); - - if (is_l3dgw_port(op)) { - ds_put_format(match, "is_chassis_resident(%s)", - op->cr_port->json_key); - } - - /* Create a single ND rule for all IPs that are used as VIPs. */ - char *lb_ips_v6_as = lr_lb_address_set_ref(op->od->tunnel_key, - AF_INET6); - build_lrouter_nd_flow(op->od, op, "nd_na", lb_ips_v6_as, NULL, - REG_INPORT_ETH_ADDR, match, false, 90, - NULL, lflows, meter_groups, - &op->lflow_dep_mgr, OBJDEP_TYPE_LPORT, - op->nbrp->name, op->od); - free(lb_ips_v6_as); - } - if (!op->od->is_gw_router && !op->od->n_l3dgw_ports) { /* UDP/TCP/SCTP port unreachable. */ for (int i = 0; i < op->lrp_networks.n_ipv4_addrs; i++) { @@ -15557,19 +15583,6 @@ build_lrouter_ipv4_ip_input(struct ovn_port *op, } } - /* Drop IP traffic destined to router owned IPs except if the IP is - * also a SNAT IP. Those are dropped later, in stage - * "lr_in_arp_resolve", if unSNAT was unsuccessful. - * - * If op->od->lb_force_snat_router_ip is true, it means the IP of the - * router port is also SNAT IP. - * - * Priority 60. - */ - if (!op->od->lb_force_snat_router_ip) { - build_lrouter_drop_own_dest(op, S_ROUTER_IN_IP_INPUT, 60, false, - lflows); - } /* ARP / ND handling for external IP addresses. * * DNAT and SNAT IP addresses are external IP addresses that need ARP @@ -15618,6 +15631,71 @@ build_lrouter_ipv4_ip_input(struct ovn_port *op, } } +static void +build_lrouter_ipv4_ip_input_lb_related(struct ovn_port *op, + struct lflow_data *lflows, + struct ds *match, + const struct shash *meter_groups) +{ + ovs_assert(op->nbrp); + /* No ingress packets are accepted on a chassisredirect + * port, so no need to program flows for that port. */ + if (is_cr_port(op)) { + return; + } + + if (sset_count(&op->od->lb_ips->ips_v4_reachable)) { + ds_clear(match); + if (is_l3dgw_port(op)) { + ds_put_format(match, "is_chassis_resident(%s)", + op->cr_port->json_key); + } + + /* Create a single ARP rule for all IPs that are used as VIPs. */ + char *lb_ips_v4_as = lr_lb_address_set_ref(op->od->tunnel_key, + AF_INET); + build_lrouter_arp_flow(op->od, op, lb_ips_v4_as, + REG_INPORT_ETH_ADDR, + match, false, 90, NULL, lflows, + &op->lflow_dep_mgr, OBJDEP_TYPE_LB, + op->nbrp->name, op->od); + free(lb_ips_v4_as); + } + + if (sset_count(&op->od->lb_ips->ips_v6_reachable)) { + ds_clear(match); + + if (is_l3dgw_port(op)) { + ds_put_format(match, "is_chassis_resident(%s)", + op->cr_port->json_key); + } + + /* Create a single ND rule for all IPs that are used as VIPs. */ + char *lb_ips_v6_as = lr_lb_address_set_ref(op->od->tunnel_key, + AF_INET6); + build_lrouter_nd_flow(op->od, op, "nd_na", lb_ips_v6_as, NULL, + REG_INPORT_ETH_ADDR, match, false, 90, + NULL, lflows, meter_groups, + &op->lflow_dep_mgr, OBJDEP_TYPE_LB, + op->nbrp->name, op->od); + free(lb_ips_v6_as); + } + + /* Drop IP traffic destined to router owned IPs except if the IP is + * also a SNAT IP. Those are dropped later, in stage + * "lr_in_arp_resolve", if unSNAT was unsuccessful. + * + * If op->od->lb_force_snat_router_ip is true, it means the IP of the + * router port is also SNAT IP. + * + * Priority 60. + */ + if (!op->od->lb_force_snat_router_ip) { + build_lrouter_drop_own_dest(op, S_ROUTER_IN_IP_INPUT, 60, false, + lflows, OBJDEP_TYPE_LB); + } +} + static void build_lrouter_in_unsnat_match(struct ovn_datapath *od, const struct nbrec_nat *nat, struct ds *match, @@ -16898,12 +16976,27 @@ build_lswitch_and_lrouter_iterate_by_lsp(struct ovn_port *op, build_lswitch_external_port(op, lflows, &op->lflow_dep_mgr); build_lswitch_ip_unicast_lookup(op, lflows, actions, match, &op->lflow_dep_mgr); + /* Build Logical Router Flows. */ - build_ip_routing_flows_for_router_type_lsp(op, lr_ports, lflows); build_arp_resolve_flows_for_lsp(op, lflows, lr_ports, match, actions, &op->lflow_dep_mgr); } +static void +build_lb_related_iterate_by_lsp(struct ovn_port *op, + const struct hmap *lr_ports, + struct ds *match, + struct ds *actions, + struct lflow_data *lflows) +{ + ovs_assert(op->nbsp); + build_lswitch_ip_unicast_lookup_lb_vips(op, lflows, + &op->lflow_dep_mgr); + build_ip_routing_flows_for_router_type_lsp(op, lr_ports, lflows); + build_arp_resolve_flows_for_routable_addrs_throug_lsp( + op, lflows, lr_ports, match, actions, &op->lflow_dep_mgr); +} + /* Helper function to combine all lflow generation which is iterated by logical * router port. All the flows built in this function are Logical Router flows. */ @@ -16929,6 +17022,15 @@ build_lswitch_and_lrouter_iterate_by_lrp(struct ovn_port *op, lsi->meter_groups); build_lrouter_ipv4_ip_input(op, lsi->lflows, &lsi->match, &lsi->actions, lsi->meter_groups); +} + +static void +build_lb_related_iterate_by_lrp(struct ovn_port *op, + struct lswitch_flow_build_info *lsi) +{ + ovs_assert(op->nbrp); + build_lrouter_ipv4_ip_input_lb_related(op, lsi->lflows, &lsi->match, + lsi->meter_groups); build_lrouter_force_snat_flows_op(op, lsi->lflows, &lsi->match, &lsi->actions); } @@ -16993,6 +17095,9 @@ build_lflows_thread(void *arg) &lsi->match, &lsi->actions, lsi->lflows); + build_lb_related_iterate_by_lsp(op, lsi->lr_ports, + &lsi->match, &lsi->actions, + lsi->lflows); } } for (bnum = control->id; @@ -17005,6 +17110,7 @@ build_lflows_thread(void *arg) return NULL; } build_lswitch_and_lrouter_iterate_by_lrp(op, lsi); + build_lb_related_iterate_by_lrp(op, lsi); } } for (bnum = control->id; @@ -17193,9 +17299,12 @@ build_lswitch_and_lrouter_flows(const struct ovn_datapaths *ls_datapaths, lsi.meter_groups, &lsi.match, &lsi.actions, lsi.lflows); + build_lb_related_iterate_by_lsp(op, lsi.lr_ports, &lsi.match, + &lsi.actions, lsi.lflows); } HMAP_FOR_EACH (op, key_node, lr_ports) { build_lswitch_and_lrouter_iterate_by_lrp(op, &lsi); + build_lb_related_iterate_by_lrp(op, &lsi); } stopwatch_stop(LFLOWS_PORTS_STOPWATCH_NAME, time_msec()); stopwatch_start(LFLOWS_LBS_STOPWATCH_NAME, time_msec()); From patchwork Fri Aug 18 08:58:33 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Numan Siddique X-Patchwork-Id: 1822764 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.137; helo=smtp4.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp4.osuosl.org (smtp4.osuosl.org [140.211.166.137]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4RRwmr1M38z1ygH for ; Fri, 18 Aug 2023 18:59:56 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 4B6C041AE9; Fri, 18 Aug 2023 08:59:53 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 4B6C041AE9 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id gRNObYHpA2IQ; Fri, 18 Aug 2023 08:59:50 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp4.osuosl.org (Postfix) with ESMTPS id 5EF97427FC; Fri, 18 Aug 2023 08:59:49 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 5EF97427FC Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 18B8DC0072; Fri, 18 Aug 2023 08:59:49 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 0D854C0039 for ; Fri, 18 Aug 2023 08:59:48 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 47BC7615B5 for ; Fri, 18 Aug 2023 08:58:41 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 47BC7615B5 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ifGJTYzxx-Vg for ; Fri, 18 Aug 2023 08:58:40 +0000 (UTC) Received: from relay8-d.mail.gandi.net (relay8-d.mail.gandi.net [217.70.183.201]) by smtp3.osuosl.org (Postfix) with ESMTPS id 88061615DE for ; Fri, 18 Aug 2023 08:58:39 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 88061615DE Received: by mail.gandi.net (Postfix) with ESMTPSA id E2C791BF208; Fri, 18 Aug 2023 08:58:36 +0000 (UTC) From: numans@ovn.org To: dev@openvswitch.org Date: Fri, 18 Aug 2023 14:28:33 +0530 Message-Id: <20230818085833.1031109-1-numans@ovn.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230818085606.1030792-1-numans@ovn.org> References: <20230818085606.1030792-1-numans@ovn.org> MIME-Version: 1.0 X-GND-Sasl: numans@ovn.org Subject: [ovs-dev] [PATCH ovn v6 13/16] northd: Refactor the northd change tracking. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Numan Siddique Previous commits made of objdep_mgr to store the lflow references for ovn_port, ovn_datapath and load balancers. This patch refactors the northd change tracking to make it easier to pass the northd changed data to lflow engine for lflow generation. In order to incrementally handle the northd changes in the lflow engine, it is sufficient to provide the below changed data to lflow engine node - changed ovn_ports - changed ovn_datapaths - changed load balancers. For every changed resource, lflow northd change handler can get the objdep_mgr for each of these resources and can regenerate the lflows easily. 'lflow' changes for ovn_port is already handled. Upcoming commits will handle the ovn_datapath and load balancer changes in lflow engine. Signed-off-by: Numan Siddique --- northd/en-lflow.c | 8 +- northd/en-northd.c | 2 +- northd/northd.c | 286 +++++++++++++++++++++++---------------------- northd/northd.h | 49 +++++--- 4 files changed, 181 insertions(+), 164 deletions(-) diff --git a/northd/en-lflow.c b/northd/en-lflow.c index b7538d6382..dacaaa549a 100644 --- a/northd/en-lflow.c +++ b/northd/en-lflow.c @@ -106,7 +106,7 @@ lflow_northd_handler(struct engine_node *node, } /* Fall back to recompute if lb related data has changed. */ - if (northd_data->lb_changed) { + if (northd_data->trk_northd_changes.lb_changed) { return false; } @@ -116,9 +116,9 @@ lflow_northd_handler(struct engine_node *node, struct lflow_input lflow_input; lflow_get_input_data(node, &lflow_input); - if (!lflow_handle_northd_ls_changes(eng_ctx->ovnsb_idl_txn, - &northd_data->tracked_ls_changes, - &lflow_input, lflow_data)) { + if (!lflow_handle_northd_changes(eng_ctx->ovnsb_idl_txn, + &northd_data->trk_northd_changes, + &lflow_input, lflow_data)) { return false; } diff --git a/northd/en-northd.c b/northd/en-northd.c index 3be0f79e19..7a4a1d8629 100644 --- a/northd/en-northd.c +++ b/northd/en-northd.c @@ -303,7 +303,7 @@ northd_lb_data_handler_post_od(struct engine_node *node, void *data) /* Indicate the depedendant engine nodes that load balancer/group * related data has changed (including association to logical * switch/router). */ - nd->lb_changed = true; + nd->trk_northd_changes.lb_changed = true; engine_set_node_state(node, EN_UPDATED); return true; } diff --git a/northd/northd.c b/northd/northd.c index 2ec768ab71..3aefe42334 100644 --- a/northd/northd.c +++ b/northd/northd.c @@ -4994,34 +4994,50 @@ build_ports(struct ovsdb_idl_txn *ovnsb_txn, } static void -destroy_tracked_ls_change(struct ls_change *ls_change) +destroy_tracked_ovn_ports(struct tracked_ovn_ports *trk_ovn_ports) { - struct ovn_port *op; - LIST_FOR_EACH (op, list, &ls_change->added_ports) { - ovs_list_remove(&op->list); + struct tracked_ovn_port *trk_op; + HMAP_FOR_EACH_POP (trk_op, hmap_node, &trk_ovn_ports->deleted) { + ovn_port_destroy_orphan(trk_op->op); + free(trk_op); } - LIST_FOR_EACH (op, list, &ls_change->updated_ports) { - ovs_list_remove(&op->list); + + HMAP_FOR_EACH_POP (trk_op, hmap_node, &trk_ovn_ports->created) { + free(trk_op); } - LIST_FOR_EACH_SAFE (op, list, &ls_change->deleted_ports) { - ovs_list_remove(&op->list); - ovn_port_destroy_orphan(op); + + HMAP_FOR_EACH_POP (trk_op, hmap_node, &trk_ovn_ports->updated) { + free(trk_op); } } void destroy_northd_data_tracked_changes(struct northd_data *nd) { - struct ls_change *ls_change; - LIST_FOR_EACH_SAFE (ls_change, list_node, - &nd->tracked_ls_changes.updated) { - destroy_tracked_ls_change(ls_change); - ovs_list_remove(&ls_change->list_node); - free(ls_change); - } + struct northd_tracked_data *trk_changes = &nd->trk_northd_changes; + destroy_tracked_ovn_ports(&trk_changes->trk_ovn_ports); nd->change_tracked = false; - nd->lb_changed = false; + trk_changes->lb_changed = false; +} + +static void +add_op_to_northd_tracked_ports(struct hmap *tracked_ovn_ports, + struct ovn_port *op) +{ + struct tracked_ovn_port *trk_op; + uint32_t hash = hash_string(op->key, 0); + HMAP_FOR_EACH_WITH_HASH (trk_op, hmap_node, hash, tracked_ovn_ports) { + if (!strcmp(trk_op->op->key, op->key)) { + break; + } + } + + if (!trk_op) { + trk_op = xzalloc(sizeof *trk_op); + trk_op->op = op; + hmap_insert(tracked_ovn_ports, &trk_op->hmap_node, hash); + } } /* Check if a changed LSP can be handled incrementally within the I-P engine @@ -5272,7 +5288,7 @@ ls_handle_lsp_changes(struct ovsdb_idl_txn *ovnsb_idl_txn, const struct northd_input *ni, struct northd_data *nd, struct ovn_datapath *od, - struct ls_change *ls_change, + struct tracked_ovn_ports *trk_ports, bool *updated) { bool ls_ports_changed = false; @@ -5333,8 +5349,7 @@ ls_handle_lsp_changes(struct ovsdb_idl_txn *ovnsb_idl_txn, if (!op) { goto fail; } - ovs_list_push_back(&ls_change->added_ports, - &op->list); + add_op_to_northd_tracked_ports(&trk_ports->created, op); } else if (ls_port_has_changed(op->nbsp, new_nbsp)) { /* Existing port updated */ bool temp = false; @@ -5365,7 +5380,8 @@ ls_handle_lsp_changes(struct ovsdb_idl_txn *ovnsb_idl_txn, ni->sbrec_chassis_by_hostname)) { goto fail; } - ovs_list_push_back(&ls_change->updated_ports, &op->list); + + add_op_to_northd_tracked_ports(&trk_ports->updated, op); } op->visited = true; } @@ -5374,20 +5390,19 @@ ls_handle_lsp_changes(struct ovsdb_idl_txn *ovnsb_idl_txn, HMAP_FOR_EACH_SAFE (op, dp_node, &od->ports) { if (!op->visited) { if (!op->lsp_can_be_inc_processed) { - goto fail_clean_deleted; + goto fail; } if (sset_contains(&nd->svc_monitor_lsps, op->key)) { /* This port was used for svc monitor, which may be * impacted by this deletion. Fallback to recompute. */ - goto fail_clean_deleted; + goto fail; } - ovs_list_push_back(&ls_change->deleted_ports, - &op->list); + add_op_to_northd_tracked_ports(&trk_ports->deleted, op); hmap_remove(&nd->ls_ports, &op->key_node); hmap_remove(&od->ports, &op->dp_node); sbrec_port_binding_delete(op->sb); delete_fdb_entry(ni->sbrec_fdb_by_dp_and_port, od->tunnel_key, - op->tunnel_key); + op->tunnel_key); } } @@ -5404,23 +5419,19 @@ ls_handle_lsp_changes(struct ovsdb_idl_txn *ovnsb_idl_txn, only_rports = (od->n_router_ports && (od->n_router_ports == hmap_count(&od->ports))); if (only_rports) { - goto fail_clean_deleted; + goto fail; } - if (!ovs_list_is_empty(&ls_change->added_ports) || - !ovs_list_is_empty(&ls_change->updated_ports) || - !ovs_list_is_empty(&ls_change->deleted_ports)) { + if (!hmap_is_empty(&trk_ports->created) + || !hmap_is_empty(&trk_ports->updated) + || !hmap_is_empty(&trk_ports->deleted)) { *updated = true; } return true; -fail_clean_deleted: - LIST_FOR_EACH_POP (op, list, &ls_change->deleted_ports) { - ovn_port_destroy_orphan(op); - } - fail: + destroy_tracked_ovn_ports(trk_ports); return false; } @@ -5438,8 +5449,8 @@ northd_handle_ls_changes(struct ovsdb_idl_txn *ovnsb_idl_txn, struct northd_data *nd) { const struct nbrec_logical_switch *changed_ls; - struct ls_change *ls_change = NULL; + struct northd_tracked_data *trk_nd_changes = &nd->trk_northd_changes; NBREC_LOGICAL_SWITCH_TABLE_FOR_EACH_TRACKED (changed_ls, ni->nbrec_logical_switch_table) { if (nbrec_logical_switch_is_new(changed_ls) || @@ -5462,33 +5473,18 @@ northd_handle_ls_changes(struct ovsdb_idl_txn *ovnsb_idl_txn, goto fail; } - ls_change = xzalloc(sizeof *ls_change); - ls_change->od = od; - ovs_list_init(&ls_change->added_ports); - ovs_list_init(&ls_change->deleted_ports); - ovs_list_init(&ls_change->updated_ports); - bool updated = false; if (!ls_handle_lsp_changes(ovnsb_idl_txn, changed_ls, - ni, nd, od, ls_change, + ni, nd, od, &trk_nd_changes->trk_ovn_ports, &updated)) { - destroy_tracked_ls_change(ls_change); - free(ls_change); goto fail; } if (updated) { - ovs_list_push_back(&nd->tracked_ls_changes.updated, - &ls_change->list_node); - } else { - free(ls_change); + nd->change_tracked = true; } } - if (!ovs_list_is_empty(&nd->tracked_ls_changes.updated)) { - nd->change_tracked = true; - } - return true; fail: @@ -17918,104 +17914,107 @@ unlink_and_sync_lflows(struct ovsdb_idl_txn *sb_txn, struct ovn_datapath *od, lflow_data, lflowdep_mgr); } -bool lflow_handle_northd_ls_changes(struct ovsdb_idl_txn *ovnsb_txn, - struct tracked_ls_changes *ls_changes, - struct lflow_input *lflow_input, - struct lflow_data *lflow_data) +bool lflow_handle_northd_changes(struct ovsdb_idl_txn *ovnsb_txn, + struct northd_tracked_data *trk_nd_changes, + struct lflow_input *lflow_input, + struct lflow_data *lflow_data) { - struct ls_change *ls_change; + struct tracked_ovn_ports *trk_ovn_ports = &trk_nd_changes->trk_ovn_ports; + + struct tracked_ovn_port *trk_op; + HMAP_FOR_EACH (trk_op, hmap_node, &trk_ovn_ports->deleted) { + struct ovn_port *op = trk_op->op; + + /* unlink old lflows. */ + unlink_and_sync_lflows(ovnsb_txn, op->od, OBJDEP_TYPE_LPORT, + op->nbsp->name, lflow_input, lflow_data, + &op->lflow_dep_mgr); + objdep_mgr_clear(&op->lflow_dep_mgr); + + /* No need to update SB multicast groups, thanks to weak + * references. */ + } + + HMAP_FOR_EACH (trk_op, hmap_node, &trk_ovn_ports->updated) { + struct ovn_port *op = trk_op->op; + + /* unlink old lflows. */ + unlink_lflows(op->od, OBJDEP_TYPE_LPORT, op->nbsp->name, + lflow_data, &op->lflow_dep_mgr); + + /* Generate new lflows. */ + struct ds match = DS_EMPTY_INITIALIZER; + struct ds actions = DS_EMPTY_INITIALIZER; + build_lswitch_and_lrouter_iterate_by_lsp(op, lflow_input->ls_ports, + lflow_input->lr_ports, + lflow_input->meter_groups, + &match, &actions, + lflow_data); + ds_destroy(&match); + ds_destroy(&actions); + + /* SB port_binding is not deleted, so don't update SB multicast + * groups. */ + + /* Sync the new flows to SB. */ + sync_lflows_for_res(ovnsb_txn, OBJDEP_TYPE_LPORT, op->nbsp->name, + lflow_input, lflow_data, &op->lflow_dep_mgr); + } + + HMAP_FOR_EACH (trk_op, hmap_node, &trk_ovn_ports->created) { + struct ovn_port *op = trk_op->op; - LIST_FOR_EACH (ls_change, list_node, &ls_changes->updated) { const struct sbrec_multicast_group *sbmc_flood = mcast_group_lookup(lflow_input->sbrec_mcast_group_by_name_dp, - MC_FLOOD, ls_change->od->sb); + MC_FLOOD, op->od->sb); const struct sbrec_multicast_group *sbmc_flood_l2 = mcast_group_lookup(lflow_input->sbrec_mcast_group_by_name_dp, - MC_FLOOD_L2, ls_change->od->sb); + MC_FLOOD_L2, op->od->sb); const struct sbrec_multicast_group *sbmc_unknown = mcast_group_lookup(lflow_input->sbrec_mcast_group_by_name_dp, - MC_UNKNOWN, ls_change->od->sb); - - struct ovn_port *op; - LIST_FOR_EACH (op, list, &ls_change->deleted_ports) { - /* unlink old lflows. */ - unlink_and_sync_lflows(ovnsb_txn, op->od, OBJDEP_TYPE_LPORT, - op->nbsp->name, lflow_input, lflow_data, - &op->lflow_dep_mgr); - objdep_mgr_clear(&op->lflow_dep_mgr); - - /* No need to update SB multicast groups, thanks to weak - * references. */ - } - - LIST_FOR_EACH (op, list, &ls_change->updated_ports) { - /* unlink old lflows. */ - unlink_lflows(op->od, OBJDEP_TYPE_LPORT, op->nbsp->name, - lflow_data, &op->lflow_dep_mgr); - - /* Generate new lflows. */ - struct ds match = DS_EMPTY_INITIALIZER; - struct ds actions = DS_EMPTY_INITIALIZER; - build_lswitch_and_lrouter_iterate_by_lsp(op, lflow_input->ls_ports, - lflow_input->lr_ports, - lflow_input->meter_groups, - &match, &actions, - lflow_data); - ds_destroy(&match); - ds_destroy(&actions); - - /* SB port_binding is not deleted, so don't update SB multicast - * groups. */ - - /* Sync the new flows to SB. */ - sync_lflows_for_res(ovnsb_txn, OBJDEP_TYPE_LPORT, - op->nbsp->name, lflow_input, - lflow_data, &op->lflow_dep_mgr); - } - - LIST_FOR_EACH (op, list, &ls_change->added_ports) { - struct ds match = DS_EMPTY_INITIALIZER; - struct ds actions = DS_EMPTY_INITIALIZER; - build_lswitch_and_lrouter_iterate_by_lsp(op, lflow_input->ls_ports, - lflow_input->lr_ports, - lflow_input->meter_groups, - &match, &actions, - lflow_data); - ds_destroy(&match); - ds_destroy(&actions); - - /* Update SB multicast groups for the new port. */ - if (!sbmc_flood) { - sbmc_flood = create_sb_multicast_group(ovnsb_txn, - ls_change->od->sb, MC_FLOOD, OVN_MCAST_FLOOD_TUNNEL_KEY); + MC_UNKNOWN, op->od->sb); + + struct ds match = DS_EMPTY_INITIALIZER; + struct ds actions = DS_EMPTY_INITIALIZER; + build_lswitch_and_lrouter_iterate_by_lsp(op, lflow_input->ls_ports, + lflow_input->lr_ports, + lflow_input->meter_groups, + &match, &actions, + lflow_data); + ds_destroy(&match); + ds_destroy(&actions); + + /* Update SB multicast groups for the new port. */ + if (!sbmc_flood) { + sbmc_flood = create_sb_multicast_group(ovnsb_txn, + op->od->sb, MC_FLOOD, OVN_MCAST_FLOOD_TUNNEL_KEY); + } + sbrec_multicast_group_update_ports_addvalue(sbmc_flood, op->sb); + + if (!sbmc_flood_l2) { + sbmc_flood_l2 = create_sb_multicast_group(ovnsb_txn, + op->od->sb, MC_FLOOD_L2, + OVN_MCAST_FLOOD_L2_TUNNEL_KEY); + } + sbrec_multicast_group_update_ports_addvalue(sbmc_flood_l2, op->sb); + + if (op->has_unknown) { + if (!sbmc_unknown) { + sbmc_unknown = create_sb_multicast_group(ovnsb_txn, + op->od->sb, MC_UNKNOWN, + OVN_MCAST_UNKNOWN_TUNNEL_KEY); } - sbrec_multicast_group_update_ports_addvalue(sbmc_flood, op->sb); - - if (!sbmc_flood_l2) { - sbmc_flood_l2 = create_sb_multicast_group(ovnsb_txn, - ls_change->od->sb, MC_FLOOD_L2, - OVN_MCAST_FLOOD_L2_TUNNEL_KEY); - } - sbrec_multicast_group_update_ports_addvalue(sbmc_flood_l2, op->sb); - - if (op->has_unknown) { - if (!sbmc_unknown) { - sbmc_unknown = create_sb_multicast_group(ovnsb_txn, - ls_change->od->sb, MC_UNKNOWN, - OVN_MCAST_UNKNOWN_TUNNEL_KEY); - } - sbrec_multicast_group_update_ports_addvalue(sbmc_unknown, - op->sb); - } - - /* Sync the newly added flows to SB. */ - sync_lflows_for_res(ovnsb_txn, OBJDEP_TYPE_LPORT, - op->nbsp->name, lflow_input, - lflow_data, &op->lflow_dep_mgr); + sbrec_multicast_group_update_ports_addvalue(sbmc_unknown, + op->sb); } + + /* Sync the newly added flows to SB. */ + sync_lflows_for_res(ovnsb_txn, OBJDEP_TYPE_LPORT, + op->nbsp->name, lflow_input, + lflow_data, &op->lflow_dep_mgr); } - return true; + return true; } /* Each port group in Port_Group table in OVN_Northbound has a corresponding @@ -18896,7 +18895,10 @@ northd_init(struct northd_data *data) sset_init(&data->svc_monitor_lsps); hmap_init(&data->svc_monitor_map); data->change_tracked = false; - ovs_list_init(&data->tracked_ls_changes.updated); + + hmap_init(&data->trk_northd_changes.trk_ovn_ports.created); + hmap_init(&data->trk_northd_changes.trk_ovn_ports.updated); + hmap_init(&data->trk_northd_changes.trk_ovn_ports.deleted); } void @@ -18949,6 +18951,10 @@ northd_destroy(struct northd_data *data) destroy_debug_config(); sset_destroy(&data->svc_monitor_lsps); + + hmap_destroy(&data->trk_northd_changes.trk_ovn_ports.created); + hmap_destroy(&data->trk_northd_changes.trk_ovn_ports.updated); + hmap_destroy(&data->trk_northd_changes.trk_ovn_ports.deleted); } void diff --git a/northd/northd.h b/northd/northd.h index d0992f142e..1634463a67 100644 --- a/northd/northd.h +++ b/northd/northd.h @@ -19,6 +19,7 @@ #include "lib/ovn-sb-idl.h" #include "lib/ovn-util.h" #include "lib/ovs-atomic.h" +#include "lib/hmapx.h" #include "lib/objdep.h" #include "lib/sset.h" #include "northd/ipam.h" @@ -87,21 +88,33 @@ struct ovn_datapaths { struct ovn_datapath **array; }; -/* Track what's changed for a single LS. - * Now only track port changes. */ -struct ls_change { - struct ovs_list list_node; - struct ovn_datapath *od; - struct ovs_list added_ports; - struct ovs_list deleted_ports; - struct ovs_list updated_ports; +/* Represents a tracked ovn_port. */ +struct tracked_ovn_port { + struct hmap_node hmap_node; + struct ovn_port *op; }; -/* Track what's changed for logical switches. - * Now only track updated ones (added or deleted may be supported in the - * future). */ -struct tracked_ls_changes { - struct ovs_list updated; /* Contains struct ls_change */ +struct tracked_ovn_ports { + /* tracked created ports. + * hmap node data is 'struct tracked_ovn_port *' */ + struct hmap created; + + /* tracked updated ports. + * hmap node data is 'struct tracked_ovn_port *' */ + struct hmap updated; + + /* tracked deleted ports. + * hmap node data is 'struct tracked_ovn_port *' */ + struct hmap deleted; +}; + +/* Track what's changed in the northd engine node. + * Now only tracks ovn_ports (of vif type) - created, updated + * and deleted and indicates if load balancers have changed. */ +struct northd_tracked_data { + struct tracked_ovn_ports trk_ovn_ports; + bool lb_changed; /* Indicates if load balancers changed or association of + * load balancer to logical switch/router changed. */ }; struct northd_data { @@ -120,9 +133,7 @@ struct northd_data { struct sset svc_monitor_lsps; struct hmap svc_monitor_map; bool change_tracked; - struct tracked_ls_changes tracked_ls_changes; - bool lb_changed; /* Indicates if load balancers changed or association of - * load balancer to logical switch/router changed. */ + struct northd_tracked_data trk_northd_changes; }; struct lflow_data { @@ -352,9 +363,9 @@ void northd_indices_create(struct northd_data *data, void build_lflows(struct ovsdb_idl_txn *ovnsb_txn, struct lflow_input *input_data, struct lflow_data *lflow_data); -bool lflow_handle_northd_ls_changes(struct ovsdb_idl_txn *ovnsb_txn, - struct tracked_ls_changes *, - struct lflow_input *, struct lflow_data *); +bool lflow_handle_northd_changes(struct ovsdb_idl_txn *ovnsb_txn, + struct northd_tracked_data *, + struct lflow_input *, struct lflow_data *); bool northd_handle_sb_port_binding_changes( const struct sbrec_port_binding_table *, struct hmap *ls_ports); From patchwork Fri Aug 18 08:58:38 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Numan Siddique X-Patchwork-Id: 1822765 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::136; helo=smtp3.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4RRwn65P51z1ygW for ; Fri, 18 Aug 2023 19:00:10 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id AEDBA61674; Fri, 18 Aug 2023 09:00:07 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org AEDBA61674 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id VbZI8tID7nQ4; Fri, 18 Aug 2023 09:00:01 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp3.osuosl.org (Postfix) with ESMTPS id 9411461647; Fri, 18 Aug 2023 09:00:00 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 9411461647 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 5E5F2C0039; Fri, 18 Aug 2023 09:00:00 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 0413AC0039 for ; Fri, 18 Aug 2023 08:59:59 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id A39A961580 for ; Fri, 18 Aug 2023 08:58:51 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org A39A961580 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 7UjiO3IrQMdk for ; Fri, 18 Aug 2023 08:58:48 +0000 (UTC) Received: from relay5-d.mail.gandi.net (relay5-d.mail.gandi.net [IPv6:2001:4b98:dc4:8::225]) by smtp3.osuosl.org (Postfix) with ESMTPS id 26C9C615E4 for ; Fri, 18 Aug 2023 08:58:47 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 26C9C615E4 Received: by mail.gandi.net (Postfix) with ESMTPSA id BC7021C0002; Fri, 18 Aug 2023 08:58:43 +0000 (UTC) From: numans@ovn.org To: dev@openvswitch.org Date: Fri, 18 Aug 2023 14:28:38 +0530 Message-Id: <20230818085838.1031124-1-numans@ovn.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230818085606.1030792-1-numans@ovn.org> References: <20230818085606.1030792-1-numans@ovn.org> MIME-Version: 1.0 X-GND-Sasl: numans@ovn.org Subject: [ovs-dev] [PATCH ovn v6 14/16] northd: Handle load balancer change in lflow engine. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Numan Siddique If a load balancer or load balancer group is changed, then the lflow engine handles these changes incrementally. This is possible because we maintain lb to lflow references (using objdep_mgr) for each lb in the 'struct lb_datapaths'. northd engine adds the changed load balancers in the northd tracked data. And for each tracked lb, lflow engine handler - Deletes the old lflow references and - Rebuilds the lflows. This patch also rebuilds the logical flows for the logical switch and logical router datapaths associated with the changed load balancers. Since we already maintain the logical flows related to load balancers in a separte objtype in the objdep_mgr of each datapath, it becomes easier to rebuild on these lflows. Below are the scale testing results done with all of these patches applied using ovn-heater. The test ran the scenario - ocp-500-density-heavy.yml [1]. With these patches applied (with load balancer I-P handling in both northd and lflow engine nodes) the resuts are: ------------------------------------------------------------------------------------------------------------------------------------------------------- Min (s) Median (s) 90%ile (s) 99%ile (s) Max (s) Mean (s) Total (s) Count Failed ------------------------------------------------------------------------------------------------------------------------------------------------------- Iteration Total 0.135651 1.130527 1.179357 1.201410 2.180203 0.674606 84.325717 125 0 Namespace.add_ports 0.005218 0.005678 0.006457 0.018936 0.020812 0.006182 0.772796 125 0 WorkerNode.bind_port 0.033631 0.043287 0.051171 0.058223 0.062819 0.043839 10.959757 250 0 WorkerNode.ping_port 0.005460 0.006791 1.041434 1.064807 1.069957 0.274352 68.587878 250 0 ------------------------------------------------------------------------------------------------------------------------------------------------------- The results with the present main are: ------------------------------------------------------------------------------------------------------------------------------------------------------- Min (s) Median (s) 90%ile (s) 99%ile (s) Max (s) Mean (s) Total (s) Count Failed ------------------------------------------------------------------------------------------------------------------------------------------------------- Iteration Total 4.377260 6.486962 7.502040 8.322587 8.334701 6.559002 819.875306 125 0 Namespace.add_ports 0.005112 0.005484 0.005953 0.009153 0.011452 0.005662 0.707752 125 0 WorkerNode.bind_port 0.035360 0.042732 0.049152 0.053698 0.056635 0.043215 10.803700 250 0 WorkerNode.ping_port 0.005338 1.599904 7.229649 7.798039 8.206537 3.209860 802.464911 250 0 ------------------------------------------------------------------------------------------------------------------------------------------------------- Signed-off-by: Numan Siddique --- northd/en-lflow.c | 17 +- northd/en-northd.c | 21 ++- northd/en-sync-sb.c | 5 +- northd/northd.c | 450 +++++++++++++++++++++++++++++++++++++++++--- northd/northd.h | 45 ++++- tests/ovn-northd.at | 159 ++++++++++++---- 6 files changed, 608 insertions(+), 89 deletions(-) diff --git a/northd/en-lflow.c b/northd/en-lflow.c index dacaaa549a..56fe564c82 100644 --- a/northd/en-lflow.c +++ b/northd/en-lflow.c @@ -105,20 +105,21 @@ lflow_northd_handler(struct engine_node *node, return false; } - /* Fall back to recompute if lb related data has changed. */ - if (northd_data->trk_northd_changes.lb_changed) { - return false; - } - const struct engine_context *eng_ctx = engine_get_context(); struct lflow_data *lflow_data = data; struct lflow_input lflow_input; lflow_get_input_data(node, &lflow_input); - if (!lflow_handle_northd_changes(eng_ctx->ovnsb_idl_txn, - &northd_data->trk_northd_changes, - &lflow_input, lflow_data)) { + if (!lflow_handle_northd_port_changes(eng_ctx->ovnsb_idl_txn, + &northd_data->trk_northd_changes.trk_ovn_ports, + &lflow_input, lflow_data)) { + return false; + } + + if (!lflow_handle_northd_lb_changes(eng_ctx->ovnsb_idl_txn, + &northd_data->trk_northd_changes.trk_lbs, + &lflow_input, lflow_data)) { return false; } diff --git a/northd/en-northd.c b/northd/en-northd.c index 7a4a1d8629..651c53a6ce 100644 --- a/northd/en-northd.c +++ b/northd/en-northd.c @@ -270,11 +270,15 @@ northd_lb_data_handler_pre_od(struct engine_node *node, void *data) &nd->ls_datapaths, &nd->lr_datapaths, &nd->lb_datapaths_map, - &nd->lb_group_datapaths_map)) { + &nd->lb_group_datapaths_map, + &nd->trk_northd_changes)) { return false; } - engine_set_node_state(node, EN_UPDATED); + if (northd_has_tracked_data(&nd->trk_northd_changes)) { + nd->change_tracked = true; + engine_set_node_state(node, EN_UPDATED); + } return true; } @@ -296,15 +300,16 @@ northd_lb_data_handler_post_od(struct engine_node *node, void *data) &nd->ls_datapaths, &nd->lr_datapaths, &nd->lb_datapaths_map, - &nd->lb_group_datapaths_map)) { + &nd->lb_group_datapaths_map, + &nd->trk_northd_changes)) { return false; } - /* Indicate the depedendant engine nodes that load balancer/group - * related data has changed (including association to logical - * switch/router). */ - nd->trk_northd_changes.lb_changed = true; - engine_set_node_state(node, EN_UPDATED); + if (northd_has_tracked_data(&nd->trk_northd_changes)) { + nd->change_tracked = true; + engine_set_node_state(node, EN_UPDATED); + } + return true; } diff --git a/northd/en-sync-sb.c b/northd/en-sync-sb.c index 552ed56452..3ef3be6af1 100644 --- a/northd/en-sync-sb.c +++ b/northd/en-sync-sb.c @@ -244,10 +244,11 @@ bool sync_to_sb_lb_northd_handler(struct engine_node *node, void *data OVS_UNUSED) { struct northd_data *nd = engine_get_input_data("northd", node); - if (nd->change_tracked) { + if (nd->change_tracked && + northd_has_only_ports_in_tracked_data(&nd->trk_northd_changes)) { /* There are only NB LSP related changes and these can be safely * ignore and returned true. However in case the northd engine - * tracking data includes other changes, we need to do additional + * tracking data includes other ports, we need to do additional * checks before safely ignoring. */ return true; } diff --git a/northd/northd.c b/northd/northd.c index 3aefe42334..2a490e9ab6 100644 --- a/northd/northd.c +++ b/northd/northd.c @@ -5011,19 +5011,68 @@ destroy_tracked_ovn_ports(struct tracked_ovn_ports *trk_ovn_ports) } } +static void +destroy_tracked_lb_datapaths(struct tracked_lb_datapaths *trk_lbs) +{ + struct hmapx_node *hmapx_node; + HMAPX_FOR_EACH_SAFE (hmapx_node, &trk_lbs->deleted) { + ovn_lb_datapaths_destroy(hmapx_node->data); + hmapx_delete(&trk_lbs->deleted, hmapx_node); + } + + hmapx_clear(&trk_lbs->crupdated); + + if (trk_lbs->nb_ls_map) { + bitmap_free(trk_lbs->nb_ls_map); + trk_lbs->nb_ls_map = NULL; + } + + trk_lbs->n_nb_ls = 0; + + if (trk_lbs->nb_lr_map) { + bitmap_free(trk_lbs->nb_lr_map); + trk_lbs->nb_lr_map = NULL; + } + + trk_lbs->n_nb_lr = 0; +} + void destroy_northd_data_tracked_changes(struct northd_data *nd) { struct northd_tracked_data *trk_changes = &nd->trk_northd_changes; destroy_tracked_ovn_ports(&trk_changes->trk_ovn_ports); - + destroy_tracked_lb_datapaths(&trk_changes->trk_lbs); nd->change_tracked = false; - trk_changes->lb_changed = false; +} + +bool northd_has_tracked_data(struct northd_tracked_data *trk_nd_changes) +{ + return (trk_nd_changes->trk_lbs.n_nb_ls + || trk_nd_changes->trk_lbs.n_nb_lr + || !hmap_is_empty(&trk_nd_changes->trk_ovn_ports.created) + || !hmap_is_empty(&trk_nd_changes->trk_ovn_ports.updated) + || !hmap_is_empty(&trk_nd_changes->trk_ovn_ports.deleted) + || !hmapx_is_empty(&trk_nd_changes->trk_lbs.crupdated) + || !hmapx_is_empty(&trk_nd_changes->trk_lbs.deleted)); +} + +bool northd_has_only_ports_in_tracked_data( + struct northd_tracked_data *trk_nd_changes) +{ + return (!trk_nd_changes->trk_lbs.n_nb_ls + && !trk_nd_changes->trk_lbs.n_nb_lr + && hmapx_is_empty(&trk_nd_changes->trk_lbs.crupdated) + && hmapx_is_empty(&trk_nd_changes->trk_lbs.deleted) + && (!hmap_is_empty(&trk_nd_changes->trk_ovn_ports.created) + || !hmap_is_empty(&trk_nd_changes->trk_ovn_ports.updated) + || !hmap_is_empty(&trk_nd_changes->trk_ovn_ports.deleted))); } static void add_op_to_northd_tracked_ports(struct hmap *tracked_ovn_ports, - struct ovn_port *op) + struct ovn_port *op, + uint8_t changes) { struct tracked_ovn_port *trk_op; uint32_t hash = hash_string(op->key, 0); @@ -5038,6 +5087,60 @@ add_op_to_northd_tracked_ports(struct hmap *tracked_ovn_ports, trk_op->op = op; hmap_insert(tracked_ovn_ports, &trk_op->hmap_node, hash); } + + trk_op->changes |= changes; +} + +static void +add_od_to_northd_lb_dps_track_data(struct northd_tracked_data *nd_changes, + struct ovn_datapath *od, + size_t n_ls_datapaths, + size_t n_lr_datapaths) +{ + struct tracked_lb_datapaths *trk_lbs = &nd_changes->trk_lbs; + + if (od->nbs) { + if (!trk_lbs->nb_ls_map) { + trk_lbs->nb_ls_map = bitmap_allocate(n_ls_datapaths); + } + + if (!bitmap_is_set(trk_lbs->nb_ls_map, od->index)) { + bitmap_set1(trk_lbs->nb_ls_map, od->index); + trk_lbs->n_nb_ls++; + + /* Also add the router ports of the logical switch + * to the northd tracked data. */ + for (size_t i = 0; i < od->n_router_ports; i++) { + add_op_to_northd_tracked_ports( + &nd_changes->trk_ovn_ports.updated, od->router_ports[i], + EN_TRACKED_OP_LB_CHANGED); + } + } + } else { + ovs_assert(od->nbr); + if (!trk_lbs->nb_lr_map) { + trk_lbs->nb_lr_map = bitmap_allocate(n_lr_datapaths); + } + + if (!bitmap_is_set(trk_lbs->nb_lr_map, od->index)) { + bitmap_set1(trk_lbs->nb_lr_map, od->index); + trk_lbs->n_nb_lr++; + + /* Also add the peer (logical switch port) of logical router ports + * to the northd tracked data. */ + struct ovn_port *op; + HMAP_FOR_EACH (op, dp_node, &od->ports) { + add_op_to_northd_tracked_ports( + &nd_changes->trk_ovn_ports.updated, op, + EN_TRACKED_OP_LB_CHANGED); + if (op->peer) { + add_op_to_northd_tracked_ports( + &nd_changes->trk_ovn_ports.updated, op->peer, + EN_TRACKED_OP_LB_CHANGED); + } + } + } + } } /* Check if a changed LSP can be handled incrementally within the I-P engine @@ -5349,7 +5452,8 @@ ls_handle_lsp_changes(struct ovsdb_idl_txn *ovnsb_idl_txn, if (!op) { goto fail; } - add_op_to_northd_tracked_ports(&trk_ports->created, op); + add_op_to_northd_tracked_ports(&trk_ports->created, op, + EN_TRACKED_OP_LPORT_CHANGED); } else if (ls_port_has_changed(op->nbsp, new_nbsp)) { /* Existing port updated */ bool temp = false; @@ -5381,7 +5485,8 @@ ls_handle_lsp_changes(struct ovsdb_idl_txn *ovnsb_idl_txn, goto fail; } - add_op_to_northd_tracked_ports(&trk_ports->updated, op); + add_op_to_northd_tracked_ports(&trk_ports->updated, op, + EN_TRACKED_OP_LPORT_CHANGED); } op->visited = true; } @@ -5397,7 +5502,8 @@ ls_handle_lsp_changes(struct ovsdb_idl_txn *ovnsb_idl_txn, * impacted by this deletion. Fallback to recompute. */ goto fail; } - add_op_to_northd_tracked_ports(&trk_ports->deleted, op); + add_op_to_northd_tracked_ports(&trk_ports->deleted, op, + EN_TRACKED_OP_LPORT_CHANGED); hmap_remove(&nd->ls_ports, &op->key_node); hmap_remove(&od->ports, &op->dp_node); sbrec_port_binding_delete(op->sb); @@ -5652,7 +5758,8 @@ northd_handle_lb_data_changes_pre_od(struct tracked_lb_data *trk_lb_data, struct ovn_datapaths *ls_datapaths, struct ovn_datapaths *lr_datapaths, struct hmap *lb_datapaths_map, - struct hmap *lbgrp_datapaths_map) + struct hmap *lbgrp_datapaths_map, + struct northd_tracked_data *nd_changes) { struct ovn_lb_datapaths *lb_dps; struct ovn_northd_lb *lb; @@ -5665,6 +5772,9 @@ northd_handle_lb_data_changes_pre_od(struct tracked_lb_data *trk_lb_data, lb_dps = ovn_lb_datapaths_find(lb_datapaths_map, lb_uuid); ovs_assert(lb_dps); + /* Add the deleted lb to the northd tracked data. */ + hmapx_add(&nd_changes->trk_lbs.deleted, lb_dps); + /* Re-evaluate 'od->has_lb_vip for od's associated with the * deleted lb. */ size_t index; @@ -5672,10 +5782,14 @@ northd_handle_lb_data_changes_pre_od(struct tracked_lb_data *trk_lb_data, lb_dps->nb_ls_map) { od = ls_datapaths->array[index]; init_lb_for_datapath(od); + + /* Add the ls datapath to the northd tracked data. */ + add_od_to_northd_lb_dps_track_data(nd_changes, od, + ods_size(ls_datapaths), + ods_size(lr_datapaths)); } hmap_remove(lb_datapaths_map, &lb_dps->hmap_node); - ovn_lb_datapaths_destroy(lb_dps); } struct crupdated_lb *clb; @@ -5731,7 +5845,8 @@ northd_handle_lb_data_changes_post_od(struct tracked_lb_data *trk_lb_data, struct ovn_datapaths *ls_datapaths, struct ovn_datapaths *lr_datapaths, struct hmap *lb_datapaths_map, - struct hmap *lbgrp_datapaths_map) + struct hmap *lbgrp_datapaths_map, + struct northd_tracked_data *nd_changes) { ovs_assert(!trk_lb_data->has_health_checks); ovs_assert(!trk_lb_data->has_dissassoc_lbs_from_lb_grops); @@ -5751,6 +5866,9 @@ northd_handle_lb_data_changes_post_od(struct tracked_lb_data *trk_lb_data, lb_dps = ovn_lb_datapaths_find(lb_datapaths_map, &uuidnode->uuid); ovs_assert(lb_dps); ovn_lb_datapaths_add_ls(lb_dps, 1, &od); + + /* Add the lb to the northd tracked data. */ + hmapx_add(&nd_changes->trk_lbs.crupdated, lb_dps); } UUIDSET_FOR_EACH (uuidnode, &codlb->assoc_lbgrps) { @@ -5766,11 +5884,19 @@ northd_handle_lb_data_changes_post_od(struct tracked_lb_data *trk_lb_data, lb_dps = ovn_lb_datapaths_find(lb_datapaths_map, lb_uuid); ovs_assert(lb_dps); ovn_lb_datapaths_add_ls(lb_dps, 1, &od); + + /* Add the lb to the northd tracked data. */ + hmapx_add(&nd_changes->trk_lbs.crupdated, lb_dps); } } /* Re-evaluate 'od->has_lb_vip' */ init_lb_for_datapath(od); + + /* Add the ls datapath to the northd tracked data. */ + add_od_to_northd_lb_dps_track_data(nd_changes, od, + ods_size(ls_datapaths), + ods_size(lr_datapaths)); } LIST_FOR_EACH (codlb, list_node, &trk_lb_data->crupdated_lr_lbs) { @@ -5786,6 +5912,9 @@ northd_handle_lb_data_changes_post_od(struct tracked_lb_data *trk_lb_data, /* Add the lb_ips of lb_dps to the od. */ build_lrouter_lb_ips(od->lb_ips, lb_dps->lb); build_lrouter_lb_reachable_ips(od, lb_dps->lb); + + /* Add the lb to the northd tracked data. */ + hmapx_add(&nd_changes->trk_lbs.crupdated, lb_dps); } UUIDSET_FOR_EACH (uuidnode, &codlb->assoc_lbgrps) { @@ -5805,11 +5934,19 @@ northd_handle_lb_data_changes_post_od(struct tracked_lb_data *trk_lb_data, /* Add the lb_ips of lb_dps to the od. */ build_lrouter_lb_ips(od->lb_ips, lb_dps->lb); build_lrouter_lb_reachable_ips(od, lb_dps->lb); + + /* Add the lb to the northd tracked data. */ + hmapx_add(&nd_changes->trk_lbs.crupdated, lb_dps); } } /* Re-evaluate 'od->has_lb_vip' */ init_lb_for_datapath(od); + + /* Add the lr datapath to the northd tracked data. */ + add_od_to_northd_lb_dps_track_data(nd_changes, od, + ods_size(ls_datapaths), + ods_size(lr_datapaths)); } struct crupdated_lb *clb; @@ -5825,6 +5962,11 @@ northd_handle_lb_data_changes_post_od(struct tracked_lb_data *trk_lb_data, od = ls_datapaths->array[index]; /* Re-evaluate 'od->has_lb_vip' */ init_lb_for_datapath(od); + + /* Add the ls datapath to the northd tracked data. */ + add_od_to_northd_lb_dps_track_data(nd_changes, od, + ods_size(ls_datapaths), + ods_size(lr_datapaths)); } BITMAP_FOR_EACH_1 (index, ods_size(lr_datapaths), @@ -5848,7 +5990,15 @@ northd_handle_lb_data_changes_post_od(struct tracked_lb_data *trk_lb_data, add_neigh_ips_to_lrouter(od, lb->neigh_mode, &clb->inserted_vips_v4, &clb->inserted_vips_v6); + + /* Add the lr datapath to the northd tracked data. */ + add_od_to_northd_lb_dps_track_data(nd_changes, od, + ods_size(ls_datapaths), + ods_size(lr_datapaths)); } + + /* Add the lb to the northd tracked data. */ + hmapx_add(&nd_changes->trk_lbs.crupdated, lb_dps); } struct ovn_lb_group *lbgrp; @@ -5877,6 +6027,11 @@ northd_handle_lb_data_changes_post_od(struct tracked_lb_data *trk_lb_data, /* Add the lb_ips of lb_dps to the od. */ build_lrouter_lb_ips(od->lb_ips, lb_dps->lb); + + /* Add the lr datapath to the northd tracked data. */ + add_od_to_northd_lb_dps_track_data(nd_changes, od, + ods_size(ls_datapaths), + ods_size(lr_datapaths)); } for (size_t i = 0; i < lbgrp_dps->n_ls; i++) { @@ -5885,7 +6040,15 @@ northd_handle_lb_data_changes_post_od(struct tracked_lb_data *trk_lb_data, /* Re-evaluate 'od->has_lb_vip' */ init_lb_for_datapath(od); + + /* Add the ls datapath to the northd tracked data. */ + add_od_to_northd_lb_dps_track_data(nd_changes, od, + ods_size(ls_datapaths), + ods_size(lr_datapaths)); } + + /* Add the lb to the northd tracked data. */ + hmapx_add(&nd_changes->trk_lbs.crupdated, lb_dps); } } @@ -17814,6 +17977,45 @@ unlink_lflows(struct ovn_datapath *od, enum objdep_type objdep_type, unlink_objres_lflows(res_node, od, lflow_data, lflowdep_mgr); } +/* Unlinks all the lflows from the lflow dep mgr for all the set datapaths + * for the specified type 'objdep_type' and resource name 'res_name'. */ +static void +unlink_all_ods_objres_lflows(struct objdep_mgr *lflowdep_mgr, + enum objdep_type objdep_type, + const char *res_name, + size_t n_ls_datapaths, + size_t n_lr_datapaths, + struct hmap *lflows_hash_map) +{ + struct resource_to_objects_node *res_node = objdep_mgr_find_objs( + lflowdep_mgr, objdep_type, res_name); + + if (!res_node) { + return; + } + + struct object_to_resources_list_node *resource_list_node; + RESOURCE_FOR_EACH_OBJ (resource_list_node, res_node) { + const struct uuid *obj_uuid = &resource_list_node->obj_uuid; + struct ovn_lflow *lflow = ovn_lflow_uuid_find(lflows_hash_map, + obj_uuid); + if (!lflow) { + continue; + } + + size_t n_datapaths; + if (ovn_stage_to_datapath_type(lflow->stage) == DP_SWITCH) { + n_datapaths = n_ls_datapaths; + } else { + n_datapaths = n_lr_datapaths; + } + size_t index; + BITMAP_FOR_EACH_1 (index, n_datapaths, lflow->dpg_bitmap) { + bitmap_set0(lflow->dpg_bitmap, index); + } + } +} + static bool sync_lflows_from_objres(struct ovsdb_idl_txn *ovnsb_txn, struct resource_to_objects_node *res_node, @@ -17914,17 +18116,19 @@ unlink_and_sync_lflows(struct ovsdb_idl_txn *sb_txn, struct ovn_datapath *od, lflow_data, lflowdep_mgr); } -bool lflow_handle_northd_changes(struct ovsdb_idl_txn *ovnsb_txn, - struct northd_tracked_data *trk_nd_changes, - struct lflow_input *lflow_input, - struct lflow_data *lflow_data) +bool lflow_handle_northd_port_changes(struct ovsdb_idl_txn *ovnsb_txn, + struct tracked_ovn_ports *trk_ovn_ports, + struct lflow_input *lflow_input, + struct lflow_data *lflow_data) { - struct tracked_ovn_ports *trk_ovn_ports = &trk_nd_changes->trk_ovn_ports; - struct tracked_ovn_port *trk_op; HMAP_FOR_EACH (trk_op, hmap_node, &trk_ovn_ports->deleted) { struct ovn_port *op = trk_op->op; + /* We don't support lflow handling for deleted logical router + * ports yet. */ + ovs_assert(op->nbsp); + /* unlink old lflows. */ unlink_and_sync_lflows(ovnsb_txn, op->od, OBJDEP_TYPE_LPORT, op->nbsp->name, lflow_input, lflow_data, @@ -17937,19 +18141,53 @@ bool lflow_handle_northd_changes(struct ovsdb_idl_txn *ovnsb_txn, HMAP_FOR_EACH (trk_op, hmap_node, &trk_ovn_ports->updated) { struct ovn_port *op = trk_op->op; + ovs_assert(trk_op->changes); - /* unlink old lflows. */ - unlink_lflows(op->od, OBJDEP_TYPE_LPORT, op->nbsp->name, - lflow_data, &op->lflow_dep_mgr); + const char *res_name = op->nbsp ? op->nbsp->name : op->nbrp->name; /* Generate new lflows. */ struct ds match = DS_EMPTY_INITIALIZER; struct ds actions = DS_EMPTY_INITIALIZER; - build_lswitch_and_lrouter_iterate_by_lsp(op, lflow_input->ls_ports, - lflow_input->lr_ports, - lflow_input->meter_groups, - &match, &actions, - lflow_data); + if (trk_op->changes & EN_TRACKED_OP_LPORT_CHANGED) { + /* We still don't support EN_TRACKED_OP_LPORT_CHANGED for + * router ports. */ + ovs_assert(op->nbsp); + + /* unlink old lflows. */ + unlink_lflows(op->od, OBJDEP_TYPE_LPORT, res_name, lflow_data, + &op->lflow_dep_mgr); + + /* Generate new lflows. */ + build_lswitch_and_lrouter_iterate_by_lsp(op, lflow_input->ls_ports, + lflow_input->lr_ports, + lflow_input->meter_groups, + &match, &actions, + lflow_data); + } + + if (trk_op->changes & EN_TRACKED_OP_LB_CHANGED) { + /* unlink old lflows. */ + unlink_lflows(op->od, OBJDEP_TYPE_LB, res_name, lflow_data, + &op->lflow_dep_mgr); + unlink_lflows(op->od, OBJDEP_TYPE_CT_LB, res_name, lflow_data, + &op->lflow_dep_mgr); + + if (op->nbsp) { + build_lswitch_ip_unicast_lookup_lb_vips(op, lflow_data, + &op->lflow_dep_mgr); + build_ip_routing_flows_for_router_type_lsp( + op, lflow_input->lr_ports, lflow_data); + build_arp_resolve_flows_for_routable_addrs_throug_lsp( + op, lflow_data, lflow_input->lr_ports, &match, &actions, + &op->lflow_dep_mgr); + } else { + build_lrouter_ipv4_ip_input_lb_related(op, lflow_data, &match, + lflow_input->meter_groups); + build_lrouter_force_snat_flows_op(op, lflow_data, &match, + &actions); + } + } + ds_destroy(&match); ds_destroy(&actions); @@ -17957,13 +18195,26 @@ bool lflow_handle_northd_changes(struct ovsdb_idl_txn *ovnsb_txn, * groups. */ /* Sync the new flows to SB. */ - sync_lflows_for_res(ovnsb_txn, OBJDEP_TYPE_LPORT, op->nbsp->name, - lflow_input, lflow_data, &op->lflow_dep_mgr); + if (trk_op->changes & EN_TRACKED_OP_LPORT_CHANGED) { + sync_lflows_for_res(ovnsb_txn, OBJDEP_TYPE_LPORT, res_name, + lflow_input, lflow_data, &op->lflow_dep_mgr); + } + + if (trk_op->changes & EN_TRACKED_OP_LB_CHANGED) { + sync_lflows_for_res(ovnsb_txn, OBJDEP_TYPE_LB, res_name, + lflow_input, lflow_data, &op->lflow_dep_mgr); + sync_lflows_for_res(ovnsb_txn, OBJDEP_TYPE_CT_LB, res_name, + lflow_input, lflow_data, &op->lflow_dep_mgr); + } } HMAP_FOR_EACH (trk_op, hmap_node, &trk_ovn_ports->created) { struct ovn_port *op = trk_op->op; + /* We don't support lflow handling for deleted logical router + * ports yet. */ + ovs_assert(op->nbsp); + const struct sbrec_multicast_group *sbmc_flood = mcast_group_lookup(lflow_input->sbrec_mcast_group_by_name_dp, MC_FLOOD, op->od->sb); @@ -18017,6 +18268,151 @@ bool lflow_handle_northd_changes(struct ovsdb_idl_txn *ovnsb_txn, return true; } +static void +lflow_update_ls_lb_lflows(struct ovn_datapath *od, + struct ovsdb_idl_txn *ovnsb_txn, + struct lflow_input *lflow_input, + struct lflow_data *lflow_data) +{ + ovs_assert(od->nbs); + + /* unlink old lflows. */ + unlink_lflows(od, OBJDEP_TYPE_LB, od->nbs->name, lflow_data, + &od->lflow_dep_mgr); + unlink_lflows(od, OBJDEP_TYPE_CT_LB, od->nbs->name, lflow_data, + &od->lflow_dep_mgr); + + /* Build the lflows related to LB and CT LB. */ + build_pre_acls(od, lflow_input->port_groups, lflow_data); + build_pre_lb_lb_related(od, lflow_data); + build_acl_hints(od, lflow_input->features, lflow_data); + build_acls(od, lflow_input->features, lflow_data, lflow_input->port_groups, + lflow_input->meter_groups); + build_lb_hairpin(od, lflow_data); + + /* Sync the new flows to SB. */ + sync_lflows_for_res(ovnsb_txn, OBJDEP_TYPE_LB, od->nbs->name, lflow_input, + lflow_data, &od->lflow_dep_mgr); + sync_lflows_for_res(ovnsb_txn, OBJDEP_TYPE_CT_LB, od->nbs->name, + lflow_input, lflow_data, &od->lflow_dep_mgr); +} + +static void +lflow_update_lr_lb_lflows(struct ovn_datapath *od, + struct ovsdb_idl_txn *ovnsb_txn, + struct lflow_input *lflow_input, + struct lflow_data *lflow_data) +{ + ovs_assert(od->nbr); + + /* unlink old lflows. */ + unlink_lflows(od, OBJDEP_TYPE_LB, od->nbr->name, lflow_data, + &od->lflow_dep_mgr); + unlink_lflows(od, OBJDEP_TYPE_CT_LB, od->nbr->name, lflow_data, + &od->lflow_dep_mgr); + + struct ds actions = DS_EMPTY_INITIALIZER; + struct ds match = DS_EMPTY_INITIALIZER; + + build_lrouter_nat_flows_lb_related(od, lflow_data, &match, + lflow_input->features); + build_lrouter_nat_flows_ct_lb_related(od, lflow_data); + + ds_destroy(&match); + ds_destroy(&actions); + + /* Sync the new flows to SB. */ + sync_lflows_for_res(ovnsb_txn, OBJDEP_TYPE_LB, od->nbr->name, lflow_input, + lflow_data, &od->lflow_dep_mgr); + sync_lflows_for_res(ovnsb_txn, OBJDEP_TYPE_CT_LB, od->nbr->name, + lflow_input, lflow_data, &od->lflow_dep_mgr); +} + +bool lflow_handle_northd_lb_changes(struct ovsdb_idl_txn *ovnsb_txn, + struct tracked_lb_datapaths *trk_lbs, + struct lflow_input *lflow_input, + struct lflow_data *lflow_data) +{ + struct ovn_lb_datapaths *lb_dps; + struct hmapx_node *hmapx_node; + HMAPX_FOR_EACH (hmapx_node, &trk_lbs->deleted) { + lb_dps = hmapx_node->data; + + /* unlink old lflows. */ + unlink_all_ods_objres_lflows(&lb_dps->lflow_dep_mgr, OBJDEP_TYPE_LB, + lb_dps->lb->nlb->name, + ods_size(lflow_input->ls_datapaths), + ods_size(lflow_input->lr_datapaths), + &lflow_data->lflows_hash_map); + sync_lflows_for_res(ovnsb_txn, OBJDEP_TYPE_LB, + lb_dps->lb->nlb->name, lflow_input, + lflow_data, &lb_dps->lflow_dep_mgr); + objdep_mgr_clear(&lb_dps->lflow_dep_mgr); + } + + HMAPX_FOR_EACH (hmapx_node, &trk_lbs->crupdated) { + lb_dps = hmapx_node->data; + + /* unlink old lflows. */ + unlink_all_ods_objres_lflows(&lb_dps->lflow_dep_mgr, OBJDEP_TYPE_LB, + lb_dps->lb->nlb->name, + ods_size(lflow_input->ls_datapaths), + ods_size(lflow_input->lr_datapaths), + &lflow_data->lflows_hash_map); + + /* Generate new lflows. */ + struct ds match = DS_EMPTY_INITIALIZER; + struct ds actions = DS_EMPTY_INITIALIZER; + + build_lswitch_arp_nd_service_monitor(lb_dps, lflow_input->ls_ports, + lflow_data, &actions, + &match); + build_lrouter_defrag_flows_for_lb(lb_dps, lflow_data, + lflow_input->lr_datapaths, &match); + build_lrouter_flows_for_lb(lb_dps, lflow_data, + lflow_input->meter_groups, + lflow_input->lr_datapaths, + lflow_input->features, + lflow_input->svc_monitor_map, + &match, &actions); + build_lswitch_flows_for_lb(lb_dps, lflow_data, + lflow_input->meter_groups, + lflow_input->ls_datapaths, + lflow_input->features, + lflow_input->svc_monitor_map, + &match, &actions); + + ds_destroy(&match); + ds_destroy(&actions); + + /* Sync the new flows to SB. */ + sync_lflows_for_res(ovnsb_txn, OBJDEP_TYPE_LB, + lb_dps->lb->nlb->name, lflow_input, + lflow_data, &lb_dps->lflow_dep_mgr); + + } + + if (trk_lbs->n_nb_ls && trk_lbs->nb_ls_map) { + size_t index; + BITMAP_FOR_EACH_1 (index, ods_size(lflow_input->ls_datapaths), + trk_lbs->nb_ls_map) { + struct ovn_datapath *od = lflow_input->ls_datapaths->array[index]; + lflow_update_ls_lb_lflows(od, ovnsb_txn, lflow_input, lflow_data); + } + } + + if (trk_lbs->n_nb_lr && trk_lbs->nb_lr_map) { + size_t index; + BITMAP_FOR_EACH_1 (index, ods_size(lflow_input->lr_datapaths), + trk_lbs->nb_lr_map) { + struct ovn_datapath *od = lflow_input->lr_datapaths->array[index]; + lflow_update_lr_lb_lflows(od, ovnsb_txn, lflow_input, lflow_data); + } + } + + return true; +} + /* Each port group in Port_Group table in OVN_Northbound has a corresponding * entry in Port_Group table in OVN_Southbound. In OVN_Northbound the entries * contains lport uuids, while in OVN_Southbound we store the lport names. @@ -18899,6 +19295,8 @@ northd_init(struct northd_data *data) hmap_init(&data->trk_northd_changes.trk_ovn_ports.created); hmap_init(&data->trk_northd_changes.trk_ovn_ports.updated); hmap_init(&data->trk_northd_changes.trk_ovn_ports.deleted); + hmapx_init(&data->trk_northd_changes.trk_lbs.crupdated); + hmapx_init(&data->trk_northd_changes.trk_lbs.deleted); } void @@ -18955,6 +19353,8 @@ northd_destroy(struct northd_data *data) hmap_destroy(&data->trk_northd_changes.trk_ovn_ports.created); hmap_destroy(&data->trk_northd_changes.trk_ovn_ports.updated); hmap_destroy(&data->trk_northd_changes.trk_ovn_ports.deleted); + hmapx_destroy(&data->trk_northd_changes.trk_lbs.crupdated); + hmapx_destroy(&data->trk_northd_changes.trk_lbs.deleted); } void diff --git a/northd/northd.h b/northd/northd.h index 1634463a67..b836aa737f 100644 --- a/northd/northd.h +++ b/northd/northd.h @@ -88,10 +88,15 @@ struct ovn_datapaths { struct ovn_datapath **array; }; +#define EN_TRACKED_OP_LPORT_CHANGED 0x01 +#define EN_TRACKED_OP_LB_CHANGED 0x02 + /* Represents a tracked ovn_port. */ struct tracked_ovn_port { struct hmap_node hmap_node; struct ovn_port *op; + /* Indicates what changed. one or any or all of EN_TRACKED_OP_* bit. */ + uint8_t changes; }; struct tracked_ovn_ports { @@ -108,13 +113,30 @@ struct tracked_ovn_ports { struct hmap deleted; }; +struct tracked_lb_datapaths { + /* Tracked created or updated load balancers. + * hmapx node data is 'struct ovn_lb_datapaths' */ + struct hmapx crupdated; + + /* Tracked deleted lbs. + * hmapx node data is 'struct ovn_lb_datapaths' */ + struct hmapx deleted; + + /* Tracked logical switches related to the tracked lbs. */ + unsigned long *nb_ls_map; + size_t n_nb_ls; + + /* Tracked logical routers related to the tracked lb. */ + unsigned long *nb_lr_map; + size_t n_nb_lr; +}; + /* Track what's changed in the northd engine node. * Now only tracks ovn_ports (of vif type) - created, updated * and deleted and indicates if load balancers have changed. */ struct northd_tracked_data { struct tracked_ovn_ports trk_ovn_ports; - bool lb_changed; /* Indicates if load balancers changed or association of - * load balancer to logical switch/router changed. */ + struct tracked_lb_datapaths trk_lbs; }; struct northd_data { @@ -363,9 +385,14 @@ void northd_indices_create(struct northd_data *data, void build_lflows(struct ovsdb_idl_txn *ovnsb_txn, struct lflow_input *input_data, struct lflow_data *lflow_data); -bool lflow_handle_northd_changes(struct ovsdb_idl_txn *ovnsb_txn, - struct northd_tracked_data *, - struct lflow_input *, struct lflow_data *); +bool lflow_handle_northd_port_changes(struct ovsdb_idl_txn *ovnsb_txn, + struct tracked_ovn_ports *, + struct lflow_input *, + struct lflow_data *); +bool lflow_handle_northd_lb_changes(struct ovsdb_idl_txn *ovnsb_txn, + struct tracked_lb_datapaths *, + struct lflow_input *lflow_input, + struct lflow_data *lflow_data); bool northd_handle_sb_port_binding_changes( const struct sbrec_port_binding_table *, struct hmap *ls_ports); @@ -374,12 +401,16 @@ bool northd_handle_lb_data_changes_pre_od(struct tracked_lb_data *, struct ovn_datapaths *ls_datapaths, struct ovn_datapaths *lr_datapaths, struct hmap *lb_datapaths_map, - struct hmap *lbgrp_datapaths_map); + struct hmap *lbgrp_datapaths_map, + struct northd_tracked_data *); bool northd_handle_lb_data_changes_post_od(struct tracked_lb_data *, struct ovn_datapaths *ls_datapaths, struct ovn_datapaths *lr_datapaths, struct hmap *lb_datapaths_map, - struct hmap *lbgrp_datapaths_map); + struct hmap *lbgrp_datapaths_map, + struct northd_tracked_data *); +bool northd_has_tracked_data(struct northd_tracked_data *); +bool northd_has_only_ports_in_tracked_data(struct northd_tracked_data *); void build_bfd_table(struct ovsdb_idl_txn *ovnsb_txn, const struct nbrec_bfd_table *, diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at index a5f64ed5bd..4424a1f64d 100644 --- a/tests/ovn-northd.at +++ b/tests/ovn-northd.at @@ -9834,7 +9834,7 @@ check ovn-nbctl --wait=sb lb-add lb1 10.0.0.10:80 10.0.0.3:80 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats @@ -9842,17 +9842,17 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb set load_balancer . ip_port_mappings:10.0.0.3=sw0-p1:10.0.0.2 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check ovn-nbctl --wait=sb set load_balancer . options:foo=bar check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check ovn-nbctl --wait=sb -- lb-add lb2 20.0.0.10:80 20.0.0.20:80 -- lb-add lb3 30.0.0.10:80 30.0.0.20:80 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats @@ -9860,7 +9860,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb -- lb-del lb2 -- lb-del lb3 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats @@ -9901,13 +9901,14 @@ ovn-nbctl --wait=sb lsp-set-options sw0-lr0 router-port=lr0-sw0 check_engine_stats lb_data norecompute compute check_engine_stats northd recompute nocompute check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE # Associate lb1 to sw0. There should be no recompute of northd engine node check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb ls-lb-add sw0 lb1 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Modify the backend of the lb1 vip @@ -9915,7 +9916,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb set load_balancer lb1 vips:'"10.0.0.10:80"'='"10.0.0.100:80"' check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Cleanup the vip of lb1. @@ -9923,7 +9924,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb clear load_Balancer lb1 vips check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Set the vips of lb1 back @@ -9931,7 +9932,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lb-add lb1 10.0.0.10:80 10.0.0.3:80 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Add another vip to lb1 @@ -9939,7 +9940,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lb-add lb1 10.0.0.20:80 10.0.0.30:8080 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Disassociate lb1 from sw0. There should be a full recompute of northd engine node. @@ -9974,7 +9975,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lr-lb-add lr0 lb1 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Modify the backend of the lb1 vip @@ -9982,7 +9983,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb set load_balancer lb1 vips:'"10.0.0.10:80"'='"10.0.0.100:80"' check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Cleanup the vip of lb1. @@ -9990,7 +9991,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb clear load_Balancer lb1 vips check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Set the vips of lb1 back @@ -9998,7 +9999,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lb-add lb1 10.0.0.10:80 10.0.0.3:80 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Add another vip to lb1 @@ -10006,7 +10007,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lb-add lb1 10.0.0.20:80 10.0.0.30:8080 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats @@ -10021,7 +10022,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats lbg1_uuid=$(ovn-nbctl create load_balancer_group name=lbg1) check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats @@ -10033,7 +10034,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl add load_balancer_group . load_Balancer $lb1_uuid check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl clear load_balancer_group . load_Balancer @@ -10046,7 +10047,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl add load_balancer_group . load_Balancer $lb1_uuid check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10054,21 +10055,23 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl add logical_switch sw0 load_balancer_group $lbg1_uuid check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute +CHECK_NO_CHANGE_AFTER_RECOMPUTE # Update lb and this should not result in northd recompute check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb set load_balancer . options:bar=foo check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute +CHECK_NO_CHANGE_AFTER_RECOMPUTE # Modify the backend of the lb1 vip check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb set load_balancer lb1 vips:'"10.0.0.10:80"'='"10.0.0.100:80"' check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Cleanup the vip of lb1. @@ -10076,7 +10079,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb clear load_Balancer lb1 vips check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Set the vips of lb1 back @@ -10084,7 +10087,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lb-add lb1 10.0.0.10:80 10.0.0.3:80 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Add another vip to lb1 @@ -10092,7 +10095,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lb-add lb1 10.0.0.20:80 10.0.0.30:8080 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats @@ -10100,12 +10103,13 @@ check ovn-nbctl clear logical_switch sw0 load_balancer_group check_engine_stats lb_data norecompute compute check_engine_stats northd recompute nocompute check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl add logical_router lr0 load_balancer_group $lbg1_uuid check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Modify the backend of the lb1 vip @@ -10113,7 +10117,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb set load_balancer lb1 vips:'"10.0.0.10:80"'='"10.0.0.100:80"' check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Cleanup the vip of lb1. @@ -10121,7 +10125,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb clear load_Balancer lb1 vips check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Set the vips of lb1 back @@ -10129,7 +10133,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lb-add lb1 10.0.0.10:80 10.0.0.3:80 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Add another vip to lb1 @@ -10137,7 +10141,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lb-add lb1 10.0.0.20:80 10.0.0.30:8080 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats @@ -10145,13 +10149,15 @@ check ovn-nbctl clear logical_router lr0 load_balancer_group check_engine_stats lb_data norecompute compute check_engine_stats northd recompute nocompute check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE # Add back lb group to logical switch and then delete it. check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl add logical_switch sw0 load_balancer_group $lbg1_uuid check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute +CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl clear logical_switch sw0 load_balancer_group -- \ @@ -10180,40 +10186,42 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats lbg1_uuid=$(ovn-nbctl create load_balancer_group name=lbg1) check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl set load_balancer_group . load_balancer="$lb2_uuid,$lb3_uuid,$lb4_uuid" check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute +CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl set logical_switch sw0 load_balancer_group=$lbg1_uuid check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl set logical_router lr1 load_balancer_group=$lbg1_uuid check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb ls-lb-add sw0 lb2 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb ls-lb-add sw0 lb3 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats @@ -10221,7 +10229,7 @@ check ovn-nbctl --wait=sb lr-lb-add lr1 lb1 check ovn-nbctl --wait=sb lr-lb-add lr1 lb2 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats @@ -10244,7 +10252,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lb-del lb4 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Deleting lb2 should result in lflow recompute as it is @@ -10253,7 +10261,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lb-del lb2 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats @@ -10265,3 +10273,76 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE AT_CLEANUP ]) + + +OVN_FOR_EACH_NORTHD_NO_HV([ +AT_SETUP([Load balancer incremental processing with stateless ACLs]) +ovn_start + +check_engine_stats() { + node=$1 + recompute=$2 + compute=$3 + + echo "__file__:__line__: Checking engine stats for node $node : recompute - \ +$recompute : compute - $compute" + + node_stat=$(as northd ovn-appctl -t NORTHD_TYPE inc-engine/show-stats $node) + # node_stat will be of this format : + # - Node: lflow - recompute: 3 - compute: 0 - abort: 0 + node_recompute_ct=$(echo $node_stat | cut -d '-' -f2 | cut -d ':' -f2) + node_compute_ct=$(echo $node_stat | cut -d '-' -f3 | cut -d ':' -f2) + + if [[ "$recompute" == "norecompute" ]]; then + # node should not be recomputed + echo "Expecting $node recompute count - $node_recompute_ct to be 0" + check test "$node_recompute_ct" -eq "0" + else + echo "Expecting $node recompute count - $node_recompute_ct not to be 0" + check test "$node_recompute_ct" -ne "0" + fi + + if [[ "$compute" == "nocompute" ]]; then + # node should not be computed + echo "Expecting $node compute count - $node_compute_ct to be 0" + check test "$node_compute_ct" -eq "0" + else + echo "Expecting $node compute count - $node_compute_ct not to be 0" + check test "$node_compute_ct" -ne "0" + fi +} + +# Test I-P for load balancers. +# Presently ovn-northd handles I-P for NB LBs in northd_lb_data engine node +# only. +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb lb-add lb1 10.0.0.10:80 10.0.0.3:80 + +check ovn-nbctl ls-add sw0 +check ovn-nbctl acl-add sw0 from-lport 1 1 allow-stateless +check ovn-nbctl --wait=sb acl-add sw0 to-lport 1 1 allow-stateless + +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb ls-lb-add sw0 lb1 +check_engine_stats lb_data norecompute compute +check_engine_stats northd norecompute compute +check_engine_stats lflow norecompute compute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +# Clear the VIPs of lb1 +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb clear load_balancer . vips +check_engine_stats lb_data norecompute compute +check_engine_stats northd norecompute compute +check_engine_stats lflow norecompute compute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb lb-del lb1 +check_engine_stats lb_data norecompute compute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +AT_CLEANUP +]) From patchwork Fri Aug 18 08:58:45 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Numan Siddique X-Patchwork-Id: 1822766 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::138; helo=smtp1.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp1.osuosl.org (smtp1.osuosl.org [IPv6:2605:bc80:3010::138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4RRwnM53V9z1ygW for ; Fri, 18 Aug 2023 19:00:23 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id DD9BF840B0; Fri, 18 Aug 2023 09:00:21 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org DD9BF840B0 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 2SypjEOCjYMX; Fri, 18 Aug 2023 09:00:17 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp1.osuosl.org (Postfix) with ESMTPS id 141AD8419B; Fri, 18 Aug 2023 09:00:14 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 141AD8419B Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id A4DD9C0039; Fri, 18 Aug 2023 09:00:13 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 1BCE7C0072 for ; Fri, 18 Aug 2023 09:00:12 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 7E216615EE for ; Fri, 18 Aug 2023 08:58:55 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 7E216615EE X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id T8UmHPcC7-49 for ; Fri, 18 Aug 2023 08:58:54 +0000 (UTC) Received: from relay4-d.mail.gandi.net (relay4-d.mail.gandi.net [IPv6:2001:4b98:dc4:8::224]) by smtp3.osuosl.org (Postfix) with ESMTPS id B8554615F0 for ; Fri, 18 Aug 2023 08:58:53 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org B8554615F0 Received: by mail.gandi.net (Postfix) with ESMTPSA id AB662E0009; Fri, 18 Aug 2023 08:58:49 +0000 (UTC) From: numans@ovn.org To: dev@openvswitch.org Date: Fri, 18 Aug 2023 14:28:45 +0530 Message-Id: <20230818085845.1031138-1-numans@ovn.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230818085606.1030792-1-numans@ovn.org> References: <20230818085606.1030792-1-numans@ovn.org> MIME-Version: 1.0 X-GND-Sasl: numans@ovn.org Subject: [ovs-dev] [PATCH ovn v6 15/16] northd: Move router ports SB PB options sync to sync_to_sb_pb node. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Numan Siddique With this we don't fall back to recompute of northd engine node when SB port bindings (of router ports) updates from SB ovsdb-server are received. Signed-off-by: Numan Siddique --- northd/en-northd.c | 2 +- northd/en-sync-sb.c | 3 +- northd/northd.c | 197 +++++++++++++++++++++++++++----------------- northd/northd.h | 6 +- tests/ovn-northd.at | 49 +++++++++++ 5 files changed, 176 insertions(+), 81 deletions(-) diff --git a/northd/en-northd.c b/northd/en-northd.c index 651c53a6ce..b3e84a6af6 100644 --- a/northd/en-northd.c +++ b/northd/en-northd.c @@ -199,7 +199,7 @@ northd_sb_port_binding_handler(struct engine_node *node, northd_get_input_data(node, &input_data); if (!northd_handle_sb_port_binding_changes( - input_data.sbrec_port_binding_table, &nd->ls_ports)) { + input_data.sbrec_port_binding_table, &nd->ls_ports, &nd->lr_ports)) { return false; } diff --git a/northd/en-sync-sb.c b/northd/en-sync-sb.c index 3ef3be6af1..3b82bd0718 100644 --- a/northd/en-sync-sb.c +++ b/northd/en-sync-sb.c @@ -276,7 +276,8 @@ en_sync_to_sb_pb_run(struct engine_node *node, void *data OVS_UNUSED) const struct engine_context *eng_ctx = engine_get_context(); struct northd_data *northd_data = engine_get_input_data("northd", node); - sync_pbs(eng_ctx->ovnsb_idl_txn, &northd_data->ls_ports); + sync_pbs(eng_ctx->ovnsb_idl_txn, &northd_data->ls_ports, + &northd_data->lr_ports); engine_set_node_state(node, EN_UPDATED); } diff --git a/northd/northd.c b/northd/northd.c index 2a490e9ab6..2ae41b5d8a 100644 --- a/northd/northd.c +++ b/northd/northd.c @@ -3222,27 +3222,33 @@ op_get_name(const struct ovn_port *op) } static void -ovn_update_ipv6_prefix(struct hmap *lr_ports) +update_nb_ipv6_prefix(const struct ovn_port *op) { - const struct ovn_port *op; - HMAP_FOR_EACH (op, key_node, lr_ports) { - ovs_assert(op->nbrp); + ovs_assert(op->nbrp); - if (!smap_get_bool(&op->nbrp->options, "prefix", false)) { - continue; - } + if (!smap_get_bool(&op->nbrp->options, "prefix", false)) { + return; + } - char prefix[IPV6_SCAN_LEN + 6]; - unsigned aid; - const char *ipv6_pd_list = smap_get(&op->sb->options, - "ipv6_ra_pd_list"); - if (!ipv6_pd_list || - !ovs_scan(ipv6_pd_list, "%u:%s", &aid, prefix)) { - continue; - } + char prefix[IPV6_SCAN_LEN + 6]; + unsigned aid; + const char *ipv6_pd_list = smap_get(&op->sb->options, + "ipv6_ra_pd_list"); + if (!ipv6_pd_list || + !ovs_scan(ipv6_pd_list, "%u:%s", &aid, prefix)) { + return; + } + + const char *prefix_ptr = prefix; + nbrec_logical_router_port_set_ipv6_prefix(op->nbrp, &prefix_ptr, 1); +} - const char *prefix_ptr = prefix; - nbrec_logical_router_port_set_ipv6_prefix(op->nbrp, &prefix_ptr, 1); +static void +ovn_update_ipv6_prefix(struct hmap *lr_ports) +{ + const struct ovn_port *op; + HMAP_FOR_EACH (op, key_node, lr_ports) { + update_nb_ipv6_prefix(op); } } @@ -3390,6 +3396,9 @@ ovn_port_update_sbrec(struct ovsdb_idl_txn *ovnsb_txn, { sbrec_port_binding_set_datapath(op->sb, op->od->sb); if (op->nbrp) { + /* Note: SB port binding options for router ports are set in + * sync_pbs(). */ + /* If the router is for l3 gateway, it resides on a chassis * and its port type is "l3gateway". */ const char *chassis_name = smap_get(&op->od->nbr->options, "chassis"); @@ -3401,15 +3410,11 @@ ovn_port_update_sbrec(struct ovsdb_idl_txn *ovnsb_txn, sbrec_port_binding_set_type(op->sb, "patch"); } - struct smap new; - smap_init(&new); if (is_cr_port(op)) { ovs_assert(sbrec_chassis_by_name); ovs_assert(sbrec_chassis_by_hostname); ovs_assert(sbrec_ha_chassis_grp_by_name); ovs_assert(active_ha_chassis_grps); - const char *redirect_type = smap_get(&op->nbrp->options, - "redirect-type"); if (op->nbrp->ha_chassis_group) { if (op->nbrp->n_gateway_chassis) { @@ -3451,49 +3456,8 @@ ovn_port_update_sbrec(struct ovsdb_idl_txn *ovnsb_txn, /* Delete the legacy gateway_chassis from the pb. */ sbrec_port_binding_set_gateway_chassis(op->sb, NULL, 0); } - smap_add(&new, "distributed-port", op->nbrp->name); - - bool always_redirect = - !op->od->has_distributed_nat && - !l3dgw_port_has_associated_vtep_lports(op->l3dgw_port); - - if (redirect_type) { - smap_add(&new, "redirect-type", redirect_type); - /* XXX Why can't we enable always-redirect when redirect-type - * is bridged? */ - if (!strcmp(redirect_type, "bridged")) { - always_redirect = false; - } - } - - if (always_redirect) { - smap_add(&new, "always-redirect", "true"); - } - } else { - if (op->peer) { - smap_add(&new, "peer", op->peer->key); - if (op->nbrp->ha_chassis_group || - op->nbrp->n_gateway_chassis) { - char *redirect_name = - ovn_chassis_redirect_name(op->nbrp->name); - smap_add(&new, "chassis-redirect-port", redirect_name); - free(redirect_name); - } - } - if (chassis_name) { - smap_add(&new, "l3gateway-chassis", chassis_name); - } } - const char *ipv6_pd_list = smap_get(&op->sb->options, - "ipv6_ra_pd_list"); - if (ipv6_pd_list) { - smap_add(&new, "ipv6_ra_pd_list", ipv6_pd_list); - } - - sbrec_port_binding_set_options(op->sb, &new); - smap_destroy(&new); - sbrec_port_binding_set_parent_port(op->sb, NULL); sbrec_port_binding_set_tag(op->sb, NULL, 0); @@ -4667,10 +4631,23 @@ sync_lbs(struct ovsdb_idl_txn *ovnsb_txn, } } -/* Sync the SB Port bindings which needs to be updated. - * Presently it syncs the nat column of port bindings corresponding to - * the logical switch ports. */ -void sync_pbs(struct ovsdb_idl_txn *ovnsb_idl_txn, struct hmap *ls_ports) +static void ovn_update_ipv6_options(struct hmap *lr_ports); +static void ovn_update_ipv6_prefix(struct hmap *lr_ports); + +/* Sync the SB Port bindings which needs to be updated after the northd + * engine is run. + * Presently it syncs + * - the nat column of port bindings corresponding to + * the logical switch ports. + * - 'always-redirect' flag of chassis resident gateway ports. + * + * Note: Ideally we should move ovn_port_update_sbrec() here. + * But until we add full incremental processing to northd engine, + * it is difficult. + * */ +void +sync_pbs(struct ovsdb_idl_txn *ovnsb_idl_txn, struct hmap *ls_ports, + struct hmap *lr_ports) { ovs_assert(ovnsb_idl_txn); @@ -4796,6 +4773,63 @@ void sync_pbs(struct ovsdb_idl_txn *ovnsb_idl_txn, struct hmap *ls_ports) sbrec_port_binding_set_nat_addresses(op->sb, NULL, 0); } } + + /* Only sets the port binding options column for the router ports. + * */ + HMAP_FOR_EACH (op, key_node, lr_ports) { + ovs_assert(op->nbrp); + struct smap new; + smap_init(&new); + + const char *chassis_name = smap_get(&op->od->nbr->options, "chassis"); + if (is_cr_port(op)) { + smap_add(&new, "distributed-port", op->nbrp->name); + + bool always_redirect = + !op->od->has_distributed_nat && + !l3dgw_port_has_associated_vtep_lports(op->l3dgw_port); + + const char *redirect_type = smap_get(&op->nbrp->options, + "redirect-type"); + if (redirect_type) { + smap_add(&new, "redirect-type", redirect_type); + /* XXX Why can't we enable always-redirect when redirect-type + * is bridged? */ + if (!strcmp(redirect_type, "bridged")) { + always_redirect = false; + } + } + + if (always_redirect) { + smap_add(&new, "always-redirect", "true"); + } + } else { + if (op->peer) { + smap_add(&new, "peer", op->peer->key); + if (op->nbrp->ha_chassis_group || + op->nbrp->n_gateway_chassis) { + char *redirect_name = + ovn_chassis_redirect_name(op->nbrp->name); + smap_add(&new, "chassis-redirect-port", redirect_name); + free(redirect_name); + } + } + if (chassis_name) { + smap_add(&new, "l3gateway-chassis", chassis_name); + } + } + + const char *ipv6_pd_list = smap_get(&op->sb->options, + "ipv6_ra_pd_list"); + if (ipv6_pd_list) { + smap_add(&new, "ipv6_ra_pd_list", ipv6_pd_list); + } + + sbrec_port_binding_set_options(op->sb, &new); + smap_destroy(&new); + } + + ovn_update_ipv6_options(lr_ports); } static bool @@ -5690,20 +5724,23 @@ fail: bool northd_handle_sb_port_binding_changes( const struct sbrec_port_binding_table *sbrec_port_binding_table, - struct hmap *ls_ports) + struct hmap *ls_ports, struct hmap *lr_ports) { const struct sbrec_port_binding *pb; static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 5); SBREC_PORT_BINDING_TABLE_FOR_EACH_TRACKED (pb, sbrec_port_binding_table) { /* Return false if the 'pb' belongs to a router port. We don't handle * I-P for router ports yet. */ - if (is_pb_router_type(pb)) { - return false; - } + bool is_router_port = is_pb_router_type(pb); - struct ovn_port *op = ovn_port_find(ls_ports, pb->logical_port); - if (op && !op->lsp_can_be_inc_processed) { - return false; + struct ovn_port *op; + if (is_router_port) { + op = ovn_port_find(lr_ports, pb->logical_port); + } else { + op = ovn_port_find(ls_ports, pb->logical_port); + if (op && !op->lsp_can_be_inc_processed) { + return false; + } } if (sbrec_port_binding_is_new(pb)) { @@ -5712,7 +5749,8 @@ northd_handle_sb_port_binding_changes( * pointer in northd data. Fallback to recompute otherwise. */ if (!op) { VLOG_WARN_RL(&rl, "A port-binding for %s is created but the " - "LSP is not found.", pb->logical_port); + "%s is not found.", pb->logical_port, + is_router_port ? "LRP" : "LSP"); return false; } op->sb = pb; @@ -5723,7 +5761,8 @@ northd_handle_sb_port_binding_changes( * sb idl pointers and other unexpected behavior. */ if (op) { VLOG_WARN_RL(&rl, "A port-binding for %s is deleted but the " - "LSP still exists.", pb->logical_port); + "%s still exists.", pb->logical_port, + is_router_port ? "LRP" : "LSP"); return false; } } else { @@ -5733,7 +5772,8 @@ northd_handle_sb_port_binding_changes( * Fallback to recompute for anything unexpected. */ if (!op) { VLOG_WARN_RL(&rl, "A port-binding for %s is updated but the " - "LSP is not found.", pb->logical_port); + "%s is not found.", pb->logical_port, + is_router_port ? "LRP" : "LSP"); return false; } if (op->sb != pb) { @@ -5741,6 +5781,10 @@ northd_handle_sb_port_binding_changes( "IDL row, which is unusual.", pb->logical_port); return false; } + + if (op && is_router_port) { + update_nb_ipv6_prefix(op); + } } } return true; @@ -19487,7 +19531,6 @@ ovnnb_db_run(struct northd_input *input_data, &data->lr_ports); stopwatch_stop(BUILD_LFLOWS_CTX_STOPWATCH_NAME, time_msec()); stopwatch_start(CLEAR_LFLOWS_CTX_STOPWATCH_NAME, time_msec()); - ovn_update_ipv6_options(&data->lr_ports); ovn_update_ipv6_prefix(&data->lr_ports); sync_port_groups(ovnsb_txn, input_data->sbrec_port_group_table, diff --git a/northd/northd.h b/northd/northd.h index b836aa737f..fc2c083612 100644 --- a/northd/northd.h +++ b/northd/northd.h @@ -394,7 +394,8 @@ bool lflow_handle_northd_lb_changes(struct ovsdb_idl_txn *ovnsb_txn, struct lflow_input *lflow_input, struct lflow_data *lflow_data); bool northd_handle_sb_port_binding_changes( - const struct sbrec_port_binding_table *, struct hmap *ls_ports); + const struct sbrec_port_binding_table *, struct hmap *ls_ports, + struct hmap *lr_ports); struct tracked_lb_data; bool northd_handle_lb_data_changes_pre_od(struct tracked_lb_data *, @@ -426,6 +427,7 @@ const char *northd_get_svc_monitor_mac(void); void sync_lbs(struct ovsdb_idl_txn *, const struct sbrec_load_balancer_table *, struct ovn_datapaths *ls_datapaths, struct hmap *lbs); -void sync_pbs(struct ovsdb_idl_txn *, struct hmap *ls_ports); +void sync_pbs(struct ovsdb_idl_txn *, struct hmap *ls_ports, + struct hmap *lr_ports); #endif /* NORTHD_H */ diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at index 4424a1f64d..37bfc3e0cf 100644 --- a/tests/ovn-northd.at +++ b/tests/ovn-northd.at @@ -10346,3 +10346,52 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE AT_CLEANUP ]) + +OVN_FOR_EACH_NORTHD_NO_HV([ +AT_SETUP([SB port binding incremental processing]) +ovn_start + +check ovn-nbctl ls-add sw0 +check ovn-nbctl lsp-add sw0 sw0-port1 +check ovn-nbctl lsp-set-addresses sw0-port1 "50:54:00:00:00:03 10.0.0.3" + +# Create a logical router and attach both logical switches +check ovn-nbctl lr-add lr0 +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb lrp-add lr0 lr0-sw0 00:00:00:00:ff:01 10.0.0.1/24 +# northd engine node should have only one recompute count. +node_recompute_ct=$(as northd ovn-appctl -t NORTHD_TYPE inc-engine/show-stats northd recompute) +check test "$node_recompute_ct" -eq "1" +lflow_recompute_ct=$(as northd ovn-appctl -t NORTHD_TYPE inc-engine/show-stats lflow recompute) +check test "$node_recompute_ct" -eq "1" + +# Add a gw port to lr0 +ovn-sbctl chassis-add gw1 geneve 127.0.0.1 +ovn-nbctl --wait=sb sync +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb lrp-add lr0 lr0-public 00:00:00:00:ff:01 10.0.0.1/24 + +node_recompute_ct=$(as northd ovn-appctl -t NORTHD_TYPE inc-engine/show-stats northd recompute) +check test "$node_recompute_ct" -eq "1" +lflow_recompute_ct=$(as northd ovn-appctl -t NORTHD_TYPE inc-engine/show-stats lflow recompute) +check test "$node_recompute_ct" -eq "1" + +as northd ovn-appctl -t NORTHD_TYPE vlog/set dbg + +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl lrp-set-gateway-chassis lr0-public gw1 10 +node_recompute_ct=$(as northd ovn-appctl -t NORTHD_TYPE inc-engine/show-stats northd recompute) +# recompute count would be 2 because we don't handle SB HA_Chassis_Group changes. +check test "$node_recompute_ct" -eq "2" +lflow_recompute_ct=$(as northd ovn-appctl -t NORTHD_TYPE inc-engine/show-stats lflow recompute) +check test "$node_recompute_ct" -eq "2" + +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl set logical_router_port lr0-public options:foo=bar +node_recompute_ct=$(as northd ovn-appctl -t NORTHD_TYPE inc-engine/show-stats northd recompute) +check test "$node_recompute_ct" -eq "1" +lflow_recompute_ct=$(as northd ovn-appctl -t NORTHD_TYPE inc-engine/show-stats lflow recompute) +check test "$node_recompute_ct" -eq "1" + +AT_CLEANUP +]) From patchwork Fri Aug 18 08:58:51 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Numan Siddique X-Patchwork-Id: 1822768 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.138; helo=smtp1.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4RRwnj2DZCz1ygW for ; Fri, 18 Aug 2023 19:00:41 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 5A2F983FB7; Fri, 18 Aug 2023 09:00:39 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 5A2F983FB7 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id IHY-ewg37s60; Fri, 18 Aug 2023 09:00:33 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp1.osuosl.org (Postfix) with ESMTPS id 0C11E841EF; Fri, 18 Aug 2023 09:00:30 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 0C11E841EF Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id B4E35C0039; Fri, 18 Aug 2023 09:00:30 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp1.osuosl.org (smtp1.osuosl.org [IPv6:2605:bc80:3010::138]) by lists.linuxfoundation.org (Postfix) with ESMTP id 05A7BC0032 for ; Fri, 18 Aug 2023 09:00:29 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 89C2B83F93 for ; Fri, 18 Aug 2023 08:59:02 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 89C2B83F93 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id CZXAVqP5a9YO for ; Fri, 18 Aug 2023 08:59:00 +0000 (UTC) Received: from relay1-d.mail.gandi.net (relay1-d.mail.gandi.net [IPv6:2001:4b98:dc4:8::221]) by smtp1.osuosl.org (Postfix) with ESMTPS id C7FC383FC6 for ; Fri, 18 Aug 2023 08:58:59 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org C7FC383FC6 Received: by mail.gandi.net (Postfix) with ESMTPSA id ADC3424000A; Fri, 18 Aug 2023 08:58:55 +0000 (UTC) From: numans@ovn.org To: dev@openvswitch.org Date: Fri, 18 Aug 2023 14:28:51 +0530 Message-Id: <20230818085851.1031153-1-numans@ovn.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230818085606.1030792-1-numans@ovn.org> References: <20230818085606.1030792-1-numans@ovn.org> MIME-Version: 1.0 X-GND-Sasl: numans@ovn.org Subject: [ovs-dev] [PATCH ovn v6 16/16] northd: Handle NAT changes for a logical router incrementally. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Numan Siddique This patch adds the I-P support for any changes to router NATs in both the northd and lflow engine nodes. Signed-off-by: Numan Siddique --- northd/en-lflow.c | 10 + northd/en-northd.c | 30 ++- northd/en-northd.h | 4 +- northd/inc-proc-northd.c | 4 +- northd/northd.c | 463 ++++++++++++++++++++++++++++++++------- northd/northd.h | 23 +- tests/ovn-northd.at | 202 +++++++++++++++++ 7 files changed, 650 insertions(+), 86 deletions(-) diff --git a/northd/en-lflow.c b/northd/en-lflow.c index 56fe564c82..ad5ebc53fc 100644 --- a/northd/en-lflow.c +++ b/northd/en-lflow.c @@ -105,6 +105,10 @@ lflow_northd_handler(struct engine_node *node, return false; } + if (northd_data->trk_northd_changes.lb_nats_changed) { + return false; + } + const struct engine_context *eng_ctx = engine_get_context(); struct lflow_data *lflow_data = data; @@ -123,6 +127,12 @@ lflow_northd_handler(struct engine_node *node, return false; } + if (!lflow_handle_northd_od_changes(eng_ctx->ovnsb_idl_txn, + &northd_data->trk_northd_changes.trk_datapaths, + &lflow_input, lflow_data)) { + return false; + } + engine_set_node_state(node, EN_UPDATED); return true; diff --git a/northd/en-northd.c b/northd/en-northd.c index b3e84a6af6..5fa9ecbbc1 100644 --- a/northd/en-northd.c +++ b/northd/en-northd.c @@ -207,19 +207,41 @@ northd_sb_port_binding_handler(struct engine_node *node, } bool -northd_nb_logical_router_handler(struct engine_node *node, - void *data) +northd_nb_logical_router_handler_pre_lb(struct engine_node *node, + void *data) { struct northd_data *nd = data; struct northd_input input_data; northd_get_input_data(node, &input_data); - if (!northd_handle_lr_changes(&input_data, nd)) { + if (!northd_handle_lr_changes_pre_lb(&input_data, nd)) { return false; } - if (nd->change_tracked) { + if (northd_has_tracked_data(&nd->trk_northd_changes)) { + nd->change_tracked = true; + engine_set_node_state(node, EN_UPDATED); + } + + return true; +} + +bool +northd_nb_logical_router_handler_post_lb(struct engine_node *node, + void *data) +{ + struct northd_data *nd = data; + struct northd_input input_data; + + northd_get_input_data(node, &input_data); + + if (!northd_handle_lr_changes_post_lb(&input_data, nd)) { + return false; + } + + if (northd_has_tracked_data(&nd->trk_northd_changes)) { + nd->change_tracked = true; engine_set_node_state(node, EN_UPDATED); } diff --git a/northd/en-northd.h b/northd/en-northd.h index 84d8673e1b..02af2a758c 100644 --- a/northd/en-northd.h +++ b/northd/en-northd.h @@ -16,7 +16,9 @@ void en_northd_cleanup(void *data); void en_northd_clear_tracked_data(void *data); bool northd_nb_nb_global_handler(struct engine_node *, void *data OVS_UNUSED); bool northd_nb_logical_switch_handler(struct engine_node *, void *data); -bool northd_nb_logical_router_handler(struct engine_node *, void *data); +bool northd_nb_logical_router_handler_pre_lb(struct engine_node *, void *data); +bool northd_nb_logical_router_handler_post_lb(struct engine_node *, + void *data); bool northd_sb_port_binding_handler(struct engine_node *, void *data); bool northd_lb_data_handler_pre_od(struct engine_node *, void *data); bool northd_lb_data_handler_post_od(struct engine_node *, void *data); diff --git a/northd/inc-proc-northd.c b/northd/inc-proc-northd.c index e3bc2bda7b..12a5990521 100644 --- a/northd/inc-proc-northd.c +++ b/northd/inc-proc-northd.c @@ -189,8 +189,10 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, engine_add_input(&en_northd, &en_nb_logical_switch, northd_nb_logical_switch_handler); engine_add_input(&en_northd, &en_nb_logical_router, - northd_nb_logical_router_handler); + northd_nb_logical_router_handler_pre_lb); engine_add_input(&en_northd, &en_lb_data, northd_lb_data_handler_post_od); + engine_add_input(&en_northd, &en_nb_logical_router, + northd_nb_logical_router_handler_post_lb); engine_add_input(&en_mac_binding_aging, &en_nb_nb_global, NULL); engine_add_input(&en_mac_binding_aging, &en_sb_mac_binding, NULL); diff --git a/northd/northd.c b/northd/northd.c index 2ae41b5d8a..28e1f43a38 100644 --- a/northd/northd.c +++ b/northd/northd.c @@ -703,6 +703,18 @@ destroy_nat_entries(struct ovn_datapath *od) } } +static void +reinit_nat_entries(struct ovn_datapath *od) +{ + ovs_assert(od->nbr); + destroy_nat_entries(od); + od->n_nat_entries = 0; + od->has_distributed_nat = false; + free(od->nat_entries); + od->nat_entries = NULL; + init_nat_entries(od); +} + static void init_router_external_ips(struct ovn_datapath *od) { @@ -724,6 +736,14 @@ destroy_router_external_ips(struct ovn_datapath *od) sset_destroy(&od->external_ips); } +static void +reinit_router_external_ips(struct ovn_datapath *od) +{ + ovs_assert(od->nbr); + destroy_router_external_ips(od); + init_router_external_ips(od); +} + static bool lb_has_vip(const struct nbrec_load_balancer *lb) { @@ -792,6 +812,47 @@ destroy_lb_for_datapath(struct ovn_datapath *od) od->lb_ips = NULL; } +/* Initiailizes the router datapath's lb_nats sset which contains + * set of NAT ips which are also in the load balancer vips + * associated with the router datapath. */ +static void +init_router_lb_nats(struct ovn_datapath *od) +{ + ovs_assert(od->nbr); + + sset_init(&od->lb_nats); + const char *external_ip; + SSET_FOR_EACH (external_ip, &od->external_ips) { + bool is_vip_nat = false; + if (addr_is_ipv6(external_ip)) { + is_vip_nat = sset_contains(&od->lb_ips->ips_v6, external_ip); + } else { + is_vip_nat = sset_contains(&od->lb_ips->ips_v4, external_ip); + } + + if (is_vip_nat) { + sset_add(&od->lb_nats, external_ip); + } + } +} + +static void +destroy_router_lb_nats(struct ovn_datapath *od) +{ + if (od->nbr) { + sset_destroy(&od->lb_nats); + } +} + +static void +build_router_lb_nats(struct hmap *lr_datapaths) +{ + struct ovn_datapath *od; + HMAP_FOR_EACH (od, key_node, lr_datapaths) { + init_router_lb_nats(od); + } +} + /* A group of logical router datapaths which are connected - either * directly or indirectly. * Each logical router can belong to only one group. */ @@ -847,6 +908,7 @@ ovn_datapath_destroy(struct hmap *datapaths, struct ovn_datapath *od) free(od->ls_peers); destroy_nat_entries(od); destroy_router_external_ips(od); + destroy_router_lb_nats(od); destroy_lb_for_datapath(od); free(od->nat_entries); free(od->localnet_ports); @@ -1594,6 +1656,8 @@ destroy_routable_addresses(struct ovn_port_routable_addresses *ra) destroy_lport_addresses(&ra->laddrs[i]); } free(ra->laddrs); + ra->laddrs = NULL; + ra->n_addrs = 0; } static char **get_nat_addresses(const struct ovn_port *op, size_t *n, @@ -1638,6 +1702,14 @@ ovn_port_set_nb(struct ovn_port *op, op->lsp_can_be_inc_processed = lsp_can_be_inc_processed(nbsp); } op->nbrp = nbrp; + + if (nbrp) { + if (is_cr_port(op)) { + /* ovn-northd doesn't have to do anything when a chassis-redirect + * port gets updated as its an internal port created by northd. */ + op->lsp_can_be_inc_processed = true; + } + } init_mcast_port_info(&op->mcast_info, op->nbsp, op->nbrp); } @@ -5071,12 +5143,20 @@ destroy_tracked_lb_datapaths(struct tracked_lb_datapaths *trk_lbs) trk_lbs->n_nb_lr = 0; } +static void +destroy_tracked_datapaths(struct tracked_datapaths *trk_dps) +{ + hmapx_clear(&trk_dps->crupdated); +} + void destroy_northd_data_tracked_changes(struct northd_data *nd) { struct northd_tracked_data *trk_changes = &nd->trk_northd_changes; destroy_tracked_ovn_ports(&trk_changes->trk_ovn_ports); destroy_tracked_lb_datapaths(&trk_changes->trk_lbs); + destroy_tracked_datapaths(&trk_changes->trk_datapaths); + trk_changes->lb_nats_changed = false; nd->change_tracked = false; } @@ -5088,7 +5168,8 @@ bool northd_has_tracked_data(struct northd_tracked_data *trk_nd_changes) || !hmap_is_empty(&trk_nd_changes->trk_ovn_ports.updated) || !hmap_is_empty(&trk_nd_changes->trk_ovn_ports.deleted) || !hmapx_is_empty(&trk_nd_changes->trk_lbs.crupdated) - || !hmapx_is_empty(&trk_nd_changes->trk_lbs.deleted)); + || !hmapx_is_empty(&trk_nd_changes->trk_lbs.deleted) + || !hmapx_is_empty(&trk_nd_changes->trk_datapaths.crupdated)); } bool northd_has_only_ports_in_tracked_data( @@ -5098,6 +5179,7 @@ bool northd_has_only_ports_in_tracked_data( && !trk_nd_changes->trk_lbs.n_nb_lr && hmapx_is_empty(&trk_nd_changes->trk_lbs.crupdated) && hmapx_is_empty(&trk_nd_changes->trk_lbs.deleted) + && hmapx_is_empty(&trk_nd_changes->trk_datapaths.crupdated) && (!hmap_is_empty(&trk_nd_changes->trk_ovn_ports.created) || !hmap_is_empty(&trk_nd_changes->trk_ovn_ports.updated) || !hmap_is_empty(&trk_nd_changes->trk_ovn_ports.deleted))); @@ -5646,7 +5728,8 @@ check_unsupported_inc_proc_for_lr_changes( for (col = 0; col < NBREC_LOGICAL_ROUTER_N_COLUMNS; col++) { if (nbrec_logical_router_is_updated(lr, col)) { if (col == NBREC_LOGICAL_ROUTER_COL_LOAD_BALANCER || - col == NBREC_LOGICAL_ROUTER_COL_LOAD_BALANCER_GROUP) { + col == NBREC_LOGICAL_ROUTER_COL_LOAD_BALANCER_GROUP || + col == NBREC_LOGICAL_ROUTER_COL_NAT) { continue; } return true; @@ -5665,12 +5748,6 @@ check_unsupported_inc_proc_for_lr_changes( OVSDB_IDL_CHANGE_MODIFY) > 0) { return true; } - for (size_t i = 0; i < lr->n_nat; i++) { - if (nbrec_nat_row_get_seqno(lr->nat[i], - OVSDB_IDL_CHANGE_MODIFY) > 0) { - return true; - } - } for (size_t i = 0; i < lr->n_policies; i++) { if (nbrec_logical_router_policy_row_get_seqno(lr->policies[i], OVSDB_IDL_CHANGE_MODIFY) > 0) { @@ -5686,6 +5763,93 @@ check_unsupported_inc_proc_for_lr_changes( return false; } +static bool +handle_lr_nat_changes(const struct nbrec_logical_router *lr, + struct ovn_datapath *od, + bool *changed) +{ + /* If nothing has changed, return */ + bool nat_changed = false; + if (nbrec_logical_router_is_updated(lr, NBREC_LOGICAL_ROUTER_COL_NAT)) { + nat_changed = true; + for (size_t i = 0; i < lr->n_nat; i++) { + if ((nbrec_nat_row_get_seqno(lr->nat[i], + OVSDB_IDL_CHANGE_INSERT) > 0) || + (nbrec_nat_row_get_seqno(lr->nat[i], + OVSDB_IDL_CHANGE_DELETE) > 0)) { + /* Check if the modified NAT has add_route options set. + * If so, return false. */ + if (smap_get_bool(&lr->nat[i]->options, "add_route", false)) { + return false; + } + nat_changed = true; + break; + } + } + } else { + for (size_t i = 0; i < lr->n_nat; i++) { + if (nbrec_nat_row_get_seqno(lr->nat[i], + OVSDB_IDL_CHANGE_MODIFY) > 0) { + /* Check if the modified NAT has add_route options set. + * If so, return false. */ + if (smap_get_bool(&lr->nat[i]->options, "add_route", false)) { + return false; + } + nat_changed = true; + break; + } + } + } + + if (!nat_changed) { + return true; + } + + *changed = true; + reinit_nat_entries(od); + reinit_router_external_ips(od); + return true; +} + +static bool +lr_lb_nats_changed(struct ovn_datapath *od) +{ + /* Check if NATs changed. */ + bool nat_changed = + nbrec_logical_router_is_updated(od->nbr, NBREC_LOGICAL_ROUTER_COL_NAT); + + if (!nat_changed) { + for (size_t i = 0; i < od->nbr->n_nat; i++) { + if (nbrec_nat_row_get_seqno(od->nbr->nat[i], + OVSDB_IDL_CHANGE_MODIFY) > 0) { + nat_changed = true; + break; + } + } + } + + if (!nat_changed) { + return false; + } + + struct sset old_lb_nats = SSET_INITIALIZER(&old_lb_nats); + bool lb_nats_changed; + + sset_swap(&od->lb_nats, &old_lb_nats); + sset_destroy(&od->lb_nats); + init_router_lb_nats(od); + + lb_nats_changed = sset_count(&od->lb_nats) != sset_count(&old_lb_nats); + + if (!lb_nats_changed) { + lb_nats_changed = !sset_equals(&od->lb_nats, &old_lb_nats); + } + + sset_destroy(&old_lb_nats); + + return lb_nats_changed; +} + /* Return true if changes are handled incrementally, false otherwise. * When there are any changes, try to track what's exactly changed and set * northd_data->change_tracked accordingly: change tracked - true, otherwise, @@ -5696,8 +5860,8 @@ check_unsupported_inc_proc_for_lr_changes( * northd_handle_lb_data_changes_post_od). * */ bool -northd_handle_lr_changes(const struct northd_input *ni, - struct northd_data *nd) +northd_handle_lr_changes_pre_lb(const struct northd_input *ni, + struct northd_data *nd) { const struct nbrec_logical_router *changed_lr; @@ -5713,6 +5877,43 @@ northd_handle_lr_changes(const struct northd_input *ni, if (check_unsupported_inc_proc_for_lr_changes(changed_lr)) { goto fail; } + + struct ovn_datapath *od = ovn_datapath_find( + &nd->lr_datapaths.datapaths, &changed_lr->header_.uuid); + if (!od) { + static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 1); + VLOG_WARN_RL(&rl, "Internal error: a tracked updated LR doesn't " + "exist in lr_datapaths: "UUID_FMT, + UUID_ARGS(&changed_lr->header_.uuid)); + return false; + } + + bool lr_nat_changed = false; + if (!handle_lr_nat_changes(changed_lr, od, + &lr_nat_changed)) { + goto fail; + } + + if (lr_nat_changed) { + nd->change_tracked = true; + hmapx_add(&nd->trk_northd_changes.trk_datapaths.crupdated, od); + + /* Also add the peer (logical switch port) of logical router ports + * to the northd tracked data. */ + struct ovn_port *op; + uint8_t lport_changes = + (EN_TRACKED_OP_LPORT_CHANGED | EN_TRACKED_OP_LB_CHANGED); + HMAP_FOR_EACH (op, dp_node, &od->ports) { + add_op_to_northd_tracked_ports( + &nd->trk_northd_changes.trk_ovn_ports.updated, op, + lport_changes); + if (op->peer) { + add_op_to_northd_tracked_ports( + &nd->trk_northd_changes.trk_ovn_ports.updated, + op->peer, lport_changes); + } + } + } } return true; @@ -5721,6 +5922,26 @@ fail: return false; } +bool +northd_handle_lr_changes_post_lb(const struct northd_input *ni, + struct northd_data *nd) +{ + const struct nbrec_logical_router *changed_lr; + + NBREC_LOGICAL_ROUTER_TABLE_FOR_EACH_TRACKED (changed_lr, + ni->nbrec_logical_router_table) { + struct ovn_datapath *od = ovn_datapath_find( + &nd->lr_datapaths.datapaths, &changed_lr->header_.uuid); + ovs_assert(od); + + if (lr_lb_nats_changed(od)) { + nd->trk_northd_changes.lb_nats_changed = true; + } + } + + return true; +} + bool northd_handle_sb_port_binding_changes( const struct sbrec_port_binding_table *sbrec_port_binding_table, @@ -17118,38 +17339,42 @@ build_lswitch_and_lrouter_iterate_by_ls(struct ovn_datapath *od, */ static void build_lswitch_and_lrouter_iterate_by_lr(struct ovn_datapath *od, - struct lswitch_flow_build_info *lsi) + const struct shash *meter_groups, + const struct hmap *lr_ports, + const struct hmap *ls_ports, + const struct chassis_features *features, + const struct hmap *bfd_connections, + struct ds *match, + struct ds *actions, + struct lflow_data *lflows) { ovs_assert(od->nbr); - build_adm_ctrl_flows_for_lrouter(od, lsi->lflows); - build_neigh_learning_flows_for_lrouter(od, lsi->lflows, &lsi->match, - &lsi->actions, lsi->meter_groups); - build_ND_RA_flows_for_lrouter(od, lsi->lflows); - build_ip_routing_pre_flows_for_lrouter(od, lsi->lflows); - build_static_route_flows_for_lrouter(od, lsi->features, - lsi->lflows, lsi->lr_ports, - lsi->bfd_connections); - build_mcast_lookup_flows_for_lrouter(od, lsi->lflows, &lsi->match, - &lsi->actions); - build_ingress_policy_flows_for_lrouter(od, lsi->lflows, lsi->lr_ports); - build_arp_resolve_flows_for_lrouter(od, lsi->lflows); - build_check_pkt_len_flows_for_lrouter(od, lsi->lflows, lsi->lr_ports, - &lsi->match, &lsi->actions, - lsi->meter_groups); - build_gateway_redirect_flows_for_lrouter(od, lsi->lflows, &lsi->match, - &lsi->actions); - build_arp_request_flows_for_lrouter(od, lsi->lflows, &lsi->match, - &lsi->actions, lsi->meter_groups); - build_misc_local_traffic_drop_flows_for_lrouter(od, lsi->lflows); - build_lrouter_arp_nd_for_datapath(od, lsi->lflows, lsi->meter_groups); - build_lrouter_nat_defrag_and_lb(od, lsi->lflows, lsi->ls_ports, - lsi->lr_ports, &lsi->match, - &lsi->actions, lsi->meter_groups, - lsi->features); - build_lrouter_nat_flows_lb_related(od, lsi->lflows, &lsi->match, - lsi->features); - build_lrouter_nat_flows_ct_lb_related(od, lsi->lflows); - build_lrouter_lb_affinity_default_flows(od, lsi->lflows); + build_adm_ctrl_flows_for_lrouter(od, lflows); + build_neigh_learning_flows_for_lrouter(od, lflows, match, + actions, meter_groups); + build_ND_RA_flows_for_lrouter(od, lflows); + build_ip_routing_pre_flows_for_lrouter(od, lflows); + build_static_route_flows_for_lrouter(od, features, + lflows, lr_ports, + bfd_connections); + build_mcast_lookup_flows_for_lrouter(od, lflows, match, + actions); + build_ingress_policy_flows_for_lrouter(od, lflows, lr_ports); + build_arp_resolve_flows_for_lrouter(od, lflows); + build_check_pkt_len_flows_for_lrouter(od, lflows, lr_ports, match, actions, + meter_groups); + build_gateway_redirect_flows_for_lrouter(od, lflows, match, actions); + build_arp_request_flows_for_lrouter(od, lflows, match, + actions, meter_groups); + build_misc_local_traffic_drop_flows_for_lrouter(od, lflows); + build_lrouter_arp_nd_for_datapath(od, lflows, meter_groups); + build_lrouter_nat_defrag_and_lb(od, lflows, ls_ports, + lr_ports, match, + actions, meter_groups, + features); + build_lrouter_nat_flows_lb_related(od, lflows, match, features); + build_lrouter_nat_flows_ct_lb_related(od, lflows); + build_lrouter_lb_affinity_default_flows(od, lflows); } /* Helper function to combine all lflow generation which is iterated by logical @@ -17205,37 +17430,38 @@ build_lb_related_iterate_by_lsp(struct ovn_port *op, */ static void build_lswitch_and_lrouter_iterate_by_lrp(struct ovn_port *op, - struct lswitch_flow_build_info *lsi) + const struct shash *meter_groups, + struct ds *match, + struct ds *actions, + struct lflow_data *lflows) { ovs_assert(op->nbrp); - build_adm_ctrl_flows_for_lrouter_port(op, lsi->lflows, &lsi->match, - &lsi->actions); - build_neigh_learning_flows_for_lrouter_port(op, lsi->lflows, &lsi->match, - &lsi->actions); - build_ip_routing_flows_for_lrp(op, lsi->lflows); - build_ND_RA_flows_for_lrouter_port(op, lsi->lflows, &lsi->match, - &lsi->actions, lsi->meter_groups); - build_arp_resolve_flows_for_lrp(op, lsi->lflows, &lsi->match, - &lsi->actions); - build_egress_delivery_flows_for_lrouter_port(op, lsi->lflows, &lsi->match, - &lsi->actions); - build_dhcpv6_reply_flows_for_lrouter_port(op, lsi->lflows, &lsi->match); - build_ipv6_input_flows_for_lrouter_port(op, lsi->lflows, - &lsi->match, &lsi->actions, - lsi->meter_groups); - build_lrouter_ipv4_ip_input(op, lsi->lflows, - &lsi->match, &lsi->actions, lsi->meter_groups); + build_adm_ctrl_flows_for_lrouter_port(op, lflows, match, + actions); + build_neigh_learning_flows_for_lrouter_port(op, lflows, match, + actions); + build_ip_routing_flows_for_lrp(op, lflows); + build_ND_RA_flows_for_lrouter_port(op, lflows, match, actions, + meter_groups); + build_arp_resolve_flows_for_lrp(op, lflows, match, actions); + build_egress_delivery_flows_for_lrouter_port(op, lflows, match, + actions); + build_dhcpv6_reply_flows_for_lrouter_port(op, lflows, match); + build_ipv6_input_flows_for_lrouter_port(op, lflows, match, actions, + meter_groups); + build_lrouter_ipv4_ip_input(op, lflows, match, actions, meter_groups); } static void build_lb_related_iterate_by_lrp(struct ovn_port *op, - struct lswitch_flow_build_info *lsi) + const struct shash *meter_groups, + struct ds *match, + struct ds *actions, + struct lflow_data *lflows) { ovs_assert(op->nbrp); - build_lrouter_ipv4_ip_input_lb_related(op, lsi->lflows, &lsi->match, - lsi->meter_groups); - build_lrouter_force_snat_flows_op(op, lsi->lflows, &lsi->match, - &lsi->actions); + build_lrouter_ipv4_ip_input_lb_related(op, lflows, match, meter_groups); + build_lrouter_force_snat_flows_op(op, lflows, match, actions); } static void * @@ -17280,7 +17506,10 @@ build_lflows_thread(void *arg) if (stop_parallel_processing()) { return NULL; } - build_lswitch_and_lrouter_iterate_by_lr(od, lsi); + build_lswitch_and_lrouter_iterate_by_lr( + od, lsi->meter_groups, lsi->lr_ports, lsi->ls_ports, + lsi->features, lsi->bfd_connections, &lsi->match, + &lsi->actions, lsi->lflows); } } for (bnum = control->id; @@ -17312,8 +17541,12 @@ build_lflows_thread(void *arg) if (stop_parallel_processing()) { return NULL; } - build_lswitch_and_lrouter_iterate_by_lrp(op, lsi); - build_lb_related_iterate_by_lrp(op, lsi); + build_lswitch_and_lrouter_iterate_by_lrp( + op, lsi->meter_groups, &lsi->match, &lsi->actions, + lsi->lflows); + build_lb_related_iterate_by_lrp( + op, lsi->meter_groups, &lsi->match, &lsi->actions, + lsi->lflows); } } for (bnum = control->id; @@ -17492,7 +17725,10 @@ build_lswitch_and_lrouter_flows(const struct ovn_datapaths *ls_datapaths, build_lswitch_and_lrouter_iterate_by_ls(od, &lsi); } HMAP_FOR_EACH (od, key_node, &lr_datapaths->datapaths) { - build_lswitch_and_lrouter_iterate_by_lr(od, &lsi); + build_lswitch_and_lrouter_iterate_by_lr( + od, lsi.meter_groups, lsi.lr_ports, lsi.ls_ports, + lsi.features, lsi.bfd_connections, &lsi.match, + &lsi.actions, lsi.lflows); } stopwatch_stop(LFLOWS_DATAPATHS_STOPWATCH_NAME, time_msec()); stopwatch_start(LFLOWS_PORTS_STOPWATCH_NAME, time_msec()); @@ -17506,8 +17742,12 @@ build_lswitch_and_lrouter_flows(const struct ovn_datapaths *ls_datapaths, &lsi.actions, lsi.lflows); } HMAP_FOR_EACH (op, key_node, lr_ports) { - build_lswitch_and_lrouter_iterate_by_lrp(op, &lsi); - build_lb_related_iterate_by_lrp(op, &lsi); + build_lswitch_and_lrouter_iterate_by_lrp(op, lsi.meter_groups, + &lsi.match, + &lsi.actions, + lsi.lflows); + build_lb_related_iterate_by_lrp(op, lsi.meter_groups, &lsi.match, + &lsi.actions, lsi.lflows); } stopwatch_stop(LFLOWS_PORTS_STOPWATCH_NAME, time_msec()); stopwatch_start(LFLOWS_LBS_STOPWATCH_NAME, time_msec()); @@ -18193,20 +18433,20 @@ bool lflow_handle_northd_port_changes(struct ovsdb_idl_txn *ovnsb_txn, struct ds match = DS_EMPTY_INITIALIZER; struct ds actions = DS_EMPTY_INITIALIZER; if (trk_op->changes & EN_TRACKED_OP_LPORT_CHANGED) { - /* We still don't support EN_TRACKED_OP_LPORT_CHANGED for - * router ports. */ - ovs_assert(op->nbsp); - /* unlink old lflows. */ unlink_lflows(op->od, OBJDEP_TYPE_LPORT, res_name, lflow_data, &op->lflow_dep_mgr); /* Generate new lflows. */ - build_lswitch_and_lrouter_iterate_by_lsp(op, lflow_input->ls_ports, - lflow_input->lr_ports, - lflow_input->meter_groups, - &match, &actions, - lflow_data); + if (op->nbsp) { + build_lswitch_and_lrouter_iterate_by_lsp( + op, lflow_input->ls_ports, lflow_input->lr_ports, + lflow_input->meter_groups, &match, &actions, lflow_data); + } else { + build_lswitch_and_lrouter_iterate_by_lrp( + op, lflow_input->meter_groups, &match, &actions, + lflow_data); + } } if (trk_op->changes & EN_TRACKED_OP_LB_CHANGED) { @@ -18457,6 +18697,70 @@ bool lflow_handle_northd_lb_changes(struct ovsdb_idl_txn *ovnsb_txn, return true; } +bool +lflow_handle_northd_od_changes(struct ovsdb_idl_txn *ovnsb_txn, + struct tracked_datapaths *trk_dps, + struct lflow_input *lflow_input, + struct lflow_data *lflow_data) +{ + struct ovn_datapath *od; + struct hmapx_node *hmapx_node; + HMAPX_FOR_EACH (hmapx_node, &trk_dps->crupdated) { + od = hmapx_node->data; + + /* We don't support handling logical switches yet. */ + ovs_assert(od->nbr); + + struct resource_to_objects_node *res_node = + objdep_mgr_find_objs(&od->lflow_dep_mgr, OBJDEP_TYPE_OD, + od->nbr->name); + + /* unlink old lflows of type OBJDEP_TYPE_OD. */ + unlink_objres_lflows(res_node, od, lflow_data, + &od->lflow_dep_mgr); + + res_node = objdep_mgr_find_objs( + &od->lflow_dep_mgr, OBJDEP_TYPE_LB, od->nbr->name); + + unlink_objres_lflows(res_node, od, lflow_data, &od->lflow_dep_mgr); + + res_node = objdep_mgr_find_objs( + &od->lflow_dep_mgr, OBJDEP_TYPE_CT_LB, od->nbr->name); + unlink_objres_lflows(res_node, od, lflow_data, &od->lflow_dep_mgr); + + struct ds match = DS_EMPTY_INITIALIZER; + struct ds actions = DS_EMPTY_INITIALIZER; + build_lswitch_and_lrouter_iterate_by_lr( + od, lflow_input->meter_groups, lflow_input->lr_ports, + lflow_input->ls_ports, lflow_input->features, + lflow_input->bfd_connections, &match, + &actions, lflow_data); + ds_destroy(&match); + ds_destroy(&actions); + + /* Sync the updated flows to SB. */ + res_node = objdep_mgr_find_objs(&od->lflow_dep_mgr, + OBJDEP_TYPE_OD, + od->nbr->name); + sync_lflows_from_objres(ovnsb_txn, res_node, lflow_input, + lflow_data, &od->lflow_dep_mgr); + + res_node = objdep_mgr_find_objs(&od->lflow_dep_mgr, + OBJDEP_TYPE_LB, + od->nbr->name); + sync_lflows_from_objres(ovnsb_txn, res_node, lflow_input, + lflow_data, &od->lflow_dep_mgr); + + res_node = objdep_mgr_find_objs(&od->lflow_dep_mgr, + OBJDEP_TYPE_CT_LB, + od->nbr->name); + sync_lflows_from_objres(ovnsb_txn, res_node, lflow_input, + lflow_data, &od->lflow_dep_mgr); + } + + return true; +} + /* Each port group in Port_Group table in OVN_Northbound has a corresponding * entry in Port_Group table in OVN_Southbound. In OVN_Northbound the entries * contains lport uuids, while in OVN_Southbound we store the lport names. @@ -19341,6 +19645,7 @@ northd_init(struct northd_data *data) hmap_init(&data->trk_northd_changes.trk_ovn_ports.deleted); hmapx_init(&data->trk_northd_changes.trk_lbs.crupdated); hmapx_init(&data->trk_northd_changes.trk_lbs.deleted); + hmapx_init(&data->trk_northd_changes.trk_datapaths.crupdated); } void @@ -19399,6 +19704,7 @@ northd_destroy(struct northd_data *data) hmap_destroy(&data->trk_northd_changes.trk_ovn_ports.deleted); hmapx_destroy(&data->trk_northd_changes.trk_lbs.crupdated); hmapx_destroy(&data->trk_northd_changes.trk_lbs.deleted); + hmapx_destroy(&data->trk_northd_changes.trk_datapaths.crupdated); } void @@ -19495,6 +19801,7 @@ ovnnb_db_run(struct northd_input *input_data, build_lb_datapaths(input_data->lbs, input_data->lb_groups, &data->ls_datapaths, &data->lr_datapaths, &data->lb_datapaths_map, &data->lb_group_datapaths_map); + build_router_lb_nats(&data->lr_datapaths.datapaths); build_ports(ovnsb_txn, input_data->sbrec_port_binding_table, input_data->sbrec_chassis_table, diff --git a/northd/northd.h b/northd/northd.h index fc2c083612..a499db00fc 100644 --- a/northd/northd.h +++ b/northd/northd.h @@ -113,6 +113,12 @@ struct tracked_ovn_ports { struct hmap deleted; }; +struct tracked_datapaths { + /* Tracked created or updated logical switches or + * routers. */ + struct hmapx crupdated; +}; + struct tracked_lb_datapaths { /* Tracked created or updated load balancers. * hmapx node data is 'struct ovn_lb_datapaths' */ @@ -137,6 +143,10 @@ struct tracked_lb_datapaths { struct northd_tracked_data { struct tracked_ovn_ports trk_ovn_ports; struct tracked_lb_datapaths trk_lbs; + struct tracked_datapaths trk_datapaths; + + /* Indicates if any router's NATs changed which were also LB vips. */ + bool lb_nats_changed; }; struct northd_data { @@ -334,6 +344,9 @@ struct ovn_datapath { /* Set of nat external ips on the router. */ struct sset external_ips; + /* Set of nat external ips which are lb vips too. */ + struct sset lb_nats; + /* SNAT IPs owned by the router (shash of 'struct ovn_snat_ip'). */ struct shash snat_ips; @@ -375,8 +388,10 @@ void ovnsb_db_run(struct ovsdb_idl_txn *ovnnb_txn, bool northd_handle_ls_changes(struct ovsdb_idl_txn *, const struct northd_input *, struct northd_data *); -bool northd_handle_lr_changes(const struct northd_input *, - struct northd_data *); +bool northd_handle_lr_changes_pre_lb(const struct northd_input *, + struct northd_data *); +bool northd_handle_lr_changes_post_lb(const struct northd_input *, + struct northd_data *); void destroy_northd_data_tracked_changes(struct northd_data *); void northd_destroy(struct northd_data *data); void northd_init(struct northd_data *data); @@ -393,6 +408,10 @@ bool lflow_handle_northd_lb_changes(struct ovsdb_idl_txn *ovnsb_txn, struct tracked_lb_datapaths *, struct lflow_input *lflow_input, struct lflow_data *lflow_data); +bool lflow_handle_northd_od_changes(struct ovsdb_idl_txn *ovnsb_txn, + struct tracked_datapaths *, + struct lflow_input *lflow_input, + struct lflow_data *lflow_data); bool northd_handle_sb_port_binding_changes( const struct sbrec_port_binding_table *, struct hmap *ls_ports, struct hmap *lr_ports); diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at index 37bfc3e0cf..ec28dea9c8 100644 --- a/tests/ovn-northd.at +++ b/tests/ovn-northd.at @@ -10395,3 +10395,205 @@ check test "$node_recompute_ct" -eq "1" AT_CLEANUP ]) + +OVN_FOR_EACH_NORTHD_NO_HV([ +AT_SETUP([Logical router incremental processing for NAT]) + +ovn_start + +net_add n1 +sim_add hv1 +as hv1 +ovs-vsctl add-br br-phys +ovn_attach n1 br-phys 192.168.0.11 + +check_engine_stats() { + node=$1 + recompute=$2 + compute=$3 + + echo "__file__:__line__: Checking engine stats for node $node : recompute - \ +$recompute : compute - $compute" + + node_stat=$(as northd ovn-appctl -t NORTHD_TYPE inc-engine/show-stats $node) + # node_stat will be of this format : + # - Node: lflow - recompute: 3 - compute: 0 - abort: 0 + node_recompute_ct=$(echo $node_stat | cut -d '-' -f2 | cut -d ':' -f2) + node_compute_ct=$(echo $node_stat | cut -d '-' -f3 | cut -d ':' -f2) + + if [[ "$recompute" == "norecompute" ]]; then + # node should not be recomputed + echo "Expecting $node recompute count - $node_recompute_ct to be 0" + check test "$node_recompute_ct" -eq "0" + else + echo "Expecting $node recompute count - $node_recompute_ct not to be 0" + check test "$node_recompute_ct" -ne "0" + fi + + if [[ "$compute" == "nocompute" ]]; then + # node should not be computed + echo "Expecting $node compute count - $node_compute_ct to be 0" + check test "$node_compute_ct" -eq "0" + else + echo "Expecting $node compute count - $node_compute_ct not to be 0" + check test "$node_compute_ct" -ne "0" + fi +} + +ovn-sbctl chassis-add gw1 geneve 127.0.0.1 + +check ovn-nbctl ls-add sw0 +check ovn-nbctl lsp-add sw0 sw0p1 -- lsp-set-addresses sw0p1 "00:00:20:20:12:01 10.0.0.4" + +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb lr-add lr0 +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute + +# Adding a logical router port should result in recompute +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl lrp-add lr0 lr0-sw0 00:00:00:00:ff:01 10.0.0.1/24 +# for northd engine there will be both recompute and compute +# first it will be recompute to handle lr0-sw0 and then a compute +# for the SB port binding change. +check_engine_stats northd recompute compute +check_engine_stats lflow recompute nocompute + +ovn-nbctl lsp-add sw0 sw0-lr0 +ovn-nbctl lsp-set-type sw0-lr0 router +ovn-nbctl lsp-set-addresses sw0-lr0 00:00:00:00:ff:01 +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb lsp-set-options sw0-lr0 router-port=lr0-sw0 +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute + +ovn-nbctl ls-add public +ovn-nbctl lrp-add lr0 lr0-public 00:00:20:20:12:13 172.168.0.100/24 +ovn-nbctl lsp-add public public-lr0 +ovn-nbctl lsp-set-type public-lr0 router +ovn-nbctl lsp-set-addresses public-lr0 router +ovn-nbctl lsp-set-options public-lr0 router-port=lr0-public + +# localnet port +ovn-nbctl lsp-add public ln-public +ovn-nbctl lsp-set-type ln-public localnet +ovn-nbctl lsp-set-addresses ln-public unknown +ovn-nbctl lsp-set-options ln-public network_name=public + +# schedule the gw router port to a chassis. Change the name of the chassis +ovn-nbctl --wait=hv lrp-set-gateway-chassis lr0-public hv1 20 + +# Modify a logical router port and it should result in recompute. +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl set logical_router_port lr0-sw0 options:foo=bar +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute + +check as northd ovn-appctl -t NORTHD_TYPE vlog/set dbg + +# Do checks for NATs. +# Add a NAT. This should not result in recompute of both northd and lflow +# engine nodes. +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb lr-nat-add lr0 dnat_and_snat 172.168.0.110 10.0.0.4 +check_engine_stats northd norecompute compute +check_engine_stats lflow norecompute compute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +# Update the NAT options column +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb set NAT . options:foo=bar +check_engine_stats northd norecompute compute +check_engine_stats lflow norecompute compute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +# Update the NAT external_ip column +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb set NAT . external_ip=172.168.0.120 +check_engine_stats northd norecompute compute +check_engine_stats lflow norecompute compute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +# Update the NAT logical_ip column +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb set NAT . logical_ip=10.0.0.10 +check_engine_stats northd norecompute compute +check_engine_stats lflow norecompute compute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +# Update the NAT type +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb set NAT . type=snat +check_engine_stats northd norecompute compute +check_engine_stats lflow norecompute compute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +# Create a dnat_and_snat NAT with external_mac and logical_port +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb lr-nat-add lr0 dnat_and_snat 172.168.0.110 10.0.0.4 sw0p1 30:54:00:00:00:03 +check_engine_stats northd norecompute compute +check_engine_stats lflow norecompute compute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +nat2_uuid=$(ovn-nbctl --bare --columns _uuid find nat logical_ip=10.0.0.4) + +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb set NAT $nat2_uuid external_mac='"30:54:00:00:00:04"' +check_engine_stats northd norecompute compute +check_engine_stats lflow norecompute compute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +# Create a load balancer and add the lb vip as NAT +check ovn-nbctl lb-add lb1 172.168.0.140 10.0.0.20 +check ovn-nbctl lb-add lb2 172.168.0.150:80 10.0.0.40:8080 +check ovn-nbctl lr-lb-add lr0 lb1 +check ovn-nbctl lr-lb-add lr0 lb2 + +# lflow engine should recompute since the nat ip 172.168.0.140 +# is a lb vip. +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb lr-nat-add lr0 dnat_and_snat 172.168.0.140 10.0.0.20 +check_engine_stats northd norecompute compute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb lr-nat-add lr0 dnat_and_snat 172.168.0.150 10.0.0.41 +check_engine_stats northd norecompute compute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb lr-nat-del lr0 dnat_and_snat 172.168.0.150 +check_engine_stats northd norecompute compute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb lr-nat-del lr0 dnat_and_snat 172.168.0.140 +check_engine_stats northd norecompute compute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +# Delete the NAT +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb clear logical_router lr0 nat +check_engine_stats northd norecompute compute +check_engine_stats lflow norecompute compute +check_engine_stats sync_to_sb_pb recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +# Create router Policy +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb lr-policy-add lr0 10 "ip4.src == 10.0.0.3" reroute 172.168.0.101,172.168.0.102 +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute + +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb lr-policy-del lr0 10 "ip4.src == 10.0.0.3" +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute + +OVN_CLEANUP([hv1]) +AT_CLEANUP +])