From patchwork Fri Jul 7 05:53:26 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Numan Siddique X-Patchwork-Id: 1804604 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::133; helo=smtp2.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp2.osuosl.org (smtp2.osuosl.org [IPv6:2605:bc80:3010::133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4Qy2dS6Jy7z20b8 for ; Fri, 7 Jul 2023 15:53:48 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id EA5B240567; Fri, 7 Jul 2023 05:53:46 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org EA5B240567 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id fz36F_Y67pzG; Fri, 7 Jul 2023 05:53:44 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp2.osuosl.org (Postfix) with ESMTPS id 791DD40407; Fri, 7 Jul 2023 05:53:43 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 791DD40407 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 5B606C0072; Fri, 7 Jul 2023 05:53:43 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by lists.linuxfoundation.org (Postfix) with ESMTP id 82FF8C0032 for ; Fri, 7 Jul 2023 05:53:41 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 5BEC940484 for ; Fri, 7 Jul 2023 05:53:41 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 5BEC940484 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id h4pCjRWWnUFc for ; Fri, 7 Jul 2023 05:53:38 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 1904740407 Received: from relay5-d.mail.gandi.net (relay5-d.mail.gandi.net [IPv6:2001:4b98:dc4:8::225]) by smtp2.osuosl.org (Postfix) with ESMTPS id 1904740407 for ; Fri, 7 Jul 2023 05:53:37 +0000 (UTC) X-GND-Sasl: numans@ovn.org X-GND-Sasl: numans@ovn.org Received: by mail.gandi.net (Postfix) with ESMTPSA id B1F751C0002; Fri, 7 Jul 2023 05:53:34 +0000 (UTC) From: numans@ovn.org To: dev@openvswitch.org Date: Fri, 7 Jul 2023 11:23:26 +0530 Message-Id: <20230707055326.961596-1-numans@ovn.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230707055147.961462-1-numans@ovn.org> References: <20230707055147.961462-1-numans@ovn.org> MIME-Version: 1.0 Subject: [ovs-dev] [PATCH ovn v2 2/8] northd: Add a new engine node - northd_lb_data. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Numan Siddique This patch separates out the 'lbs' and 'lb_groups' from the 'northd' engine node data into a new engine node 'northd_lb_data'. This new node becomes an input to the 'northd' node. This makes handling the NB load balancer and load balancer group changes easier. Signed-off-by: Numan Siddique Acked-by: Han Zhou --- lib/lb.c | 201 +++++++++-- lib/lb.h | 86 +++-- northd/automake.mk | 2 + northd/en-lflow.c | 3 +- northd/en-northd-lb-data.c | 126 +++++++ northd/en-northd-lb-data.h | 19 ++ northd/en-northd.c | 11 +- northd/en-sync-sb.c | 2 +- northd/inc-proc-northd.c | 8 +- northd/northd.c | 673 +++++++++++++++++++++---------------- northd/northd.h | 15 +- 11 files changed, 780 insertions(+), 366 deletions(-) create mode 100644 northd/en-northd-lb-data.c create mode 100644 northd/en-northd-lb-data.h diff --git a/lib/lb.c b/lib/lb.c index 7afdaed65b..429dbf15af 100644 --- a/lib/lb.c +++ b/lib/lb.c @@ -26,6 +26,7 @@ #include "openvswitch/vlog.h" #include "lib/bitmap.h" #include "lib/smap.h" +#include "socket-util.h" VLOG_DEFINE_THIS_MODULE(lb); @@ -431,11 +432,62 @@ void ovn_northd_lb_vip_init(struct ovn_northd_lb_vip *lb_vip_nb, ovn_lb_get_health_check(nbrec_lb, vip_port_str, template); } +static void +ovn_lb_vip_backends_health_check_init(const struct ovn_northd_lb *lb, + const struct ovn_lb_vip *lb_vip, + struct ovn_northd_lb_vip *lb_vip_nb) +{ + struct ds key = DS_EMPTY_INITIALIZER; + + for (size_t j = 0; j < lb_vip->n_backends; j++) { + struct ovn_lb_backend *backend = &lb_vip->backends[j]; + ds_clear(&key); + ds_put_format(&key, IN6_IS_ADDR_V4MAPPED(&lb_vip->vip) + ? "%s" : "[%s]", backend->ip_str); + + const char *s = smap_get(&lb->nlb->ip_port_mappings, ds_cstr(&key)); + if (!s) { + continue; + } + + char *svc_mon_src_ip = NULL; + char *port_name = xstrdup(s); + char *p = strstr(port_name, ":"); + if (p) { + *p = 0; + p++; + struct sockaddr_storage svc_mon_src_addr; + if (!inet_parse_address(p, &svc_mon_src_addr)) { + static struct vlog_rate_limit rl = + VLOG_RATE_LIMIT_INIT(5, 1); + VLOG_WARN_RL(&rl, "Invalid svc mon src IP %s", p); + } else { + struct ds src_ip_s = DS_EMPTY_INITIALIZER; + ss_format_address_nobracks(&svc_mon_src_addr, + &src_ip_s); + svc_mon_src_ip = ds_steal_cstr(&src_ip_s); + } + } + + if (svc_mon_src_ip) { + struct ovn_northd_lb_backend *backend_nb = + &lb_vip_nb->backends_nb[j]; + backend_nb->health_check = true; + backend_nb->logical_port = xstrdup(port_name); + backend_nb->svc_mon_src_ip = svc_mon_src_ip; + } + free(port_name); + } + + ds_destroy(&key); +} + static void ovn_northd_lb_vip_destroy(struct ovn_northd_lb_vip *vip) { free(vip->backend_ips); for (size_t i = 0; i < vip->n_backends; i++) { + free(vip->backends_nb[i].logical_port); free(vip->backends_nb[i].svc_mon_src_ip); } free(vip->backends_nb); @@ -555,8 +607,7 @@ ovn_lb_get_health_check(const struct nbrec_load_balancer *nbrec_lb, } struct ovn_northd_lb * -ovn_northd_lb_create(const struct nbrec_load_balancer *nbrec_lb, - size_t n_ls_datapaths, size_t n_lr_datapaths) +ovn_northd_lb_create(const struct nbrec_load_balancer *nbrec_lb) { bool template = smap_get_bool(&nbrec_lb->options, "template", false); bool is_udp = nullable_string_is_equal(nbrec_lb->protocol, "udp"); @@ -595,9 +646,6 @@ ovn_northd_lb_create(const struct nbrec_load_balancer *nbrec_lb, } lb->affinity_timeout = affinity_timeout; - lb->nb_ls_map = bitmap_allocate(n_ls_datapaths); - lb->nb_lr_map = bitmap_allocate(n_lr_datapaths); - sset_init(&lb->ips_v4); sset_init(&lb->ips_v6); struct smap_node *node; @@ -631,7 +679,12 @@ ovn_northd_lb_create(const struct nbrec_load_balancer *nbrec_lb, ovn_lb_vip6_template_format_internal(lb_vip), xstrdup(node->value)); } + n_vips++; + + if (lb_vip_nb->lb_health_check) { + ovn_lb_vip_backends_health_check_init(lb, lb_vip, lb_vip_nb); + } } /* It's possible that parsing VIPs fails. Update the lb->n_vips to the @@ -639,6 +692,7 @@ ovn_northd_lb_create(const struct nbrec_load_balancer *nbrec_lb, */ lb->n_vips = n_vips; + if (nbrec_lb->n_selection_fields) { char *proto = NULL; if (nbrec_lb->protocol && nbrec_lb->protocol[0]) { @@ -684,24 +738,6 @@ ovn_northd_lb_get_vips(const struct ovn_northd_lb *lb) return &lb->nlb->vips; } -void -ovn_northd_lb_add_lr(struct ovn_northd_lb *lb, size_t n, - struct ovn_datapath **ods) -{ - for (size_t i = 0; i < n; i++) { - bitmap_set1(lb->nb_lr_map, ods[i]->index); - } -} - -void -ovn_northd_lb_add_ls(struct ovn_northd_lb *lb, size_t n, - struct ovn_datapath **ods) -{ - for (size_t i = 0; i < n; i++) { - bitmap_set1(lb->nb_ls_map, ods[i]->index); - } -} - void ovn_northd_lb_destroy(struct ovn_northd_lb *lb) { @@ -715,8 +751,6 @@ ovn_northd_lb_destroy(struct ovn_northd_lb *lb) sset_destroy(&lb->ips_v4); sset_destroy(&lb->ips_v6); free(lb->selection_fields); - bitmap_free(lb->nb_lr_map); - bitmap_free(lb->nb_ls_map); free(lb); } @@ -727,8 +761,7 @@ ovn_northd_lb_destroy(struct ovn_northd_lb *lb) * with ovn_lb_group_add_ls() and ovn_lb_group_add_lr() respectively. */ struct ovn_lb_group * ovn_lb_group_create(const struct nbrec_load_balancer_group *nbrec_lb_group, - const struct hmap *lbs, size_t max_ls_datapaths, - size_t max_lr_datapaths) + const struct hmap *lbs) { struct ovn_lb_group *lb_group; @@ -736,8 +769,6 @@ ovn_lb_group_create(const struct nbrec_load_balancer_group *nbrec_lb_group, lb_group->uuid = nbrec_lb_group->header_.uuid; lb_group->n_lbs = nbrec_lb_group->n_load_balancer; lb_group->lbs = xmalloc(lb_group->n_lbs * sizeof *lb_group->lbs); - lb_group->ls = xmalloc(max_ls_datapaths * sizeof *lb_group->ls); - lb_group->lr = xmalloc(max_lr_datapaths * sizeof *lb_group->lr); lb_group->lb_ips = ovn_lb_ip_set_create(); for (size_t i = 0; i < nbrec_lb_group->n_load_balancer; i++) { @@ -758,8 +789,6 @@ ovn_lb_group_destroy(struct ovn_lb_group *lb_group) ovn_lb_ip_set_destroy(lb_group->lb_ips); free(lb_group->lbs); - free(lb_group->ls); - free(lb_group->lr); free(lb_group); } @@ -943,3 +972,113 @@ ovn_lb_5tuples_destroy(struct hmap *tuples) hmap_destroy(tuples); } + +void +build_lrouter_lb_ips(struct ovn_lb_ip_set *lb_ips, + const struct ovn_northd_lb *lb) +{ + const char *ip_address; + + SSET_FOR_EACH (ip_address, &lb->ips_v4) { + sset_add(&lb_ips->ips_v4, ip_address); + if (lb->routable) { + sset_add(&lb_ips->ips_v4_routable, ip_address); + } + } + SSET_FOR_EACH (ip_address, &lb->ips_v6) { + sset_add(&lb_ips->ips_v6, ip_address); + if (lb->routable) { + sset_add(&lb_ips->ips_v6_routable, ip_address); + } + } +} + +/* lb datapaths functions */ +struct ovn_lb_datapaths * +ovn_lb_datapaths_create(const struct ovn_northd_lb *lb, size_t n_ls_datapaths, + size_t n_lr_datapaths) +{ + struct ovn_lb_datapaths *lb_dps = xzalloc(sizeof *lb_dps); + lb_dps->lb = lb; + lb_dps->nb_ls_map = bitmap_allocate(n_ls_datapaths); + lb_dps->nb_lr_map = bitmap_allocate(n_lr_datapaths); + + return lb_dps; +} + +struct ovn_lb_datapaths * +ovn_lb_datapaths_find(const struct hmap *lb_dps_map, + const struct uuid *lb_uuid) +{ + struct ovn_lb_datapaths *lb_dps; + size_t hash = uuid_hash(lb_uuid); + HMAP_FOR_EACH_WITH_HASH (lb_dps, hmap_node, hash, lb_dps_map) { + if (uuid_equals(&lb_dps->lb->nlb->header_.uuid, lb_uuid)) { + return lb_dps; + } + } + return NULL; +} + +void +ovn_lb_datapaths_destroy(struct ovn_lb_datapaths *lb_dps) +{ + bitmap_free(lb_dps->nb_lr_map); + bitmap_free(lb_dps->nb_ls_map); + free(lb_dps); +} + +void +ovn_lb_datapaths_add_lr(struct ovn_lb_datapaths *lb_dps, size_t n, + struct ovn_datapath **ods) +{ + for (size_t i = 0; i < n; i++) { + bitmap_set1(lb_dps->nb_lr_map, ods[i]->index); + } +} + +void +ovn_lb_datapaths_add_ls(struct ovn_lb_datapaths *lb_dps, size_t n, + struct ovn_datapath **ods) +{ + for (size_t i = 0; i < n; i++) { + bitmap_set1(lb_dps->nb_ls_map, ods[i]->index); + } +} + +struct ovn_lb_group_datapaths * +ovn_lb_group_datapaths_create(const struct ovn_lb_group *lb_group, + size_t max_ls_datapaths, + size_t max_lr_datapaths) +{ + struct ovn_lb_group_datapaths *lb_group_dps = + xzalloc(sizeof *lb_group_dps); + lb_group_dps->lb_group = lb_group; + lb_group_dps->ls = xmalloc(max_ls_datapaths * sizeof *lb_group_dps->ls); + lb_group_dps->lr = xmalloc(max_lr_datapaths * sizeof *lb_group_dps->lr); + + return lb_group_dps; +} + +void +ovn_lb_group_datapaths_destroy(struct ovn_lb_group_datapaths *lb_group_dps) +{ + free(lb_group_dps->ls); + free(lb_group_dps->lr); + free(lb_group_dps); +} + +struct ovn_lb_group_datapaths * +ovn_lb_group_datapaths_find(const struct hmap *lb_group_dps_map, + const struct uuid *lb_group_uuid) +{ + struct ovn_lb_group_datapaths *lb_group_dps; + size_t hash = uuid_hash(lb_group_uuid); + + HMAP_FOR_EACH_WITH_HASH (lb_group_dps, hmap_node, hash, lb_group_dps_map) { + if (uuid_equals(&lb_group_dps->lb_group->uuid, lb_group_uuid)) { + return lb_group_dps; + } + } + return NULL; +} diff --git a/lib/lb.h b/lib/lb.h index 23d8fc9e9b..0339050cba 100644 --- a/lib/lb.h +++ b/lib/lb.h @@ -59,7 +59,6 @@ struct ovn_northd_lb { struct hmap_node hmap_node; const struct nbrec_load_balancer *nlb; /* May be NULL. */ - const struct sbrec_load_balancer *slb; /* May be NULL. */ const char *proto; char *selection_fields; struct ovn_lb_vip *vips; @@ -78,14 +77,6 @@ struct ovn_northd_lb { struct sset ips_v4; struct sset ips_v6; - - size_t n_nb_ls; - unsigned long *nb_ls_map; - - size_t n_nb_lr; - unsigned long *nb_lr_map; - - struct ovn_dp_group *dpg; }; struct ovn_lb_vip { @@ -129,23 +120,19 @@ struct ovn_northd_lb_vip { }; struct ovn_northd_lb_backend { - struct ovn_port *op; /* Logical port to which the ip belong to. */ bool health_check; + char *logical_port; /* Logical port to which the ip belong to. */ char *svc_mon_src_ip; /* Source IP to use for monitoring. */ - const struct sbrec_service_monitor *sbrec_monitor; }; -struct ovn_northd_lb *ovn_northd_lb_create(const struct nbrec_load_balancer *, - size_t n_ls_datapaths, - size_t n_lr_datapaths); +struct ovn_northd_lb *ovn_northd_lb_create(const struct nbrec_load_balancer *); struct ovn_northd_lb *ovn_northd_lb_find(const struct hmap *, const struct uuid *); const struct smap *ovn_northd_lb_get_vips(const struct ovn_northd_lb *); void ovn_northd_lb_destroy(struct ovn_northd_lb *); -void ovn_northd_lb_add_lr(struct ovn_northd_lb *lb, size_t n, - struct ovn_datapath **ods); -void ovn_northd_lb_add_ls(struct ovn_northd_lb *lb, size_t n, - struct ovn_datapath **ods); + +void build_lrouter_lb_ips(struct ovn_lb_ip_set *, + const struct ovn_northd_lb *); struct ovn_lb_group { struct hmap_node hmap_node; @@ -153,35 +140,70 @@ struct ovn_lb_group { size_t n_lbs; struct ovn_northd_lb **lbs; struct ovn_lb_ip_set *lb_ips; +}; + +struct ovn_lb_group *ovn_lb_group_create( + const struct nbrec_load_balancer_group *, + const struct hmap *lbs); +void ovn_lb_group_destroy(struct ovn_lb_group *lb_group); +struct ovn_lb_group *ovn_lb_group_find(const struct hmap *lb_groups, + const struct uuid *); + +struct ovn_lb_datapaths { + struct hmap_node hmap_node; - /* Datapaths to which this LB group is applied. */ + const struct ovn_northd_lb *lb; + size_t n_nb_ls; + unsigned long *nb_ls_map; + + size_t n_nb_lr; + unsigned long *nb_lr_map; +}; + +struct ovn_lb_datapaths *ovn_lb_datapaths_create(const struct ovn_northd_lb *, + size_t n_ls_datapaths, + size_t n_lr_datapaths); +struct ovn_lb_datapaths *ovn_lb_datapaths_find(const struct hmap *, + const struct uuid *); +void ovn_lb_datapaths_destroy(struct ovn_lb_datapaths *); +void ovn_lb_datapaths_add_lr(struct ovn_lb_datapaths *, size_t n, + struct ovn_datapath **); +void ovn_lb_datapaths_add_ls(struct ovn_lb_datapaths *, size_t n, + struct ovn_datapath **); + +struct ovn_lb_group_datapaths { + struct hmap_node hmap_node; + + const struct ovn_lb_group *lb_group; + + /* Datapaths to which 'lb_group' is applied. */ size_t n_ls; struct ovn_datapath **ls; size_t n_lr; struct ovn_datapath **lr; }; -struct ovn_lb_group *ovn_lb_group_create( - const struct nbrec_load_balancer_group *, - const struct hmap *lbs, - size_t max_ls_datapaths, +struct ovn_lb_group_datapaths *ovn_lb_group_datapaths_create( + const struct ovn_lb_group *, size_t max_ls_datapaths, size_t max_lr_datapaths); -void ovn_lb_group_destroy(struct ovn_lb_group *lb_group); -struct ovn_lb_group *ovn_lb_group_find(const struct hmap *lb_groups, - const struct uuid *); + +void ovn_lb_group_datapaths_destroy(struct ovn_lb_group_datapaths *); +struct ovn_lb_group_datapaths *ovn_lb_group_datapaths_find( + const struct hmap *lb_group_dps, const struct uuid *); static inline void -ovn_lb_group_add_ls(struct ovn_lb_group *lb_group, size_t n, - struct ovn_datapath **ods) +ovn_lb_group_datapaths_add_ls(struct ovn_lb_group_datapaths *lbg_dps, size_t n, + struct ovn_datapath **ods) { - memcpy(&lb_group->ls[lb_group->n_ls], ods, n * sizeof *ods); - lb_group->n_ls += n; + memcpy(&lbg_dps->ls[lbg_dps->n_ls], ods, n * sizeof *ods); + lbg_dps->n_ls += n; } static inline void -ovn_lb_group_add_lr(struct ovn_lb_group *lb_group, struct ovn_datapath *lr) +ovn_lb_group_datapaths_add_lr(struct ovn_lb_group_datapaths *lbg_dps, + struct ovn_datapath *lr) { - lb_group->lr[lb_group->n_lr++] = lr; + lbg_dps->lr[lbg_dps->n_lr++] = lr; } struct ovn_controller_lb { diff --git a/northd/automake.mk b/northd/automake.mk index b17f1fdb54..6f60265b73 100644 --- a/northd/automake.mk +++ b/northd/automake.mk @@ -18,6 +18,8 @@ northd_ovn_northd_SOURCES = \ northd/en-sync-sb.h \ northd/en-sync-from-sb.c \ northd/en-sync-from-sb.h \ + northd/en-northd-lb-data.c \ + northd/en-northd-lb-data.h \ northd/inc-proc-northd.c \ northd/inc-proc-northd.h \ northd/ipam.c \ diff --git a/northd/en-lflow.c b/northd/en-lflow.c index 28ab1c67fb..db1bcbccd6 100644 --- a/northd/en-lflow.c +++ b/northd/en-lflow.c @@ -57,7 +57,8 @@ lflow_get_input_data(struct engine_node *node, lflow_input->lr_ports = &northd_data->lr_ports; lflow_input->port_groups = &northd_data->port_groups; lflow_input->meter_groups = &northd_data->meter_groups; - lflow_input->lbs = &northd_data->lbs; + lflow_input->lb_datapaths_map = &northd_data->lb_datapaths_map; + lflow_input->svc_monitor_map = &northd_data->svc_monitor_map; lflow_input->features = &northd_data->features; lflow_input->ovn_internal_version_changed = northd_data->ovn_internal_version_changed; diff --git a/northd/en-northd-lb-data.c b/northd/en-northd-lb-data.c new file mode 100644 index 0000000000..d46c3c27ed --- /dev/null +++ b/northd/en-northd-lb-data.c @@ -0,0 +1,126 @@ +/* + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at: + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#include + +#include +#include +#include + +#include "openvswitch/util.h" + +#include "en-northd-lb-data.h" +#include "lib/inc-proc-eng.h" +#include "lib/lb.h" +#include "lib/ovn-nb-idl.h" +#include "lib/ovn-sb-idl.h" +#include "lib/ovn-util.h" +#include "northd.h" + +#include "openvswitch/vlog.h" + +VLOG_DEFINE_THIS_MODULE(en_northd_lb_data); + +static void northd_lb_data_init(struct northd_lb_data *); +static void northd_lb_data_destroy(struct northd_lb_data *); +static void build_lbs(const struct nbrec_load_balancer_table *, + const struct nbrec_load_balancer_group_table *, + struct hmap *lbs, struct hmap *lb_groups); + +void * +en_northd_lb_data_init(struct engine_node *node OVS_UNUSED, + struct engine_arg *arg OVS_UNUSED) +{ + struct northd_lb_data *data = xzalloc(sizeof *data); + + northd_lb_data_init(data); + + return data; +} + +void +en_northd_lb_data_run(struct engine_node *node, void *data) +{ + struct northd_lb_data *lb_data = (struct northd_lb_data *) data; + northd_lb_data_destroy(lb_data); + northd_lb_data_init(lb_data); + + const struct nbrec_load_balancer_table *nb_lb_table = + EN_OVSDB_GET(engine_get_input("NB_load_balancer", node)); + const struct nbrec_load_balancer_group_table *nb_lbg_table = + EN_OVSDB_GET(engine_get_input("NB_load_balancer_group", node)); + + build_lbs(nb_lb_table, nb_lbg_table, &lb_data->lbs, &lb_data->lb_groups); + engine_set_node_state(node, EN_UPDATED); +} + +void +en_northd_lb_data_cleanup(void *data) +{ + struct northd_lb_data *lb_data = (struct northd_lb_data *) data; + northd_lb_data_destroy(lb_data); +} + +/* static functions. */ +static void +northd_lb_data_init(struct northd_lb_data *lb_data) +{ + hmap_init(&lb_data->lbs); + hmap_init(&lb_data->lb_groups); +} + +static void +northd_lb_data_destroy(struct northd_lb_data *lb_data) +{ + struct ovn_northd_lb *lb; + HMAP_FOR_EACH_POP (lb, hmap_node, &lb_data->lbs) { + ovn_northd_lb_destroy(lb); + } + hmap_destroy(&lb_data->lbs); + + struct ovn_lb_group *lb_group; + HMAP_FOR_EACH_POP (lb_group, hmap_node, &lb_data->lb_groups) { + ovn_lb_group_destroy(lb_group); + } + hmap_destroy(&lb_data->lb_groups); +} + +static void +build_lbs(const struct nbrec_load_balancer_table *nbrec_load_balancer_table, + const struct nbrec_load_balancer_group_table *nbrec_lb_group_table, + struct hmap *lbs, struct hmap *lb_groups) +{ + struct ovn_lb_group *lb_group; + struct ovn_northd_lb *lb_nb; + + const struct nbrec_load_balancer *nbrec_lb; + NBREC_LOAD_BALANCER_TABLE_FOR_EACH (nbrec_lb, nbrec_load_balancer_table) { + lb_nb = ovn_northd_lb_create(nbrec_lb); + hmap_insert(lbs, &lb_nb->hmap_node, + uuid_hash(&nbrec_lb->header_.uuid)); + } + + const struct nbrec_load_balancer_group *nbrec_lb_group; + NBREC_LOAD_BALANCER_GROUP_TABLE_FOR_EACH (nbrec_lb_group, + nbrec_lb_group_table) { + lb_group = ovn_lb_group_create(nbrec_lb_group, lbs); + + for (size_t i = 0; i < lb_group->n_lbs; i++) { + build_lrouter_lb_ips(lb_group->lb_ips, lb_group->lbs[i]); + } + + hmap_insert(lb_groups, &lb_group->hmap_node, + uuid_hash(&lb_group->uuid)); + } +} diff --git a/northd/en-northd-lb-data.h b/northd/en-northd-lb-data.h new file mode 100644 index 0000000000..eb297e376d --- /dev/null +++ b/northd/en-northd-lb-data.h @@ -0,0 +1,19 @@ +#ifndef EN_NORTHD_LB_DATA_H +#define EN_NORTHD_LB_DATA_H 1 + +#include + +#include "openvswitch/hmap.h" + +#include "lib/inc-proc-eng.h" + +struct northd_lb_data { + struct hmap lbs; + struct hmap lb_groups; +}; + +void *en_northd_lb_data_init(struct engine_node *, struct engine_arg *); +void en_northd_lb_data_run(struct engine_node *, void *data); +void en_northd_lb_data_cleanup(void *data); + +#endif /* end of EN_NORTHD_LB_DATA_H */ diff --git a/northd/en-northd.c b/northd/en-northd.c index f9f2d04452..cc7d838451 100644 --- a/northd/en-northd.c +++ b/northd/en-northd.c @@ -20,6 +20,7 @@ #include "coverage.h" #include "en-northd.h" +#include "en-northd-lb-data.h" #include "lib/inc-proc-eng.h" #include "lib/ovn-nb-idl.h" #include "openvswitch/list.h" /* TODO This is needed for ovn-parallel-hmap.h. @@ -70,10 +71,6 @@ northd_get_input_data(struct engine_node *node, EN_OVSDB_GET(engine_get_input("NB_logical_switch", node)); input_data->nbrec_logical_router_table = EN_OVSDB_GET(engine_get_input("NB_logical_router", node)); - input_data->nbrec_load_balancer_table = - EN_OVSDB_GET(engine_get_input("NB_load_balancer", node)); - input_data->nbrec_load_balancer_group_table = - EN_OVSDB_GET(engine_get_input("NB_load_balancer_group", node)); input_data->nbrec_port_group_table = EN_OVSDB_GET(engine_get_input("NB_port_group", node)); input_data->nbrec_meter_table = @@ -117,6 +114,11 @@ northd_get_input_data(struct engine_node *node, EN_OVSDB_GET(engine_get_input("SB_chassis_template_var", node)); input_data->sbrec_mirror_table = EN_OVSDB_GET(engine_get_input("SB_mirror", node)); + + struct northd_lb_data *lb_data = + engine_get_input_data("northd_lb_data", node); + input_data->lbs = &lb_data->lbs; + input_data->lb_groups = &lb_data->lb_groups; } void @@ -130,6 +132,7 @@ en_northd_run(struct engine_node *node, void *data) northd_init(data); northd_get_input_data(node, &input_data); + COVERAGE_INC(northd_run); stopwatch_start(OVNNB_DB_RUN_STOPWATCH_NAME, time_msec()); ovnnb_db_run(&input_data, data, eng_ctx->ovnnb_idl_txn, diff --git a/northd/en-sync-sb.c b/northd/en-sync-sb.c index 821047581c..fda0ca5a68 100644 --- a/northd/en-sync-sb.c +++ b/northd/en-sync-sb.c @@ -230,7 +230,7 @@ en_sync_to_sb_lb_run(struct engine_node *node, void *data OVS_UNUSED) struct northd_data *northd_data = engine_get_input_data("northd", node); sync_lbs(eng_ctx->ovnsb_idl_txn, sb_load_balancer_table, - &northd_data->ls_datapaths, &northd_data->lbs); + &northd_data->ls_datapaths, &northd_data->lb_datapaths_map); engine_set_node_state(node, EN_UPDATED); } diff --git a/northd/inc-proc-northd.c b/northd/inc-proc-northd.c index 507348b719..b2e884962f 100644 --- a/northd/inc-proc-northd.c +++ b/northd/inc-proc-northd.c @@ -35,6 +35,7 @@ #include "en-northd-output.h" #include "en-sync-sb.h" #include "en-sync-from-sb.h" +#include "en-northd-lb-data.h" #include "unixctl.h" #include "util.h" @@ -140,6 +141,7 @@ static ENGINE_NODE(sync_to_sb_addr_set, "sync_to_sb_addr_set"); static ENGINE_NODE(fdb_aging, "fdb_aging"); static ENGINE_NODE(fdb_aging_waker, "fdb_aging_waker"); static ENGINE_NODE(sync_to_sb_lb, "sync_to_sb_lb"); +static ENGINE_NODE(northd_lb_data, "northd_lb_data"); void inc_proc_northd_init(struct ovsdb_idl_loop *nb, struct ovsdb_idl_loop *sb) @@ -147,8 +149,6 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, /* Define relationships between nodes where first argument is dependent * on the second argument */ engine_add_input(&en_northd, &en_nb_port_group, NULL); - engine_add_input(&en_northd, &en_nb_load_balancer, NULL); - engine_add_input(&en_northd, &en_nb_load_balancer_group, NULL); engine_add_input(&en_northd, &en_nb_acl, NULL); engine_add_input(&en_northd, &en_nb_logical_router, NULL); engine_add_input(&en_northd, &en_nb_mirror, NULL); @@ -178,6 +178,10 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, engine_add_input(&en_northd, &en_nb_logical_switch, northd_nb_logical_switch_handler); + engine_add_input(&en_northd_lb_data, &en_nb_load_balancer, NULL); + engine_add_input(&en_northd_lb_data, &en_nb_load_balancer_group, NULL); + engine_add_input(&en_northd, &en_northd_lb_data, NULL); + engine_add_input(&en_mac_binding_aging, &en_nb_nb_global, NULL); engine_add_input(&en_mac_binding_aging, &en_sb_mac_binding, NULL); engine_add_input(&en_mac_binding_aging, &en_northd, NULL); diff --git a/northd/northd.c b/northd/northd.c index 2390c159c3..890186b29c 100644 --- a/northd/northd.c +++ b/northd/northd.c @@ -3862,14 +3862,11 @@ struct service_monitor_info { static struct service_monitor_info * -create_or_get_service_mon(struct ovsdb_idl_txn *ovnsb_txn, - struct hmap *monitor_map, - const char *ip, const char *logical_port, - uint16_t service_port, const char *protocol) +get_service_mon(const struct hmap *monitor_map, + const char *ip, const char *logical_port, + uint16_t service_port, const char *protocol, + uint32_t hash) { - uint32_t hash = service_port; - hash = hash_string(ip, hash); - hash = hash_string(logical_port, hash); struct service_monitor_info *mon_info; HMAP_FOR_EACH_WITH_HASH (mon_info, hmap_node, hash, monitor_map) { @@ -3881,6 +3878,26 @@ create_or_get_service_mon(struct ovsdb_idl_txn *ovnsb_txn, } } + return NULL; +} + +static struct service_monitor_info * +create_or_get_service_mon(struct ovsdb_idl_txn *ovnsb_txn, + struct hmap *monitor_map, + const char *ip, const char *logical_port, + uint16_t service_port, const char *protocol) +{ + uint32_t hash = service_port; + hash = hash_string(ip, hash); + hash = hash_string(logical_port, hash); + struct service_monitor_info *mon_info = + get_service_mon(monitor_map, ip, logical_port, service_port, + protocol, hash); + + if (mon_info) { + return mon_info; + } + struct sbrec_service_monitor *sbrec_mon = sbrec_service_monitor_insert(ovnsb_txn); sbrec_service_monitor_set_ip(sbrec_mon, ip); @@ -3894,7 +3911,8 @@ create_or_get_service_mon(struct ovsdb_idl_txn *ovnsb_txn, } static void -ovn_lb_svc_create(struct ovsdb_idl_txn *ovnsb_txn, struct ovn_northd_lb *lb, +ovn_lb_svc_create(struct ovsdb_idl_txn *ovnsb_txn, + const struct ovn_northd_lb *lb, struct hmap *monitor_map, struct hmap *ls_ports, struct sset *svc_monitor_lsps) { @@ -3911,58 +3929,27 @@ ovn_lb_svc_create(struct ovsdb_idl_txn *ovnsb_txn, struct ovn_northd_lb *lb, struct ovn_northd_lb_backend *backend_nb = &lb_vip_nb->backends_nb[j]; - struct ovn_port *op = NULL; - char *svc_mon_src_ip = NULL; - - struct ds key = DS_EMPTY_INITIALIZER; - ds_put_format(&key, - IN6_IS_ADDR_V4MAPPED(&lb_vip->vip) - ? "%s" : "[%s]", backend->ip_str); - - const char *s = smap_get(&lb->nlb->ip_port_mappings, - ds_cstr(&key)); - if (s) { - char *port_name = xstrdup(s); - char *p = strstr(port_name, ":"); - if (p) { - *p = 0; - p++; - sset_add(svc_monitor_lsps, port_name); - op = ovn_port_find(ls_ports, port_name); - struct sockaddr_storage svc_mon_src_addr; - if (!inet_parse_address(p, &svc_mon_src_addr)) { - static struct vlog_rate_limit rl = - VLOG_RATE_LIMIT_INIT(5, 1); - VLOG_WARN_RL(&rl, "Invalid svc mon src IP %s", p); - } else { - struct ds src_ip_s = DS_EMPTY_INITIALIZER; - ss_format_address_nobracks(&svc_mon_src_addr, - &src_ip_s); - svc_mon_src_ip = ds_steal_cstr(&src_ip_s); - } - } - free(port_name); + if (!backend_nb->health_check) { + continue; } - ds_destroy(&key); - if (!lb_vip_nb->lb_health_check || !op || !svc_mon_src_ip || - !lsp_is_enabled(op->nbsp)) { - free(svc_mon_src_ip); + sset_add(svc_monitor_lsps, backend_nb->logical_port); + struct ovn_port *op = ovn_port_find(ls_ports, + backend_nb->logical_port); + + if (!op || !lsp_is_enabled(op->nbsp)) { continue; } - backend_nb->op = op; - backend_nb->svc_mon_src_ip = svc_mon_src_ip; - const char *protocol = lb->nlb->protocol; if (!protocol || !protocol[0]) { protocol = "tcp"; } - backend_nb->health_check = true; + struct service_monitor_info *mon_info = create_or_get_service_mon(ovnsb_txn, monitor_map, backend->ip_str, - backend_nb->op->nbsp->name, + backend_nb->logical_port, backend->port, protocol); ovs_assert(mon_info); @@ -3991,18 +3978,20 @@ ovn_lb_svc_create(struct ovsdb_idl_txn *ovnsb_txn, struct ovn_northd_lb *lb, "offline"); } - backend_nb->sbrec_monitor = mon_info->sbrec_mon; mon_info->required = true; } } } static bool -build_lb_vip_actions(struct ovn_lb_vip *lb_vip, - struct ovn_northd_lb_vip *lb_vip_nb, +build_lb_vip_actions(const struct ovn_northd_lb *lb, + const struct ovn_lb_vip *lb_vip, + const struct ovn_northd_lb_vip *lb_vip_nb, struct ds *action, char *selection_fields, - struct ds *skip_snat_action, struct ds *force_snat_action, - bool ls_dp, const struct chassis_features *features) + struct ds *skip_snat_action, + struct ds *force_snat_action, + bool ls_dp, const struct chassis_features *features, + const struct hmap *svc_monitor_map) { const char *ct_lb_action = features->ct_no_masked_label ? "ct_lb_mark" : "ct_lb"; @@ -4017,10 +4006,31 @@ build_lb_vip_actions(struct ovn_lb_vip *lb_vip, struct ovn_lb_backend *backend = &lb_vip->backends[i]; struct ovn_northd_lb_backend *backend_nb = &lb_vip_nb->backends_nb[i]; - if (!backend_nb->health_check || - (backend_nb->health_check && backend_nb->sbrec_monitor && - backend_nb->sbrec_monitor->status && - strcmp(backend_nb->sbrec_monitor->status, "online"))) { + + if (!backend_nb->health_check) { + continue; + } + + const char *protocol = lb->nlb->protocol; + if (!protocol || !protocol[0]) { + protocol = "tcp"; + } + + uint32_t hash = backend->port; + hash = hash_string(backend->ip_str, hash); + hash = hash_string(backend_nb->logical_port, hash); + + struct service_monitor_info *mon_info = get_service_mon( + svc_monitor_map, backend->ip_str, backend_nb->logical_port, + backend->port, protocol, hash); + + if (!mon_info) { + continue; + } + + ovs_assert(mon_info->sbrec_mon); + if (mon_info->sbrec_mon->status && + strcmp(mon_info->sbrec_mon->status, "online")) { continue; } @@ -4070,59 +4080,32 @@ build_lb_vip_actions(struct ovn_lb_vip *lb_vip, } static void -build_lrouter_lb_ips(struct ovn_lb_ip_set *lb_ips, - const struct ovn_northd_lb *lb) -{ - const char *ip_address; - - SSET_FOR_EACH (ip_address, &lb->ips_v4) { - sset_add(&lb_ips->ips_v4, ip_address); - if (lb->routable) { - sset_add(&lb_ips->ips_v4_routable, ip_address); - } - } - SSET_FOR_EACH (ip_address, &lb->ips_v6) { - sset_add(&lb_ips->ips_v6, ip_address); - if (lb->routable) { - sset_add(&lb_ips->ips_v6_routable, ip_address); - } - } -} - -static void -build_lbs(const struct nbrec_load_balancer_table *nbrec_load_balancer_table, - const struct nbrec_load_balancer_group_table *nbrec_lb_group_table, - struct ovn_datapaths *ls_datapaths, - struct ovn_datapaths *lr_datapaths, - struct hmap *lbs, struct hmap *lb_groups) +build_lb_datapaths(const struct hmap *lbs, const struct hmap *lb_groups, + struct ovn_datapaths *ls_datapaths, + struct ovn_datapaths *lr_datapaths, + struct hmap *lb_datapaths_map, + struct hmap *lb_group_datapaths_map) { const struct nbrec_load_balancer_group *nbrec_lb_group; - struct ovn_lb_group *lb_group; - struct ovn_northd_lb *lb; + struct ovn_lb_group_datapaths *lb_group_dps; + const struct ovn_lb_group *lb_group; + struct ovn_lb_datapaths *lb_dps; + const struct ovn_northd_lb *lb; - hmap_init(lbs); - hmap_init(lb_groups); + hmap_init(lb_datapaths_map); + hmap_init(lb_group_datapaths_map); - const struct nbrec_load_balancer *nbrec_lb; - NBREC_LOAD_BALANCER_TABLE_FOR_EACH (nbrec_lb, nbrec_load_balancer_table) { - struct ovn_northd_lb *lb_nb = ovn_northd_lb_create(nbrec_lb, - ods_size(ls_datapaths), - ods_size(lr_datapaths)); - hmap_insert(lbs, &lb_nb->hmap_node, - uuid_hash(&nbrec_lb->header_.uuid)); + HMAP_FOR_EACH (lb, hmap_node, lbs) { + lb_dps = ovn_lb_datapaths_create(lb, ods_size(ls_datapaths), + ods_size(lr_datapaths)); + hmap_insert(lb_datapaths_map, &lb_dps->hmap_node, + uuid_hash(&lb->nlb->header_.uuid)); } - NBREC_LOAD_BALANCER_GROUP_TABLE_FOR_EACH (nbrec_lb_group, - nbrec_lb_group_table) { - lb_group = ovn_lb_group_create(nbrec_lb_group, lbs, - ods_size(ls_datapaths), - ods_size(lr_datapaths)); - - for (size_t i = 0; i < lb_group->n_lbs; i++) { - build_lrouter_lb_ips(lb_group->lb_ips, lb_group->lbs[i]); - } - - hmap_insert(lb_groups, &lb_group->hmap_node, + HMAP_FOR_EACH (lb_group, hmap_node, lb_groups) { + lb_group_dps = ovn_lb_group_datapaths_create( + lb_group, ods_size(ls_datapaths), ods_size(lr_datapaths)); + hmap_insert(lb_group_datapaths_map, &lb_group_dps->hmap_node, uuid_hash(&lb_group->uuid)); } @@ -4135,22 +4118,19 @@ build_lbs(const struct nbrec_load_balancer_table *nbrec_load_balancer_table, for (size_t i = 0; i < od->nbs->n_load_balancer; i++) { const struct uuid *lb_uuid = &od->nbs->load_balancer[i]->header_.uuid; - lb = ovn_northd_lb_find(lbs, lb_uuid); - ovn_northd_lb_add_ls(lb, 1, &od); + lb_dps = ovn_lb_datapaths_find(lb_datapaths_map, lb_uuid); + ovs_assert(lb_dps); + ovn_lb_datapaths_add_ls(lb_dps, 1, &od); } for (size_t i = 0; i < od->nbs->n_load_balancer_group; i++) { nbrec_lb_group = od->nbs->load_balancer_group[i]; - lb_group = ovn_lb_group_find(lb_groups, - &nbrec_lb_group->header_.uuid); - ovn_lb_group_add_ls(lb_group, 1, &od); - } - } - - HMAP_FOR_EACH (lb_group, hmap_node, lb_groups) { - for (size_t j = 0; j < lb_group->n_lbs; j++) { - ovn_northd_lb_add_ls(lb_group->lbs[j], lb_group->n_ls, - lb_group->ls); + const struct uuid *lb_group_uuid = &nbrec_lb_group->header_.uuid; + lb_group_dps = + ovn_lb_group_datapaths_find(lb_group_datapaths_map, + lb_group_uuid); + ovs_assert(lb_group_dps); + ovn_lb_group_datapaths_add_ls(lb_group_dps, 1, &od); } } @@ -4172,15 +4152,21 @@ build_lbs(const struct nbrec_load_balancer_table *nbrec_load_balancer_table, size_t idx = (i + largest_group) % od->nbr->n_load_balancer_group; nbrec_lb_group = od->nbr->load_balancer_group[idx]; - lb_group = ovn_lb_group_find(lb_groups, - &nbrec_lb_group->header_.uuid); - ovn_lb_group_add_lr(lb_group, od); + const struct uuid *lb_group_uuid = &nbrec_lb_group->header_.uuid; + + lb_group_dps = + ovn_lb_group_datapaths_find(lb_group_datapaths_map, + lb_group_uuid); + ovs_assert(lb_group_dps); + ovn_lb_group_datapaths_add_lr(lb_group_dps, od); if (!od->lb_ips) { - od->lb_ips = ovn_lb_ip_set_clone(lb_group->lb_ips); + od->lb_ips = + ovn_lb_ip_set_clone(lb_group_dps->lb_group->lb_ips); } else { - for (size_t j = 0; j < lb_group->n_lbs; j++) { - build_lrouter_lb_ips(od->lb_ips, lb_group->lbs[j]); + for (size_t j = 0; j < lb_group_dps->lb_group->n_lbs; j++) { + build_lrouter_lb_ips(od->lb_ips, + lb_group_dps->lb_group->lbs[j]); } } } @@ -4192,16 +4178,23 @@ build_lbs(const struct nbrec_load_balancer_table *nbrec_load_balancer_table, for (size_t i = 0; i < od->nbr->n_load_balancer; i++) { const struct uuid *lb_uuid = &od->nbr->load_balancer[i]->header_.uuid; - lb = ovn_northd_lb_find(lbs, lb_uuid); - ovn_northd_lb_add_lr(lb, 1, &od); - build_lrouter_lb_ips(od->lb_ips, lb); + lb_dps = ovn_lb_datapaths_find(lb_datapaths_map, lb_uuid); + ovs_assert(lb_dps); + ovn_lb_datapaths_add_lr(lb_dps, 1, &od); + build_lrouter_lb_ips(od->lb_ips, lb_dps->lb); } } - HMAP_FOR_EACH (lb_group, hmap_node, lb_groups) { - for (size_t j = 0; j < lb_group->n_lbs; j++) { - ovn_northd_lb_add_lr(lb_group->lbs[j], lb_group->n_lr, - lb_group->lr); + HMAP_FOR_EACH (lb_group_dps, hmap_node, lb_group_datapaths_map) { + for (size_t j = 0; j < lb_group_dps->lb_group->n_lbs; j++) { + const struct uuid *lb_uuid = + &lb_group_dps->lb_group->lbs[j]->nlb->header_.uuid; + lb_dps = ovn_lb_datapaths_find(lb_datapaths_map, lb_uuid); + ovs_assert(lb_dps); + ovn_lb_datapaths_add_ls(lb_dps, lb_group_dps->n_ls, + lb_group_dps->ls); + ovn_lb_datapaths_add_lr(lb_dps, lb_group_dps->n_lr, + lb_group_dps->lr); } } } @@ -4210,10 +4203,10 @@ static void build_lb_svcs( struct ovsdb_idl_txn *ovnsb_txn, const struct sbrec_service_monitor_table *sbrec_service_monitor_table, - struct hmap *ls_ports, struct hmap *lbs, struct sset *svc_monitor_lsps) + struct hmap *ls_ports, struct hmap *lb_dps_map, + struct sset *svc_monitor_lsps, + struct hmap *svc_monitor_map) { - struct hmap monitor_map = HMAP_INITIALIZER(&monitor_map); - const struct sbrec_service_monitor *sbrec_mon; SBREC_SERVICE_MONITOR_TABLE_FOR_EACH (sbrec_mon, sbrec_service_monitor_table) { @@ -4223,24 +4216,23 @@ build_lb_svcs( struct service_monitor_info *mon_info = xzalloc(sizeof *mon_info); mon_info->sbrec_mon = sbrec_mon; mon_info->required = false; - hmap_insert(&monitor_map, &mon_info->hmap_node, hash); + hmap_insert(svc_monitor_map, &mon_info->hmap_node, hash); } - struct ovn_northd_lb *lb; - HMAP_FOR_EACH (lb, hmap_node, lbs) { - ovn_lb_svc_create(ovnsb_txn, lb, &monitor_map, ls_ports, + struct ovn_lb_datapaths *lb_dps; + HMAP_FOR_EACH (lb_dps, hmap_node, lb_dps_map) { + ovn_lb_svc_create(ovnsb_txn, lb_dps->lb, svc_monitor_map, ls_ports, svc_monitor_lsps); } struct service_monitor_info *mon_info; - HMAP_FOR_EACH_POP (mon_info, hmap_node, &monitor_map) { + HMAP_FOR_EACH_SAFE (mon_info, hmap_node, svc_monitor_map) { if (!mon_info->required) { sbrec_service_monitor_delete(mon_info->sbrec_mon); + hmap_remove(svc_monitor_map, &mon_info->hmap_node); + free(mon_info); } - - free(mon_info); } - hmap_destroy(&monitor_map); } static bool lrouter_port_ipv4_reachable(const struct ovn_port *op, @@ -4325,7 +4317,8 @@ build_lrouter_lbs_check(const struct ovn_datapaths *lr_datapaths) static void build_lrouter_lbs_reachable_ips(struct ovn_datapaths *lr_datapaths, - struct hmap *lbs, struct hmap *lb_groups) + struct hmap *lb_dps_map, + struct hmap *lb_group_dps_map) { struct ovn_datapath *od; @@ -4335,21 +4328,25 @@ build_lrouter_lbs_reachable_ips(struct ovn_datapaths *lr_datapaths, } for (size_t i = 0; i < od->nbr->n_load_balancer; i++) { - struct ovn_northd_lb *lb = - ovn_northd_lb_find(lbs, - &od->nbr->load_balancer[i]->header_.uuid); - build_lrouter_lb_reachable_ips(od, lb); + struct ovn_lb_datapaths *lb_dps = + ovn_lb_datapaths_find(lb_dps_map, + &od->nbr->load_balancer[i]->header_.uuid); + ovs_assert(lb_dps); + build_lrouter_lb_reachable_ips(od, lb_dps->lb); } for (size_t i = 0; i < od->nbr->n_load_balancer_group; i++) { const struct nbrec_load_balancer_group *nbrec_lb_group = od->nbr->load_balancer_group[i]; - struct ovn_lb_group *lb_group; - - lb_group = ovn_lb_group_find(lb_groups, - &nbrec_lb_group->header_.uuid); - for (size_t j = 0; j < lb_group->n_lbs; j++) { - build_lrouter_lb_reachable_ips(od, lb_group->lbs[j]); + struct ovn_lb_group_datapaths *lb_group_dps; + + lb_group_dps = + ovn_lb_group_datapaths_find(lb_group_dps_map, + &nbrec_lb_group->header_.uuid); + ovs_assert(lb_group_dps); + for (size_t j = 0; j < lb_group_dps->lb_group->n_lbs; j++) { + build_lrouter_lb_reachable_ips(od, + lb_group_dps->lb_group->lbs[j]); } } } @@ -4357,45 +4354,50 @@ build_lrouter_lbs_reachable_ips(struct ovn_datapaths *lr_datapaths, static void build_lswitch_lbs_from_lrouter(struct ovn_datapaths *lr_datapaths, - struct hmap *lbs, struct hmap *lb_groups) + struct hmap *lb_dps_map, + struct hmap *lb_group_dps_map) { if (!install_ls_lb_from_router) { return; } - struct ovn_northd_lb *lb; + struct ovn_lb_datapaths *lb_dps; size_t index; - HMAP_FOR_EACH (lb, hmap_node, lbs) { - BITMAP_FOR_EACH_1 (index, ods_size(lr_datapaths), lb->nb_lr_map) { + HMAP_FOR_EACH (lb_dps, hmap_node, lb_dps_map) { + BITMAP_FOR_EACH_1 (index, ods_size(lr_datapaths), lb_dps->nb_lr_map) { struct ovn_datapath *od = lr_datapaths->array[index]; - ovn_northd_lb_add_ls(lb, od->n_ls_peers, od->ls_peers); - } - } - - struct ovn_lb_group *lb_group; - HMAP_FOR_EACH (lb_group, hmap_node, lb_groups) { - for (size_t i = 0; i < lb_group->n_lr; i++) { - struct ovn_datapath *od = lb_group->lr[i]; - ovn_lb_group_add_ls(lb_group, od->n_ls_peers, od->ls_peers); - for (size_t j = 0; j < lb_group->n_lbs; j++) { - ovn_northd_lb_add_ls(lb_group->lbs[j], od->n_ls_peers, - od->ls_peers); + ovn_lb_datapaths_add_ls(lb_dps, od->n_ls_peers, od->ls_peers); + } + } + + struct ovn_lb_group_datapaths *lb_group_dps; + HMAP_FOR_EACH (lb_group_dps, hmap_node, lb_group_dps_map) { + for (size_t i = 0; i < lb_group_dps->n_lr; i++) { + struct ovn_datapath *od = lb_group_dps->lr[i]; + ovn_lb_group_datapaths_add_ls(lb_group_dps, od->n_ls_peers, + od->ls_peers); + for (size_t j = 0; j < lb_group_dps->lb_group->n_lbs; j++) { + const struct uuid *lb_uuid = + &lb_group_dps->lb_group->lbs[j]->nlb->header_.uuid; + lb_dps = ovn_lb_datapaths_find(lb_dps_map, lb_uuid); + ovs_assert(lb_dps); + ovn_lb_datapaths_add_ls(lb_dps, od->n_ls_peers, od->ls_peers); } } } } static void -build_lb_count_dps(struct hmap *lbs, +build_lb_count_dps(struct hmap *lb_dps_map, size_t n_ls_datapaths, size_t n_lr_datapaths) { - struct ovn_northd_lb *lb; + struct ovn_lb_datapaths *lb_dps; - HMAP_FOR_EACH (lb, hmap_node, lbs) { - lb->n_nb_lr = bitmap_count1(lb->nb_lr_map, n_lr_datapaths); - lb->n_nb_ls = bitmap_count1(lb->nb_ls_map, n_ls_datapaths); + HMAP_FOR_EACH (lb_dps, hmap_node, lb_dps_map) { + lb_dps->n_nb_lr = bitmap_count1(lb_dps->nb_lr_map, n_lr_datapaths); + lb_dps->n_nb_ls = bitmap_count1(lb_dps->nb_ls_map, n_ls_datapaths); } } @@ -4408,13 +4410,16 @@ build_lb_port_related_data( struct ovsdb_idl_txn *ovnsb_txn, const struct sbrec_service_monitor_table *sbrec_service_monitor_table, struct ovn_datapaths *lr_datapaths, struct hmap *ls_ports, - struct hmap *lbs, struct hmap *lb_groups, struct sset *svc_monitor_lsps) + struct hmap *lb_dps_map, struct hmap *lb_group_dps_map, + struct sset *svc_monitor_lsps, + struct hmap *svc_monitor_map) { build_lrouter_lbs_check(lr_datapaths); - build_lrouter_lbs_reachable_ips(lr_datapaths, lbs, lb_groups); - build_lb_svcs(ovnsb_txn, sbrec_service_monitor_table, ls_ports, lbs, - svc_monitor_lsps); - build_lswitch_lbs_from_lrouter(lr_datapaths, lbs, lb_groups); + build_lrouter_lbs_reachable_ips(lr_datapaths, lb_dps_map, + lb_group_dps_map); + build_lb_svcs(ovnsb_txn, sbrec_service_monitor_table, ls_ports, lb_dps_map, + svc_monitor_lsps, svc_monitor_map); + build_lswitch_lbs_from_lrouter(lr_datapaths, lb_dps_map, lb_group_dps_map); } @@ -4535,17 +4540,39 @@ ovn_dp_group_get_or_create(struct ovsdb_idl_txn *ovnsb_txn, return dpg; } +struct sb_lb { + struct hmap_node hmap_node; + + const struct sbrec_load_balancer *slb; + struct ovn_dp_group *dpg; + struct uuid lb_uuid; +}; + +static struct sb_lb * +find_slb_in_sb_lbs(struct hmap *sb_lbs, const struct uuid *lb_uuid) +{ + struct sb_lb *sb_lb; + HMAP_FOR_EACH_WITH_HASH (sb_lb, hmap_node, uuid_hash(lb_uuid), sb_lbs) { + if (uuid_equals(&sb_lb->lb_uuid, lb_uuid)) { + return sb_lb; + } + } + + return NULL; +} + /* Syncs relevant load balancers (applied to logical switches) to the * Southbound database. */ void sync_lbs(struct ovsdb_idl_txn *ovnsb_txn, const struct sbrec_load_balancer_table *sbrec_load_balancer_table, - struct ovn_datapaths *ls_datapaths, struct hmap *lbs) + struct ovn_datapaths *ls_datapaths, struct hmap *lb_dps_map) { struct hmap dp_groups = HMAP_INITIALIZER(&dp_groups); size_t bitmap_len = ods_size(ls_datapaths); - struct ovn_northd_lb *lb; + struct ovn_lb_datapaths *lb_dps; + struct hmap sb_lbs = HMAP_INITIALIZER(&sb_lbs); /* Delete any stale SB load balancer rows and create datapath * groups for existing ones. */ @@ -4568,28 +4595,32 @@ sync_lbs(struct ovsdb_idl_txn *ovnsb_txn, * "at-least-once" consistency for clustered database tables that * are not indexed in any way. */ - lb = ovn_northd_lb_find(lbs, &lb_uuid); - if (!lb || !lb->n_nb_ls || !hmapx_add(&existing_lbs, lb)) { + lb_dps = ovn_lb_datapaths_find(lb_dps_map, &lb_uuid); + if (!lb_dps || !lb_dps->n_nb_ls || !hmapx_add(&existing_lbs, lb_dps)) { sbrec_load_balancer_delete(sbrec_lb); continue; } - lb->slb = sbrec_lb; + struct sb_lb *sb_lb = xzalloc(sizeof *sb_lb); + sb_lb->lb_uuid = lb_uuid; + sb_lb->slb = sbrec_lb; + hmap_insert(&sb_lbs, &sb_lb->hmap_node, uuid_hash(&lb_uuid)); /* Find or create datapath group for this load balancer. */ - lb->dpg = ovn_dp_group_get_or_create(ovnsb_txn, &dp_groups, - lb->slb->datapath_group, - lb->n_nb_ls, lb->nb_ls_map, - bitmap_len, true, - ls_datapaths, NULL); + sb_lb->dpg = ovn_dp_group_get_or_create(ovnsb_txn, &dp_groups, + sb_lb->slb->datapath_group, + lb_dps->n_nb_ls, + lb_dps->nb_ls_map, + bitmap_len, true, + ls_datapaths, NULL); } hmapx_destroy(&existing_lbs); /* Create SB Load balancer records if not present and sync * the SB load balancer columns. */ - HMAP_FOR_EACH (lb, hmap_node, lbs) { + HMAP_FOR_EACH (lb_dps, hmap_node, lb_dps_map) { - if (!lb->n_nb_ls) { + if (!lb_dps->n_nb_ls) { continue; } @@ -4597,37 +4628,44 @@ sync_lbs(struct ovsdb_idl_txn *ovnsb_txn, * transport port) tuple. */ struct smap options; - smap_clone(&options, &lb->nlb->options); + smap_clone(&options, &lb_dps->lb->nlb->options); smap_replace(&options, "hairpin_orig_tuple", "true"); - if (!lb->slb) { + struct sb_lb *sb_lb = find_slb_in_sb_lbs(&sb_lbs, + &lb_dps->lb->nlb->header_.uuid); + ovs_assert(!sb_lb || (sb_lb->slb && sb_lb->dpg)); + struct ovn_dp_group *lb_dpg = NULL; + if (!sb_lb) { sbrec_lb = sbrec_load_balancer_insert(ovnsb_txn); - lb->slb = sbrec_lb; char *lb_id = xasprintf( - UUID_FMT, UUID_ARGS(&lb->nlb->header_.uuid)); + UUID_FMT, UUID_ARGS(&lb_dps->lb->nlb->header_.uuid)); const struct smap external_ids = SMAP_CONST1(&external_ids, "lb_id", lb_id); sbrec_load_balancer_set_external_ids(sbrec_lb, &external_ids); free(lb_id); + } else { + sbrec_lb = sb_lb->slb; + lb_dpg = sb_lb->dpg; } /* Find or create datapath group for this load balancer. */ - if (!lb->dpg) { - lb->dpg = ovn_dp_group_get_or_create(ovnsb_txn, &dp_groups, - lb->slb->datapath_group, - lb->n_nb_ls, lb->nb_ls_map, - bitmap_len, true, - ls_datapaths, NULL); + if (!lb_dpg) { + lb_dpg = ovn_dp_group_get_or_create(ovnsb_txn, &dp_groups, + sbrec_lb->datapath_group, + lb_dps->n_nb_ls, + lb_dps->nb_ls_map, bitmap_len, + true, ls_datapaths, NULL); } /* Update columns. */ - sbrec_load_balancer_set_name(lb->slb, lb->nlb->name); - sbrec_load_balancer_set_vips(lb->slb, ovn_northd_lb_get_vips(lb)); - sbrec_load_balancer_set_protocol(lb->slb, lb->nlb->protocol); - sbrec_load_balancer_set_datapath_group(lb->slb, lb->dpg->dp_group); - sbrec_load_balancer_set_options(lb->slb, &options); + sbrec_load_balancer_set_name(sbrec_lb, lb_dps->lb->nlb->name); + sbrec_load_balancer_set_vips(sbrec_lb, + ovn_northd_lb_get_vips(lb_dps->lb)); + sbrec_load_balancer_set_protocol(sbrec_lb, lb_dps->lb->nlb->protocol); + sbrec_load_balancer_set_datapath_group(sbrec_lb, lb_dpg->dp_group); + sbrec_load_balancer_set_options(sbrec_lb, &options); /* Clearing 'datapaths' column, since 'dp_group' is in use. */ - sbrec_load_balancer_set_datapaths(lb->slb, NULL, 0); + sbrec_load_balancer_set_datapaths(sbrec_lb, NULL, 0); smap_destroy(&options); } @@ -4638,6 +4676,12 @@ sync_lbs(struct ovsdb_idl_txn *ovnsb_txn, } hmap_destroy(&dp_groups); + struct sb_lb *sb_lb; + HMAP_FOR_EACH_POP (sb_lb, hmap_node, &sb_lbs) { + free(sb_lb); + } + hmap_destroy(&sb_lbs); + /* Datapath_Binding.load_balancers is not used anymore, it's still in the * schema for compatibility reasons. Reset it to empty, just in case. */ @@ -7832,15 +7876,17 @@ build_qos(struct ovn_datapath *od, struct hmap *lflows) { } static void -build_lb_rules_pre_stateful(struct hmap *lflows, struct ovn_northd_lb *lb, +build_lb_rules_pre_stateful(struct hmap *lflows, + struct ovn_lb_datapaths *lb_dps, bool ct_lb_mark, const struct ovn_datapaths *ls_datapaths, struct ds *match, struct ds *action) { - if (!lb->n_nb_ls) { + if (!lb_dps->n_nb_ls) { return; } + const struct ovn_northd_lb *lb = lb_dps->lb; for (size_t i = 0; i < lb->n_vips; i++) { struct ovn_lb_vip *lb_vip = &lb->vips[i]; ds_clear(action); @@ -7886,7 +7932,7 @@ build_lb_rules_pre_stateful(struct hmap *lflows, struct ovn_northd_lb *lb, } ovn_lflow_add_with_dp_group( - lflows, lb->nb_ls_map, ods_size(ls_datapaths), + lflows, lb_dps->nb_ls_map, ods_size(ls_datapaths), S_SWITCH_IN_PRE_STATEFUL, 120, ds_cstr(match), ds_cstr(action), &lb->nlb->header_); } @@ -7932,7 +7978,7 @@ build_lb_rules_pre_stateful(struct hmap *lflows, struct ovn_northd_lb *lb, * */ static void -build_lb_affinity_lr_flows(struct hmap *lflows, struct ovn_northd_lb *lb, +build_lb_affinity_lr_flows(struct hmap *lflows, const struct ovn_northd_lb *lb, struct ovn_lb_vip *lb_vip, char *new_lb_match, char *lb_action, const unsigned long *dp_bitmap, const struct ovn_datapaths *lr_datapaths) @@ -8118,14 +8164,16 @@ build_lb_affinity_lr_flows(struct hmap *lflows, struct ovn_northd_lb *lb, * */ static void -build_lb_affinity_ls_flows(struct hmap *lflows, struct ovn_northd_lb *lb, +build_lb_affinity_ls_flows(struct hmap *lflows, + struct ovn_lb_datapaths *lb_dps, struct ovn_lb_vip *lb_vip, const struct ovn_datapaths *ls_datapaths) { - if (!lb->affinity_timeout || !lb->n_nb_ls) { + if (!lb_dps->lb->affinity_timeout || !lb_dps->n_nb_ls) { return; } + const struct ovn_northd_lb *lb = lb_dps->lb; struct ds new_lb_match = DS_EMPTY_INITIALIZER; if (IN6_IS_ADDR_V4MAPPED(&lb_vip->vip)) { ds_put_format(&new_lb_match, @@ -8145,9 +8193,9 @@ build_lb_affinity_ls_flows(struct hmap *lflows, struct ovn_northd_lb *lb, static char *aff_check = REGBIT_KNOWN_LB_SESSION" = chk_lb_aff(); next;"; ovn_lflow_add_with_dp_group( - lflows, lb->nb_ls_map, ods_size(ls_datapaths), + lflows, lb_dps->nb_ls_map, ods_size(ls_datapaths), S_SWITCH_IN_LB_AFF_CHECK, 100, ds_cstr(&new_lb_match), aff_check, - &lb->nlb->header_); + &lb_dps->lb->nlb->header_); ds_destroy(&new_lb_match); struct ds aff_action = DS_EMPTY_INITIALIZER; @@ -8235,14 +8283,15 @@ build_lb_affinity_ls_flows(struct hmap *lflows, struct ovn_northd_lb *lb, /* Forward to OFTABLE_CHK_LB_AFFINITY table to store flow tuple. */ ovn_lflow_add_with_dp_group( - lflows, lb->nb_ls_map, ods_size(ls_datapaths), + lflows, lb_dps->nb_ls_map, ods_size(ls_datapaths), S_SWITCH_IN_LB_AFF_LEARN, 100, ds_cstr(&aff_match_learn), ds_cstr(&aff_action_learn), &lb->nlb->header_); /* Use already selected backend within affinity timeslot. */ ovn_lflow_add_with_dp_group( - lflows, lb->nb_ls_map, ods_size(ls_datapaths), S_SWITCH_IN_LB, 150, - ds_cstr(&aff_match), ds_cstr(&aff_action), &lb->nlb->header_); + lflows, lb_dps->nb_ls_map, ods_size(ls_datapaths), + S_SWITCH_IN_LB, 150, ds_cstr(&aff_match), ds_cstr(&aff_action), + &lb->nlb->header_); ds_truncate(&aff_action, aff_action_len); ds_truncate(&aff_action_learn, aff_action_learn_len); @@ -8275,11 +8324,13 @@ build_lrouter_lb_affinity_default_flows(struct ovn_datapath *od, } static void -build_lb_rules(struct hmap *lflows, struct ovn_northd_lb *lb, +build_lb_rules(struct hmap *lflows, struct ovn_lb_datapaths *lb_dps, const struct ovn_datapaths *ls_datapaths, const struct chassis_features *features, struct ds *match, - struct ds *action, const struct shash *meter_groups) + struct ds *action, const struct shash *meter_groups, + const struct hmap *svc_monitor_map) { + const struct ovn_northd_lb *lb = lb_dps->lb; for (size_t i = 0; i < lb->n_vips; i++) { struct ovn_lb_vip *lb_vip = &lb->vips[i]; struct ovn_northd_lb_vip *lb_vip_nb = &lb->vips_nb[i]; @@ -8300,9 +8351,10 @@ build_lb_rules(struct hmap *lflows, struct ovn_northd_lb *lb, /* New connections in Ingress table. */ const char *meter = NULL; - bool reject = build_lb_vip_actions(lb_vip, lb_vip_nb, action, - lb->selection_fields, NULL, - NULL, true, features); + bool reject = build_lb_vip_actions(lb, lb_vip, lb_vip_nb, action, + lb->selection_fields, + NULL, NULL, true, features, + svc_monitor_map); ds_put_format(match, "ct.new && %s.dst == %s", ip_match, lb_vip->vip_str); @@ -8313,15 +8365,17 @@ build_lb_rules(struct hmap *lflows, struct ovn_northd_lb *lb, priority = 120; } - build_lb_affinity_ls_flows(lflows, lb, lb_vip, ls_datapaths); + build_lb_affinity_ls_flows(lflows, lb_dps, lb_vip, ls_datapaths); unsigned long *dp_non_meter = NULL; bool build_non_meter = false; if (reject) { size_t index; - dp_non_meter = bitmap_clone(lb->nb_ls_map, ods_size(ls_datapaths)); - BITMAP_FOR_EACH_1 (index, ods_size(ls_datapaths), lb->nb_ls_map) { + dp_non_meter = bitmap_clone(lb_dps->nb_ls_map, + ods_size(ls_datapaths)); + BITMAP_FOR_EACH_1 (index, ods_size(ls_datapaths), + lb_dps->nb_ls_map) { struct ovn_datapath *od = ls_datapaths->array[index]; meter = copp_meter_get(COPP_REJECT, od->nbs->copp, @@ -8339,7 +8393,7 @@ build_lb_rules(struct hmap *lflows, struct ovn_northd_lb *lb, } if (!reject || build_non_meter) { ovn_lflow_add_with_dp_group( - lflows, dp_non_meter ? dp_non_meter : lb->nb_ls_map, + lflows, dp_non_meter ? dp_non_meter : lb_dps->nb_ls_map, ods_size(ls_datapaths), S_SWITCH_IN_LB, priority, ds_cstr(match), ds_cstr(action), &lb->nlb->header_); } @@ -9554,7 +9608,8 @@ build_lswitch_arp_nd_responder_default(struct ovn_datapath *od, /* Ingress table 19: ARP/ND responder for service monitor source ip. * (priority 110)*/ static void -build_lswitch_arp_nd_service_monitor(struct ovn_northd_lb *lb, +build_lswitch_arp_nd_service_monitor(const struct ovn_northd_lb *lb, + const struct hmap *ls_ports, struct hmap *lflows, struct ds *actions, struct ds *match) @@ -9569,7 +9624,14 @@ build_lswitch_arp_nd_service_monitor(struct ovn_northd_lb *lb, for (size_t j = 0; j < lb_vip_nb->n_backends; j++) { struct ovn_northd_lb_backend *backend_nb = &lb_vip_nb->backends_nb[j]; - if (!backend_nb->op || !backend_nb->svc_mon_src_ip) { + + if (!backend_nb->health_check) { + continue; + } + + struct ovn_port *op = ovn_port_find(ls_ports, + backend_nb->logical_port); + if (!op || !backend_nb->svc_mon_src_ip) { continue; } @@ -9611,7 +9673,7 @@ build_lswitch_arp_nd_service_monitor(struct ovn_northd_lb *lb, svc_monitor_mac); } ovn_lflow_add_with_hint(lflows, - backend_nb->op->od, + op->od, S_SWITCH_IN_ARP_ND_RSP, 110, ds_cstr(match), ds_cstr(actions), &lb->nlb->header_); @@ -11336,7 +11398,7 @@ struct lrouter_nat_lb_flows_ctx { struct ds *gw_redir_action; struct ovn_lb_vip *lb_vip; - struct ovn_northd_lb *lb; + const struct ovn_northd_lb *lb; bool reject; int prio; @@ -11468,14 +11530,16 @@ build_gw_lrouter_nat_flows_for_lb(struct lrouter_nat_lb_flows_ctx *ctx, static void build_lrouter_nat_flows_for_lb(struct ovn_lb_vip *lb_vip, - struct ovn_northd_lb *lb, + struct ovn_lb_datapaths *lb_dps, struct ovn_northd_lb_vip *vips_nb, const struct ovn_datapaths *lr_datapaths, struct hmap *lflows, struct ds *match, struct ds *action, const struct shash *meter_groups, - const struct chassis_features *features) + const struct chassis_features *features, + const struct hmap *svc_monitor_map) { + const struct ovn_northd_lb *lb = lb_dps->lb; bool ipv4 = lb_vip->address_family == AF_INET; const char *ip_match = ipv4 ? "ip4" : "ip6"; @@ -11490,9 +11554,10 @@ build_lrouter_nat_flows_for_lb(struct ovn_lb_vip *lb_vip, ds_clear(match); ds_clear(action); - bool reject = build_lb_vip_actions(lb_vip, vips_nb, action, + bool reject = build_lb_vip_actions(lb, lb_vip, vips_nb, action, lb->selection_fields, &skip_snat_act, - &force_snat_act, false, features); + &force_snat_act, false, features, + svc_monitor_map); /* Higher priority rules are added for load-balancing in DNAT * table. For every match (on a VIP[:port]), we add two flows. @@ -11567,7 +11632,7 @@ build_lrouter_nat_flows_for_lb(struct ovn_lb_vip *lb_vip, * lflow generation for them. */ size_t index; - BITMAP_FOR_EACH_1 (index, bitmap_len, lb->nb_lr_map) { + BITMAP_FOR_EACH_1 (index, bitmap_len, lb_dps->nb_lr_map) { struct ovn_datapath *od = lr_datapaths->array[index]; enum lrouter_nat_lb_flow_type type; @@ -11647,16 +11712,19 @@ build_lrouter_nat_flows_for_lb(struct ovn_lb_vip *lb_vip, } static void -build_lswitch_flows_for_lb(struct ovn_northd_lb *lb, struct hmap *lflows, +build_lswitch_flows_for_lb(struct ovn_lb_datapaths *lb_dps, + struct hmap *lflows, const struct shash *meter_groups, const struct ovn_datapaths *ls_datapaths, const struct chassis_features *features, + const struct hmap *svc_monitor_map, struct ds *match, struct ds *action) { - if (!lb->n_nb_ls) { + if (!lb_dps->n_nb_ls) { return; } + const struct ovn_northd_lb *lb = lb_dps->lb; for (size_t i = 0; i < lb->n_vips; i++) { struct ovn_lb_vip *lb_vip = &lb->vips[i]; @@ -11666,7 +11734,7 @@ build_lswitch_flows_for_lb(struct ovn_northd_lb *lb, struct hmap *lflows, } size_t index; - BITMAP_FOR_EACH_1 (index, ods_size(ls_datapaths), lb->nb_ls_map) { + BITMAP_FOR_EACH_1 (index, ods_size(ls_datapaths), lb_dps->nb_ls_map) { struct ovn_datapath *od = ls_datapaths->array[index]; ovn_lflow_add_with_hint__(lflows, od, @@ -11690,10 +11758,10 @@ build_lswitch_flows_for_lb(struct ovn_northd_lb *lb, struct hmap *lflows, * a higher priority rule for load balancing below also commits the * connection, so it is okay if we do not hit the above match on * REGBIT_CONNTRACK_COMMIT. */ - build_lb_rules_pre_stateful(lflows, lb, features->ct_no_masked_label, + build_lb_rules_pre_stateful(lflows, lb_dps, features->ct_no_masked_label, ls_datapaths, match, action); - build_lb_rules(lflows, lb, ls_datapaths, features, match, action, - meter_groups); + build_lb_rules(lflows, lb_dps, ls_datapaths, features, match, action, + meter_groups, svc_monitor_map); } /* If there are any load balancing rules, we should send the packet to @@ -11705,17 +11773,17 @@ build_lswitch_flows_for_lb(struct ovn_northd_lb *lb, struct hmap *lflows, * defragmentation to match on L4 ports. */ static void -build_lrouter_defrag_flows_for_lb(struct ovn_northd_lb *lb, +build_lrouter_defrag_flows_for_lb(struct ovn_lb_datapaths *lb_dps, struct hmap *lflows, const struct ovn_datapaths *lr_datapaths, struct ds *match) { - if (!lb->n_nb_lr) { + if (!lb_dps->n_nb_lr) { return; } - for (size_t i = 0; i < lb->n_vips; i++) { - struct ovn_lb_vip *lb_vip = &lb->vips[i]; + for (size_t i = 0; i < lb_dps->lb->n_vips; i++) { + struct ovn_lb_vip *lb_vip = &lb_dps->lb->vips[i]; bool ipv6 = lb_vip->address_family == AF_INET6; int prio = 100; @@ -11724,36 +11792,41 @@ build_lrouter_defrag_flows_for_lb(struct ovn_northd_lb *lb, lb_vip->vip_str); ovn_lflow_add_with_dp_group( - lflows, lb->nb_lr_map, ods_size(lr_datapaths), S_ROUTER_IN_DEFRAG, - prio, ds_cstr(match), "ct_dnat;", &lb->nlb->header_); + lflows, lb_dps->nb_lr_map, ods_size(lr_datapaths), + S_ROUTER_IN_DEFRAG, prio, ds_cstr(match), "ct_dnat;", + &lb_dps->lb->nlb->header_); } } static void -build_lrouter_flows_for_lb(struct ovn_northd_lb *lb, struct hmap *lflows, +build_lrouter_flows_for_lb(struct ovn_lb_datapaths *lb_dps, + struct hmap *lflows, const struct shash *meter_groups, const struct ovn_datapaths *lr_datapaths, const struct chassis_features *features, + const struct hmap *svc_monitor_map, struct ds *match, struct ds *action) { size_t index; - if (!lb->n_nb_lr) { + if (!lb_dps->n_nb_lr) { return; } + const struct ovn_northd_lb *lb = lb_dps->lb; for (size_t i = 0; i < lb->n_vips; i++) { struct ovn_lb_vip *lb_vip = &lb->vips[i]; - build_lrouter_nat_flows_for_lb(lb_vip, lb, &lb->vips_nb[i], + build_lrouter_nat_flows_for_lb(lb_vip, lb_dps, &lb->vips_nb[i], lr_datapaths, lflows, match, action, - meter_groups, features); + meter_groups, features, + svc_monitor_map); if (!build_empty_lb_event_flow(lb_vip, lb, match, action)) { continue; } - BITMAP_FOR_EACH_1 (index, ods_size(lr_datapaths), lb->nb_lr_map) { + BITMAP_FOR_EACH_1 (index, ods_size(lr_datapaths), lb_dps->nb_lr_map) { struct ovn_datapath *od = lr_datapaths->array[index]; ovn_lflow_add_with_hint__(lflows, od, S_ROUTER_IN_DNAT, @@ -11767,7 +11840,7 @@ build_lrouter_flows_for_lb(struct ovn_northd_lb *lb, struct hmap *lflows, } if (lb->skip_snat) { - BITMAP_FOR_EACH_1 (index, ods_size(lr_datapaths), lb->nb_lr_map) { + BITMAP_FOR_EACH_1 (index, ods_size(lr_datapaths), lb_dps->nb_lr_map) { struct ovn_datapath *od = lr_datapaths->array[index]; ovn_lflow_add(lflows, od, S_ROUTER_OUT_SNAT, 120, @@ -15484,7 +15557,8 @@ struct lswitch_flow_build_info { struct hmap *lflows; struct hmap *igmp_groups; const struct shash *meter_groups; - const struct hmap *lbs; + const struct hmap *lb_dps_map; + const struct hmap *svc_monitor_map; const struct hmap *bfd_connections; const struct chassis_features *features; char *svc_check_match; @@ -15628,7 +15702,7 @@ build_lflows_thread(void *arg) struct ovn_datapath *od; struct ovn_port *op; - struct ovn_northd_lb *lb; + struct ovn_lb_datapaths *lb_dps; struct ovn_igmp_group *igmp_group; int bnum; @@ -15695,28 +15769,33 @@ build_lflows_thread(void *arg) } } for (bnum = control->id; - bnum <= lsi->lbs->mask; + bnum <= lsi->lb_dps_map->mask; bnum += control->pool->size) { - HMAP_FOR_EACH_IN_PARALLEL (lb, hmap_node, bnum, lsi->lbs) { + HMAP_FOR_EACH_IN_PARALLEL (lb_dps, hmap_node, bnum, + lsi->lb_dps_map) { if (stop_parallel_processing()) { return NULL; } - build_lswitch_arp_nd_service_monitor(lb, lsi->lflows, + build_lswitch_arp_nd_service_monitor(lb_dps->lb, + lsi->ls_ports, + lsi->lflows, &lsi->match, &lsi->actions); - build_lrouter_defrag_flows_for_lb(lb, lsi->lflows, + build_lrouter_defrag_flows_for_lb(lb_dps, lsi->lflows, lsi->lr_datapaths, &lsi->match); - build_lrouter_flows_for_lb(lb, lsi->lflows, + build_lrouter_flows_for_lb(lb_dps, lsi->lflows, lsi->meter_groups, lsi->lr_datapaths, lsi->features, + lsi->svc_monitor_map, &lsi->match, &lsi->actions); - build_lswitch_flows_for_lb(lb, lsi->lflows, + build_lswitch_flows_for_lb(lb_dps, lsi->lflows, lsi->meter_groups, lsi->ls_datapaths, lsi->features, + lsi->svc_monitor_map, &lsi->match, &lsi->actions); } } @@ -15782,7 +15861,8 @@ build_lswitch_and_lrouter_flows(const struct ovn_datapaths *ls_datapaths, struct hmap *lflows, struct hmap *igmp_groups, const struct shash *meter_groups, - const struct hmap *lbs, + const struct hmap *lb_dps_map, + const struct hmap *svc_monitor_map, const struct hmap *bfd_connections, const struct chassis_features *features) { @@ -15809,7 +15889,8 @@ build_lswitch_and_lrouter_flows(const struct ovn_datapaths *ls_datapaths, lsiv[index].port_groups = port_groups; lsiv[index].igmp_groups = igmp_groups; lsiv[index].meter_groups = meter_groups; - lsiv[index].lbs = lbs; + lsiv[index].lb_dps_map = lb_dps_map; + lsiv[index].svc_monitor_map = svc_monitor_map; lsiv[index].bfd_connections = bfd_connections; lsiv[index].features = features; lsiv[index].svc_check_match = svc_check_match; @@ -15832,7 +15913,7 @@ build_lswitch_and_lrouter_flows(const struct ovn_datapaths *ls_datapaths, } else { struct ovn_datapath *od; struct ovn_port *op; - struct ovn_northd_lb *lb; + struct ovn_lb_datapaths *lb_dps; struct ovn_igmp_group *igmp_group; struct lswitch_flow_build_info lsi = { .ls_datapaths = ls_datapaths, @@ -15843,7 +15924,8 @@ build_lswitch_and_lrouter_flows(const struct ovn_datapaths *ls_datapaths, .lflows = lflows, .igmp_groups = igmp_groups, .meter_groups = meter_groups, - .lbs = lbs, + .lb_dps_map = lb_dps_map, + .svc_monitor_map = svc_monitor_map, .bfd_connections = bfd_connections, .features = features, .svc_check_match = svc_check_match, @@ -15875,17 +15957,19 @@ build_lswitch_and_lrouter_flows(const struct ovn_datapaths *ls_datapaths, } stopwatch_stop(LFLOWS_PORTS_STOPWATCH_NAME, time_msec()); stopwatch_start(LFLOWS_LBS_STOPWATCH_NAME, time_msec()); - HMAP_FOR_EACH (lb, hmap_node, lbs) { - build_lswitch_arp_nd_service_monitor(lb, lsi.lflows, - &lsi.actions, + HMAP_FOR_EACH (lb_dps, hmap_node, lb_dps_map) { + build_lswitch_arp_nd_service_monitor(lb_dps->lb, lsi.ls_ports, + lsi.lflows, &lsi.actions, &lsi.match); - build_lrouter_defrag_flows_for_lb(lb, lsi.lflows, lsi.lr_datapaths, - &lsi.match); - build_lrouter_flows_for_lb(lb, lsi.lflows, lsi.meter_groups, + build_lrouter_defrag_flows_for_lb(lb_dps, lsi.lflows, + lsi.lr_datapaths, &lsi.match); + build_lrouter_flows_for_lb(lb_dps, lsi.lflows, lsi.meter_groups, lsi.lr_datapaths, lsi.features, + lsi.svc_monitor_map, &lsi.match, &lsi.actions); - build_lswitch_flows_for_lb(lb, lsi.lflows, lsi.meter_groups, + build_lswitch_flows_for_lb(lb_dps, lsi.lflows, lsi.meter_groups, lsi.ls_datapaths, lsi.features, + lsi.svc_monitor_map, &lsi.match, &lsi.actions); } stopwatch_stop(LFLOWS_LBS_STOPWATCH_NAME, time_msec()); @@ -15985,7 +16069,9 @@ void build_lflows(struct ovsdb_idl_txn *ovnsb_txn, input_data->lr_ports, input_data->port_groups, lflows, &igmp_groups, - input_data->meter_groups, input_data->lbs, + input_data->meter_groups, + input_data->lb_datapaths_map, + input_data->svc_monitor_map, input_data->bfd_connections, input_data->features); @@ -17388,8 +17474,8 @@ northd_init(struct northd_data *data) hmap_init(&data->lr_ports); hmap_init(&data->port_groups); shash_init(&data->meter_groups); - hmap_init(&data->lbs); - hmap_init(&data->lb_groups); + hmap_init(&data->lb_datapaths_map); + hmap_init(&data->lb_group_datapaths_map); ovs_list_init(&data->lr_list); data->features = (struct chassis_features) { .ct_no_masked_label = true, @@ -17399,6 +17485,7 @@ northd_init(struct northd_data *data) }; data->ovn_internal_version_changed = false; sset_init(&data->svc_monitor_lsps); + hmap_init(&data->svc_monitor_map); data->change_tracked = false; ovs_list_init(&data->tracked_ls_changes.updated); } @@ -17406,17 +17493,18 @@ northd_init(struct northd_data *data) void northd_destroy(struct northd_data *data) { - struct ovn_northd_lb *lb; - HMAP_FOR_EACH_POP (lb, hmap_node, &data->lbs) { - ovn_northd_lb_destroy(lb); + struct ovn_lb_datapaths *lb_dps; + HMAP_FOR_EACH_POP (lb_dps, hmap_node, &data->lb_datapaths_map) { + ovn_lb_datapaths_destroy(lb_dps); } - hmap_destroy(&data->lbs); + hmap_destroy(&data->lb_datapaths_map); - struct ovn_lb_group *lb_group; - HMAP_FOR_EACH_POP (lb_group, hmap_node, &data->lb_groups) { - ovn_lb_group_destroy(lb_group); + struct ovn_lb_group_datapaths *lb_group_dps; + HMAP_FOR_EACH_POP (lb_group_dps, hmap_node, + &data->lb_group_datapaths_map) { + ovn_lb_group_datapaths_destroy(lb_group_dps); } - hmap_destroy(&data->lb_groups); + hmap_destroy(&data->lb_group_datapaths_map); struct ovn_port_group *pg; HMAP_FOR_EACH_SAFE (pg, key_node, &data->port_groups) { @@ -17431,6 +17519,12 @@ northd_destroy(struct northd_data *data) } shash_destroy(&data->meter_groups); + struct service_monitor_info *mon_info; + HMAP_FOR_EACH_POP (mon_info, hmap_node, &data->svc_monitor_map) { + free(mon_info); + } + hmap_destroy(&data->svc_monitor_map); + /* XXX Having to explicitly clean up macam here * is a bit strange. We don't explicitly initialize * macam in this module, but this is the logical place @@ -17539,10 +17633,9 @@ ovnnb_db_run(struct northd_input *input_data, input_data->sbrec_chassis_table, &data->ls_datapaths, &data->lr_datapaths, &data->lr_list); - build_lbs(input_data->nbrec_load_balancer_table, - input_data->nbrec_load_balancer_group_table, - &data->ls_datapaths, &data->lr_datapaths, &data->lbs, - &data->lb_groups); + build_lb_datapaths(input_data->lbs, input_data->lb_groups, + &data->ls_datapaths, &data->lr_datapaths, + &data->lb_datapaths_map, &data->lb_group_datapaths_map); build_ports(ovnsb_txn, input_data->sbrec_port_binding_table, input_data->sbrec_chassis_table, @@ -17557,9 +17650,11 @@ ovnnb_db_run(struct northd_input *input_data, build_lb_port_related_data(ovnsb_txn, input_data->sbrec_service_monitor_table, &data->lr_datapaths, &data->ls_ports, - &data->lbs, &data->lb_groups, - &data->svc_monitor_lsps); - build_lb_count_dps(&data->lbs, + &data->lb_datapaths_map, + &data->lb_group_datapaths_map, + &data->svc_monitor_lsps, + &data->svc_monitor_map); + build_lb_count_dps(&data->lb_datapaths_map, ods_size(&data->ls_datapaths), ods_size(&data->lr_datapaths)); build_ipam(&data->ls_datapaths.datapaths, &data->ls_ports); diff --git a/northd/northd.h b/northd/northd.h index 48c282476a..7d92028c7d 100644 --- a/northd/northd.h +++ b/northd/northd.h @@ -28,9 +28,6 @@ struct northd_input { const struct nbrec_nb_global_table *nbrec_nb_global_table; const struct nbrec_logical_switch_table *nbrec_logical_switch_table; const struct nbrec_logical_router_table *nbrec_logical_router_table; - const struct nbrec_load_balancer_table *nbrec_load_balancer_table; - const struct nbrec_load_balancer_group_table - *nbrec_load_balancer_group_table; const struct nbrec_port_group_table *nbrec_port_group_table; const struct nbrec_meter_table *nbrec_meter_table; const struct nbrec_acl_table *nbrec_acl_table; @@ -59,6 +56,10 @@ struct northd_input { *sbrec_chassis_template_var_table; const struct sbrec_mirror_table *sbrec_mirror_table; + /* Northd lb data node inputs*/ + const struct hmap *lbs; + const struct hmap *lb_groups; + /* Indexes */ struct ovsdb_idl_index *sbrec_chassis_by_name; struct ovsdb_idl_index *sbrec_chassis_by_hostname; @@ -110,12 +111,13 @@ struct northd_data { struct hmap lr_ports; struct hmap port_groups; struct shash meter_groups; - struct hmap lbs; - struct hmap lb_groups; + struct hmap lb_datapaths_map; + struct hmap lb_group_datapaths_map; struct ovs_list lr_list; bool ovn_internal_version_changed; struct chassis_features features; struct sset svc_monitor_lsps; + struct hmap svc_monitor_map; bool change_tracked; struct tracked_ls_changes tracked_ls_changes; }; @@ -146,9 +148,10 @@ struct lflow_input { const struct hmap *lr_ports; const struct hmap *port_groups; const struct shash *meter_groups; - const struct hmap *lbs; + const struct hmap *lb_datapaths_map; const struct hmap *bfd_connections; const struct chassis_features *features; + const struct hmap *svc_monitor_map; bool ovn_internal_version_changed; };