| Message ID | 20230912174718.258722-1-numans@ovn.org |
|---|---|
| State | Accepted |
| Headers | show |
| Series | northd: I-P for load balancer and lb groups | expand |
| Context | Check | Description |
|---|---|---|
| ovsrobot/apply-robot | success | apply and check: success |
| ovsrobot/github-robot-_Build_and_Test | success | github build: passed |
| ovsrobot/github-robot-_ovn-kubernetes | fail | github build: failed |
On Tue, Sep 12, 2023 at 10:47 AM <numans@ovn.org> wrote: > > From: Numan Siddique <numans@ovn.org> > > When a logical router gets updated due to load balancer or load balancer > groups changes, it is now incrementally handled first in 'lb_data' > engine node similar to how logical switch changes are handled. The > tracking data of 'lb_data' is updated similarly so that northd engine > handler - northd_handle_lb_data_changes() handles it. > > A new handler northd_handle_lr_changes() is added in the 'northd' engine > node for logical router changes. This handler returns true if only > load balancer or load balancer group columns are changed. It returns > false for any other changes. > > northd_handle_lb_data_changes() also sets the logical router > od's lb_ips accordingly. > > Below are the scale testing results done with these patches applied > using ovn-heater. The test ran the scenario - > ocp-500-density-heavy.yml [1]. > > With these patches applied (with load balancer I-P handling in northd > engine node) the resuts are: > > ------------------------------------------------------------------------------------------------------------------------------------------------------- > Min (s) Median (s) 90%ile (s) 99%ile (s) Max (s) Mean (s) Total (s) Count Failed > ------------------------------------------------------------------------------------------------------------------------------------------------------- > Iteration Total 0.131363 1.189994 3.213526 4.308134 4.394562 1.385713 173.214153 125 0 > Namespace.add_ports 0.005176 0.005611 0.006476 0.019867 0.024206 0.006058 0.757188 125 0 > WorkerNode.bind_port 0.033776 0.046343 0.054414 0.061719 0.063613 0.046815 11.703773 250 0 > WorkerNode.ping_port 0.005156 0.006959 2.044939 3.655328 4.241496 0.627103 156.775832 250 0 > ------------------------------------------------------------------------------------------------------------------------------------------------------- > > The results with the present main are: > > ------------------------------------------------------------------------------------------------------------------------------------------------------- > Min (s) Median (s) 90%ile (s) 99%ile (s) Max (s) Mean (s) Total (s) Count Failed > ------------------------------------------------------------------------------------------------------------------------------------------------------- > Iteration Total 3.233795 4.364926 5.400982 6.412803 7.409757 4.792270 599.033790 125 0 > Namespace.add_ports 0.005230 0.006564 0.007379 0.019060 0.037490 0.007223 0.902930 125 0 > WorkerNode.bind_port 0.033864 0.044052 0.049608 0.054849 0.056196 0.044005 11.001231 250 0 > WorkerNode.ping_port 0.005334 2.060477 5.222422 6.267332 7.284001 2.323020 580.754964 250 0 > ------------------------------------------------------------------------------------------------------------------------------------------------------- > > Few observations: > > - The total time taken has come down significantly from 599 seconds to 173. > - 99%ile with these patches is 4.3 seconds compared to 6.4 seconds for the > main. > - 99%ile with these patches is 3.2 seconds compared to 5.4 seconds for the > main. > - CPU utilization of northd during the test with these patches > is between 100% to 300% which is almost the same as main. > Main difference being that, with these patches the test duration is > less and hence overall less CPU utilization. > > [1] - https://github.com/ovn-org/ovn-heater/blob/main/test-scenarios/ocp-500-density-heavy.yml > > Reviewed-by: Ales Musil <amusil@redhat.com> > Acked-by: Mark Michelson <mmichels@redhat.com> > Signed-off-by: Numan Siddique <numans@ovn.org> Thanks Numan! I have only a small nit comment inlined. Acked-by: Han Zhou <hzhou@ovn.org> > --- > lib/lb.c | 46 +++++- > lib/lb.h | 9 ++ > northd/en-lb-data.c | 328 +++++++++++++++++++++++++++++++-------- > northd/en-lb-data.h | 14 ++ > northd/en-northd.c | 20 +++ > northd/en-northd.h | 1 + > northd/inc-proc-northd.c | 5 +- > northd/northd.c | 245 ++++++++++++++++++++++++++--- > northd/northd.h | 2 + > tests/ovn-northd.at | 42 ++--- > 10 files changed, 599 insertions(+), 113 deletions(-) > > diff --git a/lib/lb.c b/lib/lb.c > index e6c9dc2be2..d0d562b6fb 100644 > --- a/lib/lb.c > +++ b/lib/lb.c > @@ -794,6 +794,7 @@ ovn_lb_group_init(struct ovn_lb_group *lb_group, > const struct uuid *lb_uuid = > &nbrec_lb_group->load_balancer[i]->header_.uuid; > lb_group->lbs[i] = ovn_northd_lb_find(lbs, lb_uuid); > + lb_group->has_routable_lb |= lb_group->lbs[i]->routable; > } > } > > @@ -815,6 +816,7 @@ ovn_lb_group_cleanup(struct ovn_lb_group *lb_group) > { > ovn_lb_ip_set_destroy(lb_group->lb_ips); > lb_group->lb_ips = NULL; > + lb_group->has_routable_lb = false; > free(lb_group->lbs); > } > > @@ -1022,23 +1024,54 @@ ovn_lb_5tuples_destroy(struct hmap *tuples) > void > build_lrouter_lb_ips(struct ovn_lb_ip_set *lb_ips, > const struct ovn_northd_lb *lb) > +{ > + add_ips_to_lb_ip_set(lb_ips, lb->routable, &lb->ips_v4, &lb->ips_v6); > +} > + > +void > +add_ips_to_lb_ip_set(struct ovn_lb_ip_set *lb_ips, > + bool is_routable, > + const struct sset *lb_ips_v4, > + const struct sset *lb_ips_v6) > { > const char *ip_address; > > - SSET_FOR_EACH (ip_address, &lb->ips_v4) { > + SSET_FOR_EACH (ip_address, lb_ips_v4) { > sset_add(&lb_ips->ips_v4, ip_address); > - if (lb->routable) { > + if (is_routable) { > sset_add(&lb_ips->ips_v4_routable, ip_address); > } > } > - SSET_FOR_EACH (ip_address, &lb->ips_v6) { > + SSET_FOR_EACH (ip_address, lb_ips_v6) { > sset_add(&lb_ips->ips_v6, ip_address); > - if (lb->routable) { > + if (is_routable) { > sset_add(&lb_ips->ips_v6_routable, ip_address); > } > } > } > > +void > +remove_ips_from_lb_ip_set(struct ovn_lb_ip_set *lb_ips, > + bool is_routable, > + const struct sset *lb_ips_v4, > + const struct sset *lb_ips_v6) > +{ > + const char *ip_address; > + > + SSET_FOR_EACH (ip_address, lb_ips_v4) { > + sset_find_and_delete(&lb_ips->ips_v4, ip_address); > + if (is_routable) { > + sset_find_and_delete(&lb_ips->ips_v4_routable, ip_address); > + } > + } > + SSET_FOR_EACH (ip_address, lb_ips_v6) { > + sset_find_and_delete(&lb_ips->ips_v6, ip_address); > + if (is_routable) { > + sset_find_and_delete(&lb_ips->ips_v6_routable, ip_address); > + } > + } > +} > + > /* lb datapaths functions */ > struct ovn_lb_datapaths * > ovn_lb_datapaths_create(const struct ovn_northd_lb *lb, size_t n_ls_datapaths, > @@ -1079,7 +1112,10 @@ ovn_lb_datapaths_add_lr(struct ovn_lb_datapaths *lb_dps, size_t n, > struct ovn_datapath **ods) > { > for (size_t i = 0; i < n; i++) { > - bitmap_set1(lb_dps->nb_lr_map, ods[i]->index); > + if (!bitmap_is_set(lb_dps->nb_lr_map, ods[i]->index)) { > + bitmap_set1(lb_dps->nb_lr_map, ods[i]->index); > + lb_dps->n_nb_lr++; > + } > } > } > > diff --git a/lib/lb.h b/lib/lb.h > index 74905c73b7..b8e3c1e8fb 100644 > --- a/lib/lb.h > +++ b/lib/lb.h > @@ -138,6 +138,14 @@ void ovn_northd_lb_reinit(struct ovn_northd_lb *, > > void build_lrouter_lb_ips(struct ovn_lb_ip_set *, > const struct ovn_northd_lb *); > +void add_ips_to_lb_ip_set(struct ovn_lb_ip_set *lb_ips, > + bool is_routable, > + const struct sset *lb_ips_v4, > + const struct sset *lb_ips_v6); > +void remove_ips_from_lb_ip_set(struct ovn_lb_ip_set *lb_ips, > + bool is_routable, > + const struct sset *lb_ips_v4, > + const struct sset *lb_ips_v6); > > struct ovn_lb_group { > struct hmap_node hmap_node; > @@ -145,6 +153,7 @@ struct ovn_lb_group { > size_t n_lbs; > struct ovn_northd_lb **lbs; > struct ovn_lb_ip_set *lb_ips; > + bool has_routable_lb; > }; > > struct ovn_lb_group *ovn_lb_group_create( > diff --git a/northd/en-lb-data.c b/northd/en-lb-data.c > index fd09e719d8..250cec848b 100644 > --- a/northd/en-lb-data.c > +++ b/northd/en-lb-data.c > @@ -40,20 +40,30 @@ static void build_lbs(const struct nbrec_load_balancer_table *, > const struct nbrec_load_balancer_group_table *, > struct hmap *lbs, struct hmap *lb_groups); > static void build_od_lb_map(const struct nbrec_logical_switch_table *, > - struct hmap *od_lb_map); > + const struct nbrec_logical_router_table *, > + struct hmap *ls_lb_map, struct hmap *lr_lb_map); > static struct od_lb_data *find_od_lb_data(struct hmap *od_lb_map, > const struct uuid *od_uuid); > static void destroy_od_lb_data(struct od_lb_data *od_lb_data); > static struct od_lb_data *create_od_lb_data(struct hmap *od_lb_map, > const struct uuid *od_uuid); > +static void handle_od_lb_changes(struct nbrec_load_balancer **, > + size_t n_nbrec_lbs, > + struct od_lb_data *od_lb_data, > + struct ed_type_lb_data *lb_data, > + struct crupdated_od_lb_data *); > +static void handle_od_lbgrp_changes(struct nbrec_load_balancer_group **, > + size_t n_nbrec_lbs, > + struct od_lb_data *, > + struct ed_type_lb_data *lb_data, > + struct crupdated_od_lb_data *); > > static struct ovn_lb_group *create_lb_group( > const struct nbrec_load_balancer_group *, struct hmap *lbs, > struct hmap *lb_groups); > static void destroy_tracked_data(struct ed_type_lb_data *); > -static void add_crupdated_lb_to_tracked_data(struct ovn_northd_lb *, > - struct tracked_lb_data *, > - bool health_checks); > +static struct crupdated_lb *add_crupdated_lb_to_tracked_data( > + struct ovn_northd_lb *, struct tracked_lb_data *, bool health_checks); > static void add_deleted_lb_to_tracked_data(struct ovn_northd_lb *, > struct tracked_lb_data *, > bool health_checks); > @@ -64,6 +74,8 @@ static void add_deleted_lbgrp_to_tracked_data( > struct ovn_lb_group *, struct tracked_lb_data *); > static bool is_ls_lbs_changed(const struct nbrec_logical_switch *nbs); > static bool is_ls_lbgrps_changed(const struct nbrec_logical_switch *nbs); > +static bool is_lr_lbs_changed(const struct nbrec_logical_router *); > +static bool is_lr_lbgrps_changed(const struct nbrec_logical_router *); > > /* 'lb_data' engine node manages the NB load balancers and load balancer > * groups. For each NB LB, it creates 'struct ovn_northd_lb' and > @@ -92,10 +104,13 @@ en_lb_data_run(struct engine_node *node, void *data) > EN_OVSDB_GET(engine_get_input("NB_load_balancer_group", node)); > const struct nbrec_logical_switch_table *nb_ls_table = > EN_OVSDB_GET(engine_get_input("NB_logical_switch", node)); > + const struct nbrec_logical_router_table *nb_lr_table = > + EN_OVSDB_GET(engine_get_input("NB_logical_router", node)); > > lb_data->tracked = false; > build_lbs(nb_lb_table, nb_lbg_table, &lb_data->lbs, &lb_data->lbgrps); > - build_od_lb_map(nb_ls_table, &lb_data->ls_lb_map); > + build_od_lb_map(nb_ls_table, nb_lr_table, &lb_data->ls_lb_map, > + &lb_data->lr_lb_map); > > engine_set_node_state(node, EN_UPDATED); > } > @@ -137,6 +152,7 @@ lb_data_load_balancer_handler(struct engine_node *node, void *data) > uuid_hash(&tracked_lb->header_.uuid)); > add_crupdated_lb_to_tracked_data(lb, trk_lb_data, > lb->health_checks); > + trk_lb_data->has_routable_lb |= lb->routable; > } else if (nbrec_load_balancer_is_deleted(tracked_lb)) { > lb = ovn_northd_lb_find(&lb_data->lbs, > &tracked_lb->header_.uuid); > @@ -144,15 +160,44 @@ lb_data_load_balancer_handler(struct engine_node *node, void *data) > hmap_remove(&lb_data->lbs, &lb->hmap_node); > add_deleted_lb_to_tracked_data(lb, trk_lb_data, > lb->health_checks); > + trk_lb_data->has_routable_lb |= lb->routable; > } else { > /* Load balancer updated. */ > lb = ovn_northd_lb_find(&lb_data->lbs, > &tracked_lb->header_.uuid); > ovs_assert(lb); > bool health_checks = lb->health_checks; > + struct sset old_ips_v4 = SSET_INITIALIZER(&old_ips_v4); > + struct sset old_ips_v6 = SSET_INITIALIZER(&old_ips_v6); > + sset_swap(&lb->ips_v4, &old_ips_v4); > + sset_swap(&lb->ips_v6, &old_ips_v6); > ovn_northd_lb_reinit(lb, tracked_lb); > health_checks |= lb->health_checks; > - add_crupdated_lb_to_tracked_data(lb, trk_lb_data, health_checks); > + struct crupdated_lb *clb = add_crupdated_lb_to_tracked_data( > + lb, trk_lb_data, health_checks); > + trk_lb_data->has_routable_lb |= lb->routable; > + > + /* Determine the inserted and deleted vips and store them in > + * the tracked data. */ > + const char *vip; > + SSET_FOR_EACH (vip, &lb->ips_v4) { > + if (!sset_find_and_delete(&old_ips_v4, vip)) { > + sset_add(&clb->inserted_vips_v4, vip); > + } > + } > + > + sset_swap(&old_ips_v4, &clb->deleted_vips_v4); > + > + SSET_FOR_EACH (vip, &lb->ips_v6) { > + if (!sset_find_and_delete(&old_ips_v6, vip)) { > + sset_add(&clb->inserted_vips_v6, vip); > + } > + } > + > + sset_swap(&old_ips_v6, &clb->deleted_vips_v6); > + > + sset_destroy(&old_ips_v4); > + sset_destroy(&old_ips_v6); > } > } > > @@ -181,6 +226,8 @@ lb_data_load_balancer_group_handler(struct engine_node *node, void *data) > for (size_t i = 0; i < lb_group->n_lbs; i++) { > hmapx_add(&clbg->assoc_lbs, lb_group->lbs[i]); > } > + > + trk_lb_data->has_routable_lb |= lb_group->has_routable_lb; > } else if (nbrec_load_balancer_group_is_deleted(tracked_lb_group)) { > struct ovn_lb_group *lb_group; > lb_group = ovn_lb_group_find(&lb_data->lbgrps, > @@ -188,6 +235,7 @@ lb_data_load_balancer_group_handler(struct engine_node *node, void *data) > ovs_assert(lb_group); > hmap_remove(&lb_data->lbgrps, &lb_group->hmap_node); > add_deleted_lbgrp_to_tracked_data(lb_group, trk_lb_data); > + trk_lb_data->has_routable_lb |= lb_group->has_routable_lb; > } else { > > struct ovn_lb_group *lb_group; > @@ -209,6 +257,7 @@ lb_data_load_balancer_group_handler(struct engine_node *node, void *data) > build_lrouter_lb_ips(lb_group->lb_ips, lb_group->lbs[i]); > } > > + trk_lb_data->has_routable_lb |= lb_group->has_routable_lb; > struct crupdated_lbgrp *clbg = > add_crupdated_lbgrp_to_tracked_data(lb_group, trk_lb_data); > > @@ -270,6 +319,7 @@ lb_data_logical_switch_handler(struct engine_node *node, void *data) > if (!ls_lbs_changed && !ls_lbgrps_changed) { > continue; > } > + changed = true; > struct crupdated_od_lb_data *codlb = xzalloc(sizeof *codlb); > codlb->od_uuid = nbs->header_.uuid; > uuidset_init(&codlb->assoc_lbs); > @@ -283,66 +333,77 @@ lb_data_logical_switch_handler(struct engine_node *node, void *data) > } > > if (ls_lbs_changed) { > - struct uuidset *pre_lb_uuids = od_lb_data->lbs; > - od_lb_data->lbs = xzalloc(sizeof *od_lb_data->lbs); > - uuidset_init(od_lb_data->lbs); > - > - for (size_t i = 0; i < nbs->n_load_balancer; i++) { > - const struct uuid *lb_uuid = > - &nbs->load_balancer[i]->header_.uuid; > - uuidset_insert(od_lb_data->lbs, lb_uuid); > - > - struct uuidset_node *unode = uuidset_find(pre_lb_uuids, > - lb_uuid); > - > - if (!unode || (nbrec_load_balancer_row_get_seqno( > - nbs->load_balancer[i], > - OVSDB_IDL_CHANGE_MODIFY) > 0)) { > - /* Add this lb to the tracked data. */ > - uuidset_insert(&codlb->assoc_lbs, lb_uuid); > - changed = true; > - } > - > - if (unode) { > - uuidset_delete(pre_lb_uuids, unode); > - } > - } > - if (!uuidset_is_empty(pre_lb_uuids)) { > - trk_lb_data->has_dissassoc_lbs_from_od = true; > - changed = true; > - } > - > - uuidset_destroy(pre_lb_uuids); > - free(pre_lb_uuids); > + handle_od_lb_changes(nbs->load_balancer, nbs->n_load_balancer, > + od_lb_data, lb_data, codlb); > } > > if (ls_lbgrps_changed) { > - struct uuidset *pre_lbgrp_uuids = od_lb_data->lbgrps; > - od_lb_data->lbgrps = xzalloc(sizeof *od_lb_data->lbgrps); > - uuidset_init(od_lb_data->lbgrps); > - for (size_t i = 0; i < nbs->n_load_balancer_group; i++) { > - const struct uuid *lbg_uuid = > - &nbs->load_balancer_group[i]->header_.uuid; > - uuidset_insert(od_lb_data->lbgrps, lbg_uuid); > - > - if (!uuidset_find_and_delete(pre_lbgrp_uuids, > - lbg_uuid)) { > - /* Add this lb group to the tracked data. */ > - uuidset_insert(&codlb->assoc_lbgrps, lbg_uuid); > - changed = true; > - } > - } > + handle_od_lbgrp_changes(nbs->load_balancer_group, > + nbs->n_load_balancer_group, > + od_lb_data, lb_data, codlb); > + } > > - if (!uuidset_is_empty(pre_lbgrp_uuids)) { > - trk_lb_data->has_dissassoc_lbgrps_from_od = true; > - changed = true; > - } > + ovs_list_insert(&trk_lb_data->crupdated_ls_lbs, &codlb->list_node); > + } > + } > + > + if (changed) { > + engine_set_node_state(node, EN_UPDATED); > + } > + return true; > +} > + > +bool > +lb_data_logical_router_handler(struct engine_node *node, void *data) > +{ > + struct ed_type_lb_data *lb_data = (struct ed_type_lb_data *) data; > + const struct nbrec_logical_router_table *nbrec_lr_table = > + EN_OVSDB_GET(engine_get_input("NB_logical_router", node)); > + > + struct tracked_lb_data *trk_lb_data = &lb_data->tracked_lb_data; > + lb_data->tracked = true; > > - uuidset_destroy(pre_lbgrp_uuids); > - free(pre_lbgrp_uuids); > + bool changed = false; > + const struct nbrec_logical_router *nbr; > + NBREC_LOGICAL_ROUTER_TABLE_FOR_EACH_TRACKED (nbr, nbrec_lr_table) { > + if (nbrec_logical_router_is_deleted(nbr)) { > + struct od_lb_data *od_lb_data = > + find_od_lb_data(&lb_data->lr_lb_map, &nbr->header_.uuid); > + if (od_lb_data) { > + hmap_remove(&lb_data->lr_lb_map, &od_lb_data->hmap_node); > + hmapx_add(&trk_lb_data->deleted_od_lb_data, od_lb_data); > + } > + } else { > + bool lr_lbs_changed = is_lr_lbs_changed(nbr); > + bool lr_lbgrps_changed = is_lr_lbgrps_changed(nbr); > + if (!lr_lbs_changed && !lr_lbgrps_changed) { > + continue; > } > + changed = true; > + struct crupdated_od_lb_data *codlb = xzalloc(sizeof *codlb); > + codlb->od_uuid = nbr->header_.uuid; > + uuidset_init(&codlb->assoc_lbs); > + uuidset_init(&codlb->assoc_lbgrps); > > - ovs_list_insert(&trk_lb_data->crupdated_ls_lbs, &codlb->list_node); > + struct od_lb_data *od_lb_data = > + find_od_lb_data(&lb_data->lr_lb_map, &nbr->header_.uuid); > + if (!od_lb_data) { > + od_lb_data = create_od_lb_data(&lb_data->lr_lb_map, > + &nbr->header_.uuid); > + } > + > + if (lr_lbs_changed) { > + handle_od_lb_changes(nbr->load_balancer, nbr->n_load_balancer, > + od_lb_data, lb_data, codlb); > + } > + > + if (lr_lbgrps_changed) { > + handle_od_lbgrp_changes(nbr->load_balancer_group, > + nbr->n_load_balancer_group, > + od_lb_data, lb_data, codlb); > + } > + > + ovs_list_insert(&trk_lb_data->crupdated_lr_lbs, &codlb->list_node); > } > } > > @@ -359,6 +420,7 @@ lb_data_init(struct ed_type_lb_data *lb_data) > hmap_init(&lb_data->lbs); > hmap_init(&lb_data->lbgrps); > hmap_init(&lb_data->ls_lb_map); > + hmap_init(&lb_data->lr_lb_map); > > struct tracked_lb_data *trk_lb_data = &lb_data->tracked_lb_data; > hmap_init(&trk_lb_data->crupdated_lbs); > @@ -366,6 +428,7 @@ lb_data_init(struct ed_type_lb_data *lb_data) > hmap_init(&trk_lb_data->crupdated_lbgrps); > hmapx_init(&trk_lb_data->deleted_lbgrps); > ovs_list_init(&trk_lb_data->crupdated_ls_lbs); > + ovs_list_init(&trk_lb_data->crupdated_lr_lbs); > hmapx_init(&trk_lb_data->deleted_od_lb_data); > } > > @@ -390,6 +453,11 @@ lb_data_destroy(struct ed_type_lb_data *lb_data) > } > hmap_destroy(&lb_data->ls_lb_map); > > + HMAP_FOR_EACH_POP (od_lb_data, hmap_node, &lb_data->lr_lb_map) { > + destroy_od_lb_data(od_lb_data); > + } > + hmap_destroy(&lb_data->lr_lb_map); > + > destroy_tracked_data(lb_data); > hmap_destroy(&lb_data->tracked_lb_data.crupdated_lbs); > hmapx_destroy(&lb_data->tracked_lb_data.deleted_lbs); > @@ -435,7 +503,8 @@ create_lb_group(const struct nbrec_load_balancer_group *nbrec_lb_group, > > static void > build_od_lb_map(const struct nbrec_logical_switch_table *nbrec_ls_table, > - struct hmap *od_lb_map) > + const struct nbrec_logical_router_table *nbrec_lr_table, > + struct hmap *ls_lb_map, struct hmap *lr_lb_map) > { > const struct nbrec_logical_switch *nbrec_ls; > NBREC_LOGICAL_SWITCH_TABLE_FOR_EACH (nbrec_ls, nbrec_ls_table) { > @@ -443,17 +512,35 @@ build_od_lb_map(const struct nbrec_logical_switch_table *nbrec_ls_table, > continue; > } > > - struct od_lb_data *od_lb_data = > - create_od_lb_data(od_lb_map, &nbrec_ls->header_.uuid); > + struct od_lb_data *ls_lb_data = > + create_od_lb_data(ls_lb_map, &nbrec_ls->header_.uuid); > for (size_t i = 0; i < nbrec_ls->n_load_balancer; i++) { > - uuidset_insert(od_lb_data->lbs, > + uuidset_insert(ls_lb_data->lbs, > &nbrec_ls->load_balancer[i]->header_.uuid); > } > for (size_t i = 0; i < nbrec_ls->n_load_balancer_group; i++) { > - uuidset_insert(od_lb_data->lbgrps, > + uuidset_insert(ls_lb_data->lbgrps, > &nbrec_ls->load_balancer_group[i]->header_.uuid); > } > } > + > + const struct nbrec_logical_router *nbrec_lr; > + NBREC_LOGICAL_ROUTER_TABLE_FOR_EACH (nbrec_lr, nbrec_lr_table) { > + if (!nbrec_lr->n_load_balancer && !nbrec_lr->n_load_balancer_group) { > + continue; > + } > + > + struct od_lb_data *lr_lb_data = > + create_od_lb_data(lr_lb_map, &nbrec_lr->header_.uuid); > + for (size_t i = 0; i < nbrec_lr->n_load_balancer; i++) { > + uuidset_insert(lr_lb_data->lbs, > + &nbrec_lr->load_balancer[i]->header_.uuid); > + } > + for (size_t i = 0; i < nbrec_lr->n_load_balancer_group; i++) { > + uuidset_insert(lr_lb_data->lbgrps, > + &nbrec_lr->load_balancer_group[i]->header_.uuid); > + } > + } > } > > static struct od_lb_data * > @@ -495,6 +582,84 @@ destroy_od_lb_data(struct od_lb_data *od_lb_data) > free(od_lb_data); > } > > +static void > +handle_od_lb_changes(struct nbrec_load_balancer **nbrec_lbs, > + size_t n_nbrec_lbs, struct od_lb_data *od_lb_data, > + struct ed_type_lb_data *lb_data, > + struct crupdated_od_lb_data *codlb) > +{ > + struct tracked_lb_data *trk_lb_data = &lb_data->tracked_lb_data; > + struct uuidset *pre_lb_uuids = od_lb_data->lbs; > + od_lb_data->lbs = xzalloc(sizeof *od_lb_data->lbs); > + uuidset_init(od_lb_data->lbs); > + > + for (size_t i = 0; i < n_nbrec_lbs; i++) { > + const struct uuid *lb_uuid = &nbrec_lbs[i]->header_.uuid; > + uuidset_insert(od_lb_data->lbs, lb_uuid); > + > + struct uuidset_node *unode = uuidset_find(pre_lb_uuids, lb_uuid); > + > + if (!unode || (nbrec_load_balancer_row_get_seqno( > + nbrec_lbs[i], OVSDB_IDL_CHANGE_MODIFY) > 0)) { > + /* Add this lb to the tracked data. */ > + uuidset_insert(&codlb->assoc_lbs, lb_uuid); > + > + if (!trk_lb_data->has_routable_lb) { > + struct ovn_northd_lb *lb = ovn_northd_lb_find(&lb_data->lbs, > + lb_uuid); > + ovs_assert(lb); > + trk_lb_data->has_routable_lb |= lb->routable; > + } > + } > + > + if (unode) { > + uuidset_delete(pre_lb_uuids, unode); > + } > + } > + > + if (!uuidset_is_empty(pre_lb_uuids)) { > + trk_lb_data->has_dissassoc_lbs_from_od = true; > + } > + > + uuidset_destroy(pre_lb_uuids); > + free(pre_lb_uuids); > +} > + > +static void > +handle_od_lbgrp_changes(struct nbrec_load_balancer_group **nbrec_lbgrps, > + size_t n_nbrec_lbgrps, struct od_lb_data *od_lb_data, > + struct ed_type_lb_data *lb_data, > + struct crupdated_od_lb_data *codlb) > +{ > + struct tracked_lb_data *trk_lb_data = &lb_data->tracked_lb_data; > + struct uuidset *pre_lbgrp_uuids = od_lb_data->lbgrps; > + od_lb_data->lbgrps = xzalloc(sizeof *od_lb_data->lbgrps); > + uuidset_init(od_lb_data->lbgrps); > + for (size_t i = 0; i < n_nbrec_lbgrps; i++) { > + const struct uuid *lbgrp_uuid = &nbrec_lbgrps[i]->header_.uuid; > + uuidset_insert(od_lb_data->lbgrps, lbgrp_uuid); > + > + if (!uuidset_find_and_delete(pre_lbgrp_uuids, lbgrp_uuid)) { > + /* Add this lb group to the tracked data. */ > + uuidset_insert(&codlb->assoc_lbgrps, lbgrp_uuid); > + > + if (!trk_lb_data->has_routable_lb) { > + struct ovn_lb_group *lbgrp = > + ovn_lb_group_find(&lb_data->lbgrps, lbgrp_uuid); > + ovs_assert(lbgrp); > + trk_lb_data->has_routable_lb |= lbgrp->has_routable_lb; > + } > + } > + } > + > + if (!uuidset_is_empty(pre_lbgrp_uuids)) { > + trk_lb_data->has_dissassoc_lbgrps_from_od = true; > + } > + > + uuidset_destroy(pre_lbgrp_uuids); > + free(pre_lbgrp_uuids); > +} > + > static void > destroy_tracked_data(struct ed_type_lb_data *lb_data) > { > @@ -503,6 +668,7 @@ destroy_tracked_data(struct ed_type_lb_data *lb_data) > lb_data->tracked_lb_data.has_dissassoc_lbs_from_lbgrps = false; > lb_data->tracked_lb_data.has_dissassoc_lbs_from_od = false; > lb_data->tracked_lb_data.has_dissassoc_lbgrps_from_od = false; > + lb_data->tracked_lb_data.has_routable_lb = false; > > struct hmapx_node *node; > HMAPX_FOR_EACH_SAFE (node, &lb_data->tracked_lb_data.deleted_lbs) { > @@ -518,6 +684,10 @@ destroy_tracked_data(struct ed_type_lb_data *lb_data) > struct crupdated_lb *clb; > HMAP_FOR_EACH_POP (clb, hmap_node, > &lb_data->tracked_lb_data.crupdated_lbs) { > + sset_destroy(&clb->inserted_vips_v4); > + sset_destroy(&clb->inserted_vips_v6); > + sset_destroy(&clb->deleted_vips_v4); > + sset_destroy(&clb->deleted_vips_v6); > free(clb); > } > > @@ -537,13 +707,21 @@ destroy_tracked_data(struct ed_type_lb_data *lb_data) > free(codlb); > } > > + LIST_FOR_EACH_SAFE (codlb, list_node, > + &lb_data->tracked_lb_data.crupdated_lr_lbs) { > + ovs_list_remove(&codlb->list_node); > + uuidset_destroy(&codlb->assoc_lbs); > + uuidset_destroy(&codlb->assoc_lbgrps); > + free(codlb); > + } > + > HMAPX_FOR_EACH_SAFE (node, &lb_data->tracked_lb_data.deleted_od_lb_data) { > destroy_od_lb_data(node->data); > hmapx_delete(&lb_data->tracked_lb_data.deleted_od_lb_data, node); > } > } > > -static void > +static struct crupdated_lb * > add_crupdated_lb_to_tracked_data(struct ovn_northd_lb *lb, > struct tracked_lb_data *tracked_lb_data, > bool health_checks) > @@ -552,9 +730,15 @@ add_crupdated_lb_to_tracked_data(struct ovn_northd_lb *lb, > clb->lb = lb; > hmap_insert(&tracked_lb_data->crupdated_lbs, &clb->hmap_node, > uuid_hash(&lb->nlb->header_.uuid)); > + sset_init(&clb->inserted_vips_v4); > + sset_init(&clb->inserted_vips_v6); > + sset_init(&clb->deleted_vips_v4); > + sset_init(&clb->deleted_vips_v6); > if (health_checks) { > tracked_lb_data->has_health_checks = true; > } > + > + return clb; > } > > static void > @@ -600,3 +784,17 @@ is_ls_lbgrps_changed(const struct nbrec_logical_switch *nbs) { > || nbrec_logical_switch_is_updated(nbs, > NBREC_LOGICAL_SWITCH_COL_LOAD_BALANCER_GROUP)); > } > + > +static bool > +is_lr_lbs_changed(const struct nbrec_logical_router *nbr) { > + return ((nbrec_logical_router_is_new(nbr) && nbr->n_load_balancer) > + || nbrec_logical_router_is_updated(nbr, > + NBREC_LOGICAL_ROUTER_COL_LOAD_BALANCER)); > +} > + > +static bool > +is_lr_lbgrps_changed(const struct nbrec_logical_router *nbr) { > + return ((nbrec_logical_router_is_new(nbr) && nbr->n_load_balancer_group) > + || nbrec_logical_router_is_updated(nbr, > + NBREC_LOGICAL_ROUTER_COL_LOAD_BALANCER_GROUP)); > +} > diff --git a/northd/en-lb-data.h b/northd/en-lb-data.h > index ebfd547d77..9d20a539ba 100644 > --- a/northd/en-lb-data.h > +++ b/northd/en-lb-data.h > @@ -6,6 +6,7 @@ > #include "openvswitch/hmap.h" > #include "include/openvswitch/list.h" > #include "lib/hmapx.h" > +#include "lib/sset.h" > #include "lib/uuidset.h" > > #include "lib/inc-proc-eng.h" > @@ -17,6 +18,10 @@ struct crupdated_lb { > struct hmap_node hmap_node; > > struct ovn_northd_lb *lb; > + struct sset inserted_vips_v4; > + struct sset inserted_vips_v6; > + struct sset deleted_vips_v4; > + struct sset deleted_vips_v6; > }; > > struct crupdated_lbgrp { > @@ -54,6 +59,10 @@ struct tracked_lb_data { > * 'struct crupdated_od_lb_data' */ > struct ovs_list crupdated_ls_lbs; > > + /* List of logical router <-> lb changes. List node is > + * 'struct crupdated_od_lb_data' */ > + struct ovs_list crupdated_lr_lbs; > + > /* hmapx of deleted logical switches which have load balancer or lb groups > * associated with it. hmapx_node is 'struct od_lb_data'. */ > struct hmapx deleted_od_lb_data; > @@ -70,6 +79,9 @@ struct tracked_lb_data { > > /* Indicates if a lb group was disassociated from a logical switch. */ > bool has_dissassoc_lbgrps_from_od; > + > + /* Indicates if any lb (in the tracked data) has 'routable' flag set. */ > + bool has_routable_lb; > }; > > /* Datapath (logical switch) to lb/lbgrp association data. */ > @@ -90,6 +102,7 @@ struct ed_type_lb_data { > > /* hmap of ls to lb map. hmap node is 'struct od_lb_data'. */ > struct hmap ls_lb_map; > + struct hmap lr_lb_map; > > /* tracked data*/ > bool tracked; > @@ -104,5 +117,6 @@ void en_lb_data_clear_tracked_data(void *data); > bool lb_data_load_balancer_handler(struct engine_node *, void *data); > bool lb_data_load_balancer_group_handler(struct engine_node *, void *data); > bool lb_data_logical_switch_handler(struct engine_node *, void *data); > +bool lb_data_logical_router_handler(struct engine_node *, void *data); > > #endif /* end of EN_NORTHD_LB_DATA_H */ > diff --git a/northd/en-northd.c b/northd/en-northd.c > index 8487b003f7..aa0f20f0c2 100644 > --- a/northd/en-northd.c > +++ b/northd/en-northd.c > @@ -196,6 +196,26 @@ northd_sb_port_binding_handler(struct engine_node *node, > return true; > } > > +bool > +northd_nb_logical_router_handler(struct engine_node *node, > + void *data) > +{ > + struct northd_data *nd = data; > + struct northd_input input_data; > + > + northd_get_input_data(node, &input_data); > + > + if (!northd_handle_lr_changes(&input_data, nd)) { > + return false; > + } > + > + if (nd->change_tracked) { > + engine_set_node_state(node, EN_UPDATED); > + } > + > + return true; > +} > + > bool > northd_lb_data_handler(struct engine_node *node, void *data) > { > diff --git a/northd/en-northd.h b/northd/en-northd.h > index 3c77b64bb2..5a88871760 100644 > --- a/northd/en-northd.h > +++ b/northd/en-northd.h > @@ -16,6 +16,7 @@ void en_northd_cleanup(void *data); > void en_northd_clear_tracked_data(void *data); > bool northd_nb_nb_global_handler(struct engine_node *, void *data OVS_UNUSED); > bool northd_nb_logical_switch_handler(struct engine_node *, void *data); > +bool northd_nb_logical_router_handler(struct engine_node *, void *data); > bool northd_sb_port_binding_handler(struct engine_node *, void *data); > bool northd_lb_data_handler(struct engine_node *, void *data); > > diff --git a/northd/inc-proc-northd.c b/northd/inc-proc-northd.c > index e9e28c4bea..8b08171179 100644 > --- a/northd/inc-proc-northd.c > +++ b/northd/inc-proc-northd.c > @@ -158,8 +158,9 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, > lb_data_load_balancer_group_handler); > engine_add_input(&en_lb_data, &en_nb_logical_switch, > lb_data_logical_switch_handler); > + engine_add_input(&en_lb_data, &en_nb_logical_router, > + lb_data_logical_router_handler); > > - engine_add_input(&en_northd, &en_nb_logical_router, NULL); > engine_add_input(&en_northd, &en_nb_mirror, NULL); > engine_add_input(&en_northd, &en_nb_static_mac_binding, NULL); > engine_add_input(&en_northd, &en_nb_chassis_template_var, NULL); > @@ -184,6 +185,8 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, > northd_nb_nb_global_handler); > engine_add_input(&en_northd, &en_nb_logical_switch, > northd_nb_logical_switch_handler); > + engine_add_input(&en_northd, &en_nb_logical_router, > + northd_nb_logical_router_handler); > engine_add_input(&en_northd, &en_lb_data, northd_lb_data_handler); > > engine_add_input(&en_mac_binding_aging, &en_nb_nb_global, NULL); > diff --git a/northd/northd.c b/northd/northd.c > index f6258d4830..23d11d9d97 100644 > --- a/northd/northd.c > +++ b/northd/northd.c > @@ -4167,54 +4167,63 @@ static bool lrouter_port_ipv4_reachable(const struct ovn_port *op, > ovs_be32 addr); > static bool lrouter_port_ipv6_reachable(const struct ovn_port *op, > const struct in6_addr *addr); > + > static void > -build_lrouter_lb_reachable_ips(struct ovn_datapath *od, > - const struct ovn_northd_lb *lb) > +add_neigh_ips_to_lrouter(struct ovn_datapath *od, > + enum lb_neighbor_responder_mode neigh_mode, > + const struct sset *lb_ips_v4, > + const struct sset *lb_ips_v6) > { > /* If configured to not reply to any neighbor requests for all VIPs > * return early. > */ > - if (lb->neigh_mode == LB_NEIGH_RESPOND_NONE) { > + if (neigh_mode == LB_NEIGH_RESPOND_NONE) { > return; > } > > + const char *ip_address; > + > /* If configured to reply to neighbor requests for all VIPs force them > * all to be considered "reachable". > */ > - if (lb->neigh_mode == LB_NEIGH_RESPOND_ALL) { > - for (size_t i = 0; i < lb->n_vips; i++) { > - if (lb->vips[i].address_family == AF_INET) { > - sset_add(&od->lb_ips->ips_v4_reachable, lb->vips[i].vip_str); > - } else { > - sset_add(&od->lb_ips->ips_v6_reachable, lb->vips[i].vip_str); > - } > + if (neigh_mode == LB_NEIGH_RESPOND_ALL) { > + SSET_FOR_EACH (ip_address, lb_ips_v4) { > + sset_add(&od->lb_ips->ips_v4_reachable, ip_address); > + } > + SSET_FOR_EACH (ip_address, lb_ips_v6) { > + sset_add(&od->lb_ips->ips_v6_reachable, ip_address); > } > + > return; > } > > /* Otherwise, a VIP is reachable if there's at least one router > * subnet that includes it. > */ > - ovs_assert(lb->neigh_mode == LB_NEIGH_RESPOND_REACHABLE); > - for (size_t i = 0; i < lb->n_vips; i++) { > - if (lb->vips[i].address_family == AF_INET) { > - ovs_be32 vip_ip4 = in6_addr_get_mapped_ipv4(&lb->vips[i].vip); > - struct ovn_port *op; > + ovs_assert(neigh_mode == LB_NEIGH_RESPOND_REACHABLE); > > + SSET_FOR_EACH (ip_address, lb_ips_v4) { > + struct ovn_port *op; > + ovs_be32 vip_ip4; > + if (ip_parse(ip_address, &vip_ip4)) { > HMAP_FOR_EACH (op, dp_node, &od->ports) { > if (lrouter_port_ipv4_reachable(op, vip_ip4)) { > sset_add(&od->lb_ips->ips_v4_reachable, > - lb->vips[i].vip_str); > + ip_address); > break; > } > } > - } else { > - struct ovn_port *op; > + } > + } > > + SSET_FOR_EACH (ip_address, lb_ips_v6) { > + struct ovn_port *op; > + struct in6_addr vip; > + if (ipv6_parse(ip_address, &vip)) { > HMAP_FOR_EACH (op, dp_node, &od->ports) { > - if (lrouter_port_ipv6_reachable(op, &lb->vips[i].vip)) { > + if (lrouter_port_ipv6_reachable(op, &vip)) { > sset_add(&od->lb_ips->ips_v6_reachable, > - lb->vips[i].vip_str); > + ip_address); > break; > } > } > @@ -4222,6 +4231,34 @@ build_lrouter_lb_reachable_ips(struct ovn_datapath *od, > } > } > > +static void > +remove_lrouter_lb_reachable_ips(struct ovn_datapath *od, > + enum lb_neighbor_responder_mode neigh_mode, > + const struct sset *lb_ips_v4, > + const struct sset *lb_ips_v6) > +{ > + if (neigh_mode == LB_NEIGH_RESPOND_NONE) { > + return; > + } > + > + const char *ip_address; > + SSET_FOR_EACH (ip_address, lb_ips_v4) { > + sset_find_and_delete(&od->lb_ips->ips_v4_reachable, ip_address); > + } > + SSET_FOR_EACH (ip_address, lb_ips_v6) { > + sset_find_and_delete(&od->lb_ips->ips_v6_reachable, ip_address); > + } > +} > + > +static void > +build_lrouter_lb_reachable_ips(struct ovn_datapath *od, > + const struct ovn_northd_lb *lb) > +{ > + add_neigh_ips_to_lrouter(od, lb->neigh_mode, &lb->ips_v4, > + &lb->ips_v6); > +} > + > + > static void > build_lrouter_lbs_check(const struct ovn_datapaths *lr_datapaths) > { > @@ -5419,6 +5456,95 @@ fail: > return false; > } > > +/* Returns true if the logical router has changes which are not > + * incrementally handled. > + * Presently supports i-p for the below changes: > + * - load balancers and load balancer groups. > + */ > +static bool > +check_unsupported_inc_proc_for_lr_changes( nit: It would be better to be aligned with the naming convention of ls_changes_can_be_handled(), something like: lr_changes_can_be_handled(). > + const struct nbrec_logical_router *lr) > +{ > + /* Check if the columns are changed in this row. */ > + enum nbrec_logical_router_column_id col; > + for (col = 0; col < NBREC_LOGICAL_ROUTER_N_COLUMNS; col++) { > + if (nbrec_logical_router_is_updated(lr, col)) { > + if (col == NBREC_LOGICAL_ROUTER_COL_LOAD_BALANCER || > + col == NBREC_LOGICAL_ROUTER_COL_LOAD_BALANCER_GROUP) { > + continue; > + } > + return true; > + } > + } > + > + /* Check if the referenced rows are changed. > + XXX: Need a better OVSDB IDL interface for this check. */ > + for (size_t i = 0; i < lr->n_ports; i++) { > + if (nbrec_logical_router_port_row_get_seqno(lr->ports[i], > + OVSDB_IDL_CHANGE_MODIFY) > 0) { > + return true; > + } > + } > + if (lr->copp && nbrec_copp_row_get_seqno(lr->copp, > + OVSDB_IDL_CHANGE_MODIFY) > 0) { > + return true; > + } > + for (size_t i = 0; i < lr->n_nat; i++) { > + if (nbrec_nat_row_get_seqno(lr->nat[i], > + OVSDB_IDL_CHANGE_MODIFY) > 0) { > + return true; > + } > + } > + for (size_t i = 0; i < lr->n_policies; i++) { > + if (nbrec_logical_router_policy_row_get_seqno(lr->policies[i], > + OVSDB_IDL_CHANGE_MODIFY) > 0) { > + return true; > + } > + } > + for (size_t i = 0; i < lr->n_static_routes; i++) { > + if (nbrec_logical_router_static_route_row_get_seqno( > + lr->static_routes[i], OVSDB_IDL_CHANGE_MODIFY) > 0) { > + return true; > + } > + } > + return false; > +} > + > +/* Return true if changes are handled incrementally, false otherwise. > + * When there are any changes, try to track what's exactly changed and set > + * northd_data->change_tracked accordingly: change tracked - true, otherwise, > + * false. > + * Note: Changes to load balancer and load balancer groups associated with > + * the logical routers are handled separately in the lb_data change > + * handlers (northd_handle_lb_data_changes_pre_od and > + * northd_handle_lb_data_changes_post_od). > + * */ > +bool > +northd_handle_lr_changes(const struct northd_input *ni, > + struct northd_data *nd) > +{ > + const struct nbrec_logical_router *changed_lr; > + > + NBREC_LOGICAL_ROUTER_TABLE_FOR_EACH_TRACKED (changed_lr, > + ni->nbrec_logical_router_table) { > + if (nbrec_logical_router_is_new(changed_lr) || > + nbrec_logical_router_is_deleted(changed_lr)) { > + goto fail; > + } > + > + /* Presently only able to handle load balancer and > + * load balancer group changes. */ > + if (check_unsupported_inc_proc_for_lr_changes(changed_lr)) { > + goto fail; > + } > + } > + > + return true; > +fail: > + destroy_northd_data_tracked_changes(nd); > + return false; > +} > + > bool > northd_handle_sb_port_binding_changes( > const struct sbrec_port_binding_table *sbrec_port_binding_table, > @@ -5535,6 +5661,10 @@ northd_handle_lb_data_changes(struct tracked_lb_data *trk_lb_data, > return false; > } > > + if (trk_lb_data->has_routable_lb) { > + return false; > + } > + > struct ovn_lb_datapaths *lb_dps; > struct ovn_northd_lb *lb; > struct ovn_datapath *od; > @@ -5625,6 +5755,45 @@ northd_handle_lb_data_changes(struct tracked_lb_data *trk_lb_data, > init_lb_for_datapath(od); > } > > + LIST_FOR_EACH (codlb, list_node, &trk_lb_data->crupdated_lr_lbs) { > + od = ovn_datapath_find(&lr_datapaths->datapaths, &codlb->od_uuid); > + ovs_assert(od); > + > + struct uuidset_node *uuidnode; > + UUIDSET_FOR_EACH (uuidnode, &codlb->assoc_lbs) { > + lb_dps = ovn_lb_datapaths_find(lb_datapaths_map, &uuidnode->uuid); > + ovs_assert(lb_dps); > + ovn_lb_datapaths_add_lr(lb_dps, 1, &od); > + > + /* Add the lb_ips of lb_dps to the od. */ > + build_lrouter_lb_ips(od->lb_ips, lb_dps->lb); > + build_lrouter_lb_reachable_ips(od, lb_dps->lb); > + } > + > + UUIDSET_FOR_EACH (uuidnode, &codlb->assoc_lbgrps) { > + lbgrp_dps = ovn_lb_group_datapaths_find(lbgrp_datapaths_map, > + &uuidnode->uuid); > + ovs_assert(lbgrp_dps); > + ovn_lb_group_datapaths_add_lr(lbgrp_dps, od); > + > + /* Associate all the lbs of the lbgrp to the datapath 'od' */ > + for (size_t j = 0; j < lbgrp_dps->lb_group->n_lbs; j++) { > + const struct uuid *lb_uuid > + = &lbgrp_dps->lb_group->lbs[j]->nlb->header_.uuid; > + lb_dps = ovn_lb_datapaths_find(lb_datapaths_map, lb_uuid); > + ovs_assert(lb_dps); > + ovn_lb_datapaths_add_lr(lb_dps, 1, &od); > + > + /* Add the lb_ips of lb_dps to the od. */ > + build_lrouter_lb_ips(od->lb_ips, lb_dps->lb); > + build_lrouter_lb_reachable_ips(od, lb_dps->lb); > + } > + } > + > + /* Re-evaluate 'od->has_lb_vip' */ > + init_lb_for_datapath(od); > + } > + > HMAP_FOR_EACH (clb, hmap_node, &trk_lb_data->crupdated_lbs) { > lb = clb->lb; > const struct uuid *lb_uuid = &lb->nlb->header_.uuid; > @@ -5638,6 +5807,29 @@ northd_handle_lb_data_changes(struct tracked_lb_data *trk_lb_data, > /* Re-evaluate 'od->has_lb_vip' */ > init_lb_for_datapath(od); > } > + > + BITMAP_FOR_EACH_1 (index, ods_size(lr_datapaths), > + lb_dps->nb_lr_map) { > + od = lr_datapaths->array[index]; > + /* Re-evaluate 'od->has_lb_vip' */ > + init_lb_for_datapath(od); > + > + /* Update the od->lb_ips with the deleted and inserted > + * vips (if any). */ > + remove_ips_from_lb_ip_set(od->lb_ips, lb->routable, > + &clb->deleted_vips_v4, > + &clb->deleted_vips_v6); > + add_ips_to_lb_ip_set(od->lb_ips, lb->routable, > + &clb->inserted_vips_v4, > + &clb->inserted_vips_v6); > + > + remove_lrouter_lb_reachable_ips(od, lb->neigh_mode, > + &clb->deleted_vips_v4, > + &clb->deleted_vips_v6); > + add_neigh_ips_to_lrouter(od, lb->neigh_mode, > + &clb->inserted_vips_v4, > + &clb->inserted_vips_v6); > + } > } > > HMAP_FOR_EACH (crupdated_lbgrp, hmap_node, > @@ -5655,8 +5847,19 @@ northd_handle_lb_data_changes(struct tracked_lb_data *trk_lb_data, > lb_uuid = &lb->nlb->header_.uuid; > lb_dps = ovn_lb_datapaths_find(lb_datapaths_map, lb_uuid); > ovs_assert(lb_dps); > + for (size_t i = 0; i < lbgrp_dps->n_lr; i++) { > + od = lbgrp_dps->lr[i]; > + ovn_lb_datapaths_add_lr(lb_dps, 1, &od); > + > + /* Re-evaluate 'od->has_lb_vip' */ > + init_lb_for_datapath(od); > + > + /* Add the lb_ips of lb_dps to the od. */ > + build_lrouter_lb_ips(od->lb_ips, lb_dps->lb); > + } > + > for (size_t i = 0; i < lbgrp_dps->n_ls; i++) { > - od = lbgrp_dps->ls[i]; > + od = lbgrp_dps->ls[i]; > ovn_lb_datapaths_add_ls(lb_dps, 1, &od); > > /* Re-evaluate 'od->has_lb_vip' */ > diff --git a/northd/northd.h b/northd/northd.h > index 5d8ac6fea0..4d030b10da 100644 > --- a/northd/northd.h > +++ b/northd/northd.h > @@ -328,6 +328,8 @@ void ovnsb_db_run(struct ovsdb_idl_txn *ovnnb_txn, > bool northd_handle_ls_changes(struct ovsdb_idl_txn *, > const struct northd_input *, > struct northd_data *); > +bool northd_handle_lr_changes(const struct northd_input *, > + struct northd_data *); > void destroy_northd_data_tracked_changes(struct northd_data *); > void northd_destroy(struct northd_data *data); > void northd_init(struct northd_data *data); > diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at > index 1f8b264bde..399b2d91cf 100644 > --- a/tests/ovn-northd.at > +++ b/tests/ovn-northd.at > @@ -10487,8 +10487,8 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE > # Add lb1 to lr0 and then disassociate > check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats > check ovn-nbctl --wait=sb lr-lb-add lr0 lb1 > -check_engine_stats lb_data norecompute nocompute > -check_engine_stats northd recompute nocompute > +check_engine_stats lb_data norecompute compute > +check_engine_stats northd norecompute compute > check_engine_stats lflow recompute nocompute > CHECK_NO_CHANGE_AFTER_RECOMPUTE > > @@ -10496,7 +10496,7 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE > check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats > check ovn-nbctl --wait=sb set load_balancer lb1 vips:'"10.0.0.10:80"'='" 10.0.0.100:80"' > check_engine_stats lb_data norecompute compute > -check_engine_stats northd recompute nocompute > +check_engine_stats northd norecompute compute > check_engine_stats lflow recompute nocompute > CHECK_NO_CHANGE_AFTER_RECOMPUTE > > @@ -10504,7 +10504,7 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE > check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats > check ovn-nbctl --wait=sb clear load_Balancer lb1 vips > check_engine_stats lb_data norecompute compute > -check_engine_stats northd recompute nocompute > +check_engine_stats northd norecompute compute > check_engine_stats lflow recompute nocompute > CHECK_NO_CHANGE_AFTER_RECOMPUTE > > @@ -10512,7 +10512,7 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE > check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats > check ovn-nbctl --wait=sb lb-add lb1 10.0.0.10:80 10.0.0.3:80 > check_engine_stats lb_data norecompute compute > -check_engine_stats northd recompute nocompute > +check_engine_stats northd norecompute compute > check_engine_stats lflow recompute nocompute > CHECK_NO_CHANGE_AFTER_RECOMPUTE > > @@ -10520,13 +10520,13 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE > check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats > check ovn-nbctl --wait=sb lb-add lb1 10.0.0.20:80 10.0.0.30:8080 > check_engine_stats lb_data norecompute compute > -check_engine_stats northd recompute nocompute > +check_engine_stats northd norecompute compute > check_engine_stats lflow recompute nocompute > CHECK_NO_CHANGE_AFTER_RECOMPUTE > > check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats > check ovn-nbctl --wait=sb lr-lb-del lr0 lb1 > -check_engine_stats lb_data norecompute nocompute > +check_engine_stats lb_data norecompute compute > check_engine_stats northd recompute nocompute > check_engine_stats lflow recompute nocompute > CHECK_NO_CHANGE_AFTER_RECOMPUTE > @@ -10618,8 +10618,8 @@ check_engine_stats lflow recompute nocompute > > check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats > check ovn-nbctl add logical_router lr0 load_balancer_group $lbg1_uuid > -check_engine_stats lb_data norecompute nocompute > -check_engine_stats northd recompute nocompute > +check_engine_stats lb_data norecompute compute > +check_engine_stats northd norecompute compute > check_engine_stats lflow recompute nocompute > CHECK_NO_CHANGE_AFTER_RECOMPUTE > > @@ -10627,7 +10627,7 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE > check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats > check ovn-nbctl --wait=sb set load_balancer lb1 vips:'"10.0.0.10:80"'='" 10.0.0.100:80"' > check_engine_stats lb_data norecompute compute > -check_engine_stats northd recompute nocompute > +check_engine_stats northd norecompute compute > check_engine_stats lflow recompute nocompute > CHECK_NO_CHANGE_AFTER_RECOMPUTE > > @@ -10635,7 +10635,7 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE > check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats > check ovn-nbctl --wait=sb clear load_Balancer lb1 vips > check_engine_stats lb_data norecompute compute > -check_engine_stats northd recompute nocompute > +check_engine_stats northd norecompute compute > check_engine_stats lflow recompute nocompute > CHECK_NO_CHANGE_AFTER_RECOMPUTE > > @@ -10643,7 +10643,7 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE > check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats > check ovn-nbctl --wait=sb lb-add lb1 10.0.0.10:80 10.0.0.3:80 > check_engine_stats lb_data norecompute compute > -check_engine_stats northd recompute nocompute > +check_engine_stats northd norecompute compute > check_engine_stats lflow recompute nocompute > CHECK_NO_CHANGE_AFTER_RECOMPUTE > > @@ -10651,13 +10651,13 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE > check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats > check ovn-nbctl --wait=sb lb-add lb1 10.0.0.20:80 10.0.0.30:8080 > check_engine_stats lb_data norecompute compute > -check_engine_stats northd recompute nocompute > +check_engine_stats northd norecompute compute > check_engine_stats lflow recompute nocompute > CHECK_NO_CHANGE_AFTER_RECOMPUTE > > check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats > check ovn-nbctl clear logical_router lr0 load_balancer_group > -check_engine_stats lb_data norecompute nocompute > +check_engine_stats lb_data norecompute compute > check_engine_stats northd recompute nocompute > check_engine_stats lflow recompute nocompute > > @@ -10712,8 +10712,8 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE > > check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats > check ovn-nbctl set logical_router lr1 load_balancer_group=$lbg1_uuid > -check_engine_stats lb_data norecompute nocompute > -check_engine_stats northd recompute nocompute > +check_engine_stats lb_data norecompute compute > +check_engine_stats northd norecompute compute > check_engine_stats lflow recompute nocompute > CHECK_NO_CHANGE_AFTER_RECOMPUTE > > @@ -10734,8 +10734,8 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE > check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats > check ovn-nbctl --wait=sb lr-lb-add lr1 lb1 > check ovn-nbctl --wait=sb lr-lb-add lr1 lb2 > -check_engine_stats lb_data norecompute nocompute > -check_engine_stats northd recompute nocompute > +check_engine_stats lb_data norecompute compute > +check_engine_stats northd norecompute compute > check_engine_stats lflow recompute nocompute > CHECK_NO_CHANGE_AFTER_RECOMPUTE > > @@ -10748,7 +10748,7 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE > > check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats > check ovn-nbctl --wait=sb lr-lb-del lr1 lb2 > -check_engine_stats lb_data norecompute nocompute > +check_engine_stats lb_data norecompute compute > check_engine_stats northd recompute nocompute > check_engine_stats lflow recompute nocompute > CHECK_NO_CHANGE_AFTER_RECOMPUTE > @@ -10758,7 +10758,7 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE > check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats > check ovn-nbctl --wait=sb lb-del lb4 > check_engine_stats lb_data norecompute compute > -check_engine_stats northd recompute nocompute > +check_engine_stats northd norecompute compute > check_engine_stats lflow recompute nocompute > CHECK_NO_CHANGE_AFTER_RECOMPUTE > > @@ -10767,7 +10767,7 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE > check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats > check ovn-nbctl --wait=sb lb-del lb2 > check_engine_stats lb_data norecompute compute > -check_engine_stats northd recompute nocompute > +check_engine_stats northd norecompute compute > check_engine_stats lflow recompute nocompute > CHECK_NO_CHANGE_AFTER_RECOMPUTE > > -- > 2.41.0 > > _______________________________________________ > dev mailing list > dev@openvswitch.org > https://mail.openvswitch.org/mailman/listinfo/ovs-dev
On Wed, Sep 13, 2023 at 6:18 PM Han Zhou <hzhou@ovn.org> wrote: > > On Tue, Sep 12, 2023 at 10:47 AM <numans@ovn.org> wrote: > > > > From: Numan Siddique <numans@ovn.org> > > > > When a logical router gets updated due to load balancer or load balancer > > groups changes, it is now incrementally handled first in 'lb_data' > > engine node similar to how logical switch changes are handled. The > > tracking data of 'lb_data' is updated similarly so that northd engine > > handler - northd_handle_lb_data_changes() handles it. > > > > A new handler northd_handle_lr_changes() is added in the 'northd' engine > > node for logical router changes. This handler returns true if only > > load balancer or load balancer group columns are changed. It returns > > false for any other changes. > > > > northd_handle_lb_data_changes() also sets the logical router > > od's lb_ips accordingly. > > > > Below are the scale testing results done with these patches applied > > using ovn-heater. The test ran the scenario - > > ocp-500-density-heavy.yml [1]. > > > > With these patches applied (with load balancer I-P handling in northd > > engine node) the resuts are: > > > > > ------------------------------------------------------------------------------------------------------------------------------------------------------- > > Min (s) Median (s) 90%ile (s) > 99%ile (s) Max (s) Mean (s) Total (s) Count > Failed > > > ------------------------------------------------------------------------------------------------------------------------------------------------------- > > Iteration Total 0.131363 1.189994 3.213526 > 4.308134 4.394562 1.385713 173.214153 125 0 > > Namespace.add_ports 0.005176 0.005611 0.006476 > 0.019867 0.024206 0.006058 0.757188 125 0 > > WorkerNode.bind_port 0.033776 0.046343 0.054414 > 0.061719 0.063613 0.046815 11.703773 250 0 > > WorkerNode.ping_port 0.005156 0.006959 2.044939 > 3.655328 4.241496 0.627103 156.775832 250 0 > > > ------------------------------------------------------------------------------------------------------------------------------------------------------- > > > > The results with the present main are: > > > > > ------------------------------------------------------------------------------------------------------------------------------------------------------- > > Min (s) Median (s) 90%ile (s) > 99%ile (s) Max (s) Mean (s) Total (s) Count > Failed > > > ------------------------------------------------------------------------------------------------------------------------------------------------------- > > Iteration Total 3.233795 4.364926 5.400982 > 6.412803 7.409757 4.792270 599.033790 125 0 > > Namespace.add_ports 0.005230 0.006564 0.007379 > 0.019060 0.037490 0.007223 0.902930 125 0 > > WorkerNode.bind_port 0.033864 0.044052 0.049608 > 0.054849 0.056196 0.044005 11.001231 250 0 > > WorkerNode.ping_port 0.005334 2.060477 5.222422 > 6.267332 7.284001 2.323020 580.754964 250 0 > > > ------------------------------------------------------------------------------------------------------------------------------------------------------- > > > > Few observations: > > > > - The total time taken has come down significantly from 599 seconds to > 173. > > - 99%ile with these patches is 4.3 seconds compared to 6.4 seconds for > the > > main. > > - 99%ile with these patches is 3.2 seconds compared to 5.4 seconds for > the > > main. > > - CPU utilization of northd during the test with these patches > > is between 100% to 300% which is almost the same as main. > > Main difference being that, with these patches the test duration is > > less and hence overall less CPU utilization. > > > > [1] - > https://github.com/ovn-org/ovn-heater/blob/main/test-scenarios/ocp-500-density-heavy.yml > > > > Reviewed-by: Ales Musil <amusil@redhat.com> > > Acked-by: Mark Michelson <mmichels@redhat.com> > > Signed-off-by: Numan Siddique <numans@ovn.org> > > Thanks Numan! I have only a small nit comment inlined. > > Acked-by: Han Zhou <hzhou@ovn.org> Thanks Han, Mark and Ales for the reviews. I addressed your comment (with the below changes) and applied to the main and to branch-23.09 ------ diff --git a/northd/northd.c b/northd/northd.c index 23d11d9d97..bdfd15ec5b 100644 --- a/northd/northd.c +++ b/northd/northd.c @@ -5456,13 +5456,13 @@ fail: return false; } -/* Returns true if the logical router has changes which are not +/* Returns true if the logical router has changes which can be * incrementally handled. * Presently supports i-p for the below changes: * - load balancers and load balancer groups. */ static bool -check_unsupported_inc_proc_for_lr_changes( +lr_changes_can_be_handled( const struct nbrec_logical_router *lr) { /* Check if the columns are changed in this row. */ @@ -5473,7 +5473,7 @@ check_unsupported_inc_proc_for_lr_changes( col == NBREC_LOGICAL_ROUTER_COL_LOAD_BALANCER_GROUP) { continue; } - return true; + return false; } } @@ -5482,32 +5482,32 @@ check_unsupported_inc_proc_for_lr_changes( for (size_t i = 0; i < lr->n_ports; i++) { if (nbrec_logical_router_port_row_get_seqno(lr->ports[i], OVSDB_IDL_CHANGE_MODIFY) > 0) { - return true; + return false; } } if (lr->copp && nbrec_copp_row_get_seqno(lr->copp, OVSDB_IDL_CHANGE_MODIFY) > 0) { - return true; + return false; } for (size_t i = 0; i < lr->n_nat; i++) { if (nbrec_nat_row_get_seqno(lr->nat[i], OVSDB_IDL_CHANGE_MODIFY) > 0) { - return true; + return false; } } for (size_t i = 0; i < lr->n_policies; i++) { if (nbrec_logical_router_policy_row_get_seqno(lr->policies[i], OVSDB_IDL_CHANGE_MODIFY) > 0) { - return true; + return false; } } for (size_t i = 0; i < lr->n_static_routes; i++) { if (nbrec_logical_router_static_route_row_get_seqno( lr->static_routes[i], OVSDB_IDL_CHANGE_MODIFY) > 0) { - return true; + return false; } } - return false; + return true; } /* Return true if changes are handled incrementally, false otherwise. @@ -5534,7 +5534,7 @@ northd_handle_lr_changes(const struct northd_input *ni, /* Presently only able to handle load balancer and * load balancer group changes. */ - if (check_unsupported_inc_proc_for_lr_changes(changed_lr)) { + if (!lr_changes_can_be_handled(changed_lr)) { goto fail; } } ------------------------- Also, I updated the commit message with the latest scale test results for v8. There were some changes in p1 of v8 when compared to v7. In p1 of the v8, the function sync_to_sb_lb_northd_handler() returns false if nd->lb_changed. This was missing in v7 because of which the results were a little better. With v8 the engine node sync_to_sb_lb falls back to recompute for any lb changes (which is correct). We can definitely handle the LB changes in the sync_to_sb_lb_northd_handler() and improve this. I'll address this. Below are the new results ------------------------------------------------------------------------------------------------------------------------------------------------------- Min (s) Median (s) 90%ile (s) 99%ile (s) Max (s) Mean (s) Total (s) Count Failed ------------------------------------------------------------------------------------------------------------------------------------------------------- Iteration Total 0.138730 2.168997 3.224783 3.320061 3.326713 1.616405 202.050672 125 0 Namespace.add_ports 0.005276 0.005608 0.006604 0.009053 0.018615 0.005901 0.737612 125 0 WorkerNode.bind_port 0.034812 0.045776 0.053103 0.057902 0.060541 0.045659 11.414781 250 0 WorkerNode.ping_port 0.005281 0.006927 2.071924 3.186326 3.197238 0.743860 185.964955 250 0 ------------------------------------------------------------------------------------------------------------------------------------------------------- The total time changed from 172 seconds in v7 to 202 seconds in v8 because of this. Thanks Numan > > > --- > > lib/lb.c | 46 +++++- > > lib/lb.h | 9 ++ > > northd/en-lb-data.c | 328 +++++++++++++++++++++++++++++++-------- > > northd/en-lb-data.h | 14 ++ > > northd/en-northd.c | 20 +++ > > northd/en-northd.h | 1 + > > northd/inc-proc-northd.c | 5 +- > > northd/northd.c | 245 ++++++++++++++++++++++++++--- > > northd/northd.h | 2 + > > tests/ovn-northd.at | 42 ++--- > > 10 files changed, 599 insertions(+), 113 deletions(-) > > > > diff --git a/lib/lb.c b/lib/lb.c > > index e6c9dc2be2..d0d562b6fb 100644 > > --- a/lib/lb.c > > +++ b/lib/lb.c > > @@ -794,6 +794,7 @@ ovn_lb_group_init(struct ovn_lb_group *lb_group, > > const struct uuid *lb_uuid = > > &nbrec_lb_group->load_balancer[i]->header_.uuid; > > lb_group->lbs[i] = ovn_northd_lb_find(lbs, lb_uuid); > > + lb_group->has_routable_lb |= lb_group->lbs[i]->routable; > > } > > } > > > > @@ -815,6 +816,7 @@ ovn_lb_group_cleanup(struct ovn_lb_group *lb_group) > > { > > ovn_lb_ip_set_destroy(lb_group->lb_ips); > > lb_group->lb_ips = NULL; > > + lb_group->has_routable_lb = false; > > free(lb_group->lbs); > > } > > > > @@ -1022,23 +1024,54 @@ ovn_lb_5tuples_destroy(struct hmap *tuples) > > void > > build_lrouter_lb_ips(struct ovn_lb_ip_set *lb_ips, > > const struct ovn_northd_lb *lb) > > +{ > > + add_ips_to_lb_ip_set(lb_ips, lb->routable, &lb->ips_v4, &lb->ips_v6); > > +} > > + > > +void > > +add_ips_to_lb_ip_set(struct ovn_lb_ip_set *lb_ips, > > + bool is_routable, > > + const struct sset *lb_ips_v4, > > + const struct sset *lb_ips_v6) > > { > > const char *ip_address; > > > > - SSET_FOR_EACH (ip_address, &lb->ips_v4) { > > + SSET_FOR_EACH (ip_address, lb_ips_v4) { > > sset_add(&lb_ips->ips_v4, ip_address); > > - if (lb->routable) { > > + if (is_routable) { > > sset_add(&lb_ips->ips_v4_routable, ip_address); > > } > > } > > - SSET_FOR_EACH (ip_address, &lb->ips_v6) { > > + SSET_FOR_EACH (ip_address, lb_ips_v6) { > > sset_add(&lb_ips->ips_v6, ip_address); > > - if (lb->routable) { > > + if (is_routable) { > > sset_add(&lb_ips->ips_v6_routable, ip_address); > > } > > } > > } > > > > +void > > +remove_ips_from_lb_ip_set(struct ovn_lb_ip_set *lb_ips, > > + bool is_routable, > > + const struct sset *lb_ips_v4, > > + const struct sset *lb_ips_v6) > > +{ > > + const char *ip_address; > > + > > + SSET_FOR_EACH (ip_address, lb_ips_v4) { > > + sset_find_and_delete(&lb_ips->ips_v4, ip_address); > > + if (is_routable) { > > + sset_find_and_delete(&lb_ips->ips_v4_routable, ip_address); > > + } > > + } > > + SSET_FOR_EACH (ip_address, lb_ips_v6) { > > + sset_find_and_delete(&lb_ips->ips_v6, ip_address); > > + if (is_routable) { > > + sset_find_and_delete(&lb_ips->ips_v6_routable, ip_address); > > + } > > + } > > +} > > + > > /* lb datapaths functions */ > > struct ovn_lb_datapaths * > > ovn_lb_datapaths_create(const struct ovn_northd_lb *lb, size_t > n_ls_datapaths, > > @@ -1079,7 +1112,10 @@ ovn_lb_datapaths_add_lr(struct ovn_lb_datapaths > *lb_dps, size_t n, > > struct ovn_datapath **ods) > > { > > for (size_t i = 0; i < n; i++) { > > - bitmap_set1(lb_dps->nb_lr_map, ods[i]->index); > > + if (!bitmap_is_set(lb_dps->nb_lr_map, ods[i]->index)) { > > + bitmap_set1(lb_dps->nb_lr_map, ods[i]->index); > > + lb_dps->n_nb_lr++; > > + } > > } > > } > > > > diff --git a/lib/lb.h b/lib/lb.h > > index 74905c73b7..b8e3c1e8fb 100644 > > --- a/lib/lb.h > > +++ b/lib/lb.h > > @@ -138,6 +138,14 @@ void ovn_northd_lb_reinit(struct ovn_northd_lb *, > > > > void build_lrouter_lb_ips(struct ovn_lb_ip_set *, > > const struct ovn_northd_lb *); > > +void add_ips_to_lb_ip_set(struct ovn_lb_ip_set *lb_ips, > > + bool is_routable, > > + const struct sset *lb_ips_v4, > > + const struct sset *lb_ips_v6); > > +void remove_ips_from_lb_ip_set(struct ovn_lb_ip_set *lb_ips, > > + bool is_routable, > > + const struct sset *lb_ips_v4, > > + const struct sset *lb_ips_v6); > > > > struct ovn_lb_group { > > struct hmap_node hmap_node; > > @@ -145,6 +153,7 @@ struct ovn_lb_group { > > size_t n_lbs; > > struct ovn_northd_lb **lbs; > > struct ovn_lb_ip_set *lb_ips; > > + bool has_routable_lb; > > }; > > > > struct ovn_lb_group *ovn_lb_group_create( > > diff --git a/northd/en-lb-data.c b/northd/en-lb-data.c > > index fd09e719d8..250cec848b 100644 > > --- a/northd/en-lb-data.c > > +++ b/northd/en-lb-data.c > > @@ -40,20 +40,30 @@ static void build_lbs(const struct > nbrec_load_balancer_table *, > > const struct nbrec_load_balancer_group_table *, > > struct hmap *lbs, struct hmap *lb_groups); > > static void build_od_lb_map(const struct nbrec_logical_switch_table *, > > - struct hmap *od_lb_map); > > + const struct nbrec_logical_router_table *, > > + struct hmap *ls_lb_map, struct hmap > *lr_lb_map); > > static struct od_lb_data *find_od_lb_data(struct hmap *od_lb_map, > > const struct uuid *od_uuid); > > static void destroy_od_lb_data(struct od_lb_data *od_lb_data); > > static struct od_lb_data *create_od_lb_data(struct hmap *od_lb_map, > > const struct uuid *od_uuid); > > +static void handle_od_lb_changes(struct nbrec_load_balancer **, > > + size_t n_nbrec_lbs, > > + struct od_lb_data *od_lb_data, > > + struct ed_type_lb_data *lb_data, > > + struct crupdated_od_lb_data *); > > +static void handle_od_lbgrp_changes(struct nbrec_load_balancer_group **, > > + size_t n_nbrec_lbs, > > + struct od_lb_data *, > > + struct ed_type_lb_data *lb_data, > > + struct crupdated_od_lb_data *); > > > > static struct ovn_lb_group *create_lb_group( > > const struct nbrec_load_balancer_group *, struct hmap *lbs, > > struct hmap *lb_groups); > > static void destroy_tracked_data(struct ed_type_lb_data *); > > -static void add_crupdated_lb_to_tracked_data(struct ovn_northd_lb *, > > - struct > tracked_lb_data *, > > - bool health_checks); > > +static struct crupdated_lb *add_crupdated_lb_to_tracked_data( > > + struct ovn_northd_lb *, struct tracked_lb_data *, bool > health_checks); > > static void add_deleted_lb_to_tracked_data(struct ovn_northd_lb *, > > struct tracked_lb_data > *, > > bool health_checks); > > @@ -64,6 +74,8 @@ static void add_deleted_lbgrp_to_tracked_data( > > struct ovn_lb_group *, struct tracked_lb_data *); > > static bool is_ls_lbs_changed(const struct nbrec_logical_switch *nbs); > > static bool is_ls_lbgrps_changed(const struct nbrec_logical_switch *nbs); > > +static bool is_lr_lbs_changed(const struct nbrec_logical_router *); > > +static bool is_lr_lbgrps_changed(const struct nbrec_logical_router *); > > > > /* 'lb_data' engine node manages the NB load balancers and load balancer > > * groups. For each NB LB, it creates 'struct ovn_northd_lb' and > > @@ -92,10 +104,13 @@ en_lb_data_run(struct engine_node *node, void *data) > > EN_OVSDB_GET(engine_get_input("NB_load_balancer_group", node)); > > const struct nbrec_logical_switch_table *nb_ls_table = > > EN_OVSDB_GET(engine_get_input("NB_logical_switch", node)); > > + const struct nbrec_logical_router_table *nb_lr_table = > > + EN_OVSDB_GET(engine_get_input("NB_logical_router", node)); > > > > lb_data->tracked = false; > > build_lbs(nb_lb_table, nb_lbg_table, &lb_data->lbs, > &lb_data->lbgrps); > > - build_od_lb_map(nb_ls_table, &lb_data->ls_lb_map); > > + build_od_lb_map(nb_ls_table, nb_lr_table, &lb_data->ls_lb_map, > > + &lb_data->lr_lb_map); > > > > engine_set_node_state(node, EN_UPDATED); > > } > > @@ -137,6 +152,7 @@ lb_data_load_balancer_handler(struct engine_node > *node, void *data) > > uuid_hash(&tracked_lb->header_.uuid)); > > add_crupdated_lb_to_tracked_data(lb, trk_lb_data, > > lb->health_checks); > > + trk_lb_data->has_routable_lb |= lb->routable; > > } else if (nbrec_load_balancer_is_deleted(tracked_lb)) { > > lb = ovn_northd_lb_find(&lb_data->lbs, > > &tracked_lb->header_.uuid); > > @@ -144,15 +160,44 @@ lb_data_load_balancer_handler(struct engine_node > *node, void *data) > > hmap_remove(&lb_data->lbs, &lb->hmap_node); > > add_deleted_lb_to_tracked_data(lb, trk_lb_data, > > lb->health_checks); > > + trk_lb_data->has_routable_lb |= lb->routable; > > } else { > > /* Load balancer updated. */ > > lb = ovn_northd_lb_find(&lb_data->lbs, > > &tracked_lb->header_.uuid); > > ovs_assert(lb); > > bool health_checks = lb->health_checks; > > + struct sset old_ips_v4 = SSET_INITIALIZER(&old_ips_v4); > > + struct sset old_ips_v6 = SSET_INITIALIZER(&old_ips_v6); > > + sset_swap(&lb->ips_v4, &old_ips_v4); > > + sset_swap(&lb->ips_v6, &old_ips_v6); > > ovn_northd_lb_reinit(lb, tracked_lb); > > health_checks |= lb->health_checks; > > - add_crupdated_lb_to_tracked_data(lb, trk_lb_data, > health_checks); > > + struct crupdated_lb *clb = add_crupdated_lb_to_tracked_data( > > + lb, trk_lb_data, health_checks); > > + trk_lb_data->has_routable_lb |= lb->routable; > > + > > + /* Determine the inserted and deleted vips and store them in > > + * the tracked data. */ > > + const char *vip; > > + SSET_FOR_EACH (vip, &lb->ips_v4) { > > + if (!sset_find_and_delete(&old_ips_v4, vip)) { > > + sset_add(&clb->inserted_vips_v4, vip); > > + } > > + } > > + > > + sset_swap(&old_ips_v4, &clb->deleted_vips_v4); > > + > > + SSET_FOR_EACH (vip, &lb->ips_v6) { > > + if (!sset_find_and_delete(&old_ips_v6, vip)) { > > + sset_add(&clb->inserted_vips_v6, vip); > > + } > > + } > > + > > + sset_swap(&old_ips_v6, &clb->deleted_vips_v6); > > + > > + sset_destroy(&old_ips_v4); > > + sset_destroy(&old_ips_v6); > > } > > } > > > > @@ -181,6 +226,8 @@ lb_data_load_balancer_group_handler(struct > engine_node *node, void *data) > > for (size_t i = 0; i < lb_group->n_lbs; i++) { > > hmapx_add(&clbg->assoc_lbs, lb_group->lbs[i]); > > } > > + > > + trk_lb_data->has_routable_lb |= lb_group->has_routable_lb; > > } else if > (nbrec_load_balancer_group_is_deleted(tracked_lb_group)) { > > struct ovn_lb_group *lb_group; > > lb_group = ovn_lb_group_find(&lb_data->lbgrps, > > @@ -188,6 +235,7 @@ lb_data_load_balancer_group_handler(struct > engine_node *node, void *data) > > ovs_assert(lb_group); > > hmap_remove(&lb_data->lbgrps, &lb_group->hmap_node); > > add_deleted_lbgrp_to_tracked_data(lb_group, trk_lb_data); > > + trk_lb_data->has_routable_lb |= lb_group->has_routable_lb; > > } else { > > > > struct ovn_lb_group *lb_group; > > @@ -209,6 +257,7 @@ lb_data_load_balancer_group_handler(struct > engine_node *node, void *data) > > build_lrouter_lb_ips(lb_group->lb_ips, lb_group->lbs[i]); > > } > > > > + trk_lb_data->has_routable_lb |= lb_group->has_routable_lb; > > struct crupdated_lbgrp *clbg = > > add_crupdated_lbgrp_to_tracked_data(lb_group, > trk_lb_data); > > > > @@ -270,6 +319,7 @@ lb_data_logical_switch_handler(struct engine_node > *node, void *data) > > if (!ls_lbs_changed && !ls_lbgrps_changed) { > > continue; > > } > > + changed = true; > > struct crupdated_od_lb_data *codlb = xzalloc(sizeof *codlb); > > codlb->od_uuid = nbs->header_.uuid; > > uuidset_init(&codlb->assoc_lbs); > > @@ -283,66 +333,77 @@ lb_data_logical_switch_handler(struct engine_node > *node, void *data) > > } > > > > if (ls_lbs_changed) { > > - struct uuidset *pre_lb_uuids = od_lb_data->lbs; > > - od_lb_data->lbs = xzalloc(sizeof *od_lb_data->lbs); > > - uuidset_init(od_lb_data->lbs); > > - > > - for (size_t i = 0; i < nbs->n_load_balancer; i++) { > > - const struct uuid *lb_uuid = > > - &nbs->load_balancer[i]->header_.uuid; > > - uuidset_insert(od_lb_data->lbs, lb_uuid); > > - > > - struct uuidset_node *unode = > uuidset_find(pre_lb_uuids, > > - lb_uuid); > > - > > - if (!unode || (nbrec_load_balancer_row_get_seqno( > > - nbs->load_balancer[i], > > - OVSDB_IDL_CHANGE_MODIFY) > 0)) { > > - /* Add this lb to the tracked data. */ > > - uuidset_insert(&codlb->assoc_lbs, lb_uuid); > > - changed = true; > > - } > > - > > - if (unode) { > > - uuidset_delete(pre_lb_uuids, unode); > > - } > > - } > > - if (!uuidset_is_empty(pre_lb_uuids)) { > > - trk_lb_data->has_dissassoc_lbs_from_od = true; > > - changed = true; > > - } > > - > > - uuidset_destroy(pre_lb_uuids); > > - free(pre_lb_uuids); > > + handle_od_lb_changes(nbs->load_balancer, > nbs->n_load_balancer, > > + od_lb_data, lb_data, codlb); > > } > > > > if (ls_lbgrps_changed) { > > - struct uuidset *pre_lbgrp_uuids = od_lb_data->lbgrps; > > - od_lb_data->lbgrps = xzalloc(sizeof *od_lb_data->lbgrps); > > - uuidset_init(od_lb_data->lbgrps); > > - for (size_t i = 0; i < nbs->n_load_balancer_group; i++) { > > - const struct uuid *lbg_uuid = > > - &nbs->load_balancer_group[i]->header_.uuid; > > - uuidset_insert(od_lb_data->lbgrps, lbg_uuid); > > - > > - if (!uuidset_find_and_delete(pre_lbgrp_uuids, > > - lbg_uuid)) { > > - /* Add this lb group to the tracked data. */ > > - uuidset_insert(&codlb->assoc_lbgrps, lbg_uuid); > > - changed = true; > > - } > > - } > > + handle_od_lbgrp_changes(nbs->load_balancer_group, > > + nbs->n_load_balancer_group, > > + od_lb_data, lb_data, codlb); > > + } > > > > - if (!uuidset_is_empty(pre_lbgrp_uuids)) { > > - trk_lb_data->has_dissassoc_lbgrps_from_od = true; > > - changed = true; > > - } > > + ovs_list_insert(&trk_lb_data->crupdated_ls_lbs, > &codlb->list_node); > > + } > > + } > > + > > + if (changed) { > > + engine_set_node_state(node, EN_UPDATED); > > + } > > + return true; > > +} > > + > > +bool > > +lb_data_logical_router_handler(struct engine_node *node, void *data) > > +{ > > + struct ed_type_lb_data *lb_data = (struct ed_type_lb_data *) data; > > + const struct nbrec_logical_router_table *nbrec_lr_table = > > + EN_OVSDB_GET(engine_get_input("NB_logical_router", node)); > > + > > + struct tracked_lb_data *trk_lb_data = &lb_data->tracked_lb_data; > > + lb_data->tracked = true; > > > > - uuidset_destroy(pre_lbgrp_uuids); > > - free(pre_lbgrp_uuids); > > + bool changed = false; > > + const struct nbrec_logical_router *nbr; > > + NBREC_LOGICAL_ROUTER_TABLE_FOR_EACH_TRACKED (nbr, nbrec_lr_table) { > > + if (nbrec_logical_router_is_deleted(nbr)) { > > + struct od_lb_data *od_lb_data = > > + find_od_lb_data(&lb_data->lr_lb_map, &nbr->header_.uuid); > > + if (od_lb_data) { > > + hmap_remove(&lb_data->lr_lb_map, &od_lb_data->hmap_node); > > + hmapx_add(&trk_lb_data->deleted_od_lb_data, od_lb_data); > > + } > > + } else { > > + bool lr_lbs_changed = is_lr_lbs_changed(nbr); > > + bool lr_lbgrps_changed = is_lr_lbgrps_changed(nbr); > > + if (!lr_lbs_changed && !lr_lbgrps_changed) { > > + continue; > > } > > + changed = true; > > + struct crupdated_od_lb_data *codlb = xzalloc(sizeof *codlb); > > + codlb->od_uuid = nbr->header_.uuid; > > + uuidset_init(&codlb->assoc_lbs); > > + uuidset_init(&codlb->assoc_lbgrps); > > > > - ovs_list_insert(&trk_lb_data->crupdated_ls_lbs, > &codlb->list_node); > > + struct od_lb_data *od_lb_data = > > + find_od_lb_data(&lb_data->lr_lb_map, &nbr->header_.uuid); > > + if (!od_lb_data) { > > + od_lb_data = create_od_lb_data(&lb_data->lr_lb_map, > > + &nbr->header_.uuid); > > + } > > + > > + if (lr_lbs_changed) { > > + handle_od_lb_changes(nbr->load_balancer, > nbr->n_load_balancer, > > + od_lb_data, lb_data, codlb); > > + } > > + > > + if (lr_lbgrps_changed) { > > + handle_od_lbgrp_changes(nbr->load_balancer_group, > > + nbr->n_load_balancer_group, > > + od_lb_data, lb_data, codlb); > > + } > > + > > + ovs_list_insert(&trk_lb_data->crupdated_lr_lbs, > &codlb->list_node); > > } > > } > > > > @@ -359,6 +420,7 @@ lb_data_init(struct ed_type_lb_data *lb_data) > > hmap_init(&lb_data->lbs); > > hmap_init(&lb_data->lbgrps); > > hmap_init(&lb_data->ls_lb_map); > > + hmap_init(&lb_data->lr_lb_map); > > > > struct tracked_lb_data *trk_lb_data = &lb_data->tracked_lb_data; > > hmap_init(&trk_lb_data->crupdated_lbs); > > @@ -366,6 +428,7 @@ lb_data_init(struct ed_type_lb_data *lb_data) > > hmap_init(&trk_lb_data->crupdated_lbgrps); > > hmapx_init(&trk_lb_data->deleted_lbgrps); > > ovs_list_init(&trk_lb_data->crupdated_ls_lbs); > > + ovs_list_init(&trk_lb_data->crupdated_lr_lbs); > > hmapx_init(&trk_lb_data->deleted_od_lb_data); > > } > > > > @@ -390,6 +453,11 @@ lb_data_destroy(struct ed_type_lb_data *lb_data) > > } > > hmap_destroy(&lb_data->ls_lb_map); > > > > + HMAP_FOR_EACH_POP (od_lb_data, hmap_node, &lb_data->lr_lb_map) { > > + destroy_od_lb_data(od_lb_data); > > + } > > + hmap_destroy(&lb_data->lr_lb_map); > > + > > destroy_tracked_data(lb_data); > > hmap_destroy(&lb_data->tracked_lb_data.crupdated_lbs); > > hmapx_destroy(&lb_data->tracked_lb_data.deleted_lbs); > > @@ -435,7 +503,8 @@ create_lb_group(const struct > nbrec_load_balancer_group *nbrec_lb_group, > > > > static void > > build_od_lb_map(const struct nbrec_logical_switch_table *nbrec_ls_table, > > - struct hmap *od_lb_map) > > + const struct nbrec_logical_router_table *nbrec_lr_table, > > + struct hmap *ls_lb_map, struct hmap *lr_lb_map) > > { > > const struct nbrec_logical_switch *nbrec_ls; > > NBREC_LOGICAL_SWITCH_TABLE_FOR_EACH (nbrec_ls, nbrec_ls_table) { > > @@ -443,17 +512,35 @@ build_od_lb_map(const struct > nbrec_logical_switch_table *nbrec_ls_table, > > continue; > > } > > > > - struct od_lb_data *od_lb_data = > > - create_od_lb_data(od_lb_map, &nbrec_ls->header_.uuid); > > + struct od_lb_data *ls_lb_data = > > + create_od_lb_data(ls_lb_map, &nbrec_ls->header_.uuid); > > for (size_t i = 0; i < nbrec_ls->n_load_balancer; i++) { > > - uuidset_insert(od_lb_data->lbs, > > + uuidset_insert(ls_lb_data->lbs, > > &nbrec_ls->load_balancer[i]->header_.uuid); > > } > > for (size_t i = 0; i < nbrec_ls->n_load_balancer_group; i++) { > > - uuidset_insert(od_lb_data->lbgrps, > > + uuidset_insert(ls_lb_data->lbgrps, > > > &nbrec_ls->load_balancer_group[i]->header_.uuid); > > } > > } > > + > > + const struct nbrec_logical_router *nbrec_lr; > > + NBREC_LOGICAL_ROUTER_TABLE_FOR_EACH (nbrec_lr, nbrec_lr_table) { > > + if (!nbrec_lr->n_load_balancer && > !nbrec_lr->n_load_balancer_group) { > > + continue; > > + } > > + > > + struct od_lb_data *lr_lb_data = > > + create_od_lb_data(lr_lb_map, &nbrec_lr->header_.uuid); > > + for (size_t i = 0; i < nbrec_lr->n_load_balancer; i++) { > > + uuidset_insert(lr_lb_data->lbs, > > + &nbrec_lr->load_balancer[i]->header_.uuid); > > + } > > + for (size_t i = 0; i < nbrec_lr->n_load_balancer_group; i++) { > > + uuidset_insert(lr_lb_data->lbgrps, > > + > &nbrec_lr->load_balancer_group[i]->header_.uuid); > > + } > > + } > > } > > > > static struct od_lb_data * > > @@ -495,6 +582,84 @@ destroy_od_lb_data(struct od_lb_data *od_lb_data) > > free(od_lb_data); > > } > > > > +static void > > +handle_od_lb_changes(struct nbrec_load_balancer **nbrec_lbs, > > + size_t n_nbrec_lbs, struct od_lb_data *od_lb_data, > > + struct ed_type_lb_data *lb_data, > > + struct crupdated_od_lb_data *codlb) > > +{ > > + struct tracked_lb_data *trk_lb_data = &lb_data->tracked_lb_data; > > + struct uuidset *pre_lb_uuids = od_lb_data->lbs; > > + od_lb_data->lbs = xzalloc(sizeof *od_lb_data->lbs); > > + uuidset_init(od_lb_data->lbs); > > + > > + for (size_t i = 0; i < n_nbrec_lbs; i++) { > > + const struct uuid *lb_uuid = &nbrec_lbs[i]->header_.uuid; > > + uuidset_insert(od_lb_data->lbs, lb_uuid); > > + > > + struct uuidset_node *unode = uuidset_find(pre_lb_uuids, lb_uuid); > > + > > + if (!unode || (nbrec_load_balancer_row_get_seqno( > > + nbrec_lbs[i], OVSDB_IDL_CHANGE_MODIFY) > 0)) { > > + /* Add this lb to the tracked data. */ > > + uuidset_insert(&codlb->assoc_lbs, lb_uuid); > > + > > + if (!trk_lb_data->has_routable_lb) { > > + struct ovn_northd_lb *lb = > ovn_northd_lb_find(&lb_data->lbs, > > + lb_uuid); > > + ovs_assert(lb); > > + trk_lb_data->has_routable_lb |= lb->routable; > > + } > > + } > > + > > + if (unode) { > > + uuidset_delete(pre_lb_uuids, unode); > > + } > > + } > > + > > + if (!uuidset_is_empty(pre_lb_uuids)) { > > + trk_lb_data->has_dissassoc_lbs_from_od = true; > > + } > > + > > + uuidset_destroy(pre_lb_uuids); > > + free(pre_lb_uuids); > > +} > > + > > +static void > > +handle_od_lbgrp_changes(struct nbrec_load_balancer_group **nbrec_lbgrps, > > + size_t n_nbrec_lbgrps, struct od_lb_data > *od_lb_data, > > + struct ed_type_lb_data *lb_data, > > + struct crupdated_od_lb_data *codlb) > > +{ > > + struct tracked_lb_data *trk_lb_data = &lb_data->tracked_lb_data; > > + struct uuidset *pre_lbgrp_uuids = od_lb_data->lbgrps; > > + od_lb_data->lbgrps = xzalloc(sizeof *od_lb_data->lbgrps); > > + uuidset_init(od_lb_data->lbgrps); > > + for (size_t i = 0; i < n_nbrec_lbgrps; i++) { > > + const struct uuid *lbgrp_uuid = &nbrec_lbgrps[i]->header_.uuid; > > + uuidset_insert(od_lb_data->lbgrps, lbgrp_uuid); > > + > > + if (!uuidset_find_and_delete(pre_lbgrp_uuids, lbgrp_uuid)) { > > + /* Add this lb group to the tracked data. */ > > + uuidset_insert(&codlb->assoc_lbgrps, lbgrp_uuid); > > + > > + if (!trk_lb_data->has_routable_lb) { > > + struct ovn_lb_group *lbgrp = > > + ovn_lb_group_find(&lb_data->lbgrps, lbgrp_uuid); > > + ovs_assert(lbgrp); > > + trk_lb_data->has_routable_lb |= lbgrp->has_routable_lb; > > + } > > + } > > + } > > + > > + if (!uuidset_is_empty(pre_lbgrp_uuids)) { > > + trk_lb_data->has_dissassoc_lbgrps_from_od = true; > > + } > > + > > + uuidset_destroy(pre_lbgrp_uuids); > > + free(pre_lbgrp_uuids); > > +} > > + > > static void > > destroy_tracked_data(struct ed_type_lb_data *lb_data) > > { > > @@ -503,6 +668,7 @@ destroy_tracked_data(struct ed_type_lb_data *lb_data) > > lb_data->tracked_lb_data.has_dissassoc_lbs_from_lbgrps = false; > > lb_data->tracked_lb_data.has_dissassoc_lbs_from_od = false; > > lb_data->tracked_lb_data.has_dissassoc_lbgrps_from_od = false; > > + lb_data->tracked_lb_data.has_routable_lb = false; > > > > struct hmapx_node *node; > > HMAPX_FOR_EACH_SAFE (node, &lb_data->tracked_lb_data.deleted_lbs) { > > @@ -518,6 +684,10 @@ destroy_tracked_data(struct ed_type_lb_data *lb_data) > > struct crupdated_lb *clb; > > HMAP_FOR_EACH_POP (clb, hmap_node, > > &lb_data->tracked_lb_data.crupdated_lbs) { > > + sset_destroy(&clb->inserted_vips_v4); > > + sset_destroy(&clb->inserted_vips_v6); > > + sset_destroy(&clb->deleted_vips_v4); > > + sset_destroy(&clb->deleted_vips_v6); > > free(clb); > > } > > > > @@ -537,13 +707,21 @@ destroy_tracked_data(struct ed_type_lb_data > *lb_data) > > free(codlb); > > } > > > > + LIST_FOR_EACH_SAFE (codlb, list_node, > > + &lb_data->tracked_lb_data.crupdated_lr_lbs) { > > + ovs_list_remove(&codlb->list_node); > > + uuidset_destroy(&codlb->assoc_lbs); > > + uuidset_destroy(&codlb->assoc_lbgrps); > > + free(codlb); > > + } > > + > > HMAPX_FOR_EACH_SAFE (node, > &lb_data->tracked_lb_data.deleted_od_lb_data) { > > destroy_od_lb_data(node->data); > > hmapx_delete(&lb_data->tracked_lb_data.deleted_od_lb_data, node); > > } > > } > > > > -static void > > +static struct crupdated_lb * > > add_crupdated_lb_to_tracked_data(struct ovn_northd_lb *lb, > > struct tracked_lb_data *tracked_lb_data, > > bool health_checks) > > @@ -552,9 +730,15 @@ add_crupdated_lb_to_tracked_data(struct > ovn_northd_lb *lb, > > clb->lb = lb; > > hmap_insert(&tracked_lb_data->crupdated_lbs, &clb->hmap_node, > > uuid_hash(&lb->nlb->header_.uuid)); > > + sset_init(&clb->inserted_vips_v4); > > + sset_init(&clb->inserted_vips_v6); > > + sset_init(&clb->deleted_vips_v4); > > + sset_init(&clb->deleted_vips_v6); > > if (health_checks) { > > tracked_lb_data->has_health_checks = true; > > } > > + > > + return clb; > > } > > > > static void > > @@ -600,3 +784,17 @@ is_ls_lbgrps_changed(const struct > nbrec_logical_switch *nbs) { > > || nbrec_logical_switch_is_updated(nbs, > > NBREC_LOGICAL_SWITCH_COL_LOAD_BALANCER_GROUP)); > > } > > + > > +static bool > > +is_lr_lbs_changed(const struct nbrec_logical_router *nbr) { > > + return ((nbrec_logical_router_is_new(nbr) && nbr->n_load_balancer) > > + || nbrec_logical_router_is_updated(nbr, > > + NBREC_LOGICAL_ROUTER_COL_LOAD_BALANCER)); > > +} > > + > > +static bool > > +is_lr_lbgrps_changed(const struct nbrec_logical_router *nbr) { > > + return ((nbrec_logical_router_is_new(nbr) && > nbr->n_load_balancer_group) > > + || nbrec_logical_router_is_updated(nbr, > > + NBREC_LOGICAL_ROUTER_COL_LOAD_BALANCER_GROUP)); > > +} > > diff --git a/northd/en-lb-data.h b/northd/en-lb-data.h > > index ebfd547d77..9d20a539ba 100644 > > --- a/northd/en-lb-data.h > > +++ b/northd/en-lb-data.h > > @@ -6,6 +6,7 @@ > > #include "openvswitch/hmap.h" > > #include "include/openvswitch/list.h" > > #include "lib/hmapx.h" > > +#include "lib/sset.h" > > #include "lib/uuidset.h" > > > > #include "lib/inc-proc-eng.h" > > @@ -17,6 +18,10 @@ struct crupdated_lb { > > struct hmap_node hmap_node; > > > > struct ovn_northd_lb *lb; > > + struct sset inserted_vips_v4; > > + struct sset inserted_vips_v6; > > + struct sset deleted_vips_v4; > > + struct sset deleted_vips_v6; > > }; > > > > struct crupdated_lbgrp { > > @@ -54,6 +59,10 @@ struct tracked_lb_data { > > * 'struct crupdated_od_lb_data' */ > > struct ovs_list crupdated_ls_lbs; > > > > + /* List of logical router <-> lb changes. List node is > > + * 'struct crupdated_od_lb_data' */ > > + struct ovs_list crupdated_lr_lbs; > > + > > /* hmapx of deleted logical switches which have load balancer or lb > groups > > * associated with it. hmapx_node is 'struct od_lb_data'. */ > > struct hmapx deleted_od_lb_data; > > @@ -70,6 +79,9 @@ struct tracked_lb_data { > > > > /* Indicates if a lb group was disassociated from a logical switch. > */ > > bool has_dissassoc_lbgrps_from_od; > > + > > + /* Indicates if any lb (in the tracked data) has 'routable' flag > set. */ > > + bool has_routable_lb; > > }; > > > > /* Datapath (logical switch) to lb/lbgrp association data. */ > > @@ -90,6 +102,7 @@ struct ed_type_lb_data { > > > > /* hmap of ls to lb map. hmap node is 'struct od_lb_data'. */ > > struct hmap ls_lb_map; > > + struct hmap lr_lb_map; > > > > /* tracked data*/ > > bool tracked; > > @@ -104,5 +117,6 @@ void en_lb_data_clear_tracked_data(void *data); > > bool lb_data_load_balancer_handler(struct engine_node *, void *data); > > bool lb_data_load_balancer_group_handler(struct engine_node *, void > *data); > > bool lb_data_logical_switch_handler(struct engine_node *, void *data); > > +bool lb_data_logical_router_handler(struct engine_node *, void *data); > > > > #endif /* end of EN_NORTHD_LB_DATA_H */ > > diff --git a/northd/en-northd.c b/northd/en-northd.c > > index 8487b003f7..aa0f20f0c2 100644 > > --- a/northd/en-northd.c > > +++ b/northd/en-northd.c > > @@ -196,6 +196,26 @@ northd_sb_port_binding_handler(struct engine_node > *node, > > return true; > > } > > > > +bool > > +northd_nb_logical_router_handler(struct engine_node *node, > > + void *data) > > +{ > > + struct northd_data *nd = data; > > + struct northd_input input_data; > > + > > + northd_get_input_data(node, &input_data); > > + > > + if (!northd_handle_lr_changes(&input_data, nd)) { > > + return false; > > + } > > + > > + if (nd->change_tracked) { > > + engine_set_node_state(node, EN_UPDATED); > > + } > > + > > + return true; > > +} > > + > > bool > > northd_lb_data_handler(struct engine_node *node, void *data) > > { > > diff --git a/northd/en-northd.h b/northd/en-northd.h > > index 3c77b64bb2..5a88871760 100644 > > --- a/northd/en-northd.h > > +++ b/northd/en-northd.h > > @@ -16,6 +16,7 @@ void en_northd_cleanup(void *data); > > void en_northd_clear_tracked_data(void *data); > > bool northd_nb_nb_global_handler(struct engine_node *, void *data > OVS_UNUSED); > > bool northd_nb_logical_switch_handler(struct engine_node *, void *data); > > +bool northd_nb_logical_router_handler(struct engine_node *, void *data); > > bool northd_sb_port_binding_handler(struct engine_node *, void *data); > > bool northd_lb_data_handler(struct engine_node *, void *data); > > > > diff --git a/northd/inc-proc-northd.c b/northd/inc-proc-northd.c > > index e9e28c4bea..8b08171179 100644 > > --- a/northd/inc-proc-northd.c > > +++ b/northd/inc-proc-northd.c > > @@ -158,8 +158,9 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, > > lb_data_load_balancer_group_handler); > > engine_add_input(&en_lb_data, &en_nb_logical_switch, > > lb_data_logical_switch_handler); > > + engine_add_input(&en_lb_data, &en_nb_logical_router, > > + lb_data_logical_router_handler); > > > > - engine_add_input(&en_northd, &en_nb_logical_router, NULL); > > engine_add_input(&en_northd, &en_nb_mirror, NULL); > > engine_add_input(&en_northd, &en_nb_static_mac_binding, NULL); > > engine_add_input(&en_northd, &en_nb_chassis_template_var, NULL); > > @@ -184,6 +185,8 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, > > northd_nb_nb_global_handler); > > engine_add_input(&en_northd, &en_nb_logical_switch, > > northd_nb_logical_switch_handler); > > + engine_add_input(&en_northd, &en_nb_logical_router, > > + northd_nb_logical_router_handler); > > engine_add_input(&en_northd, &en_lb_data, northd_lb_data_handler); > > > > engine_add_input(&en_mac_binding_aging, &en_nb_nb_global, NULL); > > diff --git a/northd/northd.c b/northd/northd.c > > index f6258d4830..23d11d9d97 100644 > > --- a/northd/northd.c > > +++ b/northd/northd.c > > @@ -4167,54 +4167,63 @@ static bool lrouter_port_ipv4_reachable(const > struct ovn_port *op, > > ovs_be32 addr); > > static bool lrouter_port_ipv6_reachable(const struct ovn_port *op, > > const struct in6_addr *addr); > > + > > static void > > -build_lrouter_lb_reachable_ips(struct ovn_datapath *od, > > - const struct ovn_northd_lb *lb) > > +add_neigh_ips_to_lrouter(struct ovn_datapath *od, > > + enum lb_neighbor_responder_mode neigh_mode, > > + const struct sset *lb_ips_v4, > > + const struct sset *lb_ips_v6) > > { > > /* If configured to not reply to any neighbor requests for all VIPs > > * return early. > > */ > > - if (lb->neigh_mode == LB_NEIGH_RESPOND_NONE) { > > + if (neigh_mode == LB_NEIGH_RESPOND_NONE) { > > return; > > } > > > > + const char *ip_address; > > + > > /* If configured to reply to neighbor requests for all VIPs force > them > > * all to be considered "reachable". > > */ > > - if (lb->neigh_mode == LB_NEIGH_RESPOND_ALL) { > > - for (size_t i = 0; i < lb->n_vips; i++) { > > - if (lb->vips[i].address_family == AF_INET) { > > - sset_add(&od->lb_ips->ips_v4_reachable, > lb->vips[i].vip_str); > > - } else { > > - sset_add(&od->lb_ips->ips_v6_reachable, > lb->vips[i].vip_str); > > - } > > + if (neigh_mode == LB_NEIGH_RESPOND_ALL) { > > + SSET_FOR_EACH (ip_address, lb_ips_v4) { > > + sset_add(&od->lb_ips->ips_v4_reachable, ip_address); > > + } > > + SSET_FOR_EACH (ip_address, lb_ips_v6) { > > + sset_add(&od->lb_ips->ips_v6_reachable, ip_address); > > } > > + > > return; > > } > > > > /* Otherwise, a VIP is reachable if there's at least one router > > * subnet that includes it. > > */ > > - ovs_assert(lb->neigh_mode == LB_NEIGH_RESPOND_REACHABLE); > > - for (size_t i = 0; i < lb->n_vips; i++) { > > - if (lb->vips[i].address_family == AF_INET) { > > - ovs_be32 vip_ip4 = > in6_addr_get_mapped_ipv4(&lb->vips[i].vip); > > - struct ovn_port *op; > > + ovs_assert(neigh_mode == LB_NEIGH_RESPOND_REACHABLE); > > > > + SSET_FOR_EACH (ip_address, lb_ips_v4) { > > + struct ovn_port *op; > > + ovs_be32 vip_ip4; > > + if (ip_parse(ip_address, &vip_ip4)) { > > HMAP_FOR_EACH (op, dp_node, &od->ports) { > > if (lrouter_port_ipv4_reachable(op, vip_ip4)) { > > sset_add(&od->lb_ips->ips_v4_reachable, > > - lb->vips[i].vip_str); > > + ip_address); > > break; > > } > > } > > - } else { > > - struct ovn_port *op; > > + } > > + } > > > > + SSET_FOR_EACH (ip_address, lb_ips_v6) { > > + struct ovn_port *op; > > + struct in6_addr vip; > > + if (ipv6_parse(ip_address, &vip)) { > > HMAP_FOR_EACH (op, dp_node, &od->ports) { > > - if (lrouter_port_ipv6_reachable(op, &lb->vips[i].vip)) { > > + if (lrouter_port_ipv6_reachable(op, &vip)) { > > sset_add(&od->lb_ips->ips_v6_reachable, > > - lb->vips[i].vip_str); > > + ip_address); > > break; > > } > > } > > @@ -4222,6 +4231,34 @@ build_lrouter_lb_reachable_ips(struct ovn_datapath > *od, > > } > > } > > > > +static void > > +remove_lrouter_lb_reachable_ips(struct ovn_datapath *od, > > + enum lb_neighbor_responder_mode > neigh_mode, > > + const struct sset *lb_ips_v4, > > + const struct sset *lb_ips_v6) > > +{ > > + if (neigh_mode == LB_NEIGH_RESPOND_NONE) { > > + return; > > + } > > + > > + const char *ip_address; > > + SSET_FOR_EACH (ip_address, lb_ips_v4) { > > + sset_find_and_delete(&od->lb_ips->ips_v4_reachable, ip_address); > > + } > > + SSET_FOR_EACH (ip_address, lb_ips_v6) { > > + sset_find_and_delete(&od->lb_ips->ips_v6_reachable, ip_address); > > + } > > +} > > + > > +static void > > +build_lrouter_lb_reachable_ips(struct ovn_datapath *od, > > + const struct ovn_northd_lb *lb) > > +{ > > + add_neigh_ips_to_lrouter(od, lb->neigh_mode, &lb->ips_v4, > > + &lb->ips_v6); > > +} > > + > > + > > static void > > build_lrouter_lbs_check(const struct ovn_datapaths *lr_datapaths) > > { > > @@ -5419,6 +5456,95 @@ fail: > > return false; > > } > > > > +/* Returns true if the logical router has changes which are not > > + * incrementally handled. > > + * Presently supports i-p for the below changes: > > + * - load balancers and load balancer groups. > > + */ > > +static bool > > +check_unsupported_inc_proc_for_lr_changes( > > nit: It would be better to be aligned with the naming convention of > ls_changes_can_be_handled(), something like: lr_changes_can_be_handled(). > > > + const struct nbrec_logical_router *lr) > > +{ > > + /* Check if the columns are changed in this row. */ > > + enum nbrec_logical_router_column_id col; > > + for (col = 0; col < NBREC_LOGICAL_ROUTER_N_COLUMNS; col++) { > > + if (nbrec_logical_router_is_updated(lr, col)) { > > + if (col == NBREC_LOGICAL_ROUTER_COL_LOAD_BALANCER || > > + col == NBREC_LOGICAL_ROUTER_COL_LOAD_BALANCER_GROUP) { > > + continue; > > + } > > + return true; > > + } > > + } > > + > > + /* Check if the referenced rows are changed. > > + XXX: Need a better OVSDB IDL interface for this check. */ > > + for (size_t i = 0; i < lr->n_ports; i++) { > > + if (nbrec_logical_router_port_row_get_seqno(lr->ports[i], > > + OVSDB_IDL_CHANGE_MODIFY) > 0) { > > + return true; > > + } > > + } > > + if (lr->copp && nbrec_copp_row_get_seqno(lr->copp, > > + OVSDB_IDL_CHANGE_MODIFY) > 0) { > > + return true; > > + } > > + for (size_t i = 0; i < lr->n_nat; i++) { > > + if (nbrec_nat_row_get_seqno(lr->nat[i], > > + OVSDB_IDL_CHANGE_MODIFY) > 0) { > > + return true; > > + } > > + } > > + for (size_t i = 0; i < lr->n_policies; i++) { > > + if (nbrec_logical_router_policy_row_get_seqno(lr->policies[i], > > + OVSDB_IDL_CHANGE_MODIFY) > 0) { > > + return true; > > + } > > + } > > + for (size_t i = 0; i < lr->n_static_routes; i++) { > > + if (nbrec_logical_router_static_route_row_get_seqno( > > + lr->static_routes[i], OVSDB_IDL_CHANGE_MODIFY) > 0) { > > + return true; > > + } > > + } > > + return false; > > +} > > + > > +/* Return true if changes are handled incrementally, false otherwise. > > + * When there are any changes, try to track what's exactly changed and > set > > + * northd_data->change_tracked accordingly: change tracked - true, > otherwise, > > + * false. > > + * Note: Changes to load balancer and load balancer groups associated > with > > + * the logical routers are handled separately in the lb_data change > > + * handlers (northd_handle_lb_data_changes_pre_od and > > + * northd_handle_lb_data_changes_post_od). > > + * */ > > +bool > > +northd_handle_lr_changes(const struct northd_input *ni, > > + struct northd_data *nd) > > +{ > > + const struct nbrec_logical_router *changed_lr; > > + > > + NBREC_LOGICAL_ROUTER_TABLE_FOR_EACH_TRACKED (changed_lr, > > + > ni->nbrec_logical_router_table) { > > + if (nbrec_logical_router_is_new(changed_lr) || > > + nbrec_logical_router_is_deleted(changed_lr)) { > > + goto fail; > > + } > > + > > + /* Presently only able to handle load balancer and > > + * load balancer group changes. */ > > + if (check_unsupported_inc_proc_for_lr_changes(changed_lr)) { > > + goto fail; > > + } > > + } > > + > > + return true; > > +fail: > > + destroy_northd_data_tracked_changes(nd); > > + return false; > > +} > > + > > bool > > northd_handle_sb_port_binding_changes( > > const struct sbrec_port_binding_table *sbrec_port_binding_table, > > @@ -5535,6 +5661,10 @@ northd_handle_lb_data_changes(struct > tracked_lb_data *trk_lb_data, > > return false; > > } > > > > + if (trk_lb_data->has_routable_lb) { > > + return false; > > + } > > + > > struct ovn_lb_datapaths *lb_dps; > > struct ovn_northd_lb *lb; > > struct ovn_datapath *od; > > @@ -5625,6 +5755,45 @@ northd_handle_lb_data_changes(struct > tracked_lb_data *trk_lb_data, > > init_lb_for_datapath(od); > > } > > > > + LIST_FOR_EACH (codlb, list_node, &trk_lb_data->crupdated_lr_lbs) { > > + od = ovn_datapath_find(&lr_datapaths->datapaths, > &codlb->od_uuid); > > + ovs_assert(od); > > + > > + struct uuidset_node *uuidnode; > > + UUIDSET_FOR_EACH (uuidnode, &codlb->assoc_lbs) { > > + lb_dps = ovn_lb_datapaths_find(lb_datapaths_map, > &uuidnode->uuid); > > + ovs_assert(lb_dps); > > + ovn_lb_datapaths_add_lr(lb_dps, 1, &od); > > + > > + /* Add the lb_ips of lb_dps to the od. */ > > + build_lrouter_lb_ips(od->lb_ips, lb_dps->lb); > > + build_lrouter_lb_reachable_ips(od, lb_dps->lb); > > + } > > + > > + UUIDSET_FOR_EACH (uuidnode, &codlb->assoc_lbgrps) { > > + lbgrp_dps = ovn_lb_group_datapaths_find(lbgrp_datapaths_map, > > + &uuidnode->uuid); > > + ovs_assert(lbgrp_dps); > > + ovn_lb_group_datapaths_add_lr(lbgrp_dps, od); > > + > > + /* Associate all the lbs of the lbgrp to the datapath 'od' */ > > + for (size_t j = 0; j < lbgrp_dps->lb_group->n_lbs; j++) { > > + const struct uuid *lb_uuid > > + = &lbgrp_dps->lb_group->lbs[j]->nlb->header_.uuid; > > + lb_dps = ovn_lb_datapaths_find(lb_datapaths_map, > lb_uuid); > > + ovs_assert(lb_dps); > > + ovn_lb_datapaths_add_lr(lb_dps, 1, &od); > > + > > + /* Add the lb_ips of lb_dps to the od. */ > > + build_lrouter_lb_ips(od->lb_ips, lb_dps->lb); > > + build_lrouter_lb_reachable_ips(od, lb_dps->lb); > > + } > > + } > > + > > + /* Re-evaluate 'od->has_lb_vip' */ > > + init_lb_for_datapath(od); > > + } > > + > > HMAP_FOR_EACH (clb, hmap_node, &trk_lb_data->crupdated_lbs) { > > lb = clb->lb; > > const struct uuid *lb_uuid = &lb->nlb->header_.uuid; > > @@ -5638,6 +5807,29 @@ northd_handle_lb_data_changes(struct > tracked_lb_data *trk_lb_data, > > /* Re-evaluate 'od->has_lb_vip' */ > > init_lb_for_datapath(od); > > } > > + > > + BITMAP_FOR_EACH_1 (index, ods_size(lr_datapaths), > > + lb_dps->nb_lr_map) { > > + od = lr_datapaths->array[index]; > > + /* Re-evaluate 'od->has_lb_vip' */ > > + init_lb_for_datapath(od); > > + > > + /* Update the od->lb_ips with the deleted and inserted > > + * vips (if any). */ > > + remove_ips_from_lb_ip_set(od->lb_ips, lb->routable, > > + &clb->deleted_vips_v4, > > + &clb->deleted_vips_v6); > > + add_ips_to_lb_ip_set(od->lb_ips, lb->routable, > > + &clb->inserted_vips_v4, > > + &clb->inserted_vips_v6); > > + > > + remove_lrouter_lb_reachable_ips(od, lb->neigh_mode, > > + &clb->deleted_vips_v4, > > + &clb->deleted_vips_v6); > > + add_neigh_ips_to_lrouter(od, lb->neigh_mode, > > + &clb->inserted_vips_v4, > > + &clb->inserted_vips_v6); > > + } > > } > > > > HMAP_FOR_EACH (crupdated_lbgrp, hmap_node, > > @@ -5655,8 +5847,19 @@ northd_handle_lb_data_changes(struct > tracked_lb_data *trk_lb_data, > > lb_uuid = &lb->nlb->header_.uuid; > > lb_dps = ovn_lb_datapaths_find(lb_datapaths_map, lb_uuid); > > ovs_assert(lb_dps); > > + for (size_t i = 0; i < lbgrp_dps->n_lr; i++) { > > + od = lbgrp_dps->lr[i]; > > + ovn_lb_datapaths_add_lr(lb_dps, 1, &od); > > + > > + /* Re-evaluate 'od->has_lb_vip' */ > > + init_lb_for_datapath(od); > > + > > + /* Add the lb_ips of lb_dps to the od. */ > > + build_lrouter_lb_ips(od->lb_ips, lb_dps->lb); > > + } > > + > > for (size_t i = 0; i < lbgrp_dps->n_ls; i++) { > > - od = lbgrp_dps->ls[i]; > > + od = lbgrp_dps->ls[i]; > > ovn_lb_datapaths_add_ls(lb_dps, 1, &od); > > > > /* Re-evaluate 'od->has_lb_vip' */ > > diff --git a/northd/northd.h b/northd/northd.h > > index 5d8ac6fea0..4d030b10da 100644 > > --- a/northd/northd.h > > +++ b/northd/northd.h > > @@ -328,6 +328,8 @@ void ovnsb_db_run(struct ovsdb_idl_txn *ovnnb_txn, > > bool northd_handle_ls_changes(struct ovsdb_idl_txn *, > > const struct northd_input *, > > struct northd_data *); > > +bool northd_handle_lr_changes(const struct northd_input *, > > + struct northd_data *); > > void destroy_northd_data_tracked_changes(struct northd_data *); > > void northd_destroy(struct northd_data *data); > > void northd_init(struct northd_data *data); > > diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at > > index 1f8b264bde..399b2d91cf 100644 > > --- a/tests/ovn-northd.at > > +++ b/tests/ovn-northd.at > > @@ -10487,8 +10487,8 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE > > # Add lb1 to lr0 and then disassociate > > check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats > > check ovn-nbctl --wait=sb lr-lb-add lr0 lb1 > > -check_engine_stats lb_data norecompute nocompute > > -check_engine_stats northd recompute nocompute > > +check_engine_stats lb_data norecompute compute > > +check_engine_stats northd norecompute compute > > check_engine_stats lflow recompute nocompute > > CHECK_NO_CHANGE_AFTER_RECOMPUTE > > > > @@ -10496,7 +10496,7 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE > > check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats > > check ovn-nbctl --wait=sb set load_balancer lb1 vips:'"10.0.0.10:80"'='" > 10.0.0.100:80"' > > check_engine_stats lb_data norecompute compute > > -check_engine_stats northd recompute nocompute > > +check_engine_stats northd norecompute compute > > check_engine_stats lflow recompute nocompute > > CHECK_NO_CHANGE_AFTER_RECOMPUTE > > > > @@ -10504,7 +10504,7 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE > > check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats > > check ovn-nbctl --wait=sb clear load_Balancer lb1 vips > > check_engine_stats lb_data norecompute compute > > -check_engine_stats northd recompute nocompute > > +check_engine_stats northd norecompute compute > > check_engine_stats lflow recompute nocompute > > CHECK_NO_CHANGE_AFTER_RECOMPUTE > > > > @@ -10512,7 +10512,7 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE > > check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats > > check ovn-nbctl --wait=sb lb-add lb1 10.0.0.10:80 10.0.0.3:80 > > check_engine_stats lb_data norecompute compute > > -check_engine_stats northd recompute nocompute > > +check_engine_stats northd norecompute compute > > check_engine_stats lflow recompute nocompute > > CHECK_NO_CHANGE_AFTER_RECOMPUTE > > > > @@ -10520,13 +10520,13 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE > > check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats > > check ovn-nbctl --wait=sb lb-add lb1 10.0.0.20:80 10.0.0.30:8080 > > check_engine_stats lb_data norecompute compute > > -check_engine_stats northd recompute nocompute > > +check_engine_stats northd norecompute compute > > check_engine_stats lflow recompute nocompute > > CHECK_NO_CHANGE_AFTER_RECOMPUTE > > > > check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats > > check ovn-nbctl --wait=sb lr-lb-del lr0 lb1 > > -check_engine_stats lb_data norecompute nocompute > > +check_engine_stats lb_data norecompute compute > > check_engine_stats northd recompute nocompute > > check_engine_stats lflow recompute nocompute > > CHECK_NO_CHANGE_AFTER_RECOMPUTE > > @@ -10618,8 +10618,8 @@ check_engine_stats lflow recompute nocompute > > > > check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats > > check ovn-nbctl add logical_router lr0 load_balancer_group $lbg1_uuid > > -check_engine_stats lb_data norecompute nocompute > > -check_engine_stats northd recompute nocompute > > +check_engine_stats lb_data norecompute compute > > +check_engine_stats northd norecompute compute > > check_engine_stats lflow recompute nocompute > > CHECK_NO_CHANGE_AFTER_RECOMPUTE > > > > @@ -10627,7 +10627,7 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE > > check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats > > check ovn-nbctl --wait=sb set load_balancer lb1 vips:'"10.0.0.10:80"'='" > 10.0.0.100:80"' > > check_engine_stats lb_data norecompute compute > > -check_engine_stats northd recompute nocompute > > +check_engine_stats northd norecompute compute > > check_engine_stats lflow recompute nocompute > > CHECK_NO_CHANGE_AFTER_RECOMPUTE > > > > @@ -10635,7 +10635,7 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE > > check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats > > check ovn-nbctl --wait=sb clear load_Balancer lb1 vips > > check_engine_stats lb_data norecompute compute > > -check_engine_stats northd recompute nocompute > > +check_engine_stats northd norecompute compute > > check_engine_stats lflow recompute nocompute > > CHECK_NO_CHANGE_AFTER_RECOMPUTE > > > > @@ -10643,7 +10643,7 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE > > check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats > > check ovn-nbctl --wait=sb lb-add lb1 10.0.0.10:80 10.0.0.3:80 > > check_engine_stats lb_data norecompute compute > > -check_engine_stats northd recompute nocompute > > +check_engine_stats northd norecompute compute > > check_engine_stats lflow recompute nocompute > > CHECK_NO_CHANGE_AFTER_RECOMPUTE > > > > @@ -10651,13 +10651,13 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE > > check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats > > check ovn-nbctl --wait=sb lb-add lb1 10.0.0.20:80 10.0.0.30:8080 > > check_engine_stats lb_data norecompute compute > > -check_engine_stats northd recompute nocompute > > +check_engine_stats northd norecompute compute > > check_engine_stats lflow recompute nocompute > > CHECK_NO_CHANGE_AFTER_RECOMPUTE > > > > check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats > > check ovn-nbctl clear logical_router lr0 load_balancer_group > > -check_engine_stats lb_data norecompute nocompute > > +check_engine_stats lb_data norecompute compute > > check_engine_stats northd recompute nocompute > > check_engine_stats lflow recompute nocompute > > > > @@ -10712,8 +10712,8 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE > > > > check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats > > check ovn-nbctl set logical_router lr1 load_balancer_group=$lbg1_uuid > > -check_engine_stats lb_data norecompute nocompute > > -check_engine_stats northd recompute nocompute > > +check_engine_stats lb_data norecompute compute > > +check_engine_stats northd norecompute compute > > check_engine_stats lflow recompute nocompute > > CHECK_NO_CHANGE_AFTER_RECOMPUTE > > > > @@ -10734,8 +10734,8 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE > > check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats > > check ovn-nbctl --wait=sb lr-lb-add lr1 lb1 > > check ovn-nbctl --wait=sb lr-lb-add lr1 lb2 > > -check_engine_stats lb_data norecompute nocompute > > -check_engine_stats northd recompute nocompute > > +check_engine_stats lb_data norecompute compute > > +check_engine_stats northd norecompute compute > > check_engine_stats lflow recompute nocompute > > CHECK_NO_CHANGE_AFTER_RECOMPUTE > > > > @@ -10748,7 +10748,7 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE > > > > check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats > > check ovn-nbctl --wait=sb lr-lb-del lr1 lb2 > > -check_engine_stats lb_data norecompute nocompute > > +check_engine_stats lb_data norecompute compute > > check_engine_stats northd recompute nocompute > > check_engine_stats lflow recompute nocompute > > CHECK_NO_CHANGE_AFTER_RECOMPUTE > > @@ -10758,7 +10758,7 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE > > check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats > > check ovn-nbctl --wait=sb lb-del lb4 > > check_engine_stats lb_data norecompute compute > > -check_engine_stats northd recompute nocompute > > +check_engine_stats northd norecompute compute > > check_engine_stats lflow recompute nocompute > > CHECK_NO_CHANGE_AFTER_RECOMPUTE > > > > @@ -10767,7 +10767,7 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE > > check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats > > check ovn-nbctl --wait=sb lb-del lb2 > > check_engine_stats lb_data norecompute compute > > -check_engine_stats northd recompute nocompute > > +check_engine_stats northd norecompute compute > > check_engine_stats lflow recompute nocompute > > CHECK_NO_CHANGE_AFTER_RECOMPUTE > > > > -- > > 2.41.0 > > > > _______________________________________________ > > dev mailing list > > dev@openvswitch.org > > https://mail.openvswitch.org/mailman/listinfo/ovs-dev > _______________________________________________ > dev mailing list > dev@openvswitch.org > https://mail.openvswitch.org/mailman/listinfo/ovs-dev
diff --git a/lib/lb.c b/lib/lb.c index e6c9dc2be2..d0d562b6fb 100644 --- a/lib/lb.c +++ b/lib/lb.c @@ -794,6 +794,7 @@ ovn_lb_group_init(struct ovn_lb_group *lb_group, const struct uuid *lb_uuid = &nbrec_lb_group->load_balancer[i]->header_.uuid; lb_group->lbs[i] = ovn_northd_lb_find(lbs, lb_uuid); + lb_group->has_routable_lb |= lb_group->lbs[i]->routable; } } @@ -815,6 +816,7 @@ ovn_lb_group_cleanup(struct ovn_lb_group *lb_group) { ovn_lb_ip_set_destroy(lb_group->lb_ips); lb_group->lb_ips = NULL; + lb_group->has_routable_lb = false; free(lb_group->lbs); } @@ -1022,23 +1024,54 @@ ovn_lb_5tuples_destroy(struct hmap *tuples) void build_lrouter_lb_ips(struct ovn_lb_ip_set *lb_ips, const struct ovn_northd_lb *lb) +{ + add_ips_to_lb_ip_set(lb_ips, lb->routable, &lb->ips_v4, &lb->ips_v6); +} + +void +add_ips_to_lb_ip_set(struct ovn_lb_ip_set *lb_ips, + bool is_routable, + const struct sset *lb_ips_v4, + const struct sset *lb_ips_v6) { const char *ip_address; - SSET_FOR_EACH (ip_address, &lb->ips_v4) { + SSET_FOR_EACH (ip_address, lb_ips_v4) { sset_add(&lb_ips->ips_v4, ip_address); - if (lb->routable) { + if (is_routable) { sset_add(&lb_ips->ips_v4_routable, ip_address); } } - SSET_FOR_EACH (ip_address, &lb->ips_v6) { + SSET_FOR_EACH (ip_address, lb_ips_v6) { sset_add(&lb_ips->ips_v6, ip_address); - if (lb->routable) { + if (is_routable) { sset_add(&lb_ips->ips_v6_routable, ip_address); } } } +void +remove_ips_from_lb_ip_set(struct ovn_lb_ip_set *lb_ips, + bool is_routable, + const struct sset *lb_ips_v4, + const struct sset *lb_ips_v6) +{ + const char *ip_address; + + SSET_FOR_EACH (ip_address, lb_ips_v4) { + sset_find_and_delete(&lb_ips->ips_v4, ip_address); + if (is_routable) { + sset_find_and_delete(&lb_ips->ips_v4_routable, ip_address); + } + } + SSET_FOR_EACH (ip_address, lb_ips_v6) { + sset_find_and_delete(&lb_ips->ips_v6, ip_address); + if (is_routable) { + sset_find_and_delete(&lb_ips->ips_v6_routable, ip_address); + } + } +} + /* lb datapaths functions */ struct ovn_lb_datapaths * ovn_lb_datapaths_create(const struct ovn_northd_lb *lb, size_t n_ls_datapaths, @@ -1079,7 +1112,10 @@ ovn_lb_datapaths_add_lr(struct ovn_lb_datapaths *lb_dps, size_t n, struct ovn_datapath **ods) { for (size_t i = 0; i < n; i++) { - bitmap_set1(lb_dps->nb_lr_map, ods[i]->index); + if (!bitmap_is_set(lb_dps->nb_lr_map, ods[i]->index)) { + bitmap_set1(lb_dps->nb_lr_map, ods[i]->index); + lb_dps->n_nb_lr++; + } } } diff --git a/lib/lb.h b/lib/lb.h index 74905c73b7..b8e3c1e8fb 100644 --- a/lib/lb.h +++ b/lib/lb.h @@ -138,6 +138,14 @@ void ovn_northd_lb_reinit(struct ovn_northd_lb *, void build_lrouter_lb_ips(struct ovn_lb_ip_set *, const struct ovn_northd_lb *); +void add_ips_to_lb_ip_set(struct ovn_lb_ip_set *lb_ips, + bool is_routable, + const struct sset *lb_ips_v4, + const struct sset *lb_ips_v6); +void remove_ips_from_lb_ip_set(struct ovn_lb_ip_set *lb_ips, + bool is_routable, + const struct sset *lb_ips_v4, + const struct sset *lb_ips_v6); struct ovn_lb_group { struct hmap_node hmap_node; @@ -145,6 +153,7 @@ struct ovn_lb_group { size_t n_lbs; struct ovn_northd_lb **lbs; struct ovn_lb_ip_set *lb_ips; + bool has_routable_lb; }; struct ovn_lb_group *ovn_lb_group_create( diff --git a/northd/en-lb-data.c b/northd/en-lb-data.c index fd09e719d8..250cec848b 100644 --- a/northd/en-lb-data.c +++ b/northd/en-lb-data.c @@ -40,20 +40,30 @@ static void build_lbs(const struct nbrec_load_balancer_table *, const struct nbrec_load_balancer_group_table *, struct hmap *lbs, struct hmap *lb_groups); static void build_od_lb_map(const struct nbrec_logical_switch_table *, - struct hmap *od_lb_map); + const struct nbrec_logical_router_table *, + struct hmap *ls_lb_map, struct hmap *lr_lb_map); static struct od_lb_data *find_od_lb_data(struct hmap *od_lb_map, const struct uuid *od_uuid); static void destroy_od_lb_data(struct od_lb_data *od_lb_data); static struct od_lb_data *create_od_lb_data(struct hmap *od_lb_map, const struct uuid *od_uuid); +static void handle_od_lb_changes(struct nbrec_load_balancer **, + size_t n_nbrec_lbs, + struct od_lb_data *od_lb_data, + struct ed_type_lb_data *lb_data, + struct crupdated_od_lb_data *); +static void handle_od_lbgrp_changes(struct nbrec_load_balancer_group **, + size_t n_nbrec_lbs, + struct od_lb_data *, + struct ed_type_lb_data *lb_data, + struct crupdated_od_lb_data *); static struct ovn_lb_group *create_lb_group( const struct nbrec_load_balancer_group *, struct hmap *lbs, struct hmap *lb_groups); static void destroy_tracked_data(struct ed_type_lb_data *); -static void add_crupdated_lb_to_tracked_data(struct ovn_northd_lb *, - struct tracked_lb_data *, - bool health_checks); +static struct crupdated_lb *add_crupdated_lb_to_tracked_data( + struct ovn_northd_lb *, struct tracked_lb_data *, bool health_checks); static void add_deleted_lb_to_tracked_data(struct ovn_northd_lb *, struct tracked_lb_data *, bool health_checks); @@ -64,6 +74,8 @@ static void add_deleted_lbgrp_to_tracked_data( struct ovn_lb_group *, struct tracked_lb_data *); static bool is_ls_lbs_changed(const struct nbrec_logical_switch *nbs); static bool is_ls_lbgrps_changed(const struct nbrec_logical_switch *nbs); +static bool is_lr_lbs_changed(const struct nbrec_logical_router *); +static bool is_lr_lbgrps_changed(const struct nbrec_logical_router *); /* 'lb_data' engine node manages the NB load balancers and load balancer * groups. For each NB LB, it creates 'struct ovn_northd_lb' and @@ -92,10 +104,13 @@ en_lb_data_run(struct engine_node *node, void *data) EN_OVSDB_GET(engine_get_input("NB_load_balancer_group", node)); const struct nbrec_logical_switch_table *nb_ls_table = EN_OVSDB_GET(engine_get_input("NB_logical_switch", node)); + const struct nbrec_logical_router_table *nb_lr_table = + EN_OVSDB_GET(engine_get_input("NB_logical_router", node)); lb_data->tracked = false; build_lbs(nb_lb_table, nb_lbg_table, &lb_data->lbs, &lb_data->lbgrps); - build_od_lb_map(nb_ls_table, &lb_data->ls_lb_map); + build_od_lb_map(nb_ls_table, nb_lr_table, &lb_data->ls_lb_map, + &lb_data->lr_lb_map); engine_set_node_state(node, EN_UPDATED); } @@ -137,6 +152,7 @@ lb_data_load_balancer_handler(struct engine_node *node, void *data) uuid_hash(&tracked_lb->header_.uuid)); add_crupdated_lb_to_tracked_data(lb, trk_lb_data, lb->health_checks); + trk_lb_data->has_routable_lb |= lb->routable; } else if (nbrec_load_balancer_is_deleted(tracked_lb)) { lb = ovn_northd_lb_find(&lb_data->lbs, &tracked_lb->header_.uuid); @@ -144,15 +160,44 @@ lb_data_load_balancer_handler(struct engine_node *node, void *data) hmap_remove(&lb_data->lbs, &lb->hmap_node); add_deleted_lb_to_tracked_data(lb, trk_lb_data, lb->health_checks); + trk_lb_data->has_routable_lb |= lb->routable; } else { /* Load balancer updated. */ lb = ovn_northd_lb_find(&lb_data->lbs, &tracked_lb->header_.uuid); ovs_assert(lb); bool health_checks = lb->health_checks; + struct sset old_ips_v4 = SSET_INITIALIZER(&old_ips_v4); + struct sset old_ips_v6 = SSET_INITIALIZER(&old_ips_v6); + sset_swap(&lb->ips_v4, &old_ips_v4); + sset_swap(&lb->ips_v6, &old_ips_v6); ovn_northd_lb_reinit(lb, tracked_lb); health_checks |= lb->health_checks; - add_crupdated_lb_to_tracked_data(lb, trk_lb_data, health_checks); + struct crupdated_lb *clb = add_crupdated_lb_to_tracked_data( + lb, trk_lb_data, health_checks); + trk_lb_data->has_routable_lb |= lb->routable; + + /* Determine the inserted and deleted vips and store them in + * the tracked data. */ + const char *vip; + SSET_FOR_EACH (vip, &lb->ips_v4) { + if (!sset_find_and_delete(&old_ips_v4, vip)) { + sset_add(&clb->inserted_vips_v4, vip); + } + } + + sset_swap(&old_ips_v4, &clb->deleted_vips_v4); + + SSET_FOR_EACH (vip, &lb->ips_v6) { + if (!sset_find_and_delete(&old_ips_v6, vip)) { + sset_add(&clb->inserted_vips_v6, vip); + } + } + + sset_swap(&old_ips_v6, &clb->deleted_vips_v6); + + sset_destroy(&old_ips_v4); + sset_destroy(&old_ips_v6); } } @@ -181,6 +226,8 @@ lb_data_load_balancer_group_handler(struct engine_node *node, void *data) for (size_t i = 0; i < lb_group->n_lbs; i++) { hmapx_add(&clbg->assoc_lbs, lb_group->lbs[i]); } + + trk_lb_data->has_routable_lb |= lb_group->has_routable_lb; } else if (nbrec_load_balancer_group_is_deleted(tracked_lb_group)) { struct ovn_lb_group *lb_group; lb_group = ovn_lb_group_find(&lb_data->lbgrps, @@ -188,6 +235,7 @@ lb_data_load_balancer_group_handler(struct engine_node *node, void *data) ovs_assert(lb_group); hmap_remove(&lb_data->lbgrps, &lb_group->hmap_node); add_deleted_lbgrp_to_tracked_data(lb_group, trk_lb_data); + trk_lb_data->has_routable_lb |= lb_group->has_routable_lb; } else { struct ovn_lb_group *lb_group; @@ -209,6 +257,7 @@ lb_data_load_balancer_group_handler(struct engine_node *node, void *data) build_lrouter_lb_ips(lb_group->lb_ips, lb_group->lbs[i]); } + trk_lb_data->has_routable_lb |= lb_group->has_routable_lb; struct crupdated_lbgrp *clbg = add_crupdated_lbgrp_to_tracked_data(lb_group, trk_lb_data); @@ -270,6 +319,7 @@ lb_data_logical_switch_handler(struct engine_node *node, void *data) if (!ls_lbs_changed && !ls_lbgrps_changed) { continue; } + changed = true; struct crupdated_od_lb_data *codlb = xzalloc(sizeof *codlb); codlb->od_uuid = nbs->header_.uuid; uuidset_init(&codlb->assoc_lbs); @@ -283,66 +333,77 @@ lb_data_logical_switch_handler(struct engine_node *node, void *data) } if (ls_lbs_changed) { - struct uuidset *pre_lb_uuids = od_lb_data->lbs; - od_lb_data->lbs = xzalloc(sizeof *od_lb_data->lbs); - uuidset_init(od_lb_data->lbs); - - for (size_t i = 0; i < nbs->n_load_balancer; i++) { - const struct uuid *lb_uuid = - &nbs->load_balancer[i]->header_.uuid; - uuidset_insert(od_lb_data->lbs, lb_uuid); - - struct uuidset_node *unode = uuidset_find(pre_lb_uuids, - lb_uuid); - - if (!unode || (nbrec_load_balancer_row_get_seqno( - nbs->load_balancer[i], - OVSDB_IDL_CHANGE_MODIFY) > 0)) { - /* Add this lb to the tracked data. */ - uuidset_insert(&codlb->assoc_lbs, lb_uuid); - changed = true; - } - - if (unode) { - uuidset_delete(pre_lb_uuids, unode); - } - } - if (!uuidset_is_empty(pre_lb_uuids)) { - trk_lb_data->has_dissassoc_lbs_from_od = true; - changed = true; - } - - uuidset_destroy(pre_lb_uuids); - free(pre_lb_uuids); + handle_od_lb_changes(nbs->load_balancer, nbs->n_load_balancer, + od_lb_data, lb_data, codlb); } if (ls_lbgrps_changed) { - struct uuidset *pre_lbgrp_uuids = od_lb_data->lbgrps; - od_lb_data->lbgrps = xzalloc(sizeof *od_lb_data->lbgrps); - uuidset_init(od_lb_data->lbgrps); - for (size_t i = 0; i < nbs->n_load_balancer_group; i++) { - const struct uuid *lbg_uuid = - &nbs->load_balancer_group[i]->header_.uuid; - uuidset_insert(od_lb_data->lbgrps, lbg_uuid); - - if (!uuidset_find_and_delete(pre_lbgrp_uuids, - lbg_uuid)) { - /* Add this lb group to the tracked data. */ - uuidset_insert(&codlb->assoc_lbgrps, lbg_uuid); - changed = true; - } - } + handle_od_lbgrp_changes(nbs->load_balancer_group, + nbs->n_load_balancer_group, + od_lb_data, lb_data, codlb); + } - if (!uuidset_is_empty(pre_lbgrp_uuids)) { - trk_lb_data->has_dissassoc_lbgrps_from_od = true; - changed = true; - } + ovs_list_insert(&trk_lb_data->crupdated_ls_lbs, &codlb->list_node); + } + } + + if (changed) { + engine_set_node_state(node, EN_UPDATED); + } + return true; +} + +bool +lb_data_logical_router_handler(struct engine_node *node, void *data) +{ + struct ed_type_lb_data *lb_data = (struct ed_type_lb_data *) data; + const struct nbrec_logical_router_table *nbrec_lr_table = + EN_OVSDB_GET(engine_get_input("NB_logical_router", node)); + + struct tracked_lb_data *trk_lb_data = &lb_data->tracked_lb_data; + lb_data->tracked = true; - uuidset_destroy(pre_lbgrp_uuids); - free(pre_lbgrp_uuids); + bool changed = false; + const struct nbrec_logical_router *nbr; + NBREC_LOGICAL_ROUTER_TABLE_FOR_EACH_TRACKED (nbr, nbrec_lr_table) { + if (nbrec_logical_router_is_deleted(nbr)) { + struct od_lb_data *od_lb_data = + find_od_lb_data(&lb_data->lr_lb_map, &nbr->header_.uuid); + if (od_lb_data) { + hmap_remove(&lb_data->lr_lb_map, &od_lb_data->hmap_node); + hmapx_add(&trk_lb_data->deleted_od_lb_data, od_lb_data); + } + } else { + bool lr_lbs_changed = is_lr_lbs_changed(nbr); + bool lr_lbgrps_changed = is_lr_lbgrps_changed(nbr); + if (!lr_lbs_changed && !lr_lbgrps_changed) { + continue; } + changed = true; + struct crupdated_od_lb_data *codlb = xzalloc(sizeof *codlb); + codlb->od_uuid = nbr->header_.uuid; + uuidset_init(&codlb->assoc_lbs); + uuidset_init(&codlb->assoc_lbgrps); - ovs_list_insert(&trk_lb_data->crupdated_ls_lbs, &codlb->list_node); + struct od_lb_data *od_lb_data = + find_od_lb_data(&lb_data->lr_lb_map, &nbr->header_.uuid); + if (!od_lb_data) { + od_lb_data = create_od_lb_data(&lb_data->lr_lb_map, + &nbr->header_.uuid); + } + + if (lr_lbs_changed) { + handle_od_lb_changes(nbr->load_balancer, nbr->n_load_balancer, + od_lb_data, lb_data, codlb); + } + + if (lr_lbgrps_changed) { + handle_od_lbgrp_changes(nbr->load_balancer_group, + nbr->n_load_balancer_group, + od_lb_data, lb_data, codlb); + } + + ovs_list_insert(&trk_lb_data->crupdated_lr_lbs, &codlb->list_node); } } @@ -359,6 +420,7 @@ lb_data_init(struct ed_type_lb_data *lb_data) hmap_init(&lb_data->lbs); hmap_init(&lb_data->lbgrps); hmap_init(&lb_data->ls_lb_map); + hmap_init(&lb_data->lr_lb_map); struct tracked_lb_data *trk_lb_data = &lb_data->tracked_lb_data; hmap_init(&trk_lb_data->crupdated_lbs); @@ -366,6 +428,7 @@ lb_data_init(struct ed_type_lb_data *lb_data) hmap_init(&trk_lb_data->crupdated_lbgrps); hmapx_init(&trk_lb_data->deleted_lbgrps); ovs_list_init(&trk_lb_data->crupdated_ls_lbs); + ovs_list_init(&trk_lb_data->crupdated_lr_lbs); hmapx_init(&trk_lb_data->deleted_od_lb_data); } @@ -390,6 +453,11 @@ lb_data_destroy(struct ed_type_lb_data *lb_data) } hmap_destroy(&lb_data->ls_lb_map); + HMAP_FOR_EACH_POP (od_lb_data, hmap_node, &lb_data->lr_lb_map) { + destroy_od_lb_data(od_lb_data); + } + hmap_destroy(&lb_data->lr_lb_map); + destroy_tracked_data(lb_data); hmap_destroy(&lb_data->tracked_lb_data.crupdated_lbs); hmapx_destroy(&lb_data->tracked_lb_data.deleted_lbs); @@ -435,7 +503,8 @@ create_lb_group(const struct nbrec_load_balancer_group *nbrec_lb_group, static void build_od_lb_map(const struct nbrec_logical_switch_table *nbrec_ls_table, - struct hmap *od_lb_map) + const struct nbrec_logical_router_table *nbrec_lr_table, + struct hmap *ls_lb_map, struct hmap *lr_lb_map) { const struct nbrec_logical_switch *nbrec_ls; NBREC_LOGICAL_SWITCH_TABLE_FOR_EACH (nbrec_ls, nbrec_ls_table) { @@ -443,17 +512,35 @@ build_od_lb_map(const struct nbrec_logical_switch_table *nbrec_ls_table, continue; } - struct od_lb_data *od_lb_data = - create_od_lb_data(od_lb_map, &nbrec_ls->header_.uuid); + struct od_lb_data *ls_lb_data = + create_od_lb_data(ls_lb_map, &nbrec_ls->header_.uuid); for (size_t i = 0; i < nbrec_ls->n_load_balancer; i++) { - uuidset_insert(od_lb_data->lbs, + uuidset_insert(ls_lb_data->lbs, &nbrec_ls->load_balancer[i]->header_.uuid); } for (size_t i = 0; i < nbrec_ls->n_load_balancer_group; i++) { - uuidset_insert(od_lb_data->lbgrps, + uuidset_insert(ls_lb_data->lbgrps, &nbrec_ls->load_balancer_group[i]->header_.uuid); } } + + const struct nbrec_logical_router *nbrec_lr; + NBREC_LOGICAL_ROUTER_TABLE_FOR_EACH (nbrec_lr, nbrec_lr_table) { + if (!nbrec_lr->n_load_balancer && !nbrec_lr->n_load_balancer_group) { + continue; + } + + struct od_lb_data *lr_lb_data = + create_od_lb_data(lr_lb_map, &nbrec_lr->header_.uuid); + for (size_t i = 0; i < nbrec_lr->n_load_balancer; i++) { + uuidset_insert(lr_lb_data->lbs, + &nbrec_lr->load_balancer[i]->header_.uuid); + } + for (size_t i = 0; i < nbrec_lr->n_load_balancer_group; i++) { + uuidset_insert(lr_lb_data->lbgrps, + &nbrec_lr->load_balancer_group[i]->header_.uuid); + } + } } static struct od_lb_data * @@ -495,6 +582,84 @@ destroy_od_lb_data(struct od_lb_data *od_lb_data) free(od_lb_data); } +static void +handle_od_lb_changes(struct nbrec_load_balancer **nbrec_lbs, + size_t n_nbrec_lbs, struct od_lb_data *od_lb_data, + struct ed_type_lb_data *lb_data, + struct crupdated_od_lb_data *codlb) +{ + struct tracked_lb_data *trk_lb_data = &lb_data->tracked_lb_data; + struct uuidset *pre_lb_uuids = od_lb_data->lbs; + od_lb_data->lbs = xzalloc(sizeof *od_lb_data->lbs); + uuidset_init(od_lb_data->lbs); + + for (size_t i = 0; i < n_nbrec_lbs; i++) { + const struct uuid *lb_uuid = &nbrec_lbs[i]->header_.uuid; + uuidset_insert(od_lb_data->lbs, lb_uuid); + + struct uuidset_node *unode = uuidset_find(pre_lb_uuids, lb_uuid); + + if (!unode || (nbrec_load_balancer_row_get_seqno( + nbrec_lbs[i], OVSDB_IDL_CHANGE_MODIFY) > 0)) { + /* Add this lb to the tracked data. */ + uuidset_insert(&codlb->assoc_lbs, lb_uuid); + + if (!trk_lb_data->has_routable_lb) { + struct ovn_northd_lb *lb = ovn_northd_lb_find(&lb_data->lbs, + lb_uuid); + ovs_assert(lb); + trk_lb_data->has_routable_lb |= lb->routable; + } + } + + if (unode) { + uuidset_delete(pre_lb_uuids, unode); + } + } + + if (!uuidset_is_empty(pre_lb_uuids)) { + trk_lb_data->has_dissassoc_lbs_from_od = true; + } + + uuidset_destroy(pre_lb_uuids); + free(pre_lb_uuids); +} + +static void +handle_od_lbgrp_changes(struct nbrec_load_balancer_group **nbrec_lbgrps, + size_t n_nbrec_lbgrps, struct od_lb_data *od_lb_data, + struct ed_type_lb_data *lb_data, + struct crupdated_od_lb_data *codlb) +{ + struct tracked_lb_data *trk_lb_data = &lb_data->tracked_lb_data; + struct uuidset *pre_lbgrp_uuids = od_lb_data->lbgrps; + od_lb_data->lbgrps = xzalloc(sizeof *od_lb_data->lbgrps); + uuidset_init(od_lb_data->lbgrps); + for (size_t i = 0; i < n_nbrec_lbgrps; i++) { + const struct uuid *lbgrp_uuid = &nbrec_lbgrps[i]->header_.uuid; + uuidset_insert(od_lb_data->lbgrps, lbgrp_uuid); + + if (!uuidset_find_and_delete(pre_lbgrp_uuids, lbgrp_uuid)) { + /* Add this lb group to the tracked data. */ + uuidset_insert(&codlb->assoc_lbgrps, lbgrp_uuid); + + if (!trk_lb_data->has_routable_lb) { + struct ovn_lb_group *lbgrp = + ovn_lb_group_find(&lb_data->lbgrps, lbgrp_uuid); + ovs_assert(lbgrp); + trk_lb_data->has_routable_lb |= lbgrp->has_routable_lb; + } + } + } + + if (!uuidset_is_empty(pre_lbgrp_uuids)) { + trk_lb_data->has_dissassoc_lbgrps_from_od = true; + } + + uuidset_destroy(pre_lbgrp_uuids); + free(pre_lbgrp_uuids); +} + static void destroy_tracked_data(struct ed_type_lb_data *lb_data) { @@ -503,6 +668,7 @@ destroy_tracked_data(struct ed_type_lb_data *lb_data) lb_data->tracked_lb_data.has_dissassoc_lbs_from_lbgrps = false; lb_data->tracked_lb_data.has_dissassoc_lbs_from_od = false; lb_data->tracked_lb_data.has_dissassoc_lbgrps_from_od = false; + lb_data->tracked_lb_data.has_routable_lb = false; struct hmapx_node *node; HMAPX_FOR_EACH_SAFE (node, &lb_data->tracked_lb_data.deleted_lbs) { @@ -518,6 +684,10 @@ destroy_tracked_data(struct ed_type_lb_data *lb_data) struct crupdated_lb *clb; HMAP_FOR_EACH_POP (clb, hmap_node, &lb_data->tracked_lb_data.crupdated_lbs) { + sset_destroy(&clb->inserted_vips_v4); + sset_destroy(&clb->inserted_vips_v6); + sset_destroy(&clb->deleted_vips_v4); + sset_destroy(&clb->deleted_vips_v6); free(clb); } @@ -537,13 +707,21 @@ destroy_tracked_data(struct ed_type_lb_data *lb_data) free(codlb); } + LIST_FOR_EACH_SAFE (codlb, list_node, + &lb_data->tracked_lb_data.crupdated_lr_lbs) { + ovs_list_remove(&codlb->list_node); + uuidset_destroy(&codlb->assoc_lbs); + uuidset_destroy(&codlb->assoc_lbgrps); + free(codlb); + } + HMAPX_FOR_EACH_SAFE (node, &lb_data->tracked_lb_data.deleted_od_lb_data) { destroy_od_lb_data(node->data); hmapx_delete(&lb_data->tracked_lb_data.deleted_od_lb_data, node); } } -static void +static struct crupdated_lb * add_crupdated_lb_to_tracked_data(struct ovn_northd_lb *lb, struct tracked_lb_data *tracked_lb_data, bool health_checks) @@ -552,9 +730,15 @@ add_crupdated_lb_to_tracked_data(struct ovn_northd_lb *lb, clb->lb = lb; hmap_insert(&tracked_lb_data->crupdated_lbs, &clb->hmap_node, uuid_hash(&lb->nlb->header_.uuid)); + sset_init(&clb->inserted_vips_v4); + sset_init(&clb->inserted_vips_v6); + sset_init(&clb->deleted_vips_v4); + sset_init(&clb->deleted_vips_v6); if (health_checks) { tracked_lb_data->has_health_checks = true; } + + return clb; } static void @@ -600,3 +784,17 @@ is_ls_lbgrps_changed(const struct nbrec_logical_switch *nbs) { || nbrec_logical_switch_is_updated(nbs, NBREC_LOGICAL_SWITCH_COL_LOAD_BALANCER_GROUP)); } + +static bool +is_lr_lbs_changed(const struct nbrec_logical_router *nbr) { + return ((nbrec_logical_router_is_new(nbr) && nbr->n_load_balancer) + || nbrec_logical_router_is_updated(nbr, + NBREC_LOGICAL_ROUTER_COL_LOAD_BALANCER)); +} + +static bool +is_lr_lbgrps_changed(const struct nbrec_logical_router *nbr) { + return ((nbrec_logical_router_is_new(nbr) && nbr->n_load_balancer_group) + || nbrec_logical_router_is_updated(nbr, + NBREC_LOGICAL_ROUTER_COL_LOAD_BALANCER_GROUP)); +} diff --git a/northd/en-lb-data.h b/northd/en-lb-data.h index ebfd547d77..9d20a539ba 100644 --- a/northd/en-lb-data.h +++ b/northd/en-lb-data.h @@ -6,6 +6,7 @@ #include "openvswitch/hmap.h" #include "include/openvswitch/list.h" #include "lib/hmapx.h" +#include "lib/sset.h" #include "lib/uuidset.h" #include "lib/inc-proc-eng.h" @@ -17,6 +18,10 @@ struct crupdated_lb { struct hmap_node hmap_node; struct ovn_northd_lb *lb; + struct sset inserted_vips_v4; + struct sset inserted_vips_v6; + struct sset deleted_vips_v4; + struct sset deleted_vips_v6; }; struct crupdated_lbgrp { @@ -54,6 +59,10 @@ struct tracked_lb_data { * 'struct crupdated_od_lb_data' */ struct ovs_list crupdated_ls_lbs; + /* List of logical router <-> lb changes. List node is + * 'struct crupdated_od_lb_data' */ + struct ovs_list crupdated_lr_lbs; + /* hmapx of deleted logical switches which have load balancer or lb groups * associated with it. hmapx_node is 'struct od_lb_data'. */ struct hmapx deleted_od_lb_data; @@ -70,6 +79,9 @@ struct tracked_lb_data { /* Indicates if a lb group was disassociated from a logical switch. */ bool has_dissassoc_lbgrps_from_od; + + /* Indicates if any lb (in the tracked data) has 'routable' flag set. */ + bool has_routable_lb; }; /* Datapath (logical switch) to lb/lbgrp association data. */ @@ -90,6 +102,7 @@ struct ed_type_lb_data { /* hmap of ls to lb map. hmap node is 'struct od_lb_data'. */ struct hmap ls_lb_map; + struct hmap lr_lb_map; /* tracked data*/ bool tracked; @@ -104,5 +117,6 @@ void en_lb_data_clear_tracked_data(void *data); bool lb_data_load_balancer_handler(struct engine_node *, void *data); bool lb_data_load_balancer_group_handler(struct engine_node *, void *data); bool lb_data_logical_switch_handler(struct engine_node *, void *data); +bool lb_data_logical_router_handler(struct engine_node *, void *data); #endif /* end of EN_NORTHD_LB_DATA_H */ diff --git a/northd/en-northd.c b/northd/en-northd.c index 8487b003f7..aa0f20f0c2 100644 --- a/northd/en-northd.c +++ b/northd/en-northd.c @@ -196,6 +196,26 @@ northd_sb_port_binding_handler(struct engine_node *node, return true; } +bool +northd_nb_logical_router_handler(struct engine_node *node, + void *data) +{ + struct northd_data *nd = data; + struct northd_input input_data; + + northd_get_input_data(node, &input_data); + + if (!northd_handle_lr_changes(&input_data, nd)) { + return false; + } + + if (nd->change_tracked) { + engine_set_node_state(node, EN_UPDATED); + } + + return true; +} + bool northd_lb_data_handler(struct engine_node *node, void *data) { diff --git a/northd/en-northd.h b/northd/en-northd.h index 3c77b64bb2..5a88871760 100644 --- a/northd/en-northd.h +++ b/northd/en-northd.h @@ -16,6 +16,7 @@ void en_northd_cleanup(void *data); void en_northd_clear_tracked_data(void *data); bool northd_nb_nb_global_handler(struct engine_node *, void *data OVS_UNUSED); bool northd_nb_logical_switch_handler(struct engine_node *, void *data); +bool northd_nb_logical_router_handler(struct engine_node *, void *data); bool northd_sb_port_binding_handler(struct engine_node *, void *data); bool northd_lb_data_handler(struct engine_node *, void *data); diff --git a/northd/inc-proc-northd.c b/northd/inc-proc-northd.c index e9e28c4bea..8b08171179 100644 --- a/northd/inc-proc-northd.c +++ b/northd/inc-proc-northd.c @@ -158,8 +158,9 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, lb_data_load_balancer_group_handler); engine_add_input(&en_lb_data, &en_nb_logical_switch, lb_data_logical_switch_handler); + engine_add_input(&en_lb_data, &en_nb_logical_router, + lb_data_logical_router_handler); - engine_add_input(&en_northd, &en_nb_logical_router, NULL); engine_add_input(&en_northd, &en_nb_mirror, NULL); engine_add_input(&en_northd, &en_nb_static_mac_binding, NULL); engine_add_input(&en_northd, &en_nb_chassis_template_var, NULL); @@ -184,6 +185,8 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, northd_nb_nb_global_handler); engine_add_input(&en_northd, &en_nb_logical_switch, northd_nb_logical_switch_handler); + engine_add_input(&en_northd, &en_nb_logical_router, + northd_nb_logical_router_handler); engine_add_input(&en_northd, &en_lb_data, northd_lb_data_handler); engine_add_input(&en_mac_binding_aging, &en_nb_nb_global, NULL); diff --git a/northd/northd.c b/northd/northd.c index f6258d4830..23d11d9d97 100644 --- a/northd/northd.c +++ b/northd/northd.c @@ -4167,54 +4167,63 @@ static bool lrouter_port_ipv4_reachable(const struct ovn_port *op, ovs_be32 addr); static bool lrouter_port_ipv6_reachable(const struct ovn_port *op, const struct in6_addr *addr); + static void -build_lrouter_lb_reachable_ips(struct ovn_datapath *od, - const struct ovn_northd_lb *lb) +add_neigh_ips_to_lrouter(struct ovn_datapath *od, + enum lb_neighbor_responder_mode neigh_mode, + const struct sset *lb_ips_v4, + const struct sset *lb_ips_v6) { /* If configured to not reply to any neighbor requests for all VIPs * return early. */ - if (lb->neigh_mode == LB_NEIGH_RESPOND_NONE) { + if (neigh_mode == LB_NEIGH_RESPOND_NONE) { return; } + const char *ip_address; + /* If configured to reply to neighbor requests for all VIPs force them * all to be considered "reachable". */ - if (lb->neigh_mode == LB_NEIGH_RESPOND_ALL) { - for (size_t i = 0; i < lb->n_vips; i++) { - if (lb->vips[i].address_family == AF_INET) { - sset_add(&od->lb_ips->ips_v4_reachable, lb->vips[i].vip_str); - } else { - sset_add(&od->lb_ips->ips_v6_reachable, lb->vips[i].vip_str); - } + if (neigh_mode == LB_NEIGH_RESPOND_ALL) { + SSET_FOR_EACH (ip_address, lb_ips_v4) { + sset_add(&od->lb_ips->ips_v4_reachable, ip_address); + } + SSET_FOR_EACH (ip_address, lb_ips_v6) { + sset_add(&od->lb_ips->ips_v6_reachable, ip_address); } + return; } /* Otherwise, a VIP is reachable if there's at least one router * subnet that includes it. */ - ovs_assert(lb->neigh_mode == LB_NEIGH_RESPOND_REACHABLE); - for (size_t i = 0; i < lb->n_vips; i++) { - if (lb->vips[i].address_family == AF_INET) { - ovs_be32 vip_ip4 = in6_addr_get_mapped_ipv4(&lb->vips[i].vip); - struct ovn_port *op; + ovs_assert(neigh_mode == LB_NEIGH_RESPOND_REACHABLE); + SSET_FOR_EACH (ip_address, lb_ips_v4) { + struct ovn_port *op; + ovs_be32 vip_ip4; + if (ip_parse(ip_address, &vip_ip4)) { HMAP_FOR_EACH (op, dp_node, &od->ports) { if (lrouter_port_ipv4_reachable(op, vip_ip4)) { sset_add(&od->lb_ips->ips_v4_reachable, - lb->vips[i].vip_str); + ip_address); break; } } - } else { - struct ovn_port *op; + } + } + SSET_FOR_EACH (ip_address, lb_ips_v6) { + struct ovn_port *op; + struct in6_addr vip; + if (ipv6_parse(ip_address, &vip)) { HMAP_FOR_EACH (op, dp_node, &od->ports) { - if (lrouter_port_ipv6_reachable(op, &lb->vips[i].vip)) { + if (lrouter_port_ipv6_reachable(op, &vip)) { sset_add(&od->lb_ips->ips_v6_reachable, - lb->vips[i].vip_str); + ip_address); break; } } @@ -4222,6 +4231,34 @@ build_lrouter_lb_reachable_ips(struct ovn_datapath *od, } } +static void +remove_lrouter_lb_reachable_ips(struct ovn_datapath *od, + enum lb_neighbor_responder_mode neigh_mode, + const struct sset *lb_ips_v4, + const struct sset *lb_ips_v6) +{ + if (neigh_mode == LB_NEIGH_RESPOND_NONE) { + return; + } + + const char *ip_address; + SSET_FOR_EACH (ip_address, lb_ips_v4) { + sset_find_and_delete(&od->lb_ips->ips_v4_reachable, ip_address); + } + SSET_FOR_EACH (ip_address, lb_ips_v6) { + sset_find_and_delete(&od->lb_ips->ips_v6_reachable, ip_address); + } +} + +static void +build_lrouter_lb_reachable_ips(struct ovn_datapath *od, + const struct ovn_northd_lb *lb) +{ + add_neigh_ips_to_lrouter(od, lb->neigh_mode, &lb->ips_v4, + &lb->ips_v6); +} + + static void build_lrouter_lbs_check(const struct ovn_datapaths *lr_datapaths) { @@ -5419,6 +5456,95 @@ fail: return false; } +/* Returns true if the logical router has changes which are not + * incrementally handled. + * Presently supports i-p for the below changes: + * - load balancers and load balancer groups. + */ +static bool +check_unsupported_inc_proc_for_lr_changes( + const struct nbrec_logical_router *lr) +{ + /* Check if the columns are changed in this row. */ + enum nbrec_logical_router_column_id col; + for (col = 0; col < NBREC_LOGICAL_ROUTER_N_COLUMNS; col++) { + if (nbrec_logical_router_is_updated(lr, col)) { + if (col == NBREC_LOGICAL_ROUTER_COL_LOAD_BALANCER || + col == NBREC_LOGICAL_ROUTER_COL_LOAD_BALANCER_GROUP) { + continue; + } + return true; + } + } + + /* Check if the referenced rows are changed. + XXX: Need a better OVSDB IDL interface for this check. */ + for (size_t i = 0; i < lr->n_ports; i++) { + if (nbrec_logical_router_port_row_get_seqno(lr->ports[i], + OVSDB_IDL_CHANGE_MODIFY) > 0) { + return true; + } + } + if (lr->copp && nbrec_copp_row_get_seqno(lr->copp, + OVSDB_IDL_CHANGE_MODIFY) > 0) { + return true; + } + for (size_t i = 0; i < lr->n_nat; i++) { + if (nbrec_nat_row_get_seqno(lr->nat[i], + OVSDB_IDL_CHANGE_MODIFY) > 0) { + return true; + } + } + for (size_t i = 0; i < lr->n_policies; i++) { + if (nbrec_logical_router_policy_row_get_seqno(lr->policies[i], + OVSDB_IDL_CHANGE_MODIFY) > 0) { + return true; + } + } + for (size_t i = 0; i < lr->n_static_routes; i++) { + if (nbrec_logical_router_static_route_row_get_seqno( + lr->static_routes[i], OVSDB_IDL_CHANGE_MODIFY) > 0) { + return true; + } + } + return false; +} + +/* Return true if changes are handled incrementally, false otherwise. + * When there are any changes, try to track what's exactly changed and set + * northd_data->change_tracked accordingly: change tracked - true, otherwise, + * false. + * Note: Changes to load balancer and load balancer groups associated with + * the logical routers are handled separately in the lb_data change + * handlers (northd_handle_lb_data_changes_pre_od and + * northd_handle_lb_data_changes_post_od). + * */ +bool +northd_handle_lr_changes(const struct northd_input *ni, + struct northd_data *nd) +{ + const struct nbrec_logical_router *changed_lr; + + NBREC_LOGICAL_ROUTER_TABLE_FOR_EACH_TRACKED (changed_lr, + ni->nbrec_logical_router_table) { + if (nbrec_logical_router_is_new(changed_lr) || + nbrec_logical_router_is_deleted(changed_lr)) { + goto fail; + } + + /* Presently only able to handle load balancer and + * load balancer group changes. */ + if (check_unsupported_inc_proc_for_lr_changes(changed_lr)) { + goto fail; + } + } + + return true; +fail: + destroy_northd_data_tracked_changes(nd); + return false; +} + bool northd_handle_sb_port_binding_changes( const struct sbrec_port_binding_table *sbrec_port_binding_table, @@ -5535,6 +5661,10 @@ northd_handle_lb_data_changes(struct tracked_lb_data *trk_lb_data, return false; } + if (trk_lb_data->has_routable_lb) { + return false; + } + struct ovn_lb_datapaths *lb_dps; struct ovn_northd_lb *lb; struct ovn_datapath *od; @@ -5625,6 +5755,45 @@ northd_handle_lb_data_changes(struct tracked_lb_data *trk_lb_data, init_lb_for_datapath(od); } + LIST_FOR_EACH (codlb, list_node, &trk_lb_data->crupdated_lr_lbs) { + od = ovn_datapath_find(&lr_datapaths->datapaths, &codlb->od_uuid); + ovs_assert(od); + + struct uuidset_node *uuidnode; + UUIDSET_FOR_EACH (uuidnode, &codlb->assoc_lbs) { + lb_dps = ovn_lb_datapaths_find(lb_datapaths_map, &uuidnode->uuid); + ovs_assert(lb_dps); + ovn_lb_datapaths_add_lr(lb_dps, 1, &od); + + /* Add the lb_ips of lb_dps to the od. */ + build_lrouter_lb_ips(od->lb_ips, lb_dps->lb); + build_lrouter_lb_reachable_ips(od, lb_dps->lb); + } + + UUIDSET_FOR_EACH (uuidnode, &codlb->assoc_lbgrps) { + lbgrp_dps = ovn_lb_group_datapaths_find(lbgrp_datapaths_map, + &uuidnode->uuid); + ovs_assert(lbgrp_dps); + ovn_lb_group_datapaths_add_lr(lbgrp_dps, od); + + /* Associate all the lbs of the lbgrp to the datapath 'od' */ + for (size_t j = 0; j < lbgrp_dps->lb_group->n_lbs; j++) { + const struct uuid *lb_uuid + = &lbgrp_dps->lb_group->lbs[j]->nlb->header_.uuid; + lb_dps = ovn_lb_datapaths_find(lb_datapaths_map, lb_uuid); + ovs_assert(lb_dps); + ovn_lb_datapaths_add_lr(lb_dps, 1, &od); + + /* Add the lb_ips of lb_dps to the od. */ + build_lrouter_lb_ips(od->lb_ips, lb_dps->lb); + build_lrouter_lb_reachable_ips(od, lb_dps->lb); + } + } + + /* Re-evaluate 'od->has_lb_vip' */ + init_lb_for_datapath(od); + } + HMAP_FOR_EACH (clb, hmap_node, &trk_lb_data->crupdated_lbs) { lb = clb->lb; const struct uuid *lb_uuid = &lb->nlb->header_.uuid; @@ -5638,6 +5807,29 @@ northd_handle_lb_data_changes(struct tracked_lb_data *trk_lb_data, /* Re-evaluate 'od->has_lb_vip' */ init_lb_for_datapath(od); } + + BITMAP_FOR_EACH_1 (index, ods_size(lr_datapaths), + lb_dps->nb_lr_map) { + od = lr_datapaths->array[index]; + /* Re-evaluate 'od->has_lb_vip' */ + init_lb_for_datapath(od); + + /* Update the od->lb_ips with the deleted and inserted + * vips (if any). */ + remove_ips_from_lb_ip_set(od->lb_ips, lb->routable, + &clb->deleted_vips_v4, + &clb->deleted_vips_v6); + add_ips_to_lb_ip_set(od->lb_ips, lb->routable, + &clb->inserted_vips_v4, + &clb->inserted_vips_v6); + + remove_lrouter_lb_reachable_ips(od, lb->neigh_mode, + &clb->deleted_vips_v4, + &clb->deleted_vips_v6); + add_neigh_ips_to_lrouter(od, lb->neigh_mode, + &clb->inserted_vips_v4, + &clb->inserted_vips_v6); + } } HMAP_FOR_EACH (crupdated_lbgrp, hmap_node, @@ -5655,8 +5847,19 @@ northd_handle_lb_data_changes(struct tracked_lb_data *trk_lb_data, lb_uuid = &lb->nlb->header_.uuid; lb_dps = ovn_lb_datapaths_find(lb_datapaths_map, lb_uuid); ovs_assert(lb_dps); + for (size_t i = 0; i < lbgrp_dps->n_lr; i++) { + od = lbgrp_dps->lr[i]; + ovn_lb_datapaths_add_lr(lb_dps, 1, &od); + + /* Re-evaluate 'od->has_lb_vip' */ + init_lb_for_datapath(od); + + /* Add the lb_ips of lb_dps to the od. */ + build_lrouter_lb_ips(od->lb_ips, lb_dps->lb); + } + for (size_t i = 0; i < lbgrp_dps->n_ls; i++) { - od = lbgrp_dps->ls[i]; + od = lbgrp_dps->ls[i]; ovn_lb_datapaths_add_ls(lb_dps, 1, &od); /* Re-evaluate 'od->has_lb_vip' */ diff --git a/northd/northd.h b/northd/northd.h index 5d8ac6fea0..4d030b10da 100644 --- a/northd/northd.h +++ b/northd/northd.h @@ -328,6 +328,8 @@ void ovnsb_db_run(struct ovsdb_idl_txn *ovnnb_txn, bool northd_handle_ls_changes(struct ovsdb_idl_txn *, const struct northd_input *, struct northd_data *); +bool northd_handle_lr_changes(const struct northd_input *, + struct northd_data *); void destroy_northd_data_tracked_changes(struct northd_data *); void northd_destroy(struct northd_data *data); void northd_init(struct northd_data *data); diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at index 1f8b264bde..399b2d91cf 100644 --- a/tests/ovn-northd.at +++ b/tests/ovn-northd.at @@ -10487,8 +10487,8 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE # Add lb1 to lr0 and then disassociate check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lr-lb-add lr0 lb1 -check_engine_stats lb_data norecompute nocompute -check_engine_stats northd recompute nocompute +check_engine_stats lb_data norecompute compute +check_engine_stats northd norecompute compute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10496,7 +10496,7 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb set load_balancer lb1 vips:'"10.0.0.10:80"'='"10.0.0.100:80"' check_engine_stats lb_data norecompute compute -check_engine_stats northd recompute nocompute +check_engine_stats northd norecompute compute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10504,7 +10504,7 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb clear load_Balancer lb1 vips check_engine_stats lb_data norecompute compute -check_engine_stats northd recompute nocompute +check_engine_stats northd norecompute compute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10512,7 +10512,7 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lb-add lb1 10.0.0.10:80 10.0.0.3:80 check_engine_stats lb_data norecompute compute -check_engine_stats northd recompute nocompute +check_engine_stats northd norecompute compute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10520,13 +10520,13 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lb-add lb1 10.0.0.20:80 10.0.0.30:8080 check_engine_stats lb_data norecompute compute -check_engine_stats northd recompute nocompute +check_engine_stats northd norecompute compute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lr-lb-del lr0 lb1 -check_engine_stats lb_data norecompute nocompute +check_engine_stats lb_data norecompute compute check_engine_stats northd recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10618,8 +10618,8 @@ check_engine_stats lflow recompute nocompute check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl add logical_router lr0 load_balancer_group $lbg1_uuid -check_engine_stats lb_data norecompute nocompute -check_engine_stats northd recompute nocompute +check_engine_stats lb_data norecompute compute +check_engine_stats northd norecompute compute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10627,7 +10627,7 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb set load_balancer lb1 vips:'"10.0.0.10:80"'='"10.0.0.100:80"' check_engine_stats lb_data norecompute compute -check_engine_stats northd recompute nocompute +check_engine_stats northd norecompute compute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10635,7 +10635,7 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb clear load_Balancer lb1 vips check_engine_stats lb_data norecompute compute -check_engine_stats northd recompute nocompute +check_engine_stats northd norecompute compute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10643,7 +10643,7 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lb-add lb1 10.0.0.10:80 10.0.0.3:80 check_engine_stats lb_data norecompute compute -check_engine_stats northd recompute nocompute +check_engine_stats northd norecompute compute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10651,13 +10651,13 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lb-add lb1 10.0.0.20:80 10.0.0.30:8080 check_engine_stats lb_data norecompute compute -check_engine_stats northd recompute nocompute +check_engine_stats northd norecompute compute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl clear logical_router lr0 load_balancer_group -check_engine_stats lb_data norecompute nocompute +check_engine_stats lb_data norecompute compute check_engine_stats northd recompute nocompute check_engine_stats lflow recompute nocompute @@ -10712,8 +10712,8 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl set logical_router lr1 load_balancer_group=$lbg1_uuid -check_engine_stats lb_data norecompute nocompute -check_engine_stats northd recompute nocompute +check_engine_stats lb_data norecompute compute +check_engine_stats northd norecompute compute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10734,8 +10734,8 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lr-lb-add lr1 lb1 check ovn-nbctl --wait=sb lr-lb-add lr1 lb2 -check_engine_stats lb_data norecompute nocompute -check_engine_stats northd recompute nocompute +check_engine_stats lb_data norecompute compute +check_engine_stats northd norecompute compute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10748,7 +10748,7 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lr-lb-del lr1 lb2 -check_engine_stats lb_data norecompute nocompute +check_engine_stats lb_data norecompute compute check_engine_stats northd recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10758,7 +10758,7 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lb-del lb4 check_engine_stats lb_data norecompute compute -check_engine_stats northd recompute nocompute +check_engine_stats northd norecompute compute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10767,7 +10767,7 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lb-del lb2 check_engine_stats lb_data norecompute compute -check_engine_stats northd recompute nocompute +check_engine_stats northd norecompute compute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE