From patchwork Thu Jan 11 15:28:27 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Numan Siddique X-Patchwork-Id: 1885670 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::138; helo=smtp1.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp1.osuosl.org (smtp1.osuosl.org [IPv6:2605:bc80:3010::138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4T9pVD2b5Tz1yPf for ; Fri, 12 Jan 2024 02:28:52 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 6F06F813B2; Thu, 11 Jan 2024 15:28:50 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 6F06F813B2 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id X_LfkKen5arn; Thu, 11 Jan 2024 15:28:48 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp1.osuosl.org (Postfix) with ESMTPS id 7D48781330; Thu, 11 Jan 2024 15:28:47 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 7D48781330 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 4FB44C007C; Thu, 11 Jan 2024 15:28:47 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp1.osuosl.org (smtp1.osuosl.org [IPv6:2605:bc80:3010::138]) by lists.linuxfoundation.org (Postfix) with ESMTP id C2CEAC0077 for ; Thu, 11 Jan 2024 15:28:45 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 9150B81330 for ; Thu, 11 Jan 2024 15:28:45 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 9150B81330 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id qGGdt64TlAMZ for ; Thu, 11 Jan 2024 15:28:43 +0000 (UTC) Received: from relay2-d.mail.gandi.net (relay2-d.mail.gandi.net [217.70.183.194]) by smtp1.osuosl.org (Postfix) with ESMTPS id 6DA718130A for ; Thu, 11 Jan 2024 15:28:43 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 6DA718130A Received: by mail.gandi.net (Postfix) with ESMTPSA id ECC9540008; Thu, 11 Jan 2024 15:28:39 +0000 (UTC) From: numans@ovn.org To: dev@openvswitch.org Date: Thu, 11 Jan 2024 10:28:27 -0500 Message-ID: <20240111152827.2789937-1-numans@ovn.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240111152752.2789854-1-numans@ovn.org> References: <20240111152752.2789854-1-numans@ovn.org> MIME-Version: 1.0 X-GND-Sasl: numans@ovn.org Subject: [ovs-dev] [PATCH ovn v5 01/16] northd: Refactor the northd change tracking. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Numan Siddique northd engine tracking data now has the following tracking data - changed ovn_ports (right now only changed logical switch ports are tracked.) - changed load balancers. This separation becomes easier to add lflow handling for these changes in lflow northd engine handler. This patch doesn't handle the load balancer changes in lflow handler. It will be handled in upcoming commits. Before this patch, any changes to load balancers or lb groups resulted in full recompute of 'sync_to_sb_lb' and 'lflow' engine nodes. Now this scenario is optimized and it results in recomputes of these nodes only if 'northd' engine node adds any changed load balancers in its tracked data. One example is created or updating an empty load balancer group. Signed-off-by: Numan Siddique Acked-by: Dumitru Ceara Acked-by: Han Zhou --- northd/en-lflow.c | 12 +- northd/en-northd.c | 18 +- northd/en-sync-from-sb.c | 2 +- northd/en-sync-sb.c | 9 +- northd/northd.c | 436 ++++++++++++++++++++------------------- northd/northd.h | 86 +++++--- tests/ovn-northd.at | 10 +- 7 files changed, 313 insertions(+), 260 deletions(-) diff --git a/northd/en-lflow.c b/northd/en-lflow.c index 2b84fef0ef..6ba26006e0 100644 --- a/northd/en-lflow.c +++ b/northd/en-lflow.c @@ -104,12 +104,12 @@ lflow_northd_handler(struct engine_node *node, void *data) { struct northd_data *northd_data = engine_get_input_data("northd", node); - if (!northd_data->change_tracked) { + if (!northd_has_tracked_data(&northd_data->trk_data)) { return false; } - /* Fall back to recompute if lb related data has changed. */ - if (northd_data->lb_changed) { + /* Fall back to recompute if load balancers have changed. */ + if (northd_has_lbs_in_tracked_data(&northd_data->trk_data)) { return false; } @@ -119,9 +119,9 @@ lflow_northd_handler(struct engine_node *node, struct lflow_input lflow_input; lflow_get_input_data(node, &lflow_input); - if (!lflow_handle_northd_ls_changes(eng_ctx->ovnsb_idl_txn, - &northd_data->tracked_ls_changes, - &lflow_input, &lflow_data->lflows)) { + if (!lflow_handle_northd_port_changes(eng_ctx->ovnsb_idl_txn, + &northd_data->trk_data.trk_lsps, + &lflow_input, &lflow_data->lflows)) { return false; } diff --git a/northd/en-northd.c b/northd/en-northd.c index aa0f20f0c2..28559ed211 100644 --- a/northd/en-northd.c +++ b/northd/en-northd.c @@ -171,7 +171,7 @@ northd_nb_logical_switch_handler(struct engine_node *node, return false; } - if (nd->change_tracked) { + if (northd_has_lsps_in_tracked_data(&nd->trk_data)) { engine_set_node_state(node, EN_UPDATED); } @@ -209,10 +209,6 @@ northd_nb_logical_router_handler(struct engine_node *node, return false; } - if (nd->change_tracked) { - engine_set_node_state(node, EN_UPDATED); - } - return true; } @@ -230,15 +226,15 @@ northd_lb_data_handler(struct engine_node *node, void *data) &nd->ls_datapaths, &nd->lr_datapaths, &nd->lb_datapaths_map, - &nd->lb_group_datapaths_map)) { + &nd->lb_group_datapaths_map, + &nd->trk_data)) { return false; } - /* Indicate the depedendant engine nodes that load balancer/group - * related data has changed (including association to logical - * switch/router). */ - nd->lb_changed = true; - engine_set_node_state(node, EN_UPDATED); + if (northd_has_lbs_in_tracked_data(&nd->trk_data)) { + engine_set_node_state(node, EN_UPDATED); + } + return true; } diff --git a/northd/en-sync-from-sb.c b/northd/en-sync-from-sb.c index 48b1576173..8c05239b49 100644 --- a/northd/en-sync-from-sb.c +++ b/northd/en-sync-from-sb.c @@ -65,7 +65,7 @@ sync_from_sb_northd_handler(struct engine_node *node, void *data OVS_UNUSED) { struct northd_data *nd = engine_get_input_data("northd", node); - if (nd->change_tracked) { + if (northd_has_tracked_data(&nd->trk_data)) { /* So far the changes tracked in northd don't impact this node. * * In particular, for the LS related changes, the only field this node diff --git a/northd/en-sync-sb.c b/northd/en-sync-sb.c index 2ec3bf54f8..3aaab8d005 100644 --- a/northd/en-sync-sb.c +++ b/northd/en-sync-sb.c @@ -236,7 +236,8 @@ sync_to_sb_lb_northd_handler(struct engine_node *node, void *data OVS_UNUSED) { struct northd_data *nd = engine_get_input_data("northd", node); - if (!nd->change_tracked || nd->lb_changed) { + if (!northd_has_tracked_data(&nd->trk_data) || + northd_has_lbs_in_tracked_data(&nd->trk_data)) { /* Return false if no tracking data or if lbs changed. */ return false; } @@ -306,11 +307,13 @@ sync_to_sb_pb_northd_handler(struct engine_node *node, void *data OVS_UNUSED) } struct northd_data *nd = engine_get_input_data("northd", node); - if (!nd->change_tracked) { + if (!northd_has_tracked_data(&nd->trk_data) || + northd_has_lbs_in_tracked_data(&nd->trk_data)) { + /* Return false if no tracking data or if lbs changed. */ return false; } - if (!sync_pbs_for_northd_ls_changes(&nd->tracked_ls_changes)) { + if (!sync_pbs_for_northd_changed_ovn_ports(&nd->trk_data.trk_lsps)) { return false; } diff --git a/northd/northd.c b/northd/northd.c index 952f8200d4..f32e3bf21a 100644 --- a/northd/northd.c +++ b/northd/northd.c @@ -4910,21 +4910,20 @@ sync_pbs(struct ovsdb_idl_txn *ovnsb_idl_txn, struct hmap *ls_ports) } /* Sync the SB Port bindings for the added and updated logical switch ports - * of the tracked logical switches (from the northd engine node). */ + * of the tracked northd engine data. */ bool -sync_pbs_for_northd_ls_changes(struct tracked_ls_changes *ls_changes) +sync_pbs_for_northd_changed_ovn_ports( struct tracked_ovn_ports *trk_ovn_ports) { - struct ls_change *ls_change; - LIST_FOR_EACH (ls_change, list_node, &ls_changes->updated) { - struct ovn_port *op; - - LIST_FOR_EACH (op, list, &ls_change->added_ports) { - sync_pb_for_op(op); - } + struct hmapx_node *hmapx_node; + struct ovn_port *op; + HMAPX_FOR_EACH (hmapx_node, &trk_ovn_ports->created) { + op = hmapx_node->data; + sync_pb_for_op(op); + } - LIST_FOR_EACH (op, list, &ls_change->updated_ports) { - sync_pb_for_op(op); - } + HMAPX_FOR_EACH (hmapx_node, &trk_ovn_ports->updated) { + op = hmapx_node->data; + sync_pb_for_op(op); } return true; @@ -5126,34 +5125,68 @@ build_ports(struct ovsdb_idl_txn *ovnsb_txn, } static void -destroy_tracked_ls_change(struct ls_change *ls_change) +destroy_tracked_ovn_ports(struct tracked_ovn_ports *trk_ovn_ports) { - struct ovn_port *op; - LIST_FOR_EACH (op, list, &ls_change->added_ports) { - ovs_list_remove(&op->list); - } - LIST_FOR_EACH (op, list, &ls_change->updated_ports) { - ovs_list_remove(&op->list); + struct hmapx_node *hmapx_node; + HMAPX_FOR_EACH_SAFE (hmapx_node, &trk_ovn_ports->deleted) { + ovn_port_destroy_orphan(hmapx_node->data); + hmapx_delete(&trk_ovn_ports->deleted, hmapx_node); } - LIST_FOR_EACH_SAFE (op, list, &ls_change->deleted_ports) { - ovs_list_remove(&op->list); - ovn_port_destroy_orphan(op); + + hmapx_clear(&trk_ovn_ports->created); + hmapx_clear(&trk_ovn_ports->updated); +} + +static void +destroy_tracked_lbs(struct tracked_lbs *trk_lbs) +{ + struct hmapx_node *hmapx_node; + HMAPX_FOR_EACH_SAFE (hmapx_node, &trk_lbs->deleted) { + ovn_lb_datapaths_destroy(hmapx_node->data); + hmapx_delete(&trk_lbs->deleted, hmapx_node); } + + hmapx_clear(&trk_lbs->crupdated); +} + +static void +add_op_to_northd_tracked_ports(struct hmapx *tracked_ovn_ports, + struct ovn_port *op) +{ + hmapx_add(tracked_ovn_ports, op); } void destroy_northd_data_tracked_changes(struct northd_data *nd) { - struct ls_change *ls_change; - LIST_FOR_EACH_SAFE (ls_change, list_node, - &nd->tracked_ls_changes.updated) { - destroy_tracked_ls_change(ls_change); - ovs_list_remove(&ls_change->list_node); - free(ls_change); - } + struct northd_tracked_data *trk_changes = &nd->trk_data; + destroy_tracked_ovn_ports(&trk_changes->trk_lsps); + destroy_tracked_lbs(&trk_changes->trk_lbs); + nd->trk_data.type = NORTHD_TRACKED_NONE; +} + +static void +init_northd_tracked_data(struct northd_data *nd) +{ + struct northd_tracked_data *trk_data = &nd->trk_data; + trk_data->type = NORTHD_TRACKED_NONE; + hmapx_init(&trk_data->trk_lsps.created); + hmapx_init(&trk_data->trk_lsps.updated); + hmapx_init(&trk_data->trk_lsps.deleted); + hmapx_init(&trk_data->trk_lbs.crupdated); + hmapx_init(&trk_data->trk_lbs.deleted); +} - nd->change_tracked = false; - nd->lb_changed = false; +static void +destroy_northd_tracked_data(struct northd_data *nd) +{ + struct northd_tracked_data *trk_data = &nd->trk_data; + trk_data->type = NORTHD_TRACKED_NONE; + hmapx_destroy(&trk_data->trk_lsps.created); + hmapx_destroy(&trk_data->trk_lsps.updated); + hmapx_destroy(&trk_data->trk_lsps.deleted); + hmapx_destroy(&trk_data->trk_lbs.crupdated); + hmapx_destroy(&trk_data->trk_lbs.deleted); } /* Check if a changed LSP can be handled incrementally within the I-P engine @@ -5358,12 +5391,11 @@ check_lsp_changes_other_than_up(const struct nbrec_logical_switch_port *nbsp) */ static bool ls_handle_lsp_changes(struct ovsdb_idl_txn *ovnsb_idl_txn, - const struct nbrec_logical_switch *changed_ls, - const struct northd_input *ni, - struct northd_data *nd, - struct ovn_datapath *od, - struct ls_change *ls_change, - bool *updated) + const struct nbrec_logical_switch *changed_ls, + const struct northd_input *ni, + struct northd_data *nd, + struct ovn_datapath *od, + struct tracked_ovn_ports *trk_lsps) { bool ls_ports_changed = false; if (!nbrec_logical_switch_is_updated(changed_ls, @@ -5384,7 +5416,7 @@ ls_handle_lsp_changes(struct ovsdb_idl_txn *ovnsb_idl_txn, return true; } - ls_change->had_only_router_ports = (od->n_router_ports + bool ls_had_only_router_ports = (od->n_router_ports && (od->n_router_ports == hmap_count(&od->ports))); struct ovn_port *op; @@ -5410,8 +5442,7 @@ ls_handle_lsp_changes(struct ovsdb_idl_txn *ovnsb_idl_txn, if (!op) { goto fail; } - ovs_list_push_back(&ls_change->added_ports, - &op->list); + add_op_to_northd_tracked_ports(&trk_lsps->created, op); } else if (ls_port_has_changed(new_nbsp)) { /* Existing port updated */ bool temp = false; @@ -5447,7 +5478,7 @@ ls_handle_lsp_changes(struct ovsdb_idl_txn *ovnsb_idl_txn, if (!op) { goto fail; } - ovs_list_push_back(&ls_change->updated_ports, &op->list); + add_op_to_northd_tracked_ports(&trk_lsps->updated, op); } op->visited = true; } @@ -5456,15 +5487,14 @@ ls_handle_lsp_changes(struct ovsdb_idl_txn *ovnsb_idl_txn, HMAP_FOR_EACH_SAFE (op, dp_node, &od->ports) { if (!op->visited) { if (!op->lsp_can_be_inc_processed) { - goto fail_clean_deleted; + goto fail; } if (sset_contains(&nd->svc_monitor_lsps, op->key)) { /* This port was used for svc monitor, which may be * impacted by this deletion. Fallback to recompute. */ - goto fail_clean_deleted; + goto fail; } - ovs_list_push_back(&ls_change->deleted_ports, - &op->list); + add_op_to_northd_tracked_ports(&trk_lsps->deleted, op); hmap_remove(&nd->ls_ports, &op->key_node); hmap_remove(&od->ports, &op->dp_node); sbrec_port_binding_delete(op->sb); @@ -5473,27 +5503,32 @@ ls_handle_lsp_changes(struct ovsdb_idl_txn *ovnsb_idl_txn, } } - if (!ovs_list_is_empty(&ls_change->added_ports) || - !ovs_list_is_empty(&ls_change->updated_ports) || - !ovs_list_is_empty(&ls_change->deleted_ports)) { - *updated = true; + bool ls_has_only_router_ports = (od->n_router_ports + && (od->n_router_ports == hmap_count(&od->ports))); + + /* There are lflows related to router ports that depends on whether + * there are switch ports on the logical switch (see + * build_lswitch_rport_arp_req_flow() for more details). Check if this + * dependency has changed and if it has, then add the router ports + * to the tracked 'updated' ovn ports so that lflow engine can + * regenerate lflows for these router ports. */ + if (ls_had_only_router_ports != ls_has_only_router_ports) { + for (size_t i = 0; i < od->n_router_ports; i++) { + op = od->router_ports[i]; + add_op_to_northd_tracked_ports(&trk_lsps->updated, op); + } } return true; -fail_clean_deleted: - LIST_FOR_EACH_POP (op, list, &ls_change->deleted_ports) { - ovn_port_destroy_orphan(op); - } - fail: + destroy_tracked_ovn_ports(trk_lsps); return false; } /* Return true if changes are handled incrementally, false otherwise. * When there are any changes, try to track what's exactly changed and set - * northd_data->change_tracked accordingly: change tracked - true, otherwise, - * false. + * northd_data->trk_data accordingly. * * Note: Changes to load balancer and load balancer groups associated with * the logical switches are handled separately in the lb_data change handlers. @@ -5504,7 +5539,7 @@ northd_handle_ls_changes(struct ovsdb_idl_txn *ovnsb_idl_txn, struct northd_data *nd) { const struct nbrec_logical_switch *changed_ls; - struct ls_change *ls_change = NULL; + struct northd_tracked_data *trk_data = &nd->trk_data; NBREC_LOGICAL_SWITCH_TABLE_FOR_EACH_TRACKED (changed_ls, ni->nbrec_logical_switch_table) { @@ -5528,31 +5563,16 @@ northd_handle_ls_changes(struct ovsdb_idl_txn *ovnsb_idl_txn, goto fail; } - ls_change = xzalloc(sizeof *ls_change); - ls_change->od = od; - ovs_list_init(&ls_change->added_ports); - ovs_list_init(&ls_change->deleted_ports); - ovs_list_init(&ls_change->updated_ports); - - bool updated = false; if (!ls_handle_lsp_changes(ovnsb_idl_txn, changed_ls, - ni, nd, od, ls_change, - &updated)) { - destroy_tracked_ls_change(ls_change); - free(ls_change); + ni, nd, od, &trk_data->trk_lsps)) { goto fail; } - - if (updated) { - ovs_list_push_back(&nd->tracked_ls_changes.updated, - &ls_change->list_node); - } else { - free(ls_change); - } } - if (!ovs_list_is_empty(&nd->tracked_ls_changes.updated)) { - nd->change_tracked = true; + if (!hmapx_is_empty(&trk_data->trk_lsps.created) + || !hmapx_is_empty(&trk_data->trk_lsps.updated) + || !hmapx_is_empty(&trk_data->trk_lsps.deleted)) { + trk_data->type |= NORTHD_TRACKED_PORTS; } return true; @@ -5617,13 +5637,10 @@ lr_changes_can_be_handled( } /* Return true if changes are handled incrementally, false otherwise. - * When there are any changes, try to track what's exactly changed and set - * northd_data->change_tracked accordingly: change tracked - true, otherwise, - * false. + * * Note: Changes to load balancer and load balancer groups associated with * the logical routers are handled separately in the lb_data change - * handlers (northd_handle_lb_data_changes_pre_od and - * northd_handle_lb_data_changes_post_od). + * handler - northd_handle_lb_data_changes(). * */ bool northd_handle_lr_changes(const struct northd_input *ni, @@ -5729,7 +5746,8 @@ northd_handle_lb_data_changes(struct tracked_lb_data *trk_lb_data, struct ovn_datapaths *ls_datapaths, struct ovn_datapaths *lr_datapaths, struct hmap *lb_datapaths_map, - struct hmap *lbgrp_datapaths_map) + struct hmap *lbgrp_datapaths_map, + struct northd_tracked_data *nd_changes) { if (trk_lb_data->has_health_checks) { /* Fall back to recompute since a tracked load balancer @@ -5792,7 +5810,9 @@ northd_handle_lb_data_changes(struct tracked_lb_data *trk_lb_data, } hmap_remove(lb_datapaths_map, &lb_dps->hmap_node); - ovn_lb_datapaths_destroy(lb_dps); + + /* Add the deleted lb to the northd tracked data. */ + hmapx_add(&nd_changes->trk_lbs.deleted, lb_dps); } /* Create the 'lb_dps' if not already created for each @@ -5809,6 +5829,9 @@ northd_handle_lb_data_changes(struct tracked_lb_data *trk_lb_data, hmap_insert(lb_datapaths_map, &lb_dps->hmap_node, uuid_hash(lb_uuid)); } + + /* Add the updated lb to the northd tracked data. */ + hmapx_add(&nd_changes->trk_lbs.crupdated, lb_dps); } struct ovn_lb_group_datapaths *lbgrp_dps; @@ -5839,6 +5862,9 @@ northd_handle_lb_data_changes(struct tracked_lb_data *trk_lb_data, lb_dps = ovn_lb_datapaths_find(lb_datapaths_map, &uuidnode->uuid); ovs_assert(lb_dps); ovn_lb_datapaths_add_ls(lb_dps, 1, &od); + + /* Add the lb to the northd tracked data. */ + hmapx_add(&nd_changes->trk_lbs.crupdated, lb_dps); } UUIDSET_FOR_EACH (uuidnode, &codlb->assoc_lbgrps) { @@ -5854,6 +5880,9 @@ northd_handle_lb_data_changes(struct tracked_lb_data *trk_lb_data, lb_dps = ovn_lb_datapaths_find(lb_datapaths_map, lb_uuid); ovs_assert(lb_dps); ovn_lb_datapaths_add_ls(lb_dps, 1, &od); + + /* Add the lb to the northd tracked data. */ + hmapx_add(&nd_changes->trk_lbs.crupdated, lb_dps); } } @@ -5874,6 +5903,9 @@ northd_handle_lb_data_changes(struct tracked_lb_data *trk_lb_data, /* Add the lb_ips of lb_dps to the od. */ build_lrouter_lb_ips(od->lb_ips, lb_dps->lb); build_lrouter_lb_reachable_ips(od, lb_dps->lb); + + /* Add the lb to the northd tracked data. */ + hmapx_add(&nd_changes->trk_lbs.crupdated, lb_dps); } UUIDSET_FOR_EACH (uuidnode, &codlb->assoc_lbgrps) { @@ -5893,6 +5925,9 @@ northd_handle_lb_data_changes(struct tracked_lb_data *trk_lb_data, /* Add the lb_ips of lb_dps to the od. */ build_lrouter_lb_ips(od->lb_ips, lb_dps->lb); build_lrouter_lb_reachable_ips(od, lb_dps->lb); + + /* Add the lb to the northd tracked data. */ + hmapx_add(&nd_changes->trk_lbs.crupdated, lb_dps); } } @@ -5971,9 +6006,17 @@ northd_handle_lb_data_changes(struct tracked_lb_data *trk_lb_data, /* Re-evaluate 'od->has_lb_vip' */ init_lb_for_datapath(od); } + + /* Add the lb to the northd tracked data. */ + hmapx_add(&nd_changes->trk_lbs.crupdated, lb_dps); } } + if (!hmapx_is_empty(&nd_changes->trk_lbs.crupdated) + || !hmapx_is_empty(&nd_changes->trk_lbs.deleted)) { + nd_changes->type |= NORTHD_TRACKED_LBS; + } + return true; } @@ -17029,149 +17072,122 @@ delete_lflow_for_lsp(struct ovn_port *op, bool is_update, return true; } -bool lflow_handle_northd_ls_changes(struct ovsdb_idl_txn *ovnsb_txn, - struct tracked_ls_changes *ls_changes, - struct lflow_input *lflow_input, - struct hmap *lflows) +bool +lflow_handle_northd_port_changes(struct ovsdb_idl_txn *ovnsb_txn, + struct tracked_ovn_ports *trk_lsps, + struct lflow_input *lflow_input, + struct hmap *lflows) { - struct ls_change *ls_change; + struct hmapx_node *hmapx_node; + struct ovn_port *op; - LIST_FOR_EACH (ls_change, list_node, &ls_changes->updated) { - const struct sbrec_multicast_group *sbmc_flood = - mcast_group_lookup(lflow_input->sbrec_mcast_group_by_name_dp, - MC_FLOOD, ls_change->od->sb); - const struct sbrec_multicast_group *sbmc_flood_l2 = - mcast_group_lookup(lflow_input->sbrec_mcast_group_by_name_dp, - MC_FLOOD_L2, ls_change->od->sb); - const struct sbrec_multicast_group *sbmc_unknown = - mcast_group_lookup(lflow_input->sbrec_mcast_group_by_name_dp, - MC_UNKNOWN, ls_change->od->sb); + HMAPX_FOR_EACH (hmapx_node, &trk_lsps->deleted) { + op = hmapx_node->data; + /* Make sure 'op' is an lsp and not lrp. */ + ovs_assert(op->nbsp); - struct ovn_port *op; - LIST_FOR_EACH (op, list, &ls_change->deleted_ports) { - if (!delete_lflow_for_lsp(op, false, - lflow_input->sbrec_logical_flow_table, - lflows)) { + if (!delete_lflow_for_lsp(op, false, + lflow_input->sbrec_logical_flow_table, + lflows)) { return false; } - /* No need to update SB multicast groups, thanks to weak - * references. */ - } + /* No need to update SB multicast groups, thanks to weak + * references. */ + } - LIST_FOR_EACH (op, list, &ls_change->updated_ports) { - /* Delete old lflows. */ - if (!delete_lflow_for_lsp(op, true, - lflow_input->sbrec_logical_flow_table, - lflows)) { - return false; - } + HMAPX_FOR_EACH (hmapx_node, &trk_lsps->updated) { + op = hmapx_node->data; + /* Make sure 'op' is an lsp and not lrp. */ + ovs_assert(op->nbsp); - /* Generate new lflows. */ - struct ds match = DS_EMPTY_INITIALIZER; - struct ds actions = DS_EMPTY_INITIALIZER; - build_lswitch_and_lrouter_iterate_by_lsp(op, lflow_input->ls_ports, - lflow_input->lr_ports, - lflow_input->meter_groups, - &match, &actions, - lflows); - ds_destroy(&match); - ds_destroy(&actions); - - /* SB port_binding is not deleted, so don't update SB multicast - * groups. */ - - /* Sync the new flows to SB. */ - struct lflow_ref_node *lfrn; - LIST_FOR_EACH (lfrn, lflow_list_node, &op->lflows) { - sync_lsp_lflows_to_sb(ovnsb_txn, lflow_input, lflows, - lfrn->lflow); - } + /* Delete old lflows. */ + if (!delete_lflow_for_lsp(op, true, + lflow_input->sbrec_logical_flow_table, + lflows)) { + return false; } - LIST_FOR_EACH (op, list, &ls_change->added_ports) { - struct ds match = DS_EMPTY_INITIALIZER; - struct ds actions = DS_EMPTY_INITIALIZER; - build_lswitch_and_lrouter_iterate_by_lsp(op, lflow_input->ls_ports, - lflow_input->lr_ports, - lflow_input->meter_groups, - &match, &actions, - lflows); - ds_destroy(&match); - ds_destroy(&actions); - - /* Update SB multicast groups for the new port. */ - if (!sbmc_flood) { - sbmc_flood = create_sb_multicast_group(ovnsb_txn, - ls_change->od->sb, MC_FLOOD, OVN_MCAST_FLOOD_TUNNEL_KEY); - } - sbrec_multicast_group_update_ports_addvalue(sbmc_flood, op->sb); + /* Generate new lflows. */ + struct ds match = DS_EMPTY_INITIALIZER; + struct ds actions = DS_EMPTY_INITIALIZER; + build_lswitch_and_lrouter_iterate_by_lsp(op, lflow_input->ls_ports, + lflow_input->lr_ports, + lflow_input->meter_groups, + &match, &actions, + lflows); + ds_destroy(&match); + ds_destroy(&actions); - if (!sbmc_flood_l2) { - sbmc_flood_l2 = create_sb_multicast_group(ovnsb_txn, - ls_change->od->sb, MC_FLOOD_L2, - OVN_MCAST_FLOOD_L2_TUNNEL_KEY); - } - sbrec_multicast_group_update_ports_addvalue(sbmc_flood_l2, op->sb); + /* SB port_binding is not deleted, so don't update SB multicast + * groups. */ - if (op->has_unknown) { - if (!sbmc_unknown) { - sbmc_unknown = create_sb_multicast_group(ovnsb_txn, - ls_change->od->sb, MC_UNKNOWN, - OVN_MCAST_UNKNOWN_TUNNEL_KEY); - } - sbrec_multicast_group_update_ports_addvalue(sbmc_unknown, - op->sb); - } - - /* Sync the newly added flows to SB. */ - struct lflow_ref_node *lfrn; - LIST_FOR_EACH (lfrn, lflow_list_node, &op->lflows) { - sync_lsp_lflows_to_sb(ovnsb_txn, lflow_input, lflows, - lfrn->lflow); - } + /* Sync the new flows to SB. */ + struct lflow_ref_node *lfrn; + LIST_FOR_EACH (lfrn, lflow_list_node, &op->lflows) { + sync_lsp_lflows_to_sb(ovnsb_txn, lflow_input, lflows, + lfrn->lflow); } + } - bool ls_has_only_router_ports = (ls_change->od->n_router_ports && - (ls_change->od->n_router_ports == - hmap_count(&ls_change->od->ports))); - - if (ls_change->had_only_router_ports != ls_has_only_router_ports) { - /* There are lflows related to router ports that depends on whether - * there are switch ports on the logical switch (see - * build_lswitch_rport_arp_req_flow() for more details). Since this - * dependency changed, we need to regenerate lflows for each router - * port on this logical switch. */ - for (size_t i = 0; i < ls_change->od->n_router_ports; i++) { - op = ls_change->od->router_ports[i]; - - /* Delete old lflows. */ - if (!delete_lflow_for_lsp(op, "affected router", - lflow_input->sbrec_logical_flow_table, - lflows)) { - return false; - } + HMAPX_FOR_EACH (hmapx_node, &trk_lsps->created) { + op = hmapx_node->data; + /* Make sure 'op' is an lsp and not lrp. */ + ovs_assert(op->nbsp); - /* Generate new lflows. */ - struct ds match = DS_EMPTY_INITIALIZER; - struct ds actions = DS_EMPTY_INITIALIZER; - build_lswitch_and_lrouter_iterate_by_lsp(op, - lflow_input->ls_ports, lflow_input->lr_ports, - lflow_input->meter_groups, &match, &actions, lflows); - ds_destroy(&match); - ds_destroy(&actions); - - /* Sync the new flows to SB. */ - struct lflow_ref_node *lfrn; - LIST_FOR_EACH (lfrn, lflow_list_node, &op->lflows) { - sync_lsp_lflows_to_sb(ovnsb_txn, lflow_input, lflows, - lfrn->lflow); - } + const struct sbrec_multicast_group *sbmc_flood = + mcast_group_lookup(lflow_input->sbrec_mcast_group_by_name_dp, + MC_FLOOD, op->od->sb); + const struct sbrec_multicast_group *sbmc_flood_l2 = + mcast_group_lookup(lflow_input->sbrec_mcast_group_by_name_dp, + MC_FLOOD_L2, op->od->sb); + const struct sbrec_multicast_group *sbmc_unknown = + mcast_group_lookup(lflow_input->sbrec_mcast_group_by_name_dp, + MC_UNKNOWN, op->od->sb); + + struct ds match = DS_EMPTY_INITIALIZER; + struct ds actions = DS_EMPTY_INITIALIZER; + build_lswitch_and_lrouter_iterate_by_lsp(op, lflow_input->ls_ports, + lflow_input->lr_ports, + lflow_input->meter_groups, + &match, &actions, + lflows); + ds_destroy(&match); + ds_destroy(&actions); + + /* Update SB multicast groups for the new port. */ + if (!sbmc_flood) { + sbmc_flood = create_sb_multicast_group(ovnsb_txn, + op->od->sb, MC_FLOOD, OVN_MCAST_FLOOD_TUNNEL_KEY); + } + sbrec_multicast_group_update_ports_addvalue(sbmc_flood, op->sb); + + if (!sbmc_flood_l2) { + sbmc_flood_l2 = create_sb_multicast_group(ovnsb_txn, + op->od->sb, MC_FLOOD_L2, + OVN_MCAST_FLOOD_L2_TUNNEL_KEY); + } + sbrec_multicast_group_update_ports_addvalue(sbmc_flood_l2, op->sb); + + if (op->has_unknown) { + if (!sbmc_unknown) { + sbmc_unknown = create_sb_multicast_group(ovnsb_txn, + op->od->sb, MC_UNKNOWN, + OVN_MCAST_UNKNOWN_TUNNEL_KEY); } + sbrec_multicast_group_update_ports_addvalue(sbmc_unknown, + op->sb); + } + + /* Sync the newly added flows to SB. */ + struct lflow_ref_node *lfrn; + LIST_FOR_EACH (lfrn, lflow_list_node, &op->lflows) { + sync_lsp_lflows_to_sb(ovnsb_txn, lflow_input, lflows, + lfrn->lflow); } } - return true; + return true; } static bool @@ -17816,8 +17832,7 @@ northd_init(struct northd_data *data) data->ovn_internal_version_changed = false; sset_init(&data->svc_monitor_lsps); hmap_init(&data->svc_monitor_map); - data->change_tracked = false; - ovs_list_init(&data->tracked_ls_changes.updated); + init_northd_tracked_data(data); } void @@ -17857,6 +17872,7 @@ northd_destroy(struct northd_data *data) destroy_debug_config(); sset_destroy(&data->svc_monitor_lsps); + destroy_northd_tracked_data(data); } void diff --git a/northd/northd.h b/northd/northd.h index 5be7b5384d..23521065e8 100644 --- a/northd/northd.h +++ b/northd/northd.h @@ -83,22 +83,44 @@ struct ovn_datapaths { struct ovn_datapath **array; }; -/* Track what's changed for a single LS. - * Now only track port changes. */ -struct ls_change { - struct ovs_list list_node; - struct ovn_datapath *od; - struct ovs_list added_ports; - struct ovs_list deleted_ports; - struct ovs_list updated_ports; - bool had_only_router_ports; +struct tracked_ovn_ports { + /* tracked created ports. + * hmapx node data is 'struct ovn_port *' */ + struct hmapx created; + + /* tracked updated ports. + * hmapx node data is 'struct ovn_port *' */ + struct hmapx updated; + + /* tracked deleted ports. + * hmapx node data is 'struct ovn_port *' */ + struct hmapx deleted; }; -/* Track what's changed for logical switches. - * Now only track updated ones (added or deleted may be supported in the - * future). */ -struct tracked_ls_changes { - struct ovs_list updated; /* Contains struct ls_change */ +struct tracked_lbs { + /* Tracked created or updated load balancers. + * hmapx node data is 'struct ovn_lb_datapaths' */ + struct hmapx crupdated; + + /* Tracked deleted lbs. + * hmapx node data is 'struct ovn_lb_datapaths' */ + struct hmapx deleted; +}; + +enum northd_tracked_data_type { + NORTHD_TRACKED_NONE, + NORTHD_TRACKED_PORTS = (1 << 0), + NORTHD_TRACKED_LBS = (1 << 1), +}; + +/* Track what's changed in the northd engine node. + * Now only tracks ovn_ports (of vif type) - created, updated + * and deleted. */ +struct northd_tracked_data { + /* Indicates the type of data tracked. One or all of NORTHD_TRACKED_*. */ + enum northd_tracked_data_type type; + struct tracked_ovn_ports trk_lsps; + struct tracked_lbs trk_lbs; }; struct northd_data { @@ -114,10 +136,9 @@ struct northd_data { struct chassis_features features; struct sset svc_monitor_lsps; struct hmap svc_monitor_map; - bool change_tracked; - struct tracked_ls_changes tracked_ls_changes; - bool lb_changed; /* Indicates if load balancers changed or association of - * load balancer to logical switch/router changed. */ + + /* Change tracking data. */ + struct northd_tracked_data trk_data; }; struct lflow_data { @@ -338,9 +359,10 @@ void northd_indices_create(struct northd_data *data, void build_lflows(struct ovsdb_idl_txn *ovnsb_txn, struct lflow_input *input_data, struct hmap *lflows); -bool lflow_handle_northd_ls_changes(struct ovsdb_idl_txn *ovnsb_txn, - struct tracked_ls_changes *, - struct lflow_input *, struct hmap *lflows); +bool lflow_handle_northd_port_changes(struct ovsdb_idl_txn *ovnsb_txn, + struct tracked_ovn_ports *, + struct lflow_input *, + struct hmap *lflows); bool northd_handle_sb_port_binding_changes( const struct sbrec_port_binding_table *, struct hmap *ls_ports); @@ -349,7 +371,8 @@ bool northd_handle_lb_data_changes(struct tracked_lb_data *, struct ovn_datapaths *ls_datapaths, struct ovn_datapaths *lr_datapaths, struct hmap *lb_datapaths_map, - struct hmap *lbgrp_datapaths_map); + struct hmap *lbgrp_datapaths_map, + struct northd_tracked_data *); void build_bfd_table(struct ovsdb_idl_txn *ovnsb_txn, const struct nbrec_bfd_table *, @@ -372,6 +395,23 @@ void sync_lbs(struct ovsdb_idl_txn *, const struct sbrec_load_balancer_table *, bool check_sb_lb_duplicates(const struct sbrec_load_balancer_table *); void sync_pbs(struct ovsdb_idl_txn *, struct hmap *ls_ports); -bool sync_pbs_for_northd_ls_changes(struct tracked_ls_changes *); +bool sync_pbs_for_northd_changed_ovn_ports( struct tracked_ovn_ports *); + +static inline bool +northd_has_tracked_data(struct northd_tracked_data *trk_nd_changes) { + return trk_nd_changes->type != NORTHD_TRACKED_NONE; +} + +static inline bool +northd_has_lbs_in_tracked_data(struct northd_tracked_data *trk_nd_changes) +{ + return (trk_nd_changes->type & NORTHD_TRACKED_LBS); +} + +static inline bool +northd_has_lsps_in_tracked_data(struct northd_tracked_data *trk_nd_changes) +{ + return (trk_nd_changes->type & NORTHD_TRACKED_PORTS); +} #endif /* NORTHD_H */ diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at index 9a0d418e46..7bcd113d8f 100644 --- a/tests/ovn-northd.at +++ b/tests/ovn-northd.at @@ -10644,9 +10644,8 @@ check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats lbg1_uuid=$(ovn-nbctl --wait=sb create load_balancer_group name=lbg1) check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute -check_engine_stats lflow recompute nocompute -check_engine_stats sync_to_sb_lb recompute nocompute - +check_engine_stats lflow norecompute nocompute +check_engine_stats sync_to_sb_lb norecompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats @@ -10674,7 +10673,6 @@ check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute nocompute - CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats @@ -10822,8 +10820,8 @@ check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats lbg1_uuid=$(ovn-nbctl --wait=sb create load_balancer_group name=lbg1) check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute -check_engine_stats lflow recompute nocompute -check_engine_stats sync_to_sb_lb recompute nocompute +check_engine_stats lflow norecompute nocompute +check_engine_stats sync_to_sb_lb norecompute nocompute check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb set load_balancer_group . load_balancer="$lb2_uuid,$lb3_uuid,$lb4_uuid" From patchwork Thu Jan 11 15:28:41 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Numan Siddique X-Patchwork-Id: 1885671 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::138; helo=smtp1.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp1.osuosl.org (smtp1.osuosl.org [IPv6:2605:bc80:3010::138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4T9pVc1G6Nz1yPt for ; Fri, 12 Jan 2024 02:29:12 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id EECB582AF8; Thu, 11 Jan 2024 15:29:09 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org EECB582AF8 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id skFW0HvHjvsB; Thu, 11 Jan 2024 15:29:08 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp1.osuosl.org (Postfix) with ESMTPS id A808383DD3; Thu, 11 Jan 2024 15:29:06 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org A808383DD3 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 7BD3EC0077; Thu, 11 Jan 2024 15:29:06 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 521F6C0037 for ; Thu, 11 Jan 2024 15:29:05 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 8743361B1F for ; Thu, 11 Jan 2024 15:28:56 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 8743361B1F X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id vPnvARez11Nu for ; Thu, 11 Jan 2024 15:28:55 +0000 (UTC) Received: from relay5-d.mail.gandi.net (relay5-d.mail.gandi.net [IPv6:2001:4b98:dc4:8::225]) by smtp3.osuosl.org (Postfix) with ESMTPS id 2C5616F49F for ; Thu, 11 Jan 2024 15:28:54 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 2C5616F49F Received: by mail.gandi.net (Postfix) with ESMTPSA id C90C11C000C; Thu, 11 Jan 2024 15:28:52 +0000 (UTC) From: numans@ovn.org To: dev@openvswitch.org Date: Thu, 11 Jan 2024 10:28:41 -0500 Message-ID: <20240111152841.2789958-1-numans@ovn.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240111152752.2789854-1-numans@ovn.org> References: <20240111152752.2789854-1-numans@ovn.org> MIME-Version: 1.0 X-GND-Sasl: numans@ovn.org Subject: [ovs-dev] [PATCH ovn v5 02/16] tests: Add a couple of tests in ovn-northd for I-P. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Numan Siddique These tests cover scenarios for load balancers and NATs and check for the 'northd' and 'lflow' engine node recompute and compute stats. Signed-off-by: Numan Siddique Acked-by: Dumitru Ceara Acked-by: Han Zhou --- tests/ovn-macros.at | 44 ++++++++ tests/ovn-northd.at | 238 ++++++++++++++++++++++++++++++++++++++------ 2 files changed, 249 insertions(+), 33 deletions(-) diff --git a/tests/ovn-macros.at b/tests/ovn-macros.at index ca9b7a379b..9e88462b79 100644 --- a/tests/ovn-macros.at +++ b/tests/ovn-macros.at @@ -944,6 +944,50 @@ trim_zeros() { sed 's/\(00\)\{1,\}$//' } +# Checks if the provided engine node recomputed or not +# and if it computed or not. +# 1st argument is the engine node. +# 2nd argument is the expected recompute state. +# Possible values are - "norecompute" and "recompute". +# 3rd argument is the expected compute state. +# Possible values are - "nocompute" and "compute". +# +# Eg. 'check_engine_stats lflow recompute nocompute' will verify +# that if the lflow engine node recompute stat is > 0 and +# compute stat is equal to 0. It fails otherwise. +check_engine_stats() { + node=$1 + recompute=$2 + compute=$3 + + echo "Checking engine stats for node $node : recompute - \ +$recompute : compute - $compute" + + node_stat=$(as northd ovn-appctl -t ovn-northd inc-engine/show-stats $node) + # node_stat will be of this format : + # - Node: lflow - recompute: 3 - compute: 0 - abort: 0 + node_recompute_ct=$(echo $node_stat | cut -d '-' -f2 | cut -d ':' -f2) + node_compute_ct=$(echo $node_stat | cut -d '-' -f3 | cut -d ':' -f2) + + if [[ "$recompute" == "norecompute" ]]; then + # node should not be recomputed + echo "Expecting $node recompute count - $node_recompute_ct to be 0" + check test "$node_recompute_ct" -eq "0" + else + echo "Expecting $node recompute count - $node_recompute_ct not to be 0" + check test "$node_recompute_ct" -ne "0" + fi + + if [[ "$compute" == "nocompute" ]]; then + # node should not be computed + echo "Expecting $node compute count - $node_compute_ct to be 0" + check test "$node_compute_ct" -eq "0" + else + echo "Expecting $node compute count - $node_compute_ct not to be 0" + check test "$node_compute_ct" -ne "0" + fi +} + OVS_END_SHELL_HELPERS m4_define([OVN_POPULATE_ARP], [AT_CHECK(ovn_populate_arp__, [0], [ignore])]) diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at index 7bcd113d8f..4a02dacf60 100644 --- a/tests/ovn-northd.at +++ b/tests/ovn-northd.at @@ -10390,39 +10390,6 @@ OVN_FOR_EACH_NORTHD_NO_HV([ AT_SETUP([Load balancer incremental processing]) ovn_start -check_engine_stats() { - node=$1 - recompute=$2 - compute=$3 - - echo "__file__:__line__: Checking engine stats for node $node : recompute - \ -$recompute : compute - $compute" - - node_stat=$(as northd ovn-appctl -t ovn-northd inc-engine/show-stats $node) - # node_stat will be of this format : - # - Node: lflow - recompute: 3 - compute: 0 - abort: 0 - node_recompute_ct=$(echo $node_stat | cut -d '-' -f2 | cut -d ':' -f2) - node_compute_ct=$(echo $node_stat | cut -d '-' -f3 | cut -d ':' -f2) - - if [[ "$recompute" == "norecompute" ]]; then - # node should not be recomputed - echo "Expecting $node recompute count - $node_recompute_ct to be 0" - check test "$node_recompute_ct" -eq "0" - else - echo "Expecting $node recompute count - $node_recompute_ct not to be 0" - check test "$node_recompute_ct" -ne "0" - fi - - if [[ "$compute" == "nocompute" ]]; then - # node should not be computed - echo "Expecting $node compute count - $node_compute_ct to be 0" - check test "$node_compute_ct" -eq "0" - else - echo "Expecting $node compute count - $node_compute_ct not to be 0" - check test "$node_compute_ct" -ne "0" - fi -} - # Test I-P for load balancers. # Presently ovn-northd handles I-P for NB LBs in northd_lb_data engine node # only. @@ -11091,6 +11058,211 @@ AT_CHECK([ovn-sbctl dump-flows S1 | grep pre_acl | sed 's/table=./table=?/'], [0 table=? (ls_out_pre_acl ), priority=0 , match=(1), action=(next;) ]) +AT_CLEANUP +]) + +OVN_FOR_EACH_NORTHD_NO_HV([ +AT_SETUP([Load balancer incremental processing with stateless ACLs]) +ovn_start + +# Test I-P for load balancers. +# Presently ovn-northd handles I-P for NB LBs in northd_lb_data engine node +# only. +check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats +check ovn-nbctl --wait=sb lb-add lb1 10.0.0.10:80 10.0.0.3:80 + +check ovn-nbctl ls-add sw0 +check ovn-nbctl acl-add sw0 from-lport 1 1 allow-stateless +check ovn-nbctl --wait=sb acl-add sw0 to-lport 1 1 allow-stateless +check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats +check ovn-nbctl --wait=sb ls-lb-add sw0 lb1 +check_engine_stats lb_data norecompute compute +check_engine_stats northd norecompute compute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +# Clear the VIPs of lb1 +check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats +check ovn-nbctl --wait=sb clear load_balancer . vips +check_engine_stats lb_data norecompute compute +check_engine_stats northd norecompute compute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats +check ovn-nbctl --wait=sb lb-del lb1 +check_engine_stats lb_data norecompute compute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +AT_CLEANUP +]) + +OVN_FOR_EACH_NORTHD_NO_HV([ +AT_SETUP([Logical router incremental processing for NAT]) + +ovn_start + +net_add n1 +sim_add hv1 +as hv1 +ovs-vsctl add-br br-phys +ovn_attach n1 br-phys 192.168.0.11 + +ovn-sbctl chassis-add gw1 geneve 127.0.0.1 + +check ovn-nbctl ls-add sw0 +check ovn-nbctl lsp-add sw0 sw0p1 -- lsp-set-addresses sw0p1 "00:00:20:20:12:01 10.0.0.4" + +check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats +check ovn-nbctl --wait=sb lr-add lr0 +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute + +# Adding a logical router port should result in recompute +check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats +check ovn-nbctl lrp-add lr0 lr0-sw0 00:00:00:00:ff:01 10.0.0.1/24 +# for northd engine there will be both recompute and compute +# first it will be recompute to handle lr0-sw0 and then a compute +# for the SB port binding change. +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute + +ovn-nbctl lsp-add sw0 sw0-lr0 +ovn-nbctl lsp-set-type sw0-lr0 router +ovn-nbctl lsp-set-addresses sw0-lr0 00:00:00:00:ff:01 +check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats +check ovn-nbctl --wait=sb lsp-set-options sw0-lr0 router-port=lr0-sw0 +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute + +ovn-nbctl ls-add public +ovn-nbctl lrp-add lr0 lr0-public 00:00:20:20:12:13 172.168.0.100/24 +ovn-nbctl lsp-add public public-lr0 +ovn-nbctl lsp-set-type public-lr0 router +ovn-nbctl lsp-set-addresses public-lr0 router +ovn-nbctl lsp-set-options public-lr0 router-port=lr0-public + +# localnet port +ovn-nbctl lsp-add public ln-public +ovn-nbctl lsp-set-type ln-public localnet +ovn-nbctl lsp-set-addresses ln-public unknown +ovn-nbctl lsp-set-options ln-public network_name=public + +# schedule the gw router port to a chassis. Change the name of the chassis +ovn-nbctl --wait=hv lrp-set-gateway-chassis lr0-public hv1 20 + +# Modify a logical router port and it should result in recompute. +check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats +check ovn-nbctl set logical_router_port lr0-sw0 options:foo=bar +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute + +# Do checks for NATs. +# Add a NAT. This should not result in recompute of both northd and lflow +# engine nodes. +check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats +check ovn-nbctl --wait=sb lr-nat-add lr0 dnat_and_snat 172.168.0.110 10.0.0.4 +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +# Update the NAT options column +check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats +check ovn-nbctl --wait=sb set NAT . options:foo=bar +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +# Update the NAT external_ip column +check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats +check ovn-nbctl --wait=sb set NAT . external_ip=172.168.0.120 +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +# Update the NAT logical_ip column +check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats +check ovn-nbctl --wait=sb set NAT . logical_ip=10.0.0.10 +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +# Update the NAT type +check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats +check ovn-nbctl --wait=sb set NAT . type=snat +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +# Create a dnat_and_snat NAT with external_mac and logical_port +check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats +check ovn-nbctl --wait=sb lr-nat-add lr0 dnat_and_snat 172.168.0.110 10.0.0.4 sw0p1 30:54:00:00:00:03 +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +nat2_uuid=$(ovn-nbctl --bare --columns _uuid find nat logical_ip=10.0.0.4) + +check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats +check ovn-nbctl --wait=sb set NAT $nat2_uuid external_mac='"30:54:00:00:00:04"' +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +# Create a load balancer and add the lb vip as NAT +check ovn-nbctl lb-add lb1 172.168.0.140 10.0.0.20 +check ovn-nbctl lb-add lb2 172.168.0.150:80 10.0.0.40:8080 +check ovn-nbctl lr-lb-add lr0 lb1 +check ovn-nbctl lr-lb-add lr0 lb2 + +# lflow engine should recompute since the nat ip 172.168.0.140 +# is a lb vip. +check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats +check ovn-nbctl --wait=sb lr-nat-add lr0 dnat_and_snat 172.168.0.140 10.0.0.20 +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats +check ovn-nbctl --wait=sb lr-nat-add lr0 dnat_and_snat 172.168.0.150 10.0.0.41 +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats +check ovn-nbctl --wait=sb lr-nat-del lr0 dnat_and_snat 172.168.0.150 +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats +check ovn-nbctl --wait=sb lr-nat-del lr0 dnat_and_snat 172.168.0.140 +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +# Delete the NAT +check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats +check ovn-nbctl --wait=sb clear logical_router lr0 nat +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute +check_engine_stats sync_to_sb_pb recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +# Create router Policy +check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats +check ovn-nbctl --wait=sb lr-policy-add lr0 10 "ip4.src == 10.0.0.3" reroute 172.168.0.101,172.168.0.102 +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute + +check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats +check ovn-nbctl --wait=sb lr-policy-del lr0 10 "ip4.src == 10.0.0.3" +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute + +OVN_CLEANUP([hv1]) AT_CLEANUP ]) From patchwork Thu Jan 11 15:28:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Numan Siddique X-Patchwork-Id: 1885672 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::133; helo=smtp2.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp2.osuosl.org (smtp2.osuosl.org [IPv6:2605:bc80:3010::133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4T9pW45R1Zz1yPf for ; Fri, 12 Jan 2024 02:29:36 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id AEC29438A8; Thu, 11 Jan 2024 15:29:34 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org AEC29438A8 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id HXSPLC-uNt_g; Thu, 11 Jan 2024 15:29:32 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp2.osuosl.org (Postfix) with ESMTPS id 499F140A4D; Thu, 11 Jan 2024 15:29:31 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 499F140A4D Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 1DB29C007C; Thu, 11 Jan 2024 15:29:31 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by lists.linuxfoundation.org (Postfix) with ESMTP id DC079C0037 for ; Thu, 11 Jan 2024 15:29:29 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 4FA5F438B1 for ; Thu, 11 Jan 2024 15:29:26 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 4FA5F438B1 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id P1S77YLQpFEw for ; Thu, 11 Jan 2024 15:29:24 +0000 (UTC) Received: from relay8-d.mail.gandi.net (relay8-d.mail.gandi.net [IPv6:2001:4b98:dc4:8::228]) by smtp2.osuosl.org (Postfix) with ESMTPS id A7EEF438AB for ; Thu, 11 Jan 2024 15:29:23 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org A7EEF438AB Received: by mail.gandi.net (Postfix) with ESMTPSA id 460111BF20E; Thu, 11 Jan 2024 15:29:20 +0000 (UTC) From: numans@ovn.org To: dev@openvswitch.org Date: Thu, 11 Jan 2024 10:28:53 -0500 Message-ID: <20240111152853.2789978-1-numans@ovn.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240111152752.2789854-1-numans@ovn.org> References: <20240111152752.2789854-1-numans@ovn.org> MIME-Version: 1.0 X-GND-Sasl: numans@ovn.org Subject: [ovs-dev] [PATCH ovn v5 03/16] northd: Move router ports SB PB options sync to sync_to_sb_pb node. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Numan Siddique It also moves the logical router port IPv6 prefix delegation updates to "sync-from-sb" engine node. With these changes, northd engine node doesn't need to do much for SB Port binding changes other than updating 'op->sb'. And there is no need to fall back to recompute. Prior to this patch northd_handle_sb_port_binding_changes() was returning false for all SB port binding updates except for the VIF PBs. This patch now handles updates to all the SB port binding by setting 'op->sb'. Signed-off-by: Numan Siddique Acked-by: Han Zhou Acked-by: Dumitru Ceara --- northd/en-northd.c | 2 +- northd/en-sync-sb.c | 3 +- northd/northd.c | 296 ++++++++++++++++++++++++++------------------ northd/northd.h | 6 +- tests/ovn-northd.at | 158 +++++++++++++++++++++-- 5 files changed, 334 insertions(+), 131 deletions(-) diff --git a/northd/en-northd.c b/northd/en-northd.c index 28559ed211..677b2b1ab0 100644 --- a/northd/en-northd.c +++ b/northd/en-northd.c @@ -189,7 +189,7 @@ northd_sb_port_binding_handler(struct engine_node *node, northd_get_input_data(node, &input_data); if (!northd_handle_sb_port_binding_changes( - input_data.sbrec_port_binding_table, &nd->ls_ports)) { + input_data.sbrec_port_binding_table, &nd->ls_ports, &nd->lr_ports)) { return false; } diff --git a/northd/en-sync-sb.c b/northd/en-sync-sb.c index 3aaab8d005..45be7ddbcb 100644 --- a/northd/en-sync-sb.c +++ b/northd/en-sync-sb.c @@ -288,7 +288,8 @@ en_sync_to_sb_pb_run(struct engine_node *node, void *data OVS_UNUSED) const struct engine_context *eng_ctx = engine_get_context(); struct northd_data *northd_data = engine_get_input_data("northd", node); - sync_pbs(eng_ctx->ovnsb_idl_txn, &northd_data->ls_ports); + sync_pbs(eng_ctx->ovnsb_idl_txn, &northd_data->ls_ports, + &northd_data->lr_ports); engine_set_node_state(node, EN_UPDATED); } diff --git a/northd/northd.c b/northd/northd.c index f32e3bf21a..23f2dae26b 100644 --- a/northd/northd.c +++ b/northd/northd.c @@ -3435,6 +3435,9 @@ ovn_port_update_sbrec(struct ovsdb_idl_txn *ovnsb_txn, { sbrec_port_binding_set_datapath(op->sb, op->od->sb); if (op->nbrp) { + /* Note: SB port binding options for router ports are set in + * sync_pbs(). */ + /* If the router is for l3 gateway, it resides on a chassis * and its port type is "l3gateway". */ const char *chassis_name = smap_get(&op->od->nbr->options, "chassis"); @@ -3446,15 +3449,11 @@ ovn_port_update_sbrec(struct ovsdb_idl_txn *ovnsb_txn, sbrec_port_binding_set_type(op->sb, "patch"); } - struct smap new; - smap_init(&new); if (is_cr_port(op)) { ovs_assert(sbrec_chassis_by_name); ovs_assert(sbrec_chassis_by_hostname); ovs_assert(sbrec_ha_chassis_grp_by_name); ovs_assert(active_ha_chassis_grps); - const char *redirect_type = smap_get(&op->nbrp->options, - "redirect-type"); if (op->nbrp->ha_chassis_group) { if (op->nbrp->n_gateway_chassis) { @@ -3496,49 +3495,8 @@ ovn_port_update_sbrec(struct ovsdb_idl_txn *ovnsb_txn, /* Delete the legacy gateway_chassis from the pb. */ sbrec_port_binding_set_gateway_chassis(op->sb, NULL, 0); } - smap_add(&new, "distributed-port", op->nbrp->name); - - bool always_redirect = - !op->od->has_distributed_nat && - !l3dgw_port_has_associated_vtep_lports(op->l3dgw_port); - - if (redirect_type) { - smap_add(&new, "redirect-type", redirect_type); - /* XXX Why can't we enable always-redirect when redirect-type - * is bridged? */ - if (!strcmp(redirect_type, "bridged")) { - always_redirect = false; - } - } - - if (always_redirect) { - smap_add(&new, "always-redirect", "true"); - } - } else { - if (op->peer) { - smap_add(&new, "peer", op->peer->key); - if (op->nbrp->ha_chassis_group || - op->nbrp->n_gateway_chassis) { - char *redirect_name = - ovn_chassis_redirect_name(op->nbrp->name); - smap_add(&new, "chassis-redirect-port", redirect_name); - free(redirect_name); - } - } - if (chassis_name) { - smap_add(&new, "l3gateway-chassis", chassis_name); - } } - const char *ipv6_pd_list = smap_get(&op->sb->options, - "ipv6_ra_pd_list"); - if (ipv6_pd_list) { - smap_add(&new, "ipv6_ra_pd_list", ipv6_pd_list); - } - - sbrec_port_binding_set_options(op->sb, &new); - smap_destroy(&new); - sbrec_port_binding_set_parent_port(op->sb, NULL); sbrec_port_binding_set_tag(op->sb, NULL, 0); @@ -4768,12 +4726,14 @@ check_sb_lb_duplicates(const struct sbrec_load_balancer_table *table) return duplicates; } -/* Syncs the SB port binding for the ovn_port 'op'. Caller should make sure - * that the OVN SB IDL txn is not NULL. Presently it only syncs the nat - * column of port binding corresponding to the 'op->nbsp' */ +/* Syncs the SB port binding for the ovn_port 'op' of a logical switch port. + * Caller should make sure that the OVN SB IDL txn is not NULL. Presently it + * only syncs the nat column of port binding corresponding to the 'op->nbsp' */ static void -sync_pb_for_op(struct ovn_port *op) +sync_pb_for_lsp(struct ovn_port *op) { + ovs_assert(op->nbsp); + if (lsp_is_router(op->nbsp)) { const char *chassis = NULL; if (op->peer && op->peer->od && op->peer->od->nbr) { @@ -4895,18 +4855,86 @@ sync_pb_for_op(struct ovn_port *op) } } +/* Syncs the SB port binding for the ovn_port 'op' of a logical router port. + * Caller should make sure that the OVN SB IDL txn is not NULL. Presently it + * only sets the port binding options column for the router ports */ +static void +sync_pb_for_lrp(struct ovn_port *op) +{ + ovs_assert(op->nbrp); + + struct smap new; + smap_init(&new); + + const char *chassis_name = smap_get(&op->od->nbr->options, "chassis"); + if (is_cr_port(op)) { + smap_add(&new, "distributed-port", op->nbrp->name); + + bool always_redirect = + !op->od->has_distributed_nat && + !l3dgw_port_has_associated_vtep_lports(op->l3dgw_port); + + const char *redirect_type = smap_get(&op->nbrp->options, + "redirect-type"); + if (redirect_type) { + smap_add(&new, "redirect-type", redirect_type); + /* XXX Why can't we enable always-redirect when redirect-type + * is bridged? */ + if (!strcmp(redirect_type, "bridged")) { + always_redirect = false; + } + } + + if (always_redirect) { + smap_add(&new, "always-redirect", "true"); + } + } else { + if (op->peer) { + smap_add(&new, "peer", op->peer->key); + if (op->nbrp->ha_chassis_group || + op->nbrp->n_gateway_chassis) { + char *redirect_name = + ovn_chassis_redirect_name(op->nbrp->name); + smap_add(&new, "chassis-redirect-port", redirect_name); + free(redirect_name); + } + } + if (chassis_name) { + smap_add(&new, "l3gateway-chassis", chassis_name); + } + } + + const char *ipv6_pd_list = smap_get(&op->sb->options, "ipv6_ra_pd_list"); + if (ipv6_pd_list) { + smap_add(&new, "ipv6_ra_pd_list", ipv6_pd_list); + } + + sbrec_port_binding_set_options(op->sb, &new); + smap_destroy(&new); +} + +static void ovn_update_ipv6_options(struct hmap *lr_ports); +static void ovn_update_ipv6_opt_for_op(struct ovn_port *op); + /* Sync the SB Port bindings which needs to be updated. * Presently it syncs the nat column of port bindings corresponding to * the logical switch ports. */ void -sync_pbs(struct ovsdb_idl_txn *ovnsb_idl_txn, struct hmap *ls_ports) +sync_pbs(struct ovsdb_idl_txn *ovnsb_idl_txn, struct hmap *ls_ports, + struct hmap *lr_ports) { ovs_assert(ovnsb_idl_txn); struct ovn_port *op; HMAP_FOR_EACH (op, key_node, ls_ports) { - sync_pb_for_op(op); + sync_pb_for_lsp(op); + } + + HMAP_FOR_EACH (op, key_node, lr_ports) { + sync_pb_for_lrp(op); } + + ovn_update_ipv6_options(lr_ports); } /* Sync the SB Port bindings for the added and updated logical switch ports @@ -4918,12 +4946,14 @@ sync_pbs_for_northd_changed_ovn_ports( struct tracked_ovn_ports *trk_ovn_ports) struct ovn_port *op; HMAPX_FOR_EACH (hmapx_node, &trk_ovn_ports->created) { op = hmapx_node->data; - sync_pb_for_op(op); + ovs_assert(op->nbsp); + sync_pb_for_lsp(op); } HMAPX_FOR_EACH (hmapx_node, &trk_ovn_ports->updated) { op = hmapx_node->data; - sync_pb_for_op(op); + ovs_assert(op->nbsp); + sync_pb_for_lsp(op); } return true; @@ -5671,20 +5701,31 @@ fail: bool northd_handle_sb_port_binding_changes( const struct sbrec_port_binding_table *sbrec_port_binding_table, - struct hmap *ls_ports) + struct hmap *ls_ports, struct hmap *lr_ports) { const struct sbrec_port_binding *pb; static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 5); SBREC_PORT_BINDING_TABLE_FOR_EACH_TRACKED (pb, sbrec_port_binding_table) { - /* Return false if the 'pb' belongs to a router port. We don't handle - * I-P for router ports yet. */ - if (is_pb_router_type(pb)) { - return false; + bool is_router_port = is_pb_router_type(pb); + struct ovn_port *op = NULL; + + if (is_router_port) { + /* A router port binding 'pb' can belong to + * - a logical switch port of type router or + * - a logical router port. + * + * So, first search in lr_ports hmap. If not found, search in + * ls_ports hmap. + * */ + op = ovn_port_find(lr_ports, pb->logical_port); } - struct ovn_port *op = ovn_port_find(ls_ports, pb->logical_port); - if (op && !op->lsp_can_be_inc_processed) { - return false; + if (!op) { + op = ovn_port_find(ls_ports, pb->logical_port); + + if (op) { + is_router_port = false; + } } if (sbrec_port_binding_is_new(pb)) { @@ -5693,7 +5734,8 @@ northd_handle_sb_port_binding_changes( * pointer in northd data. Fallback to recompute otherwise. */ if (!op) { VLOG_WARN_RL(&rl, "A port-binding for %s is created but the " - "LSP is not found.", pb->logical_port); + "%s is not found.", pb->logical_port, + is_router_port ? "LRP" : "LSP"); return false; } op->sb = pb; @@ -5703,18 +5745,29 @@ northd_handle_sb_port_binding_changes( * case. Fallback to recompute otherwise, to avoid dangling * sb idl pointers and other unexpected behavior. */ if (op && op->sb == pb) { - VLOG_WARN_RL(&rl, "A port-binding for %s is deleted but " - "the LSP still exists.", pb->logical_port); + VLOG_WARN_RL(&rl, "A port-binding for %s is deleted but the " + "LSP/LRP still exists.", pb->logical_port); return false; } } else { - /* The PB is updated, most likely because of binding/unbinding - * to/from a chassis, and we can ignore the change (updating NB - * "up" will be handled in the engine node "sync_from_sb"). + /* The PB is updated. + * For an LSP PB it is most likely because of + * binding/unbinding to/from a chassis, and we can ignore the + * change (updating NB "up" will be handled in the engine node + * "sync_from_sb"). + * + * For an LRP PB, it is most likely because of + * - IPv6 prefix delagation updates from ovn-controller. + * This update is handled in "sync_from_sb" node. + * - ha chassis group and this can be ignored. + * + * All other changes can be ignored. + * * Fallback to recompute for anything unexpected. */ if (!op) { VLOG_WARN_RL(&rl, "A port-binding for %s is updated but the " - "LSP is not found.", pb->logical_port); + "%s is not found.", pb->logical_port, + is_router_port ? "LRP" : "LSP"); return false; } if (op->sb != pb) { @@ -7855,67 +7908,70 @@ static void copy_ra_to_sb(struct ovn_port *op, const char *address_mode); static void -ovn_update_ipv6_options(struct hmap *lr_ports) +ovn_update_ipv6_opt_for_op(struct ovn_port *op) { - struct ovn_port *op; - HMAP_FOR_EACH (op, key_node, lr_ports) { - ovs_assert(op->nbrp); - - if (op->nbrp->peer || !op->peer) { - continue; - } + if (op->nbrp->peer || !op->peer) { + return; + } - if (!op->lrp_networks.n_ipv6_addrs) { - continue; - } + if (!op->lrp_networks.n_ipv6_addrs) { + return; + } - struct smap options; - smap_clone(&options, &op->sb->options); + struct smap options; + smap_clone(&options, &op->sb->options); - /* enable IPv6 prefix delegation */ - bool prefix_delegation = smap_get_bool(&op->nbrp->options, + /* enable IPv6 prefix delegation */ + bool prefix_delegation = smap_get_bool(&op->nbrp->options, "prefix_delegation", false); - if (!lrport_is_enabled(op->nbrp)) { - prefix_delegation = false; - } - if (smap_get_bool(&options, "ipv6_prefix_delegation", - false) != prefix_delegation) { - smap_add(&options, "ipv6_prefix_delegation", - prefix_delegation ? "true" : "false"); - } + if (!lrport_is_enabled(op->nbrp)) { + prefix_delegation = false; + } + if (smap_get_bool(&options, "ipv6_prefix_delegation", + false) != prefix_delegation) { + smap_add(&options, "ipv6_prefix_delegation", + prefix_delegation ? "true" : "false"); + } - bool ipv6_prefix = smap_get_bool(&op->nbrp->options, + bool ipv6_prefix = smap_get_bool(&op->nbrp->options, "prefix", false); - if (!lrport_is_enabled(op->nbrp)) { - ipv6_prefix = false; - } - if (smap_get_bool(&options, "ipv6_prefix", false) != ipv6_prefix) { - smap_add(&options, "ipv6_prefix", - ipv6_prefix ? "true" : "false"); - } - sbrec_port_binding_set_options(op->sb, &options); + if (!lrport_is_enabled(op->nbrp)) { + ipv6_prefix = false; + } + if (smap_get_bool(&options, "ipv6_prefix", false) != ipv6_prefix) { + smap_add(&options, "ipv6_prefix", ipv6_prefix ? "true" : "false"); + } + sbrec_port_binding_set_options(op->sb, &options); - smap_destroy(&options); + smap_destroy(&options); - const char *address_mode = smap_get( - &op->nbrp->ipv6_ra_configs, "address_mode"); + const char *address_mode = smap_get(&op->nbrp->ipv6_ra_configs, + "address_mode"); - if (!address_mode) { - continue; - } - if (strcmp(address_mode, "slaac") && - strcmp(address_mode, "dhcpv6_stateful") && - strcmp(address_mode, "dhcpv6_stateless")) { - static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 5); - VLOG_WARN_RL(&rl, "Invalid address mode [%s] defined", - address_mode); - continue; - } + if (!address_mode) { + return; + } + if (strcmp(address_mode, "slaac") && + strcmp(address_mode, "dhcpv6_stateful") && + strcmp(address_mode, "dhcpv6_stateless")) { + static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 5); + VLOG_WARN_RL(&rl, "Invalid address mode [%s] defined", + address_mode); + return; + } - if (smap_get_bool(&op->nbrp->ipv6_ra_configs, "send_periodic", - false)) { - copy_ra_to_sb(op, address_mode); - } + if (smap_get_bool(&op->nbrp->ipv6_ra_configs, "send_periodic", false)) { + copy_ra_to_sb(op, address_mode); + } +} + +static void +ovn_update_ipv6_options(struct hmap *lr_ports) +{ + struct ovn_port *op; + HMAP_FOR_EACH (op, key_node, lr_ports) { + ovs_assert(op->nbrp); + ovn_update_ipv6_opt_for_op(op); } } @@ -18007,8 +18063,6 @@ ovnnb_db_run(struct northd_input *input_data, &data->lr_ports); stopwatch_stop(BUILD_LFLOWS_CTX_STOPWATCH_NAME, time_msec()); stopwatch_start(CLEAR_LFLOWS_CTX_STOPWATCH_NAME, time_msec()); - ovn_update_ipv6_options(&data->lr_ports); - ovn_update_ipv6_prefix(&data->lr_ports); sync_mirrors(ovnsb_txn, input_data->nbrec_mirror_table, input_data->sbrec_mirror_table); @@ -18339,6 +18393,8 @@ ovnsb_db_run(struct ovsdb_idl_txn *ovnnb_txn, &ha_ref_chassis_map); } shash_destroy(&ha_ref_chassis_map); + + ovn_update_ipv6_prefix(lr_ports); } const char * diff --git a/northd/northd.h b/northd/northd.h index 23521065e8..233dca8084 100644 --- a/northd/northd.h +++ b/northd/northd.h @@ -364,7 +364,8 @@ bool lflow_handle_northd_port_changes(struct ovsdb_idl_txn *ovnsb_txn, struct lflow_input *, struct hmap *lflows); bool northd_handle_sb_port_binding_changes( - const struct sbrec_port_binding_table *, struct hmap *ls_ports); + const struct sbrec_port_binding_table *, struct hmap *ls_ports, + struct hmap *lr_ports); struct tracked_lb_data; bool northd_handle_lb_data_changes(struct tracked_lb_data *, @@ -394,7 +395,8 @@ void sync_lbs(struct ovsdb_idl_txn *, const struct sbrec_load_balancer_table *, struct chassis_features *chassis_features); bool check_sb_lb_duplicates(const struct sbrec_load_balancer_table *); -void sync_pbs(struct ovsdb_idl_txn *, struct hmap *ls_ports); +void sync_pbs(struct ovsdb_idl_txn *, struct hmap *ls_ports, + struct hmap *lr_ports); bool sync_pbs_for_northd_changed_ovn_ports( struct tracked_ovn_ports *); static inline bool diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at index 4a02dacf60..4edad24e53 100644 --- a/tests/ovn-northd.at +++ b/tests/ovn-northd.at @@ -10073,7 +10073,7 @@ check ovn-nbctl --wait=hv lsp-add ls0 lsp0-0 -- lsp-set-addresses lsp0-0 "unknow ovs-vsctl add-port br-int lsp0-0 -- set interface lsp0-0 external_ids:iface-id=lsp0-0 wait_for_ports_up check ovn-nbctl --wait=hv sync -check_recompute_counter 4 5 5 5 5 5 +check_recompute_counter 3 4 3 4 3 4 check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=hv lsp-add ls0 lsp0-1 -- lsp-set-addresses lsp0-1 "aa:aa:aa:00:00:01 192.168.0.11" @@ -10235,6 +10235,126 @@ OVN_CLEANUP([hv1]) AT_CLEANUP ]) +OVN_FOR_EACH_NORTHD_NO_HV([ +AT_SETUP([SB Port binding incremental processing]) +ovn_start + +check_recompute_counter() { + northd_recomp=$(as northd ovn-appctl -t ovn-northd inc-engine/show-stats northd recompute) + echo "northd recompute count - $northd_recomp" + AT_CHECK([test x$northd_recomp = x$1]) + + lflow_recomp=$(as northd ovn-appctl -t ovn-northd inc-engine/show-stats lflow recompute) + echo "lflow recompute count - $lflow_recomp" + AT_CHECK([test x$lflow_recomp = x$2]) +} + +net_add n1 +sim_add hv1 +as hv1 +ovs-vsctl add-br br-phys +ovn_attach n1 br-phys 192.168.0.11 + +check ovn-nbctl --wait=sb ls-add ls0 +check ovn-nbctl --wait=sb lr-add lr0 + +# Test normal VIF port +check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats +check ovn-nbctl lsp-add ls0 p1 +check_recompute_counter 0 0 +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats +check ovn-nbctl lsp-set-addresses p1 "00:00:00:00:01:01 10.0.0.3" +check_recompute_counter 0 0 +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +check as northd ovn-appctl -t ovn-northd vlog/set info + +check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats +check ovn-sbctl lsp-bind p1 hv1 +check_recompute_counter 0 0 +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +# Test lsp of type router +check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats +check ovn-nbctl lsp-add ls0 rp -- lsp-set-type rp router + +# northd engine recomputes twice. Both the times for handling NB logical switch port +# changes and not because of SB port binding changes. This is because ovn-northd +# sets the "up" to true. +check_recompute_counter 2 2 +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +# Set some options to 'rp'. northd should only recompute once. +check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats +check ovn-nbctl lsp-set-options rp foo=bar +check_recompute_counter 1 1 +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +# Test lsp of type external +check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats +check ovn-nbctl lsp-add ls0 e1 -- lsp-set-type e1 external +check_recompute_counter 2 2 +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +# Set some options to 'e1'. northd should only recompute once. +check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats +check ovn-nbctl lsp-set-options e1 foo=bar +check_recompute_counter 1 1 +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +# Test lrp +check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats +check ovn-nbctl lrp-add lr0 lrp 00:00:02:01:02:03 10.0.0.1/24 +check_recompute_counter 1 1 +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +# Set some options on 'lrp'. northd should only recompute once. +check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats +check ovn-nbctl lrp-set-options lrp route_table=rtb-1 +check_recompute_counter 1 1 +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +# Make lrp a gateway port +check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats +check ovn-nbctl lrp-set-gateway-chassis lrp hv1 +# There will be 3 recomputes of northd engine node +# 1. missing handler for input NB_logical_router +# 2. missing handler for input SB_ha_chassis_group +# 3. missing handler for input NB_logical_router when ovn-northd +# updates the hosting-chassis option in NB_logical_router_port. +check_recompute_counter 3 3 +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats +check ovn-nbctl clear logical_router_port lrp gateway_chassis +# There will be 2 recomputes of northd engine node +# 1. missing handler for input NB_logical_router +# 2. missing handler for input SB_ha_chassis_group +check_recompute_counter 2 2 +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +# Delete some of the ports. +check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats +check ovn-nbctl lsp-del p1 +check_recompute_counter 0 0 +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats +check ovn-nbctl lsp-del e1 +check_recompute_counter 1 1 +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats +check ovn-nbctl lrp-del lrp +check_recompute_counter 1 1 +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +OVN_CLEANUP([hv1]) +AT_CLEANUP +]) + OVN_FOR_EACH_NORTHD_NO_HV([ AT_SETUP([ACL/Meter incremental processing - no northd recompute]) ovn_start @@ -10269,6 +10389,7 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE(1) AT_CLEANUP ]) + OVN_FOR_EACH_NORTHD_NO_HV([ AT_SETUP([check fip and lb flows]) AT_KEYWORDS([fip-lb-flows]) @@ -10471,7 +10592,7 @@ ovn-nbctl lsp-set-addresses sw0-lr0 00:00:00:00:ff:01 check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats ovn-nbctl --wait=sb lsp-set-options sw0-lr0 router-port=lr0-sw0 check_engine_stats lb_data norecompute compute -check_engine_stats northd recompute nocompute +check_engine_stats northd recompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute nocompute @@ -11114,12 +11235,14 @@ ovn_attach n1 br-phys 192.168.0.11 ovn-sbctl chassis-add gw1 geneve 127.0.0.1 check ovn-nbctl ls-add sw0 -check ovn-nbctl lsp-add sw0 sw0p1 -- lsp-set-addresses sw0p1 "00:00:20:20:12:01 10.0.0.4" +check ovn-nbctl --wait=sb lsp-add sw0 sw0p1 -- lsp-set-addresses sw0p1 "00:00:20:20:12:01 10.0.0.4" check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb lr-add lr0 check_engine_stats northd recompute nocompute +check_engine_stats sync_to_sb_pb recompute nocompute check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE # Adding a logical router port should result in recompute check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats @@ -11127,16 +11250,20 @@ check ovn-nbctl lrp-add lr0 lr0-sw0 00:00:00:00:ff:01 10.0.0.1/24 # for northd engine there will be both recompute and compute # first it will be recompute to handle lr0-sw0 and then a compute # for the SB port binding change. -check_engine_stats northd recompute nocompute +check_engine_stats northd recompute compute +check_engine_stats sync_to_sb_pb recompute nocompute check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE ovn-nbctl lsp-add sw0 sw0-lr0 ovn-nbctl lsp-set-type sw0-lr0 router ovn-nbctl lsp-set-addresses sw0-lr0 00:00:00:00:ff:01 check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb lsp-set-options sw0-lr0 router-port=lr0-sw0 -check_engine_stats northd recompute nocompute +check_engine_stats northd recompute compute +check_engine_stats sync_to_sb_pb recompute nocompute check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE ovn-nbctl ls-add public ovn-nbctl lrp-add lr0 lr0-public 00:00:20:20:12:13 172.168.0.100/24 @@ -11158,7 +11285,9 @@ ovn-nbctl --wait=hv lrp-set-gateway-chassis lr0-public hv1 20 check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl set logical_router_port lr0-sw0 options:foo=bar check_engine_stats northd recompute nocompute +check_engine_stats sync_to_sb_pb recompute nocompute check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE # Do checks for NATs. # Add a NAT. This should not result in recompute of both northd and lflow @@ -11167,6 +11296,7 @@ check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb lr-nat-add lr0 dnat_and_snat 172.168.0.110 10.0.0.4 check_engine_stats northd recompute nocompute check_engine_stats lflow recompute nocompute +check_engine_stats sync_to_sb_pb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Update the NAT options column @@ -11174,6 +11304,7 @@ check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb set NAT . options:foo=bar check_engine_stats northd recompute nocompute check_engine_stats lflow recompute nocompute +check_engine_stats sync_to_sb_pb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Update the NAT external_ip column @@ -11181,6 +11312,7 @@ check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb set NAT . external_ip=172.168.0.120 check_engine_stats northd recompute nocompute check_engine_stats lflow recompute nocompute +check_engine_stats sync_to_sb_pb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Update the NAT logical_ip column @@ -11188,6 +11320,7 @@ check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb set NAT . logical_ip=10.0.0.10 check_engine_stats northd recompute nocompute check_engine_stats lflow recompute nocompute +check_engine_stats sync_to_sb_pb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Update the NAT type @@ -11195,13 +11328,15 @@ check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb set NAT . type=snat check_engine_stats northd recompute nocompute check_engine_stats lflow recompute nocompute +check_engine_stats sync_to_sb_pb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Create a dnat_and_snat NAT with external_mac and logical_port check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb lr-nat-add lr0 dnat_and_snat 172.168.0.110 10.0.0.4 sw0p1 30:54:00:00:00:03 -check_engine_stats northd recompute nocompute +check_engine_stats northd recompute compute check_engine_stats lflow recompute nocompute +check_engine_stats sync_to_sb_pb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE nat2_uuid=$(ovn-nbctl --bare --columns _uuid find nat logical_ip=10.0.0.4) @@ -11210,6 +11345,7 @@ check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb set NAT $nat2_uuid external_mac='"30:54:00:00:00:04"' check_engine_stats northd recompute nocompute check_engine_stats lflow recompute nocompute +check_engine_stats sync_to_sb_pb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Create a load balancer and add the lb vip as NAT @@ -11223,31 +11359,35 @@ check ovn-nbctl lr-lb-add lr0 lb2 check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb lr-nat-add lr0 dnat_and_snat 172.168.0.140 10.0.0.20 check_engine_stats northd recompute nocompute +check_engine_stats sync_to_sb_pb recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb lr-nat-add lr0 dnat_and_snat 172.168.0.150 10.0.0.41 check_engine_stats northd recompute nocompute +check_engine_stats sync_to_sb_pb recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb lr-nat-del lr0 dnat_and_snat 172.168.0.150 check_engine_stats northd recompute nocompute +check_engine_stats sync_to_sb_pb recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb lr-nat-del lr0 dnat_and_snat 172.168.0.140 check_engine_stats northd recompute nocompute +check_engine_stats sync_to_sb_pb recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Delete the NAT check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb clear logical_router lr0 nat -check_engine_stats northd recompute nocompute +check_engine_stats northd recompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_pb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11256,12 +11396,16 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb lr-policy-add lr0 10 "ip4.src == 10.0.0.3" reroute 172.168.0.101,172.168.0.102 check_engine_stats northd recompute nocompute +check_engine_stats sync_to_sb_pb recompute nocompute check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb lr-policy-del lr0 10 "ip4.src == 10.0.0.3" check_engine_stats northd recompute nocompute +check_engine_stats sync_to_sb_pb recompute nocompute check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE OVN_CLEANUP([hv1]) AT_CLEANUP From patchwork Thu Jan 11 15:29:21 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Numan Siddique X-Patchwork-Id: 1885673 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::136; helo=smtp3.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4T9pWN74Vnz1yPf for ; Fri, 12 Jan 2024 02:29:52 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 244846147F; Thu, 11 Jan 2024 15:29:51 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 244846147F X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id aEjo5eQ6P4qs; Thu, 11 Jan 2024 15:29:48 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp3.osuosl.org (Postfix) with ESMTPS id 8828061B2A; Thu, 11 Jan 2024 15:29:47 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 8828061B2A Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 2E0B2C007C; Thu, 11 Jan 2024 15:29:47 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by lists.linuxfoundation.org (Postfix) with ESMTP id 5EA0DC0037 for ; Thu, 11 Jan 2024 15:29:46 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id B40708401A for ; Thu, 11 Jan 2024 15:29:39 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org B40708401A X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id qpMHu8MuzmJA for ; Thu, 11 Jan 2024 15:29:36 +0000 (UTC) Received: from relay3-d.mail.gandi.net (relay3-d.mail.gandi.net [217.70.183.195]) by smtp1.osuosl.org (Postfix) with ESMTPS id 4EDBA83FFC for ; Thu, 11 Jan 2024 15:29:35 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 4EDBA83FFC Received: by mail.gandi.net (Postfix) with ESMTPSA id 5FCE060014; Thu, 11 Jan 2024 15:29:31 +0000 (UTC) From: numans@ovn.org To: dev@openvswitch.org Date: Thu, 11 Jan 2024 10:29:21 -0500 Message-ID: <20240111152921.2790005-1-numans@ovn.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240111152752.2789854-1-numans@ovn.org> References: <20240111152752.2789854-1-numans@ovn.org> MIME-Version: 1.0 X-GND-Sasl: numans@ovn.org Subject: [ovs-dev] [PATCH ovn v5 04/16] northd: Add a new engine 'lr_nat' to manage lr NAT data. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Numan Siddique This new engine now maintains the NAT related data for each logical router which was earlier maintained by the northd engine node in the 'struct ovn_datapath'. The input to this engine node is 'northd'. A record for each logical router (lr_nat_record) is maintained in the 'lr_nats' hmap table which stores the lr's NAT dat. 'northd' engine now reports logical routers changed due to NATs in its tracking data. 'lr_nat' engine node makes use of this tracked data in its northd change handler to update the NAT data. This engine node becomes an input to 'lflow' node. Signed-off-by: Numan Siddique Acked-by: Dumitru Ceara --- lib/ovn-util.c | 6 +- lib/ovn-util.h | 2 +- lib/stopwatch-names.h | 1 + northd/automake.mk | 2 + northd/en-lflow.c | 5 + northd/en-lr-nat.c | 423 ++++++++++++++++++++++++++++ northd/en-lr-nat.h | 127 +++++++++ northd/en-northd.c | 4 + northd/en-sync-sb.c | 6 +- northd/inc-proc-northd.c | 6 + northd/northd.c | 589 ++++++++++++++++----------------------- northd/northd.h | 46 +-- northd/ovn-northd.c | 1 + tests/ovn-northd.at | 46 ++- 14 files changed, 885 insertions(+), 379 deletions(-) create mode 100644 northd/en-lr-nat.c create mode 100644 northd/en-lr-nat.h diff --git a/lib/ovn-util.c b/lib/ovn-util.c index 6ef9cac7f2..c8b89cc216 100644 --- a/lib/ovn-util.c +++ b/lib/ovn-util.c @@ -385,7 +385,7 @@ extract_sbrec_binding_first_mac(const struct sbrec_port_binding *binding, } bool -lport_addresses_is_empty(struct lport_addresses *laddrs) +lport_addresses_is_empty(const struct lport_addresses *laddrs) { return !laddrs->n_ipv4_addrs && !laddrs->n_ipv6_addrs; } @@ -395,6 +395,10 @@ destroy_lport_addresses(struct lport_addresses *laddrs) { free(laddrs->ipv4_addrs); free(laddrs->ipv6_addrs); + laddrs->ipv4_addrs = NULL; + laddrs->ipv6_addrs = NULL; + laddrs->n_ipv4_addrs = 0; + laddrs->n_ipv6_addrs = 0; } /* Returns a string of the IP address of 'laddrs' that overlaps with 'ip_s'. diff --git a/lib/ovn-util.h b/lib/ovn-util.h index aa0b3b2fb4..d245d57d56 100644 --- a/lib/ovn-util.h +++ b/lib/ovn-util.h @@ -112,7 +112,7 @@ bool extract_sbrec_binding_first_mac(const struct sbrec_port_binding *binding, bool extract_lrp_networks__(char *mac, char **networks, size_t n_networks, struct lport_addresses *laddrs); -bool lport_addresses_is_empty(struct lport_addresses *); +bool lport_addresses_is_empty(const struct lport_addresses *); void destroy_lport_addresses(struct lport_addresses *); const char *find_lport_address(const struct lport_addresses *laddrs, const char *ip_s); diff --git a/lib/stopwatch-names.h b/lib/stopwatch-names.h index 4e93c1dc14..782d64320a 100644 --- a/lib/stopwatch-names.h +++ b/lib/stopwatch-names.h @@ -29,5 +29,6 @@ #define LFLOWS_TO_SB_STOPWATCH_NAME "lflows_to_sb" #define PORT_GROUP_RUN_STOPWATCH_NAME "port_group_run" #define SYNC_METERS_RUN_STOPWATCH_NAME "sync_meters_run" +#define LR_NAT_RUN_STOPWATCH_NAME "lr_nat_run" #endif diff --git a/northd/automake.mk b/northd/automake.mk index 5d77ca67b7..a477105470 100644 --- a/northd/automake.mk +++ b/northd/automake.mk @@ -24,6 +24,8 @@ northd_ovn_northd_SOURCES = \ northd/en-sync-from-sb.h \ northd/en-lb-data.c \ northd/en-lb-data.h \ + northd/en-lr-nat.c \ + northd/en-lr-nat.h \ northd/inc-proc-northd.c \ northd/inc-proc-northd.h \ northd/ipam.c \ diff --git a/northd/en-lflow.c b/northd/en-lflow.c index 6ba26006e0..e4f875ef7c 100644 --- a/northd/en-lflow.c +++ b/northd/en-lflow.c @@ -19,6 +19,7 @@ #include #include "en-lflow.h" +#include "en-lr-nat.h" #include "en-northd.h" #include "en-meters.h" @@ -40,6 +41,9 @@ lflow_get_input_data(struct engine_node *node, engine_get_input_data("port_group", node); struct sync_meters_data *sync_meters_data = engine_get_input_data("sync_meters", node); + struct ed_type_lr_nat_data *lr_nat_data = + engine_get_input_data("lr_nat", node); + lflow_input->nbrec_bfd_table = EN_OVSDB_GET(engine_get_input("NB_bfd", node)); lflow_input->sbrec_bfd_table = @@ -61,6 +65,7 @@ lflow_get_input_data(struct engine_node *node, lflow_input->ls_ports = &northd_data->ls_ports; lflow_input->lr_ports = &northd_data->lr_ports; lflow_input->ls_port_groups = &pg_data->ls_port_groups; + lflow_input->lr_nats = &lr_nat_data->lr_nats; lflow_input->meter_groups = &sync_meters_data->meter_groups; lflow_input->lb_datapaths_map = &northd_data->lb_datapaths_map; lflow_input->svc_monitor_map = &northd_data->svc_monitor_map; diff --git a/northd/en-lr-nat.c b/northd/en-lr-nat.c new file mode 100644 index 0000000000..273c5be34b --- /dev/null +++ b/northd/en-lr-nat.c @@ -0,0 +1,423 @@ +/* + * Copyright (c) 2023, Red Hat, Inc. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at: + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#include + +#include +#include +#include + +/* OVS includes */ +#include "include/openvswitch/hmap.h" +#include "openvswitch/util.h" +#include "openvswitch/vlog.h" +#include "stopwatch.h" + +/* OVN includes */ +#include "en-lr-nat.h" +#include "lib/inc-proc-eng.h" +#include "lib/lb.h" +#include "lib/ovn-nb-idl.h" +#include "lib/ovn-sb-idl.h" +#include "lib/ovn-util.h" +#include "lib/stopwatch-names.h" +#include "northd.h" + +VLOG_DEFINE_THIS_MODULE(en_lr_nat); + +/* Static function declarations. */ +static void lr_nat_table_init(struct lr_nat_table *); +static void lr_nat_table_clear(struct lr_nat_table *); +static void lr_nat_table_destroy(struct lr_nat_table *); +static void lr_nat_table_build(struct lr_nat_table *, + const struct ovn_datapaths *lr_datapaths); +static struct lr_nat_record *lr_nat_table_find_(const struct lr_nat_table *, + const struct nbrec_logical_router *); +static struct lr_nat_record *lr_nat_table_find_by_index_( + const struct lr_nat_table *, size_t od_index); + +static struct lr_nat_record *lr_nat_record_create( + struct lr_nat_table *, const struct ovn_datapath *); +static void lr_nat_record_init(struct lr_nat_record *); +static void lr_nat_record_reinit(struct lr_nat_record *); +static void lr_nat_record_destroy(struct lr_nat_record *); + +static void lr_nat_entries_init(struct lr_nat_record *); +static void lr_nat_entries_destroy(struct lr_nat_record *); +static void lr_nat_external_ips_init(struct lr_nat_record *); +static void lr_nat_external_ips_destroy(struct lr_nat_record *); +static bool get_force_snat_ip(struct lr_nat_record *, const char *key_type, + struct lport_addresses *); + +const struct lr_nat_record * +lr_nat_table_find_by_index(const struct lr_nat_table *table, + size_t od_index) +{ + return lr_nat_table_find_by_index_(table, od_index); +} + +/* 'lr_nat' engine node manages the NB logical router NAT data. + */ +void * +en_lr_nat_init(struct engine_node *node OVS_UNUSED, + struct engine_arg *arg OVS_UNUSED) +{ + struct ed_type_lr_nat_data *data = xzalloc(sizeof *data); + lr_nat_table_init(&data->lr_nats); + hmapx_init(&data->trk_data.crupdated); + return data; +} + +void +en_lr_nat_cleanup(void *data_) +{ + struct ed_type_lr_nat_data *data = (struct ed_type_lr_nat_data *) data_; + lr_nat_table_destroy(&data->lr_nats); + hmapx_destroy(&data->trk_data.crupdated); +} + +void +en_lr_nat_clear_tracked_data(void *data_) +{ + struct ed_type_lr_nat_data *data = (struct ed_type_lr_nat_data *) data_; + hmapx_clear(&data->trk_data.crupdated); +} + +void +en_lr_nat_run(struct engine_node *node, void *data_) +{ + struct northd_data *northd_data = engine_get_input_data("northd", node); + struct ed_type_lr_nat_data *data = data_; + + stopwatch_start(LR_NAT_RUN_STOPWATCH_NAME, time_msec()); + lr_nat_table_clear(&data->lr_nats); + lr_nat_table_build(&data->lr_nats, &northd_data->lr_datapaths); + + stopwatch_stop(LR_NAT_RUN_STOPWATCH_NAME, time_msec()); + engine_set_node_state(node, EN_UPDATED); +} + +/* Handler functions. */ +bool +lr_nat_northd_handler(struct engine_node *node, void *data_) +{ + struct northd_data *northd_data = engine_get_input_data("northd", node); + if (!northd_has_tracked_data(&northd_data->trk_data)) { + return false; + } + + if (!northd_has_lr_nats_in_tracked_data(&northd_data->trk_data)) { + return true; + } + + struct ed_type_lr_nat_data *data = data_; + struct lr_nat_record *lrnat_rec; + const struct ovn_datapath *od; + struct hmapx_node *hmapx_node; + + HMAPX_FOR_EACH (hmapx_node, &northd_data->trk_data.lr_with_changed_nats) { + od = hmapx_node->data; + lrnat_rec = lr_nat_table_find_(&data->lr_nats, od->nbr); + ovs_assert(lrnat_rec); + lr_nat_record_reinit(lrnat_rec); + + /* Add the lrnet rec to the tracking data. */ + hmapx_add(&data->trk_data.crupdated, lrnat_rec); + } + + if (lr_nat_has_tracked_data(&data->trk_data)) { + engine_set_node_state(node, EN_UPDATED); + } + + return true; +} + +/* static functions. */ +static void +lr_nat_table_init(struct lr_nat_table *table) +{ + *table = (struct lr_nat_table) { + .entries = HMAP_INITIALIZER(&table->entries), + }; +} + +static void +lr_nat_table_clear(struct lr_nat_table *table) +{ + struct lr_nat_record *lrnat_rec; + HMAP_FOR_EACH_POP (lrnat_rec, key_node, &table->entries) { + lr_nat_record_destroy(lrnat_rec); + } + + free(table->array); + table->array = NULL; +} + +static void +lr_nat_table_build(struct lr_nat_table *table, + const struct ovn_datapaths *lr_datapaths) +{ + table->array = xrealloc(table->array, + ods_size(lr_datapaths) * sizeof *table->array); + + const struct ovn_datapath *od; + HMAP_FOR_EACH (od, key_node, &lr_datapaths->datapaths) { + lr_nat_record_create(table, od); + } +} + +static void +lr_nat_table_destroy(struct lr_nat_table *table) +{ + lr_nat_table_clear(table); + hmap_destroy(&table->entries); +} + +struct lr_nat_record * +lr_nat_table_find_(const struct lr_nat_table *table, + const struct nbrec_logical_router *nbr) +{ + struct lr_nat_record *lrnat_rec; + + HMAP_FOR_EACH_WITH_HASH (lrnat_rec, key_node, + uuid_hash(&nbr->header_.uuid), &table->entries) { + if (nbr == lrnat_rec->od->nbr) { + return lrnat_rec; + } + } + return NULL; +} + + +struct lr_nat_record * +lr_nat_table_find_by_index_(const struct lr_nat_table *table, + size_t od_index) +{ + ovs_assert(od_index <= hmap_count(&table->entries)); + + return table->array[od_index]; +} + +static struct lr_nat_record * +lr_nat_record_create(struct lr_nat_table *table, + const struct ovn_datapath *od) +{ + ovs_assert(od->nbr); + + struct lr_nat_record *lrnat_rec = xzalloc(sizeof *lrnat_rec); + lrnat_rec->od = od; + lr_nat_record_init(lrnat_rec); + + hmap_insert(&table->entries, &lrnat_rec->key_node, + uuid_hash(&od->nbr->header_.uuid)); + table->array[od->index] = lrnat_rec; + return lrnat_rec; +} + +static void +lr_nat_record_init(struct lr_nat_record *lrnat_rec) +{ + lr_nat_entries_init(lrnat_rec); + lr_nat_external_ips_init(lrnat_rec); +} + +static void +lr_nat_record_reinit(struct lr_nat_record *lrnat_rec) +{ + lr_nat_entries_destroy(lrnat_rec); + lr_nat_external_ips_destroy(lrnat_rec); + lr_nat_record_init(lrnat_rec); +} + +static void +lr_nat_record_destroy(struct lr_nat_record *lrnat_rec) +{ + lr_nat_entries_destroy(lrnat_rec); + lr_nat_external_ips_destroy(lrnat_rec); + free(lrnat_rec); +} + +static void +lr_nat_external_ips_init(struct lr_nat_record *lrnat_rec) +{ + sset_init(&lrnat_rec->external_ips); + for (size_t i = 0; i < lrnat_rec->od->nbr->n_nat; i++) { + sset_add(&lrnat_rec->external_ips, + lrnat_rec->od->nbr->nat[i]->external_ip); + } +} + +static void +lr_nat_external_ips_destroy(struct lr_nat_record *lrnat_rec) +{ + sset_destroy(&lrnat_rec->external_ips); +} + +static void +snat_ip_add(struct lr_nat_record *lrnat_rec, const char *ip, + struct ovn_nat *nat_entry) +{ + struct ovn_snat_ip *snat_ip = shash_find_data(&lrnat_rec->snat_ips, ip); + + if (!snat_ip) { + snat_ip = xzalloc(sizeof *snat_ip); + ovs_list_init(&snat_ip->snat_entries); + shash_add(&lrnat_rec->snat_ips, ip, snat_ip); + } + + if (nat_entry) { + ovs_list_push_back(&snat_ip->snat_entries, + &nat_entry->ext_addr_list_node); + } +} + +static void +lr_nat_entries_init(struct lr_nat_record *lrnat_rec) +{ + shash_init(&lrnat_rec->snat_ips); + sset_init(&lrnat_rec->external_macs); + lrnat_rec->has_distributed_nat = false; + + if (get_force_snat_ip(lrnat_rec, "dnat", + &lrnat_rec->dnat_force_snat_addrs)) { + if (lrnat_rec->dnat_force_snat_addrs.n_ipv4_addrs) { + snat_ip_add(lrnat_rec, + lrnat_rec->dnat_force_snat_addrs.ipv4_addrs[0].addr_s, + NULL); + } + if (lrnat_rec->dnat_force_snat_addrs.n_ipv6_addrs) { + snat_ip_add(lrnat_rec, + lrnat_rec->dnat_force_snat_addrs.ipv6_addrs[0].addr_s, + NULL); + } + } + + /* Check if 'lb_force_snat_ip' is configured with 'router_ip'. */ + const char *lb_force_snat = + smap_get(&lrnat_rec->od->nbr->options, "lb_force_snat_ip"); + if (lb_force_snat && !strcmp(lb_force_snat, "router_ip") + && smap_get(&lrnat_rec->od->nbr->options, "chassis")) { + + /* Set it to true only if its gateway router and + * options:lb_force_snat_ip=router_ip. */ + lrnat_rec->lb_force_snat_router_ip = true; + } else { + lrnat_rec->lb_force_snat_router_ip = false; + + /* Check if 'lb_force_snat_ip' is configured with a set of + * IP address(es). */ + if (get_force_snat_ip(lrnat_rec, "lb", + &lrnat_rec->lb_force_snat_addrs)) { + if (lrnat_rec->lb_force_snat_addrs.n_ipv4_addrs) { + snat_ip_add(lrnat_rec, + lrnat_rec->lb_force_snat_addrs.ipv4_addrs[0].addr_s, + NULL); + } + if (lrnat_rec->lb_force_snat_addrs.n_ipv6_addrs) { + snat_ip_add(lrnat_rec, + lrnat_rec->lb_force_snat_addrs.ipv6_addrs[0].addr_s, + NULL); + } + } + } + + if (!lrnat_rec->od->nbr->n_nat) { + return; + } + + lrnat_rec->nat_entries = + xmalloc(lrnat_rec->od->nbr->n_nat * sizeof *lrnat_rec->nat_entries); + + for (size_t i = 0; i < lrnat_rec->od->nbr->n_nat; i++) { + const struct nbrec_nat *nat = lrnat_rec->od->nbr->nat[i]; + struct ovn_nat *nat_entry = &lrnat_rec->nat_entries[i]; + + nat_entry->nb = nat; + if (!extract_ip_addresses(nat->external_ip, + &nat_entry->ext_addrs) || + !nat_entry_is_valid(nat_entry)) { + static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1); + + VLOG_WARN_RL(&rl, + "Bad ip address %s in nat configuration " + "for router %s", nat->external_ip, + lrnat_rec->od->nbr->name); + continue; + } + + /* If this is a SNAT rule add the IP to the set of unique SNAT IPs. */ + if (!strcmp(nat->type, "snat")) { + if (!nat_entry_is_v6(nat_entry)) { + snat_ip_add(lrnat_rec, + nat_entry->ext_addrs.ipv4_addrs[0].addr_s, + nat_entry); + } else { + snat_ip_add(lrnat_rec, + nat_entry->ext_addrs.ipv6_addrs[0].addr_s, + nat_entry); + } + } else { + if (!strcmp(nat->type, "dnat_and_snat") + && nat->logical_port && nat->external_mac) { + lrnat_rec->has_distributed_nat = true; + } + + if (nat->external_mac) { + sset_add(&lrnat_rec->external_macs, nat->external_mac); + } + } + } + lrnat_rec->n_nat_entries = lrnat_rec->od->nbr->n_nat; +} + +static bool +get_force_snat_ip(struct lr_nat_record *lrnat_rec, const char *key_type, + struct lport_addresses *laddrs) +{ + char *key = xasprintf("%s_force_snat_ip", key_type); + const char *addresses = smap_get(&lrnat_rec->od->nbr->options, key); + free(key); + + if (!addresses) { + return false; + } + + if (!extract_ip_address(addresses, laddrs)) { + static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1); + VLOG_WARN_RL(&rl, "bad ip %s in options of router "UUID_FMT"", + addresses, UUID_ARGS(&lrnat_rec->od->nbr->header_.uuid)); + return false; + } + + return true; +} + +static void +lr_nat_entries_destroy(struct lr_nat_record *lrnat_rec) +{ + shash_destroy_free_data(&lrnat_rec->snat_ips); + destroy_lport_addresses(&lrnat_rec->dnat_force_snat_addrs); + destroy_lport_addresses(&lrnat_rec->lb_force_snat_addrs); + + for (size_t i = 0; i < lrnat_rec->n_nat_entries; i++) { + destroy_lport_addresses(&lrnat_rec->nat_entries[i].ext_addrs); + } + + free(lrnat_rec->nat_entries); + lrnat_rec->nat_entries = NULL; + lrnat_rec->n_nat_entries = 0; + sset_destroy(&lrnat_rec->external_macs); +} diff --git a/northd/en-lr-nat.h b/northd/en-lr-nat.h new file mode 100644 index 0000000000..3ec4c7b506 --- /dev/null +++ b/northd/en-lr-nat.h @@ -0,0 +1,127 @@ +/* + * Copyright (c) 2023, Red Hat, Inc. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at: + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +#ifndef EN_LR_NAT_H +#define EN_LR_NAT_H 1 + +#include + +/* OVS includes. */ +#include "lib/hmapx.h" +#include "openvswitch/hmap.h" +#include "sset.h" + +/* OVN includes. */ +#include "lib/inc-proc-eng.h" +#include "lib/ovn-nb-idl.h" +#include "lib/ovn-sb-idl.h" +#include "lib/ovn-util.h" + +/* Contains a NAT entry with the external addresses pre-parsed. */ +struct ovn_nat { + const struct nbrec_nat *nb; + struct lport_addresses ext_addrs; + struct ovs_list ext_addr_list_node; /* Linkage in the per-external IP + * list of nat entries. Currently + * only used for SNAT. + */ +}; + +/* Stores the list of SNAT entries referencing a unique SNAT IP address. + * The 'snat_entries' list will be empty if the SNAT IP is used only for + * dnat_force_snat_ip or lb_force_snat_ip. + */ +struct ovn_snat_ip { + struct ovs_list snat_entries; +}; + +struct lr_nat_record { + struct hmap_node key_node; /* Index on 'nbr->header_.uuid'. */ + + const struct ovn_datapath *od; + + struct ovn_nat *nat_entries; + size_t n_nat_entries; + + bool has_distributed_nat; + + /* Set of nat external ips on the router. */ + struct sset external_ips; + + /* Set of nat external macs on the router. */ + struct sset external_macs; + + /* SNAT IPs owned by the router (shash of 'struct ovn_snat_ip'). */ + struct shash snat_ips; + + struct lport_addresses dnat_force_snat_addrs; + struct lport_addresses lb_force_snat_addrs; + bool lb_force_snat_router_ip; +}; + +struct lr_nat_tracked_data { + /* Created or updated logical router with NAT data. */ + struct hmapx crupdated; +}; + +struct lr_nat_table { + struct hmap entries; /* Stores struct lr_nat_record. */ + + /* The array index of each element in 'entries'. */ + struct lr_nat_record **array; +}; + +const struct lr_nat_record * lr_nat_table_find_by_index( + const struct lr_nat_table *, size_t od_index); + +struct ed_type_lr_nat_data { + struct lr_nat_table lr_nats; + + struct lr_nat_tracked_data trk_data; +}; + +void *en_lr_nat_init(struct engine_node *, struct engine_arg *); +void en_lr_nat_cleanup(void *data); +void en_lr_nat_clear_tracked_data(void *data); +void en_lr_nat_run(struct engine_node *, void *data); + +bool lr_nat_logical_router_handler(struct engine_node *, void *data); +bool lr_nat_northd_handler(struct engine_node *, void *data); + +/* Returns true if a 'nat_entry' is valid, i.e.: + * - parsing was successful. + * - the string yielded exactly one IPv4 address or exactly one IPv6 address. + */ +static inline bool +nat_entry_is_valid(const struct ovn_nat *nat_entry) +{ + const struct lport_addresses *ext_addrs = &nat_entry->ext_addrs; + + return (ext_addrs->n_ipv4_addrs == 1 && ext_addrs->n_ipv6_addrs == 0) || + (ext_addrs->n_ipv4_addrs == 0 && ext_addrs->n_ipv6_addrs == 1); +} + +static inline bool +nat_entry_is_v6(const struct ovn_nat *nat_entry) +{ + return nat_entry->ext_addrs.n_ipv6_addrs > 0; +} + +static inline bool +lr_nat_has_tracked_data(struct lr_nat_tracked_data *trk_data) { + return !hmapx_is_empty(&trk_data->crupdated); +} + +#endif /* EN_LR_NAT_H */ \ No newline at end of file diff --git a/northd/en-northd.c b/northd/en-northd.c index 677b2b1ab0..546397f3dc 100644 --- a/northd/en-northd.c +++ b/northd/en-northd.c @@ -209,6 +209,10 @@ northd_nb_logical_router_handler(struct engine_node *node, return false; } + if (northd_has_lr_nats_in_tracked_data(&nd->trk_data)) { + engine_set_node_state(node, EN_UPDATED); + } + return true; } diff --git a/northd/en-sync-sb.c b/northd/en-sync-sb.c index 45be7ddbcb..11e12428f7 100644 --- a/northd/en-sync-sb.c +++ b/northd/en-sync-sb.c @@ -21,6 +21,7 @@ #include "lib/svec.h" #include "openvswitch/util.h" +#include "en-lr-nat.h" #include "en-sync-sb.h" #include "lib/inc-proc-eng.h" #include "lib/lb.h" @@ -287,9 +288,10 @@ en_sync_to_sb_pb_run(struct engine_node *node, void *data OVS_UNUSED) { const struct engine_context *eng_ctx = engine_get_context(); struct northd_data *northd_data = engine_get_input_data("northd", node); - + struct ed_type_lr_nat_data *lr_nat_data = + engine_get_input_data("lr_nat", node); sync_pbs(eng_ctx->ovnsb_idl_txn, &northd_data->ls_ports, - &northd_data->lr_ports); + &northd_data->lr_ports, &lr_nat_data->lr_nats); engine_set_node_state(node, EN_UPDATED); } diff --git a/northd/inc-proc-northd.c b/northd/inc-proc-northd.c index 04df0b06c2..1f211b278e 100644 --- a/northd/inc-proc-northd.c +++ b/northd/inc-proc-northd.c @@ -31,6 +31,7 @@ #include "openvswitch/vlog.h" #include "inc-proc-northd.h" #include "en-lb-data.h" +#include "en-lr-nat.h" #include "en-northd.h" #include "en-lflow.h" #include "en-northd-output.h" @@ -146,6 +147,7 @@ static ENGINE_NODE(fdb_aging_waker, "fdb_aging_waker"); static ENGINE_NODE(sync_to_sb_lb, "sync_to_sb_lb"); static ENGINE_NODE(sync_to_sb_pb, "sync_to_sb_pb"); static ENGINE_NODE_WITH_CLEAR_TRACK_DATA(lb_data, "lb_data"); +static ENGINE_NODE_WITH_CLEAR_TRACK_DATA(lr_nat, "lr_nat"); void inc_proc_northd_init(struct ovsdb_idl_loop *nb, struct ovsdb_idl_loop *sb) @@ -189,6 +191,8 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, northd_nb_logical_router_handler); engine_add_input(&en_northd, &en_lb_data, northd_lb_data_handler); + engine_add_input(&en_lr_nat, &en_northd, lr_nat_northd_handler); + engine_add_input(&en_mac_binding_aging, &en_nb_nb_global, NULL); engine_add_input(&en_mac_binding_aging, &en_sb_mac_binding, NULL); engine_add_input(&en_mac_binding_aging, &en_northd, NULL); @@ -212,6 +216,7 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, engine_add_input(&en_lflow, &en_sb_igmp_group, NULL); engine_add_input(&en_lflow, &en_northd, lflow_northd_handler); engine_add_input(&en_lflow, &en_port_group, lflow_port_group_handler); + engine_add_input(&en_lflow, &en_lr_nat, NULL); engine_add_input(&en_sync_to_sb_addr_set, &en_nb_address_set, sync_to_sb_addr_set_nb_address_set_handler); @@ -235,6 +240,7 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, engine_add_input(&en_sync_to_sb_pb, &en_northd, sync_to_sb_pb_northd_handler); + engine_add_input(&en_sync_to_sb_pb, &en_lr_nat, NULL); /* en_sync_to_sb engine node syncs the SB database tables from * the NB database tables. diff --git a/northd/northd.c b/northd/northd.c index 23f2dae26b..e5e86326e3 100644 --- a/northd/northd.c +++ b/northd/northd.c @@ -44,6 +44,7 @@ #include "memory.h" #include "northd.h" #include "en-lb-data.h" +#include "en-lr-nat.h" #include "lib/ovn-parallel-hmap.h" #include "ovn/actions.h" #include "ovn/features.h" @@ -556,184 +557,6 @@ ovn_mcast_group_allocate_key(struct mcast_info *mcast_info) &mcast_info->group_tnlid_hint); } -/* Contains a NAT entry with the external addresses pre-parsed. */ -struct ovn_nat { - const struct nbrec_nat *nb; - struct lport_addresses ext_addrs; - struct ovs_list ext_addr_list_node; /* Linkage in the per-external IP - * list of nat entries. Currently - * only used for SNAT. - */ -}; - -/* Stores the list of SNAT entries referencing a unique SNAT IP address. - * The 'snat_entries' list will be empty if the SNAT IP is used only for - * dnat_force_snat_ip or lb_force_snat_ip. - */ -struct ovn_snat_ip { - struct ovs_list snat_entries; -}; - -static bool -get_force_snat_ip(struct ovn_datapath *od, const char *key_type, - struct lport_addresses *laddrs); - -/* Returns true if a 'nat_entry' is valid, i.e.: - * - parsing was successful. - * - the string yielded exactly one IPv4 address or exactly one IPv6 address. - */ -static bool -nat_entry_is_valid(const struct ovn_nat *nat_entry) -{ - const struct lport_addresses *ext_addrs = &nat_entry->ext_addrs; - - return (ext_addrs->n_ipv4_addrs == 1 && ext_addrs->n_ipv6_addrs == 0) || - (ext_addrs->n_ipv4_addrs == 0 && ext_addrs->n_ipv6_addrs == 1); -} - -static bool -nat_entry_is_v6(const struct ovn_nat *nat_entry) -{ - return nat_entry->ext_addrs.n_ipv6_addrs > 0; -} - -static void -snat_ip_add(struct ovn_datapath *od, const char *ip, struct ovn_nat *nat_entry) -{ - struct ovn_snat_ip *snat_ip = shash_find_data(&od->snat_ips, ip); - - if (!snat_ip) { - snat_ip = xzalloc(sizeof *snat_ip); - ovs_list_init(&snat_ip->snat_entries); - shash_add(&od->snat_ips, ip, snat_ip); - } - - if (nat_entry) { - ovs_list_push_back(&snat_ip->snat_entries, - &nat_entry->ext_addr_list_node); - } -} - -static void -init_nat_entries(struct ovn_datapath *od) -{ - ovs_assert(od->nbr); - - shash_init(&od->snat_ips); - if (get_force_snat_ip(od, "dnat", &od->dnat_force_snat_addrs)) { - if (od->dnat_force_snat_addrs.n_ipv4_addrs) { - snat_ip_add(od, od->dnat_force_snat_addrs.ipv4_addrs[0].addr_s, - NULL); - } - if (od->dnat_force_snat_addrs.n_ipv6_addrs) { - snat_ip_add(od, od->dnat_force_snat_addrs.ipv6_addrs[0].addr_s, - NULL); - } - } - - /* Check if 'lb_force_snat_ip' is configured with 'router_ip'. */ - const char *lb_force_snat = - smap_get(&od->nbr->options, "lb_force_snat_ip"); - if (lb_force_snat && !strcmp(lb_force_snat, "router_ip") - && smap_get(&od->nbr->options, "chassis")) { - /* Set it to true only if its gateway router and - * options:lb_force_snat_ip=router_ip. */ - od->lb_force_snat_router_ip = true; - } else { - od->lb_force_snat_router_ip = false; - - /* Check if 'lb_force_snat_ip' is configured with a set of - * IP address(es). */ - if (get_force_snat_ip(od, "lb", &od->lb_force_snat_addrs)) { - if (od->lb_force_snat_addrs.n_ipv4_addrs) { - snat_ip_add(od, od->lb_force_snat_addrs.ipv4_addrs[0].addr_s, - NULL); - } - if (od->lb_force_snat_addrs.n_ipv6_addrs) { - snat_ip_add(od, od->lb_force_snat_addrs.ipv6_addrs[0].addr_s, - NULL); - } - } - } - - if (!od->nbr->n_nat) { - return; - } - - od->nat_entries = xmalloc(od->nbr->n_nat * sizeof *od->nat_entries); - - for (size_t i = 0; i < od->nbr->n_nat; i++) { - const struct nbrec_nat *nat = od->nbr->nat[i]; - struct ovn_nat *nat_entry = &od->nat_entries[i]; - - nat_entry->nb = nat; - if (!extract_ip_addresses(nat->external_ip, - &nat_entry->ext_addrs) || - !nat_entry_is_valid(nat_entry)) { - static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1); - - VLOG_WARN_RL(&rl, - "Bad ip address %s in nat configuration " - "for router %s", nat->external_ip, od->nbr->name); - continue; - } - - /* If this is a SNAT rule add the IP to the set of unique SNAT IPs. */ - if (!strcmp(nat->type, "snat")) { - if (!nat_entry_is_v6(nat_entry)) { - snat_ip_add(od, nat_entry->ext_addrs.ipv4_addrs[0].addr_s, - nat_entry); - } else { - snat_ip_add(od, nat_entry->ext_addrs.ipv6_addrs[0].addr_s, - nat_entry); - } - } - - if (!strcmp(nat->type, "dnat_and_snat") - && nat->logical_port && nat->external_mac) { - od->has_distributed_nat = true; - } - } - od->n_nat_entries = od->nbr->n_nat; -} - -static void -destroy_nat_entries(struct ovn_datapath *od) -{ - if (!od->nbr) { - return; - } - - shash_destroy_free_data(&od->snat_ips); - destroy_lport_addresses(&od->dnat_force_snat_addrs); - destroy_lport_addresses(&od->lb_force_snat_addrs); - - for (size_t i = 0; i < od->n_nat_entries; i++) { - destroy_lport_addresses(&od->nat_entries[i].ext_addrs); - } -} - -static void -init_router_external_ips(struct ovn_datapath *od) -{ - ovs_assert(od->nbr); - - sset_init(&od->external_ips); - for (size_t i = 0; i < od->nbr->n_nat; i++) { - sset_add(&od->external_ips, od->nbr->nat[i]->external_ip); - } -} - -static void -destroy_router_external_ips(struct ovn_datapath *od) -{ - if (!od->nbr) { - return; - } - - sset_destroy(&od->external_ips); -} - static bool lb_has_vip(const struct nbrec_load_balancer *lb) { @@ -854,10 +677,7 @@ ovn_datapath_destroy(struct hmap *datapaths, struct ovn_datapath *od) destroy_ipam_info(&od->ipam_info); free(od->router_ports); free(od->ls_peers); - destroy_nat_entries(od); - destroy_router_external_ips(od); destroy_lb_for_datapath(od); - free(od->nat_entries); free(od->localnet_ports); free(od->l3dgw_ports); destroy_mcast_info_for_datapath(od); @@ -874,8 +694,8 @@ ovn_datapath_get_type(const struct ovn_datapath *od) } static struct ovn_datapath * -ovn_datapath_find(const struct hmap *datapaths, - const struct uuid *uuid) +ovn_datapath_find_(const struct hmap *datapaths, + const struct uuid *uuid) { struct ovn_datapath *od; @@ -887,6 +707,13 @@ ovn_datapath_find(const struct hmap *datapaths, return NULL; } +const struct ovn_datapath * +ovn_datapath_find(const struct hmap *datapaths, + const struct uuid *uuid) +{ + return ovn_datapath_find_(datapaths, uuid); +} + static struct ovn_datapath * ovn_datapath_find_by_key(struct hmap *datapaths, uint32_t dp_key) { @@ -925,7 +752,7 @@ ovn_datapath_from_sbrec(const struct hmap *ls_datapaths, if (!dps) { return NULL; } - struct ovn_datapath *od = ovn_datapath_find(dps, &key); + struct ovn_datapath *od = ovn_datapath_find_(dps, &key); if (od && (od->sb == sb)) { return od; } @@ -1213,7 +1040,7 @@ join_datapaths(const struct nbrec_logical_switch_table *nbrec_ls_table, continue; } - if (ovn_datapath_find(datapaths, &key)) { + if (ovn_datapath_find_(datapaths, &key)) { static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1); VLOG_INFO_RL( &rl, "deleting Datapath_Binding "UUID_FMT" with " @@ -1230,8 +1057,8 @@ join_datapaths(const struct nbrec_logical_switch_table *nbrec_ls_table, const struct nbrec_logical_switch *nbs; NBREC_LOGICAL_SWITCH_TABLE_FOR_EACH (nbs, nbrec_ls_table) { - struct ovn_datapath *od = ovn_datapath_find(datapaths, - &nbs->header_.uuid); + struct ovn_datapath *od = ovn_datapath_find_(datapaths, + &nbs->header_.uuid); if (od) { od->nbs = nbs; ovs_list_remove(&od->list); @@ -1254,8 +1081,8 @@ join_datapaths(const struct nbrec_logical_switch_table *nbrec_ls_table, continue; } - struct ovn_datapath *od = ovn_datapath_find(datapaths, - &nbr->header_.uuid); + struct ovn_datapath *od = ovn_datapath_find_(datapaths, + &nbr->header_.uuid); if (od) { if (!od->nbs) { od->nbr = nbr; @@ -1276,8 +1103,6 @@ join_datapaths(const struct nbrec_logical_switch_table *nbrec_ls_table, ovs_list_push_back(nb_only, &od->list); } init_mcast_info_for_datapath(od); - init_nat_entries(od); - init_router_external_ips(od); init_lb_for_datapath(od); if (smap_get(&od->nbr->options, "chassis")) { od->is_gw_router = true; @@ -1361,12 +1186,6 @@ ovn_datapath_assign_requested_tnl_id( } } -static inline size_t -ods_size(const struct ovn_datapaths *datapaths) -{ - return hmap_count(&datapaths->datapaths); -} - static void ods_build_array_index(struct ovn_datapaths *datapaths) { @@ -4859,7 +4678,7 @@ sync_pb_for_lsp(struct ovn_port *op) * Caller should make sure that the OVN SB IDL txn is not NULL. Presently it * only sets the port binding options column for the router ports */ static void -sync_pb_for_lrp(struct ovn_port *op) +sync_pb_for_lrp(struct ovn_port *op, const struct lr_nat_table *lr_nats) { ovs_assert(op->nbrp); @@ -4868,10 +4687,14 @@ sync_pb_for_lrp(struct ovn_port *op) const char *chassis_name = smap_get(&op->od->nbr->options, "chassis"); if (is_cr_port(op)) { + const struct lr_nat_record *lrnat_rec = + lr_nat_table_find_by_index(lr_nats, op->od->index); + ovs_assert(lrnat_rec); + smap_add(&new, "distributed-port", op->nbrp->name); bool always_redirect = - !op->od->has_distributed_nat && + !lrnat_rec->has_distributed_nat && !l3dgw_port_has_associated_vtep_lports(op->l3dgw_port); const char *redirect_type = smap_get(&op->nbrp->options, @@ -4921,7 +4744,7 @@ static void ovn_update_ipv6_opt_for_op(struct ovn_port *op); * the logical switch ports. */ void sync_pbs(struct ovsdb_idl_txn *ovnsb_idl_txn, struct hmap *ls_ports, - struct hmap *lr_ports) + struct hmap *lr_ports, const struct lr_nat_table *lr_nats) { ovs_assert(ovnsb_idl_txn); @@ -4931,7 +4754,7 @@ sync_pbs(struct ovsdb_idl_txn *ovnsb_idl_txn, struct hmap *ls_ports, } HMAP_FOR_EACH (op, key_node, lr_ports) { - sync_pb_for_lrp(op); + sync_pb_for_lrp(op, lr_nats); } ovn_update_ipv6_options(lr_ports); @@ -4940,7 +4763,7 @@ sync_pbs(struct ovsdb_idl_txn *ovnsb_idl_txn, struct hmap *ls_ports, /* Sync the SB Port bindings for the added and updated logical switch ports * of the tracked northd engine data. */ bool -sync_pbs_for_northd_changed_ovn_ports( struct tracked_ovn_ports *trk_ovn_ports) +sync_pbs_for_northd_changed_ovn_ports(struct tracked_ovn_ports *trk_ovn_ports) { struct hmapx_node *hmapx_node; struct ovn_port *op; @@ -5192,6 +5015,7 @@ destroy_northd_data_tracked_changes(struct northd_data *nd) struct northd_tracked_data *trk_changes = &nd->trk_data; destroy_tracked_ovn_ports(&trk_changes->trk_lsps); destroy_tracked_lbs(&trk_changes->trk_lbs); + hmapx_clear(&trk_changes->lr_with_changed_nats); nd->trk_data.type = NORTHD_TRACKED_NONE; } @@ -5205,6 +5029,7 @@ init_northd_tracked_data(struct northd_data *nd) hmapx_init(&trk_data->trk_lsps.deleted); hmapx_init(&trk_data->trk_lbs.crupdated); hmapx_init(&trk_data->trk_lbs.deleted); + hmapx_init(&trk_data->lr_with_changed_nats); } static void @@ -5217,6 +5042,7 @@ destroy_northd_tracked_data(struct northd_data *nd) hmapx_destroy(&trk_data->trk_lsps.deleted); hmapx_destroy(&trk_data->trk_lbs.crupdated); hmapx_destroy(&trk_data->trk_lbs.deleted); + hmapx_destroy(&trk_data->lr_with_changed_nats); } /* Check if a changed LSP can be handled incrementally within the I-P engine @@ -5577,7 +5403,7 @@ northd_handle_ls_changes(struct ovsdb_idl_txn *ovnsb_idl_txn, nbrec_logical_switch_is_deleted(changed_ls)) { goto fail; } - struct ovn_datapath *od = ovn_datapath_find( + struct ovn_datapath *od = ovn_datapath_find_( &nd->ls_datapaths.datapaths, &changed_ls->header_.uuid); if (!od) { @@ -5616,6 +5442,7 @@ fail: * incrementally handled. * Presently supports i-p for the below changes: * - load balancers and load balancer groups. + * - NAT changes */ static bool lr_changes_can_be_handled( @@ -5625,8 +5452,9 @@ lr_changes_can_be_handled( enum nbrec_logical_router_column_id col; for (col = 0; col < NBREC_LOGICAL_ROUTER_N_COLUMNS; col++) { if (nbrec_logical_router_is_updated(lr, col)) { - if (col == NBREC_LOGICAL_ROUTER_COL_LOAD_BALANCER || - col == NBREC_LOGICAL_ROUTER_COL_LOAD_BALANCER_GROUP) { + if (col == NBREC_LOGICAL_ROUTER_COL_LOAD_BALANCER + || col == NBREC_LOGICAL_ROUTER_COL_LOAD_BALANCER_GROUP + || col == NBREC_LOGICAL_ROUTER_COL_NAT) { continue; } return false; @@ -5645,12 +5473,6 @@ lr_changes_can_be_handled( OVSDB_IDL_CHANGE_MODIFY) > 0) { return false; } - for (size_t i = 0; i < lr->n_nat; i++) { - if (nbrec_nat_row_get_seqno(lr->nat[i], - OVSDB_IDL_CHANGE_MODIFY) > 0) { - return false; - } - } for (size_t i = 0; i < lr->n_policies; i++) { if (nbrec_logical_router_policy_row_get_seqno(lr->policies[i], OVSDB_IDL_CHANGE_MODIFY) > 0) { @@ -5666,6 +5488,39 @@ lr_changes_can_be_handled( return true; } +static bool +is_lr_nats_seqno_changed(const struct nbrec_logical_router *nbr) +{ + for (size_t i = 0; i < nbr->n_nat; i++) { + if (nbrec_nat_row_get_seqno(nbr->nat[i], + OVSDB_IDL_CHANGE_MODIFY) > 0) { + return true; + } + } + + return false; +} + +static bool +is_lr_nats_changed(const struct nbrec_logical_router *nbr) { + return (nbrec_logical_router_is_updated(nbr, + NBREC_LOGICAL_ROUTER_COL_NAT) + || nbrec_logical_router_is_updated( + nbr, NBREC_LOGICAL_ROUTER_COL_OPTIONS) + || is_lr_nats_seqno_changed(nbr)); +} + +static bool +lr_has_routable_nats(const struct nbrec_logical_router *nbr) { + for (size_t i = 0; i < nbr->n_nat; i++) { + if (smap_get_bool(&nbr->nat[i]->options, "add_route", false)) { + return true; + } + } + + return false; +} + /* Return true if changes are handled incrementally, false otherwise. * * Note: Changes to load balancer and load balancer groups associated with @@ -5685,11 +5540,37 @@ northd_handle_lr_changes(const struct northd_input *ni, goto fail; } - /* Presently only able to handle load balancer and - * load balancer group changes. */ + /* Presently only able to handle load balancer, + * load balancer group changes and NAT changes. */ if (!lr_changes_can_be_handled(changed_lr)) { goto fail; } + + if (is_lr_nats_changed(changed_lr)) { + if (lr_has_routable_nats(changed_lr)) { + /* router has routable NATs. We can't handle these changes + * incrementally yet. Fall back to recompute. */ + goto fail; + } + + struct ovn_datapath *od = ovn_datapath_find_( + &nd->lr_datapaths.datapaths, + &changed_lr->header_.uuid); + + if (!od) { + static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 1); + VLOG_WARN_RL(&rl, "Internal error: a tracked updated LR " + "doesn't exist in lr_datapaths: "UUID_FMT, + UUID_ARGS(&changed_lr->header_.uuid)); + goto fail; + } + + hmapx_add(&nd->trk_data.lr_with_changed_nats, od); + } + } + + if (!hmapx_is_empty(&nd->trk_data.lr_with_changed_nats)) { + nd->trk_data.type |= NORTHD_TRACKED_LR_NATS; } return true; @@ -5907,7 +5788,7 @@ northd_handle_lb_data_changes(struct tracked_lb_data *trk_lb_data, struct crupdated_od_lb_data *codlb; LIST_FOR_EACH (codlb, list_node, &trk_lb_data->crupdated_ls_lbs) { - od = ovn_datapath_find(&ls_datapaths->datapaths, &codlb->od_uuid); + od = ovn_datapath_find_(&ls_datapaths->datapaths, &codlb->od_uuid); ovs_assert(od); struct uuidset_node *uuidnode; @@ -5944,7 +5825,7 @@ northd_handle_lb_data_changes(struct tracked_lb_data *trk_lb_data, } LIST_FOR_EACH (codlb, list_node, &trk_lb_data->crupdated_lr_lbs) { - od = ovn_datapath_find(&lr_datapaths->datapaths, &codlb->od_uuid); + od = ovn_datapath_find_(&lr_datapaths->datapaths, &codlb->od_uuid); ovs_assert(od); struct uuidset_node *uuidnode; @@ -9291,31 +9172,15 @@ static void build_lswitch_rport_arp_req_self_orig_flow(struct ovn_port *op, uint32_t priority, struct ovn_datapath *od, + const struct lr_nat_table *lr_nats, struct hmap *lflows) { - struct sset all_eth_addrs = SSET_INITIALIZER(&all_eth_addrs); struct ds eth_src = DS_EMPTY_INITIALIZER; struct ds match = DS_EMPTY_INITIALIZER; - sset_add(&all_eth_addrs, op->lrp_networks.ea_s); - - for (size_t i = 0; i < op->od->nbr->n_nat; i++) { - struct ovn_nat *nat_entry = &op->od->nat_entries[i]; - const struct nbrec_nat *nat = nat_entry->nb; - - if (!nat_entry_is_valid(nat_entry)) { - continue; - } - - if (!strcmp(nat->type, "snat")) { - continue; - } - - if (!nat->external_mac) { - continue; - } - sset_add(&all_eth_addrs, nat->external_mac); - } + const struct lr_nat_record *lrnat_rec = lr_nat_table_find_by_index( + lr_nats, op->od->index); + ovs_assert(lrnat_rec); /* Self originated ARP requests/RARP/ND need to be flooded to the L2 domain * (except on router ports). Determine that packets are self originated @@ -9325,8 +9190,8 @@ build_lswitch_rport_arp_req_self_orig_flow(struct ovn_port *op, */ const char *eth_addr; - ds_put_cstr(ð_src, "{"); - SSET_FOR_EACH (eth_addr, &all_eth_addrs) { + ds_put_format(ð_src, "{%s, ", op->lrp_networks.ea_s); + SSET_FOR_EACH (eth_addr, &lrnat_rec->external_macs) { ds_put_format(ð_src, "%s, ", eth_addr); } ds_chomp(ð_src, ' '); @@ -9339,7 +9204,6 @@ build_lswitch_rport_arp_req_self_orig_flow(struct ovn_port *op, ovn_lflow_add(lflows, od, S_SWITCH_IN_L2_LKUP, priority, ds_cstr(&match), "outport = \""MC_FLOOD_L2"\"; output;"); - sset_destroy(&all_eth_addrs); ds_destroy(ð_src); ds_destroy(&match); } @@ -9445,6 +9309,7 @@ static void build_lswitch_rport_arp_req_flows(struct ovn_port *op, struct ovn_datapath *sw_od, struct ovn_port *sw_op, + const struct lr_nat_table *lr_nats, struct hmap *lflows, const struct ovsdb_idl_row *stage_hint) { @@ -9489,8 +9354,38 @@ build_lswitch_rport_arp_req_flows(struct ovn_port *op, } } - for (size_t i = 0; i < op->od->nbr->n_nat; i++) { - struct ovn_nat *nat_entry = &op->od->nat_entries[i]; + for (size_t i = 0; i < op->lrp_networks.n_ipv4_addrs; i++) { + build_lswitch_rport_arp_req_flow( + op->lrp_networks.ipv4_addrs[i].addr_s, AF_INET, sw_op, sw_od, 80, + lflows, stage_hint); + } + for (size_t i = 0; i < op->lrp_networks.n_ipv6_addrs; i++) { + build_lswitch_rport_arp_req_flow( + op->lrp_networks.ipv6_addrs[i].addr_s, AF_INET6, sw_op, sw_od, 80, + lflows, stage_hint); + } + + /* Self originated ARP requests/RARP/ND need to be flooded as usual. + * + * However, if the switch doesn't have any non-router ports we shouldn't + * even try to flood. + * + * Priority: 75. + */ + if (sw_od->n_router_ports != sw_od->nbs->n_ports) { + build_lswitch_rport_arp_req_self_orig_flow(op, 75, sw_od, lr_nats, + lflows); + } + + const struct lr_nat_record *lrnat_rec = + lr_nat_table_find_by_index(lr_nats, op->od->index); + + if (!lrnat_rec) { + return; + } + + for (size_t i = 0; i < lrnat_rec->n_nat_entries; i++) { + struct ovn_nat *nat_entry = &lrnat_rec->nat_entries[i]; const struct nbrec_nat *nat = nat_entry->nb; if (!nat_entry_is_valid(nat_entry)) { @@ -9520,7 +9415,7 @@ build_lswitch_rport_arp_req_flows(struct ovn_port *op, } struct shash_node *snat_snode; - SHASH_FOR_EACH (snat_snode, &op->od->snat_ips) { + SHASH_FOR_EACH (snat_snode, &lrnat_rec->snat_ips) { struct ovn_snat_ip *snat_ip = snat_snode->data; if (ovs_list_is_empty(&snat_ip->snat_entries)) { @@ -9549,28 +9444,6 @@ build_lswitch_rport_arp_req_flows(struct ovn_port *op, } } } - - for (size_t i = 0; i < op->lrp_networks.n_ipv4_addrs; i++) { - build_lswitch_rport_arp_req_flow( - op->lrp_networks.ipv4_addrs[i].addr_s, AF_INET, sw_op, sw_od, 80, - lflows, stage_hint); - } - for (size_t i = 0; i < op->lrp_networks.n_ipv6_addrs; i++) { - build_lswitch_rport_arp_req_flow( - op->lrp_networks.ipv6_addrs[i].addr_s, AF_INET6, sw_op, sw_od, 80, - lflows, stage_hint); - } - - /* Self originated ARP requests/RARP/ND need to be flooded as usual. - * - * However, if the switch doesn't have any non-router ports we shouldn't - * even try to flood. - * - * Priority: 75. - */ - if (sw_od->n_router_ports != sw_od->nbs->n_ports) { - build_lswitch_rport_arp_req_self_orig_flow(op, 75, sw_od, lflows); - } } static void @@ -10604,6 +10477,7 @@ build_lswitch_ip_mcast_igmp_mld(struct ovn_igmp_group *igmp_group, /* Ingress table 25: Destination lookup, unicast handling (priority 50), */ static void build_lswitch_ip_unicast_lookup(struct ovn_port *op, + const struct lr_nat_table *lr_nats, struct hmap *lflows, struct ds *actions, struct ds *match) @@ -10618,8 +10492,8 @@ build_lswitch_ip_unicast_lookup(struct ovn_port *op, * requests only to the router port that owns the IP address. */ if (lsp_is_router(op->nbsp)) { - build_lswitch_rport_arp_req_flows(op->peer, op->od, op, lflows, - &op->nbsp->header_); + build_lswitch_rport_arp_req_flows(op->peer, op->od, op, lr_nats, + lflows, &op->nbsp->header_); } for (size_t i = 0; i < op->nbsp->n_addresses; i++) { @@ -11982,27 +11856,6 @@ op_put_v6_networks(struct ds *ds, const struct ovn_port *op) ds_put_cstr(ds, "}"); } -static bool -get_force_snat_ip(struct ovn_datapath *od, const char *key_type, - struct lport_addresses *laddrs) -{ - char *key = xasprintf("%s_force_snat_ip", key_type); - const char *addresses = smap_get(&od->nbr->options, key); - free(key); - - if (!addresses) { - return false; - } - - if (!extract_ip_address(addresses, laddrs)) { - static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1); - VLOG_WARN_RL(&rl, "bad ip %s in options of router "UUID_FMT"", - addresses, UUID_ARGS(&od->key)); - return false; - } - - return true; -} enum lrouter_nat_lb_flow_type { LROUTER_NAT_LB_FLOW_NORMAL = 0, @@ -12154,6 +12007,7 @@ build_lrouter_nat_flows_for_lb(struct ovn_lb_vip *lb_vip, struct ovn_lb_datapaths *lb_dps, struct ovn_northd_lb_vip *vips_nb, const struct ovn_datapaths *lr_datapaths, + const struct lr_nat_table *lr_nats, struct hmap *lflows, struct ds *match, struct ds *action, const struct shash *meter_groups, @@ -12259,10 +12113,13 @@ build_lrouter_nat_flows_for_lb(struct ovn_lb_vip *lb_vip, struct ovn_datapath *od = lr_datapaths->array[index]; enum lrouter_nat_lb_flow_type type; + const struct lr_nat_record *lrnat_rec = + lr_nat_table_find_by_index(lr_nats, od->index); + ovs_assert(lrnat_rec); if (lb->skip_snat) { type = LROUTER_NAT_LB_FLOW_SKIP_SNAT; - } else if (!lport_addresses_is_empty(&od->lb_force_snat_addrs) || - od->lb_force_snat_router_ip) { + } else if (!lport_addresses_is_empty(&lrnat_rec->lb_force_snat_addrs) + || lrnat_rec->lb_force_snat_router_ip) { type = LROUTER_NAT_LB_FLOW_FORCE_SNAT; } else { type = LROUTER_NAT_LB_FLOW_NORMAL; @@ -12278,7 +12135,7 @@ build_lrouter_nat_flows_for_lb(struct ovn_lb_vip *lb_vip, bitmap_set1(aff_dp_bitmap[type], index); } - if (sset_contains(&od->external_ips, lb_vip->vip_str)) { + if (sset_contains(&lrnat_rec->external_ips, lb_vip->vip_str)) { /* The load balancer vip is also present in the NAT entries. * So add a high priority lflow to advance the the packet * destined to the vip (and the vip port if defined) @@ -12408,6 +12265,7 @@ build_lrouter_flows_for_lb(struct ovn_lb_datapaths *lb_dps, struct hmap *lflows, const struct shash *meter_groups, const struct ovn_datapaths *lr_datapaths, + const struct lr_nat_table *lr_nats, const struct chassis_features *features, const struct hmap *svc_monitor_map, struct ds *match, struct ds *action) @@ -12423,8 +12281,8 @@ build_lrouter_flows_for_lb(struct ovn_lb_datapaths *lb_dps, struct ovn_lb_vip *lb_vip = &lb->vips[i]; build_lrouter_nat_flows_for_lb(lb_vip, lb_dps, &lb->vips_nb[i], - lr_datapaths, lflows, match, action, - meter_groups, features, + lr_datapaths, lr_nats, lflows, match, + action, meter_groups, features, svc_monitor_map); if (!build_empty_lb_event_flow(lb_vip, lb, match, action)) { @@ -12823,7 +12681,9 @@ build_lrouter_port_nat_arp_nd_flow(struct ovn_port *op, } static void -build_lrouter_drop_own_dest(struct ovn_port *op, enum ovn_stage stage, +build_lrouter_drop_own_dest(struct ovn_port *op, + const struct lr_nat_record *lrnat_rec, + enum ovn_stage stage, uint16_t priority, bool drop_snat_ip, struct hmap *lflows) { @@ -12833,7 +12693,8 @@ build_lrouter_drop_own_dest(struct ovn_port *op, enum ovn_stage stage, for (size_t i = 0; i < op->lrp_networks.n_ipv4_addrs; i++) { const char *ip = op->lrp_networks.ipv4_addrs[i].addr_s; - bool router_ip_in_snat_ips = !!shash_find(&op->od->snat_ips, ip); + bool router_ip_in_snat_ips = !!shash_find(&lrnat_rec->snat_ips, + ip); bool router_ip_in_lb_ips = !!sset_find(&op->od->lb_ips->ips_v4, ip); bool drop_router_ip = (drop_snat_ip == (router_ip_in_snat_ips || @@ -12862,7 +12723,8 @@ build_lrouter_drop_own_dest(struct ovn_port *op, enum ovn_stage stage, for (size_t i = 0; i < op->lrp_networks.n_ipv6_addrs; i++) { const char *ip = op->lrp_networks.ipv6_addrs[i].addr_s; - bool router_ip_in_snat_ips = !!shash_find(&op->od->snat_ips, ip); + bool router_ip_in_snat_ips = !!shash_find(&lrnat_rec->snat_ips, + ip); bool router_ip_in_lb_ips = !!sset_find(&op->od->lb_ips->ips_v6, ip); bool drop_router_ip = (drop_snat_ip == (router_ip_in_snat_ips || @@ -12915,11 +12777,12 @@ build_lrouter_force_snat_flows(struct hmap *lflows, struct ovn_datapath *od, static void build_lrouter_force_snat_flows_op(struct ovn_port *op, + const struct lr_nat_record *lrnat_rec, struct hmap *lflows, struct ds *match, struct ds *actions) { ovs_assert(op->nbrp); - if (!op->peer || !op->od->lb_force_snat_router_ip) { + if (!op->peer || !lrnat_rec->lb_force_snat_router_ip) { return; } @@ -13872,8 +13735,8 @@ routable_addresses_to_lflows(struct hmap *lflows, struct ovn_port *router_port, /* This function adds ARP resolve flows related to a LRP. */ static void build_arp_resolve_flows_for_lrp( - struct ovn_port *op, struct hmap *lflows, - struct ds *match, struct ds *actions) + struct ovn_port *op, const struct lr_nat_record *lrnat_rec, + struct hmap *lflows, struct ds *match, struct ds *actions) { ovs_assert(op->nbrp); /* This is a logical router port. If next-hop IP address in @@ -13949,8 +13812,8 @@ build_arp_resolve_flows_for_lrp( * * Priority 2. */ - build_lrouter_drop_own_dest(op, S_ROUTER_IN_ARP_RESOLVE, 2, true, - lflows); + build_lrouter_drop_own_dest(op, lrnat_rec, S_ROUTER_IN_ARP_RESOLVE, 2, + true, lflows); } /* This function adds ARP resolve flows related to a LSP. */ @@ -14280,6 +14143,7 @@ build_check_pkt_len_flows_for_lrouter( static void build_gateway_redirect_flows_for_lrouter( struct ovn_datapath *od, struct hmap *lflows, + const struct lr_nat_table *lr_nats, struct ds *match, struct ds *actions) { ovs_assert(od->nbr); @@ -14315,8 +14179,16 @@ build_gateway_redirect_flows_for_lrouter( ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_GW_REDIRECT, 50, ds_cstr(match), ds_cstr(actions), stage_hint); - for (int j = 0; j < od->n_nat_entries; j++) { - const struct ovn_nat *nat = &od->nat_entries[j]; + + const struct lr_nat_record *lrnat_rec = lr_nat_table_find_by_index( + lr_nats, od->index); + + if (!lrnat_rec) { + continue; + } + + for (int j = 0; j < lrnat_rec->n_nat_entries; j++) { + const struct ovn_nat *nat = &lrnat_rec->nat_entries[j]; if (!lrouter_dnat_and_snat_is_stateless(nat->nb) || (!nat->nb->allowed_ext_ips && !nat->nb->exempted_ext_ips)) { @@ -14754,10 +14626,15 @@ build_ipv6_input_flows_for_lrouter_port( static void build_lrouter_arp_nd_for_datapath(struct ovn_datapath *od, + const struct lr_nat_table *lr_nats, struct hmap *lflows, const struct shash *meter_groups) { ovs_assert(od->nbr); + if (!od->nbr->n_nat) { + return; + } + /* Priority-90-92 flows handle ARP requests and ND packets. Most are * per logical port but DNAT addresses can be handled per datapath * for non gateway router ports. @@ -14766,8 +14643,12 @@ build_lrouter_arp_nd_for_datapath(struct ovn_datapath *od, * port to handle the special cases. In case we get the packet * on a regular port, just reply with the port's ETH address. */ - for (int i = 0; i < od->nbr->n_nat; i++) { - struct ovn_nat *nat_entry = &od->nat_entries[i]; + const struct lr_nat_record *lrnat_rec = lr_nat_table_find_by_index( + lr_nats, od->index); + ovs_assert(lrnat_rec); + + for (int i = 0; i < lrnat_rec->n_nat_entries; i++) { + struct ovn_nat *nat_entry = &lrnat_rec->nat_entries[i]; /* Skip entries we failed to parse. */ if (!nat_entry_is_valid(nat_entry)) { @@ -14775,8 +14656,8 @@ build_lrouter_arp_nd_for_datapath(struct ovn_datapath *od, } /* Skip SNAT entries for now, we handle unique SNAT IPs separately - * below. - */ + * below. + */ if (!strcmp(nat_entry->nb->type, "snat")) { continue; } @@ -14785,7 +14666,7 @@ build_lrouter_arp_nd_for_datapath(struct ovn_datapath *od, /* Now handle SNAT entries too, one per unique SNAT IP. */ struct shash_node *snat_snode; - SHASH_FOR_EACH (snat_snode, &od->snat_ips) { + SHASH_FOR_EACH (snat_snode, &lrnat_rec->snat_ips) { struct ovn_snat_ip *snat_ip = snat_snode->data; if (ovs_list_is_empty(&snat_ip->snat_entries)) { @@ -14803,6 +14684,7 @@ build_lrouter_arp_nd_for_datapath(struct ovn_datapath *od, static void build_lrouter_ipv4_ip_input(struct ovn_port *op, struct hmap *lflows, + const struct lr_nat_record *lrnat_rec, struct ds *match, struct ds *actions, const struct shash *meter_groups) { @@ -15039,14 +14921,14 @@ build_lrouter_ipv4_ip_input(struct ovn_port *op, * also a SNAT IP. Those are dropped later, in stage * "lr_in_arp_resolve", if unSNAT was unsuccessful. * - * If op->od->lb_force_snat_router_ip is true, it means the IP of the + * If lrnat_rec->lb_force_snat_router_ip is true, it means the IP of the * router port is also SNAT IP. * * Priority 60. */ - if (!op->od->lb_force_snat_router_ip) { - build_lrouter_drop_own_dest(op, S_ROUTER_IN_IP_INPUT, 60, false, - lflows); + if (!lrnat_rec->lb_force_snat_router_ip) { + build_lrouter_drop_own_dest(op, lrnat_rec, S_ROUTER_IN_IP_INPUT, 60, + false, lflows); } /* ARP / ND handling for external IP addresses. * @@ -15061,8 +14943,8 @@ build_lrouter_ipv4_ip_input(struct ovn_port *op, return; } - for (size_t i = 0; i < op->od->nbr->n_nat; i++) { - struct ovn_nat *nat_entry = &op->od->nat_entries[i]; + for (int i = 0; i < lrnat_rec->n_nat_entries; i++) { + struct ovn_nat *nat_entry = &lrnat_rec->nat_entries[i]; /* Skip entries we failed to parse. */ if (!nat_entry_is_valid(nat_entry)) { @@ -15070,18 +14952,18 @@ build_lrouter_ipv4_ip_input(struct ovn_port *op, } /* Skip SNAT entries for now, we handle unique SNAT IPs separately - * below. - */ + * below. + */ if (!strcmp(nat_entry->nb->type, "snat")) { continue; } build_lrouter_port_nat_arp_nd_flow(op, nat_entry, lflows, - meter_groups); + meter_groups); } /* Now handle SNAT entries too, one per unique SNAT IP. */ struct shash_node *snat_snode; - SHASH_FOR_EACH (snat_snode, &op->od->snat_ips) { + SHASH_FOR_EACH (snat_snode, &lrnat_rec->snat_ips) { struct ovn_snat_ip *snat_ip = snat_snode->data; if (ovs_list_is_empty(&snat_ip->snat_entries)) { @@ -15090,9 +14972,9 @@ build_lrouter_ipv4_ip_input(struct ovn_port *op, struct ovn_nat *nat_entry = CONTAINER_OF(ovs_list_front(&snat_ip->snat_entries), - struct ovn_nat, ext_addr_list_node); + struct ovn_nat, ext_addr_list_node); build_lrouter_port_nat_arp_nd_flow(op, nat_entry, lflows, - meter_groups); + meter_groups); } } @@ -15201,6 +15083,7 @@ build_lrouter_in_unsnat_flow(struct hmap *lflows, struct ovn_datapath *od, static void build_lrouter_in_dnat_flow(struct hmap *lflows, struct ovn_datapath *od, + const struct lr_nat_record *lrnat_rec, const struct nbrec_nat *nat, struct ds *match, struct ds *actions, bool distributed_nat, int cidr_bits, bool is_v6, @@ -15224,7 +15107,7 @@ build_lrouter_in_dnat_flow(struct hmap *lflows, struct ovn_datapath *od, nat->external_ip); if (od->is_gw_router) { - if (!lport_addresses_is_empty(&od->dnat_force_snat_addrs)) { + if (!lport_addresses_is_empty(&lrnat_rec->dnat_force_snat_addrs)) { /* Indicate to the future tables that a DNAT has taken * place and a force SNAT needs to be done in the * Egress SNAT table. */ @@ -15780,6 +15663,7 @@ static void build_lrouter_nat_defrag_and_lb(struct ovn_datapath *od, struct hmap *lflows, const struct hmap *ls_ports, const struct hmap *lr_ports, + const struct lr_nat_table *lr_nats, struct ds *match, struct ds *actions, const struct shash *meter_groups, @@ -15890,14 +15774,18 @@ build_lrouter_nat_defrag_and_lb(struct ovn_datapath *od, struct hmap *lflows, } struct sset nat_entries = SSET_INITIALIZER(&nat_entries); + const struct lr_nat_record *lrnat_rec = lr_nat_table_find_by_index(lr_nats, + od->index); + ovs_assert(lrnat_rec); bool dnat_force_snat_ip = - !lport_addresses_is_empty(&od->dnat_force_snat_addrs); + !lport_addresses_is_empty(&lrnat_rec->dnat_force_snat_addrs); bool lb_force_snat_ip = - !lport_addresses_is_empty(&od->lb_force_snat_addrs); + !lport_addresses_is_empty(&lrnat_rec->lb_force_snat_addrs); - for (int i = 0; i < od->nbr->n_nat; i++) { - const struct nbrec_nat *nat = od->nbr->nat[i]; + for (int i = 0; i < lrnat_rec->n_nat_entries; i++) { + struct ovn_nat *nat_entry = &lrnat_rec->nat_entries[i]; + const struct nbrec_nat *nat = nat_entry->nb; struct eth_addr mac = eth_addr_broadcast; bool is_v6, distributed_nat; ovs_be32 mask; @@ -15935,7 +15823,7 @@ build_lrouter_nat_defrag_and_lb(struct ovn_datapath *od, struct hmap *lflows, distributed_nat, is_v6, l3dgw_port); } /* S_ROUTER_IN_DNAT */ - build_lrouter_in_dnat_flow(lflows, od, nat, match, actions, + build_lrouter_in_dnat_flow(lflows, od, lrnat_rec, nat, match, actions, distributed_nat, cidr_bits, is_v6, l3dgw_port, stateless); @@ -16144,25 +16032,25 @@ build_lrouter_nat_defrag_and_lb(struct ovn_datapath *od, struct hmap *lflows, /* Handle force SNAT options set in the gateway router. */ if (od->is_gw_router) { if (dnat_force_snat_ip) { - if (od->dnat_force_snat_addrs.n_ipv4_addrs) { + if (lrnat_rec->dnat_force_snat_addrs.n_ipv4_addrs) { build_lrouter_force_snat_flows(lflows, od, "4", - od->dnat_force_snat_addrs.ipv4_addrs[0].addr_s, + lrnat_rec->dnat_force_snat_addrs.ipv4_addrs[0].addr_s, "dnat"); } - if (od->dnat_force_snat_addrs.n_ipv6_addrs) { + if (lrnat_rec->dnat_force_snat_addrs.n_ipv6_addrs) { build_lrouter_force_snat_flows(lflows, od, "6", - od->dnat_force_snat_addrs.ipv6_addrs[0].addr_s, + lrnat_rec->dnat_force_snat_addrs.ipv6_addrs[0].addr_s, "dnat"); } } if (lb_force_snat_ip) { - if (od->lb_force_snat_addrs.n_ipv4_addrs) { + if (lrnat_rec->lb_force_snat_addrs.n_ipv4_addrs) { build_lrouter_force_snat_flows(lflows, od, "4", - od->lb_force_snat_addrs.ipv4_addrs[0].addr_s, "lb"); + lrnat_rec->lb_force_snat_addrs.ipv4_addrs[0].addr_s, "lb"); } - if (od->lb_force_snat_addrs.n_ipv6_addrs) { + if (lrnat_rec->lb_force_snat_addrs.n_ipv6_addrs) { build_lrouter_force_snat_flows(lflows, od, "6", - od->lb_force_snat_addrs.ipv6_addrs[0].addr_s, "lb"); + lrnat_rec->lb_force_snat_addrs.ipv6_addrs[0].addr_s, "lb"); } } } @@ -16178,6 +16066,7 @@ struct lswitch_flow_build_info { const struct hmap *ls_ports; const struct hmap *lr_ports; const struct ls_port_group_table *ls_port_groups; + const struct lr_nat_table *lr_nats; struct hmap *lflows; struct hmap *igmp_groups; const struct shash *meter_groups; @@ -16243,14 +16132,15 @@ build_lswitch_and_lrouter_iterate_by_lr(struct ovn_datapath *od, build_check_pkt_len_flows_for_lrouter(od, lsi->lflows, lsi->lr_ports, &lsi->match, &lsi->actions, lsi->meter_groups); - build_gateway_redirect_flows_for_lrouter(od, lsi->lflows, &lsi->match, - &lsi->actions); + build_gateway_redirect_flows_for_lrouter(od, lsi->lflows, lsi->lr_nats, + &lsi->match, &lsi->actions); build_arp_request_flows_for_lrouter(od, lsi->lflows, &lsi->match, &lsi->actions, lsi->meter_groups); build_misc_local_traffic_drop_flows_for_lrouter(od, lsi->lflows); - build_lrouter_arp_nd_for_datapath(od, lsi->lflows, lsi->meter_groups); + build_lrouter_arp_nd_for_datapath(od, lsi->lr_nats, lsi->lflows, + lsi->meter_groups); build_lrouter_nat_defrag_and_lb(od, lsi->lflows, lsi->ls_ports, - lsi->lr_ports, &lsi->match, + lsi->lr_ports,lsi->lr_nats, &lsi->match, &lsi->actions, lsi->meter_groups, lsi->features); build_lrouter_lb_affinity_default_flows(od, lsi->lflows); @@ -16263,6 +16153,7 @@ static void build_lswitch_and_lrouter_iterate_by_lsp(struct ovn_port *op, const struct hmap *ls_ports, const struct hmap *lr_ports, + const struct lr_nat_table *lr_nats, const struct shash *meter_groups, struct ds *match, struct ds *actions, @@ -16279,7 +16170,7 @@ build_lswitch_and_lrouter_iterate_by_lsp(struct ovn_port *op, meter_groups, actions, match); build_lswitch_dhcp_options_and_response(op, lflows, meter_groups); build_lswitch_external_port(op, lflows); - build_lswitch_ip_unicast_lookup(op, lflows, actions, match); + build_lswitch_ip_unicast_lookup(op, lr_nats, lflows, actions, match); /* Build Logical Router Flows. */ build_ip_routing_flows_for_router_type_lsp(op, lr_ports, lflows); @@ -16297,6 +16188,11 @@ build_lswitch_and_lrouter_iterate_by_lrp(struct ovn_port *op, struct lswitch_flow_build_info *lsi) { ovs_assert(op->nbrp); + + const struct lr_nat_record *lrnet_rec = lr_nat_table_find_by_index( + lsi->lr_nats, op->od->index); + ovs_assert(lrnet_rec); + build_adm_ctrl_flows_for_lrouter_port(op, lsi->lflows, &lsi->match, &lsi->actions); build_neigh_learning_flows_for_lrouter_port(op, lsi->lflows, &lsi->match, @@ -16304,7 +16200,7 @@ build_lswitch_and_lrouter_iterate_by_lrp(struct ovn_port *op, build_ip_routing_flows_for_lrp(op, lsi->lflows); build_ND_RA_flows_for_lrouter_port(op, lsi->lflows, &lsi->match, &lsi->actions, lsi->meter_groups); - build_arp_resolve_flows_for_lrp(op, lsi->lflows, &lsi->match, + build_arp_resolve_flows_for_lrp(op, lrnet_rec, lsi->lflows, &lsi->match, &lsi->actions); build_egress_delivery_flows_for_lrouter_port(op, lsi->lflows, &lsi->match, &lsi->actions); @@ -16312,9 +16208,9 @@ build_lswitch_and_lrouter_iterate_by_lrp(struct ovn_port *op, build_ipv6_input_flows_for_lrouter_port(op, lsi->lflows, &lsi->match, &lsi->actions, lsi->meter_groups); - build_lrouter_ipv4_ip_input(op, lsi->lflows, + build_lrouter_ipv4_ip_input(op, lsi->lflows, lrnet_rec, &lsi->match, &lsi->actions, lsi->meter_groups); - build_lrouter_force_snat_flows_op(op, lsi->lflows, &lsi->match, + build_lrouter_force_snat_flows_op(op, lrnet_rec, lsi->lflows, &lsi->match, &lsi->actions); } @@ -16374,6 +16270,7 @@ build_lflows_thread(void *arg) } build_lswitch_and_lrouter_iterate_by_lsp(op, lsi->ls_ports, lsi->lr_ports, + lsi->lr_nats, lsi->meter_groups, &lsi->match, &lsi->actions, @@ -16412,6 +16309,7 @@ build_lflows_thread(void *arg) build_lrouter_flows_for_lb(lb_dps, lsi->lflows, lsi->meter_groups, lsi->lr_datapaths, + lsi->lr_nats, lsi->features, lsi->svc_monitor_map, &lsi->match, &lsi->actions); @@ -16482,6 +16380,7 @@ build_lswitch_and_lrouter_flows(const struct ovn_datapaths *ls_datapaths, const struct hmap *ls_ports, const struct hmap *lr_ports, const struct ls_port_group_table *ls_pgs, + const struct lr_nat_table *lr_nats, struct hmap *lflows, struct hmap *igmp_groups, const struct shash *meter_groups, @@ -16511,6 +16410,7 @@ build_lswitch_and_lrouter_flows(const struct ovn_datapaths *ls_datapaths, lsiv[index].ls_ports = ls_ports; lsiv[index].lr_ports = lr_ports; lsiv[index].ls_port_groups = ls_pgs; + lsiv[index].lr_nats = lr_nats; lsiv[index].igmp_groups = igmp_groups; lsiv[index].meter_groups = meter_groups; lsiv[index].lb_dps_map = lb_dps_map; @@ -16545,6 +16445,7 @@ build_lswitch_and_lrouter_flows(const struct ovn_datapaths *ls_datapaths, .ls_ports = ls_ports, .lr_ports = lr_ports, .ls_port_groups = ls_pgs, + .lr_nats = lr_nats, .lflows = lflows, .igmp_groups = igmp_groups, .meter_groups = meter_groups, @@ -16572,6 +16473,7 @@ build_lswitch_and_lrouter_flows(const struct ovn_datapaths *ls_datapaths, HMAP_FOR_EACH (op, key_node, ls_ports) { build_lswitch_and_lrouter_iterate_by_lsp(op, lsi.ls_ports, lsi.lr_ports, + lsi.lr_nats, lsi.meter_groups, &lsi.match, &lsi.actions, lsi.lflows); @@ -16588,8 +16490,8 @@ build_lswitch_and_lrouter_flows(const struct ovn_datapaths *ls_datapaths, build_lrouter_defrag_flows_for_lb(lb_dps, lsi.lflows, lsi.lr_datapaths, &lsi.match); build_lrouter_flows_for_lb(lb_dps, lsi.lflows, lsi.meter_groups, - lsi.lr_datapaths, lsi.features, - lsi.svc_monitor_map, + lsi.lr_datapaths, lsi.lr_nats, + lsi.features, lsi.svc_monitor_map, &lsi.match, &lsi.actions); build_lswitch_flows_for_lb(lb_dps, lsi.lflows, lsi.meter_groups, lsi.ls_datapaths, lsi.features, @@ -16692,6 +16594,7 @@ void build_lflows(struct ovsdb_idl_txn *ovnsb_txn, input_data->ls_ports, input_data->lr_ports, input_data->ls_port_groups, + input_data->lr_nats, lflows, &igmp_groups, input_data->meter_groups, @@ -17169,6 +17072,7 @@ lflow_handle_northd_port_changes(struct ovsdb_idl_txn *ovnsb_txn, struct ds actions = DS_EMPTY_INITIALIZER; build_lswitch_and_lrouter_iterate_by_lsp(op, lflow_input->ls_ports, lflow_input->lr_ports, + lflow_input->lr_nats, lflow_input->meter_groups, &match, &actions, lflows); @@ -17205,6 +17109,7 @@ lflow_handle_northd_port_changes(struct ovsdb_idl_txn *ovnsb_txn, struct ds actions = DS_EMPTY_INITIALIZER; build_lswitch_and_lrouter_iterate_by_lsp(op, lflow_input->ls_ports, lflow_input->lr_ports, + lflow_input->lr_nats, lflow_input->meter_groups, &match, &actions, lflows); diff --git a/northd/northd.h b/northd/northd.h index 233dca8084..4dd260761e 100644 --- a/northd/northd.h +++ b/northd/northd.h @@ -83,6 +83,12 @@ struct ovn_datapaths { struct ovn_datapath **array; }; +static inline size_t +ods_size(const struct ovn_datapaths *datapaths) +{ + return hmap_count(&datapaths->datapaths); +} + struct tracked_ovn_ports { /* tracked created ports. * hmapx node data is 'struct ovn_port *' */ @@ -109,8 +115,9 @@ struct tracked_lbs { enum northd_tracked_data_type { NORTHD_TRACKED_NONE, - NORTHD_TRACKED_PORTS = (1 << 0), - NORTHD_TRACKED_LBS = (1 << 1), + NORTHD_TRACKED_PORTS = (1 << 0), + NORTHD_TRACKED_LBS = (1 << 1), + NORTHD_TRACKED_LR_NATS = (1 << 2), }; /* Track what's changed in the northd engine node. @@ -121,6 +128,10 @@ struct northd_tracked_data { enum northd_tracked_data_type type; struct tracked_ovn_ports trk_lsps; struct tracked_lbs trk_lbs; + + /* Tracked logical routers whose NATs have changed. + * hmapx node is 'struct ovn_datapath *'. */ + struct hmapx lr_with_changed_nats; }; struct northd_data { @@ -148,6 +159,8 @@ struct lflow_data { void lflow_data_init(struct lflow_data *); void lflow_data_destroy(struct lflow_data *); +struct lr_nat_table; + struct lflow_input { /* Northbound table references */ const struct nbrec_bfd_table *nbrec_bfd_table; @@ -166,6 +179,7 @@ struct lflow_input { const struct hmap *ls_ports; const struct hmap *lr_ports; const struct ls_port_group_table *ls_port_groups; + const struct lr_nat_table *lr_nats; const struct shash *meter_groups; const struct hmap *lb_datapaths_map; const struct hmap *bfd_connections; @@ -302,24 +316,9 @@ struct ovn_datapath { struct ovn_port **l3dgw_ports; size_t n_l3dgw_ports; - /* NAT entries configured on the router. */ - struct ovn_nat *nat_entries; - size_t n_nat_entries; - - bool has_distributed_nat; /* router datapath has a logical port with redirect-type set to bridged. */ bool redirect_bridged; - /* Set of nat external ips on the router. */ - struct sset external_ips; - - /* SNAT IPs owned by the router (shash of 'struct ovn_snat_ip'). */ - struct shash snat_ips; - - struct lport_addresses dnat_force_snat_addrs; - struct lport_addresses lb_force_snat_addrs; - bool lb_force_snat_router_ip; - /* Load Balancer vIPs relevant for this datapath. */ struct ovn_lb_ip_set *lb_ips; @@ -336,6 +335,9 @@ struct ovn_datapath { struct hmap ports; }; +const struct ovn_datapath *ovn_datapath_find(const struct hmap *datapaths, + const struct uuid *uuid); + void ovnnb_db_run(struct northd_input *input_data, struct northd_data *data, struct ovsdb_idl_txn *ovnnb_txn, @@ -396,8 +398,8 @@ void sync_lbs(struct ovsdb_idl_txn *, const struct sbrec_load_balancer_table *, bool check_sb_lb_duplicates(const struct sbrec_load_balancer_table *); void sync_pbs(struct ovsdb_idl_txn *, struct hmap *ls_ports, - struct hmap *lr_ports); -bool sync_pbs_for_northd_changed_ovn_ports( struct tracked_ovn_ports *); + struct hmap *lr_ports, const struct lr_nat_table *); +bool sync_pbs_for_northd_changed_ovn_ports(struct tracked_ovn_ports *); static inline bool northd_has_tracked_data(struct northd_tracked_data *trk_nd_changes) { @@ -416,4 +418,10 @@ northd_has_lsps_in_tracked_data(struct northd_tracked_data *trk_nd_changes) return (trk_nd_changes->type & NORTHD_TRACKED_PORTS); } +static inline bool +northd_has_lr_nats_in_tracked_data(struct northd_tracked_data *trk_nd_changes) +{ + return (trk_nd_changes->type & NORTHD_TRACKED_LR_NATS); +} + #endif /* NORTHD_H */ diff --git a/northd/ovn-northd.c b/northd/ovn-northd.c index f3868068d3..40f9764b3a 100644 --- a/northd/ovn-northd.c +++ b/northd/ovn-northd.c @@ -870,6 +870,7 @@ main(int argc, char *argv[]) stopwatch_create(LFLOWS_TO_SB_STOPWATCH_NAME, SW_MS); stopwatch_create(PORT_GROUP_RUN_STOPWATCH_NAME, SW_MS); stopwatch_create(SYNC_METERS_RUN_STOPWATCH_NAME, SW_MS); + stopwatch_create(LR_NAT_RUN_STOPWATCH_NAME, SW_MS); /* Initialize incremental processing engine for ovn-northd */ inc_proc_northd_init(&ovnnb_idl_loop, &ovnsb_idl_loop); diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at index 4edad24e53..ef8c6a616b 100644 --- a/tests/ovn-northd.at +++ b/tests/ovn-northd.at @@ -11240,6 +11240,7 @@ check ovn-nbctl --wait=sb lsp-add sw0 sw0p1 -- lsp-set-addresses sw0p1 "00:00:20 check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb lr-add lr0 check_engine_stats northd recompute nocompute +check_engine_stats lr_nat recompute nocompute check_engine_stats sync_to_sb_pb recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11251,6 +11252,7 @@ check ovn-nbctl lrp-add lr0 lr0-sw0 00:00:00:00:ff:01 10.0.0.1/24 # first it will be recompute to handle lr0-sw0 and then a compute # for the SB port binding change. check_engine_stats northd recompute compute +check_engine_stats lr_nat recompute nocompute check_engine_stats sync_to_sb_pb recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11261,6 +11263,7 @@ ovn-nbctl lsp-set-addresses sw0-lr0 00:00:00:00:ff:01 check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb lsp-set-options sw0-lr0 router-port=lr0-sw0 check_engine_stats northd recompute compute +check_engine_stats lr_nat recompute nocompute check_engine_stats sync_to_sb_pb recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11285,16 +11288,18 @@ ovn-nbctl --wait=hv lrp-set-gateway-chassis lr0-public hv1 20 check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl set logical_router_port lr0-sw0 options:foo=bar check_engine_stats northd recompute nocompute +check_engine_stats lr_nat recompute nocompute check_engine_stats sync_to_sb_pb recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Do checks for NATs. -# Add a NAT. This should not result in recompute of both northd and lflow -# engine nodes. +# Add a NAT. This should not result in recompute of northd, but +# recompute of lflow node. check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb lr-nat-add lr0 dnat_and_snat 172.168.0.110 10.0.0.4 -check_engine_stats northd recompute nocompute +check_engine_stats northd norecompute compute +check_engine_stats lr_nat norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_pb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11302,7 +11307,8 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE # Update the NAT options column check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb set NAT . options:foo=bar -check_engine_stats northd recompute nocompute +check_engine_stats northd norecompute compute +check_engine_stats lr_nat norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_pb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11310,7 +11316,8 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE # Update the NAT external_ip column check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb set NAT . external_ip=172.168.0.120 -check_engine_stats northd recompute nocompute +check_engine_stats northd norecompute compute +check_engine_stats lr_nat norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_pb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11318,7 +11325,8 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE # Update the NAT logical_ip column check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb set NAT . logical_ip=10.0.0.10 -check_engine_stats northd recompute nocompute +check_engine_stats northd norecompute compute +check_engine_stats lr_nat norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_pb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11326,7 +11334,8 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE # Update the NAT type check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb set NAT . type=snat -check_engine_stats northd recompute nocompute +check_engine_stats northd norecompute compute +check_engine_stats lr_nat norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_pb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11334,7 +11343,8 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE # Create a dnat_and_snat NAT with external_mac and logical_port check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb lr-nat-add lr0 dnat_and_snat 172.168.0.110 10.0.0.4 sw0p1 30:54:00:00:00:03 -check_engine_stats northd recompute compute +check_engine_stats northd norecompute compute +check_engine_stats lr_nat norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_pb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11343,7 +11353,8 @@ nat2_uuid=$(ovn-nbctl --bare --columns _uuid find nat logical_ip=10.0.0.4) check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb set NAT $nat2_uuid external_mac='"30:54:00:00:00:04"' -check_engine_stats northd recompute nocompute +check_engine_stats northd norecompute compute +check_engine_stats lr_nat norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_pb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11358,28 +11369,32 @@ check ovn-nbctl lr-lb-add lr0 lb2 # is a lb vip. check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb lr-nat-add lr0 dnat_and_snat 172.168.0.140 10.0.0.20 -check_engine_stats northd recompute nocompute +check_engine_stats northd norecompute compute +check_engine_stats lr_nat norecompute compute check_engine_stats sync_to_sb_pb recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb lr-nat-add lr0 dnat_and_snat 172.168.0.150 10.0.0.41 -check_engine_stats northd recompute nocompute +check_engine_stats northd norecompute compute +check_engine_stats lr_nat norecompute compute check_engine_stats sync_to_sb_pb recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb lr-nat-del lr0 dnat_and_snat 172.168.0.150 -check_engine_stats northd recompute nocompute +check_engine_stats northd norecompute compute +check_engine_stats lr_nat norecompute compute check_engine_stats sync_to_sb_pb recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb lr-nat-del lr0 dnat_and_snat 172.168.0.140 -check_engine_stats northd recompute nocompute +check_engine_stats northd norecompute compute +check_engine_stats lr_nat norecompute compute check_engine_stats sync_to_sb_pb recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11387,7 +11402,8 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE # Delete the NAT check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb clear logical_router lr0 nat -check_engine_stats northd recompute compute +check_engine_stats northd norecompute compute +check_engine_stats lr_nat norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_pb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11396,6 +11412,7 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb lr-policy-add lr0 10 "ip4.src == 10.0.0.3" reroute 172.168.0.101,172.168.0.102 check_engine_stats northd recompute nocompute +check_engine_stats lr_nat recompute nocompute check_engine_stats sync_to_sb_pb recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11403,6 +11420,7 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb lr-policy-del lr0 10 "ip4.src == 10.0.0.3" check_engine_stats northd recompute nocompute +check_engine_stats lr_nat recompute nocompute check_engine_stats sync_to_sb_pb recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE From patchwork Thu Jan 11 15:29:32 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Numan Siddique X-Patchwork-Id: 1885674 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::133; helo=smtp2.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp2.osuosl.org (smtp2.osuosl.org [IPv6:2605:bc80:3010::133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4T9pWc5yDFz1yPm for ; Fri, 12 Jan 2024 02:30:04 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id E90D3438FA; Thu, 11 Jan 2024 15:30:02 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org E90D3438FA X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id kUeqQoOpQ4Yo; Thu, 11 Jan 2024 15:29:56 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp2.osuosl.org (Postfix) with ESMTPS id A9EED438D0; Thu, 11 Jan 2024 15:29:51 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org A9EED438D0 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 548BCC0DD8; Thu, 11 Jan 2024 15:29:51 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp4.osuosl.org (smtp4.osuosl.org [140.211.166.137]) by lists.linuxfoundation.org (Postfix) with ESMTP id 15994C0DD4 for ; Thu, 11 Jan 2024 15:29:50 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 68F3242C59 for ; Thu, 11 Jan 2024 15:29:49 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 68F3242C59 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id boxc3o26PDxl for ; Thu, 11 Jan 2024 15:29:46 +0000 (UTC) Received: from relay2-d.mail.gandi.net (relay2-d.mail.gandi.net [IPv6:2001:4b98:dc4:8::222]) by smtp4.osuosl.org (Postfix) with ESMTPS id BA623410A2 for ; Thu, 11 Jan 2024 15:29:45 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org BA623410A2 Received: by mail.gandi.net (Postfix) with ESMTPSA id CDF3140002; Thu, 11 Jan 2024 15:29:42 +0000 (UTC) From: numans@ovn.org To: dev@openvswitch.org Date: Thu, 11 Jan 2024 10:29:32 -0500 Message-ID: <20240111152932.2790024-1-numans@ovn.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240111152752.2789854-1-numans@ovn.org> References: <20240111152752.2789854-1-numans@ovn.org> MIME-Version: 1.0 X-GND-Sasl: numans@ovn.org Subject: [ovs-dev] [PATCH ovn v5 05/16] northd: Add a new engine 'lr_stateful' to manage lr's stateful data. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Numan Siddique This new engine now maintains the load balancer and NAT data of a logical router which was earlier part of northd engine node data. The main inputs to this engine are: - northd node - lr_nat node - lb_data node A record for each logical router is maintained in the 'lr_stateful_table' hmap table and this record - stores the lb related data - embeds the 'lr_nat' record. This engine node becomes an input to 'lflow' node. Signed-off-by: Numan Siddique Acked-by: Dumitru Ceara --- lib/stopwatch-names.h | 1 + northd/automake.mk | 2 + northd/en-lflow.c | 4 + northd/en-lr-nat.h | 3 + northd/en-lr-stateful.c | 641 +++++++++++++++++++++++++++++++++++ northd/en-lr-stateful.h | 105 ++++++ northd/en-sync-sb.c | 49 +-- northd/inc-proc-northd.c | 13 +- northd/northd.c | 711 ++++++++++++--------------------------- northd/northd.h | 139 +++++++- northd/ovn-northd.c | 1 + tests/ovn-northd.at | 62 ++++ 12 files changed, 1213 insertions(+), 518 deletions(-) create mode 100644 northd/en-lr-stateful.c create mode 100644 northd/en-lr-stateful.h diff --git a/lib/stopwatch-names.h b/lib/stopwatch-names.h index 782d64320a..e5e41fbfd8 100644 --- a/lib/stopwatch-names.h +++ b/lib/stopwatch-names.h @@ -30,5 +30,6 @@ #define PORT_GROUP_RUN_STOPWATCH_NAME "port_group_run" #define SYNC_METERS_RUN_STOPWATCH_NAME "sync_meters_run" #define LR_NAT_RUN_STOPWATCH_NAME "lr_nat_run" +#define LR_STATEFUL_RUN_STOPWATCH_NAME "lr_stateful" #endif diff --git a/northd/automake.mk b/northd/automake.mk index a477105470..b886356c9c 100644 --- a/northd/automake.mk +++ b/northd/automake.mk @@ -26,6 +26,8 @@ northd_ovn_northd_SOURCES = \ northd/en-lb-data.h \ northd/en-lr-nat.c \ northd/en-lr-nat.h \ + northd/en-lr-stateful.c \ + northd/en-lr-stateful.h \ northd/inc-proc-northd.c \ northd/inc-proc-northd.h \ northd/ipam.c \ diff --git a/northd/en-lflow.c b/northd/en-lflow.c index e4f875ef7c..bb9f78a6a1 100644 --- a/northd/en-lflow.c +++ b/northd/en-lflow.c @@ -20,6 +20,7 @@ #include "en-lflow.h" #include "en-lr-nat.h" +#include "en-lr-stateful.h" #include "en-northd.h" #include "en-meters.h" @@ -43,6 +44,8 @@ lflow_get_input_data(struct engine_node *node, engine_get_input_data("sync_meters", node); struct ed_type_lr_nat_data *lr_nat_data = engine_get_input_data("lr_nat", node); + struct ed_type_lr_stateful *lr_stateful_data = + engine_get_input_data("lr_stateful", node); lflow_input->nbrec_bfd_table = EN_OVSDB_GET(engine_get_input("NB_bfd", node)); @@ -66,6 +69,7 @@ lflow_get_input_data(struct engine_node *node, lflow_input->lr_ports = &northd_data->lr_ports; lflow_input->ls_port_groups = &pg_data->ls_port_groups; lflow_input->lr_nats = &lr_nat_data->lr_nats; + lflow_input->lr_stateful_table = &lr_stateful_data->table; lflow_input->meter_groups = &sync_meters_data->meter_groups; lflow_input->lb_datapaths_map = &northd_data->lb_datapaths_map; lflow_input->svc_monitor_map = &northd_data->svc_monitor_map; diff --git a/northd/en-lr-nat.h b/northd/en-lr-nat.h index 3ec4c7b506..4d8e142ae6 100644 --- a/northd/en-lr-nat.h +++ b/northd/en-lr-nat.h @@ -86,6 +86,9 @@ struct lr_nat_table { const struct lr_nat_record * lr_nat_table_find_by_index( const struct lr_nat_table *, size_t od_index); +#define LR_NAT_TABLE_FOR_EACH(LR_NAT_REC, TABLE) \ + HMAP_FOR_EACH (LR_NAT_REC, key_node, &(TABLE)->entries) + struct ed_type_lr_nat_data { struct lr_nat_table lr_nats; diff --git a/northd/en-lr-stateful.c b/northd/en-lr-stateful.c new file mode 100644 index 0000000000..dedfdf6747 --- /dev/null +++ b/northd/en-lr-stateful.c @@ -0,0 +1,641 @@ +/* + * Copyright (c) 2023, Red Hat, Inc. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at: + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#include + +#include +#include +#include + +/* OVS includes */ +#include "include/openvswitch/hmap.h" +#include "lib/bitmap.h" +#include "lib/socket-util.h" +#include "lib/uuidset.h" +#include "openvswitch/util.h" +#include "openvswitch/vlog.h" +#include "stopwatch.h" + +/* OVN includes */ +#include "en-lb-data.h" +#include "en-lr-nat.h" +#include "en-lr-stateful.h" +#include "lib/inc-proc-eng.h" +#include "lib/lb.h" +#include "lib/ovn-nb-idl.h" +#include "lib/ovn-sb-idl.h" +#include "lib/ovn-util.h" +#include "lib/stopwatch-names.h" +#include "northd.h" + +VLOG_DEFINE_THIS_MODULE(en_lr_stateful); + +/* Static function declarations. */ +static void lr_stateful_table_init(struct lr_stateful_table *); +static void lr_stateful_table_clear(struct lr_stateful_table *); +static void lr_stateful_table_destroy(struct lr_stateful_table *); +static struct lr_stateful_record *lr_stateful_table_find_( + const struct lr_stateful_table *, const struct nbrec_logical_router *); +static struct lr_stateful_record *lr_stateful_table_find_by_index_( + const struct lr_stateful_table *table, size_t od_index); + +static void lr_stateful_table_build(struct lr_stateful_table *, + const struct lr_nat_table *, + const struct ovn_datapaths *lr_datapaths, + const struct hmap *lb_datapaths_map, + const struct hmap *lbgrp_datapaths_map); + +static struct lr_stateful_input lr_stateful_get_input_data( + struct engine_node *); + +static struct lr_stateful_record *lr_stateful_record_create( + struct lr_stateful_table *, const struct lr_nat_record *, + const struct hmap *lb_datapaths_map, + const struct hmap *lbgrp_datapaths_map); +static void lr_stateful_record_destroy(struct lr_stateful_record *); +static void lr_stateful_record_init( + struct lr_stateful_record *, + const struct hmap *lb_datapaths_map, + const struct hmap *lbgrp_datapaths_map); + +static void build_lrouter_lb_reachable_ips(struct lr_stateful_record *, + const struct ovn_northd_lb *); +static void add_neigh_ips_to_lrouter(struct lr_stateful_record *, + enum lb_neighbor_responder_mode, + const struct sset *lb_ips_v4, + const struct sset *lb_ips_v6); +static void remove_lrouter_lb_reachable_ips(struct lr_stateful_record *, + enum lb_neighbor_responder_mode, + const struct sset *lb_ips_v4, + const struct sset *lb_ips_v6); +static void lr_stateful_build_vip_nats(struct lr_stateful_record *); + +/* 'lr_stateful' engine node manages the NB logical router LB data. + */ +void * +en_lr_stateful_init(struct engine_node *node OVS_UNUSED, + struct engine_arg *arg OVS_UNUSED) +{ + struct ed_type_lr_stateful *data = xzalloc(sizeof *data); + lr_stateful_table_init(&data->table); + hmapx_init(&data->trk_data.crupdated); + return data; +} + +void +en_lr_stateful_cleanup(void *data_) +{ + struct ed_type_lr_stateful *data = data_; + lr_stateful_table_destroy(&data->table); + hmapx_destroy(&data->trk_data.crupdated); +} + +void +en_lr_stateful_clear_tracked_data(void *data_) +{ + struct ed_type_lr_stateful *data = data_; + + hmapx_clear(&data->trk_data.crupdated); +} + +void +en_lr_stateful_run(struct engine_node *node, void *data_) +{ + struct lr_stateful_input input_data = lr_stateful_get_input_data(node); + struct ed_type_lr_stateful *data = data_; + + stopwatch_start(LR_STATEFUL_RUN_STOPWATCH_NAME, time_msec()); + + lr_stateful_table_clear(&data->table); + lr_stateful_table_build(&data->table, input_data.lr_nats, + input_data.lr_datapaths, + input_data.lb_datapaths_map, + input_data.lbgrp_datapaths_map); + + stopwatch_stop(LR_STATEFUL_RUN_STOPWATCH_NAME, time_msec()); + engine_set_node_state(node, EN_UPDATED); +} + +bool +lr_stateful_northd_handler(struct engine_node *node, void *data OVS_UNUSED) +{ + struct northd_data *northd_data = engine_get_input_data("northd", node); + if (!northd_has_tracked_data(&northd_data->trk_data)) { + return false; + } + + /* lr_stateful node needs inputs for any changes to NAT and load balancers. + * Changes to NAT is provided by the lr_nat tracked data and changes + * to lbs and lb grps is provided by lb_data's tracked data. + * So we don't need to do anything here for northd changes. + * But we do need to access the datapaths and lb_datapaths from the + * northd engine node and hence its an input. + * */ + return true; +} + +bool +lr_stateful_lb_data_handler(struct engine_node *node, void *data_) +{ + struct ed_type_lb_data *lb_data = engine_get_input_data("lb_data", node); + if (!lb_data->tracked) { + return false; + } + + struct ed_type_lr_stateful *data = data_; + struct lr_stateful_input input_data = + lr_stateful_get_input_data(node); + + const struct tracked_lb_data *trk_lb_data = &lb_data->tracked_lb_data; + const struct crupdated_od_lb_data *codlb; + + LIST_FOR_EACH (codlb, list_node, &trk_lb_data->crupdated_lr_lbs) { + const struct ovn_datapath *od = ovn_datapath_find( + &input_data.lr_datapaths->datapaths, &codlb->od_uuid); + ovs_assert(od); + + struct lr_stateful_record *lr_stateful_rec =lr_stateful_table_find_( + &data->table, od->nbr); + if (!lr_stateful_rec) { + const struct lr_nat_record *lrnat_rec = lr_nat_table_find_by_index( + input_data.lr_nats, od->index); + ovs_assert(lrnat_rec); + + lr_stateful_rec = lr_stateful_record_create(&data->table, + lrnat_rec, + input_data.lb_datapaths_map, + input_data.lbgrp_datapaths_map); + + /* Add the lr_stateful_rec rec to the tracking data. */ + hmapx_add(&data->trk_data.crupdated, lr_stateful_rec); + continue; + } + + /* Update. */ + struct uuidset_node *uuidnode; + UUIDSET_FOR_EACH (uuidnode, &codlb->assoc_lbs) { + const struct ovn_lb_datapaths *lb_dps = ovn_lb_datapaths_find( + input_data.lb_datapaths_map, &uuidnode->uuid); + ovs_assert(lb_dps); + + /* Add the lb_ips of lb_dps to the od. */ + build_lrouter_lb_ips(lr_stateful_rec->lb_ips, lb_dps->lb); + build_lrouter_lb_reachable_ips(lr_stateful_rec, lb_dps->lb); + } + + UUIDSET_FOR_EACH (uuidnode, &codlb->assoc_lbgrps) { + const struct ovn_lb_group_datapaths *lbgrp_dps = + ovn_lb_group_datapaths_find(input_data.lbgrp_datapaths_map, + &uuidnode->uuid); + ovs_assert(lbgrp_dps); + + for (size_t j = 0; j < lbgrp_dps->lb_group->n_lbs; j++) { + const struct uuid *lb_uuid + = &lbgrp_dps->lb_group->lbs[j]->nlb->header_.uuid; + const struct ovn_lb_datapaths *lb_dps = ovn_lb_datapaths_find( + input_data.lb_datapaths_map, lb_uuid); + ovs_assert(lb_dps); + + /* Add the lb_ips of lb_dps to the od. */ + build_lrouter_lb_ips(lr_stateful_rec->lb_ips, lb_dps->lb); + build_lrouter_lb_reachable_ips(lr_stateful_rec, lb_dps->lb); + } + } + + /* Add the lr_stateful_rec rec to the tracking data. */ + hmapx_add(&data->trk_data.crupdated, lr_stateful_rec); + } + + const struct crupdated_lb *clb; + HMAP_FOR_EACH (clb, hmap_node, &trk_lb_data->crupdated_lbs) { + const struct uuid *lb_uuid = &clb->lb->nlb->header_.uuid; + const struct ovn_northd_lb *lb = clb->lb; + + const struct ovn_lb_datapaths *lb_dps = ovn_lb_datapaths_find( + input_data.lb_datapaths_map, lb_uuid); + ovs_assert(lb_dps); + + size_t index; + BITMAP_FOR_EACH_1 (index, ods_size(input_data.lr_datapaths), + lb_dps->nb_lr_map) { + const struct ovn_datapath *od = + input_data.lr_datapaths->array[index]; + + struct lr_stateful_record *lr_stateful_rec = + lr_stateful_table_find_(&data->table, od->nbr); + ovs_assert(lr_stateful_rec); + + /* Update the od->lb_ips with the deleted and inserted + * vips (if any). */ + remove_ips_from_lb_ip_set(lr_stateful_rec->lb_ips, lb->routable, + &clb->deleted_vips_v4, + &clb->deleted_vips_v6); + add_ips_to_lb_ip_set(lr_stateful_rec->lb_ips, lb->routable, + &clb->inserted_vips_v4, + &clb->inserted_vips_v6); + + remove_lrouter_lb_reachable_ips(lr_stateful_rec, lb->neigh_mode, + &clb->deleted_vips_v4, + &clb->deleted_vips_v6); + add_neigh_ips_to_lrouter(lr_stateful_rec, lb->neigh_mode, + &clb->inserted_vips_v4, + &clb->inserted_vips_v6); + + /* Add the lr_stateful_rec rec to the tracking data. */ + hmapx_add(&data->trk_data.crupdated, lr_stateful_rec); + } + } + + const struct crupdated_lbgrp *crupdated_lbgrp; + HMAP_FOR_EACH (crupdated_lbgrp, hmap_node, + &trk_lb_data->crupdated_lbgrps) { + const struct uuid *lb_uuid = &crupdated_lbgrp->lbgrp->uuid; + const struct ovn_lb_group_datapaths *lbgrp_dps = + ovn_lb_group_datapaths_find(input_data.lbgrp_datapaths_map, + lb_uuid); + ovs_assert(lbgrp_dps); + + struct hmapx_node *hnode; + HMAPX_FOR_EACH (hnode, &crupdated_lbgrp->assoc_lbs) { + const struct ovn_northd_lb *lb = hnode->data; + lb_uuid = &lb->nlb->header_.uuid; + const struct ovn_lb_datapaths *lb_dps = ovn_lb_datapaths_find( + input_data.lb_datapaths_map, lb_uuid); + ovs_assert(lb_dps); + for (size_t i = 0; i < lbgrp_dps->n_lr; i++) { + const struct ovn_datapath *od = lbgrp_dps->lr[i]; + struct lr_stateful_record *lr_stateful_rec = + lr_stateful_table_find_(&data->table, od->nbr); + ovs_assert(lr_stateful_rec); + /* Add the lb_ips of lb_dps to the lr lb data. */ + build_lrouter_lb_ips(lr_stateful_rec->lb_ips, lb_dps->lb); + build_lrouter_lb_reachable_ips(lr_stateful_rec, lb_dps->lb); + + /* Add the lr_stateful_rec rec to the tracking data. */ + hmapx_add(&data->trk_data.crupdated, lr_stateful_rec); + } + } + } + + if (lr_stateful_has_tracked_data(&data->trk_data)) { + struct hmapx_node *hmapx_node; + /* For all the modified lr_stateful records (re)build the + * vip nats. */ + HMAPX_FOR_EACH (hmapx_node, &data->trk_data.crupdated) { + lr_stateful_build_vip_nats(hmapx_node->data); + } + + engine_set_node_state(node, EN_UPDATED); + } + + return true; +} + +bool +lr_stateful_lr_nat_handler(struct engine_node *node, void *data_) +{ + struct ed_type_lr_nat_data *lr_nat_data = + engine_get_input_data("lr_nat", node); + + if (!lr_nat_has_tracked_data(&lr_nat_data->trk_data)) { + return false; + } + + struct ed_type_lr_stateful *data = data_; + struct lr_stateful_input input_data = + lr_stateful_get_input_data(node); + struct hmapx_node *hmapx_node; + + HMAPX_FOR_EACH (hmapx_node, &lr_nat_data->trk_data.crupdated) { + const struct lr_nat_record *lrnat_rec = hmapx_node->data; + struct lr_stateful_record *lr_stateful_rec = + lr_stateful_table_find_(&data->table, lrnat_rec->od->nbr); + if (!lr_stateful_rec) { + lr_stateful_rec = lr_stateful_record_create(&data->table, + lrnat_rec, + input_data.lb_datapaths_map, + input_data.lbgrp_datapaths_map); + } else { + lr_stateful_build_vip_nats(lr_stateful_rec); + } + + /* Add the lr_stateful_rec rec to the tracking data. */ + hmapx_add(&data->trk_data.crupdated, lr_stateful_rec); + } + + if (lr_stateful_has_tracked_data(&data->trk_data)) { + engine_set_node_state(node, EN_UPDATED); + } + + return true; +} + +const struct lr_stateful_record * +lr_stateful_table_find_by_index(const struct lr_stateful_table *table, + size_t od_index) +{ + return lr_stateful_table_find_by_index_(table, od_index); +} + +/* static functions. */ +static void +lr_stateful_table_init(struct lr_stateful_table *table) +{ + *table = (struct lr_stateful_table) { + .entries = HMAP_INITIALIZER(&table->entries), + }; +} + +static void +lr_stateful_table_destroy(struct lr_stateful_table *table) +{ + lr_stateful_table_clear(table); + hmap_destroy(&table->entries); +} + +static void +lr_stateful_table_clear(struct lr_stateful_table *table) +{ + struct lr_stateful_record *lr_stateful_rec; + HMAP_FOR_EACH_POP (lr_stateful_rec, key_node, &table->entries) { + lr_stateful_record_destroy(lr_stateful_rec); + } + + free(table->array); + table->array = NULL; +} + +static void +lr_stateful_table_build(struct lr_stateful_table *table, + const struct lr_nat_table *lr_nats, + const struct ovn_datapaths *lr_datapaths, + const struct hmap *lb_datapaths_map, + const struct hmap *lbgrp_datapaths_map) +{ + table->array = xrealloc(table->array, + ods_size(lr_datapaths) * sizeof *table->array); + const struct lr_nat_record *lrnat_rec; + LR_NAT_TABLE_FOR_EACH (lrnat_rec, lr_nats) { + lr_stateful_record_create(table, lrnat_rec, lb_datapaths_map, + lbgrp_datapaths_map); + } +} + +static struct lr_stateful_record * +lr_stateful_table_find_(const struct lr_stateful_table *table, + const struct nbrec_logical_router *nbr) +{ + struct lr_stateful_record *lr_stateful_rec; + + HMAP_FOR_EACH_WITH_HASH (lr_stateful_rec, key_node, + uuid_hash(&nbr->header_.uuid), &table->entries) { + if (nbr == lr_stateful_rec->od->nbr) { + return lr_stateful_rec; + } + } + return NULL; +} + +static struct lr_stateful_record * +lr_stateful_table_find_by_index_(const struct lr_stateful_table *table, + size_t od_index) +{ + ovs_assert(od_index <= hmap_count(&table->entries)); + return table->array[od_index]; +} + +static struct lr_stateful_record * +lr_stateful_record_create(struct lr_stateful_table *table, + const struct lr_nat_record *lrnat_rec, + const struct hmap *lb_datapaths_map, + const struct hmap *lbgrp_datapaths_map) +{ + struct lr_stateful_record *lr_stateful_rec = + xzalloc(sizeof *lr_stateful_rec); + lr_stateful_rec->lrnat_rec = lrnat_rec; + lr_stateful_rec->od = lrnat_rec->od; + lr_stateful_record_init(lr_stateful_rec, lb_datapaths_map, + lbgrp_datapaths_map); + + hmap_insert(&table->entries, &lr_stateful_rec->key_node, + uuid_hash(&lr_stateful_rec->od->nbr->header_.uuid)); + + table->array[lr_stateful_rec->od->index] = lr_stateful_rec; + return lr_stateful_rec; +} + +static void +lr_stateful_record_destroy(struct lr_stateful_record *lr_stateful_rec) +{ + ovn_lb_ip_set_destroy(lr_stateful_rec->lb_ips); + lr_stateful_rec->lb_ips = NULL; + sset_destroy(&lr_stateful_rec->vip_nats); + free(lr_stateful_rec); +} + +static void +lr_stateful_record_init(struct lr_stateful_record *lr_stateful_rec, + const struct hmap *lb_datapaths_map, + const struct hmap *lbgrp_datapaths_map) +{ + /* Checking load balancer groups first, starting from the largest one, + * to more efficiently copy IP sets. */ + size_t largest_group = 0; + + const struct nbrec_logical_router *nbr = lr_stateful_rec->od->nbr; + for (size_t i = 1; i < nbr->n_load_balancer_group; i++) { + if (nbr->load_balancer_group[i]->n_load_balancer > + nbr->load_balancer_group[largest_group]->n_load_balancer) { + largest_group = i; + } + } + + for (size_t i = 0; i < nbr->n_load_balancer_group; i++) { + size_t idx = (i + largest_group) % nbr->n_load_balancer_group; + + const struct nbrec_load_balancer_group *nbrec_lb_group = + nbr->load_balancer_group[idx]; + const struct uuid *lbgrp_uuid = &nbrec_lb_group->header_.uuid; + + const struct ovn_lb_group_datapaths *lbgrp_dps = + ovn_lb_group_datapaths_find(lbgrp_datapaths_map, + lbgrp_uuid); + ovs_assert(lbgrp_dps); + + if (!lr_stateful_rec->lb_ips) { + lr_stateful_rec->lb_ips = + ovn_lb_ip_set_clone(lbgrp_dps->lb_group->lb_ips); + } else { + for (size_t j = 0; j < lbgrp_dps->lb_group->n_lbs; j++) { + build_lrouter_lb_ips(lr_stateful_rec->lb_ips, + lbgrp_dps->lb_group->lbs[j]); + } + } + + for (size_t j = 0; j < lbgrp_dps->lb_group->n_lbs; j++) { + build_lrouter_lb_reachable_ips(lr_stateful_rec, + lbgrp_dps->lb_group->lbs[j]); + } + } + + if (!lr_stateful_rec->lb_ips) { + lr_stateful_rec->lb_ips = ovn_lb_ip_set_create(); + } + + for (size_t i = 0; i < nbr->n_load_balancer; i++) { + const struct uuid *lb_uuid = + &nbr->load_balancer[i]->header_.uuid; + const struct ovn_lb_datapaths *lb_dps = + ovn_lb_datapaths_find(lb_datapaths_map, lb_uuid); + ovs_assert(lb_dps); + build_lrouter_lb_ips(lr_stateful_rec->lb_ips, lb_dps->lb); + build_lrouter_lb_reachable_ips(lr_stateful_rec, lb_dps->lb); + } + + sset_init(&lr_stateful_rec->vip_nats); + + if (nbr->n_nat) { + lr_stateful_build_vip_nats(lr_stateful_rec); + } +} + +static struct lr_stateful_input +lr_stateful_get_input_data(struct engine_node *node) +{ + struct northd_data *northd_data = engine_get_input_data("northd", node); + struct ed_type_lr_nat_data *lr_nat_data = + engine_get_input_data("lr_nat", node); + + return (struct lr_stateful_input) { + .lr_datapaths = &northd_data->lr_datapaths, + .lb_datapaths_map = &northd_data->lb_datapaths_map, + .lbgrp_datapaths_map = &northd_data->lb_group_datapaths_map, + .lr_nats = &lr_nat_data->lr_nats, + }; +} + +static void +build_lrouter_lb_reachable_ips(struct lr_stateful_record *lr_stateful_rec, + const struct ovn_northd_lb *lb) +{ + add_neigh_ips_to_lrouter(lr_stateful_rec, lb->neigh_mode, &lb->ips_v4, + &lb->ips_v6); +} + +static void +add_neigh_ips_to_lrouter(struct lr_stateful_record *lr_stateful_rec, + enum lb_neighbor_responder_mode neigh_mode, + const struct sset *lb_ips_v4, + const struct sset *lb_ips_v6) +{ + /* If configured to not reply to any neighbor requests for all VIPs + * return early. + */ + if (neigh_mode == LB_NEIGH_RESPOND_NONE) { + return; + } + + const char *ip_address; + + /* If configured to reply to neighbor requests for all VIPs force them + * all to be considered "reachable". + */ + if (neigh_mode == LB_NEIGH_RESPOND_ALL) { + SSET_FOR_EACH (ip_address, lb_ips_v4) { + sset_add(&lr_stateful_rec->lb_ips->ips_v4_reachable, ip_address); + } + SSET_FOR_EACH (ip_address, lb_ips_v6) { + sset_add(&lr_stateful_rec->lb_ips->ips_v6_reachable, ip_address); + } + + return; + } + + /* Otherwise, a VIP is reachable if there's at least one router + * subnet that includes it. + */ + ovs_assert(neigh_mode == LB_NEIGH_RESPOND_REACHABLE); + + SSET_FOR_EACH (ip_address, lb_ips_v4) { + struct ovn_port *op; + ovs_be32 vip_ip4; + if (ip_parse(ip_address, &vip_ip4)) { + HMAP_FOR_EACH (op, dp_node, &lr_stateful_rec->od->ports) { + if (lrouter_port_ipv4_reachable(op, vip_ip4)) { + sset_add(&lr_stateful_rec->lb_ips->ips_v4_reachable, + ip_address); + break; + } + } + } + } + + SSET_FOR_EACH (ip_address, lb_ips_v6) { + struct ovn_port *op; + struct in6_addr vip; + if (ipv6_parse(ip_address, &vip)) { + HMAP_FOR_EACH (op, dp_node, &lr_stateful_rec->od->ports) { + if (lrouter_port_ipv6_reachable(op, &vip)) { + sset_add(&lr_stateful_rec->lb_ips->ips_v6_reachable, + ip_address); + break; + } + } + } + } +} + +static void +remove_lrouter_lb_reachable_ips(struct lr_stateful_record *lr_stateful_rec, + enum lb_neighbor_responder_mode neigh_mode, + const struct sset *lb_ips_v4, + const struct sset *lb_ips_v6) +{ + if (neigh_mode == LB_NEIGH_RESPOND_NONE) { + return; + } + + const char *ip_address; + SSET_FOR_EACH (ip_address, lb_ips_v4) { + sset_find_and_delete(&lr_stateful_rec->lb_ips->ips_v4_reachable, + ip_address); + } + SSET_FOR_EACH (ip_address, lb_ips_v6) { + sset_find_and_delete(&lr_stateful_rec->lb_ips->ips_v6_reachable, + ip_address); + } +} + +static void +lr_stateful_build_vip_nats(struct lr_stateful_record *lr_stateful_rec) +{ + sset_clear(&lr_stateful_rec->vip_nats); + const char *external_ip; + SSET_FOR_EACH (external_ip, &lr_stateful_rec->lrnat_rec->external_ips) { + bool is_vip_nat = false; + if (addr_is_ipv6(external_ip)) { + is_vip_nat = sset_contains(&lr_stateful_rec->lb_ips->ips_v6, + external_ip); + } else { + is_vip_nat = sset_contains(&lr_stateful_rec->lb_ips->ips_v4, + external_ip); + } + + if (is_vip_nat) { + sset_add(&lr_stateful_rec->vip_nats, external_ip); + } + } +} diff --git a/northd/en-lr-stateful.h b/northd/en-lr-stateful.h new file mode 100644 index 0000000000..0bc1f4ee75 --- /dev/null +++ b/northd/en-lr-stateful.h @@ -0,0 +1,105 @@ +/* + * Copyright (c) 2023, Red Hat, Inc. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at: + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +#ifndef EN_LR_STATEFUL_H +#define EN_LR_STATEFUL_H 1 + +#include + +/* OVS includes. */ +#include "lib/hmapx.h" +#include "openvswitch/hmap.h" +#include "sset.h" + +/* OVN includes. */ +#include "lib/inc-proc-eng.h" +#include "lib/lb.h" +#include "lib/ovn-nb-idl.h" +#include "lib/ovn-sb-idl.h" +#include "lib/ovn-util.h" + +struct ovn_datapath; +struct lr_nat_record; + +/* lr_stateful_table: This represents a table of logical routers with + * stateful related data. + * stateful related data has two main components + * - NAT and + * - Load balancers. + * + * lr_stateful_record: It is a record in the lr_stateful_table for each + * logical router. + */ + +struct lr_stateful_record { + struct hmap_node key_node; /* Index on 'nbr->header_.uuid'. */ + + const struct ovn_datapath *od; + const struct lr_nat_record *lrnat_rec; + + /* Load Balancer vIPs relevant for this datapath. */ + struct ovn_lb_ip_set *lb_ips; + + /* sset of vips which are also part of lr nats. */ + struct sset vip_nats; +}; + +struct lr_stateful_table { + struct hmap entries; + + /* The array index of each element in 'entries'. */ + struct lr_stateful_record **array; +}; + +#define LR_STATEFUL_TABLE_FOR_EACH(LR_LB_NAT_REC, TABLE) \ + HMAP_FOR_EACH (LR_LB_NAT_REC, key_node, &(TABLE)->entries) + +struct lr_stateful_tracked_data { + /* Created or updated logical router with LB and/or NAT data. */ + struct hmapx crupdated; /* Stores 'struct lr_stateful_record'. */ +}; + +struct ed_type_lr_stateful { + struct lr_stateful_table table; + + /* Node's tracked data. */ + struct lr_stateful_tracked_data trk_data; +}; + +struct lr_stateful_input { + const struct ovn_datapaths *lr_datapaths; + const struct hmap *lb_datapaths_map; + const struct hmap *lbgrp_datapaths_map; + const struct lr_nat_table *lr_nats; +}; + +void *en_lr_stateful_init(struct engine_node *, struct engine_arg *); +void en_lr_stateful_cleanup(void *data); +void en_lr_stateful_clear_tracked_data(void *data); +void en_lr_stateful_run(struct engine_node *, void *data); + +bool lr_stateful_northd_handler(struct engine_node *, void *data); +bool lr_stateful_lr_nat_handler(struct engine_node *, void *data); +bool lr_stateful_lb_data_handler(struct engine_node *, void *data); + +const struct lr_stateful_record *lr_stateful_table_find_by_index( + const struct lr_stateful_table *, size_t od_index); + +static inline bool +lr_stateful_has_tracked_data(struct lr_stateful_tracked_data *trk_data) { + return !hmapx_is_empty(&trk_data->crupdated); +} + +#endif /* EN_lr_stateful_H */ diff --git a/northd/en-sync-sb.c b/northd/en-sync-sb.c index 11e12428f7..d39cbbf2e6 100644 --- a/northd/en-sync-sb.c +++ b/northd/en-sync-sb.c @@ -22,6 +22,7 @@ #include "openvswitch/util.h" #include "en-lr-nat.h" +#include "en-lr-stateful.h" #include "en-sync-sb.h" #include "lib/inc-proc-eng.h" #include "lib/lb.h" @@ -41,7 +42,7 @@ static void sync_addr_sets(struct ovsdb_idl_txn *ovnsb_txn, const struct nbrec_address_set_table *, const struct nbrec_port_group_table *, const struct sbrec_address_set_table *, - const struct ovn_datapaths *lr_datapaths); + const struct lr_stateful_table *); static const struct sbrec_address_set *sb_address_set_lookup_by_name( struct ovsdb_idl_index *, const char *name); static void update_sb_addr_set(struct sorted_array *, @@ -87,11 +88,11 @@ en_sync_to_sb_addr_set_run(struct engine_node *node, void *data OVS_UNUSED) EN_OVSDB_GET(engine_get_input("SB_address_set", node)); const struct engine_context *eng_ctx = engine_get_context(); - struct northd_data *northd_data = engine_get_input_data("northd", node); - + const struct ed_type_lr_stateful *lr_stateful_data = + engine_get_input_data("lr_stateful", node); sync_addr_sets(eng_ctx->ovnsb_idl_txn, nb_address_set_table, nb_port_group_table, sb_address_set_table, - &northd_data->lr_datapaths); + &lr_stateful_data->table); engine_set_node_state(node, EN_UPDATED); } @@ -288,10 +289,12 @@ en_sync_to_sb_pb_run(struct engine_node *node, void *data OVS_UNUSED) { const struct engine_context *eng_ctx = engine_get_context(); struct northd_data *northd_data = engine_get_input_data("northd", node); - struct ed_type_lr_nat_data *lr_nat_data = - engine_get_input_data("lr_nat", node); + struct ed_type_lr_stateful *lr_stateful_data = + engine_get_input_data("lr_stateful", node); + sync_pbs(eng_ctx->ovnsb_idl_txn, &northd_data->ls_ports, - &northd_data->lr_ports, &lr_nat_data->lr_nats); + &northd_data->lr_ports, + &lr_stateful_data->table); engine_set_node_state(node, EN_UPDATED); } @@ -316,7 +319,11 @@ sync_to_sb_pb_northd_handler(struct engine_node *node, void *data OVS_UNUSED) return false; } - if (!sync_pbs_for_northd_changed_ovn_ports(&nd->trk_data.trk_lsps)) { + struct ed_type_lr_stateful *lr_stateful_data = + engine_get_input_data("lr_stateful", node); + + if (!sync_pbs_for_northd_changed_ovn_ports(&nd->trk_data.trk_lsps, + &lr_stateful_data->table)) { return false; } @@ -362,7 +369,7 @@ sync_addr_sets(struct ovsdb_idl_txn *ovnsb_txn, const struct nbrec_address_set_table *nb_address_set_table, const struct nbrec_port_group_table *nb_port_group_table, const struct sbrec_address_set_table *sb_address_set_table, - const struct ovn_datapaths *lr_datapaths) + const struct lr_stateful_table *lr_statefuls) { struct shash sb_address_sets = SHASH_INITIALIZER(&sb_address_sets); @@ -406,16 +413,14 @@ sync_addr_sets(struct ovsdb_idl_txn *ovnsb_txn, } /* Sync router load balancer VIP generated address sets. */ - struct ovn_datapath *od; - HMAP_FOR_EACH (od, key_node, &lr_datapaths->datapaths) { - ovs_assert(od->nbr); - - if (sset_count(&od->lb_ips->ips_v4_reachable)) { - char *ipv4_addrs_name = lr_lb_address_set_name(od->tunnel_key, - AF_INET); + const struct lr_stateful_record *lr_sful_rec; + LR_STATEFUL_TABLE_FOR_EACH (lr_sful_rec, lr_statefuls) { + if (sset_count(&lr_sful_rec->lb_ips->ips_v4_reachable)) { + char *ipv4_addrs_name = + lr_lb_address_set_name(lr_sful_rec->od->tunnel_key, AF_INET); struct sorted_array ipv4_addrs_sorted = - sorted_array_from_sset(&od->lb_ips->ips_v4_reachable); + sorted_array_from_sset(&lr_sful_rec->lb_ips->ips_v4_reachable); sync_addr_set(ovnsb_txn, ipv4_addrs_name, &ipv4_addrs_sorted, &sb_address_sets); @@ -423,11 +428,11 @@ sync_addr_sets(struct ovsdb_idl_txn *ovnsb_txn, free(ipv4_addrs_name); } - if (sset_count(&od->lb_ips->ips_v6_reachable)) { - char *ipv6_addrs_name = lr_lb_address_set_name(od->tunnel_key, - AF_INET6); - struct sorted_array ipv6_addrs_sorted = - sorted_array_from_sset(&od->lb_ips->ips_v6_reachable); + if (sset_count(&lr_sful_rec->lb_ips->ips_v6_reachable)) { + char *ipv6_addrs_name = + lr_lb_address_set_name(lr_sful_rec->od->tunnel_key, AF_INET6); + struct sorted_array ipv6_addrs_sorted = sorted_array_from_sset( + &lr_sful_rec->lb_ips->ips_v6_reachable); sync_addr_set(ovnsb_txn, ipv6_addrs_name, &ipv6_addrs_sorted, &sb_address_sets); diff --git a/northd/inc-proc-northd.c b/northd/inc-proc-northd.c index 1f211b278e..97bcce9655 100644 --- a/northd/inc-proc-northd.c +++ b/northd/inc-proc-northd.c @@ -31,6 +31,7 @@ #include "openvswitch/vlog.h" #include "inc-proc-northd.h" #include "en-lb-data.h" +#include "en-lr-stateful.h" #include "en-lr-nat.h" #include "en-northd.h" #include "en-lflow.h" @@ -148,6 +149,7 @@ static ENGINE_NODE(sync_to_sb_lb, "sync_to_sb_lb"); static ENGINE_NODE(sync_to_sb_pb, "sync_to_sb_pb"); static ENGINE_NODE_WITH_CLEAR_TRACK_DATA(lb_data, "lb_data"); static ENGINE_NODE_WITH_CLEAR_TRACK_DATA(lr_nat, "lr_nat"); +static ENGINE_NODE_WITH_CLEAR_TRACK_DATA(lr_stateful, "lr_stateful"); void inc_proc_northd_init(struct ovsdb_idl_loop *nb, struct ovsdb_idl_loop *sb) @@ -193,6 +195,11 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, engine_add_input(&en_lr_nat, &en_northd, lr_nat_northd_handler); + engine_add_input(&en_lr_stateful, &en_northd, lr_stateful_northd_handler); + engine_add_input(&en_lr_stateful, &en_lr_nat, lr_stateful_lr_nat_handler); + engine_add_input(&en_lr_stateful, &en_lb_data, + lr_stateful_lb_data_handler); + engine_add_input(&en_mac_binding_aging, &en_nb_nb_global, NULL); engine_add_input(&en_mac_binding_aging, &en_sb_mac_binding, NULL); engine_add_input(&en_mac_binding_aging, &en_northd, NULL); @@ -214,15 +221,17 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, engine_add_input(&en_lflow, &en_sb_logical_flow, NULL); engine_add_input(&en_lflow, &en_sb_multicast_group, NULL); engine_add_input(&en_lflow, &en_sb_igmp_group, NULL); + engine_add_input(&en_lflow, &en_lr_nat, NULL); + engine_add_input(&en_lflow, &en_lr_stateful, NULL); engine_add_input(&en_lflow, &en_northd, lflow_northd_handler); engine_add_input(&en_lflow, &en_port_group, lflow_port_group_handler); - engine_add_input(&en_lflow, &en_lr_nat, NULL); engine_add_input(&en_sync_to_sb_addr_set, &en_nb_address_set, sync_to_sb_addr_set_nb_address_set_handler); engine_add_input(&en_sync_to_sb_addr_set, &en_nb_port_group, sync_to_sb_addr_set_nb_port_group_handler); engine_add_input(&en_sync_to_sb_addr_set, &en_northd, NULL); + engine_add_input(&en_sync_to_sb_addr_set, &en_lr_stateful, NULL); engine_add_input(&en_sync_to_sb_addr_set, &en_sb_address_set, NULL); engine_add_input(&en_port_group, &en_nb_port_group, @@ -240,7 +249,7 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, engine_add_input(&en_sync_to_sb_pb, &en_northd, sync_to_sb_pb_northd_handler); - engine_add_input(&en_sync_to_sb_pb, &en_lr_nat, NULL); + engine_add_input(&en_sync_to_sb_pb, &en_lr_stateful, NULL); /* en_sync_to_sb engine node syncs the SB database tables from * the NB database tables. diff --git a/northd/northd.c b/northd/northd.c index e5e86326e3..fc2fc835b2 100644 --- a/northd/northd.c +++ b/northd/northd.c @@ -45,6 +45,7 @@ #include "northd.h" #include "en-lb-data.h" #include "en-lr-nat.h" +#include "en-lr-stateful.h" #include "lib/ovn-parallel-hmap.h" #include "ovn/actions.h" #include "ovn/features.h" @@ -618,13 +619,6 @@ init_lb_for_datapath(struct ovn_datapath *od) } } -static void -destroy_lb_for_datapath(struct ovn_datapath *od) -{ - ovn_lb_ip_set_destroy(od->lb_ips); - od->lb_ips = NULL; -} - /* A group of logical router datapaths which are connected - either * directly or indirectly. * Each logical router can belong to only one group. */ @@ -677,7 +671,6 @@ ovn_datapath_destroy(struct hmap *datapaths, struct ovn_datapath *od) destroy_ipam_info(&od->ipam_info); free(od->router_ports); free(od->ls_peers); - destroy_lb_for_datapath(od); free(od->localnet_ports); free(od->l3dgw_ports); destroy_mcast_info_for_datapath(od); @@ -1318,124 +1311,6 @@ struct lflow_ref_node { struct ovn_lflow *lflow; }; -/* A logical switch port or logical router port. - * - * In steady state, an ovn_port points to a northbound Logical_Switch_Port - * record (via 'nbsp') *or* a Logical_Router_Port record (via 'nbrp'), and to a - * southbound Port_Binding record (via 'sb'). As the state of the system - * changes, join_logical_ports() may determine that there is a new LSP or LRP - * that has no corresponding Port_Binding record (in which case build_ports()) - * will create the missing Port_Binding) or that a Port_Binding record exists - * that has no coresponding LSP (in which case build_ports() will delete the - * spurious Port_Binding). Thus, after build_ports() runs, any given ovn_port - * will have 'sb' nonnull, and 'nbsp' xor 'nbrp' nonnull. - * - * Ordinarily there is only one ovn_port that points to a given LSP or LRP (but - * distributed gateway ports point a "derived" ovn_port to a duplicate LRP). - */ -struct ovn_port { - /* Port name aka key. - * - * This is ordinarily the same as nbsp->name or nbrp->name and - * sb->logical_port. (A distributed gateway port creates a "derived" - * ovn_port with key "cr-%s" % nbrp->name.) */ - struct hmap_node key_node; /* Index on 'key'. */ - char *key; /* nbsp->name, nbrp->name, sb->logical_port. */ - char *json_key; /* 'key', quoted for use in JSON. */ - - const struct sbrec_port_binding *sb; /* May be NULL. */ - - uint32_t tunnel_key; - - /* Logical switch port data. */ - const struct nbrec_logical_switch_port *nbsp; /* May be NULL. */ - - struct lport_addresses *lsp_addrs; /* Logical switch port addresses. */ - unsigned int n_lsp_addrs; /* Total length of lsp_addrs. */ - unsigned int n_lsp_non_router_addrs; /* Number of elements from the - * beginning of 'lsp_addrs' extracted - * directly from LSP 'addresses'. */ - - struct lport_addresses *ps_addrs; /* Port security addresses. */ - unsigned int n_ps_addrs; - - bool lsp_can_be_inc_processed; /* If it can be incrementally processed when - the port changes. */ - - /* Logical router port data. */ - const struct nbrec_logical_router_port *nbrp; /* May be NULL. */ - - struct lport_addresses lrp_networks; - - struct ovn_port_routable_addresses routables; - - /* Logical port multicast data. */ - struct mcast_port_info mcast_info; - - /* At most one of l3dgw_port and cr_port can be not NULL. */ - - /* This is set to a distributed gateway port if and only if this ovn_port - * is "derived" from it. Otherwise this is set to NULL. The derived - * ovn_port represents the instance of distributed gateway port on the - * gateway chassis.*/ - struct ovn_port *l3dgw_port; - - /* This is set to the "derived" chassis-redirect port of this port if and - * only if this port is a distributed gateway port. Otherwise this is set - * to NULL. */ - struct ovn_port *cr_port; - - bool has_unknown; /* If the addresses have 'unknown' defined. */ - - bool has_bfd; - - /* The port's peer: - * - * - A switch port S of type "router" has a router port R as a peer, - * and R in turn has S has its peer. - * - * - Two connected logical router ports have each other as peer. - * - * - Other kinds of ports have no peer. */ - struct ovn_port *peer; - - struct ovn_datapath *od; - - struct ovs_list list; /* In list of similar records. */ - - struct hmap_node dp_node; /* Node in od->ports. */ - - struct lport_addresses proxy_arp_addrs; - - /* Temporarily used for traversing a list (or hmap) of ports. */ - bool visited; - - /* List of struct lflow_ref_node that points to the lflows generated by - * this ovn_port. - * - * This data is initialized and destroyed by the en_northd node, but - * populated and used only by the en_lflow node. Ideally this data should - * be maintained as part of en_lflow's data (struct lflow_data): a hash - * index from ovn_port key to lflows. However, it would be less efficient - * and more complex: - * - * 1. It would require an extra search (using the index) to find the - * lflows. - * - * 2. Building the index needs to be thread-safe, using either a global - * lock which is obviously less efficient, or hash-based lock array which - * is more complex. - * - * Adding the list here is more straightforward. The drawback is that we - * need to keep in mind that this data belongs to en_lflow node, so never - * access it from any other nodes. - */ - struct ovs_list lflows; - - /* Only used for the router type LSP whose peer is l3dgw_port */ - bool enable_router_port_acl; -}; - static bool lsp_can_be_inc_processed(const struct nbrec_logical_switch_port *); static bool @@ -1460,16 +1335,21 @@ destroy_routable_addresses(struct ovn_port_routable_addresses *ra) } static char **get_nat_addresses(const struct ovn_port *op, size_t *n, - bool routable_only, bool include_lb_ips); + bool routable_only, bool include_lb_ips, + const struct lr_stateful_record *); -static void -assign_routable_addresses(struct ovn_port *op) +static struct ovn_port_routable_addresses +get_op_routable_addresses(struct ovn_port *op, + const struct lr_stateful_record *lr_stateful_rec) { size_t n; - char **nats = get_nat_addresses(op, &n, true, true); + char **nats = get_nat_addresses(op, &n, true, true, lr_stateful_rec); if (!nats) { - return; + return (struct ovn_port_routable_addresses) { + .laddrs = NULL, + .n_addrs = 0, + }; } struct lport_addresses *laddrs = xcalloc(n, sizeof(*laddrs)); @@ -1485,9 +1365,15 @@ assign_routable_addresses(struct ovn_port *op) } free(nats); - /* Everything seems to have worked out */ - op->routables.laddrs = laddrs; - op->routables.n_addrs = n_addrs; + if (!n_addrs) { + free(laddrs); + laddrs = NULL; + } + + return (struct ovn_port_routable_addresses) { + .laddrs = laddrs, + .n_addrs = n_addrs, + }; } @@ -1547,8 +1433,6 @@ ovn_port_destroy_orphan(struct ovn_port *port) } free(port->ps_addrs); - destroy_routable_addresses(&port->routables); - destroy_lport_addresses(&port->lrp_networks); destroy_lport_addresses(&port->proxy_arp_addrs); free(port->json_key); @@ -2590,9 +2474,7 @@ join_logical_ports(const struct sbrec_port_binding_table *sbrec_pb_table, sizeof *od->l3dgw_ports); } od->l3dgw_ports[od->n_l3dgw_ports++] = op; - - assign_routable_addresses(op); - } + } } } @@ -2695,7 +2577,8 @@ join_logical_ports(const struct sbrec_port_binding_table *sbrec_pb_table, * and must free the returned array when it is no longer needed. */ static char ** get_nat_addresses(const struct ovn_port *op, size_t *n, bool routable_only, - bool include_lb_ips) + bool include_lb_ips, + const struct lr_stateful_record *lr_stateful_rec) { size_t n_nats = 0; struct eth_addr mac; @@ -2780,23 +2663,25 @@ get_nat_addresses(const struct ovn_port *op, size_t *n, bool routable_only, } } - if (include_lb_ips) { + if (include_lb_ips && lr_stateful_rec) { const char *ip_address; if (routable_only) { - SSET_FOR_EACH (ip_address, &op->od->lb_ips->ips_v4_routable) { + SSET_FOR_EACH (ip_address, + &lr_stateful_rec->lb_ips->ips_v4_routable) { ds_put_format(&c_addresses, " %s", ip_address); central_ip_address = true; } - SSET_FOR_EACH (ip_address, &op->od->lb_ips->ips_v6_routable) { + SSET_FOR_EACH (ip_address, + &lr_stateful_rec->lb_ips->ips_v6_routable) { ds_put_format(&c_addresses, " %s", ip_address); central_ip_address = true; } } else { - SSET_FOR_EACH (ip_address, &op->od->lb_ips->ips_v4) { + SSET_FOR_EACH (ip_address, &lr_stateful_rec->lb_ips->ips_v4) { ds_put_format(&c_addresses, " %s", ip_address); central_ip_address = true; } - SSET_FOR_EACH (ip_address, &op->od->lb_ips->ips_v6) { + SSET_FOR_EACH (ip_address, &lr_stateful_rec->lb_ips->ips_v6) { ds_put_format(&c_addresses, " %s", ip_address); central_ip_address = true; } @@ -3867,21 +3752,8 @@ build_lb_datapaths(const struct hmap *lbs, const struct hmap *lb_groups, HMAP_FOR_EACH (od, key_node, &lr_datapaths->datapaths) { ovs_assert(od->nbr); - /* Checking load balancer groups first, starting from the largest one, - * to more efficiently copy IP sets. */ - size_t largest_group = 0; - - for (size_t i = 1; i < od->nbr->n_load_balancer_group; i++) { - if (od->nbr->load_balancer_group[i]->n_load_balancer > - od->nbr->load_balancer_group[largest_group]->n_load_balancer) { - largest_group = i; - } - } - for (size_t i = 0; i < od->nbr->n_load_balancer_group; i++) { - size_t idx = (i + largest_group) % od->nbr->n_load_balancer_group; - - nbrec_lb_group = od->nbr->load_balancer_group[idx]; + nbrec_lb_group = od->nbr->load_balancer_group[i]; const struct uuid *lb_group_uuid = &nbrec_lb_group->header_.uuid; lb_group_dps = @@ -3889,20 +3761,6 @@ build_lb_datapaths(const struct hmap *lbs, const struct hmap *lb_groups, lb_group_uuid); ovs_assert(lb_group_dps); ovn_lb_group_datapaths_add_lr(lb_group_dps, od); - - if (!od->lb_ips) { - od->lb_ips = - ovn_lb_ip_set_clone(lb_group_dps->lb_group->lb_ips); - } else { - for (size_t j = 0; j < lb_group_dps->lb_group->n_lbs; j++) { - build_lrouter_lb_ips(od->lb_ips, - lb_group_dps->lb_group->lbs[j]); - } - } - } - - if (!od->lb_ips) { - od->lb_ips = ovn_lb_ip_set_create(); } for (size_t i = 0; i < od->nbr->n_load_balancer; i++) { @@ -3911,7 +3769,6 @@ build_lb_datapaths(const struct hmap *lbs, const struct hmap *lb_groups, lb_dps = ovn_lb_datapaths_find(lb_datapaths_map, lb_uuid); ovs_assert(lb_dps); ovn_lb_datapaths_add_lr(lb_dps, 1, &od); - build_lrouter_lb_ips(od->lb_ips, lb_dps->lb); } } @@ -3965,102 +3822,6 @@ build_lb_svcs( } } -static bool lrouter_port_ipv4_reachable(const struct ovn_port *op, - ovs_be32 addr); -static bool lrouter_port_ipv6_reachable(const struct ovn_port *op, - const struct in6_addr *addr); - -static void -add_neigh_ips_to_lrouter(struct ovn_datapath *od, - enum lb_neighbor_responder_mode neigh_mode, - const struct sset *lb_ips_v4, - const struct sset *lb_ips_v6) -{ - /* If configured to not reply to any neighbor requests for all VIPs - * return early. - */ - if (neigh_mode == LB_NEIGH_RESPOND_NONE) { - return; - } - - const char *ip_address; - - /* If configured to reply to neighbor requests for all VIPs force them - * all to be considered "reachable". - */ - if (neigh_mode == LB_NEIGH_RESPOND_ALL) { - SSET_FOR_EACH (ip_address, lb_ips_v4) { - sset_add(&od->lb_ips->ips_v4_reachable, ip_address); - } - SSET_FOR_EACH (ip_address, lb_ips_v6) { - sset_add(&od->lb_ips->ips_v6_reachable, ip_address); - } - - return; - } - - /* Otherwise, a VIP is reachable if there's at least one router - * subnet that includes it. - */ - ovs_assert(neigh_mode == LB_NEIGH_RESPOND_REACHABLE); - - SSET_FOR_EACH (ip_address, lb_ips_v4) { - struct ovn_port *op; - ovs_be32 vip_ip4; - if (ip_parse(ip_address, &vip_ip4)) { - HMAP_FOR_EACH (op, dp_node, &od->ports) { - if (lrouter_port_ipv4_reachable(op, vip_ip4)) { - sset_add(&od->lb_ips->ips_v4_reachable, - ip_address); - break; - } - } - } - } - - SSET_FOR_EACH (ip_address, lb_ips_v6) { - struct ovn_port *op; - struct in6_addr vip; - if (ipv6_parse(ip_address, &vip)) { - HMAP_FOR_EACH (op, dp_node, &od->ports) { - if (lrouter_port_ipv6_reachable(op, &vip)) { - sset_add(&od->lb_ips->ips_v6_reachable, - ip_address); - break; - } - } - } - } -} - -static void -remove_lrouter_lb_reachable_ips(struct ovn_datapath *od, - enum lb_neighbor_responder_mode neigh_mode, - const struct sset *lb_ips_v4, - const struct sset *lb_ips_v6) -{ - if (neigh_mode == LB_NEIGH_RESPOND_NONE) { - return; - } - - const char *ip_address; - SSET_FOR_EACH (ip_address, lb_ips_v4) { - sset_find_and_delete(&od->lb_ips->ips_v4_reachable, ip_address); - } - SSET_FOR_EACH (ip_address, lb_ips_v6) { - sset_find_and_delete(&od->lb_ips->ips_v6_reachable, ip_address); - } -} - -static void -build_lrouter_lb_reachable_ips(struct ovn_datapath *od, - const struct ovn_northd_lb *lb) -{ - add_neigh_ips_to_lrouter(od, lb->neigh_mode, &lb->ips_v4, - &lb->ips_v6); -} - - static void build_lrouter_lbs_check(const struct ovn_datapaths *lr_datapaths) { @@ -4082,43 +3843,6 @@ build_lrouter_lbs_check(const struct ovn_datapaths *lr_datapaths) } } -static void -build_lrouter_lbs_reachable_ips(struct ovn_datapaths *lr_datapaths, - struct hmap *lb_dps_map, - struct hmap *lb_group_dps_map) -{ - struct ovn_datapath *od; - - HMAP_FOR_EACH (od, key_node, &lr_datapaths->datapaths) { - if (!od->nbr) { - continue; - } - - for (size_t i = 0; i < od->nbr->n_load_balancer; i++) { - struct ovn_lb_datapaths *lb_dps = - ovn_lb_datapaths_find(lb_dps_map, - &od->nbr->load_balancer[i]->header_.uuid); - ovs_assert(lb_dps); - build_lrouter_lb_reachable_ips(od, lb_dps->lb); - } - - for (size_t i = 0; i < od->nbr->n_load_balancer_group; i++) { - const struct nbrec_load_balancer_group *nbrec_lb_group = - od->nbr->load_balancer_group[i]; - struct ovn_lb_group_datapaths *lb_group_dps; - - lb_group_dps = - ovn_lb_group_datapaths_find(lb_group_dps_map, - &nbrec_lb_group->header_.uuid); - ovs_assert(lb_group_dps); - for (size_t j = 0; j < lb_group_dps->lb_group->n_lbs; j++) { - build_lrouter_lb_reachable_ips(od, - lb_group_dps->lb_group->lbs[j]); - } - } - } -} - static void build_lswitch_lbs_from_lrouter(struct ovn_datapaths *lr_datapaths, struct hmap *lb_dps_map, @@ -4182,8 +3906,6 @@ build_lb_port_related_data( struct hmap *svc_monitor_map) { build_lrouter_lbs_check(lr_datapaths); - build_lrouter_lbs_reachable_ips(lr_datapaths, lb_dps_map, - lb_group_dps_map); build_lb_svcs(ovnsb_txn, sbrec_service_monitor_table, ls_ports, lb_dps_map, svc_monitor_lsps, svc_monitor_map); build_lswitch_lbs_from_lrouter(lr_datapaths, lb_dps_map, lb_group_dps_map); @@ -4549,7 +4271,8 @@ check_sb_lb_duplicates(const struct sbrec_load_balancer_table *table) * Caller should make sure that the OVN SB IDL txn is not NULL. Presently it * only syncs the nat column of port binding corresponding to the 'op->nbsp' */ static void -sync_pb_for_lsp(struct ovn_port *op) +sync_pb_for_lsp(struct ovn_port *op, + const struct lr_stateful_table *lr_stateful_table) { ovs_assert(op->nbsp); @@ -4568,10 +4291,17 @@ sync_pb_for_lsp(struct ovn_port *op) if (nat_addresses && !strcmp(nat_addresses, "router")) { if (op->peer && op->peer->od && (chassis || op->peer->od->n_l3dgw_ports)) { - bool exclude_lb_vips = smap_get_bool(&op->nbsp->options, + bool include_lb_vips = !smap_get_bool(&op->nbsp->options, "exclude-lb-vips-from-garp", false); + + const struct lr_stateful_record *lr_stateful_rec = NULL; + + if (include_lb_vips) { + lr_stateful_rec = lr_stateful_table_find_by_index( + lr_stateful_table, op->peer->od->index); + } nats = get_nat_addresses(op->peer, &n_nats, false, - !exclude_lb_vips); + include_lb_vips, lr_stateful_rec); } } else if (nat_addresses && (chassis || l3dgw_ports)) { struct lport_addresses laddrs; @@ -4678,7 +4408,8 @@ sync_pb_for_lsp(struct ovn_port *op) * Caller should make sure that the OVN SB IDL txn is not NULL. Presently it * only sets the port binding options column for the router ports */ static void -sync_pb_for_lrp(struct ovn_port *op, const struct lr_nat_table *lr_nats) +sync_pb_for_lrp(struct ovn_port *op, + const struct lr_stateful_table *lr_stateful_table) { ovs_assert(op->nbrp); @@ -4687,14 +4418,14 @@ sync_pb_for_lrp(struct ovn_port *op, const struct lr_nat_table *lr_nats) const char *chassis_name = smap_get(&op->od->nbr->options, "chassis"); if (is_cr_port(op)) { - const struct lr_nat_record *lrnat_rec = - lr_nat_table_find_by_index(lr_nats, op->od->index); - ovs_assert(lrnat_rec); + const struct lr_stateful_record *lr_stateful_rec = + lr_stateful_table_find_by_index(lr_stateful_table, op->od->index); + ovs_assert(lr_stateful_rec); smap_add(&new, "distributed-port", op->nbrp->name); bool always_redirect = - !lrnat_rec->has_distributed_nat && + !lr_stateful_rec->lrnat_rec->has_distributed_nat && !l3dgw_port_has_associated_vtep_lports(op->l3dgw_port); const char *redirect_type = smap_get(&op->nbrp->options, @@ -4744,17 +4475,18 @@ static void ovn_update_ipv6_opt_for_op(struct ovn_port *op); * the logical switch ports. */ void sync_pbs(struct ovsdb_idl_txn *ovnsb_idl_txn, struct hmap *ls_ports, - struct hmap *lr_ports, const struct lr_nat_table *lr_nats) + struct hmap *lr_ports, + const struct lr_stateful_table *lr_stateful_table) { ovs_assert(ovnsb_idl_txn); struct ovn_port *op; HMAP_FOR_EACH (op, key_node, ls_ports) { - sync_pb_for_lsp(op); + sync_pb_for_lsp(op, lr_stateful_table); } HMAP_FOR_EACH (op, key_node, lr_ports) { - sync_pb_for_lrp(op, lr_nats); + sync_pb_for_lrp(op, lr_stateful_table); } ovn_update_ipv6_options(lr_ports); @@ -4763,20 +4495,22 @@ sync_pbs(struct ovsdb_idl_txn *ovnsb_idl_txn, struct hmap *ls_ports, /* Sync the SB Port bindings for the added and updated logical switch ports * of the tracked northd engine data. */ bool -sync_pbs_for_northd_changed_ovn_ports(struct tracked_ovn_ports *trk_ovn_ports) +sync_pbs_for_northd_changed_ovn_ports( + struct tracked_ovn_ports *trk_ovn_ports, + const struct lr_stateful_table *lr_stateful_table) { struct hmapx_node *hmapx_node; struct ovn_port *op; HMAPX_FOR_EACH (hmapx_node, &trk_ovn_ports->created) { op = hmapx_node->data; ovs_assert(op->nbsp); - sync_pb_for_lsp(op); + sync_pb_for_lsp(op, lr_stateful_table); } HMAPX_FOR_EACH (hmapx_node, &trk_ovn_ports->updated) { op = hmapx_node->data; ovs_assert(op->nbsp); - sync_pb_for_lsp(op); + sync_pb_for_lsp(op, lr_stateful_table); } return true; @@ -5439,14 +5173,13 @@ fail: } /* Returns true if the logical router has changes which can be - * incrementally handled. + * incrementally handled or the changes can be ignored. * Presently supports i-p for the below changes: * - load balancers and load balancer groups. * - NAT changes */ static bool -lr_changes_can_be_handled( - const struct nbrec_logical_router *lr) +lr_changes_can_be_handled(const struct nbrec_logical_router *lr) { /* Check if the columns are changed in this row. */ enum nbrec_logical_router_column_id col; @@ -5510,17 +5243,6 @@ is_lr_nats_changed(const struct nbrec_logical_router *nbr) { || is_lr_nats_seqno_changed(nbr)); } -static bool -lr_has_routable_nats(const struct nbrec_logical_router *nbr) { - for (size_t i = 0; i < nbr->n_nat; i++) { - if (smap_get_bool(&nbr->nat[i]->options, "add_route", false)) { - return true; - } - } - - return false; -} - /* Return true if changes are handled incrementally, false otherwise. * * Note: Changes to load balancer and load balancer groups associated with @@ -5547,12 +5269,6 @@ northd_handle_lr_changes(const struct northd_input *ni, } if (is_lr_nats_changed(changed_lr)) { - if (lr_has_routable_nats(changed_lr)) { - /* router has routable NATs. We can't handle these changes - * incrementally yet. Fall back to recompute. */ - goto fail; - } - struct ovn_datapath *od = ovn_datapath_find_( &nd->lr_datapaths.datapaths, &changed_lr->header_.uuid); @@ -5834,10 +5550,6 @@ northd_handle_lb_data_changes(struct tracked_lb_data *trk_lb_data, ovs_assert(lb_dps); ovn_lb_datapaths_add_lr(lb_dps, 1, &od); - /* Add the lb_ips of lb_dps to the od. */ - build_lrouter_lb_ips(od->lb_ips, lb_dps->lb); - build_lrouter_lb_reachable_ips(od, lb_dps->lb); - /* Add the lb to the northd tracked data. */ hmapx_add(&nd_changes->trk_lbs.crupdated, lb_dps); } @@ -5856,10 +5568,6 @@ northd_handle_lb_data_changes(struct tracked_lb_data *trk_lb_data, ovs_assert(lb_dps); ovn_lb_datapaths_add_lr(lb_dps, 1, &od); - /* Add the lb_ips of lb_dps to the od. */ - build_lrouter_lb_ips(od->lb_ips, lb_dps->lb); - build_lrouter_lb_reachable_ips(od, lb_dps->lb); - /* Add the lb to the northd tracked data. */ hmapx_add(&nd_changes->trk_lbs.crupdated, lb_dps); } @@ -5888,22 +5596,6 @@ northd_handle_lb_data_changes(struct tracked_lb_data *trk_lb_data, od = lr_datapaths->array[index]; /* Re-evaluate 'od->has_lb_vip' */ init_lb_for_datapath(od); - - /* Update the od->lb_ips with the deleted and inserted - * vips (if any). */ - remove_ips_from_lb_ip_set(od->lb_ips, lb->routable, - &clb->deleted_vips_v4, - &clb->deleted_vips_v6); - add_ips_to_lb_ip_set(od->lb_ips, lb->routable, - &clb->inserted_vips_v4, - &clb->inserted_vips_v6); - - remove_lrouter_lb_reachable_ips(od, lb->neigh_mode, - &clb->deleted_vips_v4, - &clb->deleted_vips_v6); - add_neigh_ips_to_lrouter(od, lb->neigh_mode, - &clb->inserted_vips_v4, - &clb->inserted_vips_v6); } } @@ -5928,9 +5620,6 @@ northd_handle_lb_data_changes(struct tracked_lb_data *trk_lb_data, /* Re-evaluate 'od->has_lb_vip' */ init_lb_for_datapath(od); - - /* Add the lb_ips of lb_dps to the od. */ - build_lrouter_lb_ips(od->lb_ips, lb_dps->lb); } for (size_t i = 0; i < lbgrp_dps->n_ls; i++) { @@ -9228,7 +8917,7 @@ arp_nd_ns_match(const char *ips, int addr_family, struct ds *match) /* Returns 'true' if the IPv4 'addr' is on the same subnet with one of the * IPs configured on the router port. */ -static bool +bool lrouter_port_ipv4_reachable(const struct ovn_port *op, ovs_be32 addr) { for (size_t i = 0; i < op->lrp_networks.n_ipv4_addrs; i++) { @@ -9244,7 +8933,7 @@ lrouter_port_ipv4_reachable(const struct ovn_port *op, ovs_be32 addr) /* Returns 'true' if the IPv6 'addr' is on the same subnet with one of the * IPs configured on the router port. */ -static bool +bool lrouter_port_ipv6_reachable(const struct ovn_port *op, const struct in6_addr *addr) { @@ -9306,12 +8995,11 @@ build_lswitch_rport_arp_req_flow(const char *ips, * - 75: ARP requests to router owned IPs (interface IP/LB/NAT). */ static void -build_lswitch_rport_arp_req_flows(struct ovn_port *op, - struct ovn_datapath *sw_od, - struct ovn_port *sw_op, - const struct lr_nat_table *lr_nats, - struct hmap *lflows, - const struct ovsdb_idl_row *stage_hint) +build_lswitch_rport_arp_req_flows( + struct ovn_port *op, struct ovn_datapath *sw_od, + struct ovn_port *sw_op, const struct lr_nat_table *lr_nats, + const struct lr_stateful_table *lr_stateful_table, + struct hmap *lflows, const struct ovsdb_idl_row *stage_hint) { if (!op || !op->nbrp) { return; @@ -9325,32 +9013,38 @@ build_lswitch_rport_arp_req_flows(struct ovn_port *op, * router port. * Priority: 80. */ - - const char *ip_addr; - SSET_FOR_EACH (ip_addr, &op->od->lb_ips->ips_v4_reachable) { - ovs_be32 ipv4_addr; - - /* Check if the ovn port has a network configured on which we could - * expect ARP requests for the LB VIP. - */ - if (ip_parse(ip_addr, &ipv4_addr) && - lrouter_port_ipv4_reachable(op, ipv4_addr)) { - build_lswitch_rport_arp_req_flow( - ip_addr, AF_INET, sw_op, sw_od, 80, lflows, - stage_hint); + const struct lr_stateful_record *lr_stateful_rec = NULL; + if (op->od->nbr->n_load_balancer || op->od->nbr->n_load_balancer_group) { + lr_stateful_rec = lr_stateful_table_find_by_index(lr_stateful_table, + op->od->index); + ovs_assert(lr_stateful_rec); + + const char *ip_addr; + SSET_FOR_EACH (ip_addr, &lr_stateful_rec->lb_ips->ips_v4_reachable) { + ovs_be32 ipv4_addr; + + /* Check if the ovn port has a network configured on which we could + * expect ARP requests for the LB VIP. + */ + if (ip_parse(ip_addr, &ipv4_addr) && + lrouter_port_ipv4_reachable(op, ipv4_addr)) { + build_lswitch_rport_arp_req_flow( + ip_addr, AF_INET, sw_op, sw_od, 80, lflows, + stage_hint); + } } - } - SSET_FOR_EACH (ip_addr, &op->od->lb_ips->ips_v6_reachable) { - struct in6_addr ipv6_addr; + SSET_FOR_EACH (ip_addr, &lr_stateful_rec->lb_ips->ips_v6_reachable) { + struct in6_addr ipv6_addr; - /* Check if the ovn port has a network configured on which we could - * expect NS requests for the LB VIP. - */ - if (ipv6_parse(ip_addr, &ipv6_addr) && - lrouter_port_ipv6_reachable(op, &ipv6_addr)) { - build_lswitch_rport_arp_req_flow( - ip_addr, AF_INET6, sw_op, sw_od, 80, lflows, - stage_hint); + /* Check if the ovn port has a network configured on which we could + * expect NS requests for the LB VIP. + */ + if (ipv6_parse(ip_addr, &ipv6_addr) && + lrouter_port_ipv6_reachable(op, &ipv6_addr)) { + build_lswitch_rport_arp_req_flow( + ip_addr, AF_INET6, sw_op, sw_od, 80, lflows, + stage_hint); + } } } @@ -9400,13 +9094,17 @@ build_lswitch_rport_arp_req_flows(struct ovn_port *op, * expect ARP requests/NS for the DNAT external_ip. */ if (nat_entry_is_v6(nat_entry)) { - if (!sset_contains(&op->od->lb_ips->ips_v6, nat->external_ip)) { + if (!lr_stateful_rec || + !sset_contains(&lr_stateful_rec->lb_ips->ips_v6, + nat->external_ip)) { build_lswitch_rport_arp_req_flow( nat->external_ip, AF_INET6, sw_op, sw_od, 80, lflows, stage_hint); } } else { - if (!sset_contains(&op->od->lb_ips->ips_v4, nat->external_ip)) { + if (!lr_stateful_rec || + !sset_contains(&lr_stateful_rec->lb_ips->ips_v4, + nat->external_ip)) { build_lswitch_rport_arp_req_flow( nat->external_ip, AF_INET, sw_op, sw_od, 80, lflows, stage_hint); @@ -9431,13 +9129,17 @@ build_lswitch_rport_arp_req_flows(struct ovn_port *op, * expect ARP requests/NS for the SNAT external_ip. */ if (nat_entry_is_v6(nat_entry)) { - if (!sset_contains(&op->od->lb_ips->ips_v6, nat->external_ip)) { + if (!lr_stateful_rec || + !sset_contains(&lr_stateful_rec->lb_ips->ips_v6, + nat->external_ip)) { build_lswitch_rport_arp_req_flow( nat->external_ip, AF_INET6, sw_op, sw_od, 80, lflows, stage_hint); } } else { - if (!sset_contains(&op->od->lb_ips->ips_v4, nat->external_ip)) { + if (!lr_stateful_rec || + !sset_contains(&lr_stateful_rec->lb_ips->ips_v4, + nat->external_ip)) { build_lswitch_rport_arp_req_flow( nat->external_ip, AF_INET, sw_op, sw_od, 80, lflows, stage_hint); @@ -10476,11 +10178,11 @@ build_lswitch_ip_mcast_igmp_mld(struct ovn_igmp_group *igmp_group, /* Ingress table 25: Destination lookup, unicast handling (priority 50), */ static void -build_lswitch_ip_unicast_lookup(struct ovn_port *op, - const struct lr_nat_table *lr_nats, - struct hmap *lflows, - struct ds *actions, - struct ds *match) +build_lswitch_ip_unicast_lookup( + struct ovn_port *op, const struct lr_nat_table *lr_nats, + const struct lr_stateful_table *lr_stateful_table, + struct hmap *lflows, struct ds *actions, + struct ds *match) { ovs_assert(op->nbsp); if (lsp_is_external(op->nbsp)) { @@ -10493,7 +10195,8 @@ build_lswitch_ip_unicast_lookup(struct ovn_port *op, */ if (lsp_is_router(op->nbsp)) { build_lswitch_rport_arp_req_flows(op->peer, op->od, op, lr_nats, - lflows, &op->nbsp->header_); + lr_stateful_table, lflows, + &op->nbsp->header_); } for (size_t i = 0; i < op->nbsp->n_addresses; i++) { @@ -12683,6 +12386,7 @@ build_lrouter_port_nat_arp_nd_flow(struct ovn_port *op, static void build_lrouter_drop_own_dest(struct ovn_port *op, const struct lr_nat_record *lrnat_rec, + const struct lr_stateful_record *lr_stateful_rec, enum ovn_stage stage, uint16_t priority, bool drop_snat_ip, struct hmap *lflows) @@ -12695,8 +12399,8 @@ build_lrouter_drop_own_dest(struct ovn_port *op, bool router_ip_in_snat_ips = !!shash_find(&lrnat_rec->snat_ips, ip); - bool router_ip_in_lb_ips = - !!sset_find(&op->od->lb_ips->ips_v4, ip); + bool router_ip_in_lb_ips = (lr_stateful_rec && + !!sset_find(&lr_stateful_rec->lb_ips->ips_v4, ip)); bool drop_router_ip = (drop_snat_ip == (router_ip_in_snat_ips || router_ip_in_lb_ips)); @@ -12725,8 +12429,8 @@ build_lrouter_drop_own_dest(struct ovn_port *op, bool router_ip_in_snat_ips = !!shash_find(&lrnat_rec->snat_ips, ip); - bool router_ip_in_lb_ips = - !!sset_find(&op->od->lb_ips->ips_v6, ip); + bool router_ip_in_lb_ips = (lr_stateful_rec && + !!sset_find(&lr_stateful_rec->lb_ips->ips_v6, ip)); bool drop_router_ip = (drop_snat_ip == (router_ip_in_snat_ips || router_ip_in_lb_ips)); @@ -13438,7 +13142,8 @@ build_ip_routing_flows_for_lrp( */ static void build_ip_routing_flows_for_router_type_lsp( - struct ovn_port *op, const struct hmap *lr_ports, struct hmap *lflows) + struct ovn_port *op, const struct lr_stateful_table *lr_stateful_table, + const struct hmap *lr_ports, struct hmap *lflows) { ovs_assert(op->nbsp); if (!lsp_is_router(op->nbsp)) { @@ -13446,7 +13151,8 @@ build_ip_routing_flows_for_router_type_lsp( } struct ovn_port *peer = ovn_port_get_peer(lr_ports, op); - if (!peer || !peer->nbrp || !peer->lrp_networks.n_ipv4_addrs) { + if (!peer || !peer->nbrp || !peer->lrp_networks.n_ipv4_addrs + || !op->od->n_router_ports) { return; } @@ -13457,19 +13163,29 @@ build_ip_routing_flows_for_router_type_lsp( continue; } - struct ovn_port_routable_addresses *ra = &router_port->routables; - for (size_t j = 0; j < ra->n_addrs; j++) { - struct lport_addresses *laddrs = &ra->laddrs[j]; - for (size_t k = 0; k < laddrs->n_ipv4_addrs; k++) { - add_route(lflows, peer->od, peer, - peer->lrp_networks.ipv4_addrs[0].addr_s, - laddrs->ipv4_addrs[k].network_s, - laddrs->ipv4_addrs[k].plen, NULL, false, 0, - &peer->nbrp->header_, false, - ROUTE_PRIO_OFFSET_CONNECTED); + const struct lr_stateful_record *lr_stateful_rec = + lr_stateful_table_find_by_index(lr_stateful_table, + router_port->od->index); + + if (router_port->nbrp->ha_chassis_group || + router_port->nbrp->n_gateway_chassis) { + struct ovn_port_routable_addresses ra = + get_op_routable_addresses(router_port, lr_stateful_rec); + for (size_t j = 0; j < ra.n_addrs; j++) { + struct lport_addresses *laddrs = &ra.laddrs[j]; + for (size_t k = 0; k < laddrs->n_ipv4_addrs; k++) { + add_route(lflows, peer->od, peer, + peer->lrp_networks.ipv4_addrs[0].addr_s, + laddrs->ipv4_addrs[k].network_s, + laddrs->ipv4_addrs[k].plen, NULL, false, 0, + &peer->nbrp->header_, false, + ROUTE_PRIO_OFFSET_CONNECTED); + } } + destroy_routable_addresses(&ra); } } + } static void @@ -13693,33 +13409,36 @@ build_arp_resolve_flows_for_lrouter( static void routable_addresses_to_lflows(struct hmap *lflows, struct ovn_port *router_port, - struct ovn_port *peer, struct ds *match, - struct ds *actions) + struct ovn_port *peer, + const struct lr_stateful_record *lr_stateful_rec, + struct ds *match, struct ds *actions) { - struct ovn_port_routable_addresses *ra = &router_port->routables; - if (!ra->n_addrs) { + struct ovn_port_routable_addresses ra = + get_op_routable_addresses(router_port, lr_stateful_rec); + if (!ra.n_addrs) { return; } - for (size_t i = 0; i < ra->n_addrs; i++) { + for (size_t i = 0; i < ra.n_addrs; i++) { ds_clear(match); ds_put_format(match, "outport == %s && "REG_NEXT_HOP_IPV4" == {", peer->json_key); bool first = true; - for (size_t j = 0; j < ra->laddrs[i].n_ipv4_addrs; j++) { + for (size_t j = 0; j < ra.laddrs[i].n_ipv4_addrs; j++) { if (!first) { ds_put_cstr(match, ", "); } - ds_put_cstr(match, ra->laddrs[i].ipv4_addrs[j].addr_s); + ds_put_cstr(match, ra.laddrs[i].ipv4_addrs[j].addr_s); first = false; } ds_put_cstr(match, "}"); ds_clear(actions); - ds_put_format(actions, "eth.dst = %s; next;", ra->laddrs[i].ea_s); + ds_put_format(actions, "eth.dst = %s; next;", ra.laddrs[i].ea_s); ovn_lflow_add(lflows, peer->od, S_ROUTER_IN_ARP_RESOLVE, 100, ds_cstr(match), ds_cstr(actions)); } + destroy_routable_addresses(&ra); } /* Local router ingress table ARP_RESOLVE: ARP Resolution. @@ -13736,6 +13455,7 @@ routable_addresses_to_lflows(struct hmap *lflows, struct ovn_port *router_port, static void build_arp_resolve_flows_for_lrp( struct ovn_port *op, const struct lr_nat_record *lrnat_rec, + const struct lr_stateful_record *lr_stateful_rec, struct hmap *lflows, struct ds *match, struct ds *actions) { ovs_assert(op->nbrp); @@ -13812,8 +13532,8 @@ build_arp_resolve_flows_for_lrp( * * Priority 2. */ - build_lrouter_drop_own_dest(op, lrnat_rec, S_ROUTER_IN_ARP_RESOLVE, 2, - true, lflows); + build_lrouter_drop_own_dest(op, lrnat_rec, lr_stateful_rec, + S_ROUTER_IN_ARP_RESOLVE, 2, true, lflows); } /* This function adds ARP resolve flows related to a LSP. */ @@ -13821,6 +13541,7 @@ static void build_arp_resolve_flows_for_lsp( struct ovn_port *op, struct hmap *lflows, const struct hmap *lr_ports, + const struct lr_stateful_table *lr_stateful_table, struct ds *match, struct ds *actions) { ovs_assert(op->nbsp); @@ -13964,8 +13685,11 @@ build_arp_resolve_flows_for_lsp( if (smap_get(&peer->od->nbr->options, "chassis") || peer->cr_port) { + const struct lr_stateful_record *lr_stateful_rec = + lr_stateful_table_find_by_index(lr_stateful_table, + router_port->od->index); routable_addresses_to_lflows(lflows, router_port, peer, - match, actions); + lr_stateful_rec, match, actions); } } } @@ -14685,6 +14409,7 @@ static void build_lrouter_ipv4_ip_input(struct ovn_port *op, struct hmap *lflows, const struct lr_nat_record *lrnat_rec, + const struct lr_stateful_record *lr_stateful_rec, struct ds *match, struct ds *actions, const struct shash *meter_groups) { @@ -14809,7 +14534,8 @@ build_lrouter_ipv4_ip_input(struct ovn_port *op, &op->nbrp->header_, lflows); } - if (sset_count(&op->od->lb_ips->ips_v4_reachable)) { + if (lr_stateful_rec && sset_count( + &lr_stateful_rec->lb_ips->ips_v4_reachable)) { ds_clear(match); if (is_l3dgw_port(op)) { ds_put_format(match, "is_chassis_resident(%s)", @@ -14825,7 +14551,8 @@ build_lrouter_ipv4_ip_input(struct ovn_port *op, free(lb_ips_v4_as); } - if (sset_count(&op->od->lb_ips->ips_v6_reachable)) { + if (lr_stateful_rec && sset_count( + &lr_stateful_rec->lb_ips->ips_v6_reachable)) { ds_clear(match); if (is_l3dgw_port(op)) { @@ -14927,8 +14654,8 @@ build_lrouter_ipv4_ip_input(struct ovn_port *op, * Priority 60. */ if (!lrnat_rec->lb_force_snat_router_ip) { - build_lrouter_drop_own_dest(op, lrnat_rec, S_ROUTER_IN_IP_INPUT, 60, - false, lflows); + build_lrouter_drop_own_dest(op, lrnat_rec, lr_stateful_rec, + S_ROUTER_IN_IP_INPUT, 60, false, lflows); } /* ARP / ND handling for external IP addresses. * @@ -16067,6 +15794,7 @@ struct lswitch_flow_build_info { const struct hmap *lr_ports; const struct ls_port_group_table *ls_port_groups; const struct lr_nat_table *lr_nats; + const struct lr_stateful_table *lr_stateful_table; struct hmap *lflows; struct hmap *igmp_groups; const struct shash *meter_groups; @@ -16150,14 +15878,15 @@ build_lswitch_and_lrouter_iterate_by_lr(struct ovn_datapath *od, * switch port. */ static void -build_lswitch_and_lrouter_iterate_by_lsp(struct ovn_port *op, - const struct hmap *ls_ports, - const struct hmap *lr_ports, - const struct lr_nat_table *lr_nats, - const struct shash *meter_groups, - struct ds *match, - struct ds *actions, - struct hmap *lflows) +build_lswitch_and_lrouter_iterate_by_lsp( + struct ovn_port *op, const struct hmap *ls_ports, + const struct hmap *lr_ports, + const struct lr_nat_table *lr_nats, + const struct lr_stateful_table *lr_stateful_table, + const struct shash *meter_groups, + struct ds *match, + struct ds *actions, + struct hmap *lflows) { ovs_assert(op->nbsp); start_collecting_lflows(); @@ -16170,11 +15899,14 @@ build_lswitch_and_lrouter_iterate_by_lsp(struct ovn_port *op, meter_groups, actions, match); build_lswitch_dhcp_options_and_response(op, lflows, meter_groups); build_lswitch_external_port(op, lflows); - build_lswitch_ip_unicast_lookup(op, lr_nats, lflows, actions, match); + build_lswitch_ip_unicast_lookup(op, lr_nats, lr_stateful_table, lflows, + actions, match); /* Build Logical Router Flows. */ - build_ip_routing_flows_for_router_type_lsp(op, lr_ports, lflows); - build_arp_resolve_flows_for_lsp(op, lflows, lr_ports, match, actions); + build_ip_routing_flows_for_router_type_lsp(op, lr_stateful_table, lr_ports, + lflows); + build_arp_resolve_flows_for_lsp(op, lflows, lr_ports, lr_stateful_table, + match, actions); link_ovn_port_to_lflows(op, &collected_lflows); end_collecting_lflows(); @@ -16193,6 +15925,8 @@ build_lswitch_and_lrouter_iterate_by_lrp(struct ovn_port *op, lsi->lr_nats, op->od->index); ovs_assert(lrnet_rec); + const struct lr_stateful_record *lr_stateful_rec = + lr_stateful_table_find_by_index(lsi->lr_stateful_table, op->od->index); build_adm_ctrl_flows_for_lrouter_port(op, lsi->lflows, &lsi->match, &lsi->actions); build_neigh_learning_flows_for_lrouter_port(op, lsi->lflows, &lsi->match, @@ -16200,15 +15934,15 @@ build_lswitch_and_lrouter_iterate_by_lrp(struct ovn_port *op, build_ip_routing_flows_for_lrp(op, lsi->lflows); build_ND_RA_flows_for_lrouter_port(op, lsi->lflows, &lsi->match, &lsi->actions, lsi->meter_groups); - build_arp_resolve_flows_for_lrp(op, lrnet_rec, lsi->lflows, &lsi->match, - &lsi->actions); + build_arp_resolve_flows_for_lrp(op, lrnet_rec, lr_stateful_rec, + lsi->lflows, &lsi->match, &lsi->actions); build_egress_delivery_flows_for_lrouter_port(op, lsi->lflows, &lsi->match, &lsi->actions); build_dhcpv6_reply_flows_for_lrouter_port(op, lsi->lflows, &lsi->match); build_ipv6_input_flows_for_lrouter_port(op, lsi->lflows, &lsi->match, &lsi->actions, lsi->meter_groups); - build_lrouter_ipv4_ip_input(op, lsi->lflows, lrnet_rec, + build_lrouter_ipv4_ip_input(op, lsi->lflows, lrnet_rec, lr_stateful_rec, &lsi->match, &lsi->actions, lsi->meter_groups); build_lrouter_force_snat_flows_op(op, lrnet_rec, lsi->lflows, &lsi->match, &lsi->actions); @@ -16268,13 +16002,10 @@ build_lflows_thread(void *arg) if (stop_parallel_processing()) { return NULL; } - build_lswitch_and_lrouter_iterate_by_lsp(op, lsi->ls_ports, - lsi->lr_ports, - lsi->lr_nats, - lsi->meter_groups, - &lsi->match, - &lsi->actions, - lsi->lflows); + build_lswitch_and_lrouter_iterate_by_lsp( + op, lsi->ls_ports, lsi->lr_ports, lsi->lr_nats, + lsi->lr_stateful_table, lsi->meter_groups, + &lsi->match, &lsi->actions, lsi->lflows); } } for (bnum = control->id; @@ -16375,19 +16106,21 @@ fix_flow_map_size(struct hmap *lflow_map, } static void -build_lswitch_and_lrouter_flows(const struct ovn_datapaths *ls_datapaths, - const struct ovn_datapaths *lr_datapaths, - const struct hmap *ls_ports, - const struct hmap *lr_ports, - const struct ls_port_group_table *ls_pgs, - const struct lr_nat_table *lr_nats, - struct hmap *lflows, - struct hmap *igmp_groups, - const struct shash *meter_groups, - const struct hmap *lb_dps_map, - const struct hmap *svc_monitor_map, - const struct hmap *bfd_connections, - const struct chassis_features *features) +build_lswitch_and_lrouter_flows( + const struct ovn_datapaths *ls_datapaths, + const struct ovn_datapaths *lr_datapaths, + const struct hmap *ls_ports, + const struct hmap *lr_ports, + const struct ls_port_group_table *ls_pgs, + const struct lr_nat_table *lr_nats, + const struct lr_stateful_table *lr_stateful_table, + struct hmap *lflows, + struct hmap *igmp_groups, + const struct shash *meter_groups, + const struct hmap *lb_dps_map, + const struct hmap *svc_monitor_map, + const struct hmap *bfd_connections, + const struct chassis_features *features) { char *svc_check_match = xasprintf("eth.dst == %s", svc_monitor_mac); @@ -16411,6 +16144,7 @@ build_lswitch_and_lrouter_flows(const struct ovn_datapaths *ls_datapaths, lsiv[index].lr_ports = lr_ports; lsiv[index].ls_port_groups = ls_pgs; lsiv[index].lr_nats = lr_nats; + lsiv[index].lr_stateful_table = lr_stateful_table; lsiv[index].igmp_groups = igmp_groups; lsiv[index].meter_groups = meter_groups; lsiv[index].lb_dps_map = lb_dps_map; @@ -16446,6 +16180,7 @@ build_lswitch_and_lrouter_flows(const struct ovn_datapaths *ls_datapaths, .lr_ports = lr_ports, .ls_port_groups = ls_pgs, .lr_nats = lr_nats, + .lr_stateful_table = lr_stateful_table, .lflows = lflows, .igmp_groups = igmp_groups, .meter_groups = meter_groups, @@ -16474,6 +16209,7 @@ build_lswitch_and_lrouter_flows(const struct ovn_datapaths *ls_datapaths, build_lswitch_and_lrouter_iterate_by_lsp(op, lsi.ls_ports, lsi.lr_ports, lsi.lr_nats, + lsi.lr_stateful_table, lsi.meter_groups, &lsi.match, &lsi.actions, lsi.lflows); @@ -16595,6 +16331,7 @@ void build_lflows(struct ovsdb_idl_txn *ovnsb_txn, input_data->lr_ports, input_data->ls_port_groups, input_data->lr_nats, + input_data->lr_stateful_table, lflows, &igmp_groups, input_data->meter_groups, @@ -17070,12 +16807,10 @@ lflow_handle_northd_port_changes(struct ovsdb_idl_txn *ovnsb_txn, /* Generate new lflows. */ struct ds match = DS_EMPTY_INITIALIZER; struct ds actions = DS_EMPTY_INITIALIZER; - build_lswitch_and_lrouter_iterate_by_lsp(op, lflow_input->ls_ports, - lflow_input->lr_ports, - lflow_input->lr_nats, - lflow_input->meter_groups, - &match, &actions, - lflows); + build_lswitch_and_lrouter_iterate_by_lsp( + op, lflow_input->ls_ports, lflow_input->lr_ports, + lflow_input->lr_nats, lflow_input->lr_stateful_table, + lflow_input->meter_groups, &match, &actions, lflows); ds_destroy(&match); ds_destroy(&actions); @@ -17107,12 +16842,10 @@ lflow_handle_northd_port_changes(struct ovsdb_idl_txn *ovnsb_txn, struct ds match = DS_EMPTY_INITIALIZER; struct ds actions = DS_EMPTY_INITIALIZER; - build_lswitch_and_lrouter_iterate_by_lsp(op, lflow_input->ls_ports, - lflow_input->lr_ports, - lflow_input->lr_nats, - lflow_input->meter_groups, - &match, &actions, - lflows); + build_lswitch_and_lrouter_iterate_by_lsp( + op, lflow_input->ls_ports, lflow_input->lr_ports, + lflow_input->lr_nats, lflow_input->lr_stateful_table, + lflow_input->meter_groups, &match, &actions, lflows); ds_destroy(&match); ds_destroy(&actions); diff --git a/northd/northd.h b/northd/northd.h index 4dd260761e..518e46ddbe 100644 --- a/northd/northd.h +++ b/northd/northd.h @@ -180,6 +180,7 @@ struct lflow_input { const struct hmap *lr_ports; const struct ls_port_group_table *ls_port_groups; const struct lr_nat_table *lr_nats; + const struct lr_stateful_table *lr_stateful_table; const struct shash *meter_groups; const struct hmap *lb_datapaths_map; const struct hmap *bfd_connections; @@ -319,9 +320,6 @@ struct ovn_datapath { /* router datapath has a logical port with redirect-type set to bridged. */ bool redirect_bridged; - /* Load Balancer vIPs relevant for this datapath. */ - struct ovn_lb_ip_set *lb_ips; - struct ovn_port **localnet_ports; size_t n_localnet_ports; @@ -338,6 +336,122 @@ struct ovn_datapath { const struct ovn_datapath *ovn_datapath_find(const struct hmap *datapaths, const struct uuid *uuid); +/* A logical switch port or logical router port. + * + * In steady state, an ovn_port points to a northbound Logical_Switch_Port + * record (via 'nbsp') *or* a Logical_Router_Port record (via 'nbrp'), and to a + * southbound Port_Binding record (via 'sb'). As the state of the system + * changes, join_logical_ports() may determine that there is a new LSP or LRP + * that has no corresponding Port_Binding record (in which case build_ports()) + * will create the missing Port_Binding) or that a Port_Binding record exists + * that has no coresponding LSP (in which case build_ports() will delete the + * spurious Port_Binding). Thus, after build_ports() runs, any given ovn_port + * will have 'sb' nonnull, and 'nbsp' xor 'nbrp' nonnull. + * + * Ordinarily there is only one ovn_port that points to a given LSP or LRP (but + * distributed gateway ports point a "derived" ovn_port to a duplicate LRP). + */ +struct ovn_port { + /* Port name aka key. + * + * This is ordinarily the same as nbsp->name or nbrp->name and + * sb->logical_port. (A distributed gateway port creates a "derived" + * ovn_port with key "cr-%s" % nbrp->name.) */ + struct hmap_node key_node; /* Index on 'key'. */ + char *key; /* nbsp->name, nbrp->name, sb->logical_port. */ + char *json_key; /* 'key', quoted for use in JSON. */ + + const struct sbrec_port_binding *sb; /* May be NULL. */ + + uint32_t tunnel_key; + + /* Logical switch port data. */ + const struct nbrec_logical_switch_port *nbsp; /* May be NULL. */ + + struct lport_addresses *lsp_addrs; /* Logical switch port addresses. */ + unsigned int n_lsp_addrs; /* Total length of lsp_addrs. */ + unsigned int n_lsp_non_router_addrs; /* Number of elements from the + * beginning of 'lsp_addrs' extracted + * directly from LSP 'addresses'. */ + + struct lport_addresses *ps_addrs; /* Port security addresses. */ + unsigned int n_ps_addrs; + + bool lsp_can_be_inc_processed; /* If it can be incrementally processed when + the port changes. */ + + /* Logical router port data. */ + const struct nbrec_logical_router_port *nbrp; /* May be NULL. */ + + struct lport_addresses lrp_networks; + + /* Logical port multicast data. */ + struct mcast_port_info mcast_info; + + /* At most one of l3dgw_port and cr_port can be not NULL. */ + + /* This is set to a distributed gateway port if and only if this ovn_port + * is "derived" from it. Otherwise this is set to NULL. The derived + * ovn_port represents the instance of distributed gateway port on the + * gateway chassis.*/ + struct ovn_port *l3dgw_port; + + /* This is set to the "derived" chassis-redirect port of this port if and + * only if this port is a distributed gateway port. Otherwise this is set + * to NULL. */ + struct ovn_port *cr_port; + + bool has_unknown; /* If the addresses have 'unknown' defined. */ + + bool has_bfd; + + /* The port's peer: + * + * - A switch port S of type "router" has a router port R as a peer, + * and R in turn has S has its peer. + * + * - Two connected logical router ports have each other as peer. + * + * - Other kinds of ports have no peer. */ + struct ovn_port *peer; + + struct ovn_datapath *od; + + struct ovs_list list; /* In list of similar records. */ + + struct hmap_node dp_node; /* Node in od->ports. */ + + struct lport_addresses proxy_arp_addrs; + + /* Temporarily used for traversing a list (or hmap) of ports. */ + bool visited; + + /* List of struct lflow_ref_node that points to the lflows generated by + * this ovn_port. + * + * This data is initialized and destroyed by the en_northd node, but + * populated and used only by the en_lflow node. Ideally this data should + * be maintained as part of en_lflow's data (struct lflow_data): a hash + * index from ovn_port key to lflows. However, it would be less efficient + * and more complex: + * + * 1. It would require an extra search (using the index) to find the + * lflows. + * + * 2. Building the index needs to be thread-safe, using either a global + * lock which is obviously less efficient, or hash-based lock array which + * is more complex. + * + * Adding the list here is more straightforward. The drawback is that we + * need to keep in mind that this data belongs to en_lflow node, so never + * access it from any other nodes. + */ + struct ovs_list lflows; + + /* Only used for the router type LSP whose peer is l3dgw_port */ + bool enable_router_port_acl; +}; + void ovnnb_db_run(struct northd_input *input_data, struct northd_data *data, struct ovsdb_idl_txn *ovnnb_txn, @@ -397,9 +511,13 @@ void sync_lbs(struct ovsdb_idl_txn *, const struct sbrec_load_balancer_table *, struct chassis_features *chassis_features); bool check_sb_lb_duplicates(const struct sbrec_load_balancer_table *); +struct lr_stateful_table; void sync_pbs(struct ovsdb_idl_txn *, struct hmap *ls_ports, - struct hmap *lr_ports, const struct lr_nat_table *); -bool sync_pbs_for_northd_changed_ovn_ports(struct tracked_ovn_ports *); + struct hmap *lr_ports, + const struct lr_stateful_table *); +bool sync_pbs_for_northd_changed_ovn_ports( + struct tracked_ovn_ports *, + const struct lr_stateful_table *); static inline bool northd_has_tracked_data(struct northd_tracked_data *trk_nd_changes) { @@ -424,4 +542,15 @@ northd_has_lr_nats_in_tracked_data(struct northd_tracked_data *trk_nd_changes) return (trk_nd_changes->type & NORTHD_TRACKED_LR_NATS); } +/* Returns 'true' if the IPv4 'addr' is on the same subnet with one of the + * IPs configured on the router port. + */ +bool lrouter_port_ipv4_reachable(const struct ovn_port *, ovs_be32 addr); + +/* Returns 'true' if the IPv6 'addr' is on the same subnet with one of the + * IPs configured on the router port. + */ +bool lrouter_port_ipv6_reachable(const struct ovn_port *, + const struct in6_addr *); + #endif /* NORTHD_H */ diff --git a/northd/ovn-northd.c b/northd/ovn-northd.c index 40f9764b3a..e85e00cc9b 100644 --- a/northd/ovn-northd.c +++ b/northd/ovn-northd.c @@ -871,6 +871,7 @@ main(int argc, char *argv[]) stopwatch_create(PORT_GROUP_RUN_STOPWATCH_NAME, SW_MS); stopwatch_create(SYNC_METERS_RUN_STOPWATCH_NAME, SW_MS); stopwatch_create(LR_NAT_RUN_STOPWATCH_NAME, SW_MS); + stopwatch_create(LR_STATEFUL_RUN_STOPWATCH_NAME, SW_MS); /* Initialize incremental processing engine for ovn-northd */ inc_proc_northd_init(&ovnnb_idl_loop, &ovnsb_idl_loop); diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at index ef8c6a616b..1825fd3e18 100644 --- a/tests/ovn-northd.at +++ b/tests/ovn-northd.at @@ -10528,18 +10528,21 @@ check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb set load_balancer . ip_port_mappings:10.0.0.3=sw0-p1:10.0.0.2 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_stateful norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute nocompute check ovn-nbctl --wait=sb set load_balancer . options:foo=bar check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_stateful norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute nocompute check ovn-nbctl --wait=sb -- lb-add lb2 20.0.0.10:80 20.0.0.20:80 -- lb-add lb3 30.0.0.10:80 30.0.0.20:80 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_stateful norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute nocompute @@ -10549,6 +10552,7 @@ check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb -- lb-del lb2 -- lb-del lb3 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_stateful norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute nocompute @@ -10562,6 +10566,7 @@ AT_CHECK([ovn-nbctl --wait=sb \ ]) check_engine_stats lb_data norecompute compute check_engine_stats northd recompute nocompute +check_engine_stats lr_stateful recompute nocompute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute nocompute @@ -10579,6 +10584,7 @@ check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb clear Load_Balancer . health_check check_engine_stats lb_data norecompute compute check_engine_stats northd recompute nocompute +check_engine_stats lr_stateful recompute nocompute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute nocompute @@ -10593,6 +10599,7 @@ check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats ovn-nbctl --wait=sb lsp-set-options sw0-lr0 router-port=lr0-sw0 check_engine_stats lb_data norecompute compute check_engine_stats northd recompute compute +check_engine_stats lr_stateful recompute nocompute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute nocompute @@ -10601,6 +10608,7 @@ check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb ls-lb-add sw0 lb1 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_stateful norecompute compute check_engine_stats lflow recompute nocompute # A LB applied to a switch/router triggers: # - a recompute in the first iteration (handling northd change) @@ -10613,6 +10621,7 @@ check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb set load_balancer lb1 vips:'"10.0.0.10:80"'='"10.0.0.100:80"' check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_stateful norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE(1) @@ -10622,6 +10631,7 @@ check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb clear load_Balancer lb1 vips check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_stateful norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE(1) @@ -10631,6 +10641,7 @@ check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb lb-add lb1 10.0.0.10:80 10.0.0.3:80 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_stateful norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE(1) @@ -10640,6 +10651,7 @@ check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb lb-add lb1 10.0.0.20:80 10.0.0.30:8080 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_stateful norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE(1) @@ -10649,6 +10661,7 @@ check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb ls-lb-del sw0 lb1 check_engine_stats lb_data norecompute compute check_engine_stats northd recompute nocompute +check_engine_stats lr_stateful recompute nocompute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE(1) @@ -10659,6 +10672,7 @@ check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb ls-lb-add sw0 lb1 -- lsp-add sw0 sw0p1 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_stateful norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute compute @@ -10679,6 +10693,7 @@ check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb lr-lb-add lr0 lb1 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_stateful norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10688,6 +10703,7 @@ check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb set load_balancer lb1 vips:'"10.0.0.10:80"'='"10.0.0.100:80"' check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_stateful norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10697,6 +10713,7 @@ check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb clear load_Balancer lb1 vips check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_stateful norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10706,6 +10723,7 @@ check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb lb-add lb1 10.0.0.10:80 10.0.0.3:80 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_stateful norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10715,6 +10733,7 @@ check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb lb-add lb1 10.0.0.20:80 10.0.0.30:8080 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_stateful norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10744,6 +10763,7 @@ check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb add load_balancer_group . load_Balancer $lb1_uuid check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_stateful norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute nocompute @@ -10751,6 +10771,7 @@ check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb clear load_balancer_group . load_Balancer check_engine_stats lb_data norecompute compute check_engine_stats northd recompute nocompute +check_engine_stats lr_stateful recompute nocompute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute nocompute @@ -10767,6 +10788,7 @@ check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb add logical_switch sw0 load_balancer_group $lbg1_uuid check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_stateful norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute compute @@ -10783,6 +10805,7 @@ check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb set load_balancer lb1 vips:'"10.0.0.10:80"'='"10.0.0.100:80"' check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_stateful norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10792,6 +10815,7 @@ check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb clear load_Balancer lb1 vips check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_stateful norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10801,6 +10825,7 @@ check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb lb-add lb1 10.0.0.10:80 10.0.0.3:80 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_stateful norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10810,6 +10835,7 @@ check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb lb-add lb1 10.0.0.20:80 10.0.0.30:8080 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_stateful norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10825,6 +10851,7 @@ check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl add logical_router lr0 load_balancer_group $lbg1_uuid check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_stateful norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10834,6 +10861,7 @@ check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb set load_balancer lb1 vips:'"10.0.0.10:80"'='"10.0.0.100:80"' check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_stateful norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10843,6 +10871,7 @@ check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb clear load_Balancer lb1 vips check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_stateful norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10852,6 +10881,7 @@ check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb lb-add lb1 10.0.0.10:80 10.0.0.3:80 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_stateful norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10861,6 +10891,7 @@ check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb lb-add lb1 10.0.0.20:80 10.0.0.30:8080 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_stateful norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10869,6 +10900,7 @@ check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb clear logical_router lr0 load_balancer_group check_engine_stats lb_data norecompute compute check_engine_stats northd recompute nocompute +check_engine_stats lr_stateful recompute nocompute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute compute @@ -10877,6 +10909,7 @@ check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb add logical_switch sw0 load_balancer_group $lbg1_uuid check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_stateful norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute compute @@ -10885,6 +10918,7 @@ check ovn-nbctl --wait=sb clear logical_switch sw0 load_balancer_group -- \ destroy load_balancer_group $lbg1_uuid check_engine_stats lb_data norecompute compute check_engine_stats northd recompute nocompute +check_engine_stats lr_stateful recompute nocompute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb compute compute @@ -10908,6 +10942,7 @@ check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats lbg1_uuid=$(ovn-nbctl --wait=sb create load_balancer_group name=lbg1) check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_stateful norecompute compute check_engine_stats lflow norecompute nocompute check_engine_stats sync_to_sb_lb norecompute nocompute @@ -10915,6 +10950,7 @@ check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb set load_balancer_group . load_balancer="$lb2_uuid,$lb3_uuid,$lb4_uuid" check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_stateful norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute nocompute @@ -10922,6 +10958,7 @@ check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb set logical_switch sw0 load_balancer_group=$lbg1_uuid check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_stateful norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10930,6 +10967,7 @@ check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb set logical_router lr1 load_balancer_group=$lbg1_uuid check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_stateful norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10938,6 +10976,7 @@ check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb ls-lb-add sw0 lb2 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_stateful norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10946,6 +10985,7 @@ check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb ls-lb-add sw0 lb3 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_stateful norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10955,6 +10995,7 @@ check ovn-nbctl --wait=sb lr-lb-add lr1 lb1 check ovn-nbctl --wait=sb lr-lb-add lr1 lb2 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_stateful norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10963,6 +11004,7 @@ check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb ls-lb-del sw0 lb2 check_engine_stats lb_data norecompute compute check_engine_stats northd recompute nocompute +check_engine_stats lr_stateful recompute nocompute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10971,6 +11013,7 @@ check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb lr-lb-del lr1 lb2 check_engine_stats lb_data norecompute compute check_engine_stats northd recompute nocompute +check_engine_stats lr_stateful recompute nocompute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10981,6 +11024,7 @@ check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb lb-del lb4 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_stateful norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10991,6 +11035,7 @@ check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb lb-del lb2 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_stateful norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10999,6 +11044,7 @@ check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb remove load_balancer_group . load_balancer $lb3_uuid check_engine_stats lb_data norecompute compute check_engine_stats northd recompute nocompute +check_engine_stats lr_stateful recompute nocompute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11241,6 +11287,7 @@ check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb lr-add lr0 check_engine_stats northd recompute nocompute check_engine_stats lr_nat recompute nocompute +check_engine_stats lr_stateful recompute nocompute check_engine_stats sync_to_sb_pb recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11253,6 +11300,7 @@ check ovn-nbctl lrp-add lr0 lr0-sw0 00:00:00:00:ff:01 10.0.0.1/24 # for the SB port binding change. check_engine_stats northd recompute compute check_engine_stats lr_nat recompute nocompute +check_engine_stats lr_stateful recompute nocompute check_engine_stats sync_to_sb_pb recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11264,6 +11312,7 @@ check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb lsp-set-options sw0-lr0 router-port=lr0-sw0 check_engine_stats northd recompute compute check_engine_stats lr_nat recompute nocompute +check_engine_stats lr_stateful recompute nocompute check_engine_stats sync_to_sb_pb recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11289,6 +11338,7 @@ check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl set logical_router_port lr0-sw0 options:foo=bar check_engine_stats northd recompute nocompute check_engine_stats lr_nat recompute nocompute +check_engine_stats lr_stateful recompute nocompute check_engine_stats sync_to_sb_pb recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11318,6 +11368,7 @@ check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb set NAT . external_ip=172.168.0.120 check_engine_stats northd norecompute compute check_engine_stats lr_nat norecompute compute +check_engine_stats lr_stateful norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_pb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11327,6 +11378,7 @@ check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb set NAT . logical_ip=10.0.0.10 check_engine_stats northd norecompute compute check_engine_stats lr_nat norecompute compute +check_engine_stats lr_stateful norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_pb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11336,6 +11388,7 @@ check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb set NAT . type=snat check_engine_stats northd norecompute compute check_engine_stats lr_nat norecompute compute +check_engine_stats lr_stateful norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_pb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11345,6 +11398,7 @@ check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb lr-nat-add lr0 dnat_and_snat 172.168.0.110 10.0.0.4 sw0p1 30:54:00:00:00:03 check_engine_stats northd norecompute compute check_engine_stats lr_nat norecompute compute +check_engine_stats lr_stateful norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_pb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11355,6 +11409,7 @@ check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb set NAT $nat2_uuid external_mac='"30:54:00:00:00:04"' check_engine_stats northd norecompute compute check_engine_stats lr_nat norecompute compute +check_engine_stats lr_stateful norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_pb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11371,6 +11426,7 @@ check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb lr-nat-add lr0 dnat_and_snat 172.168.0.140 10.0.0.20 check_engine_stats northd norecompute compute check_engine_stats lr_nat norecompute compute +check_engine_stats lr_stateful norecompute compute check_engine_stats sync_to_sb_pb recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11379,6 +11435,7 @@ check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb lr-nat-add lr0 dnat_and_snat 172.168.0.150 10.0.0.41 check_engine_stats northd norecompute compute check_engine_stats lr_nat norecompute compute +check_engine_stats lr_stateful norecompute compute check_engine_stats sync_to_sb_pb recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11387,6 +11444,7 @@ check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb lr-nat-del lr0 dnat_and_snat 172.168.0.150 check_engine_stats northd norecompute compute check_engine_stats lr_nat norecompute compute +check_engine_stats lr_stateful norecompute compute check_engine_stats sync_to_sb_pb recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11395,6 +11453,7 @@ check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb lr-nat-del lr0 dnat_and_snat 172.168.0.140 check_engine_stats northd norecompute compute check_engine_stats lr_nat norecompute compute +check_engine_stats lr_stateful norecompute compute check_engine_stats sync_to_sb_pb recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11404,6 +11463,7 @@ check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb clear logical_router lr0 nat check_engine_stats northd norecompute compute check_engine_stats lr_nat norecompute compute +check_engine_stats lr_stateful norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_pb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11413,6 +11473,7 @@ check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb lr-policy-add lr0 10 "ip4.src == 10.0.0.3" reroute 172.168.0.101,172.168.0.102 check_engine_stats northd recompute nocompute check_engine_stats lr_nat recompute nocompute +check_engine_stats lr_stateful recompute nocompute check_engine_stats sync_to_sb_pb recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11421,6 +11482,7 @@ check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb lr-policy-del lr0 10 "ip4.src == 10.0.0.3" check_engine_stats northd recompute nocompute check_engine_stats lr_nat recompute nocompute +check_engine_stats lr_stateful recompute nocompute check_engine_stats sync_to_sb_pb recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE From patchwork Thu Jan 11 15:29:44 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Numan Siddique X-Patchwork-Id: 1885675 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::138; helo=smtp1.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp1.osuosl.org (smtp1.osuosl.org [IPv6:2605:bc80:3010::138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4T9pXm2PnTz1yPp for ; Fri, 12 Jan 2024 02:31:04 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 38FD184023; Thu, 11 Jan 2024 15:31:02 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 38FD184023 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id qIwmkOKJaSxh; Thu, 11 Jan 2024 15:30:59 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp1.osuosl.org (Postfix) with ESMTPS id EED1983A4F; Thu, 11 Jan 2024 15:30:58 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org EED1983A4F Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id C7D2DC0077; Thu, 11 Jan 2024 15:30:58 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by lists.linuxfoundation.org (Postfix) with ESMTP id 1518BC0DD3 for ; Thu, 11 Jan 2024 15:30:58 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id D520A84019 for ; Thu, 11 Jan 2024 15:30:32 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org D520A84019 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id brSAekSixj59 for ; Thu, 11 Jan 2024 15:30:30 +0000 (UTC) Received: from relay5-d.mail.gandi.net (relay5-d.mail.gandi.net [217.70.183.197]) by smtp1.osuosl.org (Postfix) with ESMTPS id DDACE84042 for ; Thu, 11 Jan 2024 15:30:29 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org DDACE84042 Received: by mail.gandi.net (Postfix) with ESMTPSA id 53F4E1C000C; Thu, 11 Jan 2024 15:30:27 +0000 (UTC) From: numans@ovn.org To: dev@openvswitch.org Date: Thu, 11 Jan 2024 10:29:44 -0500 Message-ID: <20240111152944.2790086-1-numans@ovn.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240111152752.2789854-1-numans@ovn.org> References: <20240111152752.2789854-1-numans@ovn.org> MIME-Version: 1.0 X-GND-Sasl: numans@ovn.org Subject: [ovs-dev] [PATCH ovn v5 06/16] northd: Generate router's stateful flows using lr_stateful data. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Numan Siddique Previous commits added new engine nodes to store logical router's stateful (LB and NAT data). Make use of the data stored by these engine nodes to generate logical flows related to router's LBs and NATs. Signed-off-by: Numan Siddique Acked-by: Dumitru Ceara --- northd/en-lflow.c | 3 - northd/en-lr-stateful.h | 4 + northd/inc-proc-northd.c | 1 - northd/northd.c | 833 +++++++++++++++++++++++++-------------- northd/northd.h | 1 - 5 files changed, 544 insertions(+), 298 deletions(-) diff --git a/northd/en-lflow.c b/northd/en-lflow.c index bb9f78a6a1..a3a0d62f15 100644 --- a/northd/en-lflow.c +++ b/northd/en-lflow.c @@ -42,8 +42,6 @@ lflow_get_input_data(struct engine_node *node, engine_get_input_data("port_group", node); struct sync_meters_data *sync_meters_data = engine_get_input_data("sync_meters", node); - struct ed_type_lr_nat_data *lr_nat_data = - engine_get_input_data("lr_nat", node); struct ed_type_lr_stateful *lr_stateful_data = engine_get_input_data("lr_stateful", node); @@ -68,7 +66,6 @@ lflow_get_input_data(struct engine_node *node, lflow_input->ls_ports = &northd_data->ls_ports; lflow_input->lr_ports = &northd_data->lr_ports; lflow_input->ls_port_groups = &pg_data->ls_port_groups; - lflow_input->lr_nats = &lr_nat_data->lr_nats; lflow_input->lr_stateful_table = &lr_stateful_data->table; lflow_input->meter_groups = &sync_meters_data->meter_groups; lflow_input->lb_datapaths_map = &northd_data->lb_datapaths_map; diff --git a/northd/en-lr-stateful.h b/northd/en-lr-stateful.h index 0bc1f4ee75..74029c9a6c 100644 --- a/northd/en-lr-stateful.h +++ b/northd/en-lr-stateful.h @@ -66,6 +66,10 @@ struct lr_stateful_table { #define LR_STATEFUL_TABLE_FOR_EACH(LR_LB_NAT_REC, TABLE) \ HMAP_FOR_EACH (LR_LB_NAT_REC, key_node, &(TABLE)->entries) +#define LR_STATEFUL_TABLE_FOR_EACH_IN_P(LR_STATEFUL_REC, JOBID, TABLE) \ + HMAP_FOR_EACH_IN_PARALLEL (LR_STATEFUL_REC, key_node, JOBID, \ + &(TABLE)->entries) + struct lr_stateful_tracked_data { /* Created or updated logical router with LB and/or NAT data. */ struct hmapx crupdated; /* Stores 'struct lr_stateful_record'. */ diff --git a/northd/inc-proc-northd.c b/northd/inc-proc-northd.c index 97bcce9655..adb38dde78 100644 --- a/northd/inc-proc-northd.c +++ b/northd/inc-proc-northd.c @@ -221,7 +221,6 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, engine_add_input(&en_lflow, &en_sb_logical_flow, NULL); engine_add_input(&en_lflow, &en_sb_multicast_group, NULL); engine_add_input(&en_lflow, &en_sb_igmp_group, NULL); - engine_add_input(&en_lflow, &en_lr_nat, NULL); engine_add_input(&en_lflow, &en_lr_stateful, NULL); engine_add_input(&en_lflow, &en_northd, lflow_northd_handler); engine_add_input(&en_lflow, &en_port_group, lflow_port_group_handler); diff --git a/northd/northd.c b/northd/northd.c index fc2fc835b2..f56a7c5ea4 100644 --- a/northd/northd.c +++ b/northd/northd.c @@ -8859,18 +8859,14 @@ build_lrouter_groups(struct hmap *lr_ports, struct ovs_list *lr_list) */ static void build_lswitch_rport_arp_req_self_orig_flow(struct ovn_port *op, - uint32_t priority, - struct ovn_datapath *od, - const struct lr_nat_table *lr_nats, - struct hmap *lflows) + uint32_t priority, + const struct ovn_datapath *od, + const struct lr_nat_record *lrnat_rec, + struct hmap *lflows) { struct ds eth_src = DS_EMPTY_INITIALIZER; struct ds match = DS_EMPTY_INITIALIZER; - const struct lr_nat_record *lrnat_rec = lr_nat_table_find_by_index( - lr_nats, op->od->index); - ovs_assert(lrnat_rec); - /* Self originated ARP requests/RARP/ND need to be flooded to the L2 domain * (except on router ports). Determine that packets are self originated * by also matching on source MAC. Matching on ingress port is not @@ -8956,10 +8952,11 @@ lrouter_port_ipv6_reachable(const struct ovn_port *op, * switching domain as regular broadcast. */ static void -build_lswitch_rport_arp_req_flow(const char *ips, - int addr_family, struct ovn_port *patch_op, struct ovn_datapath *od, - uint32_t priority, struct hmap *lflows, - const struct ovsdb_idl_row *stage_hint) +build_lswitch_rport_arp_req_flow(const char *ips, int addr_family, + struct ovn_port *patch_op, + const struct ovn_datapath *od, + uint32_t priority, struct hmap *lflows, + const struct ovsdb_idl_row *stage_hint) { struct ds match = DS_EMPTY_INITIALIZER; struct ds actions = DS_EMPTY_INITIALIZER; @@ -8995,10 +8992,47 @@ build_lswitch_rport_arp_req_flow(const char *ips, * - 75: ARP requests to router owned IPs (interface IP/LB/NAT). */ static void -build_lswitch_rport_arp_req_flows( - struct ovn_port *op, struct ovn_datapath *sw_od, - struct ovn_port *sw_op, const struct lr_nat_table *lr_nats, - const struct lr_stateful_table *lr_stateful_table, +build_lswitch_rport_arp_req_flows(struct ovn_port *op, + struct ovn_datapath *sw_od, + struct ovn_port *sw_op, + struct hmap *lflows, + const struct ovsdb_idl_row *stage_hint) +{ + if (!op || !op->nbrp) { + return; + } + + if (!lrport_is_enabled(op->nbrp)) { + return; + } + + /* Forward ARP requests for owned IP addresses (L3, VIP, NAT) only to this + * router port. + * Priority: 80. + */ + for (size_t i = 0; i < op->lrp_networks.n_ipv4_addrs; i++) { + build_lswitch_rport_arp_req_flow( + op->lrp_networks.ipv4_addrs[i].addr_s, AF_INET, sw_op, sw_od, 80, + lflows, stage_hint); + } + for (size_t i = 0; i < op->lrp_networks.n_ipv6_addrs; i++) { + build_lswitch_rport_arp_req_flow( + op->lrp_networks.ipv6_addrs[i].addr_s, AF_INET6, sw_op, sw_od, 80, + lflows, stage_hint); + } +} + +/* + * Ingress table 25: Flows that forward ARP/ND requests only to the routers + * that own the addresses. + * Priorities: + * - 80: self originated GARPs that need to follow regular processing. + * - 75: ARP requests to router owned IPs (interface IP/LB/NAT). + */ +static void +build_lswitch_rport_arp_req_flows_for_lbnats( + struct ovn_port *op, const struct lr_stateful_record *lr_stateful_rec, + const struct ovn_datapath *sw_od, struct ovn_port *sw_op, struct hmap *lflows, const struct ovsdb_idl_row *stage_hint) { if (!op || !op->nbrp) { @@ -9009,16 +9043,13 @@ build_lswitch_rport_arp_req_flows( return; } + ovs_assert(op->od == lr_stateful_rec->od); + /* Forward ARP requests for owned IP addresses (L3, VIP, NAT) only to this * router port. * Priority: 80. */ - const struct lr_stateful_record *lr_stateful_rec = NULL; if (op->od->nbr->n_load_balancer || op->od->nbr->n_load_balancer_group) { - lr_stateful_rec = lr_stateful_table_find_by_index(lr_stateful_table, - op->od->index); - ovs_assert(lr_stateful_rec); - const char *ip_addr; SSET_FOR_EACH (ip_addr, &lr_stateful_rec->lb_ips->ips_v4_reachable) { ovs_be32 ipv4_addr; @@ -9048,17 +9079,6 @@ build_lswitch_rport_arp_req_flows( } } - for (size_t i = 0; i < op->lrp_networks.n_ipv4_addrs; i++) { - build_lswitch_rport_arp_req_flow( - op->lrp_networks.ipv4_addrs[i].addr_s, AF_INET, sw_op, sw_od, 80, - lflows, stage_hint); - } - for (size_t i = 0; i < op->lrp_networks.n_ipv6_addrs; i++) { - build_lswitch_rport_arp_req_flow( - op->lrp_networks.ipv6_addrs[i].addr_s, AF_INET6, sw_op, sw_od, 80, - lflows, stage_hint); - } - /* Self originated ARP requests/RARP/ND need to be flooded as usual. * * However, if the switch doesn't have any non-router ports we shouldn't @@ -9067,19 +9087,14 @@ build_lswitch_rport_arp_req_flows( * Priority: 75. */ if (sw_od->n_router_ports != sw_od->nbs->n_ports) { - build_lswitch_rport_arp_req_self_orig_flow(op, 75, sw_od, lr_nats, + build_lswitch_rport_arp_req_self_orig_flow(op, 75, sw_od, + lr_stateful_rec->lrnat_rec, lflows); } - const struct lr_nat_record *lrnat_rec = - lr_nat_table_find_by_index(lr_nats, op->od->index); - - if (!lrnat_rec) { - return; - } - - for (size_t i = 0; i < lrnat_rec->n_nat_entries; i++) { - struct ovn_nat *nat_entry = &lrnat_rec->nat_entries[i]; + for (size_t i = 0; i < lr_stateful_rec->lrnat_rec->n_nat_entries; i++) { + struct ovn_nat *nat_entry = + &lr_stateful_rec->lrnat_rec->nat_entries[i]; const struct nbrec_nat *nat = nat_entry->nb; if (!nat_entry_is_valid(nat_entry)) { @@ -9094,16 +9109,14 @@ build_lswitch_rport_arp_req_flows( * expect ARP requests/NS for the DNAT external_ip. */ if (nat_entry_is_v6(nat_entry)) { - if (!lr_stateful_rec || - !sset_contains(&lr_stateful_rec->lb_ips->ips_v6, + if (!sset_contains(&lr_stateful_rec->lb_ips->ips_v6, nat->external_ip)) { build_lswitch_rport_arp_req_flow( nat->external_ip, AF_INET6, sw_op, sw_od, 80, lflows, stage_hint); } } else { - if (!lr_stateful_rec || - !sset_contains(&lr_stateful_rec->lb_ips->ips_v4, + if (!sset_contains(&lr_stateful_rec->lb_ips->ips_v4, nat->external_ip)) { build_lswitch_rport_arp_req_flow( nat->external_ip, AF_INET, sw_op, sw_od, 80, lflows, @@ -9113,7 +9126,7 @@ build_lswitch_rport_arp_req_flows( } struct shash_node *snat_snode; - SHASH_FOR_EACH (snat_snode, &lrnat_rec->snat_ips) { + SHASH_FOR_EACH (snat_snode, &lr_stateful_rec->lrnat_rec->snat_ips) { struct ovn_snat_ip *snat_ip = snat_snode->data; if (ovs_list_is_empty(&snat_ip->snat_entries)) { @@ -10178,11 +10191,8 @@ build_lswitch_ip_mcast_igmp_mld(struct ovn_igmp_group *igmp_group, /* Ingress table 25: Destination lookup, unicast handling (priority 50), */ static void -build_lswitch_ip_unicast_lookup( - struct ovn_port *op, const struct lr_nat_table *lr_nats, - const struct lr_stateful_table *lr_stateful_table, - struct hmap *lflows, struct ds *actions, - struct ds *match) +build_lswitch_ip_unicast_lookup(struct ovn_port *op, struct hmap *lflows, + struct ds *actions, struct ds *match) { ovs_assert(op->nbsp); if (lsp_is_external(op->nbsp)) { @@ -10194,8 +10204,7 @@ build_lswitch_ip_unicast_lookup( * requests only to the router port that owns the IP address. */ if (lsp_is_router(op->nbsp)) { - build_lswitch_rport_arp_req_flows(op->peer, op->od, op, lr_nats, - lr_stateful_table, lflows, + build_lswitch_rport_arp_req_flows(op->peer, op->od, op, lflows, &op->nbsp->header_); } @@ -10292,33 +10301,6 @@ build_lswitch_ip_unicast_lookup( S_SWITCH_IN_L2_LKUP, 50, ds_cstr(match), ds_cstr(actions), &op->nbsp->header_); - - /* Add ethernet addresses specified in NAT rules on - * distributed logical routers. */ - if (is_l3dgw_port(op->peer)) { - for (int j = 0; j < op->peer->od->nbr->n_nat; j++) { - const struct nbrec_nat *nat - = op->peer->od->nbr->nat[j]; - if (!strcmp(nat->type, "dnat_and_snat") - && nat->logical_port && nat->external_mac - && eth_addr_from_string(nat->external_mac, &mac)) { - - ds_clear(match); - ds_put_format(match, "eth.dst == "ETH_ADDR_FMT - " && is_chassis_resident(\"%s\")", - ETH_ADDR_ARGS(mac), - nat->logical_port); - - ds_clear(actions); - ds_put_format(actions, action, op->json_key); - ovn_lflow_add_with_hint(lflows, op->od, - S_SWITCH_IN_L2_LKUP, 50, - ds_cstr(match), - ds_cstr(actions), - &op->nbsp->header_); - } - } - } } else { static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 1); @@ -10329,6 +10311,52 @@ build_lswitch_ip_unicast_lookup( } } +/* Ingress table 25: Destination lookup, unicast handling (priority 50), */ +static void +build_lswitch_ip_unicast_lookup_for_nats( + struct ovn_port *op, const struct lr_stateful_record *lr_stateful_rec, + struct hmap *lflows, struct ds *match, struct ds *actions) +{ + ovs_assert(op->nbsp); + + if (!op->peer || !is_l3dgw_port(op->peer)) { + return; + } + + ovs_assert(op->peer->od == lr_stateful_rec->od); + + const char *action = lsp_is_enabled(op->nbsp) ? + "outport = %s; output;" : + debug_drop_action(); + struct eth_addr mac; + + /* Add ethernet addresses specified in NAT rules on + * distributed logical routers. */ + for (size_t i = 0; i < lr_stateful_rec->lrnat_rec->n_nat_entries; i++) { + const struct ovn_nat *nat = + &lr_stateful_rec->lrnat_rec->nat_entries[i]; + + if (!strcmp(nat->nb->type, "dnat_and_snat") + && nat->nb->logical_port && nat->nb->external_mac + && eth_addr_from_string(nat->nb->external_mac, &mac)) { + + ds_clear(match); + ds_put_format(match, "eth.dst == "ETH_ADDR_FMT + " && is_chassis_resident(\"%s\")", + ETH_ADDR_ARGS(mac), + nat->nb->logical_port); + + ds_clear(actions); + ds_put_format(actions, action, op->json_key); + ovn_lflow_add_with_hint(lflows, op->od, + S_SWITCH_IN_L2_LKUP, 50, + ds_cstr(match), + ds_cstr(actions), + &op->nbsp->header_); + } + } +} + struct bfd_entry { struct hmap_node hmap_node; @@ -11706,16 +11734,17 @@ build_gw_lrouter_nat_flows_for_lb(struct lrouter_nat_lb_flows_ctx *ctx, } static void -build_lrouter_nat_flows_for_lb(struct ovn_lb_vip *lb_vip, - struct ovn_lb_datapaths *lb_dps, - struct ovn_northd_lb_vip *vips_nb, - const struct ovn_datapaths *lr_datapaths, - const struct lr_nat_table *lr_nats, - struct hmap *lflows, - struct ds *match, struct ds *action, - const struct shash *meter_groups, - const struct chassis_features *features, - const struct hmap *svc_monitor_map) +build_lrouter_nat_flows_for_lb( + struct ovn_lb_vip *lb_vip, + struct ovn_lb_datapaths *lb_dps, + struct ovn_northd_lb_vip *vips_nb, + const struct ovn_datapaths *lr_datapaths, + const struct lr_stateful_table *lr_stateful_table, + struct hmap *lflows, + struct ds *match, struct ds *action, + const struct shash *meter_groups, + const struct chassis_features *features, + const struct hmap *svc_monitor_map) { const struct ovn_northd_lb *lb = lb_dps->lb; bool ipv4 = lb_vip->address_family == AF_INET; @@ -11816,9 +11845,11 @@ build_lrouter_nat_flows_for_lb(struct ovn_lb_vip *lb_vip, struct ovn_datapath *od = lr_datapaths->array[index]; enum lrouter_nat_lb_flow_type type; - const struct lr_nat_record *lrnat_rec = - lr_nat_table_find_by_index(lr_nats, od->index); - ovs_assert(lrnat_rec); + const struct lr_stateful_record *lr_stateful_rec = + lr_stateful_table_find_by_index(lr_stateful_table, od->index); + ovs_assert(lr_stateful_rec); + + const struct lr_nat_record *lrnat_rec = lr_stateful_rec->lrnat_rec; if (lb->skip_snat) { type = LROUTER_NAT_LB_FLOW_SKIP_SNAT; } else if (!lport_addresses_is_empty(&lrnat_rec->lb_force_snat_addrs) @@ -11968,7 +11999,7 @@ build_lrouter_flows_for_lb(struct ovn_lb_datapaths *lb_dps, struct hmap *lflows, const struct shash *meter_groups, const struct ovn_datapaths *lr_datapaths, - const struct lr_nat_table *lr_nats, + const struct lr_stateful_table *lr_stateful_table, const struct chassis_features *features, const struct hmap *svc_monitor_map, struct ds *match, struct ds *action) @@ -11984,8 +12015,8 @@ build_lrouter_flows_for_lb(struct ovn_lb_datapaths *lb_dps, struct ovn_lb_vip *lb_vip = &lb->vips[i]; build_lrouter_nat_flows_for_lb(lb_vip, lb_dps, &lb->vips_nb[i], - lr_datapaths, lr_nats, lflows, match, - action, meter_groups, features, + lr_datapaths, lr_stateful_table, lflows, + match, action, meter_groups, features, svc_monitor_map); if (!build_empty_lb_event_flow(lb_vip, lb, match, action)) { @@ -12122,7 +12153,7 @@ lrouter_dnat_and_snat_is_stateless(const struct nbrec_nat *nat) * and action says "next" instead of ct*. */ static inline void -lrouter_nat_add_ext_ip_match(struct ovn_datapath *od, +lrouter_nat_add_ext_ip_match(const struct ovn_datapath *od, struct hmap *lflows, struct ds *match, const struct nbrec_nat *nat, bool is_v6, bool is_src, int cidr_bits) @@ -12186,7 +12217,7 @@ lrouter_nat_add_ext_ip_match(struct ovn_datapath *od, * with the given priority. */ static void -build_lrouter_arp_flow(struct ovn_datapath *od, struct ovn_port *op, +build_lrouter_arp_flow(const struct ovn_datapath *od, struct ovn_port *op, const char *ip_address, const char *eth_addr, struct ds *extra_match, bool drop, uint16_t priority, const struct ovsdb_idl_row *hint, @@ -12235,7 +12266,7 @@ build_lrouter_arp_flow(struct ovn_datapath *od, struct ovn_port *op, * 'sn_ip_address'. */ static void -build_lrouter_nd_flow(struct ovn_datapath *od, struct ovn_port *op, +build_lrouter_nd_flow(const struct ovn_datapath *od, struct ovn_port *op, const char *action, const char *ip_address, const char *sn_ip_address, const char *eth_addr, struct ds *extra_match, bool drop, uint16_t priority, @@ -12289,7 +12320,7 @@ build_lrouter_nd_flow(struct ovn_datapath *od, struct ovn_port *op, } static void -build_lrouter_nat_arp_nd_flow(struct ovn_datapath *od, +build_lrouter_nat_arp_nd_flow(const struct ovn_datapath *od, struct ovn_nat *nat_entry, struct hmap *lflows, const struct shash *meter_groups) @@ -12385,7 +12416,6 @@ build_lrouter_port_nat_arp_nd_flow(struct ovn_port *op, static void build_lrouter_drop_own_dest(struct ovn_port *op, - const struct lr_nat_record *lrnat_rec, const struct lr_stateful_record *lr_stateful_rec, enum ovn_stage stage, uint16_t priority, bool drop_snat_ip, @@ -12397,10 +12427,10 @@ build_lrouter_drop_own_dest(struct ovn_port *op, for (size_t i = 0; i < op->lrp_networks.n_ipv4_addrs; i++) { const char *ip = op->lrp_networks.ipv4_addrs[i].addr_s; - bool router_ip_in_snat_ips = !!shash_find(&lrnat_rec->snat_ips, - ip); - bool router_ip_in_lb_ips = (lr_stateful_rec && - !!sset_find(&lr_stateful_rec->lb_ips->ips_v4, ip)); + bool router_ip_in_snat_ips = + !!shash_find(&lr_stateful_rec->lrnat_rec->snat_ips, ip); + bool router_ip_in_lb_ips = + !!sset_find(&lr_stateful_rec->lb_ips->ips_v4, ip); bool drop_router_ip = (drop_snat_ip == (router_ip_in_snat_ips || router_ip_in_lb_ips)); @@ -12427,10 +12457,10 @@ build_lrouter_drop_own_dest(struct ovn_port *op, for (size_t i = 0; i < op->lrp_networks.n_ipv6_addrs; i++) { const char *ip = op->lrp_networks.ipv6_addrs[i].addr_s; - bool router_ip_in_snat_ips = !!shash_find(&lrnat_rec->snat_ips, - ip); - bool router_ip_in_lb_ips = (lr_stateful_rec && - !!sset_find(&lr_stateful_rec->lb_ips->ips_v6, ip)); + bool router_ip_in_snat_ips = + !!shash_find(&lr_stateful_rec->lrnat_rec->snat_ips, ip); + bool router_ip_in_lb_ips = + !!sset_find(&lr_stateful_rec->lb_ips->ips_v6, ip); bool drop_router_ip = (drop_snat_ip == (router_ip_in_snat_ips || router_ip_in_lb_ips)); @@ -12454,7 +12484,8 @@ build_lrouter_drop_own_dest(struct ovn_port *op, } static void -build_lrouter_force_snat_flows(struct hmap *lflows, struct ovn_datapath *od, +build_lrouter_force_snat_flows(struct hmap *lflows, + const struct ovn_datapath *od, const char *ip_version, const char *ip_addr, const char *context) { @@ -13453,10 +13484,8 @@ routable_addresses_to_lflows(struct hmap *lflows, struct ovn_port *router_port, /* This function adds ARP resolve flows related to a LRP. */ static void -build_arp_resolve_flows_for_lrp( - struct ovn_port *op, const struct lr_nat_record *lrnat_rec, - const struct lr_stateful_record *lr_stateful_rec, - struct hmap *lflows, struct ds *match, struct ds *actions) +build_arp_resolve_flows_for_lrp(struct ovn_port *op, struct hmap *lflows, + struct ds *match, struct ds *actions) { ovs_assert(op->nbrp); /* This is a logical router port. If next-hop IP address in @@ -13525,15 +13554,6 @@ build_arp_resolve_flows_for_lrp( &op->nbrp->header_); } } - - /* Drop IP traffic destined to router owned IPs. Part of it is dropped - * in stage "lr_in_ip_input" but traffic that could have been unSNATed - * but didn't match any existing session might still end up here. - * - * Priority 2. - */ - build_lrouter_drop_own_dest(op, lrnat_rec, lr_stateful_rec, - S_ROUTER_IN_ARP_RESOLVE, 2, true, lflows); } /* This function adds ARP resolve flows related to a LSP. */ @@ -13541,7 +13561,6 @@ static void build_arp_resolve_flows_for_lsp( struct ovn_port *op, struct hmap *lflows, const struct hmap *lr_ports, - const struct lr_stateful_table *lr_stateful_table, struct ds *match, struct ds *actions) { ovs_assert(op->nbsp); @@ -13682,15 +13701,50 @@ build_arp_resolve_flows_for_lsp( ds_cstr(match), ds_cstr(actions), &op->nbsp->header_); } + } + } +} - if (smap_get(&peer->od->nbr->options, "chassis") - || peer->cr_port) { - const struct lr_stateful_record *lr_stateful_rec = - lr_stateful_table_find_by_index(lr_stateful_table, - router_port->od->index); - routable_addresses_to_lflows(lflows, router_port, peer, - lr_stateful_rec, match, actions); - } +static void +build_arp_resolve_flows_for_lsp_routable_addresses( + struct ovn_port *op, struct hmap *lflows, + const struct hmap *lr_ports, + const struct lr_stateful_table *lr_stateful_table, + struct ds *match, struct ds *actions) +{ + if (!lsp_is_router(op->nbsp)) { + return; + } + + struct ovn_port *peer = ovn_port_get_peer(lr_ports, op); + if (!peer || !peer->nbrp) { + return; + } + + if (peer->od->nbr && + smap_get_bool(&peer->od->nbr->options, + "dynamic_neigh_routers", false)) { + return; + } + + for (size_t i = 0; i < op->od->n_router_ports; i++) { + struct ovn_port *router_port = + ovn_port_get_peer(lr_ports, op->od->router_ports[i]); + if (!router_port || !router_port->nbrp) { + continue; + } + + /* Skip the router port under consideration. */ + if (router_port == peer) { + continue; + } + + if (smap_get(&peer->od->nbr->options, "chassis") || peer->cr_port) { + const struct lr_stateful_record *lr_stateful_rec; + lr_stateful_rec = lr_stateful_table_find_by_index( + lr_stateful_table, router_port->od->index); + routable_addresses_to_lflows(lflows, router_port, peer, + lr_stateful_rec, match, actions); } } } @@ -13867,7 +13921,6 @@ build_check_pkt_len_flows_for_lrouter( static void build_gateway_redirect_flows_for_lrouter( struct ovn_datapath *od, struct hmap *lflows, - const struct lr_nat_table *lr_nats, struct ds *match, struct ds *actions) { ovs_assert(od->nbr); @@ -13884,7 +13937,6 @@ build_gateway_redirect_flows_for_lrouter( } const struct ovsdb_idl_row *stage_hint = NULL; - bool add_def_flow = true; if (od->l3dgw_ports[i]->nbrp) { stage_hint = &od->l3dgw_ports[i]->nbrp->header_; @@ -13903,14 +13955,33 @@ build_gateway_redirect_flows_for_lrouter( ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_GW_REDIRECT, 50, ds_cstr(match), ds_cstr(actions), stage_hint); + } - const struct lr_nat_record *lrnat_rec = lr_nat_table_find_by_index( - lr_nats, od->index); + /* Packets are allowed by default. */ + ovn_lflow_add(lflows, od, S_ROUTER_IN_GW_REDIRECT, 0, "1", "next;"); +} - if (!lrnat_rec) { +/* Logical router ingress table GW_REDIRECT: Gateway redirect. */ +static void +build_lr_gateway_redirect_flows_for_nats( + const struct ovn_datapath *od, const struct lr_nat_record *lrnat_rec, + struct hmap *lflows, struct ds *match, struct ds *actions) +{ + ovs_assert(od->nbr); + for (size_t i = 0; i < od->n_l3dgw_ports; i++) { + if (l3dgw_port_has_associated_vtep_lports(od->l3dgw_ports[i])) { + /* Skip adding redirect lflow for vtep-enabled l3dgw ports. + * Traffic from hypervisor to VTEP (ramp) switch should go in + * distributed manner. Only returning routed traffic must go + * through centralized gateway (or ha-chassis-group). + * This assumes that attached logical switch with vtep lport(s) has + * no localnet port(s) for NAT. Otherwise centralized NAT will not + * work. */ continue; } + bool add_def_flow = true; + for (int j = 0; j < lrnat_rec->n_nat_entries; j++) { const struct ovn_nat *nat = &lrnat_rec->nat_entries[j]; @@ -13919,6 +13990,12 @@ build_gateway_redirect_flows_for_lrouter( continue; } + const struct ovsdb_idl_row *stage_hint = NULL; + + if (od->l3dgw_ports[i]->nbrp) { + stage_hint = &od->l3dgw_ports[i]->nbrp->header_; + } + struct ds match_ext = DS_EMPTY_INITIALIZER; struct nbrec_address_set *as = nat->nb->allowed_ext_ips ? nat->nb->allowed_ext_ips : nat->nb->exempted_ext_ips; @@ -13948,9 +14025,6 @@ build_gateway_redirect_flows_for_lrouter( ds_destroy(&match_ext); } } - - /* Packets are allowed by default. */ - ovn_lflow_add(lflows, od, S_ROUTER_IN_GW_REDIRECT, 0, "1", "next;"); } /* Local router ingress table ARP_REQUEST: ARP request. @@ -14349,8 +14423,8 @@ build_ipv6_input_flows_for_lrouter_port( } static void -build_lrouter_arp_nd_for_datapath(struct ovn_datapath *od, - const struct lr_nat_table *lr_nats, +build_lrouter_arp_nd_for_datapath(const struct ovn_datapath *od, + const struct lr_nat_record *lrnat_rec, struct hmap *lflows, const struct shash *meter_groups) { @@ -14367,10 +14441,6 @@ build_lrouter_arp_nd_for_datapath(struct ovn_datapath *od, * port to handle the special cases. In case we get the packet * on a regular port, just reply with the port's ETH address. */ - const struct lr_nat_record *lrnat_rec = lr_nat_table_find_by_index( - lr_nats, od->index); - ovs_assert(lrnat_rec); - for (int i = 0; i < lrnat_rec->n_nat_entries; i++) { struct ovn_nat *nat_entry = &lrnat_rec->nat_entries[i]; @@ -14408,8 +14478,6 @@ build_lrouter_arp_nd_for_datapath(struct ovn_datapath *od, static void build_lrouter_ipv4_ip_input(struct ovn_port *op, struct hmap *lflows, - const struct lr_nat_record *lrnat_rec, - const struct lr_stateful_record *lr_stateful_rec, struct ds *match, struct ds *actions, const struct shash *meter_groups) { @@ -14534,41 +14602,6 @@ build_lrouter_ipv4_ip_input(struct ovn_port *op, &op->nbrp->header_, lflows); } - if (lr_stateful_rec && sset_count( - &lr_stateful_rec->lb_ips->ips_v4_reachable)) { - ds_clear(match); - if (is_l3dgw_port(op)) { - ds_put_format(match, "is_chassis_resident(%s)", - op->cr_port->json_key); - } - - /* Create a single ARP rule for all IPs that are used as VIPs. */ - char *lb_ips_v4_as = lr_lb_address_set_ref(op->od->tunnel_key, - AF_INET); - build_lrouter_arp_flow(op->od, op, lb_ips_v4_as, - REG_INPORT_ETH_ADDR, - match, false, 90, NULL, lflows); - free(lb_ips_v4_as); - } - - if (lr_stateful_rec && sset_count( - &lr_stateful_rec->lb_ips->ips_v6_reachable)) { - ds_clear(match); - - if (is_l3dgw_port(op)) { - ds_put_format(match, "is_chassis_resident(%s)", - op->cr_port->json_key); - } - - /* Create a single ND rule for all IPs that are used as VIPs. */ - char *lb_ips_v6_as = lr_lb_address_set_ref(op->od->tunnel_key, - AF_INET6); - build_lrouter_nd_flow(op->od, op, "nd_na", lb_ips_v6_as, NULL, - REG_INPORT_ETH_ADDR, match, false, 90, - NULL, lflows, meter_groups); - free(lb_ips_v6_as); - } - if (!op->od->is_gw_router && !op->od->n_l3dgw_ports) { /* UDP/TCP/SCTP port unreachable. */ for (int i = 0; i < op->lrp_networks.n_ipv4_addrs; i++) { @@ -14643,20 +14676,55 @@ build_lrouter_ipv4_ip_input(struct ovn_port *op, &op->nbrp->header_); } } +} - /* Drop IP traffic destined to router owned IPs except if the IP is - * also a SNAT IP. Those are dropped later, in stage - * "lr_in_arp_resolve", if unSNAT was unsuccessful. - * - * If lrnat_rec->lb_force_snat_router_ip is true, it means the IP of the - * router port is also SNAT IP. - * - * Priority 60. - */ - if (!lrnat_rec->lb_force_snat_router_ip) { - build_lrouter_drop_own_dest(op, lrnat_rec, lr_stateful_rec, - S_ROUTER_IN_IP_INPUT, 60, false, lflows); +/* Logical router ingress table 3: IP Input for IPv4. */ +static void +build_lrouter_ipv4_ip_input_for_lbnats( + struct ovn_port *op, struct hmap *lflows, + const struct lr_stateful_record *lr_stateful_rec, + struct ds *match, const struct shash *meter_groups) +{ + ovs_assert(op->nbrp); + /* No ingress packets are accepted on a chassisredirect + * port, so no need to program flows for that port. */ + if (is_cr_port(op)) { + return; + } + + if (sset_count(&lr_stateful_rec->lb_ips->ips_v4_reachable)) { + ds_clear(match); + if (is_l3dgw_port(op)) { + ds_put_format(match, "is_chassis_resident(%s)", + op->cr_port->json_key); + } + + /* Create a single ARP rule for all IPs that are used as VIPs. */ + char *lb_ips_v4_as = lr_lb_address_set_ref(op->od->tunnel_key, + AF_INET); + build_lrouter_arp_flow(op->od, op, lb_ips_v4_as, + REG_INPORT_ETH_ADDR, + match, false, 90, NULL, lflows); + free(lb_ips_v4_as); } + + if (sset_count(&lr_stateful_rec->lb_ips->ips_v6_reachable)) { + ds_clear(match); + + if (is_l3dgw_port(op)) { + ds_put_format(match, "is_chassis_resident(%s)", + op->cr_port->json_key); + } + + /* Create a single ND rule for all IPs that are used as VIPs. */ + char *lb_ips_v6_as = lr_lb_address_set_ref(op->od->tunnel_key, + AF_INET6); + build_lrouter_nd_flow(op->od, op, "nd_na", lb_ips_v6_as, NULL, + REG_INPORT_ETH_ADDR, match, false, 90, + NULL, lflows, meter_groups); + free(lb_ips_v6_as); + } + /* ARP / ND handling for external IP addresses. * * DNAT and SNAT IP addresses are external IP addresses that need ARP @@ -14670,8 +14738,9 @@ build_lrouter_ipv4_ip_input(struct ovn_port *op, return; } - for (int i = 0; i < lrnat_rec->n_nat_entries; i++) { - struct ovn_nat *nat_entry = &lrnat_rec->nat_entries[i]; + for (int i = 0; i < lr_stateful_rec->lrnat_rec->n_nat_entries; i++) { + struct ovn_nat *nat_entry = + &lr_stateful_rec->lrnat_rec->nat_entries[i]; /* Skip entries we failed to parse. */ if (!nat_entry_is_valid(nat_entry)) { @@ -14690,7 +14759,7 @@ build_lrouter_ipv4_ip_input(struct ovn_port *op, /* Now handle SNAT entries too, one per unique SNAT IP. */ struct shash_node *snat_snode; - SHASH_FOR_EACH (snat_snode, &lrnat_rec->snat_ips) { + SHASH_FOR_EACH (snat_snode, &lr_stateful_rec->lrnat_rec->snat_ips) { struct ovn_snat_ip *snat_ip = snat_snode->data; if (ovs_list_is_empty(&snat_ip->snat_entries)) { @@ -14706,7 +14775,7 @@ build_lrouter_ipv4_ip_input(struct ovn_port *op, } static void -build_lrouter_in_unsnat_match(struct ovn_datapath *od, +build_lrouter_in_unsnat_match(const struct ovn_datapath *od, const struct nbrec_nat *nat, struct ds *match, bool distributed_nat, bool is_v6, struct ovn_port *l3dgw_port) @@ -14733,7 +14802,7 @@ build_lrouter_in_unsnat_match(struct ovn_datapath *od, static void build_lrouter_in_unsnat_stateless_flow(struct hmap *lflows, - struct ovn_datapath *od, + const struct ovn_datapath *od, const struct nbrec_nat *nat, struct ds *match, bool distributed_nat, bool is_v6, @@ -14755,7 +14824,7 @@ build_lrouter_in_unsnat_stateless_flow(struct hmap *lflows, static void build_lrouter_in_unsnat_in_czone_flow(struct hmap *lflows, - struct ovn_datapath *od, + const struct ovn_datapath *od, const struct nbrec_nat *nat, struct ds *match, bool distributed_nat, bool is_v6, struct ovn_port *l3dgw_port) @@ -14788,7 +14857,8 @@ build_lrouter_in_unsnat_in_czone_flow(struct hmap *lflows, } static void -build_lrouter_in_unsnat_flow(struct hmap *lflows, struct ovn_datapath *od, +build_lrouter_in_unsnat_flow(struct hmap *lflows, + const struct ovn_datapath *od, const struct nbrec_nat *nat, struct ds *match, bool distributed_nat, bool is_v6, struct ovn_port *l3dgw_port) @@ -14809,7 +14879,8 @@ build_lrouter_in_unsnat_flow(struct hmap *lflows, struct ovn_datapath *od, } static void -build_lrouter_in_dnat_flow(struct hmap *lflows, struct ovn_datapath *od, +build_lrouter_in_dnat_flow(struct hmap *lflows, + const struct ovn_datapath *od, const struct lr_nat_record *lrnat_rec, const struct nbrec_nat *nat, struct ds *match, struct ds *actions, bool distributed_nat, @@ -14880,7 +14951,8 @@ build_lrouter_in_dnat_flow(struct hmap *lflows, struct ovn_datapath *od, } static void -build_lrouter_out_undnat_flow(struct hmap *lflows, struct ovn_datapath *od, +build_lrouter_out_undnat_flow(struct hmap *lflows, + const struct ovn_datapath *od, const struct nbrec_nat *nat, struct ds *match, struct ds *actions, bool distributed_nat, struct eth_addr mac, bool is_v6, @@ -14930,7 +15002,8 @@ build_lrouter_out_undnat_flow(struct hmap *lflows, struct ovn_datapath *od, } static void -build_lrouter_out_is_dnat_local(struct hmap *lflows, struct ovn_datapath *od, +build_lrouter_out_is_dnat_local(struct hmap *lflows, + const struct ovn_datapath *od, const struct nbrec_nat *nat, struct ds *match, struct ds *actions, bool distributed_nat, bool is_v6, struct ovn_port *l3dgw_port) @@ -14960,7 +15033,8 @@ build_lrouter_out_is_dnat_local(struct hmap *lflows, struct ovn_datapath *od, } static void -build_lrouter_out_snat_match(struct hmap *lflows, struct ovn_datapath *od, +build_lrouter_out_snat_match(struct hmap *lflows, + const struct ovn_datapath *od, const struct nbrec_nat *nat, struct ds *match, bool distributed_nat, int cidr_bits, bool is_v6, struct ovn_port *l3dgw_port) @@ -14989,7 +15063,7 @@ build_lrouter_out_snat_match(struct hmap *lflows, struct ovn_datapath *od, static void build_lrouter_out_snat_stateless_flow(struct hmap *lflows, - struct ovn_datapath *od, + const struct ovn_datapath *od, const struct nbrec_nat *nat, struct ds *match, struct ds *actions, bool distributed_nat, @@ -15032,7 +15106,7 @@ build_lrouter_out_snat_stateless_flow(struct hmap *lflows, static void build_lrouter_out_snat_in_czone_flow(struct hmap *lflows, - struct ovn_datapath *od, + const struct ovn_datapath *od, const struct nbrec_nat *nat, struct ds *match, struct ds *actions, bool distributed_nat, @@ -15093,7 +15167,8 @@ build_lrouter_out_snat_in_czone_flow(struct hmap *lflows, } static void -build_lrouter_out_snat_flow(struct hmap *lflows, struct ovn_datapath *od, +build_lrouter_out_snat_flow(struct hmap *lflows, + const struct ovn_datapath *od, const struct nbrec_nat *nat, struct ds *match, struct ds *actions, bool distributed_nat, struct eth_addr mac, int cidr_bits, bool is_v6, @@ -15140,9 +15215,10 @@ build_lrouter_out_snat_flow(struct hmap *lflows, struct ovn_datapath *od, static void build_lrouter_ingress_nat_check_pkt_len(struct hmap *lflows, const struct nbrec_nat *nat, - struct ovn_datapath *od, bool is_v6, - struct ds *match, struct ds *actions, - int mtu, struct ovn_port *l3dgw_port, + const struct ovn_datapath *od, + bool is_v6, struct ds *match, + struct ds *actions, int mtu, + struct ovn_port *l3dgw_port, const struct shash *meter_groups) { ds_clear(match); @@ -15209,7 +15285,8 @@ build_lrouter_ingress_nat_check_pkt_len(struct hmap *lflows, } static void -build_lrouter_ingress_flow(struct hmap *lflows, struct ovn_datapath *od, +build_lrouter_ingress_flow(struct hmap *lflows, + const struct ovn_datapath *od, const struct nbrec_nat *nat, struct ds *match, struct ds *actions, struct eth_addr mac, bool distributed_nat, bool is_v6, @@ -15259,7 +15336,8 @@ build_lrouter_ingress_flow(struct hmap *lflows, struct ovn_datapath *od, } static int -lrouter_check_nat_entry(struct ovn_datapath *od, const struct nbrec_nat *nat, +lrouter_check_nat_entry(const struct ovn_datapath *od, + const struct nbrec_nat *nat, const struct hmap *lr_ports, ovs_be32 *mask, bool *is_v6, int *cidr_bits, struct eth_addr *mac, bool *distributed, struct ovn_port **nat_l3dgw_port) @@ -15386,15 +15464,8 @@ lrouter_check_nat_entry(struct ovn_datapath *od, const struct nbrec_nat *nat, } /* NAT, Defrag and load balancing. */ -static void -build_lrouter_nat_defrag_and_lb(struct ovn_datapath *od, struct hmap *lflows, - const struct hmap *ls_ports, - const struct hmap *lr_ports, - const struct lr_nat_table *lr_nats, - struct ds *match, - struct ds *actions, - const struct shash *meter_groups, - const struct chassis_features *features) +static void build_lr_nat_defrag_and_lb_default_flows(struct ovn_datapath *od, + struct hmap *lflows) { ovs_assert(od->nbr); @@ -15411,6 +15482,23 @@ build_lrouter_nat_defrag_and_lb(struct ovn_datapath *od, struct hmap *lflows, ovn_lflow_add(lflows, od, S_ROUTER_OUT_EGR_LOOP, 0, "1", "next;"); ovn_lflow_add(lflows, od, S_ROUTER_IN_ECMP_STATEFUL, 0, "1", "next;"); + /* Send the IPv6 NS packets to next table. When ovn-controller + * generates IPv6 NS (for the action - nd_ns{}), the injected + * packet would go through conntrack - which is not required. */ + ovn_lflow_add(lflows, od, S_ROUTER_OUT_SNAT, 120, "nd_ns", "next;"); +} + +static void +build_lrouter_nat_defrag_and_lb( + const struct lr_stateful_record *lr_stateful_rec, struct hmap *lflows, + const struct hmap *ls_ports, const struct hmap *lr_ports, + struct ds *match, struct ds *actions, + const struct shash *meter_groups, + const struct chassis_features *features) +{ + const struct ovn_datapath *od = lr_stateful_rec->od; + ovs_assert(od->nbr); + const char *ct_flag_reg = features->ct_no_masked_label ? "ct_mark" : "ct_label"; @@ -15488,11 +15576,6 @@ build_lrouter_nat_defrag_and_lb(struct ovn_datapath *od, struct hmap *lflows, "ip && ct.new", "ct_commit { } ; next; "); } - /* Send the IPv6 NS packets to next table. When ovn-controller - * generates IPv6 NS (for the action - nd_ns{}), the injected - * packet would go through conntrack - which is not required. */ - ovn_lflow_add(lflows, od, S_ROUTER_OUT_SNAT, 120, "nd_ns", "next;"); - /* NAT rules are only valid on Gateway routers and routers with * l3dgw_ports (router has port(s) with gateway chassis * specified). */ @@ -15501,8 +15584,7 @@ build_lrouter_nat_defrag_and_lb(struct ovn_datapath *od, struct hmap *lflows, } struct sset nat_entries = SSET_INITIALIZER(&nat_entries); - const struct lr_nat_record *lrnat_rec = lr_nat_table_find_by_index(lr_nats, - od->index); + const struct lr_nat_record *lrnat_rec = lr_stateful_rec->lrnat_rec; ovs_assert(lrnat_rec); bool dnat_force_snat_ip = @@ -15785,7 +15867,125 @@ build_lrouter_nat_defrag_and_lb(struct ovn_datapath *od, struct hmap *lflows, sset_destroy(&nat_entries); } +static void +build_lsp_lflows_for_lbnats(struct ovn_port *lsp, + const struct lr_stateful_record *lr_stateful_rec, + const struct lr_stateful_table *lr_stateful_table, + const struct hmap *lr_ports, + struct hmap *lflows, + struct ds *match, + struct ds *actions) +{ + ovs_assert(lsp->nbsp); + ovs_assert(lsp->peer); + start_collecting_lflows(); + build_lswitch_rport_arp_req_flows_for_lbnats( + lsp->peer, lr_stateful_rec, lsp->od, lsp, + lflows, &lsp->nbsp->header_); + build_ip_routing_flows_for_router_type_lsp(lsp, lr_stateful_table, + lr_ports, lflows); + build_arp_resolve_flows_for_lsp_routable_addresses( + lsp, lflows, lr_ports, lr_stateful_table, match, actions); + build_lswitch_ip_unicast_lookup_for_nats(lsp, lr_stateful_rec, lflows, + match, actions); + link_ovn_port_to_lflows(lsp, &collected_lflows); + end_collecting_lflows(); +} + +static void +build_lbnat_lflows_iterate_by_lsp( + struct ovn_port *op, const struct lr_stateful_table *lr_stateful_table, + const struct hmap *lr_ports, struct ds *match, struct ds *actions, + struct hmap *lflows) +{ + ovs_assert(op->nbsp); + + if (!lsp_is_router(op->nbsp) || !op->peer) { + return; + } + + const struct lr_stateful_record *lr_stateful_rec; + lr_stateful_rec = lr_stateful_table_find_by_index(lr_stateful_table, + op->peer->od->index); + ovs_assert(lr_stateful_rec); + + build_lsp_lflows_for_lbnats(op, lr_stateful_rec, lr_stateful_table, + lr_ports, lflows, match, actions); +} + +static void +build_lrp_lflows_for_lbnats(struct ovn_port *op, + const struct lr_stateful_record *lr_stateful_rec, + const struct shash *meter_groups, + struct ds *match, struct ds *actions, + struct hmap *lflows) +{ + /* Drop IP traffic destined to router owned IPs except if the IP is + * also a SNAT IP. Those are dropped later, in stage + * "lr_in_arp_resolve", if unSNAT was unsuccessful. + * + * If lrnat_rec->lb_force_snat_router_ip is true, it means the IP of the + * router port is also SNAT IP. + * + * Priority 60. + */ + if (!lr_stateful_rec->lrnat_rec->lb_force_snat_router_ip) { + build_lrouter_drop_own_dest(op, lr_stateful_rec, + S_ROUTER_IN_IP_INPUT, 60, false, lflows); + } + + /* Drop IP traffic destined to router owned IPs. Part of it is dropped + * in stage "lr_in_ip_input" but traffic that could have been unSNATed + * but didn't match any existing session might still end up here. + * + * Priority 2. + */ + build_lrouter_drop_own_dest(op, lr_stateful_rec, + S_ROUTER_IN_ARP_RESOLVE, 2, true, lflows); + + build_lrouter_ipv4_ip_input_for_lbnats(op, lflows, lr_stateful_rec, + match, meter_groups); + build_lrouter_force_snat_flows_op(op, lr_stateful_rec->lrnat_rec, lflows, + match, actions); +} + +static void +build_lbnat_lflows_iterate_by_lrp( + struct ovn_port *op, const struct lr_stateful_table *lr_stateful_table, + const struct shash *meter_groups, struct ds *match, + struct ds *actions, struct hmap *lflows) +{ + ovs_assert(op->nbrp); + const struct lr_stateful_record *lr_stateful_rec; + lr_stateful_rec = lr_stateful_table_find_by_index(lr_stateful_table, + op->od->index); + ovs_assert(lr_stateful_rec); + + build_lrp_lflows_for_lbnats(op, lr_stateful_rec, meter_groups, match, + actions, lflows); +} + +static void +build_lr_stateful_flows(const struct lr_stateful_record *lr_stateful_rec, + struct hmap *lflows, + const struct hmap *ls_ports, + const struct hmap *lr_ports, + struct ds *match, + struct ds *actions, + const struct shash *meter_groups, + const struct chassis_features *features) +{ + build_lrouter_nat_defrag_and_lb(lr_stateful_rec, lflows, ls_ports, + lr_ports, match, actions, + meter_groups, features); + build_lr_gateway_redirect_flows_for_nats(lr_stateful_rec->od, + lr_stateful_rec->lrnat_rec, + lflows, match, actions); + build_lrouter_arp_nd_for_datapath(lr_stateful_rec->od, + lr_stateful_rec->lrnat_rec, lflows, + meter_groups); +} struct lswitch_flow_build_info { const struct ovn_datapaths *ls_datapaths; @@ -15793,7 +15993,6 @@ struct lswitch_flow_build_info { const struct hmap *ls_ports; const struct hmap *lr_ports; const struct ls_port_group_table *ls_port_groups; - const struct lr_nat_table *lr_nats; const struct lr_stateful_table *lr_stateful_table; struct hmap *lflows; struct hmap *igmp_groups; @@ -15860,17 +16059,13 @@ build_lswitch_and_lrouter_iterate_by_lr(struct ovn_datapath *od, build_check_pkt_len_flows_for_lrouter(od, lsi->lflows, lsi->lr_ports, &lsi->match, &lsi->actions, lsi->meter_groups); - build_gateway_redirect_flows_for_lrouter(od, lsi->lflows, lsi->lr_nats, - &lsi->match, &lsi->actions); + build_gateway_redirect_flows_for_lrouter(od, lsi->lflows, &lsi->match, + &lsi->actions); build_arp_request_flows_for_lrouter(od, lsi->lflows, &lsi->match, &lsi->actions, lsi->meter_groups); build_misc_local_traffic_drop_flows_for_lrouter(od, lsi->lflows); - build_lrouter_arp_nd_for_datapath(od, lsi->lr_nats, lsi->lflows, - lsi->meter_groups); - build_lrouter_nat_defrag_and_lb(od, lsi->lflows, lsi->ls_ports, - lsi->lr_ports,lsi->lr_nats, &lsi->match, - &lsi->actions, lsi->meter_groups, - lsi->features); + + build_lr_nat_defrag_and_lb_default_flows(od, lsi->lflows); build_lrouter_lb_affinity_default_flows(od, lsi->lflows); } @@ -15878,15 +16073,13 @@ build_lswitch_and_lrouter_iterate_by_lr(struct ovn_datapath *od, * switch port. */ static void -build_lswitch_and_lrouter_iterate_by_lsp( - struct ovn_port *op, const struct hmap *ls_ports, - const struct hmap *lr_ports, - const struct lr_nat_table *lr_nats, - const struct lr_stateful_table *lr_stateful_table, - const struct shash *meter_groups, - struct ds *match, - struct ds *actions, - struct hmap *lflows) +build_lswitch_and_lrouter_iterate_by_lsp(struct ovn_port *op, + const struct hmap *ls_ports, + const struct hmap *lr_ports, + const struct shash *meter_groups, + struct ds *match, + struct ds *actions, + struct hmap *lflows) { ovs_assert(op->nbsp); start_collecting_lflows(); @@ -15899,14 +16092,11 @@ build_lswitch_and_lrouter_iterate_by_lsp( meter_groups, actions, match); build_lswitch_dhcp_options_and_response(op, lflows, meter_groups); build_lswitch_external_port(op, lflows); - build_lswitch_ip_unicast_lookup(op, lr_nats, lr_stateful_table, lflows, - actions, match); + build_lswitch_ip_unicast_lookup(op, lflows, actions, + match); /* Build Logical Router Flows. */ - build_ip_routing_flows_for_router_type_lsp(op, lr_stateful_table, lr_ports, - lflows); - build_arp_resolve_flows_for_lsp(op, lflows, lr_ports, lr_stateful_table, - match, actions); + build_arp_resolve_flows_for_lsp(op, lflows, lr_ports, match, actions); link_ovn_port_to_lflows(op, &collected_lflows); end_collecting_lflows(); @@ -15921,12 +16111,6 @@ build_lswitch_and_lrouter_iterate_by_lrp(struct ovn_port *op, { ovs_assert(op->nbrp); - const struct lr_nat_record *lrnet_rec = lr_nat_table_find_by_index( - lsi->lr_nats, op->od->index); - ovs_assert(lrnet_rec); - - const struct lr_stateful_record *lr_stateful_rec = - lr_stateful_table_find_by_index(lsi->lr_stateful_table, op->od->index); build_adm_ctrl_flows_for_lrouter_port(op, lsi->lflows, &lsi->match, &lsi->actions); build_neigh_learning_flows_for_lrouter_port(op, lsi->lflows, &lsi->match, @@ -15934,30 +16118,28 @@ build_lswitch_and_lrouter_iterate_by_lrp(struct ovn_port *op, build_ip_routing_flows_for_lrp(op, lsi->lflows); build_ND_RA_flows_for_lrouter_port(op, lsi->lflows, &lsi->match, &lsi->actions, lsi->meter_groups); - build_arp_resolve_flows_for_lrp(op, lrnet_rec, lr_stateful_rec, - lsi->lflows, &lsi->match, &lsi->actions); + build_arp_resolve_flows_for_lrp(op, lsi->lflows, + &lsi->match, &lsi->actions); build_egress_delivery_flows_for_lrouter_port(op, lsi->lflows, &lsi->match, &lsi->actions); build_dhcpv6_reply_flows_for_lrouter_port(op, lsi->lflows, &lsi->match); build_ipv6_input_flows_for_lrouter_port(op, lsi->lflows, &lsi->match, &lsi->actions, lsi->meter_groups); - build_lrouter_ipv4_ip_input(op, lsi->lflows, lrnet_rec, lr_stateful_rec, - &lsi->match, &lsi->actions, lsi->meter_groups); - build_lrouter_force_snat_flows_op(op, lrnet_rec, lsi->lflows, &lsi->match, - &lsi->actions); + build_lrouter_ipv4_ip_input(op, lsi->lflows, &lsi->match, &lsi->actions, + lsi->meter_groups); } static void * build_lflows_thread(void *arg) { struct worker_control *control = (struct worker_control *) arg; + const struct lr_stateful_record *lr_stateful_rec; struct lswitch_flow_build_info *lsi; - + struct ovn_igmp_group *igmp_group; + struct ovn_lb_datapaths *lb_dps; struct ovn_datapath *od; struct ovn_port *op; - struct ovn_lb_datapaths *lb_dps; - struct ovn_igmp_group *igmp_group; int bnum; while (!stop_parallel_processing()) { @@ -16002,10 +16184,15 @@ build_lflows_thread(void *arg) if (stop_parallel_processing()) { return NULL; } - build_lswitch_and_lrouter_iterate_by_lsp( - op, lsi->ls_ports, lsi->lr_ports, lsi->lr_nats, - lsi->lr_stateful_table, lsi->meter_groups, - &lsi->match, &lsi->actions, lsi->lflows); + build_lswitch_and_lrouter_iterate_by_lsp(op, lsi->ls_ports, + lsi->lr_ports, + lsi->meter_groups, + &lsi->match, + &lsi->actions, + lsi->lflows); + build_lbnat_lflows_iterate_by_lsp( + op, lsi->lr_stateful_table, lsi->lr_ports, &lsi->match, + &lsi->actions, lsi->lflows); } } for (bnum = control->id; @@ -16018,6 +16205,9 @@ build_lflows_thread(void *arg) return NULL; } build_lswitch_and_lrouter_iterate_by_lrp(op, lsi); + build_lbnat_lflows_iterate_by_lrp( + op, lsi->lr_stateful_table, lsi->meter_groups, + &lsi->match, &lsi->actions, lsi->lflows); } } for (bnum = control->id; @@ -16040,7 +16230,7 @@ build_lflows_thread(void *arg) build_lrouter_flows_for_lb(lb_dps, lsi->lflows, lsi->meter_groups, lsi->lr_datapaths, - lsi->lr_nats, + lsi->lr_stateful_table, lsi->features, lsi->svc_monitor_map, &lsi->match, &lsi->actions); @@ -16052,6 +16242,23 @@ build_lflows_thread(void *arg) &lsi->match, &lsi->actions); } } + for (bnum = control->id; + bnum <= lsi->lr_stateful_table->entries.mask; + bnum += control->pool->size) + { + LR_STATEFUL_TABLE_FOR_EACH_IN_P (lr_stateful_rec, bnum, + lsi->lr_stateful_table) { + if (stop_parallel_processing()) { + return NULL; + } + build_lr_stateful_flows(lr_stateful_rec, + lsi->lflows, lsi->ls_ports, + lsi->lr_ports, &lsi->match, + &lsi->actions, + lsi->meter_groups, + lsi->features); + } + } for (bnum = control->id; bnum <= lsi->igmp_groups->mask; bnum += control->pool->size) @@ -16112,7 +16319,6 @@ build_lswitch_and_lrouter_flows( const struct hmap *ls_ports, const struct hmap *lr_ports, const struct ls_port_group_table *ls_pgs, - const struct lr_nat_table *lr_nats, const struct lr_stateful_table *lr_stateful_table, struct hmap *lflows, struct hmap *igmp_groups, @@ -16143,7 +16349,6 @@ build_lswitch_and_lrouter_flows( lsiv[index].ls_ports = ls_ports; lsiv[index].lr_ports = lr_ports; lsiv[index].ls_port_groups = ls_pgs; - lsiv[index].lr_nats = lr_nats; lsiv[index].lr_stateful_table = lr_stateful_table; lsiv[index].igmp_groups = igmp_groups; lsiv[index].meter_groups = meter_groups; @@ -16169,17 +16374,18 @@ build_lswitch_and_lrouter_flows( } free(lsiv); } else { + const struct lr_stateful_record *lr_stateful_rec; + struct ovn_igmp_group *igmp_group; + struct ovn_lb_datapaths *lb_dps; struct ovn_datapath *od; struct ovn_port *op; - struct ovn_lb_datapaths *lb_dps; - struct ovn_igmp_group *igmp_group; + struct lswitch_flow_build_info lsi = { .ls_datapaths = ls_datapaths, .lr_datapaths = lr_datapaths, .ls_ports = ls_ports, .lr_ports = lr_ports, .ls_port_groups = ls_pgs, - .lr_nats = lr_nats, .lr_stateful_table = lr_stateful_table, .lflows = lflows, .igmp_groups = igmp_groups, @@ -16208,14 +16414,21 @@ build_lswitch_and_lrouter_flows( HMAP_FOR_EACH (op, key_node, ls_ports) { build_lswitch_and_lrouter_iterate_by_lsp(op, lsi.ls_ports, lsi.lr_ports, - lsi.lr_nats, - lsi.lr_stateful_table, lsi.meter_groups, - &lsi.match, &lsi.actions, + &lsi.match, + &lsi.actions, lsi.lflows); + build_lbnat_lflows_iterate_by_lsp(op, lsi.lr_stateful_table, + lsi.lr_ports, &lsi.match, + &lsi.actions, lsi.lflows); } HMAP_FOR_EACH (op, key_node, lr_ports) { build_lswitch_and_lrouter_iterate_by_lrp(op, &lsi); + build_lbnat_lflows_iterate_by_lrp(op, lsi.lr_stateful_table, + lsi.meter_groups, + &lsi.match, + &lsi.actions, + lsi.lflows); } stopwatch_stop(LFLOWS_PORTS_STOPWATCH_NAME, time_msec()); stopwatch_start(LFLOWS_LBS_STOPWATCH_NAME, time_msec()); @@ -16226,7 +16439,7 @@ build_lswitch_and_lrouter_flows( build_lrouter_defrag_flows_for_lb(lb_dps, lsi.lflows, lsi.lr_datapaths, &lsi.match); build_lrouter_flows_for_lb(lb_dps, lsi.lflows, lsi.meter_groups, - lsi.lr_datapaths, lsi.lr_nats, + lsi.lr_datapaths, lsi.lr_stateful_table, lsi.features, lsi.svc_monitor_map, &lsi.match, &lsi.actions); build_lswitch_flows_for_lb(lb_dps, lsi.lflows, lsi.meter_groups, @@ -16235,6 +16448,14 @@ build_lswitch_and_lrouter_flows( &lsi.match, &lsi.actions); } stopwatch_stop(LFLOWS_LBS_STOPWATCH_NAME, time_msec()); + + LR_STATEFUL_TABLE_FOR_EACH (lr_stateful_rec, lr_stateful_table) { + build_lr_stateful_flows(lr_stateful_rec, lsi.lflows, + lsi.ls_ports, lsi.lr_ports, &lsi.match, + &lsi.actions, lsi.meter_groups, + lsi.features); + } + stopwatch_start(LFLOWS_IGMP_STOPWATCH_NAME, time_msec()); HMAP_FOR_EACH (igmp_group, hmap_node, igmp_groups) { build_lswitch_ip_mcast_igmp_mld(igmp_group, @@ -16330,7 +16551,6 @@ void build_lflows(struct ovsdb_idl_txn *ovnsb_txn, input_data->ls_ports, input_data->lr_ports, input_data->ls_port_groups, - input_data->lr_nats, input_data->lr_stateful_table, lflows, &igmp_groups, @@ -16807,10 +17027,24 @@ lflow_handle_northd_port_changes(struct ovsdb_idl_txn *ovnsb_txn, /* Generate new lflows. */ struct ds match = DS_EMPTY_INITIALIZER; struct ds actions = DS_EMPTY_INITIALIZER; - build_lswitch_and_lrouter_iterate_by_lsp( - op, lflow_input->ls_ports, lflow_input->lr_ports, - lflow_input->lr_nats, lflow_input->lr_stateful_table, - lflow_input->meter_groups, &match, &actions, lflows); + build_lswitch_and_lrouter_iterate_by_lsp(op, lflow_input->ls_ports, + lflow_input->lr_ports, + lflow_input->meter_groups, + &match, &actions, + lflows); + + if (lsp_is_router(op->nbsp) && op->peer && op->peer->od->nbr) { + const struct lr_stateful_record *lr_stateful_rec = + lr_stateful_table_find_by_index(lflow_input->lr_stateful_table, + op->peer->od->index); + ovs_assert(lr_stateful_rec); + + build_lsp_lflows_for_lbnats(op, lr_stateful_rec, + lflow_input->lr_stateful_table, + lflow_input->lr_ports, + lflows, &match, &actions); + } + ds_destroy(&match); ds_destroy(&actions); @@ -16842,10 +17076,23 @@ lflow_handle_northd_port_changes(struct ovsdb_idl_txn *ovnsb_txn, struct ds match = DS_EMPTY_INITIALIZER; struct ds actions = DS_EMPTY_INITIALIZER; - build_lswitch_and_lrouter_iterate_by_lsp( - op, lflow_input->ls_ports, lflow_input->lr_ports, - lflow_input->lr_nats, lflow_input->lr_stateful_table, - lflow_input->meter_groups, &match, &actions, lflows); + build_lswitch_and_lrouter_iterate_by_lsp(op, lflow_input->ls_ports, + lflow_input->lr_ports, + lflow_input->meter_groups, + &match, &actions, lflows); + + if (lsp_is_router(op->nbsp) && op->peer && op->peer->od->nbr) { + const struct lr_stateful_record *lr_stateful_rec = + lr_stateful_table_find_by_index(lflow_input->lr_stateful_table, + op->peer->od->index); + ovs_assert(lr_stateful_rec); + + build_lsp_lflows_for_lbnats(op, lr_stateful_rec, + lflow_input->lr_stateful_table, + lflow_input->lr_ports, + lflows, &match, &actions); + } + ds_destroy(&match); ds_destroy(&actions); diff --git a/northd/northd.h b/northd/northd.h index 518e46ddbe..46bd2498f4 100644 --- a/northd/northd.h +++ b/northd/northd.h @@ -179,7 +179,6 @@ struct lflow_input { const struct hmap *ls_ports; const struct hmap *lr_ports; const struct ls_port_group_table *ls_port_groups; - const struct lr_nat_table *lr_nats; const struct lr_stateful_table *lr_stateful_table; const struct shash *meter_groups; const struct hmap *lb_datapaths_map; From patchwork Thu Jan 11 15:30:28 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Numan Siddique X-Patchwork-Id: 1885676 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.136; helo=smtp3.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4T9pYP4bgmz1yPp for ; Fri, 12 Jan 2024 02:31:37 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 101076F559; Thu, 11 Jan 2024 15:31:35 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 101076F559 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Aj-l9KBhoOhR; Thu, 11 Jan 2024 15:31:31 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp3.osuosl.org (Postfix) with ESMTPS id 1D51F6F4B9; Thu, 11 Jan 2024 15:31:30 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 1D51F6F4B9 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id E8349C0077; Thu, 11 Jan 2024 15:31:29 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 539C6C0037 for ; Thu, 11 Jan 2024 15:31:29 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 8FE0B61B3D for ; Thu, 11 Jan 2024 15:31:16 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 8FE0B61B3D X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id b-v1Xvm3PxAM for ; Thu, 11 Jan 2024 15:31:13 +0000 (UTC) Received: from relay3-d.mail.gandi.net (relay3-d.mail.gandi.net [IPv6:2001:4b98:dc4:8::223]) by smtp3.osuosl.org (Postfix) with ESMTPS id 37E3B61B31 for ; Thu, 11 Jan 2024 15:31:13 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 37E3B61B31 Received: by mail.gandi.net (Postfix) with ESMTPSA id 97A4D6000A; Thu, 11 Jan 2024 15:31:10 +0000 (UTC) From: numans@ovn.org To: dev@openvswitch.org Date: Thu, 11 Jan 2024 10:30:28 -0500 Message-ID: <20240111153028.2790191-1-numans@ovn.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240111152752.2789854-1-numans@ovn.org> References: <20240111152752.2789854-1-numans@ovn.org> MIME-Version: 1.0 X-GND-Sasl: numans@ovn.org Subject: [ovs-dev] [PATCH ovn v5 07/16] northd: Add a new node 'ls_stateful'. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Numan Siddique This new engine now maintains the load balancer and ACL data of a logical switch which was earlier part of northd engine node data. The main inputs to this engine are: - northd node - Port group node A record for each logical switch is maintained in the 'ls_statefuls' hmap table and this record stores the below data which was earlier part of 'struct ovn_datapath'. - bool has_stateful_acl; - bool has_lb_vip; - bool has_acls; - uint64_t max_acl_tier; This engine node becomes an input to 'lflow' node. If a logical switch is configured with - DHCP for all or some of its ports - stateful ACLs and/or load balancers ovn-northd was adding flows to commit the DHCP response generated by ovn-controller to conntrack. With this patch we don't commit the DHCP response to conntrack anymore for the following reasons: 1. There is no need to actually commit the response. 2. Since stateful information is moved to 'ls_stateful' node, it becomes ineffecient to access ls_stateful's data ('has_lb_vip' and 'has_acls') in build_dhcpv4_options_flows(). Signed-off-by: Numan Siddique Acked-by: Dumitru Ceara --- lib/stopwatch-names.h | 1 + northd/automake.mk | 2 + northd/en-lflow.c | 4 + northd/en-lr-stateful.c | 6 +- northd/en-lr-stateful.h | 2 + northd/en-ls-stateful.c | 440 +++++++++++++++++++++++++++++++++++++++ northd/en-ls-stateful.h | 81 +++++++ northd/en-northd.c | 2 +- northd/en-port-group.h | 3 + northd/inc-proc-northd.c | 7 + northd/northd.c | 335 +++++++++++++++-------------- northd/northd.h | 29 ++- northd/ovn-northd.c | 1 + 13 files changed, 753 insertions(+), 160 deletions(-) create mode 100644 northd/en-ls-stateful.c create mode 100644 northd/en-ls-stateful.h diff --git a/lib/stopwatch-names.h b/lib/stopwatch-names.h index e5e41fbfd8..c865f125be 100644 --- a/lib/stopwatch-names.h +++ b/lib/stopwatch-names.h @@ -31,5 +31,6 @@ #define SYNC_METERS_RUN_STOPWATCH_NAME "sync_meters_run" #define LR_NAT_RUN_STOPWATCH_NAME "lr_nat_run" #define LR_STATEFUL_RUN_STOPWATCH_NAME "lr_stateful" +#define LS_STATEFUL_RUN_STOPWATCH_NAME "ls_stateful" #endif diff --git a/northd/automake.mk b/northd/automake.mk index b886356c9c..a178541759 100644 --- a/northd/automake.mk +++ b/northd/automake.mk @@ -28,6 +28,8 @@ northd_ovn_northd_SOURCES = \ northd/en-lr-nat.h \ northd/en-lr-stateful.c \ northd/en-lr-stateful.h \ + northd/en-ls-stateful.c \ + northd/en-ls-stateful.h \ northd/inc-proc-northd.c \ northd/inc-proc-northd.h \ northd/ipam.c \ diff --git a/northd/en-lflow.c b/northd/en-lflow.c index a3a0d62f15..eb6b2a8666 100644 --- a/northd/en-lflow.c +++ b/northd/en-lflow.c @@ -21,6 +21,7 @@ #include "en-lflow.h" #include "en-lr-nat.h" #include "en-lr-stateful.h" +#include "en-ls-stateful.h" #include "en-northd.h" #include "en-meters.h" @@ -44,6 +45,8 @@ lflow_get_input_data(struct engine_node *node, engine_get_input_data("sync_meters", node); struct ed_type_lr_stateful *lr_stateful_data = engine_get_input_data("lr_stateful", node); + struct ed_type_ls_stateful *ls_stateful_data = + engine_get_input_data("ls_stateful", node); lflow_input->nbrec_bfd_table = EN_OVSDB_GET(engine_get_input("NB_bfd", node)); @@ -67,6 +70,7 @@ lflow_get_input_data(struct engine_node *node, lflow_input->lr_ports = &northd_data->lr_ports; lflow_input->ls_port_groups = &pg_data->ls_port_groups; lflow_input->lr_stateful_table = &lr_stateful_data->table; + lflow_input->ls_stateful_table = &ls_stateful_data->table; lflow_input->meter_groups = &sync_meters_data->meter_groups; lflow_input->lb_datapaths_map = &northd_data->lb_datapaths_map; lflow_input->svc_monitor_map = &northd_data->svc_monitor_map; diff --git a/northd/en-lr-stateful.c b/northd/en-lr-stateful.c index dedfdf6747..4287aaddc9 100644 --- a/northd/en-lr-stateful.c +++ b/northd/en-lr-stateful.c @@ -295,7 +295,9 @@ lr_stateful_lb_data_handler(struct engine_node *node, void *data_) /* For all the modified lr_stateful records (re)build the * vip nats. */ HMAPX_FOR_EACH (hmapx_node, &data->trk_data.crupdated) { - lr_stateful_build_vip_nats(hmapx_node->data); + struct lr_stateful_record *lr_stateful_rec = hmapx_node->data; + lr_stateful_build_vip_nats(lr_stateful_rec); + lr_stateful_rec->has_lb_vip = od_has_lb_vip(lr_stateful_rec->od); } engine_set_node_state(node, EN_UPDATED); @@ -510,6 +512,8 @@ lr_stateful_record_init(struct lr_stateful_record *lr_stateful_rec, if (nbr->n_nat) { lr_stateful_build_vip_nats(lr_stateful_rec); } + + lr_stateful_rec->has_lb_vip = od_has_lb_vip(lr_stateful_rec->od); } static struct lr_stateful_input diff --git a/northd/en-lr-stateful.h b/northd/en-lr-stateful.h index 74029c9a6c..5d647d8863 100644 --- a/northd/en-lr-stateful.h +++ b/northd/en-lr-stateful.h @@ -49,6 +49,8 @@ struct lr_stateful_record { const struct ovn_datapath *od; const struct lr_nat_record *lrnat_rec; + bool has_lb_vip; + /* Load Balancer vIPs relevant for this datapath. */ struct ovn_lb_ip_set *lb_ips; diff --git a/northd/en-ls-stateful.c b/northd/en-ls-stateful.c new file mode 100644 index 0000000000..5fa305bc29 --- /dev/null +++ b/northd/en-ls-stateful.c @@ -0,0 +1,440 @@ +/* + * Copyright (c) 2023, Red Hat, Inc. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at: + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#include + +#include +#include +#include + +/* OVS includes */ +#include "include/openvswitch/hmap.h" +#include "lib/bitmap.h" +#include "lib/socket-util.h" +#include "lib/uuidset.h" +#include "openvswitch/util.h" +#include "openvswitch/vlog.h" +#include "stopwatch.h" + +/* OVN includes */ +#include "en-lb-data.h" +#include "en-ls-stateful.h" +#include "en-port-group.h" +#include "lib/inc-proc-eng.h" +#include "lib/lb.h" +#include "lib/ovn-nb-idl.h" +#include "lib/ovn-sb-idl.h" +#include "lib/ovn-util.h" +#include "lib/stopwatch-names.h" +#include "northd.h" + +VLOG_DEFINE_THIS_MODULE(en_ls_stateful); + +/* Static function declarations. */ +static void ls_stateful_table_init(struct ls_stateful_table *); +static void ls_stateful_table_clear(struct ls_stateful_table *); +static void ls_stateful_table_destroy(struct ls_stateful_table *); +static struct ls_stateful_record *ls_stateful_table_find_( + const struct ls_stateful_table *, const struct nbrec_logical_switch *); +static void ls_stateful_table_build(struct ls_stateful_table *, + const struct ovn_datapaths *ls_datapaths, + const struct ls_port_group_table *); + +static struct ls_stateful_input ls_stateful_get_input_data( + struct engine_node *); + +static struct ls_stateful_record *ls_stateful_record_create( + struct ls_stateful_table *, + const struct ovn_datapath *, + const struct ls_port_group_table *); +static void ls_stateful_record_destroy(struct ls_stateful_record *); +static void ls_stateful_record_init( + struct ls_stateful_record *, + const struct ovn_datapath *, + const struct ls_port_group *, + const struct ls_port_group_table *); +static void ls_stateful_record_reinit( + struct ls_stateful_record *, + const struct ls_port_group *, + const struct ls_port_group_table *); +static bool ls_has_lb_vip(const struct ovn_datapath *); +static void ls_stateful_record_set_acl_flags( + struct ls_stateful_record *, const struct ovn_datapath *, + const struct ls_port_group *, const struct ls_port_group_table *); +static bool ls_stateful_record_set_acl_flags_(struct ls_stateful_record *, + struct nbrec_acl **, + size_t n_acls); +static struct ls_stateful_input ls_stateful_get_input_data( + struct engine_node *); + +struct ls_stateful_input { + const struct ls_port_group_table *ls_port_groups; + const struct ovn_datapaths *ls_datapaths; +}; + +/* public functions. */ +const struct ls_stateful_record * +ls_stateful_table_find(const struct ls_stateful_table *table, + const struct nbrec_logical_switch *nbs) +{ + return ls_stateful_table_find_(table, nbs); +} + +void * +en_ls_stateful_init(struct engine_node *node OVS_UNUSED, + struct engine_arg *arg OVS_UNUSED) +{ + struct ed_type_ls_stateful *data = xzalloc(sizeof *data); + ls_stateful_table_init(&data->table); + hmapx_init(&data->trk_data.crupdated); + return data; +} + +void +en_ls_stateful_cleanup(void *data_) +{ + struct ed_type_ls_stateful *data = data_; + ls_stateful_table_destroy(&data->table); + hmapx_destroy(&data->trk_data.crupdated); +} + +void +en_ls_stateful_clear_tracked_data(void *data_) +{ + struct ed_type_ls_stateful *data = data_; + hmapx_clear(&data->trk_data.crupdated); +} + +void +en_ls_stateful_run(struct engine_node *node, void *data_) +{ + struct ls_stateful_input input_data = ls_stateful_get_input_data(node); + struct ed_type_ls_stateful *data = data_; + + stopwatch_start(LS_STATEFUL_RUN_STOPWATCH_NAME, time_msec()); + + ls_stateful_table_clear(&data->table); + ls_stateful_table_build(&data->table, input_data.ls_datapaths, + input_data.ls_port_groups); + + stopwatch_stop(LS_STATEFUL_RUN_STOPWATCH_NAME, time_msec()); + engine_set_node_state(node, EN_UPDATED); +} + +/* Handler functions. */ +bool +ls_stateful_northd_handler(struct engine_node *node, void *data_) +{ + struct northd_data *northd_data = engine_get_input_data("northd", node); + if (!northd_has_tracked_data(&northd_data->trk_data)) { + return false; + } + + if (!northd_has_ls_lbs_in_tracked_data(&northd_data->trk_data) && + !northd_has_ls_acls_in_tracked_data(&northd_data->trk_data)) { + return true; + } + + struct northd_tracked_data *nd_changes = &northd_data->trk_data; + struct ls_stateful_input input_data = ls_stateful_get_input_data(node); + struct ed_type_ls_stateful *data = data_; + struct hmapx_node *hmapx_node; + + struct hmapx changed_stateful_od = HMAPX_INITIALIZER(&changed_stateful_od); + HMAPX_FOR_EACH (hmapx_node, &nd_changes->ls_with_changed_lbs) { + hmapx_add(&changed_stateful_od, hmapx_node->data); + } + + HMAPX_FOR_EACH (hmapx_node, &nd_changes->ls_with_changed_acls) { + hmapx_add(&changed_stateful_od, hmapx_node->data); + } + + HMAPX_FOR_EACH (hmapx_node, &changed_stateful_od) { + const struct ovn_datapath *od = hmapx_node->data; + + struct ls_stateful_record *ls_stateful_rec = ls_stateful_table_find_( + &data->table, od->nbs); + ovs_assert(ls_stateful_rec); + ls_stateful_record_reinit(ls_stateful_rec, NULL, + input_data.ls_port_groups); + + /* Add the ls_stateful_rec to the tracking data. */ + hmapx_add(&data->trk_data.crupdated, ls_stateful_rec); + } + + if (ls_stateful_has_tracked_data(&data->trk_data)) { + engine_set_node_state(node, EN_UPDATED); + } + + hmapx_destroy(&changed_stateful_od); + + return true; +} + +bool +ls_stateful_port_group_handler(struct engine_node *node, void *data_) +{ + struct port_group_data *pg_data = + engine_get_input_data("port_group", node); + + if (pg_data->ls_port_groups_sets_changed) { + return false; + } + + /* port_group engine node doesn't provide the tracking data yet. + * Loop through all the ls port groups and update the ls_stateful_rec. + * This is still better than returning false. */ + struct ls_stateful_input input_data = ls_stateful_get_input_data(node); + struct ed_type_ls_stateful *data = data_; + const struct ls_port_group *ls_pg; + + LS_PORT_GROUP_TABLE_FOR_EACH (ls_pg, input_data.ls_port_groups) { + struct ls_stateful_record *ls_stateful_rec = + ls_stateful_table_find_(&data->table, ls_pg->nbs); + ovs_assert(ls_stateful_rec); + + bool had_stateful_acl = ls_stateful_rec->has_stateful_acl; + uint64_t max_acl_tier = ls_stateful_rec->max_acl_tier; + bool had_acls = ls_stateful_rec->has_acls; + bool modified = false; + + ls_stateful_record_reinit(ls_stateful_rec, ls_pg, + input_data.ls_port_groups); + + if ((had_stateful_acl != ls_stateful_rec->has_stateful_acl) + || (had_acls != ls_stateful_rec->has_acls) + || max_acl_tier != ls_stateful_rec->max_acl_tier) { + modified = true; + } + + if (modified) { + /* Add the ls_stateful_rec to the tracking data. */ + hmapx_add(&data->trk_data.crupdated, ls_stateful_rec); + } + } + + if (ls_stateful_has_tracked_data(&data->trk_data)) { + engine_set_node_state(node, EN_UPDATED); + } + return true; +} + +/* static functions. */ +static void +ls_stateful_table_init(struct ls_stateful_table *table) +{ + *table = (struct ls_stateful_table) { + .entries = HMAP_INITIALIZER(&table->entries), + }; +} + +static void +ls_stateful_table_destroy(struct ls_stateful_table *table) +{ + ls_stateful_table_clear(table); + hmap_destroy(&table->entries); +} + +static void +ls_stateful_table_clear(struct ls_stateful_table *table) +{ + struct ls_stateful_record *ls_stateful_rec; + HMAP_FOR_EACH_POP (ls_stateful_rec, key_node, &table->entries) { + ls_stateful_record_destroy(ls_stateful_rec); + } +} + +static void +ls_stateful_table_build(struct ls_stateful_table *table, + const struct ovn_datapaths *ls_datapaths, + const struct ls_port_group_table *ls_pgs) +{ + const struct ovn_datapath *od; + HMAP_FOR_EACH (od, key_node, &ls_datapaths->datapaths) { + ls_stateful_record_create(table, od, ls_pgs); + } +} + +struct ls_stateful_record * +ls_stateful_table_find_(const struct ls_stateful_table *table, + const struct nbrec_logical_switch *nbs) +{ + struct ls_stateful_record *rec; + + HMAP_FOR_EACH_WITH_HASH (rec, key_node, + uuid_hash(&nbs->header_.uuid), &table->entries) { + if (nbs == rec->od->nbs) { + return rec; + } + } + return NULL; +} + +static struct ls_stateful_record * +ls_stateful_record_create(struct ls_stateful_table *table, + const struct ovn_datapath *od, + const struct ls_port_group_table *ls_pgs) +{ + struct ls_stateful_record *ls_stateful_rec = + xzalloc(sizeof *ls_stateful_rec); + ls_stateful_rec->od = od; + ls_stateful_record_init(ls_stateful_rec, od, NULL, ls_pgs); + + hmap_insert(&table->entries, &ls_stateful_rec->key_node, + uuid_hash(&ls_stateful_rec->od->nbs->header_.uuid)); + + return ls_stateful_rec; +} + +static void +ls_stateful_record_destroy(struct ls_stateful_record *ls_stateful_rec) +{ + free(ls_stateful_rec); +} + +static void +ls_stateful_record_init(struct ls_stateful_record *ls_stateful_rec, + const struct ovn_datapath *od, + const struct ls_port_group *ls_pg, + const struct ls_port_group_table *ls_pgs) +{ + ls_stateful_rec->has_lb_vip = ls_has_lb_vip(od); + ls_stateful_record_set_acl_flags(ls_stateful_rec, od, ls_pg, ls_pgs); +} + +static void +ls_stateful_record_reinit(struct ls_stateful_record *ls_stateful_rec, + const struct ls_port_group *ls_pg, + const struct ls_port_group_table *ls_pgs) +{ + ls_stateful_record_init(ls_stateful_rec, ls_stateful_rec->od, + ls_pg, ls_pgs); +} + +static bool +lb_has_vip(const struct nbrec_load_balancer *lb) +{ + return !smap_is_empty(&lb->vips); +} + +static bool +lb_group_has_vip(const struct nbrec_load_balancer_group *lb_group) +{ + for (size_t i = 0; i < lb_group->n_load_balancer; i++) { + if (lb_has_vip(lb_group->load_balancer[i])) { + return true; + } + } + return false; +} + +static bool +ls_has_lb_vip(const struct ovn_datapath *od) +{ + for (size_t i = 0; i < od->nbs->n_load_balancer; i++) { + if (lb_has_vip(od->nbs->load_balancer[i])) { + return true; + } + } + + for (size_t i = 0; i < od->nbs->n_load_balancer_group; i++) { + if (lb_group_has_vip(od->nbs->load_balancer_group[i])) { + return true; + } + } + return false; +} + +static void +ls_stateful_record_set_acl_flags(struct ls_stateful_record *ls_stateful_rec, + const struct ovn_datapath *od, + const struct ls_port_group *ls_pg, + const struct ls_port_group_table *ls_pgs) +{ + ls_stateful_rec->has_stateful_acl = false; + ls_stateful_rec->max_acl_tier = 0; + ls_stateful_rec->has_acls = false; + + if (ls_stateful_record_set_acl_flags_(ls_stateful_rec, od->nbs->acls, + od->nbs->n_acls)) { + return; + } + + if (!ls_pg) { + ls_pg = ls_port_group_table_find(ls_pgs, od->nbs); + } + + if (!ls_pg) { + return; + } + + const struct ls_port_group_record *ls_pg_rec; + HMAP_FOR_EACH (ls_pg_rec, key_node, &ls_pg->nb_pgs) { + if (ls_stateful_record_set_acl_flags_(ls_stateful_rec, + ls_pg_rec->nb_pg->acls, + ls_pg_rec->nb_pg->n_acls)) { + return; + } + } +} + +static bool +ls_stateful_record_set_acl_flags_(struct ls_stateful_record *ls_stateful_rec, + struct nbrec_acl **acls, + size_t n_acls) +{ + /* A true return indicates that there are no possible ACL flags + * left to set on ls_stateful record. A false return indicates that + * further ACLs should be explored in case more flags need to be + * set on ls_stateful record. + */ + if (!n_acls) { + return false; + } + + ls_stateful_rec->has_acls = true; + for (size_t i = 0; i < n_acls; i++) { + const struct nbrec_acl *acl = acls[i]; + if (acl->tier > ls_stateful_rec->max_acl_tier) { + ls_stateful_rec->max_acl_tier = acl->tier; + } + if (!ls_stateful_rec->has_stateful_acl + && !strcmp(acl->action, "allow-related")) { + ls_stateful_rec->has_stateful_acl = true; + } + if (ls_stateful_rec->has_stateful_acl && + ls_stateful_rec->max_acl_tier == + nbrec_acl_col_tier.type.value.integer.max) { + return true; + } + } + + return false; +} + +static struct ls_stateful_input +ls_stateful_get_input_data(struct engine_node *node) +{ + const struct northd_data *northd_data = + engine_get_input_data("northd", node); + const struct port_group_data *pg_data = + engine_get_input_data("port_group", node); + + return (struct ls_stateful_input) { + .ls_port_groups = &pg_data->ls_port_groups, + .ls_datapaths = &northd_data->ls_datapaths, + }; +} diff --git a/northd/en-ls-stateful.h b/northd/en-ls-stateful.h new file mode 100644 index 0000000000..cba53e1f29 --- /dev/null +++ b/northd/en-ls-stateful.h @@ -0,0 +1,81 @@ +/* + * Copyright (c) 2023, Red Hat, Inc. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at: + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +#ifndef EN_LS_STATEFUL_H +#define EN_LS_STATEFUL_H 1 + +#include + +/* OVS includes. */ +#include "lib/hmapx.h" +#include "openvswitch/hmap.h" +#include "sset.h" + +/* OVN includes. */ +#include "lib/inc-proc-eng.h" +#include "lib/lb.h" +#include "lib/ovn-nb-idl.h" +#include "lib/ovn-sb-idl.h" +#include "lib/ovn-util.h" +#include "lib/stopwatch-names.h" + +struct ls_stateful_record { + struct hmap_node key_node; + + const struct ovn_datapath *od; + bool has_stateful_acl; + bool has_lb_vip; + bool has_acls; + uint64_t max_acl_tier; +}; + +struct ls_stateful_table { + struct hmap entries; +}; + +#define LS_STATEFUL_TABLE_FOR_EACH(LS_STATEFUL_REC, TABLE) \ + HMAP_FOR_EACH (LS_STATEFUL_REC, key_node, &(TABLE)->entries) + +#define LS_STATEFUL_TABLE_FOR_EACH_IN_P(LS_STATEFUL_REC, JOBID, TABLE) \ + HMAP_FOR_EACH_IN_PARALLEL (LS_STATEFUL_REC, key_node, JOBID, \ + &(TABLE)->entries) + +struct ls_stateful_tracked_data { + /* Created or updated logical switch with LB and ACL data. */ + struct hmapx crupdated; /* Stores 'struct ls_stateful_record'. */ +}; + +struct ed_type_ls_stateful { + struct ls_stateful_table table; + struct ls_stateful_tracked_data trk_data; +}; + +void *en_ls_stateful_init(struct engine_node *, struct engine_arg *); +void en_ls_stateful_cleanup(void *data); +void en_ls_stateful_clear_tracked_data(void *data); +void en_ls_stateful_run(struct engine_node *, void *data); + +bool ls_stateful_northd_handler(struct engine_node *, void *data); +bool ls_stateful_port_group_handler(struct engine_node *, void *data); + +const struct ls_stateful_record *ls_stateful_table_find( + const struct ls_stateful_table *, const struct nbrec_logical_switch *); + +static inline bool +ls_stateful_has_tracked_data(struct ls_stateful_tracked_data *trk_data) { + return !hmapx_is_empty(&trk_data->crupdated); +} + +#endif /* EN_LS_STATEFUL_H */ diff --git a/northd/en-northd.c b/northd/en-northd.c index 546397f3dc..5143603f39 100644 --- a/northd/en-northd.c +++ b/northd/en-northd.c @@ -171,7 +171,7 @@ northd_nb_logical_switch_handler(struct engine_node *node, return false; } - if (northd_has_lsps_in_tracked_data(&nd->trk_data)) { + if (northd_has_tracked_data(&nd->trk_data)) { engine_set_node_state(node, EN_UPDATED); } diff --git a/northd/en-port-group.h b/northd/en-port-group.h index 3b28a23694..54014062ce 100644 --- a/northd/en-port-group.h +++ b/northd/en-port-group.h @@ -48,6 +48,9 @@ struct ls_port_group_record { struct sset ports; /* Subset of 'nb_pg' ports in this record. */ }; +#define LS_PORT_GROUP_TABLE_FOR_EACH(LS_PG, TABLE) \ + HMAP_FOR_EACH (LS_PG, key_node, &(TABLE)->entries) + void ls_port_group_table_init(struct ls_port_group_table *); void ls_port_group_table_clear(struct ls_port_group_table *); void ls_port_group_table_destroy(struct ls_port_group_table *); diff --git a/northd/inc-proc-northd.c b/northd/inc-proc-northd.c index adb38dde78..9ce4279ee8 100644 --- a/northd/inc-proc-northd.c +++ b/northd/inc-proc-northd.c @@ -33,6 +33,7 @@ #include "en-lb-data.h" #include "en-lr-stateful.h" #include "en-lr-nat.h" +#include "en-ls-stateful.h" #include "en-northd.h" #include "en-lflow.h" #include "en-northd-output.h" @@ -150,6 +151,7 @@ static ENGINE_NODE(sync_to_sb_pb, "sync_to_sb_pb"); static ENGINE_NODE_WITH_CLEAR_TRACK_DATA(lb_data, "lb_data"); static ENGINE_NODE_WITH_CLEAR_TRACK_DATA(lr_nat, "lr_nat"); static ENGINE_NODE_WITH_CLEAR_TRACK_DATA(lr_stateful, "lr_stateful"); +static ENGINE_NODE_WITH_CLEAR_TRACK_DATA(ls_stateful, "ls_stateful"); void inc_proc_northd_init(struct ovsdb_idl_loop *nb, struct ovsdb_idl_loop *sb) @@ -200,6 +202,10 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, engine_add_input(&en_lr_stateful, &en_lb_data, lr_stateful_lb_data_handler); + engine_add_input(&en_ls_stateful, &en_northd, ls_stateful_northd_handler); + engine_add_input(&en_ls_stateful, &en_port_group, + ls_stateful_port_group_handler); + engine_add_input(&en_mac_binding_aging, &en_nb_nb_global, NULL); engine_add_input(&en_mac_binding_aging, &en_sb_mac_binding, NULL); engine_add_input(&en_mac_binding_aging, &en_northd, NULL); @@ -222,6 +228,7 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, engine_add_input(&en_lflow, &en_sb_multicast_group, NULL); engine_add_input(&en_lflow, &en_sb_igmp_group, NULL); engine_add_input(&en_lflow, &en_lr_stateful, NULL); + engine_add_input(&en_lflow, &en_ls_stateful, NULL); engine_add_input(&en_lflow, &en_northd, lflow_northd_handler); engine_add_input(&en_lflow, &en_port_group, lflow_port_group_handler); diff --git a/northd/northd.c b/northd/northd.c index f56a7c5ea4..68f2b0bab4 100644 --- a/northd/northd.c +++ b/northd/northd.c @@ -46,6 +46,7 @@ #include "en-lb-data.h" #include "en-lr-nat.h" #include "en-lr-stateful.h" +#include "en-ls-stateful.h" #include "lib/ovn-parallel-hmap.h" #include "ovn/actions.h" #include "ovn/features.h" @@ -576,7 +577,7 @@ lb_group_has_vip(const struct nbrec_load_balancer_group *lb_group) } static bool -ls_has_lb_vip(struct ovn_datapath *od) +ls_has_lb_vip(const struct ovn_datapath *od) { for (size_t i = 0; i < od->nbs->n_load_balancer; i++) { if (lb_has_vip(od->nbs->load_balancer[i])) { @@ -593,7 +594,7 @@ ls_has_lb_vip(struct ovn_datapath *od) } static bool -lr_has_lb_vip(struct ovn_datapath *od) +lr_has_lb_vip(const struct ovn_datapath *od) { for (size_t i = 0; i < od->nbr->n_load_balancer; i++) { if (lb_has_vip(od->nbr->load_balancer[i])) { @@ -609,13 +610,13 @@ lr_has_lb_vip(struct ovn_datapath *od) return false; } -static void -init_lb_for_datapath(struct ovn_datapath *od) +bool +od_has_lb_vip(const struct ovn_datapath *od) { if (od->nbs) { - od->has_lb_vip = ls_has_lb_vip(od); + return ls_has_lb_vip(od); } else { - od->has_lb_vip = lr_has_lb_vip(od); + return lr_has_lb_vip(od); } } @@ -1065,7 +1066,6 @@ join_datapaths(const struct nbrec_logical_switch_table *nbrec_ls_table, init_ipam_info_for_datapath(od); init_mcast_info_for_datapath(od); - init_lb_for_datapath(od); } const struct nbrec_logical_router *nbr; @@ -1096,7 +1096,6 @@ join_datapaths(const struct nbrec_logical_switch_table *nbrec_ls_table, ovs_list_push_back(nb_only, &od->list); } init_mcast_info_for_datapath(od); - init_lb_for_datapath(od); if (smap_get(&od->nbr->options, "chassis")) { od->is_gw_router = true; } @@ -2583,7 +2582,8 @@ get_nat_addresses(const struct ovn_port *op, size_t *n, bool routable_only, size_t n_nats = 0; struct eth_addr mac; if (!op || !op->nbrp || !op->od || !op->od->nbr - || (!op->od->nbr->n_nat && !op->od->has_lb_vip) + || (!op->od->nbr->n_nat && (!lr_stateful_rec + || !lr_stateful_rec->has_lb_vip)) || !eth_addr_from_string(op->nbrp->mac, &mac)) { *n = n_nats; return NULL; @@ -3830,7 +3830,7 @@ build_lrouter_lbs_check(const struct ovn_datapaths *lr_datapaths) HMAP_FOR_EACH (od, key_node, &lr_datapaths->datapaths) { ovs_assert(od->nbr); - if (od->has_lb_vip && od->n_l3dgw_ports > 1 + if (od_has_lb_vip(od) && od->n_l3dgw_ports > 1 && !smap_get(&od->nbr->options, "chassis")) { static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 1); VLOG_WARN_RL(&rl, "Load-balancers are configured on logical " @@ -4750,6 +4750,8 @@ destroy_northd_data_tracked_changes(struct northd_data *nd) destroy_tracked_ovn_ports(&trk_changes->trk_lsps); destroy_tracked_lbs(&trk_changes->trk_lbs); hmapx_clear(&trk_changes->lr_with_changed_nats); + hmapx_clear(&trk_changes->ls_with_changed_lbs); + hmapx_clear(&trk_changes->ls_with_changed_acls); nd->trk_data.type = NORTHD_TRACKED_NONE; } @@ -4764,6 +4766,8 @@ init_northd_tracked_data(struct northd_data *nd) hmapx_init(&trk_data->trk_lbs.crupdated); hmapx_init(&trk_data->trk_lbs.deleted); hmapx_init(&trk_data->lr_with_changed_nats); + hmapx_init(&trk_data->ls_with_changed_lbs); + hmapx_init(&trk_data->ls_with_changed_acls); } static void @@ -4777,6 +4781,8 @@ destroy_northd_tracked_data(struct northd_data *nd) hmapx_destroy(&trk_data->trk_lbs.crupdated); hmapx_destroy(&trk_data->trk_lbs.deleted); hmapx_destroy(&trk_data->lr_with_changed_nats); + hmapx_destroy(&trk_data->ls_with_changed_lbs); + hmapx_destroy(&trk_data->ls_with_changed_acls); } /* Check if a changed LSP can be handled incrementally within the I-P engine @@ -4892,6 +4898,7 @@ ls_port_create(struct ovsdb_idl_txn *ovnsb_txn, struct hmap *ls_ports, * - logical switch ports. * - load balancers. * - load balancer groups. + * - ACLs */ static bool ls_changes_can_be_handled( @@ -5116,6 +5123,25 @@ fail: return false; } +static bool +is_acls_seqno_changed(struct nbrec_acl **nb_acls, size_t n_nb_acls) +{ + for (size_t i = 0; i < n_nb_acls; i++) { + if (nbrec_acl_row_get_seqno(nb_acls[i], + OVSDB_IDL_CHANGE_MODIFY) > 0) { + return true; + } + } + + return false; +} + +static bool +is_ls_acls_changed(const struct nbrec_logical_switch *nbs) { + return (nbrec_logical_switch_is_updated(nbs, NBREC_LOGICAL_SWITCH_COL_ACLS) + || is_acls_seqno_changed(nbs->acls, nbs->n_acls)); +} + /* Return true if changes are handled incrementally, false otherwise. * When there are any changes, try to track what's exactly changed and set * northd_data->trk_data accordingly. @@ -5157,6 +5183,10 @@ northd_handle_ls_changes(struct ovsdb_idl_txn *ovnsb_idl_txn, ni, nd, od, &trk_data->trk_lsps)) { goto fail; } + + if (is_ls_acls_changed(changed_ls)) { + hmapx_add(&trk_data->ls_with_changed_acls, od); + } } if (!hmapx_is_empty(&trk_data->trk_lsps.created) @@ -5165,6 +5195,10 @@ northd_handle_ls_changes(struct ovsdb_idl_txn *ovnsb_idl_txn, trk_data->type |= NORTHD_TRACKED_PORTS; } + if (!hmapx_is_empty(&trk_data->ls_with_changed_acls)) { + trk_data->type |= NORTHD_TRACKED_LS_ACLS; + } + return true; fail: @@ -5386,10 +5420,6 @@ northd_handle_sb_port_binding_changes( * due to association of a load balancer (eg. ovn-nbctl ls-lb-add sw0 lb1), * the logical switch datapath is added to the load balancer (represented * by 'struct ovn_lb_datapaths') by calling ovn_lb_datapaths_add_ls(). - * - * - For every 'lb' in the tracked data (trk_lb_data->crupdated_lbs) , - * it gets the associated logical switches and for each switch it - * re-evaluates 'od->has_lb_vip' to reflect any changes to the lb vips. * */ bool northd_handle_lb_data_changes(struct tracked_lb_data *trk_lb_data, @@ -5450,13 +5480,13 @@ northd_handle_lb_data_changes(struct tracked_lb_data *trk_lb_data, lb_dps = ovn_lb_datapaths_find(lb_datapaths_map, lb_uuid); ovs_assert(lb_dps); - /* Re-evaluate 'od->has_lb_vip for od's associated with the - * deleted lb. */ size_t index; BITMAP_FOR_EACH_1 (index, ods_size(ls_datapaths), lb_dps->nb_ls_map) { od = ls_datapaths->array[index]; - init_lb_for_datapath(od); + + /* Add the ls datapath to the northd tracked data. */ + hmapx_add(&nd_changes->ls_with_changed_lbs, od); } hmap_remove(lb_datapaths_map, &lb_dps->hmap_node); @@ -5536,8 +5566,8 @@ northd_handle_lb_data_changes(struct tracked_lb_data *trk_lb_data, } } - /* Re-evaluate 'od->has_lb_vip' */ - init_lb_for_datapath(od); + /* Add the ls datapath to the northd tracked data. */ + hmapx_add(&nd_changes->ls_with_changed_lbs, od); } LIST_FOR_EACH (codlb, list_node, &trk_lb_data->crupdated_lr_lbs) { @@ -5572,9 +5602,6 @@ northd_handle_lb_data_changes(struct tracked_lb_data *trk_lb_data, hmapx_add(&nd_changes->trk_lbs.crupdated, lb_dps); } } - - /* Re-evaluate 'od->has_lb_vip' */ - init_lb_for_datapath(od); } HMAP_FOR_EACH (clb, hmap_node, &trk_lb_data->crupdated_lbs) { @@ -5587,15 +5614,9 @@ northd_handle_lb_data_changes(struct tracked_lb_data *trk_lb_data, BITMAP_FOR_EACH_1 (index, ods_size(ls_datapaths), lb_dps->nb_ls_map) { od = ls_datapaths->array[index]; - /* Re-evaluate 'od->has_lb_vip' */ - init_lb_for_datapath(od); - } - BITMAP_FOR_EACH_1 (index, ods_size(lr_datapaths), - lb_dps->nb_lr_map) { - od = lr_datapaths->array[index]; - /* Re-evaluate 'od->has_lb_vip' */ - init_lb_for_datapath(od); + /* Add the ls datapath to the northd tracked data. */ + hmapx_add(&nd_changes->ls_with_changed_lbs, od); } } @@ -5617,17 +5638,14 @@ northd_handle_lb_data_changes(struct tracked_lb_data *trk_lb_data, for (size_t i = 0; i < lbgrp_dps->n_lr; i++) { od = lbgrp_dps->lr[i]; ovn_lb_datapaths_add_lr(lb_dps, 1, &od); - - /* Re-evaluate 'od->has_lb_vip' */ - init_lb_for_datapath(od); } for (size_t i = 0; i < lbgrp_dps->n_ls; i++) { od = lbgrp_dps->ls[i]; ovn_lb_datapaths_add_ls(lb_dps, 1, &od); - /* Re-evaluate 'od->has_lb_vip' */ - init_lb_for_datapath(od); + /* Add the ls datapath to the northd tracked data. */ + hmapx_add(&nd_changes->ls_with_changed_lbs, od); } /* Add the lb to the northd tracked data. */ @@ -5640,6 +5658,10 @@ northd_handle_lb_data_changes(struct tracked_lb_data *trk_lb_data, nd_changes->type |= NORTHD_TRACKED_LBS; } + if (!hmapx_is_empty(&nd_changes->ls_with_changed_lbs)) { + nd_changes->type |= NORTHD_TRACKED_LS_LBS; + } + return true; } @@ -6573,63 +6595,6 @@ build_dhcpv6_action(struct ovn_port *op, struct in6_addr *offer_ip, return true; } -static bool -od_set_acl_flags(struct ovn_datapath *od, struct nbrec_acl **acls, - size_t n_acls) -{ - /* A true return indicates that there are no possible ACL flags - * left to set on od. A false return indicates that further ACLs - * should be explored in case more flags need to be set on od - */ - if (!n_acls) { - return false; - } - - od->has_acls = true; - for (size_t i = 0; i < n_acls; i++) { - const struct nbrec_acl *acl = acls[i]; - if (acl->tier > od->max_acl_tier) { - od->max_acl_tier = acl->tier; - } - if (!od->has_stateful_acl && !strcmp(acl->action, "allow-related")) { - od->has_stateful_acl = true; - } - if (od->has_stateful_acl && - od->max_acl_tier == nbrec_acl_col_tier.type.value.integer.max) { - return true; - } - } - - return false; -} - -static void -ls_get_acl_flags(struct ovn_datapath *od, - const struct ls_port_group_table *ls_port_groups) -{ - od->has_acls = false; - od->has_stateful_acl = false; - od->max_acl_tier = 0; - - if (od_set_acl_flags(od, od->nbs->acls, od->nbs->n_acls)) { - return; - } - - const struct ls_port_group *ls_pg = - ls_port_group_table_find(ls_port_groups, od->nbs); - if (!ls_pg) { - return; - } - - const struct ls_port_group_record *ls_pg_rec; - HMAP_FOR_EACH (ls_pg_rec, key_node, &ls_pg->nb_pgs) { - if (od_set_acl_flags(od, ls_pg_rec->nb_pg->acls, - ls_pg_rec->nb_pg->n_acls)) { - return; - } - } -} - /* Adds the logical flows in the (in/out) check port sec stage only if * - the lport is disabled or * - lport is of type vtep - to skip the ingress pipeline. @@ -6777,9 +6742,10 @@ build_lswitch_output_port_sec_od(struct ovn_datapath *od, } static void -skip_port_from_conntrack(struct ovn_datapath *od, struct ovn_port *op, - enum ovn_stage in_stage, enum ovn_stage out_stage, - uint16_t priority, struct hmap *lflows) +skip_port_from_conntrack(const struct ovn_datapath *od, struct ovn_port *op, + bool has_stateful_acl, enum ovn_stage in_stage, + enum ovn_stage out_stage, uint16_t priority, + struct hmap *lflows) { /* Can't use ct() for router ports. Consider the following configuration: * lp1(10.0.0.2) on hostA--ls1--lr0--ls2--lp2(10.0.1.2) on hostB, For a @@ -6792,7 +6758,7 @@ skip_port_from_conntrack(struct ovn_datapath *od, struct ovn_port *op, * conntrack state across all chassis. */ const char *ingress_action = "next;"; - const char *egress_action = od->has_stateful_acl + const char *egress_action = has_stateful_acl ? "next;" : "ct_clear; next;"; @@ -6811,7 +6777,7 @@ skip_port_from_conntrack(struct ovn_datapath *od, struct ovn_port *op, } static void -build_stateless_filter(struct ovn_datapath *od, +build_stateless_filter(const struct ovn_datapath *od, const struct nbrec_acl *acl, struct hmap *lflows) { @@ -6832,7 +6798,7 @@ build_stateless_filter(struct ovn_datapath *od, } static void -build_stateless_filters(struct ovn_datapath *od, +build_stateless_filters(const struct ovn_datapath *od, const struct ls_port_group_table *ls_port_groups, struct hmap *lflows) { @@ -6862,9 +6828,7 @@ build_stateless_filters(struct ovn_datapath *od, } static void -build_pre_acls(struct ovn_datapath *od, - const struct ls_port_group_table *ls_port_groups, - struct hmap *lflows) +build_pre_acls(struct ovn_datapath *od, struct hmap *lflows) { /* Ingress and Egress Pre-ACL Table (Priority 0): Packets are * allowed by default. */ @@ -6876,22 +6840,30 @@ build_pre_acls(struct ovn_datapath *od, ovn_lflow_add(lflows, od, S_SWITCH_OUT_PRE_ACL, 110, "eth.src == $svc_monitor_mac", "next;"); +} + +static void +build_ls_stateful_rec_pre_acls(const struct ls_stateful_record *ls_sful_rec, + const struct ls_port_group_table *ls_port_groups, + struct hmap *lflows) +{ + const struct ovn_datapath *od = ls_sful_rec->od; /* If there are any stateful ACL rules in this datapath, we may * send IP packets for some (allow) filters through the conntrack action, * which handles defragmentation, in order to match L4 headers. */ - if (od->has_stateful_acl) { + if (ls_sful_rec->has_stateful_acl) { for (size_t i = 0; i < od->n_router_ports; i++) { struct ovn_port *op = od->router_ports[i]; if (op->enable_router_port_acl) { continue; } - skip_port_from_conntrack(od, op, + skip_port_from_conntrack(od, op, true, S_SWITCH_IN_PRE_ACL, S_SWITCH_OUT_PRE_ACL, 110, lflows); } for (size_t i = 0; i < od->n_localnet_ports; i++) { - skip_port_from_conntrack(od, od->localnet_ports[i], + skip_port_from_conntrack(od, od->localnet_ports[i], true, S_SWITCH_IN_PRE_ACL, S_SWITCH_OUT_PRE_ACL, 110, lflows); @@ -6929,7 +6901,7 @@ build_pre_acls(struct ovn_datapath *od, REGBIT_CONNTRACK_DEFRAG" = 1; next;"); ovn_lflow_add(lflows, od, S_SWITCH_OUT_PRE_ACL, 100, "ip", REGBIT_CONNTRACK_DEFRAG" = 1; next;"); - } else if (od->has_lb_vip) { + } else if (ls_sful_rec->has_lb_vip) { /* We'll build stateless filters if there are LB rules so that * the stateless flows are not tracked in pre-lb. */ build_stateless_filters(od, ls_port_groups, lflows); @@ -7057,30 +7029,40 @@ build_pre_lb(struct ovn_datapath *od, const struct shash *meter_groups, ovn_lflow_add(lflows, od, S_SWITCH_IN_PRE_LB, 0, "1", "next;"); ovn_lflow_add(lflows, od, S_SWITCH_OUT_PRE_LB, 0, "1", "next;"); + /* Do not send statless flows via conntrack */ + ovn_lflow_add(lflows, od, S_SWITCH_IN_PRE_LB, 110, + REGBIT_ACL_STATELESS" == 1", "next;"); + ovn_lflow_add(lflows, od, S_SWITCH_OUT_PRE_LB, 110, + REGBIT_ACL_STATELESS" == 1", "next;"); +} + +static void +build_ls_stateful_rec_pre_lb(const struct ls_stateful_record *ls_stateful_rec, + struct hmap *lflows) +{ + const struct ovn_datapath *od = ls_stateful_rec->od; + for (size_t i = 0; i < od->n_router_ports; i++) { skip_port_from_conntrack(od, od->router_ports[i], + ls_stateful_rec->has_stateful_acl, S_SWITCH_IN_PRE_LB, S_SWITCH_OUT_PRE_LB, 110, lflows); } + /* Localnet ports have no need for going through conntrack, unless * the logical switch has a load balancer. Then, conntrack is necessary * so that traffic arriving via the localnet port can be load * balanced. */ - if (!od->has_lb_vip) { + if (!ls_stateful_rec->has_lb_vip) { for (size_t i = 0; i < od->n_localnet_ports; i++) { skip_port_from_conntrack(od, od->localnet_ports[i], + ls_stateful_rec->has_stateful_acl, S_SWITCH_IN_PRE_LB, S_SWITCH_OUT_PRE_LB, 110, lflows); } } - /* Do not sent statless flows via conntrack */ - ovn_lflow_add(lflows, od, S_SWITCH_IN_PRE_LB, 110, - REGBIT_ACL_STATELESS" == 1", "next;"); - ovn_lflow_add(lflows, od, S_SWITCH_OUT_PRE_LB, 110, - REGBIT_ACL_STATELESS" == 1", "next;"); - /* 'REGBIT_CONNTRACK_NAT' is set to let the pre-stateful table send * packet to conntrack for defragmentation and possibly for unNATting. * @@ -7111,7 +7093,7 @@ build_pre_lb(struct ovn_datapath *od, const struct shash *meter_groups, * ingress pipeline if a load balancer is configured. We can now * add a lflow to drop ct.inv packets. */ - if (od->has_lb_vip) { + if (ls_stateful_rec->has_lb_vip) { ovn_lflow_add(lflows, od, S_SWITCH_IN_PRE_LB, 100, "ip", REGBIT_CONNTRACK_NAT" = 1; next;"); ovn_lflow_add(lflows, od, S_SWITCH_OUT_PRE_LB, @@ -7152,10 +7134,12 @@ build_pre_stateful(struct ovn_datapath *od, } static void -build_acl_hints(struct ovn_datapath *od, +build_acl_hints(const struct ls_stateful_record *ls_sful_rec, const struct chassis_features *features, struct hmap *lflows) { + const struct ovn_datapath *od = ls_sful_rec->od; + /* This stage builds hints for the IN/OUT_ACL stage. Based on various * combinations of ct flags packets may hit only a subset of the logical * flows in the IN/OUT_ACL stage. @@ -7179,13 +7163,13 @@ build_acl_hints(struct ovn_datapath *od, const char *match; /* In any case, advance to the next stage. */ - if (!od->has_acls && !od->has_lb_vip) { + if (!ls_sful_rec->has_acls && !ls_sful_rec->has_lb_vip) { ovn_lflow_add(lflows, od, stage, UINT16_MAX, "1", "next;"); } else { ovn_lflow_add(lflows, od, stage, 0, "1", "next;"); } - if (!od->has_stateful_acl && !od->has_lb_vip) { + if (!ls_sful_rec->has_stateful_acl && !ls_sful_rec->has_lb_vip) { continue; } @@ -7321,10 +7305,10 @@ build_acl_log(struct ds *actions, const struct nbrec_acl *acl, } static void -consider_acl(struct hmap *lflows, struct ovn_datapath *od, +consider_acl(struct hmap *lflows, const struct ovn_datapath *od, const struct nbrec_acl *acl, bool has_stateful, bool ct_masked_mark, const struct shash *meter_groups, - struct ds *match, struct ds *actions) + uint64_t max_acl_tier, struct ds *match, struct ds *actions) { const char *ct_blocked_match = ct_masked_mark ? "ct_mark.blocked" @@ -7361,7 +7345,7 @@ consider_acl(struct hmap *lflows, struct ovn_datapath *od, /* All ACLS will start by matching on their respective tier. */ size_t match_tier_len = 0; ds_clear(match); - if (od->max_acl_tier) { + if (max_acl_tier) { ds_put_format(match, REG_ACL_TIER " == %"PRId64" && ", acl->tier); match_tier_len = match->length; } @@ -7548,12 +7532,15 @@ ovn_update_ipv6_options(struct hmap *lr_ports) #define IPV6_CT_OMIT_MATCH "nd || nd_ra || nd_rs || mldv1 || mldv2" static void -build_acl_action_lflows(struct ovn_datapath *od, struct hmap *lflows, +build_acl_action_lflows(const struct ls_stateful_record *ls_stateful_rec, + struct hmap *lflows, const char *default_acl_action, const struct shash *meter_groups, struct ds *match, struct ds *actions) { + const struct ovn_datapath *od = ls_stateful_rec->od; + enum ovn_stage stages [] = { S_SWITCH_IN_ACL_ACTION, S_SWITCH_IN_ACL_AFTER_LB_ACTION, @@ -7564,7 +7551,7 @@ build_acl_action_lflows(struct ovn_datapath *od, struct hmap *lflows, ds_put_cstr(actions, REGBIT_ACL_VERDICT_ALLOW " = 0; " REGBIT_ACL_VERDICT_DROP " = 0; " REGBIT_ACL_VERDICT_REJECT " = 0; "); - if (od->max_acl_tier) { + if (ls_stateful_rec->max_acl_tier) { ds_put_cstr(actions, REG_ACL_TIER " = 0; "); } @@ -7572,7 +7559,7 @@ build_acl_action_lflows(struct ovn_datapath *od, struct hmap *lflows, for (size_t i = 0; i < ARRAY_SIZE(stages); i++) { enum ovn_stage stage = stages[i]; - if (!od->has_acls) { + if (!ls_stateful_rec->has_acls) { ovn_lflow_add(lflows, od, stage, 0, "1", "next;"); continue; } @@ -7607,7 +7594,7 @@ build_acl_action_lflows(struct ovn_datapath *od, struct hmap *lflows, ovn_lflow_add(lflows, od, stage, 0, "1", ds_cstr(actions)); struct ds tier_actions = DS_EMPTY_INITIALIZER; - for (size_t j = 0; j < od->max_acl_tier; j++) { + for (size_t j = 0; j < ls_stateful_rec->max_acl_tier; j++) { ds_clear(match); ds_put_format(match, REG_ACL_TIER " == %"PRIuSIZE, j); ds_clear(&tier_actions); @@ -7623,7 +7610,7 @@ build_acl_action_lflows(struct ovn_datapath *od, struct hmap *lflows, } static void -build_acl_log_related_flows(struct ovn_datapath *od, struct hmap *lflows, +build_acl_log_related_flows(const struct ovn_datapath *od, struct hmap *lflows, const struct nbrec_acl *acl, bool has_stateful, bool ct_masked_mark, const struct shash *meter_groups, @@ -7696,15 +7683,19 @@ build_acl_log_related_flows(struct ovn_datapath *od, struct hmap *lflows, } static void -build_acls(struct ovn_datapath *od, const struct chassis_features *features, +build_acls(const struct ls_stateful_record *ls_stateful_rec, + const struct chassis_features *features, struct hmap *lflows, const struct ls_port_group_table *ls_port_groups, const struct shash *meter_groups) { + const struct ovn_datapath *od = ls_stateful_rec->od; + const char *default_acl_action = default_acl_drop ? debug_implicit_drop_action() : "next;"; - bool has_stateful = od->has_stateful_acl || od->has_lb_vip; + bool has_stateful = (ls_stateful_rec->has_stateful_acl + || ls_stateful_rec->has_lb_vip); const char *ct_blocked_match = features->ct_no_masked_label ? "ct_mark.blocked" : "ct_label.blocked"; @@ -7718,8 +7709,8 @@ build_acls(struct ovn_datapath *od, const struct chassis_features *features, * * A related rule at priority 1 is added below if there * are any stateful ACLs in this datapath. */ - if (!od->has_acls) { - if (!od->has_lb_vip) { + if (!ls_stateful_rec->has_acls) { + if (!ls_stateful_rec->has_lb_vip) { ovn_lflow_add(lflows, od, S_SWITCH_IN_ACL_EVAL, UINT16_MAX, "1", "next;"); ovn_lflow_add(lflows, od, S_SWITCH_OUT_ACL_EVAL, UINT16_MAX, "1", @@ -7882,7 +7873,8 @@ build_acls(struct ovn_datapath *od, const struct chassis_features *features, meter_groups, &match, &actions); consider_acl(lflows, od, acl, has_stateful, features->ct_no_masked_label, - meter_groups, &match, &actions); + meter_groups, ls_stateful_rec->max_acl_tier, + &match, &actions); } const struct ls_port_group *ls_pg = @@ -7898,7 +7890,8 @@ build_acls(struct ovn_datapath *od, const struct chassis_features *features, meter_groups, &match, &actions); consider_acl(lflows, od, acl, has_stateful, features->ct_no_masked_label, - meter_groups, &match, &actions); + meter_groups, ls_stateful_rec->max_acl_tier, + &match, &actions); } } } @@ -7916,7 +7909,7 @@ build_acls(struct ovn_datapath *od, const struct chassis_features *features, dns_actions); } - if (od->has_acls || od->has_lb_vip) { + if (ls_stateful_rec->has_acls || ls_stateful_rec->has_lb_vip) { /* Add a 34000 priority flow to advance the service monitor reply * packets to skip applying ingress ACLs. */ ovn_lflow_add(lflows, od, S_SWITCH_IN_ACL_EVAL, 34000, @@ -7930,8 +7923,8 @@ build_acls(struct ovn_datapath *od, const struct chassis_features *features, REGBIT_ACL_VERDICT_ALLOW" = 1; next;"); } - build_acl_action_lflows(od, lflows, default_acl_action, meter_groups, - &match, &actions); + build_acl_action_lflows(ls_stateful_rec, lflows, default_acl_action, + meter_groups, &match, &actions); ds_destroy(&match); ds_destroy(&actions); @@ -8576,8 +8569,11 @@ build_stateful(struct ovn_datapath *od, } static void -build_lb_hairpin(struct ovn_datapath *od, struct hmap *lflows) +build_lb_hairpin(const struct ls_stateful_record *ls_stateful_rec, + struct hmap *lflows) { + const struct ovn_datapath *od = ls_stateful_rec->od; + /* Ingress Pre-Hairpin/Nat-Hairpin/Hairpin tabled (Priority 0). * Packets that don't need hairpinning should continue processing. */ @@ -8585,7 +8581,7 @@ build_lb_hairpin(struct ovn_datapath *od, struct hmap *lflows) ovn_lflow_add(lflows, od, S_SWITCH_IN_NAT_HAIRPIN, 0, "1", "next;"); ovn_lflow_add(lflows, od, S_SWITCH_IN_HAIRPIN, 0, "1", "next;"); - if (od->has_lb_vip) { + if (ls_stateful_rec->has_lb_vip) { /* Check if the packet needs to be hairpinned. * Set REGBIT_HAIRPIN in the original direction and * REGBIT_HAIRPIN_REPLY in the reply direction. @@ -9242,9 +9238,7 @@ build_dhcpv4_options_flows(struct ovn_port *op, &op->nbsp->dhcpv4_options->options, "lease_time"); ovs_assert(server_id && server_mac && lease_time); const char *dhcp_actions = - (op->od->has_stateful_acl || op->od->has_lb_vip) - ? REGBIT_ACL_VERDICT_ALLOW" = 1; ct_commit; next;" - : REGBIT_ACL_VERDICT_ALLOW" = 1; next;"; + REGBIT_ACL_VERDICT_ALLOW" = 1; next;"; ds_clear(&match); ds_put_format(&match, "outport == %s && eth.src == %s " "&& ip4.src == %s && udp && udp.src == 67 " @@ -9327,9 +9321,7 @@ build_dhcpv6_options_flows(struct ovn_port *op, ipv6_string_mapped(server_ip, &lla); const char *dhcp6_actions = - (op->od->has_stateful_acl || op->od->has_lb_vip) - ? REGBIT_ACL_VERDICT_ALLOW" = 1; ct_commit; next;" - : REGBIT_ACL_VERDICT_ALLOW" = 1; next;"; + REGBIT_ACL_VERDICT_ALLOW" = 1; next;"; ds_clear(&match); ds_put_format(&match, "outport == %s && eth.src == %s " "&& ip6.src == %s && udp && udp.src == 547 " @@ -9436,22 +9428,16 @@ build_lswitch_lflows_l2_unknown(struct ovn_datapath *od, static void build_lswitch_lflows_pre_acl_and_acl( struct ovn_datapath *od, - const struct ls_port_group_table *ls_port_groups, const struct chassis_features *features, struct hmap *lflows, const struct shash *meter_groups) { ovs_assert(od->nbs); - ls_get_acl_flags(od, ls_port_groups); - - build_pre_acls(od, ls_port_groups, lflows); + build_pre_acls(od, lflows); build_pre_lb(od, meter_groups, lflows); build_pre_stateful(od, features, lflows); - build_acl_hints(od, features, lflows); - build_acls(od, features, lflows, ls_port_groups, meter_groups); build_qos(od, lflows); build_stateful(od, features, lflows); - build_lb_hairpin(od, lflows); build_vtep_hairpin(od, lflows); } @@ -15511,7 +15497,7 @@ build_lrouter_nat_defrag_and_lb( * a dynamically negotiated FTP data channel), but will allow * related traffic such as an ICMP Port Unreachable through * that's generated from a non-listening UDP port. */ - if (od->has_lb_vip && features->ct_lb_related) { + if (lr_stateful_rec->has_lb_vip && features->ct_lb_related) { ds_clear(match); ds_put_cstr(match, "ct.rel && !ct.est && !ct.new"); @@ -15536,7 +15522,7 @@ build_lrouter_nat_defrag_and_lb( * Pass the traffic that is already established to the next table with * proper flags set. */ - if (od->has_lb_vip) { + if (lr_stateful_rec->has_lb_vip) { ds_clear(match); ds_put_format(match, "ct.est && !ct.rel && !ct.new && %s.natted", @@ -15566,7 +15552,7 @@ build_lrouter_nat_defrag_and_lb( * not committed, it would produce ongoing datapath flows with the ct.new * flag set. Some NICs are unable to offload these flows. */ - if (od->is_gw_router && (od->nbr->n_nat || od->has_lb_vip)) { + if (od->is_gw_router && (od->nbr->n_nat || lr_stateful_rec->has_lb_vip)) { /* Do not send ND or ICMP packets to connection tracking. */ ovn_lflow_add(lflows, od, S_ROUTER_OUT_UNDNAT, 100, "nd || nd_rs || nd_ra", "next;"); @@ -15987,6 +15973,22 @@ build_lr_stateful_flows(const struct lr_stateful_record *lr_stateful_rec, meter_groups); } +static void +build_ls_stateful_flows(const struct ls_stateful_record *ls_stateful_rec, + const struct ls_port_group_table *ls_pgs, + const struct chassis_features *features, + const struct shash *meter_groups, + struct hmap *lflows) +{ + ovs_assert(ls_stateful_rec->od); + + build_ls_stateful_rec_pre_acls(ls_stateful_rec, ls_pgs, lflows); + build_ls_stateful_rec_pre_lb(ls_stateful_rec, lflows); + build_acl_hints(ls_stateful_rec, features, lflows); + build_acls(ls_stateful_rec, features, lflows, ls_pgs, meter_groups); + build_lb_hairpin(ls_stateful_rec, lflows); +} + struct lswitch_flow_build_info { const struct ovn_datapaths *ls_datapaths; const struct ovn_datapaths *lr_datapaths; @@ -15994,6 +15996,7 @@ struct lswitch_flow_build_info { const struct hmap *lr_ports; const struct ls_port_group_table *ls_port_groups; const struct lr_stateful_table *lr_stateful_table; + const struct ls_stateful_table *ls_stateful_table; struct hmap *lflows; struct hmap *igmp_groups; const struct shash *meter_groups; @@ -16018,9 +16021,7 @@ build_lswitch_and_lrouter_iterate_by_ls(struct ovn_datapath *od, struct lswitch_flow_build_info *lsi) { ovs_assert(od->nbs); - build_lswitch_lflows_pre_acl_and_acl(od, lsi->ls_port_groups, - lsi->features, - lsi->lflows, + build_lswitch_lflows_pre_acl_and_acl(od, lsi->features, lsi->lflows, lsi->meter_groups); build_fwd_group_lflows(od, lsi->lflows); @@ -16135,6 +16136,7 @@ build_lflows_thread(void *arg) { struct worker_control *control = (struct worker_control *) arg; const struct lr_stateful_record *lr_stateful_rec; + const struct ls_stateful_record *ls_stateful_rec; struct lswitch_flow_build_info *lsi; struct ovn_igmp_group *igmp_group; struct ovn_lb_datapaths *lb_dps; @@ -16259,6 +16261,20 @@ build_lflows_thread(void *arg) lsi->features); } } + + for (bnum = control->id; + bnum <= lsi->ls_stateful_table->entries.mask; + bnum += control->pool->size) + { + LS_STATEFUL_TABLE_FOR_EACH_IN_P (ls_stateful_rec, bnum, + lsi->ls_stateful_table) { + build_ls_stateful_flows(ls_stateful_rec, + lsi->ls_port_groups, + lsi->features, lsi->meter_groups, + lsi->lflows); + } + } + for (bnum = control->id; bnum <= lsi->igmp_groups->mask; bnum += control->pool->size) @@ -16320,6 +16336,7 @@ build_lswitch_and_lrouter_flows( const struct hmap *lr_ports, const struct ls_port_group_table *ls_pgs, const struct lr_stateful_table *lr_stateful_table, + const struct ls_stateful_table *ls_stateful_table, struct hmap *lflows, struct hmap *igmp_groups, const struct shash *meter_groups, @@ -16350,6 +16367,7 @@ build_lswitch_and_lrouter_flows( lsiv[index].lr_ports = lr_ports; lsiv[index].ls_port_groups = ls_pgs; lsiv[index].lr_stateful_table = lr_stateful_table; + lsiv[index].ls_stateful_table = ls_stateful_table; lsiv[index].igmp_groups = igmp_groups; lsiv[index].meter_groups = meter_groups; lsiv[index].lb_dps_map = lb_dps_map; @@ -16375,6 +16393,7 @@ build_lswitch_and_lrouter_flows( free(lsiv); } else { const struct lr_stateful_record *lr_stateful_rec; + const struct ls_stateful_record *ls_stateful_rec; struct ovn_igmp_group *igmp_group; struct ovn_lb_datapaths *lb_dps; struct ovn_datapath *od; @@ -16387,6 +16406,7 @@ build_lswitch_and_lrouter_flows( .lr_ports = lr_ports, .ls_port_groups = ls_pgs, .lr_stateful_table = lr_stateful_table, + .ls_stateful_table = ls_stateful_table, .lflows = lflows, .igmp_groups = igmp_groups, .meter_groups = meter_groups, @@ -16456,6 +16476,12 @@ build_lswitch_and_lrouter_flows( lsi.features); } + LS_STATEFUL_TABLE_FOR_EACH (ls_stateful_rec, ls_stateful_table) { + build_ls_stateful_flows(ls_stateful_rec, lsi.ls_port_groups, + lsi.features, lsi.meter_groups, + lsi.lflows); + } + stopwatch_start(LFLOWS_IGMP_STOPWATCH_NAME, time_msec()); HMAP_FOR_EACH (igmp_group, hmap_node, igmp_groups) { build_lswitch_ip_mcast_igmp_mld(igmp_group, @@ -16552,6 +16578,7 @@ void build_lflows(struct ovsdb_idl_txn *ovnsb_txn, input_data->lr_ports, input_data->ls_port_groups, input_data->lr_stateful_table, + input_data->ls_stateful_table, lflows, &igmp_groups, input_data->meter_groups, diff --git a/northd/northd.h b/northd/northd.h index 46bd2498f4..88406bffee 100644 --- a/northd/northd.h +++ b/northd/northd.h @@ -89,6 +89,8 @@ ods_size(const struct ovn_datapaths *datapaths) return hmap_count(&datapaths->datapaths); } +bool od_has_lb_vip(const struct ovn_datapath *od); + struct tracked_ovn_ports { /* tracked created ports. * hmapx node data is 'struct ovn_port *' */ @@ -118,6 +120,8 @@ enum northd_tracked_data_type { NORTHD_TRACKED_PORTS = (1 << 0), NORTHD_TRACKED_LBS = (1 << 1), NORTHD_TRACKED_LR_NATS = (1 << 2), + NORTHD_TRACKED_LS_LBS = (1 << 3), + NORTHD_TRACKED_LS_ACLS = (1 << 4), }; /* Track what's changed in the northd engine node. @@ -132,6 +136,14 @@ struct northd_tracked_data { /* Tracked logical routers whose NATs have changed. * hmapx node is 'struct ovn_datapath *'. */ struct hmapx lr_with_changed_nats; + + /* Tracked logical switches whose load balancers have changed. + * hmapx node is 'struct ovn_datapath *'. */ + struct hmapx ls_with_changed_lbs; + + /* Tracked logical switches whose ACLs have changed. + * hmapx node is 'struct ovn_datapath *'. */ + struct hmapx ls_with_changed_acls; }; struct northd_data { @@ -180,6 +192,7 @@ struct lflow_input { const struct hmap *lr_ports; const struct ls_port_group_table *ls_port_groups; const struct lr_stateful_table *lr_stateful_table; + const struct ls_stateful_table *ls_stateful_table; const struct shash *meter_groups; const struct hmap *lb_datapaths_map; const struct hmap *bfd_connections; @@ -289,11 +302,7 @@ struct ovn_datapath { struct hmap port_tnlids; uint32_t port_key_hint; - bool has_stateful_acl; - bool has_lb_vip; bool has_unknown; - bool has_acls; - uint64_t max_acl_tier; bool has_vtep_lports; bool has_arp_proxy_port; @@ -541,6 +550,18 @@ northd_has_lr_nats_in_tracked_data(struct northd_tracked_data *trk_nd_changes) return (trk_nd_changes->type & NORTHD_TRACKED_LR_NATS); } +static inline bool +northd_has_ls_lbs_in_tracked_data(struct northd_tracked_data *trk_nd_changes) +{ + return (trk_nd_changes->type & NORTHD_TRACKED_LS_LBS); +} + +static inline bool +northd_has_ls_acls_in_tracked_data(struct northd_tracked_data *trk_nd_changes) +{ + return (trk_nd_changes->type & NORTHD_TRACKED_LS_ACLS); +} + /* Returns 'true' if the IPv4 'addr' is on the same subnet with one of the * IPs configured on the router port. */ diff --git a/northd/ovn-northd.c b/northd/ovn-northd.c index e85e00cc9b..084d675644 100644 --- a/northd/ovn-northd.c +++ b/northd/ovn-northd.c @@ -872,6 +872,7 @@ main(int argc, char *argv[]) stopwatch_create(SYNC_METERS_RUN_STOPWATCH_NAME, SW_MS); stopwatch_create(LR_NAT_RUN_STOPWATCH_NAME, SW_MS); stopwatch_create(LR_STATEFUL_RUN_STOPWATCH_NAME, SW_MS); + stopwatch_create(LS_STATEFUL_RUN_STOPWATCH_NAME, SW_MS); /* Initialize incremental processing engine for ovn-northd */ inc_proc_northd_init(&ovnnb_idl_loop, &ovnsb_idl_loop); From patchwork Thu Jan 11 15:31:11 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Numan Siddique X-Patchwork-Id: 1885677 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::138; helo=smtp1.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp1.osuosl.org (smtp1.osuosl.org [IPv6:2605:bc80:3010::138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4T9pYw6SY9z1yPp for ; Fri, 12 Jan 2024 02:32:04 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 7FB7083FEE; Thu, 11 Jan 2024 15:32:02 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 7FB7083FEE X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id KJm6PUtaHyvi; Thu, 11 Jan 2024 15:31:58 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp1.osuosl.org (Postfix) with ESMTPS id 47AF984029; Thu, 11 Jan 2024 15:31:57 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 47AF984029 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 18531C007C; Thu, 11 Jan 2024 15:31:57 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp2.osuosl.org (smtp2.osuosl.org [IPv6:2605:bc80:3010::133]) by lists.linuxfoundation.org (Postfix) with ESMTP id 3EA1CC007C for ; Thu, 11 Jan 2024 15:31:56 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 09810438A1 for ; Thu, 11 Jan 2024 15:31:54 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 09810438A1 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id P4FftbjVwnJs for ; Thu, 11 Jan 2024 15:31:49 +0000 (UTC) Received: from relay1-d.mail.gandi.net (relay1-d.mail.gandi.net [IPv6:2001:4b98:dc4:8::221]) by smtp2.osuosl.org (Postfix) with ESMTPS id 6B77A438DE for ; Thu, 11 Jan 2024 15:31:48 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 6B77A438DE Received: by mail.gandi.net (Postfix) with ESMTPSA id 3CC07240009; Thu, 11 Jan 2024 15:31:43 +0000 (UTC) From: numans@ovn.org To: dev@openvswitch.org Date: Thu, 11 Jan 2024 10:31:11 -0500 Message-ID: <20240111153112.2790251-1-numans@ovn.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240111152752.2789854-1-numans@ovn.org> References: <20240111152752.2789854-1-numans@ovn.org> MIME-Version: 1.0 X-GND-Sasl: numans@ovn.org Subject: [ovs-dev] [PATCH ovn v5 08/16] northd: Refactor lflow management into a separate module. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Numan Siddique ovn_lflow_add() and other related functions/macros are now moved into a separate module - lflow-mgr.c. This module maintains a table 'struct lflow_table' for the logical flows. lflow table maintains a hmap to store the logical flows. It also maintains the logical switch and router dp groups. Previous commits which added lflow incremental processing for the VIF logical ports, stored the references to the logical ports' lflows using 'struct lflow_ref_list'. This struct is renamed to 'struct lflow_ref' and is part of lflow-mgr.c. It is modified a bit to store the resource to lflow references. Example usage of 'struct lflow_ref'. 'struct ovn_port' maintains 2 instances of lflow_ref. i,e struct ovn_port { ... ... struct lflow_ref *lflow_ref; struct lflow_ref *stateful_lflow_ref; }; All the logical flows generated by build_lswitch_and_lrouter_iterate_by_lsp() uses the ovn_port->lflow_ref. All the logical flows generated by build_lsp_lflows_for_lbnats() uses the ovn_port->stateful_lflow_ref. When handling the ovn_port changes incrementally, the lflows referenced in 'struct ovn_port' are cleared and regenerated and synced to the SB logical flows. eg. lflow_ref_clear_lflows(op->lflow_ref); build_lswitch_and_lrouter_iterate_by_lsp(op, ...); lflow_ref_sync_lflows_to_sb(op->lflow_ref, ...); This patch does few more changes: - Logical flows are now hashed without the logical datapaths. If a logical flow is referenced by just one datapath, we don't rehash it. - The synthetic 'hash' column of sbrec_logical_flow now doesn't use the logical datapath. This means that when ovn-northd is updated/upgraded and has this commit, all the logical flows with 'logical_datapath' column set will get deleted and re-added causing some disruptions. - With the commit [1] which added I-P support for logical port changes, multiple logical flows with same match 'M' and actions 'A' are generated and stored without the dp groups, which was not the case prior to that patch. One example to generate these lflows is: ovn-nbctl lsp-set-addresses sw0p1 "MAC1 IP1" ovn-nbctl lsp-set-addresses sw1p1 "MAC1 IP1" ovn-nbctl lsp-set-addresses sw2p1 "MAC1 IP1" Now with this patch we go back to the earlier way. i.e one logical flow with logical_dp_groups set. - With this patch any updates to a logical port which doesn't result in new logical flows will not result in deletion and addition of same logical flows. Eg. ovn-nbctl set logical_switch_port sw0p1 external_ids:foo=bar will be a no-op to the SB logical flow table. [1] - 8bbd678("northd: Incremental processing of VIF additions in 'lflow' node.") Signed-off-by: Numan Siddique --- lib/ovn-util.c | 18 +- lib/ovn-util.h | 2 - northd/automake.mk | 4 +- northd/en-lflow.c | 24 +- northd/en-lflow.h | 6 + northd/inc-proc-northd.c | 4 +- northd/lflow-mgr.c | 1261 +++++++++++++++++++++++++ northd/lflow-mgr.h | 186 ++++ northd/northd.c | 1918 ++++++++++---------------------------- northd/northd.h | 218 ++++- northd/ovn-northd.c | 4 + tests/ovn-northd.at | 216 +++++ 12 files changed, 2395 insertions(+), 1466 deletions(-) create mode 100644 northd/lflow-mgr.c create mode 100644 northd/lflow-mgr.h diff --git a/lib/ovn-util.c b/lib/ovn-util.c index c8b89cc216..ba29baea63 100644 --- a/lib/ovn-util.c +++ b/lib/ovn-util.c @@ -620,13 +620,10 @@ ovn_pipeline_from_name(const char *pipeline) uint32_t sbrec_logical_flow_hash(const struct sbrec_logical_flow *lf) { - const struct sbrec_datapath_binding *ld = lf->logical_datapath; - uint32_t hash = ovn_logical_flow_hash(lf->table_id, - ovn_pipeline_from_name(lf->pipeline), - lf->priority, lf->match, - lf->actions); - - return ld ? ovn_logical_flow_hash_datapath(&ld->header_.uuid, hash) : hash; + return ovn_logical_flow_hash(lf->table_id, + ovn_pipeline_from_name(lf->pipeline), + lf->priority, lf->match, + lf->actions); } uint32_t @@ -639,13 +636,6 @@ ovn_logical_flow_hash(uint8_t table_id, enum ovn_pipeline pipeline, return hash_string(actions, hash); } -uint32_t -ovn_logical_flow_hash_datapath(const struct uuid *logical_datapath, - uint32_t hash) -{ - return hash_add(hash, uuid_hash(logical_datapath)); -} - struct tnlid_node { struct hmap_node hmap_node; diff --git a/lib/ovn-util.h b/lib/ovn-util.h index d245d57d56..1c430027c6 100644 --- a/lib/ovn-util.h +++ b/lib/ovn-util.h @@ -145,8 +145,6 @@ uint32_t sbrec_logical_flow_hash(const struct sbrec_logical_flow *); uint32_t ovn_logical_flow_hash(uint8_t table_id, enum ovn_pipeline pipeline, uint16_t priority, const char *match, const char *actions); -uint32_t ovn_logical_flow_hash_datapath(const struct uuid *logical_datapath, - uint32_t hash); void ovn_conn_show(struct unixctl_conn *conn, int argc OVS_UNUSED, const char *argv[] OVS_UNUSED, void *idl_); diff --git a/northd/automake.mk b/northd/automake.mk index a178541759..7c6d56a4ff 100644 --- a/northd/automake.mk +++ b/northd/automake.mk @@ -33,7 +33,9 @@ northd_ovn_northd_SOURCES = \ northd/inc-proc-northd.c \ northd/inc-proc-northd.h \ northd/ipam.c \ - northd/ipam.h + northd/ipam.h \ + northd/lflow-mgr.c \ + northd/lflow-mgr.h northd_ovn_northd_LDADD = \ lib/libovn.la \ $(OVSDB_LIBDIR)/libovsdb.la \ diff --git a/northd/en-lflow.c b/northd/en-lflow.c index eb6b2a8666..fef9a1352d 100644 --- a/northd/en-lflow.c +++ b/northd/en-lflow.c @@ -24,6 +24,7 @@ #include "en-ls-stateful.h" #include "en-northd.h" #include "en-meters.h" +#include "lflow-mgr.h" #include "lib/inc-proc-eng.h" #include "northd.h" @@ -58,6 +59,8 @@ lflow_get_input_data(struct engine_node *node, EN_OVSDB_GET(engine_get_input("SB_multicast_group", node)); lflow_input->sbrec_igmp_group_table = EN_OVSDB_GET(engine_get_input("SB_igmp_group", node)); + lflow_input->sbrec_logical_dp_group_table = + EN_OVSDB_GET(engine_get_input("SB_logical_dp_group", node)); lflow_input->sbrec_mcast_group_by_name_dp = engine_ovsdb_node_get_index( @@ -90,17 +93,19 @@ void en_lflow_run(struct engine_node *node, void *data) struct hmap bfd_connections = HMAP_INITIALIZER(&bfd_connections); lflow_input.bfd_connections = &bfd_connections; + stopwatch_start(BUILD_LFLOWS_STOPWATCH_NAME, time_msec()); + struct lflow_data *lflow_data = data; - lflow_data_destroy(lflow_data); - lflow_data_init(lflow_data); + lflow_table_clear(lflow_data->lflow_table); + lflow_reset_northd_refs(&lflow_input); - stopwatch_start(BUILD_LFLOWS_STOPWATCH_NAME, time_msec()); build_bfd_table(eng_ctx->ovnsb_idl_txn, lflow_input.nbrec_bfd_table, lflow_input.sbrec_bfd_table, lflow_input.lr_ports, &bfd_connections); - build_lflows(eng_ctx->ovnsb_idl_txn, &lflow_input, &lflow_data->lflows); + build_lflows(eng_ctx->ovnsb_idl_txn, &lflow_input, + lflow_data->lflow_table); bfd_cleanup_connections(lflow_input.nbrec_bfd_table, &bfd_connections); hmap_destroy(&bfd_connections); @@ -131,7 +136,7 @@ lflow_northd_handler(struct engine_node *node, if (!lflow_handle_northd_port_changes(eng_ctx->ovnsb_idl_txn, &northd_data->trk_data.trk_lsps, - &lflow_input, &lflow_data->lflows)) { + &lflow_input, lflow_data->lflow_table)) { return false; } @@ -160,11 +165,14 @@ void *en_lflow_init(struct engine_node *node OVS_UNUSED, struct engine_arg *arg OVS_UNUSED) { struct lflow_data *data = xmalloc(sizeof *data); - lflow_data_init(data); + data->lflow_table = lflow_table_alloc(); + lflow_table_init(data->lflow_table); return data; } -void en_lflow_cleanup(void *data) +void en_lflow_cleanup(void *data_) { - lflow_data_destroy(data); + struct lflow_data *data = data_; + lflow_table_destroy(data->lflow_table); + data->lflow_table = NULL; } diff --git a/northd/en-lflow.h b/northd/en-lflow.h index 5417b2faff..f7325c56b1 100644 --- a/northd/en-lflow.h +++ b/northd/en-lflow.h @@ -9,6 +9,12 @@ #include "lib/inc-proc-eng.h" +struct lflow_table; + +struct lflow_data { + struct lflow_table *lflow_table; +}; + void en_lflow_run(struct engine_node *node, void *data); void *en_lflow_init(struct engine_node *node, struct engine_arg *arg); void en_lflow_cleanup(void *data); diff --git a/northd/inc-proc-northd.c b/northd/inc-proc-northd.c index 9ce4279ee8..0e17bfe2e6 100644 --- a/northd/inc-proc-northd.c +++ b/northd/inc-proc-northd.c @@ -99,7 +99,8 @@ static unixctl_cb_func chassis_features_list; SB_NODE(bfd, "bfd") \ SB_NODE(fdb, "fdb") \ SB_NODE(static_mac_binding, "static_mac_binding") \ - SB_NODE(chassis_template_var, "chassis_template_var") + SB_NODE(chassis_template_var, "chassis_template_var") \ + SB_NODE(logical_dp_group, "logical_dp_group") enum sb_engine_node { #define SB_NODE(NAME, NAME_STR) SB_##NAME, @@ -229,6 +230,7 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, engine_add_input(&en_lflow, &en_sb_igmp_group, NULL); engine_add_input(&en_lflow, &en_lr_stateful, NULL); engine_add_input(&en_lflow, &en_ls_stateful, NULL); + engine_add_input(&en_lflow, &en_sb_logical_dp_group, NULL); engine_add_input(&en_lflow, &en_northd, lflow_northd_handler); engine_add_input(&en_lflow, &en_port_group, lflow_port_group_handler); diff --git a/northd/lflow-mgr.c b/northd/lflow-mgr.c new file mode 100644 index 0000000000..3cf9696f6e --- /dev/null +++ b/northd/lflow-mgr.c @@ -0,0 +1,1261 @@ +/* + * Copyright (c) 2023, Red Hat, Inc. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at: + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#include + +#include +#include +#include + +/* OVS includes */ +#include "include/openvswitch/hmap.h" +#include "include/openvswitch/thread.h" +#include "lib/bitmap.h" +#include "lib/uuidset.h" +#include "openvswitch/util.h" +#include "openvswitch/vlog.h" +#include "ovs-thread.h" +#include "stopwatch.h" + +/* OVN includes */ +#include "debug.h" +#include "lflow-mgr.h" +#include "lib/ovn-parallel-hmap.h" +#include "northd.h" + +VLOG_DEFINE_THIS_MODULE(lflow_mgr); + +/* Static function declarations. */ +struct ovn_lflow; + +static void ovn_lflow_init(struct ovn_lflow *, struct ovn_datapath *od, + size_t dp_bitmap_len, enum ovn_stage stage, + uint16_t priority, char *match, + char *actions, char *io_port, + char *ctrl_meter, char *stage_hint, + const char *where); +static struct ovn_lflow *ovn_lflow_find(const struct hmap *lflows, + enum ovn_stage stage, + uint16_t priority, const char *match, + const char *actions, + const char *ctrl_meter, uint32_t hash); +static void ovn_lflow_destroy(struct lflow_table *lflow_table, + struct ovn_lflow *lflow); +static char *ovn_lflow_hint(const struct ovsdb_idl_row *row); + +static struct ovn_lflow *do_ovn_lflow_add( + struct lflow_table *, const struct ovn_datapath *, + const unsigned long *dp_bitmap, size_t dp_bitmap_len, uint32_t hash, + enum ovn_stage stage, uint16_t priority, const char *match, + const char *actions, const char *io_port, + const char *ctrl_meter, + const struct ovsdb_idl_row *stage_hint, + const char *where); + + +static struct ovs_mutex *lflow_hash_lock(const struct hmap *lflow_table, + uint32_t hash); +static void lflow_hash_unlock(struct ovs_mutex *hash_lock); + +static struct ovn_dp_group *ovn_dp_group_get( + struct hmap *dp_groups, size_t desired_n, + const unsigned long *desired_bitmap, + size_t bitmap_len); +static struct ovn_dp_group *ovn_dp_group_create( + struct ovsdb_idl_txn *ovnsb_txn, struct hmap *dp_groups, + struct sbrec_logical_dp_group *, size_t desired_n, + const unsigned long *desired_bitmap, + size_t bitmap_len, bool is_switch, + const struct ovn_datapaths *ls_datapaths, + const struct ovn_datapaths *lr_datapaths); +static struct ovn_dp_group *ovn_dp_group_get( + struct hmap *dp_groups, size_t desired_n, + const unsigned long *desired_bitmap, + size_t bitmap_len); +static struct sbrec_logical_dp_group *ovn_sb_insert_or_update_logical_dp_group( + struct ovsdb_idl_txn *ovnsb_txn, + struct sbrec_logical_dp_group *, + const unsigned long *dpg_bitmap, + const struct ovn_datapaths *); +static struct ovn_dp_group *ovn_dp_group_find(const struct hmap *dp_groups, + const unsigned long *dpg_bitmap, + size_t bitmap_len, + uint32_t hash); +static void inc_ovn_dp_group_ref(struct ovn_dp_group *); +static void dec_ovn_dp_group_ref(struct hmap *dp_groups, + struct ovn_dp_group *); +static void ovn_dp_group_add_with_reference(struct ovn_lflow *, + const struct ovn_datapath *od, + const unsigned long *dp_bitmap, + size_t bitmap_len); + +static bool lflow_ref_sync_lflows__( + struct lflow_ref *, struct lflow_table *, + struct ovsdb_idl_txn *ovnsb_txn, + const struct ovn_datapaths *ls_datapaths, + const struct ovn_datapaths *lr_datapaths, + bool ovn_internal_version_changed, + const struct sbrec_logical_flow_table *, + const struct sbrec_logical_dp_group_table *); +static bool sync_lflow_to_sb(struct ovn_lflow *, + struct ovsdb_idl_txn *ovnsb_txn, + struct lflow_table *, + const struct ovn_datapaths *ls_datapaths, + const struct ovn_datapaths *lr_datapaths, + bool ovn_internal_version_changed, + const struct sbrec_logical_flow *sbflow, + const struct sbrec_logical_dp_group_table *); + +extern int parallelization_state; +extern thread_local size_t thread_lflow_counter; + +struct dp_refcnt; +static struct dp_refcnt *dp_refcnt_find(struct hmap *dp_refcnts_map, + size_t dp_index); +static void inc_dp_refcnt(struct hmap *dp_refcnts_map, size_t dp_index); +static bool dec_dp_refcnt(struct hmap *dp_refcnts_map, size_t dp_index); +static void ovn_lflow_clear_dp_refcnts_map(struct ovn_lflow *); +static struct lflow_ref_node *lflow_ref_node_find(struct hmap *lflow_ref_nodes, + struct ovn_lflow *lflow, + uint32_t lflow_hash); +static void lflow_ref_node_destroy(struct lflow_ref_node *, + struct hmap *lflow_ref_nodes); + +static bool lflow_hash_lock_initialized = false; +/* The lflow_hash_lock is a mutex array that protects updates to the shared + * lflow table across threads when parallel lflow build and dp-group are both + * enabled. To avoid high contention between threads, a big array of mutexes + * are used instead of just one. This is possible because when parallel build + * is used we only use hmap_insert_fast() to update the hmap, which would not + * touch the bucket array but only the list in a single bucket. We only need to + * make sure that when adding lflows to the same hash bucket, the same lock is + * used, so that no two threads can add to the bucket at the same time. It is + * ok that the same lock is used to protect multiple buckets, so a fixed sized + * mutex array is used instead of 1-1 mapping to the hash buckets. This + * simplies the implementation while effectively reduces lock contention + * because the chance that different threads contending the same lock amongst + * the big number of locks is very low. */ +#define LFLOW_HASH_LOCK_MASK 0xFFFF +static struct ovs_mutex lflow_hash_locks[LFLOW_HASH_LOCK_MASK + 1]; + +/* Full thread safety analysis is not possible with hash locks, because + * they are taken conditionally based on the 'parallelization_state' and + * a flow hash. Also, the order in which two hash locks are taken is not + * predictable during the static analysis. + * + * Since the order of taking two locks depends on a random hash, to avoid + * ABBA deadlocks, no two hash locks can be nested. In that sense an array + * of hash locks is similar to a single mutex. + * + * Using a fake mutex to partially simulate thread safety restrictions, as + * if it were actually a single mutex. + * + * OVS_NO_THREAD_SAFETY_ANALYSIS below allows us to ignore conditional + * nature of the lock. Unlike other attributes, it applies to the + * implementation and not to the interface. So, we can define a function + * that acquires the lock without analysing the way it does that. + */ +extern struct ovs_mutex fake_hash_mutex; + +/* Represents a logical ovn flow (lflow). + * + * A logical flow with match 'M' and actions 'A' - L(M, A) is created + * when lflow engine node (northd.c) calls lflow_table_add_lflow + * (or one of the helper macros ovn_lflow_add_*). + * + * Each lflow is stored in the lflow_table (see 'struct lflow_table' below) + * and possibly referenced by zero or more lflow_refs + * (see 'struct lflow_ref' and 'struct lflow_ref_node' below). + * + * */ +struct ovn_lflow { + struct hmap_node hmap_node; + + struct ovn_datapath *od; /* 'logical_datapath' in SB schema. */ + unsigned long *dpg_bitmap; /* Bitmap of all datapaths by their 'index'.*/ + enum ovn_stage stage; + uint16_t priority; + char *match; + char *actions; + char *io_port; + char *stage_hint; + char *ctrl_meter; + size_t n_ods; /* Number of datapaths referenced by 'od' and + * 'dpg_bitmap'. */ + struct ovn_dp_group *dpg; /* Link to unique Sb datapath group. */ + const char *where; + + struct uuid sb_uuid; /* SB DB row uuid, specified by northd. */ + struct ovs_list referenced_by; /* List of struct lflow_ref_node. */ + struct hmap dp_refcnts_map; /* Maintains the number of times this ovn_lflow + * is referenced by a given datapath. + * Contains 'struct dp_refcnt' in the map. */ +}; + +/* Logical flow table. */ +struct lflow_table { + struct hmap entries; /* hmap of lflows. */ + struct hmap ls_dp_groups; /* hmap of logical switch dp groups. */ + struct hmap lr_dp_groups; /* hmap of logical router dp groups. */ + ssize_t max_seen_lflow_size; +}; + +struct lflow_table * +lflow_table_alloc(void) +{ + struct lflow_table *lflow_table = xzalloc(sizeof *lflow_table); + lflow_table->max_seen_lflow_size = 128; + + return lflow_table; +} + +void +lflow_table_init(struct lflow_table *lflow_table) +{ + fast_hmap_size_for(&lflow_table->entries, + lflow_table->max_seen_lflow_size); + ovn_dp_groups_init(&lflow_table->ls_dp_groups); + ovn_dp_groups_init(&lflow_table->lr_dp_groups); +} + +void +lflow_table_clear(struct lflow_table *lflow_table) +{ + struct ovn_lflow *lflow; + HMAP_FOR_EACH_POP (lflow, hmap_node, &lflow_table->entries) { + ovn_lflow_destroy(NULL, lflow); + } + + ovn_dp_groups_clear(&lflow_table->ls_dp_groups); + ovn_dp_groups_clear(&lflow_table->lr_dp_groups); +} + +void +lflow_table_destroy(struct lflow_table *lflow_table) +{ + lflow_table_clear(lflow_table); + hmap_destroy(&lflow_table->entries); + ovn_dp_groups_destroy(&lflow_table->ls_dp_groups); + ovn_dp_groups_destroy(&lflow_table->lr_dp_groups); + free(lflow_table); +} + +void +lflow_table_expand(struct lflow_table *lflow_table) +{ + hmap_expand(&lflow_table->entries); + + if (hmap_count(&lflow_table->entries) > + lflow_table->max_seen_lflow_size) { + lflow_table->max_seen_lflow_size = hmap_count(&lflow_table->entries); + } +} + +void +lflow_table_set_size(struct lflow_table *lflow_table, size_t size) +{ + lflow_table->entries.n = size; +} + +void +lflow_table_sync_to_sb(struct lflow_table *lflow_table, + struct ovsdb_idl_txn *ovnsb_txn, + const struct ovn_datapaths *ls_datapaths, + const struct ovn_datapaths *lr_datapaths, + bool ovn_internal_version_changed, + const struct sbrec_logical_flow_table *sb_flow_table, + const struct sbrec_logical_dp_group_table *dpgrp_table) +{ + struct hmap lflows_temp = HMAP_INITIALIZER(&lflows_temp); + struct hmap *lflows = &lflow_table->entries; + struct ovn_lflow *lflow; + + /* Push changes to the Logical_Flow table to database. */ + const struct sbrec_logical_flow *sbflow; + SBREC_LOGICAL_FLOW_TABLE_FOR_EACH_SAFE (sbflow, sb_flow_table) { + struct sbrec_logical_dp_group *dp_group = sbflow->logical_dp_group; + struct ovn_datapath *logical_datapath_od = NULL; + size_t i; + + /* Find one valid datapath to get the datapath type. */ + struct sbrec_datapath_binding *dp = sbflow->logical_datapath; + if (dp) { + logical_datapath_od = ovn_datapath_from_sbrec( + &ls_datapaths->datapaths, &lr_datapaths->datapaths, dp); + if (logical_datapath_od + && ovn_datapath_is_stale(logical_datapath_od)) { + logical_datapath_od = NULL; + } + } + for (i = 0; dp_group && i < dp_group->n_datapaths; i++) { + logical_datapath_od = ovn_datapath_from_sbrec( + &ls_datapaths->datapaths, &lr_datapaths->datapaths, + dp_group->datapaths[i]); + if (logical_datapath_od + && !ovn_datapath_is_stale(logical_datapath_od)) { + break; + } + logical_datapath_od = NULL; + } + + if (!logical_datapath_od) { + /* This lflow has no valid logical datapaths. */ + sbrec_logical_flow_delete(sbflow); + continue; + } + + enum ovn_pipeline pipeline + = !strcmp(sbflow->pipeline, "ingress") ? P_IN : P_OUT; + + lflow = ovn_lflow_find( + lflows, + ovn_stage_build(ovn_datapath_get_type(logical_datapath_od), + pipeline, sbflow->table_id), + sbflow->priority, sbflow->match, sbflow->actions, + sbflow->controller_meter, sbflow->hash); + if (lflow) { + sync_lflow_to_sb(lflow, ovnsb_txn, lflow_table, ls_datapaths, + lr_datapaths, ovn_internal_version_changed, + sbflow, dpgrp_table); + + hmap_remove(lflows, &lflow->hmap_node); + hmap_insert(&lflows_temp, &lflow->hmap_node, + hmap_node_hash(&lflow->hmap_node)); + } else { + sbrec_logical_flow_delete(sbflow); + } + } + + HMAP_FOR_EACH_SAFE (lflow, hmap_node, lflows) { + sync_lflow_to_sb(lflow, ovnsb_txn, lflow_table, ls_datapaths, + lr_datapaths, ovn_internal_version_changed, + NULL, dpgrp_table); + + hmap_remove(lflows, &lflow->hmap_node); + hmap_insert(&lflows_temp, &lflow->hmap_node, + hmap_node_hash(&lflow->hmap_node)); + } + hmap_swap(lflows, &lflows_temp); + hmap_destroy(&lflows_temp); +} + +/* lflow ref */ +struct lflow_ref { + /* head of the list 'struct lflow_ref_node'. */ + struct ovs_list lflows_ref_list; + + /* hmap_node is 'struct lflow_ref_node *'. This is used to ensure + * that there are no duplicates in 'lflow_ref_list' above. */ + struct hmap lflow_ref_nodes; +}; + +struct lflow_ref_node { + /* hmap node in the hmap - 'struct lflow_ref->lflow_ref_nodes' */ + struct hmap_node ref_node; + + /* This list follows different lflows referenced by the + * 'struct lflow_ref'. List head is lflow_ref->lflows_ref_list. */ + struct ovs_list lflow_list_node; + /* This list follows different objects that reference the same lflow. List + * head is ovn_lflow->referenced_by. */ + struct ovs_list ref_list_node; + /* The lflow. */ + struct ovn_lflow *lflow; + + /* Index id of the datapath this lflow_ref_node belongs to. */ + size_t dp_index; + + /* Indicates if the lflow_ref_node for an lflow - L(M, A) is linked + * to datapath(s) or not. + * It is set to true when an lflow L(M, A) is referenced by an lflow ref + * in lflow_table_add_lflow(). It is set to false when it is unlinked + * from the datapath when lflow_ref_unlink_lflows() is called. */ + bool linked; +}; + +struct lflow_ref * +lflow_ref_create(void) +{ + struct lflow_ref *lflow_ref = xzalloc(sizeof *lflow_ref); + ovs_list_init(&lflow_ref->lflows_ref_list); + hmap_init(&lflow_ref->lflow_ref_nodes); + return lflow_ref; +} + +void +lflow_ref_destroy(struct lflow_ref *lflow_ref) +{ + struct lflow_ref_node *l; + + LIST_FOR_EACH_SAFE (l, lflow_list_node, &lflow_ref->lflows_ref_list) { + lflow_ref_node_destroy(l, NULL); + } + + hmap_destroy(&lflow_ref->lflow_ref_nodes); + free(lflow_ref); +} + +void +lflow_ref_clear(struct lflow_ref *lflow_ref) +{ + struct lflow_ref_node *l; + LIST_FOR_EACH_SAFE (l, lflow_list_node, &lflow_ref->lflows_ref_list) { + lflow_ref_node_destroy(l, NULL); + } + + hmap_clear(&lflow_ref->lflow_ref_nodes); +} + +/* Unlinks the lflows referenced by the 'lflow_ref'. + * For each lflow_ref_node (lrn) in the lflow_ref, it basically clears + * the datapath id (lrn->dp_index) from the lrn->lflow's dpg bitmap. + */ +void +lflow_ref_unlink_lflows(struct lflow_ref *lflow_ref) +{ + struct lflow_ref_node *lrn; + + LIST_FOR_EACH (lrn, lflow_list_node, &lflow_ref->lflows_ref_list) { + if (dec_dp_refcnt(&lrn->lflow->dp_refcnts_map, + lrn->dp_index)) { + bitmap_set0(lrn->lflow->dpg_bitmap, lrn->dp_index); + } + + lrn->linked = false; + } +} + +bool +lflow_ref_resync_flows(struct lflow_ref *lflow_ref, + struct lflow_table *lflow_table, + struct ovsdb_idl_txn *ovnsb_txn, + const struct ovn_datapaths *ls_datapaths, + const struct ovn_datapaths *lr_datapaths, + bool ovn_internal_version_changed, + const struct sbrec_logical_flow_table *sbflow_table, + const struct sbrec_logical_dp_group_table *dpgrp_table) +{ + lflow_ref_unlink_lflows(lflow_ref); + return lflow_ref_sync_lflows__(lflow_ref, lflow_table, ovnsb_txn, + ls_datapaths, lr_datapaths, + ovn_internal_version_changed, sbflow_table, + dpgrp_table); +} + +bool +lflow_ref_sync_lflows(struct lflow_ref *lflow_ref, + struct lflow_table *lflow_table, + struct ovsdb_idl_txn *ovnsb_txn, + const struct ovn_datapaths *ls_datapaths, + const struct ovn_datapaths *lr_datapaths, + bool ovn_internal_version_changed, + const struct sbrec_logical_flow_table *sbflow_table, + const struct sbrec_logical_dp_group_table *dpgrp_table) +{ + return lflow_ref_sync_lflows__(lflow_ref, lflow_table, ovnsb_txn, + ls_datapaths, lr_datapaths, + ovn_internal_version_changed, sbflow_table, + dpgrp_table); +} + +void +lflow_table_add_lflow(struct lflow_table *lflow_table, + const struct ovn_datapath *od, + const unsigned long *dp_bitmap, size_t dp_bitmap_len, + enum ovn_stage stage, uint16_t priority, + const char *match, const char *actions, + const char *io_port, const char *ctrl_meter, + const struct ovsdb_idl_row *stage_hint, + const char *where, + struct lflow_ref *lflow_ref) + OVS_EXCLUDED(fake_hash_mutex) +{ + struct ovs_mutex *hash_lock; + uint32_t hash; + + ovs_assert(!od || + ovn_stage_to_datapath_type(stage) == ovn_datapath_get_type(od)); + + hash = ovn_logical_flow_hash(ovn_stage_get_table(stage), + ovn_stage_get_pipeline(stage), + priority, match, + actions); + + hash_lock = lflow_hash_lock(&lflow_table->entries, hash); + struct ovn_lflow *lflow = + do_ovn_lflow_add(lflow_table, od, dp_bitmap, + dp_bitmap_len, hash, stage, + priority, match, actions, + io_port, ctrl_meter, stage_hint, where); + + if (lflow_ref) { + /* lflow referencing is only supported if 'od' is not NULL. */ + ovs_assert(od); + + struct lflow_ref_node *lrn = + lflow_ref_node_find(&lflow_ref->lflow_ref_nodes, lflow, hash); + if (!lrn) { + lrn = xzalloc(sizeof *lrn); + lrn->lflow = lflow; + lrn->dp_index = od->index; + ovs_list_insert(&lflow_ref->lflows_ref_list, + &lrn->lflow_list_node); + inc_dp_refcnt(&lflow->dp_refcnts_map, lrn->dp_index); + ovs_list_insert(&lflow->referenced_by, &lrn->ref_list_node); + + hmap_insert(&lflow_ref->lflow_ref_nodes, &lrn->ref_node, hash); + } + + lrn->linked = true; + } + + lflow_hash_unlock(hash_lock); + +} + +void +lflow_table_add_lflow_default_drop(struct lflow_table *lflow_table, + const struct ovn_datapath *od, + enum ovn_stage stage, + const char *where, + struct lflow_ref *lflow_ref) +{ + lflow_table_add_lflow(lflow_table, od, NULL, 0, stage, 0, "1", + debug_drop_action(), NULL, NULL, NULL, + where, lflow_ref); +} + +/* Given a desired bitmap, finds a datapath group in 'dp_groups'. If it + * doesn't exist, creates a new one and adds it to 'dp_groups'. + * If 'sb_group' is provided, function will try to re-use this group by + * either taking it directly, or by modifying, if it's not already in use. */ +struct ovn_dp_group * +ovn_dp_group_get_or_create(struct ovsdb_idl_txn *ovnsb_txn, + struct hmap *dp_groups, + struct sbrec_logical_dp_group *sb_group, + size_t desired_n, + const unsigned long *desired_bitmap, + size_t bitmap_len, + bool is_switch, + const struct ovn_datapaths *ls_datapaths, + const struct ovn_datapaths *lr_datapaths) +{ + struct ovn_dp_group *dpg; + + dpg = ovn_dp_group_get(dp_groups, desired_n, desired_bitmap, bitmap_len); + if (dpg) { + return dpg; + } + + return ovn_dp_group_create(ovnsb_txn, dp_groups, sb_group, desired_n, + desired_bitmap, bitmap_len, is_switch, + ls_datapaths, lr_datapaths); +} + +void +ovn_dp_groups_clear(struct hmap *dp_groups) +{ + struct ovn_dp_group *dpg; + HMAP_FOR_EACH_POP (dpg, node, dp_groups) { + bitmap_free(dpg->bitmap); + free(dpg); + } +} + +void +ovn_dp_groups_destroy(struct hmap *dp_groups) +{ + ovn_dp_groups_clear(dp_groups); + hmap_destroy(dp_groups); +} + +void +lflow_hash_lock_init(void) +{ + if (!lflow_hash_lock_initialized) { + for (size_t i = 0; i < LFLOW_HASH_LOCK_MASK + 1; i++) { + ovs_mutex_init(&lflow_hash_locks[i]); + } + lflow_hash_lock_initialized = true; + } +} + +void +lflow_hash_lock_destroy(void) +{ + if (lflow_hash_lock_initialized) { + for (size_t i = 0; i < LFLOW_HASH_LOCK_MASK + 1; i++) { + ovs_mutex_destroy(&lflow_hash_locks[i]); + } + } + lflow_hash_lock_initialized = false; +} + +/* static functions. */ +static void +ovn_lflow_init(struct ovn_lflow *lflow, struct ovn_datapath *od, + size_t dp_bitmap_len, enum ovn_stage stage, uint16_t priority, + char *match, char *actions, char *io_port, char *ctrl_meter, + char *stage_hint, const char *where) +{ + lflow->dpg_bitmap = bitmap_allocate(dp_bitmap_len); + lflow->od = od; + lflow->stage = stage; + lflow->priority = priority; + lflow->match = match; + lflow->actions = actions; + lflow->io_port = io_port; + lflow->stage_hint = stage_hint; + lflow->ctrl_meter = ctrl_meter; + lflow->dpg = NULL; + lflow->where = where; + lflow->sb_uuid = UUID_ZERO; + hmap_init(&lflow->dp_refcnts_map); + ovs_list_init(&lflow->referenced_by); +} + +static struct ovs_mutex * +lflow_hash_lock(const struct hmap *lflow_table, uint32_t hash) + OVS_ACQUIRES(fake_hash_mutex) + OVS_NO_THREAD_SAFETY_ANALYSIS +{ + struct ovs_mutex *hash_lock = NULL; + + if (parallelization_state == STATE_USE_PARALLELIZATION) { + hash_lock = + &lflow_hash_locks[hash & lflow_table->mask & LFLOW_HASH_LOCK_MASK]; + ovs_mutex_lock(hash_lock); + } + return hash_lock; +} + +static void +lflow_hash_unlock(struct ovs_mutex *hash_lock) + OVS_RELEASES(fake_hash_mutex) + OVS_NO_THREAD_SAFETY_ANALYSIS +{ + if (hash_lock) { + ovs_mutex_unlock(hash_lock); + } +} + +static bool +ovn_lflow_equal(const struct ovn_lflow *a, enum ovn_stage stage, + uint16_t priority, const char *match, + const char *actions, const char *ctrl_meter) +{ + return (a->stage == stage + && a->priority == priority + && !strcmp(a->match, match) + && !strcmp(a->actions, actions) + && nullable_string_is_equal(a->ctrl_meter, ctrl_meter)); +} + +static struct ovn_lflow * +ovn_lflow_find(const struct hmap *lflows, + enum ovn_stage stage, uint16_t priority, + const char *match, const char *actions, + const char *ctrl_meter, uint32_t hash) +{ + struct ovn_lflow *lflow; + HMAP_FOR_EACH_WITH_HASH (lflow, hmap_node, hash, lflows) { + if (ovn_lflow_equal(lflow, stage, priority, match, actions, + ctrl_meter)) { + return lflow; + } + } + return NULL; +} + +static char * +ovn_lflow_hint(const struct ovsdb_idl_row *row) +{ + if (!row) { + return NULL; + } + return xasprintf("%08x", row->uuid.parts[0]); +} + +static void +ovn_lflow_destroy(struct lflow_table *lflow_table, struct ovn_lflow *lflow) +{ + if (lflow) { + if (lflow_table) { + hmap_remove(&lflow_table->entries, &lflow->hmap_node); + } + bitmap_free(lflow->dpg_bitmap); + free(lflow->match); + free(lflow->actions); + free(lflow->io_port); + free(lflow->stage_hint); + free(lflow->ctrl_meter); + ovn_lflow_clear_dp_refcnts_map(lflow); + struct lflow_ref_node *l; + LIST_FOR_EACH_SAFE (l, ref_list_node, &lflow->referenced_by) { + lflow_ref_node_destroy(l, NULL); + } + free(lflow); + } +} + +static struct ovn_lflow * +do_ovn_lflow_add(struct lflow_table *lflow_table, + const struct ovn_datapath *od, + const unsigned long *dp_bitmap, size_t dp_bitmap_len, + uint32_t hash, enum ovn_stage stage, uint16_t priority, + const char *match, const char *actions, + const char *io_port, const char *ctrl_meter, + const struct ovsdb_idl_row *stage_hint, + const char *where) + OVS_REQUIRES(fake_hash_mutex) +{ + struct ovn_lflow *old_lflow; + struct ovn_lflow *lflow; + + size_t bitmap_len = od ? ods_size(od->datapaths) : dp_bitmap_len; + ovs_assert(bitmap_len); + + old_lflow = ovn_lflow_find(&lflow_table->entries, stage, + priority, match, actions, ctrl_meter, hash); + if (old_lflow) { + ovn_dp_group_add_with_reference(old_lflow, od, dp_bitmap, + bitmap_len); + return old_lflow; + } + + lflow = xzalloc(sizeof *lflow); + /* While adding new logical flows we're not setting single datapath, but + * collecting a group. 'od' will be updated later for all flows with only + * one datapath in a group, so it could be hashed correctly. */ + ovn_lflow_init(lflow, NULL, bitmap_len, stage, priority, + xstrdup(match), xstrdup(actions), + io_port ? xstrdup(io_port) : NULL, + nullable_xstrdup(ctrl_meter), + ovn_lflow_hint(stage_hint), where); + + ovn_dp_group_add_with_reference(lflow, od, dp_bitmap, bitmap_len); + + if (parallelization_state != STATE_USE_PARALLELIZATION) { + hmap_insert(&lflow_table->entries, &lflow->hmap_node, hash); + } else { + hmap_insert_fast(&lflow_table->entries, &lflow->hmap_node, + hash); + thread_lflow_counter++; + } + + return lflow; +} + +static bool +sync_lflow_to_sb(struct ovn_lflow *lflow, + struct ovsdb_idl_txn *ovnsb_txn, + struct lflow_table *lflow_table, + const struct ovn_datapaths *ls_datapaths, + const struct ovn_datapaths *lr_datapaths, + bool ovn_internal_version_changed, + const struct sbrec_logical_flow *sbflow, + const struct sbrec_logical_dp_group_table *sb_dpgrp_table) +{ + struct sbrec_logical_dp_group *sbrec_dp_group = NULL; + struct ovn_dp_group *pre_sync_dpg = lflow->dpg; + struct ovn_datapath **datapaths_array; + struct hmap *dp_groups; + size_t n_datapaths; + bool is_switch; + + if (ovn_stage_to_datapath_type(lflow->stage) == DP_SWITCH) { + n_datapaths = ods_size(ls_datapaths); + datapaths_array = ls_datapaths->array; + dp_groups = &lflow_table->ls_dp_groups; + is_switch = true; + } else { + n_datapaths = ods_size(lr_datapaths); + datapaths_array = lr_datapaths->array; + dp_groups = &lflow_table->lr_dp_groups; + is_switch = false; + } + + lflow->n_ods = bitmap_count1(lflow->dpg_bitmap, n_datapaths); + ovs_assert(lflow->n_ods); + + if (lflow->n_ods == 1) { + /* There is only one datapath, so it should be moved out of the + * group to a single 'od'. */ + size_t index = bitmap_scan(lflow->dpg_bitmap, true, 0, + n_datapaths); + + lflow->od = datapaths_array[index]; + lflow->dpg = NULL; + } else { + lflow->od = NULL; + } + + if (!sbflow) { + lflow->sb_uuid = uuid_random(); + sbflow = sbrec_logical_flow_insert_persist_uuid(ovnsb_txn, + &lflow->sb_uuid); + const char *pipeline = ovn_stage_get_pipeline_name(lflow->stage); + uint8_t table = ovn_stage_get_table(lflow->stage); + sbrec_logical_flow_set_pipeline(sbflow, pipeline); + sbrec_logical_flow_set_table_id(sbflow, table); + sbrec_logical_flow_set_priority(sbflow, lflow->priority); + sbrec_logical_flow_set_match(sbflow, lflow->match); + sbrec_logical_flow_set_actions(sbflow, lflow->actions); + if (lflow->io_port) { + struct smap tags = SMAP_INITIALIZER(&tags); + smap_add(&tags, "in_out_port", lflow->io_port); + sbrec_logical_flow_set_tags(sbflow, &tags); + smap_destroy(&tags); + } + sbrec_logical_flow_set_controller_meter(sbflow, lflow->ctrl_meter); + + /* Trim the source locator lflow->where, which looks something like + * "ovn/northd/northd.c:1234", down to just the part following the + * last slash, e.g. "northd.c:1234". */ + const char *slash = strrchr(lflow->where, '/'); +#if _WIN32 + const char *backslash = strrchr(lflow->where, '\\'); + if (!slash || backslash > slash) { + slash = backslash; + } +#endif + const char *where = slash ? slash + 1 : lflow->where; + + struct smap ids = SMAP_INITIALIZER(&ids); + smap_add(&ids, "stage-name", ovn_stage_to_str(lflow->stage)); + smap_add(&ids, "source", where); + if (lflow->stage_hint) { + smap_add(&ids, "stage-hint", lflow->stage_hint); + } + sbrec_logical_flow_set_external_ids(sbflow, &ids); + smap_destroy(&ids); + + } else { + lflow->sb_uuid = sbflow->header_.uuid; + sbrec_dp_group = sbflow->logical_dp_group; + + if (ovn_internal_version_changed) { + const char *stage_name = smap_get_def(&sbflow->external_ids, + "stage-name", ""); + const char *stage_hint = smap_get_def(&sbflow->external_ids, + "stage-hint", ""); + const char *source = smap_get_def(&sbflow->external_ids, + "source", ""); + + if (strcmp(stage_name, ovn_stage_to_str(lflow->stage))) { + sbrec_logical_flow_update_external_ids_setkey( + sbflow, "stage-name", ovn_stage_to_str(lflow->stage)); + } + if (lflow->stage_hint) { + if (strcmp(stage_hint, lflow->stage_hint)) { + sbrec_logical_flow_update_external_ids_setkey( + sbflow, "stage-hint", lflow->stage_hint); + } + } + if (lflow->where) { + + /* Trim the source locator lflow->where, which looks something + * like "ovn/northd/northd.c:1234", down to just the part + * following the last slash, e.g. "northd.c:1234". */ + const char *slash = strrchr(lflow->where, '/'); +#if _WIN32 + const char *backslash = strrchr(lflow->where, '\\'); + if (!slash || backslash > slash) { + slash = backslash; + } +#endif + const char *where = slash ? slash + 1 : lflow->where; + + if (strcmp(source, where)) { + sbrec_logical_flow_update_external_ids_setkey( + sbflow, "source", where); + } + } + } + } + + if (lflow->od) { + sbrec_logical_flow_set_logical_datapath(sbflow, lflow->od->sb); + sbrec_logical_flow_set_logical_dp_group(sbflow, NULL); + } else { + sbrec_logical_flow_set_logical_datapath(sbflow, NULL); + lflow->dpg = ovn_dp_group_get(dp_groups, lflow->n_ods, + lflow->dpg_bitmap, + n_datapaths); + if (lflow->dpg) { + /* Update the dpg's sb dp_group. */ + lflow->dpg->dp_group = sbrec_logical_dp_group_table_get_for_uuid( + sb_dpgrp_table, + &lflow->dpg->dpg_uuid); + + if (!lflow->dpg->dp_group) { + /* Ideally this should not happen. But it can still happen + * due to 2 reasons: + * 1. There is a bug in the dp_group management. We should + * perhaps assert here. + * 2. A User or CMS may delete the logical_dp_groups in SB DB + * or clear the SB:Logical_flow.logical_dp_groups column + * (intentionally or accidentally) + * + * Because of (2) it is better to return false instead of + * assert,so that we recover from th inconsistent SB DB. + */ + static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 1); + VLOG_WARN_RL(&rl, "SB Logical flow ["UUID_FMT"]'s " + "logical_dp_group column is not set " + "(which is unexpected). It should have been " + "referencing the dp group ["UUID_FMT"]", + UUID_ARGS(&sbflow->header_.uuid), + UUID_ARGS(&lflow->dpg->dpg_uuid)); + return false; + } + } else { + lflow->dpg = ovn_dp_group_create( + ovnsb_txn, dp_groups, sbrec_dp_group, + lflow->n_ods, lflow->dpg_bitmap, + n_datapaths, is_switch, + ls_datapaths, + lr_datapaths); + } + sbrec_logical_flow_set_logical_dp_group(sbflow, + lflow->dpg->dp_group); + } + + if (pre_sync_dpg != lflow->dpg) { + if (lflow->dpg) { + inc_ovn_dp_group_ref(lflow->dpg); + } + if (pre_sync_dpg) { + dec_ovn_dp_group_ref(dp_groups, pre_sync_dpg); + } + } + + return true; +} + +static struct ovn_dp_group * +ovn_dp_group_find(const struct hmap *dp_groups, + const unsigned long *dpg_bitmap, size_t bitmap_len, + uint32_t hash) +{ + struct ovn_dp_group *dpg; + + HMAP_FOR_EACH_WITH_HASH (dpg, node, hash, dp_groups) { + if (bitmap_equal(dpg->bitmap, dpg_bitmap, bitmap_len)) { + return dpg; + } + } + return NULL; +} + +static void +inc_ovn_dp_group_ref(struct ovn_dp_group *dpg) +{ + dpg->refcnt++; +} + +static void +dec_ovn_dp_group_ref(struct hmap *dp_groups, struct ovn_dp_group *dpg) +{ + dpg->refcnt--; + + if (!dpg->refcnt) { + hmap_remove(dp_groups, &dpg->node); + free(dpg->bitmap); + free(dpg); + } +} + +static struct sbrec_logical_dp_group * +ovn_sb_insert_or_update_logical_dp_group( + struct ovsdb_idl_txn *ovnsb_txn, + struct sbrec_logical_dp_group *dp_group, + const unsigned long *dpg_bitmap, + const struct ovn_datapaths *datapaths) +{ + const struct sbrec_datapath_binding **sb; + size_t n = 0, index; + + sb = xmalloc(bitmap_count1(dpg_bitmap, ods_size(datapaths)) * sizeof *sb); + BITMAP_FOR_EACH_1 (index, ods_size(datapaths), dpg_bitmap) { + sb[n++] = datapaths->array[index]->sb; + } + if (!dp_group) { + struct uuid dpg_uuid = uuid_random(); + dp_group = sbrec_logical_dp_group_insert_persist_uuid( + ovnsb_txn, &dpg_uuid); + } + sbrec_logical_dp_group_set_datapaths( + dp_group, (struct sbrec_datapath_binding **) sb, n); + free(sb); + + return dp_group; +} + +static struct ovn_dp_group * +ovn_dp_group_get(struct hmap *dp_groups, size_t desired_n, + const unsigned long *desired_bitmap, + size_t bitmap_len) +{ + uint32_t hash; + + hash = hash_int(desired_n, 0); + return ovn_dp_group_find(dp_groups, desired_bitmap, bitmap_len, hash); +} + +/* Creates a new datapath group and adds it to 'dp_groups'. + * If 'sb_group' is provided, function will try to re-use this group by + * either taking it directly, or by modifying, if it's not already in use. + * Caller should first call ovn_dp_group_get() before calling this function. */ +static struct ovn_dp_group * +ovn_dp_group_create(struct ovsdb_idl_txn *ovnsb_txn, + struct hmap *dp_groups, + struct sbrec_logical_dp_group *sb_group, + size_t desired_n, + const unsigned long *desired_bitmap, + size_t bitmap_len, + bool is_switch, + const struct ovn_datapaths *ls_datapaths, + const struct ovn_datapaths *lr_datapaths) +{ + struct ovn_dp_group *dpg; + + bool update_dp_group = false, can_modify = false; + unsigned long *dpg_bitmap; + size_t i, n = 0; + + dpg_bitmap = sb_group ? bitmap_allocate(bitmap_len) : NULL; + for (i = 0; sb_group && i < sb_group->n_datapaths; i++) { + struct ovn_datapath *datapath_od; + + datapath_od = ovn_datapath_from_sbrec( + ls_datapaths ? &ls_datapaths->datapaths : NULL, + lr_datapaths ? &lr_datapaths->datapaths : NULL, + sb_group->datapaths[i]); + if (!datapath_od || ovn_datapath_is_stale(datapath_od)) { + break; + } + bitmap_set1(dpg_bitmap, datapath_od->index); + n++; + } + if (!sb_group || i != sb_group->n_datapaths) { + /* No group or stale group. Not going to be used. */ + update_dp_group = true; + can_modify = true; + } else if (!bitmap_equal(dpg_bitmap, desired_bitmap, bitmap_len)) { + /* The group in Sb is different. */ + update_dp_group = true; + /* We can modify existing group if it's not already in use. */ + can_modify = !ovn_dp_group_find(dp_groups, dpg_bitmap, + bitmap_len, hash_int(n, 0)); + } + + bitmap_free(dpg_bitmap); + + dpg = xzalloc(sizeof *dpg); + dpg->bitmap = bitmap_clone(desired_bitmap, bitmap_len); + if (!update_dp_group) { + dpg->dp_group = sb_group; + } else { + dpg->dp_group = ovn_sb_insert_or_update_logical_dp_group( + ovnsb_txn, + can_modify ? sb_group : NULL, + desired_bitmap, + is_switch ? ls_datapaths : lr_datapaths); + } + dpg->dpg_uuid = dpg->dp_group->header_.uuid; + hmap_insert(dp_groups, &dpg->node, hash_int(desired_n, 0)); + + return dpg; +} + +/* Adds an OVN datapath to a datapath group of existing logical flow. + * Version to use when hash bucket locking is NOT required or the corresponding + * hash lock is already taken. */ +static void +ovn_dp_group_add_with_reference(struct ovn_lflow *lflow_ref, + const struct ovn_datapath *od, + const unsigned long *dp_bitmap, + size_t bitmap_len) + OVS_REQUIRES(fake_hash_mutex) +{ + if (od) { + bitmap_set1(lflow_ref->dpg_bitmap, od->index); + } + if (dp_bitmap) { + bitmap_or(lflow_ref->dpg_bitmap, dp_bitmap, bitmap_len); + } +} + +static bool +lflow_ref_sync_lflows__(struct lflow_ref *lflow_ref, + struct lflow_table *lflow_table, + struct ovsdb_idl_txn *ovnsb_txn, + const struct ovn_datapaths *ls_datapaths, + const struct ovn_datapaths *lr_datapaths, + bool ovn_internal_version_changed, + const struct sbrec_logical_flow_table *sbflow_table, + const struct sbrec_logical_dp_group_table *dpgrp_table) +{ + struct lflow_ref_node *lrn; + struct ovn_lflow *lflow; + LIST_FOR_EACH_SAFE (lrn, lflow_list_node, &lflow_ref->lflows_ref_list) { + lflow = lrn->lflow; + const struct sbrec_logical_flow *sblflow = + sbrec_logical_flow_table_get_for_uuid(sbflow_table, + &lflow->sb_uuid); + + struct hmap *dp_groups = NULL; + size_t n_datapaths; + if (ovn_stage_to_datapath_type(lflow->stage) == DP_SWITCH) { + dp_groups = &lflow_table->ls_dp_groups; + n_datapaths = ods_size(ls_datapaths); + } else { + dp_groups = &lflow_table->lr_dp_groups; + n_datapaths = ods_size(lr_datapaths); + } + + size_t n_ods = bitmap_count1(lflow->dpg_bitmap, n_datapaths); + + if (n_ods) { + if (!sync_lflow_to_sb(lflow, ovnsb_txn, lflow_table, ls_datapaths, + lr_datapaths, ovn_internal_version_changed, + sblflow, dpgrp_table)) { + return false; + } + } + + if (!lrn->linked) { + lflow_ref_node_destroy(lrn, &lflow_ref->lflow_ref_nodes); + + if (ovs_list_is_empty(&lflow->referenced_by)) { + if (lflow->dpg) { + dec_ovn_dp_group_ref(dp_groups, lflow->dpg); + } + ovn_lflow_destroy(lflow_table, lflow); + + if (sblflow) { + sbrec_logical_flow_delete(sblflow); + } + } + } + } + + return true; +} + +/* Used for the datapath reference counting for a given 'struct ovn_lflow'. + * See the hmap 'dp_refcnts_map in 'struct ovn_lflow'. + * For a given lflow L(M, A) with match - M and actions - A, it can be + * referenced by multiple lflow_refs for the same datapath + * Eg. Two lflow_ref's - op->lflow_ref and op->stateful_lflow_ref of a + * datapath can have a reference to the same lflow L (M, A). In this it + * is important to maintain this reference count so that the sync to the + * SB DB logical_flow is correct. */ +struct dp_refcnt { + struct hmap_node key_node; + + size_t dp_index; /* datapath index. Also used as hmap key. */ + size_t refcnt; /* reference counter. */ +}; + +static struct dp_refcnt * +dp_refcnt_find(struct hmap *dp_refcnts_map, size_t dp_index) +{ + struct dp_refcnt *dp_refcnt; + HMAP_FOR_EACH_WITH_HASH (dp_refcnt, key_node, dp_index, dp_refcnts_map) { + if (dp_refcnt->dp_index == dp_index) { + return dp_refcnt; + } + } + + return NULL; +} + +static void +inc_dp_refcnt(struct hmap *dp_refcnts_map, size_t dp_index) +{ + struct dp_refcnt *dp_refcnt = dp_refcnt_find(dp_refcnts_map, dp_index); + + if (!dp_refcnt) { + dp_refcnt = xzalloc(sizeof *dp_refcnt); + dp_refcnt->dp_index = dp_index; + + hmap_insert(dp_refcnts_map, &dp_refcnt->key_node, dp_index); + } + + dp_refcnt->refcnt++; +} + +/* Decrements the datapath's refcnt from the 'dp_refcnts_map' if it exists + * and returns true of the refcnt is 0 or if the dp refcnt doesn't exist. */ +static bool +dec_dp_refcnt(struct hmap *dp_refcnts_map, size_t dp_index) +{ + bool retval = true; + + struct dp_refcnt *dp_refcnt = dp_refcnt_find(dp_refcnts_map, dp_index); + if (dp_refcnt) { + dp_refcnt->refcnt--; + + if (!dp_refcnt->refcnt) { + hmap_remove(dp_refcnts_map, &dp_refcnt->key_node); + free(dp_refcnt); + } else { + retval = false; + } + } + + return retval; +} + +static void +ovn_lflow_clear_dp_refcnts_map(struct ovn_lflow *lflow) +{ + struct dp_refcnt *dp_refcnt; + + HMAP_FOR_EACH_POP (dp_refcnt, key_node, &lflow->dp_refcnts_map) { + free(dp_refcnt); + } + + hmap_destroy(&lflow->dp_refcnts_map); +} + +static struct lflow_ref_node * +lflow_ref_node_find(struct hmap *lflow_ref_nodes, struct ovn_lflow *lflow, + uint32_t lflow_hash) +{ + struct lflow_ref_node *lrn; + HMAP_FOR_EACH_WITH_HASH (lrn, ref_node, lflow_hash, lflow_ref_nodes) { + if (lrn->lflow == lflow) { + return lrn; + } + } + + return NULL; +} + +static void +lflow_ref_node_destroy(struct lflow_ref_node *lrn, + struct hmap *lflow_ref_nodes) +{ + if (lflow_ref_nodes) { + hmap_remove(lflow_ref_nodes, &lrn->ref_node); + } + ovs_list_remove(&lrn->lflow_list_node); + ovs_list_remove(&lrn->ref_list_node); + free(lrn); +} diff --git a/northd/lflow-mgr.h b/northd/lflow-mgr.h new file mode 100644 index 0000000000..5a0cc28965 --- /dev/null +++ b/northd/lflow-mgr.h @@ -0,0 +1,186 @@ + /* + * Copyright (c) 2023, Red Hat, Inc. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at: + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +#ifndef LFLOW_MGR_H +#define LFLOW_MGR_H 1 + +#include "include/openvswitch/hmap.h" +#include "include/openvswitch/uuid.h" + +#include "northd.h" + +struct ovsdb_idl_txn; +struct ovn_datapath; +struct ovsdb_idl_row; + +/* lflow map which stores the logical flows. */ +struct lflow_table; +struct lflow_table *lflow_table_alloc(void); +void lflow_table_init(struct lflow_table *); +void lflow_table_clear(struct lflow_table *); +void lflow_table_destroy(struct lflow_table *); +void lflow_table_expand(struct lflow_table *); +void lflow_table_set_size(struct lflow_table *, size_t); +void lflow_table_sync_to_sb(struct lflow_table *, + struct ovsdb_idl_txn *ovnsb_txn, + const struct ovn_datapaths *ls_datapaths, + const struct ovn_datapaths *lr_datapaths, + bool ovn_internal_version_changed, + const struct sbrec_logical_flow_table *, + const struct sbrec_logical_dp_group_table *); +void lflow_table_destroy(struct lflow_table *); + +void lflow_hash_lock_init(void); +void lflow_hash_lock_destroy(void); + +/* lflow mgr manages logical flows for a resource (like logical port + * or datapath). */ +struct lflow_ref; + +struct lflow_ref *lflow_ref_create(void); +void lflow_ref_destroy(struct lflow_ref *); +void lflow_ref_clear(struct lflow_ref *lflow_ref); +void lflow_ref_unlink_lflows(struct lflow_ref *); +bool lflow_ref_resync_flows(struct lflow_ref *, + struct lflow_table *lflow_table, + struct ovsdb_idl_txn *ovnsb_txn, + const struct ovn_datapaths *ls_datapaths, + const struct ovn_datapaths *lr_datapaths, + bool ovn_internal_version_changed, + const struct sbrec_logical_flow_table *, + const struct sbrec_logical_dp_group_table *); +bool lflow_ref_sync_lflows(struct lflow_ref *, + struct lflow_table *lflow_table, + struct ovsdb_idl_txn *ovnsb_txn, + const struct ovn_datapaths *ls_datapaths, + const struct ovn_datapaths *lr_datapaths, + bool ovn_internal_version_changed, + const struct sbrec_logical_flow_table *, + const struct sbrec_logical_dp_group_table *); + + +void lflow_table_add_lflow(struct lflow_table *, const struct ovn_datapath *, + const unsigned long *dp_bitmap, + size_t dp_bitmap_len, enum ovn_stage stage, + uint16_t priority, const char *match, + const char *actions, const char *io_port, + const char *ctrl_meter, + const struct ovsdb_idl_row *stage_hint, + const char *where, struct lflow_ref *); +void lflow_table_add_lflow_default_drop(struct lflow_table *, + const struct ovn_datapath *, + enum ovn_stage stage, + const char *where, + struct lflow_ref *); + +/* Adds a row with the specified contents to the Logical_Flow table. */ +#define ovn_lflow_add_with_hint__(LFLOW_TABLE, OD, STAGE, PRIORITY, MATCH, \ + ACTIONS, IN_OUT_PORT, CTRL_METER, \ + STAGE_HINT) \ + lflow_table_add_lflow(LFLOW_TABLE, OD, NULL, 0, STAGE, PRIORITY, MATCH, \ + ACTIONS, IN_OUT_PORT, CTRL_METER, STAGE_HINT, \ + OVS_SOURCE_LOCATOR, NULL) + +#define ovn_lflow_add_with_lflow_ref_hint__(LFLOW_TABLE, OD, STAGE, PRIORITY, \ + MATCH, ACTIONS, IN_OUT_PORT, \ + CTRL_METER, STAGE_HINT, LFLOW_REF)\ + lflow_table_add_lflow(LFLOW_TABLE, OD, NULL, 0, STAGE, PRIORITY, MATCH, \ + ACTIONS, IN_OUT_PORT, CTRL_METER, STAGE_HINT, \ + OVS_SOURCE_LOCATOR, LFLOW_REF) + +#define ovn_lflow_add_with_hint(LFLOW_TABLE, OD, STAGE, PRIORITY, MATCH, \ + ACTIONS, STAGE_HINT) \ + lflow_table_add_lflow(LFLOW_TABLE, OD, NULL, 0, STAGE, PRIORITY, MATCH, \ + ACTIONS, NULL, NULL, STAGE_HINT, \ + OVS_SOURCE_LOCATOR, NULL) + +#define ovn_lflow_add_with_lflow_ref_hint(LFLOW_TABLE, OD, STAGE, PRIORITY, \ + MATCH, ACTIONS, STAGE_HINT, \ + LFLOW_REF) \ + lflow_table_add_lflow(LFLOW_TABLE, OD, NULL, 0, STAGE, PRIORITY, MATCH, \ + ACTIONS, NULL, NULL, STAGE_HINT, \ + OVS_SOURCE_LOCATOR, LFLOW_REF) + +#define ovn_lflow_add_with_dp_group(LFLOW_TABLE, DP_BITMAP, DP_BITMAP_LEN, \ + STAGE, PRIORITY, MATCH, ACTIONS, \ + STAGE_HINT) \ + lflow_table_add_lflow(LFLOW_TABLE, NULL, DP_BITMAP, DP_BITMAP_LEN, STAGE, \ + PRIORITY, MATCH, ACTIONS, NULL, NULL, STAGE_HINT, \ + OVS_SOURCE_LOCATOR, NULL) + +#define ovn_lflow_add_default_drop(LFLOW_TABLE, OD, STAGE) \ + lflow_table_add_lflow_default_drop(LFLOW_TABLE, OD, STAGE, \ + OVS_SOURCE_LOCATOR, NULL) + + +/* This macro is similar to ovn_lflow_add_with_hint, except that it requires + * the IN_OUT_PORT argument, which tells the lport name that appears in the + * MATCH, which helps ovn-controller to bypass lflows parsing when the lport is + * not local to the chassis. The critiera of the lport to be added using this + * argument: + * + * - For ingress pipeline, the lport that is used to match "inport". + * - For egress pipeline, the lport that is used to match "outport". + * + * For now, only LS pipelines should use this macro. */ +#define ovn_lflow_add_with_lport_and_hint(LFLOW_TABLE, OD, STAGE, PRIORITY, \ + MATCH, ACTIONS, IN_OUT_PORT, \ + STAGE_HINT, LFLOW_REF) \ + lflow_table_add_lflow(LFLOW_TABLE, OD, NULL, 0, STAGE, PRIORITY, MATCH, \ + ACTIONS, IN_OUT_PORT, NULL, STAGE_HINT, \ + OVS_SOURCE_LOCATOR, LFLOW_REF) + +#define ovn_lflow_add(LFLOW_TABLE, OD, STAGE, PRIORITY, MATCH, ACTIONS) \ + lflow_table_add_lflow(LFLOW_TABLE, OD, NULL, 0, STAGE, PRIORITY, MATCH, \ + ACTIONS, NULL, NULL, NULL, OVS_SOURCE_LOCATOR, NULL) + +#define ovn_lflow_add_with_lflow_ref(LFLOW_TABLE, OD, STAGE, PRIORITY, MATCH, \ + ACTIONS, LFLOW_REF) \ + lflow_table_add_lflow(LFLOW_TABLE, OD, NULL, 0, STAGE, PRIORITY, MATCH, \ + ACTIONS, NULL, NULL, NULL, OVS_SOURCE_LOCATOR, \ + LFLOW_REF) + +#define ovn_lflow_metered(LFLOW_TABLE, OD, STAGE, PRIORITY, MATCH, ACTIONS, \ + CTRL_METER) \ + ovn_lflow_add_with_hint__(LFLOW_TABLE, OD, STAGE, PRIORITY, MATCH, \ + ACTIONS, NULL, CTRL_METER, NULL) + +struct sbrec_logical_dp_group; + +struct ovn_dp_group { + unsigned long *bitmap; + const struct sbrec_logical_dp_group *dp_group; + struct uuid dpg_uuid; + struct hmap_node node; + size_t refcnt; +}; + +static inline void +ovn_dp_groups_init(struct hmap *dp_groups) +{ + hmap_init(dp_groups); +} + +void ovn_dp_groups_clear(struct hmap *dp_groups); +void ovn_dp_groups_destroy(struct hmap *dp_groups); +struct ovn_dp_group *ovn_dp_group_get_or_create( + struct ovsdb_idl_txn *ovnsb_txn, struct hmap *dp_groups, + struct sbrec_logical_dp_group *sb_group, + size_t desired_n, const unsigned long *desired_bitmap, + size_t bitmap_len, bool is_switch, + const struct ovn_datapaths *ls_datapaths, + const struct ovn_datapaths *lr_datapaths); + +#endif /* LFLOW_MGR_H */ \ No newline at end of file diff --git a/northd/northd.c b/northd/northd.c index 68f2b0bab4..f41df83c40 100644 --- a/northd/northd.c +++ b/northd/northd.c @@ -41,6 +41,7 @@ #include "lib/ovn-sb-idl.h" #include "lib/ovn-util.h" #include "lib/lb.h" +#include "lflow-mgr.h" #include "memory.h" #include "northd.h" #include "en-lb-data.h" @@ -68,7 +69,7 @@ VLOG_DEFINE_THIS_MODULE(northd); static bool controller_event_en; -static bool lflow_hash_lock_initialized = false; + static bool check_lsp_is_up; @@ -97,116 +98,6 @@ static bool default_acl_drop; #define MAX_OVN_TAGS 4096 -/* Pipeline stages. */ - -/* The two purposes for which ovn-northd uses OVN logical datapaths. */ -enum ovn_datapath_type { - DP_SWITCH, /* OVN logical switch. */ - DP_ROUTER /* OVN logical router. */ -}; - -/* Returns an "enum ovn_stage" built from the arguments. - * - * (It's better to use ovn_stage_build() for type-safety reasons, but inline - * functions can't be used in enums or switch cases.) */ -#define OVN_STAGE_BUILD(DP_TYPE, PIPELINE, TABLE) \ - (((DP_TYPE) << 9) | ((PIPELINE) << 8) | (TABLE)) - -/* A stage within an OVN logical switch or router. - * - * An "enum ovn_stage" indicates whether the stage is part of a logical switch - * or router, whether the stage is part of the ingress or egress pipeline, and - * the table within that pipeline. The first three components are combined to - * form the stage's full name, e.g. S_SWITCH_IN_PORT_SEC_L2, - * S_ROUTER_OUT_DELIVERY. */ -enum ovn_stage { -#define PIPELINE_STAGES \ - /* Logical switch ingress stages. */ \ - PIPELINE_STAGE(SWITCH, IN, CHECK_PORT_SEC, 0, "ls_in_check_port_sec") \ - PIPELINE_STAGE(SWITCH, IN, APPLY_PORT_SEC, 1, "ls_in_apply_port_sec") \ - PIPELINE_STAGE(SWITCH, IN, LOOKUP_FDB , 2, "ls_in_lookup_fdb") \ - PIPELINE_STAGE(SWITCH, IN, PUT_FDB, 3, "ls_in_put_fdb") \ - PIPELINE_STAGE(SWITCH, IN, PRE_ACL, 4, "ls_in_pre_acl") \ - PIPELINE_STAGE(SWITCH, IN, PRE_LB, 5, "ls_in_pre_lb") \ - PIPELINE_STAGE(SWITCH, IN, PRE_STATEFUL, 6, "ls_in_pre_stateful") \ - PIPELINE_STAGE(SWITCH, IN, ACL_HINT, 7, "ls_in_acl_hint") \ - PIPELINE_STAGE(SWITCH, IN, ACL_EVAL, 8, "ls_in_acl_eval") \ - PIPELINE_STAGE(SWITCH, IN, ACL_ACTION, 9, "ls_in_acl_action") \ - PIPELINE_STAGE(SWITCH, IN, QOS_MARK, 10, "ls_in_qos_mark") \ - PIPELINE_STAGE(SWITCH, IN, QOS_METER, 11, "ls_in_qos_meter") \ - PIPELINE_STAGE(SWITCH, IN, LB_AFF_CHECK, 12, "ls_in_lb_aff_check") \ - PIPELINE_STAGE(SWITCH, IN, LB, 13, "ls_in_lb") \ - PIPELINE_STAGE(SWITCH, IN, LB_AFF_LEARN, 14, "ls_in_lb_aff_learn") \ - PIPELINE_STAGE(SWITCH, IN, PRE_HAIRPIN, 15, "ls_in_pre_hairpin") \ - PIPELINE_STAGE(SWITCH, IN, NAT_HAIRPIN, 16, "ls_in_nat_hairpin") \ - PIPELINE_STAGE(SWITCH, IN, HAIRPIN, 17, "ls_in_hairpin") \ - PIPELINE_STAGE(SWITCH, IN, ACL_AFTER_LB_EVAL, 18, \ - "ls_in_acl_after_lb_eval") \ - PIPELINE_STAGE(SWITCH, IN, ACL_AFTER_LB_ACTION, 19, \ - "ls_in_acl_after_lb_action") \ - PIPELINE_STAGE(SWITCH, IN, STATEFUL, 20, "ls_in_stateful") \ - PIPELINE_STAGE(SWITCH, IN, ARP_ND_RSP, 21, "ls_in_arp_rsp") \ - PIPELINE_STAGE(SWITCH, IN, DHCP_OPTIONS, 22, "ls_in_dhcp_options") \ - PIPELINE_STAGE(SWITCH, IN, DHCP_RESPONSE, 23, "ls_in_dhcp_response") \ - PIPELINE_STAGE(SWITCH, IN, DNS_LOOKUP, 24, "ls_in_dns_lookup") \ - PIPELINE_STAGE(SWITCH, IN, DNS_RESPONSE, 25, "ls_in_dns_response") \ - PIPELINE_STAGE(SWITCH, IN, EXTERNAL_PORT, 26, "ls_in_external_port") \ - PIPELINE_STAGE(SWITCH, IN, L2_LKUP, 27, "ls_in_l2_lkup") \ - PIPELINE_STAGE(SWITCH, IN, L2_UNKNOWN, 28, "ls_in_l2_unknown") \ - \ - /* Logical switch egress stages. */ \ - PIPELINE_STAGE(SWITCH, OUT, PRE_ACL, 0, "ls_out_pre_acl") \ - PIPELINE_STAGE(SWITCH, OUT, PRE_LB, 1, "ls_out_pre_lb") \ - PIPELINE_STAGE(SWITCH, OUT, PRE_STATEFUL, 2, "ls_out_pre_stateful") \ - PIPELINE_STAGE(SWITCH, OUT, ACL_HINT, 3, "ls_out_acl_hint") \ - PIPELINE_STAGE(SWITCH, OUT, ACL_EVAL, 4, "ls_out_acl_eval") \ - PIPELINE_STAGE(SWITCH, OUT, ACL_ACTION, 5, "ls_out_acl_action") \ - PIPELINE_STAGE(SWITCH, OUT, QOS_MARK, 6, "ls_out_qos_mark") \ - PIPELINE_STAGE(SWITCH, OUT, QOS_METER, 7, "ls_out_qos_meter") \ - PIPELINE_STAGE(SWITCH, OUT, STATEFUL, 8, "ls_out_stateful") \ - PIPELINE_STAGE(SWITCH, OUT, CHECK_PORT_SEC, 9, "ls_out_check_port_sec") \ - PIPELINE_STAGE(SWITCH, OUT, APPLY_PORT_SEC, 10, "ls_out_apply_port_sec") \ - \ - /* Logical router ingress stages. */ \ - PIPELINE_STAGE(ROUTER, IN, ADMISSION, 0, "lr_in_admission") \ - PIPELINE_STAGE(ROUTER, IN, LOOKUP_NEIGHBOR, 1, "lr_in_lookup_neighbor") \ - PIPELINE_STAGE(ROUTER, IN, LEARN_NEIGHBOR, 2, "lr_in_learn_neighbor") \ - PIPELINE_STAGE(ROUTER, IN, IP_INPUT, 3, "lr_in_ip_input") \ - PIPELINE_STAGE(ROUTER, IN, UNSNAT, 4, "lr_in_unsnat") \ - PIPELINE_STAGE(ROUTER, IN, DEFRAG, 5, "lr_in_defrag") \ - PIPELINE_STAGE(ROUTER, IN, LB_AFF_CHECK, 6, "lr_in_lb_aff_check") \ - PIPELINE_STAGE(ROUTER, IN, DNAT, 7, "lr_in_dnat") \ - PIPELINE_STAGE(ROUTER, IN, LB_AFF_LEARN, 8, "lr_in_lb_aff_learn") \ - PIPELINE_STAGE(ROUTER, IN, ECMP_STATEFUL, 9, "lr_in_ecmp_stateful") \ - PIPELINE_STAGE(ROUTER, IN, ND_RA_OPTIONS, 10, "lr_in_nd_ra_options") \ - PIPELINE_STAGE(ROUTER, IN, ND_RA_RESPONSE, 11, "lr_in_nd_ra_response") \ - PIPELINE_STAGE(ROUTER, IN, IP_ROUTING_PRE, 12, "lr_in_ip_routing_pre") \ - PIPELINE_STAGE(ROUTER, IN, IP_ROUTING, 13, "lr_in_ip_routing") \ - PIPELINE_STAGE(ROUTER, IN, IP_ROUTING_ECMP, 14, "lr_in_ip_routing_ecmp") \ - PIPELINE_STAGE(ROUTER, IN, POLICY, 15, "lr_in_policy") \ - PIPELINE_STAGE(ROUTER, IN, POLICY_ECMP, 16, "lr_in_policy_ecmp") \ - PIPELINE_STAGE(ROUTER, IN, ARP_RESOLVE, 17, "lr_in_arp_resolve") \ - PIPELINE_STAGE(ROUTER, IN, CHK_PKT_LEN, 18, "lr_in_chk_pkt_len") \ - PIPELINE_STAGE(ROUTER, IN, LARGER_PKTS, 19, "lr_in_larger_pkts") \ - PIPELINE_STAGE(ROUTER, IN, GW_REDIRECT, 20, "lr_in_gw_redirect") \ - PIPELINE_STAGE(ROUTER, IN, ARP_REQUEST, 21, "lr_in_arp_request") \ - \ - /* Logical router egress stages. */ \ - PIPELINE_STAGE(ROUTER, OUT, CHECK_DNAT_LOCAL, 0, \ - "lr_out_chk_dnat_local") \ - PIPELINE_STAGE(ROUTER, OUT, UNDNAT, 1, "lr_out_undnat") \ - PIPELINE_STAGE(ROUTER, OUT, POST_UNDNAT, 2, "lr_out_post_undnat") \ - PIPELINE_STAGE(ROUTER, OUT, SNAT, 3, "lr_out_snat") \ - PIPELINE_STAGE(ROUTER, OUT, POST_SNAT, 4, "lr_out_post_snat") \ - PIPELINE_STAGE(ROUTER, OUT, EGR_LOOP, 5, "lr_out_egr_loop") \ - PIPELINE_STAGE(ROUTER, OUT, DELIVERY, 6, "lr_out_delivery") - -#define PIPELINE_STAGE(DP_TYPE, PIPELINE, STAGE, TABLE, NAME) \ - S_##DP_TYPE##_##PIPELINE##_##STAGE \ - = OVN_STAGE_BUILD(DP_##DP_TYPE, P_##PIPELINE, TABLE), - PIPELINE_STAGES -#undef PIPELINE_STAGE -}; /* Due to various hard-coded priorities need to implement ACLs, the * northbound database supports a smaller range of ACL priorities than @@ -391,51 +282,9 @@ enum ovn_stage { #define ROUTE_PRIO_OFFSET_STATIC 1 #define ROUTE_PRIO_OFFSET_CONNECTED 2 -/* Returns an "enum ovn_stage" built from the arguments. */ -static enum ovn_stage -ovn_stage_build(enum ovn_datapath_type dp_type, enum ovn_pipeline pipeline, - uint8_t table) -{ - return OVN_STAGE_BUILD(dp_type, pipeline, table); -} - -/* Returns the pipeline to which 'stage' belongs. */ -static enum ovn_pipeline -ovn_stage_get_pipeline(enum ovn_stage stage) -{ - return (stage >> 8) & 1; -} - -/* Returns the pipeline name to which 'stage' belongs. */ -static const char * -ovn_stage_get_pipeline_name(enum ovn_stage stage) -{ - return ovn_stage_get_pipeline(stage) == P_IN ? "ingress" : "egress"; -} - -/* Returns the table to which 'stage' belongs. */ -static uint8_t -ovn_stage_get_table(enum ovn_stage stage) -{ - return stage & 0xff; -} - -/* Returns a string name for 'stage'. */ -static const char * -ovn_stage_to_str(enum ovn_stage stage) -{ - switch (stage) { -#define PIPELINE_STAGE(DP_TYPE, PIPELINE, STAGE, TABLE, NAME) \ - case S_##DP_TYPE##_##PIPELINE##_##STAGE: return NAME; - PIPELINE_STAGES -#undef PIPELINE_STAGE - default: return ""; - } -} - /* Returns the type of the datapath to which a flow with the given 'stage' may * be added. */ -static enum ovn_datapath_type +enum ovn_datapath_type ovn_stage_to_datapath_type(enum ovn_stage stage) { switch (stage) { @@ -680,13 +529,6 @@ ovn_datapath_destroy(struct hmap *datapaths, struct ovn_datapath *od) } } -/* Returns 'od''s datapath type. */ -static enum ovn_datapath_type -ovn_datapath_get_type(const struct ovn_datapath *od) -{ - return od->nbs ? DP_SWITCH : DP_ROUTER; -} - static struct ovn_datapath * ovn_datapath_find_(const struct hmap *datapaths, const struct uuid *uuid) @@ -722,13 +564,7 @@ ovn_datapath_find_by_key(struct hmap *datapaths, uint32_t dp_key) return NULL; } -static bool -ovn_datapath_is_stale(const struct ovn_datapath *od) -{ - return !od->nbr && !od->nbs; -} - -static struct ovn_datapath * +struct ovn_datapath * ovn_datapath_from_sbrec(const struct hmap *ls_datapaths, const struct hmap *lr_datapaths, const struct sbrec_datapath_binding *sb) @@ -1297,19 +1133,6 @@ struct ovn_port_routable_addresses { size_t n_addrs; }; -/* A node that maintains link between an object (such as an ovn_port) and - * a lflow. */ -struct lflow_ref_node { - /* This list follows different lflows referenced by the same object. List - * head is, for example, ovn_port->lflows. */ - struct ovs_list lflow_list_node; - /* This list follows different objects that reference the same lflow. List - * head is ovn_lflow->referenced_by. */ - struct ovs_list ref_list_node; - /* The lflow. */ - struct ovn_lflow *lflow; -}; - static bool lsp_can_be_inc_processed(const struct nbrec_logical_switch_port *); static bool @@ -1389,6 +1212,8 @@ ovn_port_set_nb(struct ovn_port *op, init_mcast_port_info(&op->mcast_info, op->nbsp, op->nbrp); } +static bool lsp_is_router(const struct nbrec_logical_switch_port *nbsp); + static struct ovn_port * ovn_port_create(struct hmap *ports, const char *key, const struct nbrec_logical_switch_port *nbsp, @@ -1407,12 +1232,14 @@ ovn_port_create(struct hmap *ports, const char *key, op->l3dgw_port = op->cr_port = NULL; hmap_insert(ports, &op->key_node, hash_string(op->key, 0)); - ovs_list_init(&op->lflows); + op->lflow_ref = lflow_ref_create(); + op->stateful_lflow_ref = lflow_ref_create(); + return op; } static void -ovn_port_destroy_orphan(struct ovn_port *port) +ovn_port_cleanup(struct ovn_port *port) { if (port->tunnel_key) { ovs_assert(port->od); @@ -1422,6 +1249,8 @@ ovn_port_destroy_orphan(struct ovn_port *port) destroy_lport_addresses(&port->lsp_addrs[i]); } free(port->lsp_addrs); + port->n_lsp_addrs = 0; + port->lsp_addrs = NULL; if (port->peer) { port->peer->peer = NULL; @@ -1431,18 +1260,22 @@ ovn_port_destroy_orphan(struct ovn_port *port) destroy_lport_addresses(&port->ps_addrs[i]); } free(port->ps_addrs); + port->ps_addrs = NULL; + port->n_ps_addrs = 0; destroy_lport_addresses(&port->lrp_networks); destroy_lport_addresses(&port->proxy_arp_addrs); +} + +static void +ovn_port_destroy_orphan(struct ovn_port *port) +{ + ovn_port_cleanup(port); free(port->json_key); free(port->key); + lflow_ref_destroy(port->lflow_ref); + lflow_ref_destroy(port->stateful_lflow_ref); - struct lflow_ref_node *l; - LIST_FOR_EACH_SAFE (l, lflow_list_node, &port->lflows) { - ovs_list_remove(&l->lflow_list_node); - ovs_list_remove(&l->ref_list_node); - free(l); - } free(port); } @@ -3911,124 +3744,6 @@ build_lb_port_related_data( build_lswitch_lbs_from_lrouter(lr_datapaths, lb_dps_map, lb_group_dps_map); } - -struct ovn_dp_group { - unsigned long *bitmap; - struct sbrec_logical_dp_group *dp_group; - struct hmap_node node; -}; - -static struct ovn_dp_group * -ovn_dp_group_find(const struct hmap *dp_groups, - const unsigned long *dpg_bitmap, size_t bitmap_len, - uint32_t hash) -{ - struct ovn_dp_group *dpg; - - HMAP_FOR_EACH_WITH_HASH (dpg, node, hash, dp_groups) { - if (bitmap_equal(dpg->bitmap, dpg_bitmap, bitmap_len)) { - return dpg; - } - } - return NULL; -} - -static struct sbrec_logical_dp_group * -ovn_sb_insert_or_update_logical_dp_group( - struct ovsdb_idl_txn *ovnsb_txn, - struct sbrec_logical_dp_group *dp_group, - const unsigned long *dpg_bitmap, - const struct ovn_datapaths *datapaths) -{ - const struct sbrec_datapath_binding **sb; - size_t n = 0, index; - - sb = xmalloc(bitmap_count1(dpg_bitmap, ods_size(datapaths)) * sizeof *sb); - BITMAP_FOR_EACH_1 (index, ods_size(datapaths), dpg_bitmap) { - sb[n++] = datapaths->array[index]->sb; - } - if (!dp_group) { - dp_group = sbrec_logical_dp_group_insert(ovnsb_txn); - } - sbrec_logical_dp_group_set_datapaths( - dp_group, (struct sbrec_datapath_binding **) sb, n); - free(sb); - - return dp_group; -} - -/* Given a desired bitmap, finds a datapath group in 'dp_groups'. If it - * doesn't exist, creates a new one and adds it to 'dp_groups'. - * If 'sb_group' is provided, function will try to re-use this group by - * either taking it directly, or by modifying, if it's not already in use. */ -static struct ovn_dp_group * -ovn_dp_group_get_or_create(struct ovsdb_idl_txn *ovnsb_txn, - struct hmap *dp_groups, - struct sbrec_logical_dp_group *sb_group, - size_t desired_n, - const unsigned long *desired_bitmap, - size_t bitmap_len, - bool is_switch, - const struct ovn_datapaths *ls_datapaths, - const struct ovn_datapaths *lr_datapaths) -{ - struct ovn_dp_group *dpg; - uint32_t hash; - - hash = hash_int(desired_n, 0); - dpg = ovn_dp_group_find(dp_groups, desired_bitmap, bitmap_len, hash); - if (dpg) { - return dpg; - } - - bool update_dp_group = false, can_modify = false; - unsigned long *dpg_bitmap; - size_t i, n = 0; - - dpg_bitmap = sb_group ? bitmap_allocate(bitmap_len) : NULL; - for (i = 0; sb_group && i < sb_group->n_datapaths; i++) { - struct ovn_datapath *datapath_od; - - datapath_od = ovn_datapath_from_sbrec( - ls_datapaths ? &ls_datapaths->datapaths : NULL, - lr_datapaths ? &lr_datapaths->datapaths : NULL, - sb_group->datapaths[i]); - if (!datapath_od || ovn_datapath_is_stale(datapath_od)) { - break; - } - bitmap_set1(dpg_bitmap, datapath_od->index); - n++; - } - if (!sb_group || i != sb_group->n_datapaths) { - /* No group or stale group. Not going to be used. */ - update_dp_group = true; - can_modify = true; - } else if (!bitmap_equal(dpg_bitmap, desired_bitmap, bitmap_len)) { - /* The group in Sb is different. */ - update_dp_group = true; - /* We can modify existing group if it's not already in use. */ - can_modify = !ovn_dp_group_find(dp_groups, dpg_bitmap, - bitmap_len, hash_int(n, 0)); - } - - bitmap_free(dpg_bitmap); - - dpg = xzalloc(sizeof *dpg); - dpg->bitmap = bitmap_clone(desired_bitmap, bitmap_len); - if (!update_dp_group) { - dpg->dp_group = sb_group; - } else { - dpg->dp_group = ovn_sb_insert_or_update_logical_dp_group( - ovnsb_txn, - can_modify ? sb_group : NULL, - desired_bitmap, - is_switch ? ls_datapaths : lr_datapaths); - } - hmap_insert(dp_groups, &dpg->node, hash); - - return dpg; -} - struct sb_lb { struct hmap_node hmap_node; @@ -4846,28 +4561,20 @@ ovn_port_find_in_datapath(struct ovn_datapath *od, return NULL; } -static struct ovn_port * -ls_port_create(struct ovsdb_idl_txn *ovnsb_txn, struct hmap *ls_ports, - const char *key, const struct nbrec_logical_switch_port *nbsp, - struct ovn_datapath *od, const struct sbrec_port_binding *sb, - struct ovs_list *lflows, - const struct sbrec_mirror_table *sbrec_mirror_table, - const struct sbrec_chassis_table *sbrec_chassis_table, - struct ovsdb_idl_index *sbrec_chassis_by_name, - struct ovsdb_idl_index *sbrec_chassis_by_hostname) +static bool +ls_port_init(struct ovn_port *op, struct ovsdb_idl_txn *ovnsb_txn, + struct hmap *ls_ports, struct ovn_datapath *od, + const struct sbrec_port_binding *sb, + const struct sbrec_mirror_table *sbrec_mirror_table, + const struct sbrec_chassis_table *sbrec_chassis_table, + struct ovsdb_idl_index *sbrec_chassis_by_name, + struct ovsdb_idl_index *sbrec_chassis_by_hostname) { - struct ovn_port *op = ovn_port_create(ls_ports, key, nbsp, NULL, - NULL); - parse_lsp_addrs(op); op->od = od; - hmap_insert(&od->ports, &op->dp_node, hmap_node_hash(&op->key_node)); - if (lflows) { - ovs_list_splice(&op->lflows, lflows->next, lflows); - } - + parse_lsp_addrs(op); /* Assign explicitly requested tunnel ids first. */ if (!ovn_port_assign_requested_tnl_id(sbrec_chassis_table, op)) { - return NULL; + return false; } if (sb) { op->sb = sb; @@ -4884,14 +4591,57 @@ ls_port_create(struct ovsdb_idl_txn *ovnsb_txn, struct hmap *ls_ports, } /* Assign new tunnel ids where needed. */ if (!ovn_port_allocate_key(sbrec_chassis_table, ls_ports, op)) { - return NULL; + return false; } ovn_port_update_sbrec(ovnsb_txn, sbrec_chassis_by_name, sbrec_chassis_by_hostname, NULL, sbrec_mirror_table, op, NULL, NULL); + return true; +} + +static struct ovn_port * +ls_port_create(struct ovsdb_idl_txn *ovnsb_txn, struct hmap *ls_ports, + const char *key, const struct nbrec_logical_switch_port *nbsp, + struct ovn_datapath *od, const struct sbrec_port_binding *sb, + const struct sbrec_mirror_table *sbrec_mirror_table, + const struct sbrec_chassis_table *sbrec_chassis_table, + struct ovsdb_idl_index *sbrec_chassis_by_name, + struct ovsdb_idl_index *sbrec_chassis_by_hostname) +{ + struct ovn_port *op = ovn_port_create(ls_ports, key, nbsp, NULL, + NULL); + hmap_insert(&od->ports, &op->dp_node, hmap_node_hash(&op->key_node)); + if (!ls_port_init(op, ovnsb_txn, ls_ports, od, sb, + sbrec_mirror_table, sbrec_chassis_table, + sbrec_chassis_by_name, sbrec_chassis_by_hostname)) { + ovn_port_destroy(ls_ports, op); + return NULL; + } + return op; } +static bool +ls_port_reinit(struct ovn_port *op, struct ovsdb_idl_txn *ovnsb_txn, + struct hmap *ls_ports, + const struct nbrec_logical_switch_port *nbsp, + const struct nbrec_logical_router_port *nbrp, + struct ovn_datapath *od, + const struct sbrec_port_binding *sb, + const struct sbrec_mirror_table *sbrec_mirror_table, + const struct sbrec_chassis_table *sbrec_chassis_table, + struct ovsdb_idl_index *sbrec_chassis_by_name, + struct ovsdb_idl_index *sbrec_chassis_by_hostname) +{ + ovn_port_cleanup(op); + op->sb = sb; + ovn_port_set_nb(op, nbsp, nbrp); + op->l3dgw_port = op->cr_port = NULL; + return ls_port_init(op, ovnsb_txn, ls_ports, od, sb, + sbrec_mirror_table, sbrec_chassis_table, + sbrec_chassis_by_name, sbrec_chassis_by_hostname); +} + /* Returns true if the logical switch has changes which can be * incrementally handled. * Presently supports i-p for the below changes: @@ -5031,7 +4781,7 @@ ls_handle_lsp_changes(struct ovsdb_idl_txn *ovnsb_idl_txn, goto fail; } op = ls_port_create(ovnsb_idl_txn, &nd->ls_ports, - new_nbsp->name, new_nbsp, od, NULL, NULL, + new_nbsp->name, new_nbsp, od, NULL, ni->sbrec_mirror_table, ni->sbrec_chassis_table, ni->sbrec_chassis_by_name, @@ -5062,17 +4812,12 @@ ls_handle_lsp_changes(struct ovsdb_idl_txn *ovnsb_idl_txn, op->visited = true; continue; } - struct ovs_list lflows = OVS_LIST_INITIALIZER(&lflows); - ovs_list_splice(&lflows, op->lflows.next, &op->lflows); - ovn_port_destroy(&nd->ls_ports, op); - op = ls_port_create(ovnsb_idl_txn, &nd->ls_ports, - new_nbsp->name, new_nbsp, od, sb, &lflows, - ni->sbrec_mirror_table, + if (!ls_port_reinit(op, ovnsb_idl_txn, &nd->ls_ports, + new_nbsp, NULL, + od, sb, ni->sbrec_mirror_table, ni->sbrec_chassis_table, ni->sbrec_chassis_by_name, - ni->sbrec_chassis_by_hostname); - ovs_assert(ovs_list_is_empty(&lflows)); - if (!op) { + ni->sbrec_chassis_by_hostname)) { goto fail; } add_op_to_northd_tracked_ports(&trk_lsps->updated, op); @@ -6017,170 +5762,7 @@ ovn_igmp_group_destroy(struct hmap *igmp_groups, * function of most of the northbound database. */ -struct ovn_lflow { - struct hmap_node hmap_node; - struct ovs_list list_node; /* For temporary list of lflows. Don't remove - at destroy. */ - - struct ovn_datapath *od; /* 'logical_datapath' in SB schema. */ - unsigned long *dpg_bitmap; /* Bitmap of all datapaths by their 'index'.*/ - enum ovn_stage stage; - uint16_t priority; - char *match; - char *actions; - char *io_port; - char *stage_hint; - char *ctrl_meter; - size_t n_ods; /* Number of datapaths referenced by 'od' and - * 'dpg_bitmap'. */ - struct ovn_dp_group *dpg; /* Link to unique Sb datapath group. */ - - struct ovs_list referenced_by; /* List of struct lflow_ref_node. */ - const char *where; - - struct uuid sb_uuid; /* SB DB row uuid, specified by northd. */ -}; - -static void ovn_lflow_destroy(struct hmap *lflows, struct ovn_lflow *lflow); -static struct ovn_lflow *ovn_lflow_find(const struct hmap *lflows, - const struct ovn_datapath *od, - enum ovn_stage stage, - uint16_t priority, const char *match, - const char *actions, - const char *ctrl_meter, uint32_t hash); - -static char * -ovn_lflow_hint(const struct ovsdb_idl_row *row) -{ - if (!row) { - return NULL; - } - return xasprintf("%08x", row->uuid.parts[0]); -} - -static bool -ovn_lflow_equal(const struct ovn_lflow *a, const struct ovn_datapath *od, - enum ovn_stage stage, uint16_t priority, const char *match, - const char *actions, const char *ctrl_meter) -{ - return (a->od == od - && a->stage == stage - && a->priority == priority - && !strcmp(a->match, match) - && !strcmp(a->actions, actions) - && nullable_string_is_equal(a->ctrl_meter, ctrl_meter)); -} - -enum { - STATE_NULL, /* parallelization is off */ - STATE_INIT_HASH_SIZES, /* parallelization is on; hashes sizing needed */ - STATE_USE_PARALLELIZATION /* parallelization is on */ -}; -static int parallelization_state = STATE_NULL; - -static void -ovn_lflow_init(struct ovn_lflow *lflow, struct ovn_datapath *od, - size_t dp_bitmap_len, enum ovn_stage stage, uint16_t priority, - char *match, char *actions, char *io_port, char *ctrl_meter, - char *stage_hint, const char *where) -{ - ovs_list_init(&lflow->list_node); - ovs_list_init(&lflow->referenced_by); - lflow->dpg_bitmap = bitmap_allocate(dp_bitmap_len); - lflow->od = od; - lflow->stage = stage; - lflow->priority = priority; - lflow->match = match; - lflow->actions = actions; - lflow->io_port = io_port; - lflow->stage_hint = stage_hint; - lflow->ctrl_meter = ctrl_meter; - lflow->dpg = NULL; - lflow->where = where; - lflow->sb_uuid = UUID_ZERO; -} - -/* The lflow_hash_lock is a mutex array that protects updates to the shared - * lflow table across threads when parallel lflow build and dp-group are both - * enabled. To avoid high contention between threads, a big array of mutexes - * are used instead of just one. This is possible because when parallel build - * is used we only use hmap_insert_fast() to update the hmap, which would not - * touch the bucket array but only the list in a single bucket. We only need to - * make sure that when adding lflows to the same hash bucket, the same lock is - * used, so that no two threads can add to the bucket at the same time. It is - * ok that the same lock is used to protect multiple buckets, so a fixed sized - * mutex array is used instead of 1-1 mapping to the hash buckets. This - * simplies the implementation while effectively reduces lock contention - * because the chance that different threads contending the same lock amongst - * the big number of locks is very low. */ -#define LFLOW_HASH_LOCK_MASK 0xFFFF -static struct ovs_mutex lflow_hash_locks[LFLOW_HASH_LOCK_MASK + 1]; - -static void -lflow_hash_lock_init(void) -{ - if (!lflow_hash_lock_initialized) { - for (size_t i = 0; i < LFLOW_HASH_LOCK_MASK + 1; i++) { - ovs_mutex_init(&lflow_hash_locks[i]); - } - lflow_hash_lock_initialized = true; - } -} - -static void -lflow_hash_lock_destroy(void) -{ - if (lflow_hash_lock_initialized) { - for (size_t i = 0; i < LFLOW_HASH_LOCK_MASK + 1; i++) { - ovs_mutex_destroy(&lflow_hash_locks[i]); - } - } - lflow_hash_lock_initialized = false; -} - -/* Full thread safety analysis is not possible with hash locks, because - * they are taken conditionally based on the 'parallelization_state' and - * a flow hash. Also, the order in which two hash locks are taken is not - * predictable during the static analysis. - * - * Since the order of taking two locks depends on a random hash, to avoid - * ABBA deadlocks, no two hash locks can be nested. In that sense an array - * of hash locks is similar to a single mutex. - * - * Using a fake mutex to partially simulate thread safety restrictions, as - * if it were actually a single mutex. - * - * OVS_NO_THREAD_SAFETY_ANALYSIS below allows us to ignore conditional - * nature of the lock. Unlike other attributes, it applies to the - * implementation and not to the interface. So, we can define a function - * that acquires the lock without analysing the way it does that. - */ -extern struct ovs_mutex fake_hash_mutex; - -static struct ovs_mutex * -lflow_hash_lock(const struct hmap *lflow_map, uint32_t hash) - OVS_ACQUIRES(fake_hash_mutex) - OVS_NO_THREAD_SAFETY_ANALYSIS -{ - struct ovs_mutex *hash_lock = NULL; - - if (parallelization_state == STATE_USE_PARALLELIZATION) { - hash_lock = - &lflow_hash_locks[hash & lflow_map->mask & LFLOW_HASH_LOCK_MASK]; - ovs_mutex_lock(hash_lock); - } - return hash_lock; -} - -static void -lflow_hash_unlock(struct ovs_mutex *hash_lock) - OVS_RELEASES(fake_hash_mutex) - OVS_NO_THREAD_SAFETY_ANALYSIS -{ - if (hash_lock) { - ovs_mutex_unlock(hash_lock); - } -} +int parallelization_state = STATE_NULL; /* This thread-local var is used for parallel lflow building when dp-groups is @@ -6193,240 +5775,7 @@ lflow_hash_unlock(struct ovs_mutex *hash_lock) * threads are collected to fix the lflow hmap's size (by the function * fix_flow_map_size()). * */ -static thread_local size_t thread_lflow_counter = 0; - -/* Adds an OVN datapath to a datapath group of existing logical flow. - * Version to use when hash bucket locking is NOT required or the corresponding - * hash lock is already taken. */ -static void -ovn_dp_group_add_with_reference(struct ovn_lflow *lflow_ref, - const struct ovn_datapath *od, - const unsigned long *dp_bitmap, - size_t bitmap_len) - OVS_REQUIRES(fake_hash_mutex) -{ - if (od) { - bitmap_set1(lflow_ref->dpg_bitmap, od->index); - } - if (dp_bitmap) { - bitmap_or(lflow_ref->dpg_bitmap, dp_bitmap, bitmap_len); - } -} - -/* This global variable collects the lflows generated by do_ovn_lflow_add(). - * start_collecting_lflows() will enable the lflow collection and the calls to - * do_ovn_lflow_add (or the macros ovn_lflow_add_...) will add generated lflows - * to the list. end_collecting_lflows() will disable it. */ -static thread_local struct ovs_list collected_lflows; -static thread_local bool collecting_lflows = false; - -static void -start_collecting_lflows(void) -{ - ovs_assert(!collecting_lflows); - ovs_list_init(&collected_lflows); - collecting_lflows = true; -} - -static void -end_collecting_lflows(void) -{ - ovs_assert(collecting_lflows); - collecting_lflows = false; -} - -/* Adds a row with the specified contents to the Logical_Flow table. - * Version to use when hash bucket locking is NOT required. */ -static void -do_ovn_lflow_add(struct hmap *lflow_map, const struct ovn_datapath *od, - const unsigned long *dp_bitmap, size_t dp_bitmap_len, - uint32_t hash, enum ovn_stage stage, uint16_t priority, - const char *match, const char *actions, const char *io_port, - const struct ovsdb_idl_row *stage_hint, - const char *where, const char *ctrl_meter) - OVS_REQUIRES(fake_hash_mutex) -{ - - struct ovn_lflow *old_lflow; - struct ovn_lflow *lflow; - - size_t bitmap_len = od ? ods_size(od->datapaths) : dp_bitmap_len; - ovs_assert(bitmap_len); - - if (collecting_lflows) { - ovs_assert(od); - ovs_assert(!dp_bitmap); - } else { - old_lflow = ovn_lflow_find(lflow_map, NULL, stage, priority, match, - actions, ctrl_meter, hash); - if (old_lflow) { - ovn_dp_group_add_with_reference(old_lflow, od, dp_bitmap, - bitmap_len); - return; - } - } - - lflow = xmalloc(sizeof *lflow); - /* While adding new logical flows we're not setting single datapath, but - * collecting a group. 'od' will be updated later for all flows with only - * one datapath in a group, so it could be hashed correctly. */ - ovn_lflow_init(lflow, NULL, bitmap_len, stage, priority, - xstrdup(match), xstrdup(actions), - io_port ? xstrdup(io_port) : NULL, - nullable_xstrdup(ctrl_meter), - ovn_lflow_hint(stage_hint), where); - - ovn_dp_group_add_with_reference(lflow, od, dp_bitmap, bitmap_len); - - if (parallelization_state != STATE_USE_PARALLELIZATION) { - hmap_insert(lflow_map, &lflow->hmap_node, hash); - } else { - hmap_insert_fast(lflow_map, &lflow->hmap_node, hash); - thread_lflow_counter++; - } - - if (collecting_lflows) { - ovs_list_insert(&collected_lflows, &lflow->list_node); - } -} - -/* Adds a row with the specified contents to the Logical_Flow table. */ -static void -ovn_lflow_add_at(struct hmap *lflow_map, const struct ovn_datapath *od, - const unsigned long *dp_bitmap, size_t dp_bitmap_len, - enum ovn_stage stage, uint16_t priority, - const char *match, const char *actions, const char *io_port, - const char *ctrl_meter, - const struct ovsdb_idl_row *stage_hint, const char *where) - OVS_EXCLUDED(fake_hash_mutex) -{ - struct ovs_mutex *hash_lock; - uint32_t hash; - - ovs_assert(!od || - ovn_stage_to_datapath_type(stage) == ovn_datapath_get_type(od)); - - hash = ovn_logical_flow_hash(ovn_stage_get_table(stage), - ovn_stage_get_pipeline(stage), - priority, match, - actions); - - hash_lock = lflow_hash_lock(lflow_map, hash); - do_ovn_lflow_add(lflow_map, od, dp_bitmap, dp_bitmap_len, hash, stage, - priority, match, actions, io_port, stage_hint, where, - ctrl_meter); - lflow_hash_unlock(hash_lock); -} - -static void -__ovn_lflow_add_default_drop(struct hmap *lflow_map, - struct ovn_datapath *od, - enum ovn_stage stage, - const char *where) -{ - ovn_lflow_add_at(lflow_map, od, NULL, 0, stage, 0, "1", - debug_drop_action(), - NULL, NULL, NULL, where ); -} - -/* Adds a row with the specified contents to the Logical_Flow table. */ -#define ovn_lflow_add_with_hint__(LFLOW_MAP, OD, STAGE, PRIORITY, MATCH, \ - ACTIONS, IN_OUT_PORT, CTRL_METER, \ - STAGE_HINT) \ - ovn_lflow_add_at(LFLOW_MAP, OD, NULL, 0, STAGE, PRIORITY, MATCH, ACTIONS, \ - IN_OUT_PORT, CTRL_METER, STAGE_HINT, OVS_SOURCE_LOCATOR) - -#define ovn_lflow_add_with_hint(LFLOW_MAP, OD, STAGE, PRIORITY, MATCH, \ - ACTIONS, STAGE_HINT) \ - ovn_lflow_add_at(LFLOW_MAP, OD, NULL, 0, STAGE, PRIORITY, MATCH, ACTIONS, \ - NULL, NULL, STAGE_HINT, OVS_SOURCE_LOCATOR) - -#define ovn_lflow_add_with_dp_group(LFLOW_MAP, DP_BITMAP, DP_BITMAP_LEN, \ - STAGE, PRIORITY, MATCH, ACTIONS, \ - STAGE_HINT) \ - ovn_lflow_add_at(LFLOW_MAP, NULL, DP_BITMAP, DP_BITMAP_LEN, STAGE, \ - PRIORITY, MATCH, ACTIONS, NULL, NULL, STAGE_HINT, \ - OVS_SOURCE_LOCATOR) - -#define ovn_lflow_add_default_drop(LFLOW_MAP, OD, STAGE) \ - __ovn_lflow_add_default_drop(LFLOW_MAP, OD, STAGE, OVS_SOURCE_LOCATOR) - - -/* This macro is similar to ovn_lflow_add_with_hint, except that it requires - * the IN_OUT_PORT argument, which tells the lport name that appears in the - * MATCH, which helps ovn-controller to bypass lflows parsing when the lport is - * not local to the chassis. The critiera of the lport to be added using this - * argument: - * - * - For ingress pipeline, the lport that is used to match "inport". - * - For egress pipeline, the lport that is used to match "outport". - * - * For now, only LS pipelines should use this macro. */ -#define ovn_lflow_add_with_lport_and_hint(LFLOW_MAP, OD, STAGE, PRIORITY, \ - MATCH, ACTIONS, IN_OUT_PORT, \ - STAGE_HINT) \ - ovn_lflow_add_at(LFLOW_MAP, OD, NULL, 0, STAGE, PRIORITY, MATCH, ACTIONS, \ - IN_OUT_PORT, NULL, STAGE_HINT, OVS_SOURCE_LOCATOR) - -#define ovn_lflow_add(LFLOW_MAP, OD, STAGE, PRIORITY, MATCH, ACTIONS) \ - ovn_lflow_add_at(LFLOW_MAP, OD, NULL, 0, STAGE, PRIORITY, MATCH, ACTIONS, \ - NULL, NULL, NULL, OVS_SOURCE_LOCATOR) - -#define ovn_lflow_metered(LFLOW_MAP, OD, STAGE, PRIORITY, MATCH, ACTIONS, \ - CTRL_METER) \ - ovn_lflow_add_with_hint__(LFLOW_MAP, OD, STAGE, PRIORITY, MATCH, \ - ACTIONS, NULL, CTRL_METER, NULL) - -static struct ovn_lflow * -ovn_lflow_find(const struct hmap *lflows, const struct ovn_datapath *od, - enum ovn_stage stage, uint16_t priority, - const char *match, const char *actions, const char *ctrl_meter, - uint32_t hash) -{ - struct ovn_lflow *lflow; - HMAP_FOR_EACH_WITH_HASH (lflow, hmap_node, hash, lflows) { - if (ovn_lflow_equal(lflow, od, stage, priority, match, actions, - ctrl_meter)) { - return lflow; - } - } - return NULL; -} - -static void -ovn_lflow_destroy(struct hmap *lflows, struct ovn_lflow *lflow) -{ - if (lflow) { - if (lflows) { - hmap_remove(lflows, &lflow->hmap_node); - } - bitmap_free(lflow->dpg_bitmap); - free(lflow->match); - free(lflow->actions); - free(lflow->io_port); - free(lflow->stage_hint); - free(lflow->ctrl_meter); - struct lflow_ref_node *l; - LIST_FOR_EACH_SAFE (l, ref_list_node, &lflow->referenced_by) { - ovs_list_remove(&l->lflow_list_node); - ovs_list_remove(&l->ref_list_node); - free(l); - } - free(lflow); - } -} - -static void -link_ovn_port_to_lflows(struct ovn_port *op, struct ovs_list *lflows) -{ - struct ovn_lflow *f; - LIST_FOR_EACH (f, list_node, lflows) { - struct lflow_ref_node *lfrn = xmalloc(sizeof *lfrn); - lfrn->lflow = f; - ovs_list_insert(&op->lflows, &lfrn->lflow_list_node); - ovs_list_insert(&f->referenced_by, &lfrn->ref_list_node); - } -} +thread_local size_t thread_lflow_counter = 0; static bool build_dhcpv4_action(struct ovn_port *op, ovs_be32 offer_ip, @@ -6604,8 +5953,8 @@ build_dhcpv6_action(struct ovn_port *op, struct in6_addr *offer_ip, * build_lswitch_lflows_admission_control() handles the port security. */ static void -build_lswitch_port_sec_op(struct ovn_port *op, struct hmap *lflows, - struct ds *actions, struct ds *match) +build_lswitch_port_sec_op(struct ovn_port *op, struct lflow_table *lflows, + struct ds *actions, struct ds *match) { ovs_assert(op->nbsp); @@ -6621,13 +5970,13 @@ build_lswitch_port_sec_op(struct ovn_port *op, struct hmap *lflows, ovn_lflow_add_with_lport_and_hint( lflows, op->od, S_SWITCH_IN_CHECK_PORT_SEC, 100, ds_cstr(match), REGBIT_PORT_SEC_DROP" = 1; next;", - op->key, &op->nbsp->header_); + op->key, &op->nbsp->header_, op->lflow_ref); ds_clear(match); ds_put_format(match, "outport == %s", op->json_key); ovn_lflow_add_with_lport_and_hint( lflows, op->od, S_SWITCH_IN_L2_UNKNOWN, 50, ds_cstr(match), - debug_drop_action(), op->key, &op->nbsp->header_); + debug_drop_action(), op->key, &op->nbsp->header_, op->lflow_ref); return; } @@ -6643,14 +5992,16 @@ build_lswitch_port_sec_op(struct ovn_port *op, struct hmap *lflows, ovn_lflow_add_with_lport_and_hint(lflows, op->od, S_SWITCH_IN_CHECK_PORT_SEC, 70, ds_cstr(match), ds_cstr(actions), - op->key, &op->nbsp->header_); + op->key, &op->nbsp->header_, + op->lflow_ref); } else if (queue_id) { ds_put_cstr(actions, REGBIT_PORT_SEC_DROP" = check_in_port_sec(); next;"); ovn_lflow_add_with_lport_and_hint(lflows, op->od, S_SWITCH_IN_CHECK_PORT_SEC, 70, ds_cstr(match), ds_cstr(actions), - op->key, &op->nbsp->header_); + op->key, &op->nbsp->header_, + op->lflow_ref); if (!lsp_is_localnet(op->nbsp) && !op->od->n_localnet_ports) { return; @@ -6665,7 +6016,8 @@ build_lswitch_port_sec_op(struct ovn_port *op, struct hmap *lflows, ovn_lflow_add_with_lport_and_hint(lflows, op->od, S_SWITCH_OUT_APPLY_PORT_SEC, 100, ds_cstr(match), ds_cstr(actions), - op->key, &op->nbsp->header_); + op->key, &op->nbsp->header_, + op->lflow_ref); } else if (op->od->n_localnet_ports) { ds_put_format(match, "outport == %s && inport == %s", op->od->localnet_ports[0]->json_key, @@ -6674,15 +6026,16 @@ build_lswitch_port_sec_op(struct ovn_port *op, struct hmap *lflows, S_SWITCH_OUT_APPLY_PORT_SEC, 110, ds_cstr(match), ds_cstr(actions), op->od->localnet_ports[0]->key, - &op->od->localnet_ports[0]->nbsp->header_); + &op->od->localnet_ports[0]->nbsp->header_, + op->lflow_ref); } } } static void build_lswitch_learn_fdb_op( - struct ovn_port *op, struct hmap *lflows, - struct ds *actions, struct ds *match) + struct ovn_port *op, struct lflow_table *lflows, + struct ds *actions, struct ds *match) { ovs_assert(op->nbsp); @@ -6699,7 +6052,8 @@ build_lswitch_learn_fdb_op( ovn_lflow_add_with_lport_and_hint(lflows, op->od, S_SWITCH_IN_LOOKUP_FDB, 100, ds_cstr(match), ds_cstr(actions), - op->key, &op->nbsp->header_); + op->key, &op->nbsp->header_, + op->lflow_ref); ds_put_cstr(match, " && "REGBIT_LKUP_FDB" == 0"); ds_clear(actions); @@ -6707,13 +6061,14 @@ build_lswitch_learn_fdb_op( ovn_lflow_add_with_lport_and_hint(lflows, op->od, S_SWITCH_IN_PUT_FDB, 100, ds_cstr(match), ds_cstr(actions), op->key, - &op->nbsp->header_); + &op->nbsp->header_, + op->lflow_ref); } } static void build_lswitch_learn_fdb_od( - struct ovn_datapath *od, struct hmap *lflows) + struct ovn_datapath *od, struct lflow_table *lflows) { ovs_assert(od->nbs); ovn_lflow_add(lflows, od, S_SWITCH_IN_LOOKUP_FDB, 0, "1", "next;"); @@ -6727,7 +6082,7 @@ build_lswitch_learn_fdb_od( * (priority 100). */ static void build_lswitch_output_port_sec_od(struct ovn_datapath *od, - struct hmap *lflows) + struct lflow_table *lflows) { ovs_assert(od->nbs); ovn_lflow_add(lflows, od, S_SWITCH_OUT_CHECK_PORT_SEC, 100, @@ -6745,7 +6100,7 @@ static void skip_port_from_conntrack(const struct ovn_datapath *od, struct ovn_port *op, bool has_stateful_acl, enum ovn_stage in_stage, enum ovn_stage out_stage, uint16_t priority, - struct hmap *lflows) + struct lflow_table *lflows) { /* Can't use ct() for router ports. Consider the following configuration: * lp1(10.0.0.2) on hostA--ls1--lr0--ls2--lp2(10.0.1.2) on hostB, For a @@ -6767,10 +6122,10 @@ skip_port_from_conntrack(const struct ovn_datapath *od, struct ovn_port *op, ovn_lflow_add_with_lport_and_hint(lflows, od, in_stage, priority, ingress_match, ingress_action, - op->key, &op->nbsp->header_); + op->key, &op->nbsp->header_, NULL); ovn_lflow_add_with_lport_and_hint(lflows, od, out_stage, priority, egress_match, egress_action, - op->key, &op->nbsp->header_); + op->key, &op->nbsp->header_, NULL); free(ingress_match); free(egress_match); @@ -6779,7 +6134,7 @@ skip_port_from_conntrack(const struct ovn_datapath *od, struct ovn_port *op, static void build_stateless_filter(const struct ovn_datapath *od, const struct nbrec_acl *acl, - struct hmap *lflows) + struct lflow_table *lflows) { const char *action = REGBIT_ACL_STATELESS" = 1; next;"; if (!strcmp(acl->direction, "from-lport")) { @@ -6800,7 +6155,7 @@ build_stateless_filter(const struct ovn_datapath *od, static void build_stateless_filters(const struct ovn_datapath *od, const struct ls_port_group_table *ls_port_groups, - struct hmap *lflows) + struct lflow_table *lflows) { for (size_t i = 0; i < od->nbs->n_acls; i++) { const struct nbrec_acl *acl = od->nbs->acls[i]; @@ -6828,7 +6183,7 @@ build_stateless_filters(const struct ovn_datapath *od, } static void -build_pre_acls(struct ovn_datapath *od, struct hmap *lflows) +build_pre_acls(struct ovn_datapath *od, struct lflow_table *lflows) { /* Ingress and Egress Pre-ACL Table (Priority 0): Packets are * allowed by default. */ @@ -6843,16 +6198,17 @@ build_pre_acls(struct ovn_datapath *od, struct hmap *lflows) } static void -build_ls_stateful_rec_pre_acls(const struct ls_stateful_record *ls_sful_rec, - const struct ls_port_group_table *ls_port_groups, - struct hmap *lflows) +build_ls_stateful_rec_pre_acls( + const struct ls_stateful_record *ls_stateful_rec, + const struct ls_port_group_table *ls_port_groups, + struct lflow_table *lflows) { - const struct ovn_datapath *od = ls_sful_rec->od; + const struct ovn_datapath *od = ls_stateful_rec->od; /* If there are any stateful ACL rules in this datapath, we may * send IP packets for some (allow) filters through the conntrack action, * which handles defragmentation, in order to match L4 headers. */ - if (ls_sful_rec->has_stateful_acl) { + if (ls_stateful_rec->has_stateful_acl) { for (size_t i = 0; i < od->n_router_ports; i++) { struct ovn_port *op = od->router_ports[i]; if (op->enable_router_port_acl) { @@ -6901,7 +6257,7 @@ build_ls_stateful_rec_pre_acls(const struct ls_stateful_record *ls_sful_rec, REGBIT_CONNTRACK_DEFRAG" = 1; next;"); ovn_lflow_add(lflows, od, S_SWITCH_OUT_PRE_ACL, 100, "ip", REGBIT_CONNTRACK_DEFRAG" = 1; next;"); - } else if (ls_sful_rec->has_lb_vip) { + } else if (ls_stateful_rec->has_lb_vip) { /* We'll build stateless filters if there are LB rules so that * the stateless flows are not tracked in pre-lb. */ build_stateless_filters(od, ls_port_groups, lflows); @@ -6968,7 +6324,7 @@ build_empty_lb_event_flow(struct ovn_lb_vip *lb_vip, static void build_interconn_mcast_snoop_flows(struct ovn_datapath *od, const struct shash *meter_groups, - struct hmap *lflows) + struct lflow_table *lflows) { struct mcast_switch_info *mcast_sw_info = &od->mcast_info.sw; if (!mcast_sw_info->enabled @@ -7002,7 +6358,7 @@ build_interconn_mcast_snoop_flows(struct ovn_datapath *od, static void build_pre_lb(struct ovn_datapath *od, const struct shash *meter_groups, - struct hmap *lflows) + struct lflow_table *lflows) { /* Handle IGMP/MLD packets crossing AZs. */ build_interconn_mcast_snoop_flows(od, meter_groups, lflows); @@ -7038,7 +6394,7 @@ build_pre_lb(struct ovn_datapath *od, const struct shash *meter_groups, static void build_ls_stateful_rec_pre_lb(const struct ls_stateful_record *ls_stateful_rec, - struct hmap *lflows) + struct lflow_table *lflows) { const struct ovn_datapath *od = ls_stateful_rec->od; @@ -7104,7 +6460,7 @@ build_ls_stateful_rec_pre_lb(const struct ls_stateful_record *ls_stateful_rec, static void build_pre_stateful(struct ovn_datapath *od, const struct chassis_features *features, - struct hmap *lflows) + struct lflow_table *lflows) { /* Ingress and Egress pre-stateful Table (Priority 0): Packets are * allowed by default. */ @@ -7134,11 +6490,11 @@ build_pre_stateful(struct ovn_datapath *od, } static void -build_acl_hints(const struct ls_stateful_record *ls_sful_rec, +build_acl_hints(const struct ls_stateful_record *ls_stateful_rec, const struct chassis_features *features, - struct hmap *lflows) + struct lflow_table *lflows) { - const struct ovn_datapath *od = ls_sful_rec->od; + const struct ovn_datapath *od = ls_stateful_rec->od; /* This stage builds hints for the IN/OUT_ACL stage. Based on various * combinations of ct flags packets may hit only a subset of the logical @@ -7163,13 +6519,14 @@ build_acl_hints(const struct ls_stateful_record *ls_sful_rec, const char *match; /* In any case, advance to the next stage. */ - if (!ls_sful_rec->has_acls && !ls_sful_rec->has_lb_vip) { + if (!ls_stateful_rec->has_acls && !ls_stateful_rec->has_lb_vip) { ovn_lflow_add(lflows, od, stage, UINT16_MAX, "1", "next;"); } else { ovn_lflow_add(lflows, od, stage, 0, "1", "next;"); } - if (!ls_sful_rec->has_stateful_acl && !ls_sful_rec->has_lb_vip) { + if (!ls_stateful_rec->has_stateful_acl && + !ls_stateful_rec->has_lb_vip) { continue; } @@ -7305,7 +6662,7 @@ build_acl_log(struct ds *actions, const struct nbrec_acl *acl, } static void -consider_acl(struct hmap *lflows, const struct ovn_datapath *od, +consider_acl(struct lflow_table *lflows, const struct ovn_datapath *od, const struct nbrec_acl *acl, bool has_stateful, bool ct_masked_mark, const struct shash *meter_groups, uint64_t max_acl_tier, struct ds *match, struct ds *actions) @@ -7533,7 +6890,7 @@ ovn_update_ipv6_options(struct hmap *lr_ports) static void build_acl_action_lflows(const struct ls_stateful_record *ls_stateful_rec, - struct hmap *lflows, + struct lflow_table *lflows, const char *default_acl_action, const struct shash *meter_groups, struct ds *match, @@ -7610,7 +6967,8 @@ build_acl_action_lflows(const struct ls_stateful_record *ls_stateful_rec, } static void -build_acl_log_related_flows(const struct ovn_datapath *od, struct hmap *lflows, +build_acl_log_related_flows(const struct ovn_datapath *od, + struct lflow_table *lflows, const struct nbrec_acl *acl, bool has_stateful, bool ct_masked_mark, const struct shash *meter_groups, @@ -7685,7 +7043,7 @@ build_acl_log_related_flows(const struct ovn_datapath *od, struct hmap *lflows, static void build_acls(const struct ls_stateful_record *ls_stateful_rec, const struct chassis_features *features, - struct hmap *lflows, + struct lflow_table *lflows, const struct ls_port_group_table *ls_port_groups, const struct shash *meter_groups) { @@ -7931,7 +7289,7 @@ build_acls(const struct ls_stateful_record *ls_stateful_rec, } static void -build_qos(struct ovn_datapath *od, struct hmap *lflows) { +build_qos(struct ovn_datapath *od, struct lflow_table *lflows) { struct ds action = DS_EMPTY_INITIALIZER; ovn_lflow_add(lflows, od, S_SWITCH_IN_QOS_MARK, 0, "1", "next;"); @@ -7992,7 +7350,7 @@ build_qos(struct ovn_datapath *od, struct hmap *lflows) { } static void -build_lb_rules_pre_stateful(struct hmap *lflows, +build_lb_rules_pre_stateful(struct lflow_table *lflows, struct ovn_lb_datapaths *lb_dps, bool ct_lb_mark, const struct ovn_datapaths *ls_datapaths, @@ -8094,7 +7452,8 @@ build_lb_rules_pre_stateful(struct hmap *lflows, * */ static void -build_lb_affinity_lr_flows(struct hmap *lflows, const struct ovn_northd_lb *lb, +build_lb_affinity_lr_flows(struct lflow_table *lflows, + const struct ovn_northd_lb *lb, struct ovn_lb_vip *lb_vip, char *new_lb_match, char *lb_action, const unsigned long *dp_bitmap, const struct ovn_datapaths *lr_datapaths) @@ -8280,7 +7639,7 @@ build_lb_affinity_lr_flows(struct hmap *lflows, const struct ovn_northd_lb *lb, * */ static void -build_lb_affinity_ls_flows(struct hmap *lflows, +build_lb_affinity_ls_flows(struct lflow_table *lflows, struct ovn_lb_datapaths *lb_dps, struct ovn_lb_vip *lb_vip, const struct ovn_datapaths *ls_datapaths) @@ -8423,7 +7782,7 @@ build_lb_affinity_ls_flows(struct hmap *lflows, static void build_lswitch_lb_affinity_default_flows(struct ovn_datapath *od, - struct hmap *lflows) + struct lflow_table *lflows) { ovs_assert(od->nbs); ovn_lflow_add(lflows, od, S_SWITCH_IN_LB_AFF_CHECK, 0, "1", "next;"); @@ -8432,7 +7791,7 @@ build_lswitch_lb_affinity_default_flows(struct ovn_datapath *od, static void build_lrouter_lb_affinity_default_flows(struct ovn_datapath *od, - struct hmap *lflows) + struct lflow_table *lflows) { ovs_assert(od->nbr); ovn_lflow_add(lflows, od, S_ROUTER_IN_LB_AFF_CHECK, 0, "1", "next;"); @@ -8440,7 +7799,7 @@ build_lrouter_lb_affinity_default_flows(struct ovn_datapath *od, } static void -build_lb_rules(struct hmap *lflows, struct ovn_lb_datapaths *lb_dps, +build_lb_rules(struct lflow_table *lflows, struct ovn_lb_datapaths *lb_dps, const struct ovn_datapaths *ls_datapaths, const struct chassis_features *features, struct ds *match, struct ds *action, const struct shash *meter_groups, @@ -8520,7 +7879,7 @@ build_lb_rules(struct hmap *lflows, struct ovn_lb_datapaths *lb_dps, static void build_stateful(struct ovn_datapath *od, const struct chassis_features *features, - struct hmap *lflows) + struct lflow_table *lflows) { const char *ct_block_action = features->ct_no_masked_label ? "ct_mark.blocked" @@ -8570,7 +7929,7 @@ build_stateful(struct ovn_datapath *od, static void build_lb_hairpin(const struct ls_stateful_record *ls_stateful_rec, - struct hmap *lflows) + struct lflow_table *lflows) { const struct ovn_datapath *od = ls_stateful_rec->od; @@ -8629,7 +7988,7 @@ build_lb_hairpin(const struct ls_stateful_record *ls_stateful_rec, } static void -build_vtep_hairpin(struct ovn_datapath *od, struct hmap *lflows) +build_vtep_hairpin(struct ovn_datapath *od, struct lflow_table *lflows) { if (!od->has_vtep_lports) { /* There is no need in these flows if datapath has no vtep lports. */ @@ -8677,7 +8036,7 @@ build_vtep_hairpin(struct ovn_datapath *od, struct hmap *lflows) /* Build logical flows for the forwarding groups */ static void -build_fwd_group_lflows(struct ovn_datapath *od, struct hmap *lflows) +build_fwd_group_lflows(struct ovn_datapath *od, struct lflow_table *lflows) { ovs_assert(od->nbs); if (!od->nbs->n_forwarding_groups) { @@ -8858,7 +8217,8 @@ build_lswitch_rport_arp_req_self_orig_flow(struct ovn_port *op, uint32_t priority, const struct ovn_datapath *od, const struct lr_nat_record *lrnat_rec, - struct hmap *lflows) + struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { struct ds eth_src = DS_EMPTY_INITIALIZER; struct ds match = DS_EMPTY_INITIALIZER; @@ -8882,8 +8242,10 @@ build_lswitch_rport_arp_req_self_orig_flow(struct ovn_port *op, ds_put_format(&match, "eth.src == %s && (arp.op == 1 || rarp.op == 3 || nd_ns)", ds_cstr(ð_src)); - ovn_lflow_add(lflows, od, S_SWITCH_IN_L2_LKUP, priority, ds_cstr(&match), - "outport = \""MC_FLOOD_L2"\"; output;"); + ovn_lflow_add_with_lflow_ref(lflows, od, S_SWITCH_IN_L2_LKUP, priority, + ds_cstr(&match), + "outport = \""MC_FLOOD_L2"\"; output;", + lflow_ref); ds_destroy(ð_src); ds_destroy(&match); @@ -8948,11 +8310,11 @@ lrouter_port_ipv6_reachable(const struct ovn_port *op, * switching domain as regular broadcast. */ static void -build_lswitch_rport_arp_req_flow(const char *ips, int addr_family, - struct ovn_port *patch_op, - const struct ovn_datapath *od, - uint32_t priority, struct hmap *lflows, - const struct ovsdb_idl_row *stage_hint) +build_lswitch_rport_arp_req_flow( + const char *ips, int addr_family, struct ovn_port *patch_op, + const struct ovn_datapath *od, uint32_t priority, + struct lflow_table *lflows, const struct ovsdb_idl_row *stage_hint, + struct lflow_ref *lflow_ref) { struct ds match = DS_EMPTY_INITIALIZER; struct ds actions = DS_EMPTY_INITIALIZER; @@ -8966,14 +8328,17 @@ build_lswitch_rport_arp_req_flow(const char *ips, int addr_family, ds_put_format(&actions, "clone {outport = %s; output; }; " "outport = \""MC_FLOOD_L2"\"; output;", patch_op->json_key); - ovn_lflow_add_with_hint(lflows, od, S_SWITCH_IN_L2_LKUP, - priority, ds_cstr(&match), - ds_cstr(&actions), stage_hint); + ovn_lflow_add_with_lflow_ref_hint(lflows, od, S_SWITCH_IN_L2_LKUP, + priority, ds_cstr(&match), + ds_cstr(&actions), stage_hint, + lflow_ref); } else { ds_put_format(&actions, "outport = %s; output;", patch_op->json_key); - ovn_lflow_add_with_hint(lflows, od, S_SWITCH_IN_L2_LKUP, priority, - ds_cstr(&match), ds_cstr(&actions), - stage_hint); + ovn_lflow_add_with_lflow_ref_hint(lflows, od, S_SWITCH_IN_L2_LKUP, + priority, ds_cstr(&match), + ds_cstr(&actions), + stage_hint, + lflow_ref); } ds_destroy(&match); @@ -8991,7 +8356,7 @@ static void build_lswitch_rport_arp_req_flows(struct ovn_port *op, struct ovn_datapath *sw_od, struct ovn_port *sw_op, - struct hmap *lflows, + struct lflow_table *lflows, const struct ovsdb_idl_row *stage_hint) { if (!op || !op->nbrp) { @@ -9009,12 +8374,12 @@ build_lswitch_rport_arp_req_flows(struct ovn_port *op, for (size_t i = 0; i < op->lrp_networks.n_ipv4_addrs; i++) { build_lswitch_rport_arp_req_flow( op->lrp_networks.ipv4_addrs[i].addr_s, AF_INET, sw_op, sw_od, 80, - lflows, stage_hint); + lflows, stage_hint, sw_op->lflow_ref); } for (size_t i = 0; i < op->lrp_networks.n_ipv6_addrs; i++) { build_lswitch_rport_arp_req_flow( op->lrp_networks.ipv6_addrs[i].addr_s, AF_INET6, sw_op, sw_od, 80, - lflows, stage_hint); + lflows, stage_hint, sw_op->lflow_ref); } } @@ -9029,7 +8394,8 @@ static void build_lswitch_rport_arp_req_flows_for_lbnats( struct ovn_port *op, const struct lr_stateful_record *lr_stateful_rec, const struct ovn_datapath *sw_od, struct ovn_port *sw_op, - struct hmap *lflows, const struct ovsdb_idl_row *stage_hint) + struct lflow_table *lflows, const struct ovsdb_idl_row *stage_hint, + struct lflow_ref *lflow_ref) { if (!op || !op->nbrp) { return; @@ -9057,7 +8423,7 @@ build_lswitch_rport_arp_req_flows_for_lbnats( lrouter_port_ipv4_reachable(op, ipv4_addr)) { build_lswitch_rport_arp_req_flow( ip_addr, AF_INET, sw_op, sw_od, 80, lflows, - stage_hint); + stage_hint, lflow_ref); } } SSET_FOR_EACH (ip_addr, &lr_stateful_rec->lb_ips->ips_v6_reachable) { @@ -9070,7 +8436,7 @@ build_lswitch_rport_arp_req_flows_for_lbnats( lrouter_port_ipv6_reachable(op, &ipv6_addr)) { build_lswitch_rport_arp_req_flow( ip_addr, AF_INET6, sw_op, sw_od, 80, lflows, - stage_hint); + stage_hint, lflow_ref); } } } @@ -9085,7 +8451,7 @@ build_lswitch_rport_arp_req_flows_for_lbnats( if (sw_od->n_router_ports != sw_od->nbs->n_ports) { build_lswitch_rport_arp_req_self_orig_flow(op, 75, sw_od, lr_stateful_rec->lrnat_rec, - lflows); + lflows, lflow_ref); } for (size_t i = 0; i < lr_stateful_rec->lrnat_rec->n_nat_entries; i++) { @@ -9109,14 +8475,14 @@ build_lswitch_rport_arp_req_flows_for_lbnats( nat->external_ip)) { build_lswitch_rport_arp_req_flow( nat->external_ip, AF_INET6, sw_op, sw_od, 80, lflows, - stage_hint); + stage_hint, lflow_ref); } } else { if (!sset_contains(&lr_stateful_rec->lb_ips->ips_v4, nat->external_ip)) { build_lswitch_rport_arp_req_flow( nat->external_ip, AF_INET, sw_op, sw_od, 80, lflows, - stage_hint); + stage_hint, lflow_ref); } } } @@ -9143,7 +8509,7 @@ build_lswitch_rport_arp_req_flows_for_lbnats( nat->external_ip)) { build_lswitch_rport_arp_req_flow( nat->external_ip, AF_INET6, sw_op, sw_od, 80, lflows, - stage_hint); + stage_hint, lflow_ref); } } else { if (!lr_stateful_rec || @@ -9151,7 +8517,7 @@ build_lswitch_rport_arp_req_flows_for_lbnats( nat->external_ip)) { build_lswitch_rport_arp_req_flow( nat->external_ip, AF_INET, sw_op, sw_od, 80, lflows, - stage_hint); + stage_hint, lflow_ref); } } } @@ -9162,7 +8528,7 @@ build_dhcpv4_options_flows(struct ovn_port *op, struct lport_addresses *lsp_addrs, struct ovn_port *inport, bool is_external, const struct shash *meter_groups, - struct hmap *lflows) + struct lflow_table *lflows) { struct ds match = DS_EMPTY_INITIALIZER; @@ -9193,7 +8559,7 @@ build_dhcpv4_options_flows(struct ovn_port *op, op->json_key); } - ovn_lflow_add_with_hint__(lflows, op->od, + ovn_lflow_add_with_lflow_ref_hint__(lflows, op->od, S_SWITCH_IN_DHCP_OPTIONS, 100, ds_cstr(&match), ds_cstr(&options_action), @@ -9201,7 +8567,8 @@ build_dhcpv4_options_flows(struct ovn_port *op, copp_meter_get(COPP_DHCPV4_OPTS, op->od->nbs->copp, meter_groups), - &op->nbsp->dhcpv4_options->header_); + &op->nbsp->dhcpv4_options->header_, + op->lflow_ref); ds_clear(&match); /* If REGBIT_DHCP_OPTS_RESULT is set, it means the @@ -9220,7 +8587,8 @@ build_dhcpv4_options_flows(struct ovn_port *op, ovn_lflow_add_with_lport_and_hint( lflows, op->od, S_SWITCH_IN_DHCP_RESPONSE, 100, ds_cstr(&match), ds_cstr(&response_action), inport->key, - &op->nbsp->dhcpv4_options->header_); + &op->nbsp->dhcpv4_options->header_, + op->lflow_ref); ds_destroy(&options_action); ds_destroy(&response_action); ds_destroy(&ipv4_addr_match); @@ -9247,7 +8615,8 @@ build_dhcpv4_options_flows(struct ovn_port *op, ovn_lflow_add_with_lport_and_hint( lflows, op->od, S_SWITCH_OUT_ACL_EVAL, 34000, ds_cstr(&match),dhcp_actions, op->key, - &op->nbsp->dhcpv4_options->header_); + &op->nbsp->dhcpv4_options->header_, + op->lflow_ref); } break; } @@ -9260,7 +8629,7 @@ build_dhcpv6_options_flows(struct ovn_port *op, struct lport_addresses *lsp_addrs, struct ovn_port *inport, bool is_external, const struct shash *meter_groups, - struct hmap *lflows) + struct lflow_table *lflows) { struct ds match = DS_EMPTY_INITIALIZER; @@ -9282,7 +8651,7 @@ build_dhcpv6_options_flows(struct ovn_port *op, op->json_key); } - ovn_lflow_add_with_hint__(lflows, op->od, + ovn_lflow_add_with_lflow_ref_hint__(lflows, op->od, S_SWITCH_IN_DHCP_OPTIONS, 100, ds_cstr(&match), ds_cstr(&options_action), @@ -9290,7 +8659,8 @@ build_dhcpv6_options_flows(struct ovn_port *op, copp_meter_get(COPP_DHCPV6_OPTS, op->od->nbs->copp, meter_groups), - &op->nbsp->dhcpv6_options->header_); + &op->nbsp->dhcpv6_options->header_, + op->lflow_ref); /* If REGBIT_DHCP_OPTS_RESULT is set to 1, it means the * put_dhcpv6_opts action is successful */ @@ -9298,7 +8668,7 @@ build_dhcpv6_options_flows(struct ovn_port *op, ovn_lflow_add_with_lport_and_hint( lflows, op->od, S_SWITCH_IN_DHCP_RESPONSE, 100, ds_cstr(&match), ds_cstr(&response_action), inport->key, - &op->nbsp->dhcpv6_options->header_); + &op->nbsp->dhcpv6_options->header_, op->lflow_ref); ds_destroy(&options_action); ds_destroy(&response_action); @@ -9330,7 +8700,8 @@ build_dhcpv6_options_flows(struct ovn_port *op, ovn_lflow_add_with_lport_and_hint( lflows, op->od, S_SWITCH_OUT_ACL_EVAL, 34000, ds_cstr(&match),dhcp6_actions, op->key, - &op->nbsp->dhcpv6_options->header_); + &op->nbsp->dhcpv6_options->header_, + op->lflow_ref); } break; } @@ -9341,7 +8712,7 @@ build_dhcpv6_options_flows(struct ovn_port *op, static void build_drop_arp_nd_flows_for_unbound_router_ports(struct ovn_port *op, const struct ovn_port *port, - struct hmap *lflows) + struct lflow_table *lflows) { struct ds match = DS_EMPTY_INITIALIZER; @@ -9361,7 +8732,7 @@ build_drop_arp_nd_flows_for_unbound_router_ports(struct ovn_port *op, ovn_lflow_add_with_lport_and_hint( lflows, op->od, S_SWITCH_IN_EXTERNAL_PORT, 100, ds_cstr(&match), debug_drop_action(), port->key, - &op->nbsp->header_); + &op->nbsp->header_, op->lflow_ref); } for (size_t l = 0; l < rp->lsp_addrs[k].n_ipv6_addrs; l++) { ds_clear(&match); @@ -9377,7 +8748,7 @@ build_drop_arp_nd_flows_for_unbound_router_ports(struct ovn_port *op, ovn_lflow_add_with_lport_and_hint( lflows, op->od, S_SWITCH_IN_EXTERNAL_PORT, 100, ds_cstr(&match), debug_drop_action(), port->key, - &op->nbsp->header_); + &op->nbsp->header_, op->lflow_ref); } ds_clear(&match); @@ -9393,7 +8764,8 @@ build_drop_arp_nd_flows_for_unbound_router_ports(struct ovn_port *op, 100, ds_cstr(&match), debug_drop_action(), port->key, - &op->nbsp->header_); + &op->nbsp->header_, + op->lflow_ref); } } } @@ -9408,7 +8780,7 @@ is_vlan_transparent(const struct ovn_datapath *od) static void build_lswitch_lflows_l2_unknown(struct ovn_datapath *od, - struct hmap *lflows) + struct lflow_table *lflows) { /* Ingress table 25/26: Destination lookup for unknown MACs. */ if (od->has_unknown) { @@ -9429,7 +8801,7 @@ static void build_lswitch_lflows_pre_acl_and_acl( struct ovn_datapath *od, const struct chassis_features *features, - struct hmap *lflows, + struct lflow_table *lflows, const struct shash *meter_groups) { ovs_assert(od->nbs); @@ -9445,7 +8817,7 @@ build_lswitch_lflows_pre_acl_and_acl( * 100). */ static void build_lswitch_lflows_admission_control(struct ovn_datapath *od, - struct hmap *lflows) + struct lflow_table *lflows) { ovs_assert(od->nbs); /* Logical VLANs not supported. */ @@ -9473,7 +8845,7 @@ build_lswitch_lflows_admission_control(struct ovn_datapath *od, static void build_lswitch_arp_nd_responder_skip_local(struct ovn_port *op, - struct hmap *lflows, + struct lflow_table *lflows, struct ds *match) { ovs_assert(op->nbsp); @@ -9485,14 +8857,14 @@ build_lswitch_arp_nd_responder_skip_local(struct ovn_port *op, ovn_lflow_add_with_lport_and_hint(lflows, op->od, S_SWITCH_IN_ARP_ND_RSP, 100, ds_cstr(match), "next;", op->key, - &op->nbsp->header_); + &op->nbsp->header_, op->lflow_ref); } /* Ingress table 19: ARP/ND responder, reply for known IPs. * (priority 50). */ static void build_lswitch_arp_nd_responder_known_ips(struct ovn_port *op, - struct hmap *lflows, + struct lflow_table *lflows, const struct hmap *ls_ports, const struct shash *meter_groups, struct ds *actions, @@ -9577,7 +8949,8 @@ build_lswitch_arp_nd_responder_known_ips(struct ovn_port *op, S_SWITCH_IN_ARP_ND_RSP, 100, ds_cstr(match), ds_cstr(actions), vparent, - &vp->nbsp->header_); + &vp->nbsp->header_, + op->lflow_ref); } free(tokstr); @@ -9621,11 +8994,12 @@ build_lswitch_arp_nd_responder_known_ips(struct ovn_port *op, "output;", op->lsp_addrs[i].ea_s, op->lsp_addrs[i].ea_s, op->lsp_addrs[i].ipv4_addrs[j].addr_s); - ovn_lflow_add_with_hint(lflows, op->od, - S_SWITCH_IN_ARP_ND_RSP, 50, - ds_cstr(match), - ds_cstr(actions), - &op->nbsp->header_); + ovn_lflow_add_with_lflow_ref_hint(lflows, op->od, + S_SWITCH_IN_ARP_ND_RSP, 50, + ds_cstr(match), + ds_cstr(actions), + &op->nbsp->header_, + op->lflow_ref); /* Do not reply to an ARP request from the port that owns * the address (otherwise a DHCP client that ARPs to check @@ -9644,7 +9018,8 @@ build_lswitch_arp_nd_responder_known_ips(struct ovn_port *op, S_SWITCH_IN_ARP_ND_RSP, 100, ds_cstr(match), "next;", op->key, - &op->nbsp->header_); + &op->nbsp->header_, + op->lflow_ref); } /* For ND solicitations, we need to listen for both the @@ -9674,15 +9049,16 @@ build_lswitch_arp_nd_responder_known_ips(struct ovn_port *op, op->lsp_addrs[i].ipv6_addrs[j].addr_s, op->lsp_addrs[i].ipv6_addrs[j].addr_s, op->lsp_addrs[i].ea_s); - ovn_lflow_add_with_hint__(lflows, op->od, - S_SWITCH_IN_ARP_ND_RSP, 50, - ds_cstr(match), - ds_cstr(actions), - NULL, - copp_meter_get(COPP_ND_NA, - op->od->nbs->copp, - meter_groups), - &op->nbsp->header_); + ovn_lflow_add_with_lflow_ref_hint__(lflows, op->od, + S_SWITCH_IN_ARP_ND_RSP, 50, + ds_cstr(match), + ds_cstr(actions), + NULL, + copp_meter_get(COPP_ND_NA, + op->od->nbs->copp, + meter_groups), + &op->nbsp->header_, + op->lflow_ref); /* Do not reply to a solicitation from the port that owns * the address (otherwise DAD detection will fail). */ @@ -9691,7 +9067,8 @@ build_lswitch_arp_nd_responder_known_ips(struct ovn_port *op, S_SWITCH_IN_ARP_ND_RSP, 100, ds_cstr(match), "next;", op->key, - &op->nbsp->header_); + &op->nbsp->header_, + op->lflow_ref); } } } @@ -9737,8 +9114,12 @@ build_lswitch_arp_nd_responder_known_ips(struct ovn_port *op, ea_s, ea_s); - ovn_lflow_add_with_hint(lflows, op->od, S_SWITCH_IN_ARP_ND_RSP, - 30, ds_cstr(match), ds_cstr(actions), &op->nbsp->header_); + ovn_lflow_add_with_lflow_ref_hint(lflows, op->od, + S_SWITCH_IN_ARP_ND_RSP, + 30, ds_cstr(match), + ds_cstr(actions), + &op->nbsp->header_, + op->lflow_ref); } /* Add IPv6 NDP responses. @@ -9781,15 +9162,16 @@ build_lswitch_arp_nd_responder_known_ips(struct ovn_port *op, lsp_is_router(op->nbsp) ? "nd_na_router" : "nd_na", ea_s, ea_s); - ovn_lflow_add_with_hint__(lflows, op->od, - S_SWITCH_IN_ARP_ND_RSP, 30, - ds_cstr(match), - ds_cstr(actions), - NULL, - copp_meter_get(COPP_ND_NA, - op->od->nbs->copp, - meter_groups), - &op->nbsp->header_); + ovn_lflow_add_with_lflow_ref_hint__(lflows, op->od, + S_SWITCH_IN_ARP_ND_RSP, 30, + ds_cstr(match), + ds_cstr(actions), + NULL, + copp_meter_get(COPP_ND_NA, + op->od->nbs->copp, + meter_groups), + &op->nbsp->header_, + op->lflow_ref); ds_destroy(&ip6_dst_match); ds_destroy(&nd_target_match); } @@ -9800,7 +9182,7 @@ build_lswitch_arp_nd_responder_known_ips(struct ovn_port *op, * (priority 0)*/ static void build_lswitch_arp_nd_responder_default(struct ovn_datapath *od, - struct hmap *lflows) + struct lflow_table *lflows) { ovs_assert(od->nbs); ovn_lflow_add(lflows, od, S_SWITCH_IN_ARP_ND_RSP, 0, "1", "next;"); @@ -9811,7 +9193,7 @@ build_lswitch_arp_nd_responder_default(struct ovn_datapath *od, static void build_lswitch_arp_nd_service_monitor(const struct ovn_northd_lb *lb, const struct hmap *ls_ports, - struct hmap *lflows, + struct lflow_table *lflows, struct ds *actions, struct ds *match) { @@ -9887,7 +9269,7 @@ build_lswitch_arp_nd_service_monitor(const struct ovn_northd_lb *lb, * priority 100 flows. */ static void build_lswitch_dhcp_options_and_response(struct ovn_port *op, - struct hmap *lflows, + struct lflow_table *lflows, const struct shash *meter_groups) { ovs_assert(op->nbsp); @@ -9942,7 +9324,7 @@ build_lswitch_dhcp_options_and_response(struct ovn_port *op, * (priority 0). */ static void build_lswitch_dhcp_and_dns_defaults(struct ovn_datapath *od, - struct hmap *lflows) + struct lflow_table *lflows) { ovs_assert(od->nbs); ovn_lflow_add(lflows, od, S_SWITCH_IN_DHCP_OPTIONS, 0, "1", "next;"); @@ -9957,7 +9339,7 @@ build_lswitch_dhcp_and_dns_defaults(struct ovn_datapath *od, */ static void build_lswitch_dns_lookup_and_response(struct ovn_datapath *od, - struct hmap *lflows, + struct lflow_table *lflows, const struct shash *meter_groups) { ovs_assert(od->nbs); @@ -9988,7 +9370,7 @@ build_lswitch_dns_lookup_and_response(struct ovn_datapath *od, * binding the external ports. */ static void build_lswitch_external_port(struct ovn_port *op, - struct hmap *lflows) + struct lflow_table *lflows) { ovs_assert(op->nbsp); if (!lsp_is_external(op->nbsp)) { @@ -10004,7 +9386,7 @@ build_lswitch_external_port(struct ovn_port *op, * (priority 70 - 100). */ static void build_lswitch_destination_lookup_bmcast(struct ovn_datapath *od, - struct hmap *lflows, + struct lflow_table *lflows, struct ds *actions, const struct shash *meter_groups) { @@ -10097,7 +9479,7 @@ build_lswitch_destination_lookup_bmcast(struct ovn_datapath *od, * (priority 90). */ static void build_lswitch_ip_mcast_igmp_mld(struct ovn_igmp_group *igmp_group, - struct hmap *lflows, + struct lflow_table *lflows, struct ds *actions, struct ds *match) { @@ -10177,7 +9559,8 @@ build_lswitch_ip_mcast_igmp_mld(struct ovn_igmp_group *igmp_group, /* Ingress table 25: Destination lookup, unicast handling (priority 50), */ static void -build_lswitch_ip_unicast_lookup(struct ovn_port *op, struct hmap *lflows, +build_lswitch_ip_unicast_lookup(struct ovn_port *op, + struct lflow_table *lflows, struct ds *actions, struct ds *match) { ovs_assert(op->nbsp); @@ -10210,10 +9593,12 @@ build_lswitch_ip_unicast_lookup(struct ovn_port *op, struct hmap *lflows, ds_clear(actions); ds_put_format(actions, action, op->json_key); - ovn_lflow_add_with_hint(lflows, op->od, S_SWITCH_IN_L2_LKUP, - 50, ds_cstr(match), - ds_cstr(actions), - &op->nbsp->header_); + ovn_lflow_add_with_lflow_ref_hint(lflows, op->od, + S_SWITCH_IN_L2_LKUP, + 50, ds_cstr(match), + ds_cstr(actions), + &op->nbsp->header_, + op->lflow_ref); } else if (!strcmp(op->nbsp->addresses[i], "unknown")) { continue; } else if (is_dynamic_lsp_address(op->nbsp->addresses[i])) { @@ -10228,10 +9613,12 @@ build_lswitch_ip_unicast_lookup(struct ovn_port *op, struct hmap *lflows, ds_clear(actions); ds_put_format(actions, action, op->json_key); - ovn_lflow_add_with_hint(lflows, op->od, S_SWITCH_IN_L2_LKUP, - 50, ds_cstr(match), - ds_cstr(actions), - &op->nbsp->header_); + ovn_lflow_add_with_lflow_ref_hint(lflows, op->od, + S_SWITCH_IN_L2_LKUP, + 50, ds_cstr(match), + ds_cstr(actions), + &op->nbsp->header_, + op->lflow_ref); } else if (!strcmp(op->nbsp->addresses[i], "router")) { if (!op->peer || !op->peer->nbrp || !ovs_scan(op->peer->nbrp->mac, @@ -10283,10 +9670,11 @@ build_lswitch_ip_unicast_lookup(struct ovn_port *op, struct hmap *lflows, ds_clear(actions); ds_put_format(actions, action, op->json_key); - ovn_lflow_add_with_hint(lflows, op->od, - S_SWITCH_IN_L2_LKUP, 50, - ds_cstr(match), ds_cstr(actions), - &op->nbsp->header_); + ovn_lflow_add_with_lflow_ref_hint(lflows, op->od, + S_SWITCH_IN_L2_LKUP, 50, + ds_cstr(match), ds_cstr(actions), + &op->nbsp->header_, + op->lflow_ref); } else { static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 1); @@ -10301,7 +9689,8 @@ build_lswitch_ip_unicast_lookup(struct ovn_port *op, struct hmap *lflows, static void build_lswitch_ip_unicast_lookup_for_nats( struct ovn_port *op, const struct lr_stateful_record *lr_stateful_rec, - struct hmap *lflows, struct ds *match, struct ds *actions) + struct lflow_table *lflows, struct ds *match, struct ds *actions, + struct lflow_ref *lflow_ref) { ovs_assert(op->nbsp); @@ -10334,11 +9723,12 @@ build_lswitch_ip_unicast_lookup_for_nats( ds_clear(actions); ds_put_format(actions, action, op->json_key); - ovn_lflow_add_with_hint(lflows, op->od, + ovn_lflow_add_with_lflow_ref_hint(lflows, op->od, S_SWITCH_IN_L2_LKUP, 50, ds_cstr(match), ds_cstr(actions), - &op->nbsp->header_); + &op->nbsp->header_, + lflow_ref); } } } @@ -10578,7 +9968,7 @@ get_outport_for_routing_policy_nexthop(struct ovn_datapath *od, } static void -build_routing_policy_flow(struct hmap *lflows, struct ovn_datapath *od, +build_routing_policy_flow(struct lflow_table *lflows, struct ovn_datapath *od, const struct hmap *lr_ports, const struct nbrec_logical_router_policy *rule, const struct ovsdb_idl_row *stage_hint) @@ -10643,7 +10033,8 @@ build_routing_policy_flow(struct hmap *lflows, struct ovn_datapath *od, } static void -build_ecmp_routing_policy_flows(struct hmap *lflows, struct ovn_datapath *od, +build_ecmp_routing_policy_flows(struct lflow_table *lflows, + struct ovn_datapath *od, const struct hmap *lr_ports, const struct nbrec_logical_router_policy *rule, uint16_t ecmp_group_id) @@ -10779,7 +10170,7 @@ get_route_table_id(struct simap *route_tables, const char *route_table_name) } static void -build_route_table_lflow(struct ovn_datapath *od, struct hmap *lflows, +build_route_table_lflow(struct ovn_datapath *od, struct lflow_table *lflows, struct nbrec_logical_router_port *lrp, struct simap *route_tables) { @@ -11190,7 +10581,7 @@ find_static_route_outport(struct ovn_datapath *od, const struct hmap *lr_ports, } static void -add_ecmp_symmetric_reply_flows(struct hmap *lflows, +add_ecmp_symmetric_reply_flows(struct lflow_table *lflows, struct ovn_datapath *od, bool ct_masked_mark, const char *port_ip, @@ -11355,7 +10746,7 @@ add_ecmp_symmetric_reply_flows(struct hmap *lflows, } static void -build_ecmp_route_flow(struct hmap *lflows, struct ovn_datapath *od, +build_ecmp_route_flow(struct lflow_table *lflows, struct ovn_datapath *od, bool ct_masked_mark, const struct hmap *lr_ports, struct ecmp_groups_node *eg) @@ -11442,12 +10833,12 @@ build_ecmp_route_flow(struct hmap *lflows, struct ovn_datapath *od, } static void -add_route(struct hmap *lflows, struct ovn_datapath *od, +add_route(struct lflow_table *lflows, struct ovn_datapath *od, const struct ovn_port *op, const char *lrp_addr_s, const char *network_s, int plen, const char *gateway, bool is_src_route, const uint32_t rtb_id, const struct ovsdb_idl_row *stage_hint, bool is_discard_route, - int ofs) + int ofs, struct lflow_ref *lflow_ref) { bool is_ipv4 = strchr(network_s, '.') ? true : false; struct ds match = DS_EMPTY_INITIALIZER; @@ -11490,14 +10881,17 @@ add_route(struct hmap *lflows, struct ovn_datapath *od, ds_put_format(&actions, "ip.ttl--; %s", ds_cstr(&common_actions)); } - ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_IP_ROUTING, priority, - ds_cstr(&match), ds_cstr(&actions), - stage_hint); + ovn_lflow_add_with_lflow_ref_hint(lflows, od, S_ROUTER_IN_IP_ROUTING, + priority, ds_cstr(&match), + ds_cstr(&actions), stage_hint, + lflow_ref); if (op && op->has_bfd) { ds_put_format(&match, " && udp.dst == 3784"); - ovn_lflow_add_with_hint(lflows, op->od, S_ROUTER_IN_IP_ROUTING, - priority + 1, ds_cstr(&match), - ds_cstr(&common_actions), stage_hint); + ovn_lflow_add_with_lflow_ref_hint(lflows, op->od, + S_ROUTER_IN_IP_ROUTING, + priority + 1, ds_cstr(&match), + ds_cstr(&common_actions),\ + stage_hint, lflow_ref); } ds_destroy(&match); ds_destroy(&common_actions); @@ -11505,7 +10899,7 @@ add_route(struct hmap *lflows, struct ovn_datapath *od, } static void -build_static_route_flow(struct hmap *lflows, struct ovn_datapath *od, +build_static_route_flow(struct lflow_table *lflows, struct ovn_datapath *od, const struct hmap *lr_ports, const struct parsed_route *route_) { @@ -11531,7 +10925,7 @@ build_static_route_flow(struct hmap *lflows, struct ovn_datapath *od, add_route(lflows, route_->is_discard_route ? od : out_port->od, out_port, lrp_addr_s, prefix_s, route_->plen, route->nexthop, route_->is_src_route, route_->route_table_id, &route->header_, - route_->is_discard_route, ofs); + route_->is_discard_route, ofs, NULL); free(prefix_s); } @@ -11594,7 +10988,7 @@ struct lrouter_nat_lb_flows_ctx { int prio; - struct hmap *lflows; + struct lflow_table *lflows; const struct shash *meter_groups; }; @@ -11726,7 +11120,7 @@ build_lrouter_nat_flows_for_lb( struct ovn_northd_lb_vip *vips_nb, const struct ovn_datapaths *lr_datapaths, const struct lr_stateful_table *lr_stateful_table, - struct hmap *lflows, + struct lflow_table *lflows, struct ds *match, struct ds *action, const struct shash *meter_groups, const struct chassis_features *features, @@ -11895,7 +11289,7 @@ build_lrouter_nat_flows_for_lb( static void build_lswitch_flows_for_lb(struct ovn_lb_datapaths *lb_dps, - struct hmap *lflows, + struct lflow_table *lflows, const struct shash *meter_groups, const struct ovn_datapaths *ls_datapaths, const struct chassis_features *features, @@ -11956,7 +11350,7 @@ build_lswitch_flows_for_lb(struct ovn_lb_datapaths *lb_dps, */ static void build_lrouter_defrag_flows_for_lb(struct ovn_lb_datapaths *lb_dps, - struct hmap *lflows, + struct lflow_table *lflows, const struct ovn_datapaths *lr_datapaths, struct ds *match) { @@ -11982,7 +11376,7 @@ build_lrouter_defrag_flows_for_lb(struct ovn_lb_datapaths *lb_dps, static void build_lrouter_flows_for_lb(struct ovn_lb_datapaths *lb_dps, - struct hmap *lflows, + struct lflow_table *lflows, const struct shash *meter_groups, const struct ovn_datapaths *lr_datapaths, const struct lr_stateful_table *lr_stateful_table, @@ -12140,7 +11534,7 @@ lrouter_dnat_and_snat_is_stateless(const struct nbrec_nat *nat) */ static inline void lrouter_nat_add_ext_ip_match(const struct ovn_datapath *od, - struct hmap *lflows, struct ds *match, + struct lflow_table *lflows, struct ds *match, const struct nbrec_nat *nat, bool is_v6, bool is_src, int cidr_bits) { @@ -12207,7 +11601,7 @@ build_lrouter_arp_flow(const struct ovn_datapath *od, struct ovn_port *op, const char *ip_address, const char *eth_addr, struct ds *extra_match, bool drop, uint16_t priority, const struct ovsdb_idl_row *hint, - struct hmap *lflows) + struct lflow_table *lflows) { struct ds match = DS_EMPTY_INITIALIZER; struct ds actions = DS_EMPTY_INITIALIZER; @@ -12257,7 +11651,8 @@ build_lrouter_nd_flow(const struct ovn_datapath *od, struct ovn_port *op, const char *sn_ip_address, const char *eth_addr, struct ds *extra_match, bool drop, uint16_t priority, const struct ovsdb_idl_row *hint, - struct hmap *lflows, const struct shash *meter_groups) + struct lflow_table *lflows, + const struct shash *meter_groups) { struct ds match = DS_EMPTY_INITIALIZER; struct ds actions = DS_EMPTY_INITIALIZER; @@ -12308,7 +11703,7 @@ build_lrouter_nd_flow(const struct ovn_datapath *od, struct ovn_port *op, static void build_lrouter_nat_arp_nd_flow(const struct ovn_datapath *od, struct ovn_nat *nat_entry, - struct hmap *lflows, + struct lflow_table *lflows, const struct shash *meter_groups) { struct lport_addresses *ext_addrs = &nat_entry->ext_addrs; @@ -12331,7 +11726,7 @@ build_lrouter_nat_arp_nd_flow(const struct ovn_datapath *od, static void build_lrouter_port_nat_arp_nd_flow(struct ovn_port *op, struct ovn_nat *nat_entry, - struct hmap *lflows, + struct lflow_table *lflows, const struct shash *meter_groups) { struct lport_addresses *ext_addrs = &nat_entry->ext_addrs; @@ -12405,7 +11800,7 @@ build_lrouter_drop_own_dest(struct ovn_port *op, const struct lr_stateful_record *lr_stateful_rec, enum ovn_stage stage, uint16_t priority, bool drop_snat_ip, - struct hmap *lflows) + struct lflow_table *lflows) { struct ds match_ips = DS_EMPTY_INITIALIZER; @@ -12470,7 +11865,7 @@ build_lrouter_drop_own_dest(struct ovn_port *op, } static void -build_lrouter_force_snat_flows(struct hmap *lflows, +build_lrouter_force_snat_flows(struct lflow_table *lflows, const struct ovn_datapath *od, const char *ip_version, const char *ip_addr, const char *context) @@ -12499,7 +11894,7 @@ build_lrouter_force_snat_flows(struct hmap *lflows, static void build_lrouter_force_snat_flows_op(struct ovn_port *op, const struct lr_nat_record *lrnat_rec, - struct hmap *lflows, + struct lflow_table *lflows, struct ds *match, struct ds *actions) { ovs_assert(op->nbrp); @@ -12571,7 +11966,7 @@ build_lrouter_force_snat_flows_op(struct ovn_port *op, } static void -build_lrouter_bfd_flows(struct hmap *lflows, struct ovn_port *op, +build_lrouter_bfd_flows(struct lflow_table *lflows, struct ovn_port *op, const struct shash *meter_groups) { if (!op->has_bfd) { @@ -12626,7 +12021,7 @@ build_lrouter_bfd_flows(struct hmap *lflows, struct ovn_port *op, */ static void build_adm_ctrl_flows_for_lrouter( - struct ovn_datapath *od, struct hmap *lflows) + struct ovn_datapath *od, struct lflow_table *lflows) { ovs_assert(od->nbr); /* Logical VLANs not supported. @@ -12670,7 +12065,7 @@ build_gateway_get_l2_hdr_size(struct ovn_port *op) * function. */ static void OVS_PRINTF_FORMAT(9, 10) -build_gateway_mtu_flow(struct hmap *lflows, struct ovn_port *op, +build_gateway_mtu_flow(struct lflow_table *lflows, struct ovn_port *op, enum ovn_stage stage, uint16_t prio_low, uint16_t prio_high, struct ds *match, struct ds *actions, const struct ovsdb_idl_row *hint, @@ -12731,7 +12126,7 @@ consider_l3dgw_port_is_centralized(struct ovn_port *op) */ static void build_adm_ctrl_flows_for_lrouter_port( - struct ovn_port *op, struct hmap *lflows, + struct ovn_port *op, struct lflow_table *lflows, struct ds *match, struct ds *actions) { ovs_assert(op->nbrp); @@ -12785,7 +12180,7 @@ build_adm_ctrl_flows_for_lrouter_port( * lflows for logical routers. */ static void build_neigh_learning_flows_for_lrouter( - struct ovn_datapath *od, struct hmap *lflows, + struct ovn_datapath *od, struct lflow_table *lflows, struct ds *match, struct ds *actions, const struct shash *meter_groups) { @@ -12916,7 +12311,7 @@ build_neigh_learning_flows_for_lrouter( * for logical router ports. */ static void build_neigh_learning_flows_for_lrouter_port( - struct ovn_port *op, struct hmap *lflows, + struct ovn_port *op, struct lflow_table *lflows, struct ds *match, struct ds *actions) { ovs_assert(op->nbrp); @@ -12978,7 +12373,7 @@ build_neigh_learning_flows_for_lrouter_port( * Adv (RA) options and response. */ static void build_ND_RA_flows_for_lrouter_port( - struct ovn_port *op, struct hmap *lflows, + struct ovn_port *op, struct lflow_table *lflows, struct ds *match, struct ds *actions, const struct shash *meter_groups) { @@ -13093,7 +12488,8 @@ build_ND_RA_flows_for_lrouter_port( /* Logical router ingress table ND_RA_OPTIONS & ND_RA_RESPONSE: RS * responder, by default goto next. (priority 0). */ static void -build_ND_RA_flows_for_lrouter(struct ovn_datapath *od, struct hmap *lflows) +build_ND_RA_flows_for_lrouter(struct ovn_datapath *od, + struct lflow_table *lflows) { ovs_assert(od->nbr); ovn_lflow_add(lflows, od, S_ROUTER_IN_ND_RA_OPTIONS, 0, "1", "next;"); @@ -13104,7 +12500,7 @@ build_ND_RA_flows_for_lrouter(struct ovn_datapath *od, struct hmap *lflows) * by default goto next. (priority 0). */ static void build_ip_routing_pre_flows_for_lrouter(struct ovn_datapath *od, - struct hmap *lflows) + struct lflow_table *lflows) { ovs_assert(od->nbr); ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_ROUTING_PRE, 0, "1", @@ -13132,21 +12528,23 @@ build_ip_routing_pre_flows_for_lrouter(struct ovn_datapath *od, */ static void build_ip_routing_flows_for_lrp( - struct ovn_port *op, struct hmap *lflows) + struct ovn_port *op, struct lflow_table *lflows) { ovs_assert(op->nbrp); for (int i = 0; i < op->lrp_networks.n_ipv4_addrs; i++) { add_route(lflows, op->od, op, op->lrp_networks.ipv4_addrs[i].addr_s, op->lrp_networks.ipv4_addrs[i].network_s, op->lrp_networks.ipv4_addrs[i].plen, NULL, false, 0, - &op->nbrp->header_, false, ROUTE_PRIO_OFFSET_CONNECTED); + &op->nbrp->header_, false, ROUTE_PRIO_OFFSET_CONNECTED, + NULL); } for (int i = 0; i < op->lrp_networks.n_ipv6_addrs; i++) { add_route(lflows, op->od, op, op->lrp_networks.ipv6_addrs[i].addr_s, op->lrp_networks.ipv6_addrs[i].network_s, op->lrp_networks.ipv6_addrs[i].plen, NULL, false, 0, - &op->nbrp->header_, false, ROUTE_PRIO_OFFSET_CONNECTED); + &op->nbrp->header_, false, ROUTE_PRIO_OFFSET_CONNECTED, + NULL); } } @@ -13159,8 +12557,9 @@ build_ip_routing_flows_for_lrp( */ static void build_ip_routing_flows_for_router_type_lsp( - struct ovn_port *op, const struct lr_stateful_table *lr_stateful_table, - const struct hmap *lr_ports, struct hmap *lflows) + struct ovn_port *op, const struct lr_stateful_table *lr_stateful_table, + const struct hmap *lr_ports, struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { ovs_assert(op->nbsp); if (!lsp_is_router(op->nbsp)) { @@ -13196,7 +12595,8 @@ build_ip_routing_flows_for_router_type_lsp( laddrs->ipv4_addrs[k].network_s, laddrs->ipv4_addrs[k].plen, NULL, false, 0, &peer->nbrp->header_, false, - ROUTE_PRIO_OFFSET_CONNECTED); + ROUTE_PRIO_OFFSET_CONNECTED, + lflow_ref); } } destroy_routable_addresses(&ra); @@ -13208,7 +12608,7 @@ build_ip_routing_flows_for_router_type_lsp( static void build_static_route_flows_for_lrouter( struct ovn_datapath *od, const struct chassis_features *features, - struct hmap *lflows, const struct hmap *lr_ports, + struct lflow_table *lflows, const struct hmap *lr_ports, const struct hmap *bfd_connections) { ovs_assert(od->nbr); @@ -13272,7 +12672,7 @@ build_static_route_flows_for_lrouter( */ static void build_mcast_lookup_flows_for_lrouter( - struct ovn_datapath *od, struct hmap *lflows, + struct ovn_datapath *od, struct lflow_table *lflows, struct ds *match, struct ds *actions) { ovs_assert(od->nbr); @@ -13373,7 +12773,7 @@ build_mcast_lookup_flows_for_lrouter( * advances to the next table for ARP/ND resolution. */ static void build_ingress_policy_flows_for_lrouter( - struct ovn_datapath *od, struct hmap *lflows, + struct ovn_datapath *od, struct lflow_table *lflows, const struct hmap *lr_ports) { ovs_assert(od->nbr); @@ -13407,7 +12807,7 @@ build_ingress_policy_flows_for_lrouter( /* Local router ingress table ARP_RESOLVE: ARP Resolution. */ static void build_arp_resolve_flows_for_lrouter( - struct ovn_datapath *od, struct hmap *lflows) + struct ovn_datapath *od, struct lflow_table *lflows) { ovs_assert(od->nbr); /* Multicast packets already have the outport set so just advance to @@ -13425,10 +12825,12 @@ build_arp_resolve_flows_for_lrouter( } static void -routable_addresses_to_lflows(struct hmap *lflows, struct ovn_port *router_port, +routable_addresses_to_lflows(struct lflow_table *lflows, + struct ovn_port *router_port, struct ovn_port *peer, const struct lr_stateful_record *lr_stateful_rec, - struct ds *match, struct ds *actions) + struct ds *match, struct ds *actions, + struct lflow_ref *lflow_ref) { struct ovn_port_routable_addresses ra = get_op_routable_addresses(router_port, lr_stateful_rec); @@ -13452,8 +12854,9 @@ routable_addresses_to_lflows(struct hmap *lflows, struct ovn_port *router_port, ds_clear(actions); ds_put_format(actions, "eth.dst = %s; next;", ra.laddrs[i].ea_s); - ovn_lflow_add(lflows, peer->od, S_ROUTER_IN_ARP_RESOLVE, 100, - ds_cstr(match), ds_cstr(actions)); + ovn_lflow_add_with_lflow_ref(lflows, peer->od, S_ROUTER_IN_ARP_RESOLVE, + 100, ds_cstr(match), ds_cstr(actions), + lflow_ref); } destroy_routable_addresses(&ra); } @@ -13470,7 +12873,8 @@ routable_addresses_to_lflows(struct hmap *lflows, struct ovn_port *router_port, /* This function adds ARP resolve flows related to a LRP. */ static void -build_arp_resolve_flows_for_lrp(struct ovn_port *op, struct hmap *lflows, +build_arp_resolve_flows_for_lrp(struct ovn_port *op, + struct lflow_table *lflows, struct ds *match, struct ds *actions) { ovs_assert(op->nbrp); @@ -13545,7 +12949,7 @@ build_arp_resolve_flows_for_lrp(struct ovn_port *op, struct hmap *lflows, /* This function adds ARP resolve flows related to a LSP. */ static void build_arp_resolve_flows_for_lsp( - struct ovn_port *op, struct hmap *lflows, + struct ovn_port *op, struct lflow_table *lflows, const struct hmap *lr_ports, struct ds *match, struct ds *actions) { @@ -13587,11 +12991,12 @@ build_arp_resolve_flows_for_lsp( ds_clear(actions); ds_put_format(actions, "eth.dst = %s; next;", ea_s); - ovn_lflow_add_with_hint(lflows, peer->od, + ovn_lflow_add_with_lflow_ref_hint(lflows, peer->od, S_ROUTER_IN_ARP_RESOLVE, 100, ds_cstr(match), ds_cstr(actions), - &op->nbsp->header_); + &op->nbsp->header_, + op->lflow_ref); } } @@ -13618,11 +13023,12 @@ build_arp_resolve_flows_for_lsp( ds_clear(actions); ds_put_format(actions, "eth.dst = %s; next;", ea_s); - ovn_lflow_add_with_hint(lflows, peer->od, + ovn_lflow_add_with_lflow_ref_hint(lflows, peer->od, S_ROUTER_IN_ARP_RESOLVE, 100, ds_cstr(match), ds_cstr(actions), - &op->nbsp->header_); + &op->nbsp->header_, + op->lflow_ref); } } } @@ -13666,10 +13072,11 @@ build_arp_resolve_flows_for_lsp( ds_clear(actions); ds_put_format(actions, "eth.dst = %s; next;", router_port->lrp_networks.ea_s); - ovn_lflow_add_with_hint(lflows, peer->od, + ovn_lflow_add_with_lflow_ref_hint(lflows, peer->od, S_ROUTER_IN_ARP_RESOLVE, 100, ds_cstr(match), ds_cstr(actions), - &op->nbsp->header_); + &op->nbsp->header_, + op->lflow_ref); } if (router_port->lrp_networks.n_ipv6_addrs) { @@ -13682,10 +13089,11 @@ build_arp_resolve_flows_for_lsp( ds_clear(actions); ds_put_format(actions, "eth.dst = %s; next;", router_port->lrp_networks.ea_s); - ovn_lflow_add_with_hint(lflows, peer->od, + ovn_lflow_add_with_lflow_ref_hint(lflows, peer->od, S_ROUTER_IN_ARP_RESOLVE, 100, ds_cstr(match), ds_cstr(actions), - &op->nbsp->header_); + &op->nbsp->header_, + op->lflow_ref); } } } @@ -13693,10 +13101,11 @@ build_arp_resolve_flows_for_lsp( static void build_arp_resolve_flows_for_lsp_routable_addresses( - struct ovn_port *op, struct hmap *lflows, - const struct hmap *lr_ports, - const struct lr_stateful_table *lr_stateful_table, - struct ds *match, struct ds *actions) + struct ovn_port *op, struct lflow_table *lflows, + const struct hmap *lr_ports, + const struct lr_stateful_table *lr_stateful_table, + struct ds *match, struct ds *actions, + struct lflow_ref *lflow_ref) { if (!lsp_is_router(op->nbsp)) { return; @@ -13730,13 +13139,15 @@ build_arp_resolve_flows_for_lsp_routable_addresses( lr_stateful_rec = lr_stateful_table_find_by_index( lr_stateful_table, router_port->od->index); routable_addresses_to_lflows(lflows, router_port, peer, - lr_stateful_rec, match, actions); + lr_stateful_rec, match, actions, + lflow_ref); } } } static void -build_icmperr_pkt_big_flows(struct ovn_port *op, int mtu, struct hmap *lflows, +build_icmperr_pkt_big_flows(struct ovn_port *op, int mtu, + struct lflow_table *lflows, const struct shash *meter_groups, struct ds *match, struct ds *actions, enum ovn_stage stage, struct ovn_port *outport) @@ -13829,7 +13240,7 @@ build_icmperr_pkt_big_flows(struct ovn_port *op, int mtu, struct hmap *lflows, static void build_check_pkt_len_flows_for_lrp(struct ovn_port *op, - struct hmap *lflows, + struct lflow_table *lflows, const struct hmap *lr_ports, const struct shash *meter_groups, struct ds *match, @@ -13879,7 +13290,7 @@ build_check_pkt_len_flows_for_lrp(struct ovn_port *op, * */ static void build_check_pkt_len_flows_for_lrouter( - struct ovn_datapath *od, struct hmap *lflows, + struct ovn_datapath *od, struct lflow_table *lflows, const struct hmap *lr_ports, struct ds *match, struct ds *actions, const struct shash *meter_groups) @@ -13906,7 +13317,7 @@ build_check_pkt_len_flows_for_lrouter( /* Logical router ingress table GW_REDIRECT: Gateway redirect. */ static void build_gateway_redirect_flows_for_lrouter( - struct ovn_datapath *od, struct hmap *lflows, + struct ovn_datapath *od, struct lflow_table *lflows, struct ds *match, struct ds *actions) { ovs_assert(od->nbr); @@ -13950,8 +13361,8 @@ build_gateway_redirect_flows_for_lrouter( /* Logical router ingress table GW_REDIRECT: Gateway redirect. */ static void build_lr_gateway_redirect_flows_for_nats( - const struct ovn_datapath *od, const struct lr_nat_record *lrnat_rec, - struct hmap *lflows, struct ds *match, struct ds *actions) + const struct ovn_datapath *od, const struct lr_nat_record *lrnat_rec, + struct lflow_table *lflows, struct ds *match, struct ds *actions) { ovs_assert(od->nbr); for (size_t i = 0; i < od->n_l3dgw_ports; i++) { @@ -14020,7 +13431,7 @@ build_lr_gateway_redirect_flows_for_nats( * and sends an ARP/IPv6 NA request (priority 100). */ static void build_arp_request_flows_for_lrouter( - struct ovn_datapath *od, struct hmap *lflows, + struct ovn_datapath *od, struct lflow_table *lflows, struct ds *match, struct ds *actions, const struct shash *meter_groups) { @@ -14098,7 +13509,7 @@ build_arp_request_flows_for_lrouter( */ static void build_egress_delivery_flows_for_lrouter_port( - struct ovn_port *op, struct hmap *lflows, + struct ovn_port *op, struct lflow_table *lflows, struct ds *match, struct ds *actions) { ovs_assert(op->nbrp); @@ -14140,7 +13551,7 @@ build_egress_delivery_flows_for_lrouter_port( static void build_misc_local_traffic_drop_flows_for_lrouter( - struct ovn_datapath *od, struct hmap *lflows) + struct ovn_datapath *od, struct lflow_table *lflows) { ovs_assert(od->nbr); /* Allow IGMP and MLD packets (with TTL = 1) if the router is @@ -14222,7 +13633,7 @@ build_misc_local_traffic_drop_flows_for_lrouter( static void build_dhcpv6_reply_flows_for_lrouter_port( - struct ovn_port *op, struct hmap *lflows, + struct ovn_port *op, struct lflow_table *lflows, struct ds *match) { ovs_assert(op->nbrp); @@ -14242,7 +13653,7 @@ build_dhcpv6_reply_flows_for_lrouter_port( static void build_ipv6_input_flows_for_lrouter_port( - struct ovn_port *op, struct hmap *lflows, + struct ovn_port *op, struct lflow_table *lflows, struct ds *match, struct ds *actions, const struct shash *meter_groups) { @@ -14411,7 +13822,7 @@ build_ipv6_input_flows_for_lrouter_port( static void build_lrouter_arp_nd_for_datapath(const struct ovn_datapath *od, const struct lr_nat_record *lrnat_rec, - struct hmap *lflows, + struct lflow_table *lflows, const struct shash *meter_groups) { ovs_assert(od->nbr); @@ -14463,7 +13874,7 @@ build_lrouter_arp_nd_for_datapath(const struct ovn_datapath *od, /* Logical router ingress table 3: IP Input for IPv4. */ static void build_lrouter_ipv4_ip_input(struct ovn_port *op, - struct hmap *lflows, + struct lflow_table *lflows, struct ds *match, struct ds *actions, const struct shash *meter_groups) { @@ -14667,7 +14078,7 @@ build_lrouter_ipv4_ip_input(struct ovn_port *op, /* Logical router ingress table 3: IP Input for IPv4. */ static void build_lrouter_ipv4_ip_input_for_lbnats( - struct ovn_port *op, struct hmap *lflows, + struct ovn_port *op, struct lflow_table *lflows, const struct lr_stateful_record *lr_stateful_rec, struct ds *match, const struct shash *meter_groups) { @@ -14787,7 +14198,7 @@ build_lrouter_in_unsnat_match(const struct ovn_datapath *od, } static void -build_lrouter_in_unsnat_stateless_flow(struct hmap *lflows, +build_lrouter_in_unsnat_stateless_flow(struct lflow_table *lflows, const struct ovn_datapath *od, const struct nbrec_nat *nat, struct ds *match, @@ -14809,7 +14220,7 @@ build_lrouter_in_unsnat_stateless_flow(struct hmap *lflows, } static void -build_lrouter_in_unsnat_in_czone_flow(struct hmap *lflows, +build_lrouter_in_unsnat_in_czone_flow(struct lflow_table *lflows, const struct ovn_datapath *od, const struct nbrec_nat *nat, struct ds *match, bool distributed_nat, @@ -14843,7 +14254,7 @@ build_lrouter_in_unsnat_in_czone_flow(struct hmap *lflows, } static void -build_lrouter_in_unsnat_flow(struct hmap *lflows, +build_lrouter_in_unsnat_flow(struct lflow_table *lflows, const struct ovn_datapath *od, const struct nbrec_nat *nat, struct ds *match, bool distributed_nat, bool is_v6, @@ -14865,7 +14276,7 @@ build_lrouter_in_unsnat_flow(struct hmap *lflows, } static void -build_lrouter_in_dnat_flow(struct hmap *lflows, +build_lrouter_in_dnat_flow(struct lflow_table *lflows, const struct ovn_datapath *od, const struct lr_nat_record *lrnat_rec, const struct nbrec_nat *nat, struct ds *match, @@ -14937,7 +14348,7 @@ build_lrouter_in_dnat_flow(struct hmap *lflows, } static void -build_lrouter_out_undnat_flow(struct hmap *lflows, +build_lrouter_out_undnat_flow(struct lflow_table *lflows, const struct ovn_datapath *od, const struct nbrec_nat *nat, struct ds *match, struct ds *actions, bool distributed_nat, @@ -14988,7 +14399,7 @@ build_lrouter_out_undnat_flow(struct hmap *lflows, } static void -build_lrouter_out_is_dnat_local(struct hmap *lflows, +build_lrouter_out_is_dnat_local(struct lflow_table *lflows, const struct ovn_datapath *od, const struct nbrec_nat *nat, struct ds *match, struct ds *actions, bool distributed_nat, @@ -15019,7 +14430,7 @@ build_lrouter_out_is_dnat_local(struct hmap *lflows, } static void -build_lrouter_out_snat_match(struct hmap *lflows, +build_lrouter_out_snat_match(struct lflow_table *lflows, const struct ovn_datapath *od, const struct nbrec_nat *nat, struct ds *match, bool distributed_nat, int cidr_bits, bool is_v6, @@ -15048,7 +14459,7 @@ build_lrouter_out_snat_match(struct hmap *lflows, } static void -build_lrouter_out_snat_stateless_flow(struct hmap *lflows, +build_lrouter_out_snat_stateless_flow(struct lflow_table *lflows, const struct ovn_datapath *od, const struct nbrec_nat *nat, struct ds *match, struct ds *actions, @@ -15091,7 +14502,7 @@ build_lrouter_out_snat_stateless_flow(struct hmap *lflows, } static void -build_lrouter_out_snat_in_czone_flow(struct hmap *lflows, +build_lrouter_out_snat_in_czone_flow(struct lflow_table *lflows, const struct ovn_datapath *od, const struct nbrec_nat *nat, struct ds *match, @@ -15153,7 +14564,7 @@ build_lrouter_out_snat_in_czone_flow(struct hmap *lflows, } static void -build_lrouter_out_snat_flow(struct hmap *lflows, +build_lrouter_out_snat_flow(struct lflow_table *lflows, const struct ovn_datapath *od, const struct nbrec_nat *nat, struct ds *match, struct ds *actions, bool distributed_nat, @@ -15199,7 +14610,7 @@ build_lrouter_out_snat_flow(struct hmap *lflows, } static void -build_lrouter_ingress_nat_check_pkt_len(struct hmap *lflows, +build_lrouter_ingress_nat_check_pkt_len(struct lflow_table *lflows, const struct nbrec_nat *nat, const struct ovn_datapath *od, bool is_v6, struct ds *match, @@ -15271,7 +14682,7 @@ build_lrouter_ingress_nat_check_pkt_len(struct hmap *lflows, } static void -build_lrouter_ingress_flow(struct hmap *lflows, +build_lrouter_ingress_flow(struct lflow_table *lflows, const struct ovn_datapath *od, const struct nbrec_nat *nat, struct ds *match, struct ds *actions, struct eth_addr mac, @@ -15451,7 +14862,7 @@ lrouter_check_nat_entry(const struct ovn_datapath *od, /* NAT, Defrag and load balancing. */ static void build_lr_nat_defrag_and_lb_default_flows(struct ovn_datapath *od, - struct hmap *lflows) + struct lflow_table *lflows) { ovs_assert(od->nbr); @@ -15476,7 +14887,8 @@ static void build_lr_nat_defrag_and_lb_default_flows(struct ovn_datapath *od, static void build_lrouter_nat_defrag_and_lb( - const struct lr_stateful_record *lr_stateful_rec, struct hmap *lflows, + const struct lr_stateful_record *lr_stateful_rec, + struct lflow_table *lflows, const struct hmap *ls_ports, const struct hmap *lr_ports, struct ds *match, struct ds *actions, const struct shash *meter_groups, @@ -15858,31 +15270,30 @@ build_lsp_lflows_for_lbnats(struct ovn_port *lsp, const struct lr_stateful_record *lr_stateful_rec, const struct lr_stateful_table *lr_stateful_table, const struct hmap *lr_ports, - struct hmap *lflows, + struct lflow_table *lflows, struct ds *match, - struct ds *actions) + struct ds *actions, + struct lflow_ref *lflow_ref) { ovs_assert(lsp->nbsp); ovs_assert(lsp->peer); - start_collecting_lflows(); build_lswitch_rport_arp_req_flows_for_lbnats( lsp->peer, lr_stateful_rec, lsp->od, lsp, - lflows, &lsp->nbsp->header_); + lflows, &lsp->nbsp->header_, lflow_ref); build_ip_routing_flows_for_router_type_lsp(lsp, lr_stateful_table, - lr_ports, lflows); + lr_ports, lflows, + lflow_ref); build_arp_resolve_flows_for_lsp_routable_addresses( - lsp, lflows, lr_ports, lr_stateful_table, match, actions); + lsp, lflows, lr_ports, lr_stateful_table, match, actions, lflow_ref); build_lswitch_ip_unicast_lookup_for_nats(lsp, lr_stateful_rec, lflows, - match, actions); - link_ovn_port_to_lflows(lsp, &collected_lflows); - end_collecting_lflows(); + match, actions, lflow_ref); } static void build_lbnat_lflows_iterate_by_lsp( struct ovn_port *op, const struct lr_stateful_table *lr_stateful_table, const struct hmap *lr_ports, struct ds *match, struct ds *actions, - struct hmap *lflows) + struct lflow_table *lflows) { ovs_assert(op->nbsp); @@ -15895,8 +15306,9 @@ build_lbnat_lflows_iterate_by_lsp( op->peer->od->index); ovs_assert(lr_stateful_rec); - build_lsp_lflows_for_lbnats(op, lr_stateful_rec, lr_stateful_table, - lr_ports, lflows, match, actions); + build_lsp_lflows_for_lbnats(op, lr_stateful_rec, + lr_stateful_table, lr_ports, lflows, + match, actions, op->stateful_lflow_ref); } static void @@ -15904,7 +15316,7 @@ build_lrp_lflows_for_lbnats(struct ovn_port *op, const struct lr_stateful_record *lr_stateful_rec, const struct shash *meter_groups, struct ds *match, struct ds *actions, - struct hmap *lflows) + struct lflow_table *lflows) { /* Drop IP traffic destined to router owned IPs except if the IP is * also a SNAT IP. Those are dropped later, in stage @@ -15939,7 +15351,7 @@ static void build_lbnat_lflows_iterate_by_lrp( struct ovn_port *op, const struct lr_stateful_table *lr_stateful_table, const struct shash *meter_groups, struct ds *match, - struct ds *actions, struct hmap *lflows) + struct ds *actions, struct lflow_table *lflows) { ovs_assert(op->nbrp); @@ -15954,7 +15366,7 @@ build_lbnat_lflows_iterate_by_lrp( static void build_lr_stateful_flows(const struct lr_stateful_record *lr_stateful_rec, - struct hmap *lflows, + struct lflow_table *lflows, const struct hmap *ls_ports, const struct hmap *lr_ports, struct ds *match, @@ -15978,7 +15390,7 @@ build_ls_stateful_flows(const struct ls_stateful_record *ls_stateful_rec, const struct ls_port_group_table *ls_pgs, const struct chassis_features *features, const struct shash *meter_groups, - struct hmap *lflows) + struct lflow_table *lflows) { ovs_assert(ls_stateful_rec->od); @@ -15997,7 +15409,7 @@ struct lswitch_flow_build_info { const struct ls_port_group_table *ls_port_groups; const struct lr_stateful_table *lr_stateful_table; const struct ls_stateful_table *ls_stateful_table; - struct hmap *lflows; + struct lflow_table *lflows; struct hmap *igmp_groups; const struct shash *meter_groups; const struct hmap *lb_dps_map; @@ -16080,10 +15492,9 @@ build_lswitch_and_lrouter_iterate_by_lsp(struct ovn_port *op, const struct shash *meter_groups, struct ds *match, struct ds *actions, - struct hmap *lflows) + struct lflow_table *lflows) { ovs_assert(op->nbsp); - start_collecting_lflows(); /* Build Logical Switch Flows. */ build_lswitch_port_sec_op(op, lflows, actions, match); @@ -16098,9 +15509,6 @@ build_lswitch_and_lrouter_iterate_by_lsp(struct ovn_port *op, /* Build Logical Router Flows. */ build_arp_resolve_flows_for_lsp(op, lflows, lr_ports, match, actions); - - link_ovn_port_to_lflows(op, &collected_lflows); - end_collecting_lflows(); } /* Helper function to combine all lflow generation which is iterated by logical @@ -16307,7 +15715,7 @@ noop_callback(struct worker_pool *pool OVS_UNUSED, /* Do nothing */ } -/* Fixes the hmap size (hmap->n) after parallel building the lflow_map when +/* Fixes the hmap size (hmap->n) after parallel building the lflow_table when * dp-groups is enabled, because in that case all threads are updating the * global lflow hmap. Although the lflow_hash_lock prevents currently inserting * to the same hash bucket, the hmap->n is updated currently by all threads and @@ -16317,7 +15725,7 @@ noop_callback(struct worker_pool *pool OVS_UNUSED, * after the worker threads complete the tasks in each iteration before any * future operations on the lflow map. */ static void -fix_flow_map_size(struct hmap *lflow_map, +fix_flow_table_size(struct lflow_table *lflow_table, struct lswitch_flow_build_info *lsiv, size_t n_lsiv) { @@ -16325,7 +15733,7 @@ fix_flow_map_size(struct hmap *lflow_map, for (size_t i = 0; i < n_lsiv; i++) { total += lsiv[i].thread_lflow_counter; } - lflow_map->n = total; + lflow_table_set_size(lflow_table, total); } static void @@ -16337,7 +15745,7 @@ build_lswitch_and_lrouter_flows( const struct ls_port_group_table *ls_pgs, const struct lr_stateful_table *lr_stateful_table, const struct ls_stateful_table *ls_stateful_table, - struct hmap *lflows, + struct lflow_table *lflows, struct hmap *igmp_groups, const struct shash *meter_groups, const struct hmap *lb_dps_map, @@ -16384,7 +15792,7 @@ build_lswitch_and_lrouter_flows( /* Run thread pool. */ run_pool_callback(build_lflows_pool, NULL, NULL, noop_callback); - fix_flow_map_size(lflows, lsiv, build_lflows_pool->size); + fix_flow_table_size(lflows, lsiv, build_lflows_pool->size); for (index = 0; index < build_lflows_pool->size; index++) { ds_destroy(&lsiv[index].match); @@ -16498,24 +15906,6 @@ build_lswitch_and_lrouter_flows( free(svc_check_match); } -static ssize_t max_seen_lflow_size = 128; - -void -lflow_data_init(struct lflow_data *data) -{ - fast_hmap_size_for(&data->lflows, max_seen_lflow_size); -} - -void -lflow_data_destroy(struct lflow_data *data) -{ - struct ovn_lflow *lflow; - HMAP_FOR_EACH_SAFE (lflow, hmap_node, &data->lflows) { - ovn_lflow_destroy(&data->lflows, lflow); - } - hmap_destroy(&data->lflows); -} - void run_update_worker_pool(int n_threads) { /* If number of threads has been updated (or initially set), @@ -16561,7 +15951,7 @@ create_sb_multicast_group(struct ovsdb_idl_txn *ovnsb_txn, * constructing their contents based on the OVN_NB database. */ void build_lflows(struct ovsdb_idl_txn *ovnsb_txn, struct lflow_input *input_data, - struct hmap *lflows) + struct lflow_table *lflows) { struct hmap mcast_groups; struct hmap igmp_groups; @@ -16592,281 +15982,26 @@ void build_lflows(struct ovsdb_idl_txn *ovnsb_txn, } /* Parallel build may result in a suboptimal hash. Resize the - * hash to a correct size before doing lookups */ - - hmap_expand(lflows); - - if (hmap_count(lflows) > max_seen_lflow_size) { - max_seen_lflow_size = hmap_count(lflows); - } - - stopwatch_start(LFLOWS_DP_GROUPS_STOPWATCH_NAME, time_msec()); - /* Collecting all unique datapath groups. */ - struct hmap ls_dp_groups = HMAP_INITIALIZER(&ls_dp_groups); - struct hmap lr_dp_groups = HMAP_INITIALIZER(&lr_dp_groups); - struct hmap single_dp_lflows; - - /* Single dp_flows will never grow bigger than lflows, - * thus the two hmaps will remain the same size regardless - * of how many elements we remove from lflows and add to - * single_dp_lflows. - * Note - lflows is always sized for at least 128 flows. - */ - fast_hmap_size_for(&single_dp_lflows, max_seen_lflow_size); - - struct ovn_lflow *lflow; - HMAP_FOR_EACH_SAFE (lflow, hmap_node, lflows) { - struct ovn_datapath **datapaths_array; - size_t n_datapaths; - - if (ovn_stage_to_datapath_type(lflow->stage) == DP_SWITCH) { - n_datapaths = ods_size(input_data->ls_datapaths); - datapaths_array = input_data->ls_datapaths->array; - } else { - n_datapaths = ods_size(input_data->lr_datapaths); - datapaths_array = input_data->lr_datapaths->array; - } - - lflow->n_ods = bitmap_count1(lflow->dpg_bitmap, n_datapaths); - - ovs_assert(lflow->n_ods); - - if (lflow->n_ods == 1) { - /* There is only one datapath, so it should be moved out of the - * group to a single 'od'. */ - size_t index = bitmap_scan(lflow->dpg_bitmap, true, 0, - n_datapaths); - - bitmap_set0(lflow->dpg_bitmap, index); - lflow->od = datapaths_array[index]; - - /* Logical flow should be re-hashed to allow lookups. */ - uint32_t hash = hmap_node_hash(&lflow->hmap_node); - /* Remove from lflows. */ - hmap_remove(lflows, &lflow->hmap_node); - hash = ovn_logical_flow_hash_datapath(&lflow->od->sb->header_.uuid, - hash); - /* Add to single_dp_lflows. */ - hmap_insert_fast(&single_dp_lflows, &lflow->hmap_node, hash); - } - } - - /* Merge multiple and single dp hashes. */ - - fast_hmap_merge(lflows, &single_dp_lflows); - - hmap_destroy(&single_dp_lflows); - - stopwatch_stop(LFLOWS_DP_GROUPS_STOPWATCH_NAME, time_msec()); + * lflow map to a correct size before doing lookups */ + lflow_table_expand(lflows); + stopwatch_start(LFLOWS_TO_SB_STOPWATCH_NAME, time_msec()); - - struct hmap lflows_temp = HMAP_INITIALIZER(&lflows_temp); - /* Push changes to the Logical_Flow table to database. */ - const struct sbrec_logical_flow *sbflow; - SBREC_LOGICAL_FLOW_TABLE_FOR_EACH_SAFE (sbflow, - input_data->sbrec_logical_flow_table) { - struct sbrec_logical_dp_group *dp_group = sbflow->logical_dp_group; - struct ovn_datapath *logical_datapath_od = NULL; - size_t i; - - /* Find one valid datapath to get the datapath type. */ - struct sbrec_datapath_binding *dp = sbflow->logical_datapath; - if (dp) { - logical_datapath_od = ovn_datapath_from_sbrec( - &input_data->ls_datapaths->datapaths, - &input_data->lr_datapaths->datapaths, - dp); - if (logical_datapath_od - && ovn_datapath_is_stale(logical_datapath_od)) { - logical_datapath_od = NULL; - } - } - for (i = 0; dp_group && i < dp_group->n_datapaths; i++) { - logical_datapath_od = ovn_datapath_from_sbrec( - &input_data->ls_datapaths->datapaths, - &input_data->lr_datapaths->datapaths, - dp_group->datapaths[i]); - if (logical_datapath_od - && !ovn_datapath_is_stale(logical_datapath_od)) { - break; - } - logical_datapath_od = NULL; - } - - if (!logical_datapath_od) { - /* This lflow has no valid logical datapaths. */ - sbrec_logical_flow_delete(sbflow); - continue; - } - - enum ovn_pipeline pipeline - = !strcmp(sbflow->pipeline, "ingress") ? P_IN : P_OUT; - - lflow = ovn_lflow_find( - lflows, dp_group ? NULL : logical_datapath_od, - ovn_stage_build(ovn_datapath_get_type(logical_datapath_od), - pipeline, sbflow->table_id), - sbflow->priority, sbflow->match, sbflow->actions, - sbflow->controller_meter, sbflow->hash); - if (lflow) { - struct hmap *dp_groups; - size_t n_datapaths; - bool is_switch; - - lflow->sb_uuid = sbflow->header_.uuid; - is_switch = ovn_stage_to_datapath_type(lflow->stage) == DP_SWITCH; - if (is_switch) { - n_datapaths = ods_size(input_data->ls_datapaths); - dp_groups = &ls_dp_groups; - } else { - n_datapaths = ods_size(input_data->lr_datapaths); - dp_groups = &lr_dp_groups; - } - if (input_data->ovn_internal_version_changed) { - const char *stage_name = smap_get_def(&sbflow->external_ids, - "stage-name", ""); - const char *stage_hint = smap_get_def(&sbflow->external_ids, - "stage-hint", ""); - const char *source = smap_get_def(&sbflow->external_ids, - "source", ""); - - if (strcmp(stage_name, ovn_stage_to_str(lflow->stage))) { - sbrec_logical_flow_update_external_ids_setkey(sbflow, - "stage-name", ovn_stage_to_str(lflow->stage)); - } - if (lflow->stage_hint) { - if (strcmp(stage_hint, lflow->stage_hint)) { - sbrec_logical_flow_update_external_ids_setkey(sbflow, - "stage-hint", lflow->stage_hint); - } - } - if (lflow->where) { - if (strcmp(source, lflow->where)) { - sbrec_logical_flow_update_external_ids_setkey(sbflow, - "source", lflow->where); - } - } - } - - if (lflow->od) { - sbrec_logical_flow_set_logical_datapath(sbflow, lflow->od->sb); - sbrec_logical_flow_set_logical_dp_group(sbflow, NULL); - } else { - lflow->dpg = ovn_dp_group_get_or_create( - ovnsb_txn, dp_groups, dp_group, - lflow->n_ods, lflow->dpg_bitmap, - n_datapaths, is_switch, - input_data->ls_datapaths, - input_data->lr_datapaths); - - sbrec_logical_flow_set_logical_datapath(sbflow, NULL); - sbrec_logical_flow_set_logical_dp_group(sbflow, - lflow->dpg->dp_group); - } - - /* This lflow updated. Not needed anymore. */ - hmap_remove(lflows, &lflow->hmap_node); - hmap_insert(&lflows_temp, &lflow->hmap_node, - hmap_node_hash(&lflow->hmap_node)); - } else { - sbrec_logical_flow_delete(sbflow); - } - } - - HMAP_FOR_EACH_SAFE (lflow, hmap_node, lflows) { - const char *pipeline = ovn_stage_get_pipeline_name(lflow->stage); - uint8_t table = ovn_stage_get_table(lflow->stage); - struct hmap *dp_groups; - size_t n_datapaths; - bool is_switch; - - is_switch = ovn_stage_to_datapath_type(lflow->stage) == DP_SWITCH; - if (is_switch) { - n_datapaths = ods_size(input_data->ls_datapaths); - dp_groups = &ls_dp_groups; - } else { - n_datapaths = ods_size(input_data->lr_datapaths); - dp_groups = &lr_dp_groups; - } - - lflow->sb_uuid = uuid_random(); - sbflow = sbrec_logical_flow_insert_persist_uuid(ovnsb_txn, - &lflow->sb_uuid); - if (lflow->od) { - sbrec_logical_flow_set_logical_datapath(sbflow, lflow->od->sb); - } else { - lflow->dpg = ovn_dp_group_get_or_create( - ovnsb_txn, dp_groups, NULL, - lflow->n_ods, lflow->dpg_bitmap, - n_datapaths, is_switch, - input_data->ls_datapaths, - input_data->lr_datapaths); - - sbrec_logical_flow_set_logical_dp_group(sbflow, - lflow->dpg->dp_group); - } - - sbrec_logical_flow_set_pipeline(sbflow, pipeline); - sbrec_logical_flow_set_table_id(sbflow, table); - sbrec_logical_flow_set_priority(sbflow, lflow->priority); - sbrec_logical_flow_set_match(sbflow, lflow->match); - sbrec_logical_flow_set_actions(sbflow, lflow->actions); - if (lflow->io_port) { - struct smap tags = SMAP_INITIALIZER(&tags); - smap_add(&tags, "in_out_port", lflow->io_port); - sbrec_logical_flow_set_tags(sbflow, &tags); - smap_destroy(&tags); - } - sbrec_logical_flow_set_controller_meter(sbflow, lflow->ctrl_meter); - - /* Trim the source locator lflow->where, which looks something like - * "ovn/northd/northd.c:1234", down to just the part following the - * last slash, e.g. "northd.c:1234". */ - const char *slash = strrchr(lflow->where, '/'); -#if _WIN32 - const char *backslash = strrchr(lflow->where, '\\'); - if (!slash || backslash > slash) { - slash = backslash; - } -#endif - const char *where = slash ? slash + 1 : lflow->where; - - struct smap ids = SMAP_INITIALIZER(&ids); - smap_add(&ids, "stage-name", ovn_stage_to_str(lflow->stage)); - smap_add(&ids, "source", where); - if (lflow->stage_hint) { - smap_add(&ids, "stage-hint", lflow->stage_hint); - } - sbrec_logical_flow_set_external_ids(sbflow, &ids); - smap_destroy(&ids); - hmap_remove(lflows, &lflow->hmap_node); - hmap_insert(&lflows_temp, &lflow->hmap_node, - hmap_node_hash(&lflow->hmap_node)); - } - hmap_swap(lflows, &lflows_temp); - hmap_destroy(&lflows_temp); + lflow_table_sync_to_sb(lflows, ovnsb_txn, input_data->ls_datapaths, + input_data->lr_datapaths, + input_data->ovn_internal_version_changed, + input_data->sbrec_logical_flow_table, + input_data->sbrec_logical_dp_group_table); stopwatch_stop(LFLOWS_TO_SB_STOPWATCH_NAME, time_msec()); - struct ovn_dp_group *dpg; - HMAP_FOR_EACH_POP (dpg, node, &ls_dp_groups) { - bitmap_free(dpg->bitmap); - free(dpg); - } - hmap_destroy(&ls_dp_groups); - HMAP_FOR_EACH_POP (dpg, node, &lr_dp_groups) { - bitmap_free(dpg->bitmap); - free(dpg); - } - hmap_destroy(&lr_dp_groups); /* Push changes to the Multicast_Group table to database. */ const struct sbrec_multicast_group *sbmc; - SBREC_MULTICAST_GROUP_TABLE_FOR_EACH_SAFE (sbmc, - input_data->sbrec_multicast_group_table) { + SBREC_MULTICAST_GROUP_TABLE_FOR_EACH_SAFE ( + sbmc, input_data->sbrec_multicast_group_table) { struct ovn_datapath *od = ovn_datapath_from_sbrec( - &input_data->ls_datapaths->datapaths, - &input_data->lr_datapaths->datapaths, - sbmc->datapath); + &input_data->ls_datapaths->datapaths, + &input_data->lr_datapaths->datapaths, + sbmc->datapath); if (!od || ovn_datapath_is_stale(od)) { sbrec_multicast_group_delete(sbmc); @@ -16906,120 +16041,22 @@ void build_lflows(struct ovsdb_idl_txn *ovnsb_txn, hmap_destroy(&mcast_groups); } -static void -sync_lsp_lflows_to_sb(struct ovsdb_idl_txn *ovnsb_txn, - struct lflow_input *lflow_input, - struct hmap *lflows, - struct ovn_lflow *lflow) -{ - size_t n_datapaths; - struct ovn_datapath **datapaths_array; - if (ovn_stage_to_datapath_type(lflow->stage) == DP_SWITCH) { - n_datapaths = ods_size(lflow_input->ls_datapaths); - datapaths_array = lflow_input->ls_datapaths->array; - } else { - n_datapaths = ods_size(lflow_input->lr_datapaths); - datapaths_array = lflow_input->lr_datapaths->array; - } - uint32_t n_ods = bitmap_count1(lflow->dpg_bitmap, n_datapaths); - ovs_assert(n_ods == 1); - /* There is only one datapath, so it should be moved out of the - * group to a single 'od'. */ - size_t index = bitmap_scan(lflow->dpg_bitmap, true, 0, - n_datapaths); - - bitmap_set0(lflow->dpg_bitmap, index); - lflow->od = datapaths_array[index]; - - /* Logical flow should be re-hashed to allow lookups. */ - uint32_t hash = hmap_node_hash(&lflow->hmap_node); - /* Remove from lflows. */ - hmap_remove(lflows, &lflow->hmap_node); - hash = ovn_logical_flow_hash_datapath(&lflow->od->sb->header_.uuid, - hash); - /* Add back. */ - hmap_insert(lflows, &lflow->hmap_node, hash); - - /* Sync to SB. */ - const struct sbrec_logical_flow *sbflow; - /* Note: uuid_random acquires a global mutex. If we parallelize the sync to - * SB this may become a bottleneck. */ - lflow->sb_uuid = uuid_random(); - sbflow = sbrec_logical_flow_insert_persist_uuid(ovnsb_txn, - &lflow->sb_uuid); - const char *pipeline = ovn_stage_get_pipeline_name(lflow->stage); - uint8_t table = ovn_stage_get_table(lflow->stage); - sbrec_logical_flow_set_logical_datapath(sbflow, lflow->od->sb); - sbrec_logical_flow_set_logical_dp_group(sbflow, NULL); - sbrec_logical_flow_set_pipeline(sbflow, pipeline); - sbrec_logical_flow_set_table_id(sbflow, table); - sbrec_logical_flow_set_priority(sbflow, lflow->priority); - sbrec_logical_flow_set_match(sbflow, lflow->match); - sbrec_logical_flow_set_actions(sbflow, lflow->actions); - if (lflow->io_port) { - struct smap tags = SMAP_INITIALIZER(&tags); - smap_add(&tags, "in_out_port", lflow->io_port); - sbrec_logical_flow_set_tags(sbflow, &tags); - smap_destroy(&tags); - } - sbrec_logical_flow_set_controller_meter(sbflow, lflow->ctrl_meter); - /* Trim the source locator lflow->where, which looks something like - * "ovn/northd/northd.c:1234", down to just the part following the - * last slash, e.g. "northd.c:1234". */ - const char *slash = strrchr(lflow->where, '/'); -#if _WIN32 - const char *backslash = strrchr(lflow->where, '\\'); - if (!slash || backslash > slash) { - slash = backslash; - } -#endif - const char *where = slash ? slash + 1 : lflow->where; - - struct smap ids = SMAP_INITIALIZER(&ids); - smap_add(&ids, "stage-name", ovn_stage_to_str(lflow->stage)); - smap_add(&ids, "source", where); - if (lflow->stage_hint) { - smap_add(&ids, "stage-hint", lflow->stage_hint); - } - sbrec_logical_flow_set_external_ids(sbflow, &ids); - smap_destroy(&ids); -} - -static bool -delete_lflow_for_lsp(struct ovn_port *op, bool is_update, - const struct sbrec_logical_flow_table *sb_lflow_table, - struct hmap *lflows) -{ - struct lflow_ref_node *lfrn; - const char *operation = is_update ? "updated" : "deleted"; - LIST_FOR_EACH_SAFE (lfrn, lflow_list_node, &op->lflows) { - VLOG_DBG("Deleting SB lflow "UUID_FMT" for %s port %s", - UUID_ARGS(&lfrn->lflow->sb_uuid), operation, op->key); - - const struct sbrec_logical_flow *sblflow = - sbrec_logical_flow_table_get_for_uuid(sb_lflow_table, - &lfrn->lflow->sb_uuid); - if (sblflow) { - sbrec_logical_flow_delete(sblflow); - } else { - static struct vlog_rate_limit rl = - VLOG_RATE_LIMIT_INIT(1, 1); - VLOG_WARN_RL(&rl, "SB lflow "UUID_FMT" not found when handling " - "%s port %s. Recompute.", - UUID_ARGS(&lfrn->lflow->sb_uuid), operation, op->key); - return false; - } +void +lflow_reset_northd_refs(struct lflow_input *lflow_input) +{ + struct ovn_port *op; - ovn_lflow_destroy(lflows, lfrn->lflow); + HMAP_FOR_EACH (op, key_node, lflow_input->ls_ports) { + lflow_ref_clear(op->lflow_ref); + lflow_ref_clear(op->stateful_lflow_ref); } - return true; } bool lflow_handle_northd_port_changes(struct ovsdb_idl_txn *ovnsb_txn, struct tracked_ovn_ports *trk_lsps, struct lflow_input *lflow_input, - struct hmap *lflows) + struct lflow_table *lflows) { struct hmapx_node *hmapx_node; struct ovn_port *op; @@ -17028,13 +16065,15 @@ lflow_handle_northd_port_changes(struct ovsdb_idl_txn *ovnsb_txn, op = hmapx_node->data; /* Make sure 'op' is an lsp and not lrp. */ ovs_assert(op->nbsp); - - if (!delete_lflow_for_lsp(op, false, - lflow_input->sbrec_logical_flow_table, - lflows)) { - return false; - } - + bool handled = lflow_ref_resync_flows( + op->lflow_ref, lflows, ovnsb_txn, lflow_input->ls_datapaths, + lflow_input->lr_datapaths, + lflow_input->ovn_internal_version_changed, + lflow_input->sbrec_logical_flow_table, + lflow_input->sbrec_logical_dp_group_table); + if (!handled) { + return false; + } /* No need to update SB multicast groups, thanks to weak * references. */ } @@ -17043,13 +16082,8 @@ lflow_handle_northd_port_changes(struct ovsdb_idl_txn *ovnsb_txn, op = hmapx_node->data; /* Make sure 'op' is an lsp and not lrp. */ ovs_assert(op->nbsp); - - /* Delete old lflows. */ - if (!delete_lflow_for_lsp(op, true, - lflow_input->sbrec_logical_flow_table, - lflows)) { - return false; - } + /* Clear old lflows. */ + lflow_ref_unlink_lflows(op->lflow_ref); /* Generate new lflows. */ struct ds match = DS_EMPTY_INITIALIZER; @@ -17060,16 +16094,42 @@ lflow_handle_northd_port_changes(struct ovsdb_idl_txn *ovnsb_txn, &match, &actions, lflows); + /* Sync the new flows to SB. */ + bool handled = lflow_ref_sync_lflows( + op->lflow_ref, lflows, ovnsb_txn, lflow_input->ls_datapaths, + lflow_input->lr_datapaths, + lflow_input->ovn_internal_version_changed, + lflow_input->sbrec_logical_flow_table, + lflow_input->sbrec_logical_dp_group_table); + if (!handled) { + return false; + } + if (lsp_is_router(op->nbsp) && op->peer && op->peer->od->nbr) { const struct lr_stateful_record *lr_stateful_rec = lr_stateful_table_find_by_index(lflow_input->lr_stateful_table, op->peer->od->index); ovs_assert(lr_stateful_rec); + /* Clear old lflows. */ + lflow_ref_unlink_lflows(op->stateful_lflow_ref); + + /* Generate new lflows. */ build_lsp_lflows_for_lbnats(op, lr_stateful_rec, lflow_input->lr_stateful_table, lflow_input->lr_ports, - lflows, &match, &actions); + lflows, &match, &actions, + op->stateful_lflow_ref); + handled = lflow_ref_sync_lflows( + op->stateful_lflow_ref, lflows, ovnsb_txn, + lflow_input->ls_datapaths, + lflow_input->lr_datapaths, + lflow_input->ovn_internal_version_changed, + lflow_input->sbrec_logical_flow_table, + lflow_input->sbrec_logical_dp_group_table); + if (!handled) { + return false; + } } ds_destroy(&match); @@ -17077,13 +16137,6 @@ lflow_handle_northd_port_changes(struct ovsdb_idl_txn *ovnsb_txn, /* SB port_binding is not deleted, so don't update SB multicast * groups. */ - - /* Sync the new flows to SB. */ - struct lflow_ref_node *lfrn; - LIST_FOR_EACH (lfrn, lflow_list_node, &op->lflows) { - sync_lsp_lflows_to_sb(ovnsb_txn, lflow_input, lflows, - lfrn->lflow); - } } HMAPX_FOR_EACH (hmapx_node, &trk_lsps->created) { @@ -17108,6 +16161,17 @@ lflow_handle_northd_port_changes(struct ovsdb_idl_txn *ovnsb_txn, lflow_input->meter_groups, &match, &actions, lflows); + /* Sync the newly added flows to SB. */ + bool handled = lflow_ref_sync_lflows( + op->lflow_ref, lflows, ovnsb_txn, lflow_input->ls_datapaths, + lflow_input->lr_datapaths, + lflow_input->ovn_internal_version_changed, + lflow_input->sbrec_logical_flow_table, + lflow_input->sbrec_logical_dp_group_table); + if (!handled) { + return false; + } + if (lsp_is_router(op->nbsp) && op->peer && op->peer->od->nbr) { const struct lr_stateful_record *lr_stateful_rec = lr_stateful_table_find_by_index(lflow_input->lr_stateful_table, @@ -17117,7 +16181,18 @@ lflow_handle_northd_port_changes(struct ovsdb_idl_txn *ovnsb_txn, build_lsp_lflows_for_lbnats(op, lr_stateful_rec, lflow_input->lr_stateful_table, lflow_input->lr_ports, - lflows, &match, &actions); + lflows, &match, &actions, + op->stateful_lflow_ref); + handled = lflow_ref_sync_lflows( + op->stateful_lflow_ref, lflows, ovnsb_txn, + lflow_input->ls_datapaths, + lflow_input->lr_datapaths, + lflow_input->ovn_internal_version_changed, + lflow_input->sbrec_logical_flow_table, + lflow_input->sbrec_logical_dp_group_table); + if (!handled) { + return false; + } } ds_destroy(&match); @@ -17146,13 +16221,6 @@ lflow_handle_northd_port_changes(struct ovsdb_idl_txn *ovnsb_txn, sbrec_multicast_group_update_ports_addvalue(sbmc_unknown, op->sb); } - - /* Sync the newly added flows to SB. */ - struct lflow_ref_node *lfrn; - LIST_FOR_EACH (lfrn, lflow_list_node, &op->lflows) { - sync_lsp_lflows_to_sb(ovnsb_txn, lflow_input, lflows, - lfrn->lflow); - } } return true; diff --git a/northd/northd.h b/northd/northd.h index 88406bffee..42b4eee607 100644 --- a/northd/northd.h +++ b/northd/northd.h @@ -23,6 +23,7 @@ #include "northd/en-port-group.h" #include "northd/ipam.h" #include "openvswitch/hmap.h" +#include "ovs-thread.h" struct northd_input { /* Northbound table references */ @@ -164,13 +165,6 @@ struct northd_data { struct northd_tracked_data trk_data; }; -struct lflow_data { - struct hmap lflows; -}; - -void lflow_data_init(struct lflow_data *); -void lflow_data_destroy(struct lflow_data *); - struct lr_nat_table; struct lflow_input { @@ -182,6 +176,7 @@ struct lflow_input { const struct sbrec_logical_flow_table *sbrec_logical_flow_table; const struct sbrec_multicast_group_table *sbrec_multicast_group_table; const struct sbrec_igmp_group_table *sbrec_igmp_group_table; + const struct sbrec_logical_dp_group_table *sbrec_logical_dp_group_table; /* Indexes */ struct ovsdb_idl_index *sbrec_mcast_group_by_name_dp; @@ -201,6 +196,15 @@ struct lflow_input { bool ovn_internal_version_changed; }; +extern int parallelization_state; +enum { + STATE_NULL, /* parallelization is off */ + STATE_INIT_HASH_SIZES, /* parallelization is on; hashes sizing needed */ + STATE_USE_PARALLELIZATION /* parallelization is on */ +}; + +extern thread_local size_t thread_lflow_counter; + /* * Multicast snooping and querier per datapath configuration. */ @@ -344,6 +348,179 @@ struct ovn_datapath { const struct ovn_datapath *ovn_datapath_find(const struct hmap *datapaths, const struct uuid *uuid); +struct ovn_datapath *ovn_datapath_from_sbrec( + const struct hmap *ls_datapaths, const struct hmap *lr_datapaths, + const struct sbrec_datapath_binding *); + +static inline bool +ovn_datapath_is_stale(const struct ovn_datapath *od) +{ + return !od->nbr && !od->nbs; +}; + +/* Pipeline stages. */ + +/* The two purposes for which ovn-northd uses OVN logical datapaths. */ +enum ovn_datapath_type { + DP_SWITCH, /* OVN logical switch. */ + DP_ROUTER /* OVN logical router. */ +}; + +/* Returns an "enum ovn_stage" built from the arguments. + * + * (It's better to use ovn_stage_build() for type-safety reasons, but inline + * functions can't be used in enums or switch cases.) */ +#define OVN_STAGE_BUILD(DP_TYPE, PIPELINE, TABLE) \ + (((DP_TYPE) << 9) | ((PIPELINE) << 8) | (TABLE)) + +/* A stage within an OVN logical switch or router. + * + * An "enum ovn_stage" indicates whether the stage is part of a logical switch + * or router, whether the stage is part of the ingress or egress pipeline, and + * the table within that pipeline. The first three components are combined to + * form the stage's full name, e.g. S_SWITCH_IN_PORT_SEC_L2, + * S_ROUTER_OUT_DELIVERY. */ +enum ovn_stage { +#define PIPELINE_STAGES \ + /* Logical switch ingress stages. */ \ + PIPELINE_STAGE(SWITCH, IN, CHECK_PORT_SEC, 0, "ls_in_check_port_sec") \ + PIPELINE_STAGE(SWITCH, IN, APPLY_PORT_SEC, 1, "ls_in_apply_port_sec") \ + PIPELINE_STAGE(SWITCH, IN, LOOKUP_FDB , 2, "ls_in_lookup_fdb") \ + PIPELINE_STAGE(SWITCH, IN, PUT_FDB, 3, "ls_in_put_fdb") \ + PIPELINE_STAGE(SWITCH, IN, PRE_ACL, 4, "ls_in_pre_acl") \ + PIPELINE_STAGE(SWITCH, IN, PRE_LB, 5, "ls_in_pre_lb") \ + PIPELINE_STAGE(SWITCH, IN, PRE_STATEFUL, 6, "ls_in_pre_stateful") \ + PIPELINE_STAGE(SWITCH, IN, ACL_HINT, 7, "ls_in_acl_hint") \ + PIPELINE_STAGE(SWITCH, IN, ACL_EVAL, 8, "ls_in_acl_eval") \ + PIPELINE_STAGE(SWITCH, IN, ACL_ACTION, 9, "ls_in_acl_action") \ + PIPELINE_STAGE(SWITCH, IN, QOS_MARK, 10, "ls_in_qos_mark") \ + PIPELINE_STAGE(SWITCH, IN, QOS_METER, 11, "ls_in_qos_meter") \ + PIPELINE_STAGE(SWITCH, IN, LB_AFF_CHECK, 12, "ls_in_lb_aff_check") \ + PIPELINE_STAGE(SWITCH, IN, LB, 13, "ls_in_lb") \ + PIPELINE_STAGE(SWITCH, IN, LB_AFF_LEARN, 14, "ls_in_lb_aff_learn") \ + PIPELINE_STAGE(SWITCH, IN, PRE_HAIRPIN, 15, "ls_in_pre_hairpin") \ + PIPELINE_STAGE(SWITCH, IN, NAT_HAIRPIN, 16, "ls_in_nat_hairpin") \ + PIPELINE_STAGE(SWITCH, IN, HAIRPIN, 17, "ls_in_hairpin") \ + PIPELINE_STAGE(SWITCH, IN, ACL_AFTER_LB_EVAL, 18, \ + "ls_in_acl_after_lb_eval") \ + PIPELINE_STAGE(SWITCH, IN, ACL_AFTER_LB_ACTION, 19, \ + "ls_in_acl_after_lb_action") \ + PIPELINE_STAGE(SWITCH, IN, STATEFUL, 20, "ls_in_stateful") \ + PIPELINE_STAGE(SWITCH, IN, ARP_ND_RSP, 21, "ls_in_arp_rsp") \ + PIPELINE_STAGE(SWITCH, IN, DHCP_OPTIONS, 22, "ls_in_dhcp_options") \ + PIPELINE_STAGE(SWITCH, IN, DHCP_RESPONSE, 23, "ls_in_dhcp_response") \ + PIPELINE_STAGE(SWITCH, IN, DNS_LOOKUP, 24, "ls_in_dns_lookup") \ + PIPELINE_STAGE(SWITCH, IN, DNS_RESPONSE, 25, "ls_in_dns_response") \ + PIPELINE_STAGE(SWITCH, IN, EXTERNAL_PORT, 26, "ls_in_external_port") \ + PIPELINE_STAGE(SWITCH, IN, L2_LKUP, 27, "ls_in_l2_lkup") \ + PIPELINE_STAGE(SWITCH, IN, L2_UNKNOWN, 28, "ls_in_l2_unknown") \ + \ + /* Logical switch egress stages. */ \ + PIPELINE_STAGE(SWITCH, OUT, PRE_ACL, 0, "ls_out_pre_acl") \ + PIPELINE_STAGE(SWITCH, OUT, PRE_LB, 1, "ls_out_pre_lb") \ + PIPELINE_STAGE(SWITCH, OUT, PRE_STATEFUL, 2, "ls_out_pre_stateful") \ + PIPELINE_STAGE(SWITCH, OUT, ACL_HINT, 3, "ls_out_acl_hint") \ + PIPELINE_STAGE(SWITCH, OUT, ACL_EVAL, 4, "ls_out_acl_eval") \ + PIPELINE_STAGE(SWITCH, OUT, ACL_ACTION, 5, "ls_out_acl_action") \ + PIPELINE_STAGE(SWITCH, OUT, QOS_MARK, 6, "ls_out_qos_mark") \ + PIPELINE_STAGE(SWITCH, OUT, QOS_METER, 7, "ls_out_qos_meter") \ + PIPELINE_STAGE(SWITCH, OUT, STATEFUL, 8, "ls_out_stateful") \ + PIPELINE_STAGE(SWITCH, OUT, CHECK_PORT_SEC, 9, "ls_out_check_port_sec") \ + PIPELINE_STAGE(SWITCH, OUT, APPLY_PORT_SEC, 10, "ls_out_apply_port_sec") \ + \ + /* Logical router ingress stages. */ \ + PIPELINE_STAGE(ROUTER, IN, ADMISSION, 0, "lr_in_admission") \ + PIPELINE_STAGE(ROUTER, IN, LOOKUP_NEIGHBOR, 1, "lr_in_lookup_neighbor") \ + PIPELINE_STAGE(ROUTER, IN, LEARN_NEIGHBOR, 2, "lr_in_learn_neighbor") \ + PIPELINE_STAGE(ROUTER, IN, IP_INPUT, 3, "lr_in_ip_input") \ + PIPELINE_STAGE(ROUTER, IN, UNSNAT, 4, "lr_in_unsnat") \ + PIPELINE_STAGE(ROUTER, IN, DEFRAG, 5, "lr_in_defrag") \ + PIPELINE_STAGE(ROUTER, IN, LB_AFF_CHECK, 6, "lr_in_lb_aff_check") \ + PIPELINE_STAGE(ROUTER, IN, DNAT, 7, "lr_in_dnat") \ + PIPELINE_STAGE(ROUTER, IN, LB_AFF_LEARN, 8, "lr_in_lb_aff_learn") \ + PIPELINE_STAGE(ROUTER, IN, ECMP_STATEFUL, 9, "lr_in_ecmp_stateful") \ + PIPELINE_STAGE(ROUTER, IN, ND_RA_OPTIONS, 10, "lr_in_nd_ra_options") \ + PIPELINE_STAGE(ROUTER, IN, ND_RA_RESPONSE, 11, "lr_in_nd_ra_response") \ + PIPELINE_STAGE(ROUTER, IN, IP_ROUTING_PRE, 12, "lr_in_ip_routing_pre") \ + PIPELINE_STAGE(ROUTER, IN, IP_ROUTING, 13, "lr_in_ip_routing") \ + PIPELINE_STAGE(ROUTER, IN, IP_ROUTING_ECMP, 14, "lr_in_ip_routing_ecmp") \ + PIPELINE_STAGE(ROUTER, IN, POLICY, 15, "lr_in_policy") \ + PIPELINE_STAGE(ROUTER, IN, POLICY_ECMP, 16, "lr_in_policy_ecmp") \ + PIPELINE_STAGE(ROUTER, IN, ARP_RESOLVE, 17, "lr_in_arp_resolve") \ + PIPELINE_STAGE(ROUTER, IN, CHK_PKT_LEN, 18, "lr_in_chk_pkt_len") \ + PIPELINE_STAGE(ROUTER, IN, LARGER_PKTS, 19, "lr_in_larger_pkts") \ + PIPELINE_STAGE(ROUTER, IN, GW_REDIRECT, 20, "lr_in_gw_redirect") \ + PIPELINE_STAGE(ROUTER, IN, ARP_REQUEST, 21, "lr_in_arp_request") \ + \ + /* Logical router egress stages. */ \ + PIPELINE_STAGE(ROUTER, OUT, CHECK_DNAT_LOCAL, 0, \ + "lr_out_chk_dnat_local") \ + PIPELINE_STAGE(ROUTER, OUT, UNDNAT, 1, "lr_out_undnat") \ + PIPELINE_STAGE(ROUTER, OUT, POST_UNDNAT, 2, "lr_out_post_undnat") \ + PIPELINE_STAGE(ROUTER, OUT, SNAT, 3, "lr_out_snat") \ + PIPELINE_STAGE(ROUTER, OUT, POST_SNAT, 4, "lr_out_post_snat") \ + PIPELINE_STAGE(ROUTER, OUT, EGR_LOOP, 5, "lr_out_egr_loop") \ + PIPELINE_STAGE(ROUTER, OUT, DELIVERY, 6, "lr_out_delivery") + +#define PIPELINE_STAGE(DP_TYPE, PIPELINE, STAGE, TABLE, NAME) \ + S_##DP_TYPE##_##PIPELINE##_##STAGE \ + = OVN_STAGE_BUILD(DP_##DP_TYPE, P_##PIPELINE, TABLE), + PIPELINE_STAGES +#undef PIPELINE_STAGE +}; + +enum ovn_datapath_type ovn_stage_to_datapath_type(enum ovn_stage stage); + + +/* Returns 'od''s datapath type. */ +static inline enum ovn_datapath_type +ovn_datapath_get_type(const struct ovn_datapath *od) +{ + return od->nbs ? DP_SWITCH : DP_ROUTER; +} + +/* Returns an "enum ovn_stage" built from the arguments. */ +static inline enum ovn_stage +ovn_stage_build(enum ovn_datapath_type dp_type, enum ovn_pipeline pipeline, + uint8_t table) +{ + return OVN_STAGE_BUILD(dp_type, pipeline, table); +} + +/* Returns the pipeline to which 'stage' belongs. */ +static inline enum ovn_pipeline +ovn_stage_get_pipeline(enum ovn_stage stage) +{ + return (stage >> 8) & 1; +} + +/* Returns the pipeline name to which 'stage' belongs. */ +static inline const char * +ovn_stage_get_pipeline_name(enum ovn_stage stage) +{ + return ovn_stage_get_pipeline(stage) == P_IN ? "ingress" : "egress"; +} + +/* Returns the table to which 'stage' belongs. */ +static inline uint8_t +ovn_stage_get_table(enum ovn_stage stage) +{ + return stage & 0xff; +} + +/* Returns a string name for 'stage'. */ +static inline const char * +ovn_stage_to_str(enum ovn_stage stage) +{ + switch (stage) { +#define PIPELINE_STAGE(DP_TYPE, PIPELINE, STAGE, TABLE, NAME) \ + case S_##DP_TYPE##_##PIPELINE##_##STAGE: return NAME; + PIPELINE_STAGES +#undef PIPELINE_STAGE + default: return ""; + } +} + /* A logical switch port or logical router port. * * In steady state, an ovn_port points to a northbound Logical_Switch_Port @@ -434,8 +611,10 @@ struct ovn_port { /* Temporarily used for traversing a list (or hmap) of ports. */ bool visited; - /* List of struct lflow_ref_node that points to the lflows generated by - * this ovn_port. + /* Only used for the router type LSP whose peer is l3dgw_port */ + bool enable_router_port_acl; + + /* Reference of lflows generated for this ovn_port. * * This data is initialized and destroyed by the en_northd node, but * populated and used only by the en_lflow node. Ideally this data should @@ -453,11 +632,16 @@ struct ovn_port { * Adding the list here is more straightforward. The drawback is that we * need to keep in mind that this data belongs to en_lflow node, so never * access it from any other nodes. + * + * 'lflow_ref' is used to reference generic logical flows generated for + * this ovn_port. + * + * 'stateful_lflow_ref' is used for logical switch ports of type + * 'patch/router' to reference logical flows generated fo this ovn_port + * from the 'lr_stateful' record of the peer port's datapath. */ - struct ovs_list lflows; - - /* Only used for the router type LSP whose peer is l3dgw_port */ - bool enable_router_port_acl; + struct lflow_ref *lflow_ref; + struct lflow_ref *stateful_lflow_ref; }; void ovnnb_db_run(struct northd_input *input_data, @@ -480,13 +664,17 @@ void northd_destroy(struct northd_data *data); void northd_init(struct northd_data *data); void northd_indices_create(struct northd_data *data, struct ovsdb_idl *ovnsb_idl); + +struct lflow_table; void build_lflows(struct ovsdb_idl_txn *ovnsb_txn, struct lflow_input *input_data, - struct hmap *lflows); + struct lflow_table *); +void lflow_reset_northd_refs(struct lflow_input *); + bool lflow_handle_northd_port_changes(struct ovsdb_idl_txn *ovnsb_txn, struct tracked_ovn_ports *, struct lflow_input *, - struct hmap *lflows); + struct lflow_table *lflows); bool northd_handle_sb_port_binding_changes( const struct sbrec_port_binding_table *, struct hmap *ls_ports, struct hmap *lr_ports); diff --git a/northd/ovn-northd.c b/northd/ovn-northd.c index 084d675644..e0e60f3559 100644 --- a/northd/ovn-northd.c +++ b/northd/ovn-northd.c @@ -848,6 +848,10 @@ main(int argc, char *argv[]) ovsdb_idl_omit_alert(ovnsb_idl_loop.idl, &sbrec_port_group_columns[i]); } + for (size_t i = 0; i < SBREC_LOGICAL_DP_GROUP_N_COLUMNS; i++) { + ovsdb_idl_omit_alert(ovnsb_idl_loop.idl, + &sbrec_logical_dp_group_columns[i]); + } unixctl_command_register("sb-connection-status", "", 0, 0, ovn_conn_show, ovnsb_idl_loop.idl); diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at index 1825fd3e18..f7d47fc7e4 100644 --- a/tests/ovn-northd.at +++ b/tests/ovn-northd.at @@ -11267,6 +11267,222 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE AT_CLEANUP ]) +OVN_FOR_EACH_NORTHD_NO_HV([ +AT_SETUP([Load balancer incremental processing for multiple LBs with same VIPs]) +ovn_start + +check ovn-nbctl ls-add sw0 +check ovn-nbctl ls-add sw1 +check ovn-nbctl lb-add lb1 10.0.0.10:80 10.0.0.3:80 +check ovn-nbctl --wait=sb lb-add lb2 10.0.0.10:80 10.0.0.3:80 + +check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats +check ovn-nbctl --wait=sb ls-lb-add sw0 lb1 +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +lb_lflow_uuid=$(fetch_column Logical_flow _uuid match='"ct.new && ip4.dst == 10.0.0.10 && tcp.dst == 80"') +sw0_uuid=$(fetch_column Datapath_Binding _uuid external_ids:name=sw0) + +lb_lflow_dp=$(ovn-sbctl --bare --columns logical_datapath list logical_flow $lb_lflow_uuid) +AT_CHECK([test "$lb_lflow_dp" = "$sw0_uuid"]) + +lb_lflow_dpgrp=$(ovn-sbctl --bare --columns logical_dp_group list logical_flow $lb_lflow_uuid) +AT_CHECK([test "$lb_lflow_dpgrp" = ""]) + +check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats +check ovn-nbctl --wait=sb ls-lb-add sw1 lb2 +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +lb_lflow_dp=$(ovn-sbctl --bare --columns logical_datapath list logical_flow $lb_lflow_uuid) +AT_CHECK([test "$lb_lflow_dp" = ""]) + +lb_lflow_dpgrp=$(ovn-sbctl --bare --columns logical_dp_group list logical_flow $lb_lflow_uuid) +AT_CHECK([test "$lb_lflow_dpgrp" != ""]) + +# Clear the SB:Logical_Flow.logical_dp_groups column of all the +# logical flows and then modify the NB:Load_balancer. ovn-northd +# should resync the logical flows. +for l in $(ovn-sbctl --bare --columns _uuid list logical_flow) +do + ovn-sbctl clear logical_flow $l logical_dp_group +done + +check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats +check ovn-nbctl --wait=sb set load_balancer lb1 options:foo=bar +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +lb_lflow_uuid=$(fetch_column Logical_flow _uuid match='"ct.new && ip4.dst == 10.0.0.10 && tcp.dst == 80"') + +check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats +check ovn-nbctl --wait=sb clear load_balancer lb2 vips +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +lb_lflow_dp=$(ovn-sbctl --bare --columns logical_datapath list logical_flow $lb_lflow_uuid) +AT_CHECK([test "$lb_lflow_dp" = "$sw0_uuid"]) + +lb_lflow_dpgrp=$(ovn-sbctl --bare --columns logical_dp_group list logical_flow $lb_lflow_uuid) +AT_CHECK([test "$lb_lflow_dpgrp" = ""]) + +# Add back the vip to lb2. +check ovn-nbctl lb-add lb2 10.0.0.10:80 10.0.0.3:80 + +# Create additional logical switches and associate lb1 to sw0, sw1 and sw2 +# and associate lb2 to sw3, sw4 and sw5 +check ovn-nbctl ls-add sw2 +check ovn-nbctl ls-add sw3 +check ovn-nbctl ls-add sw4 +check ovn-nbctl ls-add sw5 +check ovn-nbctl --wait=sb ls-lb-del sw1 lb2 +check ovn-nbctl ls-lb-add sw1 lb1 +check ovn-nbctl ls-lb-add sw2 lb1 +check ovn-nbctl ls-lb-add sw3 lb2 +check ovn-nbctl ls-lb-add sw4 lb2 + +check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats +check ovn-nbctl --wait=sb ls-lb-add sw5 lb2 +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +lb_lflow_dp=$(ovn-sbctl --bare --columns logical_datapath list logical_flow $lb_lflow_uuid) +AT_CHECK([test "$lb_lflow_dp" = ""]) + +lb_lflow_dpgrp=$(ovn-sbctl --bare --columns logical_dp_group list logical_flow $lb_lflow_uuid) +AT_CHECK([test "$lb_lflow_dpgrp" != ""]) + +sw1_uuid=$(fetch_column Datapath_Binding _uuid external_ids:name=sw1) +sw2_uuid=$(fetch_column Datapath_Binding _uuid external_ids:name=sw2) +sw3_uuid=$(fetch_column Datapath_Binding _uuid external_ids:name=sw3) +sw4_uuid=$(fetch_column Datapath_Binding _uuid external_ids:name=sw4) +sw5_uuid=$(fetch_column Datapath_Binding _uuid external_ids:name=sw5) + +dpgrp_dps=$(ovn-sbctl --bare --columns datapaths list logical_dp_group $lb_lflow_dpgrp) + +AT_CHECK([echo $dpgrp_dps | grep $sw0_uuid], [0], [ignore]) +AT_CHECK([echo $dpgrp_dps | grep $sw1_uuid], [0], [ignore]) +AT_CHECK([echo $dpgrp_dps | grep $sw2_uuid], [0], [ignore]) +AT_CHECK([echo $dpgrp_dps | grep $sw3_uuid], [0], [ignore]) +AT_CHECK([echo $dpgrp_dps | grep $sw4_uuid], [0], [ignore]) +AT_CHECK([echo $dpgrp_dps | grep $sw5_uuid], [0], [ignore]) + +echo "dpgrp_dps - $dpgrp_dps" + +# Clear the vips for lb2. The logical lb logical flow dp group should have +# only sw0, sw1 and sw2 uuids. + +check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats +check ovn-nbctl --wait=sb clear load_balancer lb2 vips +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +lb_lflow_dpgrp=$(ovn-sbctl --bare --columns logical_dp_group list logical_flow $lb_lflow_uuid) +AT_CHECK([test "$lb_lflow_dpgrp" != ""]) + +dpgrp_dps=$(ovn-sbctl --bare --columns datapaths list logical_dp_group $lb_lflow_dpgrp) + +AT_CHECK([echo $dpgrp_dps | grep $sw0_uuid], [0], [ignore]) +AT_CHECK([echo $dpgrp_dps | grep $sw1_uuid], [0], [ignore]) +AT_CHECK([echo $dpgrp_dps | grep $sw2_uuid], [0], [ignore]) +AT_CHECK([echo $dpgrp_dps | grep $sw3_uuid], [1], [ignore]) +AT_CHECK([echo $dpgrp_dps | grep $sw4_uuid], [1], [ignore]) +AT_CHECK([echo $dpgrp_dps | grep $sw5_uuid], [1], [ignore]) + +# Clear the vips for lb1. The logical flow should be deleted. +check ovn-nbctl --wait=sb clear load_balancer lb1 vips + +AT_CHECK([ovn-sbctl --bare --columns logical_datapath list logical_flow $lb_lflow_uuid], [1], [ignore], [ignore]) + +lb_lflow_uuid=$(fetch_column Logical_flow _uuid match='"ct.new && ip4.dst == 10.0.0.10 && tcp.dst == 80"') +AT_CHECK([test "$lb_lflow_uuid" = ""]) + + +# Now add back the vips, create another lb with the same vips and associate to +# sw0 and sw1 +check ovn-nbctl lb-add lb1 10.0.0.10:80 10.0.0.3:80 +check ovn-nbctl lb-add lb2 10.0.0.10:80 10.0.0.3:80 +check ovn-nbctl --wait=sb lb-add lb3 10.0.0.10:80 10.0.0.3:80 + +check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats + +check ovn-nbctl ls-lb-add sw0 lb3 +check ovn-nbctl --wait=sb ls-lb-add sw1 lb3 +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +lb_lflow_uuid=$(fetch_column Logical_flow _uuid match='"ct.new && ip4.dst == 10.0.0.10 && tcp.dst == 80"') + +lb_lflow_dp=$(ovn-sbctl --bare --columns logical_datapath list logical_flow $lb_lflow_uuid) +AT_CHECK([test "$lb_lflow_dp" = ""]) + +lb_lflow_dpgrp=$(ovn-sbctl --bare --columns logical_dp_group list logical_flow $lb_lflow_uuid) +AT_CHECK([test "$lb_lflow_dpgrp" != ""]) + +dpgrp_dps=$(ovn-sbctl --bare --columns datapaths list logical_dp_group $lb_lflow_dpgrp) + +AT_CHECK([echo $dpgrp_dps | grep $sw0_uuid], [0], [ignore]) +AT_CHECK([echo $dpgrp_dps | grep $sw1_uuid], [0], [ignore]) +AT_CHECK([echo $dpgrp_dps | grep $sw2_uuid], [0], [ignore]) +AT_CHECK([echo $dpgrp_dps | grep $sw3_uuid], [0], [ignore]) +AT_CHECK([echo $dpgrp_dps | grep $sw4_uuid], [0], [ignore]) +AT_CHECK([echo $dpgrp_dps | grep $sw5_uuid], [0], [ignore]) + +# Now clear lb1 vips. +# Since lb3 is associated with sw0 and sw1, the logical flow db group +# should have reference to sw0 and sw1, but not to sw2. +check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats +check ovn-nbctl --wait=sb clear load_balancer lb1 vips +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +lb_lflow_dp=$(ovn-sbctl --bare --columns logical_datapath list logical_flow $lb_lflow_uuid) +AT_CHECK([test "$lb_lflow_dp" = ""]) + +lb_lflow_dpgrp=$(ovn-sbctl --bare --columns logical_dp_group list logical_flow $lb_lflow_uuid) +AT_CHECK([test "$lb_lflow_dpgrp" != ""]) + +dpgrp_dps=$(ovn-sbctl --bare --columns datapaths list logical_dp_group $lb_lflow_dpgrp) + +echo "dpgrp dps - $dpgrp_dps" + +AT_CHECK([echo $dpgrp_dps | grep $sw0_uuid], [0], [ignore]) +AT_CHECK([echo $dpgrp_dps | grep $sw1_uuid], [0], [ignore]) +AT_CHECK([echo $dpgrp_dps | grep $sw2_uuid], [1], [ignore]) +AT_CHECK([echo $dpgrp_dps | grep $sw3_uuid], [0], [ignore]) +AT_CHECK([echo $dpgrp_dps | grep $sw4_uuid], [0], [ignore]) +AT_CHECK([echo $dpgrp_dps | grep $sw5_uuid], [0], [ignore]) + +# Now clear lb3 vips. The logical flow db group +# should have reference only to sw3, sw4 and sw5 because lb2 is +# associated to them. + +check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats +check ovn-nbctl --wait=sb clear load_balancer lb3 vips +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +lb_lflow_dp=$(ovn-sbctl --bare --columns logical_datapath list logical_flow $lb_lflow_uuid) +AT_CHECK([test "$lb_lflow_dp" = ""]) + +lb_lflow_dpgrp=$(ovn-sbctl --bare --columns logical_dp_group list logical_flow $lb_lflow_uuid) +AT_CHECK([test "$lb_lflow_dpgrp" != ""]) + +dpgrp_dps=$(ovn-sbctl --bare --columns datapaths list logical_dp_group $lb_lflow_dpgrp) + +echo "dpgrp dps - $dpgrp_dps" + +AT_CHECK([echo $dpgrp_dps | grep $sw0_uuid], [1], [ignore]) +AT_CHECK([echo $dpgrp_dps | grep $sw1_uuid], [1], [ignore]) +AT_CHECK([echo $dpgrp_dps | grep $sw2_uuid], [1], [ignore]) +AT_CHECK([echo $dpgrp_dps | grep $sw3_uuid], [0], [ignore]) +AT_CHECK([echo $dpgrp_dps | grep $sw4_uuid], [0], [ignore]) +AT_CHECK([echo $dpgrp_dps | grep $sw5_uuid], [0], [ignore]) + +AT_CLEANUP +]) + OVN_FOR_EACH_NORTHD_NO_HV([ AT_SETUP([Logical router incremental processing for NAT]) From patchwork Thu Jan 11 15:31:45 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Numan Siddique X-Patchwork-Id: 1885678 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::133; helo=smtp2.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp2.osuosl.org (smtp2.osuosl.org [IPv6:2605:bc80:3010::133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4T9pZb5kXXz1yPp for ; Fri, 12 Jan 2024 02:32:39 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 0CDEB4390B; Thu, 11 Jan 2024 15:32:37 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 0CDEB4390B X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id R13Is20u1ChN; Thu, 11 Jan 2024 15:32:33 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp2.osuosl.org (Postfix) with ESMTPS id 8A0CF438DE; Thu, 11 Jan 2024 15:32:31 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 8A0CF438DE Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 5C821C0077; Thu, 11 Jan 2024 15:32:31 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp4.osuosl.org (smtp4.osuosl.org [140.211.166.137]) by lists.linuxfoundation.org (Postfix) with ESMTP id 8C223C0037 for ; Thu, 11 Jan 2024 15:32:30 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 03AF342C67 for ; Thu, 11 Jan 2024 15:32:28 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 03AF342C67 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id BDhHB0-RS8XA for ; Thu, 11 Jan 2024 15:32:25 +0000 (UTC) Received: from relay3-d.mail.gandi.net (relay3-d.mail.gandi.net [217.70.183.195]) by smtp4.osuosl.org (Postfix) with ESMTPS id 3D25E42C41 for ; Thu, 11 Jan 2024 15:32:23 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 3D25E42C41 Received: by mail.gandi.net (Postfix) with ESMTPSA id 344AC60002; Thu, 11 Jan 2024 15:32:20 +0000 (UTC) From: numans@ovn.org To: dev@openvswitch.org Date: Thu, 11 Jan 2024 10:31:45 -0500 Message-ID: <20240111153145.2790297-1-numans@ovn.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240111152752.2789854-1-numans@ovn.org> References: <20240111152752.2789854-1-numans@ovn.org> MIME-Version: 1.0 X-GND-Sasl: numans@ovn.org Subject: [ovs-dev] [PATCH ovn v5 09/16] northd: Use lflow_ref when adding all logical flows. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Numan Siddique ovn_lflow_add* macros now expect 'lflow_ref' to be passed and it can be NULL too. References to the logical flows generated for router ports are now stored in op->lflow_ref and op->stateful_lflow_ref. Signed-off-by: Numan Siddique Acked-by: Dumitru Ceara --- northd/lflow-mgr.h | 38 +- northd/northd.c | 1614 +++++++++++++++++++++++++++----------------- 2 files changed, 991 insertions(+), 661 deletions(-) diff --git a/northd/lflow-mgr.h b/northd/lflow-mgr.h index 5a0cc28965..f215891b97 100644 --- a/northd/lflow-mgr.h +++ b/northd/lflow-mgr.h @@ -88,41 +88,27 @@ void lflow_table_add_lflow_default_drop(struct lflow_table *, /* Adds a row with the specified contents to the Logical_Flow table. */ #define ovn_lflow_add_with_hint__(LFLOW_TABLE, OD, STAGE, PRIORITY, MATCH, \ ACTIONS, IN_OUT_PORT, CTRL_METER, \ - STAGE_HINT) \ - lflow_table_add_lflow(LFLOW_TABLE, OD, NULL, 0, STAGE, PRIORITY, MATCH, \ - ACTIONS, IN_OUT_PORT, CTRL_METER, STAGE_HINT, \ - OVS_SOURCE_LOCATOR, NULL) - -#define ovn_lflow_add_with_lflow_ref_hint__(LFLOW_TABLE, OD, STAGE, PRIORITY, \ - MATCH, ACTIONS, IN_OUT_PORT, \ - CTRL_METER, STAGE_HINT, LFLOW_REF)\ + STAGE_HINT, LFLOW_REF) \ lflow_table_add_lflow(LFLOW_TABLE, OD, NULL, 0, STAGE, PRIORITY, MATCH, \ ACTIONS, IN_OUT_PORT, CTRL_METER, STAGE_HINT, \ OVS_SOURCE_LOCATOR, LFLOW_REF) #define ovn_lflow_add_with_hint(LFLOW_TABLE, OD, STAGE, PRIORITY, MATCH, \ - ACTIONS, STAGE_HINT) \ - lflow_table_add_lflow(LFLOW_TABLE, OD, NULL, 0, STAGE, PRIORITY, MATCH, \ - ACTIONS, NULL, NULL, STAGE_HINT, \ - OVS_SOURCE_LOCATOR, NULL) - -#define ovn_lflow_add_with_lflow_ref_hint(LFLOW_TABLE, OD, STAGE, PRIORITY, \ - MATCH, ACTIONS, STAGE_HINT, \ - LFLOW_REF) \ + ACTIONS, STAGE_HINT, LFLOW_REF) \ lflow_table_add_lflow(LFLOW_TABLE, OD, NULL, 0, STAGE, PRIORITY, MATCH, \ ACTIONS, NULL, NULL, STAGE_HINT, \ OVS_SOURCE_LOCATOR, LFLOW_REF) #define ovn_lflow_add_with_dp_group(LFLOW_TABLE, DP_BITMAP, DP_BITMAP_LEN, \ STAGE, PRIORITY, MATCH, ACTIONS, \ - STAGE_HINT) \ + STAGE_HINT, LFLOW_REF) \ lflow_table_add_lflow(LFLOW_TABLE, NULL, DP_BITMAP, DP_BITMAP_LEN, STAGE, \ PRIORITY, MATCH, ACTIONS, NULL, NULL, STAGE_HINT, \ - OVS_SOURCE_LOCATOR, NULL) + OVS_SOURCE_LOCATOR, LFLOW_REF) -#define ovn_lflow_add_default_drop(LFLOW_TABLE, OD, STAGE) \ +#define ovn_lflow_add_default_drop(LFLOW_TABLE, OD, STAGE, LFLOW_REF) \ lflow_table_add_lflow_default_drop(LFLOW_TABLE, OD, STAGE, \ - OVS_SOURCE_LOCATOR, NULL) + OVS_SOURCE_LOCATOR, LFLOW_REF) /* This macro is similar to ovn_lflow_add_with_hint, except that it requires @@ -142,20 +128,16 @@ void lflow_table_add_lflow_default_drop(struct lflow_table *, ACTIONS, IN_OUT_PORT, NULL, STAGE_HINT, \ OVS_SOURCE_LOCATOR, LFLOW_REF) -#define ovn_lflow_add(LFLOW_TABLE, OD, STAGE, PRIORITY, MATCH, ACTIONS) \ - lflow_table_add_lflow(LFLOW_TABLE, OD, NULL, 0, STAGE, PRIORITY, MATCH, \ - ACTIONS, NULL, NULL, NULL, OVS_SOURCE_LOCATOR, NULL) - -#define ovn_lflow_add_with_lflow_ref(LFLOW_TABLE, OD, STAGE, PRIORITY, MATCH, \ - ACTIONS, LFLOW_REF) \ +#define ovn_lflow_add(LFLOW_TABLE, OD, STAGE, PRIORITY, MATCH, ACTIONS, \ + LFLOW_REF) \ lflow_table_add_lflow(LFLOW_TABLE, OD, NULL, 0, STAGE, PRIORITY, MATCH, \ ACTIONS, NULL, NULL, NULL, OVS_SOURCE_LOCATOR, \ LFLOW_REF) #define ovn_lflow_metered(LFLOW_TABLE, OD, STAGE, PRIORITY, MATCH, ACTIONS, \ - CTRL_METER) \ + CTRL_METER, LFLOW_REF) \ ovn_lflow_add_with_hint__(LFLOW_TABLE, OD, STAGE, PRIORITY, MATCH, \ - ACTIONS, NULL, CTRL_METER, NULL) + ACTIONS, NULL, CTRL_METER, NULL, LFLOW_REF) struct sbrec_logical_dp_group; diff --git a/northd/northd.c b/northd/northd.c index f41df83c40..a9cb0b6267 100644 --- a/northd/northd.c +++ b/northd/northd.c @@ -6068,13 +6068,16 @@ build_lswitch_learn_fdb_op( static void build_lswitch_learn_fdb_od( - struct ovn_datapath *od, struct lflow_table *lflows) + struct ovn_datapath *od, struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { ovs_assert(od->nbs); - ovn_lflow_add(lflows, od, S_SWITCH_IN_LOOKUP_FDB, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_SWITCH_IN_PUT_FDB, 0, "1", "next;"); + ovn_lflow_add(lflows, od, S_SWITCH_IN_LOOKUP_FDB, 0, "1", "next;", + lflow_ref); + ovn_lflow_add(lflows, od, S_SWITCH_IN_PUT_FDB, 0, "1", "next;", + lflow_ref); ovn_lflow_add(lflows, od, S_SWITCH_IN_L2_LKUP, 0, "1", - "outport = get_fdb(eth.dst); next;"); + "outport = get_fdb(eth.dst); next;", lflow_ref); } /* Egress tables 8: Egress port security - IP (priority 0) @@ -6082,25 +6085,30 @@ build_lswitch_learn_fdb_od( * (priority 100). */ static void build_lswitch_output_port_sec_od(struct ovn_datapath *od, - struct lflow_table *lflows) + struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { ovs_assert(od->nbs); ovn_lflow_add(lflows, od, S_SWITCH_OUT_CHECK_PORT_SEC, 100, - "eth.mcast", REGBIT_PORT_SEC_DROP" = 0; next;"); + "eth.mcast", REGBIT_PORT_SEC_DROP" = 0; next;", + lflow_ref); ovn_lflow_add(lflows, od, S_SWITCH_OUT_CHECK_PORT_SEC, 0, "1", - REGBIT_PORT_SEC_DROP" = check_out_port_sec(); next;"); + REGBIT_PORT_SEC_DROP" = check_out_port_sec(); next;", + lflow_ref); ovn_lflow_add(lflows, od, S_SWITCH_OUT_APPLY_PORT_SEC, 50, - REGBIT_PORT_SEC_DROP" == 1", debug_drop_action()); + REGBIT_PORT_SEC_DROP" == 1", debug_drop_action(), + lflow_ref); ovn_lflow_add(lflows, od, S_SWITCH_OUT_APPLY_PORT_SEC, 0, - "1", "output;"); + "1", "output;", lflow_ref); } static void skip_port_from_conntrack(const struct ovn_datapath *od, struct ovn_port *op, bool has_stateful_acl, enum ovn_stage in_stage, enum ovn_stage out_stage, uint16_t priority, - struct lflow_table *lflows) + struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { /* Can't use ct() for router ports. Consider the following configuration: * lp1(10.0.0.2) on hostA--ls1--lr0--ls2--lp2(10.0.1.2) on hostB, For a @@ -6122,10 +6130,10 @@ skip_port_from_conntrack(const struct ovn_datapath *od, struct ovn_port *op, ovn_lflow_add_with_lport_and_hint(lflows, od, in_stage, priority, ingress_match, ingress_action, - op->key, &op->nbsp->header_, NULL); + op->key, &op->nbsp->header_, lflow_ref); ovn_lflow_add_with_lport_and_hint(lflows, od, out_stage, priority, egress_match, egress_action, - op->key, &op->nbsp->header_, NULL); + op->key, &op->nbsp->header_, lflow_ref); free(ingress_match); free(egress_match); @@ -6134,7 +6142,8 @@ skip_port_from_conntrack(const struct ovn_datapath *od, struct ovn_port *op, static void build_stateless_filter(const struct ovn_datapath *od, const struct nbrec_acl *acl, - struct lflow_table *lflows) + struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { const char *action = REGBIT_ACL_STATELESS" = 1; next;"; if (!strcmp(acl->direction, "from-lport")) { @@ -6142,25 +6151,28 @@ build_stateless_filter(const struct ovn_datapath *od, acl->priority + OVN_ACL_PRI_OFFSET, acl->match, action, - &acl->header_); + &acl->header_, + lflow_ref); } else { ovn_lflow_add_with_hint(lflows, od, S_SWITCH_OUT_PRE_ACL, acl->priority + OVN_ACL_PRI_OFFSET, acl->match, action, - &acl->header_); + &acl->header_, + lflow_ref); } } static void build_stateless_filters(const struct ovn_datapath *od, const struct ls_port_group_table *ls_port_groups, - struct lflow_table *lflows) + struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { for (size_t i = 0; i < od->nbs->n_acls; i++) { const struct nbrec_acl *acl = od->nbs->acls[i]; if (!strcmp(acl->action, "allow-stateless")) { - build_stateless_filter(od, acl, lflows); + build_stateless_filter(od, acl, lflows, lflow_ref); } } @@ -6176,32 +6188,38 @@ build_stateless_filters(const struct ovn_datapath *od, const struct nbrec_acl *acl = ls_pg_rec->nb_pg->acls[i]; if (!strcmp(acl->action, "allow-stateless")) { - build_stateless_filter(od, acl, lflows); + build_stateless_filter(od, acl, lflows, lflow_ref); } } } } static void -build_pre_acls(struct ovn_datapath *od, struct lflow_table *lflows) +build_pre_acls(struct ovn_datapath *od, struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { /* Ingress and Egress Pre-ACL Table (Priority 0): Packets are * allowed by default. */ - ovn_lflow_add(lflows, od, S_SWITCH_IN_PRE_ACL, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_SWITCH_OUT_PRE_ACL, 0, "1", "next;"); + ovn_lflow_add(lflows, od, S_SWITCH_IN_PRE_ACL, 0, "1", "next;", + lflow_ref); + ovn_lflow_add(lflows, od, S_SWITCH_OUT_PRE_ACL, 0, "1", "next;", + lflow_ref); ovn_lflow_add(lflows, od, S_SWITCH_IN_PRE_ACL, 110, - "eth.dst == $svc_monitor_mac", "next;"); + "eth.dst == $svc_monitor_mac", "next;", + lflow_ref); ovn_lflow_add(lflows, od, S_SWITCH_OUT_PRE_ACL, 110, - "eth.src == $svc_monitor_mac", "next;"); + "eth.src == $svc_monitor_mac", "next;", + lflow_ref); } static void build_ls_stateful_rec_pre_acls( const struct ls_stateful_record *ls_stateful_rec, const struct ls_port_group_table *ls_port_groups, - struct lflow_table *lflows) + struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { const struct ovn_datapath *od = ls_stateful_rec->od; @@ -6216,17 +6234,17 @@ build_ls_stateful_rec_pre_acls( } skip_port_from_conntrack(od, op, true, S_SWITCH_IN_PRE_ACL, S_SWITCH_OUT_PRE_ACL, - 110, lflows); + 110, lflows, lflow_ref); } for (size_t i = 0; i < od->n_localnet_ports; i++) { skip_port_from_conntrack(od, od->localnet_ports[i], true, S_SWITCH_IN_PRE_ACL, S_SWITCH_OUT_PRE_ACL, - 110, lflows); + 110, lflows, lflow_ref); } /* stateless filters always take precedence over stateful ACLs. */ - build_stateless_filters(od, ls_port_groups, lflows); + build_stateless_filters(od, ls_port_groups, lflows, lflow_ref); /* Ingress and Egress Pre-ACL Table (Priority 110). * @@ -6234,16 +6252,18 @@ build_ls_stateful_rec_pre_acls( * unreachable packets. */ ovn_lflow_add(lflows, od, S_SWITCH_IN_PRE_ACL, 110, "nd || nd_rs || nd_ra || mldv1 || mldv2 || " - "(udp && udp.src == 546 && udp.dst == 547)", "next;"); + "(udp && udp.src == 546 && udp.dst == 547)", "next;", + lflow_ref); ovn_lflow_add(lflows, od, S_SWITCH_OUT_PRE_ACL, 110, "nd || nd_rs || nd_ra || mldv1 || mldv2 || " - "(udp && udp.src == 546 && udp.dst == 547)", "next;"); + "(udp && udp.src == 546 && udp.dst == 547)", "next;", + lflow_ref); /* Do not send multicast packets to conntrack. */ ovn_lflow_add(lflows, od, S_SWITCH_IN_PRE_ACL, 110, "eth.mcast", - "next;"); + "next;", lflow_ref); ovn_lflow_add(lflows, od, S_SWITCH_OUT_PRE_ACL, 110, "eth.mcast", - "next;"); + "next;", lflow_ref); /* Ingress and Egress Pre-ACL Table (Priority 100). * @@ -6254,13 +6274,15 @@ build_ls_stateful_rec_pre_acls( * 'REGBIT_CONNTRACK_DEFRAG' is set to let the pre-stateful table send * it to conntrack for tracking and defragmentation. */ ovn_lflow_add(lflows, od, S_SWITCH_IN_PRE_ACL, 100, "ip", - REGBIT_CONNTRACK_DEFRAG" = 1; next;"); + REGBIT_CONNTRACK_DEFRAG" = 1; next;", + lflow_ref); ovn_lflow_add(lflows, od, S_SWITCH_OUT_PRE_ACL, 100, "ip", - REGBIT_CONNTRACK_DEFRAG" = 1; next;"); + REGBIT_CONNTRACK_DEFRAG" = 1; next;", + lflow_ref); } else if (ls_stateful_rec->has_lb_vip) { /* We'll build stateless filters if there are LB rules so that * the stateless flows are not tracked in pre-lb. */ - build_stateless_filters(od, ls_port_groups, lflows); + build_stateless_filters(od, ls_port_groups, lflows, lflow_ref); } } @@ -6324,7 +6346,8 @@ build_empty_lb_event_flow(struct ovn_lb_vip *lb_vip, static void build_interconn_mcast_snoop_flows(struct ovn_datapath *od, const struct shash *meter_groups, - struct lflow_table *lflows) + struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { struct mcast_switch_info *mcast_sw_info = &od->mcast_info.sw; if (!mcast_sw_info->enabled @@ -6343,7 +6366,8 @@ build_interconn_mcast_snoop_flows(struct ovn_datapath *od, ovn_lflow_metered(lflows, od, S_SWITCH_OUT_PRE_LB, 120, match, "clone { igmp; }; next;", copp_meter_get(COPP_IGMP, od->nbs->copp, - meter_groups)); + meter_groups), + lflow_ref); free(match); /* Punt MLD traffic to controller. */ @@ -6351,50 +6375,54 @@ build_interconn_mcast_snoop_flows(struct ovn_datapath *od, ovn_lflow_metered(lflows, od, S_SWITCH_OUT_PRE_LB, 120, match, "clone { igmp; }; next;", copp_meter_get(COPP_IGMP, od->nbs->copp, - meter_groups)); + meter_groups), + lflow_ref); free(match); } } static void build_pre_lb(struct ovn_datapath *od, const struct shash *meter_groups, - struct lflow_table *lflows) + struct lflow_table *lflows, struct lflow_ref *lflow_ref) { /* Handle IGMP/MLD packets crossing AZs. */ - build_interconn_mcast_snoop_flows(od, meter_groups, lflows); + build_interconn_mcast_snoop_flows(od, meter_groups, lflows, lflow_ref); /* Do not send multicast packets to conntrack */ - ovn_lflow_add(lflows, od, S_SWITCH_IN_PRE_LB, 110, "eth.mcast", "next;"); - ovn_lflow_add(lflows, od, S_SWITCH_OUT_PRE_LB, 110, "eth.mcast", "next;"); + ovn_lflow_add(lflows, od, S_SWITCH_IN_PRE_LB, 110, "eth.mcast", "next;", + lflow_ref); + ovn_lflow_add(lflows, od, S_SWITCH_OUT_PRE_LB, 110, "eth.mcast", "next;", + lflow_ref); /* Do not send ND packets to conntrack */ ovn_lflow_add(lflows, od, S_SWITCH_IN_PRE_LB, 110, "nd || nd_rs || nd_ra || mldv1 || mldv2", - "next;"); + "next;", lflow_ref); ovn_lflow_add(lflows, od, S_SWITCH_OUT_PRE_LB, 110, "nd || nd_rs || nd_ra || mldv1 || mldv2", - "next;"); + "next;", lflow_ref); /* Do not send service monitor packets to conntrack. */ ovn_lflow_add(lflows, od, S_SWITCH_IN_PRE_LB, 110, - "eth.dst == $svc_monitor_mac", "next;"); + "eth.dst == $svc_monitor_mac", "next;", lflow_ref); ovn_lflow_add(lflows, od, S_SWITCH_OUT_PRE_LB, 110, - "eth.src == $svc_monitor_mac", "next;"); + "eth.src == $svc_monitor_mac", "next;", lflow_ref); /* Allow all packets to go to next tables by default. */ - ovn_lflow_add(lflows, od, S_SWITCH_IN_PRE_LB, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_SWITCH_OUT_PRE_LB, 0, "1", "next;"); + ovn_lflow_add(lflows, od, S_SWITCH_IN_PRE_LB, 0, "1", "next;", lflow_ref); + ovn_lflow_add(lflows, od, S_SWITCH_OUT_PRE_LB, 0, "1", "next;", lflow_ref); /* Do not send statless flows via conntrack */ ovn_lflow_add(lflows, od, S_SWITCH_IN_PRE_LB, 110, - REGBIT_ACL_STATELESS" == 1", "next;"); + REGBIT_ACL_STATELESS" == 1", "next;", lflow_ref); ovn_lflow_add(lflows, od, S_SWITCH_OUT_PRE_LB, 110, - REGBIT_ACL_STATELESS" == 1", "next;"); + REGBIT_ACL_STATELESS" == 1", "next;", lflow_ref); } static void build_ls_stateful_rec_pre_lb(const struct ls_stateful_record *ls_stateful_rec, - struct lflow_table *lflows) + struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { const struct ovn_datapath *od = ls_stateful_rec->od; @@ -6402,7 +6430,7 @@ build_ls_stateful_rec_pre_lb(const struct ls_stateful_record *ls_stateful_rec, skip_port_from_conntrack(od, od->router_ports[i], ls_stateful_rec->has_stateful_acl, S_SWITCH_IN_PRE_LB, S_SWITCH_OUT_PRE_LB, - 110, lflows); + 110, lflows, lflow_ref); } /* Localnet ports have no need for going through conntrack, unless @@ -6415,7 +6443,7 @@ build_ls_stateful_rec_pre_lb(const struct ls_stateful_record *ls_stateful_rec, skip_port_from_conntrack(od, od->localnet_ports[i], ls_stateful_rec->has_stateful_acl, S_SWITCH_IN_PRE_LB, S_SWITCH_OUT_PRE_LB, - 110, lflows); + 110, lflows, lflow_ref); } } @@ -6451,21 +6479,26 @@ build_ls_stateful_rec_pre_lb(const struct ls_stateful_record *ls_stateful_rec, */ if (ls_stateful_rec->has_lb_vip) { ovn_lflow_add(lflows, od, S_SWITCH_IN_PRE_LB, - 100, "ip", REGBIT_CONNTRACK_NAT" = 1; next;"); + 100, "ip", REGBIT_CONNTRACK_NAT" = 1; next;", + lflow_ref); ovn_lflow_add(lflows, od, S_SWITCH_OUT_PRE_LB, - 100, "ip", REGBIT_CONNTRACK_NAT" = 1; next;"); + 100, "ip", REGBIT_CONNTRACK_NAT" = 1; next;", + lflow_ref); } } static void build_pre_stateful(struct ovn_datapath *od, const struct chassis_features *features, - struct lflow_table *lflows) + struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { /* Ingress and Egress pre-stateful Table (Priority 0): Packets are * allowed by default. */ - ovn_lflow_add(lflows, od, S_SWITCH_IN_PRE_STATEFUL, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_SWITCH_OUT_PRE_STATEFUL, 0, "1", "next;"); + ovn_lflow_add(lflows, od, S_SWITCH_IN_PRE_STATEFUL, 0, "1", "next;", + lflow_ref); + ovn_lflow_add(lflows, od, S_SWITCH_OUT_PRE_STATEFUL, 0, "1", "next;", + lflow_ref); /* Note: priority-120 flows are added in build_lb_rules_pre_stateful(). */ @@ -6474,25 +6507,30 @@ build_pre_stateful(struct ovn_datapath *od, : "ct_lb;"; ovn_lflow_add(lflows, od, S_SWITCH_IN_PRE_STATEFUL, 110, - REGBIT_CONNTRACK_NAT" == 1", ct_lb_action); + REGBIT_CONNTRACK_NAT" == 1", ct_lb_action, + lflow_ref); ovn_lflow_add(lflows, od, S_SWITCH_OUT_PRE_STATEFUL, 110, - REGBIT_CONNTRACK_NAT" == 1", ct_lb_action); + REGBIT_CONNTRACK_NAT" == 1", ct_lb_action, + lflow_ref); /* If REGBIT_CONNTRACK_DEFRAG is set as 1, then the packets should be * sent to conntrack for tracking and defragmentation. */ ovn_lflow_add(lflows, od, S_SWITCH_IN_PRE_STATEFUL, 100, - REGBIT_CONNTRACK_DEFRAG" == 1", "ct_next;"); + REGBIT_CONNTRACK_DEFRAG" == 1", "ct_next;", + lflow_ref); ovn_lflow_add(lflows, od, S_SWITCH_OUT_PRE_STATEFUL, 100, - REGBIT_CONNTRACK_DEFRAG" == 1", "ct_next;"); + REGBIT_CONNTRACK_DEFRAG" == 1", "ct_next;", + lflow_ref); } static void build_acl_hints(const struct ls_stateful_record *ls_stateful_rec, const struct chassis_features *features, - struct lflow_table *lflows) + struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { const struct ovn_datapath *od = ls_stateful_rec->od; @@ -6520,9 +6558,10 @@ build_acl_hints(const struct ls_stateful_record *ls_stateful_rec, /* In any case, advance to the next stage. */ if (!ls_stateful_rec->has_acls && !ls_stateful_rec->has_lb_vip) { - ovn_lflow_add(lflows, od, stage, UINT16_MAX, "1", "next;"); + ovn_lflow_add(lflows, od, stage, UINT16_MAX, "1", "next;", + lflow_ref); } else { - ovn_lflow_add(lflows, od, stage, 0, "1", "next;"); + ovn_lflow_add(lflows, od, stage, 0, "1", "next;", lflow_ref); } if (!ls_stateful_rec->has_stateful_acl && @@ -6537,7 +6576,7 @@ build_acl_hints(const struct ls_stateful_record *ls_stateful_rec, ovn_lflow_add(lflows, od, stage, 7, "ct.new && !ct.est", REGBIT_ACL_HINT_ALLOW_NEW " = 1; " REGBIT_ACL_HINT_DROP " = 1; " - "next;"); + "next;", lflow_ref); /* Already established connections in the "request" direction that * are already marked as "blocked" may hit either: @@ -6553,13 +6592,13 @@ build_acl_hints(const struct ls_stateful_record *ls_stateful_rec, ovn_lflow_add(lflows, od, stage, 6, match, REGBIT_ACL_HINT_ALLOW_NEW " = 1; " REGBIT_ACL_HINT_DROP " = 1; " - "next;"); + "next;", lflow_ref); /* Not tracked traffic can either be allowed or dropped. */ ovn_lflow_add(lflows, od, stage, 5, "!ct.trk", REGBIT_ACL_HINT_ALLOW " = 1; " REGBIT_ACL_HINT_DROP " = 1; " - "next;"); + "next;", lflow_ref); /* Already established connections in the "request" direction may hit * either: @@ -6575,20 +6614,20 @@ build_acl_hints(const struct ls_stateful_record *ls_stateful_rec, ovn_lflow_add(lflows, od, stage, 4, match, REGBIT_ACL_HINT_ALLOW " = 1; " REGBIT_ACL_HINT_BLOCK " = 1; " - "next;"); + "next;", lflow_ref); /* Not established or established and already blocked connections may * hit drop ACLs. */ ovn_lflow_add(lflows, od, stage, 3, "!ct.est", REGBIT_ACL_HINT_DROP " = 1; " - "next;"); + "next;", lflow_ref); match = features->ct_no_masked_label ? "ct.est && ct_mark.blocked == 1" : "ct.est && ct_label.blocked == 1"; ovn_lflow_add(lflows, od, stage, 2, match, REGBIT_ACL_HINT_DROP " = 1; " - "next;"); + "next;", lflow_ref); /* Established connections that were previously allowed might hit * drop ACLs in which case the connection must be committed with @@ -6599,7 +6638,7 @@ build_acl_hints(const struct ls_stateful_record *ls_stateful_rec, : "ct.est && ct_label.blocked == 0"; ovn_lflow_add(lflows, od, stage, 1, match, REGBIT_ACL_HINT_BLOCK " = 1; " - "next;"); + "next;", lflow_ref); } } @@ -6665,7 +6704,8 @@ static void consider_acl(struct lflow_table *lflows, const struct ovn_datapath *od, const struct nbrec_acl *acl, bool has_stateful, bool ct_masked_mark, const struct shash *meter_groups, - uint64_t max_acl_tier, struct ds *match, struct ds *actions) + uint64_t max_acl_tier, struct ds *match, struct ds *actions, + struct lflow_ref *lflow_ref) { const char *ct_blocked_match = ct_masked_mark ? "ct_mark.blocked" @@ -6714,7 +6754,7 @@ consider_acl(struct lflow_table *lflows, const struct ovn_datapath *od, ds_put_format(match, "(%s)", acl->match); ovn_lflow_add_with_hint(lflows, od, stage, priority, ds_cstr(match), ds_cstr(actions), - &acl->header_); + &acl->header_, lflow_ref); return; } @@ -6751,7 +6791,7 @@ consider_acl(struct lflow_table *lflows, const struct ovn_datapath *od, ds_put_cstr(actions, "next;"); ovn_lflow_add_with_hint(lflows, od, stage, priority, ds_cstr(match), ds_cstr(actions), - &acl->header_); + &acl->header_, lflow_ref); /* Match on traffic in the request direction for an established * connection tracking entry that has not been marked for @@ -6773,7 +6813,7 @@ consider_acl(struct lflow_table *lflows, const struct ovn_datapath *od, ds_put_cstr(actions, "next;"); ovn_lflow_add_with_hint(lflows, od, stage, priority, ds_cstr(match), ds_cstr(actions), - &acl->header_); + &acl->header_, lflow_ref); } else if (!strcmp(acl->action, "drop") || !strcmp(acl->action, "reject")) { /* The implementation of "drop" differs if stateful ACLs are in @@ -6790,7 +6830,7 @@ consider_acl(struct lflow_table *lflows, const struct ovn_datapath *od, ds_put_cstr(actions, "next;"); ovn_lflow_add_with_hint(lflows, od, stage, priority, ds_cstr(match), ds_cstr(actions), - &acl->header_); + &acl->header_, lflow_ref); /* For an existing connection without ct_mark.blocked set, we've * encountered a policy change. ACLs previously allowed * this connection and we committed the connection tracking @@ -6811,7 +6851,7 @@ consider_acl(struct lflow_table *lflows, const struct ovn_datapath *od, ct_blocked_match); ovn_lflow_add_with_hint(lflows, od, stage, priority, ds_cstr(match), ds_cstr(actions), - &acl->header_); + &acl->header_, lflow_ref); } } @@ -6894,7 +6934,8 @@ build_acl_action_lflows(const struct ls_stateful_record *ls_stateful_rec, const char *default_acl_action, const struct shash *meter_groups, struct ds *match, - struct ds *actions) + struct ds *actions, + struct lflow_ref *lflow_ref) { const struct ovn_datapath *od = ls_stateful_rec->od; @@ -6917,18 +6958,20 @@ build_acl_action_lflows(const struct ls_stateful_record *ls_stateful_rec, for (size_t i = 0; i < ARRAY_SIZE(stages); i++) { enum ovn_stage stage = stages[i]; if (!ls_stateful_rec->has_acls) { - ovn_lflow_add(lflows, od, stage, 0, "1", "next;"); + ovn_lflow_add(lflows, od, stage, 0, "1", "next;", lflow_ref); continue; } ds_truncate(actions, verdict_len); ds_put_cstr(actions, "next;"); ovn_lflow_add(lflows, od, stage, 1000, - REGBIT_ACL_VERDICT_ALLOW " == 1", ds_cstr(actions)); + REGBIT_ACL_VERDICT_ALLOW " == 1", ds_cstr(actions), + lflow_ref); ds_truncate(actions, verdict_len); ds_put_cstr(actions, debug_implicit_drop_action()); ovn_lflow_add(lflows, od, stage, 1000, REGBIT_ACL_VERDICT_DROP " == 1", - ds_cstr(actions)); + ds_cstr(actions), + lflow_ref); bool ingress = ovn_stage_get_pipeline(stage) == P_IN; ds_truncate(actions, verdict_len); @@ -6944,11 +6987,11 @@ build_acl_action_lflows(const struct ls_stateful_record *ls_stateful_rec, ovn_lflow_metered(lflows, od, stage, 1000, REGBIT_ACL_VERDICT_REJECT " == 1", ds_cstr(actions), copp_meter_get(COPP_REJECT, od->nbs->copp, - meter_groups)); + meter_groups), lflow_ref); ds_truncate(actions, verdict_len); ds_put_cstr(actions, default_acl_action); - ovn_lflow_add(lflows, od, stage, 0, "1", ds_cstr(actions)); + ovn_lflow_add(lflows, od, stage, 0, "1", ds_cstr(actions), lflow_ref); struct ds tier_actions = DS_EMPTY_INITIALIZER; for (size_t j = 0; j < ls_stateful_rec->max_acl_tier; j++) { @@ -6960,7 +7003,7 @@ build_acl_action_lflows(const struct ls_stateful_record *ls_stateful_rec, j + 1, ingress ? "ingress" : "egress", ovn_stage_get_table(stage) - 1); ovn_lflow_add(lflows, od, stage, 500, ds_cstr(match), - ds_cstr(&tier_actions)); + ds_cstr(&tier_actions), lflow_ref); } ds_destroy(&tier_actions); } @@ -6972,7 +7015,8 @@ build_acl_log_related_flows(const struct ovn_datapath *od, const struct nbrec_acl *acl, bool has_stateful, bool ct_masked_mark, const struct shash *meter_groups, - struct ds *match, struct ds *actions) + struct ds *match, struct ds *actions, + struct lflow_ref *lflow_ref) { /* Related and reply traffic are universally allowed by priority * 65532 flows created in build_acls(). If logging is enabled on @@ -7026,7 +7070,7 @@ build_acl_log_related_flows(const struct ovn_datapath *od, ovn_lflow_add_with_hint(lflows, od, log_related_stage, UINT16_MAX - 2, ds_cstr(match), ds_cstr(actions), - &acl->header_); + &acl->header_, lflow_ref); ds_clear(match); ds_put_format(match, "!ct.est && ct.rel && !ct.new%s && " @@ -7037,7 +7081,7 @@ build_acl_log_related_flows(const struct ovn_datapath *od, ovn_lflow_add_with_hint(lflows, od, log_related_stage, UINT16_MAX - 2, ds_cstr(match), ds_cstr(actions), - &acl->header_); + &acl->header_, lflow_ref); } static void @@ -7045,7 +7089,8 @@ build_acls(const struct ls_stateful_record *ls_stateful_rec, const struct chassis_features *features, struct lflow_table *lflows, const struct ls_port_group_table *ls_port_groups, - const struct shash *meter_groups) + const struct shash *meter_groups, + struct lflow_ref *lflow_ref) { const struct ovn_datapath *od = ls_stateful_rec->od; @@ -7070,20 +7115,24 @@ build_acls(const struct ls_stateful_record *ls_stateful_rec, if (!ls_stateful_rec->has_acls) { if (!ls_stateful_rec->has_lb_vip) { ovn_lflow_add(lflows, od, S_SWITCH_IN_ACL_EVAL, UINT16_MAX, "1", - "next;"); + "next;", lflow_ref); ovn_lflow_add(lflows, od, S_SWITCH_OUT_ACL_EVAL, UINT16_MAX, "1", - "next;"); + "next;", lflow_ref); } else { - ovn_lflow_add(lflows, od, S_SWITCH_IN_ACL_EVAL, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_SWITCH_OUT_ACL_EVAL, 0, "1", "next;"); + ovn_lflow_add(lflows, od, S_SWITCH_IN_ACL_EVAL, 0, "1", "next;", + lflow_ref); + ovn_lflow_add(lflows, od, S_SWITCH_OUT_ACL_EVAL, 0, "1", "next;", + lflow_ref); } ovn_lflow_add(lflows, od, S_SWITCH_IN_ACL_AFTER_LB_EVAL, 0, "1", - "next;"); + "next;", lflow_ref); } else { - ovn_lflow_add(lflows, od, S_SWITCH_IN_ACL_EVAL, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_SWITCH_OUT_ACL_EVAL, 0, "1", "next;"); + ovn_lflow_add(lflows, od, S_SWITCH_IN_ACL_EVAL, 0, "1", "next;", + lflow_ref); + ovn_lflow_add(lflows, od, S_SWITCH_OUT_ACL_EVAL, 0, "1", "next;", + lflow_ref); ovn_lflow_add(lflows, od, S_SWITCH_IN_ACL_AFTER_LB_EVAL, 0, "1", - "next;"); + "next;", lflow_ref); } @@ -7115,19 +7164,21 @@ build_acls(const struct ls_stateful_record *ls_stateful_rec, ovn_lflow_add(lflows, od, S_SWITCH_IN_ACL_EVAL, 1, ds_cstr(&match), REGBIT_CONNTRACK_COMMIT" = 1; " - REGBIT_ACL_VERDICT_ALLOW" = 1; next;"); + REGBIT_ACL_VERDICT_ALLOW" = 1; next;", + lflow_ref); ovn_lflow_add(lflows, od, S_SWITCH_OUT_ACL_EVAL, 1, ds_cstr(&match), REGBIT_CONNTRACK_COMMIT" = 1; " - REGBIT_ACL_VERDICT_ALLOW" = 1; next;"); + REGBIT_ACL_VERDICT_ALLOW" = 1; next;", + lflow_ref); const char *next_action = default_acl_drop ? "next;" : REGBIT_CONNTRACK_COMMIT" = 1; next;"; ovn_lflow_add(lflows, od, S_SWITCH_IN_ACL_EVAL, 1, "ip && !ct.est", - next_action); + next_action, lflow_ref); ovn_lflow_add(lflows, od, S_SWITCH_OUT_ACL_EVAL, 1, "ip && !ct.est", - next_action); + next_action, lflow_ref); /* Ingress and Egress ACL Table (Priority 65532). * @@ -7141,9 +7192,11 @@ build_acls(const struct ls_stateful_record *ls_stateful_rec, use_ct_inv_match ? "ct.inv || " : "", ct_blocked_match); ovn_lflow_add(lflows, od, S_SWITCH_IN_ACL_EVAL, UINT16_MAX - 3, - ds_cstr(&match), REGBIT_ACL_VERDICT_DROP " = 1; next;"); + ds_cstr(&match), REGBIT_ACL_VERDICT_DROP " = 1; next;", + lflow_ref); ovn_lflow_add(lflows, od, S_SWITCH_OUT_ACL_EVAL, UINT16_MAX - 3, - ds_cstr(&match), REGBIT_ACL_VERDICT_DROP " = 1; next;"); + ds_cstr(&match), REGBIT_ACL_VERDICT_DROP " = 1; next;", + lflow_ref); /* Ingress and Egress ACL Table (Priority 65535 - 3). * @@ -7163,10 +7216,12 @@ build_acls(const struct ls_stateful_record *ls_stateful_rec, ds_cstr(&match), REGBIT_ACL_HINT_DROP" = 0; " REGBIT_ACL_HINT_BLOCK" = 0; " REGBIT_ACL_HINT_ALLOW_REL" = 1; " - REGBIT_ACL_VERDICT_ALLOW" = 1; next;"); + REGBIT_ACL_VERDICT_ALLOW" = 1; next;", + lflow_ref); ovn_lflow_add(lflows, od, S_SWITCH_OUT_ACL_EVAL, UINT16_MAX - 3, ds_cstr(&match), - REGBIT_ACL_VERDICT_ALLOW " = 1; next;"); + REGBIT_ACL_VERDICT_ALLOW " = 1; next;", + lflow_ref); /* Ingress and Egress ACL Table (Priority 65535). * @@ -7195,15 +7250,16 @@ build_acls(const struct ls_stateful_record *ls_stateful_rec, use_ct_inv_match ? " && !ct.inv" : "", ct_blocked_match); ovn_lflow_add(lflows, od, S_SWITCH_IN_ACL_EVAL, UINT16_MAX - 3, - ds_cstr(&match), ct_in_acl_action); + ds_cstr(&match), ct_in_acl_action, lflow_ref); ovn_lflow_add(lflows, od, S_SWITCH_OUT_ACL_EVAL, UINT16_MAX - 3, - ds_cstr(&match), ct_out_acl_action); + ds_cstr(&match), ct_out_acl_action, lflow_ref); /* Reply and related traffic matched by an "allow-related" ACL * should be allowed in the ls_in_acl_after_lb stage too. */ ovn_lflow_add(lflows, od, S_SWITCH_IN_ACL_AFTER_LB_EVAL, UINT16_MAX - 3, REGBIT_ACL_HINT_ALLOW_REL" == 1", - REGBIT_ACL_VERDICT_ALLOW " = 1; next;"); + REGBIT_ACL_VERDICT_ALLOW " = 1; next;", + lflow_ref); } /* Ingress and Egress ACL Table (Priority 65532). @@ -7215,24 +7271,28 @@ build_acls(const struct ls_stateful_record *ls_stateful_rec, * https://bugzilla.kernel.org/show_bug.cgi?id=11797. */ ovn_lflow_add(lflows, od, S_SWITCH_IN_ACL_EVAL, UINT16_MAX - 3, IPV6_CT_OMIT_MATCH, - REGBIT_ACL_VERDICT_ALLOW " = 1; next;"); + REGBIT_ACL_VERDICT_ALLOW " = 1; next;", + lflow_ref); ovn_lflow_add(lflows, od, S_SWITCH_OUT_ACL_EVAL, UINT16_MAX - 3, IPV6_CT_OMIT_MATCH, - REGBIT_ACL_VERDICT_ALLOW " = 1; next;"); + REGBIT_ACL_VERDICT_ALLOW " = 1; next;", + lflow_ref); ovn_lflow_add(lflows, od, S_SWITCH_IN_ACL_AFTER_LB_EVAL, UINT16_MAX - 3, IPV6_CT_OMIT_MATCH, - REGBIT_ACL_VERDICT_ALLOW " = 1; next;"); + REGBIT_ACL_VERDICT_ALLOW " = 1; next;", + lflow_ref); /* Ingress or Egress ACL Table (Various priorities). */ for (size_t i = 0; i < od->nbs->n_acls; i++) { struct nbrec_acl *acl = od->nbs->acls[i]; build_acl_log_related_flows(od, lflows, acl, has_stateful, features->ct_no_masked_label, - meter_groups, &match, &actions); + meter_groups, &match, &actions, + lflow_ref); consider_acl(lflows, od, acl, has_stateful, features->ct_no_masked_label, meter_groups, ls_stateful_rec->max_acl_tier, - &match, &actions); + &match, &actions, lflow_ref); } const struct ls_port_group *ls_pg = @@ -7245,11 +7305,12 @@ build_acls(const struct ls_stateful_record *ls_stateful_rec, build_acl_log_related_flows(od, lflows, acl, has_stateful, features->ct_no_masked_label, - meter_groups, &match, &actions); + meter_groups, &match, &actions, + lflow_ref); consider_acl(lflows, od, acl, has_stateful, features->ct_no_masked_label, meter_groups, ls_stateful_rec->max_acl_tier, - &match, &actions); + &match, &actions, lflow_ref); } } } @@ -7264,7 +7325,7 @@ build_acls(const struct ls_stateful_record *ls_stateful_rec, : REGBIT_ACL_VERDICT_ALLOW" = 1; next;"; ovn_lflow_add( lflows, od, S_SWITCH_OUT_ACL_EVAL, 34000, "udp.src == 53", - dns_actions); + dns_actions, lflow_ref); } if (ls_stateful_rec->has_acls || ls_stateful_rec->has_lb_vip) { @@ -7272,30 +7333,37 @@ build_acls(const struct ls_stateful_record *ls_stateful_rec, * packets to skip applying ingress ACLs. */ ovn_lflow_add(lflows, od, S_SWITCH_IN_ACL_EVAL, 34000, "eth.dst == $svc_monitor_mac", - REGBIT_ACL_VERDICT_ALLOW" = 1; next;"); + REGBIT_ACL_VERDICT_ALLOW" = 1; next;", + lflow_ref); /* Add a 34000 priority flow to advance the service monitor packets * generated by ovn-controller to skip applying egress ACLs. */ ovn_lflow_add(lflows, od, S_SWITCH_OUT_ACL_EVAL, 34000, "eth.src == $svc_monitor_mac", - REGBIT_ACL_VERDICT_ALLOW" = 1; next;"); + REGBIT_ACL_VERDICT_ALLOW" = 1; next;", + lflow_ref); } build_acl_action_lflows(ls_stateful_rec, lflows, default_acl_action, - meter_groups, &match, &actions); + meter_groups, &match, &actions, lflow_ref); ds_destroy(&match); ds_destroy(&actions); } static void -build_qos(struct ovn_datapath *od, struct lflow_table *lflows) { +build_qos(struct ovn_datapath *od, struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { struct ds action = DS_EMPTY_INITIALIZER; - ovn_lflow_add(lflows, od, S_SWITCH_IN_QOS_MARK, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_SWITCH_OUT_QOS_MARK, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_SWITCH_IN_QOS_METER, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_SWITCH_OUT_QOS_METER, 0, "1", "next;"); + ovn_lflow_add(lflows, od, S_SWITCH_IN_QOS_MARK, 0, "1", "next;", + lflow_ref); + ovn_lflow_add(lflows, od, S_SWITCH_OUT_QOS_MARK, 0, "1", "next;", + lflow_ref); + ovn_lflow_add(lflows, od, S_SWITCH_IN_QOS_METER, 0, "1", "next;", + lflow_ref); + ovn_lflow_add(lflows, od, S_SWITCH_OUT_QOS_METER, 0, "1", "next;", + lflow_ref); for (size_t i = 0; i < od->nbs->n_qos_rules; i++) { struct nbrec_qos *qos = od->nbs->qos_rules[i]; @@ -7312,7 +7380,7 @@ build_qos(struct ovn_datapath *od, struct lflow_table *lflows) { ovn_lflow_add_with_hint(lflows, od, stage, qos->priority, qos->match, ds_cstr(&action), - &qos->header_); + &qos->header_, lflow_ref); } } @@ -7343,7 +7411,7 @@ build_qos(struct ovn_datapath *od, struct lflow_table *lflows) { ovn_lflow_add_with_hint(lflows, od, stage, qos->priority, qos->match, ds_cstr(&action), - &qos->header_); + &qos->header_, lflow_ref); } } ds_destroy(&action); @@ -7408,7 +7476,7 @@ build_lb_rules_pre_stateful(struct lflow_table *lflows, ovn_lflow_add_with_dp_group( lflows, lb_dps->nb_ls_map, ods_size(ls_datapaths), S_SWITCH_IN_PRE_STATEFUL, 120, ds_cstr(match), ds_cstr(action), - &lb->nlb->header_); + &lb->nlb->header_, NULL); } } @@ -7456,7 +7524,8 @@ build_lb_affinity_lr_flows(struct lflow_table *lflows, const struct ovn_northd_lb *lb, struct ovn_lb_vip *lb_vip, char *new_lb_match, char *lb_action, const unsigned long *dp_bitmap, - const struct ovn_datapaths *lr_datapaths) + const struct ovn_datapaths *lr_datapaths, + struct lflow_ref *lflow_ref) { if (!lb->affinity_timeout || bitmap_is_all_zeros(dp_bitmap, ods_size(lr_datapaths))) { @@ -7495,7 +7564,8 @@ build_lb_affinity_lr_flows(struct lflow_table *lflows, ovn_lflow_add_with_dp_group( lflows, dp_bitmap, ods_size(lr_datapaths), S_ROUTER_IN_LB_AFF_CHECK, - 100, new_lb_match, ds_cstr(&aff_check_action), &lb->nlb->header_); + 100, new_lb_match, ds_cstr(&aff_check_action), &lb->nlb->header_, + lflow_ref); /* Prepare common part of affinity LB and affinity learn action. */ ds_put_format(&aff_action, "%s = %s; ", reg_vip, lb_vip->vip_str); @@ -7577,12 +7647,14 @@ build_lb_affinity_lr_flows(struct lflow_table *lflows, ovn_lflow_add_with_dp_group( lflows, dp_bitmap, ods_size(lr_datapaths), S_ROUTER_IN_LB_AFF_LEARN, 100, ds_cstr(&aff_match_learn), - ds_cstr(&aff_action_learn), &lb->nlb->header_); + ds_cstr(&aff_action_learn), &lb->nlb->header_, + lflow_ref); /* Use already selected backend within affinity timeslot. */ ovn_lflow_add_with_dp_group( lflows, dp_bitmap, ods_size(lr_datapaths), S_ROUTER_IN_DNAT, 150, - ds_cstr(&aff_match), ds_cstr(&aff_action), &lb->nlb->header_); + ds_cstr(&aff_match), ds_cstr(&aff_action), &lb->nlb->header_, + lflow_ref); ds_truncate(&aff_action, aff_action_len); ds_truncate(&aff_action_learn, aff_action_learn_len); @@ -7642,7 +7714,8 @@ static void build_lb_affinity_ls_flows(struct lflow_table *lflows, struct ovn_lb_datapaths *lb_dps, struct ovn_lb_vip *lb_vip, - const struct ovn_datapaths *ls_datapaths) + const struct ovn_datapaths *ls_datapaths, + struct lflow_ref *lflow_ref) { if (!lb_dps->lb->affinity_timeout || !lb_dps->n_nb_ls) { return; @@ -7670,7 +7743,7 @@ build_lb_affinity_ls_flows(struct lflow_table *lflows, ovn_lflow_add_with_dp_group( lflows, lb_dps->nb_ls_map, ods_size(ls_datapaths), S_SWITCH_IN_LB_AFF_CHECK, 100, ds_cstr(&new_lb_match), aff_check, - &lb_dps->lb->nlb->header_); + &lb_dps->lb->nlb->header_, lflow_ref); ds_destroy(&new_lb_match); struct ds aff_action = DS_EMPTY_INITIALIZER; @@ -7760,13 +7833,14 @@ build_lb_affinity_ls_flows(struct lflow_table *lflows, ovn_lflow_add_with_dp_group( lflows, lb_dps->nb_ls_map, ods_size(ls_datapaths), S_SWITCH_IN_LB_AFF_LEARN, 100, ds_cstr(&aff_match_learn), - ds_cstr(&aff_action_learn), &lb->nlb->header_); + ds_cstr(&aff_action_learn), &lb->nlb->header_, + lflow_ref); /* Use already selected backend within affinity timeslot. */ ovn_lflow_add_with_dp_group( lflows, lb_dps->nb_ls_map, ods_size(ls_datapaths), S_SWITCH_IN_LB, 150, ds_cstr(&aff_match), ds_cstr(&aff_action), - &lb->nlb->header_); + &lb->nlb->header_, lflow_ref); ds_truncate(&aff_action, aff_action_len); ds_truncate(&aff_action_learn, aff_action_learn_len); @@ -7782,20 +7856,26 @@ build_lb_affinity_ls_flows(struct lflow_table *lflows, static void build_lswitch_lb_affinity_default_flows(struct ovn_datapath *od, - struct lflow_table *lflows) + struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { ovs_assert(od->nbs); - ovn_lflow_add(lflows, od, S_SWITCH_IN_LB_AFF_CHECK, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_SWITCH_IN_LB_AFF_LEARN, 0, "1", "next;"); + ovn_lflow_add(lflows, od, S_SWITCH_IN_LB_AFF_CHECK, 0, "1", "next;", + lflow_ref); + ovn_lflow_add(lflows, od, S_SWITCH_IN_LB_AFF_LEARN, 0, "1", "next;", + lflow_ref); } static void build_lrouter_lb_affinity_default_flows(struct ovn_datapath *od, - struct lflow_table *lflows) + struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { ovs_assert(od->nbr); - ovn_lflow_add(lflows, od, S_ROUTER_IN_LB_AFF_CHECK, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_ROUTER_IN_LB_AFF_LEARN, 0, "1", "next;"); + ovn_lflow_add(lflows, od, S_ROUTER_IN_LB_AFF_CHECK, 0, "1", "next;", + lflow_ref); + ovn_lflow_add(lflows, od, S_ROUTER_IN_LB_AFF_LEARN, 0, "1", "next;", + lflow_ref); } static void @@ -7840,7 +7920,8 @@ build_lb_rules(struct lflow_table *lflows, struct ovn_lb_datapaths *lb_dps, priority = 120; } - build_lb_affinity_ls_flows(lflows, lb_dps, lb_vip, ls_datapaths); + build_lb_affinity_ls_flows(lflows, lb_dps, lb_vip, ls_datapaths, + NULL); unsigned long *dp_non_meter = NULL; bool build_non_meter = false; @@ -7863,14 +7944,16 @@ build_lb_rules(struct lflow_table *lflows, struct ovn_lb_datapaths *lb_dps, ovn_lflow_add_with_hint__( lflows, od, S_SWITCH_IN_LB, priority, ds_cstr(match), ds_cstr(action), - NULL, meter, &lb->nlb->header_); + NULL, meter, &lb->nlb->header_, + NULL); } } if (!reject || build_non_meter) { ovn_lflow_add_with_dp_group( lflows, dp_non_meter ? dp_non_meter : lb_dps->nb_ls_map, ods_size(ls_datapaths), S_SWITCH_IN_LB, priority, - ds_cstr(match), ds_cstr(action), &lb->nlb->header_); + ds_cstr(match), ds_cstr(action), &lb->nlb->header_, + NULL); } bitmap_free(dp_non_meter); } @@ -7879,7 +7962,8 @@ build_lb_rules(struct lflow_table *lflows, struct ovn_lb_datapaths *lb_dps, static void build_stateful(struct ovn_datapath *od, const struct chassis_features *features, - struct lflow_table *lflows) + struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { const char *ct_block_action = features->ct_no_masked_label ? "ct_mark.blocked" @@ -7888,9 +7972,11 @@ build_stateful(struct ovn_datapath *od, /* Ingress LB, Ingress and Egress stateful Table (Priority 0): Packets are * allowed by default. */ - ovn_lflow_add(lflows, od, S_SWITCH_IN_LB, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_SWITCH_IN_STATEFUL, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_SWITCH_OUT_STATEFUL, 0, "1", "next;"); + ovn_lflow_add(lflows, od, S_SWITCH_IN_LB, 0, "1", "next;", lflow_ref); + ovn_lflow_add(lflows, od, S_SWITCH_IN_STATEFUL, 0, "1", "next;", + lflow_ref); + ovn_lflow_add(lflows, od, S_SWITCH_OUT_STATEFUL, 0, "1", "next;", + lflow_ref); /* If REGBIT_CONNTRACK_COMMIT is set as 1 and * REGBIT_CONNTRACK_SET_LABEL is set to 1, then the packets should be @@ -7904,11 +7990,13 @@ build_stateful(struct ovn_datapath *od, ovn_lflow_add(lflows, od, S_SWITCH_IN_STATEFUL, 100, REGBIT_CONNTRACK_COMMIT" == 1 && " REGBIT_ACL_LABEL" == 1", - ds_cstr(&actions)); + ds_cstr(&actions), + lflow_ref); ovn_lflow_add(lflows, od, S_SWITCH_OUT_STATEFUL, 100, REGBIT_CONNTRACK_COMMIT" == 1 && " REGBIT_ACL_LABEL" == 1", - ds_cstr(&actions)); + ds_cstr(&actions), + lflow_ref); /* If REGBIT_CONNTRACK_COMMIT is set as 1, then the packets should be * committed to conntrack. We always set ct_mark.blocked to 0 here as @@ -7919,26 +8007,32 @@ build_stateful(struct ovn_datapath *od, ovn_lflow_add(lflows, od, S_SWITCH_IN_STATEFUL, 100, REGBIT_CONNTRACK_COMMIT" == 1 && " REGBIT_ACL_LABEL" == 0", - ds_cstr(&actions)); + ds_cstr(&actions), + lflow_ref); ovn_lflow_add(lflows, od, S_SWITCH_OUT_STATEFUL, 100, REGBIT_CONNTRACK_COMMIT" == 1 && " REGBIT_ACL_LABEL" == 0", - ds_cstr(&actions)); + ds_cstr(&actions), + lflow_ref); ds_destroy(&actions); } static void build_lb_hairpin(const struct ls_stateful_record *ls_stateful_rec, - struct lflow_table *lflows) + struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { const struct ovn_datapath *od = ls_stateful_rec->od; /* Ingress Pre-Hairpin/Nat-Hairpin/Hairpin tabled (Priority 0). * Packets that don't need hairpinning should continue processing. */ - ovn_lflow_add(lflows, od, S_SWITCH_IN_PRE_HAIRPIN, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_SWITCH_IN_NAT_HAIRPIN, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_SWITCH_IN_HAIRPIN, 0, "1", "next;"); + ovn_lflow_add(lflows, od, S_SWITCH_IN_PRE_HAIRPIN, 0, "1", "next;", + lflow_ref); + ovn_lflow_add(lflows, od, S_SWITCH_IN_NAT_HAIRPIN, 0, "1", "next;", + lflow_ref); + ovn_lflow_add(lflows, od, S_SWITCH_IN_HAIRPIN, 0, "1", "next;", + lflow_ref); if (ls_stateful_rec->has_lb_vip) { /* Check if the packet needs to be hairpinned. @@ -7950,7 +8044,8 @@ build_lb_hairpin(const struct ls_stateful_record *ls_stateful_rec, REGBIT_HAIRPIN " = chk_lb_hairpin(); " REGBIT_HAIRPIN_REPLY " = chk_lb_hairpin_reply(); " "next;", - &od->nbs->header_); + &od->nbs->header_, + lflow_ref); /* If packet needs to be hairpinned, snat the src ip with the VIP * for new sessions. */ @@ -7958,7 +8053,8 @@ build_lb_hairpin(const struct ls_stateful_record *ls_stateful_rec, "ip && ct.new && ct.trk" " && "REGBIT_HAIRPIN " == 1", "ct_snat_to_vip; next;", - &od->nbs->header_); + &od->nbs->header_, + lflow_ref); /* If packet needs to be hairpinned, for established sessions there * should already be an SNAT conntrack entry. @@ -7967,13 +8063,15 @@ build_lb_hairpin(const struct ls_stateful_record *ls_stateful_rec, "ip && ct.est && ct.trk" " && "REGBIT_HAIRPIN " == 1", "ct_snat;", - &od->nbs->header_); + &od->nbs->header_, + lflow_ref); /* For the reply of hairpinned traffic, snat the src ip to the VIP. */ ovn_lflow_add_with_hint(lflows, od, S_SWITCH_IN_NAT_HAIRPIN, 90, "ip && "REGBIT_HAIRPIN_REPLY " == 1", "ct_snat;", - &od->nbs->header_); + &od->nbs->header_, + lflow_ref); /* Ingress Hairpin table. * - Priority 1: Packets that were SNAT-ed for hairpinning should be @@ -7983,12 +8081,13 @@ build_lb_hairpin(const struct ls_stateful_record *ls_stateful_rec, lflows, od, S_SWITCH_IN_HAIRPIN, 1, "("REGBIT_HAIRPIN " == 1 || " REGBIT_HAIRPIN_REPLY " == 1)", "eth.dst <-> eth.src; outport = inport; flags.loopback = 1; " - "output;"); + "output;", lflow_ref); } } static void -build_vtep_hairpin(struct ovn_datapath *od, struct lflow_table *lflows) +build_vtep_hairpin(struct ovn_datapath *od, struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { if (!od->has_vtep_lports) { /* There is no need in these flows if datapath has no vtep lports. */ @@ -8001,7 +8100,7 @@ build_vtep_hairpin(struct ovn_datapath *od, struct lflow_table *lflows) char *action = xasprintf("next(pipeline=ingress, table=%d);", ovn_stage_get_table(S_SWITCH_IN_L2_LKUP)); ovn_lflow_add(lflows, od, S_SWITCH_IN_HAIRPIN, 1000, - REGBIT_FROM_RAMP" == 1", action); + REGBIT_FROM_RAMP" == 1", action, lflow_ref); free(action); /* Ingress pre-arp flow for traffic from VTEP (ramp) switch. @@ -8018,7 +8117,7 @@ build_vtep_hairpin(struct ovn_datapath *od, struct lflow_table *lflows) REGBIT_FROM_RAMP" == 1 && is_chassis_resident(%s)", op->cr_port->json_key); ovn_lflow_add(lflows, od, S_SWITCH_IN_HAIRPIN, 2000, - ds_cstr(&match), "next;"); + ds_cstr(&match), "next;", lflow_ref); } } @@ -8029,14 +8128,15 @@ build_vtep_hairpin(struct ovn_datapath *od, struct lflow_table *lflows) */ ovn_lflow_add(lflows, od, S_SWITCH_IN_ARP_ND_RSP, 65535, REGBIT_FROM_RAMP" == 1 && (arp || nd_ns)", - "flags.loopback = 1; next;"); + "flags.loopback = 1; next;", lflow_ref); ds_destroy(&match); } /* Build logical flows for the forwarding groups */ static void -build_fwd_group_lflows(struct ovn_datapath *od, struct lflow_table *lflows) +build_fwd_group_lflows(struct ovn_datapath *od, struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { ovs_assert(od->nbs); if (!od->nbs->n_forwarding_groups) { @@ -8071,7 +8171,7 @@ build_fwd_group_lflows(struct ovn_datapath *od, struct lflow_table *lflows) ovn_lflow_add_with_hint(lflows, od, S_SWITCH_IN_ARP_ND_RSP, 50, ds_cstr(&match), ds_cstr(&actions), - &fwd_group->header_); + &fwd_group->header_, lflow_ref); /* L2 lookup for the forwarding group's virtual MAC */ ds_clear(&match); @@ -8094,7 +8194,7 @@ build_fwd_group_lflows(struct ovn_datapath *od, struct lflow_table *lflows) ds_put_format(&actions, "fwd_group(%s);", ds_cstr(&group_ports)); ovn_lflow_add_with_hint(lflows, od, S_SWITCH_IN_L2_LKUP, 50, ds_cstr(&match), ds_cstr(&actions), - &fwd_group->header_); + &fwd_group->header_, lflow_ref); } ds_destroy(&match); @@ -8242,10 +8342,8 @@ build_lswitch_rport_arp_req_self_orig_flow(struct ovn_port *op, ds_put_format(&match, "eth.src == %s && (arp.op == 1 || rarp.op == 3 || nd_ns)", ds_cstr(ð_src)); - ovn_lflow_add_with_lflow_ref(lflows, od, S_SWITCH_IN_L2_LKUP, priority, - ds_cstr(&match), - "outport = \""MC_FLOOD_L2"\"; output;", - lflow_ref); + ovn_lflow_add(lflows, od, S_SWITCH_IN_L2_LKUP, priority, ds_cstr(&match), + "outport = \""MC_FLOOD_L2"\"; output;", lflow_ref); ds_destroy(ð_src); ds_destroy(&match); @@ -8328,17 +8426,17 @@ build_lswitch_rport_arp_req_flow( ds_put_format(&actions, "clone {outport = %s; output; }; " "outport = \""MC_FLOOD_L2"\"; output;", patch_op->json_key); - ovn_lflow_add_with_lflow_ref_hint(lflows, od, S_SWITCH_IN_L2_LKUP, - priority, ds_cstr(&match), - ds_cstr(&actions), stage_hint, - lflow_ref); + ovn_lflow_add_with_hint(lflows, od, S_SWITCH_IN_L2_LKUP, + priority, ds_cstr(&match), + ds_cstr(&actions), stage_hint, + lflow_ref); } else { ds_put_format(&actions, "outport = %s; output;", patch_op->json_key); - ovn_lflow_add_with_lflow_ref_hint(lflows, od, S_SWITCH_IN_L2_LKUP, - priority, ds_cstr(&match), - ds_cstr(&actions), - stage_hint, - lflow_ref); + ovn_lflow_add_with_hint(lflows, od, S_SWITCH_IN_L2_LKUP, + priority, ds_cstr(&match), + ds_cstr(&actions), + stage_hint, + lflow_ref); } ds_destroy(&match); @@ -8528,7 +8626,8 @@ build_dhcpv4_options_flows(struct ovn_port *op, struct lport_addresses *lsp_addrs, struct ovn_port *inport, bool is_external, const struct shash *meter_groups, - struct lflow_table *lflows) + struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { struct ds match = DS_EMPTY_INITIALIZER; @@ -8559,7 +8658,7 @@ build_dhcpv4_options_flows(struct ovn_port *op, op->json_key); } - ovn_lflow_add_with_lflow_ref_hint__(lflows, op->od, + ovn_lflow_add_with_hint__(lflows, op->od, S_SWITCH_IN_DHCP_OPTIONS, 100, ds_cstr(&match), ds_cstr(&options_action), @@ -8568,7 +8667,7 @@ build_dhcpv4_options_flows(struct ovn_port *op, op->od->nbs->copp, meter_groups), &op->nbsp->dhcpv4_options->header_, - op->lflow_ref); + lflow_ref); ds_clear(&match); /* If REGBIT_DHCP_OPTS_RESULT is set, it means the @@ -8588,7 +8687,7 @@ build_dhcpv4_options_flows(struct ovn_port *op, lflows, op->od, S_SWITCH_IN_DHCP_RESPONSE, 100, ds_cstr(&match), ds_cstr(&response_action), inport->key, &op->nbsp->dhcpv4_options->header_, - op->lflow_ref); + lflow_ref); ds_destroy(&options_action); ds_destroy(&response_action); ds_destroy(&ipv4_addr_match); @@ -8616,7 +8715,7 @@ build_dhcpv4_options_flows(struct ovn_port *op, lflows, op->od, S_SWITCH_OUT_ACL_EVAL, 34000, ds_cstr(&match),dhcp_actions, op->key, &op->nbsp->dhcpv4_options->header_, - op->lflow_ref); + lflow_ref); } break; } @@ -8629,7 +8728,8 @@ build_dhcpv6_options_flows(struct ovn_port *op, struct lport_addresses *lsp_addrs, struct ovn_port *inport, bool is_external, const struct shash *meter_groups, - struct lflow_table *lflows) + struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { struct ds match = DS_EMPTY_INITIALIZER; @@ -8651,7 +8751,7 @@ build_dhcpv6_options_flows(struct ovn_port *op, op->json_key); } - ovn_lflow_add_with_lflow_ref_hint__(lflows, op->od, + ovn_lflow_add_with_hint__(lflows, op->od, S_SWITCH_IN_DHCP_OPTIONS, 100, ds_cstr(&match), ds_cstr(&options_action), @@ -8660,7 +8760,7 @@ build_dhcpv6_options_flows(struct ovn_port *op, op->od->nbs->copp, meter_groups), &op->nbsp->dhcpv6_options->header_, - op->lflow_ref); + lflow_ref); /* If REGBIT_DHCP_OPTS_RESULT is set to 1, it means the * put_dhcpv6_opts action is successful */ @@ -8668,7 +8768,7 @@ build_dhcpv6_options_flows(struct ovn_port *op, ovn_lflow_add_with_lport_and_hint( lflows, op->od, S_SWITCH_IN_DHCP_RESPONSE, 100, ds_cstr(&match), ds_cstr(&response_action), inport->key, - &op->nbsp->dhcpv6_options->header_, op->lflow_ref); + &op->nbsp->dhcpv6_options->header_, lflow_ref); ds_destroy(&options_action); ds_destroy(&response_action); @@ -8701,7 +8801,7 @@ build_dhcpv6_options_flows(struct ovn_port *op, lflows, op->od, S_SWITCH_OUT_ACL_EVAL, 34000, ds_cstr(&match),dhcp6_actions, op->key, &op->nbsp->dhcpv6_options->header_, - op->lflow_ref); + lflow_ref); } break; } @@ -8712,7 +8812,8 @@ build_dhcpv6_options_flows(struct ovn_port *op, static void build_drop_arp_nd_flows_for_unbound_router_ports(struct ovn_port *op, const struct ovn_port *port, - struct lflow_table *lflows) + struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { struct ds match = DS_EMPTY_INITIALIZER; @@ -8732,7 +8833,7 @@ build_drop_arp_nd_flows_for_unbound_router_ports(struct ovn_port *op, ovn_lflow_add_with_lport_and_hint( lflows, op->od, S_SWITCH_IN_EXTERNAL_PORT, 100, ds_cstr(&match), debug_drop_action(), port->key, - &op->nbsp->header_, op->lflow_ref); + &op->nbsp->header_, lflow_ref); } for (size_t l = 0; l < rp->lsp_addrs[k].n_ipv6_addrs; l++) { ds_clear(&match); @@ -8748,7 +8849,7 @@ build_drop_arp_nd_flows_for_unbound_router_ports(struct ovn_port *op, ovn_lflow_add_with_lport_and_hint( lflows, op->od, S_SWITCH_IN_EXTERNAL_PORT, 100, ds_cstr(&match), debug_drop_action(), port->key, - &op->nbsp->header_, op->lflow_ref); + &op->nbsp->header_, lflow_ref); } ds_clear(&match); @@ -8765,7 +8866,7 @@ build_drop_arp_nd_flows_for_unbound_router_ports(struct ovn_port *op, debug_drop_action(), port->key, &op->nbsp->header_, - op->lflow_ref); + lflow_ref); } } } @@ -8780,19 +8881,22 @@ is_vlan_transparent(const struct ovn_datapath *od) static void build_lswitch_lflows_l2_unknown(struct ovn_datapath *od, - struct lflow_table *lflows) + struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { /* Ingress table 25/26: Destination lookup for unknown MACs. */ if (od->has_unknown) { ovn_lflow_add(lflows, od, S_SWITCH_IN_L2_UNKNOWN, 50, "outport == \"none\"", - "outport = \""MC_UNKNOWN "\"; output;"); + "outport = \""MC_UNKNOWN "\"; output;", + lflow_ref); } else { ovn_lflow_add(lflows, od, S_SWITCH_IN_L2_UNKNOWN, 50, - "outport == \"none\"", debug_drop_action()); + "outport == \"none\"", debug_drop_action(), + lflow_ref); } ovn_lflow_add(lflows, od, S_SWITCH_IN_L2_UNKNOWN, 0, "1", - "output;"); + "output;", lflow_ref); } /* Build pre-ACL and ACL tables for both ingress and egress. @@ -8802,42 +8906,49 @@ build_lswitch_lflows_pre_acl_and_acl( struct ovn_datapath *od, const struct chassis_features *features, struct lflow_table *lflows, - const struct shash *meter_groups) + const struct shash *meter_groups, + struct lflow_ref *lflow_ref) { ovs_assert(od->nbs); - build_pre_acls(od, lflows); - build_pre_lb(od, meter_groups, lflows); - build_pre_stateful(od, features, lflows); - build_qos(od, lflows); - build_stateful(od, features, lflows); - build_vtep_hairpin(od, lflows); + build_pre_acls(od, lflows, lflow_ref); + build_pre_lb(od, meter_groups, lflows, lflow_ref); + build_pre_stateful(od, features, lflows, lflow_ref); + build_qos(od, lflows, lflow_ref); + build_stateful(od, features, lflows, lflow_ref); + build_vtep_hairpin(od, lflows, lflow_ref); } /* Logical switch ingress table 0: Admission control framework (priority * 100). */ static void build_lswitch_lflows_admission_control(struct ovn_datapath *od, - struct lflow_table *lflows) + struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { ovs_assert(od->nbs); /* Logical VLANs not supported. */ if (!is_vlan_transparent(od)) { /* Block logical VLANs. */ ovn_lflow_add(lflows, od, S_SWITCH_IN_CHECK_PORT_SEC, 100, - "vlan.present", debug_drop_action()); + "vlan.present", debug_drop_action(), + lflow_ref); } /* Broadcast/multicast source address is invalid. */ ovn_lflow_add(lflows, od, S_SWITCH_IN_CHECK_PORT_SEC, 100, - "eth.src[40]", debug_drop_action()); + "eth.src[40]", debug_drop_action(), + lflow_ref); ovn_lflow_add(lflows, od, S_SWITCH_IN_CHECK_PORT_SEC, 50, "1", - REGBIT_PORT_SEC_DROP" = check_in_port_sec(); next;"); + REGBIT_PORT_SEC_DROP" = check_in_port_sec(); next;", + lflow_ref); ovn_lflow_add(lflows, od, S_SWITCH_IN_APPLY_PORT_SEC, 50, - REGBIT_PORT_SEC_DROP" == 1", debug_drop_action()); + REGBIT_PORT_SEC_DROP" == 1", debug_drop_action(), + lflow_ref); - ovn_lflow_add(lflows, od, S_SWITCH_IN_APPLY_PORT_SEC, 0, "1", "next;"); + ovn_lflow_add(lflows, od, S_SWITCH_IN_APPLY_PORT_SEC, 0, "1", "next;", + lflow_ref); } /* Ingress table 19: ARP/ND responder, skip requests coming from localnet @@ -8994,12 +9105,12 @@ build_lswitch_arp_nd_responder_known_ips(struct ovn_port *op, "output;", op->lsp_addrs[i].ea_s, op->lsp_addrs[i].ea_s, op->lsp_addrs[i].ipv4_addrs[j].addr_s); - ovn_lflow_add_with_lflow_ref_hint(lflows, op->od, - S_SWITCH_IN_ARP_ND_RSP, 50, - ds_cstr(match), - ds_cstr(actions), - &op->nbsp->header_, - op->lflow_ref); + ovn_lflow_add_with_hint(lflows, op->od, + S_SWITCH_IN_ARP_ND_RSP, 50, + ds_cstr(match), + ds_cstr(actions), + &op->nbsp->header_, + op->lflow_ref); /* Do not reply to an ARP request from the port that owns * the address (otherwise a DHCP client that ARPs to check @@ -9049,16 +9160,16 @@ build_lswitch_arp_nd_responder_known_ips(struct ovn_port *op, op->lsp_addrs[i].ipv6_addrs[j].addr_s, op->lsp_addrs[i].ipv6_addrs[j].addr_s, op->lsp_addrs[i].ea_s); - ovn_lflow_add_with_lflow_ref_hint__(lflows, op->od, - S_SWITCH_IN_ARP_ND_RSP, 50, - ds_cstr(match), - ds_cstr(actions), - NULL, - copp_meter_get(COPP_ND_NA, - op->od->nbs->copp, - meter_groups), - &op->nbsp->header_, - op->lflow_ref); + ovn_lflow_add_with_hint__(lflows, op->od, + S_SWITCH_IN_ARP_ND_RSP, 50, + ds_cstr(match), + ds_cstr(actions), + NULL, + copp_meter_get(COPP_ND_NA, + op->od->nbs->copp, + meter_groups), + &op->nbsp->header_, + op->lflow_ref); /* Do not reply to a solicitation from the port that owns * the address (otherwise DAD detection will fail). */ @@ -9114,12 +9225,12 @@ build_lswitch_arp_nd_responder_known_ips(struct ovn_port *op, ea_s, ea_s); - ovn_lflow_add_with_lflow_ref_hint(lflows, op->od, - S_SWITCH_IN_ARP_ND_RSP, - 30, ds_cstr(match), - ds_cstr(actions), - &op->nbsp->header_, - op->lflow_ref); + ovn_lflow_add_with_hint(lflows, op->od, + S_SWITCH_IN_ARP_ND_RSP, + 30, ds_cstr(match), + ds_cstr(actions), + &op->nbsp->header_, + op->lflow_ref); } /* Add IPv6 NDP responses. @@ -9162,16 +9273,16 @@ build_lswitch_arp_nd_responder_known_ips(struct ovn_port *op, lsp_is_router(op->nbsp) ? "nd_na_router" : "nd_na", ea_s, ea_s); - ovn_lflow_add_with_lflow_ref_hint__(lflows, op->od, - S_SWITCH_IN_ARP_ND_RSP, 30, - ds_cstr(match), - ds_cstr(actions), - NULL, - copp_meter_get(COPP_ND_NA, - op->od->nbs->copp, - meter_groups), - &op->nbsp->header_, - op->lflow_ref); + ovn_lflow_add_with_hint__(lflows, op->od, + S_SWITCH_IN_ARP_ND_RSP, 30, + ds_cstr(match), + ds_cstr(actions), + NULL, + copp_meter_get(COPP_ND_NA, + op->od->nbs->copp, + meter_groups), + &op->nbsp->header_, + op->lflow_ref); ds_destroy(&ip6_dst_match); ds_destroy(&nd_target_match); } @@ -9182,21 +9293,24 @@ build_lswitch_arp_nd_responder_known_ips(struct ovn_port *op, * (priority 0)*/ static void build_lswitch_arp_nd_responder_default(struct ovn_datapath *od, - struct lflow_table *lflows) + struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { ovs_assert(od->nbs); - ovn_lflow_add(lflows, od, S_SWITCH_IN_ARP_ND_RSP, 0, "1", "next;"); + ovn_lflow_add(lflows, od, S_SWITCH_IN_ARP_ND_RSP, 0, "1", "next;", + lflow_ref); } /* Ingress table 19: ARP/ND responder for service monitor source ip. * (priority 110)*/ static void -build_lswitch_arp_nd_service_monitor(const struct ovn_northd_lb *lb, +build_lswitch_arp_nd_service_monitor(const struct ovn_lb_datapaths *lb_dps, const struct hmap *ls_ports, struct lflow_table *lflows, struct ds *actions, struct ds *match) { + const struct ovn_northd_lb *lb = lb_dps->lb; for (size_t i = 0; i < lb->n_vips; i++) { struct ovn_northd_lb_vip *lb_vip_nb = &lb->vips_nb[i]; if (!lb_vip_nb->lb_health_check) { @@ -9259,7 +9373,8 @@ build_lswitch_arp_nd_service_monitor(const struct ovn_northd_lb *lb, op->od, S_SWITCH_IN_ARP_ND_RSP, 110, ds_cstr(match), ds_cstr(actions), - &lb->nlb->header_); + &lb->nlb->header_, + NULL); } } } @@ -9299,19 +9414,19 @@ build_lswitch_dhcp_options_and_response(struct ovn_port *op, build_dhcpv4_options_flows( op, &op->lsp_addrs[i], op->od->localnet_ports[j], is_external, - meter_groups, lflows); + meter_groups, lflows, op->lflow_ref); build_dhcpv6_options_flows( op, &op->lsp_addrs[i], op->od->localnet_ports[j], is_external, - meter_groups, lflows); + meter_groups, lflows, op->lflow_ref); } } else { build_dhcpv4_options_flows(op, &op->lsp_addrs[i], op, is_external, meter_groups, - lflows); + lflows, op->lflow_ref); build_dhcpv6_options_flows(op, &op->lsp_addrs[i], op, is_external, meter_groups, - lflows); + lflows, op->lflow_ref); } } } @@ -9324,14 +9439,20 @@ build_lswitch_dhcp_options_and_response(struct ovn_port *op, * (priority 0). */ static void build_lswitch_dhcp_and_dns_defaults(struct ovn_datapath *od, - struct lflow_table *lflows) + struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { ovs_assert(od->nbs); - ovn_lflow_add(lflows, od, S_SWITCH_IN_DHCP_OPTIONS, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_SWITCH_IN_DHCP_RESPONSE, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_SWITCH_IN_DNS_LOOKUP, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_SWITCH_IN_DNS_RESPONSE, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_SWITCH_IN_EXTERNAL_PORT, 0, "1", "next;"); + ovn_lflow_add(lflows, od, S_SWITCH_IN_DHCP_OPTIONS, 0, "1", "next;", + lflow_ref); + ovn_lflow_add(lflows, od, S_SWITCH_IN_DHCP_RESPONSE, 0, "1", "next;", + lflow_ref); + ovn_lflow_add(lflows, od, S_SWITCH_IN_DNS_LOOKUP, 0, "1", "next;", + lflow_ref); + ovn_lflow_add(lflows, od, S_SWITCH_IN_DNS_RESPONSE, 0, "1", "next;", + lflow_ref); + ovn_lflow_add(lflows, od, S_SWITCH_IN_EXTERNAL_PORT, 0, "1", "next;", + lflow_ref); } /* Logical switch ingress table 22 and 23: DNS lookup and response @@ -9340,7 +9461,8 @@ build_lswitch_dhcp_and_dns_defaults(struct ovn_datapath *od, static void build_lswitch_dns_lookup_and_response(struct ovn_datapath *od, struct lflow_table *lflows, - const struct shash *meter_groups) + const struct shash *meter_groups, + struct lflow_ref *lflow_ref) { ovs_assert(od->nbs); if (!ls_has_dns_records(od->nbs)) { @@ -9350,18 +9472,18 @@ build_lswitch_dns_lookup_and_response(struct ovn_datapath *od, "udp.dst == 53", REGBIT_DNS_LOOKUP_RESULT" = dns_lookup(); next;", copp_meter_get(COPP_DNS, od->nbs->copp, - meter_groups)); + meter_groups), lflow_ref); const char *dns_action = "eth.dst <-> eth.src; ip4.src <-> ip4.dst; " "udp.dst = udp.src; udp.src = 53; outport = inport; " "flags.loopback = 1; output;"; const char *dns_match = "udp.dst == 53 && "REGBIT_DNS_LOOKUP_RESULT; ovn_lflow_add(lflows, od, S_SWITCH_IN_DNS_RESPONSE, 100, - dns_match, dns_action); + dns_match, dns_action, lflow_ref); dns_action = "eth.dst <-> eth.src; ip6.src <-> ip6.dst; " "udp.dst = udp.src; udp.src = 53; outport = inport; " "flags.loopback = 1; output;"; ovn_lflow_add(lflows, od, S_SWITCH_IN_DNS_RESPONSE, 100, - dns_match, dns_action); + dns_match, dns_action, lflow_ref); } /* Table 24: External port. Drop ARP request for router ips from @@ -9378,7 +9500,8 @@ build_lswitch_external_port(struct ovn_port *op, } for (size_t i = 0; i < op->od->n_localnet_ports; i++) { build_drop_arp_nd_flows_for_unbound_router_ports( - op, op->od->localnet_ports[i], lflows); + op, op->od->localnet_ports[i], lflows, + op->lflow_ref); } } @@ -9388,7 +9511,8 @@ static void build_lswitch_destination_lookup_bmcast(struct ovn_datapath *od, struct lflow_table *lflows, struct ds *actions, - const struct shash *meter_groups) + const struct shash *meter_groups, + struct lflow_ref *lflow_ref) { ovs_assert(od->nbs); @@ -9396,7 +9520,7 @@ build_lswitch_destination_lookup_bmcast(struct ovn_datapath *od, "eth.dst == $svc_monitor_mac && (tcp || icmp || icmp6)", "handle_svc_check(inport);", copp_meter_get(COPP_SVC_MONITOR, od->nbs->copp, - meter_groups)); + meter_groups), lflow_ref); struct mcast_switch_info *mcast_sw_info = &od->mcast_info.sw; @@ -9407,27 +9531,31 @@ build_lswitch_destination_lookup_bmcast(struct ovn_datapath *od, ovn_lflow_metered(lflows, od, S_SWITCH_IN_L2_LKUP, 100, "igmp", ds_cstr(actions), copp_meter_get(COPP_IGMP, od->nbs->copp, - meter_groups)); + meter_groups), + lflow_ref); /* Punt MLD traffic to controller. */ ovn_lflow_metered(lflows, od, S_SWITCH_IN_L2_LKUP, 100, "mldv1 || mldv2", ds_cstr(actions), copp_meter_get(COPP_IGMP, od->nbs->copp, - meter_groups)); + meter_groups), + lflow_ref); /* Flood all IP multicast traffic destined to 224.0.0.X to all * ports - RFC 4541, section 2.1.2, item 2. */ ovn_lflow_add(lflows, od, S_SWITCH_IN_L2_LKUP, 85, "ip4.mcast && ip4.dst == 224.0.0.0/24", - "outport = \""MC_FLOOD_L2"\"; output;"); + "outport = \""MC_FLOOD_L2"\"; output;", + lflow_ref); /* Flood all IPv6 multicast traffic destined to reserved * multicast IPs (RFC 4291, 2.7.1). */ ovn_lflow_add(lflows, od, S_SWITCH_IN_L2_LKUP, 85, "ip6.mcast_flood", - "outport = \""MC_FLOOD"\"; output;"); + "outport = \""MC_FLOOD"\"; output;", + lflow_ref); /* Forward uregistered IP multicast to routers with relay enabled * and to any ports configured to flood IP multicast traffic. @@ -9459,7 +9587,7 @@ build_lswitch_destination_lookup_bmcast(struct ovn_datapath *od, ovn_lflow_add(lflows, od, S_SWITCH_IN_L2_LKUP, 80, "ip4.mcast || ip6.mcast", - ds_cstr(actions)); + ds_cstr(actions), lflow_ref); } } @@ -9467,11 +9595,12 @@ build_lswitch_destination_lookup_bmcast(struct ovn_datapath *od, "broadcast-arps-to-all-routers", true)) { ovn_lflow_add(lflows, od, S_SWITCH_IN_L2_LKUP, 72, "eth.mcast && (arp.op == 1 || nd_ns)", - "outport = \""MC_FLOOD_L2"\"; output;"); + "outport = \""MC_FLOOD_L2"\"; output;", + lflow_ref); } ovn_lflow_add(lflows, od, S_SWITCH_IN_L2_LKUP, 70, "eth.mcast", - "outport = \""MC_FLOOD"\"; output;"); + "outport = \""MC_FLOOD"\"; output;", lflow_ref); } @@ -9553,7 +9682,7 @@ build_lswitch_ip_mcast_igmp_mld(struct ovn_igmp_group *igmp_group, igmp_group->mcgroup.name); ovn_lflow_add(lflows, igmp_group->datapath, S_SWITCH_IN_L2_LKUP, - 90, ds_cstr(match), ds_cstr(actions)); + 90, ds_cstr(match), ds_cstr(actions), NULL); } } @@ -9593,12 +9722,12 @@ build_lswitch_ip_unicast_lookup(struct ovn_port *op, ds_clear(actions); ds_put_format(actions, action, op->json_key); - ovn_lflow_add_with_lflow_ref_hint(lflows, op->od, - S_SWITCH_IN_L2_LKUP, - 50, ds_cstr(match), - ds_cstr(actions), - &op->nbsp->header_, - op->lflow_ref); + ovn_lflow_add_with_hint(lflows, op->od, + S_SWITCH_IN_L2_LKUP, + 50, ds_cstr(match), + ds_cstr(actions), + &op->nbsp->header_, + op->lflow_ref); } else if (!strcmp(op->nbsp->addresses[i], "unknown")) { continue; } else if (is_dynamic_lsp_address(op->nbsp->addresses[i])) { @@ -9613,12 +9742,12 @@ build_lswitch_ip_unicast_lookup(struct ovn_port *op, ds_clear(actions); ds_put_format(actions, action, op->json_key); - ovn_lflow_add_with_lflow_ref_hint(lflows, op->od, - S_SWITCH_IN_L2_LKUP, - 50, ds_cstr(match), - ds_cstr(actions), - &op->nbsp->header_, - op->lflow_ref); + ovn_lflow_add_with_hint(lflows, op->od, + S_SWITCH_IN_L2_LKUP, + 50, ds_cstr(match), + ds_cstr(actions), + &op->nbsp->header_, + op->lflow_ref); } else if (!strcmp(op->nbsp->addresses[i], "router")) { if (!op->peer || !op->peer->nbrp || !ovs_scan(op->peer->nbrp->mac, @@ -9670,11 +9799,11 @@ build_lswitch_ip_unicast_lookup(struct ovn_port *op, ds_clear(actions); ds_put_format(actions, action, op->json_key); - ovn_lflow_add_with_lflow_ref_hint(lflows, op->od, - S_SWITCH_IN_L2_LKUP, 50, - ds_cstr(match), ds_cstr(actions), - &op->nbsp->header_, - op->lflow_ref); + ovn_lflow_add_with_hint(lflows, op->od, + S_SWITCH_IN_L2_LKUP, 50, + ds_cstr(match), ds_cstr(actions), + &op->nbsp->header_, + op->lflow_ref); } else { static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 1); @@ -9723,7 +9852,7 @@ build_lswitch_ip_unicast_lookup_for_nats( ds_clear(actions); ds_put_format(actions, action, op->json_key); - ovn_lflow_add_with_lflow_ref_hint(lflows, op->od, + ovn_lflow_add_with_hint(lflows, op->od, S_SWITCH_IN_L2_LKUP, 50, ds_cstr(match), ds_cstr(actions), @@ -9971,7 +10100,8 @@ static void build_routing_policy_flow(struct lflow_table *lflows, struct ovn_datapath *od, const struct hmap *lr_ports, const struct nbrec_logical_router_policy *rule, - const struct ovsdb_idl_row *stage_hint) + const struct ovsdb_idl_row *stage_hint, + struct lflow_ref *lflow_ref) { struct ds match = DS_EMPTY_INITIALIZER; struct ds actions = DS_EMPTY_INITIALIZER; @@ -10027,7 +10157,8 @@ build_routing_policy_flow(struct lflow_table *lflows, struct ovn_datapath *od, ds_put_format(&match, "%s", rule->match); ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_POLICY, rule->priority, - ds_cstr(&match), ds_cstr(&actions), stage_hint); + ds_cstr(&match), ds_cstr(&actions), stage_hint, + lflow_ref); ds_destroy(&match); ds_destroy(&actions); } @@ -10037,7 +10168,8 @@ build_ecmp_routing_policy_flows(struct lflow_table *lflows, struct ovn_datapath *od, const struct hmap *lr_ports, const struct nbrec_logical_router_policy *rule, - uint16_t ecmp_group_id) + uint16_t ecmp_group_id, + struct lflow_ref *lflow_ref) { ovs_assert(rule->n_nexthops > 1); @@ -10109,7 +10241,8 @@ build_ecmp_routing_policy_flows(struct lflow_table *lflows, ecmp_group_id, i + 1); ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_POLICY_ECMP, 100, ds_cstr(&match), - ds_cstr(&actions), &rule->header_); + ds_cstr(&actions), &rule->header_, + lflow_ref); } ds_clear(&actions); @@ -10127,7 +10260,8 @@ build_ecmp_routing_policy_flows(struct lflow_table *lflows, ds_put_cstr(&actions, ");"); ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_POLICY, rule->priority, rule->match, - ds_cstr(&actions), &rule->header_); + ds_cstr(&actions), &rule->header_, + lflow_ref); cleanup: ds_destroy(&match); @@ -10172,7 +10306,8 @@ get_route_table_id(struct simap *route_tables, const char *route_table_name) static void build_route_table_lflow(struct ovn_datapath *od, struct lflow_table *lflows, struct nbrec_logical_router_port *lrp, - struct simap *route_tables) + struct simap *route_tables, + struct lflow_ref *lflow_ref) { struct ds match = DS_EMPTY_INITIALIZER; struct ds actions = DS_EMPTY_INITIALIZER; @@ -10188,7 +10323,7 @@ build_route_table_lflow(struct ovn_datapath *od, struct lflow_table *lflows, REG_ROUTE_TABLE_ID, rtb_id); ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_ROUTING_PRE, 100, - ds_cstr(&match), ds_cstr(&actions)); + ds_cstr(&match), ds_cstr(&actions), lflow_ref); ds_destroy(&match); ds_destroy(&actions); @@ -10587,7 +10722,8 @@ add_ecmp_symmetric_reply_flows(struct lflow_table *lflows, const char *port_ip, struct ovn_port *out_port, const struct parsed_route *route, - struct ds *route_match) + struct ds *route_match, + struct lflow_ref *lflow_ref) { const struct nbrec_logical_router_static_route *st_route = route->route; struct ds base_match = DS_EMPTY_INITIALIZER; @@ -10610,12 +10746,12 @@ add_ecmp_symmetric_reply_flows(struct lflow_table *lflows, free(cidr); ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_DEFRAG, 100, ds_cstr(&base_match), "ct_next;", - &st_route->header_); + &st_route->header_, lflow_ref); /* And packets that go out over an ECMP route need conntrack */ ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_DEFRAG, 100, ds_cstr(route_match), "ct_next;", - &st_route->header_); + &st_route->header_, lflow_ref); /* Save src eth and inport in ct_label for packets that arrive over * an ECMP route. @@ -10632,7 +10768,8 @@ add_ecmp_symmetric_reply_flows(struct lflow_table *lflows, ct_ecmp_reply_port_match, out_port->sb->tunnel_key); ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_ECMP_STATEFUL, 100, ds_cstr(&match), ds_cstr(&actions), - &st_route->header_); + &st_route->header_, + lflow_ref); ds_clear(&match); ds_put_format(&match, "%s && (ct.new && !ct.est) && udp", ds_cstr(&base_match)); @@ -10644,7 +10781,8 @@ add_ecmp_symmetric_reply_flows(struct lflow_table *lflows, ct_ecmp_reply_port_match, out_port->sb->tunnel_key); ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_ECMP_STATEFUL, 100, ds_cstr(&match), ds_cstr(&actions), - &st_route->header_); + &st_route->header_, + lflow_ref); ds_clear(&match); ds_put_format(&match, "%s && (ct.new && !ct.est) && sctp", ds_cstr(&base_match)); @@ -10656,7 +10794,8 @@ add_ecmp_symmetric_reply_flows(struct lflow_table *lflows, ct_ecmp_reply_port_match, out_port->sb->tunnel_key); ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_ECMP_STATEFUL, 100, ds_cstr(&match), ds_cstr(&actions), - &st_route->header_); + &st_route->header_, + lflow_ref); ds_clear(&match); ds_put_format(&match, @@ -10670,7 +10809,8 @@ add_ecmp_symmetric_reply_flows(struct lflow_table *lflows, ct_ecmp_reply_port_match, out_port->sb->tunnel_key); ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_ECMP_STATEFUL, 100, ds_cstr(&match), ds_cstr(&actions), - &st_route->header_); + &st_route->header_, + lflow_ref); ds_clear(&match); ds_put_format(&match, @@ -10684,7 +10824,8 @@ add_ecmp_symmetric_reply_flows(struct lflow_table *lflows, ct_ecmp_reply_port_match, out_port->sb->tunnel_key); ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_ECMP_STATEFUL, 100, ds_cstr(&match), ds_cstr(&actions), - &st_route->header_); + &st_route->header_, + lflow_ref); ds_clear(&match); ds_put_format(&match, "%s && (!ct.rpl && ct.est) && sctp", @@ -10697,7 +10838,8 @@ add_ecmp_symmetric_reply_flows(struct lflow_table *lflows, ct_ecmp_reply_port_match, out_port->sb->tunnel_key); ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_ECMP_STATEFUL, 100, ds_cstr(&match), ds_cstr(&actions), - &st_route->header_); + &st_route->header_, + lflow_ref); /* Bypass ECMP selection if we already have ct_label information * for where to route the packet. @@ -10716,12 +10858,14 @@ add_ecmp_symmetric_reply_flows(struct lflow_table *lflows, port_ip, out_port->json_key); ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_IP_ROUTING, 10300, ds_cstr(&match), ds_cstr(&actions), - &st_route->header_); + &st_route->header_, + lflow_ref); /* Egress reply traffic for symmetric ECMP routes skips router policies. */ ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_POLICY, 65535, ds_cstr(&ecmp_reply), "next;", - &st_route->header_); + &st_route->header_, + lflow_ref); /* Use REG_ECMP_ETH_FULL to pass the eth field from ct_label to eth.dst to * avoid masked access to ct_label. Otherwise it may prevent OVS flow @@ -10737,7 +10881,8 @@ add_ecmp_symmetric_reply_flows(struct lflow_table *lflows, " pop(" REG_ECMP_ETH_FULL "); next;"; ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_ARP_RESOLVE, 200, ds_cstr(&ecmp_reply), - action, &st_route->header_); + action, &st_route->header_, + lflow_ref); ds_destroy(&base_match); ds_destroy(&match); @@ -10748,7 +10893,8 @@ add_ecmp_symmetric_reply_flows(struct lflow_table *lflows, static void build_ecmp_route_flow(struct lflow_table *lflows, struct ovn_datapath *od, bool ct_masked_mark, const struct hmap *lr_ports, - struct ecmp_groups_node *eg) + struct ecmp_groups_node *eg, + struct lflow_ref *lflow_ref) { bool is_ipv4 = IN6_IS_ADDR_V4MAPPED(&eg->prefix); @@ -10781,7 +10927,8 @@ build_ecmp_route_flow(struct lflow_table *lflows, struct ovn_datapath *od, ds_put_cstr(&actions, ");"); ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_ROUTING, priority, - ds_cstr(&route_match), ds_cstr(&actions)); + ds_cstr(&route_match), ds_cstr(&actions), + lflow_ref); /* Add per member flow */ struct ds match = DS_EMPTY_INITIALIZER; @@ -10804,7 +10951,8 @@ build_ecmp_route_flow(struct lflow_table *lflows, struct ovn_datapath *od, out_port->key)) { add_ecmp_symmetric_reply_flows(lflows, od, ct_masked_mark, lrp_addr_s, out_port, - route_, &route_match); + route_, &route_match, + lflow_ref); } ds_clear(&match); ds_put_format(&match, REG_ECMP_GROUP_ID" == %"PRIu16" && " @@ -10824,7 +10972,7 @@ build_ecmp_route_flow(struct lflow_table *lflows, struct ovn_datapath *od, out_port->json_key); ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_IP_ROUTING_ECMP, 100, ds_cstr(&match), ds_cstr(&actions), - &route->header_); + &route->header_, lflow_ref); } sset_destroy(&visited_ports); ds_destroy(&match); @@ -10881,17 +11029,17 @@ add_route(struct lflow_table *lflows, struct ovn_datapath *od, ds_put_format(&actions, "ip.ttl--; %s", ds_cstr(&common_actions)); } - ovn_lflow_add_with_lflow_ref_hint(lflows, od, S_ROUTER_IN_IP_ROUTING, - priority, ds_cstr(&match), - ds_cstr(&actions), stage_hint, - lflow_ref); + ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_IP_ROUTING, + priority, ds_cstr(&match), + ds_cstr(&actions), stage_hint, + lflow_ref); if (op && op->has_bfd) { ds_put_format(&match, " && udp.dst == 3784"); - ovn_lflow_add_with_lflow_ref_hint(lflows, op->od, - S_ROUTER_IN_IP_ROUTING, - priority + 1, ds_cstr(&match), - ds_cstr(&common_actions),\ - stage_hint, lflow_ref); + ovn_lflow_add_with_hint(lflows, op->od, + S_ROUTER_IN_IP_ROUTING, + priority + 1, ds_cstr(&match), + ds_cstr(&common_actions),\ + stage_hint, lflow_ref); } ds_destroy(&match); ds_destroy(&common_actions); @@ -10901,7 +11049,8 @@ add_route(struct lflow_table *lflows, struct ovn_datapath *od, static void build_static_route_flow(struct lflow_table *lflows, struct ovn_datapath *od, const struct hmap *lr_ports, - const struct parsed_route *route_) + const struct parsed_route *route_, + struct lflow_ref *lflow_ref) { const char *lrp_addr_s = NULL; struct ovn_port *out_port = NULL; @@ -10925,7 +11074,7 @@ build_static_route_flow(struct lflow_table *lflows, struct ovn_datapath *od, add_route(lflows, route_->is_discard_route ? od : out_port->od, out_port, lrp_addr_s, prefix_s, route_->plen, route->nexthop, route_->is_src_route, route_->route_table_id, &route->header_, - route_->is_discard_route, ofs, NULL); + route_->is_discard_route, ofs, lflow_ref); free(prefix_s); } @@ -11001,7 +11150,8 @@ lrouter_use_common_zone(const struct ovn_datapath *od) static void build_distr_lrouter_nat_flows_for_lb(struct lrouter_nat_lb_flows_ctx *ctx, enum lrouter_nat_lb_flow_type type, - struct ovn_datapath *od) + struct ovn_datapath *od, + struct lflow_ref *lflow_ref) { struct ovn_port *dgp = od->l3dgw_ports[0]; @@ -11040,7 +11190,8 @@ build_distr_lrouter_nat_flows_for_lb(struct lrouter_nat_lb_flows_ctx *ctx, ovn_lflow_add_with_hint__(ctx->lflows, od, S_ROUTER_IN_DNAT, ctx->prio, ds_cstr(ctx->new_match), ctx->new_action[type], - NULL, meter, &ctx->lb->nlb->header_); + NULL, meter, &ctx->lb->nlb->header_, + lflow_ref); ds_truncate(ctx->new_match, new_match_len); @@ -11059,7 +11210,8 @@ build_distr_lrouter_nat_flows_for_lb(struct lrouter_nat_lb_flows_ctx *ctx, ovn_lflow_add_with_hint(ctx->lflows, od, S_ROUTER_IN_GW_REDIRECT, 200, ds_cstr(ctx->undnat_match), ds_cstr(ctx->gw_redir_action), - &ctx->lb->nlb->header_); + &ctx->lb->nlb->header_, + lflow_ref); ds_truncate(ctx->undnat_match, undnat_match_len); ds_put_format(ctx->undnat_match, ") && (inport == %s || outport == %s)" @@ -11067,7 +11219,8 @@ build_distr_lrouter_nat_flows_for_lb(struct lrouter_nat_lb_flows_ctx *ctx, dgp->cr_port->json_key); ovn_lflow_add_with_hint(ctx->lflows, od, S_ROUTER_OUT_UNDNAT, 120, ds_cstr(ctx->undnat_match), undnat_action, - &ctx->lb->nlb->header_); + &ctx->lb->nlb->header_, + lflow_ref); ds_truncate(ctx->undnat_match, undnat_match_len); } @@ -11075,7 +11228,8 @@ static void build_gw_lrouter_nat_flows_for_lb(struct lrouter_nat_lb_flows_ctx *ctx, enum lrouter_nat_lb_flow_type type, const struct ovn_datapaths *lr_datapaths, - const unsigned long *dp_bitmap) + const unsigned long *dp_bitmap, + struct lflow_ref *lflow_ref) { unsigned long *dp_non_meter = NULL; bool build_non_meter = false; @@ -11101,14 +11255,14 @@ build_gw_lrouter_nat_flows_for_lb(struct lrouter_nat_lb_flows_ctx *ctx, bitmap_set0(dp_non_meter, index); ovn_lflow_add_with_hint__(ctx->lflows, od, S_ROUTER_IN_DNAT, ctx->prio, ds_cstr(ctx->new_match), ctx->new_action[type], - NULL, meter, &ctx->lb->nlb->header_); + NULL, meter, &ctx->lb->nlb->header_, lflow_ref); } } if (!ctx->reject || build_non_meter) { ovn_lflow_add_with_dp_group(ctx->lflows, dp_non_meter ? dp_non_meter : dp_bitmap, ods_size(lr_datapaths), S_ROUTER_IN_DNAT, ctx->prio, ds_cstr(ctx->new_match), - ctx->new_action[type], &ctx->lb->nlb->header_); + ctx->new_action[type], &ctx->lb->nlb->header_, lflow_ref); } bitmap_free(dp_non_meter); } @@ -11242,7 +11396,7 @@ build_lrouter_nat_flows_for_lb( if (!od->n_l3dgw_ports) { bitmap_set1(gw_dp_bitmap[type], index); } else { - build_distr_lrouter_nat_flows_for_lb(&ctx, type, od); + build_distr_lrouter_nat_flows_for_lb(&ctx, type, od, NULL); } if (lb->affinity_timeout) { @@ -11263,16 +11417,17 @@ build_lrouter_nat_flows_for_lb( * S_ROUTER_IN_DNAT stage. */ ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_UNSNAT, 120, ds_cstr(&unsnat_match), "next;", - &lb->nlb->header_); + &lb->nlb->header_, NULL); } } for (size_t type = 0; type < LROUTER_NAT_LB_FLOW_MAX; type++) { build_gw_lrouter_nat_flows_for_lb(&ctx, type, lr_datapaths, - gw_dp_bitmap[type]); + gw_dp_bitmap[type], + NULL); build_lb_affinity_lr_flows(lflows, lb, lb_vip, ds_cstr(match), aff_action[type], aff_dp_bitmap[type], - lr_datapaths); + lr_datapaths, NULL); } ds_destroy(&unsnat_match); @@ -11320,7 +11475,8 @@ build_lswitch_flows_for_lb(struct ovn_lb_datapaths *lb_dps, copp_meter_get(COPP_EVENT_ELB, od->nbs->copp, meter_groups), - &lb->nlb->header_); + &lb->nlb->header_, + NULL); } /* Ignore L4 port information in the key because fragmented packets * may not have L4 information. The pre-stateful table will send @@ -11370,7 +11526,7 @@ build_lrouter_defrag_flows_for_lb(struct ovn_lb_datapaths *lb_dps, ovn_lflow_add_with_dp_group( lflows, lb_dps->nb_lr_map, ods_size(lr_datapaths), S_ROUTER_IN_DEFRAG, prio, ds_cstr(match), "ct_dnat;", - &lb_dps->lb->nlb->header_); + &lb_dps->lb->nlb->header_, NULL); } } @@ -11412,7 +11568,7 @@ build_lrouter_flows_for_lb(struct ovn_lb_datapaths *lb_dps, copp_meter_get(COPP_EVENT_ELB, od->nbr->copp, meter_groups), - &lb->nlb->header_); + &lb->nlb->header_, NULL); } } @@ -11421,7 +11577,8 @@ build_lrouter_flows_for_lb(struct ovn_lb_datapaths *lb_dps, struct ovn_datapath *od = lr_datapaths->array[index]; ovn_lflow_add(lflows, od, S_ROUTER_OUT_SNAT, 120, - "flags.skip_snat_for_lb == 1 && ip", "next;"); + "flags.skip_snat_for_lb == 1 && ip", "next;", + NULL); } } } @@ -11536,7 +11693,8 @@ static inline void lrouter_nat_add_ext_ip_match(const struct ovn_datapath *od, struct lflow_table *lflows, struct ds *match, const struct nbrec_nat *nat, - bool is_v6, bool is_src, int cidr_bits) + bool is_v6, bool is_src, int cidr_bits, + struct lflow_ref *lflow_ref) { struct nbrec_address_set *allowed_ext_ips = nat->allowed_ext_ips; struct nbrec_address_set *exempted_ext_ips = nat->exempted_ext_ips; @@ -11587,7 +11745,7 @@ lrouter_nat_add_ext_ip_match(const struct ovn_datapath *od, ovn_lflow_add_with_hint(lflows, od, stage, priority, ds_cstr(&match_exempt), "next;", - &nat->header_); + &nat->header_, lflow_ref); ds_destroy(&match_exempt); } } @@ -11601,7 +11759,8 @@ build_lrouter_arp_flow(const struct ovn_datapath *od, struct ovn_port *op, const char *ip_address, const char *eth_addr, struct ds *extra_match, bool drop, uint16_t priority, const struct ovsdb_idl_row *hint, - struct lflow_table *lflows) + struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { struct ds match = DS_EMPTY_INITIALIZER; struct ds actions = DS_EMPTY_INITIALIZER; @@ -11633,7 +11792,8 @@ build_lrouter_arp_flow(const struct ovn_datapath *od, struct ovn_port *op, } ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_IP_INPUT, priority, - ds_cstr(&match), ds_cstr(&actions), hint); + ds_cstr(&match), ds_cstr(&actions), hint, + lflow_ref); ds_destroy(&match); ds_destroy(&actions); @@ -11652,7 +11812,8 @@ build_lrouter_nd_flow(const struct ovn_datapath *od, struct ovn_port *op, struct ds *extra_match, bool drop, uint16_t priority, const struct ovsdb_idl_row *hint, struct lflow_table *lflows, - const struct shash *meter_groups) + const struct shash *meter_groups, + struct lflow_ref *lflow_ref) { struct ds match = DS_EMPTY_INITIALIZER; struct ds actions = DS_EMPTY_INITIALIZER; @@ -11675,7 +11836,8 @@ build_lrouter_nd_flow(const struct ovn_datapath *od, struct ovn_port *op, if (drop) { ds_put_cstr(&actions, debug_drop_action()); ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_IP_INPUT, priority, - ds_cstr(&match), ds_cstr(&actions), hint); + ds_cstr(&match), ds_cstr(&actions), hint, + lflow_ref); } else { ds_put_format(&actions, "%s { " @@ -11693,7 +11855,7 @@ build_lrouter_nd_flow(const struct ovn_datapath *od, struct ovn_port *op, ds_cstr(&match), ds_cstr(&actions), NULL, copp_meter_get(COPP_ND_NA, od->nbr->copp, meter_groups), - hint); + hint, lflow_ref); } ds_destroy(&match); @@ -11704,7 +11866,8 @@ static void build_lrouter_nat_arp_nd_flow(const struct ovn_datapath *od, struct ovn_nat *nat_entry, struct lflow_table *lflows, - const struct shash *meter_groups) + const struct shash *meter_groups, + struct lflow_ref *lflow_ref) { struct lport_addresses *ext_addrs = &nat_entry->ext_addrs; const struct nbrec_nat *nat = nat_entry->nb; @@ -11714,12 +11877,14 @@ build_lrouter_nat_arp_nd_flow(const struct ovn_datapath *od, ext_addrs->ipv6_addrs[0].addr_s, ext_addrs->ipv6_addrs[0].sn_addr_s, REG_INPORT_ETH_ADDR, NULL, false, 90, - &nat->header_, lflows, meter_groups); + &nat->header_, lflows, meter_groups, + lflow_ref); } else { build_lrouter_arp_flow(od, NULL, ext_addrs->ipv4_addrs[0].addr_s, REG_INPORT_ETH_ADDR, NULL, false, 90, - &nat->header_, lflows); + &nat->header_, lflows, + lflow_ref); } } @@ -11727,7 +11892,8 @@ static void build_lrouter_port_nat_arp_nd_flow(struct ovn_port *op, struct ovn_nat *nat_entry, struct lflow_table *lflows, - const struct shash *meter_groups) + const struct shash *meter_groups, + struct lflow_ref *lflow_ref) { struct lport_addresses *ext_addrs = &nat_entry->ext_addrs; const struct nbrec_nat *nat = nat_entry->nb; @@ -11775,21 +11941,25 @@ build_lrouter_port_nat_arp_nd_flow(struct ovn_port *op, ext_addrs->ipv6_addrs[0].addr_s, ext_addrs->ipv6_addrs[0].sn_addr_s, mac_s, &match, false, 92, - &nat->header_, lflows, meter_groups); + &nat->header_, lflows, meter_groups, + lflow_ref); build_lrouter_nd_flow(op->od, op, "nd_na", ext_addrs->ipv6_addrs[0].addr_s, ext_addrs->ipv6_addrs[0].sn_addr_s, mac_s, NULL, true, 91, - &nat->header_, lflows, meter_groups); + &nat->header_, lflows, meter_groups, + lflow_ref); } else { build_lrouter_arp_flow(op->od, op, ext_addrs->ipv4_addrs[0].addr_s, mac_s, &match, false, 92, - &nat->header_, lflows); + &nat->header_, lflows, + lflow_ref); build_lrouter_arp_flow(op->od, op, ext_addrs->ipv4_addrs[0].addr_s, mac_s, NULL, true, 91, - &nat->header_, lflows); + &nat->header_, lflows, + lflow_ref); } ds_destroy(&match); @@ -11800,7 +11970,8 @@ build_lrouter_drop_own_dest(struct ovn_port *op, const struct lr_stateful_record *lr_stateful_rec, enum ovn_stage stage, uint16_t priority, bool drop_snat_ip, - struct lflow_table *lflows) + struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { struct ds match_ips = DS_EMPTY_INITIALIZER; @@ -11827,7 +11998,8 @@ build_lrouter_drop_own_dest(struct ovn_port *op, char *match = xasprintf("ip4.dst == {%s}", ds_cstr(&match_ips)); ovn_lflow_add_with_hint(lflows, op->od, stage, priority, match, debug_drop_action(), - &op->nbrp->header_); + &op->nbrp->header_, + lflow_ref); free(match); } } @@ -11857,7 +12029,8 @@ build_lrouter_drop_own_dest(struct ovn_port *op, char *match = xasprintf("ip6.dst == {%s}", ds_cstr(&match_ips)); ovn_lflow_add_with_hint(lflows, op->od, stage, priority, match, debug_drop_action(), - &op->nbrp->header_); + &op->nbrp->header_, + lflow_ref); free(match); } } @@ -11868,14 +12041,15 @@ static void build_lrouter_force_snat_flows(struct lflow_table *lflows, const struct ovn_datapath *od, const char *ip_version, const char *ip_addr, - const char *context) + const char *context, + struct lflow_ref *lflow_ref) { struct ds match = DS_EMPTY_INITIALIZER; struct ds actions = DS_EMPTY_INITIALIZER; ds_put_format(&match, "ip%s && ip%s.dst == %s", ip_version, ip_version, ip_addr); ovn_lflow_add(lflows, od, S_ROUTER_IN_UNSNAT, 110, - ds_cstr(&match), "ct_snat;"); + ds_cstr(&match), "ct_snat;", lflow_ref); /* Higher priority rules to force SNAT with the IP addresses * configured in the Gateway router. This only takes effect @@ -11885,7 +12059,8 @@ build_lrouter_force_snat_flows(struct lflow_table *lflows, context, ip_version); ds_put_format(&actions, "ct_snat(%s);", ip_addr); ovn_lflow_add(lflows, od, S_ROUTER_OUT_SNAT, 100, - ds_cstr(&match), ds_cstr(&actions)); + ds_cstr(&match), ds_cstr(&actions), + lflow_ref); ds_destroy(&match); ds_destroy(&actions); @@ -11895,7 +12070,8 @@ static void build_lrouter_force_snat_flows_op(struct ovn_port *op, const struct lr_nat_record *lrnat_rec, struct lflow_table *lflows, - struct ds *match, struct ds *actions) + struct ds *match, struct ds *actions, + struct lflow_ref *lflow_ref) { ovs_assert(op->nbrp); if (!op->peer || !lrnat_rec->lb_force_snat_router_ip) { @@ -11909,7 +12085,7 @@ build_lrouter_force_snat_flows_op(struct ovn_port *op, ds_put_format(match, "inport == %s && ip4.dst == %s", op->json_key, op->lrp_networks.ipv4_addrs[0].addr_s); ovn_lflow_add(lflows, op->od, S_ROUTER_IN_UNSNAT, 110, - ds_cstr(match), "ct_snat;"); + ds_cstr(match), "ct_snat;", lflow_ref); ds_clear(match); @@ -11921,7 +12097,8 @@ build_lrouter_force_snat_flows_op(struct ovn_port *op, ds_put_format(actions, "ct_snat(%s);", op->lrp_networks.ipv4_addrs[0].addr_s); ovn_lflow_add(lflows, op->od, S_ROUTER_OUT_SNAT, 110, - ds_cstr(match), ds_cstr(actions)); + ds_cstr(match), ds_cstr(actions), + lflow_ref); if (op->lrp_networks.n_ipv4_addrs > 1) { static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 5); VLOG_WARN_RL(&rl, "Logical router port %s is configured with " @@ -11941,7 +12118,7 @@ build_lrouter_force_snat_flows_op(struct ovn_port *op, ds_put_format(match, "inport == %s && ip6.dst == %s", op->json_key, op->lrp_networks.ipv6_addrs[0].addr_s); ovn_lflow_add(lflows, op->od, S_ROUTER_IN_UNSNAT, 110, - ds_cstr(match), "ct_snat;"); + ds_cstr(match), "ct_snat;", lflow_ref); ds_clear(match); @@ -11953,7 +12130,8 @@ build_lrouter_force_snat_flows_op(struct ovn_port *op, ds_put_format(actions, "ct_snat(%s);", op->lrp_networks.ipv6_addrs[0].addr_s); ovn_lflow_add(lflows, op->od, S_ROUTER_OUT_SNAT, 110, - ds_cstr(match), ds_cstr(actions)); + ds_cstr(match), ds_cstr(actions), + lflow_ref); if (op->lrp_networks.n_ipv6_addrs > 2) { static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 5); VLOG_WARN_RL(&rl, "Logical router port %s is configured with " @@ -11967,7 +12145,8 @@ build_lrouter_force_snat_flows_op(struct ovn_port *op, static void build_lrouter_bfd_flows(struct lflow_table *lflows, struct ovn_port *op, - const struct shash *meter_groups) + const struct shash *meter_groups, + struct lflow_ref *lflow_ref) { if (!op->has_bfd) { return; @@ -11982,7 +12161,8 @@ build_lrouter_bfd_flows(struct lflow_table *lflows, struct ovn_port *op, ds_cstr(&ip_list)); ovn_lflow_add_with_hint(lflows, op->od, S_ROUTER_IN_IP_INPUT, 110, ds_cstr(&match), "next; ", - &op->nbrp->header_); + &op->nbrp->header_, + lflow_ref); ds_clear(&match); ds_put_format(&match, "ip4.dst == %s && udp.dst == 3784", ds_cstr(&ip_list)); @@ -11990,7 +12170,8 @@ build_lrouter_bfd_flows(struct lflow_table *lflows, struct ovn_port *op, ds_cstr(&match), "handle_bfd_msg(); ", NULL, copp_meter_get(COPP_BFD, op->od->nbr->copp, meter_groups), - &op->nbrp->header_); + &op->nbrp->header_, + lflow_ref); } if (op->lrp_networks.n_ipv6_addrs) { ds_clear(&ip_list); @@ -12001,7 +12182,8 @@ build_lrouter_bfd_flows(struct lflow_table *lflows, struct ovn_port *op, ds_cstr(&ip_list)); ovn_lflow_add_with_hint(lflows, op->od, S_ROUTER_IN_IP_INPUT, 110, ds_cstr(&match), "next; ", - &op->nbrp->header_); + &op->nbrp->header_, + lflow_ref); ds_clear(&match); ds_put_format(&match, "ip6.dst == %s && udp.dst == 3784", ds_cstr(&ip_list)); @@ -12009,7 +12191,8 @@ build_lrouter_bfd_flows(struct lflow_table *lflows, struct ovn_port *op, ds_cstr(&match), "handle_bfd_msg(); ", NULL, copp_meter_get(COPP_BFD, op->od->nbr->copp, meter_groups), - &op->nbrp->header_); + &op->nbrp->header_, + lflow_ref); } ds_destroy(&ip_list); @@ -12021,16 +12204,19 @@ build_lrouter_bfd_flows(struct lflow_table *lflows, struct ovn_port *op, */ static void build_adm_ctrl_flows_for_lrouter( - struct ovn_datapath *od, struct lflow_table *lflows) + struct ovn_datapath *od, struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { ovs_assert(od->nbr); /* Logical VLANs not supported. * Broadcast/multicast source address is invalid. */ ovn_lflow_add(lflows, od, S_ROUTER_IN_ADMISSION, 100, - "vlan.present || eth.src[40]", debug_drop_action()); + "vlan.present || eth.src[40]", debug_drop_action(), + lflow_ref); /* Default action for L2 security is to drop. */ - ovn_lflow_add_default_drop(lflows, od, S_ROUTER_IN_ADMISSION); + ovn_lflow_add_default_drop(lflows, od, S_ROUTER_IN_ADMISSION, + lflow_ref); } static int @@ -12064,11 +12250,12 @@ build_gateway_get_l2_hdr_size(struct ovn_port *op) /* All 'gateway_mtu' and 'gateway_mtu_bypass' flows should be built with this * function. */ -static void OVS_PRINTF_FORMAT(9, 10) +static void OVS_PRINTF_FORMAT(10, 11) build_gateway_mtu_flow(struct lflow_table *lflows, struct ovn_port *op, enum ovn_stage stage, uint16_t prio_low, uint16_t prio_high, struct ds *match, struct ds *actions, const struct ovsdb_idl_row *hint, + struct lflow_ref *lflow_ref, const char *extra_actions_fmt, ...) { int gw_mtu = smap_get_int(&op->nbrp->options, "gateway_mtu", 0); @@ -12086,7 +12273,7 @@ build_gateway_mtu_flow(struct lflow_table *lflows, struct ovn_port *op, ds_put_format_valist(actions, extra_actions_fmt, extra_actions_args); ovn_lflow_add_with_hint(lflows, op->od, stage, prio_low, ds_cstr(match), ds_cstr(actions), - hint); + hint, lflow_ref); if (gw_mtu > 0) { const char *gw_mtu_bypass = smap_get(&op->nbrp->options, @@ -12098,7 +12285,7 @@ build_gateway_mtu_flow(struct lflow_table *lflows, struct ovn_port *op, ds_put_format(match, " && (%s)", gw_mtu_bypass); ovn_lflow_add_with_hint(lflows, op->od, stage, prio_high, ds_cstr(match), ds_cstr(actions), - hint); + hint, lflow_ref); } } va_end(extra_actions_args); @@ -12127,7 +12314,8 @@ consider_l3dgw_port_is_centralized(struct ovn_port *op) static void build_adm_ctrl_flows_for_lrouter_port( struct ovn_port *op, struct lflow_table *lflows, - struct ds *match, struct ds *actions) + struct ds *match, struct ds *actions, + struct lflow_ref *lflow_ref) { ovs_assert(op->nbrp); @@ -12151,6 +12339,7 @@ build_adm_ctrl_flows_for_lrouter_port( ds_put_format(match, "eth.mcast && inport == %s", op->json_key); build_gateway_mtu_flow(lflows, op, S_ROUTER_IN_ADMISSION, 50, 55, match, actions, &op->nbrp->header_, + lflow_ref, REG_INPORT_ETH_ADDR " = %s; next;", op->lrp_networks.ea_s); @@ -12171,6 +12360,7 @@ build_adm_ctrl_flows_for_lrouter_port( } build_gateway_mtu_flow(lflows, op, S_ROUTER_IN_ADMISSION, 50, 55, match, actions, &op->nbrp->header_, + lflow_ref, REG_INPORT_ETH_ADDR " = %s; next;", op->lrp_networks.ea_s); } @@ -12182,7 +12372,8 @@ static void build_neigh_learning_flows_for_lrouter( struct ovn_datapath *od, struct lflow_table *lflows, struct ds *match, struct ds *actions, - const struct shash *meter_groups) + const struct shash *meter_groups, + struct lflow_ref *lflow_ref) { ovs_assert(od->nbr); @@ -12229,7 +12420,7 @@ build_neigh_learning_flows_for_lrouter( learn_from_arp_request ? "" : REGBIT_LOOKUP_NEIGHBOR_IP_RESULT" = 1; "); ovn_lflow_add(lflows, od, S_ROUTER_IN_LOOKUP_NEIGHBOR, 100, - "arp.op == 2", ds_cstr(actions)); + "arp.op == 2", ds_cstr(actions), lflow_ref); ds_clear(actions); ds_put_format(actions, REGBIT_LOOKUP_NEIGHBOR_RESULT @@ -12237,7 +12428,7 @@ build_neigh_learning_flows_for_lrouter( learn_from_arp_request ? "" : REGBIT_LOOKUP_NEIGHBOR_IP_RESULT" = 1; "); ovn_lflow_add(lflows, od, S_ROUTER_IN_LOOKUP_NEIGHBOR, 100, "nd_na", - ds_cstr(actions)); + ds_cstr(actions), lflow_ref); if (!learn_from_arp_request) { /* Add flow to skip GARP LLA if we don't know it already. @@ -12254,7 +12445,7 @@ build_neigh_learning_flows_for_lrouter( " = lookup_nd_ip(inport, ip6.src); next;"); ovn_lflow_add(lflows, od, S_ROUTER_IN_LOOKUP_NEIGHBOR, 110, "nd_na && ip6.src == fe80::/10 && ip6.dst == ff00::/8", - ds_cstr(actions)); + ds_cstr(actions), lflow_ref); } ds_clear(actions); @@ -12264,12 +12455,13 @@ build_neigh_learning_flows_for_lrouter( REGBIT_LOOKUP_NEIGHBOR_IP_RESULT " = lookup_nd_ip(inport, ip6.src); "); ovn_lflow_add(lflows, od, S_ROUTER_IN_LOOKUP_NEIGHBOR, 100, "nd_ns", - ds_cstr(actions)); + ds_cstr(actions), lflow_ref); /* For other packet types, we can skip neighbor learning. * So set REGBIT_LOOKUP_NEIGHBOR_RESULT to 1. */ ovn_lflow_add(lflows, od, S_ROUTER_IN_LOOKUP_NEIGHBOR, 0, "1", - REGBIT_LOOKUP_NEIGHBOR_RESULT" = 1; next;"); + REGBIT_LOOKUP_NEIGHBOR_RESULT" = 1; next;", + lflow_ref); /* Flows for LEARN_NEIGHBOR. */ /* Skip Neighbor learning if not required. */ @@ -12278,33 +12470,40 @@ build_neigh_learning_flows_for_lrouter( learn_from_arp_request ? "" : " || "REGBIT_LOOKUP_NEIGHBOR_IP_RESULT" == 0"); ovn_lflow_add(lflows, od, S_ROUTER_IN_LEARN_NEIGHBOR, 100, - ds_cstr(match), "mac_cache_use; next;"); + ds_cstr(match), "mac_cache_use; next;", + lflow_ref); ovn_lflow_metered(lflows, od, S_ROUTER_IN_LEARN_NEIGHBOR, 90, "arp", "put_arp(inport, arp.spa, arp.sha); next;", copp_meter_get(COPP_ARP, od->nbr->copp, - meter_groups)); + meter_groups), + lflow_ref); ovn_lflow_add(lflows, od, S_ROUTER_IN_LEARN_NEIGHBOR, 95, - "nd_ns && (ip6.src == 0 || nd.sll == 0)", "next;"); + "nd_ns && (ip6.src == 0 || nd.sll == 0)", "next;", + lflow_ref); ovn_lflow_metered(lflows, od, S_ROUTER_IN_LEARN_NEIGHBOR, 95, "nd_na && nd.tll == 0", "put_nd(inport, nd.target, eth.src); next;", copp_meter_get(COPP_ND_NA, od->nbr->copp, - meter_groups)); + meter_groups), + lflow_ref); ovn_lflow_metered(lflows, od, S_ROUTER_IN_LEARN_NEIGHBOR, 90, "nd_na", "put_nd(inport, nd.target, nd.tll); next;", copp_meter_get(COPP_ND_NA, od->nbr->copp, - meter_groups)); + meter_groups), + lflow_ref); ovn_lflow_metered(lflows, od, S_ROUTER_IN_LEARN_NEIGHBOR, 90, "nd_ns", "put_nd(inport, ip6.src, nd.sll); next;", copp_meter_get(COPP_ND_NS, od->nbr->copp, - meter_groups)); + meter_groups), + lflow_ref); - ovn_lflow_add_default_drop(lflows, od, S_ROUTER_IN_LEARN_NEIGHBOR); + ovn_lflow_add_default_drop(lflows, od, S_ROUTER_IN_LEARN_NEIGHBOR, + lflow_ref); } /* Logical router ingress Table 1: Neighbor lookup lflows @@ -12312,7 +12511,8 @@ build_neigh_learning_flows_for_lrouter( static void build_neigh_learning_flows_for_lrouter_port( struct ovn_port *op, struct lflow_table *lflows, - struct ds *match, struct ds *actions) + struct ds *match, struct ds *actions, + struct lflow_ref *lflow_ref) { ovs_assert(op->nbrp); @@ -12344,7 +12544,8 @@ build_neigh_learning_flows_for_lrouter_port( ovn_lflow_add_with_hint(lflows, op->od, S_ROUTER_IN_LOOKUP_NEIGHBOR, 110, ds_cstr(match), actions_s, - &op->nbrp->header_); + &op->nbrp->header_, + lflow_ref); } ds_clear(match); ds_put_format(match, @@ -12365,7 +12566,8 @@ build_neigh_learning_flows_for_lrouter_port( ovn_lflow_add_with_hint(lflows, op->od, S_ROUTER_IN_LOOKUP_NEIGHBOR, 100, ds_cstr(match), ds_cstr(actions), - &op->nbrp->header_); + &op->nbrp->header_, + lflow_ref); } } @@ -12375,7 +12577,8 @@ static void build_ND_RA_flows_for_lrouter_port( struct ovn_port *op, struct lflow_table *lflows, struct ds *match, struct ds *actions, - const struct shash *meter_groups) + const struct shash *meter_groups, + struct lflow_ref *lflow_ref) { ovs_assert(op->nbrp); if (op->nbrp->peer || !op->peer) { @@ -12462,7 +12665,8 @@ build_ND_RA_flows_for_lrouter_port( copp_meter_get(COPP_ND_RA_OPTS, op->od->nbr->copp, meter_groups), - &op->nbrp->header_); + &op->nbrp->header_, + lflow_ref); ds_clear(actions); ds_clear(match); ds_put_format(match, "inport == %s && ip6.dst == ff02::2 && " @@ -12481,7 +12685,8 @@ build_ND_RA_flows_for_lrouter_port( ovn_lflow_add_with_hint(lflows, op->od, S_ROUTER_IN_ND_RA_RESPONSE, 50, ds_cstr(match), ds_cstr(actions), - &op->nbrp->header_); + &op->nbrp->header_, + lflow_ref); } } @@ -12489,22 +12694,26 @@ build_ND_RA_flows_for_lrouter_port( * responder, by default goto next. (priority 0). */ static void build_ND_RA_flows_for_lrouter(struct ovn_datapath *od, - struct lflow_table *lflows) + struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { ovs_assert(od->nbr); - ovn_lflow_add(lflows, od, S_ROUTER_IN_ND_RA_OPTIONS, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_ROUTER_IN_ND_RA_RESPONSE, 0, "1", "next;"); + ovn_lflow_add(lflows, od, S_ROUTER_IN_ND_RA_OPTIONS, 0, "1", "next;", + lflow_ref); + ovn_lflow_add(lflows, od, S_ROUTER_IN_ND_RA_RESPONSE, 0, "1", "next;", + lflow_ref); } /* Logical router ingress table IP_ROUTING_PRE: * by default goto next. (priority 0). */ static void build_ip_routing_pre_flows_for_lrouter(struct ovn_datapath *od, - struct lflow_table *lflows) + struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { ovs_assert(od->nbr); ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_ROUTING_PRE, 0, "1", - REG_ROUTE_TABLE_ID" = 0; next;"); + REG_ROUTE_TABLE_ID" = 0; next;", lflow_ref); } /* Logical router ingress table IP_ROUTING : IP Routing. @@ -12528,7 +12737,8 @@ build_ip_routing_pre_flows_for_lrouter(struct ovn_datapath *od, */ static void build_ip_routing_flows_for_lrp( - struct ovn_port *op, struct lflow_table *lflows) + struct ovn_port *op, struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { ovs_assert(op->nbrp); for (int i = 0; i < op->lrp_networks.n_ipv4_addrs; i++) { @@ -12536,7 +12746,7 @@ build_ip_routing_flows_for_lrp( op->lrp_networks.ipv4_addrs[i].network_s, op->lrp_networks.ipv4_addrs[i].plen, NULL, false, 0, &op->nbrp->header_, false, ROUTE_PRIO_OFFSET_CONNECTED, - NULL); + lflow_ref); } for (int i = 0; i < op->lrp_networks.n_ipv6_addrs; i++) { @@ -12544,7 +12754,7 @@ build_ip_routing_flows_for_lrp( op->lrp_networks.ipv6_addrs[i].network_s, op->lrp_networks.ipv6_addrs[i].plen, NULL, false, 0, &op->nbrp->header_, false, ROUTE_PRIO_OFFSET_CONNECTED, - NULL); + lflow_ref); } } @@ -12609,13 +12819,17 @@ static void build_static_route_flows_for_lrouter( struct ovn_datapath *od, const struct chassis_features *features, struct lflow_table *lflows, const struct hmap *lr_ports, - const struct hmap *bfd_connections) + const struct hmap *bfd_connections, + struct lflow_ref *lflow_ref) { ovs_assert(od->nbr); - ovn_lflow_add_default_drop(lflows, od, S_ROUTER_IN_IP_ROUTING_ECMP); - ovn_lflow_add_default_drop(lflows, od, S_ROUTER_IN_IP_ROUTING); + ovn_lflow_add_default_drop(lflows, od, S_ROUTER_IN_IP_ROUTING_ECMP, + lflow_ref); + ovn_lflow_add_default_drop(lflows, od, S_ROUTER_IN_IP_ROUTING, + lflow_ref); ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_ROUTING_ECMP, 150, - REG_ECMP_GROUP_ID" == 0", "next;"); + REG_ECMP_GROUP_ID" == 0", "next;", + lflow_ref); struct hmap ecmp_groups = HMAP_INITIALIZER(&ecmp_groups); struct hmap unique_routes = HMAP_INITIALIZER(&unique_routes); @@ -12625,7 +12839,7 @@ build_static_route_flows_for_lrouter( for (int i = 0; i < od->nbr->n_ports; i++) { build_route_table_lflow(od, lflows, od->nbr->ports[i], - &route_tables); + &route_tables, lflow_ref); } for (int i = 0; i < od->nbr->n_static_routes; i++) { @@ -12655,11 +12869,11 @@ build_static_route_flows_for_lrouter( /* add a flow in IP_ROUTING, and one flow for each member in * IP_ROUTING_ECMP. */ build_ecmp_route_flow(lflows, od, features->ct_no_masked_label, - lr_ports, group); + lr_ports, group, lflow_ref); } const struct unique_routes_node *ur; HMAP_FOR_EACH (ur, hmap_node, &unique_routes) { - build_static_route_flow(lflows, od, lr_ports, ur->route); + build_static_route_flow(lflows, od, lr_ports, ur->route, lflow_ref); } ecmp_groups_destroy(&ecmp_groups); unique_routes_destroy(&unique_routes); @@ -12673,7 +12887,8 @@ build_static_route_flows_for_lrouter( static void build_mcast_lookup_flows_for_lrouter( struct ovn_datapath *od, struct lflow_table *lflows, - struct ds *match, struct ds *actions) + struct ds *match, struct ds *actions, + struct lflow_ref *lflow_ref) { ovs_assert(od->nbr); @@ -12681,7 +12896,8 @@ build_mcast_lookup_flows_for_lrouter( * i.e., router solicitation and router advertisement. */ ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_ROUTING, 10550, - "nd_rs || nd_ra", debug_drop_action()); + "nd_rs || nd_ra", debug_drop_action(), + lflow_ref); if (!od->mcast_info.rtr.relay) { return; } @@ -12709,7 +12925,8 @@ build_mcast_lookup_flows_for_lrouter( ds_put_format(actions, "outport = \"%s\"; ip.ttl--; next;", igmp_group->mcgroup.name); ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_ROUTING, 10500, - ds_cstr(match), ds_cstr(actions)); + ds_cstr(match), ds_cstr(actions), + lflow_ref); } /* If needed, flood unregistered multicast on statically configured @@ -12728,13 +12945,15 @@ build_mcast_lookup_flows_for_lrouter( ds_put_format(match, "eth.src == %s && igmp", op->lrp_networks.ea_s); ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_ROUTING, 10550, - ds_cstr(match), debug_drop_action()); + ds_cstr(match), debug_drop_action(), + lflow_ref); ds_clear(match); ds_put_format(match, "eth.src == %s && (mldv1 || mldv2)", op->lrp_networks.ea_s); ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_ROUTING, 10550, - ds_cstr(match), debug_drop_action()); + ds_cstr(match), debug_drop_action(), + lflow_ref); } ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_ROUTING, 10460, @@ -12742,23 +12961,27 @@ build_mcast_lookup_flows_for_lrouter( "clone { " "outport = \""MC_STATIC"\"; " "next; " - "};"); + "};", + lflow_ref); ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_ROUTING, 10460, "mldv1 || mldv2", "clone { " "outport = \""MC_STATIC"\"; " "next; " - "};"); + "};", + lflow_ref); ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_ROUTING, 10450, "ip4.mcast || ip6.mcast", "clone { " "outport = \""MC_STATIC"\"; " "ip.ttl--; " "next; " - "};"); + "};", + lflow_ref); } else { ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_ROUTING, 10450, - "ip4.mcast || ip6.mcast", debug_drop_action()); + "ip4.mcast || ip6.mcast", debug_drop_action(), + lflow_ref); } } @@ -12774,16 +12997,20 @@ build_mcast_lookup_flows_for_lrouter( static void build_ingress_policy_flows_for_lrouter( struct ovn_datapath *od, struct lflow_table *lflows, - const struct hmap *lr_ports) + const struct hmap *lr_ports, + struct lflow_ref *lflow_ref) { ovs_assert(od->nbr); /* This is a catch-all rule. It has the lowest priority (0) * does a match-all("1") and pass-through (next) */ ovn_lflow_add(lflows, od, S_ROUTER_IN_POLICY, 0, "1", - REG_ECMP_GROUP_ID" = 0; next;"); + REG_ECMP_GROUP_ID" = 0; next;", + lflow_ref); ovn_lflow_add(lflows, od, S_ROUTER_IN_POLICY_ECMP, 150, - REG_ECMP_GROUP_ID" == 0", "next;"); - ovn_lflow_add_default_drop(lflows, od, S_ROUTER_IN_POLICY_ECMP); + REG_ECMP_GROUP_ID" == 0", "next;", + lflow_ref); + ovn_lflow_add_default_drop(lflows, od, S_ROUTER_IN_POLICY_ECMP, + lflow_ref); /* Convert routing policies to flows. */ uint16_t ecmp_group_id = 1; @@ -12795,11 +13022,11 @@ build_ingress_policy_flows_for_lrouter( if (is_ecmp_reroute) { build_ecmp_routing_policy_flows(lflows, od, lr_ports, rule, - ecmp_group_id); + ecmp_group_id, lflow_ref); ecmp_group_id++; } else { build_routing_policy_flow(lflows, od, lr_ports, rule, - &rule->header_); + &rule->header_, lflow_ref); } } } @@ -12807,21 +13034,26 @@ build_ingress_policy_flows_for_lrouter( /* Local router ingress table ARP_RESOLVE: ARP Resolution. */ static void build_arp_resolve_flows_for_lrouter( - struct ovn_datapath *od, struct lflow_table *lflows) + struct ovn_datapath *od, struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { ovs_assert(od->nbr); /* Multicast packets already have the outport set so just advance to * next table (priority 500). */ ovn_lflow_add(lflows, od, S_ROUTER_IN_ARP_RESOLVE, 500, - "ip4.mcast || ip6.mcast", "next;"); + "ip4.mcast || ip6.mcast", "next;", + lflow_ref); ovn_lflow_add(lflows, od, S_ROUTER_IN_ARP_RESOLVE, 1, "ip4", - "get_arp(outport, " REG_NEXT_HOP_IPV4 "); next;"); + "get_arp(outport, " REG_NEXT_HOP_IPV4 "); next;", + lflow_ref); ovn_lflow_add(lflows, od, S_ROUTER_IN_ARP_RESOLVE, 1, "ip6", - "get_nd(outport, " REG_NEXT_HOP_IPV6 "); next;"); + "get_nd(outport, " REG_NEXT_HOP_IPV6 "); next;", + lflow_ref); - ovn_lflow_add_default_drop(lflows, od, S_ROUTER_IN_ARP_RESOLVE); + ovn_lflow_add_default_drop(lflows, od, S_ROUTER_IN_ARP_RESOLVE, + lflow_ref); } static void @@ -12854,9 +13086,8 @@ routable_addresses_to_lflows(struct lflow_table *lflows, ds_clear(actions); ds_put_format(actions, "eth.dst = %s; next;", ra.laddrs[i].ea_s); - ovn_lflow_add_with_lflow_ref(lflows, peer->od, S_ROUTER_IN_ARP_RESOLVE, - 100, ds_cstr(match), ds_cstr(actions), - lflow_ref); + ovn_lflow_add(lflows, peer->od, S_ROUTER_IN_ARP_RESOLVE, 100, + ds_cstr(match), ds_cstr(actions), lflow_ref); } destroy_routable_addresses(&ra); } @@ -12875,7 +13106,8 @@ routable_addresses_to_lflows(struct lflow_table *lflows, static void build_arp_resolve_flows_for_lrp(struct ovn_port *op, struct lflow_table *lflows, - struct ds *match, struct ds *actions) + struct ds *match, struct ds *actions, + struct lflow_ref *lflow_ref) { ovs_assert(op->nbrp); /* This is a logical router port. If next-hop IP address in @@ -12900,7 +13132,8 @@ build_arp_resolve_flows_for_lrp(struct ovn_port *op, ovn_lflow_add_with_hint(lflows, op->peer->od, S_ROUTER_IN_ARP_RESOLVE, 100, ds_cstr(match), ds_cstr(actions), - &op->nbrp->header_); + &op->nbrp->header_, + lflow_ref); } if (op->lrp_networks.n_ipv6_addrs) { @@ -12916,7 +13149,8 @@ build_arp_resolve_flows_for_lrp(struct ovn_port *op, ovn_lflow_add_with_hint(lflows, op->peer->od, S_ROUTER_IN_ARP_RESOLVE, 100, ds_cstr(match), ds_cstr(actions), - &op->nbrp->header_); + &op->nbrp->header_, + lflow_ref); } } @@ -12941,7 +13175,8 @@ build_arp_resolve_flows_for_lrp(struct ovn_port *op, ovn_lflow_add_with_hint(lflows, op->od, S_ROUTER_IN_ARP_RESOLVE, 50, ds_cstr(match), ds_cstr(actions), - &op->nbrp->header_); + &op->nbrp->header_, + lflow_ref); } } } @@ -12991,7 +13226,7 @@ build_arp_resolve_flows_for_lsp( ds_clear(actions); ds_put_format(actions, "eth.dst = %s; next;", ea_s); - ovn_lflow_add_with_lflow_ref_hint(lflows, peer->od, + ovn_lflow_add_with_hint(lflows, peer->od, S_ROUTER_IN_ARP_RESOLVE, 100, ds_cstr(match), ds_cstr(actions), @@ -13023,7 +13258,7 @@ build_arp_resolve_flows_for_lsp( ds_clear(actions); ds_put_format(actions, "eth.dst = %s; next;", ea_s); - ovn_lflow_add_with_lflow_ref_hint(lflows, peer->od, + ovn_lflow_add_with_hint(lflows, peer->od, S_ROUTER_IN_ARP_RESOLVE, 100, ds_cstr(match), ds_cstr(actions), @@ -13072,7 +13307,7 @@ build_arp_resolve_flows_for_lsp( ds_clear(actions); ds_put_format(actions, "eth.dst = %s; next;", router_port->lrp_networks.ea_s); - ovn_lflow_add_with_lflow_ref_hint(lflows, peer->od, + ovn_lflow_add_with_hint(lflows, peer->od, S_ROUTER_IN_ARP_RESOLVE, 100, ds_cstr(match), ds_cstr(actions), &op->nbsp->header_, @@ -13089,7 +13324,7 @@ build_arp_resolve_flows_for_lsp( ds_clear(actions); ds_put_format(actions, "eth.dst = %s; next;", router_port->lrp_networks.ea_s); - ovn_lflow_add_with_lflow_ref_hint(lflows, peer->od, + ovn_lflow_add_with_hint(lflows, peer->od, S_ROUTER_IN_ARP_RESOLVE, 100, ds_cstr(match), ds_cstr(actions), &op->nbsp->header_, @@ -13150,7 +13385,8 @@ build_icmperr_pkt_big_flows(struct ovn_port *op, int mtu, struct lflow_table *lflows, const struct shash *meter_groups, struct ds *match, struct ds *actions, enum ovn_stage stage, - struct ovn_port *outport) + struct ovn_port *outport, + struct lflow_ref *lflow_ref) { char *outport_match = outport ? xasprintf("outport == %s && ", outport->json_key) @@ -13193,7 +13429,8 @@ build_icmperr_pkt_big_flows(struct ovn_port *op, int mtu, COPP_ICMP4_ERR, op->od->nbr->copp, meter_groups), - &op->nbrp->header_); + &op->nbrp->header_, + lflow_ref); } char *ip6_src = NULL; @@ -13233,7 +13470,8 @@ build_icmperr_pkt_big_flows(struct ovn_port *op, int mtu, COPP_ICMP6_ERR, op->od->nbr->copp, meter_groups), - &op->nbrp->header_); + &op->nbrp->header_, + lflow_ref); } free(outport_match); } @@ -13244,7 +13482,8 @@ build_check_pkt_len_flows_for_lrp(struct ovn_port *op, const struct hmap *lr_ports, const struct shash *meter_groups, struct ds *match, - struct ds *actions) + struct ds *actions, + struct lflow_ref *lflow_ref) { int gw_mtu = smap_get_int(&op->nbrp->options, "gateway_mtu", 0); if (gw_mtu <= 0) { @@ -13254,12 +13493,13 @@ build_check_pkt_len_flows_for_lrp(struct ovn_port *op, ds_clear(match); ds_put_format(match, "outport == %s", op->json_key); build_gateway_mtu_flow(lflows, op, S_ROUTER_IN_CHK_PKT_LEN, 50, 55, - match, actions, &op->nbrp->header_, "next;"); + match, actions, &op->nbrp->header_, + lflow_ref, "next;"); /* ingress traffic */ build_icmperr_pkt_big_flows(op, gw_mtu, lflows, meter_groups, match, actions, S_ROUTER_IN_IP_INPUT, - NULL); + NULL, lflow_ref); for (size_t i = 0; i < op->od->nbr->n_ports; i++) { struct ovn_port *rp = ovn_port_find(lr_ports, @@ -13271,7 +13511,7 @@ build_check_pkt_len_flows_for_lrp(struct ovn_port *op, /* egress traffic */ build_icmperr_pkt_big_flows(rp, gw_mtu, lflows, meter_groups, match, actions, S_ROUTER_IN_LARGER_PKTS, - op); + op, lflow_ref); } } @@ -13293,15 +13533,16 @@ build_check_pkt_len_flows_for_lrouter( struct ovn_datapath *od, struct lflow_table *lflows, const struct hmap *lr_ports, struct ds *match, struct ds *actions, - const struct shash *meter_groups) + const struct shash *meter_groups, + struct lflow_ref *lflow_ref) { ovs_assert(od->nbr); /* Packets are allowed by default. */ ovn_lflow_add(lflows, od, S_ROUTER_IN_CHK_PKT_LEN, 0, "1", - "next;"); + "next;", lflow_ref); ovn_lflow_add(lflows, od, S_ROUTER_IN_LARGER_PKTS, 0, "1", - "next;"); + "next;", lflow_ref); for (size_t i = 0; i < od->nbr->n_ports; i++) { struct ovn_port *rp = ovn_port_find(lr_ports, @@ -13310,7 +13551,7 @@ build_check_pkt_len_flows_for_lrouter( continue; } build_check_pkt_len_flows_for_lrp(rp, lflows, lr_ports, meter_groups, - match, actions); + match, actions, lflow_ref); } } @@ -13318,7 +13559,8 @@ build_check_pkt_len_flows_for_lrouter( static void build_gateway_redirect_flows_for_lrouter( struct ovn_datapath *od, struct lflow_table *lflows, - struct ds *match, struct ds *actions) + struct ds *match, struct ds *actions, + struct lflow_ref *lflow_ref) { ovs_assert(od->nbr); for (size_t i = 0; i < od->n_l3dgw_ports; i++) { @@ -13351,18 +13593,20 @@ build_gateway_redirect_flows_for_lrouter( od->l3dgw_ports[i]->cr_port->json_key); ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_GW_REDIRECT, 50, ds_cstr(match), ds_cstr(actions), - stage_hint); + stage_hint, lflow_ref); } /* Packets are allowed by default. */ - ovn_lflow_add(lflows, od, S_ROUTER_IN_GW_REDIRECT, 0, "1", "next;"); + ovn_lflow_add(lflows, od, S_ROUTER_IN_GW_REDIRECT, 0, "1", "next;", + lflow_ref); } /* Logical router ingress table GW_REDIRECT: Gateway redirect. */ static void build_lr_gateway_redirect_flows_for_nats( const struct ovn_datapath *od, const struct lr_nat_record *lrnat_rec, - struct lflow_table *lflows, struct ds *match, struct ds *actions) + struct lflow_table *lflows, struct ds *match, struct ds *actions, + struct lflow_ref *lflow_ref) { ovs_assert(od->nbr); for (size_t i = 0; i < od->n_l3dgw_ports; i++) { @@ -13403,21 +13647,23 @@ build_lr_gateway_redirect_flows_for_nats( if (nat->nb->allowed_ext_ips) { ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_GW_REDIRECT, 75, ds_cstr(&match_ext), - ds_cstr(actions), stage_hint); + ds_cstr(actions), stage_hint, + lflow_ref); if (add_def_flow) { ds_clear(&match_ext); ds_put_format(&match_ext, "ip && ip%s.dst == %s", nat_entry_is_v6(nat) ? "6" : "4", nat->nb->external_ip); ovn_lflow_add(lflows, od, S_ROUTER_IN_GW_REDIRECT, 70, - ds_cstr(&match_ext), debug_drop_action()); + ds_cstr(&match_ext), debug_drop_action(), + lflow_ref); add_def_flow = false; } } else if (nat->nb->exempted_ext_ips) { ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_GW_REDIRECT, 75, ds_cstr(&match_ext), debug_drop_action(), - stage_hint); + stage_hint, lflow_ref); } ds_destroy(&match_ext); } @@ -13433,7 +13679,8 @@ static void build_arp_request_flows_for_lrouter( struct ovn_datapath *od, struct lflow_table *lflows, struct ds *match, struct ds *actions, - const struct shash *meter_groups) + const struct shash *meter_groups, + struct lflow_ref *lflow_ref) { ovs_assert(od->nbr); for (int i = 0; i < od->nbr->n_static_routes; i++) { @@ -13475,7 +13722,8 @@ build_arp_request_flows_for_lrouter( copp_meter_get(COPP_ND_NS_RESOLVE, od->nbr->copp, meter_groups), - &route->header_); + &route->header_, + lflow_ref); } ovn_lflow_metered(lflows, od, S_ROUTER_IN_ARP_REQUEST, 100, @@ -13488,7 +13736,8 @@ build_arp_request_flows_for_lrouter( "output; " "};", copp_meter_get(COPP_ARP_RESOLVE, od->nbr->copp, - meter_groups)); + meter_groups), + lflow_ref); ovn_lflow_metered(lflows, od, S_ROUTER_IN_ARP_REQUEST, 100, "eth.dst == 00:00:00:00:00:00 && ip6", "nd_ns { " @@ -13496,8 +13745,10 @@ build_arp_request_flows_for_lrouter( "output; " "};", copp_meter_get(COPP_ND_NS_RESOLVE, od->nbr->copp, - meter_groups)); - ovn_lflow_add(lflows, od, S_ROUTER_IN_ARP_REQUEST, 0, "1", "output;"); + meter_groups), + lflow_ref); + ovn_lflow_add(lflows, od, S_ROUTER_IN_ARP_REQUEST, 0, "1", "output;", + lflow_ref); } /* Logical router egress table DELIVERY: Delivery (priority 100-110). @@ -13510,7 +13761,8 @@ build_arp_request_flows_for_lrouter( static void build_egress_delivery_flows_for_lrouter_port( struct ovn_port *op, struct lflow_table *lflows, - struct ds *match, struct ds *actions) + struct ds *match, struct ds *actions, + struct lflow_ref *lflow_ref) { ovs_assert(op->nbrp); if (!lrport_is_enabled(op->nbrp)) { @@ -13538,20 +13790,23 @@ build_egress_delivery_flows_for_lrouter_port( ds_put_format(actions, "eth.src = %s; output;", op->lrp_networks.ea_s); ovn_lflow_add(lflows, op->od, S_ROUTER_OUT_DELIVERY, 110, - ds_cstr(match), ds_cstr(actions)); + ds_cstr(match), ds_cstr(actions), + lflow_ref); } ds_clear(match); ds_put_format(match, "outport == %s", op->json_key); ovn_lflow_add(lflows, op->od, S_ROUTER_OUT_DELIVERY, 100, - ds_cstr(match), "output;"); + ds_cstr(match), "output;", lflow_ref); - ovn_lflow_add_default_drop(lflows, op->od, S_ROUTER_OUT_DELIVERY); + ovn_lflow_add_default_drop(lflows, op->od, S_ROUTER_OUT_DELIVERY, + lflow_ref); } static void build_misc_local_traffic_drop_flows_for_lrouter( - struct ovn_datapath *od, struct lflow_table *lflows) + struct ovn_datapath *od, struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { ovs_assert(od->nbr); /* Allow IGMP and MLD packets (with TTL = 1) if the router is @@ -13559,9 +13814,11 @@ build_misc_local_traffic_drop_flows_for_lrouter( */ if (od->mcast_info.rtr.flood_static) { ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_INPUT, 120, - "igmp && ip.ttl == 1", "next;"); + "igmp && ip.ttl == 1", "next;", + lflow_ref); ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_INPUT, 120, - "(mldv1 || mldv2) && ip.ttl == 1", "next;"); + "(mldv1 || mldv2) && ip.ttl == 1", "next;", + lflow_ref); } /* L3 admission control: drop multicast and broadcast source, localhost @@ -13574,7 +13831,8 @@ build_misc_local_traffic_drop_flows_for_lrouter( "ip4.dst == 127.0.0.0/8 || " "ip4.src == 0.0.0.0/8 || " "ip4.dst == 0.0.0.0/8", - debug_drop_action()); + debug_drop_action(), + lflow_ref); /* Drop ARP packets (priority 85). ARP request packets for router's own * IPs are handled with priority-90 flows. @@ -13582,28 +13840,32 @@ build_misc_local_traffic_drop_flows_for_lrouter( * IPs are handled with priority-90 flows. */ ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_INPUT, 85, - "arp || nd", debug_drop_action()); + "arp || nd", debug_drop_action(), + lflow_ref); /* Allow IPv6 multicast traffic that's supposed to reach the * router pipeline (e.g., router solicitations). */ ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_INPUT, 84, "nd_rs || nd_ra", - "next;"); + "next;", lflow_ref); /* Drop other reserved multicast. */ ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_INPUT, 83, - "ip6.mcast_rsvd", debug_drop_action()); + "ip6.mcast_rsvd", debug_drop_action(), + lflow_ref); /* Allow other multicast if relay enabled (priority 82). */ ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_INPUT, 82, "ip4.mcast || ip6.mcast", (od->mcast_info.rtr.relay ? "next;" : - debug_drop_action())); + debug_drop_action()), + lflow_ref); /* Drop Ethernet local broadcast. By definition this traffic should * not be forwarded.*/ ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_INPUT, 50, - "eth.bcast", debug_drop_action()); + "eth.bcast", debug_drop_action(), + lflow_ref); /* Avoid ICMP time exceeded for multicast, silent drop instead. * See RFC1812 section 5.3.1: @@ -13620,21 +13882,25 @@ build_misc_local_traffic_drop_flows_for_lrouter( * (priority-31 flows will send ICMP time exceeded) */ ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_INPUT, 32, "ip.ttl == {0, 1} && !ip.later_frag && " - "(ip4.mcast || ip6.mcast)", debug_drop_action()); + "(ip4.mcast || ip6.mcast)", debug_drop_action(), + lflow_ref); /* TTL discard */ ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_INPUT, 30, - "ip.ttl == {0, 1}", debug_drop_action()); + "ip.ttl == {0, 1}", debug_drop_action(), + lflow_ref); /* Pass other traffic not already handled to the next table for * routing. */ - ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_INPUT, 0, "1", "next;"); + ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_INPUT, 0, "1", "next;", + lflow_ref); } static void build_dhcpv6_reply_flows_for_lrouter_port( struct ovn_port *op, struct lflow_table *lflows, - struct ds *match) + struct ds *match, + struct lflow_ref *lflow_ref) { ovs_assert(op->nbrp); if (op->l3dgw_port) { @@ -13647,7 +13913,8 @@ build_dhcpv6_reply_flows_for_lrouter_port( op->lrp_networks.ipv6_addrs[i].addr_s); ovn_lflow_add(lflows, op->od, S_ROUTER_IN_IP_INPUT, 100, ds_cstr(match), - "reg0 = 0; handle_dhcpv6_reply;"); + "reg0 = 0; handle_dhcpv6_reply;", + lflow_ref); } } @@ -13655,7 +13922,8 @@ static void build_ipv6_input_flows_for_lrouter_port( struct ovn_port *op, struct lflow_table *lflows, struct ds *match, struct ds *actions, - const struct shash *meter_groups) + const struct shash *meter_groups, + struct lflow_ref *lflow_ref) { ovs_assert(op->nbrp); if (is_cr_port(op)) { @@ -13679,7 +13947,7 @@ build_ipv6_input_flows_for_lrouter_port( "next; "; ovn_lflow_add_with_hint(lflows, op->od, S_ROUTER_IN_IP_INPUT, 90, ds_cstr(match), lrp_actions, - &op->nbrp->header_); + &op->nbrp->header_, lflow_ref); } /* ND reply. These flows reply to ND solicitations for the @@ -13700,7 +13968,8 @@ build_ipv6_input_flows_for_lrouter_port( op->lrp_networks.ipv6_addrs[i].addr_s, op->lrp_networks.ipv6_addrs[i].sn_addr_s, REG_INPORT_ETH_ADDR, match, false, 90, - &op->nbrp->header_, lflows, meter_groups); + &op->nbrp->header_, lflows, meter_groups, + lflow_ref); } /* UDP/TCP/SCTP port unreachable */ @@ -13720,7 +13989,8 @@ build_ipv6_input_flows_for_lrouter_port( COPP_TCP_RESET, op->od->nbr->copp, meter_groups), - &op->nbrp->header_); + &op->nbrp->header_, + lflow_ref); ds_clear(match); ds_put_format(match, @@ -13736,7 +14006,8 @@ build_ipv6_input_flows_for_lrouter_port( COPP_TCP_RESET, op->od->nbr->copp, meter_groups), - &op->nbrp->header_); + &op->nbrp->header_, + lflow_ref); ds_clear(match); ds_put_format(match, @@ -13755,7 +14026,8 @@ build_ipv6_input_flows_for_lrouter_port( COPP_ICMP6_ERR, op->od->nbr->copp, meter_groups), - &op->nbrp->header_); + &op->nbrp->header_, + lflow_ref); ds_clear(match); ds_put_format(match, @@ -13774,7 +14046,8 @@ build_ipv6_input_flows_for_lrouter_port( COPP_ICMP6_ERR, op->od->nbr->copp, meter_groups), - &op->nbrp->header_); + &op->nbrp->header_, + lflow_ref); } } @@ -13814,7 +14087,7 @@ build_ipv6_input_flows_for_lrouter_port( 31, ds_cstr(match), ds_cstr(actions), NULL, copp_meter_get(COPP_ICMP6_ERR, op->od->nbr->copp, meter_groups), - &op->nbrp->header_); + &op->nbrp->header_, lflow_ref); } ds_destroy(&ip_ds); } @@ -13823,7 +14096,8 @@ static void build_lrouter_arp_nd_for_datapath(const struct ovn_datapath *od, const struct lr_nat_record *lrnat_rec, struct lflow_table *lflows, - const struct shash *meter_groups) + const struct shash *meter_groups, + struct lflow_ref *lflow_ref) { ovs_assert(od->nbr); if (!od->nbr->n_nat) { @@ -13852,7 +14126,8 @@ build_lrouter_arp_nd_for_datapath(const struct ovn_datapath *od, if (!strcmp(nat_entry->nb->type, "snat")) { continue; } - build_lrouter_nat_arp_nd_flow(od, nat_entry, lflows, meter_groups); + build_lrouter_nat_arp_nd_flow(od, nat_entry, lflows, meter_groups, + lflow_ref); } /* Now handle SNAT entries too, one per unique SNAT IP. */ @@ -13867,7 +14142,8 @@ build_lrouter_arp_nd_for_datapath(const struct ovn_datapath *od, struct ovn_nat *nat_entry = CONTAINER_OF(ovs_list_front(&snat_ip->snat_entries), struct ovn_nat, ext_addr_list_node); - build_lrouter_nat_arp_nd_flow(od, nat_entry, lflows, meter_groups); + build_lrouter_nat_arp_nd_flow(od, nat_entry, lflows, meter_groups, + lflow_ref); } } @@ -13876,7 +14152,8 @@ static void build_lrouter_ipv4_ip_input(struct ovn_port *op, struct lflow_table *lflows, struct ds *match, struct ds *actions, - const struct shash *meter_groups) + const struct shash *meter_groups, + struct lflow_ref *lflow_ref) { ovs_assert(op->nbrp); /* No ingress packets are accepted on a chassisredirect @@ -13894,7 +14171,7 @@ build_lrouter_ipv4_ip_input(struct ovn_port *op, ds_put_cstr(match, " && "REGBIT_EGRESS_LOOPBACK" == 0"); ovn_lflow_add_with_hint(lflows, op->od, S_ROUTER_IN_IP_INPUT, 100, ds_cstr(match), debug_drop_action(), - &op->nbrp->header_); + &op->nbrp->header_, lflow_ref); /* ICMP echo reply. These flows reply to ICMP echo requests * received for the router's IP address. Since packets only @@ -13913,11 +14190,11 @@ build_lrouter_ipv4_ip_input(struct ovn_port *op, "next; "; ovn_lflow_add_with_hint(lflows, op->od, S_ROUTER_IN_IP_INPUT, 90, ds_cstr(match), icmp_actions, - &op->nbrp->header_); + &op->nbrp->header_, lflow_ref); } /* BFD msg handling */ - build_lrouter_bfd_flows(lflows, op, meter_groups); + build_lrouter_bfd_flows(lflows, op, meter_groups, lflow_ref); /* ICMP time exceeded */ struct ds ip_ds = DS_EMPTY_INITIALIZER; @@ -13946,7 +14223,7 @@ build_lrouter_ipv4_ip_input(struct ovn_port *op, 31, ds_cstr(match), ds_cstr(actions), NULL, copp_meter_get(COPP_ICMP4_ERR, op->od->nbr->copp, meter_groups), - &op->nbrp->header_); + &op->nbrp->header_, lflow_ref); } ds_destroy(&ip_ds); @@ -13996,7 +14273,7 @@ build_lrouter_ipv4_ip_input(struct ovn_port *op, build_lrouter_arp_flow(op->od, op, op->lrp_networks.ipv4_addrs[i].addr_s, REG_INPORT_ETH_ADDR, match, false, 90, - &op->nbrp->header_, lflows); + &op->nbrp->header_, lflows, lflow_ref); } if (!op->od->is_gw_router && !op->od->n_l3dgw_ports) { @@ -14019,7 +14296,8 @@ build_lrouter_ipv4_ip_input(struct ovn_port *op, COPP_ICMP4_ERR, op->od->nbr->copp, meter_groups), - &op->nbrp->header_); + &op->nbrp->header_, + lflow_ref); ds_clear(match); ds_put_format(match, @@ -14035,7 +14313,8 @@ build_lrouter_ipv4_ip_input(struct ovn_port *op, COPP_TCP_RESET, op->od->nbr->copp, meter_groups), - &op->nbrp->header_); + &op->nbrp->header_, + lflow_ref); ds_clear(match); ds_put_format(match, @@ -14051,7 +14330,8 @@ build_lrouter_ipv4_ip_input(struct ovn_port *op, COPP_TCP_RESET, op->od->nbr->copp, meter_groups), - &op->nbrp->header_); + &op->nbrp->header_, + lflow_ref); ds_clear(match); ds_put_format(match, @@ -14070,7 +14350,8 @@ build_lrouter_ipv4_ip_input(struct ovn_port *op, COPP_ICMP4_ERR, op->od->nbr->copp, meter_groups), - &op->nbrp->header_); + &op->nbrp->header_, + lflow_ref); } } } @@ -14080,7 +14361,8 @@ static void build_lrouter_ipv4_ip_input_for_lbnats( struct ovn_port *op, struct lflow_table *lflows, const struct lr_stateful_record *lr_stateful_rec, - struct ds *match, const struct shash *meter_groups) + struct ds *match, const struct shash *meter_groups, + struct lflow_ref *lflow_ref) { ovs_assert(op->nbrp); /* No ingress packets are accepted on a chassisredirect @@ -14101,7 +14383,7 @@ build_lrouter_ipv4_ip_input_for_lbnats( AF_INET); build_lrouter_arp_flow(op->od, op, lb_ips_v4_as, REG_INPORT_ETH_ADDR, - match, false, 90, NULL, lflows); + match, false, 90, NULL, lflows, lflow_ref); free(lb_ips_v4_as); } @@ -14118,7 +14400,7 @@ build_lrouter_ipv4_ip_input_for_lbnats( AF_INET6); build_lrouter_nd_flow(op->od, op, "nd_na", lb_ips_v6_as, NULL, REG_INPORT_ETH_ADDR, match, false, 90, - NULL, lflows, meter_groups); + NULL, lflows, meter_groups, lflow_ref); free(lb_ips_v6_as); } @@ -14151,7 +14433,7 @@ build_lrouter_ipv4_ip_input_for_lbnats( continue; } build_lrouter_port_nat_arp_nd_flow(op, nat_entry, lflows, - meter_groups); + meter_groups, lflow_ref); } /* Now handle SNAT entries too, one per unique SNAT IP. */ @@ -14167,7 +14449,7 @@ build_lrouter_ipv4_ip_input_for_lbnats( CONTAINER_OF(ovs_list_front(&snat_ip->snat_entries), struct ovn_nat, ext_addr_list_node); build_lrouter_port_nat_arp_nd_flow(op, nat_entry, lflows, - meter_groups); + meter_groups, lflow_ref); } } @@ -14203,7 +14485,8 @@ build_lrouter_in_unsnat_stateless_flow(struct lflow_table *lflows, const struct nbrec_nat *nat, struct ds *match, bool distributed_nat, bool is_v6, - struct ovn_port *l3dgw_port) + struct ovn_port *l3dgw_port, + struct lflow_ref *lflow_ref) { if (strcmp(nat->type, "snat") && strcmp(nat->type, "dnat_and_snat")) { return; @@ -14216,7 +14499,7 @@ build_lrouter_in_unsnat_stateless_flow(struct lflow_table *lflows, ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_UNSNAT, priority, ds_cstr(match), "next;", - &nat->header_); + &nat->header_, lflow_ref); } static void @@ -14224,7 +14507,8 @@ build_lrouter_in_unsnat_in_czone_flow(struct lflow_table *lflows, const struct ovn_datapath *od, const struct nbrec_nat *nat, struct ds *match, bool distributed_nat, - bool is_v6, struct ovn_port *l3dgw_port) + bool is_v6, struct ovn_port *l3dgw_port, + struct lflow_ref *lflow_ref) { if (strcmp(nat->type, "snat") && strcmp(nat->type, "dnat_and_snat")) { return; @@ -14242,7 +14526,7 @@ build_lrouter_in_unsnat_in_czone_flow(struct lflow_table *lflows, ds_put_cstr(match, " && flags.loopback == 0"); ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_UNSNAT, 100, ds_cstr(match), "ct_snat_in_czone;", - &nat->header_); + &nat->header_, lflow_ref); ds_truncate(match, common_match_len); /* Update common zone match for the hairpin traffic. */ @@ -14250,7 +14534,7 @@ build_lrouter_in_unsnat_in_czone_flow(struct lflow_table *lflows, ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_UNSNAT, 100, ds_cstr(match), "ct_snat;", - &nat->header_); + &nat->header_, lflow_ref); } static void @@ -14258,7 +14542,8 @@ build_lrouter_in_unsnat_flow(struct lflow_table *lflows, const struct ovn_datapath *od, const struct nbrec_nat *nat, struct ds *match, bool distributed_nat, bool is_v6, - struct ovn_port *l3dgw_port) + struct ovn_port *l3dgw_port, + struct lflow_ref *lflow_ref) { if (strcmp(nat->type, "snat") && strcmp(nat->type, "dnat_and_snat")) { @@ -14272,7 +14557,7 @@ build_lrouter_in_unsnat_flow(struct lflow_table *lflows, ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_UNSNAT, priority, ds_cstr(match), "ct_snat;", - &nat->header_); + &nat->header_, lflow_ref); } static void @@ -14282,7 +14567,8 @@ build_lrouter_in_dnat_flow(struct lflow_table *lflows, const struct nbrec_nat *nat, struct ds *match, struct ds *actions, bool distributed_nat, int cidr_bits, bool is_v6, - struct ovn_port *l3dgw_port, bool stateless) + struct ovn_port *l3dgw_port, bool stateless, + struct lflow_ref *lflow_ref) { /* Ingress DNAT table: Packets enter the pipeline with destination * IP address that needs to be DNATted from a external IP address @@ -14328,7 +14614,8 @@ build_lrouter_in_dnat_flow(struct lflow_table *lflows, if (nat->allowed_ext_ips || nat->exempted_ext_ips) { lrouter_nat_add_ext_ip_match(od, lflows, match, nat, - is_v6, true, cidr_bits); + is_v6, true, cidr_bits, + lflow_ref); } if (stateless) { @@ -14344,7 +14631,7 @@ build_lrouter_in_dnat_flow(struct lflow_table *lflows, ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_DNAT, 100, ds_cstr(match), ds_cstr(actions), - &nat->header_); + &nat->header_, lflow_ref); } static void @@ -14353,7 +14640,8 @@ build_lrouter_out_undnat_flow(struct lflow_table *lflows, const struct nbrec_nat *nat, struct ds *match, struct ds *actions, bool distributed_nat, struct eth_addr mac, bool is_v6, - struct ovn_port *l3dgw_port, bool stateless) + struct ovn_port *l3dgw_port, bool stateless, + struct lflow_ref *lflow_ref) { /* Egress UNDNAT table: It is for already established connections' * reverse traffic. i.e., DNAT has already been done in ingress @@ -14395,7 +14683,7 @@ build_lrouter_out_undnat_flow(struct lflow_table *lflows, ovn_lflow_add_with_hint(lflows, od, S_ROUTER_OUT_UNDNAT, 100, ds_cstr(match), ds_cstr(actions), - &nat->header_); + &nat->header_, lflow_ref); } static void @@ -14403,7 +14691,8 @@ build_lrouter_out_is_dnat_local(struct lflow_table *lflows, const struct ovn_datapath *od, const struct nbrec_nat *nat, struct ds *match, struct ds *actions, bool distributed_nat, - bool is_v6, struct ovn_port *l3dgw_port) + bool is_v6, struct ovn_port *l3dgw_port, + struct lflow_ref *lflow_ref) { /* Note that this only applies for NAT on a distributed router. */ @@ -14426,7 +14715,7 @@ build_lrouter_out_is_dnat_local(struct lflow_table *lflows, ovn_lflow_add_with_hint(lflows, od, S_ROUTER_OUT_CHECK_DNAT_LOCAL, 50, ds_cstr(match), ds_cstr(actions), - &nat->header_); + &nat->header_, lflow_ref); } static void @@ -14434,7 +14723,8 @@ build_lrouter_out_snat_match(struct lflow_table *lflows, const struct ovn_datapath *od, const struct nbrec_nat *nat, struct ds *match, bool distributed_nat, int cidr_bits, bool is_v6, - struct ovn_port *l3dgw_port) + struct ovn_port *l3dgw_port, + struct lflow_ref *lflow_ref) { ds_clear(match); @@ -14454,7 +14744,8 @@ build_lrouter_out_snat_match(struct lflow_table *lflows, if (nat->allowed_ext_ips || nat->exempted_ext_ips) { lrouter_nat_add_ext_ip_match(od, lflows, match, nat, - is_v6, false, cidr_bits); + is_v6, false, cidr_bits, + lflow_ref); } } @@ -14465,7 +14756,8 @@ build_lrouter_out_snat_stateless_flow(struct lflow_table *lflows, struct ds *match, struct ds *actions, bool distributed_nat, struct eth_addr mac, int cidr_bits, - bool is_v6, struct ovn_port *l3dgw_port) + bool is_v6, struct ovn_port *l3dgw_port, + struct lflow_ref *lflow_ref) { if (strcmp(nat->type, "snat") && strcmp(nat->type, "dnat_and_snat")) { return; @@ -14479,7 +14771,7 @@ build_lrouter_out_snat_stateless_flow(struct lflow_table *lflows, uint16_t priority = cidr_bits + 1; build_lrouter_out_snat_match(lflows, od, nat, match, distributed_nat, - cidr_bits, is_v6, l3dgw_port); + cidr_bits, is_v6, l3dgw_port, lflow_ref); if (!od->is_gw_router) { /* Distributed router. */ @@ -14498,7 +14790,8 @@ build_lrouter_out_snat_stateless_flow(struct lflow_table *lflows, ovn_lflow_add_with_hint(lflows, od, S_ROUTER_OUT_SNAT, priority, ds_cstr(match), - ds_cstr(actions), &nat->header_); + ds_cstr(actions), &nat->header_, + lflow_ref); } static void @@ -14508,7 +14801,8 @@ build_lrouter_out_snat_in_czone_flow(struct lflow_table *lflows, struct ds *match, struct ds *actions, bool distributed_nat, struct eth_addr mac, int cidr_bits, - bool is_v6, struct ovn_port *l3dgw_port) + bool is_v6, struct ovn_port *l3dgw_port, + struct lflow_ref *lflow_ref) { if (strcmp(nat->type, "snat") && strcmp(nat->type, "dnat_and_snat")) { return; @@ -14523,7 +14817,8 @@ build_lrouter_out_snat_in_czone_flow(struct lflow_table *lflows, struct ds zone_actions = DS_EMPTY_INITIALIZER; build_lrouter_out_snat_match(lflows, od, nat, match, distributed_nat, - cidr_bits, is_v6, l3dgw_port); + cidr_bits, is_v6, l3dgw_port, + lflow_ref); if (od->n_l3dgw_ports) { priority += 128; @@ -14552,13 +14847,15 @@ build_lrouter_out_snat_in_czone_flow(struct lflow_table *lflows, ovn_lflow_add_with_hint(lflows, od, S_ROUTER_OUT_SNAT, priority, ds_cstr(match), - ds_cstr(actions), &nat->header_); + ds_cstr(actions), &nat->header_, + lflow_ref); ds_put_cstr(match, " && "REGBIT_DST_NAT_IP_LOCAL" == 1"); ovn_lflow_add_with_hint(lflows, od, S_ROUTER_OUT_SNAT, priority + 1, ds_cstr(match), - ds_cstr(&zone_actions), &nat->header_); + ds_cstr(&zone_actions), &nat->header_, + lflow_ref); ds_destroy(&zone_actions); } @@ -14569,7 +14866,8 @@ build_lrouter_out_snat_flow(struct lflow_table *lflows, const struct nbrec_nat *nat, struct ds *match, struct ds *actions, bool distributed_nat, struct eth_addr mac, int cidr_bits, bool is_v6, - struct ovn_port *l3dgw_port) + struct ovn_port *l3dgw_port, + struct lflow_ref *lflow_ref) { if (strcmp(nat->type, "snat") && strcmp(nat->type, "dnat_and_snat")) { return; @@ -14583,7 +14881,7 @@ build_lrouter_out_snat_flow(struct lflow_table *lflows, uint16_t priority = cidr_bits + 1; build_lrouter_out_snat_match(lflows, od, nat, match, distributed_nat, - cidr_bits, is_v6, l3dgw_port); + cidr_bits, is_v6, l3dgw_port, lflow_ref); if (!od->is_gw_router) { /* Distributed router. */ @@ -14606,7 +14904,8 @@ build_lrouter_out_snat_flow(struct lflow_table *lflows, ovn_lflow_add_with_hint(lflows, od, S_ROUTER_OUT_SNAT, priority, ds_cstr(match), - ds_cstr(actions), &nat->header_); + ds_cstr(actions), &nat->header_, + lflow_ref); } static void @@ -14616,7 +14915,8 @@ build_lrouter_ingress_nat_check_pkt_len(struct lflow_table *lflows, bool is_v6, struct ds *match, struct ds *actions, int mtu, struct ovn_port *l3dgw_port, - const struct shash *meter_groups) + const struct shash *meter_groups, + struct lflow_ref *lflow_ref) { ds_clear(match); ds_put_format(match, "inport == %s && "REGBIT_PKT_LARGER @@ -14650,7 +14950,8 @@ build_lrouter_ingress_nat_check_pkt_len(struct lflow_table *lflows, COPP_ICMP4_ERR, od->nbr->copp, meter_groups), - &nat->header_); + &nat->header_, + lflow_ref); } else { ds_put_format(match, " && ip6 && ip6.dst == %s", nat->external_ip); /* Set icmp6.frag_mtu to gw_mtu */ @@ -14677,7 +14978,8 @@ build_lrouter_ingress_nat_check_pkt_len(struct lflow_table *lflows, COPP_ICMP6_ERR, od->nbr->copp, meter_groups), - &nat->header_); + &nat->header_, + lflow_ref); } } @@ -14688,7 +14990,8 @@ build_lrouter_ingress_flow(struct lflow_table *lflows, struct ds *actions, struct eth_addr mac, bool distributed_nat, bool is_v6, struct ovn_port *l3dgw_port, - const struct shash *meter_groups) + const struct shash *meter_groups, + struct lflow_ref *lflow_ref) { if (od->n_l3dgw_ports && !strcmp(nat->type, "snat")) { ds_clear(match); @@ -14698,7 +15001,7 @@ build_lrouter_ingress_flow(struct lflow_table *lflows, is_v6 ? "ip6.src" : "ip4.src", nat->external_ip); ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_IP_INPUT, 120, ds_cstr(match), "next;", - &nat->header_); + &nat->header_, lflow_ref); } /* Logical router ingress table 0: * For NAT on a distributed router, add rules allowing @@ -14722,12 +15025,14 @@ build_lrouter_ingress_flow(struct lflow_table *lflows, build_gateway_mtu_flow(lflows, l3dgw_port, S_ROUTER_IN_ADMISSION, 50, 55, match, actions, &nat->header_, + lflow_ref, REG_INPORT_ETH_ADDR " = %s; next;", l3dgw_port->lrp_networks.ea_s); if (gw_mtu) { build_lrouter_ingress_nat_check_pkt_len(lflows, nat, od, is_v6, match, actions, gw_mtu, - l3dgw_port, meter_groups); + l3dgw_port, meter_groups, + lflow_ref); } } } @@ -14862,27 +15167,33 @@ lrouter_check_nat_entry(const struct ovn_datapath *od, /* NAT, Defrag and load balancing. */ static void build_lr_nat_defrag_and_lb_default_flows(struct ovn_datapath *od, - struct lflow_table *lflows) + struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { ovs_assert(od->nbr); /* Packets are allowed by default. */ - ovn_lflow_add(lflows, od, S_ROUTER_IN_DEFRAG, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_ROUTER_IN_UNSNAT, 0, "1", "next;"); + ovn_lflow_add(lflows, od, S_ROUTER_IN_DEFRAG, 0, "1", "next;", lflow_ref); + ovn_lflow_add(lflows, od, S_ROUTER_IN_UNSNAT, 0, "1", "next;", lflow_ref); ovn_lflow_add(lflows, od, S_ROUTER_OUT_CHECK_DNAT_LOCAL, 0, "1", - REGBIT_DST_NAT_IP_LOCAL" = 0; next;"); - ovn_lflow_add(lflows, od, S_ROUTER_OUT_SNAT, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_ROUTER_IN_DNAT, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_ROUTER_OUT_UNDNAT, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_ROUTER_OUT_POST_UNDNAT, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_ROUTER_OUT_POST_SNAT, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_ROUTER_OUT_EGR_LOOP, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_ROUTER_IN_ECMP_STATEFUL, 0, "1", "next;"); + REGBIT_DST_NAT_IP_LOCAL" = 0; next;", lflow_ref); + ovn_lflow_add(lflows, od, S_ROUTER_OUT_SNAT, 0, "1", "next;", lflow_ref); + ovn_lflow_add(lflows, od, S_ROUTER_IN_DNAT, 0, "1", "next;", lflow_ref); + ovn_lflow_add(lflows, od, S_ROUTER_OUT_UNDNAT, 0, "1", "next;", lflow_ref); + ovn_lflow_add(lflows, od, S_ROUTER_OUT_POST_UNDNAT, 0, "1", "next;", + lflow_ref); + ovn_lflow_add(lflows, od, S_ROUTER_OUT_POST_SNAT, 0, "1", "next;", + lflow_ref); + ovn_lflow_add(lflows, od, S_ROUTER_OUT_EGR_LOOP, 0, "1", "next;", + lflow_ref); + ovn_lflow_add(lflows, od, S_ROUTER_IN_ECMP_STATEFUL, 0, "1", "next;", + lflow_ref); /* Send the IPv6 NS packets to next table. When ovn-controller * generates IPv6 NS (for the action - nd_ns{}), the injected * packet would go through conntrack - which is not required. */ - ovn_lflow_add(lflows, od, S_ROUTER_OUT_SNAT, 120, "nd_ns", "next;"); + ovn_lflow_add(lflows, od, S_ROUTER_OUT_SNAT, 120, "nd_ns", "next;", + lflow_ref); } static void @@ -14892,7 +15203,8 @@ build_lrouter_nat_defrag_and_lb( const struct hmap *ls_ports, const struct hmap *lr_ports, struct ds *match, struct ds *actions, const struct shash *meter_groups, - const struct chassis_features *features) + const struct chassis_features *features, + struct lflow_ref *lflow_ref) { const struct ovn_datapath *od = lr_stateful_rec->od; ovs_assert(od->nbr); @@ -14917,16 +15229,18 @@ build_lrouter_nat_defrag_and_lb( ds_put_format(match, " && %s.skip_snat == 1", ct_flag_reg); ovn_lflow_add(lflows, od, S_ROUTER_IN_DNAT, 70, ds_cstr(match), - "flags.skip_snat_for_lb = 1; ct_commit_nat;"); + "flags.skip_snat_for_lb = 1; ct_commit_nat;", + lflow_ref); ds_truncate(match, match_len); ds_put_format(match, " && %s.force_snat == 1", ct_flag_reg); ovn_lflow_add(lflows, od, S_ROUTER_IN_DNAT, 70, ds_cstr(match), - "flags.force_snat_for_lb = 1; ct_commit_nat;"); + "flags.force_snat_for_lb = 1; ct_commit_nat;", + lflow_ref); ds_truncate(match, match_len); ovn_lflow_add(lflows, od, S_ROUTER_IN_DNAT, 50, ds_cstr(match), - "ct_commit_nat;"); + "ct_commit_nat;", lflow_ref); } /* Ingress DNAT (Priority 50/70). @@ -14943,16 +15257,18 @@ build_lrouter_nat_defrag_and_lb( ds_put_format(match, " && %s.skip_snat == 1", ct_flag_reg); ovn_lflow_add(lflows, od, S_ROUTER_IN_DNAT, 70, ds_cstr(match), - "flags.skip_snat_for_lb = 1; next;"); + "flags.skip_snat_for_lb = 1; next;", + lflow_ref); ds_truncate(match, match_len); ds_put_format(match, " && %s.force_snat == 1", ct_flag_reg); ovn_lflow_add(lflows, od, S_ROUTER_IN_DNAT, 70, ds_cstr(match), - "flags.force_snat_for_lb = 1; next;"); + "flags.force_snat_for_lb = 1; next;", + lflow_ref); ds_truncate(match, match_len); ovn_lflow_add(lflows, od, S_ROUTER_IN_DNAT, 50, ds_cstr(match), - "next;"); + "next;", lflow_ref); } /* If the router has load balancer or DNAT rules, re-circulate every packet @@ -14967,11 +15283,14 @@ build_lrouter_nat_defrag_and_lb( if (od->is_gw_router && (od->nbr->n_nat || lr_stateful_rec->has_lb_vip)) { /* Do not send ND or ICMP packets to connection tracking. */ ovn_lflow_add(lflows, od, S_ROUTER_OUT_UNDNAT, 100, - "nd || nd_rs || nd_ra", "next;"); + "nd || nd_rs || nd_ra", "next;", + lflow_ref); ovn_lflow_add(lflows, od, S_ROUTER_OUT_UNDNAT, 50, - "ip", "flags.loopback = 1; ct_dnat;"); + "ip", "flags.loopback = 1; ct_dnat;", + lflow_ref); ovn_lflow_add(lflows, od, S_ROUTER_OUT_POST_UNDNAT, 50, - "ip && ct.new", "ct_commit { } ; next; "); + "ip && ct.new", "ct_commit { } ; next; ", + lflow_ref); } /* NAT rules are only valid on Gateway routers and routers with @@ -15020,19 +15339,20 @@ build_lrouter_nat_defrag_and_lb( if (stateless) { build_lrouter_in_unsnat_stateless_flow(lflows, od, nat, match, distributed_nat, is_v6, - l3dgw_port); + l3dgw_port, lflow_ref); } else if (lrouter_use_common_zone(od)) { build_lrouter_in_unsnat_in_czone_flow(lflows, od, nat, match, distributed_nat, is_v6, - l3dgw_port); + l3dgw_port, lflow_ref); } else { build_lrouter_in_unsnat_flow(lflows, od, nat, match, - distributed_nat, is_v6, l3dgw_port); + distributed_nat, is_v6, l3dgw_port, + lflow_ref); } /* S_ROUTER_IN_DNAT */ build_lrouter_in_dnat_flow(lflows, od, lrnat_rec, nat, match, actions, distributed_nat, cidr_bits, is_v6, - l3dgw_port, stateless); + l3dgw_port, stateless, lflow_ref); /* ARP resolve for NAT IPs. */ if (od->is_gw_router) { @@ -15055,7 +15375,8 @@ build_lrouter_nat_defrag_and_lb( S_ROUTER_IN_ARP_RESOLVE, 150, ds_cstr(match), debug_drop_action(), - &nat->header_); + &nat->header_, + lflow_ref); /* Now for packets coming from other (downlink) LRPs, allow ARP * resolve for the NAT IP, so that such packets can be * forwarded for E/W NAT. */ @@ -15074,7 +15395,8 @@ build_lrouter_nat_defrag_and_lb( S_ROUTER_IN_ARP_RESOLVE, 100, ds_cstr(match), ds_cstr(actions), - &nat->header_); + &nat->header_, + lflow_ref); if (od->redirect_bridged && distributed_nat) { ds_clear(match); ds_put_format( @@ -15095,7 +15417,8 @@ build_lrouter_nat_defrag_and_lb( ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_ARP_RESOLVE, 90, ds_cstr(match), ds_cstr(actions), - &nat->header_); + &nat->header_, + lflow_ref); } sset_add(&nat_entries, nat->external_ip); } @@ -15105,13 +15428,13 @@ build_lrouter_nat_defrag_and_lb( /* S_ROUTER_OUT_DNAT_LOCAL */ build_lrouter_out_is_dnat_local(lflows, od, nat, match, actions, distributed_nat, is_v6, - l3dgw_port); + l3dgw_port, lflow_ref); } /* S_ROUTER_OUT_UNDNAT */ build_lrouter_out_undnat_flow(lflows, od, nat, match, actions, distributed_nat, mac, is_v6, l3dgw_port, - stateless); + stateless, lflow_ref); /* S_ROUTER_OUT_SNAT * Egress SNAT table: Packets enter the egress pipeline with * source ip address that needs to be SNATted to a external ip @@ -15120,21 +15443,22 @@ build_lrouter_nat_defrag_and_lb( build_lrouter_out_snat_stateless_flow(lflows, od, nat, match, actions, distributed_nat, mac, cidr_bits, is_v6, - l3dgw_port); + l3dgw_port, lflow_ref); } else if (lrouter_use_common_zone(od)) { build_lrouter_out_snat_in_czone_flow(lflows, od, nat, match, actions, distributed_nat, mac, - cidr_bits, is_v6, l3dgw_port); + cidr_bits, is_v6, l3dgw_port, + lflow_ref); } else { build_lrouter_out_snat_flow(lflows, od, nat, match, actions, distributed_nat, mac, cidr_bits, is_v6, - l3dgw_port); + l3dgw_port, lflow_ref); } /* S_ROUTER_IN_ADMISSION - S_ROUTER_IN_IP_INPUT */ build_lrouter_ingress_flow(lflows, od, nat, match, actions, mac, distributed_nat, is_v6, l3dgw_port, - meter_groups); + meter_groups, lflow_ref); /* Ingress Gateway Redirect Table: For NAT on a distributed * router, add flows that are specific to a NAT rule. These @@ -15161,7 +15485,8 @@ build_lrouter_nat_defrag_and_lb( if (op && op->nbsp && !strcmp(op->nbsp->type, "virtual")) { ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_GW_REDIRECT, 80, ds_cstr(match), - debug_drop_action(), &nat->header_); + debug_drop_action(), &nat->header_, + lflow_ref); } ds_put_format(match, " && is_chassis_resident(\"%s\")", nat->logical_port); @@ -15171,7 +15496,8 @@ build_lrouter_nat_defrag_and_lb( nat->external_ip); ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_GW_REDIRECT, 100, ds_cstr(match), - ds_cstr(actions), &nat->header_); + ds_cstr(actions), &nat->header_, + lflow_ref); } /* Egress Loopback table: For NAT on a distributed router. @@ -15212,7 +15538,7 @@ build_lrouter_nat_defrag_and_lb( ovn_stage_get_table(S_ROUTER_IN_ADMISSION)); ovn_lflow_add_with_hint(lflows, od, S_ROUTER_OUT_EGR_LOOP, 100, ds_cstr(match), ds_cstr(actions), - &nat->header_); + &nat->header_, lflow_ref); } } @@ -15232,7 +15558,7 @@ build_lrouter_nat_defrag_and_lb( ds_put_cstr(actions, REGBIT_DST_NAT_IP_LOCAL" = 1; next;"); ovn_lflow_add_with_hint(lflows, od, S_ROUTER_OUT_CHECK_DNAT_LOCAL, 50, ds_cstr(match), ds_cstr(actions), - &od->nbr->header_); + &od->nbr->header_, lflow_ref); } @@ -15242,22 +15568,24 @@ build_lrouter_nat_defrag_and_lb( if (lrnat_rec->dnat_force_snat_addrs.n_ipv4_addrs) { build_lrouter_force_snat_flows(lflows, od, "4", lrnat_rec->dnat_force_snat_addrs.ipv4_addrs[0].addr_s, - "dnat"); + "dnat", lflow_ref); } if (lrnat_rec->dnat_force_snat_addrs.n_ipv6_addrs) { build_lrouter_force_snat_flows(lflows, od, "6", lrnat_rec->dnat_force_snat_addrs.ipv6_addrs[0].addr_s, - "dnat"); + "dnat", lflow_ref); } } if (lb_force_snat_ip) { if (lrnat_rec->lb_force_snat_addrs.n_ipv4_addrs) { build_lrouter_force_snat_flows(lflows, od, "4", - lrnat_rec->lb_force_snat_addrs.ipv4_addrs[0].addr_s, "lb"); + lrnat_rec->lb_force_snat_addrs.ipv4_addrs[0].addr_s, "lb", + lflow_ref); } if (lrnat_rec->lb_force_snat_addrs.n_ipv6_addrs) { build_lrouter_force_snat_flows(lflows, od, "6", - lrnat_rec->lb_force_snat_addrs.ipv6_addrs[0].addr_s, "lb"); + lrnat_rec->lb_force_snat_addrs.ipv6_addrs[0].addr_s, "lb", + lflow_ref); } } } @@ -15329,7 +15657,8 @@ build_lrp_lflows_for_lbnats(struct ovn_port *op, */ if (!lr_stateful_rec->lrnat_rec->lb_force_snat_router_ip) { build_lrouter_drop_own_dest(op, lr_stateful_rec, - S_ROUTER_IN_IP_INPUT, 60, false, lflows); + S_ROUTER_IN_IP_INPUT, 60, false, lflows, + op->stateful_lflow_ref); } /* Drop IP traffic destined to router owned IPs. Part of it is dropped @@ -15339,12 +15668,14 @@ build_lrp_lflows_for_lbnats(struct ovn_port *op, * Priority 2. */ build_lrouter_drop_own_dest(op, lr_stateful_rec, - S_ROUTER_IN_ARP_RESOLVE, 2, true, lflows); + S_ROUTER_IN_ARP_RESOLVE, 2, true, + lflows, op->stateful_lflow_ref); build_lrouter_ipv4_ip_input_for_lbnats(op, lflows, lr_stateful_rec, - match, meter_groups); + match, meter_groups, + op->stateful_lflow_ref); build_lrouter_force_snat_flows_op(op, lr_stateful_rec->lrnat_rec, lflows, - match, actions); + match, actions, op->stateful_lflow_ref); } static void @@ -15376,13 +15707,13 @@ build_lr_stateful_flows(const struct lr_stateful_record *lr_stateful_rec, { build_lrouter_nat_defrag_and_lb(lr_stateful_rec, lflows, ls_ports, lr_ports, match, actions, - meter_groups, features); + meter_groups, features, NULL); build_lr_gateway_redirect_flows_for_nats(lr_stateful_rec->od, lr_stateful_rec->lrnat_rec, - lflows, match, actions); + lflows, match, actions, NULL); build_lrouter_arp_nd_for_datapath(lr_stateful_rec->od, lr_stateful_rec->lrnat_rec, lflows, - meter_groups); + meter_groups, NULL); } static void @@ -15394,11 +15725,13 @@ build_ls_stateful_flows(const struct ls_stateful_record *ls_stateful_rec, { ovs_assert(ls_stateful_rec->od); - build_ls_stateful_rec_pre_acls(ls_stateful_rec, ls_pgs, lflows); - build_ls_stateful_rec_pre_lb(ls_stateful_rec, lflows); - build_acl_hints(ls_stateful_rec, features, lflows); - build_acls(ls_stateful_rec, features, lflows, ls_pgs, meter_groups); - build_lb_hairpin(ls_stateful_rec, lflows); + build_ls_stateful_rec_pre_acls(ls_stateful_rec, ls_pgs, lflows, + NULL); + build_ls_stateful_rec_pre_lb(ls_stateful_rec, lflows, NULL); + build_acl_hints(ls_stateful_rec, features, lflows, NULL); + build_acls(ls_stateful_rec, features, lflows, ls_pgs, meter_groups, + NULL); + build_lb_hairpin(ls_stateful_rec, lflows, NULL); } struct lswitch_flow_build_info { @@ -15434,19 +15767,20 @@ build_lswitch_and_lrouter_iterate_by_ls(struct ovn_datapath *od, { ovs_assert(od->nbs); build_lswitch_lflows_pre_acl_and_acl(od, lsi->features, lsi->lflows, - lsi->meter_groups); - - build_fwd_group_lflows(od, lsi->lflows); - build_lswitch_lflows_admission_control(od, lsi->lflows); - build_lswitch_learn_fdb_od(od, lsi->lflows); - build_lswitch_arp_nd_responder_default(od, lsi->lflows); - build_lswitch_dns_lookup_and_response(od, lsi->lflows, lsi->meter_groups); - build_lswitch_dhcp_and_dns_defaults(od, lsi->lflows); + lsi->meter_groups, NULL); + + build_fwd_group_lflows(od, lsi->lflows, NULL); + build_lswitch_lflows_admission_control(od, lsi->lflows, NULL); + build_lswitch_learn_fdb_od(od, lsi->lflows, NULL); + build_lswitch_arp_nd_responder_default(od, lsi->lflows, NULL); + build_lswitch_dns_lookup_and_response(od, lsi->lflows, lsi->meter_groups, + NULL); + build_lswitch_dhcp_and_dns_defaults(od, lsi->lflows, NULL); build_lswitch_destination_lookup_bmcast(od, lsi->lflows, &lsi->actions, - lsi->meter_groups); - build_lswitch_output_port_sec_od(od, lsi->lflows); - build_lswitch_lb_affinity_default_flows(od, lsi->lflows); - build_lswitch_lflows_l2_unknown(od, lsi->lflows); + lsi->meter_groups, NULL); + build_lswitch_output_port_sec_od(od, lsi->lflows, NULL); + build_lswitch_lb_affinity_default_flows(od, lsi->lflows, NULL); + build_lswitch_lflows_l2_unknown(od, lsi->lflows, NULL); } /* Helper function to combine all lflow generation which is iterated by @@ -15457,29 +15791,34 @@ build_lswitch_and_lrouter_iterate_by_lr(struct ovn_datapath *od, struct lswitch_flow_build_info *lsi) { ovs_assert(od->nbr); - build_adm_ctrl_flows_for_lrouter(od, lsi->lflows); + build_adm_ctrl_flows_for_lrouter(od, lsi->lflows, NULL); build_neigh_learning_flows_for_lrouter(od, lsi->lflows, &lsi->match, - &lsi->actions, lsi->meter_groups); - build_ND_RA_flows_for_lrouter(od, lsi->lflows); - build_ip_routing_pre_flows_for_lrouter(od, lsi->lflows); + &lsi->actions, + lsi->meter_groups, NULL); + build_ND_RA_flows_for_lrouter(od, lsi->lflows, NULL); + build_ip_routing_pre_flows_for_lrouter(od, lsi->lflows, NULL); build_static_route_flows_for_lrouter(od, lsi->features, lsi->lflows, lsi->lr_ports, - lsi->bfd_connections); + lsi->bfd_connections, + NULL); build_mcast_lookup_flows_for_lrouter(od, lsi->lflows, &lsi->match, - &lsi->actions); - build_ingress_policy_flows_for_lrouter(od, lsi->lflows, lsi->lr_ports); - build_arp_resolve_flows_for_lrouter(od, lsi->lflows); + &lsi->actions, NULL); + build_ingress_policy_flows_for_lrouter(od, lsi->lflows, lsi->lr_ports, + NULL); + build_arp_resolve_flows_for_lrouter(od, lsi->lflows, NULL); build_check_pkt_len_flows_for_lrouter(od, lsi->lflows, lsi->lr_ports, &lsi->match, &lsi->actions, - lsi->meter_groups); + lsi->meter_groups, NULL); build_gateway_redirect_flows_for_lrouter(od, lsi->lflows, &lsi->match, - &lsi->actions); + &lsi->actions, NULL); build_arp_request_flows_for_lrouter(od, lsi->lflows, &lsi->match, - &lsi->actions, lsi->meter_groups); - build_misc_local_traffic_drop_flows_for_lrouter(od, lsi->lflows); + &lsi->actions, + lsi->meter_groups, + NULL); + build_misc_local_traffic_drop_flows_for_lrouter(od, lsi->lflows, NULL); - build_lr_nat_defrag_and_lb_default_flows(od, lsi->lflows); - build_lrouter_lb_affinity_default_flows(od, lsi->lflows); + build_lr_nat_defrag_and_lb_default_flows(od, lsi->lflows, NULL); + build_lrouter_lb_affinity_default_flows(od, lsi->lflows, NULL); } /* Helper function to combine all lflow generation which is iterated by logical @@ -15521,22 +15860,26 @@ build_lswitch_and_lrouter_iterate_by_lrp(struct ovn_port *op, ovs_assert(op->nbrp); build_adm_ctrl_flows_for_lrouter_port(op, lsi->lflows, &lsi->match, - &lsi->actions); + &lsi->actions, op->lflow_ref); build_neigh_learning_flows_for_lrouter_port(op, lsi->lflows, &lsi->match, - &lsi->actions); - build_ip_routing_flows_for_lrp(op, lsi->lflows); + &lsi->actions, op->lflow_ref); + build_ip_routing_flows_for_lrp(op, lsi->lflows, op->lflow_ref); build_ND_RA_flows_for_lrouter_port(op, lsi->lflows, &lsi->match, - &lsi->actions, lsi->meter_groups); + &lsi->actions, lsi->meter_groups, + op->lflow_ref); build_arp_resolve_flows_for_lrp(op, lsi->lflows, - &lsi->match, &lsi->actions); + &lsi->match, &lsi->actions, op->lflow_ref); build_egress_delivery_flows_for_lrouter_port(op, lsi->lflows, &lsi->match, - &lsi->actions); - build_dhcpv6_reply_flows_for_lrouter_port(op, lsi->lflows, &lsi->match); + &lsi->actions, + op->lflow_ref); + build_dhcpv6_reply_flows_for_lrouter_port(op, lsi->lflows, &lsi->match, + op->lflow_ref); build_ipv6_input_flows_for_lrouter_port(op, lsi->lflows, &lsi->match, &lsi->actions, - lsi->meter_groups); + lsi->meter_groups, + op->lflow_ref); build_lrouter_ipv4_ip_input(op, lsi->lflows, &lsi->match, &lsi->actions, - lsi->meter_groups); + lsi->meter_groups, op->lflow_ref); } static void * @@ -15629,7 +15972,7 @@ build_lflows_thread(void *arg) if (stop_parallel_processing()) { return NULL; } - build_lswitch_arp_nd_service_monitor(lb_dps->lb, + build_lswitch_arp_nd_service_monitor(lb_dps, lsi->ls_ports, lsi->lflows, &lsi->match, @@ -15861,7 +16204,7 @@ build_lswitch_and_lrouter_flows( stopwatch_stop(LFLOWS_PORTS_STOPWATCH_NAME, time_msec()); stopwatch_start(LFLOWS_LBS_STOPWATCH_NAME, time_msec()); HMAP_FOR_EACH (lb_dps, hmap_node, lb_dps_map) { - build_lswitch_arp_nd_service_monitor(lb_dps->lb, lsi.ls_ports, + build_lswitch_arp_nd_service_monitor(lb_dps, lsi.ls_ports, lsi.lflows, &lsi.actions, &lsi.match); build_lrouter_defrag_flows_for_lb(lb_dps, lsi.lflows, @@ -16050,6 +16393,11 @@ lflow_reset_northd_refs(struct lflow_input *lflow_input) lflow_ref_clear(op->lflow_ref); lflow_ref_clear(op->stateful_lflow_ref); } + + HMAP_FOR_EACH (op, key_node, lflow_input->lr_ports) { + lflow_ref_clear(op->lflow_ref); + lflow_ref_clear(op->stateful_lflow_ref); + } } bool From patchwork Thu Jan 11 15:32:22 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Numan Siddique X-Patchwork-Id: 1885679 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::138; helo=smtp1.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp1.osuosl.org (smtp1.osuosl.org [IPv6:2605:bc80:3010::138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4T9pZy2dN6z1yPp for ; Fri, 12 Jan 2024 02:32:58 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 8081584048; Thu, 11 Jan 2024 15:32:56 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 8081584048 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Om3XEKADHD9K; Thu, 11 Jan 2024 15:32:55 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp1.osuosl.org (Postfix) with ESMTPS id B27ED8404F; Thu, 11 Jan 2024 15:32:53 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org B27ED8404F Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 7B0B1C0077; Thu, 11 Jan 2024 15:32:53 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 10D9EC0037 for ; Thu, 11 Jan 2024 15:32:53 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 481A961B3D for ; Thu, 11 Jan 2024 15:32:51 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 481A961B3D X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id hmPh7vVfpjeC for ; Thu, 11 Jan 2024 15:32:50 +0000 (UTC) Received: from relay1-d.mail.gandi.net (relay1-d.mail.gandi.net [217.70.183.193]) by smtp3.osuosl.org (Postfix) with ESMTPS id B20886F535 for ; Thu, 11 Jan 2024 15:32:49 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org B20886F535 Received: by mail.gandi.net (Postfix) with ESMTPSA id 5DF9A240008; Thu, 11 Jan 2024 15:32:47 +0000 (UTC) From: numans@ovn.org To: dev@openvswitch.org Date: Thu, 11 Jan 2024 10:32:22 -0500 Message-ID: <20240111153222.2790366-1-numans@ovn.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240111152752.2789854-1-numans@ovn.org> References: <20240111152752.2789854-1-numans@ovn.org> MIME-Version: 1.0 X-GND-Sasl: numans@ovn.org Subject: [ovs-dev] [PATCH ovn v5 10/16] northd: Move ovn_lb_datapaths from lib to northd module. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Numan Siddique Signed-off-by: Numan Siddique --- lib/lb.c | 96 ------------------------------- lib/lb.h | 57 ------------------- northd/automake.mk | 4 +- northd/en-lr-stateful.c | 1 + northd/lb.c | 121 ++++++++++++++++++++++++++++++++++++++++ northd/lb.h | 82 +++++++++++++++++++++++++++ northd/northd.c | 1 + 7 files changed, 208 insertions(+), 154 deletions(-) create mode 100644 northd/lb.c create mode 100644 northd/lb.h diff --git a/lib/lb.c b/lib/lb.c index d0d562b6fb..991f20299d 100644 --- a/lib/lb.c +++ b/lib/lb.c @@ -1071,99 +1071,3 @@ remove_ips_from_lb_ip_set(struct ovn_lb_ip_set *lb_ips, } } } - -/* lb datapaths functions */ -struct ovn_lb_datapaths * -ovn_lb_datapaths_create(const struct ovn_northd_lb *lb, size_t n_ls_datapaths, - size_t n_lr_datapaths) -{ - struct ovn_lb_datapaths *lb_dps = xzalloc(sizeof *lb_dps); - lb_dps->lb = lb; - lb_dps->nb_ls_map = bitmap_allocate(n_ls_datapaths); - lb_dps->nb_lr_map = bitmap_allocate(n_lr_datapaths); - - return lb_dps; -} - -struct ovn_lb_datapaths * -ovn_lb_datapaths_find(const struct hmap *lb_dps_map, - const struct uuid *lb_uuid) -{ - struct ovn_lb_datapaths *lb_dps; - size_t hash = uuid_hash(lb_uuid); - HMAP_FOR_EACH_WITH_HASH (lb_dps, hmap_node, hash, lb_dps_map) { - if (uuid_equals(&lb_dps->lb->nlb->header_.uuid, lb_uuid)) { - return lb_dps; - } - } - return NULL; -} - -void -ovn_lb_datapaths_destroy(struct ovn_lb_datapaths *lb_dps) -{ - bitmap_free(lb_dps->nb_lr_map); - bitmap_free(lb_dps->nb_ls_map); - free(lb_dps); -} - -void -ovn_lb_datapaths_add_lr(struct ovn_lb_datapaths *lb_dps, size_t n, - struct ovn_datapath **ods) -{ - for (size_t i = 0; i < n; i++) { - if (!bitmap_is_set(lb_dps->nb_lr_map, ods[i]->index)) { - bitmap_set1(lb_dps->nb_lr_map, ods[i]->index); - lb_dps->n_nb_lr++; - } - } -} - -void -ovn_lb_datapaths_add_ls(struct ovn_lb_datapaths *lb_dps, size_t n, - struct ovn_datapath **ods) -{ - for (size_t i = 0; i < n; i++) { - if (!bitmap_is_set(lb_dps->nb_ls_map, ods[i]->index)) { - bitmap_set1(lb_dps->nb_ls_map, ods[i]->index); - lb_dps->n_nb_ls++; - } - } -} - -struct ovn_lb_group_datapaths * -ovn_lb_group_datapaths_create(const struct ovn_lb_group *lb_group, - size_t max_ls_datapaths, - size_t max_lr_datapaths) -{ - struct ovn_lb_group_datapaths *lb_group_dps = - xzalloc(sizeof *lb_group_dps); - lb_group_dps->lb_group = lb_group; - lb_group_dps->ls = xmalloc(max_ls_datapaths * sizeof *lb_group_dps->ls); - lb_group_dps->lr = xmalloc(max_lr_datapaths * sizeof *lb_group_dps->lr); - - return lb_group_dps; -} - -void -ovn_lb_group_datapaths_destroy(struct ovn_lb_group_datapaths *lb_group_dps) -{ - free(lb_group_dps->ls); - free(lb_group_dps->lr); - free(lb_group_dps); -} - -struct ovn_lb_group_datapaths * -ovn_lb_group_datapaths_find(const struct hmap *lb_group_dps_map, - const struct uuid *lb_group_uuid) -{ - struct ovn_lb_group_datapaths *lb_group_dps; - size_t hash = uuid_hash(lb_group_uuid); - - HMAP_FOR_EACH_WITH_HASH (lb_group_dps, hmap_node, hash, lb_group_dps_map) { - if (uuid_equals(&lb_group_dps->lb_group->uuid, lb_group_uuid)) { - return lb_group_dps; - } - } - return NULL; -} diff --git a/lib/lb.h b/lib/lb.h index b8e3c1e8fb..90ac39ee5a 100644 --- a/lib/lb.h +++ b/lib/lb.h @@ -167,63 +167,6 @@ void ovn_lb_group_reinit( const struct nbrec_load_balancer_group *, const struct hmap *lbs); -struct ovn_lb_datapaths { - struct hmap_node hmap_node; - - const struct ovn_northd_lb *lb; - size_t n_nb_ls; - unsigned long *nb_ls_map; - - size_t n_nb_lr; - unsigned long *nb_lr_map; -}; - -struct ovn_lb_datapaths *ovn_lb_datapaths_create(const struct ovn_northd_lb *, - size_t n_ls_datapaths, - size_t n_lr_datapaths); -struct ovn_lb_datapaths *ovn_lb_datapaths_find(const struct hmap *, - const struct uuid *); -void ovn_lb_datapaths_destroy(struct ovn_lb_datapaths *); -void ovn_lb_datapaths_add_lr(struct ovn_lb_datapaths *, size_t n, - struct ovn_datapath **); -void ovn_lb_datapaths_add_ls(struct ovn_lb_datapaths *, size_t n, - struct ovn_datapath **); - -struct ovn_lb_group_datapaths { - struct hmap_node hmap_node; - - const struct ovn_lb_group *lb_group; - - /* Datapaths to which 'lb_group' is applied. */ - size_t n_ls; - struct ovn_datapath **ls; - size_t n_lr; - struct ovn_datapath **lr; -}; - -struct ovn_lb_group_datapaths *ovn_lb_group_datapaths_create( - const struct ovn_lb_group *, size_t max_ls_datapaths, - size_t max_lr_datapaths); - -void ovn_lb_group_datapaths_destroy(struct ovn_lb_group_datapaths *); -struct ovn_lb_group_datapaths *ovn_lb_group_datapaths_find( - const struct hmap *lb_group_dps, const struct uuid *); - -static inline void -ovn_lb_group_datapaths_add_ls(struct ovn_lb_group_datapaths *lbg_dps, size_t n, - struct ovn_datapath **ods) -{ - memcpy(&lbg_dps->ls[lbg_dps->n_ls], ods, n * sizeof *ods); - lbg_dps->n_ls += n; -} - -static inline void -ovn_lb_group_datapaths_add_lr(struct ovn_lb_group_datapaths *lbg_dps, - struct ovn_datapath *lr) -{ - lbg_dps->lr[lbg_dps->n_lr++] = lr; -} - struct ovn_controller_lb { struct hmap_node hmap_node; diff --git a/northd/automake.mk b/northd/automake.mk index 7c6d56a4ff..19abb0dece 100644 --- a/northd/automake.mk +++ b/northd/automake.mk @@ -35,7 +35,9 @@ northd_ovn_northd_SOURCES = \ northd/ipam.c \ northd/ipam.h \ northd/lflow-mgr.c \ - northd/lflow-mgr.h + northd/lflow-mgr.h \ + northd/lb.c \ + northd/lb.h northd_ovn_northd_LDADD = \ lib/libovn.la \ $(OVSDB_LIBDIR)/libovsdb.la \ diff --git a/northd/en-lr-stateful.c b/northd/en-lr-stateful.c index 4287aaddc9..0c3b841050 100644 --- a/northd/en-lr-stateful.c +++ b/northd/en-lr-stateful.c @@ -33,6 +33,7 @@ #include "en-lb-data.h" #include "en-lr-nat.h" #include "en-lr-stateful.h" +#include "lb.h" #include "lib/inc-proc-eng.h" #include "lib/lb.h" #include "lib/ovn-nb-idl.h" diff --git a/northd/lb.c b/northd/lb.c new file mode 100644 index 0000000000..e6c8a51911 --- /dev/null +++ b/northd/lb.c @@ -0,0 +1,121 @@ +/* + * Copyright (c) 2024, Red Hat, Inc. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at: + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#include + +/* OVS includes */ +#include "lib/bitmap.h" + +/* OVN includes */ +#include "lb.h" +#include "lib/lb.h" +#include "northd.h" + +/* lb datapaths functions */ +struct ovn_lb_datapaths * +ovn_lb_datapaths_create(const struct ovn_northd_lb *lb, size_t n_ls_datapaths, + size_t n_lr_datapaths) +{ + struct ovn_lb_datapaths *lb_dps = xzalloc(sizeof *lb_dps); + lb_dps->lb = lb; + lb_dps->nb_ls_map = bitmap_allocate(n_ls_datapaths); + lb_dps->nb_lr_map = bitmap_allocate(n_lr_datapaths); + + return lb_dps; +} + +void +ovn_lb_datapaths_destroy(struct ovn_lb_datapaths *lb_dps) +{ + bitmap_free(lb_dps->nb_lr_map); + bitmap_free(lb_dps->nb_ls_map); + free(lb_dps); +} + +void +ovn_lb_datapaths_add_lr(struct ovn_lb_datapaths *lb_dps, size_t n, + struct ovn_datapath **ods) +{ + for (size_t i = 0; i < n; i++) { + if (!bitmap_is_set(lb_dps->nb_lr_map, ods[i]->index)) { + bitmap_set1(lb_dps->nb_lr_map, ods[i]->index); + lb_dps->n_nb_lr++; + } + } +} + +void +ovn_lb_datapaths_add_ls(struct ovn_lb_datapaths *lb_dps, size_t n, + struct ovn_datapath **ods) +{ + for (size_t i = 0; i < n; i++) { + if (!bitmap_is_set(lb_dps->nb_ls_map, ods[i]->index)) { + bitmap_set1(lb_dps->nb_ls_map, ods[i]->index); + lb_dps->n_nb_ls++; + } + } +} + +struct ovn_lb_datapaths * +ovn_lb_datapaths_find(const struct hmap *lb_dps_map, + const struct uuid *lb_uuid) +{ + struct ovn_lb_datapaths *lb_dps; + size_t hash = uuid_hash(lb_uuid); + HMAP_FOR_EACH_WITH_HASH (lb_dps, hmap_node, hash, lb_dps_map) { + if (uuid_equals(&lb_dps->lb->nlb->header_.uuid, lb_uuid)) { + return lb_dps; + } + } + return NULL; +} + +struct ovn_lb_group_datapaths * +ovn_lb_group_datapaths_create(const struct ovn_lb_group *lb_group, + size_t max_ls_datapaths, + size_t max_lr_datapaths) +{ + struct ovn_lb_group_datapaths *lb_group_dps = + xzalloc(sizeof *lb_group_dps); + lb_group_dps->lb_group = lb_group; + lb_group_dps->ls = xmalloc(max_ls_datapaths * sizeof *lb_group_dps->ls); + lb_group_dps->lr = xmalloc(max_lr_datapaths * sizeof *lb_group_dps->lr); + + return lb_group_dps; +} + +void +ovn_lb_group_datapaths_destroy(struct ovn_lb_group_datapaths *lb_group_dps) +{ + free(lb_group_dps->ls); + free(lb_group_dps->lr); + free(lb_group_dps); +} + +struct ovn_lb_group_datapaths * +ovn_lb_group_datapaths_find(const struct hmap *lb_group_dps_map, + const struct uuid *lb_group_uuid) +{ + struct ovn_lb_group_datapaths *lb_group_dps; + size_t hash = uuid_hash(lb_group_uuid); + + HMAP_FOR_EACH_WITH_HASH (lb_group_dps, hmap_node, hash, lb_group_dps_map) { + if (uuid_equals(&lb_group_dps->lb_group->uuid, lb_group_uuid)) { + return lb_group_dps; + } + } + return NULL; +} diff --git a/northd/lb.h b/northd/lb.h new file mode 100644 index 0000000000..eeef2ea260 --- /dev/null +++ b/northd/lb.h @@ -0,0 +1,82 @@ +/* + * Copyright (c) 2024, Red Hat, Inc. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at: + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#ifndef OVN_NORTHD_LB_H +#define OVN_NORTHD_LB_H 1 + +#include "openvswitch/hmap.h" +#include "uuid.h" + +struct ovn_lb_datapaths { + struct hmap_node hmap_node; + + const struct ovn_northd_lb *lb; + size_t n_nb_ls; + unsigned long *nb_ls_map; + + size_t n_nb_lr; + unsigned long *nb_lr_map; +}; + +struct ovn_lb_datapaths *ovn_lb_datapaths_create(const struct ovn_northd_lb *, + size_t n_ls_datapaths, + size_t n_lr_datapaths); +struct ovn_lb_datapaths *ovn_lb_datapaths_find(const struct hmap *, + const struct uuid *); +void ovn_lb_datapaths_destroy(struct ovn_lb_datapaths *); + +struct ovn_datapath; +void ovn_lb_datapaths_add_lr(struct ovn_lb_datapaths *, size_t n, + struct ovn_datapath **); +void ovn_lb_datapaths_add_ls(struct ovn_lb_datapaths *, size_t n, + struct ovn_datapath **); + +struct ovn_lb_group_datapaths { + struct hmap_node hmap_node; + + const struct ovn_lb_group *lb_group; + + /* Datapaths to which 'lb_group' is applied. */ + size_t n_ls; + struct ovn_datapath **ls; + size_t n_lr; + struct ovn_datapath **lr; +}; + +struct ovn_lb_group_datapaths *ovn_lb_group_datapaths_create( + const struct ovn_lb_group *, size_t max_ls_datapaths, + size_t max_lr_datapaths); + +void ovn_lb_group_datapaths_destroy(struct ovn_lb_group_datapaths *); +struct ovn_lb_group_datapaths *ovn_lb_group_datapaths_find( + const struct hmap *lb_group_dps, const struct uuid *); + +static inline void +ovn_lb_group_datapaths_add_ls(struct ovn_lb_group_datapaths *lbg_dps, size_t n, + struct ovn_datapath **ods) +{ + memcpy(&lbg_dps->ls[lbg_dps->n_ls], ods, n * sizeof *ods); + lbg_dps->n_ls += n; +} + +static inline void +ovn_lb_group_datapaths_add_lr(struct ovn_lb_group_datapaths *lbg_dps, + struct ovn_datapath *lr) +{ + lbg_dps->lr[lbg_dps->n_lr++] = lr; +} + +#endif /* OVN_NORTHD_LB_H */ \ No newline at end of file diff --git a/northd/northd.c b/northd/northd.c index a9cb0b6267..08732abbfa 100644 --- a/northd/northd.c +++ b/northd/northd.c @@ -31,6 +31,7 @@ #include "openvswitch/hmap.h" #include "openvswitch/json.h" #include "ovn/lex.h" +#include "lb.h" #include "lib/chassis-index.h" #include "lib/ip-mcast-index.h" #include "lib/static-mac-binding-index.h" From patchwork Thu Jan 11 15:32:48 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Numan Siddique X-Patchwork-Id: 1885680 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::136; helo=smtp3.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4T9pbp2FTdz1yPp for ; Fri, 12 Jan 2024 02:33:42 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 6963561B3D; Thu, 11 Jan 2024 15:33:40 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 6963561B3D X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id mwDbkJTP6PQk; Thu, 11 Jan 2024 15:33:38 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp3.osuosl.org (Postfix) with ESMTPS id 8C8AB6F4C6; Thu, 11 Jan 2024 15:33:37 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 8C8AB6F4C6 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 61B73C0077; Thu, 11 Jan 2024 15:33:37 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp4.osuosl.org (smtp4.osuosl.org [IPv6:2605:bc80:3010::137]) by lists.linuxfoundation.org (Postfix) with ESMTP id 4D9E8C0077 for ; Thu, 11 Jan 2024 15:33:36 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 2960242C86 for ; Thu, 11 Jan 2024 15:33:36 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 2960242C86 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id CumuU-sehe_L for ; Thu, 11 Jan 2024 15:33:34 +0000 (UTC) Received: from relay2-d.mail.gandi.net (relay2-d.mail.gandi.net [IPv6:2001:4b98:dc4:8::222]) by smtp4.osuosl.org (Postfix) with ESMTPS id 5F1A642C69 for ; Thu, 11 Jan 2024 15:33:34 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 5F1A642C69 Received: by mail.gandi.net (Postfix) with ESMTPSA id 1DC6640002; Thu, 11 Jan 2024 15:33:31 +0000 (UTC) From: numans@ovn.org To: dev@openvswitch.org Date: Thu, 11 Jan 2024 10:32:48 -0500 Message-ID: <20240111153248.2790435-1-numans@ovn.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240111152752.2789854-1-numans@ovn.org> References: <20240111152752.2789854-1-numans@ovn.org> MIME-Version: 1.0 X-GND-Sasl: numans@ovn.org Subject: [ovs-dev] [PATCH ovn v5 11/16] northd: Handle lb changes in lflow engine. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Numan Siddique Since northd tracked data has the changed lb data, northd engine handler for lflow engine now handles the lb changes incrementally. All the lflows generated for each lb is stored in the ovn_lb_datapaths->lflow_ref and this is used similar to how we handle ovn_port changes. Signed-off-by: Numan Siddique Acked-by: Han Zhou --- northd/en-lflow.c | 11 ++--- northd/lb.c | 3 ++ northd/lb.h | 26 ++++++++++++ northd/lflow-mgr.c | 47 +++++++++++++++++----- northd/northd.c | 98 +++++++++++++++++++++++++++++++++++++++------ northd/northd.h | 4 ++ tests/ovn-northd.at | 30 ++++++++++---- 7 files changed, 184 insertions(+), 35 deletions(-) diff --git a/northd/en-lflow.c b/northd/en-lflow.c index fef9a1352d..205605578f 100644 --- a/northd/en-lflow.c +++ b/northd/en-lflow.c @@ -123,11 +123,6 @@ lflow_northd_handler(struct engine_node *node, return false; } - /* Fall back to recompute if load balancers have changed. */ - if (northd_has_lbs_in_tracked_data(&northd_data->trk_data)) { - return false; - } - const struct engine_context *eng_ctx = engine_get_context(); struct lflow_data *lflow_data = data; @@ -140,6 +135,12 @@ lflow_northd_handler(struct engine_node *node, return false; } + if (!lflow_handle_northd_lb_changes(eng_ctx->ovnsb_idl_txn, + &northd_data->trk_data.trk_lbs, + &lflow_input, lflow_data->lflow_table)) { + return false; + } + engine_set_node_state(node, EN_UPDATED); return true; } diff --git a/northd/lb.c b/northd/lb.c index e6c8a51911..5fca41e049 100644 --- a/northd/lb.c +++ b/northd/lb.c @@ -21,6 +21,7 @@ /* OVN includes */ #include "lb.h" +#include "lflow-mgr.h" #include "lib/lb.h" #include "northd.h" @@ -33,6 +34,7 @@ ovn_lb_datapaths_create(const struct ovn_northd_lb *lb, size_t n_ls_datapaths, lb_dps->lb = lb; lb_dps->nb_ls_map = bitmap_allocate(n_ls_datapaths); lb_dps->nb_lr_map = bitmap_allocate(n_lr_datapaths); + lb_dps->lflow_ref = lflow_ref_create(); return lb_dps; } @@ -42,6 +44,7 @@ ovn_lb_datapaths_destroy(struct ovn_lb_datapaths *lb_dps) { bitmap_free(lb_dps->nb_lr_map); bitmap_free(lb_dps->nb_ls_map); + lflow_ref_destroy(lb_dps->lflow_ref); free(lb_dps); } diff --git a/northd/lb.h b/northd/lb.h index eeef2ea260..de677ca4ef 100644 --- a/northd/lb.h +++ b/northd/lb.h @@ -20,6 +20,8 @@ #include "openvswitch/hmap.h" #include "uuid.h" +struct lflow_ref; + struct ovn_lb_datapaths { struct hmap_node hmap_node; @@ -29,6 +31,30 @@ struct ovn_lb_datapaths { size_t n_nb_lr; unsigned long *nb_lr_map; + + /* Reference of lflows generated for this load balancer. + * + * This data is initialized and destroyed by the en_northd node, but + * populated and used only by the en_lflow node. Ideally this data should + * be maintained as part of en_lflow's data (struct lflow_data): a hash + * index from ovn_port key to lflows. However, it would be less efficient + * and more complex: + * + * 1. It would require an extra search (using the index) to find the + * lflows. + * + * 2. Building the index needs to be thread-safe, using either a global + * lock which is obviously less efficient, or hash-based lock array which + * is more complex. + * + * Maintaining the lflow_ref here is more straightforward. The drawback is + * that we need to keep in mind that this data belongs to en_lflow node, + * so never access it from any other nodes. + * + * 'lflow_ref' is used to reference logical flows generated for this + * load balancer. + */ + struct lflow_ref *lflow_ref; }; struct ovn_lb_datapaths *ovn_lb_datapaths_create(const struct ovn_northd_lb *, diff --git a/northd/lflow-mgr.c b/northd/lflow-mgr.c index 3cf9696f6e..6cb2a367fe 100644 --- a/northd/lflow-mgr.c +++ b/northd/lflow-mgr.c @@ -375,7 +375,15 @@ struct lflow_ref_node { /* The lflow. */ struct ovn_lflow *lflow; - /* Index id of the datapath this lflow_ref_node belongs to. */ + /* Indicates of the lflow was added with dp_group or not using + * ovn_lflow_add_with_dp_group() macro. */ + bool dpgrp_lflow; + /* dpgrp bitmap and bitmap length. Valid only of dpgrp_lflow is true. */ + unsigned long *dpgrp_bitmap; + size_t dpgrp_bitmap_len; + + /* Index id of the datapath this lflow_ref_node belongs to. + * Valid only if dpgrp_lflow is false. */ size_t dp_index; /* Indicates if the lflow_ref_node for an lflow - L(M, A) is linked @@ -429,9 +437,19 @@ lflow_ref_unlink_lflows(struct lflow_ref *lflow_ref) struct lflow_ref_node *lrn; LIST_FOR_EACH (lrn, lflow_list_node, &lflow_ref->lflows_ref_list) { - if (dec_dp_refcnt(&lrn->lflow->dp_refcnts_map, - lrn->dp_index)) { - bitmap_set0(lrn->lflow->dpg_bitmap, lrn->dp_index); + if (lrn->dpgrp_lflow) { + size_t index; + BITMAP_FOR_EACH_1 (index, lrn->dpgrp_bitmap_len, + lrn->dpgrp_bitmap) { + if (dec_dp_refcnt(&lrn->lflow->dp_refcnts_map, index)) { + bitmap_set0(lrn->lflow->dpg_bitmap, lrn->dp_index); + } + } + } else { + if (dec_dp_refcnt(&lrn->lflow->dp_refcnts_map, + lrn->dp_index)) { + bitmap_set0(lrn->lflow->dpg_bitmap, lrn->dp_index); + } } lrn->linked = false; @@ -502,18 +520,26 @@ lflow_table_add_lflow(struct lflow_table *lflow_table, io_port, ctrl_meter, stage_hint, where); if (lflow_ref) { - /* lflow referencing is only supported if 'od' is not NULL. */ - ovs_assert(od); - struct lflow_ref_node *lrn = lflow_ref_node_find(&lflow_ref->lflow_ref_nodes, lflow, hash); if (!lrn) { lrn = xzalloc(sizeof *lrn); lrn->lflow = lflow; - lrn->dp_index = od->index; + lrn->dpgrp_lflow = !od; + if (lrn->dpgrp_lflow) { + lrn->dpgrp_bitmap = bitmap_clone(dp_bitmap, dp_bitmap_len); + lrn->dpgrp_bitmap_len = dp_bitmap_len; + + size_t index; + BITMAP_FOR_EACH_1 (index, dp_bitmap_len, dp_bitmap) { + inc_dp_refcnt(&lflow->dp_refcnts_map, index); + } + } else { + lrn->dp_index = od->index; + inc_dp_refcnt(&lflow->dp_refcnts_map, lrn->dp_index); + } ovs_list_insert(&lflow_ref->lflows_ref_list, &lrn->lflow_list_node); - inc_dp_refcnt(&lflow->dp_refcnts_map, lrn->dp_index); ovs_list_insert(&lflow->referenced_by, &lrn->ref_list_node); hmap_insert(&lflow_ref->lflow_ref_nodes, &lrn->ref_node, hash); @@ -1257,5 +1283,8 @@ lflow_ref_node_destroy(struct lflow_ref_node *lrn, } ovs_list_remove(&lrn->lflow_list_node); ovs_list_remove(&lrn->ref_list_node); + if (lrn->dpgrp_lflow) { + bitmap_free(lrn->dpgrp_bitmap); + } free(lrn); } diff --git a/northd/northd.c b/northd/northd.c index 08732abbfa..6225dfe541 100644 --- a/northd/northd.c +++ b/northd/northd.c @@ -7477,7 +7477,7 @@ build_lb_rules_pre_stateful(struct lflow_table *lflows, ovn_lflow_add_with_dp_group( lflows, lb_dps->nb_ls_map, ods_size(ls_datapaths), S_SWITCH_IN_PRE_STATEFUL, 120, ds_cstr(match), ds_cstr(action), - &lb->nlb->header_, NULL); + &lb->nlb->header_, lb_dps->lflow_ref); } } @@ -7922,7 +7922,7 @@ build_lb_rules(struct lflow_table *lflows, struct ovn_lb_datapaths *lb_dps, } build_lb_affinity_ls_flows(lflows, lb_dps, lb_vip, ls_datapaths, - NULL); + lb_dps->lflow_ref); unsigned long *dp_non_meter = NULL; bool build_non_meter = false; @@ -7946,7 +7946,7 @@ build_lb_rules(struct lflow_table *lflows, struct ovn_lb_datapaths *lb_dps, lflows, od, S_SWITCH_IN_LB, priority, ds_cstr(match), ds_cstr(action), NULL, meter, &lb->nlb->header_, - NULL); + lb_dps->lflow_ref); } } if (!reject || build_non_meter) { @@ -7954,7 +7954,7 @@ build_lb_rules(struct lflow_table *lflows, struct ovn_lb_datapaths *lb_dps, lflows, dp_non_meter ? dp_non_meter : lb_dps->nb_ls_map, ods_size(ls_datapaths), S_SWITCH_IN_LB, priority, ds_cstr(match), ds_cstr(action), &lb->nlb->header_, - NULL); + lb_dps->lflow_ref); } bitmap_free(dp_non_meter); } @@ -9375,7 +9375,7 @@ build_lswitch_arp_nd_service_monitor(const struct ovn_lb_datapaths *lb_dps, S_SWITCH_IN_ARP_ND_RSP, 110, ds_cstr(match), ds_cstr(actions), &lb->nlb->header_, - NULL); + lb_dps->lflow_ref); } } } @@ -11397,7 +11397,8 @@ build_lrouter_nat_flows_for_lb( if (!od->n_l3dgw_ports) { bitmap_set1(gw_dp_bitmap[type], index); } else { - build_distr_lrouter_nat_flows_for_lb(&ctx, type, od, NULL); + build_distr_lrouter_nat_flows_for_lb(&ctx, type, od, + lb_dps->lflow_ref); } if (lb->affinity_timeout) { @@ -11418,17 +11419,17 @@ build_lrouter_nat_flows_for_lb( * S_ROUTER_IN_DNAT stage. */ ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_UNSNAT, 120, ds_cstr(&unsnat_match), "next;", - &lb->nlb->header_, NULL); + &lb->nlb->header_, lb_dps->lflow_ref); } } for (size_t type = 0; type < LROUTER_NAT_LB_FLOW_MAX; type++) { build_gw_lrouter_nat_flows_for_lb(&ctx, type, lr_datapaths, gw_dp_bitmap[type], - NULL); + lb_dps->lflow_ref); build_lb_affinity_lr_flows(lflows, lb, lb_vip, ds_cstr(match), aff_action[type], aff_dp_bitmap[type], - lr_datapaths, NULL); + lr_datapaths, lb_dps->lflow_ref); } ds_destroy(&unsnat_match); @@ -11477,7 +11478,7 @@ build_lswitch_flows_for_lb(struct ovn_lb_datapaths *lb_dps, od->nbs->copp, meter_groups), &lb->nlb->header_, - NULL); + lb_dps->lflow_ref); } /* Ignore L4 port information in the key because fragmented packets * may not have L4 information. The pre-stateful table will send @@ -11527,7 +11528,7 @@ build_lrouter_defrag_flows_for_lb(struct ovn_lb_datapaths *lb_dps, ovn_lflow_add_with_dp_group( lflows, lb_dps->nb_lr_map, ods_size(lr_datapaths), S_ROUTER_IN_DEFRAG, prio, ds_cstr(match), "ct_dnat;", - &lb_dps->lb->nlb->header_, NULL); + &lb_dps->lb->nlb->header_, lb_dps->lflow_ref); } } @@ -11569,7 +11570,7 @@ build_lrouter_flows_for_lb(struct ovn_lb_datapaths *lb_dps, copp_meter_get(COPP_EVENT_ELB, od->nbr->copp, meter_groups), - &lb->nlb->header_, NULL); + &lb->nlb->header_, lb_dps->lflow_ref); } } @@ -11579,7 +11580,7 @@ build_lrouter_flows_for_lb(struct ovn_lb_datapaths *lb_dps, ovn_lflow_add(lflows, od, S_ROUTER_OUT_SNAT, 120, "flags.skip_snat_for_lb == 1 && ip", "next;", - NULL); + lb_dps->lflow_ref); } } } @@ -16388,6 +16389,7 @@ void build_lflows(struct ovsdb_idl_txn *ovnsb_txn, void lflow_reset_northd_refs(struct lflow_input *lflow_input) { + struct ovn_lb_datapaths *lb_dps; struct ovn_port *op; HMAP_FOR_EACH (op, key_node, lflow_input->ls_ports) { @@ -16399,6 +16401,10 @@ lflow_reset_northd_refs(struct lflow_input *lflow_input) lflow_ref_clear(op->lflow_ref); lflow_ref_clear(op->stateful_lflow_ref); } + + HMAP_FOR_EACH (lb_dps, hmap_node, lflow_input->lb_datapaths_map) { + lflow_ref_clear(lb_dps->lflow_ref); + } } bool @@ -16575,6 +16581,72 @@ lflow_handle_northd_port_changes(struct ovsdb_idl_txn *ovnsb_txn, return true; } +bool +lflow_handle_northd_lb_changes(struct ovsdb_idl_txn *ovnsb_txn, + struct tracked_lbs *trk_lbs, + struct lflow_input *lflow_input, + struct lflow_table *lflows) +{ + struct ovn_lb_datapaths *lb_dps; + struct hmapx_node *hmapx_node; + HMAPX_FOR_EACH (hmapx_node, &trk_lbs->deleted) { + lb_dps = hmapx_node->data; + + lflow_ref_resync_flows( + lb_dps->lflow_ref, lflows, ovnsb_txn, lflow_input->ls_datapaths, + lflow_input->lr_datapaths, + lflow_input->ovn_internal_version_changed, + lflow_input->sbrec_logical_flow_table, + lflow_input->sbrec_logical_dp_group_table); + } + + HMAPX_FOR_EACH (hmapx_node, &trk_lbs->crupdated) { + lb_dps = hmapx_node->data; + + /* unlink old lflows. */ + lflow_ref_unlink_lflows(lb_dps->lflow_ref); + + /* Generate new lflows. */ + struct ds match = DS_EMPTY_INITIALIZER; + struct ds actions = DS_EMPTY_INITIALIZER; + + build_lswitch_arp_nd_service_monitor(lb_dps, lflow_input->ls_ports, + lflows, &actions, + &match); + build_lrouter_defrag_flows_for_lb(lb_dps, lflows, + lflow_input->lr_datapaths, &match); + build_lrouter_flows_for_lb(lb_dps, lflows, + lflow_input->meter_groups, + lflow_input->lr_datapaths, + lflow_input->lr_stateful_table, + lflow_input->features, + lflow_input->svc_monitor_map, + &match, &actions); + build_lswitch_flows_for_lb(lb_dps, lflows, + lflow_input->meter_groups, + lflow_input->ls_datapaths, + lflow_input->features, + lflow_input->svc_monitor_map, + &match, &actions); + + ds_destroy(&match); + ds_destroy(&actions); + + /* Sync the new flows to SB. */ + bool handled = lflow_ref_sync_lflows( + lb_dps->lflow_ref, lflows, ovnsb_txn, lflow_input->ls_datapaths, + lflow_input->lr_datapaths, + lflow_input->ovn_internal_version_changed, + lflow_input->sbrec_logical_flow_table, + lflow_input->sbrec_logical_dp_group_table); + if (!handled) { + return false; + } + } + + return true; +} + static bool mirror_needs_update(const struct nbrec_mirror *nb_mirror, const struct sbrec_mirror *sb_mirror) diff --git a/northd/northd.h b/northd/northd.h index 42b4eee607..d2640f38d7 100644 --- a/northd/northd.h +++ b/northd/northd.h @@ -675,6 +675,10 @@ bool lflow_handle_northd_port_changes(struct ovsdb_idl_txn *ovnsb_txn, struct tracked_ovn_ports *, struct lflow_input *, struct lflow_table *lflows); +bool lflow_handle_northd_lb_changes(struct ovsdb_idl_txn *ovnsb_txn, + struct tracked_lbs *, + struct lflow_input *, + struct lflow_table *lflows); bool northd_handle_sb_port_binding_changes( const struct sbrec_port_binding_table *, struct hmap *ls_ports, struct hmap *lr_ports); diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at index f7d47fc7e4..80374444fd 100644 --- a/tests/ovn-northd.at +++ b/tests/ovn-northd.at @@ -10519,7 +10519,7 @@ check ovn-nbctl --wait=sb lb-add lb1 10.0.0.10:80 10.0.0.3:80 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE(1) @@ -10529,21 +10529,26 @@ check ovn-nbctl --wait=sb set load_balancer . ip_port_mappings:10.0.0.3=sw0-p1:1 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE +check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats +check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb set load_balancer . options:foo=bar check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE +check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb -- lb-add lb2 20.0.0.10:80 20.0.0.20:80 -- lb-add lb3 30.0.0.10:80 30.0.0.20:80 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE(1) @@ -10553,7 +10558,7 @@ check ovn-nbctl --wait=sb -- lb-del lb2 -- lb-del lb3 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE(1) @@ -10764,8 +10769,9 @@ check ovn-nbctl --wait=sb add load_balancer_group . load_Balancer $lb1_uuid check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb clear load_balancer_group . load_Balancer @@ -10780,7 +10786,7 @@ check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb add load_balancer_group . load_Balancer $lb1_uuid check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10789,6 +10795,7 @@ check ovn-nbctl --wait=sb add logical_switch sw0 load_balancer_group $lbg1_uuid check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute +check_engine_stats ls_stateful norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute compute @@ -10797,6 +10804,7 @@ check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb set load_balancer . options:bar=foo check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats ls_stateful norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute compute @@ -10806,6 +10814,7 @@ check ovn-nbctl --wait=sb set load_balancer lb1 vips:'"10.0.0.10:80"'='"10.0.0.1 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute +check_engine_stats ls_stateful norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10903,6 +10912,7 @@ check_engine_stats northd recompute nocompute check_engine_stats lr_stateful recompute nocompute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute compute +CHECK_NO_CHANGE_AFTER_RECOMPUTE # Add back lb group to logical switch and then delete it. check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats @@ -10912,6 +10922,7 @@ check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute compute +CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb clear logical_switch sw0 load_balancer_group -- \ @@ -10945,14 +10956,17 @@ check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute check_engine_stats lflow norecompute nocompute check_engine_stats sync_to_sb_lb norecompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb set load_balancer_group . load_balancer="$lb2_uuid,$lb3_uuid,$lb4_uuid" check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats ls_stateful norecompute compute check_engine_stats lr_stateful norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb set logical_switch sw0 load_balancer_group=$lbg1_uuid From patchwork Thu Jan 11 15:33:33 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Numan Siddique X-Patchwork-Id: 1885681 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::137; helo=smtp4.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp4.osuosl.org (smtp4.osuosl.org [IPv6:2605:bc80:3010::137]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4T9pcL6dGqz1yPp for ; Fri, 12 Jan 2024 02:34:10 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 9C33F42CC3; Thu, 11 Jan 2024 15:34:08 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 9C33F42CC3 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 8oOfNRAVIVvR; Thu, 11 Jan 2024 15:34:05 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp4.osuosl.org (Postfix) with ESMTPS id 7AA6240874; Thu, 11 Jan 2024 15:34:04 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 7AA6240874 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 5E638C007C; Thu, 11 Jan 2024 15:34:04 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 76C35C0077 for ; Thu, 11 Jan 2024 15:34:02 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 0ADE26F549 for ; Thu, 11 Jan 2024 15:34:01 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 0ADE26F549 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id q9OmeaJUZe1g for ; Thu, 11 Jan 2024 15:33:59 +0000 (UTC) Received: from relay7-d.mail.gandi.net (relay7-d.mail.gandi.net [IPv6:2001:4b98:dc4:8::227]) by smtp3.osuosl.org (Postfix) with ESMTPS id 7E7A56F500 for ; Thu, 11 Jan 2024 15:33:58 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 7E7A56F500 Received: by mail.gandi.net (Postfix) with ESMTPSA id 1298320015; Thu, 11 Jan 2024 15:33:55 +0000 (UTC) From: numans@ovn.org To: dev@openvswitch.org Date: Thu, 11 Jan 2024 10:33:33 -0500 Message-ID: <20240111153333.2790501-1-numans@ovn.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240111152752.2789854-1-numans@ovn.org> References: <20240111152752.2789854-1-numans@ovn.org> MIME-Version: 1.0 X-GND-Sasl: numans@ovn.org Subject: [ovs-dev] [PATCH ovn v5 12/16] northd: Add lr_stateful handler for lflow engine node. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Numan Siddique Signed-off-by: Numan Siddique --- northd/en-lflow.c | 27 +++ northd/en-lflow.h | 1 + northd/en-lr-stateful.c | 33 +++- northd/en-lr-stateful.h | 28 ++- northd/inc-proc-northd.c | 2 +- northd/northd.c | 358 ++++++++++++++++++++++----------------- northd/northd.h | 8 +- tests/ovn-northd.at | 48 +++--- 8 files changed, 317 insertions(+), 188 deletions(-) diff --git a/northd/en-lflow.c b/northd/en-lflow.c index 205605578f..2dc4a83557 100644 --- a/northd/en-lflow.c +++ b/northd/en-lflow.c @@ -162,6 +162,33 @@ lflow_port_group_handler(struct engine_node *node, void *data OVS_UNUSED) return true; } +bool +lflow_lr_stateful_handler(struct engine_node *node, void *data) +{ + struct ed_type_lr_stateful *lr_sful_data = + engine_get_input_data("lr_stateful", node); + + if (!lr_stateful_has_tracked_data(&lr_sful_data->trk_data) + || lr_sful_data->trk_data.vip_nats_changed) { + return false; + } + + const struct engine_context *eng_ctx = engine_get_context(); + struct lflow_data *lflow_data = data; + struct lflow_input lflow_input; + + lflow_get_input_data(node, &lflow_input); + if (!lflow_handle_lr_stateful_changes(eng_ctx->ovnsb_idl_txn, + &lr_sful_data->trk_data, + &lflow_input, + lflow_data->lflow_table)) { + return false; + } + + engine_set_node_state(node, EN_UPDATED); + return true; +} + void *en_lflow_init(struct engine_node *node OVS_UNUSED, struct engine_arg *arg OVS_UNUSED) { diff --git a/northd/en-lflow.h b/northd/en-lflow.h index f7325c56b1..1d813a2a29 100644 --- a/northd/en-lflow.h +++ b/northd/en-lflow.h @@ -20,5 +20,6 @@ void *en_lflow_init(struct engine_node *node, struct engine_arg *arg); void en_lflow_cleanup(void *data); bool lflow_northd_handler(struct engine_node *, void *data); bool lflow_port_group_handler(struct engine_node *, void *data); +bool lflow_lr_stateful_handler(struct engine_node *, void *data); #endif /* EN_LFLOW_H */ diff --git a/northd/en-lr-stateful.c b/northd/en-lr-stateful.c index 0c3b841050..60e9862253 100644 --- a/northd/en-lr-stateful.c +++ b/northd/en-lr-stateful.c @@ -40,6 +40,7 @@ #include "lib/ovn-sb-idl.h" #include "lib/ovn-util.h" #include "lib/stopwatch-names.h" +#include "lflow-mgr.h" #include "northd.h" VLOG_DEFINE_THIS_MODULE(en_lr_stateful); @@ -82,7 +83,7 @@ static void remove_lrouter_lb_reachable_ips(struct lr_stateful_record *, enum lb_neighbor_responder_mode, const struct sset *lb_ips_v4, const struct sset *lb_ips_v6); -static void lr_stateful_build_vip_nats(struct lr_stateful_record *); +static bool lr_stateful_build_vip_nats(struct lr_stateful_record *); /* 'lr_stateful' engine node manages the NB logical router LB data. */ @@ -110,6 +111,7 @@ en_lr_stateful_clear_tracked_data(void *data_) struct ed_type_lr_stateful *data = data_; hmapx_clear(&data->trk_data.crupdated); + data->trk_data.vip_nats_changed = false; } void @@ -182,6 +184,10 @@ lr_stateful_lb_data_handler(struct engine_node *node, void *data_) /* Add the lr_stateful_rec rec to the tracking data. */ hmapx_add(&data->trk_data.crupdated, lr_stateful_rec); + + if (!sset_is_empty(&lr_stateful_rec->vip_nats)) { + data->trk_data.vip_nats_changed = true; + } continue; } @@ -297,7 +303,9 @@ lr_stateful_lb_data_handler(struct engine_node *node, void *data_) * vip nats. */ HMAPX_FOR_EACH (hmapx_node, &data->trk_data.crupdated) { struct lr_stateful_record *lr_stateful_rec = hmapx_node->data; - lr_stateful_build_vip_nats(lr_stateful_rec); + if (lr_stateful_build_vip_nats(lr_stateful_rec)) { + data->trk_data.vip_nats_changed = true; + } lr_stateful_rec->has_lb_vip = od_has_lb_vip(lr_stateful_rec->od); } @@ -331,8 +339,13 @@ lr_stateful_lr_nat_handler(struct engine_node *node, void *data_) lrnat_rec, input_data.lb_datapaths_map, input_data.lbgrp_datapaths_map); + if (!sset_is_empty(&lr_stateful_rec->vip_nats)) { + data->trk_data.vip_nats_changed = true; + } } else { - lr_stateful_build_vip_nats(lr_stateful_rec); + if (lr_stateful_build_vip_nats(lr_stateful_rec)) { + data->trk_data.vip_nats_changed = true; + } } /* Add the lr_stateful_rec rec to the tracking data. */ @@ -432,6 +445,7 @@ lr_stateful_record_create(struct lr_stateful_table *table, lr_stateful_rec->od = lrnat_rec->od; lr_stateful_record_init(lr_stateful_rec, lb_datapaths_map, lbgrp_datapaths_map); + lr_stateful_rec->lflow_ref = lflow_ref_create(); hmap_insert(&table->entries, &lr_stateful_rec->key_node, uuid_hash(&lr_stateful_rec->od->nbr->header_.uuid)); @@ -446,6 +460,7 @@ lr_stateful_record_destroy(struct lr_stateful_record *lr_stateful_rec) ovn_lb_ip_set_destroy(lr_stateful_rec->lb_ips); lr_stateful_rec->lb_ips = NULL; sset_destroy(&lr_stateful_rec->vip_nats); + lflow_ref_destroy(lr_stateful_rec->lflow_ref); free(lr_stateful_rec); } @@ -624,10 +639,12 @@ remove_lrouter_lb_reachable_ips(struct lr_stateful_record *lr_stateful_rec, } } -static void +static bool lr_stateful_build_vip_nats(struct lr_stateful_record *lr_stateful_rec) { - sset_clear(&lr_stateful_rec->vip_nats); + struct sset old_vip_nats = SSET_INITIALIZER(&old_vip_nats); + sset_swap(&lr_stateful_rec->vip_nats, &old_vip_nats); + const char *external_ip; SSET_FOR_EACH (external_ip, &lr_stateful_rec->lrnat_rec->external_ips) { bool is_vip_nat = false; @@ -643,4 +660,10 @@ lr_stateful_build_vip_nats(struct lr_stateful_record *lr_stateful_rec) sset_add(&lr_stateful_rec->vip_nats, external_ip); } } + + bool vip_nats_changed = !sset_equals(&lr_stateful_rec->vip_nats, + &old_vip_nats); + sset_destroy(&old_vip_nats); + + return vip_nats_changed; } diff --git a/northd/en-lr-stateful.h b/northd/en-lr-stateful.h index 5d647d8863..b5891b2716 100644 --- a/northd/en-lr-stateful.h +++ b/northd/en-lr-stateful.h @@ -32,6 +32,7 @@ struct ovn_datapath; struct lr_nat_record; +struct lflow_ref; /* lr_stateful_table: This represents a table of logical routers with * stateful related data. @@ -56,6 +57,27 @@ struct lr_stateful_record { /* sset of vips which are also part of lr nats. */ struct sset vip_nats; + + /* 'lflow_ref' is used to reference logical flows generated for + * this lr_stateful_record. + * + * This data is initialized and destroyed by the en_lr_stateful node, + * but populated and used only by the en_lflow node. Ideally this data + * should be maintained as part of en_lflow's data. However, it would + * be less efficient and more complex: + * + * 1. It would require an extra search (using the index) to find the + * lflows. + * + * 2. Building the index needs to be thread-safe, using either a global + * lock which is obviously less efficient, or hash-based lock array which + * is more complex. + * + * Adding the lflow_ref here is more straightforward. The drawback is that + * we need to keep in mind that this data belongs to en_lflow node, so + * never access it from any other nodes. + */ + struct lflow_ref *lflow_ref; }; struct lr_stateful_table { @@ -75,6 +97,10 @@ struct lr_stateful_table { struct lr_stateful_tracked_data { /* Created or updated logical router with LB and/or NAT data. */ struct hmapx crupdated; /* Stores 'struct lr_stateful_record'. */ + + /* Indicates if any router's NATs changed which were also LB vips + * or vice versa. */ + bool vip_nats_changed; }; struct ed_type_lr_stateful { @@ -105,7 +131,7 @@ const struct lr_stateful_record *lr_stateful_table_find_by_index( static inline bool lr_stateful_has_tracked_data(struct lr_stateful_tracked_data *trk_data) { - return !hmapx_is_empty(&trk_data->crupdated); + return !hmapx_is_empty(&trk_data->crupdated) || trk_data->vip_nats_changed; } #endif /* EN_lr_stateful_H */ diff --git a/northd/inc-proc-northd.c b/northd/inc-proc-northd.c index 0e17bfe2e6..dcce79510a 100644 --- a/northd/inc-proc-northd.c +++ b/northd/inc-proc-northd.c @@ -228,11 +228,11 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, engine_add_input(&en_lflow, &en_sb_logical_flow, NULL); engine_add_input(&en_lflow, &en_sb_multicast_group, NULL); engine_add_input(&en_lflow, &en_sb_igmp_group, NULL); - engine_add_input(&en_lflow, &en_lr_stateful, NULL); engine_add_input(&en_lflow, &en_ls_stateful, NULL); engine_add_input(&en_lflow, &en_sb_logical_dp_group, NULL); engine_add_input(&en_lflow, &en_northd, lflow_northd_handler); engine_add_input(&en_lflow, &en_port_group, lflow_port_group_handler); + engine_add_input(&en_lflow, &en_lr_stateful, lflow_lr_stateful_handler); engine_add_input(&en_sync_to_sb_addr_set, &en_nb_address_set, sync_to_sb_addr_set_nb_address_set_handler); diff --git a/northd/northd.c b/northd/northd.c index 6225dfe541..5e49457bbe 100644 --- a/northd/northd.c +++ b/northd/northd.c @@ -12760,63 +12760,6 @@ build_ip_routing_flows_for_lrp( } } -/* Logical router ingress table IP_ROUTING : IP Routing. - * - * For the LSP 'op' of type router, if there are logical router ports other - * than the LSP's peer connected to the logical switch, then for routable - * addresses (such as NAT IPs, LB VIPs, etc.) on each of the connected router - * ports, add routes to the LSP's peer router. - */ -static void -build_ip_routing_flows_for_router_type_lsp( - struct ovn_port *op, const struct lr_stateful_table *lr_stateful_table, - const struct hmap *lr_ports, struct lflow_table *lflows, - struct lflow_ref *lflow_ref) -{ - ovs_assert(op->nbsp); - if (!lsp_is_router(op->nbsp)) { - return; - } - - struct ovn_port *peer = ovn_port_get_peer(lr_ports, op); - if (!peer || !peer->nbrp || !peer->lrp_networks.n_ipv4_addrs - || !op->od->n_router_ports) { - return; - } - - for (int i = 0; i < op->od->n_router_ports; i++) { - struct ovn_port *router_port = ovn_port_get_peer( - lr_ports, op->od->router_ports[i]); - if (!router_port || !router_port->nbrp || router_port == peer) { - continue; - } - - const struct lr_stateful_record *lr_stateful_rec = - lr_stateful_table_find_by_index(lr_stateful_table, - router_port->od->index); - - if (router_port->nbrp->ha_chassis_group || - router_port->nbrp->n_gateway_chassis) { - struct ovn_port_routable_addresses ra = - get_op_routable_addresses(router_port, lr_stateful_rec); - for (size_t j = 0; j < ra.n_addrs; j++) { - struct lport_addresses *laddrs = &ra.laddrs[j]; - for (size_t k = 0; k < laddrs->n_ipv4_addrs; k++) { - add_route(lflows, peer->od, peer, - peer->lrp_networks.ipv4_addrs[0].addr_s, - laddrs->ipv4_addrs[k].network_s, - laddrs->ipv4_addrs[k].plen, NULL, false, 0, - &peer->nbrp->header_, false, - ROUTE_PRIO_OFFSET_CONNECTED, - lflow_ref); - } - } - destroy_routable_addresses(&ra); - } - } - -} - static void build_static_route_flows_for_lrouter( struct ovn_datapath *od, const struct chassis_features *features, @@ -13058,42 +13001,6 @@ build_arp_resolve_flows_for_lrouter( lflow_ref); } -static void -routable_addresses_to_lflows(struct lflow_table *lflows, - struct ovn_port *router_port, - struct ovn_port *peer, - const struct lr_stateful_record *lr_stateful_rec, - struct ds *match, struct ds *actions, - struct lflow_ref *lflow_ref) -{ - struct ovn_port_routable_addresses ra = - get_op_routable_addresses(router_port, lr_stateful_rec); - if (!ra.n_addrs) { - return; - } - - for (size_t i = 0; i < ra.n_addrs; i++) { - ds_clear(match); - ds_put_format(match, "outport == %s && "REG_NEXT_HOP_IPV4" == {", - peer->json_key); - bool first = true; - for (size_t j = 0; j < ra.laddrs[i].n_ipv4_addrs; j++) { - if (!first) { - ds_put_cstr(match, ", "); - } - ds_put_cstr(match, ra.laddrs[i].ipv4_addrs[j].addr_s); - first = false; - } - ds_put_cstr(match, "}"); - - ds_clear(actions); - ds_put_format(actions, "eth.dst = %s; next;", ra.laddrs[i].ea_s); - ovn_lflow_add(lflows, peer->od, S_ROUTER_IN_ARP_RESOLVE, 100, - ds_cstr(match), ds_cstr(actions), lflow_ref); - } - destroy_routable_addresses(&ra); -} - /* Local router ingress table ARP_RESOLVE: ARP Resolution. * * Any unicast packet that reaches this table is an IP packet whose @@ -13336,52 +13243,6 @@ build_arp_resolve_flows_for_lsp( } } -static void -build_arp_resolve_flows_for_lsp_routable_addresses( - struct ovn_port *op, struct lflow_table *lflows, - const struct hmap *lr_ports, - const struct lr_stateful_table *lr_stateful_table, - struct ds *match, struct ds *actions, - struct lflow_ref *lflow_ref) -{ - if (!lsp_is_router(op->nbsp)) { - return; - } - - struct ovn_port *peer = ovn_port_get_peer(lr_ports, op); - if (!peer || !peer->nbrp) { - return; - } - - if (peer->od->nbr && - smap_get_bool(&peer->od->nbr->options, - "dynamic_neigh_routers", false)) { - return; - } - - for (size_t i = 0; i < op->od->n_router_ports; i++) { - struct ovn_port *router_port = - ovn_port_get_peer(lr_ports, op->od->router_ports[i]); - if (!router_port || !router_port->nbrp) { - continue; - } - - /* Skip the router port under consideration. */ - if (router_port == peer) { - continue; - } - - if (smap_get(&peer->od->nbr->options, "chassis") || peer->cr_port) { - const struct lr_stateful_record *lr_stateful_rec; - lr_stateful_rec = lr_stateful_table_find_by_index( - lr_stateful_table, router_port->od->index); - routable_addresses_to_lflows(lflows, router_port, peer, - lr_stateful_rec, match, actions, - lflow_ref); - } - } -} - static void build_icmperr_pkt_big_flows(struct ovn_port *op, int mtu, struct lflow_table *lflows, @@ -15598,8 +15459,6 @@ build_lrouter_nat_defrag_and_lb( static void build_lsp_lflows_for_lbnats(struct ovn_port *lsp, const struct lr_stateful_record *lr_stateful_rec, - const struct lr_stateful_table *lr_stateful_table, - const struct hmap *lr_ports, struct lflow_table *lflows, struct ds *match, struct ds *actions, @@ -15610,20 +15469,107 @@ build_lsp_lflows_for_lbnats(struct ovn_port *lsp, build_lswitch_rport_arp_req_flows_for_lbnats( lsp->peer, lr_stateful_rec, lsp->od, lsp, lflows, &lsp->nbsp->header_, lflow_ref); - build_ip_routing_flows_for_router_type_lsp(lsp, lr_stateful_table, - lr_ports, lflows, - lflow_ref); - build_arp_resolve_flows_for_lsp_routable_addresses( - lsp, lflows, lr_ports, lr_stateful_table, match, actions, lflow_ref); build_lswitch_ip_unicast_lookup_for_nats(lsp, lr_stateful_rec, lflows, match, actions, lflow_ref); } +/* Logical router ingress table IP_ROUTING : IP Routing. + * + * Adds the LRP 'lrp's routable addresses (addresses which can be routed via + * the LRP's datapath) as routable flows into the other router datapaths + * which are connected to the LRP's peer's logical switch. + * + * i.e If logical switch sw0 is conencted to the routers R0, R1 and R2, + * and if LRP of R0 has routable addresses (IP1 and IP2), then it adds + * the routes to reach these IPs in the R1 and R2's datapaths. + * + * This function also adds the ARP resolve flows for these addresses + * (IP1 and IP2) in the ARP_RESOLVE table of R1 and R2. + * */ +static void +build_routable_flows_for_router_port( + struct ovn_port *lrp, const struct lr_stateful_record *lr_stateful_rec, + struct lflow_table *lflows, + struct ds *match, + struct ds *actions) +{ + ovs_assert(lrp->nbrp && lrp->od == lr_stateful_rec->od); + + struct ovn_port *lsp_peer = lrp->peer; + if (!lsp_peer || !lsp_peer->nbsp) { + return; + } + + struct ovn_datapath *peer_ls = lsp_peer->od; + ovs_assert(peer_ls->nbs); + + struct ovn_port_routable_addresses ra = + get_op_routable_addresses(lrp, lr_stateful_rec); + + struct ovn_port *router_port; + + for (size_t i = 0; i < peer_ls->n_router_ports; i++) { + router_port = peer_ls->router_ports[i]->peer; + + if (router_port == lrp) { + continue; + } + + if (lrp->nbrp->ha_chassis_group || + lrp->nbrp->n_gateway_chassis) { + for (size_t j = 0; j < ra.n_addrs; j++) { + struct lport_addresses *laddrs = &ra.laddrs[j]; + for (size_t k = 0; k < laddrs->n_ipv4_addrs; k++) { + add_route(lflows, router_port->od, router_port, + router_port->lrp_networks.ipv4_addrs[0].addr_s, + laddrs->ipv4_addrs[k].network_s, + laddrs->ipv4_addrs[k].plen, NULL, false, 0, + &router_port->nbrp->header_, false, + ROUTE_PRIO_OFFSET_CONNECTED, + lrp->stateful_lflow_ref); + } + } + } + + bool dynamic_neigh_router = + smap_get_bool(&router_port->od->nbr->options, + "dynamic_neigh_routers", false); + + if (!dynamic_neigh_router && + (router_port->od->is_gw_router || router_port->cr_port)) { + + for (size_t k = 0; k < ra.n_addrs; k++) { + ds_clear(match); + ds_put_format(match, "outport == %s && " + REG_NEXT_HOP_IPV4" == {", + router_port->json_key); + bool first = true; + for (size_t j = 0; j < ra.laddrs[k].n_ipv4_addrs; j++) { + if (!first) { + ds_put_cstr(match, ", "); + } + ds_put_cstr(match, ra.laddrs[k].ipv4_addrs[j].addr_s); + first = false; + } + ds_put_cstr(match, "}"); + + ds_clear(actions); + ds_put_format(actions, "eth.dst = %s; next;", + ra.laddrs[k].ea_s); + ovn_lflow_add(lflows, router_port->od, S_ROUTER_IN_ARP_RESOLVE, + 100, ds_cstr(match), ds_cstr(actions), + lrp->stateful_lflow_ref); + } + } + } + + destroy_routable_addresses(&ra); +} + static void build_lbnat_lflows_iterate_by_lsp( struct ovn_port *op, const struct lr_stateful_table *lr_stateful_table, - const struct hmap *lr_ports, struct ds *match, struct ds *actions, - struct lflow_table *lflows) + struct ds *match, struct ds *actions, struct lflow_table *lflows) { ovs_assert(op->nbsp); @@ -15637,8 +15583,8 @@ build_lbnat_lflows_iterate_by_lsp( ovs_assert(lr_stateful_rec); build_lsp_lflows_for_lbnats(op, lr_stateful_rec, - lr_stateful_table, lr_ports, lflows, - match, actions, op->stateful_lflow_ref); + lflows,match, actions, + op->stateful_lflow_ref); } static void @@ -15648,6 +15594,8 @@ build_lrp_lflows_for_lbnats(struct ovn_port *op, struct ds *match, struct ds *actions, struct lflow_table *lflows) { + ovs_assert(op->nbrp && op->od == lr_stateful_rec->od); + /* Drop IP traffic destined to router owned IPs except if the IP is * also a SNAT IP. Those are dropped later, in stage * "lr_in_arp_resolve", if unSNAT was unsuccessful. @@ -15680,6 +15628,9 @@ build_lrp_lflows_for_lbnats(struct ovn_port *op, match, actions, op->stateful_lflow_ref); } +/* Builds the load balancer and NAT related flows for the router port 'op'. + * It uses the op->stateful_lflow_ref for lflow referencing. + */ static void build_lbnat_lflows_iterate_by_lrp( struct ovn_port *op, const struct lr_stateful_table *lr_stateful_table, @@ -15695,6 +15646,9 @@ build_lbnat_lflows_iterate_by_lrp( build_lrp_lflows_for_lbnats(op, lr_stateful_rec, meter_groups, match, actions, lflows); + + build_routable_flows_for_router_port(op, lr_stateful_rec, lflows, match, + actions); } static void @@ -15708,14 +15662,16 @@ build_lr_stateful_flows(const struct lr_stateful_record *lr_stateful_rec, const struct chassis_features *features) { build_lrouter_nat_defrag_and_lb(lr_stateful_rec, lflows, ls_ports, - lr_ports, match, actions, - meter_groups, features, NULL); + lr_ports, match, actions, meter_groups, + features, lr_stateful_rec->lflow_ref); build_lr_gateway_redirect_flows_for_nats(lr_stateful_rec->od, lr_stateful_rec->lrnat_rec, - lflows, match, actions, NULL); + lflows, match, actions, + lr_stateful_rec->lflow_ref); build_lrouter_arp_nd_for_datapath(lr_stateful_rec->od, lr_stateful_rec->lrnat_rec, lflows, - meter_groups, NULL); + meter_groups, + lr_stateful_rec->lflow_ref); } static void @@ -15946,7 +15902,7 @@ build_lflows_thread(void *arg) &lsi->actions, lsi->lflows); build_lbnat_lflows_iterate_by_lsp( - op, lsi->lr_stateful_table, lsi->lr_ports, &lsi->match, + op, lsi->lr_stateful_table, &lsi->match, &lsi->actions, lsi->lflows); } } @@ -16192,7 +16148,7 @@ build_lswitch_and_lrouter_flows( &lsi.actions, lsi.lflows); build_lbnat_lflows_iterate_by_lsp(op, lsi.lr_stateful_table, - lsi.lr_ports, &lsi.match, + &lsi.match, &lsi.actions, lsi.lflows); } HMAP_FOR_EACH (op, key_node, lr_ports) { @@ -16389,9 +16345,15 @@ void build_lflows(struct ovsdb_idl_txn *ovnsb_txn, void lflow_reset_northd_refs(struct lflow_input *lflow_input) { + const struct lr_stateful_record *lr_stateful_rec; struct ovn_lb_datapaths *lb_dps; struct ovn_port *op; + LR_STATEFUL_TABLE_FOR_EACH (lr_stateful_rec, + lflow_input->lr_stateful_table) { + lflow_ref_clear(lr_stateful_rec->lflow_ref); + } + HMAP_FOR_EACH (op, key_node, lflow_input->ls_ports) { lflow_ref_clear(op->lflow_ref); lflow_ref_clear(op->stateful_lflow_ref); @@ -16471,8 +16433,6 @@ lflow_handle_northd_port_changes(struct ovsdb_idl_txn *ovnsb_txn, /* Generate new lflows. */ build_lsp_lflows_for_lbnats(op, lr_stateful_rec, - lflow_input->lr_stateful_table, - lflow_input->lr_ports, lflows, &match, &actions, op->stateful_lflow_ref); handled = lflow_ref_sync_lflows( @@ -16534,8 +16494,6 @@ lflow_handle_northd_port_changes(struct ovsdb_idl_txn *ovnsb_txn, ovs_assert(lr_stateful_rec); build_lsp_lflows_for_lbnats(op, lr_stateful_rec, - lflow_input->lr_stateful_table, - lflow_input->lr_ports, lflows, &match, &actions, op->stateful_lflow_ref); handled = lflow_ref_sync_lflows( @@ -16647,6 +16605,86 @@ lflow_handle_northd_lb_changes(struct ovsdb_idl_txn *ovnsb_txn, return true; } +bool +lflow_handle_lr_stateful_changes(struct ovsdb_idl_txn *ovnsb_txn, + struct lr_stateful_tracked_data *trk_data, + struct lflow_input *lflow_input, + struct lflow_table *lflows) +{ + struct lr_stateful_record *lr_stateful_rec; + struct hmapx_node *hmapx_node; + + HMAPX_FOR_EACH (hmapx_node, &trk_data->crupdated) { + lr_stateful_rec = hmapx_node->data; + /* Unlink old lflows. */ + lflow_ref_unlink_lflows(lr_stateful_rec->lflow_ref); + + /* Generate new lflows. */ + struct ds match = DS_EMPTY_INITIALIZER; + struct ds actions = DS_EMPTY_INITIALIZER; + + build_lr_stateful_flows(lr_stateful_rec, lflows, lflow_input->ls_ports, + lflow_input->lr_ports, &match, &actions, + lflow_input->meter_groups, + lflow_input->features); + + /* Sync the new flows to SB. */ + bool handled = lflow_ref_sync_lflows( + lr_stateful_rec->lflow_ref, lflows, ovnsb_txn, + lflow_input->ls_datapaths, lflow_input->lr_datapaths, + lflow_input->ovn_internal_version_changed, + lflow_input->sbrec_logical_flow_table, + lflow_input->sbrec_logical_dp_group_table); + if (!handled) { + return false; + } + + struct ovn_port *op; + HMAP_FOR_EACH (op, dp_node, &lr_stateful_rec->od->ports) { + lflow_ref_unlink_lflows(op->stateful_lflow_ref); + + build_lbnat_lflows_iterate_by_lrp(op, + lflow_input->lr_stateful_table, + lflow_input->meter_groups, + &match, &actions, + lflows); + + handled = lflow_ref_sync_lflows( + op->stateful_lflow_ref, lflows, ovnsb_txn, + lflow_input->ls_datapaths, lflow_input->lr_datapaths, + lflow_input->ovn_internal_version_changed, + lflow_input->sbrec_logical_flow_table, + lflow_input->sbrec_logical_dp_group_table); + if (!handled) { + return false; + } + + if (op->peer && op->peer->nbsp) { + lflow_ref_unlink_lflows(op->peer->stateful_lflow_ref); + + build_lbnat_lflows_iterate_by_lsp( + op->peer, lflow_input->lr_stateful_table, &match, &actions, + lflows); + + handled = lflow_ref_sync_lflows( + op->peer->stateful_lflow_ref, lflows, ovnsb_txn, + lflow_input->ls_datapaths, lflow_input->lr_datapaths, + lflow_input->ovn_internal_version_changed, + lflow_input->sbrec_logical_flow_table, + lflow_input->sbrec_logical_dp_group_table); + if (!handled) { + return false; + } + } + } + + ds_destroy(&match); + ds_destroy(&actions); + } + + return true; +} + static bool mirror_needs_update(const struct nbrec_mirror *nb_mirror, const struct sbrec_mirror *sb_mirror) diff --git a/northd/northd.h b/northd/northd.h index d2640f38d7..95120d8fd8 100644 --- a/northd/northd.h +++ b/northd/northd.h @@ -639,7 +639,7 @@ struct ovn_port { * 'stateful_lflow_ref' is used for logical switch ports of type * 'patch/router' to reference logical flows generated fo this ovn_port * from the 'lr_stateful' record of the peer port's datapath. - */ + * */ struct lflow_ref *lflow_ref; struct lflow_ref *stateful_lflow_ref; }; @@ -666,6 +666,8 @@ void northd_indices_create(struct northd_data *data, struct ovsdb_idl *ovnsb_idl); struct lflow_table; +struct lr_stateful_tracked_data; + void build_lflows(struct ovsdb_idl_txn *ovnsb_txn, struct lflow_input *input_data, struct lflow_table *); @@ -679,6 +681,10 @@ bool lflow_handle_northd_lb_changes(struct ovsdb_idl_txn *ovnsb_txn, struct tracked_lbs *, struct lflow_input *, struct lflow_table *lflows); +bool lflow_handle_lr_stateful_changes(struct ovsdb_idl_txn *, + struct lr_stateful_tracked_data *, + struct lflow_input *, + struct lflow_table *lflows); bool northd_handle_sb_port_binding_changes( const struct sbrec_port_binding_table *, struct hmap *ls_ports, struct hmap *lr_ports); diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at index 80374444fd..88dfc7bbfc 100644 --- a/tests/ovn-northd.at +++ b/tests/ovn-northd.at @@ -12,6 +12,7 @@ m4_define([_DUMP_DB_TABLES], [ ovn-sbctl list meter >> $1 ovn-sbctl list meter_band >> $1 ovn-sbctl list port_group >> $1 + ovn-sbctl dump-flows > lflows_$1 ]) # CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10699,7 +10700,7 @@ check ovn-nbctl --wait=sb lr-lb-add lr0 lb1 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10709,7 +10710,7 @@ check ovn-nbctl --wait=sb set load_balancer lb1 vips:'"10.0.0.10:80"'='"10.0.0.1 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10719,7 +10720,7 @@ check ovn-nbctl --wait=sb clear load_Balancer lb1 vips check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10729,7 +10730,7 @@ check ovn-nbctl --wait=sb lb-add lb1 10.0.0.10:80 10.0.0.3:80 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10739,7 +10740,7 @@ check ovn-nbctl --wait=sb lb-add lb1 10.0.0.20:80 10.0.0.30:8080 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10747,6 +10748,7 @@ check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb lr-lb-del lr0 lb1 check_engine_stats lb_data norecompute compute check_engine_stats northd recompute nocompute +check_engine_stats lr_stateful recompute nocompute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10861,7 +10863,7 @@ check ovn-nbctl add logical_router lr0 load_balancer_group $lbg1_uuid check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10871,7 +10873,7 @@ check ovn-nbctl --wait=sb set load_balancer lb1 vips:'"10.0.0.10:80"'='"10.0.0.1 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10881,7 +10883,7 @@ check ovn-nbctl --wait=sb clear load_Balancer lb1 vips check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10891,7 +10893,7 @@ check ovn-nbctl --wait=sb lb-add lb1 10.0.0.10:80 10.0.0.3:80 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10901,7 +10903,7 @@ check ovn-nbctl --wait=sb lb-add lb1 10.0.0.20:80 10.0.0.30:8080 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10982,7 +10984,7 @@ check ovn-nbctl --wait=sb set logical_router lr1 load_balancer_group=$lbg1_uuid check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11010,7 +11012,7 @@ check ovn-nbctl --wait=sb lr-lb-add lr1 lb2 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11580,7 +11582,7 @@ check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb lr-nat-add lr0 dnat_and_snat 172.168.0.110 10.0.0.4 check_engine_stats northd norecompute compute check_engine_stats lr_nat norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_pb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11589,7 +11591,7 @@ check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb set NAT . options:foo=bar check_engine_stats northd norecompute compute check_engine_stats lr_nat norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_pb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11599,7 +11601,7 @@ check ovn-nbctl --wait=sb set NAT . external_ip=172.168.0.120 check_engine_stats northd norecompute compute check_engine_stats lr_nat norecompute compute check_engine_stats lr_stateful norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_pb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11609,7 +11611,7 @@ check ovn-nbctl --wait=sb set NAT . logical_ip=10.0.0.10 check_engine_stats northd norecompute compute check_engine_stats lr_nat norecompute compute check_engine_stats lr_stateful norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_pb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11619,7 +11621,7 @@ check ovn-nbctl --wait=sb set NAT . type=snat check_engine_stats northd norecompute compute check_engine_stats lr_nat norecompute compute check_engine_stats lr_stateful norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_pb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11629,7 +11631,7 @@ check ovn-nbctl --wait=sb lr-nat-add lr0 dnat_and_snat 172.168.0.110 10.0.0.4 sw check_engine_stats northd norecompute compute check_engine_stats lr_nat norecompute compute check_engine_stats lr_stateful norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_pb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11640,7 +11642,7 @@ check ovn-nbctl --wait=sb set NAT $nat2_uuid external_mac='"30:54:00:00:00:04"' check_engine_stats northd norecompute compute check_engine_stats lr_nat norecompute compute check_engine_stats lr_stateful norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_pb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11661,6 +11663,8 @@ check_engine_stats sync_to_sb_pb recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE +# lflow engine should recompute since the nat ip 172.168.0.150 +# is a lb vip. check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb lr-nat-add lr0 dnat_and_snat 172.168.0.150 10.0.0.41 check_engine_stats northd norecompute compute @@ -11670,6 +11674,8 @@ check_engine_stats sync_to_sb_pb recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE +# lflow engine should recompute since the deleted nat ip 172.168.0.150 +# is a lb vip. check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb lr-nat-del lr0 dnat_and_snat 172.168.0.150 check_engine_stats northd norecompute compute @@ -11679,6 +11685,8 @@ check_engine_stats sync_to_sb_pb recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE +# lflow engine should recompute since the deleted nat ip 172.168.0.140 +# is a lb vip. check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb lr-nat-del lr0 dnat_and_snat 172.168.0.140 check_engine_stats northd norecompute compute @@ -11694,7 +11702,7 @@ check ovn-nbctl --wait=sb clear logical_router lr0 nat check_engine_stats northd norecompute compute check_engine_stats lr_nat norecompute compute check_engine_stats lr_stateful norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_pb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE From patchwork Thu Jan 11 15:33:57 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Numan Siddique X-Patchwork-Id: 1885682 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::137; helo=smtp4.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp4.osuosl.org (smtp4.osuosl.org [IPv6:2605:bc80:3010::137]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4T9pcl1S4Bz1yPp for ; Fri, 12 Jan 2024 02:34:31 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 7DBE842CC5; Thu, 11 Jan 2024 15:34:28 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 7DBE842CC5 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id p1e2mYLSgS5w; Thu, 11 Jan 2024 15:34:25 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp4.osuosl.org (Postfix) with ESMTPS id 9BF5D42CBB; Thu, 11 Jan 2024 15:34:24 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 9BF5D42CBB Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 48E6CC007C; Thu, 11 Jan 2024 15:34:24 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by lists.linuxfoundation.org (Postfix) with ESMTP id 1CD91C007C for ; Thu, 11 Jan 2024 15:34:23 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 0D76583C9A for ; Thu, 11 Jan 2024 15:34:18 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 0D76583C9A X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id wPSBZhEF2szM for ; Thu, 11 Jan 2024 15:34:16 +0000 (UTC) Received: from relay3-d.mail.gandi.net (relay3-d.mail.gandi.net [217.70.183.195]) by smtp1.osuosl.org (Postfix) with ESMTPS id 60FB684025 for ; Thu, 11 Jan 2024 15:34:15 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 60FB684025 Received: by mail.gandi.net (Postfix) with ESMTPSA id F289B60002; Thu, 11 Jan 2024 15:34:12 +0000 (UTC) From: numans@ovn.org To: dev@openvswitch.org Date: Thu, 11 Jan 2024 10:33:57 -0500 Message-ID: <20240111153357.2790547-1-numans@ovn.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240111152752.2789854-1-numans@ovn.org> References: <20240111152752.2789854-1-numans@ovn.org> MIME-Version: 1.0 X-GND-Sasl: numans@ovn.org Subject: [ovs-dev] [PATCH ovn v5 13/16] northd: Add ls_stateful handler for lflow engine node. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Numan Siddique Signed-off-by: Numan Siddique --- northd/en-lflow.c | 26 +++++++++++++ northd/en-lflow.h | 1 + northd/en-ls-stateful.c | 9 +++-- northd/en-ls-stateful.h | 23 ++++++++++++ northd/inc-proc-northd.c | 2 +- northd/lflow-mgr.c | 2 +- northd/northd.c | 52 +++++++++++++++++++++++--- northd/northd.h | 5 +++ tests/ovn-northd.at | 81 ++++++++++++++++++++++++++-------------- 9 files changed, 162 insertions(+), 39 deletions(-) diff --git a/northd/en-lflow.c b/northd/en-lflow.c index 2dc4a83557..f51ee0d2b7 100644 --- a/northd/en-lflow.c +++ b/northd/en-lflow.c @@ -189,6 +189,32 @@ lflow_lr_stateful_handler(struct engine_node *node, void *data) return true; } +bool +lflow_ls_stateful_handler(struct engine_node *node, void *data) +{ + struct ed_type_ls_stateful *ls_sful_data = + engine_get_input_data("ls_stateful", node); + + if (!ls_stateful_has_tracked_data(&ls_sful_data->trk_data)) { + return false; + } + + const struct engine_context *eng_ctx = engine_get_context(); + struct lflow_data *lflow_data = data; + struct lflow_input lflow_input; + + lflow_get_input_data(node, &lflow_input); + if (!lflow_handle_ls_stateful_changes(eng_ctx->ovnsb_idl_txn, + &ls_sful_data->trk_data, + &lflow_input, + lflow_data->lflow_table)) { + return false; + } + + engine_set_node_state(node, EN_UPDATED); + return true; +} + void *en_lflow_init(struct engine_node *node OVS_UNUSED, struct engine_arg *arg OVS_UNUSED) { diff --git a/northd/en-lflow.h b/northd/en-lflow.h index 1d813a2a29..32cae61763 100644 --- a/northd/en-lflow.h +++ b/northd/en-lflow.h @@ -21,5 +21,6 @@ void en_lflow_cleanup(void *data); bool lflow_northd_handler(struct engine_node *, void *data); bool lflow_port_group_handler(struct engine_node *, void *data); bool lflow_lr_stateful_handler(struct engine_node *, void *data); +bool lflow_ls_stateful_handler(struct engine_node *node, void *data); #endif /* EN_LFLOW_H */ diff --git a/northd/en-ls-stateful.c b/northd/en-ls-stateful.c index 5fa305bc29..df76a69c58 100644 --- a/northd/en-ls-stateful.c +++ b/northd/en-ls-stateful.c @@ -39,6 +39,7 @@ #include "lib/ovn-sb-idl.h" #include "lib/ovn-util.h" #include "lib/stopwatch-names.h" +#include "lflow-mgr.h" #include "northd.h" VLOG_DEFINE_THIS_MODULE(en_ls_stateful); @@ -292,6 +293,7 @@ ls_stateful_record_create(struct ls_stateful_table *table, xzalloc(sizeof *ls_stateful_rec); ls_stateful_rec->od = od; ls_stateful_record_init(ls_stateful_rec, od, NULL, ls_pgs); + ls_stateful_rec->lflow_ref = lflow_ref_create(); hmap_insert(&table->entries, &ls_stateful_rec->key_node, uuid_hash(&ls_stateful_rec->od->nbs->header_.uuid)); @@ -302,14 +304,15 @@ ls_stateful_record_create(struct ls_stateful_table *table, static void ls_stateful_record_destroy(struct ls_stateful_record *ls_stateful_rec) { + lflow_ref_destroy(ls_stateful_rec->lflow_ref); free(ls_stateful_rec); } static void ls_stateful_record_init(struct ls_stateful_record *ls_stateful_rec, - const struct ovn_datapath *od, - const struct ls_port_group *ls_pg, - const struct ls_port_group_table *ls_pgs) + const struct ovn_datapath *od, + const struct ls_port_group *ls_pg, + const struct ls_port_group_table *ls_pgs) { ls_stateful_rec->has_lb_vip = ls_has_lb_vip(od); ls_stateful_record_set_acl_flags(ls_stateful_rec, od, ls_pg, ls_pgs); diff --git a/northd/en-ls-stateful.h b/northd/en-ls-stateful.h index cba53e1f29..bc1b3002ef 100644 --- a/northd/en-ls-stateful.h +++ b/northd/en-ls-stateful.h @@ -31,6 +31,8 @@ #include "lib/ovn-util.h" #include "lib/stopwatch-names.h" +struct lflow_ref; + struct ls_stateful_record { struct hmap_node key_node; @@ -39,6 +41,27 @@ struct ls_stateful_record { bool has_lb_vip; bool has_acls; uint64_t max_acl_tier; + + /* 'lflow_ref' is used to reference logical flows generated for + * this ls_stateful record. + * + * This data is initialized and destroyed by the en_ls_stateful node, + * but populated and used only by the en_lflow node. Ideally this data + * should be maintained as part of en_lflow's data. However, it would + * be less efficient and more complex: + * + * 1. It would require an extra search (using the index) to find the + * lflows. + * + * 2. Building the index needs to be thread-safe, using either a global + * lock which is obviously less efficient, or hash-based lock array which + * is more complex. + * + * Adding the lflow_ref here is more straightforward. The drawback is that + * we need to keep in mind that this data belongs to en_lflow node, so + * never access it from any other nodes. + */ + struct lflow_ref *lflow_ref; }; struct ls_stateful_table { diff --git a/northd/inc-proc-northd.c b/northd/inc-proc-northd.c index dcce79510a..f7c3d2bcf5 100644 --- a/northd/inc-proc-northd.c +++ b/northd/inc-proc-northd.c @@ -228,11 +228,11 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, engine_add_input(&en_lflow, &en_sb_logical_flow, NULL); engine_add_input(&en_lflow, &en_sb_multicast_group, NULL); engine_add_input(&en_lflow, &en_sb_igmp_group, NULL); - engine_add_input(&en_lflow, &en_ls_stateful, NULL); engine_add_input(&en_lflow, &en_sb_logical_dp_group, NULL); engine_add_input(&en_lflow, &en_northd, lflow_northd_handler); engine_add_input(&en_lflow, &en_port_group, lflow_port_group_handler); engine_add_input(&en_lflow, &en_lr_stateful, lflow_lr_stateful_handler); + engine_add_input(&en_lflow, &en_ls_stateful, lflow_ls_stateful_handler); engine_add_input(&en_sync_to_sb_addr_set, &en_nb_address_set, sync_to_sb_addr_set_nb_address_set_handler); diff --git a/northd/lflow-mgr.c b/northd/lflow-mgr.c index 6cb2a367fe..d81b13a25c 100644 --- a/northd/lflow-mgr.c +++ b/northd/lflow-mgr.c @@ -442,7 +442,7 @@ lflow_ref_unlink_lflows(struct lflow_ref *lflow_ref) BITMAP_FOR_EACH_1 (index, lrn->dpgrp_bitmap_len, lrn->dpgrp_bitmap) { if (dec_dp_refcnt(&lrn->lflow->dp_refcnts_map, index)) { - bitmap_set0(lrn->lflow->dpg_bitmap, lrn->dp_index); + bitmap_set0(lrn->lflow->dpg_bitmap, index); } } } else { diff --git a/northd/northd.c b/northd/northd.c index 5e49457bbe..96a5b52127 100644 --- a/northd/northd.c +++ b/northd/northd.c @@ -15684,12 +15684,14 @@ build_ls_stateful_flows(const struct ls_stateful_record *ls_stateful_rec, ovs_assert(ls_stateful_rec->od); build_ls_stateful_rec_pre_acls(ls_stateful_rec, ls_pgs, lflows, - NULL); - build_ls_stateful_rec_pre_lb(ls_stateful_rec, lflows, NULL); - build_acl_hints(ls_stateful_rec, features, lflows, NULL); + ls_stateful_rec->lflow_ref); + build_ls_stateful_rec_pre_lb(ls_stateful_rec, lflows, + ls_stateful_rec->lflow_ref); + build_acl_hints(ls_stateful_rec, features, lflows, + ls_stateful_rec->lflow_ref); build_acls(ls_stateful_rec, features, lflows, ls_pgs, meter_groups, - NULL); - build_lb_hairpin(ls_stateful_rec, lflows, NULL); + ls_stateful_rec->lflow_ref); + build_lb_hairpin(ls_stateful_rec, lflows, ls_stateful_rec->lflow_ref); } struct lswitch_flow_build_info { @@ -16346,6 +16348,7 @@ void lflow_reset_northd_refs(struct lflow_input *lflow_input) { const struct lr_stateful_record *lr_stateful_rec; + struct ls_stateful_record *ls_stateful_rec; struct ovn_lb_datapaths *lb_dps; struct ovn_port *op; @@ -16354,6 +16357,11 @@ lflow_reset_northd_refs(struct lflow_input *lflow_input) lflow_ref_clear(lr_stateful_rec->lflow_ref); } + LS_STATEFUL_TABLE_FOR_EACH (ls_stateful_rec, + lflow_input->ls_stateful_table) { + lflow_ref_clear(ls_stateful_rec->lflow_ref); + } + HMAP_FOR_EACH (op, key_node, lflow_input->ls_ports) { lflow_ref_clear(op->lflow_ref); lflow_ref_clear(op->stateful_lflow_ref); @@ -16685,6 +16693,40 @@ lflow_handle_lr_stateful_changes(struct ovsdb_idl_txn *ovnsb_txn, return true; } +bool +lflow_handle_ls_stateful_changes(struct ovsdb_idl_txn *ovnsb_txn, + struct ls_stateful_tracked_data *trk_data, + struct lflow_input *lflow_input, + struct lflow_table *lflows) +{ + struct hmapx_node *hmapx_node; + + HMAPX_FOR_EACH (hmapx_node, &trk_data->crupdated) { + struct ls_stateful_record *ls_stateful_rec = hmapx_node->data; + lflow_ref_unlink_lflows(ls_stateful_rec->lflow_ref); + + /* Generate new lflows. */ + build_ls_stateful_flows(ls_stateful_rec, lflow_input->ls_port_groups, + lflow_input->features, + lflow_input->meter_groups, + lflows); + + /* Sync the new flows to SB. */ + bool handled = lflow_ref_sync_lflows( + ls_stateful_rec->lflow_ref, lflows, ovnsb_txn, + lflow_input->ls_datapaths, + lflow_input->lr_datapaths, + lflow_input->ovn_internal_version_changed, + lflow_input->sbrec_logical_flow_table, + lflow_input->sbrec_logical_dp_group_table); + if (!handled) { + return false; + } + } + + return true; +} + static bool mirror_needs_update(const struct nbrec_mirror *nb_mirror, const struct sbrec_mirror *sb_mirror) diff --git a/northd/northd.h b/northd/northd.h index 95120d8fd8..297b1a65b9 100644 --- a/northd/northd.h +++ b/northd/northd.h @@ -667,6 +667,7 @@ void northd_indices_create(struct northd_data *data, struct lflow_table; struct lr_stateful_tracked_data; +struct ls_stateful_tracked_data; void build_lflows(struct ovsdb_idl_txn *ovnsb_txn, struct lflow_input *input_data, @@ -685,6 +686,10 @@ bool lflow_handle_lr_stateful_changes(struct ovsdb_idl_txn *, struct lr_stateful_tracked_data *, struct lflow_input *, struct lflow_table *lflows); +bool lflow_handle_ls_stateful_changes(struct ovsdb_idl_txn *, + struct ls_stateful_tracked_data *, + struct lflow_input *, + struct lflow_table *lflows); bool northd_handle_sb_port_binding_changes( const struct sbrec_port_binding_table *, struct hmap *ls_ports, struct hmap *lr_ports); diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at index 88dfc7bbfc..263eb343eb 100644 --- a/tests/ovn-northd.at +++ b/tests/ovn-northd.at @@ -10615,12 +10615,13 @@ check ovn-nbctl --wait=sb ls-lb-add sw0 lb1 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute -check_engine_stats lflow recompute nocompute # A LB applied to a switch/router triggers: # - a recompute in the first iteration (handling northd change) # - a compute in the second iteration (handling SB update) check_engine_stats sync_to_sb_lb recompute compute -CHECK_NO_CHANGE_AFTER_RECOMPUTE(1) +check_engine_stats ls_stateful norecompute compute +check_engine_stats lflow norecompute compute +CHECK_NO_CHANGE_AFTER_RECOMPUTE((1)) # Modify the backend of the lb1 vip check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats @@ -10628,7 +10629,8 @@ check ovn-nbctl --wait=sb set load_balancer lb1 vips:'"10.0.0.10:80"'='"10.0.0.1 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats ls_stateful norecompute compute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE(1) @@ -10638,7 +10640,8 @@ check ovn-nbctl --wait=sb clear load_Balancer lb1 vips check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats ls_stateful norecompute compute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE(1) @@ -10648,7 +10651,8 @@ check ovn-nbctl --wait=sb lb-add lb1 10.0.0.10:80 10.0.0.3:80 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats ls_stateful norecompute compute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE(1) @@ -10658,7 +10662,8 @@ check ovn-nbctl --wait=sb lb-add lb1 10.0.0.20:80 10.0.0.30:8080 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats ls_stateful norecompute compute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE(1) @@ -10668,6 +10673,7 @@ check ovn-nbctl --wait=sb ls-lb-del sw0 lb1 check_engine_stats lb_data norecompute compute check_engine_stats northd recompute nocompute check_engine_stats lr_stateful recompute nocompute +check_engine_stats ls_stateful recompute nocompute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE(1) @@ -10679,7 +10685,8 @@ check ovn-nbctl --wait=sb ls-lb-add sw0 lb1 -- lsp-add sw0 sw0p1 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats ls_stateful norecompute compute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10798,7 +10805,7 @@ check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute check_engine_stats ls_stateful norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute compute # Update lb and this should not result in northd recompute @@ -10806,8 +10813,9 @@ check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb set load_balancer . options:bar=foo check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_stateful norecompute compute check_engine_stats ls_stateful norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute compute # Modify the backend of the lb1 vip @@ -10817,7 +10825,7 @@ check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute check_engine_stats ls_stateful norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10827,7 +10835,8 @@ check ovn-nbctl --wait=sb clear load_Balancer lb1 vips check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats ls_stateful norecompute compute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10837,7 +10846,8 @@ check ovn-nbctl --wait=sb lb-add lb1 10.0.0.10:80 10.0.0.3:80 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats ls_stateful norecompute compute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10847,7 +10857,8 @@ check ovn-nbctl --wait=sb lb-add lb1 10.0.0.20:80 10.0.0.30:8080 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats ls_stateful norecompute compute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10922,7 +10933,8 @@ check ovn-nbctl --wait=sb add logical_switch sw0 load_balancer_group $lbg1_uuid check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats ls_stateful norecompute compute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10975,7 +10987,8 @@ check ovn-nbctl --wait=sb set logical_switch sw0 load_balancer_group=$lbg1_uuid check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats ls_stateful norecompute compute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10993,7 +11006,8 @@ check ovn-nbctl --wait=sb ls-lb-add sw0 lb2 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats ls_stateful norecompute compute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11002,7 +11016,8 @@ check ovn-nbctl --wait=sb ls-lb-add sw0 lb3 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats ls_stateful norecompute compute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11030,6 +11045,7 @@ check ovn-nbctl --wait=sb lr-lb-del lr1 lb2 check_engine_stats lb_data norecompute compute check_engine_stats northd recompute nocompute check_engine_stats lr_stateful recompute nocompute +check_engine_stats ls_stateful recompute nocompute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11041,7 +11057,8 @@ check ovn-nbctl --wait=sb lb-del lb4 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats ls_stateful norecompute compute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11052,7 +11069,8 @@ check ovn-nbctl --wait=sb lb-del lb2 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats ls_stateful norecompute compute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11262,7 +11280,7 @@ check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb ls-lb-add sw0 lb1 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Clear the VIPs of lb1 @@ -11270,7 +11288,7 @@ check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb clear load_balancer . vips check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats @@ -11294,7 +11312,7 @@ check ovn-nbctl --wait=sb lb-add lb2 10.0.0.10:80 10.0.0.3:80 check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb ls-lb-add sw0 lb1 -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE lb_lflow_uuid=$(fetch_column Logical_flow _uuid match='"ct.new && ip4.dst == 10.0.0.10 && tcp.dst == 80"') @@ -11308,7 +11326,7 @@ AT_CHECK([test "$lb_lflow_dpgrp" = ""]) check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb ls-lb-add sw1 lb2 -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE lb_lflow_dp=$(ovn-sbctl --bare --columns logical_datapath list logical_flow $lb_lflow_uuid) @@ -11317,6 +11335,11 @@ AT_CHECK([test "$lb_lflow_dp" = ""]) lb_lflow_dpgrp=$(ovn-sbctl --bare --columns logical_dp_group list logical_flow $lb_lflow_uuid) AT_CHECK([test "$lb_lflow_dpgrp" != ""]) +check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats +check ovn-nbctl --wait=sb set load_balancer lb1 options:bar=foo +check_engine_stats lflow norecompute compute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + # Clear the SB:Logical_Flow.logical_dp_groups column of all the # logical flows and then modify the NB:Load_balancer. ovn-northd # should resync the logical flows. @@ -11334,7 +11357,7 @@ lb_lflow_uuid=$(fetch_column Logical_flow _uuid match='"ct.new && ip4.dst == 10. check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb clear load_balancer lb2 vips -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE lb_lflow_dp=$(ovn-sbctl --bare --columns logical_datapath list logical_flow $lb_lflow_uuid) @@ -11360,7 +11383,7 @@ check ovn-nbctl ls-lb-add sw4 lb2 check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb ls-lb-add sw5 lb2 -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE lb_lflow_dp=$(ovn-sbctl --bare --columns logical_datapath list logical_flow $lb_lflow_uuid) @@ -11391,7 +11414,7 @@ echo "dpgrp_dps - $dpgrp_dps" check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb clear load_balancer lb2 vips -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE lb_lflow_dpgrp=$(ovn-sbctl --bare --columns logical_dp_group list logical_flow $lb_lflow_uuid) @@ -11425,7 +11448,7 @@ check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl ls-lb-add sw0 lb3 check ovn-nbctl --wait=sb ls-lb-add sw1 lb3 -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE lb_lflow_uuid=$(fetch_column Logical_flow _uuid match='"ct.new && ip4.dst == 10.0.0.10 && tcp.dst == 80"') @@ -11450,7 +11473,7 @@ AT_CHECK([echo $dpgrp_dps | grep $sw5_uuid], [0], [ignore]) # should have reference to sw0 and sw1, but not to sw2. check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb clear load_balancer lb1 vips -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE lb_lflow_dp=$(ovn-sbctl --bare --columns logical_datapath list logical_flow $lb_lflow_uuid) @@ -11476,7 +11499,7 @@ AT_CHECK([echo $dpgrp_dps | grep $sw5_uuid], [0], [ignore]) check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb clear load_balancer lb3 vips -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE lb_lflow_dp=$(ovn-sbctl --bare --columns logical_datapath list logical_flow $lb_lflow_uuid) From patchwork Thu Jan 11 15:34:14 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Numan Siddique X-Patchwork-Id: 1885683 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::136; helo=smtp3.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4T9pcw1xkWz1yPp for ; Fri, 12 Jan 2024 02:34:40 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 07A2C6F5DA; Thu, 11 Jan 2024 15:34:38 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 07A2C6F5DA X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id VxuHaFtPBYUe; Thu, 11 Jan 2024 15:34:36 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp3.osuosl.org (Postfix) with ESMTPS id 33D3F6F5B9; Thu, 11 Jan 2024 15:34:35 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 33D3F6F5B9 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id F3092C0077; Thu, 11 Jan 2024 15:34:34 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by lists.linuxfoundation.org (Postfix) with ESMTP id 122CEC0077 for ; Thu, 11 Jan 2024 15:34:34 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id A465683CE9 for ; Thu, 11 Jan 2024 15:34:30 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org A465683CE9 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ZOCMwpKAMIPB for ; Thu, 11 Jan 2024 15:34:30 +0000 (UTC) Received: from relay8-d.mail.gandi.net (relay8-d.mail.gandi.net [217.70.183.201]) by smtp1.osuosl.org (Postfix) with ESMTPS id ADD6084397 for ; Thu, 11 Jan 2024 15:34:29 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org ADD6084397 Received: by mail.gandi.net (Postfix) with ESMTPSA id 6833E1BF213; Thu, 11 Jan 2024 15:34:27 +0000 (UTC) From: numans@ovn.org To: dev@openvswitch.org Date: Thu, 11 Jan 2024 10:34:14 -0500 Message-ID: <20240111153414.2790567-1-numans@ovn.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240111152752.2789854-1-numans@ovn.org> References: <20240111152752.2789854-1-numans@ovn.org> MIME-Version: 1.0 X-GND-Sasl: numans@ovn.org Subject: [ovs-dev] [PATCH ovn v5 14/16] northd: Add a noop handler for northd SB mac binding. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Numan Siddique northd engine node uses the sb mac binding table to cleanup mac binding entries for deleted logical ports and datapaths. Any update to SB mac binding doesn't change the northd engine node state or data. Hence it is ok to add a noop_handler. Presently, mac_binding_aging node depends on SB mac binding too and it falls back to full recompute for any SB mac binding changes. It needs to be evaluated if mac_binding_aging really needs to handle SB mac binding updates. If not, we can omit the SB mac binding updates (ovsdb_idl_omit_alert()) and also remove the noop_handler this patch adds for northd node. Signed-off-by: Numan Siddique Acked-by: Dumitru Ceara --- northd/inc-proc-northd.c | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/northd/inc-proc-northd.c b/northd/inc-proc-northd.c index f7c3d2bcf5..40a9e5e06c 100644 --- a/northd/inc-proc-northd.c +++ b/northd/inc-proc-northd.c @@ -177,7 +177,6 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, engine_add_input(&en_northd, &en_sb_mirror, NULL); engine_add_input(&en_northd, &en_sb_meter, NULL); engine_add_input(&en_northd, &en_sb_datapath_binding, NULL); - engine_add_input(&en_northd, &en_sb_mac_binding, NULL); engine_add_input(&en_northd, &en_sb_dns, NULL); engine_add_input(&en_northd, &en_sb_ha_chassis_group, NULL); engine_add_input(&en_northd, &en_sb_ip_multicast, NULL); @@ -186,6 +185,17 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, engine_add_input(&en_northd, &en_sb_static_mac_binding, NULL); engine_add_input(&en_northd, &en_sb_chassis_template_var, NULL); + /* northd engine node uses the sb mac binding table to + * cleanup mac binding entries for deleted logical ports + * and datapaths. Any update to SB mac binding doesn't + * change the northd engine node state or data. Hence + * it is ok to add a noop_handler here. + * Note: mac_binding_aging engine node depends on SB mac binding + * and it results in full recompute for any changes to it. + * */ + engine_add_input(&en_northd, &en_sb_mac_binding, + engine_noop_handler); + engine_add_input(&en_northd, &en_sb_port_binding, northd_sb_port_binding_handler); engine_add_input(&en_northd, &en_nb_nb_global, From patchwork Thu Jan 11 15:34:28 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Numan Siddique X-Patchwork-Id: 1885684 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::133; helo=smtp2.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp2.osuosl.org (smtp2.osuosl.org [IPv6:2605:bc80:3010::133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4T9pdm4dydz1yPm for ; Fri, 12 Jan 2024 02:35:24 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 6AE0D43911; Thu, 11 Jan 2024 15:35:22 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 6AE0D43911 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id QMnhZZLGFhPo; Thu, 11 Jan 2024 15:35:19 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp2.osuosl.org (Postfix) with ESMTPS id 2E7DB438C5; Thu, 11 Jan 2024 15:35:18 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 2E7DB438C5 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 05B4EC0077; Thu, 11 Jan 2024 15:35:18 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by lists.linuxfoundation.org (Postfix) with ESMTP id 1EB8EC0077 for ; Thu, 11 Jan 2024 15:35:16 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 9376A83CC3 for ; Thu, 11 Jan 2024 15:34:53 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 9376A83CC3 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 8a7Nv6QRTu_s for ; Thu, 11 Jan 2024 15:34:50 +0000 (UTC) Received: from relay4-d.mail.gandi.net (relay4-d.mail.gandi.net [IPv6:2001:4b98:dc4:8::224]) by smtp1.osuosl.org (Postfix) with ESMTPS id 3B3BE843BA for ; Thu, 11 Jan 2024 15:34:50 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 3B3BE843BA Received: by mail.gandi.net (Postfix) with ESMTPSA id 5E1C6E000B; Thu, 11 Jan 2024 15:34:46 +0000 (UTC) From: numans@ovn.org To: dev@openvswitch.org Date: Thu, 11 Jan 2024 10:34:28 -0500 Message-ID: <20240111153428.2790587-1-numans@ovn.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240111152752.2789854-1-numans@ovn.org> References: <20240111152752.2789854-1-numans@ovn.org> MIME-Version: 1.0 X-GND-Sasl: numans@ovn.org Subject: [ovs-dev] [PATCH ovn v5 15/16] northd: Add northd change handler for sync_to_sb_lb node. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Numan Siddique Any changes to northd engine node due to load balancers are now handled in 'sync_to_sb_lb' node to sync the changed load balancers to SB load balancers. The logic to sync the SB load balancers is changed a bit and it now mimics the SB lflow sync. Below are the scale testing results done with all the patches applied in this series using ovn-heater. The test ran the scenario - ocp-500-density-heavy.yml [1]. The resuts are: ------------------------------------------------------------------------------------------------------------------------------------------------------- Min (s) Median (s) 90%ile (s) 99%ile (s) Max (s) Mean (s) Total (s) Count Failed ------------------------------------------------------------------------------------------------------------------------------------------------------- Iteration Total 0.136883 1.129016 1.192001 1.204167 1.212728 0.665017 83.127099 125 0 Namespace.add_ports 0.005216 0.005736 0.007034 0.015486 0.018978 0.006211 0.776373 125 0 WorkerNode.bind_port 0.035030 0.046082 0.052469 0.058293 0.060311 0.045973 11.493259 250 0 WorkerNode.ping_port 0.005057 0.006727 1.047692 1.069253 1.071336 0.266896 66.724094 250 0 ------------------------------------------------------------------------------------------------------------------------------------------------------- The results with the present main [2] are: ------------------------------------------------------------------------------------------------------------------------------------------------------- Min (s) Median (s) 90%ile (s) 99%ile (s) Max (s) Mean (s) Total (s) Count Failed ------------------------------------------------------------------------------------------------------------------------------------------------------- Iteration Total 0.135491 2.223805 3.311270 3.339078 3.345346 1.729172 216.146495 125 0 Namespace.add_ports 0.005380 0.005744 0.006819 0.018773 0.020800 0.006292 0.786532 125 0 WorkerNode.bind_port 0.034179 0.046055 0.053488 0.058801 0.071043 0.046117 11.529311 250 0 WorkerNode.ping_port 0.004956 0.006952 3.086952 3.191743 3.192807 0.791544 197.886026 250 0 ------------------------------------------------------------------------------------------------------------------------------------------------------- [1] - https://github.com/ovn-org/ovn-heater/blob/main/test-scenarios/ocp-500-density-heavy.yml [2] - 2a12cda890a7("controller, northd: Wait for cleanup before replying to exit") Signed-off-by: Numan Siddique Acked-by: Dumitru Ceara --- northd/en-sync-sb.c | 505 +++++++++++++++++++++++++++++++++++++-- northd/inc-proc-northd.c | 1 + northd/lflow-mgr.c | 196 ++++++--------- northd/lflow-mgr.h | 23 +- northd/northd.c | 238 ------------------ northd/northd.h | 6 - tests/ovn-northd.at | 164 ++++++++++--- 7 files changed, 705 insertions(+), 428 deletions(-) diff --git a/northd/en-sync-sb.c b/northd/en-sync-sb.c index d39cbbf2e6..80f3621bb9 100644 --- a/northd/en-sync-sb.c +++ b/northd/en-sync-sb.c @@ -18,17 +18,21 @@ #include #include +/* OVS includes. */ #include "lib/svec.h" #include "openvswitch/util.h" +/* OVN includes. */ #include "en-lr-nat.h" #include "en-lr-stateful.h" #include "en-sync-sb.h" +#include "lb.h" #include "lib/inc-proc-eng.h" #include "lib/lb.h" #include "lib/ovn-nb-idl.h" #include "lib/ovn-sb-idl.h" #include "lib/ovn-util.h" +#include "lflow-mgr.h" #include "northd.h" #include "openvswitch/vlog.h" @@ -51,6 +55,40 @@ static void build_port_group_address_set(const struct nbrec_port_group *, struct svec *ipv4_addrs, struct svec *ipv6_addrs); +struct sb_lb_table; +struct sb_lb_record; +static void sb_lb_table_init(struct sb_lb_table *); +static void sb_lb_table_clear(struct sb_lb_table *); +static void sb_lb_table_destroy(struct sb_lb_table *); + +static struct sb_lb_record *sb_lb_table_find(struct hmap *sb_lbs, + const struct uuid *); +static void sb_lb_table_build_and_sync(struct sb_lb_table *, + struct ovsdb_idl_txn *ovnsb_txn, + const struct sbrec_load_balancer_table *, + const struct sbrec_logical_dp_group_table *, + struct hmap *lb_dps_map, + struct ovn_datapaths *ls_datapaths, + struct ovn_datapaths *lr_datapaths, + struct chassis_features *); +static bool sync_sb_lb_record(struct sb_lb_record *, + const struct sbrec_load_balancer *, + const struct sbrec_logical_dp_group_table *, + struct sb_lb_table *, + struct ovsdb_idl_txn *ovnsb_txn, + struct ovn_datapaths *ls_datapaths, + struct ovn_datapaths *lr_datapaths, + struct chassis_features *); +static bool sync_changed_lbs(struct sb_lb_table *, + struct ovsdb_idl_txn *ovnsb_txn, + const struct sbrec_load_balancer_table *, + const struct sbrec_logical_dp_group_table *, + struct tracked_lbs *, + struct ovn_datapaths *ls_datapaths, + struct ovn_datapaths *lr_datapaths, + struct chassis_features *); +static bool check_sb_lb_duplicates(const struct sbrec_load_balancer_table *); + void * en_sync_to_sb_init(struct engine_node *node OVS_UNUSED, struct engine_arg *arg OVS_UNUSED) @@ -206,49 +244,99 @@ sync_to_sb_addr_set_nb_port_group_handler(struct engine_node *node, /* sync_to_sb_lb engine node functions. * This engine node syncs the SB load balancers. */ +struct sb_lb_record { + struct hmap_node key_node; /* Index on 'nblb->header_.uuid'. */ + + struct ovn_lb_datapaths *lb_dps; + const struct sbrec_load_balancer *sbrec_lb; + struct ovn_dp_group *ls_dpg; + struct ovn_dp_group *lr_dpg; + struct uuid sb_uuid; +}; + +struct sb_lb_table { + struct hmap entries; /* Stores struct sb_lb_record. */ + struct hmap ls_dp_groups; + struct hmap lr_dp_groups; +}; + +struct ed_type_sync_to_sb_lb_data { + struct sb_lb_table sb_lbs; +}; + void * en_sync_to_sb_lb_init(struct engine_node *node OVS_UNUSED, struct engine_arg *arg OVS_UNUSED) { - return NULL; + struct ed_type_sync_to_sb_lb_data *data = xzalloc(sizeof *data); + sb_lb_table_init(&data->sb_lbs); + + return data; } void -en_sync_to_sb_lb_run(struct engine_node *node, void *data OVS_UNUSED) +en_sync_to_sb_lb_run(struct engine_node *node, void *data_) { + struct northd_data *northd_data = engine_get_input_data("northd", node); const struct sbrec_load_balancer_table *sb_load_balancer_table = EN_OVSDB_GET(engine_get_input("SB_load_balancer", node)); + const struct sbrec_logical_dp_group_table *sb_dpgrp_table = + EN_OVSDB_GET(engine_get_input("SB_logical_dp_group", node)); const struct engine_context *eng_ctx = engine_get_context(); - struct northd_data *northd_data = engine_get_input_data("northd", node); + struct ed_type_sync_to_sb_lb_data *data = data_; + + sb_lb_table_clear(&data->sb_lbs); + sb_lb_table_build_and_sync(&data->sb_lbs, eng_ctx->ovnsb_idl_txn, + sb_load_balancer_table, + sb_dpgrp_table, + &northd_data->lb_datapaths_map, + &northd_data->ls_datapaths, + &northd_data->lr_datapaths, + &northd_data->features); - sync_lbs(eng_ctx->ovnsb_idl_txn, sb_load_balancer_table, - &northd_data->ls_datapaths, &northd_data->lr_datapaths, - &northd_data->lb_datapaths_map, &northd_data->features); engine_set_node_state(node, EN_UPDATED); } void -en_sync_to_sb_lb_cleanup(void *data OVS_UNUSED) +en_sync_to_sb_lb_cleanup(void *data_) { - + struct ed_type_sync_to_sb_lb_data *data = data_; + sb_lb_table_destroy(&data->sb_lbs); } bool -sync_to_sb_lb_northd_handler(struct engine_node *node, void *data OVS_UNUSED) +sync_to_sb_lb_northd_handler(struct engine_node *node, void *data_) { struct northd_data *nd = engine_get_input_data("northd", node); - if (!northd_has_tracked_data(&nd->trk_data) || - northd_has_lbs_in_tracked_data(&nd->trk_data)) { - /* Return false if no tracking data or if lbs changed. */ + if (!northd_has_tracked_data(&nd->trk_data)) { + /* Return false if no tracking data. */ return false; } + if (!northd_has_lbs_in_tracked_data(&nd->trk_data)) { + return true; + } - /* There are only NB LSP related changes and these can be safely - * ignore and returned true. However in case the northd engine - * tracking data includes other changes, we need to do additional - * checks before safely ignoring. */ + const struct engine_context *eng_ctx = engine_get_context(); + if (!eng_ctx->ovnsb_idl_txn) { + return false; + } + + const struct sbrec_logical_dp_group_table *sb_dpgrp_table = + EN_OVSDB_GET(engine_get_input("SB_logical_dp_group", node)); + const struct sbrec_load_balancer_table *sb_lb_table = + EN_OVSDB_GET(engine_get_input("SB_load_balancer", node)); + struct ed_type_sync_to_sb_lb_data *data = data_; + + if (!sync_changed_lbs(&data->sb_lbs, eng_ctx->ovnsb_idl_txn, sb_lb_table, + sb_dpgrp_table, &nd->trk_data.trk_lbs, + &nd->ls_datapaths, &nd->lr_datapaths, + &nd->features)) { + return false; + } + + engine_set_node_state(node, EN_UPDATED); return true; } @@ -519,3 +607,388 @@ sb_address_set_lookup_by_name(struct ovsdb_idl_index *sbrec_addr_set_by_name, return retval; } + +/* static functions related to sync_to_sb_lb */ + +static void +sb_lb_table_init(struct sb_lb_table *sb_lbs) +{ + hmap_init(&sb_lbs->entries); + ovn_dp_groups_init(&sb_lbs->ls_dp_groups); + ovn_dp_groups_init(&sb_lbs->lr_dp_groups); +} + +static void +sb_lb_table_clear(struct sb_lb_table *sb_lbs) +{ + struct sb_lb_record *sb_lb; + HMAP_FOR_EACH_POP (sb_lb, key_node, &sb_lbs->entries) { + free(sb_lb); + } + + ovn_dp_groups_clear(&sb_lbs->ls_dp_groups); + ovn_dp_groups_clear(&sb_lbs->lr_dp_groups); +} + +static void +sb_lb_table_destroy(struct sb_lb_table *sb_lbs) +{ + sb_lb_table_clear(sb_lbs); + hmap_destroy(&sb_lbs->entries); + ovn_dp_groups_destroy(&sb_lbs->ls_dp_groups); + ovn_dp_groups_destroy(&sb_lbs->lr_dp_groups); +} + +static struct sb_lb_record * +sb_lb_table_find(struct hmap *sb_lbs, const struct uuid *lb_uuid) +{ + struct sb_lb_record *sb_lb; + HMAP_FOR_EACH_WITH_HASH (sb_lb, key_node, uuid_hash(lb_uuid), + sb_lbs) { + if (uuid_equals(&sb_lb->lb_dps->lb->nlb->header_.uuid, lb_uuid)) { + return sb_lb; + } + } + + return NULL; +} + +static void +sb_lb_table_build_and_sync(struct sb_lb_table *sb_lbs, + struct ovsdb_idl_txn *ovnsb_txn, + const struct sbrec_load_balancer_table *sb_lb_table, + const struct sbrec_logical_dp_group_table *sb_dpgrp_table, + struct hmap *lb_dps_map, + struct ovn_datapaths *ls_datapaths, + struct ovn_datapaths *lr_datapaths, + struct chassis_features *chassis_features) +{ + struct hmap tmp_sb_lbs = HMAP_INITIALIZER(&tmp_sb_lbs); + struct ovn_lb_datapaths *lb_dps; + struct sb_lb_record *sb_lb; + + HMAP_FOR_EACH (lb_dps, hmap_node, lb_dps_map) { + if (!lb_dps->n_nb_ls && !lb_dps->n_nb_lr) { + continue; + } + + sb_lb = xzalloc(sizeof *sb_lb); + sb_lb->lb_dps = lb_dps; + hmap_insert(&tmp_sb_lbs, &sb_lb->key_node, + uuid_hash(&lb_dps->lb->nlb->header_.uuid)); + } + + const struct sbrec_load_balancer *sbrec_lb; + SBREC_LOAD_BALANCER_TABLE_FOR_EACH_SAFE (sbrec_lb, + sb_lb_table) { + const char *nb_lb_uuid = smap_get(&sbrec_lb->external_ids, "lb_id"); + struct uuid lb_uuid; + if (!nb_lb_uuid || !uuid_from_string(&lb_uuid, nb_lb_uuid)) { + sbrec_load_balancer_delete(sbrec_lb); + continue; + } + + sb_lb = sb_lb_table_find(&tmp_sb_lbs, &lb_uuid); + if (sb_lb) { + sb_lb->sbrec_lb = sbrec_lb; + bool success = sync_sb_lb_record(sb_lb, sbrec_lb, sb_dpgrp_table, + sb_lbs, ovnsb_txn, ls_datapaths, + lr_datapaths, chassis_features); + /* Since we are rebuilding and syncing, sync_sb_lb_record should + * not return false. */ + ovs_assert(success); + + hmap_remove(&tmp_sb_lbs, &sb_lb->key_node); + hmap_insert(&sb_lbs->entries, &sb_lb->key_node, + uuid_hash(&sb_lb->lb_dps->lb->nlb->header_.uuid)); + } else { + sbrec_load_balancer_delete(sbrec_lb); + } + } + + HMAP_FOR_EACH_POP (sb_lb, key_node, &tmp_sb_lbs) { + bool success = sync_sb_lb_record(sb_lb, NULL, sb_dpgrp_table, sb_lbs, + ovnsb_txn, ls_datapaths, lr_datapaths, + chassis_features); + /* Since we are rebuilding and syncing, sync_sb_lb_record should not + * return false. */ + ovs_assert(success); + + hmap_insert(&sb_lbs->entries, &sb_lb->key_node, + uuid_hash(&sb_lb->lb_dps->lb->nlb->header_.uuid)); + } + + hmap_destroy(&tmp_sb_lbs); +} + +static bool +sync_sb_lb_record(struct sb_lb_record *sb_lb, + const struct sbrec_load_balancer *sbrec_lb, + const struct sbrec_logical_dp_group_table *sb_dpgrp_table, + struct sb_lb_table *sb_lbs, + struct ovsdb_idl_txn *ovnsb_txn, + struct ovn_datapaths *ls_datapaths, + struct ovn_datapaths *lr_datapaths, + struct chassis_features *chassis_features) +{ + struct sbrec_logical_dp_group *sbrec_ls_dp_group = NULL; + struct sbrec_logical_dp_group *sbrec_lr_dp_group = NULL; + const struct ovn_lb_datapaths *lb_dps; + struct ovn_dp_group *pre_sync_ls_dpg; + struct ovn_dp_group *pre_sync_lr_dpg; + + lb_dps = sb_lb->lb_dps; + pre_sync_ls_dpg = sb_lb->ls_dpg; + pre_sync_lr_dpg = sb_lb->lr_dpg; + + if (!sbrec_lb) { + sb_lb->sb_uuid = uuid_random(); + sbrec_lb = sbrec_load_balancer_insert_persist_uuid(ovnsb_txn, + &sb_lb->sb_uuid); + char *lb_id = xasprintf( + UUID_FMT, UUID_ARGS(&lb_dps->lb->nlb->header_.uuid)); + const struct smap external_ids = + SMAP_CONST1(&external_ids, "lb_id", lb_id); + sbrec_load_balancer_set_external_ids(sbrec_lb, &external_ids); + free(lb_id); + } else { + sb_lb->sb_uuid = sbrec_lb->header_.uuid; + sbrec_ls_dp_group = + chassis_features->ls_dpg_column + ? sbrec_lb->ls_datapath_group + : sbrec_lb->datapath_group; /* deprecated */ + + sbrec_lr_dp_group = sbrec_lb->lr_datapath_group; + } + + if (lb_dps->n_nb_ls) { + sb_lb->ls_dpg = ovn_dp_group_get(&sb_lbs->ls_dp_groups, + lb_dps->n_nb_ls, + lb_dps->nb_ls_map, + ods_size(ls_datapaths)); + if (sb_lb->ls_dpg) { + /* Update the dpg's sb dp_group. */ + sb_lb->ls_dpg->dp_group = + sbrec_logical_dp_group_table_get_for_uuid(sb_dpgrp_table, + &sb_lb->ls_dpg->dpg_uuid); + if (!sb_lb->ls_dpg->dp_group) { + /* Ideally this should not happen. But it can still happen + * due to 2 reasons: + * 1. There is a bug in the dp_group management. We should + * perhaps assert here. + * 2. A User or CMS may delete the logical_dp_groups in SB DB + * or clear the SB:Load_balancer.ls_datapath_group column + * (intentionally or accidentally) + * + * Because of (2) it is better to return false instead of + * assert,so that we recover from th inconsistent SB DB. + */ + static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 1); + VLOG_WARN_RL(&rl, "SB Load balancer [%s]'s ls_dp_group column " + "is not set (which is unexpected). It should " + "have been referencing the dp group ["UUID_FMT"]", + sb_lb->lb_dps->lb->nlb->name, + UUID_ARGS(&sb_lb->ls_dpg->dpg_uuid)); + return false; + } + } else { + sb_lb->ls_dpg = ovn_dp_group_create( + ovnsb_txn, &sb_lbs->ls_dp_groups, sbrec_ls_dp_group, + lb_dps->n_nb_ls, lb_dps->nb_ls_map, + ods_size(ls_datapaths), true, + ls_datapaths, lr_datapaths); + } + + if (chassis_features->ls_dpg_column) { + sbrec_load_balancer_set_ls_datapath_group(sbrec_lb, + sb_lb->ls_dpg->dp_group); + sbrec_load_balancer_set_datapath_group(sbrec_lb, NULL); + } else { + /* datapath_group column is deprecated. */ + sbrec_load_balancer_set_ls_datapath_group(sbrec_lb, NULL); + sbrec_load_balancer_set_datapath_group(sbrec_lb, + sb_lb->ls_dpg->dp_group); + + } + } else { + sbrec_load_balancer_set_ls_datapath_group(sbrec_lb, NULL); + sbrec_load_balancer_set_datapath_group(sbrec_lb, NULL); + } + + + if (lb_dps->n_nb_lr) { + sb_lb->lr_dpg = ovn_dp_group_get(&sb_lbs->lr_dp_groups, + lb_dps->n_nb_lr, + lb_dps->nb_lr_map, + ods_size(lr_datapaths)); + if (sb_lb->lr_dpg) { + /* Update the dpg's sb dp_group. */ + sb_lb->lr_dpg->dp_group = + sbrec_logical_dp_group_table_get_for_uuid(sb_dpgrp_table, + &sb_lb->lr_dpg->dpg_uuid); + if (!sb_lb->lr_dpg->dp_group) { + /* Ideally this should not happen. But it can still happen + * due to 2 reasons: + * 1. There is a bug in the dp_group management. We should + * perhaps assert here. + * 2. A User or CMS may delete the logical_dp_groups in SB DB + * or clear the SB:Load_balancer.lr_datapath_group column + * (intentionally or accidentally) + * + * Because of (2) it is better to return false instead of + * assert,so that we recover from th inconsistent SB DB. + */ + static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 1); + VLOG_WARN_RL(&rl, "SB Load balancer [%s]'s lr_dp_group column " + "is not set (which is unexpected). It should " + "have been referencing the dp group ["UUID_FMT"]", + sb_lb->lb_dps->lb->nlb->name, + UUID_ARGS(&sb_lb->lr_dpg->dpg_uuid)); + return false; + } + } else { + sb_lb->lr_dpg = ovn_dp_group_create( + ovnsb_txn, &sb_lbs->lr_dp_groups, sbrec_lr_dp_group, + lb_dps->n_nb_lr, lb_dps->nb_lr_map, + ods_size(lr_datapaths), false, + ls_datapaths, lr_datapaths); + } + + sbrec_load_balancer_set_lr_datapath_group(sbrec_lb, + sb_lb->lr_dpg->dp_group); + } else { + sbrec_load_balancer_set_lr_datapath_group(sbrec_lb, NULL); + } + + if (pre_sync_ls_dpg != sb_lb->ls_dpg) { + if (sb_lb->ls_dpg) { + inc_ovn_dp_group_ref(sb_lb->ls_dpg); + } + if (pre_sync_ls_dpg) { + dec_ovn_dp_group_ref(&sb_lbs->ls_dp_groups, pre_sync_ls_dpg); + } + } + + if (pre_sync_lr_dpg != sb_lb->lr_dpg) { + if (sb_lb->lr_dpg) { + inc_ovn_dp_group_ref(sb_lb->lr_dpg); + } + if (pre_sync_lr_dpg) { + dec_ovn_dp_group_ref(&sb_lbs->lr_dp_groups, pre_sync_lr_dpg); + } + } + + /* Update columns. */ + sbrec_load_balancer_set_name(sbrec_lb, lb_dps->lb->nlb->name); + sbrec_load_balancer_set_vips(sbrec_lb, + ovn_northd_lb_get_vips(lb_dps->lb)); + sbrec_load_balancer_set_protocol(sbrec_lb, lb_dps->lb->nlb->protocol); + + /* Store the fact that northd provides the original (destination IP + + * transport port) tuple. + */ + struct smap options; + smap_clone(&options, &lb_dps->lb->nlb->options); + smap_replace(&options, "hairpin_orig_tuple", "true"); + sbrec_load_balancer_set_options(sbrec_lb, &options); + /* Clearing 'datapaths' column, since 'dp_group' is in use. */ + sbrec_load_balancer_set_datapaths(sbrec_lb, NULL, 0); + smap_destroy(&options); + + return true; +} + +static bool +sync_changed_lbs(struct sb_lb_table *sb_lbs, + struct ovsdb_idl_txn *ovnsb_txn, + const struct sbrec_load_balancer_table *sb_lb_table, + const struct sbrec_logical_dp_group_table *sb_dpgrp_table, + struct tracked_lbs *trk_lbs, + struct ovn_datapaths *ls_datapaths, + struct ovn_datapaths *lr_datapaths, + struct chassis_features *chassis_features) +{ + struct ovn_lb_datapaths *lb_dps; + struct hmapx_node *hmapx_node; + struct sb_lb_record *sb_lb; + + HMAPX_FOR_EACH (hmapx_node, &trk_lbs->deleted) { + lb_dps = hmapx_node->data; + + sb_lb = sb_lb_table_find(&sb_lbs->entries, + &lb_dps->lb->nlb->header_.uuid); + if (sb_lb) { + const struct sbrec_load_balancer *sbrec_lb = + sbrec_load_balancer_table_get_for_uuid(sb_lb_table, + &sb_lb->sb_uuid); + if (sbrec_lb) { + sbrec_load_balancer_delete(sbrec_lb); + } + + hmap_remove(&sb_lbs->entries, &sb_lb->key_node); + free(sb_lb); + } + } + + HMAPX_FOR_EACH (hmapx_node, &trk_lbs->crupdated) { + lb_dps = hmapx_node->data; + + sb_lb = sb_lb_table_find(&sb_lbs->entries, + &lb_dps->lb->nlb->header_.uuid); + + if (!sb_lb && !lb_dps->n_nb_ls && !lb_dps->n_nb_lr) { + continue; + } + + if (!sb_lb) { + sb_lb = xzalloc(sizeof *sb_lb); + sb_lb->lb_dps = lb_dps; + hmap_insert(&sb_lbs->entries, &sb_lb->key_node, + uuid_hash(&lb_dps->lb->nlb->header_.uuid)); + } else { + sb_lb->sbrec_lb = + sbrec_load_balancer_table_get_for_uuid(sb_lb_table, + &sb_lb->sb_uuid); + } + + if (sb_lb && !lb_dps->n_nb_ls && !lb_dps->n_nb_lr) { + const struct sbrec_load_balancer *sbrec_lb = + sbrec_load_balancer_table_get_for_uuid(sb_lb_table, + &sb_lb->sb_uuid); + if (sbrec_lb) { + sbrec_load_balancer_delete(sbrec_lb); + } + + hmap_remove(&sb_lbs->entries, &sb_lb->key_node); + free(sb_lb); + } + + if (!sync_sb_lb_record(sb_lb, sb_lb->sbrec_lb, sb_dpgrp_table, sb_lbs, + ovnsb_txn, ls_datapaths, lr_datapaths, + chassis_features)) { + return false; + } + } + + return true; +} + +static bool +check_sb_lb_duplicates(const struct sbrec_load_balancer_table *table) +{ + struct sset existing_nb_lb_uuids = + SSET_INITIALIZER(&existing_nb_lb_uuids); + const struct sbrec_load_balancer *sbrec_lb; + bool duplicates = false; + + SBREC_LOAD_BALANCER_TABLE_FOR_EACH (sbrec_lb, table) { + const char *nb_lb_uuid = smap_get(&sbrec_lb->external_ids, "lb_id"); + if (nb_lb_uuid && !sset_add(&existing_nb_lb_uuids, nb_lb_uuid)) { + duplicates = true; + break; + } + } + + sset_destroy(&existing_nb_lb_uuids); + return duplicates; +} diff --git a/northd/inc-proc-northd.c b/northd/inc-proc-northd.c index 40a9e5e06c..d215c7792b 100644 --- a/northd/inc-proc-northd.c +++ b/northd/inc-proc-northd.c @@ -264,6 +264,7 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, sync_to_sb_lb_northd_handler); engine_add_input(&en_sync_to_sb_lb, &en_sb_load_balancer, sync_to_sb_lb_sb_load_balancer); + engine_add_input(&en_sync_to_sb_lb, &en_sb_logical_dp_group, NULL); engine_add_input(&en_sync_to_sb_pb, &en_northd, sync_to_sb_pb_northd_handler); diff --git a/northd/lflow-mgr.c b/northd/lflow-mgr.c index d81b13a25c..d09dc7acb5 100644 --- a/northd/lflow-mgr.c +++ b/northd/lflow-mgr.c @@ -70,21 +70,6 @@ static struct ovs_mutex *lflow_hash_lock(const struct hmap *lflow_table, uint32_t hash); static void lflow_hash_unlock(struct ovs_mutex *hash_lock); -static struct ovn_dp_group *ovn_dp_group_get( - struct hmap *dp_groups, size_t desired_n, - const unsigned long *desired_bitmap, - size_t bitmap_len); -static struct ovn_dp_group *ovn_dp_group_create( - struct ovsdb_idl_txn *ovnsb_txn, struct hmap *dp_groups, - struct sbrec_logical_dp_group *, size_t desired_n, - const unsigned long *desired_bitmap, - size_t bitmap_len, bool is_switch, - const struct ovn_datapaths *ls_datapaths, - const struct ovn_datapaths *lr_datapaths); -static struct ovn_dp_group *ovn_dp_group_get( - struct hmap *dp_groups, size_t desired_n, - const unsigned long *desired_bitmap, - size_t bitmap_len); static struct sbrec_logical_dp_group *ovn_sb_insert_or_update_logical_dp_group( struct ovsdb_idl_txn *ovnsb_txn, struct sbrec_logical_dp_group *, @@ -564,31 +549,81 @@ lflow_table_add_lflow_default_drop(struct lflow_table *lflow_table, where, lflow_ref); } -/* Given a desired bitmap, finds a datapath group in 'dp_groups'. If it - * doesn't exist, creates a new one and adds it to 'dp_groups'. +struct ovn_dp_group * +ovn_dp_group_get(struct hmap *dp_groups, size_t desired_n, + const unsigned long *desired_bitmap, + size_t bitmap_len) +{ + uint32_t hash; + + hash = hash_int(desired_n, 0); + return ovn_dp_group_find(dp_groups, desired_bitmap, bitmap_len, hash); +} + +/* Creates a new datapath group and adds it to 'dp_groups'. * If 'sb_group' is provided, function will try to re-use this group by - * either taking it directly, or by modifying, if it's not already in use. */ + * either taking it directly, or by modifying, if it's not already in use. + * Caller should first call ovn_dp_group_get() before calling this function. */ struct ovn_dp_group * -ovn_dp_group_get_or_create(struct ovsdb_idl_txn *ovnsb_txn, - struct hmap *dp_groups, - struct sbrec_logical_dp_group *sb_group, - size_t desired_n, - const unsigned long *desired_bitmap, - size_t bitmap_len, - bool is_switch, - const struct ovn_datapaths *ls_datapaths, - const struct ovn_datapaths *lr_datapaths) +ovn_dp_group_create(struct ovsdb_idl_txn *ovnsb_txn, + struct hmap *dp_groups, + struct sbrec_logical_dp_group *sb_group, + size_t desired_n, + const unsigned long *desired_bitmap, + size_t bitmap_len, + bool is_switch, + const struct ovn_datapaths *ls_datapaths, + const struct ovn_datapaths *lr_datapaths) { struct ovn_dp_group *dpg; - dpg = ovn_dp_group_get(dp_groups, desired_n, desired_bitmap, bitmap_len); - if (dpg) { - return dpg; + bool update_dp_group = false, can_modify = false; + unsigned long *dpg_bitmap; + size_t i, n = 0; + + dpg_bitmap = sb_group ? bitmap_allocate(bitmap_len) : NULL; + for (i = 0; sb_group && i < sb_group->n_datapaths; i++) { + struct ovn_datapath *datapath_od; + + datapath_od = ovn_datapath_from_sbrec( + ls_datapaths ? &ls_datapaths->datapaths : NULL, + lr_datapaths ? &lr_datapaths->datapaths : NULL, + sb_group->datapaths[i]); + if (!datapath_od || ovn_datapath_is_stale(datapath_od)) { + break; + } + bitmap_set1(dpg_bitmap, datapath_od->index); + n++; + } + if (!sb_group || i != sb_group->n_datapaths) { + /* No group or stale group. Not going to be used. */ + update_dp_group = true; + can_modify = true; + } else if (!bitmap_equal(dpg_bitmap, desired_bitmap, bitmap_len)) { + /* The group in Sb is different. */ + update_dp_group = true; + /* We can modify existing group if it's not already in use. */ + can_modify = !ovn_dp_group_find(dp_groups, dpg_bitmap, + bitmap_len, hash_int(n, 0)); } - return ovn_dp_group_create(ovnsb_txn, dp_groups, sb_group, desired_n, - desired_bitmap, bitmap_len, is_switch, - ls_datapaths, lr_datapaths); + bitmap_free(dpg_bitmap); + + dpg = xzalloc(sizeof *dpg); + dpg->bitmap = bitmap_clone(desired_bitmap, bitmap_len); + if (!update_dp_group) { + dpg->dp_group = sb_group; + } else { + dpg->dp_group = ovn_sb_insert_or_update_logical_dp_group( + ovnsb_txn, + can_modify ? sb_group : NULL, + desired_bitmap, + is_switch ? ls_datapaths : lr_datapaths); + } + dpg->dpg_uuid = dpg->dp_group->header_.uuid; + hmap_insert(dp_groups, &dpg->node, hash_int(desired_n, 0)); + + return dpg; } void @@ -987,24 +1022,6 @@ ovn_dp_group_find(const struct hmap *dp_groups, return NULL; } -static void -inc_ovn_dp_group_ref(struct ovn_dp_group *dpg) -{ - dpg->refcnt++; -} - -static void -dec_ovn_dp_group_ref(struct hmap *dp_groups, struct ovn_dp_group *dpg) -{ - dpg->refcnt--; - - if (!dpg->refcnt) { - hmap_remove(dp_groups, &dpg->node); - free(dpg->bitmap); - free(dpg); - } -} - static struct sbrec_logical_dp_group * ovn_sb_insert_or_update_logical_dp_group( struct ovsdb_idl_txn *ovnsb_txn, @@ -1031,83 +1048,6 @@ ovn_sb_insert_or_update_logical_dp_group( return dp_group; } -static struct ovn_dp_group * -ovn_dp_group_get(struct hmap *dp_groups, size_t desired_n, - const unsigned long *desired_bitmap, - size_t bitmap_len) -{ - uint32_t hash; - - hash = hash_int(desired_n, 0); - return ovn_dp_group_find(dp_groups, desired_bitmap, bitmap_len, hash); -} - -/* Creates a new datapath group and adds it to 'dp_groups'. - * If 'sb_group' is provided, function will try to re-use this group by - * either taking it directly, or by modifying, if it's not already in use. - * Caller should first call ovn_dp_group_get() before calling this function. */ -static struct ovn_dp_group * -ovn_dp_group_create(struct ovsdb_idl_txn *ovnsb_txn, - struct hmap *dp_groups, - struct sbrec_logical_dp_group *sb_group, - size_t desired_n, - const unsigned long *desired_bitmap, - size_t bitmap_len, - bool is_switch, - const struct ovn_datapaths *ls_datapaths, - const struct ovn_datapaths *lr_datapaths) -{ - struct ovn_dp_group *dpg; - - bool update_dp_group = false, can_modify = false; - unsigned long *dpg_bitmap; - size_t i, n = 0; - - dpg_bitmap = sb_group ? bitmap_allocate(bitmap_len) : NULL; - for (i = 0; sb_group && i < sb_group->n_datapaths; i++) { - struct ovn_datapath *datapath_od; - - datapath_od = ovn_datapath_from_sbrec( - ls_datapaths ? &ls_datapaths->datapaths : NULL, - lr_datapaths ? &lr_datapaths->datapaths : NULL, - sb_group->datapaths[i]); - if (!datapath_od || ovn_datapath_is_stale(datapath_od)) { - break; - } - bitmap_set1(dpg_bitmap, datapath_od->index); - n++; - } - if (!sb_group || i != sb_group->n_datapaths) { - /* No group or stale group. Not going to be used. */ - update_dp_group = true; - can_modify = true; - } else if (!bitmap_equal(dpg_bitmap, desired_bitmap, bitmap_len)) { - /* The group in Sb is different. */ - update_dp_group = true; - /* We can modify existing group if it's not already in use. */ - can_modify = !ovn_dp_group_find(dp_groups, dpg_bitmap, - bitmap_len, hash_int(n, 0)); - } - - bitmap_free(dpg_bitmap); - - dpg = xzalloc(sizeof *dpg); - dpg->bitmap = bitmap_clone(desired_bitmap, bitmap_len); - if (!update_dp_group) { - dpg->dp_group = sb_group; - } else { - dpg->dp_group = ovn_sb_insert_or_update_logical_dp_group( - ovnsb_txn, - can_modify ? sb_group : NULL, - desired_bitmap, - is_switch ? ls_datapaths : lr_datapaths); - } - dpg->dpg_uuid = dpg->dp_group->header_.uuid; - hmap_insert(dp_groups, &dpg->node, hash_int(desired_n, 0)); - - return dpg; -} - /* Adds an OVN datapath to a datapath group of existing logical flow. * Version to use when hash bucket locking is NOT required or the corresponding * hash lock is already taken. */ diff --git a/northd/lflow-mgr.h b/northd/lflow-mgr.h index f215891b97..00554ef78a 100644 --- a/northd/lflow-mgr.h +++ b/northd/lflow-mgr.h @@ -157,7 +157,10 @@ ovn_dp_groups_init(struct hmap *dp_groups) void ovn_dp_groups_clear(struct hmap *dp_groups); void ovn_dp_groups_destroy(struct hmap *dp_groups); -struct ovn_dp_group *ovn_dp_group_get_or_create( +struct ovn_dp_group *ovn_dp_group_get(struct hmap *dp_groups, size_t desired_n, + const unsigned long *desired_bitmap, + size_t bitmap_len); +struct ovn_dp_group *ovn_dp_group_create( struct ovsdb_idl_txn *ovnsb_txn, struct hmap *dp_groups, struct sbrec_logical_dp_group *sb_group, size_t desired_n, const unsigned long *desired_bitmap, @@ -165,4 +168,22 @@ struct ovn_dp_group *ovn_dp_group_get_or_create( const struct ovn_datapaths *ls_datapaths, const struct ovn_datapaths *lr_datapaths); +static inline void +inc_ovn_dp_group_ref(struct ovn_dp_group *dpg) +{ + dpg->refcnt++; +} + +static inline void +dec_ovn_dp_group_ref(struct hmap *dp_groups, struct ovn_dp_group *dpg) +{ + dpg->refcnt--; + + if (!dpg->refcnt) { + hmap_remove(dp_groups, &dpg->node); + free(dpg->bitmap); + free(dpg); + } +} + #endif /* LFLOW_MGR_H */ \ No newline at end of file diff --git a/northd/northd.c b/northd/northd.c index 96a5b52127..d21a070fdd 100644 --- a/northd/northd.c +++ b/northd/northd.c @@ -3745,244 +3745,6 @@ build_lb_port_related_data( build_lswitch_lbs_from_lrouter(lr_datapaths, lb_dps_map, lb_group_dps_map); } -struct sb_lb { - struct hmap_node hmap_node; - - const struct sbrec_load_balancer *slb; - struct ovn_dp_group *dpg; - struct ovn_dp_group *lr_dpg; - struct uuid lb_uuid; -}; - -static struct sb_lb * -find_slb_in_sb_lbs(struct hmap *sb_lbs, const struct uuid *lb_uuid) -{ - struct sb_lb *sb_lb; - HMAP_FOR_EACH_WITH_HASH (sb_lb, hmap_node, uuid_hash(lb_uuid), sb_lbs) { - if (uuid_equals(&sb_lb->lb_uuid, lb_uuid)) { - return sb_lb; - } - } - - return NULL; -} - -/* Syncs relevant load balancers (applied to logical switches) to the - * Southbound database. - */ -void -sync_lbs(struct ovsdb_idl_txn *ovnsb_txn, - const struct sbrec_load_balancer_table *sbrec_load_balancer_table, - struct ovn_datapaths *ls_datapaths, - struct ovn_datapaths *lr_datapaths, - struct hmap *lb_dps_map, - struct chassis_features *chassis_features) -{ - struct hmap ls_dp_groups = HMAP_INITIALIZER(&ls_dp_groups); - struct hmap lr_dp_groups = HMAP_INITIALIZER(&lr_dp_groups); - struct ovn_lb_datapaths *lb_dps; - struct hmap sb_lbs = HMAP_INITIALIZER(&sb_lbs); - - /* Delete any stale SB load balancer rows and create datapath - * groups for existing ones. */ - struct hmapx existing_lbs = HMAPX_INITIALIZER(&existing_lbs); - const struct sbrec_load_balancer *sbrec_lb; - SBREC_LOAD_BALANCER_TABLE_FOR_EACH_SAFE (sbrec_lb, - sbrec_load_balancer_table) { - const char *nb_lb_uuid = smap_get(&sbrec_lb->external_ids, "lb_id"); - struct uuid lb_uuid; - if (!nb_lb_uuid || !uuid_from_string(&lb_uuid, nb_lb_uuid)) { - sbrec_load_balancer_delete(sbrec_lb); - continue; - } - - /* Delete any SB load balancer entries that refer to NB load balancers - * that don't exist anymore or are not applied to switches/routers - * anymore. - * - * There is also a special case in which duplicate LBs might be created - * in the SB, e.g., due to the fact that OVSDB only ensures - * "at-least-once" consistency for clustered database tables that - * are not indexed in any way. - */ - lb_dps = ovn_lb_datapaths_find(lb_dps_map, &lb_uuid); - if (!lb_dps || (!lb_dps->n_nb_ls && !lb_dps->n_nb_lr) || - !hmapx_add(&existing_lbs, lb_dps)) { - sbrec_load_balancer_delete(sbrec_lb); - continue; - } - - struct sb_lb *sb_lb = xzalloc(sizeof *sb_lb); - sb_lb->lb_uuid = lb_uuid; - sb_lb->slb = sbrec_lb; - hmap_insert(&sb_lbs, &sb_lb->hmap_node, uuid_hash(&lb_uuid)); - - /* Find or create datapath group for this load balancer. */ - if (lb_dps->n_nb_ls) { - struct sbrec_logical_dp_group *ls_datapath_group - = chassis_features->ls_dpg_column - ? sb_lb->slb->ls_datapath_group - : sb_lb->slb->datapath_group; /* deprecated */ - sb_lb->dpg = ovn_dp_group_get_or_create( - ovnsb_txn, &ls_dp_groups, - ls_datapath_group, - lb_dps->n_nb_ls, lb_dps->nb_ls_map, - ods_size(ls_datapaths), true, - ls_datapaths, NULL); - } - if (lb_dps->n_nb_lr) { - sb_lb->lr_dpg = ovn_dp_group_get_or_create( - ovnsb_txn, &lr_dp_groups, - sb_lb->slb->lr_datapath_group, - lb_dps->n_nb_lr, lb_dps->nb_lr_map, - ods_size(lr_datapaths), false, - NULL, lr_datapaths); - } - } - hmapx_destroy(&existing_lbs); - - /* Create SB Load balancer records if not present and sync - * the SB load balancer columns. */ - HMAP_FOR_EACH (lb_dps, hmap_node, lb_dps_map) { - - if (!lb_dps->n_nb_ls && !lb_dps->n_nb_lr) { - continue; - } - - /* Store the fact that northd provides the original (destination IP + - * transport port) tuple. - */ - struct smap options; - smap_clone(&options, &lb_dps->lb->nlb->options); - smap_replace(&options, "hairpin_orig_tuple", "true"); - - struct sb_lb *sb_lb = find_slb_in_sb_lbs(&sb_lbs, - &lb_dps->lb->nlb->header_.uuid); - ovs_assert(!sb_lb || (sb_lb->slb && (sb_lb->dpg || sb_lb->lr_dpg))); - struct ovn_dp_group *lb_dpg = NULL, *lb_lr_dpg = NULL; - if (!sb_lb) { - sbrec_lb = sbrec_load_balancer_insert(ovnsb_txn); - char *lb_id = xasprintf( - UUID_FMT, UUID_ARGS(&lb_dps->lb->nlb->header_.uuid)); - const struct smap external_ids = - SMAP_CONST1(&external_ids, "lb_id", lb_id); - sbrec_load_balancer_set_external_ids(sbrec_lb, &external_ids); - free(lb_id); - } else { - sbrec_lb = sb_lb->slb; - lb_dpg = sb_lb->dpg; - lb_lr_dpg = sb_lb->lr_dpg; - } - - /* Find or create datapath group for this load balancer. */ - if (!lb_dpg && lb_dps->n_nb_ls) { - struct sbrec_logical_dp_group *ls_datapath_group - = chassis_features->ls_dpg_column - ? sbrec_lb->ls_datapath_group - : sbrec_lb->datapath_group; /* deprecated */ - lb_dpg = ovn_dp_group_get_or_create( - ovnsb_txn, &ls_dp_groups, - ls_datapath_group, - lb_dps->n_nb_ls, lb_dps->nb_ls_map, - ods_size(ls_datapaths), true, - ls_datapaths, NULL); - } - if (!lb_lr_dpg && lb_dps->n_nb_lr) { - lb_lr_dpg = ovn_dp_group_get_or_create( - ovnsb_txn, &lr_dp_groups, - sbrec_lb->lr_datapath_group, - lb_dps->n_nb_lr, lb_dps->nb_lr_map, - ods_size(lr_datapaths), false, - NULL, lr_datapaths); - } - - /* Update columns. */ - sbrec_load_balancer_set_name(sbrec_lb, lb_dps->lb->nlb->name); - sbrec_load_balancer_set_vips(sbrec_lb, - ovn_northd_lb_get_vips(lb_dps->lb)); - sbrec_load_balancer_set_protocol(sbrec_lb, lb_dps->lb->nlb->protocol); - - if (chassis_features->ls_dpg_column) { - sbrec_load_balancer_set_ls_datapath_group( - sbrec_lb, lb_dpg ? lb_dpg->dp_group : NULL - ); - sbrec_load_balancer_set_datapath_group(sbrec_lb, NULL); - } else { - /* datapath_group column is deprecated. */ - sbrec_load_balancer_set_ls_datapath_group(sbrec_lb, NULL); - sbrec_load_balancer_set_datapath_group( - sbrec_lb, lb_dpg ? lb_dpg->dp_group : NULL - ); - } - - sbrec_load_balancer_set_lr_datapath_group( - sbrec_lb, lb_lr_dpg ? lb_lr_dpg->dp_group : NULL - ); - sbrec_load_balancer_set_options(sbrec_lb, &options); - /* Clearing 'datapaths' column, since 'dp_group' is in use. */ - sbrec_load_balancer_set_datapaths(sbrec_lb, NULL, 0); - smap_destroy(&options); - } - - struct ovn_dp_group *dpg; - HMAP_FOR_EACH_POP (dpg, node, &ls_dp_groups) { - bitmap_free(dpg->bitmap); - free(dpg); - } - hmap_destroy(&ls_dp_groups); - - HMAP_FOR_EACH_POP (dpg, node, &lr_dp_groups) { - bitmap_free(dpg->bitmap); - free(dpg); - } - hmap_destroy(&lr_dp_groups); - - struct sb_lb *sb_lb; - HMAP_FOR_EACH_POP (sb_lb, hmap_node, &sb_lbs) { - free(sb_lb); - } - hmap_destroy(&sb_lbs); - - /* Datapath_Binding.load_balancers is not used anymore, it's still in the - * schema for compatibility reasons. Reset it to empty, just in case. - */ - struct ovn_datapath *od; - HMAP_FOR_EACH (od, key_node, &ls_datapaths->datapaths) { - ovs_assert(od->nbs); - - if (od->sb->n_load_balancers) { - sbrec_datapath_binding_set_load_balancers(od->sb, NULL, 0); - } - } - HMAP_FOR_EACH (od, key_node, &lr_datapaths->datapaths) { - ovs_assert(od->nbr); - - if (od->sb->n_load_balancers) { - sbrec_datapath_binding_set_load_balancers(od->sb, NULL, 0); - } - } -} - -bool -check_sb_lb_duplicates(const struct sbrec_load_balancer_table *table) -{ - struct sset existing_nb_lb_uuids = - SSET_INITIALIZER(&existing_nb_lb_uuids); - const struct sbrec_load_balancer *sbrec_lb; - bool duplicates = false; - - SBREC_LOAD_BALANCER_TABLE_FOR_EACH (sbrec_lb, table) { - const char *nb_lb_uuid = smap_get(&sbrec_lb->external_ids, "lb_id"); - if (nb_lb_uuid && !sset_add(&existing_nb_lb_uuids, nb_lb_uuid)) { - duplicates = true; - break; - } - } - - sset_destroy(&existing_nb_lb_uuids); - return duplicates; -} - /* Syncs the SB port binding for the ovn_port 'op' of a logical switch port. * Caller should make sure that the OVN SB IDL txn is not NULL. Presently it * only syncs the nat column of port binding corresponding to the 'op->nbsp' */ diff --git a/northd/northd.h b/northd/northd.h index 297b1a65b9..579edb8bdf 100644 --- a/northd/northd.h +++ b/northd/northd.h @@ -715,12 +715,6 @@ const char *northd_get_svc_monitor_mac(void); const struct ovn_datapath *northd_get_datapath_for_port( const struct hmap *ls_ports, const char *port_name); -void sync_lbs(struct ovsdb_idl_txn *, const struct sbrec_load_balancer_table *, - struct ovn_datapaths *ls_datapaths, - struct ovn_datapaths *lr_datapaths, - struct hmap *lbs, - struct chassis_features *chassis_features); -bool check_sb_lb_duplicates(const struct sbrec_load_balancer_table *); struct lr_stateful_table; void sync_pbs(struct ovsdb_idl_txn *, struct hmap *ls_ports, diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at index 263eb343eb..5c33a8a6c2 100644 --- a/tests/ovn-northd.at +++ b/tests/ovn-northd.at @@ -2938,6 +2938,8 @@ sw1_sb_uuid=$(fetch_column datapath_binding _uuid external_ids:name=sw1) echo "$sw0_sb_uuid" > sw_sb_uuids echo "$sw1_sb_uuid" >> sw_sb_uuids +lb0_dp_group=$(fetch_column sb:load_balancer ls_datapath_group name=lb0) + echo echo "__file__:__line__: Check that SB lb0 has sw0 and sw1 in datapaths column." AT_CHECK_UNQUOTED([ovn-sbctl --bare --columns _uuid,datapaths find Logical_DP_Group dnl @@ -2945,6 +2947,8 @@ AT_CHECK_UNQUOTED([ovn-sbctl --bare --columns _uuid,datapaths find Logical_DP_Gr $(cat sw_sb_uuids | sort) ]) +lbg0_dp_group=$(fetch_column sb:load_balancer ls_datapath_group name=lbg0) + echo echo "__file__:__line__: Check that SB lbg0 has sw0 and sw1 in datapaths column." AT_CHECK_UNQUOTED([ovn-sbctl --bare --columns _uuid,datapaths find Logical_DP_Group dnl @@ -3014,6 +3018,64 @@ echo "__file__:__line__: Set hairpin_snat_ip on lb1 and check that SB DB is upda check ovn-nbctl --wait=sb set Load_Balancer lb1 options:hairpin_snat_ip="42.42.42.42 4242::4242" check_column "$lb1_uuid" sb:load_balancer _uuid name=lb1 options='{hairpin_orig_tuple="true", hairpin_snat_ip="42.42.42.42 4242::4242"}' +echo +echo "__file__:__line__: Set option:bar=foo on lbg1 and check that sync_to_sb_lb engine node didn't recompute." + +check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats +check ovn-nbctl --wait=sb set load_balancer lbg1 options:bar=foo +check_engine_stats sync_to_sb_lb norecompute compute + +# Manually clear the sb:load_balancer.ls_datapath_group column for +# all load balancers and check that it is resynced back when the +# NB lbg1 is modified. + +echo +echo "__file__:__line__: Clear the ls_datapath_group column of all the SB load balancers." + +for l in $(ovn-sbctl --bare --columns _uuid list load_balancer) +do + check ovn-sbctl clear load_balancer $l ls_datapath_group +done + +lbg1_dp_group=$(fetch_column sb:load_balancer ls_datapath_group name=lbg1) +AT_CHECK([test "$lbg1_dp_group" = ""]) + +echo +echo "__file__:__line__: Set option:foo=bar on lbg1 and check that sync_to_sb_lb engine node recomputed." + +check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats +check ovn-nbctl --wait=sb set load_balancer lbg1 options:foo=bar +check_engine_stats sync_to_sb_lb recompute compute + +# Manually clear the sb:load_balancer.lr_datapath_group column for +# all load balancers and check that it is resynced back when the +# NB lbg1 is modified. + +echo +echo "__file__:__line__: Clear the lr_datapath_group column of all the SB load balancers." + +for l in $(ovn-sbctl --bare --columns _uuid list load_balancer) +do + check ovn-sbctl clear load_balancer $l lr_datapath_group +done + +echo +echo "__file__:__line__: Set option:foo=foo on lbg1 and check that sync_to_sb_lb engine node recomputed." + +check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats +check ovn-nbctl --wait=sb set load_balancer lbg1 options:foo=foo +check_engine_stats sync_to_sb_lb recompute compute + +lbg1_dp_group=$(fetch_column sb:load_balancer ls_datapath_group name=lbg1) + +echo +echo "__file__:__line__: Check that SB lbg1 has sw0 and sw1 in datapaths column." + +AT_CHECK_UNQUOTED([ovn-sbctl --bare --columns _uuid,datapaths find Logical_DP_Group dnl + | grep -A1 $lbg1_dp_group | tail -1 | tr ' ' '\n' | sort], [0], [dnl +$(cat sw_sb_uuids | sort) +]) + echo echo "__file__:__line__: Delete load balancers lb1 and lbg1 and check that datapath sw1's load_balancers is still empty." @@ -3024,6 +3086,8 @@ echo echo "__file__:__line__: Delete switch sw0." check ovn-nbctl --wait=sb ls-del sw0 +lb0_dp_group=$(fetch_column sb:load_balancer ls_datapath_group name=lb0) + echo echo "__file__:__line__: Check that SB lb0 has only sw1 in datapaths column." AT_CHECK_UNQUOTED([ovn-sbctl --bare --columns _uuid,datapaths find Logical_DP_Group dnl @@ -3031,6 +3095,8 @@ AT_CHECK_UNQUOTED([ovn-sbctl --bare --columns _uuid,datapaths find Logical_DP_Gr $sw1_sb_uuid ]) +lbg0_dp_group=$(fetch_column sb:load_balancer ls_datapath_group name=lbg0) + echo echo "__file__:__line__: Check that SB lbg0 has only sw1 in datapaths column." AT_CHECK_UNQUOTED([ovn-sbctl --bare --columns _uuid,datapaths find Logical_DP_Group dnl @@ -10521,8 +10587,7 @@ check ovn-nbctl --wait=sb lb-add lb1 10.0.0.10:80 10.0.0.3:80 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute nocompute - +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE(1) check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats @@ -10531,7 +10596,7 @@ check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute nocompute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats @@ -10541,7 +10606,7 @@ check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute nocompute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats @@ -10550,7 +10615,7 @@ check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute nocompute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE(1) check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats @@ -10560,7 +10625,7 @@ check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute nocompute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE(1) check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats @@ -10618,7 +10683,7 @@ check_engine_stats lr_stateful norecompute compute # A LB applied to a switch/router triggers: # - a recompute in the first iteration (handling northd change) # - a compute in the second iteration (handling SB update) -check_engine_stats sync_to_sb_lb recompute compute +check_engine_stats sync_to_sb_lb norecompute compute check_engine_stats ls_stateful norecompute compute check_engine_stats lflow norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE((1)) @@ -10631,7 +10696,7 @@ check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute check_engine_stats ls_stateful norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute compute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE(1) # Cleanup the vip of lb1. @@ -10642,7 +10707,7 @@ check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute check_engine_stats ls_stateful norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute compute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE(1) # Set the vips of lb1 back @@ -10653,7 +10718,7 @@ check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute check_engine_stats ls_stateful norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute compute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE(1) # Add another vip to lb1 @@ -10664,7 +10729,7 @@ check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute check_engine_stats ls_stateful norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute compute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE(1) # Disassociate lb1 from sw0. There should be a full recompute of northd engine node. @@ -10687,7 +10752,7 @@ check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute check_engine_stats ls_stateful norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute compute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats @@ -10708,7 +10773,7 @@ check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute compute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Modify the backend of the lb1 vip @@ -10718,7 +10783,7 @@ check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute compute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Cleanup the vip of lb1. @@ -10728,7 +10793,7 @@ check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute compute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Set the vips of lb1 back @@ -10738,7 +10803,7 @@ check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute compute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Add another vip to lb1 @@ -10748,7 +10813,7 @@ check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute compute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats @@ -10779,7 +10844,7 @@ check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute nocompute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats @@ -10796,7 +10861,7 @@ check ovn-nbctl --wait=sb add load_balancer_group . load_Balancer $lb1_uuid check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute nocompute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats @@ -10806,7 +10871,7 @@ check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute check_engine_stats ls_stateful norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute compute +check_engine_stats sync_to_sb_lb norecompute compute # Update lb and this should not result in northd recompute check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats @@ -10816,7 +10881,7 @@ check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute check_engine_stats ls_stateful norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute compute +check_engine_stats sync_to_sb_lb norecompute compute # Modify the backend of the lb1 vip check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats @@ -10826,7 +10891,7 @@ check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute check_engine_stats ls_stateful norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute compute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Cleanup the vip of lb1. @@ -10837,7 +10902,7 @@ check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute check_engine_stats ls_stateful norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute compute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Set the vips of lb1 back @@ -10848,7 +10913,7 @@ check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute check_engine_stats ls_stateful norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute compute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Add another vip to lb1 @@ -10859,7 +10924,7 @@ check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute check_engine_stats ls_stateful norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute compute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats @@ -10875,7 +10940,7 @@ check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute compute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Modify the backend of the lb1 vip @@ -10885,7 +10950,7 @@ check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute compute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Cleanup the vip of lb1. @@ -10895,7 +10960,7 @@ check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute compute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Set the vips of lb1 back @@ -10905,7 +10970,7 @@ check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute compute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Add another vip to lb1 @@ -10915,7 +10980,7 @@ check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute compute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats @@ -10935,7 +11000,7 @@ check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute check_engine_stats ls_stateful norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute compute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats @@ -10979,7 +11044,7 @@ check_engine_stats northd norecompute compute check_engine_stats ls_stateful norecompute compute check_engine_stats lr_stateful norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute nocompute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats @@ -10989,7 +11054,7 @@ check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute check_engine_stats ls_stateful norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute compute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats @@ -10998,7 +11063,7 @@ check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute compute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats @@ -11008,7 +11073,7 @@ check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute check_engine_stats ls_stateful norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute nocompute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats @@ -11018,7 +11083,7 @@ check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute check_engine_stats ls_stateful norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute nocompute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats @@ -11028,7 +11093,7 @@ check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute compute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats @@ -11059,7 +11124,7 @@ check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute check_engine_stats ls_stateful norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute compute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Deleting lb2 should result in lflow recompute as it is @@ -11071,7 +11136,7 @@ check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute check_engine_stats ls_stateful norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute compute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats @@ -11281,6 +11346,7 @@ check ovn-nbctl --wait=sb ls-lb-add sw0 lb1 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lflow norecompute compute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Clear the VIPs of lb1 @@ -11289,6 +11355,7 @@ check ovn-nbctl --wait=sb clear load_balancer . vips check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lflow norecompute compute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats @@ -11296,6 +11363,7 @@ check ovn-nbctl --wait=sb lb-del lb1 check_engine_stats lb_data norecompute compute check_engine_stats northd recompute nocompute check_engine_stats lflow recompute nocompute +check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE AT_CLEANUP @@ -11544,6 +11612,7 @@ check_engine_stats northd recompute nocompute check_engine_stats lr_nat recompute nocompute check_engine_stats lr_stateful recompute nocompute check_engine_stats sync_to_sb_pb recompute nocompute +check_engine_stats sync_to_sb_lb recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11557,6 +11626,7 @@ check_engine_stats northd recompute compute check_engine_stats lr_nat recompute nocompute check_engine_stats lr_stateful recompute nocompute check_engine_stats sync_to_sb_pb recompute nocompute +check_engine_stats sync_to_sb_lb recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11569,6 +11639,7 @@ check_engine_stats northd recompute compute check_engine_stats lr_nat recompute nocompute check_engine_stats lr_stateful recompute nocompute check_engine_stats sync_to_sb_pb recompute nocompute +check_engine_stats sync_to_sb_lb recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11595,6 +11666,7 @@ check_engine_stats northd recompute nocompute check_engine_stats lr_nat recompute nocompute check_engine_stats lr_stateful recompute nocompute check_engine_stats sync_to_sb_pb recompute nocompute +check_engine_stats sync_to_sb_lb recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11607,6 +11679,7 @@ check_engine_stats northd norecompute compute check_engine_stats lr_nat norecompute compute check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_pb recompute nocompute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Update the NAT options column @@ -11616,6 +11689,7 @@ check_engine_stats northd norecompute compute check_engine_stats lr_nat norecompute compute check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_pb recompute nocompute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Update the NAT external_ip column @@ -11626,6 +11700,7 @@ check_engine_stats lr_nat norecompute compute check_engine_stats lr_stateful norecompute compute check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_pb recompute nocompute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Update the NAT logical_ip column @@ -11636,6 +11711,7 @@ check_engine_stats lr_nat norecompute compute check_engine_stats lr_stateful norecompute compute check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_pb recompute nocompute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Update the NAT type @@ -11646,6 +11722,7 @@ check_engine_stats lr_nat norecompute compute check_engine_stats lr_stateful norecompute compute check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_pb recompute nocompute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Create a dnat_and_snat NAT with external_mac and logical_port @@ -11656,6 +11733,7 @@ check_engine_stats lr_nat norecompute compute check_engine_stats lr_stateful norecompute compute check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_pb recompute nocompute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE nat2_uuid=$(ovn-nbctl --bare --columns _uuid find nat logical_ip=10.0.0.4) @@ -11667,6 +11745,7 @@ check_engine_stats lr_nat norecompute compute check_engine_stats lr_stateful norecompute compute check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_pb recompute nocompute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Create a load balancer and add the lb vip as NAT @@ -11683,6 +11762,7 @@ check_engine_stats northd norecompute compute check_engine_stats lr_nat norecompute compute check_engine_stats lr_stateful norecompute compute check_engine_stats sync_to_sb_pb recompute nocompute +check_engine_stats sync_to_sb_lb norecompute compute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11694,6 +11774,7 @@ check_engine_stats northd norecompute compute check_engine_stats lr_nat norecompute compute check_engine_stats lr_stateful norecompute compute check_engine_stats sync_to_sb_pb recompute nocompute +check_engine_stats sync_to_sb_lb norecompute compute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11705,6 +11786,7 @@ check_engine_stats northd norecompute compute check_engine_stats lr_nat norecompute compute check_engine_stats lr_stateful norecompute compute check_engine_stats sync_to_sb_pb recompute nocompute +check_engine_stats sync_to_sb_lb norecompute compute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11716,6 +11798,7 @@ check_engine_stats northd norecompute compute check_engine_stats lr_nat norecompute compute check_engine_stats lr_stateful norecompute compute check_engine_stats sync_to_sb_pb recompute nocompute +check_engine_stats sync_to_sb_lb norecompute compute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11727,6 +11810,7 @@ check_engine_stats lr_nat norecompute compute check_engine_stats lr_stateful norecompute compute check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_pb recompute nocompute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Create router Policy @@ -11736,6 +11820,7 @@ check_engine_stats northd recompute nocompute check_engine_stats lr_nat recompute nocompute check_engine_stats lr_stateful recompute nocompute check_engine_stats sync_to_sb_pb recompute nocompute +check_engine_stats sync_to_sb_lb recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11745,6 +11830,7 @@ check_engine_stats northd recompute nocompute check_engine_stats lr_nat recompute nocompute check_engine_stats lr_stateful recompute nocompute check_engine_stats sync_to_sb_pb recompute nocompute +check_engine_stats sync_to_sb_lb recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE From patchwork Thu Jan 11 15:34:47 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Numan Siddique X-Patchwork-Id: 1885685 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::133; helo=smtp2.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp2.osuosl.org (smtp2.osuosl.org [IPv6:2605:bc80:3010::133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4T9pfP1t4Wz1yPm for ; Fri, 12 Jan 2024 02:35:57 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 3CFBB438F2; Thu, 11 Jan 2024 15:35:55 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 3CFBB438F2 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id LP1VCHTOIfjp; Thu, 11 Jan 2024 15:35:50 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp2.osuosl.org (Postfix) with ESMTPS id 6E1B64390A; Thu, 11 Jan 2024 15:35:49 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 6E1B64390A Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 303C4C0DD4; Thu, 11 Jan 2024 15:35:49 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 497E6C0037 for ; Thu, 11 Jan 2024 15:35:47 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 5114E6F5B1 for ; Thu, 11 Jan 2024 15:35:27 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 5114E6F5B1 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id E4Y4xqvd9Mmb for ; Thu, 11 Jan 2024 15:35:23 +0000 (UTC) Received: from relay6-d.mail.gandi.net (relay6-d.mail.gandi.net [217.70.183.198]) by smtp3.osuosl.org (Postfix) with ESMTPS id 72CFB6F620 for ; Thu, 11 Jan 2024 15:35:21 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 72CFB6F620 Received: by mail.gandi.net (Postfix) with ESMTPSA id CF1ACC0009; Thu, 11 Jan 2024 15:35:17 +0000 (UTC) From: numans@ovn.org To: dev@openvswitch.org Date: Thu, 11 Jan 2024 10:34:47 -0500 Message-ID: <20240111153447.2790612-1-numans@ovn.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240111152752.2789854-1-numans@ovn.org> References: <20240111152752.2789854-1-numans@ovn.org> MIME-Version: 1.0 X-GND-Sasl: numans@ovn.org Subject: [ovs-dev] [PATCH ovn v5 16/16] northd: Add I-P for NB_Global and SB_Global. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Numan Siddique A new engine node "global_config" is added which handles the changes to NB_Global an SB_Global tables. It also creates these rows if not present. Without the I-P, any changes to the options column of these tables result in recompute of 'northd' and 'lflow' engine nodes. Signed-off-by: Numan Siddique Acked-by: Dumitru Ceara --- northd/aging.c | 21 +- northd/automake.mk | 2 + northd/en-global-config.c | 588 ++++++++++++++++++++++++++++++++++++++ northd/en-global-config.h | 65 +++++ northd/en-lflow.c | 11 +- northd/en-northd.c | 52 ++-- northd/en-northd.h | 2 +- northd/en-sync-sb.c | 26 +- northd/inc-proc-northd.c | 38 ++- northd/northd.c | 230 +++------------ northd/northd.h | 24 +- tests/ovn-northd.at | 256 +++++++++++++++-- 12 files changed, 1017 insertions(+), 298 deletions(-) create mode 100644 northd/en-global-config.c create mode 100644 northd/en-global-config.h diff --git a/northd/aging.c b/northd/aging.c index cdf5f4464e..b76963a2dd 100644 --- a/northd/aging.c +++ b/northd/aging.c @@ -15,6 +15,7 @@ #include +#include "en-global-config.h" #include "lib/inc-proc-eng.h" #include "lib/ovn-nb-idl.h" #include "lib/ovn-sb-idl.h" @@ -347,15 +348,10 @@ aging_context_handle_timestamp(struct aging_context *ctx, int64_t timestamp, static uint32_t get_removal_limit(struct engine_node *node, const char *name) { - const struct nbrec_nb_global_table *nb_global_table = - EN_OVSDB_GET(engine_get_input("NB_nb_global", node)); - const struct nbrec_nb_global *nb = - nbrec_nb_global_table_first(nb_global_table); - if (!nb) { - return 0; - } + struct ed_type_global_config *global_config = + engine_get_input_data("global_config", node); - return smap_get_uint(&nb->options, name, 0); + return smap_get_uint(&global_config->nb_options, name, 0); } /* MAC binding aging */ @@ -394,11 +390,14 @@ en_mac_binding_aging_run(struct engine_node *node, void *data OVS_UNUSED) { const struct engine_context *eng_ctx = engine_get_context(); struct northd_data *northd_data = engine_get_input_data("northd", node); + struct ed_type_global_config *global_config = + engine_get_input_data("global_config", node); + struct aging_waker *waker = engine_get_input_data("mac_binding_aging_waker", node); if (!eng_ctx->ovnsb_idl_txn || - !northd_data->features.mac_binding_timestamp || + !global_config->features.mac_binding_timestamp || time_msec() < waker->next_wake_msec) { return; } @@ -530,9 +529,11 @@ en_fdb_aging_run(struct engine_node *node, void *data OVS_UNUSED) const struct engine_context *eng_ctx = engine_get_context(); struct northd_data *northd_data = engine_get_input_data("northd", node); struct aging_waker *waker = engine_get_input_data("fdb_aging_waker", node); + struct ed_type_global_config *global_config = + engine_get_input_data("global_config", node); if (!eng_ctx->ovnsb_idl_txn || - !northd_data->features.fdb_timestamp || + !global_config->features.fdb_timestamp || time_msec() < waker->next_wake_msec) { return; } diff --git a/northd/automake.mk b/northd/automake.mk index 19abb0dece..d491973a8b 100644 --- a/northd/automake.mk +++ b/northd/automake.mk @@ -8,6 +8,8 @@ northd_ovn_northd_SOURCES = \ northd/northd.c \ northd/northd.h \ northd/ovn-northd.c \ + northd/en-global-config.c \ + northd/en-global-config.h \ northd/en-northd.c \ northd/en-northd.h \ northd/en-lflow.c \ diff --git a/northd/en-global-config.c b/northd/en-global-config.c new file mode 100644 index 0000000000..0d218f2ab5 --- /dev/null +++ b/northd/en-global-config.c @@ -0,0 +1,588 @@ +/* + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at: + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#include + +#include +#include +#include + +/* OVS includes */ +#include "openvswitch/vlog.h" + +/* OVN includes */ +#include "debug.h" +#include "en-global-config.h" +#include "include/ovn/features.h" +#include "ipam.h" +#include "lib/ovn-nb-idl.h" +#include "lib/ovn-sb-idl.h" +#include "northd.h" + + +VLOG_DEFINE_THIS_MODULE(en_global_config); + +/* static function declarations. */ +static void northd_enable_all_features(struct ed_type_global_config *); +static void build_chassis_features(const struct sbrec_chassis_table *, + struct chassis_features *); +static bool chassis_features_changed(const struct chassis_features *, + const struct chassis_features *); +static bool config_out_of_sync(const struct smap *config, + const struct smap *saved_config, + const char *key, bool must_be_present); +static bool check_nb_options_out_of_sync(const struct nbrec_nb_global *, + struct ed_type_global_config *); +static void update_sb_config_options_to_sbrec(struct ed_type_global_config *, + const struct sbrec_sb_global *); + +void * +en_global_config_init(struct engine_node *node OVS_UNUSED, + struct engine_arg *args OVS_UNUSED) +{ + struct ed_type_global_config *data = xzalloc(sizeof *data); + smap_init(&data->nb_options); + smap_init(&data->sb_options); + northd_enable_all_features(data); + return data; +} + +void +en_global_config_run(struct engine_node *node , void *data) +{ + const struct engine_context *eng_ctx = engine_get_context(); + if (!eng_ctx->ovnnb_idl_txn || !eng_ctx->ovnsb_idl_txn) { + return; + } + + const struct nbrec_nb_global_table *nb_global_table = + EN_OVSDB_GET(engine_get_input("NB_nb_global", node)); + const struct sbrec_sb_global_table *sb_global_table = + EN_OVSDB_GET(engine_get_input("SB_sb_global", node)); + const struct sbrec_chassis_table *sbrec_chassis_table = + EN_OVSDB_GET(engine_get_input("SB_chassis", node)); + + en_global_config_clear_tracked_data(data); + + struct ed_type_global_config *config_data = data; + + /* Sync ipsec configuration. + * Copy nb_cfg from northbound to southbound database. + * Also set up to update sb_cfg once our southbound transaction commits. */ + const struct nbrec_nb_global *nb = + nbrec_nb_global_table_first(nb_global_table); + if (!nb) { + nb = nbrec_nb_global_insert(eng_ctx->ovnnb_idl_txn); + } + + const char *mac_addr_prefix = set_mac_prefix(smap_get(&nb->options, + "mac_prefix")); + + const char *monitor_mac = smap_get(&nb->options, "svc_monitor_mac"); + if (monitor_mac) { + if (eth_addr_from_string(monitor_mac, + &config_data->svc_monitor_mac_ea)) { + snprintf(config_data->svc_monitor_mac, + sizeof config_data->svc_monitor_mac, + ETH_ADDR_FMT, + ETH_ADDR_ARGS(config_data->svc_monitor_mac_ea)); + } else { + monitor_mac = NULL; + } + } + + struct smap *options = &config_data->nb_options; + smap_destroy(options); + smap_clone(options, &nb->options); + + smap_replace(options, "mac_prefix", mac_addr_prefix); + + if (!monitor_mac) { + eth_addr_random(&config_data->svc_monitor_mac_ea); + snprintf(config_data->svc_monitor_mac, + sizeof config_data->svc_monitor_mac, ETH_ADDR_FMT, + ETH_ADDR_ARGS(config_data->svc_monitor_mac_ea)); + smap_replace(options, "svc_monitor_mac", + config_data->svc_monitor_mac); + } + + char *max_tunid = xasprintf("%d", + get_ovn_max_dp_key_local(sbrec_chassis_table)); + smap_replace(options, "max_tunid", max_tunid); + free(max_tunid); + + char *ovn_internal_version = ovn_get_internal_version(); + if (strcmp(ovn_internal_version, + smap_get_def(options, "northd_internal_version", ""))) { + smap_replace(options, "northd_internal_version", + ovn_internal_version); + config_data->ovn_internal_version_changed = true; + } else { + config_data->ovn_internal_version_changed = false; + } + + free(ovn_internal_version); + + if (!smap_equal(&nb->options, options)) { + nbrec_nb_global_verify_options(nb); + nbrec_nb_global_set_options(nb, options); + } + + if (smap_get_bool(&nb->options, "ignore_chassis_features", false)) { + northd_enable_all_features(config_data); + } else { + build_chassis_features(sbrec_chassis_table, &config_data->features); + } + + init_debug_config(nb); + + const struct sbrec_sb_global *sb = + sbrec_sb_global_table_first(sb_global_table); + if (!sb) { + sb = sbrec_sb_global_insert(eng_ctx->ovnsb_idl_txn); + } + if (nb->ipsec != sb->ipsec) { + sbrec_sb_global_set_ipsec(sb, nb->ipsec); + } + + /* Set up SB_Global (depends on chassis features). */ + update_sb_config_options_to_sbrec(config_data, sb); + + engine_set_node_state(node, EN_UPDATED); +} + +void en_global_config_cleanup(void *data OVS_UNUSED) +{ + struct ed_type_global_config *config_data = data; + smap_destroy(&config_data->nb_options); + smap_destroy(&config_data->sb_options); + destroy_debug_config(); +} + +void +en_global_config_clear_tracked_data(void *data) +{ + struct ed_type_global_config *config_data = data; + config_data->tracked = false; + config_data->tracked_data.nb_options_changed = false; + config_data->tracked_data.chassis_features_changed = false; +} + +bool +global_config_nb_global_handler(struct engine_node *node, void *data) +{ + const struct nbrec_nb_global_table *nb_global_table = + EN_OVSDB_GET(engine_get_input("NB_nb_global", node)); + const struct sbrec_sb_global_table *sb_global_table = + EN_OVSDB_GET(engine_get_input("SB_sb_global", node)); + + const struct nbrec_nb_global *nb = + nbrec_nb_global_table_first(nb_global_table); + if (!nb) { + return false; + } + + const struct sbrec_sb_global *sb = + sbrec_sb_global_table_first(sb_global_table); + if (!sb) { + return false; + } + + /* We are only interested in ipsec and options column. */ + bool changes_relevant = false; + if (nbrec_nb_global_is_updated(nb, NBREC_NB_GLOBAL_COL_IPSEC) + || nbrec_nb_global_is_updated(nb, NBREC_NB_GLOBAL_COL_OPTIONS)) { + changes_relevant = true; + } + + if (!changes_relevant) { + return true; + } + + const struct engine_context *eng_ctx = engine_get_context(); + if (!eng_ctx->ovnsb_idl_txn) { + return false; + } + + if (nb->ipsec != sb->ipsec) { + sbrec_sb_global_set_ipsec(sb, nb->ipsec); + } + + struct ed_type_global_config *config_data = data; + config_data->tracked = true; + + if (smap_equal(&nb->options, &config_data->nb_options)) { + return true; + } + + /* Return false if an option is out of sync and requires updating the + * NB config. (Like svc_monitor_mac, max_tunid and mac_prefix). */ + /* Check if svc_monitor_mac has changed or not. */ + if (config_out_of_sync(&nb->options, &config_data->nb_options, + "svc_monitor_mac", true)) { + return false; + } + + /* Check if max_tunid has changed or not. */ + if (config_out_of_sync(&nb->options, &config_data->nb_options, + "max_tunid", true)) { + return false; + } + + /* Check if mac_prefix has changed or not. */ + if (config_out_of_sync(&nb->options, &config_data->nb_options, + "mac_prefix", true)) { + return false; + } + + /* Check if ignore_chassis_features has changed or not. */ + if (config_out_of_sync(&nb->options, &config_data->nb_options, + "ignore_chassis_features", false)) { + return false; + } + + /* Check if northd_internal_version has changed or not. */ + if (config_out_of_sync(&nb->options, &config_data->nb_options, + "northd_internal_version", false)) { + return false; + } + + if (check_nb_options_out_of_sync(nb, config_data)) { + config_data->tracked_data.nb_options_changed = true; + } + + smap_destroy(&config_data->nb_options); + smap_clone(&config_data->nb_options, &nb->options); + + update_sb_config_options_to_sbrec(config_data, sb); + + engine_set_node_state(node, EN_UPDATED); + return true; +} + +bool +global_config_sb_global_handler(struct engine_node *node, void *data) +{ + const struct sbrec_sb_global_table *sb_global_table = + EN_OVSDB_GET(engine_get_input("SB_sb_global", node)); + + const struct sbrec_sb_global *sb = + sbrec_sb_global_table_first(sb_global_table); + if (!sb) { + return false; + } + + struct ed_type_global_config *config_data = data; + + if (!smap_equal(&sb->options, &config_data->sb_options)) { + return false; + } + + /* No need to update the engine node. */ + return true; +} + +bool +global_config_sb_chassis_handler(struct engine_node *node, void *data) +{ + struct ed_type_global_config *config_data = data; + + const struct sbrec_chassis_table *sbrec_chassis_table = + EN_OVSDB_GET(engine_get_input("SB_chassis", node)); + const struct sbrec_chassis *chassis; + + SBREC_CHASSIS_TABLE_FOR_EACH_TRACKED (chassis, sbrec_chassis_table) { + if (sbrec_chassis_is_new(chassis) + || sbrec_chassis_is_deleted(chassis) + || sbrec_chassis_is_updated(chassis, + SBREC_CHASSIS_COL_ENCAPS)) { + return false; + } + + for (size_t i = 0; i < chassis->n_encaps; i++) { + if (sbrec_encap_row_get_seqno(chassis->encaps[i], + OVSDB_IDL_CHANGE_MODIFY) > 0) { + return false; + } + } + } + + if (smap_get_bool(&config_data->nb_options, "ignore_chassis_features", + false)) { + return true; + } + + bool reevaluate_chassis_features = false; + + /* Check and evaluate chassis features. */ + SBREC_CHASSIS_TABLE_FOR_EACH_TRACKED (chassis, sbrec_chassis_table) { + if (sbrec_chassis_is_updated(chassis, + SBREC_CHASSIS_COL_OTHER_CONFIG)) { + reevaluate_chassis_features = true; + break; + } + } + + if (!reevaluate_chassis_features) { + return true; + } + + struct chassis_features present_features = config_data->features; + + /* Enable all features before calling build_chassis_features() as + * build_chassis_features() only sets the feature flags to false. */ + northd_enable_all_features(config_data); + build_chassis_features(sbrec_chassis_table, &config_data->features); + + if (chassis_features_changed(&present_features, &config_data->features)) { + config_data->tracked_data.chassis_features_changed = true; + config_data->tracked = true; + engine_set_node_state(node, EN_UPDATED); + } + + return true; +} + +/* generic global config handler for any engine node which has global_config + * has an input node . */ +bool +node_global_config_handler(struct engine_node *node, void *data OVS_UNUSED) +{ + struct ed_type_global_config *global_config = + engine_get_input_data("global_config", node); + + if (!global_config->tracked + || global_config->tracked_data.chassis_features_changed + || global_config->tracked_data.nb_options_changed) { + return false; + } + + return true; +} + +/* static functions. */ +static void +northd_enable_all_features(struct ed_type_global_config *data) +{ + data->features = (struct chassis_features) { + .ct_no_masked_label = true, + .mac_binding_timestamp = true, + .ct_lb_related = true, + .fdb_timestamp = true, + .ls_dpg_column = true, + }; +} + +static void +build_chassis_features(const struct sbrec_chassis_table *sbrec_chassis_table, + struct chassis_features *chassis_features) +{ + const struct sbrec_chassis *chassis; + + SBREC_CHASSIS_TABLE_FOR_EACH (chassis, sbrec_chassis_table) { + /* Only consider local AZ chassis. Remote ones don't install + * flows generated by the local northd. + */ + if (smap_get_bool(&chassis->other_config, "is-remote", false)) { + continue; + } + + bool ct_no_masked_label = + smap_get_bool(&chassis->other_config, + OVN_FEATURE_CT_NO_MASKED_LABEL, + false); + if (!ct_no_masked_label && chassis_features->ct_no_masked_label) { + chassis_features->ct_no_masked_label = false; + } + + bool mac_binding_timestamp = + smap_get_bool(&chassis->other_config, + OVN_FEATURE_MAC_BINDING_TIMESTAMP, + false); + if (!mac_binding_timestamp && + chassis_features->mac_binding_timestamp) { + chassis_features->mac_binding_timestamp = false; + } + + bool ct_lb_related = + smap_get_bool(&chassis->other_config, + OVN_FEATURE_CT_LB_RELATED, + false); + if (!ct_lb_related && + chassis_features->ct_lb_related) { + chassis_features->ct_lb_related = false; + } + + bool fdb_timestamp = + smap_get_bool(&chassis->other_config, + OVN_FEATURE_FDB_TIMESTAMP, + false); + if (!fdb_timestamp && + chassis_features->fdb_timestamp) { + chassis_features->fdb_timestamp = false; + } + + bool ls_dpg_column = + smap_get_bool(&chassis->other_config, + OVN_FEATURE_LS_DPG_COLUMN, + false); + if (!ls_dpg_column && + chassis_features->ls_dpg_column) { + chassis_features->ls_dpg_column = false; + } + } +} + +static bool +config_out_of_sync(const struct smap *config, const struct smap *saved_config, + const char *key, bool must_be_present) +{ + const char *value = smap_get(config, key); + if (!value && must_be_present) { + return true; + } + + const char *saved_value = smap_get(saved_config, key); + if (!saved_value && must_be_present) { + return true; + } + + if (!value && !saved_value) { + return false; + } + + if ((!value && saved_value) || (value && !saved_value)) { + return true; + } + + return strcmp(value, saved_value); +} + +static bool +check_nb_options_out_of_sync(const struct nbrec_nb_global *nb, + struct ed_type_global_config *config_data) +{ + if (config_out_of_sync(&nb->options, &config_data->nb_options, + "mac_binding_removal_limit", false)) { + return true; + } + + if (config_out_of_sync(&nb->options, &config_data->nb_options, + "fdb_removal_limit", false)) { + return true; + } + + if (config_out_of_sync(&nb->options, &config_data->nb_options, + "controller_event", false)) { + return true; + } + + if (config_out_of_sync(&nb->options, &config_data->nb_options, + "ignore_lsp_down", false)) { + return true; + } + + if (config_out_of_sync(&nb->options, &config_data->nb_options, + "use_ct_inv_match", false)) { + return true; + } + + if (config_out_of_sync(&nb->options, &config_data->nb_options, + "default_acl_drop", false)) { + return true; + } + + if (config_out_of_sync(&nb->options, &config_data->nb_options, + "debug_drop_domain_id", false)) { + init_debug_config(nb); + return true; + } + + if (config_out_of_sync(&nb->options, &config_data->nb_options, + "debug_drop_collector_set", false)) { + init_debug_config(nb); + return true; + } + + if (config_out_of_sync(&nb->options, &config_data->nb_options, + "use_common_zone", false)) { + return true; + } + + if (config_out_of_sync(&nb->options, &config_data->nb_options, + "install_ls_lb_from_router", false)) { + return true; + } + + return false; +} + +static void +update_sb_config_options_to_sbrec(struct ed_type_global_config *config_data, + const struct sbrec_sb_global *sb) +{ + struct smap *options = &config_data->sb_options; + + smap_destroy(options); + smap_clone(options, &config_data->nb_options); + + /* Inform ovn-controllers whether LB flows will use ct_mark (i.e., only + * if all chassis support it). If not explicitly present in the database + * the default value to be used for this option is 'true'. + */ + if (!config_data->features.ct_no_masked_label) { + smap_replace(options, "lb_hairpin_use_ct_mark", "false"); + } else { + smap_remove(options, "lb_hairpin_use_ct_mark"); + } + + /* Hackaround SB_global.options overwrite by NB_Global.options for + * 'sbctl_probe_interval' option. + */ + const char *sip = smap_get(&sb->options, "sbctl_probe_interval"); + if (sip) { + smap_replace(options, "sbctl_probe_interval", sip); + } + + if (!smap_equal(&sb->options, options)) { + sbrec_sb_global_set_options(sb, options); + } +} + +static bool +chassis_features_changed(const struct chassis_features *present, + const struct chassis_features *updated) +{ + if (present->ct_no_masked_label != updated->ct_no_masked_label) { + return true; + } + + if (present->mac_binding_timestamp != updated->mac_binding_timestamp) { + return true; + } + + if (present->ct_lb_related != updated->ct_lb_related) { + return true; + } + + if (present->fdb_timestamp != updated->fdb_timestamp) { + return true; + } + + if (present->ls_dpg_column != updated->ls_dpg_column) { + return true; + } + + return false; +} diff --git a/northd/en-global-config.h b/northd/en-global-config.h new file mode 100644 index 0000000000..436bc7fa35 --- /dev/null +++ b/northd/en-global-config.h @@ -0,0 +1,65 @@ +#ifndef EN_GLOBAL_CONFIG_H +#define EN_GLOBAL_CONFIG_H 1 + +#include + +/* OVS includes. */ +#include "lib/packets.h" +#include "lib/smap.h" + +/* OVN includes. */ +#include "lib/inc-proc-eng.h" + +struct nbrec_nb_global; +struct sbrec_sb_global; + +struct chassis_features { + bool ct_no_masked_label; + bool mac_binding_timestamp; + bool ct_lb_related; + bool fdb_timestamp; + bool ls_dpg_column; +}; + +struct global_config_tracked_data { + bool nb_options_changed; + bool chassis_features_changed; +}; + +/* struct which maintains the data of the engine node global_config. */ +struct ed_type_global_config { + struct smap nb_options; + struct smap sb_options; + const struct nbrec_nb_global *nb_global; + const struct sbrec_sb_global *sb_global; + + /* MAC allocated for service monitor usage. Just one mac is allocated + * for this purpose and ovn-controller's on each chassis will make use + * of this mac when sending out the packets to monitor the services + * defined in Service_Monitor Southbound table. Since these packets + * are locally handled, having just one mac is good enough. */ + char svc_monitor_mac[ETH_ADDR_STRLEN + 1]; + struct eth_addr svc_monitor_mac_ea; + + struct chassis_features features; + + bool ovn_internal_version_changed; + + bool tracked; + struct global_config_tracked_data tracked_data; +}; + +void *en_global_config_init(struct engine_node *, struct engine_arg *); +void en_global_config_run(struct engine_node *, void *data); +void en_global_config_cleanup(void *data); +void en_global_config_clear_tracked_data(void *data); + +bool global_config_nb_global_handler(struct engine_node *, void *data); +bool global_config_sb_global_handler(struct engine_node *, void *data); +bool global_config_sb_chassis_handler(struct engine_node *, void *data); + +/* generic global config handler for any engine node which has global_config + * has an input node . */ +bool node_global_config_handler(struct engine_node *, void *data); + +#endif /* EN_GLOBAL_CONFIG_H */ diff --git a/northd/en-lflow.c b/northd/en-lflow.c index f51ee0d2b7..21e8131562 100644 --- a/northd/en-lflow.c +++ b/northd/en-lflow.c @@ -18,6 +18,7 @@ #include #include +#include "en-global-config.h" #include "en-lflow.h" #include "en-lr-nat.h" #include "en-lr-stateful.h" @@ -77,10 +78,14 @@ lflow_get_input_data(struct engine_node *node, lflow_input->meter_groups = &sync_meters_data->meter_groups; lflow_input->lb_datapaths_map = &northd_data->lb_datapaths_map; lflow_input->svc_monitor_map = &northd_data->svc_monitor_map; - lflow_input->features = &northd_data->features; - lflow_input->ovn_internal_version_changed = - northd_data->ovn_internal_version_changed; lflow_input->bfd_connections = NULL; + + struct ed_type_global_config *global_config = + engine_get_input_data("global_config", node); + lflow_input->features = &global_config->features; + lflow_input->ovn_internal_version_changed = + global_config->ovn_internal_version_changed; + lflow_input->svc_monitor_mac = global_config->svc_monitor_mac; } void en_lflow_run(struct engine_node *node, void *data) diff --git a/northd/en-northd.c b/northd/en-northd.c index 5143603f39..4479b4aff2 100644 --- a/northd/en-northd.c +++ b/northd/en-northd.c @@ -19,6 +19,7 @@ #include #include "coverage.h" +#include "en-global-config.h" #include "en-northd.h" #include "en-lb-data.h" #include "lib/inc-proc-eng.h" @@ -65,8 +66,6 @@ northd_get_input_data(struct engine_node *node, engine_get_input("SB_fdb", node), "sbrec_fdb_by_dp_and_port"); - input_data->nbrec_nb_global_table = - EN_OVSDB_GET(engine_get_input("NB_nb_global", node)); input_data->nbrec_logical_switch_table = EN_OVSDB_GET(engine_get_input("NB_logical_switch", node)); input_data->nbrec_logical_router_table = @@ -78,8 +77,6 @@ northd_get_input_data(struct engine_node *node, input_data->nbrec_mirror_table = EN_OVSDB_GET(engine_get_input("NB_mirror", node)); - input_data->sbrec_sb_global_table = - EN_OVSDB_GET(engine_get_input("SB_sb_global", node)); input_data->sbrec_datapath_binding_table = EN_OVSDB_GET(engine_get_input("SB_datapath_binding", node)); input_data->sbrec_port_binding_table = @@ -109,6 +106,14 @@ northd_get_input_data(struct engine_node *node, engine_get_input_data("lb_data", node); input_data->lbs = &lb_data->lbs; input_data->lbgrps = &lb_data->lbgrps; + + struct ed_type_global_config *global_config = + engine_get_input_data("global_config", node); + input_data->nb_options = &global_config->nb_options; + input_data->sb_options = &global_config->sb_options; + input_data->svc_monitor_mac = global_config->svc_monitor_mac; + input_data->svc_monitor_mac_ea = global_config->svc_monitor_mac_ea; + input_data->features = &global_config->features; } void @@ -129,31 +134,6 @@ en_northd_run(struct engine_node *node, void *data) eng_ctx->ovnsb_idl_txn); stopwatch_stop(OVNNB_DB_RUN_STOPWATCH_NAME, time_msec()); engine_set_node_state(node, EN_UPDATED); - -} - -bool -northd_nb_nb_global_handler(struct engine_node *node, - void *data OVS_UNUSED) -{ - const struct nbrec_nb_global_table *nb_global_table - = EN_OVSDB_GET(engine_get_input("NB_nb_global", node)); - - const struct nbrec_nb_global *nb = - nbrec_nb_global_table_first(nb_global_table); - - if (!nb) { - static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 1); - VLOG_WARN_RL(&rl, "NB_Global is updated but has no record."); - return false; - } - - /* We care about the 'options' and 'ipsec' columns only. */ - if (nbrec_nb_global_is_updated(nb, NBREC_NB_GLOBAL_COL_OPTIONS) || - nbrec_nb_global_is_updated(nb, NBREC_NB_GLOBAL_COL_IPSEC)) { - return false; - } - return true; } bool @@ -242,6 +222,20 @@ northd_lb_data_handler(struct engine_node *node, void *data) return true; } +bool +northd_global_config_handler(struct engine_node *node, void *data OVS_UNUSED) +{ + struct ed_type_global_config *global_config = + engine_get_input_data("global_config", node); + + if (!global_config->tracked + || global_config->tracked_data.nb_options_changed) { + return false; + } + + return true; +} + void *en_northd_init(struct engine_node *node OVS_UNUSED, struct engine_arg *arg OVS_UNUSED) diff --git a/northd/en-northd.h b/northd/en-northd.h index 5a88871760..9b7bda32aa 100644 --- a/northd/en-northd.h +++ b/northd/en-northd.h @@ -14,7 +14,7 @@ void *en_northd_init(struct engine_node *node OVS_UNUSED, struct engine_arg *arg); void en_northd_cleanup(void *data); void en_northd_clear_tracked_data(void *data); -bool northd_nb_nb_global_handler(struct engine_node *, void *data OVS_UNUSED); +bool northd_global_config_handler(struct engine_node *, void *data OVS_UNUSED); bool northd_nb_logical_switch_handler(struct engine_node *, void *data); bool northd_nb_logical_router_handler(struct engine_node *, void *data); bool northd_sb_port_binding_handler(struct engine_node *, void *data); diff --git a/northd/en-sync-sb.c b/northd/en-sync-sb.c index 80f3621bb9..0c8428c61d 100644 --- a/northd/en-sync-sb.c +++ b/northd/en-sync-sb.c @@ -24,6 +24,7 @@ /* OVN includes. */ #include "en-lr-nat.h" +#include "en-global-config.h" #include "en-lr-stateful.h" #include "en-sync-sb.h" #include "lb.h" @@ -46,7 +47,8 @@ static void sync_addr_sets(struct ovsdb_idl_txn *ovnsb_txn, const struct nbrec_address_set_table *, const struct nbrec_port_group_table *, const struct sbrec_address_set_table *, - const struct lr_stateful_table *); + const struct lr_stateful_table *, + const char *svc_monitor_macp); static const struct sbrec_address_set *sb_address_set_lookup_by_name( struct ovsdb_idl_index *, const char *name); static void update_sb_addr_set(struct sorted_array *, @@ -128,9 +130,12 @@ en_sync_to_sb_addr_set_run(struct engine_node *node, void *data OVS_UNUSED) const struct engine_context *eng_ctx = engine_get_context(); const struct ed_type_lr_stateful *lr_stateful_data = engine_get_input_data("lr_stateful", node); + struct ed_type_global_config *global_config = + engine_get_input_data("global_config", node); sync_addr_sets(eng_ctx->ovnsb_idl_txn, nb_address_set_table, nb_port_group_table, sb_address_set_table, - &lr_stateful_data->table); + &lr_stateful_data->table, + global_config->svc_monitor_mac); engine_set_node_state(node, EN_UPDATED); } @@ -282,6 +287,8 @@ en_sync_to_sb_lb_run(struct engine_node *node, void *data_) EN_OVSDB_GET(engine_get_input("SB_load_balancer", node)); const struct sbrec_logical_dp_group_table *sb_dpgrp_table = EN_OVSDB_GET(engine_get_input("SB_logical_dp_group", node)); + struct ed_type_global_config *global_config = + engine_get_input_data("global_config", node); const struct engine_context *eng_ctx = engine_get_context(); struct ed_type_sync_to_sb_lb_data *data = data_; @@ -292,7 +299,7 @@ en_sync_to_sb_lb_run(struct engine_node *node, void *data_) &northd_data->lb_datapaths_map, &northd_data->ls_datapaths, &northd_data->lr_datapaths, - &northd_data->features); + &global_config->features); engine_set_node_state(node, EN_UPDATED); } @@ -327,12 +334,14 @@ sync_to_sb_lb_northd_handler(struct engine_node *node, void *data_) EN_OVSDB_GET(engine_get_input("SB_logical_dp_group", node)); const struct sbrec_load_balancer_table *sb_lb_table = EN_OVSDB_GET(engine_get_input("SB_load_balancer", node)); + struct ed_type_global_config *global_config = + engine_get_input_data("global_config", node); struct ed_type_sync_to_sb_lb_data *data = data_; if (!sync_changed_lbs(&data->sb_lbs, eng_ctx->ovnsb_idl_txn, sb_lb_table, sb_dpgrp_table, &nd->trk_data.trk_lbs, &nd->ls_datapaths, &nd->lr_datapaths, - &nd->features)) { + &global_config->features)) { return false; } @@ -457,7 +466,8 @@ sync_addr_sets(struct ovsdb_idl_txn *ovnsb_txn, const struct nbrec_address_set_table *nb_address_set_table, const struct nbrec_port_group_table *nb_port_group_table, const struct sbrec_address_set_table *sb_address_set_table, - const struct lr_stateful_table *lr_statefuls) + const struct lr_stateful_table *lr_statefuls, + const char *svc_monitor_macp) { struct shash sb_address_sets = SHASH_INITIALIZER(&sb_address_sets); @@ -468,8 +478,10 @@ sync_addr_sets(struct ovsdb_idl_txn *ovnsb_txn, } /* Service monitor MAC. */ - const char *svc_monitor_macp = northd_get_svc_monitor_mac(); - struct sorted_array svc = sorted_array_create(&svc_monitor_macp, 1, false); + struct sorted_array svc = { + .arr = &svc_monitor_macp, + .n = 1, + }; sync_addr_set(ovnsb_txn, "svc_monitor_mac", &svc, &sb_address_sets); sorted_array_destroy(&svc); diff --git a/northd/inc-proc-northd.c b/northd/inc-proc-northd.c index d215c7792b..e1073812c8 100644 --- a/northd/inc-proc-northd.c +++ b/northd/inc-proc-northd.c @@ -30,6 +30,7 @@ #include "openvswitch/poll-loop.h" #include "openvswitch/vlog.h" #include "inc-proc-northd.h" +#include "en-global-config.h" #include "en-lb-data.h" #include "en-lr-stateful.h" #include "en-lr-nat.h" @@ -149,6 +150,7 @@ static ENGINE_NODE(fdb_aging, "fdb_aging"); static ENGINE_NODE(fdb_aging_waker, "fdb_aging_waker"); static ENGINE_NODE(sync_to_sb_lb, "sync_to_sb_lb"); static ENGINE_NODE(sync_to_sb_pb, "sync_to_sb_pb"); +static ENGINE_NODE_WITH_CLEAR_TRACK_DATA(global_config, "global_config"); static ENGINE_NODE_WITH_CLEAR_TRACK_DATA(lb_data, "lb_data"); static ENGINE_NODE_WITH_CLEAR_TRACK_DATA(lr_nat, "lr_nat"); static ENGINE_NODE_WITH_CLEAR_TRACK_DATA(lr_stateful, "lr_stateful"); @@ -168,11 +170,17 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, engine_add_input(&en_lb_data, &en_nb_logical_router, lb_data_logical_router_handler); + engine_add_input(&en_global_config, &en_nb_nb_global, + global_config_nb_global_handler); + engine_add_input(&en_global_config, &en_sb_sb_global, + global_config_sb_global_handler); + engine_add_input(&en_global_config, &en_sb_chassis, + global_config_sb_chassis_handler); + engine_add_input(&en_northd, &en_nb_mirror, NULL); engine_add_input(&en_northd, &en_nb_static_mac_binding, NULL); engine_add_input(&en_northd, &en_nb_chassis_template_var, NULL); - engine_add_input(&en_northd, &en_sb_sb_global, NULL); engine_add_input(&en_northd, &en_sb_chassis, NULL); engine_add_input(&en_northd, &en_sb_mirror, NULL); engine_add_input(&en_northd, &en_sb_meter, NULL); @@ -184,6 +192,8 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, engine_add_input(&en_northd, &en_sb_fdb, NULL); engine_add_input(&en_northd, &en_sb_static_mac_binding, NULL); engine_add_input(&en_northd, &en_sb_chassis_template_var, NULL); + engine_add_input(&en_northd, &en_global_config, + northd_global_config_handler); /* northd engine node uses the sb mac binding table to * cleanup mac binding entries for deleted logical ports @@ -198,8 +208,6 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, engine_add_input(&en_northd, &en_sb_port_binding, northd_sb_port_binding_handler); - engine_add_input(&en_northd, &en_nb_nb_global, - northd_nb_nb_global_handler); engine_add_input(&en_northd, &en_nb_logical_switch, northd_nb_logical_switch_handler); engine_add_input(&en_northd, &en_nb_logical_router, @@ -217,15 +225,17 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, engine_add_input(&en_ls_stateful, &en_port_group, ls_stateful_port_group_handler); - engine_add_input(&en_mac_binding_aging, &en_nb_nb_global, NULL); engine_add_input(&en_mac_binding_aging, &en_sb_mac_binding, NULL); engine_add_input(&en_mac_binding_aging, &en_northd, NULL); engine_add_input(&en_mac_binding_aging, &en_mac_binding_aging_waker, NULL); + engine_add_input(&en_mac_binding_aging, &en_global_config, + node_global_config_handler); - engine_add_input(&en_fdb_aging, &en_nb_nb_global, NULL); engine_add_input(&en_fdb_aging, &en_sb_fdb, NULL); engine_add_input(&en_fdb_aging, &en_northd, NULL); engine_add_input(&en_fdb_aging, &en_fdb_aging_waker, NULL); + engine_add_input(&en_fdb_aging, &en_global_config, + node_global_config_handler); engine_add_input(&en_sync_meters, &en_nb_acl, NULL); engine_add_input(&en_sync_meters, &en_nb_meter, NULL); @@ -239,18 +249,22 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, engine_add_input(&en_lflow, &en_sb_multicast_group, NULL); engine_add_input(&en_lflow, &en_sb_igmp_group, NULL); engine_add_input(&en_lflow, &en_sb_logical_dp_group, NULL); + engine_add_input(&en_lflow, &en_global_config, + node_global_config_handler); engine_add_input(&en_lflow, &en_northd, lflow_northd_handler); engine_add_input(&en_lflow, &en_port_group, lflow_port_group_handler); engine_add_input(&en_lflow, &en_lr_stateful, lflow_lr_stateful_handler); engine_add_input(&en_lflow, &en_ls_stateful, lflow_ls_stateful_handler); + engine_add_input(&en_sync_to_sb_addr_set, &en_northd, NULL); + engine_add_input(&en_sync_to_sb_addr_set, &en_lr_stateful, NULL); + engine_add_input(&en_sync_to_sb_addr_set, &en_sb_address_set, NULL); engine_add_input(&en_sync_to_sb_addr_set, &en_nb_address_set, sync_to_sb_addr_set_nb_address_set_handler); engine_add_input(&en_sync_to_sb_addr_set, &en_nb_port_group, sync_to_sb_addr_set_nb_port_group_handler); - engine_add_input(&en_sync_to_sb_addr_set, &en_northd, NULL); - engine_add_input(&en_sync_to_sb_addr_set, &en_lr_stateful, NULL); - engine_add_input(&en_sync_to_sb_addr_set, &en_sb_address_set, NULL); + engine_add_input(&en_sync_to_sb_addr_set, &en_global_config, + node_global_config_handler); engine_add_input(&en_port_group, &en_nb_port_group, port_group_nb_port_group_handler); @@ -260,6 +274,8 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, * table too (because of the explicit dependency in the schema). */ engine_add_input(&en_port_group, &en_northd, engine_noop_handler); + engine_add_input(&en_sync_to_sb_lb, &en_global_config, + node_global_config_handler); engine_add_input(&en_sync_to_sb_lb, &en_northd, sync_to_sb_lb_northd_handler); engine_add_input(&en_sync_to_sb_lb, &en_sb_load_balancer, @@ -365,11 +381,11 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, "sbrec_fdb_by_dp_and_port", sbrec_fdb_by_dp_and_port); - struct northd_data *northd_data = - engine_get_internal_data(&en_northd); + struct ed_type_global_config *global_config = + engine_get_internal_data(&en_global_config); unixctl_command_register("debug/chassis-features-list", "", 0, 0, chassis_features_list, - &northd_data->features); + &global_config->features); } /* Returns true if the incremental processing ended up updating nodes. */ diff --git a/northd/northd.c b/northd/northd.c index d21a070fdd..159409ea48 100644 --- a/northd/northd.c +++ b/northd/northd.c @@ -45,6 +45,7 @@ #include "lflow-mgr.h" #include "memory.h" #include "northd.h" +#include "en-global-config.h" #include "en-lb-data.h" #include "en-lr-nat.h" #include "en-lr-stateful.h" @@ -79,14 +80,6 @@ static bool install_ls_lb_from_router; /* Use common zone for SNAT and DNAT if this option is set to "true". */ static bool use_common_zone = false; -/* MAC allocated for service monitor usage. Just one mac is allocatedg5534 - * for this purpose and ovn-controller's on each chassis will make use - * of this mac when sending out the packets to monitor the services - * defined in Service_Monitor Southbound table. Since these packets - * all locally handled, having just one mac is good enough. */ -static char svc_monitor_mac[ETH_ADDR_STRLEN + 1]; -static struct eth_addr svc_monitor_mac_ea; - /* If this option is 'true' northd will make use of ct.inv match fields. * Otherwise, it will avoid using it. The default is true. */ static bool use_ct_inv_match = true; @@ -297,66 +290,6 @@ ovn_stage_to_datapath_type(enum ovn_stage stage) } } -static void -build_chassis_features(const struct sbrec_chassis_table *sbrec_chassis_table, - struct chassis_features *chassis_features) -{ - const struct sbrec_chassis *chassis; - - SBREC_CHASSIS_TABLE_FOR_EACH (chassis, sbrec_chassis_table) { - /* Only consider local AZ chassis. Remote ones don't install - * flows generated by the local northd. - */ - if (smap_get_bool(&chassis->other_config, "is-remote", false)) { - continue; - } - - bool ct_no_masked_label = - smap_get_bool(&chassis->other_config, - OVN_FEATURE_CT_NO_MASKED_LABEL, - false); - if (!ct_no_masked_label && chassis_features->ct_no_masked_label) { - chassis_features->ct_no_masked_label = false; - } - - bool mac_binding_timestamp = - smap_get_bool(&chassis->other_config, - OVN_FEATURE_MAC_BINDING_TIMESTAMP, - false); - if (!mac_binding_timestamp && - chassis_features->mac_binding_timestamp) { - chassis_features->mac_binding_timestamp = false; - } - - bool ct_lb_related = - smap_get_bool(&chassis->other_config, - OVN_FEATURE_CT_LB_RELATED, - false); - if (!ct_lb_related && - chassis_features->ct_lb_related) { - chassis_features->ct_lb_related = false; - } - - bool fdb_timestamp = - smap_get_bool(&chassis->other_config, - OVN_FEATURE_FDB_TIMESTAMP, - false); - if (!fdb_timestamp && - chassis_features->fdb_timestamp) { - chassis_features->fdb_timestamp = false; - } - - bool ls_dpg_column = - smap_get_bool(&chassis->other_config, - OVN_FEATURE_LS_DPG_COLUMN, - false); - if (!ls_dpg_column && - chassis_features->ls_dpg_column) { - chassis_features->ls_dpg_column = false; - } - } -} - static uint32_t allocate_queueid(unsigned long *queue_id_bitmap) { @@ -954,7 +887,7 @@ is_vxlan_mode(const struct sbrec_chassis_table *sbrec_chassis_table) return false; } -static uint32_t +uint32_t get_ovn_max_dp_key_local(const struct sbrec_chassis_table *sbrec_chassis_table) { if (is_vxlan_mode(sbrec_chassis_table)) { @@ -3367,6 +3300,8 @@ create_or_get_service_mon(struct ovsdb_idl_txn *ovnsb_txn, static void ovn_lb_svc_create(struct ovsdb_idl_txn *ovnsb_txn, const struct ovn_northd_lb *lb, + const char *svc_monitor_mac, + const struct eth_addr *svc_monitor_mac_ea, struct hmap *monitor_map, struct hmap *ls_ports, struct sset *svc_monitor_lsps) { @@ -3412,7 +3347,7 @@ ovn_lb_svc_create(struct ovsdb_idl_txn *ovnsb_txn, struct eth_addr ea; if (!mon_info->sbrec_mon->src_mac || !eth_addr_from_string(mon_info->sbrec_mon->src_mac, &ea) || - !eth_addr_equals(ea, svc_monitor_mac_ea)) { + !eth_addr_equals(ea, *svc_monitor_mac_ea)) { sbrec_service_monitor_set_src_mac(mon_info->sbrec_mon, svc_monitor_mac); } @@ -3624,6 +3559,8 @@ static void build_lb_svcs( struct ovsdb_idl_txn *ovnsb_txn, const struct sbrec_service_monitor_table *sbrec_service_monitor_table, + const char *svc_monitor_mac, + const struct eth_addr *svc_monitor_mac_ea, struct hmap *ls_ports, struct hmap *lb_dps_map, struct sset *svc_monitor_lsps, struct hmap *svc_monitor_map) @@ -3642,7 +3579,8 @@ build_lb_svcs( struct ovn_lb_datapaths *lb_dps; HMAP_FOR_EACH (lb_dps, hmap_node, lb_dps_map) { - ovn_lb_svc_create(ovnsb_txn, lb_dps->lb, svc_monitor_map, ls_ports, + ovn_lb_svc_create(ovnsb_txn, lb_dps->lb, svc_monitor_mac, + svc_monitor_mac_ea, svc_monitor_map, ls_ports, svc_monitor_lsps); } @@ -3734,13 +3672,16 @@ static void build_lb_port_related_data( struct ovsdb_idl_txn *ovnsb_txn, const struct sbrec_service_monitor_table *sbrec_service_monitor_table, + const char *svc_monitor_mac, + const struct eth_addr *svc_monitor_mac_ea, struct ovn_datapaths *lr_datapaths, struct hmap *ls_ports, struct hmap *lb_dps_map, struct hmap *lb_group_dps_map, struct sset *svc_monitor_lsps, struct hmap *svc_monitor_map) { build_lrouter_lbs_check(lr_datapaths); - build_lb_svcs(ovnsb_txn, sbrec_service_monitor_table, ls_ports, lb_dps_map, + build_lb_svcs(ovnsb_txn, sbrec_service_monitor_table, svc_monitor_mac, + svc_monitor_mac_ea, ls_ports, lb_dps_map, svc_monitor_lsps, svc_monitor_map); build_lswitch_lbs_from_lrouter(lr_datapaths, lb_dps_map, lb_group_dps_map); } @@ -9069,6 +9010,7 @@ build_lswitch_arp_nd_responder_default(struct ovn_datapath *od, static void build_lswitch_arp_nd_service_monitor(const struct ovn_lb_datapaths *lb_dps, const struct hmap *ls_ports, + const char *svc_monitor_mac, struct lflow_table *lflows, struct ds *actions, struct ds *match) @@ -15475,6 +15417,7 @@ struct lswitch_flow_build_info { struct ds match; struct ds actions; size_t thread_lflow_counter; + const char *svc_monitor_mac; }; /* Helper function to combine all lflow generation which is iterated by @@ -15696,6 +15639,7 @@ build_lflows_thread(void *arg) } build_lswitch_arp_nd_service_monitor(lb_dps, lsi->ls_ports, + lsi->svc_monitor_mac, lsi->lflows, &lsi->match, &lsi->actions); @@ -15816,7 +15760,8 @@ build_lswitch_and_lrouter_flows( const struct hmap *lb_dps_map, const struct hmap *svc_monitor_map, const struct hmap *bfd_connections, - const struct chassis_features *features) + const struct chassis_features *features, + const char *svc_monitor_mac) { char *svc_check_match = xasprintf("eth.dst == %s", svc_monitor_mac); @@ -15849,6 +15794,7 @@ build_lswitch_and_lrouter_flows( lsiv[index].features = features; lsiv[index].svc_check_match = svc_check_match; lsiv[index].thread_lflow_counter = 0; + lsiv[index].svc_monitor_mac = svc_monitor_mac; ds_init(&lsiv[index].match); ds_init(&lsiv[index].actions); @@ -15888,6 +15834,7 @@ build_lswitch_and_lrouter_flows( .bfd_connections = bfd_connections, .features = features, .svc_check_match = svc_check_match, + .svc_monitor_mac = svc_monitor_mac, .match = DS_EMPTY_INITIALIZER, .actions = DS_EMPTY_INITIALIZER, }; @@ -15927,6 +15874,7 @@ build_lswitch_and_lrouter_flows( stopwatch_start(LFLOWS_LBS_STOPWATCH_NAME, time_msec()); HMAP_FOR_EACH (lb_dps, hmap_node, lb_dps_map) { build_lswitch_arp_nd_service_monitor(lb_dps, lsi.ls_ports, + lsi.svc_monitor_mac, lsi.lflows, &lsi.actions, &lsi.match); build_lrouter_defrag_flows_for_lb(lb_dps, lsi.lflows, @@ -16040,7 +15988,8 @@ void build_lflows(struct ovsdb_idl_txn *ovnsb_txn, input_data->lb_datapaths_map, input_data->svc_monitor_map, input_data->bfd_connections, - input_data->features); + input_data->features, + input_data->svc_monitor_mac); if (parallelization_state == STATE_INIT_HASH_SIZES) { parallelization_state = STATE_USE_PARALLELIZATION; @@ -16339,6 +16288,7 @@ lflow_handle_northd_lb_changes(struct ovsdb_idl_txn *ovnsb_txn, struct ds actions = DS_EMPTY_INITIALIZER; build_lswitch_arp_nd_service_monitor(lb_dps, lflow_input->ls_ports, + lflow_input->svc_monitor_mac, lflows, &actions, &match); build_lrouter_defrag_flows_for_lb(lb_dps, lflows, @@ -17105,18 +17055,6 @@ destroy_datapaths_and_ports(struct ovn_datapaths *ls_datapaths, ovn_datapaths_destroy(lr_datapaths); } -static void -northd_enable_all_features(struct northd_data *data) -{ - data->features = (struct chassis_features) { - .ct_no_masked_label = true, - .mac_binding_timestamp = true, - .ct_lb_related = true, - .fdb_timestamp = true, - .ls_dpg_column = true, - }; -} - void northd_init(struct northd_data *data) { @@ -17127,8 +17065,6 @@ northd_init(struct northd_data *data) hmap_init(&data->lb_datapaths_map); hmap_init(&data->lb_group_datapaths_map); ovs_list_init(&data->lr_list); - northd_enable_all_features(data); - data->ovn_internal_version_changed = false; sset_init(&data->svc_monitor_lsps); hmap_init(&data->svc_monitor_map); init_northd_tracked_data(data); @@ -17168,7 +17104,6 @@ northd_destroy(struct northd_data *data) destroy_datapaths_and_ports(&data->ls_datapaths, &data->lr_datapaths, &data->ls_ports, &data->lr_ports, &data->lr_list); - destroy_debug_config(); sset_destroy(&data->svc_monitor_lsps); destroy_northd_tracked_data(data); @@ -17185,83 +17120,22 @@ ovnnb_db_run(struct northd_input *input_data, } stopwatch_start(BUILD_LFLOWS_CTX_STOPWATCH_NAME, time_msec()); - /* Sync ipsec configuration. - * Copy nb_cfg from northbound to southbound database. - * Also set up to update sb_cfg once our southbound transaction commits. */ - const struct nbrec_nb_global *nb = nbrec_nb_global_table_first( - input_data->nbrec_nb_global_table); - if (!nb) { - nb = nbrec_nb_global_insert(ovnnb_txn); - } - - const char *mac_addr_prefix = set_mac_prefix(smap_get(&nb->options, - "mac_prefix")); - - const char *monitor_mac = smap_get(&nb->options, "svc_monitor_mac"); - if (monitor_mac) { - if (eth_addr_from_string(monitor_mac, &svc_monitor_mac_ea)) { - snprintf(svc_monitor_mac, sizeof svc_monitor_mac, - ETH_ADDR_FMT, ETH_ADDR_ARGS(svc_monitor_mac_ea)); - } else { - monitor_mac = NULL; - } - } - - struct smap options; - smap_clone(&options, &nb->options); - - smap_replace(&options, "mac_prefix", mac_addr_prefix); - - if (!monitor_mac) { - eth_addr_random(&svc_monitor_mac_ea); - snprintf(svc_monitor_mac, sizeof svc_monitor_mac, - ETH_ADDR_FMT, ETH_ADDR_ARGS(svc_monitor_mac_ea)); - smap_replace(&options, "svc_monitor_mac", svc_monitor_mac); - } - - char *max_tunid = xasprintf("%d", - get_ovn_max_dp_key_local(input_data->sbrec_chassis_table)); - smap_replace(&options, "max_tunid", max_tunid); - free(max_tunid); - - char *ovn_internal_version = ovn_get_internal_version(); - if (!strcmp(ovn_internal_version, - smap_get_def(&options, "northd_internal_version", ""))) { - data->ovn_internal_version_changed = false; - } else { - smap_replace(&options, "northd_internal_version", - ovn_internal_version); - } - free(ovn_internal_version); - - if (!smap_equal(&nb->options, &options)) { - nbrec_nb_global_verify_options(nb); - nbrec_nb_global_set_options(nb, &options); - } - - use_ct_inv_match = smap_get_bool(&nb->options, + use_ct_inv_match = smap_get_bool(input_data->nb_options, "use_ct_inv_match", true); /* deprecated, use --event instead */ - controller_event_en = smap_get_bool(&nb->options, + controller_event_en = smap_get_bool(input_data->nb_options, "controller_event", false); - check_lsp_is_up = !smap_get_bool(&nb->options, + check_lsp_is_up = !smap_get_bool(input_data->nb_options, "ignore_lsp_down", true); - default_acl_drop = smap_get_bool(&nb->options, "default_acl_drop", false); + default_acl_drop = smap_get_bool(input_data->nb_options, + "default_acl_drop", false); - install_ls_lb_from_router = smap_get_bool(&nb->options, + install_ls_lb_from_router = smap_get_bool(input_data->nb_options, "install_ls_lb_from_router", false); - use_common_zone = smap_get_bool(&nb->options, "use_common_zone", false); - - if (smap_get_bool(&nb->options, "ignore_chassis_features", false)) { - northd_enable_all_features(data); - } else { - build_chassis_features(input_data->sbrec_chassis_table, - &data->features); - } - - init_debug_config(nb); + use_common_zone = smap_get_bool(input_data->nb_options, "use_common_zone", + false); build_datapaths(ovnsb_txn, input_data->nbrec_logical_switch_table, @@ -17286,6 +17160,8 @@ ovnnb_db_run(struct northd_input *input_data, &data->ls_ports, &data->lr_ports); build_lb_port_related_data(ovnsb_txn, input_data->sbrec_service_monitor_table, + input_data->svc_monitor_mac, + &input_data->svc_monitor_mac_ea, &data->lr_datapaths, &data->ls_ports, &data->lb_datapaths_map, &data->lb_group_datapaths_map, @@ -17318,38 +17194,6 @@ ovnnb_db_run(struct northd_input *input_data, &data->ls_datapaths.datapaths); stopwatch_stop(CLEAR_LFLOWS_CTX_STOPWATCH_NAME, time_msec()); - /* Set up SB_Global (depends on chassis features). */ - const struct sbrec_sb_global *sb = sbrec_sb_global_table_first( - input_data->sbrec_sb_global_table); - if (!sb) { - sb = sbrec_sb_global_insert(ovnsb_txn); - } - if (nb->ipsec != sb->ipsec) { - sbrec_sb_global_set_ipsec(sb, nb->ipsec); - } - - /* Inform ovn-controllers whether LB flows will use ct_mark (i.e., only - * if all chassis support it). If not explicitly present in the database - * the default value to be used for this option is 'true'. - */ - if (!data->features.ct_no_masked_label) { - smap_replace(&options, "lb_hairpin_use_ct_mark", "false"); - } else { - smap_remove(&options, "lb_hairpin_use_ct_mark"); - } - - /* Hackaround SB_global.options overwrite by NB_Global.options for - * 'sbctl_probe_interval' option. - */ - const char *sip = smap_get(&sb->options, "sbctl_probe_interval"); - if (sip) { - smap_replace(&options, "sbctl_probe_interval", sip); - } - - if (!smap_equal(&sb->options, &options)) { - sbrec_sb_global_set_options(sb, &options); - } - smap_destroy(&options); } /* Stores the set of chassis which references an ha_chassis_group. @@ -17640,12 +17484,6 @@ ovnsb_db_run(struct ovsdb_idl_txn *ovnnb_txn, ovn_update_ipv6_prefix(lr_ports); } -const char * -northd_get_svc_monitor_mac(void) -{ - return svc_monitor_mac; -} - const struct ovn_datapath * northd_get_datapath_for_port(const struct hmap *ls_ports, const char *port_name) diff --git a/northd/northd.h b/northd/northd.h index 579edb8bdf..7015402c5f 100644 --- a/northd/northd.h +++ b/northd/northd.h @@ -27,7 +27,6 @@ struct northd_input { /* Northbound table references */ - const struct nbrec_nb_global_table *nbrec_nb_global_table; const struct nbrec_logical_switch_table *nbrec_logical_switch_table; const struct nbrec_logical_router_table *nbrec_logical_router_table; const struct nbrec_static_mac_binding_table @@ -37,7 +36,6 @@ struct northd_input { const struct nbrec_mirror_table *nbrec_mirror_table; /* Southbound table references */ - const struct sbrec_sb_global_table *sbrec_sb_global_table; const struct sbrec_datapath_binding_table *sbrec_datapath_binding_table; const struct sbrec_port_binding_table *sbrec_port_binding_table; const struct sbrec_mac_binding_table *sbrec_mac_binding_table; @@ -57,6 +55,13 @@ struct northd_input { const struct hmap *lbs; const struct hmap *lbgrps; + /* Global config data node inputs. */ + const struct smap *nb_options; + const struct smap *sb_options; + const char *svc_monitor_mac; + struct eth_addr svc_monitor_mac_ea; + const struct chassis_features *features; + /* Indexes */ struct ovsdb_idl_index *sbrec_chassis_by_name; struct ovsdb_idl_index *sbrec_chassis_by_hostname; @@ -66,14 +71,6 @@ struct northd_input { struct ovsdb_idl_index *sbrec_fdb_by_dp_and_port; }; -struct chassis_features { - bool ct_no_masked_label; - bool mac_binding_timestamp; - bool ct_lb_related; - bool fdb_timestamp; - bool ls_dpg_column; -}; - /* A collection of datapaths. E.g. all logical switch datapaths, or all * logical router datapaths. */ struct ovn_datapaths { @@ -156,8 +153,6 @@ struct northd_data { struct hmap lb_datapaths_map; struct hmap lb_group_datapaths_map; struct ovs_list lr_list; - bool ovn_internal_version_changed; - struct chassis_features features; struct sset svc_monitor_lsps; struct hmap svc_monitor_map; @@ -194,6 +189,7 @@ struct lflow_input { const struct chassis_features *features; const struct hmap *svc_monitor_map; bool ovn_internal_version_changed; + const char *svc_monitor_mac; }; extern int parallelization_state; @@ -711,8 +707,6 @@ void bfd_cleanup_connections(const struct nbrec_bfd_table *, struct hmap *bfd_map); void run_update_worker_pool(int n_threads); -const char *northd_get_svc_monitor_mac(void); - const struct ovn_datapath *northd_get_datapath_for_port( const struct hmap *ls_ports, const char *port_name); @@ -770,4 +764,6 @@ bool lrouter_port_ipv4_reachable(const struct ovn_port *, ovs_be32 addr); bool lrouter_port_ipv6_reachable(const struct ovn_port *, const struct in6_addr *); +uint32_t get_ovn_max_dp_key_local(const struct sbrec_chassis_table *); + #endif /* NORTHD_H */ diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at index 5c33a8a6c2..fc9787bc40 100644 --- a/tests/ovn-northd.at +++ b/tests/ovn-northd.at @@ -8782,7 +8782,7 @@ AT_CHECK([grep "ls_in_lb " S1flows | sed 's/table=../table=??/' | sort], [0], [d table=??(ls_in_lb ), priority=0 , match=(1), action=(next;) ]) -ovn-nbctl --wait=sb set NB_Global . options:install_ls_lb_from_router=true +check ovn-nbctl --wait=sb set NB_Global . options:install_ls_lb_from_router=true ovn-sbctl dump-flows S0 > S0flows ovn-sbctl dump-flows S1 > S1flows @@ -8801,6 +8801,7 @@ AT_CHECK([grep "ls_in_lb " S1flows | sed 's/table=../table=??/' | sort], [0], [d table=??(ls_in_lb ), priority=120 , match=(ct.new && ip4.dst == 172.16.0.11 && tcp.dst == 8080), action=(reg0[[1]] = 0; ct_lb_mark(backends=10.0.0.2:8080);) ]) + ovn-sbctl get datapath S0 _uuid > dp_uuids ovn-sbctl get datapath S1 _uuid >> dp_uuids lb_dp_group=$(ovn-sbctl --bare --columns ls_datapath_group find Load_Balancer name=lb0) @@ -8809,7 +8810,7 @@ AT_CHECK_UNQUOTED([ovn-sbctl --bare --columns _uuid,datapaths find Logical_DP_Gr $(cat dp_uuids | sort) ]) -ovn-nbctl --wait=sb set NB_Global . options:install_ls_lb_from_router=false +check ovn-nbctl --wait=sb set NB_Global . options:install_ls_lb_from_router=false ovn-sbctl dump-flows S0 > S0flows ovn-sbctl dump-flows S1 > S1flows @@ -9166,12 +9167,11 @@ $4 AS_BOX([Create new PG1 and PG2]) check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats check ovn-nbctl --wait=sb -- pg-add pg1 -- pg-add pg2 -dnl The northd node should not recompute, it should handle nb_global update -dnl though, therefore "compute: 1". +dnl The northd node should not recompute. AT_CHECK([as northd ovn-appctl -t ovn-northd inc-engine/show-stats northd], [0], [dnl Node: northd - recompute: 0 -- compute: 1 +- compute: 0 - abort: 0 ]) dnl The port_group node recomputes every time a NB port group is added/deleted. @@ -9204,12 +9204,11 @@ check ovn-nbctl --wait=sb \ check_column "sw1.1" sb:Port_Group ports name="${sw1_key}_pg1" check_column "sw2.1" sb:Port_Group ports name="${sw2_key}_pg1" -dnl The northd node should not recompute, it should handle nb_global update -dnl though, therefore "compute: 1". +dnl The northd node should not recompute. AT_CHECK([as northd ovn-appctl -t ovn-northd inc-engine/show-stats northd], [0], [dnl Node: northd - recompute: 0 -- compute: 1 +- compute: 0 - abort: 0 ]) dnl The port_group node recomputes also every time a port from a new switch @@ -9241,12 +9240,11 @@ check_column "sw2.1" sb:Port_Group ports name="${sw2_key}_pg1" check_column "sw1.2" sb:Port_Group ports name="${sw1_key}_pg2" check_column "sw2.2" sb:Port_Group ports name="${sw2_key}_pg2" -dnl The northd node should not recompute, it should handle nb_global update -dnl though, therefore "compute: 1". +dnl The northd node should not recompute. AT_CHECK([as northd ovn-appctl -t ovn-northd inc-engine/show-stats northd], [0], [dnl Node: northd - recompute: 0 -- compute: 1 +- compute: 0 - abort: 0 ]) dnl The port_group node recomputes also every time a port from a new switch @@ -9279,12 +9277,11 @@ check_column "sw2.1 sw2.3" sb:Port_Group ports name="${sw2_key}_pg1" check_column "sw1.2 sw1.3" sb:Port_Group ports name="${sw1_key}_pg2" check_column "sw2.2 sw2.3" sb:Port_Group ports name="${sw2_key}_pg2" -dnl The northd node should not recompute, it should handle nb_global update -dnl though, therefore "compute: 1". +dnl The northd node should not recompute. AT_CHECK([as northd ovn-appctl -t ovn-northd inc-engine/show-stats northd], [0], [dnl Node: northd - recompute: 0 -- compute: 1 +- compute: 0 - abort: 0 ]) dnl We did not change the set of switches a pg is applied to, there should be @@ -9322,7 +9319,7 @@ dnl though, therefore "compute: 1". AT_CHECK([as northd ovn-appctl -t ovn-northd inc-engine/show-stats northd], [0], [dnl Node: northd - recompute: 0 -- compute: 1 +- compute: 0 - abort: 0 ]) dnl We did not change the set of switches a pg is applied to, there should be @@ -9354,12 +9351,11 @@ check_column "sw1.2" sb:Port_Group ports name="${sw1_key}_pg2" AT_CHECK([fetch_column sb:Port_Group ports name="${sw2_key}_pg2"], [0], [ ]) -dnl The northd node should not recompute, it should handle nb_global update -dnl though, therefore "compute: 1". +dnl The northd node should not recompute. AT_CHECK([as northd ovn-appctl -t ovn-northd inc-engine/show-stats northd], [0], [dnl Node: northd - recompute: 0 -- compute: 1 +- compute: 0 - abort: 0 ]) dnl We changed the set of switches a pg is applied to, there should be @@ -9392,12 +9388,11 @@ check_column "sw1.2" sb:Port_Group ports name="${sw1_key}_pg2" AT_CHECK([fetch_column sb:Port_Group ports name="${sw2_key}_pg2"], [0], [ ]) -dnl The northd node should not recompute, it should handle nb_global update -dnl though, therefore "compute: 1". +dnl The northd node should not recompute. AT_CHECK([as northd ovn-appctl -t ovn-northd inc-engine/show-stats northd], [0], [dnl Node: northd - recompute: 0 -- compute: 1 +- compute: 0 - abort: 0 ]) dnl We changed the set of switches a pg is applied to, there should be @@ -9430,12 +9425,11 @@ check_column "sw2.1" sb:Port_Group ports name="${sw2_key}_pg1" check_column "sw1.2" sb:Port_Group ports name="${sw1_key}_pg2" check_column "sw2.2" sb:Port_Group ports name="${sw2_key}_pg2" -dnl The northd node should not recompute, it should handle nb_global update -dnl though, therefore "compute: 1". +dnl The northd node should not recompute. AT_CHECK([as northd ovn-appctl -t ovn-northd inc-engine/show-stats northd], [0], [dnl Node: northd - recompute: 0 -- compute: 1 +- compute: 0 - abort: 0 ]) dnl We changed the set of switches a pg is applied to, there should be a @@ -9470,12 +9464,11 @@ check_column "sw1.2" sb:Port_Group ports name="${sw1_key}_pg2" AT_CHECK([fetch_column sb:Port_Group ports name="${sw2_key}_pg2"], [0], [ ]) -dnl The northd node should not recompute, it should handle nb_global update -dnl though, therefore "compute: 1". +dnl The northd node should not recompute,. AT_CHECK([as northd ovn-appctl -t ovn-northd inc-engine/show-stats northd], [0], [dnl Node: northd - recompute: 0 -- compute: 1 +- compute: 0 - abort: 0 ]) dnl We changed the set of switches a pg is applied to, there should be a @@ -11837,3 +11830,212 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE OVN_CLEANUP([hv1]) AT_CLEANUP ]) + +OVN_FOR_EACH_NORTHD_NO_HV([ +AT_SETUP([NB_Global and SB_Global incremental processing]) + +ovn_start + +check_engine_stats() { + node=$1 + recompute=$2 + compute=$3 + + echo "__file__:__line__: Checking engine stats for node $node : recompute - \ +$recompute : compute - $compute" + + node_stat=$(as northd ovn-appctl -t ovn-northd inc-engine/show-stats $node) + # node_stat will be of this format : + # - Node: lflow - recompute: 3 - compute: 0 - abort: 0 + node_recompute_ct=$(echo $node_stat | cut -d '-' -f2 | cut -d ':' -f2) + node_compute_ct=$(echo $node_stat | cut -d '-' -f3 | cut -d ':' -f2) + + if [[ "$recompute" == "norecompute" ]]; then + # node should not be recomputed + echo "Expecting $node recompute count - $node_recompute_ct to be 0" + check test "$node_recompute_ct" -eq "0" + else + echo "Expecting $node recompute count - $node_recompute_ct not to be 0" + check test "$node_recompute_ct" -ne "0" + fi + + if [[ "$compute" == "nocompute" ]]; then + # node should not be computed + echo "Expecting $node compute count - $node_compute_ct to be 0" + check test "$node_compute_ct" -eq "0" + else + echo "Expecting $node compute count - $node_compute_ct not to be 0" + check test "$node_compute_ct" -ne "0" + fi +} + +check ovn-nbctl ls-add sw0 +check ovn-nbctl lr-add lr0 +check ovn-nbctl lsp-add sw0 sw0-p1 -- lsp-set-addresses sw0-p1 "00:00:20:20:00:03 10.0.0.3" +check ovn-nbctl lrp-add lr0 lr0-sw0 00:00:20:20:12:14 10.0.0.1/24 +check ovn-nbctl lsp-add sw0 sw0-lr0 +check ovn-nbctl lsp-set-type sw0-lr0 router +check ovn-nbctl lsp-set-addresses sw0-lr0 router +check ovn-nbctl --wait=sb lsp-set-options sw0-lr0 router-port=lr0-sw0 +check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats + +# This should not result in recomputes. +check ovn-nbctl --wait=sb set NB_Global . options:foo=bar +check_engine_stats global_config norecompute compute +check_engine_stats northd norecompute compute +check_engine_stats lflow norecompute compute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats +# This should result in recomputes. +check ovn-sbctl set SB_Global . options:bar=foo +check_engine_stats global_config recompute compute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +# Clears an nb option and checks that recomputes were triggered +# and the option was added back by ovn-northd or not depending +# on the 'added_back' argument. +clear_nb_option() { + option=$1 + add_back=$2 + echo "clearing the nb option - $option" + check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats + check ovn-nbctl --wait=sb remove NB_Global . options $option + check_engine_stats global_config recompute compute + check_engine_stats northd recompute nocompute + check_engine_stats lflow recompute nocompute + + local retval=1 + if [ "$add_back" == "true" ]; then + retval=0 + fi + AT_CHECK([ovn-nbctl get NB_Global . options:$option], [$retval], [ignore], [ignore]) +} + +# Clear svc_monitor_mac and few other options which result in recompute. +# and ovn-northd should update the nb options back. +clear_nb_option svc_monitor_mac true +clear_nb_option max_tunid true +clear_nb_option mac_prefix true +clear_nb_option northd_internal_version true + +check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats +check ovn-nbctl --wait=sb set NB_Global . options:ignore_chassis_features=true +check_engine_stats global_config recompute compute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute + +clear_nb_option ignore_chassis_features false + +set_nb_option_lflow_recompute() { + local option=$1 + local value=$2 + check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats + check ovn-nbctl --wait=sb set NB_Global . options:$option=$value + check_engine_stats global_config norecompute compute + check_engine_stats northd recompute nocompute + check_engine_stats lflow recompute nocompute + check_engine_stats mac_binding_aging recompute nocompute + CHECK_NO_CHANGE_AFTER_RECOMPUTE +} + +clear_nb_option_lflow_recompute() { + local option=$1 + check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats + check ovn-nbctl --wait=sb remove NB_Global . options $option + check_engine_stats global_config norecompute compute + check_engine_stats northd recompute nocompute + check_engine_stats lflow recompute nocompute + check_engine_stats mac_binding_aging recompute nocompute + CHECK_NO_CHANGE_AFTER_RECOMPUTE +} + +set_nb_option_lflow_recompute debug_drop_domain_id 1 +clear_nb_option_lflow_recompute debug_drop_domain_id + +set_nb_option_lflow_recompute debug_drop_collector_set 1 +clear_nb_option_lflow_recompute debug_drop_collector_set + +set_nb_option_lflow_recompute mac_binding_removal_limit 100 +clear_nb_option_lflow_recompute mac_binding_removal_limit + +set_nb_option_lflow_recompute fdb_removal_limit 100 +clear_nb_option_lflow_recompute fdb_removal_limit + +set_nb_option_lflow_recompute controller_event true +clear_nb_option_lflow_recompute controller_event + +set_nb_option_lflow_recompute ignore_lsp_down true +clear_nb_option_lflow_recompute ignore_lsp_down + +set_nb_option_lflow_recompute use_ct_inv_match true +clear_nb_option_lflow_recompute use_ct_inv_match + +set_nb_option_lflow_recompute default_acl_drop true +clear_nb_option_lflow_recompute default_acl_drop + +set_nb_option_lflow_recompute use_common_zone true +clear_nb_option_lflow_recompute use_common_zone + +set_nb_option_lflow_recompute install_ls_lb_from_router true +clear_nb_option_lflow_recompute install_ls_lb_from_router + +# Now test changes to chassis for feature changes. +check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats +check ovn-sbctl chassis-add ch1 geneve 127.0.0.1 +check ovn-nbctl --wait=sb sync +check_engine_stats global_config recompute compute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute + +check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats +check ovn-sbctl chassis-add ch2 geneve 127.0.0.2 +check ovn-nbctl --wait=sb sync +check_engine_stats global_config recompute compute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute + +AT_CHECK([ovn-nbctl get NB_Global . options:max_tunid | \ +sed s/":"//g | sed s/\"//g], [0], [16711680 +], []) + +check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats +check ovn-sbctl chassis-del ch2 +check ovn-nbctl --wait=sb sync +check_engine_stats global_config recompute compute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute + +check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats +check ovn-sbctl set encap . type=vxlan +check ovn-nbctl --wait=sb sync +check_engine_stats global_config recompute compute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute + +AT_CHECK([ovn-nbctl get NB_Global . options:max_tunid | \ +sed s/":"//g | sed s/\"//g], [0], [4095 +], []) + +check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats +check ovn-sbctl set chassis . other_config:foo=bar +check ovn-nbctl --wait=sb sync +check_engine_stats global_config norecompute compute +check_engine_stats mac_binding_aging recompute nocompute +check_engine_stats fdb_aging recompute nocompute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute + +check as northd ovn-appctl -t ovn-northd inc-engine/clear-stats +check ovn-sbctl set chassis . other_config:ct-no-masked-label=true +check ovn-nbctl --wait=sb sync +check_engine_stats global_config norecompute compute +check_engine_stats mac_binding_aging recompute nocompute +check_engine_stats fdb_aging recompute nocompute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute + +AT_CLEANUP +])