From patchwork Tue Nov 28 02:34:33 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Numan Siddique X-Patchwork-Id: 1869072 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::138; helo=smtp1.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp1.osuosl.org (smtp1.osuosl.org [IPv6:2605:bc80:3010::138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4SfRPk660Vz1yST for ; Tue, 28 Nov 2023 13:35:06 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 25F5F81A5F; Tue, 28 Nov 2023 02:35:04 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 25F5F81A5F X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id N1Q1eStA5-67; Tue, 28 Nov 2023 02:35:01 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp1.osuosl.org (Postfix) with ESMTPS id CA8D38169A; Tue, 28 Nov 2023 02:35:00 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org CA8D38169A Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 8CB2EC0072; Tue, 28 Nov 2023 02:35:00 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by lists.linuxfoundation.org (Postfix) with ESMTP id 832ADC0039 for ; Tue, 28 Nov 2023 02:34:59 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 661068188B for ; Tue, 28 Nov 2023 02:34:53 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 661068188B X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 6A8mK7pAE_Dq for ; Tue, 28 Nov 2023 02:34:51 +0000 (UTC) Received: from relay3-d.mail.gandi.net (relay3-d.mail.gandi.net [IPv6:2001:4b98:dc4:8::223]) by smtp1.osuosl.org (Postfix) with ESMTPS id CFE87818DC for ; Tue, 28 Nov 2023 02:34:50 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org CFE87818DC Received: by mail.gandi.net (Postfix) with ESMTPSA id 47B6560004; Tue, 28 Nov 2023 02:34:46 +0000 (UTC) From: numans@ovn.org To: dev@openvswitch.org Date: Mon, 27 Nov 2023 21:34:33 -0500 Message-ID: <20231128023433.569570-1-numans@ovn.org> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20231128023410.569539-1-numans@ovn.org> References: <20231128023410.569539-1-numans@ovn.org> MIME-Version: 1.0 X-GND-Sasl: numans@ovn.org Subject: [ovs-dev] [PATCH ovn v3 01/16] northd: Refactor the northd change tracking. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Numan Siddique northd engine tracking data now has the following tracking data - changed ovn_ports (right now only changed logical switch ports are tracked.) - changed load balancers. This separation becomes easier to add lflow handling for these changes in lflow northd engine handler. This patch doesn't handle the load balancer changes in lflow handler. It will be handled in upcoming commits. Before this patch, any changes to load balancers or lb groups resulted in full recompute of 'sync_to_sb_lb' and 'lflow' engine nodes. Now this scenario is optimized and it results in recomputes of these nodes only if 'northd' engine node adds any changed load balancers in its tracked data. One example is created or updating an empty load balancer group. Signed-off-by: Numan Siddique --- northd/en-lflow.c | 12 +- northd/en-northd.c | 18 +- northd/en-sync-from-sb.c | 2 +- northd/en-sync-sb.c | 9 +- northd/northd.c | 436 ++++++++++++++++++++------------------- northd/northd.h | 86 +++++--- tests/ovn-northd.at | 11 +- 7 files changed, 314 insertions(+), 260 deletions(-) diff --git a/northd/en-lflow.c b/northd/en-lflow.c index 2b84fef0ef..6ba26006e0 100644 --- a/northd/en-lflow.c +++ b/northd/en-lflow.c @@ -104,12 +104,12 @@ lflow_northd_handler(struct engine_node *node, void *data) { struct northd_data *northd_data = engine_get_input_data("northd", node); - if (!northd_data->change_tracked) { + if (!northd_has_tracked_data(&northd_data->trk_data)) { return false; } - /* Fall back to recompute if lb related data has changed. */ - if (northd_data->lb_changed) { + /* Fall back to recompute if load balancers have changed. */ + if (northd_has_lbs_in_tracked_data(&northd_data->trk_data)) { return false; } @@ -119,9 +119,9 @@ lflow_northd_handler(struct engine_node *node, struct lflow_input lflow_input; lflow_get_input_data(node, &lflow_input); - if (!lflow_handle_northd_ls_changes(eng_ctx->ovnsb_idl_txn, - &northd_data->tracked_ls_changes, - &lflow_input, &lflow_data->lflows)) { + if (!lflow_handle_northd_port_changes(eng_ctx->ovnsb_idl_txn, + &northd_data->trk_data.trk_lsps, + &lflow_input, &lflow_data->lflows)) { return false; } diff --git a/northd/en-northd.c b/northd/en-northd.c index aa0f20f0c2..28559ed211 100644 --- a/northd/en-northd.c +++ b/northd/en-northd.c @@ -171,7 +171,7 @@ northd_nb_logical_switch_handler(struct engine_node *node, return false; } - if (nd->change_tracked) { + if (northd_has_lsps_in_tracked_data(&nd->trk_data)) { engine_set_node_state(node, EN_UPDATED); } @@ -209,10 +209,6 @@ northd_nb_logical_router_handler(struct engine_node *node, return false; } - if (nd->change_tracked) { - engine_set_node_state(node, EN_UPDATED); - } - return true; } @@ -230,15 +226,15 @@ northd_lb_data_handler(struct engine_node *node, void *data) &nd->ls_datapaths, &nd->lr_datapaths, &nd->lb_datapaths_map, - &nd->lb_group_datapaths_map)) { + &nd->lb_group_datapaths_map, + &nd->trk_data)) { return false; } - /* Indicate the depedendant engine nodes that load balancer/group - * related data has changed (including association to logical - * switch/router). */ - nd->lb_changed = true; - engine_set_node_state(node, EN_UPDATED); + if (northd_has_lbs_in_tracked_data(&nd->trk_data)) { + engine_set_node_state(node, EN_UPDATED); + } + return true; } diff --git a/northd/en-sync-from-sb.c b/northd/en-sync-from-sb.c index 48b1576173..8c05239b49 100644 --- a/northd/en-sync-from-sb.c +++ b/northd/en-sync-from-sb.c @@ -65,7 +65,7 @@ sync_from_sb_northd_handler(struct engine_node *node, void *data OVS_UNUSED) { struct northd_data *nd = engine_get_input_data("northd", node); - if (nd->change_tracked) { + if (northd_has_tracked_data(&nd->trk_data)) { /* So far the changes tracked in northd don't impact this node. * * In particular, for the LS related changes, the only field this node diff --git a/northd/en-sync-sb.c b/northd/en-sync-sb.c index 2ec3bf54f8..3aaab8d005 100644 --- a/northd/en-sync-sb.c +++ b/northd/en-sync-sb.c @@ -236,7 +236,8 @@ sync_to_sb_lb_northd_handler(struct engine_node *node, void *data OVS_UNUSED) { struct northd_data *nd = engine_get_input_data("northd", node); - if (!nd->change_tracked || nd->lb_changed) { + if (!northd_has_tracked_data(&nd->trk_data) || + northd_has_lbs_in_tracked_data(&nd->trk_data)) { /* Return false if no tracking data or if lbs changed. */ return false; } @@ -306,11 +307,13 @@ sync_to_sb_pb_northd_handler(struct engine_node *node, void *data OVS_UNUSED) } struct northd_data *nd = engine_get_input_data("northd", node); - if (!nd->change_tracked) { + if (!northd_has_tracked_data(&nd->trk_data) || + northd_has_lbs_in_tracked_data(&nd->trk_data)) { + /* Return false if no tracking data or if lbs changed. */ return false; } - if (!sync_pbs_for_northd_ls_changes(&nd->tracked_ls_changes)) { + if (!sync_pbs_for_northd_changed_ovn_ports(&nd->trk_data.trk_lsps)) { return false; } diff --git a/northd/northd.c b/northd/northd.c index 29245d6333..abb06bfc6a 100644 --- a/northd/northd.c +++ b/northd/northd.c @@ -4894,21 +4894,20 @@ sync_pbs(struct ovsdb_idl_txn *ovnsb_idl_txn, struct hmap *ls_ports) } /* Sync the SB Port bindings for the added and updated logical switch ports - * of the tracked logical switches (from the northd engine node). */ + * of the tracked northd engine data. */ bool -sync_pbs_for_northd_ls_changes(struct tracked_ls_changes *ls_changes) +sync_pbs_for_northd_changed_ovn_ports( struct tracked_ovn_ports *trk_ovn_ports) { - struct ls_change *ls_change; - LIST_FOR_EACH (ls_change, list_node, &ls_changes->updated) { - struct ovn_port *op; - - LIST_FOR_EACH (op, list, &ls_change->added_ports) { - sync_pb_for_op(op); - } + struct hmapx_node *hmapx_node; + struct ovn_port *op; + HMAPX_FOR_EACH (hmapx_node, &trk_ovn_ports->created) { + op = hmapx_node->data; + sync_pb_for_op(op); + } - LIST_FOR_EACH (op, list, &ls_change->updated_ports) { - sync_pb_for_op(op); - } + HMAPX_FOR_EACH (hmapx_node, &trk_ovn_ports->updated) { + op = hmapx_node->data; + sync_pb_for_op(op); } return true; @@ -5110,34 +5109,68 @@ build_ports(struct ovsdb_idl_txn *ovnsb_txn, } static void -destroy_tracked_ls_change(struct ls_change *ls_change) +destroy_tracked_ovn_ports(struct tracked_ovn_ports *trk_ovn_ports) { - struct ovn_port *op; - LIST_FOR_EACH (op, list, &ls_change->added_ports) { - ovs_list_remove(&op->list); - } - LIST_FOR_EACH (op, list, &ls_change->updated_ports) { - ovs_list_remove(&op->list); + struct hmapx_node *hmapx_node; + HMAPX_FOR_EACH_SAFE (hmapx_node, &trk_ovn_ports->deleted) { + ovn_port_destroy_orphan(hmapx_node->data); + hmapx_delete(&trk_ovn_ports->deleted, hmapx_node); } - LIST_FOR_EACH_SAFE (op, list, &ls_change->deleted_ports) { - ovs_list_remove(&op->list); - ovn_port_destroy_orphan(op); + + hmapx_clear(&trk_ovn_ports->created); + hmapx_clear(&trk_ovn_ports->updated); +} + +static void +destroy_tracked_lbs(struct tracked_lbs *trk_lbs) +{ + struct hmapx_node *hmapx_node; + HMAPX_FOR_EACH_SAFE (hmapx_node, &trk_lbs->deleted) { + ovn_lb_datapaths_destroy(hmapx_node->data); + hmapx_delete(&trk_lbs->deleted, hmapx_node); } + + hmapx_clear(&trk_lbs->crupdated); +} + +static void +add_op_to_northd_tracked_ports(struct hmapx *tracked_ovn_ports, + struct ovn_port *op) +{ + hmapx_add(tracked_ovn_ports, op); } void destroy_northd_data_tracked_changes(struct northd_data *nd) { - struct ls_change *ls_change; - LIST_FOR_EACH_SAFE (ls_change, list_node, - &nd->tracked_ls_changes.updated) { - destroy_tracked_ls_change(ls_change); - ovs_list_remove(&ls_change->list_node); - free(ls_change); - } + struct northd_tracked_data *trk_changes = &nd->trk_data; + destroy_tracked_ovn_ports(&trk_changes->trk_lsps); + destroy_tracked_lbs(&trk_changes->trk_lbs); + nd->trk_data.type = NORTHD_TRACKED_NONE; +} + +static void +init_northd_tracked_data(struct northd_data *nd) +{ + struct northd_tracked_data *trk_data = &nd->trk_data; + trk_data->type = NORTHD_TRACKED_NONE; + hmapx_init(&trk_data->trk_lsps.created); + hmapx_init(&trk_data->trk_lsps.updated); + hmapx_init(&trk_data->trk_lsps.deleted); + hmapx_init(&trk_data->trk_lbs.crupdated); + hmapx_init(&trk_data->trk_lbs.deleted); +} - nd->change_tracked = false; - nd->lb_changed = false; +static void +destroy_northd_tracked_data(struct northd_data *nd) +{ + struct northd_tracked_data *trk_data = &nd->trk_data; + trk_data->type = NORTHD_TRACKED_NONE; + hmapx_destroy(&trk_data->trk_lsps.created); + hmapx_destroy(&trk_data->trk_lsps.updated); + hmapx_destroy(&trk_data->trk_lsps.deleted); + hmapx_destroy(&trk_data->trk_lbs.crupdated); + hmapx_destroy(&trk_data->trk_lbs.deleted); } /* Check if a changed LSP can be handled incrementally within the I-P engine @@ -5344,12 +5377,11 @@ check_lsp_changes_other_than_up(const struct nbrec_logical_switch_port *nbsp) */ static bool ls_handle_lsp_changes(struct ovsdb_idl_txn *ovnsb_idl_txn, - const struct nbrec_logical_switch *changed_ls, - const struct northd_input *ni, - struct northd_data *nd, - struct ovn_datapath *od, - struct ls_change *ls_change, - bool *updated) + const struct nbrec_logical_switch *changed_ls, + const struct northd_input *ni, + struct northd_data *nd, + struct ovn_datapath *od, + struct tracked_ovn_ports *trk_lsps) { bool ls_ports_changed = false; if (!nbrec_logical_switch_is_updated(changed_ls, @@ -5370,7 +5402,7 @@ ls_handle_lsp_changes(struct ovsdb_idl_txn *ovnsb_idl_txn, return true; } - ls_change->had_only_router_ports = (od->n_router_ports + bool ls_had_only_router_ports = (od->n_router_ports && (od->n_router_ports == hmap_count(&od->ports))); struct ovn_port *op; @@ -5396,8 +5428,7 @@ ls_handle_lsp_changes(struct ovsdb_idl_txn *ovnsb_idl_txn, if (!op) { goto fail; } - ovs_list_push_back(&ls_change->added_ports, - &op->list); + add_op_to_northd_tracked_ports(&trk_lsps->created, op); } else if (ls_port_has_changed(op->nbsp, new_nbsp)) { /* Existing port updated */ bool temp = false; @@ -5433,7 +5464,7 @@ ls_handle_lsp_changes(struct ovsdb_idl_txn *ovnsb_idl_txn, if (!op) { goto fail; } - ovs_list_push_back(&ls_change->updated_ports, &op->list); + add_op_to_northd_tracked_ports(&trk_lsps->updated, op); } op->visited = true; } @@ -5442,15 +5473,14 @@ ls_handle_lsp_changes(struct ovsdb_idl_txn *ovnsb_idl_txn, HMAP_FOR_EACH_SAFE (op, dp_node, &od->ports) { if (!op->visited) { if (!op->lsp_can_be_inc_processed) { - goto fail_clean_deleted; + goto fail; } if (sset_contains(&nd->svc_monitor_lsps, op->key)) { /* This port was used for svc monitor, which may be * impacted by this deletion. Fallback to recompute. */ - goto fail_clean_deleted; + goto fail; } - ovs_list_push_back(&ls_change->deleted_ports, - &op->list); + add_op_to_northd_tracked_ports(&trk_lsps->deleted, op); hmap_remove(&nd->ls_ports, &op->key_node); hmap_remove(&od->ports, &op->dp_node); sbrec_port_binding_delete(op->sb); @@ -5459,27 +5489,32 @@ ls_handle_lsp_changes(struct ovsdb_idl_txn *ovnsb_idl_txn, } } - if (!ovs_list_is_empty(&ls_change->added_ports) || - !ovs_list_is_empty(&ls_change->updated_ports) || - !ovs_list_is_empty(&ls_change->deleted_ports)) { - *updated = true; + bool ls_has_only_router_ports = (od->n_router_ports + && (od->n_router_ports == hmap_count(&od->ports))); + + /* There are lflows related to router ports that depends on whether + * there are switch ports on the logical switch (see + * build_lswitch_rport_arp_req_flow() for more details). Check if this + * dependency has changed and if it has, then add the router ports + * to the tracked 'updated' ovn ports so that lflow engine can + * regenerate lflows for these router ports. */ + if (ls_had_only_router_ports != ls_has_only_router_ports) { + for (size_t i = 0; i < od->n_router_ports; i++) { + op = od->router_ports[i]; + add_op_to_northd_tracked_ports(&trk_lsps->updated, op); + } } return true; -fail_clean_deleted: - LIST_FOR_EACH_POP (op, list, &ls_change->deleted_ports) { - ovn_port_destroy_orphan(op); - } - fail: + destroy_tracked_ovn_ports(trk_lsps); return false; } /* Return true if changes are handled incrementally, false otherwise. * When there are any changes, try to track what's exactly changed and set - * northd_data->change_tracked accordingly: change tracked - true, otherwise, - * false. + * northd_data->trk_data accordingly. * * Note: Changes to load balancer and load balancer groups associated with * the logical switches are handled separately in the lb_data change handlers. @@ -5490,7 +5525,7 @@ northd_handle_ls_changes(struct ovsdb_idl_txn *ovnsb_idl_txn, struct northd_data *nd) { const struct nbrec_logical_switch *changed_ls; - struct ls_change *ls_change = NULL; + struct northd_tracked_data *trk_data = &nd->trk_data; NBREC_LOGICAL_SWITCH_TABLE_FOR_EACH_TRACKED (changed_ls, ni->nbrec_logical_switch_table) { @@ -5514,31 +5549,16 @@ northd_handle_ls_changes(struct ovsdb_idl_txn *ovnsb_idl_txn, goto fail; } - ls_change = xzalloc(sizeof *ls_change); - ls_change->od = od; - ovs_list_init(&ls_change->added_ports); - ovs_list_init(&ls_change->deleted_ports); - ovs_list_init(&ls_change->updated_ports); - - bool updated = false; if (!ls_handle_lsp_changes(ovnsb_idl_txn, changed_ls, - ni, nd, od, ls_change, - &updated)) { - destroy_tracked_ls_change(ls_change); - free(ls_change); + ni, nd, od, &trk_data->trk_lsps)) { goto fail; } - - if (updated) { - ovs_list_push_back(&nd->tracked_ls_changes.updated, - &ls_change->list_node); - } else { - free(ls_change); - } } - if (!ovs_list_is_empty(&nd->tracked_ls_changes.updated)) { - nd->change_tracked = true; + if (!hmapx_is_empty(&trk_data->trk_lsps.created) + || !hmapx_is_empty(&trk_data->trk_lsps.updated) + || !hmapx_is_empty(&trk_data->trk_lsps.deleted)) { + trk_data->type |= NORTHD_TRACKED_PORTS; } return true; @@ -5603,13 +5623,10 @@ lr_changes_can_be_handled( } /* Return true if changes are handled incrementally, false otherwise. - * When there are any changes, try to track what's exactly changed and set - * northd_data->change_tracked accordingly: change tracked - true, otherwise, - * false. + * * Note: Changes to load balancer and load balancer groups associated with * the logical routers are handled separately in the lb_data change - * handlers (northd_handle_lb_data_changes_pre_od and - * northd_handle_lb_data_changes_post_od). + * handler - northd_handle_lb_data_changes(). * */ bool northd_handle_lr_changes(const struct northd_input *ni, @@ -5715,7 +5732,8 @@ northd_handle_lb_data_changes(struct tracked_lb_data *trk_lb_data, struct ovn_datapaths *ls_datapaths, struct ovn_datapaths *lr_datapaths, struct hmap *lb_datapaths_map, - struct hmap *lbgrp_datapaths_map) + struct hmap *lbgrp_datapaths_map, + struct northd_tracked_data *nd_changes) { if (trk_lb_data->has_health_checks) { /* Fall back to recompute since a tracked load balancer @@ -5778,7 +5796,9 @@ northd_handle_lb_data_changes(struct tracked_lb_data *trk_lb_data, } hmap_remove(lb_datapaths_map, &lb_dps->hmap_node); - ovn_lb_datapaths_destroy(lb_dps); + + /* Add the deleted lb to the northd tracked data. */ + hmapx_add(&nd_changes->trk_lbs.deleted, lb_dps); } /* Create the 'lb_dps' if not already created for each @@ -5795,6 +5815,9 @@ northd_handle_lb_data_changes(struct tracked_lb_data *trk_lb_data, hmap_insert(lb_datapaths_map, &lb_dps->hmap_node, uuid_hash(lb_uuid)); } + + /* Add the updated lb to the northd tracked data. */ + hmapx_add(&nd_changes->trk_lbs.crupdated, lb_dps); } struct ovn_lb_group_datapaths *lbgrp_dps; @@ -5825,6 +5848,9 @@ northd_handle_lb_data_changes(struct tracked_lb_data *trk_lb_data, lb_dps = ovn_lb_datapaths_find(lb_datapaths_map, &uuidnode->uuid); ovs_assert(lb_dps); ovn_lb_datapaths_add_ls(lb_dps, 1, &od); + + /* Add the lb to the northd tracked data. */ + hmapx_add(&nd_changes->trk_lbs.crupdated, lb_dps); } UUIDSET_FOR_EACH (uuidnode, &codlb->assoc_lbgrps) { @@ -5840,6 +5866,9 @@ northd_handle_lb_data_changes(struct tracked_lb_data *trk_lb_data, lb_dps = ovn_lb_datapaths_find(lb_datapaths_map, lb_uuid); ovs_assert(lb_dps); ovn_lb_datapaths_add_ls(lb_dps, 1, &od); + + /* Add the lb to the northd tracked data. */ + hmapx_add(&nd_changes->trk_lbs.crupdated, lb_dps); } } @@ -5860,6 +5889,9 @@ northd_handle_lb_data_changes(struct tracked_lb_data *trk_lb_data, /* Add the lb_ips of lb_dps to the od. */ build_lrouter_lb_ips(od->lb_ips, lb_dps->lb); build_lrouter_lb_reachable_ips(od, lb_dps->lb); + + /* Add the lb to the northd tracked data. */ + hmapx_add(&nd_changes->trk_lbs.crupdated, lb_dps); } UUIDSET_FOR_EACH (uuidnode, &codlb->assoc_lbgrps) { @@ -5879,6 +5911,9 @@ northd_handle_lb_data_changes(struct tracked_lb_data *trk_lb_data, /* Add the lb_ips of lb_dps to the od. */ build_lrouter_lb_ips(od->lb_ips, lb_dps->lb); build_lrouter_lb_reachable_ips(od, lb_dps->lb); + + /* Add the lb to the northd tracked data. */ + hmapx_add(&nd_changes->trk_lbs.crupdated, lb_dps); } } @@ -5957,9 +5992,17 @@ northd_handle_lb_data_changes(struct tracked_lb_data *trk_lb_data, /* Re-evaluate 'od->has_lb_vip' */ init_lb_for_datapath(od); } + + /* Add the lb to the northd tracked data. */ + hmapx_add(&nd_changes->trk_lbs.crupdated, lb_dps); } } + if (!hmapx_is_empty(&nd_changes->trk_lbs.crupdated) + || !hmapx_is_empty(&nd_changes->trk_lbs.deleted)) { + nd_changes->type |= NORTHD_TRACKED_LBS; + } + return true; } @@ -16980,149 +17023,122 @@ delete_lflow_for_lsp(struct ovn_port *op, bool is_update, return true; } -bool lflow_handle_northd_ls_changes(struct ovsdb_idl_txn *ovnsb_txn, - struct tracked_ls_changes *ls_changes, - struct lflow_input *lflow_input, - struct hmap *lflows) +bool +lflow_handle_northd_port_changes(struct ovsdb_idl_txn *ovnsb_txn, + struct tracked_ovn_ports *trk_lsps, + struct lflow_input *lflow_input, + struct hmap *lflows) { - struct ls_change *ls_change; + struct hmapx_node *hmapx_node; + struct ovn_port *op; - LIST_FOR_EACH (ls_change, list_node, &ls_changes->updated) { - const struct sbrec_multicast_group *sbmc_flood = - mcast_group_lookup(lflow_input->sbrec_mcast_group_by_name_dp, - MC_FLOOD, ls_change->od->sb); - const struct sbrec_multicast_group *sbmc_flood_l2 = - mcast_group_lookup(lflow_input->sbrec_mcast_group_by_name_dp, - MC_FLOOD_L2, ls_change->od->sb); - const struct sbrec_multicast_group *sbmc_unknown = - mcast_group_lookup(lflow_input->sbrec_mcast_group_by_name_dp, - MC_UNKNOWN, ls_change->od->sb); + HMAPX_FOR_EACH (hmapx_node, &trk_lsps->deleted) { + op = hmapx_node->data; + /* Make sure 'op' is an lsp and not lrp. */ + ovs_assert(op->nbsp); - struct ovn_port *op; - LIST_FOR_EACH (op, list, &ls_change->deleted_ports) { - if (!delete_lflow_for_lsp(op, false, - lflow_input->sbrec_logical_flow_table, - lflows)) { + if (!delete_lflow_for_lsp(op, false, + lflow_input->sbrec_logical_flow_table, + lflows)) { return false; } - /* No need to update SB multicast groups, thanks to weak - * references. */ - } + /* No need to update SB multicast groups, thanks to weak + * references. */ + } - LIST_FOR_EACH (op, list, &ls_change->updated_ports) { - /* Delete old lflows. */ - if (!delete_lflow_for_lsp(op, true, - lflow_input->sbrec_logical_flow_table, - lflows)) { - return false; - } + HMAPX_FOR_EACH (hmapx_node, &trk_lsps->updated) { + op = hmapx_node->data; + /* Make sure 'op' is an lsp and not lrp. */ + ovs_assert(op->nbsp); - /* Generate new lflows. */ - struct ds match = DS_EMPTY_INITIALIZER; - struct ds actions = DS_EMPTY_INITIALIZER; - build_lswitch_and_lrouter_iterate_by_lsp(op, lflow_input->ls_ports, - lflow_input->lr_ports, - lflow_input->meter_groups, - &match, &actions, - lflows); - ds_destroy(&match); - ds_destroy(&actions); - - /* SB port_binding is not deleted, so don't update SB multicast - * groups. */ - - /* Sync the new flows to SB. */ - struct lflow_ref_node *lfrn; - LIST_FOR_EACH (lfrn, lflow_list_node, &op->lflows) { - sync_lsp_lflows_to_sb(ovnsb_txn, lflow_input, lflows, - lfrn->lflow); - } + /* Delete old lflows. */ + if (!delete_lflow_for_lsp(op, true, + lflow_input->sbrec_logical_flow_table, + lflows)) { + return false; } - LIST_FOR_EACH (op, list, &ls_change->added_ports) { - struct ds match = DS_EMPTY_INITIALIZER; - struct ds actions = DS_EMPTY_INITIALIZER; - build_lswitch_and_lrouter_iterate_by_lsp(op, lflow_input->ls_ports, - lflow_input->lr_ports, - lflow_input->meter_groups, - &match, &actions, - lflows); - ds_destroy(&match); - ds_destroy(&actions); - - /* Update SB multicast groups for the new port. */ - if (!sbmc_flood) { - sbmc_flood = create_sb_multicast_group(ovnsb_txn, - ls_change->od->sb, MC_FLOOD, OVN_MCAST_FLOOD_TUNNEL_KEY); - } - sbrec_multicast_group_update_ports_addvalue(sbmc_flood, op->sb); + /* Generate new lflows. */ + struct ds match = DS_EMPTY_INITIALIZER; + struct ds actions = DS_EMPTY_INITIALIZER; + build_lswitch_and_lrouter_iterate_by_lsp(op, lflow_input->ls_ports, + lflow_input->lr_ports, + lflow_input->meter_groups, + &match, &actions, + lflows); + ds_destroy(&match); + ds_destroy(&actions); - if (!sbmc_flood_l2) { - sbmc_flood_l2 = create_sb_multicast_group(ovnsb_txn, - ls_change->od->sb, MC_FLOOD_L2, - OVN_MCAST_FLOOD_L2_TUNNEL_KEY); - } - sbrec_multicast_group_update_ports_addvalue(sbmc_flood_l2, op->sb); + /* SB port_binding is not deleted, so don't update SB multicast + * groups. */ - if (op->has_unknown) { - if (!sbmc_unknown) { - sbmc_unknown = create_sb_multicast_group(ovnsb_txn, - ls_change->od->sb, MC_UNKNOWN, - OVN_MCAST_UNKNOWN_TUNNEL_KEY); - } - sbrec_multicast_group_update_ports_addvalue(sbmc_unknown, - op->sb); - } - - /* Sync the newly added flows to SB. */ - struct lflow_ref_node *lfrn; - LIST_FOR_EACH (lfrn, lflow_list_node, &op->lflows) { - sync_lsp_lflows_to_sb(ovnsb_txn, lflow_input, lflows, - lfrn->lflow); - } + /* Sync the new flows to SB. */ + struct lflow_ref_node *lfrn; + LIST_FOR_EACH (lfrn, lflow_list_node, &op->lflows) { + sync_lsp_lflows_to_sb(ovnsb_txn, lflow_input, lflows, + lfrn->lflow); } + } - bool ls_has_only_router_ports = (ls_change->od->n_router_ports && - (ls_change->od->n_router_ports == - hmap_count(&ls_change->od->ports))); - - if (ls_change->had_only_router_ports != ls_has_only_router_ports) { - /* There are lflows related to router ports that depends on whether - * there are switch ports on the logical switch (see - * build_lswitch_rport_arp_req_flow() for more details). Since this - * dependency changed, we need to regenerate lflows for each router - * port on this logical switch. */ - for (size_t i = 0; i < ls_change->od->n_router_ports; i++) { - op = ls_change->od->router_ports[i]; - - /* Delete old lflows. */ - if (!delete_lflow_for_lsp(op, "affected router", - lflow_input->sbrec_logical_flow_table, - lflows)) { - return false; - } + HMAPX_FOR_EACH (hmapx_node, &trk_lsps->created) { + op = hmapx_node->data; + /* Make sure 'op' is an lsp and not lrp. */ + ovs_assert(op->nbsp); - /* Generate new lflows. */ - struct ds match = DS_EMPTY_INITIALIZER; - struct ds actions = DS_EMPTY_INITIALIZER; - build_lswitch_and_lrouter_iterate_by_lsp(op, - lflow_input->ls_ports, lflow_input->lr_ports, - lflow_input->meter_groups, &match, &actions, lflows); - ds_destroy(&match); - ds_destroy(&actions); - - /* Sync the new flows to SB. */ - struct lflow_ref_node *lfrn; - LIST_FOR_EACH (lfrn, lflow_list_node, &op->lflows) { - sync_lsp_lflows_to_sb(ovnsb_txn, lflow_input, lflows, - lfrn->lflow); - } + const struct sbrec_multicast_group *sbmc_flood = + mcast_group_lookup(lflow_input->sbrec_mcast_group_by_name_dp, + MC_FLOOD, op->od->sb); + const struct sbrec_multicast_group *sbmc_flood_l2 = + mcast_group_lookup(lflow_input->sbrec_mcast_group_by_name_dp, + MC_FLOOD_L2, op->od->sb); + const struct sbrec_multicast_group *sbmc_unknown = + mcast_group_lookup(lflow_input->sbrec_mcast_group_by_name_dp, + MC_UNKNOWN, op->od->sb); + + struct ds match = DS_EMPTY_INITIALIZER; + struct ds actions = DS_EMPTY_INITIALIZER; + build_lswitch_and_lrouter_iterate_by_lsp(op, lflow_input->ls_ports, + lflow_input->lr_ports, + lflow_input->meter_groups, + &match, &actions, + lflows); + ds_destroy(&match); + ds_destroy(&actions); + + /* Update SB multicast groups for the new port. */ + if (!sbmc_flood) { + sbmc_flood = create_sb_multicast_group(ovnsb_txn, + op->od->sb, MC_FLOOD, OVN_MCAST_FLOOD_TUNNEL_KEY); + } + sbrec_multicast_group_update_ports_addvalue(sbmc_flood, op->sb); + + if (!sbmc_flood_l2) { + sbmc_flood_l2 = create_sb_multicast_group(ovnsb_txn, + op->od->sb, MC_FLOOD_L2, + OVN_MCAST_FLOOD_L2_TUNNEL_KEY); + } + sbrec_multicast_group_update_ports_addvalue(sbmc_flood_l2, op->sb); + + if (op->has_unknown) { + if (!sbmc_unknown) { + sbmc_unknown = create_sb_multicast_group(ovnsb_txn, + op->od->sb, MC_UNKNOWN, + OVN_MCAST_UNKNOWN_TUNNEL_KEY); } + sbrec_multicast_group_update_ports_addvalue(sbmc_unknown, + op->sb); + } + + /* Sync the newly added flows to SB. */ + struct lflow_ref_node *lfrn; + LIST_FOR_EACH (lfrn, lflow_list_node, &op->lflows) { + sync_lsp_lflows_to_sb(ovnsb_txn, lflow_input, lflows, + lfrn->lflow); } } - return true; + return true; } static bool @@ -17767,8 +17783,7 @@ northd_init(struct northd_data *data) data->ovn_internal_version_changed = false; sset_init(&data->svc_monitor_lsps); hmap_init(&data->svc_monitor_map); - data->change_tracked = false; - ovs_list_init(&data->tracked_ls_changes.updated); + init_northd_tracked_data(data); } void @@ -17808,6 +17823,7 @@ northd_destroy(struct northd_data *data) destroy_debug_config(); sset_destroy(&data->svc_monitor_lsps); + destroy_northd_tracked_data(data); } void diff --git a/northd/northd.h b/northd/northd.h index 5be7b5384d..23521065e8 100644 --- a/northd/northd.h +++ b/northd/northd.h @@ -83,22 +83,44 @@ struct ovn_datapaths { struct ovn_datapath **array; }; -/* Track what's changed for a single LS. - * Now only track port changes. */ -struct ls_change { - struct ovs_list list_node; - struct ovn_datapath *od; - struct ovs_list added_ports; - struct ovs_list deleted_ports; - struct ovs_list updated_ports; - bool had_only_router_ports; +struct tracked_ovn_ports { + /* tracked created ports. + * hmapx node data is 'struct ovn_port *' */ + struct hmapx created; + + /* tracked updated ports. + * hmapx node data is 'struct ovn_port *' */ + struct hmapx updated; + + /* tracked deleted ports. + * hmapx node data is 'struct ovn_port *' */ + struct hmapx deleted; }; -/* Track what's changed for logical switches. - * Now only track updated ones (added or deleted may be supported in the - * future). */ -struct tracked_ls_changes { - struct ovs_list updated; /* Contains struct ls_change */ +struct tracked_lbs { + /* Tracked created or updated load balancers. + * hmapx node data is 'struct ovn_lb_datapaths' */ + struct hmapx crupdated; + + /* Tracked deleted lbs. + * hmapx node data is 'struct ovn_lb_datapaths' */ + struct hmapx deleted; +}; + +enum northd_tracked_data_type { + NORTHD_TRACKED_NONE, + NORTHD_TRACKED_PORTS = (1 << 0), + NORTHD_TRACKED_LBS = (1 << 1), +}; + +/* Track what's changed in the northd engine node. + * Now only tracks ovn_ports (of vif type) - created, updated + * and deleted. */ +struct northd_tracked_data { + /* Indicates the type of data tracked. One or all of NORTHD_TRACKED_*. */ + enum northd_tracked_data_type type; + struct tracked_ovn_ports trk_lsps; + struct tracked_lbs trk_lbs; }; struct northd_data { @@ -114,10 +136,9 @@ struct northd_data { struct chassis_features features; struct sset svc_monitor_lsps; struct hmap svc_monitor_map; - bool change_tracked; - struct tracked_ls_changes tracked_ls_changes; - bool lb_changed; /* Indicates if load balancers changed or association of - * load balancer to logical switch/router changed. */ + + /* Change tracking data. */ + struct northd_tracked_data trk_data; }; struct lflow_data { @@ -338,9 +359,10 @@ void northd_indices_create(struct northd_data *data, void build_lflows(struct ovsdb_idl_txn *ovnsb_txn, struct lflow_input *input_data, struct hmap *lflows); -bool lflow_handle_northd_ls_changes(struct ovsdb_idl_txn *ovnsb_txn, - struct tracked_ls_changes *, - struct lflow_input *, struct hmap *lflows); +bool lflow_handle_northd_port_changes(struct ovsdb_idl_txn *ovnsb_txn, + struct tracked_ovn_ports *, + struct lflow_input *, + struct hmap *lflows); bool northd_handle_sb_port_binding_changes( const struct sbrec_port_binding_table *, struct hmap *ls_ports); @@ -349,7 +371,8 @@ bool northd_handle_lb_data_changes(struct tracked_lb_data *, struct ovn_datapaths *ls_datapaths, struct ovn_datapaths *lr_datapaths, struct hmap *lb_datapaths_map, - struct hmap *lbgrp_datapaths_map); + struct hmap *lbgrp_datapaths_map, + struct northd_tracked_data *); void build_bfd_table(struct ovsdb_idl_txn *ovnsb_txn, const struct nbrec_bfd_table *, @@ -372,6 +395,23 @@ void sync_lbs(struct ovsdb_idl_txn *, const struct sbrec_load_balancer_table *, bool check_sb_lb_duplicates(const struct sbrec_load_balancer_table *); void sync_pbs(struct ovsdb_idl_txn *, struct hmap *ls_ports); -bool sync_pbs_for_northd_ls_changes(struct tracked_ls_changes *); +bool sync_pbs_for_northd_changed_ovn_ports( struct tracked_ovn_ports *); + +static inline bool +northd_has_tracked_data(struct northd_tracked_data *trk_nd_changes) { + return trk_nd_changes->type != NORTHD_TRACKED_NONE; +} + +static inline bool +northd_has_lbs_in_tracked_data(struct northd_tracked_data *trk_nd_changes) +{ + return (trk_nd_changes->type & NORTHD_TRACKED_LBS); +} + +static inline bool +northd_has_lsps_in_tracked_data(struct northd_tracked_data *trk_nd_changes) +{ + return (trk_nd_changes->type & NORTHD_TRACKED_PORTS); +} #endif /* NORTHD_H */ diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at index a8099786c1..4fc9e6a4e0 100644 --- a/tests/ovn-northd.at +++ b/tests/ovn-northd.at @@ -10632,10 +10632,10 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats lbg1_uuid=$(ovn-nbctl --wait=sb create load_balancer_group name=lbg1) check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute -check_engine_stats lflow recompute nocompute -check_engine_stats sync_to_sb_lb recompute nocompute - +check_engine_stats lflow norecompute nocompute +check_engine_stats sync_to_sb_lb norecompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE + check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats lb1_uuid=$(fetch_column nb:Load_Balancer _uuid) @@ -10662,7 +10662,6 @@ check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute nocompute - CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats @@ -10810,8 +10809,8 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats lbg1_uuid=$(ovn-nbctl --wait=sb create load_balancer_group name=lbg1) check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute -check_engine_stats lflow recompute nocompute -check_engine_stats sync_to_sb_lb recompute nocompute +check_engine_stats lflow norecompute nocompute +check_engine_stats sync_to_sb_lb norecompute nocompute check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb set load_balancer_group . load_balancer="$lb2_uuid,$lb3_uuid,$lb4_uuid" From patchwork Tue Nov 28 02:34:48 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Numan Siddique X-Patchwork-Id: 1869073 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.138; helo=smtp1.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4SfRPr2wkTz1yRy for ; Tue, 28 Nov 2023 13:35:12 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 30D1B81AF3; Tue, 28 Nov 2023 02:35:10 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 30D1B81AF3 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 1yNeEDcsQ5Qp; Tue, 28 Nov 2023 02:35:08 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp1.osuosl.org (Postfix) with ESMTPS id 1C1C981A6D; Tue, 28 Nov 2023 02:35:05 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 1C1C981A6D Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id C9111C0DCE; Tue, 28 Nov 2023 02:35:04 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp4.osuosl.org (smtp4.osuosl.org [140.211.166.137]) by lists.linuxfoundation.org (Postfix) with ESMTP id 52B09C0072 for ; Tue, 28 Nov 2023 02:35:03 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 385D84096E for ; Tue, 28 Nov 2023 02:35:03 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 385D84096E X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id S4a34HXTmZdh for ; Tue, 28 Nov 2023 02:35:02 +0000 (UTC) Received: from relay3-d.mail.gandi.net (relay3-d.mail.gandi.net [217.70.183.195]) by smtp4.osuosl.org (Postfix) with ESMTPS id B0CAD4088F for ; Tue, 28 Nov 2023 02:35:01 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org B0CAD4088F Received: by mail.gandi.net (Postfix) with ESMTPSA id 4510660002; Tue, 28 Nov 2023 02:34:58 +0000 (UTC) From: numans@ovn.org To: dev@openvswitch.org Date: Mon, 27 Nov 2023 21:34:48 -0500 Message-ID: <20231128023448.569590-1-numans@ovn.org> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20231128023410.569539-1-numans@ovn.org> References: <20231128023410.569539-1-numans@ovn.org> MIME-Version: 1.0 X-GND-Sasl: numans@ovn.org Subject: [ovs-dev] [PATCH ovn v3 02/16] tests: Add a couple of tests in ovn-northd for I-P. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Numan Siddique These tests cover scenarios for load balancers and NATs and check for the 'northd' and 'lflow' engine node recompute and compute stats. Signed-off-by: Numan Siddique --- tests/ovn-macros.at | 44 ++++++++ tests/ovn-northd.at | 241 ++++++++++++++++++++++++++++++++++++++------ 2 files changed, 252 insertions(+), 33 deletions(-) diff --git a/tests/ovn-macros.at b/tests/ovn-macros.at index 30646372f6..38bf875184 100644 --- a/tests/ovn-macros.at +++ b/tests/ovn-macros.at @@ -933,6 +933,50 @@ trim_zeros() { sed 's/\(00\)\{1,\}$//' } +# Checks if the provided engine node recomputed or not +# and if it computed or not. +# 1st argument is the engine node. +# 2nd argument is the expected recompute state. +# Possible values are - "norecompute" and "recompute". +# 3rd argument is the expected compute state. +# Possible values are - "nocompute" and "compute". +# +# Eg. 'check_engine_stats lflow recompute nocompute' will verify +# that if the lflow engine node recompute stat is > 0 and +# compute stat is equal to 0. It fails otherwise. +check_engine_stats() { + node=$1 + recompute=$2 + compute=$3 + + echo "Checking engine stats for node $node : recompute - \ +$recompute : compute - $compute" + + node_stat=$(as northd ovn-appctl -t ovn-northd inc-engine/show-stats $node) + # node_stat will be of this format : + # - Node: lflow - recompute: 3 - compute: 0 - abort: 0 + node_recompute_ct=$(echo $node_stat | cut -d '-' -f2 | cut -d ':' -f2) + node_compute_ct=$(echo $node_stat | cut -d '-' -f3 | cut -d ':' -f2) + + if [[ "$recompute" == "norecompute" ]]; then + # node should not be recomputed + echo "Expecting $node recompute count - $node_recompute_ct to be 0" + check test "$node_recompute_ct" -eq "0" + else + echo "Expecting $node recompute count - $node_recompute_ct not to be 0" + check test "$node_recompute_ct" -ne "0" + fi + + if [[ "$compute" == "nocompute" ]]; then + # node should not be computed + echo "Expecting $node compute count - $node_compute_ct to be 0" + check test "$node_compute_ct" -eq "0" + else + echo "Expecting $node compute count - $node_compute_ct not to be 0" + check test "$node_compute_ct" -ne "0" + fi +} + OVS_END_SHELL_HELPERS m4_define([OVN_POPULATE_ARP], [AT_CHECK(ovn_populate_arp__, [0], [ignore])]) diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at index 4fc9e6a4e0..7204afe67a 100644 --- a/tests/ovn-northd.at +++ b/tests/ovn-northd.at @@ -10378,39 +10378,6 @@ OVN_FOR_EACH_NORTHD_NO_HV([ AT_SETUP([Load balancer incremental processing]) ovn_start -check_engine_stats() { - node=$1 - recompute=$2 - compute=$3 - - echo "__file__:__line__: Checking engine stats for node $node : recompute - \ -$recompute : compute - $compute" - - node_stat=$(as northd ovn-appctl -t NORTHD_TYPE inc-engine/show-stats $node) - # node_stat will be of this format : - # - Node: lflow - recompute: 3 - compute: 0 - abort: 0 - node_recompute_ct=$(echo $node_stat | cut -d '-' -f2 | cut -d ':' -f2) - node_compute_ct=$(echo $node_stat | cut -d '-' -f3 | cut -d ':' -f2) - - if [[ "$recompute" == "norecompute" ]]; then - # node should not be recomputed - echo "Expecting $node recompute count - $node_recompute_ct to be 0" - check test "$node_recompute_ct" -eq "0" - else - echo "Expecting $node recompute count - $node_recompute_ct not to be 0" - check test "$node_recompute_ct" -ne "0" - fi - - if [[ "$compute" == "nocompute" ]]; then - # node should not be computed - echo "Expecting $node compute count - $node_compute_ct to be 0" - check test "$node_compute_ct" -eq "0" - else - echo "Expecting $node compute count - $node_compute_ct not to be 0" - check test "$node_compute_ct" -ne "0" - fi -} - # Test I-P for load balancers. # Presently ovn-northd handles I-P for NB LBs in northd_lb_data engine node # only. @@ -10906,3 +10873,211 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE AT_CLEANUP ]) + +OVN_FOR_EACH_NORTHD_NO_HV([ +AT_SETUP([Load balancer incremental processing with stateless ACLs]) +ovn_start + +# Test I-P for load balancers. +# Presently ovn-northd handles I-P for NB LBs in northd_lb_data engine node +# only. +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb lb-add lb1 10.0.0.10:80 10.0.0.3:80 + +check ovn-nbctl ls-add sw0 +check ovn-nbctl acl-add sw0 from-lport 1 1 allow-stateless +check ovn-nbctl --wait=sb acl-add sw0 to-lport 1 1 allow-stateless + +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb ls-lb-add sw0 lb1 +check_engine_stats lb_data norecompute compute +check_engine_stats northd norecompute compute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +# Clear the VIPs of lb1 +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb clear load_balancer . vips +check_engine_stats lb_data norecompute compute +check_engine_stats northd norecompute compute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb lb-del lb1 +check_engine_stats lb_data norecompute compute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +AT_CLEANUP +]) + +OVN_FOR_EACH_NORTHD_NO_HV([ +AT_SETUP([Logical router incremental processing for NAT]) + +ovn_start + +net_add n1 +sim_add hv1 +as hv1 +ovs-vsctl add-br br-phys +ovn_attach n1 br-phys 192.168.0.11 + +ovn-sbctl chassis-add gw1 geneve 127.0.0.1 + +check ovn-nbctl ls-add sw0 +check ovn-nbctl lsp-add sw0 sw0p1 -- lsp-set-addresses sw0p1 "00:00:20:20:12:01 10.0.0.4" + +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb lr-add lr0 +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute + +# Adding a logical router port should result in recompute +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl lrp-add lr0 lr0-sw0 00:00:00:00:ff:01 10.0.0.1/24 +# for northd engine there will be both recompute and compute +# first it will be recompute to handle lr0-sw0 and then a compute +# for the SB port binding change. +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute + +ovn-nbctl lsp-add sw0 sw0-lr0 +ovn-nbctl lsp-set-type sw0-lr0 router +ovn-nbctl lsp-set-addresses sw0-lr0 00:00:00:00:ff:01 +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb lsp-set-options sw0-lr0 router-port=lr0-sw0 +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute + +ovn-nbctl ls-add public +ovn-nbctl lrp-add lr0 lr0-public 00:00:20:20:12:13 172.168.0.100/24 +ovn-nbctl lsp-add public public-lr0 +ovn-nbctl lsp-set-type public-lr0 router +ovn-nbctl lsp-set-addresses public-lr0 router +ovn-nbctl lsp-set-options public-lr0 router-port=lr0-public + +# localnet port +ovn-nbctl lsp-add public ln-public +ovn-nbctl lsp-set-type ln-public localnet +ovn-nbctl lsp-set-addresses ln-public unknown +ovn-nbctl lsp-set-options ln-public network_name=public + +# schedule the gw router port to a chassis. Change the name of the chassis +ovn-nbctl --wait=hv lrp-set-gateway-chassis lr0-public hv1 20 + +# Modify a logical router port and it should result in recompute. +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl set logical_router_port lr0-sw0 options:foo=bar +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute + +check as northd ovn-appctl -t NORTHD_TYPE vlog/set dbg + +# Do checks for NATs. +# Add a NAT. This should not result in recompute of both northd and lflow +# engine nodes. +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb lr-nat-add lr0 dnat_and_snat 172.168.0.110 10.0.0.4 +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +# Update the NAT options column +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb set NAT . options:foo=bar +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +# Update the NAT external_ip column +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb set NAT . external_ip=172.168.0.120 +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +# Update the NAT logical_ip column +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb set NAT . logical_ip=10.0.0.10 +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +# Update the NAT type +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb set NAT . type=snat +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +# Create a dnat_and_snat NAT with external_mac and logical_port +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb lr-nat-add lr0 dnat_and_snat 172.168.0.110 10.0.0.4 sw0p1 30:54:00:00:00:03 +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +nat2_uuid=$(ovn-nbctl --bare --columns _uuid find nat logical_ip=10.0.0.4) + +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb set NAT $nat2_uuid external_mac='"30:54:00:00:00:04"' +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +# Create a load balancer and add the lb vip as NAT +check ovn-nbctl lb-add lb1 172.168.0.140 10.0.0.20 +check ovn-nbctl lb-add lb2 172.168.0.150:80 10.0.0.40:8080 +check ovn-nbctl lr-lb-add lr0 lb1 +check ovn-nbctl lr-lb-add lr0 lb2 + +# lflow engine should recompute since the nat ip 172.168.0.140 +# is a lb vip. +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb lr-nat-add lr0 dnat_and_snat 172.168.0.140 10.0.0.20 +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb lr-nat-add lr0 dnat_and_snat 172.168.0.150 10.0.0.41 +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb lr-nat-del lr0 dnat_and_snat 172.168.0.150 +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb lr-nat-del lr0 dnat_and_snat 172.168.0.140 +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +# Delete the NAT +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb clear logical_router lr0 nat +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute +check_engine_stats sync_to_sb_pb recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +# Create router Policy +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb lr-policy-add lr0 10 "ip4.src == 10.0.0.3" reroute 172.168.0.101,172.168.0.102 +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute + +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb lr-policy-del lr0 10 "ip4.src == 10.0.0.3" +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute + +OVN_CLEANUP([hv1]) +AT_CLEANUP +]) From patchwork Tue Nov 28 02:35:00 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Numan Siddique X-Patchwork-Id: 1869074 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::137; helo=smtp4.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp4.osuosl.org (smtp4.osuosl.org [IPv6:2605:bc80:3010::137]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4SfRQJ06VFz1yRy for ; Tue, 28 Nov 2023 13:35:35 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id C2548410AF; Tue, 28 Nov 2023 02:35:32 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org C2548410AF X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ACf3uaG2xNUF; Tue, 28 Nov 2023 02:35:30 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp4.osuosl.org (Postfix) with ESMTPS id 9E61A409C3; Tue, 28 Nov 2023 02:35:29 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 9E61A409C3 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 82FB4C0072; Tue, 28 Nov 2023 02:35:29 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp2.osuosl.org (smtp2.osuosl.org [IPv6:2605:bc80:3010::133]) by lists.linuxfoundation.org (Postfix) with ESMTP id E0425C0039 for ; Tue, 28 Nov 2023 02:35:28 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 02FE24090E for ; Tue, 28 Nov 2023 02:35:20 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 02FE24090E X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Ih9AcHXlnJC5 for ; Tue, 28 Nov 2023 02:35:17 +0000 (UTC) Received: from relay9-d.mail.gandi.net (relay9-d.mail.gandi.net [IPv6:2001:4b98:dc4:8::229]) by smtp2.osuosl.org (Postfix) with ESMTPS id 0A2A7409B5 for ; Tue, 28 Nov 2023 02:35:16 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 0A2A7409B5 Received: by mail.gandi.net (Postfix) with ESMTPSA id 5D7C3FF803; Tue, 28 Nov 2023 02:35:13 +0000 (UTC) From: numans@ovn.org To: dev@openvswitch.org Date: Mon, 27 Nov 2023 21:35:00 -0500 Message-ID: <20231128023500.569653-1-numans@ovn.org> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20231128023410.569539-1-numans@ovn.org> References: <20231128023410.569539-1-numans@ovn.org> MIME-Version: 1.0 X-GND-Sasl: numans@ovn.org Subject: [ovs-dev] [PATCH ovn v3 03/16] northd: Move router ports SB PB options sync to sync_to_sb_pb node. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Numan Siddique It also moves the logical router port IPv6 prefix delegation updates to "sync-from-sb" engine node. With these changes, northd engine node doesn't need to do much for SB Port binding changes other than updating 'op->sb'. And there is no need to fall back to recompute. Prior to this patch northd_handle_sb_port_binding_changes() was returning false for all SB port binding updates except for the VIF PBs. This patch now handles updates to all the SB port binding by setting 'op->sb'. Signed-off-by: Numan Siddique --- northd/en-northd.c | 2 +- northd/en-sync-sb.c | 3 +- northd/northd.c | 294 ++++++++++++++++++++++++++------------------ northd/northd.h | 6 +- tests/ovn-northd.at | 160 ++++++++++++++++++++++-- 5 files changed, 333 insertions(+), 132 deletions(-) diff --git a/northd/en-northd.c b/northd/en-northd.c index 28559ed211..677b2b1ab0 100644 --- a/northd/en-northd.c +++ b/northd/en-northd.c @@ -189,7 +189,7 @@ northd_sb_port_binding_handler(struct engine_node *node, northd_get_input_data(node, &input_data); if (!northd_handle_sb_port_binding_changes( - input_data.sbrec_port_binding_table, &nd->ls_ports)) { + input_data.sbrec_port_binding_table, &nd->ls_ports, &nd->lr_ports)) { return false; } diff --git a/northd/en-sync-sb.c b/northd/en-sync-sb.c index 3aaab8d005..45be7ddbcb 100644 --- a/northd/en-sync-sb.c +++ b/northd/en-sync-sb.c @@ -288,7 +288,8 @@ en_sync_to_sb_pb_run(struct engine_node *node, void *data OVS_UNUSED) const struct engine_context *eng_ctx = engine_get_context(); struct northd_data *northd_data = engine_get_input_data("northd", node); - sync_pbs(eng_ctx->ovnsb_idl_txn, &northd_data->ls_ports); + sync_pbs(eng_ctx->ovnsb_idl_txn, &northd_data->ls_ports, + &northd_data->lr_ports); engine_set_node_state(node, EN_UPDATED); } diff --git a/northd/northd.c b/northd/northd.c index abb06bfc6a..aa97dfc7ed 100644 --- a/northd/northd.c +++ b/northd/northd.c @@ -3419,6 +3419,9 @@ ovn_port_update_sbrec(struct ovsdb_idl_txn *ovnsb_txn, { sbrec_port_binding_set_datapath(op->sb, op->od->sb); if (op->nbrp) { + /* Note: SB port binding options for router ports are set in + * sync_pbs(). */ + /* If the router is for l3 gateway, it resides on a chassis * and its port type is "l3gateway". */ const char *chassis_name = smap_get(&op->od->nbr->options, "chassis"); @@ -3430,15 +3433,11 @@ ovn_port_update_sbrec(struct ovsdb_idl_txn *ovnsb_txn, sbrec_port_binding_set_type(op->sb, "patch"); } - struct smap new; - smap_init(&new); if (is_cr_port(op)) { ovs_assert(sbrec_chassis_by_name); ovs_assert(sbrec_chassis_by_hostname); ovs_assert(sbrec_ha_chassis_grp_by_name); ovs_assert(active_ha_chassis_grps); - const char *redirect_type = smap_get(&op->nbrp->options, - "redirect-type"); if (op->nbrp->ha_chassis_group) { if (op->nbrp->n_gateway_chassis) { @@ -3480,49 +3479,8 @@ ovn_port_update_sbrec(struct ovsdb_idl_txn *ovnsb_txn, /* Delete the legacy gateway_chassis from the pb. */ sbrec_port_binding_set_gateway_chassis(op->sb, NULL, 0); } - smap_add(&new, "distributed-port", op->nbrp->name); - - bool always_redirect = - !op->od->has_distributed_nat && - !l3dgw_port_has_associated_vtep_lports(op->l3dgw_port); - - if (redirect_type) { - smap_add(&new, "redirect-type", redirect_type); - /* XXX Why can't we enable always-redirect when redirect-type - * is bridged? */ - if (!strcmp(redirect_type, "bridged")) { - always_redirect = false; - } - } - - if (always_redirect) { - smap_add(&new, "always-redirect", "true"); - } - } else { - if (op->peer) { - smap_add(&new, "peer", op->peer->key); - if (op->nbrp->ha_chassis_group || - op->nbrp->n_gateway_chassis) { - char *redirect_name = - ovn_chassis_redirect_name(op->nbrp->name); - smap_add(&new, "chassis-redirect-port", redirect_name); - free(redirect_name); - } - } - if (chassis_name) { - smap_add(&new, "l3gateway-chassis", chassis_name); - } - } - - const char *ipv6_pd_list = smap_get(&op->sb->options, - "ipv6_ra_pd_list"); - if (ipv6_pd_list) { - smap_add(&new, "ipv6_ra_pd_list", ipv6_pd_list); } - sbrec_port_binding_set_options(op->sb, &new); - smap_destroy(&new); - sbrec_port_binding_set_parent_port(op->sb, NULL); sbrec_port_binding_set_tag(op->sb, NULL, 0); @@ -4752,12 +4710,14 @@ check_sb_lb_duplicates(const struct sbrec_load_balancer_table *table) return duplicates; } -/* Syncs the SB port binding for the ovn_port 'op'. Caller should make sure - * that the OVN SB IDL txn is not NULL. Presently it only syncs the nat - * column of port binding corresponding to the 'op->nbsp' */ +/* Syncs the SB port binding for the ovn_port 'op' of a logical switch port. + * Caller should make sure that the OVN SB IDL txn is not NULL. Presently it + * only syncs the nat column of port binding corresponding to the 'op->nbsp' */ static void -sync_pb_for_op(struct ovn_port *op) +sync_pb_for_lsp(struct ovn_port *op) { + ovs_assert(op->nbsp); + if (lsp_is_router(op->nbsp)) { const char *chassis = NULL; if (op->peer && op->peer->od && op->peer->od->nbr) { @@ -4879,18 +4839,86 @@ sync_pb_for_op(struct ovn_port *op) } } +/* Syncs the SB port binding for the ovn_port 'op' of a logical router port. + * Caller should make sure that the OVN SB IDL txn is not NULL. Presently it + * only sets the port binding options column for the router ports */ +static void +sync_pb_for_lrp(struct ovn_port *op) +{ + ovs_assert(op->nbrp); + + struct smap new; + smap_init(&new); + + const char *chassis_name = smap_get(&op->od->nbr->options, "chassis"); + if (is_cr_port(op)) { + smap_add(&new, "distributed-port", op->nbrp->name); + + bool always_redirect = + !op->od->has_distributed_nat && + !l3dgw_port_has_associated_vtep_lports(op->l3dgw_port); + + const char *redirect_type = smap_get(&op->nbrp->options, + "redirect-type"); + if (redirect_type) { + smap_add(&new, "redirect-type", redirect_type); + /* XXX Why can't we enable always-redirect when redirect-type + * is bridged? */ + if (!strcmp(redirect_type, "bridged")) { + always_redirect = false; + } + } + + if (always_redirect) { + smap_add(&new, "always-redirect", "true"); + } + } else { + if (op->peer) { + smap_add(&new, "peer", op->peer->key); + if (op->nbrp->ha_chassis_group || + op->nbrp->n_gateway_chassis) { + char *redirect_name = + ovn_chassis_redirect_name(op->nbrp->name); + smap_add(&new, "chassis-redirect-port", redirect_name); + free(redirect_name); + } + } + if (chassis_name) { + smap_add(&new, "l3gateway-chassis", chassis_name); + } + } + + const char *ipv6_pd_list = smap_get(&op->sb->options, "ipv6_ra_pd_list"); + if (ipv6_pd_list) { + smap_add(&new, "ipv6_ra_pd_list", ipv6_pd_list); + } + + sbrec_port_binding_set_options(op->sb, &new); + smap_destroy(&new); +} + +static void ovn_update_ipv6_options(struct hmap *lr_ports); +static void ovn_update_ipv6_opt_for_op(struct ovn_port *op); + /* Sync the SB Port bindings which needs to be updated. * Presently it syncs the nat column of port bindings corresponding to * the logical switch ports. */ void -sync_pbs(struct ovsdb_idl_txn *ovnsb_idl_txn, struct hmap *ls_ports) +sync_pbs(struct ovsdb_idl_txn *ovnsb_idl_txn, struct hmap *ls_ports, + struct hmap *lr_ports) { ovs_assert(ovnsb_idl_txn); struct ovn_port *op; HMAP_FOR_EACH (op, key_node, ls_ports) { - sync_pb_for_op(op); + sync_pb_for_lsp(op); + } + + HMAP_FOR_EACH (op, key_node, lr_ports) { + sync_pb_for_lrp(op); } + + ovn_update_ipv6_options(lr_ports); } /* Sync the SB Port bindings for the added and updated logical switch ports @@ -4902,12 +4930,14 @@ sync_pbs_for_northd_changed_ovn_ports( struct tracked_ovn_ports *trk_ovn_ports) struct ovn_port *op; HMAPX_FOR_EACH (hmapx_node, &trk_ovn_ports->created) { op = hmapx_node->data; - sync_pb_for_op(op); + ovs_assert(op->nbsp); + sync_pb_for_lsp(op); } HMAPX_FOR_EACH (hmapx_node, &trk_ovn_ports->updated) { op = hmapx_node->data; - sync_pb_for_op(op); + ovs_assert(op->nbsp); + sync_pb_for_lsp(op); } return true; @@ -5657,20 +5687,31 @@ fail: bool northd_handle_sb_port_binding_changes( const struct sbrec_port_binding_table *sbrec_port_binding_table, - struct hmap *ls_ports) + struct hmap *ls_ports, struct hmap *lr_ports) { const struct sbrec_port_binding *pb; static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 5); SBREC_PORT_BINDING_TABLE_FOR_EACH_TRACKED (pb, sbrec_port_binding_table) { - /* Return false if the 'pb' belongs to a router port. We don't handle - * I-P for router ports yet. */ - if (is_pb_router_type(pb)) { - return false; + bool is_router_port = is_pb_router_type(pb); + struct ovn_port *op = NULL; + + if (is_router_port) { + /* A router port binding 'pb' can belong to + * - a logical switch port of type router or + * - a logical router port. + * + * So, first search in lr_ports hmap. If not found, search in + * ls_ports hmap. + * */ + op = ovn_port_find(lr_ports, pb->logical_port); } - struct ovn_port *op = ovn_port_find(ls_ports, pb->logical_port); - if (op && !op->lsp_can_be_inc_processed) { - return false; + if (!op) { + op = ovn_port_find(ls_ports, pb->logical_port); + + if (op) { + is_router_port = false; + } } if (sbrec_port_binding_is_new(pb)) { @@ -5679,7 +5720,8 @@ northd_handle_sb_port_binding_changes( * pointer in northd data. Fallback to recompute otherwise. */ if (!op) { VLOG_WARN_RL(&rl, "A port-binding for %s is created but the " - "LSP is not found.", pb->logical_port); + "%s is not found.", pb->logical_port, + is_router_port ? "LRP" : "LSP"); return false; } op->sb = pb; @@ -5690,17 +5732,28 @@ northd_handle_sb_port_binding_changes( * sb idl pointers and other unexpected behavior. */ if (op) { VLOG_WARN_RL(&rl, "A port-binding for %s is deleted but the " - "LSP still exists.", pb->logical_port); + "LSP/LRP still exists.", pb->logical_port); return false; } } else { - /* The PB is updated, most likely because of binding/unbinding - * to/from a chassis, and we can ignore the change (updating NB - * "up" will be handled in the engine node "sync_from_sb"). + /* The PB is updated. + * For an LSP PB it is most likely because of + * binding/unbinding to/from a chassis, and we can ignore the + * change (updating NB "up" will be handled in the engine node + * "sync_from_sb"). + * + * For an LRP PB, it is most likely because of + * - IPv6 prefix delagation updates from ovn-controller. + * This update is handled in "sync_from_sb" node. + * - ha chassis group and this can be ignored. + * + * All other changes can be ignored. + * * Fallback to recompute for anything unexpected. */ if (!op) { VLOG_WARN_RL(&rl, "A port-binding for %s is updated but the " - "LSP is not found.", pb->logical_port); + "%s is not found.", pb->logical_port, + is_router_port ? "LRP" : "LSP"); return false; } if (op->sb != pb) { @@ -7837,67 +7890,70 @@ static void copy_ra_to_sb(struct ovn_port *op, const char *address_mode); static void -ovn_update_ipv6_options(struct hmap *lr_ports) +ovn_update_ipv6_opt_for_op(struct ovn_port *op) { - struct ovn_port *op; - HMAP_FOR_EACH (op, key_node, lr_ports) { - ovs_assert(op->nbrp); - - if (op->nbrp->peer || !op->peer) { - continue; - } + if (op->nbrp->peer || !op->peer) { + return; + } - if (!op->lrp_networks.n_ipv6_addrs) { - continue; - } + if (!op->lrp_networks.n_ipv6_addrs) { + return; + } - struct smap options; - smap_clone(&options, &op->sb->options); + struct smap options; + smap_clone(&options, &op->sb->options); - /* enable IPv6 prefix delegation */ - bool prefix_delegation = smap_get_bool(&op->nbrp->options, + /* enable IPv6 prefix delegation */ + bool prefix_delegation = smap_get_bool(&op->nbrp->options, "prefix_delegation", false); - if (!lrport_is_enabled(op->nbrp)) { - prefix_delegation = false; - } - if (smap_get_bool(&options, "ipv6_prefix_delegation", - false) != prefix_delegation) { - smap_add(&options, "ipv6_prefix_delegation", - prefix_delegation ? "true" : "false"); - } + if (!lrport_is_enabled(op->nbrp)) { + prefix_delegation = false; + } + if (smap_get_bool(&options, "ipv6_prefix_delegation", + false) != prefix_delegation) { + smap_add(&options, "ipv6_prefix_delegation", + prefix_delegation ? "true" : "false"); + } - bool ipv6_prefix = smap_get_bool(&op->nbrp->options, + bool ipv6_prefix = smap_get_bool(&op->nbrp->options, "prefix", false); - if (!lrport_is_enabled(op->nbrp)) { - ipv6_prefix = false; - } - if (smap_get_bool(&options, "ipv6_prefix", false) != ipv6_prefix) { - smap_add(&options, "ipv6_prefix", - ipv6_prefix ? "true" : "false"); - } - sbrec_port_binding_set_options(op->sb, &options); + if (!lrport_is_enabled(op->nbrp)) { + ipv6_prefix = false; + } + if (smap_get_bool(&options, "ipv6_prefix", false) != ipv6_prefix) { + smap_add(&options, "ipv6_prefix", ipv6_prefix ? "true" : "false"); + } + sbrec_port_binding_set_options(op->sb, &options); - smap_destroy(&options); + smap_destroy(&options); - const char *address_mode = smap_get( - &op->nbrp->ipv6_ra_configs, "address_mode"); + const char *address_mode = smap_get(&op->nbrp->ipv6_ra_configs, + "address_mode"); - if (!address_mode) { - continue; - } - if (strcmp(address_mode, "slaac") && - strcmp(address_mode, "dhcpv6_stateful") && - strcmp(address_mode, "dhcpv6_stateless")) { - static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 5); - VLOG_WARN_RL(&rl, "Invalid address mode [%s] defined", - address_mode); - continue; - } + if (!address_mode) { + return; + } + if (strcmp(address_mode, "slaac") && + strcmp(address_mode, "dhcpv6_stateful") && + strcmp(address_mode, "dhcpv6_stateless")) { + static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 5); + VLOG_WARN_RL(&rl, "Invalid address mode [%s] defined", + address_mode); + return; + } - if (smap_get_bool(&op->nbrp->ipv6_ra_configs, "send_periodic", - false)) { - copy_ra_to_sb(op, address_mode); - } + if (smap_get_bool(&op->nbrp->ipv6_ra_configs, "send_periodic", false)) { + copy_ra_to_sb(op, address_mode); + } +} + +static void +ovn_update_ipv6_options(struct hmap *lr_ports) +{ + struct ovn_port *op; + HMAP_FOR_EACH (op, key_node, lr_ports) { + ovs_assert(op->nbrp); + ovn_update_ipv6_opt_for_op(op); } } @@ -17958,8 +18014,6 @@ ovnnb_db_run(struct northd_input *input_data, &data->lr_ports); stopwatch_stop(BUILD_LFLOWS_CTX_STOPWATCH_NAME, time_msec()); stopwatch_start(CLEAR_LFLOWS_CTX_STOPWATCH_NAME, time_msec()); - ovn_update_ipv6_options(&data->lr_ports); - ovn_update_ipv6_prefix(&data->lr_ports); sync_mirrors(ovnsb_txn, input_data->nbrec_mirror_table, input_data->sbrec_mirror_table); @@ -18290,6 +18344,8 @@ ovnsb_db_run(struct ovsdb_idl_txn *ovnnb_txn, &ha_ref_chassis_map); } shash_destroy(&ha_ref_chassis_map); + + ovn_update_ipv6_prefix(lr_ports); } const char * diff --git a/northd/northd.h b/northd/northd.h index 23521065e8..233dca8084 100644 --- a/northd/northd.h +++ b/northd/northd.h @@ -364,7 +364,8 @@ bool lflow_handle_northd_port_changes(struct ovsdb_idl_txn *ovnsb_txn, struct lflow_input *, struct hmap *lflows); bool northd_handle_sb_port_binding_changes( - const struct sbrec_port_binding_table *, struct hmap *ls_ports); + const struct sbrec_port_binding_table *, struct hmap *ls_ports, + struct hmap *lr_ports); struct tracked_lb_data; bool northd_handle_lb_data_changes(struct tracked_lb_data *, @@ -394,7 +395,8 @@ void sync_lbs(struct ovsdb_idl_txn *, const struct sbrec_load_balancer_table *, struct chassis_features *chassis_features); bool check_sb_lb_duplicates(const struct sbrec_load_balancer_table *); -void sync_pbs(struct ovsdb_idl_txn *, struct hmap *ls_ports); +void sync_pbs(struct ovsdb_idl_txn *, struct hmap *ls_ports, + struct hmap *lr_ports); bool sync_pbs_for_northd_changed_ovn_ports( struct tracked_ovn_ports *); static inline bool diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at index 7204afe67a..c2ba96e753 100644 --- a/tests/ovn-northd.at +++ b/tests/ovn-northd.at @@ -10061,7 +10061,7 @@ check ovn-nbctl --wait=hv lsp-add ls0 lsp0-0 -- lsp-set-addresses lsp0-0 "unknow ovs-vsctl add-port br-int lsp0-0 -- set interface lsp0-0 external_ids:iface-id=lsp0-0 wait_for_ports_up check ovn-nbctl --wait=hv sync -check_recompute_counter 4 5 5 5 5 5 +check_recompute_counter 3 4 3 4 3 4 check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=hv lsp-add ls0 lsp0-1 -- lsp-set-addresses lsp0-1 "aa:aa:aa:00:00:01 192.168.0.11" @@ -10223,6 +10223,126 @@ OVN_CLEANUP([hv1]) AT_CLEANUP ]) +OVN_FOR_EACH_NORTHD_NO_HV([ +AT_SETUP([SB Port binding incremental processing]) +ovn_start + +check_recompute_counter() { + northd_recomp=$(as northd ovn-appctl -t NORTHD_TYPE inc-engine/show-stats northd recompute) + echo "northd recompute count - $northd_recomp" + AT_CHECK([test x$northd_recomp = x$1]) + + lflow_recomp=$(as northd ovn-appctl -t NORTHD_TYPE inc-engine/show-stats lflow recompute) + echo "lflow recompute count - $lflow_recomp" + AT_CHECK([test x$lflow_recomp = x$2]) +} + +net_add n1 +sim_add hv1 +as hv1 +ovs-vsctl add-br br-phys +ovn_attach n1 br-phys 192.168.0.11 + +check ovn-nbctl --wait=sb ls-add ls0 +check ovn-nbctl --wait=sb lr-add lr0 + +# Test normal VIF port +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl lsp-add ls0 p1 +check_recompute_counter 0 0 +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl lsp-set-addresses p1 "00:00:00:00:01:01 10.0.0.3" +check_recompute_counter 0 0 +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +check as northd ovn-appctl -t NORTHD_TYPE vlog/set info + +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-sbctl lsp-bind p1 hv1 +check_recompute_counter 0 0 +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +# Test lsp of type router +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl lsp-add ls0 rp -- lsp-set-type rp router + +# northd engine recomputes twice. Both the times for handling NB logical switch port +# changes and not because of SB port binding changes. This is because ovn-northd +# sets the "up" to true. +check_recompute_counter 2 2 +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +# Set some options to 'rp'. northd should only recompute once. +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl lsp-set-options rp foo=bar +check_recompute_counter 1 1 +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +# Test lsp of type external +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl lsp-add ls0 e1 -- lsp-set-type e1 external +check_recompute_counter 2 2 +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +# Set some options to 'e1'. northd should only recompute once. +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl lsp-set-options e1 foo=bar +check_recompute_counter 1 1 +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +# Test lrp +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl lrp-add lr0 lrp 00:00:02:01:02:03 10.0.0.1/24 +check_recompute_counter 1 1 +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +# Set some options on 'lrp'. northd should only recompute once. +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl lrp-set-options lrp route_table=rtb-1 +check_recompute_counter 1 1 +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +# Make lrp a gateway port +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl lrp-set-gateway-chassis lrp hv1 +# There will be 3 recomputes of northd engine node +# 1. missing handler for input NB_logical_router +# 2. missing handler for input SB_ha_chassis_group +# 3. missing handler for input NB_logical_router when ovn-northd +# updates the hosting-chassis option in NB_logical_router_port. +check_recompute_counter 3 3 +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl clear logical_router_port lrp gateway_chassis +# There will be 2 recomputes of northd engine node +# 1. missing handler for input NB_logical_router +# 2. missing handler for input SB_ha_chassis_group +check_recompute_counter 2 2 +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +# Delete some of the ports. +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl lsp-del p1 +check_recompute_counter 0 0 +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl lsp-del e1 +check_recompute_counter 1 1 +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl lrp-del lrp +check_recompute_counter 1 1 +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +OVN_CLEANUP([hv1]) +AT_CLEANUP +]) + OVN_FOR_EACH_NORTHD_NO_HV([ AT_SETUP([ACL/Meter incremental processing - no northd recompute]) ovn_start @@ -10257,6 +10377,7 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE(1) AT_CLEANUP ]) + OVN_FOR_EACH_NORTHD_NO_HV([ AT_SETUP([check fip and lb flows]) AT_KEYWORDS([fip-lb-flows]) @@ -10459,7 +10580,7 @@ ovn-nbctl lsp-set-addresses sw0-lr0 00:00:00:00:ff:01 check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats ovn-nbctl --wait=sb lsp-set-options sw0-lr0 router-port=lr0-sw0 check_engine_stats lb_data norecompute compute -check_engine_stats northd recompute nocompute +check_engine_stats northd recompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute nocompute @@ -10927,12 +11048,14 @@ ovn_attach n1 br-phys 192.168.0.11 ovn-sbctl chassis-add gw1 geneve 127.0.0.1 check ovn-nbctl ls-add sw0 -check ovn-nbctl lsp-add sw0 sw0p1 -- lsp-set-addresses sw0p1 "00:00:20:20:12:01 10.0.0.4" +check ovn-nbctl --wait=sb lsp-add sw0 sw0p1 -- lsp-set-addresses sw0p1 "00:00:20:20:12:01 10.0.0.4" check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lr-add lr0 check_engine_stats northd recompute nocompute +check_engine_stats sync_to_sb_pb recompute nocompute check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE # Adding a logical router port should result in recompute check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats @@ -10940,16 +11063,20 @@ check ovn-nbctl lrp-add lr0 lr0-sw0 00:00:00:00:ff:01 10.0.0.1/24 # for northd engine there will be both recompute and compute # first it will be recompute to handle lr0-sw0 and then a compute # for the SB port binding change. -check_engine_stats northd recompute nocompute +check_engine_stats northd recompute compute +check_engine_stats sync_to_sb_pb recompute nocompute check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE ovn-nbctl lsp-add sw0 sw0-lr0 ovn-nbctl lsp-set-type sw0-lr0 router ovn-nbctl lsp-set-addresses sw0-lr0 00:00:00:00:ff:01 check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lsp-set-options sw0-lr0 router-port=lr0-sw0 -check_engine_stats northd recompute nocompute +check_engine_stats northd recompute compute +check_engine_stats sync_to_sb_pb recompute nocompute check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE ovn-nbctl ls-add public ovn-nbctl lrp-add lr0 lr0-public 00:00:20:20:12:13 172.168.0.100/24 @@ -10971,9 +11098,9 @@ ovn-nbctl --wait=hv lrp-set-gateway-chassis lr0-public hv1 20 check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl set logical_router_port lr0-sw0 options:foo=bar check_engine_stats northd recompute nocompute +check_engine_stats sync_to_sb_pb recompute nocompute check_engine_stats lflow recompute nocompute - -check as northd ovn-appctl -t NORTHD_TYPE vlog/set dbg +CHECK_NO_CHANGE_AFTER_RECOMPUTE # Do checks for NATs. # Add a NAT. This should not result in recompute of both northd and lflow @@ -10982,6 +11109,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lr-nat-add lr0 dnat_and_snat 172.168.0.110 10.0.0.4 check_engine_stats northd recompute nocompute check_engine_stats lflow recompute nocompute +check_engine_stats sync_to_sb_pb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Update the NAT options column @@ -10989,6 +11117,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb set NAT . options:foo=bar check_engine_stats northd recompute nocompute check_engine_stats lflow recompute nocompute +check_engine_stats sync_to_sb_pb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Update the NAT external_ip column @@ -10996,6 +11125,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb set NAT . external_ip=172.168.0.120 check_engine_stats northd recompute nocompute check_engine_stats lflow recompute nocompute +check_engine_stats sync_to_sb_pb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Update the NAT logical_ip column @@ -11003,6 +11133,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb set NAT . logical_ip=10.0.0.10 check_engine_stats northd recompute nocompute check_engine_stats lflow recompute nocompute +check_engine_stats sync_to_sb_pb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Update the NAT type @@ -11010,13 +11141,15 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb set NAT . type=snat check_engine_stats northd recompute nocompute check_engine_stats lflow recompute nocompute +check_engine_stats sync_to_sb_pb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Create a dnat_and_snat NAT with external_mac and logical_port check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lr-nat-add lr0 dnat_and_snat 172.168.0.110 10.0.0.4 sw0p1 30:54:00:00:00:03 -check_engine_stats northd recompute nocompute +check_engine_stats northd recompute compute check_engine_stats lflow recompute nocompute +check_engine_stats sync_to_sb_pb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE nat2_uuid=$(ovn-nbctl --bare --columns _uuid find nat logical_ip=10.0.0.4) @@ -11025,6 +11158,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb set NAT $nat2_uuid external_mac='"30:54:00:00:00:04"' check_engine_stats northd recompute nocompute check_engine_stats lflow recompute nocompute +check_engine_stats sync_to_sb_pb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Create a load balancer and add the lb vip as NAT @@ -11038,31 +11172,35 @@ check ovn-nbctl lr-lb-add lr0 lb2 check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lr-nat-add lr0 dnat_and_snat 172.168.0.140 10.0.0.20 check_engine_stats northd recompute nocompute +check_engine_stats sync_to_sb_pb recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lr-nat-add lr0 dnat_and_snat 172.168.0.150 10.0.0.41 check_engine_stats northd recompute nocompute +check_engine_stats sync_to_sb_pb recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lr-nat-del lr0 dnat_and_snat 172.168.0.150 check_engine_stats northd recompute nocompute +check_engine_stats sync_to_sb_pb recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lr-nat-del lr0 dnat_and_snat 172.168.0.140 check_engine_stats northd recompute nocompute +check_engine_stats sync_to_sb_pb recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Delete the NAT check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb clear logical_router lr0 nat -check_engine_stats northd recompute nocompute +check_engine_stats northd recompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_pb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11071,12 +11209,16 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lr-policy-add lr0 10 "ip4.src == 10.0.0.3" reroute 172.168.0.101,172.168.0.102 check_engine_stats northd recompute nocompute +check_engine_stats sync_to_sb_pb recompute nocompute check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lr-policy-del lr0 10 "ip4.src == 10.0.0.3" check_engine_stats northd recompute nocompute +check_engine_stats sync_to_sb_pb recompute nocompute check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE OVN_CLEANUP([hv1]) AT_CLEANUP From patchwork Tue Nov 28 02:35:14 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Numan Siddique X-Patchwork-Id: 1869075 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::133; helo=smtp2.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp2.osuosl.org (smtp2.osuosl.org [IPv6:2605:bc80:3010::133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4SfRQf5mFkz1yRy for ; Tue, 28 Nov 2023 13:35:54 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id AEED140A14; Tue, 28 Nov 2023 02:35:52 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org AEED140A14 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id fAFs-pmSq-UI; Tue, 28 Nov 2023 02:35:49 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp2.osuosl.org (Postfix) with ESMTPS id D0893409F1; Tue, 28 Nov 2023 02:35:48 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org D0893409F1 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id A26ADC0072; Tue, 28 Nov 2023 02:35:48 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp2.osuosl.org (smtp2.osuosl.org [IPv6:2605:bc80:3010::133]) by lists.linuxfoundation.org (Postfix) with ESMTP id 6FFEFC0039 for ; Tue, 28 Nov 2023 02:35:47 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id F1E16409C4 for ; Tue, 28 Nov 2023 02:35:33 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org F1E16409C4 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 4fY0x4uofmt3 for ; Tue, 28 Nov 2023 02:35:30 +0000 (UTC) Received: from relay2-d.mail.gandi.net (relay2-d.mail.gandi.net [217.70.183.194]) by smtp2.osuosl.org (Postfix) with ESMTPS id E49904090E for ; Tue, 28 Nov 2023 02:35:29 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org E49904090E Received: by mail.gandi.net (Postfix) with ESMTPSA id 2950140002; Tue, 28 Nov 2023 02:35:25 +0000 (UTC) From: numans@ovn.org To: dev@openvswitch.org Date: Mon, 27 Nov 2023 21:35:14 -0500 Message-ID: <20231128023514.569672-1-numans@ovn.org> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20231128023410.569539-1-numans@ovn.org> References: <20231128023410.569539-1-numans@ovn.org> MIME-Version: 1.0 X-GND-Sasl: numans@ovn.org Subject: [ovs-dev] [PATCH ovn v3 04/16] northd: Add a new engine 'lr_nat' to manage lr NAT data. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Numan Siddique This new engine now maintains the NAT related data for each logical router which was earlier maintained by the northd engine node in the 'struct ovn_datapath'. The input to this engine node is 'northd'. A record for each logical router (lr_nat_record) is maintained in the 'lr_nats' hmap table which stores the lr's NAT dat. 'northd' engine now reports logical routers changed due to NATs in its tracking data. 'lr_nat' engine node makes use of this tracked data in its northd change handler to update the NAT data. This engine node becomes an input to 'lflow' node. Signed-off-by: Numan Siddique --- lib/ovn-util.c | 6 +- lib/ovn-util.h | 2 +- lib/stopwatch-names.h | 1 + northd/automake.mk | 2 + northd/en-lflow.c | 5 + northd/en-lr-nat.c | 423 ++++++++++++++++++++++++++++ northd/en-lr-nat.h | 127 +++++++++ northd/en-northd.c | 4 + northd/en-sync-sb.c | 6 +- northd/inc-proc-northd.c | 6 + northd/northd.c | 587 ++++++++++++++++----------------------- northd/northd.h | 46 +-- northd/ovn-northd.c | 1 + tests/ovn-northd.at | 46 ++- 14 files changed, 884 insertions(+), 378 deletions(-) create mode 100644 northd/en-lr-nat.c create mode 100644 northd/en-lr-nat.h diff --git a/lib/ovn-util.c b/lib/ovn-util.c index 33105202f2..05e635a6b4 100644 --- a/lib/ovn-util.c +++ b/lib/ovn-util.c @@ -395,7 +395,7 @@ extract_sbrec_binding_first_mac(const struct sbrec_port_binding *binding, } bool -lport_addresses_is_empty(struct lport_addresses *laddrs) +lport_addresses_is_empty(const struct lport_addresses *laddrs) { return !laddrs->n_ipv4_addrs && !laddrs->n_ipv6_addrs; } @@ -405,6 +405,10 @@ destroy_lport_addresses(struct lport_addresses *laddrs) { free(laddrs->ipv4_addrs); free(laddrs->ipv6_addrs); + laddrs->ipv4_addrs = NULL; + laddrs->ipv6_addrs = NULL; + laddrs->n_ipv4_addrs = 0; + laddrs->n_ipv6_addrs = 0; } /* Returns a string of the IP address of 'laddrs' that overlaps with 'ip_s'. diff --git a/lib/ovn-util.h b/lib/ovn-util.h index bff50dbde9..5805415885 100644 --- a/lib/ovn-util.h +++ b/lib/ovn-util.h @@ -112,7 +112,7 @@ bool extract_sbrec_binding_first_mac(const struct sbrec_port_binding *binding, bool extract_lrp_networks__(char *mac, char **networks, size_t n_networks, struct lport_addresses *laddrs); -bool lport_addresses_is_empty(struct lport_addresses *); +bool lport_addresses_is_empty(const struct lport_addresses *); void destroy_lport_addresses(struct lport_addresses *); const char *find_lport_address(const struct lport_addresses *laddrs, const char *ip_s); diff --git a/lib/stopwatch-names.h b/lib/stopwatch-names.h index 3452cc71cf..0a16da211e 100644 --- a/lib/stopwatch-names.h +++ b/lib/stopwatch-names.h @@ -32,5 +32,6 @@ #define LFLOWS_TO_SB_STOPWATCH_NAME "lflows_to_sb" #define PORT_GROUP_RUN_STOPWATCH_NAME "port_group_run" #define SYNC_METERS_RUN_STOPWATCH_NAME "sync_meters_run" +#define LR_NAT_RUN_STOPWATCH_NAME "lr_nat_run" #endif diff --git a/northd/automake.mk b/northd/automake.mk index cf622fc3c9..ae367a2a8b 100644 --- a/northd/automake.mk +++ b/northd/automake.mk @@ -24,6 +24,8 @@ northd_ovn_northd_SOURCES = \ northd/en-sync-from-sb.h \ northd/en-lb-data.c \ northd/en-lb-data.h \ + northd/en-lr-nat.c \ + northd/en-lr-nat.h \ northd/inc-proc-northd.c \ northd/inc-proc-northd.h \ northd/ipam.c \ diff --git a/northd/en-lflow.c b/northd/en-lflow.c index 6ba26006e0..e4f875ef7c 100644 --- a/northd/en-lflow.c +++ b/northd/en-lflow.c @@ -19,6 +19,7 @@ #include #include "en-lflow.h" +#include "en-lr-nat.h" #include "en-northd.h" #include "en-meters.h" @@ -40,6 +41,9 @@ lflow_get_input_data(struct engine_node *node, engine_get_input_data("port_group", node); struct sync_meters_data *sync_meters_data = engine_get_input_data("sync_meters", node); + struct ed_type_lr_nat_data *lr_nat_data = + engine_get_input_data("lr_nat", node); + lflow_input->nbrec_bfd_table = EN_OVSDB_GET(engine_get_input("NB_bfd", node)); lflow_input->sbrec_bfd_table = @@ -61,6 +65,7 @@ lflow_get_input_data(struct engine_node *node, lflow_input->ls_ports = &northd_data->ls_ports; lflow_input->lr_ports = &northd_data->lr_ports; lflow_input->ls_port_groups = &pg_data->ls_port_groups; + lflow_input->lr_nats = &lr_nat_data->lr_nats; lflow_input->meter_groups = &sync_meters_data->meter_groups; lflow_input->lb_datapaths_map = &northd_data->lb_datapaths_map; lflow_input->svc_monitor_map = &northd_data->svc_monitor_map; diff --git a/northd/en-lr-nat.c b/northd/en-lr-nat.c new file mode 100644 index 0000000000..273c5be34b --- /dev/null +++ b/northd/en-lr-nat.c @@ -0,0 +1,423 @@ +/* + * Copyright (c) 2023, Red Hat, Inc. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at: + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#include + +#include +#include +#include + +/* OVS includes */ +#include "include/openvswitch/hmap.h" +#include "openvswitch/util.h" +#include "openvswitch/vlog.h" +#include "stopwatch.h" + +/* OVN includes */ +#include "en-lr-nat.h" +#include "lib/inc-proc-eng.h" +#include "lib/lb.h" +#include "lib/ovn-nb-idl.h" +#include "lib/ovn-sb-idl.h" +#include "lib/ovn-util.h" +#include "lib/stopwatch-names.h" +#include "northd.h" + +VLOG_DEFINE_THIS_MODULE(en_lr_nat); + +/* Static function declarations. */ +static void lr_nat_table_init(struct lr_nat_table *); +static void lr_nat_table_clear(struct lr_nat_table *); +static void lr_nat_table_destroy(struct lr_nat_table *); +static void lr_nat_table_build(struct lr_nat_table *, + const struct ovn_datapaths *lr_datapaths); +static struct lr_nat_record *lr_nat_table_find_(const struct lr_nat_table *, + const struct nbrec_logical_router *); +static struct lr_nat_record *lr_nat_table_find_by_index_( + const struct lr_nat_table *, size_t od_index); + +static struct lr_nat_record *lr_nat_record_create( + struct lr_nat_table *, const struct ovn_datapath *); +static void lr_nat_record_init(struct lr_nat_record *); +static void lr_nat_record_reinit(struct lr_nat_record *); +static void lr_nat_record_destroy(struct lr_nat_record *); + +static void lr_nat_entries_init(struct lr_nat_record *); +static void lr_nat_entries_destroy(struct lr_nat_record *); +static void lr_nat_external_ips_init(struct lr_nat_record *); +static void lr_nat_external_ips_destroy(struct lr_nat_record *); +static bool get_force_snat_ip(struct lr_nat_record *, const char *key_type, + struct lport_addresses *); + +const struct lr_nat_record * +lr_nat_table_find_by_index(const struct lr_nat_table *table, + size_t od_index) +{ + return lr_nat_table_find_by_index_(table, od_index); +} + +/* 'lr_nat' engine node manages the NB logical router NAT data. + */ +void * +en_lr_nat_init(struct engine_node *node OVS_UNUSED, + struct engine_arg *arg OVS_UNUSED) +{ + struct ed_type_lr_nat_data *data = xzalloc(sizeof *data); + lr_nat_table_init(&data->lr_nats); + hmapx_init(&data->trk_data.crupdated); + return data; +} + +void +en_lr_nat_cleanup(void *data_) +{ + struct ed_type_lr_nat_data *data = (struct ed_type_lr_nat_data *) data_; + lr_nat_table_destroy(&data->lr_nats); + hmapx_destroy(&data->trk_data.crupdated); +} + +void +en_lr_nat_clear_tracked_data(void *data_) +{ + struct ed_type_lr_nat_data *data = (struct ed_type_lr_nat_data *) data_; + hmapx_clear(&data->trk_data.crupdated); +} + +void +en_lr_nat_run(struct engine_node *node, void *data_) +{ + struct northd_data *northd_data = engine_get_input_data("northd", node); + struct ed_type_lr_nat_data *data = data_; + + stopwatch_start(LR_NAT_RUN_STOPWATCH_NAME, time_msec()); + lr_nat_table_clear(&data->lr_nats); + lr_nat_table_build(&data->lr_nats, &northd_data->lr_datapaths); + + stopwatch_stop(LR_NAT_RUN_STOPWATCH_NAME, time_msec()); + engine_set_node_state(node, EN_UPDATED); +} + +/* Handler functions. */ +bool +lr_nat_northd_handler(struct engine_node *node, void *data_) +{ + struct northd_data *northd_data = engine_get_input_data("northd", node); + if (!northd_has_tracked_data(&northd_data->trk_data)) { + return false; + } + + if (!northd_has_lr_nats_in_tracked_data(&northd_data->trk_data)) { + return true; + } + + struct ed_type_lr_nat_data *data = data_; + struct lr_nat_record *lrnat_rec; + const struct ovn_datapath *od; + struct hmapx_node *hmapx_node; + + HMAPX_FOR_EACH (hmapx_node, &northd_data->trk_data.lr_with_changed_nats) { + od = hmapx_node->data; + lrnat_rec = lr_nat_table_find_(&data->lr_nats, od->nbr); + ovs_assert(lrnat_rec); + lr_nat_record_reinit(lrnat_rec); + + /* Add the lrnet rec to the tracking data. */ + hmapx_add(&data->trk_data.crupdated, lrnat_rec); + } + + if (lr_nat_has_tracked_data(&data->trk_data)) { + engine_set_node_state(node, EN_UPDATED); + } + + return true; +} + +/* static functions. */ +static void +lr_nat_table_init(struct lr_nat_table *table) +{ + *table = (struct lr_nat_table) { + .entries = HMAP_INITIALIZER(&table->entries), + }; +} + +static void +lr_nat_table_clear(struct lr_nat_table *table) +{ + struct lr_nat_record *lrnat_rec; + HMAP_FOR_EACH_POP (lrnat_rec, key_node, &table->entries) { + lr_nat_record_destroy(lrnat_rec); + } + + free(table->array); + table->array = NULL; +} + +static void +lr_nat_table_build(struct lr_nat_table *table, + const struct ovn_datapaths *lr_datapaths) +{ + table->array = xrealloc(table->array, + ods_size(lr_datapaths) * sizeof *table->array); + + const struct ovn_datapath *od; + HMAP_FOR_EACH (od, key_node, &lr_datapaths->datapaths) { + lr_nat_record_create(table, od); + } +} + +static void +lr_nat_table_destroy(struct lr_nat_table *table) +{ + lr_nat_table_clear(table); + hmap_destroy(&table->entries); +} + +struct lr_nat_record * +lr_nat_table_find_(const struct lr_nat_table *table, + const struct nbrec_logical_router *nbr) +{ + struct lr_nat_record *lrnat_rec; + + HMAP_FOR_EACH_WITH_HASH (lrnat_rec, key_node, + uuid_hash(&nbr->header_.uuid), &table->entries) { + if (nbr == lrnat_rec->od->nbr) { + return lrnat_rec; + } + } + return NULL; +} + + +struct lr_nat_record * +lr_nat_table_find_by_index_(const struct lr_nat_table *table, + size_t od_index) +{ + ovs_assert(od_index <= hmap_count(&table->entries)); + + return table->array[od_index]; +} + +static struct lr_nat_record * +lr_nat_record_create(struct lr_nat_table *table, + const struct ovn_datapath *od) +{ + ovs_assert(od->nbr); + + struct lr_nat_record *lrnat_rec = xzalloc(sizeof *lrnat_rec); + lrnat_rec->od = od; + lr_nat_record_init(lrnat_rec); + + hmap_insert(&table->entries, &lrnat_rec->key_node, + uuid_hash(&od->nbr->header_.uuid)); + table->array[od->index] = lrnat_rec; + return lrnat_rec; +} + +static void +lr_nat_record_init(struct lr_nat_record *lrnat_rec) +{ + lr_nat_entries_init(lrnat_rec); + lr_nat_external_ips_init(lrnat_rec); +} + +static void +lr_nat_record_reinit(struct lr_nat_record *lrnat_rec) +{ + lr_nat_entries_destroy(lrnat_rec); + lr_nat_external_ips_destroy(lrnat_rec); + lr_nat_record_init(lrnat_rec); +} + +static void +lr_nat_record_destroy(struct lr_nat_record *lrnat_rec) +{ + lr_nat_entries_destroy(lrnat_rec); + lr_nat_external_ips_destroy(lrnat_rec); + free(lrnat_rec); +} + +static void +lr_nat_external_ips_init(struct lr_nat_record *lrnat_rec) +{ + sset_init(&lrnat_rec->external_ips); + for (size_t i = 0; i < lrnat_rec->od->nbr->n_nat; i++) { + sset_add(&lrnat_rec->external_ips, + lrnat_rec->od->nbr->nat[i]->external_ip); + } +} + +static void +lr_nat_external_ips_destroy(struct lr_nat_record *lrnat_rec) +{ + sset_destroy(&lrnat_rec->external_ips); +} + +static void +snat_ip_add(struct lr_nat_record *lrnat_rec, const char *ip, + struct ovn_nat *nat_entry) +{ + struct ovn_snat_ip *snat_ip = shash_find_data(&lrnat_rec->snat_ips, ip); + + if (!snat_ip) { + snat_ip = xzalloc(sizeof *snat_ip); + ovs_list_init(&snat_ip->snat_entries); + shash_add(&lrnat_rec->snat_ips, ip, snat_ip); + } + + if (nat_entry) { + ovs_list_push_back(&snat_ip->snat_entries, + &nat_entry->ext_addr_list_node); + } +} + +static void +lr_nat_entries_init(struct lr_nat_record *lrnat_rec) +{ + shash_init(&lrnat_rec->snat_ips); + sset_init(&lrnat_rec->external_macs); + lrnat_rec->has_distributed_nat = false; + + if (get_force_snat_ip(lrnat_rec, "dnat", + &lrnat_rec->dnat_force_snat_addrs)) { + if (lrnat_rec->dnat_force_snat_addrs.n_ipv4_addrs) { + snat_ip_add(lrnat_rec, + lrnat_rec->dnat_force_snat_addrs.ipv4_addrs[0].addr_s, + NULL); + } + if (lrnat_rec->dnat_force_snat_addrs.n_ipv6_addrs) { + snat_ip_add(lrnat_rec, + lrnat_rec->dnat_force_snat_addrs.ipv6_addrs[0].addr_s, + NULL); + } + } + + /* Check if 'lb_force_snat_ip' is configured with 'router_ip'. */ + const char *lb_force_snat = + smap_get(&lrnat_rec->od->nbr->options, "lb_force_snat_ip"); + if (lb_force_snat && !strcmp(lb_force_snat, "router_ip") + && smap_get(&lrnat_rec->od->nbr->options, "chassis")) { + + /* Set it to true only if its gateway router and + * options:lb_force_snat_ip=router_ip. */ + lrnat_rec->lb_force_snat_router_ip = true; + } else { + lrnat_rec->lb_force_snat_router_ip = false; + + /* Check if 'lb_force_snat_ip' is configured with a set of + * IP address(es). */ + if (get_force_snat_ip(lrnat_rec, "lb", + &lrnat_rec->lb_force_snat_addrs)) { + if (lrnat_rec->lb_force_snat_addrs.n_ipv4_addrs) { + snat_ip_add(lrnat_rec, + lrnat_rec->lb_force_snat_addrs.ipv4_addrs[0].addr_s, + NULL); + } + if (lrnat_rec->lb_force_snat_addrs.n_ipv6_addrs) { + snat_ip_add(lrnat_rec, + lrnat_rec->lb_force_snat_addrs.ipv6_addrs[0].addr_s, + NULL); + } + } + } + + if (!lrnat_rec->od->nbr->n_nat) { + return; + } + + lrnat_rec->nat_entries = + xmalloc(lrnat_rec->od->nbr->n_nat * sizeof *lrnat_rec->nat_entries); + + for (size_t i = 0; i < lrnat_rec->od->nbr->n_nat; i++) { + const struct nbrec_nat *nat = lrnat_rec->od->nbr->nat[i]; + struct ovn_nat *nat_entry = &lrnat_rec->nat_entries[i]; + + nat_entry->nb = nat; + if (!extract_ip_addresses(nat->external_ip, + &nat_entry->ext_addrs) || + !nat_entry_is_valid(nat_entry)) { + static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1); + + VLOG_WARN_RL(&rl, + "Bad ip address %s in nat configuration " + "for router %s", nat->external_ip, + lrnat_rec->od->nbr->name); + continue; + } + + /* If this is a SNAT rule add the IP to the set of unique SNAT IPs. */ + if (!strcmp(nat->type, "snat")) { + if (!nat_entry_is_v6(nat_entry)) { + snat_ip_add(lrnat_rec, + nat_entry->ext_addrs.ipv4_addrs[0].addr_s, + nat_entry); + } else { + snat_ip_add(lrnat_rec, + nat_entry->ext_addrs.ipv6_addrs[0].addr_s, + nat_entry); + } + } else { + if (!strcmp(nat->type, "dnat_and_snat") + && nat->logical_port && nat->external_mac) { + lrnat_rec->has_distributed_nat = true; + } + + if (nat->external_mac) { + sset_add(&lrnat_rec->external_macs, nat->external_mac); + } + } + } + lrnat_rec->n_nat_entries = lrnat_rec->od->nbr->n_nat; +} + +static bool +get_force_snat_ip(struct lr_nat_record *lrnat_rec, const char *key_type, + struct lport_addresses *laddrs) +{ + char *key = xasprintf("%s_force_snat_ip", key_type); + const char *addresses = smap_get(&lrnat_rec->od->nbr->options, key); + free(key); + + if (!addresses) { + return false; + } + + if (!extract_ip_address(addresses, laddrs)) { + static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1); + VLOG_WARN_RL(&rl, "bad ip %s in options of router "UUID_FMT"", + addresses, UUID_ARGS(&lrnat_rec->od->nbr->header_.uuid)); + return false; + } + + return true; +} + +static void +lr_nat_entries_destroy(struct lr_nat_record *lrnat_rec) +{ + shash_destroy_free_data(&lrnat_rec->snat_ips); + destroy_lport_addresses(&lrnat_rec->dnat_force_snat_addrs); + destroy_lport_addresses(&lrnat_rec->lb_force_snat_addrs); + + for (size_t i = 0; i < lrnat_rec->n_nat_entries; i++) { + destroy_lport_addresses(&lrnat_rec->nat_entries[i].ext_addrs); + } + + free(lrnat_rec->nat_entries); + lrnat_rec->nat_entries = NULL; + lrnat_rec->n_nat_entries = 0; + sset_destroy(&lrnat_rec->external_macs); +} diff --git a/northd/en-lr-nat.h b/northd/en-lr-nat.h new file mode 100644 index 0000000000..3ec4c7b506 --- /dev/null +++ b/northd/en-lr-nat.h @@ -0,0 +1,127 @@ +/* + * Copyright (c) 2023, Red Hat, Inc. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at: + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +#ifndef EN_LR_NAT_H +#define EN_LR_NAT_H 1 + +#include + +/* OVS includes. */ +#include "lib/hmapx.h" +#include "openvswitch/hmap.h" +#include "sset.h" + +/* OVN includes. */ +#include "lib/inc-proc-eng.h" +#include "lib/ovn-nb-idl.h" +#include "lib/ovn-sb-idl.h" +#include "lib/ovn-util.h" + +/* Contains a NAT entry with the external addresses pre-parsed. */ +struct ovn_nat { + const struct nbrec_nat *nb; + struct lport_addresses ext_addrs; + struct ovs_list ext_addr_list_node; /* Linkage in the per-external IP + * list of nat entries. Currently + * only used for SNAT. + */ +}; + +/* Stores the list of SNAT entries referencing a unique SNAT IP address. + * The 'snat_entries' list will be empty if the SNAT IP is used only for + * dnat_force_snat_ip or lb_force_snat_ip. + */ +struct ovn_snat_ip { + struct ovs_list snat_entries; +}; + +struct lr_nat_record { + struct hmap_node key_node; /* Index on 'nbr->header_.uuid'. */ + + const struct ovn_datapath *od; + + struct ovn_nat *nat_entries; + size_t n_nat_entries; + + bool has_distributed_nat; + + /* Set of nat external ips on the router. */ + struct sset external_ips; + + /* Set of nat external macs on the router. */ + struct sset external_macs; + + /* SNAT IPs owned by the router (shash of 'struct ovn_snat_ip'). */ + struct shash snat_ips; + + struct lport_addresses dnat_force_snat_addrs; + struct lport_addresses lb_force_snat_addrs; + bool lb_force_snat_router_ip; +}; + +struct lr_nat_tracked_data { + /* Created or updated logical router with NAT data. */ + struct hmapx crupdated; +}; + +struct lr_nat_table { + struct hmap entries; /* Stores struct lr_nat_record. */ + + /* The array index of each element in 'entries'. */ + struct lr_nat_record **array; +}; + +const struct lr_nat_record * lr_nat_table_find_by_index( + const struct lr_nat_table *, size_t od_index); + +struct ed_type_lr_nat_data { + struct lr_nat_table lr_nats; + + struct lr_nat_tracked_data trk_data; +}; + +void *en_lr_nat_init(struct engine_node *, struct engine_arg *); +void en_lr_nat_cleanup(void *data); +void en_lr_nat_clear_tracked_data(void *data); +void en_lr_nat_run(struct engine_node *, void *data); + +bool lr_nat_logical_router_handler(struct engine_node *, void *data); +bool lr_nat_northd_handler(struct engine_node *, void *data); + +/* Returns true if a 'nat_entry' is valid, i.e.: + * - parsing was successful. + * - the string yielded exactly one IPv4 address or exactly one IPv6 address. + */ +static inline bool +nat_entry_is_valid(const struct ovn_nat *nat_entry) +{ + const struct lport_addresses *ext_addrs = &nat_entry->ext_addrs; + + return (ext_addrs->n_ipv4_addrs == 1 && ext_addrs->n_ipv6_addrs == 0) || + (ext_addrs->n_ipv4_addrs == 0 && ext_addrs->n_ipv6_addrs == 1); +} + +static inline bool +nat_entry_is_v6(const struct ovn_nat *nat_entry) +{ + return nat_entry->ext_addrs.n_ipv6_addrs > 0; +} + +static inline bool +lr_nat_has_tracked_data(struct lr_nat_tracked_data *trk_data) { + return !hmapx_is_empty(&trk_data->crupdated); +} + +#endif /* EN_LR_NAT_H */ \ No newline at end of file diff --git a/northd/en-northd.c b/northd/en-northd.c index 677b2b1ab0..546397f3dc 100644 --- a/northd/en-northd.c +++ b/northd/en-northd.c @@ -209,6 +209,10 @@ northd_nb_logical_router_handler(struct engine_node *node, return false; } + if (northd_has_lr_nats_in_tracked_data(&nd->trk_data)) { + engine_set_node_state(node, EN_UPDATED); + } + return true; } diff --git a/northd/en-sync-sb.c b/northd/en-sync-sb.c index 45be7ddbcb..11e12428f7 100644 --- a/northd/en-sync-sb.c +++ b/northd/en-sync-sb.c @@ -21,6 +21,7 @@ #include "lib/svec.h" #include "openvswitch/util.h" +#include "en-lr-nat.h" #include "en-sync-sb.h" #include "lib/inc-proc-eng.h" #include "lib/lb.h" @@ -287,9 +288,10 @@ en_sync_to_sb_pb_run(struct engine_node *node, void *data OVS_UNUSED) { const struct engine_context *eng_ctx = engine_get_context(); struct northd_data *northd_data = engine_get_input_data("northd", node); - + struct ed_type_lr_nat_data *lr_nat_data = + engine_get_input_data("lr_nat", node); sync_pbs(eng_ctx->ovnsb_idl_txn, &northd_data->ls_ports, - &northd_data->lr_ports); + &northd_data->lr_ports, &lr_nat_data->lr_nats); engine_set_node_state(node, EN_UPDATED); } diff --git a/northd/inc-proc-northd.c b/northd/inc-proc-northd.c index 04df0b06c2..1f211b278e 100644 --- a/northd/inc-proc-northd.c +++ b/northd/inc-proc-northd.c @@ -31,6 +31,7 @@ #include "openvswitch/vlog.h" #include "inc-proc-northd.h" #include "en-lb-data.h" +#include "en-lr-nat.h" #include "en-northd.h" #include "en-lflow.h" #include "en-northd-output.h" @@ -146,6 +147,7 @@ static ENGINE_NODE(fdb_aging_waker, "fdb_aging_waker"); static ENGINE_NODE(sync_to_sb_lb, "sync_to_sb_lb"); static ENGINE_NODE(sync_to_sb_pb, "sync_to_sb_pb"); static ENGINE_NODE_WITH_CLEAR_TRACK_DATA(lb_data, "lb_data"); +static ENGINE_NODE_WITH_CLEAR_TRACK_DATA(lr_nat, "lr_nat"); void inc_proc_northd_init(struct ovsdb_idl_loop *nb, struct ovsdb_idl_loop *sb) @@ -189,6 +191,8 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, northd_nb_logical_router_handler); engine_add_input(&en_northd, &en_lb_data, northd_lb_data_handler); + engine_add_input(&en_lr_nat, &en_northd, lr_nat_northd_handler); + engine_add_input(&en_mac_binding_aging, &en_nb_nb_global, NULL); engine_add_input(&en_mac_binding_aging, &en_sb_mac_binding, NULL); engine_add_input(&en_mac_binding_aging, &en_northd, NULL); @@ -212,6 +216,7 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, engine_add_input(&en_lflow, &en_sb_igmp_group, NULL); engine_add_input(&en_lflow, &en_northd, lflow_northd_handler); engine_add_input(&en_lflow, &en_port_group, lflow_port_group_handler); + engine_add_input(&en_lflow, &en_lr_nat, NULL); engine_add_input(&en_sync_to_sb_addr_set, &en_nb_address_set, sync_to_sb_addr_set_nb_address_set_handler); @@ -235,6 +240,7 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, engine_add_input(&en_sync_to_sb_pb, &en_northd, sync_to_sb_pb_northd_handler); + engine_add_input(&en_sync_to_sb_pb, &en_lr_nat, NULL); /* en_sync_to_sb engine node syncs the SB database tables from * the NB database tables. diff --git a/northd/northd.c b/northd/northd.c index aa97dfc7ed..7f59415bbd 100644 --- a/northd/northd.c +++ b/northd/northd.c @@ -43,6 +43,7 @@ #include "memory.h" #include "northd.h" #include "en-lb-data.h" +#include "en-lr-nat.h" #include "lib/ovn-parallel-hmap.h" #include "ovn/actions.h" #include "ovn/features.h" @@ -555,184 +556,6 @@ ovn_mcast_group_allocate_key(struct mcast_info *mcast_info) &mcast_info->group_tnlid_hint); } -/* Contains a NAT entry with the external addresses pre-parsed. */ -struct ovn_nat { - const struct nbrec_nat *nb; - struct lport_addresses ext_addrs; - struct ovs_list ext_addr_list_node; /* Linkage in the per-external IP - * list of nat entries. Currently - * only used for SNAT. - */ -}; - -/* Stores the list of SNAT entries referencing a unique SNAT IP address. - * The 'snat_entries' list will be empty if the SNAT IP is used only for - * dnat_force_snat_ip or lb_force_snat_ip. - */ -struct ovn_snat_ip { - struct ovs_list snat_entries; -}; - -static bool -get_force_snat_ip(struct ovn_datapath *od, const char *key_type, - struct lport_addresses *laddrs); - -/* Returns true if a 'nat_entry' is valid, i.e.: - * - parsing was successful. - * - the string yielded exactly one IPv4 address or exactly one IPv6 address. - */ -static bool -nat_entry_is_valid(const struct ovn_nat *nat_entry) -{ - const struct lport_addresses *ext_addrs = &nat_entry->ext_addrs; - - return (ext_addrs->n_ipv4_addrs == 1 && ext_addrs->n_ipv6_addrs == 0) || - (ext_addrs->n_ipv4_addrs == 0 && ext_addrs->n_ipv6_addrs == 1); -} - -static bool -nat_entry_is_v6(const struct ovn_nat *nat_entry) -{ - return nat_entry->ext_addrs.n_ipv6_addrs > 0; -} - -static void -snat_ip_add(struct ovn_datapath *od, const char *ip, struct ovn_nat *nat_entry) -{ - struct ovn_snat_ip *snat_ip = shash_find_data(&od->snat_ips, ip); - - if (!snat_ip) { - snat_ip = xzalloc(sizeof *snat_ip); - ovs_list_init(&snat_ip->snat_entries); - shash_add(&od->snat_ips, ip, snat_ip); - } - - if (nat_entry) { - ovs_list_push_back(&snat_ip->snat_entries, - &nat_entry->ext_addr_list_node); - } -} - -static void -init_nat_entries(struct ovn_datapath *od) -{ - ovs_assert(od->nbr); - - shash_init(&od->snat_ips); - if (get_force_snat_ip(od, "dnat", &od->dnat_force_snat_addrs)) { - if (od->dnat_force_snat_addrs.n_ipv4_addrs) { - snat_ip_add(od, od->dnat_force_snat_addrs.ipv4_addrs[0].addr_s, - NULL); - } - if (od->dnat_force_snat_addrs.n_ipv6_addrs) { - snat_ip_add(od, od->dnat_force_snat_addrs.ipv6_addrs[0].addr_s, - NULL); - } - } - - /* Check if 'lb_force_snat_ip' is configured with 'router_ip'. */ - const char *lb_force_snat = - smap_get(&od->nbr->options, "lb_force_snat_ip"); - if (lb_force_snat && !strcmp(lb_force_snat, "router_ip") - && smap_get(&od->nbr->options, "chassis")) { - /* Set it to true only if its gateway router and - * options:lb_force_snat_ip=router_ip. */ - od->lb_force_snat_router_ip = true; - } else { - od->lb_force_snat_router_ip = false; - - /* Check if 'lb_force_snat_ip' is configured with a set of - * IP address(es). */ - if (get_force_snat_ip(od, "lb", &od->lb_force_snat_addrs)) { - if (od->lb_force_snat_addrs.n_ipv4_addrs) { - snat_ip_add(od, od->lb_force_snat_addrs.ipv4_addrs[0].addr_s, - NULL); - } - if (od->lb_force_snat_addrs.n_ipv6_addrs) { - snat_ip_add(od, od->lb_force_snat_addrs.ipv6_addrs[0].addr_s, - NULL); - } - } - } - - if (!od->nbr->n_nat) { - return; - } - - od->nat_entries = xmalloc(od->nbr->n_nat * sizeof *od->nat_entries); - - for (size_t i = 0; i < od->nbr->n_nat; i++) { - const struct nbrec_nat *nat = od->nbr->nat[i]; - struct ovn_nat *nat_entry = &od->nat_entries[i]; - - nat_entry->nb = nat; - if (!extract_ip_addresses(nat->external_ip, - &nat_entry->ext_addrs) || - !nat_entry_is_valid(nat_entry)) { - static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1); - - VLOG_WARN_RL(&rl, - "Bad ip address %s in nat configuration " - "for router %s", nat->external_ip, od->nbr->name); - continue; - } - - /* If this is a SNAT rule add the IP to the set of unique SNAT IPs. */ - if (!strcmp(nat->type, "snat")) { - if (!nat_entry_is_v6(nat_entry)) { - snat_ip_add(od, nat_entry->ext_addrs.ipv4_addrs[0].addr_s, - nat_entry); - } else { - snat_ip_add(od, nat_entry->ext_addrs.ipv6_addrs[0].addr_s, - nat_entry); - } - } - - if (!strcmp(nat->type, "dnat_and_snat") - && nat->logical_port && nat->external_mac) { - od->has_distributed_nat = true; - } - } - od->n_nat_entries = od->nbr->n_nat; -} - -static void -destroy_nat_entries(struct ovn_datapath *od) -{ - if (!od->nbr) { - return; - } - - shash_destroy_free_data(&od->snat_ips); - destroy_lport_addresses(&od->dnat_force_snat_addrs); - destroy_lport_addresses(&od->lb_force_snat_addrs); - - for (size_t i = 0; i < od->n_nat_entries; i++) { - destroy_lport_addresses(&od->nat_entries[i].ext_addrs); - } -} - -static void -init_router_external_ips(struct ovn_datapath *od) -{ - ovs_assert(od->nbr); - - sset_init(&od->external_ips); - for (size_t i = 0; i < od->nbr->n_nat; i++) { - sset_add(&od->external_ips, od->nbr->nat[i]->external_ip); - } -} - -static void -destroy_router_external_ips(struct ovn_datapath *od) -{ - if (!od->nbr) { - return; - } - - sset_destroy(&od->external_ips); -} - static bool lb_has_vip(const struct nbrec_load_balancer *lb) { @@ -853,10 +676,7 @@ ovn_datapath_destroy(struct hmap *datapaths, struct ovn_datapath *od) destroy_ipam_info(&od->ipam_info); free(od->router_ports); free(od->ls_peers); - destroy_nat_entries(od); - destroy_router_external_ips(od); destroy_lb_for_datapath(od); - free(od->nat_entries); free(od->localnet_ports); free(od->l3dgw_ports); destroy_mcast_info_for_datapath(od); @@ -873,8 +693,8 @@ ovn_datapath_get_type(const struct ovn_datapath *od) } static struct ovn_datapath * -ovn_datapath_find(const struct hmap *datapaths, - const struct uuid *uuid) +ovn_datapath_find_(const struct hmap *datapaths, + const struct uuid *uuid) { struct ovn_datapath *od; @@ -886,6 +706,13 @@ ovn_datapath_find(const struct hmap *datapaths, return NULL; } +const struct ovn_datapath * +ovn_datapath_find(const struct hmap *datapaths, + const struct uuid *uuid) +{ + return ovn_datapath_find_(datapaths, uuid); +} + static struct ovn_datapath * ovn_datapath_find_by_key(struct hmap *datapaths, uint32_t dp_key) { @@ -924,7 +751,7 @@ ovn_datapath_from_sbrec(const struct hmap *ls_datapaths, if (!dps) { return NULL; } - struct ovn_datapath *od = ovn_datapath_find(dps, &key); + struct ovn_datapath *od = ovn_datapath_find_(dps, &key); if (od && (od->sb == sb)) { return od; } @@ -1206,7 +1033,7 @@ join_datapaths(const struct nbrec_logical_switch_table *nbrec_ls_table, continue; } - if (ovn_datapath_find(datapaths, &key)) { + if (ovn_datapath_find_(datapaths, &key)) { static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1); VLOG_INFO_RL( &rl, "deleting Datapath_Binding "UUID_FMT" with " @@ -1223,8 +1050,8 @@ join_datapaths(const struct nbrec_logical_switch_table *nbrec_ls_table, const struct nbrec_logical_switch *nbs; NBREC_LOGICAL_SWITCH_TABLE_FOR_EACH (nbs, nbrec_ls_table) { - struct ovn_datapath *od = ovn_datapath_find(datapaths, - &nbs->header_.uuid); + struct ovn_datapath *od = ovn_datapath_find_(datapaths, + &nbs->header_.uuid); if (od) { od->nbs = nbs; ovs_list_remove(&od->list); @@ -1247,8 +1074,8 @@ join_datapaths(const struct nbrec_logical_switch_table *nbrec_ls_table, continue; } - struct ovn_datapath *od = ovn_datapath_find(datapaths, - &nbr->header_.uuid); + struct ovn_datapath *od = ovn_datapath_find_(datapaths, + &nbr->header_.uuid); if (od) { if (!od->nbs) { od->nbr = nbr; @@ -1269,8 +1096,6 @@ join_datapaths(const struct nbrec_logical_switch_table *nbrec_ls_table, ovs_list_push_back(nb_only, &od->list); } init_mcast_info_for_datapath(od); - init_nat_entries(od); - init_router_external_ips(od); init_lb_for_datapath(od); if (smap_get(&od->nbr->options, "chassis")) { od->is_gw_router = true; @@ -1354,12 +1179,6 @@ ovn_datapath_assign_requested_tnl_id( } } -static inline size_t -ods_size(const struct ovn_datapaths *datapaths) -{ - return hmap_count(&datapaths->datapaths); -} - static void ods_build_array_index(struct ovn_datapaths *datapaths) { @@ -4843,7 +4662,7 @@ sync_pb_for_lsp(struct ovn_port *op) * Caller should make sure that the OVN SB IDL txn is not NULL. Presently it * only sets the port binding options column for the router ports */ static void -sync_pb_for_lrp(struct ovn_port *op) +sync_pb_for_lrp(struct ovn_port *op, const struct lr_nat_table *lr_nats) { ovs_assert(op->nbrp); @@ -4852,10 +4671,14 @@ sync_pb_for_lrp(struct ovn_port *op) const char *chassis_name = smap_get(&op->od->nbr->options, "chassis"); if (is_cr_port(op)) { + const struct lr_nat_record *lrnat_rec = + lr_nat_table_find_by_index(lr_nats, op->od->index); + ovs_assert(lrnat_rec); + smap_add(&new, "distributed-port", op->nbrp->name); bool always_redirect = - !op->od->has_distributed_nat && + !lrnat_rec->has_distributed_nat && !l3dgw_port_has_associated_vtep_lports(op->l3dgw_port); const char *redirect_type = smap_get(&op->nbrp->options, @@ -4905,7 +4728,7 @@ static void ovn_update_ipv6_opt_for_op(struct ovn_port *op); * the logical switch ports. */ void sync_pbs(struct ovsdb_idl_txn *ovnsb_idl_txn, struct hmap *ls_ports, - struct hmap *lr_ports) + struct hmap *lr_ports, const struct lr_nat_table *lr_nats) { ovs_assert(ovnsb_idl_txn); @@ -4915,7 +4738,7 @@ sync_pbs(struct ovsdb_idl_txn *ovnsb_idl_txn, struct hmap *ls_ports, } HMAP_FOR_EACH (op, key_node, lr_ports) { - sync_pb_for_lrp(op); + sync_pb_for_lrp(op, lr_nats); } ovn_update_ipv6_options(lr_ports); @@ -4924,7 +4747,7 @@ sync_pbs(struct ovsdb_idl_txn *ovnsb_idl_txn, struct hmap *ls_ports, /* Sync the SB Port bindings for the added and updated logical switch ports * of the tracked northd engine data. */ bool -sync_pbs_for_northd_changed_ovn_ports( struct tracked_ovn_ports *trk_ovn_ports) +sync_pbs_for_northd_changed_ovn_ports(struct tracked_ovn_ports *trk_ovn_ports) { struct hmapx_node *hmapx_node; struct ovn_port *op; @@ -5176,6 +4999,7 @@ destroy_northd_data_tracked_changes(struct northd_data *nd) struct northd_tracked_data *trk_changes = &nd->trk_data; destroy_tracked_ovn_ports(&trk_changes->trk_lsps); destroy_tracked_lbs(&trk_changes->trk_lbs); + hmapx_clear(&trk_changes->lr_with_changed_nats); nd->trk_data.type = NORTHD_TRACKED_NONE; } @@ -5189,6 +5013,7 @@ init_northd_tracked_data(struct northd_data *nd) hmapx_init(&trk_data->trk_lsps.deleted); hmapx_init(&trk_data->trk_lbs.crupdated); hmapx_init(&trk_data->trk_lbs.deleted); + hmapx_init(&trk_data->lr_with_changed_nats); } static void @@ -5201,6 +5026,7 @@ destroy_northd_tracked_data(struct northd_data *nd) hmapx_destroy(&trk_data->trk_lsps.deleted); hmapx_destroy(&trk_data->trk_lbs.crupdated); hmapx_destroy(&trk_data->trk_lbs.deleted); + hmapx_destroy(&trk_data->lr_with_changed_nats); } /* Check if a changed LSP can be handled incrementally within the I-P engine @@ -5563,7 +5389,7 @@ northd_handle_ls_changes(struct ovsdb_idl_txn *ovnsb_idl_txn, nbrec_logical_switch_is_deleted(changed_ls)) { goto fail; } - struct ovn_datapath *od = ovn_datapath_find( + struct ovn_datapath *od = ovn_datapath_find_( &nd->ls_datapaths.datapaths, &changed_ls->header_.uuid); if (!od) { @@ -5602,6 +5428,7 @@ fail: * incrementally handled. * Presently supports i-p for the below changes: * - load balancers and load balancer groups. + * - NAT changes */ static bool lr_changes_can_be_handled( @@ -5611,8 +5438,9 @@ lr_changes_can_be_handled( enum nbrec_logical_router_column_id col; for (col = 0; col < NBREC_LOGICAL_ROUTER_N_COLUMNS; col++) { if (nbrec_logical_router_is_updated(lr, col)) { - if (col == NBREC_LOGICAL_ROUTER_COL_LOAD_BALANCER || - col == NBREC_LOGICAL_ROUTER_COL_LOAD_BALANCER_GROUP) { + if (col == NBREC_LOGICAL_ROUTER_COL_LOAD_BALANCER + || col == NBREC_LOGICAL_ROUTER_COL_LOAD_BALANCER_GROUP + || col == NBREC_LOGICAL_ROUTER_COL_NAT) { continue; } return false; @@ -5631,12 +5459,6 @@ lr_changes_can_be_handled( OVSDB_IDL_CHANGE_MODIFY) > 0) { return false; } - for (size_t i = 0; i < lr->n_nat; i++) { - if (nbrec_nat_row_get_seqno(lr->nat[i], - OVSDB_IDL_CHANGE_MODIFY) > 0) { - return false; - } - } for (size_t i = 0; i < lr->n_policies; i++) { if (nbrec_logical_router_policy_row_get_seqno(lr->policies[i], OVSDB_IDL_CHANGE_MODIFY) > 0) { @@ -5652,6 +5474,39 @@ lr_changes_can_be_handled( return true; } +static bool +is_lr_nats_seqno_changed(const struct nbrec_logical_router *nbr) +{ + for (size_t i = 0; i < nbr->n_nat; i++) { + if (nbrec_nat_row_get_seqno(nbr->nat[i], + OVSDB_IDL_CHANGE_MODIFY) > 0) { + return true; + } + } + + return false; +} + +static bool +is_lr_nats_changed(const struct nbrec_logical_router *nbr) { + return (nbrec_logical_router_is_updated(nbr, + NBREC_LOGICAL_ROUTER_COL_NAT) + || nbrec_logical_router_is_updated( + nbr, NBREC_LOGICAL_ROUTER_COL_OPTIONS) + || is_lr_nats_seqno_changed(nbr)); +} + +static bool +lr_has_routable_nats(const struct nbrec_logical_router *nbr) { + for (size_t i = 0; i < nbr->n_nat; i++) { + if (smap_get_bool(&nbr->nat[i]->options, "add_route", false)) { + return true; + } + } + + return false; +} + /* Return true if changes are handled incrementally, false otherwise. * * Note: Changes to load balancer and load balancer groups associated with @@ -5671,11 +5526,37 @@ northd_handle_lr_changes(const struct northd_input *ni, goto fail; } - /* Presently only able to handle load balancer and - * load balancer group changes. */ + /* Presently only able to handle load balancer, + * load balancer group changes and NAT changes. */ if (!lr_changes_can_be_handled(changed_lr)) { goto fail; } + + if (is_lr_nats_changed(changed_lr)) { + if (lr_has_routable_nats(changed_lr)) { + /* router has routable NATs. We can't handle these changes + * incrementally yet. Fall back to recompute. */ + goto fail; + } + + struct ovn_datapath *od = ovn_datapath_find_( + &nd->lr_datapaths.datapaths, + &changed_lr->header_.uuid); + + if (!od) { + static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 1); + VLOG_WARN_RL(&rl, "Internal error: a tracked updated LR " + "doesn't exist in lr_datapaths: "UUID_FMT, + UUID_ARGS(&changed_lr->header_.uuid)); + goto fail; + } + + hmapx_add(&nd->trk_data.lr_with_changed_nats, od); + } + } + + if (!hmapx_is_empty(&nd->trk_data.lr_with_changed_nats)) { + nd->trk_data.type |= NORTHD_TRACKED_LR_NATS; } return true; @@ -5893,7 +5774,7 @@ northd_handle_lb_data_changes(struct tracked_lb_data *trk_lb_data, struct crupdated_od_lb_data *codlb; LIST_FOR_EACH (codlb, list_node, &trk_lb_data->crupdated_ls_lbs) { - od = ovn_datapath_find(&ls_datapaths->datapaths, &codlb->od_uuid); + od = ovn_datapath_find_(&ls_datapaths->datapaths, &codlb->od_uuid); ovs_assert(od); struct uuidset_node *uuidnode; @@ -5930,7 +5811,7 @@ northd_handle_lb_data_changes(struct tracked_lb_data *trk_lb_data, } LIST_FOR_EACH (codlb, list_node, &trk_lb_data->crupdated_lr_lbs) { - od = ovn_datapath_find(&lr_datapaths->datapaths, &codlb->od_uuid); + od = ovn_datapath_find_(&lr_datapaths->datapaths, &codlb->od_uuid); ovs_assert(od); struct uuidset_node *uuidnode; @@ -9273,31 +9154,15 @@ static void build_lswitch_rport_arp_req_self_orig_flow(struct ovn_port *op, uint32_t priority, struct ovn_datapath *od, + const struct lr_nat_table *lr_nats, struct hmap *lflows) { - struct sset all_eth_addrs = SSET_INITIALIZER(&all_eth_addrs); struct ds eth_src = DS_EMPTY_INITIALIZER; struct ds match = DS_EMPTY_INITIALIZER; - sset_add(&all_eth_addrs, op->lrp_networks.ea_s); - - for (size_t i = 0; i < op->od->nbr->n_nat; i++) { - struct ovn_nat *nat_entry = &op->od->nat_entries[i]; - const struct nbrec_nat *nat = nat_entry->nb; - - if (!nat_entry_is_valid(nat_entry)) { - continue; - } - - if (!strcmp(nat->type, "snat")) { - continue; - } - - if (!nat->external_mac) { - continue; - } - sset_add(&all_eth_addrs, nat->external_mac); - } + const struct lr_nat_record *lrnat_rec = lr_nat_table_find_by_index( + lr_nats, op->od->index); + ovs_assert(lrnat_rec); /* Self originated ARP requests/RARP/ND need to be flooded to the L2 domain * (except on router ports). Determine that packets are self originated @@ -9307,8 +9172,8 @@ build_lswitch_rport_arp_req_self_orig_flow(struct ovn_port *op, */ const char *eth_addr; - ds_put_cstr(ð_src, "{"); - SSET_FOR_EACH (eth_addr, &all_eth_addrs) { + ds_put_format(ð_src, "{%s, ", op->lrp_networks.ea_s); + SSET_FOR_EACH (eth_addr, &lrnat_rec->external_macs) { ds_put_format(ð_src, "%s, ", eth_addr); } ds_chomp(ð_src, ' '); @@ -9321,7 +9186,6 @@ build_lswitch_rport_arp_req_self_orig_flow(struct ovn_port *op, ovn_lflow_add(lflows, od, S_SWITCH_IN_L2_LKUP, priority, ds_cstr(&match), "outport = \""MC_FLOOD_L2"\"; output;"); - sset_destroy(&all_eth_addrs); ds_destroy(ð_src); ds_destroy(&match); } @@ -9427,6 +9291,7 @@ static void build_lswitch_rport_arp_req_flows(struct ovn_port *op, struct ovn_datapath *sw_od, struct ovn_port *sw_op, + const struct lr_nat_table *lr_nats, struct hmap *lflows, const struct ovsdb_idl_row *stage_hint) { @@ -9471,8 +9336,38 @@ build_lswitch_rport_arp_req_flows(struct ovn_port *op, } } - for (size_t i = 0; i < op->od->nbr->n_nat; i++) { - struct ovn_nat *nat_entry = &op->od->nat_entries[i]; + for (size_t i = 0; i < op->lrp_networks.n_ipv4_addrs; i++) { + build_lswitch_rport_arp_req_flow( + op->lrp_networks.ipv4_addrs[i].addr_s, AF_INET, sw_op, sw_od, 80, + lflows, stage_hint); + } + for (size_t i = 0; i < op->lrp_networks.n_ipv6_addrs; i++) { + build_lswitch_rport_arp_req_flow( + op->lrp_networks.ipv6_addrs[i].addr_s, AF_INET6, sw_op, sw_od, 80, + lflows, stage_hint); + } + + /* Self originated ARP requests/RARP/ND need to be flooded as usual. + * + * However, if the switch doesn't have any non-router ports we shouldn't + * even try to flood. + * + * Priority: 75. + */ + if (sw_od->n_router_ports != sw_od->nbs->n_ports) { + build_lswitch_rport_arp_req_self_orig_flow(op, 75, sw_od, lr_nats, + lflows); + } + + const struct lr_nat_record *lrnat_rec = + lr_nat_table_find_by_index(lr_nats, op->od->index); + + if (!lrnat_rec) { + return; + } + + for (size_t i = 0; i < lrnat_rec->n_nat_entries; i++) { + struct ovn_nat *nat_entry = &lrnat_rec->nat_entries[i]; const struct nbrec_nat *nat = nat_entry->nb; if (!nat_entry_is_valid(nat_entry)) { @@ -9500,28 +9395,6 @@ build_lswitch_rport_arp_req_flows(struct ovn_port *op, } } } - - for (size_t i = 0; i < op->lrp_networks.n_ipv4_addrs; i++) { - build_lswitch_rport_arp_req_flow( - op->lrp_networks.ipv4_addrs[i].addr_s, AF_INET, sw_op, sw_od, 80, - lflows, stage_hint); - } - for (size_t i = 0; i < op->lrp_networks.n_ipv6_addrs; i++) { - build_lswitch_rport_arp_req_flow( - op->lrp_networks.ipv6_addrs[i].addr_s, AF_INET6, sw_op, sw_od, 80, - lflows, stage_hint); - } - - /* Self originated ARP requests/RARP/ND need to be flooded as usual. - * - * However, if the switch doesn't have any non-router ports we shouldn't - * even try to flood. - * - * Priority: 75. - */ - if (sw_od->n_router_ports != sw_od->nbs->n_ports) { - build_lswitch_rport_arp_req_self_orig_flow(op, 75, sw_od, lflows); - } } static void @@ -10555,6 +10428,7 @@ build_lswitch_ip_mcast_igmp_mld(struct ovn_igmp_group *igmp_group, /* Ingress table 25: Destination lookup, unicast handling (priority 50), */ static void build_lswitch_ip_unicast_lookup(struct ovn_port *op, + const struct lr_nat_table *lr_nats, struct hmap *lflows, struct ds *actions, struct ds *match) @@ -10569,8 +10443,8 @@ build_lswitch_ip_unicast_lookup(struct ovn_port *op, * requests only to the router port that owns the IP address. */ if (lsp_is_router(op->nbsp)) { - build_lswitch_rport_arp_req_flows(op->peer, op->od, op, lflows, - &op->nbsp->header_); + build_lswitch_rport_arp_req_flows(op->peer, op->od, op, lr_nats, + lflows, &op->nbsp->header_); } for (size_t i = 0; i < op->nbsp->n_addresses; i++) { @@ -11933,27 +11807,6 @@ op_put_v6_networks(struct ds *ds, const struct ovn_port *op) ds_put_cstr(ds, "}"); } -static bool -get_force_snat_ip(struct ovn_datapath *od, const char *key_type, - struct lport_addresses *laddrs) -{ - char *key = xasprintf("%s_force_snat_ip", key_type); - const char *addresses = smap_get(&od->nbr->options, key); - free(key); - - if (!addresses) { - return false; - } - - if (!extract_ip_address(addresses, laddrs)) { - static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1); - VLOG_WARN_RL(&rl, "bad ip %s in options of router "UUID_FMT"", - addresses, UUID_ARGS(&od->key)); - return false; - } - - return true; -} enum lrouter_nat_lb_flow_type { LROUTER_NAT_LB_FLOW_NORMAL = 0, @@ -12105,6 +11958,7 @@ build_lrouter_nat_flows_for_lb(struct ovn_lb_vip *lb_vip, struct ovn_lb_datapaths *lb_dps, struct ovn_northd_lb_vip *vips_nb, const struct ovn_datapaths *lr_datapaths, + const struct lr_nat_table *lr_nats, struct hmap *lflows, struct ds *match, struct ds *action, const struct shash *meter_groups, @@ -12210,10 +12064,13 @@ build_lrouter_nat_flows_for_lb(struct ovn_lb_vip *lb_vip, struct ovn_datapath *od = lr_datapaths->array[index]; enum lrouter_nat_lb_flow_type type; + const struct lr_nat_record *lrnat_rec = + lr_nat_table_find_by_index(lr_nats, od->index); + ovs_assert(lrnat_rec); if (lb->skip_snat) { type = LROUTER_NAT_LB_FLOW_SKIP_SNAT; - } else if (!lport_addresses_is_empty(&od->lb_force_snat_addrs) || - od->lb_force_snat_router_ip) { + } else if (!lport_addresses_is_empty(&lrnat_rec->lb_force_snat_addrs) + || lrnat_rec->lb_force_snat_router_ip) { type = LROUTER_NAT_LB_FLOW_FORCE_SNAT; } else { type = LROUTER_NAT_LB_FLOW_NORMAL; @@ -12229,7 +12086,7 @@ build_lrouter_nat_flows_for_lb(struct ovn_lb_vip *lb_vip, bitmap_set1(aff_dp_bitmap[type], index); } - if (sset_contains(&od->external_ips, lb_vip->vip_str)) { + if (sset_contains(&lrnat_rec->external_ips, lb_vip->vip_str)) { /* The load balancer vip is also present in the NAT entries. * So add a high priority lflow to advance the the packet * destined to the vip (and the vip port if defined) @@ -12359,6 +12216,7 @@ build_lrouter_flows_for_lb(struct ovn_lb_datapaths *lb_dps, struct hmap *lflows, const struct shash *meter_groups, const struct ovn_datapaths *lr_datapaths, + const struct lr_nat_table *lr_nats, const struct chassis_features *features, const struct hmap *svc_monitor_map, struct ds *match, struct ds *action) @@ -12374,8 +12232,8 @@ build_lrouter_flows_for_lb(struct ovn_lb_datapaths *lb_dps, struct ovn_lb_vip *lb_vip = &lb->vips[i]; build_lrouter_nat_flows_for_lb(lb_vip, lb_dps, &lb->vips_nb[i], - lr_datapaths, lflows, match, action, - meter_groups, features, + lr_datapaths, lr_nats, lflows, match, + action, meter_groups, features, svc_monitor_map); if (!build_empty_lb_event_flow(lb_vip, lb, match, action)) { @@ -12774,7 +12632,9 @@ build_lrouter_port_nat_arp_nd_flow(struct ovn_port *op, } static void -build_lrouter_drop_own_dest(struct ovn_port *op, enum ovn_stage stage, +build_lrouter_drop_own_dest(struct ovn_port *op, + const struct lr_nat_record *lrnat_rec, + enum ovn_stage stage, uint16_t priority, bool drop_snat_ip, struct hmap *lflows) { @@ -12784,7 +12644,8 @@ build_lrouter_drop_own_dest(struct ovn_port *op, enum ovn_stage stage, for (size_t i = 0; i < op->lrp_networks.n_ipv4_addrs; i++) { const char *ip = op->lrp_networks.ipv4_addrs[i].addr_s; - bool router_ip_in_snat_ips = !!shash_find(&op->od->snat_ips, ip); + bool router_ip_in_snat_ips = !!shash_find(&lrnat_rec->snat_ips, + ip); bool router_ip_in_lb_ips = !!sset_find(&op->od->lb_ips->ips_v4, ip); bool drop_router_ip = (drop_snat_ip == (router_ip_in_snat_ips || @@ -12813,7 +12674,8 @@ build_lrouter_drop_own_dest(struct ovn_port *op, enum ovn_stage stage, for (size_t i = 0; i < op->lrp_networks.n_ipv6_addrs; i++) { const char *ip = op->lrp_networks.ipv6_addrs[i].addr_s; - bool router_ip_in_snat_ips = !!shash_find(&op->od->snat_ips, ip); + bool router_ip_in_snat_ips = !!shash_find(&lrnat_rec->snat_ips, + ip); bool router_ip_in_lb_ips = !!sset_find(&op->od->lb_ips->ips_v6, ip); bool drop_router_ip = (drop_snat_ip == (router_ip_in_snat_ips || @@ -12866,11 +12728,12 @@ build_lrouter_force_snat_flows(struct hmap *lflows, struct ovn_datapath *od, static void build_lrouter_force_snat_flows_op(struct ovn_port *op, + const struct lr_nat_record *lrnat_rec, struct hmap *lflows, struct ds *match, struct ds *actions) { ovs_assert(op->nbrp); - if (!op->peer || !op->od->lb_force_snat_router_ip) { + if (!op->peer || !lrnat_rec->lb_force_snat_router_ip) { return; } @@ -13823,8 +13686,8 @@ routable_addresses_to_lflows(struct hmap *lflows, struct ovn_port *router_port, /* This function adds ARP resolve flows related to a LRP. */ static void build_arp_resolve_flows_for_lrp( - struct ovn_port *op, struct hmap *lflows, - struct ds *match, struct ds *actions) + struct ovn_port *op, const struct lr_nat_record *lrnat_rec, + struct hmap *lflows, struct ds *match, struct ds *actions) { ovs_assert(op->nbrp); /* This is a logical router port. If next-hop IP address in @@ -13900,8 +13763,8 @@ build_arp_resolve_flows_for_lrp( * * Priority 2. */ - build_lrouter_drop_own_dest(op, S_ROUTER_IN_ARP_RESOLVE, 2, true, - lflows); + build_lrouter_drop_own_dest(op, lrnat_rec, S_ROUTER_IN_ARP_RESOLVE, 2, + true, lflows); } /* This function adds ARP resolve flows related to a LSP. */ @@ -14231,6 +14094,7 @@ build_check_pkt_len_flows_for_lrouter( static void build_gateway_redirect_flows_for_lrouter( struct ovn_datapath *od, struct hmap *lflows, + const struct lr_nat_table *lr_nats, struct ds *match, struct ds *actions) { ovs_assert(od->nbr); @@ -14266,8 +14130,16 @@ build_gateway_redirect_flows_for_lrouter( ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_GW_REDIRECT, 50, ds_cstr(match), ds_cstr(actions), stage_hint); - for (int j = 0; j < od->n_nat_entries; j++) { - const struct ovn_nat *nat = &od->nat_entries[j]; + + const struct lr_nat_record *lrnat_rec = lr_nat_table_find_by_index( + lr_nats, od->index); + + if (!lrnat_rec) { + continue; + } + + for (int j = 0; j < lrnat_rec->n_nat_entries; j++) { + const struct ovn_nat *nat = &lrnat_rec->nat_entries[j]; if (!lrouter_dnat_and_snat_is_stateless(nat->nb) || (!nat->nb->allowed_ext_ips && !nat->nb->exempted_ext_ips)) { @@ -14705,10 +14577,15 @@ build_ipv6_input_flows_for_lrouter_port( static void build_lrouter_arp_nd_for_datapath(struct ovn_datapath *od, + const struct lr_nat_table *lr_nats, struct hmap *lflows, const struct shash *meter_groups) { ovs_assert(od->nbr); + if (!od->nbr->n_nat) { + return; + } + /* Priority-90-92 flows handle ARP requests and ND packets. Most are * per logical port but DNAT addresses can be handled per datapath * for non gateway router ports. @@ -14717,8 +14594,12 @@ build_lrouter_arp_nd_for_datapath(struct ovn_datapath *od, * port to handle the special cases. In case we get the packet * on a regular port, just reply with the port's ETH address. */ - for (int i = 0; i < od->nbr->n_nat; i++) { - struct ovn_nat *nat_entry = &od->nat_entries[i]; + const struct lr_nat_record *lrnat_rec = lr_nat_table_find_by_index( + lr_nats, od->index); + ovs_assert(lrnat_rec); + + for (int i = 0; i < lrnat_rec->n_nat_entries; i++) { + struct ovn_nat *nat_entry = &lrnat_rec->nat_entries[i]; /* Skip entries we failed to parse. */ if (!nat_entry_is_valid(nat_entry)) { @@ -14726,8 +14607,8 @@ build_lrouter_arp_nd_for_datapath(struct ovn_datapath *od, } /* Skip SNAT entries for now, we handle unique SNAT IPs separately - * below. - */ + * below. + */ if (!strcmp(nat_entry->nb->type, "snat")) { continue; } @@ -14736,7 +14617,7 @@ build_lrouter_arp_nd_for_datapath(struct ovn_datapath *od, /* Now handle SNAT entries too, one per unique SNAT IP. */ struct shash_node *snat_snode; - SHASH_FOR_EACH (snat_snode, &od->snat_ips) { + SHASH_FOR_EACH (snat_snode, &lrnat_rec->snat_ips) { struct ovn_snat_ip *snat_ip = snat_snode->data; if (ovs_list_is_empty(&snat_ip->snat_entries)) { @@ -14754,6 +14635,7 @@ build_lrouter_arp_nd_for_datapath(struct ovn_datapath *od, static void build_lrouter_ipv4_ip_input(struct ovn_port *op, struct hmap *lflows, + const struct lr_nat_record *lrnat_rec, struct ds *match, struct ds *actions, const struct shash *meter_groups) { @@ -14990,14 +14872,14 @@ build_lrouter_ipv4_ip_input(struct ovn_port *op, * also a SNAT IP. Those are dropped later, in stage * "lr_in_arp_resolve", if unSNAT was unsuccessful. * - * If op->od->lb_force_snat_router_ip is true, it means the IP of the + * If lrnat_rec->lb_force_snat_router_ip is true, it means the IP of the * router port is also SNAT IP. * * Priority 60. */ - if (!op->od->lb_force_snat_router_ip) { - build_lrouter_drop_own_dest(op, S_ROUTER_IN_IP_INPUT, 60, false, - lflows); + if (!lrnat_rec->lb_force_snat_router_ip) { + build_lrouter_drop_own_dest(op, lrnat_rec, S_ROUTER_IN_IP_INPUT, 60, + false, lflows); } /* ARP / ND handling for external IP addresses. * @@ -15012,8 +14894,8 @@ build_lrouter_ipv4_ip_input(struct ovn_port *op, return; } - for (size_t i = 0; i < op->od->nbr->n_nat; i++) { - struct ovn_nat *nat_entry = &op->od->nat_entries[i]; + for (int i = 0; i < lrnat_rec->n_nat_entries; i++) { + struct ovn_nat *nat_entry = &lrnat_rec->nat_entries[i]; /* Skip entries we failed to parse. */ if (!nat_entry_is_valid(nat_entry)) { @@ -15021,18 +14903,18 @@ build_lrouter_ipv4_ip_input(struct ovn_port *op, } /* Skip SNAT entries for now, we handle unique SNAT IPs separately - * below. - */ + * below. + */ if (!strcmp(nat_entry->nb->type, "snat")) { continue; } build_lrouter_port_nat_arp_nd_flow(op, nat_entry, lflows, - meter_groups); + meter_groups); } /* Now handle SNAT entries too, one per unique SNAT IP. */ struct shash_node *snat_snode; - SHASH_FOR_EACH (snat_snode, &op->od->snat_ips) { + SHASH_FOR_EACH (snat_snode, &lrnat_rec->snat_ips) { struct ovn_snat_ip *snat_ip = snat_snode->data; if (ovs_list_is_empty(&snat_ip->snat_entries)) { @@ -15041,9 +14923,9 @@ build_lrouter_ipv4_ip_input(struct ovn_port *op, struct ovn_nat *nat_entry = CONTAINER_OF(ovs_list_front(&snat_ip->snat_entries), - struct ovn_nat, ext_addr_list_node); + struct ovn_nat, ext_addr_list_node); build_lrouter_port_nat_arp_nd_flow(op, nat_entry, lflows, - meter_groups); + meter_groups); } } @@ -15152,6 +15034,7 @@ build_lrouter_in_unsnat_flow(struct hmap *lflows, struct ovn_datapath *od, static void build_lrouter_in_dnat_flow(struct hmap *lflows, struct ovn_datapath *od, + const struct lr_nat_record *lrnat_rec, const struct nbrec_nat *nat, struct ds *match, struct ds *actions, bool distributed_nat, int cidr_bits, bool is_v6, @@ -15175,7 +15058,7 @@ build_lrouter_in_dnat_flow(struct hmap *lflows, struct ovn_datapath *od, nat->external_ip); if (od->is_gw_router) { - if (!lport_addresses_is_empty(&od->dnat_force_snat_addrs)) { + if (!lport_addresses_is_empty(&lrnat_rec->dnat_force_snat_addrs)) { /* Indicate to the future tables that a DNAT has taken * place and a force SNAT needs to be done in the * Egress SNAT table. */ @@ -15731,6 +15614,7 @@ static void build_lrouter_nat_defrag_and_lb(struct ovn_datapath *od, struct hmap *lflows, const struct hmap *ls_ports, const struct hmap *lr_ports, + const struct lr_nat_table *lr_nats, struct ds *match, struct ds *actions, const struct shash *meter_groups, @@ -15841,14 +15725,18 @@ build_lrouter_nat_defrag_and_lb(struct ovn_datapath *od, struct hmap *lflows, } struct sset nat_entries = SSET_INITIALIZER(&nat_entries); + const struct lr_nat_record *lrnat_rec = lr_nat_table_find_by_index(lr_nats, + od->index); + ovs_assert(lrnat_rec); bool dnat_force_snat_ip = - !lport_addresses_is_empty(&od->dnat_force_snat_addrs); + !lport_addresses_is_empty(&lrnat_rec->dnat_force_snat_addrs); bool lb_force_snat_ip = - !lport_addresses_is_empty(&od->lb_force_snat_addrs); + !lport_addresses_is_empty(&lrnat_rec->lb_force_snat_addrs); - for (int i = 0; i < od->nbr->n_nat; i++) { - const struct nbrec_nat *nat = od->nbr->nat[i]; + for (int i = 0; i < lrnat_rec->n_nat_entries; i++) { + struct ovn_nat *nat_entry = &lrnat_rec->nat_entries[i]; + const struct nbrec_nat *nat = nat_entry->nb; struct eth_addr mac = eth_addr_broadcast; bool is_v6, distributed_nat; ovs_be32 mask; @@ -15886,7 +15774,7 @@ build_lrouter_nat_defrag_and_lb(struct ovn_datapath *od, struct hmap *lflows, distributed_nat, is_v6, l3dgw_port); } /* S_ROUTER_IN_DNAT */ - build_lrouter_in_dnat_flow(lflows, od, nat, match, actions, + build_lrouter_in_dnat_flow(lflows, od, lrnat_rec, nat, match, actions, distributed_nat, cidr_bits, is_v6, l3dgw_port, stateless); @@ -16095,25 +15983,25 @@ build_lrouter_nat_defrag_and_lb(struct ovn_datapath *od, struct hmap *lflows, /* Handle force SNAT options set in the gateway router. */ if (od->is_gw_router) { if (dnat_force_snat_ip) { - if (od->dnat_force_snat_addrs.n_ipv4_addrs) { + if (lrnat_rec->dnat_force_snat_addrs.n_ipv4_addrs) { build_lrouter_force_snat_flows(lflows, od, "4", - od->dnat_force_snat_addrs.ipv4_addrs[0].addr_s, + lrnat_rec->dnat_force_snat_addrs.ipv4_addrs[0].addr_s, "dnat"); } - if (od->dnat_force_snat_addrs.n_ipv6_addrs) { + if (lrnat_rec->dnat_force_snat_addrs.n_ipv6_addrs) { build_lrouter_force_snat_flows(lflows, od, "6", - od->dnat_force_snat_addrs.ipv6_addrs[0].addr_s, + lrnat_rec->dnat_force_snat_addrs.ipv6_addrs[0].addr_s, "dnat"); } } if (lb_force_snat_ip) { - if (od->lb_force_snat_addrs.n_ipv4_addrs) { + if (lrnat_rec->lb_force_snat_addrs.n_ipv4_addrs) { build_lrouter_force_snat_flows(lflows, od, "4", - od->lb_force_snat_addrs.ipv4_addrs[0].addr_s, "lb"); + lrnat_rec->lb_force_snat_addrs.ipv4_addrs[0].addr_s, "lb"); } - if (od->lb_force_snat_addrs.n_ipv6_addrs) { + if (lrnat_rec->lb_force_snat_addrs.n_ipv6_addrs) { build_lrouter_force_snat_flows(lflows, od, "6", - od->lb_force_snat_addrs.ipv6_addrs[0].addr_s, "lb"); + lrnat_rec->lb_force_snat_addrs.ipv6_addrs[0].addr_s, "lb"); } } } @@ -16129,6 +16017,7 @@ struct lswitch_flow_build_info { const struct hmap *ls_ports; const struct hmap *lr_ports; const struct ls_port_group_table *ls_port_groups; + const struct lr_nat_table *lr_nats; struct hmap *lflows; struct hmap *igmp_groups; const struct shash *meter_groups; @@ -16194,14 +16083,15 @@ build_lswitch_and_lrouter_iterate_by_lr(struct ovn_datapath *od, build_check_pkt_len_flows_for_lrouter(od, lsi->lflows, lsi->lr_ports, &lsi->match, &lsi->actions, lsi->meter_groups); - build_gateway_redirect_flows_for_lrouter(od, lsi->lflows, &lsi->match, - &lsi->actions); + build_gateway_redirect_flows_for_lrouter(od, lsi->lflows, lsi->lr_nats, + &lsi->match, &lsi->actions); build_arp_request_flows_for_lrouter(od, lsi->lflows, &lsi->match, &lsi->actions, lsi->meter_groups); build_misc_local_traffic_drop_flows_for_lrouter(od, lsi->lflows); - build_lrouter_arp_nd_for_datapath(od, lsi->lflows, lsi->meter_groups); + build_lrouter_arp_nd_for_datapath(od, lsi->lr_nats, lsi->lflows, + lsi->meter_groups); build_lrouter_nat_defrag_and_lb(od, lsi->lflows, lsi->ls_ports, - lsi->lr_ports, &lsi->match, + lsi->lr_ports,lsi->lr_nats, &lsi->match, &lsi->actions, lsi->meter_groups, lsi->features); build_lrouter_lb_affinity_default_flows(od, lsi->lflows); @@ -16214,6 +16104,7 @@ static void build_lswitch_and_lrouter_iterate_by_lsp(struct ovn_port *op, const struct hmap *ls_ports, const struct hmap *lr_ports, + const struct lr_nat_table *lr_nats, const struct shash *meter_groups, struct ds *match, struct ds *actions, @@ -16230,7 +16121,7 @@ build_lswitch_and_lrouter_iterate_by_lsp(struct ovn_port *op, meter_groups, actions, match); build_lswitch_dhcp_options_and_response(op, lflows, meter_groups); build_lswitch_external_port(op, lflows); - build_lswitch_ip_unicast_lookup(op, lflows, actions, match); + build_lswitch_ip_unicast_lookup(op, lr_nats, lflows, actions, match); /* Build Logical Router Flows. */ build_ip_routing_flows_for_router_type_lsp(op, lr_ports, lflows); @@ -16248,6 +16139,11 @@ build_lswitch_and_lrouter_iterate_by_lrp(struct ovn_port *op, struct lswitch_flow_build_info *lsi) { ovs_assert(op->nbrp); + + const struct lr_nat_record *lrnet_rec = lr_nat_table_find_by_index( + lsi->lr_nats, op->od->index); + ovs_assert(lrnet_rec); + build_adm_ctrl_flows_for_lrouter_port(op, lsi->lflows, &lsi->match, &lsi->actions); build_neigh_learning_flows_for_lrouter_port(op, lsi->lflows, &lsi->match, @@ -16255,7 +16151,7 @@ build_lswitch_and_lrouter_iterate_by_lrp(struct ovn_port *op, build_ip_routing_flows_for_lrp(op, lsi->lflows); build_ND_RA_flows_for_lrouter_port(op, lsi->lflows, &lsi->match, &lsi->actions, lsi->meter_groups); - build_arp_resolve_flows_for_lrp(op, lsi->lflows, &lsi->match, + build_arp_resolve_flows_for_lrp(op, lrnet_rec, lsi->lflows, &lsi->match, &lsi->actions); build_egress_delivery_flows_for_lrouter_port(op, lsi->lflows, &lsi->match, &lsi->actions); @@ -16263,9 +16159,9 @@ build_lswitch_and_lrouter_iterate_by_lrp(struct ovn_port *op, build_ipv6_input_flows_for_lrouter_port(op, lsi->lflows, &lsi->match, &lsi->actions, lsi->meter_groups); - build_lrouter_ipv4_ip_input(op, lsi->lflows, + build_lrouter_ipv4_ip_input(op, lsi->lflows, lrnet_rec, &lsi->match, &lsi->actions, lsi->meter_groups); - build_lrouter_force_snat_flows_op(op, lsi->lflows, &lsi->match, + build_lrouter_force_snat_flows_op(op, lrnet_rec, lsi->lflows, &lsi->match, &lsi->actions); } @@ -16325,6 +16221,7 @@ build_lflows_thread(void *arg) } build_lswitch_and_lrouter_iterate_by_lsp(op, lsi->ls_ports, lsi->lr_ports, + lsi->lr_nats, lsi->meter_groups, &lsi->match, &lsi->actions, @@ -16363,6 +16260,7 @@ build_lflows_thread(void *arg) build_lrouter_flows_for_lb(lb_dps, lsi->lflows, lsi->meter_groups, lsi->lr_datapaths, + lsi->lr_nats, lsi->features, lsi->svc_monitor_map, &lsi->match, &lsi->actions); @@ -16433,6 +16331,7 @@ build_lswitch_and_lrouter_flows(const struct ovn_datapaths *ls_datapaths, const struct hmap *ls_ports, const struct hmap *lr_ports, const struct ls_port_group_table *ls_pgs, + const struct lr_nat_table *lr_nats, struct hmap *lflows, struct hmap *igmp_groups, const struct shash *meter_groups, @@ -16462,6 +16361,7 @@ build_lswitch_and_lrouter_flows(const struct ovn_datapaths *ls_datapaths, lsiv[index].ls_ports = ls_ports; lsiv[index].lr_ports = lr_ports; lsiv[index].ls_port_groups = ls_pgs; + lsiv[index].lr_nats = lr_nats; lsiv[index].igmp_groups = igmp_groups; lsiv[index].meter_groups = meter_groups; lsiv[index].lb_dps_map = lb_dps_map; @@ -16496,6 +16396,7 @@ build_lswitch_and_lrouter_flows(const struct ovn_datapaths *ls_datapaths, .ls_ports = ls_ports, .lr_ports = lr_ports, .ls_port_groups = ls_pgs, + .lr_nats = lr_nats, .lflows = lflows, .igmp_groups = igmp_groups, .meter_groups = meter_groups, @@ -16523,6 +16424,7 @@ build_lswitch_and_lrouter_flows(const struct ovn_datapaths *ls_datapaths, HMAP_FOR_EACH (op, key_node, ls_ports) { build_lswitch_and_lrouter_iterate_by_lsp(op, lsi.ls_ports, lsi.lr_ports, + lsi.lr_nats, lsi.meter_groups, &lsi.match, &lsi.actions, lsi.lflows); @@ -16539,8 +16441,8 @@ build_lswitch_and_lrouter_flows(const struct ovn_datapaths *ls_datapaths, build_lrouter_defrag_flows_for_lb(lb_dps, lsi.lflows, lsi.lr_datapaths, &lsi.match); build_lrouter_flows_for_lb(lb_dps, lsi.lflows, lsi.meter_groups, - lsi.lr_datapaths, lsi.features, - lsi.svc_monitor_map, + lsi.lr_datapaths, lsi.lr_nats, + lsi.features, lsi.svc_monitor_map, &lsi.match, &lsi.actions); build_lswitch_flows_for_lb(lb_dps, lsi.lflows, lsi.meter_groups, lsi.ls_datapaths, lsi.features, @@ -16643,6 +16545,7 @@ void build_lflows(struct ovsdb_idl_txn *ovnsb_txn, input_data->ls_ports, input_data->lr_ports, input_data->ls_port_groups, + input_data->lr_nats, lflows, &igmp_groups, input_data->meter_groups, @@ -17120,6 +17023,7 @@ lflow_handle_northd_port_changes(struct ovsdb_idl_txn *ovnsb_txn, struct ds actions = DS_EMPTY_INITIALIZER; build_lswitch_and_lrouter_iterate_by_lsp(op, lflow_input->ls_ports, lflow_input->lr_ports, + lflow_input->lr_nats, lflow_input->meter_groups, &match, &actions, lflows); @@ -17156,6 +17060,7 @@ lflow_handle_northd_port_changes(struct ovsdb_idl_txn *ovnsb_txn, struct ds actions = DS_EMPTY_INITIALIZER; build_lswitch_and_lrouter_iterate_by_lsp(op, lflow_input->ls_ports, lflow_input->lr_ports, + lflow_input->lr_nats, lflow_input->meter_groups, &match, &actions, lflows); diff --git a/northd/northd.h b/northd/northd.h index 233dca8084..4dd260761e 100644 --- a/northd/northd.h +++ b/northd/northd.h @@ -83,6 +83,12 @@ struct ovn_datapaths { struct ovn_datapath **array; }; +static inline size_t +ods_size(const struct ovn_datapaths *datapaths) +{ + return hmap_count(&datapaths->datapaths); +} + struct tracked_ovn_ports { /* tracked created ports. * hmapx node data is 'struct ovn_port *' */ @@ -109,8 +115,9 @@ struct tracked_lbs { enum northd_tracked_data_type { NORTHD_TRACKED_NONE, - NORTHD_TRACKED_PORTS = (1 << 0), - NORTHD_TRACKED_LBS = (1 << 1), + NORTHD_TRACKED_PORTS = (1 << 0), + NORTHD_TRACKED_LBS = (1 << 1), + NORTHD_TRACKED_LR_NATS = (1 << 2), }; /* Track what's changed in the northd engine node. @@ -121,6 +128,10 @@ struct northd_tracked_data { enum northd_tracked_data_type type; struct tracked_ovn_ports trk_lsps; struct tracked_lbs trk_lbs; + + /* Tracked logical routers whose NATs have changed. + * hmapx node is 'struct ovn_datapath *'. */ + struct hmapx lr_with_changed_nats; }; struct northd_data { @@ -148,6 +159,8 @@ struct lflow_data { void lflow_data_init(struct lflow_data *); void lflow_data_destroy(struct lflow_data *); +struct lr_nat_table; + struct lflow_input { /* Northbound table references */ const struct nbrec_bfd_table *nbrec_bfd_table; @@ -166,6 +179,7 @@ struct lflow_input { const struct hmap *ls_ports; const struct hmap *lr_ports; const struct ls_port_group_table *ls_port_groups; + const struct lr_nat_table *lr_nats; const struct shash *meter_groups; const struct hmap *lb_datapaths_map; const struct hmap *bfd_connections; @@ -302,24 +316,9 @@ struct ovn_datapath { struct ovn_port **l3dgw_ports; size_t n_l3dgw_ports; - /* NAT entries configured on the router. */ - struct ovn_nat *nat_entries; - size_t n_nat_entries; - - bool has_distributed_nat; /* router datapath has a logical port with redirect-type set to bridged. */ bool redirect_bridged; - /* Set of nat external ips on the router. */ - struct sset external_ips; - - /* SNAT IPs owned by the router (shash of 'struct ovn_snat_ip'). */ - struct shash snat_ips; - - struct lport_addresses dnat_force_snat_addrs; - struct lport_addresses lb_force_snat_addrs; - bool lb_force_snat_router_ip; - /* Load Balancer vIPs relevant for this datapath. */ struct ovn_lb_ip_set *lb_ips; @@ -336,6 +335,9 @@ struct ovn_datapath { struct hmap ports; }; +const struct ovn_datapath *ovn_datapath_find(const struct hmap *datapaths, + const struct uuid *uuid); + void ovnnb_db_run(struct northd_input *input_data, struct northd_data *data, struct ovsdb_idl_txn *ovnnb_txn, @@ -396,8 +398,8 @@ void sync_lbs(struct ovsdb_idl_txn *, const struct sbrec_load_balancer_table *, bool check_sb_lb_duplicates(const struct sbrec_load_balancer_table *); void sync_pbs(struct ovsdb_idl_txn *, struct hmap *ls_ports, - struct hmap *lr_ports); -bool sync_pbs_for_northd_changed_ovn_ports( struct tracked_ovn_ports *); + struct hmap *lr_ports, const struct lr_nat_table *); +bool sync_pbs_for_northd_changed_ovn_ports(struct tracked_ovn_ports *); static inline bool northd_has_tracked_data(struct northd_tracked_data *trk_nd_changes) { @@ -416,4 +418,10 @@ northd_has_lsps_in_tracked_data(struct northd_tracked_data *trk_nd_changes) return (trk_nd_changes->type & NORTHD_TRACKED_PORTS); } +static inline bool +northd_has_lr_nats_in_tracked_data(struct northd_tracked_data *trk_nd_changes) +{ + return (trk_nd_changes->type & NORTHD_TRACKED_LR_NATS); +} + #endif /* NORTHD_H */ diff --git a/northd/ovn-northd.c b/northd/ovn-northd.c index f3868068d3..40f9764b3a 100644 --- a/northd/ovn-northd.c +++ b/northd/ovn-northd.c @@ -870,6 +870,7 @@ main(int argc, char *argv[]) stopwatch_create(LFLOWS_TO_SB_STOPWATCH_NAME, SW_MS); stopwatch_create(PORT_GROUP_RUN_STOPWATCH_NAME, SW_MS); stopwatch_create(SYNC_METERS_RUN_STOPWATCH_NAME, SW_MS); + stopwatch_create(LR_NAT_RUN_STOPWATCH_NAME, SW_MS); /* Initialize incremental processing engine for ovn-northd */ inc_proc_northd_init(&ovnnb_idl_loop, &ovnsb_idl_loop); diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at index c2ba96e753..dc01e43de0 100644 --- a/tests/ovn-northd.at +++ b/tests/ovn-northd.at @@ -11053,6 +11053,7 @@ check ovn-nbctl --wait=sb lsp-add sw0 sw0p1 -- lsp-set-addresses sw0p1 "00:00:20 check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lr-add lr0 check_engine_stats northd recompute nocompute +check_engine_stats lr_nat recompute nocompute check_engine_stats sync_to_sb_pb recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11064,6 +11065,7 @@ check ovn-nbctl lrp-add lr0 lr0-sw0 00:00:00:00:ff:01 10.0.0.1/24 # first it will be recompute to handle lr0-sw0 and then a compute # for the SB port binding change. check_engine_stats northd recompute compute +check_engine_stats lr_nat recompute nocompute check_engine_stats sync_to_sb_pb recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11074,6 +11076,7 @@ ovn-nbctl lsp-set-addresses sw0-lr0 00:00:00:00:ff:01 check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lsp-set-options sw0-lr0 router-port=lr0-sw0 check_engine_stats northd recompute compute +check_engine_stats lr_nat recompute nocompute check_engine_stats sync_to_sb_pb recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11098,16 +11101,18 @@ ovn-nbctl --wait=hv lrp-set-gateway-chassis lr0-public hv1 20 check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl set logical_router_port lr0-sw0 options:foo=bar check_engine_stats northd recompute nocompute +check_engine_stats lr_nat recompute nocompute check_engine_stats sync_to_sb_pb recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Do checks for NATs. -# Add a NAT. This should not result in recompute of both northd and lflow -# engine nodes. +# Add a NAT. This should not result in recompute of northd, but +# recompute of lflow node. check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lr-nat-add lr0 dnat_and_snat 172.168.0.110 10.0.0.4 -check_engine_stats northd recompute nocompute +check_engine_stats northd norecompute compute +check_engine_stats lr_nat norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_pb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11115,7 +11120,8 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE # Update the NAT options column check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb set NAT . options:foo=bar -check_engine_stats northd recompute nocompute +check_engine_stats northd norecompute compute +check_engine_stats lr_nat norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_pb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11123,7 +11129,8 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE # Update the NAT external_ip column check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb set NAT . external_ip=172.168.0.120 -check_engine_stats northd recompute nocompute +check_engine_stats northd norecompute compute +check_engine_stats lr_nat norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_pb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11131,7 +11138,8 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE # Update the NAT logical_ip column check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb set NAT . logical_ip=10.0.0.10 -check_engine_stats northd recompute nocompute +check_engine_stats northd norecompute compute +check_engine_stats lr_nat norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_pb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11139,7 +11147,8 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE # Update the NAT type check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb set NAT . type=snat -check_engine_stats northd recompute nocompute +check_engine_stats northd norecompute compute +check_engine_stats lr_nat norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_pb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11147,7 +11156,8 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE # Create a dnat_and_snat NAT with external_mac and logical_port check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lr-nat-add lr0 dnat_and_snat 172.168.0.110 10.0.0.4 sw0p1 30:54:00:00:00:03 -check_engine_stats northd recompute compute +check_engine_stats northd norecompute compute +check_engine_stats lr_nat norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_pb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11156,7 +11166,8 @@ nat2_uuid=$(ovn-nbctl --bare --columns _uuid find nat logical_ip=10.0.0.4) check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb set NAT $nat2_uuid external_mac='"30:54:00:00:00:04"' -check_engine_stats northd recompute nocompute +check_engine_stats northd norecompute compute +check_engine_stats lr_nat norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_pb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11171,28 +11182,32 @@ check ovn-nbctl lr-lb-add lr0 lb2 # is a lb vip. check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lr-nat-add lr0 dnat_and_snat 172.168.0.140 10.0.0.20 -check_engine_stats northd recompute nocompute +check_engine_stats northd norecompute compute +check_engine_stats lr_nat norecompute compute check_engine_stats sync_to_sb_pb recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lr-nat-add lr0 dnat_and_snat 172.168.0.150 10.0.0.41 -check_engine_stats northd recompute nocompute +check_engine_stats northd norecompute compute +check_engine_stats lr_nat norecompute compute check_engine_stats sync_to_sb_pb recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lr-nat-del lr0 dnat_and_snat 172.168.0.150 -check_engine_stats northd recompute nocompute +check_engine_stats northd norecompute compute +check_engine_stats lr_nat norecompute compute check_engine_stats sync_to_sb_pb recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lr-nat-del lr0 dnat_and_snat 172.168.0.140 -check_engine_stats northd recompute nocompute +check_engine_stats northd norecompute compute +check_engine_stats lr_nat norecompute compute check_engine_stats sync_to_sb_pb recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11200,7 +11215,8 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE # Delete the NAT check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb clear logical_router lr0 nat -check_engine_stats northd recompute compute +check_engine_stats northd norecompute compute +check_engine_stats lr_nat norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_pb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11209,6 +11225,7 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lr-policy-add lr0 10 "ip4.src == 10.0.0.3" reroute 172.168.0.101,172.168.0.102 check_engine_stats northd recompute nocompute +check_engine_stats lr_nat recompute nocompute check_engine_stats sync_to_sb_pb recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11216,6 +11233,7 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lr-policy-del lr0 10 "ip4.src == 10.0.0.3" check_engine_stats northd recompute nocompute +check_engine_stats lr_nat recompute nocompute check_engine_stats sync_to_sb_pb recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE From patchwork Tue Nov 28 02:35:27 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Numan Siddique X-Patchwork-Id: 1869076 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::138; helo=smtp1.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp1.osuosl.org (smtp1.osuosl.org [IPv6:2605:bc80:3010::138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4SfRRL2l0gz1yRy for ; Tue, 28 Nov 2023 13:36:30 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 8E07F81970; Tue, 28 Nov 2023 02:36:28 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 8E07F81970 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id vKozie8JeITU; Tue, 28 Nov 2023 02:36:25 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp1.osuosl.org (Postfix) with ESMTPS id D3AB980A82; Tue, 28 Nov 2023 02:36:24 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org D3AB980A82 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 92A8EC0DD0; Tue, 28 Nov 2023 02:36:24 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 0E823C0039 for ; Tue, 28 Nov 2023 02:36:23 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 605C460F92 for ; Tue, 28 Nov 2023 02:35:46 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 605C460F92 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id zM50zMKALD1m for ; Tue, 28 Nov 2023 02:35:43 +0000 (UTC) Received: from relay8-d.mail.gandi.net (relay8-d.mail.gandi.net [217.70.183.201]) by smtp3.osuosl.org (Postfix) with ESMTPS id C051860FA1 for ; Tue, 28 Nov 2023 02:35:42 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org C051860FA1 Received: by mail.gandi.net (Postfix) with ESMTPSA id 0F1561BF208; Tue, 28 Nov 2023 02:35:38 +0000 (UTC) From: numans@ovn.org To: dev@openvswitch.org Date: Mon, 27 Nov 2023 21:35:27 -0500 Message-ID: <20231128023527.569693-1-numans@ovn.org> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20231128023410.569539-1-numans@ovn.org> References: <20231128023410.569539-1-numans@ovn.org> MIME-Version: 1.0 X-GND-Sasl: numans@ovn.org Subject: [ovs-dev] [PATCH ovn v3 05/16] northd: Add a new engine 'lr_stateful' to manage lr's stateful data. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Numan Siddique This new engine now maintains the load balancer and NAT data of a logical router which was earlier part of northd engine node data. The main inputs to this engine are: - northd node - lr_nat node - lb_data node A record for each logical router is maintained in the 'lr_stateful_table' hmap table and this record - stores the lb related data - embeds the 'lr_nat' record. This engine node becomes an input to 'lflow' node. Signed-off-by: Numan Siddique --- lib/stopwatch-names.h | 1 + northd/automake.mk | 2 + northd/en-lflow.c | 4 + northd/en-lr-nat.h | 3 + northd/en-lr-stateful.c | 647 ++++++++++++++++++++++++++++++++++++++ northd/en-lr-stateful.h | 104 +++++++ northd/en-sync-sb.c | 49 +-- northd/inc-proc-northd.c | 13 +- northd/northd.c | 655 ++++++++++++--------------------------- northd/northd.h | 136 +++++++- northd/ovn-northd.c | 1 + tests/ovn-northd.at | 62 ++++ 12 files changed, 1191 insertions(+), 486 deletions(-) create mode 100644 northd/en-lr-stateful.c create mode 100644 northd/en-lr-stateful.h diff --git a/lib/stopwatch-names.h b/lib/stopwatch-names.h index 0a16da211e..63a04db9ca 100644 --- a/lib/stopwatch-names.h +++ b/lib/stopwatch-names.h @@ -33,5 +33,6 @@ #define PORT_GROUP_RUN_STOPWATCH_NAME "port_group_run" #define SYNC_METERS_RUN_STOPWATCH_NAME "sync_meters_run" #define LR_NAT_RUN_STOPWATCH_NAME "lr_nat_run" +#define LR_STATEFUL_RUN_STOPWATCH_NAME "lr_stateful" #endif diff --git a/northd/automake.mk b/northd/automake.mk index ae367a2a8b..3666cd452c 100644 --- a/northd/automake.mk +++ b/northd/automake.mk @@ -26,6 +26,8 @@ northd_ovn_northd_SOURCES = \ northd/en-lb-data.h \ northd/en-lr-nat.c \ northd/en-lr-nat.h \ + northd/en-lr-stateful.c \ + northd/en-lr-stateful.h \ northd/inc-proc-northd.c \ northd/inc-proc-northd.h \ northd/ipam.c \ diff --git a/northd/en-lflow.c b/northd/en-lflow.c index e4f875ef7c..0aee073520 100644 --- a/northd/en-lflow.c +++ b/northd/en-lflow.c @@ -20,6 +20,7 @@ #include "en-lflow.h" #include "en-lr-nat.h" +#include "en-lr-stateful.h" #include "en-northd.h" #include "en-meters.h" @@ -43,6 +44,8 @@ lflow_get_input_data(struct engine_node *node, engine_get_input_data("sync_meters", node); struct ed_type_lr_nat_data *lr_nat_data = engine_get_input_data("lr_nat", node); + struct ed_type_lr_stateful *lr_sful_data = + engine_get_input_data("lr_stateful", node); lflow_input->nbrec_bfd_table = EN_OVSDB_GET(engine_get_input("NB_bfd", node)); @@ -66,6 +69,7 @@ lflow_get_input_data(struct engine_node *node, lflow_input->lr_ports = &northd_data->lr_ports; lflow_input->ls_port_groups = &pg_data->ls_port_groups; lflow_input->lr_nats = &lr_nat_data->lr_nats; + lflow_input->lr_sful_table = &lr_sful_data->lr_sful_table; lflow_input->meter_groups = &sync_meters_data->meter_groups; lflow_input->lb_datapaths_map = &northd_data->lb_datapaths_map; lflow_input->svc_monitor_map = &northd_data->svc_monitor_map; diff --git a/northd/en-lr-nat.h b/northd/en-lr-nat.h index 3ec4c7b506..4d8e142ae6 100644 --- a/northd/en-lr-nat.h +++ b/northd/en-lr-nat.h @@ -86,6 +86,9 @@ struct lr_nat_table { const struct lr_nat_record * lr_nat_table_find_by_index( const struct lr_nat_table *, size_t od_index); +#define LR_NAT_TABLE_FOR_EACH(LR_NAT_REC, TABLE) \ + HMAP_FOR_EACH (LR_NAT_REC, key_node, &(TABLE)->entries) + struct ed_type_lr_nat_data { struct lr_nat_table lr_nats; diff --git a/northd/en-lr-stateful.c b/northd/en-lr-stateful.c new file mode 100644 index 0000000000..bd03c586da --- /dev/null +++ b/northd/en-lr-stateful.c @@ -0,0 +1,647 @@ +/* + * Copyright (c) 2023, Red Hat, Inc. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at: + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#include + +#include +#include +#include + +/* OVS includes */ +#include "include/openvswitch/hmap.h" +#include "lib/bitmap.h" +#include "lib/socket-util.h" +#include "lib/uuidset.h" +#include "openvswitch/util.h" +#include "openvswitch/vlog.h" +#include "stopwatch.h" + +/* OVN includes */ +#include "en-lb-data.h" +#include "en-lr-nat.h" +#include "en-lr-stateful.h" +#include "lib/inc-proc-eng.h" +#include "lib/lb.h" +#include "lib/ovn-nb-idl.h" +#include "lib/ovn-sb-idl.h" +#include "lib/ovn-util.h" +#include "lib/stopwatch-names.h" +#include "northd.h" + +VLOG_DEFINE_THIS_MODULE(en_lr_stateful); + +/* Static function declarations. */ +static void lr_stateful_table_init(struct lr_stateful_table *); +static void lr_stateful_table_clear(struct lr_stateful_table *); +static void lr_stateful_table_destroy(struct lr_stateful_table *); +static struct lr_stateful_record *lr_stateful_table_find_( + const struct lr_stateful_table *, const struct nbrec_logical_router *); +static struct lr_stateful_record *lr_stateful_table_find_by_index_( + const struct lr_stateful_table *table, size_t od_index); + +static void lr_stateful_table_build(struct lr_stateful_table *, + const struct lr_nat_table *, + const struct ovn_datapaths *lr_datapaths, + const struct hmap *lb_datapaths_map, + const struct hmap *lbgrp_datapaths_map); + +static struct lr_stateful_input lr_stateful_get_input_data( + struct engine_node *); + +static struct lr_stateful_record *lr_stateful_record_create( + struct lr_stateful_table *, const struct lr_nat_record *, + const struct hmap *lb_datapaths_map, + const struct hmap *lbgrp_datapaths_map); +static void lr_stateful_record_destroy(struct lr_stateful_record *); +static void lr_stateful_record_init( + struct lr_stateful_record *, + const struct hmap *lb_datapaths_map, + const struct hmap *lbgrp_datapaths_map); + +static void build_lrouter_lb_reachable_ips(struct lr_stateful_record *, + const struct ovn_northd_lb *); +static void add_neigh_ips_to_lrouter(struct lr_stateful_record *, + enum lb_neighbor_responder_mode, + const struct sset *lb_ips_v4, + const struct sset *lb_ips_v6); +static void remove_lrouter_lb_reachable_ips(struct lr_stateful_record *, + enum lb_neighbor_responder_mode, + const struct sset *lb_ips_v4, + const struct sset *lb_ips_v6); +static void lr_stateful_build_vip_nats(struct lr_stateful_record *); + +/* 'lr_stateful' engine node manages the NB logical router LB data. + */ +void * +en_lr_stateful_init(struct engine_node *node OVS_UNUSED, + struct engine_arg *arg OVS_UNUSED) +{ + struct ed_type_lr_stateful *data = xzalloc(sizeof *data); + lr_stateful_table_init(&data->lr_sful_table); + hmapx_init(&data->trk_data.crupdated); + return data; +} + +void +en_lr_stateful_cleanup(void *data_) +{ + struct ed_type_lr_stateful *data = + (struct ed_type_lr_stateful *) data_; + lr_stateful_table_destroy(&data->lr_sful_table); + hmapx_destroy(&data->trk_data.crupdated); +} + +void +en_lr_stateful_clear_tracked_data(void *data_) +{ + struct ed_type_lr_stateful *data = (struct ed_type_lr_stateful *) data_; + + hmapx_clear(&data->trk_data.crupdated); +} + +void +en_lr_stateful_run(struct engine_node *node, void *data_) +{ + struct lr_stateful_input input_data = lr_stateful_get_input_data(node); + struct ed_type_lr_stateful *data = data_; + + stopwatch_start(LR_STATEFUL_RUN_STOPWATCH_NAME, time_msec()); + + lr_stateful_table_clear(&data->lr_sful_table); + lr_stateful_table_build(&data->lr_sful_table, input_data.lr_nats, + input_data.lr_datapaths, + input_data.lb_datapaths_map, + input_data.lbgrp_datapaths_map); + + stopwatch_stop(LR_STATEFUL_RUN_STOPWATCH_NAME, time_msec()); + engine_set_node_state(node, EN_UPDATED); +} + +bool +lr_stateful_northd_handler(struct engine_node *node, void *data OVS_UNUSED) +{ + struct northd_data *northd_data = engine_get_input_data("northd", node); + if (!northd_has_tracked_data(&northd_data->trk_data)) { + return false; + } + + /* lr_stateful node needs inputs for any changes to NAT and load balancers. + * Changes to NAT is provided by the lr_nat tracked data and changes + * to lbs and lb grps is provided by lb_data's tracked data. + * So we don't need to do anything here for northd changes. + * But we do need to access the datapaths and lb_datapaths from the + * northd engine node and hence its an input. + * */ + return true; +} + +bool +lr_stateful_lb_data_handler(struct engine_node *node, void *data_) +{ + struct ed_type_lb_data *lb_data = engine_get_input_data("lb_data", node); + if (!lb_data->tracked) { + return false; + } + + struct ed_type_lr_stateful *data = + (struct ed_type_lr_stateful *) data_; + struct lr_stateful_input input_data = + lr_stateful_get_input_data(node); + struct lr_stateful_record *lr_sful_rec; + size_t index; + + const struct tracked_lb_data *trk_lb_data = &lb_data->tracked_lb_data; + const struct ovn_lb_group_datapaths *lbgrp_dps; + const struct crupdated_lbgrp *crupdated_lbgrp; + const struct crupdated_od_lb_data *codlb; + const struct ovn_lb_datapaths *lb_dps; + const struct crupdated_lb *clb; + const struct ovn_northd_lb *lb; + const struct ovn_datapath *od; + + LIST_FOR_EACH (codlb, list_node, &trk_lb_data->crupdated_lr_lbs) { + od = ovn_datapath_find(&input_data.lr_datapaths->datapaths, + &codlb->od_uuid); + ovs_assert(od); + + lr_sful_rec = lr_stateful_table_find_(&data->lr_sful_table, od->nbr); + if (!lr_sful_rec) { + const struct lr_nat_record *lrnat_rec = lr_nat_table_find_by_index( + input_data.lr_nats, od->index); + ovs_assert(lrnat_rec); + + lr_sful_rec = lr_stateful_record_create(&data->lr_sful_table, + lrnat_rec, + input_data.lb_datapaths_map, + input_data.lbgrp_datapaths_map); + + /* Add the lr_sful_rec rec to the tracking data. */ + hmapx_add(&data->trk_data.crupdated, lr_sful_rec); + continue; + } + + struct uuidset_node *uuidnode; + UUIDSET_FOR_EACH (uuidnode, &codlb->assoc_lbs) { + lb_dps = ovn_lb_datapaths_find( + input_data.lb_datapaths_map, &uuidnode->uuid); + ovs_assert(lb_dps); + + /* Add the lb_ips of lb_dps to the od. */ + build_lrouter_lb_ips(lr_sful_rec->lb_ips, lb_dps->lb); + build_lrouter_lb_reachable_ips(lr_sful_rec, lb_dps->lb); + } + + UUIDSET_FOR_EACH (uuidnode, &codlb->assoc_lbgrps) { + lbgrp_dps = ovn_lb_group_datapaths_find( + input_data.lbgrp_datapaths_map, &uuidnode->uuid); + ovs_assert(lbgrp_dps); + + for (size_t j = 0; j < lbgrp_dps->lb_group->n_lbs; j++) { + const struct uuid *lb_uuid + = &lbgrp_dps->lb_group->lbs[j]->nlb->header_.uuid; + lb_dps = ovn_lb_datapaths_find(input_data.lb_datapaths_map, + lb_uuid); + ovs_assert(lb_dps); + + /* Add the lb_ips of lb_dps to the od. */ + build_lrouter_lb_ips(lr_sful_rec->lb_ips, lb_dps->lb); + build_lrouter_lb_reachable_ips(lr_sful_rec, lb_dps->lb); + } + } + + /* Add the lr_sful_rec rec to the tracking data. */ + hmapx_add(&data->trk_data.crupdated, lr_sful_rec); + } + + HMAP_FOR_EACH (clb, hmap_node, &trk_lb_data->crupdated_lbs) { + lb = clb->lb; + const struct uuid *lb_uuid = &lb->nlb->header_.uuid; + + lb_dps = ovn_lb_datapaths_find(input_data.lb_datapaths_map, lb_uuid); + ovs_assert(lb_dps); + + BITMAP_FOR_EACH_1 (index, ods_size(input_data.lr_datapaths), + lb_dps->nb_lr_map) { + od = input_data.lr_datapaths->array[index]; + + lr_sful_rec = lr_stateful_table_find_(&data->lr_sful_table, + od->nbr); + ovs_assert(lr_sful_rec); + + /* Update the od->lb_ips with the deleted and inserted + * vips (if any). */ + remove_ips_from_lb_ip_set(lr_sful_rec->lb_ips, lb->routable, + &clb->deleted_vips_v4, + &clb->deleted_vips_v6); + add_ips_to_lb_ip_set(lr_sful_rec->lb_ips, lb->routable, + &clb->inserted_vips_v4, + &clb->inserted_vips_v6); + + remove_lrouter_lb_reachable_ips(lr_sful_rec, lb->neigh_mode, + &clb->deleted_vips_v4, + &clb->deleted_vips_v6); + add_neigh_ips_to_lrouter(lr_sful_rec, lb->neigh_mode, + &clb->inserted_vips_v4, + &clb->inserted_vips_v6); + + /* Add the lr_sful_rec rec to the tracking data. */ + hmapx_add(&data->trk_data.crupdated, lr_sful_rec); + } + } + + HMAP_FOR_EACH (crupdated_lbgrp, hmap_node, + &trk_lb_data->crupdated_lbgrps) { + const struct uuid *lb_uuid = &crupdated_lbgrp->lbgrp->uuid; + + lbgrp_dps = ovn_lb_group_datapaths_find(input_data.lbgrp_datapaths_map, + lb_uuid); + ovs_assert(lbgrp_dps); + + struct hmapx_node *hnode; + HMAPX_FOR_EACH (hnode, &crupdated_lbgrp->assoc_lbs) { + lb = hnode->data; + lb_uuid = &lb->nlb->header_.uuid; + lb_dps = ovn_lb_datapaths_find(input_data.lb_datapaths_map, + lb_uuid); + ovs_assert(lb_dps); + for (size_t i = 0; i < lbgrp_dps->n_lr; i++) { + od = lbgrp_dps->lr[i]; + lr_sful_rec = lr_stateful_table_find_(&data->lr_sful_table, + od->nbr); + ovs_assert(lr_sful_rec); + /* Add the lb_ips of lb_dps to the lr lb data. */ + build_lrouter_lb_ips(lr_sful_rec->lb_ips, lb_dps->lb); + build_lrouter_lb_reachable_ips(lr_sful_rec, lb_dps->lb); + + /* Add the lr_sful_rec rec to the tracking data. */ + hmapx_add(&data->trk_data.crupdated, lr_sful_rec); + } + } + } + + if (lr_stateful_has_tracked_data(&data->trk_data)) { + struct hmapx_node *hmapx_node; + /* For all the modified lr_stateful records (re)build the + * vip nats. */ + HMAPX_FOR_EACH (hmapx_node, &data->trk_data.crupdated) { + lr_stateful_build_vip_nats(hmapx_node->data); + } + + engine_set_node_state(node, EN_UPDATED); + } + + return true; +} + +bool +lr_stateful_lr_nat_handler(struct engine_node *node, void *data_) +{ + struct ed_type_lr_nat_data *lr_nat_data = + engine_get_input_data("lr_nat", node); + + if (!lr_nat_has_tracked_data(&lr_nat_data->trk_data)) { + return false; + } + + struct ed_type_lr_stateful *data = + (struct ed_type_lr_stateful *) data_; + struct lr_stateful_input input_data = + lr_stateful_get_input_data(node); + const struct lr_nat_record *lrnat_rec; + struct lr_stateful_record *lr_sful_rec; + struct hmapx_node *hmapx_node; + + HMAPX_FOR_EACH (hmapx_node, &lr_nat_data->trk_data.crupdated) { + lrnat_rec = hmapx_node->data; + lr_sful_rec = lr_stateful_table_find_(&data->lr_sful_table, + lrnat_rec->od->nbr); + if (!lr_sful_rec) { + lr_sful_rec = lr_stateful_record_create(&data->lr_sful_table, + lrnat_rec, + input_data.lb_datapaths_map, + input_data.lbgrp_datapaths_map); + } else { + lr_stateful_build_vip_nats(lr_sful_rec); + } + + /* Add the lr_sful_rec rec to the tracking data. */ + hmapx_add(&data->trk_data.crupdated, lr_sful_rec); + } + + if (lr_stateful_has_tracked_data(&data->trk_data)) { + engine_set_node_state(node, EN_UPDATED); + } + + return true; +} + +const struct lr_stateful_record * +lr_stateful_table_find_by_index(const struct lr_stateful_table *table, + size_t od_index) +{ + return lr_stateful_table_find_by_index_(table, od_index); +} + +/* static functions. */ +static void +lr_stateful_table_init(struct lr_stateful_table *table) +{ + *table = (struct lr_stateful_table) { + .entries = HMAP_INITIALIZER(&table->entries), + }; +} + +static void +lr_stateful_table_destroy(struct lr_stateful_table *table) +{ + lr_stateful_table_clear(table); + hmap_destroy(&table->entries); +} + +static void +lr_stateful_table_clear(struct lr_stateful_table *table) +{ + struct lr_stateful_record *lr_sful_rec; + HMAP_FOR_EACH_POP (lr_sful_rec, key_node, &table->entries) { + lr_stateful_record_destroy(lr_sful_rec); + } + + free(table->array); + table->array = NULL; +} + +static void +lr_stateful_table_build(struct lr_stateful_table *table, + const struct lr_nat_table *lr_nats, + const struct ovn_datapaths *lr_datapaths, + const struct hmap *lb_datapaths_map, + const struct hmap *lbgrp_datapaths_map) +{ + table->array = xrealloc(table->array, + ods_size(lr_datapaths) * sizeof *table->array); + const struct lr_nat_record *lrnat_rec; + LR_NAT_TABLE_FOR_EACH (lrnat_rec, lr_nats) { + lr_stateful_record_create(table, lrnat_rec, lb_datapaths_map, + lbgrp_datapaths_map); + } +} + +static struct lr_stateful_record * +lr_stateful_table_find_(const struct lr_stateful_table *table, + const struct nbrec_logical_router *nbr) +{ + struct lr_stateful_record *lr_sful_rec; + + HMAP_FOR_EACH_WITH_HASH (lr_sful_rec, key_node, + uuid_hash(&nbr->header_.uuid), &table->entries) { + if (nbr == lr_sful_rec->od->nbr) { + return lr_sful_rec; + } + } + return NULL; +} + +static struct lr_stateful_record * +lr_stateful_table_find_by_index_(const struct lr_stateful_table *table, + size_t od_index) +{ + ovs_assert(od_index <= hmap_count(&table->entries)); + return table->array[od_index]; +} + +static struct lr_stateful_record * +lr_stateful_record_create(struct lr_stateful_table *table, + const struct lr_nat_record *lrnat_rec, + const struct hmap *lb_datapaths_map, + const struct hmap *lbgrp_datapaths_map) +{ + struct lr_stateful_record *lr_sful_rec = xzalloc(sizeof *lr_sful_rec); + lr_sful_rec->lrnat_rec = lrnat_rec; + lr_sful_rec->od = lrnat_rec->od; + lr_stateful_record_init(lr_sful_rec, lb_datapaths_map, + lbgrp_datapaths_map); + + hmap_insert(&table->entries, &lr_sful_rec->key_node, + uuid_hash(&lr_sful_rec->od->nbr->header_.uuid)); + + table->array[lr_sful_rec->od->index] = lr_sful_rec; + return lr_sful_rec; +} + +static void +lr_stateful_record_destroy(struct lr_stateful_record *lr_sful_rec) +{ + ovn_lb_ip_set_destroy(lr_sful_rec->lb_ips); + lr_sful_rec->lb_ips = NULL; + sset_destroy(&lr_sful_rec->vip_nats); + free(lr_sful_rec); +} + +static void +lr_stateful_record_init(struct lr_stateful_record *lr_sful_rec, + const struct hmap *lb_datapaths_map, + const struct hmap *lbgrp_datapaths_map) +{ + const struct nbrec_load_balancer_group *nbrec_lb_group; + const struct ovn_lb_group_datapaths *lb_group_dps; + const struct ovn_lb_datapaths *lb_dps; + + /* Checking load balancer groups first, starting from the largest one, + * to more efficiently copy IP sets. */ + size_t largest_group = 0; + + const struct nbrec_logical_router *nbr = lr_sful_rec->od->nbr; + for (size_t i = 1; i < nbr->n_load_balancer_group; i++) { + if (nbr->load_balancer_group[i]->n_load_balancer > + nbr->load_balancer_group[largest_group]->n_load_balancer) { + largest_group = i; + } + } + + for (size_t i = 0; i < nbr->n_load_balancer_group; i++) { + size_t idx = (i + largest_group) % nbr->n_load_balancer_group; + + nbrec_lb_group = nbr->load_balancer_group[idx]; + const struct uuid *lbgrp_uuid = &nbrec_lb_group->header_.uuid; + + lb_group_dps = + ovn_lb_group_datapaths_find(lbgrp_datapaths_map, + lbgrp_uuid); + ovs_assert(lb_group_dps); + + if (!lr_sful_rec->lb_ips) { + lr_sful_rec->lb_ips = + ovn_lb_ip_set_clone(lb_group_dps->lb_group->lb_ips); + } else { + for (size_t j = 0; j < lb_group_dps->lb_group->n_lbs; j++) { + build_lrouter_lb_ips(lr_sful_rec->lb_ips, + lb_group_dps->lb_group->lbs[j]); + } + } + + for (size_t j = 0; j < lb_group_dps->lb_group->n_lbs; j++) { + build_lrouter_lb_reachable_ips(lr_sful_rec, + lb_group_dps->lb_group->lbs[j]); + } + } + + if (!lr_sful_rec->lb_ips) { + lr_sful_rec->lb_ips = ovn_lb_ip_set_create(); + } + + for (size_t i = 0; i < nbr->n_load_balancer; i++) { + const struct uuid *lb_uuid = + &nbr->load_balancer[i]->header_.uuid; + lb_dps = ovn_lb_datapaths_find(lb_datapaths_map, lb_uuid); + ovs_assert(lb_dps); + build_lrouter_lb_ips(lr_sful_rec->lb_ips, lb_dps->lb); + build_lrouter_lb_reachable_ips(lr_sful_rec, lb_dps->lb); + } + + sset_init(&lr_sful_rec->vip_nats); + + if (!nbr->n_nat) { + lr_stateful_build_vip_nats(lr_sful_rec); + } +} + +static struct lr_stateful_input +lr_stateful_get_input_data(struct engine_node *node) +{ + struct northd_data *northd_data = engine_get_input_data("northd", node); + struct ed_type_lr_nat_data *lr_nat_data = + engine_get_input_data("lr_nat", node); + + return (struct lr_stateful_input) { + .lr_datapaths = &northd_data->lr_datapaths, + .lb_datapaths_map = &northd_data->lb_datapaths_map, + .lbgrp_datapaths_map = &northd_data->lb_group_datapaths_map, + .lr_nats = &lr_nat_data->lr_nats, + }; +} + +static void +build_lrouter_lb_reachable_ips(struct lr_stateful_record *lr_sful_rec, + const struct ovn_northd_lb *lb) +{ + add_neigh_ips_to_lrouter(lr_sful_rec, lb->neigh_mode, &lb->ips_v4, + &lb->ips_v6); +} + +static void +add_neigh_ips_to_lrouter(struct lr_stateful_record *lr_sful_rec, + enum lb_neighbor_responder_mode neigh_mode, + const struct sset *lb_ips_v4, + const struct sset *lb_ips_v6) +{ + /* If configured to not reply to any neighbor requests for all VIPs + * return early. + */ + if (neigh_mode == LB_NEIGH_RESPOND_NONE) { + return; + } + + const char *ip_address; + + /* If configured to reply to neighbor requests for all VIPs force them + * all to be considered "reachable". + */ + if (neigh_mode == LB_NEIGH_RESPOND_ALL) { + SSET_FOR_EACH (ip_address, lb_ips_v4) { + sset_add(&lr_sful_rec->lb_ips->ips_v4_reachable, ip_address); + } + SSET_FOR_EACH (ip_address, lb_ips_v6) { + sset_add(&lr_sful_rec->lb_ips->ips_v6_reachable, ip_address); + } + + return; + } + + /* Otherwise, a VIP is reachable if there's at least one router + * subnet that includes it. + */ + ovs_assert(neigh_mode == LB_NEIGH_RESPOND_REACHABLE); + + SSET_FOR_EACH (ip_address, lb_ips_v4) { + struct ovn_port *op; + ovs_be32 vip_ip4; + if (ip_parse(ip_address, &vip_ip4)) { + HMAP_FOR_EACH (op, dp_node, &lr_sful_rec->od->ports) { + if (lrouter_port_ipv4_reachable(op, vip_ip4)) { + sset_add(&lr_sful_rec->lb_ips->ips_v4_reachable, + ip_address); + break; + } + } + } + } + + SSET_FOR_EACH (ip_address, lb_ips_v6) { + struct ovn_port *op; + struct in6_addr vip; + if (ipv6_parse(ip_address, &vip)) { + HMAP_FOR_EACH (op, dp_node, &lr_sful_rec->od->ports) { + if (lrouter_port_ipv6_reachable(op, &vip)) { + sset_add(&lr_sful_rec->lb_ips->ips_v6_reachable, + ip_address); + break; + } + } + } + } +} + +static void +remove_lrouter_lb_reachable_ips(struct lr_stateful_record *lr_sful_rec, + enum lb_neighbor_responder_mode neigh_mode, + const struct sset *lb_ips_v4, + const struct sset *lb_ips_v6) +{ + if (neigh_mode == LB_NEIGH_RESPOND_NONE) { + return; + } + + const char *ip_address; + SSET_FOR_EACH (ip_address, lb_ips_v4) { + sset_find_and_delete(&lr_sful_rec->lb_ips->ips_v4_reachable, + ip_address); + } + SSET_FOR_EACH (ip_address, lb_ips_v6) { + sset_find_and_delete(&lr_sful_rec->lb_ips->ips_v6_reachable, + ip_address); + } +} + +static void +lr_stateful_build_vip_nats(struct lr_stateful_record *lr_sful_rec) +{ + sset_clear(&lr_sful_rec->vip_nats); + const char *external_ip; + SSET_FOR_EACH (external_ip, &lr_sful_rec->lrnat_rec->external_ips) { + bool is_vip_nat = false; + if (addr_is_ipv6(external_ip)) { + is_vip_nat = sset_contains(&lr_sful_rec->lb_ips->ips_v6, + external_ip); + } else { + is_vip_nat = sset_contains(&lr_sful_rec->lb_ips->ips_v4, + external_ip); + } + + if (is_vip_nat) { + sset_add(&lr_sful_rec->vip_nats, external_ip); + } + } +} diff --git a/northd/en-lr-stateful.h b/northd/en-lr-stateful.h new file mode 100644 index 0000000000..a72f000d36 --- /dev/null +++ b/northd/en-lr-stateful.h @@ -0,0 +1,104 @@ +/* + * Copyright (c) 2023, Red Hat, Inc. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at: + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +#ifndef EN_LR_STATEFUL_H +#define EN_LR_STATEFUL_H 1 + +#include + +/* OVS includes. */ +#include "lib/hmapx.h" +#include "openvswitch/hmap.h" +#include "sset.h" + +/* OVN includes. */ +#include "lib/inc-proc-eng.h" +#include "lib/lb.h" +#include "lib/ovn-nb-idl.h" +#include "lib/ovn-sb-idl.h" +#include "lib/ovn-util.h" + +struct ovn_datapath; +struct lr_nat_record; + +/* lr_stateful_table: This represents a table of logical routers with + * stateful related data. + * stateful related data has two main components + * - NAT and + * - Load balancers. + * + * lr_stateful_record: It is a record in the lr_stateful_table for each + * logical router. + */ + +struct lr_stateful_record { + struct hmap_node key_node; /* Index on 'nbr->header_.uuid'. */ + + const struct ovn_datapath *od; + const struct lr_nat_record *lrnat_rec; + + /* Load Balancer vIPs relevant for this datapath. */ + struct ovn_lb_ip_set *lb_ips; + + /* sset of vips which are also part of lr nats. */ + struct sset vip_nats; +}; + +struct lr_stateful_table { + struct hmap entries; + + /* The array index of each element in 'entries'. */ + struct lr_stateful_record **array; +}; + +#define LR_STATEFUL_TABLE_FOR_EACH(LR_LB_NAT_REC, TABLE) \ + HMAP_FOR_EACH (LR_LB_NAT_REC, key_node, &(TABLE)->entries) + +struct lr_stateful_tracked_data { + /* Created or updated logical router with LB and/or NAT data. */ + struct hmapx crupdated; /* Stores 'struct lr_stateful_record'. */ +}; + +struct ed_type_lr_stateful { + struct lr_stateful_table lr_sful_table; + + struct lr_stateful_tracked_data trk_data; +}; + +struct lr_stateful_input { + const struct ovn_datapaths *lr_datapaths; + const struct hmap *lb_datapaths_map; + const struct hmap *lbgrp_datapaths_map; + const struct lr_nat_table *lr_nats; +}; + +void *en_lr_stateful_init(struct engine_node *, struct engine_arg *); +void en_lr_stateful_cleanup(void *data); +void en_lr_stateful_clear_tracked_data(void *data); +void en_lr_stateful_run(struct engine_node *, void *data); + +bool lr_stateful_northd_handler(struct engine_node *, void *data); +bool lr_stateful_lr_nat_handler(struct engine_node *, void *data); +bool lr_stateful_lb_data_handler(struct engine_node *, void *data); + +const struct lr_stateful_record *lr_stateful_table_find_by_index( + const struct lr_stateful_table *, size_t od_index); + +static inline bool +lr_stateful_has_tracked_data(struct lr_stateful_tracked_data *trk_data) { + return !hmapx_is_empty(&trk_data->crupdated); +} + +#endif /* EN_lr_stateful_H */ diff --git a/northd/en-sync-sb.c b/northd/en-sync-sb.c index 11e12428f7..733fb9024e 100644 --- a/northd/en-sync-sb.c +++ b/northd/en-sync-sb.c @@ -22,6 +22,7 @@ #include "openvswitch/util.h" #include "en-lr-nat.h" +#include "en-lr-stateful.h" #include "en-sync-sb.h" #include "lib/inc-proc-eng.h" #include "lib/lb.h" @@ -41,7 +42,7 @@ static void sync_addr_sets(struct ovsdb_idl_txn *ovnsb_txn, const struct nbrec_address_set_table *, const struct nbrec_port_group_table *, const struct sbrec_address_set_table *, - const struct ovn_datapaths *lr_datapaths); + const struct lr_stateful_table *); static const struct sbrec_address_set *sb_address_set_lookup_by_name( struct ovsdb_idl_index *, const char *name); static void update_sb_addr_set(struct sorted_array *, @@ -87,11 +88,11 @@ en_sync_to_sb_addr_set_run(struct engine_node *node, void *data OVS_UNUSED) EN_OVSDB_GET(engine_get_input("SB_address_set", node)); const struct engine_context *eng_ctx = engine_get_context(); - struct northd_data *northd_data = engine_get_input_data("northd", node); - + const struct ed_type_lr_stateful *lr_stateful_data = + engine_get_input_data("lr_stateful", node); sync_addr_sets(eng_ctx->ovnsb_idl_txn, nb_address_set_table, nb_port_group_table, sb_address_set_table, - &northd_data->lr_datapaths); + &lr_stateful_data->lr_sful_table); engine_set_node_state(node, EN_UPDATED); } @@ -288,10 +289,12 @@ en_sync_to_sb_pb_run(struct engine_node *node, void *data OVS_UNUSED) { const struct engine_context *eng_ctx = engine_get_context(); struct northd_data *northd_data = engine_get_input_data("northd", node); - struct ed_type_lr_nat_data *lr_nat_data = - engine_get_input_data("lr_nat", node); + struct ed_type_lr_stateful *lr_stateful_data = + engine_get_input_data("lr_stateful", node); + sync_pbs(eng_ctx->ovnsb_idl_txn, &northd_data->ls_ports, - &northd_data->lr_ports, &lr_nat_data->lr_nats); + &northd_data->lr_ports, + &lr_stateful_data->lr_sful_table); engine_set_node_state(node, EN_UPDATED); } @@ -316,7 +319,11 @@ sync_to_sb_pb_northd_handler(struct engine_node *node, void *data OVS_UNUSED) return false; } - if (!sync_pbs_for_northd_changed_ovn_ports(&nd->trk_data.trk_lsps)) { + struct ed_type_lr_stateful *lr_sful_data = + engine_get_input_data("lr_stateful", node); + + if (!sync_pbs_for_northd_changed_ovn_ports(&nd->trk_data.trk_lsps, + &lr_sful_data->lr_sful_table)) { return false; } @@ -362,7 +369,7 @@ sync_addr_sets(struct ovsdb_idl_txn *ovnsb_txn, const struct nbrec_address_set_table *nb_address_set_table, const struct nbrec_port_group_table *nb_port_group_table, const struct sbrec_address_set_table *sb_address_set_table, - const struct ovn_datapaths *lr_datapaths) + const struct lr_stateful_table *lr_statefuls) { struct shash sb_address_sets = SHASH_INITIALIZER(&sb_address_sets); @@ -406,16 +413,14 @@ sync_addr_sets(struct ovsdb_idl_txn *ovnsb_txn, } /* Sync router load balancer VIP generated address sets. */ - struct ovn_datapath *od; - HMAP_FOR_EACH (od, key_node, &lr_datapaths->datapaths) { - ovs_assert(od->nbr); - - if (sset_count(&od->lb_ips->ips_v4_reachable)) { - char *ipv4_addrs_name = lr_lb_address_set_name(od->tunnel_key, - AF_INET); + const struct lr_stateful_record *lr_sful_rec; + LR_STATEFUL_TABLE_FOR_EACH (lr_sful_rec, lr_statefuls) { + if (sset_count(&lr_sful_rec->lb_ips->ips_v4_reachable)) { + char *ipv4_addrs_name = + lr_lb_address_set_name(lr_sful_rec->od->tunnel_key, AF_INET); struct sorted_array ipv4_addrs_sorted = - sorted_array_from_sset(&od->lb_ips->ips_v4_reachable); + sorted_array_from_sset(&lr_sful_rec->lb_ips->ips_v4_reachable); sync_addr_set(ovnsb_txn, ipv4_addrs_name, &ipv4_addrs_sorted, &sb_address_sets); @@ -423,11 +428,11 @@ sync_addr_sets(struct ovsdb_idl_txn *ovnsb_txn, free(ipv4_addrs_name); } - if (sset_count(&od->lb_ips->ips_v6_reachable)) { - char *ipv6_addrs_name = lr_lb_address_set_name(od->tunnel_key, - AF_INET6); - struct sorted_array ipv6_addrs_sorted = - sorted_array_from_sset(&od->lb_ips->ips_v6_reachable); + if (sset_count(&lr_sful_rec->lb_ips->ips_v6_reachable)) { + char *ipv6_addrs_name = + lr_lb_address_set_name(lr_sful_rec->od->tunnel_key, AF_INET6); + struct sorted_array ipv6_addrs_sorted = sorted_array_from_sset( + &lr_sful_rec->lb_ips->ips_v6_reachable); sync_addr_set(ovnsb_txn, ipv6_addrs_name, &ipv6_addrs_sorted, &sb_address_sets); diff --git a/northd/inc-proc-northd.c b/northd/inc-proc-northd.c index 1f211b278e..97bcce9655 100644 --- a/northd/inc-proc-northd.c +++ b/northd/inc-proc-northd.c @@ -31,6 +31,7 @@ #include "openvswitch/vlog.h" #include "inc-proc-northd.h" #include "en-lb-data.h" +#include "en-lr-stateful.h" #include "en-lr-nat.h" #include "en-northd.h" #include "en-lflow.h" @@ -148,6 +149,7 @@ static ENGINE_NODE(sync_to_sb_lb, "sync_to_sb_lb"); static ENGINE_NODE(sync_to_sb_pb, "sync_to_sb_pb"); static ENGINE_NODE_WITH_CLEAR_TRACK_DATA(lb_data, "lb_data"); static ENGINE_NODE_WITH_CLEAR_TRACK_DATA(lr_nat, "lr_nat"); +static ENGINE_NODE_WITH_CLEAR_TRACK_DATA(lr_stateful, "lr_stateful"); void inc_proc_northd_init(struct ovsdb_idl_loop *nb, struct ovsdb_idl_loop *sb) @@ -193,6 +195,11 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, engine_add_input(&en_lr_nat, &en_northd, lr_nat_northd_handler); + engine_add_input(&en_lr_stateful, &en_northd, lr_stateful_northd_handler); + engine_add_input(&en_lr_stateful, &en_lr_nat, lr_stateful_lr_nat_handler); + engine_add_input(&en_lr_stateful, &en_lb_data, + lr_stateful_lb_data_handler); + engine_add_input(&en_mac_binding_aging, &en_nb_nb_global, NULL); engine_add_input(&en_mac_binding_aging, &en_sb_mac_binding, NULL); engine_add_input(&en_mac_binding_aging, &en_northd, NULL); @@ -214,15 +221,17 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, engine_add_input(&en_lflow, &en_sb_logical_flow, NULL); engine_add_input(&en_lflow, &en_sb_multicast_group, NULL); engine_add_input(&en_lflow, &en_sb_igmp_group, NULL); + engine_add_input(&en_lflow, &en_lr_nat, NULL); + engine_add_input(&en_lflow, &en_lr_stateful, NULL); engine_add_input(&en_lflow, &en_northd, lflow_northd_handler); engine_add_input(&en_lflow, &en_port_group, lflow_port_group_handler); - engine_add_input(&en_lflow, &en_lr_nat, NULL); engine_add_input(&en_sync_to_sb_addr_set, &en_nb_address_set, sync_to_sb_addr_set_nb_address_set_handler); engine_add_input(&en_sync_to_sb_addr_set, &en_nb_port_group, sync_to_sb_addr_set_nb_port_group_handler); engine_add_input(&en_sync_to_sb_addr_set, &en_northd, NULL); + engine_add_input(&en_sync_to_sb_addr_set, &en_lr_stateful, NULL); engine_add_input(&en_sync_to_sb_addr_set, &en_sb_address_set, NULL); engine_add_input(&en_port_group, &en_nb_port_group, @@ -240,7 +249,7 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, engine_add_input(&en_sync_to_sb_pb, &en_northd, sync_to_sb_pb_northd_handler); - engine_add_input(&en_sync_to_sb_pb, &en_lr_nat, NULL); + engine_add_input(&en_sync_to_sb_pb, &en_lr_stateful, NULL); /* en_sync_to_sb engine node syncs the SB database tables from * the NB database tables. diff --git a/northd/northd.c b/northd/northd.c index 7f59415bbd..e9b8332e21 100644 --- a/northd/northd.c +++ b/northd/northd.c @@ -44,6 +44,7 @@ #include "northd.h" #include "en-lb-data.h" #include "en-lr-nat.h" +#include "en-lr-stateful.h" #include "lib/ovn-parallel-hmap.h" #include "ovn/actions.h" #include "ovn/features.h" @@ -617,13 +618,6 @@ init_lb_for_datapath(struct ovn_datapath *od) } } -static void -destroy_lb_for_datapath(struct ovn_datapath *od) -{ - ovn_lb_ip_set_destroy(od->lb_ips); - od->lb_ips = NULL; -} - /* A group of logical router datapaths which are connected - either * directly or indirectly. * Each logical router can belong to only one group. */ @@ -676,7 +670,6 @@ ovn_datapath_destroy(struct hmap *datapaths, struct ovn_datapath *od) destroy_ipam_info(&od->ipam_info); free(od->router_ports); free(od->ls_peers); - destroy_lb_for_datapath(od); free(od->localnet_ports); free(od->l3dgw_ports); destroy_mcast_info_for_datapath(od); @@ -1311,121 +1304,6 @@ struct lflow_ref_node { struct ovn_lflow *lflow; }; -/* A logical switch port or logical router port. - * - * In steady state, an ovn_port points to a northbound Logical_Switch_Port - * record (via 'nbsp') *or* a Logical_Router_Port record (via 'nbrp'), and to a - * southbound Port_Binding record (via 'sb'). As the state of the system - * changes, join_logical_ports() may determine that there is a new LSP or LRP - * that has no corresponding Port_Binding record (in which case build_ports()) - * will create the missing Port_Binding) or that a Port_Binding record exists - * that has no coresponding LSP (in which case build_ports() will delete the - * spurious Port_Binding). Thus, after build_ports() runs, any given ovn_port - * will have 'sb' nonnull, and 'nbsp' xor 'nbrp' nonnull. - * - * Ordinarily there is only one ovn_port that points to a given LSP or LRP (but - * distributed gateway ports point a "derived" ovn_port to a duplicate LRP). - */ -struct ovn_port { - /* Port name aka key. - * - * This is ordinarily the same as nbsp->name or nbrp->name and - * sb->logical_port. (A distributed gateway port creates a "derived" - * ovn_port with key "cr-%s" % nbrp->name.) */ - struct hmap_node key_node; /* Index on 'key'. */ - char *key; /* nbsp->name, nbrp->name, sb->logical_port. */ - char *json_key; /* 'key', quoted for use in JSON. */ - - const struct sbrec_port_binding *sb; /* May be NULL. */ - - uint32_t tunnel_key; - - /* Logical switch port data. */ - const struct nbrec_logical_switch_port *nbsp; /* May be NULL. */ - - struct lport_addresses *lsp_addrs; /* Logical switch port addresses. */ - unsigned int n_lsp_addrs; /* Total length of lsp_addrs. */ - unsigned int n_lsp_non_router_addrs; /* Number of elements from the - * beginning of 'lsp_addrs' extracted - * directly from LSP 'addresses'. */ - - struct lport_addresses *ps_addrs; /* Port security addresses. */ - unsigned int n_ps_addrs; - - bool lsp_can_be_inc_processed; /* If it can be incrementally processed when - the port changes. */ - - /* Logical router port data. */ - const struct nbrec_logical_router_port *nbrp; /* May be NULL. */ - - struct lport_addresses lrp_networks; - - struct ovn_port_routable_addresses routables; - - /* Logical port multicast data. */ - struct mcast_port_info mcast_info; - - /* At most one of l3dgw_port and cr_port can be not NULL. */ - - /* This is set to a distributed gateway port if and only if this ovn_port - * is "derived" from it. Otherwise this is set to NULL. The derived - * ovn_port represents the instance of distributed gateway port on the - * gateway chassis.*/ - struct ovn_port *l3dgw_port; - - /* This is set to the "derived" chassis-redirect port of this port if and - * only if this port is a distributed gateway port. Otherwise this is set - * to NULL. */ - struct ovn_port *cr_port; - - bool has_unknown; /* If the addresses have 'unknown' defined. */ - - bool has_bfd; - - /* The port's peer: - * - * - A switch port S of type "router" has a router port R as a peer, - * and R in turn has S has its peer. - * - * - Two connected logical router ports have each other as peer. - * - * - Other kinds of ports have no peer. */ - struct ovn_port *peer; - - struct ovn_datapath *od; - - struct ovs_list list; /* In list of similar records. */ - - struct hmap_node dp_node; /* Node in od->ports. */ - - struct lport_addresses proxy_arp_addrs; - - /* Temporarily used for traversing a list (or hmap) of ports. */ - bool visited; - - /* List of struct lflow_ref_node that points to the lflows generated by - * this ovn_port. - * - * This data is initialized and destroyed by the en_northd node, but - * populated and used only by the en_lflow node. Ideally this data should - * be maintained as part of en_lflow's data (struct lflow_data): a hash - * index from ovn_port key to lflows. However, it would be less efficient - * and more complex: - * - * 1. It would require an extra search (using the index) to find the - * lflows. - * - * 2. Building the index needs to be thread-safe, using either a global - * lock which is obviously less efficient, or hash-based lock array which - * is more complex. - * - * Adding the list here is more straightforward. The drawback is that we - * need to keep in mind that this data belongs to en_lflow node, so never - * access it from any other nodes. - */ - struct ovs_list lflows; -}; - static bool lsp_can_be_inc_processed(const struct nbrec_logical_switch_port *); static bool @@ -1450,16 +1328,21 @@ destroy_routable_addresses(struct ovn_port_routable_addresses *ra) } static char **get_nat_addresses(const struct ovn_port *op, size_t *n, - bool routable_only, bool include_lb_ips); + bool routable_only, bool include_lb_ips, + const struct lr_stateful_record *); -static void -assign_routable_addresses(struct ovn_port *op) +static struct ovn_port_routable_addresses +get_op_routable_addresses(struct ovn_port *op, + const struct lr_stateful_record *lr_sful_rec) { size_t n; - char **nats = get_nat_addresses(op, &n, true, true); + char **nats = get_nat_addresses(op, &n, true, true, lr_sful_rec); if (!nats) { - return; + return (struct ovn_port_routable_addresses) { + .laddrs = NULL, + .n_addrs = 0, + }; } struct lport_addresses *laddrs = xcalloc(n, sizeof(*laddrs)); @@ -1475,9 +1358,15 @@ assign_routable_addresses(struct ovn_port *op) } free(nats); - /* Everything seems to have worked out */ - op->routables.laddrs = laddrs; - op->routables.n_addrs = n_addrs; + if (!n_addrs) { + free(laddrs); + laddrs = NULL; + } + + return (struct ovn_port_routable_addresses) { + .laddrs = laddrs, + .n_addrs = n_addrs, + }; } @@ -1537,8 +1426,6 @@ ovn_port_destroy_orphan(struct ovn_port *port) } free(port->ps_addrs); - destroy_routable_addresses(&port->routables); - destroy_lport_addresses(&port->lrp_networks); destroy_lport_addresses(&port->proxy_arp_addrs); free(port->json_key); @@ -2580,9 +2467,7 @@ join_logical_ports(const struct sbrec_port_binding_table *sbrec_pb_table, sizeof *od->l3dgw_ports); } od->l3dgw_ports[od->n_l3dgw_ports++] = op; - - assign_routable_addresses(op); - } + } } } @@ -2679,7 +2564,8 @@ join_logical_ports(const struct sbrec_port_binding_table *sbrec_pb_table, * and must free the returned array when it is no longer needed. */ static char ** get_nat_addresses(const struct ovn_port *op, size_t *n, bool routable_only, - bool include_lb_ips) + bool include_lb_ips, + const struct lr_stateful_record *lr_sful_rec) { size_t n_nats = 0; struct eth_addr mac; @@ -2764,23 +2650,25 @@ get_nat_addresses(const struct ovn_port *op, size_t *n, bool routable_only, } } - if (include_lb_ips) { + if (include_lb_ips && lr_sful_rec) { const char *ip_address; if (routable_only) { - SSET_FOR_EACH (ip_address, &op->od->lb_ips->ips_v4_routable) { + SSET_FOR_EACH (ip_address, + &lr_sful_rec->lb_ips->ips_v4_routable) { ds_put_format(&c_addresses, " %s", ip_address); central_ip_address = true; } - SSET_FOR_EACH (ip_address, &op->od->lb_ips->ips_v6_routable) { + SSET_FOR_EACH (ip_address, + &lr_sful_rec->lb_ips->ips_v6_routable) { ds_put_format(&c_addresses, " %s", ip_address); central_ip_address = true; } } else { - SSET_FOR_EACH (ip_address, &op->od->lb_ips->ips_v4) { + SSET_FOR_EACH (ip_address, &lr_sful_rec->lb_ips->ips_v4) { ds_put_format(&c_addresses, " %s", ip_address); central_ip_address = true; } - SSET_FOR_EACH (ip_address, &op->od->lb_ips->ips_v6) { + SSET_FOR_EACH (ip_address, &lr_sful_rec->lb_ips->ips_v6) { ds_put_format(&c_addresses, " %s", ip_address); central_ip_address = true; } @@ -3851,21 +3739,8 @@ build_lb_datapaths(const struct hmap *lbs, const struct hmap *lb_groups, HMAP_FOR_EACH (od, key_node, &lr_datapaths->datapaths) { ovs_assert(od->nbr); - /* Checking load balancer groups first, starting from the largest one, - * to more efficiently copy IP sets. */ - size_t largest_group = 0; - - for (size_t i = 1; i < od->nbr->n_load_balancer_group; i++) { - if (od->nbr->load_balancer_group[i]->n_load_balancer > - od->nbr->load_balancer_group[largest_group]->n_load_balancer) { - largest_group = i; - } - } - for (size_t i = 0; i < od->nbr->n_load_balancer_group; i++) { - size_t idx = (i + largest_group) % od->nbr->n_load_balancer_group; - - nbrec_lb_group = od->nbr->load_balancer_group[idx]; + nbrec_lb_group = od->nbr->load_balancer_group[i]; const struct uuid *lb_group_uuid = &nbrec_lb_group->header_.uuid; lb_group_dps = @@ -3873,20 +3748,6 @@ build_lb_datapaths(const struct hmap *lbs, const struct hmap *lb_groups, lb_group_uuid); ovs_assert(lb_group_dps); ovn_lb_group_datapaths_add_lr(lb_group_dps, od); - - if (!od->lb_ips) { - od->lb_ips = - ovn_lb_ip_set_clone(lb_group_dps->lb_group->lb_ips); - } else { - for (size_t j = 0; j < lb_group_dps->lb_group->n_lbs; j++) { - build_lrouter_lb_ips(od->lb_ips, - lb_group_dps->lb_group->lbs[j]); - } - } - } - - if (!od->lb_ips) { - od->lb_ips = ovn_lb_ip_set_create(); } for (size_t i = 0; i < od->nbr->n_load_balancer; i++) { @@ -3895,7 +3756,6 @@ build_lb_datapaths(const struct hmap *lbs, const struct hmap *lb_groups, lb_dps = ovn_lb_datapaths_find(lb_datapaths_map, lb_uuid); ovs_assert(lb_dps); ovn_lb_datapaths_add_lr(lb_dps, 1, &od); - build_lrouter_lb_ips(od->lb_ips, lb_dps->lb); } } @@ -3949,102 +3809,6 @@ build_lb_svcs( } } -static bool lrouter_port_ipv4_reachable(const struct ovn_port *op, - ovs_be32 addr); -static bool lrouter_port_ipv6_reachable(const struct ovn_port *op, - const struct in6_addr *addr); - -static void -add_neigh_ips_to_lrouter(struct ovn_datapath *od, - enum lb_neighbor_responder_mode neigh_mode, - const struct sset *lb_ips_v4, - const struct sset *lb_ips_v6) -{ - /* If configured to not reply to any neighbor requests for all VIPs - * return early. - */ - if (neigh_mode == LB_NEIGH_RESPOND_NONE) { - return; - } - - const char *ip_address; - - /* If configured to reply to neighbor requests for all VIPs force them - * all to be considered "reachable". - */ - if (neigh_mode == LB_NEIGH_RESPOND_ALL) { - SSET_FOR_EACH (ip_address, lb_ips_v4) { - sset_add(&od->lb_ips->ips_v4_reachable, ip_address); - } - SSET_FOR_EACH (ip_address, lb_ips_v6) { - sset_add(&od->lb_ips->ips_v6_reachable, ip_address); - } - - return; - } - - /* Otherwise, a VIP is reachable if there's at least one router - * subnet that includes it. - */ - ovs_assert(neigh_mode == LB_NEIGH_RESPOND_REACHABLE); - - SSET_FOR_EACH (ip_address, lb_ips_v4) { - struct ovn_port *op; - ovs_be32 vip_ip4; - if (ip_parse(ip_address, &vip_ip4)) { - HMAP_FOR_EACH (op, dp_node, &od->ports) { - if (lrouter_port_ipv4_reachable(op, vip_ip4)) { - sset_add(&od->lb_ips->ips_v4_reachable, - ip_address); - break; - } - } - } - } - - SSET_FOR_EACH (ip_address, lb_ips_v6) { - struct ovn_port *op; - struct in6_addr vip; - if (ipv6_parse(ip_address, &vip)) { - HMAP_FOR_EACH (op, dp_node, &od->ports) { - if (lrouter_port_ipv6_reachable(op, &vip)) { - sset_add(&od->lb_ips->ips_v6_reachable, - ip_address); - break; - } - } - } - } -} - -static void -remove_lrouter_lb_reachable_ips(struct ovn_datapath *od, - enum lb_neighbor_responder_mode neigh_mode, - const struct sset *lb_ips_v4, - const struct sset *lb_ips_v6) -{ - if (neigh_mode == LB_NEIGH_RESPOND_NONE) { - return; - } - - const char *ip_address; - SSET_FOR_EACH (ip_address, lb_ips_v4) { - sset_find_and_delete(&od->lb_ips->ips_v4_reachable, ip_address); - } - SSET_FOR_EACH (ip_address, lb_ips_v6) { - sset_find_and_delete(&od->lb_ips->ips_v6_reachable, ip_address); - } -} - -static void -build_lrouter_lb_reachable_ips(struct ovn_datapath *od, - const struct ovn_northd_lb *lb) -{ - add_neigh_ips_to_lrouter(od, lb->neigh_mode, &lb->ips_v4, - &lb->ips_v6); -} - - static void build_lrouter_lbs_check(const struct ovn_datapaths *lr_datapaths) { @@ -4066,43 +3830,6 @@ build_lrouter_lbs_check(const struct ovn_datapaths *lr_datapaths) } } -static void -build_lrouter_lbs_reachable_ips(struct ovn_datapaths *lr_datapaths, - struct hmap *lb_dps_map, - struct hmap *lb_group_dps_map) -{ - struct ovn_datapath *od; - - HMAP_FOR_EACH (od, key_node, &lr_datapaths->datapaths) { - if (!od->nbr) { - continue; - } - - for (size_t i = 0; i < od->nbr->n_load_balancer; i++) { - struct ovn_lb_datapaths *lb_dps = - ovn_lb_datapaths_find(lb_dps_map, - &od->nbr->load_balancer[i]->header_.uuid); - ovs_assert(lb_dps); - build_lrouter_lb_reachable_ips(od, lb_dps->lb); - } - - for (size_t i = 0; i < od->nbr->n_load_balancer_group; i++) { - const struct nbrec_load_balancer_group *nbrec_lb_group = - od->nbr->load_balancer_group[i]; - struct ovn_lb_group_datapaths *lb_group_dps; - - lb_group_dps = - ovn_lb_group_datapaths_find(lb_group_dps_map, - &nbrec_lb_group->header_.uuid); - ovs_assert(lb_group_dps); - for (size_t j = 0; j < lb_group_dps->lb_group->n_lbs; j++) { - build_lrouter_lb_reachable_ips(od, - lb_group_dps->lb_group->lbs[j]); - } - } - } -} - static void build_lswitch_lbs_from_lrouter(struct ovn_datapaths *lr_datapaths, struct hmap *lb_dps_map, @@ -4166,8 +3893,6 @@ build_lb_port_related_data( struct hmap *svc_monitor_map) { build_lrouter_lbs_check(lr_datapaths); - build_lrouter_lbs_reachable_ips(lr_datapaths, lb_dps_map, - lb_group_dps_map); build_lb_svcs(ovnsb_txn, sbrec_service_monitor_table, ls_ports, lb_dps_map, svc_monitor_lsps, svc_monitor_map); build_lswitch_lbs_from_lrouter(lr_datapaths, lb_dps_map, lb_group_dps_map); @@ -4533,7 +4258,8 @@ check_sb_lb_duplicates(const struct sbrec_load_balancer_table *table) * Caller should make sure that the OVN SB IDL txn is not NULL. Presently it * only syncs the nat column of port binding corresponding to the 'op->nbsp' */ static void -sync_pb_for_lsp(struct ovn_port *op) +sync_pb_for_lsp(struct ovn_port *op, + const struct lr_stateful_table *lr_sful_table) { ovs_assert(op->nbsp); @@ -4552,10 +4278,17 @@ sync_pb_for_lsp(struct ovn_port *op) if (nat_addresses && !strcmp(nat_addresses, "router")) { if (op->peer && op->peer->od && (chassis || op->peer->od->n_l3dgw_ports)) { - bool exclude_lb_vips = smap_get_bool(&op->nbsp->options, + bool include_lb_vips = !smap_get_bool(&op->nbsp->options, "exclude-lb-vips-from-garp", false); + + const struct lr_stateful_record *lr_sful_rec = NULL; + + if (include_lb_vips) { + lr_sful_rec = lr_stateful_table_find_by_index( + lr_sful_table, op->peer->od->index); + } nats = get_nat_addresses(op->peer, &n_nats, false, - !exclude_lb_vips); + include_lb_vips, lr_sful_rec); } } else if (nat_addresses && (chassis || l3dgw_ports)) { struct lport_addresses laddrs; @@ -4662,7 +4395,8 @@ sync_pb_for_lsp(struct ovn_port *op) * Caller should make sure that the OVN SB IDL txn is not NULL. Presently it * only sets the port binding options column for the router ports */ static void -sync_pb_for_lrp(struct ovn_port *op, const struct lr_nat_table *lr_nats) +sync_pb_for_lrp(struct ovn_port *op, + const struct lr_stateful_table *lr_sful_table) { ovs_assert(op->nbrp); @@ -4671,14 +4405,14 @@ sync_pb_for_lrp(struct ovn_port *op, const struct lr_nat_table *lr_nats) const char *chassis_name = smap_get(&op->od->nbr->options, "chassis"); if (is_cr_port(op)) { - const struct lr_nat_record *lrnat_rec = - lr_nat_table_find_by_index(lr_nats, op->od->index); - ovs_assert(lrnat_rec); + const struct lr_stateful_record *lr_sful_rec = + lr_stateful_table_find_by_index(lr_sful_table, op->od->index); + ovs_assert(lr_sful_rec); smap_add(&new, "distributed-port", op->nbrp->name); bool always_redirect = - !lrnat_rec->has_distributed_nat && + !lr_sful_rec->lrnat_rec->has_distributed_nat && !l3dgw_port_has_associated_vtep_lports(op->l3dgw_port); const char *redirect_type = smap_get(&op->nbrp->options, @@ -4728,17 +4462,18 @@ static void ovn_update_ipv6_opt_for_op(struct ovn_port *op); * the logical switch ports. */ void sync_pbs(struct ovsdb_idl_txn *ovnsb_idl_txn, struct hmap *ls_ports, - struct hmap *lr_ports, const struct lr_nat_table *lr_nats) + struct hmap *lr_ports, + const struct lr_stateful_table *lr_sful_table) { ovs_assert(ovnsb_idl_txn); struct ovn_port *op; HMAP_FOR_EACH (op, key_node, ls_ports) { - sync_pb_for_lsp(op); + sync_pb_for_lsp(op, lr_sful_table); } HMAP_FOR_EACH (op, key_node, lr_ports) { - sync_pb_for_lrp(op, lr_nats); + sync_pb_for_lrp(op, lr_sful_table); } ovn_update_ipv6_options(lr_ports); @@ -4747,20 +4482,22 @@ sync_pbs(struct ovsdb_idl_txn *ovnsb_idl_txn, struct hmap *ls_ports, /* Sync the SB Port bindings for the added and updated logical switch ports * of the tracked northd engine data. */ bool -sync_pbs_for_northd_changed_ovn_ports(struct tracked_ovn_ports *trk_ovn_ports) +sync_pbs_for_northd_changed_ovn_ports( + struct tracked_ovn_ports *trk_ovn_ports, + const struct lr_stateful_table *lr_sful_table) { struct hmapx_node *hmapx_node; struct ovn_port *op; HMAPX_FOR_EACH (hmapx_node, &trk_ovn_ports->created) { op = hmapx_node->data; ovs_assert(op->nbsp); - sync_pb_for_lsp(op); + sync_pb_for_lsp(op, lr_sful_table); } HMAPX_FOR_EACH (hmapx_node, &trk_ovn_ports->updated) { op = hmapx_node->data; ovs_assert(op->nbsp); - sync_pb_for_lsp(op); + sync_pb_for_lsp(op, lr_sful_table); } return true; @@ -5425,14 +5162,13 @@ fail: } /* Returns true if the logical router has changes which can be - * incrementally handled. + * incrementally handled or the changes can be ignored. * Presently supports i-p for the below changes: * - load balancers and load balancer groups. * - NAT changes */ static bool -lr_changes_can_be_handled( - const struct nbrec_logical_router *lr) +lr_changes_can_be_handled(const struct nbrec_logical_router *lr) { /* Check if the columns are changed in this row. */ enum nbrec_logical_router_column_id col; @@ -5496,17 +5232,6 @@ is_lr_nats_changed(const struct nbrec_logical_router *nbr) { || is_lr_nats_seqno_changed(nbr)); } -static bool -lr_has_routable_nats(const struct nbrec_logical_router *nbr) { - for (size_t i = 0; i < nbr->n_nat; i++) { - if (smap_get_bool(&nbr->nat[i]->options, "add_route", false)) { - return true; - } - } - - return false; -} - /* Return true if changes are handled incrementally, false otherwise. * * Note: Changes to load balancer and load balancer groups associated with @@ -5533,12 +5258,6 @@ northd_handle_lr_changes(const struct northd_input *ni, } if (is_lr_nats_changed(changed_lr)) { - if (lr_has_routable_nats(changed_lr)) { - /* router has routable NATs. We can't handle these changes - * incrementally yet. Fall back to recompute. */ - goto fail; - } - struct ovn_datapath *od = ovn_datapath_find_( &nd->lr_datapaths.datapaths, &changed_lr->header_.uuid); @@ -5820,10 +5539,6 @@ northd_handle_lb_data_changes(struct tracked_lb_data *trk_lb_data, ovs_assert(lb_dps); ovn_lb_datapaths_add_lr(lb_dps, 1, &od); - /* Add the lb_ips of lb_dps to the od. */ - build_lrouter_lb_ips(od->lb_ips, lb_dps->lb); - build_lrouter_lb_reachable_ips(od, lb_dps->lb); - /* Add the lb to the northd tracked data. */ hmapx_add(&nd_changes->trk_lbs.crupdated, lb_dps); } @@ -5842,10 +5557,6 @@ northd_handle_lb_data_changes(struct tracked_lb_data *trk_lb_data, ovs_assert(lb_dps); ovn_lb_datapaths_add_lr(lb_dps, 1, &od); - /* Add the lb_ips of lb_dps to the od. */ - build_lrouter_lb_ips(od->lb_ips, lb_dps->lb); - build_lrouter_lb_reachable_ips(od, lb_dps->lb); - /* Add the lb to the northd tracked data. */ hmapx_add(&nd_changes->trk_lbs.crupdated, lb_dps); } @@ -5874,22 +5585,6 @@ northd_handle_lb_data_changes(struct tracked_lb_data *trk_lb_data, od = lr_datapaths->array[index]; /* Re-evaluate 'od->has_lb_vip' */ init_lb_for_datapath(od); - - /* Update the od->lb_ips with the deleted and inserted - * vips (if any). */ - remove_ips_from_lb_ip_set(od->lb_ips, lb->routable, - &clb->deleted_vips_v4, - &clb->deleted_vips_v6); - add_ips_to_lb_ip_set(od->lb_ips, lb->routable, - &clb->inserted_vips_v4, - &clb->inserted_vips_v6); - - remove_lrouter_lb_reachable_ips(od, lb->neigh_mode, - &clb->deleted_vips_v4, - &clb->deleted_vips_v6); - add_neigh_ips_to_lrouter(od, lb->neigh_mode, - &clb->inserted_vips_v4, - &clb->inserted_vips_v6); } } @@ -5914,9 +5609,6 @@ northd_handle_lb_data_changes(struct tracked_lb_data *trk_lb_data, /* Re-evaluate 'od->has_lb_vip' */ init_lb_for_datapath(od); - - /* Add the lb_ips of lb_dps to the od. */ - build_lrouter_lb_ips(od->lb_ips, lb_dps->lb); } for (size_t i = 0; i < lbgrp_dps->n_ls; i++) { @@ -9210,7 +8902,7 @@ arp_nd_ns_match(const char *ips, int addr_family, struct ds *match) /* Returns 'true' if the IPv4 'addr' is on the same subnet with one of the * IPs configured on the router port. */ -static bool +bool lrouter_port_ipv4_reachable(const struct ovn_port *op, ovs_be32 addr) { for (size_t i = 0; i < op->lrp_networks.n_ipv4_addrs; i++) { @@ -9226,7 +8918,7 @@ lrouter_port_ipv4_reachable(const struct ovn_port *op, ovs_be32 addr) /* Returns 'true' if the IPv6 'addr' is on the same subnet with one of the * IPs configured on the router port. */ -static bool +bool lrouter_port_ipv6_reachable(const struct ovn_port *op, const struct in6_addr *addr) { @@ -9289,11 +8981,12 @@ build_lswitch_rport_arp_req_flow(const char *ips, */ static void build_lswitch_rport_arp_req_flows(struct ovn_port *op, - struct ovn_datapath *sw_od, - struct ovn_port *sw_op, - const struct lr_nat_table *lr_nats, - struct hmap *lflows, - const struct ovsdb_idl_row *stage_hint) + struct ovn_datapath *sw_od, + struct ovn_port *sw_op, + const struct lr_nat_table *lr_nats, + const struct lr_stateful_table *lr_sful_table, + struct hmap *lflows, + const struct ovsdb_idl_row *stage_hint) { if (!op || !op->nbrp) { return; @@ -9307,32 +9000,38 @@ build_lswitch_rport_arp_req_flows(struct ovn_port *op, * router port. * Priority: 80. */ - - const char *ip_addr; - SSET_FOR_EACH (ip_addr, &op->od->lb_ips->ips_v4_reachable) { - ovs_be32 ipv4_addr; - - /* Check if the ovn port has a network configured on which we could - * expect ARP requests for the LB VIP. - */ - if (ip_parse(ip_addr, &ipv4_addr) && - lrouter_port_ipv4_reachable(op, ipv4_addr)) { - build_lswitch_rport_arp_req_flow( - ip_addr, AF_INET, sw_op, sw_od, 80, lflows, - stage_hint); + const struct lr_stateful_record *lr_sful_rec = NULL; + if (op->od->nbr->n_load_balancer || op->od->nbr->n_load_balancer_group) { + lr_sful_rec = lr_stateful_table_find_by_index(lr_sful_table, + op->od->index); + ovs_assert(lr_sful_rec); + + const char *ip_addr; + SSET_FOR_EACH (ip_addr, &lr_sful_rec->lb_ips->ips_v4_reachable) { + ovs_be32 ipv4_addr; + + /* Check if the ovn port has a network configured on which we could + * expect ARP requests for the LB VIP. + */ + if (ip_parse(ip_addr, &ipv4_addr) && + lrouter_port_ipv4_reachable(op, ipv4_addr)) { + build_lswitch_rport_arp_req_flow( + ip_addr, AF_INET, sw_op, sw_od, 80, lflows, + stage_hint); + } } - } - SSET_FOR_EACH (ip_addr, &op->od->lb_ips->ips_v6_reachable) { - struct in6_addr ipv6_addr; + SSET_FOR_EACH (ip_addr, &lr_sful_rec->lb_ips->ips_v6_reachable) { + struct in6_addr ipv6_addr; - /* Check if the ovn port has a network configured on which we could - * expect NS requests for the LB VIP. - */ - if (ipv6_parse(ip_addr, &ipv6_addr) && - lrouter_port_ipv6_reachable(op, &ipv6_addr)) { - build_lswitch_rport_arp_req_flow( - ip_addr, AF_INET6, sw_op, sw_od, 80, lflows, - stage_hint); + /* Check if the ovn port has a network configured on which we could + * expect NS requests for the LB VIP. + */ + if (ipv6_parse(ip_addr, &ipv6_addr) && + lrouter_port_ipv6_reachable(op, &ipv6_addr)) { + build_lswitch_rport_arp_req_flow( + ip_addr, AF_INET6, sw_op, sw_od, 80, lflows, + stage_hint); + } } } @@ -9382,13 +9081,15 @@ build_lswitch_rport_arp_req_flows(struct ovn_port *op, * expect ARP requests/NS for the DNAT external_ip. */ if (nat_entry_is_v6(nat_entry)) { - if (!sset_contains(&op->od->lb_ips->ips_v6, nat->external_ip)) { + if (!lr_sful_rec || !sset_contains(&lr_sful_rec->lb_ips->ips_v6, + nat->external_ip)) { build_lswitch_rport_arp_req_flow( nat->external_ip, AF_INET6, sw_op, sw_od, 80, lflows, stage_hint); } } else { - if (!sset_contains(&op->od->lb_ips->ips_v4, nat->external_ip)) { + if (!lr_sful_rec || !sset_contains(&lr_sful_rec->lb_ips->ips_v4, + nat->external_ip)) { build_lswitch_rport_arp_req_flow( nat->external_ip, AF_INET, sw_op, sw_od, 80, lflows, stage_hint); @@ -10429,6 +10130,7 @@ build_lswitch_ip_mcast_igmp_mld(struct ovn_igmp_group *igmp_group, static void build_lswitch_ip_unicast_lookup(struct ovn_port *op, const struct lr_nat_table *lr_nats, + const struct lr_stateful_table *lr_sful_table, struct hmap *lflows, struct ds *actions, struct ds *match) @@ -10444,7 +10146,8 @@ build_lswitch_ip_unicast_lookup(struct ovn_port *op, */ if (lsp_is_router(op->nbsp)) { build_lswitch_rport_arp_req_flows(op->peer, op->od, op, lr_nats, - lflows, &op->nbsp->header_); + lr_sful_table, lflows, + &op->nbsp->header_); } for (size_t i = 0; i < op->nbsp->n_addresses; i++) { @@ -12634,6 +12337,7 @@ build_lrouter_port_nat_arp_nd_flow(struct ovn_port *op, static void build_lrouter_drop_own_dest(struct ovn_port *op, const struct lr_nat_record *lrnat_rec, + const struct lr_stateful_record *lr_sful_rec, enum ovn_stage stage, uint16_t priority, bool drop_snat_ip, struct hmap *lflows) @@ -12646,8 +12350,9 @@ build_lrouter_drop_own_dest(struct ovn_port *op, bool router_ip_in_snat_ips = !!shash_find(&lrnat_rec->snat_ips, ip); - bool router_ip_in_lb_ips = - !!sset_find(&op->od->lb_ips->ips_v4, ip); + bool router_ip_in_lb_ips = (lr_sful_rec && + !!sset_find(&lr_sful_rec->lb_ips->ips_v4, + ip)); bool drop_router_ip = (drop_snat_ip == (router_ip_in_snat_ips || router_ip_in_lb_ips)); @@ -12676,8 +12381,9 @@ build_lrouter_drop_own_dest(struct ovn_port *op, bool router_ip_in_snat_ips = !!shash_find(&lrnat_rec->snat_ips, ip); - bool router_ip_in_lb_ips = - !!sset_find(&op->od->lb_ips->ips_v6, ip); + bool router_ip_in_lb_ips = (lr_sful_rec && + !!sset_find(&lr_sful_rec->lb_ips->ips_v6, + ip)); bool drop_router_ip = (drop_snat_ip == (router_ip_in_snat_ips || router_ip_in_lb_ips)); @@ -13389,7 +13095,8 @@ build_ip_routing_flows_for_lrp( */ static void build_ip_routing_flows_for_router_type_lsp( - struct ovn_port *op, const struct hmap *lr_ports, struct hmap *lflows) + struct ovn_port *op, const struct lr_stateful_table *lr_sful_table, + const struct hmap *lr_ports, struct hmap *lflows) { ovs_assert(op->nbsp); if (!lsp_is_router(op->nbsp)) { @@ -13397,7 +13104,8 @@ build_ip_routing_flows_for_router_type_lsp( } struct ovn_port *peer = ovn_port_get_peer(lr_ports, op); - if (!peer || !peer->nbrp || !peer->lrp_networks.n_ipv4_addrs) { + if (!peer || !peer->nbrp || !peer->lrp_networks.n_ipv4_addrs + || !op->od->n_router_ports) { return; } @@ -13408,19 +13116,29 @@ build_ip_routing_flows_for_router_type_lsp( continue; } - struct ovn_port_routable_addresses *ra = &router_port->routables; - for (size_t j = 0; j < ra->n_addrs; j++) { - struct lport_addresses *laddrs = &ra->laddrs[j]; - for (size_t k = 0; k < laddrs->n_ipv4_addrs; k++) { - add_route(lflows, peer->od, peer, - peer->lrp_networks.ipv4_addrs[0].addr_s, - laddrs->ipv4_addrs[k].network_s, - laddrs->ipv4_addrs[k].plen, NULL, false, 0, - &peer->nbrp->header_, false, - ROUTE_PRIO_OFFSET_CONNECTED); + const struct lr_stateful_record *lr_sful_rec = + lr_stateful_table_find_by_index(lr_sful_table, + router_port->od->index); + + if (router_port->nbrp->ha_chassis_group || + router_port->nbrp->n_gateway_chassis) { + struct ovn_port_routable_addresses ra = + get_op_routable_addresses(router_port, lr_sful_rec); + for (size_t j = 0; j < ra.n_addrs; j++) { + struct lport_addresses *laddrs = &ra.laddrs[j]; + for (size_t k = 0; k < laddrs->n_ipv4_addrs; k++) { + add_route(lflows, peer->od, peer, + peer->lrp_networks.ipv4_addrs[0].addr_s, + laddrs->ipv4_addrs[k].network_s, + laddrs->ipv4_addrs[k].plen, NULL, false, 0, + &peer->nbrp->header_, false, + ROUTE_PRIO_OFFSET_CONNECTED); + } } + destroy_routable_addresses(&ra); } } + } static void @@ -13644,33 +13362,36 @@ build_arp_resolve_flows_for_lrouter( static void routable_addresses_to_lflows(struct hmap *lflows, struct ovn_port *router_port, - struct ovn_port *peer, struct ds *match, - struct ds *actions) + struct ovn_port *peer, + const struct lr_stateful_record *lr_sful_rec, + struct ds *match, struct ds *actions) { - struct ovn_port_routable_addresses *ra = &router_port->routables; - if (!ra->n_addrs) { + struct ovn_port_routable_addresses ra = + get_op_routable_addresses(router_port, lr_sful_rec); + if (!ra.n_addrs) { return; } - for (size_t i = 0; i < ra->n_addrs; i++) { + for (size_t i = 0; i < ra.n_addrs; i++) { ds_clear(match); ds_put_format(match, "outport == %s && "REG_NEXT_HOP_IPV4" == {", peer->json_key); bool first = true; - for (size_t j = 0; j < ra->laddrs[i].n_ipv4_addrs; j++) { + for (size_t j = 0; j < ra.laddrs[i].n_ipv4_addrs; j++) { if (!first) { ds_put_cstr(match, ", "); } - ds_put_cstr(match, ra->laddrs[i].ipv4_addrs[j].addr_s); + ds_put_cstr(match, ra.laddrs[i].ipv4_addrs[j].addr_s); first = false; } ds_put_cstr(match, "}"); ds_clear(actions); - ds_put_format(actions, "eth.dst = %s; next;", ra->laddrs[i].ea_s); + ds_put_format(actions, "eth.dst = %s; next;", ra.laddrs[i].ea_s); ovn_lflow_add(lflows, peer->od, S_ROUTER_IN_ARP_RESOLVE, 100, ds_cstr(match), ds_cstr(actions)); } + destroy_routable_addresses(&ra); } /* Local router ingress table ARP_RESOLVE: ARP Resolution. @@ -13687,6 +13408,7 @@ routable_addresses_to_lflows(struct hmap *lflows, struct ovn_port *router_port, static void build_arp_resolve_flows_for_lrp( struct ovn_port *op, const struct lr_nat_record *lrnat_rec, + const struct lr_stateful_record *lr_sful_rec, struct hmap *lflows, struct ds *match, struct ds *actions) { ovs_assert(op->nbrp); @@ -13763,8 +13485,8 @@ build_arp_resolve_flows_for_lrp( * * Priority 2. */ - build_lrouter_drop_own_dest(op, lrnat_rec, S_ROUTER_IN_ARP_RESOLVE, 2, - true, lflows); + build_lrouter_drop_own_dest(op, lrnat_rec, lr_sful_rec, + S_ROUTER_IN_ARP_RESOLVE, 2, true, lflows); } /* This function adds ARP resolve flows related to a LSP. */ @@ -13772,6 +13494,7 @@ static void build_arp_resolve_flows_for_lsp( struct ovn_port *op, struct hmap *lflows, const struct hmap *lr_ports, + const struct lr_stateful_table *lr_sful_table, struct ds *match, struct ds *actions) { ovs_assert(op->nbsp); @@ -13915,8 +13638,11 @@ build_arp_resolve_flows_for_lsp( if (smap_get(&peer->od->nbr->options, "chassis") || peer->cr_port) { + const struct lr_stateful_record *lr_sful_rec; + lr_sful_rec = lr_stateful_table_find_by_index(lr_sful_table, + router_port->od->index); routable_addresses_to_lflows(lflows, router_port, peer, - match, actions); + lr_sful_rec, match, actions); } } } @@ -14636,6 +14362,7 @@ static void build_lrouter_ipv4_ip_input(struct ovn_port *op, struct hmap *lflows, const struct lr_nat_record *lrnat_rec, + const struct lr_stateful_record *lr_sful_rec, struct ds *match, struct ds *actions, const struct shash *meter_groups) { @@ -14760,7 +14487,7 @@ build_lrouter_ipv4_ip_input(struct ovn_port *op, &op->nbrp->header_, lflows); } - if (sset_count(&op->od->lb_ips->ips_v4_reachable)) { + if (lr_sful_rec && sset_count(&lr_sful_rec->lb_ips->ips_v4_reachable)) { ds_clear(match); if (is_l3dgw_port(op)) { ds_put_format(match, "is_chassis_resident(%s)", @@ -14776,7 +14503,7 @@ build_lrouter_ipv4_ip_input(struct ovn_port *op, free(lb_ips_v4_as); } - if (sset_count(&op->od->lb_ips->ips_v6_reachable)) { + if (lr_sful_rec && sset_count(&lr_sful_rec->lb_ips->ips_v6_reachable)) { ds_clear(match); if (is_l3dgw_port(op)) { @@ -14878,8 +14605,8 @@ build_lrouter_ipv4_ip_input(struct ovn_port *op, * Priority 60. */ if (!lrnat_rec->lb_force_snat_router_ip) { - build_lrouter_drop_own_dest(op, lrnat_rec, S_ROUTER_IN_IP_INPUT, 60, - false, lflows); + build_lrouter_drop_own_dest(op, lrnat_rec, lr_sful_rec, + S_ROUTER_IN_IP_INPUT, 60, false, lflows); } /* ARP / ND handling for external IP addresses. * @@ -16018,6 +15745,7 @@ struct lswitch_flow_build_info { const struct hmap *lr_ports; const struct ls_port_group_table *ls_port_groups; const struct lr_nat_table *lr_nats; + const struct lr_stateful_table *lr_sful_table; struct hmap *lflows; struct hmap *igmp_groups; const struct shash *meter_groups; @@ -16101,14 +15829,15 @@ build_lswitch_and_lrouter_iterate_by_lr(struct ovn_datapath *od, * switch port. */ static void -build_lswitch_and_lrouter_iterate_by_lsp(struct ovn_port *op, - const struct hmap *ls_ports, - const struct hmap *lr_ports, - const struct lr_nat_table *lr_nats, - const struct shash *meter_groups, - struct ds *match, - struct ds *actions, - struct hmap *lflows) +build_lswitch_and_lrouter_iterate_by_lsp( + struct ovn_port *op, const struct hmap *ls_ports, + const struct hmap *lr_ports, + const struct lr_nat_table *lr_nats, + const struct lr_stateful_table *lr_sful_table, + const struct shash *meter_groups, + struct ds *match, + struct ds *actions, + struct hmap *lflows) { ovs_assert(op->nbsp); start_collecting_lflows(); @@ -16121,11 +15850,14 @@ build_lswitch_and_lrouter_iterate_by_lsp(struct ovn_port *op, meter_groups, actions, match); build_lswitch_dhcp_options_and_response(op, lflows, meter_groups); build_lswitch_external_port(op, lflows); - build_lswitch_ip_unicast_lookup(op, lr_nats, lflows, actions, match); + build_lswitch_ip_unicast_lookup(op, lr_nats, lr_sful_table, lflows, + actions, match); /* Build Logical Router Flows. */ - build_ip_routing_flows_for_router_type_lsp(op, lr_ports, lflows); - build_arp_resolve_flows_for_lsp(op, lflows, lr_ports, match, actions); + build_ip_routing_flows_for_router_type_lsp(op, lr_sful_table, lr_ports, + lflows); + build_arp_resolve_flows_for_lsp(op, lflows, lr_ports, lr_sful_table, + match, actions); link_ovn_port_to_lflows(op, &collected_lflows); end_collecting_lflows(); @@ -16144,6 +15876,8 @@ build_lswitch_and_lrouter_iterate_by_lrp(struct ovn_port *op, lsi->lr_nats, op->od->index); ovs_assert(lrnet_rec); + const struct lr_stateful_record *lr_sful_rec = + lr_stateful_table_find_by_index(lsi->lr_sful_table, op->od->index); build_adm_ctrl_flows_for_lrouter_port(op, lsi->lflows, &lsi->match, &lsi->actions); build_neigh_learning_flows_for_lrouter_port(op, lsi->lflows, &lsi->match, @@ -16151,15 +15885,15 @@ build_lswitch_and_lrouter_iterate_by_lrp(struct ovn_port *op, build_ip_routing_flows_for_lrp(op, lsi->lflows); build_ND_RA_flows_for_lrouter_port(op, lsi->lflows, &lsi->match, &lsi->actions, lsi->meter_groups); - build_arp_resolve_flows_for_lrp(op, lrnet_rec, lsi->lflows, &lsi->match, - &lsi->actions); + build_arp_resolve_flows_for_lrp(op, lrnet_rec, lr_sful_rec, lsi->lflows, + &lsi->match, &lsi->actions); build_egress_delivery_flows_for_lrouter_port(op, lsi->lflows, &lsi->match, &lsi->actions); build_dhcpv6_reply_flows_for_lrouter_port(op, lsi->lflows, &lsi->match); build_ipv6_input_flows_for_lrouter_port(op, lsi->lflows, &lsi->match, &lsi->actions, lsi->meter_groups); - build_lrouter_ipv4_ip_input(op, lsi->lflows, lrnet_rec, + build_lrouter_ipv4_ip_input(op, lsi->lflows, lrnet_rec, lr_sful_rec, &lsi->match, &lsi->actions, lsi->meter_groups); build_lrouter_force_snat_flows_op(op, lrnet_rec, lsi->lflows, &lsi->match, &lsi->actions); @@ -16220,12 +15954,13 @@ build_lflows_thread(void *arg) return NULL; } build_lswitch_and_lrouter_iterate_by_lsp(op, lsi->ls_ports, - lsi->lr_ports, - lsi->lr_nats, - lsi->meter_groups, - &lsi->match, - &lsi->actions, - lsi->lflows); + lsi->lr_ports, + lsi->lr_nats, + lsi->lr_sful_table, + lsi->meter_groups, + &lsi->match, + &lsi->actions, + lsi->lflows); } } for (bnum = control->id; @@ -16332,6 +16067,7 @@ build_lswitch_and_lrouter_flows(const struct ovn_datapaths *ls_datapaths, const struct hmap *lr_ports, const struct ls_port_group_table *ls_pgs, const struct lr_nat_table *lr_nats, + const struct lr_stateful_table *lr_sful_table, struct hmap *lflows, struct hmap *igmp_groups, const struct shash *meter_groups, @@ -16362,6 +16098,7 @@ build_lswitch_and_lrouter_flows(const struct ovn_datapaths *ls_datapaths, lsiv[index].lr_ports = lr_ports; lsiv[index].ls_port_groups = ls_pgs; lsiv[index].lr_nats = lr_nats; + lsiv[index].lr_sful_table = lr_sful_table; lsiv[index].igmp_groups = igmp_groups; lsiv[index].meter_groups = meter_groups; lsiv[index].lb_dps_map = lb_dps_map; @@ -16397,6 +16134,7 @@ build_lswitch_and_lrouter_flows(const struct ovn_datapaths *ls_datapaths, .lr_ports = lr_ports, .ls_port_groups = ls_pgs, .lr_nats = lr_nats, + .lr_sful_table = lr_sful_table, .lflows = lflows, .igmp_groups = igmp_groups, .meter_groups = meter_groups, @@ -16425,6 +16163,7 @@ build_lswitch_and_lrouter_flows(const struct ovn_datapaths *ls_datapaths, build_lswitch_and_lrouter_iterate_by_lsp(op, lsi.ls_ports, lsi.lr_ports, lsi.lr_nats, + lsi.lr_sful_table, lsi.meter_groups, &lsi.match, &lsi.actions, lsi.lflows); @@ -16546,6 +16285,7 @@ void build_lflows(struct ovsdb_idl_txn *ovnsb_txn, input_data->lr_ports, input_data->ls_port_groups, input_data->lr_nats, + input_data->lr_sful_table, lflows, &igmp_groups, input_data->meter_groups, @@ -17024,6 +16764,7 @@ lflow_handle_northd_port_changes(struct ovsdb_idl_txn *ovnsb_txn, build_lswitch_and_lrouter_iterate_by_lsp(op, lflow_input->ls_ports, lflow_input->lr_ports, lflow_input->lr_nats, + lflow_input->lr_sful_table, lflow_input->meter_groups, &match, &actions, lflows); @@ -17059,11 +16800,11 @@ lflow_handle_northd_port_changes(struct ovsdb_idl_txn *ovnsb_txn, struct ds match = DS_EMPTY_INITIALIZER; struct ds actions = DS_EMPTY_INITIALIZER; build_lswitch_and_lrouter_iterate_by_lsp(op, lflow_input->ls_ports, - lflow_input->lr_ports, - lflow_input->lr_nats, - lflow_input->meter_groups, - &match, &actions, - lflows); + lflow_input->lr_ports, + lflow_input->lr_nats, + lflow_input->lr_sful_table, + lflow_input->meter_groups, + &match, &actions, lflows); ds_destroy(&match); ds_destroy(&actions); diff --git a/northd/northd.h b/northd/northd.h index 4dd260761e..6d64fff626 100644 --- a/northd/northd.h +++ b/northd/northd.h @@ -180,6 +180,7 @@ struct lflow_input { const struct hmap *lr_ports; const struct ls_port_group_table *ls_port_groups; const struct lr_nat_table *lr_nats; + const struct lr_stateful_table *lr_sful_table; const struct shash *meter_groups; const struct hmap *lb_datapaths_map; const struct hmap *bfd_connections; @@ -319,9 +320,6 @@ struct ovn_datapath { /* router datapath has a logical port with redirect-type set to bridged. */ bool redirect_bridged; - /* Load Balancer vIPs relevant for this datapath. */ - struct ovn_lb_ip_set *lb_ips; - struct ovn_port **localnet_ports; size_t n_localnet_ports; @@ -338,6 +336,119 @@ struct ovn_datapath { const struct ovn_datapath *ovn_datapath_find(const struct hmap *datapaths, const struct uuid *uuid); +/* A logical switch port or logical router port. + * + * In steady state, an ovn_port points to a northbound Logical_Switch_Port + * record (via 'nbsp') *or* a Logical_Router_Port record (via 'nbrp'), and to a + * southbound Port_Binding record (via 'sb'). As the state of the system + * changes, join_logical_ports() may determine that there is a new LSP or LRP + * that has no corresponding Port_Binding record (in which case build_ports()) + * will create the missing Port_Binding) or that a Port_Binding record exists + * that has no coresponding LSP (in which case build_ports() will delete the + * spurious Port_Binding). Thus, after build_ports() runs, any given ovn_port + * will have 'sb' nonnull, and 'nbsp' xor 'nbrp' nonnull. + * + * Ordinarily there is only one ovn_port that points to a given LSP or LRP (but + * distributed gateway ports point a "derived" ovn_port to a duplicate LRP). + */ +struct ovn_port { + /* Port name aka key. + * + * This is ordinarily the same as nbsp->name or nbrp->name and + * sb->logical_port. (A distributed gateway port creates a "derived" + * ovn_port with key "cr-%s" % nbrp->name.) */ + struct hmap_node key_node; /* Index on 'key'. */ + char *key; /* nbsp->name, nbrp->name, sb->logical_port. */ + char *json_key; /* 'key', quoted for use in JSON. */ + + const struct sbrec_port_binding *sb; /* May be NULL. */ + + uint32_t tunnel_key; + + /* Logical switch port data. */ + const struct nbrec_logical_switch_port *nbsp; /* May be NULL. */ + + struct lport_addresses *lsp_addrs; /* Logical switch port addresses. */ + unsigned int n_lsp_addrs; /* Total length of lsp_addrs. */ + unsigned int n_lsp_non_router_addrs; /* Number of elements from the + * beginning of 'lsp_addrs' extracted + * directly from LSP 'addresses'. */ + + struct lport_addresses *ps_addrs; /* Port security addresses. */ + unsigned int n_ps_addrs; + + bool lsp_can_be_inc_processed; /* If it can be incrementally processed when + the port changes. */ + + /* Logical router port data. */ + const struct nbrec_logical_router_port *nbrp; /* May be NULL. */ + + struct lport_addresses lrp_networks; + + /* Logical port multicast data. */ + struct mcast_port_info mcast_info; + + /* At most one of l3dgw_port and cr_port can be not NULL. */ + + /* This is set to a distributed gateway port if and only if this ovn_port + * is "derived" from it. Otherwise this is set to NULL. The derived + * ovn_port represents the instance of distributed gateway port on the + * gateway chassis.*/ + struct ovn_port *l3dgw_port; + + /* This is set to the "derived" chassis-redirect port of this port if and + * only if this port is a distributed gateway port. Otherwise this is set + * to NULL. */ + struct ovn_port *cr_port; + + bool has_unknown; /* If the addresses have 'unknown' defined. */ + + bool has_bfd; + + /* The port's peer: + * + * - A switch port S of type "router" has a router port R as a peer, + * and R in turn has S has its peer. + * + * - Two connected logical router ports have each other as peer. + * + * - Other kinds of ports have no peer. */ + struct ovn_port *peer; + + struct ovn_datapath *od; + + struct ovs_list list; /* In list of similar records. */ + + struct hmap_node dp_node; /* Node in od->ports. */ + + struct lport_addresses proxy_arp_addrs; + + /* Temporarily used for traversing a list (or hmap) of ports. */ + bool visited; + + /* List of struct lflow_ref_node that points to the lflows generated by + * this ovn_port. + * + * This data is initialized and destroyed by the en_northd node, but + * populated and used only by the en_lflow node. Ideally this data should + * be maintained as part of en_lflow's data (struct lflow_data): a hash + * index from ovn_port key to lflows. However, it would be less efficient + * and more complex: + * + * 1. It would require an extra search (using the index) to find the + * lflows. + * + * 2. Building the index needs to be thread-safe, using either a global + * lock which is obviously less efficient, or hash-based lock array which + * is more complex. + * + * Adding the list here is more straightforward. The drawback is that we + * need to keep in mind that this data belongs to en_lflow node, so never + * access it from any other nodes. + */ + struct ovs_list lflows; +}; + void ovnnb_db_run(struct northd_input *input_data, struct northd_data *data, struct ovsdb_idl_txn *ovnnb_txn, @@ -397,9 +508,13 @@ void sync_lbs(struct ovsdb_idl_txn *, const struct sbrec_load_balancer_table *, struct chassis_features *chassis_features); bool check_sb_lb_duplicates(const struct sbrec_load_balancer_table *); +struct lr_stateful_table; void sync_pbs(struct ovsdb_idl_txn *, struct hmap *ls_ports, - struct hmap *lr_ports, const struct lr_nat_table *); -bool sync_pbs_for_northd_changed_ovn_ports(struct tracked_ovn_ports *); + struct hmap *lr_ports, + const struct lr_stateful_table *); +bool sync_pbs_for_northd_changed_ovn_ports( + struct tracked_ovn_ports *, + const struct lr_stateful_table *); static inline bool northd_has_tracked_data(struct northd_tracked_data *trk_nd_changes) { @@ -424,4 +539,15 @@ northd_has_lr_nats_in_tracked_data(struct northd_tracked_data *trk_nd_changes) return (trk_nd_changes->type & NORTHD_TRACKED_LR_NATS); } +/* Returns 'true' if the IPv4 'addr' is on the same subnet with one of the + * IPs configured on the router port. + */ +bool lrouter_port_ipv4_reachable(const struct ovn_port *, ovs_be32 addr); + +/* Returns 'true' if the IPv6 'addr' is on the same subnet with one of the + * IPs configured on the router port. + */ +bool lrouter_port_ipv6_reachable(const struct ovn_port *, + const struct in6_addr *); + #endif /* NORTHD_H */ diff --git a/northd/ovn-northd.c b/northd/ovn-northd.c index 40f9764b3a..e85e00cc9b 100644 --- a/northd/ovn-northd.c +++ b/northd/ovn-northd.c @@ -871,6 +871,7 @@ main(int argc, char *argv[]) stopwatch_create(PORT_GROUP_RUN_STOPWATCH_NAME, SW_MS); stopwatch_create(SYNC_METERS_RUN_STOPWATCH_NAME, SW_MS); stopwatch_create(LR_NAT_RUN_STOPWATCH_NAME, SW_MS); + stopwatch_create(LR_STATEFUL_RUN_STOPWATCH_NAME, SW_MS); /* Initialize incremental processing engine for ovn-northd */ inc_proc_northd_init(&ovnnb_idl_loop, &ovnsb_idl_loop); diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at index dc01e43de0..03d62695db 100644 --- a/tests/ovn-northd.at +++ b/tests/ovn-northd.at @@ -10516,18 +10516,21 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb set load_balancer . ip_port_mappings:10.0.0.3=sw0-p1:10.0.0.2 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_stateful norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute nocompute check ovn-nbctl --wait=sb set load_balancer . options:foo=bar check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_stateful norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute nocompute check ovn-nbctl --wait=sb -- lb-add lb2 20.0.0.10:80 20.0.0.20:80 -- lb-add lb3 30.0.0.10:80 30.0.0.20:80 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_stateful norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute nocompute @@ -10537,6 +10540,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb -- lb-del lb2 -- lb-del lb3 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_stateful norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute nocompute @@ -10550,6 +10554,7 @@ AT_CHECK([ovn-nbctl --wait=sb \ ]) check_engine_stats lb_data norecompute compute check_engine_stats northd recompute nocompute +check_engine_stats lr_stateful recompute nocompute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute nocompute @@ -10567,6 +10572,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb clear Load_Balancer . health_check check_engine_stats lb_data norecompute compute check_engine_stats northd recompute nocompute +check_engine_stats lr_stateful recompute nocompute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute nocompute @@ -10581,6 +10587,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats ovn-nbctl --wait=sb lsp-set-options sw0-lr0 router-port=lr0-sw0 check_engine_stats lb_data norecompute compute check_engine_stats northd recompute compute +check_engine_stats lr_stateful recompute nocompute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute nocompute @@ -10589,6 +10596,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb ls-lb-add sw0 lb1 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_stateful norecompute compute check_engine_stats lflow recompute nocompute # A LB applied to a switch/router triggers: # - a recompute in the first iteration (handling northd change) @@ -10601,6 +10609,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb set load_balancer lb1 vips:'"10.0.0.10:80"'='"10.0.0.100:80"' check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_stateful norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE(1) @@ -10610,6 +10619,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb clear load_Balancer lb1 vips check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_stateful norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE(1) @@ -10619,6 +10629,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lb-add lb1 10.0.0.10:80 10.0.0.3:80 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_stateful norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE(1) @@ -10628,6 +10639,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lb-add lb1 10.0.0.20:80 10.0.0.30:8080 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_stateful norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE(1) @@ -10637,6 +10649,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb ls-lb-del sw0 lb1 check_engine_stats lb_data norecompute compute check_engine_stats northd recompute nocompute +check_engine_stats lr_stateful recompute nocompute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE(1) @@ -10647,6 +10660,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb ls-lb-add sw0 lb1 -- lsp-add sw0 sw0p1 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_stateful norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute compute @@ -10667,6 +10681,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lr-lb-add lr0 lb1 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_stateful norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10676,6 +10691,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb set load_balancer lb1 vips:'"10.0.0.10:80"'='"10.0.0.100:80"' check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_stateful norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10685,6 +10701,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb clear load_Balancer lb1 vips check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_stateful norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10694,6 +10711,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lb-add lb1 10.0.0.10:80 10.0.0.3:80 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_stateful norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10703,6 +10721,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lb-add lb1 10.0.0.20:80 10.0.0.30:8080 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_stateful norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10733,6 +10752,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb add load_balancer_group . load_Balancer $lb1_uuid check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_stateful norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute nocompute @@ -10740,6 +10760,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb clear load_balancer_group . load_Balancer check_engine_stats lb_data norecompute compute check_engine_stats northd recompute nocompute +check_engine_stats lr_stateful recompute nocompute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute nocompute @@ -10756,6 +10777,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb add logical_switch sw0 load_balancer_group $lbg1_uuid check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_stateful norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute compute @@ -10772,6 +10794,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb set load_balancer lb1 vips:'"10.0.0.10:80"'='"10.0.0.100:80"' check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_stateful norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10781,6 +10804,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb clear load_Balancer lb1 vips check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_stateful norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10790,6 +10814,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lb-add lb1 10.0.0.10:80 10.0.0.3:80 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_stateful norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10799,6 +10824,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lb-add lb1 10.0.0.20:80 10.0.0.30:8080 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_stateful norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10814,6 +10840,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl add logical_router lr0 load_balancer_group $lbg1_uuid check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_stateful norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10823,6 +10850,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb set load_balancer lb1 vips:'"10.0.0.10:80"'='"10.0.0.100:80"' check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_stateful norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10832,6 +10860,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb clear load_Balancer lb1 vips check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_stateful norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10841,6 +10870,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lb-add lb1 10.0.0.10:80 10.0.0.3:80 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_stateful norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10850,6 +10880,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lb-add lb1 10.0.0.20:80 10.0.0.30:8080 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_stateful norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10858,6 +10889,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb clear logical_router lr0 load_balancer_group check_engine_stats lb_data norecompute compute check_engine_stats northd recompute nocompute +check_engine_stats lr_stateful recompute nocompute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute compute @@ -10866,6 +10898,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb add logical_switch sw0 load_balancer_group $lbg1_uuid check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_stateful norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute compute @@ -10874,6 +10907,7 @@ check ovn-nbctl --wait=sb clear logical_switch sw0 load_balancer_group -- \ destroy load_balancer_group $lbg1_uuid check_engine_stats lb_data norecompute compute check_engine_stats northd recompute nocompute +check_engine_stats lr_stateful recompute nocompute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb compute compute @@ -10897,6 +10931,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats lbg1_uuid=$(ovn-nbctl --wait=sb create load_balancer_group name=lbg1) check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_stateful norecompute compute check_engine_stats lflow norecompute nocompute check_engine_stats sync_to_sb_lb norecompute nocompute @@ -10904,6 +10939,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb set load_balancer_group . load_balancer="$lb2_uuid,$lb3_uuid,$lb4_uuid" check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_stateful norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute nocompute @@ -10911,6 +10947,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb set logical_switch sw0 load_balancer_group=$lbg1_uuid check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_stateful norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10919,6 +10956,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb set logical_router lr1 load_balancer_group=$lbg1_uuid check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_stateful norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10927,6 +10965,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb ls-lb-add sw0 lb2 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_stateful norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10935,6 +10974,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb ls-lb-add sw0 lb3 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_stateful norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10944,6 +10984,7 @@ check ovn-nbctl --wait=sb lr-lb-add lr1 lb1 check ovn-nbctl --wait=sb lr-lb-add lr1 lb2 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_stateful norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10952,6 +10993,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb ls-lb-del sw0 lb2 check_engine_stats lb_data norecompute compute check_engine_stats northd recompute nocompute +check_engine_stats lr_stateful recompute nocompute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10960,6 +11002,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lr-lb-del lr1 lb2 check_engine_stats lb_data norecompute compute check_engine_stats northd recompute nocompute +check_engine_stats lr_stateful recompute nocompute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10970,6 +11013,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lb-del lb4 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_stateful norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10980,6 +11024,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lb-del lb2 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_stateful norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10988,6 +11033,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb remove load_balancer_group . load_balancer $lb3_uuid check_engine_stats lb_data norecompute compute check_engine_stats northd recompute nocompute +check_engine_stats lr_stateful recompute nocompute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11054,6 +11100,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lr-add lr0 check_engine_stats northd recompute nocompute check_engine_stats lr_nat recompute nocompute +check_engine_stats lr_stateful recompute nocompute check_engine_stats sync_to_sb_pb recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11066,6 +11113,7 @@ check ovn-nbctl lrp-add lr0 lr0-sw0 00:00:00:00:ff:01 10.0.0.1/24 # for the SB port binding change. check_engine_stats northd recompute compute check_engine_stats lr_nat recompute nocompute +check_engine_stats lr_stateful recompute nocompute check_engine_stats sync_to_sb_pb recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11077,6 +11125,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lsp-set-options sw0-lr0 router-port=lr0-sw0 check_engine_stats northd recompute compute check_engine_stats lr_nat recompute nocompute +check_engine_stats lr_stateful recompute nocompute check_engine_stats sync_to_sb_pb recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11102,6 +11151,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl set logical_router_port lr0-sw0 options:foo=bar check_engine_stats northd recompute nocompute check_engine_stats lr_nat recompute nocompute +check_engine_stats lr_stateful recompute nocompute check_engine_stats sync_to_sb_pb recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11131,6 +11181,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb set NAT . external_ip=172.168.0.120 check_engine_stats northd norecompute compute check_engine_stats lr_nat norecompute compute +check_engine_stats lr_stateful norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_pb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11140,6 +11191,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb set NAT . logical_ip=10.0.0.10 check_engine_stats northd norecompute compute check_engine_stats lr_nat norecompute compute +check_engine_stats lr_stateful norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_pb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11149,6 +11201,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb set NAT . type=snat check_engine_stats northd norecompute compute check_engine_stats lr_nat norecompute compute +check_engine_stats lr_stateful norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_pb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11158,6 +11211,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lr-nat-add lr0 dnat_and_snat 172.168.0.110 10.0.0.4 sw0p1 30:54:00:00:00:03 check_engine_stats northd norecompute compute check_engine_stats lr_nat norecompute compute +check_engine_stats lr_stateful norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_pb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11168,6 +11222,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb set NAT $nat2_uuid external_mac='"30:54:00:00:00:04"' check_engine_stats northd norecompute compute check_engine_stats lr_nat norecompute compute +check_engine_stats lr_stateful norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_pb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11184,6 +11239,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lr-nat-add lr0 dnat_and_snat 172.168.0.140 10.0.0.20 check_engine_stats northd norecompute compute check_engine_stats lr_nat norecompute compute +check_engine_stats lr_stateful norecompute compute check_engine_stats sync_to_sb_pb recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11192,6 +11248,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lr-nat-add lr0 dnat_and_snat 172.168.0.150 10.0.0.41 check_engine_stats northd norecompute compute check_engine_stats lr_nat norecompute compute +check_engine_stats lr_stateful norecompute compute check_engine_stats sync_to_sb_pb recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11200,6 +11257,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lr-nat-del lr0 dnat_and_snat 172.168.0.150 check_engine_stats northd norecompute compute check_engine_stats lr_nat norecompute compute +check_engine_stats lr_stateful norecompute compute check_engine_stats sync_to_sb_pb recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11208,6 +11266,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lr-nat-del lr0 dnat_and_snat 172.168.0.140 check_engine_stats northd norecompute compute check_engine_stats lr_nat norecompute compute +check_engine_stats lr_stateful norecompute compute check_engine_stats sync_to_sb_pb recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11217,6 +11276,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb clear logical_router lr0 nat check_engine_stats northd norecompute compute check_engine_stats lr_nat norecompute compute +check_engine_stats lr_stateful norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_pb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11226,6 +11286,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lr-policy-add lr0 10 "ip4.src == 10.0.0.3" reroute 172.168.0.101,172.168.0.102 check_engine_stats northd recompute nocompute check_engine_stats lr_nat recompute nocompute +check_engine_stats lr_stateful recompute nocompute check_engine_stats sync_to_sb_pb recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11234,6 +11295,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lr-policy-del lr0 10 "ip4.src == 10.0.0.3" check_engine_stats northd recompute nocompute check_engine_stats lr_nat recompute nocompute +check_engine_stats lr_stateful recompute nocompute check_engine_stats sync_to_sb_pb recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE From patchwork Tue Nov 28 02:35:40 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Numan Siddique X-Patchwork-Id: 1869077 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.138; helo=smtp1.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4SfRRX6mxHz1yRy for ; Tue, 28 Nov 2023 13:36:40 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 617D881BD9; Tue, 28 Nov 2023 02:36:38 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 617D881BD9 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id e7yaK28nWPPO; Tue, 28 Nov 2023 02:36:33 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp1.osuosl.org (Postfix) with ESMTPS id 3779781B8A; Tue, 28 Nov 2023 02:36:32 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 3779781B8A Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id CEDCEC0072; Tue, 28 Nov 2023 02:36:31 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by lists.linuxfoundation.org (Postfix) with ESMTP id 1AF45C0072 for ; Tue, 28 Nov 2023 02:36:30 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id B9F878169A for ; Tue, 28 Nov 2023 02:36:14 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org B9F878169A X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id tPyD7r1J4gc0 for ; Tue, 28 Nov 2023 02:36:12 +0000 (UTC) Received: from relay8-d.mail.gandi.net (relay8-d.mail.gandi.net [IPv6:2001:4b98:dc4:8::228]) by smtp1.osuosl.org (Postfix) with ESMTPS id 1C51781A34 for ; Tue, 28 Nov 2023 02:36:11 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 1C51781A34 Received: by mail.gandi.net (Postfix) with ESMTPSA id A9C3A1BF203; Tue, 28 Nov 2023 02:36:09 +0000 (UTC) From: numans@ovn.org To: dev@openvswitch.org Date: Mon, 27 Nov 2023 21:35:40 -0500 Message-ID: <20231128023540.569754-1-numans@ovn.org> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20231128023410.569539-1-numans@ovn.org> References: <20231128023410.569539-1-numans@ovn.org> MIME-Version: 1.0 X-GND-Sasl: numans@ovn.org Subject: [ovs-dev] [PATCH ovn v3 06/16] northd: Generate router's stateful flows using lr_stateful data. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Numan Siddique Previous commits added new engine nodes to store logical router's stateful (LB and NAT data). Make use of the data stored by these engine nodes to generate logical flows related to router's LBs and NATs. Signed-off-by: Numan Siddique --- northd/en-lflow.c | 3 - northd/en-lr-stateful.h | 4 + northd/inc-proc-northd.c | 1 - northd/northd.c | 772 +++++++++++++++++++++++++-------------- northd/northd.h | 1 - 5 files changed, 506 insertions(+), 275 deletions(-) diff --git a/northd/en-lflow.c b/northd/en-lflow.c index 0aee073520..ef5aa11c17 100644 --- a/northd/en-lflow.c +++ b/northd/en-lflow.c @@ -42,8 +42,6 @@ lflow_get_input_data(struct engine_node *node, engine_get_input_data("port_group", node); struct sync_meters_data *sync_meters_data = engine_get_input_data("sync_meters", node); - struct ed_type_lr_nat_data *lr_nat_data = - engine_get_input_data("lr_nat", node); struct ed_type_lr_stateful *lr_sful_data = engine_get_input_data("lr_stateful", node); @@ -68,7 +66,6 @@ lflow_get_input_data(struct engine_node *node, lflow_input->ls_ports = &northd_data->ls_ports; lflow_input->lr_ports = &northd_data->lr_ports; lflow_input->ls_port_groups = &pg_data->ls_port_groups; - lflow_input->lr_nats = &lr_nat_data->lr_nats; lflow_input->lr_sful_table = &lr_sful_data->lr_sful_table; lflow_input->meter_groups = &sync_meters_data->meter_groups; lflow_input->lb_datapaths_map = &northd_data->lb_datapaths_map; diff --git a/northd/en-lr-stateful.h b/northd/en-lr-stateful.h index a72f000d36..4360419788 100644 --- a/northd/en-lr-stateful.h +++ b/northd/en-lr-stateful.h @@ -66,6 +66,10 @@ struct lr_stateful_table { #define LR_STATEFUL_TABLE_FOR_EACH(LR_LB_NAT_REC, TABLE) \ HMAP_FOR_EACH (LR_LB_NAT_REC, key_node, &(TABLE)->entries) +#define LR_STATEFUL_TABLE_FOR_EACH_IN_P(LR_SFUL_REC, JOBID, TABLE) \ + HMAP_FOR_EACH_IN_PARALLEL (LR_SFUL_REC, key_node, JOBID, \ + &(TABLE)->entries) + struct lr_stateful_tracked_data { /* Created or updated logical router with LB and/or NAT data. */ struct hmapx crupdated; /* Stores 'struct lr_stateful_record'. */ diff --git a/northd/inc-proc-northd.c b/northd/inc-proc-northd.c index 97bcce9655..adb38dde78 100644 --- a/northd/inc-proc-northd.c +++ b/northd/inc-proc-northd.c @@ -221,7 +221,6 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, engine_add_input(&en_lflow, &en_sb_logical_flow, NULL); engine_add_input(&en_lflow, &en_sb_multicast_group, NULL); engine_add_input(&en_lflow, &en_sb_igmp_group, NULL); - engine_add_input(&en_lflow, &en_lr_nat, NULL); engine_add_input(&en_lflow, &en_lr_stateful, NULL); engine_add_input(&en_lflow, &en_northd, lflow_northd_handler); engine_add_input(&en_lflow, &en_port_group, lflow_port_group_handler); diff --git a/northd/northd.c b/northd/northd.c index e9b8332e21..176433a6c5 100644 --- a/northd/northd.c +++ b/northd/northd.c @@ -8844,18 +8844,14 @@ build_lrouter_groups(struct hmap *lr_ports, struct ovs_list *lr_list) */ static void build_lswitch_rport_arp_req_self_orig_flow(struct ovn_port *op, - uint32_t priority, - struct ovn_datapath *od, - const struct lr_nat_table *lr_nats, - struct hmap *lflows) + uint32_t priority, + const struct ovn_datapath *od, + const struct lr_nat_record *lrnat_rec, + struct hmap *lflows) { struct ds eth_src = DS_EMPTY_INITIALIZER; struct ds match = DS_EMPTY_INITIALIZER; - const struct lr_nat_record *lrnat_rec = lr_nat_table_find_by_index( - lr_nats, op->od->index); - ovs_assert(lrnat_rec); - /* Self originated ARP requests/RARP/ND need to be flooded to the L2 domain * (except on router ports). Determine that packets are self originated * by also matching on source MAC. Matching on ingress port is not @@ -8942,7 +8938,8 @@ lrouter_port_ipv6_reachable(const struct ovn_port *op, */ static void build_lswitch_rport_arp_req_flow(const char *ips, - int addr_family, struct ovn_port *patch_op, struct ovn_datapath *od, + int addr_family, struct ovn_port *patch_op, + const struct ovn_datapath *od, uint32_t priority, struct hmap *lflows, const struct ovsdb_idl_row *stage_hint) { @@ -8981,12 +8978,10 @@ build_lswitch_rport_arp_req_flow(const char *ips, */ static void build_lswitch_rport_arp_req_flows(struct ovn_port *op, - struct ovn_datapath *sw_od, - struct ovn_port *sw_op, - const struct lr_nat_table *lr_nats, - const struct lr_stateful_table *lr_sful_table, - struct hmap *lflows, - const struct ovsdb_idl_row *stage_hint) + struct ovn_datapath *sw_od, + struct ovn_port *sw_op, + struct hmap *lflows, + const struct ovsdb_idl_row *stage_hint) { if (!op || !op->nbrp) { return; @@ -9000,12 +8995,48 @@ build_lswitch_rport_arp_req_flows(struct ovn_port *op, * router port. * Priority: 80. */ - const struct lr_stateful_record *lr_sful_rec = NULL; - if (op->od->nbr->n_load_balancer || op->od->nbr->n_load_balancer_group) { - lr_sful_rec = lr_stateful_table_find_by_index(lr_sful_table, - op->od->index); - ovs_assert(lr_sful_rec); + for (size_t i = 0; i < op->lrp_networks.n_ipv4_addrs; i++) { + build_lswitch_rport_arp_req_flow( + op->lrp_networks.ipv4_addrs[i].addr_s, AF_INET, sw_op, sw_od, 80, + lflows, stage_hint); + } + for (size_t i = 0; i < op->lrp_networks.n_ipv6_addrs; i++) { + build_lswitch_rport_arp_req_flow( + op->lrp_networks.ipv6_addrs[i].addr_s, AF_INET6, sw_op, sw_od, 80, + lflows, stage_hint); + } +} +/* + * Ingress table 25: Flows that forward ARP/ND requests only to the routers + * that own the addresses. + * Priorities: + * - 80: self originated GARPs that need to follow regular processing. + * - 75: ARP requests to router owned IPs (interface IP/LB/NAT). + */ +static void +build_lswitch_rport_arp_req_flows_for_lbnats(struct ovn_port *op, + const struct lr_stateful_record *lr_sful_rec, + const struct ovn_datapath *sw_od, + struct ovn_port *sw_op, + struct hmap *lflows, + const struct ovsdb_idl_row *stage_hint) +{ + if (!op || !op->nbrp) { + return; + } + + if (!lrport_is_enabled(op->nbrp)) { + return; + } + + ovs_assert(op->od == lr_sful_rec->od); + + /* Forward ARP requests for owned IP addresses (L3, VIP, NAT) only to this + * router port. + * Priority: 80. + */ + if (op->od->nbr->n_load_balancer || op->od->nbr->n_load_balancer_group) { const char *ip_addr; SSET_FOR_EACH (ip_addr, &lr_sful_rec->lb_ips->ips_v4_reachable) { ovs_be32 ipv4_addr; @@ -9035,17 +9066,6 @@ build_lswitch_rport_arp_req_flows(struct ovn_port *op, } } - for (size_t i = 0; i < op->lrp_networks.n_ipv4_addrs; i++) { - build_lswitch_rport_arp_req_flow( - op->lrp_networks.ipv4_addrs[i].addr_s, AF_INET, sw_op, sw_od, 80, - lflows, stage_hint); - } - for (size_t i = 0; i < op->lrp_networks.n_ipv6_addrs; i++) { - build_lswitch_rport_arp_req_flow( - op->lrp_networks.ipv6_addrs[i].addr_s, AF_INET6, sw_op, sw_od, 80, - lflows, stage_hint); - } - /* Self originated ARP requests/RARP/ND need to be flooded as usual. * * However, if the switch doesn't have any non-router ports we shouldn't @@ -9054,19 +9074,13 @@ build_lswitch_rport_arp_req_flows(struct ovn_port *op, * Priority: 75. */ if (sw_od->n_router_ports != sw_od->nbs->n_ports) { - build_lswitch_rport_arp_req_self_orig_flow(op, 75, sw_od, lr_nats, + build_lswitch_rport_arp_req_self_orig_flow(op, 75, sw_od, + lr_sful_rec->lrnat_rec, lflows); } - const struct lr_nat_record *lrnat_rec = - lr_nat_table_find_by_index(lr_nats, op->od->index); - - if (!lrnat_rec) { - return; - } - - for (size_t i = 0; i < lrnat_rec->n_nat_entries; i++) { - struct ovn_nat *nat_entry = &lrnat_rec->nat_entries[i]; + for (size_t i = 0; i < lr_sful_rec->lrnat_rec->n_nat_entries; i++) { + struct ovn_nat *nat_entry = &lr_sful_rec->lrnat_rec->nat_entries[i]; const struct nbrec_nat *nat = nat_entry->nb; if (!nat_entry_is_valid(nat_entry)) { @@ -9081,15 +9095,15 @@ build_lswitch_rport_arp_req_flows(struct ovn_port *op, * expect ARP requests/NS for the DNAT external_ip. */ if (nat_entry_is_v6(nat_entry)) { - if (!lr_sful_rec || !sset_contains(&lr_sful_rec->lb_ips->ips_v6, - nat->external_ip)) { + if (!sset_contains(&lr_sful_rec->lb_ips->ips_v6, + nat->external_ip)) { build_lswitch_rport_arp_req_flow( nat->external_ip, AF_INET6, sw_op, sw_od, 80, lflows, stage_hint); } } else { - if (!lr_sful_rec || !sset_contains(&lr_sful_rec->lb_ips->ips_v4, - nat->external_ip)) { + if (!sset_contains(&lr_sful_rec->lb_ips->ips_v4, + nat->external_ip)) { build_lswitch_rport_arp_req_flow( nat->external_ip, AF_INET, sw_op, sw_od, 80, lflows, stage_hint); @@ -10128,12 +10142,8 @@ build_lswitch_ip_mcast_igmp_mld(struct ovn_igmp_group *igmp_group, /* Ingress table 25: Destination lookup, unicast handling (priority 50), */ static void -build_lswitch_ip_unicast_lookup(struct ovn_port *op, - const struct lr_nat_table *lr_nats, - const struct lr_stateful_table *lr_sful_table, - struct hmap *lflows, - struct ds *actions, - struct ds *match) +build_lswitch_ip_unicast_lookup(struct ovn_port *op, struct hmap *lflows, + struct ds *actions, struct ds *match) { ovs_assert(op->nbsp); if (lsp_is_external(op->nbsp)) { @@ -10145,8 +10155,7 @@ build_lswitch_ip_unicast_lookup(struct ovn_port *op, * requests only to the router port that owns the IP address. */ if (lsp_is_router(op->nbsp)) { - build_lswitch_rport_arp_req_flows(op->peer, op->od, op, lr_nats, - lr_sful_table, lflows, + build_lswitch_rport_arp_req_flows(op->peer, op->od, op, lflows, &op->nbsp->header_); } @@ -10243,33 +10252,6 @@ build_lswitch_ip_unicast_lookup(struct ovn_port *op, S_SWITCH_IN_L2_LKUP, 50, ds_cstr(match), ds_cstr(actions), &op->nbsp->header_); - - /* Add ethernet addresses specified in NAT rules on - * distributed logical routers. */ - if (is_l3dgw_port(op->peer)) { - for (int j = 0; j < op->peer->od->nbr->n_nat; j++) { - const struct nbrec_nat *nat - = op->peer->od->nbr->nat[j]; - if (!strcmp(nat->type, "dnat_and_snat") - && nat->logical_port && nat->external_mac - && eth_addr_from_string(nat->external_mac, &mac)) { - - ds_clear(match); - ds_put_format(match, "eth.dst == "ETH_ADDR_FMT - " && is_chassis_resident(\"%s\")", - ETH_ADDR_ARGS(mac), - nat->logical_port); - - ds_clear(actions); - ds_put_format(actions, action, op->json_key); - ovn_lflow_add_with_hint(lflows, op->od, - S_SWITCH_IN_L2_LKUP, 50, - ds_cstr(match), - ds_cstr(actions), - &op->nbsp->header_); - } - } - } } else { static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 1); @@ -10280,6 +10262,52 @@ build_lswitch_ip_unicast_lookup(struct ovn_port *op, } } +/* Ingress table 25: Destination lookup, unicast handling (priority 50), */ +static void +build_lswitch_ip_unicast_lookup_for_nats(struct ovn_port *op, + const struct lr_stateful_record *lr_sful_rec, + struct hmap *lflows, struct ds *match, + struct ds *actions) +{ + ovs_assert(op->nbsp); + + if (!op->peer || !is_l3dgw_port(op->peer)) { + return; + } + + ovs_assert(op->peer->od == lr_sful_rec->od); + + const char *action = lsp_is_enabled(op->nbsp) ? + "outport = %s; output;" : + debug_drop_action(); + struct eth_addr mac; + + /* Add ethernet addresses specified in NAT rules on + * distributed logical routers. */ + for (size_t i = 0; i < lr_sful_rec->lrnat_rec->n_nat_entries; i++) { + const struct ovn_nat *nat = &lr_sful_rec->lrnat_rec->nat_entries[i]; + + if (!strcmp(nat->nb->type, "dnat_and_snat") + && nat->nb->logical_port && nat->nb->external_mac + && eth_addr_from_string(nat->nb->external_mac, &mac)) { + + ds_clear(match); + ds_put_format(match, "eth.dst == "ETH_ADDR_FMT + " && is_chassis_resident(\"%s\")", + ETH_ADDR_ARGS(mac), + nat->nb->logical_port); + + ds_clear(actions); + ds_put_format(actions, action, op->json_key); + ovn_lflow_add_with_hint(lflows, op->od, + S_SWITCH_IN_L2_LKUP, 50, + ds_cstr(match), + ds_cstr(actions), + &op->nbsp->header_); + } + } +} + struct bfd_entry { struct hmap_node hmap_node; @@ -11661,7 +11689,7 @@ build_lrouter_nat_flows_for_lb(struct ovn_lb_vip *lb_vip, struct ovn_lb_datapaths *lb_dps, struct ovn_northd_lb_vip *vips_nb, const struct ovn_datapaths *lr_datapaths, - const struct lr_nat_table *lr_nats, + const struct lr_stateful_table *lr_sful_table, struct hmap *lflows, struct ds *match, struct ds *action, const struct shash *meter_groups, @@ -11767,9 +11795,11 @@ build_lrouter_nat_flows_for_lb(struct ovn_lb_vip *lb_vip, struct ovn_datapath *od = lr_datapaths->array[index]; enum lrouter_nat_lb_flow_type type; - const struct lr_nat_record *lrnat_rec = - lr_nat_table_find_by_index(lr_nats, od->index); - ovs_assert(lrnat_rec); + const struct lr_stateful_record *lr_sful_rec = + lr_stateful_table_find_by_index(lr_sful_table, od->index); + ovs_assert(lr_sful_rec); + + const struct lr_nat_record *lrnat_rec = lr_sful_rec->lrnat_rec; if (lb->skip_snat) { type = LROUTER_NAT_LB_FLOW_SKIP_SNAT; } else if (!lport_addresses_is_empty(&lrnat_rec->lb_force_snat_addrs) @@ -11919,7 +11949,7 @@ build_lrouter_flows_for_lb(struct ovn_lb_datapaths *lb_dps, struct hmap *lflows, const struct shash *meter_groups, const struct ovn_datapaths *lr_datapaths, - const struct lr_nat_table *lr_nats, + const struct lr_stateful_table *lr_sful_table, const struct chassis_features *features, const struct hmap *svc_monitor_map, struct ds *match, struct ds *action) @@ -11935,7 +11965,7 @@ build_lrouter_flows_for_lb(struct ovn_lb_datapaths *lb_dps, struct ovn_lb_vip *lb_vip = &lb->vips[i]; build_lrouter_nat_flows_for_lb(lb_vip, lb_dps, &lb->vips_nb[i], - lr_datapaths, lr_nats, lflows, match, + lr_datapaths, lr_sful_table, lflows, match, action, meter_groups, features, svc_monitor_map); @@ -12073,7 +12103,7 @@ lrouter_dnat_and_snat_is_stateless(const struct nbrec_nat *nat) * and action says "next" instead of ct*. */ static inline void -lrouter_nat_add_ext_ip_match(struct ovn_datapath *od, +lrouter_nat_add_ext_ip_match(const struct ovn_datapath *od, struct hmap *lflows, struct ds *match, const struct nbrec_nat *nat, bool is_v6, bool is_src, int cidr_bits) @@ -12137,7 +12167,7 @@ lrouter_nat_add_ext_ip_match(struct ovn_datapath *od, * with the given priority. */ static void -build_lrouter_arp_flow(struct ovn_datapath *od, struct ovn_port *op, +build_lrouter_arp_flow(const struct ovn_datapath *od, struct ovn_port *op, const char *ip_address, const char *eth_addr, struct ds *extra_match, bool drop, uint16_t priority, const struct ovsdb_idl_row *hint, @@ -12186,7 +12216,7 @@ build_lrouter_arp_flow(struct ovn_datapath *od, struct ovn_port *op, * 'sn_ip_address'. */ static void -build_lrouter_nd_flow(struct ovn_datapath *od, struct ovn_port *op, +build_lrouter_nd_flow(const struct ovn_datapath *od, struct ovn_port *op, const char *action, const char *ip_address, const char *sn_ip_address, const char *eth_addr, struct ds *extra_match, bool drop, uint16_t priority, @@ -12240,7 +12270,7 @@ build_lrouter_nd_flow(struct ovn_datapath *od, struct ovn_port *op, } static void -build_lrouter_nat_arp_nd_flow(struct ovn_datapath *od, +build_lrouter_nat_arp_nd_flow(const struct ovn_datapath *od, struct ovn_nat *nat_entry, struct hmap *lflows, const struct shash *meter_groups) @@ -12336,7 +12366,6 @@ build_lrouter_port_nat_arp_nd_flow(struct ovn_port *op, static void build_lrouter_drop_own_dest(struct ovn_port *op, - const struct lr_nat_record *lrnat_rec, const struct lr_stateful_record *lr_sful_rec, enum ovn_stage stage, uint16_t priority, bool drop_snat_ip, @@ -12348,11 +12377,10 @@ build_lrouter_drop_own_dest(struct ovn_port *op, for (size_t i = 0; i < op->lrp_networks.n_ipv4_addrs; i++) { const char *ip = op->lrp_networks.ipv4_addrs[i].addr_s; - bool router_ip_in_snat_ips = !!shash_find(&lrnat_rec->snat_ips, - ip); - bool router_ip_in_lb_ips = (lr_sful_rec && - !!sset_find(&lr_sful_rec->lb_ips->ips_v4, - ip)); + bool router_ip_in_snat_ips = + !!shash_find(&lr_sful_rec->lrnat_rec->snat_ips, ip); + bool router_ip_in_lb_ips = + !!sset_find(&lr_sful_rec->lb_ips->ips_v4, ip); bool drop_router_ip = (drop_snat_ip == (router_ip_in_snat_ips || router_ip_in_lb_ips)); @@ -12379,11 +12407,10 @@ build_lrouter_drop_own_dest(struct ovn_port *op, for (size_t i = 0; i < op->lrp_networks.n_ipv6_addrs; i++) { const char *ip = op->lrp_networks.ipv6_addrs[i].addr_s; - bool router_ip_in_snat_ips = !!shash_find(&lrnat_rec->snat_ips, - ip); - bool router_ip_in_lb_ips = (lr_sful_rec && - !!sset_find(&lr_sful_rec->lb_ips->ips_v6, - ip)); + bool router_ip_in_snat_ips = + !!shash_find(&lr_sful_rec->lrnat_rec->snat_ips, ip); + bool router_ip_in_lb_ips = + !!sset_find(&lr_sful_rec->lb_ips->ips_v6, ip); bool drop_router_ip = (drop_snat_ip == (router_ip_in_snat_ips || router_ip_in_lb_ips)); @@ -12407,7 +12434,8 @@ build_lrouter_drop_own_dest(struct ovn_port *op, } static void -build_lrouter_force_snat_flows(struct hmap *lflows, struct ovn_datapath *od, +build_lrouter_force_snat_flows(struct hmap *lflows, + const struct ovn_datapath *od, const char *ip_version, const char *ip_addr, const char *context) { @@ -13407,8 +13435,7 @@ routable_addresses_to_lflows(struct hmap *lflows, struct ovn_port *router_port, /* This function adds ARP resolve flows related to a LRP. */ static void build_arp_resolve_flows_for_lrp( - struct ovn_port *op, const struct lr_nat_record *lrnat_rec, - const struct lr_stateful_record *lr_sful_rec, + struct ovn_port *op, struct hmap *lflows, struct ds *match, struct ds *actions) { ovs_assert(op->nbrp); @@ -13478,15 +13505,6 @@ build_arp_resolve_flows_for_lrp( &op->nbrp->header_); } } - - /* Drop IP traffic destined to router owned IPs. Part of it is dropped - * in stage "lr_in_ip_input" but traffic that could have been unSNATed - * but didn't match any existing session might still end up here. - * - * Priority 2. - */ - build_lrouter_drop_own_dest(op, lrnat_rec, lr_sful_rec, - S_ROUTER_IN_ARP_RESOLVE, 2, true, lflows); } /* This function adds ARP resolve flows related to a LSP. */ @@ -13494,7 +13512,6 @@ static void build_arp_resolve_flows_for_lsp( struct ovn_port *op, struct hmap *lflows, const struct hmap *lr_ports, - const struct lr_stateful_table *lr_sful_table, struct ds *match, struct ds *actions) { ovs_assert(op->nbsp); @@ -13635,15 +13652,50 @@ build_arp_resolve_flows_for_lsp( ds_cstr(match), ds_cstr(actions), &op->nbsp->header_); } + } + } +} - if (smap_get(&peer->od->nbr->options, "chassis") - || peer->cr_port) { - const struct lr_stateful_record *lr_sful_rec; - lr_sful_rec = lr_stateful_table_find_by_index(lr_sful_table, +static void +build_arp_resolve_flows_for_lsp_routable_addresses( + struct ovn_port *op, struct hmap *lflows, + const struct hmap *lr_ports, + const struct lr_stateful_table *lr_sful_table, + struct ds *match, struct ds *actions) +{ + if (!lsp_is_router(op->nbsp)) { + return; + } + + struct ovn_port *peer = ovn_port_get_peer(lr_ports, op); + if (!peer || !peer->nbrp) { + return; + } + + if (peer->od->nbr && + smap_get_bool(&peer->od->nbr->options, + "dynamic_neigh_routers", false)) { + return; + } + + for (size_t i = 0; i < op->od->n_router_ports; i++) { + struct ovn_port *router_port = + ovn_port_get_peer(lr_ports, op->od->router_ports[i]); + if (!router_port || !router_port->nbrp) { + continue; + } + + /* Skip the router port under consideration. */ + if (router_port == peer) { + continue; + } + + if (smap_get(&peer->od->nbr->options, "chassis") || peer->cr_port) { + const struct lr_stateful_record *lr_sful_rec; + lr_sful_rec = lr_stateful_table_find_by_index(lr_sful_table, router_port->od->index); - routable_addresses_to_lflows(lflows, router_port, peer, - lr_sful_rec, match, actions); - } + routable_addresses_to_lflows(lflows, router_port, peer, + lr_sful_rec, match, actions); } } } @@ -13820,7 +13872,6 @@ build_check_pkt_len_flows_for_lrouter( static void build_gateway_redirect_flows_for_lrouter( struct ovn_datapath *od, struct hmap *lflows, - const struct lr_nat_table *lr_nats, struct ds *match, struct ds *actions) { ovs_assert(od->nbr); @@ -13837,7 +13888,6 @@ build_gateway_redirect_flows_for_lrouter( } const struct ovsdb_idl_row *stage_hint = NULL; - bool add_def_flow = true; if (od->l3dgw_ports[i]->nbrp) { stage_hint = &od->l3dgw_ports[i]->nbrp->header_; @@ -13856,14 +13906,33 @@ build_gateway_redirect_flows_for_lrouter( ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_GW_REDIRECT, 50, ds_cstr(match), ds_cstr(actions), stage_hint); + } - const struct lr_nat_record *lrnat_rec = lr_nat_table_find_by_index( - lr_nats, od->index); + /* Packets are allowed by default. */ + ovn_lflow_add(lflows, od, S_ROUTER_IN_GW_REDIRECT, 0, "1", "next;"); +} - if (!lrnat_rec) { +/* Logical router ingress table GW_REDIRECT: Gateway redirect. */ +static void +build_lr_gateway_redirect_flows_for_nats( + const struct ovn_datapath *od, const struct lr_nat_record *lrnat_rec, + struct hmap *lflows, struct ds *match, struct ds *actions) +{ + ovs_assert(od->nbr); + for (size_t i = 0; i < od->n_l3dgw_ports; i++) { + if (l3dgw_port_has_associated_vtep_lports(od->l3dgw_ports[i])) { + /* Skip adding redirect lflow for vtep-enabled l3dgw ports. + * Traffic from hypervisor to VTEP (ramp) switch should go in + * distributed manner. Only returning routed traffic must go + * through centralized gateway (or ha-chassis-group). + * This assumes that attached logical switch with vtep lport(s) has + * no localnet port(s) for NAT. Otherwise centralized NAT will not + * work. */ continue; } + bool add_def_flow = true; + for (int j = 0; j < lrnat_rec->n_nat_entries; j++) { const struct ovn_nat *nat = &lrnat_rec->nat_entries[j]; @@ -13872,6 +13941,12 @@ build_gateway_redirect_flows_for_lrouter( continue; } + const struct ovsdb_idl_row *stage_hint = NULL; + + if (od->l3dgw_ports[i]->nbrp) { + stage_hint = &od->l3dgw_ports[i]->nbrp->header_; + } + struct ds match_ext = DS_EMPTY_INITIALIZER; struct nbrec_address_set *as = nat->nb->allowed_ext_ips ? nat->nb->allowed_ext_ips : nat->nb->exempted_ext_ips; @@ -13901,9 +13976,6 @@ build_gateway_redirect_flows_for_lrouter( ds_destroy(&match_ext); } } - - /* Packets are allowed by default. */ - ovn_lflow_add(lflows, od, S_ROUTER_IN_GW_REDIRECT, 0, "1", "next;"); } /* Local router ingress table ARP_REQUEST: ARP request. @@ -14302,8 +14374,8 @@ build_ipv6_input_flows_for_lrouter_port( } static void -build_lrouter_arp_nd_for_datapath(struct ovn_datapath *od, - const struct lr_nat_table *lr_nats, +build_lrouter_arp_nd_for_datapath(const struct ovn_datapath *od, + const struct lr_nat_record *lrnat_rec, struct hmap *lflows, const struct shash *meter_groups) { @@ -14320,10 +14392,6 @@ build_lrouter_arp_nd_for_datapath(struct ovn_datapath *od, * port to handle the special cases. In case we get the packet * on a regular port, just reply with the port's ETH address. */ - const struct lr_nat_record *lrnat_rec = lr_nat_table_find_by_index( - lr_nats, od->index); - ovs_assert(lrnat_rec); - for (int i = 0; i < lrnat_rec->n_nat_entries; i++) { struct ovn_nat *nat_entry = &lrnat_rec->nat_entries[i]; @@ -14361,8 +14429,6 @@ build_lrouter_arp_nd_for_datapath(struct ovn_datapath *od, static void build_lrouter_ipv4_ip_input(struct ovn_port *op, struct hmap *lflows, - const struct lr_nat_record *lrnat_rec, - const struct lr_stateful_record *lr_sful_rec, struct ds *match, struct ds *actions, const struct shash *meter_groups) { @@ -14487,39 +14553,6 @@ build_lrouter_ipv4_ip_input(struct ovn_port *op, &op->nbrp->header_, lflows); } - if (lr_sful_rec && sset_count(&lr_sful_rec->lb_ips->ips_v4_reachable)) { - ds_clear(match); - if (is_l3dgw_port(op)) { - ds_put_format(match, "is_chassis_resident(%s)", - op->cr_port->json_key); - } - - /* Create a single ARP rule for all IPs that are used as VIPs. */ - char *lb_ips_v4_as = lr_lb_address_set_ref(op->od->tunnel_key, - AF_INET); - build_lrouter_arp_flow(op->od, op, lb_ips_v4_as, - REG_INPORT_ETH_ADDR, - match, false, 90, NULL, lflows); - free(lb_ips_v4_as); - } - - if (lr_sful_rec && sset_count(&lr_sful_rec->lb_ips->ips_v6_reachable)) { - ds_clear(match); - - if (is_l3dgw_port(op)) { - ds_put_format(match, "is_chassis_resident(%s)", - op->cr_port->json_key); - } - - /* Create a single ND rule for all IPs that are used as VIPs. */ - char *lb_ips_v6_as = lr_lb_address_set_ref(op->od->tunnel_key, - AF_INET6); - build_lrouter_nd_flow(op->od, op, "nd_na", lb_ips_v6_as, NULL, - REG_INPORT_ETH_ADDR, match, false, 90, - NULL, lflows, meter_groups); - free(lb_ips_v6_as); - } - if (!op->od->is_gw_router && !op->od->n_l3dgw_ports) { /* UDP/TCP/SCTP port unreachable. */ for (int i = 0; i < op->lrp_networks.n_ipv4_addrs; i++) { @@ -14594,20 +14627,55 @@ build_lrouter_ipv4_ip_input(struct ovn_port *op, &op->nbrp->header_); } } +} - /* Drop IP traffic destined to router owned IPs except if the IP is - * also a SNAT IP. Those are dropped later, in stage - * "lr_in_arp_resolve", if unSNAT was unsuccessful. - * - * If lrnat_rec->lb_force_snat_router_ip is true, it means the IP of the - * router port is also SNAT IP. - * - * Priority 60. - */ - if (!lrnat_rec->lb_force_snat_router_ip) { - build_lrouter_drop_own_dest(op, lrnat_rec, lr_sful_rec, - S_ROUTER_IN_IP_INPUT, 60, false, lflows); +/* Logical router ingress table 3: IP Input for IPv4. */ +static void +build_lrouter_ipv4_ip_input_for_lbnats(struct ovn_port *op, + struct hmap *lflows, + const struct lr_stateful_record *lr_sful_rec, + struct ds *match, const struct shash *meter_groups) +{ + ovs_assert(op->nbrp); + /* No ingress packets are accepted on a chassisredirect + * port, so no need to program flows for that port. */ + if (is_cr_port(op)) { + return; } + + if (sset_count(&lr_sful_rec->lb_ips->ips_v4_reachable)) { + ds_clear(match); + if (is_l3dgw_port(op)) { + ds_put_format(match, "is_chassis_resident(%s)", + op->cr_port->json_key); + } + + /* Create a single ARP rule for all IPs that are used as VIPs. */ + char *lb_ips_v4_as = lr_lb_address_set_ref(op->od->tunnel_key, + AF_INET); + build_lrouter_arp_flow(op->od, op, lb_ips_v4_as, + REG_INPORT_ETH_ADDR, + match, false, 90, NULL, lflows); + free(lb_ips_v4_as); + } + + if (sset_count(&lr_sful_rec->lb_ips->ips_v6_reachable)) { + ds_clear(match); + + if (is_l3dgw_port(op)) { + ds_put_format(match, "is_chassis_resident(%s)", + op->cr_port->json_key); + } + + /* Create a single ND rule for all IPs that are used as VIPs. */ + char *lb_ips_v6_as = lr_lb_address_set_ref(op->od->tunnel_key, + AF_INET6); + build_lrouter_nd_flow(op->od, op, "nd_na", lb_ips_v6_as, NULL, + REG_INPORT_ETH_ADDR, match, false, 90, + NULL, lflows, meter_groups); + free(lb_ips_v6_as); + } + /* ARP / ND handling for external IP addresses. * * DNAT and SNAT IP addresses are external IP addresses that need ARP @@ -14621,8 +14689,8 @@ build_lrouter_ipv4_ip_input(struct ovn_port *op, return; } - for (int i = 0; i < lrnat_rec->n_nat_entries; i++) { - struct ovn_nat *nat_entry = &lrnat_rec->nat_entries[i]; + for (int i = 0; i < lr_sful_rec->lrnat_rec->n_nat_entries; i++) { + struct ovn_nat *nat_entry = &lr_sful_rec->lrnat_rec->nat_entries[i]; /* Skip entries we failed to parse. */ if (!nat_entry_is_valid(nat_entry)) { @@ -14641,7 +14709,7 @@ build_lrouter_ipv4_ip_input(struct ovn_port *op, /* Now handle SNAT entries too, one per unique SNAT IP. */ struct shash_node *snat_snode; - SHASH_FOR_EACH (snat_snode, &lrnat_rec->snat_ips) { + SHASH_FOR_EACH (snat_snode, &lr_sful_rec->lrnat_rec->snat_ips) { struct ovn_snat_ip *snat_ip = snat_snode->data; if (ovs_list_is_empty(&snat_ip->snat_entries)) { @@ -14657,7 +14725,7 @@ build_lrouter_ipv4_ip_input(struct ovn_port *op, } static void -build_lrouter_in_unsnat_match(struct ovn_datapath *od, +build_lrouter_in_unsnat_match(const struct ovn_datapath *od, const struct nbrec_nat *nat, struct ds *match, bool distributed_nat, bool is_v6, struct ovn_port *l3dgw_port) @@ -14684,7 +14752,7 @@ build_lrouter_in_unsnat_match(struct ovn_datapath *od, static void build_lrouter_in_unsnat_stateless_flow(struct hmap *lflows, - struct ovn_datapath *od, + const struct ovn_datapath *od, const struct nbrec_nat *nat, struct ds *match, bool distributed_nat, bool is_v6, @@ -14706,7 +14774,7 @@ build_lrouter_in_unsnat_stateless_flow(struct hmap *lflows, static void build_lrouter_in_unsnat_in_czone_flow(struct hmap *lflows, - struct ovn_datapath *od, + const struct ovn_datapath *od, const struct nbrec_nat *nat, struct ds *match, bool distributed_nat, bool is_v6, struct ovn_port *l3dgw_port) @@ -14739,7 +14807,8 @@ build_lrouter_in_unsnat_in_czone_flow(struct hmap *lflows, } static void -build_lrouter_in_unsnat_flow(struct hmap *lflows, struct ovn_datapath *od, +build_lrouter_in_unsnat_flow(struct hmap *lflows, + const struct ovn_datapath *od, const struct nbrec_nat *nat, struct ds *match, bool distributed_nat, bool is_v6, struct ovn_port *l3dgw_port) @@ -14760,7 +14829,8 @@ build_lrouter_in_unsnat_flow(struct hmap *lflows, struct ovn_datapath *od, } static void -build_lrouter_in_dnat_flow(struct hmap *lflows, struct ovn_datapath *od, +build_lrouter_in_dnat_flow(struct hmap *lflows, + const struct ovn_datapath *od, const struct lr_nat_record *lrnat_rec, const struct nbrec_nat *nat, struct ds *match, struct ds *actions, bool distributed_nat, @@ -14831,7 +14901,8 @@ build_lrouter_in_dnat_flow(struct hmap *lflows, struct ovn_datapath *od, } static void -build_lrouter_out_undnat_flow(struct hmap *lflows, struct ovn_datapath *od, +build_lrouter_out_undnat_flow(struct hmap *lflows, + const struct ovn_datapath *od, const struct nbrec_nat *nat, struct ds *match, struct ds *actions, bool distributed_nat, struct eth_addr mac, bool is_v6, @@ -14881,7 +14952,8 @@ build_lrouter_out_undnat_flow(struct hmap *lflows, struct ovn_datapath *od, } static void -build_lrouter_out_is_dnat_local(struct hmap *lflows, struct ovn_datapath *od, +build_lrouter_out_is_dnat_local(struct hmap *lflows, + const struct ovn_datapath *od, const struct nbrec_nat *nat, struct ds *match, struct ds *actions, bool distributed_nat, bool is_v6, struct ovn_port *l3dgw_port) @@ -14911,7 +14983,8 @@ build_lrouter_out_is_dnat_local(struct hmap *lflows, struct ovn_datapath *od, } static void -build_lrouter_out_snat_match(struct hmap *lflows, struct ovn_datapath *od, +build_lrouter_out_snat_match(struct hmap *lflows, + const struct ovn_datapath *od, const struct nbrec_nat *nat, struct ds *match, bool distributed_nat, int cidr_bits, bool is_v6, struct ovn_port *l3dgw_port) @@ -14940,7 +15013,7 @@ build_lrouter_out_snat_match(struct hmap *lflows, struct ovn_datapath *od, static void build_lrouter_out_snat_stateless_flow(struct hmap *lflows, - struct ovn_datapath *od, + const struct ovn_datapath *od, const struct nbrec_nat *nat, struct ds *match, struct ds *actions, bool distributed_nat, @@ -14983,7 +15056,7 @@ build_lrouter_out_snat_stateless_flow(struct hmap *lflows, static void build_lrouter_out_snat_in_czone_flow(struct hmap *lflows, - struct ovn_datapath *od, + const struct ovn_datapath *od, const struct nbrec_nat *nat, struct ds *match, struct ds *actions, bool distributed_nat, @@ -15044,7 +15117,8 @@ build_lrouter_out_snat_in_czone_flow(struct hmap *lflows, } static void -build_lrouter_out_snat_flow(struct hmap *lflows, struct ovn_datapath *od, +build_lrouter_out_snat_flow(struct hmap *lflows, + const struct ovn_datapath *od, const struct nbrec_nat *nat, struct ds *match, struct ds *actions, bool distributed_nat, struct eth_addr mac, int cidr_bits, bool is_v6, @@ -15091,9 +15165,10 @@ build_lrouter_out_snat_flow(struct hmap *lflows, struct ovn_datapath *od, static void build_lrouter_ingress_nat_check_pkt_len(struct hmap *lflows, const struct nbrec_nat *nat, - struct ovn_datapath *od, bool is_v6, - struct ds *match, struct ds *actions, - int mtu, struct ovn_port *l3dgw_port, + const struct ovn_datapath *od, + bool is_v6, struct ds *match, + struct ds *actions, int mtu, + struct ovn_port *l3dgw_port, const struct shash *meter_groups) { ds_clear(match); @@ -15160,7 +15235,8 @@ build_lrouter_ingress_nat_check_pkt_len(struct hmap *lflows, } static void -build_lrouter_ingress_flow(struct hmap *lflows, struct ovn_datapath *od, +build_lrouter_ingress_flow(struct hmap *lflows, + const struct ovn_datapath *od, const struct nbrec_nat *nat, struct ds *match, struct ds *actions, struct eth_addr mac, bool distributed_nat, bool is_v6, @@ -15210,7 +15286,8 @@ build_lrouter_ingress_flow(struct hmap *lflows, struct ovn_datapath *od, } static int -lrouter_check_nat_entry(struct ovn_datapath *od, const struct nbrec_nat *nat, +lrouter_check_nat_entry(const struct ovn_datapath *od, + const struct nbrec_nat *nat, const struct hmap *lr_ports, ovs_be32 *mask, bool *is_v6, int *cidr_bits, struct eth_addr *mac, bool *distributed, struct ovn_port **nat_l3dgw_port) @@ -15337,15 +15414,8 @@ lrouter_check_nat_entry(struct ovn_datapath *od, const struct nbrec_nat *nat, } /* NAT, Defrag and load balancing. */ -static void -build_lrouter_nat_defrag_and_lb(struct ovn_datapath *od, struct hmap *lflows, - const struct hmap *ls_ports, - const struct hmap *lr_ports, - const struct lr_nat_table *lr_nats, - struct ds *match, - struct ds *actions, - const struct shash *meter_groups, - const struct chassis_features *features) +static void build_lr_nat_defrag_and_lb_default_flows(struct ovn_datapath *od, + struct hmap *lflows) { ovs_assert(od->nbr); @@ -15362,6 +15432,23 @@ build_lrouter_nat_defrag_and_lb(struct ovn_datapath *od, struct hmap *lflows, ovn_lflow_add(lflows, od, S_ROUTER_OUT_EGR_LOOP, 0, "1", "next;"); ovn_lflow_add(lflows, od, S_ROUTER_IN_ECMP_STATEFUL, 0, "1", "next;"); + /* Send the IPv6 NS packets to next table. When ovn-controller + * generates IPv6 NS (for the action - nd_ns{}), the injected + * packet would go through conntrack - which is not required. */ + ovn_lflow_add(lflows, od, S_ROUTER_OUT_SNAT, 120, "nd_ns", "next;"); +} + +static void +build_lrouter_nat_defrag_and_lb( + const struct lr_stateful_record *lr_sful_rec, struct hmap *lflows, + const struct hmap *ls_ports, const struct hmap *lr_ports, + struct ds *match, struct ds *actions, + const struct shash *meter_groups, + const struct chassis_features *features) +{ + const struct ovn_datapath *od = lr_sful_rec->od; + ovs_assert(od->nbr); + const char *ct_flag_reg = features->ct_no_masked_label ? "ct_mark" : "ct_label"; @@ -15439,11 +15526,6 @@ build_lrouter_nat_defrag_and_lb(struct ovn_datapath *od, struct hmap *lflows, "ip && ct.new", "ct_commit { } ; next; "); } - /* Send the IPv6 NS packets to next table. When ovn-controller - * generates IPv6 NS (for the action - nd_ns{}), the injected - * packet would go through conntrack - which is not required. */ - ovn_lflow_add(lflows, od, S_ROUTER_OUT_SNAT, 120, "nd_ns", "next;"); - /* NAT rules are only valid on Gateway routers and routers with * l3dgw_ports (router has port(s) with gateway chassis * specified). */ @@ -15452,8 +15534,7 @@ build_lrouter_nat_defrag_and_lb(struct ovn_datapath *od, struct hmap *lflows, } struct sset nat_entries = SSET_INITIALIZER(&nat_entries); - const struct lr_nat_record *lrnat_rec = lr_nat_table_find_by_index(lr_nats, - od->index); + const struct lr_nat_record *lrnat_rec = lr_sful_rec->lrnat_rec; ovs_assert(lrnat_rec); bool dnat_force_snat_ip = @@ -15736,7 +15817,129 @@ build_lrouter_nat_defrag_and_lb(struct ovn_datapath *od, struct hmap *lflows, sset_destroy(&nat_entries); } +static void +build_lsp_lflows_for_lbnats(struct ovn_port *lsp, + struct ovn_port *lrp_peer, + const struct lr_stateful_record *lr_sful_rec, + const struct lr_stateful_table *lr_sful_table, + const struct hmap *lr_ports, + struct hmap *lflows, + struct ds *match, + struct ds *actions) +{ + ovs_assert(lsp->nbsp); + start_collecting_lflows(); + build_lswitch_rport_arp_req_flows_for_lbnats( + lrp_peer, lr_sful_rec, lsp->od, lsp, + lflows, &lsp->nbsp->header_); + build_ip_routing_flows_for_router_type_lsp(lsp, lr_sful_table, + lr_ports, lflows); + build_arp_resolve_flows_for_lsp_routable_addresses( + lsp, lflows, lr_ports, lr_sful_table, match, actions); + build_lswitch_ip_unicast_lookup_for_nats(lsp, lr_sful_rec, lflows, + match, actions); + link_ovn_port_to_lflows(lsp, &collected_lflows); + end_collecting_lflows(); +} + +static void +build_lbnat_lflows_iterate_by_lsp(struct ovn_port *op, + const struct lr_stateful_table *lr_sful_table, + const struct hmap *lr_ports, + struct ds *match, + struct ds *actions, + struct hmap *lflows) +{ + ovs_assert(op->nbsp); + + if (!lsp_is_router(op->nbsp) || !op->peer) { + return; + } + + const struct lr_stateful_record *lr_sful_rec; + lr_sful_rec = lr_stateful_table_find_by_index(lr_sful_table, + op->peer->od->index); + ovs_assert(lr_sful_rec); + build_lsp_lflows_for_lbnats(op, op->peer, lr_sful_rec, + lr_sful_table, lr_ports, lflows, + match, actions); +} + +static void +build_lrp_lflows_for_lbnats(struct ovn_port *op, + const struct lr_stateful_record *lr_sful_rec, + const struct shash *meter_groups, + struct ds *match, struct ds *actions, + struct hmap *lflows) +{ + /* Drop IP traffic destined to router owned IPs except if the IP is + * also a SNAT IP. Those are dropped later, in stage + * "lr_in_arp_resolve", if unSNAT was unsuccessful. + * + * If lrnat_rec->lb_force_snat_router_ip is true, it means the IP of the + * router port is also SNAT IP. + * + * Priority 60. + */ + if (!lr_sful_rec->lrnat_rec->lb_force_snat_router_ip) { + build_lrouter_drop_own_dest(op, lr_sful_rec, + S_ROUTER_IN_IP_INPUT, 60, false, lflows); + } + + /* Drop IP traffic destined to router owned IPs. Part of it is dropped + * in stage "lr_in_ip_input" but traffic that could have been unSNATed + * but didn't match any existing session might still end up here. + * + * Priority 2. + */ + build_lrouter_drop_own_dest(op, lr_sful_rec, + S_ROUTER_IN_ARP_RESOLVE, 2, true, lflows); + + build_lrouter_ipv4_ip_input_for_lbnats(op, lflows, lr_sful_rec, + match, meter_groups); + build_lrouter_force_snat_flows_op(op, lr_sful_rec->lrnat_rec, lflows, + match, actions); +} + +static void +build_lbnat_lflows_iterate_by_lrp(struct ovn_port *op, + const struct lr_stateful_table *lr_sful_table, + const struct shash *meter_groups, + struct ds *match, + struct ds *actions, + struct hmap *lflows) +{ + ovs_assert(op->nbrp); + + const struct lr_stateful_record *lr_sful_rec; + lr_sful_rec = lr_stateful_table_find_by_index(lr_sful_table, + op->od->index); + ovs_assert(lr_sful_rec); + + build_lrp_lflows_for_lbnats(op, lr_sful_rec, meter_groups, match, + actions, lflows); +} + +static void +build_lr_stateful_flows(const struct lr_stateful_record *lr_sful_rec, + struct hmap *lflows, + const struct hmap *ls_ports, + const struct hmap *lr_ports, + struct ds *match, + struct ds *actions, + const struct shash *meter_groups, + const struct chassis_features *features) +{ + build_lrouter_nat_defrag_and_lb(lr_sful_rec, lflows, ls_ports, lr_ports, + match, actions, meter_groups, features); + build_lr_gateway_redirect_flows_for_nats(lr_sful_rec->od, + lr_sful_rec->lrnat_rec, lflows, + match, actions); + build_lrouter_arp_nd_for_datapath(lr_sful_rec->od, + lr_sful_rec->lrnat_rec, lflows, + meter_groups); +} struct lswitch_flow_build_info { const struct ovn_datapaths *ls_datapaths; @@ -15744,7 +15947,6 @@ struct lswitch_flow_build_info { const struct hmap *ls_ports; const struct hmap *lr_ports; const struct ls_port_group_table *ls_port_groups; - const struct lr_nat_table *lr_nats; const struct lr_stateful_table *lr_sful_table; struct hmap *lflows; struct hmap *igmp_groups; @@ -15811,17 +16013,13 @@ build_lswitch_and_lrouter_iterate_by_lr(struct ovn_datapath *od, build_check_pkt_len_flows_for_lrouter(od, lsi->lflows, lsi->lr_ports, &lsi->match, &lsi->actions, lsi->meter_groups); - build_gateway_redirect_flows_for_lrouter(od, lsi->lflows, lsi->lr_nats, - &lsi->match, &lsi->actions); + build_gateway_redirect_flows_for_lrouter(od, lsi->lflows, &lsi->match, + &lsi->actions); build_arp_request_flows_for_lrouter(od, lsi->lflows, &lsi->match, &lsi->actions, lsi->meter_groups); build_misc_local_traffic_drop_flows_for_lrouter(od, lsi->lflows); - build_lrouter_arp_nd_for_datapath(od, lsi->lr_nats, lsi->lflows, - lsi->meter_groups); - build_lrouter_nat_defrag_and_lb(od, lsi->lflows, lsi->ls_ports, - lsi->lr_ports,lsi->lr_nats, &lsi->match, - &lsi->actions, lsi->meter_groups, - lsi->features); + + build_lr_nat_defrag_and_lb_default_flows(od, lsi->lflows); build_lrouter_lb_affinity_default_flows(od, lsi->lflows); } @@ -15832,8 +16030,6 @@ static void build_lswitch_and_lrouter_iterate_by_lsp( struct ovn_port *op, const struct hmap *ls_ports, const struct hmap *lr_ports, - const struct lr_nat_table *lr_nats, - const struct lr_stateful_table *lr_sful_table, const struct shash *meter_groups, struct ds *match, struct ds *actions, @@ -15850,14 +16046,11 @@ build_lswitch_and_lrouter_iterate_by_lsp( meter_groups, actions, match); build_lswitch_dhcp_options_and_response(op, lflows, meter_groups); build_lswitch_external_port(op, lflows); - build_lswitch_ip_unicast_lookup(op, lr_nats, lr_sful_table, lflows, - actions, match); + build_lswitch_ip_unicast_lookup(op, lflows, actions, + match); /* Build Logical Router Flows. */ - build_ip_routing_flows_for_router_type_lsp(op, lr_sful_table, lr_ports, - lflows); - build_arp_resolve_flows_for_lsp(op, lflows, lr_ports, lr_sful_table, - match, actions); + build_arp_resolve_flows_for_lsp(op, lflows, lr_ports, match, actions); link_ovn_port_to_lflows(op, &collected_lflows); end_collecting_lflows(); @@ -15872,12 +16065,6 @@ build_lswitch_and_lrouter_iterate_by_lrp(struct ovn_port *op, { ovs_assert(op->nbrp); - const struct lr_nat_record *lrnet_rec = lr_nat_table_find_by_index( - lsi->lr_nats, op->od->index); - ovs_assert(lrnet_rec); - - const struct lr_stateful_record *lr_sful_rec = - lr_stateful_table_find_by_index(lsi->lr_sful_table, op->od->index); build_adm_ctrl_flows_for_lrouter_port(op, lsi->lflows, &lsi->match, &lsi->actions); build_neigh_learning_flows_for_lrouter_port(op, lsi->lflows, &lsi->match, @@ -15885,7 +16072,7 @@ build_lswitch_and_lrouter_iterate_by_lrp(struct ovn_port *op, build_ip_routing_flows_for_lrp(op, lsi->lflows); build_ND_RA_flows_for_lrouter_port(op, lsi->lflows, &lsi->match, &lsi->actions, lsi->meter_groups); - build_arp_resolve_flows_for_lrp(op, lrnet_rec, lr_sful_rec, lsi->lflows, + build_arp_resolve_flows_for_lrp(op, lsi->lflows, &lsi->match, &lsi->actions); build_egress_delivery_flows_for_lrouter_port(op, lsi->lflows, &lsi->match, &lsi->actions); @@ -15893,22 +16080,20 @@ build_lswitch_and_lrouter_iterate_by_lrp(struct ovn_port *op, build_ipv6_input_flows_for_lrouter_port(op, lsi->lflows, &lsi->match, &lsi->actions, lsi->meter_groups); - build_lrouter_ipv4_ip_input(op, lsi->lflows, lrnet_rec, lr_sful_rec, - &lsi->match, &lsi->actions, lsi->meter_groups); - build_lrouter_force_snat_flows_op(op, lrnet_rec, lsi->lflows, &lsi->match, - &lsi->actions); + build_lrouter_ipv4_ip_input(op, lsi->lflows, &lsi->match, &lsi->actions, + lsi->meter_groups); } static void * build_lflows_thread(void *arg) { struct worker_control *control = (struct worker_control *) arg; + const struct lr_stateful_record *lr_sful_rec; struct lswitch_flow_build_info *lsi; - + struct ovn_igmp_group *igmp_group; + struct ovn_lb_datapaths *lb_dps; struct ovn_datapath *od; struct ovn_port *op; - struct ovn_lb_datapaths *lb_dps; - struct ovn_igmp_group *igmp_group; int bnum; while (!stop_parallel_processing()) { @@ -15954,13 +16139,16 @@ build_lflows_thread(void *arg) return NULL; } build_lswitch_and_lrouter_iterate_by_lsp(op, lsi->ls_ports, - lsi->lr_ports, - lsi->lr_nats, - lsi->lr_sful_table, - lsi->meter_groups, - &lsi->match, - &lsi->actions, - lsi->lflows); + lsi->lr_ports, + lsi->meter_groups, + &lsi->match, + &lsi->actions, + lsi->lflows); + build_lbnat_lflows_iterate_by_lsp(op, lsi->lr_sful_table, + lsi->lr_ports, + &lsi->match, + &lsi->actions, + lsi->lflows); } } for (bnum = control->id; @@ -15973,6 +16161,11 @@ build_lflows_thread(void *arg) return NULL; } build_lswitch_and_lrouter_iterate_by_lrp(op, lsi); + build_lbnat_lflows_iterate_by_lrp(op, lsi->lr_sful_table, + lsi->meter_groups, + &lsi->match, + &lsi->actions, + lsi->lflows); } } for (bnum = control->id; @@ -15995,7 +16188,7 @@ build_lflows_thread(void *arg) build_lrouter_flows_for_lb(lb_dps, lsi->lflows, lsi->meter_groups, lsi->lr_datapaths, - lsi->lr_nats, + lsi->lr_sful_table, lsi->features, lsi->svc_monitor_map, &lsi->match, &lsi->actions); @@ -16007,6 +16200,23 @@ build_lflows_thread(void *arg) &lsi->match, &lsi->actions); } } + for (bnum = control->id; + bnum <= lsi->lr_sful_table->entries.mask; + bnum += control->pool->size) + { + LR_STATEFUL_TABLE_FOR_EACH_IN_P (lr_sful_rec, bnum, + lsi->lr_sful_table) { + if (stop_parallel_processing()) { + return NULL; + } + build_lr_stateful_flows(lr_sful_rec, + lsi->lflows, lsi->ls_ports, + lsi->lr_ports, &lsi->match, + &lsi->actions, + lsi->meter_groups, + lsi->features); + } + } for (bnum = control->id; bnum <= lsi->igmp_groups->mask; bnum += control->pool->size) @@ -16066,7 +16276,6 @@ build_lswitch_and_lrouter_flows(const struct ovn_datapaths *ls_datapaths, const struct hmap *ls_ports, const struct hmap *lr_ports, const struct ls_port_group_table *ls_pgs, - const struct lr_nat_table *lr_nats, const struct lr_stateful_table *lr_sful_table, struct hmap *lflows, struct hmap *igmp_groups, @@ -16097,7 +16306,6 @@ build_lswitch_and_lrouter_flows(const struct ovn_datapaths *ls_datapaths, lsiv[index].ls_ports = ls_ports; lsiv[index].lr_ports = lr_ports; lsiv[index].ls_port_groups = ls_pgs; - lsiv[index].lr_nats = lr_nats; lsiv[index].lr_sful_table = lr_sful_table; lsiv[index].igmp_groups = igmp_groups; lsiv[index].meter_groups = meter_groups; @@ -16123,17 +16331,18 @@ build_lswitch_and_lrouter_flows(const struct ovn_datapaths *ls_datapaths, } free(lsiv); } else { + const struct lr_stateful_record *lr_sful_rec; + struct ovn_igmp_group *igmp_group; + struct ovn_lb_datapaths *lb_dps; struct ovn_datapath *od; struct ovn_port *op; - struct ovn_lb_datapaths *lb_dps; - struct ovn_igmp_group *igmp_group; + struct lswitch_flow_build_info lsi = { .ls_datapaths = ls_datapaths, .lr_datapaths = lr_datapaths, .ls_ports = ls_ports, .lr_ports = lr_ports, .ls_port_groups = ls_pgs, - .lr_nats = lr_nats, .lr_sful_table = lr_sful_table, .lflows = lflows, .igmp_groups = igmp_groups, @@ -16162,14 +16371,21 @@ build_lswitch_and_lrouter_flows(const struct ovn_datapaths *ls_datapaths, HMAP_FOR_EACH (op, key_node, ls_ports) { build_lswitch_and_lrouter_iterate_by_lsp(op, lsi.ls_ports, lsi.lr_ports, - lsi.lr_nats, - lsi.lr_sful_table, lsi.meter_groups, - &lsi.match, &lsi.actions, + &lsi.match, + &lsi.actions, lsi.lflows); + build_lbnat_lflows_iterate_by_lsp(op, lsi.lr_sful_table, lsi.lr_ports, + &lsi.match, &lsi.actions, + lsi.lflows); } HMAP_FOR_EACH (op, key_node, lr_ports) { build_lswitch_and_lrouter_iterate_by_lrp(op, &lsi); + build_lbnat_lflows_iterate_by_lrp(op, lsi.lr_sful_table, + lsi.meter_groups, + &lsi.match, + &lsi.actions, + lsi.lflows); } stopwatch_stop(LFLOWS_PORTS_STOPWATCH_NAME, time_msec()); stopwatch_start(LFLOWS_LBS_STOPWATCH_NAME, time_msec()); @@ -16180,7 +16396,7 @@ build_lswitch_and_lrouter_flows(const struct ovn_datapaths *ls_datapaths, build_lrouter_defrag_flows_for_lb(lb_dps, lsi.lflows, lsi.lr_datapaths, &lsi.match); build_lrouter_flows_for_lb(lb_dps, lsi.lflows, lsi.meter_groups, - lsi.lr_datapaths, lsi.lr_nats, + lsi.lr_datapaths, lsi.lr_sful_table, lsi.features, lsi.svc_monitor_map, &lsi.match, &lsi.actions); build_lswitch_flows_for_lb(lb_dps, lsi.lflows, lsi.meter_groups, @@ -16189,6 +16405,14 @@ build_lswitch_and_lrouter_flows(const struct ovn_datapaths *ls_datapaths, &lsi.match, &lsi.actions); } stopwatch_stop(LFLOWS_LBS_STOPWATCH_NAME, time_msec()); + + LR_STATEFUL_TABLE_FOR_EACH (lr_sful_rec, lr_sful_table) { + build_lr_stateful_flows(lr_sful_rec, lsi.lflows, + lsi.ls_ports, lsi.lr_ports, &lsi.match, + &lsi.actions, lsi.meter_groups, + lsi.features); + } + stopwatch_start(LFLOWS_IGMP_STOPWATCH_NAME, time_msec()); HMAP_FOR_EACH (igmp_group, hmap_node, igmp_groups) { build_lswitch_ip_mcast_igmp_mld(igmp_group, @@ -16284,7 +16508,6 @@ void build_lflows(struct ovsdb_idl_txn *ovnsb_txn, input_data->ls_ports, input_data->lr_ports, input_data->ls_port_groups, - input_data->lr_nats, input_data->lr_sful_table, lflows, &igmp_groups, @@ -16763,11 +16986,22 @@ lflow_handle_northd_port_changes(struct ovsdb_idl_txn *ovnsb_txn, struct ds actions = DS_EMPTY_INITIALIZER; build_lswitch_and_lrouter_iterate_by_lsp(op, lflow_input->ls_ports, lflow_input->lr_ports, - lflow_input->lr_nats, - lflow_input->lr_sful_table, lflow_input->meter_groups, &match, &actions, lflows); + + if (lsp_is_router(op->nbsp) && op->peer && op->peer->od->nbr) { + const struct lr_stateful_record *lr_sful_rec; + lr_sful_rec = lr_stateful_table_find_by_index( + lflow_input->lr_sful_table, op->peer->od->index); + ovs_assert(lr_sful_rec); + + build_lsp_lflows_for_lbnats(op, op->peer, lr_sful_rec, + lflow_input->lr_sful_table, + lflow_input->lr_ports, + lflows, &match, &actions); + } + ds_destroy(&match); ds_destroy(&actions); @@ -16801,8 +17035,6 @@ lflow_handle_northd_port_changes(struct ovsdb_idl_txn *ovnsb_txn, struct ds actions = DS_EMPTY_INITIALIZER; build_lswitch_and_lrouter_iterate_by_lsp(op, lflow_input->ls_ports, lflow_input->lr_ports, - lflow_input->lr_nats, - lflow_input->lr_sful_table, lflow_input->meter_groups, &match, &actions, lflows); ds_destroy(&match); diff --git a/northd/northd.h b/northd/northd.h index 6d64fff626..612b292da1 100644 --- a/northd/northd.h +++ b/northd/northd.h @@ -179,7 +179,6 @@ struct lflow_input { const struct hmap *ls_ports; const struct hmap *lr_ports; const struct ls_port_group_table *ls_port_groups; - const struct lr_nat_table *lr_nats; const struct lr_stateful_table *lr_sful_table; const struct shash *meter_groups; const struct hmap *lb_datapaths_map; From patchwork Tue Nov 28 02:36:10 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Numan Siddique X-Patchwork-Id: 1869078 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::133; helo=smtp2.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp2.osuosl.org (smtp2.osuosl.org [IPv6:2605:bc80:3010::133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4SfRRw3MRWz1yRy for ; Tue, 28 Nov 2023 13:37:00 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id EB9D740A18; Tue, 28 Nov 2023 02:36:57 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org EB9D740A18 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id kv2vYwFvFhGE; Tue, 28 Nov 2023 02:36:52 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp2.osuosl.org (Postfix) with ESMTPS id 3E47D40A2F; Tue, 28 Nov 2023 02:36:50 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 3E47D40A2F Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 14EE0C0DCE; Tue, 28 Nov 2023 02:36:50 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by lists.linuxfoundation.org (Postfix) with ESMTP id 12DB6C0DCE for ; Tue, 28 Nov 2023 02:36:48 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id A885C81AD7 for ; Tue, 28 Nov 2023 02:36:40 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org A885C81AD7 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 0wo6j-d7Qa3B for ; Tue, 28 Nov 2023 02:36:34 +0000 (UTC) Received: from relay6-d.mail.gandi.net (relay6-d.mail.gandi.net [217.70.183.198]) by smtp1.osuosl.org (Postfix) with ESMTPS id 3A97281BCB for ; Tue, 28 Nov 2023 02:36:33 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 3A97281BCB Received: by mail.gandi.net (Postfix) with ESMTPSA id 33C7AC0004; Tue, 28 Nov 2023 02:36:29 +0000 (UTC) From: numans@ovn.org To: dev@openvswitch.org Date: Mon, 27 Nov 2023 21:36:10 -0500 Message-ID: <20231128023610.569811-1-numans@ovn.org> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20231128023410.569539-1-numans@ovn.org> References: <20231128023410.569539-1-numans@ovn.org> MIME-Version: 1.0 X-GND-Sasl: numans@ovn.org Subject: [ovs-dev] [PATCH ovn v3 07/16] northd: Add a new node 'ls_stateful'. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Numan Siddique This new engine now maintains the load balancer and ACL data of a logical switch which was earlier part of northd engine node data. The main inputs to this engine are: - northd node - Port group node A record for each logical switch is maintained in the 'ls_statefuls' hmap table and this record stores the below data which was earlier part of 'struct ovn_datapath'. - bool has_stateful_acl; - bool has_lb_vip; - bool has_acls; - uint64_t max_acl_tier; This engine node becomes an input to 'lflow' node. If a logical switch is configured with - DHCP for all or some of its ports - stateful ACLs and/or load balancers ovn-northd was adding flows to commit the DHCP response generated by ovn-controller to conntrack. With this patch we don't commit the DHCP response to conntrack anymore for the following reasons: 1. There is no need to actually commit the response. 2. Since stateful information is moved to 'ls_stateful' node, it becomes ineffecient to access ls_stateful's data ('has_lb_vip' and 'has_acls') in build_dhcpv4_options_flows(). Signed-off-by: Numan Siddique --- lib/stopwatch-names.h | 1 + northd/automake.mk | 2 + northd/en-lflow.c | 4 + northd/en-lr-stateful.c | 6 +- northd/en-lr-stateful.h | 2 + northd/en-ls-stateful.c | 445 +++++++++++++++++++++++++++++++++++++++ northd/en-ls-stateful.h | 81 +++++++ northd/en-northd.c | 2 +- northd/en-port-group.h | 3 + northd/inc-proc-northd.c | 7 + northd/northd.c | 329 ++++++++++++++++------------- northd/northd.h | 29 ++- northd/ovn-northd.c | 1 + 13 files changed, 758 insertions(+), 154 deletions(-) create mode 100644 northd/en-ls-stateful.c create mode 100644 northd/en-ls-stateful.h diff --git a/lib/stopwatch-names.h b/lib/stopwatch-names.h index 63a04db9ca..374fd29b16 100644 --- a/lib/stopwatch-names.h +++ b/lib/stopwatch-names.h @@ -34,5 +34,6 @@ #define SYNC_METERS_RUN_STOPWATCH_NAME "sync_meters_run" #define LR_NAT_RUN_STOPWATCH_NAME "lr_nat_run" #define LR_STATEFUL_RUN_STOPWATCH_NAME "lr_stateful" +#define LS_STATEFUL_RUN_STOPWATCH_NAME "ls_stateful" #endif diff --git a/northd/automake.mk b/northd/automake.mk index 3666cd452c..44140ce203 100644 --- a/northd/automake.mk +++ b/northd/automake.mk @@ -28,6 +28,8 @@ northd_ovn_northd_SOURCES = \ northd/en-lr-nat.h \ northd/en-lr-stateful.c \ northd/en-lr-stateful.h \ + northd/en-ls-stateful.c \ + northd/en-ls-stateful.h \ northd/inc-proc-northd.c \ northd/inc-proc-northd.h \ northd/ipam.c \ diff --git a/northd/en-lflow.c b/northd/en-lflow.c index ef5aa11c17..83f11850ce 100644 --- a/northd/en-lflow.c +++ b/northd/en-lflow.c @@ -21,6 +21,7 @@ #include "en-lflow.h" #include "en-lr-nat.h" #include "en-lr-stateful.h" +#include "en-ls-stateful.h" #include "en-northd.h" #include "en-meters.h" @@ -44,6 +45,8 @@ lflow_get_input_data(struct engine_node *node, engine_get_input_data("sync_meters", node); struct ed_type_lr_stateful *lr_sful_data = engine_get_input_data("lr_stateful", node); + struct ed_type_ls_stateful *ls_sful_data = + engine_get_input_data("ls_stateful", node); lflow_input->nbrec_bfd_table = EN_OVSDB_GET(engine_get_input("NB_bfd", node)); @@ -67,6 +70,7 @@ lflow_get_input_data(struct engine_node *node, lflow_input->lr_ports = &northd_data->lr_ports; lflow_input->ls_port_groups = &pg_data->ls_port_groups; lflow_input->lr_sful_table = &lr_sful_data->lr_sful_table; + lflow_input->ls_sful_table = &ls_sful_data->ls_sful_table; lflow_input->meter_groups = &sync_meters_data->meter_groups; lflow_input->lb_datapaths_map = &northd_data->lb_datapaths_map; lflow_input->svc_monitor_map = &northd_data->svc_monitor_map; diff --git a/northd/en-lr-stateful.c b/northd/en-lr-stateful.c index bd03c586da..a54749ad93 100644 --- a/northd/en-lr-stateful.c +++ b/northd/en-lr-stateful.c @@ -297,7 +297,9 @@ lr_stateful_lb_data_handler(struct engine_node *node, void *data_) /* For all the modified lr_stateful records (re)build the * vip nats. */ HMAPX_FOR_EACH (hmapx_node, &data->trk_data.crupdated) { - lr_stateful_build_vip_nats(hmapx_node->data); + lr_sful_rec = hmapx_node->data; + lr_stateful_build_vip_nats(lr_sful_rec); + lr_sful_rec->has_lb_vip = od_has_lb_vip(lr_sful_rec->od); } engine_set_node_state(node, EN_UPDATED); @@ -516,6 +518,8 @@ lr_stateful_record_init(struct lr_stateful_record *lr_sful_rec, if (!nbr->n_nat) { lr_stateful_build_vip_nats(lr_sful_rec); } + + lr_sful_rec->has_lb_vip = od_has_lb_vip(lr_sful_rec->od); } static struct lr_stateful_input diff --git a/northd/en-lr-stateful.h b/northd/en-lr-stateful.h index 4360419788..0571e057f8 100644 --- a/northd/en-lr-stateful.h +++ b/northd/en-lr-stateful.h @@ -49,6 +49,8 @@ struct lr_stateful_record { const struct ovn_datapath *od; const struct lr_nat_record *lrnat_rec; + bool has_lb_vip; + /* Load Balancer vIPs relevant for this datapath. */ struct ovn_lb_ip_set *lb_ips; diff --git a/northd/en-ls-stateful.c b/northd/en-ls-stateful.c new file mode 100644 index 0000000000..117def531b --- /dev/null +++ b/northd/en-ls-stateful.c @@ -0,0 +1,445 @@ +/* + * Copyright (c) 2023, Red Hat, Inc. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at: + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#include + +#include +#include +#include + +/* OVS includes */ +#include "include/openvswitch/hmap.h" +#include "lib/bitmap.h" +#include "lib/socket-util.h" +#include "lib/uuidset.h" +#include "openvswitch/util.h" +#include "openvswitch/vlog.h" +#include "stopwatch.h" + +/* OVN includes */ +#include "en-lb-data.h" +#include "en-ls-stateful.h" +#include "en-port-group.h" +#include "lib/inc-proc-eng.h" +#include "lib/lb.h" +#include "lib/ovn-nb-idl.h" +#include "lib/ovn-sb-idl.h" +#include "lib/ovn-util.h" +#include "lib/stopwatch-names.h" +#include "northd.h" + +VLOG_DEFINE_THIS_MODULE(en_ls_stateful); + +/* Static function declarations. */ +static void ls_stateful_table_init(struct ls_stateful_table *); +static void ls_stateful_table_clear(struct ls_stateful_table *); +static void ls_stateful_table_destroy(struct ls_stateful_table *); +static struct ls_stateful_record *ls_stateful_table_find_( + const struct ls_stateful_table *, const struct nbrec_logical_switch *); +static void ls_stateful_table_build(struct ls_stateful_table *, + const struct ovn_datapaths *ls_datapaths, + const struct ls_port_group_table *); + +static struct ls_stateful_input ls_stateful_get_input_data( + struct engine_node *); + +static struct ls_stateful_record *ls_stateful_record_create( + struct ls_stateful_table *, + const struct ovn_datapath *, + const struct ls_port_group_table *); +static void ls_stateful_record_destroy(struct ls_stateful_record *); +static void ls_stateful_record_init( + struct ls_stateful_record *, + const struct ovn_datapath *, + const struct ls_port_group *, + const struct ls_port_group_table *); +static void ls_stateful_record_reinit( + struct ls_stateful_record *, + const struct ls_port_group *, + const struct ls_port_group_table *); +static bool ls_has_lb_vip(const struct ovn_datapath *); +static void ls_stateful_record_set_acl_flags(struct ls_stateful_record *, + const struct ovn_datapath *, + const struct ls_port_group *, + const struct ls_port_group_table *); +static bool ls_stateful_record_set_acl_flags_(struct ls_stateful_record *, + struct nbrec_acl **, + size_t n_acls); +static struct ls_stateful_input ls_stateful_get_input_data( + struct engine_node *); + +struct ls_stateful_input { + const struct ls_port_group_table *ls_port_groups; + const struct ovn_datapaths *ls_datapaths; +}; + +/* public functions. */ +const struct ls_stateful_record * +ls_stateful_table_find( + const struct ls_stateful_table *table, + const struct nbrec_logical_switch *nbs) +{ + return ls_stateful_table_find_(table, nbs); +} + +void * +en_ls_stateful_init(struct engine_node *node OVS_UNUSED, + struct engine_arg *arg OVS_UNUSED) +{ + struct ed_type_ls_stateful *data = xzalloc(sizeof *data); + ls_stateful_table_init(&data->ls_sful_table); + hmapx_init(&data->trk_data.crupdated); + return data; +} + +void +en_ls_stateful_cleanup(void *data_) +{ + struct ed_type_ls_stateful *data = + (struct ed_type_ls_stateful *) data_; + ls_stateful_table_destroy(&data->ls_sful_table); + hmapx_destroy(&data->trk_data.crupdated); +} + +void +en_ls_stateful_clear_tracked_data(void *data_) +{ + struct ed_type_ls_stateful *data = + (struct ed_type_ls_stateful *) data_; + + hmapx_clear(&data->trk_data.crupdated); +} + +void +en_ls_stateful_run(struct engine_node *node, void *data_) +{ + struct ls_stateful_input input_data = ls_stateful_get_input_data(node); + struct ed_type_ls_stateful *data = data_; + + stopwatch_start(LS_STATEFUL_RUN_STOPWATCH_NAME, time_msec()); + + ls_stateful_table_clear(&data->ls_sful_table); + ls_stateful_table_build(&data->ls_sful_table, input_data.ls_datapaths, + input_data.ls_port_groups); + + stopwatch_stop(LS_STATEFUL_RUN_STOPWATCH_NAME, time_msec()); + engine_set_node_state(node, EN_UPDATED); +} + +/* Handler functions. */ +bool +ls_stateful_northd_handler(struct engine_node *node, void *data_) +{ + struct northd_data *northd_data = engine_get_input_data("northd", node); + if (!northd_has_tracked_data(&northd_data->trk_data)) { + return false; + } + + if (!northd_has_ls_lbs_in_tracked_data(&northd_data->trk_data) && + !northd_has_ls_acls_in_tracked_data(&northd_data->trk_data)) { + return true; + } + + struct northd_tracked_data *nd_changes = &northd_data->trk_data; + struct ls_stateful_input input_data = ls_stateful_get_input_data(node); + struct ls_stateful_record *ls_stateful_rec; + struct ed_type_ls_stateful *data = data_; + const struct ovn_datapath *od; + struct hmapx_node *hmapx_node; + + struct hmapx changed_sful_od = HMAPX_INITIALIZER(&changed_sful_od); + HMAPX_FOR_EACH (hmapx_node, &nd_changes->ls_with_changed_lbs) { + hmapx_add(&changed_sful_od, hmapx_node->data); + } + + HMAPX_FOR_EACH (hmapx_node, &nd_changes->ls_with_changed_acls) { + hmapx_add(&changed_sful_od, hmapx_node->data); + } + + HMAPX_FOR_EACH (hmapx_node, &changed_sful_od) { + od = hmapx_node->data; + + ls_stateful_rec = ls_stateful_table_find_(&data->ls_sful_table, + od->nbs); + ovs_assert(ls_stateful_rec); + ls_stateful_record_reinit(ls_stateful_rec, NULL, + input_data.ls_port_groups); + + /* Add the ls_stateful_rec to the tracking data. */ + hmapx_add(&data->trk_data.crupdated, ls_stateful_rec); + } + + if (ls_stateful_has_tracked_data(&data->trk_data)) { + engine_set_node_state(node, EN_UPDATED); + } + + hmapx_destroy(&changed_sful_od); + + return true; +} + +bool +ls_stateful_port_group_handler(struct engine_node *node, void *data_) +{ + struct port_group_data *pg_data = + engine_get_input_data("port_group", node); + + if (pg_data->ls_port_groups_sets_changed) { + return false; + } + + /* port_group engine node doesn't provide the tracking data yet. + * Loop through all the ls port groups and update the ls_stateful_rec. + * This is still better than returning false. */ + struct ls_stateful_input input_data = ls_stateful_get_input_data(node); + struct ed_type_ls_stateful *data = data_; + const struct ls_port_group *ls_pg; + + LS_PORT_GROUP_TABLE_FOR_EACH (ls_pg, input_data.ls_port_groups) { + struct ls_stateful_record *ls_sful_rec = + ls_stateful_table_find_(&data->ls_sful_table, ls_pg->nbs); + ovs_assert(ls_sful_rec); + + bool had_stateful_acl = ls_sful_rec->has_stateful_acl; + uint64_t max_acl_tier = ls_sful_rec->max_acl_tier; + bool had_acls = ls_sful_rec->has_acls; + bool modified = false; + + ls_stateful_record_reinit(ls_sful_rec, ls_pg, + input_data.ls_port_groups); + + if ((had_stateful_acl != ls_sful_rec->has_stateful_acl) + || (had_acls != ls_sful_rec->has_acls) + || max_acl_tier != ls_sful_rec->max_acl_tier) { + modified = true; + } + + if (modified) { + /* Add the ls_sful_rec to the tracking data. */ + hmapx_add(&data->trk_data.crupdated, ls_sful_rec); + } + } + + if (ls_stateful_has_tracked_data(&data->trk_data)) { + engine_set_node_state(node, EN_UPDATED); + } + return true; +} + +/* static functions. */ +static void +ls_stateful_table_init(struct ls_stateful_table *table) +{ + *table = (struct ls_stateful_table) { + .entries = HMAP_INITIALIZER(&table->entries), + }; +} + +static void +ls_stateful_table_destroy(struct ls_stateful_table *table) +{ + ls_stateful_table_clear(table); + hmap_destroy(&table->entries); +} + +static void +ls_stateful_table_clear(struct ls_stateful_table *table) +{ + struct ls_stateful_record *ls_stateful_rec; + HMAP_FOR_EACH_POP (ls_stateful_rec, key_node, &table->entries) { + ls_stateful_record_destroy(ls_stateful_rec); + } +} + +static void +ls_stateful_table_build(struct ls_stateful_table *table, + const struct ovn_datapaths *ls_datapaths, + const struct ls_port_group_table *ls_pgs) +{ + const struct ovn_datapath *od; + HMAP_FOR_EACH (od, key_node, &ls_datapaths->datapaths) { + ls_stateful_record_create(table, od, ls_pgs); + } +} + +struct ls_stateful_record * +ls_stateful_table_find_(const struct ls_stateful_table *table, + const struct nbrec_logical_switch *nbs) +{ + struct ls_stateful_record *rec; + + HMAP_FOR_EACH_WITH_HASH (rec, key_node, + uuid_hash(&nbs->header_.uuid), &table->entries) { + if (nbs == rec->od->nbs) { + return rec; + } + } + return NULL; +} + +static struct ls_stateful_record * +ls_stateful_record_create(struct ls_stateful_table *table, + const struct ovn_datapath *od, + const struct ls_port_group_table *ls_pgs) +{ + struct ls_stateful_record *ls_sful_rec = xzalloc(sizeof *ls_sful_rec); + ls_sful_rec->od = od; + ls_stateful_record_init(ls_sful_rec, od, NULL, ls_pgs); + + hmap_insert(&table->entries, &ls_sful_rec->key_node, + uuid_hash(&ls_sful_rec->od->nbs->header_.uuid)); + + return ls_sful_rec; +} + +static void +ls_stateful_record_destroy(struct ls_stateful_record *ls_sful_rec) +{ + free(ls_sful_rec); +} + +static void +ls_stateful_record_init(struct ls_stateful_record *ls_sful_rec, + const struct ovn_datapath *od, + const struct ls_port_group *ls_pg, + const struct ls_port_group_table *ls_pgs) +{ + ls_sful_rec->has_lb_vip = ls_has_lb_vip(od); + ls_stateful_record_set_acl_flags(ls_sful_rec, od, ls_pg, ls_pgs); +} + +static void +ls_stateful_record_reinit(struct ls_stateful_record *ls_sful_rec, + const struct ls_port_group *ls_pg, + const struct ls_port_group_table *ls_pgs) +{ + ls_stateful_record_init(ls_sful_rec, ls_sful_rec->od, ls_pg, ls_pgs); +} + +static bool +lb_has_vip(const struct nbrec_load_balancer *lb) +{ + return !smap_is_empty(&lb->vips); +} + +static bool +lb_group_has_vip(const struct nbrec_load_balancer_group *lb_group) +{ + for (size_t i = 0; i < lb_group->n_load_balancer; i++) { + if (lb_has_vip(lb_group->load_balancer[i])) { + return true; + } + } + return false; +} + +static bool +ls_has_lb_vip(const struct ovn_datapath *od) +{ + for (size_t i = 0; i < od->nbs->n_load_balancer; i++) { + if (lb_has_vip(od->nbs->load_balancer[i])) { + return true; + } + } + + for (size_t i = 0; i < od->nbs->n_load_balancer_group; i++) { + if (lb_group_has_vip(od->nbs->load_balancer_group[i])) { + return true; + } + } + return false; +} + +static void +ls_stateful_record_set_acl_flags(struct ls_stateful_record *ls_sful_rec, + const struct ovn_datapath *od, + const struct ls_port_group *ls_pg, + const struct ls_port_group_table *ls_pgs) +{ + ls_sful_rec->has_stateful_acl = false; + ls_sful_rec->max_acl_tier = 0; + ls_sful_rec->has_acls = false; + + if (ls_stateful_record_set_acl_flags_(ls_sful_rec, od->nbs->acls, + od->nbs->n_acls)) { + return; + } + + if (!ls_pg) { + ls_pg = ls_port_group_table_find(ls_pgs, od->nbs); + } + + if (!ls_pg) { + return; + } + + const struct ls_port_group_record *ls_pg_rec; + HMAP_FOR_EACH (ls_pg_rec, key_node, &ls_pg->nb_pgs) { + if (ls_stateful_record_set_acl_flags_(ls_sful_rec, + ls_pg_rec->nb_pg->acls, + ls_pg_rec->nb_pg->n_acls)) { + return; + } + } +} + +static bool +ls_stateful_record_set_acl_flags_(struct ls_stateful_record *ls_sful_rec, + struct nbrec_acl **acls, + size_t n_acls) +{ + /* A true return indicates that there are no possible ACL flags + * left to set on ls_stateful record. A false return indicates that + * further ACLs should be explored in case more flags need to be + * set on ls_stateful record. + */ + if (!n_acls) { + return false; + } + + ls_sful_rec->has_acls = true; + for (size_t i = 0; i < n_acls; i++) { + const struct nbrec_acl *acl = acls[i]; + if (acl->tier > ls_sful_rec->max_acl_tier) { + ls_sful_rec->max_acl_tier = acl->tier; + } + if (!ls_sful_rec->has_stateful_acl + && !strcmp(acl->action, "allow-related")) { + ls_sful_rec->has_stateful_acl = true; + } + if (ls_sful_rec->has_stateful_acl && + ls_sful_rec->max_acl_tier == + nbrec_acl_col_tier.type.value.integer.max) { + return true; + } + } + + return false; +} + +static struct ls_stateful_input +ls_stateful_get_input_data(struct engine_node *node) +{ + const struct northd_data *northd_data = + engine_get_input_data("northd", node); + const struct port_group_data *pg_data = + engine_get_input_data("port_group", node); + + return (struct ls_stateful_input) { + .ls_port_groups = &pg_data->ls_port_groups, + .ls_datapaths = &northd_data->ls_datapaths, + }; +} diff --git a/northd/en-ls-stateful.h b/northd/en-ls-stateful.h new file mode 100644 index 0000000000..e409bb0141 --- /dev/null +++ b/northd/en-ls-stateful.h @@ -0,0 +1,81 @@ +/* + * Copyright (c) 2023, Red Hat, Inc. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at: + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +#ifndef EN_LS_STATEFUL_H +#define EN_LS_STATEFUL_H 1 + +#include + +/* OVS includes. */ +#include "lib/hmapx.h" +#include "openvswitch/hmap.h" +#include "sset.h" + +/* OVN includes. */ +#include "lib/inc-proc-eng.h" +#include "lib/lb.h" +#include "lib/ovn-nb-idl.h" +#include "lib/ovn-sb-idl.h" +#include "lib/ovn-util.h" +#include "lib/stopwatch-names.h" + +struct ls_stateful_record { + struct hmap_node key_node; + + const struct ovn_datapath *od; + bool has_stateful_acl; + bool has_lb_vip; + bool has_acls; + uint64_t max_acl_tier; +}; + +struct ls_stateful_table { + struct hmap entries; +}; + +#define LS_STATEFUL_TABLE_FOR_EACH(LS_STATEFUL_REC, TABLE) \ + HMAP_FOR_EACH (LS_STATEFUL_REC, key_node, &(TABLE)->entries) + +#define LS_STATEFUL_TABLE_FOR_EACH_IN_P(LS_STATEFUL_REC, JOBID, TABLE) \ + HMAP_FOR_EACH_IN_PARALLEL (LS_STATEFUL_REC, key_node, JOBID, \ + &(TABLE)->entries) + +struct ls_stateful_tracked_data { + /* Created or updated logical switch with LB and ACL data. */ + struct hmapx crupdated; /* Stores 'struct ls_stateful_record'. */ +}; + +struct ed_type_ls_stateful { + struct ls_stateful_table ls_sful_table; + struct ls_stateful_tracked_data trk_data; +}; + +void *en_ls_stateful_init(struct engine_node *, struct engine_arg *); +void en_ls_stateful_cleanup(void *data); +void en_ls_stateful_clear_tracked_data(void *data); +void en_ls_stateful_run(struct engine_node *, void *data); + +bool ls_stateful_northd_handler(struct engine_node *, void *data); +bool ls_stateful_port_group_handler(struct engine_node *, void *data); + +const struct ls_stateful_record *ls_stateful_table_find( + const struct ls_stateful_table *, const struct nbrec_logical_switch *); + +static inline bool +ls_stateful_has_tracked_data(struct ls_stateful_tracked_data *trk_data) { + return !hmapx_is_empty(&trk_data->crupdated); +} + +#endif /* EN_LS_STATEFUL_H */ diff --git a/northd/en-northd.c b/northd/en-northd.c index 546397f3dc..5143603f39 100644 --- a/northd/en-northd.c +++ b/northd/en-northd.c @@ -171,7 +171,7 @@ northd_nb_logical_switch_handler(struct engine_node *node, return false; } - if (northd_has_lsps_in_tracked_data(&nd->trk_data)) { + if (northd_has_tracked_data(&nd->trk_data)) { engine_set_node_state(node, EN_UPDATED); } diff --git a/northd/en-port-group.h b/northd/en-port-group.h index 3b28a23694..54014062ce 100644 --- a/northd/en-port-group.h +++ b/northd/en-port-group.h @@ -48,6 +48,9 @@ struct ls_port_group_record { struct sset ports; /* Subset of 'nb_pg' ports in this record. */ }; +#define LS_PORT_GROUP_TABLE_FOR_EACH(LS_PG, TABLE) \ + HMAP_FOR_EACH (LS_PG, key_node, &(TABLE)->entries) + void ls_port_group_table_init(struct ls_port_group_table *); void ls_port_group_table_clear(struct ls_port_group_table *); void ls_port_group_table_destroy(struct ls_port_group_table *); diff --git a/northd/inc-proc-northd.c b/northd/inc-proc-northd.c index adb38dde78..9ce4279ee8 100644 --- a/northd/inc-proc-northd.c +++ b/northd/inc-proc-northd.c @@ -33,6 +33,7 @@ #include "en-lb-data.h" #include "en-lr-stateful.h" #include "en-lr-nat.h" +#include "en-ls-stateful.h" #include "en-northd.h" #include "en-lflow.h" #include "en-northd-output.h" @@ -150,6 +151,7 @@ static ENGINE_NODE(sync_to_sb_pb, "sync_to_sb_pb"); static ENGINE_NODE_WITH_CLEAR_TRACK_DATA(lb_data, "lb_data"); static ENGINE_NODE_WITH_CLEAR_TRACK_DATA(lr_nat, "lr_nat"); static ENGINE_NODE_WITH_CLEAR_TRACK_DATA(lr_stateful, "lr_stateful"); +static ENGINE_NODE_WITH_CLEAR_TRACK_DATA(ls_stateful, "ls_stateful"); void inc_proc_northd_init(struct ovsdb_idl_loop *nb, struct ovsdb_idl_loop *sb) @@ -200,6 +202,10 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, engine_add_input(&en_lr_stateful, &en_lb_data, lr_stateful_lb_data_handler); + engine_add_input(&en_ls_stateful, &en_northd, ls_stateful_northd_handler); + engine_add_input(&en_ls_stateful, &en_port_group, + ls_stateful_port_group_handler); + engine_add_input(&en_mac_binding_aging, &en_nb_nb_global, NULL); engine_add_input(&en_mac_binding_aging, &en_sb_mac_binding, NULL); engine_add_input(&en_mac_binding_aging, &en_northd, NULL); @@ -222,6 +228,7 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, engine_add_input(&en_lflow, &en_sb_multicast_group, NULL); engine_add_input(&en_lflow, &en_sb_igmp_group, NULL); engine_add_input(&en_lflow, &en_lr_stateful, NULL); + engine_add_input(&en_lflow, &en_ls_stateful, NULL); engine_add_input(&en_lflow, &en_northd, lflow_northd_handler); engine_add_input(&en_lflow, &en_port_group, lflow_port_group_handler); diff --git a/northd/northd.c b/northd/northd.c index 176433a6c5..c8e2bd4790 100644 --- a/northd/northd.c +++ b/northd/northd.c @@ -45,6 +45,7 @@ #include "en-lb-data.h" #include "en-lr-nat.h" #include "en-lr-stateful.h" +#include "en-ls-stateful.h" #include "lib/ovn-parallel-hmap.h" #include "ovn/actions.h" #include "ovn/features.h" @@ -575,7 +576,7 @@ lb_group_has_vip(const struct nbrec_load_balancer_group *lb_group) } static bool -ls_has_lb_vip(struct ovn_datapath *od) +ls_has_lb_vip(const struct ovn_datapath *od) { for (size_t i = 0; i < od->nbs->n_load_balancer; i++) { if (lb_has_vip(od->nbs->load_balancer[i])) { @@ -592,7 +593,7 @@ ls_has_lb_vip(struct ovn_datapath *od) } static bool -lr_has_lb_vip(struct ovn_datapath *od) +lr_has_lb_vip(const struct ovn_datapath *od) { for (size_t i = 0; i < od->nbr->n_load_balancer; i++) { if (lb_has_vip(od->nbr->load_balancer[i])) { @@ -608,13 +609,13 @@ lr_has_lb_vip(struct ovn_datapath *od) return false; } -static void -init_lb_for_datapath(struct ovn_datapath *od) +bool +od_has_lb_vip(const struct ovn_datapath *od) { if (od->nbs) { - od->has_lb_vip = ls_has_lb_vip(od); + return ls_has_lb_vip(od); } else { - od->has_lb_vip = lr_has_lb_vip(od); + return lr_has_lb_vip(od); } } @@ -1058,7 +1059,6 @@ join_datapaths(const struct nbrec_logical_switch_table *nbrec_ls_table, init_ipam_info_for_datapath(od); init_mcast_info_for_datapath(od); - init_lb_for_datapath(od); } const struct nbrec_logical_router *nbr; @@ -1089,7 +1089,6 @@ join_datapaths(const struct nbrec_logical_switch_table *nbrec_ls_table, ovs_list_push_back(nb_only, &od->list); } init_mcast_info_for_datapath(od); - init_lb_for_datapath(od); if (smap_get(&od->nbr->options, "chassis")) { od->is_gw_router = true; } @@ -2570,7 +2569,8 @@ get_nat_addresses(const struct ovn_port *op, size_t *n, bool routable_only, size_t n_nats = 0; struct eth_addr mac; if (!op || !op->nbrp || !op->od || !op->od->nbr - || (!op->od->nbr->n_nat && !op->od->has_lb_vip) + || (!op->od->nbr->n_nat && (!lr_sful_rec + || !lr_sful_rec->has_lb_vip)) || !eth_addr_from_string(op->nbrp->mac, &mac)) { *n = n_nats; return NULL; @@ -3817,7 +3817,7 @@ build_lrouter_lbs_check(const struct ovn_datapaths *lr_datapaths) HMAP_FOR_EACH (od, key_node, &lr_datapaths->datapaths) { ovs_assert(od->nbr); - if (od->has_lb_vip && od->n_l3dgw_ports > 1 + if (od_has_lb_vip(od) && od->n_l3dgw_ports > 1 && !smap_get(&od->nbr->options, "chassis")) { static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 1); VLOG_WARN_RL(&rl, "Load-balancers are configured on logical " @@ -4737,6 +4737,8 @@ destroy_northd_data_tracked_changes(struct northd_data *nd) destroy_tracked_ovn_ports(&trk_changes->trk_lsps); destroy_tracked_lbs(&trk_changes->trk_lbs); hmapx_clear(&trk_changes->lr_with_changed_nats); + hmapx_clear(&trk_changes->ls_with_changed_lbs); + hmapx_clear(&trk_changes->ls_with_changed_acls); nd->trk_data.type = NORTHD_TRACKED_NONE; } @@ -4751,6 +4753,8 @@ init_northd_tracked_data(struct northd_data *nd) hmapx_init(&trk_data->trk_lbs.crupdated); hmapx_init(&trk_data->trk_lbs.deleted); hmapx_init(&trk_data->lr_with_changed_nats); + hmapx_init(&trk_data->ls_with_changed_lbs); + hmapx_init(&trk_data->ls_with_changed_acls); } static void @@ -4764,6 +4768,8 @@ destroy_northd_tracked_data(struct northd_data *nd) hmapx_destroy(&trk_data->trk_lbs.crupdated); hmapx_destroy(&trk_data->trk_lbs.deleted); hmapx_destroy(&trk_data->lr_with_changed_nats); + hmapx_destroy(&trk_data->ls_with_changed_lbs); + hmapx_destroy(&trk_data->ls_with_changed_acls); } /* Check if a changed LSP can be handled incrementally within the I-P engine @@ -4881,6 +4887,7 @@ ls_port_create(struct ovsdb_idl_txn *ovnsb_txn, struct hmap *ls_ports, * - logical switch ports. * - load balancers. * - load balancer groups. + * - ACLs */ static bool ls_changes_can_be_handled( @@ -5105,6 +5112,25 @@ fail: return false; } +static bool +is_acls_seqno_changed(struct nbrec_acl **nb_acls, size_t n_nb_acls) +{ + for (size_t i = 0; i < n_nb_acls; i++) { + if (nbrec_acl_row_get_seqno(nb_acls[i], + OVSDB_IDL_CHANGE_MODIFY) > 0) { + return true; + } + } + + return false; +} + +static bool +is_ls_acls_changed(const struct nbrec_logical_switch *nbs) { + return (nbrec_logical_switch_is_updated(nbs, NBREC_LOGICAL_SWITCH_COL_ACLS) + || is_acls_seqno_changed(nbs->acls, nbs->n_acls)); +} + /* Return true if changes are handled incrementally, false otherwise. * When there are any changes, try to track what's exactly changed and set * northd_data->trk_data accordingly. @@ -5146,6 +5172,10 @@ northd_handle_ls_changes(struct ovsdb_idl_txn *ovnsb_idl_txn, ni, nd, od, &trk_data->trk_lsps)) { goto fail; } + + if (is_ls_acls_changed(changed_ls)) { + hmapx_add(&trk_data->ls_with_changed_acls, od); + } } if (!hmapx_is_empty(&trk_data->trk_lsps.created) @@ -5154,6 +5184,10 @@ northd_handle_ls_changes(struct ovsdb_idl_txn *ovnsb_idl_txn, trk_data->type |= NORTHD_TRACKED_PORTS; } + if (!hmapx_is_empty(&trk_data->ls_with_changed_acls)) { + trk_data->type |= NORTHD_TRACKED_LS_ACLS; + } + return true; fail: @@ -5445,7 +5479,9 @@ northd_handle_lb_data_changes(struct tracked_lb_data *trk_lb_data, BITMAP_FOR_EACH_1 (index, ods_size(ls_datapaths), lb_dps->nb_ls_map) { od = ls_datapaths->array[index]; - init_lb_for_datapath(od); + + /* Add the ls datapath to the northd tracked data. */ + hmapx_add(&nd_changes->ls_with_changed_lbs, od); } hmap_remove(lb_datapaths_map, &lb_dps->hmap_node); @@ -5525,8 +5561,8 @@ northd_handle_lb_data_changes(struct tracked_lb_data *trk_lb_data, } } - /* Re-evaluate 'od->has_lb_vip' */ - init_lb_for_datapath(od); + /* Add the ls datapath to the northd tracked data. */ + hmapx_add(&nd_changes->ls_with_changed_lbs, od); } LIST_FOR_EACH (codlb, list_node, &trk_lb_data->crupdated_lr_lbs) { @@ -5561,9 +5597,6 @@ northd_handle_lb_data_changes(struct tracked_lb_data *trk_lb_data, hmapx_add(&nd_changes->trk_lbs.crupdated, lb_dps); } } - - /* Re-evaluate 'od->has_lb_vip' */ - init_lb_for_datapath(od); } HMAP_FOR_EACH (clb, hmap_node, &trk_lb_data->crupdated_lbs) { @@ -5576,15 +5609,9 @@ northd_handle_lb_data_changes(struct tracked_lb_data *trk_lb_data, BITMAP_FOR_EACH_1 (index, ods_size(ls_datapaths), lb_dps->nb_ls_map) { od = ls_datapaths->array[index]; - /* Re-evaluate 'od->has_lb_vip' */ - init_lb_for_datapath(od); - } - BITMAP_FOR_EACH_1 (index, ods_size(lr_datapaths), - lb_dps->nb_lr_map) { - od = lr_datapaths->array[index]; - /* Re-evaluate 'od->has_lb_vip' */ - init_lb_for_datapath(od); + /* Add the ls datapath to the northd tracked data. */ + hmapx_add(&nd_changes->ls_with_changed_lbs, od); } } @@ -5606,17 +5633,14 @@ northd_handle_lb_data_changes(struct tracked_lb_data *trk_lb_data, for (size_t i = 0; i < lbgrp_dps->n_lr; i++) { od = lbgrp_dps->lr[i]; ovn_lb_datapaths_add_lr(lb_dps, 1, &od); - - /* Re-evaluate 'od->has_lb_vip' */ - init_lb_for_datapath(od); } for (size_t i = 0; i < lbgrp_dps->n_ls; i++) { od = lbgrp_dps->ls[i]; ovn_lb_datapaths_add_ls(lb_dps, 1, &od); - /* Re-evaluate 'od->has_lb_vip' */ - init_lb_for_datapath(od); + /* Add the ls datapath to the northd tracked data. */ + hmapx_add(&nd_changes->ls_with_changed_lbs, od); } /* Add the lb to the northd tracked data. */ @@ -5629,6 +5653,10 @@ northd_handle_lb_data_changes(struct tracked_lb_data *trk_lb_data, nd_changes->type |= NORTHD_TRACKED_LBS; } + if (!hmapx_is_empty(&nd_changes->ls_with_changed_lbs)) { + nd_changes->type |= NORTHD_TRACKED_LS_LBS; + } + return true; } @@ -6562,63 +6590,6 @@ build_dhcpv6_action(struct ovn_port *op, struct in6_addr *offer_ip, return true; } -static bool -od_set_acl_flags(struct ovn_datapath *od, struct nbrec_acl **acls, - size_t n_acls) -{ - /* A true return indicates that there are no possible ACL flags - * left to set on od. A false return indicates that further ACLs - * should be explored in case more flags need to be set on od - */ - if (!n_acls) { - return false; - } - - od->has_acls = true; - for (size_t i = 0; i < n_acls; i++) { - const struct nbrec_acl *acl = acls[i]; - if (acl->tier > od->max_acl_tier) { - od->max_acl_tier = acl->tier; - } - if (!od->has_stateful_acl && !strcmp(acl->action, "allow-related")) { - od->has_stateful_acl = true; - } - if (od->has_stateful_acl && - od->max_acl_tier == nbrec_acl_col_tier.type.value.integer.max) { - return true; - } - } - - return false; -} - -static void -ls_get_acl_flags(struct ovn_datapath *od, - const struct ls_port_group_table *ls_port_groups) -{ - od->has_acls = false; - od->has_stateful_acl = false; - od->max_acl_tier = 0; - - if (od_set_acl_flags(od, od->nbs->acls, od->nbs->n_acls)) { - return; - } - - const struct ls_port_group *ls_pg = - ls_port_group_table_find(ls_port_groups, od->nbs); - if (!ls_pg) { - return; - } - - const struct ls_port_group_record *ls_pg_rec; - HMAP_FOR_EACH (ls_pg_rec, key_node, &ls_pg->nb_pgs) { - if (od_set_acl_flags(od, ls_pg_rec->nb_pg->acls, - ls_pg_rec->nb_pg->n_acls)) { - return; - } - } -} - /* Adds the logical flows in the (in/out) check port sec stage only if * - the lport is disabled or * - lport is of type vtep - to skip the ingress pipeline. @@ -6766,9 +6737,10 @@ build_lswitch_output_port_sec_od(struct ovn_datapath *od, } static void -skip_port_from_conntrack(struct ovn_datapath *od, struct ovn_port *op, - enum ovn_stage in_stage, enum ovn_stage out_stage, - uint16_t priority, struct hmap *lflows) +skip_port_from_conntrack(const struct ovn_datapath *od, struct ovn_port *op, + bool has_stateful_acl, enum ovn_stage in_stage, + enum ovn_stage out_stage, uint16_t priority, + struct hmap *lflows) { /* Can't use ct() for router ports. Consider the following configuration: * lp1(10.0.0.2) on hostA--ls1--lr0--ls2--lp2(10.0.1.2) on hostB, For a @@ -6781,7 +6753,7 @@ skip_port_from_conntrack(struct ovn_datapath *od, struct ovn_port *op, * conntrack state across all chassis. */ const char *ingress_action = "next;"; - const char *egress_action = od->has_stateful_acl + const char *egress_action = has_stateful_acl ? "next;" : "ct_clear; next;"; @@ -6800,7 +6772,7 @@ skip_port_from_conntrack(struct ovn_datapath *od, struct ovn_port *op, } static void -build_stateless_filter(struct ovn_datapath *od, +build_stateless_filter(const struct ovn_datapath *od, const struct nbrec_acl *acl, struct hmap *lflows) { @@ -6821,7 +6793,7 @@ build_stateless_filter(struct ovn_datapath *od, } static void -build_stateless_filters(struct ovn_datapath *od, +build_stateless_filters(const struct ovn_datapath *od, const struct ls_port_group_table *ls_port_groups, struct hmap *lflows) { @@ -6851,9 +6823,7 @@ build_stateless_filters(struct ovn_datapath *od, } static void -build_pre_acls(struct ovn_datapath *od, - const struct ls_port_group_table *ls_port_groups, - struct hmap *lflows) +build_pre_acls(struct ovn_datapath *od, struct hmap *lflows) { /* Ingress and Egress Pre-ACL Table (Priority 0): Packets are * allowed by default. */ @@ -6865,18 +6835,26 @@ build_pre_acls(struct ovn_datapath *od, ovn_lflow_add(lflows, od, S_SWITCH_OUT_PRE_ACL, 110, "eth.src == $svc_monitor_mac", "next;"); +} + +static void +build_ls_stateful_rec_pre_acls(const struct ls_stateful_record *ls_sful_rec, + const struct ls_port_group_table *ls_port_groups, + struct hmap *lflows) +{ + const struct ovn_datapath *od = ls_sful_rec->od; /* If there are any stateful ACL rules in this datapath, we may * send IP packets for some (allow) filters through the conntrack action, * which handles defragmentation, in order to match L4 headers. */ - if (od->has_stateful_acl) { + if (ls_sful_rec->has_stateful_acl) { for (size_t i = 0; i < od->n_router_ports; i++) { - skip_port_from_conntrack(od, od->router_ports[i], + skip_port_from_conntrack(od, od->router_ports[i], true, S_SWITCH_IN_PRE_ACL, S_SWITCH_OUT_PRE_ACL, 110, lflows); } for (size_t i = 0; i < od->n_localnet_ports; i++) { - skip_port_from_conntrack(od, od->localnet_ports[i], + skip_port_from_conntrack(od, od->localnet_ports[i], true, S_SWITCH_IN_PRE_ACL, S_SWITCH_OUT_PRE_ACL, 110, lflows); @@ -6914,7 +6892,7 @@ build_pre_acls(struct ovn_datapath *od, REGBIT_CONNTRACK_DEFRAG" = 1; next;"); ovn_lflow_add(lflows, od, S_SWITCH_OUT_PRE_ACL, 100, "ip", REGBIT_CONNTRACK_DEFRAG" = 1; next;"); - } else if (od->has_lb_vip) { + } else if (ls_sful_rec->has_lb_vip) { /* We'll build stateless filters if there are LB rules so that * the stateless flows are not tracked in pre-lb. */ build_stateless_filters(od, ls_port_groups, lflows); @@ -7042,30 +7020,40 @@ build_pre_lb(struct ovn_datapath *od, const struct shash *meter_groups, ovn_lflow_add(lflows, od, S_SWITCH_IN_PRE_LB, 0, "1", "next;"); ovn_lflow_add(lflows, od, S_SWITCH_OUT_PRE_LB, 0, "1", "next;"); + /* Do not send statless flows via conntrack */ + ovn_lflow_add(lflows, od, S_SWITCH_IN_PRE_LB, 110, + REGBIT_ACL_STATELESS" == 1", "next;"); + ovn_lflow_add(lflows, od, S_SWITCH_OUT_PRE_LB, 110, + REGBIT_ACL_STATELESS" == 1", "next;"); +} + +static void +build_ls_stateful_rec_pre_lb(const struct ls_stateful_record *ls_stateful_rec, + struct hmap *lflows) +{ + const struct ovn_datapath *od = ls_stateful_rec->od; + for (size_t i = 0; i < od->n_router_ports; i++) { skip_port_from_conntrack(od, od->router_ports[i], + ls_stateful_rec->has_stateful_acl, S_SWITCH_IN_PRE_LB, S_SWITCH_OUT_PRE_LB, 110, lflows); } + /* Localnet ports have no need for going through conntrack, unless * the logical switch has a load balancer. Then, conntrack is necessary * so that traffic arriving via the localnet port can be load * balanced. */ - if (!od->has_lb_vip) { + if (!ls_stateful_rec->has_lb_vip) { for (size_t i = 0; i < od->n_localnet_ports; i++) { skip_port_from_conntrack(od, od->localnet_ports[i], + ls_stateful_rec->has_stateful_acl, S_SWITCH_IN_PRE_LB, S_SWITCH_OUT_PRE_LB, 110, lflows); } } - /* Do not sent statless flows via conntrack */ - ovn_lflow_add(lflows, od, S_SWITCH_IN_PRE_LB, 110, - REGBIT_ACL_STATELESS" == 1", "next;"); - ovn_lflow_add(lflows, od, S_SWITCH_OUT_PRE_LB, 110, - REGBIT_ACL_STATELESS" == 1", "next;"); - /* 'REGBIT_CONNTRACK_NAT' is set to let the pre-stateful table send * packet to conntrack for defragmentation and possibly for unNATting. * @@ -7096,7 +7084,7 @@ build_pre_lb(struct ovn_datapath *od, const struct shash *meter_groups, * ingress pipeline if a load balancer is configured. We can now * add a lflow to drop ct.inv packets. */ - if (od->has_lb_vip) { + if (ls_stateful_rec->has_lb_vip) { ovn_lflow_add(lflows, od, S_SWITCH_IN_PRE_LB, 100, "ip", REGBIT_CONNTRACK_NAT" = 1; next;"); ovn_lflow_add(lflows, od, S_SWITCH_OUT_PRE_LB, @@ -7137,10 +7125,12 @@ build_pre_stateful(struct ovn_datapath *od, } static void -build_acl_hints(struct ovn_datapath *od, +build_acl_hints(const struct ls_stateful_record *ls_sful_rec, const struct chassis_features *features, struct hmap *lflows) { + const struct ovn_datapath *od = ls_sful_rec->od; + /* This stage builds hints for the IN/OUT_ACL stage. Based on various * combinations of ct flags packets may hit only a subset of the logical * flows in the IN/OUT_ACL stage. @@ -7164,13 +7154,13 @@ build_acl_hints(struct ovn_datapath *od, const char *match; /* In any case, advance to the next stage. */ - if (!od->has_acls && !od->has_lb_vip) { + if (!ls_sful_rec->has_acls && !ls_sful_rec->has_lb_vip) { ovn_lflow_add(lflows, od, stage, UINT16_MAX, "1", "next;"); } else { ovn_lflow_add(lflows, od, stage, 0, "1", "next;"); } - if (!od->has_stateful_acl && !od->has_lb_vip) { + if (!ls_sful_rec->has_stateful_acl && !ls_sful_rec->has_lb_vip) { continue; } @@ -7306,10 +7296,10 @@ build_acl_log(struct ds *actions, const struct nbrec_acl *acl, } static void -consider_acl(struct hmap *lflows, struct ovn_datapath *od, +consider_acl(struct hmap *lflows, const struct ovn_datapath *od, const struct nbrec_acl *acl, bool has_stateful, bool ct_masked_mark, const struct shash *meter_groups, - struct ds *match, struct ds *actions) + uint64_t max_acl_tier, struct ds *match, struct ds *actions) { const char *ct_blocked_match = ct_masked_mark ? "ct_mark.blocked" @@ -7346,7 +7336,7 @@ consider_acl(struct hmap *lflows, struct ovn_datapath *od, /* All ACLS will start by matching on their respective tier. */ size_t match_tier_len = 0; ds_clear(match); - if (od->max_acl_tier) { + if (max_acl_tier) { ds_put_format(match, REG_ACL_TIER " == %"PRId64" && ", acl->tier); match_tier_len = match->length; } @@ -7533,12 +7523,15 @@ ovn_update_ipv6_options(struct hmap *lr_ports) #define IPV6_CT_OMIT_MATCH "nd || nd_ra || nd_rs || mldv1 || mldv2" static void -build_acl_action_lflows(struct ovn_datapath *od, struct hmap *lflows, +build_acl_action_lflows(const struct ls_stateful_record *ls_stateful_rec, + struct hmap *lflows, const char *default_acl_action, const struct shash *meter_groups, struct ds *match, struct ds *actions) { + const struct ovn_datapath *od = ls_stateful_rec->od; + enum ovn_stage stages [] = { S_SWITCH_IN_ACL_ACTION, S_SWITCH_IN_ACL_AFTER_LB_ACTION, @@ -7549,7 +7542,7 @@ build_acl_action_lflows(struct ovn_datapath *od, struct hmap *lflows, ds_put_cstr(actions, REGBIT_ACL_VERDICT_ALLOW " = 0; " REGBIT_ACL_VERDICT_DROP " = 0; " REGBIT_ACL_VERDICT_REJECT " = 0; "); - if (od->max_acl_tier) { + if (ls_stateful_rec->max_acl_tier) { ds_put_cstr(actions, REG_ACL_TIER " = 0; "); } @@ -7557,7 +7550,7 @@ build_acl_action_lflows(struct ovn_datapath *od, struct hmap *lflows, for (size_t i = 0; i < ARRAY_SIZE(stages); i++) { enum ovn_stage stage = stages[i]; - if (!od->has_acls) { + if (!ls_stateful_rec->has_acls) { ovn_lflow_add(lflows, od, stage, 0, "1", "next;"); continue; } @@ -7592,7 +7585,7 @@ build_acl_action_lflows(struct ovn_datapath *od, struct hmap *lflows, ovn_lflow_add(lflows, od, stage, 0, "1", ds_cstr(actions)); struct ds tier_actions = DS_EMPTY_INITIALIZER; - for (size_t j = 0; j < od->max_acl_tier; j++) { + for (size_t j = 0; j < ls_stateful_rec->max_acl_tier; j++) { ds_clear(match); ds_put_format(match, REG_ACL_TIER " == %"PRIuSIZE, j); ds_clear(&tier_actions); @@ -7608,7 +7601,7 @@ build_acl_action_lflows(struct ovn_datapath *od, struct hmap *lflows, } static void -build_acl_log_related_flows(struct ovn_datapath *od, struct hmap *lflows, +build_acl_log_related_flows(const struct ovn_datapath *od, struct hmap *lflows, const struct nbrec_acl *acl, bool has_stateful, bool ct_masked_mark, const struct shash *meter_groups, @@ -7681,15 +7674,19 @@ build_acl_log_related_flows(struct ovn_datapath *od, struct hmap *lflows, } static void -build_acls(struct ovn_datapath *od, const struct chassis_features *features, +build_acls(const struct ls_stateful_record *ls_stateful_rec, + const struct chassis_features *features, struct hmap *lflows, const struct ls_port_group_table *ls_port_groups, const struct shash *meter_groups) { + const struct ovn_datapath *od = ls_stateful_rec->od; + const char *default_acl_action = default_acl_drop ? debug_implicit_drop_action() : "next;"; - bool has_stateful = od->has_stateful_acl || od->has_lb_vip; + bool has_stateful = (ls_stateful_rec->has_stateful_acl + || ls_stateful_rec->has_lb_vip); const char *ct_blocked_match = features->ct_no_masked_label ? "ct_mark.blocked" : "ct_label.blocked"; @@ -7703,8 +7700,8 @@ build_acls(struct ovn_datapath *od, const struct chassis_features *features, * * A related rule at priority 1 is added below if there * are any stateful ACLs in this datapath. */ - if (!od->has_acls) { - if (!od->has_lb_vip) { + if (!ls_stateful_rec->has_acls) { + if (!ls_stateful_rec->has_lb_vip) { ovn_lflow_add(lflows, od, S_SWITCH_IN_ACL_EVAL, UINT16_MAX, "1", "next;"); ovn_lflow_add(lflows, od, S_SWITCH_OUT_ACL_EVAL, UINT16_MAX, "1", @@ -7867,7 +7864,8 @@ build_acls(struct ovn_datapath *od, const struct chassis_features *features, meter_groups, &match, &actions); consider_acl(lflows, od, acl, has_stateful, features->ct_no_masked_label, - meter_groups, &match, &actions); + meter_groups, ls_stateful_rec->max_acl_tier, + &match, &actions); } const struct ls_port_group *ls_pg = @@ -7883,7 +7881,8 @@ build_acls(struct ovn_datapath *od, const struct chassis_features *features, meter_groups, &match, &actions); consider_acl(lflows, od, acl, has_stateful, features->ct_no_masked_label, - meter_groups, &match, &actions); + meter_groups, ls_stateful_rec->max_acl_tier, + &match, &actions); } } } @@ -7901,7 +7900,7 @@ build_acls(struct ovn_datapath *od, const struct chassis_features *features, dns_actions); } - if (od->has_acls || od->has_lb_vip) { + if (ls_stateful_rec->has_acls || ls_stateful_rec->has_lb_vip) { /* Add a 34000 priority flow to advance the service monitor reply * packets to skip applying ingress ACLs. */ ovn_lflow_add(lflows, od, S_SWITCH_IN_ACL_EVAL, 34000, @@ -7915,8 +7914,8 @@ build_acls(struct ovn_datapath *od, const struct chassis_features *features, REGBIT_ACL_VERDICT_ALLOW" = 1; next;"); } - build_acl_action_lflows(od, lflows, default_acl_action, meter_groups, - &match, &actions); + build_acl_action_lflows(ls_stateful_rec, lflows, default_acl_action, + meter_groups, &match, &actions); ds_destroy(&match); ds_destroy(&actions); @@ -8561,8 +8560,11 @@ build_stateful(struct ovn_datapath *od, } static void -build_lb_hairpin(struct ovn_datapath *od, struct hmap *lflows) +build_lb_hairpin(const struct ls_stateful_record *ls_stateful_rec, + struct hmap *lflows) { + const struct ovn_datapath *od = ls_stateful_rec->od; + /* Ingress Pre-Hairpin/Nat-Hairpin/Hairpin tabled (Priority 0). * Packets that don't need hairpinning should continue processing. */ @@ -8570,7 +8572,7 @@ build_lb_hairpin(struct ovn_datapath *od, struct hmap *lflows) ovn_lflow_add(lflows, od, S_SWITCH_IN_NAT_HAIRPIN, 0, "1", "next;"); ovn_lflow_add(lflows, od, S_SWITCH_IN_HAIRPIN, 0, "1", "next;"); - if (od->has_lb_vip) { + if (ls_stateful_rec->has_lb_vip) { /* Check if the packet needs to be hairpinned. * Set REGBIT_HAIRPIN in the original direction and * REGBIT_HAIRPIN_REPLY in the reply direction. @@ -9193,9 +9195,7 @@ build_dhcpv4_options_flows(struct ovn_port *op, &op->nbsp->dhcpv4_options->options, "lease_time"); ovs_assert(server_id && server_mac && lease_time); const char *dhcp_actions = - (op->od->has_stateful_acl || op->od->has_lb_vip) - ? REGBIT_ACL_VERDICT_ALLOW" = 1; ct_commit; next;" - : REGBIT_ACL_VERDICT_ALLOW" = 1; next;"; + REGBIT_ACL_VERDICT_ALLOW" = 1; next;"; ds_clear(&match); ds_put_format(&match, "outport == %s && eth.src == %s " "&& ip4.src == %s && udp && udp.src == 67 " @@ -9278,9 +9278,7 @@ build_dhcpv6_options_flows(struct ovn_port *op, ipv6_string_mapped(server_ip, &lla); const char *dhcp6_actions = - (op->od->has_stateful_acl || op->od->has_lb_vip) - ? REGBIT_ACL_VERDICT_ALLOW" = 1; ct_commit; next;" - : REGBIT_ACL_VERDICT_ALLOW" = 1; next;"; + REGBIT_ACL_VERDICT_ALLOW" = 1; next;"; ds_clear(&match); ds_put_format(&match, "outport == %s && eth.src == %s " "&& ip6.src == %s && udp && udp.src == 547 " @@ -9387,22 +9385,16 @@ build_lswitch_lflows_l2_unknown(struct ovn_datapath *od, static void build_lswitch_lflows_pre_acl_and_acl( struct ovn_datapath *od, - const struct ls_port_group_table *ls_port_groups, const struct chassis_features *features, struct hmap *lflows, const struct shash *meter_groups) { ovs_assert(od->nbs); - ls_get_acl_flags(od, ls_port_groups); - - build_pre_acls(od, ls_port_groups, lflows); + build_pre_acls(od, lflows); build_pre_lb(od, meter_groups, lflows); build_pre_stateful(od, features, lflows); - build_acl_hints(od, features, lflows); - build_acls(od, features, lflows, ls_port_groups, meter_groups); build_qos(od, lflows); build_stateful(od, features, lflows); - build_lb_hairpin(od, lflows); build_vtep_hairpin(od, lflows); } @@ -15461,7 +15453,7 @@ build_lrouter_nat_defrag_and_lb( * a dynamically negotiated FTP data channel), but will allow * related traffic such as an ICMP Port Unreachable through * that's generated from a non-listening UDP port. */ - if (od->has_lb_vip && features->ct_lb_related) { + if (lr_sful_rec->has_lb_vip && features->ct_lb_related) { ds_clear(match); ds_put_cstr(match, "ct.rel && !ct.est && !ct.new"); @@ -15486,7 +15478,7 @@ build_lrouter_nat_defrag_and_lb( * Pass the traffic that is already established to the next table with * proper flags set. */ - if (od->has_lb_vip) { + if (lr_sful_rec->has_lb_vip) { ds_clear(match); ds_put_format(match, "ct.est && !ct.rel && !ct.new && %s.natted", @@ -15516,7 +15508,7 @@ build_lrouter_nat_defrag_and_lb( * not committed, it would produce ongoing datapath flows with the ct.new * flag set. Some NICs are unable to offload these flows. */ - if (od->is_gw_router && (od->nbr->n_nat || od->has_lb_vip)) { + if (od->is_gw_router && (od->nbr->n_nat || lr_sful_rec->has_lb_vip)) { /* Do not send ND or ICMP packets to connection tracking. */ ovn_lflow_add(lflows, od, S_ROUTER_OUT_UNDNAT, 100, "nd || nd_rs || nd_ra", "next;"); @@ -15941,6 +15933,22 @@ build_lr_stateful_flows(const struct lr_stateful_record *lr_sful_rec, meter_groups); } +static void +build_ls_stateful_flows(const struct ls_stateful_record *ls_stateful_rec, + const struct ls_port_group_table *ls_pgs, + const struct chassis_features *features, + const struct shash *meter_groups, + struct hmap *lflows) +{ + ovs_assert(ls_stateful_rec->od); + + build_ls_stateful_rec_pre_acls(ls_stateful_rec, ls_pgs, lflows); + build_ls_stateful_rec_pre_lb(ls_stateful_rec, lflows); + build_acl_hints(ls_stateful_rec, features, lflows); + build_acls(ls_stateful_rec, features, lflows, ls_pgs, meter_groups); + build_lb_hairpin(ls_stateful_rec, lflows); +} + struct lswitch_flow_build_info { const struct ovn_datapaths *ls_datapaths; const struct ovn_datapaths *lr_datapaths; @@ -15948,6 +15956,7 @@ struct lswitch_flow_build_info { const struct hmap *lr_ports; const struct ls_port_group_table *ls_port_groups; const struct lr_stateful_table *lr_sful_table; + const struct ls_stateful_table *ls_sful_table; struct hmap *lflows; struct hmap *igmp_groups; const struct shash *meter_groups; @@ -15972,9 +15981,7 @@ build_lswitch_and_lrouter_iterate_by_ls(struct ovn_datapath *od, struct lswitch_flow_build_info *lsi) { ovs_assert(od->nbs); - build_lswitch_lflows_pre_acl_and_acl(od, lsi->ls_port_groups, - lsi->features, - lsi->lflows, + build_lswitch_lflows_pre_acl_and_acl(od, lsi->features, lsi->lflows, lsi->meter_groups); build_fwd_group_lflows(od, lsi->lflows); @@ -16089,6 +16096,7 @@ build_lflows_thread(void *arg) { struct worker_control *control = (struct worker_control *) arg; const struct lr_stateful_record *lr_sful_rec; + const struct ls_stateful_record *ls_stateful_rec; struct lswitch_flow_build_info *lsi; struct ovn_igmp_group *igmp_group; struct ovn_lb_datapaths *lb_dps; @@ -16217,6 +16225,20 @@ build_lflows_thread(void *arg) lsi->features); } } + + for (bnum = control->id; + bnum <= lsi->ls_sful_table->entries.mask; + bnum += control->pool->size) + { + LS_STATEFUL_TABLE_FOR_EACH_IN_P (ls_stateful_rec, bnum, + lsi->ls_sful_table) { + build_ls_stateful_flows(ls_stateful_rec, + lsi->ls_port_groups, + lsi->features, lsi->meter_groups, + lsi->lflows); + } + } + for (bnum = control->id; bnum <= lsi->igmp_groups->mask; bnum += control->pool->size) @@ -16277,6 +16299,7 @@ build_lswitch_and_lrouter_flows(const struct ovn_datapaths *ls_datapaths, const struct hmap *lr_ports, const struct ls_port_group_table *ls_pgs, const struct lr_stateful_table *lr_sful_table, + const struct ls_stateful_table *ls_sful_table, struct hmap *lflows, struct hmap *igmp_groups, const struct shash *meter_groups, @@ -16307,6 +16330,7 @@ build_lswitch_and_lrouter_flows(const struct ovn_datapaths *ls_datapaths, lsiv[index].lr_ports = lr_ports; lsiv[index].ls_port_groups = ls_pgs; lsiv[index].lr_sful_table = lr_sful_table; + lsiv[index].ls_sful_table = ls_sful_table; lsiv[index].igmp_groups = igmp_groups; lsiv[index].meter_groups = meter_groups; lsiv[index].lb_dps_map = lb_dps_map; @@ -16332,6 +16356,7 @@ build_lswitch_and_lrouter_flows(const struct ovn_datapaths *ls_datapaths, free(lsiv); } else { const struct lr_stateful_record *lr_sful_rec; + const struct ls_stateful_record *ls_stateful_rec; struct ovn_igmp_group *igmp_group; struct ovn_lb_datapaths *lb_dps; struct ovn_datapath *od; @@ -16344,6 +16369,7 @@ build_lswitch_and_lrouter_flows(const struct ovn_datapaths *ls_datapaths, .lr_ports = lr_ports, .ls_port_groups = ls_pgs, .lr_sful_table = lr_sful_table, + .ls_sful_table = ls_sful_table, .lflows = lflows, .igmp_groups = igmp_groups, .meter_groups = meter_groups, @@ -16413,6 +16439,12 @@ build_lswitch_and_lrouter_flows(const struct ovn_datapaths *ls_datapaths, lsi.features); } + LS_STATEFUL_TABLE_FOR_EACH (ls_stateful_rec, ls_sful_table) { + build_ls_stateful_flows(ls_stateful_rec, lsi.ls_port_groups, + lsi.features, lsi.meter_groups, + lsi.lflows); + } + stopwatch_start(LFLOWS_IGMP_STOPWATCH_NAME, time_msec()); HMAP_FOR_EACH (igmp_group, hmap_node, igmp_groups) { build_lswitch_ip_mcast_igmp_mld(igmp_group, @@ -16509,6 +16541,7 @@ void build_lflows(struct ovsdb_idl_txn *ovnsb_txn, input_data->lr_ports, input_data->ls_port_groups, input_data->lr_sful_table, + input_data->ls_sful_table, lflows, &igmp_groups, input_data->meter_groups, diff --git a/northd/northd.h b/northd/northd.h index 612b292da1..7727b5a8f6 100644 --- a/northd/northd.h +++ b/northd/northd.h @@ -89,6 +89,8 @@ ods_size(const struct ovn_datapaths *datapaths) return hmap_count(&datapaths->datapaths); } +bool od_has_lb_vip(const struct ovn_datapath *od); + struct tracked_ovn_ports { /* tracked created ports. * hmapx node data is 'struct ovn_port *' */ @@ -118,6 +120,8 @@ enum northd_tracked_data_type { NORTHD_TRACKED_PORTS = (1 << 0), NORTHD_TRACKED_LBS = (1 << 1), NORTHD_TRACKED_LR_NATS = (1 << 2), + NORTHD_TRACKED_LS_LBS = (1 << 3), + NORTHD_TRACKED_LS_ACLS = (1 << 4), }; /* Track what's changed in the northd engine node. @@ -132,6 +136,14 @@ struct northd_tracked_data { /* Tracked logical routers whose NATs have changed. * hmapx node is 'struct ovn_datapath *'. */ struct hmapx lr_with_changed_nats; + + /* Tracked logical switches whose load balancers have changed. + * hmapx node is 'struct ovn_datapath *'. */ + struct hmapx ls_with_changed_lbs; + + /* Tracked logical switches whose ACLs have changed. + * hmapx node is 'struct ovn_datapath *'. */ + struct hmapx ls_with_changed_acls; }; struct northd_data { @@ -180,6 +192,7 @@ struct lflow_input { const struct hmap *lr_ports; const struct ls_port_group_table *ls_port_groups; const struct lr_stateful_table *lr_sful_table; + const struct ls_stateful_table *ls_sful_table; const struct shash *meter_groups; const struct hmap *lb_datapaths_map; const struct hmap *bfd_connections; @@ -289,11 +302,7 @@ struct ovn_datapath { struct hmap port_tnlids; uint32_t port_key_hint; - bool has_stateful_acl; - bool has_lb_vip; bool has_unknown; - bool has_acls; - uint64_t max_acl_tier; bool has_vtep_lports; bool has_arp_proxy_port; @@ -538,6 +547,18 @@ northd_has_lr_nats_in_tracked_data(struct northd_tracked_data *trk_nd_changes) return (trk_nd_changes->type & NORTHD_TRACKED_LR_NATS); } +static inline bool +northd_has_ls_lbs_in_tracked_data(struct northd_tracked_data *trk_nd_changes) +{ + return (trk_nd_changes->type & NORTHD_TRACKED_LS_LBS); +} + +static inline bool +northd_has_ls_acls_in_tracked_data(struct northd_tracked_data *trk_nd_changes) +{ + return (trk_nd_changes->type & NORTHD_TRACKED_LS_ACLS); +} + /* Returns 'true' if the IPv4 'addr' is on the same subnet with one of the * IPs configured on the router port. */ diff --git a/northd/ovn-northd.c b/northd/ovn-northd.c index e85e00cc9b..084d675644 100644 --- a/northd/ovn-northd.c +++ b/northd/ovn-northd.c @@ -872,6 +872,7 @@ main(int argc, char *argv[]) stopwatch_create(SYNC_METERS_RUN_STOPWATCH_NAME, SW_MS); stopwatch_create(LR_NAT_RUN_STOPWATCH_NAME, SW_MS); stopwatch_create(LR_STATEFUL_RUN_STOPWATCH_NAME, SW_MS); + stopwatch_create(LS_STATEFUL_RUN_STOPWATCH_NAME, SW_MS); /* Initialize incremental processing engine for ovn-northd */ inc_proc_northd_init(&ovnnb_idl_loop, &ovnsb_idl_loop); From patchwork Tue Nov 28 02:36:31 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Numan Siddique X-Patchwork-Id: 1869079 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::136; helo=smtp3.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4SfRSK28fCz1yRy for ; Tue, 28 Nov 2023 13:37:21 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 233D360FB5; Tue, 28 Nov 2023 02:37:18 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 233D360FB5 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id JAV73nxmWiI8; Tue, 28 Nov 2023 02:37:13 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp3.osuosl.org (Postfix) with ESMTPS id 63F1460F9B; Tue, 28 Nov 2023 02:37:12 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 63F1460F9B Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 23877C0DD0; Tue, 28 Nov 2023 02:37:12 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp2.osuosl.org (smtp2.osuosl.org [IPv6:2605:bc80:3010::133]) by lists.linuxfoundation.org (Postfix) with ESMTP id B97BEC0039 for ; Tue, 28 Nov 2023 02:37:10 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 9BA0040A41 for ; Tue, 28 Nov 2023 02:36:51 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 9BA0040A41 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id K0a-XvblZxDQ for ; Tue, 28 Nov 2023 02:36:48 +0000 (UTC) Received: from relay5-d.mail.gandi.net (relay5-d.mail.gandi.net [217.70.183.197]) by smtp2.osuosl.org (Postfix) with ESMTPS id 1AFF240A1E for ; Tue, 28 Nov 2023 02:36:46 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 1AFF240A1E Received: by mail.gandi.net (Postfix) with ESMTPSA id DAC6F1C0002; Tue, 28 Nov 2023 02:36:42 +0000 (UTC) From: numans@ovn.org To: dev@openvswitch.org Date: Mon, 27 Nov 2023 21:36:31 -0500 Message-ID: <20231128023631.569839-1-numans@ovn.org> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20231128023410.569539-1-numans@ovn.org> References: <20231128023410.569539-1-numans@ovn.org> MIME-Version: 1.0 X-GND-Sasl: numans@ovn.org Subject: [ovs-dev] [PATCH ovn v3 08/16] northd: Refactor lflow management into a separate module. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Numan Siddique ovn_lflow_add() and other related functions/macros are now moved into a separate module - lflow-mgr.c. This module maintains a table 'struct lflow_table' for the logical flows. lflow table maintains a hmap to store the logical flows. It also maintains the logical switch and router dp groups. Previous commits which added lflow incremental processing for the VIF logical ports, stored the references to the logical ports' lflows using 'struct lflow_ref_list'. This struct is renamed to 'struct lflow_ref' and is part of lflow-mgr.c. It is modified a bit to store the resource to lflow references. Example usage of 'struct lflow_ref'. 'struct ovn_port' maintains 2 instances of lflow_ref. i,e struct ovn_port { ... ... struct lflow_ref *lflow_ref; struct lflow_ref *stateful_lflow_ref; }; All the logical flows generated by build_lswitch_and_lrouter_iterate_by_lsp() uses the ovn_port->lflow_ref. All the logical flows generated by build_lsp_lflows_for_lbnats() uses the ovn_port->stateful_lflow_ref. When handling the ovn_port changes incrementally, the lflows referenced in 'struct ovn_port' are cleared and regenerated and synced to the SB logical flows. eg. lflow_ref_clear_lflows(op->lflow_ref); build_lswitch_and_lrouter_iterate_by_lsp(op, ...); lflow_ref_sync_lflows_to_sb(op->lflow_ref, ...); This patch does few more changes: - Logical flows are now hashed without the logical datapaths. If a logical flow is referenced by just one datapath, we don't rehash it. - The synthetic 'hash' column of sbrec_logical_flow now doesn't use the logical datapath. This means that when ovn-northd is updated/upgraded and has this commit, all the logical flows with 'logical_datapath' column set will get deleted and re-added causing some disruptions. - With the commit [1] which added I-P support for logical port changes, multiple logical flows with same match 'M' and actions 'A' are generated and stored without the dp groups, which was not the case prior to that patch. One example to generate these lflows is: ovn-nbctl lsp-set-addresses sw0p1 "MAC1 IP1" ovn-nbctl lsp-set-addresses sw1p1 "MAC1 IP1" ovn-nbctl lsp-set-addresses sw2p1 "MAC1 IP1" Now with this patch we go back to the earlier way. i.e one logical flow with logical_dp_groups set. - With this patch any updates to a logical port which doesn't result in new logical flows will not result in deletion and addition of same logical flows. Eg. ovn-nbctl set logical_switch_port sw0p1 external_ids:foo=bar will be a no-op to the SB logical flow table. [1] - 8bbd678("northd: Incremental processing of VIF additions in 'lflow' node.") Signed-off-by: Numan Siddique --- lib/ovn-util.c | 11 +- northd/automake.mk | 4 +- northd/en-lflow.c | 26 +- northd/en-lflow.h | 6 + northd/inc-proc-northd.c | 4 +- northd/lflow-mgr.c | 1099 +++++++++++++++++++++++ northd/lflow-mgr.h | 186 ++++ northd/northd.c | 1836 +++++++++----------------------------- northd/northd.h | 212 ++++- northd/ovn-northd.c | 4 + 10 files changed, 1960 insertions(+), 1428 deletions(-) create mode 100644 northd/lflow-mgr.c create mode 100644 northd/lflow-mgr.h diff --git a/lib/ovn-util.c b/lib/ovn-util.c index 05e635a6b4..29464f24a3 100644 --- a/lib/ovn-util.c +++ b/lib/ovn-util.c @@ -630,13 +630,10 @@ ovn_pipeline_from_name(const char *pipeline) uint32_t sbrec_logical_flow_hash(const struct sbrec_logical_flow *lf) { - const struct sbrec_datapath_binding *ld = lf->logical_datapath; - uint32_t hash = ovn_logical_flow_hash(lf->table_id, - ovn_pipeline_from_name(lf->pipeline), - lf->priority, lf->match, - lf->actions); - - return ld ? ovn_logical_flow_hash_datapath(&ld->header_.uuid, hash) : hash; + return ovn_logical_flow_hash(lf->table_id, + ovn_pipeline_from_name(lf->pipeline), + lf->priority, lf->match, + lf->actions); } uint32_t diff --git a/northd/automake.mk b/northd/automake.mk index 44140ce203..aac6a8ecdd 100644 --- a/northd/automake.mk +++ b/northd/automake.mk @@ -33,7 +33,9 @@ northd_ovn_northd_SOURCES = \ northd/inc-proc-northd.c \ northd/inc-proc-northd.h \ northd/ipam.c \ - northd/ipam.h + northd/ipam.h \ + northd/lflow-mgr.c \ + northd/lflow-mgr.h northd_ovn_northd_LDADD = \ lib/libovn.la \ $(OVSDB_LIBDIR)/libovsdb.la \ diff --git a/northd/en-lflow.c b/northd/en-lflow.c index 83f11850ce..65d2f45ebc 100644 --- a/northd/en-lflow.c +++ b/northd/en-lflow.c @@ -24,6 +24,7 @@ #include "en-ls-stateful.h" #include "en-northd.h" #include "en-meters.h" +#include "lflow-mgr.h" #include "lib/inc-proc-eng.h" #include "northd.h" @@ -58,6 +59,8 @@ lflow_get_input_data(struct engine_node *node, EN_OVSDB_GET(engine_get_input("SB_multicast_group", node)); lflow_input->sbrec_igmp_group_table = EN_OVSDB_GET(engine_get_input("SB_igmp_group", node)); + lflow_input->sbrec_logical_dp_group_table = + EN_OVSDB_GET(engine_get_input("SB_logical_dp_group", node)); lflow_input->sbrec_mcast_group_by_name_dp = engine_ovsdb_node_get_index( @@ -90,17 +93,21 @@ void en_lflow_run(struct engine_node *node, void *data) struct hmap bfd_connections = HMAP_INITIALIZER(&bfd_connections); lflow_input.bfd_connections = &bfd_connections; + stopwatch_start(BUILD_LFLOWS_STOPWATCH_NAME, time_msec()); + struct lflow_data *lflow_data = data; - lflow_data_destroy(lflow_data); - lflow_data_init(lflow_data); + lflow_table_clear(lflow_data->lflow_table); + lflow_table_init(lflow_data->lflow_table); + + reset_lflow_refs_for_northd_resources(&lflow_input); - stopwatch_start(BUILD_LFLOWS_STOPWATCH_NAME, time_msec()); build_bfd_table(eng_ctx->ovnsb_idl_txn, lflow_input.nbrec_bfd_table, lflow_input.sbrec_bfd_table, lflow_input.lr_ports, &bfd_connections); - build_lflows(eng_ctx->ovnsb_idl_txn, &lflow_input, &lflow_data->lflows); + build_lflows(eng_ctx->ovnsb_idl_txn, &lflow_input, + lflow_data->lflow_table); bfd_cleanup_connections(lflow_input.nbrec_bfd_table, &bfd_connections); hmap_destroy(&bfd_connections); @@ -131,7 +138,7 @@ lflow_northd_handler(struct engine_node *node, if (!lflow_handle_northd_port_changes(eng_ctx->ovnsb_idl_txn, &northd_data->trk_data.trk_lsps, - &lflow_input, &lflow_data->lflows)) { + &lflow_input, lflow_data->lflow_table)) { return false; } @@ -160,11 +167,14 @@ void *en_lflow_init(struct engine_node *node OVS_UNUSED, struct engine_arg *arg OVS_UNUSED) { struct lflow_data *data = xmalloc(sizeof *data); - lflow_data_init(data); + data->lflow_table = lflow_table_alloc(); + lflow_table_init(data->lflow_table); return data; } -void en_lflow_cleanup(void *data) +void en_lflow_cleanup(void *data_) { - lflow_data_destroy(data); + struct lflow_data *data = data_; + lflow_table_destroy(data->lflow_table); + data->lflow_table = NULL; } diff --git a/northd/en-lflow.h b/northd/en-lflow.h index 5417b2faff..f7325c56b1 100644 --- a/northd/en-lflow.h +++ b/northd/en-lflow.h @@ -9,6 +9,12 @@ #include "lib/inc-proc-eng.h" +struct lflow_table; + +struct lflow_data { + struct lflow_table *lflow_table; +}; + void en_lflow_run(struct engine_node *node, void *data); void *en_lflow_init(struct engine_node *node, struct engine_arg *arg); void en_lflow_cleanup(void *data); diff --git a/northd/inc-proc-northd.c b/northd/inc-proc-northd.c index 9ce4279ee8..0e17bfe2e6 100644 --- a/northd/inc-proc-northd.c +++ b/northd/inc-proc-northd.c @@ -99,7 +99,8 @@ static unixctl_cb_func chassis_features_list; SB_NODE(bfd, "bfd") \ SB_NODE(fdb, "fdb") \ SB_NODE(static_mac_binding, "static_mac_binding") \ - SB_NODE(chassis_template_var, "chassis_template_var") + SB_NODE(chassis_template_var, "chassis_template_var") \ + SB_NODE(logical_dp_group, "logical_dp_group") enum sb_engine_node { #define SB_NODE(NAME, NAME_STR) SB_##NAME, @@ -229,6 +230,7 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, engine_add_input(&en_lflow, &en_sb_igmp_group, NULL); engine_add_input(&en_lflow, &en_lr_stateful, NULL); engine_add_input(&en_lflow, &en_ls_stateful, NULL); + engine_add_input(&en_lflow, &en_sb_logical_dp_group, NULL); engine_add_input(&en_lflow, &en_northd, lflow_northd_handler); engine_add_input(&en_lflow, &en_port_group, lflow_port_group_handler); diff --git a/northd/lflow-mgr.c b/northd/lflow-mgr.c new file mode 100644 index 0000000000..08962e9172 --- /dev/null +++ b/northd/lflow-mgr.c @@ -0,0 +1,1099 @@ +/* + * Copyright (c) 2023, Red Hat, Inc. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at: + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#include + +#include +#include +#include + +/* OVS includes */ +#include "include/openvswitch/hmap.h" +#include "include/openvswitch/thread.h" +#include "lib/bitmap.h" +#include "lib/uuidset.h" +#include "openvswitch/util.h" +#include "openvswitch/vlog.h" +#include "ovs-thread.h" +#include "stopwatch.h" + +/* OVN includes */ +#include "debug.h" +#include "lflow-mgr.h" +#include "lib/ovn-parallel-hmap.h" +#include "northd.h" + +VLOG_DEFINE_THIS_MODULE(lflow_mgr); + +/* Static function declarations. */ +struct ovn_lflow; + +static void ovn_lflow_init(struct ovn_lflow *, struct ovn_datapath *od, + size_t dp_bitmap_len, enum ovn_stage stage, + uint16_t priority, char *match, + char *actions, char *io_port, + char *ctrl_meter, char *stage_hint, + const char *where, uint32_t hash); +static struct ovn_lflow *ovn_lflow_find(const struct hmap *lflows, + enum ovn_stage stage, + uint16_t priority, const char *match, + const char *actions, + const char *ctrl_meter, uint32_t hash); +static void ovn_lflow_destroy(struct lflow_table *lflow_table, + struct ovn_lflow *lflow); +static void inc_ovn_lflow_ref(struct ovn_lflow *); +static void dec_ovn_lflow_ref(struct lflow_table *, struct ovn_lflow *); +static char *ovn_lflow_hint(const struct ovsdb_idl_row *row); + +static struct ovn_lflow *do_ovn_lflow_add( + struct lflow_table *, const struct ovn_datapath *, + const unsigned long *dp_bitmap, size_t dp_bitmap_len, uint32_t hash, + enum ovn_stage stage, uint16_t priority, const char *match, + const char *actions, const char *io_port, + const char *ctrl_meter, + const struct ovsdb_idl_row *stage_hint, + const char *where); + + +static struct ovs_mutex *lflow_hash_lock(const struct hmap *lflow_table, + uint32_t hash); +static void lflow_hash_unlock(struct ovs_mutex *hash_lock); + +static struct ovn_dp_group *ovn_dp_group_get( + struct hmap *dp_groups, size_t desired_n, + const unsigned long *desired_bitmap, + size_t bitmap_len); +static struct ovn_dp_group *ovn_dp_group_create( + struct ovsdb_idl_txn *ovnsb_txn, struct hmap *dp_groups, + struct sbrec_logical_dp_group *, size_t desired_n, + const unsigned long *desired_bitmap, + size_t bitmap_len, bool is_switch, + const struct ovn_datapaths *ls_datapaths, + const struct ovn_datapaths *lr_datapaths); +static struct ovn_dp_group *ovn_dp_group_get( + struct hmap *dp_groups, size_t desired_n, + const unsigned long *desired_bitmap, + size_t bitmap_len); +static struct sbrec_logical_dp_group *ovn_sb_insert_or_update_logical_dp_group( + struct ovsdb_idl_txn *ovnsb_txn, + struct sbrec_logical_dp_group *, + const unsigned long *dpg_bitmap, + const struct ovn_datapaths *); +static struct ovn_dp_group *ovn_dp_group_find(const struct hmap *dp_groups, + const unsigned long *dpg_bitmap, + size_t bitmap_len, + uint32_t hash); +static void inc_ovn_dp_group_ref(struct ovn_dp_group *); +static void dec_ovn_dp_group_ref(struct hmap *dp_groups, + struct ovn_dp_group *); +static void ovn_dp_group_add_with_reference(struct ovn_lflow *, + const struct ovn_datapath *od, + const unsigned long *dp_bitmap, + size_t bitmap_len); + +static void unlink_lflows_from_datapath(struct lflow_ref *); +static void lflow_ref_sync_lflows_to_sb__(struct lflow_ref *, + struct lflow_table *, + struct ovsdb_idl_txn *ovnsb_txn, + const struct ovn_datapaths *ls_datapaths, + const struct ovn_datapaths *lr_datapaths, + bool ovn_internal_version_changed, + const struct sbrec_logical_flow_table *, + const struct sbrec_logical_dp_group_table *); +static void sync_lflow_to_sb(struct ovn_lflow *, + struct ovsdb_idl_txn *ovnsb_txn, + struct lflow_table *, + const struct ovn_datapaths *ls_datapaths, + const struct ovn_datapaths *lr_datapaths, + bool ovn_internal_version_changed, + const struct sbrec_logical_flow *sbflow, + const struct sbrec_logical_dp_group_table *); + +extern int parallelization_state; +extern thread_local size_t thread_lflow_counter; + +static bool lflow_hash_lock_initialized = false; +/* The lflow_hash_lock is a mutex array that protects updates to the shared + * lflow table across threads when parallel lflow build and dp-group are both + * enabled. To avoid high contention between threads, a big array of mutexes + * are used instead of just one. This is possible because when parallel build + * is used we only use hmap_insert_fast() to update the hmap, which would not + * touch the bucket array but only the list in a single bucket. We only need to + * make sure that when adding lflows to the same hash bucket, the same lock is + * used, so that no two threads can add to the bucket at the same time. It is + * ok that the same lock is used to protect multiple buckets, so a fixed sized + * mutex array is used instead of 1-1 mapping to the hash buckets. This + * simplies the implementation while effectively reduces lock contention + * because the chance that different threads contending the same lock amongst + * the big number of locks is very low. */ +#define LFLOW_HASH_LOCK_MASK 0xFFFF +static struct ovs_mutex lflow_hash_locks[LFLOW_HASH_LOCK_MASK + 1]; + +/* Full thread safety analysis is not possible with hash locks, because + * they are taken conditionally based on the 'parallelization_state' and + * a flow hash. Also, the order in which two hash locks are taken is not + * predictable during the static analysis. + * + * Since the order of taking two locks depends on a random hash, to avoid + * ABBA deadlocks, no two hash locks can be nested. In that sense an array + * of hash locks is similar to a single mutex. + * + * Using a fake mutex to partially simulate thread safety restrictions, as + * if it were actually a single mutex. + * + * OVS_NO_THREAD_SAFETY_ANALYSIS below allows us to ignore conditional + * nature of the lock. Unlike other attributes, it applies to the + * implementation and not to the interface. So, we can define a function + * that acquires the lock without analysing the way it does that. + */ +extern struct ovs_mutex fake_hash_mutex; + +struct ovn_lflow { + struct hmap_node hmap_node; + + struct ovn_datapath *od; /* 'logical_datapath' in SB schema. */ + unsigned long *dpg_bitmap; /* Bitmap of all datapaths by their 'index'.*/ + enum ovn_stage stage; + uint16_t priority; + char *match; + char *actions; + char *io_port; + char *stage_hint; + char *ctrl_meter; + size_t n_ods; /* Number of datapaths referenced by 'od' and + * 'dpg_bitmap'. */ + struct ovn_dp_group *dpg; /* Link to unique Sb datapath group. */ + + const char *where; + + struct uuid sb_uuid; /* SB DB row uuid, specified by northd. */ + struct uuid lflow_uuid; + + size_t refcnt; +}; + +struct lflow_table { + struct hmap entries; + struct hmap ls_dp_groups; + struct hmap lr_dp_groups; + ssize_t max_seen_lflow_size; +}; + +struct lflow_table * +lflow_table_alloc(void) +{ + struct lflow_table *lflow_table = xzalloc(sizeof *lflow_table); + lflow_table->max_seen_lflow_size = 128; + + return lflow_table; +} + +void +lflow_table_init(struct lflow_table *lflow_table) +{ + fast_hmap_size_for(&lflow_table->entries, + lflow_table->max_seen_lflow_size); + ovn_dp_groups_init(&lflow_table->ls_dp_groups); + ovn_dp_groups_init(&lflow_table->lr_dp_groups); +} + +void +lflow_table_clear(struct lflow_table *lflow_table) +{ + struct ovn_lflow *lflow; + HMAP_FOR_EACH_SAFE (lflow, hmap_node, &lflow_table->entries) { + ovn_lflow_destroy(lflow_table, lflow); + } + hmap_destroy(&lflow_table->entries); + + ovn_dp_groups_destroy(&lflow_table->ls_dp_groups); + ovn_dp_groups_destroy(&lflow_table->lr_dp_groups); +} + +void +lflow_table_destroy(struct lflow_table *lflow_table) +{ + lflow_table_clear(lflow_table); + free(lflow_table); +} + +void +lflow_table_expand(struct lflow_table *lflow_table) +{ + hmap_expand(&lflow_table->entries); + + if (hmap_count(&lflow_table->entries) > + lflow_table->max_seen_lflow_size) { + lflow_table->max_seen_lflow_size = hmap_count(&lflow_table->entries); + } +} + +void +lflow_table_set_size(struct lflow_table *lflow_table, size_t size) +{ + lflow_table->entries.n = size; +} + +void +lflow_table_sync_to_sb(struct lflow_table *lflow_table, + struct ovsdb_idl_txn *ovnsb_txn, + const struct ovn_datapaths *ls_datapaths, + const struct ovn_datapaths *lr_datapaths, + bool ovn_internal_version_changed, + const struct sbrec_logical_flow_table *sb_flow_table, + const struct sbrec_logical_dp_group_table *dpgrp_table) +{ + struct hmap lflows_temp = HMAP_INITIALIZER(&lflows_temp); + struct hmap *lflows = &lflow_table->entries; + struct ovn_lflow *lflow; + + /* Push changes to the Logical_Flow table to database. */ + const struct sbrec_logical_flow *sbflow; + SBREC_LOGICAL_FLOW_TABLE_FOR_EACH_SAFE (sbflow, sb_flow_table) { + struct sbrec_logical_dp_group *dp_group = sbflow->logical_dp_group; + struct ovn_datapath *logical_datapath_od = NULL; + size_t i; + + /* Find one valid datapath to get the datapath type. */ + struct sbrec_datapath_binding *dp = sbflow->logical_datapath; + if (dp) { + logical_datapath_od = ovn_datapath_from_sbrec( + &ls_datapaths->datapaths, + &lr_datapaths->datapaths, + dp); + if (logical_datapath_od + && ovn_datapath_is_stale(logical_datapath_od)) { + logical_datapath_od = NULL; + } + } + for (i = 0; dp_group && i < dp_group->n_datapaths; i++) { + logical_datapath_od = ovn_datapath_from_sbrec( + &ls_datapaths->datapaths, + &lr_datapaths->datapaths, + dp_group->datapaths[i]); + if (logical_datapath_od + && !ovn_datapath_is_stale(logical_datapath_od)) { + break; + } + logical_datapath_od = NULL; + } + + if (!logical_datapath_od) { + /* This lflow has no valid logical datapaths. */ + sbrec_logical_flow_delete(sbflow); + continue; + } + + enum ovn_pipeline pipeline + = !strcmp(sbflow->pipeline, "ingress") ? P_IN : P_OUT; + + lflow = ovn_lflow_find( + lflows, + ovn_stage_build(ovn_datapath_get_type(logical_datapath_od), + pipeline, sbflow->table_id), + sbflow->priority, sbflow->match, sbflow->actions, + sbflow->controller_meter, sbflow->hash); + if (lflow) { + sync_lflow_to_sb(lflow, ovnsb_txn, lflow_table, ls_datapaths, + lr_datapaths, ovn_internal_version_changed, + sbflow, dpgrp_table); + + hmap_remove(lflows, &lflow->hmap_node); + hmap_insert(&lflows_temp, &lflow->hmap_node, + hmap_node_hash(&lflow->hmap_node)); + } else { + sbrec_logical_flow_delete(sbflow); + } + } + + HMAP_FOR_EACH_SAFE (lflow, hmap_node, lflows) { + sync_lflow_to_sb(lflow, ovnsb_txn, lflow_table, ls_datapaths, + lr_datapaths, ovn_internal_version_changed, + NULL, dpgrp_table); + + hmap_remove(lflows, &lflow->hmap_node); + hmap_insert(&lflows_temp, &lflow->hmap_node, + hmap_node_hash(&lflow->hmap_node)); + } + hmap_swap(lflows, &lflows_temp); + hmap_destroy(&lflows_temp); +} + +/* lflow ref mgr */ +struct lflow_ref { + char *res_name; + + /* head of the list 'struct lflow_ref_node'. */ + struct ovs_list lflows_ref_list; + + /* hmapx_node is 'struct lflow *'. This is used to ensure + * that there are no duplicates in 'lflow_ref_list' above. */ + struct hmapx lflows; +}; + +struct lflow_ref_node { + struct ovs_list ref_list_node; + + struct ovn_lflow *lflow; + size_t dp_index; +}; + +struct lflow_ref * +lflow_ref_alloc(const char *res_name) +{ + struct lflow_ref *lflow_ref = xzalloc(sizeof *lflow_ref); + lflow_ref->res_name = xstrdup(res_name); + ovs_list_init(&lflow_ref->lflows_ref_list); + hmapx_init(&lflow_ref->lflows); + return lflow_ref; +} + +void +lflow_ref_destroy(struct lflow_ref *lflow_ref) +{ + free(lflow_ref->res_name); + + struct lflow_ref_node *l; + LIST_FOR_EACH_SAFE (l, ref_list_node, &lflow_ref->lflows_ref_list) { + ovs_list_remove(&l->ref_list_node); + free(l); + } + + hmapx_destroy(&lflow_ref->lflows); + free(lflow_ref); +} + +void +lflow_ref_reset(struct lflow_ref *lflow_ref) +{ + struct lflow_ref_node *l; + LIST_FOR_EACH_SAFE (l, ref_list_node, &lflow_ref->lflows_ref_list) { + ovs_list_remove(&l->ref_list_node); + free(l); + } + + hmapx_clear(&lflow_ref->lflows); +} + +void +lflow_ref_clear_lflows(struct lflow_ref *lflow_ref) +{ + unlink_lflows_from_datapath(lflow_ref); +} + +void +lflow_ref_clear_and_sync_lflows(struct lflow_ref *lflow_ref, + struct lflow_table *lflow_table, + struct ovsdb_idl_txn *ovnsb_txn, + const struct ovn_datapaths *ls_datapaths, + const struct ovn_datapaths *lr_datapaths, + bool ovn_internal_version_changed, + const struct sbrec_logical_flow_table *sbflow_table, + const struct sbrec_logical_dp_group_table *dpgrp_table) +{ + unlink_lflows_from_datapath(lflow_ref); + lflow_ref_sync_lflows_to_sb__(lflow_ref, lflow_table, ovnsb_txn, + ls_datapaths, lr_datapaths, + ovn_internal_version_changed, sbflow_table, + dpgrp_table); +} + +void +lflow_ref_sync_lflows_to_sb(struct lflow_ref *lflow_ref, + struct lflow_table *lflow_table, + struct ovsdb_idl_txn *ovnsb_txn, + const struct ovn_datapaths *ls_datapaths, + const struct ovn_datapaths *lr_datapaths, + bool ovn_internal_version_changed, + const struct sbrec_logical_flow_table *sbflow_table, + const struct sbrec_logical_dp_group_table *dpgrp_table) +{ + lflow_ref_sync_lflows_to_sb__(lflow_ref, lflow_table, ovnsb_txn, + ls_datapaths, lr_datapaths, + ovn_internal_version_changed, sbflow_table, + dpgrp_table); +} + +void +lflow_table_add_lflow(struct lflow_table *lflow_table, + const struct ovn_datapath *od, + const unsigned long *dp_bitmap, size_t dp_bitmap_len, + enum ovn_stage stage, uint16_t priority, + const char *match, const char *actions, + const char *io_port, const char *ctrl_meter, + const struct ovsdb_idl_row *stage_hint, + const char *where, + struct lflow_ref *lflow_ref) + OVS_EXCLUDED(fake_hash_mutex) +{ + struct ovs_mutex *hash_lock; + uint32_t hash; + + ovs_assert(!od || + ovn_stage_to_datapath_type(stage) == ovn_datapath_get_type(od)); + + hash = ovn_logical_flow_hash(ovn_stage_get_table(stage), + ovn_stage_get_pipeline(stage), + priority, match, + actions); + + hash_lock = lflow_hash_lock(&lflow_table->entries, hash); + struct ovn_lflow *lflow = + do_ovn_lflow_add(lflow_table, od, dp_bitmap, + dp_bitmap_len, hash, stage, + priority, match, actions, + io_port, ctrl_meter, stage_hint, where); + + if (lflow_ref) { + if (hmapx_add(&lflow_ref->lflows, lflow)) { + /* lflow_ref_node for this lflow doesn't exist yet. Add it. */ + struct lflow_ref_node *ref_node = xzalloc(sizeof *ref_node); + ref_node->lflow = lflow; + ref_node->dp_index = od->index; + ovs_list_insert(&lflow_ref->lflows_ref_list, + &ref_node->ref_list_node); + + inc_ovn_lflow_ref(lflow); + } + } + + lflow_hash_unlock(hash_lock); + +} + +void +lflow_table_add_lflow_default_drop(struct lflow_table *lflow_table, + const struct ovn_datapath *od, + enum ovn_stage stage, + const char *where, + struct lflow_ref *lflow_ref) +{ + lflow_table_add_lflow(lflow_table, od, NULL, 0, stage, 0, "1", + debug_drop_action(), NULL, NULL, NULL, + where, lflow_ref); +} + +/* Given a desired bitmap, finds a datapath group in 'dp_groups'. If it + * doesn't exist, creates a new one and adds it to 'dp_groups'. + * If 'sb_group' is provided, function will try to re-use this group by + * either taking it directly, or by modifying, if it's not already in use. */ +struct ovn_dp_group * +ovn_dp_group_get_or_create(struct ovsdb_idl_txn *ovnsb_txn, + struct hmap *dp_groups, + struct sbrec_logical_dp_group *sb_group, + size_t desired_n, + const unsigned long *desired_bitmap, + size_t bitmap_len, + bool is_switch, + const struct ovn_datapaths *ls_datapaths, + const struct ovn_datapaths *lr_datapaths) +{ + struct ovn_dp_group *dpg; + + dpg = ovn_dp_group_get(dp_groups, desired_n, desired_bitmap, bitmap_len); + if (dpg) { + return dpg; + } + + return ovn_dp_group_create(ovnsb_txn, dp_groups, sb_group, desired_n, + desired_bitmap, bitmap_len, is_switch, + ls_datapaths, lr_datapaths); +} + +void +ovn_dp_groups_destroy(struct hmap *dp_groups) +{ + struct ovn_dp_group *dpg; + HMAP_FOR_EACH_POP (dpg, node, dp_groups) { + bitmap_free(dpg->bitmap); + free(dpg); + } + hmap_destroy(dp_groups); +} + + +void +lflow_hash_lock_init(void) +{ + if (!lflow_hash_lock_initialized) { + for (size_t i = 0; i < LFLOW_HASH_LOCK_MASK + 1; i++) { + ovs_mutex_init(&lflow_hash_locks[i]); + } + lflow_hash_lock_initialized = true; + } +} + +void +lflow_hash_lock_destroy(void) +{ + if (lflow_hash_lock_initialized) { + for (size_t i = 0; i < LFLOW_HASH_LOCK_MASK + 1; i++) { + ovs_mutex_destroy(&lflow_hash_locks[i]); + } + } + lflow_hash_lock_initialized = false; +} + +/* static functions. */ +static void +ovn_lflow_init(struct ovn_lflow *lflow, struct ovn_datapath *od, + size_t dp_bitmap_len, enum ovn_stage stage, uint16_t priority, + char *match, char *actions, char *io_port, char *ctrl_meter, + char *stage_hint, const char *where, uint32_t hash) +{ + lflow->dpg_bitmap = bitmap_allocate(dp_bitmap_len); + lflow->od = od; + lflow->stage = stage; + lflow->priority = priority; + lflow->match = match; + lflow->actions = actions; + lflow->io_port = io_port; + lflow->stage_hint = stage_hint; + lflow->ctrl_meter = ctrl_meter; + lflow->dpg = NULL; + lflow->where = where; + lflow->sb_uuid = UUID_ZERO; + lflow->lflow_uuid = uuid_random(); + lflow->lflow_uuid.parts[0] = hash; +} + +static struct ovs_mutex * +lflow_hash_lock(const struct hmap *lflow_table, uint32_t hash) + OVS_ACQUIRES(fake_hash_mutex) + OVS_NO_THREAD_SAFETY_ANALYSIS +{ + struct ovs_mutex *hash_lock = NULL; + + if (parallelization_state == STATE_USE_PARALLELIZATION) { + hash_lock = + &lflow_hash_locks[hash & lflow_table->mask & LFLOW_HASH_LOCK_MASK]; + ovs_mutex_lock(hash_lock); + } + return hash_lock; +} + +static void +lflow_hash_unlock(struct ovs_mutex *hash_lock) + OVS_RELEASES(fake_hash_mutex) + OVS_NO_THREAD_SAFETY_ANALYSIS +{ + if (hash_lock) { + ovs_mutex_unlock(hash_lock); + } +} + +static bool +ovn_lflow_equal(const struct ovn_lflow *a, enum ovn_stage stage, + uint16_t priority, const char *match, + const char *actions, const char *ctrl_meter) +{ + return (a->stage == stage + && a->priority == priority + && !strcmp(a->match, match) + && !strcmp(a->actions, actions) + && nullable_string_is_equal(a->ctrl_meter, ctrl_meter)); +} + +static struct ovn_lflow * +ovn_lflow_find(const struct hmap *lflows, + enum ovn_stage stage, uint16_t priority, + const char *match, const char *actions, + const char *ctrl_meter, uint32_t hash) +{ + struct ovn_lflow *lflow; + HMAP_FOR_EACH_WITH_HASH (lflow, hmap_node, hash, lflows) { + if (ovn_lflow_equal(lflow, stage, priority, match, actions, + ctrl_meter)) { + return lflow; + } + } + return NULL; +} + +static char * +ovn_lflow_hint(const struct ovsdb_idl_row *row) +{ + if (!row) { + return NULL; + } + return xasprintf("%08x", row->uuid.parts[0]); +} + +static void +ovn_lflow_destroy(struct lflow_table *lflow_table, struct ovn_lflow *lflow) +{ + if (lflow) { + if (lflow_table) { + hmap_remove(&lflow_table->entries, &lflow->hmap_node); + } + bitmap_free(lflow->dpg_bitmap); + free(lflow->match); + free(lflow->actions); + free(lflow->io_port); + free(lflow->stage_hint); + free(lflow->ctrl_meter); + free(lflow); + } +} + +static +void inc_ovn_lflow_ref(struct ovn_lflow *lflow) +{ + lflow->refcnt++; +} + +static +void dec_ovn_lflow_ref(struct lflow_table *lflow_table, + struct ovn_lflow *lflow) +{ + lflow->refcnt--; + + if (!lflow->refcnt) { + ovn_lflow_destroy(lflow_table, lflow); + } +} + +static struct ovn_lflow * +do_ovn_lflow_add(struct lflow_table *lflow_table, + const struct ovn_datapath *od, + const unsigned long *dp_bitmap, size_t dp_bitmap_len, + uint32_t hash, enum ovn_stage stage, uint16_t priority, + const char *match, const char *actions, + const char *io_port, const char *ctrl_meter, + const struct ovsdb_idl_row *stage_hint, + const char *where) + OVS_REQUIRES(fake_hash_mutex) +{ + struct ovn_lflow *old_lflow; + struct ovn_lflow *lflow; + + size_t bitmap_len = od ? ods_size(od->datapaths) : dp_bitmap_len; + ovs_assert(bitmap_len); + + old_lflow = ovn_lflow_find(&lflow_table->entries, stage, + priority, match, actions, ctrl_meter, hash); + if (old_lflow) { + ovn_dp_group_add_with_reference(old_lflow, od, dp_bitmap, + bitmap_len); + return old_lflow; + } + + lflow = xmalloc(sizeof *lflow); + /* While adding new logical flows we're not setting single datapath, but + * collecting a group. 'od' will be updated later for all flows with only + * one datapath in a group, so it could be hashed correctly. */ + ovn_lflow_init(lflow, NULL, bitmap_len, stage, priority, + xstrdup(match), xstrdup(actions), + io_port ? xstrdup(io_port) : NULL, + nullable_xstrdup(ctrl_meter), + ovn_lflow_hint(stage_hint), where, hash); + + ovn_dp_group_add_with_reference(lflow, od, dp_bitmap, bitmap_len); + + if (parallelization_state != STATE_USE_PARALLELIZATION) { + hmap_insert(&lflow_table->entries, &lflow->hmap_node, hash); + } else { + hmap_insert_fast(&lflow_table->entries, &lflow->hmap_node, + hash); + thread_lflow_counter++; + } + + return lflow; +} + +static void +sync_lflow_to_sb(struct ovn_lflow *lflow, + struct ovsdb_idl_txn *ovnsb_txn, + struct lflow_table *lflow_table, + const struct ovn_datapaths *ls_datapaths, + const struct ovn_datapaths *lr_datapaths, + bool ovn_internal_version_changed, + const struct sbrec_logical_flow *sbflow, + const struct sbrec_logical_dp_group_table *sb_dpgrp_table) +{ + struct sbrec_logical_dp_group *sbrec_dp_group = NULL; + struct ovn_dp_group *pre_sync_dpg = lflow->dpg; + struct ovn_datapath **datapaths_array; + struct hmap *dp_groups; + size_t n_datapaths; + bool is_switch; + + if (ovn_stage_to_datapath_type(lflow->stage) == DP_SWITCH) { + n_datapaths = ods_size(ls_datapaths); + datapaths_array = ls_datapaths->array; + dp_groups = &lflow_table->ls_dp_groups; + is_switch = true; + } else { + n_datapaths = ods_size(lr_datapaths); + datapaths_array = lr_datapaths->array; + dp_groups = &lflow_table->lr_dp_groups; + is_switch = false; + } + + lflow->n_ods = bitmap_count1(lflow->dpg_bitmap, n_datapaths); + ovs_assert(lflow->n_ods); + + if (lflow->n_ods == 1) { + /* There is only one datapath, so it should be moved out of the + * group to a single 'od'. */ + size_t index = bitmap_scan(lflow->dpg_bitmap, true, 0, + n_datapaths); + + lflow->od = datapaths_array[index]; + lflow->dpg = NULL; + } else { + lflow->od = NULL; + } + + if (!sbflow) { + lflow->sb_uuid = uuid_random(); + sbflow = sbrec_logical_flow_insert_persist_uuid(ovnsb_txn, + &lflow->sb_uuid); + const char *pipeline = ovn_stage_get_pipeline_name(lflow->stage); + uint8_t table = ovn_stage_get_table(lflow->stage); + sbrec_logical_flow_set_pipeline(sbflow, pipeline); + sbrec_logical_flow_set_table_id(sbflow, table); + sbrec_logical_flow_set_priority(sbflow, lflow->priority); + sbrec_logical_flow_set_match(sbflow, lflow->match); + sbrec_logical_flow_set_actions(sbflow, lflow->actions); + if (lflow->io_port) { + struct smap tags = SMAP_INITIALIZER(&tags); + smap_add(&tags, "in_out_port", lflow->io_port); + sbrec_logical_flow_set_tags(sbflow, &tags); + smap_destroy(&tags); + } + sbrec_logical_flow_set_controller_meter(sbflow, lflow->ctrl_meter); + + /* Trim the source locator lflow->where, which looks something like + * "ovn/northd/northd.c:1234", down to just the part following the + * last slash, e.g. "northd.c:1234". */ + const char *slash = strrchr(lflow->where, '/'); +#if _WIN32 + const char *backslash = strrchr(lflow->where, '\\'); + if (!slash || backslash > slash) { + slash = backslash; + } +#endif + const char *where = slash ? slash + 1 : lflow->where; + + struct smap ids = SMAP_INITIALIZER(&ids); + smap_add(&ids, "stage-name", ovn_stage_to_str(lflow->stage)); + smap_add(&ids, "source", where); + if (lflow->stage_hint) { + smap_add(&ids, "stage-hint", lflow->stage_hint); + } + sbrec_logical_flow_set_external_ids(sbflow, &ids); + smap_destroy(&ids); + + } else { + lflow->sb_uuid = sbflow->header_.uuid; + sbrec_dp_group = sbflow->logical_dp_group; + + if (ovn_internal_version_changed) { + const char *stage_name = smap_get_def(&sbflow->external_ids, + "stage-name", ""); + const char *stage_hint = smap_get_def(&sbflow->external_ids, + "stage-hint", ""); + const char *source = smap_get_def(&sbflow->external_ids, + "source", ""); + + if (strcmp(stage_name, ovn_stage_to_str(lflow->stage))) { + sbrec_logical_flow_update_external_ids_setkey( + sbflow, "stage-name", ovn_stage_to_str(lflow->stage)); + } + if (lflow->stage_hint) { + if (strcmp(stage_hint, lflow->stage_hint)) { + sbrec_logical_flow_update_external_ids_setkey( + sbflow, "stage-hint", lflow->stage_hint); + } + } + if (lflow->where) { + + /* Trim the source locator lflow->where, which looks something + * like "ovn/northd/northd.c:1234", down to just the part + * following the last slash, e.g. "northd.c:1234". */ + const char *slash = strrchr(lflow->where, '/'); +#if _WIN32 + const char *backslash = strrchr(lflow->where, '\\'); + if (!slash || backslash > slash) { + slash = backslash; + } +#endif + const char *where = slash ? slash + 1 : lflow->where; + + if (strcmp(source, where)) { + sbrec_logical_flow_update_external_ids_setkey( + sbflow, "source", where); + } + } + } + } + + if (lflow->od) { + sbrec_logical_flow_set_logical_datapath(sbflow, lflow->od->sb); + sbrec_logical_flow_set_logical_dp_group(sbflow, NULL); + } else { + sbrec_logical_flow_set_logical_datapath(sbflow, NULL); + lflow->dpg = ovn_dp_group_get(dp_groups, lflow->n_ods, + lflow->dpg_bitmap, + n_datapaths); + if (lflow->dpg) { + /* Update the dpg's sb dp_group. */ + lflow->dpg->dp_group = sbrec_logical_dp_group_table_get_for_uuid( + sb_dpgrp_table, + &lflow->dpg->dpg_uuid); + ovs_assert(lflow->dpg->dp_group); + } else { + lflow->dpg = ovn_dp_group_create( + ovnsb_txn, dp_groups, sbrec_dp_group, + lflow->n_ods, lflow->dpg_bitmap, + n_datapaths, is_switch, + ls_datapaths, + lr_datapaths); + } + sbrec_logical_flow_set_logical_dp_group(sbflow, + lflow->dpg->dp_group); + } + + if (pre_sync_dpg != lflow->dpg) { + if (lflow->dpg) { + inc_ovn_dp_group_ref(lflow->dpg); + } + if (pre_sync_dpg) { + dec_ovn_dp_group_ref(dp_groups, pre_sync_dpg); + } + } +} + +static struct ovn_dp_group * +ovn_dp_group_find(const struct hmap *dp_groups, + const unsigned long *dpg_bitmap, size_t bitmap_len, + uint32_t hash) +{ + struct ovn_dp_group *dpg; + + HMAP_FOR_EACH_WITH_HASH (dpg, node, hash, dp_groups) { + if (bitmap_equal(dpg->bitmap, dpg_bitmap, bitmap_len)) { + return dpg; + } + } + return NULL; +} + +static void +inc_ovn_dp_group_ref(struct ovn_dp_group *dpg) +{ + dpg->refcnt++; +} + +static void +dec_ovn_dp_group_ref(struct hmap *dp_groups, struct ovn_dp_group *dpg) +{ + dpg->refcnt--; + + if (!dpg->refcnt) { + hmap_remove(dp_groups, &dpg->node); + free(dpg->bitmap); + free(dpg); + } +} + +static struct sbrec_logical_dp_group * +ovn_sb_insert_or_update_logical_dp_group( + struct ovsdb_idl_txn *ovnsb_txn, + struct sbrec_logical_dp_group *dp_group, + const unsigned long *dpg_bitmap, + const struct ovn_datapaths *datapaths) +{ + const struct sbrec_datapath_binding **sb; + size_t n = 0, index; + + sb = xmalloc(bitmap_count1(dpg_bitmap, ods_size(datapaths)) * sizeof *sb); + BITMAP_FOR_EACH_1 (index, ods_size(datapaths), dpg_bitmap) { + sb[n++] = datapaths->array[index]->sb; + } + if (!dp_group) { + struct uuid dpg_uuid = uuid_random(); + dp_group = sbrec_logical_dp_group_insert_persist_uuid( + ovnsb_txn, &dpg_uuid); + } + sbrec_logical_dp_group_set_datapaths( + dp_group, (struct sbrec_datapath_binding **) sb, n); + free(sb); + + return dp_group; +} + +static struct ovn_dp_group * +ovn_dp_group_get(struct hmap *dp_groups, size_t desired_n, + const unsigned long *desired_bitmap, + size_t bitmap_len) +{ + uint32_t hash; + + hash = hash_int(desired_n, 0); + return ovn_dp_group_find(dp_groups, desired_bitmap, bitmap_len, hash); +} + +/* Creates a new datapath group and adds it to 'dp_groups'. + * If 'sb_group' is provided, function will try to re-use this group by + * either taking it directly, or by modifying, if it's not already in use. + * Caller should first call ovn_dp_group_get() before calling this function. */ +static struct ovn_dp_group * +ovn_dp_group_create(struct ovsdb_idl_txn *ovnsb_txn, + struct hmap *dp_groups, + struct sbrec_logical_dp_group *sb_group, + size_t desired_n, + const unsigned long *desired_bitmap, + size_t bitmap_len, + bool is_switch, + const struct ovn_datapaths *ls_datapaths, + const struct ovn_datapaths *lr_datapaths) +{ + struct ovn_dp_group *dpg; + + bool update_dp_group = false, can_modify = false; + unsigned long *dpg_bitmap; + size_t i, n = 0; + + dpg_bitmap = sb_group ? bitmap_allocate(bitmap_len) : NULL; + for (i = 0; sb_group && i < sb_group->n_datapaths; i++) { + struct ovn_datapath *datapath_od; + + datapath_od = ovn_datapath_from_sbrec( + ls_datapaths ? &ls_datapaths->datapaths : NULL, + lr_datapaths ? &lr_datapaths->datapaths : NULL, + sb_group->datapaths[i]); + if (!datapath_od || ovn_datapath_is_stale(datapath_od)) { + break; + } + bitmap_set1(dpg_bitmap, datapath_od->index); + n++; + } + if (!sb_group || i != sb_group->n_datapaths) { + /* No group or stale group. Not going to be used. */ + update_dp_group = true; + can_modify = true; + } else if (!bitmap_equal(dpg_bitmap, desired_bitmap, bitmap_len)) { + /* The group in Sb is different. */ + update_dp_group = true; + /* We can modify existing group if it's not already in use. */ + can_modify = !ovn_dp_group_find(dp_groups, dpg_bitmap, + bitmap_len, hash_int(n, 0)); + } + + bitmap_free(dpg_bitmap); + + dpg = xzalloc(sizeof *dpg); + dpg->bitmap = bitmap_clone(desired_bitmap, bitmap_len); + if (!update_dp_group) { + dpg->dp_group = sb_group; + } else { + dpg->dp_group = ovn_sb_insert_or_update_logical_dp_group( + ovnsb_txn, + can_modify ? sb_group : NULL, + desired_bitmap, + is_switch ? ls_datapaths : lr_datapaths); + } + dpg->dpg_uuid = dpg->dp_group->header_.uuid; + hmap_insert(dp_groups, &dpg->node, hash_int(desired_n, 0)); + + return dpg; +} + +/* Adds an OVN datapath to a datapath group of existing logical flow. + * Version to use when hash bucket locking is NOT required or the corresponding + * hash lock is already taken. */ +static void +ovn_dp_group_add_with_reference(struct ovn_lflow *lflow_ref, + const struct ovn_datapath *od, + const unsigned long *dp_bitmap, + size_t bitmap_len) + OVS_REQUIRES(fake_hash_mutex) +{ + if (od) { + bitmap_set1(lflow_ref->dpg_bitmap, od->index); + } + if (dp_bitmap) { + bitmap_or(lflow_ref->dpg_bitmap, dp_bitmap, bitmap_len); + } +} + +/* Unlinks the lflows stored in the resource to object nodes for the + * datapath 'od' from the lflow dependecy manager. + * It basically clears the datapath id of the 'od' for the lflows + * in the 'res_node'. + */ +static void +unlink_lflows_from_datapath(struct lflow_ref *lflow_ref) +{ + struct lflow_ref_node *ref_node; + + LIST_FOR_EACH (ref_node, ref_list_node, &lflow_ref->lflows_ref_list) { + bitmap_set0(ref_node->lflow->dpg_bitmap, ref_node->dp_index); + } +} + +static void +lflow_ref_sync_lflows_to_sb__(struct lflow_ref *lflow_ref, + struct lflow_table *lflow_table, + struct ovsdb_idl_txn *ovnsb_txn, + const struct ovn_datapaths *ls_datapaths, + const struct ovn_datapaths *lr_datapaths, + bool ovn_internal_version_changed, + const struct sbrec_logical_flow_table *sbflow_table, + const struct sbrec_logical_dp_group_table *dpgrp_table) +{ + struct lflow_ref_node *lrn; + struct ovn_lflow *lflow; + LIST_FOR_EACH (lrn, ref_list_node, &lflow_ref->lflows_ref_list) { + lflow = lrn->lflow; + + const struct sbrec_logical_flow *sblflow = + sbrec_logical_flow_table_get_for_uuid(sbflow_table, + &lflow->sb_uuid); + + size_t n_datapaths; + if (ovn_stage_to_datapath_type(lflow->stage) == DP_SWITCH) { + n_datapaths = ods_size(ls_datapaths); + } else { + n_datapaths = ods_size(lr_datapaths); + } + + size_t n_ods = bitmap_count1(lflow->dpg_bitmap, n_datapaths); + + if (n_ods) { + sync_lflow_to_sb(lflow, ovnsb_txn, lflow_table, ls_datapaths, + lr_datapaths, ovn_internal_version_changed, + sblflow, dpgrp_table); + } else { + if (sblflow) { + sbrec_logical_flow_delete(sblflow); + dec_ovn_lflow_ref(lflow_table, lflow); + } + + lrn->lflow = NULL; + hmapx_find_and_delete(&lflow_ref->lflows, lflow); + } + } + + LIST_FOR_EACH_SAFE (lrn, ref_list_node, &lflow_ref->lflows_ref_list) { + if (!lrn->lflow) { + ovs_list_remove(&lrn->ref_list_node); + free(lrn); + } + } +} diff --git a/northd/lflow-mgr.h b/northd/lflow-mgr.h new file mode 100644 index 0000000000..7809c939e6 --- /dev/null +++ b/northd/lflow-mgr.h @@ -0,0 +1,186 @@ + /* + * Copyright (c) 2023, Red Hat, Inc. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at: + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +#ifndef LFLOW_MGR_H +#define LFLOW_MGR_H 1 + +#include "include/openvswitch/hmap.h" +#include "include/openvswitch/uuid.h" + +#include "northd.h" + +struct ovsdb_idl_txn; +struct ovn_datapath; +struct ovsdb_idl_row; + +/* lflow map which stores the logical flows. */ +struct lflow_table; +struct lflow_table *lflow_table_alloc(void); +void lflow_table_init(struct lflow_table *); +void lflow_table_clear(struct lflow_table *); +void lflow_table_destroy(struct lflow_table *); +void lflow_table_expand(struct lflow_table *); +void lflow_table_set_size(struct lflow_table *, size_t); +void lflow_table_sync_to_sb(struct lflow_table *, + struct ovsdb_idl_txn *ovnsb_txn, + const struct ovn_datapaths *ls_datapaths, + const struct ovn_datapaths *lr_datapaths, + bool ovn_internal_version_changed, + const struct sbrec_logical_flow_table *, + const struct sbrec_logical_dp_group_table *); +void lflow_table_destroy(struct lflow_table *); + +void lflow_hash_lock_init(void); +void lflow_hash_lock_destroy(void); + +/* lflow_mgr manages logical flows for a resource (like logical port + * or datapath). */ +struct lflow_ref; + +/* Allocates an lflow manager. */ +struct lflow_ref *lflow_ref_alloc(const char *res_name); +void lflow_ref_destroy(struct lflow_ref *); +void lflow_ref_reset(struct lflow_ref *lflow_ref); +void lflow_ref_clear_lflows(struct lflow_ref *); +void lflow_ref_clear_and_sync_lflows(struct lflow_ref *, + struct lflow_table *lflow_table, + struct ovsdb_idl_txn *ovnsb_txn, + const struct ovn_datapaths *ls_datapaths, + const struct ovn_datapaths *lr_datapaths, + bool ovn_internal_version_changed, + const struct sbrec_logical_flow_table *, + const struct sbrec_logical_dp_group_table *); +void lflow_ref_sync_lflows_to_sb(struct lflow_ref *, + struct lflow_table *lflow_table, + struct ovsdb_idl_txn *ovnsb_txn, + const struct ovn_datapaths *ls_datapaths, + const struct ovn_datapaths *lr_datapaths, + bool ovn_internal_version_changed, + const struct sbrec_logical_flow_table *, + const struct sbrec_logical_dp_group_table *); + + +void lflow_table_add_lflow(struct lflow_table *, const struct ovn_datapath *, + const unsigned long *dp_bitmap, + size_t dp_bitmap_len, enum ovn_stage stage, + uint16_t priority, const char *match, + const char *actions, const char *io_port, + const char *ctrl_meter, + const struct ovsdb_idl_row *stage_hint, + const char *where, struct lflow_ref *); +void lflow_table_add_lflow_default_drop(struct lflow_table *, + const struct ovn_datapath *, + enum ovn_stage stage, + const char *where, + struct lflow_ref *); + +/* Adds a row with the specified contents to the Logical_Flow table. */ +#define ovn_lflow_add_with_hint__(LFLOW_TABLE, OD, STAGE, PRIORITY, MATCH, \ + ACTIONS, IN_OUT_PORT, CTRL_METER, \ + STAGE_HINT) \ + lflow_table_add_lflow(LFLOW_TABLE, OD, NULL, 0, STAGE, PRIORITY, MATCH, \ + ACTIONS, IN_OUT_PORT, CTRL_METER, STAGE_HINT, \ + OVS_SOURCE_LOCATOR, NULL) + +#define ovn_lflow_add_with_lflow_ref_hint__(LFLOW_TABLE, OD, STAGE, PRIORITY, \ + MATCH, ACTIONS, IN_OUT_PORT, \ + CTRL_METER, STAGE_HINT, LFLOW_REF)\ + lflow_table_add_lflow(LFLOW_TABLE, OD, NULL, 0, STAGE, PRIORITY, MATCH, \ + ACTIONS, IN_OUT_PORT, CTRL_METER, STAGE_HINT, \ + OVS_SOURCE_LOCATOR, LFLOW_REF) + +#define ovn_lflow_add_with_hint(LFLOW_TABLE, OD, STAGE, PRIORITY, MATCH, \ + ACTIONS, STAGE_HINT) \ + lflow_table_add_lflow(LFLOW_TABLE, OD, NULL, 0, STAGE, PRIORITY, MATCH, \ + ACTIONS, NULL, NULL, STAGE_HINT, \ + OVS_SOURCE_LOCATOR, NULL) + +#define ovn_lflow_add_with_lflow_ref_hint(LFLOW_TABLE, OD, STAGE, PRIORITY, \ + MATCH, ACTIONS, STAGE_HINT, \ + LFLOW_REF) \ + lflow_table_add_lflow(LFLOW_TABLE, OD, NULL, 0, STAGE, PRIORITY, MATCH, \ + ACTIONS, NULL, NULL, STAGE_HINT, \ + OVS_SOURCE_LOCATOR, LFLOW_REF) + +#define ovn_lflow_add_with_dp_group(LFLOW_TABLE, DP_BITMAP, DP_BITMAP_LEN, \ + STAGE, PRIORITY, MATCH, ACTIONS, \ + STAGE_HINT) \ + lflow_table_add_lflow(LFLOW_TABLE, NULL, DP_BITMAP, DP_BITMAP_LEN, STAGE, \ + PRIORITY, MATCH, ACTIONS, NULL, NULL, STAGE_HINT, \ + OVS_SOURCE_LOCATOR, NULL) + +#define ovn_lflow_add_default_drop(LFLOW_TABLE, OD, STAGE) \ + lflow_table_add_lflow_default_drop(LFLOW_TABLE, OD, STAGE, \ + OVS_SOURCE_LOCATOR, NULL) + + +/* This macro is similar to ovn_lflow_add_with_hint, except that it requires + * the IN_OUT_PORT argument, which tells the lport name that appears in the + * MATCH, which helps ovn-controller to bypass lflows parsing when the lport is + * not local to the chassis. The critiera of the lport to be added using this + * argument: + * + * - For ingress pipeline, the lport that is used to match "inport". + * - For egress pipeline, the lport that is used to match "outport". + * + * For now, only LS pipelines should use this macro. */ +#define ovn_lflow_add_with_lport_and_hint(LFLOW_TABLE, OD, STAGE, PRIORITY, \ + MATCH, ACTIONS, IN_OUT_PORT, \ + STAGE_HINT, LFLOW_REF) \ + lflow_table_add_lflow(LFLOW_TABLE, OD, NULL, 0, STAGE, PRIORITY, MATCH, \ + ACTIONS, IN_OUT_PORT, NULL, STAGE_HINT, \ + OVS_SOURCE_LOCATOR, LFLOW_REF) + +#define ovn_lflow_add(LFLOW_TABLE, OD, STAGE, PRIORITY, MATCH, ACTIONS) \ + lflow_table_add_lflow(LFLOW_TABLE, OD, NULL, 0, STAGE, PRIORITY, MATCH, \ + ACTIONS, NULL, NULL, NULL, OVS_SOURCE_LOCATOR, NULL) + +#define ovn_lflow_add_with_lflow_ref(LFLOW_TABLE, OD, STAGE, PRIORITY, MATCH, \ + ACTIONS, LFLOW_REF) \ + lflow_table_add_lflow(LFLOW_TABLE, OD, NULL, 0, STAGE, PRIORITY, MATCH, \ + ACTIONS, NULL, NULL, NULL, OVS_SOURCE_LOCATOR, \ + LFLOW_REF) + +#define ovn_lflow_metered(LFLOW_TABLE, OD, STAGE, PRIORITY, MATCH, ACTIONS, \ + CTRL_METER) \ + ovn_lflow_add_with_hint__(LFLOW_TABLE, OD, STAGE, PRIORITY, MATCH, \ + ACTIONS, NULL, CTRL_METER, NULL) + +struct sbrec_logical_dp_group; + +struct ovn_dp_group { + unsigned long *bitmap; + const struct sbrec_logical_dp_group *dp_group; + struct uuid dpg_uuid; + struct hmap_node node; + size_t refcnt; +}; + +static inline void +ovn_dp_groups_init(struct hmap *dp_groups) +{ + hmap_init(dp_groups); +} + +void ovn_dp_groups_destroy(struct hmap *dp_groups); +struct ovn_dp_group *ovn_dp_group_get_or_create( + struct ovsdb_idl_txn *ovnsb_txn, struct hmap *dp_groups, + struct sbrec_logical_dp_group *sb_group, + size_t desired_n, const unsigned long *desired_bitmap, + size_t bitmap_len, bool is_switch, + const struct ovn_datapaths *ls_datapaths, + const struct ovn_datapaths *lr_datapaths); + +#endif /* LFLOW_MGR_H */ \ No newline at end of file diff --git a/northd/northd.c b/northd/northd.c index c8e2bd4790..13d0db57e2 100644 --- a/northd/northd.c +++ b/northd/northd.c @@ -40,6 +40,7 @@ #include "lib/ovn-sb-idl.h" #include "lib/ovn-util.h" #include "lib/lb.h" +#include "lflow-mgr.h" #include "memory.h" #include "northd.h" #include "en-lb-data.h" @@ -67,7 +68,7 @@ VLOG_DEFINE_THIS_MODULE(northd); static bool controller_event_en; -static bool lflow_hash_lock_initialized = false; + static bool check_lsp_is_up; @@ -96,116 +97,6 @@ static bool default_acl_drop; #define MAX_OVN_TAGS 4096 -/* Pipeline stages. */ - -/* The two purposes for which ovn-northd uses OVN logical datapaths. */ -enum ovn_datapath_type { - DP_SWITCH, /* OVN logical switch. */ - DP_ROUTER /* OVN logical router. */ -}; - -/* Returns an "enum ovn_stage" built from the arguments. - * - * (It's better to use ovn_stage_build() for type-safety reasons, but inline - * functions can't be used in enums or switch cases.) */ -#define OVN_STAGE_BUILD(DP_TYPE, PIPELINE, TABLE) \ - (((DP_TYPE) << 9) | ((PIPELINE) << 8) | (TABLE)) - -/* A stage within an OVN logical switch or router. - * - * An "enum ovn_stage" indicates whether the stage is part of a logical switch - * or router, whether the stage is part of the ingress or egress pipeline, and - * the table within that pipeline. The first three components are combined to - * form the stage's full name, e.g. S_SWITCH_IN_PORT_SEC_L2, - * S_ROUTER_OUT_DELIVERY. */ -enum ovn_stage { -#define PIPELINE_STAGES \ - /* Logical switch ingress stages. */ \ - PIPELINE_STAGE(SWITCH, IN, CHECK_PORT_SEC, 0, "ls_in_check_port_sec") \ - PIPELINE_STAGE(SWITCH, IN, APPLY_PORT_SEC, 1, "ls_in_apply_port_sec") \ - PIPELINE_STAGE(SWITCH, IN, LOOKUP_FDB , 2, "ls_in_lookup_fdb") \ - PIPELINE_STAGE(SWITCH, IN, PUT_FDB, 3, "ls_in_put_fdb") \ - PIPELINE_STAGE(SWITCH, IN, PRE_ACL, 4, "ls_in_pre_acl") \ - PIPELINE_STAGE(SWITCH, IN, PRE_LB, 5, "ls_in_pre_lb") \ - PIPELINE_STAGE(SWITCH, IN, PRE_STATEFUL, 6, "ls_in_pre_stateful") \ - PIPELINE_STAGE(SWITCH, IN, ACL_HINT, 7, "ls_in_acl_hint") \ - PIPELINE_STAGE(SWITCH, IN, ACL_EVAL, 8, "ls_in_acl_eval") \ - PIPELINE_STAGE(SWITCH, IN, ACL_ACTION, 9, "ls_in_acl_action") \ - PIPELINE_STAGE(SWITCH, IN, QOS_MARK, 10, "ls_in_qos_mark") \ - PIPELINE_STAGE(SWITCH, IN, QOS_METER, 11, "ls_in_qos_meter") \ - PIPELINE_STAGE(SWITCH, IN, LB_AFF_CHECK, 12, "ls_in_lb_aff_check") \ - PIPELINE_STAGE(SWITCH, IN, LB, 13, "ls_in_lb") \ - PIPELINE_STAGE(SWITCH, IN, LB_AFF_LEARN, 14, "ls_in_lb_aff_learn") \ - PIPELINE_STAGE(SWITCH, IN, PRE_HAIRPIN, 15, "ls_in_pre_hairpin") \ - PIPELINE_STAGE(SWITCH, IN, NAT_HAIRPIN, 16, "ls_in_nat_hairpin") \ - PIPELINE_STAGE(SWITCH, IN, HAIRPIN, 17, "ls_in_hairpin") \ - PIPELINE_STAGE(SWITCH, IN, ACL_AFTER_LB_EVAL, 18, \ - "ls_in_acl_after_lb_eval") \ - PIPELINE_STAGE(SWITCH, IN, ACL_AFTER_LB_ACTION, 19, \ - "ls_in_acl_after_lb_action") \ - PIPELINE_STAGE(SWITCH, IN, STATEFUL, 20, "ls_in_stateful") \ - PIPELINE_STAGE(SWITCH, IN, ARP_ND_RSP, 21, "ls_in_arp_rsp") \ - PIPELINE_STAGE(SWITCH, IN, DHCP_OPTIONS, 22, "ls_in_dhcp_options") \ - PIPELINE_STAGE(SWITCH, IN, DHCP_RESPONSE, 23, "ls_in_dhcp_response") \ - PIPELINE_STAGE(SWITCH, IN, DNS_LOOKUP, 24, "ls_in_dns_lookup") \ - PIPELINE_STAGE(SWITCH, IN, DNS_RESPONSE, 25, "ls_in_dns_response") \ - PIPELINE_STAGE(SWITCH, IN, EXTERNAL_PORT, 26, "ls_in_external_port") \ - PIPELINE_STAGE(SWITCH, IN, L2_LKUP, 27, "ls_in_l2_lkup") \ - PIPELINE_STAGE(SWITCH, IN, L2_UNKNOWN, 28, "ls_in_l2_unknown") \ - \ - /* Logical switch egress stages. */ \ - PIPELINE_STAGE(SWITCH, OUT, PRE_ACL, 0, "ls_out_pre_acl") \ - PIPELINE_STAGE(SWITCH, OUT, PRE_LB, 1, "ls_out_pre_lb") \ - PIPELINE_STAGE(SWITCH, OUT, PRE_STATEFUL, 2, "ls_out_pre_stateful") \ - PIPELINE_STAGE(SWITCH, OUT, ACL_HINT, 3, "ls_out_acl_hint") \ - PIPELINE_STAGE(SWITCH, OUT, ACL_EVAL, 4, "ls_out_acl_eval") \ - PIPELINE_STAGE(SWITCH, OUT, ACL_ACTION, 5, "ls_out_acl_action") \ - PIPELINE_STAGE(SWITCH, OUT, QOS_MARK, 6, "ls_out_qos_mark") \ - PIPELINE_STAGE(SWITCH, OUT, QOS_METER, 7, "ls_out_qos_meter") \ - PIPELINE_STAGE(SWITCH, OUT, STATEFUL, 8, "ls_out_stateful") \ - PIPELINE_STAGE(SWITCH, OUT, CHECK_PORT_SEC, 9, "ls_out_check_port_sec") \ - PIPELINE_STAGE(SWITCH, OUT, APPLY_PORT_SEC, 10, "ls_out_apply_port_sec") \ - \ - /* Logical router ingress stages. */ \ - PIPELINE_STAGE(ROUTER, IN, ADMISSION, 0, "lr_in_admission") \ - PIPELINE_STAGE(ROUTER, IN, LOOKUP_NEIGHBOR, 1, "lr_in_lookup_neighbor") \ - PIPELINE_STAGE(ROUTER, IN, LEARN_NEIGHBOR, 2, "lr_in_learn_neighbor") \ - PIPELINE_STAGE(ROUTER, IN, IP_INPUT, 3, "lr_in_ip_input") \ - PIPELINE_STAGE(ROUTER, IN, UNSNAT, 4, "lr_in_unsnat") \ - PIPELINE_STAGE(ROUTER, IN, DEFRAG, 5, "lr_in_defrag") \ - PIPELINE_STAGE(ROUTER, IN, LB_AFF_CHECK, 6, "lr_in_lb_aff_check") \ - PIPELINE_STAGE(ROUTER, IN, DNAT, 7, "lr_in_dnat") \ - PIPELINE_STAGE(ROUTER, IN, LB_AFF_LEARN, 8, "lr_in_lb_aff_learn") \ - PIPELINE_STAGE(ROUTER, IN, ECMP_STATEFUL, 9, "lr_in_ecmp_stateful") \ - PIPELINE_STAGE(ROUTER, IN, ND_RA_OPTIONS, 10, "lr_in_nd_ra_options") \ - PIPELINE_STAGE(ROUTER, IN, ND_RA_RESPONSE, 11, "lr_in_nd_ra_response") \ - PIPELINE_STAGE(ROUTER, IN, IP_ROUTING_PRE, 12, "lr_in_ip_routing_pre") \ - PIPELINE_STAGE(ROUTER, IN, IP_ROUTING, 13, "lr_in_ip_routing") \ - PIPELINE_STAGE(ROUTER, IN, IP_ROUTING_ECMP, 14, "lr_in_ip_routing_ecmp") \ - PIPELINE_STAGE(ROUTER, IN, POLICY, 15, "lr_in_policy") \ - PIPELINE_STAGE(ROUTER, IN, POLICY_ECMP, 16, "lr_in_policy_ecmp") \ - PIPELINE_STAGE(ROUTER, IN, ARP_RESOLVE, 17, "lr_in_arp_resolve") \ - PIPELINE_STAGE(ROUTER, IN, CHK_PKT_LEN, 18, "lr_in_chk_pkt_len") \ - PIPELINE_STAGE(ROUTER, IN, LARGER_PKTS, 19, "lr_in_larger_pkts") \ - PIPELINE_STAGE(ROUTER, IN, GW_REDIRECT, 20, "lr_in_gw_redirect") \ - PIPELINE_STAGE(ROUTER, IN, ARP_REQUEST, 21, "lr_in_arp_request") \ - \ - /* Logical router egress stages. */ \ - PIPELINE_STAGE(ROUTER, OUT, CHECK_DNAT_LOCAL, 0, \ - "lr_out_chk_dnat_local") \ - PIPELINE_STAGE(ROUTER, OUT, UNDNAT, 1, "lr_out_undnat") \ - PIPELINE_STAGE(ROUTER, OUT, POST_UNDNAT, 2, "lr_out_post_undnat") \ - PIPELINE_STAGE(ROUTER, OUT, SNAT, 3, "lr_out_snat") \ - PIPELINE_STAGE(ROUTER, OUT, POST_SNAT, 4, "lr_out_post_snat") \ - PIPELINE_STAGE(ROUTER, OUT, EGR_LOOP, 5, "lr_out_egr_loop") \ - PIPELINE_STAGE(ROUTER, OUT, DELIVERY, 6, "lr_out_delivery") - -#define PIPELINE_STAGE(DP_TYPE, PIPELINE, STAGE, TABLE, NAME) \ - S_##DP_TYPE##_##PIPELINE##_##STAGE \ - = OVN_STAGE_BUILD(DP_##DP_TYPE, P_##PIPELINE, TABLE), - PIPELINE_STAGES -#undef PIPELINE_STAGE -}; /* Due to various hard-coded priorities need to implement ACLs, the * northbound database supports a smaller range of ACL priorities than @@ -390,51 +281,9 @@ enum ovn_stage { #define ROUTE_PRIO_OFFSET_STATIC 1 #define ROUTE_PRIO_OFFSET_CONNECTED 2 -/* Returns an "enum ovn_stage" built from the arguments. */ -static enum ovn_stage -ovn_stage_build(enum ovn_datapath_type dp_type, enum ovn_pipeline pipeline, - uint8_t table) -{ - return OVN_STAGE_BUILD(dp_type, pipeline, table); -} - -/* Returns the pipeline to which 'stage' belongs. */ -static enum ovn_pipeline -ovn_stage_get_pipeline(enum ovn_stage stage) -{ - return (stage >> 8) & 1; -} - -/* Returns the pipeline name to which 'stage' belongs. */ -static const char * -ovn_stage_get_pipeline_name(enum ovn_stage stage) -{ - return ovn_stage_get_pipeline(stage) == P_IN ? "ingress" : "egress"; -} - -/* Returns the table to which 'stage' belongs. */ -static uint8_t -ovn_stage_get_table(enum ovn_stage stage) -{ - return stage & 0xff; -} - -/* Returns a string name for 'stage'. */ -static const char * -ovn_stage_to_str(enum ovn_stage stage) -{ - switch (stage) { -#define PIPELINE_STAGE(DP_TYPE, PIPELINE, STAGE, TABLE, NAME) \ - case S_##DP_TYPE##_##PIPELINE##_##STAGE: return NAME; - PIPELINE_STAGES -#undef PIPELINE_STAGE - default: return ""; - } -} - /* Returns the type of the datapath to which a flow with the given 'stage' may * be added. */ -static enum ovn_datapath_type +enum ovn_datapath_type ovn_stage_to_datapath_type(enum ovn_stage stage) { switch (stage) { @@ -679,13 +528,6 @@ ovn_datapath_destroy(struct hmap *datapaths, struct ovn_datapath *od) } } -/* Returns 'od''s datapath type. */ -static enum ovn_datapath_type -ovn_datapath_get_type(const struct ovn_datapath *od) -{ - return od->nbs ? DP_SWITCH : DP_ROUTER; -} - static struct ovn_datapath * ovn_datapath_find_(const struct hmap *datapaths, const struct uuid *uuid) @@ -721,13 +563,7 @@ ovn_datapath_find_by_key(struct hmap *datapaths, uint32_t dp_key) return NULL; } -static bool -ovn_datapath_is_stale(const struct ovn_datapath *od) -{ - return !od->nbr && !od->nbs; -} - -static struct ovn_datapath * +struct ovn_datapath * ovn_datapath_from_sbrec(const struct hmap *ls_datapaths, const struct hmap *lr_datapaths, const struct sbrec_datapath_binding *sb) @@ -1290,19 +1126,6 @@ struct ovn_port_routable_addresses { size_t n_addrs; }; -/* A node that maintains link between an object (such as an ovn_port) and - * a lflow. */ -struct lflow_ref_node { - /* This list follows different lflows referenced by the same object. List - * head is, for example, ovn_port->lflows. */ - struct ovs_list lflow_list_node; - /* This list follows different objects that reference the same lflow. List - * head is ovn_lflow->referenced_by. */ - struct ovs_list ref_list_node; - /* The lflow. */ - struct ovn_lflow *lflow; -}; - static bool lsp_can_be_inc_processed(const struct nbrec_logical_switch_port *); static bool @@ -1382,6 +1205,8 @@ ovn_port_set_nb(struct ovn_port *op, init_mcast_port_info(&op->mcast_info, op->nbsp, op->nbrp); } +static bool lsp_is_router(const struct nbrec_logical_switch_port *nbsp); + static struct ovn_port * ovn_port_create(struct hmap *ports, const char *key, const struct nbrec_logical_switch_port *nbsp, @@ -1400,12 +1225,14 @@ ovn_port_create(struct hmap *ports, const char *key, op->l3dgw_port = op->cr_port = NULL; hmap_insert(ports, &op->key_node, hash_string(op->key, 0)); - ovs_list_init(&op->lflows); + op->lflow_ref = lflow_ref_alloc(key); + op->stateful_lflow_ref = lflow_ref_alloc(key); + return op; } static void -ovn_port_destroy_orphan(struct ovn_port *port) +ovn_port_cleanup(struct ovn_port *port) { if (port->tunnel_key) { ovs_assert(port->od); @@ -1415,6 +1242,8 @@ ovn_port_destroy_orphan(struct ovn_port *port) destroy_lport_addresses(&port->lsp_addrs[i]); } free(port->lsp_addrs); + port->n_lsp_addrs = 0; + port->lsp_addrs = NULL; if (port->peer) { port->peer->peer = NULL; @@ -1424,18 +1253,22 @@ ovn_port_destroy_orphan(struct ovn_port *port) destroy_lport_addresses(&port->ps_addrs[i]); } free(port->ps_addrs); + port->ps_addrs = NULL; + port->n_ps_addrs = 0; destroy_lport_addresses(&port->lrp_networks); destroy_lport_addresses(&port->proxy_arp_addrs); +} + +static void +ovn_port_destroy_orphan(struct ovn_port *port) +{ + ovn_port_cleanup(port); free(port->json_key); free(port->key); + lflow_ref_destroy(port->lflow_ref); + lflow_ref_destroy(port->stateful_lflow_ref); - struct lflow_ref_node *l; - LIST_FOR_EACH_SAFE (l, lflow_list_node, &port->lflows) { - ovs_list_remove(&l->lflow_list_node); - ovs_list_remove(&l->ref_list_node); - free(l); - } free(port); } @@ -3898,124 +3731,6 @@ build_lb_port_related_data( build_lswitch_lbs_from_lrouter(lr_datapaths, lb_dps_map, lb_group_dps_map); } - -struct ovn_dp_group { - unsigned long *bitmap; - struct sbrec_logical_dp_group *dp_group; - struct hmap_node node; -}; - -static struct ovn_dp_group * -ovn_dp_group_find(const struct hmap *dp_groups, - const unsigned long *dpg_bitmap, size_t bitmap_len, - uint32_t hash) -{ - struct ovn_dp_group *dpg; - - HMAP_FOR_EACH_WITH_HASH (dpg, node, hash, dp_groups) { - if (bitmap_equal(dpg->bitmap, dpg_bitmap, bitmap_len)) { - return dpg; - } - } - return NULL; -} - -static struct sbrec_logical_dp_group * -ovn_sb_insert_or_update_logical_dp_group( - struct ovsdb_idl_txn *ovnsb_txn, - struct sbrec_logical_dp_group *dp_group, - const unsigned long *dpg_bitmap, - const struct ovn_datapaths *datapaths) -{ - const struct sbrec_datapath_binding **sb; - size_t n = 0, index; - - sb = xmalloc(bitmap_count1(dpg_bitmap, ods_size(datapaths)) * sizeof *sb); - BITMAP_FOR_EACH_1 (index, ods_size(datapaths), dpg_bitmap) { - sb[n++] = datapaths->array[index]->sb; - } - if (!dp_group) { - dp_group = sbrec_logical_dp_group_insert(ovnsb_txn); - } - sbrec_logical_dp_group_set_datapaths( - dp_group, (struct sbrec_datapath_binding **) sb, n); - free(sb); - - return dp_group; -} - -/* Given a desired bitmap, finds a datapath group in 'dp_groups'. If it - * doesn't exist, creates a new one and adds it to 'dp_groups'. - * If 'sb_group' is provided, function will try to re-use this group by - * either taking it directly, or by modifying, if it's not already in use. */ -static struct ovn_dp_group * -ovn_dp_group_get_or_create(struct ovsdb_idl_txn *ovnsb_txn, - struct hmap *dp_groups, - struct sbrec_logical_dp_group *sb_group, - size_t desired_n, - const unsigned long *desired_bitmap, - size_t bitmap_len, - bool is_switch, - const struct ovn_datapaths *ls_datapaths, - const struct ovn_datapaths *lr_datapaths) -{ - struct ovn_dp_group *dpg; - uint32_t hash; - - hash = hash_int(desired_n, 0); - dpg = ovn_dp_group_find(dp_groups, desired_bitmap, bitmap_len, hash); - if (dpg) { - return dpg; - } - - bool update_dp_group = false, can_modify = false; - unsigned long *dpg_bitmap; - size_t i, n = 0; - - dpg_bitmap = sb_group ? bitmap_allocate(bitmap_len) : NULL; - for (i = 0; sb_group && i < sb_group->n_datapaths; i++) { - struct ovn_datapath *datapath_od; - - datapath_od = ovn_datapath_from_sbrec( - ls_datapaths ? &ls_datapaths->datapaths : NULL, - lr_datapaths ? &lr_datapaths->datapaths : NULL, - sb_group->datapaths[i]); - if (!datapath_od || ovn_datapath_is_stale(datapath_od)) { - break; - } - bitmap_set1(dpg_bitmap, datapath_od->index); - n++; - } - if (!sb_group || i != sb_group->n_datapaths) { - /* No group or stale group. Not going to be used. */ - update_dp_group = true; - can_modify = true; - } else if (!bitmap_equal(dpg_bitmap, desired_bitmap, bitmap_len)) { - /* The group in Sb is different. */ - update_dp_group = true; - /* We can modify existing group if it's not already in use. */ - can_modify = !ovn_dp_group_find(dp_groups, dpg_bitmap, - bitmap_len, hash_int(n, 0)); - } - - bitmap_free(dpg_bitmap); - - dpg = xzalloc(sizeof *dpg); - dpg->bitmap = bitmap_clone(desired_bitmap, bitmap_len); - if (!update_dp_group) { - dpg->dp_group = sb_group; - } else { - dpg->dp_group = ovn_sb_insert_or_update_logical_dp_group( - ovnsb_txn, - can_modify ? sb_group : NULL, - desired_bitmap, - is_switch ? ls_datapaths : lr_datapaths); - } - hmap_insert(dp_groups, &dpg->node, hash); - - return dpg; -} - struct sb_lb { struct hmap_node hmap_node; @@ -4835,28 +4550,20 @@ ovn_port_find_in_datapath(struct ovn_datapath *od, const char *name) return NULL; } -static struct ovn_port * -ls_port_create(struct ovsdb_idl_txn *ovnsb_txn, struct hmap *ls_ports, - const char *key, const struct nbrec_logical_switch_port *nbsp, - struct ovn_datapath *od, const struct sbrec_port_binding *sb, - struct ovs_list *lflows, - const struct sbrec_mirror_table *sbrec_mirror_table, - const struct sbrec_chassis_table *sbrec_chassis_table, - struct ovsdb_idl_index *sbrec_chassis_by_name, - struct ovsdb_idl_index *sbrec_chassis_by_hostname) +static bool +ls_port_init(struct ovn_port *op, struct ovsdb_idl_txn *ovnsb_txn, + struct hmap *ls_ports, struct ovn_datapath *od, + const struct sbrec_port_binding *sb, + const struct sbrec_mirror_table *sbrec_mirror_table, + const struct sbrec_chassis_table *sbrec_chassis_table, + struct ovsdb_idl_index *sbrec_chassis_by_name, + struct ovsdb_idl_index *sbrec_chassis_by_hostname) { - struct ovn_port *op = ovn_port_create(ls_ports, key, nbsp, NULL, - NULL); - parse_lsp_addrs(op); op->od = od; - hmap_insert(&od->ports, &op->dp_node, hmap_node_hash(&op->key_node)); - if (lflows) { - ovs_list_splice(&op->lflows, lflows->next, lflows); - } - + parse_lsp_addrs(op); /* Assign explicitly requested tunnel ids first. */ if (!ovn_port_assign_requested_tnl_id(sbrec_chassis_table, op)) { - return NULL; + return false; } if (sb) { op->sb = sb; @@ -4873,14 +4580,57 @@ ls_port_create(struct ovsdb_idl_txn *ovnsb_txn, struct hmap *ls_ports, } /* Assign new tunnel ids where needed. */ if (!ovn_port_allocate_key(sbrec_chassis_table, ls_ports, op)) { - return NULL; + return false; } ovn_port_update_sbrec(ovnsb_txn, sbrec_chassis_by_name, sbrec_chassis_by_hostname, NULL, sbrec_mirror_table, op, NULL, NULL); + return true; +} + +static struct ovn_port * +ls_port_create(struct ovsdb_idl_txn *ovnsb_txn, struct hmap *ls_ports, + const char *key, const struct nbrec_logical_switch_port *nbsp, + struct ovn_datapath *od, const struct sbrec_port_binding *sb, + const struct sbrec_mirror_table *sbrec_mirror_table, + const struct sbrec_chassis_table *sbrec_chassis_table, + struct ovsdb_idl_index *sbrec_chassis_by_name, + struct ovsdb_idl_index *sbrec_chassis_by_hostname) +{ + struct ovn_port *op = ovn_port_create(ls_ports, key, nbsp, NULL, + NULL); + hmap_insert(&od->ports, &op->dp_node, hmap_node_hash(&op->key_node)); + if (!ls_port_init(op, ovnsb_txn, ls_ports, od, sb, + sbrec_mirror_table, sbrec_chassis_table, + sbrec_chassis_by_name, sbrec_chassis_by_hostname)) { + ovn_port_destroy(ls_ports, op); + return NULL; + } + return op; } +static bool +ls_port_reinit(struct ovn_port *op, struct ovsdb_idl_txn *ovnsb_txn, + struct hmap *ls_ports, + const struct nbrec_logical_switch_port *nbsp, + const struct nbrec_logical_router_port *nbrp, + struct ovn_datapath *od, + const struct sbrec_port_binding *sb, + const struct sbrec_mirror_table *sbrec_mirror_table, + const struct sbrec_chassis_table *sbrec_chassis_table, + struct ovsdb_idl_index *sbrec_chassis_by_name, + struct ovsdb_idl_index *sbrec_chassis_by_hostname) +{ + ovn_port_cleanup(op); + op->sb = sb; + ovn_port_set_nb(op, nbsp, nbrp); + op->l3dgw_port = op->cr_port = NULL; + return ls_port_init(op, ovnsb_txn, ls_ports, od, sb, + sbrec_mirror_table, sbrec_chassis_table, + sbrec_chassis_by_name, sbrec_chassis_by_hostname); +} + /* Returns true if the logical switch has changes which can be * incrementally handled. * Presently supports i-p for the below changes: @@ -5020,7 +4770,7 @@ ls_handle_lsp_changes(struct ovsdb_idl_txn *ovnsb_idl_txn, goto fail; } op = ls_port_create(ovnsb_idl_txn, &nd->ls_ports, - new_nbsp->name, new_nbsp, od, NULL, NULL, + new_nbsp->name, new_nbsp, od, NULL, ni->sbrec_mirror_table, ni->sbrec_chassis_table, ni->sbrec_chassis_by_name, @@ -5051,17 +4801,12 @@ ls_handle_lsp_changes(struct ovsdb_idl_txn *ovnsb_idl_txn, op->visited = true; continue; } - struct ovs_list lflows = OVS_LIST_INITIALIZER(&lflows); - ovs_list_splice(&lflows, op->lflows.next, &op->lflows); - ovn_port_destroy(&nd->ls_ports, op); - op = ls_port_create(ovnsb_idl_txn, &nd->ls_ports, - new_nbsp->name, new_nbsp, od, sb, &lflows, - ni->sbrec_mirror_table, + if (!ls_port_reinit(op, ovnsb_idl_txn, &nd->ls_ports, + new_nbsp, NULL, + od, sb, ni->sbrec_mirror_table, ni->sbrec_chassis_table, ni->sbrec_chassis_by_name, - ni->sbrec_chassis_by_hostname); - ovs_assert(ovs_list_is_empty(&lflows)); - if (!op) { + ni->sbrec_chassis_by_hostname)) { goto fail; } add_op_to_northd_tracked_ports(&trk_lsps->updated, op); @@ -6012,170 +5757,7 @@ ovn_igmp_group_destroy(struct hmap *igmp_groups, * function of most of the northbound database. */ -struct ovn_lflow { - struct hmap_node hmap_node; - struct ovs_list list_node; /* For temporary list of lflows. Don't remove - at destroy. */ - - struct ovn_datapath *od; /* 'logical_datapath' in SB schema. */ - unsigned long *dpg_bitmap; /* Bitmap of all datapaths by their 'index'.*/ - enum ovn_stage stage; - uint16_t priority; - char *match; - char *actions; - char *io_port; - char *stage_hint; - char *ctrl_meter; - size_t n_ods; /* Number of datapaths referenced by 'od' and - * 'dpg_bitmap'. */ - struct ovn_dp_group *dpg; /* Link to unique Sb datapath group. */ - - struct ovs_list referenced_by; /* List of struct lflow_ref_node. */ - const char *where; - - struct uuid sb_uuid; /* SB DB row uuid, specified by northd. */ -}; - -static void ovn_lflow_destroy(struct hmap *lflows, struct ovn_lflow *lflow); -static struct ovn_lflow *ovn_lflow_find(const struct hmap *lflows, - const struct ovn_datapath *od, - enum ovn_stage stage, - uint16_t priority, const char *match, - const char *actions, - const char *ctrl_meter, uint32_t hash); - -static char * -ovn_lflow_hint(const struct ovsdb_idl_row *row) -{ - if (!row) { - return NULL; - } - return xasprintf("%08x", row->uuid.parts[0]); -} - -static bool -ovn_lflow_equal(const struct ovn_lflow *a, const struct ovn_datapath *od, - enum ovn_stage stage, uint16_t priority, const char *match, - const char *actions, const char *ctrl_meter) -{ - return (a->od == od - && a->stage == stage - && a->priority == priority - && !strcmp(a->match, match) - && !strcmp(a->actions, actions) - && nullable_string_is_equal(a->ctrl_meter, ctrl_meter)); -} - -enum { - STATE_NULL, /* parallelization is off */ - STATE_INIT_HASH_SIZES, /* parallelization is on; hashes sizing needed */ - STATE_USE_PARALLELIZATION /* parallelization is on */ -}; -static int parallelization_state = STATE_NULL; - -static void -ovn_lflow_init(struct ovn_lflow *lflow, struct ovn_datapath *od, - size_t dp_bitmap_len, enum ovn_stage stage, uint16_t priority, - char *match, char *actions, char *io_port, char *ctrl_meter, - char *stage_hint, const char *where) -{ - ovs_list_init(&lflow->list_node); - ovs_list_init(&lflow->referenced_by); - lflow->dpg_bitmap = bitmap_allocate(dp_bitmap_len); - lflow->od = od; - lflow->stage = stage; - lflow->priority = priority; - lflow->match = match; - lflow->actions = actions; - lflow->io_port = io_port; - lflow->stage_hint = stage_hint; - lflow->ctrl_meter = ctrl_meter; - lflow->dpg = NULL; - lflow->where = where; - lflow->sb_uuid = UUID_ZERO; -} - -/* The lflow_hash_lock is a mutex array that protects updates to the shared - * lflow table across threads when parallel lflow build and dp-group are both - * enabled. To avoid high contention between threads, a big array of mutexes - * are used instead of just one. This is possible because when parallel build - * is used we only use hmap_insert_fast() to update the hmap, which would not - * touch the bucket array but only the list in a single bucket. We only need to - * make sure that when adding lflows to the same hash bucket, the same lock is - * used, so that no two threads can add to the bucket at the same time. It is - * ok that the same lock is used to protect multiple buckets, so a fixed sized - * mutex array is used instead of 1-1 mapping to the hash buckets. This - * simplies the implementation while effectively reduces lock contention - * because the chance that different threads contending the same lock amongst - * the big number of locks is very low. */ -#define LFLOW_HASH_LOCK_MASK 0xFFFF -static struct ovs_mutex lflow_hash_locks[LFLOW_HASH_LOCK_MASK + 1]; - -static void -lflow_hash_lock_init(void) -{ - if (!lflow_hash_lock_initialized) { - for (size_t i = 0; i < LFLOW_HASH_LOCK_MASK + 1; i++) { - ovs_mutex_init(&lflow_hash_locks[i]); - } - lflow_hash_lock_initialized = true; - } -} - -static void -lflow_hash_lock_destroy(void) -{ - if (lflow_hash_lock_initialized) { - for (size_t i = 0; i < LFLOW_HASH_LOCK_MASK + 1; i++) { - ovs_mutex_destroy(&lflow_hash_locks[i]); - } - } - lflow_hash_lock_initialized = false; -} - -/* Full thread safety analysis is not possible with hash locks, because - * they are taken conditionally based on the 'parallelization_state' and - * a flow hash. Also, the order in which two hash locks are taken is not - * predictable during the static analysis. - * - * Since the order of taking two locks depends on a random hash, to avoid - * ABBA deadlocks, no two hash locks can be nested. In that sense an array - * of hash locks is similar to a single mutex. - * - * Using a fake mutex to partially simulate thread safety restrictions, as - * if it were actually a single mutex. - * - * OVS_NO_THREAD_SAFETY_ANALYSIS below allows us to ignore conditional - * nature of the lock. Unlike other attributes, it applies to the - * implementation and not to the interface. So, we can define a function - * that acquires the lock without analysing the way it does that. - */ -extern struct ovs_mutex fake_hash_mutex; - -static struct ovs_mutex * -lflow_hash_lock(const struct hmap *lflow_map, uint32_t hash) - OVS_ACQUIRES(fake_hash_mutex) - OVS_NO_THREAD_SAFETY_ANALYSIS -{ - struct ovs_mutex *hash_lock = NULL; - - if (parallelization_state == STATE_USE_PARALLELIZATION) { - hash_lock = - &lflow_hash_locks[hash & lflow_map->mask & LFLOW_HASH_LOCK_MASK]; - ovs_mutex_lock(hash_lock); - } - return hash_lock; -} - -static void -lflow_hash_unlock(struct ovs_mutex *hash_lock) - OVS_RELEASES(fake_hash_mutex) - OVS_NO_THREAD_SAFETY_ANALYSIS -{ - if (hash_lock) { - ovs_mutex_unlock(hash_lock); - } -} +int parallelization_state = STATE_NULL; /* This thread-local var is used for parallel lflow building when dp-groups is @@ -6188,240 +5770,7 @@ lflow_hash_unlock(struct ovs_mutex *hash_lock) * threads are collected to fix the lflow hmap's size (by the function * fix_flow_map_size()). * */ -static thread_local size_t thread_lflow_counter = 0; - -/* Adds an OVN datapath to a datapath group of existing logical flow. - * Version to use when hash bucket locking is NOT required or the corresponding - * hash lock is already taken. */ -static void -ovn_dp_group_add_with_reference(struct ovn_lflow *lflow_ref, - const struct ovn_datapath *od, - const unsigned long *dp_bitmap, - size_t bitmap_len) - OVS_REQUIRES(fake_hash_mutex) -{ - if (od) { - bitmap_set1(lflow_ref->dpg_bitmap, od->index); - } - if (dp_bitmap) { - bitmap_or(lflow_ref->dpg_bitmap, dp_bitmap, bitmap_len); - } -} - -/* This global variable collects the lflows generated by do_ovn_lflow_add(). - * start_collecting_lflows() will enable the lflow collection and the calls to - * do_ovn_lflow_add (or the macros ovn_lflow_add_...) will add generated lflows - * to the list. end_collecting_lflows() will disable it. */ -static thread_local struct ovs_list collected_lflows; -static thread_local bool collecting_lflows = false; - -static void -start_collecting_lflows(void) -{ - ovs_assert(!collecting_lflows); - ovs_list_init(&collected_lflows); - collecting_lflows = true; -} - -static void -end_collecting_lflows(void) -{ - ovs_assert(collecting_lflows); - collecting_lflows = false; -} - -/* Adds a row with the specified contents to the Logical_Flow table. - * Version to use when hash bucket locking is NOT required. */ -static void -do_ovn_lflow_add(struct hmap *lflow_map, const struct ovn_datapath *od, - const unsigned long *dp_bitmap, size_t dp_bitmap_len, - uint32_t hash, enum ovn_stage stage, uint16_t priority, - const char *match, const char *actions, const char *io_port, - const struct ovsdb_idl_row *stage_hint, - const char *where, const char *ctrl_meter) - OVS_REQUIRES(fake_hash_mutex) -{ - - struct ovn_lflow *old_lflow; - struct ovn_lflow *lflow; - - size_t bitmap_len = od ? ods_size(od->datapaths) : dp_bitmap_len; - ovs_assert(bitmap_len); - - if (collecting_lflows) { - ovs_assert(od); - ovs_assert(!dp_bitmap); - } else { - old_lflow = ovn_lflow_find(lflow_map, NULL, stage, priority, match, - actions, ctrl_meter, hash); - if (old_lflow) { - ovn_dp_group_add_with_reference(old_lflow, od, dp_bitmap, - bitmap_len); - return; - } - } - - lflow = xmalloc(sizeof *lflow); - /* While adding new logical flows we're not setting single datapath, but - * collecting a group. 'od' will be updated later for all flows with only - * one datapath in a group, so it could be hashed correctly. */ - ovn_lflow_init(lflow, NULL, bitmap_len, stage, priority, - xstrdup(match), xstrdup(actions), - io_port ? xstrdup(io_port) : NULL, - nullable_xstrdup(ctrl_meter), - ovn_lflow_hint(stage_hint), where); - - ovn_dp_group_add_with_reference(lflow, od, dp_bitmap, bitmap_len); - - if (parallelization_state != STATE_USE_PARALLELIZATION) { - hmap_insert(lflow_map, &lflow->hmap_node, hash); - } else { - hmap_insert_fast(lflow_map, &lflow->hmap_node, hash); - thread_lflow_counter++; - } - - if (collecting_lflows) { - ovs_list_insert(&collected_lflows, &lflow->list_node); - } -} - -/* Adds a row with the specified contents to the Logical_Flow table. */ -static void -ovn_lflow_add_at(struct hmap *lflow_map, const struct ovn_datapath *od, - const unsigned long *dp_bitmap, size_t dp_bitmap_len, - enum ovn_stage stage, uint16_t priority, - const char *match, const char *actions, const char *io_port, - const char *ctrl_meter, - const struct ovsdb_idl_row *stage_hint, const char *where) - OVS_EXCLUDED(fake_hash_mutex) -{ - struct ovs_mutex *hash_lock; - uint32_t hash; - - ovs_assert(!od || - ovn_stage_to_datapath_type(stage) == ovn_datapath_get_type(od)); - - hash = ovn_logical_flow_hash(ovn_stage_get_table(stage), - ovn_stage_get_pipeline(stage), - priority, match, - actions); - - hash_lock = lflow_hash_lock(lflow_map, hash); - do_ovn_lflow_add(lflow_map, od, dp_bitmap, dp_bitmap_len, hash, stage, - priority, match, actions, io_port, stage_hint, where, - ctrl_meter); - lflow_hash_unlock(hash_lock); -} - -static void -__ovn_lflow_add_default_drop(struct hmap *lflow_map, - struct ovn_datapath *od, - enum ovn_stage stage, - const char *where) -{ - ovn_lflow_add_at(lflow_map, od, NULL, 0, stage, 0, "1", - debug_drop_action(), - NULL, NULL, NULL, where ); -} - -/* Adds a row with the specified contents to the Logical_Flow table. */ -#define ovn_lflow_add_with_hint__(LFLOW_MAP, OD, STAGE, PRIORITY, MATCH, \ - ACTIONS, IN_OUT_PORT, CTRL_METER, \ - STAGE_HINT) \ - ovn_lflow_add_at(LFLOW_MAP, OD, NULL, 0, STAGE, PRIORITY, MATCH, ACTIONS, \ - IN_OUT_PORT, CTRL_METER, STAGE_HINT, OVS_SOURCE_LOCATOR) - -#define ovn_lflow_add_with_hint(LFLOW_MAP, OD, STAGE, PRIORITY, MATCH, \ - ACTIONS, STAGE_HINT) \ - ovn_lflow_add_at(LFLOW_MAP, OD, NULL, 0, STAGE, PRIORITY, MATCH, ACTIONS, \ - NULL, NULL, STAGE_HINT, OVS_SOURCE_LOCATOR) - -#define ovn_lflow_add_with_dp_group(LFLOW_MAP, DP_BITMAP, DP_BITMAP_LEN, \ - STAGE, PRIORITY, MATCH, ACTIONS, \ - STAGE_HINT) \ - ovn_lflow_add_at(LFLOW_MAP, NULL, DP_BITMAP, DP_BITMAP_LEN, STAGE, \ - PRIORITY, MATCH, ACTIONS, NULL, NULL, STAGE_HINT, \ - OVS_SOURCE_LOCATOR) - -#define ovn_lflow_add_default_drop(LFLOW_MAP, OD, STAGE) \ - __ovn_lflow_add_default_drop(LFLOW_MAP, OD, STAGE, OVS_SOURCE_LOCATOR) - - -/* This macro is similar to ovn_lflow_add_with_hint, except that it requires - * the IN_OUT_PORT argument, which tells the lport name that appears in the - * MATCH, which helps ovn-controller to bypass lflows parsing when the lport is - * not local to the chassis. The critiera of the lport to be added using this - * argument: - * - * - For ingress pipeline, the lport that is used to match "inport". - * - For egress pipeline, the lport that is used to match "outport". - * - * For now, only LS pipelines should use this macro. */ -#define ovn_lflow_add_with_lport_and_hint(LFLOW_MAP, OD, STAGE, PRIORITY, \ - MATCH, ACTIONS, IN_OUT_PORT, \ - STAGE_HINT) \ - ovn_lflow_add_at(LFLOW_MAP, OD, NULL, 0, STAGE, PRIORITY, MATCH, ACTIONS, \ - IN_OUT_PORT, NULL, STAGE_HINT, OVS_SOURCE_LOCATOR) - -#define ovn_lflow_add(LFLOW_MAP, OD, STAGE, PRIORITY, MATCH, ACTIONS) \ - ovn_lflow_add_at(LFLOW_MAP, OD, NULL, 0, STAGE, PRIORITY, MATCH, ACTIONS, \ - NULL, NULL, NULL, OVS_SOURCE_LOCATOR) - -#define ovn_lflow_metered(LFLOW_MAP, OD, STAGE, PRIORITY, MATCH, ACTIONS, \ - CTRL_METER) \ - ovn_lflow_add_with_hint__(LFLOW_MAP, OD, STAGE, PRIORITY, MATCH, \ - ACTIONS, NULL, CTRL_METER, NULL) - -static struct ovn_lflow * -ovn_lflow_find(const struct hmap *lflows, const struct ovn_datapath *od, - enum ovn_stage stage, uint16_t priority, - const char *match, const char *actions, const char *ctrl_meter, - uint32_t hash) -{ - struct ovn_lflow *lflow; - HMAP_FOR_EACH_WITH_HASH (lflow, hmap_node, hash, lflows) { - if (ovn_lflow_equal(lflow, od, stage, priority, match, actions, - ctrl_meter)) { - return lflow; - } - } - return NULL; -} - -static void -ovn_lflow_destroy(struct hmap *lflows, struct ovn_lflow *lflow) -{ - if (lflow) { - if (lflows) { - hmap_remove(lflows, &lflow->hmap_node); - } - bitmap_free(lflow->dpg_bitmap); - free(lflow->match); - free(lflow->actions); - free(lflow->io_port); - free(lflow->stage_hint); - free(lflow->ctrl_meter); - struct lflow_ref_node *l; - LIST_FOR_EACH_SAFE (l, ref_list_node, &lflow->referenced_by) { - ovs_list_remove(&l->lflow_list_node); - ovs_list_remove(&l->ref_list_node); - free(l); - } - free(lflow); - } -} - -static void -link_ovn_port_to_lflows(struct ovn_port *op, struct ovs_list *lflows) -{ - struct ovn_lflow *f; - LIST_FOR_EACH (f, list_node, lflows) { - struct lflow_ref_node *lfrn = xmalloc(sizeof *lfrn); - lfrn->lflow = f; - ovs_list_insert(&op->lflows, &lfrn->lflow_list_node); - ovs_list_insert(&f->referenced_by, &lfrn->ref_list_node); - } -} +thread_local size_t thread_lflow_counter = 0; static bool build_dhcpv4_action(struct ovn_port *op, ovs_be32 offer_ip, @@ -6599,7 +5948,7 @@ build_dhcpv6_action(struct ovn_port *op, struct in6_addr *offer_ip, * build_lswitch_lflows_admission_control() handles the port security. */ static void -build_lswitch_port_sec_op(struct ovn_port *op, struct hmap *lflows, +build_lswitch_port_sec_op(struct ovn_port *op, struct lflow_table *lflows, struct ds *actions, struct ds *match) { ovs_assert(op->nbsp); @@ -6616,13 +5965,13 @@ build_lswitch_port_sec_op(struct ovn_port *op, struct hmap *lflows, ovn_lflow_add_with_lport_and_hint( lflows, op->od, S_SWITCH_IN_CHECK_PORT_SEC, 100, ds_cstr(match), REGBIT_PORT_SEC_DROP" = 1; next;", - op->key, &op->nbsp->header_); + op->key, &op->nbsp->header_, op->lflow_ref); ds_clear(match); ds_put_format(match, "outport == %s", op->json_key); ovn_lflow_add_with_lport_and_hint( lflows, op->od, S_SWITCH_IN_L2_UNKNOWN, 50, ds_cstr(match), - debug_drop_action(), op->key, &op->nbsp->header_); + debug_drop_action(), op->key, &op->nbsp->header_, op->lflow_ref); return; } @@ -6638,14 +5987,16 @@ build_lswitch_port_sec_op(struct ovn_port *op, struct hmap *lflows, ovn_lflow_add_with_lport_and_hint(lflows, op->od, S_SWITCH_IN_CHECK_PORT_SEC, 70, ds_cstr(match), ds_cstr(actions), - op->key, &op->nbsp->header_); + op->key, &op->nbsp->header_, + op->lflow_ref); } else if (queue_id) { ds_put_cstr(actions, REGBIT_PORT_SEC_DROP" = check_in_port_sec(); next;"); ovn_lflow_add_with_lport_and_hint(lflows, op->od, S_SWITCH_IN_CHECK_PORT_SEC, 70, ds_cstr(match), ds_cstr(actions), - op->key, &op->nbsp->header_); + op->key, &op->nbsp->header_, + op->lflow_ref); if (!lsp_is_localnet(op->nbsp) && !op->od->n_localnet_ports) { return; @@ -6660,7 +6011,8 @@ build_lswitch_port_sec_op(struct ovn_port *op, struct hmap *lflows, ovn_lflow_add_with_lport_and_hint(lflows, op->od, S_SWITCH_OUT_APPLY_PORT_SEC, 100, ds_cstr(match), ds_cstr(actions), - op->key, &op->nbsp->header_); + op->key, &op->nbsp->header_, + op->lflow_ref); } else if (op->od->n_localnet_ports) { ds_put_format(match, "outport == %s && inport == %s", op->od->localnet_ports[0]->json_key, @@ -6669,14 +6021,15 @@ build_lswitch_port_sec_op(struct ovn_port *op, struct hmap *lflows, S_SWITCH_OUT_APPLY_PORT_SEC, 110, ds_cstr(match), ds_cstr(actions), op->od->localnet_ports[0]->key, - &op->od->localnet_ports[0]->nbsp->header_); + &op->od->localnet_ports[0]->nbsp->header_, + op->lflow_ref); } } } static void build_lswitch_learn_fdb_op( - struct ovn_port *op, struct hmap *lflows, + struct ovn_port *op, struct lflow_table *lflows, struct ds *actions, struct ds *match) { ovs_assert(op->nbsp); @@ -6694,7 +6047,8 @@ build_lswitch_learn_fdb_op( ovn_lflow_add_with_lport_and_hint(lflows, op->od, S_SWITCH_IN_LOOKUP_FDB, 100, ds_cstr(match), ds_cstr(actions), - op->key, &op->nbsp->header_); + op->key, &op->nbsp->header_, + op->lflow_ref); ds_put_cstr(match, " && "REGBIT_LKUP_FDB" == 0"); ds_clear(actions); @@ -6702,13 +6056,14 @@ build_lswitch_learn_fdb_op( ovn_lflow_add_with_lport_and_hint(lflows, op->od, S_SWITCH_IN_PUT_FDB, 100, ds_cstr(match), ds_cstr(actions), op->key, - &op->nbsp->header_); + &op->nbsp->header_, + op->lflow_ref); } } static void build_lswitch_learn_fdb_od( - struct ovn_datapath *od, struct hmap *lflows) + struct ovn_datapath *od, struct lflow_table *lflows) { ovs_assert(od->nbs); ovn_lflow_add(lflows, od, S_SWITCH_IN_LOOKUP_FDB, 0, "1", "next;"); @@ -6722,7 +6077,7 @@ build_lswitch_learn_fdb_od( * (priority 100). */ static void build_lswitch_output_port_sec_od(struct ovn_datapath *od, - struct hmap *lflows) + struct lflow_table *lflows) { ovs_assert(od->nbs); ovn_lflow_add(lflows, od, S_SWITCH_OUT_CHECK_PORT_SEC, 100, @@ -6740,7 +6095,7 @@ static void skip_port_from_conntrack(const struct ovn_datapath *od, struct ovn_port *op, bool has_stateful_acl, enum ovn_stage in_stage, enum ovn_stage out_stage, uint16_t priority, - struct hmap *lflows) + struct lflow_table *lflows) { /* Can't use ct() for router ports. Consider the following configuration: * lp1(10.0.0.2) on hostA--ls1--lr0--ls2--lp2(10.0.1.2) on hostB, For a @@ -6762,10 +6117,10 @@ skip_port_from_conntrack(const struct ovn_datapath *od, struct ovn_port *op, ovn_lflow_add_with_lport_and_hint(lflows, od, in_stage, priority, ingress_match, ingress_action, - op->key, &op->nbsp->header_); + op->key, &op->nbsp->header_, NULL); ovn_lflow_add_with_lport_and_hint(lflows, od, out_stage, priority, egress_match, egress_action, - op->key, &op->nbsp->header_); + op->key, &op->nbsp->header_, NULL); free(ingress_match); free(egress_match); @@ -6774,7 +6129,7 @@ skip_port_from_conntrack(const struct ovn_datapath *od, struct ovn_port *op, static void build_stateless_filter(const struct ovn_datapath *od, const struct nbrec_acl *acl, - struct hmap *lflows) + struct lflow_table *lflows) { const char *action = REGBIT_ACL_STATELESS" = 1; next;"; if (!strcmp(acl->direction, "from-lport")) { @@ -6795,7 +6150,7 @@ build_stateless_filter(const struct ovn_datapath *od, static void build_stateless_filters(const struct ovn_datapath *od, const struct ls_port_group_table *ls_port_groups, - struct hmap *lflows) + struct lflow_table *lflows) { for (size_t i = 0; i < od->nbs->n_acls; i++) { const struct nbrec_acl *acl = od->nbs->acls[i]; @@ -6823,7 +6178,7 @@ build_stateless_filters(const struct ovn_datapath *od, } static void -build_pre_acls(struct ovn_datapath *od, struct hmap *lflows) +build_pre_acls(struct ovn_datapath *od, struct lflow_table *lflows) { /* Ingress and Egress Pre-ACL Table (Priority 0): Packets are * allowed by default. */ @@ -6840,7 +6195,7 @@ build_pre_acls(struct ovn_datapath *od, struct hmap *lflows) static void build_ls_stateful_rec_pre_acls(const struct ls_stateful_record *ls_sful_rec, const struct ls_port_group_table *ls_port_groups, - struct hmap *lflows) + struct lflow_table *lflows) { const struct ovn_datapath *od = ls_sful_rec->od; @@ -6959,7 +6314,7 @@ build_empty_lb_event_flow(struct ovn_lb_vip *lb_vip, static void build_interconn_mcast_snoop_flows(struct ovn_datapath *od, const struct shash *meter_groups, - struct hmap *lflows) + struct lflow_table *lflows) { struct mcast_switch_info *mcast_sw_info = &od->mcast_info.sw; if (!mcast_sw_info->enabled @@ -6993,7 +6348,7 @@ build_interconn_mcast_snoop_flows(struct ovn_datapath *od, static void build_pre_lb(struct ovn_datapath *od, const struct shash *meter_groups, - struct hmap *lflows) + struct lflow_table *lflows) { /* Handle IGMP/MLD packets crossing AZs. */ build_interconn_mcast_snoop_flows(od, meter_groups, lflows); @@ -7029,7 +6384,7 @@ build_pre_lb(struct ovn_datapath *od, const struct shash *meter_groups, static void build_ls_stateful_rec_pre_lb(const struct ls_stateful_record *ls_stateful_rec, - struct hmap *lflows) + struct lflow_table *lflows) { const struct ovn_datapath *od = ls_stateful_rec->od; @@ -7095,7 +6450,7 @@ build_ls_stateful_rec_pre_lb(const struct ls_stateful_record *ls_stateful_rec, static void build_pre_stateful(struct ovn_datapath *od, const struct chassis_features *features, - struct hmap *lflows) + struct lflow_table *lflows) { /* Ingress and Egress pre-stateful Table (Priority 0): Packets are * allowed by default. */ @@ -7127,7 +6482,7 @@ build_pre_stateful(struct ovn_datapath *od, static void build_acl_hints(const struct ls_stateful_record *ls_sful_rec, const struct chassis_features *features, - struct hmap *lflows) + struct lflow_table *lflows) { const struct ovn_datapath *od = ls_sful_rec->od; @@ -7296,7 +6651,7 @@ build_acl_log(struct ds *actions, const struct nbrec_acl *acl, } static void -consider_acl(struct hmap *lflows, const struct ovn_datapath *od, +consider_acl(struct lflow_table *lflows, const struct ovn_datapath *od, const struct nbrec_acl *acl, bool has_stateful, bool ct_masked_mark, const struct shash *meter_groups, uint64_t max_acl_tier, struct ds *match, struct ds *actions) @@ -7524,7 +6879,7 @@ ovn_update_ipv6_options(struct hmap *lr_ports) static void build_acl_action_lflows(const struct ls_stateful_record *ls_stateful_rec, - struct hmap *lflows, + struct lflow_table *lflows, const char *default_acl_action, const struct shash *meter_groups, struct ds *match, @@ -7601,7 +6956,8 @@ build_acl_action_lflows(const struct ls_stateful_record *ls_stateful_rec, } static void -build_acl_log_related_flows(const struct ovn_datapath *od, struct hmap *lflows, +build_acl_log_related_flows(const struct ovn_datapath *od, + struct lflow_table *lflows, const struct nbrec_acl *acl, bool has_stateful, bool ct_masked_mark, const struct shash *meter_groups, @@ -7676,7 +7032,7 @@ build_acl_log_related_flows(const struct ovn_datapath *od, struct hmap *lflows, static void build_acls(const struct ls_stateful_record *ls_stateful_rec, const struct chassis_features *features, - struct hmap *lflows, + struct lflow_table *lflows, const struct ls_port_group_table *ls_port_groups, const struct shash *meter_groups) { @@ -7922,7 +7278,7 @@ build_acls(const struct ls_stateful_record *ls_stateful_rec, } static void -build_qos(struct ovn_datapath *od, struct hmap *lflows) { +build_qos(struct ovn_datapath *od, struct lflow_table *lflows) { struct ds action = DS_EMPTY_INITIALIZER; ovn_lflow_add(lflows, od, S_SWITCH_IN_QOS_MARK, 0, "1", "next;"); @@ -7983,7 +7339,7 @@ build_qos(struct ovn_datapath *od, struct hmap *lflows) { } static void -build_lb_rules_pre_stateful(struct hmap *lflows, +build_lb_rules_pre_stateful(struct lflow_table *lflows, struct ovn_lb_datapaths *lb_dps, bool ct_lb_mark, const struct ovn_datapaths *ls_datapaths, @@ -8085,7 +7441,8 @@ build_lb_rules_pre_stateful(struct hmap *lflows, * */ static void -build_lb_affinity_lr_flows(struct hmap *lflows, const struct ovn_northd_lb *lb, +build_lb_affinity_lr_flows(struct lflow_table *lflows, + const struct ovn_northd_lb *lb, struct ovn_lb_vip *lb_vip, char *new_lb_match, char *lb_action, const unsigned long *dp_bitmap, const struct ovn_datapaths *lr_datapaths) @@ -8271,7 +7628,7 @@ build_lb_affinity_lr_flows(struct hmap *lflows, const struct ovn_northd_lb *lb, * */ static void -build_lb_affinity_ls_flows(struct hmap *lflows, +build_lb_affinity_ls_flows(struct lflow_table *lflows, struct ovn_lb_datapaths *lb_dps, struct ovn_lb_vip *lb_vip, const struct ovn_datapaths *ls_datapaths) @@ -8414,7 +7771,7 @@ build_lb_affinity_ls_flows(struct hmap *lflows, static void build_lswitch_lb_affinity_default_flows(struct ovn_datapath *od, - struct hmap *lflows) + struct lflow_table *lflows) { ovs_assert(od->nbs); ovn_lflow_add(lflows, od, S_SWITCH_IN_LB_AFF_CHECK, 0, "1", "next;"); @@ -8423,7 +7780,7 @@ build_lswitch_lb_affinity_default_flows(struct ovn_datapath *od, static void build_lrouter_lb_affinity_default_flows(struct ovn_datapath *od, - struct hmap *lflows) + struct lflow_table *lflows) { ovs_assert(od->nbr); ovn_lflow_add(lflows, od, S_ROUTER_IN_LB_AFF_CHECK, 0, "1", "next;"); @@ -8431,7 +7788,7 @@ build_lrouter_lb_affinity_default_flows(struct ovn_datapath *od, } static void -build_lb_rules(struct hmap *lflows, struct ovn_lb_datapaths *lb_dps, +build_lb_rules(struct lflow_table *lflows, struct ovn_lb_datapaths *lb_dps, const struct ovn_datapaths *ls_datapaths, const struct chassis_features *features, struct ds *match, struct ds *action, const struct shash *meter_groups, @@ -8511,7 +7868,7 @@ build_lb_rules(struct hmap *lflows, struct ovn_lb_datapaths *lb_dps, static void build_stateful(struct ovn_datapath *od, const struct chassis_features *features, - struct hmap *lflows) + struct lflow_table *lflows) { const char *ct_block_action = features->ct_no_masked_label ? "ct_mark.blocked" @@ -8561,7 +7918,7 @@ build_stateful(struct ovn_datapath *od, static void build_lb_hairpin(const struct ls_stateful_record *ls_stateful_rec, - struct hmap *lflows) + struct lflow_table *lflows) { const struct ovn_datapath *od = ls_stateful_rec->od; @@ -8620,7 +7977,7 @@ build_lb_hairpin(const struct ls_stateful_record *ls_stateful_rec, } static void -build_vtep_hairpin(struct ovn_datapath *od, struct hmap *lflows) +build_vtep_hairpin(struct ovn_datapath *od, struct lflow_table *lflows) { if (!od->has_vtep_lports) { /* There is no need in these flows if datapath has no vtep lports. */ @@ -8668,7 +8025,7 @@ build_vtep_hairpin(struct ovn_datapath *od, struct hmap *lflows) /* Build logical flows for the forwarding groups */ static void -build_fwd_group_lflows(struct ovn_datapath *od, struct hmap *lflows) +build_fwd_group_lflows(struct ovn_datapath *od, struct lflow_table *lflows) { ovs_assert(od->nbs); if (!od->nbs->n_forwarding_groups) { @@ -8849,7 +8206,8 @@ build_lswitch_rport_arp_req_self_orig_flow(struct ovn_port *op, uint32_t priority, const struct ovn_datapath *od, const struct lr_nat_record *lrnat_rec, - struct hmap *lflows) + struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { struct ds eth_src = DS_EMPTY_INITIALIZER; struct ds match = DS_EMPTY_INITIALIZER; @@ -8873,8 +8231,10 @@ build_lswitch_rport_arp_req_self_orig_flow(struct ovn_port *op, ds_put_format(&match, "eth.src == %s && (arp.op == 1 || rarp.op == 3 || nd_ns)", ds_cstr(ð_src)); - ovn_lflow_add(lflows, od, S_SWITCH_IN_L2_LKUP, priority, ds_cstr(&match), - "outport = \""MC_FLOOD_L2"\"; output;"); + ovn_lflow_add_with_lflow_ref(lflows, od, S_SWITCH_IN_L2_LKUP, priority, + ds_cstr(&match), + "outport = \""MC_FLOOD_L2"\"; output;", + lflow_ref); ds_destroy(ð_src); ds_destroy(&match); @@ -8942,8 +8302,9 @@ static void build_lswitch_rport_arp_req_flow(const char *ips, int addr_family, struct ovn_port *patch_op, const struct ovn_datapath *od, - uint32_t priority, struct hmap *lflows, - const struct ovsdb_idl_row *stage_hint) + uint32_t priority, struct lflow_table *lflows, + const struct ovsdb_idl_row *stage_hint, + struct lflow_ref *lflow_ref) { struct ds match = DS_EMPTY_INITIALIZER; struct ds actions = DS_EMPTY_INITIALIZER; @@ -8957,14 +8318,17 @@ build_lswitch_rport_arp_req_flow(const char *ips, ds_put_format(&actions, "clone {outport = %s; output; }; " "outport = \""MC_FLOOD_L2"\"; output;", patch_op->json_key); - ovn_lflow_add_with_hint(lflows, od, S_SWITCH_IN_L2_LKUP, - priority, ds_cstr(&match), - ds_cstr(&actions), stage_hint); + ovn_lflow_add_with_lflow_ref_hint(lflows, od, S_SWITCH_IN_L2_LKUP, + priority, ds_cstr(&match), + ds_cstr(&actions), stage_hint, + lflow_ref); } else { ds_put_format(&actions, "outport = %s; output;", patch_op->json_key); - ovn_lflow_add_with_hint(lflows, od, S_SWITCH_IN_L2_LKUP, priority, - ds_cstr(&match), ds_cstr(&actions), - stage_hint); + ovn_lflow_add_with_lflow_ref_hint(lflows, od, S_SWITCH_IN_L2_LKUP, + priority, ds_cstr(&match), + ds_cstr(&actions), + stage_hint, + lflow_ref); } ds_destroy(&match); @@ -8982,7 +8346,7 @@ static void build_lswitch_rport_arp_req_flows(struct ovn_port *op, struct ovn_datapath *sw_od, struct ovn_port *sw_op, - struct hmap *lflows, + struct lflow_table *lflows, const struct ovsdb_idl_row *stage_hint) { if (!op || !op->nbrp) { @@ -9000,12 +8364,12 @@ build_lswitch_rport_arp_req_flows(struct ovn_port *op, for (size_t i = 0; i < op->lrp_networks.n_ipv4_addrs; i++) { build_lswitch_rport_arp_req_flow( op->lrp_networks.ipv4_addrs[i].addr_s, AF_INET, sw_op, sw_od, 80, - lflows, stage_hint); + lflows, stage_hint, sw_op->lflow_ref); } for (size_t i = 0; i < op->lrp_networks.n_ipv6_addrs; i++) { build_lswitch_rport_arp_req_flow( op->lrp_networks.ipv6_addrs[i].addr_s, AF_INET6, sw_op, sw_od, 80, - lflows, stage_hint); + lflows, stage_hint, sw_op->lflow_ref); } } @@ -9021,8 +8385,9 @@ build_lswitch_rport_arp_req_flows_for_lbnats(struct ovn_port *op, const struct lr_stateful_record *lr_sful_rec, const struct ovn_datapath *sw_od, struct ovn_port *sw_op, - struct hmap *lflows, - const struct ovsdb_idl_row *stage_hint) + struct lflow_table *lflows, + const struct ovsdb_idl_row *stage_hint, + struct lflow_ref *lflow_ref) { if (!op || !op->nbrp) { return; @@ -9050,7 +8415,7 @@ build_lswitch_rport_arp_req_flows_for_lbnats(struct ovn_port *op, lrouter_port_ipv4_reachable(op, ipv4_addr)) { build_lswitch_rport_arp_req_flow( ip_addr, AF_INET, sw_op, sw_od, 80, lflows, - stage_hint); + stage_hint, lflow_ref); } } SSET_FOR_EACH (ip_addr, &lr_sful_rec->lb_ips->ips_v6_reachable) { @@ -9063,7 +8428,7 @@ build_lswitch_rport_arp_req_flows_for_lbnats(struct ovn_port *op, lrouter_port_ipv6_reachable(op, &ipv6_addr)) { build_lswitch_rport_arp_req_flow( ip_addr, AF_INET6, sw_op, sw_od, 80, lflows, - stage_hint); + stage_hint, lflow_ref); } } } @@ -9078,7 +8443,7 @@ build_lswitch_rport_arp_req_flows_for_lbnats(struct ovn_port *op, if (sw_od->n_router_ports != sw_od->nbs->n_ports) { build_lswitch_rport_arp_req_self_orig_flow(op, 75, sw_od, lr_sful_rec->lrnat_rec, - lflows); + lflows, lflow_ref); } for (size_t i = 0; i < lr_sful_rec->lrnat_rec->n_nat_entries; i++) { @@ -9101,14 +8466,14 @@ build_lswitch_rport_arp_req_flows_for_lbnats(struct ovn_port *op, nat->external_ip)) { build_lswitch_rport_arp_req_flow( nat->external_ip, AF_INET6, sw_op, sw_od, 80, lflows, - stage_hint); + stage_hint, lflow_ref); } } else { if (!sset_contains(&lr_sful_rec->lb_ips->ips_v4, nat->external_ip)) { build_lswitch_rport_arp_req_flow( nat->external_ip, AF_INET, sw_op, sw_od, 80, lflows, - stage_hint); + stage_hint, lflow_ref); } } } @@ -9119,7 +8484,7 @@ build_dhcpv4_options_flows(struct ovn_port *op, struct lport_addresses *lsp_addrs, struct ovn_port *inport, bool is_external, const struct shash *meter_groups, - struct hmap *lflows) + struct lflow_table *lflows) { struct ds match = DS_EMPTY_INITIALIZER; @@ -9150,7 +8515,7 @@ build_dhcpv4_options_flows(struct ovn_port *op, op->json_key); } - ovn_lflow_add_with_hint__(lflows, op->od, + ovn_lflow_add_with_lflow_ref_hint__(lflows, op->od, S_SWITCH_IN_DHCP_OPTIONS, 100, ds_cstr(&match), ds_cstr(&options_action), @@ -9158,7 +8523,8 @@ build_dhcpv4_options_flows(struct ovn_port *op, copp_meter_get(COPP_DHCPV4_OPTS, op->od->nbs->copp, meter_groups), - &op->nbsp->dhcpv4_options->header_); + &op->nbsp->dhcpv4_options->header_, + op->lflow_ref); ds_clear(&match); /* If REGBIT_DHCP_OPTS_RESULT is set, it means the @@ -9177,7 +8543,8 @@ build_dhcpv4_options_flows(struct ovn_port *op, ovn_lflow_add_with_lport_and_hint( lflows, op->od, S_SWITCH_IN_DHCP_RESPONSE, 100, ds_cstr(&match), ds_cstr(&response_action), inport->key, - &op->nbsp->dhcpv4_options->header_); + &op->nbsp->dhcpv4_options->header_, + op->lflow_ref); ds_destroy(&options_action); ds_destroy(&response_action); ds_destroy(&ipv4_addr_match); @@ -9204,7 +8571,8 @@ build_dhcpv4_options_flows(struct ovn_port *op, ovn_lflow_add_with_lport_and_hint( lflows, op->od, S_SWITCH_OUT_ACL_EVAL, 34000, ds_cstr(&match),dhcp_actions, op->key, - &op->nbsp->dhcpv4_options->header_); + &op->nbsp->dhcpv4_options->header_, + op->lflow_ref); } break; } @@ -9217,7 +8585,7 @@ build_dhcpv6_options_flows(struct ovn_port *op, struct lport_addresses *lsp_addrs, struct ovn_port *inport, bool is_external, const struct shash *meter_groups, - struct hmap *lflows) + struct lflow_table *lflows) { struct ds match = DS_EMPTY_INITIALIZER; @@ -9239,7 +8607,7 @@ build_dhcpv6_options_flows(struct ovn_port *op, op->json_key); } - ovn_lflow_add_with_hint__(lflows, op->od, + ovn_lflow_add_with_lflow_ref_hint__(lflows, op->od, S_SWITCH_IN_DHCP_OPTIONS, 100, ds_cstr(&match), ds_cstr(&options_action), @@ -9247,7 +8615,8 @@ build_dhcpv6_options_flows(struct ovn_port *op, copp_meter_get(COPP_DHCPV6_OPTS, op->od->nbs->copp, meter_groups), - &op->nbsp->dhcpv6_options->header_); + &op->nbsp->dhcpv6_options->header_, + op->lflow_ref); /* If REGBIT_DHCP_OPTS_RESULT is set to 1, it means the * put_dhcpv6_opts action is successful */ @@ -9255,7 +8624,7 @@ build_dhcpv6_options_flows(struct ovn_port *op, ovn_lflow_add_with_lport_and_hint( lflows, op->od, S_SWITCH_IN_DHCP_RESPONSE, 100, ds_cstr(&match), ds_cstr(&response_action), inport->key, - &op->nbsp->dhcpv6_options->header_); + &op->nbsp->dhcpv6_options->header_, op->lflow_ref); ds_destroy(&options_action); ds_destroy(&response_action); @@ -9287,7 +8656,8 @@ build_dhcpv6_options_flows(struct ovn_port *op, ovn_lflow_add_with_lport_and_hint( lflows, op->od, S_SWITCH_OUT_ACL_EVAL, 34000, ds_cstr(&match),dhcp6_actions, op->key, - &op->nbsp->dhcpv6_options->header_); + &op->nbsp->dhcpv6_options->header_, + op->lflow_ref); } break; } @@ -9298,7 +8668,7 @@ build_dhcpv6_options_flows(struct ovn_port *op, static void build_drop_arp_nd_flows_for_unbound_router_ports(struct ovn_port *op, const struct ovn_port *port, - struct hmap *lflows) + struct lflow_table *lflows) { struct ds match = DS_EMPTY_INITIALIZER; @@ -9318,7 +8688,7 @@ build_drop_arp_nd_flows_for_unbound_router_ports(struct ovn_port *op, ovn_lflow_add_with_lport_and_hint( lflows, op->od, S_SWITCH_IN_EXTERNAL_PORT, 100, ds_cstr(&match), debug_drop_action(), port->key, - &op->nbsp->header_); + &op->nbsp->header_, op->lflow_ref); } for (size_t l = 0; l < rp->lsp_addrs[k].n_ipv6_addrs; l++) { ds_clear(&match); @@ -9334,7 +8704,7 @@ build_drop_arp_nd_flows_for_unbound_router_ports(struct ovn_port *op, ovn_lflow_add_with_lport_and_hint( lflows, op->od, S_SWITCH_IN_EXTERNAL_PORT, 100, ds_cstr(&match), debug_drop_action(), port->key, - &op->nbsp->header_); + &op->nbsp->header_, op->lflow_ref); } ds_clear(&match); @@ -9350,7 +8720,8 @@ build_drop_arp_nd_flows_for_unbound_router_ports(struct ovn_port *op, 100, ds_cstr(&match), debug_drop_action(), port->key, - &op->nbsp->header_); + &op->nbsp->header_, + op->lflow_ref); } } } @@ -9365,7 +8736,7 @@ is_vlan_transparent(const struct ovn_datapath *od) static void build_lswitch_lflows_l2_unknown(struct ovn_datapath *od, - struct hmap *lflows) + struct lflow_table *lflows) { /* Ingress table 25/26: Destination lookup for unknown MACs. */ if (od->has_unknown) { @@ -9386,7 +8757,7 @@ static void build_lswitch_lflows_pre_acl_and_acl( struct ovn_datapath *od, const struct chassis_features *features, - struct hmap *lflows, + struct lflow_table *lflows, const struct shash *meter_groups) { ovs_assert(od->nbs); @@ -9402,7 +8773,7 @@ build_lswitch_lflows_pre_acl_and_acl( * 100). */ static void build_lswitch_lflows_admission_control(struct ovn_datapath *od, - struct hmap *lflows) + struct lflow_table *lflows) { ovs_assert(od->nbs); /* Logical VLANs not supported. */ @@ -9430,7 +8801,7 @@ build_lswitch_lflows_admission_control(struct ovn_datapath *od, static void build_lswitch_arp_nd_responder_skip_local(struct ovn_port *op, - struct hmap *lflows, + struct lflow_table *lflows, struct ds *match) { ovs_assert(op->nbsp); @@ -9442,14 +8813,14 @@ build_lswitch_arp_nd_responder_skip_local(struct ovn_port *op, ovn_lflow_add_with_lport_and_hint(lflows, op->od, S_SWITCH_IN_ARP_ND_RSP, 100, ds_cstr(match), "next;", op->key, - &op->nbsp->header_); + &op->nbsp->header_, op->lflow_ref); } /* Ingress table 19: ARP/ND responder, reply for known IPs. * (priority 50). */ static void build_lswitch_arp_nd_responder_known_ips(struct ovn_port *op, - struct hmap *lflows, + struct lflow_table *lflows, const struct hmap *ls_ports, const struct shash *meter_groups, struct ds *actions, @@ -9534,7 +8905,8 @@ build_lswitch_arp_nd_responder_known_ips(struct ovn_port *op, S_SWITCH_IN_ARP_ND_RSP, 100, ds_cstr(match), ds_cstr(actions), vparent, - &vp->nbsp->header_); + &vp->nbsp->header_, + op->lflow_ref); } free(tokstr); @@ -9578,11 +8950,12 @@ build_lswitch_arp_nd_responder_known_ips(struct ovn_port *op, "output;", op->lsp_addrs[i].ea_s, op->lsp_addrs[i].ea_s, op->lsp_addrs[i].ipv4_addrs[j].addr_s); - ovn_lflow_add_with_hint(lflows, op->od, - S_SWITCH_IN_ARP_ND_RSP, 50, - ds_cstr(match), - ds_cstr(actions), - &op->nbsp->header_); + ovn_lflow_add_with_lflow_ref_hint(lflows, op->od, + S_SWITCH_IN_ARP_ND_RSP, 50, + ds_cstr(match), + ds_cstr(actions), + &op->nbsp->header_, + op->lflow_ref); /* Do not reply to an ARP request from the port that owns * the address (otherwise a DHCP client that ARPs to check @@ -9601,7 +8974,8 @@ build_lswitch_arp_nd_responder_known_ips(struct ovn_port *op, S_SWITCH_IN_ARP_ND_RSP, 100, ds_cstr(match), "next;", op->key, - &op->nbsp->header_); + &op->nbsp->header_, + op->lflow_ref); } /* For ND solicitations, we need to listen for both the @@ -9631,15 +9005,16 @@ build_lswitch_arp_nd_responder_known_ips(struct ovn_port *op, op->lsp_addrs[i].ipv6_addrs[j].addr_s, op->lsp_addrs[i].ipv6_addrs[j].addr_s, op->lsp_addrs[i].ea_s); - ovn_lflow_add_with_hint__(lflows, op->od, - S_SWITCH_IN_ARP_ND_RSP, 50, - ds_cstr(match), - ds_cstr(actions), - NULL, - copp_meter_get(COPP_ND_NA, - op->od->nbs->copp, - meter_groups), - &op->nbsp->header_); + ovn_lflow_add_with_lflow_ref_hint__(lflows, op->od, + S_SWITCH_IN_ARP_ND_RSP, 50, + ds_cstr(match), + ds_cstr(actions), + NULL, + copp_meter_get(COPP_ND_NA, + op->od->nbs->copp, + meter_groups), + &op->nbsp->header_, + op->lflow_ref); /* Do not reply to a solicitation from the port that owns * the address (otherwise DAD detection will fail). */ @@ -9648,7 +9023,8 @@ build_lswitch_arp_nd_responder_known_ips(struct ovn_port *op, S_SWITCH_IN_ARP_ND_RSP, 100, ds_cstr(match), "next;", op->key, - &op->nbsp->header_); + &op->nbsp->header_, + op->lflow_ref); } } } @@ -9694,8 +9070,12 @@ build_lswitch_arp_nd_responder_known_ips(struct ovn_port *op, ea_s, ea_s); - ovn_lflow_add_with_hint(lflows, op->od, S_SWITCH_IN_ARP_ND_RSP, - 30, ds_cstr(match), ds_cstr(actions), &op->nbsp->header_); + ovn_lflow_add_with_lflow_ref_hint(lflows, op->od, + S_SWITCH_IN_ARP_ND_RSP, + 30, ds_cstr(match), + ds_cstr(actions), + &op->nbsp->header_, + op->lflow_ref); } /* Add IPv6 NDP responses. @@ -9738,15 +9118,16 @@ build_lswitch_arp_nd_responder_known_ips(struct ovn_port *op, lsp_is_router(op->nbsp) ? "nd_na_router" : "nd_na", ea_s, ea_s); - ovn_lflow_add_with_hint__(lflows, op->od, - S_SWITCH_IN_ARP_ND_RSP, 30, - ds_cstr(match), - ds_cstr(actions), - NULL, - copp_meter_get(COPP_ND_NA, - op->od->nbs->copp, - meter_groups), - &op->nbsp->header_); + ovn_lflow_add_with_lflow_ref_hint__(lflows, op->od, + S_SWITCH_IN_ARP_ND_RSP, 30, + ds_cstr(match), + ds_cstr(actions), + NULL, + copp_meter_get(COPP_ND_NA, + op->od->nbs->copp, + meter_groups), + &op->nbsp->header_, + op->lflow_ref); ds_destroy(&ip6_dst_match); ds_destroy(&nd_target_match); } @@ -9757,7 +9138,7 @@ build_lswitch_arp_nd_responder_known_ips(struct ovn_port *op, * (priority 0)*/ static void build_lswitch_arp_nd_responder_default(struct ovn_datapath *od, - struct hmap *lflows) + struct lflow_table *lflows) { ovs_assert(od->nbs); ovn_lflow_add(lflows, od, S_SWITCH_IN_ARP_ND_RSP, 0, "1", "next;"); @@ -9768,7 +9149,7 @@ build_lswitch_arp_nd_responder_default(struct ovn_datapath *od, static void build_lswitch_arp_nd_service_monitor(const struct ovn_northd_lb *lb, const struct hmap *ls_ports, - struct hmap *lflows, + struct lflow_table *lflows, struct ds *actions, struct ds *match) { @@ -9844,7 +9225,7 @@ build_lswitch_arp_nd_service_monitor(const struct ovn_northd_lb *lb, * priority 100 flows. */ static void build_lswitch_dhcp_options_and_response(struct ovn_port *op, - struct hmap *lflows, + struct lflow_table *lflows, const struct shash *meter_groups) { ovs_assert(op->nbsp); @@ -9899,7 +9280,7 @@ build_lswitch_dhcp_options_and_response(struct ovn_port *op, * (priority 0). */ static void build_lswitch_dhcp_and_dns_defaults(struct ovn_datapath *od, - struct hmap *lflows) + struct lflow_table *lflows) { ovs_assert(od->nbs); ovn_lflow_add(lflows, od, S_SWITCH_IN_DHCP_OPTIONS, 0, "1", "next;"); @@ -9914,7 +9295,7 @@ build_lswitch_dhcp_and_dns_defaults(struct ovn_datapath *od, */ static void build_lswitch_dns_lookup_and_response(struct ovn_datapath *od, - struct hmap *lflows, + struct lflow_table *lflows, const struct shash *meter_groups) { ovs_assert(od->nbs); @@ -9945,7 +9326,7 @@ build_lswitch_dns_lookup_and_response(struct ovn_datapath *od, * binding the external ports. */ static void build_lswitch_external_port(struct ovn_port *op, - struct hmap *lflows) + struct lflow_table *lflows) { ovs_assert(op->nbsp); if (!lsp_is_external(op->nbsp)) { @@ -9961,7 +9342,7 @@ build_lswitch_external_port(struct ovn_port *op, * (priority 70 - 100). */ static void build_lswitch_destination_lookup_bmcast(struct ovn_datapath *od, - struct hmap *lflows, + struct lflow_table *lflows, struct ds *actions, const struct shash *meter_groups) { @@ -10054,7 +9435,7 @@ build_lswitch_destination_lookup_bmcast(struct ovn_datapath *od, * (priority 90). */ static void build_lswitch_ip_mcast_igmp_mld(struct ovn_igmp_group *igmp_group, - struct hmap *lflows, + struct lflow_table *lflows, struct ds *actions, struct ds *match) { @@ -10134,7 +9515,8 @@ build_lswitch_ip_mcast_igmp_mld(struct ovn_igmp_group *igmp_group, /* Ingress table 25: Destination lookup, unicast handling (priority 50), */ static void -build_lswitch_ip_unicast_lookup(struct ovn_port *op, struct hmap *lflows, +build_lswitch_ip_unicast_lookup(struct ovn_port *op, + struct lflow_table *lflows, struct ds *actions, struct ds *match) { ovs_assert(op->nbsp); @@ -10167,10 +9549,12 @@ build_lswitch_ip_unicast_lookup(struct ovn_port *op, struct hmap *lflows, ds_clear(actions); ds_put_format(actions, action, op->json_key); - ovn_lflow_add_with_hint(lflows, op->od, S_SWITCH_IN_L2_LKUP, - 50, ds_cstr(match), - ds_cstr(actions), - &op->nbsp->header_); + ovn_lflow_add_with_lflow_ref_hint(lflows, op->od, + S_SWITCH_IN_L2_LKUP, + 50, ds_cstr(match), + ds_cstr(actions), + &op->nbsp->header_, + op->lflow_ref); } else if (!strcmp(op->nbsp->addresses[i], "unknown")) { continue; } else if (is_dynamic_lsp_address(op->nbsp->addresses[i])) { @@ -10185,10 +9569,12 @@ build_lswitch_ip_unicast_lookup(struct ovn_port *op, struct hmap *lflows, ds_clear(actions); ds_put_format(actions, action, op->json_key); - ovn_lflow_add_with_hint(lflows, op->od, S_SWITCH_IN_L2_LKUP, - 50, ds_cstr(match), - ds_cstr(actions), - &op->nbsp->header_); + ovn_lflow_add_with_lflow_ref_hint(lflows, op->od, + S_SWITCH_IN_L2_LKUP, + 50, ds_cstr(match), + ds_cstr(actions), + &op->nbsp->header_, + op->lflow_ref); } else if (!strcmp(op->nbsp->addresses[i], "router")) { if (!op->peer || !op->peer->nbrp || !ovs_scan(op->peer->nbrp->mac, @@ -10240,10 +9626,11 @@ build_lswitch_ip_unicast_lookup(struct ovn_port *op, struct hmap *lflows, ds_clear(actions); ds_put_format(actions, action, op->json_key); - ovn_lflow_add_with_hint(lflows, op->od, - S_SWITCH_IN_L2_LKUP, 50, - ds_cstr(match), ds_cstr(actions), - &op->nbsp->header_); + ovn_lflow_add_with_lflow_ref_hint(lflows, op->od, + S_SWITCH_IN_L2_LKUP, 50, + ds_cstr(match), ds_cstr(actions), + &op->nbsp->header_, + op->lflow_ref); } else { static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 1); @@ -10258,8 +9645,8 @@ build_lswitch_ip_unicast_lookup(struct ovn_port *op, struct hmap *lflows, static void build_lswitch_ip_unicast_lookup_for_nats(struct ovn_port *op, const struct lr_stateful_record *lr_sful_rec, - struct hmap *lflows, struct ds *match, - struct ds *actions) + struct lflow_table *lflows, struct ds *match, + struct ds *actions, struct lflow_ref *lflow_ref) { ovs_assert(op->nbsp); @@ -10291,11 +9678,12 @@ build_lswitch_ip_unicast_lookup_for_nats(struct ovn_port *op, ds_clear(actions); ds_put_format(actions, action, op->json_key); - ovn_lflow_add_with_hint(lflows, op->od, + ovn_lflow_add_with_lflow_ref_hint(lflows, op->od, S_SWITCH_IN_L2_LKUP, 50, ds_cstr(match), ds_cstr(actions), - &op->nbsp->header_); + &op->nbsp->header_, + lflow_ref); } } } @@ -10535,7 +9923,7 @@ get_outport_for_routing_policy_nexthop(struct ovn_datapath *od, } static void -build_routing_policy_flow(struct hmap *lflows, struct ovn_datapath *od, +build_routing_policy_flow(struct lflow_table *lflows, struct ovn_datapath *od, const struct hmap *lr_ports, const struct nbrec_logical_router_policy *rule, const struct ovsdb_idl_row *stage_hint) @@ -10600,7 +9988,8 @@ build_routing_policy_flow(struct hmap *lflows, struct ovn_datapath *od, } static void -build_ecmp_routing_policy_flows(struct hmap *lflows, struct ovn_datapath *od, +build_ecmp_routing_policy_flows(struct lflow_table *lflows, + struct ovn_datapath *od, const struct hmap *lr_ports, const struct nbrec_logical_router_policy *rule, uint16_t ecmp_group_id) @@ -10736,7 +10125,7 @@ get_route_table_id(struct simap *route_tables, const char *route_table_name) } static void -build_route_table_lflow(struct ovn_datapath *od, struct hmap *lflows, +build_route_table_lflow(struct ovn_datapath *od, struct lflow_table *lflows, struct nbrec_logical_router_port *lrp, struct simap *route_tables) { @@ -11147,7 +10536,7 @@ find_static_route_outport(struct ovn_datapath *od, const struct hmap *lr_ports, } static void -add_ecmp_symmetric_reply_flows(struct hmap *lflows, +add_ecmp_symmetric_reply_flows(struct lflow_table *lflows, struct ovn_datapath *od, bool ct_masked_mark, const char *port_ip, @@ -11312,7 +10701,7 @@ add_ecmp_symmetric_reply_flows(struct hmap *lflows, } static void -build_ecmp_route_flow(struct hmap *lflows, struct ovn_datapath *od, +build_ecmp_route_flow(struct lflow_table *lflows, struct ovn_datapath *od, bool ct_masked_mark, const struct hmap *lr_ports, struct ecmp_groups_node *eg) @@ -11399,12 +10788,12 @@ build_ecmp_route_flow(struct hmap *lflows, struct ovn_datapath *od, } static void -add_route(struct hmap *lflows, struct ovn_datapath *od, +add_route(struct lflow_table *lflows, struct ovn_datapath *od, const struct ovn_port *op, const char *lrp_addr_s, const char *network_s, int plen, const char *gateway, bool is_src_route, const uint32_t rtb_id, const struct ovsdb_idl_row *stage_hint, bool is_discard_route, - int ofs) + int ofs, struct lflow_ref *lflow_ref) { bool is_ipv4 = strchr(network_s, '.') ? true : false; struct ds match = DS_EMPTY_INITIALIZER; @@ -11447,14 +10836,17 @@ add_route(struct hmap *lflows, struct ovn_datapath *od, ds_put_format(&actions, "ip.ttl--; %s", ds_cstr(&common_actions)); } - ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_IP_ROUTING, priority, - ds_cstr(&match), ds_cstr(&actions), - stage_hint); + ovn_lflow_add_with_lflow_ref_hint(lflows, od, S_ROUTER_IN_IP_ROUTING, + priority, ds_cstr(&match), + ds_cstr(&actions), stage_hint, + lflow_ref); if (op && op->has_bfd) { ds_put_format(&match, " && udp.dst == 3784"); - ovn_lflow_add_with_hint(lflows, op->od, S_ROUTER_IN_IP_ROUTING, - priority + 1, ds_cstr(&match), - ds_cstr(&common_actions), stage_hint); + ovn_lflow_add_with_lflow_ref_hint(lflows, op->od, + S_ROUTER_IN_IP_ROUTING, + priority + 1, ds_cstr(&match), + ds_cstr(&common_actions),\ + stage_hint, lflow_ref); } ds_destroy(&match); ds_destroy(&common_actions); @@ -11462,7 +10854,7 @@ add_route(struct hmap *lflows, struct ovn_datapath *od, } static void -build_static_route_flow(struct hmap *lflows, struct ovn_datapath *od, +build_static_route_flow(struct lflow_table *lflows, struct ovn_datapath *od, const struct hmap *lr_ports, const struct parsed_route *route_) { @@ -11488,7 +10880,7 @@ build_static_route_flow(struct hmap *lflows, struct ovn_datapath *od, add_route(lflows, route_->is_discard_route ? od : out_port->od, out_port, lrp_addr_s, prefix_s, route_->plen, route->nexthop, route_->is_src_route, route_->route_table_id, &route->header_, - route_->is_discard_route, ofs); + route_->is_discard_route, ofs, NULL); free(prefix_s); } @@ -11551,7 +10943,7 @@ struct lrouter_nat_lb_flows_ctx { int prio; - struct hmap *lflows; + struct lflow_table *lflows; const struct shash *meter_groups; }; @@ -11682,7 +11074,7 @@ build_lrouter_nat_flows_for_lb(struct ovn_lb_vip *lb_vip, struct ovn_northd_lb_vip *vips_nb, const struct ovn_datapaths *lr_datapaths, const struct lr_stateful_table *lr_sful_table, - struct hmap *lflows, + struct lflow_table *lflows, struct ds *match, struct ds *action, const struct shash *meter_groups, const struct chassis_features *features, @@ -11851,7 +11243,7 @@ build_lrouter_nat_flows_for_lb(struct ovn_lb_vip *lb_vip, static void build_lswitch_flows_for_lb(struct ovn_lb_datapaths *lb_dps, - struct hmap *lflows, + struct lflow_table *lflows, const struct shash *meter_groups, const struct ovn_datapaths *ls_datapaths, const struct chassis_features *features, @@ -11912,7 +11304,7 @@ build_lswitch_flows_for_lb(struct ovn_lb_datapaths *lb_dps, */ static void build_lrouter_defrag_flows_for_lb(struct ovn_lb_datapaths *lb_dps, - struct hmap *lflows, + struct lflow_table *lflows, const struct ovn_datapaths *lr_datapaths, struct ds *match) { @@ -11938,7 +11330,7 @@ build_lrouter_defrag_flows_for_lb(struct ovn_lb_datapaths *lb_dps, static void build_lrouter_flows_for_lb(struct ovn_lb_datapaths *lb_dps, - struct hmap *lflows, + struct lflow_table *lflows, const struct shash *meter_groups, const struct ovn_datapaths *lr_datapaths, const struct lr_stateful_table *lr_sful_table, @@ -12096,7 +11488,7 @@ lrouter_dnat_and_snat_is_stateless(const struct nbrec_nat *nat) */ static inline void lrouter_nat_add_ext_ip_match(const struct ovn_datapath *od, - struct hmap *lflows, struct ds *match, + struct lflow_table *lflows, struct ds *match, const struct nbrec_nat *nat, bool is_v6, bool is_src, int cidr_bits) { @@ -12163,7 +11555,7 @@ build_lrouter_arp_flow(const struct ovn_datapath *od, struct ovn_port *op, const char *ip_address, const char *eth_addr, struct ds *extra_match, bool drop, uint16_t priority, const struct ovsdb_idl_row *hint, - struct hmap *lflows) + struct lflow_table *lflows) { struct ds match = DS_EMPTY_INITIALIZER; struct ds actions = DS_EMPTY_INITIALIZER; @@ -12213,7 +11605,8 @@ build_lrouter_nd_flow(const struct ovn_datapath *od, struct ovn_port *op, const char *sn_ip_address, const char *eth_addr, struct ds *extra_match, bool drop, uint16_t priority, const struct ovsdb_idl_row *hint, - struct hmap *lflows, const struct shash *meter_groups) + struct lflow_table *lflows, + const struct shash *meter_groups) { struct ds match = DS_EMPTY_INITIALIZER; struct ds actions = DS_EMPTY_INITIALIZER; @@ -12264,7 +11657,7 @@ build_lrouter_nd_flow(const struct ovn_datapath *od, struct ovn_port *op, static void build_lrouter_nat_arp_nd_flow(const struct ovn_datapath *od, struct ovn_nat *nat_entry, - struct hmap *lflows, + struct lflow_table *lflows, const struct shash *meter_groups) { struct lport_addresses *ext_addrs = &nat_entry->ext_addrs; @@ -12287,7 +11680,7 @@ build_lrouter_nat_arp_nd_flow(const struct ovn_datapath *od, static void build_lrouter_port_nat_arp_nd_flow(struct ovn_port *op, struct ovn_nat *nat_entry, - struct hmap *lflows, + struct lflow_table *lflows, const struct shash *meter_groups) { struct lport_addresses *ext_addrs = &nat_entry->ext_addrs; @@ -12361,7 +11754,7 @@ build_lrouter_drop_own_dest(struct ovn_port *op, const struct lr_stateful_record *lr_sful_rec, enum ovn_stage stage, uint16_t priority, bool drop_snat_ip, - struct hmap *lflows) + struct lflow_table *lflows) { struct ds match_ips = DS_EMPTY_INITIALIZER; @@ -12426,7 +11819,7 @@ build_lrouter_drop_own_dest(struct ovn_port *op, } static void -build_lrouter_force_snat_flows(struct hmap *lflows, +build_lrouter_force_snat_flows(struct lflow_table *lflows, const struct ovn_datapath *od, const char *ip_version, const char *ip_addr, const char *context) @@ -12455,7 +11848,7 @@ build_lrouter_force_snat_flows(struct hmap *lflows, static void build_lrouter_force_snat_flows_op(struct ovn_port *op, const struct lr_nat_record *lrnat_rec, - struct hmap *lflows, + struct lflow_table *lflows, struct ds *match, struct ds *actions) { ovs_assert(op->nbrp); @@ -12527,7 +11920,7 @@ build_lrouter_force_snat_flows_op(struct ovn_port *op, } static void -build_lrouter_bfd_flows(struct hmap *lflows, struct ovn_port *op, +build_lrouter_bfd_flows(struct lflow_table *lflows, struct ovn_port *op, const struct shash *meter_groups) { if (!op->has_bfd) { @@ -12582,7 +11975,7 @@ build_lrouter_bfd_flows(struct hmap *lflows, struct ovn_port *op, */ static void build_adm_ctrl_flows_for_lrouter( - struct ovn_datapath *od, struct hmap *lflows) + struct ovn_datapath *od, struct lflow_table *lflows) { ovs_assert(od->nbr); /* Logical VLANs not supported. @@ -12626,7 +12019,7 @@ build_gateway_get_l2_hdr_size(struct ovn_port *op) * function. */ static void OVS_PRINTF_FORMAT(9, 10) -build_gateway_mtu_flow(struct hmap *lflows, struct ovn_port *op, +build_gateway_mtu_flow(struct lflow_table *lflows, struct ovn_port *op, enum ovn_stage stage, uint16_t prio_low, uint16_t prio_high, struct ds *match, struct ds *actions, const struct ovsdb_idl_row *hint, @@ -12687,7 +12080,7 @@ consider_l3dgw_port_is_centralized(struct ovn_port *op) */ static void build_adm_ctrl_flows_for_lrouter_port( - struct ovn_port *op, struct hmap *lflows, + struct ovn_port *op, struct lflow_table *lflows, struct ds *match, struct ds *actions) { ovs_assert(op->nbrp); @@ -12741,7 +12134,7 @@ build_adm_ctrl_flows_for_lrouter_port( * lflows for logical routers. */ static void build_neigh_learning_flows_for_lrouter( - struct ovn_datapath *od, struct hmap *lflows, + struct ovn_datapath *od, struct lflow_table *lflows, struct ds *match, struct ds *actions, const struct shash *meter_groups) { @@ -12872,7 +12265,7 @@ build_neigh_learning_flows_for_lrouter( * for logical router ports. */ static void build_neigh_learning_flows_for_lrouter_port( - struct ovn_port *op, struct hmap *lflows, + struct ovn_port *op, struct lflow_table *lflows, struct ds *match, struct ds *actions) { ovs_assert(op->nbrp); @@ -12934,7 +12327,7 @@ build_neigh_learning_flows_for_lrouter_port( * Adv (RA) options and response. */ static void build_ND_RA_flows_for_lrouter_port( - struct ovn_port *op, struct hmap *lflows, + struct ovn_port *op, struct lflow_table *lflows, struct ds *match, struct ds *actions, const struct shash *meter_groups) { @@ -13049,7 +12442,8 @@ build_ND_RA_flows_for_lrouter_port( /* Logical router ingress table ND_RA_OPTIONS & ND_RA_RESPONSE: RS * responder, by default goto next. (priority 0). */ static void -build_ND_RA_flows_for_lrouter(struct ovn_datapath *od, struct hmap *lflows) +build_ND_RA_flows_for_lrouter(struct ovn_datapath *od, + struct lflow_table *lflows) { ovs_assert(od->nbr); ovn_lflow_add(lflows, od, S_ROUTER_IN_ND_RA_OPTIONS, 0, "1", "next;"); @@ -13060,7 +12454,7 @@ build_ND_RA_flows_for_lrouter(struct ovn_datapath *od, struct hmap *lflows) * by default goto next. (priority 0). */ static void build_ip_routing_pre_flows_for_lrouter(struct ovn_datapath *od, - struct hmap *lflows) + struct lflow_table *lflows) { ovs_assert(od->nbr); ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_ROUTING_PRE, 0, "1", @@ -13088,21 +12482,23 @@ build_ip_routing_pre_flows_for_lrouter(struct ovn_datapath *od, */ static void build_ip_routing_flows_for_lrp( - struct ovn_port *op, struct hmap *lflows) + struct ovn_port *op, struct lflow_table *lflows) { ovs_assert(op->nbrp); for (int i = 0; i < op->lrp_networks.n_ipv4_addrs; i++) { add_route(lflows, op->od, op, op->lrp_networks.ipv4_addrs[i].addr_s, op->lrp_networks.ipv4_addrs[i].network_s, op->lrp_networks.ipv4_addrs[i].plen, NULL, false, 0, - &op->nbrp->header_, false, ROUTE_PRIO_OFFSET_CONNECTED); + &op->nbrp->header_, false, ROUTE_PRIO_OFFSET_CONNECTED, + NULL); } for (int i = 0; i < op->lrp_networks.n_ipv6_addrs; i++) { add_route(lflows, op->od, op, op->lrp_networks.ipv6_addrs[i].addr_s, op->lrp_networks.ipv6_addrs[i].network_s, op->lrp_networks.ipv6_addrs[i].plen, NULL, false, 0, - &op->nbrp->header_, false, ROUTE_PRIO_OFFSET_CONNECTED); + &op->nbrp->header_, false, ROUTE_PRIO_OFFSET_CONNECTED, + NULL); } } @@ -13116,7 +12512,8 @@ build_ip_routing_flows_for_lrp( static void build_ip_routing_flows_for_router_type_lsp( struct ovn_port *op, const struct lr_stateful_table *lr_sful_table, - const struct hmap *lr_ports, struct hmap *lflows) + const struct hmap *lr_ports, struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { ovs_assert(op->nbsp); if (!lsp_is_router(op->nbsp)) { @@ -13152,7 +12549,8 @@ build_ip_routing_flows_for_router_type_lsp( laddrs->ipv4_addrs[k].network_s, laddrs->ipv4_addrs[k].plen, NULL, false, 0, &peer->nbrp->header_, false, - ROUTE_PRIO_OFFSET_CONNECTED); + ROUTE_PRIO_OFFSET_CONNECTED, + lflow_ref); } } destroy_routable_addresses(&ra); @@ -13164,7 +12562,7 @@ build_ip_routing_flows_for_router_type_lsp( static void build_static_route_flows_for_lrouter( struct ovn_datapath *od, const struct chassis_features *features, - struct hmap *lflows, const struct hmap *lr_ports, + struct lflow_table *lflows, const struct hmap *lr_ports, const struct hmap *bfd_connections) { ovs_assert(od->nbr); @@ -13228,7 +12626,7 @@ build_static_route_flows_for_lrouter( */ static void build_mcast_lookup_flows_for_lrouter( - struct ovn_datapath *od, struct hmap *lflows, + struct ovn_datapath *od, struct lflow_table *lflows, struct ds *match, struct ds *actions) { ovs_assert(od->nbr); @@ -13329,7 +12727,7 @@ build_mcast_lookup_flows_for_lrouter( * advances to the next table for ARP/ND resolution. */ static void build_ingress_policy_flows_for_lrouter( - struct ovn_datapath *od, struct hmap *lflows, + struct ovn_datapath *od, struct lflow_table *lflows, const struct hmap *lr_ports) { ovs_assert(od->nbr); @@ -13363,7 +12761,7 @@ build_ingress_policy_flows_for_lrouter( /* Local router ingress table ARP_RESOLVE: ARP Resolution. */ static void build_arp_resolve_flows_for_lrouter( - struct ovn_datapath *od, struct hmap *lflows) + struct ovn_datapath *od, struct lflow_table *lflows) { ovs_assert(od->nbr); /* Multicast packets already have the outport set so just advance to @@ -13381,10 +12779,12 @@ build_arp_resolve_flows_for_lrouter( } static void -routable_addresses_to_lflows(struct hmap *lflows, struct ovn_port *router_port, +routable_addresses_to_lflows(struct lflow_table *lflows, + struct ovn_port *router_port, struct ovn_port *peer, const struct lr_stateful_record *lr_sful_rec, - struct ds *match, struct ds *actions) + struct ds *match, struct ds *actions, + struct lflow_ref *lflow_ref) { struct ovn_port_routable_addresses ra = get_op_routable_addresses(router_port, lr_sful_rec); @@ -13408,8 +12808,9 @@ routable_addresses_to_lflows(struct hmap *lflows, struct ovn_port *router_port, ds_clear(actions); ds_put_format(actions, "eth.dst = %s; next;", ra.laddrs[i].ea_s); - ovn_lflow_add(lflows, peer->od, S_ROUTER_IN_ARP_RESOLVE, 100, - ds_cstr(match), ds_cstr(actions)); + ovn_lflow_add_with_lflow_ref(lflows, peer->od, S_ROUTER_IN_ARP_RESOLVE, + 100, ds_cstr(match), ds_cstr(actions), + lflow_ref); } destroy_routable_addresses(&ra); } @@ -13428,7 +12829,7 @@ routable_addresses_to_lflows(struct hmap *lflows, struct ovn_port *router_port, static void build_arp_resolve_flows_for_lrp( struct ovn_port *op, - struct hmap *lflows, struct ds *match, struct ds *actions) + struct lflow_table *lflows, struct ds *match, struct ds *actions) { ovs_assert(op->nbrp); /* This is a logical router port. If next-hop IP address in @@ -13502,7 +12903,7 @@ build_arp_resolve_flows_for_lrp( /* This function adds ARP resolve flows related to a LSP. */ static void build_arp_resolve_flows_for_lsp( - struct ovn_port *op, struct hmap *lflows, + struct ovn_port *op, struct lflow_table *lflows, const struct hmap *lr_ports, struct ds *match, struct ds *actions) { @@ -13544,11 +12945,12 @@ build_arp_resolve_flows_for_lsp( ds_clear(actions); ds_put_format(actions, "eth.dst = %s; next;", ea_s); - ovn_lflow_add_with_hint(lflows, peer->od, + ovn_lflow_add_with_lflow_ref_hint(lflows, peer->od, S_ROUTER_IN_ARP_RESOLVE, 100, ds_cstr(match), ds_cstr(actions), - &op->nbsp->header_); + &op->nbsp->header_, + op->lflow_ref); } } @@ -13575,11 +12977,12 @@ build_arp_resolve_flows_for_lsp( ds_clear(actions); ds_put_format(actions, "eth.dst = %s; next;", ea_s); - ovn_lflow_add_with_hint(lflows, peer->od, + ovn_lflow_add_with_lflow_ref_hint(lflows, peer->od, S_ROUTER_IN_ARP_RESOLVE, 100, ds_cstr(match), ds_cstr(actions), - &op->nbsp->header_); + &op->nbsp->header_, + op->lflow_ref); } } } @@ -13623,10 +13026,11 @@ build_arp_resolve_flows_for_lsp( ds_clear(actions); ds_put_format(actions, "eth.dst = %s; next;", router_port->lrp_networks.ea_s); - ovn_lflow_add_with_hint(lflows, peer->od, + ovn_lflow_add_with_lflow_ref_hint(lflows, peer->od, S_ROUTER_IN_ARP_RESOLVE, 100, ds_cstr(match), ds_cstr(actions), - &op->nbsp->header_); + &op->nbsp->header_, + op->lflow_ref); } if (router_port->lrp_networks.n_ipv6_addrs) { @@ -13639,10 +13043,11 @@ build_arp_resolve_flows_for_lsp( ds_clear(actions); ds_put_format(actions, "eth.dst = %s; next;", router_port->lrp_networks.ea_s); - ovn_lflow_add_with_hint(lflows, peer->od, + ovn_lflow_add_with_lflow_ref_hint(lflows, peer->od, S_ROUTER_IN_ARP_RESOLVE, 100, ds_cstr(match), ds_cstr(actions), - &op->nbsp->header_); + &op->nbsp->header_, + op->lflow_ref); } } } @@ -13650,10 +13055,11 @@ build_arp_resolve_flows_for_lsp( static void build_arp_resolve_flows_for_lsp_routable_addresses( - struct ovn_port *op, struct hmap *lflows, + struct ovn_port *op, struct lflow_table *lflows, const struct hmap *lr_ports, const struct lr_stateful_table *lr_sful_table, - struct ds *match, struct ds *actions) + struct ds *match, struct ds *actions, + struct lflow_ref *lflow_ref) { if (!lsp_is_router(op->nbsp)) { return; @@ -13687,13 +13093,15 @@ build_arp_resolve_flows_for_lsp_routable_addresses( lr_sful_rec = lr_stateful_table_find_by_index(lr_sful_table, router_port->od->index); routable_addresses_to_lflows(lflows, router_port, peer, - lr_sful_rec, match, actions); + lr_sful_rec, match, actions, + lflow_ref); } } } static void -build_icmperr_pkt_big_flows(struct ovn_port *op, int mtu, struct hmap *lflows, +build_icmperr_pkt_big_flows(struct ovn_port *op, int mtu, + struct lflow_table *lflows, const struct shash *meter_groups, struct ds *match, struct ds *actions, enum ovn_stage stage, struct ovn_port *outport) @@ -13786,7 +13194,7 @@ build_icmperr_pkt_big_flows(struct ovn_port *op, int mtu, struct hmap *lflows, static void build_check_pkt_len_flows_for_lrp(struct ovn_port *op, - struct hmap *lflows, + struct lflow_table *lflows, const struct hmap *lr_ports, const struct shash *meter_groups, struct ds *match, @@ -13836,7 +13244,7 @@ build_check_pkt_len_flows_for_lrp(struct ovn_port *op, * */ static void build_check_pkt_len_flows_for_lrouter( - struct ovn_datapath *od, struct hmap *lflows, + struct ovn_datapath *od, struct lflow_table *lflows, const struct hmap *lr_ports, struct ds *match, struct ds *actions, const struct shash *meter_groups) @@ -13863,7 +13271,7 @@ build_check_pkt_len_flows_for_lrouter( /* Logical router ingress table GW_REDIRECT: Gateway redirect. */ static void build_gateway_redirect_flows_for_lrouter( - struct ovn_datapath *od, struct hmap *lflows, + struct ovn_datapath *od, struct lflow_table *lflows, struct ds *match, struct ds *actions) { ovs_assert(od->nbr); @@ -13908,7 +13316,7 @@ build_gateway_redirect_flows_for_lrouter( static void build_lr_gateway_redirect_flows_for_nats( const struct ovn_datapath *od, const struct lr_nat_record *lrnat_rec, - struct hmap *lflows, struct ds *match, struct ds *actions) + struct lflow_table *lflows, struct ds *match, struct ds *actions) { ovs_assert(od->nbr); for (size_t i = 0; i < od->n_l3dgw_ports; i++) { @@ -13977,7 +13385,7 @@ build_lr_gateway_redirect_flows_for_nats( * and sends an ARP/IPv6 NA request (priority 100). */ static void build_arp_request_flows_for_lrouter( - struct ovn_datapath *od, struct hmap *lflows, + struct ovn_datapath *od, struct lflow_table *lflows, struct ds *match, struct ds *actions, const struct shash *meter_groups) { @@ -14055,7 +13463,7 @@ build_arp_request_flows_for_lrouter( */ static void build_egress_delivery_flows_for_lrouter_port( - struct ovn_port *op, struct hmap *lflows, + struct ovn_port *op, struct lflow_table *lflows, struct ds *match, struct ds *actions) { ovs_assert(op->nbrp); @@ -14097,7 +13505,7 @@ build_egress_delivery_flows_for_lrouter_port( static void build_misc_local_traffic_drop_flows_for_lrouter( - struct ovn_datapath *od, struct hmap *lflows) + struct ovn_datapath *od, struct lflow_table *lflows) { ovs_assert(od->nbr); /* Allow IGMP and MLD packets (with TTL = 1) if the router is @@ -14179,7 +13587,7 @@ build_misc_local_traffic_drop_flows_for_lrouter( static void build_dhcpv6_reply_flows_for_lrouter_port( - struct ovn_port *op, struct hmap *lflows, + struct ovn_port *op, struct lflow_table *lflows, struct ds *match) { ovs_assert(op->nbrp); @@ -14199,7 +13607,7 @@ build_dhcpv6_reply_flows_for_lrouter_port( static void build_ipv6_input_flows_for_lrouter_port( - struct ovn_port *op, struct hmap *lflows, + struct ovn_port *op, struct lflow_table *lflows, struct ds *match, struct ds *actions, const struct shash *meter_groups) { @@ -14368,7 +13776,7 @@ build_ipv6_input_flows_for_lrouter_port( static void build_lrouter_arp_nd_for_datapath(const struct ovn_datapath *od, const struct lr_nat_record *lrnat_rec, - struct hmap *lflows, + struct lflow_table *lflows, const struct shash *meter_groups) { ovs_assert(od->nbr); @@ -14420,7 +13828,7 @@ build_lrouter_arp_nd_for_datapath(const struct ovn_datapath *od, /* Logical router ingress table 3: IP Input for IPv4. */ static void build_lrouter_ipv4_ip_input(struct ovn_port *op, - struct hmap *lflows, + struct lflow_table *lflows, struct ds *match, struct ds *actions, const struct shash *meter_groups) { @@ -14624,7 +14032,7 @@ build_lrouter_ipv4_ip_input(struct ovn_port *op, /* Logical router ingress table 3: IP Input for IPv4. */ static void build_lrouter_ipv4_ip_input_for_lbnats(struct ovn_port *op, - struct hmap *lflows, + struct lflow_table *lflows, const struct lr_stateful_record *lr_sful_rec, struct ds *match, const struct shash *meter_groups) { @@ -14743,7 +14151,7 @@ build_lrouter_in_unsnat_match(const struct ovn_datapath *od, } static void -build_lrouter_in_unsnat_stateless_flow(struct hmap *lflows, +build_lrouter_in_unsnat_stateless_flow(struct lflow_table *lflows, const struct ovn_datapath *od, const struct nbrec_nat *nat, struct ds *match, @@ -14765,7 +14173,7 @@ build_lrouter_in_unsnat_stateless_flow(struct hmap *lflows, } static void -build_lrouter_in_unsnat_in_czone_flow(struct hmap *lflows, +build_lrouter_in_unsnat_in_czone_flow(struct lflow_table *lflows, const struct ovn_datapath *od, const struct nbrec_nat *nat, struct ds *match, bool distributed_nat, @@ -14799,7 +14207,7 @@ build_lrouter_in_unsnat_in_czone_flow(struct hmap *lflows, } static void -build_lrouter_in_unsnat_flow(struct hmap *lflows, +build_lrouter_in_unsnat_flow(struct lflow_table *lflows, const struct ovn_datapath *od, const struct nbrec_nat *nat, struct ds *match, bool distributed_nat, bool is_v6, @@ -14821,7 +14229,7 @@ build_lrouter_in_unsnat_flow(struct hmap *lflows, } static void -build_lrouter_in_dnat_flow(struct hmap *lflows, +build_lrouter_in_dnat_flow(struct lflow_table *lflows, const struct ovn_datapath *od, const struct lr_nat_record *lrnat_rec, const struct nbrec_nat *nat, struct ds *match, @@ -14893,7 +14301,7 @@ build_lrouter_in_dnat_flow(struct hmap *lflows, } static void -build_lrouter_out_undnat_flow(struct hmap *lflows, +build_lrouter_out_undnat_flow(struct lflow_table *lflows, const struct ovn_datapath *od, const struct nbrec_nat *nat, struct ds *match, struct ds *actions, bool distributed_nat, @@ -14944,7 +14352,7 @@ build_lrouter_out_undnat_flow(struct hmap *lflows, } static void -build_lrouter_out_is_dnat_local(struct hmap *lflows, +build_lrouter_out_is_dnat_local(struct lflow_table *lflows, const struct ovn_datapath *od, const struct nbrec_nat *nat, struct ds *match, struct ds *actions, bool distributed_nat, @@ -14975,7 +14383,7 @@ build_lrouter_out_is_dnat_local(struct hmap *lflows, } static void -build_lrouter_out_snat_match(struct hmap *lflows, +build_lrouter_out_snat_match(struct lflow_table *lflows, const struct ovn_datapath *od, const struct nbrec_nat *nat, struct ds *match, bool distributed_nat, int cidr_bits, bool is_v6, @@ -15004,7 +14412,7 @@ build_lrouter_out_snat_match(struct hmap *lflows, } static void -build_lrouter_out_snat_stateless_flow(struct hmap *lflows, +build_lrouter_out_snat_stateless_flow(struct lflow_table *lflows, const struct ovn_datapath *od, const struct nbrec_nat *nat, struct ds *match, struct ds *actions, @@ -15047,7 +14455,7 @@ build_lrouter_out_snat_stateless_flow(struct hmap *lflows, } static void -build_lrouter_out_snat_in_czone_flow(struct hmap *lflows, +build_lrouter_out_snat_in_czone_flow(struct lflow_table *lflows, const struct ovn_datapath *od, const struct nbrec_nat *nat, struct ds *match, @@ -15109,7 +14517,7 @@ build_lrouter_out_snat_in_czone_flow(struct hmap *lflows, } static void -build_lrouter_out_snat_flow(struct hmap *lflows, +build_lrouter_out_snat_flow(struct lflow_table *lflows, const struct ovn_datapath *od, const struct nbrec_nat *nat, struct ds *match, struct ds *actions, bool distributed_nat, @@ -15155,7 +14563,7 @@ build_lrouter_out_snat_flow(struct hmap *lflows, } static void -build_lrouter_ingress_nat_check_pkt_len(struct hmap *lflows, +build_lrouter_ingress_nat_check_pkt_len(struct lflow_table *lflows, const struct nbrec_nat *nat, const struct ovn_datapath *od, bool is_v6, struct ds *match, @@ -15227,7 +14635,7 @@ build_lrouter_ingress_nat_check_pkt_len(struct hmap *lflows, } static void -build_lrouter_ingress_flow(struct hmap *lflows, +build_lrouter_ingress_flow(struct lflow_table *lflows, const struct ovn_datapath *od, const struct nbrec_nat *nat, struct ds *match, struct ds *actions, struct eth_addr mac, @@ -15407,7 +14815,7 @@ lrouter_check_nat_entry(const struct ovn_datapath *od, /* NAT, Defrag and load balancing. */ static void build_lr_nat_defrag_and_lb_default_flows(struct ovn_datapath *od, - struct hmap *lflows) + struct lflow_table *lflows) { ovs_assert(od->nbr); @@ -15432,7 +14840,8 @@ static void build_lr_nat_defrag_and_lb_default_flows(struct ovn_datapath *od, static void build_lrouter_nat_defrag_and_lb( - const struct lr_stateful_record *lr_sful_rec, struct hmap *lflows, + const struct lr_stateful_record *lr_sful_rec, + struct lflow_table *lflows, const struct hmap *ls_ports, const struct hmap *lr_ports, struct ds *match, struct ds *actions, const struct shash *meter_groups, @@ -15815,23 +15224,22 @@ build_lsp_lflows_for_lbnats(struct ovn_port *lsp, const struct lr_stateful_record *lr_sful_rec, const struct lr_stateful_table *lr_sful_table, const struct hmap *lr_ports, - struct hmap *lflows, + struct lflow_table *lflows, struct ds *match, - struct ds *actions) + struct ds *actions, + struct lflow_ref *lflow_ref) { ovs_assert(lsp->nbsp); - start_collecting_lflows(); build_lswitch_rport_arp_req_flows_for_lbnats( lrp_peer, lr_sful_rec, lsp->od, lsp, - lflows, &lsp->nbsp->header_); + lflows, &lsp->nbsp->header_, lflow_ref); build_ip_routing_flows_for_router_type_lsp(lsp, lr_sful_table, - lr_ports, lflows); + lr_ports, lflows, + lflow_ref); build_arp_resolve_flows_for_lsp_routable_addresses( - lsp, lflows, lr_ports, lr_sful_table, match, actions); + lsp, lflows, lr_ports, lr_sful_table, match, actions, lflow_ref); build_lswitch_ip_unicast_lookup_for_nats(lsp, lr_sful_rec, lflows, - match, actions); - link_ovn_port_to_lflows(lsp, &collected_lflows); - end_collecting_lflows(); + match, actions, lflow_ref); } static void @@ -15840,7 +15248,7 @@ build_lbnat_lflows_iterate_by_lsp(struct ovn_port *op, const struct hmap *lr_ports, struct ds *match, struct ds *actions, - struct hmap *lflows) + struct lflow_table *lflows) { ovs_assert(op->nbsp); @@ -15855,7 +15263,7 @@ build_lbnat_lflows_iterate_by_lsp(struct ovn_port *op, build_lsp_lflows_for_lbnats(op, op->peer, lr_sful_rec, lr_sful_table, lr_ports, lflows, - match, actions); + match, actions, op->stateful_lflow_ref); } static void @@ -15863,7 +15271,7 @@ build_lrp_lflows_for_lbnats(struct ovn_port *op, const struct lr_stateful_record *lr_sful_rec, const struct shash *meter_groups, struct ds *match, struct ds *actions, - struct hmap *lflows) + struct lflow_table *lflows) { /* Drop IP traffic destined to router owned IPs except if the IP is * also a SNAT IP. Those are dropped later, in stage @@ -15900,7 +15308,7 @@ build_lbnat_lflows_iterate_by_lrp(struct ovn_port *op, const struct shash *meter_groups, struct ds *match, struct ds *actions, - struct hmap *lflows) + struct lflow_table *lflows) { ovs_assert(op->nbrp); @@ -15915,7 +15323,7 @@ build_lbnat_lflows_iterate_by_lrp(struct ovn_port *op, static void build_lr_stateful_flows(const struct lr_stateful_record *lr_sful_rec, - struct hmap *lflows, + struct lflow_table *lflows, const struct hmap *ls_ports, const struct hmap *lr_ports, struct ds *match, @@ -15938,7 +15346,7 @@ build_ls_stateful_flows(const struct ls_stateful_record *ls_stateful_rec, const struct ls_port_group_table *ls_pgs, const struct chassis_features *features, const struct shash *meter_groups, - struct hmap *lflows) + struct lflow_table *lflows) { ovs_assert(ls_stateful_rec->od); @@ -15957,7 +15365,7 @@ struct lswitch_flow_build_info { const struct ls_port_group_table *ls_port_groups; const struct lr_stateful_table *lr_sful_table; const struct ls_stateful_table *ls_sful_table; - struct hmap *lflows; + struct lflow_table *lflows; struct hmap *igmp_groups; const struct shash *meter_groups; const struct hmap *lb_dps_map; @@ -16040,10 +15448,9 @@ build_lswitch_and_lrouter_iterate_by_lsp( const struct shash *meter_groups, struct ds *match, struct ds *actions, - struct hmap *lflows) + struct lflow_table *lflows) { ovs_assert(op->nbsp); - start_collecting_lflows(); /* Build Logical Switch Flows. */ build_lswitch_port_sec_op(op, lflows, actions, match); @@ -16058,9 +15465,6 @@ build_lswitch_and_lrouter_iterate_by_lsp( /* Build Logical Router Flows. */ build_arp_resolve_flows_for_lsp(op, lflows, lr_ports, match, actions); - - link_ovn_port_to_lflows(op, &collected_lflows); - end_collecting_lflows(); } /* Helper function to combine all lflow generation which is iterated by logical @@ -16271,7 +15675,7 @@ noop_callback(struct worker_pool *pool OVS_UNUSED, /* Do nothing */ } -/* Fixes the hmap size (hmap->n) after parallel building the lflow_map when +/* Fixes the hmap size (hmap->n) after parallel building the lflow_table when * dp-groups is enabled, because in that case all threads are updating the * global lflow hmap. Although the lflow_hash_lock prevents currently inserting * to the same hash bucket, the hmap->n is updated currently by all threads and @@ -16281,7 +15685,7 @@ noop_callback(struct worker_pool *pool OVS_UNUSED, * after the worker threads complete the tasks in each iteration before any * future operations on the lflow map. */ static void -fix_flow_map_size(struct hmap *lflow_map, +fix_flow_table_size(struct lflow_table *lflow_table, struct lswitch_flow_build_info *lsiv, size_t n_lsiv) { @@ -16289,7 +15693,7 @@ fix_flow_map_size(struct hmap *lflow_map, for (size_t i = 0; i < n_lsiv; i++) { total += lsiv[i].thread_lflow_counter; } - lflow_map->n = total; + lflow_table_set_size(lflow_table, total); } static void @@ -16300,7 +15704,7 @@ build_lswitch_and_lrouter_flows(const struct ovn_datapaths *ls_datapaths, const struct ls_port_group_table *ls_pgs, const struct lr_stateful_table *lr_sful_table, const struct ls_stateful_table *ls_sful_table, - struct hmap *lflows, + struct lflow_table *lflows, struct hmap *igmp_groups, const struct shash *meter_groups, const struct hmap *lb_dps_map, @@ -16347,7 +15751,7 @@ build_lswitch_and_lrouter_flows(const struct ovn_datapaths *ls_datapaths, /* Run thread pool. */ run_pool_callback(build_lflows_pool, NULL, NULL, noop_callback); - fix_flow_map_size(lflows, lsiv, build_lflows_pool->size); + fix_flow_table_size(lflows, lsiv, build_lflows_pool->size); for (index = 0; index < build_lflows_pool->size; index++) { ds_destroy(&lsiv[index].match); @@ -16461,24 +15865,6 @@ build_lswitch_and_lrouter_flows(const struct ovn_datapaths *ls_datapaths, free(svc_check_match); } -static ssize_t max_seen_lflow_size = 128; - -void -lflow_data_init(struct lflow_data *data) -{ - fast_hmap_size_for(&data->lflows, max_seen_lflow_size); -} - -void -lflow_data_destroy(struct lflow_data *data) -{ - struct ovn_lflow *lflow; - HMAP_FOR_EACH_SAFE (lflow, hmap_node, &data->lflows) { - ovn_lflow_destroy(&data->lflows, lflow); - } - hmap_destroy(&data->lflows); -} - void run_update_worker_pool(int n_threads) { /* If number of threads has been updated (or initially set), @@ -16524,7 +15910,7 @@ create_sb_multicast_group(struct ovsdb_idl_txn *ovnsb_txn, * constructing their contents based on the OVN_NB database. */ void build_lflows(struct ovsdb_idl_txn *ovnsb_txn, struct lflow_input *input_data, - struct hmap *lflows) + struct lflow_table *lflows) { struct hmap mcast_groups; struct hmap igmp_groups; @@ -16555,281 +15941,26 @@ void build_lflows(struct ovsdb_idl_txn *ovnsb_txn, } /* Parallel build may result in a suboptimal hash. Resize the - * hash to a correct size before doing lookups */ - - hmap_expand(lflows); - - if (hmap_count(lflows) > max_seen_lflow_size) { - max_seen_lflow_size = hmap_count(lflows); - } - - stopwatch_start(LFLOWS_DP_GROUPS_STOPWATCH_NAME, time_msec()); - /* Collecting all unique datapath groups. */ - struct hmap ls_dp_groups = HMAP_INITIALIZER(&ls_dp_groups); - struct hmap lr_dp_groups = HMAP_INITIALIZER(&lr_dp_groups); - struct hmap single_dp_lflows; - - /* Single dp_flows will never grow bigger than lflows, - * thus the two hmaps will remain the same size regardless - * of how many elements we remove from lflows and add to - * single_dp_lflows. - * Note - lflows is always sized for at least 128 flows. - */ - fast_hmap_size_for(&single_dp_lflows, max_seen_lflow_size); - - struct ovn_lflow *lflow; - HMAP_FOR_EACH_SAFE (lflow, hmap_node, lflows) { - struct ovn_datapath **datapaths_array; - size_t n_datapaths; - - if (ovn_stage_to_datapath_type(lflow->stage) == DP_SWITCH) { - n_datapaths = ods_size(input_data->ls_datapaths); - datapaths_array = input_data->ls_datapaths->array; - } else { - n_datapaths = ods_size(input_data->lr_datapaths); - datapaths_array = input_data->lr_datapaths->array; - } - - lflow->n_ods = bitmap_count1(lflow->dpg_bitmap, n_datapaths); - - ovs_assert(lflow->n_ods); - - if (lflow->n_ods == 1) { - /* There is only one datapath, so it should be moved out of the - * group to a single 'od'. */ - size_t index = bitmap_scan(lflow->dpg_bitmap, true, 0, - n_datapaths); - - bitmap_set0(lflow->dpg_bitmap, index); - lflow->od = datapaths_array[index]; - - /* Logical flow should be re-hashed to allow lookups. */ - uint32_t hash = hmap_node_hash(&lflow->hmap_node); - /* Remove from lflows. */ - hmap_remove(lflows, &lflow->hmap_node); - hash = ovn_logical_flow_hash_datapath(&lflow->od->sb->header_.uuid, - hash); - /* Add to single_dp_lflows. */ - hmap_insert_fast(&single_dp_lflows, &lflow->hmap_node, hash); - } - } - - /* Merge multiple and single dp hashes. */ - - fast_hmap_merge(lflows, &single_dp_lflows); - - hmap_destroy(&single_dp_lflows); - - stopwatch_stop(LFLOWS_DP_GROUPS_STOPWATCH_NAME, time_msec()); + * lflow map to a correct size before doing lookups */ + lflow_table_expand(lflows); + stopwatch_start(LFLOWS_TO_SB_STOPWATCH_NAME, time_msec()); - - struct hmap lflows_temp = HMAP_INITIALIZER(&lflows_temp); - /* Push changes to the Logical_Flow table to database. */ - const struct sbrec_logical_flow *sbflow; - SBREC_LOGICAL_FLOW_TABLE_FOR_EACH_SAFE (sbflow, - input_data->sbrec_logical_flow_table) { - struct sbrec_logical_dp_group *dp_group = sbflow->logical_dp_group; - struct ovn_datapath *logical_datapath_od = NULL; - size_t i; - - /* Find one valid datapath to get the datapath type. */ - struct sbrec_datapath_binding *dp = sbflow->logical_datapath; - if (dp) { - logical_datapath_od = ovn_datapath_from_sbrec( - &input_data->ls_datapaths->datapaths, - &input_data->lr_datapaths->datapaths, - dp); - if (logical_datapath_od - && ovn_datapath_is_stale(logical_datapath_od)) { - logical_datapath_od = NULL; - } - } - for (i = 0; dp_group && i < dp_group->n_datapaths; i++) { - logical_datapath_od = ovn_datapath_from_sbrec( - &input_data->ls_datapaths->datapaths, - &input_data->lr_datapaths->datapaths, - dp_group->datapaths[i]); - if (logical_datapath_od - && !ovn_datapath_is_stale(logical_datapath_od)) { - break; - } - logical_datapath_od = NULL; - } - - if (!logical_datapath_od) { - /* This lflow has no valid logical datapaths. */ - sbrec_logical_flow_delete(sbflow); - continue; - } - - enum ovn_pipeline pipeline - = !strcmp(sbflow->pipeline, "ingress") ? P_IN : P_OUT; - - lflow = ovn_lflow_find( - lflows, dp_group ? NULL : logical_datapath_od, - ovn_stage_build(ovn_datapath_get_type(logical_datapath_od), - pipeline, sbflow->table_id), - sbflow->priority, sbflow->match, sbflow->actions, - sbflow->controller_meter, sbflow->hash); - if (lflow) { - struct hmap *dp_groups; - size_t n_datapaths; - bool is_switch; - - lflow->sb_uuid = sbflow->header_.uuid; - is_switch = ovn_stage_to_datapath_type(lflow->stage) == DP_SWITCH; - if (is_switch) { - n_datapaths = ods_size(input_data->ls_datapaths); - dp_groups = &ls_dp_groups; - } else { - n_datapaths = ods_size(input_data->lr_datapaths); - dp_groups = &lr_dp_groups; - } - if (input_data->ovn_internal_version_changed) { - const char *stage_name = smap_get_def(&sbflow->external_ids, - "stage-name", ""); - const char *stage_hint = smap_get_def(&sbflow->external_ids, - "stage-hint", ""); - const char *source = smap_get_def(&sbflow->external_ids, - "source", ""); - - if (strcmp(stage_name, ovn_stage_to_str(lflow->stage))) { - sbrec_logical_flow_update_external_ids_setkey(sbflow, - "stage-name", ovn_stage_to_str(lflow->stage)); - } - if (lflow->stage_hint) { - if (strcmp(stage_hint, lflow->stage_hint)) { - sbrec_logical_flow_update_external_ids_setkey(sbflow, - "stage-hint", lflow->stage_hint); - } - } - if (lflow->where) { - if (strcmp(source, lflow->where)) { - sbrec_logical_flow_update_external_ids_setkey(sbflow, - "source", lflow->where); - } - } - } - - if (lflow->od) { - sbrec_logical_flow_set_logical_datapath(sbflow, lflow->od->sb); - sbrec_logical_flow_set_logical_dp_group(sbflow, NULL); - } else { - lflow->dpg = ovn_dp_group_get_or_create( - ovnsb_txn, dp_groups, dp_group, - lflow->n_ods, lflow->dpg_bitmap, - n_datapaths, is_switch, - input_data->ls_datapaths, - input_data->lr_datapaths); - - sbrec_logical_flow_set_logical_datapath(sbflow, NULL); - sbrec_logical_flow_set_logical_dp_group(sbflow, - lflow->dpg->dp_group); - } - - /* This lflow updated. Not needed anymore. */ - hmap_remove(lflows, &lflow->hmap_node); - hmap_insert(&lflows_temp, &lflow->hmap_node, - hmap_node_hash(&lflow->hmap_node)); - } else { - sbrec_logical_flow_delete(sbflow); - } - } - - HMAP_FOR_EACH_SAFE (lflow, hmap_node, lflows) { - const char *pipeline = ovn_stage_get_pipeline_name(lflow->stage); - uint8_t table = ovn_stage_get_table(lflow->stage); - struct hmap *dp_groups; - size_t n_datapaths; - bool is_switch; - - is_switch = ovn_stage_to_datapath_type(lflow->stage) == DP_SWITCH; - if (is_switch) { - n_datapaths = ods_size(input_data->ls_datapaths); - dp_groups = &ls_dp_groups; - } else { - n_datapaths = ods_size(input_data->lr_datapaths); - dp_groups = &lr_dp_groups; - } - - lflow->sb_uuid = uuid_random(); - sbflow = sbrec_logical_flow_insert_persist_uuid(ovnsb_txn, - &lflow->sb_uuid); - if (lflow->od) { - sbrec_logical_flow_set_logical_datapath(sbflow, lflow->od->sb); - } else { - lflow->dpg = ovn_dp_group_get_or_create( - ovnsb_txn, dp_groups, NULL, - lflow->n_ods, lflow->dpg_bitmap, - n_datapaths, is_switch, - input_data->ls_datapaths, - input_data->lr_datapaths); - - sbrec_logical_flow_set_logical_dp_group(sbflow, - lflow->dpg->dp_group); - } - - sbrec_logical_flow_set_pipeline(sbflow, pipeline); - sbrec_logical_flow_set_table_id(sbflow, table); - sbrec_logical_flow_set_priority(sbflow, lflow->priority); - sbrec_logical_flow_set_match(sbflow, lflow->match); - sbrec_logical_flow_set_actions(sbflow, lflow->actions); - if (lflow->io_port) { - struct smap tags = SMAP_INITIALIZER(&tags); - smap_add(&tags, "in_out_port", lflow->io_port); - sbrec_logical_flow_set_tags(sbflow, &tags); - smap_destroy(&tags); - } - sbrec_logical_flow_set_controller_meter(sbflow, lflow->ctrl_meter); - - /* Trim the source locator lflow->where, which looks something like - * "ovn/northd/northd.c:1234", down to just the part following the - * last slash, e.g. "northd.c:1234". */ - const char *slash = strrchr(lflow->where, '/'); -#if _WIN32 - const char *backslash = strrchr(lflow->where, '\\'); - if (!slash || backslash > slash) { - slash = backslash; - } -#endif - const char *where = slash ? slash + 1 : lflow->where; - - struct smap ids = SMAP_INITIALIZER(&ids); - smap_add(&ids, "stage-name", ovn_stage_to_str(lflow->stage)); - smap_add(&ids, "source", where); - if (lflow->stage_hint) { - smap_add(&ids, "stage-hint", lflow->stage_hint); - } - sbrec_logical_flow_set_external_ids(sbflow, &ids); - smap_destroy(&ids); - hmap_remove(lflows, &lflow->hmap_node); - hmap_insert(&lflows_temp, &lflow->hmap_node, - hmap_node_hash(&lflow->hmap_node)); - } - hmap_swap(lflows, &lflows_temp); - hmap_destroy(&lflows_temp); + lflow_table_sync_to_sb(lflows, ovnsb_txn, input_data->ls_datapaths, + input_data->lr_datapaths, + input_data->ovn_internal_version_changed, + input_data->sbrec_logical_flow_table, + input_data->sbrec_logical_dp_group_table); stopwatch_stop(LFLOWS_TO_SB_STOPWATCH_NAME, time_msec()); - struct ovn_dp_group *dpg; - HMAP_FOR_EACH_POP (dpg, node, &ls_dp_groups) { - bitmap_free(dpg->bitmap); - free(dpg); - } - hmap_destroy(&ls_dp_groups); - HMAP_FOR_EACH_POP (dpg, node, &lr_dp_groups) { - bitmap_free(dpg->bitmap); - free(dpg); - } - hmap_destroy(&lr_dp_groups); /* Push changes to the Multicast_Group table to database. */ const struct sbrec_multicast_group *sbmc; SBREC_MULTICAST_GROUP_TABLE_FOR_EACH_SAFE (sbmc, input_data->sbrec_multicast_group_table) { struct ovn_datapath *od = ovn_datapath_from_sbrec( - &input_data->ls_datapaths->datapaths, - &input_data->lr_datapaths->datapaths, - sbmc->datapath); + &input_data->ls_datapaths->datapaths, + &input_data->lr_datapaths->datapaths, + sbmc->datapath); if (!od || ovn_datapath_is_stale(od)) { sbrec_multicast_group_delete(sbmc); @@ -16869,120 +16000,21 @@ void build_lflows(struct ovsdb_idl_txn *ovnsb_txn, hmap_destroy(&mcast_groups); } -static void -sync_lsp_lflows_to_sb(struct ovsdb_idl_txn *ovnsb_txn, - struct lflow_input *lflow_input, - struct hmap *lflows, - struct ovn_lflow *lflow) -{ - size_t n_datapaths; - struct ovn_datapath **datapaths_array; - if (ovn_stage_to_datapath_type(lflow->stage) == DP_SWITCH) { - n_datapaths = ods_size(lflow_input->ls_datapaths); - datapaths_array = lflow_input->ls_datapaths->array; - } else { - n_datapaths = ods_size(lflow_input->lr_datapaths); - datapaths_array = lflow_input->lr_datapaths->array; - } - uint32_t n_ods = bitmap_count1(lflow->dpg_bitmap, n_datapaths); - ovs_assert(n_ods == 1); - /* There is only one datapath, so it should be moved out of the - * group to a single 'od'. */ - size_t index = bitmap_scan(lflow->dpg_bitmap, true, 0, - n_datapaths); - - bitmap_set0(lflow->dpg_bitmap, index); - lflow->od = datapaths_array[index]; - - /* Logical flow should be re-hashed to allow lookups. */ - uint32_t hash = hmap_node_hash(&lflow->hmap_node); - /* Remove from lflows. */ - hmap_remove(lflows, &lflow->hmap_node); - hash = ovn_logical_flow_hash_datapath(&lflow->od->sb->header_.uuid, - hash); - /* Add back. */ - hmap_insert(lflows, &lflow->hmap_node, hash); - - /* Sync to SB. */ - const struct sbrec_logical_flow *sbflow; - /* Note: uuid_random acquires a global mutex. If we parallelize the sync to - * SB this may become a bottleneck. */ - lflow->sb_uuid = uuid_random(); - sbflow = sbrec_logical_flow_insert_persist_uuid(ovnsb_txn, - &lflow->sb_uuid); - const char *pipeline = ovn_stage_get_pipeline_name(lflow->stage); - uint8_t table = ovn_stage_get_table(lflow->stage); - sbrec_logical_flow_set_logical_datapath(sbflow, lflow->od->sb); - sbrec_logical_flow_set_logical_dp_group(sbflow, NULL); - sbrec_logical_flow_set_pipeline(sbflow, pipeline); - sbrec_logical_flow_set_table_id(sbflow, table); - sbrec_logical_flow_set_priority(sbflow, lflow->priority); - sbrec_logical_flow_set_match(sbflow, lflow->match); - sbrec_logical_flow_set_actions(sbflow, lflow->actions); - if (lflow->io_port) { - struct smap tags = SMAP_INITIALIZER(&tags); - smap_add(&tags, "in_out_port", lflow->io_port); - sbrec_logical_flow_set_tags(sbflow, &tags); - smap_destroy(&tags); - } - sbrec_logical_flow_set_controller_meter(sbflow, lflow->ctrl_meter); - /* Trim the source locator lflow->where, which looks something like - * "ovn/northd/northd.c:1234", down to just the part following the - * last slash, e.g. "northd.c:1234". */ - const char *slash = strrchr(lflow->where, '/'); -#if _WIN32 - const char *backslash = strrchr(lflow->where, '\\'); - if (!slash || backslash > slash) { - slash = backslash; - } -#endif - const char *where = slash ? slash + 1 : lflow->where; - - struct smap ids = SMAP_INITIALIZER(&ids); - smap_add(&ids, "stage-name", ovn_stage_to_str(lflow->stage)); - smap_add(&ids, "source", where); - if (lflow->stage_hint) { - smap_add(&ids, "stage-hint", lflow->stage_hint); - } - sbrec_logical_flow_set_external_ids(sbflow, &ids); - smap_destroy(&ids); -} - -static bool -delete_lflow_for_lsp(struct ovn_port *op, bool is_update, - const struct sbrec_logical_flow_table *sb_lflow_table, - struct hmap *lflows) -{ - struct lflow_ref_node *lfrn; - const char *operation = is_update ? "updated" : "deleted"; - LIST_FOR_EACH_SAFE (lfrn, lflow_list_node, &op->lflows) { - VLOG_DBG("Deleting SB lflow "UUID_FMT" for %s port %s", - UUID_ARGS(&lfrn->lflow->sb_uuid), operation, op->key); - - const struct sbrec_logical_flow *sblflow = - sbrec_logical_flow_table_get_for_uuid(sb_lflow_table, - &lfrn->lflow->sb_uuid); - if (sblflow) { - sbrec_logical_flow_delete(sblflow); - } else { - static struct vlog_rate_limit rl = - VLOG_RATE_LIMIT_INIT(1, 1); - VLOG_WARN_RL(&rl, "SB lflow "UUID_FMT" not found when handling " - "%s port %s. Recompute.", - UUID_ARGS(&lfrn->lflow->sb_uuid), operation, op->key); - return false; - } - - ovn_lflow_destroy(lflows, lfrn->lflow); +void +reset_lflow_refs_for_northd_resources(struct lflow_input *lflow_input) +{ + struct ovn_port *op; + HMAP_FOR_EACH (op, key_node, lflow_input->ls_ports) { + lflow_ref_reset(op->lflow_ref); + lflow_ref_reset(op->stateful_lflow_ref); } - return true; } bool lflow_handle_northd_port_changes(struct ovsdb_idl_txn *ovnsb_txn, struct tracked_ovn_ports *trk_lsps, struct lflow_input *lflow_input, - struct hmap *lflows) + struct lflow_table *lflows) { struct hmapx_node *hmapx_node; struct ovn_port *op; @@ -16991,12 +16023,13 @@ lflow_handle_northd_port_changes(struct ovsdb_idl_txn *ovnsb_txn, op = hmapx_node->data; /* Make sure 'op' is an lsp and not lrp. */ ovs_assert(op->nbsp); - - if (!delete_lflow_for_lsp(op, false, - lflow_input->sbrec_logical_flow_table, - lflows)) { - return false; - } + lflow_ref_clear_and_sync_lflows(op->lflow_ref, lflows, + ovnsb_txn, + lflow_input->ls_datapaths, + lflow_input->lr_datapaths, + lflow_input->ovn_internal_version_changed, + lflow_input->sbrec_logical_flow_table, + lflow_input->sbrec_logical_dp_group_table); /* No need to update SB multicast groups, thanks to weak * references. */ @@ -17007,12 +16040,8 @@ lflow_handle_northd_port_changes(struct ovsdb_idl_txn *ovnsb_txn, /* Make sure 'op' is an lsp and not lrp. */ ovs_assert(op->nbsp); - /* Delete old lflows. */ - if (!delete_lflow_for_lsp(op, true, - lflow_input->sbrec_logical_flow_table, - lflows)) { - return false; - } + /* Clear old lflows. */ + lflow_ref_clear_lflows(op->lflow_ref); /* Generate new lflows. */ struct ds match = DS_EMPTY_INITIALIZER; @@ -17028,11 +16057,18 @@ lflow_handle_northd_port_changes(struct ovsdb_idl_txn *ovnsb_txn, lr_sful_rec = lr_stateful_table_find_by_index( lflow_input->lr_sful_table, op->peer->od->index); ovs_assert(lr_sful_rec); - + lflow_ref_clear_lflows(op->stateful_lflow_ref); build_lsp_lflows_for_lbnats(op, op->peer, lr_sful_rec, lflow_input->lr_sful_table, lflow_input->lr_ports, - lflows, &match, &actions); + lflows, &match, &actions, + op->stateful_lflow_ref); + lflow_ref_sync_lflows_to_sb(op->stateful_lflow_ref, lflows, ovnsb_txn, + lflow_input->ls_datapaths, + lflow_input->lr_datapaths, + lflow_input->ovn_internal_version_changed, + lflow_input->sbrec_logical_flow_table, + lflow_input->sbrec_logical_dp_group_table); } ds_destroy(&match); @@ -17042,11 +16078,12 @@ lflow_handle_northd_port_changes(struct ovsdb_idl_txn *ovnsb_txn, * groups. */ /* Sync the new flows to SB. */ - struct lflow_ref_node *lfrn; - LIST_FOR_EACH (lfrn, lflow_list_node, &op->lflows) { - sync_lsp_lflows_to_sb(ovnsb_txn, lflow_input, lflows, - lfrn->lflow); - } + lflow_ref_sync_lflows_to_sb(op->lflow_ref, lflows, ovnsb_txn, + lflow_input->ls_datapaths, + lflow_input->lr_datapaths, + lflow_input->ovn_internal_version_changed, + lflow_input->sbrec_logical_flow_table, + lflow_input->sbrec_logical_dp_group_table); } HMAPX_FOR_EACH (hmapx_node, &trk_lsps->created) { @@ -17098,11 +16135,12 @@ lflow_handle_northd_port_changes(struct ovsdb_idl_txn *ovnsb_txn, } /* Sync the newly added flows to SB. */ - struct lflow_ref_node *lfrn; - LIST_FOR_EACH (lfrn, lflow_list_node, &op->lflows) { - sync_lsp_lflows_to_sb(ovnsb_txn, lflow_input, lflows, - lfrn->lflow); - } + lflow_ref_sync_lflows_to_sb(op->lflow_ref, lflows, ovnsb_txn, + lflow_input->ls_datapaths, + lflow_input->lr_datapaths, + lflow_input->ovn_internal_version_changed, + lflow_input->sbrec_logical_flow_table, + lflow_input->sbrec_logical_dp_group_table); } return true; diff --git a/northd/northd.h b/northd/northd.h index 7727b5a8f6..cce465b699 100644 --- a/northd/northd.h +++ b/northd/northd.h @@ -23,6 +23,7 @@ #include "northd/en-port-group.h" #include "northd/ipam.h" #include "openvswitch/hmap.h" +#include "ovs-thread.h" struct northd_input { /* Northbound table references */ @@ -164,13 +165,6 @@ struct northd_data { struct northd_tracked_data trk_data; }; -struct lflow_data { - struct hmap lflows; -}; - -void lflow_data_init(struct lflow_data *); -void lflow_data_destroy(struct lflow_data *); - struct lr_nat_table; struct lflow_input { @@ -182,6 +176,7 @@ struct lflow_input { const struct sbrec_logical_flow_table *sbrec_logical_flow_table; const struct sbrec_multicast_group_table *sbrec_multicast_group_table; const struct sbrec_igmp_group_table *sbrec_igmp_group_table; + const struct sbrec_logical_dp_group_table *sbrec_logical_dp_group_table; /* Indexes */ struct ovsdb_idl_index *sbrec_mcast_group_by_name_dp; @@ -201,6 +196,15 @@ struct lflow_input { bool ovn_internal_version_changed; }; +extern int parallelization_state; +enum { + STATE_NULL, /* parallelization is off */ + STATE_INIT_HASH_SIZES, /* parallelization is on; hashes sizing needed */ + STATE_USE_PARALLELIZATION /* parallelization is on */ +}; + +extern thread_local size_t thread_lflow_counter; + /* * Multicast snooping and querier per datapath configuration. */ @@ -344,6 +348,179 @@ struct ovn_datapath { const struct ovn_datapath *ovn_datapath_find(const struct hmap *datapaths, const struct uuid *uuid); +struct ovn_datapath *ovn_datapath_from_sbrec( + const struct hmap *ls_datapaths, const struct hmap *lr_datapaths, + const struct sbrec_datapath_binding *); + +static inline bool +ovn_datapath_is_stale(const struct ovn_datapath *od) +{ + return !od->nbr && !od->nbs; +}; + +/* Pipeline stages. */ + +/* The two purposes for which ovn-northd uses OVN logical datapaths. */ +enum ovn_datapath_type { + DP_SWITCH, /* OVN logical switch. */ + DP_ROUTER /* OVN logical router. */ +}; + +/* Returns an "enum ovn_stage" built from the arguments. + * + * (It's better to use ovn_stage_build() for type-safety reasons, but inline + * functions can't be used in enums or switch cases.) */ +#define OVN_STAGE_BUILD(DP_TYPE, PIPELINE, TABLE) \ + (((DP_TYPE) << 9) | ((PIPELINE) << 8) | (TABLE)) + +/* A stage within an OVN logical switch or router. + * + * An "enum ovn_stage" indicates whether the stage is part of a logical switch + * or router, whether the stage is part of the ingress or egress pipeline, and + * the table within that pipeline. The first three components are combined to + * form the stage's full name, e.g. S_SWITCH_IN_PORT_SEC_L2, + * S_ROUTER_OUT_DELIVERY. */ +enum ovn_stage { +#define PIPELINE_STAGES \ + /* Logical switch ingress stages. */ \ + PIPELINE_STAGE(SWITCH, IN, CHECK_PORT_SEC, 0, "ls_in_check_port_sec") \ + PIPELINE_STAGE(SWITCH, IN, APPLY_PORT_SEC, 1, "ls_in_apply_port_sec") \ + PIPELINE_STAGE(SWITCH, IN, LOOKUP_FDB , 2, "ls_in_lookup_fdb") \ + PIPELINE_STAGE(SWITCH, IN, PUT_FDB, 3, "ls_in_put_fdb") \ + PIPELINE_STAGE(SWITCH, IN, PRE_ACL, 4, "ls_in_pre_acl") \ + PIPELINE_STAGE(SWITCH, IN, PRE_LB, 5, "ls_in_pre_lb") \ + PIPELINE_STAGE(SWITCH, IN, PRE_STATEFUL, 6, "ls_in_pre_stateful") \ + PIPELINE_STAGE(SWITCH, IN, ACL_HINT, 7, "ls_in_acl_hint") \ + PIPELINE_STAGE(SWITCH, IN, ACL_EVAL, 8, "ls_in_acl_eval") \ + PIPELINE_STAGE(SWITCH, IN, ACL_ACTION, 9, "ls_in_acl_action") \ + PIPELINE_STAGE(SWITCH, IN, QOS_MARK, 10, "ls_in_qos_mark") \ + PIPELINE_STAGE(SWITCH, IN, QOS_METER, 11, "ls_in_qos_meter") \ + PIPELINE_STAGE(SWITCH, IN, LB_AFF_CHECK, 12, "ls_in_lb_aff_check") \ + PIPELINE_STAGE(SWITCH, IN, LB, 13, "ls_in_lb") \ + PIPELINE_STAGE(SWITCH, IN, LB_AFF_LEARN, 14, "ls_in_lb_aff_learn") \ + PIPELINE_STAGE(SWITCH, IN, PRE_HAIRPIN, 15, "ls_in_pre_hairpin") \ + PIPELINE_STAGE(SWITCH, IN, NAT_HAIRPIN, 16, "ls_in_nat_hairpin") \ + PIPELINE_STAGE(SWITCH, IN, HAIRPIN, 17, "ls_in_hairpin") \ + PIPELINE_STAGE(SWITCH, IN, ACL_AFTER_LB_EVAL, 18, \ + "ls_in_acl_after_lb_eval") \ + PIPELINE_STAGE(SWITCH, IN, ACL_AFTER_LB_ACTION, 19, \ + "ls_in_acl_after_lb_action") \ + PIPELINE_STAGE(SWITCH, IN, STATEFUL, 20, "ls_in_stateful") \ + PIPELINE_STAGE(SWITCH, IN, ARP_ND_RSP, 21, "ls_in_arp_rsp") \ + PIPELINE_STAGE(SWITCH, IN, DHCP_OPTIONS, 22, "ls_in_dhcp_options") \ + PIPELINE_STAGE(SWITCH, IN, DHCP_RESPONSE, 23, "ls_in_dhcp_response") \ + PIPELINE_STAGE(SWITCH, IN, DNS_LOOKUP, 24, "ls_in_dns_lookup") \ + PIPELINE_STAGE(SWITCH, IN, DNS_RESPONSE, 25, "ls_in_dns_response") \ + PIPELINE_STAGE(SWITCH, IN, EXTERNAL_PORT, 26, "ls_in_external_port") \ + PIPELINE_STAGE(SWITCH, IN, L2_LKUP, 27, "ls_in_l2_lkup") \ + PIPELINE_STAGE(SWITCH, IN, L2_UNKNOWN, 28, "ls_in_l2_unknown") \ + \ + /* Logical switch egress stages. */ \ + PIPELINE_STAGE(SWITCH, OUT, PRE_ACL, 0, "ls_out_pre_acl") \ + PIPELINE_STAGE(SWITCH, OUT, PRE_LB, 1, "ls_out_pre_lb") \ + PIPELINE_STAGE(SWITCH, OUT, PRE_STATEFUL, 2, "ls_out_pre_stateful") \ + PIPELINE_STAGE(SWITCH, OUT, ACL_HINT, 3, "ls_out_acl_hint") \ + PIPELINE_STAGE(SWITCH, OUT, ACL_EVAL, 4, "ls_out_acl_eval") \ + PIPELINE_STAGE(SWITCH, OUT, ACL_ACTION, 5, "ls_out_acl_action") \ + PIPELINE_STAGE(SWITCH, OUT, QOS_MARK, 6, "ls_out_qos_mark") \ + PIPELINE_STAGE(SWITCH, OUT, QOS_METER, 7, "ls_out_qos_meter") \ + PIPELINE_STAGE(SWITCH, OUT, STATEFUL, 8, "ls_out_stateful") \ + PIPELINE_STAGE(SWITCH, OUT, CHECK_PORT_SEC, 9, "ls_out_check_port_sec") \ + PIPELINE_STAGE(SWITCH, OUT, APPLY_PORT_SEC, 10, "ls_out_apply_port_sec") \ + \ + /* Logical router ingress stages. */ \ + PIPELINE_STAGE(ROUTER, IN, ADMISSION, 0, "lr_in_admission") \ + PIPELINE_STAGE(ROUTER, IN, LOOKUP_NEIGHBOR, 1, "lr_in_lookup_neighbor") \ + PIPELINE_STAGE(ROUTER, IN, LEARN_NEIGHBOR, 2, "lr_in_learn_neighbor") \ + PIPELINE_STAGE(ROUTER, IN, IP_INPUT, 3, "lr_in_ip_input") \ + PIPELINE_STAGE(ROUTER, IN, UNSNAT, 4, "lr_in_unsnat") \ + PIPELINE_STAGE(ROUTER, IN, DEFRAG, 5, "lr_in_defrag") \ + PIPELINE_STAGE(ROUTER, IN, LB_AFF_CHECK, 6, "lr_in_lb_aff_check") \ + PIPELINE_STAGE(ROUTER, IN, DNAT, 7, "lr_in_dnat") \ + PIPELINE_STAGE(ROUTER, IN, LB_AFF_LEARN, 8, "lr_in_lb_aff_learn") \ + PIPELINE_STAGE(ROUTER, IN, ECMP_STATEFUL, 9, "lr_in_ecmp_stateful") \ + PIPELINE_STAGE(ROUTER, IN, ND_RA_OPTIONS, 10, "lr_in_nd_ra_options") \ + PIPELINE_STAGE(ROUTER, IN, ND_RA_RESPONSE, 11, "lr_in_nd_ra_response") \ + PIPELINE_STAGE(ROUTER, IN, IP_ROUTING_PRE, 12, "lr_in_ip_routing_pre") \ + PIPELINE_STAGE(ROUTER, IN, IP_ROUTING, 13, "lr_in_ip_routing") \ + PIPELINE_STAGE(ROUTER, IN, IP_ROUTING_ECMP, 14, "lr_in_ip_routing_ecmp") \ + PIPELINE_STAGE(ROUTER, IN, POLICY, 15, "lr_in_policy") \ + PIPELINE_STAGE(ROUTER, IN, POLICY_ECMP, 16, "lr_in_policy_ecmp") \ + PIPELINE_STAGE(ROUTER, IN, ARP_RESOLVE, 17, "lr_in_arp_resolve") \ + PIPELINE_STAGE(ROUTER, IN, CHK_PKT_LEN, 18, "lr_in_chk_pkt_len") \ + PIPELINE_STAGE(ROUTER, IN, LARGER_PKTS, 19, "lr_in_larger_pkts") \ + PIPELINE_STAGE(ROUTER, IN, GW_REDIRECT, 20, "lr_in_gw_redirect") \ + PIPELINE_STAGE(ROUTER, IN, ARP_REQUEST, 21, "lr_in_arp_request") \ + \ + /* Logical router egress stages. */ \ + PIPELINE_STAGE(ROUTER, OUT, CHECK_DNAT_LOCAL, 0, \ + "lr_out_chk_dnat_local") \ + PIPELINE_STAGE(ROUTER, OUT, UNDNAT, 1, "lr_out_undnat") \ + PIPELINE_STAGE(ROUTER, OUT, POST_UNDNAT, 2, "lr_out_post_undnat") \ + PIPELINE_STAGE(ROUTER, OUT, SNAT, 3, "lr_out_snat") \ + PIPELINE_STAGE(ROUTER, OUT, POST_SNAT, 4, "lr_out_post_snat") \ + PIPELINE_STAGE(ROUTER, OUT, EGR_LOOP, 5, "lr_out_egr_loop") \ + PIPELINE_STAGE(ROUTER, OUT, DELIVERY, 6, "lr_out_delivery") + +#define PIPELINE_STAGE(DP_TYPE, PIPELINE, STAGE, TABLE, NAME) \ + S_##DP_TYPE##_##PIPELINE##_##STAGE \ + = OVN_STAGE_BUILD(DP_##DP_TYPE, P_##PIPELINE, TABLE), + PIPELINE_STAGES +#undef PIPELINE_STAGE +}; + +enum ovn_datapath_type ovn_stage_to_datapath_type(enum ovn_stage stage); + + +/* Returns 'od''s datapath type. */ +static inline enum ovn_datapath_type +ovn_datapath_get_type(const struct ovn_datapath *od) +{ + return od->nbs ? DP_SWITCH : DP_ROUTER; +} + +/* Returns an "enum ovn_stage" built from the arguments. */ +static inline enum ovn_stage +ovn_stage_build(enum ovn_datapath_type dp_type, enum ovn_pipeline pipeline, + uint8_t table) +{ + return OVN_STAGE_BUILD(dp_type, pipeline, table); +} + +/* Returns the pipeline to which 'stage' belongs. */ +static inline enum ovn_pipeline +ovn_stage_get_pipeline(enum ovn_stage stage) +{ + return (stage >> 8) & 1; +} + +/* Returns the pipeline name to which 'stage' belongs. */ +static inline const char * +ovn_stage_get_pipeline_name(enum ovn_stage stage) +{ + return ovn_stage_get_pipeline(stage) == P_IN ? "ingress" : "egress"; +} + +/* Returns the table to which 'stage' belongs. */ +static inline uint8_t +ovn_stage_get_table(enum ovn_stage stage) +{ + return stage & 0xff; +} + +/* Returns a string name for 'stage'. */ +static inline const char * +ovn_stage_to_str(enum ovn_stage stage) +{ + switch (stage) { +#define PIPELINE_STAGE(DP_TYPE, PIPELINE, STAGE, TABLE, NAME) \ + case S_##DP_TYPE##_##PIPELINE##_##STAGE: return NAME; + PIPELINE_STAGES +#undef PIPELINE_STAGE + default: return ""; + } +} + /* A logical switch port or logical router port. * * In steady state, an ovn_port points to a northbound Logical_Switch_Port @@ -434,8 +611,7 @@ struct ovn_port { /* Temporarily used for traversing a list (or hmap) of ports. */ bool visited; - /* List of struct lflow_ref_node that points to the lflows generated by - * this ovn_port. + /* Reference of lflows generated for this ovn_port. * * This data is initialized and destroyed by the en_northd node, but * populated and used only by the en_lflow node. Ideally this data should @@ -453,8 +629,16 @@ struct ovn_port { * Adding the list here is more straightforward. The drawback is that we * need to keep in mind that this data belongs to en_lflow node, so never * access it from any other nodes. + * + * 'lflow_ref' is used to reference generic logical flows generated for + * this ovn_port. + * + * 'stateful_lflow_ref' is used for logical switch ports of type + * 'patch/router' to referenece logical flows generated fo this ovn_port + * from the 'lr_stateful' record of the peer port's datapath. */ - struct ovs_list lflows; + struct lflow_ref *lflow_ref; + struct lflow_ref *stateful_lflow_ref; }; void ovnnb_db_run(struct northd_input *input_data, @@ -477,13 +661,17 @@ void northd_destroy(struct northd_data *data); void northd_init(struct northd_data *data); void northd_indices_create(struct northd_data *data, struct ovsdb_idl *ovnsb_idl); + +struct lflow_table; void build_lflows(struct ovsdb_idl_txn *ovnsb_txn, struct lflow_input *input_data, - struct hmap *lflows); + struct lflow_table *); +void reset_lflow_refs_for_northd_resources(struct lflow_input *); + bool lflow_handle_northd_port_changes(struct ovsdb_idl_txn *ovnsb_txn, struct tracked_ovn_ports *, struct lflow_input *, - struct hmap *lflows); + struct lflow_table *lflows); bool northd_handle_sb_port_binding_changes( const struct sbrec_port_binding_table *, struct hmap *ls_ports, struct hmap *lr_ports); diff --git a/northd/ovn-northd.c b/northd/ovn-northd.c index 084d675644..e0e60f3559 100644 --- a/northd/ovn-northd.c +++ b/northd/ovn-northd.c @@ -848,6 +848,10 @@ main(int argc, char *argv[]) ovsdb_idl_omit_alert(ovnsb_idl_loop.idl, &sbrec_port_group_columns[i]); } + for (size_t i = 0; i < SBREC_LOGICAL_DP_GROUP_N_COLUMNS; i++) { + ovsdb_idl_omit_alert(ovnsb_idl_loop.idl, + &sbrec_logical_dp_group_columns[i]); + } unixctl_command_register("sb-connection-status", "", 0, 0, ovn_conn_show, ovnsb_idl_loop.idl); From patchwork Tue Nov 28 02:36:44 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Numan Siddique X-Patchwork-Id: 1869082 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::133; helo=smtp2.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp2.osuosl.org (smtp2.osuosl.org [IPv6:2605:bc80:3010::133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4SfRTN62Tnz1yRy for ; Tue, 28 Nov 2023 13:38:16 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id ECC6640993; Tue, 28 Nov 2023 02:38:14 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org ECC6640993 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id gP_nRC-FEWpd; Tue, 28 Nov 2023 02:38:11 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp2.osuosl.org (Postfix) with ESMTPS id 3482C409C4; Tue, 28 Nov 2023 02:38:10 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 3482C409C4 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 066DFC0DD6; Tue, 28 Nov 2023 02:38:08 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 05A6EC0039 for ; Tue, 28 Nov 2023 02:38:07 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 5505460F93 for ; Tue, 28 Nov 2023 02:37:04 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 5505460F93 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id G1jzTLJUhlTS for ; Tue, 28 Nov 2023 02:37:01 +0000 (UTC) Received: from relay7-d.mail.gandi.net (relay7-d.mail.gandi.net [IPv6:2001:4b98:dc4:8::227]) by smtp3.osuosl.org (Postfix) with ESMTPS id 1E37B60FD5 for ; Tue, 28 Nov 2023 02:36:59 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 1E37B60FD5 Received: by mail.gandi.net (Postfix) with ESMTPSA id 1D97820005; Tue, 28 Nov 2023 02:36:55 +0000 (UTC) From: numans@ovn.org To: dev@openvswitch.org Date: Mon, 27 Nov 2023 21:36:44 -0500 Message-ID: <20231128023644.569861-1-numans@ovn.org> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20231128023410.569539-1-numans@ovn.org> References: <20231128023410.569539-1-numans@ovn.org> MIME-Version: 1.0 X-GND-Sasl: numans@ovn.org Subject: [ovs-dev] [PATCH ovn v3 09/16] northd: Use lflow_ref when adding all logical flows. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Numan Siddique ovn_lflow_add* macros now expect 'lflow_ref' to be passed and it can be NULL too. Signed-off-by: Numan Siddique --- northd/lflow-mgr.h | 38 +- northd/northd.c | 1689 +++++++++++++++++++++++++++----------------- 2 files changed, 1039 insertions(+), 688 deletions(-) diff --git a/northd/lflow-mgr.h b/northd/lflow-mgr.h index 7809c939e6..02b74aa131 100644 --- a/northd/lflow-mgr.h +++ b/northd/lflow-mgr.h @@ -89,41 +89,27 @@ void lflow_table_add_lflow_default_drop(struct lflow_table *, /* Adds a row with the specified contents to the Logical_Flow table. */ #define ovn_lflow_add_with_hint__(LFLOW_TABLE, OD, STAGE, PRIORITY, MATCH, \ ACTIONS, IN_OUT_PORT, CTRL_METER, \ - STAGE_HINT) \ - lflow_table_add_lflow(LFLOW_TABLE, OD, NULL, 0, STAGE, PRIORITY, MATCH, \ - ACTIONS, IN_OUT_PORT, CTRL_METER, STAGE_HINT, \ - OVS_SOURCE_LOCATOR, NULL) - -#define ovn_lflow_add_with_lflow_ref_hint__(LFLOW_TABLE, OD, STAGE, PRIORITY, \ - MATCH, ACTIONS, IN_OUT_PORT, \ - CTRL_METER, STAGE_HINT, LFLOW_REF)\ + STAGE_HINT, LFLOW_REF) \ lflow_table_add_lflow(LFLOW_TABLE, OD, NULL, 0, STAGE, PRIORITY, MATCH, \ ACTIONS, IN_OUT_PORT, CTRL_METER, STAGE_HINT, \ OVS_SOURCE_LOCATOR, LFLOW_REF) #define ovn_lflow_add_with_hint(LFLOW_TABLE, OD, STAGE, PRIORITY, MATCH, \ - ACTIONS, STAGE_HINT) \ - lflow_table_add_lflow(LFLOW_TABLE, OD, NULL, 0, STAGE, PRIORITY, MATCH, \ - ACTIONS, NULL, NULL, STAGE_HINT, \ - OVS_SOURCE_LOCATOR, NULL) - -#define ovn_lflow_add_with_lflow_ref_hint(LFLOW_TABLE, OD, STAGE, PRIORITY, \ - MATCH, ACTIONS, STAGE_HINT, \ - LFLOW_REF) \ + ACTIONS, STAGE_HINT, LFLOW_REF) \ lflow_table_add_lflow(LFLOW_TABLE, OD, NULL, 0, STAGE, PRIORITY, MATCH, \ ACTIONS, NULL, NULL, STAGE_HINT, \ OVS_SOURCE_LOCATOR, LFLOW_REF) #define ovn_lflow_add_with_dp_group(LFLOW_TABLE, DP_BITMAP, DP_BITMAP_LEN, \ STAGE, PRIORITY, MATCH, ACTIONS, \ - STAGE_HINT) \ + STAGE_HINT, LFLOW_REF) \ lflow_table_add_lflow(LFLOW_TABLE, NULL, DP_BITMAP, DP_BITMAP_LEN, STAGE, \ PRIORITY, MATCH, ACTIONS, NULL, NULL, STAGE_HINT, \ - OVS_SOURCE_LOCATOR, NULL) + OVS_SOURCE_LOCATOR, LFLOW_REF) -#define ovn_lflow_add_default_drop(LFLOW_TABLE, OD, STAGE) \ +#define ovn_lflow_add_default_drop(LFLOW_TABLE, OD, STAGE, LFLOW_REF) \ lflow_table_add_lflow_default_drop(LFLOW_TABLE, OD, STAGE, \ - OVS_SOURCE_LOCATOR, NULL) + OVS_SOURCE_LOCATOR, LFLOW_REF) /* This macro is similar to ovn_lflow_add_with_hint, except that it requires @@ -143,20 +129,16 @@ void lflow_table_add_lflow_default_drop(struct lflow_table *, ACTIONS, IN_OUT_PORT, NULL, STAGE_HINT, \ OVS_SOURCE_LOCATOR, LFLOW_REF) -#define ovn_lflow_add(LFLOW_TABLE, OD, STAGE, PRIORITY, MATCH, ACTIONS) \ - lflow_table_add_lflow(LFLOW_TABLE, OD, NULL, 0, STAGE, PRIORITY, MATCH, \ - ACTIONS, NULL, NULL, NULL, OVS_SOURCE_LOCATOR, NULL) - -#define ovn_lflow_add_with_lflow_ref(LFLOW_TABLE, OD, STAGE, PRIORITY, MATCH, \ - ACTIONS, LFLOW_REF) \ +#define ovn_lflow_add(LFLOW_TABLE, OD, STAGE, PRIORITY, MATCH, ACTIONS, \ + LFLOW_REF) \ lflow_table_add_lflow(LFLOW_TABLE, OD, NULL, 0, STAGE, PRIORITY, MATCH, \ ACTIONS, NULL, NULL, NULL, OVS_SOURCE_LOCATOR, \ LFLOW_REF) #define ovn_lflow_metered(LFLOW_TABLE, OD, STAGE, PRIORITY, MATCH, ACTIONS, \ - CTRL_METER) \ + CTRL_METER, LFLOW_REF) \ ovn_lflow_add_with_hint__(LFLOW_TABLE, OD, STAGE, PRIORITY, MATCH, \ - ACTIONS, NULL, CTRL_METER, NULL) + ACTIONS, NULL, CTRL_METER, NULL, LFLOW_REF) struct sbrec_logical_dp_group; diff --git a/northd/northd.c b/northd/northd.c index 13d0db57e2..7c5fe60e9a 100644 --- a/northd/northd.c +++ b/northd/northd.c @@ -6063,13 +6063,16 @@ build_lswitch_learn_fdb_op( static void build_lswitch_learn_fdb_od( - struct ovn_datapath *od, struct lflow_table *lflows) + struct ovn_datapath *od, struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { ovs_assert(od->nbs); - ovn_lflow_add(lflows, od, S_SWITCH_IN_LOOKUP_FDB, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_SWITCH_IN_PUT_FDB, 0, "1", "next;"); + ovn_lflow_add(lflows, od, S_SWITCH_IN_LOOKUP_FDB, 0, "1", "next;", + lflow_ref); + ovn_lflow_add(lflows, od, S_SWITCH_IN_PUT_FDB, 0, "1", "next;", + lflow_ref); ovn_lflow_add(lflows, od, S_SWITCH_IN_L2_LKUP, 0, "1", - "outport = get_fdb(eth.dst); next;"); + "outport = get_fdb(eth.dst); next;", lflow_ref); } /* Egress tables 8: Egress port security - IP (priority 0) @@ -6077,25 +6080,30 @@ build_lswitch_learn_fdb_od( * (priority 100). */ static void build_lswitch_output_port_sec_od(struct ovn_datapath *od, - struct lflow_table *lflows) + struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { ovs_assert(od->nbs); ovn_lflow_add(lflows, od, S_SWITCH_OUT_CHECK_PORT_SEC, 100, - "eth.mcast", REGBIT_PORT_SEC_DROP" = 0; next;"); + "eth.mcast", REGBIT_PORT_SEC_DROP" = 0; next;", + lflow_ref); ovn_lflow_add(lflows, od, S_SWITCH_OUT_CHECK_PORT_SEC, 0, "1", - REGBIT_PORT_SEC_DROP" = check_out_port_sec(); next;"); + REGBIT_PORT_SEC_DROP" = check_out_port_sec(); next;", + lflow_ref); ovn_lflow_add(lflows, od, S_SWITCH_OUT_APPLY_PORT_SEC, 50, - REGBIT_PORT_SEC_DROP" == 1", debug_drop_action()); + REGBIT_PORT_SEC_DROP" == 1", debug_drop_action(), + lflow_ref); ovn_lflow_add(lflows, od, S_SWITCH_OUT_APPLY_PORT_SEC, 0, - "1", "output;"); + "1", "output;", lflow_ref); } static void skip_port_from_conntrack(const struct ovn_datapath *od, struct ovn_port *op, bool has_stateful_acl, enum ovn_stage in_stage, enum ovn_stage out_stage, uint16_t priority, - struct lflow_table *lflows) + struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { /* Can't use ct() for router ports. Consider the following configuration: * lp1(10.0.0.2) on hostA--ls1--lr0--ls2--lp2(10.0.1.2) on hostB, For a @@ -6117,10 +6125,10 @@ skip_port_from_conntrack(const struct ovn_datapath *od, struct ovn_port *op, ovn_lflow_add_with_lport_and_hint(lflows, od, in_stage, priority, ingress_match, ingress_action, - op->key, &op->nbsp->header_, NULL); + op->key, &op->nbsp->header_, lflow_ref); ovn_lflow_add_with_lport_and_hint(lflows, od, out_stage, priority, egress_match, egress_action, - op->key, &op->nbsp->header_, NULL); + op->key, &op->nbsp->header_, lflow_ref); free(ingress_match); free(egress_match); @@ -6129,7 +6137,8 @@ skip_port_from_conntrack(const struct ovn_datapath *od, struct ovn_port *op, static void build_stateless_filter(const struct ovn_datapath *od, const struct nbrec_acl *acl, - struct lflow_table *lflows) + struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { const char *action = REGBIT_ACL_STATELESS" = 1; next;"; if (!strcmp(acl->direction, "from-lport")) { @@ -6137,25 +6146,28 @@ build_stateless_filter(const struct ovn_datapath *od, acl->priority + OVN_ACL_PRI_OFFSET, acl->match, action, - &acl->header_); + &acl->header_, + lflow_ref); } else { ovn_lflow_add_with_hint(lflows, od, S_SWITCH_OUT_PRE_ACL, acl->priority + OVN_ACL_PRI_OFFSET, acl->match, action, - &acl->header_); + &acl->header_, + lflow_ref); } } static void build_stateless_filters(const struct ovn_datapath *od, const struct ls_port_group_table *ls_port_groups, - struct lflow_table *lflows) + struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { for (size_t i = 0; i < od->nbs->n_acls; i++) { const struct nbrec_acl *acl = od->nbs->acls[i]; if (!strcmp(acl->action, "allow-stateless")) { - build_stateless_filter(od, acl, lflows); + build_stateless_filter(od, acl, lflows, lflow_ref); } } @@ -6171,31 +6183,37 @@ build_stateless_filters(const struct ovn_datapath *od, const struct nbrec_acl *acl = ls_pg_rec->nb_pg->acls[i]; if (!strcmp(acl->action, "allow-stateless")) { - build_stateless_filter(od, acl, lflows); + build_stateless_filter(od, acl, lflows, lflow_ref); } } } } static void -build_pre_acls(struct ovn_datapath *od, struct lflow_table *lflows) +build_pre_acls(struct ovn_datapath *od, struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { /* Ingress and Egress Pre-ACL Table (Priority 0): Packets are * allowed by default. */ - ovn_lflow_add(lflows, od, S_SWITCH_IN_PRE_ACL, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_SWITCH_OUT_PRE_ACL, 0, "1", "next;"); + ovn_lflow_add(lflows, od, S_SWITCH_IN_PRE_ACL, 0, "1", "next;", + lflow_ref); + ovn_lflow_add(lflows, od, S_SWITCH_OUT_PRE_ACL, 0, "1", "next;", + lflow_ref); ovn_lflow_add(lflows, od, S_SWITCH_IN_PRE_ACL, 110, - "eth.dst == $svc_monitor_mac", "next;"); + "eth.dst == $svc_monitor_mac", "next;", + lflow_ref); ovn_lflow_add(lflows, od, S_SWITCH_OUT_PRE_ACL, 110, - "eth.src == $svc_monitor_mac", "next;"); + "eth.src == $svc_monitor_mac", "next;", + lflow_ref); } static void build_ls_stateful_rec_pre_acls(const struct ls_stateful_record *ls_sful_rec, const struct ls_port_group_table *ls_port_groups, - struct lflow_table *lflows) + struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { const struct ovn_datapath *od = ls_sful_rec->od; @@ -6206,17 +6224,17 @@ build_ls_stateful_rec_pre_acls(const struct ls_stateful_record *ls_sful_rec, for (size_t i = 0; i < od->n_router_ports; i++) { skip_port_from_conntrack(od, od->router_ports[i], true, S_SWITCH_IN_PRE_ACL, S_SWITCH_OUT_PRE_ACL, - 110, lflows); + 110, lflows, lflow_ref); } for (size_t i = 0; i < od->n_localnet_ports; i++) { skip_port_from_conntrack(od, od->localnet_ports[i], true, S_SWITCH_IN_PRE_ACL, S_SWITCH_OUT_PRE_ACL, - 110, lflows); + 110, lflows, lflow_ref); } /* stateless filters always take precedence over stateful ACLs. */ - build_stateless_filters(od, ls_port_groups, lflows); + build_stateless_filters(od, ls_port_groups, lflows, lflow_ref); /* Ingress and Egress Pre-ACL Table (Priority 110). * @@ -6224,16 +6242,18 @@ build_ls_stateful_rec_pre_acls(const struct ls_stateful_record *ls_sful_rec, * unreachable packets. */ ovn_lflow_add(lflows, od, S_SWITCH_IN_PRE_ACL, 110, "nd || nd_rs || nd_ra || mldv1 || mldv2 || " - "(udp && udp.src == 546 && udp.dst == 547)", "next;"); + "(udp && udp.src == 546 && udp.dst == 547)", "next;", + lflow_ref); ovn_lflow_add(lflows, od, S_SWITCH_OUT_PRE_ACL, 110, "nd || nd_rs || nd_ra || mldv1 || mldv2 || " - "(udp && udp.src == 546 && udp.dst == 547)", "next;"); + "(udp && udp.src == 546 && udp.dst == 547)", "next;", + lflow_ref); /* Do not send multicast packets to conntrack. */ ovn_lflow_add(lflows, od, S_SWITCH_IN_PRE_ACL, 110, "eth.mcast", - "next;"); + "next;", lflow_ref); ovn_lflow_add(lflows, od, S_SWITCH_OUT_PRE_ACL, 110, "eth.mcast", - "next;"); + "next;", lflow_ref); /* Ingress and Egress Pre-ACL Table (Priority 100). * @@ -6244,13 +6264,15 @@ build_ls_stateful_rec_pre_acls(const struct ls_stateful_record *ls_sful_rec, * 'REGBIT_CONNTRACK_DEFRAG' is set to let the pre-stateful table send * it to conntrack for tracking and defragmentation. */ ovn_lflow_add(lflows, od, S_SWITCH_IN_PRE_ACL, 100, "ip", - REGBIT_CONNTRACK_DEFRAG" = 1; next;"); + REGBIT_CONNTRACK_DEFRAG" = 1; next;", + lflow_ref); ovn_lflow_add(lflows, od, S_SWITCH_OUT_PRE_ACL, 100, "ip", - REGBIT_CONNTRACK_DEFRAG" = 1; next;"); + REGBIT_CONNTRACK_DEFRAG" = 1; next;", + lflow_ref); } else if (ls_sful_rec->has_lb_vip) { /* We'll build stateless filters if there are LB rules so that * the stateless flows are not tracked in pre-lb. */ - build_stateless_filters(od, ls_port_groups, lflows); + build_stateless_filters(od, ls_port_groups, lflows, lflow_ref); } } @@ -6314,7 +6336,8 @@ build_empty_lb_event_flow(struct ovn_lb_vip *lb_vip, static void build_interconn_mcast_snoop_flows(struct ovn_datapath *od, const struct shash *meter_groups, - struct lflow_table *lflows) + struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { struct mcast_switch_info *mcast_sw_info = &od->mcast_info.sw; if (!mcast_sw_info->enabled @@ -6333,7 +6356,8 @@ build_interconn_mcast_snoop_flows(struct ovn_datapath *od, ovn_lflow_metered(lflows, od, S_SWITCH_OUT_PRE_LB, 120, match, "clone { igmp; }; next;", copp_meter_get(COPP_IGMP, od->nbs->copp, - meter_groups)); + meter_groups), + lflow_ref); free(match); /* Punt MLD traffic to controller. */ @@ -6341,50 +6365,54 @@ build_interconn_mcast_snoop_flows(struct ovn_datapath *od, ovn_lflow_metered(lflows, od, S_SWITCH_OUT_PRE_LB, 120, match, "clone { igmp; }; next;", copp_meter_get(COPP_IGMP, od->nbs->copp, - meter_groups)); + meter_groups), + lflow_ref); free(match); } } static void build_pre_lb(struct ovn_datapath *od, const struct shash *meter_groups, - struct lflow_table *lflows) + struct lflow_table *lflows, struct lflow_ref *lflow_ref) { /* Handle IGMP/MLD packets crossing AZs. */ - build_interconn_mcast_snoop_flows(od, meter_groups, lflows); + build_interconn_mcast_snoop_flows(od, meter_groups, lflows, lflow_ref); /* Do not send multicast packets to conntrack */ - ovn_lflow_add(lflows, od, S_SWITCH_IN_PRE_LB, 110, "eth.mcast", "next;"); - ovn_lflow_add(lflows, od, S_SWITCH_OUT_PRE_LB, 110, "eth.mcast", "next;"); + ovn_lflow_add(lflows, od, S_SWITCH_IN_PRE_LB, 110, "eth.mcast", "next;", + lflow_ref); + ovn_lflow_add(lflows, od, S_SWITCH_OUT_PRE_LB, 110, "eth.mcast", "next;", + lflow_ref); /* Do not send ND packets to conntrack */ ovn_lflow_add(lflows, od, S_SWITCH_IN_PRE_LB, 110, "nd || nd_rs || nd_ra || mldv1 || mldv2", - "next;"); + "next;", lflow_ref); ovn_lflow_add(lflows, od, S_SWITCH_OUT_PRE_LB, 110, "nd || nd_rs || nd_ra || mldv1 || mldv2", - "next;"); + "next;", lflow_ref); /* Do not send service monitor packets to conntrack. */ ovn_lflow_add(lflows, od, S_SWITCH_IN_PRE_LB, 110, - "eth.dst == $svc_monitor_mac", "next;"); + "eth.dst == $svc_monitor_mac", "next;", lflow_ref); ovn_lflow_add(lflows, od, S_SWITCH_OUT_PRE_LB, 110, - "eth.src == $svc_monitor_mac", "next;"); + "eth.src == $svc_monitor_mac", "next;", lflow_ref); /* Allow all packets to go to next tables by default. */ - ovn_lflow_add(lflows, od, S_SWITCH_IN_PRE_LB, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_SWITCH_OUT_PRE_LB, 0, "1", "next;"); + ovn_lflow_add(lflows, od, S_SWITCH_IN_PRE_LB, 0, "1", "next;", lflow_ref); + ovn_lflow_add(lflows, od, S_SWITCH_OUT_PRE_LB, 0, "1", "next;", lflow_ref); /* Do not send statless flows via conntrack */ ovn_lflow_add(lflows, od, S_SWITCH_IN_PRE_LB, 110, - REGBIT_ACL_STATELESS" == 1", "next;"); + REGBIT_ACL_STATELESS" == 1", "next;", lflow_ref); ovn_lflow_add(lflows, od, S_SWITCH_OUT_PRE_LB, 110, - REGBIT_ACL_STATELESS" == 1", "next;"); + REGBIT_ACL_STATELESS" == 1", "next;", lflow_ref); } static void build_ls_stateful_rec_pre_lb(const struct ls_stateful_record *ls_stateful_rec, - struct lflow_table *lflows) + struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { const struct ovn_datapath *od = ls_stateful_rec->od; @@ -6392,7 +6420,7 @@ build_ls_stateful_rec_pre_lb(const struct ls_stateful_record *ls_stateful_rec, skip_port_from_conntrack(od, od->router_ports[i], ls_stateful_rec->has_stateful_acl, S_SWITCH_IN_PRE_LB, S_SWITCH_OUT_PRE_LB, - 110, lflows); + 110, lflows, lflow_ref); } /* Localnet ports have no need for going through conntrack, unless @@ -6405,7 +6433,7 @@ build_ls_stateful_rec_pre_lb(const struct ls_stateful_record *ls_stateful_rec, skip_port_from_conntrack(od, od->localnet_ports[i], ls_stateful_rec->has_stateful_acl, S_SWITCH_IN_PRE_LB, S_SWITCH_OUT_PRE_LB, - 110, lflows); + 110, lflows, lflow_ref); } } @@ -6441,21 +6469,26 @@ build_ls_stateful_rec_pre_lb(const struct ls_stateful_record *ls_stateful_rec, */ if (ls_stateful_rec->has_lb_vip) { ovn_lflow_add(lflows, od, S_SWITCH_IN_PRE_LB, - 100, "ip", REGBIT_CONNTRACK_NAT" = 1; next;"); + 100, "ip", REGBIT_CONNTRACK_NAT" = 1; next;", + lflow_ref); ovn_lflow_add(lflows, od, S_SWITCH_OUT_PRE_LB, - 100, "ip", REGBIT_CONNTRACK_NAT" = 1; next;"); + 100, "ip", REGBIT_CONNTRACK_NAT" = 1; next;", + lflow_ref); } } static void build_pre_stateful(struct ovn_datapath *od, const struct chassis_features *features, - struct lflow_table *lflows) + struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { /* Ingress and Egress pre-stateful Table (Priority 0): Packets are * allowed by default. */ - ovn_lflow_add(lflows, od, S_SWITCH_IN_PRE_STATEFUL, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_SWITCH_OUT_PRE_STATEFUL, 0, "1", "next;"); + ovn_lflow_add(lflows, od, S_SWITCH_IN_PRE_STATEFUL, 0, "1", "next;", + lflow_ref); + ovn_lflow_add(lflows, od, S_SWITCH_OUT_PRE_STATEFUL, 0, "1", "next;", + lflow_ref); /* Note: priority-120 flows are added in build_lb_rules_pre_stateful(). */ @@ -6464,25 +6497,30 @@ build_pre_stateful(struct ovn_datapath *od, : "ct_lb;"; ovn_lflow_add(lflows, od, S_SWITCH_IN_PRE_STATEFUL, 110, - REGBIT_CONNTRACK_NAT" == 1", ct_lb_action); + REGBIT_CONNTRACK_NAT" == 1", ct_lb_action, + lflow_ref); ovn_lflow_add(lflows, od, S_SWITCH_OUT_PRE_STATEFUL, 110, - REGBIT_CONNTRACK_NAT" == 1", ct_lb_action); + REGBIT_CONNTRACK_NAT" == 1", ct_lb_action, + lflow_ref); /* If REGBIT_CONNTRACK_DEFRAG is set as 1, then the packets should be * sent to conntrack for tracking and defragmentation. */ ovn_lflow_add(lflows, od, S_SWITCH_IN_PRE_STATEFUL, 100, - REGBIT_CONNTRACK_DEFRAG" == 1", "ct_next;"); + REGBIT_CONNTRACK_DEFRAG" == 1", "ct_next;", + lflow_ref); ovn_lflow_add(lflows, od, S_SWITCH_OUT_PRE_STATEFUL, 100, - REGBIT_CONNTRACK_DEFRAG" == 1", "ct_next;"); + REGBIT_CONNTRACK_DEFRAG" == 1", "ct_next;", + lflow_ref); } static void build_acl_hints(const struct ls_stateful_record *ls_sful_rec, const struct chassis_features *features, - struct lflow_table *lflows) + struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { const struct ovn_datapath *od = ls_sful_rec->od; @@ -6510,9 +6548,10 @@ build_acl_hints(const struct ls_stateful_record *ls_sful_rec, /* In any case, advance to the next stage. */ if (!ls_sful_rec->has_acls && !ls_sful_rec->has_lb_vip) { - ovn_lflow_add(lflows, od, stage, UINT16_MAX, "1", "next;"); + ovn_lflow_add(lflows, od, stage, UINT16_MAX, "1", "next;", + lflow_ref); } else { - ovn_lflow_add(lflows, od, stage, 0, "1", "next;"); + ovn_lflow_add(lflows, od, stage, 0, "1", "next;", lflow_ref); } if (!ls_sful_rec->has_stateful_acl && !ls_sful_rec->has_lb_vip) { @@ -6526,7 +6565,7 @@ build_acl_hints(const struct ls_stateful_record *ls_sful_rec, ovn_lflow_add(lflows, od, stage, 7, "ct.new && !ct.est", REGBIT_ACL_HINT_ALLOW_NEW " = 1; " REGBIT_ACL_HINT_DROP " = 1; " - "next;"); + "next;", lflow_ref); /* Already established connections in the "request" direction that * are already marked as "blocked" may hit either: @@ -6542,13 +6581,13 @@ build_acl_hints(const struct ls_stateful_record *ls_sful_rec, ovn_lflow_add(lflows, od, stage, 6, match, REGBIT_ACL_HINT_ALLOW_NEW " = 1; " REGBIT_ACL_HINT_DROP " = 1; " - "next;"); + "next;", lflow_ref); /* Not tracked traffic can either be allowed or dropped. */ ovn_lflow_add(lflows, od, stage, 5, "!ct.trk", REGBIT_ACL_HINT_ALLOW " = 1; " REGBIT_ACL_HINT_DROP " = 1; " - "next;"); + "next;", lflow_ref); /* Already established connections in the "request" direction may hit * either: @@ -6564,20 +6603,20 @@ build_acl_hints(const struct ls_stateful_record *ls_sful_rec, ovn_lflow_add(lflows, od, stage, 4, match, REGBIT_ACL_HINT_ALLOW " = 1; " REGBIT_ACL_HINT_BLOCK " = 1; " - "next;"); + "next;", lflow_ref); /* Not established or established and already blocked connections may * hit drop ACLs. */ ovn_lflow_add(lflows, od, stage, 3, "!ct.est", REGBIT_ACL_HINT_DROP " = 1; " - "next;"); + "next;", lflow_ref); match = features->ct_no_masked_label ? "ct.est && ct_mark.blocked == 1" : "ct.est && ct_label.blocked == 1"; ovn_lflow_add(lflows, od, stage, 2, match, REGBIT_ACL_HINT_DROP " = 1; " - "next;"); + "next;", lflow_ref); /* Established connections that were previously allowed might hit * drop ACLs in which case the connection must be committed with @@ -6588,7 +6627,7 @@ build_acl_hints(const struct ls_stateful_record *ls_sful_rec, : "ct.est && ct_label.blocked == 0"; ovn_lflow_add(lflows, od, stage, 1, match, REGBIT_ACL_HINT_BLOCK " = 1; " - "next;"); + "next;", lflow_ref); } } @@ -6654,7 +6693,8 @@ static void consider_acl(struct lflow_table *lflows, const struct ovn_datapath *od, const struct nbrec_acl *acl, bool has_stateful, bool ct_masked_mark, const struct shash *meter_groups, - uint64_t max_acl_tier, struct ds *match, struct ds *actions) + uint64_t max_acl_tier, struct ds *match, struct ds *actions, + struct lflow_ref *lflow_ref) { const char *ct_blocked_match = ct_masked_mark ? "ct_mark.blocked" @@ -6703,7 +6743,7 @@ consider_acl(struct lflow_table *lflows, const struct ovn_datapath *od, ds_put_format(match, "(%s)", acl->match); ovn_lflow_add_with_hint(lflows, od, stage, priority, ds_cstr(match), ds_cstr(actions), - &acl->header_); + &acl->header_, lflow_ref); return; } @@ -6740,7 +6780,7 @@ consider_acl(struct lflow_table *lflows, const struct ovn_datapath *od, ds_put_cstr(actions, "next;"); ovn_lflow_add_with_hint(lflows, od, stage, priority, ds_cstr(match), ds_cstr(actions), - &acl->header_); + &acl->header_, lflow_ref); /* Match on traffic in the request direction for an established * connection tracking entry that has not been marked for @@ -6762,7 +6802,7 @@ consider_acl(struct lflow_table *lflows, const struct ovn_datapath *od, ds_put_cstr(actions, "next;"); ovn_lflow_add_with_hint(lflows, od, stage, priority, ds_cstr(match), ds_cstr(actions), - &acl->header_); + &acl->header_, lflow_ref); } else if (!strcmp(acl->action, "drop") || !strcmp(acl->action, "reject")) { /* The implementation of "drop" differs if stateful ACLs are in @@ -6779,7 +6819,7 @@ consider_acl(struct lflow_table *lflows, const struct ovn_datapath *od, ds_put_cstr(actions, "next;"); ovn_lflow_add_with_hint(lflows, od, stage, priority, ds_cstr(match), ds_cstr(actions), - &acl->header_); + &acl->header_, lflow_ref); /* For an existing connection without ct_mark.blocked set, we've * encountered a policy change. ACLs previously allowed * this connection and we committed the connection tracking @@ -6800,7 +6840,7 @@ consider_acl(struct lflow_table *lflows, const struct ovn_datapath *od, ct_blocked_match); ovn_lflow_add_with_hint(lflows, od, stage, priority, ds_cstr(match), ds_cstr(actions), - &acl->header_); + &acl->header_, lflow_ref); } } @@ -6883,7 +6923,8 @@ build_acl_action_lflows(const struct ls_stateful_record *ls_stateful_rec, const char *default_acl_action, const struct shash *meter_groups, struct ds *match, - struct ds *actions) + struct ds *actions, + struct lflow_ref *lflow_ref) { const struct ovn_datapath *od = ls_stateful_rec->od; @@ -6906,18 +6947,20 @@ build_acl_action_lflows(const struct ls_stateful_record *ls_stateful_rec, for (size_t i = 0; i < ARRAY_SIZE(stages); i++) { enum ovn_stage stage = stages[i]; if (!ls_stateful_rec->has_acls) { - ovn_lflow_add(lflows, od, stage, 0, "1", "next;"); + ovn_lflow_add(lflows, od, stage, 0, "1", "next;", lflow_ref); continue; } ds_truncate(actions, verdict_len); ds_put_cstr(actions, "next;"); ovn_lflow_add(lflows, od, stage, 1000, - REGBIT_ACL_VERDICT_ALLOW " == 1", ds_cstr(actions)); + REGBIT_ACL_VERDICT_ALLOW " == 1", ds_cstr(actions), + lflow_ref); ds_truncate(actions, verdict_len); ds_put_cstr(actions, debug_implicit_drop_action()); ovn_lflow_add(lflows, od, stage, 1000, REGBIT_ACL_VERDICT_DROP " == 1", - ds_cstr(actions)); + ds_cstr(actions), + lflow_ref); bool ingress = ovn_stage_get_pipeline(stage) == P_IN; ds_truncate(actions, verdict_len); @@ -6933,11 +6976,11 @@ build_acl_action_lflows(const struct ls_stateful_record *ls_stateful_rec, ovn_lflow_metered(lflows, od, stage, 1000, REGBIT_ACL_VERDICT_REJECT " == 1", ds_cstr(actions), copp_meter_get(COPP_REJECT, od->nbs->copp, - meter_groups)); + meter_groups), lflow_ref); ds_truncate(actions, verdict_len); ds_put_cstr(actions, default_acl_action); - ovn_lflow_add(lflows, od, stage, 0, "1", ds_cstr(actions)); + ovn_lflow_add(lflows, od, stage, 0, "1", ds_cstr(actions), lflow_ref); struct ds tier_actions = DS_EMPTY_INITIALIZER; for (size_t j = 0; j < ls_stateful_rec->max_acl_tier; j++) { @@ -6949,7 +6992,7 @@ build_acl_action_lflows(const struct ls_stateful_record *ls_stateful_rec, j + 1, ingress ? "ingress" : "egress", ovn_stage_get_table(stage) - 1); ovn_lflow_add(lflows, od, stage, 500, ds_cstr(match), - ds_cstr(&tier_actions)); + ds_cstr(&tier_actions), lflow_ref); } ds_destroy(&tier_actions); } @@ -6961,7 +7004,8 @@ build_acl_log_related_flows(const struct ovn_datapath *od, const struct nbrec_acl *acl, bool has_stateful, bool ct_masked_mark, const struct shash *meter_groups, - struct ds *match, struct ds *actions) + struct ds *match, struct ds *actions, + struct lflow_ref *lflow_ref) { /* Related and reply traffic are universally allowed by priority * 65532 flows created in build_acls(). If logging is enabled on @@ -7015,7 +7059,7 @@ build_acl_log_related_flows(const struct ovn_datapath *od, ovn_lflow_add_with_hint(lflows, od, log_related_stage, UINT16_MAX - 2, ds_cstr(match), ds_cstr(actions), - &acl->header_); + &acl->header_, lflow_ref); ds_clear(match); ds_put_format(match, "!ct.est && ct.rel && !ct.new%s && " @@ -7026,7 +7070,7 @@ build_acl_log_related_flows(const struct ovn_datapath *od, ovn_lflow_add_with_hint(lflows, od, log_related_stage, UINT16_MAX - 2, ds_cstr(match), ds_cstr(actions), - &acl->header_); + &acl->header_, lflow_ref); } static void @@ -7034,7 +7078,8 @@ build_acls(const struct ls_stateful_record *ls_stateful_rec, const struct chassis_features *features, struct lflow_table *lflows, const struct ls_port_group_table *ls_port_groups, - const struct shash *meter_groups) + const struct shash *meter_groups, + struct lflow_ref *lflow_ref) { const struct ovn_datapath *od = ls_stateful_rec->od; @@ -7059,20 +7104,24 @@ build_acls(const struct ls_stateful_record *ls_stateful_rec, if (!ls_stateful_rec->has_acls) { if (!ls_stateful_rec->has_lb_vip) { ovn_lflow_add(lflows, od, S_SWITCH_IN_ACL_EVAL, UINT16_MAX, "1", - "next;"); + "next;", lflow_ref); ovn_lflow_add(lflows, od, S_SWITCH_OUT_ACL_EVAL, UINT16_MAX, "1", - "next;"); + "next;", lflow_ref); } else { - ovn_lflow_add(lflows, od, S_SWITCH_IN_ACL_EVAL, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_SWITCH_OUT_ACL_EVAL, 0, "1", "next;"); + ovn_lflow_add(lflows, od, S_SWITCH_IN_ACL_EVAL, 0, "1", "next;", + lflow_ref); + ovn_lflow_add(lflows, od, S_SWITCH_OUT_ACL_EVAL, 0, "1", "next;", + lflow_ref); } ovn_lflow_add(lflows, od, S_SWITCH_IN_ACL_AFTER_LB_EVAL, 0, "1", - "next;"); + "next;", lflow_ref); } else { - ovn_lflow_add(lflows, od, S_SWITCH_IN_ACL_EVAL, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_SWITCH_OUT_ACL_EVAL, 0, "1", "next;"); + ovn_lflow_add(lflows, od, S_SWITCH_IN_ACL_EVAL, 0, "1", "next;", + lflow_ref); + ovn_lflow_add(lflows, od, S_SWITCH_OUT_ACL_EVAL, 0, "1", "next;", + lflow_ref); ovn_lflow_add(lflows, od, S_SWITCH_IN_ACL_AFTER_LB_EVAL, 0, "1", - "next;"); + "next;", lflow_ref); } @@ -7104,19 +7153,21 @@ build_acls(const struct ls_stateful_record *ls_stateful_rec, ovn_lflow_add(lflows, od, S_SWITCH_IN_ACL_EVAL, 1, ds_cstr(&match), REGBIT_CONNTRACK_COMMIT" = 1; " - REGBIT_ACL_VERDICT_ALLOW" = 1; next;"); + REGBIT_ACL_VERDICT_ALLOW" = 1; next;", + lflow_ref); ovn_lflow_add(lflows, od, S_SWITCH_OUT_ACL_EVAL, 1, ds_cstr(&match), REGBIT_CONNTRACK_COMMIT" = 1; " - REGBIT_ACL_VERDICT_ALLOW" = 1; next;"); + REGBIT_ACL_VERDICT_ALLOW" = 1; next;", + lflow_ref); const char *next_action = default_acl_drop ? "next;" : REGBIT_CONNTRACK_COMMIT" = 1; next;"; ovn_lflow_add(lflows, od, S_SWITCH_IN_ACL_EVAL, 1, "ip && !ct.est", - next_action); + next_action, lflow_ref); ovn_lflow_add(lflows, od, S_SWITCH_OUT_ACL_EVAL, 1, "ip && !ct.est", - next_action); + next_action, lflow_ref); /* Ingress and Egress ACL Table (Priority 65532). * @@ -7130,9 +7181,11 @@ build_acls(const struct ls_stateful_record *ls_stateful_rec, use_ct_inv_match ? "ct.inv || " : "", ct_blocked_match); ovn_lflow_add(lflows, od, S_SWITCH_IN_ACL_EVAL, UINT16_MAX - 3, - ds_cstr(&match), REGBIT_ACL_VERDICT_DROP " = 1; next;"); + ds_cstr(&match), REGBIT_ACL_VERDICT_DROP " = 1; next;", + lflow_ref); ovn_lflow_add(lflows, od, S_SWITCH_OUT_ACL_EVAL, UINT16_MAX - 3, - ds_cstr(&match), REGBIT_ACL_VERDICT_DROP " = 1; next;"); + ds_cstr(&match), REGBIT_ACL_VERDICT_DROP " = 1; next;", + lflow_ref); /* Ingress and Egress ACL Table (Priority 65535 - 3). * @@ -7152,10 +7205,12 @@ build_acls(const struct ls_stateful_record *ls_stateful_rec, ds_cstr(&match), REGBIT_ACL_HINT_DROP" = 0; " REGBIT_ACL_HINT_BLOCK" = 0; " REGBIT_ACL_HINT_ALLOW_REL" = 1; " - REGBIT_ACL_VERDICT_ALLOW" = 1; next;"); + REGBIT_ACL_VERDICT_ALLOW" = 1; next;", + lflow_ref); ovn_lflow_add(lflows, od, S_SWITCH_OUT_ACL_EVAL, UINT16_MAX - 3, ds_cstr(&match), - REGBIT_ACL_VERDICT_ALLOW " = 1; next;"); + REGBIT_ACL_VERDICT_ALLOW " = 1; next;", + lflow_ref); /* Ingress and Egress ACL Table (Priority 65535). * @@ -7184,15 +7239,16 @@ build_acls(const struct ls_stateful_record *ls_stateful_rec, use_ct_inv_match ? " && !ct.inv" : "", ct_blocked_match); ovn_lflow_add(lflows, od, S_SWITCH_IN_ACL_EVAL, UINT16_MAX - 3, - ds_cstr(&match), ct_in_acl_action); + ds_cstr(&match), ct_in_acl_action, lflow_ref); ovn_lflow_add(lflows, od, S_SWITCH_OUT_ACL_EVAL, UINT16_MAX - 3, - ds_cstr(&match), ct_out_acl_action); + ds_cstr(&match), ct_out_acl_action, lflow_ref); /* Reply and related traffic matched by an "allow-related" ACL * should be allowed in the ls_in_acl_after_lb stage too. */ ovn_lflow_add(lflows, od, S_SWITCH_IN_ACL_AFTER_LB_EVAL, UINT16_MAX - 3, REGBIT_ACL_HINT_ALLOW_REL" == 1", - REGBIT_ACL_VERDICT_ALLOW " = 1; next;"); + REGBIT_ACL_VERDICT_ALLOW " = 1; next;", + lflow_ref); } /* Ingress and Egress ACL Table (Priority 65532). @@ -7204,24 +7260,28 @@ build_acls(const struct ls_stateful_record *ls_stateful_rec, * https://bugzilla.kernel.org/show_bug.cgi?id=11797. */ ovn_lflow_add(lflows, od, S_SWITCH_IN_ACL_EVAL, UINT16_MAX - 3, IPV6_CT_OMIT_MATCH, - REGBIT_ACL_VERDICT_ALLOW " = 1; next;"); + REGBIT_ACL_VERDICT_ALLOW " = 1; next;", + lflow_ref); ovn_lflow_add(lflows, od, S_SWITCH_OUT_ACL_EVAL, UINT16_MAX - 3, IPV6_CT_OMIT_MATCH, - REGBIT_ACL_VERDICT_ALLOW " = 1; next;"); + REGBIT_ACL_VERDICT_ALLOW " = 1; next;", + lflow_ref); ovn_lflow_add(lflows, od, S_SWITCH_IN_ACL_AFTER_LB_EVAL, UINT16_MAX - 3, IPV6_CT_OMIT_MATCH, - REGBIT_ACL_VERDICT_ALLOW " = 1; next;"); + REGBIT_ACL_VERDICT_ALLOW " = 1; next;", + lflow_ref); /* Ingress or Egress ACL Table (Various priorities). */ for (size_t i = 0; i < od->nbs->n_acls; i++) { struct nbrec_acl *acl = od->nbs->acls[i]; build_acl_log_related_flows(od, lflows, acl, has_stateful, features->ct_no_masked_label, - meter_groups, &match, &actions); + meter_groups, &match, &actions, + lflow_ref); consider_acl(lflows, od, acl, has_stateful, features->ct_no_masked_label, meter_groups, ls_stateful_rec->max_acl_tier, - &match, &actions); + &match, &actions, lflow_ref); } const struct ls_port_group *ls_pg = @@ -7234,11 +7294,12 @@ build_acls(const struct ls_stateful_record *ls_stateful_rec, build_acl_log_related_flows(od, lflows, acl, has_stateful, features->ct_no_masked_label, - meter_groups, &match, &actions); + meter_groups, &match, &actions, + lflow_ref); consider_acl(lflows, od, acl, has_stateful, features->ct_no_masked_label, meter_groups, ls_stateful_rec->max_acl_tier, - &match, &actions); + &match, &actions, lflow_ref); } } } @@ -7253,7 +7314,7 @@ build_acls(const struct ls_stateful_record *ls_stateful_rec, : REGBIT_ACL_VERDICT_ALLOW" = 1; next;"; ovn_lflow_add( lflows, od, S_SWITCH_OUT_ACL_EVAL, 34000, "udp.src == 53", - dns_actions); + dns_actions, lflow_ref); } if (ls_stateful_rec->has_acls || ls_stateful_rec->has_lb_vip) { @@ -7261,30 +7322,37 @@ build_acls(const struct ls_stateful_record *ls_stateful_rec, * packets to skip applying ingress ACLs. */ ovn_lflow_add(lflows, od, S_SWITCH_IN_ACL_EVAL, 34000, "eth.dst == $svc_monitor_mac", - REGBIT_ACL_VERDICT_ALLOW" = 1; next;"); + REGBIT_ACL_VERDICT_ALLOW" = 1; next;", + lflow_ref); /* Add a 34000 priority flow to advance the service monitor packets * generated by ovn-controller to skip applying egress ACLs. */ ovn_lflow_add(lflows, od, S_SWITCH_OUT_ACL_EVAL, 34000, "eth.src == $svc_monitor_mac", - REGBIT_ACL_VERDICT_ALLOW" = 1; next;"); + REGBIT_ACL_VERDICT_ALLOW" = 1; next;", + lflow_ref); } build_acl_action_lflows(ls_stateful_rec, lflows, default_acl_action, - meter_groups, &match, &actions); + meter_groups, &match, &actions, lflow_ref); ds_destroy(&match); ds_destroy(&actions); } static void -build_qos(struct ovn_datapath *od, struct lflow_table *lflows) { +build_qos(struct ovn_datapath *od, struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { struct ds action = DS_EMPTY_INITIALIZER; - ovn_lflow_add(lflows, od, S_SWITCH_IN_QOS_MARK, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_SWITCH_OUT_QOS_MARK, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_SWITCH_IN_QOS_METER, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_SWITCH_OUT_QOS_METER, 0, "1", "next;"); + ovn_lflow_add(lflows, od, S_SWITCH_IN_QOS_MARK, 0, "1", "next;", + lflow_ref); + ovn_lflow_add(lflows, od, S_SWITCH_OUT_QOS_MARK, 0, "1", "next;", + lflow_ref); + ovn_lflow_add(lflows, od, S_SWITCH_IN_QOS_METER, 0, "1", "next;", + lflow_ref); + ovn_lflow_add(lflows, od, S_SWITCH_OUT_QOS_METER, 0, "1", "next;", + lflow_ref); for (size_t i = 0; i < od->nbs->n_qos_rules; i++) { struct nbrec_qos *qos = od->nbs->qos_rules[i]; @@ -7301,7 +7369,7 @@ build_qos(struct ovn_datapath *od, struct lflow_table *lflows) { ovn_lflow_add_with_hint(lflows, od, stage, qos->priority, qos->match, ds_cstr(&action), - &qos->header_); + &qos->header_, lflow_ref); } } @@ -7332,7 +7400,7 @@ build_qos(struct ovn_datapath *od, struct lflow_table *lflows) { ovn_lflow_add_with_hint(lflows, od, stage, qos->priority, qos->match, ds_cstr(&action), - &qos->header_); + &qos->header_, lflow_ref); } } ds_destroy(&action); @@ -7343,7 +7411,8 @@ build_lb_rules_pre_stateful(struct lflow_table *lflows, struct ovn_lb_datapaths *lb_dps, bool ct_lb_mark, const struct ovn_datapaths *ls_datapaths, - struct ds *match, struct ds *action) + struct ds *match, struct ds *action, + struct lflow_ref *lflow_ref) { if (!lb_dps->n_nb_ls) { return; @@ -7397,7 +7466,7 @@ build_lb_rules_pre_stateful(struct lflow_table *lflows, ovn_lflow_add_with_dp_group( lflows, lb_dps->nb_ls_map, ods_size(ls_datapaths), S_SWITCH_IN_PRE_STATEFUL, 120, ds_cstr(match), ds_cstr(action), - &lb->nlb->header_); + &lb->nlb->header_, lflow_ref); } } @@ -7445,7 +7514,8 @@ build_lb_affinity_lr_flows(struct lflow_table *lflows, const struct ovn_northd_lb *lb, struct ovn_lb_vip *lb_vip, char *new_lb_match, char *lb_action, const unsigned long *dp_bitmap, - const struct ovn_datapaths *lr_datapaths) + const struct ovn_datapaths *lr_datapaths, + struct lflow_ref *lflow_ref) { if (!lb->affinity_timeout || bitmap_is_all_zeros(dp_bitmap, ods_size(lr_datapaths))) { @@ -7484,7 +7554,8 @@ build_lb_affinity_lr_flows(struct lflow_table *lflows, ovn_lflow_add_with_dp_group( lflows, dp_bitmap, ods_size(lr_datapaths), S_ROUTER_IN_LB_AFF_CHECK, - 100, new_lb_match, ds_cstr(&aff_check_action), &lb->nlb->header_); + 100, new_lb_match, ds_cstr(&aff_check_action), &lb->nlb->header_, + lflow_ref); /* Prepare common part of affinity LB and affinity learn action. */ ds_put_format(&aff_action, "%s = %s; ", reg_vip, lb_vip->vip_str); @@ -7566,12 +7637,14 @@ build_lb_affinity_lr_flows(struct lflow_table *lflows, ovn_lflow_add_with_dp_group( lflows, dp_bitmap, ods_size(lr_datapaths), S_ROUTER_IN_LB_AFF_LEARN, 100, ds_cstr(&aff_match_learn), - ds_cstr(&aff_action_learn), &lb->nlb->header_); + ds_cstr(&aff_action_learn), &lb->nlb->header_, + lflow_ref); /* Use already selected backend within affinity timeslot. */ ovn_lflow_add_with_dp_group( lflows, dp_bitmap, ods_size(lr_datapaths), S_ROUTER_IN_DNAT, 150, - ds_cstr(&aff_match), ds_cstr(&aff_action), &lb->nlb->header_); + ds_cstr(&aff_match), ds_cstr(&aff_action), &lb->nlb->header_, + lflow_ref); ds_truncate(&aff_action, aff_action_len); ds_truncate(&aff_action_learn, aff_action_learn_len); @@ -7631,7 +7704,8 @@ static void build_lb_affinity_ls_flows(struct lflow_table *lflows, struct ovn_lb_datapaths *lb_dps, struct ovn_lb_vip *lb_vip, - const struct ovn_datapaths *ls_datapaths) + const struct ovn_datapaths *ls_datapaths, + struct lflow_ref *lflow_ref) { if (!lb_dps->lb->affinity_timeout || !lb_dps->n_nb_ls) { return; @@ -7659,7 +7733,7 @@ build_lb_affinity_ls_flows(struct lflow_table *lflows, ovn_lflow_add_with_dp_group( lflows, lb_dps->nb_ls_map, ods_size(ls_datapaths), S_SWITCH_IN_LB_AFF_CHECK, 100, ds_cstr(&new_lb_match), aff_check, - &lb_dps->lb->nlb->header_); + &lb_dps->lb->nlb->header_, lflow_ref); ds_destroy(&new_lb_match); struct ds aff_action = DS_EMPTY_INITIALIZER; @@ -7749,13 +7823,14 @@ build_lb_affinity_ls_flows(struct lflow_table *lflows, ovn_lflow_add_with_dp_group( lflows, lb_dps->nb_ls_map, ods_size(ls_datapaths), S_SWITCH_IN_LB_AFF_LEARN, 100, ds_cstr(&aff_match_learn), - ds_cstr(&aff_action_learn), &lb->nlb->header_); + ds_cstr(&aff_action_learn), &lb->nlb->header_, + lflow_ref); /* Use already selected backend within affinity timeslot. */ ovn_lflow_add_with_dp_group( lflows, lb_dps->nb_ls_map, ods_size(ls_datapaths), S_SWITCH_IN_LB, 150, ds_cstr(&aff_match), ds_cstr(&aff_action), - &lb->nlb->header_); + &lb->nlb->header_, lflow_ref); ds_truncate(&aff_action, aff_action_len); ds_truncate(&aff_action_learn, aff_action_learn_len); @@ -7771,20 +7846,26 @@ build_lb_affinity_ls_flows(struct lflow_table *lflows, static void build_lswitch_lb_affinity_default_flows(struct ovn_datapath *od, - struct lflow_table *lflows) + struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { ovs_assert(od->nbs); - ovn_lflow_add(lflows, od, S_SWITCH_IN_LB_AFF_CHECK, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_SWITCH_IN_LB_AFF_LEARN, 0, "1", "next;"); + ovn_lflow_add(lflows, od, S_SWITCH_IN_LB_AFF_CHECK, 0, "1", "next;", + lflow_ref); + ovn_lflow_add(lflows, od, S_SWITCH_IN_LB_AFF_LEARN, 0, "1", "next;", + lflow_ref); } static void build_lrouter_lb_affinity_default_flows(struct ovn_datapath *od, - struct lflow_table *lflows) + struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { ovs_assert(od->nbr); - ovn_lflow_add(lflows, od, S_ROUTER_IN_LB_AFF_CHECK, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_ROUTER_IN_LB_AFF_LEARN, 0, "1", "next;"); + ovn_lflow_add(lflows, od, S_ROUTER_IN_LB_AFF_CHECK, 0, "1", "next;", + lflow_ref); + ovn_lflow_add(lflows, od, S_ROUTER_IN_LB_AFF_LEARN, 0, "1", "next;", + lflow_ref); } static void @@ -7792,7 +7873,8 @@ build_lb_rules(struct lflow_table *lflows, struct ovn_lb_datapaths *lb_dps, const struct ovn_datapaths *ls_datapaths, const struct chassis_features *features, struct ds *match, struct ds *action, const struct shash *meter_groups, - const struct hmap *svc_monitor_map) + const struct hmap *svc_monitor_map, + struct lflow_ref *lflow_ref) { const struct ovn_northd_lb *lb = lb_dps->lb; for (size_t i = 0; i < lb->n_vips; i++) { @@ -7829,7 +7911,8 @@ build_lb_rules(struct lflow_table *lflows, struct ovn_lb_datapaths *lb_dps, priority = 120; } - build_lb_affinity_ls_flows(lflows, lb_dps, lb_vip, ls_datapaths); + build_lb_affinity_ls_flows(lflows, lb_dps, lb_vip, ls_datapaths, + lflow_ref); unsigned long *dp_non_meter = NULL; bool build_non_meter = false; @@ -7852,14 +7935,16 @@ build_lb_rules(struct lflow_table *lflows, struct ovn_lb_datapaths *lb_dps, ovn_lflow_add_with_hint__( lflows, od, S_SWITCH_IN_LB, priority, ds_cstr(match), ds_cstr(action), - NULL, meter, &lb->nlb->header_); + NULL, meter, &lb->nlb->header_, + lflow_ref); } } if (!reject || build_non_meter) { ovn_lflow_add_with_dp_group( lflows, dp_non_meter ? dp_non_meter : lb_dps->nb_ls_map, ods_size(ls_datapaths), S_SWITCH_IN_LB, priority, - ds_cstr(match), ds_cstr(action), &lb->nlb->header_); + ds_cstr(match), ds_cstr(action), &lb->nlb->header_, + lflow_ref); } bitmap_free(dp_non_meter); } @@ -7868,7 +7953,8 @@ build_lb_rules(struct lflow_table *lflows, struct ovn_lb_datapaths *lb_dps, static void build_stateful(struct ovn_datapath *od, const struct chassis_features *features, - struct lflow_table *lflows) + struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { const char *ct_block_action = features->ct_no_masked_label ? "ct_mark.blocked" @@ -7877,9 +7963,11 @@ build_stateful(struct ovn_datapath *od, /* Ingress LB, Ingress and Egress stateful Table (Priority 0): Packets are * allowed by default. */ - ovn_lflow_add(lflows, od, S_SWITCH_IN_LB, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_SWITCH_IN_STATEFUL, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_SWITCH_OUT_STATEFUL, 0, "1", "next;"); + ovn_lflow_add(lflows, od, S_SWITCH_IN_LB, 0, "1", "next;", lflow_ref); + ovn_lflow_add(lflows, od, S_SWITCH_IN_STATEFUL, 0, "1", "next;", + lflow_ref); + ovn_lflow_add(lflows, od, S_SWITCH_OUT_STATEFUL, 0, "1", "next;", + lflow_ref); /* If REGBIT_CONNTRACK_COMMIT is set as 1 and * REGBIT_CONNTRACK_SET_LABEL is set to 1, then the packets should be @@ -7893,11 +7981,13 @@ build_stateful(struct ovn_datapath *od, ovn_lflow_add(lflows, od, S_SWITCH_IN_STATEFUL, 100, REGBIT_CONNTRACK_COMMIT" == 1 && " REGBIT_ACL_LABEL" == 1", - ds_cstr(&actions)); + ds_cstr(&actions), + lflow_ref); ovn_lflow_add(lflows, od, S_SWITCH_OUT_STATEFUL, 100, REGBIT_CONNTRACK_COMMIT" == 1 && " REGBIT_ACL_LABEL" == 1", - ds_cstr(&actions)); + ds_cstr(&actions), + lflow_ref); /* If REGBIT_CONNTRACK_COMMIT is set as 1, then the packets should be * committed to conntrack. We always set ct_mark.blocked to 0 here as @@ -7908,26 +7998,32 @@ build_stateful(struct ovn_datapath *od, ovn_lflow_add(lflows, od, S_SWITCH_IN_STATEFUL, 100, REGBIT_CONNTRACK_COMMIT" == 1 && " REGBIT_ACL_LABEL" == 0", - ds_cstr(&actions)); + ds_cstr(&actions), + lflow_ref); ovn_lflow_add(lflows, od, S_SWITCH_OUT_STATEFUL, 100, REGBIT_CONNTRACK_COMMIT" == 1 && " REGBIT_ACL_LABEL" == 0", - ds_cstr(&actions)); + ds_cstr(&actions), + lflow_ref); ds_destroy(&actions); } static void build_lb_hairpin(const struct ls_stateful_record *ls_stateful_rec, - struct lflow_table *lflows) + struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { const struct ovn_datapath *od = ls_stateful_rec->od; /* Ingress Pre-Hairpin/Nat-Hairpin/Hairpin tabled (Priority 0). * Packets that don't need hairpinning should continue processing. */ - ovn_lflow_add(lflows, od, S_SWITCH_IN_PRE_HAIRPIN, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_SWITCH_IN_NAT_HAIRPIN, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_SWITCH_IN_HAIRPIN, 0, "1", "next;"); + ovn_lflow_add(lflows, od, S_SWITCH_IN_PRE_HAIRPIN, 0, "1", "next;", + lflow_ref); + ovn_lflow_add(lflows, od, S_SWITCH_IN_NAT_HAIRPIN, 0, "1", "next;", + lflow_ref); + ovn_lflow_add(lflows, od, S_SWITCH_IN_HAIRPIN, 0, "1", "next;", + lflow_ref); if (ls_stateful_rec->has_lb_vip) { /* Check if the packet needs to be hairpinned. @@ -7939,7 +8035,8 @@ build_lb_hairpin(const struct ls_stateful_record *ls_stateful_rec, REGBIT_HAIRPIN " = chk_lb_hairpin(); " REGBIT_HAIRPIN_REPLY " = chk_lb_hairpin_reply(); " "next;", - &od->nbs->header_); + &od->nbs->header_, + lflow_ref); /* If packet needs to be hairpinned, snat the src ip with the VIP * for new sessions. */ @@ -7947,7 +8044,8 @@ build_lb_hairpin(const struct ls_stateful_record *ls_stateful_rec, "ip && ct.new && ct.trk" " && "REGBIT_HAIRPIN " == 1", "ct_snat_to_vip; next;", - &od->nbs->header_); + &od->nbs->header_, + lflow_ref); /* If packet needs to be hairpinned, for established sessions there * should already be an SNAT conntrack entry. @@ -7956,13 +8054,15 @@ build_lb_hairpin(const struct ls_stateful_record *ls_stateful_rec, "ip && ct.est && ct.trk" " && "REGBIT_HAIRPIN " == 1", "ct_snat;", - &od->nbs->header_); + &od->nbs->header_, + lflow_ref); /* For the reply of hairpinned traffic, snat the src ip to the VIP. */ ovn_lflow_add_with_hint(lflows, od, S_SWITCH_IN_NAT_HAIRPIN, 90, "ip && "REGBIT_HAIRPIN_REPLY " == 1", "ct_snat;", - &od->nbs->header_); + &od->nbs->header_, + lflow_ref); /* Ingress Hairpin table. * - Priority 1: Packets that were SNAT-ed for hairpinning should be @@ -7972,12 +8072,13 @@ build_lb_hairpin(const struct ls_stateful_record *ls_stateful_rec, lflows, od, S_SWITCH_IN_HAIRPIN, 1, "("REGBIT_HAIRPIN " == 1 || " REGBIT_HAIRPIN_REPLY " == 1)", "eth.dst <-> eth.src; outport = inport; flags.loopback = 1; " - "output;"); + "output;", lflow_ref); } } static void -build_vtep_hairpin(struct ovn_datapath *od, struct lflow_table *lflows) +build_vtep_hairpin(struct ovn_datapath *od, struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { if (!od->has_vtep_lports) { /* There is no need in these flows if datapath has no vtep lports. */ @@ -7990,7 +8091,7 @@ build_vtep_hairpin(struct ovn_datapath *od, struct lflow_table *lflows) char *action = xasprintf("next(pipeline=ingress, table=%d);", ovn_stage_get_table(S_SWITCH_IN_L2_LKUP)); ovn_lflow_add(lflows, od, S_SWITCH_IN_HAIRPIN, 1000, - REGBIT_FROM_RAMP" == 1", action); + REGBIT_FROM_RAMP" == 1", action, lflow_ref); free(action); /* Ingress pre-arp flow for traffic from VTEP (ramp) switch. @@ -8007,7 +8108,7 @@ build_vtep_hairpin(struct ovn_datapath *od, struct lflow_table *lflows) REGBIT_FROM_RAMP" == 1 && is_chassis_resident(%s)", op->cr_port->json_key); ovn_lflow_add(lflows, od, S_SWITCH_IN_HAIRPIN, 2000, - ds_cstr(&match), "next;"); + ds_cstr(&match), "next;", lflow_ref); } } @@ -8018,14 +8119,15 @@ build_vtep_hairpin(struct ovn_datapath *od, struct lflow_table *lflows) */ ovn_lflow_add(lflows, od, S_SWITCH_IN_ARP_ND_RSP, 65535, REGBIT_FROM_RAMP" == 1 && (arp || nd_ns)", - "flags.loopback = 1; next;"); + "flags.loopback = 1; next;", lflow_ref); ds_destroy(&match); } /* Build logical flows for the forwarding groups */ static void -build_fwd_group_lflows(struct ovn_datapath *od, struct lflow_table *lflows) +build_fwd_group_lflows(struct ovn_datapath *od, struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { ovs_assert(od->nbs); if (!od->nbs->n_forwarding_groups) { @@ -8060,7 +8162,7 @@ build_fwd_group_lflows(struct ovn_datapath *od, struct lflow_table *lflows) ovn_lflow_add_with_hint(lflows, od, S_SWITCH_IN_ARP_ND_RSP, 50, ds_cstr(&match), ds_cstr(&actions), - &fwd_group->header_); + &fwd_group->header_, lflow_ref); /* L2 lookup for the forwarding group's virtual MAC */ ds_clear(&match); @@ -8083,7 +8185,7 @@ build_fwd_group_lflows(struct ovn_datapath *od, struct lflow_table *lflows) ds_put_format(&actions, "fwd_group(%s);", ds_cstr(&group_ports)); ovn_lflow_add_with_hint(lflows, od, S_SWITCH_IN_L2_LKUP, 50, ds_cstr(&match), ds_cstr(&actions), - &fwd_group->header_); + &fwd_group->header_, lflow_ref); } ds_destroy(&match); @@ -8231,10 +8333,8 @@ build_lswitch_rport_arp_req_self_orig_flow(struct ovn_port *op, ds_put_format(&match, "eth.src == %s && (arp.op == 1 || rarp.op == 3 || nd_ns)", ds_cstr(ð_src)); - ovn_lflow_add_with_lflow_ref(lflows, od, S_SWITCH_IN_L2_LKUP, priority, - ds_cstr(&match), - "outport = \""MC_FLOOD_L2"\"; output;", - lflow_ref); + ovn_lflow_add(lflows, od, S_SWITCH_IN_L2_LKUP, priority, ds_cstr(&match), + "outport = \""MC_FLOOD_L2"\"; output;", lflow_ref); ds_destroy(ð_src); ds_destroy(&match); @@ -8318,17 +8418,17 @@ build_lswitch_rport_arp_req_flow(const char *ips, ds_put_format(&actions, "clone {outport = %s; output; }; " "outport = \""MC_FLOOD_L2"\"; output;", patch_op->json_key); - ovn_lflow_add_with_lflow_ref_hint(lflows, od, S_SWITCH_IN_L2_LKUP, - priority, ds_cstr(&match), - ds_cstr(&actions), stage_hint, - lflow_ref); + ovn_lflow_add_with_hint(lflows, od, S_SWITCH_IN_L2_LKUP, + priority, ds_cstr(&match), + ds_cstr(&actions), stage_hint, + lflow_ref); } else { ds_put_format(&actions, "outport = %s; output;", patch_op->json_key); - ovn_lflow_add_with_lflow_ref_hint(lflows, od, S_SWITCH_IN_L2_LKUP, - priority, ds_cstr(&match), - ds_cstr(&actions), - stage_hint, - lflow_ref); + ovn_lflow_add_with_hint(lflows, od, S_SWITCH_IN_L2_LKUP, + priority, ds_cstr(&match), + ds_cstr(&actions), + stage_hint, + lflow_ref); } ds_destroy(&match); @@ -8484,7 +8584,8 @@ build_dhcpv4_options_flows(struct ovn_port *op, struct lport_addresses *lsp_addrs, struct ovn_port *inport, bool is_external, const struct shash *meter_groups, - struct lflow_table *lflows) + struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { struct ds match = DS_EMPTY_INITIALIZER; @@ -8515,7 +8616,7 @@ build_dhcpv4_options_flows(struct ovn_port *op, op->json_key); } - ovn_lflow_add_with_lflow_ref_hint__(lflows, op->od, + ovn_lflow_add_with_hint__(lflows, op->od, S_SWITCH_IN_DHCP_OPTIONS, 100, ds_cstr(&match), ds_cstr(&options_action), @@ -8524,7 +8625,7 @@ build_dhcpv4_options_flows(struct ovn_port *op, op->od->nbs->copp, meter_groups), &op->nbsp->dhcpv4_options->header_, - op->lflow_ref); + lflow_ref); ds_clear(&match); /* If REGBIT_DHCP_OPTS_RESULT is set, it means the @@ -8544,7 +8645,7 @@ build_dhcpv4_options_flows(struct ovn_port *op, lflows, op->od, S_SWITCH_IN_DHCP_RESPONSE, 100, ds_cstr(&match), ds_cstr(&response_action), inport->key, &op->nbsp->dhcpv4_options->header_, - op->lflow_ref); + lflow_ref); ds_destroy(&options_action); ds_destroy(&response_action); ds_destroy(&ipv4_addr_match); @@ -8572,7 +8673,7 @@ build_dhcpv4_options_flows(struct ovn_port *op, lflows, op->od, S_SWITCH_OUT_ACL_EVAL, 34000, ds_cstr(&match),dhcp_actions, op->key, &op->nbsp->dhcpv4_options->header_, - op->lflow_ref); + lflow_ref); } break; } @@ -8585,7 +8686,8 @@ build_dhcpv6_options_flows(struct ovn_port *op, struct lport_addresses *lsp_addrs, struct ovn_port *inport, bool is_external, const struct shash *meter_groups, - struct lflow_table *lflows) + struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { struct ds match = DS_EMPTY_INITIALIZER; @@ -8607,7 +8709,7 @@ build_dhcpv6_options_flows(struct ovn_port *op, op->json_key); } - ovn_lflow_add_with_lflow_ref_hint__(lflows, op->od, + ovn_lflow_add_with_hint__(lflows, op->od, S_SWITCH_IN_DHCP_OPTIONS, 100, ds_cstr(&match), ds_cstr(&options_action), @@ -8616,7 +8718,7 @@ build_dhcpv6_options_flows(struct ovn_port *op, op->od->nbs->copp, meter_groups), &op->nbsp->dhcpv6_options->header_, - op->lflow_ref); + lflow_ref); /* If REGBIT_DHCP_OPTS_RESULT is set to 1, it means the * put_dhcpv6_opts action is successful */ @@ -8624,7 +8726,7 @@ build_dhcpv6_options_flows(struct ovn_port *op, ovn_lflow_add_with_lport_and_hint( lflows, op->od, S_SWITCH_IN_DHCP_RESPONSE, 100, ds_cstr(&match), ds_cstr(&response_action), inport->key, - &op->nbsp->dhcpv6_options->header_, op->lflow_ref); + &op->nbsp->dhcpv6_options->header_, lflow_ref); ds_destroy(&options_action); ds_destroy(&response_action); @@ -8657,7 +8759,7 @@ build_dhcpv6_options_flows(struct ovn_port *op, lflows, op->od, S_SWITCH_OUT_ACL_EVAL, 34000, ds_cstr(&match),dhcp6_actions, op->key, &op->nbsp->dhcpv6_options->header_, - op->lflow_ref); + lflow_ref); } break; } @@ -8668,7 +8770,8 @@ build_dhcpv6_options_flows(struct ovn_port *op, static void build_drop_arp_nd_flows_for_unbound_router_ports(struct ovn_port *op, const struct ovn_port *port, - struct lflow_table *lflows) + struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { struct ds match = DS_EMPTY_INITIALIZER; @@ -8688,7 +8791,7 @@ build_drop_arp_nd_flows_for_unbound_router_ports(struct ovn_port *op, ovn_lflow_add_with_lport_and_hint( lflows, op->od, S_SWITCH_IN_EXTERNAL_PORT, 100, ds_cstr(&match), debug_drop_action(), port->key, - &op->nbsp->header_, op->lflow_ref); + &op->nbsp->header_, lflow_ref); } for (size_t l = 0; l < rp->lsp_addrs[k].n_ipv6_addrs; l++) { ds_clear(&match); @@ -8704,7 +8807,7 @@ build_drop_arp_nd_flows_for_unbound_router_ports(struct ovn_port *op, ovn_lflow_add_with_lport_and_hint( lflows, op->od, S_SWITCH_IN_EXTERNAL_PORT, 100, ds_cstr(&match), debug_drop_action(), port->key, - &op->nbsp->header_, op->lflow_ref); + &op->nbsp->header_, lflow_ref); } ds_clear(&match); @@ -8721,7 +8824,7 @@ build_drop_arp_nd_flows_for_unbound_router_ports(struct ovn_port *op, debug_drop_action(), port->key, &op->nbsp->header_, - op->lflow_ref); + lflow_ref); } } } @@ -8736,19 +8839,22 @@ is_vlan_transparent(const struct ovn_datapath *od) static void build_lswitch_lflows_l2_unknown(struct ovn_datapath *od, - struct lflow_table *lflows) + struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { /* Ingress table 25/26: Destination lookup for unknown MACs. */ if (od->has_unknown) { ovn_lflow_add(lflows, od, S_SWITCH_IN_L2_UNKNOWN, 50, "outport == \"none\"", - "outport = \""MC_UNKNOWN "\"; output;"); + "outport = \""MC_UNKNOWN "\"; output;", + lflow_ref); } else { ovn_lflow_add(lflows, od, S_SWITCH_IN_L2_UNKNOWN, 50, - "outport == \"none\"", debug_drop_action()); + "outport == \"none\"", debug_drop_action(), + lflow_ref); } ovn_lflow_add(lflows, od, S_SWITCH_IN_L2_UNKNOWN, 0, "1", - "output;"); + "output;", lflow_ref); } /* Build pre-ACL and ACL tables for both ingress and egress. @@ -8758,42 +8864,49 @@ build_lswitch_lflows_pre_acl_and_acl( struct ovn_datapath *od, const struct chassis_features *features, struct lflow_table *lflows, - const struct shash *meter_groups) + const struct shash *meter_groups, + struct lflow_ref *lflow_ref) { ovs_assert(od->nbs); - build_pre_acls(od, lflows); - build_pre_lb(od, meter_groups, lflows); - build_pre_stateful(od, features, lflows); - build_qos(od, lflows); - build_stateful(od, features, lflows); - build_vtep_hairpin(od, lflows); + build_pre_acls(od, lflows, lflow_ref); + build_pre_lb(od, meter_groups, lflows, lflow_ref); + build_pre_stateful(od, features, lflows, lflow_ref); + build_qos(od, lflows, lflow_ref); + build_stateful(od, features, lflows, lflow_ref); + build_vtep_hairpin(od, lflows, lflow_ref); } /* Logical switch ingress table 0: Admission control framework (priority * 100). */ static void build_lswitch_lflows_admission_control(struct ovn_datapath *od, - struct lflow_table *lflows) + struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { ovs_assert(od->nbs); /* Logical VLANs not supported. */ if (!is_vlan_transparent(od)) { /* Block logical VLANs. */ ovn_lflow_add(lflows, od, S_SWITCH_IN_CHECK_PORT_SEC, 100, - "vlan.present", debug_drop_action()); + "vlan.present", debug_drop_action(), + lflow_ref); } /* Broadcast/multicast source address is invalid. */ ovn_lflow_add(lflows, od, S_SWITCH_IN_CHECK_PORT_SEC, 100, - "eth.src[40]", debug_drop_action()); + "eth.src[40]", debug_drop_action(), + lflow_ref); ovn_lflow_add(lflows, od, S_SWITCH_IN_CHECK_PORT_SEC, 50, "1", - REGBIT_PORT_SEC_DROP" = check_in_port_sec(); next;"); + REGBIT_PORT_SEC_DROP" = check_in_port_sec(); next;", + lflow_ref); ovn_lflow_add(lflows, od, S_SWITCH_IN_APPLY_PORT_SEC, 50, - REGBIT_PORT_SEC_DROP" == 1", debug_drop_action()); + REGBIT_PORT_SEC_DROP" == 1", debug_drop_action(), + lflow_ref); - ovn_lflow_add(lflows, od, S_SWITCH_IN_APPLY_PORT_SEC, 0, "1", "next;"); + ovn_lflow_add(lflows, od, S_SWITCH_IN_APPLY_PORT_SEC, 0, "1", "next;", + lflow_ref); } /* Ingress table 19: ARP/ND responder, skip requests coming from localnet @@ -8950,12 +9063,12 @@ build_lswitch_arp_nd_responder_known_ips(struct ovn_port *op, "output;", op->lsp_addrs[i].ea_s, op->lsp_addrs[i].ea_s, op->lsp_addrs[i].ipv4_addrs[j].addr_s); - ovn_lflow_add_with_lflow_ref_hint(lflows, op->od, - S_SWITCH_IN_ARP_ND_RSP, 50, - ds_cstr(match), - ds_cstr(actions), - &op->nbsp->header_, - op->lflow_ref); + ovn_lflow_add_with_hint(lflows, op->od, + S_SWITCH_IN_ARP_ND_RSP, 50, + ds_cstr(match), + ds_cstr(actions), + &op->nbsp->header_, + op->lflow_ref); /* Do not reply to an ARP request from the port that owns * the address (otherwise a DHCP client that ARPs to check @@ -9005,16 +9118,16 @@ build_lswitch_arp_nd_responder_known_ips(struct ovn_port *op, op->lsp_addrs[i].ipv6_addrs[j].addr_s, op->lsp_addrs[i].ipv6_addrs[j].addr_s, op->lsp_addrs[i].ea_s); - ovn_lflow_add_with_lflow_ref_hint__(lflows, op->od, - S_SWITCH_IN_ARP_ND_RSP, 50, - ds_cstr(match), - ds_cstr(actions), - NULL, - copp_meter_get(COPP_ND_NA, - op->od->nbs->copp, - meter_groups), - &op->nbsp->header_, - op->lflow_ref); + ovn_lflow_add_with_hint__(lflows, op->od, + S_SWITCH_IN_ARP_ND_RSP, 50, + ds_cstr(match), + ds_cstr(actions), + NULL, + copp_meter_get(COPP_ND_NA, + op->od->nbs->copp, + meter_groups), + &op->nbsp->header_, + op->lflow_ref); /* Do not reply to a solicitation from the port that owns * the address (otherwise DAD detection will fail). */ @@ -9070,12 +9183,12 @@ build_lswitch_arp_nd_responder_known_ips(struct ovn_port *op, ea_s, ea_s); - ovn_lflow_add_with_lflow_ref_hint(lflows, op->od, - S_SWITCH_IN_ARP_ND_RSP, - 30, ds_cstr(match), - ds_cstr(actions), - &op->nbsp->header_, - op->lflow_ref); + ovn_lflow_add_with_hint(lflows, op->od, + S_SWITCH_IN_ARP_ND_RSP, + 30, ds_cstr(match), + ds_cstr(actions), + &op->nbsp->header_, + op->lflow_ref); } /* Add IPv6 NDP responses. @@ -9118,16 +9231,16 @@ build_lswitch_arp_nd_responder_known_ips(struct ovn_port *op, lsp_is_router(op->nbsp) ? "nd_na_router" : "nd_na", ea_s, ea_s); - ovn_lflow_add_with_lflow_ref_hint__(lflows, op->od, - S_SWITCH_IN_ARP_ND_RSP, 30, - ds_cstr(match), - ds_cstr(actions), - NULL, - copp_meter_get(COPP_ND_NA, - op->od->nbs->copp, - meter_groups), - &op->nbsp->header_, - op->lflow_ref); + ovn_lflow_add_with_hint__(lflows, op->od, + S_SWITCH_IN_ARP_ND_RSP, 30, + ds_cstr(match), + ds_cstr(actions), + NULL, + copp_meter_get(COPP_ND_NA, + op->od->nbs->copp, + meter_groups), + &op->nbsp->header_, + op->lflow_ref); ds_destroy(&ip6_dst_match); ds_destroy(&nd_target_match); } @@ -9138,10 +9251,12 @@ build_lswitch_arp_nd_responder_known_ips(struct ovn_port *op, * (priority 0)*/ static void build_lswitch_arp_nd_responder_default(struct ovn_datapath *od, - struct lflow_table *lflows) + struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { ovs_assert(od->nbs); - ovn_lflow_add(lflows, od, S_SWITCH_IN_ARP_ND_RSP, 0, "1", "next;"); + ovn_lflow_add(lflows, od, S_SWITCH_IN_ARP_ND_RSP, 0, "1", "next;", + lflow_ref); } /* Ingress table 19: ARP/ND responder for service monitor source ip. @@ -9151,7 +9266,8 @@ build_lswitch_arp_nd_service_monitor(const struct ovn_northd_lb *lb, const struct hmap *ls_ports, struct lflow_table *lflows, struct ds *actions, - struct ds *match) + struct ds *match, + struct lflow_ref *lflow_ref) { for (size_t i = 0; i < lb->n_vips; i++) { struct ovn_northd_lb_vip *lb_vip_nb = &lb->vips_nb[i]; @@ -9215,7 +9331,8 @@ build_lswitch_arp_nd_service_monitor(const struct ovn_northd_lb *lb, op->od, S_SWITCH_IN_ARP_ND_RSP, 110, ds_cstr(match), ds_cstr(actions), - &lb->nlb->header_); + &lb->nlb->header_, + lflow_ref); } } } @@ -9255,19 +9372,19 @@ build_lswitch_dhcp_options_and_response(struct ovn_port *op, build_dhcpv4_options_flows( op, &op->lsp_addrs[i], op->od->localnet_ports[j], is_external, - meter_groups, lflows); + meter_groups, lflows, op->lflow_ref); build_dhcpv6_options_flows( op, &op->lsp_addrs[i], op->od->localnet_ports[j], is_external, - meter_groups, lflows); + meter_groups, lflows, op->lflow_ref); } } else { build_dhcpv4_options_flows(op, &op->lsp_addrs[i], op, is_external, meter_groups, - lflows); + lflows, op->lflow_ref); build_dhcpv6_options_flows(op, &op->lsp_addrs[i], op, is_external, meter_groups, - lflows); + lflows, op->lflow_ref); } } } @@ -9280,14 +9397,20 @@ build_lswitch_dhcp_options_and_response(struct ovn_port *op, * (priority 0). */ static void build_lswitch_dhcp_and_dns_defaults(struct ovn_datapath *od, - struct lflow_table *lflows) + struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { ovs_assert(od->nbs); - ovn_lflow_add(lflows, od, S_SWITCH_IN_DHCP_OPTIONS, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_SWITCH_IN_DHCP_RESPONSE, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_SWITCH_IN_DNS_LOOKUP, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_SWITCH_IN_DNS_RESPONSE, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_SWITCH_IN_EXTERNAL_PORT, 0, "1", "next;"); + ovn_lflow_add(lflows, od, S_SWITCH_IN_DHCP_OPTIONS, 0, "1", "next;", + lflow_ref); + ovn_lflow_add(lflows, od, S_SWITCH_IN_DHCP_RESPONSE, 0, "1", "next;", + lflow_ref); + ovn_lflow_add(lflows, od, S_SWITCH_IN_DNS_LOOKUP, 0, "1", "next;", + lflow_ref); + ovn_lflow_add(lflows, od, S_SWITCH_IN_DNS_RESPONSE, 0, "1", "next;", + lflow_ref); + ovn_lflow_add(lflows, od, S_SWITCH_IN_EXTERNAL_PORT, 0, "1", "next;", + lflow_ref); } /* Logical switch ingress table 22 and 23: DNS lookup and response @@ -9296,7 +9419,8 @@ build_lswitch_dhcp_and_dns_defaults(struct ovn_datapath *od, static void build_lswitch_dns_lookup_and_response(struct ovn_datapath *od, struct lflow_table *lflows, - const struct shash *meter_groups) + const struct shash *meter_groups, + struct lflow_ref *lflow_ref) { ovs_assert(od->nbs); if (!ls_has_dns_records(od->nbs)) { @@ -9306,18 +9430,18 @@ build_lswitch_dns_lookup_and_response(struct ovn_datapath *od, "udp.dst == 53", REGBIT_DNS_LOOKUP_RESULT" = dns_lookup(); next;", copp_meter_get(COPP_DNS, od->nbs->copp, - meter_groups)); + meter_groups), lflow_ref); const char *dns_action = "eth.dst <-> eth.src; ip4.src <-> ip4.dst; " "udp.dst = udp.src; udp.src = 53; outport = inport; " "flags.loopback = 1; output;"; const char *dns_match = "udp.dst == 53 && "REGBIT_DNS_LOOKUP_RESULT; ovn_lflow_add(lflows, od, S_SWITCH_IN_DNS_RESPONSE, 100, - dns_match, dns_action); + dns_match, dns_action, lflow_ref); dns_action = "eth.dst <-> eth.src; ip6.src <-> ip6.dst; " "udp.dst = udp.src; udp.src = 53; outport = inport; " "flags.loopback = 1; output;"; ovn_lflow_add(lflows, od, S_SWITCH_IN_DNS_RESPONSE, 100, - dns_match, dns_action); + dns_match, dns_action, lflow_ref); } /* Table 24: External port. Drop ARP request for router ips from @@ -9334,7 +9458,8 @@ build_lswitch_external_port(struct ovn_port *op, } for (size_t i = 0; i < op->od->n_localnet_ports; i++) { build_drop_arp_nd_flows_for_unbound_router_ports( - op, op->od->localnet_ports[i], lflows); + op, op->od->localnet_ports[i], lflows, + op->lflow_ref); } } @@ -9344,7 +9469,8 @@ static void build_lswitch_destination_lookup_bmcast(struct ovn_datapath *od, struct lflow_table *lflows, struct ds *actions, - const struct shash *meter_groups) + const struct shash *meter_groups, + struct lflow_ref *lflow_ref) { ovs_assert(od->nbs); @@ -9352,7 +9478,7 @@ build_lswitch_destination_lookup_bmcast(struct ovn_datapath *od, "eth.dst == $svc_monitor_mac && (tcp || icmp || icmp6)", "handle_svc_check(inport);", copp_meter_get(COPP_SVC_MONITOR, od->nbs->copp, - meter_groups)); + meter_groups), lflow_ref); struct mcast_switch_info *mcast_sw_info = &od->mcast_info.sw; @@ -9363,27 +9489,31 @@ build_lswitch_destination_lookup_bmcast(struct ovn_datapath *od, ovn_lflow_metered(lflows, od, S_SWITCH_IN_L2_LKUP, 100, "igmp", ds_cstr(actions), copp_meter_get(COPP_IGMP, od->nbs->copp, - meter_groups)); + meter_groups), + lflow_ref); /* Punt MLD traffic to controller. */ ovn_lflow_metered(lflows, od, S_SWITCH_IN_L2_LKUP, 100, "mldv1 || mldv2", ds_cstr(actions), copp_meter_get(COPP_IGMP, od->nbs->copp, - meter_groups)); + meter_groups), + lflow_ref); /* Flood all IP multicast traffic destined to 224.0.0.X to all * ports - RFC 4541, section 2.1.2, item 2. */ ovn_lflow_add(lflows, od, S_SWITCH_IN_L2_LKUP, 85, "ip4.mcast && ip4.dst == 224.0.0.0/24", - "outport = \""MC_FLOOD_L2"\"; output;"); + "outport = \""MC_FLOOD_L2"\"; output;", + lflow_ref); /* Flood all IPv6 multicast traffic destined to reserved * multicast IPs (RFC 4291, 2.7.1). */ ovn_lflow_add(lflows, od, S_SWITCH_IN_L2_LKUP, 85, "ip6.mcast_flood", - "outport = \""MC_FLOOD"\"; output;"); + "outport = \""MC_FLOOD"\"; output;", + lflow_ref); /* Forward uregistered IP multicast to routers with relay enabled * and to any ports configured to flood IP multicast traffic. @@ -9415,7 +9545,7 @@ build_lswitch_destination_lookup_bmcast(struct ovn_datapath *od, ovn_lflow_add(lflows, od, S_SWITCH_IN_L2_LKUP, 80, "ip4.mcast || ip6.mcast", - ds_cstr(actions)); + ds_cstr(actions), lflow_ref); } } @@ -9423,11 +9553,12 @@ build_lswitch_destination_lookup_bmcast(struct ovn_datapath *od, "broadcast-arps-to-all-routers", true)) { ovn_lflow_add(lflows, od, S_SWITCH_IN_L2_LKUP, 72, "eth.mcast && (arp.op == 1 || nd_ns)", - "outport = \""MC_FLOOD_L2"\"; output;"); + "outport = \""MC_FLOOD_L2"\"; output;", + lflow_ref); } ovn_lflow_add(lflows, od, S_SWITCH_IN_L2_LKUP, 70, "eth.mcast", - "outport = \""MC_FLOOD"\"; output;"); + "outport = \""MC_FLOOD"\"; output;", lflow_ref); } @@ -9437,7 +9568,8 @@ static void build_lswitch_ip_mcast_igmp_mld(struct ovn_igmp_group *igmp_group, struct lflow_table *lflows, struct ds *actions, - struct ds *match) + struct ds *match, + struct lflow_ref *lflow_ref) { uint64_t dummy; @@ -9509,7 +9641,7 @@ build_lswitch_ip_mcast_igmp_mld(struct ovn_igmp_group *igmp_group, igmp_group->mcgroup.name); ovn_lflow_add(lflows, igmp_group->datapath, S_SWITCH_IN_L2_LKUP, - 90, ds_cstr(match), ds_cstr(actions)); + 90, ds_cstr(match), ds_cstr(actions), lflow_ref); } } @@ -9549,12 +9681,12 @@ build_lswitch_ip_unicast_lookup(struct ovn_port *op, ds_clear(actions); ds_put_format(actions, action, op->json_key); - ovn_lflow_add_with_lflow_ref_hint(lflows, op->od, - S_SWITCH_IN_L2_LKUP, - 50, ds_cstr(match), - ds_cstr(actions), - &op->nbsp->header_, - op->lflow_ref); + ovn_lflow_add_with_hint(lflows, op->od, + S_SWITCH_IN_L2_LKUP, + 50, ds_cstr(match), + ds_cstr(actions), + &op->nbsp->header_, + op->lflow_ref); } else if (!strcmp(op->nbsp->addresses[i], "unknown")) { continue; } else if (is_dynamic_lsp_address(op->nbsp->addresses[i])) { @@ -9569,12 +9701,12 @@ build_lswitch_ip_unicast_lookup(struct ovn_port *op, ds_clear(actions); ds_put_format(actions, action, op->json_key); - ovn_lflow_add_with_lflow_ref_hint(lflows, op->od, - S_SWITCH_IN_L2_LKUP, - 50, ds_cstr(match), - ds_cstr(actions), - &op->nbsp->header_, - op->lflow_ref); + ovn_lflow_add_with_hint(lflows, op->od, + S_SWITCH_IN_L2_LKUP, + 50, ds_cstr(match), + ds_cstr(actions), + &op->nbsp->header_, + op->lflow_ref); } else if (!strcmp(op->nbsp->addresses[i], "router")) { if (!op->peer || !op->peer->nbrp || !ovs_scan(op->peer->nbrp->mac, @@ -9626,11 +9758,11 @@ build_lswitch_ip_unicast_lookup(struct ovn_port *op, ds_clear(actions); ds_put_format(actions, action, op->json_key); - ovn_lflow_add_with_lflow_ref_hint(lflows, op->od, - S_SWITCH_IN_L2_LKUP, 50, - ds_cstr(match), ds_cstr(actions), - &op->nbsp->header_, - op->lflow_ref); + ovn_lflow_add_with_hint(lflows, op->od, + S_SWITCH_IN_L2_LKUP, 50, + ds_cstr(match), ds_cstr(actions), + &op->nbsp->header_, + op->lflow_ref); } else { static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 1); @@ -9678,7 +9810,7 @@ build_lswitch_ip_unicast_lookup_for_nats(struct ovn_port *op, ds_clear(actions); ds_put_format(actions, action, op->json_key); - ovn_lflow_add_with_lflow_ref_hint(lflows, op->od, + ovn_lflow_add_with_hint(lflows, op->od, S_SWITCH_IN_L2_LKUP, 50, ds_cstr(match), ds_cstr(actions), @@ -9926,7 +10058,8 @@ static void build_routing_policy_flow(struct lflow_table *lflows, struct ovn_datapath *od, const struct hmap *lr_ports, const struct nbrec_logical_router_policy *rule, - const struct ovsdb_idl_row *stage_hint) + const struct ovsdb_idl_row *stage_hint, + struct lflow_ref *lflow_ref) { struct ds match = DS_EMPTY_INITIALIZER; struct ds actions = DS_EMPTY_INITIALIZER; @@ -9982,7 +10115,8 @@ build_routing_policy_flow(struct lflow_table *lflows, struct ovn_datapath *od, ds_put_format(&match, "%s", rule->match); ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_POLICY, rule->priority, - ds_cstr(&match), ds_cstr(&actions), stage_hint); + ds_cstr(&match), ds_cstr(&actions), stage_hint, + lflow_ref); ds_destroy(&match); ds_destroy(&actions); } @@ -9992,7 +10126,8 @@ build_ecmp_routing_policy_flows(struct lflow_table *lflows, struct ovn_datapath *od, const struct hmap *lr_ports, const struct nbrec_logical_router_policy *rule, - uint16_t ecmp_group_id) + uint16_t ecmp_group_id, + struct lflow_ref *lflow_ref) { ovs_assert(rule->n_nexthops > 1); @@ -10064,7 +10199,8 @@ build_ecmp_routing_policy_flows(struct lflow_table *lflows, ecmp_group_id, i + 1); ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_POLICY_ECMP, 100, ds_cstr(&match), - ds_cstr(&actions), &rule->header_); + ds_cstr(&actions), &rule->header_, + lflow_ref); } ds_clear(&actions); @@ -10082,7 +10218,8 @@ build_ecmp_routing_policy_flows(struct lflow_table *lflows, ds_put_cstr(&actions, ");"); ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_POLICY, rule->priority, rule->match, - ds_cstr(&actions), &rule->header_); + ds_cstr(&actions), &rule->header_, + lflow_ref); cleanup: ds_destroy(&match); @@ -10127,7 +10264,8 @@ get_route_table_id(struct simap *route_tables, const char *route_table_name) static void build_route_table_lflow(struct ovn_datapath *od, struct lflow_table *lflows, struct nbrec_logical_router_port *lrp, - struct simap *route_tables) + struct simap *route_tables, + struct lflow_ref *lflow_ref) { struct ds match = DS_EMPTY_INITIALIZER; struct ds actions = DS_EMPTY_INITIALIZER; @@ -10143,7 +10281,7 @@ build_route_table_lflow(struct ovn_datapath *od, struct lflow_table *lflows, REG_ROUTE_TABLE_ID, rtb_id); ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_ROUTING_PRE, 100, - ds_cstr(&match), ds_cstr(&actions)); + ds_cstr(&match), ds_cstr(&actions), lflow_ref); ds_destroy(&match); ds_destroy(&actions); @@ -10542,7 +10680,8 @@ add_ecmp_symmetric_reply_flows(struct lflow_table *lflows, const char *port_ip, struct ovn_port *out_port, const struct parsed_route *route, - struct ds *route_match) + struct ds *route_match, + struct lflow_ref *lflow_ref) { const struct nbrec_logical_router_static_route *st_route = route->route; struct ds base_match = DS_EMPTY_INITIALIZER; @@ -10565,12 +10704,12 @@ add_ecmp_symmetric_reply_flows(struct lflow_table *lflows, free(cidr); ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_DEFRAG, 100, ds_cstr(&base_match), "ct_next;", - &st_route->header_); + &st_route->header_, lflow_ref); /* And packets that go out over an ECMP route need conntrack */ ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_DEFRAG, 100, ds_cstr(route_match), "ct_next;", - &st_route->header_); + &st_route->header_, lflow_ref); /* Save src eth and inport in ct_label for packets that arrive over * an ECMP route. @@ -10587,7 +10726,8 @@ add_ecmp_symmetric_reply_flows(struct lflow_table *lflows, ct_ecmp_reply_port_match, out_port->sb->tunnel_key); ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_ECMP_STATEFUL, 100, ds_cstr(&match), ds_cstr(&actions), - &st_route->header_); + &st_route->header_, + lflow_ref); ds_clear(&match); ds_put_format(&match, "%s && (ct.new && !ct.est) && udp", ds_cstr(&base_match)); @@ -10599,7 +10739,8 @@ add_ecmp_symmetric_reply_flows(struct lflow_table *lflows, ct_ecmp_reply_port_match, out_port->sb->tunnel_key); ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_ECMP_STATEFUL, 100, ds_cstr(&match), ds_cstr(&actions), - &st_route->header_); + &st_route->header_, + lflow_ref); ds_clear(&match); ds_put_format(&match, "%s && (ct.new && !ct.est) && sctp", ds_cstr(&base_match)); @@ -10611,7 +10752,8 @@ add_ecmp_symmetric_reply_flows(struct lflow_table *lflows, ct_ecmp_reply_port_match, out_port->sb->tunnel_key); ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_ECMP_STATEFUL, 100, ds_cstr(&match), ds_cstr(&actions), - &st_route->header_); + &st_route->header_, + lflow_ref); ds_clear(&match); ds_put_format(&match, @@ -10625,7 +10767,8 @@ add_ecmp_symmetric_reply_flows(struct lflow_table *lflows, ct_ecmp_reply_port_match, out_port->sb->tunnel_key); ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_ECMP_STATEFUL, 100, ds_cstr(&match), ds_cstr(&actions), - &st_route->header_); + &st_route->header_, + lflow_ref); ds_clear(&match); ds_put_format(&match, @@ -10639,7 +10782,8 @@ add_ecmp_symmetric_reply_flows(struct lflow_table *lflows, ct_ecmp_reply_port_match, out_port->sb->tunnel_key); ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_ECMP_STATEFUL, 100, ds_cstr(&match), ds_cstr(&actions), - &st_route->header_); + &st_route->header_, + lflow_ref); ds_clear(&match); ds_put_format(&match, "%s && (!ct.rpl && ct.est) && sctp", @@ -10652,7 +10796,8 @@ add_ecmp_symmetric_reply_flows(struct lflow_table *lflows, ct_ecmp_reply_port_match, out_port->sb->tunnel_key); ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_ECMP_STATEFUL, 100, ds_cstr(&match), ds_cstr(&actions), - &st_route->header_); + &st_route->header_, + lflow_ref); /* Bypass ECMP selection if we already have ct_label information * for where to route the packet. @@ -10671,12 +10816,14 @@ add_ecmp_symmetric_reply_flows(struct lflow_table *lflows, port_ip, out_port->json_key); ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_IP_ROUTING, 10300, ds_cstr(&match), ds_cstr(&actions), - &st_route->header_); + &st_route->header_, + lflow_ref); /* Egress reply traffic for symmetric ECMP routes skips router policies. */ ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_POLICY, 65535, ds_cstr(&ecmp_reply), "next;", - &st_route->header_); + &st_route->header_, + lflow_ref); /* Use REG_ECMP_ETH_FULL to pass the eth field from ct_label to eth.dst to * avoid masked access to ct_label. Otherwise it may prevent OVS flow @@ -10692,7 +10839,8 @@ add_ecmp_symmetric_reply_flows(struct lflow_table *lflows, " pop(" REG_ECMP_ETH_FULL "); next;"; ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_ARP_RESOLVE, 200, ds_cstr(&ecmp_reply), - action, &st_route->header_); + action, &st_route->header_, + lflow_ref); ds_destroy(&base_match); ds_destroy(&match); @@ -10703,7 +10851,8 @@ add_ecmp_symmetric_reply_flows(struct lflow_table *lflows, static void build_ecmp_route_flow(struct lflow_table *lflows, struct ovn_datapath *od, bool ct_masked_mark, const struct hmap *lr_ports, - struct ecmp_groups_node *eg) + struct ecmp_groups_node *eg, + struct lflow_ref *lflow_ref) { bool is_ipv4 = IN6_IS_ADDR_V4MAPPED(&eg->prefix); @@ -10736,7 +10885,8 @@ build_ecmp_route_flow(struct lflow_table *lflows, struct ovn_datapath *od, ds_put_cstr(&actions, ");"); ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_ROUTING, priority, - ds_cstr(&route_match), ds_cstr(&actions)); + ds_cstr(&route_match), ds_cstr(&actions), + lflow_ref); /* Add per member flow */ struct ds match = DS_EMPTY_INITIALIZER; @@ -10759,7 +10909,8 @@ build_ecmp_route_flow(struct lflow_table *lflows, struct ovn_datapath *od, out_port->key)) { add_ecmp_symmetric_reply_flows(lflows, od, ct_masked_mark, lrp_addr_s, out_port, - route_, &route_match); + route_, &route_match, + lflow_ref); } ds_clear(&match); ds_put_format(&match, REG_ECMP_GROUP_ID" == %"PRIu16" && " @@ -10779,7 +10930,7 @@ build_ecmp_route_flow(struct lflow_table *lflows, struct ovn_datapath *od, out_port->json_key); ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_IP_ROUTING_ECMP, 100, ds_cstr(&match), ds_cstr(&actions), - &route->header_); + &route->header_, lflow_ref); } sset_destroy(&visited_ports); ds_destroy(&match); @@ -10836,17 +10987,17 @@ add_route(struct lflow_table *lflows, struct ovn_datapath *od, ds_put_format(&actions, "ip.ttl--; %s", ds_cstr(&common_actions)); } - ovn_lflow_add_with_lflow_ref_hint(lflows, od, S_ROUTER_IN_IP_ROUTING, - priority, ds_cstr(&match), - ds_cstr(&actions), stage_hint, - lflow_ref); + ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_IP_ROUTING, + priority, ds_cstr(&match), + ds_cstr(&actions), stage_hint, + lflow_ref); if (op && op->has_bfd) { ds_put_format(&match, " && udp.dst == 3784"); - ovn_lflow_add_with_lflow_ref_hint(lflows, op->od, - S_ROUTER_IN_IP_ROUTING, - priority + 1, ds_cstr(&match), - ds_cstr(&common_actions),\ - stage_hint, lflow_ref); + ovn_lflow_add_with_hint(lflows, op->od, + S_ROUTER_IN_IP_ROUTING, + priority + 1, ds_cstr(&match), + ds_cstr(&common_actions),\ + stage_hint, lflow_ref); } ds_destroy(&match); ds_destroy(&common_actions); @@ -10856,7 +11007,8 @@ add_route(struct lflow_table *lflows, struct ovn_datapath *od, static void build_static_route_flow(struct lflow_table *lflows, struct ovn_datapath *od, const struct hmap *lr_ports, - const struct parsed_route *route_) + const struct parsed_route *route_, + struct lflow_ref *lflow_ref) { const char *lrp_addr_s = NULL; struct ovn_port *out_port = NULL; @@ -10880,7 +11032,7 @@ build_static_route_flow(struct lflow_table *lflows, struct ovn_datapath *od, add_route(lflows, route_->is_discard_route ? od : out_port->od, out_port, lrp_addr_s, prefix_s, route_->plen, route->nexthop, route_->is_src_route, route_->route_table_id, &route->header_, - route_->is_discard_route, ofs, NULL); + route_->is_discard_route, ofs, lflow_ref); free(prefix_s); } @@ -10956,7 +11108,8 @@ lrouter_use_common_zone(const struct ovn_datapath *od) static void build_distr_lrouter_nat_flows_for_lb(struct lrouter_nat_lb_flows_ctx *ctx, enum lrouter_nat_lb_flow_type type, - struct ovn_datapath *od) + struct ovn_datapath *od, + struct lflow_ref *lflow_ref) { struct ovn_port *dgp = od->l3dgw_ports[0]; @@ -10995,7 +11148,8 @@ build_distr_lrouter_nat_flows_for_lb(struct lrouter_nat_lb_flows_ctx *ctx, ovn_lflow_add_with_hint__(ctx->lflows, od, S_ROUTER_IN_DNAT, ctx->prio, ds_cstr(ctx->new_match), ctx->new_action[type], - NULL, meter, &ctx->lb->nlb->header_); + NULL, meter, &ctx->lb->nlb->header_, + lflow_ref); ds_truncate(ctx->new_match, new_match_len); @@ -11014,7 +11168,8 @@ build_distr_lrouter_nat_flows_for_lb(struct lrouter_nat_lb_flows_ctx *ctx, ovn_lflow_add_with_hint(ctx->lflows, od, S_ROUTER_IN_GW_REDIRECT, 200, ds_cstr(ctx->undnat_match), ds_cstr(ctx->gw_redir_action), - &ctx->lb->nlb->header_); + &ctx->lb->nlb->header_, + lflow_ref); ds_truncate(ctx->undnat_match, undnat_match_len); ds_put_format(ctx->undnat_match, ") && (inport == %s || outport == %s)" @@ -11022,7 +11177,8 @@ build_distr_lrouter_nat_flows_for_lb(struct lrouter_nat_lb_flows_ctx *ctx, dgp->cr_port->json_key); ovn_lflow_add_with_hint(ctx->lflows, od, S_ROUTER_OUT_UNDNAT, 120, ds_cstr(ctx->undnat_match), undnat_action, - &ctx->lb->nlb->header_); + &ctx->lb->nlb->header_, + lflow_ref); ds_truncate(ctx->undnat_match, undnat_match_len); } @@ -11030,7 +11186,8 @@ static void build_gw_lrouter_nat_flows_for_lb(struct lrouter_nat_lb_flows_ctx *ctx, enum lrouter_nat_lb_flow_type type, const struct ovn_datapaths *lr_datapaths, - const unsigned long *dp_bitmap) + const unsigned long *dp_bitmap, + struct lflow_ref *lflow_ref) { unsigned long *dp_non_meter = NULL; bool build_non_meter = false; @@ -11056,14 +11213,14 @@ build_gw_lrouter_nat_flows_for_lb(struct lrouter_nat_lb_flows_ctx *ctx, bitmap_set0(dp_non_meter, index); ovn_lflow_add_with_hint__(ctx->lflows, od, S_ROUTER_IN_DNAT, ctx->prio, ds_cstr(ctx->new_match), ctx->new_action[type], - NULL, meter, &ctx->lb->nlb->header_); + NULL, meter, &ctx->lb->nlb->header_, lflow_ref); } } if (!ctx->reject || build_non_meter) { ovn_lflow_add_with_dp_group(ctx->lflows, dp_non_meter ? dp_non_meter : dp_bitmap, ods_size(lr_datapaths), S_ROUTER_IN_DNAT, ctx->prio, ds_cstr(ctx->new_match), - ctx->new_action[type], &ctx->lb->nlb->header_); + ctx->new_action[type], &ctx->lb->nlb->header_, lflow_ref); } bitmap_free(dp_non_meter); } @@ -11078,7 +11235,8 @@ build_lrouter_nat_flows_for_lb(struct ovn_lb_vip *lb_vip, struct ds *match, struct ds *action, const struct shash *meter_groups, const struct chassis_features *features, - const struct hmap *svc_monitor_map) + const struct hmap *svc_monitor_map, + struct lflow_ref *lflow_ref) { const struct ovn_northd_lb *lb = lb_dps->lb; bool ipv4 = lb_vip->address_family == AF_INET; @@ -11196,7 +11354,7 @@ build_lrouter_nat_flows_for_lb(struct ovn_lb_vip *lb_vip, if (!od->n_l3dgw_ports) { bitmap_set1(gw_dp_bitmap[type], index); } else { - build_distr_lrouter_nat_flows_for_lb(&ctx, type, od); + build_distr_lrouter_nat_flows_for_lb(&ctx, type, od, lflow_ref); } if (lb->affinity_timeout) { @@ -11217,16 +11375,17 @@ build_lrouter_nat_flows_for_lb(struct ovn_lb_vip *lb_vip, * S_ROUTER_IN_DNAT stage. */ ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_UNSNAT, 120, ds_cstr(&unsnat_match), "next;", - &lb->nlb->header_); + &lb->nlb->header_, lflow_ref); } } for (size_t type = 0; type < LROUTER_NAT_LB_FLOW_MAX; type++) { build_gw_lrouter_nat_flows_for_lb(&ctx, type, lr_datapaths, - gw_dp_bitmap[type]); + gw_dp_bitmap[type], + lflow_ref); build_lb_affinity_lr_flows(lflows, lb, lb_vip, ds_cstr(match), aff_action[type], aff_dp_bitmap[type], - lr_datapaths); + lr_datapaths, lflow_ref); } ds_destroy(&unsnat_match); @@ -11248,7 +11407,8 @@ build_lswitch_flows_for_lb(struct ovn_lb_datapaths *lb_dps, const struct ovn_datapaths *ls_datapaths, const struct chassis_features *features, const struct hmap *svc_monitor_map, - struct ds *match, struct ds *action) + struct ds *match, struct ds *action, + struct lflow_ref *lflow_ref) { if (!lb_dps->n_nb_ls) { return; @@ -11274,7 +11434,8 @@ build_lswitch_flows_for_lb(struct ovn_lb_datapaths *lb_dps, copp_meter_get(COPP_EVENT_ELB, od->nbs->copp, meter_groups), - &lb->nlb->header_); + &lb->nlb->header_, + lflow_ref); } /* Ignore L4 port information in the key because fragmented packets * may not have L4 information. The pre-stateful table will send @@ -11289,9 +11450,9 @@ build_lswitch_flows_for_lb(struct ovn_lb_datapaths *lb_dps, * connection, so it is okay if we do not hit the above match on * REGBIT_CONNTRACK_COMMIT. */ build_lb_rules_pre_stateful(lflows, lb_dps, features->ct_no_masked_label, - ls_datapaths, match, action); + ls_datapaths, match, action, lflow_ref); build_lb_rules(lflows, lb_dps, ls_datapaths, features, match, action, - meter_groups, svc_monitor_map); + meter_groups, svc_monitor_map, lflow_ref); } /* If there are any load balancing rules, we should send the packet to @@ -11306,7 +11467,8 @@ static void build_lrouter_defrag_flows_for_lb(struct ovn_lb_datapaths *lb_dps, struct lflow_table *lflows, const struct ovn_datapaths *lr_datapaths, - struct ds *match) + struct ds *match, + struct lflow_ref *lflow_ref) { if (!lb_dps->n_nb_lr) { return; @@ -11324,7 +11486,7 @@ build_lrouter_defrag_flows_for_lb(struct ovn_lb_datapaths *lb_dps, ovn_lflow_add_with_dp_group( lflows, lb_dps->nb_lr_map, ods_size(lr_datapaths), S_ROUTER_IN_DEFRAG, prio, ds_cstr(match), "ct_dnat;", - &lb_dps->lb->nlb->header_); + &lb_dps->lb->nlb->header_, lflow_ref); } } @@ -11336,7 +11498,8 @@ build_lrouter_flows_for_lb(struct ovn_lb_datapaths *lb_dps, const struct lr_stateful_table *lr_sful_table, const struct chassis_features *features, const struct hmap *svc_monitor_map, - struct ds *match, struct ds *action) + struct ds *match, struct ds *action, + struct lflow_ref *lflow_ref) { size_t index; @@ -11351,7 +11514,7 @@ build_lrouter_flows_for_lb(struct ovn_lb_datapaths *lb_dps, build_lrouter_nat_flows_for_lb(lb_vip, lb_dps, &lb->vips_nb[i], lr_datapaths, lr_sful_table, lflows, match, action, meter_groups, features, - svc_monitor_map); + svc_monitor_map, lflow_ref); if (!build_empty_lb_event_flow(lb_vip, lb, match, action)) { continue; @@ -11366,7 +11529,7 @@ build_lrouter_flows_for_lb(struct ovn_lb_datapaths *lb_dps, copp_meter_get(COPP_EVENT_ELB, od->nbr->copp, meter_groups), - &lb->nlb->header_); + &lb->nlb->header_, lflow_ref); } } @@ -11375,7 +11538,8 @@ build_lrouter_flows_for_lb(struct ovn_lb_datapaths *lb_dps, struct ovn_datapath *od = lr_datapaths->array[index]; ovn_lflow_add(lflows, od, S_ROUTER_OUT_SNAT, 120, - "flags.skip_snat_for_lb == 1 && ip", "next;"); + "flags.skip_snat_for_lb == 1 && ip", "next;", + lflow_ref); } } } @@ -11490,7 +11654,8 @@ static inline void lrouter_nat_add_ext_ip_match(const struct ovn_datapath *od, struct lflow_table *lflows, struct ds *match, const struct nbrec_nat *nat, - bool is_v6, bool is_src, int cidr_bits) + bool is_v6, bool is_src, int cidr_bits, + struct lflow_ref *lflow_ref) { struct nbrec_address_set *allowed_ext_ips = nat->allowed_ext_ips; struct nbrec_address_set *exempted_ext_ips = nat->exempted_ext_ips; @@ -11541,7 +11706,7 @@ lrouter_nat_add_ext_ip_match(const struct ovn_datapath *od, ovn_lflow_add_with_hint(lflows, od, stage, priority, ds_cstr(&match_exempt), "next;", - &nat->header_); + &nat->header_, lflow_ref); ds_destroy(&match_exempt); } } @@ -11555,7 +11720,8 @@ build_lrouter_arp_flow(const struct ovn_datapath *od, struct ovn_port *op, const char *ip_address, const char *eth_addr, struct ds *extra_match, bool drop, uint16_t priority, const struct ovsdb_idl_row *hint, - struct lflow_table *lflows) + struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { struct ds match = DS_EMPTY_INITIALIZER; struct ds actions = DS_EMPTY_INITIALIZER; @@ -11587,7 +11753,8 @@ build_lrouter_arp_flow(const struct ovn_datapath *od, struct ovn_port *op, } ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_IP_INPUT, priority, - ds_cstr(&match), ds_cstr(&actions), hint); + ds_cstr(&match), ds_cstr(&actions), hint, + lflow_ref); ds_destroy(&match); ds_destroy(&actions); @@ -11606,7 +11773,8 @@ build_lrouter_nd_flow(const struct ovn_datapath *od, struct ovn_port *op, struct ds *extra_match, bool drop, uint16_t priority, const struct ovsdb_idl_row *hint, struct lflow_table *lflows, - const struct shash *meter_groups) + const struct shash *meter_groups, + struct lflow_ref *lflow_ref) { struct ds match = DS_EMPTY_INITIALIZER; struct ds actions = DS_EMPTY_INITIALIZER; @@ -11629,7 +11797,8 @@ build_lrouter_nd_flow(const struct ovn_datapath *od, struct ovn_port *op, if (drop) { ds_put_cstr(&actions, debug_drop_action()); ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_IP_INPUT, priority, - ds_cstr(&match), ds_cstr(&actions), hint); + ds_cstr(&match), ds_cstr(&actions), hint, + lflow_ref); } else { ds_put_format(&actions, "%s { " @@ -11647,7 +11816,7 @@ build_lrouter_nd_flow(const struct ovn_datapath *od, struct ovn_port *op, ds_cstr(&match), ds_cstr(&actions), NULL, copp_meter_get(COPP_ND_NA, od->nbr->copp, meter_groups), - hint); + hint, lflow_ref); } ds_destroy(&match); @@ -11658,7 +11827,8 @@ static void build_lrouter_nat_arp_nd_flow(const struct ovn_datapath *od, struct ovn_nat *nat_entry, struct lflow_table *lflows, - const struct shash *meter_groups) + const struct shash *meter_groups, + struct lflow_ref *lflow_ref) { struct lport_addresses *ext_addrs = &nat_entry->ext_addrs; const struct nbrec_nat *nat = nat_entry->nb; @@ -11668,12 +11838,14 @@ build_lrouter_nat_arp_nd_flow(const struct ovn_datapath *od, ext_addrs->ipv6_addrs[0].addr_s, ext_addrs->ipv6_addrs[0].sn_addr_s, REG_INPORT_ETH_ADDR, NULL, false, 90, - &nat->header_, lflows, meter_groups); + &nat->header_, lflows, meter_groups, + lflow_ref); } else { build_lrouter_arp_flow(od, NULL, ext_addrs->ipv4_addrs[0].addr_s, REG_INPORT_ETH_ADDR, NULL, false, 90, - &nat->header_, lflows); + &nat->header_, lflows, + lflow_ref); } } @@ -11681,7 +11853,8 @@ static void build_lrouter_port_nat_arp_nd_flow(struct ovn_port *op, struct ovn_nat *nat_entry, struct lflow_table *lflows, - const struct shash *meter_groups) + const struct shash *meter_groups, + struct lflow_ref *lflow_ref) { struct lport_addresses *ext_addrs = &nat_entry->ext_addrs; const struct nbrec_nat *nat = nat_entry->nb; @@ -11729,21 +11902,25 @@ build_lrouter_port_nat_arp_nd_flow(struct ovn_port *op, ext_addrs->ipv6_addrs[0].addr_s, ext_addrs->ipv6_addrs[0].sn_addr_s, mac_s, &match, false, 92, - &nat->header_, lflows, meter_groups); + &nat->header_, lflows, meter_groups, + lflow_ref); build_lrouter_nd_flow(op->od, op, "nd_na", ext_addrs->ipv6_addrs[0].addr_s, ext_addrs->ipv6_addrs[0].sn_addr_s, mac_s, NULL, true, 91, - &nat->header_, lflows, meter_groups); + &nat->header_, lflows, meter_groups, + lflow_ref); } else { build_lrouter_arp_flow(op->od, op, ext_addrs->ipv4_addrs[0].addr_s, mac_s, &match, false, 92, - &nat->header_, lflows); + &nat->header_, lflows, + lflow_ref); build_lrouter_arp_flow(op->od, op, ext_addrs->ipv4_addrs[0].addr_s, mac_s, NULL, true, 91, - &nat->header_, lflows); + &nat->header_, lflows, + lflow_ref); } ds_destroy(&match); @@ -11754,7 +11931,8 @@ build_lrouter_drop_own_dest(struct ovn_port *op, const struct lr_stateful_record *lr_sful_rec, enum ovn_stage stage, uint16_t priority, bool drop_snat_ip, - struct lflow_table *lflows) + struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { struct ds match_ips = DS_EMPTY_INITIALIZER; @@ -11781,7 +11959,8 @@ build_lrouter_drop_own_dest(struct ovn_port *op, char *match = xasprintf("ip4.dst == {%s}", ds_cstr(&match_ips)); ovn_lflow_add_with_hint(lflows, op->od, stage, priority, match, debug_drop_action(), - &op->nbrp->header_); + &op->nbrp->header_, + lflow_ref); free(match); } } @@ -11811,7 +11990,8 @@ build_lrouter_drop_own_dest(struct ovn_port *op, char *match = xasprintf("ip6.dst == {%s}", ds_cstr(&match_ips)); ovn_lflow_add_with_hint(lflows, op->od, stage, priority, match, debug_drop_action(), - &op->nbrp->header_); + &op->nbrp->header_, + lflow_ref); free(match); } } @@ -11822,14 +12002,15 @@ static void build_lrouter_force_snat_flows(struct lflow_table *lflows, const struct ovn_datapath *od, const char *ip_version, const char *ip_addr, - const char *context) + const char *context, + struct lflow_ref *lflow_ref) { struct ds match = DS_EMPTY_INITIALIZER; struct ds actions = DS_EMPTY_INITIALIZER; ds_put_format(&match, "ip%s && ip%s.dst == %s", ip_version, ip_version, ip_addr); ovn_lflow_add(lflows, od, S_ROUTER_IN_UNSNAT, 110, - ds_cstr(&match), "ct_snat;"); + ds_cstr(&match), "ct_snat;", lflow_ref); /* Higher priority rules to force SNAT with the IP addresses * configured in the Gateway router. This only takes effect @@ -11839,7 +12020,8 @@ build_lrouter_force_snat_flows(struct lflow_table *lflows, context, ip_version); ds_put_format(&actions, "ct_snat(%s);", ip_addr); ovn_lflow_add(lflows, od, S_ROUTER_OUT_SNAT, 100, - ds_cstr(&match), ds_cstr(&actions)); + ds_cstr(&match), ds_cstr(&actions), + lflow_ref); ds_destroy(&match); ds_destroy(&actions); @@ -11849,7 +12031,8 @@ static void build_lrouter_force_snat_flows_op(struct ovn_port *op, const struct lr_nat_record *lrnat_rec, struct lflow_table *lflows, - struct ds *match, struct ds *actions) + struct ds *match, struct ds *actions, + struct lflow_ref *lflow_ref) { ovs_assert(op->nbrp); if (!op->peer || !lrnat_rec->lb_force_snat_router_ip) { @@ -11863,7 +12046,7 @@ build_lrouter_force_snat_flows_op(struct ovn_port *op, ds_put_format(match, "inport == %s && ip4.dst == %s", op->json_key, op->lrp_networks.ipv4_addrs[0].addr_s); ovn_lflow_add(lflows, op->od, S_ROUTER_IN_UNSNAT, 110, - ds_cstr(match), "ct_snat;"); + ds_cstr(match), "ct_snat;", lflow_ref); ds_clear(match); @@ -11875,7 +12058,8 @@ build_lrouter_force_snat_flows_op(struct ovn_port *op, ds_put_format(actions, "ct_snat(%s);", op->lrp_networks.ipv4_addrs[0].addr_s); ovn_lflow_add(lflows, op->od, S_ROUTER_OUT_SNAT, 110, - ds_cstr(match), ds_cstr(actions)); + ds_cstr(match), ds_cstr(actions), + lflow_ref); if (op->lrp_networks.n_ipv4_addrs > 1) { static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 5); VLOG_WARN_RL(&rl, "Logical router port %s is configured with " @@ -11895,7 +12079,7 @@ build_lrouter_force_snat_flows_op(struct ovn_port *op, ds_put_format(match, "inport == %s && ip6.dst == %s", op->json_key, op->lrp_networks.ipv6_addrs[0].addr_s); ovn_lflow_add(lflows, op->od, S_ROUTER_IN_UNSNAT, 110, - ds_cstr(match), "ct_snat;"); + ds_cstr(match), "ct_snat;", lflow_ref); ds_clear(match); @@ -11907,7 +12091,8 @@ build_lrouter_force_snat_flows_op(struct ovn_port *op, ds_put_format(actions, "ct_snat(%s);", op->lrp_networks.ipv6_addrs[0].addr_s); ovn_lflow_add(lflows, op->od, S_ROUTER_OUT_SNAT, 110, - ds_cstr(match), ds_cstr(actions)); + ds_cstr(match), ds_cstr(actions), + lflow_ref); if (op->lrp_networks.n_ipv6_addrs > 2) { static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 5); VLOG_WARN_RL(&rl, "Logical router port %s is configured with " @@ -11921,7 +12106,8 @@ build_lrouter_force_snat_flows_op(struct ovn_port *op, static void build_lrouter_bfd_flows(struct lflow_table *lflows, struct ovn_port *op, - const struct shash *meter_groups) + const struct shash *meter_groups, + struct lflow_ref *lflow_ref) { if (!op->has_bfd) { return; @@ -11936,7 +12122,8 @@ build_lrouter_bfd_flows(struct lflow_table *lflows, struct ovn_port *op, ds_cstr(&ip_list)); ovn_lflow_add_with_hint(lflows, op->od, S_ROUTER_IN_IP_INPUT, 110, ds_cstr(&match), "next; ", - &op->nbrp->header_); + &op->nbrp->header_, + lflow_ref); ds_clear(&match); ds_put_format(&match, "ip4.dst == %s && udp.dst == 3784", ds_cstr(&ip_list)); @@ -11944,7 +12131,8 @@ build_lrouter_bfd_flows(struct lflow_table *lflows, struct ovn_port *op, ds_cstr(&match), "handle_bfd_msg(); ", NULL, copp_meter_get(COPP_BFD, op->od->nbr->copp, meter_groups), - &op->nbrp->header_); + &op->nbrp->header_, + lflow_ref); } if (op->lrp_networks.n_ipv6_addrs) { ds_clear(&ip_list); @@ -11955,7 +12143,8 @@ build_lrouter_bfd_flows(struct lflow_table *lflows, struct ovn_port *op, ds_cstr(&ip_list)); ovn_lflow_add_with_hint(lflows, op->od, S_ROUTER_IN_IP_INPUT, 110, ds_cstr(&match), "next; ", - &op->nbrp->header_); + &op->nbrp->header_, + lflow_ref); ds_clear(&match); ds_put_format(&match, "ip6.dst == %s && udp.dst == 3784", ds_cstr(&ip_list)); @@ -11963,7 +12152,8 @@ build_lrouter_bfd_flows(struct lflow_table *lflows, struct ovn_port *op, ds_cstr(&match), "handle_bfd_msg(); ", NULL, copp_meter_get(COPP_BFD, op->od->nbr->copp, meter_groups), - &op->nbrp->header_); + &op->nbrp->header_, + lflow_ref); } ds_destroy(&ip_list); @@ -11975,16 +12165,19 @@ build_lrouter_bfd_flows(struct lflow_table *lflows, struct ovn_port *op, */ static void build_adm_ctrl_flows_for_lrouter( - struct ovn_datapath *od, struct lflow_table *lflows) + struct ovn_datapath *od, struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { ovs_assert(od->nbr); /* Logical VLANs not supported. * Broadcast/multicast source address is invalid. */ ovn_lflow_add(lflows, od, S_ROUTER_IN_ADMISSION, 100, - "vlan.present || eth.src[40]", debug_drop_action()); + "vlan.present || eth.src[40]", debug_drop_action(), + lflow_ref); /* Default action for L2 security is to drop. */ - ovn_lflow_add_default_drop(lflows, od, S_ROUTER_IN_ADMISSION); + ovn_lflow_add_default_drop(lflows, od, S_ROUTER_IN_ADMISSION, + lflow_ref); } static int @@ -12018,11 +12211,12 @@ build_gateway_get_l2_hdr_size(struct ovn_port *op) /* All 'gateway_mtu' and 'gateway_mtu_bypass' flows should be built with this * function. */ -static void OVS_PRINTF_FORMAT(9, 10) +static void OVS_PRINTF_FORMAT(10, 11) build_gateway_mtu_flow(struct lflow_table *lflows, struct ovn_port *op, enum ovn_stage stage, uint16_t prio_low, uint16_t prio_high, struct ds *match, struct ds *actions, const struct ovsdb_idl_row *hint, + struct lflow_ref *lflow_ref, const char *extra_actions_fmt, ...) { int gw_mtu = smap_get_int(&op->nbrp->options, "gateway_mtu", 0); @@ -12040,7 +12234,7 @@ build_gateway_mtu_flow(struct lflow_table *lflows, struct ovn_port *op, ds_put_format_valist(actions, extra_actions_fmt, extra_actions_args); ovn_lflow_add_with_hint(lflows, op->od, stage, prio_low, ds_cstr(match), ds_cstr(actions), - hint); + hint, lflow_ref); if (gw_mtu > 0) { const char *gw_mtu_bypass = smap_get(&op->nbrp->options, @@ -12052,7 +12246,7 @@ build_gateway_mtu_flow(struct lflow_table *lflows, struct ovn_port *op, ds_put_format(match, " && (%s)", gw_mtu_bypass); ovn_lflow_add_with_hint(lflows, op->od, stage, prio_high, ds_cstr(match), ds_cstr(actions), - hint); + hint, lflow_ref); } } va_end(extra_actions_args); @@ -12081,7 +12275,8 @@ consider_l3dgw_port_is_centralized(struct ovn_port *op) static void build_adm_ctrl_flows_for_lrouter_port( struct ovn_port *op, struct lflow_table *lflows, - struct ds *match, struct ds *actions) + struct ds *match, struct ds *actions, + struct lflow_ref *lflow_ref) { ovs_assert(op->nbrp); @@ -12105,6 +12300,7 @@ build_adm_ctrl_flows_for_lrouter_port( ds_put_format(match, "eth.mcast && inport == %s", op->json_key); build_gateway_mtu_flow(lflows, op, S_ROUTER_IN_ADMISSION, 50, 55, match, actions, &op->nbrp->header_, + lflow_ref, REG_INPORT_ETH_ADDR " = %s; next;", op->lrp_networks.ea_s); @@ -12125,6 +12321,7 @@ build_adm_ctrl_flows_for_lrouter_port( } build_gateway_mtu_flow(lflows, op, S_ROUTER_IN_ADMISSION, 50, 55, match, actions, &op->nbrp->header_, + lflow_ref, REG_INPORT_ETH_ADDR " = %s; next;", op->lrp_networks.ea_s); } @@ -12136,7 +12333,8 @@ static void build_neigh_learning_flows_for_lrouter( struct ovn_datapath *od, struct lflow_table *lflows, struct ds *match, struct ds *actions, - const struct shash *meter_groups) + const struct shash *meter_groups, + struct lflow_ref *lflow_ref) { ovs_assert(od->nbr); @@ -12183,7 +12381,7 @@ build_neigh_learning_flows_for_lrouter( learn_from_arp_request ? "" : REGBIT_LOOKUP_NEIGHBOR_IP_RESULT" = 1; "); ovn_lflow_add(lflows, od, S_ROUTER_IN_LOOKUP_NEIGHBOR, 100, - "arp.op == 2", ds_cstr(actions)); + "arp.op == 2", ds_cstr(actions), lflow_ref); ds_clear(actions); ds_put_format(actions, REGBIT_LOOKUP_NEIGHBOR_RESULT @@ -12191,7 +12389,7 @@ build_neigh_learning_flows_for_lrouter( learn_from_arp_request ? "" : REGBIT_LOOKUP_NEIGHBOR_IP_RESULT" = 1; "); ovn_lflow_add(lflows, od, S_ROUTER_IN_LOOKUP_NEIGHBOR, 100, "nd_na", - ds_cstr(actions)); + ds_cstr(actions), lflow_ref); if (!learn_from_arp_request) { /* Add flow to skip GARP LLA if we don't know it already. @@ -12208,7 +12406,7 @@ build_neigh_learning_flows_for_lrouter( " = lookup_nd_ip(inport, ip6.src); next;"); ovn_lflow_add(lflows, od, S_ROUTER_IN_LOOKUP_NEIGHBOR, 110, "nd_na && ip6.src == fe80::/10 && ip6.dst == ff00::/8", - ds_cstr(actions)); + ds_cstr(actions), lflow_ref); } ds_clear(actions); @@ -12218,12 +12416,13 @@ build_neigh_learning_flows_for_lrouter( REGBIT_LOOKUP_NEIGHBOR_IP_RESULT " = lookup_nd_ip(inport, ip6.src); "); ovn_lflow_add(lflows, od, S_ROUTER_IN_LOOKUP_NEIGHBOR, 100, "nd_ns", - ds_cstr(actions)); + ds_cstr(actions), lflow_ref); /* For other packet types, we can skip neighbor learning. * So set REGBIT_LOOKUP_NEIGHBOR_RESULT to 1. */ ovn_lflow_add(lflows, od, S_ROUTER_IN_LOOKUP_NEIGHBOR, 0, "1", - REGBIT_LOOKUP_NEIGHBOR_RESULT" = 1; next;"); + REGBIT_LOOKUP_NEIGHBOR_RESULT" = 1; next;", + lflow_ref); /* Flows for LEARN_NEIGHBOR. */ /* Skip Neighbor learning if not required. */ @@ -12232,33 +12431,40 @@ build_neigh_learning_flows_for_lrouter( learn_from_arp_request ? "" : " || "REGBIT_LOOKUP_NEIGHBOR_IP_RESULT" == 0"); ovn_lflow_add(lflows, od, S_ROUTER_IN_LEARN_NEIGHBOR, 100, - ds_cstr(match), "mac_cache_use; next;"); + ds_cstr(match), "mac_cache_use; next;", + lflow_ref); ovn_lflow_metered(lflows, od, S_ROUTER_IN_LEARN_NEIGHBOR, 90, "arp", "put_arp(inport, arp.spa, arp.sha); next;", copp_meter_get(COPP_ARP, od->nbr->copp, - meter_groups)); + meter_groups), + lflow_ref); ovn_lflow_add(lflows, od, S_ROUTER_IN_LEARN_NEIGHBOR, 95, - "nd_ns && (ip6.src == 0 || nd.sll == 0)", "next;"); + "nd_ns && (ip6.src == 0 || nd.sll == 0)", "next;", + lflow_ref); ovn_lflow_metered(lflows, od, S_ROUTER_IN_LEARN_NEIGHBOR, 95, "nd_na && nd.tll == 0", "put_nd(inport, nd.target, eth.src); next;", copp_meter_get(COPP_ND_NA, od->nbr->copp, - meter_groups)); + meter_groups), + lflow_ref); ovn_lflow_metered(lflows, od, S_ROUTER_IN_LEARN_NEIGHBOR, 90, "nd_na", "put_nd(inport, nd.target, nd.tll); next;", copp_meter_get(COPP_ND_NA, od->nbr->copp, - meter_groups)); + meter_groups), + lflow_ref); ovn_lflow_metered(lflows, od, S_ROUTER_IN_LEARN_NEIGHBOR, 90, "nd_ns", "put_nd(inport, ip6.src, nd.sll); next;", copp_meter_get(COPP_ND_NS, od->nbr->copp, - meter_groups)); + meter_groups), + lflow_ref); - ovn_lflow_add_default_drop(lflows, od, S_ROUTER_IN_LEARN_NEIGHBOR); + ovn_lflow_add_default_drop(lflows, od, S_ROUTER_IN_LEARN_NEIGHBOR, + lflow_ref); } /* Logical router ingress Table 1: Neighbor lookup lflows @@ -12266,7 +12472,8 @@ build_neigh_learning_flows_for_lrouter( static void build_neigh_learning_flows_for_lrouter_port( struct ovn_port *op, struct lflow_table *lflows, - struct ds *match, struct ds *actions) + struct ds *match, struct ds *actions, + struct lflow_ref *lflow_ref) { ovs_assert(op->nbrp); @@ -12298,7 +12505,8 @@ build_neigh_learning_flows_for_lrouter_port( ovn_lflow_add_with_hint(lflows, op->od, S_ROUTER_IN_LOOKUP_NEIGHBOR, 110, ds_cstr(match), actions_s, - &op->nbrp->header_); + &op->nbrp->header_, + lflow_ref); } ds_clear(match); ds_put_format(match, @@ -12319,7 +12527,8 @@ build_neigh_learning_flows_for_lrouter_port( ovn_lflow_add_with_hint(lflows, op->od, S_ROUTER_IN_LOOKUP_NEIGHBOR, 100, ds_cstr(match), ds_cstr(actions), - &op->nbrp->header_); + &op->nbrp->header_, + lflow_ref); } } @@ -12329,7 +12538,8 @@ static void build_ND_RA_flows_for_lrouter_port( struct ovn_port *op, struct lflow_table *lflows, struct ds *match, struct ds *actions, - const struct shash *meter_groups) + const struct shash *meter_groups, + struct lflow_ref *lflow_ref) { ovs_assert(op->nbrp); if (op->nbrp->peer || !op->peer) { @@ -12416,7 +12626,8 @@ build_ND_RA_flows_for_lrouter_port( copp_meter_get(COPP_ND_RA_OPTS, op->od->nbr->copp, meter_groups), - &op->nbrp->header_); + &op->nbrp->header_, + lflow_ref); ds_clear(actions); ds_clear(match); ds_put_format(match, "inport == %s && ip6.dst == ff02::2 && " @@ -12435,7 +12646,8 @@ build_ND_RA_flows_for_lrouter_port( ovn_lflow_add_with_hint(lflows, op->od, S_ROUTER_IN_ND_RA_RESPONSE, 50, ds_cstr(match), ds_cstr(actions), - &op->nbrp->header_); + &op->nbrp->header_, + lflow_ref); } } @@ -12443,22 +12655,26 @@ build_ND_RA_flows_for_lrouter_port( * responder, by default goto next. (priority 0). */ static void build_ND_RA_flows_for_lrouter(struct ovn_datapath *od, - struct lflow_table *lflows) + struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { ovs_assert(od->nbr); - ovn_lflow_add(lflows, od, S_ROUTER_IN_ND_RA_OPTIONS, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_ROUTER_IN_ND_RA_RESPONSE, 0, "1", "next;"); + ovn_lflow_add(lflows, od, S_ROUTER_IN_ND_RA_OPTIONS, 0, "1", "next;", + lflow_ref); + ovn_lflow_add(lflows, od, S_ROUTER_IN_ND_RA_RESPONSE, 0, "1", "next;", + lflow_ref); } /* Logical router ingress table IP_ROUTING_PRE: * by default goto next. (priority 0). */ static void build_ip_routing_pre_flows_for_lrouter(struct ovn_datapath *od, - struct lflow_table *lflows) + struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { ovs_assert(od->nbr); ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_ROUTING_PRE, 0, "1", - REG_ROUTE_TABLE_ID" = 0; next;"); + REG_ROUTE_TABLE_ID" = 0; next;", lflow_ref); } /* Logical router ingress table IP_ROUTING : IP Routing. @@ -12482,7 +12698,8 @@ build_ip_routing_pre_flows_for_lrouter(struct ovn_datapath *od, */ static void build_ip_routing_flows_for_lrp( - struct ovn_port *op, struct lflow_table *lflows) + struct ovn_port *op, struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { ovs_assert(op->nbrp); for (int i = 0; i < op->lrp_networks.n_ipv4_addrs; i++) { @@ -12490,7 +12707,7 @@ build_ip_routing_flows_for_lrp( op->lrp_networks.ipv4_addrs[i].network_s, op->lrp_networks.ipv4_addrs[i].plen, NULL, false, 0, &op->nbrp->header_, false, ROUTE_PRIO_OFFSET_CONNECTED, - NULL); + lflow_ref); } for (int i = 0; i < op->lrp_networks.n_ipv6_addrs; i++) { @@ -12498,7 +12715,7 @@ build_ip_routing_flows_for_lrp( op->lrp_networks.ipv6_addrs[i].network_s, op->lrp_networks.ipv6_addrs[i].plen, NULL, false, 0, &op->nbrp->header_, false, ROUTE_PRIO_OFFSET_CONNECTED, - NULL); + lflow_ref); } } @@ -12563,13 +12780,17 @@ static void build_static_route_flows_for_lrouter( struct ovn_datapath *od, const struct chassis_features *features, struct lflow_table *lflows, const struct hmap *lr_ports, - const struct hmap *bfd_connections) + const struct hmap *bfd_connections, + struct lflow_ref *lflow_ref) { ovs_assert(od->nbr); - ovn_lflow_add_default_drop(lflows, od, S_ROUTER_IN_IP_ROUTING_ECMP); - ovn_lflow_add_default_drop(lflows, od, S_ROUTER_IN_IP_ROUTING); + ovn_lflow_add_default_drop(lflows, od, S_ROUTER_IN_IP_ROUTING_ECMP, + lflow_ref); + ovn_lflow_add_default_drop(lflows, od, S_ROUTER_IN_IP_ROUTING, + lflow_ref); ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_ROUTING_ECMP, 150, - REG_ECMP_GROUP_ID" == 0", "next;"); + REG_ECMP_GROUP_ID" == 0", "next;", + lflow_ref); struct hmap ecmp_groups = HMAP_INITIALIZER(&ecmp_groups); struct hmap unique_routes = HMAP_INITIALIZER(&unique_routes); @@ -12579,7 +12800,7 @@ build_static_route_flows_for_lrouter( for (int i = 0; i < od->nbr->n_ports; i++) { build_route_table_lflow(od, lflows, od->nbr->ports[i], - &route_tables); + &route_tables, lflow_ref); } for (int i = 0; i < od->nbr->n_static_routes; i++) { @@ -12609,11 +12830,11 @@ build_static_route_flows_for_lrouter( /* add a flow in IP_ROUTING, and one flow for each member in * IP_ROUTING_ECMP. */ build_ecmp_route_flow(lflows, od, features->ct_no_masked_label, - lr_ports, group); + lr_ports, group, lflow_ref); } const struct unique_routes_node *ur; HMAP_FOR_EACH (ur, hmap_node, &unique_routes) { - build_static_route_flow(lflows, od, lr_ports, ur->route); + build_static_route_flow(lflows, od, lr_ports, ur->route, lflow_ref); } ecmp_groups_destroy(&ecmp_groups); unique_routes_destroy(&unique_routes); @@ -12627,7 +12848,8 @@ build_static_route_flows_for_lrouter( static void build_mcast_lookup_flows_for_lrouter( struct ovn_datapath *od, struct lflow_table *lflows, - struct ds *match, struct ds *actions) + struct ds *match, struct ds *actions, + struct lflow_ref *lflow_ref) { ovs_assert(od->nbr); @@ -12635,7 +12857,8 @@ build_mcast_lookup_flows_for_lrouter( * i.e., router solicitation and router advertisement. */ ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_ROUTING, 10550, - "nd_rs || nd_ra", debug_drop_action()); + "nd_rs || nd_ra", debug_drop_action(), + lflow_ref); if (!od->mcast_info.rtr.relay) { return; } @@ -12663,7 +12886,8 @@ build_mcast_lookup_flows_for_lrouter( ds_put_format(actions, "outport = \"%s\"; ip.ttl--; next;", igmp_group->mcgroup.name); ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_ROUTING, 10500, - ds_cstr(match), ds_cstr(actions)); + ds_cstr(match), ds_cstr(actions), + lflow_ref); } /* If needed, flood unregistered multicast on statically configured @@ -12682,13 +12906,15 @@ build_mcast_lookup_flows_for_lrouter( ds_put_format(match, "eth.src == %s && igmp", op->lrp_networks.ea_s); ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_ROUTING, 10550, - ds_cstr(match), debug_drop_action()); + ds_cstr(match), debug_drop_action(), + lflow_ref); ds_clear(match); ds_put_format(match, "eth.src == %s && (mldv1 || mldv2)", op->lrp_networks.ea_s); ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_ROUTING, 10550, - ds_cstr(match), debug_drop_action()); + ds_cstr(match), debug_drop_action(), + lflow_ref); } ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_ROUTING, 10460, @@ -12696,23 +12922,27 @@ build_mcast_lookup_flows_for_lrouter( "clone { " "outport = \""MC_STATIC"\"; " "next; " - "};"); + "};", + lflow_ref); ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_ROUTING, 10460, "mldv1 || mldv2", "clone { " "outport = \""MC_STATIC"\"; " "next; " - "};"); + "};", + lflow_ref); ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_ROUTING, 10450, "ip4.mcast || ip6.mcast", "clone { " "outport = \""MC_STATIC"\"; " "ip.ttl--; " "next; " - "};"); + "};", + lflow_ref); } else { ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_ROUTING, 10450, - "ip4.mcast || ip6.mcast", debug_drop_action()); + "ip4.mcast || ip6.mcast", debug_drop_action(), + lflow_ref); } } @@ -12728,16 +12958,20 @@ build_mcast_lookup_flows_for_lrouter( static void build_ingress_policy_flows_for_lrouter( struct ovn_datapath *od, struct lflow_table *lflows, - const struct hmap *lr_ports) + const struct hmap *lr_ports, + struct lflow_ref *lflow_ref) { ovs_assert(od->nbr); /* This is a catch-all rule. It has the lowest priority (0) * does a match-all("1") and pass-through (next) */ ovn_lflow_add(lflows, od, S_ROUTER_IN_POLICY, 0, "1", - REG_ECMP_GROUP_ID" = 0; next;"); + REG_ECMP_GROUP_ID" = 0; next;", + lflow_ref); ovn_lflow_add(lflows, od, S_ROUTER_IN_POLICY_ECMP, 150, - REG_ECMP_GROUP_ID" == 0", "next;"); - ovn_lflow_add_default_drop(lflows, od, S_ROUTER_IN_POLICY_ECMP); + REG_ECMP_GROUP_ID" == 0", "next;", + lflow_ref); + ovn_lflow_add_default_drop(lflows, od, S_ROUTER_IN_POLICY_ECMP, + lflow_ref); /* Convert routing policies to flows. */ uint16_t ecmp_group_id = 1; @@ -12749,11 +12983,11 @@ build_ingress_policy_flows_for_lrouter( if (is_ecmp_reroute) { build_ecmp_routing_policy_flows(lflows, od, lr_ports, rule, - ecmp_group_id); + ecmp_group_id, lflow_ref); ecmp_group_id++; } else { build_routing_policy_flow(lflows, od, lr_ports, rule, - &rule->header_); + &rule->header_, lflow_ref); } } } @@ -12761,21 +12995,26 @@ build_ingress_policy_flows_for_lrouter( /* Local router ingress table ARP_RESOLVE: ARP Resolution. */ static void build_arp_resolve_flows_for_lrouter( - struct ovn_datapath *od, struct lflow_table *lflows) + struct ovn_datapath *od, struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { ovs_assert(od->nbr); /* Multicast packets already have the outport set so just advance to * next table (priority 500). */ ovn_lflow_add(lflows, od, S_ROUTER_IN_ARP_RESOLVE, 500, - "ip4.mcast || ip6.mcast", "next;"); + "ip4.mcast || ip6.mcast", "next;", + lflow_ref); ovn_lflow_add(lflows, od, S_ROUTER_IN_ARP_RESOLVE, 1, "ip4", - "get_arp(outport, " REG_NEXT_HOP_IPV4 "); next;"); + "get_arp(outport, " REG_NEXT_HOP_IPV4 "); next;", + lflow_ref); ovn_lflow_add(lflows, od, S_ROUTER_IN_ARP_RESOLVE, 1, "ip6", - "get_nd(outport, " REG_NEXT_HOP_IPV6 "); next;"); + "get_nd(outport, " REG_NEXT_HOP_IPV6 "); next;", + lflow_ref); - ovn_lflow_add_default_drop(lflows, od, S_ROUTER_IN_ARP_RESOLVE); + ovn_lflow_add_default_drop(lflows, od, S_ROUTER_IN_ARP_RESOLVE, + lflow_ref); } static void @@ -12808,9 +13047,8 @@ routable_addresses_to_lflows(struct lflow_table *lflows, ds_clear(actions); ds_put_format(actions, "eth.dst = %s; next;", ra.laddrs[i].ea_s); - ovn_lflow_add_with_lflow_ref(lflows, peer->od, S_ROUTER_IN_ARP_RESOLVE, - 100, ds_cstr(match), ds_cstr(actions), - lflow_ref); + ovn_lflow_add(lflows, peer->od, S_ROUTER_IN_ARP_RESOLVE, 100, + ds_cstr(match), ds_cstr(actions), lflow_ref); } destroy_routable_addresses(&ra); } @@ -12829,7 +13067,8 @@ routable_addresses_to_lflows(struct lflow_table *lflows, static void build_arp_resolve_flows_for_lrp( struct ovn_port *op, - struct lflow_table *lflows, struct ds *match, struct ds *actions) + struct lflow_table *lflows, struct ds *match, struct ds *actions, + struct lflow_ref *lflow_ref) { ovs_assert(op->nbrp); /* This is a logical router port. If next-hop IP address in @@ -12854,7 +13093,8 @@ build_arp_resolve_flows_for_lrp( ovn_lflow_add_with_hint(lflows, op->peer->od, S_ROUTER_IN_ARP_RESOLVE, 100, ds_cstr(match), ds_cstr(actions), - &op->nbrp->header_); + &op->nbrp->header_, + lflow_ref); } if (op->lrp_networks.n_ipv6_addrs) { @@ -12870,7 +13110,8 @@ build_arp_resolve_flows_for_lrp( ovn_lflow_add_with_hint(lflows, op->peer->od, S_ROUTER_IN_ARP_RESOLVE, 100, ds_cstr(match), ds_cstr(actions), - &op->nbrp->header_); + &op->nbrp->header_, + lflow_ref); } } @@ -12895,7 +13136,8 @@ build_arp_resolve_flows_for_lrp( ovn_lflow_add_with_hint(lflows, op->od, S_ROUTER_IN_ARP_RESOLVE, 50, ds_cstr(match), ds_cstr(actions), - &op->nbrp->header_); + &op->nbrp->header_, + lflow_ref); } } } @@ -12945,7 +13187,7 @@ build_arp_resolve_flows_for_lsp( ds_clear(actions); ds_put_format(actions, "eth.dst = %s; next;", ea_s); - ovn_lflow_add_with_lflow_ref_hint(lflows, peer->od, + ovn_lflow_add_with_hint(lflows, peer->od, S_ROUTER_IN_ARP_RESOLVE, 100, ds_cstr(match), ds_cstr(actions), @@ -12977,7 +13219,7 @@ build_arp_resolve_flows_for_lsp( ds_clear(actions); ds_put_format(actions, "eth.dst = %s; next;", ea_s); - ovn_lflow_add_with_lflow_ref_hint(lflows, peer->od, + ovn_lflow_add_with_hint(lflows, peer->od, S_ROUTER_IN_ARP_RESOLVE, 100, ds_cstr(match), ds_cstr(actions), @@ -13026,7 +13268,7 @@ build_arp_resolve_flows_for_lsp( ds_clear(actions); ds_put_format(actions, "eth.dst = %s; next;", router_port->lrp_networks.ea_s); - ovn_lflow_add_with_lflow_ref_hint(lflows, peer->od, + ovn_lflow_add_with_hint(lflows, peer->od, S_ROUTER_IN_ARP_RESOLVE, 100, ds_cstr(match), ds_cstr(actions), &op->nbsp->header_, @@ -13043,7 +13285,7 @@ build_arp_resolve_flows_for_lsp( ds_clear(actions); ds_put_format(actions, "eth.dst = %s; next;", router_port->lrp_networks.ea_s); - ovn_lflow_add_with_lflow_ref_hint(lflows, peer->od, + ovn_lflow_add_with_hint(lflows, peer->od, S_ROUTER_IN_ARP_RESOLVE, 100, ds_cstr(match), ds_cstr(actions), &op->nbsp->header_, @@ -13104,7 +13346,8 @@ build_icmperr_pkt_big_flows(struct ovn_port *op, int mtu, struct lflow_table *lflows, const struct shash *meter_groups, struct ds *match, struct ds *actions, enum ovn_stage stage, - struct ovn_port *outport) + struct ovn_port *outport, + struct lflow_ref *lflow_ref) { char *outport_match = outport ? xasprintf("outport == %s && ", outport->json_key) @@ -13147,7 +13390,8 @@ build_icmperr_pkt_big_flows(struct ovn_port *op, int mtu, COPP_ICMP4_ERR, op->od->nbr->copp, meter_groups), - &op->nbrp->header_); + &op->nbrp->header_, + lflow_ref); } char *ip6_src = NULL; @@ -13187,7 +13431,8 @@ build_icmperr_pkt_big_flows(struct ovn_port *op, int mtu, COPP_ICMP6_ERR, op->od->nbr->copp, meter_groups), - &op->nbrp->header_); + &op->nbrp->header_, + lflow_ref); } free(outport_match); } @@ -13198,7 +13443,8 @@ build_check_pkt_len_flows_for_lrp(struct ovn_port *op, const struct hmap *lr_ports, const struct shash *meter_groups, struct ds *match, - struct ds *actions) + struct ds *actions, + struct lflow_ref *lflow_ref) { int gw_mtu = smap_get_int(&op->nbrp->options, "gateway_mtu", 0); if (gw_mtu <= 0) { @@ -13208,12 +13454,13 @@ build_check_pkt_len_flows_for_lrp(struct ovn_port *op, ds_clear(match); ds_put_format(match, "outport == %s", op->json_key); build_gateway_mtu_flow(lflows, op, S_ROUTER_IN_CHK_PKT_LEN, 50, 55, - match, actions, &op->nbrp->header_, "next;"); + match, actions, &op->nbrp->header_, + lflow_ref, "next;"); /* ingress traffic */ build_icmperr_pkt_big_flows(op, gw_mtu, lflows, meter_groups, match, actions, S_ROUTER_IN_IP_INPUT, - NULL); + NULL, lflow_ref); for (size_t i = 0; i < op->od->nbr->n_ports; i++) { struct ovn_port *rp = ovn_port_find(lr_ports, @@ -13225,7 +13472,7 @@ build_check_pkt_len_flows_for_lrp(struct ovn_port *op, /* egress traffic */ build_icmperr_pkt_big_flows(rp, gw_mtu, lflows, meter_groups, match, actions, S_ROUTER_IN_LARGER_PKTS, - op); + op, lflow_ref); } } @@ -13247,15 +13494,16 @@ build_check_pkt_len_flows_for_lrouter( struct ovn_datapath *od, struct lflow_table *lflows, const struct hmap *lr_ports, struct ds *match, struct ds *actions, - const struct shash *meter_groups) + const struct shash *meter_groups, + struct lflow_ref *lflow_ref) { ovs_assert(od->nbr); /* Packets are allowed by default. */ ovn_lflow_add(lflows, od, S_ROUTER_IN_CHK_PKT_LEN, 0, "1", - "next;"); + "next;", lflow_ref); ovn_lflow_add(lflows, od, S_ROUTER_IN_LARGER_PKTS, 0, "1", - "next;"); + "next;", lflow_ref); for (size_t i = 0; i < od->nbr->n_ports; i++) { struct ovn_port *rp = ovn_port_find(lr_ports, @@ -13264,7 +13512,7 @@ build_check_pkt_len_flows_for_lrouter( continue; } build_check_pkt_len_flows_for_lrp(rp, lflows, lr_ports, meter_groups, - match, actions); + match, actions, lflow_ref); } } @@ -13272,7 +13520,8 @@ build_check_pkt_len_flows_for_lrouter( static void build_gateway_redirect_flows_for_lrouter( struct ovn_datapath *od, struct lflow_table *lflows, - struct ds *match, struct ds *actions) + struct ds *match, struct ds *actions, + struct lflow_ref *lflow_ref) { ovs_assert(od->nbr); for (size_t i = 0; i < od->n_l3dgw_ports; i++) { @@ -13305,18 +13554,20 @@ build_gateway_redirect_flows_for_lrouter( od->l3dgw_ports[i]->cr_port->json_key); ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_GW_REDIRECT, 50, ds_cstr(match), ds_cstr(actions), - stage_hint); + stage_hint, lflow_ref); } /* Packets are allowed by default. */ - ovn_lflow_add(lflows, od, S_ROUTER_IN_GW_REDIRECT, 0, "1", "next;"); + ovn_lflow_add(lflows, od, S_ROUTER_IN_GW_REDIRECT, 0, "1", "next;", + lflow_ref); } /* Logical router ingress table GW_REDIRECT: Gateway redirect. */ static void build_lr_gateway_redirect_flows_for_nats( const struct ovn_datapath *od, const struct lr_nat_record *lrnat_rec, - struct lflow_table *lflows, struct ds *match, struct ds *actions) + struct lflow_table *lflows, struct ds *match, struct ds *actions, + struct lflow_ref *lflow_ref) { ovs_assert(od->nbr); for (size_t i = 0; i < od->n_l3dgw_ports; i++) { @@ -13357,21 +13608,23 @@ build_lr_gateway_redirect_flows_for_nats( if (nat->nb->allowed_ext_ips) { ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_GW_REDIRECT, 75, ds_cstr(&match_ext), - ds_cstr(actions), stage_hint); + ds_cstr(actions), stage_hint, + lflow_ref); if (add_def_flow) { ds_clear(&match_ext); ds_put_format(&match_ext, "ip && ip%s.dst == %s", nat_entry_is_v6(nat) ? "6" : "4", nat->nb->external_ip); ovn_lflow_add(lflows, od, S_ROUTER_IN_GW_REDIRECT, 70, - ds_cstr(&match_ext), debug_drop_action()); + ds_cstr(&match_ext), debug_drop_action(), + lflow_ref); add_def_flow = false; } } else if (nat->nb->exempted_ext_ips) { ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_GW_REDIRECT, 75, ds_cstr(&match_ext), debug_drop_action(), - stage_hint); + stage_hint, lflow_ref); } ds_destroy(&match_ext); } @@ -13387,7 +13640,8 @@ static void build_arp_request_flows_for_lrouter( struct ovn_datapath *od, struct lflow_table *lflows, struct ds *match, struct ds *actions, - const struct shash *meter_groups) + const struct shash *meter_groups, + struct lflow_ref *lflow_ref) { ovs_assert(od->nbr); for (int i = 0; i < od->nbr->n_static_routes; i++) { @@ -13429,7 +13683,8 @@ build_arp_request_flows_for_lrouter( copp_meter_get(COPP_ND_NS_RESOLVE, od->nbr->copp, meter_groups), - &route->header_); + &route->header_, + lflow_ref); } ovn_lflow_metered(lflows, od, S_ROUTER_IN_ARP_REQUEST, 100, @@ -13442,7 +13697,8 @@ build_arp_request_flows_for_lrouter( "output; " "};", copp_meter_get(COPP_ARP_RESOLVE, od->nbr->copp, - meter_groups)); + meter_groups), + lflow_ref); ovn_lflow_metered(lflows, od, S_ROUTER_IN_ARP_REQUEST, 100, "eth.dst == 00:00:00:00:00:00 && ip6", "nd_ns { " @@ -13450,8 +13706,10 @@ build_arp_request_flows_for_lrouter( "output; " "};", copp_meter_get(COPP_ND_NS_RESOLVE, od->nbr->copp, - meter_groups)); - ovn_lflow_add(lflows, od, S_ROUTER_IN_ARP_REQUEST, 0, "1", "output;"); + meter_groups), + lflow_ref); + ovn_lflow_add(lflows, od, S_ROUTER_IN_ARP_REQUEST, 0, "1", "output;", + lflow_ref); } /* Logical router egress table DELIVERY: Delivery (priority 100-110). @@ -13464,7 +13722,8 @@ build_arp_request_flows_for_lrouter( static void build_egress_delivery_flows_for_lrouter_port( struct ovn_port *op, struct lflow_table *lflows, - struct ds *match, struct ds *actions) + struct ds *match, struct ds *actions, + struct lflow_ref *lflow_ref) { ovs_assert(op->nbrp); if (!lrport_is_enabled(op->nbrp)) { @@ -13492,20 +13751,23 @@ build_egress_delivery_flows_for_lrouter_port( ds_put_format(actions, "eth.src = %s; output;", op->lrp_networks.ea_s); ovn_lflow_add(lflows, op->od, S_ROUTER_OUT_DELIVERY, 110, - ds_cstr(match), ds_cstr(actions)); + ds_cstr(match), ds_cstr(actions), + lflow_ref); } ds_clear(match); ds_put_format(match, "outport == %s", op->json_key); ovn_lflow_add(lflows, op->od, S_ROUTER_OUT_DELIVERY, 100, - ds_cstr(match), "output;"); + ds_cstr(match), "output;", lflow_ref); - ovn_lflow_add_default_drop(lflows, op->od, S_ROUTER_OUT_DELIVERY); + ovn_lflow_add_default_drop(lflows, op->od, S_ROUTER_OUT_DELIVERY, + lflow_ref); } static void build_misc_local_traffic_drop_flows_for_lrouter( - struct ovn_datapath *od, struct lflow_table *lflows) + struct ovn_datapath *od, struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { ovs_assert(od->nbr); /* Allow IGMP and MLD packets (with TTL = 1) if the router is @@ -13513,9 +13775,11 @@ build_misc_local_traffic_drop_flows_for_lrouter( */ if (od->mcast_info.rtr.flood_static) { ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_INPUT, 120, - "igmp && ip.ttl == 1", "next;"); + "igmp && ip.ttl == 1", "next;", + lflow_ref); ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_INPUT, 120, - "(mldv1 || mldv2) && ip.ttl == 1", "next;"); + "(mldv1 || mldv2) && ip.ttl == 1", "next;", + lflow_ref); } /* L3 admission control: drop multicast and broadcast source, localhost @@ -13528,7 +13792,8 @@ build_misc_local_traffic_drop_flows_for_lrouter( "ip4.dst == 127.0.0.0/8 || " "ip4.src == 0.0.0.0/8 || " "ip4.dst == 0.0.0.0/8", - debug_drop_action()); + debug_drop_action(), + lflow_ref); /* Drop ARP packets (priority 85). ARP request packets for router's own * IPs are handled with priority-90 flows. @@ -13536,28 +13801,32 @@ build_misc_local_traffic_drop_flows_for_lrouter( * IPs are handled with priority-90 flows. */ ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_INPUT, 85, - "arp || nd", debug_drop_action()); + "arp || nd", debug_drop_action(), + lflow_ref); /* Allow IPv6 multicast traffic that's supposed to reach the * router pipeline (e.g., router solicitations). */ ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_INPUT, 84, "nd_rs || nd_ra", - "next;"); + "next;", lflow_ref); /* Drop other reserved multicast. */ ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_INPUT, 83, - "ip6.mcast_rsvd", debug_drop_action()); + "ip6.mcast_rsvd", debug_drop_action(), + lflow_ref); /* Allow other multicast if relay enabled (priority 82). */ ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_INPUT, 82, "ip4.mcast || ip6.mcast", (od->mcast_info.rtr.relay ? "next;" : - debug_drop_action())); + debug_drop_action()), + lflow_ref); /* Drop Ethernet local broadcast. By definition this traffic should * not be forwarded.*/ ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_INPUT, 50, - "eth.bcast", debug_drop_action()); + "eth.bcast", debug_drop_action(), + lflow_ref); /* Avoid ICMP time exceeded for multicast, silent drop instead. * See RFC1812 section 5.3.1: @@ -13574,21 +13843,25 @@ build_misc_local_traffic_drop_flows_for_lrouter( * (priority-31 flows will send ICMP time exceeded) */ ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_INPUT, 32, "ip.ttl == {0, 1} && !ip.later_frag && " - "(ip4.mcast || ip6.mcast)", debug_drop_action()); + "(ip4.mcast || ip6.mcast)", debug_drop_action(), + lflow_ref); /* TTL discard */ ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_INPUT, 30, - "ip.ttl == {0, 1}", debug_drop_action()); + "ip.ttl == {0, 1}", debug_drop_action(), + lflow_ref); /* Pass other traffic not already handled to the next table for * routing. */ - ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_INPUT, 0, "1", "next;"); + ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_INPUT, 0, "1", "next;", + lflow_ref); } static void build_dhcpv6_reply_flows_for_lrouter_port( struct ovn_port *op, struct lflow_table *lflows, - struct ds *match) + struct ds *match, + struct lflow_ref *lflow_ref) { ovs_assert(op->nbrp); if (op->l3dgw_port) { @@ -13601,7 +13874,8 @@ build_dhcpv6_reply_flows_for_lrouter_port( op->lrp_networks.ipv6_addrs[i].addr_s); ovn_lflow_add(lflows, op->od, S_ROUTER_IN_IP_INPUT, 100, ds_cstr(match), - "reg0 = 0; handle_dhcpv6_reply;"); + "reg0 = 0; handle_dhcpv6_reply;", + lflow_ref); } } @@ -13609,7 +13883,8 @@ static void build_ipv6_input_flows_for_lrouter_port( struct ovn_port *op, struct lflow_table *lflows, struct ds *match, struct ds *actions, - const struct shash *meter_groups) + const struct shash *meter_groups, + struct lflow_ref *lflow_ref) { ovs_assert(op->nbrp); if (is_cr_port(op)) { @@ -13633,7 +13908,7 @@ build_ipv6_input_flows_for_lrouter_port( "next; "; ovn_lflow_add_with_hint(lflows, op->od, S_ROUTER_IN_IP_INPUT, 90, ds_cstr(match), lrp_actions, - &op->nbrp->header_); + &op->nbrp->header_, lflow_ref); } /* ND reply. These flows reply to ND solicitations for the @@ -13654,7 +13929,8 @@ build_ipv6_input_flows_for_lrouter_port( op->lrp_networks.ipv6_addrs[i].addr_s, op->lrp_networks.ipv6_addrs[i].sn_addr_s, REG_INPORT_ETH_ADDR, match, false, 90, - &op->nbrp->header_, lflows, meter_groups); + &op->nbrp->header_, lflows, meter_groups, + lflow_ref); } /* UDP/TCP/SCTP port unreachable */ @@ -13674,7 +13950,8 @@ build_ipv6_input_flows_for_lrouter_port( COPP_TCP_RESET, op->od->nbr->copp, meter_groups), - &op->nbrp->header_); + &op->nbrp->header_, + lflow_ref); ds_clear(match); ds_put_format(match, @@ -13690,7 +13967,8 @@ build_ipv6_input_flows_for_lrouter_port( COPP_TCP_RESET, op->od->nbr->copp, meter_groups), - &op->nbrp->header_); + &op->nbrp->header_, + lflow_ref); ds_clear(match); ds_put_format(match, @@ -13709,7 +13987,8 @@ build_ipv6_input_flows_for_lrouter_port( COPP_ICMP6_ERR, op->od->nbr->copp, meter_groups), - &op->nbrp->header_); + &op->nbrp->header_, + lflow_ref); ds_clear(match); ds_put_format(match, @@ -13728,7 +14007,8 @@ build_ipv6_input_flows_for_lrouter_port( COPP_ICMP6_ERR, op->od->nbr->copp, meter_groups), - &op->nbrp->header_); + &op->nbrp->header_, + lflow_ref); } } @@ -13768,7 +14048,7 @@ build_ipv6_input_flows_for_lrouter_port( 31, ds_cstr(match), ds_cstr(actions), NULL, copp_meter_get(COPP_ICMP6_ERR, op->od->nbr->copp, meter_groups), - &op->nbrp->header_); + &op->nbrp->header_, lflow_ref); } ds_destroy(&ip_ds); } @@ -13777,7 +14057,8 @@ static void build_lrouter_arp_nd_for_datapath(const struct ovn_datapath *od, const struct lr_nat_record *lrnat_rec, struct lflow_table *lflows, - const struct shash *meter_groups) + const struct shash *meter_groups, + struct lflow_ref *lflow_ref) { ovs_assert(od->nbr); if (!od->nbr->n_nat) { @@ -13806,7 +14087,8 @@ build_lrouter_arp_nd_for_datapath(const struct ovn_datapath *od, if (!strcmp(nat_entry->nb->type, "snat")) { continue; } - build_lrouter_nat_arp_nd_flow(od, nat_entry, lflows, meter_groups); + build_lrouter_nat_arp_nd_flow(od, nat_entry, lflows, meter_groups, + lflow_ref); } /* Now handle SNAT entries too, one per unique SNAT IP. */ @@ -13821,7 +14103,8 @@ build_lrouter_arp_nd_for_datapath(const struct ovn_datapath *od, struct ovn_nat *nat_entry = CONTAINER_OF(ovs_list_front(&snat_ip->snat_entries), struct ovn_nat, ext_addr_list_node); - build_lrouter_nat_arp_nd_flow(od, nat_entry, lflows, meter_groups); + build_lrouter_nat_arp_nd_flow(od, nat_entry, lflows, meter_groups, + lflow_ref); } } @@ -13830,7 +14113,8 @@ static void build_lrouter_ipv4_ip_input(struct ovn_port *op, struct lflow_table *lflows, struct ds *match, struct ds *actions, - const struct shash *meter_groups) + const struct shash *meter_groups, + struct lflow_ref *lflow_ref) { ovs_assert(op->nbrp); /* No ingress packets are accepted on a chassisredirect @@ -13848,7 +14132,7 @@ build_lrouter_ipv4_ip_input(struct ovn_port *op, ds_put_cstr(match, " && "REGBIT_EGRESS_LOOPBACK" == 0"); ovn_lflow_add_with_hint(lflows, op->od, S_ROUTER_IN_IP_INPUT, 100, ds_cstr(match), debug_drop_action(), - &op->nbrp->header_); + &op->nbrp->header_, lflow_ref); /* ICMP echo reply. These flows reply to ICMP echo requests * received for the router's IP address. Since packets only @@ -13867,11 +14151,11 @@ build_lrouter_ipv4_ip_input(struct ovn_port *op, "next; "; ovn_lflow_add_with_hint(lflows, op->od, S_ROUTER_IN_IP_INPUT, 90, ds_cstr(match), icmp_actions, - &op->nbrp->header_); + &op->nbrp->header_, lflow_ref); } /* BFD msg handling */ - build_lrouter_bfd_flows(lflows, op, meter_groups); + build_lrouter_bfd_flows(lflows, op, meter_groups, lflow_ref); /* ICMP time exceeded */ struct ds ip_ds = DS_EMPTY_INITIALIZER; @@ -13900,7 +14184,7 @@ build_lrouter_ipv4_ip_input(struct ovn_port *op, 31, ds_cstr(match), ds_cstr(actions), NULL, copp_meter_get(COPP_ICMP4_ERR, op->od->nbr->copp, meter_groups), - &op->nbrp->header_); + &op->nbrp->header_, lflow_ref); } ds_destroy(&ip_ds); @@ -13950,7 +14234,7 @@ build_lrouter_ipv4_ip_input(struct ovn_port *op, build_lrouter_arp_flow(op->od, op, op->lrp_networks.ipv4_addrs[i].addr_s, REG_INPORT_ETH_ADDR, match, false, 90, - &op->nbrp->header_, lflows); + &op->nbrp->header_, lflows, lflow_ref); } if (!op->od->is_gw_router && !op->od->n_l3dgw_ports) { @@ -13973,7 +14257,8 @@ build_lrouter_ipv4_ip_input(struct ovn_port *op, COPP_ICMP4_ERR, op->od->nbr->copp, meter_groups), - &op->nbrp->header_); + &op->nbrp->header_, + lflow_ref); ds_clear(match); ds_put_format(match, @@ -13989,7 +14274,8 @@ build_lrouter_ipv4_ip_input(struct ovn_port *op, COPP_TCP_RESET, op->od->nbr->copp, meter_groups), - &op->nbrp->header_); + &op->nbrp->header_, + lflow_ref); ds_clear(match); ds_put_format(match, @@ -14005,7 +14291,8 @@ build_lrouter_ipv4_ip_input(struct ovn_port *op, COPP_TCP_RESET, op->od->nbr->copp, meter_groups), - &op->nbrp->header_); + &op->nbrp->header_, + lflow_ref); ds_clear(match); ds_put_format(match, @@ -14024,7 +14311,8 @@ build_lrouter_ipv4_ip_input(struct ovn_port *op, COPP_ICMP4_ERR, op->od->nbr->copp, meter_groups), - &op->nbrp->header_); + &op->nbrp->header_, + lflow_ref); } } } @@ -14034,7 +14322,8 @@ static void build_lrouter_ipv4_ip_input_for_lbnats(struct ovn_port *op, struct lflow_table *lflows, const struct lr_stateful_record *lr_sful_rec, - struct ds *match, const struct shash *meter_groups) + struct ds *match, const struct shash *meter_groups, + struct lflow_ref *lflow_ref) { ovs_assert(op->nbrp); /* No ingress packets are accepted on a chassisredirect @@ -14055,7 +14344,7 @@ build_lrouter_ipv4_ip_input_for_lbnats(struct ovn_port *op, AF_INET); build_lrouter_arp_flow(op->od, op, lb_ips_v4_as, REG_INPORT_ETH_ADDR, - match, false, 90, NULL, lflows); + match, false, 90, NULL, lflows, lflow_ref); free(lb_ips_v4_as); } @@ -14072,7 +14361,7 @@ build_lrouter_ipv4_ip_input_for_lbnats(struct ovn_port *op, AF_INET6); build_lrouter_nd_flow(op->od, op, "nd_na", lb_ips_v6_as, NULL, REG_INPORT_ETH_ADDR, match, false, 90, - NULL, lflows, meter_groups); + NULL, lflows, meter_groups, lflow_ref); free(lb_ips_v6_as); } @@ -14104,7 +14393,7 @@ build_lrouter_ipv4_ip_input_for_lbnats(struct ovn_port *op, continue; } build_lrouter_port_nat_arp_nd_flow(op, nat_entry, lflows, - meter_groups); + meter_groups, lflow_ref); } /* Now handle SNAT entries too, one per unique SNAT IP. */ @@ -14120,7 +14409,7 @@ build_lrouter_ipv4_ip_input_for_lbnats(struct ovn_port *op, CONTAINER_OF(ovs_list_front(&snat_ip->snat_entries), struct ovn_nat, ext_addr_list_node); build_lrouter_port_nat_arp_nd_flow(op, nat_entry, lflows, - meter_groups); + meter_groups, lflow_ref); } } @@ -14156,7 +14445,8 @@ build_lrouter_in_unsnat_stateless_flow(struct lflow_table *lflows, const struct nbrec_nat *nat, struct ds *match, bool distributed_nat, bool is_v6, - struct ovn_port *l3dgw_port) + struct ovn_port *l3dgw_port, + struct lflow_ref *lflow_ref) { if (strcmp(nat->type, "snat") && strcmp(nat->type, "dnat_and_snat")) { return; @@ -14169,7 +14459,7 @@ build_lrouter_in_unsnat_stateless_flow(struct lflow_table *lflows, ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_UNSNAT, priority, ds_cstr(match), "next;", - &nat->header_); + &nat->header_, lflow_ref); } static void @@ -14177,7 +14467,8 @@ build_lrouter_in_unsnat_in_czone_flow(struct lflow_table *lflows, const struct ovn_datapath *od, const struct nbrec_nat *nat, struct ds *match, bool distributed_nat, - bool is_v6, struct ovn_port *l3dgw_port) + bool is_v6, struct ovn_port *l3dgw_port, + struct lflow_ref *lflow_ref) { if (strcmp(nat->type, "snat") && strcmp(nat->type, "dnat_and_snat")) { return; @@ -14195,7 +14486,7 @@ build_lrouter_in_unsnat_in_czone_flow(struct lflow_table *lflows, ds_put_cstr(match, " && flags.loopback == 0"); ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_UNSNAT, 100, ds_cstr(match), "ct_snat_in_czone;", - &nat->header_); + &nat->header_, lflow_ref); ds_truncate(match, common_match_len); /* Update common zone match for the hairpin traffic. */ @@ -14203,7 +14494,7 @@ build_lrouter_in_unsnat_in_czone_flow(struct lflow_table *lflows, ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_UNSNAT, 100, ds_cstr(match), "ct_snat;", - &nat->header_); + &nat->header_, lflow_ref); } static void @@ -14211,7 +14502,8 @@ build_lrouter_in_unsnat_flow(struct lflow_table *lflows, const struct ovn_datapath *od, const struct nbrec_nat *nat, struct ds *match, bool distributed_nat, bool is_v6, - struct ovn_port *l3dgw_port) + struct ovn_port *l3dgw_port, + struct lflow_ref *lflow_ref) { if (strcmp(nat->type, "snat") && strcmp(nat->type, "dnat_and_snat")) { @@ -14225,7 +14517,7 @@ build_lrouter_in_unsnat_flow(struct lflow_table *lflows, ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_UNSNAT, priority, ds_cstr(match), "ct_snat;", - &nat->header_); + &nat->header_, lflow_ref); } static void @@ -14235,7 +14527,8 @@ build_lrouter_in_dnat_flow(struct lflow_table *lflows, const struct nbrec_nat *nat, struct ds *match, struct ds *actions, bool distributed_nat, int cidr_bits, bool is_v6, - struct ovn_port *l3dgw_port, bool stateless) + struct ovn_port *l3dgw_port, bool stateless, + struct lflow_ref *lflow_ref) { /* Ingress DNAT table: Packets enter the pipeline with destination * IP address that needs to be DNATted from a external IP address @@ -14281,7 +14574,8 @@ build_lrouter_in_dnat_flow(struct lflow_table *lflows, if (nat->allowed_ext_ips || nat->exempted_ext_ips) { lrouter_nat_add_ext_ip_match(od, lflows, match, nat, - is_v6, true, cidr_bits); + is_v6, true, cidr_bits, + lflow_ref); } if (stateless) { @@ -14297,7 +14591,7 @@ build_lrouter_in_dnat_flow(struct lflow_table *lflows, ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_DNAT, 100, ds_cstr(match), ds_cstr(actions), - &nat->header_); + &nat->header_, lflow_ref); } static void @@ -14306,7 +14600,8 @@ build_lrouter_out_undnat_flow(struct lflow_table *lflows, const struct nbrec_nat *nat, struct ds *match, struct ds *actions, bool distributed_nat, struct eth_addr mac, bool is_v6, - struct ovn_port *l3dgw_port, bool stateless) + struct ovn_port *l3dgw_port, bool stateless, + struct lflow_ref *lflow_ref) { /* Egress UNDNAT table: It is for already established connections' * reverse traffic. i.e., DNAT has already been done in ingress @@ -14348,7 +14643,7 @@ build_lrouter_out_undnat_flow(struct lflow_table *lflows, ovn_lflow_add_with_hint(lflows, od, S_ROUTER_OUT_UNDNAT, 100, ds_cstr(match), ds_cstr(actions), - &nat->header_); + &nat->header_, lflow_ref); } static void @@ -14356,7 +14651,8 @@ build_lrouter_out_is_dnat_local(struct lflow_table *lflows, const struct ovn_datapath *od, const struct nbrec_nat *nat, struct ds *match, struct ds *actions, bool distributed_nat, - bool is_v6, struct ovn_port *l3dgw_port) + bool is_v6, struct ovn_port *l3dgw_port, + struct lflow_ref *lflow_ref) { /* Note that this only applies for NAT on a distributed router. */ @@ -14379,7 +14675,7 @@ build_lrouter_out_is_dnat_local(struct lflow_table *lflows, ovn_lflow_add_with_hint(lflows, od, S_ROUTER_OUT_CHECK_DNAT_LOCAL, 50, ds_cstr(match), ds_cstr(actions), - &nat->header_); + &nat->header_, lflow_ref); } static void @@ -14387,7 +14683,8 @@ build_lrouter_out_snat_match(struct lflow_table *lflows, const struct ovn_datapath *od, const struct nbrec_nat *nat, struct ds *match, bool distributed_nat, int cidr_bits, bool is_v6, - struct ovn_port *l3dgw_port) + struct ovn_port *l3dgw_port, + struct lflow_ref *lflow_ref) { ds_clear(match); @@ -14407,7 +14704,8 @@ build_lrouter_out_snat_match(struct lflow_table *lflows, if (nat->allowed_ext_ips || nat->exempted_ext_ips) { lrouter_nat_add_ext_ip_match(od, lflows, match, nat, - is_v6, false, cidr_bits); + is_v6, false, cidr_bits, + lflow_ref); } } @@ -14418,7 +14716,8 @@ build_lrouter_out_snat_stateless_flow(struct lflow_table *lflows, struct ds *match, struct ds *actions, bool distributed_nat, struct eth_addr mac, int cidr_bits, - bool is_v6, struct ovn_port *l3dgw_port) + bool is_v6, struct ovn_port *l3dgw_port, + struct lflow_ref *lflow_ref) { if (strcmp(nat->type, "snat") && strcmp(nat->type, "dnat_and_snat")) { return; @@ -14432,7 +14731,7 @@ build_lrouter_out_snat_stateless_flow(struct lflow_table *lflows, uint16_t priority = cidr_bits + 1; build_lrouter_out_snat_match(lflows, od, nat, match, distributed_nat, - cidr_bits, is_v6, l3dgw_port); + cidr_bits, is_v6, l3dgw_port, lflow_ref); if (!od->is_gw_router) { /* Distributed router. */ @@ -14451,7 +14750,8 @@ build_lrouter_out_snat_stateless_flow(struct lflow_table *lflows, ovn_lflow_add_with_hint(lflows, od, S_ROUTER_OUT_SNAT, priority, ds_cstr(match), - ds_cstr(actions), &nat->header_); + ds_cstr(actions), &nat->header_, + lflow_ref); } static void @@ -14461,7 +14761,8 @@ build_lrouter_out_snat_in_czone_flow(struct lflow_table *lflows, struct ds *match, struct ds *actions, bool distributed_nat, struct eth_addr mac, int cidr_bits, - bool is_v6, struct ovn_port *l3dgw_port) + bool is_v6, struct ovn_port *l3dgw_port, + struct lflow_ref *lflow_ref) { if (strcmp(nat->type, "snat") && strcmp(nat->type, "dnat_and_snat")) { return; @@ -14476,7 +14777,8 @@ build_lrouter_out_snat_in_czone_flow(struct lflow_table *lflows, struct ds zone_actions = DS_EMPTY_INITIALIZER; build_lrouter_out_snat_match(lflows, od, nat, match, distributed_nat, - cidr_bits, is_v6, l3dgw_port); + cidr_bits, is_v6, l3dgw_port, + lflow_ref); if (od->n_l3dgw_ports) { priority += 128; @@ -14505,13 +14807,15 @@ build_lrouter_out_snat_in_czone_flow(struct lflow_table *lflows, ovn_lflow_add_with_hint(lflows, od, S_ROUTER_OUT_SNAT, priority, ds_cstr(match), - ds_cstr(actions), &nat->header_); + ds_cstr(actions), &nat->header_, + lflow_ref); ds_put_cstr(match, " && "REGBIT_DST_NAT_IP_LOCAL" == 1"); ovn_lflow_add_with_hint(lflows, od, S_ROUTER_OUT_SNAT, priority + 1, ds_cstr(match), - ds_cstr(&zone_actions), &nat->header_); + ds_cstr(&zone_actions), &nat->header_, + lflow_ref); ds_destroy(&zone_actions); } @@ -14522,7 +14826,8 @@ build_lrouter_out_snat_flow(struct lflow_table *lflows, const struct nbrec_nat *nat, struct ds *match, struct ds *actions, bool distributed_nat, struct eth_addr mac, int cidr_bits, bool is_v6, - struct ovn_port *l3dgw_port) + struct ovn_port *l3dgw_port, + struct lflow_ref *lflow_ref) { if (strcmp(nat->type, "snat") && strcmp(nat->type, "dnat_and_snat")) { return; @@ -14536,7 +14841,7 @@ build_lrouter_out_snat_flow(struct lflow_table *lflows, uint16_t priority = cidr_bits + 1; build_lrouter_out_snat_match(lflows, od, nat, match, distributed_nat, - cidr_bits, is_v6, l3dgw_port); + cidr_bits, is_v6, l3dgw_port, lflow_ref); if (!od->is_gw_router) { /* Distributed router. */ @@ -14559,7 +14864,8 @@ build_lrouter_out_snat_flow(struct lflow_table *lflows, ovn_lflow_add_with_hint(lflows, od, S_ROUTER_OUT_SNAT, priority, ds_cstr(match), - ds_cstr(actions), &nat->header_); + ds_cstr(actions), &nat->header_, + lflow_ref); } static void @@ -14569,7 +14875,8 @@ build_lrouter_ingress_nat_check_pkt_len(struct lflow_table *lflows, bool is_v6, struct ds *match, struct ds *actions, int mtu, struct ovn_port *l3dgw_port, - const struct shash *meter_groups) + const struct shash *meter_groups, + struct lflow_ref *lflow_ref) { ds_clear(match); ds_put_format(match, "inport == %s && "REGBIT_PKT_LARGER @@ -14603,7 +14910,8 @@ build_lrouter_ingress_nat_check_pkt_len(struct lflow_table *lflows, COPP_ICMP4_ERR, od->nbr->copp, meter_groups), - &nat->header_); + &nat->header_, + lflow_ref); } else { ds_put_format(match, " && ip6 && ip6.dst == %s", nat->external_ip); /* Set icmp6.frag_mtu to gw_mtu */ @@ -14630,7 +14938,8 @@ build_lrouter_ingress_nat_check_pkt_len(struct lflow_table *lflows, COPP_ICMP6_ERR, od->nbr->copp, meter_groups), - &nat->header_); + &nat->header_, + lflow_ref); } } @@ -14641,7 +14950,8 @@ build_lrouter_ingress_flow(struct lflow_table *lflows, struct ds *actions, struct eth_addr mac, bool distributed_nat, bool is_v6, struct ovn_port *l3dgw_port, - const struct shash *meter_groups) + const struct shash *meter_groups, + struct lflow_ref *lflow_ref) { if (od->n_l3dgw_ports && !strcmp(nat->type, "snat")) { ds_clear(match); @@ -14651,7 +14961,7 @@ build_lrouter_ingress_flow(struct lflow_table *lflows, is_v6 ? "ip6.src" : "ip4.src", nat->external_ip); ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_IP_INPUT, 120, ds_cstr(match), "next;", - &nat->header_); + &nat->header_, lflow_ref); } /* Logical router ingress table 0: * For NAT on a distributed router, add rules allowing @@ -14675,12 +14985,14 @@ build_lrouter_ingress_flow(struct lflow_table *lflows, build_gateway_mtu_flow(lflows, l3dgw_port, S_ROUTER_IN_ADMISSION, 50, 55, match, actions, &nat->header_, + lflow_ref, REG_INPORT_ETH_ADDR " = %s; next;", l3dgw_port->lrp_networks.ea_s); if (gw_mtu) { build_lrouter_ingress_nat_check_pkt_len(lflows, nat, od, is_v6, match, actions, gw_mtu, - l3dgw_port, meter_groups); + l3dgw_port, meter_groups, + lflow_ref); } } } @@ -14815,27 +15127,33 @@ lrouter_check_nat_entry(const struct ovn_datapath *od, /* NAT, Defrag and load balancing. */ static void build_lr_nat_defrag_and_lb_default_flows(struct ovn_datapath *od, - struct lflow_table *lflows) + struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { ovs_assert(od->nbr); /* Packets are allowed by default. */ - ovn_lflow_add(lflows, od, S_ROUTER_IN_DEFRAG, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_ROUTER_IN_UNSNAT, 0, "1", "next;"); + ovn_lflow_add(lflows, od, S_ROUTER_IN_DEFRAG, 0, "1", "next;", lflow_ref); + ovn_lflow_add(lflows, od, S_ROUTER_IN_UNSNAT, 0, "1", "next;", lflow_ref); ovn_lflow_add(lflows, od, S_ROUTER_OUT_CHECK_DNAT_LOCAL, 0, "1", - REGBIT_DST_NAT_IP_LOCAL" = 0; next;"); - ovn_lflow_add(lflows, od, S_ROUTER_OUT_SNAT, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_ROUTER_IN_DNAT, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_ROUTER_OUT_UNDNAT, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_ROUTER_OUT_POST_UNDNAT, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_ROUTER_OUT_POST_SNAT, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_ROUTER_OUT_EGR_LOOP, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_ROUTER_IN_ECMP_STATEFUL, 0, "1", "next;"); + REGBIT_DST_NAT_IP_LOCAL" = 0; next;", lflow_ref); + ovn_lflow_add(lflows, od, S_ROUTER_OUT_SNAT, 0, "1", "next;", lflow_ref); + ovn_lflow_add(lflows, od, S_ROUTER_IN_DNAT, 0, "1", "next;", lflow_ref); + ovn_lflow_add(lflows, od, S_ROUTER_OUT_UNDNAT, 0, "1", "next;", lflow_ref); + ovn_lflow_add(lflows, od, S_ROUTER_OUT_POST_UNDNAT, 0, "1", "next;", + lflow_ref); + ovn_lflow_add(lflows, od, S_ROUTER_OUT_POST_SNAT, 0, "1", "next;", + lflow_ref); + ovn_lflow_add(lflows, od, S_ROUTER_OUT_EGR_LOOP, 0, "1", "next;", + lflow_ref); + ovn_lflow_add(lflows, od, S_ROUTER_IN_ECMP_STATEFUL, 0, "1", "next;", + lflow_ref); /* Send the IPv6 NS packets to next table. When ovn-controller * generates IPv6 NS (for the action - nd_ns{}), the injected * packet would go through conntrack - which is not required. */ - ovn_lflow_add(lflows, od, S_ROUTER_OUT_SNAT, 120, "nd_ns", "next;"); + ovn_lflow_add(lflows, od, S_ROUTER_OUT_SNAT, 120, "nd_ns", "next;", + lflow_ref); } static void @@ -14845,7 +15163,8 @@ build_lrouter_nat_defrag_and_lb( const struct hmap *ls_ports, const struct hmap *lr_ports, struct ds *match, struct ds *actions, const struct shash *meter_groups, - const struct chassis_features *features) + const struct chassis_features *features, + struct lflow_ref *lflow_ref) { const struct ovn_datapath *od = lr_sful_rec->od; ovs_assert(od->nbr); @@ -14870,16 +15189,18 @@ build_lrouter_nat_defrag_and_lb( ds_put_format(match, " && %s.skip_snat == 1", ct_flag_reg); ovn_lflow_add(lflows, od, S_ROUTER_IN_DNAT, 70, ds_cstr(match), - "flags.skip_snat_for_lb = 1; ct_commit_nat;"); + "flags.skip_snat_for_lb = 1; ct_commit_nat;", + lflow_ref); ds_truncate(match, match_len); ds_put_format(match, " && %s.force_snat == 1", ct_flag_reg); ovn_lflow_add(lflows, od, S_ROUTER_IN_DNAT, 70, ds_cstr(match), - "flags.force_snat_for_lb = 1; ct_commit_nat;"); + "flags.force_snat_for_lb = 1; ct_commit_nat;", + lflow_ref); ds_truncate(match, match_len); ovn_lflow_add(lflows, od, S_ROUTER_IN_DNAT, 50, ds_cstr(match), - "ct_commit_nat;"); + "ct_commit_nat;", lflow_ref); } /* Ingress DNAT (Priority 50/70). @@ -14896,16 +15217,18 @@ build_lrouter_nat_defrag_and_lb( ds_put_format(match, " && %s.skip_snat == 1", ct_flag_reg); ovn_lflow_add(lflows, od, S_ROUTER_IN_DNAT, 70, ds_cstr(match), - "flags.skip_snat_for_lb = 1; next;"); + "flags.skip_snat_for_lb = 1; next;", + lflow_ref); ds_truncate(match, match_len); ds_put_format(match, " && %s.force_snat == 1", ct_flag_reg); ovn_lflow_add(lflows, od, S_ROUTER_IN_DNAT, 70, ds_cstr(match), - "flags.force_snat_for_lb = 1; next;"); + "flags.force_snat_for_lb = 1; next;", + lflow_ref); ds_truncate(match, match_len); ovn_lflow_add(lflows, od, S_ROUTER_IN_DNAT, 50, ds_cstr(match), - "next;"); + "next;", lflow_ref); } /* If the router has load balancer or DNAT rules, re-circulate every packet @@ -14920,11 +15243,14 @@ build_lrouter_nat_defrag_and_lb( if (od->is_gw_router && (od->nbr->n_nat || lr_sful_rec->has_lb_vip)) { /* Do not send ND or ICMP packets to connection tracking. */ ovn_lflow_add(lflows, od, S_ROUTER_OUT_UNDNAT, 100, - "nd || nd_rs || nd_ra", "next;"); + "nd || nd_rs || nd_ra", "next;", + lflow_ref); ovn_lflow_add(lflows, od, S_ROUTER_OUT_UNDNAT, 50, - "ip", "flags.loopback = 1; ct_dnat;"); + "ip", "flags.loopback = 1; ct_dnat;", + lflow_ref); ovn_lflow_add(lflows, od, S_ROUTER_OUT_POST_UNDNAT, 50, - "ip && ct.new", "ct_commit { } ; next; "); + "ip && ct.new", "ct_commit { } ; next; ", + lflow_ref); } /* NAT rules are only valid on Gateway routers and routers with @@ -14973,19 +15299,20 @@ build_lrouter_nat_defrag_and_lb( if (stateless) { build_lrouter_in_unsnat_stateless_flow(lflows, od, nat, match, distributed_nat, is_v6, - l3dgw_port); + l3dgw_port, lflow_ref); } else if (lrouter_use_common_zone(od)) { build_lrouter_in_unsnat_in_czone_flow(lflows, od, nat, match, distributed_nat, is_v6, - l3dgw_port); + l3dgw_port, lflow_ref); } else { build_lrouter_in_unsnat_flow(lflows, od, nat, match, - distributed_nat, is_v6, l3dgw_port); + distributed_nat, is_v6, l3dgw_port, + lflow_ref); } /* S_ROUTER_IN_DNAT */ build_lrouter_in_dnat_flow(lflows, od, lrnat_rec, nat, match, actions, distributed_nat, cidr_bits, is_v6, - l3dgw_port, stateless); + l3dgw_port, stateless, lflow_ref); /* ARP resolve for NAT IPs. */ if (od->is_gw_router) { @@ -15008,7 +15335,8 @@ build_lrouter_nat_defrag_and_lb( S_ROUTER_IN_ARP_RESOLVE, 150, ds_cstr(match), debug_drop_action(), - &nat->header_); + &nat->header_, + lflow_ref); /* Now for packets coming from other (downlink) LRPs, allow ARP * resolve for the NAT IP, so that such packets can be * forwarded for E/W NAT. */ @@ -15027,7 +15355,8 @@ build_lrouter_nat_defrag_and_lb( S_ROUTER_IN_ARP_RESOLVE, 100, ds_cstr(match), ds_cstr(actions), - &nat->header_); + &nat->header_, + lflow_ref); if (od->redirect_bridged && distributed_nat) { ds_clear(match); ds_put_format( @@ -15048,7 +15377,8 @@ build_lrouter_nat_defrag_and_lb( ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_ARP_RESOLVE, 90, ds_cstr(match), ds_cstr(actions), - &nat->header_); + &nat->header_, + lflow_ref); } sset_add(&nat_entries, nat->external_ip); } @@ -15058,13 +15388,13 @@ build_lrouter_nat_defrag_and_lb( /* S_ROUTER_OUT_DNAT_LOCAL */ build_lrouter_out_is_dnat_local(lflows, od, nat, match, actions, distributed_nat, is_v6, - l3dgw_port); + l3dgw_port, lflow_ref); } /* S_ROUTER_OUT_UNDNAT */ build_lrouter_out_undnat_flow(lflows, od, nat, match, actions, distributed_nat, mac, is_v6, l3dgw_port, - stateless); + stateless, lflow_ref); /* S_ROUTER_OUT_SNAT * Egress SNAT table: Packets enter the egress pipeline with * source ip address that needs to be SNATted to a external ip @@ -15073,21 +15403,22 @@ build_lrouter_nat_defrag_and_lb( build_lrouter_out_snat_stateless_flow(lflows, od, nat, match, actions, distributed_nat, mac, cidr_bits, is_v6, - l3dgw_port); + l3dgw_port, lflow_ref); } else if (lrouter_use_common_zone(od)) { build_lrouter_out_snat_in_czone_flow(lflows, od, nat, match, actions, distributed_nat, mac, - cidr_bits, is_v6, l3dgw_port); + cidr_bits, is_v6, l3dgw_port, + lflow_ref); } else { build_lrouter_out_snat_flow(lflows, od, nat, match, actions, distributed_nat, mac, cidr_bits, is_v6, - l3dgw_port); + l3dgw_port, lflow_ref); } /* S_ROUTER_IN_ADMISSION - S_ROUTER_IN_IP_INPUT */ build_lrouter_ingress_flow(lflows, od, nat, match, actions, mac, distributed_nat, is_v6, l3dgw_port, - meter_groups); + meter_groups, lflow_ref); /* Ingress Gateway Redirect Table: For NAT on a distributed * router, add flows that are specific to a NAT rule. These @@ -15114,7 +15445,8 @@ build_lrouter_nat_defrag_and_lb( if (op && op->nbsp && !strcmp(op->nbsp->type, "virtual")) { ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_GW_REDIRECT, 80, ds_cstr(match), - debug_drop_action(), &nat->header_); + debug_drop_action(), &nat->header_, + lflow_ref); } ds_put_format(match, " && is_chassis_resident(\"%s\")", nat->logical_port); @@ -15124,7 +15456,8 @@ build_lrouter_nat_defrag_and_lb( nat->external_ip); ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_GW_REDIRECT, 100, ds_cstr(match), - ds_cstr(actions), &nat->header_); + ds_cstr(actions), &nat->header_, + lflow_ref); } /* Egress Loopback table: For NAT on a distributed router. @@ -15165,7 +15498,7 @@ build_lrouter_nat_defrag_and_lb( ovn_stage_get_table(S_ROUTER_IN_ADMISSION)); ovn_lflow_add_with_hint(lflows, od, S_ROUTER_OUT_EGR_LOOP, 100, ds_cstr(match), ds_cstr(actions), - &nat->header_); + &nat->header_, lflow_ref); } } @@ -15185,7 +15518,7 @@ build_lrouter_nat_defrag_and_lb( ds_put_cstr(actions, REGBIT_DST_NAT_IP_LOCAL" = 1; next;"); ovn_lflow_add_with_hint(lflows, od, S_ROUTER_OUT_CHECK_DNAT_LOCAL, 50, ds_cstr(match), ds_cstr(actions), - &od->nbr->header_); + &od->nbr->header_, lflow_ref); } @@ -15195,22 +15528,24 @@ build_lrouter_nat_defrag_and_lb( if (lrnat_rec->dnat_force_snat_addrs.n_ipv4_addrs) { build_lrouter_force_snat_flows(lflows, od, "4", lrnat_rec->dnat_force_snat_addrs.ipv4_addrs[0].addr_s, - "dnat"); + "dnat", lflow_ref); } if (lrnat_rec->dnat_force_snat_addrs.n_ipv6_addrs) { build_lrouter_force_snat_flows(lflows, od, "6", lrnat_rec->dnat_force_snat_addrs.ipv6_addrs[0].addr_s, - "dnat"); + "dnat", lflow_ref); } } if (lb_force_snat_ip) { if (lrnat_rec->lb_force_snat_addrs.n_ipv4_addrs) { build_lrouter_force_snat_flows(lflows, od, "4", - lrnat_rec->lb_force_snat_addrs.ipv4_addrs[0].addr_s, "lb"); + lrnat_rec->lb_force_snat_addrs.ipv4_addrs[0].addr_s, "lb", + lflow_ref); } if (lrnat_rec->lb_force_snat_addrs.n_ipv6_addrs) { build_lrouter_force_snat_flows(lflows, od, "6", - lrnat_rec->lb_force_snat_addrs.ipv6_addrs[0].addr_s, "lb"); + lrnat_rec->lb_force_snat_addrs.ipv6_addrs[0].addr_s, "lb", + lflow_ref); } } } @@ -15248,7 +15583,8 @@ build_lbnat_lflows_iterate_by_lsp(struct ovn_port *op, const struct hmap *lr_ports, struct ds *match, struct ds *actions, - struct lflow_table *lflows) + struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { ovs_assert(op->nbsp); @@ -15263,7 +15599,7 @@ build_lbnat_lflows_iterate_by_lsp(struct ovn_port *op, build_lsp_lflows_for_lbnats(op, op->peer, lr_sful_rec, lr_sful_table, lr_ports, lflows, - match, actions, op->stateful_lflow_ref); + match, actions, lflow_ref); } static void @@ -15271,7 +15607,8 @@ build_lrp_lflows_for_lbnats(struct ovn_port *op, const struct lr_stateful_record *lr_sful_rec, const struct shash *meter_groups, struct ds *match, struct ds *actions, - struct lflow_table *lflows) + struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { /* Drop IP traffic destined to router owned IPs except if the IP is * also a SNAT IP. Those are dropped later, in stage @@ -15284,7 +15621,8 @@ build_lrp_lflows_for_lbnats(struct ovn_port *op, */ if (!lr_sful_rec->lrnat_rec->lb_force_snat_router_ip) { build_lrouter_drop_own_dest(op, lr_sful_rec, - S_ROUTER_IN_IP_INPUT, 60, false, lflows); + S_ROUTER_IN_IP_INPUT, 60, false, + lflows, lflow_ref); } /* Drop IP traffic destined to router owned IPs. Part of it is dropped @@ -15294,12 +15632,14 @@ build_lrp_lflows_for_lbnats(struct ovn_port *op, * Priority 2. */ build_lrouter_drop_own_dest(op, lr_sful_rec, - S_ROUTER_IN_ARP_RESOLVE, 2, true, lflows); + S_ROUTER_IN_ARP_RESOLVE, 2, true, + lflows, lflow_ref); build_lrouter_ipv4_ip_input_for_lbnats(op, lflows, lr_sful_rec, - match, meter_groups); + match, meter_groups, + lflow_ref); build_lrouter_force_snat_flows_op(op, lr_sful_rec->lrnat_rec, lflows, - match, actions); + match, actions, lflow_ref); } static void @@ -15308,7 +15648,8 @@ build_lbnat_lflows_iterate_by_lrp(struct ovn_port *op, const struct shash *meter_groups, struct ds *match, struct ds *actions, - struct lflow_table *lflows) + struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { ovs_assert(op->nbrp); @@ -15318,7 +15659,7 @@ build_lbnat_lflows_iterate_by_lrp(struct ovn_port *op, ovs_assert(lr_sful_rec); build_lrp_lflows_for_lbnats(op, lr_sful_rec, meter_groups, match, - actions, lflows); + actions, lflows, lflow_ref); } static void @@ -15332,13 +15673,16 @@ build_lr_stateful_flows(const struct lr_stateful_record *lr_sful_rec, const struct chassis_features *features) { build_lrouter_nat_defrag_and_lb(lr_sful_rec, lflows, ls_ports, lr_ports, - match, actions, meter_groups, features); + match, actions, meter_groups, features, + NULL); build_lr_gateway_redirect_flows_for_nats(lr_sful_rec->od, lr_sful_rec->lrnat_rec, lflows, - match, actions); + match, actions, + NULL); build_lrouter_arp_nd_for_datapath(lr_sful_rec->od, lr_sful_rec->lrnat_rec, lflows, - meter_groups); + meter_groups, + NULL); } static void @@ -15350,11 +15694,13 @@ build_ls_stateful_flows(const struct ls_stateful_record *ls_stateful_rec, { ovs_assert(ls_stateful_rec->od); - build_ls_stateful_rec_pre_acls(ls_stateful_rec, ls_pgs, lflows); - build_ls_stateful_rec_pre_lb(ls_stateful_rec, lflows); - build_acl_hints(ls_stateful_rec, features, lflows); - build_acls(ls_stateful_rec, features, lflows, ls_pgs, meter_groups); - build_lb_hairpin(ls_stateful_rec, lflows); + build_ls_stateful_rec_pre_acls(ls_stateful_rec, ls_pgs, lflows, + NULL); + build_ls_stateful_rec_pre_lb(ls_stateful_rec, lflows, NULL); + build_acl_hints(ls_stateful_rec, features, lflows, NULL); + build_acls(ls_stateful_rec, features, lflows, ls_pgs, meter_groups, + NULL); + build_lb_hairpin(ls_stateful_rec, lflows, NULL); } struct lswitch_flow_build_info { @@ -15390,19 +15736,20 @@ build_lswitch_and_lrouter_iterate_by_ls(struct ovn_datapath *od, { ovs_assert(od->nbs); build_lswitch_lflows_pre_acl_and_acl(od, lsi->features, lsi->lflows, - lsi->meter_groups); - - build_fwd_group_lflows(od, lsi->lflows); - build_lswitch_lflows_admission_control(od, lsi->lflows); - build_lswitch_learn_fdb_od(od, lsi->lflows); - build_lswitch_arp_nd_responder_default(od, lsi->lflows); - build_lswitch_dns_lookup_and_response(od, lsi->lflows, lsi->meter_groups); - build_lswitch_dhcp_and_dns_defaults(od, lsi->lflows); + lsi->meter_groups, NULL); + + build_fwd_group_lflows(od, lsi->lflows, NULL); + build_lswitch_lflows_admission_control(od, lsi->lflows, NULL); + build_lswitch_learn_fdb_od(od, lsi->lflows, NULL); + build_lswitch_arp_nd_responder_default(od, lsi->lflows, NULL); + build_lswitch_dns_lookup_and_response(od, lsi->lflows, lsi->meter_groups, + NULL); + build_lswitch_dhcp_and_dns_defaults(od, lsi->lflows, NULL); build_lswitch_destination_lookup_bmcast(od, lsi->lflows, &lsi->actions, - lsi->meter_groups); - build_lswitch_output_port_sec_od(od, lsi->lflows); - build_lswitch_lb_affinity_default_flows(od, lsi->lflows); - build_lswitch_lflows_l2_unknown(od, lsi->lflows); + lsi->meter_groups, NULL); + build_lswitch_output_port_sec_od(od, lsi->lflows, NULL); + build_lswitch_lb_affinity_default_flows(od, lsi->lflows, NULL); + build_lswitch_lflows_l2_unknown(od, lsi->lflows, NULL); } /* Helper function to combine all lflow generation which is iterated by @@ -15413,29 +15760,34 @@ build_lswitch_and_lrouter_iterate_by_lr(struct ovn_datapath *od, struct lswitch_flow_build_info *lsi) { ovs_assert(od->nbr); - build_adm_ctrl_flows_for_lrouter(od, lsi->lflows); + build_adm_ctrl_flows_for_lrouter(od, lsi->lflows, NULL); build_neigh_learning_flows_for_lrouter(od, lsi->lflows, &lsi->match, - &lsi->actions, lsi->meter_groups); - build_ND_RA_flows_for_lrouter(od, lsi->lflows); - build_ip_routing_pre_flows_for_lrouter(od, lsi->lflows); + &lsi->actions, + lsi->meter_groups, NULL); + build_ND_RA_flows_for_lrouter(od, lsi->lflows, NULL); + build_ip_routing_pre_flows_for_lrouter(od, lsi->lflows, NULL); build_static_route_flows_for_lrouter(od, lsi->features, lsi->lflows, lsi->lr_ports, - lsi->bfd_connections); + lsi->bfd_connections, + NULL); build_mcast_lookup_flows_for_lrouter(od, lsi->lflows, &lsi->match, - &lsi->actions); - build_ingress_policy_flows_for_lrouter(od, lsi->lflows, lsi->lr_ports); - build_arp_resolve_flows_for_lrouter(od, lsi->lflows); + &lsi->actions, NULL); + build_ingress_policy_flows_for_lrouter(od, lsi->lflows, lsi->lr_ports, + NULL); + build_arp_resolve_flows_for_lrouter(od, lsi->lflows, NULL); build_check_pkt_len_flows_for_lrouter(od, lsi->lflows, lsi->lr_ports, &lsi->match, &lsi->actions, - lsi->meter_groups); + lsi->meter_groups, NULL); build_gateway_redirect_flows_for_lrouter(od, lsi->lflows, &lsi->match, - &lsi->actions); + &lsi->actions, NULL); build_arp_request_flows_for_lrouter(od, lsi->lflows, &lsi->match, - &lsi->actions, lsi->meter_groups); - build_misc_local_traffic_drop_flows_for_lrouter(od, lsi->lflows); + &lsi->actions, + lsi->meter_groups, + NULL); + build_misc_local_traffic_drop_flows_for_lrouter(od, lsi->lflows, NULL); - build_lr_nat_defrag_and_lb_default_flows(od, lsi->lflows); - build_lrouter_lb_affinity_default_flows(od, lsi->lflows); + build_lr_nat_defrag_and_lb_default_flows(od, lsi->lflows, NULL); + build_lrouter_lb_affinity_default_flows(od, lsi->lflows, NULL); } /* Helper function to combine all lflow generation which is iterated by logical @@ -15477,22 +15829,26 @@ build_lswitch_and_lrouter_iterate_by_lrp(struct ovn_port *op, ovs_assert(op->nbrp); build_adm_ctrl_flows_for_lrouter_port(op, lsi->lflows, &lsi->match, - &lsi->actions); + &lsi->actions, op->lflow_ref); build_neigh_learning_flows_for_lrouter_port(op, lsi->lflows, &lsi->match, - &lsi->actions); - build_ip_routing_flows_for_lrp(op, lsi->lflows); + &lsi->actions, op->lflow_ref); + build_ip_routing_flows_for_lrp(op, lsi->lflows, op->lflow_ref); build_ND_RA_flows_for_lrouter_port(op, lsi->lflows, &lsi->match, - &lsi->actions, lsi->meter_groups); + &lsi->actions, lsi->meter_groups, + op->lflow_ref); build_arp_resolve_flows_for_lrp(op, lsi->lflows, - &lsi->match, &lsi->actions); + &lsi->match, &lsi->actions, op->lflow_ref); build_egress_delivery_flows_for_lrouter_port(op, lsi->lflows, &lsi->match, - &lsi->actions); - build_dhcpv6_reply_flows_for_lrouter_port(op, lsi->lflows, &lsi->match); + &lsi->actions, + op->lflow_ref); + build_dhcpv6_reply_flows_for_lrouter_port(op, lsi->lflows, &lsi->match, + op->lflow_ref); build_ipv6_input_flows_for_lrouter_port(op, lsi->lflows, &lsi->match, &lsi->actions, - lsi->meter_groups); + lsi->meter_groups, + op->lflow_ref); build_lrouter_ipv4_ip_input(op, lsi->lflows, &lsi->match, &lsi->actions, - lsi->meter_groups); + lsi->meter_groups, op->lflow_ref); } static void * @@ -15560,7 +15916,8 @@ build_lflows_thread(void *arg) lsi->lr_ports, &lsi->match, &lsi->actions, - lsi->lflows); + lsi->lflows, + op->stateful_lflow_ref); } } for (bnum = control->id; @@ -15577,7 +15934,8 @@ build_lflows_thread(void *arg) lsi->meter_groups, &lsi->match, &lsi->actions, - lsi->lflows); + lsi->lflows, + op->stateful_lflow_ref); } } for (bnum = control->id; @@ -15593,23 +15951,27 @@ build_lflows_thread(void *arg) lsi->ls_ports, lsi->lflows, &lsi->match, - &lsi->actions); + &lsi->actions, + NULL); build_lrouter_defrag_flows_for_lb(lb_dps, lsi->lflows, lsi->lr_datapaths, - &lsi->match); + &lsi->match, + NULL); build_lrouter_flows_for_lb(lb_dps, lsi->lflows, lsi->meter_groups, lsi->lr_datapaths, lsi->lr_sful_table, lsi->features, lsi->svc_monitor_map, - &lsi->match, &lsi->actions); + &lsi->match, &lsi->actions, + NULL); build_lswitch_flows_for_lb(lb_dps, lsi->lflows, lsi->meter_groups, lsi->ls_datapaths, lsi->features, lsi->svc_monitor_map, - &lsi->match, &lsi->actions); + &lsi->match, &lsi->actions, + NULL); } } for (bnum = control->id; @@ -15654,7 +16016,7 @@ build_lflows_thread(void *arg) } build_lswitch_ip_mcast_igmp_mld(igmp_group, lsi->lflows, &lsi->match, - &lsi->actions); + &lsi->actions, NULL); } } } @@ -15807,7 +16169,8 @@ build_lswitch_and_lrouter_flows(const struct ovn_datapaths *ls_datapaths, lsi.lflows); build_lbnat_lflows_iterate_by_lsp(op, lsi.lr_sful_table, lsi.lr_ports, &lsi.match, &lsi.actions, - lsi.lflows); + lsi.lflows, + op->stateful_lflow_ref); } HMAP_FOR_EACH (op, key_node, lr_ports) { build_lswitch_and_lrouter_iterate_by_lrp(op, &lsi); @@ -15815,24 +16178,29 @@ build_lswitch_and_lrouter_flows(const struct ovn_datapaths *ls_datapaths, lsi.meter_groups, &lsi.match, &lsi.actions, - lsi.lflows); + lsi.lflows, + op->stateful_lflow_ref); } stopwatch_stop(LFLOWS_PORTS_STOPWATCH_NAME, time_msec()); stopwatch_start(LFLOWS_LBS_STOPWATCH_NAME, time_msec()); HMAP_FOR_EACH (lb_dps, hmap_node, lb_dps_map) { build_lswitch_arp_nd_service_monitor(lb_dps->lb, lsi.ls_ports, lsi.lflows, &lsi.actions, - &lsi.match); + &lsi.match, + NULL); build_lrouter_defrag_flows_for_lb(lb_dps, lsi.lflows, - lsi.lr_datapaths, &lsi.match); + lsi.lr_datapaths, &lsi.match, + NULL); build_lrouter_flows_for_lb(lb_dps, lsi.lflows, lsi.meter_groups, lsi.lr_datapaths, lsi.lr_sful_table, lsi.features, lsi.svc_monitor_map, - &lsi.match, &lsi.actions); + &lsi.match, &lsi.actions, + NULL); build_lswitch_flows_for_lb(lb_dps, lsi.lflows, lsi.meter_groups, lsi.ls_datapaths, lsi.features, lsi.svc_monitor_map, - &lsi.match, &lsi.actions); + &lsi.match, &lsi.actions, + NULL); } stopwatch_stop(LFLOWS_LBS_STOPWATCH_NAME, time_msec()); @@ -15854,7 +16222,8 @@ build_lswitch_and_lrouter_flows(const struct ovn_datapaths *ls_datapaths, build_lswitch_ip_mcast_igmp_mld(igmp_group, lsi.lflows, &lsi.actions, - &lsi.match); + &lsi.match, + NULL); } stopwatch_stop(LFLOWS_IGMP_STOPWATCH_NAME, time_msec()); From patchwork Tue Nov 28 02:36:57 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Numan Siddique X-Patchwork-Id: 1869080 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.133; helo=smtp2.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4SfRSf169rz1yRy for ; Tue, 28 Nov 2023 13:37:38 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 3E72340A21; Tue, 28 Nov 2023 02:37:36 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 3E72340A21 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 46QGPylxuFxc; Tue, 28 Nov 2023 02:37:34 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp2.osuosl.org (Postfix) with ESMTPS id BC72D40A38; Tue, 28 Nov 2023 02:37:33 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org BC72D40A38 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 9496FC0072; Tue, 28 Nov 2023 02:37:33 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp4.osuosl.org (smtp4.osuosl.org [IPv6:2605:bc80:3010::137]) by lists.linuxfoundation.org (Postfix) with ESMTP id 1D7CEC0039 for ; Tue, 28 Nov 2023 02:37:33 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 3881541514 for ; Tue, 28 Nov 2023 02:37:14 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 3881541514 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id jUkl9uWljtir for ; Tue, 28 Nov 2023 02:37:13 +0000 (UTC) Received: from relay1-d.mail.gandi.net (relay1-d.mail.gandi.net [IPv6:2001:4b98:dc4:8::221]) by smtp4.osuosl.org (Postfix) with ESMTPS id 9A896414A2 for ; Tue, 28 Nov 2023 02:37:12 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 9A896414A2 Received: by mail.gandi.net (Postfix) with ESMTPSA id 31A2D240004; Tue, 28 Nov 2023 02:37:09 +0000 (UTC) From: numans@ovn.org To: dev@openvswitch.org Date: Mon, 27 Nov 2023 21:36:57 -0500 Message-ID: <20231128023657.569883-1-numans@ovn.org> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20231128023410.569539-1-numans@ovn.org> References: <20231128023410.569539-1-numans@ovn.org> MIME-Version: 1.0 X-GND-Sasl: numans@ovn.org Subject: [ovs-dev] [PATCH ovn v3 10/16] northd: Move ovn_lb_datapaths from lib to northd module. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Numan Siddique Signed-off-by: Numan Siddique --- lib/lb.c | 96 ----------------------------------------- lib/lb.h | 57 ------------------------- northd/northd.c | 111 ++++++++++++++++++++++++++++++++++++++++++++++++ northd/northd.h | 28 ++++++++++++ 4 files changed, 139 insertions(+), 153 deletions(-) diff --git a/lib/lb.c b/lib/lb.c index d0d562b6fb..991f20299d 100644 --- a/lib/lb.c +++ b/lib/lb.c @@ -1071,99 +1071,3 @@ remove_ips_from_lb_ip_set(struct ovn_lb_ip_set *lb_ips, } } } - -/* lb datapaths functions */ -struct ovn_lb_datapaths * -ovn_lb_datapaths_create(const struct ovn_northd_lb *lb, size_t n_ls_datapaths, - size_t n_lr_datapaths) -{ - struct ovn_lb_datapaths *lb_dps = xzalloc(sizeof *lb_dps); - lb_dps->lb = lb; - lb_dps->nb_ls_map = bitmap_allocate(n_ls_datapaths); - lb_dps->nb_lr_map = bitmap_allocate(n_lr_datapaths); - - return lb_dps; -} - -struct ovn_lb_datapaths * -ovn_lb_datapaths_find(const struct hmap *lb_dps_map, - const struct uuid *lb_uuid) -{ - struct ovn_lb_datapaths *lb_dps; - size_t hash = uuid_hash(lb_uuid); - HMAP_FOR_EACH_WITH_HASH (lb_dps, hmap_node, hash, lb_dps_map) { - if (uuid_equals(&lb_dps->lb->nlb->header_.uuid, lb_uuid)) { - return lb_dps; - } - } - return NULL; -} - -void -ovn_lb_datapaths_destroy(struct ovn_lb_datapaths *lb_dps) -{ - bitmap_free(lb_dps->nb_lr_map); - bitmap_free(lb_dps->nb_ls_map); - free(lb_dps); -} - -void -ovn_lb_datapaths_add_lr(struct ovn_lb_datapaths *lb_dps, size_t n, - struct ovn_datapath **ods) -{ - for (size_t i = 0; i < n; i++) { - if (!bitmap_is_set(lb_dps->nb_lr_map, ods[i]->index)) { - bitmap_set1(lb_dps->nb_lr_map, ods[i]->index); - lb_dps->n_nb_lr++; - } - } -} - -void -ovn_lb_datapaths_add_ls(struct ovn_lb_datapaths *lb_dps, size_t n, - struct ovn_datapath **ods) -{ - for (size_t i = 0; i < n; i++) { - if (!bitmap_is_set(lb_dps->nb_ls_map, ods[i]->index)) { - bitmap_set1(lb_dps->nb_ls_map, ods[i]->index); - lb_dps->n_nb_ls++; - } - } -} - -struct ovn_lb_group_datapaths * -ovn_lb_group_datapaths_create(const struct ovn_lb_group *lb_group, - size_t max_ls_datapaths, - size_t max_lr_datapaths) -{ - struct ovn_lb_group_datapaths *lb_group_dps = - xzalloc(sizeof *lb_group_dps); - lb_group_dps->lb_group = lb_group; - lb_group_dps->ls = xmalloc(max_ls_datapaths * sizeof *lb_group_dps->ls); - lb_group_dps->lr = xmalloc(max_lr_datapaths * sizeof *lb_group_dps->lr); - - return lb_group_dps; -} - -void -ovn_lb_group_datapaths_destroy(struct ovn_lb_group_datapaths *lb_group_dps) -{ - free(lb_group_dps->ls); - free(lb_group_dps->lr); - free(lb_group_dps); -} - -struct ovn_lb_group_datapaths * -ovn_lb_group_datapaths_find(const struct hmap *lb_group_dps_map, - const struct uuid *lb_group_uuid) -{ - struct ovn_lb_group_datapaths *lb_group_dps; - size_t hash = uuid_hash(lb_group_uuid); - - HMAP_FOR_EACH_WITH_HASH (lb_group_dps, hmap_node, hash, lb_group_dps_map) { - if (uuid_equals(&lb_group_dps->lb_group->uuid, lb_group_uuid)) { - return lb_group_dps; - } - } - return NULL; -} diff --git a/lib/lb.h b/lib/lb.h index b8e3c1e8fb..90ac39ee5a 100644 --- a/lib/lb.h +++ b/lib/lb.h @@ -167,63 +167,6 @@ void ovn_lb_group_reinit( const struct nbrec_load_balancer_group *, const struct hmap *lbs); -struct ovn_lb_datapaths { - struct hmap_node hmap_node; - - const struct ovn_northd_lb *lb; - size_t n_nb_ls; - unsigned long *nb_ls_map; - - size_t n_nb_lr; - unsigned long *nb_lr_map; -}; - -struct ovn_lb_datapaths *ovn_lb_datapaths_create(const struct ovn_northd_lb *, - size_t n_ls_datapaths, - size_t n_lr_datapaths); -struct ovn_lb_datapaths *ovn_lb_datapaths_find(const struct hmap *, - const struct uuid *); -void ovn_lb_datapaths_destroy(struct ovn_lb_datapaths *); -void ovn_lb_datapaths_add_lr(struct ovn_lb_datapaths *, size_t n, - struct ovn_datapath **); -void ovn_lb_datapaths_add_ls(struct ovn_lb_datapaths *, size_t n, - struct ovn_datapath **); - -struct ovn_lb_group_datapaths { - struct hmap_node hmap_node; - - const struct ovn_lb_group *lb_group; - - /* Datapaths to which 'lb_group' is applied. */ - size_t n_ls; - struct ovn_datapath **ls; - size_t n_lr; - struct ovn_datapath **lr; -}; - -struct ovn_lb_group_datapaths *ovn_lb_group_datapaths_create( - const struct ovn_lb_group *, size_t max_ls_datapaths, - size_t max_lr_datapaths); - -void ovn_lb_group_datapaths_destroy(struct ovn_lb_group_datapaths *); -struct ovn_lb_group_datapaths *ovn_lb_group_datapaths_find( - const struct hmap *lb_group_dps, const struct uuid *); - -static inline void -ovn_lb_group_datapaths_add_ls(struct ovn_lb_group_datapaths *lbg_dps, size_t n, - struct ovn_datapath **ods) -{ - memcpy(&lbg_dps->ls[lbg_dps->n_ls], ods, n * sizeof *ods); - lbg_dps->n_ls += n; -} - -static inline void -ovn_lb_group_datapaths_add_lr(struct ovn_lb_group_datapaths *lbg_dps, - struct ovn_datapath *lr) -{ - lbg_dps->lr[lbg_dps->n_lr++] = lr; -} - struct ovn_controller_lb { struct hmap_node hmap_node; diff --git a/northd/northd.c b/northd/northd.c index 7c5fe60e9a..104068e293 100644 --- a/northd/northd.c +++ b/northd/northd.c @@ -3514,6 +3514,117 @@ build_lb_vip_actions(const struct ovn_northd_lb *lb, return reject; } +/* lb datapaths functions */ +static struct ovn_lb_datapaths * +ovn_lb_datapaths_create(const struct ovn_northd_lb *lb, size_t n_ls_datapaths, + size_t n_lr_datapaths) +{ + struct ovn_lb_datapaths *lb_dps = xzalloc(sizeof *lb_dps); + lb_dps->lb = lb; + lb_dps->nb_ls_map = bitmap_allocate(n_ls_datapaths); + lb_dps->nb_lr_map = bitmap_allocate(n_lr_datapaths); + + return lb_dps; +} + +static void +ovn_lb_datapaths_destroy(struct ovn_lb_datapaths *lb_dps) +{ + bitmap_free(lb_dps->nb_lr_map); + bitmap_free(lb_dps->nb_ls_map); + free(lb_dps); +} + +static void +ovn_lb_datapaths_add_lr(struct ovn_lb_datapaths *lb_dps, size_t n, + struct ovn_datapath **ods) +{ + for (size_t i = 0; i < n; i++) { + if (!bitmap_is_set(lb_dps->nb_lr_map, ods[i]->index)) { + bitmap_set1(lb_dps->nb_lr_map, ods[i]->index); + lb_dps->n_nb_lr++; + } + } +} + +static void +ovn_lb_datapaths_add_ls(struct ovn_lb_datapaths *lb_dps, size_t n, + struct ovn_datapath **ods) +{ + for (size_t i = 0; i < n; i++) { + if (!bitmap_is_set(lb_dps->nb_ls_map, ods[i]->index)) { + bitmap_set1(lb_dps->nb_ls_map, ods[i]->index); + lb_dps->n_nb_ls++; + } + } +} + +static struct ovn_lb_group_datapaths * +ovn_lb_group_datapaths_create(const struct ovn_lb_group *lb_group, + size_t max_ls_datapaths, + size_t max_lr_datapaths) +{ + struct ovn_lb_group_datapaths *lb_group_dps = + xzalloc(sizeof *lb_group_dps); + lb_group_dps->lb_group = lb_group; + lb_group_dps->ls = xmalloc(max_ls_datapaths * sizeof *lb_group_dps->ls); + lb_group_dps->lr = xmalloc(max_lr_datapaths * sizeof *lb_group_dps->lr); + + return lb_group_dps; +} + +static void +ovn_lb_group_datapaths_destroy(struct ovn_lb_group_datapaths *lb_group_dps) +{ + free(lb_group_dps->ls); + free(lb_group_dps->lr); + free(lb_group_dps); +} + +static void +ovn_lb_group_datapaths_add_ls(struct ovn_lb_group_datapaths *lbg_dps, size_t n, + struct ovn_datapath **ods) +{ + memcpy(&lbg_dps->ls[lbg_dps->n_ls], ods, n * sizeof *ods); + lbg_dps->n_ls += n; +} + +static void +ovn_lb_group_datapaths_add_lr(struct ovn_lb_group_datapaths *lbg_dps, + struct ovn_datapath *lr) +{ + lbg_dps->lr[lbg_dps->n_lr++] = lr; +} + +struct ovn_lb_datapaths * +ovn_lb_datapaths_find(const struct hmap *lb_dps_map, + const struct uuid *lb_uuid) +{ + struct ovn_lb_datapaths *lb_dps; + size_t hash = uuid_hash(lb_uuid); + HMAP_FOR_EACH_WITH_HASH (lb_dps, hmap_node, hash, lb_dps_map) { + if (uuid_equals(&lb_dps->lb->nlb->header_.uuid, lb_uuid)) { + return lb_dps; + } + } + return NULL; +} + +struct ovn_lb_group_datapaths * +ovn_lb_group_datapaths_find(const struct hmap *lb_group_dps_map, + const struct uuid *lb_group_uuid) +{ + struct ovn_lb_group_datapaths *lb_group_dps; + size_t hash = uuid_hash(lb_group_uuid); + + HMAP_FOR_EACH_WITH_HASH (lb_group_dps, hmap_node, hash, lb_group_dps_map) { + if (uuid_equals(&lb_group_dps->lb_group->uuid, lb_group_uuid)) { + return lb_group_dps; + } + } + return NULL; +} + static void build_lb_datapaths(const struct hmap *lbs, const struct hmap *lb_groups, struct ovn_datapaths *ls_datapaths, diff --git a/northd/northd.h b/northd/northd.h index cce465b699..e2df53e6f8 100644 --- a/northd/northd.h +++ b/northd/northd.h @@ -92,6 +92,29 @@ ods_size(const struct ovn_datapaths *datapaths) bool od_has_lb_vip(const struct ovn_datapath *od); +struct ovn_lb_datapaths { + struct hmap_node hmap_node; + + const struct ovn_northd_lb *lb; + size_t n_nb_ls; + unsigned long *nb_ls_map; + + size_t n_nb_lr; + unsigned long *nb_lr_map; +}; + +struct ovn_lb_group_datapaths { + struct hmap_node hmap_node; + + const struct ovn_lb_group *lb_group; + + /* Datapaths to which 'lb_group' is applied. */ + size_t n_ls; + struct ovn_datapath **ls; + size_t n_lr; + struct ovn_datapath **lr; +}; + struct tracked_ovn_ports { /* tracked created ports. * hmapx node data is 'struct ovn_port *' */ @@ -106,6 +129,11 @@ struct tracked_ovn_ports { struct hmapx deleted; }; +struct ovn_lb_datapaths *ovn_lb_datapaths_find(const struct hmap *, + const struct uuid *); +struct ovn_lb_group_datapaths *ovn_lb_group_datapaths_find( + const struct hmap *lb_group_dps, const struct uuid *); + struct tracked_lbs { /* Tracked created or updated load balancers. * hmapx node data is 'struct ovn_lb_datapaths' */ From patchwork Tue Nov 28 02:37:11 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Numan Siddique X-Patchwork-Id: 1869084 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::138; helo=smtp1.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp1.osuosl.org (smtp1.osuosl.org [IPv6:2605:bc80:3010::138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4SfRTx4VGtz1yRy for ; Tue, 28 Nov 2023 13:38:45 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 3CD3B81CE7; Tue, 28 Nov 2023 02:38:43 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 3CD3B81CE7 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Bk_3v1R11NRY; Tue, 28 Nov 2023 02:38:38 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp1.osuosl.org (Postfix) with ESMTPS id C0DC2811BA; Tue, 28 Nov 2023 02:38:36 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org C0DC2811BA Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 7A201C0072; Tue, 28 Nov 2023 02:38:36 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by lists.linuxfoundation.org (Postfix) with ESMTP id ADAADC0072 for ; Tue, 28 Nov 2023 02:38:35 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id AB41161000 for ; Tue, 28 Nov 2023 02:37:26 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org AB41161000 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id nLNe-pH6awvz for ; Tue, 28 Nov 2023 02:37:24 +0000 (UTC) Received: from relay3-d.mail.gandi.net (relay3-d.mail.gandi.net [IPv6:2001:4b98:dc4:8::223]) by smtp3.osuosl.org (Postfix) with ESMTPS id E27A360FFA for ; Tue, 28 Nov 2023 02:37:23 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org E27A360FFA Received: by mail.gandi.net (Postfix) with ESMTPSA id A7DB660002; Tue, 28 Nov 2023 02:37:21 +0000 (UTC) From: numans@ovn.org To: dev@openvswitch.org Date: Mon, 27 Nov 2023 21:37:11 -0500 Message-ID: <20231128023711.569923-1-numans@ovn.org> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20231128023410.569539-1-numans@ovn.org> References: <20231128023410.569539-1-numans@ovn.org> MIME-Version: 1.0 X-GND-Sasl: numans@ovn.org Subject: [ovs-dev] [PATCH ovn v3 11/16] northd: Handle lb changes in lflow engine. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Numan Siddique Since northd tracked data has the changed lb data, northd engine handler for lflow engine now handles the lb changes incrementally. All the lflows generated for each lb is stored in the ovn_lb_datapaths->lflow_ref and this is used similar to how we handle ovn_port changes. Signed-off-by: Numan Siddique --- northd/en-lflow.c | 11 +++--- northd/lflow-mgr.c | 41 ++++++++++++++++++- northd/lflow-mgr.h | 3 ++ northd/northd.c | 95 +++++++++++++++++++++++++++++++++++++++++---- northd/northd.h | 28 +++++++++++++ tests/ovn-northd.at | 30 ++++++++++---- 6 files changed, 186 insertions(+), 22 deletions(-) diff --git a/northd/en-lflow.c b/northd/en-lflow.c index 65d2f45ebc..13284b5556 100644 --- a/northd/en-lflow.c +++ b/northd/en-lflow.c @@ -125,11 +125,6 @@ lflow_northd_handler(struct engine_node *node, return false; } - /* Fall back to recompute if load balancers have changed. */ - if (northd_has_lbs_in_tracked_data(&northd_data->trk_data)) { - return false; - } - const struct engine_context *eng_ctx = engine_get_context(); struct lflow_data *lflow_data = data; @@ -142,6 +137,12 @@ lflow_northd_handler(struct engine_node *node, return false; } + if (!lflow_handle_northd_lb_changes(eng_ctx->ovnsb_idl_txn, + &northd_data->trk_data.trk_lbs, + &lflow_input, lflow_data->lflow_table)) { + return false; + } + engine_set_node_state(node, EN_UPDATED); return true; } diff --git a/northd/lflow-mgr.c b/northd/lflow-mgr.c index 08962e9172..d779e7e087 100644 --- a/northd/lflow-mgr.c +++ b/northd/lflow-mgr.c @@ -105,6 +105,10 @@ static void ovn_dp_group_add_with_reference(struct ovn_lflow *, size_t bitmap_len); static void unlink_lflows_from_datapath(struct lflow_ref *); +static void unlink_lflows_from_all_datapaths(struct lflow_ref *, + size_t n_ls_datapaths, + size_t n_lr_datapaths); + static void lflow_ref_sync_lflows_to_sb__(struct lflow_ref *, struct lflow_table *, struct ovsdb_idl_txn *ovnsb_txn, @@ -394,6 +398,15 @@ lflow_ref_clear_lflows(struct lflow_ref *lflow_ref) unlink_lflows_from_datapath(lflow_ref); } +void +lflow_ref_clear_lflows_for_all_dps(struct lflow_ref *lflow_ref, + size_t n_ls_datapaths, + size_t n_lr_datapaths) +{ + unlink_lflows_from_all_datapaths(lflow_ref, n_ls_datapaths, + n_lr_datapaths); +} + void lflow_ref_clear_and_sync_lflows(struct lflow_ref *lflow_ref, struct lflow_table *lflow_table, @@ -462,7 +475,9 @@ lflow_table_add_lflow(struct lflow_table *lflow_table, /* lflow_ref_node for this lflow doesn't exist yet. Add it. */ struct lflow_ref_node *ref_node = xzalloc(sizeof *ref_node); ref_node->lflow = lflow; - ref_node->dp_index = od->index; + if (od) { + ref_node->dp_index = od->index; + } ovs_list_insert(&lflow_ref->lflows_ref_list, &ref_node->ref_list_node); @@ -1047,6 +1062,30 @@ unlink_lflows_from_datapath(struct lflow_ref *lflow_ref) } } +static void +unlink_lflows_from_all_datapaths(struct lflow_ref *lflow_ref, + size_t n_ls_datapaths, + size_t n_lr_datapaths) +{ + struct lflow_ref_node *ref_node; + struct ovn_lflow *lflow; + LIST_FOR_EACH (ref_node, ref_list_node, &lflow_ref->lflows_ref_list) { + size_t n_datapaths; + size_t index; + + lflow = ref_node->lflow; + if (ovn_stage_to_datapath_type(lflow->stage) == DP_SWITCH) { + n_datapaths = n_ls_datapaths; + } else { + n_datapaths = n_lr_datapaths; + } + + BITMAP_FOR_EACH_1 (index, n_datapaths, lflow->dpg_bitmap) { + bitmap_set0(lflow->dpg_bitmap, index); + } + } +} + static void lflow_ref_sync_lflows_to_sb__(struct lflow_ref *lflow_ref, struct lflow_table *lflow_table, diff --git a/northd/lflow-mgr.h b/northd/lflow-mgr.h index 02b74aa131..c65cd70e71 100644 --- a/northd/lflow-mgr.h +++ b/northd/lflow-mgr.h @@ -54,6 +54,9 @@ struct lflow_ref *lflow_ref_alloc(const char *res_name); void lflow_ref_destroy(struct lflow_ref *); void lflow_ref_reset(struct lflow_ref *lflow_ref); void lflow_ref_clear_lflows(struct lflow_ref *); +void lflow_ref_clear_lflows_for_all_dps(struct lflow_ref *, + size_t n_ls_datapaths, + size_t n_lr_datapaths); void lflow_ref_clear_and_sync_lflows(struct lflow_ref *, struct lflow_table *lflow_table, struct ovsdb_idl_txn *ovnsb_txn, diff --git a/northd/northd.c b/northd/northd.c index 104068e293..f56916b3da 100644 --- a/northd/northd.c +++ b/northd/northd.c @@ -3523,6 +3523,7 @@ ovn_lb_datapaths_create(const struct ovn_northd_lb *lb, size_t n_ls_datapaths, lb_dps->lb = lb; lb_dps->nb_ls_map = bitmap_allocate(n_ls_datapaths); lb_dps->nb_lr_map = bitmap_allocate(n_lr_datapaths); + lb_dps->lflow_ref = lflow_ref_alloc(lb->nlb->name); return lb_dps; } @@ -3532,6 +3533,7 @@ ovn_lb_datapaths_destroy(struct ovn_lb_datapaths *lb_dps) { bitmap_free(lb_dps->nb_lr_map); bitmap_free(lb_dps->nb_ls_map); + lflow_ref_destroy(lb_dps->lflow_ref); free(lb_dps); } @@ -16063,11 +16065,11 @@ build_lflows_thread(void *arg) lsi->lflows, &lsi->match, &lsi->actions, - NULL); + lb_dps->lflow_ref); build_lrouter_defrag_flows_for_lb(lb_dps, lsi->lflows, lsi->lr_datapaths, &lsi->match, - NULL); + lb_dps->lflow_ref); build_lrouter_flows_for_lb(lb_dps, lsi->lflows, lsi->meter_groups, lsi->lr_datapaths, @@ -16075,14 +16077,14 @@ build_lflows_thread(void *arg) lsi->features, lsi->svc_monitor_map, &lsi->match, &lsi->actions, - NULL); + lb_dps->lflow_ref); build_lswitch_flows_for_lb(lb_dps, lsi->lflows, lsi->meter_groups, lsi->ls_datapaths, lsi->features, lsi->svc_monitor_map, &lsi->match, &lsi->actions, - NULL); + lb_dps->lflow_ref); } } for (bnum = control->id; @@ -16298,20 +16300,20 @@ build_lswitch_and_lrouter_flows(const struct ovn_datapaths *ls_datapaths, build_lswitch_arp_nd_service_monitor(lb_dps->lb, lsi.ls_ports, lsi.lflows, &lsi.actions, &lsi.match, - NULL); + lb_dps->lflow_ref); build_lrouter_defrag_flows_for_lb(lb_dps, lsi.lflows, lsi.lr_datapaths, &lsi.match, - NULL); + lb_dps->lflow_ref); build_lrouter_flows_for_lb(lb_dps, lsi.lflows, lsi.meter_groups, lsi.lr_datapaths, lsi.lr_sful_table, lsi.features, lsi.svc_monitor_map, &lsi.match, &lsi.actions, - NULL); + lb_dps->lflow_ref); build_lswitch_flows_for_lb(lb_dps, lsi.lflows, lsi.meter_groups, lsi.ls_datapaths, lsi.features, lsi.svc_monitor_map, &lsi.match, &lsi.actions, - NULL); + lb_dps->lflow_ref); } stopwatch_stop(LFLOWS_LBS_STOPWATCH_NAME, time_msec()); @@ -16483,11 +16485,17 @@ void build_lflows(struct ovsdb_idl_txn *ovnsb_txn, void reset_lflow_refs_for_northd_resources(struct lflow_input *lflow_input) { + struct ovn_lb_datapaths *lb_dps; struct ovn_port *op; + HMAP_FOR_EACH (op, key_node, lflow_input->ls_ports) { lflow_ref_reset(op->lflow_ref); lflow_ref_reset(op->stateful_lflow_ref); } + + HMAP_FOR_EACH (lb_dps, hmap_node, lflow_input->lb_datapaths_map) { + lflow_ref_reset(lb_dps->lflow_ref); + } } bool @@ -16626,6 +16634,77 @@ lflow_handle_northd_port_changes(struct ovsdb_idl_txn *ovnsb_txn, return true; } +bool +lflow_handle_northd_lb_changes(struct ovsdb_idl_txn *ovnsb_txn, + struct tracked_lbs *trk_lbs, + struct lflow_input *lflow_input, + struct lflow_table *lflows) +{ + struct ovn_lb_datapaths *lb_dps; + struct hmapx_node *hmapx_node; + HMAPX_FOR_EACH (hmapx_node, &trk_lbs->deleted) { + lb_dps = hmapx_node->data; + + lflow_ref_clear_lflows_for_all_dps(lb_dps->lflow_ref, + ods_size(lflow_input->ls_datapaths), + ods_size(lflow_input->lr_datapaths)); + lflow_ref_sync_lflows_to_sb(lb_dps->lflow_ref, lflows, ovnsb_txn, + lflow_input->ls_datapaths, + lflow_input->lr_datapaths, + lflow_input->ovn_internal_version_changed, + lflow_input->sbrec_logical_flow_table, + lflow_input->sbrec_logical_dp_group_table); + } + + HMAPX_FOR_EACH (hmapx_node, &trk_lbs->crupdated) { + lb_dps = hmapx_node->data; + + /* unlink old lflows. */ + lflow_ref_clear_lflows_for_all_dps(lb_dps->lflow_ref, + ods_size(lflow_input->ls_datapaths), + ods_size(lflow_input->lr_datapaths)); + + /* Generate new lflows. */ + struct ds match = DS_EMPTY_INITIALIZER; + struct ds actions = DS_EMPTY_INITIALIZER; + + build_lswitch_arp_nd_service_monitor(lb_dps->lb, lflow_input->ls_ports, + lflows, &actions, + &match, lb_dps->lflow_ref); + build_lrouter_defrag_flows_for_lb(lb_dps, lflows, + lflow_input->lr_datapaths, &match, + lb_dps->lflow_ref); + build_lrouter_flows_for_lb(lb_dps, lflows, + lflow_input->meter_groups, + lflow_input->lr_datapaths, + lflow_input->lr_sful_table, + lflow_input->features, + lflow_input->svc_monitor_map, + &match, &actions, + lb_dps->lflow_ref); + build_lswitch_flows_for_lb(lb_dps, lflows, + lflow_input->meter_groups, + lflow_input->ls_datapaths, + lflow_input->features, + lflow_input->svc_monitor_map, + &match, &actions, + lb_dps->lflow_ref); + + ds_destroy(&match); + ds_destroy(&actions); + + /* Sync the new flows to SB. */ + lflow_ref_sync_lflows_to_sb(lb_dps->lflow_ref, lflows, ovnsb_txn, + lflow_input->ls_datapaths, + lflow_input->lr_datapaths, + lflow_input->ovn_internal_version_changed, + lflow_input->sbrec_logical_flow_table, + lflow_input->sbrec_logical_dp_group_table); + } + + return true; +} + static bool mirror_needs_update(const struct nbrec_mirror *nb_mirror, const struct sbrec_mirror *sb_mirror) diff --git a/northd/northd.h b/northd/northd.h index e2df53e6f8..863bcce001 100644 --- a/northd/northd.h +++ b/northd/northd.h @@ -101,6 +101,30 @@ struct ovn_lb_datapaths { size_t n_nb_lr; unsigned long *nb_lr_map; + + /* Reference of lflows generated for this load balancer. + * + * This data is initialized and destroyed by the en_northd node, but + * populated and used only by the en_lflow node. Ideally this data should + * be maintained as part of en_lflow's data (struct lflow_data): a hash + * index from ovn_port key to lflows. However, it would be less efficient + * and more complex: + * + * 1. It would require an extra search (using the index) to find the + * lflows. + * + * 2. Building the index needs to be thread-safe, using either a global + * lock which is obviously less efficient, or hash-based lock array which + * is more complex. + * + * Maintaining the lflow_ref here is more straightforward. The drawback is + * that we need to keep in mind that this data belongs to en_lflow node, + * so never access it from any other nodes. + * + * 'lflow_ref' is used to reference logical flows generated for this + * load balancer. + */ + struct lflow_ref *lflow_ref; }; struct ovn_lb_group_datapaths { @@ -700,6 +724,10 @@ bool lflow_handle_northd_port_changes(struct ovsdb_idl_txn *ovnsb_txn, struct tracked_ovn_ports *, struct lflow_input *, struct lflow_table *lflows); +bool lflow_handle_northd_lb_changes(struct ovsdb_idl_txn *ovnsb_txn, + struct tracked_lbs *, + struct lflow_input *, + struct lflow_table *lflows); bool northd_handle_sb_port_binding_changes( const struct sbrec_port_binding_table *, struct hmap *ls_ports, struct hmap *lr_ports); diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at index 03d62695db..50d88fecd5 100644 --- a/tests/ovn-northd.at +++ b/tests/ovn-northd.at @@ -10507,7 +10507,7 @@ check ovn-nbctl --wait=sb lb-add lb1 10.0.0.10:80 10.0.0.3:80 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE(1) @@ -10517,21 +10517,26 @@ check ovn-nbctl --wait=sb set load_balancer . ip_port_mappings:10.0.0.3=sw0-p1:1 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb set load_balancer . options:foo=bar check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb -- lb-add lb2 20.0.0.10:80 20.0.0.20:80 -- lb-add lb3 30.0.0.10:80 30.0.0.20:80 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE(1) @@ -10541,7 +10546,7 @@ check ovn-nbctl --wait=sb -- lb-del lb2 -- lb-del lb3 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE(1) @@ -10753,8 +10758,9 @@ check ovn-nbctl --wait=sb add load_balancer_group . load_Balancer $lb1_uuid check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb clear load_balancer_group . load_Balancer @@ -10769,7 +10775,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb add load_balancer_group . load_Balancer $lb1_uuid check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10778,6 +10784,7 @@ check ovn-nbctl --wait=sb add logical_switch sw0 load_balancer_group $lbg1_uuid check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute +check_engine_stats ls_stateful norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute compute @@ -10786,6 +10793,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb set load_balancer . options:bar=foo check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats ls_stateful norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute compute @@ -10795,6 +10803,7 @@ check ovn-nbctl --wait=sb set load_balancer lb1 vips:'"10.0.0.10:80"'='"10.0.0.1 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute +check_engine_stats ls_stateful norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10892,6 +10901,7 @@ check_engine_stats northd recompute nocompute check_engine_stats lr_stateful recompute nocompute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute compute +CHECK_NO_CHANGE_AFTER_RECOMPUTE # Add back lb group to logical switch and then delete it. check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats @@ -10901,6 +10911,7 @@ check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute compute +CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb clear logical_switch sw0 load_balancer_group -- \ @@ -10934,14 +10945,17 @@ check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute check_engine_stats lflow norecompute nocompute check_engine_stats sync_to_sb_lb norecompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb set load_balancer_group . load_balancer="$lb2_uuid,$lb3_uuid,$lb4_uuid" check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats ls_stateful norecompute compute check_engine_stats lr_stateful norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb set logical_switch sw0 load_balancer_group=$lbg1_uuid From patchwork Tue Nov 28 02:37:22 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Numan Siddique X-Patchwork-Id: 1869081 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.136; helo=smtp3.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4SfRTK1jHpz1yRy for ; Tue, 28 Nov 2023 13:38:13 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id EBAA760F93; Tue, 28 Nov 2023 02:38:10 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org EBAA760F93 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id vG1wKXxKjRpJ; Tue, 28 Nov 2023 02:38:08 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp3.osuosl.org (Postfix) with ESMTPS id 81BF96100A; Tue, 28 Nov 2023 02:38:07 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 81BF96100A Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 4F4EAC0DD0; Tue, 28 Nov 2023 02:38:07 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp4.osuosl.org (smtp4.osuosl.org [IPv6:2605:bc80:3010::137]) by lists.linuxfoundation.org (Postfix) with ESMTP id 20BB8C0039 for ; Tue, 28 Nov 2023 02:38:06 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id C1F7641526 for ; Tue, 28 Nov 2023 02:37:42 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org C1F7641526 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Oxvx6saD8B-H for ; Tue, 28 Nov 2023 02:37:40 +0000 (UTC) Received: from relay2-d.mail.gandi.net (relay2-d.mail.gandi.net [IPv6:2001:4b98:dc4:8::222]) by smtp4.osuosl.org (Postfix) with ESMTPS id D9D5741514 for ; Tue, 28 Nov 2023 02:37:39 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org D9D5741514 Received: by mail.gandi.net (Postfix) with ESMTPSA id 2C9B540002; Tue, 28 Nov 2023 02:37:36 +0000 (UTC) From: numans@ovn.org To: dev@openvswitch.org Date: Mon, 27 Nov 2023 21:37:22 -0500 Message-ID: <20231128023722.569941-1-numans@ovn.org> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20231128023410.569539-1-numans@ovn.org> References: <20231128023410.569539-1-numans@ovn.org> MIME-Version: 1.0 X-GND-Sasl: numans@ovn.org Subject: [ovs-dev] [PATCH ovn v3 12/16] northd: Add lr_stateful handler for lflow engine node. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Numan Siddique Signed-off-by: Numan Siddique --- northd/en-lflow.c | 29 +++ northd/en-lflow.h | 1 + northd/en-lr-stateful.c | 39 ++++- northd/en-lr-stateful.h | 26 +++ northd/inc-proc-northd.c | 3 +- northd/northd.c | 370 ++++++++++++++++++++++----------------- northd/northd.h | 10 ++ tests/ovn-northd.at | 48 ++--- 8 files changed, 336 insertions(+), 190 deletions(-) diff --git a/northd/en-lflow.c b/northd/en-lflow.c index 13284b5556..09748f570b 100644 --- a/northd/en-lflow.c +++ b/northd/en-lflow.c @@ -164,6 +164,35 @@ lflow_port_group_handler(struct engine_node *node, void *data OVS_UNUSED) return true; } +bool +lflow_lr_stateful_handler(struct engine_node *node, void *data) +{ + struct ed_type_lr_stateful *lr_sful_data = + engine_get_input_data("lr_stateful", node); + + if (!lr_stateful_has_tracked_data(&lr_sful_data->trk_data) + || lr_sful_data->trk_data.vip_nats_changed) { + return false; + } + + const struct engine_context *eng_ctx = engine_get_context(); + struct lflow_data *lflow_data = data; + + struct lflow_input lflow_input; + lflow_get_input_data(node, &lflow_input); + + if (!lflow_handle_lr_stateful_changes(eng_ctx->ovnsb_idl_txn, + &lr_sful_data->trk_data, + &lflow_input, + lflow_data->lflow_table)) { + return false; + } + + engine_set_node_state(node, EN_UPDATED); + + return true; +} + void *en_lflow_init(struct engine_node *node OVS_UNUSED, struct engine_arg *arg OVS_UNUSED) { diff --git a/northd/en-lflow.h b/northd/en-lflow.h index f7325c56b1..1d813a2a29 100644 --- a/northd/en-lflow.h +++ b/northd/en-lflow.h @@ -20,5 +20,6 @@ void *en_lflow_init(struct engine_node *node, struct engine_arg *arg); void en_lflow_cleanup(void *data); bool lflow_northd_handler(struct engine_node *, void *data); bool lflow_port_group_handler(struct engine_node *, void *data); +bool lflow_lr_stateful_handler(struct engine_node *, void *data); #endif /* EN_LFLOW_H */ diff --git a/northd/en-lr-stateful.c b/northd/en-lr-stateful.c index a54749ad93..8e025f057e 100644 --- a/northd/en-lr-stateful.c +++ b/northd/en-lr-stateful.c @@ -39,6 +39,7 @@ #include "lib/ovn-sb-idl.h" #include "lib/ovn-util.h" #include "lib/stopwatch-names.h" +#include "lflow-mgr.h" #include "northd.h" VLOG_DEFINE_THIS_MODULE(en_lr_stateful); @@ -81,7 +82,7 @@ static void remove_lrouter_lb_reachable_ips(struct lr_stateful_record *, enum lb_neighbor_responder_mode, const struct sset *lb_ips_v4, const struct sset *lb_ips_v6); -static void lr_stateful_build_vip_nats(struct lr_stateful_record *); +static bool lr_stateful_build_vip_nats(struct lr_stateful_record *); /* 'lr_stateful' engine node manages the NB logical router LB data. */ @@ -110,6 +111,7 @@ en_lr_stateful_clear_tracked_data(void *data_) struct ed_type_lr_stateful *data = (struct ed_type_lr_stateful *) data_; hmapx_clear(&data->trk_data.crupdated); + data->trk_data.vip_nats_changed = false; } void @@ -190,6 +192,10 @@ lr_stateful_lb_data_handler(struct engine_node *node, void *data_) /* Add the lr_sful_rec rec to the tracking data. */ hmapx_add(&data->trk_data.crupdated, lr_sful_rec); + + if (!sset_is_empty(&lr_sful_rec->vip_nats)) { + data->trk_data.vip_nats_changed = true; + } continue; } @@ -298,7 +304,9 @@ lr_stateful_lb_data_handler(struct engine_node *node, void *data_) * vip nats. */ HMAPX_FOR_EACH (hmapx_node, &data->trk_data.crupdated) { lr_sful_rec = hmapx_node->data; - lr_stateful_build_vip_nats(lr_sful_rec); + if (lr_stateful_build_vip_nats(lr_sful_rec)) { + data->trk_data.vip_nats_changed = true; + } lr_sful_rec->has_lb_vip = od_has_lb_vip(lr_sful_rec->od); } @@ -335,8 +343,13 @@ lr_stateful_lr_nat_handler(struct engine_node *node, void *data_) lrnat_rec, input_data.lb_datapaths_map, input_data.lbgrp_datapaths_map); + if (!sset_is_empty(&lr_sful_rec->vip_nats)) { + data->trk_data.vip_nats_changed = true; + } } else { - lr_stateful_build_vip_nats(lr_sful_rec); + if (lr_stateful_build_vip_nats(lr_sful_rec)) { + data->trk_data.vip_nats_changed = true; + } } /* Add the lr_sful_rec rec to the tracking data. */ @@ -435,6 +448,7 @@ lr_stateful_record_create(struct lr_stateful_table *table, lr_sful_rec->od = lrnat_rec->od; lr_stateful_record_init(lr_sful_rec, lb_datapaths_map, lbgrp_datapaths_map); + lr_sful_rec->lflow_ref = lflow_ref_alloc(lr_sful_rec->od->nbr->name); hmap_insert(&table->entries, &lr_sful_rec->key_node, uuid_hash(&lr_sful_rec->od->nbr->header_.uuid)); @@ -449,6 +463,7 @@ lr_stateful_record_destroy(struct lr_stateful_record *lr_sful_rec) ovn_lb_ip_set_destroy(lr_sful_rec->lb_ips); lr_sful_rec->lb_ips = NULL; sset_destroy(&lr_sful_rec->vip_nats); + lflow_ref_destroy(lr_sful_rec->lflow_ref); free(lr_sful_rec); } @@ -515,7 +530,7 @@ lr_stateful_record_init(struct lr_stateful_record *lr_sful_rec, sset_init(&lr_sful_rec->vip_nats); - if (!nbr->n_nat) { + if (nbr->n_nat) { lr_stateful_build_vip_nats(lr_sful_rec); } @@ -629,10 +644,12 @@ remove_lrouter_lb_reachable_ips(struct lr_stateful_record *lr_sful_rec, } } -static void +static bool lr_stateful_build_vip_nats(struct lr_stateful_record *lr_sful_rec) { - sset_clear(&lr_sful_rec->vip_nats); + struct sset old_vip_nats = SSET_INITIALIZER(&old_vip_nats); + sset_swap(&lr_sful_rec->vip_nats, &old_vip_nats); + const char *external_ip; SSET_FOR_EACH (external_ip, &lr_sful_rec->lrnat_rec->external_ips) { bool is_vip_nat = false; @@ -648,4 +665,14 @@ lr_stateful_build_vip_nats(struct lr_stateful_record *lr_sful_rec) sset_add(&lr_sful_rec->vip_nats, external_ip); } } + + bool vip_nats_changed = + sset_count(&lr_sful_rec->vip_nats) != sset_count(&old_vip_nats); + if (!vip_nats_changed) { + vip_nats_changed = !sset_equals(&lr_sful_rec->vip_nats, + &old_vip_nats); + } + sset_destroy(&old_vip_nats); + + return vip_nats_changed; } diff --git a/northd/en-lr-stateful.h b/northd/en-lr-stateful.h index 0571e057f8..077de1bad1 100644 --- a/northd/en-lr-stateful.h +++ b/northd/en-lr-stateful.h @@ -32,6 +32,7 @@ struct ovn_datapath; struct lr_nat_record; +struct lflow_ref; /* lr_stateful_table: This represents a table of logical routers with * stateful related data. @@ -56,6 +57,27 @@ struct lr_stateful_record { /* sset of vips which are also part of lr nats. */ struct sset vip_nats; + + /* 'lflow_ref' is used to reference logical flows generated for + * this lr_lb_nat_data record. + * + * This data is initialized and destroyed by the en_lr_lb_nat_data node, + * but populated and used only by the en_lflow node. Ideally this data + * should be maintained as part of en_lflow's data. However, it would + * be less efficient and more complex: + * + * 1. It would require an extra search (using the index) to find the + * lflows. + * + * 2. Building the index needs to be thread-safe, using either a global + * lock which is obviously less efficient, or hash-based lock array which + * is more complex. + * + * Adding the lflow_ref here is more straightforward. The drawback is that + * we need to keep in mind that this data belongs to en_lflow node, so + * never access it from any other nodes. + */ + struct lflow_ref *lflow_ref; }; struct lr_stateful_table { @@ -75,6 +97,10 @@ struct lr_stateful_table { struct lr_stateful_tracked_data { /* Created or updated logical router with LB and/or NAT data. */ struct hmapx crupdated; /* Stores 'struct lr_stateful_record'. */ + + /* Indicates if any router's NATs changed which were also LB vips + * or vice versa. */ + bool vip_nats_changed; }; struct ed_type_lr_stateful { diff --git a/northd/inc-proc-northd.c b/northd/inc-proc-northd.c index 0e17bfe2e6..8be413200e 100644 --- a/northd/inc-proc-northd.c +++ b/northd/inc-proc-northd.c @@ -228,11 +228,12 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, engine_add_input(&en_lflow, &en_sb_logical_flow, NULL); engine_add_input(&en_lflow, &en_sb_multicast_group, NULL); engine_add_input(&en_lflow, &en_sb_igmp_group, NULL); - engine_add_input(&en_lflow, &en_lr_stateful, NULL); engine_add_input(&en_lflow, &en_ls_stateful, NULL); engine_add_input(&en_lflow, &en_sb_logical_dp_group, NULL); + engine_add_input(&en_lflow, &en_ls_stateful, NULL); engine_add_input(&en_lflow, &en_northd, lflow_northd_handler); engine_add_input(&en_lflow, &en_port_group, lflow_port_group_handler); + engine_add_input(&en_lflow, &en_lr_stateful, lflow_lr_stateful_handler); engine_add_input(&en_sync_to_sb_addr_set, &en_nb_address_set, sync_to_sb_addr_set_nb_address_set_handler); diff --git a/northd/northd.c b/northd/northd.c index f56916b3da..235b9e100f 100644 --- a/northd/northd.c +++ b/northd/northd.c @@ -1227,7 +1227,7 @@ ovn_port_create(struct hmap *ports, const char *key, op->lflow_ref = lflow_ref_alloc(key); op->stateful_lflow_ref = lflow_ref_alloc(key); - + op->routable_lflow_ref = lflow_ref_alloc(key); return op; } @@ -1268,6 +1268,7 @@ ovn_port_destroy_orphan(struct ovn_port *port) free(port->key); lflow_ref_destroy(port->lflow_ref); lflow_ref_destroy(port->stateful_lflow_ref); + lflow_ref_destroy(port->routable_lflow_ref); free(port); } @@ -12832,63 +12833,6 @@ build_ip_routing_flows_for_lrp( } } -/* Logical router ingress table IP_ROUTING : IP Routing. - * - * For the LSP 'op' of type router, if there are logical router ports other - * than the LSP's peer connected to the logical switch, then for routable - * addresses (such as NAT IPs, LB VIPs, etc.) on each of the connected router - * ports, add routes to the LSP's peer router. - */ -static void -build_ip_routing_flows_for_router_type_lsp( - struct ovn_port *op, const struct lr_stateful_table *lr_sful_table, - const struct hmap *lr_ports, struct lflow_table *lflows, - struct lflow_ref *lflow_ref) -{ - ovs_assert(op->nbsp); - if (!lsp_is_router(op->nbsp)) { - return; - } - - struct ovn_port *peer = ovn_port_get_peer(lr_ports, op); - if (!peer || !peer->nbrp || !peer->lrp_networks.n_ipv4_addrs - || !op->od->n_router_ports) { - return; - } - - for (int i = 0; i < op->od->n_router_ports; i++) { - struct ovn_port *router_port = ovn_port_get_peer( - lr_ports, op->od->router_ports[i]); - if (!router_port || !router_port->nbrp || router_port == peer) { - continue; - } - - const struct lr_stateful_record *lr_sful_rec = - lr_stateful_table_find_by_index(lr_sful_table, - router_port->od->index); - - if (router_port->nbrp->ha_chassis_group || - router_port->nbrp->n_gateway_chassis) { - struct ovn_port_routable_addresses ra = - get_op_routable_addresses(router_port, lr_sful_rec); - for (size_t j = 0; j < ra.n_addrs; j++) { - struct lport_addresses *laddrs = &ra.laddrs[j]; - for (size_t k = 0; k < laddrs->n_ipv4_addrs; k++) { - add_route(lflows, peer->od, peer, - peer->lrp_networks.ipv4_addrs[0].addr_s, - laddrs->ipv4_addrs[k].network_s, - laddrs->ipv4_addrs[k].plen, NULL, false, 0, - &peer->nbrp->header_, false, - ROUTE_PRIO_OFFSET_CONNECTED, - lflow_ref); - } - } - destroy_routable_addresses(&ra); - } - } - -} - static void build_static_route_flows_for_lrouter( struct ovn_datapath *od, const struct chassis_features *features, @@ -13130,42 +13074,6 @@ build_arp_resolve_flows_for_lrouter( lflow_ref); } -static void -routable_addresses_to_lflows(struct lflow_table *lflows, - struct ovn_port *router_port, - struct ovn_port *peer, - const struct lr_stateful_record *lr_sful_rec, - struct ds *match, struct ds *actions, - struct lflow_ref *lflow_ref) -{ - struct ovn_port_routable_addresses ra = - get_op_routable_addresses(router_port, lr_sful_rec); - if (!ra.n_addrs) { - return; - } - - for (size_t i = 0; i < ra.n_addrs; i++) { - ds_clear(match); - ds_put_format(match, "outport == %s && "REG_NEXT_HOP_IPV4" == {", - peer->json_key); - bool first = true; - for (size_t j = 0; j < ra.laddrs[i].n_ipv4_addrs; j++) { - if (!first) { - ds_put_cstr(match, ", "); - } - ds_put_cstr(match, ra.laddrs[i].ipv4_addrs[j].addr_s); - first = false; - } - ds_put_cstr(match, "}"); - - ds_clear(actions); - ds_put_format(actions, "eth.dst = %s; next;", ra.laddrs[i].ea_s); - ovn_lflow_add(lflows, peer->od, S_ROUTER_IN_ARP_RESOLVE, 100, - ds_cstr(match), ds_cstr(actions), lflow_ref); - } - destroy_routable_addresses(&ra); -} - /* Local router ingress table ARP_RESOLVE: ARP Resolution. * * Any unicast packet that reaches this table is an IP packet whose @@ -13408,52 +13316,6 @@ build_arp_resolve_flows_for_lsp( } } -static void -build_arp_resolve_flows_for_lsp_routable_addresses( - struct ovn_port *op, struct lflow_table *lflows, - const struct hmap *lr_ports, - const struct lr_stateful_table *lr_sful_table, - struct ds *match, struct ds *actions, - struct lflow_ref *lflow_ref) -{ - if (!lsp_is_router(op->nbsp)) { - return; - } - - struct ovn_port *peer = ovn_port_get_peer(lr_ports, op); - if (!peer || !peer->nbrp) { - return; - } - - if (peer->od->nbr && - smap_get_bool(&peer->od->nbr->options, - "dynamic_neigh_routers", false)) { - return; - } - - for (size_t i = 0; i < op->od->n_router_ports; i++) { - struct ovn_port *router_port = - ovn_port_get_peer(lr_ports, op->od->router_ports[i]); - if (!router_port || !router_port->nbrp) { - continue; - } - - /* Skip the router port under consideration. */ - if (router_port == peer) { - continue; - } - - if (smap_get(&peer->od->nbr->options, "chassis") || peer->cr_port) { - const struct lr_stateful_record *lr_sful_rec; - lr_sful_rec = lr_stateful_table_find_by_index(lr_sful_table, - router_port->od->index); - routable_addresses_to_lflows(lflows, router_port, peer, - lr_sful_rec, match, actions, - lflow_ref); - } - } -} - static void build_icmperr_pkt_big_flows(struct ovn_port *op, int mtu, struct lflow_table *lflows, @@ -15670,8 +15532,6 @@ static void build_lsp_lflows_for_lbnats(struct ovn_port *lsp, struct ovn_port *lrp_peer, const struct lr_stateful_record *lr_sful_rec, - const struct lr_stateful_table *lr_sful_table, - const struct hmap *lr_ports, struct lflow_table *lflows, struct ds *match, struct ds *actions, @@ -15681,19 +15541,101 @@ build_lsp_lflows_for_lbnats(struct ovn_port *lsp, build_lswitch_rport_arp_req_flows_for_lbnats( lrp_peer, lr_sful_rec, lsp->od, lsp, lflows, &lsp->nbsp->header_, lflow_ref); - build_ip_routing_flows_for_router_type_lsp(lsp, lr_sful_table, - lr_ports, lflows, - lflow_ref); - build_arp_resolve_flows_for_lsp_routable_addresses( - lsp, lflows, lr_ports, lr_sful_table, match, actions, lflow_ref); build_lswitch_ip_unicast_lookup_for_nats(lsp, lr_sful_rec, lflows, match, actions, lflow_ref); } +/* Logical router ingress table IP_ROUTING : IP Routing. + * + * For the LRP 'lrp's peer's logical switch, if there are logical router + * ports ('peer_lrp's) other than the 'lrp', then for routable addresses + * (such as NAT IPs, LB VIPs, etc.) of the 'lrp' add routes to the + * peer_lrp's datapath. + */ +static void +build_routable_flows_for_router_port( + struct ovn_port *lrp, const struct lr_stateful_record *lr_sful_rec, + struct lflow_table *lflows, + struct ds *match, + struct ds *actions, + struct lflow_ref *lflow_ref) +{ + ovs_assert(lrp->nbrp && lrp->od == lr_sful_rec->od); + + struct ovn_port *lsp_peer = lrp->peer; + if (!lsp_peer || !lsp_peer->nbsp) { + return; + } + + struct ovn_datapath *peer_ls = lsp_peer->od; + ovs_assert(peer_ls->nbs); + + struct ovn_port_routable_addresses ra = + get_op_routable_addresses(lrp, lr_sful_rec); + + struct ovn_port *router_port; + + for (size_t i = 0; i < peer_ls->n_router_ports; i++) { + router_port = peer_ls->router_ports[i]->peer; + + if (router_port == lrp) { + continue; + } + + if (lrp->nbrp->ha_chassis_group || + lrp->nbrp->n_gateway_chassis) { + for (size_t j = 0; j < ra.n_addrs; j++) { + struct lport_addresses *laddrs = &ra.laddrs[j]; + for (size_t k = 0; k < laddrs->n_ipv4_addrs; k++) { + add_route(lflows, router_port->od, router_port, + router_port->lrp_networks.ipv4_addrs[0].addr_s, + laddrs->ipv4_addrs[k].network_s, + laddrs->ipv4_addrs[k].plen, NULL, false, 0, + &router_port->nbrp->header_, false, + ROUTE_PRIO_OFFSET_CONNECTED, + lflow_ref); + } + } + } + + bool dynamic_neigh_router = + smap_get_bool(&router_port->od->nbr->options, + "dynamic_neigh_routers", false); + + if (!dynamic_neigh_router && + (router_port->od->is_gw_router || router_port->cr_port)) { + + for (size_t k = 0; k < ra.n_addrs; k++) { + ds_clear(match); + ds_put_format(match, "outport == %s && " + REG_NEXT_HOP_IPV4" == {", + router_port->json_key); + bool first = true; + for (size_t j = 0; j < ra.laddrs[k].n_ipv4_addrs; j++) { + if (!first) { + ds_put_cstr(match, ", "); + } + ds_put_cstr(match, ra.laddrs[k].ipv4_addrs[j].addr_s); + first = false; + } + ds_put_cstr(match, "}"); + + ds_clear(actions); + ds_put_format(actions, "eth.dst = %s; next;", + ra.laddrs[k].ea_s); + ovn_lflow_add(lflows, router_port->od, S_ROUTER_IN_ARP_RESOLVE, + 100, ds_cstr(match), ds_cstr(actions), + lflow_ref); + } + } + } + + destroy_routable_addresses(&ra); +} + static void build_lbnat_lflows_iterate_by_lsp(struct ovn_port *op, const struct lr_stateful_table *lr_sful_table, - const struct hmap *lr_ports, struct ds *match, struct ds *actions, struct lflow_table *lflows, @@ -15711,8 +15653,8 @@ build_lbnat_lflows_iterate_by_lsp(struct ovn_port *op, ovs_assert(lr_sful_rec); build_lsp_lflows_for_lbnats(op, op->peer, lr_sful_rec, - lr_sful_table, lr_ports, lflows, - match, actions, lflow_ref); + lflows,match, actions, + lflow_ref); } static void @@ -15762,7 +15704,8 @@ build_lbnat_lflows_iterate_by_lrp(struct ovn_port *op, struct ds *match, struct ds *actions, struct lflow_table *lflows, - struct lflow_ref *lflow_ref) + struct lflow_ref *lflow_ref, + struct lflow_ref *routable_lflow_ref) { ovs_assert(op->nbrp); @@ -15773,6 +15716,9 @@ build_lbnat_lflows_iterate_by_lrp(struct ovn_port *op, build_lrp_lflows_for_lbnats(op, lr_sful_rec, meter_groups, match, actions, lflows, lflow_ref); + + build_routable_flows_for_router_port(op, lr_sful_rec, lflows, match, + actions, routable_lflow_ref); } static void @@ -15787,15 +15733,15 @@ build_lr_stateful_flows(const struct lr_stateful_record *lr_sful_rec, { build_lrouter_nat_defrag_and_lb(lr_sful_rec, lflows, ls_ports, lr_ports, match, actions, meter_groups, features, - NULL); + lr_sful_rec->lflow_ref); build_lr_gateway_redirect_flows_for_nats(lr_sful_rec->od, lr_sful_rec->lrnat_rec, lflows, match, actions, - NULL); + lr_sful_rec->lflow_ref); build_lrouter_arp_nd_for_datapath(lr_sful_rec->od, lr_sful_rec->lrnat_rec, lflows, meter_groups, - NULL); + lr_sful_rec->lflow_ref); } static void @@ -16026,7 +15972,6 @@ build_lflows_thread(void *arg) &lsi->actions, lsi->lflows); build_lbnat_lflows_iterate_by_lsp(op, lsi->lr_sful_table, - lsi->lr_ports, &lsi->match, &lsi->actions, lsi->lflows, @@ -16048,7 +15993,8 @@ build_lflows_thread(void *arg) &lsi->match, &lsi->actions, lsi->lflows, - op->stateful_lflow_ref); + op->stateful_lflow_ref, + op->routable_lflow_ref); } } for (bnum = control->id; @@ -16280,9 +16226,9 @@ build_lswitch_and_lrouter_flows(const struct ovn_datapaths *ls_datapaths, &lsi.match, &lsi.actions, lsi.lflows); - build_lbnat_lflows_iterate_by_lsp(op, lsi.lr_sful_table, lsi.lr_ports, - &lsi.match, &lsi.actions, - lsi.lflows, + build_lbnat_lflows_iterate_by_lsp(op, lsi.lr_sful_table, + &lsi.match, + &lsi.actions, lsi.lflows, op->stateful_lflow_ref); } HMAP_FOR_EACH (op, key_node, lr_ports) { @@ -16292,7 +16238,8 @@ build_lswitch_and_lrouter_flows(const struct ovn_datapaths *ls_datapaths, &lsi.match, &lsi.actions, lsi.lflows, - op->stateful_lflow_ref); + op->stateful_lflow_ref, + op->routable_lflow_ref); } stopwatch_stop(LFLOWS_PORTS_STOPWATCH_NAME, time_msec()); stopwatch_start(LFLOWS_LBS_STOPWATCH_NAME, time_msec()); @@ -16485,12 +16432,24 @@ void build_lflows(struct ovsdb_idl_txn *ovnsb_txn, void reset_lflow_refs_for_northd_resources(struct lflow_input *lflow_input) { + const struct lr_stateful_record *lr_sful_rec; struct ovn_lb_datapaths *lb_dps; struct ovn_port *op; + LR_STATEFUL_TABLE_FOR_EACH (lr_sful_rec, lflow_input->lr_sful_table) { + lflow_ref_reset(lr_sful_rec->lflow_ref); + } + HMAP_FOR_EACH (op, key_node, lflow_input->ls_ports) { lflow_ref_reset(op->lflow_ref); lflow_ref_reset(op->stateful_lflow_ref); + lflow_ref_reset(op->routable_lflow_ref); + } + + HMAP_FOR_EACH (op, key_node, lflow_input->lr_ports) { + lflow_ref_reset(op->lflow_ref); + lflow_ref_reset(op->stateful_lflow_ref); + lflow_ref_reset(op->routable_lflow_ref); } HMAP_FOR_EACH (lb_dps, hmap_node, lflow_input->lb_datapaths_map) { @@ -16547,11 +16506,10 @@ lflow_handle_northd_port_changes(struct ovsdb_idl_txn *ovnsb_txn, ovs_assert(lr_sful_rec); lflow_ref_clear_lflows(op->stateful_lflow_ref); build_lsp_lflows_for_lbnats(op, op->peer, lr_sful_rec, - lflow_input->lr_sful_table, - lflow_input->lr_ports, lflows, &match, &actions, op->stateful_lflow_ref); - lflow_ref_sync_lflows_to_sb(op->stateful_lflow_ref, lflows, ovnsb_txn, + lflow_ref_sync_lflows_to_sb(op->stateful_lflow_ref, lflows, + ovnsb_txn, lflow_input->ls_datapaths, lflow_input->lr_datapaths, lflow_input->ovn_internal_version_changed, @@ -16705,6 +16663,92 @@ lflow_handle_northd_lb_changes(struct ovsdb_idl_txn *ovnsb_txn, return true; } +bool +lflow_handle_lr_stateful_changes(struct ovsdb_idl_txn *ovnsb_txn, + struct lr_stateful_tracked_data *trk_data, + struct lflow_input *lflow_input, + struct lflow_table *lflows) +{ + struct lr_stateful_record *lr_sful_rec; + struct hmapx_node *hmapx_node; + + HMAPX_FOR_EACH (hmapx_node, &trk_data->crupdated) { + lr_sful_rec = hmapx_node->data; + + lflow_ref_clear_lflows(lr_sful_rec->lflow_ref); + + /* Generate new lflows. */ + struct ds match = DS_EMPTY_INITIALIZER; + struct ds actions = DS_EMPTY_INITIALIZER; + + build_lr_stateful_flows(lr_sful_rec, lflows, lflow_input->ls_ports, + lflow_input->lr_ports, &match, &actions, + lflow_input->meter_groups, + lflow_input->features); + + /* Sync the new flows to SB. */ + lflow_ref_sync_lflows_to_sb(lr_sful_rec->lflow_ref, lflows, ovnsb_txn, + lflow_input->ls_datapaths, + lflow_input->lr_datapaths, + lflow_input->ovn_internal_version_changed, + lflow_input->sbrec_logical_flow_table, + lflow_input->sbrec_logical_dp_group_table); + + struct ovn_port *op; + HMAP_FOR_EACH (op, dp_node, &lr_sful_rec->od->ports) { + lflow_ref_clear_lflows(op->stateful_lflow_ref); + lflow_ref_clear_lflows_for_all_dps(op->routable_lflow_ref, + ods_size(lflow_input->ls_datapaths), + ods_size(lflow_input->lr_datapaths)); + + build_lbnat_lflows_iterate_by_lrp(op, lflow_input->lr_sful_table, + lflow_input->meter_groups, + &match, &actions, + lflows, + op->stateful_lflow_ref, + op->routable_lflow_ref); + + lflow_ref_sync_lflows_to_sb(op->stateful_lflow_ref, lflows, + ovnsb_txn, + lflow_input->ls_datapaths, + lflow_input->lr_datapaths, + lflow_input->ovn_internal_version_changed, + lflow_input->sbrec_logical_flow_table, + lflow_input->sbrec_logical_dp_group_table); + + lflow_ref_sync_lflows_to_sb(op->routable_lflow_ref, lflows, + ovnsb_txn, lflow_input->ls_datapaths, + lflow_input->lr_datapaths, + lflow_input->ovn_internal_version_changed, + lflow_input->sbrec_logical_flow_table, + lflow_input->sbrec_logical_dp_group_table); + + if (op->peer && op->peer->nbsp) { + lflow_ref_clear_lflows(op->peer->stateful_lflow_ref); + + build_lbnat_lflows_iterate_by_lsp(op->peer, + lflow_input->lr_sful_table, + &match, &actions, + lflows, + op->peer->stateful_lflow_ref); + + lflow_ref_sync_lflows_to_sb(op->peer->stateful_lflow_ref, + lflows, ovnsb_txn, + lflow_input->ls_datapaths, + lflow_input->lr_datapaths, + lflow_input->ovn_internal_version_changed, + lflow_input->sbrec_logical_flow_table, + lflow_input->sbrec_logical_dp_group_table); + } + } + + ds_destroy(&match); + ds_destroy(&actions); + } + + return true; +} + static bool mirror_needs_update(const struct nbrec_mirror *nb_mirror, const struct sbrec_mirror *sb_mirror) diff --git a/northd/northd.h b/northd/northd.h index 863bcce001..6adf06c26f 100644 --- a/northd/northd.h +++ b/northd/northd.h @@ -688,9 +688,13 @@ struct ovn_port { * 'stateful_lflow_ref' is used for logical switch ports of type * 'patch/router' to referenece logical flows generated fo this ovn_port * from the 'lr_stateful' record of the peer port's datapath. + * + * 'routable_lflow_ref' is used to reference logical flows generated for + * the routable ips of a logical router port. */ struct lflow_ref *lflow_ref; struct lflow_ref *stateful_lflow_ref; + struct lflow_ref *routable_lflow_ref; }; void ovnnb_db_run(struct northd_input *input_data, @@ -715,6 +719,8 @@ void northd_indices_create(struct northd_data *data, struct ovsdb_idl *ovnsb_idl); struct lflow_table; +struct lr_stateful_tracked_data; + void build_lflows(struct ovsdb_idl_txn *ovnsb_txn, struct lflow_input *input_data, struct lflow_table *); @@ -728,6 +734,10 @@ bool lflow_handle_northd_lb_changes(struct ovsdb_idl_txn *ovnsb_txn, struct tracked_lbs *, struct lflow_input *, struct lflow_table *lflows); +bool lflow_handle_lr_stateful_changes(struct ovsdb_idl_txn *, + struct lr_stateful_tracked_data *, + struct lflow_input *, + struct lflow_table *lflows); bool northd_handle_sb_port_binding_changes( const struct sbrec_port_binding_table *, struct hmap *ls_ports, struct hmap *lr_ports); diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at index 50d88fecd5..f517057534 100644 --- a/tests/ovn-northd.at +++ b/tests/ovn-northd.at @@ -12,6 +12,7 @@ m4_define([_DUMP_DB_TABLES], [ ovn-sbctl list meter >> $1 ovn-sbctl list meter_band >> $1 ovn-sbctl list port_group >> $1 + ovn-sbctl dump-flows > lflows_$1 ]) # CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10687,7 +10688,7 @@ check ovn-nbctl --wait=sb lr-lb-add lr0 lb1 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10697,7 +10698,7 @@ check ovn-nbctl --wait=sb set load_balancer lb1 vips:'"10.0.0.10:80"'='"10.0.0.1 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10707,7 +10708,7 @@ check ovn-nbctl --wait=sb clear load_Balancer lb1 vips check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10717,7 +10718,7 @@ check ovn-nbctl --wait=sb lb-add lb1 10.0.0.10:80 10.0.0.3:80 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10727,7 +10728,7 @@ check ovn-nbctl --wait=sb lb-add lb1 10.0.0.20:80 10.0.0.30:8080 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10735,6 +10736,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lr-lb-del lr0 lb1 check_engine_stats lb_data norecompute compute check_engine_stats northd recompute nocompute +check_engine_stats lr_stateful recompute nocompute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10850,7 +10852,7 @@ check ovn-nbctl add logical_router lr0 load_balancer_group $lbg1_uuid check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10860,7 +10862,7 @@ check ovn-nbctl --wait=sb set load_balancer lb1 vips:'"10.0.0.10:80"'='"10.0.0.1 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10870,7 +10872,7 @@ check ovn-nbctl --wait=sb clear load_Balancer lb1 vips check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10880,7 +10882,7 @@ check ovn-nbctl --wait=sb lb-add lb1 10.0.0.10:80 10.0.0.3:80 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10890,7 +10892,7 @@ check ovn-nbctl --wait=sb lb-add lb1 10.0.0.20:80 10.0.0.30:8080 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10971,7 +10973,7 @@ check ovn-nbctl --wait=sb set logical_router lr1 load_balancer_group=$lbg1_uuid check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10999,7 +11001,7 @@ check ovn-nbctl --wait=sb lr-lb-add lr1 lb2 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11177,7 +11179,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lr-nat-add lr0 dnat_and_snat 172.168.0.110 10.0.0.4 check_engine_stats northd norecompute compute check_engine_stats lr_nat norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_pb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11186,7 +11188,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb set NAT . options:foo=bar check_engine_stats northd norecompute compute check_engine_stats lr_nat norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_pb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11196,7 +11198,7 @@ check ovn-nbctl --wait=sb set NAT . external_ip=172.168.0.120 check_engine_stats northd norecompute compute check_engine_stats lr_nat norecompute compute check_engine_stats lr_stateful norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_pb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11206,7 +11208,7 @@ check ovn-nbctl --wait=sb set NAT . logical_ip=10.0.0.10 check_engine_stats northd norecompute compute check_engine_stats lr_nat norecompute compute check_engine_stats lr_stateful norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_pb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11216,7 +11218,7 @@ check ovn-nbctl --wait=sb set NAT . type=snat check_engine_stats northd norecompute compute check_engine_stats lr_nat norecompute compute check_engine_stats lr_stateful norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_pb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11226,7 +11228,7 @@ check ovn-nbctl --wait=sb lr-nat-add lr0 dnat_and_snat 172.168.0.110 10.0.0.4 sw check_engine_stats northd norecompute compute check_engine_stats lr_nat norecompute compute check_engine_stats lr_stateful norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_pb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11237,7 +11239,7 @@ check ovn-nbctl --wait=sb set NAT $nat2_uuid external_mac='"30:54:00:00:00:04"' check_engine_stats northd norecompute compute check_engine_stats lr_nat norecompute compute check_engine_stats lr_stateful norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_pb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11258,6 +11260,8 @@ check_engine_stats sync_to_sb_pb recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE +# lflow engine should recompute since the nat ip 172.168.0.150 +# is a lb vip. check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lr-nat-add lr0 dnat_and_snat 172.168.0.150 10.0.0.41 check_engine_stats northd norecompute compute @@ -11267,6 +11271,8 @@ check_engine_stats sync_to_sb_pb recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE +# lflow engine should recompute since the deleted nat ip 172.168.0.150 +# is a lb vip. check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lr-nat-del lr0 dnat_and_snat 172.168.0.150 check_engine_stats northd norecompute compute @@ -11276,6 +11282,8 @@ check_engine_stats sync_to_sb_pb recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE +# lflow engine should recompute since the deleted nat ip 172.168.0.140 +# is a lb vip. check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lr-nat-del lr0 dnat_and_snat 172.168.0.140 check_engine_stats northd norecompute compute @@ -11291,7 +11299,7 @@ check ovn-nbctl --wait=sb clear logical_router lr0 nat check_engine_stats northd norecompute compute check_engine_stats lr_nat norecompute compute check_engine_stats lr_stateful norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_pb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE From patchwork Tue Nov 28 02:37:38 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Numan Siddique X-Patchwork-Id: 1869087 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::136; helo=smtp3.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4SfRVx0f0Qz1yRy for ; Tue, 28 Nov 2023 13:39:37 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 7733560FE6; Tue, 28 Nov 2023 02:39:34 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 7733560FE6 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id l7VGwv83Y7F9; Tue, 28 Nov 2023 02:39:31 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp3.osuosl.org (Postfix) with ESMTPS id 6142E61071; Tue, 28 Nov 2023 02:39:29 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 6142E61071 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id DF7DDC0DCE; Tue, 28 Nov 2023 02:39:28 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 2338DC0072 for ; Tue, 28 Nov 2023 02:39:28 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 3722E61016 for ; Tue, 28 Nov 2023 02:37:55 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 3722E61016 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id H-Cz6AUyMWWE for ; Tue, 28 Nov 2023 02:37:53 +0000 (UTC) Received: from relay2-d.mail.gandi.net (relay2-d.mail.gandi.net [217.70.183.194]) by smtp3.osuosl.org (Postfix) with ESMTPS id 4E93A60FCF for ; Tue, 28 Nov 2023 02:37:53 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 4E93A60FCF Received: by mail.gandi.net (Postfix) with ESMTPSA id E731A40003; Tue, 28 Nov 2023 02:37:50 +0000 (UTC) From: numans@ovn.org To: dev@openvswitch.org Date: Mon, 27 Nov 2023 21:37:38 -0500 Message-ID: <20231128023738.569962-1-numans@ovn.org> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20231128023410.569539-1-numans@ovn.org> References: <20231128023410.569539-1-numans@ovn.org> MIME-Version: 1.0 X-GND-Sasl: numans@ovn.org Subject: [ovs-dev] [PATCH ovn v3 13/16] northd: Add ls_stateful handler for lflow engine node. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Numan Siddique Signed-off-by: Numan Siddique --- northd/en-lflow.c | 29 +++++++++++++++++++ northd/en-lflow.h | 1 + northd/en-ls-stateful.c | 3 ++ northd/en-ls-stateful.h | 23 +++++++++++++++ northd/inc-proc-northd.c | 3 +- northd/northd.c | 56 +++++++++++++++++++++++++++++++++---- northd/northd.h | 5 ++++ tests/ovn-northd.at | 60 ++++++++++++++++++++++++++-------------- 8 files changed, 152 insertions(+), 28 deletions(-) diff --git a/northd/en-lflow.c b/northd/en-lflow.c index 09748f570b..784c3c2efb 100644 --- a/northd/en-lflow.c +++ b/northd/en-lflow.c @@ -193,6 +193,35 @@ lflow_lr_stateful_handler(struct engine_node *node, void *data) return true; } +bool +lflow_ls_stateful_handler(struct engine_node *node, void *data) +{ + struct ed_type_ls_stateful *ls_sful_data = + engine_get_input_data("ls_stateful", node); + + if (!ls_stateful_has_tracked_data(&ls_sful_data->trk_data)) { + return false; + } + + const struct engine_context *eng_ctx = engine_get_context(); + struct lflow_data *lflow_data = data; + + struct lflow_input lflow_input; + lflow_get_input_data(node, &lflow_input); + + if (!lflow_handle_ls_stateful_changes(eng_ctx->ovnsb_idl_txn, + &ls_sful_data->trk_data, + &lflow_input, + lflow_data->lflow_table)) { + return false; + } + + + engine_set_node_state(node, EN_UPDATED); + + return true; +} + void *en_lflow_init(struct engine_node *node OVS_UNUSED, struct engine_arg *arg OVS_UNUSED) { diff --git a/northd/en-lflow.h b/northd/en-lflow.h index 1d813a2a29..32cae61763 100644 --- a/northd/en-lflow.h +++ b/northd/en-lflow.h @@ -21,5 +21,6 @@ void en_lflow_cleanup(void *data); bool lflow_northd_handler(struct engine_node *, void *data); bool lflow_port_group_handler(struct engine_node *, void *data); bool lflow_lr_stateful_handler(struct engine_node *, void *data); +bool lflow_ls_stateful_handler(struct engine_node *node, void *data); #endif /* EN_LFLOW_H */ diff --git a/northd/en-ls-stateful.c b/northd/en-ls-stateful.c index 117def531b..8b695e04fa 100644 --- a/northd/en-ls-stateful.c +++ b/northd/en-ls-stateful.c @@ -39,6 +39,7 @@ #include "lib/ovn-sb-idl.h" #include "lib/ovn-util.h" #include "lib/stopwatch-names.h" +#include "lflow-mgr.h" #include "northd.h" VLOG_DEFINE_THIS_MODULE(en_ls_stateful); @@ -298,6 +299,7 @@ ls_stateful_record_create(struct ls_stateful_table *table, struct ls_stateful_record *ls_sful_rec = xzalloc(sizeof *ls_sful_rec); ls_sful_rec->od = od; ls_stateful_record_init(ls_sful_rec, od, NULL, ls_pgs); + ls_sful_rec->lflow_ref = lflow_ref_alloc(od->nbs->name); hmap_insert(&table->entries, &ls_sful_rec->key_node, uuid_hash(&ls_sful_rec->od->nbs->header_.uuid)); @@ -308,6 +310,7 @@ ls_stateful_record_create(struct ls_stateful_table *table, static void ls_stateful_record_destroy(struct ls_stateful_record *ls_sful_rec) { + lflow_ref_destroy(ls_sful_rec->lflow_ref); free(ls_sful_rec); } diff --git a/northd/en-ls-stateful.h b/northd/en-ls-stateful.h index e409bb0141..5d7df849b9 100644 --- a/northd/en-ls-stateful.h +++ b/northd/en-ls-stateful.h @@ -31,6 +31,8 @@ #include "lib/ovn-util.h" #include "lib/stopwatch-names.h" +struct lflow_ref; + struct ls_stateful_record { struct hmap_node key_node; @@ -39,6 +41,27 @@ struct ls_stateful_record { bool has_lb_vip; bool has_acls; uint64_t max_acl_tier; + + /* 'lflow_ref' is used to reference logical flows generated for + * this ls_stateful record. + * + * This data is initialized and destroyed by the en_ls_stateful node, + * but populated and used only by the en_lflow node. Ideally this data + * should be maintained as part of en_lflow's data. However, it would + * be less efficient and more complex: + * + * 1. It would require an extra search (using the index) to find the + * lflows. + * + * 2. Building the index needs to be thread-safe, using either a global + * lock which is obviously less efficient, or hash-based lock array which + * is more complex. + * + * Adding the lflow_ref here is more straightforward. The drawback is that + * we need to keep in mind that this data belongs to en_lflow node, so + * never access it from any other nodes. + */ + struct lflow_ref *lflow_ref; }; struct ls_stateful_table { diff --git a/northd/inc-proc-northd.c b/northd/inc-proc-northd.c index 8be413200e..f7c3d2bcf5 100644 --- a/northd/inc-proc-northd.c +++ b/northd/inc-proc-northd.c @@ -228,12 +228,11 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, engine_add_input(&en_lflow, &en_sb_logical_flow, NULL); engine_add_input(&en_lflow, &en_sb_multicast_group, NULL); engine_add_input(&en_lflow, &en_sb_igmp_group, NULL); - engine_add_input(&en_lflow, &en_ls_stateful, NULL); engine_add_input(&en_lflow, &en_sb_logical_dp_group, NULL); - engine_add_input(&en_lflow, &en_ls_stateful, NULL); engine_add_input(&en_lflow, &en_northd, lflow_northd_handler); engine_add_input(&en_lflow, &en_port_group, lflow_port_group_handler); engine_add_input(&en_lflow, &en_lr_stateful, lflow_lr_stateful_handler); + engine_add_input(&en_lflow, &en_ls_stateful, lflow_ls_stateful_handler); engine_add_input(&en_sync_to_sb_addr_set, &en_nb_address_set, sync_to_sb_addr_set_nb_address_set_handler); diff --git a/northd/northd.c b/northd/northd.c index 235b9e100f..59b30c5dc8 100644 --- a/northd/northd.c +++ b/northd/northd.c @@ -15754,12 +15754,14 @@ build_ls_stateful_flows(const struct ls_stateful_record *ls_stateful_rec, ovs_assert(ls_stateful_rec->od); build_ls_stateful_rec_pre_acls(ls_stateful_rec, ls_pgs, lflows, - NULL); - build_ls_stateful_rec_pre_lb(ls_stateful_rec, lflows, NULL); - build_acl_hints(ls_stateful_rec, features, lflows, NULL); + ls_stateful_rec->lflow_ref); + build_ls_stateful_rec_pre_lb(ls_stateful_rec, lflows, + ls_stateful_rec->lflow_ref); + build_acl_hints(ls_stateful_rec, features, lflows, + ls_stateful_rec->lflow_ref); build_acls(ls_stateful_rec, features, lflows, ls_pgs, meter_groups, - NULL); - build_lb_hairpin(ls_stateful_rec, lflows, NULL); + ls_stateful_rec->lflow_ref); + build_lb_hairpin(ls_stateful_rec, lflows, ls_stateful_rec->lflow_ref); } struct lswitch_flow_build_info { @@ -16433,6 +16435,7 @@ void reset_lflow_refs_for_northd_resources(struct lflow_input *lflow_input) { const struct lr_stateful_record *lr_sful_rec; + struct ls_stateful_record *ls_sful_rec; struct ovn_lb_datapaths *lb_dps; struct ovn_port *op; @@ -16440,6 +16443,10 @@ reset_lflow_refs_for_northd_resources(struct lflow_input *lflow_input) lflow_ref_reset(lr_sful_rec->lflow_ref); } + LS_STATEFUL_TABLE_FOR_EACH (ls_sful_rec, lflow_input->ls_sful_table) { + lflow_ref_reset(ls_sful_rec->lflow_ref); + } + HMAP_FOR_EACH (op, key_node, lflow_input->ls_ports) { lflow_ref_reset(op->lflow_ref); lflow_ref_reset(op->stateful_lflow_ref); @@ -16749,6 +16756,45 @@ lflow_handle_lr_stateful_changes(struct ovsdb_idl_txn *ovnsb_txn, return true; } +bool +lflow_handle_ls_stateful_changes(struct ovsdb_idl_txn *ovnsb_txn, + struct ls_stateful_tracked_data *trk_data, + struct lflow_input *lflow_input, + struct lflow_table *lflows) +{ + struct ls_stateful_record *ls_sful_rec; + struct hmapx_node *hmapx_node; + + HMAPX_FOR_EACH (hmapx_node, &trk_data->crupdated) { + ls_sful_rec = hmapx_node->data; + + lflow_ref_clear_lflows(ls_sful_rec->lflow_ref); + + /* Generate new lflows. */ + struct ds match = DS_EMPTY_INITIALIZER; + struct ds actions = DS_EMPTY_INITIALIZER; + + build_ls_stateful_flows(ls_sful_rec, lflow_input->ls_port_groups, + lflow_input->features, + lflow_input->meter_groups, + lflows); + + ds_destroy(&match); + ds_destroy(&actions); + + /* Sync the new flows to SB. */ + lflow_ref_sync_lflows_to_sb(ls_sful_rec->lflow_ref, lflows, + ovnsb_txn, + lflow_input->ls_datapaths, + lflow_input->lr_datapaths, + lflow_input->ovn_internal_version_changed, + lflow_input->sbrec_logical_flow_table, + lflow_input->sbrec_logical_dp_group_table); + } + + return true; +} + static bool mirror_needs_update(const struct nbrec_mirror *nb_mirror, const struct sbrec_mirror *sb_mirror) diff --git a/northd/northd.h b/northd/northd.h index 6adf06c26f..c665092a09 100644 --- a/northd/northd.h +++ b/northd/northd.h @@ -720,6 +720,7 @@ void northd_indices_create(struct northd_data *data, struct lflow_table; struct lr_stateful_tracked_data; +struct ls_stateful_tracked_data; void build_lflows(struct ovsdb_idl_txn *ovnsb_txn, struct lflow_input *input_data, @@ -738,6 +739,10 @@ bool lflow_handle_lr_stateful_changes(struct ovsdb_idl_txn *, struct lr_stateful_tracked_data *, struct lflow_input *, struct lflow_table *lflows); +bool lflow_handle_ls_stateful_changes(struct ovsdb_idl_txn *, + struct ls_stateful_tracked_data *, + struct lflow_input *, + struct lflow_table *lflows); bool northd_handle_sb_port_binding_changes( const struct sbrec_port_binding_table *, struct hmap *ls_ports, struct hmap *lr_ports); diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at index f517057534..9b2886493c 100644 --- a/tests/ovn-northd.at +++ b/tests/ovn-northd.at @@ -10603,12 +10603,13 @@ check ovn-nbctl --wait=sb ls-lb-add sw0 lb1 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute -check_engine_stats lflow recompute nocompute # A LB applied to a switch/router triggers: # - a recompute in the first iteration (handling northd change) # - a compute in the second iteration (handling SB update) check_engine_stats sync_to_sb_lb recompute compute -CHECK_NO_CHANGE_AFTER_RECOMPUTE(1) +check_engine_stats ls_stateful norecompute compute +check_engine_stats lflow norecompute compute +CHECK_NO_CHANGE_AFTER_RECOMPUTE((1)) # Modify the backend of the lb1 vip check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats @@ -10616,7 +10617,8 @@ check ovn-nbctl --wait=sb set load_balancer lb1 vips:'"10.0.0.10:80"'='"10.0.0.1 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats ls_stateful norecompute compute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE(1) @@ -10626,7 +10628,8 @@ check ovn-nbctl --wait=sb clear load_Balancer lb1 vips check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats ls_stateful norecompute compute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE(1) @@ -10636,7 +10639,8 @@ check ovn-nbctl --wait=sb lb-add lb1 10.0.0.10:80 10.0.0.3:80 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats ls_stateful norecompute compute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE(1) @@ -10646,7 +10650,8 @@ check ovn-nbctl --wait=sb lb-add lb1 10.0.0.20:80 10.0.0.30:8080 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats ls_stateful norecompute compute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE(1) @@ -10656,6 +10661,7 @@ check ovn-nbctl --wait=sb ls-lb-del sw0 lb1 check_engine_stats lb_data norecompute compute check_engine_stats northd recompute nocompute check_engine_stats lr_stateful recompute nocompute +check_engine_stats ls_stateful recompute nocompute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE(1) @@ -10667,7 +10673,8 @@ check ovn-nbctl --wait=sb ls-lb-add sw0 lb1 -- lsp-add sw0 sw0p1 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats ls_stateful norecompute compute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10787,7 +10794,7 @@ check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute check_engine_stats ls_stateful norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute compute # Update lb and this should not result in northd recompute @@ -10795,8 +10802,9 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb set load_balancer . options:bar=foo check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_stateful norecompute compute check_engine_stats ls_stateful norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute compute # Modify the backend of the lb1 vip @@ -10806,7 +10814,7 @@ check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute check_engine_stats ls_stateful norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10816,7 +10824,8 @@ check ovn-nbctl --wait=sb clear load_Balancer lb1 vips check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats ls_stateful norecompute compute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10826,7 +10835,8 @@ check ovn-nbctl --wait=sb lb-add lb1 10.0.0.10:80 10.0.0.3:80 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats ls_stateful norecompute compute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10836,7 +10846,8 @@ check ovn-nbctl --wait=sb lb-add lb1 10.0.0.20:80 10.0.0.30:8080 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats ls_stateful norecompute compute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10911,7 +10922,8 @@ check ovn-nbctl --wait=sb add logical_switch sw0 load_balancer_group $lbg1_uuid check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats ls_stateful norecompute compute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10964,7 +10976,8 @@ check ovn-nbctl --wait=sb set logical_switch sw0 load_balancer_group=$lbg1_uuid check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats ls_stateful norecompute compute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10982,7 +10995,8 @@ check ovn-nbctl --wait=sb ls-lb-add sw0 lb2 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats ls_stateful norecompute compute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10991,7 +11005,8 @@ check ovn-nbctl --wait=sb ls-lb-add sw0 lb3 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats ls_stateful norecompute compute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11019,6 +11034,7 @@ check ovn-nbctl --wait=sb lr-lb-del lr1 lb2 check_engine_stats lb_data norecompute compute check_engine_stats northd recompute nocompute check_engine_stats lr_stateful recompute nocompute +check_engine_stats ls_stateful recompute nocompute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11030,7 +11046,8 @@ check ovn-nbctl --wait=sb lb-del lb4 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats ls_stateful norecompute compute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11041,7 +11058,8 @@ check ovn-nbctl --wait=sb lb-del lb2 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats ls_stateful norecompute compute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11075,7 +11093,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb ls-lb-add sw0 lb1 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Clear the VIPs of lb1 @@ -11083,7 +11101,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb clear load_balancer . vips check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats From patchwork Tue Nov 28 02:37:52 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Numan Siddique X-Patchwork-Id: 1869083 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::137; helo=smtp4.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp4.osuosl.org (smtp4.osuosl.org [IPv6:2605:bc80:3010::137]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4SfRTm0gGyz1yRy for ; Tue, 28 Nov 2023 13:38:36 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 9EFC64159F; Tue, 28 Nov 2023 02:38:33 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 9EFC64159F X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 8gI3TheQLbOk; Tue, 28 Nov 2023 02:38:30 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp4.osuosl.org (Postfix) with ESMTPS id 7D975410D8; Tue, 28 Nov 2023 02:38:29 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 7D975410D8 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 4A44FC0DCE; Tue, 28 Nov 2023 02:38:29 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp4.osuosl.org (smtp4.osuosl.org [IPv6:2605:bc80:3010::137]) by lists.linuxfoundation.org (Postfix) with ESMTP id 103D7C0039 for ; Tue, 28 Nov 2023 02:38:28 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 3779E4136C for ; Tue, 28 Nov 2023 02:38:08 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 3779E4136C X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id m6yo8dLum2Sc for ; Tue, 28 Nov 2023 02:38:05 +0000 (UTC) Received: from relay5-d.mail.gandi.net (relay5-d.mail.gandi.net [IPv6:2001:4b98:dc4:8::225]) by smtp4.osuosl.org (Postfix) with ESMTPS id 0B77440A00 for ; Tue, 28 Nov 2023 02:38:04 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 0B77440A00 Received: by mail.gandi.net (Postfix) with ESMTPSA id 3BF2B1C0005; Tue, 28 Nov 2023 02:38:02 +0000 (UTC) From: numans@ovn.org To: dev@openvswitch.org Date: Mon, 27 Nov 2023 21:37:52 -0500 Message-ID: <20231128023752.569985-1-numans@ovn.org> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20231128023410.569539-1-numans@ovn.org> References: <20231128023410.569539-1-numans@ovn.org> MIME-Version: 1.0 X-GND-Sasl: numans@ovn.org Subject: [ovs-dev] [PATCH ovn v3 14/16] northd: Add I-P for NB_Global and SB_Global. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Numan Siddique A new engine node "global_config" is added which handles the changes to NB_Global an SB_Global tables. It also creates these rows if not present. Without the I-P, any changes to the options column of these tables result in recompute of 'northd' and 'lflow' engine nodes. Signed-off-by: Numan Siddique --- northd/aging.c | 21 +- northd/automake.mk | 2 + northd/en-global-config.c | 588 ++++++++++++++++++++++++++++++++++++++ northd/en-global-config.h | 65 +++++ northd/en-lflow.c | 11 +- northd/en-northd.c | 52 ++-- northd/en-northd.h | 2 +- northd/en-sync-sb.c | 22 +- northd/inc-proc-northd.c | 38 ++- northd/northd.c | 230 +++------------ northd/northd.h | 24 +- tests/ovn-northd.at | 256 +++++++++++++++-- 12 files changed, 1014 insertions(+), 297 deletions(-) create mode 100644 northd/en-global-config.c create mode 100644 northd/en-global-config.h diff --git a/northd/aging.c b/northd/aging.c index f626c72c8c..cf988b39c4 100644 --- a/northd/aging.c +++ b/northd/aging.c @@ -15,6 +15,7 @@ #include +#include "en-global-config.h" #include "lib/inc-proc-eng.h" #include "lib/ovn-nb-idl.h" #include "lib/ovn-sb-idl.h" @@ -100,15 +101,10 @@ aging_context_handle_timestamp(struct aging_context *ctx, int64_t timestamp) static uint32_t get_removal_limit(struct engine_node *node, const char *name) { - const struct nbrec_nb_global_table *nb_global_table = - EN_OVSDB_GET(engine_get_input("NB_nb_global", node)); - const struct nbrec_nb_global *nb = - nbrec_nb_global_table_first(nb_global_table); - if (!nb) { - return 0; - } + struct ed_type_global_config *global_config = + engine_get_input_data("global_config", node); - return smap_get_uint(&nb->options, name, 0); + return smap_get_uint(&global_config->nb_options, name, 0); } /* MAC binding aging */ @@ -142,11 +138,14 @@ en_mac_binding_aging_run(struct engine_node *node, void *data OVS_UNUSED) { const struct engine_context *eng_ctx = engine_get_context(); struct northd_data *northd_data = engine_get_input_data("northd", node); + struct ed_type_global_config *global_config = + engine_get_input_data("global_config", node); + struct aging_waker *waker = engine_get_input_data("mac_binding_aging_waker", node); if (!eng_ctx->ovnsb_idl_txn || - !northd_data->features.mac_binding_timestamp || + !global_config->features.mac_binding_timestamp || time_msec() < waker->next_wake_msec) { return; } @@ -271,9 +270,11 @@ en_fdb_aging_run(struct engine_node *node, void *data OVS_UNUSED) const struct engine_context *eng_ctx = engine_get_context(); struct northd_data *northd_data = engine_get_input_data("northd", node); struct aging_waker *waker = engine_get_input_data("fdb_aging_waker", node); + struct ed_type_global_config *global_config = + engine_get_input_data("global_config", node); if (!eng_ctx->ovnsb_idl_txn || - !northd_data->features.fdb_timestamp || + !global_config->features.fdb_timestamp || time_msec() < waker->next_wake_msec) { return; } diff --git a/northd/automake.mk b/northd/automake.mk index aac6a8ecdd..534b31c4dd 100644 --- a/northd/automake.mk +++ b/northd/automake.mk @@ -8,6 +8,8 @@ northd_ovn_northd_SOURCES = \ northd/northd.c \ northd/northd.h \ northd/ovn-northd.c \ + northd/en-global-config.c \ + northd/en-global-config.h \ northd/en-northd.c \ northd/en-northd.h \ northd/en-lflow.c \ diff --git a/northd/en-global-config.c b/northd/en-global-config.c new file mode 100644 index 0000000000..0d218f2ab5 --- /dev/null +++ b/northd/en-global-config.c @@ -0,0 +1,588 @@ +/* + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at: + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#include + +#include +#include +#include + +/* OVS includes */ +#include "openvswitch/vlog.h" + +/* OVN includes */ +#include "debug.h" +#include "en-global-config.h" +#include "include/ovn/features.h" +#include "ipam.h" +#include "lib/ovn-nb-idl.h" +#include "lib/ovn-sb-idl.h" +#include "northd.h" + + +VLOG_DEFINE_THIS_MODULE(en_global_config); + +/* static function declarations. */ +static void northd_enable_all_features(struct ed_type_global_config *); +static void build_chassis_features(const struct sbrec_chassis_table *, + struct chassis_features *); +static bool chassis_features_changed(const struct chassis_features *, + const struct chassis_features *); +static bool config_out_of_sync(const struct smap *config, + const struct smap *saved_config, + const char *key, bool must_be_present); +static bool check_nb_options_out_of_sync(const struct nbrec_nb_global *, + struct ed_type_global_config *); +static void update_sb_config_options_to_sbrec(struct ed_type_global_config *, + const struct sbrec_sb_global *); + +void * +en_global_config_init(struct engine_node *node OVS_UNUSED, + struct engine_arg *args OVS_UNUSED) +{ + struct ed_type_global_config *data = xzalloc(sizeof *data); + smap_init(&data->nb_options); + smap_init(&data->sb_options); + northd_enable_all_features(data); + return data; +} + +void +en_global_config_run(struct engine_node *node , void *data) +{ + const struct engine_context *eng_ctx = engine_get_context(); + if (!eng_ctx->ovnnb_idl_txn || !eng_ctx->ovnsb_idl_txn) { + return; + } + + const struct nbrec_nb_global_table *nb_global_table = + EN_OVSDB_GET(engine_get_input("NB_nb_global", node)); + const struct sbrec_sb_global_table *sb_global_table = + EN_OVSDB_GET(engine_get_input("SB_sb_global", node)); + const struct sbrec_chassis_table *sbrec_chassis_table = + EN_OVSDB_GET(engine_get_input("SB_chassis", node)); + + en_global_config_clear_tracked_data(data); + + struct ed_type_global_config *config_data = data; + + /* Sync ipsec configuration. + * Copy nb_cfg from northbound to southbound database. + * Also set up to update sb_cfg once our southbound transaction commits. */ + const struct nbrec_nb_global *nb = + nbrec_nb_global_table_first(nb_global_table); + if (!nb) { + nb = nbrec_nb_global_insert(eng_ctx->ovnnb_idl_txn); + } + + const char *mac_addr_prefix = set_mac_prefix(smap_get(&nb->options, + "mac_prefix")); + + const char *monitor_mac = smap_get(&nb->options, "svc_monitor_mac"); + if (monitor_mac) { + if (eth_addr_from_string(monitor_mac, + &config_data->svc_monitor_mac_ea)) { + snprintf(config_data->svc_monitor_mac, + sizeof config_data->svc_monitor_mac, + ETH_ADDR_FMT, + ETH_ADDR_ARGS(config_data->svc_monitor_mac_ea)); + } else { + monitor_mac = NULL; + } + } + + struct smap *options = &config_data->nb_options; + smap_destroy(options); + smap_clone(options, &nb->options); + + smap_replace(options, "mac_prefix", mac_addr_prefix); + + if (!monitor_mac) { + eth_addr_random(&config_data->svc_monitor_mac_ea); + snprintf(config_data->svc_monitor_mac, + sizeof config_data->svc_monitor_mac, ETH_ADDR_FMT, + ETH_ADDR_ARGS(config_data->svc_monitor_mac_ea)); + smap_replace(options, "svc_monitor_mac", + config_data->svc_monitor_mac); + } + + char *max_tunid = xasprintf("%d", + get_ovn_max_dp_key_local(sbrec_chassis_table)); + smap_replace(options, "max_tunid", max_tunid); + free(max_tunid); + + char *ovn_internal_version = ovn_get_internal_version(); + if (strcmp(ovn_internal_version, + smap_get_def(options, "northd_internal_version", ""))) { + smap_replace(options, "northd_internal_version", + ovn_internal_version); + config_data->ovn_internal_version_changed = true; + } else { + config_data->ovn_internal_version_changed = false; + } + + free(ovn_internal_version); + + if (!smap_equal(&nb->options, options)) { + nbrec_nb_global_verify_options(nb); + nbrec_nb_global_set_options(nb, options); + } + + if (smap_get_bool(&nb->options, "ignore_chassis_features", false)) { + northd_enable_all_features(config_data); + } else { + build_chassis_features(sbrec_chassis_table, &config_data->features); + } + + init_debug_config(nb); + + const struct sbrec_sb_global *sb = + sbrec_sb_global_table_first(sb_global_table); + if (!sb) { + sb = sbrec_sb_global_insert(eng_ctx->ovnsb_idl_txn); + } + if (nb->ipsec != sb->ipsec) { + sbrec_sb_global_set_ipsec(sb, nb->ipsec); + } + + /* Set up SB_Global (depends on chassis features). */ + update_sb_config_options_to_sbrec(config_data, sb); + + engine_set_node_state(node, EN_UPDATED); +} + +void en_global_config_cleanup(void *data OVS_UNUSED) +{ + struct ed_type_global_config *config_data = data; + smap_destroy(&config_data->nb_options); + smap_destroy(&config_data->sb_options); + destroy_debug_config(); +} + +void +en_global_config_clear_tracked_data(void *data) +{ + struct ed_type_global_config *config_data = data; + config_data->tracked = false; + config_data->tracked_data.nb_options_changed = false; + config_data->tracked_data.chassis_features_changed = false; +} + +bool +global_config_nb_global_handler(struct engine_node *node, void *data) +{ + const struct nbrec_nb_global_table *nb_global_table = + EN_OVSDB_GET(engine_get_input("NB_nb_global", node)); + const struct sbrec_sb_global_table *sb_global_table = + EN_OVSDB_GET(engine_get_input("SB_sb_global", node)); + + const struct nbrec_nb_global *nb = + nbrec_nb_global_table_first(nb_global_table); + if (!nb) { + return false; + } + + const struct sbrec_sb_global *sb = + sbrec_sb_global_table_first(sb_global_table); + if (!sb) { + return false; + } + + /* We are only interested in ipsec and options column. */ + bool changes_relevant = false; + if (nbrec_nb_global_is_updated(nb, NBREC_NB_GLOBAL_COL_IPSEC) + || nbrec_nb_global_is_updated(nb, NBREC_NB_GLOBAL_COL_OPTIONS)) { + changes_relevant = true; + } + + if (!changes_relevant) { + return true; + } + + const struct engine_context *eng_ctx = engine_get_context(); + if (!eng_ctx->ovnsb_idl_txn) { + return false; + } + + if (nb->ipsec != sb->ipsec) { + sbrec_sb_global_set_ipsec(sb, nb->ipsec); + } + + struct ed_type_global_config *config_data = data; + config_data->tracked = true; + + if (smap_equal(&nb->options, &config_data->nb_options)) { + return true; + } + + /* Return false if an option is out of sync and requires updating the + * NB config. (Like svc_monitor_mac, max_tunid and mac_prefix). */ + /* Check if svc_monitor_mac has changed or not. */ + if (config_out_of_sync(&nb->options, &config_data->nb_options, + "svc_monitor_mac", true)) { + return false; + } + + /* Check if max_tunid has changed or not. */ + if (config_out_of_sync(&nb->options, &config_data->nb_options, + "max_tunid", true)) { + return false; + } + + /* Check if mac_prefix has changed or not. */ + if (config_out_of_sync(&nb->options, &config_data->nb_options, + "mac_prefix", true)) { + return false; + } + + /* Check if ignore_chassis_features has changed or not. */ + if (config_out_of_sync(&nb->options, &config_data->nb_options, + "ignore_chassis_features", false)) { + return false; + } + + /* Check if northd_internal_version has changed or not. */ + if (config_out_of_sync(&nb->options, &config_data->nb_options, + "northd_internal_version", false)) { + return false; + } + + if (check_nb_options_out_of_sync(nb, config_data)) { + config_data->tracked_data.nb_options_changed = true; + } + + smap_destroy(&config_data->nb_options); + smap_clone(&config_data->nb_options, &nb->options); + + update_sb_config_options_to_sbrec(config_data, sb); + + engine_set_node_state(node, EN_UPDATED); + return true; +} + +bool +global_config_sb_global_handler(struct engine_node *node, void *data) +{ + const struct sbrec_sb_global_table *sb_global_table = + EN_OVSDB_GET(engine_get_input("SB_sb_global", node)); + + const struct sbrec_sb_global *sb = + sbrec_sb_global_table_first(sb_global_table); + if (!sb) { + return false; + } + + struct ed_type_global_config *config_data = data; + + if (!smap_equal(&sb->options, &config_data->sb_options)) { + return false; + } + + /* No need to update the engine node. */ + return true; +} + +bool +global_config_sb_chassis_handler(struct engine_node *node, void *data) +{ + struct ed_type_global_config *config_data = data; + + const struct sbrec_chassis_table *sbrec_chassis_table = + EN_OVSDB_GET(engine_get_input("SB_chassis", node)); + const struct sbrec_chassis *chassis; + + SBREC_CHASSIS_TABLE_FOR_EACH_TRACKED (chassis, sbrec_chassis_table) { + if (sbrec_chassis_is_new(chassis) + || sbrec_chassis_is_deleted(chassis) + || sbrec_chassis_is_updated(chassis, + SBREC_CHASSIS_COL_ENCAPS)) { + return false; + } + + for (size_t i = 0; i < chassis->n_encaps; i++) { + if (sbrec_encap_row_get_seqno(chassis->encaps[i], + OVSDB_IDL_CHANGE_MODIFY) > 0) { + return false; + } + } + } + + if (smap_get_bool(&config_data->nb_options, "ignore_chassis_features", + false)) { + return true; + } + + bool reevaluate_chassis_features = false; + + /* Check and evaluate chassis features. */ + SBREC_CHASSIS_TABLE_FOR_EACH_TRACKED (chassis, sbrec_chassis_table) { + if (sbrec_chassis_is_updated(chassis, + SBREC_CHASSIS_COL_OTHER_CONFIG)) { + reevaluate_chassis_features = true; + break; + } + } + + if (!reevaluate_chassis_features) { + return true; + } + + struct chassis_features present_features = config_data->features; + + /* Enable all features before calling build_chassis_features() as + * build_chassis_features() only sets the feature flags to false. */ + northd_enable_all_features(config_data); + build_chassis_features(sbrec_chassis_table, &config_data->features); + + if (chassis_features_changed(&present_features, &config_data->features)) { + config_data->tracked_data.chassis_features_changed = true; + config_data->tracked = true; + engine_set_node_state(node, EN_UPDATED); + } + + return true; +} + +/* generic global config handler for any engine node which has global_config + * has an input node . */ +bool +node_global_config_handler(struct engine_node *node, void *data OVS_UNUSED) +{ + struct ed_type_global_config *global_config = + engine_get_input_data("global_config", node); + + if (!global_config->tracked + || global_config->tracked_data.chassis_features_changed + || global_config->tracked_data.nb_options_changed) { + return false; + } + + return true; +} + +/* static functions. */ +static void +northd_enable_all_features(struct ed_type_global_config *data) +{ + data->features = (struct chassis_features) { + .ct_no_masked_label = true, + .mac_binding_timestamp = true, + .ct_lb_related = true, + .fdb_timestamp = true, + .ls_dpg_column = true, + }; +} + +static void +build_chassis_features(const struct sbrec_chassis_table *sbrec_chassis_table, + struct chassis_features *chassis_features) +{ + const struct sbrec_chassis *chassis; + + SBREC_CHASSIS_TABLE_FOR_EACH (chassis, sbrec_chassis_table) { + /* Only consider local AZ chassis. Remote ones don't install + * flows generated by the local northd. + */ + if (smap_get_bool(&chassis->other_config, "is-remote", false)) { + continue; + } + + bool ct_no_masked_label = + smap_get_bool(&chassis->other_config, + OVN_FEATURE_CT_NO_MASKED_LABEL, + false); + if (!ct_no_masked_label && chassis_features->ct_no_masked_label) { + chassis_features->ct_no_masked_label = false; + } + + bool mac_binding_timestamp = + smap_get_bool(&chassis->other_config, + OVN_FEATURE_MAC_BINDING_TIMESTAMP, + false); + if (!mac_binding_timestamp && + chassis_features->mac_binding_timestamp) { + chassis_features->mac_binding_timestamp = false; + } + + bool ct_lb_related = + smap_get_bool(&chassis->other_config, + OVN_FEATURE_CT_LB_RELATED, + false); + if (!ct_lb_related && + chassis_features->ct_lb_related) { + chassis_features->ct_lb_related = false; + } + + bool fdb_timestamp = + smap_get_bool(&chassis->other_config, + OVN_FEATURE_FDB_TIMESTAMP, + false); + if (!fdb_timestamp && + chassis_features->fdb_timestamp) { + chassis_features->fdb_timestamp = false; + } + + bool ls_dpg_column = + smap_get_bool(&chassis->other_config, + OVN_FEATURE_LS_DPG_COLUMN, + false); + if (!ls_dpg_column && + chassis_features->ls_dpg_column) { + chassis_features->ls_dpg_column = false; + } + } +} + +static bool +config_out_of_sync(const struct smap *config, const struct smap *saved_config, + const char *key, bool must_be_present) +{ + const char *value = smap_get(config, key); + if (!value && must_be_present) { + return true; + } + + const char *saved_value = smap_get(saved_config, key); + if (!saved_value && must_be_present) { + return true; + } + + if (!value && !saved_value) { + return false; + } + + if ((!value && saved_value) || (value && !saved_value)) { + return true; + } + + return strcmp(value, saved_value); +} + +static bool +check_nb_options_out_of_sync(const struct nbrec_nb_global *nb, + struct ed_type_global_config *config_data) +{ + if (config_out_of_sync(&nb->options, &config_data->nb_options, + "mac_binding_removal_limit", false)) { + return true; + } + + if (config_out_of_sync(&nb->options, &config_data->nb_options, + "fdb_removal_limit", false)) { + return true; + } + + if (config_out_of_sync(&nb->options, &config_data->nb_options, + "controller_event", false)) { + return true; + } + + if (config_out_of_sync(&nb->options, &config_data->nb_options, + "ignore_lsp_down", false)) { + return true; + } + + if (config_out_of_sync(&nb->options, &config_data->nb_options, + "use_ct_inv_match", false)) { + return true; + } + + if (config_out_of_sync(&nb->options, &config_data->nb_options, + "default_acl_drop", false)) { + return true; + } + + if (config_out_of_sync(&nb->options, &config_data->nb_options, + "debug_drop_domain_id", false)) { + init_debug_config(nb); + return true; + } + + if (config_out_of_sync(&nb->options, &config_data->nb_options, + "debug_drop_collector_set", false)) { + init_debug_config(nb); + return true; + } + + if (config_out_of_sync(&nb->options, &config_data->nb_options, + "use_common_zone", false)) { + return true; + } + + if (config_out_of_sync(&nb->options, &config_data->nb_options, + "install_ls_lb_from_router", false)) { + return true; + } + + return false; +} + +static void +update_sb_config_options_to_sbrec(struct ed_type_global_config *config_data, + const struct sbrec_sb_global *sb) +{ + struct smap *options = &config_data->sb_options; + + smap_destroy(options); + smap_clone(options, &config_data->nb_options); + + /* Inform ovn-controllers whether LB flows will use ct_mark (i.e., only + * if all chassis support it). If not explicitly present in the database + * the default value to be used for this option is 'true'. + */ + if (!config_data->features.ct_no_masked_label) { + smap_replace(options, "lb_hairpin_use_ct_mark", "false"); + } else { + smap_remove(options, "lb_hairpin_use_ct_mark"); + } + + /* Hackaround SB_global.options overwrite by NB_Global.options for + * 'sbctl_probe_interval' option. + */ + const char *sip = smap_get(&sb->options, "sbctl_probe_interval"); + if (sip) { + smap_replace(options, "sbctl_probe_interval", sip); + } + + if (!smap_equal(&sb->options, options)) { + sbrec_sb_global_set_options(sb, options); + } +} + +static bool +chassis_features_changed(const struct chassis_features *present, + const struct chassis_features *updated) +{ + if (present->ct_no_masked_label != updated->ct_no_masked_label) { + return true; + } + + if (present->mac_binding_timestamp != updated->mac_binding_timestamp) { + return true; + } + + if (present->ct_lb_related != updated->ct_lb_related) { + return true; + } + + if (present->fdb_timestamp != updated->fdb_timestamp) { + return true; + } + + if (present->ls_dpg_column != updated->ls_dpg_column) { + return true; + } + + return false; +} diff --git a/northd/en-global-config.h b/northd/en-global-config.h new file mode 100644 index 0000000000..436bc7fa35 --- /dev/null +++ b/northd/en-global-config.h @@ -0,0 +1,65 @@ +#ifndef EN_GLOBAL_CONFIG_H +#define EN_GLOBAL_CONFIG_H 1 + +#include + +/* OVS includes. */ +#include "lib/packets.h" +#include "lib/smap.h" + +/* OVN includes. */ +#include "lib/inc-proc-eng.h" + +struct nbrec_nb_global; +struct sbrec_sb_global; + +struct chassis_features { + bool ct_no_masked_label; + bool mac_binding_timestamp; + bool ct_lb_related; + bool fdb_timestamp; + bool ls_dpg_column; +}; + +struct global_config_tracked_data { + bool nb_options_changed; + bool chassis_features_changed; +}; + +/* struct which maintains the data of the engine node global_config. */ +struct ed_type_global_config { + struct smap nb_options; + struct smap sb_options; + const struct nbrec_nb_global *nb_global; + const struct sbrec_sb_global *sb_global; + + /* MAC allocated for service monitor usage. Just one mac is allocated + * for this purpose and ovn-controller's on each chassis will make use + * of this mac when sending out the packets to monitor the services + * defined in Service_Monitor Southbound table. Since these packets + * are locally handled, having just one mac is good enough. */ + char svc_monitor_mac[ETH_ADDR_STRLEN + 1]; + struct eth_addr svc_monitor_mac_ea; + + struct chassis_features features; + + bool ovn_internal_version_changed; + + bool tracked; + struct global_config_tracked_data tracked_data; +}; + +void *en_global_config_init(struct engine_node *, struct engine_arg *); +void en_global_config_run(struct engine_node *, void *data); +void en_global_config_cleanup(void *data); +void en_global_config_clear_tracked_data(void *data); + +bool global_config_nb_global_handler(struct engine_node *, void *data); +bool global_config_sb_global_handler(struct engine_node *, void *data); +bool global_config_sb_chassis_handler(struct engine_node *, void *data); + +/* generic global config handler for any engine node which has global_config + * has an input node . */ +bool node_global_config_handler(struct engine_node *, void *data); + +#endif /* EN_GLOBAL_CONFIG_H */ diff --git a/northd/en-lflow.c b/northd/en-lflow.c index 784c3c2efb..1f4cf94c28 100644 --- a/northd/en-lflow.c +++ b/northd/en-lflow.c @@ -18,6 +18,7 @@ #include #include +#include "en-global-config.h" #include "en-lflow.h" #include "en-lr-nat.h" #include "en-lr-stateful.h" @@ -77,10 +78,14 @@ lflow_get_input_data(struct engine_node *node, lflow_input->meter_groups = &sync_meters_data->meter_groups; lflow_input->lb_datapaths_map = &northd_data->lb_datapaths_map; lflow_input->svc_monitor_map = &northd_data->svc_monitor_map; - lflow_input->features = &northd_data->features; - lflow_input->ovn_internal_version_changed = - northd_data->ovn_internal_version_changed; lflow_input->bfd_connections = NULL; + + struct ed_type_global_config *global_config = + engine_get_input_data("global_config", node); + lflow_input->features = &global_config->features; + lflow_input->ovn_internal_version_changed = + global_config->ovn_internal_version_changed; + lflow_input->svc_monitor_mac = global_config->svc_monitor_mac; } void en_lflow_run(struct engine_node *node, void *data) diff --git a/northd/en-northd.c b/northd/en-northd.c index 5143603f39..4479b4aff2 100644 --- a/northd/en-northd.c +++ b/northd/en-northd.c @@ -19,6 +19,7 @@ #include #include "coverage.h" +#include "en-global-config.h" #include "en-northd.h" #include "en-lb-data.h" #include "lib/inc-proc-eng.h" @@ -65,8 +66,6 @@ northd_get_input_data(struct engine_node *node, engine_get_input("SB_fdb", node), "sbrec_fdb_by_dp_and_port"); - input_data->nbrec_nb_global_table = - EN_OVSDB_GET(engine_get_input("NB_nb_global", node)); input_data->nbrec_logical_switch_table = EN_OVSDB_GET(engine_get_input("NB_logical_switch", node)); input_data->nbrec_logical_router_table = @@ -78,8 +77,6 @@ northd_get_input_data(struct engine_node *node, input_data->nbrec_mirror_table = EN_OVSDB_GET(engine_get_input("NB_mirror", node)); - input_data->sbrec_sb_global_table = - EN_OVSDB_GET(engine_get_input("SB_sb_global", node)); input_data->sbrec_datapath_binding_table = EN_OVSDB_GET(engine_get_input("SB_datapath_binding", node)); input_data->sbrec_port_binding_table = @@ -109,6 +106,14 @@ northd_get_input_data(struct engine_node *node, engine_get_input_data("lb_data", node); input_data->lbs = &lb_data->lbs; input_data->lbgrps = &lb_data->lbgrps; + + struct ed_type_global_config *global_config = + engine_get_input_data("global_config", node); + input_data->nb_options = &global_config->nb_options; + input_data->sb_options = &global_config->sb_options; + input_data->svc_monitor_mac = global_config->svc_monitor_mac; + input_data->svc_monitor_mac_ea = global_config->svc_monitor_mac_ea; + input_data->features = &global_config->features; } void @@ -129,31 +134,6 @@ en_northd_run(struct engine_node *node, void *data) eng_ctx->ovnsb_idl_txn); stopwatch_stop(OVNNB_DB_RUN_STOPWATCH_NAME, time_msec()); engine_set_node_state(node, EN_UPDATED); - -} - -bool -northd_nb_nb_global_handler(struct engine_node *node, - void *data OVS_UNUSED) -{ - const struct nbrec_nb_global_table *nb_global_table - = EN_OVSDB_GET(engine_get_input("NB_nb_global", node)); - - const struct nbrec_nb_global *nb = - nbrec_nb_global_table_first(nb_global_table); - - if (!nb) { - static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 1); - VLOG_WARN_RL(&rl, "NB_Global is updated but has no record."); - return false; - } - - /* We care about the 'options' and 'ipsec' columns only. */ - if (nbrec_nb_global_is_updated(nb, NBREC_NB_GLOBAL_COL_OPTIONS) || - nbrec_nb_global_is_updated(nb, NBREC_NB_GLOBAL_COL_IPSEC)) { - return false; - } - return true; } bool @@ -242,6 +222,20 @@ northd_lb_data_handler(struct engine_node *node, void *data) return true; } +bool +northd_global_config_handler(struct engine_node *node, void *data OVS_UNUSED) +{ + struct ed_type_global_config *global_config = + engine_get_input_data("global_config", node); + + if (!global_config->tracked + || global_config->tracked_data.nb_options_changed) { + return false; + } + + return true; +} + void *en_northd_init(struct engine_node *node OVS_UNUSED, struct engine_arg *arg OVS_UNUSED) diff --git a/northd/en-northd.h b/northd/en-northd.h index 5a88871760..9b7bda32aa 100644 --- a/northd/en-northd.h +++ b/northd/en-northd.h @@ -14,7 +14,7 @@ void *en_northd_init(struct engine_node *node OVS_UNUSED, struct engine_arg *arg); void en_northd_cleanup(void *data); void en_northd_clear_tracked_data(void *data); -bool northd_nb_nb_global_handler(struct engine_node *, void *data OVS_UNUSED); +bool northd_global_config_handler(struct engine_node *, void *data OVS_UNUSED); bool northd_nb_logical_switch_handler(struct engine_node *, void *data); bool northd_nb_logical_router_handler(struct engine_node *, void *data); bool northd_sb_port_binding_handler(struct engine_node *, void *data); diff --git a/northd/en-sync-sb.c b/northd/en-sync-sb.c index 733fb9024e..73b30272c4 100644 --- a/northd/en-sync-sb.c +++ b/northd/en-sync-sb.c @@ -22,6 +22,7 @@ #include "openvswitch/util.h" #include "en-lr-nat.h" +#include "en-global-config.h" #include "en-lr-stateful.h" #include "en-sync-sb.h" #include "lib/inc-proc-eng.h" @@ -42,7 +43,8 @@ static void sync_addr_sets(struct ovsdb_idl_txn *ovnsb_txn, const struct nbrec_address_set_table *, const struct nbrec_port_group_table *, const struct sbrec_address_set_table *, - const struct lr_stateful_table *); + const struct lr_stateful_table *, + const char *svc_monitor_macp); static const struct sbrec_address_set *sb_address_set_lookup_by_name( struct ovsdb_idl_index *, const char *name); static void update_sb_addr_set(struct sorted_array *, @@ -90,9 +92,12 @@ en_sync_to_sb_addr_set_run(struct engine_node *node, void *data OVS_UNUSED) const struct engine_context *eng_ctx = engine_get_context(); const struct ed_type_lr_stateful *lr_stateful_data = engine_get_input_data("lr_stateful", node); + struct ed_type_global_config *global_config = + engine_get_input_data("global_config", node); sync_addr_sets(eng_ctx->ovnsb_idl_txn, nb_address_set_table, nb_port_group_table, sb_address_set_table, - &lr_stateful_data->lr_sful_table); + &lr_stateful_data->lr_sful_table, + global_config->svc_monitor_mac); engine_set_node_state(node, EN_UPDATED); } @@ -218,12 +223,14 @@ en_sync_to_sb_lb_run(struct engine_node *node, void *data OVS_UNUSED) { const struct sbrec_load_balancer_table *sb_load_balancer_table = EN_OVSDB_GET(engine_get_input("SB_load_balancer", node)); + struct ed_type_global_config *global_config = + engine_get_input_data("global_config", node); const struct engine_context *eng_ctx = engine_get_context(); struct northd_data *northd_data = engine_get_input_data("northd", node); sync_lbs(eng_ctx->ovnsb_idl_txn, sb_load_balancer_table, &northd_data->ls_datapaths, &northd_data->lr_datapaths, - &northd_data->lb_datapaths_map, &northd_data->features); + &northd_data->lb_datapaths_map, &global_config->features); engine_set_node_state(node, EN_UPDATED); } @@ -369,7 +376,8 @@ sync_addr_sets(struct ovsdb_idl_txn *ovnsb_txn, const struct nbrec_address_set_table *nb_address_set_table, const struct nbrec_port_group_table *nb_port_group_table, const struct sbrec_address_set_table *sb_address_set_table, - const struct lr_stateful_table *lr_statefuls) + const struct lr_stateful_table *lr_statefuls, + const char *svc_monitor_macp) { struct shash sb_address_sets = SHASH_INITIALIZER(&sb_address_sets); @@ -380,8 +388,10 @@ sync_addr_sets(struct ovsdb_idl_txn *ovnsb_txn, } /* Service monitor MAC. */ - const char *svc_monitor_macp = northd_get_svc_monitor_mac(); - struct sorted_array svc = sorted_array_create(&svc_monitor_macp, 1, false); + struct sorted_array svc = { + .arr = &svc_monitor_macp, + .n = 1, + }; sync_addr_set(ovnsb_txn, "svc_monitor_mac", &svc, &sb_address_sets); sorted_array_destroy(&svc); diff --git a/northd/inc-proc-northd.c b/northd/inc-proc-northd.c index f7c3d2bcf5..7b1c6597e2 100644 --- a/northd/inc-proc-northd.c +++ b/northd/inc-proc-northd.c @@ -30,6 +30,7 @@ #include "openvswitch/poll-loop.h" #include "openvswitch/vlog.h" #include "inc-proc-northd.h" +#include "en-global-config.h" #include "en-lb-data.h" #include "en-lr-stateful.h" #include "en-lr-nat.h" @@ -149,6 +150,7 @@ static ENGINE_NODE(fdb_aging, "fdb_aging"); static ENGINE_NODE(fdb_aging_waker, "fdb_aging_waker"); static ENGINE_NODE(sync_to_sb_lb, "sync_to_sb_lb"); static ENGINE_NODE(sync_to_sb_pb, "sync_to_sb_pb"); +static ENGINE_NODE_WITH_CLEAR_TRACK_DATA(global_config, "global_config"); static ENGINE_NODE_WITH_CLEAR_TRACK_DATA(lb_data, "lb_data"); static ENGINE_NODE_WITH_CLEAR_TRACK_DATA(lr_nat, "lr_nat"); static ENGINE_NODE_WITH_CLEAR_TRACK_DATA(lr_stateful, "lr_stateful"); @@ -168,11 +170,17 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, engine_add_input(&en_lb_data, &en_nb_logical_router, lb_data_logical_router_handler); + engine_add_input(&en_global_config, &en_nb_nb_global, + global_config_nb_global_handler); + engine_add_input(&en_global_config, &en_sb_sb_global, + global_config_sb_global_handler); + engine_add_input(&en_global_config, &en_sb_chassis, + global_config_sb_chassis_handler); + engine_add_input(&en_northd, &en_nb_mirror, NULL); engine_add_input(&en_northd, &en_nb_static_mac_binding, NULL); engine_add_input(&en_northd, &en_nb_chassis_template_var, NULL); - engine_add_input(&en_northd, &en_sb_sb_global, NULL); engine_add_input(&en_northd, &en_sb_chassis, NULL); engine_add_input(&en_northd, &en_sb_mirror, NULL); engine_add_input(&en_northd, &en_sb_meter, NULL); @@ -185,11 +193,11 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, engine_add_input(&en_northd, &en_sb_fdb, NULL); engine_add_input(&en_northd, &en_sb_static_mac_binding, NULL); engine_add_input(&en_northd, &en_sb_chassis_template_var, NULL); + engine_add_input(&en_northd, &en_global_config, + northd_global_config_handler); engine_add_input(&en_northd, &en_sb_port_binding, northd_sb_port_binding_handler); - engine_add_input(&en_northd, &en_nb_nb_global, - northd_nb_nb_global_handler); engine_add_input(&en_northd, &en_nb_logical_switch, northd_nb_logical_switch_handler); engine_add_input(&en_northd, &en_nb_logical_router, @@ -207,15 +215,17 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, engine_add_input(&en_ls_stateful, &en_port_group, ls_stateful_port_group_handler); - engine_add_input(&en_mac_binding_aging, &en_nb_nb_global, NULL); engine_add_input(&en_mac_binding_aging, &en_sb_mac_binding, NULL); engine_add_input(&en_mac_binding_aging, &en_northd, NULL); engine_add_input(&en_mac_binding_aging, &en_mac_binding_aging_waker, NULL); + engine_add_input(&en_mac_binding_aging, &en_global_config, + node_global_config_handler); - engine_add_input(&en_fdb_aging, &en_nb_nb_global, NULL); engine_add_input(&en_fdb_aging, &en_sb_fdb, NULL); engine_add_input(&en_fdb_aging, &en_northd, NULL); engine_add_input(&en_fdb_aging, &en_fdb_aging_waker, NULL); + engine_add_input(&en_fdb_aging, &en_global_config, + node_global_config_handler); engine_add_input(&en_sync_meters, &en_nb_acl, NULL); engine_add_input(&en_sync_meters, &en_nb_meter, NULL); @@ -229,18 +239,22 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, engine_add_input(&en_lflow, &en_sb_multicast_group, NULL); engine_add_input(&en_lflow, &en_sb_igmp_group, NULL); engine_add_input(&en_lflow, &en_sb_logical_dp_group, NULL); + engine_add_input(&en_lflow, &en_global_config, + node_global_config_handler); engine_add_input(&en_lflow, &en_northd, lflow_northd_handler); engine_add_input(&en_lflow, &en_port_group, lflow_port_group_handler); engine_add_input(&en_lflow, &en_lr_stateful, lflow_lr_stateful_handler); engine_add_input(&en_lflow, &en_ls_stateful, lflow_ls_stateful_handler); + engine_add_input(&en_sync_to_sb_addr_set, &en_northd, NULL); + engine_add_input(&en_sync_to_sb_addr_set, &en_lr_stateful, NULL); + engine_add_input(&en_sync_to_sb_addr_set, &en_sb_address_set, NULL); engine_add_input(&en_sync_to_sb_addr_set, &en_nb_address_set, sync_to_sb_addr_set_nb_address_set_handler); engine_add_input(&en_sync_to_sb_addr_set, &en_nb_port_group, sync_to_sb_addr_set_nb_port_group_handler); - engine_add_input(&en_sync_to_sb_addr_set, &en_northd, NULL); - engine_add_input(&en_sync_to_sb_addr_set, &en_lr_stateful, NULL); - engine_add_input(&en_sync_to_sb_addr_set, &en_sb_address_set, NULL); + engine_add_input(&en_sync_to_sb_addr_set, &en_global_config, + node_global_config_handler); engine_add_input(&en_port_group, &en_nb_port_group, port_group_nb_port_group_handler); @@ -250,6 +264,8 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, * table too (because of the explicit dependency in the schema). */ engine_add_input(&en_port_group, &en_northd, engine_noop_handler); + engine_add_input(&en_sync_to_sb_lb, &en_global_config, + node_global_config_handler); engine_add_input(&en_sync_to_sb_lb, &en_northd, sync_to_sb_lb_northd_handler); engine_add_input(&en_sync_to_sb_lb, &en_sb_load_balancer, @@ -354,11 +370,11 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, "sbrec_fdb_by_dp_and_port", sbrec_fdb_by_dp_and_port); - struct northd_data *northd_data = - engine_get_internal_data(&en_northd); + struct ed_type_global_config *global_config = + engine_get_internal_data(&en_global_config); unixctl_command_register("debug/chassis-features-list", "", 0, 0, chassis_features_list, - &northd_data->features); + &global_config->features); } /* Returns true if the incremental processing ended up updating nodes. */ diff --git a/northd/northd.c b/northd/northd.c index 59b30c5dc8..e1eda9fba5 100644 --- a/northd/northd.c +++ b/northd/northd.c @@ -43,6 +43,7 @@ #include "lflow-mgr.h" #include "memory.h" #include "northd.h" +#include "en-global-config.h" #include "en-lb-data.h" #include "en-lr-nat.h" #include "en-lr-stateful.h" @@ -77,14 +78,6 @@ static bool install_ls_lb_from_router; /* Use common zone for SNAT and DNAT if this option is set to "true". */ static bool use_common_zone = false; -/* MAC allocated for service monitor usage. Just one mac is allocatedg5534 - * for this purpose and ovn-controller's on each chassis will make use - * of this mac when sending out the packets to monitor the services - * defined in Service_Monitor Southbound table. Since these packets - * all locally handled, having just one mac is good enough. */ -static char svc_monitor_mac[ETH_ADDR_STRLEN + 1]; -static struct eth_addr svc_monitor_mac_ea; - /* If this option is 'true' northd will make use of ct.inv match fields. * Otherwise, it will avoid using it. The default is true. */ static bool use_ct_inv_match = true; @@ -295,66 +288,6 @@ ovn_stage_to_datapath_type(enum ovn_stage stage) } } -static void -build_chassis_features(const struct sbrec_chassis_table *sbrec_chassis_table, - struct chassis_features *chassis_features) -{ - const struct sbrec_chassis *chassis; - - SBREC_CHASSIS_TABLE_FOR_EACH (chassis, sbrec_chassis_table) { - /* Only consider local AZ chassis. Remote ones don't install - * flows generated by the local northd. - */ - if (smap_get_bool(&chassis->other_config, "is-remote", false)) { - continue; - } - - bool ct_no_masked_label = - smap_get_bool(&chassis->other_config, - OVN_FEATURE_CT_NO_MASKED_LABEL, - false); - if (!ct_no_masked_label && chassis_features->ct_no_masked_label) { - chassis_features->ct_no_masked_label = false; - } - - bool mac_binding_timestamp = - smap_get_bool(&chassis->other_config, - OVN_FEATURE_MAC_BINDING_TIMESTAMP, - false); - if (!mac_binding_timestamp && - chassis_features->mac_binding_timestamp) { - chassis_features->mac_binding_timestamp = false; - } - - bool ct_lb_related = - smap_get_bool(&chassis->other_config, - OVN_FEATURE_CT_LB_RELATED, - false); - if (!ct_lb_related && - chassis_features->ct_lb_related) { - chassis_features->ct_lb_related = false; - } - - bool fdb_timestamp = - smap_get_bool(&chassis->other_config, - OVN_FEATURE_FDB_TIMESTAMP, - false); - if (!fdb_timestamp && - chassis_features->fdb_timestamp) { - chassis_features->fdb_timestamp = false; - } - - bool ls_dpg_column = - smap_get_bool(&chassis->other_config, - OVN_FEATURE_LS_DPG_COLUMN, - false); - if (!ls_dpg_column && - chassis_features->ls_dpg_column) { - chassis_features->ls_dpg_column = false; - } - } -} - static uint32_t allocate_queueid(unsigned long *queue_id_bitmap) { @@ -946,7 +879,7 @@ is_vxlan_mode(const struct sbrec_chassis_table *sbrec_chassis_table) return false; } -static uint32_t +uint32_t get_ovn_max_dp_key_local(const struct sbrec_chassis_table *sbrec_chassis_table) { if (is_vxlan_mode(sbrec_chassis_table)) { @@ -3354,6 +3287,8 @@ create_or_get_service_mon(struct ovsdb_idl_txn *ovnsb_txn, static void ovn_lb_svc_create(struct ovsdb_idl_txn *ovnsb_txn, const struct ovn_northd_lb *lb, + const char *svc_monitor_mac, + const struct eth_addr *svc_monitor_mac_ea, struct hmap *monitor_map, struct hmap *ls_ports, struct sset *svc_monitor_lsps) { @@ -3399,7 +3334,7 @@ ovn_lb_svc_create(struct ovsdb_idl_txn *ovnsb_txn, struct eth_addr ea; if (!mon_info->sbrec_mon->src_mac || !eth_addr_from_string(mon_info->sbrec_mon->src_mac, &ea) || - !eth_addr_equals(ea, svc_monitor_mac_ea)) { + !eth_addr_equals(ea, *svc_monitor_mac_ea)) { sbrec_service_monitor_set_src_mac(mon_info->sbrec_mon, svc_monitor_mac); } @@ -3724,6 +3659,8 @@ static void build_lb_svcs( struct ovsdb_idl_txn *ovnsb_txn, const struct sbrec_service_monitor_table *sbrec_service_monitor_table, + const char *svc_monitor_mac, + const struct eth_addr *svc_monitor_mac_ea, struct hmap *ls_ports, struct hmap *lb_dps_map, struct sset *svc_monitor_lsps, struct hmap *svc_monitor_map) @@ -3742,7 +3679,8 @@ build_lb_svcs( struct ovn_lb_datapaths *lb_dps; HMAP_FOR_EACH (lb_dps, hmap_node, lb_dps_map) { - ovn_lb_svc_create(ovnsb_txn, lb_dps->lb, svc_monitor_map, ls_ports, + ovn_lb_svc_create(ovnsb_txn, lb_dps->lb, svc_monitor_mac, + svc_monitor_mac_ea, svc_monitor_map, ls_ports, svc_monitor_lsps); } @@ -3834,13 +3772,16 @@ static void build_lb_port_related_data( struct ovsdb_idl_txn *ovnsb_txn, const struct sbrec_service_monitor_table *sbrec_service_monitor_table, + const char *svc_monitor_mac, + const struct eth_addr *svc_monitor_mac_ea, struct ovn_datapaths *lr_datapaths, struct hmap *ls_ports, struct hmap *lb_dps_map, struct hmap *lb_group_dps_map, struct sset *svc_monitor_lsps, struct hmap *svc_monitor_map) { build_lrouter_lbs_check(lr_datapaths); - build_lb_svcs(ovnsb_txn, sbrec_service_monitor_table, ls_ports, lb_dps_map, + build_lb_svcs(ovnsb_txn, sbrec_service_monitor_table, svc_monitor_mac, + svc_monitor_mac_ea, ls_ports, lb_dps_map, svc_monitor_lsps, svc_monitor_map); build_lswitch_lbs_from_lrouter(lr_datapaths, lb_dps_map, lb_group_dps_map); } @@ -9378,6 +9319,7 @@ build_lswitch_arp_nd_responder_default(struct ovn_datapath *od, static void build_lswitch_arp_nd_service_monitor(const struct ovn_northd_lb *lb, const struct hmap *ls_ports, + const char *svc_monitor_mac, struct lflow_table *lflows, struct ds *actions, struct ds *match, @@ -15783,6 +15725,7 @@ struct lswitch_flow_build_info { struct ds match; struct ds actions; size_t thread_lflow_counter; + const char *svc_monitor_mac; }; /* Helper function to combine all lflow generation which is iterated by @@ -16010,6 +15953,7 @@ build_lflows_thread(void *arg) } build_lswitch_arp_nd_service_monitor(lb_dps->lb, lsi->ls_ports, + lsi->svc_monitor_mac, lsi->lflows, &lsi->match, &lsi->actions, @@ -16133,7 +16077,8 @@ build_lswitch_and_lrouter_flows(const struct ovn_datapaths *ls_datapaths, const struct hmap *lb_dps_map, const struct hmap *svc_monitor_map, const struct hmap *bfd_connections, - const struct chassis_features *features) + const struct chassis_features *features, + const char *svc_monitor_mac) { char *svc_check_match = xasprintf("eth.dst == %s", svc_monitor_mac); @@ -16166,6 +16111,7 @@ build_lswitch_and_lrouter_flows(const struct ovn_datapaths *ls_datapaths, lsiv[index].features = features; lsiv[index].svc_check_match = svc_check_match; lsiv[index].thread_lflow_counter = 0; + lsiv[index].svc_monitor_mac = svc_monitor_mac; ds_init(&lsiv[index].match); ds_init(&lsiv[index].actions); @@ -16205,6 +16151,7 @@ build_lswitch_and_lrouter_flows(const struct ovn_datapaths *ls_datapaths, .bfd_connections = bfd_connections, .features = features, .svc_check_match = svc_check_match, + .svc_monitor_mac = svc_monitor_mac, .match = DS_EMPTY_INITIALIZER, .actions = DS_EMPTY_INITIALIZER, }; @@ -16247,6 +16194,7 @@ build_lswitch_and_lrouter_flows(const struct ovn_datapaths *ls_datapaths, stopwatch_start(LFLOWS_LBS_STOPWATCH_NAME, time_msec()); HMAP_FOR_EACH (lb_dps, hmap_node, lb_dps_map) { build_lswitch_arp_nd_service_monitor(lb_dps->lb, lsi.ls_ports, + lsi.svc_monitor_mac, lsi.lflows, &lsi.actions, &lsi.match, lb_dps->lflow_ref); @@ -16365,7 +16313,8 @@ void build_lflows(struct ovsdb_idl_txn *ovnsb_txn, input_data->lb_datapaths_map, input_data->svc_monitor_map, input_data->bfd_connections, - input_data->features); + input_data->features, + input_data->svc_monitor_mac); if (parallelization_state == STATE_INIT_HASH_SIZES) { parallelization_state = STATE_USE_PARALLELIZATION; @@ -16634,6 +16583,7 @@ lflow_handle_northd_lb_changes(struct ovsdb_idl_txn *ovnsb_txn, struct ds actions = DS_EMPTY_INITIALIZER; build_lswitch_arp_nd_service_monitor(lb_dps->lb, lflow_input->ls_ports, + lflow_input->svc_monitor_mac, lflows, &actions, &match, lb_dps->lflow_ref); build_lrouter_defrag_flows_for_lb(lb_dps, lflows, @@ -17411,18 +17361,6 @@ destroy_datapaths_and_ports(struct ovn_datapaths *ls_datapaths, ovn_datapaths_destroy(lr_datapaths); } -static void -northd_enable_all_features(struct northd_data *data) -{ - data->features = (struct chassis_features) { - .ct_no_masked_label = true, - .mac_binding_timestamp = true, - .ct_lb_related = true, - .fdb_timestamp = true, - .ls_dpg_column = true, - }; -} - void northd_init(struct northd_data *data) { @@ -17433,8 +17371,6 @@ northd_init(struct northd_data *data) hmap_init(&data->lb_datapaths_map); hmap_init(&data->lb_group_datapaths_map); ovs_list_init(&data->lr_list); - northd_enable_all_features(data); - data->ovn_internal_version_changed = false; sset_init(&data->svc_monitor_lsps); hmap_init(&data->svc_monitor_map); init_northd_tracked_data(data); @@ -17474,7 +17410,6 @@ northd_destroy(struct northd_data *data) destroy_datapaths_and_ports(&data->ls_datapaths, &data->lr_datapaths, &data->ls_ports, &data->lr_ports, &data->lr_list); - destroy_debug_config(); sset_destroy(&data->svc_monitor_lsps); destroy_northd_tracked_data(data); @@ -17491,83 +17426,22 @@ ovnnb_db_run(struct northd_input *input_data, } stopwatch_start(BUILD_LFLOWS_CTX_STOPWATCH_NAME, time_msec()); - /* Sync ipsec configuration. - * Copy nb_cfg from northbound to southbound database. - * Also set up to update sb_cfg once our southbound transaction commits. */ - const struct nbrec_nb_global *nb = nbrec_nb_global_table_first( - input_data->nbrec_nb_global_table); - if (!nb) { - nb = nbrec_nb_global_insert(ovnnb_txn); - } - - const char *mac_addr_prefix = set_mac_prefix(smap_get(&nb->options, - "mac_prefix")); - - const char *monitor_mac = smap_get(&nb->options, "svc_monitor_mac"); - if (monitor_mac) { - if (eth_addr_from_string(monitor_mac, &svc_monitor_mac_ea)) { - snprintf(svc_monitor_mac, sizeof svc_monitor_mac, - ETH_ADDR_FMT, ETH_ADDR_ARGS(svc_monitor_mac_ea)); - } else { - monitor_mac = NULL; - } - } - - struct smap options; - smap_clone(&options, &nb->options); - - smap_replace(&options, "mac_prefix", mac_addr_prefix); - - if (!monitor_mac) { - eth_addr_random(&svc_monitor_mac_ea); - snprintf(svc_monitor_mac, sizeof svc_monitor_mac, - ETH_ADDR_FMT, ETH_ADDR_ARGS(svc_monitor_mac_ea)); - smap_replace(&options, "svc_monitor_mac", svc_monitor_mac); - } - - char *max_tunid = xasprintf("%d", - get_ovn_max_dp_key_local(input_data->sbrec_chassis_table)); - smap_replace(&options, "max_tunid", max_tunid); - free(max_tunid); - - char *ovn_internal_version = ovn_get_internal_version(); - if (!strcmp(ovn_internal_version, - smap_get_def(&options, "northd_internal_version", ""))) { - data->ovn_internal_version_changed = false; - } else { - smap_replace(&options, "northd_internal_version", - ovn_internal_version); - } - free(ovn_internal_version); - - if (!smap_equal(&nb->options, &options)) { - nbrec_nb_global_verify_options(nb); - nbrec_nb_global_set_options(nb, &options); - } - - use_ct_inv_match = smap_get_bool(&nb->options, + use_ct_inv_match = smap_get_bool(input_data->nb_options, "use_ct_inv_match", true); /* deprecated, use --event instead */ - controller_event_en = smap_get_bool(&nb->options, + controller_event_en = smap_get_bool(input_data->nb_options, "controller_event", false); - check_lsp_is_up = !smap_get_bool(&nb->options, + check_lsp_is_up = !smap_get_bool(input_data->nb_options, "ignore_lsp_down", true); - default_acl_drop = smap_get_bool(&nb->options, "default_acl_drop", false); + default_acl_drop = smap_get_bool(input_data->nb_options, + "default_acl_drop", false); - install_ls_lb_from_router = smap_get_bool(&nb->options, + install_ls_lb_from_router = smap_get_bool(input_data->nb_options, "install_ls_lb_from_router", false); - use_common_zone = smap_get_bool(&nb->options, "use_common_zone", false); - - if (smap_get_bool(&nb->options, "ignore_chassis_features", false)) { - northd_enable_all_features(data); - } else { - build_chassis_features(input_data->sbrec_chassis_table, - &data->features); - } - - init_debug_config(nb); + use_common_zone = smap_get_bool(input_data->nb_options, "use_common_zone", + false); build_datapaths(ovnsb_txn, input_data->nbrec_logical_switch_table, @@ -17592,6 +17466,8 @@ ovnnb_db_run(struct northd_input *input_data, &data->ls_ports, &data->lr_ports); build_lb_port_related_data(ovnsb_txn, input_data->sbrec_service_monitor_table, + input_data->svc_monitor_mac, + &input_data->svc_monitor_mac_ea, &data->lr_datapaths, &data->ls_ports, &data->lb_datapaths_map, &data->lb_group_datapaths_map, @@ -17624,38 +17500,6 @@ ovnnb_db_run(struct northd_input *input_data, &data->ls_datapaths.datapaths); stopwatch_stop(CLEAR_LFLOWS_CTX_STOPWATCH_NAME, time_msec()); - /* Set up SB_Global (depends on chassis features). */ - const struct sbrec_sb_global *sb = sbrec_sb_global_table_first( - input_data->sbrec_sb_global_table); - if (!sb) { - sb = sbrec_sb_global_insert(ovnsb_txn); - } - if (nb->ipsec != sb->ipsec) { - sbrec_sb_global_set_ipsec(sb, nb->ipsec); - } - - /* Inform ovn-controllers whether LB flows will use ct_mark (i.e., only - * if all chassis support it). If not explicitly present in the database - * the default value to be used for this option is 'true'. - */ - if (!data->features.ct_no_masked_label) { - smap_replace(&options, "lb_hairpin_use_ct_mark", "false"); - } else { - smap_remove(&options, "lb_hairpin_use_ct_mark"); - } - - /* Hackaround SB_global.options overwrite by NB_Global.options for - * 'sbctl_probe_interval' option. - */ - const char *sip = smap_get(&sb->options, "sbctl_probe_interval"); - if (sip) { - smap_replace(&options, "sbctl_probe_interval", sip); - } - - if (!smap_equal(&sb->options, &options)) { - sbrec_sb_global_set_options(sb, &options); - } - smap_destroy(&options); } /* Stores the set of chassis which references an ha_chassis_group. @@ -17946,12 +17790,6 @@ ovnsb_db_run(struct ovsdb_idl_txn *ovnnb_txn, ovn_update_ipv6_prefix(lr_ports); } -const char * -northd_get_svc_monitor_mac(void) -{ - return svc_monitor_mac; -} - const struct ovn_datapath * northd_get_datapath_for_port(const struct hmap *ls_ports, const char *port_name) diff --git a/northd/northd.h b/northd/northd.h index c665092a09..7e48bb966d 100644 --- a/northd/northd.h +++ b/northd/northd.h @@ -27,7 +27,6 @@ struct northd_input { /* Northbound table references */ - const struct nbrec_nb_global_table *nbrec_nb_global_table; const struct nbrec_logical_switch_table *nbrec_logical_switch_table; const struct nbrec_logical_router_table *nbrec_logical_router_table; const struct nbrec_static_mac_binding_table @@ -37,7 +36,6 @@ struct northd_input { const struct nbrec_mirror_table *nbrec_mirror_table; /* Southbound table references */ - const struct sbrec_sb_global_table *sbrec_sb_global_table; const struct sbrec_datapath_binding_table *sbrec_datapath_binding_table; const struct sbrec_port_binding_table *sbrec_port_binding_table; const struct sbrec_mac_binding_table *sbrec_mac_binding_table; @@ -57,6 +55,13 @@ struct northd_input { const struct hmap *lbs; const struct hmap *lbgrps; + /* Global config data node inputs. */ + const struct smap *nb_options; + const struct smap *sb_options; + const char *svc_monitor_mac; + struct eth_addr svc_monitor_mac_ea; + const struct chassis_features *features; + /* Indexes */ struct ovsdb_idl_index *sbrec_chassis_by_name; struct ovsdb_idl_index *sbrec_chassis_by_hostname; @@ -66,14 +71,6 @@ struct northd_input { struct ovsdb_idl_index *sbrec_fdb_by_dp_and_port; }; -struct chassis_features { - bool ct_no_masked_label; - bool mac_binding_timestamp; - bool ct_lb_related; - bool fdb_timestamp; - bool ls_dpg_column; -}; - /* A collection of datapaths. E.g. all logical switch datapaths, or all * logical router datapaths. */ struct ovn_datapaths { @@ -208,8 +205,6 @@ struct northd_data { struct hmap lb_datapaths_map; struct hmap lb_group_datapaths_map; struct ovs_list lr_list; - bool ovn_internal_version_changed; - struct chassis_features features; struct sset svc_monitor_lsps; struct hmap svc_monitor_map; @@ -246,6 +241,7 @@ struct lflow_input { const struct chassis_features *features; const struct hmap *svc_monitor_map; bool ovn_internal_version_changed; + const char *svc_monitor_mac; }; extern int parallelization_state; @@ -764,8 +760,6 @@ void bfd_cleanup_connections(const struct nbrec_bfd_table *, struct hmap *bfd_map); void run_update_worker_pool(int n_threads); -const char *northd_get_svc_monitor_mac(void); - const struct ovn_datapath *northd_get_datapath_for_port( const struct hmap *ls_ports, const char *port_name); void sync_lbs(struct ovsdb_idl_txn *, const struct sbrec_load_balancer_table *, @@ -829,4 +823,6 @@ bool lrouter_port_ipv4_reachable(const struct ovn_port *, ovs_be32 addr); bool lrouter_port_ipv6_reachable(const struct ovn_port *, const struct in6_addr *); +uint32_t get_ovn_max_dp_key_local(const struct sbrec_chassis_table *); + #endif /* NORTHD_H */ diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at index 9b2886493c..03d525bdc2 100644 --- a/tests/ovn-northd.at +++ b/tests/ovn-northd.at @@ -8704,7 +8704,7 @@ AT_CHECK([grep "ls_in_lb " S1flows | sed 's/table=../table=??/' | sort], [0], [d table=??(ls_in_lb ), priority=0 , match=(1), action=(next;) ]) -ovn-nbctl --wait=sb set NB_Global . options:install_ls_lb_from_router=true +check ovn-nbctl --wait=sb set NB_Global . options:install_ls_lb_from_router=true ovn-sbctl dump-flows S0 > S0flows ovn-sbctl dump-flows S1 > S1flows @@ -8723,6 +8723,7 @@ AT_CHECK([grep "ls_in_lb " S1flows | sed 's/table=../table=??/' | sort], [0], [d table=??(ls_in_lb ), priority=120 , match=(ct.new && ip4.dst == 172.16.0.11 && tcp.dst == 8080), action=(reg0[[1]] = 0; ct_lb_mark(backends=10.0.0.2:8080);) ]) + ovn-sbctl get datapath S0 _uuid > dp_uuids ovn-sbctl get datapath S1 _uuid >> dp_uuids lb_dp_group=$(ovn-sbctl --bare --columns ls_datapath_group find Load_Balancer name=lb0) @@ -8731,7 +8732,7 @@ AT_CHECK_UNQUOTED([ovn-sbctl --bare --columns _uuid,datapaths find Logical_DP_Gr $(cat dp_uuids | sort) ]) -ovn-nbctl --wait=sb set NB_Global . options:install_ls_lb_from_router=false +check ovn-nbctl --wait=sb set NB_Global . options:install_ls_lb_from_router=false ovn-sbctl dump-flows S0 > S0flows ovn-sbctl dump-flows S1 > S1flows @@ -9088,12 +9089,11 @@ $4 AS_BOX([Create new PG1 and PG2]) check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb -- pg-add pg1 -- pg-add pg2 -dnl The northd node should not recompute, it should handle nb_global update -dnl though, therefore "compute: 1". +dnl The northd node should not recompute. AT_CHECK([as northd ovn-appctl -t NORTHD_TYPE inc-engine/show-stats northd], [0], [dnl Node: northd - recompute: 0 -- compute: 1 +- compute: 0 - abort: 0 ]) dnl The port_group node recomputes every time a NB port group is added/deleted. @@ -9126,12 +9126,11 @@ check ovn-nbctl --wait=sb \ check_column "sw1.1" sb:Port_Group ports name="${sw1_key}_pg1" check_column "sw2.1" sb:Port_Group ports name="${sw2_key}_pg1" -dnl The northd node should not recompute, it should handle nb_global update -dnl though, therefore "compute: 1". +dnl The northd node should not recompute. AT_CHECK([as northd ovn-appctl -t NORTHD_TYPE inc-engine/show-stats northd], [0], [dnl Node: northd - recompute: 0 -- compute: 1 +- compute: 0 - abort: 0 ]) dnl The port_group node recomputes also every time a port from a new switch @@ -9163,12 +9162,11 @@ check_column "sw2.1" sb:Port_Group ports name="${sw2_key}_pg1" check_column "sw1.2" sb:Port_Group ports name="${sw1_key}_pg2" check_column "sw2.2" sb:Port_Group ports name="${sw2_key}_pg2" -dnl The northd node should not recompute, it should handle nb_global update -dnl though, therefore "compute: 1". +dnl The northd node should not recompute. AT_CHECK([as northd ovn-appctl -t NORTHD_TYPE inc-engine/show-stats northd], [0], [dnl Node: northd - recompute: 0 -- compute: 1 +- compute: 0 - abort: 0 ]) dnl The port_group node recomputes also every time a port from a new switch @@ -9201,12 +9199,11 @@ check_column "sw2.1 sw2.3" sb:Port_Group ports name="${sw2_key}_pg1" check_column "sw1.2 sw1.3" sb:Port_Group ports name="${sw1_key}_pg2" check_column "sw2.2 sw2.3" sb:Port_Group ports name="${sw2_key}_pg2" -dnl The northd node should not recompute, it should handle nb_global update -dnl though, therefore "compute: 1". +dnl The northd node should not recompute. AT_CHECK([as northd ovn-appctl -t NORTHD_TYPE inc-engine/show-stats northd], [0], [dnl Node: northd - recompute: 0 -- compute: 1 +- compute: 0 - abort: 0 ]) dnl We did not change the set of switches a pg is applied to, there should be @@ -9244,7 +9241,7 @@ dnl though, therefore "compute: 1". AT_CHECK([as northd ovn-appctl -t NORTHD_TYPE inc-engine/show-stats northd], [0], [dnl Node: northd - recompute: 0 -- compute: 1 +- compute: 0 - abort: 0 ]) dnl We did not change the set of switches a pg is applied to, there should be @@ -9276,12 +9273,11 @@ check_column "sw1.2" sb:Port_Group ports name="${sw1_key}_pg2" AT_CHECK([fetch_column sb:Port_Group ports name="${sw2_key}_pg2"], [0], [ ]) -dnl The northd node should not recompute, it should handle nb_global update -dnl though, therefore "compute: 1". +dnl The northd node should not recompute. AT_CHECK([as northd ovn-appctl -t NORTHD_TYPE inc-engine/show-stats northd], [0], [dnl Node: northd - recompute: 0 -- compute: 1 +- compute: 0 - abort: 0 ]) dnl We changed the set of switches a pg is applied to, there should be @@ -9314,12 +9310,11 @@ check_column "sw1.2" sb:Port_Group ports name="${sw1_key}_pg2" AT_CHECK([fetch_column sb:Port_Group ports name="${sw2_key}_pg2"], [0], [ ]) -dnl The northd node should not recompute, it should handle nb_global update -dnl though, therefore "compute: 1". +dnl The northd node should not recompute. AT_CHECK([as northd ovn-appctl -t NORTHD_TYPE inc-engine/show-stats northd], [0], [dnl Node: northd - recompute: 0 -- compute: 1 +- compute: 0 - abort: 0 ]) dnl We changed the set of switches a pg is applied to, there should be @@ -9352,12 +9347,11 @@ check_column "sw2.1" sb:Port_Group ports name="${sw2_key}_pg1" check_column "sw1.2" sb:Port_Group ports name="${sw1_key}_pg2" check_column "sw2.2" sb:Port_Group ports name="${sw2_key}_pg2" -dnl The northd node should not recompute, it should handle nb_global update -dnl though, therefore "compute: 1". +dnl The northd node should not recompute. AT_CHECK([as northd ovn-appctl -t NORTHD_TYPE inc-engine/show-stats northd], [0], [dnl Node: northd - recompute: 0 -- compute: 1 +- compute: 0 - abort: 0 ]) dnl We changed the set of switches a pg is applied to, there should be a @@ -9392,12 +9386,11 @@ check_column "sw1.2" sb:Port_Group ports name="${sw1_key}_pg2" AT_CHECK([fetch_column sb:Port_Group ports name="${sw2_key}_pg2"], [0], [ ]) -dnl The northd node should not recompute, it should handle nb_global update -dnl though, therefore "compute: 1". +dnl The northd node should not recompute,. AT_CHECK([as northd ovn-appctl -t NORTHD_TYPE inc-engine/show-stats northd], [0], [dnl Node: northd - recompute: 0 -- compute: 1 +- compute: 0 - abort: 0 ]) dnl We changed the set of switches a pg is applied to, there should be a @@ -11343,3 +11336,212 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE OVN_CLEANUP([hv1]) AT_CLEANUP ]) + +OVN_FOR_EACH_NORTHD_NO_HV([ +AT_SETUP([NB_Global and SB_Global incremental processing]) + +ovn_start + +check_engine_stats() { + node=$1 + recompute=$2 + compute=$3 + + echo "__file__:__line__: Checking engine stats for node $node : recompute - \ +$recompute : compute - $compute" + + node_stat=$(as northd ovn-appctl -t NORTHD_TYPE inc-engine/show-stats $node) + # node_stat will be of this format : + # - Node: lflow - recompute: 3 - compute: 0 - abort: 0 + node_recompute_ct=$(echo $node_stat | cut -d '-' -f2 | cut -d ':' -f2) + node_compute_ct=$(echo $node_stat | cut -d '-' -f3 | cut -d ':' -f2) + + if [[ "$recompute" == "norecompute" ]]; then + # node should not be recomputed + echo "Expecting $node recompute count - $node_recompute_ct to be 0" + check test "$node_recompute_ct" -eq "0" + else + echo "Expecting $node recompute count - $node_recompute_ct not to be 0" + check test "$node_recompute_ct" -ne "0" + fi + + if [[ "$compute" == "nocompute" ]]; then + # node should not be computed + echo "Expecting $node compute count - $node_compute_ct to be 0" + check test "$node_compute_ct" -eq "0" + else + echo "Expecting $node compute count - $node_compute_ct not to be 0" + check test "$node_compute_ct" -ne "0" + fi +} + +check ovn-nbctl ls-add sw0 +check ovn-nbctl lr-add lr0 +check ovn-nbctl lsp-add sw0 sw0-p1 -- lsp-set-addresses sw0-p1 "00:00:20:20:00:03 10.0.0.3" +check ovn-nbctl lrp-add lr0 lr0-sw0 00:00:20:20:12:14 10.0.0.1/24 +check ovn-nbctl lsp-add sw0 sw0-lr0 +check ovn-nbctl lsp-set-type sw0-lr0 router +check ovn-nbctl lsp-set-addresses sw0-lr0 router +check ovn-nbctl --wait=sb lsp-set-options sw0-lr0 router-port=lr0-sw0 +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats + +# This should not result in recomputes. +check ovn-nbctl --wait=sb set NB_Global . options:foo=bar +check_engine_stats global_config norecompute compute +check_engine_stats northd norecompute compute +check_engine_stats lflow norecompute compute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +# This should result in recomputes. +check ovn-sbctl set SB_Global . options:bar=foo +check_engine_stats global_config recompute compute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +# Clears an nb option and checks that recomputes were triggered +# and the option was added back by ovn-northd or not depending +# on the 'added_back' argument. +clear_nb_option() { + option=$1 + add_back=$2 + echo "clearing the nb option - $option" + check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats + check ovn-nbctl --wait=sb remove NB_Global . options $option + check_engine_stats global_config recompute compute + check_engine_stats northd recompute nocompute + check_engine_stats lflow recompute nocompute + + local retval=1 + if [ "$add_back" == "true" ]; then + retval=0 + fi + AT_CHECK([ovn-nbctl get NB_Global . options:$option], [$retval], [ignore], [ignore]) +} + +# Clear svc_monitor_mac and few other options which result in recompute. +# and ovn-northd should update the nb options back. +clear_nb_option svc_monitor_mac true +clear_nb_option max_tunid true +clear_nb_option mac_prefix true +clear_nb_option northd_internal_version true + +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb set NB_Global . options:ignore_chassis_features=true +check_engine_stats global_config recompute compute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute + +clear_nb_option ignore_chassis_features false + +set_nb_option_lflow_recompute() { + local option=$1 + local value=$2 + check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats + check ovn-nbctl --wait=sb set NB_Global . options:$option=$value + check_engine_stats global_config norecompute compute + check_engine_stats northd recompute nocompute + check_engine_stats lflow recompute nocompute + check_engine_stats mac_binding_aging recompute nocompute + CHECK_NO_CHANGE_AFTER_RECOMPUTE +} + +clear_nb_option_lflow_recompute() { + local option=$1 + check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats + check ovn-nbctl --wait=sb remove NB_Global . options $option + check_engine_stats global_config norecompute compute + check_engine_stats northd recompute nocompute + check_engine_stats lflow recompute nocompute + check_engine_stats mac_binding_aging recompute nocompute + CHECK_NO_CHANGE_AFTER_RECOMPUTE +} + +set_nb_option_lflow_recompute debug_drop_domain_id 1 +clear_nb_option_lflow_recompute debug_drop_domain_id + +set_nb_option_lflow_recompute debug_drop_collector_set 1 +clear_nb_option_lflow_recompute debug_drop_collector_set + +set_nb_option_lflow_recompute mac_binding_removal_limit 100 +clear_nb_option_lflow_recompute mac_binding_removal_limit + +set_nb_option_lflow_recompute fdb_removal_limit 100 +clear_nb_option_lflow_recompute fdb_removal_limit + +set_nb_option_lflow_recompute controller_event true +clear_nb_option_lflow_recompute controller_event + +set_nb_option_lflow_recompute ignore_lsp_down true +clear_nb_option_lflow_recompute ignore_lsp_down + +set_nb_option_lflow_recompute use_ct_inv_match true +clear_nb_option_lflow_recompute use_ct_inv_match + +set_nb_option_lflow_recompute default_acl_drop true +clear_nb_option_lflow_recompute default_acl_drop + +set_nb_option_lflow_recompute use_common_zone true +clear_nb_option_lflow_recompute use_common_zone + +set_nb_option_lflow_recompute install_ls_lb_from_router true +clear_nb_option_lflow_recompute install_ls_lb_from_router + +# Now test changes to chassis for feature changes. +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-sbctl chassis-add ch1 geneve 127.0.0.1 +check ovn-nbctl --wait=sb sync +check_engine_stats global_config recompute compute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute + +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-sbctl chassis-add ch2 geneve 127.0.0.2 +check ovn-nbctl --wait=sb sync +check_engine_stats global_config recompute compute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute + +AT_CHECK([ovn-nbctl get NB_Global . options:max_tunid | \ +sed s/":"//g | sed s/\"//g], [0], [16711680 +], []) + +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-sbctl chassis-del ch2 +check ovn-nbctl --wait=sb sync +check_engine_stats global_config recompute compute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute + +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-sbctl set encap . type=vxlan +check ovn-nbctl --wait=sb sync +check_engine_stats global_config recompute compute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute + +AT_CHECK([ovn-nbctl get NB_Global . options:max_tunid | \ +sed s/":"//g | sed s/\"//g], [0], [4095 +], []) + +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-sbctl set chassis . other_config:foo=bar +check ovn-nbctl --wait=sb sync +check_engine_stats global_config norecompute compute +check_engine_stats mac_binding_aging recompute nocompute +check_engine_stats fdb_aging recompute nocompute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute + +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-sbctl set chassis . other_config:ct-no-masked-label=true +check ovn-nbctl --wait=sb sync +check_engine_stats global_config norecompute compute +check_engine_stats mac_binding_aging recompute nocompute +check_engine_stats fdb_aging recompute nocompute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute + +AT_CLEANUP +]) From patchwork Tue Nov 28 02:38:03 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Numan Siddique X-Patchwork-Id: 1869085 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::136; helo=smtp3.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4SfRVV0PvDz1yRy for ; Tue, 28 Nov 2023 13:39:14 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 3A93F61056; Tue, 28 Nov 2023 02:39:12 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 3A93F61056 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id swfeHbj-tHgG; Tue, 28 Nov 2023 02:39:11 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp3.osuosl.org (Postfix) with ESMTPS id 5F29761010; Tue, 28 Nov 2023 02:39:10 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 5F29761010 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 36459C0DCE; Tue, 28 Nov 2023 02:39:10 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by lists.linuxfoundation.org (Postfix) with ESMTP id C3070C0039 for ; Tue, 28 Nov 2023 02:39:09 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 708C281E0A for ; Tue, 28 Nov 2023 02:38:21 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 708C281E0A X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id yucOziSVUaW4 for ; Tue, 28 Nov 2023 02:38:21 +0000 (UTC) Received: from relay3-d.mail.gandi.net (relay3-d.mail.gandi.net [217.70.183.195]) by smtp1.osuosl.org (Postfix) with ESMTPS id A85E681E05 for ; Tue, 28 Nov 2023 02:38:20 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org A85E681E05 Received: by mail.gandi.net (Postfix) with ESMTPSA id 66C9A60005; Tue, 28 Nov 2023 02:38:18 +0000 (UTC) From: numans@ovn.org To: dev@openvswitch.org Date: Mon, 27 Nov 2023 21:38:03 -0500 Message-ID: <20231128023803.570013-1-numans@ovn.org> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20231128023410.569539-1-numans@ovn.org> References: <20231128023410.569539-1-numans@ovn.org> MIME-Version: 1.0 X-GND-Sasl: numans@ovn.org Subject: [ovs-dev] [PATCH ovn v3 15/16] northd: Add a noop handler for northd SB mac binding. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Numan Siddique Signed-off-by: Numan Siddique --- northd/inc-proc-northd.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/northd/inc-proc-northd.c b/northd/inc-proc-northd.c index 7b1c6597e2..28f397ff39 100644 --- a/northd/inc-proc-northd.c +++ b/northd/inc-proc-northd.c @@ -185,7 +185,6 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, engine_add_input(&en_northd, &en_sb_mirror, NULL); engine_add_input(&en_northd, &en_sb_meter, NULL); engine_add_input(&en_northd, &en_sb_datapath_binding, NULL); - engine_add_input(&en_northd, &en_sb_mac_binding, NULL); engine_add_input(&en_northd, &en_sb_dns, NULL); engine_add_input(&en_northd, &en_sb_ha_chassis_group, NULL); engine_add_input(&en_northd, &en_sb_ip_multicast, NULL); @@ -196,6 +195,14 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, engine_add_input(&en_northd, &en_global_config, northd_global_config_handler); + /* northd engine node uses the sb mac binding table to + * cleanup mac binding entries for deleted logical ports + * and datapaths. Any update to to SB mac binding doesn't + * change the northd engine node state or data. Hence + * it is ok to add a noop_handler here. */ + engine_add_input(&en_northd, &en_sb_mac_binding, + engine_noop_handler); + engine_add_input(&en_northd, &en_sb_port_binding, northd_sb_port_binding_handler); engine_add_input(&en_northd, &en_nb_logical_switch, From patchwork Tue Nov 28 02:38:19 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Numan Siddique X-Patchwork-Id: 1869086 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::133; helo=smtp2.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp2.osuosl.org (smtp2.osuosl.org [IPv6:2605:bc80:3010::133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4SfRVs4LLBz1yRy for ; Tue, 28 Nov 2023 13:39:33 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 55A5240AB7; Tue, 28 Nov 2023 02:39:30 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 55A5240AB7 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id EunGoxwlnRe2; Tue, 28 Nov 2023 02:39:27 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp2.osuosl.org (Postfix) with ESMTPS id 2FFC740A61; Tue, 28 Nov 2023 02:39:26 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 2FFC740A61 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id F3F27C0072; Tue, 28 Nov 2023 02:39:25 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp4.osuosl.org (smtp4.osuosl.org [IPv6:2605:bc80:3010::137]) by lists.linuxfoundation.org (Postfix) with ESMTP id 9BD63C0039 for ; Tue, 28 Nov 2023 02:39:24 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 2C65C4154A for ; Tue, 28 Nov 2023 02:38:43 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 2C65C4154A X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id jdkXlksO4FMg for ; Tue, 28 Nov 2023 02:38:38 +0000 (UTC) Received: from relay4-d.mail.gandi.net (relay4-d.mail.gandi.net [217.70.183.196]) by smtp4.osuosl.org (Postfix) with ESMTPS id 273F641554 for ; Tue, 28 Nov 2023 02:38:37 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 273F641554 Received: by mail.gandi.net (Postfix) with ESMTPSA id 6BCB7E0002; Tue, 28 Nov 2023 02:38:34 +0000 (UTC) From: numans@ovn.org To: dev@openvswitch.org Date: Mon, 27 Nov 2023 21:38:19 -0500 Message-ID: <20231128023819.570057-1-numans@ovn.org> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20231128023410.569539-1-numans@ovn.org> References: <20231128023410.569539-1-numans@ovn.org> MIME-Version: 1.0 X-GND-Sasl: numans@ovn.org Subject: [ovs-dev] [PATCH ovn v3 16/16] northd: Add northd change handler for sync_to_sb_lb node. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Numan Siddique Any changes to northd engine node due to load balancers are now handled in 'sync_to_sb_lb' node to sync the changed load balancers to SB load balancers. The logic to sync the SB load balancers is changed a bit and it now mimics the SB lflow sync. Below are the scale testing results done with all the patches applied in this series using ovn-heater. The test ran the scenario - ocp-500-density-heavy.yml [1]. The resuts are: ------------------------------------------------------------------------------------------------------------------------------------------------------- Min (s) Median (s) 90%ile (s) 99%ile (s) Max (s) Mean (s) Total (s) Count Failed ------------------------------------------------------------------------------------------------------------------------------------------------------- Iteration Total 0.136883 1.129016 1.192001 1.204167 1.212728 0.665017 83.127099 125 0 Namespace.add_ports 0.005216 0.005736 0.007034 0.015486 0.018978 0.006211 0.776373 125 0 WorkerNode.bind_port 0.035030 0.046082 0.052469 0.058293 0.060311 0.045973 11.493259 250 0 WorkerNode.ping_port 0.005057 0.006727 1.047692 1.069253 1.071336 0.266896 66.724094 250 0 ------------------------------------------------------------------------------------------------------------------------------------------------------- The results with the present main [2] are: ------------------------------------------------------------------------------------------------------------------------------------------------------- Min (s) Median (s) 90%ile (s) 99%ile (s) Max (s) Mean (s) Total (s) Count Failed ------------------------------------------------------------------------------------------------------------------------------------------------------- Iteration Total 0.135491 2.223805 3.311270 3.339078 3.345346 1.729172 216.146495 125 0 Namespace.add_ports 0.005380 0.005744 0.006819 0.018773 0.020800 0.006292 0.786532 125 0 WorkerNode.bind_port 0.034179 0.046055 0.053488 0.058801 0.071043 0.046117 11.529311 250 0 WorkerNode.ping_port 0.004956 0.006952 3.086952 3.191743 3.192807 0.791544 197.886026 250 0 ------------------------------------------------------------------------------------------------------------------------------------------------------- [1] - https://github.com/ovn-org/ovn-heater/blob/main/test-scenarios/ocp-500-density-heavy.yml [2] - 2a12cda890a7("controller, northd: Wait for cleanup before replying to exit") Signed-off-by: Numan Siddique --- northd/en-sync-sb.c | 445 +++++++++++++++++++++++++++++++++++++-- northd/inc-proc-northd.c | 1 + northd/lflow-mgr.c | 196 ++++++----------- northd/lflow-mgr.h | 23 +- northd/northd.c | 238 --------------------- northd/northd.h | 6 - tests/ovn-northd.at | 103 +++++---- 7 files changed, 585 insertions(+), 427 deletions(-) diff --git a/northd/en-sync-sb.c b/northd/en-sync-sb.c index 73b30272c4..62c5dbd20f 100644 --- a/northd/en-sync-sb.c +++ b/northd/en-sync-sb.c @@ -30,6 +30,7 @@ #include "lib/ovn-nb-idl.h" #include "lib/ovn-sb-idl.h" #include "lib/ovn-util.h" +#include "lflow-mgr.h" #include "northd.h" #include "openvswitch/vlog.h" @@ -53,6 +54,38 @@ static void build_port_group_address_set(const struct nbrec_port_group *, struct svec *ipv4_addrs, struct svec *ipv6_addrs); +struct sb_lb_table; +struct sb_lb_record; +static void sb_lb_table_init(struct sb_lb_table *); +static void sb_lb_table_clear(struct sb_lb_table *); +static struct sb_lb_record *sb_lb_table_find(struct hmap *sb_lbs, + const struct uuid *); +static void sb_lb_table_build_and_sync(struct sb_lb_table *, + struct ovsdb_idl_txn *ovnsb_txn, + const struct sbrec_load_balancer_table *, + const struct sbrec_logical_dp_group_table *, + struct hmap *lb_dps_map, + struct ovn_datapaths *ls_datapaths, + struct ovn_datapaths *lr_datapaths, + struct chassis_features *); +static void sync_sb_lb_record(struct sb_lb_record *, + const struct sbrec_load_balancer *, + const struct sbrec_logical_dp_group_table *, + struct sb_lb_table *, + struct ovsdb_idl_txn *ovnsb_txn, + struct ovn_datapaths *ls_datapaths, + struct ovn_datapaths *lr_datapaths, + struct chassis_features *); +static void sync_changed_lbs(struct sb_lb_table *, + struct ovsdb_idl_txn *ovnsb_txn, + const struct sbrec_load_balancer_table *, + const struct sbrec_logical_dp_group_table *, + struct tracked_lbs *, + struct ovn_datapaths *ls_datapaths, + struct ovn_datapaths *lr_datapaths, + struct chassis_features *); +static bool check_sb_lb_duplicates(const struct sbrec_load_balancer_table *); + void * en_sync_to_sb_init(struct engine_node *node OVS_UNUSED, struct engine_arg *arg OVS_UNUSED) @@ -211,51 +244,103 @@ sync_to_sb_addr_set_nb_port_group_handler(struct engine_node *node, /* sync_to_sb_lb engine node functions. * This engine node syncs the SB load balancers. */ +struct sb_lb_record { + struct hmap_node key_node; /* Index on 'nblb->header_.uuid'. */ + + struct ovn_lb_datapaths *lb_dps; + const struct sbrec_load_balancer *sbrec_lb; + struct ovn_dp_group *ls_dpg; + struct ovn_dp_group *lr_dpg; + struct uuid sb_uuid; +}; + +struct sb_lb_table { + struct hmap entries; /* Stores struct sb_lb_record. */ + struct hmap ls_dp_groups; + struct hmap lr_dp_groups; +}; + +struct ed_type_sync_to_sb_lb_data { + struct sb_lb_table sb_lbs; +}; + void * en_sync_to_sb_lb_init(struct engine_node *node OVS_UNUSED, struct engine_arg *arg OVS_UNUSED) { - return NULL; + struct ed_type_sync_to_sb_lb_data *data = xzalloc(sizeof *data); + sb_lb_table_init(&data->sb_lbs); + + return data; } void -en_sync_to_sb_lb_run(struct engine_node *node, void *data OVS_UNUSED) +en_sync_to_sb_lb_run(struct engine_node *node, void *data_) { + struct northd_data *northd_data = engine_get_input_data("northd", node); const struct sbrec_load_balancer_table *sb_load_balancer_table = EN_OVSDB_GET(engine_get_input("SB_load_balancer", node)); + const struct sbrec_logical_dp_group_table *sb_dpgrp_table = + EN_OVSDB_GET(engine_get_input("SB_logical_dp_group", node)); struct ed_type_global_config *global_config = engine_get_input_data("global_config", node); + const struct engine_context *eng_ctx = engine_get_context(); - struct northd_data *northd_data = engine_get_input_data("northd", node); + struct ed_type_sync_to_sb_lb_data *data = data_; + + sb_lb_table_clear(&data->sb_lbs); + sb_lb_table_init(&data->sb_lbs); + sb_lb_table_build_and_sync(&data->sb_lbs, eng_ctx->ovnsb_idl_txn, + sb_load_balancer_table, + sb_dpgrp_table, + &northd_data->lb_datapaths_map, + &northd_data->ls_datapaths, + &northd_data->lr_datapaths, + &global_config->features); - sync_lbs(eng_ctx->ovnsb_idl_txn, sb_load_balancer_table, - &northd_data->ls_datapaths, &northd_data->lr_datapaths, - &northd_data->lb_datapaths_map, &global_config->features); engine_set_node_state(node, EN_UPDATED); } void -en_sync_to_sb_lb_cleanup(void *data OVS_UNUSED) +en_sync_to_sb_lb_cleanup(void *data_) { - + struct ed_type_sync_to_sb_lb_data *data = data_; + sb_lb_table_clear(&data->sb_lbs); } bool -sync_to_sb_lb_northd_handler(struct engine_node *node, void *data OVS_UNUSED) +sync_to_sb_lb_northd_handler(struct engine_node *node, void *data_) { struct northd_data *nd = engine_get_input_data("northd", node); - if (!northd_has_tracked_data(&nd->trk_data) || - northd_has_lbs_in_tracked_data(&nd->trk_data)) { + if (!northd_has_tracked_data(&nd->trk_data)) { /* Return false if no tracking data or if lbs changed. */ return false; } + if (!northd_has_lbs_in_tracked_data(&nd->trk_data)) { + return true; + } + + const struct engine_context *eng_ctx = engine_get_context(); + if (!eng_ctx->ovnsb_idl_txn) { + return false; + } + + const struct sbrec_logical_dp_group_table *sb_dpgrp_table = + EN_OVSDB_GET(engine_get_input("SB_logical_dp_group", node)); + const struct sbrec_load_balancer_table *sb_lb_table = + EN_OVSDB_GET(engine_get_input("SB_load_balancer", node)); + struct ed_type_global_config *global_config = + engine_get_input_data("global_config", node); + struct ed_type_sync_to_sb_lb_data *data = data_; + + sync_changed_lbs(&data->sb_lbs, eng_ctx->ovnsb_idl_txn, sb_lb_table, + sb_dpgrp_table, &nd->trk_data.trk_lbs, + &nd->ls_datapaths, &nd->lr_datapaths, + &global_config->features); - /* There are only NB LSP related changes and these can be safely - * ignore and returned true. However in case the northd engine - * tracking data includes other changes, we need to do additional - * checks before safely ignoring. */ + engine_set_node_state(node, EN_UPDATED); return true; } @@ -529,3 +614,333 @@ sb_address_set_lookup_by_name(struct ovsdb_idl_index *sbrec_addr_set_by_name, return retval; } + +/* static functions related to sync_to_sb_lb */ + +static void +sb_lb_table_init(struct sb_lb_table *sb_lbs) +{ + hmap_init(&sb_lbs->entries); + ovn_dp_groups_init(&sb_lbs->ls_dp_groups); + ovn_dp_groups_init(&sb_lbs->lr_dp_groups); +} + +static void +sb_lb_table_clear(struct sb_lb_table *sb_lbs) +{ + struct sb_lb_record *sb_lb; + HMAP_FOR_EACH_POP (sb_lb, key_node, &sb_lbs->entries) { + free(sb_lb); + } + hmap_destroy(&sb_lbs->entries); + + ovn_dp_groups_destroy(&sb_lbs->ls_dp_groups); + ovn_dp_groups_destroy(&sb_lbs->lr_dp_groups); +} + +static struct sb_lb_record * +sb_lb_table_find(struct hmap *sb_lbs, const struct uuid *lb_uuid) +{ + struct sb_lb_record *sb_lb; + HMAP_FOR_EACH_WITH_HASH (sb_lb, key_node, uuid_hash(lb_uuid), + sb_lbs) { + if (uuid_equals(&sb_lb->lb_dps->lb->nlb->header_.uuid, lb_uuid)) { + return sb_lb; + } + } + + return NULL; +} + +static void +sb_lb_table_build_and_sync(struct sb_lb_table *sb_lbs, + struct ovsdb_idl_txn *ovnsb_txn, + const struct sbrec_load_balancer_table *sb_lb_table, + const struct sbrec_logical_dp_group_table *sb_dpgrp_table, + struct hmap *lb_dps_map, + struct ovn_datapaths *ls_datapaths, + struct ovn_datapaths *lr_datapaths, + struct chassis_features *chassis_features) +{ + struct hmap tmp_sb_lbs = HMAP_INITIALIZER(&tmp_sb_lbs); + struct ovn_lb_datapaths *lb_dps; + struct sb_lb_record *sb_lb; + + HMAP_FOR_EACH (lb_dps, hmap_node, lb_dps_map) { + if (!lb_dps->n_nb_ls && !lb_dps->n_nb_lr) { + continue; + } + + sb_lb = xzalloc(sizeof *sb_lb); + sb_lb->lb_dps = lb_dps; + hmap_insert(&tmp_sb_lbs, &sb_lb->key_node, + uuid_hash(&lb_dps->lb->nlb->header_.uuid)); + } + + const struct sbrec_load_balancer *sbrec_lb; + SBREC_LOAD_BALANCER_TABLE_FOR_EACH_SAFE (sbrec_lb, + sb_lb_table) { + const char *nb_lb_uuid = smap_get(&sbrec_lb->external_ids, "lb_id"); + struct uuid lb_uuid; + if (!nb_lb_uuid || !uuid_from_string(&lb_uuid, nb_lb_uuid)) { + sbrec_load_balancer_delete(sbrec_lb); + continue; + } + + sb_lb = sb_lb_table_find(&tmp_sb_lbs, &lb_uuid); + if (sb_lb) { + sb_lb->sbrec_lb = sbrec_lb; + sync_sb_lb_record(sb_lb, sbrec_lb, sb_dpgrp_table, sb_lbs, + ovnsb_txn, ls_datapaths, lr_datapaths, + chassis_features); + + hmap_remove(&tmp_sb_lbs, &sb_lb->key_node); + hmap_insert(&sb_lbs->entries, &sb_lb->key_node, + uuid_hash(&sb_lb->lb_dps->lb->nlb->header_.uuid)); + } else { + sbrec_load_balancer_delete(sbrec_lb); + } + } + + HMAP_FOR_EACH_POP (sb_lb, key_node, &tmp_sb_lbs) { + sync_sb_lb_record(sb_lb, NULL, sb_dpgrp_table, sb_lbs, + ovnsb_txn, ls_datapaths, lr_datapaths, + chassis_features); + hmap_insert(&sb_lbs->entries, &sb_lb->key_node, + uuid_hash(&sb_lb->lb_dps->lb->nlb->header_.uuid)); + } + + hmap_destroy(&tmp_sb_lbs); +} + +static void +sync_sb_lb_record(struct sb_lb_record *sb_lb, + const struct sbrec_load_balancer *sbrec_lb, + const struct sbrec_logical_dp_group_table *sb_dpgrp_table, + struct sb_lb_table *sb_lbs, + struct ovsdb_idl_txn *ovnsb_txn, + struct ovn_datapaths *ls_datapaths, + struct ovn_datapaths *lr_datapaths, + struct chassis_features *chassis_features) +{ + struct sbrec_logical_dp_group *sbrec_ls_dp_group = NULL; + struct sbrec_logical_dp_group *sbrec_lr_dp_group = NULL; + const struct ovn_lb_datapaths *lb_dps; + struct ovn_dp_group *pre_sync_ls_dpg; + struct ovn_dp_group *pre_sync_lr_dpg; + + lb_dps = sb_lb->lb_dps; + pre_sync_ls_dpg = sb_lb->ls_dpg; + pre_sync_lr_dpg = sb_lb->lr_dpg; + + if (!sbrec_lb) { + sb_lb->sb_uuid = uuid_random(); + sbrec_lb = sbrec_load_balancer_insert_persist_uuid(ovnsb_txn, + &sb_lb->sb_uuid); + char *lb_id = xasprintf( + UUID_FMT, UUID_ARGS(&lb_dps->lb->nlb->header_.uuid)); + const struct smap external_ids = + SMAP_CONST1(&external_ids, "lb_id", lb_id); + sbrec_load_balancer_set_external_ids(sbrec_lb, &external_ids); + free(lb_id); + } else { + sb_lb->sb_uuid = sbrec_lb->header_.uuid; + sbrec_ls_dp_group = + chassis_features->ls_dpg_column + ? sbrec_lb->ls_datapath_group + : sbrec_lb->datapath_group; /* deprecated */ + + sbrec_lr_dp_group = sbrec_lb->lr_datapath_group; + } + + if (lb_dps->n_nb_ls) { + sb_lb->ls_dpg = ovn_dp_group_get(&sb_lbs->ls_dp_groups, + lb_dps->n_nb_ls, + lb_dps->nb_ls_map, + ods_size(ls_datapaths)); + if (sb_lb->ls_dpg) { + /* Update the dpg's sb dp_group. */ + sb_lb->ls_dpg->dp_group = + sbrec_logical_dp_group_table_get_for_uuid(sb_dpgrp_table, + &sb_lb->ls_dpg->dpg_uuid); + ovs_assert(sb_lb->ls_dpg->dp_group); + } else { + sb_lb->ls_dpg = ovn_dp_group_create( + ovnsb_txn, &sb_lbs->ls_dp_groups, + sbrec_ls_dp_group, + lb_dps->n_nb_ls, lb_dps->nb_ls_map, + ods_size(ls_datapaths), true, + ls_datapaths, + lr_datapaths); + } + + if (chassis_features->ls_dpg_column) { + sbrec_load_balancer_set_ls_datapath_group(sbrec_lb, + sb_lb->ls_dpg->dp_group); + sbrec_load_balancer_set_datapath_group(sbrec_lb, NULL); + } else { + /* datapath_group column is deprecated. */ + sbrec_load_balancer_set_ls_datapath_group(sbrec_lb, NULL); + sbrec_load_balancer_set_datapath_group(sbrec_lb, + sb_lb->ls_dpg->dp_group); + + } + } else { + sbrec_load_balancer_set_ls_datapath_group(sbrec_lb, NULL); + sbrec_load_balancer_set_datapath_group(sbrec_lb, NULL); + } + + + if (lb_dps->n_nb_lr) { + sb_lb->lr_dpg = ovn_dp_group_get(&sb_lbs->lr_dp_groups, + lb_dps->n_nb_lr, + lb_dps->nb_lr_map, + ods_size(lr_datapaths)); + if (sb_lb->lr_dpg) { + /* Update the dpg's sb dp_group. */ + sb_lb->lr_dpg->dp_group = + sbrec_logical_dp_group_table_get_for_uuid(sb_dpgrp_table, + &sb_lb->lr_dpg->dpg_uuid); + ovs_assert(sb_lb->lr_dpg->dp_group); + } else { + sb_lb->lr_dpg = ovn_dp_group_create( + ovnsb_txn, &sb_lbs->lr_dp_groups, + sbrec_lr_dp_group, + lb_dps->n_nb_lr, lb_dps->nb_lr_map, + ods_size(lr_datapaths), false, + ls_datapaths, + lr_datapaths); + } + + sbrec_load_balancer_set_lr_datapath_group(sbrec_lb, + sb_lb->lr_dpg->dp_group); + } else { + sbrec_load_balancer_set_lr_datapath_group(sbrec_lb, NULL); + } + + if (pre_sync_ls_dpg != sb_lb->ls_dpg) { + if (sb_lb->ls_dpg) { + inc_ovn_dp_group_ref(sb_lb->ls_dpg); + } + if (pre_sync_ls_dpg) { + dec_ovn_dp_group_ref(&sb_lbs->ls_dp_groups, pre_sync_ls_dpg); + } + } + + if (pre_sync_lr_dpg != sb_lb->lr_dpg) { + if (sb_lb->lr_dpg) { + inc_ovn_dp_group_ref(sb_lb->lr_dpg); + } + if (pre_sync_lr_dpg) { + dec_ovn_dp_group_ref(&sb_lbs->lr_dp_groups, pre_sync_lr_dpg); + } + } + + /* Update columns. */ + sbrec_load_balancer_set_name(sbrec_lb, lb_dps->lb->nlb->name); + sbrec_load_balancer_set_vips(sbrec_lb, + ovn_northd_lb_get_vips(lb_dps->lb)); + sbrec_load_balancer_set_protocol(sbrec_lb, lb_dps->lb->nlb->protocol); + + /* Store the fact that northd provides the original (destination IP + + * transport port) tuple. + */ + struct smap options; + smap_clone(&options, &lb_dps->lb->nlb->options); + smap_replace(&options, "hairpin_orig_tuple", "true"); + sbrec_load_balancer_set_options(sbrec_lb, &options); + /* Clearing 'datapaths' column, since 'dp_group' is in use. */ + sbrec_load_balancer_set_datapaths(sbrec_lb, NULL, 0); + smap_destroy(&options); +} + +static void +sync_changed_lbs(struct sb_lb_table *sb_lbs, + struct ovsdb_idl_txn *ovnsb_txn, + const struct sbrec_load_balancer_table *sb_lb_table, + const struct sbrec_logical_dp_group_table *sb_dpgrp_table, + struct tracked_lbs *trk_lbs, + struct ovn_datapaths *ls_datapaths, + struct ovn_datapaths *lr_datapaths, + struct chassis_features *chassis_features) +{ + struct ovn_lb_datapaths *lb_dps; + struct hmapx_node *hmapx_node; + struct sb_lb_record *sb_lb; + + HMAPX_FOR_EACH (hmapx_node, &trk_lbs->deleted) { + lb_dps = hmapx_node->data; + + sb_lb = sb_lb_table_find(&sb_lbs->entries, + &lb_dps->lb->nlb->header_.uuid); + if (sb_lb) { + const struct sbrec_load_balancer *sbrec_lb = + sbrec_load_balancer_table_get_for_uuid(sb_lb_table, + &sb_lb->sb_uuid); + if (sbrec_lb) { + sbrec_load_balancer_delete(sbrec_lb); + } + + hmap_remove(&sb_lbs->entries, &sb_lb->key_node); + free(sb_lb); + } + } + + HMAPX_FOR_EACH (hmapx_node, &trk_lbs->crupdated) { + lb_dps = hmapx_node->data; + + sb_lb = sb_lb_table_find(&sb_lbs->entries, + &lb_dps->lb->nlb->header_.uuid); + + if (!sb_lb && !lb_dps->n_nb_ls && !lb_dps->n_nb_lr) { + continue; + } + + if (!sb_lb) { + sb_lb = xzalloc(sizeof *sb_lb); + sb_lb->lb_dps = lb_dps; + hmap_insert(&sb_lbs->entries, &sb_lb->key_node, + uuid_hash(&lb_dps->lb->nlb->header_.uuid)); + } else { + sb_lb->sbrec_lb = + sbrec_load_balancer_table_get_for_uuid(sb_lb_table, + &sb_lb->sb_uuid); + } + + if (sb_lb && !lb_dps->n_nb_ls && !lb_dps->n_nb_lr) { + const struct sbrec_load_balancer *sbrec_lb = + sbrec_load_balancer_table_get_for_uuid(sb_lb_table, + &sb_lb->sb_uuid); + if (sbrec_lb) { + sbrec_load_balancer_delete(sbrec_lb); + } + + hmap_remove(&sb_lbs->entries, &sb_lb->key_node); + free(sb_lb); + } + + sync_sb_lb_record(sb_lb, sb_lb->sbrec_lb, sb_dpgrp_table, sb_lbs, + ovnsb_txn, ls_datapaths, lr_datapaths, + chassis_features); + } +} + +static bool +check_sb_lb_duplicates(const struct sbrec_load_balancer_table *table) +{ + struct sset existing_nb_lb_uuids = + SSET_INITIALIZER(&existing_nb_lb_uuids); + const struct sbrec_load_balancer *sbrec_lb; + bool duplicates = false; + + SBREC_LOAD_BALANCER_TABLE_FOR_EACH (sbrec_lb, table) { + const char *nb_lb_uuid = smap_get(&sbrec_lb->external_ids, "lb_id"); + if (nb_lb_uuid && !sset_add(&existing_nb_lb_uuids, nb_lb_uuid)) { + duplicates = true; + break; + } + } + + sset_destroy(&existing_nb_lb_uuids); + return duplicates; +} diff --git a/northd/inc-proc-northd.c b/northd/inc-proc-northd.c index 28f397ff39..7688e7af93 100644 --- a/northd/inc-proc-northd.c +++ b/northd/inc-proc-northd.c @@ -277,6 +277,7 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, sync_to_sb_lb_northd_handler); engine_add_input(&en_sync_to_sb_lb, &en_sb_load_balancer, sync_to_sb_lb_sb_load_balancer); + engine_add_input(&en_sync_to_sb_lb, &en_sb_logical_dp_group, NULL); engine_add_input(&en_sync_to_sb_pb, &en_northd, sync_to_sb_pb_northd_handler); diff --git a/northd/lflow-mgr.c b/northd/lflow-mgr.c index d779e7e087..6f0e9ae9cb 100644 --- a/northd/lflow-mgr.c +++ b/northd/lflow-mgr.c @@ -72,21 +72,6 @@ static struct ovs_mutex *lflow_hash_lock(const struct hmap *lflow_table, uint32_t hash); static void lflow_hash_unlock(struct ovs_mutex *hash_lock); -static struct ovn_dp_group *ovn_dp_group_get( - struct hmap *dp_groups, size_t desired_n, - const unsigned long *desired_bitmap, - size_t bitmap_len); -static struct ovn_dp_group *ovn_dp_group_create( - struct ovsdb_idl_txn *ovnsb_txn, struct hmap *dp_groups, - struct sbrec_logical_dp_group *, size_t desired_n, - const unsigned long *desired_bitmap, - size_t bitmap_len, bool is_switch, - const struct ovn_datapaths *ls_datapaths, - const struct ovn_datapaths *lr_datapaths); -static struct ovn_dp_group *ovn_dp_group_get( - struct hmap *dp_groups, size_t desired_n, - const unsigned long *desired_bitmap, - size_t bitmap_len); static struct sbrec_logical_dp_group *ovn_sb_insert_or_update_logical_dp_group( struct ovsdb_idl_txn *ovnsb_txn, struct sbrec_logical_dp_group *, @@ -501,31 +486,81 @@ lflow_table_add_lflow_default_drop(struct lflow_table *lflow_table, where, lflow_ref); } -/* Given a desired bitmap, finds a datapath group in 'dp_groups'. If it - * doesn't exist, creates a new one and adds it to 'dp_groups'. +struct ovn_dp_group * +ovn_dp_group_get(struct hmap *dp_groups, size_t desired_n, + const unsigned long *desired_bitmap, + size_t bitmap_len) +{ + uint32_t hash; + + hash = hash_int(desired_n, 0); + return ovn_dp_group_find(dp_groups, desired_bitmap, bitmap_len, hash); +} + +/* Creates a new datapath group and adds it to 'dp_groups'. * If 'sb_group' is provided, function will try to re-use this group by - * either taking it directly, or by modifying, if it's not already in use. */ + * either taking it directly, or by modifying, if it's not already in use. + * Caller should first call ovn_dp_group_get() before calling this function. */ struct ovn_dp_group * -ovn_dp_group_get_or_create(struct ovsdb_idl_txn *ovnsb_txn, - struct hmap *dp_groups, - struct sbrec_logical_dp_group *sb_group, - size_t desired_n, - const unsigned long *desired_bitmap, - size_t bitmap_len, - bool is_switch, - const struct ovn_datapaths *ls_datapaths, - const struct ovn_datapaths *lr_datapaths) +ovn_dp_group_create(struct ovsdb_idl_txn *ovnsb_txn, + struct hmap *dp_groups, + struct sbrec_logical_dp_group *sb_group, + size_t desired_n, + const unsigned long *desired_bitmap, + size_t bitmap_len, + bool is_switch, + const struct ovn_datapaths *ls_datapaths, + const struct ovn_datapaths *lr_datapaths) { struct ovn_dp_group *dpg; - dpg = ovn_dp_group_get(dp_groups, desired_n, desired_bitmap, bitmap_len); - if (dpg) { - return dpg; + bool update_dp_group = false, can_modify = false; + unsigned long *dpg_bitmap; + size_t i, n = 0; + + dpg_bitmap = sb_group ? bitmap_allocate(bitmap_len) : NULL; + for (i = 0; sb_group && i < sb_group->n_datapaths; i++) { + struct ovn_datapath *datapath_od; + + datapath_od = ovn_datapath_from_sbrec( + ls_datapaths ? &ls_datapaths->datapaths : NULL, + lr_datapaths ? &lr_datapaths->datapaths : NULL, + sb_group->datapaths[i]); + if (!datapath_od || ovn_datapath_is_stale(datapath_od)) { + break; + } + bitmap_set1(dpg_bitmap, datapath_od->index); + n++; + } + if (!sb_group || i != sb_group->n_datapaths) { + /* No group or stale group. Not going to be used. */ + update_dp_group = true; + can_modify = true; + } else if (!bitmap_equal(dpg_bitmap, desired_bitmap, bitmap_len)) { + /* The group in Sb is different. */ + update_dp_group = true; + /* We can modify existing group if it's not already in use. */ + can_modify = !ovn_dp_group_find(dp_groups, dpg_bitmap, + bitmap_len, hash_int(n, 0)); + } + + bitmap_free(dpg_bitmap); + + dpg = xzalloc(sizeof *dpg); + dpg->bitmap = bitmap_clone(desired_bitmap, bitmap_len); + if (!update_dp_group) { + dpg->dp_group = sb_group; + } else { + dpg->dp_group = ovn_sb_insert_or_update_logical_dp_group( + ovnsb_txn, + can_modify ? sb_group : NULL, + desired_bitmap, + is_switch ? ls_datapaths : lr_datapaths); } + dpg->dpg_uuid = dpg->dp_group->header_.uuid; + hmap_insert(dp_groups, &dpg->node, hash_int(desired_n, 0)); - return ovn_dp_group_create(ovnsb_txn, dp_groups, sb_group, desired_n, - desired_bitmap, bitmap_len, is_switch, - ls_datapaths, lr_datapaths); + return dpg; } void @@ -908,24 +943,6 @@ ovn_dp_group_find(const struct hmap *dp_groups, return NULL; } -static void -inc_ovn_dp_group_ref(struct ovn_dp_group *dpg) -{ - dpg->refcnt++; -} - -static void -dec_ovn_dp_group_ref(struct hmap *dp_groups, struct ovn_dp_group *dpg) -{ - dpg->refcnt--; - - if (!dpg->refcnt) { - hmap_remove(dp_groups, &dpg->node); - free(dpg->bitmap); - free(dpg); - } -} - static struct sbrec_logical_dp_group * ovn_sb_insert_or_update_logical_dp_group( struct ovsdb_idl_txn *ovnsb_txn, @@ -952,83 +969,6 @@ ovn_sb_insert_or_update_logical_dp_group( return dp_group; } -static struct ovn_dp_group * -ovn_dp_group_get(struct hmap *dp_groups, size_t desired_n, - const unsigned long *desired_bitmap, - size_t bitmap_len) -{ - uint32_t hash; - - hash = hash_int(desired_n, 0); - return ovn_dp_group_find(dp_groups, desired_bitmap, bitmap_len, hash); -} - -/* Creates a new datapath group and adds it to 'dp_groups'. - * If 'sb_group' is provided, function will try to re-use this group by - * either taking it directly, or by modifying, if it's not already in use. - * Caller should first call ovn_dp_group_get() before calling this function. */ -static struct ovn_dp_group * -ovn_dp_group_create(struct ovsdb_idl_txn *ovnsb_txn, - struct hmap *dp_groups, - struct sbrec_logical_dp_group *sb_group, - size_t desired_n, - const unsigned long *desired_bitmap, - size_t bitmap_len, - bool is_switch, - const struct ovn_datapaths *ls_datapaths, - const struct ovn_datapaths *lr_datapaths) -{ - struct ovn_dp_group *dpg; - - bool update_dp_group = false, can_modify = false; - unsigned long *dpg_bitmap; - size_t i, n = 0; - - dpg_bitmap = sb_group ? bitmap_allocate(bitmap_len) : NULL; - for (i = 0; sb_group && i < sb_group->n_datapaths; i++) { - struct ovn_datapath *datapath_od; - - datapath_od = ovn_datapath_from_sbrec( - ls_datapaths ? &ls_datapaths->datapaths : NULL, - lr_datapaths ? &lr_datapaths->datapaths : NULL, - sb_group->datapaths[i]); - if (!datapath_od || ovn_datapath_is_stale(datapath_od)) { - break; - } - bitmap_set1(dpg_bitmap, datapath_od->index); - n++; - } - if (!sb_group || i != sb_group->n_datapaths) { - /* No group or stale group. Not going to be used. */ - update_dp_group = true; - can_modify = true; - } else if (!bitmap_equal(dpg_bitmap, desired_bitmap, bitmap_len)) { - /* The group in Sb is different. */ - update_dp_group = true; - /* We can modify existing group if it's not already in use. */ - can_modify = !ovn_dp_group_find(dp_groups, dpg_bitmap, - bitmap_len, hash_int(n, 0)); - } - - bitmap_free(dpg_bitmap); - - dpg = xzalloc(sizeof *dpg); - dpg->bitmap = bitmap_clone(desired_bitmap, bitmap_len); - if (!update_dp_group) { - dpg->dp_group = sb_group; - } else { - dpg->dp_group = ovn_sb_insert_or_update_logical_dp_group( - ovnsb_txn, - can_modify ? sb_group : NULL, - desired_bitmap, - is_switch ? ls_datapaths : lr_datapaths); - } - dpg->dpg_uuid = dpg->dp_group->header_.uuid; - hmap_insert(dp_groups, &dpg->node, hash_int(desired_n, 0)); - - return dpg; -} - /* Adds an OVN datapath to a datapath group of existing logical flow. * Version to use when hash bucket locking is NOT required or the corresponding * hash lock is already taken. */ diff --git a/northd/lflow-mgr.h b/northd/lflow-mgr.h index c65cd70e71..583afa6cb3 100644 --- a/northd/lflow-mgr.h +++ b/northd/lflow-mgr.h @@ -160,7 +160,10 @@ ovn_dp_groups_init(struct hmap *dp_groups) } void ovn_dp_groups_destroy(struct hmap *dp_groups); -struct ovn_dp_group *ovn_dp_group_get_or_create( +struct ovn_dp_group *ovn_dp_group_get(struct hmap *dp_groups, size_t desired_n, + const unsigned long *desired_bitmap, + size_t bitmap_len); +struct ovn_dp_group *ovn_dp_group_create( struct ovsdb_idl_txn *ovnsb_txn, struct hmap *dp_groups, struct sbrec_logical_dp_group *sb_group, size_t desired_n, const unsigned long *desired_bitmap, @@ -168,4 +171,22 @@ struct ovn_dp_group *ovn_dp_group_get_or_create( const struct ovn_datapaths *ls_datapaths, const struct ovn_datapaths *lr_datapaths); +static inline void +inc_ovn_dp_group_ref(struct ovn_dp_group *dpg) +{ + dpg->refcnt++; +} + +static inline void +dec_ovn_dp_group_ref(struct hmap *dp_groups, struct ovn_dp_group *dpg) +{ + dpg->refcnt--; + + if (!dpg->refcnt) { + hmap_remove(dp_groups, &dpg->node); + free(dpg->bitmap); + free(dpg); + } +} + #endif /* LFLOW_MGR_H */ \ No newline at end of file diff --git a/northd/northd.c b/northd/northd.c index e1eda9fba5..b5a7f73fdb 100644 --- a/northd/northd.c +++ b/northd/northd.c @@ -3786,244 +3786,6 @@ build_lb_port_related_data( build_lswitch_lbs_from_lrouter(lr_datapaths, lb_dps_map, lb_group_dps_map); } -struct sb_lb { - struct hmap_node hmap_node; - - const struct sbrec_load_balancer *slb; - struct ovn_dp_group *dpg; - struct ovn_dp_group *lr_dpg; - struct uuid lb_uuid; -}; - -static struct sb_lb * -find_slb_in_sb_lbs(struct hmap *sb_lbs, const struct uuid *lb_uuid) -{ - struct sb_lb *sb_lb; - HMAP_FOR_EACH_WITH_HASH (sb_lb, hmap_node, uuid_hash(lb_uuid), sb_lbs) { - if (uuid_equals(&sb_lb->lb_uuid, lb_uuid)) { - return sb_lb; - } - } - - return NULL; -} - -/* Syncs relevant load balancers (applied to logical switches) to the - * Southbound database. - */ -void -sync_lbs(struct ovsdb_idl_txn *ovnsb_txn, - const struct sbrec_load_balancer_table *sbrec_load_balancer_table, - struct ovn_datapaths *ls_datapaths, - struct ovn_datapaths *lr_datapaths, - struct hmap *lb_dps_map, - struct chassis_features *chassis_features) -{ - struct hmap ls_dp_groups = HMAP_INITIALIZER(&ls_dp_groups); - struct hmap lr_dp_groups = HMAP_INITIALIZER(&lr_dp_groups); - struct ovn_lb_datapaths *lb_dps; - struct hmap sb_lbs = HMAP_INITIALIZER(&sb_lbs); - - /* Delete any stale SB load balancer rows and create datapath - * groups for existing ones. */ - struct hmapx existing_lbs = HMAPX_INITIALIZER(&existing_lbs); - const struct sbrec_load_balancer *sbrec_lb; - SBREC_LOAD_BALANCER_TABLE_FOR_EACH_SAFE (sbrec_lb, - sbrec_load_balancer_table) { - const char *nb_lb_uuid = smap_get(&sbrec_lb->external_ids, "lb_id"); - struct uuid lb_uuid; - if (!nb_lb_uuid || !uuid_from_string(&lb_uuid, nb_lb_uuid)) { - sbrec_load_balancer_delete(sbrec_lb); - continue; - } - - /* Delete any SB load balancer entries that refer to NB load balancers - * that don't exist anymore or are not applied to switches/routers - * anymore. - * - * There is also a special case in which duplicate LBs might be created - * in the SB, e.g., due to the fact that OVSDB only ensures - * "at-least-once" consistency for clustered database tables that - * are not indexed in any way. - */ - lb_dps = ovn_lb_datapaths_find(lb_dps_map, &lb_uuid); - if (!lb_dps || (!lb_dps->n_nb_ls && !lb_dps->n_nb_lr) || - !hmapx_add(&existing_lbs, lb_dps)) { - sbrec_load_balancer_delete(sbrec_lb); - continue; - } - - struct sb_lb *sb_lb = xzalloc(sizeof *sb_lb); - sb_lb->lb_uuid = lb_uuid; - sb_lb->slb = sbrec_lb; - hmap_insert(&sb_lbs, &sb_lb->hmap_node, uuid_hash(&lb_uuid)); - - /* Find or create datapath group for this load balancer. */ - if (lb_dps->n_nb_ls) { - struct sbrec_logical_dp_group *ls_datapath_group - = chassis_features->ls_dpg_column - ? sb_lb->slb->ls_datapath_group - : sb_lb->slb->datapath_group; /* deprecated */ - sb_lb->dpg = ovn_dp_group_get_or_create( - ovnsb_txn, &ls_dp_groups, - ls_datapath_group, - lb_dps->n_nb_ls, lb_dps->nb_ls_map, - ods_size(ls_datapaths), true, - ls_datapaths, NULL); - } - if (lb_dps->n_nb_lr) { - sb_lb->lr_dpg = ovn_dp_group_get_or_create( - ovnsb_txn, &lr_dp_groups, - sb_lb->slb->lr_datapath_group, - lb_dps->n_nb_lr, lb_dps->nb_lr_map, - ods_size(lr_datapaths), false, - NULL, lr_datapaths); - } - } - hmapx_destroy(&existing_lbs); - - /* Create SB Load balancer records if not present and sync - * the SB load balancer columns. */ - HMAP_FOR_EACH (lb_dps, hmap_node, lb_dps_map) { - - if (!lb_dps->n_nb_ls && !lb_dps->n_nb_lr) { - continue; - } - - /* Store the fact that northd provides the original (destination IP + - * transport port) tuple. - */ - struct smap options; - smap_clone(&options, &lb_dps->lb->nlb->options); - smap_replace(&options, "hairpin_orig_tuple", "true"); - - struct sb_lb *sb_lb = find_slb_in_sb_lbs(&sb_lbs, - &lb_dps->lb->nlb->header_.uuid); - ovs_assert(!sb_lb || (sb_lb->slb && (sb_lb->dpg || sb_lb->lr_dpg))); - struct ovn_dp_group *lb_dpg = NULL, *lb_lr_dpg = NULL; - if (!sb_lb) { - sbrec_lb = sbrec_load_balancer_insert(ovnsb_txn); - char *lb_id = xasprintf( - UUID_FMT, UUID_ARGS(&lb_dps->lb->nlb->header_.uuid)); - const struct smap external_ids = - SMAP_CONST1(&external_ids, "lb_id", lb_id); - sbrec_load_balancer_set_external_ids(sbrec_lb, &external_ids); - free(lb_id); - } else { - sbrec_lb = sb_lb->slb; - lb_dpg = sb_lb->dpg; - lb_lr_dpg = sb_lb->lr_dpg; - } - - /* Find or create datapath group for this load balancer. */ - if (!lb_dpg && lb_dps->n_nb_ls) { - struct sbrec_logical_dp_group *ls_datapath_group - = chassis_features->ls_dpg_column - ? sbrec_lb->ls_datapath_group - : sbrec_lb->datapath_group; /* deprecated */ - lb_dpg = ovn_dp_group_get_or_create( - ovnsb_txn, &ls_dp_groups, - ls_datapath_group, - lb_dps->n_nb_ls, lb_dps->nb_ls_map, - ods_size(ls_datapaths), true, - ls_datapaths, NULL); - } - if (!lb_lr_dpg && lb_dps->n_nb_lr) { - lb_lr_dpg = ovn_dp_group_get_or_create( - ovnsb_txn, &lr_dp_groups, - sbrec_lb->lr_datapath_group, - lb_dps->n_nb_lr, lb_dps->nb_lr_map, - ods_size(lr_datapaths), false, - NULL, lr_datapaths); - } - - /* Update columns. */ - sbrec_load_balancer_set_name(sbrec_lb, lb_dps->lb->nlb->name); - sbrec_load_balancer_set_vips(sbrec_lb, - ovn_northd_lb_get_vips(lb_dps->lb)); - sbrec_load_balancer_set_protocol(sbrec_lb, lb_dps->lb->nlb->protocol); - - if (chassis_features->ls_dpg_column) { - sbrec_load_balancer_set_ls_datapath_group( - sbrec_lb, lb_dpg ? lb_dpg->dp_group : NULL - ); - sbrec_load_balancer_set_datapath_group(sbrec_lb, NULL); - } else { - /* datapath_group column is deprecated. */ - sbrec_load_balancer_set_ls_datapath_group(sbrec_lb, NULL); - sbrec_load_balancer_set_datapath_group( - sbrec_lb, lb_dpg ? lb_dpg->dp_group : NULL - ); - } - - sbrec_load_balancer_set_lr_datapath_group( - sbrec_lb, lb_lr_dpg ? lb_lr_dpg->dp_group : NULL - ); - sbrec_load_balancer_set_options(sbrec_lb, &options); - /* Clearing 'datapaths' column, since 'dp_group' is in use. */ - sbrec_load_balancer_set_datapaths(sbrec_lb, NULL, 0); - smap_destroy(&options); - } - - struct ovn_dp_group *dpg; - HMAP_FOR_EACH_POP (dpg, node, &ls_dp_groups) { - bitmap_free(dpg->bitmap); - free(dpg); - } - hmap_destroy(&ls_dp_groups); - - HMAP_FOR_EACH_POP (dpg, node, &lr_dp_groups) { - bitmap_free(dpg->bitmap); - free(dpg); - } - hmap_destroy(&lr_dp_groups); - - struct sb_lb *sb_lb; - HMAP_FOR_EACH_POP (sb_lb, hmap_node, &sb_lbs) { - free(sb_lb); - } - hmap_destroy(&sb_lbs); - - /* Datapath_Binding.load_balancers is not used anymore, it's still in the - * schema for compatibility reasons. Reset it to empty, just in case. - */ - struct ovn_datapath *od; - HMAP_FOR_EACH (od, key_node, &ls_datapaths->datapaths) { - ovs_assert(od->nbs); - - if (od->sb->n_load_balancers) { - sbrec_datapath_binding_set_load_balancers(od->sb, NULL, 0); - } - } - HMAP_FOR_EACH (od, key_node, &lr_datapaths->datapaths) { - ovs_assert(od->nbr); - - if (od->sb->n_load_balancers) { - sbrec_datapath_binding_set_load_balancers(od->sb, NULL, 0); - } - } -} - -bool -check_sb_lb_duplicates(const struct sbrec_load_balancer_table *table) -{ - struct sset existing_nb_lb_uuids = - SSET_INITIALIZER(&existing_nb_lb_uuids); - const struct sbrec_load_balancer *sbrec_lb; - bool duplicates = false; - - SBREC_LOAD_BALANCER_TABLE_FOR_EACH (sbrec_lb, table) { - const char *nb_lb_uuid = smap_get(&sbrec_lb->external_ids, "lb_id"); - if (nb_lb_uuid && !sset_add(&existing_nb_lb_uuids, nb_lb_uuid)) { - duplicates = true; - break; - } - } - - sset_destroy(&existing_nb_lb_uuids); - return duplicates; -} - /* Syncs the SB port binding for the ovn_port 'op' of a logical switch port. * Caller should make sure that the OVN SB IDL txn is not NULL. Presently it * only syncs the nat column of port binding corresponding to the 'op->nbsp' */ diff --git a/northd/northd.h b/northd/northd.h index 7e48bb966d..8dabc17eb7 100644 --- a/northd/northd.h +++ b/northd/northd.h @@ -762,12 +762,6 @@ void run_update_worker_pool(int n_threads); const struct ovn_datapath *northd_get_datapath_for_port( const struct hmap *ls_ports, const char *port_name); -void sync_lbs(struct ovsdb_idl_txn *, const struct sbrec_load_balancer_table *, - struct ovn_datapaths *ls_datapaths, - struct ovn_datapaths *lr_datapaths, - struct hmap *lbs, - struct chassis_features *chassis_features); -bool check_sb_lb_duplicates(const struct sbrec_load_balancer_table *); struct lr_stateful_table; void sync_pbs(struct ovsdb_idl_txn *, struct hmap *ls_ports, diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at index 03d525bdc2..b0d3fde1da 100644 --- a/tests/ovn-northd.at +++ b/tests/ovn-northd.at @@ -2938,6 +2938,8 @@ sw1_sb_uuid=$(fetch_column datapath_binding _uuid external_ids:name=sw1) echo "$sw0_sb_uuid" > sw_sb_uuids echo "$sw1_sb_uuid" >> sw_sb_uuids +lb0_dp_group=$(fetch_column sb:load_balancer ls_datapath_group name=lb0) + echo echo "__file__:__line__: Check that SB lb0 has sw0 and sw1 in datapaths column." AT_CHECK_UNQUOTED([ovn-sbctl --bare --columns _uuid,datapaths find Logical_DP_Group dnl @@ -2945,6 +2947,8 @@ AT_CHECK_UNQUOTED([ovn-sbctl --bare --columns _uuid,datapaths find Logical_DP_Gr $(cat sw_sb_uuids | sort) ]) +lbg0_dp_group=$(fetch_column sb:load_balancer ls_datapath_group name=lbg0) + echo echo "__file__:__line__: Check that SB lbg0 has sw0 and sw1 in datapaths column." AT_CHECK_UNQUOTED([ovn-sbctl --bare --columns _uuid,datapaths find Logical_DP_Group dnl @@ -10502,9 +10506,9 @@ check ovn-nbctl --wait=sb lb-add lb1 10.0.0.10:80 10.0.0.3:80 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute nocompute - +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE(1) + check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb set load_balancer . ip_port_mappings:10.0.0.3=sw0-p1:10.0.0.2 @@ -10512,7 +10516,7 @@ check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute nocompute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats @@ -10522,7 +10526,7 @@ check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute nocompute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats @@ -10531,7 +10535,7 @@ check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute nocompute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE(1) check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats @@ -10541,7 +10545,7 @@ check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute nocompute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE(1) check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats @@ -10599,7 +10603,7 @@ check_engine_stats lr_stateful norecompute compute # A LB applied to a switch/router triggers: # - a recompute in the first iteration (handling northd change) # - a compute in the second iteration (handling SB update) -check_engine_stats sync_to_sb_lb recompute compute +check_engine_stats sync_to_sb_lb norecompute compute check_engine_stats ls_stateful norecompute compute check_engine_stats lflow norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE((1)) @@ -10612,7 +10616,7 @@ check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute check_engine_stats ls_stateful norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute compute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE(1) # Cleanup the vip of lb1. @@ -10623,7 +10627,7 @@ check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute check_engine_stats ls_stateful norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute compute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE(1) # Set the vips of lb1 back @@ -10634,7 +10638,7 @@ check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute check_engine_stats ls_stateful norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute compute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE(1) # Add another vip to lb1 @@ -10645,7 +10649,7 @@ check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute check_engine_stats ls_stateful norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute compute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE(1) # Disassociate lb1 from sw0. There should be a full recompute of northd engine node. @@ -10668,7 +10672,7 @@ check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute check_engine_stats ls_stateful norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute compute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats @@ -10689,7 +10693,7 @@ check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute compute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Modify the backend of the lb1 vip @@ -10699,7 +10703,7 @@ check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute compute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Cleanup the vip of lb1. @@ -10709,7 +10713,7 @@ check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute compute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Set the vips of lb1 back @@ -10719,7 +10723,7 @@ check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute compute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Add another vip to lb1 @@ -10729,7 +10733,7 @@ check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute compute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats @@ -10761,7 +10765,7 @@ check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute nocompute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats @@ -10778,7 +10782,7 @@ check ovn-nbctl --wait=sb add load_balancer_group . load_Balancer $lb1_uuid check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute nocompute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats @@ -10788,7 +10792,7 @@ check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute check_engine_stats ls_stateful norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute compute +check_engine_stats sync_to_sb_lb norecompute compute # Update lb and this should not result in northd recompute check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats @@ -10798,7 +10802,7 @@ check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute check_engine_stats ls_stateful norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute compute +check_engine_stats sync_to_sb_lb norecompute compute # Modify the backend of the lb1 vip check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats @@ -10808,7 +10812,7 @@ check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute check_engine_stats ls_stateful norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute compute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Cleanup the vip of lb1. @@ -10819,7 +10823,7 @@ check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute check_engine_stats ls_stateful norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute compute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Set the vips of lb1 back @@ -10830,7 +10834,7 @@ check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute check_engine_stats ls_stateful norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute compute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Add another vip to lb1 @@ -10841,7 +10845,7 @@ check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute check_engine_stats ls_stateful norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute compute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats @@ -10857,7 +10861,7 @@ check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute compute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Modify the backend of the lb1 vip @@ -10867,7 +10871,7 @@ check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute compute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Cleanup the vip of lb1. @@ -10877,7 +10881,7 @@ check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute compute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Set the vips of lb1 back @@ -10887,7 +10891,7 @@ check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute compute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Add another vip to lb1 @@ -10897,7 +10901,7 @@ check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute compute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats @@ -10917,7 +10921,7 @@ check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute check_engine_stats ls_stateful norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute compute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats @@ -10961,7 +10965,7 @@ check_engine_stats northd norecompute compute check_engine_stats ls_stateful norecompute compute check_engine_stats lr_stateful norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute nocompute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats @@ -10971,7 +10975,7 @@ check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute check_engine_stats ls_stateful norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute compute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats @@ -10980,7 +10984,7 @@ check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute compute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats @@ -10990,7 +10994,7 @@ check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute check_engine_stats ls_stateful norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute nocompute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats @@ -11000,7 +11004,7 @@ check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute check_engine_stats ls_stateful norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute nocompute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats @@ -11010,7 +11014,7 @@ check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute compute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats @@ -11041,7 +11045,7 @@ check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute check_engine_stats ls_stateful norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute compute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Deleting lb2 should result in lflow recompute as it is @@ -11053,7 +11057,7 @@ check_engine_stats northd norecompute compute check_engine_stats lr_stateful norecompute compute check_engine_stats ls_stateful norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute compute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats @@ -11087,6 +11091,7 @@ check ovn-nbctl --wait=sb ls-lb-add sw0 lb1 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lflow norecompute compute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Clear the VIPs of lb1 @@ -11095,6 +11100,7 @@ check ovn-nbctl --wait=sb clear load_balancer . vips check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lflow norecompute compute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats @@ -11102,6 +11108,7 @@ check ovn-nbctl --wait=sb lb-del lb1 check_engine_stats lb_data norecompute compute check_engine_stats northd recompute nocompute check_engine_stats lflow recompute nocompute +check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE AT_CLEANUP @@ -11129,6 +11136,7 @@ check_engine_stats northd recompute nocompute check_engine_stats lr_nat recompute nocompute check_engine_stats lr_stateful recompute nocompute check_engine_stats sync_to_sb_pb recompute nocompute +check_engine_stats sync_to_sb_lb recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11142,6 +11150,7 @@ check_engine_stats northd recompute compute check_engine_stats lr_nat recompute nocompute check_engine_stats lr_stateful recompute nocompute check_engine_stats sync_to_sb_pb recompute nocompute +check_engine_stats sync_to_sb_lb recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11154,6 +11163,7 @@ check_engine_stats northd recompute compute check_engine_stats lr_nat recompute nocompute check_engine_stats lr_stateful recompute nocompute check_engine_stats sync_to_sb_pb recompute nocompute +check_engine_stats sync_to_sb_lb recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11180,6 +11190,7 @@ check_engine_stats northd recompute nocompute check_engine_stats lr_nat recompute nocompute check_engine_stats lr_stateful recompute nocompute check_engine_stats sync_to_sb_pb recompute nocompute +check_engine_stats sync_to_sb_lb recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11192,6 +11203,7 @@ check_engine_stats northd norecompute compute check_engine_stats lr_nat norecompute compute check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_pb recompute nocompute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Update the NAT options column @@ -11201,6 +11213,7 @@ check_engine_stats northd norecompute compute check_engine_stats lr_nat norecompute compute check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_pb recompute nocompute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Update the NAT external_ip column @@ -11211,6 +11224,7 @@ check_engine_stats lr_nat norecompute compute check_engine_stats lr_stateful norecompute compute check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_pb recompute nocompute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Update the NAT logical_ip column @@ -11221,6 +11235,7 @@ check_engine_stats lr_nat norecompute compute check_engine_stats lr_stateful norecompute compute check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_pb recompute nocompute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Update the NAT type @@ -11231,6 +11246,7 @@ check_engine_stats lr_nat norecompute compute check_engine_stats lr_stateful norecompute compute check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_pb recompute nocompute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Create a dnat_and_snat NAT with external_mac and logical_port @@ -11241,6 +11257,7 @@ check_engine_stats lr_nat norecompute compute check_engine_stats lr_stateful norecompute compute check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_pb recompute nocompute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE nat2_uuid=$(ovn-nbctl --bare --columns _uuid find nat logical_ip=10.0.0.4) @@ -11252,6 +11269,7 @@ check_engine_stats lr_nat norecompute compute check_engine_stats lr_stateful norecompute compute check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_pb recompute nocompute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Create a load balancer and add the lb vip as NAT @@ -11268,6 +11286,7 @@ check_engine_stats northd norecompute compute check_engine_stats lr_nat norecompute compute check_engine_stats lr_stateful norecompute compute check_engine_stats sync_to_sb_pb recompute nocompute +check_engine_stats sync_to_sb_lb norecompute compute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11279,6 +11298,7 @@ check_engine_stats northd norecompute compute check_engine_stats lr_nat norecompute compute check_engine_stats lr_stateful norecompute compute check_engine_stats sync_to_sb_pb recompute nocompute +check_engine_stats sync_to_sb_lb norecompute compute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11290,6 +11310,7 @@ check_engine_stats northd norecompute compute check_engine_stats lr_nat norecompute compute check_engine_stats lr_stateful norecompute compute check_engine_stats sync_to_sb_pb recompute nocompute +check_engine_stats sync_to_sb_lb norecompute compute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11301,6 +11322,7 @@ check_engine_stats northd norecompute compute check_engine_stats lr_nat norecompute compute check_engine_stats lr_stateful norecompute compute check_engine_stats sync_to_sb_pb recompute nocompute +check_engine_stats sync_to_sb_lb norecompute compute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11312,6 +11334,7 @@ check_engine_stats lr_nat norecompute compute check_engine_stats lr_stateful norecompute compute check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_pb recompute nocompute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Create router Policy @@ -11321,6 +11344,7 @@ check_engine_stats northd recompute nocompute check_engine_stats lr_nat recompute nocompute check_engine_stats lr_stateful recompute nocompute check_engine_stats sync_to_sb_pb recompute nocompute +check_engine_stats sync_to_sb_lb recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11330,6 +11354,7 @@ check_engine_stats northd recompute nocompute check_engine_stats lr_nat recompute nocompute check_engine_stats lr_stateful recompute nocompute check_engine_stats sync_to_sb_pb recompute nocompute +check_engine_stats sync_to_sb_lb recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE