From patchwork Fri Nov 4 22:11:12 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dumitru Ceara X-Patchwork-Id: 1699893 X-Patchwork-Delegate: mmichels@redhat.com Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::138; helo=smtp1.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=Mkx7KdVS; dkim-atps=neutral Received: from smtp1.osuosl.org (smtp1.osuosl.org [IPv6:2605:bc80:3010::138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4N3vwh1pX7z23lT for ; Sat, 5 Nov 2022 09:11:32 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 6D96C8221C; Fri, 4 Nov 2022 22:11:30 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 6D96C8221C Authentication-Results: smtp1.osuosl.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=Mkx7KdVS X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id kgBSSde6Z7b5; Fri, 4 Nov 2022 22:11:28 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp1.osuosl.org (Postfix) with ESMTPS id 218AB8222C; Fri, 4 Nov 2022 22:11:27 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 218AB8222C Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id DE30EC0032; Fri, 4 Nov 2022 22:11:26 +0000 (UTC) X-Original-To: ovs-dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp1.osuosl.org (smtp1.osuosl.org [IPv6:2605:bc80:3010::138]) by lists.linuxfoundation.org (Postfix) with ESMTP id 81563C002D for ; Fri, 4 Nov 2022 22:11:25 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 0710A82213 for ; Fri, 4 Nov 2022 22:11:23 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 0710A82213 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 2fhINK3_7XmL for ; Fri, 4 Nov 2022 22:11:20 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 5CC2D8221C Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by smtp1.osuosl.org (Postfix) with ESMTPS id 5CC2D8221C for ; Fri, 4 Nov 2022 22:11:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1667599879; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=TnSVgyat/WcOCWPg+88PBl/SR29Vxt3m6r7NnqD2Dgc=; b=Mkx7KdVS0ZBuC8GUYXrplpnF60cRjl/JzPB1UK7w+VjW9ZmfgZnpPFp5oLra+uEjXBhGX3 R99StSeiB1NE6MlXpM/2uI3GRvujun34ndkEAckAzw1QybOZCWFUtoI/6XO++BKYOADylU V6fyJAAkyEHNRpEtwN/zthUnwDmzrFY= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-308-UHtt9Gx_OR6zqVec2pv6hw-1; Fri, 04 Nov 2022 18:11:16 -0400 X-MC-Unique: UHtt9Gx_OR6zqVec2pv6hw-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id E30AE29AB3F7; Fri, 4 Nov 2022 22:11:15 +0000 (UTC) Received: from dceara.remote.csb (unknown [10.39.192.36]) by smtp.corp.redhat.com (Postfix) with ESMTP id 4D3CDC2C8C5; Fri, 4 Nov 2022 22:11:14 +0000 (UTC) From: Dumitru Ceara To: ovs-dev@openvswitch.org Date: Fri, 4 Nov 2022 23:11:12 +0100 Message-Id: <166759987050.515460.12406491945779417131.stgit@dceara.remote.csb> In-Reply-To: <166759985955.515460.16275468131013093936.stgit@dceara.remote.csb> References: <166759985955.515460.16275468131013093936.stgit@dceara.remote.csb> User-Agent: StGit/0.23 MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.8 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Cc: surya@redhat.com, i.maximets@ovn.org Subject: [ovs-dev] [PATCH ovn v2 1/5] lflow: Factor out the lflow reference handling code into a new module. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" This makes it easier to have an overview of what the code does and at the same time it allows multiple users to define and manage "resource <-> object" dependencies. Acked-by: Han Zhou Signed-off-by: Dumitru Ceara Acked-by: Mark Michelson --- V2: - Addressed Mark's comments: - Fixed typos in comments in objdep.h. - Made objdep_change_handler return bool (handled successfully or not). - Reverted some unrelated style changes. - Fixed cast style. - Addressed Han's comments: - Removed superfluous 'type' argument name in prototypes. - Added Han's ack. --- controller/lflow.c | 330 ++++++++++--------------------------------- controller/lflow.h | 67 +-------- controller/ovn-controller.c | 55 +++++-- lib/automake.mk | 2 lib/objdep.c | 260 ++++++++++++++++++++++++++++++++++ lib/objdep.h | 122 ++++++++++++++++ 6 files changed, 508 insertions(+), 328 deletions(-) create mode 100644 lib/objdep.c create mode 100644 lib/objdep.h diff --git a/controller/lflow.c b/controller/lflow.c index cc0f31db06..d4434bdee8 100644 --- a/controller/lflow.c +++ b/controller/lflow.c @@ -61,7 +61,7 @@ struct lookup_port_aux { struct ovsdb_idl_index *sbrec_port_binding_by_name; const struct sbrec_datapath_binding *dp; const struct sbrec_logical_flow *lflow; - struct lflow_resource_ref *lfrr; + struct objdep_mgr *deps_mgr; const struct hmap *chassis_tunnels; }; @@ -72,8 +72,8 @@ struct condition_aux { const struct sset *active_tunnels; const struct sbrec_logical_flow *lflow; /* Resource reference to store the port name referenced - * in is_chassis_resident() to the logical flow. */ - struct lflow_resource_ref *lfrr; + * in is_chassis_resident() to the object (logical flow). */ + struct objdep_mgr *deps_mgr; }; static struct expr * @@ -81,7 +81,7 @@ convert_match_to_expr(const struct sbrec_logical_flow *, const struct local_datapath *ldp, struct expr **prereqs, const struct shash *addr_sets, const struct shash *port_groups, - struct lflow_resource_ref *, bool *pg_addr_set_ref); + struct objdep_mgr *, bool *pg_addr_set_ref); static void add_matches_to_flow_table(const struct sbrec_logical_flow *, const struct local_datapath *, @@ -94,17 +94,6 @@ consider_logical_flow(const struct sbrec_logical_flow *lflow, bool is_recompute, struct lflow_ctx_in *l_ctx_in, struct lflow_ctx_out *l_ctx_out); -static void lflow_resource_add(struct lflow_resource_ref *, enum ref_type, - const char *ref_name, const struct uuid *, - size_t ref_count); -static struct ref_lflow_node *ref_lflow_lookup(struct hmap *ref_lflow_table, - enum ref_type, - const char *ref_name); -static struct lflow_ref_node *lflow_ref_lookup(struct hmap *lflow_ref_table, - const struct uuid *lflow_uuid); -static void ref_lflow_node_destroy(struct ref_lflow_node *); -static void lflow_resource_destroy_lflow(struct lflow_resource_ref *, - const struct uuid *lflow_uuid); static void add_port_sec_flows(const struct shash *binding_lports, const struct sbrec_chassis *, @@ -125,8 +114,8 @@ lookup_port_cb(const void *aux_, const char *port_name, unsigned int *portp) /* Store the name that used to lookup the lport to lflow reference, so that * in the future when the lport's port binding changes, the logical flow * that references this lport can be reprocessed. */ - lflow_resource_add(aux->lfrr, REF_TYPE_PORTBINDING, port_name, - &aux->lflow->header_.uuid, 0); + objdep_mgr_add(aux->deps_mgr, OBJDEP_TYPE_PORTBINDING, port_name, + &aux->lflow->header_.uuid); const struct sbrec_port_binding *pb = lport_lookup_by_name(aux->sbrec_port_binding_by_name, port_name); @@ -141,8 +130,8 @@ lookup_port_cb(const void *aux_, const char *port_name, unsigned int *portp) * this multicast group can be reprocessed. */ struct ds mg_key = DS_EMPTY_INITIALIZER; get_mc_group_key(port_name, aux->dp->tunnel_key, &mg_key); - lflow_resource_add(aux->lfrr, REF_TYPE_MC_GROUP, ds_cstr(&mg_key), - &aux->lflow->header_.uuid, 0); + objdep_mgr_add(aux->deps_mgr, OBJDEP_TYPE_MC_GROUP, ds_cstr(&mg_key), + &aux->lflow->header_.uuid); ds_destroy(&mg_key); const struct sbrec_multicast_group *mg = mcgroup_lookup_by_dp_name( @@ -180,11 +169,11 @@ is_chassis_resident_cb(const void *c_aux_, const char *port_name) { const struct condition_aux *c_aux = c_aux_; - /* Store the port name that used to lookup the lport to lflow reference, so - * that in the future when the lport's port-binding changes the logical + /* Store the port name that used to lookup the lport to object reference, + * so that in the future when the lport's port-binding changes the logical * flow that references this lport can be reprocessed. */ - lflow_resource_add(c_aux->lfrr, REF_TYPE_PORTBINDING, port_name, - &c_aux->lflow->header_.uuid, 0); + objdep_mgr_add(c_aux->deps_mgr, OBJDEP_TYPE_PORTBINDING, port_name, + &c_aux->lflow->header_.uuid); const struct sbrec_port_binding *pb = lport_lookup_by_name(c_aux->sbrec_port_binding_by_name, port_name); @@ -207,155 +196,6 @@ is_chassis_resident_cb(const void *c_aux_, const char *port_name) } } -void -lflow_resource_init(struct lflow_resource_ref *lfrr) -{ - hmap_init(&lfrr->ref_lflow_table); - hmap_init(&lfrr->lflow_ref_table); -} - -void -lflow_resource_destroy(struct lflow_resource_ref *lfrr) -{ - struct ref_lflow_node *rlfn; - HMAP_FOR_EACH_SAFE (rlfn, node, &lfrr->ref_lflow_table) { - struct lflow_ref_list_node *lrln; - HMAP_FOR_EACH_SAFE (lrln, hmap_node, &rlfn->lflow_uuids) { - ovs_list_remove(&lrln->list_node); - hmap_remove(&rlfn->lflow_uuids, &lrln->hmap_node); - free(lrln); - } - hmap_remove(&lfrr->ref_lflow_table, &rlfn->node); - ref_lflow_node_destroy(rlfn); - } - hmap_destroy(&lfrr->ref_lflow_table); - - struct lflow_ref_node *lfrn; - HMAP_FOR_EACH_SAFE (lfrn, node, &lfrr->lflow_ref_table) { - hmap_remove(&lfrr->lflow_ref_table, &lfrn->node); - free(lfrn); - } - hmap_destroy(&lfrr->lflow_ref_table); -} - -void -lflow_resource_clear(struct lflow_resource_ref *lfrr) -{ - lflow_resource_destroy(lfrr); - lflow_resource_init(lfrr); -} - -static struct ref_lflow_node* -ref_lflow_lookup(struct hmap *ref_lflow_table, - enum ref_type type, const char *ref_name) -{ - struct ref_lflow_node *rlfn; - - HMAP_FOR_EACH_WITH_HASH (rlfn, node, hash_string(ref_name, type), - ref_lflow_table) { - if (rlfn->type == type && !strcmp(rlfn->ref_name, ref_name)) { - return rlfn; - } - } - return NULL; -} - -static struct lflow_ref_node* -lflow_ref_lookup(struct hmap *lflow_ref_table, - const struct uuid *lflow_uuid) -{ - struct lflow_ref_node *lfrn; - - HMAP_FOR_EACH_WITH_HASH (lfrn, node, uuid_hash(lflow_uuid), - lflow_ref_table) { - if (uuid_equals(&lfrn->lflow_uuid, lflow_uuid)) { - return lfrn; - } - } - return NULL; -} - -static void -lflow_resource_add(struct lflow_resource_ref *lfrr, enum ref_type type, - const char *ref_name, const struct uuid *lflow_uuid, - size_t ref_count) -{ - struct ref_lflow_node *rlfn = ref_lflow_lookup(&lfrr->ref_lflow_table, - type, ref_name); - struct lflow_ref_node *lfrn = lflow_ref_lookup(&lfrr->lflow_ref_table, - lflow_uuid); - if (rlfn && lfrn) { - /* Check if the mapping already existed before adding a new one. */ - struct lflow_ref_list_node *n; - HMAP_FOR_EACH_WITH_HASH (n, hmap_node, uuid_hash(lflow_uuid), - &rlfn->lflow_uuids) { - if (uuid_equals(&n->lflow_uuid, lflow_uuid)) { - return; - } - } - } - - if (!rlfn) { - rlfn = xzalloc(sizeof *rlfn); - rlfn->node.hash = hash_string(ref_name, type); - rlfn->type = type; - rlfn->ref_name = xstrdup(ref_name); - hmap_init(&rlfn->lflow_uuids); - hmap_insert(&lfrr->ref_lflow_table, &rlfn->node, rlfn->node.hash); - } - - if (!lfrn) { - lfrn = xzalloc(sizeof *lfrn); - lfrn->node.hash = uuid_hash(lflow_uuid); - lfrn->lflow_uuid = *lflow_uuid; - ovs_list_init(&lfrn->lflow_ref_head); - hmap_insert(&lfrr->lflow_ref_table, &lfrn->node, lfrn->node.hash); - } - - struct lflow_ref_list_node *lrln = xzalloc(sizeof *lrln); - lrln->lflow_uuid = *lflow_uuid; - lrln->ref_count = ref_count; - lrln->rlfn = rlfn; - hmap_insert(&rlfn->lflow_uuids, &lrln->hmap_node, uuid_hash(lflow_uuid)); - ovs_list_push_back(&lfrn->lflow_ref_head, &lrln->list_node); -} - -static void -ref_lflow_node_destroy(struct ref_lflow_node *rlfn) -{ - free(rlfn->ref_name); - hmap_destroy(&rlfn->lflow_uuids); - free(rlfn); -} - -static void -lflow_resource_destroy_lflow(struct lflow_resource_ref *lfrr, - const struct uuid *lflow_uuid) -{ - struct lflow_ref_node *lfrn = lflow_ref_lookup(&lfrr->lflow_ref_table, - lflow_uuid); - if (!lfrn) { - return; - } - - hmap_remove(&lfrr->lflow_ref_table, &lfrn->node); - struct lflow_ref_list_node *lrln; - LIST_FOR_EACH_SAFE (lrln, list_node, &lfrn->lflow_ref_head) { - ovs_list_remove(&lrln->list_node); - hmap_remove(&lrln->rlfn->lflow_uuids, &lrln->hmap_node); - - /* Clean up the node in ref_lflow_table if the resource is not - * referred by any logical flows. */ - if (hmap_is_empty(&lrln->rlfn->lflow_uuids)) { - hmap_remove(&lfrr->ref_lflow_table, &lrln->rlfn->node); - ref_lflow_node_destroy(lrln->rlfn); - } - - free(lrln); - } - free(lfrn); -} - /* Adds the logical flows from the Logical_Flow table to flow tables. */ static void add_logical_flows(struct lflow_ctx_in *l_ctx_in, @@ -400,7 +240,7 @@ lflow_handle_changed_flows(struct lflow_ctx_in *l_ctx_in, struct uuidset_node *ofrn; UUIDSET_FOR_EACH (ofrn, &flood_remove_nodes) { /* Delete entries from lflow resource reference. */ - lflow_resource_destroy_lflow(l_ctx_out->lfrr, &ofrn->uuid); + objdep_mgr_remove_obj(l_ctx_out->lflow_deps_mgr, &ofrn->uuid); /* Delete conj_ids owned by the lflow. */ lflow_conj_ids_free(l_ctx_out->conj_ids, &ofrn->uuid); /* Reprocessing the lflow if the sb record is not deleted. */ @@ -537,7 +377,7 @@ consider_lflow_for_added_as_ips__( .sbrec_port_binding_by_name = l_ctx_in->sbrec_port_binding_by_name, .dp = dp, .lflow = lflow, - .lfrr = l_ctx_out->lfrr, + .deps_mgr = l_ctx_out->lflow_deps_mgr, }; struct condition_aux cond_aux = { .sbrec_port_binding_by_name = l_ctx_in->sbrec_port_binding_by_name, @@ -545,7 +385,7 @@ consider_lflow_for_added_as_ips__( .chassis = l_ctx_in->chassis, .active_tunnels = l_ctx_in->active_tunnels, .lflow = lflow, - .lfrr = l_ctx_out->lfrr, + .deps_mgr = l_ctx_out->lflow_deps_mgr, }; struct hmap matches = HMAP_INITIALIZER(&matches); @@ -591,7 +431,7 @@ consider_lflow_for_added_as_ips__( struct expr *expr = convert_match_to_expr(lflow, ldp, &prereqs, l_ctx_in->addr_sets, l_ctx_in->port_groups, - l_ctx_out->lfrr, NULL); + l_ctx_out->lflow_deps_mgr, NULL); shash_replace((struct shash *)l_ctx_in->addr_sets, as_name, real_as); if (new_fake_as) { expr_constant_set_destroy(new_fake_as); @@ -787,10 +627,10 @@ lflow_handle_addr_set_update(const char *as_name, return false; } - struct ref_lflow_node *rlfn = - ref_lflow_lookup(&l_ctx_out->lfrr->ref_lflow_table, REF_TYPE_ADDRSET, - as_name); - if (!rlfn) { + struct resource_to_objects_node *resource_node = + objdep_mgr_find_objs(l_ctx_out->lflow_deps_mgr, OBJDEP_TYPE_ADDRSET, + as_name); + if (!resource_node) { *changed = false; return true; } @@ -798,22 +638,23 @@ lflow_handle_addr_set_update(const char *as_name, *changed = false; bool ret = true; - struct lflow_ref_list_node *lrln; - HMAP_FOR_EACH (lrln, hmap_node, &rlfn->lflow_uuids) { - if (uuidset_find(l_ctx_out->lflows_processed, &lrln->lflow_uuid)) { + struct object_to_resources_list_node *resource_list_node; + RESOURCE_FOR_EACH_OBJ (resource_list_node, resource_node) { + const struct uuid *obj_uuid = &resource_list_node->obj_uuid; + if (uuidset_find(l_ctx_out->lflows_processed, obj_uuid)) { VLOG_DBG("lflow "UUID_FMT"has been processed, skip.", - UUID_ARGS(&lrln->lflow_uuid)); + UUID_ARGS(obj_uuid)); continue; } const struct sbrec_logical_flow *lflow = sbrec_logical_flow_table_get_for_uuid(l_ctx_in->logical_flow_table, - &lrln->lflow_uuid); + obj_uuid); if (!lflow) { /* lflow deletion should be handled in the corresponding input * handler, so we can skip here. */ VLOG_DBG("lflow "UUID_FMT" not found while handling updates of " "address set %s, skip.", - UUID_ARGS(&lrln->lflow_uuid), as_name); + UUID_ARGS(obj_uuid), as_name); continue; } *changed = true; @@ -825,9 +666,9 @@ lflow_handle_addr_set_update(const char *as_name, if (!as_info_from_expr_const(as_name, c, &as_info)) { continue; } - if (!ofctrl_remove_flows_for_as_ip(l_ctx_out->flow_table, - &lrln->lflow_uuid, &as_info, - lrln->ref_count)) { + if (!ofctrl_remove_flows_for_as_ip( + l_ctx_out->flow_table, obj_uuid, &as_info, + resource_list_node->ref_count)) { ret = false; goto done; } @@ -836,7 +677,7 @@ lflow_handle_addr_set_update(const char *as_name, if (as_diff->added) { if (!consider_lflow_for_added_as_ips(lflow, as_name, - lrln->ref_count, + resource_list_node->ref_count, as_diff->added, l_ctx_in, l_ctx_out)) { ret = false; @@ -850,59 +691,32 @@ done: } bool -lflow_handle_changed_ref(enum ref_type ref_type, const char *ref_name, - struct lflow_ctx_in *l_ctx_in, - struct lflow_ctx_out *l_ctx_out, - bool *changed) +lflow_handle_changed_ref(enum objdep_type type, const char *res_name, + struct ovs_list *objs_todo, + const void *in_arg, void *out_arg) { - struct ref_lflow_node *rlfn = - ref_lflow_lookup(&l_ctx_out->lfrr->ref_lflow_table, ref_type, - ref_name); - if (!rlfn) { - *changed = false; - return true; - } - VLOG_DBG("Handle changed lflow reference for resource type: %d," - " name: %s.", ref_type, ref_name); - *changed = false; - bool ret = true; - - struct ovs_list lflows_todo = OVS_LIST_INITIALIZER(&lflows_todo); - - struct lflow_ref_list_node *lrln, *lrln_uuid; - HMAP_FOR_EACH (lrln, hmap_node, &rlfn->lflow_uuids) { - if (uuidset_find(l_ctx_out->lflows_processed, &lrln->lflow_uuid)) { - continue; - } - /* Use lflow_ref_list_node as list node to store the uuid. - * Other fields are not used here. */ - lrln_uuid = xmalloc(sizeof *lrln_uuid); - lrln_uuid->lflow_uuid = lrln->lflow_uuid; - ovs_list_push_back(&lflows_todo, &lrln_uuid->list_node); - } - if (ovs_list_is_empty(&lflows_todo)) { - return true; - } - *changed = true; + struct lflow_ctx_in *l_ctx_in = CONST_CAST(struct lflow_ctx_in *, in_arg); + struct lflow_ctx_out *l_ctx_out = out_arg; /* Re-parse the related lflows. */ /* Firstly, flood remove the flows from desired flow table. */ + struct object_to_resources_list_node *resource_list_node_uuid; struct uuidset flood_remove_nodes = UUIDSET_INITIALIZER(&flood_remove_nodes); - LIST_FOR_EACH_SAFE (lrln_uuid, list_node, &lflows_todo) { - VLOG_DBG("Reprocess lflow "UUID_FMT" for resource type: %d," + LIST_FOR_EACH_SAFE (resource_list_node_uuid, list_node, objs_todo) { + const struct uuid *obj_uuid = &resource_list_node_uuid->obj_uuid; + VLOG_DBG("Reprocess lflow "UUID_FMT" for resource type: %s," " name: %s.", - UUID_ARGS(&lrln_uuid->lflow_uuid), - ref_type, ref_name); - uuidset_insert(&flood_remove_nodes, &lrln_uuid->lflow_uuid); - free(lrln_uuid); + UUID_ARGS(obj_uuid), objdep_type_name(type), res_name); + uuidset_insert(&flood_remove_nodes, obj_uuid); + free(resource_list_node_uuid); } ofctrl_flood_remove_flows(l_ctx_out->flow_table, &flood_remove_nodes); /* Secondly, for each lflow that is actually removed, reprocessing it. */ struct uuidset_node *ofrn; UUIDSET_FOR_EACH (ofrn, &flood_remove_nodes) { - lflow_resource_destroy_lflow(l_ctx_out->lfrr, &ofrn->uuid); + objdep_mgr_remove_obj(l_ctx_out->lflow_deps_mgr, &ofrn->uuid); lflow_conj_ids_free(l_ctx_out->conj_ids, &ofrn->uuid); const struct sbrec_logical_flow *lflow = @@ -910,9 +724,9 @@ lflow_handle_changed_ref(enum ref_type ref_type, const char *ref_name, &ofrn->uuid); if (!lflow) { VLOG_DBG("lflow "UUID_FMT" not found while reprocessing for" - " resource type: %d, name: %s.", + " resource type: %s, name: %s.", UUID_ARGS(&ofrn->uuid), - ref_type, ref_name); + objdep_type_name(type), res_name); continue; } @@ -929,8 +743,7 @@ lflow_handle_changed_ref(enum ref_type ref_type, const char *ref_name, consider_logical_flow(lflow, false, l_ctx_in, l_ctx_out); } uuidset_destroy(&flood_remove_nodes); - - return ret; + return true; } static void @@ -985,7 +798,7 @@ add_matches_to_flow_table(const struct sbrec_logical_flow *lflow, .sbrec_port_binding_by_name = l_ctx_in->sbrec_port_binding_by_name, .dp = ldp->datapath, .lflow = lflow, - .lfrr = l_ctx_out->lfrr, + .deps_mgr = l_ctx_out->lflow_deps_mgr, .chassis_tunnels = l_ctx_in->chassis_tunnels, }; @@ -1100,7 +913,7 @@ convert_match_to_expr(const struct sbrec_logical_flow *lflow, struct expr **prereqs, const struct shash *addr_sets, const struct shash *port_groups, - struct lflow_resource_ref *lfrr, + struct objdep_mgr *mgr, bool *pg_addr_set_ref) { struct shash addr_sets_ref = SHASH_INITIALIZER(&addr_sets_ref); @@ -1114,14 +927,15 @@ convert_match_to_expr(const struct sbrec_logical_flow *lflow, &error); struct shash_node *addr_sets_ref_node; SHASH_FOR_EACH (addr_sets_ref_node, &addr_sets_ref) { - lflow_resource_add(lfrr, REF_TYPE_ADDRSET, addr_sets_ref_node->name, - &lflow->header_.uuid, - *(size_t *)addr_sets_ref_node->data); + objdep_mgr_add_with_refcount(mgr, OBJDEP_TYPE_ADDRSET, + addr_sets_ref_node->name, + &lflow->header_.uuid, + *(size_t *) addr_sets_ref_node->data); } const char *port_group_name; SSET_FOR_EACH (port_group_name, &port_groups_ref) { - lflow_resource_add(lfrr, REF_TYPE_PORTGROUP, port_group_name, - &lflow->header_.uuid, 0); + objdep_mgr_add(mgr, OBJDEP_TYPE_PORTGROUP, port_group_name, + &lflow->header_.uuid); } if (pg_addr_set_ref) { @@ -1165,8 +979,8 @@ consider_logical_flow__(const struct sbrec_logical_flow *lflow, const char *io_port = smap_get(&lflow->tags, "in_out_port"); if (io_port) { - lflow_resource_add(l_ctx_out->lfrr, REF_TYPE_PORTBINDING, io_port, - &lflow->header_.uuid, 0); + objdep_mgr_add(l_ctx_out->lflow_deps_mgr, OBJDEP_TYPE_PORTBINDING, + io_port, &lflow->header_.uuid); const struct sbrec_port_binding *pb = lport_lookup_by_name(l_ctx_in->sbrec_port_binding_by_name, io_port); @@ -1232,7 +1046,7 @@ consider_logical_flow__(const struct sbrec_logical_flow *lflow, .sbrec_port_binding_by_name = l_ctx_in->sbrec_port_binding_by_name, .dp = dp, .lflow = lflow, - .lfrr = l_ctx_out->lfrr, + .deps_mgr = l_ctx_out->lflow_deps_mgr, }; struct condition_aux cond_aux = { .sbrec_port_binding_by_name = l_ctx_in->sbrec_port_binding_by_name, @@ -1240,7 +1054,7 @@ consider_logical_flow__(const struct sbrec_logical_flow *lflow, .chassis = l_ctx_in->chassis, .active_tunnels = l_ctx_in->active_tunnels, .lflow = lflow, - .lfrr = l_ctx_out->lfrr, + .deps_mgr = l_ctx_out->lflow_deps_mgr, }; struct lflow_cache_value *lcv = @@ -1272,7 +1086,8 @@ consider_logical_flow__(const struct sbrec_logical_flow *lflow, switch (lcv_type) { case LCACHE_T_NONE: expr = convert_match_to_expr(lflow, ldp, &prereqs, l_ctx_in->addr_sets, - l_ctx_in->port_groups, l_ctx_out->lfrr, + l_ctx_in->port_groups, + l_ctx_out->lflow_deps_mgr, &pg_addr_set_ref); if (!expr) { goto done; @@ -1345,8 +1160,8 @@ consider_logical_flow__(const struct sbrec_logical_flow *lflow, /* Cache new entry if caching is enabled. */ if (lflow_cache_is_enabled(l_ctx_out->lflow_cache)) { if (cached_expr - && !lflow_ref_lookup(&l_ctx_out->lfrr->lflow_ref_table, - &lflow->header_.uuid)) { + && !objdep_mgr_contains_obj(l_ctx_out->lflow_deps_mgr, + &lflow->header_.uuid)) { lflow_cache_add_matches(l_ctx_out->lflow_cache, &lflow->header_.uuid, start_conj_id, n_conjs, matches, matches_size); @@ -2448,7 +2263,7 @@ lflow_add_flows_for_datapath(const struct sbrec_datapath_binding *dp, &lflow->header_.uuid)) { continue; } - /* Don't call lflows_processed_add() because here we process the + /* Don't call uuidset_insert() because here we process the * lflow only for one of the DPs in the DP group, which may be * incomplete. */ consider_logical_flow__(lflow, dp, l_ctx_in, l_ctx_out); @@ -2514,7 +2329,11 @@ lflow_handle_flows_for_lport(const struct sbrec_port_binding *pb, { bool changed; - if (!lflow_handle_changed_ref(REF_TYPE_PORTBINDING, pb->logical_port, + if (!objdep_mgr_handle_change(l_ctx_out->lflow_deps_mgr, + OBJDEP_TYPE_PORTBINDING, + pb->logical_port, + lflow_handle_changed_ref, + l_ctx_out->lflows_processed, l_ctx_in, l_ctx_out, &changed)) { return false; } @@ -2549,7 +2368,11 @@ lflow_handle_changed_port_bindings(struct lflow_ctx_in *l_ctx_in, && !sbrec_port_binding_is_deleted(pb)) { continue; } - if (!lflow_handle_changed_ref(REF_TYPE_PORTBINDING, pb->logical_port, + if (!objdep_mgr_handle_change(l_ctx_out->lflow_deps_mgr, + OBJDEP_TYPE_PORTBINDING, + pb->logical_port, + lflow_handle_changed_ref, + l_ctx_out->lflows_processed, l_ctx_in, l_ctx_out, &changed)) { ret = false; break; @@ -2573,7 +2396,10 @@ lflow_handle_changed_mc_groups(struct lflow_ctx_in *l_ctx_in, && !sbrec_multicast_group_is_deleted(mg)) { continue; } - if (!lflow_handle_changed_ref(REF_TYPE_MC_GROUP, ds_cstr(&mg_key), + if (!objdep_mgr_handle_change(l_ctx_out->lflow_deps_mgr, + OBJDEP_TYPE_MC_GROUP, ds_cstr(&mg_key), + lflow_handle_changed_ref, + l_ctx_out->lflows_processed, l_ctx_in, l_ctx_out, &changed)) { ret = false; break; diff --git a/controller/lflow.h b/controller/lflow.h index 8cbe312cac..3f369ebfb6 100644 --- a/controller/lflow.h +++ b/controller/lflow.h @@ -16,6 +16,9 @@ #ifndef OVN_LFLOW_H #define OVN_LFLOW_H 1 +#include "lib/ovn-util.h" +#include "lib/objdep.h" +#include "lib/uuidset.h" #include "ovn/logical-fields.h" /* Logical_Flow table translation to OpenFlow @@ -80,59 +83,6 @@ struct uuid; #define OFTABLE_ECMP_NH_MAC 76 #define OFTABLE_ECMP_NH 77 -enum ref_type { - REF_TYPE_ADDRSET, - REF_TYPE_PORTGROUP, - REF_TYPE_PORTBINDING, - REF_TYPE_MC_GROUP -}; - -struct ref_lflow_node { - struct hmap_node node; /* node in lflow_resource_ref.ref_lflow_table. */ - enum ref_type type; /* key */ - char *ref_name; /* key */ - struct hmap lflow_uuids; /* Contains lflow_ref_list_node. Use hmap instead - of list so that lflow_resource_add() can check - and avoid adding redundant entires in O(1). */ -}; - -struct lflow_ref_node { - struct hmap_node node; /* node in lflow_resource_ref.lflow_ref_table. */ - struct uuid lflow_uuid; /* key */ - struct ovs_list lflow_ref_head; /* Contains lflow_ref_list_node. */ -}; - -/* Maintains the relationship for a pair of named resource and - * a lflow, indexed by both ref_lflow_table and lflow_ref_table. */ -struct lflow_ref_list_node { - struct ovs_list list_node; /* node in lflow_ref_node.lflow_ref_head. */ - struct hmap_node hmap_node; /* node in ref_lflow_node.lflow_uuids. */ - struct uuid lflow_uuid; - size_t ref_count; /* Reference count of the resource by this lflow. - Currently used for the resource type REF_TYPE_ADDRSET - only, and for other types it is always 0. */ - struct ref_lflow_node *rlfn; -}; - -struct lflow_resource_ref { - /* A map from a referenced resource type & name (e.g. address_set AS1) - * to a list of lflows that are referencing the named resource. Data - * type of each node in this hmap is struct ref_lflow_node. The - * ref_lflow_head in each node points to a list of - * lflow_ref_list_node.ref_list. */ - struct hmap ref_lflow_table; - - /* A map from a lflow uuid to a list of named resources that are - * referenced by the lflow. Data type of each node in this hmap is - * struct lflow_ref_node. The lflow_ref_head in each node points to - * a list of lflow_ref_list_node.lflow_list. */ - struct hmap lflow_ref_table; -}; - -void lflow_resource_init(struct lflow_resource_ref *); -void lflow_resource_destroy(struct lflow_resource_ref *); -void lflow_resource_clear(struct lflow_resource_ref *); - struct lflow_ctx_in { struct ovsdb_idl_index *sbrec_multicast_group_by_name_datapath; struct ovsdb_idl_index *sbrec_logical_flow_by_logical_datapath; @@ -169,7 +119,7 @@ struct lflow_ctx_out { struct ovn_desired_flow_table *flow_table; struct ovn_extend_table *group_table; struct ovn_extend_table *meter_table; - struct lflow_resource_ref *lfrr; + struct objdep_mgr *lflow_deps_mgr; struct lflow_cache *lflow_cache; struct conj_ids *conj_ids; struct uuidset *lflows_processed; @@ -181,10 +131,8 @@ void lflow_init(void); void lflow_run(struct lflow_ctx_in *, struct lflow_ctx_out *); void lflow_handle_cached_flows(struct lflow_cache *, const struct sbrec_logical_flow_table *); -bool lflow_handle_changed_flows(struct lflow_ctx_in *, struct lflow_ctx_out *); -bool lflow_handle_changed_ref(enum ref_type, const char *ref_name, - struct lflow_ctx_in *, struct lflow_ctx_out *, - bool *changed); +bool lflow_handle_changed_flows(struct lflow_ctx_in *, + struct lflow_ctx_out *); struct addr_set_diff { struct expr_constant_set *added; @@ -194,6 +142,9 @@ bool lflow_handle_addr_set_update(const char *as_name, struct addr_set_diff *, struct lflow_ctx_in *, struct lflow_ctx_out *, bool *changed); +bool lflow_handle_changed_ref(enum objdep_type, const char *res_name, + struct ovs_list *objs_todo, + const void *in_arg, void *out_arg); void lflow_handle_changed_mac_bindings( struct ovsdb_idl_index *sbrec_port_binding_by_name, diff --git a/controller/ovn-controller.c b/controller/ovn-controller.c index 8895c7a2bd..f62303e3cb 100644 --- a/controller/ovn-controller.c +++ b/controller/ovn-controller.c @@ -2578,8 +2578,8 @@ struct ed_type_lflow_output { struct ovn_extend_table group_table; /* meter ids for QoS */ struct ovn_extend_table meter_table; - /* lflow resource cross reference */ - struct lflow_resource_ref lflow_resource_ref; + /* lflow <-> resource cross reference */ + struct objdep_mgr lflow_deps_mgr;; /* conjunciton ID usage information of lflows */ struct conj_ids conj_ids; @@ -2738,7 +2738,7 @@ init_lflow_ctx(struct engine_node *node, l_ctx_out->flow_table = &fo->flow_table; l_ctx_out->group_table = &fo->group_table; l_ctx_out->meter_table = &fo->meter_table; - l_ctx_out->lfrr = &fo->lflow_resource_ref; + l_ctx_out->lflow_deps_mgr = &fo->lflow_deps_mgr; l_ctx_out->conj_ids = &fo->conj_ids; l_ctx_out->lflows_processed = &fo->lflows_processed; l_ctx_out->lflow_cache = fo->pd.lflow_cache; @@ -2754,7 +2754,7 @@ en_lflow_output_init(struct engine_node *node OVS_UNUSED, ovn_desired_flow_table_init(&data->flow_table); ovn_extend_table_init(&data->group_table); ovn_extend_table_init(&data->meter_table); - lflow_resource_init(&data->lflow_resource_ref); + objdep_mgr_init(&data->lflow_deps_mgr); lflow_conj_ids_init(&data->conj_ids); uuidset_init(&data->lflows_processed); simap_init(&data->hd.ids); @@ -2778,7 +2778,7 @@ en_lflow_output_cleanup(void *data) ovn_desired_flow_table_destroy(&flow_output_data->flow_table); ovn_extend_table_destroy(&flow_output_data->group_table); ovn_extend_table_destroy(&flow_output_data->meter_table); - lflow_resource_destroy(&flow_output_data->lflow_resource_ref); + objdep_mgr_destroy(&flow_output_data->lflow_deps_mgr); lflow_conj_ids_destroy(&flow_output_data->conj_ids); uuidset_destroy(&flow_output_data->lflows_processed); lflow_cache_destroy(flow_output_data->pd.lflow_cache); @@ -2814,7 +2814,7 @@ en_lflow_output_run(struct engine_node *node, void *data) struct ovn_desired_flow_table *lflow_table = &fo->flow_table; struct ovn_extend_table *group_table = &fo->group_table; struct ovn_extend_table *meter_table = &fo->meter_table; - struct lflow_resource_ref *lfrr = &fo->lflow_resource_ref; + struct objdep_mgr *lflow_deps_mgr = &fo->lflow_deps_mgr; static bool first_run = true; if (first_run) { @@ -2823,7 +2823,7 @@ en_lflow_output_run(struct engine_node *node, void *data) ovn_desired_flow_table_clear(lflow_table); ovn_extend_table_clear(group_table, false /* desired */); ovn_extend_table_clear(meter_table, false /* desired */); - lflow_resource_clear(lfrr); + objdep_mgr_clear(lflow_deps_mgr); lflow_conj_ids_clear(&fo->conj_ids); } @@ -2954,8 +2954,11 @@ lflow_output_addr_sets_handler(struct engine_node *node, void *data) } SSET_FOR_EACH (ref_name, &as_data->deleted) { - if (!lflow_handle_changed_ref(REF_TYPE_ADDRSET, ref_name, &l_ctx_in, - &l_ctx_out, &changed)) { + if (!objdep_mgr_handle_change(l_ctx_out.lflow_deps_mgr, + OBJDEP_TYPE_ADDRSET, ref_name, + lflow_handle_changed_ref, + l_ctx_out.lflows_processed, + &l_ctx_in, &l_ctx_out, &changed)) { return false; } if (changed) { @@ -2969,7 +2972,11 @@ lflow_output_addr_sets_handler(struct engine_node *node, void *data) &l_ctx_out, &changed)) { VLOG_DBG("Can't incrementally handle the change of address set %s." " Reprocess related lflows.", shash_node->name); - if (!lflow_handle_changed_ref(REF_TYPE_ADDRSET, shash_node->name, + if (!objdep_mgr_handle_change(l_ctx_out.lflow_deps_mgr, + OBJDEP_TYPE_ADDRSET, + shash_node->name, + lflow_handle_changed_ref, + l_ctx_out.lflows_processed, &l_ctx_in, &l_ctx_out, &changed)) { return false; } @@ -2979,8 +2986,11 @@ lflow_output_addr_sets_handler(struct engine_node *node, void *data) } } SSET_FOR_EACH (ref_name, &as_data->new) { - if (!lflow_handle_changed_ref(REF_TYPE_ADDRSET, ref_name, &l_ctx_in, - &l_ctx_out, &changed)) { + if (!objdep_mgr_handle_change(l_ctx_out.lflow_deps_mgr, + OBJDEP_TYPE_ADDRSET, ref_name, + lflow_handle_changed_ref, + l_ctx_out.lflows_processed, + &l_ctx_in, &l_ctx_out, &changed)) { return false; } if (changed) { @@ -3011,8 +3021,11 @@ lflow_output_port_groups_handler(struct engine_node *node, void *data) } SSET_FOR_EACH (ref_name, &pg_data->deleted) { - if (!lflow_handle_changed_ref(REF_TYPE_PORTGROUP, ref_name, &l_ctx_in, - &l_ctx_out, &changed)) { + if (!objdep_mgr_handle_change(l_ctx_out.lflow_deps_mgr, + OBJDEP_TYPE_PORTGROUP, ref_name, + lflow_handle_changed_ref, + l_ctx_out.lflows_processed, + &l_ctx_in, &l_ctx_out, &changed)) { return false; } if (changed) { @@ -3020,8 +3033,11 @@ lflow_output_port_groups_handler(struct engine_node *node, void *data) } } SSET_FOR_EACH (ref_name, &pg_data->updated) { - if (!lflow_handle_changed_ref(REF_TYPE_PORTGROUP, ref_name, &l_ctx_in, - &l_ctx_out, &changed)) { + if (!objdep_mgr_handle_change(l_ctx_out.lflow_deps_mgr, + OBJDEP_TYPE_PORTGROUP, ref_name, + lflow_handle_changed_ref, + l_ctx_out.lflows_processed, + &l_ctx_in, &l_ctx_out, &changed)) { return false; } if (changed) { @@ -3029,8 +3045,11 @@ lflow_output_port_groups_handler(struct engine_node *node, void *data) } } SSET_FOR_EACH (ref_name, &pg_data->new) { - if (!lflow_handle_changed_ref(REF_TYPE_PORTGROUP, ref_name, &l_ctx_in, - &l_ctx_out, &changed)) { + if (!objdep_mgr_handle_change(l_ctx_out.lflow_deps_mgr, + OBJDEP_TYPE_PORTGROUP, ref_name, + lflow_handle_changed_ref, + l_ctx_out.lflows_processed, + &l_ctx_in, &l_ctx_out, &changed)) { return false; } if (changed) { diff --git a/lib/automake.mk b/lib/automake.mk index 60bead6a6a..15d4f84bbb 100644 --- a/lib/automake.mk +++ b/lib/automake.mk @@ -31,6 +31,8 @@ lib_libovn_la_SOURCES = \ lib/mcast-group-index.c \ lib/mcast-group-index.h \ lib/lex.c \ + lib/objdep.c \ + lib/objdep.h \ lib/ovn-l7.h \ lib/ovn-l7.c \ lib/ovn-util.c \ diff --git a/lib/objdep.c b/lib/objdep.c new file mode 100644 index 0000000000..092d4af261 --- /dev/null +++ b/lib/objdep.c @@ -0,0 +1,260 @@ +/* Copyright (c) 2015, 2016, 2017 Nicira, Inc. + * Copyright (c) 2022, Red Hat, Inc. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at: + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#include + +#include "lib/objdep.h" +#include "lib/hash.h" +#include "lib/util.h" +#include "openvswitch/vlog.h" + +VLOG_DEFINE_THIS_MODULE(resource_dep); + +static void resource_node_destroy(struct resource_to_objects_node *); + +void +objdep_mgr_init(struct objdep_mgr *mgr) +{ + hmap_init(&mgr->resource_to_objects_table); + hmap_init(&mgr->object_to_resources_table); +} + +void +objdep_mgr_destroy(struct objdep_mgr *mgr) +{ + objdep_mgr_clear(mgr); + + hmap_destroy(&mgr->resource_to_objects_table); + hmap_destroy(&mgr->object_to_resources_table); +} + +void +objdep_mgr_clear(struct objdep_mgr *mgr) +{ + struct resource_to_objects_node *resource_node; + HMAP_FOR_EACH_SAFE (resource_node, node, &mgr->resource_to_objects_table) { + struct object_to_resources_list_node *object_list_node; + HMAP_FOR_EACH_SAFE (object_list_node, hmap_node, + &resource_node->objs) { + ovs_list_remove(&object_list_node->list_node); + hmap_remove(&resource_node->objs, &object_list_node->hmap_node); + free(object_list_node); + } + hmap_remove(&mgr->resource_to_objects_table, &resource_node->node); + resource_node_destroy(resource_node); + } + + struct object_to_resources_node *object_node; + HMAP_FOR_EACH_SAFE (object_node, node, &mgr->object_to_resources_table) { + hmap_remove(&mgr->object_to_resources_table, &object_node->node); + free(object_node); + } +} + +void +objdep_mgr_add(struct objdep_mgr *mgr, enum objdep_type type, + const char *res_name, const struct uuid *obj_uuid) +{ + objdep_mgr_add_with_refcount(mgr, type, res_name, obj_uuid, 0); +} + +void +objdep_mgr_add_with_refcount(struct objdep_mgr *mgr, enum objdep_type type, + const char *res_name, const struct uuid *obj_uuid, + size_t ref_count) +{ + struct resource_to_objects_node *resource_node = + objdep_mgr_find_objs(mgr, type, res_name); + struct object_to_resources_node *object_node = + objdep_mgr_find_resources(mgr, obj_uuid); + if (resource_node && object_node) { + /* Check if the mapping already existed before adding a new one. */ + struct object_to_resources_list_node *n; + HMAP_FOR_EACH_WITH_HASH (n, hmap_node, uuid_hash(obj_uuid), + &resource_node->objs) { + if (uuid_equals(&n->obj_uuid, obj_uuid)) { + return; + } + } + } + + /* Create the resource node if we didn't have one already (for a + * different object). */ + if (!resource_node) { + resource_node = xzalloc(sizeof *resource_node); + resource_node->node.hash = hash_string(res_name, type); + resource_node->type = type; + resource_node->res_name = xstrdup(res_name); + hmap_init(&resource_node->objs); + hmap_insert(&mgr->resource_to_objects_table, + &resource_node->node, + resource_node->node.hash); + } + + /* Create the object node if we didn't have one already (for a + * different resource). */ + if (!object_node) { + object_node = xzalloc(sizeof *object_node); + object_node->node.hash = uuid_hash(obj_uuid); + object_node->obj_uuid = *obj_uuid; + ovs_list_init(&object_node->resources_head); + hmap_insert(&mgr->object_to_resources_table, + &object_node->node, + object_node->node.hash); + } + + struct object_to_resources_list_node *resource_list_node = + xzalloc(sizeof *resource_list_node); + resource_list_node->obj_uuid = *obj_uuid; + resource_list_node->ref_count = ref_count; + resource_list_node->resource_node = resource_node; + hmap_insert(&resource_node->objs, &resource_list_node->hmap_node, + uuid_hash(obj_uuid)); + ovs_list_push_back(&object_node->resources_head, + &resource_list_node->list_node); +} + +void +objdep_mgr_remove_obj(struct objdep_mgr *mgr, const struct uuid *obj_uuid) +{ + struct object_to_resources_node *object_node = + objdep_mgr_find_resources(mgr, obj_uuid); + if (!object_node) { + return; + } + + hmap_remove(&mgr->object_to_resources_table, &object_node->node); + + struct object_to_resources_list_node *resource_list_node; + LIST_FOR_EACH_SAFE (resource_list_node, list_node, + &object_node->resources_head) { + struct resource_to_objects_node *resource_node = + resource_list_node->resource_node; + ovs_list_remove(&resource_list_node->list_node); + hmap_remove(&resource_node->objs, &resource_list_node->hmap_node); + + /* Clean up the node in ref_obj_table if the resource is not + * referred by any logical flows. */ + if (hmap_is_empty(&resource_node->objs)) { + hmap_remove(&mgr->resource_to_objects_table, &resource_node->node); + resource_node_destroy(resource_list_node->resource_node); + } + + free(resource_list_node); + } + free(object_node); +} + +struct resource_to_objects_node * +objdep_mgr_find_objs(struct objdep_mgr *mgr, enum objdep_type type, + const char *res_name) +{ + struct resource_to_objects_node *resource_node; + + HMAP_FOR_EACH_WITH_HASH (resource_node, node, hash_string(res_name, type), + &mgr->resource_to_objects_table) { + if (resource_node->type == type && + !strcmp(resource_node->res_name, res_name)) { + return resource_node; + } + } + return NULL; +} + +struct object_to_resources_node * +objdep_mgr_find_resources(struct objdep_mgr *mgr, + const struct uuid *obj_uuid) +{ + struct object_to_resources_node *object_node; + + HMAP_FOR_EACH_WITH_HASH (object_node, node, uuid_hash(obj_uuid), + &mgr->object_to_resources_table) { + if (uuid_equals(&object_node->obj_uuid, obj_uuid)) { + return object_node; + } + } + return NULL; +} + +bool +objdep_mgr_contains_obj(struct objdep_mgr *mgr, const struct uuid *obj_uuid) +{ + return !!objdep_mgr_find_resources(mgr, obj_uuid); +} + +bool +objdep_mgr_handle_change(struct objdep_mgr *mgr, + enum objdep_type type, + const char *res_name, + objdep_change_handler handler, + struct uuidset *objs_processed, + const void *in_arg, void *out_arg, + bool *changed) +{ + struct resource_to_objects_node *resource_node = + objdep_mgr_find_objs(mgr, type, res_name); + if (!resource_node) { + *changed = false; + return true; + } + VLOG_DBG("Handle changed object reference for resource type: %s," + " name: %s.", objdep_type_name(type), res_name); + *changed = false; + + struct ovs_list objs_todo = OVS_LIST_INITIALIZER(&objs_todo); + + struct object_to_resources_list_node *resource_list_node; + HMAP_FOR_EACH (resource_list_node, hmap_node, &resource_node->objs) { + if (uuidset_find(objs_processed, &resource_list_node->obj_uuid)) { + continue; + } + /* Use object_to_resources_list_node as list node to store the uuid. + * Other fields are not used here. */ + struct object_to_resources_list_node *resource_list_node_uuid = + xmalloc(sizeof *resource_list_node_uuid); + resource_list_node_uuid->obj_uuid = resource_list_node->obj_uuid; + ovs_list_push_back(&objs_todo, &resource_list_node_uuid->list_node); + } + if (ovs_list_is_empty(&objs_todo)) { + return true; + } + *changed = true; + + /* This takes ownership of objs_todo. */ + return handler(type, res_name, &objs_todo, in_arg, out_arg); +} + +const char * +objdep_type_name(enum objdep_type type) +{ + static const char *type_names[OBJDEP_TYPE_MAX] = { + [OBJDEP_TYPE_ADDRSET] = "Address_Set", + [OBJDEP_TYPE_PORTGROUP] = "Port_Group", + [OBJDEP_TYPE_PORTBINDING] = "Port_Binding", + [OBJDEP_TYPE_MC_GROUP] = "Multicast_Group", + }; + + ovs_assert(type < OBJDEP_TYPE_MAX); + return type_names[type]; +} + +static void +resource_node_destroy(struct resource_to_objects_node *resource_node) +{ + free(resource_node->res_name); + hmap_destroy(&resource_node->objs); + free(resource_node); +} diff --git a/lib/objdep.h b/lib/objdep.h new file mode 100644 index 0000000000..50c7b01ef1 --- /dev/null +++ b/lib/objdep.h @@ -0,0 +1,122 @@ +/* Copyright (c) 2015, 2016, 2017 Nicira, Inc. + * Copyright (c) 2022, Red Hat, Inc. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at: + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#ifndef OVN_OBJDEP_H +#define OVN_OBJDEP_H 1 + +#include "lib/uuidset.h" +#include "openvswitch/hmap.h" +#include "openvswitch/list.h" + +enum objdep_type { + OBJDEP_TYPE_ADDRSET, + OBJDEP_TYPE_PORTGROUP, + OBJDEP_TYPE_PORTBINDING, + OBJDEP_TYPE_MC_GROUP, + OBJDEP_TYPE_MAX, +}; + +/* Callbacks provided by users to process changes to resources referred by + * various objects. They should return true if the change has been + * handled successfully. */ +typedef bool (*objdep_change_handler)(enum objdep_type, + const char *res_name, + struct ovs_list *ref_nodes, + const void *in_arg, void *out_arg); + +/* A node pointing to all objects that refer to a given resource. */ +struct resource_to_objects_node { + struct hmap_node node; /* node in objdep_mgr.resource_to_objects_table. */ + enum objdep_type type; /* key */ + char *res_name; /* key */ + struct hmap objs; /* Contains object_to_resources_list_node. + * Use hmap instead of list so + * that obj_resource_add() can check and avoid + * and redundant entries in O(1). */ +}; + +#define RESOURCE_FOR_EACH_OBJ(NODE, MAP) \ + HMAP_FOR_EACH (NODE, hmap_node, &(MAP)->objs) + +/* A node pointing to all resources used by a given object (specified by + * uuid). + */ +struct object_to_resources_node { + struct hmap_node node; /* node in objdep_mgr.object_to_resources_table. */ + struct uuid obj_uuid; /* key */ + struct ovs_list resources_head; /* Contains elements of type + * object_to_resources_list_node. */ +}; + +/* Maintains the relationship for a pair of named resource and + * an object, indexed by both resource_to_object_table and + * object_to_resources_table. */ +struct object_to_resources_list_node { + /* node in object_to_resources_node.resources_head. */ + struct ovs_list list_node; + struct hmap_node hmap_node; /* node in resource_to_objects_node.objs. */ + struct uuid obj_uuid; + size_t ref_count; /* Reference count of the resource by this object. + * Currently only used for the resource type + * OBJDEP_TYPE_ADDRSET and for other types always + * set to 0. */ + struct resource_to_objects_node *resource_node; +}; + +struct objdep_mgr { + /* A map from a referenced resource type & name (e.g. address_set AS1) + * to a list of object UUIDs (e.g., lflow) that are referencing the named + * resource. Data type of each node in this hmap is struct + * resource_to_objects_node. The objs in each node point + * to a map of object_to_resources_list_node.ref_list. */ + struct hmap resource_to_objects_table; + + /* A map from a obj uuid to a list of named resources that are + * referenced by the object. Data type of each node in this hmap is + * struct object_to_resources_node. The resources_head in each node + * points to a list of object_to_resources_list_node.obj_list. */ + struct hmap object_to_resources_table; +}; + +void objdep_mgr_init(struct objdep_mgr *); +void objdep_mgr_destroy(struct objdep_mgr *); +void objdep_mgr_clear(struct objdep_mgr *); + +void objdep_mgr_add(struct objdep_mgr *, enum objdep_type, + const char *res_name, const struct uuid *); +void objdep_mgr_add_with_refcount(struct objdep_mgr *, + enum objdep_type, + const char *res_name, + const struct uuid *, + size_t ref_count); +void objdep_mgr_remove_obj(struct objdep_mgr *, const struct uuid *); + +struct resource_to_objects_node *objdep_mgr_find_objs( + struct objdep_mgr *, enum objdep_type, const char *res_name); +struct object_to_resources_node *objdep_mgr_find_resources( + struct objdep_mgr *, const struct uuid *); +bool objdep_mgr_contains_obj(struct objdep_mgr *, const struct uuid *); + +bool objdep_mgr_handle_change(struct objdep_mgr *, enum objdep_type, + const char *res_name, + objdep_change_handler handler, + struct uuidset *objs_processed, + const void *in_arg, void *out_arg, + bool *changed); + +const char *objdep_type_name(enum objdep_type); + +#endif /* lib/objdep.h */ From patchwork Fri Nov 4 22:11:23 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dumitru Ceara X-Patchwork-Id: 1699894 X-Patchwork-Delegate: mmichels@redhat.com Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::136; helo=smtp3.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=KPYzUo1T; dkim-atps=neutral Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4N3vx04ryxz23lT for ; Sat, 5 Nov 2022 09:11:48 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 6747B61183; Fri, 4 Nov 2022 22:11:46 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 6747B61183 Authentication-Results: smtp3.osuosl.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=KPYzUo1T X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Q538UjFGE2Na; Fri, 4 Nov 2022 22:11:44 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp3.osuosl.org (Postfix) with ESMTPS id 12049611A0; Fri, 4 Nov 2022 22:11:43 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 12049611A0 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id DC298C0032; Fri, 4 Nov 2022 22:11:42 +0000 (UTC) X-Original-To: ovs-dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp2.osuosl.org (smtp2.osuosl.org [IPv6:2605:bc80:3010::133]) by lists.linuxfoundation.org (Postfix) with ESMTP id 4FC41C002D for ; Fri, 4 Nov 2022 22:11:41 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 2E55F40DA0 for ; Fri, 4 Nov 2022 22:11:33 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 2E55F40DA0 Authentication-Results: smtp2.osuosl.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=KPYzUo1T X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 6DRJNKzuXbpM for ; Fri, 4 Nov 2022 22:11:31 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 0FC9340D78 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by smtp2.osuosl.org (Postfix) with ESMTPS id 0FC9340D78 for ; Fri, 4 Nov 2022 22:11:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1667599889; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=L40aNOGeusroGVKHuC7yFHr8tc8//IsCSR4izVBnV9c=; b=KPYzUo1TS2jWcA7DL5g6o7UlHmbAkZRtfyB16M24bZgkUxv5aDecrmgJANEg/L+gn9YKNB 3n/FPDAsF5PixnAV2BtK8CqZD/QvvhfgiI4HUHWEG1TnfuKOzC6qjnXArN65C4FYYkfwhL 6E0oCt0X1m88B2FFR/coeQb4m8vwKfI= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-479-j8Pg9huhNFGPZwfeOPjdtQ-1; Fri, 04 Nov 2022 18:11:27 -0400 X-MC-Unique: j8Pg9huhNFGPZwfeOPjdtQ-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id A857D85A583; Fri, 4 Nov 2022 22:11:26 +0000 (UTC) Received: from dceara.remote.csb (unknown [10.39.192.36]) by smtp.corp.redhat.com (Postfix) with ESMTP id 9C698C16932; Fri, 4 Nov 2022 22:11:25 +0000 (UTC) From: Dumitru Ceara To: ovs-dev@openvswitch.org Date: Fri, 4 Nov 2022 23:11:23 +0100 Message-Id: <166759988115.515460.8092347969356435254.stgit@dceara.remote.csb> In-Reply-To: <166759985955.515460.16275468131013093936.stgit@dceara.remote.csb> References: <166759985955.515460.16275468131013093936.stgit@dceara.remote.csb> User-Agent: StGit/0.23 MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.8 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Cc: surya@redhat.com, i.maximets@ovn.org Subject: [ovs-dev] [PATCH ovn v2 2/5] Add NB and SB Template_Var tables. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" Propagate the contents of the NB table to the Southbound. Signed-off-by: Dumitru Ceara --- Note: - ovn-trace doesn't support template variables (yet). V2: - Fixed TEMPLATE_VAR_TABLE_INITIALIZER definition so that GCC doesn't complain anymore. - Addressed Han's comments: - Rename tables to Chassis_Template_Var. - Fix man page. - Simplify function prototypes. - Changed schema as suggested by Ilya. --- northd/automake.mk | 4 ++ northd/en-northd.c | 4 ++ northd/inc-proc-northd.c | 8 ++++- northd/northd.c | 41 +++++++++++++++++++++++++ northd/northd.h | 4 ++ northd/template-var.c | 74 ++++++++++++++++++++++++++++++++++++++++++++++ northd/template-var.h | 58 ++++++++++++++++++++++++++++++++++++ ovn-nb.ovsschema | 17 +++++++++-- ovn-nb.xml | 29 ++++++++++++++++++ ovn-sb.ovsschema | 12 ++++++- ovn-sb.xml | 15 +++++++++ tests/ovn-northd.at | 35 ++++++++++++++++++++++ utilities/ovn-nbctl.c | 3 ++ utilities/ovn-sbctl.c | 3 ++ 14 files changed, 299 insertions(+), 8 deletions(-) create mode 100644 northd/template-var.c create mode 100644 northd/template-var.h diff --git a/northd/automake.mk b/northd/automake.mk index 81582867dc..31134bc329 100644 --- a/northd/automake.mk +++ b/northd/automake.mk @@ -13,7 +13,9 @@ northd_ovn_northd_SOURCES = \ northd/inc-proc-northd.c \ northd/inc-proc-northd.h \ northd/ipam.c \ - northd/ipam.h + northd/ipam.h \ + northd/template-var.c \ + northd/template-var.h northd_ovn_northd_LDADD = \ lib/libovn.la \ $(OVSDB_LIBDIR)/libovsdb.la \ diff --git a/northd/en-northd.c b/northd/en-northd.c index 7fe83db642..030ee25d8f 100644 --- a/northd/en-northd.c +++ b/northd/en-northd.c @@ -80,6 +80,8 @@ void en_northd_run(struct engine_node *node, void *data) EN_OVSDB_GET(engine_get_input("NB_acl", node)); input_data.nbrec_static_mac_binding_table = EN_OVSDB_GET(engine_get_input("NB_static_mac_binding", node)); + input_data.nbrec_chassis_template_var_table = + EN_OVSDB_GET(engine_get_input("NB_chassis_template_var", node)); input_data.sbrec_sb_global_table = EN_OVSDB_GET(engine_get_input("SB_sb_global", node)); @@ -113,6 +115,8 @@ void en_northd_run(struct engine_node *node, void *data) EN_OVSDB_GET(engine_get_input("SB_chassis_private", node)); input_data.sbrec_static_mac_binding_table = EN_OVSDB_GET(engine_get_input("SB_static_mac_binding", node)); + input_data.sbrec_chassis_template_var_table = + EN_OVSDB_GET(engine_get_input("SB_chassis_template_var", node)); northd_run(&input_data, data, eng_ctx->ovnnb_idl_txn, diff --git a/northd/inc-proc-northd.c b/northd/inc-proc-northd.c index 54e0ad3b05..da791f035d 100644 --- a/northd/inc-proc-northd.c +++ b/northd/inc-proc-northd.c @@ -64,7 +64,8 @@ VLOG_DEFINE_THIS_MODULE(inc_proc_northd); NB_NODE(ha_chassis_group, "ha_chassis_group") \ NB_NODE(ha_chassis, "ha_chassis") \ NB_NODE(bfd, "bfd") \ - NB_NODE(static_mac_binding, "static_mac_binding") + NB_NODE(static_mac_binding, "static_mac_binding") \ + NB_NODE(chassis_template_var, "chassis_template_var") enum nb_engine_node { #define NB_NODE(NAME, NAME_STR) NB_##NAME, @@ -114,7 +115,8 @@ VLOG_DEFINE_THIS_MODULE(inc_proc_northd); SB_NODE(load_balancer, "load_balancer") \ SB_NODE(bfd, "bfd") \ SB_NODE(fdb, "fdb") \ - SB_NODE(static_mac_binding, "static_mac_binding") + SB_NODE(static_mac_binding, "static_mac_binding") \ + SB_NODE(chassis_template_var, "chassis_template_var") enum sb_engine_node { #define SB_NODE(NAME, NAME_STR) SB_##NAME, @@ -186,6 +188,7 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, engine_add_input(&en_northd, &en_nb_ha_chassis_group, NULL); engine_add_input(&en_northd, &en_nb_ha_chassis, NULL); engine_add_input(&en_northd, &en_nb_static_mac_binding, NULL); + engine_add_input(&en_northd, &en_nb_chassis_template_var, NULL); engine_add_input(&en_northd, &en_sb_sb_global, NULL); engine_add_input(&en_northd, &en_sb_chassis, NULL); @@ -215,6 +218,7 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, engine_add_input(&en_northd, &en_sb_load_balancer, NULL); engine_add_input(&en_northd, &en_sb_fdb, NULL); engine_add_input(&en_northd, &en_sb_static_mac_binding, NULL); + engine_add_input(&en_northd, &en_sb_chassis_template_var, NULL); engine_add_input(&en_mac_binding_aging, &en_nb_nb_global, NULL); engine_add_input(&en_mac_binding_aging, &en_sb_mac_binding, NULL); engine_add_input(&en_mac_binding_aging, &en_northd, NULL); diff --git a/northd/northd.c b/northd/northd.c index b7388afc58..170b4f95c8 100644 --- a/northd/northd.c +++ b/northd/northd.c @@ -51,6 +51,7 @@ #include "lib/stopwatch-names.h" #include "stream.h" #include "timeval.h" +#include "template-var.h" #include "util.h" #include "uuid.h" #include "ovs-thread.h" @@ -15129,6 +15130,44 @@ sync_dns_entries(struct northd_input *input_data, } hmap_destroy(&dns_map); } + +static void +sync_template_vars(struct northd_input *input_data, + struct ovsdb_idl_txn *ovnsb_txn) +{ + struct template_var_table nb_tvs = TEMPLATE_VAR_TABLE_INITIALIZER(&nb_tvs); + + const struct nbrec_chassis_template_var *nb_tv; + const struct sbrec_chassis_template_var *sb_tv; + struct template_var *tv; + + NBREC_CHASSIS_TEMPLATE_VAR_TABLE_FOR_EACH ( + nb_tv, input_data->nbrec_chassis_template_var_table) { + template_var_insert(&nb_tvs, nb_tv); + } + + SBREC_CHASSIS_TEMPLATE_VAR_TABLE_FOR_EACH_SAFE ( + sb_tv, input_data->sbrec_chassis_template_var_table) { + tv = template_var_find(&nb_tvs, sb_tv->chassis); + if (!tv) { + sbrec_chassis_template_var_delete(sb_tv); + continue; + } + if (!smap_equal(&sb_tv->variables, &tv->nb->variables)) { + sbrec_chassis_template_var_set_variables(sb_tv, + &tv->nb->variables); + } + template_var_remove(&nb_tvs, tv); + template_var_destroy(tv); + } + + TEMPLATE_VAR_TABLE_FOR_EACH (tv, &nb_tvs) { + sb_tv = sbrec_chassis_template_var_insert(ovnsb_txn); + sbrec_chassis_template_var_set_chassis(sb_tv, tv->nb->chassis); + sbrec_chassis_template_var_set_variables(sb_tv, &tv->nb->variables); + } + template_var_table_destroy(&nb_tvs); +} static void destroy_datapaths_and_ports(struct hmap *datapaths, struct hmap *ports, @@ -15645,6 +15684,8 @@ ovnnb_db_run(struct northd_input *input_data, sync_port_groups(input_data, ovnsb_txn, &data->port_groups); sync_meters(input_data, ovnsb_txn, &data->meter_groups); sync_dns_entries(input_data, ovnsb_txn, &data->datapaths); + sync_template_vars(input_data, ovnsb_txn); + cleanup_stale_fdb_entries(input_data, &data->datapaths); stopwatch_stop(CLEAR_LFLOWS_CTX_STOPWATCH_NAME, time_msec()); diff --git a/northd/northd.h b/northd/northd.h index da90e28155..1935c80d40 100644 --- a/northd/northd.h +++ b/northd/northd.h @@ -36,6 +36,8 @@ struct northd_input { const struct nbrec_acl_table *nbrec_acl_table; const struct nbrec_static_mac_binding_table *nbrec_static_mac_binding_table; + const struct nbrec_chassis_template_var_table + *nbrec_chassis_template_var_table; /* Southbound table references */ const struct sbrec_sb_global_table *sbrec_sb_global_table; @@ -55,6 +57,8 @@ struct northd_input { const struct sbrec_chassis_private_table *sbrec_chassis_private_table; const struct sbrec_static_mac_binding_table *sbrec_static_mac_binding_table; + const struct sbrec_chassis_template_var_table + *sbrec_chassis_template_var_table; /* Indexes */ struct ovsdb_idl_index *sbrec_chassis_by_name; diff --git a/northd/template-var.c b/northd/template-var.c new file mode 100644 index 0000000000..a1069a5b51 --- /dev/null +++ b/northd/template-var.c @@ -0,0 +1,74 @@ +/* Copyright (c) 2022, Red Hat, Inc. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at: + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#include + +#include "template-var.h" + +struct template_var_table * +template_var_table_create(void) +{ + struct template_var_table *table = xmalloc(sizeof *table); + + hmap_init(&table->vars); + return table; +} + +void +template_var_table_destroy(struct template_var_table *table) +{ + struct template_var *tv; + + HMAP_FOR_EACH_POP (tv, hmap_node, &table->vars) { + template_var_destroy(tv); + } + hmap_destroy(&table->vars); +} + +void +template_var_insert(struct template_var_table *table, + const struct nbrec_chassis_template_var *nbrec_tv) +{ + struct template_var *tv = xmalloc(sizeof *tv); + tv->nb = nbrec_tv; + hmap_insert(&table->vars, &tv->hmap_node, + template_var_hash(nbrec_tv->chassis)); +} + +struct template_var * +template_var_find(struct template_var_table *table, const char *chassis_name) +{ + struct template_var *tv; + + HMAP_FOR_EACH_WITH_HASH (tv, hmap_node, template_var_hash(chassis_name), + &table->vars) { + if (!strcmp(chassis_name, tv->nb->chassis)) { + return tv; + } + } + return NULL; +} + +void +template_var_remove(struct template_var_table *table, struct template_var *tv) +{ + hmap_remove(&table->vars, &tv->hmap_node); +} + +void +template_var_destroy(struct template_var *tv) +{ + free(tv); +} diff --git a/northd/template-var.h b/northd/template-var.h new file mode 100644 index 0000000000..2de0e6cfe4 --- /dev/null +++ b/northd/template-var.h @@ -0,0 +1,58 @@ +/* Copyright (c) 2022, Red Hat, Inc. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at: + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#ifndef OVN_NORTHD_TEMPLATE_VAR_H +#define OVN_NORTHD_TEMPLATE_VAR_H 1 + +#include "openvswitch/hmap.h" +#include "lib/ovn-nb-idl.h" + +struct template_var { + struct hmap_node hmap_node; + + const struct nbrec_chassis_template_var *nb; +}; + +struct template_var_table { + struct hmap vars; +}; + +#define TEMPLATE_VAR_TABLE_INITIALIZER(TBL) \ + { .vars = HMAP_INITIALIZER(&(TBL)->vars) } + +#define TEMPLATE_VAR_TABLE_FOR_EACH(NODE, TBL) \ + HMAP_FOR_EACH (NODE, hmap_node, &(TBL)->vars) + +struct template_var_table *template_var_table_create(void); +void template_var_table_destroy(struct template_var_table *); + +static inline uint32_t +template_var_hash(const char *tv_chassis) +{ + return hash_string(tv_chassis, 0); +} + +void template_var_insert(struct template_var_table *, + const struct nbrec_chassis_template_var *); + +struct template_var * +template_var_find(struct template_var_table *, const char *chassis_name); + +void template_var_remove(struct template_var_table *, + struct template_var *); + +void template_var_destroy(struct template_var *); + +#endif /* OVN_NORTHD_TEMPLATE_VAR_H 1 */ diff --git a/ovn-nb.ovsschema b/ovn-nb.ovsschema index 174364c8b1..6f9d38f47b 100644 --- a/ovn-nb.ovsschema +++ b/ovn-nb.ovsschema @@ -1,7 +1,7 @@ { "name": "OVN_Northbound", - "version": "6.3.0", - "cksum": "4042813038 31869", + "version": "6.4.0", + "cksum": "3512158873 32360", "tables": { "NB_Global": { "columns": { @@ -620,6 +620,17 @@ "mac": {"type": "string"}, "override_dynamic_mac": {"type": "boolean"}}, "indexes": [["logical_port", "ip"]], - "isRoot": true} + "isRoot": true}, + "Chassis_Template_Var": { + "columns": { + "chassis": {"type": "string"}, + "variables": { + "type": {"key": "string", "value": "string", + "min": 0, "max": "unlimited"}}, + "external_ids": { + "type": {"key": "string", "value": "string", + "min": 0, "max": "unlimited"}}}, + "indexes": [["chassis"]], + "isRoot": true} } } diff --git a/ovn-nb.xml b/ovn-nb.xml index f41e9d7c0e..45b75e66df 100644 --- a/ovn-nb.xml +++ b/ovn-nb.xml @@ -4436,4 +4436,33 @@ + + +

+ One record per chassis, each containing a map, variables, + between template variable names and their value for that specific + chassis. A template variable has a name and potentially different + values on different hypervisors in the OVN cluster. For example, + two rows, R1 = (.chassis=C1, variables={(N: V1)} and + R2 = (.chassis=C2, variables={(N: V2)} will make + ovn-controller running on chassis C1 and + C2 interpret the token N either as + V1 (on C1) or as V2 (on + C2). Users can refer to template variables from + within other logical components, e.g., within ACL, QoS or + Logical_Router_Policy matches or from Load_Balancer VIP and + backend definitions. +

+ + The chassis this set of variable values applies to. + + + The set of variable values for a given chassis. + + + + See External IDs at the beginning of this document. + + +
diff --git a/ovn-sb.ovsschema b/ovn-sb.ovsschema index 576ebbdeb0..95c7c2d7e3 100644 --- a/ovn-sb.ovsschema +++ b/ovn-sb.ovsschema @@ -1,7 +1,7 @@ { "name": "OVN_Southbound", - "version": "20.25.0", - "cksum": "53184112 28845", + "version": "20.26.0", + "cksum": "3311869408 29176", "tables": { "SB_Global": { "columns": { @@ -565,6 +565,14 @@ "key": {"type": "uuid", "refTable": "Datapath_Binding"}}}}, "indexes": [["logical_port", "ip"]], + "isRoot": true}, + "Chassis_Template_Var": { + "columns": { + "chassis": {"type": "string"}, + "variables": { + "type": {"key": "string", "value": "string", + "min": 0, "max": "unlimited"}}}, + "indexes": [["chassis"]], "isRoot": true} } } diff --git a/ovn-sb.xml b/ovn-sb.xml index 315d608534..169a53aaa9 100644 --- a/ovn-sb.xml +++ b/ovn-sb.xml @@ -4801,4 +4801,19 @@ tcp.flags = RST; The logical datapath to which the logical router port belongs. + + +

+ Each record represents the set of template variable instantiations + for a given chassis and is populated by ovn-northd + from the contents of the OVN_Northbound.Chassis_Template_Var + table. +

+ + The chassis this set of variable values applies to. + + + The set of variable values for a given chassis. + +
diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at index 4f399eccb6..c7112b805d 100644 --- a/tests/ovn-northd.at +++ b/tests/ovn-northd.at @@ -7929,3 +7929,38 @@ AT_CHECK([grep "lr_in_arp_resolve" R1flows | grep priority=90 | sort], [0], [dnl AT_CLEANUP ]) + +OVN_FOR_EACH_NORTHD_NO_HV([ +AT_SETUP([NB to SB Chassis_Template_Var propagation]) +AT_KEYWORDS([templates]) +ovn_start + +AT_CHECK([ovn-nbctl create Chassis_Template_Var chassis="hv1"], [0], [ignore]) +AT_CHECK([ovn-nbctl create Chassis_Template_Var chassis="hv2"], [0], [ignore]) + +check ovn-nbctl set Chassis_Template_Var hv1 variables:tv=v1 +check ovn-nbctl set Chassis_Template_Var hv2 variables:tv=v2 + +AS_BOX([Ensure values are propagated to SB]) +check ovn-nbctl --wait=sb sync +check_column "tv=v1" sb:Chassis_Template_Var variables chassis="hv1" +check_column "tv=v2" sb:Chassis_Template_Var variables chassis="hv2" + +AS_BOX([Ensure SB is reconciled]) +check ovn-sbctl --all destroy Chassis_Template_Var +check ovn-nbctl --wait=sb sync +check_column "tv=v1" sb:Chassis_Template_Var variables chassis="hv1" +check_column "tv=v2" sb:Chassis_Template_Var variables chassis="hv2" + +AS_BOX([Ensure SB is reconciled - deletion]) +check ovn-nbctl destroy Chassis_Template_Var hv1 +check ovn-nbctl --wait=sb sync +check_column "tv=v2" sb:Chassis_Template_Var variables chassis="hv2" + +AS_BOX([Ensure SB is reconciled - cleanup]) +check ovn-nbctl destroy Chassis_Template_Var hv2 +check ovn-nbctl --wait=sb sync +check_row_count sb:Chassis_Template_Var 0 + +AT_CLEANUP +]) diff --git a/utilities/ovn-nbctl.c b/utilities/ovn-nbctl.c index 811468dc66..d2dee6b31c 100644 --- a/utilities/ovn-nbctl.c +++ b/utilities/ovn-nbctl.c @@ -7300,6 +7300,9 @@ static const struct ctl_table_class tables[NBREC_N_TABLES] = { [NBREC_TABLE_NAT].row_ids[0] = {&nbrec_nat_col_external_ip, NULL, NULL}, + [NBREC_TABLE_CHASSIS_TEMPLATE_VAR].row_ids[0] + = {&nbrec_chassis_template_var_col_chassis, NULL, NULL}, + [NBREC_TABLE_CONNECTION].row_ids[0] = {&nbrec_connection_col_target, NULL, NULL}, }; diff --git a/utilities/ovn-sbctl.c b/utilities/ovn-sbctl.c index f60dde1b67..00b2f785a5 100644 --- a/utilities/ovn-sbctl.c +++ b/utilities/ovn-sbctl.c @@ -1457,6 +1457,9 @@ static const struct ctl_table_class tables[SBREC_N_TABLES] = { [SBREC_TABLE_LOAD_BALANCER].row_ids[0] = {&sbrec_load_balancer_col_name, NULL, NULL}, + + [SBREC_TABLE_CHASSIS_TEMPLATE_VAR].row_ids[0] + = {&sbrec_chassis_template_var_col_chassis, NULL, NULL}, }; static const struct ctl_command_syntax sbctl_commands[] = { From patchwork Fri Nov 4 22:11:33 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dumitru Ceara X-Patchwork-Id: 1699895 X-Patchwork-Delegate: mmichels@redhat.com Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::137; helo=smtp4.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=K6a6ip0N; dkim-atps=neutral Received: from smtp4.osuosl.org (smtp4.osuosl.org [IPv6:2605:bc80:3010::137]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4N3vxC1h76z23lT for ; Sat, 5 Nov 2022 09:11:59 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 82EBF41A01; Fri, 4 Nov 2022 22:11:56 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 82EBF41A01 Authentication-Results: smtp4.osuosl.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=K6a6ip0N X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id wTuw0C-fs7Zt; Fri, 4 Nov 2022 22:11:52 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp4.osuosl.org (Postfix) with ESMTPS id C009541A0E; Fri, 4 Nov 2022 22:11:51 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org C009541A0E Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 96466C0032; Fri, 4 Nov 2022 22:11:51 +0000 (UTC) X-Original-To: ovs-dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by lists.linuxfoundation.org (Postfix) with ESMTP id 4AD2EC002D for ; Fri, 4 Nov 2022 22:11:50 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id E0CF38220F for ; Fri, 4 Nov 2022 22:11:41 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org E0CF38220F Authentication-Results: smtp1.osuosl.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=K6a6ip0N X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id L7YGWHb-miR2 for ; Fri, 4 Nov 2022 22:11:39 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 504C9821FD Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by smtp1.osuosl.org (Postfix) with ESMTPS id 504C9821FD for ; Fri, 4 Nov 2022 22:11:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1667599898; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=4zBQ75Cmn4xbovZWRZKnYDuRUu+RkR1NJWt0/ZKIJSA=; b=K6a6ip0N+DWkF98AZjo+uPdifh2FgS2EzU5TXp+gQTfkUnyEdoZ7Hg9b+LimFWxic9heeQ +UMsPN+pp2RDID0WC4z2heq7/uYpwxahli1F1CP5m4w6c81HXoC8jq5TAGxoVIhWfylDjA SrHat/2DvkN/r48t9NKW9ldDnuw61iw= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-649-grzHndD3Pwy1q5su3ueRHQ-1; Fri, 04 Nov 2022 18:11:37 -0400 X-MC-Unique: grzHndD3Pwy1q5su3ueRHQ-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id E4494380450C; Fri, 4 Nov 2022 22:11:36 +0000 (UTC) Received: from dceara.remote.csb (unknown [10.39.192.36]) by smtp.corp.redhat.com (Postfix) with ESMTP id 8770840C6DC7; Fri, 4 Nov 2022 22:11:35 +0000 (UTC) From: Dumitru Ceara To: ovs-dev@openvswitch.org Date: Fri, 4 Nov 2022 23:11:33 +0100 Message-Id: <166759989186.515460.7366980327085014395.stgit@dceara.remote.csb> In-Reply-To: <166759985955.515460.16275468131013093936.stgit@dceara.remote.csb> References: <166759985955.515460.16275468131013093936.stgit@dceara.remote.csb> User-Agent: StGit/0.23 MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.2 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Cc: surya@redhat.com, i.maximets@ovn.org Subject: [ovs-dev] [PATCH ovn v2 3/5] controller: Add support for templated actions and matches. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" Expand SB.Template_Var records in two stages: 1. first expand them to local values in match/action strings 2. then reparse the expanded strings For the case when a lflow references a Chassis_Template_Var also track references (similar to the ones maintained for multicast groups, address sets, port_groups, port bindings). Signed-off-by: Dumitru Ceara --- V2: - Fix GCC build due to missing newline. - Handle SB table rename Template_Var -> Chassis_Template_Var. - Address Han's comments: - Add new function to parse lflow actions. - Move xstrdup inside lexer_parse_template_string() and execute it only if needed. - Match template vars only by chassis name. - Change local_templates table to a plain smap. - Some indentation updates. - Use NULL template variable change handlers for chassis/Open_vSwitch changes. - Added more tests. - Fix tracking of template references in lflow actions. --- controller/lflow.c | 136 ++++++++++++++------ controller/lflow.h | 1 controller/ofctrl.c | 11 +- controller/ofctrl.h | 3 controller/ovn-controller.c | 289 +++++++++++++++++++++++++++++++++++++++++++ include/ovn/expr.h | 4 - include/ovn/lex.h | 15 ++ lib/actions.c | 9 + lib/expr.c | 18 ++- lib/lex.c | 57 ++++++++ lib/objdep.c | 1 lib/objdep.h | 1 tests/ovn-controller.at | 50 +++++++ tests/ovn.at | 80 ++++++++++++ tests/test-ovn.c | 17 ++- utilities/ovn-trace.c | 33 ++++- 16 files changed, 658 insertions(+), 67 deletions(-) diff --git a/controller/lflow.c b/controller/lflow.c index d4434bdee8..fc4371d0df 100644 --- a/controller/lflow.c +++ b/controller/lflow.c @@ -81,6 +81,8 @@ convert_match_to_expr(const struct sbrec_logical_flow *, const struct local_datapath *ldp, struct expr **prereqs, const struct shash *addr_sets, const struct shash *port_groups, + const struct smap *template_vars, + struct sset *template_vars_ref, struct objdep_mgr *, bool *pg_addr_set_ref); static void add_matches_to_flow_table(const struct sbrec_logical_flow *, @@ -297,6 +299,43 @@ as_info_from_expr_const(const char *as_name, const union expr_constant *c, return true; } +static bool +lflow_parse_actions(const struct sbrec_logical_flow *lflow, + const struct lflow_ctx_in *l_ctx_in, + struct sset *template_vars_ref, + struct ofpbuf *ovnacts_out, + struct expr **prereqs_out) +{ + bool ingress = !strcmp(lflow->pipeline, "ingress"); + struct ovnact_parse_params pp = { + .symtab = &symtab, + .dhcp_opts = l_ctx_in->dhcp_opts, + .dhcpv6_opts = l_ctx_in->dhcpv6_opts, + .nd_ra_opts = l_ctx_in->nd_ra_opts, + .controller_event_opts = l_ctx_in->controller_event_opts, + + .pipeline = ingress ? OVNACT_P_INGRESS : OVNACT_P_EGRESS, + .n_tables = LOG_PIPELINE_LEN, + .cur_ltable = lflow->table_id, + }; + + char *actions_expanded_s = NULL; + const char *actions_s = + lexer_parse_template_string(lflow->actions, l_ctx_in->template_vars, + template_vars_ref, &actions_expanded_s); + char *error = ovnacts_parse_string(actions_s, &pp, + ovnacts_out, prereqs_out); + free(actions_expanded_s); + if (error) { + static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 1); + VLOG_WARN_RL(&rl, "error parsing actions \"%s\": %s", + lflow->actions, error); + free(error); + return false; + } + return true; +} + /* Parses the lflow regarding the changed address set 'as_name', and generates * ovs flows for the newly added addresses in 'as_diff_added' only. It is * similar to consider_logical_flow__, with the below differences: @@ -347,27 +386,14 @@ consider_lflow_for_added_as_ips__( uint64_t ovnacts_stub[1024 / 8]; struct ofpbuf ovnacts = OFPBUF_STUB_INITIALIZER(ovnacts_stub); - struct ovnact_parse_params pp = { - .symtab = &symtab, - .dhcp_opts = l_ctx_in->dhcp_opts, - .dhcpv6_opts = l_ctx_in->dhcpv6_opts, - .nd_ra_opts = l_ctx_in->nd_ra_opts, - .controller_event_opts = l_ctx_in->controller_event_opts, - .pipeline = ingress ? OVNACT_P_INGRESS : OVNACT_P_EGRESS, - .n_tables = LOG_PIPELINE_LEN, - .cur_ltable = lflow->table_id, - }; + struct sset template_vars_ref = SSET_INITIALIZER(&template_vars_ref); struct expr *prereqs = NULL; - char *error; - error = ovnacts_parse_string(lflow->actions, &pp, &ovnacts, &prereqs); - if (error) { - static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 1); - VLOG_WARN_RL(&rl, "error parsing actions \"%s\": %s", - lflow->actions, error); - free(error); + if (!lflow_parse_actions(lflow, l_ctx_in, &template_vars_ref, + &ovnacts, &prereqs)) { ovnacts_free(ovnacts.data, ovnacts.size); ofpbuf_uninit(&ovnacts); + sset_destroy(&template_vars_ref); return true; } @@ -431,6 +457,8 @@ consider_lflow_for_added_as_ips__( struct expr *expr = convert_match_to_expr(lflow, ldp, &prereqs, l_ctx_in->addr_sets, l_ctx_in->port_groups, + l_ctx_in->template_vars, + &template_vars_ref, l_ctx_out->lflow_deps_mgr, NULL); shash_replace((struct shash *)l_ctx_in->addr_sets, as_name, real_as); if (new_fake_as) { @@ -502,6 +530,14 @@ done: ofpbuf_uninit(&ovnacts); expr_destroy(expr); expr_matches_destroy(&matches); + + const char *tv_name; + SSET_FOR_EACH (tv_name, &template_vars_ref) { + objdep_mgr_add(l_ctx_out->lflow_deps_mgr, OBJDEP_TYPE_TEMPLATE, + tv_name, &lflow->header_.uuid); + } + sset_destroy(&template_vars_ref); + return handled; } @@ -913,6 +949,8 @@ convert_match_to_expr(const struct sbrec_logical_flow *lflow, struct expr **prereqs, const struct shash *addr_sets, const struct shash *port_groups, + const struct smap *template_vars, + struct sset *template_vars_ref, struct objdep_mgr *mgr, bool *pg_addr_set_ref) { @@ -920,11 +958,18 @@ convert_match_to_expr(const struct sbrec_logical_flow *lflow, struct sset port_groups_ref = SSET_INITIALIZER(&port_groups_ref); char *error = NULL; - struct expr *e = expr_parse_string(lflow->match, &symtab, addr_sets, + char *match_expanded_s = NULL; + const char *match_s = lexer_parse_template_string(lflow->match, + template_vars, + template_vars_ref, + &match_expanded_s); + struct expr *e = expr_parse_string(match_s, &symtab, addr_sets, port_groups, &addr_sets_ref, &port_groups_ref, ldp->datapath->tunnel_key, &error); + free(match_expanded_s); + struct shash_node *addr_sets_ref_node; SHASH_FOR_EACH (addr_sets_ref_node, &addr_sets_ref) { objdep_mgr_add_with_refcount(mgr, OBJDEP_TYPE_ADDRSET, @@ -963,6 +1008,18 @@ convert_match_to_expr(const struct sbrec_logical_flow *lflow, return expr_simplify(e); } +static void +store_lflow_template_refs(struct objdep_mgr *lflow_deps_mgr, + const struct sset *template_vars_ref, + const struct sbrec_logical_flow *lflow) +{ + const char *tv_name; + SSET_FOR_EACH (tv_name, template_vars_ref) { + objdep_mgr_add(lflow_deps_mgr, OBJDEP_TYPE_TEMPLATE, tv_name, + &lflow->header_.uuid); + } +} + static void consider_logical_flow__(const struct sbrec_logical_flow *lflow, const struct sbrec_datapath_binding *dp, @@ -1015,28 +1072,16 @@ consider_logical_flow__(const struct sbrec_logical_flow *lflow, * XXX Deny changes to 'outport' in egress pipeline. */ uint64_t ovnacts_stub[1024 / 8]; struct ofpbuf ovnacts = OFPBUF_STUB_INITIALIZER(ovnacts_stub); - struct ovnact_parse_params pp = { - .symtab = &symtab, - .dhcp_opts = l_ctx_in->dhcp_opts, - .dhcpv6_opts = l_ctx_in->dhcpv6_opts, - .nd_ra_opts = l_ctx_in->nd_ra_opts, - .controller_event_opts = l_ctx_in->controller_event_opts, - - .pipeline = ingress ? OVNACT_P_INGRESS : OVNACT_P_EGRESS, - .n_tables = LOG_PIPELINE_LEN, - .cur_ltable = lflow->table_id, - }; + struct sset template_vars_ref = SSET_INITIALIZER(&template_vars_ref); struct expr *prereqs = NULL; - char *error; - error = ovnacts_parse_string(lflow->actions, &pp, &ovnacts, &prereqs); - if (error) { - static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 1); - VLOG_WARN_RL(&rl, "error parsing actions \"%s\": %s", - lflow->actions, error); - free(error); + if (!lflow_parse_actions(lflow, l_ctx_in, &template_vars_ref, + &ovnacts, &prereqs)) { ovnacts_free(ovnacts.data, ovnacts.size); ofpbuf_uninit(&ovnacts); + store_lflow_template_refs(l_ctx_out->lflow_deps_mgr, + &template_vars_ref, lflow); + sset_destroy(&template_vars_ref); return; } @@ -1087,6 +1132,8 @@ consider_logical_flow__(const struct sbrec_logical_flow *lflow, case LCACHE_T_NONE: expr = convert_match_to_expr(lflow, ldp, &prereqs, l_ctx_in->addr_sets, l_ctx_in->port_groups, + l_ctx_in->template_vars, + &template_vars_ref, l_ctx_out->lflow_deps_mgr, &pg_addr_set_ref); if (!expr) { @@ -1101,11 +1148,13 @@ consider_logical_flow__(const struct sbrec_logical_flow *lflow, } /* If caching is enabled and this is a not cached expr that doesn't refer - * to address sets or port groups, save it to potentially cache it later. + * to address sets, port groups, or template variables, save it to + * potentially cache it later. */ if (lcv_type == LCACHE_T_NONE && lflow_cache_is_enabled(l_ctx_out->lflow_cache) - && !pg_addr_set_ref) { + && !pg_addr_set_ref + && sset_is_empty(&template_vars_ref)) { cached_expr = expr_clone(expr); } @@ -1190,6 +1239,10 @@ done: expr_destroy(cached_expr); expr_matches_destroy(matches); free(matches); + + store_lflow_template_refs(l_ctx_out->lflow_deps_mgr, + &template_vars_ref, lflow); + sset_destroy(&template_vars_ref); } static void @@ -1953,8 +2006,7 @@ add_lb_ct_snat_hairpin_flows(struct ovn_controller_lb *lb, static void consider_lb_hairpin_flows(const struct sbrec_load_balancer *sbrec_lb, - const struct hmap *local_datapaths, - bool use_ct_mark, + const struct hmap *local_datapaths, bool use_ct_mark, struct ovn_desired_flow_table *flow_table, struct simap *ids) { @@ -2043,8 +2095,8 @@ add_lb_hairpin_flows(const struct sbrec_load_balancer_table *lb_table, ovs_assert(id_pool_alloc_id(pool, &id)); simap_put(ids, lb->name, id); } - consider_lb_hairpin_flows(lb, local_datapaths, use_ct_mark, - flow_table, ids); + consider_lb_hairpin_flows(lb, local_datapaths, use_ct_mark, flow_table, + ids); } } diff --git a/controller/lflow.h b/controller/lflow.h index 3f369ebfb6..9a7079f99e 100644 --- a/controller/lflow.h +++ b/controller/lflow.h @@ -112,6 +112,7 @@ struct lflow_ctx_in { const struct hmap *dhcp_opts; const struct hmap *dhcpv6_opts; const struct controller_event_options *controller_event_opts; + const struct smap *template_vars; bool lb_hairpin_use_ct_mark; }; diff --git a/controller/ofctrl.c b/controller/ofctrl.c index c779912583..f92e2017ce 100644 --- a/controller/ofctrl.c +++ b/controller/ofctrl.c @@ -2873,7 +2873,8 @@ ofctrl_lookup_port(const void *br_int_, const char *port_name, char * ofctrl_inject_pkt(const struct ovsrec_bridge *br_int, const char *flow_s, const struct shash *addr_sets, - const struct shash *port_groups) + const struct shash *port_groups, + const struct smap *template_vars) { int version = rconn_get_version(swconn); if (version < 0) { @@ -2881,9 +2882,15 @@ ofctrl_inject_pkt(const struct ovsrec_bridge *br_int, const char *flow_s, } struct flow uflow; - char *error = expr_parse_microflow(flow_s, &symtab, addr_sets, + char *flow_expanded_s = NULL; + const char *flow_exp_s = lexer_parse_template_string(flow_s, + template_vars, + NULL, + &flow_expanded_s); + char *error = expr_parse_microflow(flow_exp_s, &symtab, addr_sets, port_groups, ofctrl_lookup_port, br_int, &uflow); + free(flow_expanded_s); if (error) { return error; } diff --git a/controller/ofctrl.h b/controller/ofctrl.h index 71d3f5838a..3968245126 100644 --- a/controller/ofctrl.h +++ b/controller/ofctrl.h @@ -71,7 +71,8 @@ void ofctrl_ct_flush_zone(uint16_t zone_id); char *ofctrl_inject_pkt(const struct ovsrec_bridge *br_int, const char *flow_s, const struct shash *addr_sets, - const struct shash *port_groups); + const struct shash *port_groups, + const struct smap *template_vars); /* Flow table interfaces to the rest of ovn-controller. */ diff --git a/controller/ovn-controller.c b/controller/ovn-controller.c index f62303e3cb..c374bd0f33 100644 --- a/controller/ovn-controller.c +++ b/controller/ovn-controller.c @@ -91,6 +91,7 @@ static unixctl_cb_func debug_pause_execution; static unixctl_cb_func debug_resume_execution; static unixctl_cb_func debug_status_execution; static unixctl_cb_func debug_dump_local_bindings; +static unixctl_cb_func debug_dump_local_template_vars; static unixctl_cb_func debug_dump_lflow_conj_ids; static unixctl_cb_func lflow_cache_flush_cmd; static unixctl_cb_func lflow_cache_show_stats_cmd; @@ -170,6 +171,8 @@ update_sb_monitors(struct ovsdb_idl *ovnsb_idl, * * Monitor IGMP_Groups for local chassis. * + * Monitor Template_Var for local chassis. + * * We always monitor patch ports because they allow us to see the linkages * between related logical datapaths. That way, when we know that we have * a VIF on a particular logical switch, we immediately know to monitor all @@ -184,6 +187,7 @@ update_sb_monitors(struct ovsdb_idl *ovnsb_idl, struct ovsdb_idl_condition ip_mcast = OVSDB_IDL_CONDITION_INIT(&ip_mcast); struct ovsdb_idl_condition igmp = OVSDB_IDL_CONDITION_INIT(&igmp); struct ovsdb_idl_condition chprv = OVSDB_IDL_CONDITION_INIT(&chprv); + struct ovsdb_idl_condition tv = OVSDB_IDL_CONDITION_INIT(&tv); /* Always monitor all logical datapath groups. Otherwise, DPG updates may * be received *after* the lflows using it are seen by ovn-controller. @@ -201,6 +205,7 @@ update_sb_monitors(struct ovsdb_idl *ovnsb_idl, ovsdb_idl_condition_add_clause_true(&ip_mcast); ovsdb_idl_condition_add_clause_true(&igmp); ovsdb_idl_condition_add_clause_true(&chprv); + ovsdb_idl_condition_add_clause_true(&tv); goto out; } @@ -239,6 +244,9 @@ update_sb_monitors(struct ovsdb_idl *ovnsb_idl, /* Monitors Chassis_Private record for current chassis only. */ sbrec_chassis_private_add_clause_name(&chprv, OVSDB_F_EQ, chassis->name); + + sbrec_chassis_template_var_add_clause_chassis(&tv, OVSDB_F_EQ, + chassis->name); } else { /* During initialization, we monitor all records in Chassis_Private so * that we don't try to recreate existing ones. */ @@ -290,6 +298,7 @@ out:; sbrec_ip_multicast_set_condition(ovnsb_idl, &ip_mcast), sbrec_igmp_group_set_condition(ovnsb_idl, &igmp), sbrec_chassis_private_set_condition(ovnsb_idl, &chprv), + sbrec_chassis_template_var_set_condition(ovnsb_idl, &tv), }; unsigned int expected_cond_seqno = 0; @@ -307,6 +316,7 @@ out:; ovsdb_idl_condition_destroy(&ip_mcast); ovsdb_idl_condition_destroy(&igmp); ovsdb_idl_condition_destroy(&chprv); + ovsdb_idl_condition_destroy(&tv); return expected_cond_seqno; } @@ -986,7 +996,8 @@ ctrl_register_ovs_idl(struct ovsdb_idl *ovs_idl) SB_NODE(load_balancer, "load_balancer") \ SB_NODE(fdb, "fdb") \ SB_NODE(meter, "meter") \ - SB_NODE(static_mac_binding, "static_mac_binding") + SB_NODE(static_mac_binding, "static_mac_binding") \ + SB_NODE(chassis_template_var, "chassis_template_var") enum sb_engine_node { #define SB_NODE(NAME, NAME_STR) SB_##NAME, @@ -1648,6 +1659,184 @@ runtime_data_sb_datapath_binding_handler(struct engine_node *node OVS_UNUSED, return true; } +static void +local_templates_to_string(const struct smap *local_templates, + struct ds *out_data) +{ + const struct smap_node **nodes = smap_sort(local_templates); + + for (size_t i = 0; i < smap_count(local_templates); i++) { + const struct smap_node *node = nodes[i]; + + ds_put_format(out_data, "name: '%s' value: '%s'\n", + node->key, node->value); + } + free(nodes); +} + +struct ed_type_template_vars { + struct smap local_templates; + + bool change_tracked; + struct sset new; + struct sset deleted; + struct sset updated; +}; + +static void +template_vars_init(struct ovsdb_idl_index *tv_index_by_chassis, + const struct sbrec_chassis *chassis, + struct smap *local_templates) +{ + const struct sbrec_chassis_template_var *tv; + struct sbrec_chassis_template_var *tv_key = + sbrec_chassis_template_var_index_init_row(tv_index_by_chassis); + sbrec_chassis_template_var_index_set_chassis(tv_key, chassis->name); + + tv = sbrec_chassis_template_var_index_find(tv_index_by_chassis, tv_key); + if (tv) { + smap_destroy(local_templates); + smap_clone(local_templates, &tv->variables); + } + + sbrec_chassis_template_var_index_destroy_row(tv_key); +} + +static void +template_vars_update(const struct sbrec_chassis_template_var_table *tv_table, + const struct sbrec_chassis *chassis, + struct smap *local_templates, struct sset *new, + struct sset *deleted, struct sset *updated) +{ + const struct sbrec_chassis_template_var *tv; + struct smap_node *node; + SBREC_CHASSIS_TEMPLATE_VAR_TABLE_FOR_EACH_TRACKED (tv, tv_table) { + if (strcmp(tv->chassis, chassis->name)) { + continue; + } + + if (sbrec_chassis_template_var_is_deleted(tv)) { + SMAP_FOR_EACH (node, local_templates) { + sset_add(deleted, node->key); + } + } else if (sbrec_chassis_template_var_is_new(tv)) { + ovs_assert(smap_count(local_templates) == 0); + SMAP_FOR_EACH (node, &tv->variables) { + sset_add(new, node->key); + } + } else { + SMAP_FOR_EACH (node, &tv->variables) { + struct smap_node *old = smap_get_node(local_templates, + node->key); + if (old) { + if (strcmp(old->value, node->value)) { + sset_add(updated, node->key); + } + } else { + sset_add(new, node->key); + } + } + SMAP_FOR_EACH (node, local_templates) { + sset_add (deleted, node->key); + } + } + + smap_destroy(local_templates); + smap_clone(local_templates, &tv->variables); + return; + } +} + +static void * +en_template_vars_init(struct engine_node *node OVS_UNUSED, + struct engine_arg *arg OVS_UNUSED) +{ + struct ed_type_template_vars *tv_data = xzalloc(sizeof *tv_data); + smap_init(&tv_data->local_templates); + tv_data->change_tracked = false; + sset_init(&tv_data->new); + sset_init(&tv_data->deleted); + sset_init(&tv_data->updated); + return tv_data; +} + +static void +en_template_vars_run(struct engine_node *node, void *data) +{ + struct ed_type_template_vars *tv_data = data; + + const struct ovsrec_open_vswitch_table *ovs_table = + EN_OVSDB_GET(engine_get_input("OVS_open_vswitch", node)); + struct ovsdb_idl_index *sbrec_chassis_by_name = + engine_ovsdb_node_get_index(engine_get_input("SB_chassis", node), + "name"); + const struct sbrec_chassis *chassis = + chassis_lookup_by_name(sbrec_chassis_by_name, + get_ovs_chassis_id(ovs_table)); + struct ovsdb_idl_index *sbrec_chassis_template_var_index_by_chassis = + engine_ovsdb_node_get_index( + engine_get_input("SB_chassis_template_var", node), + "chassis"); + + smap_clear(&tv_data->local_templates); + template_vars_init(sbrec_chassis_template_var_index_by_chassis, + chassis, &tv_data->local_templates); + engine_set_node_state(node, EN_UPDATED); +} + +static bool +template_vars_sb_chassis_template_var_handler(struct engine_node *node, + void *data) +{ + struct ed_type_template_vars *tv_data = data; + + const struct sbrec_chassis_template_var_table *tv_table = + EN_OVSDB_GET(engine_get_input("SB_chassis_template_var", node)); + const struct ovsrec_open_vswitch_table *ovs_table = + EN_OVSDB_GET(engine_get_input("OVS_open_vswitch", node)); + struct ovsdb_idl_index *sbrec_chassis_by_name = + engine_ovsdb_node_get_index(engine_get_input("SB_chassis", node), + "name"); + const struct sbrec_chassis *chassis = + chassis_lookup_by_name(sbrec_chassis_by_name, + get_ovs_chassis_id(ovs_table)); + + template_vars_update(tv_table, chassis, &tv_data->local_templates, + &tv_data->new, &tv_data->deleted, &tv_data->updated); + + if (!sset_is_empty(&tv_data->new) || !sset_is_empty(&tv_data->deleted) || + !sset_is_empty(&tv_data->updated)) { + engine_set_node_state(node, EN_UPDATED); + } else { + engine_set_node_state(node, EN_UNCHANGED); + } + + tv_data->change_tracked = true; + return true; +} + +static void +en_template_vars_clear_tracked_data(void *data) +{ + struct ed_type_template_vars *tv_data = data; + + sset_clear(&tv_data->new); + sset_clear(&tv_data->deleted); + sset_clear(&tv_data->updated); + tv_data->change_tracked = false; +} + +static void +en_template_vars_cleanup(void *data) +{ + struct ed_type_template_vars *tv_data = data; + + smap_destroy(&tv_data->local_templates); + sset_destroy(&tv_data->new); + sset_destroy(&tv_data->deleted); + sset_destroy(&tv_data->updated); +} + struct ed_type_addr_sets { struct shash addr_sets; bool change_tracked; @@ -2702,6 +2891,9 @@ init_lflow_ctx(struct engine_node *node, struct ed_type_dhcp_options *dhcp_opts = engine_get_input_data("dhcp_options", node); + struct ed_type_template_vars *template_vars = + engine_get_input_data("template_vars", node); + l_ctx_in->sbrec_multicast_group_by_name_datapath = sbrec_mc_group_by_name_dp; l_ctx_in->sbrec_logical_flow_by_logical_datapath = @@ -2734,6 +2926,7 @@ init_lflow_ctx(struct engine_node *node, l_ctx_in->dhcp_opts = &dhcp_opts->v4_opts; l_ctx_in->dhcpv6_opts = &dhcp_opts->v6_opts; l_ctx_in->controller_event_opts = &fo->controller_event_opts; + l_ctx_in->template_vars = &template_vars->local_templates; l_ctx_out->flow_table = &fo->flow_table; l_ctx_out->group_table = &fo->group_table; @@ -3060,6 +3253,64 @@ lflow_output_port_groups_handler(struct engine_node *node, void *data) return true; } +static bool +lflow_output_template_vars_handler(struct engine_node *node, void *data) +{ + struct ed_type_template_vars *tv_data = + engine_get_input_data("template_vars", node); + + struct ed_type_lflow_output *fo = data; + struct lflow_ctx_out l_ctx_out; + struct lflow_ctx_in l_ctx_in; + init_lflow_ctx(node, fo, &l_ctx_in, &l_ctx_out); + + const char *res_name; + bool changed; + + if (!tv_data->change_tracked) { + return false; + } + + SSET_FOR_EACH (res_name, &tv_data->deleted) { + if (!objdep_mgr_handle_change(l_ctx_out.lflow_deps_mgr, + OBJDEP_TYPE_TEMPLATE, + res_name, lflow_handle_changed_ref, + l_ctx_out.lflows_processed, + &l_ctx_in, &l_ctx_out, &changed)) { + return false; + } + if (changed) { + engine_set_node_state(node, EN_UPDATED); + } + } + SSET_FOR_EACH (res_name, &tv_data->updated) { + if (!objdep_mgr_handle_change(l_ctx_out.lflow_deps_mgr, + OBJDEP_TYPE_TEMPLATE, + res_name, lflow_handle_changed_ref, + l_ctx_out.lflows_processed, + &l_ctx_in, &l_ctx_out, &changed)) { + return false; + } + if (changed) { + engine_set_node_state(node, EN_UPDATED); + } + } + SSET_FOR_EACH (res_name, &tv_data->new) { + if (!objdep_mgr_handle_change(l_ctx_out.lflow_deps_mgr, + OBJDEP_TYPE_TEMPLATE, + res_name, lflow_handle_changed_ref, + l_ctx_out.lflows_processed, + &l_ctx_in, &l_ctx_out, &changed)) { + return false; + } + if (changed) { + engine_set_node_state(node, EN_UPDATED); + } + } + + return true; +} + static bool lflow_output_runtime_data_handler(struct engine_node *node, void *data OVS_UNUSED) @@ -3646,6 +3897,9 @@ main(int argc, char *argv[]) struct ovsdb_idl_index *sbrec_static_mac_binding_by_datapath = ovsdb_idl_index_create1(ovnsb_idl_loop.idl, &sbrec_static_mac_binding_col_datapath); + struct ovsdb_idl_index *sbrec_chassis_template_var_index_by_chassis + = ovsdb_idl_index_create1(ovnsb_idl_loop.idl, + &sbrec_chassis_template_var_col_chassis); ovsdb_idl_track_add_all(ovnsb_idl_loop.idl); ovsdb_idl_omit_alert(ovnsb_idl_loop.idl, @@ -3707,6 +3961,7 @@ main(int argc, char *argv[]) /* Define inc-proc-engine nodes. */ ENGINE_NODE(sb_ro, "sb_ro"); + ENGINE_NODE_WITH_CLEAR_TRACK_DATA(template_vars, "template_vars"); ENGINE_NODE_WITH_CLEAR_TRACK_DATA_IS_VALID(ct_zones, "ct_zones"); ENGINE_NODE_WITH_CLEAR_TRACK_DATA(ovs_interface_shadow, "ovs_interface_shadow"); @@ -3733,6 +3988,10 @@ main(int argc, char *argv[]) #undef OVS_NODE /* Add dependencies between inc-proc-engine nodes. */ + engine_add_input(&en_template_vars, &en_ovs_open_vswitch, NULL); + engine_add_input(&en_template_vars, &en_sb_chassis, NULL); + engine_add_input(&en_template_vars, &en_sb_chassis_template_var, + template_vars_sb_chassis_template_var_handler); engine_add_input(&en_addr_sets, &en_sb_address_set, addr_sets_sb_address_set_handler); @@ -3800,6 +4059,8 @@ main(int argc, char *argv[]) lflow_output_addr_sets_handler); engine_add_input(&en_lflow_output, &en_port_groups, lflow_output_port_groups_handler); + engine_add_input(&en_lflow_output, &en_template_vars, + lflow_output_template_vars_handler); engine_add_input(&en_lflow_output, &en_runtime_data, lflow_output_runtime_data_handler); engine_add_input(&en_lflow_output, &en_non_vif_data, @@ -3908,6 +4169,8 @@ main(int argc, char *argv[]) sbrec_mac_binding_by_datapath); engine_ovsdb_node_add_index(&en_sb_static_mac_binding, "datapath", sbrec_static_mac_binding_by_datapath); + engine_ovsdb_node_add_index(&en_sb_chassis_template_var, "chassis", + sbrec_chassis_template_var_index_by_chassis); struct ed_type_lflow_output *lflow_output_data = engine_get_internal_data(&en_lflow_output); @@ -3917,6 +4180,8 @@ main(int argc, char *argv[]) engine_get_internal_data(&en_ct_zones); struct ed_type_runtime_data *runtime_data = engine_get_internal_data(&en_runtime_data); + struct ed_type_template_vars *template_vars_data = + engine_get_internal_data(&en_template_vars); ofctrl_init(&lflow_output_data->group_table, &lflow_output_data->meter_table, @@ -3978,6 +4243,10 @@ main(int argc, char *argv[]) debug_dump_lflow_conj_ids, &lflow_output_data->conj_ids); + unixctl_command_register("debug/dump-local-template-vars", "", 0, 0, + debug_dump_local_template_vars, + &template_vars_data->local_templates); + unixctl_command_register("debug/ignore-startup-delay", "", 0, 0, debug_ignore_startup_delay, NULL); @@ -4354,9 +4623,12 @@ main(int argc, char *argv[]) engine_get_data(&en_addr_sets); struct ed_type_port_groups *pg_data = engine_get_data(&en_port_groups); - if (br_int && chassis && as_data && pg_data) { + struct ed_type_template_vars *tv_data = + engine_get_data(&en_template_vars); + if (br_int && chassis && as_data && pg_data && tv_data) { char *error = ofctrl_inject_pkt(br_int, pending_pkt.flow_s, - &as_data->addr_sets, &pg_data->port_groups_cs_local); + &as_data->addr_sets, &pg_data->port_groups_cs_local, + &tv_data->local_templates); if (error) { unixctl_command_reply_error(pending_pkt.conn, error); free(error); @@ -4825,6 +5097,17 @@ debug_dump_lflow_conj_ids(struct unixctl_conn *conn, int argc OVS_UNUSED, ds_destroy(&conj_ids_dump); } +static void +debug_dump_local_template_vars(struct unixctl_conn *conn, int argc OVS_UNUSED, + const char *argv[] OVS_UNUSED, void *local_vars) +{ + struct ds tv_str = DS_EMPTY_INITIALIZER; + ds_put_cstr(&tv_str, "Local template vars:\n"); + local_templates_to_string(local_vars, &tv_str); + unixctl_command_reply(conn, ds_cstr(&tv_str)); + ds_destroy(&tv_str); +} + static void debug_ignore_startup_delay(struct unixctl_conn *conn, int argc OVS_UNUSED, const char *argv[] OVS_UNUSED, void *arg OVS_UNUSED) diff --git a/include/ovn/expr.h b/include/ovn/expr.h index 3b141b034c..80d95ff675 100644 --- a/include/ovn/expr.h +++ b/include/ovn/expr.h @@ -59,6 +59,7 @@ #include "openvswitch/match.h" #include "openvswitch/meta-flow.h" #include "logical-fields.h" +#include "smap.h" struct ds; struct expr; @@ -521,7 +522,8 @@ union expr_constant { char *string; }; -bool expr_constant_parse(struct lexer *, const struct expr_field *, +bool expr_constant_parse(struct lexer *, + const struct expr_field *, union expr_constant *); void expr_constant_format(const union expr_constant *, enum expr_constant_type, struct ds *); diff --git a/include/ovn/lex.h b/include/ovn/lex.h index ecb7ace243..516bbc2bd8 100644 --- a/include/ovn/lex.h +++ b/include/ovn/lex.h @@ -23,6 +23,8 @@ * This is a simple lexical analyzer (or tokenizer) for OVN match expressions * and ACLs. */ +#include "smap.h" +#include "sset.h" #include "openvswitch/meta-flow.h" struct ds; @@ -37,7 +39,8 @@ enum lex_type { LEX_T_INTEGER, /* 12345 or 1.2.3.4 or ::1 or 01:02:03:04:05 */ LEX_T_MASKED_INTEGER, /* 12345/10 or 1.2.0.0/16 or ::2/127 or... */ LEX_T_MACRO, /* $NAME */ - LEX_T_PORT_GROUP, /* @NAME */ + LEX_T_PORT_GROUP, /* @NAME */ + LEX_T_TEMPLATE, /* ^NAME */ LEX_T_ERROR, /* invalid input */ /* Bare tokens. */ @@ -86,9 +89,9 @@ struct lex_token { /* One of LEX_*. */ enum lex_type type; - /* Meaningful for LEX_T_ID, LEX_T_STRING, LEX_T_ERROR, LEX_T_MACRO only. - * For these token types, 's' may point to 'buffer'; otherwise, it points - * to malloc()ed memory owned by the token. + /* Meaningful for LEX_T_ID, LEX_T_STRING, LEX_T_ERROR, LEX_T_MACRO, + * LEX_T_TEMPLATE only. For these token types, 's' may point to 'buffer'; + * otherwise, it points to malloc()ed memory owned by the token. * * Must be NULL for other token types. * @@ -151,4 +154,8 @@ void lexer_syntax_error(struct lexer *, const char *message, ...) char *lexer_steal_error(struct lexer *); +const char *lexer_parse_template_string(const char *s, + const struct smap *template_vars, + struct sset *template_vars_ref, + char **out_expanded); #endif /* ovn/lex.h */ diff --git a/lib/actions.c b/lib/actions.c index adbb42db4f..d7d4868835 100644 --- a/lib/actions.c +++ b/lib/actions.c @@ -258,8 +258,8 @@ add_prerequisite(struct action_context *ctx, const char *prerequisite) struct expr *expr; char *error; - expr = expr_parse_string(prerequisite, ctx->pp->symtab, NULL, NULL, - NULL, NULL, 0, &error); + expr = expr_parse_string(prerequisite, ctx->pp->symtab, NULL, NULL, NULL, + NULL, 0, &error); ovs_assert(!error); ctx->prereqs = expr_combine(EXPR_T_AND, ctx->prereqs, expr); } @@ -4695,6 +4695,11 @@ parse_set_action(struct action_context *ctx) static bool parse_action(struct action_context *ctx) { + if (ctx->lexer->token.type == LEX_T_TEMPLATE) { + lexer_error(ctx->lexer, "Unexpanded template."); + return false; + } + if (ctx->lexer->token.type != LEX_T_ID) { lexer_syntax_error(ctx->lexer, NULL); return false; diff --git a/lib/expr.c b/lib/expr.c index d1f9d28ca6..43f44771cf 100644 --- a/lib/expr.c +++ b/lib/expr.c @@ -695,7 +695,9 @@ parse_field(struct expr_context *ctx, struct expr_field *f) return false; } - symbol = shash_find_data(ctx->symtab, ctx->lexer->token.s); + symbol = ctx->symtab + ? shash_find_data(ctx->symtab, ctx->lexer->token.s) + : NULL; if (!symbol) { lexer_syntax_error(ctx->lexer, "expecting field name"); return false; @@ -894,7 +896,10 @@ parse_constant(struct expr_context *ctx, struct expr_constant_set *cs, cs->as_name = NULL; } - if (ctx->lexer->token.type == LEX_T_STRING) { + if (ctx->lexer->token.type == LEX_T_TEMPLATE) { + lexer_error(ctx->lexer, "Unexpanded template."); + return false; + } else if (ctx->lexer->token.type == LEX_T_STRING) { if (!assign_constant_set_type(ctx, cs, EXPR_C_STRING)) { return false; } @@ -978,7 +983,9 @@ expr_constant_parse(struct lexer *lexer, const struct expr_field *f, return false; } - struct expr_context ctx = { .lexer = lexer }; + struct expr_context ctx = { + .lexer = lexer, + }; struct expr_constant_set cs; memset(&cs, 0, sizeof cs); @@ -1332,7 +1339,10 @@ expr_parse_primary(struct expr_context *ctx, bool *atomic) return e; } - if (ctx->lexer->token.type == LEX_T_ID) { + if (ctx->lexer->token.type == LEX_T_TEMPLATE) { + lexer_error(ctx->lexer, "Unexpanded template."); + return NULL; + } else if (ctx->lexer->token.type == LEX_T_ID) { struct expr_field f; enum expr_relop r; struct expr_constant_set c; diff --git a/lib/lex.c b/lib/lex.c index c84d52aa8d..7b44bc3537 100644 --- a/lib/lex.c +++ b/lib/lex.c @@ -235,6 +235,10 @@ lex_token_format(const struct lex_token *token, struct ds *s) ds_put_format(s, "@%s", token->s); break; + case LEX_T_TEMPLATE: + ds_put_format(s, "^%s", token->s); + break; + case LEX_T_LPAREN: ds_put_cstr(s, "("); break; @@ -588,6 +592,18 @@ lex_parse_port_group(const char *p, struct lex_token *token) return lex_parse_id(p, LEX_T_PORT_GROUP, token); } +static const char * +lex_parse_template(const char *p, struct lex_token *token) +{ + p++; + if (!lex_is_id1(*p)) { + lex_error(token, "`^' must be followed by a valid identifier."); + return p; + } + + return lex_parse_id(p, LEX_T_TEMPLATE, token); +} + /* Initializes 'token' and parses the first token from the beginning of * null-terminated string 'p' into 'token'. Stores a pointer to the start of * the token (after skipping white space and comments, if any) into '*startp'. @@ -766,6 +782,10 @@ next: p = lex_parse_port_group(p, token); break; + case '^': + p = lex_parse_template(p, token); + break; + case ':': if (p[1] != ':') { token->type = LEX_T_COLON; @@ -1031,3 +1051,40 @@ lexer_steal_error(struct lexer *lexer) lexer->error = NULL; return error; } + +/* Takes ownership of 's' and expands all templates that are encountered + * in the contents of 's', if possible. Adds the encountered template names + * to 'template_vars_ref'. + */ +const char * +lexer_parse_template_string(const char *s, const struct smap *template_vars, + struct sset *template_vars_ref, + char **out_expanded) +{ + /* No '^' means no templates. */ + if (!strchr(s, '^')) { + *out_expanded = NULL; + return s; + } + + struct ds expanded = DS_EMPTY_INITIALIZER; + + struct lexer lexer; + lexer_init(&lexer, s); + + while (lexer_get(&lexer) != LEX_T_END) { + if (lexer.token.type == LEX_T_TEMPLATE) { + ds_put_cstr(&expanded, smap_get_def(template_vars, lexer.token.s, + lexer.token.s)); + if (template_vars_ref) { + sset_add(template_vars_ref, lexer.token.s); + } + } else { + lex_token_format(&lexer.token, &expanded); + } + } + + lexer_destroy(&lexer); + *out_expanded = ds_steal_cstr(&expanded); + return *out_expanded; +} diff --git a/lib/objdep.c b/lib/objdep.c index 092d4af261..06cf126f12 100644 --- a/lib/objdep.c +++ b/lib/objdep.c @@ -245,6 +245,7 @@ objdep_type_name(enum objdep_type type) [OBJDEP_TYPE_PORTGROUP] = "Port_Group", [OBJDEP_TYPE_PORTBINDING] = "Port_Binding", [OBJDEP_TYPE_MC_GROUP] = "Multicast_Group", + [OBJDEP_TYPE_TEMPLATE] = "Template", }; ovs_assert(type < OBJDEP_TYPE_MAX); diff --git a/lib/objdep.h b/lib/objdep.h index 50c7b01ef1..1ea781947c 100644 --- a/lib/objdep.h +++ b/lib/objdep.h @@ -26,6 +26,7 @@ enum objdep_type { OBJDEP_TYPE_PORTGROUP, OBJDEP_TYPE_PORTBINDING, OBJDEP_TYPE_MC_GROUP, + OBJDEP_TYPE_TEMPLATE, OBJDEP_TYPE_MAX, }; diff --git a/tests/ovn-controller.at b/tests/ovn-controller.at index 3c3fb31c7c..5fb6c65eba 100644 --- a/tests/ovn-controller.at +++ b/tests/ovn-controller.at @@ -2337,3 +2337,53 @@ done AT_CHECK([grep "deleted interface patch" hv1/ovs-vswitchd.log], [1], [ignore]) OVN_CLEANUP([hv1]) AT_CLEANUP + +OVN_FOR_EACH_NORTHD([ +AT_SETUP([ovn-controller - Local Chassis_Template_Var updates]) +AT_KEYWORDS([templates]) +ovn_start + +net_add n1 +sim_add hv +as hv +ovs-vsctl \ + -- add-br br-phys \ + -- add-br br-eth0 \ + -- add-br br-eth1 \ + -- add-br br-eth2 +ovn_attach n1 br-phys 192.168.0.1 + +m4_define([GET_LOCAL_TEMPLATE_VARS], + [ovn-appctl debug/dump-local-template-vars | grep -v 'Local template vars']) + +dnl Expect no local vars initially. +AT_CHECK([GET_LOCAL_TEMPLATE_VARS], [1], []) + +AT_CHECK([ovn-nbctl --wait=hv create Chassis_Template_Var chassis="hv"], [0], [ignore]) +AT_CHECK([GET_LOCAL_TEMPLATE_VARS], [1], []) + +dnl Expect new vars to be handled properly. +check ovn-nbctl --wait=hv set Chassis_Template_Var hv variables:foo=foo-hv variables:bar=bar-hv +AT_CHECK([GET_LOCAL_TEMPLATE_VARS], [0], [dnl +name: 'bar' value: 'bar-hv' +name: 'foo' value: 'foo-hv' +]) + +dnl Expect var updates to be handled properly. +check ovn-nbctl --wait=hv set Chassis_Template_Var hv variables:bar=bar-new-hv +AT_CHECK([GET_LOCAL_TEMPLATE_VARS], [0], [dnl +name: 'bar' value: 'bar-new-hv' +name: 'foo' value: 'foo-hv' +]) + +dnl Expect var deletions to be handled properly. +check ovn-nbctl --wait=hv remove Chassis_Template_Var hv variables bar +AT_CHECK([GET_LOCAL_TEMPLATE_VARS], [0], [dnl +name: 'foo' value: 'foo-hv' +]) + +check ovn-nbctl --wait=hv remove Chassis_Template_Var hv variables foo +AT_CHECK([GET_LOCAL_TEMPLATE_VARS], [1], []) + +AT_CLEANUP +]) diff --git a/tests/ovn.at b/tests/ovn.at index f8b8db4df8..2a59235c59 100644 --- a/tests/ovn.at +++ b/tests/ovn.at @@ -259,7 +259,7 @@ fe:x => error("Invalid numeric constant.") & => error("`&' is only valid as part of `&&'.") | => error("`|' is only valid as part of `||'.") -^ => error("Invalid character `^' in input.") +^ => error("`^' must be followed by a valid identifier.") ]) AT_CAPTURE_FILE([input.txt]) sed 's/ =>.*//' test-cases.txt > input.txt @@ -32975,3 +32975,81 @@ check ovn-nbctl --wait=hv sync OVN_CLEANUP([hv1]) AT_CLEANUP ]) + +OVN_FOR_EACH_NORTHD([ +AT_SETUP([Logical flows with Chassis_Template_Var references]) +AT_KEYWORDS([templates]) +ovn_start +net_add n1 + +sim_add hv1 +as hv1 +ovs-vsctl add-br br-phys +ovn_attach n1 br-phys 192.168.0.1 + +check ovn-nbctl ls-add sw + +dnl Use --wait=sb to ensure lsp1 getting a tunnel_key before lsp2. +check ovn-nbctl --wait=sb lsp-add sw lsp1 +check ovn-nbctl --wait=sb lsp-add sw lsp2 + +AT_CHECK([ovn-nbctl create Chassis_Template_Var chassis=hv1], [0], [ignore]) + +check ovn-nbctl pg-add pg1 lsp1 lsp2 +AT_CHECK([ovn-nbctl create address_set name=as1 addresses=\"1.1.1.1\",\"1.1.1.2\"], [0], [ignore]) + +dnl Create an ACL that uses an "uninstantiated" template. +check ovn-nbctl acl-add sw from-lport 1 "ip4.src == 42.42.42.42 && ^CONDITION" allow + +check ovs-vsctl add-port br-int p1 -- set interface p1 external_ids:iface-id=lsp1 +check ovs-vsctl add-port br-int p2 -- set interface p2 external_ids:iface-id=lsp2 + +wait_for_ports_up +ovn-nbctl --wait=hv sync + +dnl Ensure the ACL is not translated to OpenFlow. +as hv1 +AT_CHECK([ovs-ofctl dump-flows br-int | grep '42\.42\.42\.42'], [1], []) + +dnl Create a Chassis_Template_Var mapping for CONDITION. +check ovn-nbctl --wait=hv set Chassis_Template_Var hv1 variables:CONDITION='inport == @pg1' + +lsp1=0x$(fetch_column Port_Binding tunnel_key logical_port=lsp1) +lsp2=0x$(fetch_column Port_Binding tunnel_key logical_port=lsp2) + +dnl Ensure the ACL is translated to OpenFlows expanding pg1. +as hv1 +AT_CHECK_UNQUOTED([ovs-ofctl dump-flows br-int | grep '42\.42\.42\.42' | ofctl_strip_all], [0], [dnl + table=16, priority=1001,ip,reg14=$lsp1,metadata=0x1,nw_src=42.42.42.42 actions=resubmit(,17) + table=16, priority=1001,ip,reg14=$lsp2,metadata=0x1,nw_src=42.42.42.42 actions=resubmit(,17) +]) + +dnl Remove a port from pg1 and expect OpenFlows to be correctly updated. +check ovn-nbctl --wait=hv pg-set-ports pg1 lsp2 +AT_CHECK_UNQUOTED([ovs-ofctl dump-flows br-int | grep '42\.42\.42\.42' | ofctl_strip_all], [0], [dnl + table=16, priority=1001,ip,reg14=$lsp2,metadata=0x1,nw_src=42.42.42.42 actions=resubmit(,17) +]) + +dnl Change the Chassis_Template_Var mapping to use the address set. +check ovn-nbctl --wait=hv set Chassis_Template_Var hv1 variables:CONDITION='ip4.dst == $as1' + +dnl Ensure the ACL is translated to OpenFlows expanding as1. +as hv1 +AT_CHECK_UNQUOTED([ovs-ofctl dump-flows br-int | grep '42\.42\.42\.42' | ofctl_strip_all], [0], [dnl + table=16, priority=1001,ip,metadata=0x1,nw_src=42.42.42.42,nw_dst=1.1.1.1 actions=resubmit(,17) + table=16, priority=1001,ip,metadata=0x1,nw_src=42.42.42.42,nw_dst=1.1.1.2 actions=resubmit(,17) +]) + +dnl Remove an IP from AS1 and expect OpenFlows to be correctly updated. +check ovn-nbctl set address_set as1 addresses=\"1.1.1.1\" +AT_CHECK_UNQUOTED([ovs-ofctl dump-flows br-int | grep '42\.42\.42\.42' | ofctl_strip_all], [0], [dnl + table=16, priority=1001,ip,metadata=0x1,nw_src=42.42.42.42,nw_dst=1.1.1.1 actions=resubmit(,17) +]) + +dnl Remove the mapping and expect OpenFlows to be removed. +check ovn-nbctl --wait=hv clear Chassis_Template_Var hv1 variables +AT_CHECK([ovs-ofctl dump-flows br-int | grep '42\.42\.42\.42'], [1], []) + +OVN_CLEANUP([hv1]) +AT_CLEANUP +]) diff --git a/tests/test-ovn.c b/tests/test-ovn.c index a241f150d1..c422bafd70 100644 --- a/tests/test-ovn.c +++ b/tests/test-ovn.c @@ -291,12 +291,14 @@ test_parse_expr__(int steps) struct shash symtab; struct shash addr_sets; struct shash port_groups; + struct smap template_vars; struct simap ports; struct ds input; create_symtab(&symtab); create_addr_sets(&addr_sets); create_port_groups(&port_groups); + smap_init(&template_vars); simap_init(&ports); simap_put(&ports, "eth0", 5); @@ -355,6 +357,7 @@ test_parse_expr__(int steps) shash_destroy(&addr_sets); expr_const_sets_destroy(&port_groups); shash_destroy(&port_groups); + smap_destroy(&template_vars); } static void @@ -913,8 +916,8 @@ test_tree_shape_exhaustively(struct expr *expr, struct shash *symtab, expr_format(expr, &s); char *error; - modified = expr_parse_string(ds_cstr(&s), symtab, NULL, - NULL, NULL, NULL, 0, &error); + modified = expr_parse_string(ds_cstr(&s), symtab, NULL, NULL, NULL, + NULL, 0, &error); if (error) { fprintf(stderr, "%s fails to parse (%s)\n", ds_cstr(&s), error); @@ -1286,6 +1289,8 @@ test_parse_actions(struct ovs_cmdl_context *ctx OVS_UNUSED) struct ds input; bool ok = true; + struct smap template_vars = SMAP_INITIALIZER(&template_vars); + create_symtab(&symtab); create_gen_opts(&dhcp_opts, &dhcpv6_opts, &nd_ra_opts, &event_opts); @@ -1322,7 +1327,11 @@ test_parse_actions(struct ovs_cmdl_context *ctx OVS_UNUSED) .n_tables = 24, .cur_ltable = 10, }; - error = ovnacts_parse_string(ds_cstr(&input), &pp, &ovnacts, &prereqs); + char *exp_input_expanded = NULL; + const char *exp_input = + lexer_parse_template_string(ds_cstr(&input), &template_vars, NULL, + &exp_input_expanded); + error = ovnacts_parse_string(exp_input, &pp, &ovnacts, &prereqs); if (!error) { /* Convert the parsed representation back to a string and print it, * if it's different from the input. */ @@ -1409,6 +1418,7 @@ test_parse_actions(struct ovs_cmdl_context *ctx OVS_UNUSED) expr_destroy(prereqs); ovnacts_free(ovnacts.data, ovnacts.size); ofpbuf_uninit(&ovnacts); + free(exp_input_expanded); } ds_destroy(&input); @@ -1419,6 +1429,7 @@ test_parse_actions(struct ovs_cmdl_context *ctx OVS_UNUSED) dhcp_opts_destroy(&dhcpv6_opts); nd_ra_opts_destroy(&nd_ra_opts); controller_event_opts_destroy(&event_opts); + smap_destroy(&template_vars); ovn_extend_table_destroy(&group_table); ovn_extend_table_destroy(&meter_table); exit(ok ? EXIT_SUCCESS : EXIT_FAILURE); diff --git a/utilities/ovn-trace.c b/utilities/ovn-trace.c index 6fa5137d95..b837884ccc 100644 --- a/utilities/ovn-trace.c +++ b/utilities/ovn-trace.c @@ -505,6 +505,7 @@ static struct hmap dhcp_opts; /* Contains "struct gen_opts_map"s. */ static struct hmap dhcpv6_opts; /* Contains "struct gen_opts_map"s. */ static struct hmap nd_ra_opts; /* Contains "struct gen_opts_map"s. */ static struct controller_event_options event_opts; +static struct smap template_vars; static struct ovntrace_datapath * ovntrace_datapath_find_by_sb_uuid(const struct uuid *sb_uuid) @@ -955,9 +956,15 @@ parse_lflow_for_datapath(const struct sbrec_logical_flow *sblf, char *error; struct expr *match; - match = expr_parse_string(sblf->match, &symtab, &address_sets, + char *match_expanded_s = NULL; + const char *match_s = lexer_parse_template_string(sblf->match, + &template_vars, + NULL, + &match_expanded_s); + match = expr_parse_string(match_s, &symtab, &address_sets, &port_groups, NULL, NULL, dp->tunnel_key, &error); + free(match_expanded_s); if (error) { VLOG_WARN("%s: parsing expression failed (%s)", sblf->match, error); @@ -980,7 +987,12 @@ parse_lflow_for_datapath(const struct sbrec_logical_flow *sblf, uint64_t stub[1024 / 8]; struct ofpbuf ovnacts = OFPBUF_STUB_INITIALIZER(stub); struct expr *prereqs; - error = ovnacts_parse_string(sblf->actions, &pp, &ovnacts, &prereqs); + char *actions_expanded_s = NULL; + const char *actions_s = + lexer_parse_template_string(sblf->actions, &template_vars, + NULL, &actions_expanded_s); + error = ovnacts_parse_string(actions_s, &pp, &ovnacts, &prereqs); + free(actions_expanded_s); if (error) { VLOG_WARN("%s: parsing actions failed (%s)", sblf->actions, error); free(error); @@ -1078,6 +1090,7 @@ read_gen_opts(void) nd_ra_opts_init(&nd_ra_opts); controller_event_opts_init(&event_opts); + smap_init(&template_vars); } static void @@ -3420,9 +3433,15 @@ trace_parse(const char *dp_s, const char *flow_s, * * First make sure that the expression parses. */ char *error; - struct expr *e = expr_parse_string(flow_s, &symtab, &address_sets, + char *flow_expanded_s = NULL; + const char *flow_exp_s = lexer_parse_template_string(flow_s, + &template_vars, + NULL, + &flow_expanded_s); + struct expr *e = expr_parse_string(flow_exp_s, &symtab, &address_sets, &port_groups, NULL, NULL, 0, &error); + free(flow_expanded_s); if (!e) { return trace_parse_error(error); } @@ -3447,9 +3466,15 @@ trace_parse(const char *dp_s, const char *flow_s, free(port_name); } - char *error = expr_parse_microflow(flow_s, &symtab, &address_sets, + char *flow_expanded_s = NULL; + const char *flow_exp_s = lexer_parse_template_string(flow_s, + &template_vars, + NULL, + &flow_expanded_s); + char *error = expr_parse_microflow(flow_exp_s, &symtab, &address_sets, &port_groups, ovntrace_lookup_port, *dpp, uflow); + free(flow_expanded_s); if (error) { return trace_parse_error(error); } From patchwork Fri Nov 4 22:11:43 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dumitru Ceara X-Patchwork-Id: 1699896 X-Patchwork-Delegate: mmichels@redhat.com Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::136; helo=smtp3.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=Th/aGnPw; dkim-atps=neutral Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4N3vxh3yfxz23m0 for ; Sat, 5 Nov 2022 09:12:24 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id C21DF611AF; Fri, 4 Nov 2022 22:12:22 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org C21DF611AF Authentication-Results: smtp3.osuosl.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=Th/aGnPw X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id U5Znf0lm3fsQ; Fri, 4 Nov 2022 22:12:20 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp3.osuosl.org (Postfix) with ESMTPS id D12EE61197; Fri, 4 Nov 2022 22:12:19 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org D12EE61197 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id B7D3FC0032; Fri, 4 Nov 2022 22:12:19 +0000 (UTC) X-Original-To: ovs-dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp2.osuosl.org (smtp2.osuosl.org [IPv6:2605:bc80:3010::133]) by lists.linuxfoundation.org (Postfix) with ESMTP id C4673C002D for ; Fri, 4 Nov 2022 22:12:18 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 7F65240FE2 for ; Fri, 4 Nov 2022 22:11:54 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 7F65240FE2 Authentication-Results: smtp2.osuosl.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=Th/aGnPw X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Z1Xt8c9jcziF for ; Fri, 4 Nov 2022 22:11:52 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org CB51F40FCE Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by smtp2.osuosl.org (Postfix) with ESMTPS id CB51F40FCE for ; Fri, 4 Nov 2022 22:11:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1667599910; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Zd2cUzpBe0Ou4I5ZplH2Ey1jJKUcKjKhe6i7IR+PEN4=; b=Th/aGnPwHN/BC8ha3Our/kDY2wvA3zYcRdy/+1VPAWNcJirAQVxjCG3Tc1MBo9DZc3mn3W huCFA7JLrseCbxBAmv846Yk3t8R9U9PsPF6KhOt2lQmqFnvPB6qWSGR7ctlUM6G9UAoaJa t7Std2Ho8kUq9ShNFF9CQFNSniQbFuA= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-641-V67qIpOsPGG_2qm1xaw0UQ-1; Fri, 04 Nov 2022 18:11:47 -0400 X-MC-Unique: V67qIpOsPGG_2qm1xaw0UQ-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 118263C02189; Fri, 4 Nov 2022 22:11:47 +0000 (UTC) Received: from dceara.remote.csb (unknown [10.39.192.36]) by smtp.corp.redhat.com (Postfix) with ESMTP id 77C3039D7C; Fri, 4 Nov 2022 22:11:45 +0000 (UTC) From: Dumitru Ceara To: ovs-dev@openvswitch.org Date: Fri, 4 Nov 2022 23:11:43 +0100 Message-Id: <166759990211.515460.9734961082398063748.stgit@dceara.remote.csb> In-Reply-To: <166759985955.515460.16275468131013093936.stgit@dceara.remote.csb> References: <166759985955.515460.16275468131013093936.stgit@dceara.remote.csb> User-Agent: StGit/0.23 MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.5 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Cc: surya@redhat.com, i.maximets@ovn.org Subject: [ovs-dev] [PATCH ovn v2 4/5] lb: Support using templates. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" Allow the CMS to configure template LBs. The following configurations are supported: - VIPs of the form: ^vip_variable[:^port_variable|:port] - Backends of the form: ^backendip_variable1[:^port_variable1|:port],^backendip_variable2[:^port_variable2|:port] OR ^backends_variable1,^backends_variable2 The CMS needs to provide a bit more information than with non-template load balancers and must explicitly specify the address family to be used. There is currently no support for template load balancers with options:add_route=true set. That is because ovn-northd does not instantiate template variables. While this is a limitation in a way, its impact is not huge. The load balancer 'add_route' option was added as a way to make the CMS life easier and to avoid having to explicitly add a route for the VIP. The CMS can still achieve the same logical topology by explicitly adding the VIP route. Template load balancers don't support the "reachable" neighbor-responder mode. Instead the CMS can explicitly configure the responder mode to either "all" or "none". To properly handle template updates in ovn-controller we also add a Chassis_Template_Var <- LB reference in ovn-controller. This way, when a Chassis_Template_Var changes value all load balancers that refer to it will also get updated. Signed-off-by: Dumitru Ceara --- V2: - Fix GCC build due to missing explicit return. - Fix ls_in_pre_stateful flows due to using wrong lb field. - Use new lexer_parse_template_string(). - Changed lb_handle_changed_ref() signature to return bool. - Update documentation with info about responder mode=none, LB template supported formats, lb explicit address family requirements. - Squashed the template LB patches into a single one - Added more tests. - Squashed the system tests patch into this one. --- controller/lflow.c | 118 +++++++++-- controller/lflow.h | 7 + controller/ovn-controller.c | 67 +++++- lib/lb.c | 457 ++++++++++++++++++++++++++++++++++++++----- lib/lb.h | 40 +++- lib/ovn-util.c | 3 northd/northd.c | 89 ++++---- ovn-nb.xml | 53 +++++ tests/ovn-nbctl.at | 23 +- tests/ovn-northd.at | 7 + tests/ovn.at | 131 ++++++++++++ tests/system-ovn.at | 183 +++++++++++++++++ utilities/ovn-nbctl.c | 122 ++++++----- 13 files changed, 1071 insertions(+), 229 deletions(-) diff --git a/controller/lflow.c b/controller/lflow.c index fc4371d0df..7f880bd62b 100644 --- a/controller/lflow.c +++ b/controller/lflow.c @@ -97,6 +97,15 @@ consider_logical_flow(const struct sbrec_logical_flow *lflow, struct lflow_ctx_in *l_ctx_in, struct lflow_ctx_out *l_ctx_out); +static void +consider_lb_hairpin_flows(struct objdep_mgr *mgr, + const struct sbrec_load_balancer *sbrec_lb, + const struct hmap *local_datapaths, + const struct smap *template_vars, + bool use_ct_mark, + struct ovn_desired_flow_table *flow_table, + struct simap *ids); + static void add_port_sec_flows(const struct shash *binding_lports, const struct sbrec_chassis *, struct ovn_desired_flow_table *); @@ -223,7 +232,7 @@ lflow_handle_changed_flows(struct lflow_ctx_in *l_ctx_in, UUIDSET_INITIALIZER(&flood_remove_nodes); SBREC_LOGICAL_FLOW_TABLE_FOR_EACH_TRACKED (lflow, l_ctx_in->logical_flow_table) { - if (uuidset_find(l_ctx_out->lflows_processed, &lflow->header_.uuid)) { + if (uuidset_find(l_ctx_out->objs_processed, &lflow->header_.uuid)) { VLOG_DBG("lflow "UUID_FMT"has been processed, skip.", UUID_ARGS(&lflow->header_.uuid)); continue; @@ -253,14 +262,14 @@ lflow_handle_changed_flows(struct lflow_ctx_in *l_ctx_in, UUID_ARGS(&lflow->header_.uuid)); /* For the extra lflows that need to be reprocessed because of the - * flood remove, remove it from lflows_processed. */ + * flood remove, remove it from objs_processed. */ struct uuidset_node *unode = - uuidset_find(l_ctx_out->lflows_processed, + uuidset_find(l_ctx_out->objs_processed, &lflow->header_.uuid); if (unode) { VLOG_DBG("lflow "UUID_FMT"has been processed, now reprocess.", UUID_ARGS(&lflow->header_.uuid)); - uuidset_delete(l_ctx_out->lflows_processed, unode); + uuidset_delete(l_ctx_out->objs_processed, unode); } consider_logical_flow(lflow, false, l_ctx_in, l_ctx_out); @@ -677,7 +686,7 @@ lflow_handle_addr_set_update(const char *as_name, struct object_to_resources_list_node *resource_list_node; RESOURCE_FOR_EACH_OBJ (resource_list_node, resource_node) { const struct uuid *obj_uuid = &resource_list_node->obj_uuid; - if (uuidset_find(l_ctx_out->lflows_processed, obj_uuid)) { + if (uuidset_find(l_ctx_out->objs_processed, obj_uuid)) { VLOG_DBG("lflow "UUID_FMT"has been processed, skip.", UUID_ARGS(obj_uuid)); continue; @@ -767,13 +776,13 @@ lflow_handle_changed_ref(enum objdep_type type, const char *res_name, } /* For the extra lflows that need to be reprocessed because of the - * flood remove, remove it from lflows_processed. */ + * flood remove, remove it from objs_processed. */ struct uuidset_node *unode = - uuidset_find(l_ctx_out->lflows_processed, &lflow->header_.uuid); + uuidset_find(l_ctx_out->objs_processed, &lflow->header_.uuid); if (unode) { VLOG_DBG("lflow "UUID_FMT"has been processed, now reprocess.", UUID_ARGS(&lflow->header_.uuid)); - uuidset_delete(l_ctx_out->lflows_processed, unode); + uuidset_delete(l_ctx_out->objs_processed, unode); } consider_logical_flow(lflow, false, l_ctx_in, l_ctx_out); @@ -782,6 +791,43 @@ lflow_handle_changed_ref(enum objdep_type type, const char *res_name, return true; } +bool +lb_handle_changed_ref(enum objdep_type type, const char *res_name, + struct ovs_list *objs_todo, + const void *in_arg, void *out_arg) +{ + struct lflow_ctx_in *l_ctx_in = CONST_CAST(struct lflow_ctx_in *, in_arg); + struct lflow_ctx_out *l_ctx_out = out_arg; + + struct object_to_resources_list_node *resource_lb_uuid; + LIST_FOR_EACH_POP (resource_lb_uuid, list_node, objs_todo) { + VLOG_DBG("Reprocess LB "UUID_FMT" for resource type: %s, name: %s", + UUID_ARGS(&resource_lb_uuid->obj_uuid), + objdep_type_name(type), res_name); + + const struct sbrec_load_balancer *lb = + sbrec_load_balancer_table_get_for_uuid( + l_ctx_in->lb_table, &resource_lb_uuid->obj_uuid); + if (!lb) { + VLOG_DBG("Failed to find LB "UUID_FMT" referred by: %s", + UUID_ARGS(&resource_lb_uuid->obj_uuid), res_name); + } else { + ofctrl_remove_flows(l_ctx_out->flow_table, + &resource_lb_uuid->obj_uuid); + + consider_lb_hairpin_flows(l_ctx_out->lb_deps_mgr, lb, + l_ctx_in->local_datapaths, + l_ctx_in->template_vars, + l_ctx_in->lb_hairpin_use_ct_mark, + l_ctx_out->flow_table, + l_ctx_out->hairpin_lb_ids); + } + + free(resource_lb_uuid); + } + return true; +} + static void lflow_parse_ctrl_meter(const struct sbrec_logical_flow *lflow, struct ovn_extend_table *meter_table, @@ -1263,9 +1309,9 @@ consider_logical_flow(const struct sbrec_logical_flow *lflow, COVERAGE_INC(consider_logical_flow); if (!is_recompute) { - ovs_assert(!uuidset_find(l_ctx_out->lflows_processed, + ovs_assert(!uuidset_find(l_ctx_out->objs_processed, &lflow->header_.uuid)); - uuidset_insert(l_ctx_out->lflows_processed, &lflow->header_.uuid); + uuidset_insert(l_ctx_out->objs_processed, &lflow->header_.uuid); } if (dp) { @@ -2005,8 +2051,11 @@ add_lb_ct_snat_hairpin_flows(struct ovn_controller_lb *lb, } static void -consider_lb_hairpin_flows(const struct sbrec_load_balancer *sbrec_lb, - const struct hmap *local_datapaths, bool use_ct_mark, +consider_lb_hairpin_flows(struct objdep_mgr *mgr, + const struct sbrec_load_balancer *sbrec_lb, + const struct hmap *local_datapaths, + const struct smap *template_vars, + bool use_ct_mark, struct ovn_desired_flow_table *flow_table, struct simap *ids) { @@ -2042,7 +2091,9 @@ consider_lb_hairpin_flows(const struct sbrec_load_balancer *sbrec_lb, return; } - struct ovn_controller_lb *lb = ovn_controller_lb_create(sbrec_lb); + struct sset template_vars_ref = SSET_INITIALIZER(&template_vars_ref); + struct ovn_controller_lb *lb = + ovn_controller_lb_create(sbrec_lb, template_vars, &template_vars_ref); uint8_t lb_proto = IPPROTO_TCP; if (lb->slb->protocol && lb->slb->protocol[0]) { if (!strcmp(lb->slb->protocol, "udp")) { @@ -2052,6 +2103,11 @@ consider_lb_hairpin_flows(const struct sbrec_load_balancer *sbrec_lb, } } + const char *tv_name; + SSET_FOR_EACH (tv_name, &template_vars_ref) { + objdep_mgr_add(mgr, OBJDEP_TYPE_TEMPLATE, tv_name, + &sbrec_lb->header_.uuid); + } for (i = 0; i < lb->n_vips; i++) { struct ovn_lb_vip *lb_vip = &lb->vips[i]; @@ -2066,13 +2122,17 @@ consider_lb_hairpin_flows(const struct sbrec_load_balancer *sbrec_lb, add_lb_ct_snat_hairpin_flows(lb, id, lb_proto, flow_table); ovn_controller_lb_destroy(lb); + sset_destroy(&template_vars_ref); } /* Adds OpenFlow flows to flow tables for each Load balancer VIPs and * backends to handle the load balanced hairpin traffic. */ static void -add_lb_hairpin_flows(const struct sbrec_load_balancer_table *lb_table, - const struct hmap *local_datapaths, bool use_ct_mark, +add_lb_hairpin_flows(struct objdep_mgr *mgr, + const struct sbrec_load_balancer_table *lb_table, + const struct hmap *local_datapaths, + const struct smap *template_vars, + bool use_ct_mark, struct ovn_desired_flow_table *flow_table, struct simap *ids, struct id_pool *pool) @@ -2095,8 +2155,8 @@ add_lb_hairpin_flows(const struct sbrec_load_balancer_table *lb_table, ovs_assert(id_pool_alloc_id(pool, &id)); simap_put(ids, lb->name, id); } - consider_lb_hairpin_flows(lb, local_datapaths, use_ct_mark, flow_table, - ids); + consider_lb_hairpin_flows(mgr, lb, local_datapaths, template_vars, + use_ct_mark, flow_table, ids); } } @@ -2232,7 +2292,9 @@ lflow_run(struct lflow_ctx_in *l_ctx_in, struct lflow_ctx_out *l_ctx_out) l_ctx_in->static_mac_binding_table, l_ctx_in->local_datapaths, l_ctx_out->flow_table); - add_lb_hairpin_flows(l_ctx_in->lb_table, l_ctx_in->local_datapaths, + add_lb_hairpin_flows(l_ctx_out->lb_deps_mgr, l_ctx_in->lb_table, + l_ctx_in->local_datapaths, + l_ctx_in->template_vars, l_ctx_in->lb_hairpin_use_ct_mark, l_ctx_out->flow_table, l_ctx_out->hairpin_lb_ids, @@ -2283,10 +2345,10 @@ lflow_add_flows_for_datapath(const struct sbrec_datapath_binding *dp, const struct sbrec_logical_flow *lflow; SBREC_LOGICAL_FLOW_FOR_EACH_EQUAL ( lflow, lf_row, l_ctx_in->sbrec_logical_flow_by_logical_datapath) { - if (uuidset_find(l_ctx_out->lflows_processed, &lflow->header_.uuid)) { + if (uuidset_find(l_ctx_out->objs_processed, &lflow->header_.uuid)) { continue; } - uuidset_insert(l_ctx_out->lflows_processed, &lflow->header_.uuid); + uuidset_insert(l_ctx_out->objs_processed, &lflow->header_.uuid); consider_logical_flow__(lflow, dp, l_ctx_in, l_ctx_out); } sbrec_logical_flow_index_destroy_row(lf_row); @@ -2311,7 +2373,7 @@ lflow_add_flows_for_datapath(const struct sbrec_datapath_binding *dp, sbrec_logical_flow_index_set_logical_dp_group(lf_row, ldpg); SBREC_LOGICAL_FLOW_FOR_EACH_EQUAL ( lflow, lf_row, l_ctx_in->sbrec_logical_flow_by_logical_dp_group) { - if (uuidset_find(l_ctx_out->lflows_processed, + if (uuidset_find(l_ctx_out->objs_processed, &lflow->header_.uuid)) { continue; } @@ -2363,7 +2425,9 @@ lflow_add_flows_for_datapath(const struct sbrec_datapath_binding *dp, /* Add load balancer hairpin flows if the datapath has any load balancers * associated. */ for (size_t i = 0; i < n_dp_lbs; i++) { - consider_lb_hairpin_flows(dp_lbs[i], l_ctx_in->local_datapaths, + consider_lb_hairpin_flows(l_ctx_out->lb_deps_mgr, dp_lbs[i], + l_ctx_in->local_datapaths, + l_ctx_in->template_vars, l_ctx_in->lb_hairpin_use_ct_mark, l_ctx_out->flow_table, l_ctx_out->hairpin_lb_ids); @@ -2385,7 +2449,7 @@ lflow_handle_flows_for_lport(const struct sbrec_port_binding *pb, OBJDEP_TYPE_PORTBINDING, pb->logical_port, lflow_handle_changed_ref, - l_ctx_out->lflows_processed, + l_ctx_out->objs_processed, l_ctx_in, l_ctx_out, &changed)) { return false; } @@ -2424,7 +2488,7 @@ lflow_handle_changed_port_bindings(struct lflow_ctx_in *l_ctx_in, OBJDEP_TYPE_PORTBINDING, pb->logical_port, lflow_handle_changed_ref, - l_ctx_out->lflows_processed, + l_ctx_out->objs_processed, l_ctx_in, l_ctx_out, &changed)) { ret = false; break; @@ -2451,7 +2515,7 @@ lflow_handle_changed_mc_groups(struct lflow_ctx_in *l_ctx_in, if (!objdep_mgr_handle_change(l_ctx_out->lflow_deps_mgr, OBJDEP_TYPE_MC_GROUP, ds_cstr(&mg_key), lflow_handle_changed_ref, - l_ctx_out->lflows_processed, + l_ctx_out->objs_processed, l_ctx_in, l_ctx_out, &changed)) { ret = false; break; @@ -2505,7 +2569,9 @@ lflow_handle_changed_lbs(struct lflow_ctx_in *l_ctx_in, VLOG_DBG("Add load balancer hairpin flows for "UUID_FMT, UUID_ARGS(&lb->header_.uuid)); - consider_lb_hairpin_flows(lb, l_ctx_in->local_datapaths, + consider_lb_hairpin_flows(l_ctx_out->lb_deps_mgr, lb, + l_ctx_in->local_datapaths, + l_ctx_in->template_vars, l_ctx_in->lb_hairpin_use_ct_mark, l_ctx_out->flow_table, l_ctx_out->hairpin_lb_ids); diff --git a/controller/lflow.h b/controller/lflow.h index 9a7079f99e..069b5d795f 100644 --- a/controller/lflow.h +++ b/controller/lflow.h @@ -121,9 +121,10 @@ struct lflow_ctx_out { struct ovn_extend_table *group_table; struct ovn_extend_table *meter_table; struct objdep_mgr *lflow_deps_mgr; + struct objdep_mgr *lb_deps_mgr; struct lflow_cache *lflow_cache; struct conj_ids *conj_ids; - struct uuidset *lflows_processed; + struct uuidset *objs_processed; struct simap *hairpin_lb_ids; struct id_pool *hairpin_id_pool; }; @@ -173,4 +174,8 @@ bool lflow_handle_changed_mc_groups(struct lflow_ctx_in *, struct lflow_ctx_out *); bool lflow_handle_changed_port_bindings(struct lflow_ctx_in *, struct lflow_ctx_out *); + +bool lb_handle_changed_ref(enum objdep_type type, const char *res_name, + struct ovs_list *objs_todo, + const void *in_arg, void *out_arg); #endif /* controller/lflow.h */ diff --git a/controller/ovn-controller.c b/controller/ovn-controller.c index c374bd0f33..f0be783ee6 100644 --- a/controller/ovn-controller.c +++ b/controller/ovn-controller.c @@ -2769,13 +2769,15 @@ struct ed_type_lflow_output { struct ovn_extend_table meter_table; /* lflow <-> resource cross reference */ struct objdep_mgr lflow_deps_mgr;; + /* load balancer <-> resource cross reference */ + struct objdep_mgr lb_deps_mgr; /* conjunciton ID usage information of lflows */ struct conj_ids conj_ids; - /* lflows processed in the current engine execution. + /* objects (lflows and lbs) processed in the current engine execution. * Cleared by en_lflow_output_clear_tracked_data before each engine * execution. */ - struct uuidset lflows_processed; + struct uuidset objs_processed; /* Data which is persistent and not cleared during * full recompute. */ @@ -2932,8 +2934,9 @@ init_lflow_ctx(struct engine_node *node, l_ctx_out->group_table = &fo->group_table; l_ctx_out->meter_table = &fo->meter_table; l_ctx_out->lflow_deps_mgr = &fo->lflow_deps_mgr; + l_ctx_out->lb_deps_mgr = &fo->lb_deps_mgr; l_ctx_out->conj_ids = &fo->conj_ids; - l_ctx_out->lflows_processed = &fo->lflows_processed; + l_ctx_out->objs_processed = &fo->objs_processed; l_ctx_out->lflow_cache = fo->pd.lflow_cache; l_ctx_out->hairpin_id_pool = fo->hd.pool; l_ctx_out->hairpin_lb_ids = &fo->hd.ids; @@ -2948,8 +2951,9 @@ en_lflow_output_init(struct engine_node *node OVS_UNUSED, ovn_extend_table_init(&data->group_table); ovn_extend_table_init(&data->meter_table); objdep_mgr_init(&data->lflow_deps_mgr); + objdep_mgr_init(&data->lb_deps_mgr); lflow_conj_ids_init(&data->conj_ids); - uuidset_init(&data->lflows_processed); + uuidset_init(&data->objs_processed); simap_init(&data->hd.ids); data->hd.pool = id_pool_create(1, UINT32_MAX - 1); nd_ra_opts_init(&data->nd_ra_opts); @@ -2961,7 +2965,7 @@ static void en_lflow_output_clear_tracked_data(void *data) { struct ed_type_lflow_output *flow_output_data = data; - uuidset_clear(&flow_output_data->lflows_processed); + uuidset_clear(&flow_output_data->objs_processed); } static void @@ -2972,8 +2976,9 @@ en_lflow_output_cleanup(void *data) ovn_extend_table_destroy(&flow_output_data->group_table); ovn_extend_table_destroy(&flow_output_data->meter_table); objdep_mgr_destroy(&flow_output_data->lflow_deps_mgr); + objdep_mgr_destroy(&flow_output_data->lb_deps_mgr); lflow_conj_ids_destroy(&flow_output_data->conj_ids); - uuidset_destroy(&flow_output_data->lflows_processed); + uuidset_destroy(&flow_output_data->objs_processed); lflow_cache_destroy(flow_output_data->pd.lflow_cache); simap_destroy(&flow_output_data->hd.ids); id_pool_destroy(flow_output_data->hd.pool); @@ -3008,6 +3013,7 @@ en_lflow_output_run(struct engine_node *node, void *data) struct ovn_extend_table *group_table = &fo->group_table; struct ovn_extend_table *meter_table = &fo->meter_table; struct objdep_mgr *lflow_deps_mgr = &fo->lflow_deps_mgr; + struct objdep_mgr *lb_deps_mgr = &fo->lb_deps_mgr; static bool first_run = true; if (first_run) { @@ -3017,6 +3023,7 @@ en_lflow_output_run(struct engine_node *node, void *data) ovn_extend_table_clear(group_table, false /* desired */); ovn_extend_table_clear(meter_table, false /* desired */); objdep_mgr_clear(lflow_deps_mgr); + objdep_mgr_clear(lb_deps_mgr); lflow_conj_ids_clear(&fo->conj_ids); } @@ -3150,7 +3157,7 @@ lflow_output_addr_sets_handler(struct engine_node *node, void *data) if (!objdep_mgr_handle_change(l_ctx_out.lflow_deps_mgr, OBJDEP_TYPE_ADDRSET, ref_name, lflow_handle_changed_ref, - l_ctx_out.lflows_processed, + l_ctx_out.objs_processed, &l_ctx_in, &l_ctx_out, &changed)) { return false; } @@ -3169,7 +3176,7 @@ lflow_output_addr_sets_handler(struct engine_node *node, void *data) OBJDEP_TYPE_ADDRSET, shash_node->name, lflow_handle_changed_ref, - l_ctx_out.lflows_processed, + l_ctx_out.objs_processed, &l_ctx_in, &l_ctx_out, &changed)) { return false; } @@ -3182,7 +3189,7 @@ lflow_output_addr_sets_handler(struct engine_node *node, void *data) if (!objdep_mgr_handle_change(l_ctx_out.lflow_deps_mgr, OBJDEP_TYPE_ADDRSET, ref_name, lflow_handle_changed_ref, - l_ctx_out.lflows_processed, + l_ctx_out.objs_processed, &l_ctx_in, &l_ctx_out, &changed)) { return false; } @@ -3217,7 +3224,7 @@ lflow_output_port_groups_handler(struct engine_node *node, void *data) if (!objdep_mgr_handle_change(l_ctx_out.lflow_deps_mgr, OBJDEP_TYPE_PORTGROUP, ref_name, lflow_handle_changed_ref, - l_ctx_out.lflows_processed, + l_ctx_out.objs_processed, &l_ctx_in, &l_ctx_out, &changed)) { return false; } @@ -3229,7 +3236,7 @@ lflow_output_port_groups_handler(struct engine_node *node, void *data) if (!objdep_mgr_handle_change(l_ctx_out.lflow_deps_mgr, OBJDEP_TYPE_PORTGROUP, ref_name, lflow_handle_changed_ref, - l_ctx_out.lflows_processed, + l_ctx_out.objs_processed, &l_ctx_in, &l_ctx_out, &changed)) { return false; } @@ -3241,7 +3248,7 @@ lflow_output_port_groups_handler(struct engine_node *node, void *data) if (!objdep_mgr_handle_change(l_ctx_out.lflow_deps_mgr, OBJDEP_TYPE_PORTGROUP, ref_name, lflow_handle_changed_ref, - l_ctx_out.lflows_processed, + l_ctx_out.objs_processed, &l_ctx_in, &l_ctx_out, &changed)) { return false; } @@ -3275,7 +3282,17 @@ lflow_output_template_vars_handler(struct engine_node *node, void *data) if (!objdep_mgr_handle_change(l_ctx_out.lflow_deps_mgr, OBJDEP_TYPE_TEMPLATE, res_name, lflow_handle_changed_ref, - l_ctx_out.lflows_processed, + l_ctx_out.objs_processed, + &l_ctx_in, &l_ctx_out, &changed)) { + return false; + } + if (changed) { + engine_set_node_state(node, EN_UPDATED); + } + if (!objdep_mgr_handle_change(l_ctx_out.lb_deps_mgr, + OBJDEP_TYPE_TEMPLATE, + res_name, lb_handle_changed_ref, + l_ctx_out.objs_processed, &l_ctx_in, &l_ctx_out, &changed)) { return false; } @@ -3287,7 +3304,17 @@ lflow_output_template_vars_handler(struct engine_node *node, void *data) if (!objdep_mgr_handle_change(l_ctx_out.lflow_deps_mgr, OBJDEP_TYPE_TEMPLATE, res_name, lflow_handle_changed_ref, - l_ctx_out.lflows_processed, + l_ctx_out.objs_processed, + &l_ctx_in, &l_ctx_out, &changed)) { + return false; + } + if (changed) { + engine_set_node_state(node, EN_UPDATED); + } + if (!objdep_mgr_handle_change(l_ctx_out.lb_deps_mgr, + OBJDEP_TYPE_TEMPLATE, + res_name, lb_handle_changed_ref, + l_ctx_out.objs_processed, &l_ctx_in, &l_ctx_out, &changed)) { return false; } @@ -3299,7 +3326,17 @@ lflow_output_template_vars_handler(struct engine_node *node, void *data) if (!objdep_mgr_handle_change(l_ctx_out.lflow_deps_mgr, OBJDEP_TYPE_TEMPLATE, res_name, lflow_handle_changed_ref, - l_ctx_out.lflows_processed, + l_ctx_out.objs_processed, + &l_ctx_in, &l_ctx_out, &changed)) { + return false; + } + if (changed) { + engine_set_node_state(node, EN_UPDATED); + } + if (!objdep_mgr_handle_change(l_ctx_out.lb_deps_mgr, + OBJDEP_TYPE_TEMPLATE, + res_name, lb_handle_changed_ref, + l_ctx_out.objs_processed, &l_ctx_in, &l_ctx_out, &changed)) { return false; } diff --git a/lib/lb.c b/lib/lb.c index ab5de38a89..caeb9a8be7 100644 --- a/lib/lb.c +++ b/lib/lb.c @@ -19,6 +19,7 @@ #include "lib/ovn-nb-idl.h" #include "lib/ovn-sb-idl.h" #include "lib/ovn-util.h" +#include "ovn/lex.h" /* OpenvSwitch lib includes. */ #include "openvswitch/vlog.h" @@ -26,6 +27,16 @@ VLOG_DEFINE_THIS_MODULE(lb); +static const char *lb_neighbor_responder_mode_names[] = { + [LB_NEIGH_RESPOND_REACHABLE] = "reachable", + [LB_NEIGH_RESPOND_ALL] = "all", + [LB_NEIGH_RESPOND_NONE] = "none", +}; + +static struct nbrec_load_balancer_health_check * +ovn_lb_get_health_check(const struct nbrec_load_balancer *nbrec_lb, + const char *vip_port_str, bool template); + struct ovn_lb_ip_set * ovn_lb_ip_set_create(void) { @@ -71,94 +82,297 @@ ovn_lb_ip_set_clone(struct ovn_lb_ip_set *lb_ip_set) return clone; } -static -bool ovn_lb_vip_init(struct ovn_lb_vip *lb_vip, const char *lb_key, - const char *lb_value) +/* Format for backend ips: "IP1:port1,IP2:port2,...". */ +static char * +ovn_lb_backends_init_explicit(struct ovn_lb_vip *lb_vip, const char *value, + size_t *n_backends) { - int addr_family; - - if (!ip_address_and_port_from_lb_key(lb_key, &lb_vip->vip_str, - &lb_vip->vip, &lb_vip->vip_port, - &addr_family)) { - return false; - } - - /* Format for backend ips: "IP1:port1,IP2:port2,...". */ - size_t n_backends = 0; + struct ds errors = DS_EMPTY_INITIALIZER; size_t n_allocated_backends = 0; - char *tokstr = xstrdup(lb_value); + char *tokstr = xstrdup(value); char *save_ptr = NULL; + *n_backends = 0; + for (char *token = strtok_r(tokstr, ",", &save_ptr); token != NULL; token = strtok_r(NULL, ",", &save_ptr)) { - if (n_backends == n_allocated_backends) { + if (*n_backends == n_allocated_backends) { lb_vip->backends = x2nrealloc(lb_vip->backends, &n_allocated_backends, sizeof *lb_vip->backends); } - struct ovn_lb_backend *backend = &lb_vip->backends[n_backends]; + struct ovn_lb_backend *backend = &lb_vip->backends[(*n_backends)]; int backend_addr_family; if (!ip_address_and_port_from_lb_key(token, &backend->ip_str, &backend->ip, &backend->port, &backend_addr_family)) { + if (lb_vip->port_str) { + ds_put_format(&errors, "%s: should be an IP address and a " + "port number with : as a separator, ", + token); + } else { + ds_put_format(&errors, "%s: should be an IP address, ", token); + } continue; } - if (addr_family != backend_addr_family) { + if (lb_vip->address_family != backend_addr_family) { free(backend->ip_str); + ds_put_format(&errors, "%s: IP address family is different from " + "VIP %s, ", + token, lb_vip->vip_str); continue; } - n_backends++; + if (lb_vip->port_str) { + if (!backend->port) { + free(backend->ip_str); + ds_put_format(&errors, "%s: should be an IP address and " + "a port number with : as a separator, ", + token); + continue; + } + } else { + if (backend->port) { + free(backend->ip_str); + ds_put_format(&errors, "%s: should be an IP address, ", token); + continue; + } + } + + backend->port_str = + backend->port ? xasprintf("%"PRIu16, backend->port) : NULL; + (*n_backends)++; } free(tokstr); - lb_vip->n_backends = n_backends; - return true; + + if (ds_last(&errors) != EOF) { + ds_chomp(&errors, ' '); + ds_chomp(&errors, ','); + ds_put_char(&errors, '.'); + return ds_steal_cstr(&errors); + } + return NULL; } static -void ovn_lb_vip_destroy(struct ovn_lb_vip *vip) +char *ovn_lb_vip_init_explicit(struct ovn_lb_vip *lb_vip, const char *lb_key, + const char *lb_value) +{ + if (!ip_address_and_port_from_lb_key(lb_key, &lb_vip->vip_str, + &lb_vip->vip, &lb_vip->vip_port, + &lb_vip->address_family)) { + return xasprintf("%s: should be an IP address (or an IP address " + "and a port number with : as a separator).", lb_key); + } + + lb_vip->port_str = lb_vip->vip_port + ? xasprintf("%"PRIu16, lb_vip->vip_port) + : NULL; + + return ovn_lb_backends_init_explicit(lb_vip, lb_value, + &lb_vip->n_backends); +} + +/* Parses backends of a templated LB VIP. + * For now only the following template forms are supported: + * A. + * ^backendip_variable1[:^port_variable1|:port], + * ^backendip_variable2[:^port_variable2|:port] + * + * B. + * ^backends_variable1,^backends_variable2 is also a thing + * where 'backends_variable1' may expand to IP1_1:PORT1_1 on chassis-1 + * IP1_2:PORT1_2 on chassis-2 + * and 'backends_variable2' may expand to IP2_1:PORT2_1 on chassis-1 + * IP2_2:PORT2_2 on chassis-2 + */ +static char * +ovn_lb_backends_init_template(struct ovn_lb_vip *lb_vip, const char *value_, + size_t *n_backends) +{ + struct ds errors = DS_EMPTY_INITIALIZER; + char *value = xstrdup(value_); + char *save_ptr = NULL; + size_t n_allocated_backends = 0; + *n_backends = 0; + + for (char *backend = strtok_r(value, ",", &save_ptr); backend; + backend = strtok_r(NULL, ",", &save_ptr)) { + + char *atom = xstrdup(backend); + char *save_ptr2 = NULL; + bool success = false; + char *backend_ip = NULL; + char *backend_port = NULL; + + for (char *subatom = strtok_r(atom, ":", &save_ptr2); subatom; + subatom = strtok_r(NULL, ":", &save_ptr2)) { + if (backend_ip && backend_port) { + success = false; + break; + } + success = true; + if (!backend_ip) { + backend_ip = xstrdup(subatom); + } else { + backend_port = xstrdup(subatom); + } + } + + if (success) { + if (*n_backends == n_allocated_backends) { + lb_vip->backends = x2nrealloc(lb_vip->backends, + &n_allocated_backends, + sizeof *lb_vip->backends); + } + + struct ovn_lb_backend *lb_backend = + &lb_vip->backends[(*n_backends)]; + lb_backend->ip_str = backend_ip; + lb_backend->port_str = backend_port; + lb_backend->port = 0; + (*n_backends)++; + } else { + ds_put_format(&errors, "%s: should be a template of the form: " + "'^backendip_variable1[:^port_variable1|:port]', ", + atom); + } + free(atom); + } + + free(value); + if (ds_last(&errors) != EOF) { + ds_chomp(&errors, ' '); + ds_chomp(&errors, ','); + ds_put_char(&errors, '.'); + return ds_steal_cstr(&errors); + } + return NULL; +} + +/* Parses a VIP of a templated LB. + * For now only the following template forms are supported: + * ^vip_variable[:^port_variable|:port] + */ +static char * +ovn_lb_vip_init_template(struct ovn_lb_vip *lb_vip, const char *lb_key_, + const char *lb_value, int address_family) +{ + char *save_ptr = NULL; + char *lb_key = xstrdup(lb_key_); + bool success = false; + + for (char *atom = strtok_r(lb_key, ":", &save_ptr); atom; + atom = strtok_r(NULL, ":", &save_ptr)) { + if (lb_vip->vip_str && lb_vip->port_str) { + success = false; + break; + } + success = true; + if (!lb_vip->vip_str) { + lb_vip->vip_str = xstrdup(atom); + } else { + lb_vip->port_str = xstrdup(atom); + } + } + free(lb_key); + + if (!success) { + return xasprintf("%s: should be a template of the form: " + "'^vip_variable[:^port_variable|:port]'.", + lb_key_); + } + + lb_vip->address_family = address_family; + return ovn_lb_backends_init_template(lb_vip, lb_value, + &lb_vip->n_backends); +} + +/* Returns NULL on success, an error string on failure. The caller is + * responsible for destroying 'lb_vip' in all cases. + */ +char * +ovn_lb_vip_init(struct ovn_lb_vip *lb_vip, const char *lb_key, + const char *lb_value, bool template, int address_family) +{ + memset(lb_vip, 0, sizeof *lb_vip); + + return !template + ? ovn_lb_vip_init_explicit(lb_vip, lb_key, lb_value) + : ovn_lb_vip_init_template(lb_vip, lb_key, lb_value, + address_family); +} + +void +ovn_lb_vip_destroy(struct ovn_lb_vip *vip) { free(vip->vip_str); + free(vip->port_str); for (size_t i = 0; i < vip->n_backends; i++) { free(vip->backends[i].ip_str); + free(vip->backends[i].port_str); } free(vip->backends); } +void +ovn_lb_vip_format(const struct ovn_lb_vip *vip, struct ds *s, bool template) +{ + bool needs_brackets = vip->address_family == AF_INET6 && vip->port_str + && !template; + if (needs_brackets) { + ds_put_char(s, '['); + } + ds_put_cstr(s, vip->vip_str); + if (needs_brackets) { + ds_put_char(s, ']'); + } + if (vip->port_str) { + ds_put_format(s, ":%s", vip->port_str); + } +} + +void +ovn_lb_vip_backends_format(const struct ovn_lb_vip *vip, struct ds *s, + bool template) +{ + bool needs_brackets = vip->address_family == AF_INET6 && vip->port_str + && !template; + for (size_t i = 0; i < vip->n_backends; i++) { + struct ovn_lb_backend *backend = &vip->backends[i]; + + if (needs_brackets) { + ds_put_char(s, '['); + } + ds_put_cstr(s, backend->ip_str); + if (needs_brackets) { + ds_put_char(s, ']'); + } + if (backend->port_str) { + ds_put_format(s, ":%s", backend->port_str); + } + if (i != vip->n_backends - 1) { + ds_put_char(s, ','); + } + } +} + static void ovn_northd_lb_vip_init(struct ovn_northd_lb_vip *lb_vip_nb, const struct ovn_lb_vip *lb_vip, const struct nbrec_load_balancer *nbrec_lb, - const char *vip_port_str, const char *backend_ips) + const char *vip_port_str, const char *backend_ips, + bool template) { lb_vip_nb->backend_ips = xstrdup(backend_ips); lb_vip_nb->n_backends = lb_vip->n_backends; lb_vip_nb->backends_nb = xcalloc(lb_vip_nb->n_backends, sizeof *lb_vip_nb->backends_nb); - - struct nbrec_load_balancer_health_check *lb_health_check = NULL; - if (nbrec_lb->protocol && !strcmp(nbrec_lb->protocol, "sctp")) { - if (nbrec_lb->n_health_check > 0) { - static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 1); - VLOG_WARN_RL(&rl, - "SCTP load balancers do not currently support " - "health checks. Not creating health checks for " - "load balancer " UUID_FMT, - UUID_ARGS(&nbrec_lb->header_.uuid)); - } - } else { - for (size_t j = 0; j < nbrec_lb->n_health_check; j++) { - if (!strcmp(nbrec_lb->health_check[j]->vip, vip_port_str)) { - lb_health_check = nbrec_lb->health_check[j]; - break; - } - } - } - - lb_vip_nb->lb_health_check = lb_health_check; + lb_vip_nb->lb_health_check = + ovn_lb_get_health_check(nbrec_lb, vip_port_str, template); } static @@ -189,12 +403,112 @@ ovn_lb_get_hairpin_snat_ip(const struct uuid *lb_uuid, } } +static bool +ovn_lb_get_routable_mode(const struct nbrec_load_balancer *nbrec_lb, + bool routable, bool template) +{ + if (template && routable) { + static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 1); + VLOG_WARN_RL(&rl, "Template load balancer "UUID_FMT" does not suport " + "option 'add_route'. Forcing it to disabled.", + UUID_ARGS(&nbrec_lb->header_.uuid)); + return false; + } + return routable; +} + +static bool +ovn_lb_neigh_mode_is_valid(enum lb_neighbor_responder_mode mode, bool template) +{ + if (!template) { + return true; + } + + switch (mode) { + case LB_NEIGH_RESPOND_REACHABLE: + return false; + case LB_NEIGH_RESPOND_ALL: + case LB_NEIGH_RESPOND_NONE: + return true; + } + return false; +} + +static enum lb_neighbor_responder_mode +ovn_lb_get_neigh_mode(const struct nbrec_load_balancer *nbrec_lb, + const char *mode, bool template) +{ + static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 1); + enum lb_neighbor_responder_mode default_mode = + template ? LB_NEIGH_RESPOND_NONE : LB_NEIGH_RESPOND_REACHABLE; + + if (!mode) { + mode = lb_neighbor_responder_mode_names[default_mode]; + } + + for (size_t i = 0; i < ARRAY_SIZE(lb_neighbor_responder_mode_names); i++) { + if (!strcmp(mode, lb_neighbor_responder_mode_names[i])) { + if (ovn_lb_neigh_mode_is_valid(i, template)) { + return i; + } + break; + } + } + + VLOG_WARN_RL(&rl, "Invalid neighbor responder mode %s for load balancer " + UUID_FMT", forcing it to %s", + mode, UUID_ARGS(&nbrec_lb->header_.uuid), + lb_neighbor_responder_mode_names[default_mode]); + return default_mode; +} + +static struct nbrec_load_balancer_health_check * +ovn_lb_get_health_check(const struct nbrec_load_balancer *nbrec_lb, + const char *vip_port_str, bool template) +{ + static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 1); + + if (!nbrec_lb->n_health_check) { + return NULL; + } + + if (nbrec_lb->protocol && !strcmp(nbrec_lb->protocol, "sctp")) { + VLOG_WARN_RL(&rl, + "SCTP load balancers do not currently support " + "health checks. Not creating health checks for " + "load balancer " UUID_FMT, + UUID_ARGS(&nbrec_lb->header_.uuid)); + return NULL; + } + + if (template) { + VLOG_WARN_RL(&rl, + "Template load balancers do not currently support " + "health checks. Not creating health checks for " + "load balancer " UUID_FMT, + UUID_ARGS(&nbrec_lb->header_.uuid)); + return NULL; + } + + for (size_t i = 0; i < nbrec_lb->n_health_check; i++) { + if (!strcmp(nbrec_lb->health_check[i]->vip, vip_port_str)) { + return nbrec_lb->health_check[i]; + } + } + return NULL; +} + struct ovn_northd_lb * ovn_northd_lb_create(const struct nbrec_load_balancer *nbrec_lb) { bool is_udp = nullable_string_is_equal(nbrec_lb->protocol, "udp"); bool is_sctp = nullable_string_is_equal(nbrec_lb->protocol, "sctp"); struct ovn_northd_lb *lb = xzalloc(sizeof *lb); + int address_family = !strcmp(smap_get_def(&nbrec_lb->options, + "address-family", "ipv4"), + "ipv4") + ? AF_INET + : AF_INET6; lb->nlb = nbrec_lb; lb->proto = is_udp ? "udp" : is_sctp ? "sctp" : "tcp"; @@ -202,12 +516,16 @@ ovn_northd_lb_create(const struct nbrec_load_balancer *nbrec_lb) lb->vips = xcalloc(lb->n_vips, sizeof *lb->vips); lb->vips_nb = xcalloc(lb->n_vips, sizeof *lb->vips_nb); lb->controller_event = smap_get_bool(&nbrec_lb->options, "event", false); - lb->routable = smap_get_bool(&nbrec_lb->options, "add_route", false); + + bool routable = smap_get_bool(&nbrec_lb->options, "add_route", false); + lb->routable = ovn_lb_get_routable_mode(nbrec_lb, routable, lb->template); + lb->skip_snat = smap_get_bool(&nbrec_lb->options, "skip_snat", false); - const char *mode = - smap_get_def(&nbrec_lb->options, "neighbor_responder", "reachable"); - lb->neigh_mode = strcmp(mode, "all") ? LB_NEIGH_RESPOND_REACHABLE - : LB_NEIGH_RESPOND_ALL; + lb->template = smap_get_bool(&nbrec_lb->options, "template", false); + + const char *mode = smap_get(&nbrec_lb->options, "neighbor_responder"); + lb->neigh_mode = ovn_lb_get_neigh_mode(nbrec_lb, mode, lb->template); + sset_init(&lb->ips_v4); sset_init(&lb->ips_v6); struct smap_node *node; @@ -217,13 +535,19 @@ ovn_northd_lb_create(const struct nbrec_load_balancer *nbrec_lb) struct ovn_lb_vip *lb_vip = &lb->vips[n_vips]; struct ovn_northd_lb_vip *lb_vip_nb = &lb->vips_nb[n_vips]; - lb_vip->empty_backend_rej = smap_get_bool(&nbrec_lb->options, - "reject", false); - if (!ovn_lb_vip_init(lb_vip, node->key, node->value)) { + char *error = ovn_lb_vip_init(lb_vip, node->key, node->value, + lb->template, address_family); + if (error) { + static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1); + VLOG_WARN_RL(&rl, "Failed to initialize LB VIP: %s", error); + ovn_lb_vip_destroy(lb_vip); + free(error); continue; } + lb_vip->empty_backend_rej = smap_get_bool(&nbrec_lb->options, + "reject", false); ovn_northd_lb_vip_init(lb_vip_nb, lb_vip, nbrec_lb, - node->key, node->value); + node->key, node->value, lb->template); if (IN6_IS_ADDR_V4MAPPED(&lb_vip->vip)) { sset_add(&lb->ips_v4, lb_vip->vip_str); } else { @@ -371,9 +695,12 @@ ovn_lb_group_find(const struct hmap *lb_groups, const struct uuid *uuid) } struct ovn_controller_lb * -ovn_controller_lb_create(const struct sbrec_load_balancer *sbrec_lb) +ovn_controller_lb_create(const struct sbrec_load_balancer *sbrec_lb, + const struct smap *template_vars, + struct sset *template_vars_ref) { struct ovn_controller_lb *lb = xzalloc(sizeof *lb); + bool template = smap_get_bool(&sbrec_lb->options, "template", false); lb->slb = sbrec_lb; lb->n_vips = smap_count(&sbrec_lb->vips); @@ -385,10 +712,28 @@ ovn_controller_lb_create(const struct sbrec_load_balancer *sbrec_lb) SMAP_FOR_EACH (node, &sbrec_lb->vips) { struct ovn_lb_vip *lb_vip = &lb->vips[n_vips]; - if (!ovn_lb_vip_init(lb_vip, node->key, node->value)) { - continue; + char *key_expanded_s = NULL; + const char *key_s = template + ? lexer_parse_template_string(node->key, + template_vars, + template_vars_ref, + &key_expanded_s) + : node->key; + char *value_expanded_s = NULL; + const char *value_s = template + ? lexer_parse_template_string(node->value, + template_vars, + template_vars_ref, + &value_expanded_s) + : node->value; + char *error = ovn_lb_vip_init_explicit(lb_vip, key_s, value_s); + if (error) { + free(error); + } else { + n_vips++; } - n_vips++; + free(key_expanded_s); + free(value_expanded_s); } /* It's possible that parsing VIPs fails. Update the lb->n_vips to the diff --git a/lib/lb.h b/lib/lb.h index c1aadd6dd5..42bc5afecb 100644 --- a/lib/lb.h +++ b/lib/lb.h @@ -35,6 +35,7 @@ struct uuid; enum lb_neighbor_responder_mode { LB_NEIGH_RESPOND_REACHABLE, LB_NEIGH_RESPOND_ALL, + LB_NEIGH_RESPOND_NONE, }; /* The "routable" ssets are subsets of the load balancer IPs for which IP @@ -67,6 +68,7 @@ struct ovn_northd_lb { bool controller_event; bool routable; bool skip_snat; + bool template; struct sset ips_v4; struct sset ips_v6; @@ -81,19 +83,31 @@ struct ovn_northd_lb { }; struct ovn_lb_vip { - struct in6_addr vip; - char *vip_str; - uint16_t vip_port; - + struct in6_addr vip; /* Only used in ovn-controller. */ + char *vip_str; /* Actual VIP string representation (without port). + * To be used in ovn-northd. + */ + uint16_t vip_port; /* Only used in ovn-controller. */ + char *port_str; /* Actual port string representation. To be used + * in ovn-controller. + */ struct ovn_lb_backend *backends; size_t n_backends; bool empty_backend_rej; + int address_family; }; struct ovn_lb_backend { - struct in6_addr ip; - char *ip_str; - uint16_t port; + struct in6_addr ip; /* Only used in ovn-controller. */ + char *ip_str; /* Actual IP string representation. To be used in + * ovn-northd. + */ + uint16_t port; /* Mostly used in ovn-controller but also for + * healthcheck in ovn-northd. + */ + char *port_str; /* Actual port string representation. To be used + * in ovn-northd. + */ }; /* ovn-northd specific backend information. */ @@ -173,7 +187,17 @@ struct ovn_controller_lb { }; struct ovn_controller_lb *ovn_controller_lb_create( - const struct sbrec_load_balancer *); + const struct sbrec_load_balancer *, + const struct smap *template_vars, + struct sset *template_vars_ref); void ovn_controller_lb_destroy(struct ovn_controller_lb *); +char *ovn_lb_vip_init(struct ovn_lb_vip *lb_vip, const char *lb_key, + const char *lb_value, bool template, int address_family); +void ovn_lb_vip_destroy(struct ovn_lb_vip *vip); +void ovn_lb_vip_format(const struct ovn_lb_vip *vip, struct ds *s, + bool template); +void ovn_lb_vip_backends_format(const struct ovn_lb_vip *vip, struct ds *s, + bool template); + #endif /* OVN_LIB_LB_H 1 */ diff --git a/lib/ovn-util.c b/lib/ovn-util.c index 5dca727146..91b572feb3 100644 --- a/lib/ovn-util.c +++ b/lib/ovn-util.c @@ -793,9 +793,6 @@ ip_address_and_port_from_lb_key(const char *key, char **ip_address, { struct sockaddr_storage ss; if (!inet_parse_active(key, 0, &ss, false, NULL)) { - static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1); - VLOG_WARN_RL(&rl, "bad ip address or port for load balancer key %s", - key); *ip_address = NULL; memset(ip, 0, sizeof(*ip)); *port = 0; diff --git a/northd/northd.c b/northd/northd.c index 170b4f95c8..e12b91a3a6 100644 --- a/northd/northd.c +++ b/northd/northd.c @@ -3720,6 +3720,10 @@ static void ovn_lb_svc_create(struct ovsdb_idl_txn *ovnsb_txn, struct ovn_northd_lb *lb, struct hmap *monitor_map, struct hmap *ports) { + if (lb->template) { + return; + } + for (size_t i = 0; i < lb->n_vips; i++) { struct ovn_lb_vip *lb_vip = &lb->vips[i]; struct ovn_northd_lb_vip *lb_vip_nb = &lb->vips_nb[i]; @@ -4036,12 +4040,19 @@ static void build_lrouter_lb_reachable_ips(struct ovn_datapath *od, const struct ovn_northd_lb *lb) { + /* If configured to not reply to any neighbor requests for all VIPs + * return early. + */ + if (lb->neigh_mode == LB_NEIGH_RESPOND_NONE) { + return; + } + /* If configured to reply to neighbor requests for all VIPs force them * all to be considered "reachable". */ if (lb->neigh_mode == LB_NEIGH_RESPOND_ALL) { for (size_t i = 0; i < lb->n_vips; i++) { - if (IN6_IS_ADDR_V4MAPPED(&lb->vips[i].vip)) { + if (lb->vips[i].address_family == AF_INET) { sset_add(&od->lb_ips->ips_v4_reachable, lb->vips[i].vip_str); } else { sset_add(&od->lb_ips->ips_v6_reachable, lb->vips[i].vip_str); @@ -4053,8 +4064,9 @@ build_lrouter_lb_reachable_ips(struct ovn_datapath *od, /* Otherwise, a VIP is reachable if there's at least one router * subnet that includes it. */ + ovs_assert(lb->neigh_mode == LB_NEIGH_RESPOND_REACHABLE); for (size_t i = 0; i < lb->n_vips; i++) { - if (IN6_IS_ADDR_V4MAPPED(&lb->vips[i].vip)) { + if (lb->vips[i].address_family == AF_INET) { ovs_be32 vip_ip4 = in6_addr_get_mapped_ipv4(&lb->vips[i].vip); struct ovn_port *op; @@ -5814,16 +5826,16 @@ build_empty_lb_event_flow(struct ovn_lb_vip *lb_vip, ds_clear(action); ds_clear(match); - bool ipv4 = IN6_IS_ADDR_V4MAPPED(&lb_vip->vip); + bool ipv4 = lb_vip->address_family == AF_INET; ds_put_format(match, "ip%s.dst == %s && %s", ipv4 ? "4": "6", lb_vip->vip_str, lb->proto); char *vip = lb_vip->vip_str; - if (lb_vip->vip_port) { - ds_put_format(match, " && %s.dst == %u", lb->proto, lb_vip->vip_port); - vip = xasprintf("%s%s%s:%u", ipv4 ? "" : "[", lb_vip->vip_str, - ipv4 ? "" : "]", lb_vip->vip_port); + if (lb_vip->port_str) { + ds_put_format(match, " && %s.dst == %s", lb->proto, lb_vip->port_str); + vip = xasprintf("%s%s%s:%s", ipv4 ? "" : "[", lb_vip->vip_str, + ipv4 ? "" : "]", lb_vip->port_str); } ds_put_format(action, @@ -5834,7 +5846,7 @@ build_empty_lb_event_flow(struct ovn_lb_vip *lb_vip, event_to_string(OVN_EVENT_EMPTY_LB_BACKENDS), vip, lb->proto, UUID_ARGS(&lb->nlb->header_.uuid)); - if (lb_vip->vip_port) { + if (lb_vip->port_str) { free(vip); } return true; @@ -6890,7 +6902,7 @@ build_lb_rules_pre_stateful(struct hmap *lflows, struct ovn_northd_lb *lb, /* Store the original destination IP to be used when generating * hairpin flows. */ - if (IN6_IS_ADDR_V4MAPPED(&lb_vip->vip)) { + if (lb->vips[i].address_family == AF_INET) { ip_match = "ip4"; ds_put_format(action, REG_ORIG_DIP_IPV4 " = %s; ", lb_vip->vip_str); @@ -6901,7 +6913,7 @@ build_lb_rules_pre_stateful(struct hmap *lflows, struct ovn_northd_lb *lb, } const char *proto = NULL; - if (lb_vip->vip_port) { + if (lb_vip->port_str) { proto = "tcp"; if (lb->nlb->protocol) { if (!strcmp(lb->nlb->protocol, "udp")) { @@ -6914,14 +6926,14 @@ build_lb_rules_pre_stateful(struct hmap *lflows, struct ovn_northd_lb *lb, /* Store the original destination port to be used when generating * hairpin flows. */ - ds_put_format(action, REG_ORIG_TP_DPORT " = %"PRIu16"; ", - lb_vip->vip_port); + ds_put_format(action, REG_ORIG_TP_DPORT " = %s; ", + lb_vip->port_str); } ds_put_format(action, "%s;", ct_lb_mark ? "ct_lb_mark" : "ct_lb"); ds_put_format(match, "%s.dst == %s", ip_match, lb_vip->vip_str); - if (lb_vip->vip_port) { - ds_put_format(match, " && %s.dst == %d", proto, lb_vip->vip_port); + if (lb_vip->port_str) { + ds_put_format(match, " && %s.dst == %s", proto, lb_vip->port_str); } struct ovn_lflow *lflow_ref = NULL; @@ -6953,24 +6965,12 @@ build_lb_rules(struct hmap *lflows, struct ovn_northd_lb *lb, bool ct_lb_mark, struct ovn_lb_vip *lb_vip = &lb->vips[i]; struct ovn_northd_lb_vip *lb_vip_nb = &lb->vips_nb[i]; const char *ip_match = NULL; - if (IN6_IS_ADDR_V4MAPPED(&lb_vip->vip)) { + if (lb_vip->address_family == AF_INET) { ip_match = "ip4"; } else { ip_match = "ip6"; } - const char *proto = NULL; - if (lb_vip->vip_port) { - proto = "tcp"; - if (lb->nlb->protocol) { - if (!strcmp(lb->nlb->protocol, "udp")) { - proto = "udp"; - } else if (!strcmp(lb->nlb->protocol, "sctp")) { - proto = "sctp"; - } - } - } - ds_clear(action); ds_clear(match); @@ -6988,8 +6988,9 @@ build_lb_rules(struct hmap *lflows, struct ovn_northd_lb *lb, bool ct_lb_mark, ds_put_format(match, "ct.new && %s.dst == %s", ip_match, lb_vip->vip_str); int priority = 110; - if (lb_vip->vip_port) { - ds_put_format(match, " && %s.dst == %d", proto, lb_vip->vip_port); + if (lb_vip->port_str) { + ds_put_format(match, " && %s.dst == %s", lb->proto, + lb_vip->port_str); priority = 120; } @@ -9989,7 +9990,7 @@ build_lrouter_nat_flows_for_lb(struct ovn_lb_vip *lb_vip, * of "ct_lb_mark($targets);". The other flow is for ct.est with * an action of "next;". */ - if (IN6_IS_ADDR_V4MAPPED(&lb_vip->vip)) { + if (lb_vip->address_family == AF_INET) { ds_put_format(match, "ip4 && "REG_NEXT_HOP_IPV4" == %s", lb_vip->vip_str); } else { @@ -10005,14 +10006,14 @@ build_lrouter_nat_flows_for_lb(struct ovn_lb_vip *lb_vip, } int prio = 110; - if (lb_vip->vip_port) { + if (lb_vip->port_str) { prio = 120; new_match = xasprintf("ct.new && %s && %s && " - REG_ORIG_TP_DPORT_ROUTER" == %d", - ds_cstr(match), lb->proto, lb_vip->vip_port); + REG_ORIG_TP_DPORT_ROUTER" == %s", + ds_cstr(match), lb->proto, lb_vip->port_str); est_match = xasprintf("ct.est && %s && %s && " - REG_ORIG_TP_DPORT_ROUTER" == %d && %s == 1", - ds_cstr(match), lb->proto, lb_vip->vip_port, + REG_ORIG_TP_DPORT_ROUTER" == %s && %s == 1", + ds_cstr(match), lb->proto, lb_vip->port_str, ct_natted); } else { new_match = xasprintf("ct.new && %s", ds_cstr(match)); @@ -10021,7 +10022,7 @@ build_lrouter_nat_flows_for_lb(struct ovn_lb_vip *lb_vip, } const char *ip_match = NULL; - if (IN6_IS_ADDR_V4MAPPED(&lb_vip->vip)) { + if (lb_vip->address_family == AF_INET) { ip_match = "ip4"; } else { ip_match = "ip6"; @@ -10039,9 +10040,9 @@ build_lrouter_nat_flows_for_lb(struct ovn_lb_vip *lb_vip, ds_put_format(&undnat_match, "(%s.src == %s", ip_match, backend->ip_str); - if (backend->port) { - ds_put_format(&undnat_match, " && %s.src == %d) || ", - lb->proto, backend->port); + if (backend->port_str) { + ds_put_format(&undnat_match, " && %s.src == %s) || ", + lb->proto, backend->port_str); } else { ds_put_cstr(&undnat_match, ") || "); } @@ -10054,9 +10055,9 @@ build_lrouter_nat_flows_for_lb(struct ovn_lb_vip *lb_vip, struct ds unsnat_match = DS_EMPTY_INITIALIZER; ds_put_format(&unsnat_match, "%s && %s.dst == %s && %s", ip_match, ip_match, lb_vip->vip_str, lb->proto); - if (lb_vip->vip_port) { - ds_put_format(&unsnat_match, " && %s.dst == %d", lb->proto, - lb_vip->vip_port); + if (lb_vip->port_str) { + ds_put_format(&unsnat_match, " && %s.dst == %s", lb->proto, + lb_vip->port_str); } struct ovn_datapath **gw_router_skip_snat = @@ -10296,7 +10297,7 @@ build_lrouter_defrag_flows_for_lb(struct ovn_northd_lb *lb, ds_clear(&defrag_actions); ds_clear(match); - if (IN6_IS_ADDR_V4MAPPED(&lb_vip->vip)) { + if (lb_vip->address_family == AF_INET) { ds_put_format(match, "ip && ip4.dst == %s", lb_vip->vip_str); ds_put_format(&defrag_actions, REG_NEXT_HOP_IPV4" = %s; ", lb_vip->vip_str); @@ -10306,7 +10307,7 @@ build_lrouter_defrag_flows_for_lb(struct ovn_northd_lb *lb, lb_vip->vip_str); } - if (lb_vip->vip_port) { + if (lb_vip->port_str) { ds_put_format(match, " && %s", lb->proto); prio = 110; diff --git a/ovn-nb.xml b/ovn-nb.xml index 45b75e66df..af3ac261bc 100644 --- a/ovn-nb.xml +++ b/ovn-nb.xml @@ -1905,8 +1905,57 @@ is applied reply to ARP/neighbor discovery requests for all VIPs of the load balancer. If set to reachable, then routers on which the load balancer is applied reply to ARP/neighbor discovery - requests only for VIPs that are part of a router's subnet. The default - value of this option, if not specified, is reachable. + requests only for VIPs that are part of a router's subnet. If set to + none, then routers on which the load balancer is applied + never reply to ARP/neighbor discovery requests for any of the load + balancer VIPs. Load balancers with options:template=true + do not support reachable as a valid mode. The default + value of this option, if not specified, is reachable for + regular load balancers and none for template load + balancers. + + + +

+ Option to be set to true, if the load balancer is a + template. In this the load balancer VIPs and/or backends may be + using in their definition. +

+ +

+ Load balancer template VIP supported formats are: +

+ 
+^VIP_VAR[:^PORT_VAR|:port]
+
+ +

+ where VIP_VAR and PORT_VAR are names of + records. +

+ +

+ Load balancer template backend supported formats are: +

+ 
+^BACKEND_VAR1[:^PORT_VAR1|:port],^BACKEND_VAR2[:^PORT_VAR2|:port]
+
+or
+
+^BACKENDS_VAR1,^BACKENDS_VAR2
+
+

+ where BACKEND_VAR1, PORT_VAR1, + BACKEND_VAR2, PORT_VAR2, + BACKENDS_VAR1 and BACKENDS_VAR2 are names + of records. +

+ + + + Address family used by the load balancer. Supported values are + ipv4 and ipv6. This value is used and is + mandatory for load balancer with options:template=true. diff --git a/tests/ovn-nbctl.at b/tests/ovn-nbctl.at index 4d480e3573..9da7c26b31 100644 --- a/tests/ovn-nbctl.at +++ b/tests/ovn-nbctl.at @@ -857,23 +857,19 @@ AT_CHECK([ovn-nbctl -vsocket_util:off lb-add lb0 30.0.0.10 192.168.10.10:a80], [ [ovn-nbctl: 192.168.10.10:a80: should be an IP address. ]) -AT_CHECK([ovn-nbctl -vsocket_util:off lb-add lb0 30.0.0.10 192.168.10.10:], [1], [], -[ovn-nbctl: 192.168.10.10:: should be an IP address. -]) - AT_CHECK([ovn-nbctl -vsocket_util:off lb-add lb0 30.0.0.10 192.168.10.1a], [1], [], [ovn-nbctl: 192.168.10.1a: should be an IP address. ]) AT_CHECK([ovn-nbctl lb-add lb0 30.0.0.10: 192.168.10.10:80,192.168.10.20:80 tcp], [1], [], -[ovn-nbctl: Protocol is unnecessary when no port of vip is given. +[ovn-nbctl: 192.168.10.10:80: should be an IP address, 192.168.10.20:80: should be an IP address. ]) AT_CHECK([ovn-nbctl lb-add lb0 30.0.0.10 192.168.10.10 tcp], [1], [], [ovn-nbctl: Protocol is unnecessary when no port of vip is given. ]) -AT_CHECK([ovn-nbctl lb-add lb0 30.0.0.10 192.168.10.10:900 tcp], [1], [], +AT_CHECK([ovn-nbctl lb-add lb0 30.0.0.10 192.168.10.10 tcp], [1], [], [ovn-nbctl: Protocol is unnecessary when no port of vip is given. ]) @@ -1111,7 +1107,7 @@ AT_CHECK([ovn-nbctl -vsocket_util:off lb-add lb0 ae0f::10fff [[fd0f::10]]:80,fd0 AT_CHECK([ovn-nbctl -vsocket_util:off lb-add lb0 ae0f::10 [[fd0f::10]]:80,[[fd0f::20]]:80], [1], [], -[ovn-nbctl: [[fd0f::10]]:80: should be an IP address. +[ovn-nbctl: [[fd0f::10]]:80: should be an IP address, [[fd0f::20]]:80: should be an IP address. ]) @@ -1125,18 +1121,13 @@ AT_CHECK([ovn-nbctl -vsocket_util:off lb-add lb0 ae0f::10 [[fd0f::10]]:a80], [1] ]) -AT_CHECK([ovn-nbctl -vsocket_util:off lb-add lb0 ae0f::10 [[fd0f::10]]:], [1], [], -[ovn-nbctl: [[fd0f::10]]:: should be an IP address. -]) - - AT_CHECK([ovn-nbctl -vsocket_util:off lb-add lb0 ae0f::10 fd0f::1001a], [1], [], [ovn-nbctl: fd0f::1001a: should be an IP address. ]) AT_CHECK([ovn-nbctl -vsocket_util:off lb-add lb0 [[ae0f::10]]: [[fd0f::10]]:80,[[fd0f::20]]:80 tcp], [1], [], -[ovn-nbctl: Protocol is unnecessary when no port of vip is given. +[ovn-nbctl: [[fd0f::10]]:80: should be an IP address, [[fd0f::20]]:80: should be an IP address. ]) @@ -1146,7 +1137,7 @@ AT_CHECK([ovn-nbctl lb-add lb0 ae0f::10 fd0f::10 tcp], [1], [], AT_CHECK([ovn-nbctl lb-add lb0 ae0f::10 [[fd0f::10]]:900 tcp], [1], [], -[ovn-nbctl: Protocol is unnecessary when no port of vip is given. +[ovn-nbctl: [[fd0f::10]]:900: should be an IP address. ]) AT_CHECK([ovn-nbctl lb-add lb0 ae0f::10 192.168.10.10], [1], [], @@ -1158,7 +1149,7 @@ AT_CHECK([ovn-nbctl lb-add lb0 ae0f::10 192.168.10.10], [1], [], ]) AT_CHECK([ovn-nbctl lb-add lb0 [[ae0f::10]]:80 192.168.10.10:80], [1], [], -[ovn-nbctl: 192.168.10.10:80: IP address family is different from VIP [[ae0f::10]]:80. +[ovn-nbctl: 192.168.10.10:80: IP address family is different from VIP ae0f::10. ]) AT_CHECK([ovn-nbctl lb-add lb0 30.0.0.10 ae0f::10], [1], [], @@ -1166,7 +1157,7 @@ AT_CHECK([ovn-nbctl lb-add lb0 30.0.0.10 ae0f::10], [1], [], ]) AT_CHECK([ovn-nbctl lb-add lb0 30.0.0.10:80 [[ae0f::10]]:80], [1], [], -[ovn-nbctl: [[ae0f::10]]:80: IP address family is different from VIP 30.0.0.10:80. +[ovn-nbctl: [[ae0f::10]]:80: IP address family is different from VIP 30.0.0.10. ]) AT_CHECK([ovn-nbctl lb-add lb0 ae0f::10 fd0f::10]) diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at index c7112b805d..d088961819 100644 --- a/tests/ovn-northd.at +++ b/tests/ovn-northd.at @@ -1828,6 +1828,11 @@ ovn-nbctl set Load_Balancer lb8 options:neighbor_responder=all ovn-nbctl lb-add lb9 "[[4444::4444]]:8080" "[[10::10]]:8080" udp ovn-nbctl set Load_Balancer lb9 options:neighbor_responder=all +ovn-nbctl lb-add lb10 "55.55.55.55:8080" "10.0.0.8:8080" udp +ovn-nbctl set Load_Balancer lb10 options:neighbor_responder=none +ovn-nbctl lb-add lb11 "[[5555::5555]]:8080" "[[10::10]]:8080" udp +ovn-nbctl set Load_Balancer lb11 options:neighbor_responder=none + ovn-nbctl lr-lb-add lr lb1 ovn-nbctl lr-lb-add lr lb2 ovn-nbctl lr-lb-add lr lb3 @@ -1837,6 +1842,8 @@ ovn-nbctl lr-lb-add lr lb6 ovn-nbctl lr-lb-add lr lb7 ovn-nbctl lr-lb-add lr lb8 ovn-nbctl lr-lb-add lr lb9 +ovn-nbctl lr-lb-add lr lb10 +ovn-nbctl lr-lb-add lr lb11 ovn-nbctl --wait=sb sync lr_key=$(fetch_column sb:datapath_binding tunnel_key external_ids:name=lr) diff --git a/tests/ovn.at b/tests/ovn.at index 2a59235c59..45241d7e20 100644 --- a/tests/ovn.at +++ b/tests/ovn.at @@ -33053,3 +33053,134 @@ AT_CHECK([ovs-ofctl dump-flows br-int | grep '42\.42\.42\.42'], [1], []) OVN_CLEANUP([hv1]) AT_CLEANUP ]) + +OVN_FOR_EACH_NORTHD([ +AT_SETUP([Load balancers with Chassis_Template_Var references]) +AT_KEYWORDS([templates]) +ovn_start +net_add n1 + +sim_add hv1 +as hv1 +ovs-vsctl add-br br-phys +ovn_attach n1 br-phys 192.168.0.1 + +check ovn-nbctl ls-add sw + +dnl Use --wait=sb to ensure lsp1 getting a tunnel_key before lsp2. +check ovn-nbctl --wait=sb lsp-add sw lsp1 +check ovn-nbctl --wait=sb lsp-add sw lsp2 + +AT_CHECK([ovn-nbctl create Chassis_Template_Var chassis=hv1], [0], [ignore]) + +dnl Create a few LBs that use "uninstantiated" templates. +check ovn-nbctl --template lb-add lb-test1 "^VIP1:^VPORT1" "^BACKENDS1" tcp +check ovn-nbctl --template lb-add lb-test2 "^VIP2:^VPORT2" "^BACKENDS21,^BACKENDS22" tcp +check ovn-nbctl --template lb-add lb-test3 "^VIP3:^VPORT3" "^BACKENDS31:^BPORT1,^BACKENDS32:^BPORT2" tcp +check ovn-nbctl ls-lb-add sw lb-test1 +check ovn-nbctl ls-lb-add sw lb-test2 +check ovn-nbctl ls-lb-add sw lb-test3 + +check ovs-vsctl add-port br-int p1 -- set interface p1 external_ids:iface-id=lsp1 +check ovs-vsctl add-port br-int p2 -- set interface p2 external_ids:iface-id=lsp2 + +wait_for_ports_up +ovn-nbctl --wait=hv sync + +dnl Ensure the LBs are not translated to OpenFlow. +as hv1 +AT_CHECK([ovs-ofctl dump-groups br-int | grep 'nat'], [1], []) + +dnl Create Chassis_Template_Var mappings. +check ovn-nbctl --wait=hv set Chassis_Template_Var hv1 \ + variables:VIP1='43.43.43.1' variables:VPORT1='4301' \ + variables:BACKENDS1='85.85.85.1:8501' \ + variables:VIP2='43.43.43.2' variables:VPORT2='4302' \ + variables:BACKENDS21='85.85.85.21:8502' \ + variables:BACKENDS22='85.85.85.22:8502' \ + variables:VIP3='43.43.43.3' variables:VPORT3='4303' \ + variables:BACKENDS31='85.85.85.31' \ + variables:BACKENDS32='85.85.85.32' \ + variables:BPORT1='8503' variables:BPORT2='8503' + +dnl Ensure the LBs are translated to OpenFlow. +as hv1 +AT_CHECK([ovs-ofctl dump-groups br-int | grep 'nat(dst=85.85.85.1:8501)' -c], [0], [dnl +1 +]) +AT_CHECK([ovs-ofctl dump-groups br-int | grep 'nat(dst=85.85.85.21:8502)' -c], [0], [dnl +1 +]) +AT_CHECK([ovs-ofctl dump-groups br-int | grep 'nat(dst=85.85.85.22:8502)' -c], [0], [dnl +1 +]) +AT_CHECK([ovs-ofctl dump-groups br-int | grep 'nat(dst=85.85.85.31:8503)' -c], [0], [dnl +1 +]) +AT_CHECK([ovs-ofctl dump-groups br-int | grep 'nat(dst=85.85.85.32:8503)' -c], [0], [dnl +1 +]) + +dnl Ensure hairpin flows are correct. +as hv1 +AT_CHECK([ovs-ofctl dump-flows br-int | grep table=68 | ofctl_strip_all], [0], [dnl + table=68, priority=100,ct_mark=0x2/0x2,tcp,reg1=0x2b2b2b01,reg2=0x10cd/0xffff,nw_src=85.85.85.1,nw_dst=85.85.85.1,tp_dst=8501 actions=load:0x1->NXM_NX_REG10[[7]],learn(table=69,delete_learned,OXM_OF_METADATA[[]],eth_type=0x800,NXM_OF_IP_SRC[[]],ip_dst=43.43.43.1,nw_proto=6,NXM_OF_TCP_SRC[[]]=NXM_OF_TCP_DST[[]],load:0x1->NXM_NX_REG10[[7]]) + table=68, priority=100,ct_mark=0x2/0x2,tcp,reg1=0x2b2b2b02,reg2=0x10ce/0xffff,nw_src=85.85.85.21,nw_dst=85.85.85.21,tp_dst=8502 actions=load:0x1->NXM_NX_REG10[[7]],learn(table=69,delete_learned,OXM_OF_METADATA[[]],eth_type=0x800,NXM_OF_IP_SRC[[]],ip_dst=43.43.43.2,nw_proto=6,NXM_OF_TCP_SRC[[]]=NXM_OF_TCP_DST[[]],load:0x1->NXM_NX_REG10[[7]]) + table=68, priority=100,ct_mark=0x2/0x2,tcp,reg1=0x2b2b2b02,reg2=0x10ce/0xffff,nw_src=85.85.85.22,nw_dst=85.85.85.22,tp_dst=8502 actions=load:0x1->NXM_NX_REG10[[7]],learn(table=69,delete_learned,OXM_OF_METADATA[[]],eth_type=0x800,NXM_OF_IP_SRC[[]],ip_dst=43.43.43.2,nw_proto=6,NXM_OF_TCP_SRC[[]]=NXM_OF_TCP_DST[[]],load:0x1->NXM_NX_REG10[[7]]) + table=68, priority=100,ct_mark=0x2/0x2,tcp,reg1=0x2b2b2b03,reg2=0x10cf/0xffff,nw_src=85.85.85.31,nw_dst=85.85.85.31,tp_dst=8503 actions=load:0x1->NXM_NX_REG10[[7]],learn(table=69,delete_learned,OXM_OF_METADATA[[]],eth_type=0x800,NXM_OF_IP_SRC[[]],ip_dst=43.43.43.3,nw_proto=6,NXM_OF_TCP_SRC[[]]=NXM_OF_TCP_DST[[]],load:0x1->NXM_NX_REG10[[7]]) + table=68, priority=100,ct_mark=0x2/0x2,tcp,reg1=0x2b2b2b03,reg2=0x10cf/0xffff,nw_src=85.85.85.32,nw_dst=85.85.85.32,tp_dst=8503 actions=load:0x1->NXM_NX_REG10[[7]],learn(table=69,delete_learned,OXM_OF_METADATA[[]],eth_type=0x800,NXM_OF_IP_SRC[[]],ip_dst=43.43.43.3,nw_proto=6,NXM_OF_TCP_SRC[[]]=NXM_OF_TCP_DST[[]],load:0x1->NXM_NX_REG10[[7]]) +]) + +dnl Change Chassis_Template_Var mappings +check ovn-nbctl --wait=hv set Chassis_Template_Var hv1 \ + variables:VIP1='42.42.42.1' variables:VPORT1='4201' \ + variables:BACKENDS1='84.84.84.1:8401' \ + variables:VIP2='42.42.42.2' variables:VPORT2='4202' \ + variables:BACKENDS21='84.84.84.21:8402' \ + variables:BACKENDS22='84.84.84.22:8402' \ + variables:VIP3='42.42.42.3' variables:VPORT3='4203' \ + variables:BACKENDS31='84.84.84.31' \ + variables:BACKENDS32='84.84.84.32' \ + variables:BPORT1='8403' variables:BPORT2='8403' + +dnl Ensure the LBs are translated to OpenFlow. +as hv1 +AT_CHECK([ovs-ofctl dump-groups br-int | grep 'nat(dst=84.84.84.1:8401)' -c], [0], [dnl +1 +]) +AT_CHECK([ovs-ofctl dump-groups br-int | grep 'nat(dst=84.84.84.21:8402)' -c], [0], [dnl +1 +]) +AT_CHECK([ovs-ofctl dump-groups br-int | grep 'nat(dst=84.84.84.22:8402)' -c], [0], [dnl +1 +]) +AT_CHECK([ovs-ofctl dump-groups br-int | grep 'nat(dst=84.84.84.31:8403)' -c], [0], [dnl +1 +]) +AT_CHECK([ovs-ofctl dump-groups br-int | grep 'nat(dst=84.84.84.32:8403)' -c], [0], [dnl +1 +]) + +dnl Ensure hairpin flows are correct. +as hv1 +AT_CHECK([ovs-ofctl dump-flows br-int | grep table=68 | ofctl_strip_all], [0], [dnl + table=68, priority=100,ct_mark=0x2/0x2,tcp,reg1=0x2a2a2a01,reg2=0x1069/0xffff,nw_src=84.84.84.1,nw_dst=84.84.84.1,tp_dst=8401 actions=load:0x1->NXM_NX_REG10[[7]],learn(table=69,delete_learned,OXM_OF_METADATA[[]],eth_type=0x800,NXM_OF_IP_SRC[[]],ip_dst=42.42.42.1,nw_proto=6,NXM_OF_TCP_SRC[[]]=NXM_OF_TCP_DST[[]],load:0x1->NXM_NX_REG10[[7]]) + table=68, priority=100,ct_mark=0x2/0x2,tcp,reg1=0x2a2a2a02,reg2=0x106a/0xffff,nw_src=84.84.84.21,nw_dst=84.84.84.21,tp_dst=8402 actions=load:0x1->NXM_NX_REG10[[7]],learn(table=69,delete_learned,OXM_OF_METADATA[[]],eth_type=0x800,NXM_OF_IP_SRC[[]],ip_dst=42.42.42.2,nw_proto=6,NXM_OF_TCP_SRC[[]]=NXM_OF_TCP_DST[[]],load:0x1->NXM_NX_REG10[[7]]) + table=68, priority=100,ct_mark=0x2/0x2,tcp,reg1=0x2a2a2a02,reg2=0x106a/0xffff,nw_src=84.84.84.22,nw_dst=84.84.84.22,tp_dst=8402 actions=load:0x1->NXM_NX_REG10[[7]],learn(table=69,delete_learned,OXM_OF_METADATA[[]],eth_type=0x800,NXM_OF_IP_SRC[[]],ip_dst=42.42.42.2,nw_proto=6,NXM_OF_TCP_SRC[[]]=NXM_OF_TCP_DST[[]],load:0x1->NXM_NX_REG10[[7]]) + table=68, priority=100,ct_mark=0x2/0x2,tcp,reg1=0x2a2a2a03,reg2=0x106b/0xffff,nw_src=84.84.84.31,nw_dst=84.84.84.31,tp_dst=8403 actions=load:0x1->NXM_NX_REG10[[7]],learn(table=69,delete_learned,OXM_OF_METADATA[[]],eth_type=0x800,NXM_OF_IP_SRC[[]],ip_dst=42.42.42.3,nw_proto=6,NXM_OF_TCP_SRC[[]]=NXM_OF_TCP_DST[[]],load:0x1->NXM_NX_REG10[[7]]) + table=68, priority=100,ct_mark=0x2/0x2,tcp,reg1=0x2a2a2a03,reg2=0x106b/0xffff,nw_src=84.84.84.32,nw_dst=84.84.84.32,tp_dst=8403 actions=load:0x1->NXM_NX_REG10[[7]],learn(table=69,delete_learned,OXM_OF_METADATA[[]],eth_type=0x800,NXM_OF_IP_SRC[[]],ip_dst=42.42.42.3,nw_proto=6,NXM_OF_TCP_SRC[[]]=NXM_OF_TCP_DST[[]],load:0x1->NXM_NX_REG10[[7]]) +]) + +dnl Remove Chassis_Template_Variables and check that everything is +dnl removed from OpenFlow. +check ovn-nbctl --wait=hv clear Chassis_Template_Var hv1 variables + +as hv1 +AT_CHECK([ovs-ofctl dump-groups br-int | grep 'nat'], [1], []) + +as hv1 +AT_CHECK([ovs-ofctl dump-flows br-int | grep table=68 | ofctl_strip_all], [0], []) + +OVN_CLEANUP([hv1]) +AT_CLEANUP +]) diff --git a/tests/system-ovn.at b/tests/system-ovn.at index 20c0584151..8d0ec68479 100644 --- a/tests/system-ovn.at +++ b/tests/system-ovn.at @@ -8597,3 +8597,186 @@ OVS_TRAFFIC_VSWITCHD_STOP(["/failed to query port patch-.*/d /connection dropped.*/d"]) AT_CLEANUP ]) + +OVN_FOR_EACH_NORTHD([ +AT_SETUP([load-balancer template IPv4]) +AT_SKIP_IF([test $HAVE_NC = no]) +AT_KEYWORDS([ovnlb templates]) + +CHECK_CONNTRACK() +CHECK_CONNTRACK_NAT() +ovn_start +OVS_TRAFFIC_VSWITCHD_START() +OVS_CHECK_CT_ZERO_SNAT() +ADD_BR([br-int]) + +# Set external-ids in br-int needed for ovn-controller +ovs-vsctl \ + -- set Open_vSwitch . external-ids:system-id=hv1 \ + -- set Open_vSwitch . external-ids:ovn-remote=unix:$ovs_base/ovn-sb/ovn-sb.sock \ + -- set Open_vSwitch . external-ids:ovn-encap-type=geneve \ + -- set Open_vSwitch . external-ids:ovn-encap-ip=169.0.0.1 \ + -- set bridge br-int fail-mode=secure other-config:disable-in-band=true + +# Start ovn-controller +start_daemon ovn-controller + +# Logical network: +# VM1 -- LS1 -- GW-Router -- LS2 -- VM3 +# | +# VM2 ----+ +# +# A templated load balancer applied on LS1 and GW-Router with +# VM1 as backend. The VIP should be accessible from both VM2 and VM3. + +check ovn-nbctl \ + -- lr-add rtr \ + -- set Logical_Router rtr options:chassis=hv1 \ + -- lrp-add rtr rtr-ls1 00:00:00:00:01:00 42.42.42.1/24 \ + -- lrp-add rtr rtr-ls2 00:00:00:00:02:00 43.43.43.1/24 \ + -- ls-add ls1 \ + -- lsp-add ls1 ls1-rtr \ + -- lsp-set-addresses ls1-rtr 00:00:00:00:01:00 \ + -- lsp-set-type ls1-rtr router \ + -- lsp-set-options ls1-rtr router-port=rtr-ls1 \ + -- lsp-add ls1 vm1 -- lsp-set-addresses vm1 00:00:00:00:00:01 \ + -- lsp-add ls1 vm2 -- lsp-set-addresses vm2 00:00:00:00:00:02 \ + -- ls-add ls2 \ + -- lsp-add ls2 ls2-rtr \ + -- lsp-set-addresses ls2-rtr 00:00:00:00:02:00 \ + -- lsp-set-type ls2-rtr router \ + -- lsp-set-options ls2-rtr router-port=rtr-ls2 \ + -- lsp-add ls2 vm3 -- lsp-set-addresses vm3 00:00:00:00:00:03 + +# Add a template LB that eventually expands to: +# VIP=66.66.66.66:666 backends=42.42.42.2:4242 proto=tcp + +AT_CHECK([ovn-nbctl -- create chassis_template_var chassis="hv1" variables="{vip=66.66.66.66,vport=666,backends=\"42.42.42.2:4242\"}"], + [0], [ignore]) + +check ovn-nbctl --template lb-add lb-test "^vip:^vport" "^backends" tcp \ + -- ls-lb-add ls1 lb-test \ + -- lr-lb-add rtr lb-test + +ADD_NAMESPACES(vm1) +ADD_VETH(vm1, vm1, br-int, "42.42.42.2/24", "00:00:00:00:00:01", "42.42.42.1") + +ADD_NAMESPACES(vm2) +ADD_VETH(vm2, vm2, br-int, "42.42.42.3/24", "00:00:00:00:00:02", "42.42.42.1") + +ADD_NAMESPACES(vm3) +ADD_VETH(vm3, vm3, br-int, "43.43.43.2/24", "00:00:00:00:00:03", "43.43.43.1") + +# Wait for ovn-controller to catch up. +wait_for_ports_up +check ovn-nbctl --wait=hv sync + +AT_CHECK([ovn-appctl -t ovn-controller debug/dump-local-template-vars | sort], [0], [dnl +Local template vars: +name: 'backends' value: '42.42.42.2:4242' +name: 'vip' value: '66.66.66.66' +name: 'vport' value: '666' +]) + +# Start IPv4 TCP server on vm1. +NETNS_DAEMONIZE([vm1], [nc -k -l 42.42.42.2 4242], [nc-vm1.pid]) + +# Make sure connecting to the VIP works. +NS_CHECK_EXEC([vm2], [nc 66.66.66.66 666 -z], [0], [ignore], [ignore]) +NS_CHECK_EXEC([vm3], [nc 66.66.66.66 666 -z], [0], [ignore], [ignore]) + +AT_CLEANUP +]) + +OVN_FOR_EACH_NORTHD([ +AT_SETUP([load-balancer template IPv6]) +AT_SKIP_IF([test $HAVE_NC = no]) +AT_KEYWORDS([ovnlb templates]) + +CHECK_CONNTRACK() +CHECK_CONNTRACK_NAT() +ovn_start +OVS_TRAFFIC_VSWITCHD_START() +OVS_CHECK_CT_ZERO_SNAT() +ADD_BR([br-int]) + +# Set external-ids in br-int needed for ovn-controller +ovs-vsctl \ + -- set Open_vSwitch . external-ids:system-id=hv1 \ + -- set Open_vSwitch . external-ids:ovn-remote=unix:$ovs_base/ovn-sb/ovn-sb.sock \ + -- set Open_vSwitch . external-ids:ovn-encap-type=geneve \ + -- set Open_vSwitch . external-ids:ovn-encap-ip=169.0.0.1 \ + -- set bridge br-int fail-mode=secure other-config:disable-in-band=true + +# Start ovn-controller +start_daemon ovn-controller + +# Logical network: +# VM1 -- LS1 -- GW-Router -- LS2 -- VM3 +# | +# VM2 ----+ +# +# A templated load balancer applied on LS1 and GW-Router with +# VM1 as backend. The VIP should be accessible from both VM2 and VM3. + +check ovn-nbctl \ + -- lr-add rtr \ + -- set Logical_Router rtr options:chassis=hv1 \ + -- lrp-add rtr rtr-ls1 00:00:00:00:01:00 4242::1/64 \ + -- lrp-add rtr rtr-ls2 00:00:00:00:02:00 4343::1/64 \ + -- ls-add ls1 \ + -- lsp-add ls1 ls1-rtr \ + -- lsp-set-addresses ls1-rtr 00:00:00:00:01:00 \ + -- lsp-set-type ls1-rtr router \ + -- lsp-set-options ls1-rtr router-port=rtr-ls1 \ + -- lsp-add ls1 vm1 -- lsp-set-addresses vm1 00:00:00:00:00:01 \ + -- lsp-add ls1 vm2 -- lsp-set-addresses vm2 00:00:00:00:00:02 \ + -- ls-add ls2 \ + -- lsp-add ls2 ls2-rtr \ + -- lsp-set-addresses ls2-rtr 00:00:00:00:02:00 \ + -- lsp-set-type ls2-rtr router \ + -- lsp-set-options ls2-rtr router-port=rtr-ls2 \ + -- lsp-add ls2 vm3 -- lsp-set-addresses vm3 00:00:00:00:00:03 + +# Add a template LB that eventually expands to: +# VIP=6666::1 backends=[4242::2]:4242 proto=tcp + +AT_CHECK([ovn-nbctl -- create chassis_template_var chassis="hv1" variables="{vip=\"6666::1\",vport=666,backends=\"[[4242::2]]:4242\"}"], + [0], [ignore]) + +check ovn-nbctl --template lb-add lb-test "^vip:^vport" "^backends" tcp ipv6 \ + -- ls-lb-add ls1 lb-test \ + -- lr-lb-add rtr lb-test + +ADD_NAMESPACES(vm1) +ADD_VETH(vm1, vm1, br-int, "4242::2/64", "00:00:00:00:00:01", "4242::1") +OVS_WAIT_UNTIL([test "$(ip netns exec vm1 ip a | grep 4242::2 | grep tentative)" = ""]) + +ADD_NAMESPACES(vm2) +ADD_VETH(vm2, vm2, br-int, "4242::3/64", "00:00:00:00:00:02", "4242::1") +OVS_WAIT_UNTIL([test "$(ip netns exec vm2 ip a | grep 4242::3 | grep tentative)" = ""]) + +ADD_NAMESPACES(vm3) +ADD_VETH(vm3, vm3, br-int, "4343::2/64", "00:00:00:00:00:03", "4343::1") +OVS_WAIT_UNTIL([test "$(ip netns exec vm3 ip a | grep 4343::2 | grep tentative)" = ""]) + +# Wait for ovn-controller to catch up. +wait_for_ports_up +check ovn-nbctl --wait=hv sync + +AT_CHECK([ovn-appctl -t ovn-controller debug/dump-local-template-vars | sort], [0], [dnl +Local template vars: +name: 'backends' value: '[[4242::2]]:4242' +name: 'vip' value: '6666::1' +name: 'vport' value: '666' +]) + +# Start IPv6 TCP server on vm1. +NETNS_DAEMONIZE([vm1], [nc -k -l 4242::2 4242], [nc-vm1.pid]) + +# Make sure connecting to the VIP works. +NS_CHECK_EXEC([vm2], [nc 6666::1 666 -z], [0], [ignore], [ignore]) +NS_CHECK_EXEC([vm3], [nc 6666::1 666 -z], [0], [ignore], [ignore]) + +AT_CLEANUP +]) diff --git a/utilities/ovn-nbctl.c b/utilities/ovn-nbctl.c index d2dee6b31c..dea7218f21 100644 --- a/utilities/ovn-nbctl.c +++ b/utilities/ovn-nbctl.c @@ -28,6 +28,7 @@ #include "openvswitch/json.h" #include "lib/acl-log.h" #include "lib/copp.h" +#include "lib/lb.h" #include "lib/ovn-nb-idl.h" #include "lib/ovn-util.h" #include "memory.h" @@ -2837,6 +2838,7 @@ nbctl_lb_add(struct ctl_context *ctx) bool empty_backend_rej = shash_find(&ctx->options, "--reject") != NULL; bool empty_backend_event = shash_find(&ctx->options, "--event") != NULL; bool add_route = shash_find(&ctx->options, "--add-route") != NULL; + bool template = shash_find(&ctx->options, "--template") != NULL; if (empty_backend_event && empty_backend_rej) { ctl_error(ctx, @@ -2844,10 +2846,12 @@ nbctl_lb_add(struct ctl_context *ctx) return; } + const char *lb_address_family_str = "ipv4"; + int lb_address_family = AF_INET; const char *lb_proto; bool is_update_proto = false; - if (ctx->argc == 4) { + if (ctx->argc <= 4) { /* Default protocol. */ lb_proto = "tcp"; } else { @@ -2863,79 +2867,61 @@ nbctl_lb_add(struct ctl_context *ctx) } } - struct sockaddr_storage ss_vip; - if (!inet_parse_active(lb_vip, 0, &ss_vip, false, NULL)) { - ctl_error(ctx, "%s: should be an IP address (or an IP address " - "and a port number with : as a separator).", lb_vip); - return; - } - - struct ds lb_vip_normalized_ds = DS_EMPTY_INITIALIZER; - uint16_t lb_vip_port = ss_get_port(&ss_vip); - if (lb_vip_port) { - ss_format_address(&ss_vip, &lb_vip_normalized_ds); - ds_put_format(&lb_vip_normalized_ds, ":%d", lb_vip_port); - } else { - ss_format_address_nobracks(&ss_vip, &lb_vip_normalized_ds); - } - const char *lb_vip_normalized = ds_cstr(&lb_vip_normalized_ds); + if (ctx->argc > 5) { + lb_address_family_str = ctx->argv[5]; + lb_address_family = !strcmp(lb_address_family_str, "ipv4") + ? AF_INET : AF_INET6; - if (!lb_vip_port && is_update_proto) { - ds_destroy(&lb_vip_normalized_ds); - ctl_error(ctx, "Protocol is unnecessary when no port of vip " - "is given."); - return; } - char *token = NULL, *save_ptr = NULL; + struct ds lb_vip_normalized = DS_EMPTY_INITIALIZER; struct ds lb_ips_new = DS_EMPTY_INITIALIZER; - for (token = strtok_r(lb_ips, ",", &save_ptr); - token != NULL; token = strtok_r(NULL, ",", &save_ptr)) { - struct sockaddr_storage ss_dst; + struct ovn_lb_vip lb_vip_parsed; - if (lb_vip_port) { - if (!inet_parse_active(token, -1, &ss_dst, false, NULL)) { - ctl_error(ctx, "%s: should be an IP address and a port " - "number with : as a separator.", token); - goto out; - } - } else { - if (!inet_parse_address(token, &ss_dst)) { - ctl_error(ctx, "%s: should be an IP address.", token); - goto out; - } - } + char *error = ovn_lb_vip_init(&lb_vip_parsed, lb_vip, lb_ips, template, + lb_address_family); + if (error) { + ctl_error(ctx, "%s", error); + ovn_lb_vip_destroy(&lb_vip_parsed); + free(error); + return; + } - if (ss_vip.ss_family != ss_dst.ss_family) { - ctl_error(ctx, "%s: IP address family is different from VIP %s.", - token, lb_vip_normalized); - goto out; - } - ds_put_format(&lb_ips_new, "%s%s", - lb_ips_new.length ? "," : "", token); + if (is_update_proto && !lb_vip_parsed.port_str) { + ctl_error(ctx, "Protocol is unnecessary when no port of vip is " + "given."); + ovn_lb_vip_destroy(&lb_vip_parsed); + return; } + ovn_lb_vip_format(&lb_vip_parsed, &lb_vip_normalized, template); + ovn_lb_vip_backends_format(&lb_vip_parsed, &lb_ips_new, template); + ovn_lb_vip_destroy(&lb_vip_parsed); + const struct nbrec_load_balancer *lb = NULL; if (!add_duplicate) { - char *error = lb_by_name_or_uuid(ctx, lb_name, false, &lb); + error = lb_by_name_or_uuid(ctx, lb_name, false, &lb); if (error) { ctx->error = error; goto out; } if (lb) { - if (smap_get(&lb->vips, lb_vip_normalized)) { + if (smap_get(&lb->vips, ds_cstr(&lb_vip_normalized))) { if (!may_exist) { ctl_error(ctx, "%s: a load balancer with this vip (%s) " - "already exists", lb_name, lb_vip_normalized); + "already exists", lb_name, + ds_cstr(&lb_vip_normalized)); goto out; } /* Update the vips. */ smap_replace(CONST_CAST(struct smap *, &lb->vips), - lb_vip_normalized, ds_cstr(&lb_ips_new)); + ds_cstr(&lb_vip_normalized), + ds_cstr(&lb_ips_new)); } else { /* Add the new vips. */ smap_add(CONST_CAST(struct smap *, &lb->vips), - lb_vip_normalized, ds_cstr(&lb_ips_new)); + ds_cstr(&lb_vip_normalized), + ds_cstr(&lb_ips_new)); } /* Update the load balancer. */ @@ -2954,7 +2940,7 @@ nbctl_lb_add(struct ctl_context *ctx) nbrec_load_balancer_set_name(lb, lb_name); nbrec_load_balancer_set_protocol(lb, lb_proto); smap_add(CONST_CAST(struct smap *, &lb->vips), - lb_vip_normalized, ds_cstr(&lb_ips_new)); + ds_cstr(&lb_vip_normalized), ds_cstr(&lb_ips_new)); nbrec_load_balancer_set_vips(lb, &lb->vips); struct smap options = SMAP_INITIALIZER(&options); if (empty_backend_rej) { @@ -2966,12 +2952,16 @@ nbctl_lb_add(struct ctl_context *ctx) if (add_route) { smap_add(&options, "add_route", "true"); } + if (template) { + smap_add(&options, "template", "true"); + smap_add(&options, "address-family", lb_address_family_str); + } nbrec_load_balancer_set_options(lb, &options); smap_destroy(&options); out: ds_destroy(&lb_ips_new); - ds_destroy(&lb_vip_normalized_ds); + ds_destroy(&lb_vip_normalized); } static void @@ -3025,6 +3015,7 @@ static void nbctl_pre_lb_list(struct ctl_context *ctx) { ovsdb_idl_add_column(ctx->idl, &nbrec_load_balancer_col_name); + ovsdb_idl_add_column(ctx->idl, &nbrec_load_balancer_col_options); ovsdb_idl_add_column(ctx->idl, &nbrec_load_balancer_col_protocol); ovsdb_idl_add_column(ctx->idl, &nbrec_load_balancer_col_vips); } @@ -3033,6 +3024,7 @@ static void lb_info_add_smap(const struct nbrec_load_balancer *lb, struct smap *lbs, int vip_width) { + bool template = smap_get_bool(&lb->options, "template", false); const struct smap_node **nodes = smap_sort(&lb->vips); if (!nodes) { return; @@ -3041,13 +3033,24 @@ lb_info_add_smap(const struct nbrec_load_balancer *lb, struct ds val = DS_EMPTY_INITIALIZER; for (size_t i = 0; i < smap_count(&lb->vips); i++) { const struct smap_node *node = nodes[i]; + const char *protocol = lb->protocol; - struct sockaddr_storage ss; - if (!inet_parse_active(node->key, 0, &ss, false, NULL)) { - continue; + if (!template) { + struct sockaddr_storage ss; + if (!inet_parse_active(node->key, 0, &ss, false, NULL)) { + continue; + } + protocol = ss_get_port(&ss) ? lb->protocol : ""; + } else { + if (!lb->protocol) { + VLOG_WARN("Load Balancer "UUID_FMT" (%s) is a template and " + "misses protocol", UUID_ARGS(&lb->header_.uuid), + lb->name); + continue; + } + protocol = lb->protocol; } - char *protocol = ss_get_port(&ss) ? lb->protocol : ""; if (i == 0) { ds_put_format(&val, UUID_FMT " %-20.16s%-11.7s%-*.*s%s", UUID_ARGS(&lb->header_.uuid), @@ -3239,6 +3242,7 @@ nbctl_pre_lr_lb_list(struct ctl_context *ctx) &nbrec_logical_router_col_load_balancer_group); ovsdb_idl_add_column(ctx->idl, &nbrec_load_balancer_col_name); + ovsdb_idl_add_column(ctx->idl, &nbrec_load_balancer_col_options); ovsdb_idl_add_column(ctx->idl, &nbrec_load_balancer_col_protocol); ovsdb_idl_add_column(ctx->idl, &nbrec_load_balancer_col_vips); @@ -3402,6 +3406,7 @@ nbctl_pre_ls_lb_list(struct ctl_context *ctx) &nbrec_logical_switch_col_load_balancer_group); ovsdb_idl_add_column(ctx->idl, &nbrec_load_balancer_col_name); + ovsdb_idl_add_column(ctx->idl, &nbrec_load_balancer_col_options); ovsdb_idl_add_column(ctx->idl, &nbrec_load_balancer_col_protocol); ovsdb_idl_add_column(ctx->idl, &nbrec_load_balancer_col_vips); @@ -7472,9 +7477,10 @@ static const struct ctl_command_syntax nbctl_commands[] = { nbctl_pre_lr_nat_set_ext_ips, nbctl_lr_nat_set_ext_ips, NULL, "--is-exempted", RW}, /* load balancer commands. */ - { "lb-add", 3, 4, "LB VIP[:PORT] IP[:PORT]... [PROTOCOL]", + { "lb-add", 3, 5, "LB VIP[:PORT] IP[:PORT]... [PROTOCOL] [ADDRESS_FAMILY]", nbctl_pre_lb_add, nbctl_lb_add, NULL, - "--may-exist,--add-duplicate,--reject,--event,--add-route", RW }, + "--may-exist,--add-duplicate,--reject,--event,--add-route,--template", + RW }, { "lb-del", 1, 2, "LB [VIP]", nbctl_pre_lb_del, nbctl_lb_del, NULL, "--if-exists", RW }, { "lb-list", 0, 1, "[LB]", nbctl_pre_lb_list, nbctl_lb_list, NULL, "", RO }, From patchwork Fri Nov 4 22:11:53 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dumitru Ceara X-Patchwork-Id: 1699897 X-Patchwork-Delegate: mmichels@redhat.com Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::136; helo=smtp3.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=cLbN+I6k; dkim-atps=neutral Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4N3vy31MrKz23m0 for ; Sat, 5 Nov 2022 09:12:43 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 6833F611A9; Fri, 4 Nov 2022 22:12:41 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 6833F611A9 Authentication-Results: smtp3.osuosl.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=cLbN+I6k X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 3-WUu6b3adf1; Fri, 4 Nov 2022 22:12:40 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp3.osuosl.org (Postfix) with ESMTPS id 3E87661193; Fri, 4 Nov 2022 22:12:39 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 3E87661193 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id DF7B6C0032; Fri, 4 Nov 2022 22:12:38 +0000 (UTC) X-Original-To: ovs-dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by lists.linuxfoundation.org (Postfix) with ESMTP id DA4AFC002D for ; Fri, 4 Nov 2022 22:12:36 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id C0E0082210 for ; Fri, 4 Nov 2022 22:12:02 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org C0E0082210 Authentication-Results: smtp1.osuosl.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=cLbN+I6k X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id UWydkWXme8ut for ; Fri, 4 Nov 2022 22:12:02 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org D9AEF82264 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by smtp1.osuosl.org (Postfix) with ESMTPS id D9AEF82264 for ; Fri, 4 Nov 2022 22:12:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1667599920; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=R/x3akD9R3bs2XvUcQ54E6UMvPCZ2S8FQHKeHrGvj7Q=; b=cLbN+I6kiA1FgZoiJX90Eaez2BSXPxHlCShAzUyYQy94NbhkYpSatGsPwwodfEcaW/B8qN xKPCZ3crOMur9ZPeKYep6YUzzHzGn699YIx118m1juESCbm6IIx6F8kgXl3FL4jctwimT1 rD+UKJNC21xGnB71T0ipBWlGfLlBdN0= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-630-skhV_D4EO5CILxsvmbFTQg-1; Fri, 04 Nov 2022 18:11:57 -0400 X-MC-Unique: skhV_D4EO5CILxsvmbFTQg-1 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 935C0380451D; Fri, 4 Nov 2022 22:11:57 +0000 (UTC) Received: from dceara.remote.csb (unknown [10.39.192.36]) by smtp.corp.redhat.com (Postfix) with ESMTP id 3EBB74B3FC6; Fri, 4 Nov 2022 22:11:56 +0000 (UTC) From: Dumitru Ceara To: ovs-dev@openvswitch.org Date: Fri, 4 Nov 2022 23:11:53 +0100 Message-Id: <166759991227.515460.16643724109191119263.stgit@dceara.remote.csb> In-Reply-To: <166759985955.515460.16275468131013093936.stgit@dceara.remote.csb> References: <166759985955.515460.16275468131013093936.stgit@dceara.remote.csb> User-Agent: StGit/0.23 MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.9 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Cc: surya@redhat.com, i.maximets@ovn.org Subject: [ovs-dev] [PATCH ovn v2 5/5] tutorial: Add scripts to simulate node-port ovn-k8s services. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" In a sandbox run: $ ./ovn-lb-benchmark.sh to simulate an ovn-k8s-like topology with N nodes, VIPS NodePort services applied to all nodes. Each service has BACKENDS backends. If USE_TEMPLATES is "yes" then the configuration will be optimized to use Chassis_Template_Vars. Otherwise it will create N LBs per service, one for every node. Signed-off-by: Dumitru Ceara Acked-by: Mark Michelson --- tutorial/automake.mk | 4 + tutorial/ovn-gen-lb-template-vars.py | 116 ++++++++++++++++++++++++++++++++++ tutorial/ovn-lb-benchmark.sh | 110 ++++++++++++++++++++++++++++++++ 3 files changed, 229 insertions(+), 1 deletion(-) create mode 100755 tutorial/ovn-gen-lb-template-vars.py create mode 100755 tutorial/ovn-lb-benchmark.sh diff --git a/tutorial/automake.mk b/tutorial/automake.mk index 046962c000..171da8de66 100644 --- a/tutorial/automake.mk +++ b/tutorial/automake.mk @@ -1,6 +1,8 @@ EXTRA_DIST += \ tutorial/ovs-sandbox \ - tutorial/ovn-setup.sh + tutorial/ovn-setup.sh \ + tutorial/ovn-lb-benchmark.sh \ + tutorial/ovn-gen-lb-template-vars.py sandbox: all cd $(srcdir)/tutorial && MAKE=$(MAKE) HAVE_OPENSSL=$(HAVE_OPENSSL) \ ./ovs-sandbox -b $(abs_builddir) --ovs-src $(ovs_srcdir) --ovs-build $(ovs_builddir) $(SANDBOXFLAGS) diff --git a/tutorial/ovn-gen-lb-template-vars.py b/tutorial/ovn-gen-lb-template-vars.py new file mode 100755 index 0000000000..fe7e6b93f6 --- /dev/null +++ b/tutorial/ovn-gen-lb-template-vars.py @@ -0,0 +1,116 @@ +import getopt +import os +import re +import sys +import uuid + +import ovs.db.idl +import ovs.db.schema +import ovs.db.types +import ovs.ovsuuid +import ovs.poller +import ovs.stream +import ovs.util +import ovs.vlog +from ovs.db import data +from ovs.db import error +from ovs.db.idl import _row_to_uuid as row_to_uuid +from ovs.fatal_signal import signal_alarm + +vlog = ovs.vlog.Vlog("template-lb-stress") +vlog.set_levels_from_string("console:info") +vlog.init(None) + +SCHEMA = '../ovn-nb.ovsschema' + + +def add_chassis_template_vars(idl, n, n_vips, n_backends): + for i in range(1, n + 1): + print(f'ADDING LBs for node {i}') + txn = ovs.db.idl.Transaction(idl) + tv = txn.insert(idl.tables["Chassis_Template_Var"]) + tv.chassis = f'chassis-{i}' + tv.setkey('variables', 'vip', f'42.42.42.{i}') + + for j in range(1, n_vips + 1): + backends = '' + for k in range(0, n_backends): + j1 = j // 250 + j2 = j % 250 + backends = f'42.{k}.{j1}.{j2}:{j},{backends}' + tv.setkey('variables', f'backends{j}', backends) + status = txn.commit_block() + sys.stdout.write( + f'commit status = {ovs.db.idl.Transaction.status_to_string(status)}\n' + ) + + +def run(remote, n, n_vips, n_backends): + schema_helper = ovs.db.idl.SchemaHelper(SCHEMA) + schema_helper.register_all() + idl = ovs.db.idl.Idl(remote, schema_helper, leader_only=False) + + seqno = 0 + + error, stream = ovs.stream.Stream.open_block( + ovs.stream.Stream.open(remote), 2000 + ) + if error: + sys.stderr.write(f'failed to connect to \"{remote}\"') + sys.exit(1) + + if not stream: + sys.stderr.write(f'failed to connect to \"{remote}\"') + sys.exit(1) + rpc = ovs.jsonrpc.Connection(stream) + + while idl.change_seqno == seqno and not idl.run(): + rpc.run() + + poller = ovs.poller.Poller() + idl.wait(poller) + rpc.wait(poller) + poller.block() + + add_chassis_template_vars(idl, n, n_vips, n_backends) + + +def main(argv): + try: + options, args = getopt.gnu_getopt( + argv[1:], 'n:v:b:r:', ['vips', 'backends', 'remote'] + ) + except getopt.GetoptError as geo: + sys.stderr.write(f'{ovs.util.PROGRAM_NAME}: {geo.msg}\n') + sys.exit(1) + + n = None + vips = None + backends = None + remote = None + for key, value in options: + if key == '-n': + n = int(value) + elif key in ['-v', '--vips']: + vips = int(value) + elif key in ['-b', '--backends']: + backends = int(value) + elif key in ['-r', '--remote']: + remote = value + else: + sys.stderr.write(f'{ovs.util.PROGRAM_NAME}: unknown input args') + sys.exit(1) + + if not n or not vips or not backends: + sys.stderr.write(f'{ovs.util.PROGRAM_NAME}: invalid input args') + sys.exit(1) + + run(remote, n, vips, backends) + + +if __name__ == '__main__': + try: + main(sys.argv) + except error.Error as e: + sys.stderr.write(f'{e}\n') + sys.exit(1) diff --git a/tutorial/ovn-lb-benchmark.sh b/tutorial/ovn-lb-benchmark.sh new file mode 100755 index 0000000000..6e8129ab97 --- /dev/null +++ b/tutorial/ovn-lb-benchmark.sh @@ -0,0 +1,110 @@ +#!/bin/bash + +nrtr=$1 +nlb=$2 +nbackends=$3 +use_template=$4 + +echo "ROUTERS : $nrtr" +echo "LBS : $nlb" +echo "BACKENDS PER LB: $nbackends" +echo "USE TEMPLATE : ${use_template}" + +export OVN_NB_DAEMON=$(ovn-nbctl --detach) +export OVN_SB_DAEMON=$(ovn-sbctl --detach) +trap "killall -9 ovn-nbctl; killall -9 ovn-sbctl" EXIT + +lbg=$(ovn-nbctl create load_balancer_group name=lbg) + +ovn-nbctl ls-add join +ovn-nbctl lr-add cluster +ovn-nbctl lrp-add cluster rcj 00:00:00:00:00:01 10.0.0.1/8 +ovn-nbctl lsp-add join sjc \ + -- lsp-set-type sjc router \ + -- lsp-set-addresses sjc router \ + -- lsp-set-options sjc router-port=rcj + +for i in $(seq $nrtr); do + ch="chassis-$i" + gwr=lr-$i + echo "ROUTER $gwr" + gwr2join=lr2j-$i + join2gwr=j2lr-$i + + ovn-nbctl lr-add $gwr \ + -- set logical_router $gwr load_balancer_group=$lbg \ + -- set logical_router $gwr options:chassis=$ch + ovn-nbctl lrp-add $gwr $gwr2join 00:00:00:00:00:01 10.0.0.1/8 + ovn-nbctl lsp-add join $join2gwr \ + -- lsp-set-type $join2gwr router \ + -- lsp-set-addresses $join2gwr router \ + -- lsp-set-options $join2gwr router-port=$gwr2join + + s=ls-$i + echo "SWITCH $s" + s2cluster=s2c-$s + cluster2s=c2s-$s + ovn-nbctl ls-add $s \ + -- set logical_switch $s load_balancer_group=$lbg + ovn-nbctl lsp-add $s $s2cluster \ + -- lsp-set-type $s2cluster router \ + -- lsp-set-addresses $s2cluster router \ + -- lsp-set-options $s2cluster router-port=$cluster2s + ovn-nbctl lrp-add cluster $cluster2s 00:00:00:00:00:01 10.0.0.1/8 + ovn-nbctl lrp-set-gateway-chassis $cluster2s $ch 1 + + lsp=lsp-$i + echo "LSP $lsp" + ovn-nbctl lsp-add $s $lsp +done + +# Bind a port from the first LS locally. +ovs-vsctl add-port br-int lsp-1 \ + -- set interface lsp-1 external_ids:iface-id=lsp-1 + +# Add NodePort-like LBs using templates. +function add_template_lbs() { + for l in $(seq $nlb); do + lb=lb-$l + echo "LB $lb" + ovn-nbctl --template lb-add $lb "^vip:$l" "^backends$l" tcp + lb_uuid=$(ovn-nbctl --columns _uuid --bare find load_balancer name=$lb) + ovn-nbctl add load_balancer_group $lbg load_balancer $lb_uuid + done + + # Generate LBs chassis_template_vars. + python ovn-gen-lb-template-vars.py -n $nrtr -v $nlb -b $nbackends \ + -r unix:$PWD/sandbox/nb1.ovsdb +} + +# Add NodePort-like LBs without using templates. +function add_non_template_lbs() { + for i in $(seq $nrtr); do + echo ITERATION $i + gwr=lr-$i + s=ls-$i + for l in $(seq $nlb); do + lb=lb-$l-$i + echo LB $lb + backends="" + for k in $(seq $nbackends); do + l1=$(expr $l / 250) + l2=$(expr $l % 250) + backends="42.$k.$l1.$l2:$l,$backends" + done + lb_uuid=$(ovn-nbctl create load_balancer name=$lb \ + protocol=tcp vips:"\"42.42.42.$i:$l\""="\"$backends\"") + ovn-nbctl add logical_switch $s load_balancer ${lb_uuid} + ovn-nbctl add logical_router $gwr load_balancer ${lb_uuid} + done + done +} + +if [ "${use_template}" = "yes" ]; then + add_template_lbs +else + add_non_template_lbs +fi + +ovs-appctl -t $PWD/sandbox/nb1 ovsdb-server/compact +ovs-appctl -t $PWD/sandbox/sb1 ovsdb-server/compact