From patchwork Thu Mar 14 19:31:32 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Numan Siddique X-Patchwork-Id: 1056668 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=openvswitch.org (client-ip=140.211.169.12; helo=mail.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Received: from mail.linuxfoundation.org (mail.linuxfoundation.org [140.211.169.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 44KzNc2X7cz9ryj for ; Fri, 15 Mar 2019 06:31:44 +1100 (AEDT) Received: from mail.linux-foundation.org (localhost [127.0.0.1]) by mail.linuxfoundation.org (Postfix) with ESMTP id EB223CCE; Thu, 14 Mar 2019 19:31:40 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@mail.linuxfoundation.org Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 0D4B2CC6 for ; Thu, 14 Mar 2019 19:31:40 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 6CA82D3 for ; Thu, 14 Mar 2019 19:31:39 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 07453C058CAD; Thu, 14 Mar 2019 19:31:39 +0000 (UTC) Received: from nusiddiq.mac (unknown [10.67.116.14]) by smtp.corp.redhat.com (Postfix) with ESMTP id 9D5781001DC7; Thu, 14 Mar 2019 19:31:37 +0000 (UTC) From: nusiddiq@redhat.com To: dev@openvswitch.org Date: Fri, 15 Mar 2019 01:01:32 +0530 Message-Id: <20190314193132.24343-1-nusiddiq@redhat.com> In-Reply-To: <20190314193056.24241-1-nusiddiq@redhat.com> References: <20190314193056.24241-1-nusiddiq@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.32]); Thu, 14 Mar 2019 19:31:39 +0000 (UTC) X-Spam-Status: No, score=-6.9 required=5.0 tests=BAYES_00,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Subject: [ovs-dev] [PATCH v4 1/5] ovn-northd: Reuse the hmaps - datapaths and ports in ovnsb_db_run() X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: ovs-dev-bounces@openvswitch.org Errors-To: ovs-dev-bounces@openvswitch.org From: Numan Siddique We can reuse the datapaths and ports built during ovnnb_db_run() in ovnsb_db_run(). This way we avoid creating the logical ports hash nodes during the ovnsb_db_run(). An upcoming patch will make further use of these hashmaps during ovnsb_db_run(). This patch refactors the code accordingly. Signed-off-by: Numan Siddique Acked-by: Han Zhou --- ovn/northd/ovn-northd.c | 109 ++++++++++++++++++---------------------- 1 file changed, 48 insertions(+), 61 deletions(-) diff --git a/ovn/northd/ovn-northd.c b/ovn/northd/ovn-northd.c index 2843969f4..63fc0f13b 100644 --- a/ovn/northd/ovn-northd.c +++ b/ovn/northd/ovn-northd.c @@ -667,7 +667,6 @@ join_datapaths(struct northd_context *ctx, struct hmap *datapaths, struct ovs_list *sb_only, struct ovs_list *nb_only, struct ovs_list *both) { - hmap_init(datapaths); ovs_list_init(sb_only); ovs_list_init(nb_only); ovs_list_init(both); @@ -1579,7 +1578,6 @@ join_logical_ports(struct northd_context *ctx, struct hmap *tag_alloc_table, struct ovs_list *sb_only, struct ovs_list *nb_only, struct ovs_list *both) { - hmap_init(ports); ovs_list_init(sb_only); ovs_list_init(nb_only); ovs_list_init(both); @@ -7221,28 +7219,43 @@ sync_dns_entries(struct northd_context *ctx, struct hmap *datapaths) } hmap_destroy(&dns_map); } + +static void +destroy_datapaths_and_ports(struct hmap *datapaths, struct hmap *ports) +{ + struct ovn_datapath *dp, *next_dp; + HMAP_FOR_EACH_SAFE (dp, next_dp, key_node, datapaths) { + ovn_datapath_destroy(datapaths, dp); + } + hmap_destroy(datapaths); + struct ovn_port *port, *next_port; + HMAP_FOR_EACH_SAFE (port, next_port, key_node, ports) { + ovn_port_destroy(ports, port); + } + hmap_destroy(ports); +} - static void ovnnb_db_run(struct northd_context *ctx, struct ovsdb_idl_index *sbrec_chassis_by_name, - struct ovsdb_idl_loop *sb_loop) + struct ovsdb_idl_loop *sb_loop, + struct hmap *datapaths, struct hmap *ports) { if (!ctx->ovnsb_txn || !ctx->ovnnb_txn) { return; } - struct hmap datapaths, ports, port_groups; - build_datapaths(ctx, &datapaths); - build_ports(ctx, sbrec_chassis_by_name, &datapaths, &ports); - build_ipam(&datapaths, &ports); - build_port_group_lswitches(ctx, &port_groups, &ports); - build_lflows(ctx, &datapaths, &ports, &port_groups); + struct hmap port_groups; + build_datapaths(ctx, datapaths); + build_ports(ctx, sbrec_chassis_by_name, datapaths, ports); + build_ipam(datapaths, ports); + build_port_group_lswitches(ctx, &port_groups, ports); + build_lflows(ctx, datapaths, ports, &port_groups); sync_address_sets(ctx); sync_port_groups(ctx); sync_meters(ctx); - sync_dns_entries(ctx, &datapaths); + sync_dns_entries(ctx, datapaths); struct ovn_port_group *pg, *next_pg; HMAP_FOR_EACH_SAFE (pg, next_pg, key_node, &port_groups) { @@ -7250,18 +7263,6 @@ ovnnb_db_run(struct northd_context *ctx, } hmap_destroy(&port_groups); - struct ovn_datapath *dp, *next_dp; - HMAP_FOR_EACH_SAFE (dp, next_dp, key_node, &datapaths) { - ovn_datapath_destroy(&datapaths, dp); - } - hmap_destroy(&datapaths); - - struct ovn_port *port, *next_port; - HMAP_FOR_EACH_SAFE (port, next_port, key_node, &ports) { - ovn_port_destroy(&ports, port); - } - hmap_destroy(&ports); - /* Sync ipsec configuration. * Copy nb_cfg from northbound to southbound database. * Also set up to update sb_cfg once our southbound transaction commits. */ @@ -7312,53 +7313,25 @@ ovnnb_db_run(struct northd_context *ctx, * this column is not empty, it means we need to set the corresponding logical * port as 'up' in the northbound DB. */ static void -update_logical_port_status(struct northd_context *ctx) +update_logical_port_status(struct northd_context *ctx, struct hmap *ports) { - struct hmap lports_hmap; const struct sbrec_port_binding *sb; - const struct nbrec_logical_switch_port *nbsp; - - struct lport_hash_node { - struct hmap_node node; - const struct nbrec_logical_switch_port *nbsp; - } *hash_node; - - hmap_init(&lports_hmap); - - NBREC_LOGICAL_SWITCH_PORT_FOR_EACH(nbsp, ctx->ovnnb_idl) { - hash_node = xzalloc(sizeof *hash_node); - hash_node->nbsp = nbsp; - hmap_insert(&lports_hmap, &hash_node->node, hash_string(nbsp->name, 0)); - } SBREC_PORT_BINDING_FOR_EACH(sb, ctx->ovnsb_idl) { - nbsp = NULL; - HMAP_FOR_EACH_WITH_HASH(hash_node, node, - hash_string(sb->logical_port, 0), - &lports_hmap) { - if (!strcmp(sb->logical_port, hash_node->nbsp->name)) { - nbsp = hash_node->nbsp; - break; - } - } + struct ovn_port *op = ovn_port_find(ports, sb->logical_port); - if (!nbsp) { + if (!op || !op->nbsp) { /* The logical port doesn't exist for this port binding. This can * happen under normal circumstances when ovn-northd hasn't gotten * around to pruning the Port_Binding yet. */ continue; } - bool up = (sb->chassis || !strcmp(nbsp->type, "router")); - if (!nbsp->up || *nbsp->up != up) { - nbrec_logical_switch_port_set_up(nbsp, &up, 1); + bool up = (sb->chassis || !strcmp(op->nbsp->type, "router")); + if (!op->nbsp->up || *op->nbsp->up != up) { + nbrec_logical_switch_port_set_up(op->nbsp, &up, 1); } } - - HMAP_FOR_EACH_POP(hash_node, node, &lports_hmap) { - free(hash_node); - } - hmap_destroy(&lports_hmap); } static struct gen_opts_map supported_dhcp_opts[] = { @@ -7680,15 +7653,30 @@ update_northbound_cfg(struct northd_context *ctx, /* Handle a fairly small set of changes in the southbound database. */ static void -ovnsb_db_run(struct northd_context *ctx, struct ovsdb_idl_loop *sb_loop) +ovnsb_db_run(struct northd_context *ctx, struct ovsdb_idl_loop *sb_loop, + struct hmap *ports) { if (!ctx->ovnnb_txn || !ovsdb_idl_has_ever_connected(ctx->ovnsb_idl)) { return; } - update_logical_port_status(ctx); + update_logical_port_status(ctx, ports); update_northbound_cfg(ctx, sb_loop); } + +static void +ovn_db_run(struct northd_context *ctx, + struct ovsdb_idl_index *sbrec_chassis_by_name, + struct ovsdb_idl_loop *ovnsb_idl_loop) +{ + struct hmap datapaths, ports; + hmap_init(&datapaths); + hmap_init(&ports); + ovnnb_db_run(ctx, sbrec_chassis_by_name, ovnsb_idl_loop, + &datapaths, &ports); + ovnsb_db_run(ctx, ovnsb_idl_loop, &ports); + destroy_datapaths_and_ports(&datapaths, &ports); +} static void parse_options(int argc OVS_UNUSED, char *argv[] OVS_UNUSED) @@ -7946,8 +7934,7 @@ main(int argc, char *argv[]) } if (ovsdb_idl_has_lock(ovnsb_idl_loop.idl)) { - ovnnb_db_run(&ctx, sbrec_chassis_by_name, &ovnsb_idl_loop); - ovnsb_db_run(&ctx, &ovnsb_idl_loop); + ovn_db_run(&ctx, sbrec_chassis_by_name, &ovnsb_idl_loop); if (ctx.ovnsb_txn) { check_and_add_supported_dhcp_opts_to_sb_db(&ctx); check_and_add_supported_dhcpv6_opts_to_sb_db(&ctx); From patchwork Thu Mar 14 19:31:47 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Numan Siddique X-Patchwork-Id: 1056669 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=openvswitch.org (client-ip=140.211.169.12; helo=mail.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Received: from mail.linuxfoundation.org (mail.linuxfoundation.org [140.211.169.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 44KzPC2cpVz9s6w for ; Fri, 15 Mar 2019 06:32:15 +1100 (AEDT) Received: from mail.linux-foundation.org (localhost [127.0.0.1]) by mail.linuxfoundation.org (Postfix) with ESMTP id 98E51D05; Thu, 14 Mar 2019 19:32:00 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@mail.linuxfoundation.org Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 54311CDF for ; Thu, 14 Mar 2019 19:31:59 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id A7C63D3 for ; Thu, 14 Mar 2019 19:31:53 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 3E9E330671ED for ; Thu, 14 Mar 2019 19:31:53 +0000 (UTC) Received: from nusiddiq.mac (unknown [10.67.116.14]) by smtp.corp.redhat.com (Postfix) with ESMTP id EC59E60BE6; Thu, 14 Mar 2019 19:31:50 +0000 (UTC) From: nusiddiq@redhat.com To: dev@openvswitch.org Date: Fri, 15 Mar 2019 01:01:47 +0530 Message-Id: <20190314193147.24444-1-nusiddiq@redhat.com> In-Reply-To: <20190314193056.24241-1-nusiddiq@redhat.com> References: <20190314193056.24241-1-nusiddiq@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.48]); Thu, 14 Mar 2019 19:31:53 +0000 (UTC) X-Spam-Status: No, score=-6.9 required=5.0 tests=BAYES_00,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Subject: [ovs-dev] [PATCH v4 2/5] ovn: Add generic HA chassis group X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: ovs-dev-bounces@openvswitch.org Errors-To: ovs-dev-bounces@openvswitch.org From: Numan Siddique This patch adds the tables - 'HA_Chassis_Group' and 'HA_Chassis' in both OVN Northbound and Southbound DBs to support generic HA Chassis groups in OVN. CMS can create a group of HA chassis with the priorities assigned to each chassis in the group. An HA chassis group can be associated to a distributed logical router port. An upcoming patch will make use of it while supporting 'external'* logical ports. HA chassis group is similar to the existing gateway chassis support in OVN which is used by the distributed gateway router ports. This patch tries to abstract this so that, the HA chassis support can be leveraged by not just distributed gateway router ports. If a logical router port has a set of gateway chassis associated to it, ovn-northd will create HA chassis group in Southbound DB and add these gateway chassis to this group. ovn-northd would still create gateway chassis in Southbound DB as ovn-controller still doesn't support using the HA chassis group. Next patch in the series will add the support in ovn-controller to make use of HA chassis group instead of gateway chassis. The patch following that will delete creation of gateway chassis in Southbound DB. HA_Chasss_Group table in Southbound DB has a column - 'ref_chassis'. This column is used to store the list of chassis which references the HA chassis group. This information will be used by ovn-controller in an upcoming patch to establish BFD sessions with the required chassis. Suppose if there is an HA chassis group - 'hagrp1' in the Southbound DB and it has HA chasiss list - ha1, ha2 and ha3 and this HA chassis group is used by a distributed logical router port, then ovn-northd will update the 'ref_chassis' with the list of chassis which has claimed all the logical switch ports which are connected to the logical router which has this distributed logical router port. Signed-off-by: Numan Siddique --- ovn/lib/chassis-index.c | 26 ++ ovn/lib/chassis-index.h | 4 + ovn/northd/ovn-northd.c | 449 ++++++++++++++++++++++++++++++++-- ovn/ovn-nb.ovsschema | 36 ++- ovn/ovn-nb.xml | 75 ++++++ ovn/ovn-sb.ovsschema | 43 +++- ovn/ovn-sb.xml | 63 +++++ ovn/utilities/ovn-nbctl.8.xml | 41 ++++ ovn/utilities/ovn-nbctl.c | 221 +++++++++++++++++ ovn/utilities/ovn-sbctl.c | 6 + tests/ovn-northd.at | 262 ++++++++++++++++++++ tests/ovn.at | 150 +++++++++--- 12 files changed, 1314 insertions(+), 62 deletions(-) diff --git a/ovn/lib/chassis-index.c b/ovn/lib/chassis-index.c index a5dbf4ace..34d4a31eb 100644 --- a/ovn/lib/chassis-index.c +++ b/ovn/lib/chassis-index.c @@ -39,3 +39,29 @@ chassis_lookup_by_name(struct ovsdb_idl_index *sbrec_chassis_by_name, return retval; } + +struct ovsdb_idl_index * +ha_chassis_group_index_create(struct ovsdb_idl *idl) +{ + return ovsdb_idl_index_create1(idl, &sbrec_ha_chassis_group_col_name); +} + +/* Finds and returns the HA chassis group with the given 'name', or NULL + * if no such HA chassis group exists. */ +const struct sbrec_ha_chassis_group * +ha_chassis_group_lookup_by_name( + struct ovsdb_idl_index *sbrec_ha_chassis_grp_by_name, + const char *name) +{ + struct sbrec_ha_chassis_group *target = + sbrec_ha_chassis_group_index_init_row(sbrec_ha_chassis_grp_by_name); + sbrec_ha_chassis_group_set_name(target, name); + + struct sbrec_ha_chassis_group *retval = + sbrec_ha_chassis_group_index_find(sbrec_ha_chassis_grp_by_name, + target); + + sbrec_ha_chassis_group_index_destroy_row(target); + + return retval; +} diff --git a/ovn/lib/chassis-index.h b/ovn/lib/chassis-index.h index d5e5df926..9bc610ad2 100644 --- a/ovn/lib/chassis-index.h +++ b/ovn/lib/chassis-index.h @@ -23,4 +23,8 @@ struct ovsdb_idl_index *chassis_index_create(struct ovsdb_idl *); const struct sbrec_chassis *chassis_lookup_by_name( struct ovsdb_idl_index *sbrec_chassis_by_name, const char *name); +struct ovsdb_idl_index *ha_chassis_group_index_create(struct ovsdb_idl *idl); +const struct sbrec_ha_chassis_group *ha_chassis_group_lookup_by_name( + struct ovsdb_idl_index *sbrec_ha_chassis_grp_by_name, const char *name); + #endif /* ovn/lib/chassis-index.h */ diff --git a/ovn/northd/ovn-northd.c b/ovn/northd/ovn-northd.c index 63fc0f13b..48092d8f5 100644 --- a/ovn/northd/ovn-northd.c +++ b/ovn/northd/ovn-northd.c @@ -56,6 +56,7 @@ struct northd_context { struct ovsdb_idl *ovnsb_idl; struct ovsdb_idl_txn *ovnnb_txn; struct ovsdb_idl_txn *ovnsb_txn; + struct ovsdb_idl_index *sbrec_ha_chassis_grp_by_name; }; static const char *ovnnb_db; @@ -450,10 +451,25 @@ struct ovn_datapath { struct ovn_port *l3redirect_port; struct ovn_port *localnet_port; + struct ovs_list lr_list; /* In list of logical router datapaths. */ + /* The logical router group to which this datapath belongs. + * Valid only if it is logical router datapath. NULL otherwise. */ + struct lrouter_group *lr_group; + /* Port groups related to the datapath, used only when nbs is NOT NULL. */ struct hmap nb_pgs; }; +/* A group of logical router datapaths which are connected - either + * directly or indirectly. + * Each logical router can belong to only one group. */ +struct lrouter_group { + struct ovn_datapath **router_dps; + int n_router_dps; + /* Set of ha_chassis_groups which are associated with the router dps. */ + struct sset ha_chassis_groups; +}; + struct macam_node { struct hmap_node hmap_node; struct eth_addr mac_addr; /* Allocated MAC address. */ @@ -483,6 +499,7 @@ ovn_datapath_create(struct hmap *datapaths, const struct uuid *key, hmap_init(&od->nb_pgs); od->port_key_hint = 0; hmap_insert(datapaths, &od->key_node, uuid_hash(&od->key)); + od->lr_group = NULL; return od; } @@ -665,7 +682,7 @@ ovn_datapath_update_external_ids(struct ovn_datapath *od) static void join_datapaths(struct northd_context *ctx, struct hmap *datapaths, struct ovs_list *sb_only, struct ovs_list *nb_only, - struct ovs_list *both) + struct ovs_list *both, struct ovs_list *lr_list) { ovs_list_init(sb_only); ovs_list_init(nb_only); @@ -746,6 +763,7 @@ join_datapaths(struct northd_context *ctx, struct hmap *datapaths, NULL, nbr, NULL); ovs_list_push_back(nb_only, &od->list); } + ovs_list_push_back(lr_list, &od->lr_list); } } @@ -762,11 +780,12 @@ ovn_datapath_allocate_key(struct hmap *dp_tnlids) * Initializes 'datapaths' to contain a "struct ovn_datapath" for every logical * switch and router. */ static void -build_datapaths(struct northd_context *ctx, struct hmap *datapaths) +build_datapaths(struct northd_context *ctx, struct hmap *datapaths, + struct ovs_list *lr_list) { struct ovs_list sb_only, nb_only, both; - join_datapaths(ctx, datapaths, &sb_only, &nb_only, &both); + join_datapaths(ctx, datapaths, &sb_only, &nb_only, &both, lr_list); if (!ovs_list_is_empty(&nb_only)) { /* First index the in-use datapath tunnel IDs. */ @@ -1980,6 +1999,13 @@ sbpb_gw_chassis_needs_update( return false; } + if (lrp->n_gateway_chassis && !port_binding->ha_chassis_group) { + /* If there are gateway chassis in the NB DB, but there is + * no corresponding HA chassis group in SB DB we need to + * create the HA chassis group in SB DB for this lrp. */ + return true; + } + /* These arrays are used to collect valid Gateway_Chassis and valid * Chassis records from the Logical_Router_Port Gateway_Chassis list, * we ignore the ones we can't match on the SBDB */ @@ -2049,31 +2075,63 @@ sbpb_gw_chassis_needs_update( return false; } +static struct sbrec_ha_chassis * +create_sb_ha_chassis(struct northd_context *ctx, + const struct sbrec_chassis *chassis, int priority) +{ + struct sbrec_ha_chassis *sb_ha_chassis = + sbrec_ha_chassis_insert(ctx->ovnsb_txn); + sbrec_ha_chassis_set_chassis(sb_ha_chassis, chassis); + sbrec_ha_chassis_set_priority(sb_ha_chassis, priority); + return sb_ha_chassis; +} + /* This functions translates the gw chassis on the nb database * to sb database entries, the only difference is that SB database * Gateway_Chassis table references the chassis directly instead - * of using the name */ + * of using the name. + * + * This function also creates a HA Chassis group in SB DB for + * the gateway chassis associated to a distributed gateway + * router port in the NB DB. + * + * An upcoming patch will delete the code to create the Gateway chassis + * in SB DB.*/ static void copy_gw_chassis_from_nbrp_to_sbpb( struct northd_context *ctx, struct ovsdb_idl_index *sbrec_chassis_by_name, const struct nbrec_logical_router_port *lrp, - const struct sbrec_port_binding *port_binding) { - - if (!lrp || !port_binding || !lrp->n_gateway_chassis) { - return; - } - + const struct sbrec_port_binding *port_binding) +{ struct sbrec_gateway_chassis **gw_chassis = NULL; int n_gwc = 0; int n; + /* Make use of the new HA chassis group table to support HA + * for the distributed gateway router port. We can delete + * the old gateway_chassis code once ovn-controller supports + * HA chassis group. */ + const struct sbrec_ha_chassis_group *sb_ha_chassis_group = + ha_chassis_group_lookup_by_name( + ctx->sbrec_ha_chassis_grp_by_name, lrp->name); + if (!sb_ha_chassis_group) { + sb_ha_chassis_group = sbrec_ha_chassis_group_insert(ctx->ovnsb_txn); + sbrec_ha_chassis_group_set_name(sb_ha_chassis_group, lrp->name); + } + + struct sbrec_ha_chassis **sb_ha_chassis = xcalloc(lrp->n_gateway_chassis, + sizeof *sb_ha_chassis); + size_t n_sb_ha_ch = 0; /* XXX: This can be improved. This code will generate a set of new * Gateway_Chassis and push them all in a single transaction, instead * this would be more optimal if we just add/update/remove the rows in * the southbound db that need to change. We don't expect lots of * changes to the Gateway_Chassis table, but if that proves to be wrong - * we should optimize this. */ + * we should optimize this. + * + * Note: Remove the below code to add gateway_chassis row in OVN + * Southbound db once ovn-controller supports HA chassis group. */ for (n = 0; n < lrp->n_gateway_chassis; n++) { struct nbrec_gateway_chassis *lrp_gwc = lrp->gateway_chassis[n]; if (!lrp_gwc->chassis_name) { @@ -2086,6 +2144,8 @@ copy_gw_chassis_from_nbrp_to_sbpb( gw_chassis = xrealloc(gw_chassis, (n_gwc + 1) * sizeof *gw_chassis); + /* This code to create gateway_chassis in SB DB needs to be deleted + * once ovn-controller supports making use of HA chassis groups. */ struct sbrec_gateway_chassis *pb_gwc = sbrec_gateway_chassis_insert(ctx->ovnsb_txn); @@ -2096,16 +2156,26 @@ copy_gw_chassis_from_nbrp_to_sbpb( sbrec_gateway_chassis_set_external_ids(pb_gwc, &lrp_gwc->external_ids); gw_chassis[n_gwc++] = pb_gwc; + + sb_ha_chassis[n_sb_ha_ch] = + create_sb_ha_chassis(ctx, chassis, lrp_gwc->priority); + n_sb_ha_ch++; } sbrec_port_binding_set_gateway_chassis(port_binding, gw_chassis, n_gwc); free(gw_chassis); + + sbrec_ha_chassis_group_set_ha_chassis(sb_ha_chassis_group, + sb_ha_chassis, n_sb_ha_ch); + sbrec_port_binding_set_ha_chassis_group(port_binding, sb_ha_chassis_group); + free(sb_ha_chassis); } static void ovn_port_update_sbrec(struct northd_context *ctx, struct ovsdb_idl_index *sbrec_chassis_by_name, const struct ovn_port *op, - struct hmap *chassis_qdisc_queues) + struct hmap *chassis_qdisc_queues, + struct sset *active_ha_chassis_grps) { sbrec_port_binding_set_datapath(op->sb, op->od->sb); if (op->nbrp) { @@ -2142,6 +2212,7 @@ ovn_port_update_sbrec(struct northd_context *ctx, op->nbrp, op->sb); } + sset_add(active_ha_chassis_grps, op->nbrp->name); } else if (redirect_chassis) { /* Handle ports that had redirect-chassis option attached * to them, and for backwards compatibility convert them @@ -2152,20 +2223,23 @@ ovn_port_update_sbrec(struct northd_context *ctx, if (chassis) { /* If we found the chassis, and the gw chassis on record * differs from what we expect go ahead and update */ + char *gwc_name = xasprintf("%s_%s", op->nbrp->name, + chassis->name); if (op->sb->n_gateway_chassis != 1 || !op->sb->gateway_chassis[0]->chassis || strcmp(op->sb->gateway_chassis[0]->chassis->name, chassis->name) || op->sb->gateway_chassis[0]->priority != 0) { + /* This code to create gateway_chassis in SB DB needs + * to be deleted once ovn-controller supports making + * use of HA chassis groups. */ + /* Construct a single Gateway_Chassis entry on the * Port_Binding attached to the redirect_chassis * name */ struct sbrec_gateway_chassis *gw_chassis = sbrec_gateway_chassis_insert(ctx->ovnsb_txn); - char *gwc_name = xasprintf("%s_%s", op->nbrp->name, - chassis->name); - /* XXX: Again, here, we could just update an existing * Gateway_Chassis, instead of creating a new one * and replacing it */ @@ -2176,8 +2250,31 @@ ovn_port_update_sbrec(struct northd_context *ctx, &op->nbrp->external_ids); sbrec_port_binding_set_gateway_chassis(op->sb, &gw_chassis, 1); - free(gwc_name); } + + /* Create HA chassis group in SB DB for the + * redirect-chassis option. */ + const struct sbrec_ha_chassis_group *sb_ha_ch_grp; + sb_ha_ch_grp = ha_chassis_group_lookup_by_name( + ctx->sbrec_ha_chassis_grp_by_name, gwc_name); + if (!sb_ha_ch_grp) { + sb_ha_ch_grp = + sbrec_ha_chassis_group_insert(ctx->ovnsb_txn); + sbrec_ha_chassis_group_set_name(sb_ha_ch_grp, + gwc_name); + } + + if (sb_ha_ch_grp->n_ha_chassis != 1) { + struct sbrec_ha_chassis **sb_ha_ch = + xcalloc(1, sizeof *sb_ha_ch); + sb_ha_ch[0] = create_sb_ha_chassis(ctx, chassis, 0); + sbrec_ha_chassis_group_set_ha_chassis(sb_ha_ch_grp, + sb_ha_ch, 1); + } + sbrec_port_binding_set_ha_chassis_group(op->sb, + sb_ha_ch_grp); + sset_add(active_ha_chassis_grps, gwc_name); + free(gwc_name); } else { VLOG_WARN("chassis name '%s' from redirect from logical " " router port '%s' redirect-chassis not found", @@ -2348,6 +2445,18 @@ cleanup_mac_bindings(struct northd_context *ctx, struct hmap *ports) } } +static void +cleanup_sb_ha_chassis_groups(struct northd_context *ctx, + struct sset *active_ha_chassis_groups) +{ + const struct sbrec_ha_chassis_group *b, *n; + SBREC_HA_CHASSIS_GROUP_FOR_EACH_SAFE (b, n, ctx->ovnsb_idl) { + if (!sset_contains(active_ha_chassis_groups, b->name)) { + sbrec_ha_chassis_group_delete(b); + } + } +} + /* Updates the southbound Port_Binding table so that it contains the logical * switch ports specified by the northbound database. * @@ -2363,6 +2472,10 @@ build_ports(struct northd_context *ctx, struct hmap tag_alloc_table = HMAP_INITIALIZER(&tag_alloc_table); struct hmap chassis_qdisc_queues = HMAP_INITIALIZER(&chassis_qdisc_queues); + /* sset which stores the set of ha chassis group names used. */ + struct sset active_ha_chassis_grps = + SSET_INITIALIZER(&active_ha_chassis_grps); + join_logical_ports(ctx, datapaths, ports, &chassis_qdisc_queues, &tag_alloc_table, &sb_only, &nb_only, &both); @@ -2376,8 +2489,8 @@ build_ports(struct northd_context *ctx, tag_alloc_create_new_tag(&tag_alloc_table, op->nbsp); } ovn_port_update_sbrec(ctx, sbrec_chassis_by_name, - op, &chassis_qdisc_queues); - + op, &chassis_qdisc_queues, + &active_ha_chassis_grps); add_tnlid(&op->od->port_tnlids, op->sb->tunnel_key); if (op->sb->tunnel_key > op->od->port_key_hint) { op->od->port_key_hint = op->sb->tunnel_key; @@ -2393,8 +2506,8 @@ build_ports(struct northd_context *ctx, op->sb = sbrec_port_binding_insert(ctx->ovnsb_txn); ovn_port_update_sbrec(ctx, sbrec_chassis_by_name, op, - &chassis_qdisc_queues); - + &chassis_qdisc_queues, + &active_ha_chassis_grps); sbrec_port_binding_set_logical_port(op->sb, op->key); sbrec_port_binding_set_tunnel_key(op->sb, tunnel_key); } @@ -2416,6 +2529,8 @@ build_ports(struct northd_context *ctx, tag_alloc_destroy(&tag_alloc_table); destroy_chassis_queues(&chassis_qdisc_queues); + cleanup_sb_ha_chassis_groups(ctx, &active_ha_chassis_grps); + sset_destroy(&active_ha_chassis_grps); } #define OVN_MIN_MULTICAST 32768 @@ -4046,6 +4161,108 @@ build_stateful(struct ovn_datapath *od, struct hmap *lflows) } } +static void +build_lrouter_groups__(struct hmap *ports, struct ovn_datapath *od) +{ + ovs_assert((od && od->nbr && od->lr_group)); + + if (od->l3dgw_port && od->l3redirect_port) { + /* It's a logical router with gateway port. If it + * has HA_Chassis_Group associated to it in SB DB, then store the + * ha chassis group name. */ + if (od->l3redirect_port->sb->ha_chassis_group) { + sset_add(&od->lr_group->ha_chassis_groups, + od->l3redirect_port->sb->ha_chassis_group->name); + } + } + + for (size_t i = 0; i < od->nbr->n_ports; i++) { + struct ovn_port *router_port = + ovn_port_find(ports, od->nbr->ports[i]->name); + + if (!router_port || !router_port->peer) { + continue; + } + + /* Get the peer logical switch/logical router datapath. */ + struct ovn_datapath *peer_dp = router_port->peer->od; + if (peer_dp->nbr) { + if (!peer_dp->lr_group) { + peer_dp->lr_group = od->lr_group; + od->lr_group->router_dps[od->lr_group->n_router_dps++] + = peer_dp; + build_lrouter_groups__(ports, peer_dp); + } + } else { + for (size_t j = 0; j < peer_dp->n_router_ports; j++) { + if (!peer_dp->router_ports[j]->peer) { + /* If there is no peer port connecting to the + * router datapath, ignore it. */ + continue; + } + + struct ovn_datapath *router_dp; + router_dp = peer_dp->router_ports[j]->peer->od; + if (router_dp == od) { + continue; + } + + if (router_dp->lr_group == od->lr_group) { + /* 'router_dp' and 'od' already belong to the same + * lrouter group. Nothing to be done. */ + continue; + } + + router_dp->lr_group = od->lr_group; + od->lr_group->router_dps[od->lr_group->n_router_dps++] + = router_dp; + build_lrouter_groups__(ports, router_dp); + } + } + } +} + +/* Adds each logical router into a logical router group. All the + * logical routers which belong to a group are connected to + * each other either directly or indirectly (via transit logical switches + * in between). + * + * Suppose if 'lr_list' has lr0, lr1, lr2, lr3, lr4, lr5 + * and the topology is like + * sw0 <-> lr0 <-> sw1 <-> lr1 <->sw2 <-> lr2 + * sw3 <-> lr3 <-> lr4 <-> sw5 + * sw6 <-> lr5 <-> sw7 + * Then 3 groups are created. + * Group 1 -> lr0, lr1 and lr2 + * lr0, lr1 and lr2's ovn_datapath->lr_group will point to this + * group. This means sw0's logical ports can send packets to sw2's + * logical ports if proper static route's are added. + * Group 2 -> lr3 and lr4 + * lr3 and lr4's ovn_datapath->lr_group will point to this group. + * Group 3 -> lr5 + * + * Each logical router can belong to only one group. + */ +static void +build_lrouter_groups(struct hmap *ports, struct ovs_list *lr_list) +{ + struct ovn_datapath *od; + size_t n_router_dps = ovs_list_size(lr_list); + + LIST_FOR_EACH (od, lr_list, lr_list) { + if (!od->lr_group) { + od->lr_group = xzalloc(sizeof *od->lr_group); + /* Each logical router group can have max + * 'n_router_dps'. So allocate enough memory. */ + od->lr_group->router_dps = xcalloc(sizeof *od, n_router_dps); + od->lr_group->router_dps[od->lr_group->n_router_dps] = od; + od->lr_group->n_router_dps = 1; + sset_init(&od->lr_group->ha_chassis_groups); + build_lrouter_groups__(ports, od); + } + } +} + static void build_lswitch_flows(struct hmap *datapaths, struct hmap *ports, struct hmap *port_groups, struct hmap *lflows, @@ -7221,8 +7438,27 @@ sync_dns_entries(struct northd_context *ctx, struct hmap *datapaths) } static void -destroy_datapaths_and_ports(struct hmap *datapaths, struct hmap *ports) +destroy_datapaths_and_ports(struct hmap *datapaths, struct hmap *ports, + struct ovs_list *lr_list) { + struct ovn_datapath *router_dp; + LIST_FOR_EACH_POP (router_dp, lr_list, lr_list) { + if (router_dp->lr_group) { + struct lrouter_group *lr_group = router_dp->lr_group; + + for (size_t i = 0; i < router_dp->lr_group->n_router_dps; i++) { + if (router_dp->lr_group->router_dps[i] != router_dp) { + router_dp->lr_group->router_dps[i]->lr_group = NULL; + } + } + + free(lr_group->router_dps); + sset_destroy(&lr_group->ha_chassis_groups); + free(lr_group); + router_dp->lr_group = NULL; + } + } + struct ovn_datapath *dp, *next_dp; HMAP_FOR_EACH_SAFE (dp, next_dp, key_node, datapaths) { ovn_datapath_destroy(datapaths, dp); @@ -7240,16 +7476,19 @@ static void ovnnb_db_run(struct northd_context *ctx, struct ovsdb_idl_index *sbrec_chassis_by_name, struct ovsdb_idl_loop *sb_loop, - struct hmap *datapaths, struct hmap *ports) + struct hmap *datapaths, struct hmap *ports, + struct ovs_list *lr_list) { if (!ctx->ovnsb_txn || !ctx->ovnnb_txn) { return; } struct hmap port_groups; - build_datapaths(ctx, datapaths); + + build_datapaths(ctx, datapaths, lr_list); build_ports(ctx, sbrec_chassis_by_name, datapaths, ports); build_ipam(datapaths, ports); build_port_group_lswitches(ctx, &port_groups, ports); + build_lrouter_groups(ports, lr_list); build_lflows(ctx, datapaths, ports, &port_groups); sync_address_sets(ctx); @@ -7309,13 +7548,131 @@ ovnnb_db_run(struct northd_context *ctx, cleanup_macam(&macam); } +/* Stores the list of chassis which references an ha_chassis_group. + */ +struct ha_ref_chassis_info { + const struct sbrec_ha_chassis_group *ha_chassis_group; + struct sbrec_chassis **ref_chassis; + size_t n_ref_chassis; + size_t free_slots; +}; + +static void +add_to_ha_ref_chassis_info(struct ha_ref_chassis_info *ref_ch_info, + const struct sbrec_chassis *chassis) +{ + for (size_t j = 0; j < ref_ch_info->n_ref_chassis; j++) { + if (ref_ch_info->ref_chassis[j] == chassis) { + return; + } + } + + /* Allocate space for 3 chassis at a time. */ + if (!ref_ch_info->free_slots) { + ref_ch_info->ref_chassis = + xrealloc(ref_ch_info->ref_chassis, + sizeof *ref_ch_info->ref_chassis * + (ref_ch_info->n_ref_chassis + 3)); + ref_ch_info->free_slots = 3; + } + + ref_ch_info->ref_chassis[ref_ch_info->n_ref_chassis] = + CONST_CAST(struct sbrec_chassis *, chassis); + ref_ch_info->n_ref_chassis++; + ref_ch_info->free_slots--; +} + +static void +update_sb_ha_group_ref_chassis(struct shash *ha_ref_chassis_map) +{ + struct shash_node *node, *next; + SHASH_FOR_EACH_SAFE (node, next, ha_ref_chassis_map) { + struct ha_ref_chassis_info *ha_ref_info = node->data; + sbrec_ha_chassis_group_set_ref_chassis(ha_ref_info->ha_chassis_group, + ha_ref_info->ref_chassis, + ha_ref_info->n_ref_chassis); + free(ha_ref_info->ref_chassis); + free(ha_ref_info); + shash_delete(ha_ref_chassis_map, node); + } +} + +/* This function checks if the port binding 'sb' references + * a HA chassis group. + * Eg. Suppose a distributed logical router port - lr0-public + * uses an HA chassis group - hagrp1 and if hagrp1 has 3 ha + * chassis - gw1, gw2 and gw3. + * Or + * If the distributed logical router port - lr0-public has + * 3 gateway chassis - gw1, gw2 and gw3. + * ovn-northd creates ha chassis group - hagrp1 in SB DB + * and adds gw1, gw2 and gw3 to its ha_chassis list. + * + * If port binding 'sb' represents a logical switch port 'p1' + * and its logical switch is connected to the logical router + * 'lr0' directly or indirectly (i.e p1's logical switch is + * connected to a router 'lr1' and 'lr1' has a path to lr0 via + * transit logical switches) and 'sb' is claimed by chassis - 'c1' then + * this function adds c1 to the list of the reference chassis + * - 'ref_chassis' of hagrp1. + */ +static void +build_ha_chassis_group_ref_chassis(struct northd_context *ctx, + const struct sbrec_port_binding *sb, + struct ovn_port *op, + struct shash *ha_ref_chassis_map) +{ + struct lrouter_group *lr_group = NULL; + for (size_t i = 0; i < op->od->n_router_ports; i++) { + if (!op->od->router_ports[i]->peer) { + continue; + } + + lr_group = op->od->router_ports[i]->peer->od->lr_group; + /* If a logical switch has multiple router ports, then + * all the logical routers belong to the same logical + * router group. */ + break; + } + + if (!lr_group) { + return; + } + + const char *ha_group_name; + SSET_FOR_EACH (ha_group_name, &lr_group->ha_chassis_groups) { + const struct sbrec_ha_chassis_group *sb_ha_chassis_grp; + sb_ha_chassis_grp = ha_chassis_group_lookup_by_name( + ctx->sbrec_ha_chassis_grp_by_name, ha_group_name); + + if (sb_ha_chassis_grp) { + struct ha_ref_chassis_info *ref_ch_info = + shash_find_data(ha_ref_chassis_map, sb_ha_chassis_grp->name); + ovs_assert(ref_ch_info); + add_to_ha_ref_chassis_info(ref_ch_info, sb->chassis); + } + } +} + /* Handle changes to the 'chassis' column of the 'Port_Binding' table. When * this column is not empty, it means we need to set the corresponding logical * port as 'up' in the northbound DB. */ static void -update_logical_port_status(struct northd_context *ctx, struct hmap *ports) +handle_port_binding_changes(struct northd_context *ctx, struct hmap *ports, + struct shash *ha_ref_chassis_map) { const struct sbrec_port_binding *sb; + bool build_ha_chassis_ref = false; + if (ctx->ovnsb_txn) { + const struct sbrec_ha_chassis_group *ha_ch_grp; + SBREC_HA_CHASSIS_GROUP_FOR_EACH (ha_ch_grp, ctx->ovnsb_idl) { + struct ha_ref_chassis_info *ref_ch_info = + xzalloc(sizeof *ref_ch_info); + ref_ch_info->ha_chassis_group = ha_ch_grp; + build_ha_chassis_ref = true; + shash_add(ha_ref_chassis_map, ha_ch_grp->name, ref_ch_info); + } + } SBREC_PORT_BINDING_FOR_EACH(sb, ctx->ovnsb_idl) { struct ovn_port *op = ovn_port_find(ports, sb->logical_port); @@ -7331,6 +7688,13 @@ update_logical_port_status(struct northd_context *ctx, struct hmap *ports) if (!op->nbsp->up || *op->nbsp->up != up) { nbrec_logical_switch_port_set_up(op->nbsp, &up, 1); } + + if (build_ha_chassis_ref && ctx->ovnsb_txn && sb->chassis) { + /* Check and add the chassis which has claimed this 'sb' + * to the ha chassis group's ref_chassis if required. */ + build_ha_chassis_group_ref_chassis(ctx, sb, op, + ha_ref_chassis_map); + } } } @@ -7660,8 +8024,13 @@ ovnsb_db_run(struct northd_context *ctx, struct ovsdb_idl_loop *sb_loop, return; } - update_logical_port_status(ctx, ports); + struct shash ha_ref_chassis_map = SHASH_INITIALIZER(&ha_ref_chassis_map); + handle_port_binding_changes(ctx, ports, &ha_ref_chassis_map); update_northbound_cfg(ctx, sb_loop); + if (ctx->ovnsb_txn) { + update_sb_ha_group_ref_chassis(&ha_ref_chassis_map); + } + shash_destroy(&ha_ref_chassis_map); } static void @@ -7670,12 +8039,14 @@ ovn_db_run(struct northd_context *ctx, struct ovsdb_idl_loop *ovnsb_idl_loop) { struct hmap datapaths, ports; + struct ovs_list lr_list; + ovs_list_init(&lr_list); hmap_init(&datapaths); hmap_init(&ports); ovnnb_db_run(ctx, sbrec_chassis_by_name, ovnsb_idl_loop, - &datapaths, &ports); + &datapaths, &ports, &lr_list); ovnsb_db_run(ctx, ovnsb_idl_loop, &ports); - destroy_datapaths_and_ports(&datapaths, &ports); + destroy_datapaths_and_ports(&datapaths, &ports, &lr_list); } static void @@ -7840,6 +8211,8 @@ main(int argc, char *argv[]) ovsdb_idl_add_column(ovnsb_idl_loop.idl, &sbrec_port_binding_col_chassis); ovsdb_idl_add_column(ovnsb_idl_loop.idl, &sbrec_port_binding_col_gateway_chassis); + ovsdb_idl_add_column(ovnsb_idl_loop.idl, + &sbrec_port_binding_col_ha_chassis_group); ovsdb_idl_add_column(ovnsb_idl_loop.idl, &sbrec_gateway_chassis_col_chassis); ovsdb_idl_add_column(ovnsb_idl_loop.idl, &sbrec_gateway_chassis_col_name); @@ -7904,9 +8277,30 @@ main(int argc, char *argv[]) ovsdb_idl_add_column(ovnsb_idl_loop.idl, &sbrec_chassis_col_nb_cfg); ovsdb_idl_add_column(ovnsb_idl_loop.idl, &sbrec_chassis_col_name); + ovsdb_idl_add_table(ovnsb_idl_loop.idl, &sbrec_table_ha_chassis); + add_column_noalert(ovnsb_idl_loop.idl, + &sbrec_ha_chassis_col_chassis); + add_column_noalert(ovnsb_idl_loop.idl, + &sbrec_ha_chassis_col_priority); + add_column_noalert(ovnsb_idl_loop.idl, + &sbrec_ha_chassis_col_external_ids); + + ovsdb_idl_add_table(ovnsb_idl_loop.idl, &sbrec_table_ha_chassis_group); + add_column_noalert(ovnsb_idl_loop.idl, + &sbrec_ha_chassis_group_col_name); + add_column_noalert(ovnsb_idl_loop.idl, + &sbrec_ha_chassis_group_col_ha_chassis); + add_column_noalert(ovnsb_idl_loop.idl, + &sbrec_ha_chassis_group_col_external_ids); + add_column_noalert(ovnsb_idl_loop.idl, + &sbrec_ha_chassis_group_col_ref_chassis); + struct ovsdb_idl_index *sbrec_chassis_by_name = chassis_index_create(ovnsb_idl_loop.idl); + struct ovsdb_idl_index *sbrec_ha_chassis_grp_by_name + = ha_chassis_group_index_create(ovnsb_idl_loop.idl); + /* Ensure that only a single ovn-northd is active in the deployment by * acquiring a lock called "ovn_northd" on the southbound database * and then only performing DB transactions if the lock is held. */ @@ -7921,6 +8315,7 @@ main(int argc, char *argv[]) .ovnnb_txn = ovsdb_idl_loop_run(&ovnnb_idl_loop), .ovnsb_idl = ovnsb_idl_loop.idl, .ovnsb_txn = ovsdb_idl_loop_run(&ovnsb_idl_loop), + .sbrec_ha_chassis_grp_by_name = sbrec_ha_chassis_grp_by_name, }; if (!had_lock && ovsdb_idl_has_lock(ovnsb_idl_loop.idl)) { diff --git a/ovn/ovn-nb.ovsschema b/ovn/ovn-nb.ovsschema index 10a59649a..48d27b960 100644 --- a/ovn/ovn-nb.ovsschema +++ b/ovn/ovn-nb.ovsschema @@ -1,7 +1,7 @@ { "name": "OVN_Northbound", - "version": "5.14.1", - "cksum": "3758097843 20509", + "version": "5.15.0", + "cksum": "1078795414 21917", "tables": { "NB_Global": { "columns": { @@ -271,6 +271,12 @@ "refType": "strong"}, "min": 0, "max": "unlimited"}}, + "ha_chassis_group": { + "type": {"key": {"type": "uuid", + "refTable": "HA_Chassis_Group", + "refType": "weak"}, + "min": 0, + "max": 1}}, "options": { "type": {"key": "string", "value": "string", @@ -392,5 +398,29 @@ "type": {"key": "string", "value": "string", "min": 0, "max": "unlimited"}}}, "indexes": [["name"]], - "isRoot": false}} + "isRoot": false}, + "HA_Chassis": { + "columns": { + "chassis_name": {"type": "string"}, + "priority": {"type": {"key": {"type": "integer", + "minInteger": 0, + "maxInteger": 32767}}}, + "external_ids": { + "type": {"key": "string", "value": "string", + "min": 0, "max": "unlimited"}}}, + "isRoot": false}, + "HA_Chassis_Group": { + "columns": { + "name": {"type": "string"}, + "ha_chassis": { + "type": {"key": {"type": "uuid", + "refTable": "HA_Chassis", + "refType": "strong"}, + "min": 0, + "max": "unlimited"}}, + "external_ids": { + "type": {"key": "string", "value": "string", + "min": 0, "max": "unlimited"}}}, + "indexes": [["name"]], + "isRoot": true}} } diff --git a/ovn/ovn-nb.xml b/ovn/ovn-nb.xml index 61a57110a..75d27a0eb 100644 --- a/ovn/ovn-nb.xml +++ b/ovn/ovn-nb.xml @@ -1522,6 +1522,12 @@ +

+ This column is ignored if the column + . + is set. +

+

If set, this indicates that this logical router port represents a distributed gateway port that connects this router to a logical @@ -1549,6 +1555,24 @@

+ +

+ If set, this indicates that this logical router port represents + a distributed gateway port that connects this router to a logical + switch with a localnet port. There may be at most one such + logical router port on each logical router. The HA chassis which + are part of the HA chassis group will provide the gateway high + availability. Please see the for + more details. +

+ +

+ When this column is set, the column + will + be ignored. +

+ +

The IP addresses and netmasks of the router. For example, @@ -2622,4 +2646,55 @@ + +

+ Table representing a group of chassis which can provide High availability + services. Each chassis in the group is represented by the table + . The HA chassis with highest priority will + be the master of this group. If the master chassis failover is detected, + the HA chassis with the next higher priority takes over the + responsibility of providing the HA. If a distributed gateway router port + references a row in this table, then the master HA chassis in this group + provides the gateway functionality. +

+ + + Name of the . Name should be unique. + + + + A list of HA chassis which belongs to this group. + + + + + See External IDs at the beginning of this document. + + +
+ + + +

+ Name of the chassis which is part of the HA chassis group. + The value must match the + column + of the table in the + database. +

+ + + +

+ Priority of the chassis. Chassis with highest priority will be + the master. +

+ + + + + See External IDs at the beginning of this document. + + +
diff --git a/ovn/ovn-sb.ovsschema b/ovn/ovn-sb.ovsschema index cc8c771a7..e05f964b0 100644 --- a/ovn/ovn-sb.ovsschema +++ b/ovn/ovn-sb.ovsschema @@ -1,7 +1,7 @@ { "name": "OVN_Southbound", - "version": "2.1.0", - "cksum": "3806083220 15332", + "version": "2.2.0", + "cksum": "2001312516 17220", "tables": { "SB_Global": { "columns": { @@ -147,6 +147,12 @@ "refType": "strong"}, "min": 0, "max": "unlimited"}}, + "ha_chassis_group": { + "type": {"key": {"type": "uuid", + "refTable": "HA_Chassis_Group", + "refType": "weak"}, + "min": 0, + "max": 1}}, "options": { "type": {"key": "string", "value": "string", @@ -309,4 +315,35 @@ "type": {"key": "string", "value": "string", "min": 0, "max": "unlimited"}}}, "indexes": [["name"]], - "isRoot": false}}} + "isRoot": false}, + "HA_Chassis": { + "columns": { + "chassis": {"type": {"key": {"type": "uuid", + "refTable": "Chassis", + "refType": "weak"}, + "min": 0, "max": 1}}, + "priority": {"type": {"key": {"type": "integer", + "minInteger": 0, + "maxInteger": 32767}}}, + "external_ids": { + "type": {"key": "string", "value": "string", + "min": 0, "max": "unlimited"}}}, + "isRoot": false}, + "HA_Chassis_Group": { + "columns": { + "name": {"type": "string"}, + "ha_chassis": { + "type": {"key": {"type": "uuid", + "refTable": "HA_Chassis", + "refType": "strong"}, + "min": 0, + "max": "unlimited"}}, + "ref_chassis": {"type": {"key": {"type": "uuid", + "refTable": "Chassis", + "refType": "weak"}, + "min": 0, "max": "unlimited"}}, + "external_ids": { + "type": {"key": "string", "value": "string", + "min": 0, "max": "unlimited"}}}, + "indexes": [["name"]], + "isRoot": true}}} diff --git a/ovn/ovn-sb.xml b/ovn/ovn-sb.xml index 4e080abff..5c4a852a5 100644 --- a/ovn/ovn-sb.xml +++ b/ovn/ovn-sb.xml @@ -2246,6 +2246,16 @@ tcp.flags = RST;

+ +

+ This should only be populated for ports with + set to chassisredirect. + This column defines the HA chassis group with a list of + HA chassis used as gateways where traffic will be redirected + through. +

+ +

A number that represents the logical port in the key (e.g. STT key or @@ -3339,4 +3349,57 @@ tcp.flags = RST; + + + +

+ The which provides the HA functionality. +

+ + + +

+ Priority of the HA chassis. Chassis with highest priority will be + the master in the HA chassis group. +

+ + + + + See External IDs at the beginning of this document. + + +
+ + +

+ Table representing a group of chassis which can provide High availability + services. Each chassis in the group is represented by the table + . The HA chassis with highest priority will + be the master of this group. If the master chassis failover is detected, + the HA chassis with the next higher priority takes over the + responsibility of providing the HA. If column of the table + references this table, + then this HA chassis group provides the gateway functionality and + redirects the gateway traffic to the master of this group. +

+ + Name of the . Name should be unique. + + + + A list of which belongs to this group. + + + + A list of which references this HA chassis group. + + + + + See External IDs at the beginning of this document. + + +
diff --git a/ovn/utilities/ovn-nbctl.8.xml b/ovn/utilities/ovn-nbctl.8.xml index a7a9c2701..483602970 100644 --- a/ovn/utilities/ovn-nbctl.8.xml +++ b/ovn/utilities/ovn-nbctl.8.xml @@ -908,6 +908,47 @@ +

HA Chassis Group commands

+ +
+
ha-chassis-group-add group
+
+ Creates a new HA chassis group in the HA_Chassis_Group + table named group. +
+ +
ha-chassis-group-del group
+
+ Deletes the HA chassis group group. It is an error if + group does not exist. +
+ +
ha-chassis-group-list
+
+ Lists the HA chassis group group along with the + HA chassis if any associated with it. +
+ +
ha-chassis-group-add-chassis group + chassis priority
+
+ Adds a new HA chassis chassis to the + HA Chassis group group with the specified priority. + If the chassis already exists, then the + priority is updated. + The chassis should be the name of the chassis in the + OVN_Southbound. +
+ +
ha-chassis-group-remove-chassis group + chassis
+
+ Removes the HA chassis chassis from the HA chassis + group group. It is an error if chassis does + not exist. +
+
+

Database Commands

These commands query and modify the contents of ovsdb tables. They are a slight abstraction of the ovsdb interface and diff --git a/ovn/utilities/ovn-nbctl.c b/ovn/utilities/ovn-nbctl.c index 2727b410a..a2954abd4 100644 --- a/ovn/utilities/ovn-nbctl.c +++ b/ovn/utilities/ovn-nbctl.c @@ -694,6 +694,15 @@ Port group commands:\n\ pg-set-ports PG PORTS Set PORTS on port group PG\n\ pg-del PG Delete port group PG\n\ \n\ +HA chassis group commands:\n\ + ha-chassis-group-add GRP Create an HA chassis group GRP\n\ + ha-chassis-group-del GRP Delete the HA chassis group GRP\n\ + ha-chassis-group-list List the HA chassis groups\n\ + ha-chassis-group-add-chassis GRP CHASSIS [PRIORITY] Adds an HA\ +chassis with optional PRIORITY to the HA chassis group GRP\n\ + ha-chassis-group-del-chassis GRP CHASSIS Deletes the HA chassis\ +CHASSIS from the HA chassis group GRP\n\ +\n\ %s\ %s\ \n\ @@ -4756,6 +4765,201 @@ cmd_pg_del(struct ctl_context *ctx) nbrec_port_group_delete(pg); } +static const struct nbrec_ha_chassis_group* +ha_chassis_group_by_name_or_uuid(struct ctl_context *ctx, const char *id, + bool must_exist) +{ + struct uuid ch_grp_uuid; + const struct nbrec_ha_chassis_group *ha_ch_grp = NULL; + bool is_uuid = uuid_from_string(&ch_grp_uuid, id); + if (is_uuid) { + ha_ch_grp = nbrec_ha_chassis_group_get_for_uuid(ctx->idl, + &ch_grp_uuid); + } + + if (!ha_ch_grp) { + const struct nbrec_ha_chassis_group *iter; + NBREC_HA_CHASSIS_GROUP_FOR_EACH (iter, ctx->idl) { + if (!strcmp(iter->name, id)) { + ha_ch_grp = iter; + break; + } + } + } + + if (!ha_ch_grp && must_exist) { + ctx->error = xasprintf("%s: ha_chassi_group %s not found", + id, is_uuid ? "UUID" : "name"); + } + + return ha_ch_grp; +} + +static void +cmd_ha_ch_grp_add(struct ctl_context *ctx) +{ + const char *name = ctx->argv[1]; + struct nbrec_ha_chassis_group *ha_ch_grp = + nbrec_ha_chassis_group_insert(ctx->txn); + nbrec_ha_chassis_group_set_name(ha_ch_grp, name); +} + +static void +cmd_ha_ch_grp_del(struct ctl_context *ctx) +{ + const char *name_or_id = ctx->argv[1]; + + const struct nbrec_ha_chassis_group *ha_ch_grp = + ha_chassis_group_by_name_or_uuid(ctx, name_or_id, true); + + if (ha_ch_grp) { + nbrec_ha_chassis_group_delete(ha_ch_grp); + } +} + +static void +cmd_ha_ch_grp_list(struct ctl_context *ctx) +{ + const struct nbrec_ha_chassis_group *ha_ch_grp; + + NBREC_HA_CHASSIS_GROUP_FOR_EACH (ha_ch_grp, ctx->idl) { + ds_put_format(&ctx->output, UUID_FMT " (%s)\n", + UUID_ARGS(&ha_ch_grp->header_.uuid), ha_ch_grp->name); + const struct nbrec_ha_chassis *ha_ch; + for (size_t i = 0; i < ha_ch_grp->n_ha_chassis; i++) { + ha_ch = ha_ch_grp->ha_chassis[i]; + ds_put_format(&ctx->output, + " "UUID_FMT " (%s)\n" + " priority %lu\n\n", + UUID_ARGS(&ha_ch->header_.uuid), ha_ch->chassis_name, + ha_ch->priority); + } + ds_put_cstr(&ctx->output, "\n"); + } +} + +static void +cmd_ha_ch_grp_add_chassis(struct ctl_context *ctx) +{ + const struct nbrec_ha_chassis_group *ha_ch_grp = + ha_chassis_group_by_name_or_uuid(ctx, ctx->argv[1], true); + + if (!ha_ch_grp) { + return; + } + + const char *chassis_name = ctx->argv[2]; + int64_t priority; + char *error = parse_priority(ctx->argv[3], &priority); + if (error) { + ctx->error = error; + return; + } + + struct nbrec_ha_chassis *ha_chassis = NULL; + for (size_t i = 0; i < ha_ch_grp->n_ha_chassis; i++) { + if (!strcmp(ha_ch_grp->ha_chassis[i]->chassis_name, chassis_name)) { + ha_chassis = ha_ch_grp->ha_chassis[i]; + break; + } + } + + if (ha_chassis) { + nbrec_ha_chassis_set_priority(ha_chassis, priority); + return; + } + + ha_chassis = nbrec_ha_chassis_insert(ctx->txn); + nbrec_ha_chassis_set_chassis_name(ha_chassis, chassis_name); + nbrec_ha_chassis_set_priority(ha_chassis, priority); + + nbrec_ha_chassis_group_verify_ha_chassis(ha_ch_grp); + + struct nbrec_ha_chassis **new_ha_chs = + xmalloc(sizeof *new_ha_chs * (ha_ch_grp->n_ha_chassis + 1)); + nullable_memcpy(new_ha_chs, ha_ch_grp->ha_chassis, + sizeof *new_ha_chs * ha_ch_grp->n_ha_chassis); + new_ha_chs[ha_ch_grp->n_ha_chassis] = + CONST_CAST(struct nbrec_ha_chassis *, ha_chassis); + nbrec_ha_chassis_group_set_ha_chassis(ha_ch_grp, new_ha_chs, + ha_ch_grp->n_ha_chassis + 1); + free(new_ha_chs); +} + +static void +cmd_ha_ch_grp_remove_chassis(struct ctl_context *ctx) +{ + const struct nbrec_ha_chassis_group *ha_ch_grp = + ha_chassis_group_by_name_or_uuid(ctx, ctx->argv[1], true); + + if (!ha_ch_grp) { + return; + } + + const char *chassis_name = ctx->argv[2]; + struct nbrec_ha_chassis *ha_chassis = NULL; + size_t idx = 0; + for (size_t i = 0; i < ha_ch_grp->n_ha_chassis; i++) { + if (!strcmp(ha_ch_grp->ha_chassis[i]->chassis_name, chassis_name)) { + ha_chassis = ha_ch_grp->ha_chassis[i]; + idx = i; + break; + } + } + + if (!ha_chassis) { + ctx->error = xasprintf("%s: ha chassis not found in %s ha " + "chassis group", chassis_name, ctx->argv[1]); + return; + } + + struct nbrec_ha_chassis **new_ha_ch + = xmemdup(ha_ch_grp->ha_chassis, + sizeof *new_ha_ch * ha_ch_grp->n_ha_chassis); + new_ha_ch[idx] = new_ha_ch[ha_ch_grp->n_ha_chassis - 1]; + nbrec_ha_chassis_group_verify_ha_chassis(ha_ch_grp); + nbrec_ha_chassis_group_set_ha_chassis(ha_ch_grp, new_ha_ch, + ha_ch_grp->n_ha_chassis - 1); + free(new_ha_ch); + nbrec_ha_chassis_delete(ha_chassis); +} + +static void +cmd_ha_ch_grp_set_chassis_prio(struct ctl_context *ctx) +{ + const struct nbrec_ha_chassis_group *ha_ch_grp = + ha_chassis_group_by_name_or_uuid(ctx, ctx->argv[1], true); + + if (!ha_ch_grp) { + return; + } + + int64_t priority; + char *error = parse_priority(ctx->argv[3], &priority); + if (error) { + ctx->error = error; + return; + } + + const char *chassis_name = ctx->argv[2]; + struct nbrec_ha_chassis *ha_chassis = NULL; + + for (size_t i = 0; i < ha_ch_grp->n_ha_chassis; i++) { + if (!strcmp(ha_ch_grp->ha_chassis[i]->chassis_name, chassis_name)) { + ha_chassis = ha_ch_grp->ha_chassis[i]; + break; + } + } + + if (!ha_chassis) { + ctx->error = xasprintf("%s: ha chassis not found in %s ha " + "chassis group", chassis_name, ctx->argv[1]); + return; + } + + nbrec_ha_chassis_set_priority(ha_chassis, priority); +} + static const struct ctl_table_class tables[NBREC_N_TABLES] = { [NBREC_TABLE_DHCP_OPTIONS].row_ids = {{&nbrec_logical_switch_port_col_name, NULL, @@ -4790,6 +4994,9 @@ static const struct ctl_table_class tables[NBREC_N_TABLES] = { = {&nbrec_port_group_col_name, NULL, NULL}, [NBREC_TABLE_ACL].row_ids[0] = {&nbrec_acl_col_name, NULL, NULL}, + + [NBREC_TABLE_HA_CHASSIS_GROUP].row_ids[0] + = {&nbrec_ha_chassis_group_col_name, NULL, NULL}, }; static char * @@ -5230,6 +5437,20 @@ static const struct ctl_command_syntax nbctl_commands[] = { {"pg-set-ports", 2, INT_MAX, "", NULL, cmd_pg_set_ports, NULL, "", RW }, {"pg-del", 1, 1, "", NULL, cmd_pg_del, NULL, "", RW }, + /* HA chassis group commands. */ + {"ha-chassis-group-add", 1, 1, "[CHASSIS GROUP]", NULL, + cmd_ha_ch_grp_add, NULL, "", RW }, + {"ha-chassis-group-del", 1, 1, "[CHASSIS GROUP]", NULL, + cmd_ha_ch_grp_del, NULL, "", RW }, + {"ha-chassis-group-list", 0, 0, "[CHASSIS GROUP]", NULL, + cmd_ha_ch_grp_list, NULL, "", RO }, + {"ha-chassis-group-add-chassis", 3, 3, "[CHASSIS GROUP]", NULL, + cmd_ha_ch_grp_add_chassis, NULL, "", RW }, + {"ha-chassis-group-remove-chassis", 2, 2, "[CHASSIS GROUP]", NULL, + cmd_ha_ch_grp_remove_chassis, NULL, "", RW }, + {"ha-chassis-group-set-chassis-prio", 3, 3, "[CHASSIS GROUP]", NULL, + cmd_ha_ch_grp_set_chassis_prio, NULL, "", RW }, + {NULL, 0, 0, NULL, NULL, NULL, NULL, "", RO}, }; diff --git a/ovn/utilities/ovn-sbctl.c b/ovn/utilities/ovn-sbctl.c index ee97a4710..c5ff9318f 100644 --- a/ovn/utilities/ovn-sbctl.c +++ b/ovn/utilities/ovn-sbctl.c @@ -1189,6 +1189,12 @@ static const struct ctl_table_class tables[SBREC_N_TABLES] = { [SBREC_TABLE_ADDRESS_SET].row_ids[0] = {&sbrec_address_set_col_name, NULL, NULL}, + + [SBREC_TABLE_HA_CHASSIS_GROUP].row_ids[0] + = {&sbrec_ha_chassis_group_col_name, NULL, NULL}, + + [SBREC_TABLE_HA_CHASSIS].row_ids[0] + = {&sbrec_ha_chassis_col_chassis, NULL, NULL}, }; diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at index 1878eb2df..b4e8995be 100644 --- a/tests/ovn-northd.at +++ b/tests/ovn-northd.at @@ -34,6 +34,33 @@ AT_CHECK([ovn-sbctl --bare --columns gateway_chassis find port_binding logical_p AT_CHECK([ovn-sbctl --bare --columns gateway_chassis find port_binding logical_port="cr-alice" | grep $gwc2_uuid | wc -l], [0], [1 ]) +# There should be one ha_chassis_group with the name "alice" +ha_chassi_grp_name=`ovn-sbctl --bare --columns name find \ +ha_chassis_group name="alice"` + +AT_CHECK([test $ha_chassi_grp_name = alice]) + +ha_chgrp_uuid=`ovn-sbctl --bare --columns _uuid find ha_chassis_group name=alice` + +AT_CHECK([ovn-sbctl --bare --columns ha_chassis_group find port_binding \ +logical_port="cr-alice" | grep $ha_chgrp_uuid | wc -l], [0], [1 +]) + +ha_ch=`ovn-sbctl --bare --columns ha_chassis find ha_chassis_group` +# Trim the spaces. +ha_ch=`echo $ha_ch | sed 's/ //g'` + +ha_ch_list='' +for i in `ovn-sbctl --bare --columns _uuid list ha_chassis | sort` +do + ha_ch_list="$ha_ch_list $i" +done + +# Trim the spaces. +ha_ch_list=`echo $ha_ch_list | sed 's/ //g'` + +AT_CHECK([test "$ha_ch_list" = "$ha_ch"]) + # delete the 2nd Gateway_Chassis on NBDB for alice port ovn-nbctl --wait=sb set Logical_Router_Port alice gateway_chassis=${nb_gwc1_uuid} @@ -45,6 +72,10 @@ AT_CHECK([ovn-sbctl --bare --columns gateway_chassis find port_binding logical_p AT_CHECK([ovn-sbctl find Gateway_Chassis name=alice_gw2], [0], []) +# There should be only 1 row in ha_chassis SB DB table. +AT_CHECK([ovn-sbctl --bare --columns _uuid list ha_chassis | wc -l], [0], [1 +]) + # delete all the gateway_chassis on NBDB for alice port ovn-nbctl --wait=sb clear Logical_Router_Port alice gateway_chassis @@ -54,6 +85,12 @@ ovn-nbctl --wait=sb clear Logical_Router_Port alice gateway_chassis AT_CHECK([ovn-sbctl find Gateway_Chassis name=alice_gw1], [0], []) AT_CHECK([ovn-sbctl find Gateway_Chassis name=alice_gw2], [0], []) +# expect that the ha_chassis doesn't exist anymore +AT_CHECK([ovn-sbctl list ha_chassis | wc -l], [0], [0 +]) +AT_CHECK([ovn-sbctl list ha_chassis_group | wc -l], [0], [0 +]) + AT_CLEANUP AT_SETUP([ovn -- check Gateway_Chassis propagation from NBDB to SBDB backwards compatibility]) @@ -301,3 +338,228 @@ as northd OVS_APP_EXIT_AND_WAIT([ovn-northd]) AT_CLEANUP + +AT_SETUP([ovn -- check HA_Chassis_Group propagation from NBDB to SBDB]) +AT_SKIP_IF([test $HAVE_PYTHON = no]) +ovn_start + +ovn-nbctl --wait=sb ha-chassis-group-add hagrp1 + +# ovn-northd should not create HA chassis group and HA chassis rows +# unless the HA chassis group in OVN NB DB is associated to +# a logical router port. ovn-northd still doesn't support +# associating a HA chassis group to a logical router port. +AT_CHECK([ovn-sbctl --bare --columns name find ha_chassis_group name="hagrp1" \ +| wc -l], [0], [0 +]) + +ovn-nbctl --wait=sb ha-chassis-group-add-chassis hagrp1 ch1 30 +ovn-nbctl --wait=sb ha-chassis-group-add-chassis hagrp1 ch2 20 +ovn-nbctl --wait=sb ha-chassis-group-add-chassis hagrp1 ch3 10 + +# There should be no HA_Chassis rows in SB DB. +AT_CHECK([ovn-sbctl list ha_chassis | grep chassis | awk '{print $3}' \ +| grep -v '-' | wc -l ], [0], [0 +]) + +# Add chassis ch1. +ovn-sbctl chassis-add ch1 geneve 127.0.0.2 + +OVS_WAIT_UNTIL([test 1 = `ovn-sbctl list chassis | grep ch1 | wc -l`]) + +# There should be no HA_Chassis rows +AT_CHECK([ovn-sbctl list ha_chassis | grep chassis | awk '{print $3}' \ +| grep -v '-' | wc -l ], [0], [0 +]) + +ovn-nbctl ha-chassis-group-del hagrp1 +OVS_WAIT_UNTIL([test 0 = `ovn-sbctl list ha_chassis_group | wc -l`]) +OVS_WAIT_UNTIL([test 0 = `ovn-sbctl list ha_chassis | wc -l`]) + +# Create a logical router port and attach Gateway chassis. +# ovn-northd should create HA chassis group rows in SB DB. +ovn-nbctl lr-add lr0 + +ovn-nbctl lrp-add lr0 lr0-public 00:00:20:20:12:13 172.168.0.100/24 +ovn-nbctl lrp-set-gateway-chassis lr0-public ch1 20 + +OVS_WAIT_UNTIL([test 1 = `ovn-sbctl --bare --columns name find ha_chassis_group \ +name="lr0-public" | wc -l`]) + +OVS_WAIT_UNTIL([test 1 = `ovn-sbctl --bare --columns _uuid \ +find ha_chassis | wc -l`]) + +ovn-nbctl lrp-set-gateway-chassis lr0-public ch2 10 + +ovn-sbctl --bare --columns _uuid find ha_chassis +OVS_WAIT_UNTIL([test 2 = `ovn-sbctl list ha_chassis | grep chassis | wc -l`]) + +# Test if 'ref_chassis' column is properly set or not in +# SB DB ha_chassis_group. +ovn-nbctl ls-add sw0 +ovn-nbctl lsp-add sw0 sw0-p1 + +ovn-sbctl chassis-add ch2 geneve 127.0.0.3 +ovn-sbctl chassis-add ch3 geneve 127.0.0.4 +ovn-sbctl chassis-add comp1 geneve 127.0.0.5 +ovn-sbctl chassis-add comp2 geneve 127.0.0.6 + +ovn-nbctl lrp-add lr0 lr0-sw0 00:00:20:20:12:14 10.0.0.1/24 +ovn-nbctl lsp-add sw0 sw0-lr0 +ovn-nbctl lsp-set-type sw0-lr0 router +ovn-nbctl lsp-set-addresses sw0-lr0 router +ovn-nbctl lsp-set-options sw0-lr0 router-port=lr0-sw0 + +ovn-sbctl lsp-bind sw0-p1 comp1 +OVS_WAIT_UNTIL([test x`ovn-nbctl lsp-get-up sw0-p1` = xup]) + +comp1_ch_uuid=`ovn-sbctl --bare --columns _uuid find chassis name="comp1"` +comp2_ch_uuid=`ovn-sbctl --bare --columns _uuid find chassis name="comp2"` +ch2_ch_uuid=`ovn-sbctl --bare --columns _uuid find chassis name="comp1"` + +echo "comp1_ch_uuid = $comp1_ch_uuid" +OVS_WAIT_UNTIL( + [ref_ch_list=`ovn-sbctl --bare --columns ref_chassis find ha_chassis_group | sort` + # Trim the spaces. + ref_ch_list=`echo $ref_ch_list | sed 's/ //g'` + test "$comp1_ch_uuid" = "$ref_ch_list"]) + +# unbind sw0-p1 +ovn-sbctl lsp-unbind sw0-p1 +OVS_WAIT_UNTIL([test x`ovn-nbctl lsp-get-up sw0-p1` = xdown]) +OVS_WAIT_UNTIL( + [ref_ch_list=`ovn-sbctl --bare --columns ref_chassis find ha_chassis_group | sort` + # Trim the spaces. + ref_ch_list=`echo $ref_ch_list | sed 's/ //g'` + test "" = "$ref_ch_list"]) + +# Bind sw0-p1 in comp2 +ovn-sbctl lsp-bind sw0-p1 comp2 +OVS_WAIT_UNTIL( + [ref_ch_list=`ovn-sbctl --bare --columns ref_chassis find ha_chassis_group | sort` + # Trim the spaces. + ref_ch_list=`echo $ref_ch_list | sed 's/ //g'` + test "$comp2_ch_uuid" = "$ref_ch_list"]) + +ovn-nbctl ls-add sw1 +ovn-nbctl lsp-add sw1 sw1-p1 +ovn-nbctl lr-add lr1 +ovn-nbctl lrp-add lr1 lr1-sw1 00:00:20:20:12:15 20.0.0.1/24 +ovn-nbctl lsp-add sw1 sw1-lr1 +ovn-nbctl lsp-set-type sw1-lr1 router +ovn-nbctl lsp-set-addresses sw1-lr1 router +ovn-nbctl lsp-set-options sw1-lr1 router-port=lr1-sw1 + +# Bind sw1-p1 in comp1. +ovn-sbctl lsp-bind sw1-p1 comp1 +# Wait until sw1-p1 is up +OVS_WAIT_UNTIL([test x`ovn-nbctl lsp-get-up sw1-p1` = xup]) + +# sw1-p1 is not connected to lr0. So comp1 should not be in 'ref_chassis' +OVS_WAIT_UNTIL( + [ref_ch_list=`ovn-sbctl --bare --columns ref_chassis find ha_chassis_group | sort` + # Trim the spaces. + ref_ch_list=`echo $ref_ch_list | sed 's/ //g'` + test "$comp2_ch_uuid" = "$ref_ch_list"]) + +# Now attach sw0 to lr1 +ovn-nbctl lrp-add lr1 lr1-sw0 00:00:20:20:12:16 10.0.0.10/24 +ovn-nbctl lsp-add sw0 sw0-lr1 +ovn-nbctl lsp-set-type sw0-lr1 router +ovn-nbctl lsp-set-addresses sw0-lr1 router +ovn-nbctl lsp-set-options sw0-lr1 router-port=lr1-sw0 + +# Both comp1 and comp2 should be in 'ref_chassis' as sw1 is indirectly +# connected to lr0 +exp_ref_ch_list='' +for i in `ovn-sbctl --bare --columns _uuid list chassis | sort` +do + if test $i = $comp1_ch_uuid; then + exp_ref_ch_list="${exp_ref_ch_list}$i" + elif test $i = $comp2_ch_uuid; then + exp_ref_ch_list="${exp_ref_ch_list}$i" + fi +done + +OVS_WAIT_UNTIL( + [ref_ch_list=`ovn-sbctl --bare --columns ref_chassis find ha_chassis_group | sort` + # Trim the spaces. + ref_ch_list=`echo $ref_ch_list | sed 's/ //g'` + test "$exp_ref_ch_list" = "$ref_ch_list"]) + +# Unind sw1-p1. comp2 should not be in the ref_chassis. +ovn-sbctl lsp-unbind sw1-p1 +OVS_WAIT_UNTIL([test x`ovn-nbctl lsp-get-up sw1-p1` = xdown]) +OVS_WAIT_UNTIL( + [ref_ch_list=`ovn-sbctl --bare --columns ref_chassis find ha_chassis_group | sort` + # Trim the spaces. + ref_ch_list=`echo $ref_ch_list | sed 's/ //g'` + test "$comp2_ch_uuid" = "$ref_ch_list"]) + +# Create sw2 and attach it to lr2 +ovn-nbctl ls-add sw2 +ovn-nbctl lsp-add sw2 sw2-p1 +ovn-nbctl lr-add lr2 +ovn-nbctl lrp-add lr2 lr2-sw2 00:00:20:20:12:17 30.0.0.1/24 +ovn-nbctl lsp-add sw2 sw2-lr2 +ovn-nbctl lsp-set-type sw2-lr2 router +ovn-nbctl lsp-set-addresses sw2-lr2 router +ovn-nbctl lsp-set-options sw2-lr2 router-port=lr2-sw2 + +# Bind sw2-p1 to comp1 +ovn-sbctl lsp-bind sw2-p1 comp1 +# Wait until sw2-p1 is up +OVS_WAIT_UNTIL([test x`ovn-nbctl lsp-get-up sw2-p1` = xup]) + +# sw2-p1 is not connected to lr0. So comp1 should not be in 'ref_chassis' +OVS_WAIT_UNTIL( + [ref_ch_list=`ovn-sbctl --bare --columns ref_chassis find ha_chassis_group | sort` + # Trim the spaces. + ref_ch_list=`echo $ref_ch_list | sed 's/ //g'` + test "$comp2_ch_uuid" = "$ref_ch_list"]) + +# Now attach sw1 to lr2. With this sw2-p1 is indirectly connected to lr0. +ovn-nbctl lrp-add lr2 lr2-sw1 00:00:20:20:12:18 20.0.0.10/24 +ovn-nbctl lsp-add sw1 sw1-lr2 +ovn-nbctl lsp-set-type sw1-lr2 router +ovn-nbctl lsp-set-addresses sw1-lr2 router +ovn-nbctl lsp-set-options sw1-lr2 router-port=lr2-sw1 + +# sw2-p1 is indirectly connected to lr0. So comp1 (and comp2) should be in +# 'ref_chassis' +OVS_WAIT_UNTIL( + [ref_ch_list=`ovn-sbctl --bare --columns ref_chassis find ha_chassis_group | sort` + # Trim the spaces. + ref_ch_list=`echo $ref_ch_list | sed 's/ //g'` + test "$exp_ref_ch_list" = "$ref_ch_list"]) + +# Create sw0-p2 and bind it to comp1 +ovn-nbctl lsp-add sw0 sw0-p2 +ovn-sbctl lsp-bind sw0-p2 comp1 +OVS_WAIT_UNTIL([test x`ovn-nbctl lsp-get-up sw0-p2` = xup]) +OVS_WAIT_UNTIL( + [ref_ch_list=`ovn-sbctl --bare --columns ref_chassis find ha_chassis_group | sort` + # Trim the spaces. + ref_ch_list=`echo $ref_ch_list | sed 's/ //g'` + test "$exp_ref_ch_list" = "$ref_ch_list"]) + +# unbind sw0-p2 +ovn-sbctl lsp-unbind sw0-p2 +OVS_WAIT_UNTIL([test x`ovn-nbctl lsp-get-up sw0-p2` = xdown]) +OVS_WAIT_UNTIL( + [ref_ch_list=`ovn-sbctl --bare --columns ref_chassis find ha_chassis_group | sort` + # Trim the spaces. + ref_ch_list=`echo $ref_ch_list | sed 's/ //g'` + test "$exp_ref_ch_list" = "$ref_ch_list"]) + +# Delete lr1-sw0. comp1 should be deleted from ref_chassis as there is no link +# from sw1 and sw2 to lr0. +ovn-nbctl lrp-del lr1-sw0 + +OVS_WAIT_UNTIL( + [ref_ch_list=`ovn-sbctl --bare --columns ref_chassis find ha_chassis_group | sort` + # Trim the spaces. + ref_ch_list=`echo $ref_ch_list | sed 's/ //g'` + test "$comp2_ch_uuid" = "$ref_ch_list"]) + +AT_CLEANUP diff --git a/tests/ovn.at b/tests/ovn.at index f2f2bc405..04a618801 100644 --- a/tests/ovn.at +++ b/tests/ovn.at @@ -8398,6 +8398,16 @@ as ext1 ovs-vsctl set open . external-ids:ovn-bridge-mappings=phys:br-phys AT_CHECK([ovn-nbctl --timeout=3 --wait=sb sync], [0], [ignore]) +# hv1 should be in 'ref_chassis' of the ha_chasssi_group as logical +# switch 'foo' can reach the router 'R1' (which has gw router port) +# via foo1 -> foo -> R0 -> join -> R1 +hv1_ch_uuid=`ovn-sbctl --bare --columns _uuid find chassis name="hv1"` +OVS_WAIT_UNTIL( + [ref_ch_list=`ovn-sbctl --bare --columns ref_chassis find ha_chassis_group | sort` + # Trim the spaces. + ref_ch_list=`echo $ref_ch_list | sed 's/ //g'` + test "$hv1_ch_uuid" = "$ref_ch_list"]) + # Allow some time for ovn-northd and ovn-controller to catch up. # XXX This should be more systematic. sleep 2 @@ -9766,6 +9776,32 @@ echo "------ Port_Binding chassisredirect -------" ovn-sbctl find Port_Binding type=chassisredirect echo "-------------------------------------------" +# There should be one ha_chassis_group with the name "outside" +ha_chassi_grp_name=`ovn-sbctl --bare --columns name find \ +ha_chassis_group name="outside"` + +AT_CHECK([test $ha_chassi_grp_name = outside]) + +# There should be 2 ha_chassis rows in SB DB. +AT_CHECK([ovn-sbctl list ha_chassis | grep chassis | awk '{print $3}' \ +| grep '-' | wc -l ], [0], [2 +]) + +ha_ch=`ovn-sbctl --bare --columns ha_chassis find ha_chassis_group` +# Trim the spaces. +ha_ch=`echo $ha_ch | sed 's/ //g'` + +ha_ch_list='' +for i in `ovn-sbctl --bare --columns _uuid list ha_chassis | sort` +do + ha_ch_list="$ha_ch_list $i" +done + +# Trim the spaces. +ha_ch_list=`echo $ha_ch_list | sed 's/ //g'` + +AT_CHECK([test "$ha_ch_list" = "$ha_ch"]) + for chassis in gw1 gw2 hv1 hv2; do as $chassis echo "------ $chassis dump ----------" @@ -9810,33 +9846,56 @@ as hv2 ovs-ofctl dump-flows br-int table=32 gw1_chassis=$(ovn-sbctl --bare --columns=_uuid find Chassis name=gw1) gw2_chassis=$(ovn-sbctl --bare --columns=_uuid find Chassis name=gw2) -AT_CHECK([as hv1 ovs-ofctl dump-flows br-int table=32 | grep active_backup | grep slaves:$hv1_gw1_ofport,$hv1_gw2_ofport | wc -l], [0], [1 +OVS_WAIT_UNTIL([as hv1 ovs-ofctl dump-flows br-int table=32 | \ +grep active_backup | grep slaves:$hv1_gw1_ofport,$hv1_gw2_ofport \ +| wc -l], [0], [1 ]) -AT_CHECK([as hv2 ovs-ofctl dump-flows br-int table=32 | grep active_backup | grep slaves:$hv2_gw1_ofport,$hv2_gw2_ofport | wc -l], [0], [1 +OVS_WAIT_UNTIL([as hv2 ovs-ofctl dump-flows br-int table=32 | \ +grep active_backup | grep slaves:$hv2_gw1_ofport,$hv2_gw2_ofport \ +| wc -l], [0], [1 ]) -sleep 3 # let BFD sessions settle so we get the right flows on the right chassis - # make sure that flows for handling the outside router port reside on gw1 -AT_CHECK([as gw1 ovs-ofctl dump-flows br-int table=24 | grep 00:00:02:01:02:04 | wc -l], [0], [[1 +OVS_WAIT_UNTIL([as gw1 ovs-ofctl dump-flows br-int table=24 | \ +grep 00:00:02:01:02:04 | wc -l], [0], [[1 ]]) -AT_CHECK([as gw2 ovs-ofctl dump-flows br-int table=24 | grep 00:00:02:01:02:04 | wc -l], [0], [[0 + +OVS_WAIT_UNTIL([as gw2 ovs-ofctl dump-flows br-int table=24 | \ +grep 00:00:02:01:02:04 | wc -l], [0], [[0 ]]) # make sure ARP responder flows for outside router port reside on gw1 too -AT_CHECK([as gw1 ovs-ofctl dump-flows br-int table=9 | grep arp_tpa=192.168.0.101 | wc -l], [0], [[1 +OVS_WAIT_UNTIL([as gw1 ovs-ofctl dump-flows br-int table=9 | \ +grep arp_tpa=192.168.0.101 | wc -l], [0], [[1 ]]) -AT_CHECK([as gw2 ovs-ofctl dump-flows br-int table=9 | grep arp_tpa=192.168.0.101 | wc -l], [0], [[0 +OVS_WAIT_UNTIL([as gw2 ovs-ofctl dump-flows br-int table=9 | grep arp_tpa=192.168.0.101 | wc -l], [0], [[0 ]]) - - # check that the chassis redirect port has been claimed by the gw1 chassis -AT_CHECK([ovn-sbctl --columns chassis --bare find Port_Binding logical_port=cr-outside | grep $gw1_chassis | wc -l], - [0],[[1 +OVS_WAIT_UNTIL([ovn-sbctl --columns chassis --bare find Port_Binding \ +logical_port=cr-outside | grep $gw1_chassis | wc -l], [0],[[1 ]]) +hv1_ch_uuid=`ovn-sbctl --bare --columns _uuid find chassis name="hv1"` +hv2_ch_uuid=`ovn-sbctl --bare --columns _uuid find chassis name="hv2"` + +exp_ref_ch_list='' +for i in `ovn-sbctl --bare --columns _uuid list chassis | sort` +do + if test $i = $hv1_ch_uuid; then + exp_ref_ch_list="${exp_ref_ch_list}$i" + elif test $i = $hv2_ch_uuid; then + exp_ref_ch_list="${exp_ref_ch_list}$i" + fi +done + +OVS_WAIT_UNTIL( + [ref_ch_list=`ovn-sbctl --bare --columns ref_chassis find ha_chassis_group | sort` + # Trim the spaces. + ref_ch_list=`echo $ref_ch_list | sed 's/ //g'` + test "$exp_ref_ch_list" = "$ref_ch_list"]) + # at this point, we invert the priority of the gw chassis between gw1 and gw2 @@ -9851,15 +9910,19 @@ ovn-nbctl --id=@gc0 create Gateway_Chassis \ ovn-nbctl --wait=hv --timeout=3 sync # we make sure that the hypervisors noticed, and inverted the slave ports -AT_CHECK([as hv1 ovs-ofctl dump-flows br-int table=32 | grep active_backup | grep slaves:$hv1_gw2_ofport,$hv1_gw1_ofport | wc -l], [0], [1 +OVS_WAIT_UNTIL([as hv1 ovs-ofctl dump-flows br-int table=32 | \ +grep active_backup | grep slaves:$hv1_gw2_ofport,$hv1_gw1_ofport \ +| wc -l], [0], [1 ]) -AT_CHECK([as hv2 ovs-ofctl dump-flows br-int table=32 | grep active_backup | grep slaves:$hv2_gw2_ofport,$hv2_gw1_ofport | wc -l], [0], [1 +OVS_WAIT_UNTIL([as hv2 ovs-ofctl dump-flows br-int table=32 | \ +grep active_backup | grep slaves:$hv2_gw2_ofport,$hv2_gw1_ofport \ +| wc -l], [0], [1 ]) # check that the chassis redirect port has been reclaimed by the gw2 chassis -AT_CHECK([ovn-sbctl --columns chassis --bare find Port_Binding logical_port=cr-outside | grep $gw2_chassis | wc -l], - [0],[[1 +OVS_WAIT_UNTIL([ovn-sbctl --columns chassis --bare find Port_Binding \ +logical_port=cr-outside | grep $gw2_chassis | wc -l], [0],[[1 ]]) # check BFD enablement on tunnel ports from gw1 ######### @@ -9908,31 +9971,32 @@ AT_CHECK([ovs-vsctl --bare --columns bfd find Interface name=ovn-hv1-0],[0], [[ ]]) -sleep 3 # let BFD sessions settle so we get the right flows on the right chassis - # make sure that flows for handling the outside router port reside on gw2 now -AT_CHECK([as gw2 ovs-ofctl dump-flows br-int table=24 | grep 00:00:02:01:02:04 | wc -l], [0], [[1 +OVS_WAIT_UNTIL([as gw2 ovs-ofctl dump-flows br-int table=24 | \ +grep 00:00:02:01:02:04 | wc -l], [0], [[1 ]]) -AT_CHECK([as gw1 ovs-ofctl dump-flows br-int table=24 | grep 00:00:02:01:02:04 | wc -l], [0], [[0 +OVS_WAIT_UNTIL([as gw1 ovs-ofctl dump-flows br-int table=24 | \ +grep 00:00:02:01:02:04 | wc -l], [0], [[0 ]]) # disconnect GW2 from the network, GW1 should take over as gw2 port=${sandbox}_br-phys as main ovs-vsctl del-port n1 $port -sleep 4 bfd_dump # make sure that flows for handling the outside router port reside on gw2 now -AT_CHECK([as gw1 ovs-ofctl dump-flows br-int table=24 | grep 00:00:02:01:02:04 | wc -l], [0], [[1 +OVS_WAIT_UNTIL([as gw1 ovs-ofctl dump-flows br-int table=24 | \ +grep 00:00:02:01:02:04 | wc -l], [0], [[1 ]]) -AT_CHECK([as gw2 ovs-ofctl dump-flows br-int table=24 | grep 00:00:02:01:02:04 | wc -l], [0], [[0 +OVS_WAIT_UNTIL([as gw2 ovs-ofctl dump-flows br-int table=24 | \ +grep 00:00:02:01:02:04 | wc -l], [0], [[0 ]]) # check that the chassis redirect port has been reclaimed by the gw1 chassis -AT_CHECK([ovn-sbctl --columns chassis --bare find Port_Binding logical_port=cr-outside | grep $gw1_chassis | wc -l], - [0],[[1 +OVS_WAIT_UNTIL([ovn-sbctl --columns chassis --bare find Port_Binding \ +logical_port=cr-outside | grep $gw1_chassis | wc -l], [0],[[1 ]]) ovn-nbctl --wait=hv set NB_Global . options:"bfd-min-rx"=2000 @@ -9962,6 +10026,37 @@ for chassis in gw1 hv1 hv2; do ]) done +# Delete the inside1 vif. The ref_chassis in ha_chassis_group shouldn't have +# reference to hv1. +as hv1 ovs-vsctl del-port hv1-vif1 + +OVS_WAIT_UNTIL( + [ref_ch_list=`ovn-sbctl --bare --columns ref_chassis find ha_chassis_group | sort` + # Trim the spaces. + ref_ch_list=`echo $ref_ch_list | sed 's/ //g'` + test "$hv2_ch_uuid" = "$ref_ch_list"]) + +# Delete the inside2 vif. +ovn-sbctl show + +echo "Deleting hv2-vif1" +as hv2 ovs-vsctl del-port hv2-vif1 + +# ref_chassis of ha_chassis_group should be empty +OVS_WAIT_UNTIL( + [ref_ch_list=`ovn-sbctl --bare --columns ref_chassis find ha_chassis_group | sort` + # Trim the spaces. + ref_ch_list=`echo $ref_ch_list | sed 's/ //g'` + exp_ref_ch_list="" + test "$exp_ref_ch_list" = "$ref_ch_list"]) + +# Delete the Gateway_Chassis for lrp - outside +ovn-nbctl clear Logical_Router_Port outside gateway_chassis + +# There shoud be no ha_chassis_group rows in SB DB. +OVS_WAIT_UNTIL([test 0 = `ovn-sbctl list ha_chassis_group | wc -l`]) +OVS_WAIT_UNTIL([test 0 = `ovn-sbctl list ha_chassis | wc -l`]) + OVN_CLEANUP([gw1],[gw2],[hv1],[hv2]) AT_CLEANUP @@ -10198,10 +10293,7 @@ ovn-nbctl --wait=hv --timeout=3 sync gw2_chassis=$(ovn-sbctl --bare --columns=_uuid find Chassis name=gw2) ovn-sbctl destroy Chassis $gw2_chassis -# Ensure ovn-controller has processed latest sbdb update -# ovn-nbctl --wait=hv sync - -AT_CHECK([grep "Releasing lport" gw1/ovn-controller.log], [1], []) +OVS_WAIT_UNTIL([test 0 = `grep -c "Releasing lport" gw1/ovn-controller.log`]) OVN_CLEANUP([gw1],[gw2],[hv1]) From patchwork Thu Mar 14 19:31:58 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Numan Siddique X-Patchwork-Id: 1056670 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=openvswitch.org (client-ip=140.211.169.12; helo=mail.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Received: from mail.linuxfoundation.org (mail.linuxfoundation.org [140.211.169.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 44KzQ54Lg1z9ryj for ; Fri, 15 Mar 2019 06:33:01 +1100 (AEDT) Received: from mail.linux-foundation.org (localhost [127.0.0.1]) by mail.linuxfoundation.org (Postfix) with ESMTP id 8CA4BD48; Thu, 14 Mar 2019 19:32:10 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@mail.linuxfoundation.org Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 66257CF3 for ; Thu, 14 Mar 2019 19:32:09 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 309A085B for ; Thu, 14 Mar 2019 19:32:06 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id B55552D80B for ; Thu, 14 Mar 2019 19:32:05 +0000 (UTC) Received: from nusiddiq.mac (unknown [10.67.116.14]) by smtp.corp.redhat.com (Postfix) with ESMTP id AE95C5D75D; Thu, 14 Mar 2019 19:32:02 +0000 (UTC) From: nusiddiq@redhat.com To: dev@openvswitch.org Date: Fri, 15 Mar 2019 01:01:58 +0530 Message-Id: <20190314193158.24511-1-nusiddiq@redhat.com> In-Reply-To: <20190314193056.24241-1-nusiddiq@redhat.com> References: <20190314193056.24241-1-nusiddiq@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.30]); Thu, 14 Mar 2019 19:32:05 +0000 (UTC) X-Spam-Status: No, score=-6.9 required=5.0 tests=BAYES_00,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Subject: [ovs-dev] [PATCH v4 3/5] ovn-controller: Make use of ha_chassis_group table to bind the chassisredirect ports X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: ovs-dev-bounces@openvswitch.org Errors-To: ovs-dev-bounces@openvswitch.org From: Numan Siddique This patch uses the newly added ha_chassis_group table in Southbound DB - to bind the chassisredirect ports. - to establish BFD sessions with the required chassis. The previous patch in this series sets the list of chassis which references a ha chassis group in the 'ref_chassis' column of 'ha_chassis_group' table (in ovn-northd). This patch uses that information to establish BFD sessions with only the required chassis. There is no need to traverse the local_datapath list to determine if a local chasis has to establish a BFD session with another chassis. For eg, if chassis - HV1, HV2 and HV3 are part of a chassis group G1 and G1 is referenced by compute chassis - C1 and C2, the chassis C1 will establish BFD sessions with HV1, HV2 and HV3 since C1 references the group G1. The ha chassis HV1, HV2 and HV3 also establish BFD sessions amongst themselves and also with C1 and C2. This patch also deletes the old code (which used gateway_chassis table) to bind the chassisredirect port. The rational behind the refactor is to make the ha chassis binding support generic, so that logical ports of type 'external' (which will be added in the upcoming patch) can also make use of it and to simplify the gateway chassis support code in OVN. Functionally this new approach is same as the older one. Signed-off-by: Numan Siddique --- ovn/controller/automake.mk | 4 +- ovn/controller/bfd.c | 229 +++++++++++--------------------- ovn/controller/bfd.h | 11 +- ovn/controller/binding.c | 19 +-- ovn/controller/binding.h | 1 - ovn/controller/gchassis.c | 222 ------------------------------- ovn/controller/gchassis.h | 71 ---------- ovn/controller/ha-chassis.c | 203 ++++++++++++++++++++++++++++ ovn/controller/ha-chassis.h | 50 +++++++ ovn/controller/lflow.c | 29 ++-- ovn/controller/lflow.h | 3 +- ovn/controller/ovn-controller.c | 13 +- ovn/controller/physical.c | 109 +++++++-------- ovn/controller/physical.h | 3 +- ovn/controller/pinctrl.c | 38 ++---- ovn/controller/pinctrl.h | 1 - 16 files changed, 425 insertions(+), 581 deletions(-) delete mode 100644 ovn/controller/gchassis.c delete mode 100644 ovn/controller/gchassis.h create mode 100644 ovn/controller/ha-chassis.c create mode 100644 ovn/controller/ha-chassis.h diff --git a/ovn/controller/automake.mk b/ovn/controller/automake.mk index cd5505ca6..fcdf7a431 100644 --- a/ovn/controller/automake.mk +++ b/ovn/controller/automake.mk @@ -8,8 +8,8 @@ ovn_controller_ovn_controller_SOURCES = \ ovn/controller/chassis.h \ ovn/controller/encaps.c \ ovn/controller/encaps.h \ - ovn/controller/gchassis.c \ - ovn/controller/gchassis.h \ + ovn/controller/ha-chassis.c \ + ovn/controller/ha-chassis.h \ ovn/controller/lflow.c \ ovn/controller/lflow.h \ ovn/controller/lport.c \ diff --git a/ovn/controller/bfd.c b/ovn/controller/bfd.c index 10ab6ac37..d016e27b7 100644 --- a/ovn/controller/bfd.c +++ b/ovn/controller/bfd.c @@ -15,7 +15,6 @@ #include #include "bfd.h" -#include "gchassis.h" #include "lport.h" #include "ovn-controller.h" @@ -89,184 +88,104 @@ bfd_calculate_active_tunnels(const struct ovsrec_bridge *br_int, } } -struct local_datapath_node { - struct ovs_list node; - const struct local_datapath *dp; -}; - +/* Loops through the HA chassis groups in the SB DB and returns + * the set of chassis which the call can establish the BFD sessions + * with. + * Eg. + * If there are 2 HA chassis groups. + * Group name - hapgrp1 + * - HA chassis - (HA1, HA2, HA3) + * - ref chassis - (C1, C2) + * + * Group name - hapgrp2 + * - HA chassis - (HA1, HA4, HA5) + * - ref chassis - (C1, C3, C4) + * + * If 'our_chassis' is HA1 then this function returns + * bfd chassis set - (HA2, HA3, HA4 HA5, C1, C2, C3, C4) + * + * If 'our_chassis' is C1 then this function returns + * bfd chassis set - (HA1, HA2, HA3, HA4, HA5) + * + * If 'our_chassis' is HA5 then this function returns + * bfd chassis set - (HA1, HA4, C1, C3, C4) + * + * If 'our_chassis' is C2 then this function returns + * bfd chassis set - (HA1, HA2, HA3) + * + * If 'our_chassis' is C5 then this function returns empty bfd set. + */ static void -bfd_travel_gw_related_chassis( - struct ovsdb_idl_index *sbrec_port_binding_by_datapath, - const struct local_datapath *dp, - const struct hmap *local_datapaths, +bfd_calculate_chassis( + const struct sbrec_chassis *our_chassis, + const struct sbrec_ha_chassis_group_table *ha_chassis_grp_table, struct sset *bfd_chassis) { - struct ovs_list dp_list; - const struct sbrec_port_binding *pb; - struct sset visited_dp = SSET_INITIALIZER(&visited_dp); - const char *dp_key; - struct local_datapath_node *dp_binding; - - if (!(dp_key = smap_get(&dp->datapath->external_ids, "logical-router")) && - !(dp_key = smap_get(&dp->datapath->external_ids, "logical-switch"))) { - VLOG_INFO("datapath has no uuid, cannot travel graph"); - return; - } - - sset_add(&visited_dp, dp_key); - - ovs_list_init(&dp_list); - dp_binding = xmalloc(sizeof *dp_binding); - dp_binding->dp = dp; - ovs_list_push_back(&dp_list, &dp_binding->node); - - struct sbrec_port_binding *target = sbrec_port_binding_index_init_row( - sbrec_port_binding_by_datapath); - - /* Go through whole graph to figure out all chassis which may deliver - * packets to gateway. */ - LIST_FOR_EACH_POP (dp_binding, node, &dp_list) { - dp = dp_binding->dp; - free(dp_binding); - for (size_t i = 0; i < dp->n_peer_dps; i++) { - const struct sbrec_datapath_binding *pdp = dp->peer_dps[i]; - if (!pdp) { - continue; - } - - if (!(dp_key = smap_get(&pdp->external_ids, "logical-router")) && - !(dp_key = smap_get(&pdp->external_ids, "logical-switch"))) { + const struct sbrec_ha_chassis_group *ha_chassis_grp; + SBREC_HA_CHASSIS_GROUP_TABLE_FOR_EACH (ha_chassis_grp, + ha_chassis_grp_table) { + bool is_ha_chassis = false; + struct sset grp_chassis = SSET_INITIALIZER(&grp_chassis); + const struct sbrec_ha_chassis *ha_ch; + bool bfd_setup_required = false; + if (ha_chassis_grp->n_ha_chassis < 2) { + /* No need to consider the chassis group for BFD if + * there is 1 or no chassis in it. */ + continue; + } + for (size_t i = 0; i < ha_chassis_grp->n_ha_chassis; i++) { + ha_ch = ha_chassis_grp->ha_chassis[i]; + if (!ha_ch->chassis) { continue; } - - if (sset_contains(&visited_dp, dp_key)) { - continue; + sset_add(&grp_chassis, ha_ch->chassis->name); + if (our_chassis == ha_ch->chassis) { + is_ha_chassis = true; + bfd_setup_required = true; } + } - sset_add(&visited_dp, dp_key); - - struct hmap_node *node = hmap_first_with_hash(local_datapaths, - pdp->tunnel_key); - if (!node) { - continue; + if (is_ha_chassis) { + /* It's an HA chassis. So add the ref_chassis to the bfd set. */ + for (size_t i = 0; i < ha_chassis_grp->n_ref_chassis; i++) { + sset_add(&grp_chassis, ha_chassis_grp->ref_chassis[i]->name); } - - dp_binding = xmalloc(sizeof *dp_binding); - dp_binding->dp = CONTAINER_OF(node, struct local_datapath, - hmap_node); - ovs_list_push_back(&dp_list, &dp_binding->node); - - sbrec_port_binding_index_set_datapath(target, pdp); - SBREC_PORT_BINDING_FOR_EACH_EQUAL (pb, target, - sbrec_port_binding_by_datapath) { - if (pb->chassis) { - const char *chassis_name = pb->chassis->name; - if (chassis_name) { - sset_add(bfd_chassis, chassis_name); - } + } else { + /* This is not an HA chassis. Check if this chassis is present + * in the ref_chassis list. If so add the ha_chassis to the + * sset .*/ + for (size_t i = 0; i < ha_chassis_grp->n_ref_chassis; i++) { + if (our_chassis == ha_chassis_grp->ref_chassis[i]) { + bfd_setup_required = true; + break; } } } - } - sbrec_port_binding_index_destroy_row(target); - - sset_destroy(&visited_dp); -} - -static struct ovs_list * -bfd_find_ha_gateway_chassis( - struct ovsdb_idl_index *sbrec_chassis_by_name, - struct ovsdb_idl_index *sbrec_port_binding_by_datapath, - const struct sbrec_datapath_binding *datapath) -{ - struct sbrec_port_binding *target = sbrec_port_binding_index_init_row( - sbrec_port_binding_by_datapath); - sbrec_port_binding_index_set_datapath(target, datapath); - - struct ovs_list *ha_gateway_chassis = NULL; - const struct sbrec_port_binding *pb; - SBREC_PORT_BINDING_FOR_EACH_EQUAL (pb, target, - sbrec_port_binding_by_datapath) { - if (strcmp(pb->type, "chassisredirect")) { - continue; - } - - struct ovs_list *gateway_chassis = gateway_chassis_get_ordered( - sbrec_chassis_by_name, pb); - if (!gateway_chassis || ovs_list_is_short(gateway_chassis)) { - /* We don't need BFD for non-HA chassisredirect. */ - gateway_chassis_destroy(gateway_chassis); - continue; - } - - ha_gateway_chassis = gateway_chassis; - break; - } - sbrec_port_binding_index_destroy_row(target); - return ha_gateway_chassis; -} - -static void -bfd_calculate_chassis( - struct ovsdb_idl_index *sbrec_chassis_by_name, - struct ovsdb_idl_index *sbrec_port_binding_by_datapath, - const struct sbrec_chassis *our_chassis, - const struct hmap *local_datapaths, - struct sset *bfd_chassis) -{ - /* Identify all chassis nodes to which we need to enable bfd. - * 1) Any chassis hosting the chassisredirect ports for known - * router datapaths. - * 2) Chassis hosting peer datapaths (with ports) connected - * to a router datapath when our chassis is hosting a router - * with a chassis redirect port. */ - const struct local_datapath *dp; - HMAP_FOR_EACH (dp, hmap_node, local_datapaths) { - const char *is_router = smap_get(&dp->datapath->external_ids, - "logical-router"); - bool our_chassis_is_gw_for_dp = false; - if (is_router) { - struct ovs_list *ha_gateway_chassis - = bfd_find_ha_gateway_chassis(sbrec_chassis_by_name, - sbrec_port_binding_by_datapath, - dp->datapath); - if (ha_gateway_chassis) { - our_chassis_is_gw_for_dp = gateway_chassis_contains( - ha_gateway_chassis, our_chassis); - struct gateway_chassis *gwc; - LIST_FOR_EACH (gwc, node, ha_gateway_chassis) { - if (gwc->db->chassis) { - sset_add(bfd_chassis, gwc->db->chassis->name); - } - } - gateway_chassis_destroy(ha_gateway_chassis); + if (bfd_setup_required) { + const char *name; + SSET_FOR_EACH (name, &grp_chassis) { + sset_add(bfd_chassis, name); } } - if (our_chassis_is_gw_for_dp) { - bfd_travel_gw_related_chassis(sbrec_port_binding_by_datapath, - dp, local_datapaths, bfd_chassis); - } + sset_destroy(&grp_chassis); } } void -bfd_run(struct ovsdb_idl_index *sbrec_chassis_by_name, - struct ovsdb_idl_index *sbrec_port_binding_by_datapath, - const struct ovsrec_interface_table *interface_table, +bfd_run(const struct ovsrec_interface_table *interface_table, const struct ovsrec_bridge *br_int, const struct sbrec_chassis *chassis_rec, - const struct sbrec_sb_global_table *sb_global_table, - const struct hmap *local_datapaths) + const struct sbrec_ha_chassis_group_table *ha_chassis_grp_table, + const struct sbrec_sb_global_table *sb_global_table) { - if (!chassis_rec) { return; } struct sset bfd_chassis = SSET_INITIALIZER(&bfd_chassis); - bfd_calculate_chassis(sbrec_chassis_by_name, - sbrec_port_binding_by_datapath, - chassis_rec, local_datapaths, &bfd_chassis); + bfd_calculate_chassis(chassis_rec, ha_chassis_grp_table, + &bfd_chassis); + /* Identify tunnels ports(connected to remote chassis id) to enable bfd */ struct sset tunnels = SSET_INITIALIZER(&tunnels); struct sset bfd_ifaces = SSET_INITIALIZER(&bfd_ifaces); @@ -336,7 +255,7 @@ bfd_run(struct ovsdb_idl_index *sbrec_chassis_by_name, VLOG_INFO("Disabled BFD on interface %s", iface->name); } } - } + } } smap_destroy(&bfd); diff --git a/ovn/controller/bfd.h b/ovn/controller/bfd.h index e36820afb..789f7b269 100644 --- a/ovn/controller/bfd.h +++ b/ovn/controller/bfd.h @@ -24,16 +24,17 @@ struct ovsrec_interface_table; struct ovsrec_open_vswitch_table; struct sbrec_chassis; struct sbrec_sb_global_table; +struct sbrec_ha_chassis_group_table; struct sset; void bfd_register_ovs_idl(struct ovsdb_idl *); -void bfd_run(struct ovsdb_idl_index *sbrec_chassis_by_name, - struct ovsdb_idl_index *sbrec_port_binding_by_datapath, - const struct ovsrec_interface_table *interface_table, + +void bfd_run(const struct ovsrec_interface_table *interface_table, const struct ovsrec_bridge *br_int, const struct sbrec_chassis *chassis_rec, - const struct sbrec_sb_global_table *sb_global_table, - const struct hmap *local_datapaths); + const struct sbrec_ha_chassis_group_table *ha_chassis_grp_table, + const struct sbrec_sb_global_table *sb_global_table); + void bfd_calculate_active_tunnels(const struct ovsrec_bridge *br_int, struct sset *active_tunnels); diff --git a/ovn/controller/binding.c b/ovn/controller/binding.c index 74ba12743..2929eccd7 100644 --- a/ovn/controller/binding.c +++ b/ovn/controller/binding.c @@ -15,7 +15,7 @@ #include #include "binding.h" -#include "gchassis.h" +#include "ha-chassis.h" #include "lflow.h" #include "lport.h" @@ -430,7 +430,6 @@ sbrec_get_port_encap(const struct sbrec_chassis *chassis_rec, static void consider_local_datapath(struct ovsdb_idl_txn *ovnsb_idl_txn, struct ovsdb_idl_txn *ovs_idl_txn, - struct ovsdb_idl_index *sbrec_chassis_by_name, struct ovsdb_idl_index *sbrec_datapath_binding_by_key, struct ovsdb_idl_index *sbrec_port_binding_by_datapath, struct ovsdb_idl_index *sbrec_port_binding_by_name, @@ -445,7 +444,6 @@ consider_local_datapath(struct ovsdb_idl_txn *ovnsb_idl_txn, { const struct ovsrec_interface *iface_rec = shash_find_data(lport_to_iface, binding_rec->logical_port); - struct ovs_list *gateway_chassis = NULL; bool our_chassis = false; if (iface_rec @@ -478,20 +476,17 @@ consider_local_datapath(struct ovsdb_idl_txn *ovnsb_idl_txn, binding_rec->datapath, false, local_datapaths); } } else if (!strcmp(binding_rec->type, "chassisredirect")) { - gateway_chassis = gateway_chassis_get_ordered(sbrec_chassis_by_name, - binding_rec); - if (gateway_chassis && - gateway_chassis_contains(gateway_chassis, chassis_rec)) { - - our_chassis = gateway_chassis_is_active( - gateway_chassis, chassis_rec, active_tunnels); + if (ha_chassis_group_contains(binding_rec->ha_chassis_group, + chassis_rec)) { + our_chassis = ha_chassis_group_is_active( + binding_rec->ha_chassis_group, + active_tunnels, chassis_rec); add_local_datapath(sbrec_datapath_binding_by_key, sbrec_port_binding_by_datapath, sbrec_port_binding_by_name, binding_rec->datapath, false, local_datapaths); } - gateway_chassis_destroy(gateway_chassis); } else if (!strcmp(binding_rec->type, "l3gateway")) { const char *chassis_id = smap_get(&binding_rec->options, "l3gateway-chassis"); @@ -592,7 +587,6 @@ consider_localnet_port(const struct sbrec_port_binding *binding_rec, void binding_run(struct ovsdb_idl_txn *ovnsb_idl_txn, struct ovsdb_idl_txn *ovs_idl_txn, - struct ovsdb_idl_index *sbrec_chassis_by_name, struct ovsdb_idl_index *sbrec_datapath_binding_by_key, struct ovsdb_idl_index *sbrec_port_binding_by_datapath, struct ovsdb_idl_index *sbrec_port_binding_by_name, @@ -625,7 +619,6 @@ binding_run(struct ovsdb_idl_txn *ovnsb_idl_txn, * directly connected logical ports and children of those ports. */ SBREC_PORT_BINDING_TABLE_FOR_EACH (binding_rec, port_binding_table) { consider_local_datapath(ovnsb_idl_txn, ovs_idl_txn, - sbrec_chassis_by_name, sbrec_datapath_binding_by_key, sbrec_port_binding_by_datapath, sbrec_port_binding_by_name, diff --git a/ovn/controller/binding.h b/ovn/controller/binding.h index 837e1099f..0f24a0efd 100644 --- a/ovn/controller/binding.h +++ b/ovn/controller/binding.h @@ -33,7 +33,6 @@ struct sset; void binding_register_ovs_idl(struct ovsdb_idl *); void binding_run(struct ovsdb_idl_txn *ovnsb_idl_txn, struct ovsdb_idl_txn *ovs_idl_txn, - struct ovsdb_idl_index *sbrec_chassis_by_name, struct ovsdb_idl_index *sbrec_datapath_binding_by_key, struct ovsdb_idl_index *sbrec_port_binding_by_datapath, struct ovsdb_idl_index *sbrec_port_binding_by_name, diff --git a/ovn/controller/gchassis.c b/ovn/controller/gchassis.c deleted file mode 100644 index 34b78bcc0..000000000 --- a/ovn/controller/gchassis.c +++ /dev/null @@ -1,222 +0,0 @@ -/* Copyright (c) 2017, Red Hat, Inc. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at: - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -#include - -#include "gchassis.h" -#include "lport.h" -#include "lib/sset.h" -#include "openvswitch/vlog.h" -#include "ovn/lib/chassis-index.h" -#include "ovn/lib/ovn-sb-idl.h" - -VLOG_DEFINE_THIS_MODULE(gchassis); - -/* gateway_chassis ordering - */ -static int -compare_chassis_prio_(const void *a_, const void *b_) -{ - const struct gateway_chassis *gc_a = a_; - const struct gateway_chassis *gc_b = b_; - int prio_diff = gc_b->db->priority - gc_a->db->priority; - if (!prio_diff) { - return strcmp(gc_b->db->name, gc_a->db->name); - } - return prio_diff; -} - -struct ovs_list* -gateway_chassis_get_ordered(struct ovsdb_idl_index *sbrec_chassis_by_name, - const struct sbrec_port_binding *binding) -{ - const char *redir_chassis_str; - const struct sbrec_chassis *redirect_chassis = NULL; - - /* XXX: redirect-chassis SBDB option handling is supported for backwards - * compatibility with N-1 version of ovn-northd. This support can - * be removed in OVS 2.9 where Gateway_Chassis list on the port binding - * will always be populated by northd */ - redir_chassis_str = smap_get(&binding->options, "redirect-chassis"); - - if (redir_chassis_str) { - redirect_chassis = chassis_lookup_by_name(sbrec_chassis_by_name, - redir_chassis_str); - if (!redirect_chassis) { - static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 1); - VLOG_WARN_RL(&rl, "chassis name (%s) in redirect-chassis option " - "of logical port %s not known", - redir_chassis_str, binding->logical_port); - } - } - - if (!redirect_chassis && binding->n_gateway_chassis == 0) { - return NULL; - } - - struct gateway_chassis *gateway_chassis = NULL; - int n = 0; - - if (binding->n_gateway_chassis) { - gateway_chassis = xmalloc(sizeof *gateway_chassis * - binding->n_gateway_chassis); - for (n = 0; n < binding->n_gateway_chassis; n++) { - gateway_chassis[n].db = binding->gateway_chassis[n]; - gateway_chassis[n].virtual_gwc = false; - } - qsort(gateway_chassis, n, sizeof *gateway_chassis, - compare_chassis_prio_); - } else if (redirect_chassis) { - /* When only redirect_chassis is available, return a single - * virtual entry that it's not on OVSDB, this way the code - * handling the returned list will be uniform, regardless - * of gateway_chassis being populated or redirect-chassis option - * being used */ - gateway_chassis = xmalloc(sizeof *gateway_chassis); - struct sbrec_gateway_chassis *gwc = - xzalloc(sizeof *gateway_chassis->db); - sbrec_gateway_chassis_init(gwc); - gwc->name = xasprintf("%s_%s", binding->logical_port, - redirect_chassis->name); - gwc->chassis = CONST_CAST(struct sbrec_chassis *, redirect_chassis); - gateway_chassis->db = gwc; - gateway_chassis->virtual_gwc = true; - n++; - } - - struct ovs_list *list = NULL; - if (n) { - list = xmalloc(sizeof *list); - ovs_list_init(list); - - int i; - for (i = 0; i < n; i++) { - ovs_list_push_back(list, &gateway_chassis[i].node); - } - } - - return list; -} - -bool -gateway_chassis_contains(const struct ovs_list *gateway_chassis, - const struct sbrec_chassis *chassis) { - struct gateway_chassis *chassis_item; - if (gateway_chassis) { - LIST_FOR_EACH (chassis_item, node, gateway_chassis) { - if (chassis_item->db->chassis - && !strcmp(chassis_item->db->chassis->name, chassis->name)) { - return true; - } - } - } - return false; -} - -void -gateway_chassis_destroy(struct ovs_list *list) -{ - if (!list) { - return; - } - - /* XXX: This loop is for backwards compatibility with redirect-chassis - * which we insert as a single virtual Gateway_Chassis on the ordered - * list */ - struct gateway_chassis *chassis_item; - LIST_FOR_EACH (chassis_item, node, list) { - if (chassis_item->virtual_gwc) { - free(chassis_item->db->name); - free(CONST_CAST(struct sbrec_gateway_chassis *, chassis_item->db)); - } - } - - free(ovs_list_front(list)); - free(list); -} - -bool -gateway_chassis_in_pb_contains(const struct sbrec_port_binding *binding, - const struct sbrec_chassis *chassis) -{ - if (!binding || !chassis) { - return false; - } - - /* XXX: redirect-chassis handling for backwards compatibility, - * with older ovs-northd during upgrade phase, can be removed - * for OVS 2.9 */ - const char *redirect_chassis = smap_get(&binding->options, - "redirect-chassis"); - if (binding->n_gateway_chassis) { - int n; - for (n = 0; n < binding->n_gateway_chassis; n++) { - if (binding->gateway_chassis[n]->chassis - && !strcmp(binding->gateway_chassis[n]->chassis->name, - chassis->name)) { - return true; - } - } - } else if (redirect_chassis) { - return !strcmp(redirect_chassis, chassis->name); - } - - return false; -} - -bool -gateway_chassis_is_active(const struct ovs_list *gateway_chassis, - const struct sbrec_chassis *local_chassis, - const struct sset *active_tunnels) -{ - struct gateway_chassis *gwc; - - if (!gateway_chassis - || (gateway_chassis && ovs_list_is_empty(gateway_chassis))) { - return false; - } - /* if there's only one chassis, and local chassis is on the list - * it's not HA and it's the equivalent of being active */ - if (ovs_list_is_singleton(gateway_chassis) && - gateway_chassis_contains(gateway_chassis, local_chassis)) { - return true; - } - - /* if there are no other tunnels active, we assume that the - * connection providing tunneling is down, hence we're down */ - if (sset_is_empty(active_tunnels)) { - return false; - } - - /* gateway_chassis is an ordered list, by priority, of chassis - * hosting the redirect of the port */ - LIST_FOR_EACH (gwc, node, gateway_chassis) { - if (!gwc->db->chassis) { - continue; - } - /* if we found the chassis on the list, and we didn't exit before - * on the active_tunnels check for other higher priority chassis - * being active, then this chassis is master. */ - if (!strcmp(gwc->db->chassis->name, local_chassis->name)) { - return true; - } - /* if we find this specific chassis on the list to have an active - * tunnel, then 'local_chassis' is not master */ - if (sset_contains(active_tunnels, gwc->db->chassis->name)) { - return false; - } - } - return false; -} diff --git a/ovn/controller/gchassis.h b/ovn/controller/gchassis.h deleted file mode 100644 index 901be4491..000000000 --- a/ovn/controller/gchassis.h +++ /dev/null @@ -1,71 +0,0 @@ -/* Copyright (c) 2017 Red Hat, Inc. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at: - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -#ifndef OVN_GCHASSIS_H -#define OVN_GCHASSIS_H 1 - -#include -#include "lib/uuid.h" -#include "openvswitch/hmap.h" -#include "openvswitch/list.h" - -struct ovsdb_idl; -struct ovsdb_idl_index; -struct sbrec_chassis; -struct sbrec_gateway_chassis; -struct sbrec_port_binding; -struct sset; - - -/* Gateway_Chassis management - * ========================== - * - * The following structure and methods handle ordering of Gateway_Chassis - * entries in a chassisredirect port. And parsing redirect-chassis option - * for backwards compatibility with older (N-1 version of ovn-northd). - */ -struct gateway_chassis { - struct ovs_list node; - const struct sbrec_gateway_chassis *db; /* sbrec row for the gwc */ - bool virtual_gwc; /* db entry not from SBDB, but from redirect-chassis */ -}; - -/* Gets, and orders by priority/name the list of Gateway_Chassis */ -struct ovs_list *gateway_chassis_get_ordered( - struct ovsdb_idl_index *sbrec_chassis_by_name, - const struct sbrec_port_binding *binding); - -/* Checks if an specific chassis is contained in the gateway_chassis - * list */ -bool gateway_chassis_contains(const struct ovs_list *gateway_chassis, - const struct sbrec_chassis *chassis); - -/* Destroy a gateway_chassis list from memory */ -void gateway_chassis_destroy(struct ovs_list *list); - -/* Checks if a chassis is referenced in the port_binding gateway_chassis - * list or redirect-chassis option (backwards compatibility) */ -bool gateway_chassis_in_pb_contains( - const struct sbrec_port_binding *binding, - const struct sbrec_chassis *chassis); - -/* Returns true if the local chassis is the active gateway among a set - * of gateway_chassis. Return false if the local chassis is currently a - * backup in a set of multiple gateway_chassis. */ -bool gateway_chassis_is_active( - const struct ovs_list *gateway_chassis, - const struct sbrec_chassis *local_chassis, - const struct sset *active_tunnels); -#endif /* ovn/controller/gchassis.h */ diff --git a/ovn/controller/ha-chassis.c b/ovn/controller/ha-chassis.c new file mode 100644 index 000000000..498e5ce5a --- /dev/null +++ b/ovn/controller/ha-chassis.c @@ -0,0 +1,203 @@ +/* Copyright (c) 2019 Red Hat, Inc. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at: + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#include + +#include "ha-chassis.h" +#include "lib/sset.h" +#include "openvswitch/vlog.h" +#include "ovn/lib/ovn-sb-idl.h" + +VLOG_DEFINE_THIS_MODULE(ha_chassis); + +static int +compare_chassis_prio_(const void *a_, const void *b_) +{ + const struct sbrec_ha_chassis *ch_a = a_; + const struct sbrec_ha_chassis *ch_b = b_; + int prio_diff = ch_b->priority - ch_a->priority; + if (!prio_diff) { + return strcmp(ch_b->chassis->name, ch_a->chassis->name); + } + return prio_diff; +} + +/* Returns the ordered HA chassis list in the HA chassis group. + * Eg. If an HA chassis group has 3 HA chassis + * - HA1 - pri 30 + * - HA2 - pri 40 and + * - HA3 - pri 20 + * and the ref_chassis of HA chassis group is set to - C1 and C2. + * + * If active_tunnels is NULL, then it returns the ordered list + * - (HA2, HA1, HA3) + * + * If active_tunnels is set to - (HA1, HA2, C1, C2) and + * local_chassis is HA3, then it returns the ordered list + * - (HA2, HA1, HA3) + * + * If active_tunnels is set to - (HA1, C1, C2) and + * local_chassis is HA3, then it returns the ordered list + * - (HA1, HA3) + * + * If active_tunnels is set to - (C1, C2) and + * local_chassis is HA3, then it returns the ordered list + * - (HA3) + * + * If active_tunnels is set is empty and local_chassis is HA3, + * then it returns NULL. + */ +static struct ha_chassis_ordered * +get_ordered_ha_chassis_list(const struct sbrec_ha_chassis_group *ha_ch_grp, + const struct sset *active_tunnels, + const struct sbrec_chassis *local_chassis) +{ + struct sbrec_ha_chassis *ha_ch_order = + xzalloc(sizeof *ha_ch_order * ha_ch_grp->n_ha_chassis); + + size_t n_ha_ch = 0; + + for (size_t i = 0; i < ha_ch_grp->n_ha_chassis; i++) { + if (!ha_ch_grp->ha_chassis[i]->chassis) { + continue; + } + + /* Don't add it to the list for ordering if it is not active. */ + if (ha_ch_grp->ha_chassis[i]->chassis != local_chassis && + active_tunnels && + !sset_contains(active_tunnels, + ha_ch_grp->ha_chassis[i]->chassis->name)) { + continue; + } + + ha_ch_order[n_ha_ch].chassis = ha_ch_grp->ha_chassis[i]->chassis; + ha_ch_order[n_ha_ch].priority = ha_ch_grp->ha_chassis[i]->priority; + n_ha_ch++; + } + + if (!n_ha_ch) { + free(ha_ch_order); + return NULL; + } + + struct ha_chassis_ordered *ordered_ha_ch; + if (n_ha_ch == 1) { + if (active_tunnels) { + /* If n_ha_ch is 1, it means only the local chassis is in the + * ha_ch_order list. Check if this local chassis has active + * bfd session with any of the referenced chassis. If so, + * then the local chassis can be active. Otherwise it can't. + * This can happen in the following scenario. + * Lets say we have chassis HA1 (prioirty 20) and HA2 (priority 10) + * in the ha_chasis_group and compute chassis C1 and C2 are in the + * reference chassis list. If HA1 chassis has lost the link and + * when this function is called for HA2 we need to consider + * HA2 as active since it has active BFD sessions with C1 and C2. + * On HA1 chassis, this function won't be called since + * active_tunnels set will be empty. + * */ + bool can_local_chassis_be_active = false; + for (size_t i = 0; i < ha_ch_grp->n_ref_chassis; i++) { + if (sset_contains(active_tunnels, + ha_ch_grp->ref_chassis[i]->name)) { + can_local_chassis_be_active = true; + break; + } + } + if (!can_local_chassis_be_active) { + free(ha_ch_order); + return NULL; + } + } + } else { + qsort(ha_ch_order, n_ha_ch, sizeof *ha_ch_order, + compare_chassis_prio_); + } + + ordered_ha_ch = xmalloc(sizeof *ordered_ha_ch); + ordered_ha_ch->ha_ch = ha_ch_order; + ordered_ha_ch->n_ha_ch = n_ha_ch; + + return ordered_ha_ch; +} + +void +ha_chassis_destroy_ordered(struct ha_chassis_ordered *ordered_ha_ch) +{ + if (ordered_ha_ch) { + free(ordered_ha_ch->ha_ch); + free(ordered_ha_ch); + } +} + + +/* Returns true if the local_chassis is the master of + * the HA chassis group, false otherwise. */ +bool +ha_chassis_group_is_active( + const struct sbrec_ha_chassis_group *ha_ch_grp, + const struct sset *active_tunnels, + const struct sbrec_chassis *local_chassis) +{ + if (!ha_ch_grp || !ha_ch_grp->n_ha_chassis) { + return false; + } + + if (ha_ch_grp->n_ha_chassis == 1) { + return (ha_ch_grp->ha_chassis[0]->chassis == local_chassis); + } + + if (sset_is_empty(active_tunnels)) { + /* If active tunnel sset is empty, it means it has lost + * connectivity with other chassis. */ + return false; + } + + struct ha_chassis_ordered *ordered_ha_ch = + get_ordered_ha_chassis_list(ha_ch_grp, active_tunnels, local_chassis); + if (!ordered_ha_ch) { + return false; + } + + struct sbrec_chassis *active_ch = ordered_ha_ch->ha_ch[0].chassis; + ha_chassis_destroy_ordered(ordered_ha_ch); + + return (active_ch == local_chassis); +} + +bool +ha_chassis_group_contains( + const struct sbrec_ha_chassis_group *ha_chassis_grp, + const struct sbrec_chassis *chassis) +{ + if (ha_chassis_grp && chassis) { + for (size_t i = 0; i < ha_chassis_grp->n_ha_chassis; i++) { + if (ha_chassis_grp->ha_chassis[i]->chassis == chassis) { + return true; + } + } + } + return false; +} + +struct ha_chassis_ordered * +ha_chassis_get_ordered(const struct sbrec_ha_chassis_group *ha_chassis_grp) +{ + if (!ha_chassis_grp || !ha_chassis_grp->n_ha_chassis) { + return NULL; + } + + return get_ordered_ha_chassis_list(ha_chassis_grp, NULL, NULL); +} diff --git a/ovn/controller/ha-chassis.h b/ovn/controller/ha-chassis.h new file mode 100644 index 000000000..3768c2a5c --- /dev/null +++ b/ovn/controller/ha-chassis.h @@ -0,0 +1,50 @@ +/* Copyright (c) 2019 Red Hat, Inc. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at: + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#ifndef OVN_HA_CHASSIS_H +#define OVN_HA_CHASSIS_H 1 + +#include +#include "openvswitch/hmap.h" +#include "openvswitch/list.h" + +struct sbrec_chassis; +struct sbrec_ha_chassis_group; +struct sset; + +struct ha_chassis_ordered { + struct sbrec_ha_chassis *ha_ch; + size_t n_ha_ch; +}; + +/* Returns true if the local chassis is the active gateway among a set + * of gateway_chassis. Return false if the local chassis is currently a + * backup in a set of multiple gateway_chassis. */ +bool ha_chassis_group_is_active( + const struct sbrec_ha_chassis_group *ha_chassis_grp, + const struct sset *active_tunnels, + const struct sbrec_chassis *local_chassis); + +bool ha_chassis_group_contains( + const struct sbrec_ha_chassis_group *ha_chassis_grp, + const struct sbrec_chassis *chassis); + +struct ha_chassis_ordered *ha_chassis_get_ordered( + const struct sbrec_ha_chassis_group *ha_chassis_grp); + +void ha_chassis_destroy_ordered( + struct ha_chassis_ordered *ordered_ha_ch); + +#endif /* OVN_HA_CHASSIS_H */ diff --git a/ovn/controller/lflow.c b/ovn/controller/lflow.c index 8db81927e..e4a8004fc 100644 --- a/ovn/controller/lflow.c +++ b/ovn/controller/lflow.c @@ -16,7 +16,7 @@ #include #include "lflow.h" #include "coverage.h" -#include "gchassis.h" +#include "ha-chassis.h" #include "lport.h" #include "ofctrl.h" #include "openvswitch/dynamic-string.h" @@ -56,14 +56,12 @@ struct lookup_port_aux { }; struct condition_aux { - struct ovsdb_idl_index *sbrec_chassis_by_name; struct ovsdb_idl_index *sbrec_port_binding_by_name; const struct sbrec_chassis *chassis; const struct sset *active_tunnels; }; static void consider_logical_flow( - struct ovsdb_idl_index *sbrec_chassis_by_name, struct ovsdb_idl_index *sbrec_multicast_group_by_name_datapath, struct ovsdb_idl_index *sbrec_port_binding_by_name, const struct sbrec_logical_flow *, @@ -117,14 +115,11 @@ is_chassis_resident_cb(const void *c_aux_, const char *port_name) /* for non-chassisredirect ports */ return pb->chassis && pb->chassis == c_aux->chassis; } else { - struct ovs_list *gateway_chassis; - gateway_chassis = gateway_chassis_get_ordered( - c_aux->sbrec_chassis_by_name, pb); - if (gateway_chassis) { - bool active = gateway_chassis_is_active(gateway_chassis, - c_aux->chassis, - c_aux->active_tunnels); - gateway_chassis_destroy(gateway_chassis); + if (ha_chassis_group_contains(pb->ha_chassis_group, + c_aux->chassis)) { + bool active = ha_chassis_group_is_active(pb->ha_chassis_group, + c_aux->active_tunnels, + c_aux->chassis); return active; } return false; @@ -141,7 +136,6 @@ is_switch(const struct sbrec_datapath_binding *ldp) /* Adds the logical flows from the Logical_Flow table to flow tables. */ static void add_logical_flows( - struct ovsdb_idl_index *sbrec_chassis_by_name, struct ovsdb_idl_index *sbrec_multicast_group_by_name_datapath, struct ovsdb_idl_index *sbrec_port_binding_by_name, const struct sbrec_dhcp_options_table *dhcp_options_table, @@ -180,8 +174,7 @@ add_logical_flows( nd_ra_opts_init(&nd_ra_opts); SBREC_LOGICAL_FLOW_TABLE_FOR_EACH (lflow, logical_flow_table) { - consider_logical_flow(sbrec_chassis_by_name, - sbrec_multicast_group_by_name_datapath, + consider_logical_flow(sbrec_multicast_group_by_name_datapath, sbrec_port_binding_by_name, lflow, local_datapaths, chassis, &dhcp_opts, &dhcpv6_opts, &nd_ra_opts, @@ -197,7 +190,6 @@ add_logical_flows( static void consider_logical_flow( - struct ovsdb_idl_index *sbrec_chassis_by_name, struct ovsdb_idl_index *sbrec_multicast_group_by_name_datapath, struct ovsdb_idl_index *sbrec_port_binding_by_name, const struct sbrec_logical_flow *lflow, @@ -295,7 +287,6 @@ consider_logical_flow( .dp = lflow->logical_datapath }; struct condition_aux cond_aux = { - .sbrec_chassis_by_name = sbrec_chassis_by_name, .sbrec_port_binding_by_name = sbrec_port_binding_by_name, .chassis = chassis, .active_tunnels = active_tunnels, @@ -460,8 +451,7 @@ add_neighbor_flows(struct ovsdb_idl_index *sbrec_port_binding_by_name, /* Translates logical flows in the Logical_Flow table in the OVN_SB database * into OpenFlow flows. See ovn-architecture(7) for more information. */ void -lflow_run(struct ovsdb_idl_index *sbrec_chassis_by_name, - struct ovsdb_idl_index *sbrec_multicast_group_by_name_datapath, +lflow_run(struct ovsdb_idl_index *sbrec_multicast_group_by_name_datapath, struct ovsdb_idl_index *sbrec_port_binding_by_name, const struct sbrec_dhcp_options_table *dhcp_options_table, const struct sbrec_dhcpv6_options_table *dhcpv6_options_table, @@ -479,8 +469,7 @@ lflow_run(struct ovsdb_idl_index *sbrec_chassis_by_name, { COVERAGE_INC(lflow_run); - add_logical_flows(sbrec_chassis_by_name, - sbrec_multicast_group_by_name_datapath, + add_logical_flows(sbrec_multicast_group_by_name_datapath, sbrec_port_binding_by_name, dhcp_options_table, dhcpv6_options_table, logical_flow_table, local_datapaths, chassis, addr_sets, port_groups, diff --git a/ovn/controller/lflow.h b/ovn/controller/lflow.h index d19338140..05d479d1a 100644 --- a/ovn/controller/lflow.h +++ b/ovn/controller/lflow.h @@ -65,8 +65,7 @@ struct uuid; #define LOG_PIPELINE_LEN 24 void lflow_init(void); -void lflow_run(struct ovsdb_idl_index *sbrec_chassis_by_name, - struct ovsdb_idl_index *sbrec_multicast_group_by_name_datapath, +void lflow_run(struct ovsdb_idl_index *sbrec_multicast_group_by_name_datapath, struct ovsdb_idl_index *sbrec_port_binding_by_name, const struct sbrec_dhcp_options_table *, const struct sbrec_dhcpv6_options_table *, diff --git a/ovn/controller/ovn-controller.c b/ovn/controller/ovn-controller.c index 882cc195f..986a28c8a 100644 --- a/ovn/controller/ovn-controller.c +++ b/ovn/controller/ovn-controller.c @@ -710,7 +710,7 @@ main(int argc, char *argv[]) bfd_calculate_active_tunnels(br_int, &active_tunnels); } - binding_run(ovnsb_idl_txn, ovs_idl_txn, sbrec_chassis_by_name, + binding_run(ovnsb_idl_txn, ovs_idl_txn, sbrec_datapath_binding_by_key, sbrec_port_binding_by_datapath, sbrec_port_binding_by_name, @@ -740,7 +740,7 @@ main(int argc, char *argv[]) enum mf_field_id mff_ovn_geneve = ofctrl_run( br_int, &pending_ct_zones); - pinctrl_run(ovnsb_idl_txn, sbrec_chassis_by_name, + pinctrl_run(ovnsb_idl_txn, sbrec_datapath_binding_by_key, sbrec_port_binding_by_datapath, sbrec_port_binding_by_key, @@ -760,7 +760,6 @@ main(int argc, char *argv[]) struct hmap flow_table = HMAP_INITIALIZER(&flow_table); lflow_run( - sbrec_chassis_by_name, sbrec_multicast_group_by_name_datapath, sbrec_port_binding_by_name, sbrec_dhcp_options_table_get(ovnsb_idl_loop.idl), @@ -774,15 +773,13 @@ main(int argc, char *argv[]) if (chassis_id) { bfd_run( - sbrec_chassis_by_name, - sbrec_port_binding_by_datapath, ovsrec_interface_table_get(ovs_idl_loop.idl), br_int, chassis, - sbrec_sb_global_table_get(ovnsb_idl_loop.idl), - &local_datapaths); + sbrec_ha_chassis_group_table_get( + ovnsb_idl_loop.idl), + sbrec_sb_global_table_get(ovnsb_idl_loop.idl)); } physical_run( - sbrec_chassis_by_name, sbrec_port_binding_by_name, sbrec_multicast_group_table_get( ovnsb_idl_loop.idl), diff --git a/ovn/controller/physical.c b/ovn/controller/physical.c index da89890ac..738640446 100644 --- a/ovn/controller/physical.c +++ b/ovn/controller/physical.c @@ -17,7 +17,7 @@ #include "binding.h" #include "byte-order.h" #include "flow.h" -#include "gchassis.h" +#include "ha-chassis.h" #include "lflow.h" #include "lport.h" #include "lib/bundle.h" @@ -377,8 +377,7 @@ load_logical_ingress_metadata(const struct sbrec_port_binding *binding, } static void -consider_port_binding(struct ovsdb_idl_index *sbrec_chassis_by_name, - struct ovsdb_idl_index *sbrec_port_binding_by_name, +consider_port_binding(struct ovsdb_idl_index *sbrec_port_binding_by_name, enum mf_field_id mff_ovn_geneve, const struct simap *ct_zones, const struct sset *active_tunnels, @@ -446,13 +445,13 @@ consider_port_binding(struct ovsdb_idl_index *sbrec_chassis_by_name, return; } - struct ovs_list *gateway_chassis - = gateway_chassis_get_ordered(sbrec_chassis_by_name, binding); + struct ha_chassis_ordered *ha_ch_ordered + = ha_chassis_get_ordered(binding->ha_chassis_group); if (!strcmp(binding->type, "chassisredirect") && (binding->chassis == chassis - || gateway_chassis_is_active(gateway_chassis, chassis, - active_tunnels))) { + || ha_chassis_group_is_active(binding->ha_chassis_group, + active_tunnels, chassis))) { /* Table 33, priority 100. * ======================= @@ -588,7 +587,7 @@ consider_port_binding(struct ovsdb_idl_index *sbrec_chassis_by_name, goto out; } } else { - if (!gateway_chassis || ovs_list_is_short(gateway_chassis)) { + if (!ha_ch_ordered || ha_ch_ordered->n_ha_ch < 2) { /* It's on a single remote chassis */ if (!binding->chassis) { goto out; @@ -599,7 +598,8 @@ consider_port_binding(struct ovsdb_idl_index *sbrec_chassis_by_name, } ofport = tun->ofport; } else { - /* It's distributed across the "gateway_chassis" list */ + /* It's distributed across the chassis belonging to + * an HA chassis group. */ is_ha_remote = true; } } @@ -753,34 +753,36 @@ consider_port_binding(struct ovsdb_idl_index *sbrec_chassis_by_name, /* Output to tunnel. */ ofpact_put_OUTPUT(ofpacts_p)->port = rem_tun->ofport; } else { - struct gateway_chassis *gwc; /* Make sure all tunnel endpoints use the same encapsulation, * and set it up */ - LIST_FOR_EACH (gwc, node, gateway_chassis) { - if (gwc->db->chassis) { - if (!tun) { - tun = chassis_tunnel_find(gwc->db->chassis->name, NULL); - } else { - struct chassis_tunnel *chassis_tunnel = - chassis_tunnel_find(gwc->db->chassis->name, NULL); - if (chassis_tunnel && - tun->type != chassis_tunnel->type) { - static struct vlog_rate_limit rl = - VLOG_RATE_LIMIT_INIT(1, 1); - VLOG_ERR_RL(&rl, "Port %s has Gateway_Chassis " - "with mixed encapsulations, only " - "uniform encapsulations are " - "supported.", - binding->logical_port); - goto out; - } + for (size_t i = 0; i < ha_ch_ordered->n_ha_ch; i++) { + const struct sbrec_chassis *ch = + ha_ch_ordered->ha_ch[i].chassis; + if (!ch) { + continue; + } + if (!tun) { + tun = chassis_tunnel_find(ch->name, NULL); + } else { + struct chassis_tunnel *chassis_tunnel = + chassis_tunnel_find(ch->name, NULL); + if (chassis_tunnel && + tun->type != chassis_tunnel->type) { + static struct vlog_rate_limit rl = + VLOG_RATE_LIMIT_INIT(1, 1); + VLOG_ERR_RL(&rl, "Port %s has Gateway_Chassis " + "with mixed encapsulations, only " + "uniform encapsulations are " + "supported.", + binding->logical_port); + goto out; } } } if (!tun) { static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 1); - VLOG_ERR_RL(&rl, "No tunnel endpoint found for gateways in " - "Gateway_Chassis of port %s", + VLOG_ERR_RL(&rl, "No tunnel endpoint found for HA chassis in " + "HA chassis group of port %s", binding->logical_port); goto out; } @@ -791,24 +793,27 @@ consider_port_binding(struct ovsdb_idl_index *sbrec_chassis_by_name, /* Output to tunnels with active/backup */ struct ofpact_bundle *bundle = ofpact_put_BUNDLE(ofpacts_p); - LIST_FOR_EACH (gwc, node, gateway_chassis) { - if (gwc->db->chassis) { - tun = chassis_tunnel_find(gwc->db->chassis->name, NULL); - if (!tun) { - continue; - } - if (bundle->n_slaves >= BUNDLE_MAX_SLAVES) { - static struct vlog_rate_limit rl = - VLOG_RATE_LIMIT_INIT(1, 1); - VLOG_WARN_RL(&rl, "Remote endpoints for port beyond " - "BUNDLE_MAX_SLAVES"); - break; - } - ofpbuf_put(ofpacts_p, &tun->ofport, - sizeof tun->ofport); - bundle = ofpacts_p->header; - bundle->n_slaves++; + for (size_t i = 0; i < ha_ch_ordered->n_ha_ch; i++) { + const struct sbrec_chassis *ch = + ha_ch_ordered->ha_ch[i].chassis; + if (!ch) { + continue; + } + tun = chassis_tunnel_find(ch->name, NULL); + if (!tun) { + continue; + } + if (bundle->n_slaves >= BUNDLE_MAX_SLAVES) { + static struct vlog_rate_limit rl = + VLOG_RATE_LIMIT_INIT(1, 1); + VLOG_WARN_RL(&rl, "Remote endpoints for port beyond " + "BUNDLE_MAX_SLAVES"); + break; } + ofpbuf_put(ofpacts_p, &tun->ofport, + sizeof tun->ofport); + bundle = ofpacts_p->header; + bundle->n_slaves++; } bundle->algorithm = NX_BD_ALG_ACTIVE_BACKUP; @@ -822,8 +827,8 @@ consider_port_binding(struct ovsdb_idl_index *sbrec_chassis_by_name, &match, ofpacts_p); } out: - if (gateway_chassis) { - gateway_chassis_destroy(gateway_chassis); + if (ha_ch_ordered) { + ha_chassis_destroy_ordered(ha_ch_ordered); } } @@ -967,8 +972,7 @@ update_ofports(struct simap *old, struct simap *new) } void -physical_run(struct ovsdb_idl_index *sbrec_chassis_by_name, - struct ovsdb_idl_index *sbrec_port_binding_by_name, +physical_run(struct ovsdb_idl_index *sbrec_port_binding_by_name, const struct sbrec_multicast_group_table *multicast_group_table, const struct sbrec_port_binding_table *port_binding_table, enum mf_field_id mff_ovn_geneve, @@ -1119,8 +1123,7 @@ physical_run(struct ovsdb_idl_index *sbrec_chassis_by_name, * 64 for logical-to-physical translation. */ const struct sbrec_port_binding *binding; SBREC_PORT_BINDING_TABLE_FOR_EACH (binding, port_binding_table) { - consider_port_binding(sbrec_chassis_by_name, - sbrec_port_binding_by_name, + consider_port_binding(sbrec_port_binding_by_name, mff_ovn_geneve, ct_zones, active_tunnels, local_datapaths, binding, chassis, diff --git a/ovn/controller/physical.h b/ovn/controller/physical.h index a7a4def37..38c3a8771 100644 --- a/ovn/controller/physical.h +++ b/ovn/controller/physical.h @@ -43,8 +43,7 @@ struct sset; #define OVN_GENEVE_LEN 4 void physical_register_ovs_idl(struct ovsdb_idl *); -void physical_run(struct ovsdb_idl_index *sbrec_chassis_by_name, - struct ovsdb_idl_index *sbrec_port_binding_by_name, +void physical_run(struct ovsdb_idl_index *sbrec_port_binding_by_name, const struct sbrec_multicast_group_table *, const struct sbrec_port_binding_table *, enum mf_field_id mff_ovn_geneve, diff --git a/ovn/controller/pinctrl.c b/ovn/controller/pinctrl.c index 100a20ff2..78a617210 100644 --- a/ovn/controller/pinctrl.c +++ b/ovn/controller/pinctrl.c @@ -23,7 +23,7 @@ #include "dirs.h" #include "dp-packet.h" #include "flow.h" -#include "gchassis.h" +#include "ha-chassis.h" #include "lport.h" #include "nx-match.h" #include "ovn-controller.h" @@ -81,7 +81,6 @@ static void init_send_garps(void); static void destroy_send_garps(void); static void send_garp_wait(void); static void send_garp_run( - struct ovsdb_idl_index *sbrec_chassis_by_name, struct ovsdb_idl_index *sbrec_port_binding_by_datapath, struct ovsdb_idl_index *sbrec_port_binding_by_name, const struct ovsrec_bridge *, @@ -1505,7 +1504,6 @@ pinctrl_recv(const struct sbrec_dns_table *dns_table, void pinctrl_run(struct ovsdb_idl_txn *ovnsb_idl_txn, - struct ovsdb_idl_index *sbrec_chassis_by_name, struct ovsdb_idl_index *sbrec_datapath_binding_by_key, struct ovsdb_idl_index *sbrec_port_binding_by_datapath, struct ovsdb_idl_index *sbrec_port_binding_by_key, @@ -1554,7 +1552,7 @@ pinctrl_run(struct ovsdb_idl_txn *ovnsb_idl_txn, run_put_mac_bindings(ovnsb_idl_txn, sbrec_datapath_binding_by_key, sbrec_port_binding_by_key, sbrec_mac_binding_by_lport_ip); - send_garp_run(sbrec_chassis_by_name, sbrec_port_binding_by_datapath, + send_garp_run(sbrec_port_binding_by_datapath, sbrec_port_binding_by_name, br_int, chassis, local_datapaths, active_tunnels); send_ipv6_ras(sbrec_port_binding_by_datapath, @@ -2329,8 +2327,7 @@ get_localnet_vifs_l3gwports( } static bool -pinctrl_is_chassis_resident(struct ovsdb_idl_index *sbrec_chassis_by_name, - struct ovsdb_idl_index *sbrec_port_binding_by_name, +pinctrl_is_chassis_resident(struct ovsdb_idl_index *sbrec_port_binding_by_name, const struct sbrec_chassis *chassis, const struct sset *active_tunnels, const char *port_name) @@ -2343,13 +2340,8 @@ pinctrl_is_chassis_resident(struct ovsdb_idl_index *sbrec_chassis_by_name, if (strcmp(pb->type, "chassisredirect")) { return pb->chassis == chassis; } else { - struct ovs_list *gateway_chassis = - gateway_chassis_get_ordered(sbrec_chassis_by_name, pb); - bool active = gateway_chassis_is_active(gateway_chassis, - chassis, - active_tunnels); - gateway_chassis_destroy(gateway_chassis); - return active; + return ha_chassis_group_is_active(pb->ha_chassis_group, + active_tunnels, chassis); } } @@ -2420,8 +2412,7 @@ extract_addresses_with_port(const char *addresses, } static void -consider_nat_address(struct ovsdb_idl_index *sbrec_chassis_by_name, - struct ovsdb_idl_index *sbrec_port_binding_by_name, +consider_nat_address(struct ovsdb_idl_index *sbrec_port_binding_by_name, const char *nat_address, const struct sbrec_port_binding *pb, struct sset *nat_address_keys, @@ -2434,7 +2425,7 @@ consider_nat_address(struct ovsdb_idl_index *sbrec_chassis_by_name, if (!extract_addresses_with_port(nat_address, laddrs, &lport) || (!lport && !strcmp(pb->type, "patch")) || (lport && !pinctrl_is_chassis_resident( - sbrec_chassis_by_name, sbrec_port_binding_by_name, chassis, + sbrec_port_binding_by_name, chassis, active_tunnels, lport))) { destroy_lport_addresses(laddrs); free(laddrs); @@ -2454,8 +2445,7 @@ consider_nat_address(struct ovsdb_idl_index *sbrec_chassis_by_name, } static void -get_nat_addresses_and_keys(struct ovsdb_idl_index *sbrec_chassis_by_name, - struct ovsdb_idl_index *sbrec_port_binding_by_name, +get_nat_addresses_and_keys(struct ovsdb_idl_index *sbrec_port_binding_by_name, struct sset *nat_address_keys, struct sset *local_l3gw_ports, const struct sbrec_chassis *chassis, @@ -2473,8 +2463,7 @@ get_nat_addresses_and_keys(struct ovsdb_idl_index *sbrec_chassis_by_name, if (pb->n_nat_addresses) { for (int i = 0; i < pb->n_nat_addresses; i++) { - consider_nat_address(sbrec_chassis_by_name, - sbrec_port_binding_by_name, + consider_nat_address(sbrec_port_binding_by_name, pb->nat_addresses[i], pb, nat_address_keys, chassis, active_tunnels, @@ -2486,8 +2475,7 @@ get_nat_addresses_and_keys(struct ovsdb_idl_index *sbrec_chassis_by_name, const char *nat_addresses_options = smap_get(&pb->options, "nat-addresses"); if (nat_addresses_options) { - consider_nat_address(sbrec_chassis_by_name, - sbrec_port_binding_by_name, + consider_nat_address(sbrec_port_binding_by_name, nat_addresses_options, pb, nat_address_keys, chassis, active_tunnels, @@ -2504,8 +2492,7 @@ send_garp_wait(void) } static void -send_garp_run(struct ovsdb_idl_index *sbrec_chassis_by_name, - struct ovsdb_idl_index *sbrec_port_binding_by_datapath, +send_garp_run(struct ovsdb_idl_index *sbrec_port_binding_by_datapath, struct ovsdb_idl_index *sbrec_port_binding_by_name, const struct ovsrec_bridge *br_int, const struct sbrec_chassis *chassis, @@ -2524,8 +2511,7 @@ send_garp_run(struct ovsdb_idl_index *sbrec_chassis_by_name, br_int, chassis, local_datapaths, &localnet_vifs, &local_l3gw_ports); - get_nat_addresses_and_keys(sbrec_chassis_by_name, - sbrec_port_binding_by_name, + get_nat_addresses_and_keys(sbrec_port_binding_by_name, &nat_ip_keys, &local_l3gw_ports, chassis, active_tunnels, &nat_addresses); diff --git a/ovn/controller/pinctrl.h b/ovn/controller/pinctrl.h index 697d02478..f61d7056e 100644 --- a/ovn/controller/pinctrl.h +++ b/ovn/controller/pinctrl.h @@ -32,7 +32,6 @@ struct sbrec_dns_table; void pinctrl_init(void); void pinctrl_run(struct ovsdb_idl_txn *ovnsb_idl_txn, - struct ovsdb_idl_index *sbrec_chassis_by_name, struct ovsdb_idl_index *sbrec_datapath_binding_by_key, struct ovsdb_idl_index *sbrec_port_binding_by_datapath, struct ovsdb_idl_index *sbrec_port_binding_by_key, From patchwork Thu Mar 14 19:32:19 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Numan Siddique X-Patchwork-Id: 1056679 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=openvswitch.org (client-ip=140.211.169.12; helo=mail.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Received: from mail.linuxfoundation.org (mail.linuxfoundation.org [140.211.169.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 44KzQx130kz9s6w for ; Fri, 15 Mar 2019 06:33:45 +1100 (AEDT) Received: from mail.linux-foundation.org (localhost [127.0.0.1]) by mail.linuxfoundation.org (Postfix) with ESMTP id 71A43CC6; Thu, 14 Mar 2019 19:32:28 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@mail.linuxfoundation.org Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 060D1D07 for ; Thu, 14 Mar 2019 19:32:28 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 9188B83A for ; Thu, 14 Mar 2019 19:32:25 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 3FB6C308794E for ; Thu, 14 Mar 2019 19:32:25 +0000 (UTC) Received: from nusiddiq.mac (unknown [10.67.116.14]) by smtp.corp.redhat.com (Postfix) with ESMTP id 3303317CEB; Thu, 14 Mar 2019 19:32:22 +0000 (UTC) From: nusiddiq@redhat.com To: dev@openvswitch.org Date: Fri, 15 Mar 2019 01:02:19 +0530 Message-Id: <20190314193219.24627-1-nusiddiq@redhat.com> In-Reply-To: <20190314193056.24241-1-nusiddiq@redhat.com> References: <20190314193056.24241-1-nusiddiq@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.45]); Thu, 14 Mar 2019 19:32:25 +0000 (UTC) X-Spam-Status: No, score=-6.9 required=5.0 tests=BAYES_00,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Subject: [ovs-dev] [PATCH v4 4/5] ovn-northd: Delete the references to gateway_chasss in SB DB X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: ovs-dev-bounces@openvswitch.org Errors-To: ovs-dev-bounces@openvswitch.org From: Numan Siddique Previous patch in the series added the support in ovn-controller to use ha_chassis_group table in SB DB to support HA chassis and establishing BFD tunnels instead of the gateway_chassis table. There is no need for ovn-northd to create any gateway_chassis rows in SB DB. This patch does that and deletes the code which is not required anymore. This patch also now supports 'ha_chassis_group' to be associated with a distributed logical router port and ignores 'gateway_chassis' and 'redirect-chassis' if set along with 'ha_chassis_group'. Signed-off-by: Numan Siddique --- NEWS | 1 + ovn/northd/ovn-northd.c | 321 +++++++++++++++++++--------------------- tests/ovn-northd.at | 231 +++++++++++++++++++++++++---- tests/ovn.at | 175 ++++++++++++++++++++++ 4 files changed, 529 insertions(+), 199 deletions(-) diff --git a/NEWS b/NEWS index 89d0f19d6..74adb2562 100644 --- a/NEWS +++ b/NEWS @@ -24,6 +24,7 @@ Post-v2.11.0 protocol extension. - OVN: * Select IPAM mac_prefix in a random manner if not provided by the user + * Added the HA chassis group support and deprecated Gateway chassis. v2.11.0 - 19 Feb 2019 --------------------- diff --git a/ovn/northd/ovn-northd.c b/ovn/northd/ovn-northd.c index 48092d8f5..be8dc36a6 100644 --- a/ovn/northd/ovn-northd.c +++ b/ovn/northd/ovn-northd.c @@ -1732,7 +1732,8 @@ join_logical_ports(struct northd_context *ctx, const char *redirect_chassis = smap_get(&op->nbrp->options, "redirect-chassis"); - if (redirect_chassis || op->nbrp->n_gateway_chassis) { + if (op->nbrp->ha_chassis_group || redirect_chassis || + op->nbrp->n_gateway_chassis) { /* Additional "derived" ovn_port crp represents the * instance of op on the "redirect-chassis". */ const char *gw_chassis = smap_get(&op->od->nbr->options, @@ -1966,112 +1967,54 @@ get_nat_addresses(const struct ovn_port *op, size_t *n) return addresses; } -static bool -gateway_chassis_equal(const struct nbrec_gateway_chassis *nb_gwc, - const struct sbrec_chassis *nb_gwc_c, - const struct sbrec_gateway_chassis *sb_gwc) -{ - bool equal = !strcmp(nb_gwc->name, sb_gwc->name) - && nb_gwc->priority == sb_gwc->priority - && smap_equal(&nb_gwc->options, &sb_gwc->options) - && smap_equal(&nb_gwc->external_ids, &sb_gwc->external_ids); - - if (!equal) { - return false; - } - - /* If everything else matched and we were unable to find the SBDB - * Chassis entry at this time, assume a match and return true. - * This happens when an ovn-controller is restarting and the Chassis - * entry is gone away momentarily */ - return !nb_gwc_c - || (sb_gwc->chassis && !strcmp(nb_gwc_c->name, - sb_gwc->chassis->name)); -} - static bool sbpb_gw_chassis_needs_update( - struct ovsdb_idl_index *sbrec_chassis_by_name, - const struct sbrec_port_binding *port_binding, + const struct sbrec_port_binding *pb, const struct nbrec_logical_router_port *lrp) { - if (!lrp || !port_binding) { + if (!lrp || !pb) { return false; } - if (lrp->n_gateway_chassis && !port_binding->ha_chassis_group) { + if (lrp->n_gateway_chassis && !pb->ha_chassis_group) { /* If there are gateway chassis in the NB DB, but there is * no corresponding HA chassis group in SB DB we need to * create the HA chassis group in SB DB for this lrp. */ return true; } - /* These arrays are used to collect valid Gateway_Chassis and valid - * Chassis records from the Logical_Router_Port Gateway_Chassis list, - * we ignore the ones we can't match on the SBDB */ - struct nbrec_gateway_chassis **lrp_gwc = xzalloc(lrp->n_gateway_chassis * - sizeof *lrp_gwc); - const struct sbrec_chassis **lrp_gwc_c = xzalloc(lrp->n_gateway_chassis * - sizeof *lrp_gwc_c); - - /* Count the number of gateway chassis chassis names from the logical - * router port that we are able to match on the southbound database */ - int lrp_n_gateway_chassis = 0; - int n; - for (n = 0; n < lrp->n_gateway_chassis; n++) { - - if (!lrp->gateway_chassis[n]->chassis_name) { - continue; - } - - const struct sbrec_chassis *chassis = - chassis_lookup_by_name(sbrec_chassis_by_name, - lrp->gateway_chassis[n]->chassis_name); - - lrp_gwc_c[lrp_n_gateway_chassis] = chassis; - lrp_gwc[lrp_n_gateway_chassis] = lrp->gateway_chassis[n]; - lrp_n_gateway_chassis++; - if (!chassis) { - static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 1); - VLOG_WARN_RL( - &rl, "Chassis name %s referenced in NBDB via Gateway_Chassis " - "on logical router port %s does not exist in SBDB", - lrp->gateway_chassis[n]->chassis_name, lrp->name); - } + if (strcmp(pb->ha_chassis_group->name, lrp->name)) { + /* Name doesn't match. */ + return true; } - /* Basic check, different amount of Gateway_Chassis means that we - * need to update southbound database Port_Binding */ - if (lrp_n_gateway_chassis != port_binding->n_gateway_chassis) { - free(lrp_gwc_c); - free(lrp_gwc); + if (lrp->n_gateway_chassis != pb->ha_chassis_group->n_ha_chassis) { return true; } - for (n = 0; n < lrp_n_gateway_chassis; n++) { - int i; - /* For each of the valid gw chassis on the lrp, check if there's - * a match on the Port_Binding list, we assume order is not - * persisted */ - for (i = 0; i < port_binding->n_gateway_chassis; i++) { - if (gateway_chassis_equal(lrp_gwc[n], - lrp_gwc_c[n], - port_binding->gateway_chassis[i])) { - break; /* we found a match */ + for (size_t i = 0; i < lrp->n_gateway_chassis; i++) { + struct nbrec_gateway_chassis *nbgw_ch = lrp->gateway_chassis[i]; + bool found = false; + for (size_t j = 0; j < pb->ha_chassis_group->n_ha_chassis; j++) { + struct sbrec_ha_chassis *sbha_ch = + pb->ha_chassis_group->ha_chassis[j]; + if (sbha_ch->chassis && + !strcmp(nbgw_ch->chassis_name, sbha_ch->chassis->name)) { + if (nbgw_ch->priority == sbha_ch->priority) { + found = true; + break; + } + /* Priority has changed. Return true. */ + return true; } } - /* if no Port_Binding gateway chassis matched for the entry... */ - if (i == port_binding->n_gateway_chassis) { - free(lrp_gwc_c); - free(lrp_gwc); - return true; /* found no match for this gateway chassis on lrp */ + if (!found) { + return true; } } - /* no need for update, all ports matched */ - free(lrp_gwc_c); - free(lrp_gwc); + /* No need to update SB DB. Its in sync. */ return false; } @@ -2086,17 +2029,101 @@ create_sb_ha_chassis(struct northd_context *ctx, return sb_ha_chassis; } +static bool +chassis_group_list_changed( + const struct nbrec_ha_chassis_group *nb_ha_grp, + const struct sbrec_ha_chassis_group *sb_ha_grp) +{ + if (nb_ha_grp->n_ha_chassis != sb_ha_grp->n_ha_chassis) { + return true; + } + + struct shash nb_ha_chassis_list = SHASH_INITIALIZER(&nb_ha_chassis_list); + for (size_t i = 0; i < nb_ha_grp->n_ha_chassis; i++) { + shash_add(&nb_ha_chassis_list, + nb_ha_grp->ha_chassis[i]->chassis_name, + nb_ha_grp->ha_chassis[i]); + } + + bool changed = false; + const struct sbrec_ha_chassis *sb_ha_chassis; + const struct nbrec_ha_chassis *nb_ha_chassis; + for (size_t i = 0; i < sb_ha_grp->n_ha_chassis; i++) { + sb_ha_chassis = sb_ha_grp->ha_chassis[i]; + if (!sb_ha_chassis->chassis) { + /* This can happen, if ovn-controller exits gracefully in + * a chassis, in which case, chassis row for that chassis + * would be NULL. */ + changed = true; + break; + } + + nb_ha_chassis = shash_find_and_delete(&nb_ha_chassis_list, + sb_ha_chassis->chassis->name); + if (!nb_ha_chassis || + nb_ha_chassis->priority != sb_ha_chassis->priority) { + changed = true; + break; + } + } + + struct shash_node *node, *next; + SHASH_FOR_EACH_SAFE (node, next, &nb_ha_chassis_list) { + shash_delete(&nb_ha_chassis_list, node); + changed = true; + } + shash_destroy(&nb_ha_chassis_list); + + return changed; +} + +static void +sync_ha_chassis_group_for_sbpb(struct northd_context *ctx, + const struct nbrec_ha_chassis_group *nb_ha_grp, + struct ovsdb_idl_index *sbrec_chassis_by_name, + const struct sbrec_port_binding *pb) +{ + bool new_sb_chassis_group = false; + const struct sbrec_ha_chassis_group *sb_ha_grp = + ha_chassis_group_lookup_by_name( + ctx->sbrec_ha_chassis_grp_by_name, nb_ha_grp->name); + + if (!sb_ha_grp) { + sb_ha_grp = sbrec_ha_chassis_group_insert(ctx->ovnsb_txn); + sbrec_ha_chassis_group_set_name(sb_ha_grp, nb_ha_grp->name); + new_sb_chassis_group = true; + } + + if (new_sb_chassis_group || + chassis_group_list_changed(nb_ha_grp, sb_ha_grp)) { + struct sbrec_ha_chassis **sb_ha_chassis = NULL; + size_t n_ha_chassis = nb_ha_grp->n_ha_chassis; + sb_ha_chassis = xcalloc(n_ha_chassis, sizeof *sb_ha_chassis); + for (size_t i = 0; i < nb_ha_grp->n_ha_chassis; i++) { + const struct nbrec_ha_chassis *nb_ha_chassis + = nb_ha_grp->ha_chassis[i]; + const struct sbrec_chassis *chassis = + chassis_lookup_by_name(sbrec_chassis_by_name, + nb_ha_chassis->chassis_name); + sb_ha_chassis[i] = sbrec_ha_chassis_insert(ctx->ovnsb_txn); + /* It's perfectly ok if the chassis is NULL. This could + * happen when ovn-controller exits and removes its row + * from the chassis table in OVN SB DB. */ + sbrec_ha_chassis_set_chassis(sb_ha_chassis[i], chassis); + sbrec_ha_chassis_set_priority(sb_ha_chassis[i], + nb_ha_chassis->priority); + } + sbrec_ha_chassis_group_set_ha_chassis(sb_ha_grp, sb_ha_chassis, + n_ha_chassis); + free(sb_ha_chassis); + } + + sbrec_port_binding_set_ha_chassis_group(pb, sb_ha_grp); +} + /* This functions translates the gw chassis on the nb database - * to sb database entries, the only difference is that SB database - * Gateway_Chassis table references the chassis directly instead - * of using the name. - * - * This function also creates a HA Chassis group in SB DB for - * the gateway chassis associated to a distributed gateway - * router port in the NB DB. - * - * An upcoming patch will delete the code to create the Gateway chassis - * in SB DB.*/ + * to HA chassis group in the sb database entries. + */ static void copy_gw_chassis_from_nbrp_to_sbpb( struct northd_context *ctx, @@ -2104,14 +2131,9 @@ copy_gw_chassis_from_nbrp_to_sbpb( const struct nbrec_logical_router_port *lrp, const struct sbrec_port_binding *port_binding) { - struct sbrec_gateway_chassis **gw_chassis = NULL; - int n_gwc = 0; - int n; /* Make use of the new HA chassis group table to support HA - * for the distributed gateway router port. We can delete - * the old gateway_chassis code once ovn-controller supports - * HA chassis group. */ + * for the distributed gateway router port. */ const struct sbrec_ha_chassis_group *sb_ha_chassis_group = ha_chassis_group_lookup_by_name( ctx->sbrec_ha_chassis_grp_by_name, lrp->name); @@ -2123,16 +2145,7 @@ copy_gw_chassis_from_nbrp_to_sbpb( struct sbrec_ha_chassis **sb_ha_chassis = xcalloc(lrp->n_gateway_chassis, sizeof *sb_ha_chassis); size_t n_sb_ha_ch = 0; - /* XXX: This can be improved. This code will generate a set of new - * Gateway_Chassis and push them all in a single transaction, instead - * this would be more optimal if we just add/update/remove the rows in - * the southbound db that need to change. We don't expect lots of - * changes to the Gateway_Chassis table, but if that proves to be wrong - * we should optimize this. - * - * Note: Remove the below code to add gateway_chassis row in OVN - * Southbound db once ovn-controller supports HA chassis group. */ - for (n = 0; n < lrp->n_gateway_chassis; n++) { + for (size_t n = 0; n < lrp->n_gateway_chassis; n++) { struct nbrec_gateway_chassis *lrp_gwc = lrp->gateway_chassis[n]; if (!lrp_gwc->chassis_name) { continue; @@ -2142,27 +2155,10 @@ copy_gw_chassis_from_nbrp_to_sbpb( chassis_lookup_by_name(sbrec_chassis_by_name, lrp_gwc->chassis_name); - gw_chassis = xrealloc(gw_chassis, (n_gwc + 1) * sizeof *gw_chassis); - - /* This code to create gateway_chassis in SB DB needs to be deleted - * once ovn-controller supports making use of HA chassis groups. */ - struct sbrec_gateway_chassis *pb_gwc = - sbrec_gateway_chassis_insert(ctx->ovnsb_txn); - - sbrec_gateway_chassis_set_name(pb_gwc, lrp_gwc->name); - sbrec_gateway_chassis_set_priority(pb_gwc, lrp_gwc->priority); - sbrec_gateway_chassis_set_chassis(pb_gwc, chassis); - sbrec_gateway_chassis_set_options(pb_gwc, &lrp_gwc->options); - sbrec_gateway_chassis_set_external_ids(pb_gwc, &lrp_gwc->external_ids); - - gw_chassis[n_gwc++] = pb_gwc; - sb_ha_chassis[n_sb_ha_ch] = create_sb_ha_chassis(ctx, chassis, lrp_gwc->priority); n_sb_ha_ch++; } - sbrec_port_binding_set_gateway_chassis(port_binding, gw_chassis, n_gwc); - free(gw_chassis); sbrec_ha_chassis_group_set_ha_chassis(sb_ha_chassis_group, sb_ha_chassis, n_sb_ha_ch); @@ -2195,18 +2191,27 @@ ovn_port_update_sbrec(struct northd_context *ctx, if (op->derived) { const char *redirect_chassis = smap_get(&op->nbrp->options, "redirect-chassis"); - if (op->nbrp->n_gateway_chassis && redirect_chassis) { + if (op->nbrp->ha_chassis_group && op->nbrp->n_gateway_chassis) { static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 1); VLOG_WARN_RL( - &rl, "logical router port %s has both options:" - "redirect-chassis and gateway_chassis populated " - "redirect-chassis will be ignored in favour of " - "gateway chassis", op->nbrp->name); + &rl, "logical router port %s has both:" + "ha_chassis_group and gateway_chassis populated " + "gateway_chassis will be ignored in favour of " + "ha_chassis_group", op->nbrp->name); } - if (op->nbrp->n_gateway_chassis) { - if (sbpb_gw_chassis_needs_update(sbrec_chassis_by_name, - op->sb, op->nbrp)) { + if (op->nbrp->ha_chassis_group) { + /* HA Chassis group is set. Ignore 'gateway_chassis' + * column and redirect-chassis option. */ + sync_ha_chassis_group_for_sbpb(ctx, op->nbrp->ha_chassis_group, + sbrec_chassis_by_name, op->sb); + sset_add(active_ha_chassis_grps, + op->nbrp->ha_chassis_group->name); + } else if (op->nbrp->n_gateway_chassis) { + /* Legacy gateway_chassis support. + * Create ha_chassis_group for the Northbound gateway_chassis + * associated with the lrp. */ + if (sbpb_gw_chassis_needs_update(op->sb, op->nbrp)) { copy_gw_chassis_from_nbrp_to_sbpb(ctx, sbrec_chassis_by_name, op->nbrp, op->sb); @@ -2216,7 +2221,7 @@ ovn_port_update_sbrec(struct northd_context *ctx, } else if (redirect_chassis) { /* Handle ports that had redirect-chassis option attached * to them, and for backwards compatibility convert them - * to a single Gateway_Chassis entry */ + * to a single HA Chassis group entry */ const struct sbrec_chassis *chassis = chassis_lookup_by_name(sbrec_chassis_by_name, redirect_chassis); @@ -2224,36 +2229,7 @@ ovn_port_update_sbrec(struct northd_context *ctx, /* If we found the chassis, and the gw chassis on record * differs from what we expect go ahead and update */ char *gwc_name = xasprintf("%s_%s", op->nbrp->name, - chassis->name); - if (op->sb->n_gateway_chassis != 1 - || !op->sb->gateway_chassis[0]->chassis - || strcmp(op->sb->gateway_chassis[0]->chassis->name, - chassis->name) - || op->sb->gateway_chassis[0]->priority != 0) { - /* This code to create gateway_chassis in SB DB needs - * to be deleted once ovn-controller supports making - * use of HA chassis groups. */ - - /* Construct a single Gateway_Chassis entry on the - * Port_Binding attached to the redirect_chassis - * name */ - struct sbrec_gateway_chassis *gw_chassis = - sbrec_gateway_chassis_insert(ctx->ovnsb_txn); - - /* XXX: Again, here, we could just update an existing - * Gateway_Chassis, instead of creating a new one - * and replacing it */ - sbrec_gateway_chassis_set_name(gw_chassis, gwc_name); - sbrec_gateway_chassis_set_priority(gw_chassis, 0); - sbrec_gateway_chassis_set_chassis(gw_chassis, chassis); - sbrec_gateway_chassis_set_external_ids(gw_chassis, - &op->nbrp->external_ids); - sbrec_port_binding_set_gateway_chassis(op->sb, - &gw_chassis, 1); - } - - /* Create HA chassis group in SB DB for the - * redirect-chassis option. */ + chassis->name); const struct sbrec_ha_chassis_group *sb_ha_ch_grp; sb_ha_ch_grp = ha_chassis_group_lookup_by_name( ctx->sbrec_ha_chassis_grp_by_name, gwc_name); @@ -2279,11 +2255,20 @@ ovn_port_update_sbrec(struct northd_context *ctx, VLOG_WARN("chassis name '%s' from redirect from logical " " router port '%s' redirect-chassis not found", redirect_chassis, op->nbrp->name); - if (op->sb->n_gateway_chassis) { - sbrec_port_binding_set_gateway_chassis(op->sb, NULL, - 0); + if (op->sb->ha_chassis_group) { + sbrec_port_binding_set_ha_chassis_group(op->sb, NULL); } } + } else { + /* Nothing is set. Clear ha_chassis_group from pb. */ + if (op->sb->ha_chassis_group) { + sbrec_port_binding_set_ha_chassis_group(op->sb, NULL); + } + } + + if (op->sb->n_gateway_chassis) { + /* Delete the legacy gateway_chassis from the pb. */ + sbrec_port_binding_set_gateway_chassis(op->sb, NULL, 0); } smap_add(&new, "distributed-port", op->nbrp->name); } else { diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at index b4e8995be..efaec250f 100644 --- a/tests/ovn-northd.at +++ b/tests/ovn-northd.at @@ -20,18 +20,22 @@ ovn-nbctl --wait=sb \ set Logical_Router_Port alice 'gateway_chassis=[@gc0,@gc1]' nb_gwc1_uuid=`ovn-nbctl --bare --columns _uuid find Gateway_Chassis name="alice_gw1"` -gwc1_uuid=`ovn-sbctl --bare --columns _uuid find Gateway_Chassis name="alice_gw1"` -gwc2_uuid=`ovn-sbctl --bare --columns _uuid find Gateway_Chassis name="alice_gw2"` +# With the new ha_chassis_group table added, there should be no rows in +# gateway_chassis table in SB DB. +AT_CHECK([ovn-sbctl list gateway_chassis | wc -l], [0], [0 +]) -echo "Port_Binding for cr-alice:" -ovn-sbctl find port_binding logical_port="cr-alice" -echo "Gateway_Chassis list:" -ovn-sbctl list Gateway_Chassis +# There should be one ha_chassis_group with the name "alice" +ha_chassi_grp_name=`ovn-sbctl --bare --columns name find \ +ha_chassis_group name="alice"` -AT_CHECK([ovn-sbctl --bare --columns gateway_chassis find port_binding logical_port="cr-alice" | grep $gwc1_uuid | wc -l], [0], [1 -]) -AT_CHECK([ovn-sbctl --bare --columns gateway_chassis find port_binding logical_port="cr-alice" | grep $gwc2_uuid | wc -l], [0], [1 +AT_CHECK([test $ha_chassi_grp_name = alice]) + +ha_chgrp_uuid=`ovn-sbctl --bare --columns _uuid find ha_chassis_group name=alice` + +AT_CHECK([ovn-sbctl --bare --columns ha_chassis_group find port_binding \ +logical_port="cr-alice" | grep $ha_chgrp_uuid | wc -l], [0], [1 ]) # There should be one ha_chassis_group with the name "alice" @@ -62,15 +66,35 @@ ha_ch_list=`echo $ha_ch_list | sed 's/ //g'` AT_CHECK([test "$ha_ch_list" = "$ha_ch"]) # delete the 2nd Gateway_Chassis on NBDB for alice port +gw_ch=`ovn-sbctl --bare --columns gateway_chassis find port_binding \ +logical_port="cr-alice"` +AT_CHECK([test "$gw_ch" = ""]) -ovn-nbctl --wait=sb set Logical_Router_Port alice gateway_chassis=${nb_gwc1_uuid} +ha_ch=`ovn-sbctl --bare --columns ha_chassis find ha_chassis_group` +ha_ch=`echo $ha_ch | sed 's/ //g'` +# Trim the spaces. +echo "ha ch in grp = $ha_ch" + +ha_ch_list='' +for i in `ovn-sbctl --bare --columns _uuid list ha_chassis | sort` +do + ha_ch_list="$ha_ch_list $i" +done + +# Trim the spaces. +ha_ch_list=`echo $ha_ch_list | sed 's/ //g'` + +AT_CHECK([test "$ha_ch_list" = "$ha_ch"]) -gwc1_uuid=`ovn-sbctl --bare --columns _uuid find Gateway_Chassis name="alice_gw1"` +# delete the 2nd Gateway_Chassis on NBDB for alice port +ovn-nbctl --wait=sb set Logical_Router_Port alice gateway_chassis=${nb_gwc1_uuid} -AT_CHECK([ovn-sbctl --bare --columns gateway_chassis find port_binding logical_port="cr-alice" | grep $gwc1_uuid | wc -l], [0], [1 +# There should be only 1 row in ha_chassis SB DB table. +AT_CHECK([ovn-sbctl --bare --columns _uuid list ha_chassis | wc -l], [0], [1 ]) -AT_CHECK([ovn-sbctl find Gateway_Chassis name=alice_gw2], [0], []) +AT_CHECK([ovn-sbctl list gateway_chassis | wc -l], [0], [0 +]) # There should be only 1 row in ha_chassis SB DB table. AT_CHECK([ovn-sbctl --bare --columns _uuid list ha_chassis | wc -l], [0], [1 @@ -80,10 +104,15 @@ AT_CHECK([ovn-sbctl --bare --columns _uuid list ha_chassis | wc -l], [0], [1 ovn-nbctl --wait=sb clear Logical_Router_Port alice gateway_chassis -# expect that the Gateway_Chassis doesn't exist anymore +# expect that the ha_chassis doesn't exist anymore +AT_CHECK([ovn-sbctl list gateway_chassis | wc -l], [0], [0 +]) -AT_CHECK([ovn-sbctl find Gateway_Chassis name=alice_gw1], [0], []) -AT_CHECK([ovn-sbctl find Gateway_Chassis name=alice_gw2], [0], []) +AT_CHECK([ovn-sbctl list ha_chassis | wc -l], [0], [0 +]) + +AT_CHECK([ovn-sbctl list ha_chassis_group | wc -l], [0], [0 +]) # expect that the ha_chassis doesn't exist anymore AT_CHECK([ovn-sbctl list ha_chassis | wc -l], [0], [0 @@ -105,19 +134,40 @@ ovn-nbctl --wait=sb lrp-add R1 bob 00:00:02:01:02:03 172.16.1.1/24 \ -- set Logical_Router_Port bob options:redirect-chassis="gw1" -# It should be converted to Gateway_Chassis entries in SBDB, and +# It should be converted to ha_chassis_group entries in SBDB, and # still redirect-chassis is kept for backwards compatibility -gwc1_uuid=`ovn-sbctl --bare --columns _uuid find Gateway_Chassis name="bob_gw1"` +AT_CHECK([ovn-sbctl list gateway_chassis | wc -l], [0], [0 +]) + +AT_CHECK([ovn-sbctl --bare --columns _uuid list ha_chassis | wc -l], [0], [1 +]) -AT_CHECK([ovn-sbctl --bare --columns gateway_chassis find port_binding logical_port="cr-bob" | grep $gwc1_uuid | wc -l], [0], [1 +AT_CHECK([ovn-sbctl --bare --columns _uuid list ha_chassis_group | wc -l], [0], [1 +]) + +# There should be one ha_chassis_group with the name "bob_gw1" +ha_chassi_grp_name=`ovn-sbctl --bare --columns name find \ +ha_chassis_group name="bob_gw1"` + +AT_CHECK([test $ha_chassi_grp_name = bob_gw1]) + +ha_chgrp_uuid=`ovn-sbctl --bare --columns _uuid find ha_chassis_group name=bob_gw1` + +AT_CHECK([ovn-sbctl --bare --columns ha_chassis_group find port_binding \ +logical_port="cr-bob" | grep $ha_chgrp_uuid | wc -l], [0], [1 ]) ovn-nbctl --wait=sb remove Logical_Router_Port bob options redirect-chassis -# expect that the Gateway_Chassis doesn't exist anymore +# expect that the ha_chassis/ha_chassis_group doesn't exist anymore AT_CHECK([ovn-sbctl find Gateway_Chassis name=bob_gw1], [0], []) +AT_CHECK([ovn-sbctl list ha_chassis | wc -l], [0], [0 +]) + +AT_CHECK([ovn-sbctl list ha_chassis_group | wc -l], [0], [0 +]) AT_CLEANUP @@ -347,8 +397,7 @@ ovn-nbctl --wait=sb ha-chassis-group-add hagrp1 # ovn-northd should not create HA chassis group and HA chassis rows # unless the HA chassis group in OVN NB DB is associated to -# a logical router port. ovn-northd still doesn't support -# associating a HA chassis group to a logical router port. +# a logical router port. AT_CHECK([ovn-sbctl --bare --columns name find ha_chassis_group name="hagrp1" \ | wc -l], [0], [0 ]) @@ -372,25 +421,112 @@ AT_CHECK([ovn-sbctl list ha_chassis | grep chassis | awk '{print $3}' \ | grep -v '-' | wc -l ], [0], [0 ]) -ovn-nbctl ha-chassis-group-del hagrp1 -OVS_WAIT_UNTIL([test 0 = `ovn-sbctl list ha_chassis_group | wc -l`]) -OVS_WAIT_UNTIL([test 0 = `ovn-sbctl list ha_chassis | wc -l`]) - -# Create a logical router port and attach Gateway chassis. -# ovn-northd should create HA chassis group rows in SB DB. +# Create a logical router port and attach ha chassis group. ovn-nbctl lr-add lr0 +ovn-nbctl lrp-add lr0 lr0-public 00:00:20:20:12:13 172.168.0.100/24 + +hagrp1_uuid=`ovn-nbctl --bare --columns _uuid find ha_chassis_group name=hagrp1` +ovn-nbctl set logical_router_port lr0-public ha_chassis_group=$hagrp1_uuid + +OVS_WAIT_UNTIL([test 1 = `ovn-sbctl --bare --columns name find \ +ha_chassis_group name="hagrp1" | wc -l`]) + +AT_CHECK([test 3 = `ovn-sbctl list ha_chassis | grep chassis | wc -l`]) + +# 2 HA chassis should be created with 'chassis' column empty because +# we have not added hv1 and hv2 chassis to the SB DB. +AT_CHECK([test 2 = `ovn-sbctl list ha_chassis | grep chassis | awk '{print $3}' \ +| grep -v '-' | wc -l`]) + +# We should have 1 ha chassis with 'chassis' column set for hv1 +AT_CHECK([test 1 = `ovn-sbctl list ha_chassis | grep chassis | awk '{print $3}' \ +| grep '-' | wc -l`]) + +# Create another logical router port and associate to the same ha_chasis_group +ovn-nbctl lr-add lr1 +ovn-nbctl lrp-add lr1 lr1-public 00:00:20:20:12:14 182.168.0.100/24 + +ovn-nbctl set logical_router_port lr1-public ha_chassis_group=$hagrp1_uuid + +# We should still have 1 HA chassis group and 3 HA chassis in SB DB. +OVS_WAIT_UNTIL([test 1 = `ovn-sbctl --bare --columns name find \ +ha_chassis_group name="hagrp1" | wc -l`]) + +AT_CHECK([test 3 = `ovn-sbctl list ha_chassis | grep chassis | wc -l`]) + +# Change the priority of ch1 - ha chassis in NB DB. It should get +# reflected in SB DB. +ovn-nbctl --wait=sb ha-chassis-group-add-chassis hagrp1 ch1 100 + +OVS_WAIT_UNTIL([test 1 = `ovn-sbctl --bare --columns priority find \ +ha_chassis | grep 100 | wc -l`]) + +# Delete ch1 HA chassis in NB DB. +ovn-nbctl --wait=sb ha-chassis-group-remove-chassis hagrp1 ch1 + +OVS_WAIT_UNTIL([test 2 = `ovn-sbctl list ha_chassis | grep chassis | wc -l`]) + +# Add back the ha chassis +ovn-nbctl --wait=sb ha-chassis-group-add-chassis hagrp1 ch1 40 +OVS_WAIT_UNTIL([test 3 = `ovn-sbctl list ha_chassis | grep chassis | wc -l`]) + +# Delete lr0-public. We should still have 1 HA chassis group and +# 3 HA chassis in SB DB. +ovn-nbctl --wait=sb lrp-del lr0-public + +OVS_WAIT_UNTIL([test 1 = `ovn-sbctl --bare --columns name find \ +ha_chassis_group name="hagrp1" | wc -l`]) + +AT_CHECK([test 3 = `ovn-sbctl list ha_chassis | grep chassis | wc -l`]) + +# Delete lr1-public. There should be no HA chassis group in SB DB. +ovn-nbctl --wait=sb lrp-del lr1-public + +OVS_WAIT_UNTIL([test 0 = `ovn-sbctl --bare --columns name find \ +ha_chassis_group name="hagrp1" | wc -l`]) +AT_CHECK([test 0 = `ovn-sbctl list ha_chassis | grep chassis | wc -l`]) + +# Add lr0-public again ovn-nbctl lrp-add lr0 lr0-public 00:00:20:20:12:13 172.168.0.100/24 -ovn-nbctl lrp-set-gateway-chassis lr0-public ch1 20 +ovn-nbctl set logical_router_port lr0-public ha_chassis_group=$hagrp1_uuid + +OVS_WAIT_UNTIL([test 1 = `ovn-sbctl --bare --columns name find \ +ha_chassis_group name="hagrp1" | wc -l`]) + +AT_CHECK([test 3 = `ovn-sbctl list ha_chassis | grep chassis | wc -l`]) + +# Create a Gateway chassis. ovn-northd should ignore this. +ovn-nbctl lrp-set-gateway-chassis lr0-public ch-1 20 -OVS_WAIT_UNTIL([test 1 = `ovn-sbctl --bare --columns name find ha_chassis_group \ -name="lr0-public" | wc -l`]) +# There should be only 1 HA chassis group in SB DB with the +# name hagrp1. +OVS_WAIT_UNTIL([test 1 = `ovn-sbctl --bare --columns name find \ +ha_chassis_group | wc -l`]) + +OVS_WAIT_UNTIL([test 1 = `ovn-sbctl --bare --columns name find \ +ha_chassis_group name="hagrp1" | wc -l`]) + +AT_CHECK([test 3 = `ovn-sbctl list ha_chassis | grep chassis | wc -l`]) + +# Now delete HA chassis group. ovn-northd should create HA chassis group +# with the Gateway chassis name +ovn-nbctl ha-chassis-group-del hagrp1 + +OVS_WAIT_UNTIL([test 0 = `ovn-sbctl --bare --columns name find \ +ha_chassis_group name="hagrp1" | wc -l`]) + +OVS_WAIT_UNTIL([test 1 = `ovn-sbctl --bare --columns name find \ +ha_chassis_group name="lr0-public" | wc -l`]) OVS_WAIT_UNTIL([test 1 = `ovn-sbctl --bare --columns _uuid \ find ha_chassis | wc -l`]) ovn-nbctl lrp-set-gateway-chassis lr0-public ch2 10 +OVS_WAIT_UNTIL([test 1 = `ovn-sbctl --bare --columns name find \ +ha_chassis_group name="lr0-public" | wc -l`]) + ovn-sbctl --bare --columns _uuid find ha_chassis OVS_WAIT_UNTIL([test 2 = `ovn-sbctl list ha_chassis | grep chassis | wc -l`]) @@ -562,4 +698,37 @@ OVS_WAIT_UNTIL( ref_ch_list=`echo $ref_ch_list | sed 's/ //g'` test "$comp2_ch_uuid" = "$ref_ch_list"]) +# Set redirect-chassis option to lr0-public. It should be ignored. +ovn-nbctl set logical_router_port lr0-public options:redirect-chassis=ch1 + +OVS_WAIT_UNTIL([test 1 = `ovn-sbctl --bare --columns name find \ +ha_chassis_group | wc -l`]) + +OVS_WAIT_UNTIL([test 1 = `ovn-sbctl --bare --columns name find \ +ha_chassis_group name="lr0-public" | wc -l`]) + +ovn-sbctl --bare --columns _uuid find ha_chassis +OVS_WAIT_UNTIL([test 2 = `ovn-sbctl list ha_chassis | grep chassis | wc -l`]) + +# Delete the gateway chassis. HA chassis group should be created in SB DB +# for the redirect-chassis option. +ovn-nbctl clear logical_router_port lr0-public gateway_chassis + +OVS_WAIT_UNTIL([test 1 = `ovn-sbctl --bare --columns name find \ +ha_chassis_group | wc -l`]) + +ovn-sbctl list ha_chassis_group + +OVS_WAIT_UNTIL([test 1 = `ovn-sbctl --bare --columns name find \ +ha_chassis_group name="lr0-public_ch1" | wc -l`]) + +ovn-sbctl --bare --columns _uuid find ha_chassis +OVS_WAIT_UNTIL([test 1 = `ovn-sbctl list ha_chassis | grep chassis | wc -l`]) + +# Clear the redirect-chassis option. +ovn-nbctl clear logical_router_port lr0-public options + +OVS_WAIT_UNTIL([test 0 = `ovn-sbctl list ha_chassis_group | wc -l`]) +AT_CHECK([test 0 = `ovn-sbctl list ha_chassis | wc -l`]) + AT_CLEANUP diff --git a/tests/ovn.at b/tests/ovn.at index 04a618801..d6a068e0a 100644 --- a/tests/ovn.at +++ b/tests/ovn.at @@ -10057,6 +10057,181 @@ ovn-nbctl clear Logical_Router_Port outside gateway_chassis OVS_WAIT_UNTIL([test 0 = `ovn-sbctl list ha_chassis_group | wc -l`]) OVS_WAIT_UNTIL([test 0 = `ovn-sbctl list ha_chassis | wc -l`]) +ovn-nbctl remove NB_Global . options "bfd-min-rx" +ovn-nbctl remove NB_Global . options "bfd-min-tx" +ovn-nbctl remove NB_Global . options "bfd-mult" + +# Now test with HA chassis group instead of Gateway chassis in NB DB +ovn-nbctl --wait=sb ha-chassis-group-add hagrp1 + +ovn-nbctl list ha_chassis_group +ovn-nbctl --bare --columns _uuid find ha_chassis_group name=hagrp1 +hagrp1_uuid=`ovn-nbctl --bare --columns _uuid find ha_chassis_group name=hagrp1` +ovn-nbctl --wait=sb ha-chassis-group-add-chassis hagrp1 gw1 30 +ovn-nbctl --wait=sb ha-chassis-group-add-chassis hagrp1 gw2 20 + +# ovn-northd should not create HA chassis group and HA chassis rows +# unless the HA chassis group in OVN NB DB is associated to +# a logical router port. +OVS_WAIT_UNTIL([test 0 = `ovn-sbctl list ha_chassis | wc -l`]) + +# Associate hagrp1 to outside logical router port +ovn-nbctl set Logical_Router_Port outside ha_chassis_group=$hagrp1_uuid + +OVS_WAIT_UNTIL([test 1 = `ovn-sbctl --bare --columns _uuid \ +find ha_chassis_group | wc -l`]) + +OVS_WAIT_UNTIL([test 2 = `ovn-sbctl list ha_chassis | grep chassis | wc -l`]) + +OVS_WAIT_UNTIL([as hv1 ovs-ofctl dump-flows br-int table=32 | \ +grep active_backup | grep slaves:$hv1_gw1_ofport,$hv1_gw2_ofport \ +| wc -l], [0], [1 +]) + +OVS_WAIT_UNTIL([as hv2 ovs-ofctl dump-flows br-int table=32 | \ +grep active_backup | grep slaves:$hv2_gw1_ofport,$hv2_gw2_ofport \ +| wc -l], [0], [1 +]) + +# make sure that flows for handling the outside router port reside on gw1 +OVS_WAIT_UNTIL([as gw1 ovs-ofctl dump-flows br-int table=24 | \ +grep 00:00:02:01:02:04 | wc -l], [0], [[1 +]]) +OVS_WAIT_UNTIL([as gw2 ovs-ofctl dump-flows br-int table=24 | \ +grep 00:00:02:01:02:04 | wc -l], [0], [[0 +]]) + +# make sure ARP responder flows for outside router port reside on gw1 too +OVS_WAIT_UNTIL([as gw1 ovs-ofctl dump-flows br-int table=9 | \ +grep arp_tpa=192.168.0.101 | wc -l], [0], [[1 +]]) +OVS_WAIT_UNTIL([as gw2 ovs-ofctl dump-flows br-int table=9 | grep arp_tpa=192.168.0.101 | wc -l], [0], [[0 +]]) + +# check that the chassis redirect port has been claimed by the gw1 chassis +OVS_WAIT_UNTIL([ovn-sbctl --columns chassis --bare find Port_Binding \ +logical_port=cr-outside | grep $gw1_chassis | wc -l], [0],[[1 +]]) + +# Re add the ovs ports. +for i in 1 2; do + as hv$i + ovs-vsctl -- add-port br-int hv$i-vif1 -- \ + set interface hv$i-vif1 external-ids:iface-id=inside$i \ + options:tx_pcap=hv$i/vif1-tx.pcap \ + options:rxq_pcap=hv$i/vif1-rx.pcap \ + ofport-request=1 +done + +hv1_ch_uuid=`ovn-sbctl --bare --columns _uuid find chassis name="hv1"` +hv2_ch_uuid=`ovn-sbctl --bare --columns _uuid find chassis name="hv2"` + +exp_ref_ch_list='' +for i in `ovn-sbctl --bare --columns _uuid list chassis | sort` +do + if test $i = $hv1_ch_uuid; then + exp_ref_ch_list="${exp_ref_ch_list}$i" + elif test $i = $hv2_ch_uuid; then + exp_ref_ch_list="${exp_ref_ch_list}$i" + fi +done + +OVS_WAIT_UNTIL( + [ref_ch_list=`ovn-sbctl --bare --columns ref_chassis find ha_chassis_group | sort` + # Trim the spaces. + ref_ch_list=`echo $ref_ch_list | sed 's/ //g'` + test "$exp_ref_ch_list" = "$ref_ch_list"]) + +# Increase the priority of gw2 +ovn-nbctl --wait=sb ha-chassis-group-add-chassis hagrp1 gw2 40 + +OVS_WAIT_UNTIL([as hv1 ovs-ofctl dump-flows br-int table=32 | \ +grep active_backup | grep slaves:$hv1_gw2_ofport,$hv1_gw1_ofport \ +| wc -l], [0], [1 +]) + +OVS_WAIT_UNTIL([as hv2 ovs-ofctl dump-flows br-int table=32 | \ +grep active_backup | grep slaves:$hv2_gw2_ofport,$hv2_gw1_ofport \ +| wc -l], [0], [1 +]) + +# check that the chassis redirect port has been reclaimed by the gw2 chassis +OVS_WAIT_UNTIL([ovn-sbctl --columns chassis --bare find Port_Binding \ +logical_port=cr-outside | grep $gw2_chassis | wc -l], [0],[[1 +]]) + +# check BFD enablement on tunnel ports from gw1 ######### +as gw1 +for chassis in gw2 hv1 hv2; do + echo "checking gw1 -> $chassis" + AT_CHECK([ovs-vsctl --bare --columns bfd find Interface name=ovn-$chassis-0],[0], + [[enable=true +]]) +done + +# check BFD enablement on tunnel ports from gw2 ########## +as gw2 +for chassis in gw1 hv1 hv2; do + echo "checking gw2 -> $chassis" + AT_CHECK([ovs-vsctl --bare --columns bfd find Interface name=ovn-$chassis-0],[0], + [[enable=true +]]) +done + +# check BFD enablement on tunnel ports from hv1 ########### +as hv1 +for chassis in gw1 gw2; do + echo "checking hv1 -> $chassis" + AT_CHECK([ovs-vsctl --bare --columns bfd find Interface name=ovn-$chassis-0],[0], + [[enable=true +]]) +done +# make sure BFD is not enabled to hv2, we don't need it +AT_CHECK([ovs-vsctl --bare --columns bfd find Interface name=ovn-hv2-0],[0], + [[ +]]) + +# check BFD enablement on tunnel ports from hv2 ########## +as hv2 +for chassis in gw1 gw2; do + echo "checking hv2 -> $chassis" + AT_CHECK([ovs-vsctl --bare --columns bfd find Interface name=ovn-$chassis-0],[0], + [[enable=true +]]) +done +# make sure BFD is not enabled to hv1, we don't need it +AT_CHECK([ovs-vsctl --bare --columns bfd find Interface name=ovn-hv1-0],[0], + [[ +]]) + +# make sure that flows for handling the outside router port reside on gw2 now +OVS_WAIT_UNTIL([as gw2 ovs-ofctl dump-flows br-int table=24 | \ +grep 00:00:02:01:02:04 | wc -l], [0], [[1 +]]) +OVS_WAIT_UNTIL([as gw1 ovs-ofctl dump-flows br-int table=24 | \ +grep 00:00:02:01:02:04 | wc -l], [0], [[0 +]]) + +# disconnect GW2 from the network, GW1 should take over +as gw2 +port=${sandbox}_br-phys +as main ovs-vsctl del-port n1 $port + +bfd_dump + +# make sure that flows for handling the outside router port reside on gw2 now +OVS_WAIT_UNTIL([as gw1 ovs-ofctl dump-flows br-int table=24 | \ +grep 00:00:02:01:02:04 | wc -l], [0], [[1 +]]) +OVS_WAIT_UNTIL([as gw2 ovs-ofctl dump-flows br-int table=24 | \ +grep 00:00:02:01:02:04 | wc -l], [0], [[0 +]]) + +# check that the chassis redirect port has been reclaimed by the gw1 chassis +OVS_WAIT_UNTIL([ovn-sbctl --columns chassis --bare find Port_Binding \ +logical_port=cr-outside | grep $gw1_chassis | wc -l], [0],[[1 +]]) + OVN_CLEANUP([gw1],[gw2],[hv1],[hv2]) AT_CLEANUP From patchwork Thu Mar 14 19:32:35 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Numan Siddique X-Patchwork-Id: 1056680 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=openvswitch.org (client-ip=140.211.169.12; helo=mail.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Received: from mail.linuxfoundation.org (mail.linuxfoundation.org [140.211.169.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 44KzRc74Hlz9ryj for ; Fri, 15 Mar 2019 06:34:20 +1100 (AEDT) Received: from mail.linux-foundation.org (localhost [127.0.0.1]) by mail.linuxfoundation.org (Postfix) with ESMTP id 5BFF2D88; Thu, 14 Mar 2019 19:32:48 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@mail.linuxfoundation.org Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id B4A60CC8 for ; Thu, 14 Mar 2019 19:32:46 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 7C015D3 for ; Thu, 14 Mar 2019 19:32:41 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 0409B30FAF31 for ; Thu, 14 Mar 2019 19:32:41 +0000 (UTC) Received: from nusiddiq.mac (unknown [10.67.116.14]) by smtp.corp.redhat.com (Postfix) with ESMTP id 9998460F9B; Thu, 14 Mar 2019 19:32:38 +0000 (UTC) From: nusiddiq@redhat.com To: dev@openvswitch.org Date: Fri, 15 Mar 2019 01:02:35 +0530 Message-Id: <20190314193235.24698-1-nusiddiq@redhat.com> In-Reply-To: <20190314193056.24241-1-nusiddiq@redhat.com> References: <20190314193056.24241-1-nusiddiq@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.44]); Thu, 14 Mar 2019 19:32:41 +0000 (UTC) X-Spam-Status: No, score=-6.9 required=5.0 tests=BAYES_00,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Subject: [ovs-dev] [PATCH v4 5/5] ovn: Support a new Logical_Switch_Port.type - 'external' X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: ovs-dev-bounces@openvswitch.org Errors-To: ovs-dev-bounces@openvswitch.org From: Numan Siddique In the case of OpenStack + OVN, when the VMs are booted on hypervisors supporting SR-IOV nics, there are no OVS ports for these VMs. When these VMs sends DHCPv4, DHPCv6 or IPv6 Router Solicitation requests, the local ovn-controller cannot reply to these packets. OpenStack Neutron dhcp agent service needs to be run to serve these requests. With the new logical port type - 'external', OVN itself can handle these requests avoiding the need to deploy any external services like neutron dhcp agent. To make use of this feature, CMS has to - create a logical port for such VMs - set the type to 'external' - create an HA chassis group and associate the logical port to it or associate an already existing HA chassis group. - create a localnet port for the logical switch - configure the ovn-bridge-mappings option in the OVS db. HA chassis with the highest priority becomes the master of the HA chassis group and the ovn-controller running in that 'chassis', claims the Port_Binding for that logical port and it adds the necessary DHCPv4/v6 OF flows. Since the packet enters the logical switch pipeline via the localnet port, the inport register (reg14) is set to the tunnel key of localnet port in the match conditions. In case the chassis goes down for some reason, next higher priority HA chassis becomes the master and claims the port. When the VM with the external port, sends an ARP request for the router ips, only the chassis which has claimed the port, will reply to the ARP requests. Rest of the chassis on receiving these packets drop them in the ingress switch datapath stage - S_SWITCH_IN_EXTERNAL_PORT which is just before S_SWITCH_IN_L2_LKUP. This would guarantee that only the chassis which has claimed the external ports will run the router datapath pipeline. Signed-off-by: Numan Siddique Acked-by: Han Zhou --- NEWS | 1 + ovn/controller/binding.c | 12 + ovn/controller/ovn-controller.c | 1 + ovn/lib/ovn-util.c | 1 + ovn/northd/ovn-northd.8.xml | 37 +- ovn/northd/ovn-northd.c | 143 ++++++- ovn/ovn-architecture.7.xml | 71 ++++ ovn/ovn-nb.ovsschema | 10 +- ovn/ovn-nb.xml | 56 +++ tests/ovn-northd.at | 102 ++++- tests/ovn.at | 730 +++++++++++++++++++++++++++++++- 11 files changed, 1141 insertions(+), 23 deletions(-) diff --git a/NEWS b/NEWS index 74adb2562..a2e4dd426 100644 --- a/NEWS +++ b/NEWS @@ -25,6 +25,7 @@ Post-v2.11.0 - OVN: * Select IPAM mac_prefix in a random manner if not provided by the user * Added the HA chassis group support and deprecated Gateway chassis. + * Added 'external' logical port support. v2.11.0 - 19 Feb 2019 --------------------- diff --git a/ovn/controller/binding.c b/ovn/controller/binding.c index 2929eccd7..404f0e757 100644 --- a/ovn/controller/binding.c +++ b/ovn/controller/binding.c @@ -502,6 +502,18 @@ consider_local_datapath(struct ovsdb_idl_txn *ovnsb_idl_txn, * for them. */ sset_add(local_lports, binding_rec->logical_port); our_chassis = false; + } else if (!strcmp(binding_rec->type, "external")) { + if (ha_chassis_group_contains(binding_rec->ha_chassis_group, + chassis_rec)) { + our_chassis = ha_chassis_group_is_active( + binding_rec->ha_chassis_group, + active_tunnels, chassis_rec); + + add_local_datapath(sbrec_datapath_binding_by_key, + sbrec_port_binding_by_datapath, + sbrec_port_binding_by_name, + binding_rec->datapath, false, local_datapaths); + } } if (our_chassis diff --git a/ovn/controller/ovn-controller.c b/ovn/controller/ovn-controller.c index 986a28c8a..ad626f617 100644 --- a/ovn/controller/ovn-controller.c +++ b/ovn/controller/ovn-controller.c @@ -145,6 +145,7 @@ update_sb_monitors(struct ovsdb_idl *ovnsb_idl, * ports that have a Gateway_Chassis that point's to our own * chassis */ sbrec_port_binding_add_clause_type(&pb, OVSDB_F_EQ, "chassisredirect"); + sbrec_port_binding_add_clause_type(&pb, OVSDB_F_EQ, "external"); if (chassis) { /* This should be mostly redundant with the other clauses for port * bindings, but it allows us to catch any ports that are assigned to diff --git a/ovn/lib/ovn-util.c b/ovn/lib/ovn-util.c index aa03919bb..a9d4b8736 100644 --- a/ovn/lib/ovn-util.c +++ b/ovn/lib/ovn-util.c @@ -319,6 +319,7 @@ static const char *OVN_NB_LSP_TYPES[] = { "localport", "router", "vtep", + "external", }; bool diff --git a/ovn/northd/ovn-northd.8.xml b/ovn/northd/ovn-northd.8.xml index 392a5efc9..c8883d60d 100644 --- a/ovn/northd/ovn-northd.8.xml +++ b/ovn/northd/ovn-northd.8.xml @@ -626,7 +626,8 @@ nd_na_router {

This table adds the DHCPv4 options to a DHCPv4 packet from the logical ports configured with IPv4 address(es) and DHCPv4 options, - and similarly for DHCPv6 options. + and similarly for DHCPv6 options. This table also adds flows for the + logical ports of type external.

    @@ -827,7 +828,39 @@ output;
-

Ingress Table 16 Destination Lookup

+

Ingress table 16 External ports

+ +

+ Traffic from the external logical ports enter the ingress + datapath pipeline via the localnet port. This table adds the + below logical flows to handle the traffic from these ports. +

+ +
    +
  • +

    + A priority-100 flow is added for each external logical + port which doesn't reside on a chassis to drop the ARP/IPv6 NS + request to the router IP(s) (of the logical switch) which matches + on the inport of the external logical port + and the valid eth.src address(es) of the + external logical port. +

    + +

    + This flow guarantees that the ARP/NS request to the router IP + address from the external ports is responded by only the chassis + which has claimed these external ports. All the other chassis, + drops these packets. +

    +
    • + +
  • + A priority-0 flow that matches all packets to advances to table 17. +
    • +
+ +

Ingress Table 17 Destination Lookup

This table implements switching behavior. It contains these logical diff --git a/ovn/northd/ovn-northd.c b/ovn/northd/ovn-northd.c index be8dc36a6..a82eb18ee 100644 --- a/ovn/northd/ovn-northd.c +++ b/ovn/northd/ovn-northd.c @@ -119,7 +119,8 @@ enum ovn_stage { PIPELINE_STAGE(SWITCH, IN, DHCP_RESPONSE, 13, "ls_in_dhcp_response") \ PIPELINE_STAGE(SWITCH, IN, DNS_LOOKUP, 14, "ls_in_dns_lookup") \ PIPELINE_STAGE(SWITCH, IN, DNS_RESPONSE, 15, "ls_in_dns_response") \ - PIPELINE_STAGE(SWITCH, IN, L2_LKUP, 16, "ls_in_l2_lkup") \ + PIPELINE_STAGE(SWITCH, IN, EXTERNAL_PORT, 16, "ls_in_external_port") \ + PIPELINE_STAGE(SWITCH, IN, L2_LKUP, 17, "ls_in_l2_lkup") \ \ /* Logical switch egress stages. */ \ PIPELINE_STAGE(SWITCH, OUT, PRE_LB, 0, "ls_out_pre_lb") \ @@ -2332,6 +2333,18 @@ ovn_port_update_sbrec(struct northd_context *ctx, } sbrec_port_binding_set_nat_addresses(op->sb, NULL, 0); + + if (!strcmp(op->nbsp->type, "external")) { + if (op->nbsp->ha_chassis_group) { + sync_ha_chassis_group_for_sbpb( + ctx, op->nbsp->ha_chassis_group, + sbrec_chassis_by_name, op->sb); + sset_add(active_ha_chassis_grps, + op->nbsp->ha_chassis_group->name); + } else { + sbrec_port_binding_set_ha_chassis_group(op->sb, NULL); + } + } } else { const char *chassis = NULL; if (op->peer && op->peer->od && op->peer->od->nbr) { @@ -3029,6 +3042,12 @@ lsp_is_up(const struct nbrec_logical_switch_port *lsp) return !lsp->up || *lsp->up; } +static bool +lsp_is_external(const struct nbrec_logical_switch_port *nbsp) +{ + return !strcmp(nbsp->type, "external"); +} + static bool build_dhcpv4_action(struct ovn_port *op, ovs_be32 offer_ip, struct ds *options_action, struct ds *response_action, @@ -3926,6 +3945,10 @@ build_acls(struct ovn_datapath *od, struct hmap *lflows, * logical ports of the datapath if the CMS has configured DHCPv4 options. * */ for (size_t i = 0; i < od->nbs->n_ports; i++) { + if (lsp_is_external(od->nbs->ports[i])) { + continue; + } + if (od->nbs->ports[i]->dhcpv4_options) { const char *server_id = smap_get( &od->nbs->ports[i]->dhcpv4_options->options, "server_id"); @@ -4312,6 +4335,10 @@ build_lswitch_flows(struct hmap *datapaths, struct hmap *ports, continue; } + if (lsp_is_external(op->nbsp)) { + continue; + } + ds_clear(&match); ds_clear(&actions); ds_put_format(&match, "inport == %s", op->json_key); @@ -4378,6 +4405,10 @@ build_lswitch_flows(struct hmap *datapaths, struct hmap *ports, continue; } + if (lsp_is_external(op->nbsp)) { + continue; + } + for (size_t i = 0; i < op->n_lsp_addrs; i++) { for (size_t j = 0; j < op->lsp_addrs[i].n_ipv4_addrs; j++) { ds_clear(&match); @@ -4486,6 +4517,14 @@ build_lswitch_flows(struct hmap *datapaths, struct hmap *ports, continue; } + bool is_external = lsp_is_external(op->nbsp); + if (is_external && (!op->od->localnet_port || + !op->nbsp->ha_chassis_group)) { + /* If it's an external port and there is no localnet port + * and if it doesn't belong to an HA chassis group ignore it. */ + continue; + } + for (size_t i = 0; i < op->n_lsp_addrs; i++) { for (size_t j = 0; j < op->lsp_addrs[i].n_ipv4_addrs; j++) { struct ds options_action = DS_EMPTY_INITIALIZER; @@ -4498,9 +4537,16 @@ build_lswitch_flows(struct hmap *datapaths, struct hmap *ports, ds_put_format( &match, "inport == %s && eth.src == %s && " "ip4.src == 0.0.0.0 && ip4.dst == 255.255.255.255 && " - "udp.src == 68 && udp.dst == 67", op->json_key, + "udp.src == 68 && udp.dst == 67", + is_external ? op->od->localnet_port->json_key : + op->json_key, op->lsp_addrs[i].ea_s); + if (is_external) { + ds_put_format(&match, " && is_chassis_resident(%s)", + op->json_key); + } + ovn_lflow_add(lflows, op->od, S_SWITCH_IN_DHCP_OPTIONS, 100, ds_cstr(&match), ds_cstr(&options_action)); @@ -4515,9 +4561,16 @@ build_lswitch_flows(struct hmap *datapaths, struct hmap *ports, */ ds_put_format( &match, "inport == %s && eth.src == %s && " - "%s && udp.src == 68 && udp.dst == 67", op->json_key, + "%s && udp.src == 68 && udp.dst == 67", + is_external ? op->od->localnet_port->json_key : + op->json_key, op->lsp_addrs[i].ea_s, ds_cstr(&ipv4_addr_match)); + if (is_external) { + ds_put_format(&match, " && is_chassis_resident(%s)", + op->json_key); + } + ovn_lflow_add(lflows, op->od, S_SWITCH_IN_DHCP_OPTIONS, 100, ds_cstr(&match), ds_cstr(&options_action)); @@ -4528,8 +4581,16 @@ build_lswitch_flows(struct hmap *datapaths, struct hmap *ports, ds_put_format( &match, "inport == %s && eth.src == %s && " "ip4 && udp.src == 68 && udp.dst == 67" - " && "REGBIT_DHCP_OPTS_RESULT, op->json_key, + " && "REGBIT_DHCP_OPTS_RESULT, + is_external ? op->od->localnet_port->json_key : + op->json_key, op->lsp_addrs[i].ea_s); + + if (is_external) { + ds_put_format(&match, " && is_chassis_resident(%s)", + op->json_key); + } + ovn_lflow_add(lflows, op->od, S_SWITCH_IN_DHCP_RESPONSE, 100, ds_cstr(&match), ds_cstr(&response_action)); @@ -4550,9 +4611,16 @@ build_lswitch_flows(struct hmap *datapaths, struct hmap *ports, ds_put_format( &match, "inport == %s && eth.src == %s" " && ip6.dst == ff02::1:2 && udp.src == 546 &&" - " udp.dst == 547", op->json_key, + " udp.dst == 547", + is_external ? op->od->localnet_port->json_key : + op->json_key, op->lsp_addrs[i].ea_s); + if (is_external) { + ds_put_format(&match, " && is_chassis_resident(%s)", + op->json_key); + } + ovn_lflow_add(lflows, op->od, S_SWITCH_IN_DHCP_OPTIONS, 100, ds_cstr(&match), ds_cstr(&options_action)); @@ -4604,7 +4672,9 @@ build_lswitch_flows(struct hmap *datapaths, struct hmap *ports, /* Ingress table 12 and 13: DHCP options and response, by default goto * next. (priority 0). * Ingress table 14 and 15: DNS lookup and response, by default goto next. - * (priority 0).*/ + * (priority 0). + * Ingress table 16 - External port handling, by default goto next. + * (priority 0). */ HMAP_FOR_EACH (od, key_node, datapaths) { if (!od->nbs) { @@ -4615,9 +4685,60 @@ build_lswitch_flows(struct hmap *datapaths, struct hmap *ports, ovn_lflow_add(lflows, od, S_SWITCH_IN_DHCP_RESPONSE, 0, "1", "next;"); ovn_lflow_add(lflows, od, S_SWITCH_IN_DNS_LOOKUP, 0, "1", "next;"); ovn_lflow_add(lflows, od, S_SWITCH_IN_DNS_RESPONSE, 0, "1", "next;"); + ovn_lflow_add(lflows, od, S_SWITCH_IN_EXTERNAL_PORT, 0, "1", "next;"); } - /* Ingress table 16: Destination lookup, broadcast and multicast handling + HMAP_FOR_EACH (op, key_node, ports) { + if (!op->nbsp || !lsp_is_external(op->nbsp) || + !op->od->localnet_port) { + continue; + } + + /* Table 16: External port. Drop ARP request for router ips from + * external ports on chassis not binding those ports. + * This makes the router pipeline to be run only on the chassis + * binding the external ports. */ + + for (size_t i = 0; i < op->n_lsp_addrs; i++) { + for (size_t j = 0; j < op->od->n_router_ports; j++) { + struct ovn_port *rp = op->od->router_ports[j]; + for (size_t k = 0; k < rp->n_lsp_addrs; k++) { + for (size_t l = 0; l < rp->lsp_addrs[k].n_ipv4_addrs; + l++) { + ds_clear(&match); + ds_put_format( + &match, "inport == %s && eth.src == %s" + " && !is_chassis_resident(%s)" + " && arp.tpa == %s && arp.op == 1", + op->od->localnet_port->json_key, + op->lsp_addrs[i].ea_s, op->json_key, + rp->lsp_addrs[k].ipv4_addrs[l].addr_s); + ovn_lflow_add(lflows, op->od, + S_SWITCH_IN_EXTERNAL_PORT, 100, + ds_cstr(&match), "drop;"); + } + for (size_t l = 0; l < rp->lsp_addrs[k].n_ipv6_addrs; + l++) { + ds_clear(&match); + ds_put_format( + &match, "inport == %s && eth.src == %s" + " && !is_chassis_resident(%s)" + " && nd_ns && ip6.dst == {%s, %s} && " + "nd.target == %s", + op->od->localnet_port->json_key, + op->lsp_addrs[i].ea_s, op->json_key, + rp->lsp_addrs[k].ipv6_addrs[l].addr_s, + rp->lsp_addrs[k].ipv6_addrs[l].sn_addr_s, + rp->lsp_addrs[k].ipv6_addrs[l].addr_s); + ovn_lflow_add(lflows, op->od, + S_SWITCH_IN_EXTERNAL_PORT, 100, + ds_cstr(&match), "drop;"); + } + } + } + } + } + /* Ingress table 17: Destination lookup, broadcast and multicast handling * (priority 100). */ HMAP_FOR_EACH (op, key_node, ports) { if (!op->nbsp) { @@ -4637,9 +4758,9 @@ build_lswitch_flows(struct hmap *datapaths, struct hmap *ports, "outport = \""MC_FLOOD"\"; output;"); } - /* Ingress table 16: Destination lookup, unicast handling (priority 50), */ + /* Ingress table 17: Destination lookup, unicast handling (priority 50), */ HMAP_FOR_EACH (op, key_node, ports) { - if (!op->nbsp) { + if (!op->nbsp || lsp_is_external(op->nbsp)) { continue; } @@ -4756,7 +4877,7 @@ build_lswitch_flows(struct hmap *datapaths, struct hmap *ports, } } - /* Ingress table 16: Destination lookup for unknown MACs (priority 0). */ + /* Ingress table 17: Destination lookup for unknown MACs (priority 0). */ HMAP_FOR_EACH (od, key_node, datapaths) { if (!od->nbs) { continue; @@ -4791,7 +4912,7 @@ build_lswitch_flows(struct hmap *datapaths, struct hmap *ports, * Priority 150 rules drop packets to disabled logical ports, so that they * don't even receive multicast or broadcast packets. */ HMAP_FOR_EACH (op, key_node, ports) { - if (!op->nbsp) { + if (!op->nbsp || lsp_is_external(op->nbsp)) { continue; } diff --git a/ovn/ovn-architecture.7.xml b/ovn/ovn-architecture.7.xml index 3936e6016..8c9e10635 100644 --- a/ovn/ovn-architecture.7.xml +++ b/ovn/ovn-architecture.7.xml @@ -1678,6 +1678,77 @@ +

Native OVN services for external logical ports

+ +

+ To support OVN native services (like DHCP/IPv6 RA/DNS lookup) to the + cloud resources which are external, OVN supports external + logical ports. +

+ +

+ Below are some of the use cases where external ports can be + used. +

+ +
    +
  • + VMs connected to SR-IOV nics - Traffic from these VMs by passes the + kernel stack and local ovn-controller do not bind these + ports and cannot serve the native services. +
    • +
  • + When CMS supports provisioning baremetal servers. +
    • +
+ +

+ OVN will provide the native services if CMS has done the below + configuration in the OVN Northbound Database. +

+ +
    +
  • + A row is created in Logical_Switch_Port, configuring the + column + and setting the to external. +
    • + +
  • + column is configured. +
    • + +
  • + The HA chassis which belongs to the HA chassis group has the + ovn-bridge-mappings configured and has proper L2 + connectivity so that it can receive the DHCP and other related request + packets from these external resources. +
    • + +
  • + The Logical_Switch of this port has a localnet port. +
    • + +
  • + Native OVN services are enabled by configuring the DHCP and other + options like the way it is done for the normal logical ports. +
    • +
+ +

+ It is recommended to use the same HA chassis group for all the external + ports of a logical switch. Otherwise, the physical switch might see MAC + flap issue when different chassis provide the native services. For + example when supporting native DHCPv4 service, DHCPv4 server mac + (configured in column in table ) originating + from different ports can cause MAC flap issue. + The MAC of the logical router IP(s) can also flap if the same HA chassis + group is not set for all the external ports of a logical switch. +

+

Security

Role-Based Access Controls for the Soutbound DB

diff --git a/ovn/ovn-nb.ovsschema b/ovn/ovn-nb.ovsschema index 48d27b960..884130b7a 100644 --- a/ovn/ovn-nb.ovsschema +++ b/ovn/ovn-nb.ovsschema @@ -1,7 +1,7 @@ { "name": "OVN_Northbound", - "version": "5.15.0", - "cksum": "1078795414 21917", + "version": "5.15.1", + "cksum": "2715809578 22214", "tables": { "NB_Global": { "columns": { @@ -102,6 +102,12 @@ "refType": "weak"}, "min": 0, "max": 1}}, + "ha_chassis_group": { + "type": {"key": {"type": "uuid", + "refTable": "HA_Chassis_Group", + "refType": "weak"}, + "min": 0, + "max": 1}}, "external_ids": { "type": {"key": "string", "value": "string", "min": 0, "max": "unlimited"}}}, diff --git a/ovn/ovn-nb.xml b/ovn/ovn-nb.xml index 75d27a0eb..68aea46ab 100644 --- a/ovn/ovn-nb.xml +++ b/ovn/ovn-nb.xml @@ -353,6 +353,53 @@
A port to a logical switch on a VTEP gateway.
+ +
external
+
+

+ Represents a logical port which is external and not having + an OVS port in the integration bridge. + OVN will never receive any traffic from this port or + send any traffic to this port. OVN can support + native services like DHCPv4/DHCPv6/DNS for this port. + If is defined, + ovn-controller running in the master chassis of + the HA chassis group will bind this port to provide these native + services. It is expected that this port belong to a bridged + logical switch (with a localnet port). +

+ +

+ It is recommended to use the same HA chassis group for all the + external ports of a logical switch. Otherwise, the physical + switch might see MAC flap issue when different chassis provide + the native services. For example when supporting native DHCPv4 + service, DHCPv4 server mac (configured in + column in table ) + originating from different ports can cause MAC flap issue. + The MAC of the logical router IP(s) can also flap if the + same HA chassis group is not set for all the external ports + of a logical switch. +

+ +

+ Below are some of the use cases where external + ports can be used. +

+ +
    +
  • + VMs connected to SR-IOV nics - Traffic from these VMs by passes + the kernel stack and local ovn-controller do not + bind these ports and cannot serve the native services. +
    • + +
  • + When CMS supports provisioning baremetal servers. +
    • +
+
@@ -901,6 +948,15 @@ + + References a row in the OVN Northbound database's + table. + It indicates the HA chassis group to use if the + is set to external. + If is not to external, this + column is ignored. + +

diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at index efaec250f..993c5afea 100644 --- a/tests/ovn-northd.at +++ b/tests/ovn-northd.at @@ -397,7 +397,7 @@ ovn-nbctl --wait=sb ha-chassis-group-add hagrp1 # ovn-northd should not create HA chassis group and HA chassis rows # unless the HA chassis group in OVN NB DB is associated to -# a logical router port. +# a logical router port or logical port of type external. AT_CHECK([ovn-sbctl --bare --columns name find ha_chassis_group name="hagrp1" \ | wc -l], [0], [0 ]) @@ -731,4 +731,104 @@ ovn-nbctl clear logical_router_port lr0-public options OVS_WAIT_UNTIL([test 0 = `ovn-sbctl list ha_chassis_group | wc -l`]) AT_CHECK([test 0 = `ovn-sbctl list ha_chassis | wc -l`]) +# Delete old sw0. +ovn-nbctl ls-del sw0 + +# Create external logical ports and associate ha_chassis_group +ovn-nbctl ls-add sw0 +ovn-nbctl lsp-add sw0 sw0-pext1 +ovn-nbctl lsp-add sw0 sw0-pext2 +ovn-nbctl lsp-add sw0 sw0-p1 + +ovn-nbctl lsp-set-addresses sw0-pext1 "00:00:00:00:00:03 10.0.0.3" +ovn-nbctl lsp-set-addresses sw0-pext2 "00:00:00:00:00:03 10.0.0.4" +ovn-nbctl lsp-set-addresses sw0-p1 "00:00:00:00:00:03 10.0.0.5" + +ovn-nbctl --wait=sb ha-chassis-group-add hagrp1 + +ovn-nbctl --wait=sb ha-chassis-group-add-chassis hagrp1 ch1 30 +ovn-nbctl --wait=sb ha-chassis-group-add-chassis hagrp1 ch2 20 +ovn-nbctl --wait=sb ha-chassis-group-add-chassis hagrp1 ch3 10 + +# ovn-northd should not create HA chassis group and HA chassis rows +# unless the HA chassis group in OVN NB DB is associated to +# a logical router port or logical port of type external. +OVS_WAIT_UNTIL([test 0 = `ovn-sbctl list ha_chassis_group | wc -l`]) +AT_CHECK([test 0 = `ovn-sbctl list ha_chassis | wc -l`]) + +hagrp1_uuid=`ovn-nbctl --bare --columns _uuid find ha_chassis_group \ +name=hagrp1` + +# The type of the lsp - sw0-pext1 is still not set to external. +# So ha_chassis_group should be ignored. +ovn-nbctl set logical_switch_port sw0-pext1 ha_chassis_group=$hagrp1_uuid + +OVS_WAIT_UNTIL([test 0 = `ovn-sbctl --bare --columns name find \ +ha_chassis_group name="hagrp1" | wc -l`]) + +AT_CHECK([test 0 = `ovn-sbctl list ha_chassis | grep chassis | wc -l`]) + +# Set the type of sw0-pext1 to external +ovn-nbctl lsp-set-type sw0-pext1 external + +OVS_WAIT_UNTIL([test 1 = `ovn-sbctl --bare --columns name find \ +ha_chassis_group name="hagrp1" | wc -l`]) + +AT_CHECK([test 3 = `ovn-sbctl list ha_chassis | grep chassis | wc -l`]) + +sb_hagrp1_uuid=`ovn-sbctl --bare --columns _uuid find ha_chassis_group \ +name=hagrp1` + +AT_CHECK([test "$sb_hagrp1_uuid" = `ovn-sbctl --bare --columns \ +ha_chassis_group find port_binding logical_port=sw0-pext1`]) + +# Set the type of sw0-pext2 to external and associate ha_chassis_group +ovn-nbctl lsp-set-type sw0-pext2 external +ovn-nbctl set logical_switch_port sw0-pext2 ha_chassis_group=$hagrp1_uuid + +OVS_WAIT_UNTIL([test 1 = `ovn-sbctl --bare --columns name find \ +ha_chassis_group name="hagrp1" | wc -l`]) + +AT_CHECK([test 3 = `ovn-sbctl list ha_chassis | grep chassis | wc -l`]) +AT_CHECK([test "$sb_hagrp1_uuid" = `ovn-sbctl --bare --columns \ +ha_chassis_group find port_binding logical_port=sw0-pext1`]) + +OVS_WAIT_UNTIL([test "$sb_hagrp1_uuid" = `ovn-sbctl --bare --columns \ +ha_chassis_group find port_binding logical_port=sw0-pext2`]) + +# sw0-p1 is a normal port. So ha_chassis_group should not be set +# in port_binding. +ovn-nbctl --wait=sb set logical_switch_port sw0-p1 \ +ha_chassis_group=$hagrp1_uuid + +OVS_WAIT_UNTIL([test x$(ovn-sbctl --bare --columns chassis find port_binding \ +logical_port=sw0-p1) = x], [0], []) + +# Clear ha_chassis_group for sw0-pext1 +ovn-nbctl --wait=sb clear logical_switch_port sw0-pext1 ha_chassis_group + +OVS_WAIT_UNTIL([test x$(ovn-sbctl --bare --columns chassis find port_binding \ +logical_port=sw0-pext1) = x], [0], []) + +OVS_WAIT_UNTIL([test 1 = `ovn-sbctl --bare --columns name find \ +ha_chassis_group name="hagrp1" | wc -l`]) + +AT_CHECK([test 3 = `ovn-sbctl list ha_chassis | grep chassis | wc -l`]) + +# Clear ha_chassis_group for sw0-pext2 +ovn-nbctl --wait=sb clear logical_switch_port sw0-pext2 ha_chassis_group + +OVS_WAIT_UNTIL([test x$(ovn-sbctl --bare --columns chassis find port_binding \ +logical_port=sw0-pext2) = x], [0], []) + +OVS_WAIT_UNTIL([test 0 = `ovn-sbctl list ha_chassis_group | wc -l`]) +AT_CHECK([test 0 = `ovn-sbctl list ha_chassis | wc -l`]) + +as ovn-sb +OVS_APP_EXIT_AND_WAIT([ovsdb-server]) +as ovn-nb +OVS_APP_EXIT_AND_WAIT([ovsdb-server]) +as northd +OVS_APP_EXIT_AND_WAIT([ovn-northd]) + AT_CLEANUP diff --git a/tests/ovn.at b/tests/ovn.at index d6a068e0a..89fde43f8 100644 --- a/tests/ovn.at +++ b/tests/ovn.at @@ -9857,11 +9857,10 @@ grep active_backup | grep slaves:$hv2_gw1_ofport,$hv2_gw2_ofport \ ]) # make sure that flows for handling the outside router port reside on gw1 -OVS_WAIT_UNTIL([as gw1 ovs-ofctl dump-flows br-int table=24 | \ +OVS_WAIT_UNTIL([as gw1 ovs-ofctl dump-flows br-int table=25 | \ grep 00:00:02:01:02:04 | wc -l], [0], [[1 ]]) - -OVS_WAIT_UNTIL([as gw2 ovs-ofctl dump-flows br-int table=24 | \ +OVS_WAIT_UNTIL([as gw2 ovs-ofctl dump-flows br-int table=25 | \ grep 00:00:02:01:02:04 | wc -l], [0], [[0 ]]) @@ -9972,10 +9971,10 @@ AT_CHECK([ovs-vsctl --bare --columns bfd find Interface name=ovn-hv1-0],[0], ]]) # make sure that flows for handling the outside router port reside on gw2 now -OVS_WAIT_UNTIL([as gw2 ovs-ofctl dump-flows br-int table=24 | \ +OVS_WAIT_UNTIL([as gw2 ovs-ofctl dump-flows br-int table=25 | \ grep 00:00:02:01:02:04 | wc -l], [0], [[1 ]]) -OVS_WAIT_UNTIL([as gw1 ovs-ofctl dump-flows br-int table=24 | \ +OVS_WAIT_UNTIL([as gw1 ovs-ofctl dump-flows br-int table=25 | \ grep 00:00:02:01:02:04 | wc -l], [0], [[0 ]]) @@ -9987,10 +9986,10 @@ as main ovs-vsctl del-port n1 $port bfd_dump # make sure that flows for handling the outside router port reside on gw2 now -OVS_WAIT_UNTIL([as gw1 ovs-ofctl dump-flows br-int table=24 | \ +OVS_WAIT_UNTIL([as gw1 ovs-ofctl dump-flows br-int table=25 | \ grep 00:00:02:01:02:04 | wc -l], [0], [[1 ]]) -OVS_WAIT_UNTIL([as gw2 ovs-ofctl dump-flows br-int table=24 | \ +OVS_WAIT_UNTIL([as gw2 ovs-ofctl dump-flows br-int table=25 | \ grep 00:00:02:01:02:04 | wc -l], [0], [[0 ]]) @@ -12111,6 +12110,723 @@ as hv2 start_daemon ovn-controller OVN_CLEANUP([hv1],[hv2]) AT_CLEANUP +AT_SETUP([ovn -- external logical port]) +AT_SKIP_IF([test $HAVE_PYTHON = no]) +ovn_start + +net_add n1 +sim_add hv1 +sim_add hv2 +sim_add hv3 + +ovn-nbctl ls-add ls1 +ovn-nbctl lsp-add ls1 ls1-lp1 \ +-- lsp-set-addresses ls1-lp1 "f0:00:00:00:00:01 10.0.0.4 ae70::4" + +# Add a couple of external logical port +ovn-nbctl lsp-add ls1 ls1-lp_ext1 \ +-- lsp-set-addresses ls1-lp_ext1 "f0:00:00:00:00:03 10.0.0.6 ae70::6" +ovn-nbctl lsp-set-port-security ls1-lp_ext1 \ +"f0:00:00:00:00:03 10.0.0.6 ae70::6" +ovn-nbctl lsp-set-type ls1-lp_ext1 external + +ovn-nbctl lsp-add ls1 ls1-lp_ext2 \ +-- lsp-set-addresses ls1-lp_ext2 "f0:00:00:00:00:04 10.0.0.7 ae70::7" +ovn-nbctl lsp-set-port-security ls1-lp_ext2 \ +"f0:00:00:00:00:04 10.0.0.7 ae70::8" +ovn-nbctl lsp-set-type ls1-lp_ext2 external + +d1="$(ovn-nbctl create DHCP_Options cidr=10.0.0.0/24 \ +options="\"server_id\"=\"10.0.0.1\" \"server_mac\"=\"ff:10:00:00:00:01\" \ +\"lease_time\"=\"3600\" \"router\"=\"10.0.0.1\"")" + +d2="$(ovn-nbctl create DHCP_Options cidr="ae70\:\:/64" \ +options="\"server_id\"=\"00:00:00:10:00:01\"")" + +ovn-nbctl lsp-set-dhcpv4-options ls1-lp1 ${d1} +ovn-nbctl lsp-set-dhcpv4-options ls1-lp_ext1 ${d1} +ovn-nbctl lsp-set-dhcpv4-options ls1-lp_ext2 ${d1} + +ovn-nbctl lsp-set-dhcpv6-options ls1-lp1 ${d2} +ovn-nbctl lsp-set-dhcpv6-options ls1-lp_ext1 ${d2} +ovn-nbctl lsp-set-dhcpv6-options ls1-lp_ext2 ${d2} + +# Create a logical router and connect it to ls1 +ovn-nbctl lr-add lr0 +ovn-nbctl lrp-add lr0 lr0-ls1 a0:10:00:00:00:01 10.0.0.1/24 +ovn-nbctl lsp-add ls1 ls1-lr0 +ovn-nbctl set Logical_Switch_Port ls1-lr0 type=router \ + options:router-port=lr0-ls1 addresses=router + +# Create HA chassis group +ovn-nbctl ha-chassis-group-add hagrp1 +ovn-nbctl ha-chassis-group-add-chassis hagrp1 hv1 30 + +hagrp1_uuid=`ovn-nbctl --bare --columns _uuid find ha_chassis_group name="hagrp1"` + +# There should be 1 HA_Chassis rows with chassis sets +OVS_WAIT_UNTIL([ovn-sbctl list ha_chassis | grep chassis | awk '{print $3}' \ +| grep '-' | wc -l ], [0], [1 +]) + +as hv1 +ovs-vsctl add-br br-phys +ovn_attach n1 br-phys 192.168.0.1 +ovs-vsctl -- add-port br-phys hv1-ext1 -- \ + set interface hv1-ext1 options:tx_pcap=hv1/ext1-tx.pcap \ + options:rxq_pcap=hv1/ext1-rx.pcap \ + ofport-request=2 +ovs-vsctl set open . external-ids:ovn-bridge-mappings=phys:br-phys + +as hv2 +ovs-vsctl add-br br-phys +ovn_attach n1 br-phys 192.168.0.2 +ovs-vsctl -- add-port br-phys hv2-ext2 -- \ + set interface hv2-ext2 options:tx_pcap=hv2/ext2-tx.pcap \ + options:rxq_pcap=hv2/ext2-rx.pcap \ + ofport-request=2 +ovs-vsctl set open . external-ids:ovn-bridge-mappings=phys:br-phys + +as hv3 +ovs-vsctl add-br br-phys +ovn_attach n1 br-phys 192.168.0.3 +ovs-vsctl -- add-port br-phys hv3-ext3 -- \ + set interface hv3-ext3 options:tx_pcap=hv3/ext3-tx.pcap \ + options:rxq_pcap=hv3/ext3-rx.pcap \ + ofport-request=2 +ovs-vsctl set open . external-ids:ovn-bridge-mappings=phys:br-phys + +# No DHCPv4/v6 flows for the external port - ls1-lp_ext1 - 10.0.0.6 in hv1 and +# hv2 as ha-chassis-group is not set and no localnet port added to ls1. +AT_CHECK([ovn-sbctl dump-flows ls1 | grep "offerip = 10.0.0.6" | \ +wc -l], [0], [0 +]) +AT_CHECK([as hv1 ovs-ofctl dump-flows br-int | grep table=20 | \ +grep controller | grep "0a.00.00.06" | wc -l], [0], [0 +]) +AT_CHECK([as hv2 ovs-ofctl dump-flows br-int | grep table=20 | \ +grep controller | grep "0a.00.00.06" | wc -l], [0], [0 +]) +AT_CHECK([as hv1 ovs-ofctl dump-flows br-int | grep table=20 | \ +grep controller | grep tp_src=546 | grep \ +"ae.70.00.00.00.00.00.00.00.00.00.00.00.00.00.06" | wc -l], [0], [0 +]) +AT_CHECK([as hv2 ovs-ofctl dump-flows br-int | grep table=20 | \ +grep controller | grep tp_src=546 | grep \ +"ae.70.00.00.00.00.00.00.00.00.00.00.00.00.00.06" | wc -l], [0], [0 +]) + +hv1_uuid=$(ovn-sbctl list chassis hv1 | grep uuid | awk '{print $3}') +hv2_uuid=$(ovn-sbctl list chassis hv2 | grep uuid | awk '{print $3}') +hv3_uuid=$(ovn-sbctl list chassis hv3 | grep uuid | awk '{print $3}') + +# The port_binding row for ls1-lp_ext1 should have empty chassis +chassis=`ovn-sbctl --bare --columns chassis find port_binding \ +logical_port=ls1-lp_ext1` + +AT_CHECK([test x$chassis == x], [0], []) + +# Associate hagrp1 ha-chassis-group to ls1-lp_ext1 +ovn-nbctl --wait=hv set Logical_Switch_Port ls1-lp_ext1 \ +ha-chassis-group=$hagrp1_uuid + +# Get the hagrp1 uuid in SB DB. +sb_hagrp1_uuid=`ovn-sbctl --bare --columns _uuid find ha_chassis_group \ +name="hagrp1"` + +# Wait till ls1-lp_ext1 port_binding has ha_chassis_group set +OVS_WAIT_UNTIL( + [sb_pb_hagrp=`ovn-sbctl --bare --columns ha_chassis_group find \ +port_binding logical_port=ls1-lp_ext1` + test "$sb_pb_hagrp" = "$sb_hagrp1_uuid"]) + +# No DHCPv4/v6 flows for the external port - ls1-lp_ext1 - 10.0.0.6 in hv1 and hv2 +# as no localnet port added to ls1 yet. +AT_CHECK([as hv1 ovs-ofctl dump-flows br-int | grep table=20 | \ +grep controller | grep "0a.00.00.06" | wc -l], [0], [0 +]) +AT_CHECK([as hv2 ovs-ofctl dump-flows br-int | grep table=20 | \ +grep controller | grep "0a.00.00.06" | wc -l], [0], [0 +]) +AT_CHECK([as hv1 ovs-ofctl dump-flows br-int | grep table=20 | \ +grep controller | grep tp_src=546 | grep \ +"ae.70.00.00.00.00.00.00.00.00.00.00.00.00.00.06" | wc -l], [0], [0 +]) +AT_CHECK([as hv2 ovs-ofctl dump-flows br-int | grep table=20 | \ +grep controller | grep tp_src=546 | grep \ +"ae.70.00.00.00.00.00.00.00.00.00.00.00.00.00.06" | wc -l], [0], [0 +]) + +# Add the localnet port to the logical switch ls1 +ovn-nbctl lsp-add ls1 ln-public +ovn-nbctl lsp-set-addresses ln-public unknown +ovn-nbctl lsp-set-type ln-public localnet +ovn-nbctl --wait=hv lsp-set-options ln-public network_name=phys + +ln_public_key=$(ovn-sbctl list port_binding ln-public | grep tunnel_key | \ +awk '{print $3}') + +# The ls1-lp_ext1 should be bound to hv1 as only hv1 is part of the +# ha chassis group. +OVS_WAIT_UNTIL( + [chassis=`ovn-sbctl --bare --columns chassis find port_binding \ +logical_port=ls1-lp_ext1` + test "$chassis" = "$hv1_uuid"]) + +# There should be DHCPv4/v6 OF flows for the ls1-lp_ext1 port in hv1 +AT_CHECK([as hv1 ovs-ofctl dump-flows br-int | grep table=20 | \ +grep controller | grep "0a.00.00.06" | grep reg14=0x$ln_public_key | \ +wc -l], [0], [3 +]) +AT_CHECK([as hv1 ovs-ofctl dump-flows br-int | grep table=20 | \ +grep controller | grep tp_src=546 | grep \ +"ae.70.00.00.00.00.00.00.00.00.00.00.00.00.00.06" | \ +grep reg14=0x$ln_public_key | wc -l], [0], [1 +]) + +# There should be no DHCPv4/v6 flows for ls1-lp_ext1 on hv2 +AT_CHECK([as hv2 ovs-ofctl dump-flows br-int | grep table=20 | \ +grep controller | grep "0a.00.00.06" | wc -l], [0], [0 +]) +AT_CHECK([as hv2 ovs-ofctl dump-flows br-int | grep table=20 | \ +grep controller | grep tp_src=546 | grep \ +"ae.70.00.00.00.00.00.00.00.00.00.00.00.00.00.06" | wc -l], [0], [0 +]) + +# No DHCPv4/v6 flows for the external port - ls1-lp_ext2 - 10.0.0.7 in hv1 and +# hv2 as requested-chassis option is not set. +AT_CHECK([as hv1 ovs-ofctl dump-flows br-int | grep table=20 | \ +grep controller | grep "0a.00.00.07" | wc -l], [0], [0 +]) +AT_CHECK([as hv2 ovs-ofctl dump-flows br-int | grep table=20 | \ +grep controller | grep "0a.00.00.07" | wc -l], [0], [0 +]) +AT_CHECK([as hv1 ovs-ofctl dump-flows br-int | grep table=20 | \ +grep controller | grep tp_src=546 | grep \ +"ae.70.00.00.00.00.00.00.00.00.00.00.00.00.00.07" | wc -l], [0], [0 +]) +AT_CHECK([as hv2 ovs-ofctl dump-flows br-int | grep table=20 | \ +grep controller | grep tp_src=546 | grep \ +"ae.70.00.00.00.00.00.00.00.00.00.00.00.00.00.07" | wc -l], [0], [0 +]) + +as hv1 +ovs-vsctl show + +# This shell function sends a DHCP request packet +# test_dhcp INPORT SRC_MAC DHCP_TYPE OFFER_IP ... +test_dhcp() { + local inport=$1 src_mac=$2 dhcp_type=$3 offer_ip=$4 use_ip=$5 + shift; shift; shift; shift; shift; + if test $use_ip != 0; then + src_ip=$1 + dst_ip=$2 + shift; shift; + else + src_ip=`ip_to_hex 0 0 0 0` + dst_ip=`ip_to_hex 255 255 255 255` + fi + local request=ffffffffffff${src_mac}0800451001100000000080110000${src_ip}${dst_ip} + # udp header and dhcp header + request=${request}0044004300fc0000 + request=${request}010106006359aa760000000000000000000000000000000000000000${src_mac} + # client hardware padding + request=${request}00000000000000000000 + # server hostname + request=${request}0000000000000000000000000000000000000000000000000000000000000000 + request=${request}0000000000000000000000000000000000000000000000000000000000000000 + # boot file name + request=${request}0000000000000000000000000000000000000000000000000000000000000000 + request=${request}0000000000000000000000000000000000000000000000000000000000000000 + request=${request}0000000000000000000000000000000000000000000000000000000000000000 + request=${request}0000000000000000000000000000000000000000000000000000000000000000 + # dhcp magic cookie + request=${request}63825363 + # dhcp message type + request=${request}3501${dhcp_type}ff + + local srv_mac=$1 srv_ip=$2 expected_dhcp_opts=$3 + # total IP length will be the IP length of the request packet + # (which is 272 in our case) + 8 (padding bytes) + (expected_dhcp_opts / 2) + ip_len=`expr 280 + ${#expected_dhcp_opts} / 2` + udp_len=`expr $ip_len - 20` + ip_len=$(printf "%x" $ip_len) + udp_len=$(printf "%x" $udp_len) + # $ip_len var will be in 3 digits i.e 134. So adding a '0' before $ip_len + local reply=${src_mac}${srv_mac}080045100${ip_len}000000008011XXXX${srv_ip}${offer_ip} + # udp header and dhcp header. + # $udp_len var will be in 3 digits. So adding a '0' before $udp_len + reply=${reply}004300440${udp_len}0000020106006359aa760000000000000000 + # your ip address + reply=${reply}${offer_ip} + # next server ip address, relay agent ip address, client mac address + reply=${reply}0000000000000000${src_mac} + # client hardware padding + reply=${reply}00000000000000000000 + # server hostname + reply=${reply}0000000000000000000000000000000000000000000000000000000000000000 + reply=${reply}0000000000000000000000000000000000000000000000000000000000000000 + # boot file name + reply=${reply}0000000000000000000000000000000000000000000000000000000000000000 + reply=${reply}0000000000000000000000000000000000000000000000000000000000000000 + reply=${reply}0000000000000000000000000000000000000000000000000000000000000000 + reply=${reply}0000000000000000000000000000000000000000000000000000000000000000 + # dhcp magic cookie + reply=${reply}63825363 + # dhcp message type + local dhcp_reply_type=02 + if test $dhcp_type = 03; then + dhcp_reply_type=05 + fi + reply=${reply}3501${dhcp_reply_type}${expected_dhcp_opts}00000000ff00000000 + echo $reply >> ext1_v4.expected + + as hv1 ovs-appctl netdev-dummy/receive hv${inport}-ext${inport} $request +} + + +trim_zeros() { + sed 's/\(00\)\{1,\}$//' +} + +# This shell function sends a DHCPv6 request packet +# test_dhcpv6 INPORT SRC_MAC SRC_LLA DHCPv6_MSG_TYPE OFFER_IP OUTPORT... +# The OUTPORTs (zero or more) list the VIFs on which the original DHCPv6 +# packet should be received twice (one from ovn-controller and the other +# from the "ovs-ofctl monitor br-int resume" +test_dhcpv6() { + local inport=$1 src_mac=$2 src_lla=$3 msg_code=$4 offer_ip=$5 + local req_pkt_in_expected=$6 + local request=ffffffffffff${src_mac}86dd00000000002a1101${src_lla} + # dst ip ff02::1:2 + request=${request}ff020000000000000000000000010002 + # udp header and dhcpv6 header + request=${request}02220223002affff${msg_code}010203 + # Client identifier + request=${request}0001000a00030001${src_mac} + # IA-NA (Identity Association for Non Temporary Address) + request=${request}0003000c0102030400000e1000001518 + shift; shift; shift; shift; shift; + + local server_mac=000000100001 + local server_lla=fe80000000000000020000fffe100001 + local reply_code=07 + if test $msg_code = 01; then + reply_code=02 + fi + local msg_len=54 + if test $offer_ip = 1; then + msg_len=28 + fi + local reply=${src_mac}${server_mac}86dd0000000000${msg_len}1101 + reply=${reply}${server_lla}${src_lla} + + # udp header and dhcpv6 header + reply=${reply}0223022200${msg_len}ffff${reply_code}010203 + # Client identifier + reply=${reply}0001000a00030001${src_mac} + # IA-NA + if test $offer_ip != 1; then + reply=${reply}0003002801020304ffffffffffffffff00050018${offer_ip} + reply=${reply}ffffffffffffffff + fi + # Server identifier + reply=${reply}0002000a00030001${server_mac} + + echo $reply | trim_zeros >> ext${inport}_v6.expected + # The inport also receives the request packet since it is connected + # to the br-phys. + #echo $request >> ext${inport}_v6.expected + + as hv1 ovs-appctl netdev-dummy/receive hv${inport}-ext${inport} $request +} + +reset_pcap_file() { + local iface=$1 + local pcap_file=$2 + ovs-vsctl -- set Interface $iface options:tx_pcap=dummy-tx.pcap \ +options:rxq_pcap=dummy-rx.pcap + rm -f ${pcap_file}*.pcap + ovs-vsctl -- set Interface $iface options:tx_pcap=${pcap_file}-tx.pcap \ +options:rxq_pcap=${pcap_file}-rx.pcap +} + +ip_to_hex() { + printf "%02x%02x%02x%02x" "$@" +} + +AT_CAPTURE_FILE([ofctl_monitor0_hv1.log]) +as hv1 ovs-ofctl monitor br-int resume --detach --no-chdir \ +--pidfile=ovs-ofctl0.pid 2> ofctl_monitor0_hv1.log + +AT_CAPTURE_FILE([ofctl_monitor0_hv2.log]) +as hv2 ovs-ofctl monitor br-int resume --detach --no-chdir \ +--pidfile=ovs-ofctl0.pid 2> ofctl_monitor0_hv2.log + +AT_CAPTURE_FILE([ofctl_monitor0_hv3.log]) +as hv3 ovs-ofctl monitor br-int resume --detach --no-chdir \ +--pidfile=ovs-ofctl0.pid 2> ofctl_monitor0_hv3.log + +as hv1 +reset_pcap_file hv1-ext1 hv1/ext1 + +# Send DHCPDISCOVER. +offer_ip=`ip_to_hex 10 0 0 6` +server_ip=`ip_to_hex 10 0 0 1` +server_mac=ff1000000001 +expected_dhcp_opts=330400000e100104ffffff0003040a00000136040a000001 +test_dhcp 1 f00000000003 01 $offer_ip 0 $server_mac $server_ip \ +$expected_dhcp_opts + +# NXT_RESUMEs should be 1 in hv1. +OVS_WAIT_UNTIL([test 1 = `cat ofctl_monitor0_hv1.log | grep -c NXT_RESUME`]) + +# NXT_RESUMEs should be 0 in hv2. +OVS_WAIT_UNTIL([test 0 = `cat ofctl_monitor0_hv2.log | grep -c NXT_RESUME`]) + +$PYTHON "$top_srcdir/utilities/ovs-pcap.in" hv1/ext1-tx.pcap > ext1_v4.packets +cat ext1_v4.expected | cut -c -48 > expout +AT_CHECK([cat ext1_v4.packets | cut -c -48], [0], [expout]) +# Skipping the IPv4 checksum. +cat ext1_v4.expected | cut -c 53- > expout +AT_CHECK([cat ext1_v4.packets | cut -c 53-], [0], [expout]) + +# ovs-ofctl also resumes the packets and this causes other ports to receive +# the DHCP request packet. So reset the pcap files so that its easier to test. +as hv1 +reset_pcap_file hv1-ext1 hv1/ext1 + +rm -f ext1_v4.expected +rm -f ext1_v4.packets + +# Send DHCPv6 request +src_mac=f00000000003 +src_lla=fe80000000000000f20000fffe000003 +offer_ip=ae700000000000000000000000000006 +test_dhcpv6 1 $src_mac $src_lla 01 $offer_ip + +# NXT_RESUMEs should be 2 in hv1. +OVS_WAIT_UNTIL([test 2 = `cat ofctl_monitor0_hv1.log | grep -c NXT_RESUME`]) + +# NXT_RESUMEs should be 0 in hv2. +OVS_WAIT_UNTIL([test 0 = `cat ofctl_monitor0_hv2.log | grep -c NXT_RESUME`]) + +$PYTHON "$top_srcdir/utilities/ovs-pcap.in" hv1/ext1-tx.pcap | \ +sort > ext1_v6.packets +cat ext1_v6.expected | cut -c -120 > expout +AT_CHECK([cat ext1_v6.packets | cut -c -120], [0], [expout]) +# Skipping the UDP checksum +cat ext1_v6.expected | cut -c 125- > expout +AT_CHECK([cat ext1_v6.packets | cut -c 125-], [0], [expout]) + +rm -f ext1_v6.expected +rm -f ext1_v6.packets + +as hv1 +reset_pcap_file hv1-ext1 hv1/ext1 + +# Delete the ha-chassis hv1. +ovn-nbctl ha-chassis-group-remove-chassis hagrp1 hv1 +OVS_WAIT_UNTIL( + [chassis=`ovn-sbctl --bare --columns chassis find port_binding \ +logical_port=ls1-lp_ext1` + test "$chassis" = ""]) + +# Add hv2 to the ha chassis group +ovn-nbctl --wait=hv ha-chassis-group-add-chassis hagrp1 hv2 20 + +ovn-sbctl list ha_chassis_group +ovn-sbctl list ha_chassis + +ovn-sbctl find port_binding logical_port=ls1-lp_ext1 + +# The ls1-lp_ext1 should be bound to hv2 +OVS_WAIT_UNTIL( + [chassis=`ovn-sbctl --bare --columns chassis find port_binding \ +logical_port=ls1-lp_ext1` + test "$chassis" = "$hv2_uuid"]) + +# There should be OF flows for DHCP4/v6 for the ls1-lp_ext1 port in hv2 +AT_CHECK([as hv2 ovs-ofctl dump-flows br-int | grep table=20 | \ +grep controller | grep "0a.00.00.06" | grep reg14=0x$ln_public_key | \ +wc -l], [0], [3 +]) +AT_CHECK([as hv2 ovs-ofctl dump-flows br-int | grep table=20 | \ +grep controller | grep tp_src=546 | grep \ +"ae.70.00.00.00.00.00.00.00.00.00.00.00.00.00.06" | \ +grep reg14=0x$ln_public_key | wc -l], [0], [1 +]) + +# There should be no DHCPv4/v6 flows for ls1-lp_ext1 on hv1 +AT_CHECK([as hv1 ovs-ofctl dump-flows br-int | grep table=20 | \ +grep controller | grep "0a.00.00.06" | wc -l], [0], [0 +]) +AT_CHECK([as hv1 ovs-ofctl dump-flows br-int | grep table=20 | \ +grep controller | grep tp_src=546 | grep \ +"ae.70.00.00.00.00.00.00.00.00.00.00.00.00.00.06" | \ +grep reg14=0x$ln_public_key | wc -l], [0], [0 +]) + +# Send DHCPDISCOVER again for hv1/ext1. The DHCP response should come from +# hv2 ovn-controller. +offer_ip=`ip_to_hex 10 0 0 6` +server_ip=`ip_to_hex 10 0 0 1` +server_mac=ff1000000001 +expected_dhcp_opts=330400000e100104ffffff0003040a00000136040a000001 +test_dhcp 1 f00000000003 01 $offer_ip 0 $server_mac $server_ip \ +$expected_dhcp_opts + +# NXT_RESUMEs should be 2 in hv1. +OVS_WAIT_UNTIL([test 2 = `cat ofctl_monitor0_hv1.log | grep -c NXT_RESUME`]) + +# NXT_RESUMEs should be 1 in hv2. +OVS_WAIT_UNTIL([test 1 = `cat ofctl_monitor0_hv2.log | grep -c NXT_RESUME`]) + +$PYTHON "$top_srcdir/utilities/ovs-pcap.in" hv1/ext1-tx.pcap > ext1_v4.packets +cat ext1_v4.expected | cut -c -48 > expout +AT_CHECK([cat ext1_v4.packets | cut -c -48], [0], [expout]) +# Skipping the IPv4 checksum. +cat ext1_v4.expected | cut -c 53- > expout +AT_CHECK([cat ext1_v4.packets | cut -c 53-], [0], [expout]) + +# ovs-ofctl also resumes the packets and this causes other ports to receive +# the DHCP request packet. So reset the pcap files so that its easier to test. +as hv1 +reset_pcap_file hv1-ext1 hv1/ext1 + +rm -f ext1_v4.expected + +# Send DHCPv6 request again +src_mac=f00000000003 +src_lla=fe80000000000000f20000fffe000003 +offer_ip=ae700000000000000000000000000006 +test_dhcpv6 1 $src_mac $src_lla 01 $offer_ip 1 + +# NXT_RESUMEs should be 2 in hv1. +OVS_WAIT_UNTIL([test 2 = `cat ofctl_monitor0_hv1.log | grep -c NXT_RESUME`]) + +# NXT_RESUMEs should be 2 in hv2. +OVS_WAIT_UNTIL([test 2 = `cat ofctl_monitor0_hv2.log | grep -c NXT_RESUME`]) + +$PYTHON "$top_srcdir/utilities/ovs-pcap.in" hv1/ext1-tx.pcap | \ +sort > ext1_v6.packets +cat ext1_v6.expected | cut -c -120 > expout +AT_CHECK([cat ext1_v6.packets | cut -c -120], [0], [expout]) +# Skipping the UDP checksum +cat ext1_v6.expected | cut -c 125- > expout +AT_CHECK([cat ext1_v6.packets | cut -c 125-], [0], [expout]) + +rm -f ext1_v6.expected +rm -f ext1_v6.packets + +as hv1 +ovs-vsctl show +reset_pcap_file hv1-ext1 hv1/ext1 +reset_pcap_file br-phys_n1 hv1/br-phys_n1 +reset_pcap_file br-phys hv1/br-phys + +as hv2 +ovs-vsctl show +reset_pcap_file hv2-ext2 hv2/ext2 +reset_pcap_file br-phys_n1 hv2/br-phys_n1 +reset_pcap_file br-phys hv2/br-phys + +# From ls1-lp_ext1, send ARP request for the router ip. The ARP +# response should come from the router pipeline of hv2. +ext1_mac=f00000000003 +router_mac=a01000000001 +ext1_ip=`ip_to_hex 10 0 0 6` +router_ip=`ip_to_hex 10 0 0 1` +arp_request=ffffffffffff${ext1_mac}08060001080006040001${ext1_mac}${ext1_ip}000000000000${router_ip} + +as hv1 ovs-appctl netdev-dummy/receive hv1-ext1 $arp_request +expected_response=${src_mac}${router_mac}08060001080006040002${router_mac}${router_ip}${ext1_mac}${ext1_ip} +echo $expected_response > expout +$PYTHON "$top_srcdir/utilities/ovs-pcap.in" hv1/ext1-tx.pcap > ext1_arp_resp +AT_CHECK([cat ext1_arp_resp], [0], [expout]) + +# Verify that the response came from hv2 +$PYTHON "$top_srcdir/utilities/ovs-pcap.in" hv2/br-phys_n1-tx.pcap > ext1_arp_resp +AT_CHECK([cat ext1_arp_resp], [0], [expout]) + +# Now add 3 ha chassis to the ha chassis group +ovn-nbctl ha-chassis-group-add-chassis hagrp1 hv1 30 +ovn-nbctl ha-chassis-group-add-chassis hagrp1 hv2 20 +ovn-nbctl ha-chassis-group-add-chassis hagrp1 hv3 10 + +# hv1 should be master and claim ls1-lp_ext1 +OVS_WAIT_UNTIL( + [chassis=`ovn-sbctl --bare --columns chassis find port_binding \ +logical_port=ls1-lp_ext1` + test "$chassis" = "$hv1_uuid"]) + +as hv1 +ovs-vsctl show +reset_pcap_file hv1-ext1 hv1/ext1 +reset_pcap_file br-phys_n1 hv1/br-phys_n1 +reset_pcap_file br-phys hv1/br-phys + +as hv2 +ovs-vsctl show +reset_pcap_file hv2-ext2 hv2/ext2 +reset_pcap_file br-phys_n1 hv2/br-phys_n1 +reset_pcap_file br-phys hv2/br-phys + +as hv3 +ovs-vsctl show +reset_pcap_file hv3-ext3 hv3/ext3 +reset_pcap_file br-phys_n1 hv3/br-phys_n1 +reset_pcap_file br-phys hv3/br-phys + +# Send DHCPDISCOVER. +offer_ip=`ip_to_hex 10 0 0 6` +server_ip=`ip_to_hex 10 0 0 1` +server_mac=ff1000000001 +expected_dhcp_opts=330400000e100104ffffff0003040a00000136040a000001 +test_dhcp 1 f00000000003 01 $offer_ip 0 $server_mac $server_ip \ +$expected_dhcp_opts + +# NXT_RESUMEs should be 3 in hv1. +OVS_WAIT_UNTIL([test 3 = `cat ofctl_monitor0_hv1.log | grep -c NXT_RESUME`]) + +# NXT_RESUMEs should be 2 in hv2. +OVS_WAIT_UNTIL([test 2 = `cat ofctl_monitor0_hv2.log | grep -c NXT_RESUME`]) + +$PYTHON "$top_srcdir/utilities/ovs-pcap.in" hv1/ext1-tx.pcap > ext1_v4.packets +cat ext1_v4.expected | cut -c -48 > expout +AT_CHECK([cat ext1_v4.packets | cut -c -48], [0], [expout]) +# Skipping the IPv4 checksum. +cat ext1_v4.expected | cut -c 53- > expout +AT_CHECK([cat ext1_v4.packets | cut -c 53-], [0], [expout]) + +# ovs-ofctl also resumes the packets and this causes other ports to receive +# the DHCP request packet. So reset the pcap files so that its easier to test. +as hv1 +reset_pcap_file hv1-ext1 hv1/ext1 + +rm -f ext1_v4.expected +rm -f ext1_v4.packets + +# Send DHCPv6 request +src_mac=f00000000003 +src_lla=fe80000000000000f20000fffe000003 +offer_ip=ae700000000000000000000000000006 +test_dhcpv6 1 $src_mac $src_lla 01 $offer_ip + +# NXT_RESUMEs should be 4 in hv1. +OVS_WAIT_UNTIL([test 4 = `cat ofctl_monitor0_hv1.log | grep -c NXT_RESUME`]) + +# NXT_RESUMEs should be 2 in hv2. +OVS_WAIT_UNTIL([test 2 = `cat ofctl_monitor0_hv2.log | grep -c NXT_RESUME`]) + +$PYTHON "$top_srcdir/utilities/ovs-pcap.in" hv1/ext1-tx.pcap | \ +sort > ext1_v6.packets +cat ext1_v6.expected | cut -c -120 > expout +AT_CHECK([cat ext1_v6.packets | cut -c -120], [0], [expout]) +# Skipping the UDP checksum +cat ext1_v6.expected | cut -c 125- > expout +AT_CHECK([cat ext1_v6.packets | cut -c 125-], [0], [expout]) + +rm -f ext1_v6.expected +rm -f ext1_v6.packets +as hv1 reset_pcap_file hv1-ext1 hv1/ext1 + +# Now increase the priority of hv3 so it becomes master. +ovn-nbctl ha-chassis-group-add-chassis hagrp1 hv3 50 + +# hv3 should be master and claim ls1-lp_ext1 +OVS_WAIT_UNTIL( + [chassis=`ovn-sbctl --bare --columns chassis find port_binding \ +logical_port=ls1-lp_ext1` + test "$chassis" = "$hv3_uuid"]) + +as hv1 +ovs-vsctl show +reset_pcap_file hv1-ext1 hv1/ext1 +reset_pcap_file br-phys_n1 hv1/br-phys_n1 +reset_pcap_file br-phys hv1/br-phys + +as hv2 +ovs-vsctl show +reset_pcap_file hv2-ext2 hv2/ext2 +reset_pcap_file br-phys_n1 hv2/br-phys_n1 +reset_pcap_file br-phys hv2/br-phys + +as hv2 +ovs-vsctl show +reset_pcap_file hv3-ext3 hv3/ext3 +reset_pcap_file br-phys_n1 hv3/br-phys_n1 +reset_pcap_file br-phys hv3/br-phys + +# Send DHCPDISCOVER. +offer_ip=`ip_to_hex 10 0 0 6` +server_ip=`ip_to_hex 10 0 0 1` +server_mac=ff1000000001 +expected_dhcp_opts=330400000e100104ffffff0003040a00000136040a000001 +test_dhcp 1 f00000000003 01 $offer_ip 0 $server_mac $server_ip \ +$expected_dhcp_opts + +# NXT_RESUMEs should be 4 in hv1. +OVS_WAIT_UNTIL([test 4 = `cat ofctl_monitor0_hv1.log | grep -c NXT_RESUME`]) + +# NXT_RESUMEs should be 2 in hv2. +OVS_WAIT_UNTIL([test 2 = `cat ofctl_monitor0_hv2.log | grep -c NXT_RESUME`]) + +# NXT_RESUMEs should be 1 in hv3. +OVS_WAIT_UNTIL([test 1 = `cat ofctl_monitor0_hv3.log | grep -c NXT_RESUME`]) + +$PYTHON "$top_srcdir/utilities/ovs-pcap.in" hv1/ext1-tx.pcap > ext1_v4.packets +cat ext1_v4.expected | cut -c -48 > expout +AT_CHECK([cat ext1_v4.packets | cut -c -48], [0], [expout]) +# Skipping the IPv4 checksum. +cat ext1_v4.expected | cut -c 53- > expout +AT_CHECK([cat ext1_v4.packets | cut -c 53-], [0], [expout]) + +# ovs-ofctl also resumes the packets and this causes other ports to receive +# the DHCP request packet. So reset the pcap files so that its easier to test. +as hv1 +reset_pcap_file hv1-ext1 hv1/ext1 + +rm -f ext1_v4.expected +rm -f ext1_v4.packets + +# Send DHCPv6 request +src_mac=f00000000003 +src_lla=fe80000000000000f20000fffe000003 +offer_ip=ae700000000000000000000000000006 +test_dhcpv6 1 $src_mac $src_lla 01 $offer_ip + +# NXT_RESUMEs should be 4 in hv1. +OVS_WAIT_UNTIL([test 4 = `cat ofctl_monitor0_hv1.log | grep -c NXT_RESUME`]) + +# NXT_RESUMEs should be 2 in hv2. +OVS_WAIT_UNTIL([test 2 = `cat ofctl_monitor0_hv2.log | grep -c NXT_RESUME`]) + +# NXT_RESUMEs should be 2 in hv3. +OVS_WAIT_UNTIL([test 2 = `cat ofctl_monitor0_hv3.log | grep -c NXT_RESUME`]) + +$PYTHON "$top_srcdir/utilities/ovs-pcap.in" hv1/ext1-tx.pcap | \ +sort > ext1_v6.packets +cat ext1_v6.expected | cut -c -120 > expout +AT_CHECK([cat ext1_v6.packets | cut -c -120], [0], [expout]) +# Skipping the UDP checksum +cat ext1_v6.expected | cut -c 125- > expout +AT_CHECK([cat ext1_v6.packets | cut -c 125-], [0], [expout]) + +# disconnect hv3 from the network, hv1 should take over +as hv3 +port=${sandbox}_br-phys +as main ovs-vsctl del-port n1 $port + +# hv1 should be master and claim ls1-lp_ext1 +OVS_WAIT_UNTIL( + [chassis=`ovn-sbctl --bare --columns chassis find port_binding \ +logical_port=ls1-lp_ext1` + test "$chassis" = "$hv1_uuid"]) + +OVN_CLEANUP([hv1],[hv2],[hv3]) +AT_CLEANUP + AT_SETUP([ovn -- ovn-controller restart]) AT_SKIP_IF([test $HAVE_PYTHON = no]) ovn_start