From patchwork Tue Jan 28 02:55:33 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Han Zhou X-Patchwork-Id: 1230138 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.138; helo=whitealder.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=ovn.org Received: from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 486B8X35pQz9sR1 for ; Tue, 28 Jan 2020 13:56:27 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by whitealder.osuosl.org (Postfix) with ESMTP id D415986917; Tue, 28 Jan 2020 02:56:25 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from whitealder.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Us6lRGpZcilp; Tue, 28 Jan 2020 02:56:20 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by whitealder.osuosl.org (Postfix) with ESMTP id 3FA80867A1; Tue, 28 Jan 2020 02:56:12 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 203D8C1D8A; Tue, 28 Jan 2020 02:56:12 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by lists.linuxfoundation.org (Postfix) with ESMTP id EF870C0171 for ; Tue, 28 Jan 2020 02:56:10 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id DE4E887CDD for ; Tue, 28 Jan 2020 02:56:10 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id irt+nrXcAQj7 for ; Tue, 28 Jan 2020 02:56:06 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from relay8-d.mail.gandi.net (relay8-d.mail.gandi.net [217.70.183.201]) by hemlock.osuosl.org (Postfix) with ESMTPS id 10CEA87BC0 for ; Tue, 28 Jan 2020 02:56:05 +0000 (UTC) X-Originating-IP: 216.113.160.77 Received: from localhost.localdomain.localdomain (unknown [216.113.160.77]) (Authenticated sender: hzhou@ovn.org) by relay8-d.mail.gandi.net (Postfix) with ESMTPSA id D5BB61BF203; Tue, 28 Jan 2020 02:56:03 +0000 (UTC) From: Han Zhou To: dev@openvswitch.org Date: Mon, 27 Jan 2020 18:55:33 -0800 Message-Id: <1580180138-82118-9-git-send-email-hzhou@ovn.org> X-Mailer: git-send-email 2.1.0 In-Reply-To: <1580180138-82118-1-git-send-email-hzhou@ovn.org> References: <1580180138-82118-1-git-send-email-hzhou@ovn.org> Cc: Han Zhou Subject: [ovs-dev] [PATCH ovn v3 08/13] ovn-ic: Interconnection gateway controller. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" Sync local and remote gateways between SB and IC-SB. Signed-off-by: Han Zhou --- ic/ovn-ic.c | 147 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++ tests/ovn-ic.at | 55 +++++++++++++++++++++ 2 files changed, 202 insertions(+) diff --git a/ic/ovn-ic.c b/ic/ovn-ic.c index 82bd86e..f352036 100644 --- a/ic/ovn-ic.c +++ b/ic/ovn-ic.c @@ -248,6 +248,152 @@ ts_run(struct ic_context *ctx) shash_destroy(&isb_dps); } +/* Returns true if any information in gw and chassis is different. */ +static bool +is_gateway_data_changed(const struct icsbrec_gateway *gw, + const struct sbrec_chassis *chassis) +{ + if (strcmp(gw->hostname, chassis->hostname)) { + return true; + } + + if (gw->n_encaps != chassis->n_encaps) { + return true; + } + + for (int g = 0; g < gw->n_encaps; g++) { + + bool found = false; + const struct icsbrec_encap *gw_encap = gw->encaps[g]; + for (int s = 0; s < chassis->n_encaps; s++) { + const struct sbrec_encap *chassis_encap = chassis->encaps[s]; + if (!strcmp(gw_encap->type, chassis_encap->type) && + !strcmp(gw_encap->ip, chassis_encap->ip)) { + found = true; + if (!smap_equal(&gw_encap->options, &chassis_encap->options)) { + return true; + } + break; + } + } + if (!found) { + return true; + } + } + + return false; +} + +static void +sync_isb_gw_to_sb(struct ic_context *ctx, + const struct icsbrec_gateway *gw, + const struct sbrec_chassis *chassis) +{ + sbrec_chassis_set_hostname(chassis, gw->hostname); + sbrec_chassis_update_external_ids_setkey(chassis, "is-remote", "true"); + + /* Sync encaps used by this gateway. */ + ovs_assert(gw->n_encaps); + struct sbrec_encap *sb_encap; + struct sbrec_encap **sb_encaps = + xmalloc(gw->n_encaps * sizeof *sb_encaps); + for (int i = 0; i < gw->n_encaps; i++) { + sb_encap = sbrec_encap_insert(ctx->ovnsb_txn); + sbrec_encap_set_chassis_name(sb_encap, gw->name); + sbrec_encap_set_ip(sb_encap, gw->encaps[i]->ip); + sbrec_encap_set_type(sb_encap, gw->encaps[i]->type); + sbrec_encap_set_options(sb_encap, &gw->encaps[i]->options); + sb_encaps[i] = sb_encap; + } + sbrec_chassis_set_encaps(chassis, sb_encaps, gw->n_encaps); + free(sb_encaps); +} + +static void +sync_sb_gw_to_isb(struct ic_context *ctx, + const struct sbrec_chassis *chassis, + const struct icsbrec_gateway *gw) +{ + icsbrec_gateway_set_hostname(gw, chassis->hostname); + + /* Sync encaps used by this chassis. */ + ovs_assert(chassis->n_encaps); + struct icsbrec_encap *isb_encap; + struct icsbrec_encap **isb_encaps = + xmalloc(chassis->n_encaps * sizeof *isb_encaps); + for (int i = 0; i < chassis->n_encaps; i++) { + isb_encap = icsbrec_encap_insert(ctx->ovnisb_txn); + icsbrec_encap_set_gateway_name(isb_encap, + chassis->name); + icsbrec_encap_set_ip(isb_encap, chassis->encaps[i]->ip); + icsbrec_encap_set_type(isb_encap, + chassis->encaps[i]->type); + icsbrec_encap_set_options(isb_encap, + &chassis->encaps[i]->options); + isb_encaps[i] = isb_encap; + } + icsbrec_gateway_set_encaps(gw, isb_encaps, + chassis->n_encaps); + free(isb_encaps); +} + +static void +gateway_run(struct ic_context *ctx, const struct icsbrec_availability_zone *az) +{ + if (!ctx->ovnisb_txn || !ctx->ovnsb_txn) { + return; + } + + struct shash local_gws = SHASH_INITIALIZER(&local_gws); + struct shash remote_gws = SHASH_INITIALIZER(&remote_gws); + const struct icsbrec_gateway *gw; + ICSBREC_GATEWAY_FOR_EACH (gw, ctx->ovnisb_idl) { + if (gw->availability_zone == az) { + shash_add(&local_gws, gw->name, gw); + } else { + shash_add(&remote_gws, gw->name, gw); + } + } + + const struct sbrec_chassis *chassis; + SBREC_CHASSIS_FOR_EACH (chassis, ctx->ovnsb_idl) { + if (smap_get_bool(&chassis->external_ids, "is-interconn", false)) { + gw = shash_find_and_delete(&local_gws, chassis->name); + if (!gw) { + gw = icsbrec_gateway_insert(ctx->ovnisb_txn); + icsbrec_gateway_set_availability_zone(gw, az); + icsbrec_gateway_set_name(gw, chassis->name); + sync_sb_gw_to_isb(ctx, chassis, gw); + } else if (is_gateway_data_changed(gw, chassis)) { + sync_sb_gw_to_isb(ctx, chassis, gw); + } + } else if (smap_get_bool(&chassis->external_ids, "is-remote", false)) { + gw = shash_find_and_delete(&remote_gws, chassis->name); + if (!gw) { + sbrec_chassis_delete(chassis); + } else if (is_gateway_data_changed(gw, chassis)) { + sync_isb_gw_to_sb(ctx, gw, chassis); + } + } + } + + /* Delete extra gateways from ISB for the local AZ */ + struct shash_node *node; + SHASH_FOR_EACH (node, &local_gws) { + icsbrec_gateway_delete(node->data); + } + shash_destroy(&local_gws); + + /* Create SB chassis for remote gateways in ISB */ + SHASH_FOR_EACH (node, &remote_gws) { + gw = node->data; + chassis = sbrec_chassis_insert(ctx->ovnsb_txn); + sbrec_chassis_set_name(chassis, gw->name); + sync_isb_gw_to_sb(ctx, gw, chassis); + } + shash_destroy(&remote_gws); +} + static void ovn_db_run(struct ic_context *ctx) { @@ -259,6 +405,7 @@ ovn_db_run(struct ic_context *ctx) } ts_run(ctx); + gateway_run(ctx, az); } static void diff --git a/tests/ovn-ic.at b/tests/ovn-ic.at index 3d77a7d..62cf88f 100644 --- a/tests/ovn-ic.at +++ b/tests/ovn-ic.at @@ -66,3 +66,58 @@ ts2 OVN_CLEANUP_IC([az1]) AT_CLEANUP + +AT_SETUP([ovn-ic -- gateway sync]) + +ovn_init_ic_db +net_add n1 +ovn_start az1 +ovn_start az2 +sim_add gw1 +as gw1 +ovs-vsctl add-br br-phys +ovn_az_attach az1 n1 br-phys 192.168.0.1 +ovs-vsctl set open . external-ids:ovn-is-interconn=true external-ids:hostname=gw1 + +AT_CHECK([ovn_as az2 ovn-sbctl show | sort -r], [0], [dnl +Chassis gw1 + hostname: gw1 + Encap vxlan + Encap geneve + options: {csum="true"} + options: {csum="true"} + ip: "192.168.0.1" + ip: "192.168.0.1" +]) + +AT_CHECK([ovn_as az2 ovn-sbctl -f csv -d bare --no-headings --columns external_ids list chassis], [0], [dnl +is-remote=true +]) + +ovs-vsctl set open . external-ids:ovn-is-interconn=false +AT_CHECK([ovn_as az2 ovn-sbctl show], [0], [dnl +]) + +ovs-vsctl set open . external-ids:ovn-is-interconn=true +AT_CHECK([ovn_as az2 ovn-sbctl show | grep gw1], [0], [ignore]) + +OVN_CLEANUP_SBOX(gw1) +AT_CHECK([ovn_as az2 ovn-sbctl show], [0], [dnl +]) + +# Test encap change +sim_add gw2 +as gw2 +ovs-vsctl add-br br-phys +ovn_az_attach az1 n1 br-phys 192.168.0.1 +ovs-vsctl set open . external-ids:ovn-is-interconn=true +OVS_WAIT_UNTIL([ovn_as az2 ovn-sbctl show | grep "192.168.0.1"]) +ovs-vsctl set open . external_ids:ovn-encap-ip=192.168.0.2 +OVS_WAIT_UNTIL([ovn_as az2 ovn-sbctl show | grep "192.168.0.2"]) +ovs-vsctl set open . external_ids:ovn-encap-type="geneve,stt" +OVS_WAIT_UNTIL([ovn_as az2 ovn-sbctl show | grep stt]) + +OVN_CLEANUP_SBOX(gw2) +OVN_CLEANUP_IC([az1], [az2]) + +AT_CLEANUP