From patchwork Wed Jan 29 19:55:56 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Han Zhou X-Patchwork-Id: 1231076 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.137; helo=fraxinus.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=ovn.org Received: from fraxinus.osuosl.org (smtp4.osuosl.org [140.211.166.137]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 487Dl76cHsz9sPJ for ; Thu, 30 Jan 2020 06:56:35 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by fraxinus.osuosl.org (Postfix) with ESMTP id BC1338600D; Wed, 29 Jan 2020 19:56:33 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from fraxinus.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id qR2bqE4cOQ5D; Wed, 29 Jan 2020 19:56:31 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by fraxinus.osuosl.org (Postfix) with ESMTP id 4BE5285FE4; Wed, 29 Jan 2020 19:56:31 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 33BEDC1D85; Wed, 29 Jan 2020 19:56:31 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by lists.linuxfoundation.org (Postfix) with ESMTP id 983D3C0881 for ; Wed, 29 Jan 2020 19:56:28 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id 8769086113 for ; Wed, 29 Jan 2020 19:56:28 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id xti2jMMIcnBT for ; Wed, 29 Jan 2020 19:56:27 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from relay1-d.mail.gandi.net (relay1-d.mail.gandi.net [217.70.183.193]) by hemlock.osuosl.org (Postfix) with ESMTPS id C5ADF8820C for ; Wed, 29 Jan 2020 19:56:26 +0000 (UTC) X-Originating-IP: 216.113.160.77 Received: from localhost.localdomain.localdomain (unknown [216.113.160.77]) (Authenticated sender: hzhou@ovn.org) by relay1-d.mail.gandi.net (Postfix) with ESMTPSA id 6F40C240004; Wed, 29 Jan 2020 19:56:24 +0000 (UTC) From: Han Zhou To: dev@openvswitch.org Date: Wed, 29 Jan 2020 11:55:56 -0800 Message-Id: <1580327768-36501-2-git-send-email-hzhou@ovn.org> X-Mailer: git-send-email 2.1.0 In-Reply-To: <1580327768-36501-1-git-send-email-hzhou@ovn.org> References: <1580327768-36501-1-git-send-email-hzhou@ovn.org> Cc: Han Zhou Subject: [ovs-dev] [PATCH ovn v4 01/13] ovn-architecture: Add documentation for OVN interconnection feature. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" Signed-off-by: Han Zhou --- ovn-architecture.7.xml | 144 ++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 143 insertions(+), 1 deletion(-) diff --git a/ovn-architecture.7.xml b/ovn-architecture.7.xml index c43f16d..defcdc9 100644 --- a/ovn-architecture.7.xml +++ b/ovn-architecture.7.xml @@ -1246,7 +1246,14 @@

Distributed gateway ports are logical router patch ports that directly connect distributed logical routers to logical - switches with localnet ports. + switches with external connection. +

+ +

+ There are two types of external connections. Firstly, connection to + physical network through a localnet port. Secondly, connection to + another OVN deployment, which will be introduced in section "OVN + Deployments Interconnection".

@@ -1820,6 +1827,141 @@ +

OVN Deployments Interconnection (TODO)

+ +

+ It is not uncommon for an operator to deploy multiple OVN clusters, for + two main reasons. Firstly, an operator may prefer to deploy one OVN + cluster for each availability zone, e.g. in different physical regions, + to avoid single point of failure. Secondly, there is always an upper limit + for a single OVN control plane to scale. +

+ +

+ Although the control planes of the different availability zone (AZ)s are + independent from each other, the workloads from different AZs may need + to communicate across the zones. The OVN interconnection feature provides + a native way to interconnect different AZs by L3 routing through transit + overlay networks between logical routers of different AZs. +

+ +

+ A global OVN Interconnection Northbound database is introduced for the + operator (probably through CMS systems) to configure transit logical + switches that connect logical routers from different AZs. A transit + switch is similar to a regular logical switch, but it is used for + interconnection purpose only. Typically, each transit switch can be used + to connect all logical routers that belong to same tenant across all AZs. +

+ +

+ A dedicated daemon process ovn-ic, OVN interconnection + controller, in each AZ will consume this data and populate corresponding + logical switches to their own northbound databases for each AZ, so that + logical routers can be connected to the transit switch by creating + patch port pairs in their northbound databases. Any router ports + connected to the transit switches are considered interconnection ports, + which will be exchanged between AZs. +

+ +

+ Physically, when workloads from different AZs communicate, packets + need to go through multiple hops: source chassis, source gateway, + destination gateway and destination chassis. All these hops are connected + through tunnels so that the packets never leave overlay networks. + A distributed gateway port is required to connect the logical router to a + transit switch, with a gateway chassis specified, so that the traffic can + be forwarded through the gateway chassis. +

+ +

+ A global OVN Interconnection Southbound database is introduced for + exchanging control plane information between the AZs. The data in + this database is populated and consumed by the ovn-ic, + of each AZ. The main information in this database includes: +

+ +
    +
  • + Datapath bindings for transit switches, which mainly contains the tunnel + keys generated for each transit switch. Separate key ranges are reserved + for transit switches so that they will never conflict with any tunnel + keys locally assigned for datapaths within each AZ. +
    • +
  • + Availability zones, which are registerd by ovn-ic + from each AZ. +
    • +
  • + Gateways. Each AZ specifies chassises that are supposed to work + as interconnection gateways, and the ovn-ic will + populate this information to the interconnection southbound DB. + The ovn-ic from all the other AZs will learn the + gateways and populate to their own southbound DB as a chassis. +
    • +
  • + Port bindings for logical switch ports created on the transit switch. + Each AZ maintains their logical router to transit switch connections + independently, but ovn-ic automatically populates + local port bindings on transit switches to the global interconnection + southbound DB, and learns remote port bindings from other AZs back + to its own northbound and southbound DBs, so that logical flows + can be produced and then translated to OVS flows locally, which finally + enables data plane communication. +
    • +
+ +

+ The tunnel keys for transit switch datapaths and related port bindings + must be agreed across all AZs. This is ensured by generating and storing + the keys in the global interconnection southbound database. Any + ovn-ic from any AZ can allocate the key, but race conditions + are solved by enforcing unique index for the column in the database. +

+ +

+ Once each AZ's NB and SB databases are populated with interconnection + switches and ports, and agreed upon the tunnel keys, data plane + communication between the AZs are established. +

+ +

A day in the life of a packet crossing AZs

+
    +
  1. + An IP packet is sent out from a VIF on a hypervisor (HV1) of AZ1, with + destination IP belonging to a VIF in AZ2. +
    2. +
  2. + In HV1's OVS flow tables, the packet goes through logical switch and + logical router pipelines, and in a logical router pipeline, the routing + stage finds out the next hop for the destination IP, which belongs to + a remote logical router port in AZ2, and the output port, which is a + chassis-redirect port located on an interconnection gateway (GW1 in AZ1), + so HV1 sends the packet to GW1 through tunnel. +
    3. +
  3. + On GW1, it continues with the logical router pipe line and switches to + the transit switch's pipeline through the peer port of the chassis + redirect port. In the transit switch's pipeline it outputs to the + remote logical port which is located on a gateway (GW2) in AZ2, so + the GW1 sends the packet to GW2 in tunnel. +
    4. +
  4. + On GW2, it continues with the transit switch pipeline and switches to + the logical router pipeline through the peer port, which is a chassis + redirect port that is located on GW2. The logical router pipeline + then forwards the packet to relevant logical pipelines according to + the destination IP address, and figures out the MAC and location + of the destination VIF port - a hypervisor (HV2). The GW2 then sends + the packet to HV2 in tunnel. +
    5. +
  5. + On HV2, the packet is delivered to the final destination VIF port by + the logical switch egress pipeline, just the same way as for intra-AZ + communications. +
    6. +
+

Native OVN services for external logical ports

From patchwork Wed Jan 29 19:55:57 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Han Zhou X-Patchwork-Id: 1231082 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.136; helo=silver.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=ovn.org Received: from silver.osuosl.org (smtp3.osuosl.org [140.211.166.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 487Dlp4ttlz9sNT for ; Thu, 30 Jan 2020 06:57:10 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by silver.osuosl.org (Postfix) with ESMTP id D0F002268D; Wed, 29 Jan 2020 19:57:08 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from silver.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 2iF1u2W3lDTL; Wed, 29 Jan 2020 19:56:36 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by silver.osuosl.org (Postfix) with ESMTP id 1B320220C5; Wed, 29 Jan 2020 19:56:36 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 03001C0881; Wed, 29 Jan 2020 19:56:36 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from fraxinus.osuosl.org (smtp4.osuosl.org [140.211.166.137]) by lists.linuxfoundation.org (Postfix) with ESMTP id 753DBC0171 for ; Wed, 29 Jan 2020 19:56:34 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by fraxinus.osuosl.org (Postfix) with ESMTP id 56DEA86054 for ; Wed, 29 Jan 2020 19:56:34 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from fraxinus.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id UZrZgXcVNsBz for ; Wed, 29 Jan 2020 19:56:28 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from relay1-d.mail.gandi.net (relay1-d.mail.gandi.net [217.70.183.193]) by fraxinus.osuosl.org (Postfix) with ESMTPS id 24C2985F85 for ; Wed, 29 Jan 2020 19:56:27 +0000 (UTC) X-Originating-IP: 216.113.160.77 Received: from localhost.localdomain.localdomain (unknown [216.113.160.77]) (Authenticated sender: hzhou@ovn.org) by relay1-d.mail.gandi.net (Postfix) with ESMTPSA id BF559240002; Wed, 29 Jan 2020 19:56:25 +0000 (UTC) From: Han Zhou To: dev@openvswitch.org Date: Wed, 29 Jan 2020 11:55:57 -0800 Message-Id: <1580327768-36501-3-git-send-email-hzhou@ovn.org> X-Mailer: git-send-email 2.1.0 In-Reply-To: <1580327768-36501-1-git-send-email-hzhou@ovn.org> References: <1580327768-36501-1-git-send-email-hzhou@ovn.org> Cc: Han Zhou Subject: [ovs-dev] [PATCH ovn v4 02/13] ovn-ic-nb: Interconnection northbound DB schema and CLI. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" This patch introduces OVN_IC_Northbound DB schema and the CLI ovn-ic-nbctl that manages the DB. Signed-off-by: Han Zhou --- .gitignore | 3 + automake.mk | 35 ++ debian/ovn-common.install | 1 + debian/ovn-common.manpages | 2 + lib/.gitignore | 3 + lib/automake.mk | 16 +- lib/ovn-ic-nb-idl.ann | 9 + lib/ovn-util.c | 13 + lib/ovn-util.h | 1 + ovn-ic-nb.ovsschema | 75 ++++ ovn-ic-nb.xml | 371 +++++++++++++++++ tests/automake.mk | 2 + tests/ovn-ic-nbctl.at | 65 +++ tests/testsuite.at | 1 + utilities/.gitignore | 2 + utilities/automake.mk | 8 + utilities/ovn-ic-nbctl.8.xml | 174 ++++++++ utilities/ovn-ic-nbctl.c | 950 +++++++++++++++++++++++++++++++++++++++++++ 18 files changed, 1730 insertions(+), 1 deletion(-) create mode 100644 lib/ovn-ic-nb-idl.ann create mode 100644 ovn-ic-nb.ovsschema create mode 100644 ovn-ic-nb.xml create mode 100644 tests/ovn-ic-nbctl.at create mode 100644 utilities/ovn-ic-nbctl.8.xml create mode 100644 utilities/ovn-ic-nbctl.c diff --git a/.gitignore b/.gitignore index 6fee075..d4f8c10 100644 --- a/.gitignore +++ b/.gitignore @@ -67,6 +67,9 @@ /ovn-sb.5 /ovn-sb.gv /ovn-sb.pic +/ovn-ic-nb.5 +/ovn-ic-nb.gv +/ovn-ic-nb.pic /package.m4 /stamp-h1 /_build-gcc diff --git a/automake.mk b/automake.mk index 591e007..59063c2 100644 --- a/automake.mk +++ b/automake.mk @@ -62,6 +62,34 @@ ovn-sb.5: \ $(srcdir)/ovn-sb.xml > $@.tmp && \ mv $@.tmp $@ +# OVN interconnection northbound E-R diagram +# +# If "python" or "dot" is not available, then we do not add graphical diagram +# to the documentation. +if HAVE_DOT +ovn-ic-nb.gv: ${OVSDIR}/ovsdb/ovsdb-dot.in $(srcdir)/ovn-ic-nb.ovsschema + $(AM_V_GEN)$(OVSDB_DOT) --no-arrows $(srcdir)/ovn-ic-nb.ovsschema > $@ +ovn-ic-nb.pic: ovn-ic-nb.gv ${OVSDIR}/ovsdb/dot2pic + $(AM_V_GEN)(dot -T plain < ovn-ic-nb.gv | $(PYTHON) ${OVSDIR}/ovsdb/dot2pic -f 3) > $@.tmp && \ + mv $@.tmp $@ +OVN_IC_NB_PIC = ovn-ic-nb.pic +OVN_IC_NB_DOT_DIAGRAM_ARG = --er-diagram=$(OVN_IC_NB_PIC) +CLEANFILES += ovn-ic-nb.gv ovn-ic-nb.pic +endif + +# OVN interconnection northbound schema documentation +EXTRA_DIST += ovn-ic-nb.xml +CLEANFILES += ovn-ic-nb.5 +man_MANS += ovn-ic-nb.5 + +ovn-ic-nb.5: \ + ${OVSDIR}/ovsdb/ovsdb-doc $(srcdir)/ovn-ic-nb.xml $(srcdir)/ovn-ic-nb.ovsschema $(OVN_IC_NB_PIC) + $(AM_V_GEN)$(OVSDB_DOC) \ + $(OVN_IC_NB_DOT_DIAGRAM_ARG) \ + --version=$(VERSION) \ + $(srcdir)/ovn-ic-nb.ovsschema \ + $(srcdir)/ovn-ic-nb.xml > $@.tmp && \ + mv $@.tmp $@ # Version checking for ovn-nb.ovsschema. ALL_LOCAL += ovn-nb.ovsschema.stamp @@ -74,7 +102,14 @@ ALL_LOCAL += ovn-sb.ovsschema.stamp ovn-sb.ovsschema.stamp: ovn-sb.ovsschema $(srcdir)/build-aux/cksum-schema-check $? $@ +# Version checking for ovn-ic-nb.ovsschema. +ALL_LOCAL += ovn-ic-nb.ovsschema.stamp +ovn-ic-nb.ovsschema.stamp: ovn-ic-nb.ovsschema + $(srcdir)/build-aux/cksum-schema-check $? $@ +CLEANFILES += ovn-ic-nb.ovsschema.stamp + pkgdata_DATA += ovn-nb.ovsschema pkgdata_DATA += ovn-sb.ovsschema +pkgdata_DATA += ovn-ic-nb.ovsschema CLEANFILES += ovn-sb.ovsschema.stamp diff --git a/debian/ovn-common.install b/debian/ovn-common.install index 90484d2..59b8018 100644 --- a/debian/ovn-common.install +++ b/debian/ovn-common.install @@ -1,5 +1,6 @@ usr/bin/ovn-nbctl usr/bin/ovn-sbctl +usr/bin/ovn-ic-nbctl usr/bin/ovn-trace usr/bin/ovn-detrace usr/share/openvswitch/scripts/ovn-ctl diff --git a/debian/ovn-common.manpages b/debian/ovn-common.manpages index 249349e..e7d3e4d 100644 --- a/debian/ovn-common.manpages +++ b/debian/ovn-common.manpages @@ -1,8 +1,10 @@ ovn/ovn-architecture.7 ovn/ovn-nb.5 ovn/ovn-sb.5 +ovn/ovn-ic-nb.5 ovn/utilities/ovn-ctl.8 ovn/utilities/ovn-nbctl.8 ovn/utilities/ovn-sbctl.8 +ovn/utilities/ovn-ic-nbctl.8 ovn/utilities/ovn-trace.8 ovn/utilities/ovn-detrace.1 diff --git a/lib/.gitignore b/lib/.gitignore index 3eed458..3af2923 100644 --- a/lib/.gitignore +++ b/lib/.gitignore @@ -5,4 +5,7 @@ /ovn-sb-idl.c /ovn-sb-idl.h /ovn-sb-idl.ovsidl +/ovn-ic-nb-idl.c +/ovn-ic-nb-idl.h +/ovn-ic-nb-idl.ovsidl /ovn-dirs.c diff --git a/lib/automake.mk b/lib/automake.mk index 0c8245c..5f6561a 100644 --- a/lib/automake.mk +++ b/lib/automake.mk @@ -29,7 +29,9 @@ nodist_lib_libovn_la_SOURCES = \ lib/ovn-nb-idl.c \ lib/ovn-nb-idl.h \ lib/ovn-sb-idl.c \ - lib/ovn-sb-idl.h + lib/ovn-sb-idl.h \ + lib/ovn-ic-nb-idl.c \ + lib/ovn-ic-nb-idl.h CLEANFILES += $(nodist_lib_libovn_la_SOURCES) @@ -74,3 +76,15 @@ lib/ovn-nb-idl.ovsidl: $(OVN_NB_IDL_FILES) $(AM_V_GEN)$(OVSDB_IDLC) annotate $(OVN_NB_IDL_FILES) > $@.tmp && \ mv $@.tmp $@ +# ovn-ic-nb IDL +OVSIDL_BUILT += \ + lib/ovn-ic-nb-idl.c \ + lib/ovn-ic-nb-idl.h \ + lib/ovn-ic-nb-idl.ovsidl +EXTRA_DIST += lib/ovn-ic-nb-idl.ann +OVN_IC_NB_IDL_FILES = \ + $(srcdir)/ovn-ic-nb.ovsschema \ + $(srcdir)/lib/ovn-ic-nb-idl.ann +lib/ovn-ic-nb-idl.ovsidl: $(OVN_IC_NB_IDL_FILES) + $(AM_V_GEN)$(OVSDB_IDLC) annotate $(OVN_IC_NB_IDL_FILES) > $@.tmp && \ + mv $@.tmp $@ diff --git a/lib/ovn-ic-nb-idl.ann b/lib/ovn-ic-nb-idl.ann new file mode 100644 index 0000000..9dfbafa --- /dev/null +++ b/lib/ovn-ic-nb-idl.ann @@ -0,0 +1,9 @@ +# -*- python -*- + +# This code, when invoked by "ovsdb-idlc annotate" (by the build +# process), annotates ovn-ic-nb.ovsschema with additional data that give +# the ovsdb-idl engine information about the types involved, so that +# it can generate more programmer-friendly data structures. + +s["idlPrefix"] = "icnbrec_" +s["idlHeader"] = "\"lib/ovn-ic-nb-idl.h\"" diff --git a/lib/ovn-util.c b/lib/ovn-util.c index 6e0bba2..5b891ec 100644 --- a/lib/ovn-util.c +++ b/lib/ovn-util.c @@ -350,6 +350,19 @@ default_sb_db(void) return def; } +const char * +default_ic_nb_db(void) +{ + static char *def; + if (!def) { + def = getenv("OVN_IC_NB_DB"); + if (!def) { + def = xasprintf("unix:%s/ovn_ic_nb_db.sock", ovn_rundir()); + } + } + return def; +} + char * get_abs_unix_ctl_path(void) { diff --git a/lib/ovn-util.h b/lib/ovn-util.h index 422d69e..cda245b 100644 --- a/lib/ovn-util.h +++ b/lib/ovn-util.h @@ -73,6 +73,7 @@ char *alloc_nat_zone_key(const struct uuid *key, const char *type); const char *default_nb_db(void); const char *default_sb_db(void); +const char *default_ic_nb_db(void); char *get_abs_unix_ctl_path(void); struct ovsdb_idl_table_class; diff --git a/ovn-ic-nb.ovsschema b/ovn-ic-nb.ovsschema new file mode 100644 index 0000000..894db83 --- /dev/null +++ b/ovn-ic-nb.ovsschema @@ -0,0 +1,75 @@ +{ + "name": "OVN_IC_Northbound", + "version": "1.0.0", + "cksum": "45589876 3383", + "tables": { + "IC_NB_Global": { + "columns": { + "external_ids": { + "type": {"key": "string", "value": "string", + "min": 0, "max": "unlimited"}}, + "connections": { + "type": {"key": {"type": "uuid", + "refTable": "Connection"}, + "min": 0, + "max": "unlimited"}}, + "ssl": { + "type": {"key": {"type": "uuid", + "refTable": "SSL"}, + "min": 0, "max": 1}}, + "options": { + "type": {"key": "string", "value": "string", + "min": 0, "max": "unlimited"}}}, + "maxRows": 1, + "isRoot": true}, + "Transit_Switch": { + "columns": { + "name": {"type": "string"}, + "other_config": { + "type": {"key": "string", "value": "string", + "min": 0, "max": "unlimited"}}, + "external_ids": { + "type": {"key": "string", "value": "string", + "min": 0, "max": "unlimited"}}}, + "isRoot": true, + "indexes": [["name"]]}, + "Connection": { + "columns": { + "target": {"type": "string"}, + "max_backoff": {"type": {"key": {"type": "integer", + "minInteger": 1000}, + "min": 0, + "max": 1}}, + "inactivity_probe": {"type": {"key": "integer", + "min": 0, + "max": 1}}, + "other_config": {"type": {"key": "string", + "value": "string", + "min": 0, + "max": "unlimited"}}, + "external_ids": {"type": {"key": "string", + "value": "string", + "min": 0, + "max": "unlimited"}}, + "is_connected": {"type": "boolean", "ephemeral": true}, + "status": {"type": {"key": "string", + "value": "string", + "min": 0, + "max": "unlimited"}, + "ephemeral": true}}, + "indexes": [["target"]]}, + "SSL": { + "columns": { + "private_key": {"type": "string"}, + "certificate": {"type": "string"}, + "ca_cert": {"type": "string"}, + "bootstrap_ca_cert": {"type": "boolean"}, + "ssl_protocols": {"type": "string"}, + "ssl_ciphers": {"type": "string"}, + "external_ids": {"type": {"key": "string", + "value": "string", + "min": 0, + "max": "unlimited"}}}, + "maxRows": 1} + } +} diff --git a/ovn-ic-nb.xml b/ovn-ic-nb.xml new file mode 100644 index 0000000..a693611 --- /dev/null +++ b/ovn-ic-nb.xml @@ -0,0 +1,371 @@ + + +

+ This database is the interface for cloud management system (CMS), such as + OpenStack, to configure OVN interconnection settings. The CMS produces + almost all of the contents of the database. The ovn-ic + program monitors the database contents, transforms it, and stores it into + the database. +

+ +

+ We generally speak of ``the'' CMS, but one can imagine scenarios in + which multiple CMSes manage different parts of OVN interconnection. +

+ +

External IDs

+ +

+ Each of the tables in this database contains a special column, named + external_ids. This column has the same form and purpose each + place it appears. +

+ +
+
external_ids: map of string-string pairs
+
+ Key-value pairs for use by the CMS. The CMS might use certain pairs, for + example, to identify entities in its own configuration that correspond to + those in this database. +
+
+ + +

+ Northbound configuration for OVN interconnection. This table must have exactly + one row. +

+ + + + See External IDs at the beginning of this document. + + + + + + This column provides general key/value settings. The supported + options are described individually below. + + + + + + Database clients to which the Open vSwitch database server should + connect or on which it should listen, along with options for how these + connections should be configured. See the + table for more information. + + + Global SSL configuration. + + +
+ + +

+ Each row represents one transit logical switch for interconnection between + different OVN deployments (availability zones). +

+ + + + A name that uniquely identifies the transit logical switch. + + + + + + + See External IDs at the beginning of this document. + + +
+ + + SSL configuration for ovn-nb database access. + + + Name of a PEM file containing the private key used as the switch's + identity for SSL connections to the controller. + + + + Name of a PEM file containing a certificate, signed by the + certificate authority (CA) used by the controller and manager, + that certifies the switch's private key, identifying a trustworthy + switch. + + + + Name of a PEM file containing the CA certificate used to verify + that the switch is connected to a trustworthy controller. + + + + If set to true, then Open vSwitch will attempt to + obtain the CA certificate from the controller on its first SSL + connection and save it to the named PEM file. If it is successful, + it will immediately drop the connection and reconnect, and from then + on all SSL connections must be authenticated by a certificate signed + by the CA certificate thus obtained. This option exposes the + SSL connection to a man-in-the-middle attack obtaining the initial + CA certificate. It may still be useful for bootstrapping. + + + + List of SSL protocols to be enabled for SSL connections. The default + when this option is omitted is TLSv1,TLSv1.1,TLSv1.2. + + + + List of ciphers (in OpenSSL cipher string format) to be supported + for SSL connections. The default when this option is omitted is + HIGH:!aNULL:!MD5. + + + + The overall purpose of these columns is described under Common + Columns at the beginning of this document. + + + +
+ + +

+ Configuration for a database connection to an Open vSwitch database + (OVSDB) client. +

+ +

+ This table primarily configures the Open vSwitch database server + (ovsdb-server). +

+ +

+ The Open vSwitch database server can initiate and maintain active + connections to remote clients. It can also listen for database + connections. +

+ + + +

Connection methods for clients.

+

+ The following connection methods are currently supported: +

+
+
ssl:host[:port]
+
+

+ The specified SSL port on the host at the given + host, which can either be a DNS name (if built with + unbound library) or an IP address. A valid SSL configuration must + be provided when this form is used, this configuration can be + specified via command-line options or the table. +

+

+ If port is not specified, it defaults to 6640. +

+

+ SSL support is an optional feature that is not always + built as part of Open vSwitch. +

+
+ +
tcp:host[:port]
+
+

+ The specified TCP port on the host at the given + host, which can either be a DNS name (if built with + unbound library) or an IP address. If host is an IPv6 + address, wrap it in square brackets, e.g. tcp:[::1]:6640. +

+

+ If port is not specified, it defaults to 6640. +

+
+
pssl:[port][:host]
+
+

+ Listens for SSL connections on the specified TCP port. + Specify 0 for port to have the kernel automatically + choose an available port. If host, which can either + be a DNS name (if built with unbound library) or an IP address, + is specified, then connections are restricted to the resolved or + specified local IPaddress (either IPv4 or IPv6 address). If + host is an IPv6 address, wrap in square brackets, + e.g. pssl:6640:[::1]. If host is not + specified then it listens only on IPv4 (but not IPv6) addresses. + A valid SSL configuration must be provided when this form is used, + this can be specified either via command-line options or the + table. +

+

+ If port is not specified, it defaults to 6640. +

+

+ SSL support is an optional feature that is not always built as + part of Open vSwitch. +

+
+
ptcp:[port][:host]
+
+

+ Listens for connections on the specified TCP port. + Specify 0 for port to have the kernel automatically + choose an available port. If host, which can either + be a DNS name (if built with unbound library) or an IP address, + is specified, then connections are restricted to the resolved or + specified local IP address (either IPv4 or IPv6 address). If + host is an IPv6 address, wrap it in square brackets, + e.g. ptcp:6640:[::1]. If host is not + specified then it listens only on IPv4 addresses. +

+

+ If port is not specified, it defaults to 6640. +

+
+
+

When multiple clients are configured, the + values must be unique. Duplicate values yield + unspecified results.

+ + + + + + Maximum number of milliseconds to wait between connection attempts. + Default is implementation-specific. + + + + Maximum number of milliseconds of idle time on connection to the client + before sending an inactivity probe message. If Open vSwitch does not + communicate with the client for the specified number of seconds, it + will send a probe. If a response is not received for the same + additional amount of time, Open vSwitch assumes the connection has been + broken and attempts to reconnect. Default is implementation-specific. + A value of 0 disables inactivity probes. + + + + +

+ Key-value pair of is always updated. + Other key-value pairs in the status columns may be updated depends + on the type. +

+ +

+ When specifies a connection method that + listens for inbound connections (e.g. ptcp: or + punix:), both and + may also be updated while the + remaining key-value pairs are omitted. +

+ +

+ On the other hand, when specifies an + outbound connection, all key-value pairs may be updated, except + the above-mentioned two key-value pairs associated with inbound + connection targets. They are omitted. +

+ + + true if currently connected to this client, + false otherwise. + + + + A human-readable description of the last error on the connection + to the manager; i.e. strerror(errno). This key + will exist only if an error has occurred. + + + +

+ The state of the connection to the manager: +

+
+
VOID
+
Connection is disabled.
+ +
BACKOFF
+
Attempting to reconnect at an increasing period.
+ +
CONNECTING
+
Attempting to connect.
+ +
ACTIVE
+
Connected, remote host responsive.
+ +
IDLE
+
Connection is idle. Waiting for response to keep-alive.
+
+

+ These values may change in the future. They are provided only for + human consumption. +

+ + + + The amount of time since this client last successfully connected + to the database (in seconds). Value is empty if client has never + successfully been connected. + + + + The amount of time since this client last disconnected from the + database (in seconds). Value is empty if client has never + disconnected. + + + + Space-separated list of the names of OVSDB locks that the connection + holds. Omitted if the connection does not hold any locks. + + + + Space-separated list of the names of OVSDB locks that the connection is + currently waiting to acquire. Omitted if the connection is not waiting + for any locks. + + + + Space-separated list of the names of OVSDB locks that the connection + has had stolen by another OVSDB client. Omitted if no locks have been + stolen from this connection. + + + + When specifies a connection method that + listens for inbound connections (e.g. ptcp: or + pssl:) and more than one connection is actually active, + the value is the number of active connections. Otherwise, this + key-value pair is omitted. + + + + When is ptcp: or + pssl:, this is the TCP port on which the OVSDB server is + listening. (This is particularly useful when specifies a port of 0, allowing the kernel to + choose any available port.) + + + + + The overall purpose of these columns is described under Common + Columns at the beginning of this document. + + + + +
+ diff --git a/tests/automake.mk b/tests/automake.mk index ed6240c..8c1630c 100644 --- a/tests/automake.mk +++ b/tests/automake.mk @@ -24,6 +24,7 @@ TESTSUITE_AT = \ tests/ovn-northd.at \ tests/ovn-nbctl.at \ tests/ovn-sbctl.at \ + tests/ovn-ic-nbctl.at \ tests/ovn-controller.at \ tests/ovn-controller-vtep.at \ tests/ovn-macros.at \ @@ -99,6 +100,7 @@ valgrind_wrappers = \ tests/valgrind/ovn-nbctl \ tests/valgrind/ovn-northd \ tests/valgrind/ovn-sbctl \ + tests/valgrind/ovn-ic-nbctl \ tests/valgrind/ovs-appctl \ tests/valgrind/ovs-ofctl \ tests/valgrind/ovs-vsctl \ diff --git a/tests/ovn-ic-nbctl.at b/tests/ovn-ic-nbctl.at new file mode 100644 index 0000000..2402d64 --- /dev/null +++ b/tests/ovn-ic-nbctl.at @@ -0,0 +1,65 @@ +AT_BANNER([ovn-ic-nbctl]) + +# OVN_IC_NBCTL_TEST_START +m4_define([OVN_IC_NBCTL_TEST_START], + [dnl Create database (ovn-ic-nb). + AT_KEYWORDS([ic_nbctl]) + AT_CHECK([ovsdb-tool create ovn-ic-nb.db $abs_top_srcdir/ovn-ic-nb.ovsschema]) + + dnl Start ovsdb-servers. + AT_CHECK([ovsdb-server --detach --no-chdir --pidfile=ovn_ic_nb_db.pid --unixctl=$OVS_RUNDIR/ovn_ic_nb_db.ctl --log-file=ovsdb_ic_nb.log --remote=punix:$OVS_RUNDIR/ovn_ic_nb_db.sock ovn-ic-nb.db ], [0], [], [stderr]) + on_exit "kill `cat ovn_ic_nb_db.pid`" + AT_CHECK([[sed < stderr ' +/vlog|INFO|opened log file/d +/ovsdb_server|INFO|ovsdb-server (Open vSwitch)/d']]) + AT_CAPTURE_FILE([ovsdb-server.log]) +]) + +# OVN_IC_NBCTL_TEST_STOP +m4_define([OVN_IC_NBCTL_TEST_STOP], + [AT_CHECK([check_logs "$1"]) + OVS_APP_EXIT_AND_WAIT_BY_TARGET([$OVS_RUNDIR/ovn_ic_nb_db.ctl], [$OVS_RUNDIR/ovn_ic_nb_db.pid])]) + +dnl --------------------------------------------------------------------- + +AT_SETUP([ovn-ic-nbctl]) +OVN_IC_NBCTL_TEST_START + +AT_CHECK([ovn-ic-nbctl ts-add ts0]) +AT_CHECK([ovn-ic-nbctl ts-list | uuidfilt], [0], [dnl +<0> (ts0) +]) + +AT_CHECK([ovn-ic-nbctl ts-add ts1]) +AT_CHECK([ovn-ic-nbctl ts-list | uuidfilt], [0], [dnl +<0> (ts0) +<1> (ts1) +]) + +AT_CHECK([ovn-ic-nbctl show | sort], [0], [dnl +Transit_Switch ts0 +Transit_Switch ts1 +]) + +AT_CHECK([ovn-ic-nbctl ts-del ts1]) +AT_CHECK([ovn-ic-nbctl ts-list | uuidfilt], [0], [dnl +<0> (ts0) +]) + +AT_CHECK([ovn-ic-nbctl ts-add ts0], [1], [], + [ovn-ic-nbctl: ts0: a transit switch with this name already exists +]) + +AT_CHECK([ovn-ic-nbctl --may-exist ts-add ts0]) +AT_CHECK([ovn-ic-nbctl ts-list | uuidfilt], [0], [dnl +<0> (ts0) +]) + +AT_CHECK([ovn-ic-nbctl ts-del ts2], [1], [], + [ovn-ic-nbctl: ts2: switch name not found +]) + +AT_CHECK([ovn-ic-nbctl --if-exists ts-del ts2]) + +OVN_IC_NBCTL_TEST_STOP +AT_CLEANUP diff --git a/tests/testsuite.at b/tests/testsuite.at index da8157d..aa44a2e 100644 --- a/tests/testsuite.at +++ b/tests/testsuite.at @@ -26,6 +26,7 @@ m4_include([tests/ovn-performance.at]) m4_include([tests/ovn-northd.at]) m4_include([tests/ovn-nbctl.at]) m4_include([tests/ovn-sbctl.at]) +m4_include([tests/ovn-ic-nbctl.at]) m4_include([tests/ovn-controller.at]) m4_include([tests/ovn-controller-vtep.at]) m4_include([tests/checkpatch.at]) diff --git a/utilities/.gitignore b/utilities/.gitignore index ab063fa..6c710fb 100644 --- a/utilities/.gitignore +++ b/utilities/.gitignore @@ -3,6 +3,8 @@ /ovn-nbctl.8 /ovn-sbctl /ovn-sbctl.8 +/ovn-ic-nbctl +/ovn-ic-nbctl.8 /ovn-appctl /ovn-appctl.8 /ovn-trace diff --git a/utilities/automake.mk b/utilities/automake.mk index 4792799..ac2e034 100644 --- a/utilities/automake.mk +++ b/utilities/automake.mk @@ -7,6 +7,7 @@ man_MANS += \ utilities/ovn-ctl.8 \ utilities/ovn-nbctl.8 \ utilities/ovn-sbctl.8 \ + utilities/ovn-ic-nbctl.8 \ utilities/ovn-trace.8 \ utilities/ovn-detrace.1 \ utilities/ovn-appctl.8 @@ -28,6 +29,7 @@ EXTRA_DIST += \ utilities/ovn-docker-overlay-driver.in \ utilities/ovn-docker-underlay-driver.in \ utilities/ovn-nbctl.8.xml \ + utilities/ovn-ic-nbctl.8.xml \ utilities/ovn-appctl.8.xml \ utilities/ovn-trace.8.xml \ utilities/ovn-detrace.in \ @@ -51,6 +53,7 @@ CLEANFILES += \ utilities/ovn-docker-underlay-driver \ utilities/ovn-nbctl.8 \ utilities/ovn-sbctl.8 \ + utilities/ovn-ic-nbctl.8 \ utilities/ovn-trace.8 \ utilities/ovn-detrace.1 \ utilities/ovn-detrace \ @@ -73,6 +76,11 @@ bin_PROGRAMS += utilities/ovn-sbctl utilities_ovn_sbctl_SOURCES = utilities/ovn-sbctl.c utilities_ovn_sbctl_LDADD = lib/libovn.la $(OVSDB_LIBDIR)/libovsdb.la $(OVS_LIBDIR)/libopenvswitch.la +# ovn-ic-nbctl +bin_PROGRAMS += utilities/ovn-ic-nbctl +utilities_ovn_ic_nbctl_SOURCES = utilities/ovn-ic-nbctl.c +utilities_ovn_ic_nbctl_LDADD = lib/libovn.la $(OVSDB_LIBDIR)/libovsdb.la $(OVS_LIBDIR)/libopenvswitch.la + # ovn-trace bin_PROGRAMS += utilities/ovn-trace utilities_ovn_trace_SOURCES = utilities/ovn-trace.c diff --git a/utilities/ovn-ic-nbctl.8.xml b/utilities/ovn-ic-nbctl.8.xml new file mode 100644 index 0000000..4a70994 --- /dev/null +++ b/utilities/ovn-ic-nbctl.8.xml @@ -0,0 +1,174 @@ + + +

Name

+

ovn-ic-nbctl -- Open Virtual Network interconnection northbound db management utility

+ +

Synopsis

+

ovn-ic-nbctl [options] command [arg...]

+ +

Description

+

This utility can be used to manage the OVN interconnection northbound database.

+ +

General Commands

+ +
+
init
+
+ Initializes the database, if it is empty. If the database has already + been initialized, this command has no effect. +
+ +
show
+
+ Prints a brief overview of the database contents. +
+
+ +

Transit Switch Commands

+ +
+
[--may-exist] ts-add switch
+
+

+ Creates a new transit switch named switch. +

+ +

+ Transit switch names must be unique. Adding a duplicated name results + in error. With --may-exist, adding a duplicate name + succeeds but does not create a new transit switch. +

+
+ +
[--if-exists] ts-del switch
+
+ Deletes switch. It is an error if switch does + not exist, unless --if-exists is specified. +
+ +
ts-list
+
+ Lists all existing switches on standard output, one per line. +
+
+ +

Database Commands

+

These commands query and modify the contents of ovsdb tables. + They are a slight abstraction of the ovsdb interface and + as such they operate at a lower level than other ovn-ic-nbctl commands.

+

Identifying Tables, Records, and Columns

+

Each of these commands has a table parameter to identify a table + within the database. Many of them also take a record parameter + that identifies a particular record within a table. The record + parameter may be the UUID for a record, which may be abbreviated to its + first 4 (or more) hex digits, as long as that is unique. Many tables offer + additional ways to identify records. Some commands also take + column parameters that identify a particular field within the + records in a table.

+ +

+ For a list of tables and their columns, see ovn-ic-nb(5) or + see the table listing from the --help option. +

+ +

+ Record names must be specified in full and with correct capitalization, + except that UUIDs may be abbreviated to their first 4 (or more) hex + digits, as long as that is unique within the table. Names of tables and + columns are not case-sensitive, and - and _ are + treated interchangeably. Unique abbreviations of table and column names + are acceptable, e.g. t or transit is sufficient + to identify the Transit_Switch table. +

+ + + +

Remote Connectivity Commands

+
+
get-connection
+
+ Prints the configured connection(s). +
+ +
del-connection
+
+ Deletes the configured connection(s). +
+ +
[--inactivity-probe=msecs] set-connection target...
+
+ Sets the configured manager target or targets. Use + --inactivity-probe=msecs to override the default + idle connection inactivity probe time. Use 0 to disable inactivity probes. +
+
+ +

SSL Configuration Commands

+
+
get-ssl
+
+ Prints the SSL configuration. +
+ +
del-ssl
+
+ Deletes the current SSL configuration. +
+ +
[--bootstrap] set-ssl + private-key certificate ca-cert + [ssl-protocol-list [ssl-cipher-list]]
+
+ Sets the SSL configuration. +
+
+ +

Options

+ +
+
--db database
+
+ The OVSDB database remote to contact. If the OVN_IC_NB_DB + environment variable is set, its value is used as the default. + Otherwise, the default is unix:@RUNDIR@/ovn_ic_nb_db.sock, but this + default is unlikely to be useful outside of single-machine OVN test + environments. +
+ +
--leader-only
+
--no-leader-only
+
+ By default, or with --leader-only, when the database server + is a clustered database, ovn-ic-nbctl will avoid servers other + than the cluster leader. This ensures that any data that + ovn-ic-nbctl reads and reports is up-to-date. With + --no-leader-only, ovn-ic-nbctl will use any server + in the cluster, which means that for read-only transactions it can report + and act on stale data (transactions that modify the database are always + serialized even with --no-leader-only). Refer to + Understanding Cluster Consistency in ovsdb(7) + for more information. +
+
+ +

Logging options

+ + +

Table Formatting Options

+ These options control the format of output from the list and + find commands. + + +

PKI Options

+

+ PKI configuration is required to use SSL for the connection to the + database. +

+ + + +

Other Options

+ + + + diff --git a/utilities/ovn-ic-nbctl.c b/utilities/ovn-ic-nbctl.c new file mode 100644 index 0000000..9f209a9 --- /dev/null +++ b/utilities/ovn-ic-nbctl.c @@ -0,0 +1,950 @@ +/* + * Copyright (c) 2020 eBay Inc. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at: + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#include + +#include +#include +#include +#include +#include + +#include "command-line.h" +#include "compiler.h" +#include "db-ctl-base.h" +#include "dirs.h" +#include "fatal-signal.h" +#include "openvswitch/dynamic-string.h" +#include "openvswitch/json.h" +#include "openvswitch/shash.h" +#include "openvswitch/vconn.h" +#include "openvswitch/vlog.h" +#include "lib/ovn-ic-nb-idl.h" +#include "lib/ovn-util.h" +#include "openvswitch/poll-loop.h" +#include "process.h" +#include "sset.h" +#include "stream-ssl.h" +#include "stream.h" +#include "table.h" +#include "timeval.h" +#include "util.h" +#include "svec.h" + +VLOG_DEFINE_THIS_MODULE(ic_nbctl); + +struct ic_nbctl_context; + +/* --db: The database server to contact. */ +static const char *db; + +/* --oneline: Write each command's output as a single line? */ +static bool oneline; + +/* --dry-run: Do not commit any changes. */ +static bool dry_run; + +/* --timeout: Time to wait for a connection to 'db'. */ +static unsigned int timeout; + +/* Format for table output. */ +static struct table_style table_style = TABLE_STYLE_DEFAULT; + +/* The IDL we're using and the current transaction, if any. + * This is for use by ic_nbctl_exit() only, to allow it to clean up. + * Other code should use its context arguments. */ +static struct ovsdb_idl *the_idl; +static struct ovsdb_idl_txn *the_idl_txn; +OVS_NO_RETURN static void ic_nbctl_exit(int status); + +/* --leader-only, --no-leader-only: Only accept the leader in a cluster. */ +static int leader_only = true; + +static void ic_nbctl_cmd_init(void); +OVS_NO_RETURN static void usage(void); +static void parse_options(int argc, char *argv[], struct shash *local_options); +static void run_prerequisites(struct ctl_command[], size_t n_commands, + struct ovsdb_idl *); +static bool do_ic_nbctl(const char *args, struct ctl_command *, size_t n, + struct ovsdb_idl *); + +int +main(int argc, char *argv[]) +{ + struct ovsdb_idl *idl; + struct ctl_command *commands; + struct shash local_options; + unsigned int seqno; + size_t n_commands; + + set_program_name(argv[0]); + fatal_ignore_sigpipe(); + vlog_set_levels(NULL, VLF_CONSOLE, VLL_WARN); + vlog_set_levels_from_string_assert("reconnect:warn"); + + ic_nbctl_cmd_init(); + + /* Parse command line. */ + char *args = process_escape_args(argv); + shash_init(&local_options); + parse_options(argc, argv, &local_options); + char *error = ctl_parse_commands(argc - optind, argv + optind, + &local_options, &commands, &n_commands); + if (error) { + ctl_fatal("%s", error); + } + VLOG(ctl_might_write_to_db(commands, n_commands) ? VLL_INFO : VLL_DBG, + "Called as %s", args); + + ctl_timeout_setup(timeout); + + /* Initialize IDL. */ + idl = the_idl = ovsdb_idl_create(db, &icnbrec_idl_class, true, false); + ovsdb_idl_set_leader_only(idl, leader_only); + run_prerequisites(commands, n_commands, idl); + + /* Execute the commands. + * + * 'seqno' is the database sequence number for which we last tried to + * execute our transaction. There's no point in trying to commit more than + * once for any given sequence number, because if the transaction fails + * it's because the database changed and we need to obtain an up-to-date + * view of the database before we try the transaction again. */ + seqno = ovsdb_idl_get_seqno(idl); + for (;;) { + ovsdb_idl_run(idl); + if (!ovsdb_idl_is_alive(idl)) { + int retval = ovsdb_idl_get_last_error(idl); + ctl_fatal("%s: database connection failed (%s)", + db, ovs_retval_to_string(retval)); + } + + if (seqno != ovsdb_idl_get_seqno(idl)) { + seqno = ovsdb_idl_get_seqno(idl); + if (do_ic_nbctl(args, commands, n_commands, idl)) { + free(args); + exit(EXIT_SUCCESS); + } + } + + if (seqno == ovsdb_idl_get_seqno(idl)) { + ovsdb_idl_wait(idl); + poll_block(); + } + } +} + +static void +parse_options(int argc, char *argv[], struct shash *local_options) +{ + enum { + OPT_DB = UCHAR_MAX + 1, + OPT_ONELINE, + OPT_NO_SYSLOG, + OPT_DRY_RUN, + OPT_LOCAL, + OPT_COMMANDS, + OPT_OPTIONS, + OPT_BOOTSTRAP_CA_CERT, + VLOG_OPTION_ENUMS, + TABLE_OPTION_ENUMS, + SSL_OPTION_ENUMS, + }; + static const struct option global_long_options[] = { + {"db", required_argument, NULL, OPT_DB}, + {"no-syslog", no_argument, NULL, OPT_NO_SYSLOG}, + {"dry-run", no_argument, NULL, OPT_DRY_RUN}, + {"oneline", no_argument, NULL, OPT_ONELINE}, + {"timeout", required_argument, NULL, 't'}, + {"help", no_argument, NULL, 'h'}, + {"commands", no_argument, NULL, OPT_COMMANDS}, + {"options", no_argument, NULL, OPT_OPTIONS}, + {"leader-only", no_argument, &leader_only, true}, + {"no-leader-only", no_argument, &leader_only, false}, + {"version", no_argument, NULL, 'V'}, + VLOG_LONG_OPTIONS, + STREAM_SSL_LONG_OPTIONS, + {"bootstrap-ca-cert", required_argument, NULL, OPT_BOOTSTRAP_CA_CERT}, + TABLE_LONG_OPTIONS, + {NULL, 0, NULL, 0}, + }; + const int n_global_long_options = ARRAY_SIZE(global_long_options) - 1; + char *tmp, *short_options; + + struct option *options; + size_t allocated_options; + size_t n_options; + size_t i; + + tmp = ovs_cmdl_long_options_to_short_options(global_long_options); + short_options = xasprintf("+%s", tmp); + free(tmp); + + /* We want to parse both global and command-specific options here, but + * getopt_long() isn't too convenient for the job. We copy our global + * options into a dynamic array, then append all of the command-specific + * options. */ + options = xmemdup(global_long_options, sizeof global_long_options); + allocated_options = ARRAY_SIZE(global_long_options); + n_options = n_global_long_options; + ctl_add_cmd_options(&options, &n_options, &allocated_options, OPT_LOCAL); + + for (;;) { + int idx; + int c; + + c = getopt_long(argc, argv, short_options, options, &idx); + if (c == -1) { + break; + } + + switch (c) { + case OPT_DB: + db = optarg; + break; + + case OPT_ONELINE: + oneline = true; + break; + + case OPT_NO_SYSLOG: + vlog_set_levels(&this_module, VLF_SYSLOG, VLL_WARN); + break; + + case OPT_DRY_RUN: + dry_run = true; + break; + + case OPT_LOCAL: + if (shash_find(local_options, options[idx].name)) { + ctl_fatal("'%s' option specified multiple times", + options[idx].name); + } + shash_add_nocopy(local_options, + xasprintf("--%s", options[idx].name), + nullable_xstrdup(optarg)); + break; + + case 'h': + usage(); + + case OPT_COMMANDS: + ctl_print_commands(); + /* fall through */ + + case OPT_OPTIONS: + ctl_print_options(global_long_options); + /* fall through */ + + case 'V': + ovs_print_version(0, 0); + printf("DB Schema %s\n", icnbrec_get_db_version()); + exit(EXIT_SUCCESS); + + case 't': + if (!str_to_uint(optarg, 10, &timeout) || !timeout) { + ctl_fatal("value %s on -t or --timeout is invalid", optarg); + } + break; + + VLOG_OPTION_HANDLERS + TABLE_OPTION_HANDLERS(&table_style) + STREAM_SSL_OPTION_HANDLERS + + case OPT_BOOTSTRAP_CA_CERT: + stream_ssl_set_ca_cert_file(optarg, true); + break; + + case '?': + exit(EXIT_FAILURE); + + default: + ovs_abort(0, "Internal error when parsing option %d.", c); + + case 0: + break; + } + } + free(short_options); + + if (!db) { + db = default_ic_nb_db(); + } + + for (i = n_global_long_options; options[i].name; i++) { + free(CONST_CAST(char *, options[i].name)); + } + free(options); +} + +static void +usage(void) +{ + printf("\ +%s: OVN interconnection northbound DB management utility\n\ +\n\ +usage: %s [OPTIONS] COMMAND [ARG...]\n\ +\n\ +General commands:\n\ + init initialize the database\n\ + show print overview of database contents\n\ +\n\ +Transit switch commands:\n\ + ts-add SWITCH create a transit switch named SWITCH\n\ + ts-del SWITCH delete SWITCH\n\ + ts-list print all transit switches\n\ +\n\ +Connection commands:\n\ + get-connection print the connections\n\ + del-connection delete the connections\n\ + [--inactivity-probe=MSECS]\n\ + set-connection TARGET... set the list of connections to TARGET...\n\ +\n\ +SSL commands:\n\ + get-ssl print the SSL configuration\n\ + del-ssl delete the SSL configuration\n\ + set-ssl PRIV-KEY CERT CA-CERT [SSL-PROTOS [SSL-CIPHERS]] \ +set the SSL configuration\n\ +\n\ +%s\ +%s\ +\n\ +Options:\n\ + --db=DATABASE connect to DATABASE\n\ + (default: %s)\n\ + --no-leader-only accept any cluster member, not just the leader\n\ + -t, --timeout=SECS wait at most SECS seconds\n\ + --dry-run do not commit changes to database\n\ + --oneline print exactly one line of output per command\n", + program_name, program_name, ctl_get_db_cmd_usage(), + ctl_list_db_tables_usage(), default_ic_nb_db()); + table_usage(); + vlog_usage(); + printf("\ + --no-syslog equivalent to --verbose=ic_nbctl:syslog:warn\n"); + printf("\n\ +Other options:\n\ + -h, --help display this help message\n\ + -V, --version display version information\n"); + stream_usage("database", true, true, true); + exit(EXIT_SUCCESS); +} + + +/* ovs-ic_nbctl specific context. Inherits the 'struct ctl_context' as base. + * Now empty, just keep the framework for future additions. */ +struct ic_nbctl_context { + struct ctl_context base; + + /* A cache of the contents of the database. + * + * A command that needs to use any of this information must first call + * ic_nbctl_context_populate_cache(). A command that changes anything that + * could invalidate the cache must either call + * ic_nbctl_context_invalidate_cache() or manually update the cache to + * maintain its correctness. */ + bool cache_valid; +}; + +static struct cmd_show_table cmd_show_tables[] = { + {&icnbrec_table_transit_switch, + &icnbrec_transit_switch_col_name, + {NULL}, + {NULL, NULL, NULL}}, + + {NULL, NULL, {NULL, NULL, NULL}, {NULL, NULL, NULL}}, +}; + +static void +ic_nbctl_init(struct ctl_context *ctx OVS_UNUSED) +{ +} + +static void +ic_nbctl_ts_add(struct ctl_context *ctx) +{ + const char *ts_name = ctx->argv[1]; + bool may_exist = shash_find(&ctx->options, "--may-exist") != NULL; + + const struct icnbrec_transit_switch *ts; + ICNBREC_TRANSIT_SWITCH_FOR_EACH (ts, ctx->idl) { + if (!strcmp(ts->name, ts_name)) { + if (may_exist) { + return; + } + ctl_error(ctx, "%s: a transit switch with this name already " + "exists", ts_name); + return; + } + } + + ts = icnbrec_transit_switch_insert(ctx->txn); + icnbrec_transit_switch_set_name(ts, ts_name); +} + +static char * +ts_by_name_or_uuid(struct ctl_context *ctx, const char *id, bool must_exist, + const struct icnbrec_transit_switch **ts_p) +{ + const struct icnbrec_transit_switch *ts = NULL; + *ts_p = NULL; + + struct uuid ts_uuid; + bool is_uuid = uuid_from_string(&ts_uuid, id); + if (is_uuid) { + ts = icnbrec_transit_switch_get_for_uuid(ctx->idl, &ts_uuid); + } + + if (!ts) { + const struct icnbrec_transit_switch *iter; + + ICNBREC_TRANSIT_SWITCH_FOR_EACH (iter, ctx->idl) { + if (!strcmp(iter->name, id)) { + ts = iter; + break; + } + } + } + + if (!ts && must_exist) { + return xasprintf("%s: switch %s not found", + id, is_uuid ? "UUID" : "name"); + } + + *ts_p = ts; + return NULL; +} + +static void +ic_nbctl_ts_del(struct ctl_context *ctx) +{ + bool must_exist = !shash_find(&ctx->options, "--if-exists"); + const char *id = ctx->argv[1]; + const struct icnbrec_transit_switch *ts = NULL; + + char *error = ts_by_name_or_uuid(ctx, id, must_exist, &ts); + if (error) { + ctx->error = error; + return; + } + if (!ts) { + return; + } + + icnbrec_transit_switch_delete(ts); +} + +static void +ic_nbctl_ts_list(struct ctl_context *ctx) +{ + const struct icnbrec_transit_switch *ts; + struct smap switches; + + smap_init(&switches); + ICNBREC_TRANSIT_SWITCH_FOR_EACH (ts, ctx->idl) { + smap_add_format(&switches, ts->name, UUID_FMT " (%s)", + UUID_ARGS(&ts->header_.uuid), ts->name); + } + const struct smap_node **nodes = smap_sort(&switches); + for (size_t i = 0; i < smap_count(&switches); i++) { + const struct smap_node *node = nodes[i]; + ds_put_format(&ctx->output, "%s\n", node->value); + } + smap_destroy(&switches); + free(nodes); +} +static void +verify_connections(struct ctl_context *ctx) +{ + const struct icnbrec_ic_nb_global *ic_nb_global = + icnbrec_ic_nb_global_first(ctx->idl); + const struct icnbrec_connection *conn; + + icnbrec_ic_nb_global_verify_connections(ic_nb_global); + + ICNBREC_CONNECTION_FOR_EACH (conn, ctx->idl) { + icnbrec_connection_verify_target(conn); + } +} + +static void +pre_connection(struct ctl_context *ctx) +{ + ovsdb_idl_add_column(ctx->idl, &icnbrec_ic_nb_global_col_connections); + ovsdb_idl_add_column(ctx->idl, &icnbrec_connection_col_target); + ovsdb_idl_add_column(ctx->idl, &icnbrec_connection_col_inactivity_probe); +} + +static void +cmd_get_connection(struct ctl_context *ctx) +{ + const struct icnbrec_connection *conn; + struct svec targets; + size_t i; + + verify_connections(ctx); + + /* Print the targets in sorted order for reproducibility. */ + svec_init(&targets); + + ICNBREC_CONNECTION_FOR_EACH (conn, ctx->idl) { + svec_add(&targets, conn->target); + } + + svec_sort_unique(&targets); + for (i = 0; i < targets.n; i++) { + ds_put_format(&ctx->output, "%s\n", targets.names[i]); + } + svec_destroy(&targets); +} + +static void +delete_connections(struct ctl_context *ctx) +{ + const struct icnbrec_ic_nb_global *ic_nb_global = + icnbrec_ic_nb_global_first(ctx->idl); + const struct icnbrec_connection *conn, *next; + + /* Delete Manager rows pointed to by 'connection_options' column. */ + ICNBREC_CONNECTION_FOR_EACH_SAFE (conn, next, ctx->idl) { + icnbrec_connection_delete(conn); + } + + /* Delete 'Manager' row refs in 'manager_options' column. */ + icnbrec_ic_nb_global_set_connections(ic_nb_global, NULL, 0); +} + +static void +cmd_del_connection(struct ctl_context *ctx) +{ + verify_connections(ctx); + delete_connections(ctx); +} + +static void +insert_connections(struct ctl_context *ctx, char *targets[], size_t n) +{ + const struct icnbrec_ic_nb_global *ic_nb_global = + icnbrec_ic_nb_global_first(ctx->idl); + struct icnbrec_connection **connections; + size_t i, conns = 0; + const char *inactivity_probe = shash_find_data(&ctx->options, + "--inactivity-probe"); + + /* Insert each connection in a new row in Connection table. */ + connections = xmalloc(n * sizeof *connections); + for (i = 0; i < n; i++) { + if (stream_verify_name(targets[i]) && + pstream_verify_name(targets[i])) { + VLOG_WARN("target type \"%s\" is possibly erroneous", targets[i]); + } + + connections[conns] = icnbrec_connection_insert(ctx->txn); + icnbrec_connection_set_target(connections[conns], targets[i]); + if (inactivity_probe) { + int64_t msecs = atoll(inactivity_probe); + icnbrec_connection_set_inactivity_probe(connections[conns], + &msecs, 1); + } + conns++; + } + + /* Store uuids of new connection rows in 'connection' column. */ + icnbrec_ic_nb_global_set_connections(ic_nb_global, connections, conns); + free(connections); +} + +static void +cmd_set_connection(struct ctl_context *ctx) +{ + const size_t n = ctx->argc - 1; + + verify_connections(ctx); + delete_connections(ctx); + insert_connections(ctx, &ctx->argv[1], n); +} + +static void +pre_cmd_get_ssl(struct ctl_context *ctx) +{ + ovsdb_idl_add_column(ctx->idl, &icnbrec_ic_nb_global_col_ssl); + + ovsdb_idl_add_column(ctx->idl, &icnbrec_ssl_col_private_key); + ovsdb_idl_add_column(ctx->idl, &icnbrec_ssl_col_certificate); + ovsdb_idl_add_column(ctx->idl, &icnbrec_ssl_col_ca_cert); + ovsdb_idl_add_column(ctx->idl, &icnbrec_ssl_col_bootstrap_ca_cert); +} + +static void +cmd_get_ssl(struct ctl_context *ctx) +{ + const struct icnbrec_ic_nb_global *ic_nb_global = + icnbrec_ic_nb_global_first(ctx->idl); + const struct icnbrec_ssl *ssl = icnbrec_ssl_first(ctx->idl); + + icnbrec_ic_nb_global_verify_ssl(ic_nb_global); + if (ssl) { + icnbrec_ssl_verify_private_key(ssl); + icnbrec_ssl_verify_certificate(ssl); + icnbrec_ssl_verify_ca_cert(ssl); + icnbrec_ssl_verify_bootstrap_ca_cert(ssl); + + ds_put_format(&ctx->output, "Private key: %s\n", ssl->private_key); + ds_put_format(&ctx->output, "Certificate: %s\n", ssl->certificate); + ds_put_format(&ctx->output, "CA Certificate: %s\n", ssl->ca_cert); + ds_put_format(&ctx->output, "Bootstrap: %s\n", + ssl->bootstrap_ca_cert ? "true" : "false"); + } +} + +static void +pre_cmd_del_ssl(struct ctl_context *ctx) +{ + ovsdb_idl_add_column(ctx->idl, &icnbrec_ic_nb_global_col_ssl); +} + +static void +cmd_del_ssl(struct ctl_context *ctx) +{ + const struct icnbrec_ic_nb_global *ic_nb_global = + icnbrec_ic_nb_global_first(ctx->idl); + const struct icnbrec_ssl *ssl = icnbrec_ssl_first(ctx->idl); + + if (ssl) { + icnbrec_ic_nb_global_verify_ssl(ic_nb_global); + icnbrec_ssl_delete(ssl); + icnbrec_ic_nb_global_set_ssl(ic_nb_global, NULL); + } +} + +static void +pre_cmd_set_ssl(struct ctl_context *ctx) +{ + ovsdb_idl_add_column(ctx->idl, &icnbrec_ic_nb_global_col_ssl); +} + +static void +cmd_set_ssl(struct ctl_context *ctx) +{ + bool bootstrap = shash_find(&ctx->options, "--bootstrap"); + const struct icnbrec_ic_nb_global *ic_nb_global = + icnbrec_ic_nb_global_first(ctx->idl); + const struct icnbrec_ssl *ssl = icnbrec_ssl_first(ctx->idl); + + icnbrec_ic_nb_global_verify_ssl(ic_nb_global); + if (ssl) { + icnbrec_ssl_delete(ssl); + } + ssl = icnbrec_ssl_insert(ctx->txn); + + icnbrec_ssl_set_private_key(ssl, ctx->argv[1]); + icnbrec_ssl_set_certificate(ssl, ctx->argv[2]); + icnbrec_ssl_set_ca_cert(ssl, ctx->argv[3]); + + icnbrec_ssl_set_bootstrap_ca_cert(ssl, bootstrap); + + if (ctx->argc == 5) { + icnbrec_ssl_set_ssl_protocols(ssl, ctx->argv[4]); + } else if (ctx->argc == 6) { + icnbrec_ssl_set_ssl_protocols(ssl, ctx->argv[4]); + icnbrec_ssl_set_ssl_ciphers(ssl, ctx->argv[5]); + } + + icnbrec_ic_nb_global_set_ssl(ic_nb_global, ssl); +} + + +static const struct ctl_table_class tables[ICNBREC_N_TABLES] = { + [ICNBREC_TABLE_TRANSIT_SWITCH].row_ids[0] = + {&icnbrec_transit_switch_col_name, NULL, NULL}, +}; + + +static void +ic_nbctl_context_init_command(struct ic_nbctl_context *ic_nbctl_ctx, + struct ctl_command *command) +{ + ctl_context_init_command(&ic_nbctl_ctx->base, command); +} + +static void +ic_nbctl_context_init(struct ic_nbctl_context *ic_nbctl_ctx, + struct ctl_command *command, struct ovsdb_idl *idl, + struct ovsdb_idl_txn *txn, + struct ovsdb_symbol_table *symtab) +{ + ctl_context_init(&ic_nbctl_ctx->base, command, idl, txn, symtab, + NULL); + ic_nbctl_ctx->cache_valid = false; +} + +static void +ic_nbctl_context_done_command(struct ic_nbctl_context *ic_nbctl_ctx, + struct ctl_command *command) +{ + ctl_context_done_command(&ic_nbctl_ctx->base, command); +} + +static void +ic_nbctl_context_done(struct ic_nbctl_context *ic_nbctl_ctx, + struct ctl_command *command) +{ + ctl_context_done(&ic_nbctl_ctx->base, command); +} + +static void +run_prerequisites(struct ctl_command *commands, size_t n_commands, + struct ovsdb_idl *idl) +{ + ovsdb_idl_add_table(idl, &icnbrec_table_ic_nb_global); + + for (struct ctl_command *c = commands; c < &commands[n_commands]; c++) { + if (c->syntax->prerequisites) { + struct ic_nbctl_context ic_nbctl_ctx; + + ds_init(&c->output); + c->table = NULL; + + ic_nbctl_context_init(&ic_nbctl_ctx, c, idl, NULL, NULL); + (c->syntax->prerequisites)(&ic_nbctl_ctx.base); + if (ic_nbctl_ctx.base.error) { + ctl_fatal("%s", ic_nbctl_ctx.base.error); + } + ic_nbctl_context_done(&ic_nbctl_ctx, c); + + ovs_assert(!c->output.string); + ovs_assert(!c->table); + } + } +} + +static bool +do_ic_nbctl(const char *args, struct ctl_command *commands, size_t n_commands, + struct ovsdb_idl *idl) +{ + struct ovsdb_idl_txn *txn; + enum ovsdb_idl_txn_status status; + struct ovsdb_symbol_table *symtab; + struct ic_nbctl_context ic_nbctl_ctx; + struct ctl_command *c; + struct shash_node *node; + + txn = the_idl_txn = ovsdb_idl_txn_create(idl); + if (dry_run) { + ovsdb_idl_txn_set_dry_run(txn); + } + + ovsdb_idl_txn_add_comment(txn, "ovs-ic_nbctl: %s", args); + + const struct icnbrec_ic_nb_global *inb = icnbrec_ic_nb_global_first(idl); + if (!inb) { + /* XXX add verification that table is empty */ + icnbrec_ic_nb_global_insert(txn); + } + + symtab = ovsdb_symbol_table_create(); + for (c = commands; c < &commands[n_commands]; c++) { + ds_init(&c->output); + c->table = NULL; + } + ic_nbctl_context_init(&ic_nbctl_ctx, NULL, idl, txn, symtab); + for (c = commands; c < &commands[n_commands]; c++) { + ic_nbctl_context_init_command(&ic_nbctl_ctx, c); + if (c->syntax->run) { + (c->syntax->run)(&ic_nbctl_ctx.base); + } + if (ic_nbctl_ctx.base.error) { + ctl_fatal("%s", ic_nbctl_ctx.base.error); + } + ic_nbctl_context_done_command(&ic_nbctl_ctx, c); + + if (ic_nbctl_ctx.base.try_again) { + ic_nbctl_context_done(&ic_nbctl_ctx, NULL); + goto try_again; + } + } + ic_nbctl_context_done(&ic_nbctl_ctx, NULL); + + SHASH_FOR_EACH (node, &symtab->sh) { + struct ovsdb_symbol *symbol = node->data; + if (!symbol->created) { + ctl_fatal("row id \"%s\" is referenced but never created (e.g. " + "with \"-- --id=%s create ...\")", + node->name, node->name); + } + if (!symbol->strong_ref) { + if (!symbol->weak_ref) { + VLOG_WARN("row id \"%s\" was created but no reference to it " + "was inserted, so it will not actually appear in " + "the database", node->name); + } else { + VLOG_WARN("row id \"%s\" was created but only a weak " + "reference to it was inserted, so it will not " + "actually appear in the database", node->name); + } + } + } + + status = ovsdb_idl_txn_commit_block(txn); + if (status == TXN_UNCHANGED || status == TXN_SUCCESS) { + for (c = commands; c < &commands[n_commands]; c++) { + if (c->syntax->postprocess) { + ic_nbctl_context_init(&ic_nbctl_ctx, c, idl, txn, symtab); + (c->syntax->postprocess)(&ic_nbctl_ctx.base); + if (ic_nbctl_ctx.base.error) { + ctl_fatal("%s", ic_nbctl_ctx.base.error); + } + ic_nbctl_context_done(&ic_nbctl_ctx, c); + } + } + } + + switch (status) { + case TXN_UNCOMMITTED: + case TXN_INCOMPLETE: + OVS_NOT_REACHED(); + + case TXN_ABORTED: + /* Should not happen--we never call ovsdb_idl_txn_abort(). */ + ctl_fatal("transaction aborted"); + + case TXN_UNCHANGED: + case TXN_SUCCESS: + break; + + case TXN_TRY_AGAIN: + goto try_again; + + case TXN_ERROR: + ctl_fatal("transaction error: %s", ovsdb_idl_txn_get_error(txn)); + + case TXN_NOT_LOCKED: + /* Should not happen--we never call ovsdb_idl_set_lock(). */ + ctl_fatal("database not locked"); + + default: + OVS_NOT_REACHED(); + } + + ovsdb_symbol_table_destroy(symtab); + + for (c = commands; c < &commands[n_commands]; c++) { + struct ds *ds = &c->output; + + if (c->table) { + table_print(c->table, &table_style); + } else if (oneline) { + size_t j; + + ds_chomp(ds, '\n'); + for (j = 0; j < ds->length; j++) { + int ch = ds->string[j]; + switch (ch) { + case '\n': + fputs("\\n", stdout); + break; + + case '\\': + fputs("\\\\", stdout); + break; + + default: + putchar(ch); + } + } + putchar('\n'); + } else { + fputs(ds_cstr(ds), stdout); + } + ds_destroy(&c->output); + table_destroy(c->table); + free(c->table); + + shash_destroy_free_data(&c->options); + } + free(commands); + ovsdb_idl_txn_destroy(txn); + ovsdb_idl_destroy(idl); + + return true; + +try_again: + /* Our transaction needs to be rerun, or a prerequisite was not met. Free + * resources and return so that the caller can try again. */ + ovsdb_idl_txn_abort(txn); + ovsdb_idl_txn_destroy(txn); + the_idl_txn = NULL; + + ovsdb_symbol_table_destroy(symtab); + for (c = commands; c < &commands[n_commands]; c++) { + ds_destroy(&c->output); + table_destroy(c->table); + free(c->table); + } + return false; +} + +/* Frees the current transaction and the underlying IDL and then calls + * exit(status). + * + * Freeing the transaction and the IDL is not strictly necessary, but it makes + * for a clean memory leak report from valgrind in the normal case. That makes + * it easier to notice real memory leaks. */ +static void +ic_nbctl_exit(int status) +{ + if (the_idl_txn) { + ovsdb_idl_txn_abort(the_idl_txn); + ovsdb_idl_txn_destroy(the_idl_txn); + } + ovsdb_idl_destroy(the_idl); + exit(status); +} + +static const struct ctl_command_syntax ic_nbctl_commands[] = { + { "init", 0, 0, "", NULL, ic_nbctl_init, NULL, "", RW }, + + /* transit switch commands. */ + { "ts-add", 1, 1, "SWITCH", NULL, ic_nbctl_ts_add, NULL, "--may-exist", RW }, + { "ts-del", 1, 1, "SWITCH", NULL, ic_nbctl_ts_del, NULL, "--if-exists", RW }, + { "ts-list", 0, 0, "", NULL, ic_nbctl_ts_list, NULL, "", RO }, + + /* Connection commands. */ + {"get-connection", 0, 0, "", pre_connection, cmd_get_connection, NULL, "", + RO}, + {"del-connection", 0, 0, "", pre_connection, cmd_del_connection, NULL, "", + RW}, + {"set-connection", 1, INT_MAX, "TARGET...", pre_connection, + cmd_set_connection, NULL, "--inactivity-probe=", RW}, + + /* SSL commands. */ + {"get-ssl", 0, 0, "", pre_cmd_get_ssl, cmd_get_ssl, NULL, "", RO}, + {"del-ssl", 0, 0, "", pre_cmd_del_ssl, cmd_del_ssl, NULL, "", RW}, + {"set-ssl", 3, 5, + "PRIVATE-KEY CERTIFICATE CA-CERT [SSL-PROTOS [SSL-CIPHERS]]", + pre_cmd_set_ssl, cmd_set_ssl, NULL, "--bootstrap", RW}, + + {NULL, 0, 0, NULL, NULL, NULL, NULL, NULL, RO}, +}; + +/* Registers ic_nbctl and common db commands. */ +static void +ic_nbctl_cmd_init(void) +{ + ctl_init(&icnbrec_idl_class, icnbrec_table_classes, tables, + cmd_show_tables, ic_nbctl_exit); + ctl_register_commands(ic_nbctl_commands); +} From patchwork Wed Jan 29 19:55:58 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Han Zhou X-Patchwork-Id: 1231086 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.136; helo=silver.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=ovn.org Received: from silver.osuosl.org (smtp3.osuosl.org [140.211.166.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 487DmK35qdz9sNT for ; Thu, 30 Jan 2020 06:57:37 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by silver.osuosl.org (Postfix) with ESMTP id D3C26227E1; Wed, 29 Jan 2020 19:57:35 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from silver.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Bnczl2mTF2fR; Wed, 29 Jan 2020 19:56:57 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by silver.osuosl.org (Postfix) with ESMTP id EEFA6221DC; Wed, 29 Jan 2020 19:56:43 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id D0FAEC0881; Wed, 29 Jan 2020 19:56:43 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by lists.linuxfoundation.org (Postfix) with ESMTP id 0D3E3C0171 for ; Wed, 29 Jan 2020 19:56:42 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id 0709088299 for ; Wed, 29 Jan 2020 19:56:42 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id l3z5kfTkk49s for ; Wed, 29 Jan 2020 19:56:30 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from relay1-d.mail.gandi.net (relay1-d.mail.gandi.net [217.70.183.193]) by hemlock.osuosl.org (Postfix) with ESMTPS id 7910887082 for ; Wed, 29 Jan 2020 19:56:29 +0000 (UTC) X-Originating-IP: 216.113.160.77 Received: from localhost.localdomain.localdomain (unknown [216.113.160.77]) (Authenticated sender: hzhou@ovn.org) by relay1-d.mail.gandi.net (Postfix) with ESMTPSA id 45B56240009; Wed, 29 Jan 2020 19:56:26 +0000 (UTC) From: Han Zhou To: dev@openvswitch.org Date: Wed, 29 Jan 2020 11:55:58 -0800 Message-Id: <1580327768-36501-4-git-send-email-hzhou@ovn.org> X-Mailer: git-send-email 2.1.0 In-Reply-To: <1580327768-36501-1-git-send-email-hzhou@ovn.org> References: <1580327768-36501-1-git-send-email-hzhou@ovn.org> Cc: Han Zhou Subject: [ovs-dev] [PATCH ovn v4 03/13] ovn-ic-sb: Interconnection southbound DB schema and CLI. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" This patch introduces OVN_IC_Southbound DB schema and the CLI ovn-ic-sbctl that manages the DB. Signed-off-by: Han Zhou --- .gitignore | 3 + automake.mk | 36 ++ debian/ovn-common.install | 1 + debian/ovn-common.manpages | 2 + lib/.gitignore | 3 + lib/automake.mk | 17 +- lib/ovn-ic-sb-idl.ann | 9 + lib/ovn-util.c | 13 + lib/ovn-util.h | 1 + ovn-ic-sb.ovsschema | 129 ++++++ ovn-ic-sb.xml | 582 ++++++++++++++++++++++++ tests/automake.mk | 2 + tests/ovn-ic-sbctl.at | 112 +++++ tests/testsuite.at | 1 + utilities/.gitignore | 2 + utilities/automake.mk | 8 + utilities/ovn-ic-sbctl.8.xml | 148 ++++++ utilities/ovn-ic-sbctl.c | 1017 ++++++++++++++++++++++++++++++++++++++++++ 18 files changed, 2085 insertions(+), 1 deletion(-) create mode 100644 lib/ovn-ic-sb-idl.ann create mode 100644 ovn-ic-sb.ovsschema create mode 100644 ovn-ic-sb.xml create mode 100644 tests/ovn-ic-sbctl.at create mode 100644 utilities/ovn-ic-sbctl.8.xml create mode 100644 utilities/ovn-ic-sbctl.c diff --git a/.gitignore b/.gitignore index d4f8c10..7ca9b38 100644 --- a/.gitignore +++ b/.gitignore @@ -70,6 +70,9 @@ /ovn-ic-nb.5 /ovn-ic-nb.gv /ovn-ic-nb.pic +/ovn-ic-sb.5 +/ovn-ic-sb.gv +/ovn-ic-sb.pic /package.m4 /stamp-h1 /_build-gcc diff --git a/automake.mk b/automake.mk index 59063c2..2b0bb4b 100644 --- a/automake.mk +++ b/automake.mk @@ -91,6 +91,35 @@ ovn-ic-nb.5: \ $(srcdir)/ovn-ic-nb.xml > $@.tmp && \ mv $@.tmp $@ +# OVN interconnection southbound E-R diagram +# +# If "python" or "dot" is not available, then we do not add graphical diagram +# to the documentation. +if HAVE_DOT +ovn-ic-sb.gv: ${OVSDIR}/ovsdb/ovsdb-dot.in $(srcdir)/ovn-ic-sb.ovsschema + $(AM_V_GEN)$(OVSDB_DOT) --no-arrows $(srcdir)/ovn-ic-sb.ovsschema > $@ +ovn-ic-sb.pic: ovn-ic-sb.gv ${OVSDIR}/ovsdb/dot2pic + $(AM_V_GEN)(dot -T plain < ovn-ic-sb.gv | $(PYTHON) ${OVSDIR}/ovsdb/dot2pic -f 3) > $@.tmp && \ + mv $@.tmp $@ +OVN_IC_SB_PIC = ovn-ic-sb.pic +OVN_IC_SB_DOT_DIAGRAM_ARG = --er-diagram=$(OVN_IC_SB_PIC) +CLEANFILES += ovn-ic-sb.gv ovn-ic-sb.pic +endif + +# OVN interconnection southbound schema documentation +EXTRA_DIST += ovn-ic-sb.xml +CLEANFILES += ovn-ic-sb.5 +man_MANS += ovn-ic-sb.5 + +ovn-ic-sb.5: \ + ${OVSDIR}/ovsdb/ovsdb-doc $(srcdir)/ovn-ic-sb.xml $(srcdir)/ovn-ic-sb.ovsschema $(OVN_IC_SB_PIC) + $(AM_V_GEN)$(OVSDB_DOC) \ + $(OVN_IC_SB_DOT_DIAGRAM_ARG) \ + --version=$(VERSION) \ + $(srcdir)/ovn-ic-sb.ovsschema \ + $(srcdir)/ovn-ic-sb.xml > $@.tmp && \ + mv $@.tmp $@ + # Version checking for ovn-nb.ovsschema. ALL_LOCAL += ovn-nb.ovsschema.stamp ovn-nb.ovsschema.stamp: ovn-nb.ovsschema @@ -108,8 +137,15 @@ ovn-ic-nb.ovsschema.stamp: ovn-ic-nb.ovsschema $(srcdir)/build-aux/cksum-schema-check $? $@ CLEANFILES += ovn-ic-nb.ovsschema.stamp +# Version checking for ovn-ic-sb.ovsschema. +ALL_LOCAL += ovn-ic-sb.ovsschema.stamp +ovn-ic-sb.ovsschema.stamp: ovn-ic-sb.ovsschema + $(srcdir)/build-aux/cksum-schema-check $? $@ +CLEANFILES += ovn-ic-sb.ovsschema.stamp + pkgdata_DATA += ovn-nb.ovsschema pkgdata_DATA += ovn-sb.ovsschema pkgdata_DATA += ovn-ic-nb.ovsschema +pkgdata_DATA += ovn-ic-sb.ovsschema CLEANFILES += ovn-sb.ovsschema.stamp diff --git a/debian/ovn-common.install b/debian/ovn-common.install index 59b8018..e3c3c00 100644 --- a/debian/ovn-common.install +++ b/debian/ovn-common.install @@ -1,6 +1,7 @@ usr/bin/ovn-nbctl usr/bin/ovn-sbctl usr/bin/ovn-ic-nbctl +usr/bin/ovn-ic-sbctl usr/bin/ovn-trace usr/bin/ovn-detrace usr/share/openvswitch/scripts/ovn-ctl diff --git a/debian/ovn-common.manpages b/debian/ovn-common.manpages index e7d3e4d..ba0fe8a 100644 --- a/debian/ovn-common.manpages +++ b/debian/ovn-common.manpages @@ -2,9 +2,11 @@ ovn/ovn-architecture.7 ovn/ovn-nb.5 ovn/ovn-sb.5 ovn/ovn-ic-nb.5 +ovn/ovn-ic-sb.5 ovn/utilities/ovn-ctl.8 ovn/utilities/ovn-nbctl.8 ovn/utilities/ovn-sbctl.8 ovn/utilities/ovn-ic-nbctl.8 +ovn/utilities/ovn-ic-sbctl.8 ovn/utilities/ovn-trace.8 ovn/utilities/ovn-detrace.1 diff --git a/lib/.gitignore b/lib/.gitignore index 3af2923..7f67f1d 100644 --- a/lib/.gitignore +++ b/lib/.gitignore @@ -8,4 +8,7 @@ /ovn-ic-nb-idl.c /ovn-ic-nb-idl.h /ovn-ic-nb-idl.ovsidl +/ovn-ic-sb-idl.c +/ovn-ic-sb-idl.h +/ovn-ic-sb-idl.ovsidl /ovn-dirs.c diff --git a/lib/automake.mk b/lib/automake.mk index 5f6561a..f3e9c88 100644 --- a/lib/automake.mk +++ b/lib/automake.mk @@ -31,7 +31,9 @@ nodist_lib_libovn_la_SOURCES = \ lib/ovn-sb-idl.c \ lib/ovn-sb-idl.h \ lib/ovn-ic-nb-idl.c \ - lib/ovn-ic-nb-idl.h + lib/ovn-ic-nb-idl.h \ + lib/ovn-ic-sb-idl.c \ + lib/ovn-ic-sb-idl.h CLEANFILES += $(nodist_lib_libovn_la_SOURCES) @@ -88,3 +90,16 @@ OVN_IC_NB_IDL_FILES = \ lib/ovn-ic-nb-idl.ovsidl: $(OVN_IC_NB_IDL_FILES) $(AM_V_GEN)$(OVSDB_IDLC) annotate $(OVN_IC_NB_IDL_FILES) > $@.tmp && \ mv $@.tmp $@ + +# ovn-ic-sb IDL +OVSIDL_BUILT += \ + lib/ovn-ic-sb-idl.c \ + lib/ovn-ic-sb-idl.h \ + lib/ovn-ic-sb-idl.ovsidl +EXTRA_DIST += lib/ovn-ic-sb-idl.ann +OVN_IC_SB_IDL_FILES = \ + $(srcdir)/ovn-ic-sb.ovsschema \ + $(srcdir)/lib/ovn-ic-sb-idl.ann +lib/ovn-ic-sb-idl.ovsidl: $(OVN_IC_SB_IDL_FILES) + $(AM_V_GEN)$(OVSDB_IDLC) annotate $(OVN_IC_SB_IDL_FILES) > $@.tmp && \ + mv $@.tmp $@ diff --git a/lib/ovn-ic-sb-idl.ann b/lib/ovn-ic-sb-idl.ann new file mode 100644 index 0000000..def68dd --- /dev/null +++ b/lib/ovn-ic-sb-idl.ann @@ -0,0 +1,9 @@ +# -*- python -*- + +# This code, when invoked by "ovsdb-idlc annotate" (by the build +# process), annotates ovn-ic-sb.ovsschema with additional data that give +# the ovsdb-idl engine information about the types involved, so that +# it can generate more programmer-friendly data structures. + +s["idlPrefix"] = "icsbrec_" +s["idlHeader"] = "\"lib/ovn-ic-sb-idl.h\"" diff --git a/lib/ovn-util.c b/lib/ovn-util.c index 5b891ec..18c13a8 100644 --- a/lib/ovn-util.c +++ b/lib/ovn-util.c @@ -363,6 +363,19 @@ default_ic_nb_db(void) return def; } +const char * +default_ic_sb_db(void) +{ + static char *def; + if (!def) { + def = getenv("OVN_IC_SB_DB"); + if (!def) { + def = xasprintf("unix:%s/ovn_ic_sb_db.sock", ovn_rundir()); + } + } + return def; +} + char * get_abs_unix_ctl_path(void) { diff --git a/lib/ovn-util.h b/lib/ovn-util.h index cda245b..5685f43 100644 --- a/lib/ovn-util.h +++ b/lib/ovn-util.h @@ -74,6 +74,7 @@ char *alloc_nat_zone_key(const struct uuid *key, const char *type); const char *default_nb_db(void); const char *default_sb_db(void); const char *default_ic_nb_db(void); +const char *default_ic_sb_db(void); char *get_abs_unix_ctl_path(void); struct ovsdb_idl_table_class; diff --git a/ovn-ic-sb.ovsschema b/ovn-ic-sb.ovsschema new file mode 100644 index 0000000..819c28a --- /dev/null +++ b/ovn-ic-sb.ovsschema @@ -0,0 +1,129 @@ +{ + "name": "OVN_IC_Southbound", + "version": "1.0.0", + "cksum": "1702378250 6056", + "tables": { + "IC_SB_Global": { + "columns": { + "external_ids": { + "type": {"key": "string", "value": "string", + "min": 0, "max": "unlimited"}}, + "connections": { + "type": {"key": {"type": "uuid", + "refTable": "Connection"}, + "min": 0, + "max": "unlimited"}}, + "ssl": { + "type": {"key": {"type": "uuid", + "refTable": "SSL"}, + "min": 0, "max": 1}}, + "options": { + "type": {"key": "string", "value": "string", + "min": 0, "max": "unlimited"}}}, + "maxRows": 1, + "isRoot": true}, + "Availability_Zone": { + "columns": { + "name": {"type": "string"}}, + "isRoot": true, + "indexes": [["name"]]}, + "Gateway": { + "columns": { + "name": {"type": "string"}, + "availability_zone": {"type": {"key": {"type": "uuid", + "refTable": "Availability_Zone"}}}, + "hostname": {"type": "string"}, + "encaps": {"type": {"key": {"type": "uuid", + "refTable": "Encap"}, + "min": 1, "max": "unlimited"}}, + "external_ids": { + "type": {"key": "string", "value": "string", + "min": 0, "max": "unlimited"}}}, + "isRoot": true, + "indexes": [["name"]]}, + "Encap": { + "columns": { + "type": {"type": {"key": { + "type": "string", + "enum": ["set", ["geneve", "stt", "vxlan"]]}}}, + "options": {"type": {"key": "string", + "value": "string", + "min": 0, + "max": "unlimited"}}, + "ip": {"type": "string"}, + "gateway_name": {"type": "string"}}, + "indexes": [["type", "ip"]]}, + "Datapath_Binding": { + "columns": { + "transit_switch": {"type": "string"}, + "tunnel_key": { + "type": {"key": {"type": "integer", + "minInteger": 1, + "maxInteger": 16777215}}}, + "external_ids": { + "type": {"key": "string", "value": "string", + "min": 0, "max": "unlimited"}}}, + "indexes": [["tunnel_key"]], + "isRoot": true}, + "Port_Binding": { + "columns": { + "logical_port": {"type": "string"}, + "transit_switch": {"type": "string"}, + "availability_zone": {"type": {"key": {"type": "uuid", + "refTable": "Availability_Zone"}}}, + "tunnel_key": { + "type": {"key": {"type": "integer", + "minInteger": 1, + "maxInteger": 32767}}}, + "gateway": {"type": "string"}, + "encap": {"type": {"key": {"type": "uuid", + "refTable": "Encap", + "refType": "weak"}, + "min": 0, "max": 1}}, + "address": {"type": "string"}, + "external_ids": {"type": {"key": "string", + "value": "string", + "min": 0, + "max": "unlimited"}}}, + "indexes": [["transit_switch", "tunnel_key"], ["logical_port"]], + "isRoot": true}, + "Connection": { + "columns": { + "target": {"type": "string"}, + "max_backoff": {"type": {"key": {"type": "integer", + "minInteger": 1000}, + "min": 0, + "max": 1}}, + "inactivity_probe": {"type": {"key": "integer", + "min": 0, + "max": 1}}, + "other_config": {"type": {"key": "string", + "value": "string", + "min": 0, + "max": "unlimited"}}, + "external_ids": {"type": {"key": "string", + "value": "string", + "min": 0, + "max": "unlimited"}}, + "is_connected": {"type": "boolean", "ephemeral": true}, + "status": {"type": {"key": "string", + "value": "string", + "min": 0, + "max": "unlimited"}, + "ephemeral": true}}, + "indexes": [["target"]]}, + "SSL": { + "columns": { + "private_key": {"type": "string"}, + "certificate": {"type": "string"}, + "ca_cert": {"type": "string"}, + "bootstrap_ca_cert": {"type": "boolean"}, + "ssl_protocols": {"type": "string"}, + "ssl_ciphers": {"type": "string"}, + "external_ids": {"type": {"key": "string", + "value": "string", + "min": 0, + "max": "unlimited"}}}, + "maxRows": 1} + } +} diff --git a/ovn-ic-sb.xml b/ovn-ic-sb.xml new file mode 100644 index 0000000..dad6405 --- /dev/null +++ b/ovn-ic-sb.xml @@ -0,0 +1,582 @@ + + +

+ This database holds configuration and state for interconnecting different + OVN deployments. The content of the database is populated and used by the + ovn-ic program in each OVN deployment, and not supposed to be + directly used by CMS or end user. +

+ +

+ The OVN Interconnection Southbound database is shared by + ovn-ic program in each OVN deployment. It contains + interconnection information from all related OVN deployments, and is used + as the intermediate store for each OVN deployment to exchange the + information. The ovn-ic program in each deployment is + responsible for syncing the data between this database and the its own + northbound and southbound databases. +

+ +

Database Structure

+ +

+ The OVN Interconnection Southbound database contains classes of data with + different properties, as described in the sections below. +

+ +

Availability Zone Specific Information

+ +

+ These tables contain objects that are availability zone specific. Each object + is owned and populated by one availability zone, and read by other availability + zones. +

+ +

+ The , , and tables are the availability + zone specific tables. +

+ +

Global Information

+ +

+ The data that does not belong to any specific availability zone but is + common for all availability zones. +

+ +

+ The table contains the common datapath + binding information. +

+ +

Common Columns

+

+ Each of the tables in this database contains a special column, named + external_ids. This column has the same form and purpose each + place it appears. +

+ +
+
external_ids: map of string-string pairs
+
+ Key-value pairs for use by ovn-ic. +
+
+ +

+ Interconnection Southbound configuration. This table must have exactly + one row. +

+ + + + See External IDs at the beginning of this document. + + + + + + + + + Database clients to which the Open vSwitch database server should + connect or on which it should listen, along with options for how these + connections should be configured. See the + table for more information. + + + Global SSL configuration. + + +
+ + +

+ Each row in this table represents an Availability Zone. Each OVN deployment + is considered an availability zone from OVN control plane perspective, with + its own central components, such as northbound and southbound databases and + ovn-northd daemon. +

+ + + A name that uniquely identifies the availability zone. + +
+ + +

+ Each row in this table represents a interconnection gateway chassis in an + availability zone. +

+ + + The name of the gateway. See column + of the OVN Southbound database's table. + + + + The availabilty zone that the gateway belongs to. + + + + The hostname of the gateway. + + + + The overall purpose of these columns is described under Common + Columns at the beginning of this document. + + + + + +

+ OVN uses encapsulation to transmit logical dataplane packets + between gateways. +

+ + + Points to supported encapsulation configurations to transmit + logical dataplane packets to this gateway. Each entry is a record that describes the configuration. + See column + of the OVN Southbound database's table. + + + +
+ + +

+ The column in the table refers to rows in this table to identify + how OVN may transmit logical dataplane packets to this gateway. +

+ + + The encapsulation to use to transmit packets to this gateway. + See column + of the OVN Southbound database's table. + + + + Options for configuring the encapsulation, which may be specific. See + column of the OVN Southbound database's table. + + + + The IPv4 address of the encapsulation tunnel endpoint. + + + + The name of the gateway that created this encap. + +
+ + +

+ Each row in this table represents a logical datapath for a transit + logical switch configured in the OVN Interconnection Northbound + database's table. +

+ + + The name of the transit logical switch that is configured in the OVN + Interconnection Northbound database's table. + + + +

+ The tunnel key value to which the logical datapath is bound. The key + can be generated by any ovn-ic but the same key is shared + by all availability zones so that the logical datapaths can be peered + across them. A tunnel key for transit switch datapath binding must be + globally unique. +

+ +

+ For more information about the meanings of a tunnel key, + see column of the OVN + Southbound database's + table. +

+ + + + The overall purpose of these columns is described under Common + Columns at the beginning of this document. + + + +
+ + +

+ Each row in this table binds a logical port on the transit switch to a + physical gateway and a tunnel key. Each port on the transit switch + belongs to a specific availability zone. +

+ + + + The name of the transit switch that the corresponding logical port + belongs to. + + + + A logical port, taken from in the OVN_Northbound + database's + table. The logical port name must be unique across all availability + zones. + + + + The availability zone that the port belongs to. + + + + Points to supported encapsulation configurations to transmit + logical dataplane packets to this gateway. Each entry is a record that describes the configuration. + + + + The name of the gateway that this port is physically located. + + + +

+ A number that represents the logical port in the key (e.g. STT key or + Geneve TLV) field carried within tunnel protocol packets. The key + can be generated by any ovn-ic but the same key is + shared by all availability zones so that the packets can go through + the datapath pipelines of different availability zones. +

+

+ The tunnel ID must be unique within the scope of a logical datapath. +

+

+ For more information about tunnel key, see column of the OVN Southbound database's table. +

+ + + +

+ The Ethernet address and IP addresses used by the corresponding logical + router port peering with the transit switch port. It is a string combined + with the value of + column followed by the values in column in table. +

+ + + + + + +

+ See External IDs at the beginning of this document. +

+ + +
+ + +

+ Configuration for a database connection to an Open vSwitch database + (OVSDB) client. +

+ +

+ This table primarily configures the Open vSwitch database server + (ovsdb-server). +

+ +

+ The Open vSwitch database server can initiate and maintain active + connections to remote clients. It can also listen for database + connections. +

+ + + +

Connection methods for clients.

+

+ The following connection methods are currently supported: +

+
+
ssl:host[:port]
+
+

+ The specified SSL port on the given host, + which can either be a DNS name (if built with unbound library) or + an IP address. A valid SSL configuration must be provided when + this form is used, this configuration can be specified via + command-line options or the table. +

+

+ If port is not specified, it defaults to 6640. +

+

+ SSL support is an optional feature that is not always + built as part of Open vSwitch. +

+
+ +
tcp:host[:port]
+
+

+ The specified TCP port on the given host, + which can either be a DNS name (if built with unbound library) or + an IP address (IPv4 or IPv6). If host is an IPv6 + address, wrap it in square brackets, e.g. tcp:[::1]:6640. +

+

+ If port is not specified, it defaults to 6640. +

+
+
pssl:[port][:host]
+
+

+ Listens for SSL connections on the specified TCP port. + Specify 0 for port to have the kernel automatically + choose an available port. If host, which can either + be a DNS name (if built with unbound library) or an IP address, + is specified, then connections are restricted to the resolved or + specified local IP address (either IPv4 or IPv6 address). If + host is an IPv6 address, wrap in square brackets, + e.g. pssl:6640:[::1]. If host is not + specified then it listens only on IPv4 (but not IPv6) addresses. + A valid SSL configuration must be provided when this form is used, + this can be specified either via command-line options or the + table. +

+

+ If port is not specified, it defaults to 6640. +

+

+ SSL support is an optional feature that is not always built as + part of Open vSwitch. +

+
+
ptcp:[port][:host]
+
+

+ Listens for connections on the specified TCP port. + Specify 0 for port to have the kernel automatically + choose an available port. If host, which can either + be a DNS name (if built with unbound library) or an IP address, + is specified, then connections are restricted to the resolved or + specified local IP address (either IPv4 or IPv6 address). If + host is an IPv6 address, wrap it in square brackets, + e.g. ptcp:6640:[::1]. If host is not + specified then it listens only on IPv4 addresses. +

+

+ If port is not specified, it defaults to 6640. +

+
+
+

When multiple clients are configured, the + values must be unique. Duplicate values yield + unspecified results.

+ + + + + + + Maximum number of milliseconds to wait between connection attempts. + Default is implementation-specific. + + + + Maximum number of milliseconds of idle time on connection to the client + before sending an inactivity probe message. If Open vSwitch does not + communicate with the client for the specified number of seconds, it + will send a probe. If a response is not received for the same + additional amount of time, Open vSwitch assumes the connection has been + broken and attempts to reconnect. Default is implementation-specific. + A value of 0 disables inactivity probes. + + + + +

+ Key-value pair of is always updated. + Other key-value pairs in the status columns may be updated depends + on the type. +

+ +

+ When specifies a connection method that + listens for inbound connections (e.g. ptcp: or + punix:), both and + may also be updated while the + remaining key-value pairs are omitted. +

+ +

+ On the other hand, when specifies an + outbound connection, all key-value pairs may be updated, except + the above-mentioned two key-value pairs associated with inbound + connection targets. They are omitted. +

+ + + true if currently connected to this client, + false otherwise. + + + + A human-readable description of the last error on the connection + to the manager; i.e. strerror(errno). This key + will exist only if an error has occurred. + + + +

+ The state of the connection to the manager: +

+
+
VOID
+
Connection is disabled.
+ +
BACKOFF
+
Attempting to reconnect at an increasing period.
+ +
CONNECTING
+
Attempting to connect.
+ +
ACTIVE
+
Connected, remote host responsive.
+ +
IDLE
+
Connection is idle. Waiting for response to keep-alive.
+
+

+ These values may change in the future. They are provided only for + human consumption. +

+ + + + The amount of time since this client last successfully connected + to the database (in seconds). Value is empty if client has never + successfully been connected. + + + + The amount of time since this client last disconnected from the + database (in seconds). Value is empty if client has never + disconnected. + + + + Space-separated list of the names of OVSDB locks that the connection + holds. Omitted if the connection does not hold any locks. + + + + Space-separated list of the names of OVSDB locks that the connection is + currently waiting to acquire. Omitted if the connection is not waiting + for any locks. + + + + Space-separated list of the names of OVSDB locks that the connection + has had stolen by another OVSDB client. Omitted if no locks have been + stolen from this connection. + + + + When specifies a connection method that + listens for inbound connections (e.g. ptcp: or + pssl:) and more than one connection is actually active, + the value is the number of active connections. Otherwise, this + key-value pair is omitted. + + + + When is ptcp: or + pssl:, this is the TCP port on which the OVSDB server is + listening. (This is particularly useful when specifies a port of 0, allowing the kernel to + choose any available port.) + + + + + The overall purpose of these columns is described under Common + Columns at the beginning of this document. + + + + +
+ + SSL configuration for ovn-sb database access. + + + Name of a PEM file containing the private key used as the switch's + identity for SSL connections to the controller. + + + + Name of a PEM file containing a certificate, signed by the + certificate authority (CA) used by the controller and manager, + that certifies the switch's private key, identifying a trustworthy + switch. + + + + Name of a PEM file containing the CA certificate used to verify + that the switch is connected to a trustworthy controller. + + + + If set to true, then Open vSwitch will attempt to + obtain the CA certificate from the controller on its first SSL + connection and save it to the named PEM file. If it is successful, + it will immediately drop the connection and reconnect, and from then + on all SSL connections must be authenticated by a certificate signed + by the CA certificate thus obtained. This option exposes the + SSL connection to a man-in-the-middle attack obtaining the initial + CA certificate. It may still be useful for bootstrapping. + + + + List of SSL protocols to be enabled for SSL connections. The default + when this option is omitted is TLSv1,TLSv1.1,TLSv1.2. + + + + List of ciphers (in OpenSSL cipher string format) to be supported + for SSL connections. The default when this option is omitted is + HIGH:!aNULL:!MD5. + + + + The overall purpose of these columns is described under Common + Columns at the beginning of this document. + + + +
+ + diff --git a/tests/automake.mk b/tests/automake.mk index 8c1630c..c70a213 100644 --- a/tests/automake.mk +++ b/tests/automake.mk @@ -25,6 +25,7 @@ TESTSUITE_AT = \ tests/ovn-nbctl.at \ tests/ovn-sbctl.at \ tests/ovn-ic-nbctl.at \ + tests/ovn-ic-sbctl.at \ tests/ovn-controller.at \ tests/ovn-controller-vtep.at \ tests/ovn-macros.at \ @@ -101,6 +102,7 @@ valgrind_wrappers = \ tests/valgrind/ovn-northd \ tests/valgrind/ovn-sbctl \ tests/valgrind/ovn-ic-nbctl \ + tests/valgrind/ovn-ic-sbctl \ tests/valgrind/ovs-appctl \ tests/valgrind/ovs-ofctl \ tests/valgrind/ovs-vsctl \ diff --git a/tests/ovn-ic-sbctl.at b/tests/ovn-ic-sbctl.at new file mode 100644 index 0000000..86d6b00 --- /dev/null +++ b/tests/ovn-ic-sbctl.at @@ -0,0 +1,112 @@ +AT_BANNER([ovn-ic-sbctl]) + +# OVN_IC_SBCTL_TEST_START +m4_define([OVN_IC_SBCTL_TEST_START], + [dnl Create database (ovn-ic-sb). + AT_KEYWORDS([ic_sbctl]) + AT_CHECK([ovsdb-tool create ovn-ic-sb.db $abs_top_srcdir/ovn-ic-sb.ovsschema]) + + dnl Start ovsdb-servers. + AT_CHECK([ovsdb-server --detach --no-chdir --pidfile=ovn_ic_sb_db.pid --unixctl=$OVS_RUNDIR/ovn_ic_sb_db.ctl --log-file=ovsdb_ic_nb.log --remote=punix:$OVS_RUNDIR/ovn_ic_sb_db.sock ovn-ic-sb.db ], [0], [], [stderr]) + on_exit "kill `cat ovn_ic_sb_db.pid`" + AT_CHECK([[sed < stderr ' +/vlog|INFO|opened log file/d +/ovsdb_server|INFO|ovsdb-server (Open vSwitch)/d']]) + AT_CAPTURE_FILE([ovsdb-server.log]) +]) + +# OVN_IC_SBCTL_TEST_STOP +m4_define([OVN_IC_SBCTL_TEST_STOP], + [AT_CHECK([check_logs "$1"]) + OVS_APP_EXIT_AND_WAIT_BY_TARGET([$OVS_RUNDIR/ovn_ic_sb_db.ctl], [$OVS_RUNDIR/ovn_ic_sb_db.pid])]) + +dnl --------------------------------------------------------------------- + +AT_SETUP([ovn-ic-sbctl]) +OVN_IC_SBCTL_TEST_START + +for az in 1 2; do + az_uuid=$(ovn-ic-sbctl create availability_zone name=az$az) + for gw in 1 2; do + ovn-ic-sbctl --id=@encap create encap type=geneve ip=192.168.0.$az$gw options="csum=true" \ + -- create gateway availability_zone=$az_uuid name=gw$az$gw encap=@encap hostname=host.gw$az$gw + for pb in 1 2; do + ovn-ic-sbctl create port_binding logical_port=lp$az$gw$pb transit_switch="ts$pb" \ + address="\"aa:aa:aa:aa:0$az:$gw$pb 169.254.$pb.$az$gw/24\"" tunnel_key=$az$gw \ + availability_zone=$az_uuid gateway=gw$az$gw + done + done +done + +AT_CHECK([ovn-ic-sbctl show], [0], [dnl +availability-zone az1 + gateway gw11 + hostname: host.gw11 + type: geneve + ip: 192.168.0.11 + port lp111 + transit switch: ts1 + address: [["aa:aa:aa:aa:01:11 169.254.1.11/24"]] + port lp112 + transit switch: ts2 + address: [["aa:aa:aa:aa:01:12 169.254.2.11/24"]] + gateway gw12 + hostname: host.gw12 + type: geneve + ip: 192.168.0.12 + port lp121 + transit switch: ts1 + address: [["aa:aa:aa:aa:01:21 169.254.1.12/24"]] + port lp122 + transit switch: ts2 + address: [["aa:aa:aa:aa:01:22 169.254.2.12/24"]] +availability-zone az2 + gateway gw21 + hostname: host.gw21 + type: geneve + ip: 192.168.0.21 + port lp211 + transit switch: ts1 + address: [["aa:aa:aa:aa:02:11 169.254.1.21/24"]] + port lp212 + transit switch: ts2 + address: [["aa:aa:aa:aa:02:12 169.254.2.21/24"]] + gateway gw22 + hostname: host.gw22 + type: geneve + ip: 192.168.0.22 + port lp221 + transit switch: ts1 + address: [["aa:aa:aa:aa:02:21 169.254.1.22/24"]] + port lp222 + transit switch: ts2 + address: [["aa:aa:aa:aa:02:22 169.254.2.22/24"]] +]) + +AT_CHECK([ovn-ic-sbctl show az2], [0], [dnl +availability-zone az2 + gateway gw21 + hostname: host.gw21 + type: geneve + ip: 192.168.0.21 + port lp211 + transit switch: ts1 + address: [["aa:aa:aa:aa:02:11 169.254.1.21/24"]] + port lp212 + transit switch: ts2 + address: [["aa:aa:aa:aa:02:12 169.254.2.21/24"]] + gateway gw22 + hostname: host.gw22 + type: geneve + ip: 192.168.0.22 + port lp221 + transit switch: ts1 + address: [["aa:aa:aa:aa:02:21 169.254.1.22/24"]] + port lp222 + transit switch: ts2 + address: [["aa:aa:aa:aa:02:22 169.254.2.22/24"]] +]) + + +OVN_IC_SBCTL_TEST_STOP +AT_CLEANUP diff --git a/tests/testsuite.at b/tests/testsuite.at index aa44a2e..eff6732 100644 --- a/tests/testsuite.at +++ b/tests/testsuite.at @@ -27,6 +27,7 @@ m4_include([tests/ovn-northd.at]) m4_include([tests/ovn-nbctl.at]) m4_include([tests/ovn-sbctl.at]) m4_include([tests/ovn-ic-nbctl.at]) +m4_include([tests/ovn-ic-sbctl.at]) m4_include([tests/ovn-controller.at]) m4_include([tests/ovn-controller-vtep.at]) m4_include([tests/checkpatch.at]) diff --git a/utilities/.gitignore b/utilities/.gitignore index 6c710fb..763f760 100644 --- a/utilities/.gitignore +++ b/utilities/.gitignore @@ -5,6 +5,8 @@ /ovn-sbctl.8 /ovn-ic-nbctl /ovn-ic-nbctl.8 +/ovn-ic-sbctl +/ovn-ic-sbctl.8 /ovn-appctl /ovn-appctl.8 /ovn-trace diff --git a/utilities/automake.mk b/utilities/automake.mk index ac2e034..c4a6d24 100644 --- a/utilities/automake.mk +++ b/utilities/automake.mk @@ -8,6 +8,7 @@ man_MANS += \ utilities/ovn-nbctl.8 \ utilities/ovn-sbctl.8 \ utilities/ovn-ic-nbctl.8 \ + utilities/ovn-ic-sbctl.8 \ utilities/ovn-trace.8 \ utilities/ovn-detrace.1 \ utilities/ovn-appctl.8 @@ -30,6 +31,7 @@ EXTRA_DIST += \ utilities/ovn-docker-underlay-driver.in \ utilities/ovn-nbctl.8.xml \ utilities/ovn-ic-nbctl.8.xml \ + utilities/ovn-ic-sbctl.8.xml \ utilities/ovn-appctl.8.xml \ utilities/ovn-trace.8.xml \ utilities/ovn-detrace.in \ @@ -54,6 +56,7 @@ CLEANFILES += \ utilities/ovn-nbctl.8 \ utilities/ovn-sbctl.8 \ utilities/ovn-ic-nbctl.8 \ + utilities/ovn-ic-sbctl.8 \ utilities/ovn-trace.8 \ utilities/ovn-detrace.1 \ utilities/ovn-detrace \ @@ -81,6 +84,11 @@ bin_PROGRAMS += utilities/ovn-ic-nbctl utilities_ovn_ic_nbctl_SOURCES = utilities/ovn-ic-nbctl.c utilities_ovn_ic_nbctl_LDADD = lib/libovn.la $(OVSDB_LIBDIR)/libovsdb.la $(OVS_LIBDIR)/libopenvswitch.la +# ovn-ic-sbctl +bin_PROGRAMS += utilities/ovn-ic-sbctl +utilities_ovn_ic_sbctl_SOURCES = utilities/ovn-ic-sbctl.c +utilities_ovn_ic_sbctl_LDADD = lib/libovn.la $(OVSDB_LIBDIR)/libovsdb.la $(OVS_LIBDIR)/libopenvswitch.la + # ovn-trace bin_PROGRAMS += utilities/ovn-trace utilities_ovn_trace_SOURCES = utilities/ovn-trace.c diff --git a/utilities/ovn-ic-sbctl.8.xml b/utilities/ovn-ic-sbctl.8.xml new file mode 100644 index 0000000..9952e6c --- /dev/null +++ b/utilities/ovn-ic-sbctl.8.xml @@ -0,0 +1,148 @@ + + +

Name

+

ovn-ic-sbctl -- Open Virtual Network interconnection southbound db management utility

+ +

Synopsis

+

ovn-ic-sbctl [options] command [arg...]

+ +

Description

+

This utility can be used to manage the OVN interconnection southbound database.

+ +

General Commands

+ +
+
init
+
+ Initializes the database, if it is empty. If the database has already + been initialized, this command has no effect. +
+ +
show [availability_zone]
+
+ Prints a brief overview of the database contents. If + availability_zone is provided, only records related to that + availability zone are shown. +
+
+ +

Database Commands

+

These commands query and modify the contents of ovsdb tables. + They are a slight abstraction of the ovsdb interface and + as such they operate at a lower level than other ovn-ic-sbctl commands.

+

Identifying Tables, Records, and Columns

+

Each of these commands has a table parameter to identify a table + within the database. Many of them also take a record parameter + that identifies a particular record within a table. The record + parameter may be the UUID for a record, which may be abbreviated to its + first 4 (or more) hex digits, as long as that is unique. Many tables offer + additional ways to identify records. Some commands also take + column parameters that identify a particular field within the + records in a table.

+ +

+ For a list of tables and their columns, see ovn-ic-sb(5) or + see the table listing from the --help option. +

+ +

+ Record names must be specified in full and with correct capitalization, + except that UUIDs may be abbreviated to their first 4 (or more) hex + digits, as long as that is unique within the table. Names of tables and + columns are not case-sensitive, and - and _ are + treated interchangeably. Unique abbreviations of table and column names + are acceptable, e.g. g or gatew is sufficient + to identify the Gateway table. +

+ + + +

Remote Connectivity Commands

+
+
get-connection
+
+ Prints the configured connection(s). +
+ +
del-connection
+
+ Deletes the configured connection(s). +
+ +
[--inactivity-probe=msecs] set-connection target...
+
+ Sets the configured manager target or targets. Use + --inactivity-probe=msecs to override the default + idle connection inactivity probe time. Use 0 to disable inactivity probes. +
+
+ +

SSL Configuration Commands

+
+
get-ssl
+
+ Prints the SSL configuration. +
+ +
del-ssl
+
+ Deletes the current SSL configuration. +
+ +
[--bootstrap] set-ssl + private-key certificate ca-cert + [ssl-protocol-list [ssl-cipher-list]]
+
+ Sets the SSL configuration. +
+
+ +

Options

+ +
+
--db database
+
+ The OVSDB database remote to contact. If the OVN_IC_SB_DB + environment variable is set, its value is used as the default. + Otherwise, the default is unix:@RUNDIR@/ovn_ic_sb_db.sock, but this + default is unlikely to be useful outside of single-machine OVN test + environments. +
+ +
--leader-only
+
--no-leader-only
+
+ By default, or with --leader-only, when the database server + is a clustered database, ovn-ic-sbctl will avoid servers other + than the cluster leader. This ensures that any data that + ovn-ic-sbctl reads and reports is up-to-date. With + --no-leader-only, ovn-ic-sbctl will use any server + in the cluster, which means that for read-only transactions it can report + and act on stale data (transactions that modify the database are always + serialized even with --no-leader-only). Refer to + Understanding Cluster Consistency in ovsdb(7) + for more information. +
+
+ +

Logging options

+ + +

Table Formatting Options

+ These options control the format of output from the list and + find commands. + + +

PKI Options

+

+ PKI configuration is required to use SSL for the connection to the + database. +

+ + + +

Other Options

+ + + + diff --git a/utilities/ovn-ic-sbctl.c b/utilities/ovn-ic-sbctl.c new file mode 100644 index 0000000..f01a77c --- /dev/null +++ b/utilities/ovn-ic-sbctl.c @@ -0,0 +1,1017 @@ +/* + * Copyright (c) 2020 eBay Inc. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at: + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#include + +#include +#include +#include +#include +#include + +#include "command-line.h" +#include "compiler.h" +#include "db-ctl-base.h" +#include "dirs.h" +#include "fatal-signal.h" +#include "openvswitch/dynamic-string.h" +#include "openvswitch/json.h" +#include "openvswitch/shash.h" +#include "openvswitch/vconn.h" +#include "openvswitch/vlog.h" +#include "lib/ovn-ic-sb-idl.h" +#include "lib/ovn-util.h" +#include "openvswitch/poll-loop.h" +#include "process.h" +#include "sset.h" +#include "stream-ssl.h" +#include "stream.h" +#include "table.h" +#include "util.h" +#include "svec.h" + +VLOG_DEFINE_THIS_MODULE(ic_sbctl); + +struct ic_sbctl_context; + +/* --db: The database server to contact. */ +static const char *db; + +/* --oneline: Write each command's output as a single line? */ +static bool oneline; + +/* --dry-run: Do not commit any changes. */ +static bool dry_run; + +/* --timeout: Time to wait for a connection to 'db'. */ +static unsigned int timeout; + +/* Format for table output. */ +static struct table_style table_style = TABLE_STYLE_DEFAULT; + +/* The IDL we're using and the current transaction, if any. + * This is for use by ic_sbctl_exit() only, to allow it to clean up. + * Other code should use its context arguments. */ +static struct ovsdb_idl *the_idl; +static struct ovsdb_idl_txn *the_idl_txn; +OVS_NO_RETURN static void ic_sbctl_exit(int status); + +/* --leader-only, --no-leader-only: Only accept the leader in a cluster. */ +static int leader_only = true; + +static void ic_sbctl_cmd_init(void); +OVS_NO_RETURN static void usage(void); +static void parse_options(int argc, char *argv[], struct shash *local_options); +static void run_prerequisites(struct ctl_command[], size_t n_commands, + struct ovsdb_idl *); +static bool do_ic_sbctl(const char *args, struct ctl_command *, size_t n, + struct ovsdb_idl *); + +int +main(int argc, char *argv[]) +{ + struct ovsdb_idl *idl; + struct ctl_command *commands; + struct shash local_options; + unsigned int seqno; + size_t n_commands; + + set_program_name(argv[0]); + fatal_ignore_sigpipe(); + vlog_set_levels(NULL, VLF_CONSOLE, VLL_WARN); + vlog_set_levels_from_string_assert("reconnect:warn"); + + ic_sbctl_cmd_init(); + + /* Parse command line. */ + char *args = process_escape_args(argv); + shash_init(&local_options); + parse_options(argc, argv, &local_options); + char *error = ctl_parse_commands(argc - optind, argv + optind, + &local_options, &commands, &n_commands); + if (error) { + ctl_fatal("%s", error); + } + VLOG(ctl_might_write_to_db(commands, n_commands) ? VLL_INFO : VLL_DBG, + "Called as %s", args); + + ctl_timeout_setup(timeout); + + /* Initialize IDL. */ + idl = the_idl = ovsdb_idl_create(db, &icsbrec_idl_class, true, false); + ovsdb_idl_set_leader_only(idl, leader_only); + run_prerequisites(commands, n_commands, idl); + + /* Execute the commands. + * + * 'seqno' is the database sequence number for which we last tried to + * execute our transaction. There's no point in trying to commit more than + * once for any given sequence number, because if the transaction fails + * it's because the database changed and we need to obtain an up-to-date + * view of the database before we try the transaction again. */ + seqno = ovsdb_idl_get_seqno(idl); + for (;;) { + ovsdb_idl_run(idl); + if (!ovsdb_idl_is_alive(idl)) { + int retval = ovsdb_idl_get_last_error(idl); + ctl_fatal("%s: database connection failed (%s)", + db, ovs_retval_to_string(retval)); + } + + if (seqno != ovsdb_idl_get_seqno(idl)) { + seqno = ovsdb_idl_get_seqno(idl); + if (do_ic_sbctl(args, commands, n_commands, idl)) { + free(args); + exit(EXIT_SUCCESS); + } + } + + if (seqno == ovsdb_idl_get_seqno(idl)) { + ovsdb_idl_wait(idl); + poll_block(); + } + } +} + +static void +parse_options(int argc, char *argv[], struct shash *local_options) +{ + enum { + OPT_DB = UCHAR_MAX + 1, + OPT_ONELINE, + OPT_NO_SYSLOG, + OPT_DRY_RUN, + OPT_LOCAL, + OPT_COMMANDS, + OPT_OPTIONS, + OPT_BOOTSTRAP_CA_CERT, + VLOG_OPTION_ENUMS, + TABLE_OPTION_ENUMS, + SSL_OPTION_ENUMS, + }; + static const struct option global_long_options[] = { + {"db", required_argument, NULL, OPT_DB}, + {"no-syslog", no_argument, NULL, OPT_NO_SYSLOG}, + {"dry-run", no_argument, NULL, OPT_DRY_RUN}, + {"oneline", no_argument, NULL, OPT_ONELINE}, + {"timeout", required_argument, NULL, 't'}, + {"help", no_argument, NULL, 'h'}, + {"commands", no_argument, NULL, OPT_COMMANDS}, + {"options", no_argument, NULL, OPT_OPTIONS}, + {"leader-only", no_argument, &leader_only, true}, + {"no-leader-only", no_argument, &leader_only, false}, + {"version", no_argument, NULL, 'V'}, + VLOG_LONG_OPTIONS, + STREAM_SSL_LONG_OPTIONS, + {"bootstrap-ca-cert", required_argument, NULL, OPT_BOOTSTRAP_CA_CERT}, + TABLE_LONG_OPTIONS, + {NULL, 0, NULL, 0}, + }; + const int n_global_long_options = ARRAY_SIZE(global_long_options) - 1; + char *tmp, *short_options; + + struct option *options; + size_t allocated_options; + size_t n_options; + size_t i; + + tmp = ovs_cmdl_long_options_to_short_options(global_long_options); + short_options = xasprintf("+%s", tmp); + free(tmp); + + /* We want to parse both global and command-specific options here, but + * getopt_long() isn't too convenient for the job. We copy our global + * options into a dynamic array, then append all of the command-specific + * options. */ + options = xmemdup(global_long_options, sizeof global_long_options); + allocated_options = ARRAY_SIZE(global_long_options); + n_options = n_global_long_options; + ctl_add_cmd_options(&options, &n_options, &allocated_options, OPT_LOCAL); + + for (;;) { + int idx; + int c; + + c = getopt_long(argc, argv, short_options, options, &idx); + if (c == -1) { + break; + } + + switch (c) { + case OPT_DB: + db = optarg; + break; + + case OPT_ONELINE: + oneline = true; + break; + + case OPT_NO_SYSLOG: + vlog_set_levels(&this_module, VLF_SYSLOG, VLL_WARN); + break; + + case OPT_DRY_RUN: + dry_run = true; + break; + + case OPT_LOCAL: + if (shash_find(local_options, options[idx].name)) { + ctl_fatal("'%s' option specified multiple times", + options[idx].name); + } + shash_add_nocopy(local_options, + xasprintf("--%s", options[idx].name), + nullable_xstrdup(optarg)); + break; + + case 'h': + usage(); + + case OPT_COMMANDS: + ctl_print_commands(); + /* fall through */ + + case OPT_OPTIONS: + ctl_print_options(global_long_options); + /* fall through */ + + case 'V': + ovs_print_version(0, 0); + printf("DB Schema %s\n", icsbrec_get_db_version()); + exit(EXIT_SUCCESS); + + case 't': + if (!str_to_uint(optarg, 10, &timeout) || !timeout) { + ctl_fatal("value %s on -t or --timeout is invalid", optarg); + } + break; + + VLOG_OPTION_HANDLERS + TABLE_OPTION_HANDLERS(&table_style) + STREAM_SSL_OPTION_HANDLERS + + case OPT_BOOTSTRAP_CA_CERT: + stream_ssl_set_ca_cert_file(optarg, true); + break; + + case '?': + exit(EXIT_FAILURE); + + default: + ovs_abort(0, "Internal error when parsing option %d.", c); + + case 0: + break; + } + } + free(short_options); + + if (!db) { + db = default_ic_sb_db(); + } + + for (i = n_global_long_options; options[i].name; i++) { + free(CONST_CAST(char *, options[i].name)); + } + free(options); +} + +static void +usage(void) +{ + printf("\ +%s: OVN interconnection southbound DB management utility\n\ +\n\ +usage: %s [OPTIONS] COMMAND [ARG...]\n\ +\n\ +General commands:\n\ + show print overview of database contents\n\ +\n\ +Connection commands:\n\ + get-connection print the connections\n\ + del-connection delete the connections\n\ + [--inactivity-probe=MSECS]\n\ + set-connection TARGET... set the list of connections to TARGET...\n\ +\n\ +SSL commands:\n\ + get-ssl print the SSL configuration\n\ + del-ssl delete the SSL configuration\n\ + set-ssl PRIV-KEY CERT CA-CERT [SSL-PROTOS [SSL-CIPHERS]] \ +set the SSL configuration\n\ +\n\ +%s\ +%s\ +\n\ +Options:\n\ + --db=DATABASE connect to DATABASE\n\ + (default: %s)\n\ + --no-leader-only accept any cluster member, not just the leader\n\ + -t, --timeout=SECS wait at most SECS seconds\n\ + --dry-run do not commit changes to database\n\ + --oneline print exactly one line of output per command\n", + program_name, program_name, ctl_get_db_cmd_usage(), + ctl_list_db_tables_usage(), default_ic_sb_db()); + table_usage(); + vlog_usage(); + printf("\ + --no-syslog equivalent to --verbose=ic_sbctl:syslog:warn\n"); + printf("\n\ +Other options:\n\ + -h, --help display this help message\n\ + -V, --version display version information\n"); + stream_usage("database", true, true, true); + exit(EXIT_SUCCESS); +} + + +/* ovs-ic_sbctl specific context. Inherits the 'struct ctl_context' as base. + * Now empty, just keep the framework for future additions. */ +struct ic_sbctl_context { + struct ctl_context base; + + /* A cache of the contents of the database. + * + * A command that needs to use any of this information must first call + * ic_sbctl_context_populate_cache(). A command that changes anything that + * could invalidate the cache must either call + * ic_sbctl_context_invalidate_cache() or manually update the cache to + * maintain its correctness. */ + bool cache_valid; +}; + +static void +ic_sbctl_init(struct ctl_context *ctx OVS_UNUSED) +{ +} + +static char * +az_by_name_or_uuid(struct ctl_context *ctx, const char *id, bool must_exist, + const struct icsbrec_availability_zone **az_p) +{ + const struct icsbrec_availability_zone *az = NULL; + *az_p = NULL; + + struct uuid az_uuid; + bool is_uuid = uuid_from_string(&az_uuid, id); + if (is_uuid) { + az = icsbrec_availability_zone_get_for_uuid(ctx->idl, &az_uuid); + } + + if (!az) { + const struct icsbrec_availability_zone *iter; + + ICSBREC_AVAILABILITY_ZONE_FOR_EACH (iter, ctx->idl) { + if (!strcmp(iter->name, id)) { + az = iter; + break; + } + } + } + + if (!az && must_exist) { + return xasprintf("%s: availability zone %s not found", + id, is_uuid ? "UUID" : "name"); + } + + *az_p = az; + return NULL; +} + +/* Data structures for the show command, which constructs the hierarchy + * of availability_zone:gateway:port_binding. */ +struct gw_data { + const struct icsbrec_gateway *isb_gw; + /* With node type struct icsbrec_port_binding. + * We don't need to search port-binding, but using shash makes + * sorting convenient. */ + struct shash pbs; +}; + +struct az_data { + const struct icsbrec_availability_zone *isb_az; + /* With node type struct gw_data */ + struct shash gws; +}; + +static void +init_az_info(struct ctl_context *ctx, struct shash *azs) +{ + const struct icsbrec_availability_zone *az; + ICSBREC_AVAILABILITY_ZONE_FOR_EACH (az, ctx->idl) { + struct az_data *az_data = xmalloc(sizeof *az_data); + shash_init(&az_data->gws); + az_data->isb_az = az; + shash_add(azs, az->name, az_data); + } + + const struct icsbrec_gateway *gw; + ICSBREC_GATEWAY_FOR_EACH (gw, ctx->idl) { + struct az_data *az_data = shash_find_data(azs, + gw->availability_zone->name); + ovs_assert(az_data); + struct gw_data *gw_data = xmalloc(sizeof *gw_data); + gw_data->isb_gw = gw; + shash_init(&gw_data->pbs); + shash_add(&az_data->gws, gw->name, gw_data); + } + + const struct icsbrec_port_binding *pb; + ICSBREC_PORT_BINDING_FOR_EACH (pb, ctx->idl) { + struct az_data *az_data = shash_find_data(azs, + pb->availability_zone->name); + ovs_assert(az_data); + struct gw_data *gw_data = shash_find_data(&az_data->gws, pb->gateway); + if (!gw_data) { + continue; + } + shash_add(&gw_data->pbs, pb->logical_port, pb); + } +} + +static void +destroy_az_info(struct shash *azs) +{ + struct shash_node *az_node; + SHASH_FOR_EACH (az_node, azs) { + struct az_data *az_data = az_node->data; + struct shash_node *gw_node; + SHASH_FOR_EACH (gw_node, &az_data->gws) { + struct gw_data *gw_data = gw_node->data; + shash_destroy(&gw_data->pbs); + free(gw_data); + } + shash_destroy(&az_data->gws); + free(az_data); + } + shash_destroy(azs); +} + +static void +print_az(struct az_data *az, struct ds *s) +{ + ds_put_format(s, "availability-zone %s\n", az->isb_az->name); + + const struct shash_node **nodes = shash_sort(&az->gws); + for (int g = 0; g < shash_count(&az->gws); g++) { + struct gw_data *gw = nodes[g]->data; + + ds_put_format(s, " gateway %s\n", gw->isb_gw->name); + if (gw->isb_gw->hostname[0]) { + ds_put_format(s, " hostname: %s\n", gw->isb_gw->hostname); + } + + for (int i = 0; i < gw->isb_gw->n_encaps; i++) { + ds_put_format(s, " type: %s\n", + gw->isb_gw->encaps[i]->type); + ds_put_format(s, " ip: %s\n", + gw->isb_gw->encaps[i]->ip); + } + + const struct shash_node **pb_nodes = shash_sort(&gw->pbs); + for (int p = 0; p < shash_count(&gw->pbs); p++) { + const struct icsbrec_port_binding *pb = pb_nodes[p]->data; + ds_put_format(s, " port %s\n", + pb->logical_port); + ds_put_format(s, " transit switch: %s\n", + pb->transit_switch); + ds_put_format(s, " address: [\"%s\"]\n", + pb->address); + } + free(pb_nodes); + } + free(nodes); +} + +static void +ic_sbctl_show(struct ctl_context *ctx) +{ + /* Availability zones with the gateways and port-bindings data. Node + * type is struct az_data. */ + struct shash azs = SHASH_INITIALIZER(&azs); + init_az_info(ctx, &azs); + + const struct icsbrec_availability_zone *az; + + if (ctx->argc == 2) { + char *error = az_by_name_or_uuid(ctx, ctx->argv[1], false, &az); + if (error) { + ctx->error = error; + goto error; + } + if (az) { + struct az_data *az_data = shash_find_data(&azs, az->name); + ovs_assert(az_data); + print_az(az_data, &ctx->output); + } + } else { + const struct shash_node **nodes = shash_sort(&azs); + for (int i = 0; i < shash_count(&azs); i++) { + struct az_data *az_data = nodes[i]->data; + print_az(az_data, &ctx->output); + } + free(nodes); + } + +error: + destroy_az_info(&azs); +} + +static void +verify_connections(struct ctl_context *ctx) +{ + const struct icsbrec_ic_sb_global *ic_sb_global = + icsbrec_ic_sb_global_first(ctx->idl); + const struct icsbrec_connection *conn; + + icsbrec_ic_sb_global_verify_connections(ic_sb_global); + + ICSBREC_CONNECTION_FOR_EACH (conn, ctx->idl) { + icsbrec_connection_verify_target(conn); + } +} + +static void +pre_connection(struct ctl_context *ctx) +{ + ovsdb_idl_add_column(ctx->idl, &icsbrec_ic_sb_global_col_connections); + ovsdb_idl_add_column(ctx->idl, &icsbrec_connection_col_target); + ovsdb_idl_add_column(ctx->idl, &icsbrec_connection_col_inactivity_probe); +} + +static void +cmd_get_connection(struct ctl_context *ctx) +{ + const struct icsbrec_connection *conn; + struct svec targets; + size_t i; + + verify_connections(ctx); + + /* Print the targets in sorted order for reproducibility. */ + svec_init(&targets); + + ICSBREC_CONNECTION_FOR_EACH (conn, ctx->idl) { + svec_add(&targets, conn->target); + } + + svec_sort_unique(&targets); + for (i = 0; i < targets.n; i++) { + ds_put_format(&ctx->output, "%s\n", targets.names[i]); + } + svec_destroy(&targets); +} + +static void +delete_connections(struct ctl_context *ctx) +{ + const struct icsbrec_ic_sb_global *ic_sb_global = + icsbrec_ic_sb_global_first(ctx->idl); + const struct icsbrec_connection *conn, *next; + + /* Delete Manager rows pointed to by 'connection_options' column. */ + ICSBREC_CONNECTION_FOR_EACH_SAFE (conn, next, ctx->idl) { + icsbrec_connection_delete(conn); + } + + /* Delete 'Manager' row refs in 'manager_options' column. */ + icsbrec_ic_sb_global_set_connections(ic_sb_global, NULL, 0); +} + +static void +cmd_del_connection(struct ctl_context *ctx) +{ + verify_connections(ctx); + delete_connections(ctx); +} + +static void +insert_connections(struct ctl_context *ctx, char *targets[], size_t n) +{ + const struct icsbrec_ic_sb_global *ic_sb_global = + icsbrec_ic_sb_global_first(ctx->idl); + struct icsbrec_connection **connections; + size_t i, conns = 0; + const char *inactivity_probe = shash_find_data(&ctx->options, + "--inactivity-probe"); + + /* Insert each connection in a new row in Connection table. */ + connections = xmalloc(n * sizeof *connections); + for (i = 0; i < n; i++) { + if (stream_verify_name(targets[i]) && + pstream_verify_name(targets[i])) { + VLOG_WARN("target type \"%s\" is possibly erroneous", targets[i]); + } + + connections[conns] = icsbrec_connection_insert(ctx->txn); + icsbrec_connection_set_target(connections[conns], targets[i]); + if (inactivity_probe) { + int64_t msecs = atoll(inactivity_probe); + icsbrec_connection_set_inactivity_probe(connections[conns], + &msecs, 1); + } + conns++; + } + + /* Store uuids of new connection rows in 'connection' column. */ + icsbrec_ic_sb_global_set_connections(ic_sb_global, connections, conns); + free(connections); +} + +static void +cmd_set_connection(struct ctl_context *ctx) +{ + const size_t n = ctx->argc - 1; + + verify_connections(ctx); + delete_connections(ctx); + insert_connections(ctx, &ctx->argv[1], n); +} + +static void +pre_cmd_get_ssl(struct ctl_context *ctx) +{ + ovsdb_idl_add_column(ctx->idl, &icsbrec_ic_sb_global_col_ssl); + + ovsdb_idl_add_column(ctx->idl, &icsbrec_ssl_col_private_key); + ovsdb_idl_add_column(ctx->idl, &icsbrec_ssl_col_certificate); + ovsdb_idl_add_column(ctx->idl, &icsbrec_ssl_col_ca_cert); + ovsdb_idl_add_column(ctx->idl, &icsbrec_ssl_col_bootstrap_ca_cert); +} + +static void +cmd_get_ssl(struct ctl_context *ctx) +{ + const struct icsbrec_ic_sb_global *ic_sb_global = + icsbrec_ic_sb_global_first(ctx->idl); + const struct icsbrec_ssl *ssl = icsbrec_ssl_first(ctx->idl); + + icsbrec_ic_sb_global_verify_ssl(ic_sb_global); + if (ssl) { + icsbrec_ssl_verify_private_key(ssl); + icsbrec_ssl_verify_certificate(ssl); + icsbrec_ssl_verify_ca_cert(ssl); + icsbrec_ssl_verify_bootstrap_ca_cert(ssl); + + ds_put_format(&ctx->output, "Private key: %s\n", ssl->private_key); + ds_put_format(&ctx->output, "Certificate: %s\n", ssl->certificate); + ds_put_format(&ctx->output, "CA Certificate: %s\n", ssl->ca_cert); + ds_put_format(&ctx->output, "Bootstrap: %s\n", + ssl->bootstrap_ca_cert ? "true" : "false"); + } +} + +static void +pre_cmd_del_ssl(struct ctl_context *ctx) +{ + ovsdb_idl_add_column(ctx->idl, &icsbrec_ic_sb_global_col_ssl); +} + +static void +cmd_del_ssl(struct ctl_context *ctx) +{ + const struct icsbrec_ic_sb_global *ic_sb_global = + icsbrec_ic_sb_global_first(ctx->idl); + const struct icsbrec_ssl *ssl = icsbrec_ssl_first(ctx->idl); + + if (ssl) { + icsbrec_ic_sb_global_verify_ssl(ic_sb_global); + icsbrec_ssl_delete(ssl); + icsbrec_ic_sb_global_set_ssl(ic_sb_global, NULL); + } +} + +static void +pre_cmd_set_ssl(struct ctl_context *ctx) +{ + ovsdb_idl_add_column(ctx->idl, &icsbrec_ic_sb_global_col_ssl); +} + +static void +cmd_set_ssl(struct ctl_context *ctx) +{ + bool bootstrap = shash_find(&ctx->options, "--bootstrap"); + const struct icsbrec_ic_sb_global *ic_sb_global = + icsbrec_ic_sb_global_first(ctx->idl); + const struct icsbrec_ssl *ssl = icsbrec_ssl_first(ctx->idl); + + icsbrec_ic_sb_global_verify_ssl(ic_sb_global); + if (ssl) { + icsbrec_ssl_delete(ssl); + } + ssl = icsbrec_ssl_insert(ctx->txn); + + icsbrec_ssl_set_private_key(ssl, ctx->argv[1]); + icsbrec_ssl_set_certificate(ssl, ctx->argv[2]); + icsbrec_ssl_set_ca_cert(ssl, ctx->argv[3]); + + icsbrec_ssl_set_bootstrap_ca_cert(ssl, bootstrap); + + if (ctx->argc == 5) { + icsbrec_ssl_set_ssl_protocols(ssl, ctx->argv[4]); + } else if (ctx->argc == 6) { + icsbrec_ssl_set_ssl_protocols(ssl, ctx->argv[4]); + icsbrec_ssl_set_ssl_ciphers(ssl, ctx->argv[5]); + } + + icsbrec_ic_sb_global_set_ssl(ic_sb_global, ssl); +} + + +static const struct ctl_table_class tables[ICSBREC_N_TABLES] = { + [ICSBREC_TABLE_AVAILABILITY_ZONE].row_ids[0] = + {&icsbrec_availability_zone_col_name, NULL, NULL}, + + [ICSBREC_TABLE_GATEWAY].row_ids[0] = + {&icsbrec_gateway_col_name, NULL, NULL}, + + [ICSBREC_TABLE_PORT_BINDING].row_ids[0] = + {&icsbrec_port_binding_col_logical_port, NULL, NULL}, + + [ICSBREC_TABLE_DATAPATH_BINDING].row_ids[0] = + {&icsbrec_datapath_binding_col_transit_switch, NULL, NULL}, +}; + + +static void +ic_sbctl_context_init_command(struct ic_sbctl_context *ic_sbctl_ctx, + struct ctl_command *command) +{ + ctl_context_init_command(&ic_sbctl_ctx->base, command); +} + +static void +ic_sbctl_context_init(struct ic_sbctl_context *ic_sbctl_ctx, + struct ctl_command *command, struct ovsdb_idl *idl, + struct ovsdb_idl_txn *txn, + struct ovsdb_symbol_table *symtab) +{ + ctl_context_init(&ic_sbctl_ctx->base, command, idl, txn, symtab, NULL); + ic_sbctl_ctx->cache_valid = false; +} + +static void +ic_sbctl_context_done_command(struct ic_sbctl_context *ic_sbctl_ctx, + struct ctl_command *command) +{ + ctl_context_done_command(&ic_sbctl_ctx->base, command); +} + +static void +ic_sbctl_context_done(struct ic_sbctl_context *ic_sbctl_ctx, + struct ctl_command *command) +{ + ctl_context_done(&ic_sbctl_ctx->base, command); +} + +static void +run_prerequisites(struct ctl_command *commands, size_t n_commands, + struct ovsdb_idl *idl) +{ + ovsdb_idl_add_table(idl, &icsbrec_table_ic_sb_global); + + for (struct ctl_command *c = commands; c < &commands[n_commands]; c++) { + if (c->syntax->prerequisites) { + struct ic_sbctl_context ic_sbctl_ctx; + + ds_init(&c->output); + c->table = NULL; + + ic_sbctl_context_init(&ic_sbctl_ctx, c, idl, NULL, NULL); + (c->syntax->prerequisites)(&ic_sbctl_ctx.base); + if (ic_sbctl_ctx.base.error) { + ctl_fatal("%s", ic_sbctl_ctx.base.error); + } + ic_sbctl_context_done(&ic_sbctl_ctx, c); + + ovs_assert(!c->output.string); + ovs_assert(!c->table); + } + } +} + +static bool +do_ic_sbctl(const char *args, struct ctl_command *commands, size_t n_commands, + struct ovsdb_idl *idl) +{ + struct ovsdb_idl_txn *txn; + enum ovsdb_idl_txn_status status; + struct ovsdb_symbol_table *symtab; + struct ic_sbctl_context ic_sbctl_ctx; + struct ctl_command *c; + struct shash_node *node; + + txn = the_idl_txn = ovsdb_idl_txn_create(idl); + if (dry_run) { + ovsdb_idl_txn_set_dry_run(txn); + } + + ovsdb_idl_txn_add_comment(txn, "ovs-ic_sbctl: %s", args); + + const struct icsbrec_ic_sb_global *isb = icsbrec_ic_sb_global_first(idl); + if (!isb) { + /* XXX add verification that table is empty */ + icsbrec_ic_sb_global_insert(txn); + } + + symtab = ovsdb_symbol_table_create(); + for (c = commands; c < &commands[n_commands]; c++) { + ds_init(&c->output); + c->table = NULL; + } + ic_sbctl_context_init(&ic_sbctl_ctx, NULL, idl, txn, symtab); + for (c = commands; c < &commands[n_commands]; c++) { + ic_sbctl_context_init_command(&ic_sbctl_ctx, c); + if (c->syntax->run) { + (c->syntax->run)(&ic_sbctl_ctx.base); + } + if (ic_sbctl_ctx.base.error) { + ctl_fatal("%s", ic_sbctl_ctx.base.error); + } + ic_sbctl_context_done_command(&ic_sbctl_ctx, c); + + if (ic_sbctl_ctx.base.try_again) { + ic_sbctl_context_done(&ic_sbctl_ctx, NULL); + goto try_again; + } + } + ic_sbctl_context_done(&ic_sbctl_ctx, NULL); + + SHASH_FOR_EACH (node, &symtab->sh) { + struct ovsdb_symbol *symbol = node->data; + if (!symbol->created) { + ctl_fatal("row id \"%s\" is referenced but never created (e.g. " + "with \"-- --id=%s create ...\")", + node->name, node->name); + } + if (!symbol->strong_ref) { + if (!symbol->weak_ref) { + VLOG_WARN("row id \"%s\" was created but no reference to it " + "was inserted, so it will not actually appear in " + "the database", node->name); + } else { + VLOG_WARN("row id \"%s\" was created but only a weak " + "reference to it was inserted, so it will not " + "actually appear in the database", node->name); + } + } + } + + status = ovsdb_idl_txn_commit_block(txn); + if (status == TXN_UNCHANGED || status == TXN_SUCCESS) { + for (c = commands; c < &commands[n_commands]; c++) { + if (c->syntax->postprocess) { + ic_sbctl_context_init(&ic_sbctl_ctx, c, idl, txn, symtab); + (c->syntax->postprocess)(&ic_sbctl_ctx.base); + if (ic_sbctl_ctx.base.error) { + ctl_fatal("%s", ic_sbctl_ctx.base.error); + } + ic_sbctl_context_done(&ic_sbctl_ctx, c); + } + } + } + + switch (status) { + case TXN_UNCOMMITTED: + case TXN_INCOMPLETE: + OVS_NOT_REACHED(); + + case TXN_ABORTED: + /* Should not happen--we never call ovsdb_idl_txn_abort(). */ + ctl_fatal("transaction aborted"); + + case TXN_UNCHANGED: + case TXN_SUCCESS: + break; + + case TXN_TRY_AGAIN: + goto try_again; + + case TXN_ERROR: + ctl_fatal("transaction error: %s", ovsdb_idl_txn_get_error(txn)); + + case TXN_NOT_LOCKED: + /* Should not happen--we never call ovsdb_idl_set_lock(). */ + ctl_fatal("database not locked"); + + default: + OVS_NOT_REACHED(); + } + + ovsdb_symbol_table_destroy(symtab); + + for (c = commands; c < &commands[n_commands]; c++) { + struct ds *ds = &c->output; + + if (c->table) { + table_print(c->table, &table_style); + } else if (oneline) { + size_t j; + + ds_chomp(ds, '\n'); + for (j = 0; j < ds->length; j++) { + int ch = ds->string[j]; + switch (ch) { + case '\n': + fputs("\\n", stdout); + break; + + case '\\': + fputs("\\\\", stdout); + break; + + default: + putchar(ch); + } + } + putchar('\n'); + } else { + fputs(ds_cstr(ds), stdout); + } + ds_destroy(&c->output); + table_destroy(c->table); + free(c->table); + + shash_destroy_free_data(&c->options); + } + free(commands); + ovsdb_idl_txn_destroy(txn); + ovsdb_idl_destroy(idl); + + return true; + +try_again: + /* Our transaction needs to be rerun, or a prerequisite was not met. Free + * resources and return so that the caller can try again. */ + ovsdb_idl_txn_abort(txn); + ovsdb_idl_txn_destroy(txn); + the_idl_txn = NULL; + + ovsdb_symbol_table_destroy(symtab); + for (c = commands; c < &commands[n_commands]; c++) { + ds_destroy(&c->output); + table_destroy(c->table); + free(c->table); + } + return false; +} + +/* Frees the current transaction and the underlying IDL and then calls + * exit(status). + * + * Freeing the transaction and the IDL is not strictly necessary, but it makes + * for a clean memory leak report from valgrind in the normal case. That makes + * it easier to notice real memory leaks. */ +static void +ic_sbctl_exit(int status) +{ + if (the_idl_txn) { + ovsdb_idl_txn_abort(the_idl_txn); + ovsdb_idl_txn_destroy(the_idl_txn); + } + ovsdb_idl_destroy(the_idl); + exit(status); +} + +static const struct ctl_command_syntax ic_sbctl_commands[] = { + { "init", 0, 0, "", NULL, ic_sbctl_init, NULL, "", RW }, + { "show", 0, 1, "[AZ]", NULL, ic_sbctl_show, NULL, "", RO }, + + /* Connection commands. */ + {"get-connection", 0, 0, "", pre_connection, cmd_get_connection, NULL, "", + RO}, + {"del-connection", 0, 0, "", pre_connection, cmd_del_connection, NULL, "", + RW}, + {"set-connection", 1, INT_MAX, "TARGET...", pre_connection, + cmd_set_connection, NULL, "--inactivity-probe=", RW}, + + /* SSL commands. */ + {"get-ssl", 0, 0, "", pre_cmd_get_ssl, cmd_get_ssl, NULL, "", RO}, + {"del-ssl", 0, 0, "", pre_cmd_del_ssl, cmd_del_ssl, NULL, "", RW}, + {"set-ssl", 3, 5, + "PRIVATE-KEY CERTIFICATE CA-CERT [SSL-PROTOS [SSL-CIPHERS]]", + pre_cmd_set_ssl, cmd_set_ssl, NULL, "--bootstrap", RW}, + + {NULL, 0, 0, NULL, NULL, NULL, NULL, NULL, RO}, +}; + +/* Registers ic_sbctl and common db commands. */ +static void +ic_sbctl_cmd_init(void) +{ + ctl_init(&icsbrec_idl_class, icsbrec_table_classes, tables, + NULL, ic_sbctl_exit); + ctl_register_commands(ic_sbctl_commands); +} From patchwork Wed Jan 29 19:55:59 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Han Zhou X-Patchwork-Id: 1231087 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.133; helo=hemlock.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=ovn.org Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 487DmL1wySz9sPJ for ; Thu, 30 Jan 2020 06:57:38 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id BB63588313; Wed, 29 Jan 2020 19:57:36 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id krEJb5TKqDvV; Wed, 29 Jan 2020 19:57:25 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by hemlock.osuosl.org (Postfix) with ESMTP id 916F788349; Wed, 29 Jan 2020 19:56:57 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 766C3C1D8B; Wed, 29 Jan 2020 19:56:57 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by lists.linuxfoundation.org (Postfix) with ESMTP id D5390C0881 for ; Wed, 29 Jan 2020 19:56:54 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by whitealder.osuosl.org (Postfix) with ESMTP id B1F37876D5 for ; Wed, 29 Jan 2020 19:56:54 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from whitealder.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Jk95pEUHad2e for ; Wed, 29 Jan 2020 19:56:33 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from relay1-d.mail.gandi.net (relay1-d.mail.gandi.net [217.70.183.193]) by whitealder.osuosl.org (Postfix) with ESMTPS id 25195876B4 for ; Wed, 29 Jan 2020 19:56:30 +0000 (UTC) X-Originating-IP: 216.113.160.77 Received: from localhost.localdomain.localdomain (unknown [216.113.160.77]) (Authenticated sender: hzhou@ovn.org) by relay1-d.mail.gandi.net (Postfix) with ESMTPSA id 99112240003; Wed, 29 Jan 2020 19:56:28 +0000 (UTC) From: Han Zhou To: dev@openvswitch.org Date: Wed, 29 Jan 2020 11:55:59 -0800 Message-Id: <1580327768-36501-5-git-send-email-hzhou@ovn.org> X-Mailer: git-send-email 2.1.0 In-Reply-To: <1580327768-36501-1-git-send-email-hzhou@ovn.org> References: <1580327768-36501-1-git-send-email-hzhou@ovn.org> Cc: Han Zhou Subject: [ovs-dev] [PATCH ovn v4 04/13] ovn-ic: Interconnection controller with AZ registeration. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" This patch introduces interconnection controller, ovn-ic, and implements the basic AZ registration feature: taking the AZ name from NB DB and create an Availability_Zone entry in IC-SB DB. Signed-off-by: Han Zhou --- Makefile.am | 1 + ic/.gitignore | 2 + ic/automake.mk | 10 ++ ic/ovn-ic.8.xml | 120 +++++++++++++ ic/ovn-ic.c | 467 +++++++++++++++++++++++++++++++++++++++++++++++++++ ovn-nb.ovsschema | 5 +- ovn-nb.xml | 7 + tests/automake.mk | 4 +- tests/ovn-ic.at | 28 +++ tests/ovn-macros.at | 161 +++++++++++++++--- tests/testsuite.at | 1 + tutorial/ovs-sandbox | 78 ++++++++- 12 files changed, 854 insertions(+), 30 deletions(-) create mode 100644 ic/.gitignore create mode 100644 ic/automake.mk create mode 100644 ic/ovn-ic.8.xml create mode 100644 ic/ovn-ic.c create mode 100644 tests/ovn-ic.at diff --git a/Makefile.am b/Makefile.am index 8eed7a7..d524c0d 100644 --- a/Makefile.am +++ b/Makefile.am @@ -500,4 +500,5 @@ include selinux/automake.mk include controller/automake.mk include controller-vtep/automake.mk include northd/automake.mk +include ic/automake.mk include build-aux/automake.mk diff --git a/ic/.gitignore b/ic/.gitignore new file mode 100644 index 0000000..1b73eb4 --- /dev/null +++ b/ic/.gitignore @@ -0,0 +1,2 @@ +/ovn-ic +/ovn-ic.8 diff --git a/ic/automake.mk b/ic/automake.mk new file mode 100644 index 0000000..8e71bc3 --- /dev/null +++ b/ic/automake.mk @@ -0,0 +1,10 @@ +# ovn-ic +bin_PROGRAMS += ic/ovn-ic +ic_ovn_ic_SOURCES = ic/ovn-ic.c +ic_ovn_ic_LDADD = \ + lib/libovn.la \ + $(OVSDB_LIBDIR)/libovsdb.la \ + $(OVS_LIBDIR)/libopenvswitch.la +man_MANS += ic/ovn-ic.8 +EXTRA_DIST += ic/ovn-ic.8.xml +CLEANFILES += ic/ovn-ic.8 diff --git a/ic/ovn-ic.8.xml b/ic/ovn-ic.8.xml new file mode 100644 index 0000000..f5ef994 --- /dev/null +++ b/ic/ovn-ic.8.xml @@ -0,0 +1,120 @@ + + +

Name

+

ovn-ic -- Open Virtual Network interconnection controller

+ +

Synopsis

+

ovn-ic [options]

+ +

Description

+

+ ovn-ic, OVN interconnection controller, is a centralized + daemon which communicates with global interconnection databases IC_NB/IC_SB + to configure and exchange data with local NB/SB for interconnecting + with other OVN deployments. +

+ +

Options

+
+
--ovnnb-db=database
+
+ The OVSDB database containing the OVN Northbound Database. If the + OVN_NB_DB environment variable is set, its value is used + as the default. Otherwise, the default is + unix:@RUNDIR@/ovnnb_db.sock. +
+
--ovnsb-db=database
+
+ The OVSDB database containing the OVN Southbound Database. If the + OVN_SB_DB environment variable is set, its value is used + as the default. Otherwise, the default is + unix:@RUNDIR@/ovnsb_db.sock. +
+
--ic-nb-db=database
+
+ The OVSDB database containing the OVN Interconnection Northbound + Database. If the OVN_IC_NB_DB environment variable is set, + its value is used as the default. Otherwise, the default is + unix:@RUNDIR@/ovn_ic_nb_db.sock. +
+
--ic-sb-db=database
+
+ The OVSDB database containing the OVN Interconnection Southbound + Database. If the OVN_IC_SB_DB environment variable is set, + its value is used as the default. Otherwise, the default is + unix:@RUNDIR@/ovn_ic_sb_db.sock. +
+
+

+ database in the above options must be an OVSDB active or + passive connection method, as described in ovsdb(7). +

+ +

Daemon Options

+ + +

Logging Options

+ + +

PKI Options

+

+ PKI configuration is required in order to use SSL for the connections to + the Northbound and Southbound databases. +

+ + +

Other Options

+ +

+ + +

Runtime Management Commands

+

+ ovs-appctl can send commands to a running + ovn-ic process. The currently supported commands + are described below. +

+
exit
+
+ Causes ovn-ic to gracefully terminate. +
+ +
pause
+
+ Pauses the ovn-ic operation from processing any database changes. + This will also instruct ovn-ic to drop any lock on SB DB. +
+ +
resume
+
+ Resumes the ovn-ic operation to process database contents. This will + also instruct ovn-northd to aspire for the lock on SB DB. +
+ +
is-paused
+
+ Returns "true" if ovn-ic is currently paused, "false" otherwise. +
+ +
status
+
+ Prints this server's status. Status will be "active" if ovn-ic has + acquired OVSDB lock on SB DB, "standby" if it has not or "paused" if + this instance is paused. +
+
+ +

+ +

Active-Standby for High Availability

+

+ You may run ovn-ic more than once in an OVN deployment. + When connected to a standalone or clustered DB setup, OVN will + automatically ensure that only one of them is active at a time. + If multiple instances of ovn-ic are running and the + active ovn-ic fails, one of the hot standby instances + of ovn-ic will automatically take over. +

+ diff --git a/ic/ovn-ic.c b/ic/ovn-ic.c new file mode 100644 index 0000000..a6d27c0 --- /dev/null +++ b/ic/ovn-ic.c @@ -0,0 +1,467 @@ +/* + * Copyright (c) 2020 eBay Inc. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at: + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#include + +#include +#include +#include + +#include "bitmap.h" +#include "command-line.h" +#include "daemon.h" +#include "dirs.h" +#include "openvswitch/dynamic-string.h" +#include "fatal-signal.h" +#include "hash.h" +#include "openvswitch/hmap.h" +#include "lib/ovn-ic-nb-idl.h" +#include "lib/ovn-ic-sb-idl.h" +#include "lib/ovn-nb-idl.h" +#include "lib/ovn-sb-idl.h" +#include "lib/ovn-util.h" +#include "openvswitch/poll-loop.h" +#include "smap.h" +#include "sset.h" +#include "stream.h" +#include "stream-ssl.h" +#include "unixctl.h" +#include "util.h" +#include "uuid.h" +#include "openvswitch/vlog.h" + +VLOG_DEFINE_THIS_MODULE(ovn_ic); + +static unixctl_cb_func ovn_ic_exit; +static unixctl_cb_func ovn_ic_pause; +static unixctl_cb_func ovn_ic_resume; +static unixctl_cb_func ovn_ic_is_paused; +static unixctl_cb_func ovn_ic_status; + +struct ic_context { + struct ovsdb_idl *ovnnb_idl; + struct ovsdb_idl *ovnsb_idl; + struct ovsdb_idl *ovninb_idl; + struct ovsdb_idl *ovnisb_idl; + struct ovsdb_idl_txn *ovnnb_txn; + struct ovsdb_idl_txn *ovnsb_txn; + struct ovsdb_idl_txn *ovninb_txn; + struct ovsdb_idl_txn *ovnisb_txn; +}; + +struct ic_state { + bool had_lock; + bool paused; +}; + +static const char *ovnnb_db; +static const char *ovnsb_db; +static const char *ovn_ic_nb_db; +static const char *ovn_ic_sb_db; +static const char *unixctl_path; + + +static void +usage(void) +{ + printf("\ +%s: OVN interconnection management daemon\n\ +usage: %s [OPTIONS]\n\ +\n\ +Options:\n\ + --ovnnb-db=DATABASE connect to ovn-nb database at DATABASE\n\ + (default: %s)\n\ + --ovnsb-db=DATABASE connect to ovn-sb database at DATABASE\n\ + (default: %s)\n\ + --unixctl=SOCKET override default control socket name\n\ + -h, --help display this help message\n\ + -o, --options list available options\n\ + -V, --version display version information\n\ +", program_name, program_name, default_nb_db(), default_sb_db()); + daemon_usage(); + vlog_usage(); + stream_usage("database", true, true, false); +} + +static const struct icsbrec_availability_zone * +az_run(struct ic_context *ctx) +{ + const struct nbrec_nb_global *nb_global = + nbrec_nb_global_first(ctx->ovnnb_idl); + + if (!nb_global) { + VLOG_INFO("NB Global not exist."); + return NULL; + } + + /* Delete old AZ if name changes. Note: if name changed when ovn-ic + * is not running, one has to manually delete the old AZ with: + * "ovn-ic-sbctl destroy avail ". */ + static char *az_name; + const struct icsbrec_availability_zone *az; + if (az_name && strcmp(az_name, nb_global->name)) { + ICSBREC_AVAILABILITY_ZONE_FOR_EACH (az, ctx->ovnisb_idl) { + if (!strcmp(az->name, az_name)) { + icsbrec_availability_zone_delete(az); + break; + } + } + free(az_name); + az_name = NULL; + } + + if (!nb_global->name[0]) { + return NULL; + } + + if (!az_name) { + az_name = xstrdup(nb_global->name); + } + + ICSBREC_AVAILABILITY_ZONE_FOR_EACH (az, ctx->ovnisb_idl) { + if (!strcmp(az->name, az_name)) { + return az; + } + } + + /* Create AZ in ISB */ + if (ctx->ovnisb_txn) { + VLOG_INFO("Register AZ %s to interconnection DB.", az_name); + az = icsbrec_availability_zone_insert(ctx->ovnisb_txn); + icsbrec_availability_zone_set_name(az, az_name); + return az; + } + return NULL; +} + +static void +ovn_db_run(struct ic_context *ctx) +{ + const struct icsbrec_availability_zone *az = az_run(ctx); + VLOG_DBG("Availability zone: %s", az ? az->name : "not created yet."); +} + +static void +parse_options(int argc OVS_UNUSED, char *argv[] OVS_UNUSED) +{ + enum { + DAEMON_OPTION_ENUMS, + VLOG_OPTION_ENUMS, + SSL_OPTION_ENUMS, + }; + static const struct option long_options[] = { + {"ovnsb-db", required_argument, NULL, 'd'}, + {"ovnnb-db", required_argument, NULL, 'D'}, + {"ic-sb-db", required_argument, NULL, 'i'}, + {"ic-nb-db", required_argument, NULL, 'I'}, + {"unixctl", required_argument, NULL, 'u'}, + {"help", no_argument, NULL, 'h'}, + {"options", no_argument, NULL, 'o'}, + {"version", no_argument, NULL, 'V'}, + DAEMON_LONG_OPTIONS, + VLOG_LONG_OPTIONS, + STREAM_SSL_LONG_OPTIONS, + {NULL, 0, NULL, 0}, + }; + char *short_options = ovs_cmdl_long_options_to_short_options(long_options); + + for (;;) { + int c; + + c = getopt_long(argc, argv, short_options, long_options, NULL); + if (c == -1) { + break; + } + + switch (c) { + DAEMON_OPTION_HANDLERS; + VLOG_OPTION_HANDLERS; + STREAM_SSL_OPTION_HANDLERS; + + case 'd': + ovnsb_db = optarg; + break; + + case 'D': + ovnnb_db = optarg; + break; + + case 'i': + ovn_ic_sb_db = optarg; + break; + + case 'I': + ovn_ic_nb_db = optarg; + break; + + case 'u': + unixctl_path = optarg; + break; + + case 'h': + usage(); + exit(EXIT_SUCCESS); + + case 'o': + ovs_cmdl_print_options(long_options); + exit(EXIT_SUCCESS); + + case 'V': + ovs_print_version(0, 0); + exit(EXIT_SUCCESS); + + default: + break; + } + } + + if (!ovnsb_db) { + ovnsb_db = default_sb_db(); + } + + if (!ovnnb_db) { + ovnnb_db = default_nb_db(); + } + + if (!ovn_ic_sb_db) { + ovn_ic_sb_db = default_ic_sb_db(); + } + + if (!ovn_ic_nb_db) { + ovn_ic_nb_db = default_ic_nb_db(); + } + + free(short_options); +} + +static void OVS_UNUSED +add_column_noalert(struct ovsdb_idl *idl, + const struct ovsdb_idl_column *column) +{ + ovsdb_idl_add_column(idl, column); + ovsdb_idl_omit_alert(idl, column); +} + +int +main(int argc, char *argv[]) +{ + int res = EXIT_SUCCESS; + struct unixctl_server *unixctl; + int retval; + bool exiting; + struct ic_state state; + + fatal_ignore_sigpipe(); + ovs_cmdl_proctitle_init(argc, argv); + set_program_name(argv[0]); + service_start(&argc, &argv); + parse_options(argc, argv); + + daemonize_start(false); + + if (!unixctl_path) { + char *abs_unixctl_path = get_abs_unix_ctl_path(); + retval = unixctl_server_create(abs_unixctl_path, &unixctl); + free(abs_unixctl_path); + } else { + retval = unixctl_server_create(unixctl_path, &unixctl); + } + + if (retval) { + exit(EXIT_FAILURE); + } + unixctl_command_register("exit", "", 0, 0, ovn_ic_exit, &exiting); + unixctl_command_register("pause", "", 0, 0, ovn_ic_pause, &state); + unixctl_command_register("resume", "", 0, 0, ovn_ic_resume, &state); + unixctl_command_register("is-paused", "", 0, 0, ovn_ic_is_paused, &state); + unixctl_command_register("status", "", 0, 0, ovn_ic_status, &state); + + daemonize_complete(); + + /* ovn-ic-nb db. */ + struct ovsdb_idl_loop ovninb_idl_loop = OVSDB_IDL_LOOP_INITIALIZER( + ovsdb_idl_create(ovn_ic_nb_db, &icnbrec_idl_class, true, true)); + + /* ovn-ic-sb db. */ + struct ovsdb_idl_loop ovnisb_idl_loop = OVSDB_IDL_LOOP_INITIALIZER( + ovsdb_idl_create(ovn_ic_sb_db, &icsbrec_idl_class, true, true)); + + /* ovn-nb db. XXX: add only needed tables and columns */ + struct ovsdb_idl_loop ovnnb_idl_loop = OVSDB_IDL_LOOP_INITIALIZER( + ovsdb_idl_create(ovnnb_db, &nbrec_idl_class, true, true)); + + /* ovn-sb db. XXX: add only needed tables and columns */ + struct ovsdb_idl_loop ovnsb_idl_loop = OVSDB_IDL_LOOP_INITIALIZER( + ovsdb_idl_create(ovnsb_db, &sbrec_idl_class, true, true)); + + /* Main loop. */ + exiting = false; + state.had_lock = false; + state.paused = false; + while (!exiting) { + if (!state.paused) { + if (!ovsdb_idl_has_lock(ovnsb_idl_loop.idl) && + !ovsdb_idl_is_lock_contended(ovnsb_idl_loop.idl)) + { + /* Ensure that only a single ovn-ic is active in the deployment + * by acquiring a lock called "ovn_ic" on the southbound + * database and then only performing DB transactions if the + * lock is held. */ + ovsdb_idl_set_lock(ovnsb_idl_loop.idl, "ovn_ic"); + } + + struct ic_context ctx = { + .ovnnb_idl = ovnnb_idl_loop.idl, + .ovnnb_txn = ovsdb_idl_loop_run(&ovnnb_idl_loop), + .ovnsb_idl = ovnsb_idl_loop.idl, + .ovnsb_txn = ovsdb_idl_loop_run(&ovnsb_idl_loop), + .ovninb_idl = ovninb_idl_loop.idl, + .ovninb_txn = ovsdb_idl_loop_run(&ovninb_idl_loop), + .ovnisb_idl = ovnisb_idl_loop.idl, + .ovnisb_txn = ovsdb_idl_loop_run(&ovnisb_idl_loop), + }; + + if (!state.had_lock && ovsdb_idl_has_lock(ovnsb_idl_loop.idl)) { + VLOG_INFO("ovn-ic lock acquired. " + "This ovn-ic instance is now active."); + state.had_lock = true; + } else if (state.had_lock && + !ovsdb_idl_has_lock(ovnsb_idl_loop.idl)) { + VLOG_INFO("ovn-ic lock lost. " + "This ovn-ic instance is now on standby."); + state.had_lock = false; + } + + if (ovsdb_idl_has_lock(ovnsb_idl_loop.idl)) { + ovn_db_run(&ctx); + } + + ovsdb_idl_loop_commit_and_wait(&ovnnb_idl_loop); + ovsdb_idl_loop_commit_and_wait(&ovnsb_idl_loop); + ovsdb_idl_loop_commit_and_wait(&ovninb_idl_loop); + ovsdb_idl_loop_commit_and_wait(&ovnisb_idl_loop); + } else { + /* ovn-ic is paused + * - we still want to handle any db updates and update the + * local IDL. Otherwise, when it is resumed, the local IDL + * copy will be out of sync. + * - but we don't want to create any txns. + * */ + if (ovsdb_idl_has_lock(ovnsb_idl_loop.idl) || + ovsdb_idl_is_lock_contended(ovnsb_idl_loop.idl)) + { + /* make sure we don't hold the lock while paused */ + VLOG_INFO("This ovn-ic instance is now paused."); + ovsdb_idl_set_lock(ovnsb_idl_loop.idl, NULL); + state.had_lock = false; + } + + ovsdb_idl_run(ovnnb_idl_loop.idl); + ovsdb_idl_run(ovnsb_idl_loop.idl); + ovsdb_idl_run(ovninb_idl_loop.idl); + ovsdb_idl_run(ovnisb_idl_loop.idl); + ovsdb_idl_wait(ovnnb_idl_loop.idl); + ovsdb_idl_wait(ovnsb_idl_loop.idl); + ovsdb_idl_wait(ovninb_idl_loop.idl); + ovsdb_idl_wait(ovnisb_idl_loop.idl); + } + + unixctl_server_run(unixctl); + unixctl_server_wait(unixctl); + if (exiting) { + poll_immediate_wake(); + } + + poll_block(); + if (should_service_stop()) { + exiting = true; + } + } + + unixctl_server_destroy(unixctl); + ovsdb_idl_loop_destroy(&ovnnb_idl_loop); + ovsdb_idl_loop_destroy(&ovnsb_idl_loop); + ovsdb_idl_loop_destroy(&ovninb_idl_loop); + ovsdb_idl_loop_destroy(&ovnisb_idl_loop); + service_stop(); + + exit(res); +} + +static void +ovn_ic_exit(struct unixctl_conn *conn, int argc OVS_UNUSED, + const char *argv[] OVS_UNUSED, void *exiting_) +{ + bool *exiting = exiting_; + *exiting = true; + + unixctl_command_reply(conn, NULL); +} + +static void +ovn_ic_pause(struct unixctl_conn *conn, int argc OVS_UNUSED, + const char *argv[] OVS_UNUSED, void *state_) +{ + struct ic_state *state = state_; + state->paused = true; + + unixctl_command_reply(conn, NULL); +} + +static void +ovn_ic_resume(struct unixctl_conn *conn, int argc OVS_UNUSED, + const char *argv[] OVS_UNUSED, void *state_) +{ + struct ic_state *state = state_; + state->paused = false; + + unixctl_command_reply(conn, NULL); +} + +static void +ovn_ic_is_paused(struct unixctl_conn *conn, int argc OVS_UNUSED, + const char *argv[] OVS_UNUSED, void *state_) +{ + struct ic_state *state = state_; + if (state->paused) { + unixctl_command_reply(conn, "true"); + } else { + unixctl_command_reply(conn, "false"); + } +} + +static void +ovn_ic_status(struct unixctl_conn *conn, int argc OVS_UNUSED, + const char *argv[] OVS_UNUSED, void *state_) +{ + struct ic_state *state = state_; + char *status; + + if (state->paused) { + status = "paused"; + } else { + status = state->had_lock ? "active" : "standby"; + } + + /* + * Use a labelled formatted output so we can add more to the status command + * later without breaking any consuming scripts + */ + struct ds s = DS_EMPTY_INITIALIZER; + ds_put_format(&s, "Status: %s\n", status); + unixctl_command_reply(conn, ds_cstr(&s)); + ds_destroy(&s); +} diff --git a/ovn-nb.ovsschema b/ovn-nb.ovsschema index 7d969fb..bbd6c25 100644 --- a/ovn-nb.ovsschema +++ b/ovn-nb.ovsschema @@ -1,10 +1,11 @@ { "name": "OVN_Northbound", - "version": "5.19.0", - "cksum": "4258826789 24879", + "version": "5.20.0", + "cksum": "987891875 24923", "tables": { "NB_Global": { "columns": { + "name": {"type": "string"}, "nb_cfg": {"type": {"key": "integer"}}, "sb_cfg": {"type": {"key": "integer"}}, "hv_cfg": {"type": {"key": "integer"}}, diff --git a/ovn-nb.xml b/ovn-nb.xml index 7ff0bde..6f8baa0 100644 --- a/ovn-nb.xml +++ b/ovn-nb.xml @@ -36,6 +36,13 @@ one row.

+ + + The name of the OVN cluster, which uniquely identifies the OVN cluster + throughout all OVN clusters supposed to interconnect with each other. + + + These columns allow a client to track the overall configuration state of the system. diff --git a/tests/automake.mk b/tests/automake.mk index c70a213..215fb43 100644 --- a/tests/automake.mk +++ b/tests/automake.mk @@ -28,6 +28,7 @@ TESTSUITE_AT = \ tests/ovn-ic-sbctl.at \ tests/ovn-controller.at \ tests/ovn-controller-vtep.at \ + tests/ovn-ic.at \ tests/ovn-macros.at \ tests/ovn-performance.at @@ -54,7 +55,7 @@ SYSTEM_KMOD_TESTSUITE = $(srcdir)/tests/system-kmod-testsuite SYSTEM_USERSPACE_TESTSUITE = $(srcdir)/tests/system-userspace-testsuite DISTCLEANFILES += tests/atconfig tests/atlocal -AUTOTEST_PATH = $(ovs_builddir)/utilities:$(ovs_builddir)/vswitchd:$(ovs_builddir)/ovsdb:$(ovs_builddir)/vtep:tests:$(PTHREAD_WIN32_DIR_DLL):$(SSL_DIR):controller-vtep:northd:utilities:controller +AUTOTEST_PATH = $(ovs_builddir)/utilities:$(ovs_builddir)/vswitchd:$(ovs_builddir)/ovsdb:$(ovs_builddir)/vtep:tests:$(PTHREAD_WIN32_DIR_DLL):$(SSL_DIR):controller-vtep:northd:utilities:controller:ic export ovs_srcdir @@ -103,6 +104,7 @@ valgrind_wrappers = \ tests/valgrind/ovn-sbctl \ tests/valgrind/ovn-ic-nbctl \ tests/valgrind/ovn-ic-sbctl \ + tests/valgrind/ovn-ic \ tests/valgrind/ovs-appctl \ tests/valgrind/ovs-ofctl \ tests/valgrind/ovs-vsctl \ diff --git a/tests/ovn-ic.at b/tests/ovn-ic.at new file mode 100644 index 0000000..b20105a --- /dev/null +++ b/tests/ovn-ic.at @@ -0,0 +1,28 @@ +AT_BANNER([OVN Interconnection Controller]) +AT_SETUP([ovn-ic -- AZ register]) + +ovn_init_ic_db +ovn_start az1 +ovn_start az2 + +AT_CHECK([ovn-ic-sbctl show], [0], [dnl +availability-zone az1 +availability-zone az2 +]) + +ovn_as az1 +ovn-nbctl set NB_Global . name=az3 +AT_CHECK([ovn-ic-sbctl show], [0], [dnl +availability-zone az2 +availability-zone az3 +]) + +ovn_as az2 +ovn-nbctl set NB_Global . name=\"\" +AT_CHECK([ovn-ic-sbctl show], [0], [dnl +availability-zone az3 +]) + +OVN_CLEANUP_IC([az1], [az2]) + +AT_CLEANUP diff --git a/tests/ovn-macros.at b/tests/ovn-macros.at index 4045ce0..53a22a8 100644 --- a/tests/ovn-macros.at +++ b/tests/ovn-macros.at @@ -48,9 +48,49 @@ m4_define([OVN_CLEANUP],[ OVN_CLEANUP_VSWITCH([main]) ]) +# OVN_CLEANUP_AZ(az) +# +# Gracefully terminate all OVN daemons, including those in the +# specified sandbox instances. +m4_define([OVN_CLEANUP_AZ],[ + as $1/ovn-sb + OVS_APP_EXIT_AND_WAIT([ovsdb-server]) + + as $1/ovn-nb + OVS_APP_EXIT_AND_WAIT([ovsdb-server]) + + as $1/northd + OVS_APP_EXIT_AND_WAIT([ovn-northd]) + + as $1/northd-backup + OVS_APP_EXIT_AND_WAIT([ovn-northd]) + + as $1/ic + OVS_APP_EXIT_AND_WAIT([ovn-ic]) +]) + +# OVN_CLEANUP_IC([az ...]) +# +# Gracefully terminate all interconnection DBs, and daemons in the +# specified AZs, if any. +m4_define([OVN_CLEANUP_IC],[ + m4_foreach([az], [$@], [ + OVN_CLEANUP_AZ([az]) + ]) + as ovn-ic-sb + OVS_APP_EXIT_AND_WAIT([ovsdb-server]) + + as ovn-ic-nb + OVS_APP_EXIT_AND_WAIT([ovsdb-server]) + + if test -d "$ovs_base"/main; then + OVN_CLEANUP_VSWITCH([main]) + fi +]) + m4_divert_push([PREPARE_TESTS]) -# ovn_init_db DATABASE +# ovn_init_db DATABASE [AZ] # # Creates and initializes the given DATABASE (one of "ovn-sb" or "ovn-nb"), # starts its ovsdb-server instance, and sets the appropriate environment @@ -60,36 +100,77 @@ m4_divert_push([PREPARE_TESTS]) # Usually invoked from ovn_start. ovn_init_db () { echo "creating $1 database" - local d=$ovs_base/$1 + local as_d=$1 + if test -n "$2"; then + as_d=$2/$as_d + fi + local d=$ovs_base/$as_d mkdir "$d" || return 1 : > "$d"/.$1.db.~lock~ - as $1 ovsdb-tool create "$d"/$1.db "$abs_top_srcdir"/$1.ovsschema - as $1 start_daemon ovsdb-server --remote=punix:"$d"/$1.sock "$d"/$1.db + as $as_d ovsdb-tool create "$d"/$1.db "$abs_top_srcdir"/$1.ovsschema + as $as_d start_daemon ovsdb-server --remote=punix:"$d"/$1.sock "$d"/$1.db local var=`echo $1_db | tr a-z- A-Z_` - AS_VAR_SET([$var], [unix:$ovs_base/$1/$1.sock]); export $var + AS_VAR_SET([$var], [unix:"$d"/$1.sock]); export $var } -# ovn_start +# ovn_init_ic_db +# +# Creates and initializes ovn-ic-nb and ovn-ic-sb databases and starts their +# ovsdb-server instances, for OVN interconnection. +ovn_init_ic_db () { + ovn_init_db ovn-ic-nb + ovn_init_db ovn-ic-sb +} + +# ovn_start [AZ] # # Creates and initializes ovn-sb and ovn-nb databases and starts their # ovsdb-server instance, sets appropriate environment variables so that # ovn-sbctl and ovn-nbctl use them by default, and starts ovn-northd running # against them. ovn_start () { - ovn_init_db ovn-sb; ovn-sbctl init - ovn_init_db ovn-nb; ovn-nbctl init + if test -n "$1"; then + mkdir "$ovs_base"/$1 + fi + ovn_init_db ovn-sb $1; ovn-sbctl init + ovn_init_db ovn-nb $1; ovn-nbctl init + if test -n "$1"; then + ovn-nbctl set NB_Global . name=$1 + fi + local ovn_sb_db=$OVN_SB_DB + local ovn_nb_db=$OVN_NB_DB + + local as_d=northd + if test -n "$1"; then + as_d=$1/$as_d + fi echo "starting ovn-northd" - mkdir "$ovs_base"/northd - as northd start_daemon ovn-northd -v \ - --ovnnb-db=unix:"$ovs_base"/ovn-nb/ovn-nb.sock \ - --ovnsb-db=unix:"$ovs_base"/ovn-sb/ovn-sb.sock + mkdir "$ovs_base"/$as_d + as $as_d start_daemon ovn-northd -v \ + --ovnnb-db=$ovn_nb_db \ + --ovnsb-db=$ovn_sb_db + as_d=northd-backup + if test -n "$1"; then + as_d=$1/$as_d + fi echo "starting backup ovn-northd" - mkdir "$ovs_base"/northd-backup - as northd-backup start_daemon ovn-northd -v \ - --ovnnb-db=unix:"$ovs_base"/ovn-nb/ovn-nb.sock \ - --ovnsb-db=unix:"$ovs_base"/ovn-sb/ovn-sb.sock + mkdir "$ovs_base"/$as_d + as $as_d start_daemon ovn-northd -v \ + --ovnnb-db=$ovn_nb_db \ + --ovnsb-db=$ovn_sb_db + + if test -n "$1"; then + as_d=$1/ic + echo "starting ovn-ic" + mkdir "$ovs_base"/$as_d + as $as_d start_daemon ovn-ic -v \ + --ovnnb-db=$ovn_nb_db \ + --ovnsb-db=$ovn_sb_db \ + --ic-nb-db=unix:"$ovs_base"/ovn-ic-nb/ovn-ic-nb.sock \ + --ic-sb-db=unix:"$ovs_base"/ovn-ic-sb/ovn-ic-sb.sock + fi } # Interconnection networks. @@ -132,24 +213,25 @@ net_attach () { || return 1 } -# ovn_attach NETWORK BRIDGE IP [MASKLEN] -# -# First, this command attaches BRIDGE to interconnection network NETWORK, just -# like "net_attach NETWORK BRIDGE". Second, it configures (simulated) IP -# address IP (with network mask length MASKLEN, which defaults to 24) on -# BRIDGE. Finally, it configures the Open vSwitch database to work with OVN -# and starts ovn-controller. -ovn_attach() { - local net=$1 bridge=$2 ip=$3 masklen=${4-24} +# ovn_az_attach AZ NETWORK BRIDGE IP [MASKLEN] +ovn_az_attach() { + local az=$1 net=$2 bridge=$3 ip=$4 masklen=${5-24} net_attach $net $bridge || return 1 mac=`ovs-vsctl get Interface $bridge mac_in_use | sed s/\"//g` arp_table="$arp_table $sandbox,$bridge,$ip,$mac" ovs-appctl netdev-dummy/ip4addr $bridge $ip/$masklen >/dev/null || return 1 ovs-appctl ovs/route/add $ip/$masklen $bridge >/dev/null || return 1 + + local ovn_remote + if test X"$az" = XNONE; then + ovn_remote=unix:$ovs_base/ovn-sb/ovn-sb.sock + else + ovn_remote=unix:$ovs_base/$az/ovn-sb/ovn-sb.sock + fi ovs-vsctl \ -- set Open_vSwitch . external-ids:system-id=$sandbox \ - -- set Open_vSwitch . external-ids:ovn-remote=unix:$ovs_base/ovn-sb/ovn-sb.sock \ + -- set Open_vSwitch . external-ids:ovn-remote=$ovn_remote \ -- set Open_vSwitch . external-ids:ovn-encap-type=geneve,vxlan \ -- set Open_vSwitch . external-ids:ovn-encap-ip=$ip \ -- add-br br-int \ @@ -158,6 +240,33 @@ ovn_attach() { start_daemon ovn-controller || return 1 } +# ovn_attach NETWORK BRIDGE IP [MASKLEN] +# +# First, this command attaches BRIDGE to interconnection network NETWORK, just +# like "net_attach NETWORK BRIDGE". Second, it configures (simulated) IP +# address IP (with network mask length MASKLEN, which defaults to 24) on +# BRIDGE. Finally, it configures the Open vSwitch database to work with OVN +# and starts ovn-controller. +ovn_attach() { + ovn_az_attach NONE $@ +} + +# ovn_setenv AZ +ovn_setenv () { + local d=$ovs_base/$1 + AS_VAR_SET([OVN_NB_DB], [unix:"$d"/ovn-nb/ovn-nb.sock]); export $var + AS_VAR_SET([OVN_SB_DB], [unix:"$d"/ovn-sb/ovn-sb.sock]); export $var +} + +# ovs_as AZ +ovn_as() { + if test "X$2" != X; then + (ovn_setenv $1; shift; "$@") + else + ovn_setenv $1 + fi +} + # OVN_POPULATE_ARP # # This pre-populates the ARP tables of all of the OVN instances that have been diff --git a/tests/testsuite.at b/tests/testsuite.at index eff6732..1985923 100644 --- a/tests/testsuite.at +++ b/tests/testsuite.at @@ -30,4 +30,5 @@ m4_include([tests/ovn-ic-nbctl.at]) m4_include([tests/ovn-ic-sbctl.at]) m4_include([tests/ovn-controller.at]) m4_include([tests/ovn-controller-vtep.at]) +m4_include([tests/ovn-ic.at]) m4_include([tests/checkpatch.at]) diff --git a/tutorial/ovs-sandbox b/tutorial/ovs-sandbox index 36cdb6b..1841776 100755 --- a/tutorial/ovs-sandbox +++ b/tutorial/ovs-sandbox @@ -58,6 +58,8 @@ gdb_vswitchd_ex=false gdb_ovsdb_ex=false gdb_ovn_northd=false gdb_ovn_northd_ex=false +gdb_ovn_ic=false +gdb_ovn_ic_ex=false gdb_ovn_controller=false gdb_ovn_controller_ex=false gdb_ovn_controller_vtep=false @@ -72,13 +74,20 @@ built=false ovn=true ovnsb_schema= ovnnb_schema= +ic_sb_schema= +ic_nb_schema= ovn_rbac=true n_northds=1 +n_ics=1 n_controllers=1 nbdb_model=standalone nbdb_servers=3 sbdb_model=backup sbdb_servers=3 +ic_nb_model=clustered +ic_nb_servers=3 +ic_sb_model=clustered +ic_sb_servers=3 dummy=override for option; do @@ -125,6 +134,7 @@ General options: -g, --gdb-vswitchd run ovs-vswitchd under gdb -d, --gdb-ovsdb run ovsdb-server under gdb --gdb-ovn-northd run ovn-northd under gdb + --gdb-ovn-ic run ovn-ic under gdb --gdb-ovn-controller run ovn-controller under gdb --gdb-ovn-controller-vtep run ovn-controller-vtep under gdb --dummy=ARG pass --enable-dummy=ARG to vswitchd (default: override) @@ -135,10 +145,15 @@ General options: OVN options: --no-ovn-rbac disable role-based access control for OVN --n-northds=NUMBER run NUMBER copies of northd (default: 1) + --n-ics=NUMBER run NUMBER copies of ic (default: 1) --nbdb-model=standalone|backup|clustered northbound database model --nbdb-servers=N number of servers in nbdb cluster (default: 3) --sbdb-model=standalone|backup|clustered southbound database model --sbdb-servers=N number of servers in sbdb cluster (default: 3) + --ic-nb-model=standalone|backup|clustered ic-northbound database model + --ic-nb-servers=N number of servers in IC NB cluster (default: 3) + --ic-sb-model=standalone|backup|clustered ic-southbound database model + --ic-sb-servers=N number of servers in IC SB cluster (default: 3) Other options: -h, --help Print this usage message. @@ -210,6 +225,9 @@ EOF --gdb-ovn-northd) gdb_ovn_northd=true ;; + --gdb-ovn-ic) + gdb_ovn_ic=true + ;; --gdb-ovn-controller) gdb_ovn_controller=true ;; @@ -225,6 +243,12 @@ EOF --n-northd*) prev=n_northds ;; + --n-ic*=*) + n_ics=$optarg + ;; + --n-ic*) + prev=n_ics_ + ;; --n-controller*=*) n_controllers=$optarg ;; @@ -259,10 +283,39 @@ EOF --sbdb-m*) prev=sbdb_model ;; + --ic-nb-s*=*) + ic_nb_servers=$optarg + ic_nb_model=clustered + ;; + --ic-nb-s*) + prev=ic_nb_servers + ic_nb_model=clustered + ;; + --ic-nb-m*=*) + ic_nb_model=$optarg + ;; + --ic-nb-m*) + prev=ic_nb_model + ;; + --ic-sb-s*=*) + ic_sb_servers=$optarg + ic_sb_model=clustered + ;; + --ic-sb-s*) + prev=ic_sb_servers + ic_sb_model=clustered + ;; + --ic-sb-m*=*) + ic_sb_model=$optarg + ;; + --ic-sb-m*) + prev=ic_sb_model + ;; -R|--gdb-run) gdb_vswitchd_ex=true gdb_ovsdb_ex=true gdb_ovn_northd_ex=true + gdb_ovn_ic_ex=true gdb_ovn_controller_ex=true gdb_ovn_controller_vtep_ex=true ;; @@ -327,6 +380,16 @@ if $built; then echo >&2 'source directory not found, please use --srcdir' exit 1 fi + ic_sb_schema=$srcdir/ovn-ic-sb.ovsschema + if test ! -e "$ic_sb_schema"; then + echo >&2 'source directory not found, please use --srcdir' + exit 1 + fi + ic_nb_schema=$srcdir/ovn-ic-nb.ovsschema + if test ! -e "$ic_nb_schema"; then + echo >&2 'source directory not found, please use --srcdir' + exit 1 + fi vtep_schema=$ovssrcdir/vtep/vtep.ovsschema if test ! -e "$vtep_schema"; then echo >&2 'source directory not found, please use --srcdir' @@ -340,7 +403,7 @@ if $built; then exit 1 fi PATH=$ovsbuilddir/ovsdb:$ovsbuilddir/vswitchd:$ovsbuilddir/utilities:$ovsbuilddir/vtep:$PATH - PATH=$builddir/controller:$builddir/controller-vtep:$builddir/northd:$builddir/utilities:$PATH + PATH=$builddir/controller:$builddir/controller-vtep:$builddir/northd:$builddir/ic:$builddir/utilities:$PATH export PATH else case $schema in @@ -490,6 +553,8 @@ The backup database file is sandbox/${db}2.db backup_note= ovn_start_db nb "$nbdb_model" "$nbdb_servers" "$ovnnb_schema" ovn_start_db sb "$sbdb_model" "$sbdb_servers" "$ovnsb_schema" +ovn_start_db ic_nb "$ic_nb_model" "$ic_nb_servers" "$ic_nb_schema" +ovn_start_db ic_sb "$ic_sb_model" "$ic_sb_servers" "$ic_sb_schema" #Add a small delay to allow ovsdb-server to launch. sleep 0.1 @@ -512,6 +577,9 @@ rungdb $gdb_vswitchd $gdb_vswitchd_ex ovs-vswitchd --detach --no-chdir --pidfile ovn-nbctl init ovn-sbctl init +ovn-ic-nbctl init +ovn-ic-sbctl init +ovn-nbctl set NB_Global . name=az-1 ovs-vsctl set open . external-ids:system-id=chassis-1 ovs-vsctl set open . external-ids:hostname=sandbox @@ -533,6 +601,14 @@ else ovs-vsctl set open . external-ids:ovn-remote=$OVN_SB_DB OVN_CTRLR_PKI="" fi +for i in $(seq $n_ics); do + if [ $i -eq 1 ]; then inst=""; else inst=$i; fi + rungdb $gdb_ovn_ic $gdb_ovn_ic_ex ovn-ic --detach \ + --no-chdir --pidfile=ovn-ic${inst}.pid -vconsole:off \ + --log-file=ovn-ic${inst}.log -vsyslog:off \ + --ovnsb-db="$OVN_SB_DB" --ovnnb-db="$OVN_NB_DB" \ + --ic-sb-db="$OVN_IC_SB_DB" --ic-nb-db="$OVN_IC_NB_DB" +done for i in $(seq $n_northds); do if [ $i -eq 1 ]; then inst=""; else inst=$i; fi rungdb $gdb_ovn_northd $gdb_ovn_northd_ex ovn-northd --detach \ From patchwork Wed Jan 29 19:56:00 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Han Zhou X-Patchwork-Id: 1231078 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.137; helo=fraxinus.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=ovn.org Received: from fraxinus.osuosl.org (smtp4.osuosl.org [140.211.166.137]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 487DlK42Jgz9sNT for ; Thu, 30 Jan 2020 06:56:45 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by fraxinus.osuosl.org (Postfix) with ESMTP id C51DE860EF; Wed, 29 Jan 2020 19:56:43 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from fraxinus.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id kKKW0He5XnEY; Wed, 29 Jan 2020 19:56:41 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by fraxinus.osuosl.org (Postfix) with ESMTP id 15F4385FD0; Wed, 29 Jan 2020 19:56:37 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id E30D2C1D8A; Wed, 29 Jan 2020 19:56:36 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from silver.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 8A425C0881 for ; Wed, 29 Jan 2020 19:56:34 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by silver.osuosl.org (Postfix) with ESMTP id 752D3203CF for ; Wed, 29 Jan 2020 19:56:34 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from silver.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id fPa6Qjvvs3Mm for ; Wed, 29 Jan 2020 19:56:32 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from relay1-d.mail.gandi.net (relay1-d.mail.gandi.net [217.70.183.193]) by silver.osuosl.org (Postfix) with ESMTPS id 10BFC2206D for ; Wed, 29 Jan 2020 19:56:31 +0000 (UTC) X-Originating-IP: 216.113.160.77 Received: from localhost.localdomain.localdomain (unknown [216.113.160.77]) (Authenticated sender: hzhou@ovn.org) by relay1-d.mail.gandi.net (Postfix) with ESMTPSA id E9988240002; Wed, 29 Jan 2020 19:56:29 +0000 (UTC) From: Han Zhou To: dev@openvswitch.org Date: Wed, 29 Jan 2020 11:56:00 -0800 Message-Id: <1580327768-36501-6-git-send-email-hzhou@ovn.org> X-Mailer: git-send-email 2.1.0 In-Reply-To: <1580327768-36501-1-git-send-email-hzhou@ovn.org> References: <1580327768-36501-1-git-send-email-hzhou@ovn.org> Cc: Han Zhou Subject: [ovs-dev] [PATCH ovn v4 05/13] ovn-northd.c: Refactor allocate_tnlid. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" Move allocate_tnlid() and related interfaces to ovn_util module, so that they be reused by ovn-ic (in next patches). At the same time, define macros for the range of datapath tunnel keys, and reserve a range with ((1u << 16) - 1) keys for global transit switch datapaths, among the ((1u << 24) - 1) datapath tunnel key space. Signed-off-by: Han Zhou --- lib/ovn-util.c | 59 +++++++++++++++++++++++++++++++++++++ lib/ovn-util.h | 12 ++++++++ northd/ovn-northd.c | 85 +++++++++-------------------------------------------- 3 files changed, 85 insertions(+), 71 deletions(-) diff --git a/lib/ovn-util.c b/lib/ovn-util.c index 18c13a8..70007fd 100644 --- a/lib/ovn-util.c +++ b/lib/ovn-util.c @@ -455,3 +455,62 @@ datapath_is_switch(const struct sbrec_datapath_binding *ldp) { return smap_get(&ldp->external_ids, "logical-switch") != NULL; } + +struct tnlid_node { + struct hmap_node hmap_node; + uint32_t tnlid; +}; + +void +ovn_destroy_tnlids(struct hmap *tnlids) +{ + struct tnlid_node *node; + HMAP_FOR_EACH_POP (node, hmap_node, tnlids) { + free(node); + } + hmap_destroy(tnlids); +} + +void +ovn_add_tnlid(struct hmap *set, uint32_t tnlid) +{ + struct tnlid_node *node = xmalloc(sizeof *node); + hmap_insert(set, &node->hmap_node, hash_int(tnlid, 0)); + node->tnlid = tnlid; +} + +static bool +tnlid_in_use(const struct hmap *set, uint32_t tnlid) +{ + const struct tnlid_node *node; + HMAP_FOR_EACH_IN_BUCKET (node, hmap_node, hash_int(tnlid, 0), set) { + if (node->tnlid == tnlid) { + return true; + } + } + return false; +} + +static uint32_t +next_tnlid(uint32_t tnlid, uint32_t min, uint32_t max) +{ + return tnlid + 1 <= max ? tnlid + 1 : min; +} + +uint32_t +ovn_allocate_tnlid(struct hmap *set, const char *name, uint32_t min, + uint32_t max, uint32_t *hint) +{ + for (uint32_t tnlid = next_tnlid(*hint, min, max); tnlid != *hint; + tnlid = next_tnlid(tnlid, min, max)) { + if (!tnlid_in_use(set, tnlid)) { + ovn_add_tnlid(set, tnlid); + *hint = tnlid; + return tnlid; + } + } + + static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 1); + VLOG_WARN_RL(&rl, "all %s tunnel ids exhausted", name); + return 0; +} diff --git a/lib/ovn-util.h b/lib/ovn-util.h index 5685f43..4f9bbf6 100644 --- a/lib/ovn-util.h +++ b/lib/ovn-util.h @@ -90,4 +90,16 @@ uint32_t ovn_logical_flow_hash(const struct uuid *logical_datapath, uint16_t priority, const char *match, const char *actions); bool datapath_is_switch(const struct sbrec_datapath_binding *); + +#define OVN_MAX_DP_KEY ((1u << 24) - 1) +#define OVN_MAX_DP_GLOBAL_NUM ((1u << 16) - 1) +#define OVN_MIN_DP_KEY_LOCAL 1 +#define OVN_MAX_DP_KEY_LOCAL (OVN_MAX_DP_KEY - OVN_MAX_DP_GLOBAL_NUM) +#define OVN_MIN_DP_KEY_GLOBAL (OVN_MAX_DP_KEY_LOCAL + 1) +#define OVN_MAX_DP_KEY_GLOBAL OVN_MAX_DP_KEY +struct hmap; +void ovn_destroy_tnlids(struct hmap *tnlids); +void ovn_add_tnlid(struct hmap *set, uint32_t tnlid); +uint32_t ovn_allocate_tnlid(struct hmap *set, const char *name, uint32_t min, + uint32_t max, uint32_t *hint); #endif diff --git a/northd/ovn-northd.c b/northd/ovn-northd.c index 44fe6cf..5157690 100644 --- a/northd/ovn-northd.c +++ b/northd/ovn-northd.c @@ -307,65 +307,6 @@ Options:\n\ stream_usage("database", true, true, false); } -struct tnlid_node { - struct hmap_node hmap_node; - uint32_t tnlid; -}; - -static void -destroy_tnlids(struct hmap *tnlids) -{ - struct tnlid_node *node; - HMAP_FOR_EACH_POP (node, hmap_node, tnlids) { - free(node); - } - hmap_destroy(tnlids); -} - -static void -add_tnlid(struct hmap *set, uint32_t tnlid) -{ - struct tnlid_node *node = xmalloc(sizeof *node); - hmap_insert(set, &node->hmap_node, hash_int(tnlid, 0)); - node->tnlid = tnlid; -} - -static bool -tnlid_in_use(const struct hmap *set, uint32_t tnlid) -{ - const struct tnlid_node *node; - HMAP_FOR_EACH_IN_BUCKET (node, hmap_node, hash_int(tnlid, 0), set) { - if (node->tnlid == tnlid) { - return true; - } - } - return false; -} - -static uint32_t -next_tnlid(uint32_t tnlid, uint32_t min, uint32_t max) -{ - return tnlid + 1 <= max ? tnlid + 1 : min; -} - -static uint32_t -allocate_tnlid(struct hmap *set, const char *name, uint32_t min, uint32_t max, - uint32_t *hint) -{ - for (uint32_t tnlid = next_tnlid(*hint, min, max); tnlid != *hint; - tnlid = next_tnlid(tnlid, min, max)) { - if (!tnlid_in_use(set, tnlid)) { - add_tnlid(set, tnlid); - *hint = tnlid; - return tnlid; - } - } - - static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 1); - VLOG_WARN_RL(&rl, "all %s tunnel ids exhausted", name); - return 0; -} - struct ovn_chassis_qdisc_queues { struct hmap_node key_node; uint32_t queue_id; @@ -564,9 +505,9 @@ init_mcast_port_info(struct mcast_port_info *mcast_info, static uint32_t ovn_mcast_group_allocate_key(struct mcast_info *mcast_info) { - return allocate_tnlid(&mcast_info->group_tnlids, "multicast group", - OVN_MIN_IP_MULTICAST, OVN_MAX_IP_MULTICAST, - &mcast_info->group_tnlid_hint); + return ovn_allocate_tnlid(&mcast_info->group_tnlids, "multicast group", + OVN_MIN_IP_MULTICAST, OVN_MAX_IP_MULTICAST, + &mcast_info->group_tnlid_hint); } /* The 'key' comes from nbs->header_.uuid or nbr->header_.uuid or @@ -669,7 +610,7 @@ ovn_datapath_destroy(struct hmap *datapaths, struct ovn_datapath *od) * private list and once we've exited that function it is not safe to * use it. */ hmap_remove(datapaths, &od->key_node); - destroy_tnlids(&od->port_tnlids); + ovn_destroy_tnlids(&od->port_tnlids); bitmap_free(od->ipam_info.allocated_ipv4s); free(od->router_ports); ovn_ls_port_group_destroy(&od->nb_pgs); @@ -909,7 +850,7 @@ destroy_mcast_info_for_datapath(struct ovn_datapath *od) destroy_mcast_info_for_switch_datapath(od); } - destroy_tnlids(&od->mcast_info.group_tnlids); + ovn_destroy_tnlids(&od->mcast_info.group_tnlids); } static void @@ -1064,7 +1005,8 @@ static uint32_t ovn_datapath_allocate_key(struct hmap *dp_tnlids) { static uint32_t hint; - return allocate_tnlid(dp_tnlids, "datapath", 1, (1u << 24) - 1, &hint); + return ovn_allocate_tnlid(dp_tnlids, "datapath", OVN_MIN_DP_KEY_LOCAL, + OVN_MAX_DP_KEY_LOCAL, &hint); } /* Updates the southbound Datapath_Binding table so that it contains the @@ -1085,7 +1027,7 @@ build_datapaths(struct northd_context *ctx, struct hmap *datapaths, struct hmap dp_tnlids = HMAP_INITIALIZER(&dp_tnlids); struct ovn_datapath *od; LIST_FOR_EACH (od, list, &both) { - add_tnlid(&dp_tnlids, od->sb->tunnel_key); + ovn_add_tnlid(&dp_tnlids, od->sb->tunnel_key); } /* Add southbound record for each unmatched northbound record. */ @@ -1099,7 +1041,7 @@ build_datapaths(struct northd_context *ctx, struct hmap *datapaths, ovn_datapath_update_external_ids(od); sbrec_datapath_binding_set_tunnel_key(od->sb, tunnel_key); } - destroy_tnlids(&dp_tnlids); + ovn_destroy_tnlids(&dp_tnlids); } /* Delete southbound records without northbound matches. */ @@ -1230,8 +1172,8 @@ ovn_port_find(const struct hmap *ports, const char *name) static uint32_t ovn_port_allocate_key(struct ovn_datapath *od) { - return allocate_tnlid(&od->port_tnlids, "port", - 1, (1u << 15) - 1, &od->port_key_hint); + return ovn_allocate_tnlid(&od->port_tnlids, "port", + 1, (1u << 15) - 1, &od->port_key_hint); } /* Returns true if the logical switch port 'enabled' column is empty or @@ -3370,7 +3312,7 @@ build_ports(struct northd_context *ctx, ovn_port_update_sbrec(ctx, sbrec_chassis_by_name, op, &chassis_qdisc_queues, &active_ha_chassis_grps); - add_tnlid(&op->od->port_tnlids, op->sb->tunnel_key); + ovn_add_tnlid(&op->od->port_tnlids, op->sb->tunnel_key); if (op->sb->tunnel_key > op->od->port_key_hint) { op->od->port_key_hint = op->sb->tunnel_key; } @@ -3606,7 +3548,8 @@ ovn_igmp_group_add(struct northd_context *ctx, struct hmap *igmp_groups, igmp_group->address = *address; if (mcgroup) { igmp_group->mcgroup.key = mcgroup->tunnel_key; - add_tnlid(&datapath->mcast_info.group_tnlids, mcgroup->tunnel_key); + ovn_add_tnlid(&datapath->mcast_info.group_tnlids, + mcgroup->tunnel_key); } else { igmp_group->mcgroup.key = 0; } From patchwork Wed Jan 29 19:56:01 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Han Zhou X-Patchwork-Id: 1231079 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.136; helo=silver.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=ovn.org Received: from silver.osuosl.org (smtp3.osuosl.org [140.211.166.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 487DlP2W7mz9sNT for ; Thu, 30 Jan 2020 06:56:49 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by silver.osuosl.org (Postfix) with ESMTP id C0D4B2263E; Wed, 29 Jan 2020 19:56:47 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from silver.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id JW0P1epud1cr; Wed, 29 Jan 2020 19:56:39 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by silver.osuosl.org (Postfix) with ESMTP id 731702155F; Wed, 29 Jan 2020 19:56:39 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 1B673C1D85; Wed, 29 Jan 2020 19:56:39 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from silver.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 77966C0881 for ; Wed, 29 Jan 2020 19:56:35 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by silver.osuosl.org (Postfix) with ESMTP id 5CD7C220DA for ; Wed, 29 Jan 2020 19:56:35 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from silver.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id X7oOMtlGAol2 for ; Wed, 29 Jan 2020 19:56:33 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from relay1-d.mail.gandi.net (relay1-d.mail.gandi.net [217.70.183.193]) by silver.osuosl.org (Postfix) with ESMTPS id 47FD42155F for ; Wed, 29 Jan 2020 19:56:33 +0000 (UTC) X-Originating-IP: 216.113.160.77 Received: from localhost.localdomain.localdomain (unknown [216.113.160.77]) (Authenticated sender: hzhou@ovn.org) by relay1-d.mail.gandi.net (Postfix) with ESMTPSA id 19EFF240005; Wed, 29 Jan 2020 19:56:30 +0000 (UTC) From: Han Zhou To: dev@openvswitch.org Date: Wed, 29 Jan 2020 11:56:01 -0800 Message-Id: <1580327768-36501-7-git-send-email-hzhou@ovn.org> X-Mailer: git-send-email 2.1.0 In-Reply-To: <1580327768-36501-1-git-send-email-hzhou@ovn.org> References: <1580327768-36501-1-git-send-email-hzhou@ovn.org> Cc: Han Zhou Subject: [ovs-dev] [PATCH ovn v4 06/13] ovn-ic: Transit switch controller. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" Processing transit switches and sync between IC-NB, IC-SB and NB. Signed-off-by: Han Zhou --- ic/ovn-ic.c | 107 ++++++++++++++++++++++++++++++++++++++++++++++++++++ lib/ovn-util.c | 6 +-- lib/ovn-util.h | 1 + northd/ovn-northd.c | 70 ++++++++++++++++++++++++++++------ ovn-nb.xml | 24 ++++++++++++ ovn-sb.xml | 9 +++++ tests/ovn-ic.at | 40 ++++++++++++++++++++ 7 files changed, 242 insertions(+), 15 deletions(-) diff --git a/ic/ovn-ic.c b/ic/ovn-ic.c index a6d27c0..82bd86e 100644 --- a/ic/ovn-ic.c +++ b/ic/ovn-ic.c @@ -147,11 +147,118 @@ az_run(struct ic_context *ctx) return NULL; } +static uint32_t +allocate_ts_dp_key(struct hmap *dp_tnlids) +{ + static uint32_t hint = OVN_MIN_DP_KEY_GLOBAL; + return ovn_allocate_tnlid(dp_tnlids, "transit switch datapath", + OVN_MIN_DP_KEY_GLOBAL, OVN_MAX_DP_KEY_GLOBAL, + &hint); +} + +static void +ts_run(struct ic_context *ctx) +{ + const struct icnbrec_transit_switch *ts; + + struct hmap dp_tnlids = HMAP_INITIALIZER(&dp_tnlids); + struct shash isb_dps = SHASH_INITIALIZER(&isb_dps); + const struct icsbrec_datapath_binding *isb_dp; + ICSBREC_DATAPATH_BINDING_FOR_EACH (isb_dp, ctx->ovnisb_idl) { + shash_add(&isb_dps, isb_dp->transit_switch, isb_dp); + ovn_add_tnlid(&dp_tnlids, isb_dp->tunnel_key); + } + + /* Sync INB TS to AZ NB */ + if (ctx->ovnnb_txn) { + struct shash nb_tses = SHASH_INITIALIZER(&nb_tses); + const struct nbrec_logical_switch *ls; + + /* Get current NB Logical_Switch with other_config:interconn-ts */ + NBREC_LOGICAL_SWITCH_FOR_EACH (ls, ctx->ovnnb_idl) { + const char *ts_name = smap_get(&ls->other_config, "interconn-ts"); + if (ts_name) { + shash_add(&nb_tses, ts_name, ls); + } + } + + /* Create NB Logical_Switch for each TS */ + ICNBREC_TRANSIT_SWITCH_FOR_EACH (ts, ctx->ovninb_idl) { + ls = shash_find_and_delete(&nb_tses, ts->name); + if (!ls) { + ls = nbrec_logical_switch_insert(ctx->ovnnb_txn); + nbrec_logical_switch_set_name(ls, ts->name); + nbrec_logical_switch_update_other_config_setkey(ls, + "interconn-ts", + ts->name); + } + isb_dp = shash_find_data(&isb_dps, ts->name); + if (isb_dp) { + int64_t nb_tnl_key = smap_get_int(&ls->other_config, + "requested-tnl-key", + 0); + if (nb_tnl_key != isb_dp->tunnel_key) { + VLOG_DBG("Set other_config:requested-tnl-key %"PRId64 + " for transit switch %s in NB.", + isb_dp->tunnel_key, ts->name); + char *tnl_key_str = xasprintf("%"PRId64, + isb_dp->tunnel_key); + nbrec_logical_switch_update_other_config_setkey( + ls, "requested-tnl-key", tnl_key_str); + free(tnl_key_str); + } + } + } + + /* Delete extra NB Logical_Switch with other_config:interconn-ts */ + struct shash_node *node; + SHASH_FOR_EACH (node, &nb_tses) { + nbrec_logical_switch_delete(node->data); + } + shash_destroy(&nb_tses); + } + + /* Sync TS between INB and ISB. This is performed after syncing with AZ + * SB, to avoid uncommitted ISB datapath tunnel key to be synced back to + * AZ. */ + if (ctx->ovnisb_txn) { + /* Create ISB Datapath_Binding */ + ICNBREC_TRANSIT_SWITCH_FOR_EACH (ts, ctx->ovninb_idl) { + isb_dp = shash_find_and_delete(&isb_dps, ts->name); + if (!isb_dp) { + /* Allocate tunnel key */ + int64_t dp_key = allocate_ts_dp_key(&dp_tnlids); + if (!dp_key) { + continue; + } + + isb_dp = icsbrec_datapath_binding_insert(ctx->ovnisb_txn); + icsbrec_datapath_binding_set_transit_switch(isb_dp, ts->name); + icsbrec_datapath_binding_set_tunnel_key(isb_dp, dp_key); + } + } + + /* Delete extra ISB Datapath_Binding */ + struct shash_node *node; + SHASH_FOR_EACH (node, &isb_dps) { + icsbrec_datapath_binding_delete(node->data); + } + } + ovn_destroy_tnlids(&dp_tnlids); + shash_destroy(&isb_dps); +} + static void ovn_db_run(struct ic_context *ctx) { const struct icsbrec_availability_zone *az = az_run(ctx); VLOG_DBG("Availability zone: %s", az ? az->name : "not created yet."); + + if (!az) { + return; + } + + ts_run(ctx); } static void diff --git a/lib/ovn-util.c b/lib/ovn-util.c index 70007fd..f604532 100644 --- a/lib/ovn-util.c +++ b/lib/ovn-util.c @@ -479,8 +479,8 @@ ovn_add_tnlid(struct hmap *set, uint32_t tnlid) node->tnlid = tnlid; } -static bool -tnlid_in_use(const struct hmap *set, uint32_t tnlid) +bool +ovn_tnlid_in_use(const struct hmap *set, uint32_t tnlid) { const struct tnlid_node *node; HMAP_FOR_EACH_IN_BUCKET (node, hmap_node, hash_int(tnlid, 0), set) { @@ -503,7 +503,7 @@ ovn_allocate_tnlid(struct hmap *set, const char *name, uint32_t min, { for (uint32_t tnlid = next_tnlid(*hint, min, max); tnlid != *hint; tnlid = next_tnlid(tnlid, min, max)) { - if (!tnlid_in_use(set, tnlid)) { + if (!ovn_tnlid_in_use(set, tnlid)) { ovn_add_tnlid(set, tnlid); *hint = tnlid; return tnlid; diff --git a/lib/ovn-util.h b/lib/ovn-util.h index 4f9bbf6..d99123f 100644 --- a/lib/ovn-util.h +++ b/lib/ovn-util.h @@ -100,6 +100,7 @@ bool datapath_is_switch(const struct sbrec_datapath_binding *); struct hmap; void ovn_destroy_tnlids(struct hmap *tnlids); void ovn_add_tnlid(struct hmap *set, uint32_t tnlid); +bool ovn_tnlid_in_use(const struct hmap *set, uint32_t tnlid); uint32_t ovn_allocate_tnlid(struct hmap *set, const char *name, uint32_t min, uint32_t max, uint32_t *hint); #endif diff --git a/northd/ovn-northd.c b/northd/ovn-northd.c index 5157690..4b58bea 100644 --- a/northd/ovn-northd.c +++ b/northd/ovn-northd.c @@ -907,6 +907,14 @@ ovn_datapath_update_external_ids(struct ovn_datapath *od) if (name2 && name2[0]) { smap_add(&ids, "name2", name2); } + + /* Set interconn-ts. */ + if (od->nbs) { + const char *ts = smap_get(&od->nbs->other_config, "interconn-ts"); + if (ts) { + smap_add(&ids, "interconn-ts", ts); + } + } sbrec_datapath_binding_set_external_ids(od->sb, &ids); smap_destroy(&ids); } @@ -1022,30 +1030,68 @@ build_datapaths(struct northd_context *ctx, struct hmap *datapaths, join_datapaths(ctx, datapaths, &sb_only, &nb_only, &both, lr_list); - if (!ovs_list_is_empty(&nb_only)) { - /* First index the in-use datapath tunnel IDs. */ - struct hmap dp_tnlids = HMAP_INITIALIZER(&dp_tnlids); - struct ovn_datapath *od; + /* First index the in-use datapath tunnel IDs. */ + struct hmap dp_tnlids = HMAP_INITIALIZER(&dp_tnlids); + struct ovn_datapath *od, *next; + if (!ovs_list_is_empty(&nb_only) || !ovs_list_is_empty(&both)) { LIST_FOR_EACH (od, list, &both) { ovn_add_tnlid(&dp_tnlids, od->sb->tunnel_key); } + } - /* Add southbound record for each unmatched northbound record. */ - LIST_FOR_EACH (od, list, &nb_only) { - uint32_t tunnel_key = ovn_datapath_allocate_key(&dp_tnlids); + /* Add southbound record for each unmatched northbound record. */ + LIST_FOR_EACH (od, list, &nb_only) { + int64_t tunnel_key = 0; + if (od->nbs) { + tunnel_key = smap_get_int(&od->nbs->other_config, + "requested-tnl-key", + 0); + if (tunnel_key && ovn_tnlid_in_use(&dp_tnlids, tunnel_key)) { + static struct vlog_rate_limit rl = + VLOG_RATE_LIMIT_INIT(1, 1); + VLOG_WARN_RL(&rl, "Cannot create datapath binding for " + "logical switch %s due to duplicate key set " + "in other_config:requested-tnl-key: %"PRId64, + od->nbs->name, tunnel_key); + continue; + } + } + if (!tunnel_key) { + tunnel_key = ovn_datapath_allocate_key(&dp_tnlids); if (!tunnel_key) { break; } + } - od->sb = sbrec_datapath_binding_insert(ctx->ovnsb_txn); - ovn_datapath_update_external_ids(od); - sbrec_datapath_binding_set_tunnel_key(od->sb, tunnel_key); + od->sb = sbrec_datapath_binding_insert(ctx->ovnsb_txn); + ovn_datapath_update_external_ids(od); + sbrec_datapath_binding_set_tunnel_key(od->sb, tunnel_key); + } + + /* Sync from northbound to southbound record for od existed in both. */ + LIST_FOR_EACH (od, list, &both) { + if (od->nbs) { + int64_t tunnel_key = smap_get_int(&od->nbs->other_config, + "requested-tnl-key", + 0); + if (tunnel_key && tunnel_key != od->sb->tunnel_key) { + if (ovn_tnlid_in_use(&dp_tnlids, tunnel_key)) { + static struct vlog_rate_limit rl = + VLOG_RATE_LIMIT_INIT(1, 1); + VLOG_WARN_RL(&rl, "Cannot update datapath binding key for " + "logical switch %s due to duplicate key set " + "in other_config:requested-tnl-key: %"PRId64, + od->nbs->name, tunnel_key); + continue; + } + sbrec_datapath_binding_set_tunnel_key(od->sb, tunnel_key); + } } - ovn_destroy_tnlids(&dp_tnlids); } + ovn_destroy_tnlids(&dp_tnlids); + /* Delete southbound records without northbound matches. */ - struct ovn_datapath *od, *next; LIST_FOR_EACH_SAFE (od, next, list, &sb_only) { ovs_list_remove(&od->list); sbrec_datapath_binding_delete(od->sb); diff --git a/ovn-nb.xml b/ovn-nb.xml index 6f8baa0..a092af8 100644 --- a/ovn-nb.xml +++ b/ovn-nb.xml @@ -363,6 +363,30 @@ + + + The + of corresponding transit switch in + database. This kind of logical switch is created and controlled + by ovn-ic. + + + + + + Configures the datapath tunnel key for the logical switch. Usually + this is not needed because ovn-northd will assign an + unique key for each datapath by itself. However, if it is configured, + ovn-northd honors the configured value. The typical use + case is for interconnection: the tunnel keys for transit switches need + to be unique globally, so they are maintained in the global + database, and ovn-ic simply + syncs the value from through this config. + + + See External IDs at the beginning of this document. diff --git a/ovn-sb.xml b/ovn-sb.xml index cf29430..c08ceda 100644 --- a/ovn-sb.xml +++ b/ovn-sb.xml @@ -2361,6 +2361,15 @@ tcp.flags = RST; the database. + + For a logical datapath that represents a logical switch that represents + a transit switch for interconnection, ovn-northd stores in + this key the value of the same interconn-ts key of the + column of the corresponding row in the database. + +

ovn-northd copies these from the name fields in the X-Patchwork-Id: 1231080 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.133; helo=hemlock.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=ovn.org Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 487DlT2bkDz9sNT for ; Thu, 30 Jan 2020 06:56:53 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id C124E8828D; Wed, 29 Jan 2020 19:56:51 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id bh-oAxFs8vDp; Wed, 29 Jan 2020 19:56:45 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by hemlock.osuosl.org (Postfix) with ESMTP id 1176A88287; Wed, 29 Jan 2020 19:56:41 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id D79AFC1D92; Wed, 29 Jan 2020 19:56:40 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by lists.linuxfoundation.org (Postfix) with ESMTP id E5821C1D8C for ; Wed, 29 Jan 2020 19:56:37 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id D0E7087E62 for ; Wed, 29 Jan 2020 19:56:37 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id IeEAfSWAyajh for ; Wed, 29 Jan 2020 19:56:35 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from relay1-d.mail.gandi.net (relay1-d.mail.gandi.net [217.70.183.193]) by hemlock.osuosl.org (Postfix) with ESMTPS id 990A987A96 for ; Wed, 29 Jan 2020 19:56:34 +0000 (UTC) X-Originating-IP: 216.113.160.77 Received: from localhost.localdomain.localdomain (unknown [216.113.160.77]) (Authenticated sender: hzhou@ovn.org) by relay1-d.mail.gandi.net (Postfix) with ESMTPSA id 6AE87240009; Wed, 29 Jan 2020 19:56:32 +0000 (UTC) From: Han Zhou To: dev@openvswitch.org Date: Wed, 29 Jan 2020 11:56:02 -0800 Message-Id: <1580327768-36501-8-git-send-email-hzhou@ovn.org> X-Mailer: git-send-email 2.1.0 In-Reply-To: <1580327768-36501-1-git-send-email-hzhou@ovn.org> References: <1580327768-36501-1-git-send-email-hzhou@ovn.org> Cc: Han Zhou Subject: [ovs-dev] [PATCH ovn v4 07/13] ovn-sb: Add keys is_interconn and is_remote to Chassis's external_ids. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" Support the new keys in external_ids column of Chassis table for OVN interconnection. Also, populate the is_interconn key according to external_ids:ovn-is-interconn key of Open_vSwitch table on the chassis. This patch also avoids creating tunnel or bfd sessions with remote chassis. Signed-off-by: Han Zhou --- controller/bfd.c | 6 +++++- controller/chassis.c | 25 +++++++++++++++++++++++-- controller/encaps.c | 15 ++++++++++++--- controller/encaps.h | 3 ++- controller/ovn-controller.8.xml | 6 ++++++ controller/ovn-controller.c | 2 +- northd/ovn-northd.c | 4 +++- ovn-sb.xml | 15 +++++++++++++++ 8 files changed, 67 insertions(+), 9 deletions(-) diff --git a/controller/bfd.c b/controller/bfd.c index 10cd5fc..2b1e87f 100644 --- a/controller/bfd.c +++ b/controller/bfd.c @@ -151,7 +151,11 @@ bfd_calculate_chassis( if (is_ha_chassis) { /* It's an HA chassis. So add the ref_chassis to the bfd set. */ for (size_t i = 0; i < ha_chassis_grp->n_ref_chassis; i++) { - sset_add(&grp_chassis, ha_chassis_grp->ref_chassis[i]->name); + struct sbrec_chassis *ref_ch = ha_chassis_grp->ref_chassis[i]; + if (smap_get_bool(&ref_ch->external_ids, "is-remote", false)) { + continue; + } + sset_add(&grp_chassis, ref_ch->name); } } else { /* This is not an HA chassis. Check if this chassis is present diff --git a/controller/chassis.c b/controller/chassis.c index 978273e..522893e 100644 --- a/controller/chassis.c +++ b/controller/chassis.c @@ -92,6 +92,8 @@ struct ovs_chassis_cfg { struct sset encap_ip_set; /* Interface type list formatted in the OVN-SB Chassis required format. */ struct ds iface_types; + /* Is this chassis an interconnection gateway. */ + bool is_interconn; }; static void @@ -172,6 +174,12 @@ get_datapath_type(const struct ovsrec_bridge *br_int) return ""; } +static bool +get_is_interconn(const struct smap *ext_ids) +{ + return smap_get_bool(ext_ids, "ovn-is-interconn", false); +} + static void update_chassis_transport_zones(const struct sset *transport_zones, const struct sbrec_chassis *chassis_rec) @@ -285,19 +293,23 @@ chassis_parse_ovs_config(const struct ovsrec_open_vswitch_table *ovs_table, sset_destroy(&ovs_cfg->encap_ip_set); } + ovs_cfg->is_interconn = get_is_interconn(&cfg->external_ids); + return true; } static void chassis_build_external_ids(struct smap *ext_ids, const char *bridge_mappings, const char *datapath_type, const char *cms_options, - const char *chassis_macs, const char *iface_types) + const char *chassis_macs, const char *iface_types, + bool is_interconn) { smap_replace(ext_ids, "ovn-bridge-mappings", bridge_mappings); smap_replace(ext_ids, "datapath-type", datapath_type); smap_replace(ext_ids, "ovn-cms-options", cms_options); smap_replace(ext_ids, "iface-types", iface_types); smap_replace(ext_ids, "ovn-chassis-mac-mappings", chassis_macs); + smap_replace(ext_ids, "is-interconn", is_interconn ? "true" : "false"); } /* @@ -309,6 +321,7 @@ chassis_external_ids_changed(const char *bridge_mappings, const char *cms_options, const char *chassis_macs, const struct ds *iface_types, + bool is_interconn, const struct sbrec_chassis *chassis_rec) { const char *chassis_bridge_mappings = @@ -345,6 +358,12 @@ chassis_external_ids_changed(const char *bridge_mappings, return true; } + bool chassis_is_interconn = + smap_get_bool(&chassis_rec->external_ids, "is-interconn", false); + if (chassis_is_interconn != is_interconn) { + return true; + } + return false; } @@ -524,6 +543,7 @@ chassis_update(const struct sbrec_chassis *chassis_rec, ovs_cfg->cms_options, ovs_cfg->chassis_macs, &ovs_cfg->iface_types, + ovs_cfg->is_interconn, chassis_rec)) { struct smap ext_ids; @@ -532,7 +552,8 @@ chassis_update(const struct sbrec_chassis *chassis_rec, ovs_cfg->datapath_type, ovs_cfg->cms_options, ovs_cfg->chassis_macs, - ds_cstr_ro(&ovs_cfg->iface_types)); + ds_cstr_ro(&ovs_cfg->iface_types), + ovs_cfg->is_interconn); sbrec_chassis_verify_external_ids(chassis_rec); sbrec_chassis_set_external_ids(chassis_rec, &ext_ids); smap_destroy(&ext_ids); diff --git a/controller/encaps.c b/controller/encaps.c index db48b70..846628a 100644 --- a/controller/encaps.c +++ b/controller/encaps.c @@ -296,7 +296,7 @@ encaps_run(struct ovsdb_idl_txn *ovs_idl_txn, const struct ovsrec_bridge_table *bridge_table, const struct ovsrec_bridge *br_int, const struct sbrec_chassis_table *chassis_table, - const char *chassis_id, + const struct sbrec_chassis *this_chassis, const struct sbrec_sb_global *sbg, const struct sset *transport_zones) { @@ -316,7 +316,7 @@ encaps_run(struct ovsdb_idl_txn *ovs_idl_txn, tc.ovs_txn = ovs_idl_txn; ovsdb_idl_txn_add_comment(tc.ovs_txn, "ovn-controller: modifying OVS tunnels '%s'", - chassis_id); + this_chassis->name); /* Collect all port names into tc.port_names. * @@ -347,7 +347,7 @@ encaps_run(struct ovsdb_idl_txn *ovs_idl_txn, } SBREC_CHASSIS_TABLE_FOR_EACH (chassis_rec, chassis_table) { - if (strcmp(chassis_rec->name, chassis_id)) { + if (strcmp(chassis_rec->name, this_chassis->name)) { /* Create tunnels to the other Chassis belonging to the * same transport zone */ if (!chassis_tzones_overlap(transport_zones, chassis_rec)) { @@ -357,6 +357,15 @@ encaps_run(struct ovsdb_idl_txn *ovs_idl_txn, continue; } + if (smap_get_bool(&chassis_rec->external_ids, "is-remote", false) + && !smap_get_bool(&this_chassis->external_ids, "is-interconn", + false)) { + VLOG_DBG("Skipping encap creation for Chassis '%s' because " + "it is remote but this chassis is not interconn.", + chassis_rec->name); + continue; + } + if (chassis_tunnel_add(chassis_rec, sbg, &tc) == 0) { VLOG_INFO("Creating encap for '%s' failed", chassis_rec->name); continue; diff --git a/controller/encaps.h b/controller/encaps.h index c919d18..f488393 100644 --- a/controller/encaps.h +++ b/controller/encaps.h @@ -23,6 +23,7 @@ struct ovsdb_idl_txn; struct ovsrec_bridge; struct ovsrec_bridge_table; struct sbrec_chassis_table; +struct sbrec_chassis; struct sbrec_sb_global; struct ovsrec_open_vswitch_table; struct sset; @@ -32,7 +33,7 @@ void encaps_run(struct ovsdb_idl_txn *ovs_idl_txn, const struct ovsrec_bridge_table *, const struct ovsrec_bridge *br_int, const struct sbrec_chassis_table *, - const char *chassis_id, + const struct sbrec_chassis *, const struct sbrec_sb_global *, const struct sset *transport_zones); diff --git a/controller/ovn-controller.8.xml b/controller/ovn-controller.8.xml index a163ff5..76bbbdc 100644 --- a/controller/ovn-controller.8.xml +++ b/controller/ovn-controller.8.xml @@ -227,6 +227,12 @@ mac with, if packet is going from a distributed router port on vlan type logical switch. + +

external_ids:ovn-is-interconn
+
+ The boolean flag indicates if the chassis is used as an + interconnection gateway. +

diff --git a/controller/ovn-controller.c b/controller/ovn-controller.c index 29a3792..caef80b 100644 --- a/controller/ovn-controller.c +++ b/controller/ovn-controller.c @@ -2020,7 +2020,7 @@ main(int argc, char *argv[]) encaps_run(ovs_idl_txn, bridge_table, br_int, sbrec_chassis_table_get(ovnsb_idl_loop.idl), - chassis_id, + chassis, sbrec_sb_global_first(ovnsb_idl_loop.idl), &transport_zones); diff --git a/northd/ovn-northd.c b/northd/ovn-northd.c index 4b58bea..a3f96b0 100644 --- a/northd/ovn-northd.c +++ b/northd/ovn-northd.c @@ -11178,7 +11178,8 @@ update_northbound_cfg(struct northd_context *ctx, const struct sbrec_chassis *chassis; int64_t hv_cfg = nbg->nb_cfg; SBREC_CHASSIS_FOR_EACH (chassis, ctx->ovnsb_idl) { - if (chassis->nb_cfg < hv_cfg) { + if (!smap_get_bool(&chassis->external_ids, "is-remote", false) && + chassis->nb_cfg < hv_cfg) { hv_cfg = chassis->nb_cfg; } } @@ -11466,6 +11467,7 @@ main(int argc, char *argv[]) ovsdb_idl_add_table(ovnsb_idl_loop.idl, &sbrec_table_chassis); ovsdb_idl_add_column(ovnsb_idl_loop.idl, &sbrec_chassis_col_nb_cfg); ovsdb_idl_add_column(ovnsb_idl_loop.idl, &sbrec_chassis_col_name); + ovsdb_idl_add_column(ovnsb_idl_loop.idl, &sbrec_chassis_col_external_ids); ovsdb_idl_add_table(ovnsb_idl_loop.idl, &sbrec_table_ha_chassis); add_column_noalert(ovnsb_idl_loop.idl, diff --git a/ovn-sb.xml b/ovn-sb.xml index c08ceda..3ae9d4f 100644 --- a/ovn-sb.xml +++ b/ovn-sb.xml @@ -293,6 +293,21 @@ See ovn-controller(8) for more information. + + ovn-controller populates this key with the setting + configured in the column of the Open_vSwitch + database's table. + If set to true, the chassis is used as an interconnection gateway. + See ovn-controller(8) for more information. + + + + ovn-ic set this key to true for remote interconnection + gateway chassises learned from the interconnection southbound database. + See ovn-ic(8) for more information. + + ovn-controller populates this key with the transport zones configured in the X-Patchwork-Id: 1231084 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.136; helo=silver.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=ovn.org Received: from silver.osuosl.org (smtp3.osuosl.org [140.211.166.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 487Dlv75ZWz9sNT for ; Thu, 30 Jan 2020 06:57:15 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by silver.osuosl.org (Postfix) with ESMTP id 8362022728; Wed, 29 Jan 2020 19:57:14 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from silver.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id WJQNpoUqrFCY; Wed, 29 Jan 2020 19:57:07 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by silver.osuosl.org (Postfix) with ESMTP id C11472221C; Wed, 29 Jan 2020 19:56:44 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id B3E8BC1D85; Wed, 29 Jan 2020 19:56:44 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by lists.linuxfoundation.org (Postfix) with ESMTP id 57443C1D84 for ; Wed, 29 Jan 2020 19:56:42 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by whitealder.osuosl.org (Postfix) with ESMTP id 3B5528769B for ; Wed, 29 Jan 2020 19:56:42 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from whitealder.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 3zTjkb3VRw1j for ; Wed, 29 Jan 2020 19:56:36 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from relay1-d.mail.gandi.net (relay1-d.mail.gandi.net [217.70.183.193]) by whitealder.osuosl.org (Postfix) with ESMTPS id E9A0A876E2 for ; Wed, 29 Jan 2020 19:56:35 +0000 (UTC) X-Originating-IP: 216.113.160.77 Received: from localhost.localdomain.localdomain (unknown [216.113.160.77]) (Authenticated sender: hzhou@ovn.org) by relay1-d.mail.gandi.net (Postfix) with ESMTPSA id BA12D240007; Wed, 29 Jan 2020 19:56:33 +0000 (UTC) From: Han Zhou To: dev@openvswitch.org Date: Wed, 29 Jan 2020 11:56:03 -0800 Message-Id: <1580327768-36501-9-git-send-email-hzhou@ovn.org> X-Mailer: git-send-email 2.1.0 In-Reply-To: <1580327768-36501-1-git-send-email-hzhou@ovn.org> References: <1580327768-36501-1-git-send-email-hzhou@ovn.org> Cc: Han Zhou Subject: [ovs-dev] [PATCH ovn v4 08/13] ovn-ic: Interconnection gateway controller. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" Sync local and remote gateways between SB and IC-SB. Signed-off-by: Han Zhou --- ic/ovn-ic.c | 147 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++ tests/ovn-ic.at | 55 +++++++++++++++++++++ 2 files changed, 202 insertions(+) diff --git a/ic/ovn-ic.c b/ic/ovn-ic.c index 82bd86e..f352036 100644 --- a/ic/ovn-ic.c +++ b/ic/ovn-ic.c @@ -248,6 +248,152 @@ ts_run(struct ic_context *ctx) shash_destroy(&isb_dps); } +/* Returns true if any information in gw and chassis is different. */ +static bool +is_gateway_data_changed(const struct icsbrec_gateway *gw, + const struct sbrec_chassis *chassis) +{ + if (strcmp(gw->hostname, chassis->hostname)) { + return true; + } + + if (gw->n_encaps != chassis->n_encaps) { + return true; + } + + for (int g = 0; g < gw->n_encaps; g++) { + + bool found = false; + const struct icsbrec_encap *gw_encap = gw->encaps[g]; + for (int s = 0; s < chassis->n_encaps; s++) { + const struct sbrec_encap *chassis_encap = chassis->encaps[s]; + if (!strcmp(gw_encap->type, chassis_encap->type) && + !strcmp(gw_encap->ip, chassis_encap->ip)) { + found = true; + if (!smap_equal(&gw_encap->options, &chassis_encap->options)) { + return true; + } + break; + } + } + if (!found) { + return true; + } + } + + return false; +} + +static void +sync_isb_gw_to_sb(struct ic_context *ctx, + const struct icsbrec_gateway *gw, + const struct sbrec_chassis *chassis) +{ + sbrec_chassis_set_hostname(chassis, gw->hostname); + sbrec_chassis_update_external_ids_setkey(chassis, "is-remote", "true"); + + /* Sync encaps used by this gateway. */ + ovs_assert(gw->n_encaps); + struct sbrec_encap *sb_encap; + struct sbrec_encap **sb_encaps = + xmalloc(gw->n_encaps * sizeof *sb_encaps); + for (int i = 0; i < gw->n_encaps; i++) { + sb_encap = sbrec_encap_insert(ctx->ovnsb_txn); + sbrec_encap_set_chassis_name(sb_encap, gw->name); + sbrec_encap_set_ip(sb_encap, gw->encaps[i]->ip); + sbrec_encap_set_type(sb_encap, gw->encaps[i]->type); + sbrec_encap_set_options(sb_encap, &gw->encaps[i]->options); + sb_encaps[i] = sb_encap; + } + sbrec_chassis_set_encaps(chassis, sb_encaps, gw->n_encaps); + free(sb_encaps); +} + +static void +sync_sb_gw_to_isb(struct ic_context *ctx, + const struct sbrec_chassis *chassis, + const struct icsbrec_gateway *gw) +{ + icsbrec_gateway_set_hostname(gw, chassis->hostname); + + /* Sync encaps used by this chassis. */ + ovs_assert(chassis->n_encaps); + struct icsbrec_encap *isb_encap; + struct icsbrec_encap **isb_encaps = + xmalloc(chassis->n_encaps * sizeof *isb_encaps); + for (int i = 0; i < chassis->n_encaps; i++) { + isb_encap = icsbrec_encap_insert(ctx->ovnisb_txn); + icsbrec_encap_set_gateway_name(isb_encap, + chassis->name); + icsbrec_encap_set_ip(isb_encap, chassis->encaps[i]->ip); + icsbrec_encap_set_type(isb_encap, + chassis->encaps[i]->type); + icsbrec_encap_set_options(isb_encap, + &chassis->encaps[i]->options); + isb_encaps[i] = isb_encap; + } + icsbrec_gateway_set_encaps(gw, isb_encaps, + chassis->n_encaps); + free(isb_encaps); +} + +static void +gateway_run(struct ic_context *ctx, const struct icsbrec_availability_zone *az) +{ + if (!ctx->ovnisb_txn || !ctx->ovnsb_txn) { + return; + } + + struct shash local_gws = SHASH_INITIALIZER(&local_gws); + struct shash remote_gws = SHASH_INITIALIZER(&remote_gws); + const struct icsbrec_gateway *gw; + ICSBREC_GATEWAY_FOR_EACH (gw, ctx->ovnisb_idl) { + if (gw->availability_zone == az) { + shash_add(&local_gws, gw->name, gw); + } else { + shash_add(&remote_gws, gw->name, gw); + } + } + + const struct sbrec_chassis *chassis; + SBREC_CHASSIS_FOR_EACH (chassis, ctx->ovnsb_idl) { + if (smap_get_bool(&chassis->external_ids, "is-interconn", false)) { + gw = shash_find_and_delete(&local_gws, chassis->name); + if (!gw) { + gw = icsbrec_gateway_insert(ctx->ovnisb_txn); + icsbrec_gateway_set_availability_zone(gw, az); + icsbrec_gateway_set_name(gw, chassis->name); + sync_sb_gw_to_isb(ctx, chassis, gw); + } else if (is_gateway_data_changed(gw, chassis)) { + sync_sb_gw_to_isb(ctx, chassis, gw); + } + } else if (smap_get_bool(&chassis->external_ids, "is-remote", false)) { + gw = shash_find_and_delete(&remote_gws, chassis->name); + if (!gw) { + sbrec_chassis_delete(chassis); + } else if (is_gateway_data_changed(gw, chassis)) { + sync_isb_gw_to_sb(ctx, gw, chassis); + } + } + } + + /* Delete extra gateways from ISB for the local AZ */ + struct shash_node *node; + SHASH_FOR_EACH (node, &local_gws) { + icsbrec_gateway_delete(node->data); + } + shash_destroy(&local_gws); + + /* Create SB chassis for remote gateways in ISB */ + SHASH_FOR_EACH (node, &remote_gws) { + gw = node->data; + chassis = sbrec_chassis_insert(ctx->ovnsb_txn); + sbrec_chassis_set_name(chassis, gw->name); + sync_isb_gw_to_sb(ctx, gw, chassis); + } + shash_destroy(&remote_gws); +} + static void ovn_db_run(struct ic_context *ctx) { @@ -259,6 +405,7 @@ ovn_db_run(struct ic_context *ctx) } ts_run(ctx); + gateway_run(ctx, az); } static void diff --git a/tests/ovn-ic.at b/tests/ovn-ic.at index 3d77a7d..62cf88f 100644 --- a/tests/ovn-ic.at +++ b/tests/ovn-ic.at @@ -66,3 +66,58 @@ ts2 OVN_CLEANUP_IC([az1]) AT_CLEANUP + +AT_SETUP([ovn-ic -- gateway sync]) + +ovn_init_ic_db +net_add n1 +ovn_start az1 +ovn_start az2 +sim_add gw1 +as gw1 +ovs-vsctl add-br br-phys +ovn_az_attach az1 n1 br-phys 192.168.0.1 +ovs-vsctl set open . external-ids:ovn-is-interconn=true external-ids:hostname=gw1 + +AT_CHECK([ovn_as az2 ovn-sbctl show | sort -r], [0], [dnl +Chassis gw1 + hostname: gw1 + Encap vxlan + Encap geneve + options: {csum="true"} + options: {csum="true"} + ip: "192.168.0.1" + ip: "192.168.0.1" +]) + +AT_CHECK([ovn_as az2 ovn-sbctl -f csv -d bare --no-headings --columns external_ids list chassis], [0], [dnl +is-remote=true +]) + +ovs-vsctl set open . external-ids:ovn-is-interconn=false +AT_CHECK([ovn_as az2 ovn-sbctl show], [0], [dnl +]) + +ovs-vsctl set open . external-ids:ovn-is-interconn=true +AT_CHECK([ovn_as az2 ovn-sbctl show | grep gw1], [0], [ignore]) + +OVN_CLEANUP_SBOX(gw1) +AT_CHECK([ovn_as az2 ovn-sbctl show], [0], [dnl +]) + +# Test encap change +sim_add gw2 +as gw2 +ovs-vsctl add-br br-phys +ovn_az_attach az1 n1 br-phys 192.168.0.1 +ovs-vsctl set open . external-ids:ovn-is-interconn=true +OVS_WAIT_UNTIL([ovn_as az2 ovn-sbctl show | grep "192.168.0.1"]) +ovs-vsctl set open . external_ids:ovn-encap-ip=192.168.0.2 +OVS_WAIT_UNTIL([ovn_as az2 ovn-sbctl show | grep "192.168.0.2"]) +ovs-vsctl set open . external_ids:ovn-encap-type="geneve,stt" +OVS_WAIT_UNTIL([ovn_as az2 ovn-sbctl show | grep stt]) + +OVN_CLEANUP_SBOX(gw2) +OVN_CLEANUP_IC([az1], [az2]) + +AT_CLEANUP From patchwork Wed Jan 29 19:56:04 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Han Zhou X-Patchwork-Id: 1231083 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.137; helo=fraxinus.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=ovn.org Received: from fraxinus.osuosl.org (smtp4.osuosl.org [140.211.166.137]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 487Dlr5vwTz9sNT for ; Thu, 30 Jan 2020 06:57:12 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by fraxinus.osuosl.org (Postfix) with ESMTP id 209528614E; Wed, 29 Jan 2020 19:57:11 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from fraxinus.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 0zhuEv2LIJI9; Wed, 29 Jan 2020 19:57:05 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by fraxinus.osuosl.org (Postfix) with ESMTP id BA5298606C; Wed, 29 Jan 2020 19:56:56 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id B028FC0881; Wed, 29 Jan 2020 19:56:56 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from silver.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by lists.linuxfoundation.org (Postfix) with ESMTP id A1733C0881 for ; Wed, 29 Jan 2020 19:56:54 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by silver.osuosl.org (Postfix) with ESMTP id 90C9622128 for ; Wed, 29 Jan 2020 19:56:54 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from silver.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id PoAy1nlIA6CS for ; Wed, 29 Jan 2020 19:56:38 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from relay1-d.mail.gandi.net (relay1-d.mail.gandi.net [217.70.183.193]) by silver.osuosl.org (Postfix) with ESMTPS id 54308203CF for ; Wed, 29 Jan 2020 19:56:37 +0000 (UTC) X-Originating-IP: 216.113.160.77 Received: from localhost.localdomain.localdomain (unknown [216.113.160.77]) (Authenticated sender: hzhou@ovn.org) by relay1-d.mail.gandi.net (Postfix) with ESMTPSA id 22EF2240005; Wed, 29 Jan 2020 19:56:34 +0000 (UTC) From: Han Zhou To: dev@openvswitch.org Date: Wed, 29 Jan 2020 11:56:04 -0800 Message-Id: <1580327768-36501-10-git-send-email-hzhou@ovn.org> X-Mailer: git-send-email 2.1.0 In-Reply-To: <1580327768-36501-1-git-send-email-hzhou@ovn.org> References: <1580327768-36501-1-git-send-email-hzhou@ovn.org> Cc: Han Zhou Subject: [ovs-dev] [PATCH ovn v4 09/13] ovn-ic: Interconnection port controller. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" Sync interconnection logical ports and bindings between NB, SB and IC-SB. With this patch, the OVN interconnection works end to end. Signed-off-by: Han Zhou --- controller/binding.c | 6 +- ic/ovn-ic.c | 385 ++++++++++++++++++++++++++++++++++++++++++++++++- lib/ovn-util.c | 7 + lib/ovn-util.h | 2 + northd/ovn-northd.c | 69 +++++++-- ovn-architecture.7.xml | 2 +- ovn-nb.xml | 35 ++++- tests/ovn-ic.at | 65 +++++++++ 8 files changed, 552 insertions(+), 19 deletions(-) diff --git a/controller/binding.c b/controller/binding.c index 6e752db..c3376e2 100644 --- a/controller/binding.c +++ b/controller/binding.c @@ -834,11 +834,13 @@ binding_evaluate_port_binding_changes( * - If a regular VIF is unbound from this chassis, the local ovsdb * interface table will be updated, which will trigger recompute. * - * - If the port is not a regular VIF, always trigger recompute. */ + * - If the port is not a regular VIF, and not a "remote" port, + * always trigger recompute. */ if (binding_rec->chassis == chassis_rec || is_our_chassis(chassis_rec, binding_rec, active_tunnels, &lport_to_iface, local_lports) - || strcmp(binding_rec->type, "")) { + || (strcmp(binding_rec->type, "") && strcmp(binding_rec->type, + "remote"))) { changed = true; break; } diff --git a/ic/ovn-ic.c b/ic/ovn-ic.c index f352036..25ca3f7 100644 --- a/ic/ovn-ic.c +++ b/ic/ovn-ic.c @@ -60,6 +60,10 @@ struct ic_context { struct ovsdb_idl_txn *ovnsb_txn; struct ovsdb_idl_txn *ovninb_txn; struct ovsdb_idl_txn *ovnisb_txn; + struct ovsdb_idl_index *nbrec_ls_by_name; + struct ovsdb_idl_index *sbrec_chassis_by_name; + struct ovsdb_idl_index *sbrec_port_binding_by_name; + struct ovsdb_idl_index *icsbrec_port_binding_by_ts; }; struct ic_state { @@ -182,7 +186,7 @@ ts_run(struct ic_context *ctx) } } - /* Create NB Logical_Switch for each TS */ + /* Create/update NB Logical_Switch for each TS */ ICNBREC_TRANSIT_SWITCH_FOR_EACH (ts, ctx->ovninb_idl) { ls = shash_find_and_delete(&nb_tses, ts->name); if (!ls) { @@ -394,6 +398,366 @@ gateway_run(struct ic_context *ctx, const struct icsbrec_availability_zone *az) shash_destroy(&remote_gws); } +static const struct nbrec_logical_switch * +find_ts_in_nb(struct ic_context *ctx, char *ts_name) +{ + const struct nbrec_logical_switch *key = + nbrec_logical_switch_index_init_row(ctx->nbrec_ls_by_name); + nbrec_logical_switch_index_set_name(key, ts_name); + + const struct nbrec_logical_switch *ls; + bool found = false; + NBREC_LOGICAL_SWITCH_FOR_EACH_EQUAL (ls, key, ctx->nbrec_ls_by_name) { + const char *ls_ts_name = smap_get(&ls->other_config, "interconn-ts"); + if (ls_ts_name && !strcmp(ts_name, ls_ts_name)) { + found = true; + break; + } + } + nbrec_logical_switch_index_destroy_row(key); + + if (found) { + return ls; + } + return NULL; +} + +static const struct sbrec_port_binding * +find_sb_pb_by_name(struct ovsdb_idl_index *sbrec_port_binding_by_name, + const char *name) +{ + const struct sbrec_port_binding *key = + sbrec_port_binding_index_init_row(sbrec_port_binding_by_name); + sbrec_port_binding_index_set_logical_port(key, name); + + const struct sbrec_port_binding *pb = + sbrec_port_binding_index_find(sbrec_port_binding_by_name, key); + sbrec_port_binding_index_destroy_row(key); + + return pb; +} + +static const struct sbrec_port_binding * +find_peer_port(struct ic_context *ctx, + const struct sbrec_port_binding *sb_pb) +{ + const char *peer_name = smap_get(&sb_pb->options, "peer"); + if (!peer_name) { + return NULL; + } + + return find_sb_pb_by_name(ctx->sbrec_port_binding_by_name, peer_name); +} + +static const struct sbrec_port_binding * +find_crp_from_lrp(struct ic_context *ctx, + const struct sbrec_port_binding *lrp_pb) +{ + char *crp_name = ovn_chassis_redirect_name(lrp_pb->logical_port); + + const struct sbrec_port_binding *pb = + find_sb_pb_by_name(ctx->sbrec_port_binding_by_name, crp_name); + + free(crp_name); + return pb; +} + +static const struct sbrec_port_binding * +find_crp_for_sb_pb(struct ic_context *ctx, + const struct sbrec_port_binding *sb_pb) +{ + const struct sbrec_port_binding *peer = find_peer_port(ctx, sb_pb); + if (!peer) { + return NULL; + } + + return find_crp_from_lrp(ctx, peer); +} + +static const char * +get_lrp_address_for_sb_pb(struct ic_context *ctx, + const struct sbrec_port_binding *sb_pb) +{ + const struct sbrec_port_binding *peer = find_peer_port(ctx, sb_pb); + if (!peer) { + return NULL; + } + + return peer->n_mac ? *peer->mac : NULL; +} + +static const struct sbrec_chassis * +find_sb_chassis(struct ic_context *ctx, const char *name) +{ + const struct sbrec_chassis *key = + sbrec_chassis_index_init_row(ctx->sbrec_chassis_by_name); + sbrec_chassis_index_set_name(key, name); + + const struct sbrec_chassis *chassis = + sbrec_chassis_index_find(ctx->sbrec_chassis_by_name, key); + sbrec_chassis_index_destroy_row(key); + + return chassis; +} + +static void +sync_lsp_tnl_key(const struct nbrec_logical_switch_port *lsp, + int64_t isb_tnl_key) +{ + int64_t tnl_key = smap_get_int(&lsp->options, "requested-tnl-key", 0); + if (tnl_key != isb_tnl_key) { + VLOG_DBG("Set options:requested-tnl-key %"PRId64 + " for lsp %s in NB.", isb_tnl_key, lsp->name); + char *tnl_key_str = xasprintf("%"PRId64, isb_tnl_key); + nbrec_logical_switch_port_update_options_setkey(lsp, + "requested-tnl-key", + tnl_key_str); + free(tnl_key_str); + } + +} + +/* For each local port: + * - Sync from NB to ISB. + * - Sync gateway from SB to ISB. + * - Sync tunnel key from ISB to NB. + */ +static void +sync_local_port(struct ic_context *ctx, + const struct icsbrec_port_binding *isb_pb, + const struct sbrec_port_binding *sb_pb, + const struct nbrec_logical_switch_port *lsp) +{ + /* Sync address from NB to ISB */ + const char *address = get_lrp_address_for_sb_pb(ctx, sb_pb); + if (!address) { + VLOG_DBG("Can't get logical router port address for logical" + " switch port %s", sb_pb->logical_port); + if (isb_pb->address[0]) { + icsbrec_port_binding_set_address(isb_pb, ""); + } + } else { + if (strcmp(address, isb_pb->address)) { + icsbrec_port_binding_set_address(isb_pb, address); + } + } + + /* Sync gateway from SB to ISB */ + const struct sbrec_port_binding *crp = find_crp_for_sb_pb(ctx, sb_pb); + if (crp && crp->chassis) { + if (strcmp(crp->chassis->name, isb_pb->gateway)) { + icsbrec_port_binding_set_gateway(isb_pb, crp->chassis->name); + } + } else { + if (isb_pb->gateway[0]) { + icsbrec_port_binding_set_gateway(isb_pb, ""); + } + } + + /* Sync back tunnel key from ISB to NB */ + sync_lsp_tnl_key(lsp, isb_pb->tunnel_key); +} + +/* For each remote port: + * - Sync from ISB to NB + * - Sync gateway from ISB to SB + */ +static void +sync_remote_port(struct ic_context *ctx, + const struct icsbrec_port_binding *isb_pb, + const struct nbrec_logical_switch_port *lsp, + const struct sbrec_port_binding *sb_pb) +{ + /* Sync address from ISB to NB */ + if (isb_pb->address[0]) { + if (lsp->n_addresses != 1 || + strcmp(isb_pb->address, lsp->addresses[0])) { + nbrec_logical_switch_port_set_addresses( + lsp, (const char **)&isb_pb->address, 1); + } + } else { + if (lsp->n_addresses != 0) { + nbrec_logical_switch_port_set_addresses(lsp, NULL, 0); + } + } + + /* Sync tunnel key from ISB to NB */ + sync_lsp_tnl_key(lsp, isb_pb->tunnel_key); + + /* Sync gateway from ISB to SB */ + if (isb_pb->gateway[0]) { + if (!sb_pb->chassis || strcmp(sb_pb->chassis->name, isb_pb->gateway)) { + const struct sbrec_chassis *chassis = + find_sb_chassis(ctx, isb_pb->gateway); + if (!chassis) { + VLOG_DBG("Chassis %s is not found in SB, syncing from ISB " + "to SB skipped for logical port %s.", + isb_pb->gateway, lsp->name); + return; + } + sbrec_port_binding_set_chassis(sb_pb, chassis); + } + } else { + if (sb_pb->chassis) { + sbrec_port_binding_set_chassis(sb_pb, NULL); + } + } +} + +static void +create_nb_lsp(struct ic_context *ctx, + const struct icsbrec_port_binding *isb_pb, + const struct nbrec_logical_switch *ls) +{ + const struct nbrec_logical_switch_port *lsp = + nbrec_logical_switch_port_insert(ctx->ovnnb_txn); + nbrec_logical_switch_port_set_name(lsp, isb_pb->logical_port); + nbrec_logical_switch_port_set_type(lsp, "remote"); + + bool up = true; + nbrec_logical_switch_port_set_up(lsp, &up, 1); + + if (isb_pb->address[0]) { + nbrec_logical_switch_port_set_addresses( + lsp, (const char **)&isb_pb->address, 1); + } + sync_lsp_tnl_key(lsp, isb_pb->tunnel_key); + nbrec_logical_switch_update_ports_addvalue(ls, lsp); +} + +static void +create_isb_pb(struct ic_context *ctx, + const struct sbrec_port_binding *sb_pb, + const struct icsbrec_availability_zone *az, + const char *ts_name, + uint32_t pb_tnl_key) +{ + const struct icsbrec_port_binding *isb_pb = + icsbrec_port_binding_insert(ctx->ovnisb_txn); + icsbrec_port_binding_set_availability_zone(isb_pb, az); + icsbrec_port_binding_set_transit_switch(isb_pb, ts_name); + icsbrec_port_binding_set_logical_port(isb_pb, sb_pb->logical_port); + icsbrec_port_binding_set_tunnel_key(isb_pb, pb_tnl_key); + + const char *address = get_lrp_address_for_sb_pb(ctx, sb_pb); + if (address) { + icsbrec_port_binding_set_address(isb_pb, address); + } + + const struct sbrec_port_binding *crp = find_crp_for_sb_pb(ctx, sb_pb); + if (crp && crp->chassis) { + icsbrec_port_binding_set_gateway(isb_pb, crp->chassis->name); + } + + /* XXX: Sync encap so that multiple encaps can be used for the same + * gateway. However, it is not needed for now, since we don't yet + * support specifying encap type/ip for gateway chassis or ha-chassis + * for logical router port in NB DB, and now encap should always be + * empty. The sync can be added if we add such support for gateway + * chassis/ha-chassis in NB DB. */ +} + +static const struct sbrec_port_binding * +find_lsp_in_sb(struct ic_context *ctx, + const struct nbrec_logical_switch_port *lsp) +{ + return find_sb_pb_by_name(ctx->sbrec_port_binding_by_name, lsp->name); +} + +static uint32_t +allocate_port_key(struct hmap *pb_tnlids) +{ + static uint32_t hint; + return ovn_allocate_tnlid(pb_tnlids, "transit port", + 1, (1u << 15) - 1, &hint); +} + +static void +port_binding_run(struct ic_context *ctx, + const struct icsbrec_availability_zone *az) +{ + if (!ctx->ovnisb_txn || !ctx->ovnnb_txn || !ctx->ovnsb_txn) { + return; + } + + const struct icnbrec_transit_switch *ts; + ICNBREC_TRANSIT_SWITCH_FOR_EACH (ts, ctx->ovninb_idl) { + const struct nbrec_logical_switch *ls = find_ts_in_nb(ctx, ts->name); + if (!ls) { + VLOG_DBG("Transit switch %s not found in NB.", ts->name); + continue; + } + struct shash local_pbs = SHASH_INITIALIZER(&local_pbs); + struct shash remote_pbs = SHASH_INITIALIZER(&remote_pbs); + struct hmap pb_tnlids = HMAP_INITIALIZER(&pb_tnlids); + const struct icsbrec_port_binding *isb_pb; + const struct icsbrec_port_binding *isb_pb_key = + icsbrec_port_binding_index_init_row( + ctx->icsbrec_port_binding_by_ts); + icsbrec_port_binding_index_set_transit_switch(isb_pb_key, ts->name); + + ICSBREC_PORT_BINDING_FOR_EACH_EQUAL (isb_pb, isb_pb_key, + ctx->icsbrec_port_binding_by_ts) { + if (isb_pb->availability_zone == az) { + shash_add(&local_pbs, isb_pb->logical_port, isb_pb); + } else { + shash_add(&remote_pbs, isb_pb->logical_port, isb_pb); + } + ovn_add_tnlid(&pb_tnlids, isb_pb->tunnel_key); + } + icsbrec_port_binding_index_destroy_row(isb_pb_key); + + const struct nbrec_logical_switch_port *lsp; + for (int i = 0; i < ls->n_ports; i++) { + lsp = ls->ports[i]; + + const struct sbrec_port_binding *sb_pb = find_lsp_in_sb(ctx, lsp); + if (!strcmp(lsp->type, "router")) { + /* The port is local. */ + if (!sb_pb) { + continue; + } + isb_pb = shash_find_and_delete(&local_pbs, lsp->name); + if (!isb_pb) { + uint32_t pb_tnl_key = allocate_port_key(&pb_tnlids); + create_isb_pb(ctx, sb_pb, az, ts->name, pb_tnl_key); + } else { + sync_local_port(ctx, isb_pb, sb_pb, lsp); + } + } else if (!strcmp(lsp->type, "remote")) { + /* The port is remote. */ + isb_pb = shash_find_and_delete(&remote_pbs, lsp->name); + if (!isb_pb) { + nbrec_logical_switch_update_ports_delvalue(ls, lsp); + } else { + if (!sb_pb) { + continue; + } + sync_remote_port(ctx, isb_pb, lsp, sb_pb); + } + } else { + VLOG_DBG("Ignore lsp %s on ts %s with type %s.", + lsp->name, ts->name, lsp->type); + } + } + + /* Delete extra port-binding from ISB */ + struct shash_node *node; + SHASH_FOR_EACH (node, &local_pbs) { + icsbrec_port_binding_delete(node->data); + } + + /* Create lsp in NB for remote ports */ + SHASH_FOR_EACH (node, &remote_pbs) { + create_nb_lsp(ctx, node->data, ls); + } + + shash_destroy(&local_pbs); + shash_destroy(&remote_pbs); + ovn_destroy_tnlids(&pb_tnlids); + } +} + static void ovn_db_run(struct ic_context *ctx) { @@ -406,6 +770,7 @@ ovn_db_run(struct ic_context *ctx) ts_run(ctx); gateway_run(ctx, az); + port_binding_run(ctx, az); } static void @@ -561,6 +926,20 @@ main(int argc, char *argv[]) struct ovsdb_idl_loop ovnsb_idl_loop = OVSDB_IDL_LOOP_INITIALIZER( ovsdb_idl_create(ovnsb_db, &sbrec_idl_class, true, true)); + /* Create IDL indexes */ + struct ovsdb_idl_index *nbrec_ls_by_name + = ovsdb_idl_index_create1(ovnnb_idl_loop.idl, + &nbrec_logical_switch_col_name); + struct ovsdb_idl_index *sbrec_port_binding_by_name + = ovsdb_idl_index_create1(ovnsb_idl_loop.idl, + &sbrec_port_binding_col_logical_port); + struct ovsdb_idl_index *sbrec_chassis_by_name + = ovsdb_idl_index_create1(ovnsb_idl_loop.idl, + &sbrec_chassis_col_name); + struct ovsdb_idl_index *icsbrec_port_binding_by_ts + = ovsdb_idl_index_create1(ovnisb_idl_loop.idl, + &icsbrec_port_binding_col_transit_switch); + /* Main loop. */ exiting = false; state.had_lock = false; @@ -586,6 +965,10 @@ main(int argc, char *argv[]) .ovninb_txn = ovsdb_idl_loop_run(&ovninb_idl_loop), .ovnisb_idl = ovnisb_idl_loop.idl, .ovnisb_txn = ovsdb_idl_loop_run(&ovnisb_idl_loop), + .nbrec_ls_by_name = nbrec_ls_by_name, + .sbrec_port_binding_by_name = sbrec_port_binding_by_name, + .sbrec_chassis_by_name = sbrec_chassis_by_name, + .icsbrec_port_binding_by_ts = icsbrec_port_binding_by_ts, }; if (!state.had_lock && ovsdb_idl_has_lock(ovnsb_idl_loop.idl)) { diff --git a/lib/ovn-util.c b/lib/ovn-util.c index f604532..ba643c5 100644 --- a/lib/ovn-util.c +++ b/lib/ovn-util.c @@ -404,6 +404,7 @@ static const char *OVN_NB_LSP_TYPES[] = { "vtep", "external", "virtual", + "remote", }; bool @@ -514,3 +515,9 @@ ovn_allocate_tnlid(struct hmap *set, const char *name, uint32_t min, VLOG_WARN_RL(&rl, "all %s tunnel ids exhausted", name); return 0; } + +char * +ovn_chassis_redirect_name(const char *port_name) +{ + return xasprintf("cr-%s", port_name); +} diff --git a/lib/ovn-util.h b/lib/ovn-util.h index d99123f..d0a2645 100644 --- a/lib/ovn-util.h +++ b/lib/ovn-util.h @@ -103,4 +103,6 @@ void ovn_add_tnlid(struct hmap *set, uint32_t tnlid); bool ovn_tnlid_in_use(const struct hmap *set, uint32_t tnlid); uint32_t ovn_allocate_tnlid(struct hmap *set, const char *name, uint32_t min, uint32_t max, uint32_t *hint); + +char *ovn_chassis_redirect_name(const char *port_name); #endif diff --git a/northd/ovn-northd.c b/northd/ovn-northd.c index a3f96b0..a7103cb 100644 --- a/northd/ovn-northd.c +++ b/northd/ovn-northd.c @@ -1250,12 +1250,6 @@ lrport_is_enabled(const struct nbrec_logical_router_port *lrport) return !lrport->enabled || *lrport->enabled; } -static char * -chassis_redirect_name(const char *port_name) -{ - return xasprintf("cr-%s", port_name); -} - static bool ipam_is_duplicate_mac(struct eth_addr *ea, uint64_t mac64, bool warn) { @@ -2142,7 +2136,8 @@ join_logical_ports(struct northd_context *ctx, continue; } - char *redirect_name = chassis_redirect_name(nbrp->name); + char *redirect_name = + ovn_chassis_redirect_name(nbrp->name); struct ovn_port *crp = ovn_port_find(ports, redirect_name); if (crp) { crp->derived = true; @@ -2650,6 +2645,24 @@ copy_gw_chassis_from_nbrp_to_sbpb( free(sb_ha_chassis); } +static int64_t +op_get_requested_tnl_key(const struct ovn_port *op) +{ + ovs_assert(op->nbsp || op->nbrp); + const struct smap *op_options = op->nbsp ? &op->nbsp->options + : &op->nbrp->options; + return smap_get_int(op_options, "requested-tnl-key", 0); +} + +static const char* +op_get_name(const struct ovn_port *op) +{ + ovs_assert(op->nbsp || op->nbrp); + const char *name = op->nbsp ? op->nbsp->name + : op->nbrp->name; + return name; +} + static void ovn_port_update_sbrec(struct northd_context *ctx, struct ovsdb_idl_index *sbrec_chassis_by_name, @@ -2978,6 +2991,18 @@ ovn_port_update_sbrec(struct northd_context *ctx, sbrec_port_binding_set_external_ids(op->sb, &ids); smap_destroy(&ids); } + int64_t tnl_key = op_get_requested_tnl_key(op); + if (tnl_key && tnl_key != op->sb->tunnel_key) { + if (ovn_tnlid_in_use(&op->od->port_tnlids, tnl_key)) { + static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 1); + VLOG_WARN_RL(&rl, "Cannot update port binding for " + "%s due to duplicate key set " + "in options:requested-tnl-key: %"PRId64, + op_get_name(op), tnl_key); + } else { + sbrec_port_binding_set_tunnel_key(op->sb, tnl_key); + } + } } /* Remove mac_binding entries that refer to logical_ports which are @@ -3347,8 +3372,16 @@ build_ports(struct northd_context *ctx, &tag_alloc_table, &sb_only, &nb_only, &both); struct ovn_port *op, *next; + /* For logical ports that are in both databases, index the in-use + * tunnel_keys. */ + LIST_FOR_EACH (op, list, &both) { + ovn_add_tnlid(&op->od->port_tnlids, op->sb->tunnel_key); + if (op->sb->tunnel_key > op->od->port_key_hint) { + op->od->port_key_hint = op->sb->tunnel_key; + } + } /* For logical ports that are in both databases, update the southbound - * record based on northbound data. Also index the in-use tunnel_keys. + * record based on northbound data. * For logical ports that are in NB database, do any tag allocation * needed. */ LIST_FOR_EACH_SAFE (op, next, list, &both) { @@ -3358,19 +3391,27 @@ build_ports(struct northd_context *ctx, ovn_port_update_sbrec(ctx, sbrec_chassis_by_name, op, &chassis_qdisc_queues, &active_ha_chassis_grps); - ovn_add_tnlid(&op->od->port_tnlids, op->sb->tunnel_key); - if (op->sb->tunnel_key > op->od->port_key_hint) { - op->od->port_key_hint = op->sb->tunnel_key; - } } /* Add southbound record for each unmatched northbound record. */ LIST_FOR_EACH_SAFE (op, next, list, &nb_only) { - uint16_t tunnel_key = ovn_port_allocate_key(op->od); - if (!tunnel_key) { + int64_t tunnel_key = op_get_requested_tnl_key(op); + if (tunnel_key && ovn_tnlid_in_use(&op->od->port_tnlids, tunnel_key)) { + static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 1); + VLOG_WARN_RL(&rl, "Cannot create port binding for " + "%s due to duplicate key set " + "in options:requested-tnl-key: %"PRId64, + op_get_name(op), tunnel_key); continue; } + if (!tunnel_key) { + tunnel_key = ovn_port_allocate_key(op->od); + if (!tunnel_key) { + continue; + } + } + ovn_port_set_sb(op, sbrec_port_binding_insert(ctx->ovnsb_txn)); ovn_port_update_sbrec(ctx, sbrec_chassis_by_name, op, &chassis_qdisc_queues, diff --git a/ovn-architecture.7.xml b/ovn-architecture.7.xml index defcdc9..71efa41 100644 --- a/ovn-architecture.7.xml +++ b/ovn-architecture.7.xml @@ -1827,7 +1827,7 @@ -

OVN Deployments Interconnection (TODO)

+

OVN Deployments Interconnection

It is not uncommon for an operator to deploy multiple OVN clusters, for diff --git a/ovn-nb.xml b/ovn-nb.xml index a092af8..7bae14c 100644 --- a/ovn-nb.xml +++ b/ovn-nb.xml @@ -537,7 +537,16 @@ parent owning the virtual ip. - + + +

remote
+
+ A remote port is to model a port that resides remotely on another + OVN, which is on the other side of a transit logical switch for OVN + interconnection. This type of ports are created by + ovn-ic instead of by CMS. Any change to the port will + be automatically overwritten by ovn-ic. +
@@ -1176,6 +1185,21 @@ + + + Configures the port binding tunnel key for the port. Usually + this is not needed because ovn-northd will assign an + unique key for each port by itself. However, if it is configured, + ovn-northd honors the configured value. The typical use + case is for interconnection: the tunnel keys for ports on transit + switches need to be unique globally, so they are maintained in the + global database, and ovn-ic + simply syncs the value from through this + config. + + +

@@ -2212,6 +2236,15 @@ to true.

+ + + Configures the port binding tunnel key for the port. Usually + this is not needed because ovn-northd will assign an + unique key for each port by itself. However, if it is configured, + ovn-northd honors the configured value. + + diff --git a/tests/ovn-ic.at b/tests/ovn-ic.at index 62cf88f..d506442 100644 --- a/tests/ovn-ic.at +++ b/tests/ovn-ic.at @@ -121,3 +121,68 @@ OVN_CLEANUP_SBOX(gw2) OVN_CLEANUP_IC([az1], [az2]) AT_CLEANUP + + +AT_SETUP([ovn-ic -- port sync]) + +ovn_init_ic_db +ovn-ic-nbctl ts-add ts1 +net_add n1 +ovn_start az1 +ovn_start az2 +sim_add gw1 +as gw1 +ovs-vsctl add-br br-phys +ovn_az_attach az1 n1 br-phys 192.168.0.1 +ovs-vsctl set open . external-ids:ovn-is-interconn=true + +ovn_as az1 +OVS_WAIT_UNTIL([ovn-sbctl list datapath_binding | grep interconn-ts | grep ts1]) + +# Create LRP and connect to TS +ovn-nbctl lr-add lr1 +ovn-nbctl lrp-add lr1 lrp-lr1-ts1 aa:aa:aa:aa:aa:01 169.254.100.1/24 +ovn-nbctl lsp-add ts1 lsp-ts1-lr1 +ovn-nbctl lsp-set-addresses lsp-ts1-lr1 router +ovn-nbctl lsp-set-type lsp-ts1-lr1 router +ovn-nbctl lsp-set-options lsp-ts1-lr1 router-port=lrp-lr1-ts1 + +AT_CHECK([ovn_as az2 ovn-nbctl show | uuidfilt], [0], [dnl +switch <0> (ts1) + port lsp-ts1-lr1 + type: remote + addresses: [["aa:aa:aa:aa:aa:01 169.254.100.1/24"]] +]) + +AT_CHECK([ovn_as az2 ovn-sbctl -f csv -d bare --no-headings --columns logical_port,type list port_binding], [0], [dnl +lsp-ts1-lr1,remote +]) + +ovn-nbctl lrp-set-gateway-chassis lrp-lr1-ts1 gw1 +OVS_WAIT_UNTIL([ovn_as az2 ovn-sbctl show | grep lsp-ts1-lr1]) + +ovn-nbctl lrp-del-gateway-chassis lrp-lr1-ts1 gw1 +OVS_WAIT_WHILE([ovn_as az2 ovn-sbctl show | grep lsp-ts1-lr1]) + +ovn-nbctl set logical_router_port lrp-lr1-ts1 mac="\"aa:aa:aa:aa:aa:02\"" \ + networks="169.254.100.2/24 169.254.200.3/24" +OVS_WAIT_UNTIL([ovn_as az2 ovn-nbctl show | grep "aa:aa:aa:aa:aa:02 169.254.100.2/24 169.254.200.3/24"]) + +# Delete the router port from az1, the remote port in az2 should still remain +# but just lost address. +ovn-nbctl lrp-del lrp-lr1-ts1 +OVS_WAIT_WHILE([ovn_as az2 ovn-nbctl show | grep "aa:aa:aa:aa:aa:02 169.254.100.2/24 169.254.200.3/24"]) +AT_CHECK([ovn_as az2 ovn-nbctl show | uuidfilt], [0], [dnl +switch <0> (ts1) + port lsp-ts1-lr1 + type: remote +]) + +# Delete the lsp from az1, the remote port in az2 should be gone +ovn-nbctl lsp-del lsp-ts1-lr1 +OVS_WAIT_WHILE([ovn_as az2 ovn-nbctl show | grep lsp-ts1-lr1]) + +OVN_CLEANUP_SBOX(gw1) +OVN_CLEANUP_IC([az1], [az2]) + +AT_CLEANUP From patchwork Wed Jan 29 19:56:05 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Han Zhou X-Patchwork-Id: 1231085 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.136; helo=silver.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=ovn.org Received: from silver.osuosl.org (smtp3.osuosl.org [140.211.166.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 487Dm86nJ5z9sNT for ; Thu, 30 Jan 2020 06:57:28 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by silver.osuosl.org (Postfix) with ESMTP id 63BFC227AA; Wed, 29 Jan 2020 19:57:27 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from silver.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id WamkljR+pnsy; Wed, 29 Jan 2020 19:57:22 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by silver.osuosl.org (Postfix) with ESMTP id EFEFD22618; Wed, 29 Jan 2020 19:56:46 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id E519AC1D84; Wed, 29 Jan 2020 19:56:46 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by lists.linuxfoundation.org (Postfix) with ESMTP id 3D0EFC0171 for ; Wed, 29 Jan 2020 19:56:43 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id 217E088154 for ; Wed, 29 Jan 2020 19:56:43 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id NBPAWfyMXkA6 for ; Wed, 29 Jan 2020 19:56:39 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from relay1-d.mail.gandi.net (relay1-d.mail.gandi.net [217.70.183.193]) by hemlock.osuosl.org (Postfix) with ESMTPS id AFA5B87E62 for ; Wed, 29 Jan 2020 19:56:38 +0000 (UTC) X-Originating-IP: 216.113.160.77 Received: from localhost.localdomain.localdomain (unknown [216.113.160.77]) (Authenticated sender: hzhou@ovn.org) by relay1-d.mail.gandi.net (Postfix) with ESMTPSA id 76F4C24000A; Wed, 29 Jan 2020 19:56:36 +0000 (UTC) From: Han Zhou To: dev@openvswitch.org Date: Wed, 29 Jan 2020 11:56:05 -0800 Message-Id: <1580327768-36501-11-git-send-email-hzhou@ovn.org> X-Mailer: git-send-email 2.1.0 In-Reply-To: <1580327768-36501-1-git-send-email-hzhou@ovn.org> References: <1580327768-36501-1-git-send-email-hzhou@ovn.org> Cc: Han Zhou Subject: [ovs-dev] [PATCH ovn v4 10/13] ovn.at: e2e test for OVN interconnection. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" Test with 5 AZs, each with 1 GW, 1 HV, 5 VIFs, 5 LRs, connected to 5 transit switches. Verify traffic through each TS between each pair of AZs. Signed-off-by: Han Zhou --- tests/ovn.at | 147 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 147 insertions(+) diff --git a/tests/ovn.at b/tests/ovn.at index f130441..1ae26fd 100644 --- a/tests/ovn.at +++ b/tests/ovn.at @@ -18022,6 +18022,153 @@ ovn-nbctl --wait=hv --timeout=3 sync AT_CHECK([ovn-trace --ovs lsw0 'inport == "lp1" && eth.type == 0x1234' | grep "dl_type=0x1234 actions="], [0], [ignore]) OVN_CLEANUP([hv1]) +AT_CLEANUP + +AT_SETUP([ovn -- interconnection]) +ovn_init_ic_db +n_az=5 +n_ts=5 +for i in `seq 1 $n_az`; do + ovn_start az$i +done + +net_add n1 + +# 1 HV and 1 GW per AZ +for az in `seq 1 $n_az`; do + sim_add hv$az + as hv$az + ovs-vsctl add-br br-phys + ovn_az_attach az$az n1 br-phys 192.168.$az.1 16 + for p in `seq 1 $n_ts`; do + ovs-vsctl -- add-port br-int vif$p -- \ + set interface vif$p external-ids:iface-id=lsp$az-$p \ + options:tx_pcap=hv$az/vif$p-tx.pcap \ + options:rxq_pcap=hv$az/vif$p-rx.pcap \ + ofport-request=$p + done + + sim_add gw$az + as gw$az + ovs-vsctl add-br br-phys + ovn_az_attach az$az n1 br-phys 192.168.$az.2 16 + ovs-vsctl set open . external-ids:ovn-is-interconn=true +done + +for ts in `seq 1 $n_ts`; do + ovn-ic-nbctl create Transit_Switch name=ts$ts +done + +for az in `seq 1 $n_az`; do + ovn_as az$az + + # Each AZ has n_ts LSPi->LSi->LRi connecting to each TSi + for i in `seq 1 $n_ts`; do + lsp_mac=00:00:00:0$az:0$i:00 + lrp_ls_mac=00:00:00:0$az:0$i:01 + lrp_ts_mac=00:00:00:0$az:0$i:02 + lsp_ip=10.$az.$i.123 + lrp_ls_ip=10.$az.$i.1 + lrp_ts_ip=169.254.$i.$az + + ovn-nbctl ls-add ls$az-$i + ovn-nbctl lsp-add ls$az-$i lsp$az-$i + ovn-nbctl lsp-set-addresses lsp$az-$i "$lsp_mac $lsp_ip" + + ovn-nbctl lr-add lr$az-$i + + ovn-nbctl lrp-add lr$az-$i lrp-lr$az-$i-ls$az-$i $lrp_ls_mac $lrp_ls_ip/24 + ovn-nbctl lsp-add ls$az-$i lsp-ls$az-$i-lr$az-$i + ovn-nbctl lsp-set-addresses lsp-ls$az-$i-lr$az-$i router + ovn-nbctl lsp-set-type lsp-ls$az-$i-lr$az-$i router + ovn-nbctl lsp-set-options lsp-ls$az-$i-lr$az-$i router-port=lrp-lr$az-$i-ls$az-$i + + ovn-nbctl lrp-add lr$az-$i lrp-lr$az-$i-ts$i $lrp_ts_mac $lrp_ts_ip/24 + ovn-nbctl lsp-add ts$i lsp-ts$i-lr$az-$i + ovn-nbctl lsp-set-addresses lsp-ts$i-lr$az-$i router + ovn-nbctl lsp-set-type lsp-ts$i-lr$az-$i router + ovn-nbctl lsp-set-options lsp-ts$i-lr$az-$i router-port=lrp-lr$az-$i-ts$i + ovn-nbctl lrp-set-gateway-chassis lrp-lr$az-$i-ts$i gw$az + + for remote in `seq 1 $n_az`; do + if test $az = $remote; then + continue + fi + ovn-nbctl lr-route-add lr$az-$i 10.$remote.$i.0/24 169.254.$i.$remote + done + done +done + +# Pre-populate the hypervisors' ARP tables so that we don't lose any +# packets for ARP resolution (native tunneling doesn't queue packets +# for ARP resolution). +OVN_POPULATE_ARP + +for i in `seq 1 $n_az`; do + AT_CHECK([ovn_as az$i ovn-nbctl --timeout=3 --wait=sb sync], [0], [ignore]) +done + +# Allow some time for ovn-northd and ovn-controller to catch up. +# XXX This should be more systematic. +sleep 2 + +# Send packets between AZs on each TS +for s_az in `seq 1 $n_az`; do + for d_az in `seq 1 $n_az`; do + if test $s_az = $d_az; then + continue + fi + + for i in `seq 1 $n_ts`; do + lsp_smac=00:00:00:0${s_az}:0$i:00 + lsp_dmac=00:00:00:0${d_az}:0$i:00 + lrp_ls_smac=00:00:00:0${s_az}:0$i:01 + lrp_ls_dmac=00:00:00:0${d_az}:0$i:01 + lsp_sip=10.${s_az}.$i.123 + lsp_dip=10.${d_az}.$i.123 + + packet="inport==\"lsp${s_az}-$i\" && eth.src==$lsp_smac && eth.dst==$lrp_ls_smac && + ip4 && ip.ttl==64 && ip4.src==$lsp_sip && ip4.dst==$lsp_dip && + udp && udp.src==53 && udp.dst==4369" + AT_CHECK([as hv${s_az} ovs-appctl -t ovn-controller inject-pkt "$packet"]) + + # Packet to Expect + # The TTL should be decremented by 2. + packet="eth.src==$lrp_ls_dmac && eth.dst==$lsp_dmac && + ip4 && ip.ttl==62 && ip4.src==$lsp_sip && ip4.dst==$lsp_dip && + udp && udp.src==53 && udp.dst==4369" + echo $packet | ovstest test-ovn expr-to-packets >> ${d_az}-$i.expected + done + done +done + +echo "---------INB dump-----" +ovn-ic-nbctl show +echo "---------------------" + +echo "---------ISB dump-----" +ovn-ic-sbctl show +echo "---------------------" +ovn-ic-sbctl list gateway +echo "---------------------" +ovn-ic-sbctl list datapath_binding +echo "---------------------" +ovn-ic-sbctl list port_binding +echo "---------------------" + +for az in `seq 1 $n_az`; do + for i in `seq 1 $n_ts`; do + OVN_CHECK_PACKETS([hv$az/vif$i-tx.pcap], [$az-$i.expected]) + done +done + +for az in `seq 1 $n_az`; do + OVN_CLEANUP_SBOX([hv$az]) + OVN_CLEANUP_SBOX([gw$az]) + OVN_CLEANUP_AZ([az$az]) +done + +OVN_CLEANUP_IC AT_CLEANUP From patchwork Wed Jan 29 19:56:06 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Han Zhou X-Patchwork-Id: 1231081 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.137; helo=fraxinus.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=ovn.org Received: from fraxinus.osuosl.org (smtp4.osuosl.org [140.211.166.137]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 487Dlc4mbRz9sNT for ; Thu, 30 Jan 2020 06:57:00 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by fraxinus.osuosl.org (Postfix) with ESMTP id 2545786193; Wed, 29 Jan 2020 19:56:59 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from fraxinus.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id T4gO27KpXVIJ; Wed, 29 Jan 2020 19:56:57 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by fraxinus.osuosl.org (Postfix) with ESMTP id BC53186054; Wed, 29 Jan 2020 19:56:47 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 91A13C1D8B; Wed, 29 Jan 2020 19:56:47 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from fraxinus.osuosl.org (smtp4.osuosl.org [140.211.166.137]) by lists.linuxfoundation.org (Postfix) with ESMTP id 17861C0171 for ; Wed, 29 Jan 2020 19:56:46 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by fraxinus.osuosl.org (Postfix) with ESMTP id 1268286132 for ; Wed, 29 Jan 2020 19:56:46 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from fraxinus.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id aVLZVcmIxFM5 for ; Wed, 29 Jan 2020 19:56:44 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from relay1-d.mail.gandi.net (relay1-d.mail.gandi.net [217.70.183.193]) by fraxinus.osuosl.org (Postfix) with ESMTPS id E8C84860C5 for ; Wed, 29 Jan 2020 19:56:39 +0000 (UTC) X-Originating-IP: 216.113.160.77 Received: from localhost.localdomain.localdomain (unknown [216.113.160.77]) (Authenticated sender: hzhou@ovn.org) by relay1-d.mail.gandi.net (Postfix) with ESMTPSA id C123C240007; Wed, 29 Jan 2020 19:56:37 +0000 (UTC) From: Han Zhou To: dev@openvswitch.org Date: Wed, 29 Jan 2020 11:56:06 -0800 Message-Id: <1580327768-36501-12-git-send-email-hzhou@ovn.org> X-Mailer: git-send-email 2.1.0 In-Reply-To: <1580327768-36501-1-git-send-email-hzhou@ovn.org> References: <1580327768-36501-1-git-send-email-hzhou@ovn.org> Cc: Han Zhou Subject: [ovs-dev] [PATCH ovn v4 11/13] ovn-ctl: Refactor to reduce redundant code. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" This patch helps reducing redundant code in next patch for adding support for interconnection related DBs and daemon. Signed-off-by: Han Zhou --- utilities/ovn-ctl | 61 +++++++++++++++++++++++++++++++------------------------ 1 file changed, 35 insertions(+), 26 deletions(-) diff --git a/utilities/ovn-ctl b/utilities/ovn-ctl index 576a983..2e4e773 100755 --- a/utilities/ovn-ctl +++ b/utilities/ovn-ctl @@ -42,16 +42,18 @@ pidfile_is_running () { test -e "$pidfile" && [ -s "$pidfile" ] && pid=`cat "$pidfile"` && pid_exists "$pid" } >/dev/null 2>&1 -stop_nb_ovsdb() { - if pidfile_is_running $DB_NB_PID; then - ovn-appctl -t $OVN_RUNDIR/ovnnb_db.ctl exit +stop_xx_ovsdb() { + if pidfile_is_running $1; then + ovn-appctl -t $OVN_RUNDIR/$2 exit fi } +stop_nb_ovsdb() { + stop_xx_ovsdb $DB_NB_PID ovnnb_db.ctl +} + stop_sb_ovsdb() { - if pidfile_is_running $DB_SB_PID; then - ovn-appctl -t $OVN_RUNDIR/ovnsb_db.ctl exit - fi + stop_xx_ovsdb $DB_SB_PID ovnsb_db.ctl } stop_ovsdb () { @@ -59,42 +61,49 @@ stop_ovsdb () { stop_sb_ovsdb } -demote_ovnnb() { - if test ! -z "$DB_NB_SYNC_FROM_ADDR"; then - echo "$DB_NB_SYNC_FROM_PROTO:$DB_NB_SYNC_FROM_ADDR:$DB_NB_SYNC_FROM_PORT" > $ovnnb_active_conf_file +demote_xx_ovsdb () { + local sync_from_addr=$1 + local sync_from_proto=$2 + local sync_from_port=$3 + local active_conf_file=$4 + local ctl_file=$5 + + if test ! -z "$sync_from_addr"; then + echo "$sync_from_proto:$sync_from_addr:$sync_from_port" > $active_conf_file fi - if test -e $ovnnb_active_conf_file; then - ovn-appctl -t $OVN_RUNDIR/ovnnb_db.ctl ovsdb-server/set-active-ovsdb-server `cat $ovnnb_active_conf_file` - ovn-appctl -t $OVN_RUNDIR/ovnnb_db.ctl ovsdb-server/connect-active-ovsdb-server + if test -e $active_conf_file; then + ovn-appctl -t $OVN_RUNDIR/$ctl_file ovsdb-server/set-active-ovsdb-server `cat $active_conf_file` + ovn-appctl -t $OVN_RUNDIR/$ctl_file ovsdb-server/connect-active-ovsdb-server else echo >&2 "$0: active server details not set" exit 1 fi } +demote_ovnnb() { + demote_xx_ovsdb $DB_NB_SYNC_FROM_ADDR $DB_NB_SYNC_FROM_PROTO \ + $DB_NB_SYNC_FROM_PORT $ovnnb_active_conf_file ovnnb_db.ctl +} + demote_ovnsb() { - if test ! -z "$DB_SB_SYNC_FROM_ADDR"; then - echo "$DB_SB_SYNC_FROM_PROTO:$DB_SB_SYNC_FROM_ADDR:$DB_SB_SYNC_FROM_PORT" > $ovnsb_active_conf_file - fi + demote_xx_ovsdb $DB_SB_SYNC_FROM_ADDR $DB_SB_SYNC_FROM_PROTO \ + $DB_SB_SYNC_FROM_PORT $ovnsb_active_conf_file ovnsb_db.ctl +} - if test -e $ovnsb_active_conf_file; then - ovn-appctl -t $OVN_RUNDIR/ovnsb_db.ctl ovsdb-server/set-active-ovsdb-server `cat $ovnsb_active_conf_file` - ovn-appctl -t $OVN_RUNDIR/ovnsb_db.ctl ovsdb-server/connect-active-ovsdb-server - else - echo >&2 "$0: active server details not set" - exit 1 - fi +promote_xx_ovsdb() { + local active_conf_file=$1 + local ctl_file=$2 + rm -f $active_conf_file + ovn-appctl -t $OVN_RUNDIR/$2 ovsdb-server/disconnect-active-ovsdb-server } promote_ovnnb() { - rm -f $ovnnb_active_conf_file - ovn-appctl -t $OVN_RUNDIR/ovnnb_db.ctl ovsdb-server/disconnect-active-ovsdb-server + promote_xx_ovsdb $ovnnb_active_conf_file ovnnb_db.ctl } promote_ovnsb() { - rm -f $ovnsb_active_conf_file - ovn-appctl -t $OVN_RUNDIR/ovnsb_db.ctl ovsdb-server/disconnect-active-ovsdb-server + promote_xx_ovsdb $ovnsb_active_conf_file ovnsb_db.ctl } start_ovsdb__() { From patchwork Wed Jan 29 19:56:07 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Han Zhou X-Patchwork-Id: 1231089 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.136; helo=silver.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=ovn.org Received: from silver.osuosl.org (smtp3.osuosl.org [140.211.166.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 487Dnd5R2Kz9sNT for ; Thu, 30 Jan 2020 06:58:45 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by silver.osuosl.org (Postfix) with ESMTP id 3D38522268; Wed, 29 Jan 2020 19:58:44 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from silver.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id GdgX1oNurPsB; Wed, 29 Jan 2020 19:58:30 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by silver.osuosl.org (Postfix) with ESMTP id 06B602278E; Wed, 29 Jan 2020 19:57:25 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id E9F5FC0881; Wed, 29 Jan 2020 19:57:24 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by lists.linuxfoundation.org (Postfix) with ESMTP id BBE98C0171 for ; Wed, 29 Jan 2020 19:57:23 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by whitealder.osuosl.org (Postfix) with ESMTP id A602887749 for ; Wed, 29 Jan 2020 19:57:23 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from whitealder.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id C6VxXxyuLTCG for ; Wed, 29 Jan 2020 19:57:09 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from relay1-d.mail.gandi.net (relay1-d.mail.gandi.net [217.70.183.193]) by whitealder.osuosl.org (Postfix) with ESMTPS id 438678773A for ; Wed, 29 Jan 2020 19:56:41 +0000 (UTC) X-Originating-IP: 216.113.160.77 Received: from localhost.localdomain.localdomain (unknown [216.113.160.77]) (Authenticated sender: hzhou@ovn.org) by relay1-d.mail.gandi.net (Postfix) with ESMTPSA id 12DDA240002; Wed, 29 Jan 2020 19:56:38 +0000 (UTC) From: Han Zhou To: dev@openvswitch.org Date: Wed, 29 Jan 2020 11:56:07 -0800 Message-Id: <1580327768-36501-13-git-send-email-hzhou@ovn.org> X-Mailer: git-send-email 2.1.0 In-Reply-To: <1580327768-36501-1-git-send-email-hzhou@ovn.org> References: <1580327768-36501-1-git-send-email-hzhou@ovn.org> Cc: Han Zhou Subject: [ovs-dev] [PATCH ovn v4 12/13] ovn-ctl: Support commands for interconnection. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" Add support for managing IC-NB and IC-SB DBs, and ovn-ic daemon. Signed-off-by: Han Zhou --- utilities/ovn-ctl | 364 +++++++++++++++++++++++++++++++++++++++++++++++- utilities/ovn-ctl.8.xml | 91 ++++++++++++ 2 files changed, 453 insertions(+), 2 deletions(-) diff --git a/utilities/ovn-ctl b/utilities/ovn-ctl index 2e4e773..c7cb42b 100755 --- a/utilities/ovn-ctl +++ b/utilities/ovn-ctl @@ -33,6 +33,9 @@ done ovnnb_active_conf_file="$ovn_etcdir/ovnnb-active.conf" ovnsb_active_conf_file="$ovn_etcdir/ovnsb-active.conf" ovn_northd_db_conf_file="$ovn_etcdir/ovn-northd-db-params.conf" +ic_nb_active_conf_file="$ovn_etcdir/ic-nb-active.conf" +ic_sb_active_conf_file="$ovn_etcdir/ic-sb-active.conf" +ovn_ic_db_conf_file="$ovn_etcdir/ovn-ic-db-params.conf" ## ----- ## ## start ## ## ----- ## @@ -61,6 +64,19 @@ stop_ovsdb () { stop_sb_ovsdb } +stop_ic_nb_ovsdb() { + stop_xx_ovsdb $DB_IC_NB_PID ovn_ic_nb_db.ctl +} + +stop_ic_sb_ovsdb() { + stop_xx_ovsdb $DB_IC_SB_PID ovn_ic_sb_db.ctl +} + +stop_ic_ovsdb () { + stop_ic_nb_ovsdb + stop_ic_sb_ovsdb +} + demote_xx_ovsdb () { local sync_from_addr=$1 local sync_from_proto=$2 @@ -91,6 +107,16 @@ demote_ovnsb() { $DB_SB_SYNC_FROM_PORT $ovnsb_active_conf_file ovnsb_db.ctl } +demote_ic_nb() { + demote_xx_ovsdb $DB_IC_NB_SYNC_FROM_ADDR $DB_IC_NB_SYNC_FROM_PROTO \ + $DB_IC_NB_SYNC_FROM_PORT $ic_nb_active_conf_file ovn_ic_nb_db.ctl +} + +demote_ic_sb() { + demote_xx_ovsdb $DB_IC_SB_SYNC_FROM_ADDR $DB_IC_SB_SYNC_FROM_PROTO \ + $DB_IC_SB_SYNC_FROM_PORT $ic_sb_active_conf_file ovn_ic_sb_db.ctl +} + promote_xx_ovsdb() { local active_conf_file=$1 local ctl_file=$2 @@ -106,6 +132,14 @@ promote_ovnsb() { promote_xx_ovsdb $ovnsb_active_conf_file ovnsb_db.ctl } +promote_ic_nb() { + promote_xx_ovsdb $ic_nb_active_conf_file ovn_ic_nb_db.ctl +} + +promote_ic_sb() { + promote_xx_ovsdb $ic_sb_active_conf_file ovn_ic_sb_db.ctl +} + start_ovsdb__() { local DB=$1 db=$2 schema_name=$3 table_name=$4 local db_pid_file @@ -255,7 +289,7 @@ $cluster_remote_port # Initialize the database if it's running standalone, # active-passive, or is the first server in a cluster. if test -z "$cluster_remote_addr"; then - ovn-${db}ctl init + $(echo ovn-${db}ctl | tr _ -) init fi if test $mode = cluster; then @@ -284,6 +318,19 @@ start_ovsdb () { start_sb_ovsdb } +start_ic_nb_ovsdb() { + start_ovsdb__ IC_NB ic_nb OVN_IC_Northbound IC_NB_Global +} + +start_ic_sb_ovsdb() { + start_ovsdb__ IC_SB ic_sb OVN_IC_Southbound IC_SB_Global +} + +start_ic_ovsdb () { + start_ic_nb_ovsdb + start_ic_sb_ovsdb +} + sync_status() { ovn-appctl -t $OVN_RUNDIR/ovn${1}_db.ctl ovsdb-server/sync-status | awk '{if(NR==1) print $2}' } @@ -318,6 +365,36 @@ status_ovsdb () { fi } +status_ic_nb() { + if ! pidfile_is_running $DB_IC_NB_PID; then + echo "not-running" + else + echo "running/$(sync_status ic_nb)" + fi +} + +status_ic_sb() { + if ! pidfile_is_running $DB_IC_SB_PID; then + echo "not-running" + else + echo "running/$(sync_status ic_sb)" + fi +} + +status_ic_ovsdb () { + if ! pidfile_is_running $DB_IC_NB_PID; then + log_success_msg "OVN IC-Northbound DB is not running" + else + log_success_msg "OVN IC-Northbound DB is running" + fi + + if ! pidfile_is_running $DB_IC_SB_PID; then + log_success_msg "OVN IC-Southbound DB is not running" + else + log_success_msg "OVN IC-Southbound DB is running" + fi +} + run_nb_ovsdb() { DB_NB_DETACH=no start_nb_ovsdb @@ -328,6 +405,16 @@ run_sb_ovsdb() { start_sb_ovsdb } +run_ic_nb_ovsdb() { + DB_IC_NB_DETACH=no + start_ic_nb_ovsdb +} + +run_ic_sb_ovsdb() { + DB_IC_SB_DETACH=no + start_ic_sb_ovsdb +} + start_northd () { if [ ! -e $ovn_northd_db_conf_file ]; then if test X"$OVN_MANAGE_OVSDB" = Xyes; then @@ -373,6 +460,41 @@ start_northd () { fi } +start_ic () { + if [ ! -e $ovn_ic_db_conf_file ]; then + ovn_ic_params="--ovnnb-db=$OVN_NORTHD_NB_DB \ + --ovnsb-db=$OVN_NORTHD_SB_DB \ + --ic-nb-db=$OVN_IC_NB_DB \ + --ic-sb-db=$OVN_IC_SB_DB" + else + ovn_ic_params="`cat $ovn_ic_db_conf_file`" + fi + + if daemon_is_running ovn-ic; then + log_success_msg "ovn-ic is already running" + else + set ovn-ic + if test X"$OVN_IC_LOGFILE" != X; then + set "$@" --log-file=$OVN_IC_LOGFILE + fi + if test X"$OVN_IC_SSL_KEY" != X; then + set "$@" --private-key=$OVN_IC_SSL_KEY + fi + if test X"$OVN_IC_SSL_CERT" != X; then + set "$@" --certificate=$OVN_IC_SSL_CERT + fi + if test X"$OVN_IC_SSL_CA_CERT" != X; then + set "$@" --ca-cert=$OVN_IC_SSL_CA_CERT + fi + + [ "$OVN_USER" != "" ] && set "$@" --user "$OVN_USER" + + set "$@" $OVN_IC_LOG $ovn_ic_params + + OVS_RUNDIR=${OVS_RUNDIR} start_ovn_daemon "$OVN_IC_PRIORITY" "$OVN_IC_WRAPPER" "$@" + fi +} + start_controller () { set ovn-controller "unix:$DB_SOCK" set "$@" $OVN_CONTROLLER_LOG @@ -435,6 +557,10 @@ stop_northd () { fi } +stop_ic () { + OVS_RUNDIR=${OVS_RUNDIR} stop_ovn_daemon ovn-ic +} + stop_controller () { OVS_RUNDIR=${OVS_RUNDIR} stop_ovn_daemon ovn-controller "$@" } @@ -452,6 +578,11 @@ restart_northd () { start_northd } +restart_ic () { + stop_ic + start_ic +} + restart_controller () { stop_controller --restart start_controller @@ -477,6 +608,21 @@ restart_sb_ovsdb () { start_sb_ovsdb } +restart_ic_ovsdb () { + stop_ic_ovsdb + start_ic_ovsdb +} + +restart_ic_nb_ovsdb () { + stop_ic_nb_ovsdb + start_ic_nb_ovsdb +} + +restart_ic_sb_ovsdb () { + stop_ic_sb_ovsdb + start_ic_sb_ovsdb +} + ## ---- ## ## main ## ## ---- ## @@ -507,14 +653,38 @@ set_defaults () { DB_SB_SYNC_FROM_ADDR= DB_SB_SYNC_FROM_PORT=6642 + DB_IC_NB_SOCK=$OVN_RUNDIR/ovn_ic_nb_db.sock + DB_IC_NB_PID=$OVN_RUNDIR/ovn_ic_nb_db.pid + DB_IC_NB_CTRL_SOCK=$OVN_RUNDIR/ovn_ic_nb_db.ctl + DB_IC_NB_FILE=$ovn_dbdir/ovn_ic_nb_db.db + DB_IC_NB_ADDR=0.0.0.0 + DB_IC_NB_PORT=6645 + DB_IC_NB_SYNC_FROM_PROTO=tcp + DB_IC_NB_SYNC_FROM_ADDR= + DB_IC_NB_SYNC_FROM_PORT=6645 + + DB_IC_SB_SOCK=$OVN_RUNDIR/ovn_ic_sb_db.sock + DB_IC_SB_PID=$OVN_RUNDIR/ovn_ic_sb_db.pid + DB_IC_SB_CTRL_SOCK=$OVN_RUNDIR/ovn_ic_sb_db.ctl + DB_IC_SB_FILE=$ovn_dbdir/ovn_ic_sb_db.db + DB_IC_SB_ADDR=0.0.0.0 + DB_IC_SB_PORT=6646 + DB_IC_SB_SYNC_FROM_PROTO=tcp + DB_IC_SB_SYNC_FROM_ADDR= + DB_IC_SB_SYNC_FROM_PORT=6646 + DB_NB_SCHEMA=$ovn_datadir/ovn-nb.ovsschema DB_SB_SCHEMA=$ovn_datadir/ovn-sb.ovsschema + DB_IC_NB_SCHEMA=$ovn_datadir/ovn-ic-nb.ovsschema + DB_IC_SB_SCHEMA=$ovn_datadir/ovn-ic-sb.ovsschema DB_SOCK=$OVS_RUNDIR/db.sock DB_CONF_FILE=$dbdir/conf.db OVN_NORTHD_PRIORITY=-10 OVN_NORTHD_WRAPPER= + OVN_IC_PRIORITY=-10 + OVN_IC_WRAPPER= OVN_CONTROLLER_PRIORITY=-10 OVN_CONTROLLER_WRAPPER= @@ -523,10 +693,16 @@ set_defaults () { OVN_CONTROLLER_LOG="-vconsole:emer -vsyslog:err -vfile:info" OVN_NORTHD_LOG="-vconsole:emer -vsyslog:err -vfile:info" OVN_NORTHD_LOGFILE="" + OVN_IC_LOG="-vconsole:emer -vsyslog:err -vfile:info" + OVN_IC_LOGFILE="" OVN_NB_LOG="-vconsole:off -vfile:info" OVN_SB_LOG="-vconsole:off -vfile:info" OVN_NB_LOGFILE="$ovn_logdir/ovsdb-server-nb.log" OVN_SB_LOGFILE="$ovn_logdir/ovsdb-server-sb.log" + OVN_IC_NB_LOG="-vconsole:off -vfile:info" + OVN_IC_SB_LOG="-vconsole:off -vfile:info" + OVN_IC_NB_LOGFILE="$ovn_logdir/ovsdb-server-ic-nb.log" + OVN_IC_SB_LOGFILE="$ovn_logdir/ovsdb-server-ic-sb.log" OVN_CONTROLLER_SSL_KEY="" OVN_CONTROLLER_SSL_CERT="" @@ -537,14 +713,24 @@ set_defaults () { OVN_NORTHD_SSL_CERT="" OVN_NORTHD_SSL_CA_CERT="" + OVN_IC_SSL_KEY="" + OVN_IC_SSL_CERT="" + OVN_IC_SSL_CA_CERT="" + DB_SB_CREATE_INSECURE_REMOTE="no" DB_NB_CREATE_INSECURE_REMOTE="no" + DB_IC_SB_CREATE_INSECURE_REMOTE="no" + DB_IC_NB_CREATE_INSECURE_REMOTE="no" + MONITOR="yes" DB_NB_DETACH="yes" DB_SB_DETACH="yes" + DB_IC_NB_DETACH="yes" + DB_IC_SB_DETACH="yes" + DB_NB_CLUSTER_LOCAL_ADDR="" DB_NB_CLUSTER_LOCAL_PROTO="tcp" DB_NB_CLUSTER_LOCAL_PORT=6643 @@ -559,11 +745,30 @@ set_defaults () { DB_SB_CLUSTER_REMOTE_PROTO="tcp" DB_SB_CLUSTER_REMOTE_PORT=6644 + DB_IC_NB_CLUSTER_LOCAL_ADDR="" + DB_IC_NB_CLUSTER_LOCAL_PROTO="tcp" + DB_IC_NB_CLUSTER_LOCAL_PORT=6647 + DB_IC_NB_CLUSTER_REMOTE_ADDR="" + DB_IC_NB_CLUSTER_REMOTE_PROTO="tcp" + DB_IC_NB_CLUSTER_REMOTE_PORT=6647 + + DB_IC_SB_CLUSTER_LOCAL_ADDR="" + DB_IC_SB_CLUSTER_LOCAL_PROTO="tcp" + DB_IC_SB_CLUSTER_LOCAL_PORT=6648 + DB_IC_SB_CLUSTER_REMOTE_ADDR="" + DB_IC_SB_CLUSTER_REMOTE_PROTO="tcp" + DB_IC_SB_CLUSTER_REMOTE_PORT=6648 + OVN_NORTHD_NB_DB="unix:$DB_NB_SOCK" OVN_NORTHD_SB_DB="unix:$DB_SB_SOCK" DB_NB_USE_REMOTE_IN_DB="yes" DB_SB_USE_REMOTE_IN_DB="yes" + OVN_IC_NB_DB="unix:$DB_IC_NB_SOCK" + OVN_IC_SB_DB="unix:$DB_IC_SB_SOCK" + DB_IC_NB_USE_REMOTE_IN_DB="yes" + DB_IC_SB_USE_REMOTE_IN_DB="yes" + OVN_NB_DB_SSL_KEY="" OVN_NB_DB_SSL_CERT="" OVN_NB_DB_SSL_CA_CERT="" @@ -572,6 +777,14 @@ set_defaults () { OVN_SB_DB_SSL_CERT="" OVN_SB_DB_SSL_CA_CERT="" + OVN_IC_NB_DB_SSL_KEY="" + OVN_IC_NB_DB_SSL_CERT="" + OVN_IC_NB_DB_SSL_CA_CERT="" + + OVN_IC_SB_DB_SSL_KEY="" + OVN_IC_SB_DB_SSL_CERT="" + OVN_IC_SB_DB_SSL_CA_CERT="" + } set_option () { @@ -601,22 +814,36 @@ Commands: start_ovsdb start ovn related ovsdb-server processes start_nb_ovsdb start ovn northbound db ovsdb-server process start_sb_ovsdb start ovn southbound db ovsdb-server process + start_ic start ovn-ic + start_ic_ovsdb start ovn interconnection ovsdb-server processes + start_ic_nb_ovsdb start ovn ic-northbound db ovsdb-server process + start_ic_sb_ovsdb start ovn ic-southbound db ovsdb-server process start_controller start ovn-controller start_controller_vtep start ovn-controller-vtep stop_northd stop ovn-northd stop_ovsdb stop ovn related ovsdb-server processes stop_nb_ovsdb stop ovn northbound db ovsdb-server process stop_sb_ovsdb stop ovn southbound db ovsdb-server process + stop_ic stop ovn-ic + stop_ic_ovsdb stop ovn interconnection ovsdb-server processes + stop_ic_nb_ovsdb stop ovn ic-northbound db ovsdb-server process + stop_ic_sb_ovsdb stop ovn ic-southbound db ovsdb-server process stop_controller stop ovn-controller stop_controller_vtep stop ovn-controller-vtep restart_northd restart ovn-northd restart_ovsdb restart ovn related ovsdb-server processes restart_nb_ovsdb restart ovn northbound db ovsdb-server process restart_sb_ovsdb restart ovn southbound db ovsdb-server process + restart_ic restart ovn-ic + restart_ic_ovsdb restart ovn interconnection ovsdb-server processes + restart_ic_nb_ovsdb restart ovn ic-northbound db ovsdb-server process + restart_ic_sb_ovsdb restart ovn ic-southbound db ovsdb-server process restart_controller restart ovn-controller restart_controller_vtep restart ovn-controller-vtep status_northd status ovs-northd status_ovsdb status related ovsdb-server processes + status_ic status ovn-ic + status_ic_ovsdb status ovn interconnection ovsdb-server processes status_controller status ovn-controller status_controller_vtep status ovn-controller-vtep promote_ovnnb promote ovn northbound db backup server to active @@ -625,6 +852,12 @@ Commands: demote_ovnsb demote ovn southbound db active server to backup run_nb_ovsdb run ovn northbound db ovsdb-server process run_sb_ovsdb run ovn southbound db ovsdb-server process + promote_ic_nb promote ovn ic-northbound db backup server to active + promote_ic_sb promote ovn ic-southbound db backup server to active + demote_ic_nb demote ovn ic-northbound db active server to backup + demote_ic_sb demote ovn ic-southbound db active server to backup + run_ic_nb_ovsdb run ovn ic-northbound db ovsdb-server process + run_ic_sb_ovsdb run ovn ic-southbound db ovsdb-server process Options: --ovn-northd-priority=NICE set ovn-northd's niceness (default: $OVN_NORTHD_PRIORITY) @@ -644,7 +877,7 @@ Options: --ovn-northd-ssl-key=KEY OVN Northd SSL private key file --ovn-northd-ssl-cert=CERT OVN Northd SSL certificate file --ovn-northd-ssl-ca-cert=CERT OVN Northd SSL CA certificate file - --ovn-manage-ovsdb=yes|no Whether or not the OVN databases should be + --ovn-manage-ovsdb=yes|no Whether or not the OVN NB/SB databases should be automatically started and stopped along with ovn-northd. The default is "yes". If this is set to "no", the "start_ovsdb" and @@ -655,6 +888,19 @@ Options: --ovn-northd-logfile=STRING ovn northd process log file (default: $OVN_NORTHD_LOGFILE) --ovn-nb-log=STRING ovn NB ovsdb-server processes logging params (default: $OVN_NB_LOG) --ovn-sb-log=STRING ovn SB ovsdb-server processes logging params (default: $OVN_SB_LOG) + --ovn-ic-priority=NICE set ovn-ic's niceness (default: $OVN_IC_PRIORITY) + --ovn-ic-wrapper=WRAPPER run with a wrapper like valgrind for debugging + --ovn-ic-ssl-key=KEY OVN IC SSL private key file + --ovn-ic-ssl-cert=CERT OVN IC SSL certificate file + --ovn-ic-ssl-ca-cert=CERT OVN IC SSL CA certificate file + --ovn-ic-log=STRING ovn-ic process logging params (default: $OVN_IC_LOG) + --ovn-ic-logfile=STRING ovn-ic process log file (default: $OVN_IC_LOGFILE) + --ovn-ic-nb-db-ssl-key=KEY OVN IC Northbound DB SSL private key file + --ovn-ic-nb-db-ssl-cert=CERT OVN IC Northbound DB SSL certificate file + --ovn-ic-nb-db-ssl-ca-cert=CERT OVN IC Northbound DB SSL CA certificate file + --ovn-ic-sb-db-ssl-key=KEY OVN IC Southbound DB SSL private key file + --ovn-ic-sb-db-ssl-cert=CERT OVN IC Southbound DB SSL certificate file + --ovn-ic-sb-db-ssl-ca-cert=CERT OVN IC Southbound DB SSL CA certificate file --ovn-user="user[:group]" pass the --user flag to the ovn daemons --ovs-user="user[:group]" pass the --user flag to ovs daemons -h, --help display this help message @@ -709,6 +955,54 @@ File location options: --ovn-northd-sb-db=SB DB address(es) (default: $OVN_NORTHD_SB_DB) --db-nb-use-remote-in-db=yes|no OVN_Northbound db listen on target connection table (default: $DB_NB_USE_REMOTE_IN_DB) --db-sb-use-remote-in-db=yes|no OVN_Southbound db listen on target connection table (default: $DB_SB_USE_REMOTE_IN_DB) + --db-ic-nb-sock=SOCKET OVN_IC_Northbound db socket (default: $DB_IC_NB_SOCK) + --db-ic-sb-scok=SOCKET OVN_IC_Southbound db socket (default: $DB_IC_SB_SOCK) + --db-ic-nb-file=FILE OVN_IC_Northbound db file (default: $DB_IC_NB_FILE) + --db-ic-sb-file=FILE OVN_IC_Southbound db file (default: $DB_IC_SB_FILE) + --db-ic-nb-schema=FILE OVN_IC_Northbound db file (default: $DB_IC_NB_SCHEMA) + --db-ic-sb-schema=FILE OVN_IC_Southbound db file (default: $DB_IC_SB_SCHEMA) + --db-ic-nb-addr=ADDR OVN IC Northbound db ptcp address (default: $DB_IC_NB_ADDR) + --db-ic-nb-port=PORT OVN IC Northbound db ptcp port (default: $DB_IC_NB_PORT) + --db-ic-sb-addr=ADDR OVN IC Southbound db ptcp address (default: $DB_IC_SB_ADDR) + --db-ic-sb-port=PORT OVN IC Southbound db ptcp port (default: $DB_IC_SB_PORT) + --ovn-ic-nb-logfile=FILE OVN IC Northbound log file (default: $OVN_IC_NB_LOGFILE) + --ovn-ic-sb-logfile=FILE OVN IC Southbound log file (default: $OVN_IC_SB_LOGFILE) + --db-ic-nb-sync-from-addr=ADDR OVN IC Northbound active db tcp address (default: $DB_IC_NB_SYNC_FROM_ADDR) + --db-ic-nb-sync-from-port=PORT OVN IC Northbound active db tcp port (default: $DB_IC_NB_SYNC_FROM_PORT) + --db-ic-nb-sync-from-proto=PROTO OVN IC Northbound active db transport (default: $DB_IC_NB_SYNC_FROM_PROTO) + --db-ic-nb-create-insecure-remote=yes|no Create ptcp OVN IC Northbound remote (default: $DB_IC_NB_CREATE_INSECURE_REMOTE) + --db-ic-sb-sync-from-addr=ADDR OVN IC Southbound active db tcp address (default: $DB_IC_SB_SYNC_FROM_ADDR) + --db-ic-sb-sync-from-port=ADDR OVN IC Southbound active db tcp port (default: $DB_IC_SB_SYNC_FROM_PORT) + --db-ic-sb-sync-from-proto=PROTO OVN IC Southbound active db transport (default: $DB_IC_SB_SYNC_FROM_PROTO) + --db-ic-sb-create-insecure-remote=yes|no Create ptcp OVN IC Southbound remote (default: $DB_IC_SB_CREATE_INSECURE_REMOTE) + --db-ic-nb-cluster-local-addr=ADDR OVN_IC_Northbound cluster local address \ + (default: $DB_IC_NB_CLUSTER_LOCAL_ADDR) + --db-ic-nb-cluster-local-port=PORT OVN_IC_Northbound cluster local tcp port \ + (default: $DB_IC_NB_CLUSTER_LOCAL_PORT) + --db-ic-nb-cluster-local-proto=PROTO OVN_IC_Northbound cluster local db transport \ + (default: $DB_IC_NB_CLUSTER_LOCAL_PROTO) + --db-ic-nb-cluster-remote-addr=ADDR OVN_IC_Northbound cluster remote address \ + (default: $DB_IC_NB_CLUSTER_REMOTE_ADDR) + --db-ic-nb-cluster-remote-port=PORT OVN_IC_Northbound cluster remote tcp port \ + (default: $DB_IC_NB_CLUSTER_REMOTE_PORT) + --db-ic-nb-cluster-remote-proto=PROTO OVN_IC_Northbound cluster remote db \ + transport (default: $DB_IC_NB_CLUSTER_REMOTE_PROTO) + --db-ic-sb-cluster-local-addr=ADDR OVN_IC_Southbound cluster local address \ + (default: $DB_IC_SB_CLUSTER_LOCAL_ADDR) + --db-ic-sb-cluster-local-port=PORT OVN_IC_Southbound cluster local tcp port \ + (default: $DB_IC_SB_CLUSTER_LOCAL_PORT) + --db-ic-sb-cluster-local-proto=PROTO OVN_IC_Southbound cluster local db transport \ + (default: $DB_IC_SB_CLUSTER_LOCAL_PROTO) + --db-ic-sb-cluster-remote-addr=ADDR OVN_IC_Southbound cluster remote address \ + (default: $DB_IC_SB_CLUSTER_REMOTE_ADDR) + --db-ic-sb-cluster-remote-port=PORT OVN_IC_Southbound cluster remote tcp port \ + (default: $DB_IC_SB_CLUSTER_REMOTE_PORT) + --db-ic-sb-cluster-remote-proto=PROTO OVN_IC_Southbound cluster remote db \ + transport (default: $DB_IC_SB_CLUSTER_REMOTE_PROTO) + --ovn-ic-nb-db=IC NB DB address(es) (default: $OVN_IC_NB_DB) + --ovn-ic-sb-db=IC SB DB address(es) (default: $OVN_IC_SB_DB) + --db-ic-nb-use-remote-in-db=yes|no OVN_IC_Northbound db listen on target connection table (default: $DB_IC_NB_USE_REMOTE_IN_DB) + --db-ic-sb-use-remote-in-db=yes|no OVN_IC_Southbound db listen on target connection table (default: $DB_IC_SB_USE_REMOTE_IN_DB) Default directories with "configure" option and environment variable override: logs: /usr/local/var/log/ovn (--with-logdir, OVN_LOGDIR) @@ -781,6 +1075,18 @@ case $command in start_controller_vtep) start_controller_vtep ;; + start_ic) + start_ic + ;; + start_ic_ovsdb) + start_ic_ovsdb + ;; + start_ic_nb_ovsdb) + start_ic_nb_ovsdb + ;; + start_ic_sb_ovsdb) + start_ic_sb_ovsdb + ;; stop_northd) stop_northd ;; @@ -799,6 +1105,18 @@ case $command in stop_controller_vtep) stop_controller_vtep ;; + stop_ic) + stop_ic + ;; + stop_ic_ovsdb) + stop_ic_ovsdb + ;; + stop_ic_nb_ovsdb) + stop_ic_nb_ovsdb + ;; + stop_ic_sb_ovsdb) + stop_ic_sb_ovsdb + ;; restart_northd) restart_northd ;; @@ -817,6 +1135,18 @@ case $command in restart_controller_vtep) restart_controller_vtep ;; + restart_ic) + restart_ic + ;; + restart_ic_ovsdb) + restart_ic_ovsdb + ;; + restart_ic_nb_ovsdb) + restart_ic_nb_ovsdb + ;; + restart_ic_sb_ovsdb) + restart_ic_sb_ovsdb + ;; status_northd) daemon_status ovn-northd || exit 1 ;; @@ -829,6 +1159,12 @@ case $command in status_controller_vtep) daemon_status ovn-controller-vtep || exit 1 ;; + status_ic) + daemon_status ovn-ic || exit 1 + ;; + status_ic_ovsdb) + status_ic_ovsdb + ;; promote_ovnnb) promote_ovnnb ;; @@ -847,12 +1183,36 @@ case $command in status_ovnsb) status_ovnsb ;; + promote_ic_nb) + promote_ic_nb + ;; + promote_ic_sb) + promote_ic_sb + ;; + demote_ic_nb) + demote_ic_nb + ;; + demote_ic_sb) + demote_ic_sb + ;; + status_ic_nb) + status_ic_nb + ;; + status_ic_sb) + status_ic_sb + ;; run_nb_ovsdb) run_nb_ovsdb ;; run_sb_ovsdb) run_sb_ovsdb ;; + run_ic_nb_ovsdb) + run_ic_nb_ovsdb + ;; + run_ic_sb_ovsdb) + run_ic_sb_ovsdb + ;; help) usage ;; diff --git a/utilities/ovn-ctl.8.xml b/utilities/ovn-ctl.8.xml index f70bd6f..8167013 100644 --- a/utilities/ovn-ctl.8.xml +++ b/utilities/ovn-ctl.8.xml @@ -16,12 +16,15 @@
start_northd
start_controller
start_controller_vtep
+
start_ic
stop_northd
stop_controller
stop_controller_vtep
+
stop_ic
restart_northd
restart_controller
restart_controller_vtep
+
restart_ic
promote_ovnnb
promote_ovnsb
demote_ovnnb
@@ -37,6 +40,21 @@
restart_ovsdb
run_nb_ovsdb
run_sb_ovsdb
+
promote_ic_nb
+
promote_ic_sb
+
demote_ic_nb
+
demote_ic_sb
+
status_ic_nb
+
status_ic_sb
+
start_ic_ovsdb
+
start_ic_nb_ovsdb
+
start_ic_sb_ovsdb
+
stop_ic_ovsdb
+
stop_ic_nb_ovsdb
+
stop_ic_sb_ovsdb
+
restart_ic_ovsdb
+
run_ic_nb_ovsdb
+
run_ic_sb_ovsdb

Options

@@ -44,6 +62,8 @@

--ovn-northd-wrapper=WRAPPER

--ovn-controller-priority=NICE

--ovn-controller-wrapper=WRAPPER

+

--ovn-ic-priority=NICE

+

--ovn-ic-wrapper=WRAPPER

--ovn-user=USER:GROUP

--ovs-user=USER:GROUP

-h | --help

@@ -56,6 +76,12 @@

--db-sb-schema=FILE

--db-sb-create-insecure-remote=yes|no

--db-nb-create-insecure-remote=yes|no

+

--db-ic-nb-file=FILE

+

--db-ic-sb-file=FILE

+

--db-ic-nb-schema=FILE

+

--db-ic-sb-schema=FILE

+

--db-ic-sb-create-insecure-remote=yes|no

+

--db-ic-nb-create-insecure-remote=yes|no

--ovn-controller-ssl-key=KEY

--ovn-controller-ssl-cert=CERT

--ovn-controller-ssl-ca-cert=CERT

@@ -68,6 +94,12 @@

--db-sb-sync-from-addr=IP ADDRESS

--db-sb-sync-from-port=PORT NUMBER

--db-sb-sync-from-proto=PROTO

+

--db-ic-nb-sync-from-addr=IP ADDRESS

+

--db-ic-nb-sync-from-port=PORT NUMBER

+

--db-ic-nb-sync-from-proto=PROTO

+

--db-ic-sb-sync-from-addr=IP ADDRESS

+

--db-ic-sb-sync-from-port=PORT NUMBER

+

--db-ic-sb-sync-from-proto=PROTO

--ovn-northd-nb-db=PROTO:IP ADDRESS: @@ -80,6 +112,18 @@ PORT..

+

+ + --ovn-ic-nb-db=PROTO:IP ADDRESS: + PORT.. + +

+

+ + --ovn-ic-sb-db=PROTO:IP ADDRESS: + PORT.. + +

Clustering options

--db-nb-cluster-local-addr=IP ADDRESS

--db-nb-cluster-local-port=PORT NUMBER

@@ -93,6 +137,18 @@

--db-sb-cluster-remote-addr=IP ADDRESS

--db-sb-cluster-remote-port=PORT NUMBER

--db-sb-cluster-remote-proto=PROTO (tcp/ssl)

+

--db-ic-nb-cluster-local-addr=IP ADDRESS

+

--db-ic-nb-cluster-local-port=PORT NUMBER

+

--db-ic-nb-cluster-local-proto=PROTO (tcp/ssl)

+

--db-ic-nb-cluster-remote-addr=IP ADDRESS

+

--db-ic-nb-cluster-remote-port=PORT NUMBER

+

--db-ic-nb-cluster-remote-proto=PROTO (tcp/ssl)

+

--db-ic-sb-cluster-local-addr=IP ADDRESS

+

--db-ic-sb-cluster-local-port=PORT NUMBER

+

--db-ic-sb-cluster-local-proto=PROTO (tcp/ssl)

+

--db-ic-sb-cluster-remote-addr=IP ADDRESS

+

--db-ic-sb-cluster-remote-port=PORT NUMBER

+

--db-ic-sb-cluster-remote-proto=PROTO (tcp/ssl)

Configuration files

Following are the optional configuration files. If present, it should be located in the etc dir

@@ -119,6 +175,27 @@

--ovnnb-db=tcp:x.x.x.x:6641 --ovnsb-db=tcp:x.x.x.x:6642

+

ic-nb-active.conf

+

+ If present, this file should hold the url to connect to the active + Interconnection Northbound DB server +

+

tcp:x.x.x.x:6645

+ +

ic-sb-active.conf

+

+ If present, this file should hold the url to connect to the active + Interconnection Southbound DB server +

+

tcp:x.x.x.x:6646

+ +

ovn-ic-db-params.conf

+

+ If present, this file should hold the database url parameters to be passed + to ovn-ic. +

+

--ic-nb-db=tcp:x.x.x.x:6645 --ic-sb-db=tcp:x.x.x.x:6646

+

Running OVN db servers without detaching

# ovn-ctl run_nb_ovsdb

@@ -134,6 +211,20 @@ This command will be useful for starting the OVN sb ovsdb-server in a container.

+

# ovn-ctl run_ic_nb_ovsdb

+

+ This command runs the OVN IC-NB ovsdb-server without passing the + detach option, making it to block until ovsdb-server exits. + This command will be useful for starting the OVN IC-NB ovsdb-server in a + container. +

+

# ovn-ctl run_ic_sb_ovsdb

+

+ This command runs the OVN IC-SB ovsdb-server without passing the + detach option, making it to block until ovsdb-server exits. + This command will be useful for starting the OVN IC-SB ovsdb-server in a + container. +

Example Usage

Run ovn-controller on a host already running OVS

From patchwork Wed Jan 29 19:56:08 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Han Zhou X-Patchwork-Id: 1231088 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.133; helo=hemlock.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=ovn.org Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 487Dmb5cGsz9sNT for ; Thu, 30 Jan 2020 06:57:51 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id 40B5A882EC; Wed, 29 Jan 2020 19:57:50 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id FUJO2J6R0wVU; Wed, 29 Jan 2020 19:57:42 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by hemlock.osuosl.org (Postfix) with ESMTP id 54875883D2; Wed, 29 Jan 2020 19:56:59 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 3D13EC1D84; Wed, 29 Jan 2020 19:56:59 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by lists.linuxfoundation.org (Postfix) with ESMTP id C2608C0171 for ; Wed, 29 Jan 2020 19:56:55 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id A867987082 for ; Wed, 29 Jan 2020 19:56:55 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id UtIifNmesnu8 for ; Wed, 29 Jan 2020 19:56:51 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from relay1-d.mail.gandi.net (relay1-d.mail.gandi.net [217.70.183.193]) by hemlock.osuosl.org (Postfix) with ESMTPS id 9251F88266 for ; Wed, 29 Jan 2020 19:56:42 +0000 (UTC) X-Originating-IP: 216.113.160.77 Received: from localhost.localdomain.localdomain (unknown [216.113.160.77]) (Authenticated sender: hzhou@ovn.org) by relay1-d.mail.gandi.net (Postfix) with ESMTPSA id 637FF240004; Wed, 29 Jan 2020 19:56:40 +0000 (UTC) From: Han Zhou To: dev@openvswitch.org Date: Wed, 29 Jan 2020 11:56:08 -0800 Message-Id: <1580327768-36501-14-git-send-email-hzhou@ovn.org> X-Mailer: git-send-email 2.1.0 In-Reply-To: <1580327768-36501-1-git-send-email-hzhou@ovn.org> References: <1580327768-36501-1-git-send-email-hzhou@ovn.org> Cc: Han Zhou Subject: [ovs-dev] [PATCH ovn v4 13/13] tutorial: Add tutorial for OVN Interconnection. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" Added tutorial, and also updated NEWS and TODO. Tested-by: Aliasgar Ginwala Signed-off-by: Han Zhou --- Documentation/automake.mk | 1 + Documentation/tutorials/index.rst | 1 + Documentation/tutorials/ovn-interconnection.rst | 188 ++++++++++++++++++++++++ NEWS | 5 + TODO.rst | 6 + 5 files changed, 201 insertions(+) create mode 100644 Documentation/tutorials/ovn-interconnection.rst diff --git a/Documentation/automake.mk b/Documentation/automake.mk index bf21663..2f33753 100644 --- a/Documentation/automake.mk +++ b/Documentation/automake.mk @@ -20,6 +20,7 @@ DOC_SOURCE = \ Documentation/tutorials/ovn-sandbox.rst \ Documentation/tutorials/ovn-ipsec.rst \ Documentation/tutorials/ovn-rbac.rst \ + Documentation/tutorials/ovn-interconnection.rst \ Documentation/topics/index.rst \ Documentation/topics/testing.rst \ Documentation/topics/high-availability.rst \ diff --git a/Documentation/tutorials/index.rst b/Documentation/tutorials/index.rst index 1cf083e..4ff6e16 100644 --- a/Documentation/tutorials/index.rst +++ b/Documentation/tutorials/index.rst @@ -43,3 +43,4 @@ vSwitch. ovn-openstack ovn-rbac ovn-ipsec + ovn-interconnection diff --git a/Documentation/tutorials/ovn-interconnection.rst b/Documentation/tutorials/ovn-interconnection.rst new file mode 100644 index 0000000..2f9d6d7 --- /dev/null +++ b/Documentation/tutorials/ovn-interconnection.rst @@ -0,0 +1,188 @@ +.. + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + + Convention for heading levels in OVN documentation: + + ======= Heading 0 (reserved for the title in a document) + ------- Heading 1 + ~~~~~~~ Heading 2 + +++++++ Heading 3 + ''''''' Heading 4 + + Avoid deeper levels because they do not render well. + +=================== +OVN Interconnection +=================== + +This document provides a guide for interconnecting multiple OVN deployements +with OVN managed tunneling. More details about the OVN Interconnectiong design +can be found in ``ovn-architecture``\(7) manpage. + +This document assumes two or more OVN deployments are setup and runs normally, +possibly at different data-centers, and the gateway chassises of each OVN +are with IP addresses that are reachable between each other. + +Setup Interconnection Databases +------------------------------- + +To interconnect different OVNs, you need to create global OVSDB databases that +store interconnection data. The databases can be setup on any nodes that are +accessible from all the central nodes of each OVN deployment. It is +recommended that the global databases are setup with HA, with nodes in +different avaialbility zones, to avoid single point of failure. + +1. Install OVN packages on each global database node. + +2. Start OVN IC-NB and IC-SB databases. + + On each global database node :: + + $ ovn-ctl [options] start_ic_ovsdb + + Options depends on the HA mode you use. To start standalone mode with TCP + connections, use :: + + $ ovn-ctl --db-ic-nb-create-insecure-remote=yes \ + --db-ic-sb-create-insecure-remote=yes start_ic_ovsdb + + This command starts IC database servers that accept both unix socket and + TCP connections. For other modes, see more details with :: + + $ ovn-ctl --help + +Register OVN to Interconnection Databases +----------------------------------------- + +For each OVN deployment, set an availability zone name :: + + $ ovn-nbctl set NB_Global . name= + +The name should be unique across all OVN deployments, e.g. ovn-east, +ovn-west, etc. + +For each OVN deployment, start the ``ovn-ic`` daemon on central nodes :: + + $ ovn-ctl --ovn-ic-nb-db= --ovn-ic-sb-db= \ + --ovn-northd-nb-db= --ovn-northd-sb-db= [more options] start_ic + +An example of ```` is ``tcp::6645``, or for +clustered DB: ``tcp::6645,tcp::6645,tcp::6645``. +```` is similar, but usually with a different port number, typically, +6646. + +For ```` and ````, use same connection methods as for starting +``northd``. + +Verify each OVN registration from global IC-SB database, using +``ovn-ic-sbctl``, either on a global DB node or other nodes but with property +DB connection method specified in options :: + + $ ovn-ic-sbctl show + +Configure Gateways +------------------ + +For each OVN deployment, specify some chassises as interconnection gateways. +The number of gateways you need depends on the scale and bandwidth you need for +the traffic between the OVN deployments. + +For a node to work as an interconnection gateway, it must firstly be installed +and configured as a regular OVN chassis, with OVS and ``ovn-controller`` +running. To make a chassis as an interconnection gateway, simply run the +command on the chassis :: + + $ ovs-vsctl set open_vswitch . external_ids:ovn-is-interconn=true + +After configuring gateways, verify from the global IC-SB database :: + + $ ovn-ic-sbctl show + +Create Transit Logical Switches +------------------------------- + +Transit Logical Switches, or Transit Switches, are virtual switches for +connecting logical routers in different OVN setups. :: + + $ ovn-ic-nbctl ts-add + +After creating a transit switch, it can be seen from each OVN deployment's +Northbound database, which can be seen using :: + + $ ovn-nbctl find logical_switch other_config:interconn-ts= + +You will also see it with simply ``ovn-nbctl ls-list``. + +If there are multiple tenants that require traffic being isolated from each +other, then multiple transit switches can be created accordingly. + +Connect Logical Routers to Transit Switches +------------------------------------------- + +Connect logical routers from each OVN deployment to the desired transit +switches just as if they are regular logical switches, which includes below +steps (from each OVN, for each logical router you want to connect). + +Assume a transit switch named ``ts1`` is already created in ``IC-NB`` and a +logical router ``lr1`` created in current OVN deployment. + +1. Create a logical router port. :: + + $ ovn-nbctl lrp-add lr1 lrp-lr1-ts1 aa:aa:aa:aa:aa:01 169.254.100.1/24 + + (The mac and IP are examples.) + +2. Create a logical switch port on the transit switch and peer with the logical + router port. :: + + $ ovn-nbctl lsp-add ts1 lsp-ts1-lr1 -- \ + lsp-set-addresses lsp-ts1-lr1 router -- \ + lsp-set-type lsp-ts1-lr1 router -- \ + lsp-set-options lsp-ts1-lr1 router-port=lrp-lr1-ts1 + +3. Assign gateway(s) for the logical router port. :: + + $ ovn-nbctl lrp-set-gateway-chassis lrp-lr1-ts1 [priority] + + Optionally, you can assign more gateways and specify priorities, to achieve + HA, just as usual for a distributed gateway port. + +Similarly in another OVN deployment, you can connect a logical router (e.g. +lr2) to the same transit switch the same way, with a different IP (e.g. +169.254.100.2) on the same subnet. + +The ports connected to transit switches will be automatically populated to +``IC-SB`` database, which can be verified by :: + + $ ovn-ic-sbctl show + +Create Static Routes +-------------------- + +Now that you have all the physical and logical topologies ready, simply create +static routes between the OVN deployments so that packets can be forwarded by +the logical routers through transit switches to the remote OVN. + +For example, in ovn-east, there are workloads using 10.0.1.0/24 under lr1, and +in ovn-west, there are workloads using 10.0.2.0/24 under lr2. + +In ovn-east, add below route :: + + $ ovn-nbctl lr-route-add lr1 10.0.2.0/24 169.254.100.2 + +In ovs-west, add below route :: + + $ ovn-nbctl lr-route-add lr2 10.0.1.0/24 169.254.100.1 + +Now the traffic should be able to go through between the workloads through +tunnels crossing gateway nodes of ovn-east and ovn-west. diff --git a/NEWS b/NEWS index 2b8cd6f..bf1565f 100644 --- a/NEWS +++ b/NEWS @@ -7,6 +7,11 @@ Post-OVS-v2.12.0 - Added Forwarding Group support in OVN. - Added support for MLD Snooping and MLD Querier. + - OVN Interconnection: + * Support for L3 interconnection of multiple OVN deployments with tunnels + managed by OVN. See instructions in + Documentation/tutorials/ovn-interconnection.rst. + v2.12.0 - 03 Sep 2019 --------------------- - DPDK: diff --git a/TODO.rst b/TODO.rst index 943d9bf..fbab508 100644 --- a/TODO.rst +++ b/TODO.rst @@ -145,3 +145,9 @@ OVN To-do List * Support FTP ALGs. * Support reject action. + +* OVN Interconnection + + * Packaging for RHEL, Debian, etc. + + * Route advertisement between edge routers.