@@ -20,6 +20,7 @@ DOC_SOURCE = \
Documentation/tutorials/ovn-sandbox.rst \
Documentation/tutorials/ovn-ipsec.rst \
Documentation/tutorials/ovn-rbac.rst \
+ Documentation/tutorials/ovn-interconnection.rst \
Documentation/topics/index.rst \
Documentation/topics/testing.rst \
Documentation/topics/high-availability.rst \
@@ -43,3 +43,4 @@ vSwitch.
ovn-openstack
ovn-rbac
ovn-ipsec
+ ovn-interconnection
new file mode 100644
@@ -0,0 +1,188 @@
+..
+ Licensed under the Apache License, Version 2.0 (the "License"); you may
+ not use this file except in compliance with the License. You may obtain
+ a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+ WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+ License for the specific language governing permissions and limitations
+ under the License.
+
+ Convention for heading levels in OVN documentation:
+
+ ======= Heading 0 (reserved for the title in a document)
+ ------- Heading 1
+ ~~~~~~~ Heading 2
+ +++++++ Heading 3
+ ''''''' Heading 4
+
+ Avoid deeper levels because they do not render well.
+
+===================
+OVN Interconnection
+===================
+
+This document provides a guide for interconnecting multiple OVN deployements
+with OVN managed tunneling. More details about the OVN Interconnectiong design
+can be found in ``ovn-architecture``\(7) manpage.
+
+This document assumes two or more OVN deployments are setup and runs normally,
+possibly at different data-centers, and the gateway chassises of each OVN
+are with IP addresses that are reachable between each other.
+
+Setup Interconnection Databases
+-------------------------------
+
+To interconnect different OVNs, you need to create global OVSDB databases that
+store interconnection data. The databases can be setup on any nodes that are
+accessible from all the central nodes of each OVN deployment. It is
+recommended that the global databases are setup with HA, with nodes in
+different avaialbility zones, to avoid single point of failure.
+
+1. Install OVN packages on each global database node.
+
+2. Start OVN IC-NB and IC-SB databases.
+
+ On each global database node ::
+
+ $ ovn-ctl [options] start_ic_ovsdb
+
+ Options depends on the HA mode you use. To start standalone mode with TCP
+ connections, use ::
+
+ $ ovn-ctl --db-ic-nb-create-insecure-remote=yes \
+ --db-ic-sb-create-insecure-remote=yes start_ic_ovsdb
+
+ This command starts IC database servers that accept both unix socket and
+ TCP connections. For other modes, see more details with ::
+
+ $ ovn-ctl --help
+
+Register OVN to Interconnection Databases
+-----------------------------------------
+
+For each OVN deployment, set an availability zone name ::
+
+ $ ovn-nbctl set NB_Global . name=<availability zone name>
+
+The name should be unique across all OVN deployments, e.g. ovn-east,
+ovn-west, etc.
+
+For each OVN deployment, start the ``ovn-ic`` daemon on central nodes ::
+
+ $ ovn-ctl --ovn-ic-nb-db=<IC-NB> --ovn-ic-sb-db=<IC-SB> \
+ --ovn-northd-nb-db=<NB> --ovn-northd-sb-db=<SB> [more options] start_ic
+
+An example of ``<IC-NB>`` is ``tcp:<global db hostname>:6645``, or for
+clustered DB: ``tcp:<node1>:6645,tcp:<node2>:6645,tcp:<node3>:6645``.
+``<IC-SB>`` is similar, but usually with a different port number, typically,
+6646.
+
+For ``<NB>`` and ``<SB>``, use same connection methods as for starting
+``northd``.
+
+Verify each OVN registration from global IC-SB database, using
+``ovn-ic-sbctl``, either on a global DB node or other nodes but with property
+DB connection method specified in options ::
+
+ $ ovn-ic-sbctl show
+
+Configure Gateways
+------------------
+
+For each OVN deployment, specify some chassises as interconnection gateways.
+The number of gateways you need depends on the scale and bandwidth you need for
+the traffic between the OVN deployments.
+
+For a node to work as an interconnection gateway, it must firstly be installed
+and configured as a regular OVN chassis, with OVS and ``ovn-controller``
+running. To make a chassis as an interconnection gateway, simply run the
+command on the chassis ::
+
+ $ ovs-vsctl set open_vswitch . external_ids:ovn-is-interconn=true
+
+After configuring gateways, verify from the global IC-SB database ::
+
+ $ ovn-ic-sbctl show
+
+Create Transit Logical Switches
+-------------------------------
+
+Transit Logical Switches, or Transit Switches, are virtual switches for
+connecting logical routers in different OVN setups. ::
+
+ $ ovn-ic-nbctl ts-add <name>
+
+After creating a transit switch, it can be seen from each OVN deployment's
+Northbound database, which can be seen using ::
+
+ $ ovn-nbctl find logical_switch other_config:interconn-ts=<name>
+
+You will also see it with simply ``ovn-nbctl ls-list``.
+
+If there are multiple tenants that require traffic being isolated from each
+other, then multiple transit switches can be created accordingly.
+
+Connect Logical Routers to Transit Switches
+-------------------------------------------
+
+Connect logical routers from each OVN deployment to the desired transit
+switches just as if they are regular logical switches, which includes below
+steps (from each OVN, for each logical router you want to connect).
+
+Assume a transit switch named ``ts1`` is already created in ``IC-NB`` and a
+logical router ``lr1`` created in current OVN deployment.
+
+1. Create a logical router port. ::
+
+ $ ovn-nbctl lrp-add lr1 lrp-lr1-ts1 aa:aa:aa:aa:aa:01 169.254.100.1/24
+
+ (The mac and IP are examples.)
+
+2. Create a logical switch port on the transit switch and peer with the logical
+ router port. ::
+
+ $ ovn-nbctl lsp-add ts1 lsp-ts1-lr1 -- \
+ lsp-set-addresses lsp-ts1-lr1 router -- \
+ lsp-set-type lsp-ts1-lr1 router -- \
+ lsp-set-options lsp-ts1-lr1 router-port=lrp-lr1-ts1
+
+3. Assign gateway(s) for the logical router port. ::
+
+ $ ovn-nbctl lrp-set-gateway-chassis lrp-lr1-ts1 <gateway name> [priority]
+
+ Optionally, you can assign more gateways and specify priorities, to achieve
+ HA, just as usual for a distributed gateway port.
+
+Similarly in another OVN deployment, you can connect a logical router (e.g.
+lr2) to the same transit switch the same way, with a different IP (e.g.
+169.254.100.2) on the same subnet.
+
+The ports connected to transit switches will be automatically populated to
+``IC-SB`` database, which can be verified by ::
+
+ $ ovn-ic-sbctl show
+
+Create Static Routes
+--------------------
+
+Now that you have all the physical and logical topologies ready, simply create
+static routes between the OVN deployments so that packets can be forwarded by
+the logical routers through transit switches to the remote OVN.
+
+For example, in ovn-east, there are workloads using 10.0.1.0/24 under lr1, and
+in ovn-west, there are workloads using 10.0.2.0/24 under lr2.
+
+In ovn-east, add below route ::
+
+ $ ovn-nbctl lr-route-add lr1 10.0.2.0/24 169.254.100.2
+
+In ovs-west, add below route ::
+
+ $ ovn-nbctl lr-route-add lr2 10.0.1.0/24 169.254.100.1
+
+Now the traffic should be able to go through between the workloads through
+tunnels crossing gateway nodes of ovn-east and ovn-west.
@@ -6,6 +6,11 @@ Post-OVS-v2.12.0
- Added Stateless Floating IP support in OVN.
- Added Forwarding Group support in OVN.
+ - OVN Interconnection:
+ * Support for L3 interconnection of multiple OVN deployments with tunnels
+ managed by OVN. See instructions in
+ Documentation/tutorials/ovn-interconnection.rst.
+
v2.12.0 - 03 Sep 2019
---------------------
- DPDK:
@@ -145,3 +145,9 @@ OVN To-do List
* Support FTP ALGs.
* Support reject action.
+
+* OVN Interconnection
+
+ * Packaging for RHEL, Debian, etc.
+
+ * Route advertisement between edge routers.