[ovs-dev,ovn,19/19] tutorial: Add tutorial for OVN Interconnection.
diff mbox series

Message ID 1571619079-75503-20-git-send-email-hzhou@ovn.org
State New
Headers show
Series
  • OVN Interconnection
Related show

Commit Message

Han Zhou Oct. 21, 2019, 12:51 a.m. UTC
Added tutorial, and also updated NEWS and TODO.

Signed-off-by: Han Zhou <hzhou@ovn.org>
---
 Documentation/automake.mk                       |   1 +
 Documentation/tutorials/index.rst               |   1 +
 Documentation/tutorials/ovn-interconnection.rst | 181 ++++++++++++++++++++++++
 NEWS                                            |   5 +
 TODO.rst                                        |  10 ++
 5 files changed, 198 insertions(+)
 create mode 100644 Documentation/tutorials/ovn-interconnection.rst

Comments

Han Zhou Oct. 24, 2019, 5:16 p.m. UTC | #1
On Sun, Oct 20, 2019 at 5:52 PM Han Zhou <hzhou@ovn.org> wrote:

> diff --git a/Documentation/tutorials/ovn-interconnection.rst
b/Documentation/tutorials/ovn-interconnection.rst
...
> +
> +For each OVN deployment, start the ``ovn-ic`` daemon on central nodes ::
> +
> +    $ ovn-ctl --ovninb-db=<IC-NB> --ovnisb-db=<IC-SB> \
> +              --ovnnb-db=<NB> --ovnsb-db=<SB> [more options] start_ic
> +

Sorry that here the options names are wrong. It should be:
    $ ovn-ctl --ovn-ic-inb-db=<IC-NB> --ovn-ic-isb-db=<IC-SB> \
              --ovn-northd-nb-db=<NB> --ovn-northd-sb-db=<SB> [more
options] start_ic

I will correct it in v2, but I want to wait for more feedbacks on other
patches in the series.
aginwala Oct. 30, 2019, 12:49 a.m. UTC | #2
Thanks Han for the correction. Just one more minor typo in the tutorial
below.

I hit some roadblocks to start ic controller on different az but got my
setup running with workarounds in current code and have posted in comments
which needs fix for sure. I tried with 2 different ovn setups with 2 AZs
where each ovn az uses1 hv, 1 gw and 1 lport bound to hv. I am able to
access the workloads across azs.


Tested-by: Aliasgar Ginwala <aginwala@ebay.com <gvrose8192@gmail.com>>



On Sun, Oct 20, 2019 at 5:55 PM Han Zhou <hzhou@ovn.org> wrote:

> Added tutorial, and also updated NEWS and TODO.
>
> Signed-off-by: Han Zhou <hzhou@ovn.org>
> ---
>  Documentation/automake.mk                       |   1 +
>  Documentation/tutorials/index.rst               |   1 +
>  Documentation/tutorials/ovn-interconnection.rst | 181
> ++++++++++++++++++++++++
>  NEWS                                            |   5 +
>  TODO.rst                                        |  10 ++
>  5 files changed, 198 insertions(+)
>  create mode 100644 Documentation/tutorials/ovn-interconnection.rst
>
> diff --git a/Documentation/automake.mk b/Documentation/automake.mk
> index 5968d69..15d261d 100644
> --- a/Documentation/automake.mk
> +++ b/Documentation/automake.mk
> @@ -20,6 +20,7 @@ DOC_SOURCE = \
>         Documentation/tutorials/ovn-sandbox.rst \
>         Documentation/tutorials/ovn-ipsec.rst \
>         Documentation/tutorials/ovn-rbac.rst \
> +       Documentation/tutorials/ovn-interconnection.rst \
>         Documentation/topics/index.rst \
>         Documentation/topics/testing.rst \
>         Documentation/topics/high-availability.rst \
> diff --git a/Documentation/tutorials/index.rst
> b/Documentation/tutorials/index.rst
> index 1cf083e..4ff6e16 100644
> --- a/Documentation/tutorials/index.rst
> +++ b/Documentation/tutorials/index.rst
> @@ -43,3 +43,4 @@ vSwitch.
>     ovn-openstack
>     ovn-rbac
>     ovn-ipsec
> +   ovn-interconnection
> diff --git a/Documentation/tutorials/ovn-interconnection.rst
> b/Documentation/tutorials/ovn-interconnection.rst
> new file mode 100644
> index 0000000..1320d41
> --- /dev/null
> +++ b/Documentation/tutorials/ovn-interconnection.rst
> @@ -0,0 +1,181 @@
> +..
> +      Licensed under the Apache License, Version 2.0 (the "License"); you
> may
> +      not use this file except in compliance with the License. You may
> obtain
> +      a copy of the License at
> +
> +          http://www.apache.org/licenses/LICENSE-2.0
> +
> +      Unless required by applicable law or agreed to in writing, software
> +      distributed under the License is distributed on an "AS IS" BASIS,
> WITHOUT
> +      WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
> See the
> +      License for the specific language governing permissions and
> limitations
> +      under the License.
> +
> +      Convention for heading levels in OVN documentation:
> +
> +      =======  Heading 0 (reserved for the title in a document)
> +      -------  Heading 1
> +      ~~~~~~~  Heading 2
> +      +++++++  Heading 3
> +      '''''''  Heading 4
> +
> +      Avoid deeper levels because they do not render well.
> +
> +===================
> +OVN Interconnection
> +===================
> +
> +This document provides a guide for interconnecting multiple OVN
> deployements
> +with OVN managed tunneling.  More details about the OVN Interconnectiong
> design
> +can be found in ``ovn-architecture``\(7) manpage.
> +
> +This document assumes two or more OVN deployments are setup and runs
> normally,
> +possibly at different data-centers, and the gateway chassises of each OVN
> +are with IP addresses that are reachable between each other.
> +
> +Setup Interconnection Databases
> +-------------------------------
> +
> +To interconnect different OVNs, you need to create global OVSDB databases
> that
> +store interconnection data.  The databases can be setup on any nodes that
> are
> +accessible from all the central nodes of each OVN deployment.  It is
> +recommended that the global databases are setup with HA, with nodes in
> +different avaialbility zones, to avoid single point of failure.
> +
> +1. Install OVN packages on each global database node.
> +
> +2. Start OVN IC-NB and IC-SB databases.
> +
> +   On each global database node ::
> +
> +    $ ovn-ctl [options] start_ic_ovsdb
> +
> +   Options depends on the HA mode you use.  See details with ::
> +
> +    $ ovn-ctl --help.
> +
> +Register OVN to Interconnection Databases
> +-----------------------------------------
> +
> +For each OVN deployment, set an availability zone name ::
> +
> +    $ ovn-nbctl set NB_Global . name=<availability zone name>
> +
> +The name should be unique across all OVN deployments, e.g. ovn-east,
> +ovn-west, etc.
> +
> +For each OVN deployment, start the ``ovn-ic`` daemon on central nodes ::
> +
> +    $ ovn-ctl --ovninb-db=<IC-NB> --ovnisb-db=<IC-SB> \
> +              --ovnnb-db=<NB> --ovnsb-db=<SB> [more options] start_ic
> +
> +An example of ``<IC-NB>`` is ``tcp:<global db hostname>:6645``, or for
> +clustered DB: ``tcp:<node1>:6645,tcp:<node2>:6645,tcp:<node3>:6645``.
> +``<IC-SB>`` is similar, but usually with a different port number,
> typically,
> +6646.
> +
> +For ``<NB>`` and ``<SB>``, use same connection methods as for starting
> +``northd``.
> +
> +Verify each OVN registration from global IC-SB database, using
> +``ovn-isbctl``, either on a global DB node or other nodes but with
> property
> +DB connection method specified in options ::
> +
> +    $ ovn-isbctl show
> +
> +Configure Gateways
> +------------------
> +
> +For each OVN deployment, specify some chassises as interconnection
> gateways.
> +The number of gateways you need depends on the scale and bandwidth you
> need for
> +the traffic between the OVN deployments.
> +
> +For a node to work as an interconnection gateway, it must firstly be
> installed
> +and configured as a regular OVN chassis, with OVS and ``ovn-controller``
> +running.  To make a chassis as an interconnection gateway, simply run the
> +command on the chassis ::
> +
> +    $ ovs-vsctl set open_vswitch . external_ids:is-interconn=true
> +
> +After configuring gateways, verify from the global IC-SB database ::
> +
> +    $ ovn-isbctl show
> +
> +Create Transit Logical Switches
> +-------------------------------
> +
> +Transit Logical Switches, or Transit Switches, are virtual switches for
> +connecting logical routers in different OVN setups. ::
> +
> +    $ ovn-inbctl ts-add <name>
> +
> +After creating a transit switch, it can be seen from each OVN deployment's
> +Northbound database, which can be seen using ::
> +
> +    $ ovn-nbctl find logical_switch other_config:interconn-ts=<name>
> +
> +You will also see it with simply ``ovn-nbctl ls-list``.
> +
> +If there are multiple tenants that require traffic being isolated from
> each
> +other, then multiple transit switches can be created accordingly.
> +
> +Connect Logical Routers to Transit Switches
> +-------------------------------------------
> +
> +Connect logical routers from each OVN deployment to the desired transit
> +switches just as if they are regular logical switches, which includes
> below
> +steps (from each OVN, for each logical router you want to connect).
> +
> +Assume a transit switch named ``ts1`` is already created in ``IC-NB`` and
> a
> +logical router ``lr1`` created in current OVN deployment.
> +
> +1. Create a logical router port. ::
> +
> +    $ ovn-nbctl lrp-add lr1 lrp-lr1-ts1 aa:aa:aa:aa:aa:01
> 169.254.100.1/24
> +
> +   (The mac and IP are examples.)
> +
> +2. Create a logical switch port on the transit switch and peer with the
> logical
> +   router port. ::
> +
> +    $ ovn-nbctl lsp-add ts1 lsp-ts1-lr1 -- \
> +            ovn-nbctl lsp-set-addresses lsp-ts1-lr1 router -- \
> +            ovn-nbctl lsp-set-type lsp-ts1-lr1 router -- \
> +            ovn-nbctl lsp-set-options lsp-ts1-lr1
> +
> +3. Assign gateway(s) for the logical router port. ::
> +
> +    $ ovn-nbctl lrp-set-gateway-chassis lrp-lr1-ts1 <gateway name>
> [priority]
> +
> +   Optionally, you can assign more gateways and specify priorities, to
> achieve
> +   HA, just as usual for a distributed gateway port.
> +
> +Similarly in another OVN deployment, you can connect a logical router
> (e.g.
> +lr2) to the same transit switch the same way, with a different IP (e.g.
> +169.254.100.2) on the same subnet.
> +
> +The ports connected to transit switches will be automatically populated to
> +``IC-SB`` database, which can be verified by ::
> +
> +    $ ovn-isbctl show
> +
> +Create Static Routes
> +--------------------
> +
> +Now that you have all the physical and logical topologies ready, simply
> create
> +static routes between the OVN deployments so that packets can be
> forwarded by
> +the logical routers through transit switches to the remote OVN.
> +
> +For example, in ovn-east, there are workloads using 10.0.1.0/24 under
> lr1, and
> +in ovn-west, there are workloads using 10.0.2.0/24 under lr2.
> +
> +In ovn-east, add below route ::
> +
> +    $ ovn-nbctl lr-route-add lr1 10.0.2.0/24 169.254.100.2
> +
> +In ovs-west, add below route ::
> +
> +    $ ovn-nbctl lr-route-add lr1 10.0.1.0/24 169.254.100.1
>
should be lr2.

> +
> +Now the traffic should be able to go through between the workloads through
> +tunnels crossing gateway nodes of ovn-east and ovn-west.
> diff --git a/NEWS b/NEWS
> index 72e52b9..41a0d95 100644
> --- a/NEWS
> +++ b/NEWS
> @@ -3,6 +3,11 @@ Post-v2.12.0
>     - This is the first release after OVN is split from Open vSwitch
>       project.
>
> +   - OVN Interconnection:
> +     * Support for L3 interconnection of multiple OVN deployments with
> tunnels
> +       managed by OVN.  See instructions in
> +       Documentation/tutorials/ovn-interconnection.rst.
> +
>  v2.12.0 - 03 Sep 2019
>  ---------------------
>     - DPDK:
> diff --git a/TODO.rst b/TODO.rst
> index ed55ea2..5de1420 100644
> --- a/TODO.rst
> +++ b/TODO.rst
> @@ -155,3 +155,13 @@ OVN To-do List
>
>        match(ip4.src == {IP1, IP2, IP3} && ip4.dst == {IP4, IP5, IP6} &&
>        tcp.dst >= 1000 && tcp.dst <= 2000) actions=allow
> +
> +* OVN Interconnection
> +
> +  * Packaging for RHEL, Debian, etc.
> +
> +  * Gateway HA enhancement. Currently gateway HA relies on each OVN's BFD
> +    monitoring detecting gateway failover and updating port-binding in
> +    SB DB, which then is synced to IC-SB DB by ovn-ic. This may have
> longer
> +    latency for failover than monitoring between each gateway pairs
> +    acrossing OVN.
> --
> 2.1.0
>
> _______________________________________________
> dev mailing list
> dev@openvswitch.org
> https://mail.openvswitch.org/mailman/listinfo/ovs-dev
>
Han Zhou Oct. 30, 2019, 9:24 p.m. UTC | #3
On Tue, Oct 29, 2019 at 5:49 PM aginwala <aginwala@asu.edu> wrote:

>
> Thanks Han for the correction. Just one more minor typo in the tutorial
> below.
>
> I hit some roadblocks to start ic controller on different az but got my
> setup running with workarounds in current code and have posted in comments
> which needs fix for sure. I tried with 2 different ovn setups with 2 AZs
> where each ovn az uses1 hv, 1 gw and 1 lport bound to hv. I am able to
> access the workloads across azs.
>
>
> Tested-by: Aliasgar Ginwala <aginwala@ebay.com <gvrose8192@gmail.com>>
>
>
>
>
> +In ovs-west, add below route ::
>> +
>> +    $ ovn-nbctl lr-route-add lr1 10.0.1.0/24 169.254.100.1
>>
> should be lr2.
>
>>
>>
Thanks for the testing and the findings. I corrected it in v2.
https://patchwork.ozlabs.org/project/openvswitch/list/?series=139731

Patch
diff mbox series

diff --git a/Documentation/automake.mk b/Documentation/automake.mk
index 5968d69..15d261d 100644
--- a/Documentation/automake.mk
+++ b/Documentation/automake.mk
@@ -20,6 +20,7 @@  DOC_SOURCE = \
 	Documentation/tutorials/ovn-sandbox.rst \
 	Documentation/tutorials/ovn-ipsec.rst \
 	Documentation/tutorials/ovn-rbac.rst \
+	Documentation/tutorials/ovn-interconnection.rst \
 	Documentation/topics/index.rst \
 	Documentation/topics/testing.rst \
 	Documentation/topics/high-availability.rst \
diff --git a/Documentation/tutorials/index.rst b/Documentation/tutorials/index.rst
index 1cf083e..4ff6e16 100644
--- a/Documentation/tutorials/index.rst
+++ b/Documentation/tutorials/index.rst
@@ -43,3 +43,4 @@  vSwitch.
    ovn-openstack
    ovn-rbac
    ovn-ipsec
+   ovn-interconnection
diff --git a/Documentation/tutorials/ovn-interconnection.rst b/Documentation/tutorials/ovn-interconnection.rst
new file mode 100644
index 0000000..1320d41
--- /dev/null
+++ b/Documentation/tutorials/ovn-interconnection.rst
@@ -0,0 +1,181 @@ 
+..
+      Licensed under the Apache License, Version 2.0 (the "License"); you may
+      not use this file except in compliance with the License. You may obtain
+      a copy of the License at
+
+          http://www.apache.org/licenses/LICENSE-2.0
+
+      Unless required by applicable law or agreed to in writing, software
+      distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+      WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+      License for the specific language governing permissions and limitations
+      under the License.
+
+      Convention for heading levels in OVN documentation:
+
+      =======  Heading 0 (reserved for the title in a document)
+      -------  Heading 1
+      ~~~~~~~  Heading 2
+      +++++++  Heading 3
+      '''''''  Heading 4
+
+      Avoid deeper levels because they do not render well.
+
+===================
+OVN Interconnection
+===================
+
+This document provides a guide for interconnecting multiple OVN deployements
+with OVN managed tunneling.  More details about the OVN Interconnectiong design
+can be found in ``ovn-architecture``\(7) manpage.
+
+This document assumes two or more OVN deployments are setup and runs normally,
+possibly at different data-centers, and the gateway chassises of each OVN
+are with IP addresses that are reachable between each other.
+
+Setup Interconnection Databases
+-------------------------------
+
+To interconnect different OVNs, you need to create global OVSDB databases that
+store interconnection data.  The databases can be setup on any nodes that are
+accessible from all the central nodes of each OVN deployment.  It is
+recommended that the global databases are setup with HA, with nodes in
+different avaialbility zones, to avoid single point of failure.
+
+1. Install OVN packages on each global database node.
+
+2. Start OVN IC-NB and IC-SB databases.
+
+   On each global database node ::
+
+    $ ovn-ctl [options] start_ic_ovsdb
+
+   Options depends on the HA mode you use.  See details with ::
+
+    $ ovn-ctl --help.
+
+Register OVN to Interconnection Databases
+-----------------------------------------
+
+For each OVN deployment, set an availability zone name ::
+
+    $ ovn-nbctl set NB_Global . name=<availability zone name>
+
+The name should be unique across all OVN deployments, e.g. ovn-east,
+ovn-west, etc.
+
+For each OVN deployment, start the ``ovn-ic`` daemon on central nodes ::
+
+    $ ovn-ctl --ovninb-db=<IC-NB> --ovnisb-db=<IC-SB> \
+              --ovnnb-db=<NB> --ovnsb-db=<SB> [more options] start_ic
+
+An example of ``<IC-NB>`` is ``tcp:<global db hostname>:6645``, or for
+clustered DB: ``tcp:<node1>:6645,tcp:<node2>:6645,tcp:<node3>:6645``.
+``<IC-SB>`` is similar, but usually with a different port number, typically,
+6646.
+
+For ``<NB>`` and ``<SB>``, use same connection methods as for starting
+``northd``.
+
+Verify each OVN registration from global IC-SB database, using
+``ovn-isbctl``, either on a global DB node or other nodes but with property
+DB connection method specified in options ::
+
+    $ ovn-isbctl show
+
+Configure Gateways
+------------------
+
+For each OVN deployment, specify some chassises as interconnection gateways.
+The number of gateways you need depends on the scale and bandwidth you need for
+the traffic between the OVN deployments.
+
+For a node to work as an interconnection gateway, it must firstly be installed
+and configured as a regular OVN chassis, with OVS and ``ovn-controller``
+running.  To make a chassis as an interconnection gateway, simply run the
+command on the chassis ::
+
+    $ ovs-vsctl set open_vswitch . external_ids:is-interconn=true
+
+After configuring gateways, verify from the global IC-SB database ::
+
+    $ ovn-isbctl show
+
+Create Transit Logical Switches
+-------------------------------
+
+Transit Logical Switches, or Transit Switches, are virtual switches for
+connecting logical routers in different OVN setups. ::
+
+    $ ovn-inbctl ts-add <name>
+
+After creating a transit switch, it can be seen from each OVN deployment's
+Northbound database, which can be seen using ::
+
+    $ ovn-nbctl find logical_switch other_config:interconn-ts=<name>
+
+You will also see it with simply ``ovn-nbctl ls-list``.
+
+If there are multiple tenants that require traffic being isolated from each
+other, then multiple transit switches can be created accordingly.
+
+Connect Logical Routers to Transit Switches
+-------------------------------------------
+
+Connect logical routers from each OVN deployment to the desired transit
+switches just as if they are regular logical switches, which includes below
+steps (from each OVN, for each logical router you want to connect).
+
+Assume a transit switch named ``ts1`` is already created in ``IC-NB`` and a
+logical router ``lr1`` created in current OVN deployment.
+
+1. Create a logical router port. ::
+
+    $ ovn-nbctl lrp-add lr1 lrp-lr1-ts1 aa:aa:aa:aa:aa:01 169.254.100.1/24
+
+   (The mac and IP are examples.)
+
+2. Create a logical switch port on the transit switch and peer with the logical
+   router port. ::
+
+    $ ovn-nbctl lsp-add ts1 lsp-ts1-lr1 -- \
+            ovn-nbctl lsp-set-addresses lsp-ts1-lr1 router -- \
+            ovn-nbctl lsp-set-type lsp-ts1-lr1 router -- \
+            ovn-nbctl lsp-set-options lsp-ts1-lr1
+
+3. Assign gateway(s) for the logical router port. ::
+
+    $ ovn-nbctl lrp-set-gateway-chassis lrp-lr1-ts1 <gateway name> [priority]
+
+   Optionally, you can assign more gateways and specify priorities, to achieve
+   HA, just as usual for a distributed gateway port.
+
+Similarly in another OVN deployment, you can connect a logical router (e.g.
+lr2) to the same transit switch the same way, with a different IP (e.g.
+169.254.100.2) on the same subnet.
+
+The ports connected to transit switches will be automatically populated to
+``IC-SB`` database, which can be verified by ::
+
+    $ ovn-isbctl show
+
+Create Static Routes
+--------------------
+
+Now that you have all the physical and logical topologies ready, simply create
+static routes between the OVN deployments so that packets can be forwarded by
+the logical routers through transit switches to the remote OVN.
+
+For example, in ovn-east, there are workloads using 10.0.1.0/24 under lr1, and
+in ovn-west, there are workloads using 10.0.2.0/24 under lr2.
+
+In ovn-east, add below route ::
+
+    $ ovn-nbctl lr-route-add lr1 10.0.2.0/24 169.254.100.2
+
+In ovs-west, add below route ::
+
+    $ ovn-nbctl lr-route-add lr1 10.0.1.0/24 169.254.100.1
+
+Now the traffic should be able to go through between the workloads through
+tunnels crossing gateway nodes of ovn-east and ovn-west.
diff --git a/NEWS b/NEWS
index 72e52b9..41a0d95 100644
--- a/NEWS
+++ b/NEWS
@@ -3,6 +3,11 @@  Post-v2.12.0
    - This is the first release after OVN is split from Open vSwitch
      project.
 
+   - OVN Interconnection:
+     * Support for L3 interconnection of multiple OVN deployments with tunnels
+       managed by OVN.  See instructions in
+       Documentation/tutorials/ovn-interconnection.rst.
+
 v2.12.0 - 03 Sep 2019
 ---------------------
    - DPDK:
diff --git a/TODO.rst b/TODO.rst
index ed55ea2..5de1420 100644
--- a/TODO.rst
+++ b/TODO.rst
@@ -155,3 +155,13 @@  OVN To-do List
 
       match(ip4.src == {IP1, IP2, IP3} && ip4.dst == {IP4, IP5, IP6} &&
       tcp.dst >= 1000 && tcp.dst <= 2000) actions=allow
+
+* OVN Interconnection
+
+  * Packaging for RHEL, Debian, etc.
+
+  * Gateway HA enhancement. Currently gateway HA relies on each OVN's BFD
+    monitoring detecting gateway failover and updating port-binding in
+    SB DB, which then is synced to IC-SB DB by ovn-ic. This may have longer
+    latency for failover than monitoring between each gateway pairs
+    acrossing OVN.