[ovs-dev,patch_v6,2/2] ovn: Reformat some ovn design documentation.

Message ID 1469305082-58994-3-git-send-email-dlu998@gmail.com
State Deferred
Headers show

Commit Message

Darrell Ball July 23, 2016, 8:18 p.m.
Some design micro-details (e.g.) register assignments) that may change
over time were moved from the ovn-architecture.7.xml document to the
OVN-DESIGN.md document.  A table is added to summarize and quantify
MFF logical metadata and register usage.  The OVN-DESIGN.md file was
tested using the following markdown parsers:

https://jbt.github.io/markdown-editor/
http://dillinger.io/

The new flags register usage is also documented in the new
OVN-DESIGN.md document.

Signed-off-by: Darrell Ball <dlu998@gmail.com>
---
 ovn/OVN-DESIGN.md          | 199 +++++++++++++++++++++++++++++++++++++++++++++
 ovn/automake.mk            |   1 +
 ovn/ovn-architecture.7.xml | 193 -------------------------------------------
 3 files changed, 200 insertions(+), 193 deletions(-)
 create mode 100644 ovn/OVN-DESIGN.md

Comments

Flavio Fernandes July 25, 2016, 2:06 a.m. | #1
Embarrassingly minor nitpicks follow...


On Sat, Jul 23, 2016 at 3:18 PM, Darrell Ball <dlu998@gmail.com> wrote:

> Some design micro-details (e.g.) register assignments) that may change
> over time were moved from the ovn-architecture.7.xml document to the
> OVN-DESIGN.md document.  A table is added to summarize and quantify
> MFF logical metadata and register usage.  The OVN-DESIGN.md file was
> tested using the following markdown parsers:
>
> https://jbt.github.io/markdown-editor/
> http://dillinger.io/
>
> The new flags register usage is also documented in the new
> OVN-DESIGN.md document.
>
> Signed-off-by: Darrell Ball <dlu998@gmail.com>
> ---
>  ovn/OVN-DESIGN.md          | 199
> +++++++++++++++++++++++++++++++++++++++++++++
>  ovn/automake.mk            |   1 +
>  ovn/ovn-architecture.7.xml | 193
> -------------------------------------------
>  3 files changed, 200 insertions(+), 193 deletions(-)
>  create mode 100644 ovn/OVN-DESIGN.md
>
> diff --git a/ovn/OVN-DESIGN.md b/ovn/OVN-DESIGN.md
> new file mode 100644
> index 0000000..2162c65
> --- /dev/null
> +++ b/ovn/OVN-DESIGN.md
> @@ -0,0 +1,199 @@
> +OVN register and metadata usage:
> +-------------------------------
> +
> +logical datapath field:
> +
> +A field that denotes the logical datapath through which a packet is being
> +processed.
> +*Keep the following in sync with MFF_LOG_DATAPATH in*
> +*ovn/lib/logical-fields.h.*
> +OVN uses the field that OpenFlow 1.1+ simply (and confusingly) calls
> +'metadata' to store the logical datapath.  (This field is passed across
> +tunnels as part of the tunnel key.)
> +
> +
> +logical input port field:
> +
> +A field that denotes the logical port from which the packet entered the
> +logical datapath.
> +*Keep the following in sync with MFF_LOG_INPORT in*
> +*ovn/lib/logical-fields.h.*
> +OVN stores this in Nicira extension register number 14.
> +
> +Geneve and STT tunnels pass this field as part of the tunnel key.
> +Although VXLAN tunnels do not explicitly carry a logical input port,
> +OVN only uses VXLAN to communicate with gateways that from OVN's
> +perspective consist of only a single logical port, so that OVN can set
> +the logical input port field to this one on ingress to the OVN logical
> +pipeline.
> +
> +
> +logical output port field:
> +
> +A field that denotes the logical port from which the packet will leave
> +the logical datapath.  This is initialized to 0 at the beginning of the
> +logical ingress pipeline.
> +*Keep the following in sync with MFF_LOG_OUTPORT in*
> +*ovn/lib/logical-fields.h.*
> +OVN stores this in Nicira extension register number 15.
> +
> +Geneve and STT tunnels pass this field as part of the tunnel key.  VXLAN
> +tunnels do not transmit the logical output port field.
> +
> +
> +conntrack zone field for logical ports:
> +
> +A field that denotes the connection tracking zone for logical ports.
> +The value only has local significance and is not meaningful between
> +chassis.  This is initialized to 0 at the beginning of the logical
> +ingress pipeline.
> +*Keep the following in sync with MFF_LOG_CT_ZONE in*
> +*ovn/lib/logical-fields.h.*
> +OVN stores this in Nicira extension register number 13.
> +
> +
> +conntrack zone fields for Gateway router:
> +
> +Fields that denote the connection tracking zones for Gateway routers.
> +These values only have local significance (only on chassis that have
> +Gateway routers instantiated) and is not meaningful between
> +chassis.
> +*Keep the following in sync with MFF_LOG_DNAT_ZONE and*
> +*MFF_LOG_DNAT_ZONE inovn/lib/logical-fields.h.*
>

Missing space between "in" and "ovn/lib/logical-fields.h.*"



> +OVN stores the zone information for DNATting in Nicira
> +extension register number 11 and zone information for SNATing in Nicira
> +extension register number 12.
> +
> +
> +flags field:
> +
> +Scratchpad flags that denote the pipeline state between tables.  The
> +values only have local significance and are not meaningful between
> +chassis.  This is initialized to 0 at the beginning of the logical
> +ingress pipeline.
> +*Keep the following in sync with MFF_LOG_FLAGS in*
> +*ovn/lib/logical-fields.h.*
> +OVN stores this in Nicira extension register number 10.
> +
> +
> +VLAN ID:
> +
> +The VLAN ID is used as an interface between OVN and containers nested
> +inside a VM (see Life Cycle of a container interface inside a VM, in
> +ovn-architecture.7.xml, for more information).
> +
> +
> +The following table summarizes the register and metadata usage for OVN.
> +*Keep the following in sync with MFF_LOG_FLAGS in*
> +*ovn/lib/logical-fields.h.*:
> +
> +```
> +
> +  Register/Metadata         Usage                      Bits (Used/Total)
> +  ----------------        ---------                    --------------
> +  MFF_METADATA         logical datapath                      24/64
> +  MFF_REG0             generic scratch                       32/32
> +  MFF_REG1             generic scratch                       32/32
> +  MFF_REG2             generic scratch                       32/32
> +  MFF_REG3             generic scratch                       32/32
> +  MFF_REG4             generic scratch                       32/32
> +  MFF_REG5             generic scratch                       32/32
> +  MFF_REG6             generic scratch                       32/32
> +  MFF_REG7             generic scratch                       32/32
> +  MFF_REG8             unused                                 0/32
> +  MFF_REG9             unused                                 0/32
> +  MFF_REG10            flags                                  1/32
> +  MFF_REG11            conntrack dnat zone gateway           16/32
> +  MFF_REG12            conntrack snat zone gateway           16/32
> +  MFF_REG13            conntrack zone logical ports          16/32
> +  MFF_REG14            logical input port                    15/32
> +  MFF_REG15            logical output port                   16/32
> +
> +```
> +
> +OVN Tunnel Encapsulations:
> +-------------------------
> +
> +tunnel key:
> +
> +When OVN encapsulates a packet in Geneve or another tunnel, it attaches
> +extra data to it to allow the receiving OVN instance to process it
> +correctly.  This takes different forms depending on the particular
> +encapsulation, but in each case we refer to it here as the 'tunnel key'.
> +
> +OVN supports three types of IP tunnel encapsulations used in the IP mesh
> +connecting HVs, physical switches, appliances and/or servers.  OVN passes
> +packet context information across tunnels.
> +
> +OVN annotates logical network packets that it sends from one hypervisor to
> +another with the following three pieces of metadata, which are encoded in
> an
> +encapsulation-specific fashion.
> +
> +
> +24-bit logical datapath identifier:
> +This is derived from the tunnel_key column in the OVN Southbound
> +Datapath_Binding table.
> +
> +15-bit logical ingress port identifier:
> +ID 0 is reserved for internal use within OVN.  IDs 1 through 32767,
> inclusive,
> +may be assigned to logical ports (see the tunnel_key column in the OVN
> +Southbound Port_Binding table).
> +
> +16-bit logical egress port identifier:
> +IDs 0 through 32767 have the same meaning as for logical ingress ports.
> IDs
> +32768 through 65535, inclusive, may be assigned to logical multicast
> groups
> +(see the tunnel_key column in the OVN Southbound Multicast_Group table).
> +
> +For hypervisor-to-hypervisor traffic, OVN supports only Geneve and STT
> +encapsulations, for the following reasons:
> +
> +Only STT and Geneve support the large amounts of metadata (over 32 bits
> per
> +packet) that OVN uses (as described above).
> +
> +STT and Geneve use randomized UDP or TCP source ports that allows
> efficient
>


... source ports that _allow_ efficient

To be fair/clear, this 'issue' was actually in the original location. ;)



> +distribution among multiple paths in environments that use ECMP in their
> +underlay.
> +
> +NICs are available to offload STT and Geneve encapsulation and
> decapsulation.
> +
> +Due to its flexibility, the preferred encapsulation between hypervisors is
> +Geneve.  For Geneve encapsulation, OVN transmits the logical datapath
> +identifier in the Geneve VNI.
> +
> +*Keep the following in sync with ovn/controller/physical.h.*
> +OVN transmits the logical ingress and logical egress ports in a TLV with
> +class 0x0102, type 0, and a 32-bit value encoded as follows, from MSB to
> +LSB:
> +
> +```
> +
> +      1          15             16
> +  -----------------------------------------
> +  |       |                |              |
> +  |  resv |  ingress port  |  egress port |
> +  |       |                |              |
> +  -----------------------------------------
> + ```
> +
> +Environments whose NICs lack Geneve offload may prefer STT encapsulation
> +for performance reasons.  For STT encapsulation, OVN encodes all three
> +pieces of logical metadata in the STT 64-bit tunnel ID as follows, from
> MSB
> +to LSB:
> +
> +```
> +
> +      9          15              16             24
> +  ------------------------------------------------------
> +  |       |                |              |            |
> +  |  resv |  ingress port  |  egress port |  datapath  |
> +  |       |                |              |            |
> +  ------------------------------------------------------
> +```
> +
> +For connecting to gateways, in addition to Geneve and STT, OVN supports
> +VXLAN, because only VXLAN support is common on top-of-rack (ToR) switches.
> +Currently, gateways have a feature set that matches the capabilities as
> +defined by the VTEP schema, so fewer bits of metadata are necessary.  In
> +the future, gateways that do not support encapsulations with large amounts
> +of metadata may continue to have a reduced feature set.
> +
> diff --git a/ovn/automake.mk b/ovn/automake.mk
> index f3f40e5..39277de 100644
> --- a/ovn/automake.mk
> +++ b/ovn/automake.mk
> @@ -73,6 +73,7 @@ DISTCLEANFILES += ovn/ovn-architecture.7
>  EXTRA_DIST += \
>         ovn/TODO \
>         ovn/CONTAINERS.OpenStack.md \
> +        ovn/OVN-DESIGN.md \
>         ovn/OVN-GW-HA.md
>
>  # Version checking for ovn-nb.ovsschema.
> diff --git a/ovn/ovn-architecture.7.xml b/ovn/ovn-architecture.7.xml
> index ead4feb..e023818 100644
> --- a/ovn/ovn-architecture.7.xml
> +++ b/ovn/ovn-architecture.7.xml
> @@ -609,100 +609,6 @@
>    </p>
>
>    <p>
> -    This section mentions several data and metadata fields, for clarity
> -    summarized here:
> -  </p>
> -
> -  <dl>
> -    <dt>tunnel key</dt>
> -    <dd>
> -      When OVN encapsulates a packet in Geneve or another tunnel, it
> attaches
> -      extra data to it to allow the receiving OVN instance to process it
> -      correctly.  This takes different forms depending on the particular
> -      encapsulation, but in each case we refer to it here as the ``tunnel
> -      key.''  See <code>Tunnel Encapsulations</code>, below, for details.
> -    </dd>
> -
> -    <dt>logical datapath field</dt>
> -    <dd>
> -      A field that denotes the logical datapath through which a packet is
> being
> -      processed.
> -      <!-- Keep the following in sync with MFF_LOG_DATAPATH in
> -           ovn/lib/logical-fields.h. -->
> -      OVN uses the field that OpenFlow 1.1+ simply (and confusingly) calls
> -      ``metadata'' to store the logical datapath.  (This field is passed
> across
> -      tunnels as part of the tunnel key.)
> -    </dd>
> -
> -    <dt>logical input port field</dt>
> -    <dd>
> -      <p>
> -        A field that denotes the logical port from which the packet
> -        entered the logical datapath.
> -        <!-- Keep the following in sync with MFF_LOG_INPORT in
> -             ovn/lib/logical-fields.h. -->
> -        OVN stores this in Nicira extension register number 14.
> -      </p>
> -
> -      <p>
> -        Geneve and STT tunnels pass this field as part of the tunnel key.
> -        Although VXLAN tunnels do not explicitly carry a logical input
> port,
> -        OVN only uses VXLAN to communicate with gateways that from OVN's
> -        perspective consist of only a single logical port, so that OVN
> can set
> -        the logical input port field to this one on ingress to the OVN
> logical
> -        pipeline.
> -      </p>
> -    </dd>
> -
> -    <dt>logical output port field</dt>
> -    <dd>
> -      <p>
> -        A field that denotes the logical port from which the packet will
> -        leave the logical datapath.  This is initialized to 0 at the
> -        beginning of the logical ingress pipeline.
> -        <!-- Keep the following in sync with MFF_LOG_OUTPORT in
> -             ovn/lib/logical-fields.h. -->
> -        OVN stores this in Nicira extension register number 15.
> -      </p>
> -
> -      <p>
> -        Geneve and STT tunnels pass this field as part of the tunnel key.
> -        VXLAN tunnels do not transmit the logical output port field.
> -      </p>
> -    </dd>
> -
> -    <dt>conntrack zone field for logical ports</dt>
> -    <dd>
> -      A field that denotes the connection tracking zone for logical ports.
> -      The value only has local significance and is not meaningful between
> -      chassis.  This is initialized to 0 at the beginning of the logical
> -        <!-- Keep the following in sync with MFF_LOG_CT_ZONE in
> -             ovn/lib/logical-fields.h. -->
> -      ingress pipeline.  OVN stores this in Nicira extension register
> -      number 13.
> -    </dd>
> -
> -    <dt>conntrack zone fields for Gateway router</dt>
> -    <dd>
> -      Fields that denote the connection tracking zones for Gateway
> routers.
> -      These values only have local significance (only on chassis that have
> -      Gateway routers instantiated) and is not meaningful between
> -      chassis.  OVN stores the zone information for DNATting in Nicira
> -        <!-- Keep the following in sync with MFF_LOG_DNAT_ZONE and
> -        MFF_LOG_SNAT_ZONE in ovn/lib/logical-fields.h. -->
> -      extension register number 11 and zone information for SNATing in
> Nicira
> -      extension register number 12.
> -    </dd>
> -
> -    <dt>VLAN ID</dt>
> -    <dd>
> -      The VLAN ID is used as an interface between OVN and containers
> nested
> -      inside a VM (see <code>Life Cycle of a container interface inside a
> -      VM</code>, above, for more information).
> -    </dd>
> -  </dl>
> -
> -  <p>
>      Initially, a VM or container on the ingress hypervisor sends a packet
> on a
>      port attached to the OVN integration bridge.  Then:
>    </p>
> @@ -1003,103 +909,4 @@
>        entries and the <code>Logical_Switch</code> tunnel keys.
>      </li>
>    </ol>
> -
> -  <h1>Design Decisions</h1>
>

Maybe leave a bread crumb here, just to point folks to OVN-DESIGN.md ?


> -
> -  <h2>Tunnel Encapsulations</h2>
> -
> -  <p>
> -    OVN annotates logical network packets that it sends from one
> hypervisor to
> -    another with the following three pieces of metadata, which are
> encoded in
> -    an encapsulation-specific fashion:
> -  </p>
> -
> -  <ul>
> -    <li>
> -      24-bit logical datapath identifier, from the <code>tunnel_key</code>
> -      column in the OVN Southbound <code>Datapath_Binding</code> table.
> -    </li>
> -
> -    <li>
> -      15-bit logical ingress port identifier.  ID 0 is reserved for
> internal
> -      use within OVN.  IDs 1 through 32767, inclusive, may be assigned to
> -      logical ports (see the <code>tunnel_key</code> column in the OVN
> -      Southbound <code>Port_Binding</code> table).
> -    </li>
> -
> -    <li>
> -      16-bit logical egress port identifier.  IDs 0 through 32767 have
> the same
> -      meaning as for logical ingress ports.  IDs 32768 through 65535,
> -      inclusive, may be assigned to logical multicast groups (see the
> -      <code>tunnel_key</code> column in the OVN Southbound
> -      <code>Multicast_Group</code> table).
> -    </li>
> -  </ul>
> -
> -  <p>
> -    For hypervisor-to-hypervisor traffic, OVN supports only Geneve and STT
> -    encapsulations, for the following reasons:
> -  </p>
> -
> -  <ul>
> -    <li>
> -      Only STT and Geneve support the large amounts of metadata (over 32
> bits
> -      per packet) that OVN uses (as described above).
> -    </li>
> -
> -    <li>
> -      STT and Geneve use randomized UDP or TCP source ports that allows
> -      efficient distribution among multiple paths in environments that
> use ECMP
> -      in their underlay.
> -    </li>
> -
> -    <li>
> -      NICs are available to offload STT and Geneve encapsulation and
> -      decapsulation.
> -    </li>
> -  </ul>
> -
> -  <p>
> -    Due to its flexibility, the preferred encapsulation between
> hypervisors is
> -    Geneve.  For Geneve encapsulation, OVN transmits the logical datapath
> -    identifier in the Geneve VNI.
> -
> -    <!-- Keep the following in sync with ovn/controller/physical.h. -->
> -    OVN transmits the logical ingress and logical egress ports in a TLV
> with
> -    class 0x0102, type 0, and a 32-bit value encoded as follows, from MSB
> to
> -    LSB:
> -  </p>
> -
> -  <diagram>
> -    <header name="">
> -      <bits name="rsv" above="1" below="0" width=".25"/>
> -      <bits name="ingress port" above="15" width=".75"/>
> -      <bits name="egress port" above="16" width=".75"/>
> -    </header>
> -  </diagram>
> -
> -  <p>
> -    Environments whose NICs lack Geneve offload may prefer STT
> encapsulation
> -    for performance reasons.  For STT encapsulation, OVN encodes all three
> -    pieces of logical metadata in the STT 64-bit tunnel ID as follows,
> from MSB
> -    to LSB:
> -  </p>
> -
> -  <diagram>
> -    <header name="">
> -      <bits name="reserved" above="9" below="0" width=".5"/>
> -      <bits name="ingress port" above="15" width=".75"/>
> -      <bits name="egress port" above="16" width=".75"/>
> -      <bits name="datapath" above="24" width="1.25"/>
> -    </header>
> -  </diagram>
> -
> -  <p>
> -    For connecting to gateways, in addition to Geneve and STT, OVN
> supports
> -    VXLAN, because only VXLAN support is common on top-of-rack (ToR)
> switches.
> -    Currently, gateways have a feature set that matches the capabilities
> as
> -    defined by the VTEP schema, so fewer bits of metadata are necessary.
> In
> -    the future, gateways that do not support encapsulations with large
> amounts
> -    of metadata may continue to have a reduced feature set.
> -  </p>
>  </manpage>
> --
> 1.9.1
>
> _______________________________________________
> dev mailing list
> dev@openvswitch.org
> http://openvswitch.org/mailman/listinfo/dev
>

Patch

diff --git a/ovn/OVN-DESIGN.md b/ovn/OVN-DESIGN.md
new file mode 100644
index 0000000..2162c65
--- /dev/null
+++ b/ovn/OVN-DESIGN.md
@@ -0,0 +1,199 @@ 
+OVN register and metadata usage:
+-------------------------------
+
+logical datapath field:
+
+A field that denotes the logical datapath through which a packet is being
+processed.
+*Keep the following in sync with MFF_LOG_DATAPATH in*
+*ovn/lib/logical-fields.h.*
+OVN uses the field that OpenFlow 1.1+ simply (and confusingly) calls
+'metadata' to store the logical datapath.  (This field is passed across
+tunnels as part of the tunnel key.)
+
+
+logical input port field:
+
+A field that denotes the logical port from which the packet entered the
+logical datapath.
+*Keep the following in sync with MFF_LOG_INPORT in*
+*ovn/lib/logical-fields.h.*
+OVN stores this in Nicira extension register number 14.
+
+Geneve and STT tunnels pass this field as part of the tunnel key.
+Although VXLAN tunnels do not explicitly carry a logical input port,
+OVN only uses VXLAN to communicate with gateways that from OVN's
+perspective consist of only a single logical port, so that OVN can set
+the logical input port field to this one on ingress to the OVN logical
+pipeline.
+
+
+logical output port field:
+
+A field that denotes the logical port from which the packet will leave
+the logical datapath.  This is initialized to 0 at the beginning of the
+logical ingress pipeline.
+*Keep the following in sync with MFF_LOG_OUTPORT in*
+*ovn/lib/logical-fields.h.*
+OVN stores this in Nicira extension register number 15.
+
+Geneve and STT tunnels pass this field as part of the tunnel key.  VXLAN
+tunnels do not transmit the logical output port field.
+
+
+conntrack zone field for logical ports:
+
+A field that denotes the connection tracking zone for logical ports.
+The value only has local significance and is not meaningful between
+chassis.  This is initialized to 0 at the beginning of the logical
+ingress pipeline.
+*Keep the following in sync with MFF_LOG_CT_ZONE in*
+*ovn/lib/logical-fields.h.*
+OVN stores this in Nicira extension register number 13.
+
+
+conntrack zone fields for Gateway router:
+
+Fields that denote the connection tracking zones for Gateway routers.
+These values only have local significance (only on chassis that have
+Gateway routers instantiated) and is not meaningful between
+chassis.
+*Keep the following in sync with MFF_LOG_DNAT_ZONE and* 
+*MFF_LOG_DNAT_ZONE inovn/lib/logical-fields.h.*
+OVN stores the zone information for DNATting in Nicira
+extension register number 11 and zone information for SNATing in Nicira
+extension register number 12.
+
+
+flags field:
+
+Scratchpad flags that denote the pipeline state between tables.  The
+values only have local significance and are not meaningful between
+chassis.  This is initialized to 0 at the beginning of the logical
+ingress pipeline. 
+*Keep the following in sync with MFF_LOG_FLAGS in*
+*ovn/lib/logical-fields.h.*
+OVN stores this in Nicira extension register number 10.
+
+
+VLAN ID:
+
+The VLAN ID is used as an interface between OVN and containers nested
+inside a VM (see Life Cycle of a container interface inside a VM, in 
+ovn-architecture.7.xml, for more information).
+
+
+The following table summarizes the register and metadata usage for OVN.
+*Keep the following in sync with MFF_LOG_FLAGS in*
+*ovn/lib/logical-fields.h.*:
+
+```
+
+  Register/Metadata         Usage                      Bits (Used/Total)
+  ----------------        ---------                    --------------
+  MFF_METADATA         logical datapath                      24/64
+  MFF_REG0             generic scratch                       32/32
+  MFF_REG1             generic scratch                       32/32
+  MFF_REG2             generic scratch                       32/32
+  MFF_REG3             generic scratch                       32/32
+  MFF_REG4             generic scratch                       32/32
+  MFF_REG5             generic scratch                       32/32
+  MFF_REG6             generic scratch                       32/32
+  MFF_REG7             generic scratch                       32/32
+  MFF_REG8             unused                                 0/32
+  MFF_REG9             unused                                 0/32
+  MFF_REG10            flags                                  1/32
+  MFF_REG11            conntrack dnat zone gateway           16/32
+  MFF_REG12            conntrack snat zone gateway           16/32
+  MFF_REG13            conntrack zone logical ports          16/32
+  MFF_REG14            logical input port                    15/32
+  MFF_REG15            logical output port                   16/32
+
+```
+
+OVN Tunnel Encapsulations:
+-------------------------
+
+tunnel key:
+
+When OVN encapsulates a packet in Geneve or another tunnel, it attaches
+extra data to it to allow the receiving OVN instance to process it
+correctly.  This takes different forms depending on the particular
+encapsulation, but in each case we refer to it here as the 'tunnel key'.
+
+OVN supports three types of IP tunnel encapsulations used in the IP mesh
+connecting HVs, physical switches, appliances and/or servers.  OVN passes
+packet context information across tunnels.
+
+OVN annotates logical network packets that it sends from one hypervisor to
+another with the following three pieces of metadata, which are encoded in an
+encapsulation-specific fashion.
+
+
+24-bit logical datapath identifier:
+This is derived from the tunnel_key column in the OVN Southbound
+Datapath_Binding table.
+ 
+15-bit logical ingress port identifier:
+ID 0 is reserved for internal use within OVN.  IDs 1 through 32767, inclusive,
+may be assigned to logical ports (see the tunnel_key column in the OVN
+Southbound Port_Binding table).
+
+16-bit logical egress port identifier:
+IDs 0 through 32767 have the same meaning as for logical ingress ports. IDs
+32768 through 65535, inclusive, may be assigned to logical multicast groups
+(see the tunnel_key column in the OVN Southbound Multicast_Group table).
+
+For hypervisor-to-hypervisor traffic, OVN supports only Geneve and STT
+encapsulations, for the following reasons:
+
+Only STT and Geneve support the large amounts of metadata (over 32 bits per
+packet) that OVN uses (as described above).
+
+STT and Geneve use randomized UDP or TCP source ports that allows efficient
+distribution among multiple paths in environments that use ECMP in their
+underlay.
+
+NICs are available to offload STT and Geneve encapsulation and decapsulation.
+ 
+Due to its flexibility, the preferred encapsulation between hypervisors is
+Geneve.  For Geneve encapsulation, OVN transmits the logical datapath
+identifier in the Geneve VNI.
+
+*Keep the following in sync with ovn/controller/physical.h.*
+OVN transmits the logical ingress and logical egress ports in a TLV with
+class 0x0102, type 0, and a 32-bit value encoded as follows, from MSB to
+LSB:
+
+```
+
+      1          15             16
+  -----------------------------------------
+  |       |                |              |
+  |  resv |  ingress port  |  egress port |
+  |       |                |              |
+  -----------------------------------------
+ ``` 
+
+Environments whose NICs lack Geneve offload may prefer STT encapsulation
+for performance reasons.  For STT encapsulation, OVN encodes all three
+pieces of logical metadata in the STT 64-bit tunnel ID as follows, from MSB
+to LSB:
+
+```
+
+      9          15              16             24
+  ------------------------------------------------------
+  |       |                |              |            |
+  |  resv |  ingress port  |  egress port |  datapath  |
+  |       |                |              |            |
+  ------------------------------------------------------
+```
+
+For connecting to gateways, in addition to Geneve and STT, OVN supports
+VXLAN, because only VXLAN support is common on top-of-rack (ToR) switches.
+Currently, gateways have a feature set that matches the capabilities as
+defined by the VTEP schema, so fewer bits of metadata are necessary.  In
+the future, gateways that do not support encapsulations with large amounts
+of metadata may continue to have a reduced feature set.
+
diff --git a/ovn/automake.mk b/ovn/automake.mk
index f3f40e5..39277de 100644
--- a/ovn/automake.mk
+++ b/ovn/automake.mk
@@ -73,6 +73,7 @@  DISTCLEANFILES += ovn/ovn-architecture.7
 EXTRA_DIST += \
 	ovn/TODO \
 	ovn/CONTAINERS.OpenStack.md \
+        ovn/OVN-DESIGN.md \
 	ovn/OVN-GW-HA.md
 
 # Version checking for ovn-nb.ovsschema.
diff --git a/ovn/ovn-architecture.7.xml b/ovn/ovn-architecture.7.xml
index ead4feb..e023818 100644
--- a/ovn/ovn-architecture.7.xml
+++ b/ovn/ovn-architecture.7.xml
@@ -609,100 +609,6 @@ 
   </p>
 
   <p>
-    This section mentions several data and metadata fields, for clarity
-    summarized here:
-  </p>
-
-  <dl>
-    <dt>tunnel key</dt>
-    <dd>
-      When OVN encapsulates a packet in Geneve or another tunnel, it attaches
-      extra data to it to allow the receiving OVN instance to process it
-      correctly.  This takes different forms depending on the particular
-      encapsulation, but in each case we refer to it here as the ``tunnel
-      key.''  See <code>Tunnel Encapsulations</code>, below, for details.
-    </dd>
-
-    <dt>logical datapath field</dt>
-    <dd>
-      A field that denotes the logical datapath through which a packet is being
-      processed.
-      <!-- Keep the following in sync with MFF_LOG_DATAPATH in
-           ovn/lib/logical-fields.h. -->
-      OVN uses the field that OpenFlow 1.1+ simply (and confusingly) calls
-      ``metadata'' to store the logical datapath.  (This field is passed across
-      tunnels as part of the tunnel key.)
-    </dd>
-
-    <dt>logical input port field</dt>
-    <dd>
-      <p>
-        A field that denotes the logical port from which the packet
-        entered the logical datapath.
-        <!-- Keep the following in sync with MFF_LOG_INPORT in
-             ovn/lib/logical-fields.h. -->
-        OVN stores this in Nicira extension register number 14.
-      </p>
-
-      <p>
-        Geneve and STT tunnels pass this field as part of the tunnel key.
-        Although VXLAN tunnels do not explicitly carry a logical input port,
-        OVN only uses VXLAN to communicate with gateways that from OVN's
-        perspective consist of only a single logical port, so that OVN can set
-        the logical input port field to this one on ingress to the OVN logical
-        pipeline.
-      </p>
-    </dd>
-
-    <dt>logical output port field</dt>
-    <dd>
-      <p>
-        A field that denotes the logical port from which the packet will
-        leave the logical datapath.  This is initialized to 0 at the
-        beginning of the logical ingress pipeline.
-        <!-- Keep the following in sync with MFF_LOG_OUTPORT in
-             ovn/lib/logical-fields.h. -->
-        OVN stores this in Nicira extension register number 15.
-      </p>
-
-      <p>
-        Geneve and STT tunnels pass this field as part of the tunnel key.
-        VXLAN tunnels do not transmit the logical output port field.
-      </p>
-    </dd>
-
-    <dt>conntrack zone field for logical ports</dt>
-    <dd>
-      A field that denotes the connection tracking zone for logical ports.
-      The value only has local significance and is not meaningful between
-      chassis.  This is initialized to 0 at the beginning of the logical
-        <!-- Keep the following in sync with MFF_LOG_CT_ZONE in
-             ovn/lib/logical-fields.h. -->
-      ingress pipeline.  OVN stores this in Nicira extension register
-      number 13.
-    </dd>
-
-    <dt>conntrack zone fields for Gateway router</dt>
-    <dd>
-      Fields that denote the connection tracking zones for Gateway routers.
-      These values only have local significance (only on chassis that have
-      Gateway routers instantiated) and is not meaningful between
-      chassis.  OVN stores the zone information for DNATting in Nicira
-        <!-- Keep the following in sync with MFF_LOG_DNAT_ZONE and
-        MFF_LOG_SNAT_ZONE in ovn/lib/logical-fields.h. -->
-      extension register number 11 and zone information for SNATing in Nicira
-      extension register number 12.
-    </dd>
-
-    <dt>VLAN ID</dt>
-    <dd>
-      The VLAN ID is used as an interface between OVN and containers nested
-      inside a VM (see <code>Life Cycle of a container interface inside a
-      VM</code>, above, for more information).
-    </dd>
-  </dl>
-
-  <p>
     Initially, a VM or container on the ingress hypervisor sends a packet on a
     port attached to the OVN integration bridge.  Then:
   </p>
@@ -1003,103 +909,4 @@ 
       entries and the <code>Logical_Switch</code> tunnel keys.
     </li>
   </ol>
-
-  <h1>Design Decisions</h1>
-
-  <h2>Tunnel Encapsulations</h2>
-
-  <p>
-    OVN annotates logical network packets that it sends from one hypervisor to
-    another with the following three pieces of metadata, which are encoded in
-    an encapsulation-specific fashion:
-  </p>
-
-  <ul>
-    <li>
-      24-bit logical datapath identifier, from the <code>tunnel_key</code>
-      column in the OVN Southbound <code>Datapath_Binding</code> table.
-    </li>
-
-    <li>
-      15-bit logical ingress port identifier.  ID 0 is reserved for internal
-      use within OVN.  IDs 1 through 32767, inclusive, may be assigned to
-      logical ports (see the <code>tunnel_key</code> column in the OVN
-      Southbound <code>Port_Binding</code> table).
-    </li>
-
-    <li>
-      16-bit logical egress port identifier.  IDs 0 through 32767 have the same
-      meaning as for logical ingress ports.  IDs 32768 through 65535,
-      inclusive, may be assigned to logical multicast groups (see the
-      <code>tunnel_key</code> column in the OVN Southbound
-      <code>Multicast_Group</code> table).
-    </li>
-  </ul>
-
-  <p>
-    For hypervisor-to-hypervisor traffic, OVN supports only Geneve and STT
-    encapsulations, for the following reasons:
-  </p>
-
-  <ul>
-    <li>
-      Only STT and Geneve support the large amounts of metadata (over 32 bits
-      per packet) that OVN uses (as described above).
-    </li>
-
-    <li>
-      STT and Geneve use randomized UDP or TCP source ports that allows
-      efficient distribution among multiple paths in environments that use ECMP
-      in their underlay.
-    </li>
-
-    <li>
-      NICs are available to offload STT and Geneve encapsulation and
-      decapsulation.
-    </li>
-  </ul>
-
-  <p>
-    Due to its flexibility, the preferred encapsulation between hypervisors is
-    Geneve.  For Geneve encapsulation, OVN transmits the logical datapath
-    identifier in the Geneve VNI.
-
-    <!-- Keep the following in sync with ovn/controller/physical.h. -->
-    OVN transmits the logical ingress and logical egress ports in a TLV with
-    class 0x0102, type 0, and a 32-bit value encoded as follows, from MSB to
-    LSB:
-  </p>
-
-  <diagram>
-    <header name="">
-      <bits name="rsv" above="1" below="0" width=".25"/>
-      <bits name="ingress port" above="15" width=".75"/>
-      <bits name="egress port" above="16" width=".75"/>
-    </header>
-  </diagram>
-
-  <p>
-    Environments whose NICs lack Geneve offload may prefer STT encapsulation
-    for performance reasons.  For STT encapsulation, OVN encodes all three
-    pieces of logical metadata in the STT 64-bit tunnel ID as follows, from MSB
-    to LSB:
-  </p>
-
-  <diagram>
-    <header name="">
-      <bits name="reserved" above="9" below="0" width=".5"/>
-      <bits name="ingress port" above="15" width=".75"/>
-      <bits name="egress port" above="16" width=".75"/>
-      <bits name="datapath" above="24" width="1.25"/>
-    </header>
-  </diagram>
-
-  <p>
-    For connecting to gateways, in addition to Geneve and STT, OVN supports
-    VXLAN, because only VXLAN support is common on top-of-rack (ToR) switches.
-    Currently, gateways have a feature set that matches the capabilities as
-    defined by the VTEP schema, so fewer bits of metadata are necessary.  In
-    the future, gateways that do not support encapsulations with large amounts
-    of metadata may continue to have a reduced feature set.
-  </p>
 </manpage>