From patchwork Wed Mar 18 18:45:44 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ben Pfaff X-Patchwork-Id: 1257711 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.136; helo=silver.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=ovn.org Received: from silver.osuosl.org (smtp3.osuosl.org [140.211.166.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 48jJsK0rX0z9sR4 for ; Thu, 19 Mar 2020 05:46:13 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by silver.osuosl.org (Postfix) with ESMTP id 984E326866; Wed, 18 Mar 2020 18:46:11 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from silver.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id fKwHYexMJrI7; Wed, 18 Mar 2020 18:46:05 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by silver.osuosl.org (Postfix) with ESMTP id 6D52E26963; Wed, 18 Mar 2020 18:46:05 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 540EEC18DA; Wed, 18 Mar 2020 18:46:05 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from fraxinus.osuosl.org (smtp4.osuosl.org [140.211.166.137]) by lists.linuxfoundation.org (Postfix) with ESMTP id 48621C1D90 for ; Wed, 18 Mar 2020 18:46:04 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by fraxinus.osuosl.org (Postfix) with ESMTP id 439E4866C8 for ; Wed, 18 Mar 2020 18:46:04 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from fraxinus.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id pt1zDPpq5lSU for ; Wed, 18 Mar 2020 18:46:03 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from relay3-d.mail.gandi.net (relay3-d.mail.gandi.net [217.70.183.195]) by fraxinus.osuosl.org (Postfix) with ESMTPS id C90688654C for ; Wed, 18 Mar 2020 18:46:02 +0000 (UTC) X-Originating-IP: 75.54.222.30 Received: from sigfpe.attlocal.net (75-54-222-30.lightspeed.rdcyca.sbcglobal.net [75.54.222.30]) (Authenticated sender: blp@ovn.org) by relay3-d.mail.gandi.net (Postfix) with ESMTPSA id 37A2D60002; Wed, 18 Mar 2020 18:45:58 +0000 (UTC) From: Ben Pfaff To: dev@openvswitch.org Date: Wed, 18 Mar 2020 11:45:44 -0700 Message-Id: <20200318184548.3000357-4-blp@ovn.org> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200318184548.3000357-1-blp@ovn.org> References: <20200318184548.3000357-1-blp@ovn.org> MIME-Version: 1.0 Cc: Ben Pfaff Subject: [ovs-dev] [PATCH ovn v2 3/7] ovn-nb: Better document distributed gateway ports. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" The documentation was scattered across two places. This brings it together. It also documents the two solutions for physical VLAN MTU issues in a unified way. Signed-off-by: Ben Pfaff --- ovn-architecture.7.xml | 110 +++++++++++++++++++++++++-- ovn-nb.xml | 166 ++++++++++++++++++++--------------------- 2 files changed, 181 insertions(+), 95 deletions(-) diff --git a/ovn-architecture.7.xml b/ovn-architecture.7.xml index d32978a9d5be..94a897fa693e 100644 --- a/ovn-architecture.7.xml +++ b/ovn-architecture.7.xml @@ -688,6 +688,100 @@ highest-priority gateway that is online.

+

Physical VLAN MTU Issues

+ +

+ Consider the preceding diagram again: +

+ + 
+                                LSlocal
+                                   |
+                                  LR1
+                                   |
+                              +----+----+
+                              |    |    |
+                             LS1  ...  LSn
+
+ +

+ Suppose that each logical switch LS1, ..., LSn is bridged to a physical + VLAN-tagged network attached to a localnet port on LSlocal, + over a distributed gateway port on LR1. If a packet originating on + LSi is destined to the external network, OVN sends it to the + gateway chassis over a tunnel. There, the packet traverses LR1's logical + router pipeline, possibly undergoes NAT, and eventually ends up at + LSlocal's localnet port. If all of the physical links in the + network have the same MTU, then the packet's transit across a tunnel causes + an MTU problem: tunnel overhead prevents a packet that uses the full + physical MTU from crossing the tunnel to the gateway chassis (without + fragmentation). +

+ +

+ OVN offers two solutions to this problem, the + reside-on-redirect-chassis and redirect-type + options. Both solutions require each logical switch LS1, ..., LSn to + include a localnet logical switch port LN1, ..., LNn + respectively, that is present on each chassis. Both cause packets to be + sent over the localnet ports instead of tunnels. They differ + in which packets--some or all--are sent this way. The most prominent + tradeoff between these options is that + reside-on-redirect-chassis is easier to configure and that + redirect-type performs better for east-west traffic. +

+ +

+ The first solution is the reside-on-redirect-chassis option + for logical router ports. Setting this option on a LRP from (e.g.) LS1 to + LR1 disables forwarding from LS1 to LR1 except on the gateway chassis. On + chassis other than the gateway chassis, this single change means that + packets that would otherwise have been forwarded to LR1 are instead + forwarded to LN1. The instance of LN1 on the gateway chassis then receives + the packet and forwards it to LR1. The packet traverses the LR1 logical + router pipeline, possibly undergoes NAT, and eventually ends up at + LSlocal's localnet port. The packet never traverses a tunnel, + avoiding the MTU issue. +

+ +

+ This option has the further consequence of centralizing ``distributed'' + logical router LR1, since no packets are forwarded from LS1 to LR1 on any + chassis other than the gateway chassis. Therefore, east-west traffic + passes through the gateway chassis, not just north-south. (The naive + ``fix'' of allowing east-west traffic to flow directly between chassis over + LN1 does not work because routing sets the Ethernet source address to LR1's + source address. Seeing this single Ethernet source address originate from + all of the chassis will confuse the physical switch.) +

+ +

+ Do not set the reside-on-redirect-chassis option on a + distributed gateway port. In the diagram above, it would be set on the + LRPs connecting LS1, ..., LSn to LR1. +

+ +

+ The second solution is the redirect-chassis option for + distributed gateway ports. Setting this option to bridged + causes packets that are redirected to the gateway chassis to go over the + localnet ports instead of being tunneled. This option does + not change how OVN treats packets not redirected to the gateway chassis. +

+ +

+ The redirect-chassis option requires the administrator or the + CMS to configure each participating chassis with a unique Ethernet address + for the locgical router by setting ovn-chassis-mac-mappings in + the Open vSwitch database, for use by ovn-controller. This + makes it more difficult to configure than + reside-on-redirect-chassis. +

+ +

+ Set the redirect-chassis option on a distributed gateway port. +

+

Life Cycle of a VIF

@@ -1892,14 +1986,14 @@

- VLAN-based redirection: - - As an enhancement to reside-on-redirect-chassis we support - VLAN-based redirection as well. By setting - options:redirect-type to bridged on a gateway - chassis attached router port, user can enforce that redirected packet - should not use tunnel port but rather use localnet port of peer logical - switch to go out on a physical VLAN. + As an alternative to reside-on-redirect-chassis, OVN supports + VLAN-based redirection. Whereas reside-on-redirect-chassis + centralizes all router functionality, VLAN-based redirection only changes + how OVN redirects packets to the gateway chassis. By setting + options:redirect-type to bridged on a distributed + gateway port, OVN redirects packets to the gateway chassis using the + localnet port of the router's peer logical switch, instead of + a tunnel.

diff --git a/ovn-nb.xml b/ovn-nb.xml index ccd9bae991af..7f142bd35031 100644 --- a/ovn-nb.xml +++ b/ovn-nb.xml @@ -2038,8 +2038,9 @@

If any of these are set, this logical router port represents a distributed gateway port that connects this router to a - logical switch with a localnet port. There may be at most one - such logical router port on each logical router. + logical switch with a localnet port. There may + be at most one such logical router port on each logical + router.

@@ -2096,6 +2097,82 @@ Designates the named chassis as the gateway. + + +

+ MTU issues arise in mixing tunnels with logical networks that are + bridged to a physical VLAN. For an explanation of the MTU issues, + see Physical VLAN MTU Issues in the OVN architecture + document. The following options, which are alternatives, provide + solutions. Both of them cause packets to be sent over + localnet instead of tunnels, but they differ in whether + some or all packets are sent this way. The most prominent + tradeoff between these options is that + reside-on-redirect-chassis is easier to configure and + that redirect-type performs better for east-west + traffic. +

+ + +

+ If set to true, this option forces all traffic across + the logical router port to pass through the gateway chassis using a + hop across a localnet port. This changes behavior in + two ways: +

+ +
    +
  • + Without this option, east-west traffic passes directly between + source and destination chassis (or even within a single chassis, + for co-located VMs). With this option, all east-west traffic + passes through the gateway chassis. +
    • + +
  • + Without this option, traffic between the gateway chassis and + other chassis is encapsulated in tunnels. With this option, + traffic passes over a localnet interface. +
    • +
+ +

+ This option may usefully be set only on logical router ports that + connect a distributed logical router to a logical switch with VIFs. + It should not be set on a distributed gateway port. +

+ +

+ OVN honors this option only if the logical router has a distributed + gateway port and if the LRP's peer switch has a + localnet port. +

+ + + +

+ If set to bridged on a distributed gateway port, this + option causes OVN to redirect packets to the gateway chassis over a + localnet port instead of a tunnel. The relevant + chassis must share a localnet port. +

+ +

+ This feature requires the administrator or the CMS to configure + each participating chassis with a unique Ethernet address for the + locgical router by setting ovn-chassis-mac-mappings in + the Open vSwitch database, for use by ovn-controller. +

+ +

+ Setting this option to overlay or leaving it unset has + no effect. This option may usefully be set only on a distributed + gateway port. It is otherwise ignored. +

+ + @@ -2198,91 +2275,6 @@ Additional options for the logical router port.

- -

- Generally routing is distributed in OVN. The packet - from a logical port which needs to be routed hits the router pipeline - in the source chassis. For the East-West traffic, the packet is - sent directly to the destination chassis. For the outside traffic - the packet is sent to the gateway chassis. -

- -

- When this option is set, OVN considers this only if -

- -
    -
  • - The logical router to which this logical router port belongs to - has a distributed gateway port. -
    • - -
  • - The peer's logical switch has a localnet port (representing - a VLAN tagged network) -
    • -
- -

- When this option is set to true, then the packet - which needs to be routed hits the router pipeline in the chassis - hosting the distributed gateway router port. The source chassis - pushes out this traffic via the localnet port. With this the - East-West traffic is no more distributed and will always go through - the gateway chassis. -

- -

- Without this option set, for any traffic destined to outside from a - logical port which belongs to a logical switch with localnet port, - the source chassis will send the traffic to the gateway chassis via - the tunnel port instead of the localnet port and this could cause MTU - issues. -

- - - -

- This options dictates if a packet redirected to - gateway chassis will be overlay encapsulated - or go as a regular packet via the localnet port. -

- -

- Option takes following values -

- -
    -
  • - OVERLAY -
    • - -
  • - BRIDGED -
    • -
- -

- OVERLAY option will ensure that redirected packet goes out as - encapsulation via the tunnel port. -

- -

- BRIDGED option will ensure that redirected packet goes out - via the localnet port tagged with vlan (if configured). -

- -

- OVERLAY is the default redirection type. -

- -

- Option is applicable only to gateway chassis attached logical - router ports. -

- - -