From patchwork Fri Oct 16 09:32:10 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: "Chandran, Sugesh" X-Patchwork-Id: 531171 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from archives.nicira.com (unknown [IPv6:2600:3c00::f03c:91ff:fe6e:bdf7]) by ozlabs.org (Postfix) with ESMTP id 49E2B14012C for ; Fri, 16 Oct 2015 20:32:39 +1100 (AEDT) Received: from archives.nicira.com (localhost [127.0.0.1]) by archives.nicira.com (Postfix) with ESMTP id 9DC2810C83; Fri, 16 Oct 2015 02:32:37 -0700 (PDT) X-Original-To: dev@openvswitch.org Delivered-To: dev@openvswitch.org Received: from mx3v1.cudamail.com (mx3.cudamail.com [64.34.241.5]) by archives.nicira.com (Postfix) with ESMTPS id 9E3BA10C82 for ; Fri, 16 Oct 2015 02:32:36 -0700 (PDT) Received: from bar3.cudamail.com (bar1 [192.168.15.1]) by mx3v1.cudamail.com (Postfix) with ESMTP id 03D5661814F for ; Fri, 16 Oct 2015 03:32:35 -0600 (MDT) X-ASG-Debug-ID: 1444987954-03dd7b106c418c0001-byXFYA Received: from mx3-pf3.cudamail.com ([192.168.14.3]) by bar3.cudamail.com with ESMTP id 64ljPIxXJm9ZcxvH (version=TLSv1 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Fri, 16 Oct 2015 03:32:34 -0600 (MDT) X-Barracuda-Envelope-From: sugesh.chandran@intel.com X-Barracuda-RBL-Trusted-Forwarder: 192.168.14.3 Received: from unknown (HELO mga14.intel.com) (192.55.52.115) by mx3-pf3.cudamail.com with SMTP; 16 Oct 2015 09:32:28 -0000 Received-SPF: pass (mx3-pf3.cudamail.com: SPF record at intel.com designates 192.55.52.115 as permitted sender) X-Barracuda-Apparent-Source-IP: 192.55.52.115 X-Barracuda-RBL-IP: 192.55.52.115 Received: from orsmga002.jf.intel.com ([10.7.209.21]) by fmsmga103.fm.intel.com with ESMTP; 16 Oct 2015 02:32:27 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.17,688,1437462000"; d="scan'208";a="828121652" Received: from sugeshch-mobl.ger.corp.intel.com (HELO sugesh-VirtualBox.ir.intel.com) ([10.243.20.19]) by orsmga002.jf.intel.com with ESMTP; 16 Oct 2015 02:32:26 -0700 X-CudaMail-Envelope-Sender: sugesh.chandran@intel.com From: Sugesh Chandran To: dev@openvswitch.org X-CudaMail-MID: CM-V3-1015003675 X-CudaMail-DTE: 101615 X-CudaMail-Originating-IP: 192.55.52.115 Date: Fri, 16 Oct 2015 10:32:10 +0100 X-ASG-Orig-Subj: [##CM-V3-1015003675##][PATCH] Detailed documentation for configuring native userspace-tunneling in OVS with/without DPDK. Message-Id: <1444987930-4120-1-git-send-email-sugesh.chandran@intel.com> X-Mailer: git-send-email 2.1.4 MIME-Version: 1.0 X-GBUdb-Analysis: 0, 192.55.52.115, Ugly c=0 p=0 Source New X-MessageSniffer-Rules: 0-0-0-19057-c X-Barracuda-Connect: UNKNOWN[192.168.14.3] X-Barracuda-Start-Time: 1444987954 X-Barracuda-Encrypted: DHE-RSA-AES256-SHA X-Barracuda-URL: https://web.cudamail.com:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at cudamail.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.60 X-Barracuda-Spam-Status: No, SCORE=0.60 using per-user scores of TAG_LEVEL=3.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=3.0 tests=BSF_SC5_MJ1963, RDNS_NONE X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.23541 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.10 RDNS_NONE Delivered to trusted network by a host with no rDNS 0.50 BSF_SC5_MJ1963 Custom Rule MJ1963 Subject: [ovs-dev] [PATCH] Detailed documentation for configuring native userspace-tunneling in OVS with/without DPDK. X-BeenThere: dev@openvswitch.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@openvswitch.org Sender: "dev" Adding a self-guide for configuring native userspace tunneling in OVS with/without DPDK ports. This document also provides necessary debugging commands to identify and resolve the userspace tunneling issues in OVS. Signed-off-by: Sugesh Chandran --- README-native-tunneling-DPDK.md | 206 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 206 insertions(+) create mode 100644 README-native-tunneling-DPDK.md diff --git a/README-native-tunneling-DPDK.md b/README-native-tunneling-DPDK.md new file mode 100644 index 0000000..47609b5 --- /dev/null +++ b/README-native-tunneling-DPDK.md @@ -0,0 +1,206 @@ +Openvswitch Native tunneling configuration guide +======================================================= + +This guide is for configuring userspace tunneling in Open-vSwitch. The +traditional OVS with kernel datapath uses kernel module to perform the tunneling +however this setup performs all the tunneling operations purely in the +userspace. This way userspace-tunnelling is platform independent. + +This setup needs an additional bridge called “br-phy1” when compared to the +kernel based OVS. The purpose of this bridge is to make available the kernel +network stack for routing and arp resolution. The data path needs to look-up +the routing table and arp table to prepare the tunnel header and xmit data to +the output port. + +The tunneling setup is found as below: + + +--------------+ + | VM-0 | 192.168.1.1/24 + +--------------+ + (vm_port0) + | + | + | + | + | + +--------------+ + | br-int | 192.168.1.2/24 + +--------------+ +--------------+ + | vxlan0 | | vxlan0 | + +--------------+ +--------------+ + | | + | | + | | + | | + | | + | | + 172.168.1.1/24 | + +--------------+ | + | br-phy1 | 172.168.1.2/24 + +--------------+ +---------------+ + | dpdk0/eth1 |==========================| eth1 | + +--------------+ +---------------+ + Host A with OVS. Remote host. + +#### Prerequisites +The host must be pre-installed with OVS-DPDK, QEMU and virtual machine images. +Please refer the installation guide of relevant modules for these instructions. +ovs-vswitchd and OVSDB server must be up and running before proceeding to any +of the configuration steps. + +This setup guide covers only the required steps for setting up VxLAN userspace +tunneling. The same approach can be used for any other tunneling protocols, by +specifying the appropriate tunneling protocol type. + +Note:- This configuration guide is for setting up the VxLAN tunneling on one +host(local host). The same steps have to perform on the remote host as well for +setting up a VM<->VM VxLAN tunnel setup. The only difference for setting up the +userspace-tunneling in remote node is , VM and VxLAN tunnel ip addresses are +different. + +#### Configuration steps +1. Create the “br-int” bridge as below, + + ***HOST-A$ sudo ovs-vsctl --may-exist add-br br-int -- set Bridge br-int datapath_type=netdev -- br-set-external-id br-int bridge-id br-int -- set bridge br-int fail-mode=standalone*** + +2. Add the VM interface to the “br-int”. + + ***HOST-A$ sudo ovs-vsctl add-port br-int vm_port0 -- set Interface vm_port0 type=dpdkvhostuser*** + + `[“vm_port0” is the vhost-user interface name for the VM0. This interface name +must be used while qemu starting the virtual machine.]` + +3. Start the VM(VM-0) using vhost-user network backend. The vhost-user +interface name is "vm_port0". + +4. Set the Ip address 192.168.1.1 to the VM interface. Run the following +command inside the VM to set the Ip address. + + ***VM-0$ sudo ip addr add 192.168.1.1/24 dev eth0*** + + `[“eth0” is the interface inside VM. its possible to set the ip address using +"ifconfig" command as "sudo ifconfig eth0 192.168.1.1/24". Please use either +one command to set the ip address.]` + +5. Attach the VxLAN tunnel interface to the “br-int“ bridge. + + ***HOST-A$ sudo ovs-vsctl add-port br-int vxlan0 -- set interface vxlan0 type=vxlan options:remote_ip=172.168.1.2*** + + `[“172.168.1.2” is the remote tunnel end point address, On remote host this +will be 172.168.1.1.]` + +6. Create the “br-phy1” bridge for attaching the physical interface. + + ***HOST-A$ sudo ovs-vsctl --may-exist add-br br-phy1 -- set Bridge br-phy1 datapath_type=netdev -- br-set-external-id br-phy1 bridge-id br-phy1 -- set bridge br-phy1 fail-mode=standalone other_config:hwaddr="<mac address of eth1 interface>"*** + + `[“eth1” is the physical interface on the host, Assign mac address of “eth1” +to the “br-phy1”.]` + +7. The physical port "eth1" can operate either as a KNI(kernel network +interface) or a DPDK interface. Depending on the operating mode, attach "eth1" +to the "br-phy1" bridge as follows. + + Use step-7 if "eth1" is KNI. Please follow Step :8 rather than this step in +case "eth1" is a DPDK Interface.This step will cause to loose the connectivity +through “eth1” (refer configuration problems in https://github.com/openvswitch/ovs/blob/master/FAQ.md for more details) for a +while. The connectivity can be restored by moving the IP address to the +“br-phy1” internal interface. The following command-set will do that, + + ***HOST-A$ sudo ovs-vsctl --timeout 10 add-port br-phy1 eth1*** + + ***HOST-A$ sudo ip addr add 172.168.1.1/24 dev br-phy1*** + + ***HOST-A$ sudo ip link set br-phy1 up*** + + ***HOST-A$ sudo ip addr flush dev eth1 2>/dev/null*** + + ***HOST-A$ sudo ip link set eth1 up*** + + ***HOST-A$ sudo iptables –F*** + +8. Steps for attaching “eth1” to “br-phy1” is slightly different in case +“eth1” is a DPDK interface. DPDK interfaces are not managed by the kernel, so +the port details are not visible on any “ip” commands. + + ***HOST-A$ sudo ./tools/dpdk_nic_bind.py --bind=igb_uio eth1*** + + `["dpdk_nic_bind.py" is a utility script to bind/unbind interfaces to DPDK/Linux +kernel.More details on bind/unbind can be found at http://dpdk.org/doc/guides/linux_gsg/build_dpdk.html#binding-and-unbinding-network-ports-to-from-the-kernel-modules]` + + ***HOST-A$ sudo ovs-vsctl --timeout 10 add-port br-phy1 dpdk0 -- set Interface dpdk0 type=dpdk*** + + ***HOST-A$ sudo ip addr add 172.168.1.1/24 dev br-phy1*** + + ***HOST-A$ sudo ip link set br-phy1 up*** + + ***HOST-A$ sudo iptables –F*** + + `[“eth1” is mapped to DPDK port “dpdk0”. DPDK driver assign port names that +starts with “dpdk”]` + +Now the traffic from “VM0” will be VxLAN encapsulated and send out over eth1/dpdk0 +interface. + +TCPDUMP doesnt work on DPDK interfaces(eth0) as its no longer managed by the kernel. + +#### Debugging + +* Verify the created tunnel port details. + + ***HOST-A$ sudo ovs-appctl tnl/ports/show*** + +* As mentioned before, its necessary that the vswitch should have the arp-table +entries to do tunnelling at userspace. The learned arp entries in the vswitch +can be verified by, + + ***HOST-A$ sudo ovs-appctl tnl/arp/show*** + + the arp entries can be flushed using, + + ***HOST-A$ sudo ovs-appctl tnl/arp/flush*** + + To set a specific arp entry, + + ***HOST-A$ sudo ovs-appctl tnl/arp/set <bridge> <ip addr> <mac addr>*** + + In the above test setup, the following arp entries has to set in case they are + not present. + + ***HOST-A$ sudo ovs-appctl tnl/arp/set br-int 172.168.1.1 <mac addr of br-phy1>*** + + ***HOST-A$ sudo ovs-appctl tnl/arp/set br-phy1 172.168.1.2 <mac addr of remote TEP>*** + +* Similarly OVS uses routing table entries to xmit the tunnel packets. The +vswitch routing entries can be verified by, + + ***HOST-A$ sudo ovs-appctl ovs/route/show*** + + To delete the route entries. + + ***HOST-A$ sudo ovs-appctl ovs/route/del*** + +* To lookup a route entry for a destination please use, + + ***HOST-A$ ovs-appctl ovs/route/lookup*** + +* In case the route entries are missing for tunnel packet forwarding, it can be +added using the following command, + + ***HOST-A$ sudo ovs-appctl ovs/route/add 172.168.1.1/24 br-phy1*** + + This command instructs OVS to route all traffic destined for “172.168.1.2” to + bridge “br-phy1” + +* To verify the range of tunneling source ports, + + ***HOST-A$ sudo ovs-appctl tnl/egress_port_range*** + +* To see the configured datapath ports, + + ***HOST-A$ sudo ovs-appctl dpif/show*** + +* To verify the flows programmed on the OVS datapath, + + ***HOST-A$ sudo ovs-appctl dpctl/dump-flows*** + + [This command shows how OVS forwards the packets in datapath.]