| Message ID | 1444987930-4120-1-git-send-email-sugesh.chandran@intel.com |
|---|---|
| State | Not Applicable |
| Headers | show |
Thanks for writing this up, it will definitely help many users. I'm not sure the appropriate place for this is the OVS source tree (given that's similar to README-native-tunneling.md), but this seems a perfect candidate for a configuration cookbook (http://openvswitch.org/support/config-cookbooks/). We keep the openvswitch.org website repository here (https://github.com/openvswitch/openvswitch.github.io), will you consider creating a pull request to add this guide to the configuration cookbooks? I believe that there's no need to rewrite this in HTML, Markdown should be supported. Thanks, Daniele On 16/10/2015 02:32, "Sugesh Chandran" <sugesh.chandran@intel.com> wrote: >Adding a self-guide for configuring native userspace tunneling in OVS >with/without DPDK ports. This document also provides necessary debugging >commands to identify and resolve the userspace tunneling issues in OVS. > >Signed-off-by: Sugesh Chandran <sugesh.chandran@intel.com> >--- > README-native-tunneling-DPDK.md | 206 >++++++++++++++++++++++++++++++++++++++++ > 1 file changed, 206 insertions(+) > create mode 100644 README-native-tunneling-DPDK.md > >diff --git a/README-native-tunneling-DPDK.md >b/README-native-tunneling-DPDK.md >new file mode 100644 >index 0000000..47609b5 >--- /dev/null >+++ b/README-native-tunneling-DPDK.md >@@ -0,0 +1,206 @@ >+Openvswitch Native tunneling configuration guide >+======================================================= >+ >+This guide is for configuring userspace tunneling in Open-vSwitch. The >+traditional OVS with kernel datapath uses kernel module to perform the >tunneling >+however this setup performs all the tunneling operations purely in the >+userspace. This way userspace-tunnelling is platform independent. >+ >+This setup needs an additional bridge called ³br-phy1² when compared to >the >+kernel based OVS. The purpose of this bridge is to make available the >kernel >+network stack for routing and arp resolution. The data path needs to >look-up >+the routing table and arp table to prepare the tunnel header and xmit >data to >+the output port. >+ >+The tunneling setup is found as below: >+ >+ +--------------+ >+ | VM-0 | 192.168.1.1/24 >+ +--------------+ >+ (vm_port0) >+ | >+ | >+ | >+ | >+ | >+ +--------------+ >+ | br-int | 192.168.1.2/24 >+ +--------------+ +--------------+ >+ | vxlan0 | | vxlan0 | >+ +--------------+ +--------------+ >+ | | >+ | | >+ | | >+ | | >+ | | >+ | | >+ 172.168.1.1/24 | >+ +--------------+ | >+ | br-phy1 | 172.168.1.2/24 >+ +--------------+ +---------------+ >+ | dpdk0/eth1 |==========================| eth1 | >+ +--------------+ +---------------+ >+ Host A with OVS. Remote host. >+ >+#### Prerequisites >+The host must be pre-installed with OVS-DPDK, QEMU and virtual machine >images. >+Please refer the installation guide of relevant modules for these >instructions. >+ovs-vswitchd and OVSDB server must be up and running before proceeding >to any >+of the configuration steps. >+ >+This setup guide covers only the required steps for setting up VxLAN >userspace >+tunneling. The same approach can be used for any other tunneling >protocols, by >+specifying the appropriate tunneling protocol type. >+ >+Note:- This configuration guide is for setting up the VxLAN tunneling on >one >+host(local host). The same steps have to perform on the remote host as >well for >+setting up a VM<->VM VxLAN tunnel setup. The only difference for setting >up the >+userspace-tunneling in remote node is , VM and VxLAN tunnel ip addresses >are >+different. >+ >+#### Configuration steps >+1. Create the ³br-int² bridge as below, >+ >+ ***HOST-A$ sudo ovs-vsctl --may-exist add-br br-int -- set Bridge >br-int datapath_type=netdev -- br-set-external-id br-int bridge-id >br-int -- set bridge br-int fail-mode=standalone*** >+ >+2. Add the VM interface to the ³br-int². >+ >+ ***HOST-A$ sudo ovs-vsctl add-port br-int vm_port0 -- set Interface >vm_port0 type=dpdkvhostuser*** >+ >+ `[³vm_port0² is the vhost-user interface name for the VM0. This >interface name >+must be used while qemu starting the virtual machine.]` >+ >+3. Start the VM(VM-0) using vhost-user network backend. The vhost-user >+interface name is "vm_port0". >+ >+4. Set the Ip address 192.168.1.1 to the VM interface. Run the following >+command inside the VM to set the Ip address. >+ >+ ***VM-0$ sudo ip addr add 192.168.1.1/24 dev eth0*** >+ >+ `[³eth0² is the interface inside VM. its possible to set the ip >address using >+"ifconfig" command as "sudo ifconfig eth0 192.168.1.1/24". Please use >either >+one command to set the ip address.]` >+ >+5. Attach the VxLAN tunnel interface to the ³br-int³ bridge. >+ >+ ***HOST-A$ sudo ovs-vsctl add-port br-int vxlan0 -- set interface >vxlan0 type=vxlan options:remote_ip=172.168.1.2*** >+ >+ `[³172.168.1.2² is the remote tunnel end point address, On remote host >this >+will be 172.168.1.1.]` >+ >+6. Create the ³br-phy1² bridge for attaching the physical interface. >+ >+ ***HOST-A$ sudo ovs-vsctl --may-exist add-br br-phy1 -- set Bridge >br-phy1 datapath_type=netdev -- br-set-external-id br-phy1 bridge-id >br-phy1 -- set bridge br-phy1 fail-mode=standalone >other_config:hwaddr="<mac address of eth1 interface>"*** >+ >+ `[³eth1² is the physical interface on the host, Assign mac address of >³eth1² >+to the ³br-phy1².]` >+ >+7. The physical port "eth1" can operate either as a KNI(kernel network >+interface) or a DPDK interface. Depending on the operating mode, attach >"eth1" >+to the "br-phy1" bridge as follows. >+ >+ Use step-7 if "eth1" is KNI. Please follow Step :8 rather than this >step in >+case "eth1" is a DPDK Interface.This step will cause to loose the >connectivity >+through ³eth1² (refer configuration problems in >https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_openvswitc >h_ovs_blob_master_FAQ.md&d=BQIGaQ&c=Sqcl0Ez6M0X8aeM67LKIiDJAXVeAw-YihVMNtX >t-uEs&r=SmB5nZacmXNq0gKCC1s_Cw5yUNjxgD4v5kJqZ2uWLlE&m=LdO039X1zZ5EL5hDBSK- >X5UByLfxhThnfXUHoZaJp2M&s=8zuC3ssoCA-R5eXDWhdYSsNszPcmW0vmQFE21KE9vBg&e= >for more details) for a >+while. The connectivity can be restored by moving the IP address to the >+³br-phy1² internal interface. The following command-set will do that, >+ >+ ***HOST-A$ sudo ovs-vsctl --timeout 10 add-port br-phy1 eth1*** >+ >+ ***HOST-A$ sudo ip addr add 172.168.1.1/24 dev br-phy1*** >+ >+ ***HOST-A$ sudo ip link set br-phy1 up*** >+ >+ ***HOST-A$ sudo ip addr flush dev eth1 2>/dev/null*** >+ >+ ***HOST-A$ sudo ip link set eth1 up*** >+ >+ ***HOST-A$ sudo iptables F*** >+ >+8. Steps for attaching ³eth1² to ³br-phy1² is slightly different in case >+³eth1² is a DPDK interface. DPDK interfaces are not managed by the >kernel, so >+the port details are not visible on any ³ip² commands. >+ >+ ***HOST-A$ sudo ./tools/dpdk_nic_bind.py --bind=igb_uio eth1*** >+ >+ `["dpdk_nic_bind.py" is a utility script to bind/unbind interfaces to >DPDK/Linux >+kernel.More details on bind/unbind can be found at >https://urldefense.proofpoint.com/v2/url?u=http-3A__dpdk.org_doc_guides_li >nux-5Fgsg_build-5Fdpdk.html-23binding-2Dand-2Dunbinding-2Dnetwork-2Dports- >2Dto-2Dfrom-2Dthe-2Dkernel-2Dmodules&d=BQIGaQ&c=Sqcl0Ez6M0X8aeM67LKIiDJAXV >eAw-YihVMNtXt-uEs&r=SmB5nZacmXNq0gKCC1s_Cw5yUNjxgD4v5kJqZ2uWLlE&m=LdO039X1 >zZ5EL5hDBSK-X5UByLfxhThnfXUHoZaJp2M&s=WAahc2Mv28gVX8eCe913qdXbU_pYWFi_cjzc >PBxOgLc&e= ]` >+ >+ ***HOST-A$ sudo ovs-vsctl --timeout 10 add-port br-phy1 dpdk0 -- set >Interface dpdk0 type=dpdk*** >+ >+ ***HOST-A$ sudo ip addr add 172.168.1.1/24 dev br-phy1*** >+ >+ ***HOST-A$ sudo ip link set br-phy1 up*** >+ >+ ***HOST-A$ sudo iptables F*** >+ >+ `[³eth1² is mapped to DPDK port ³dpdk0². DPDK driver assign port names >that >+starts with ³dpdk²]` >+ >+Now the traffic from ³VM0² will be VxLAN encapsulated and send out over >eth1/dpdk0 >+interface. >+ >+TCPDUMP doesnt work on DPDK interfaces(eth0) as its no longer managed by >the kernel. >+ >+#### Debugging >+ >+* Verify the created tunnel port details. >+ >+ ***HOST-A$ sudo ovs-appctl tnl/ports/show*** >+ >+* As mentioned before, its necessary that the vswitch should have the >arp-table >+entries to do tunnelling at userspace. The learned arp entries in the >vswitch >+can be verified by, >+ >+ ***HOST-A$ sudo ovs-appctl tnl/arp/show*** >+ >+ the arp entries can be flushed using, >+ >+ ***HOST-A$ sudo ovs-appctl tnl/arp/flush*** >+ >+ To set a specific arp entry, >+ >+ ***HOST-A$ sudo ovs-appctl tnl/arp/set <bridge> <ip addr> ><mac addr>*** >+ >+ In the above test setup, the following arp entries has to set in case >they are >+ not present. >+ >+ ***HOST-A$ sudo ovs-appctl tnl/arp/set br-int 172.168.1.1 <mac addr >of br-phy1>*** >+ >+ ***HOST-A$ sudo ovs-appctl tnl/arp/set br-phy1 172.168.1.2 <mac >addr of remote TEP>*** >+ >+* Similarly OVS uses routing table entries to xmit the tunnel packets. >The >+vswitch routing entries can be verified by, >+ >+ ***HOST-A$ sudo ovs-appctl ovs/route/show*** >+ >+ To delete the route entries. >+ >+ ***HOST-A$ sudo ovs-appctl ovs/route/del*** >+ >+* To lookup a route entry for a destination please use, >+ >+ ***HOST-A$ ovs-appctl ovs/route/lookup*** >+ >+* In case the route entries are missing for tunnel packet forwarding, it >can be >+added using the following command, >+ >+ ***HOST-A$ sudo ovs-appctl ovs/route/add 172.168.1.1/24 br-phy1*** >+ >+ This command instructs OVS to route all traffic destined for >³172.168.1.2² to >+ bridge ³br-phy1² >+ >+* To verify the range of tunneling source ports, >+ >+ ***HOST-A$ sudo ovs-appctl tnl/egress_port_range*** >+ >+* To see the configured datapath ports, >+ >+ ***HOST-A$ sudo ovs-appctl dpif/show*** >+ >+* To verify the flows programmed on the OVS datapath, >+ >+ ***HOST-A$ sudo ovs-appctl dpctl/dump-flows*** >+ >+ [This command shows how OVS forwards the packets in datapath.] >-- >2.1.4 > >_______________________________________________ >dev mailing list >dev@openvswitch.org >https://urldefense.proofpoint.com/v2/url?u=http-3A__openvswitch.org_mailma >n_listinfo_dev&d=BQIGaQ&c=Sqcl0Ez6M0X8aeM67LKIiDJAXVeAw-YihVMNtXt-uEs&r=Sm >B5nZacmXNq0gKCC1s_Cw5yUNjxgD4v5kJqZ2uWLlE&m=LdO039X1zZ5EL5hDBSK-X5UByLfxhT >hnfXUHoZaJp2M&s=8IMF4lSKT6LrB5cEJkINsDKtlW_2tedPcqV0OFXLhUY&e=
Hi Daniele, Thank you for the inputs! As you suggested, I have created a pull request in "https://github.com/openvswitch/openvswitch.github.io" to add this guide under configuration cookbooks. You can find the details of pull request in the following link. https://github.com/openvswitch/openvswitch.github.io/pull/23 Please have a look on it and let me know the suggestions if any. Regards _Sugesh -----Original Message----- From: Daniele Di Proietto [mailto:diproiettod@vmware.com] Sent: Tuesday, October 20, 2015 9:16 PM To: Chandran, Sugesh Cc: dev@openvswitch.org Subject: Re: [ovs-dev] [PATCH] Detailed documentation for configuring native userspace-tunneling in OVS with/without DPDK. Thanks for writing this up, it will definitely help many users. I'm not sure the appropriate place for this is the OVS source tree (given that's similar to README-native-tunneling.md), but this seems a perfect candidate for a configuration cookbook (http://openvswitch.org/support/config-cookbooks/). We keep the openvswitch.org website repository here (https://github.com/openvswitch/openvswitch.github.io), will you consider creating a pull request to add this guide to the configuration cookbooks? I believe that there's no need to rewrite this in HTML, Markdown should be supported. Thanks, Daniele On 16/10/2015 02:32, "Sugesh Chandran" <sugesh.chandran@intel.com> wrote: >Adding a self-guide for configuring native userspace tunneling in OVS >with/without DPDK ports. This document also provides necessary >debugging commands to identify and resolve the userspace tunneling issues in OVS. > >Signed-off-by: Sugesh Chandran <sugesh.chandran@intel.com> >--- > README-native-tunneling-DPDK.md | 206 >++++++++++++++++++++++++++++++++++++++++ > 1 file changed, 206 insertions(+) > create mode 100644 README-native-tunneling-DPDK.md > >diff --git a/README-native-tunneling-DPDK.md >b/README-native-tunneling-DPDK.md >new file mode 100644 >index 0000000..47609b5 >--- /dev/null >+++ b/README-native-tunneling-DPDK.md >@@ -0,0 +1,206 @@ >+Openvswitch Native tunneling configuration guide >+======================================================= >+ >+This guide is for configuring userspace tunneling in Open-vSwitch. The >+traditional OVS with kernel datapath uses kernel module to perform the >tunneling >+however this setup performs all the tunneling operations purely in the >+userspace. This way userspace-tunnelling is platform independent. >+ >+This setup needs an additional bridge called ³br-phy1² when compared >+to >the >+kernel based OVS. The purpose of this bridge is to make available the >kernel >+network stack for routing and arp resolution. The data path needs to >look-up >+the routing table and arp table to prepare the tunnel header and xmit >data to >+the output port. >+ >+The tunneling setup is found as below: >+ >+ +--------------+ >+ | VM-0 | 192.168.1.1/24 >+ +--------------+ >+ (vm_port0) >+ | >+ | >+ | >+ | >+ | >+ +--------------+ >+ | br-int | 192.168.1.2/24 >+ +--------------+ +--------------+ >+ | vxlan0 | | vxlan0 | >+ +--------------+ +--------------+ >+ | | >+ | | >+ | | >+ | | >+ | | >+ | | >+ 172.168.1.1/24 | >+ +--------------+ | >+ | br-phy1 | 172.168.1.2/24 >+ +--------------+ +---------------+ >+ | dpdk0/eth1 |==========================| eth1 | >+ +--------------+ +---------------+ >+ Host A with OVS. Remote host. >+ >+#### Prerequisites >+The host must be pre-installed with OVS-DPDK, QEMU and virtual machine >images. >+Please refer the installation guide of relevant modules for these >instructions. >+ovs-vswitchd and OVSDB server must be up and running before proceeding >to any >+of the configuration steps. >+ >+This setup guide covers only the required steps for setting up VxLAN >userspace >+tunneling. The same approach can be used for any other tunneling >protocols, by >+specifying the appropriate tunneling protocol type. >+ >+Note:- This configuration guide is for setting up the VxLAN tunneling >+on >one >+host(local host). The same steps have to perform on the remote host as >well for >+setting up a VM<->VM VxLAN tunnel setup. The only difference for >+setting >up the >+userspace-tunneling in remote node is , VM and VxLAN tunnel ip >+addresses >are >+different. >+ >+#### Configuration steps >+1. Create the ³br-int² bridge as below, >+ >+ ***HOST-A$ sudo ovs-vsctl --may-exist add-br br-int -- set Bridge >br-int datapath_type=netdev -- br-set-external-id br-int bridge-id >br-int -- set bridge br-int fail-mode=standalone*** >+ >+2. Add the VM interface to the ³br-int². >+ >+ ***HOST-A$ sudo ovs-vsctl add-port br-int vm_port0 -- set Interface >vm_port0 type=dpdkvhostuser*** >+ >+ `[³vm_port0² is the vhost-user interface name for the VM0. This >interface name >+must be used while qemu starting the virtual machine.]` >+ >+3. Start the VM(VM-0) using vhost-user network backend. The vhost-user >+interface name is "vm_port0". >+ >+4. Set the Ip address 192.168.1.1 to the VM interface. Run the following >+command inside the VM to set the Ip address. >+ >+ ***VM-0$ sudo ip addr add 192.168.1.1/24 dev eth0*** >+ >+ `[³eth0² is the interface inside VM. its possible to set the ip >address using >+"ifconfig" command as "sudo ifconfig eth0 192.168.1.1/24". Please use >either >+one command to set the ip address.]` >+ >+5. Attach the VxLAN tunnel interface to the ³br-int³ bridge. >+ >+ ***HOST-A$ sudo ovs-vsctl add-port br-int vxlan0 -- set interface >vxlan0 type=vxlan options:remote_ip=172.168.1.2*** >+ >+ `[³172.168.1.2² is the remote tunnel end point address, On remote >+ host >this >+will be 172.168.1.1.]` >+ >+6. Create the ³br-phy1² bridge for attaching the physical interface. >+ >+ ***HOST-A$ sudo ovs-vsctl --may-exist add-br br-phy1 -- set Bridge >br-phy1 datapath_type=netdev -- br-set-external-id br-phy1 bridge-id >br-phy1 -- set bridge br-phy1 fail-mode=standalone >other_config:hwaddr="<mac address of eth1 interface>"*** >+ >+ `[³eth1² is the physical interface on the host, Assign mac address >+ of >³eth1² >+to the ³br-phy1².]` >+ >+7. The physical port "eth1" can operate either as a KNI(kernel network >+interface) or a DPDK interface. Depending on the operating mode, >+attach >"eth1" >+to the "br-phy1" bridge as follows. >+ >+ Use step-7 if "eth1" is KNI. Please follow Step :8 rather than this >step in >+case "eth1" is a DPDK Interface.This step will cause to loose the >connectivity >+through ³eth1² (refer configuration problems in >https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_openvsw >itc >h_ovs_blob_master_FAQ.md&d=BQIGaQ&c=Sqcl0Ez6M0X8aeM67LKIiDJAXVeAw-YihVM >NtX >t-uEs&r=SmB5nZacmXNq0gKCC1s_Cw5yUNjxgD4v5kJqZ2uWLlE&m=LdO039X1zZ5EL5hDB >SK- >X5UByLfxhThnfXUHoZaJp2M&s=8zuC3ssoCA-R5eXDWhdYSsNszPcmW0vmQFE21KE9vBg&e >= >for more details) for a >+while. The connectivity can be restored by moving the IP address to >+the ³br-phy1² internal interface. The following command-set will do >+that, >+ >+ ***HOST-A$ sudo ovs-vsctl --timeout 10 add-port br-phy1 eth1*** >+ >+ ***HOST-A$ sudo ip addr add 172.168.1.1/24 dev br-phy1*** >+ >+ ***HOST-A$ sudo ip link set br-phy1 up*** >+ >+ ***HOST-A$ sudo ip addr flush dev eth1 2>/dev/null*** >+ >+ ***HOST-A$ sudo ip link set eth1 up*** >+ >+ ***HOST-A$ sudo iptables F*** >+ >+8. Steps for attaching ³eth1² to ³br-phy1² is slightly different in case >+³eth1² is a DPDK interface. DPDK interfaces are not managed by the >kernel, so >+the port details are not visible on any ³ip² commands. >+ >+ ***HOST-A$ sudo ./tools/dpdk_nic_bind.py --bind=igb_uio eth1*** >+ >+ `["dpdk_nic_bind.py" is a utility script to bind/unbind interfaces >+ to >DPDK/Linux >+kernel.More details on bind/unbind can be found at >https://urldefense.proofpoint.com/v2/url?u=http-3A__dpdk.org_doc_guides >_li >nux-5Fgsg_build-5Fdpdk.html-23binding-2Dand-2Dunbinding-2Dnetwork-2Dpor >ts- >2Dto-2Dfrom-2Dthe-2Dkernel-2Dmodules&d=BQIGaQ&c=Sqcl0Ez6M0X8aeM67LKIiDJ >AXV >eAw-YihVMNtXt-uEs&r=SmB5nZacmXNq0gKCC1s_Cw5yUNjxgD4v5kJqZ2uWLlE&m=LdO03 >9X1 >zZ5EL5hDBSK-X5UByLfxhThnfXUHoZaJp2M&s=WAahc2Mv28gVX8eCe913qdXbU_pYWFi_c >jzc >PBxOgLc&e= ]` >+ >+ ***HOST-A$ sudo ovs-vsctl --timeout 10 add-port br-phy1 dpdk0 -- set >Interface dpdk0 type=dpdk*** >+ >+ ***HOST-A$ sudo ip addr add 172.168.1.1/24 dev br-phy1*** >+ >+ ***HOST-A$ sudo ip link set br-phy1 up*** >+ >+ ***HOST-A$ sudo iptables F*** >+ >+ `[³eth1² is mapped to DPDK port ³dpdk0². DPDK driver assign port >+ names >that >+starts with ³dpdk²]` >+ >+Now the traffic from ³VM0² will be VxLAN encapsulated and send out >+over >eth1/dpdk0 >+interface. >+ >+TCPDUMP doesnt work on DPDK interfaces(eth0) as its no longer managed >+by >the kernel. >+ >+#### Debugging >+ >+* Verify the created tunnel port details. >+ >+ ***HOST-A$ sudo ovs-appctl tnl/ports/show*** >+ >+* As mentioned before, its necessary that the vswitch should have the >arp-table >+entries to do tunnelling at userspace. The learned arp entries in the >vswitch >+can be verified by, >+ >+ ***HOST-A$ sudo ovs-appctl tnl/arp/show*** >+ >+ the arp entries can be flushed using, >+ >+ ***HOST-A$ sudo ovs-appctl tnl/arp/flush*** >+ >+ To set a specific arp entry, >+ >+ ***HOST-A$ sudo ovs-appctl tnl/arp/set <bridge> <ip >+ addr> ><mac addr>*** >+ >+ In the above test setup, the following arp entries has to set in case >they are >+ not present. >+ >+ ***HOST-A$ sudo ovs-appctl tnl/arp/set br-int 172.168.1.1 <mac >+ addr >of br-phy1>*** >+ >+ ***HOST-A$ sudo ovs-appctl tnl/arp/set br-phy1 172.168.1.2 <mac >addr of remote TEP>*** >+ >+* Similarly OVS uses routing table entries to xmit the tunnel packets. >The >+vswitch routing entries can be verified by, >+ >+ ***HOST-A$ sudo ovs-appctl ovs/route/show*** >+ >+ To delete the route entries. >+ >+ ***HOST-A$ sudo ovs-appctl ovs/route/del*** >+ >+* To lookup a route entry for a destination please use, >+ >+ ***HOST-A$ ovs-appctl ovs/route/lookup*** >+ >+* In case the route entries are missing for tunnel packet forwarding, >+it >can be >+added using the following command, >+ >+ ***HOST-A$ sudo ovs-appctl ovs/route/add 172.168.1.1/24 br-phy1*** >+ >+ This command instructs OVS to route all traffic destined for >³172.168.1.2² to >+ bridge ³br-phy1² >+ >+* To verify the range of tunneling source ports, >+ >+ ***HOST-A$ sudo ovs-appctl tnl/egress_port_range*** >+ >+* To see the configured datapath ports, >+ >+ ***HOST-A$ sudo ovs-appctl dpif/show*** >+ >+* To verify the flows programmed on the OVS datapath, >+ >+ ***HOST-A$ sudo ovs-appctl dpctl/dump-flows*** >+ >+ [This command shows how OVS forwards the packets in datapath.] >-- >2.1.4 > >_______________________________________________ >dev mailing list >dev@openvswitch.org >https://urldefense.proofpoint.com/v2/url?u=http-3A__openvswitch.org_mai >lma >n_listinfo_dev&d=BQIGaQ&c=Sqcl0Ez6M0X8aeM67LKIiDJAXVeAw-YihVMNtXt-uEs&r >=Sm >B5nZacmXNq0gKCC1s_Cw5yUNjxgD4v5kJqZ2uWLlE&m=LdO039X1zZ5EL5hDBSK-X5UByLf >xhT hnfXUHoZaJp2M&s=8IMF4lSKT6LrB5cEJkINsDKtlW_2tedPcqV0OFXLhUY&e=
diff --git a/README-native-tunneling-DPDK.md b/README-native-tunneling-DPDK.md new file mode 100644 index 0000000..47609b5 --- /dev/null +++ b/README-native-tunneling-DPDK.md @@ -0,0 +1,206 @@ +Openvswitch Native tunneling configuration guide +======================================================= + +This guide is for configuring userspace tunneling in Open-vSwitch. The +traditional OVS with kernel datapath uses kernel module to perform the tunneling +however this setup performs all the tunneling operations purely in the +userspace. This way userspace-tunnelling is platform independent. + +This setup needs an additional bridge called “br-phy1” when compared to the +kernel based OVS. The purpose of this bridge is to make available the kernel +network stack for routing and arp resolution. The data path needs to look-up +the routing table and arp table to prepare the tunnel header and xmit data to +the output port. + +The tunneling setup is found as below: + + +--------------+ + | VM-0 | 192.168.1.1/24 + +--------------+ + (vm_port0) + | + | + | + | + | + +--------------+ + | br-int | 192.168.1.2/24 + +--------------+ +--------------+ + | vxlan0 | | vxlan0 | + +--------------+ +--------------+ + | | + | | + | | + | | + | | + | | + 172.168.1.1/24 | + +--------------+ | + | br-phy1 | 172.168.1.2/24 + +--------------+ +---------------+ + | dpdk0/eth1 |==========================| eth1 | + +--------------+ +---------------+ + Host A with OVS. Remote host. + +#### Prerequisites +The host must be pre-installed with OVS-DPDK, QEMU and virtual machine images. +Please refer the installation guide of relevant modules for these instructions. +ovs-vswitchd and OVSDB server must be up and running before proceeding to any +of the configuration steps. + +This setup guide covers only the required steps for setting up VxLAN userspace +tunneling. The same approach can be used for any other tunneling protocols, by +specifying the appropriate tunneling protocol type. + +Note:- This configuration guide is for setting up the VxLAN tunneling on one +host(local host). The same steps have to perform on the remote host as well for +setting up a VM<->VM VxLAN tunnel setup. The only difference for setting up the +userspace-tunneling in remote node is , VM and VxLAN tunnel ip addresses are +different. + +#### Configuration steps +1. Create the “br-int” bridge as below, + + ***HOST-A$ sudo ovs-vsctl --may-exist add-br br-int -- set Bridge br-int datapath_type=netdev -- br-set-external-id br-int bridge-id br-int -- set bridge br-int fail-mode=standalone*** + +2. Add the VM interface to the “br-int”. + + ***HOST-A$ sudo ovs-vsctl add-port br-int vm_port0 -- set Interface vm_port0 type=dpdkvhostuser*** + + `[“vm_port0” is the vhost-user interface name for the VM0. This interface name +must be used while qemu starting the virtual machine.]` + +3. Start the VM(VM-0) using vhost-user network backend. The vhost-user +interface name is "vm_port0". + +4. Set the Ip address 192.168.1.1 to the VM interface. Run the following +command inside the VM to set the Ip address. + + ***VM-0$ sudo ip addr add 192.168.1.1/24 dev eth0*** + + `[“eth0” is the interface inside VM. its possible to set the ip address using +"ifconfig" command as "sudo ifconfig eth0 192.168.1.1/24". Please use either +one command to set the ip address.]` + +5. Attach the VxLAN tunnel interface to the “br-int“ bridge. + + ***HOST-A$ sudo ovs-vsctl add-port br-int vxlan0 -- set interface vxlan0 type=vxlan options:remote_ip=172.168.1.2*** + + `[“172.168.1.2” is the remote tunnel end point address, On remote host this +will be 172.168.1.1.]` + +6. Create the “br-phy1” bridge for attaching the physical interface. + + ***HOST-A$ sudo ovs-vsctl --may-exist add-br br-phy1 -- set Bridge br-phy1 datapath_type=netdev -- br-set-external-id br-phy1 bridge-id br-phy1 -- set bridge br-phy1 fail-mode=standalone other_config:hwaddr="<mac address of eth1 interface>"*** + + `[“eth1” is the physical interface on the host, Assign mac address of “eth1” +to the “br-phy1”.]` + +7. The physical port "eth1" can operate either as a KNI(kernel network +interface) or a DPDK interface. Depending on the operating mode, attach "eth1" +to the "br-phy1" bridge as follows. + + Use step-7 if "eth1" is KNI. Please follow Step :8 rather than this step in +case "eth1" is a DPDK Interface.This step will cause to loose the connectivity +through “eth1” (refer configuration problems in https://github.com/openvswitch/ovs/blob/master/FAQ.md for more details) for a +while. The connectivity can be restored by moving the IP address to the +“br-phy1” internal interface. The following command-set will do that, + + ***HOST-A$ sudo ovs-vsctl --timeout 10 add-port br-phy1 eth1*** + + ***HOST-A$ sudo ip addr add 172.168.1.1/24 dev br-phy1*** + + ***HOST-A$ sudo ip link set br-phy1 up*** + + ***HOST-A$ sudo ip addr flush dev eth1 2>/dev/null*** + + ***HOST-A$ sudo ip link set eth1 up*** + + ***HOST-A$ sudo iptables –F*** + +8. Steps for attaching “eth1” to “br-phy1” is slightly different in case +“eth1” is a DPDK interface. DPDK interfaces are not managed by the kernel, so +the port details are not visible on any “ip” commands. + + ***HOST-A$ sudo ./tools/dpdk_nic_bind.py --bind=igb_uio eth1*** + + `["dpdk_nic_bind.py" is a utility script to bind/unbind interfaces to DPDK/Linux +kernel.More details on bind/unbind can be found at http://dpdk.org/doc/guides/linux_gsg/build_dpdk.html#binding-and-unbinding-network-ports-to-from-the-kernel-modules]` + + ***HOST-A$ sudo ovs-vsctl --timeout 10 add-port br-phy1 dpdk0 -- set Interface dpdk0 type=dpdk*** + + ***HOST-A$ sudo ip addr add 172.168.1.1/24 dev br-phy1*** + + ***HOST-A$ sudo ip link set br-phy1 up*** + + ***HOST-A$ sudo iptables –F*** + + `[“eth1” is mapped to DPDK port “dpdk0”. DPDK driver assign port names that +starts with “dpdk”]` + +Now the traffic from “VM0” will be VxLAN encapsulated and send out over eth1/dpdk0 +interface. + +TCPDUMP doesnt work on DPDK interfaces(eth0) as its no longer managed by the kernel. + +#### Debugging + +* Verify the created tunnel port details. + + ***HOST-A$ sudo ovs-appctl tnl/ports/show*** + +* As mentioned before, its necessary that the vswitch should have the arp-table +entries to do tunnelling at userspace. The learned arp entries in the vswitch +can be verified by, + + ***HOST-A$ sudo ovs-appctl tnl/arp/show*** + + the arp entries can be flushed using, + + ***HOST-A$ sudo ovs-appctl tnl/arp/flush*** + + To set a specific arp entry, + + ***HOST-A$ sudo ovs-appctl tnl/arp/set <bridge> <ip addr> <mac addr>*** + + In the above test setup, the following arp entries has to set in case they are + not present. + + ***HOST-A$ sudo ovs-appctl tnl/arp/set br-int 172.168.1.1 <mac addr of br-phy1>*** + + ***HOST-A$ sudo ovs-appctl tnl/arp/set br-phy1 172.168.1.2 <mac addr of remote TEP>*** + +* Similarly OVS uses routing table entries to xmit the tunnel packets. The +vswitch routing entries can be verified by, + + ***HOST-A$ sudo ovs-appctl ovs/route/show*** + + To delete the route entries. + + ***HOST-A$ sudo ovs-appctl ovs/route/del*** + +* To lookup a route entry for a destination please use, + + ***HOST-A$ ovs-appctl ovs/route/lookup*** + +* In case the route entries are missing for tunnel packet forwarding, it can be +added using the following command, + + ***HOST-A$ sudo ovs-appctl ovs/route/add 172.168.1.1/24 br-phy1*** + + This command instructs OVS to route all traffic destined for “172.168.1.2” to + bridge “br-phy1” + +* To verify the range of tunneling source ports, + + ***HOST-A$ sudo ovs-appctl tnl/egress_port_range*** + +* To see the configured datapath ports, + + ***HOST-A$ sudo ovs-appctl dpif/show*** + +* To verify the flows programmed on the OVS datapath, + + ***HOST-A$ sudo ovs-appctl dpctl/dump-flows*** + + [This command shows how OVS forwards the packets in datapath.]
Adding a self-guide for configuring native userspace tunneling in OVS with/without DPDK ports. This document also provides necessary debugging commands to identify and resolve the userspace tunneling issues in OVS. Signed-off-by: Sugesh Chandran <sugesh.chandran@intel.com> --- README-native-tunneling-DPDK.md | 206 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 206 insertions(+) create mode 100644 README-native-tunneling-DPDK.md