From patchwork Sat May 16 05:53:53 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gowrishankar Muthukrishnan X-Patchwork-Id: 1291844 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.138; helo=whitealder.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=F47QeYXm; dkim-atps=neutral Received: from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 49PDxx0TY2z9sTD for ; Sat, 16 May 2020 15:54:45 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by whitealder.osuosl.org (Postfix) with ESMTP id 5713888D75; Sat, 16 May 2020 05:54:43 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from whitealder.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 3TrS+Thef4qz; Sat, 16 May 2020 05:54:34 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by whitealder.osuosl.org (Postfix) with ESMTP id 9A54C8902D; Sat, 16 May 2020 05:54:32 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 755D1C088F; Sat, 16 May 2020 05:54:32 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from silver.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 8B682C0865 for ; Sat, 16 May 2020 05:54:31 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by silver.osuosl.org (Postfix) with ESMTP id 77FEF207A6 for ; Sat, 16 May 2020 05:54:31 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from silver.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 6oeVuEUThGhW for ; Sat, 16 May 2020 05:54:16 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [205.139.110.120]) by silver.osuosl.org (Postfix) with ESMTPS id D1BF82152E for ; Sat, 16 May 2020 05:54:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1589608453; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=9nmYTSi2rHkzqKYGTMvQiMEADo9ZlH/ctmD1tXeWQbo=; b=F47QeYXm9rXqP0tD9/jzZ0pEoKMVCVhGSkwJKb+UQb1myJm89NkB8Lx0ixEYTuBssfCld3 fBONCHWMa6ShGt8+mbAAQTF/iQx34Nktr499RzmwMstZ/jXQHsx4x9GTn0A1t4b5n6Bke7 mLcekuYxaClFYCPAGzFgM2jh8+QMZKc= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-482-ycjQW0fXPo2nC-AlNRbFsw-1; Sat, 16 May 2020 01:54:09 -0400 X-MC-Unique: ycjQW0fXPo2nC-AlNRbFsw-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id A64058005AD; Sat, 16 May 2020 05:54:08 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-116-41.sin2.redhat.com [10.67.116.41]) by smtp.corp.redhat.com (Postfix) with ESMTP id 41790579A0; Sat, 16 May 2020 05:54:05 +0000 (UTC) From: Gowrishankar Muthukrishnan To: dev@openvswitch.org Date: Sat, 16 May 2020 11:23:53 +0530 Message-Id: <0c4c167ad644d3dda51b992e51ec1c27b8492457.1589605823.git.gmuthukr@redhat.com> In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Cc: Flavio Leitner Subject: [ovs-dev] [PATCH v2 3/3] system-dpdk: add negotiation check for userspace-tso X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" This patch adds minimal check for userspace-tso in system-dpdk tests, starting with verification on virtio negotiation. Signed-off-by: Gowrishankar Muthukrishnan --- v2: - removed tso check in existing tests (added in v1) - improved validations in host and guest end - added tests for non-tso backend. Testing info: Found a bug in virtio pmd for virtio negotiation with TSO in backend: http://patchwork.dpdk.org/patch/70233/ In summary, host side do not have bug after applying above patch in dpdk, even though reported failure (test #6) in host side is due to pmd in guest side (I am working on fixing it). Without patch, host side would not be able to enable TSO (as explained in it). With virtio pmd patch: 6: OVS-DPDK - validate tso negotiation (with userspace-tso) FAILED (system-dpdk.at:330) userspace-tcp on testpmd - vhostuser client ovs - vhostuser server host side: ovs received set_features for 0x910008183 which has VIRTIO_NET_F_CSUM but no VIRTIO_NET_F_HOST_TSO4 and VIRTIO_NET_F_HOST_TSO6. 6: OVS-DPDK - validate tso negotiation (with userspace-tso) FAILED (system-dpdk.at:333) userspace-tcp on testpmd - vhostuser client ovs - vhostuser server guest side: TCP_TSO is not turned on in virtio_user pmd and having: UDP_CKSUM TCP_CKSUM 7: OVS-DPDK - validate tso negotiation (with userspace-tso) FAILED (system-dpdk.at:395) userspace-tcp off testpmd - vhostuser server ovs - vhostuser client guest side: TSO features were not turned off in virtio_user pmd but having: UDP_CKSUM TCP_CKSUM TCP_TSO Without virtio pmd patch: 6: OVS-DPDK - validate tso negotiation (with userspace-tso) FAILED (system-dpdk.at:295) userspace-tcp on testpmd - vhostuser server ovs - vhostuser client host side: ovs received set_features for 0x910008000 which do not have any offload flags. VIRTIO_NET_F_CSUM, VIRTIO_NET_F_HOST_TSO4 and VIRTIO_NET_F_HOST_TSO6. 6: OVS-DPDK - validate tso negotiation (with userspace-tso) FAILED (system-dpdk.at:330) userspace-tcp on testpmd - vhostuser client ovs - vhostuser server host side: ovs received set_features for 0x910008000 which do not have any offload flags. VIRTIO_NET_F_CSUM, VIRTIO_NET_F_HOST_TSO4 and VIRTIO_NET_F_HOST_TSO6. 7: OVS-DPDK - validate tso negotiation (with userspace-tso) FAILED (system-dpdk.at:395) userspace-tcp off testpmd - vhostuser server ovs - vhostuser client guest side: TSO features were not turned off in virtio_user pmd but having: UDP_CKSUM TCP_CKSUM TCP_TSO --- tests/system-dpdk-macros.at | 17 ++-- tests/system-dpdk.at | 199 ++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 211 insertions(+), 5 deletions(-) diff --git a/tests/system-dpdk-macros.at b/tests/system-dpdk-macros.at index e5ac4f4..e804153 100644 --- a/tests/system-dpdk-macros.at +++ b/tests/system-dpdk-macros.at @@ -48,13 +48,11 @@ m4_define([OVS_DPDK_PRE_PHY_SKIP], ]) -# OVS_DPDK_START() +# OVS_DB_START() # -# Create an empty database and start ovsdb-server. Add special configuration -# dpdk-init to enable DPDK functionality. Start ovs-vswitchd connected to that -# database using system devices (no dummies). +# Create an empty database and start ovsdb-server. # -m4_define([OVS_DPDK_START], +m4_define([OVS_DB_START], [dnl Create database. AT_CHECK([touch .conf.db.~lock~]) AT_CHECK([ovsdb-tool create conf.db $abs_top_srcdir/vswitchd/vswitch.ovsschema]) @@ -69,7 +67,16 @@ m4_define([OVS_DPDK_START], dnl Initialize database. AT_CHECK([ovs-vsctl --no-wait init]) +]) + +# OVS_DPDK_START() +# +# Add special configuration dpdk-init to enable DPDK functionality. +# Start ovs-vswitchd connected to that database using system devices (no dummies). +# +m4_define([OVS_DPDK_START], + [ dnl Enable DPDK functionality AT_CHECK([ovs-vsctl --no-wait set Open_vSwitch . other_config:dpdk-init=true]) OVS_DPDK_SET_SOCKET_MEM() diff --git a/tests/system-dpdk.at b/tests/system-dpdk.at index b032c5b..016bb43 100644 --- a/tests/system-dpdk.at +++ b/tests/system-dpdk.at @@ -8,6 +8,7 @@ dnl Check if EAL init is successful AT_SETUP([OVS-DPDK - EAL init]) AT_KEYWORDS([dpdk]) OVS_DPDK_PRE_CHECK() +OVS_DB_START() OVS_DPDK_START() AT_CHECK([grep "DPDK Enabled - initializing..." ovs-vswitchd.log], [], [stdout]) AT_CHECK([grep "EAL" ovs-vswitchd.log], [], [stdout]) @@ -28,6 +29,7 @@ AT_SETUP([OVS-DPDK - add standard DPDK port]) AT_KEYWORDS([dpdk]) OVS_DPDK_PRE_PHY_SKIP() +OVS_DB_START() OVS_DPDK_START() dnl Add userspace bridge and attach it to OVS @@ -55,6 +57,7 @@ dnl Add vhost-user-client port AT_SETUP([OVS-DPDK - add vhost-user-client port]) AT_KEYWORDS([dpdk]) OVS_DPDK_PRE_CHECK() +OVS_DB_START() OVS_DPDK_START() dnl Add userspace bridge and attach it to OVS @@ -89,6 +92,7 @@ AT_SETUP([OVS-DPDK - ping vhost-user ports]) AT_KEYWORDS([dpdk]) OVS_DPDK_PRE_CHECK() AT_SKIP_IF([! which testpmd >/dev/null 2>/dev/null]) +OVS_DB_START() OVS_DPDK_START() dnl Find number of sockets @@ -169,6 +173,7 @@ AT_SETUP([OVS-DPDK - ping vhost-user-client ports]) AT_KEYWORDS([dpdk]) OVS_DPDK_PRE_CHECK() AT_SKIP_IF([! which testpmd >/dev/null 2>/dev/null]) +OVS_DB_START() OVS_DPDK_START() dnl Find number of sockets @@ -241,3 +246,197 @@ OVS_DPDK_HUGEPAGE_CLEANUP([rtemap_*]) OVS_DPDK_HUGEPAGE_CLEANUP([page0map_0]) AT_CLEANUP dnl -------------------------------------------------------------------------- + +dnl -------------------------------------------------------------------------- +dnl validate tso negotiation (with userspace-tso) +AT_SETUP([OVS-DPDK - validate tso negotiation (with userspace-tso)]) +AT_KEYWORDS([dpdk]) +OVS_DPDK_PRE_CHECK() +AT_SKIP_IF([! which testpmd >/dev/null 2>/dev/null]) +OVS_DB_START() +AT_CHECK([ovs-vsctl --no-wait set Open_vSwitch . other_config:userspace-tso-enable=true]) +OVS_DPDK_START() +AT_CHECK([grep -c 'Userspace TCP Segmentation Offloading support enabled' \ + ovs-vswitchd.log],[ignore],[dnl +1 +]) +dnl Find number of sockets +AT_CHECK([lscpu], [], [stdout]) +AT_CHECK([cat stdout | grep "NUMA node(s)" | awk '{c=1; while (c++<$(3)) {printf "448,"}; print "448"}' > NUMA_NODE]) + +dnl Add userspace bridge and attach it to OVS +AT_CHECK([ovs-vsctl add-br br10 -- set bridge br10 datapath_type=netdev]) + +dnl Add vhostuser port (client mode) +AT_CHECK([ovs-vsctl add-port br10 dpdkvhostuserclient0 -- set Interface \ + dpdkvhostuserclient0 \ + type=dpdkvhostuserclient \ + options:vhost-server-path=$OVS_RUNDIR/dpdkvhostclient0], [], + [stdout], [stderr]) +AT_CHECK([ovs-vsctl show], [], [stdout]) + +dnl Execute testpmd in background +on_exit "pkill -f -x -9 'tail -f /dev/null'" +AT_CHECK([echo "show device info all" > CMDFILE]) +AT_CHECK([echo "show port 0 tx_offload capabilities" >> CMDFILE]) +tail -f /dev/null | testpmd --no-pci \ + --socket-mem="$(cat NUMA_NODE)" --socket-limit="$(cat NUMA_NODE)" \ + --vdev="net_virtio_user,path=$OVS_RUNDIR/dpdkvhostclient0,server=1" \ + --vdev="net_tap0,iface=tap0" --file-prefix page0 \ + --single-file-segments -- --cmdline-file=CMDFILE \ + -a >$OVS_RUNDIR/testpmd-dpdkvhostuserclient0.log 2>&1 & + +dnl Give settling time to the testpmd processes - NOTE: this is bad form. +sleep 10 + +dnl Check whether TSO is turned on (host side) +AT_CHECK([awk '/negotiated Virtio features/ {a=$NF} END{print a}' \ + $OVS_RUNDIR/ovs-vswitchd.log],[],[stdout]) +AT_CHECK([printf "%X" $(( $(cat stdout) & ((1<<0)|(1<<11)|(1<<12)) ))],[],[1801]) + +dnl Check whether TSO is turned on (guest side) +AT_CHECK([awk 'BEGIN{n=0} /Per Port/ && /(TCP|UDP)_CKSUM/ && /TCP_TSO/ {n++} END{printf n}' \ + $OVS_RUNDIR/testpmd-dpdkvhostuserclient0.log],[0],[1]) + +dnl Clean up +AT_CHECK([ovs-vsctl del-port br10 dpdkvhostuserclient0], [], [stdout], [stderr]) + +dnl Clean up the testpmd now +pkill -f -x -9 'tail -f /dev/null' + +dnl Add vhostuser port (server mode) +AT_CHECK([ovs-vsctl add-port br10 dpdkvhostuser0 -- set Interface dpdkvhostuser0 \ + type=dpdkvhostuser], [], + [stdout], [stderr]) +AT_CHECK([ovs-vsctl show], [], [stdout]) + +dnl Execute testpmd in background +on_exit "pkill -f -x -9 'tail -f /dev/null'" +AT_CHECK([echo "show device info all" > CMDFILE]) +AT_CHECK([echo "show port 0 tx_offload capabilities" >> CMDFILE]) +tail -f /dev/null | testpmd --no-pci \ + --socket-mem="$(cat NUMA_NODE)" --socket-limit="$(cat NUMA_NODE)" \ + --vdev="net_virtio_user,path=$OVS_RUNDIR/dpdkvhostuser0" \ + --vdev="net_tap0,iface=tap0" --file-prefix page0 \ + --single-file-segments -- --cmdline-file=CMDFILE \ + -a >$OVS_RUNDIR/testpmd-dpdkvhostuser0.log 2>&1 & + +dnl Give settling time to the testpmd processes - NOTE: this is bad form. +sleep 10 + +dnl Check whether TSO is turned on (host side) +AT_CHECK([awk '/negotiated Virtio features/ {a=$NF} END{print a}' \ + $OVS_RUNDIR/ovs-vswitchd.log],[],[stdout]) +AT_CHECK([printf "%X" $(( $(cat stdout) & ((1<<0)|(1<<11)|(1<<12)) ))],[],[1801]) + +dnl Check whether TSO is turned on (guest side) +AT_CHECK([awk 'BEGIN{n=0} /Per Port/ && /(TCP|UDP)_CKSUM/ && /TCP_TSO/ {n++} END{printf n}' \ + $OVS_RUNDIR/testpmd-dpdkvhostuser0.log],[0],[1]) + +dnl Clean up the testpmd now +pkill -f -x -9 'tail -f /dev/null' + +dnl Clean up +AT_CHECK([ovs-vsctl del-port br10 dpdkvhostuser0], [], [stdout], [stderr]) +OVS_VSWITCHD_STOP() +OVS_DPDK_HUGEPAGE_CLEANUP([rtemap_*]) +OVS_DPDK_HUGEPAGE_CLEANUP([page0map_0]) +AT_CLEANUP +dnl -------------------------------------------------------------------------- + +dnl -------------------------------------------------------------------------- +dnl validate tso negotiation (without userspace-tso) +AT_SETUP([OVS-DPDK - validate tso negotiation (with userspace-tso)]) +AT_KEYWORDS([dpdk]) +OVS_DPDK_PRE_CHECK() +AT_SKIP_IF([! which testpmd >/dev/null 2>/dev/null]) +OVS_DB_START() +AT_CHECK([ovs-vsctl --no-wait set Open_vSwitch . other_config:userspace-tso-enable=false]) +OVS_DPDK_START() +AT_CHECK([grep -c 'Userspace TCP Segmentation Offloading support enabled' \ + ovs-vswitchd.log],[ignore],[dnl +0 +]) +dnl Find number of sockets +AT_CHECK([lscpu], [], [stdout]) +AT_CHECK([cat stdout | grep "NUMA node(s)" | awk '{c=1; while (c++<$(3)) {printf "448,"}; print "448"}' > NUMA_NODE]) + +dnl Add userspace bridge and attach it to OVS +AT_CHECK([ovs-vsctl add-br br10 -- set bridge br10 datapath_type=netdev]) + +dnl Add vhostuser port (client mode) +AT_CHECK([ovs-vsctl add-port br10 dpdkvhostuserclient0 -- set Interface \ + dpdkvhostuserclient0 \ + type=dpdkvhostuserclient \ + options:vhost-server-path=$OVS_RUNDIR/dpdkvhostclient0], [], + [stdout], [stderr]) +AT_CHECK([ovs-vsctl show], [], [stdout]) + +dnl Execute testpmd in background +on_exit "pkill -f -x -9 'tail -f /dev/null'" +AT_CHECK([echo "show device info all" > CMDFILE]) +AT_CHECK([echo "show port 0 tx_offload capabilities" >> CMDFILE]) +tail -f /dev/null | testpmd --no-pci \ + --socket-mem="$(cat NUMA_NODE)" --socket-limit="$(cat NUMA_NODE)" \ + --vdev="net_virtio_user,path=$OVS_RUNDIR/dpdkvhostclient0,server=1" \ + --vdev="net_tap0,iface=tap0" --file-prefix page0 \ + --single-file-segments -- --cmdline-file=CMDFILE \ + -a >$OVS_RUNDIR/testpmd-dpdkvhostuserclient0.log 2>&1 & + +dnl Give settling time to the testpmd processes - NOTE: this is bad form. +sleep 10 + +dnl Check whether TSO is turned off (host side) +AT_CHECK([awk '/negotiated Virtio features/ {a=$NF} END{print a}' \ + $OVS_RUNDIR/ovs-vswitchd.log],[],[stdout]) +AT_CHECK([printf "%X" $(( $(cat stdout) & ((1<<0)|(1<<11)|(1<<12)) ))],[],[0]) + +dnl Check whether TSO is turned off (guest side) +AT_CHECK([awk 'BEGIN{n=0} /Per Port/ && /(TCP|UDP)_CKSUM/ && /TCP_TSO/ {n++} END{printf n}' \ + $OVS_RUNDIR/testpmd-dpdkvhostuserclient0.log],[0],[0]) + +dnl Clean up the testpmd now +pkill -f -x -9 'tail -f /dev/null' + +dnl Clean up +AT_CHECK([ovs-vsctl del-port br10 dpdkvhostuserclient0], [], [stdout], [stderr]) + +dnl Add vhostuser port (server mode) +AT_CHECK([ovs-vsctl add-port br10 dpdkvhostuser0 -- set Interface dpdkvhostuser0 \ + type=dpdkvhostuser], [], + [stdout], [stderr]) +AT_CHECK([ovs-vsctl show], [], [stdout]) + +dnl Execute testpmd in background +on_exit "pkill -f -x -9 'tail -f /dev/null'" +AT_CHECK([echo "show device info all" > CMDFILE]) +AT_CHECK([echo "show port 0 tx_offload capabilities" >> CMDFILE]) +tail -f /dev/null | testpmd --no-pci \ + --socket-mem="$(cat NUMA_NODE)" --socket-limit="$(cat NUMA_NODE)" \ + --vdev="net_virtio_user,path=$OVS_RUNDIR/dpdkvhostuser0" \ + --vdev="net_tap0,iface=tap0" --file-prefix page0 \ + --single-file-segments -- --cmdline-file=CMDFILE \ + -a >$OVS_RUNDIR/testpmd-dpdkvhostuser0.log 2>&1 & + +dnl Give settling time to the testpmd processes - NOTE: this is bad form. +sleep 10 + +dnl Check whether TSO is turned off (host side) +AT_CHECK([awk '/negotiated Virtio features/ {a=$NF} END{print a}' \ + $OVS_RUNDIR/ovs-vswitchd.log],[],[stdout]) +AT_CHECK([printf "%X" $(( $(cat stdout) & ((1<<0)|(1<<11)|(1<<12)) ))],[],[0]) + +dnl Check whether TSO is turned off (guest side) +AT_CHECK([awk 'BEGIN{n=0} /Per Port/ && /(TCP|UDP)_CKSUM/ && /TCP_TSO/ {n++} END{printf n}' \ + $OVS_RUNDIR/testpmd-dpdkvhostuser0.log],[0],[0]) + +dnl Clean up the testpmd now +pkill -f -x -9 'tail -f /dev/null' + +dnl Clean up +AT_CHECK([ovs-vsctl del-port br10 dpdkvhostuser0], [], [stdout], [stderr]) +OVS_VSWITCHD_STOP() +OVS_DPDK_HUGEPAGE_CLEANUP([rtemap_*]) +OVS_DPDK_HUGEPAGE_CLEANUP([page0map_0]) +AT_CLEANUP +dnl --------------------------------------------------------------------------