From patchwork Sat May 16 05:53:51 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gowrishankar Muthukrishnan X-Patchwork-Id: 1291842 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.133; helo=hemlock.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=ejDsTxmm; dkim-atps=neutral Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 49PDxM1n5Lz9sTD for ; Sat, 16 May 2020 15:54:14 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id 97BE4888CF; Sat, 16 May 2020 05:54:12 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id YY822B2p3Xla; Sat, 16 May 2020 05:54:12 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by hemlock.osuosl.org (Postfix) with ESMTP id 0370C88AC7; Sat, 16 May 2020 05:54:12 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id DE1D7C088F; Sat, 16 May 2020 05:54:11 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by lists.linuxfoundation.org (Postfix) with ESMTP id AB1F1C016F for ; Sat, 16 May 2020 05:54:10 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id 9535D888B6 for ; Sat, 16 May 2020 05:54:10 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id sZcvij92xArA for ; Sat, 16 May 2020 05:54:08 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [205.139.110.120]) by hemlock.osuosl.org (Postfix) with ESMTPS id 7FFF8888B5 for ; Sat, 16 May 2020 05:54:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1589608447; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=CQmZtcEC4VEncUZ/xTX3axZ2DW4vPpcKUB35WFWybro=; b=ejDsTxmm6o20he8N8RTGvcyIFA8AQ0uubo6kD4kxdFCy66lM1bTML4Ph1oU0HBytDocUj5 n2ATBTuUX3TSnUd+lygvzHwVu7Ro7xYCEmDNLseQ54+tXsVdGxcs2Yz1S7iSdJMrOocLxy FVkAo70HmijNt5AXAI87wjxqjZyA3h0= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-76-KyMnnGpmM2WY4y-Pz7z0BA-1; Sat, 16 May 2020 01:54:03 -0400 X-MC-Unique: KyMnnGpmM2WY4y-Pz7z0BA-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 661D81005510; Sat, 16 May 2020 05:54:02 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-116-41.sin2.redhat.com [10.67.116.41]) by smtp.corp.redhat.com (Postfix) with ESMTP id 4B9A4579A0; Sat, 16 May 2020 05:53:59 +0000 (UTC) From: Gowrishankar Muthukrishnan To: dev@openvswitch.org Date: Sat, 16 May 2020 11:23:51 +0530 Message-Id: <3435decd0cc431e438bedfbcb92c0619e15e7489.1589605823.git.gmuthukr@redhat.com> In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Cc: Flavio Leitner Subject: [ovs-dev] [PATCH v2 1/3] system-dpdk: cleanup stale hugepage files after tests X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" After dpdk tests completes, cleaning up hugepage map files created by tests is helpful to release used memory into hugepage memory allocator. Signed-off-by: Gowrishankar Muthukrishnan --- tests/system-dpdk-macros.at | 13 +++++++++++++ tests/system-dpdk.at | 7 +++++++ 2 files changed, 20 insertions(+) diff --git a/tests/system-dpdk-macros.at b/tests/system-dpdk-macros.at index c6708ca..d3a3aea 100644 --- a/tests/system-dpdk-macros.at +++ b/tests/system-dpdk-macros.at @@ -63,3 +63,16 @@ m4_define([OVS_DPDK_START], AT_CAPTURE_FILE([ovs-vswitchd.log]) on_exit "kill_ovs_vswitchd `cat ovs-vswitchd.pid`" ]) + + +# OVS_DPDK_HUGEPAGE_CLEANUP([file]) +# +# Cleanup system for stale hugepages. +# +m4_define([OVS_DPDK_HUGEPAGE_CLEANUP], + [dnl Cleanup mapping files in hugetlbfs mount point + AT_CHECK([cat /proc/mounts | grep hugetlbfs], [], [stdout]) + AT_CHECK([cut -d ' ' -f 2 stdout], [], [stdout]) + AT_CHECK([rm -f $(cat stdout)/$1], [], []) + +]) diff --git a/tests/system-dpdk.at b/tests/system-dpdk.at index a015d52..01ac970 100644 --- a/tests/system-dpdk.at +++ b/tests/system-dpdk.at @@ -16,6 +16,7 @@ OVS_VSWITCHD_STOP(["/Global register is changed during/d /EAL: Invalid NUMA socket, default to 0/d /EAL: WARNING: cpu flags constant_tsc=yes nonstop_tsc=no -> using unreliable clock cycles !/d /EAL: No free hugepages reported in hugepages-1048576kB/d"]) +OVS_DPDK_HUGEPAGE_CLEANUP([rtemap_*]) AT_CLEANUP dnl -------------------------------------------------------------------------- @@ -43,6 +44,7 @@ OVS_VSWITCHD_STOP("/does not exist. The Open vSwitch kernel module is probably n /EAL: WARNING: cpu flags constant_tsc=yes nonstop_tsc=no -> using unreliable clock cycles !/d /EAL: No free hugepages reported in hugepages-1048576kB/d ") +OVS_DPDK_HUGEPAGE_CLEANUP([rtemap_*]) AT_CLEANUP dnl -------------------------------------------------------------------------- @@ -75,6 +77,7 @@ OVS_VSWITCHD_STOP(["\@does not exist. The Open vSwitch kernel module is probably \@EAL: Invalid NUMA socket, default to 0@d \@EAL: WARNING: cpu flags constant_tsc=yes nonstop_tsc=no -> using unreliable clock cycles !@d \@EAL: No free hugepages reported in hugepages-1048576kB@d"]) +OVS_DPDK_HUGEPAGE_CLEANUP([rtemap_*]) AT_CLEANUP dnl -------------------------------------------------------------------------- @@ -154,6 +157,8 @@ OVS_VSWITCHD_STOP(["\@does not exist. The Open vSwitch kernel module is probably \@EAL: Invalid NUMA socket, default to 0@d \@EAL: WARNING: cpu flags constant_tsc=yes nonstop_tsc=no -> using unreliable clock cycles !@d \@EAL: No free hugepages reported in hugepages-1048576kB@d"]) +OVS_DPDK_HUGEPAGE_CLEANUP([rtemap_*]) +OVS_DPDK_HUGEPAGE_CLEANUP([page0map_0]) AT_CLEANUP dnl -------------------------------------------------------------------------- @@ -230,5 +235,7 @@ OVS_VSWITCHD_STOP(["\@does not exist. The Open vSwitch kernel module is probably \@EAL: Invalid NUMA socket, default to 0@d \@EAL: WARNING: cpu flags constant_tsc=yes nonstop_tsc=no -> using unreliable clock cycles !@d \@EAL: No free hugepages reported in hugepages-1048576kB@d"]) +OVS_DPDK_HUGEPAGE_CLEANUP([rtemap_*]) +OVS_DPDK_HUGEPAGE_CLEANUP([page0map_0]) AT_CLEANUP dnl -------------------------------------------------------------------------- From patchwork Sat May 16 05:53:52 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gowrishankar Muthukrishnan X-Patchwork-Id: 1291843 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.138; helo=whitealder.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=bOn8NN66; dkim-atps=neutral Received: from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 49PDxf5r4yz9sTD for ; Sat, 16 May 2020 15:54:30 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by whitealder.osuosl.org (Postfix) with ESMTP id 496F088E9D; Sat, 16 May 2020 05:54:29 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from whitealder.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id FMGMf0W8vSel; Sat, 16 May 2020 05:54:27 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by whitealder.osuosl.org (Postfix) with ESMTP id 27A4688E8B; Sat, 16 May 2020 05:54:27 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 0B68BC088F; Sat, 16 May 2020 05:54:27 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by lists.linuxfoundation.org (Postfix) with ESMTP id 25F2DC016F for ; Sat, 16 May 2020 05:54:26 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by whitealder.osuosl.org (Postfix) with ESMTP id 0D34188E9D for ; Sat, 16 May 2020 05:54:26 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from whitealder.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id LODPyBNJgJJJ for ; Sat, 16 May 2020 05:54:24 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [205.139.110.120]) by whitealder.osuosl.org (Postfix) with ESMTPS id BC71388E88 for ; Sat, 16 May 2020 05:54:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1589608463; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=H9T+93Xn1NOKOkRxQliH+fbzboemzE0+NNQSFnfif9c=; b=bOn8NN66YDPalMNehQRmqua15D+0dqG832JcYi7rMt6kP2TKoUR2LK0CBB67meKrtiP9OR LIdAM0E9Mj0mXgayJCieYcsTga1uD/FQuvV4ySlNdYQd+u3KbdpwTtWkT0iyzl9T6ZYfo8 fMcgDPQBa+OCEO+yhnC9tp6prfD5TvA= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-342-SeW-i9-zMrmenK23lWFaKA-1; Sat, 16 May 2020 01:54:06 -0400 X-MC-Unique: SeW-i9-zMrmenK23lWFaKA-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 6CB261005510; Sat, 16 May 2020 05:54:05 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-116-41.sin2.redhat.com [10.67.116.41]) by smtp.corp.redhat.com (Postfix) with ESMTP id 4696E6918A; Sat, 16 May 2020 05:54:02 +0000 (UTC) From: Gowrishankar Muthukrishnan To: dev@openvswitch.org Date: Sat, 16 May 2020 11:23:52 +0530 Message-Id: <7bcb3150fa36e23de78d725bd91718c344ed896f.1589605823.git.gmuthukr@redhat.com> In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Cc: Flavio Leitner Subject: [ovs-dev] [PATCH v2 2/3] system-dpdk: use optimum hugepages for dpdk tests X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" Today we need 1GB from hugepages for running dpdk tests (i.e 512MB for ovs-vswitchd including phy ports and 512MB for testpmd app). This patch optimize the usage as: - 1GB for dpdk tests including phy ports, vhu ports and testpmd - 512MB for dpdk tests including vhu ports and testpmd Signed-off-by: Gowrishankar Muthukrishnan --- v2: - fixed hunk to reflect intended changes. --- tests/system-dpdk-macros.at | 19 +++++++++++++++++++ tests/system-dpdk.at | 10 ++++++---- 2 files changed, 25 insertions(+), 4 deletions(-) diff --git a/tests/system-dpdk-macros.at b/tests/system-dpdk-macros.at index d3a3aea..e5ac4f4 100644 --- a/tests/system-dpdk-macros.at +++ b/tests/system-dpdk-macros.at @@ -1,3 +1,18 @@ +# OVS_DPDK_SET_SOCKET_MEM() +# +# Set optimum memory required for tests. +# +m4_define([OVS_DPDK_SET_SOCKET_MEM], + [ + AT_CHECK([$DPDK_DIR/usertools/dpdk-devbind.py -s],[],[stdout]) + AT_CHECK([sed -n '/DPDK/,/kernel/{//!p;}' stdout|sed '/^=/d'],[],[stdout]) + AT_CHECK([egrep -c '^....:..:..\..' stdout],[ignore],[stdout]) + AS_IF([test $(cat stdout) -eq 0],[echo 64 > SOCKET_MEM], + [echo $((64+512)) > SOCKET_MEM]) + +]) + + # OVS_DPDK_PRE_CHECK() # # Check prerequisites for DPDK tests. Following settings are checked: @@ -57,6 +72,10 @@ m4_define([OVS_DPDK_START], dnl Enable DPDK functionality AT_CHECK([ovs-vsctl --no-wait set Open_vSwitch . other_config:dpdk-init=true]) + OVS_DPDK_SET_SOCKET_MEM() + AT_CHECK([lscpu], [], [stdout]) + AT_CHECK([cat stdout | grep "NUMA node(s)" | awk -v m=$(cat SOCKET_MEM) '{c=1; while (c++<$(3)) {printf m","}; print m}' > NUMA_NODE]) + AT_CHECK([ovs-vsctl --no-wait set Open_vSwitch . other_config:dpdk-socket-mem=$(cat NUMA_NODE)]) dnl Start ovs-vswitchd. AT_CHECK([ovs-vswitchd --detach --no-chdir --pidfile --log-file -vvconn -vofproto_dpif -vunixctl], [0], [stdout], [stderr]) diff --git a/tests/system-dpdk.at b/tests/system-dpdk.at index 01ac970..b032c5b 100644 --- a/tests/system-dpdk.at +++ b/tests/system-dpdk.at @@ -93,7 +93,7 @@ OVS_DPDK_START() dnl Find number of sockets AT_CHECK([lscpu], [], [stdout]) -AT_CHECK([cat stdout | grep "NUMA node(s)" | awk '{c=1; while (c++<$(3)) {printf "512,"}; print "512"}' > NUMA_NODE]) +AT_CHECK([cat stdout | grep "NUMA node(s)" | awk '{c=1; while (c++<$(3)) {printf "448,"}; print "448"}' > NUMA_NODE]) dnl Add userspace bridge and attach it to OVS AT_CHECK([ovs-vsctl add-br br10 -- set bridge br10 datapath_type=netdev]) @@ -118,7 +118,8 @@ ADD_VETH(tap1, ns2, br10, "172.31.110.12/24") dnl Execute testpmd in background on_exit "pkill -f -x -9 'tail -f /dev/null'" -tail -f /dev/null | testpmd --socket-mem="$(cat NUMA_NODE)" --no-pci\ +tail -f /dev/null | testpmd --no-pci \ + --socket-mem="$(cat NUMA_NODE)" --socket-limit="$(cat NUMA_NODE)" \ --vdev="net_virtio_user,path=$OVS_RUNDIR/dpdkvhostuser0" \ --vdev="net_tap0,iface=tap0" --file-prefix page0 \ --single-file-segments -- -a >$OVS_RUNDIR/testpmd-dpdkvhostuser0.log 2>&1 & @@ -172,7 +173,7 @@ OVS_DPDK_START() dnl Find number of sockets AT_CHECK([lscpu], [], [stdout]) -AT_CHECK([cat stdout | grep "NUMA node(s)" | awk '{c=1; while (c++<$(3)) {printf "512,"}; print "512"}' > NUMA_NODE]) +AT_CHECK([cat stdout | grep "NUMA node(s)" | awk '{c=1; while (c++<$(3)) {printf "448,"}; print "448"}' > NUMA_NODE]) dnl Add userspace bridge and attach it to OVS AT_CHECK([ovs-vsctl add-br br10 -- set bridge br10 datapath_type=netdev]) @@ -196,7 +197,8 @@ ADD_VETH(tap1, ns2, br10, "172.31.110.12/24") dnl Execute testpmd in background on_exit "pkill -f -x -9 'tail -f /dev/null'" -tail -f /dev/null | testpmd --socket-mem="$(cat NUMA_NODE)" --no-pci\ +tail -f /dev/null | testpmd --no-pci \ + --socket-mem="$(cat NUMA_NODE)" --socket-limit="$(cat NUMA_NODE)" \ --vdev="net_virtio_user,path=$OVS_RUNDIR/dpdkvhostclient0,server=1" \ --vdev="net_tap0,iface=tap0" --file-prefix page0 \ --single-file-segments -- -a >$OVS_RUNDIR/testpmd-dpdkvhostuserclient0.log 2>&1 & From patchwork Sat May 16 05:53:53 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gowrishankar Muthukrishnan X-Patchwork-Id: 1291844 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.138; helo=whitealder.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=F47QeYXm; dkim-atps=neutral Received: from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 49PDxx0TY2z9sTD for ; Sat, 16 May 2020 15:54:45 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by whitealder.osuosl.org (Postfix) with ESMTP id 5713888D75; Sat, 16 May 2020 05:54:43 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from whitealder.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 3TrS+Thef4qz; Sat, 16 May 2020 05:54:34 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by whitealder.osuosl.org (Postfix) with ESMTP id 9A54C8902D; Sat, 16 May 2020 05:54:32 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 755D1C088F; Sat, 16 May 2020 05:54:32 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from silver.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 8B682C0865 for ; Sat, 16 May 2020 05:54:31 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by silver.osuosl.org (Postfix) with ESMTP id 77FEF207A6 for ; Sat, 16 May 2020 05:54:31 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from silver.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 6oeVuEUThGhW for ; Sat, 16 May 2020 05:54:16 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [205.139.110.120]) by silver.osuosl.org (Postfix) with ESMTPS id D1BF82152E for ; Sat, 16 May 2020 05:54:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1589608453; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=9nmYTSi2rHkzqKYGTMvQiMEADo9ZlH/ctmD1tXeWQbo=; b=F47QeYXm9rXqP0tD9/jzZ0pEoKMVCVhGSkwJKb+UQb1myJm89NkB8Lx0ixEYTuBssfCld3 fBONCHWMa6ShGt8+mbAAQTF/iQx34Nktr499RzmwMstZ/jXQHsx4x9GTn0A1t4b5n6Bke7 mLcekuYxaClFYCPAGzFgM2jh8+QMZKc= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-482-ycjQW0fXPo2nC-AlNRbFsw-1; Sat, 16 May 2020 01:54:09 -0400 X-MC-Unique: ycjQW0fXPo2nC-AlNRbFsw-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id A64058005AD; Sat, 16 May 2020 05:54:08 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-116-41.sin2.redhat.com [10.67.116.41]) by smtp.corp.redhat.com (Postfix) with ESMTP id 41790579A0; Sat, 16 May 2020 05:54:05 +0000 (UTC) From: Gowrishankar Muthukrishnan To: dev@openvswitch.org Date: Sat, 16 May 2020 11:23:53 +0530 Message-Id: <0c4c167ad644d3dda51b992e51ec1c27b8492457.1589605823.git.gmuthukr@redhat.com> In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Cc: Flavio Leitner Subject: [ovs-dev] [PATCH v2 3/3] system-dpdk: add negotiation check for userspace-tso X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" This patch adds minimal check for userspace-tso in system-dpdk tests, starting with verification on virtio negotiation. Signed-off-by: Gowrishankar Muthukrishnan --- v2: - removed tso check in existing tests (added in v1) - improved validations in host and guest end - added tests for non-tso backend. Testing info: Found a bug in virtio pmd for virtio negotiation with TSO in backend: http://patchwork.dpdk.org/patch/70233/ In summary, host side do not have bug after applying above patch in dpdk, even though reported failure (test #6) in host side is due to pmd in guest side (I am working on fixing it). Without patch, host side would not be able to enable TSO (as explained in it). With virtio pmd patch: 6: OVS-DPDK - validate tso negotiation (with userspace-tso) FAILED (system-dpdk.at:330) userspace-tcp on testpmd - vhostuser client ovs - vhostuser server host side: ovs received set_features for 0x910008183 which has VIRTIO_NET_F_CSUM but no VIRTIO_NET_F_HOST_TSO4 and VIRTIO_NET_F_HOST_TSO6. 6: OVS-DPDK - validate tso negotiation (with userspace-tso) FAILED (system-dpdk.at:333) userspace-tcp on testpmd - vhostuser client ovs - vhostuser server guest side: TCP_TSO is not turned on in virtio_user pmd and having: UDP_CKSUM TCP_CKSUM 7: OVS-DPDK - validate tso negotiation (with userspace-tso) FAILED (system-dpdk.at:395) userspace-tcp off testpmd - vhostuser server ovs - vhostuser client guest side: TSO features were not turned off in virtio_user pmd but having: UDP_CKSUM TCP_CKSUM TCP_TSO Without virtio pmd patch: 6: OVS-DPDK - validate tso negotiation (with userspace-tso) FAILED (system-dpdk.at:295) userspace-tcp on testpmd - vhostuser server ovs - vhostuser client host side: ovs received set_features for 0x910008000 which do not have any offload flags. VIRTIO_NET_F_CSUM, VIRTIO_NET_F_HOST_TSO4 and VIRTIO_NET_F_HOST_TSO6. 6: OVS-DPDK - validate tso negotiation (with userspace-tso) FAILED (system-dpdk.at:330) userspace-tcp on testpmd - vhostuser client ovs - vhostuser server host side: ovs received set_features for 0x910008000 which do not have any offload flags. VIRTIO_NET_F_CSUM, VIRTIO_NET_F_HOST_TSO4 and VIRTIO_NET_F_HOST_TSO6. 7: OVS-DPDK - validate tso negotiation (with userspace-tso) FAILED (system-dpdk.at:395) userspace-tcp off testpmd - vhostuser server ovs - vhostuser client guest side: TSO features were not turned off in virtio_user pmd but having: UDP_CKSUM TCP_CKSUM TCP_TSO --- tests/system-dpdk-macros.at | 17 ++-- tests/system-dpdk.at | 199 ++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 211 insertions(+), 5 deletions(-) diff --git a/tests/system-dpdk-macros.at b/tests/system-dpdk-macros.at index e5ac4f4..e804153 100644 --- a/tests/system-dpdk-macros.at +++ b/tests/system-dpdk-macros.at @@ -48,13 +48,11 @@ m4_define([OVS_DPDK_PRE_PHY_SKIP], ]) -# OVS_DPDK_START() +# OVS_DB_START() # -# Create an empty database and start ovsdb-server. Add special configuration -# dpdk-init to enable DPDK functionality. Start ovs-vswitchd connected to that -# database using system devices (no dummies). +# Create an empty database and start ovsdb-server. # -m4_define([OVS_DPDK_START], +m4_define([OVS_DB_START], [dnl Create database. AT_CHECK([touch .conf.db.~lock~]) AT_CHECK([ovsdb-tool create conf.db $abs_top_srcdir/vswitchd/vswitch.ovsschema]) @@ -69,7 +67,16 @@ m4_define([OVS_DPDK_START], dnl Initialize database. AT_CHECK([ovs-vsctl --no-wait init]) +]) + +# OVS_DPDK_START() +# +# Add special configuration dpdk-init to enable DPDK functionality. +# Start ovs-vswitchd connected to that database using system devices (no dummies). +# +m4_define([OVS_DPDK_START], + [ dnl Enable DPDK functionality AT_CHECK([ovs-vsctl --no-wait set Open_vSwitch . other_config:dpdk-init=true]) OVS_DPDK_SET_SOCKET_MEM() diff --git a/tests/system-dpdk.at b/tests/system-dpdk.at index b032c5b..016bb43 100644 --- a/tests/system-dpdk.at +++ b/tests/system-dpdk.at @@ -8,6 +8,7 @@ dnl Check if EAL init is successful AT_SETUP([OVS-DPDK - EAL init]) AT_KEYWORDS([dpdk]) OVS_DPDK_PRE_CHECK() +OVS_DB_START() OVS_DPDK_START() AT_CHECK([grep "DPDK Enabled - initializing..." ovs-vswitchd.log], [], [stdout]) AT_CHECK([grep "EAL" ovs-vswitchd.log], [], [stdout]) @@ -28,6 +29,7 @@ AT_SETUP([OVS-DPDK - add standard DPDK port]) AT_KEYWORDS([dpdk]) OVS_DPDK_PRE_PHY_SKIP() +OVS_DB_START() OVS_DPDK_START() dnl Add userspace bridge and attach it to OVS @@ -55,6 +57,7 @@ dnl Add vhost-user-client port AT_SETUP([OVS-DPDK - add vhost-user-client port]) AT_KEYWORDS([dpdk]) OVS_DPDK_PRE_CHECK() +OVS_DB_START() OVS_DPDK_START() dnl Add userspace bridge and attach it to OVS @@ -89,6 +92,7 @@ AT_SETUP([OVS-DPDK - ping vhost-user ports]) AT_KEYWORDS([dpdk]) OVS_DPDK_PRE_CHECK() AT_SKIP_IF([! which testpmd >/dev/null 2>/dev/null]) +OVS_DB_START() OVS_DPDK_START() dnl Find number of sockets @@ -169,6 +173,7 @@ AT_SETUP([OVS-DPDK - ping vhost-user-client ports]) AT_KEYWORDS([dpdk]) OVS_DPDK_PRE_CHECK() AT_SKIP_IF([! which testpmd >/dev/null 2>/dev/null]) +OVS_DB_START() OVS_DPDK_START() dnl Find number of sockets @@ -241,3 +246,197 @@ OVS_DPDK_HUGEPAGE_CLEANUP([rtemap_*]) OVS_DPDK_HUGEPAGE_CLEANUP([page0map_0]) AT_CLEANUP dnl -------------------------------------------------------------------------- + +dnl -------------------------------------------------------------------------- +dnl validate tso negotiation (with userspace-tso) +AT_SETUP([OVS-DPDK - validate tso negotiation (with userspace-tso)]) +AT_KEYWORDS([dpdk]) +OVS_DPDK_PRE_CHECK() +AT_SKIP_IF([! which testpmd >/dev/null 2>/dev/null]) +OVS_DB_START() +AT_CHECK([ovs-vsctl --no-wait set Open_vSwitch . other_config:userspace-tso-enable=true]) +OVS_DPDK_START() +AT_CHECK([grep -c 'Userspace TCP Segmentation Offloading support enabled' \ + ovs-vswitchd.log],[ignore],[dnl +1 +]) +dnl Find number of sockets +AT_CHECK([lscpu], [], [stdout]) +AT_CHECK([cat stdout | grep "NUMA node(s)" | awk '{c=1; while (c++<$(3)) {printf "448,"}; print "448"}' > NUMA_NODE]) + +dnl Add userspace bridge and attach it to OVS +AT_CHECK([ovs-vsctl add-br br10 -- set bridge br10 datapath_type=netdev]) + +dnl Add vhostuser port (client mode) +AT_CHECK([ovs-vsctl add-port br10 dpdkvhostuserclient0 -- set Interface \ + dpdkvhostuserclient0 \ + type=dpdkvhostuserclient \ + options:vhost-server-path=$OVS_RUNDIR/dpdkvhostclient0], [], + [stdout], [stderr]) +AT_CHECK([ovs-vsctl show], [], [stdout]) + +dnl Execute testpmd in background +on_exit "pkill -f -x -9 'tail -f /dev/null'" +AT_CHECK([echo "show device info all" > CMDFILE]) +AT_CHECK([echo "show port 0 tx_offload capabilities" >> CMDFILE]) +tail -f /dev/null | testpmd --no-pci \ + --socket-mem="$(cat NUMA_NODE)" --socket-limit="$(cat NUMA_NODE)" \ + --vdev="net_virtio_user,path=$OVS_RUNDIR/dpdkvhostclient0,server=1" \ + --vdev="net_tap0,iface=tap0" --file-prefix page0 \ + --single-file-segments -- --cmdline-file=CMDFILE \ + -a >$OVS_RUNDIR/testpmd-dpdkvhostuserclient0.log 2>&1 & + +dnl Give settling time to the testpmd processes - NOTE: this is bad form. +sleep 10 + +dnl Check whether TSO is turned on (host side) +AT_CHECK([awk '/negotiated Virtio features/ {a=$NF} END{print a}' \ + $OVS_RUNDIR/ovs-vswitchd.log],[],[stdout]) +AT_CHECK([printf "%X" $(( $(cat stdout) & ((1<<0)|(1<<11)|(1<<12)) ))],[],[1801]) + +dnl Check whether TSO is turned on (guest side) +AT_CHECK([awk 'BEGIN{n=0} /Per Port/ && /(TCP|UDP)_CKSUM/ && /TCP_TSO/ {n++} END{printf n}' \ + $OVS_RUNDIR/testpmd-dpdkvhostuserclient0.log],[0],[1]) + +dnl Clean up +AT_CHECK([ovs-vsctl del-port br10 dpdkvhostuserclient0], [], [stdout], [stderr]) + +dnl Clean up the testpmd now +pkill -f -x -9 'tail -f /dev/null' + +dnl Add vhostuser port (server mode) +AT_CHECK([ovs-vsctl add-port br10 dpdkvhostuser0 -- set Interface dpdkvhostuser0 \ + type=dpdkvhostuser], [], + [stdout], [stderr]) +AT_CHECK([ovs-vsctl show], [], [stdout]) + +dnl Execute testpmd in background +on_exit "pkill -f -x -9 'tail -f /dev/null'" +AT_CHECK([echo "show device info all" > CMDFILE]) +AT_CHECK([echo "show port 0 tx_offload capabilities" >> CMDFILE]) +tail -f /dev/null | testpmd --no-pci \ + --socket-mem="$(cat NUMA_NODE)" --socket-limit="$(cat NUMA_NODE)" \ + --vdev="net_virtio_user,path=$OVS_RUNDIR/dpdkvhostuser0" \ + --vdev="net_tap0,iface=tap0" --file-prefix page0 \ + --single-file-segments -- --cmdline-file=CMDFILE \ + -a >$OVS_RUNDIR/testpmd-dpdkvhostuser0.log 2>&1 & + +dnl Give settling time to the testpmd processes - NOTE: this is bad form. +sleep 10 + +dnl Check whether TSO is turned on (host side) +AT_CHECK([awk '/negotiated Virtio features/ {a=$NF} END{print a}' \ + $OVS_RUNDIR/ovs-vswitchd.log],[],[stdout]) +AT_CHECK([printf "%X" $(( $(cat stdout) & ((1<<0)|(1<<11)|(1<<12)) ))],[],[1801]) + +dnl Check whether TSO is turned on (guest side) +AT_CHECK([awk 'BEGIN{n=0} /Per Port/ && /(TCP|UDP)_CKSUM/ && /TCP_TSO/ {n++} END{printf n}' \ + $OVS_RUNDIR/testpmd-dpdkvhostuser0.log],[0],[1]) + +dnl Clean up the testpmd now +pkill -f -x -9 'tail -f /dev/null' + +dnl Clean up +AT_CHECK([ovs-vsctl del-port br10 dpdkvhostuser0], [], [stdout], [stderr]) +OVS_VSWITCHD_STOP() +OVS_DPDK_HUGEPAGE_CLEANUP([rtemap_*]) +OVS_DPDK_HUGEPAGE_CLEANUP([page0map_0]) +AT_CLEANUP +dnl -------------------------------------------------------------------------- + +dnl -------------------------------------------------------------------------- +dnl validate tso negotiation (without userspace-tso) +AT_SETUP([OVS-DPDK - validate tso negotiation (with userspace-tso)]) +AT_KEYWORDS([dpdk]) +OVS_DPDK_PRE_CHECK() +AT_SKIP_IF([! which testpmd >/dev/null 2>/dev/null]) +OVS_DB_START() +AT_CHECK([ovs-vsctl --no-wait set Open_vSwitch . other_config:userspace-tso-enable=false]) +OVS_DPDK_START() +AT_CHECK([grep -c 'Userspace TCP Segmentation Offloading support enabled' \ + ovs-vswitchd.log],[ignore],[dnl +0 +]) +dnl Find number of sockets +AT_CHECK([lscpu], [], [stdout]) +AT_CHECK([cat stdout | grep "NUMA node(s)" | awk '{c=1; while (c++<$(3)) {printf "448,"}; print "448"}' > NUMA_NODE]) + +dnl Add userspace bridge and attach it to OVS +AT_CHECK([ovs-vsctl add-br br10 -- set bridge br10 datapath_type=netdev]) + +dnl Add vhostuser port (client mode) +AT_CHECK([ovs-vsctl add-port br10 dpdkvhostuserclient0 -- set Interface \ + dpdkvhostuserclient0 \ + type=dpdkvhostuserclient \ + options:vhost-server-path=$OVS_RUNDIR/dpdkvhostclient0], [], + [stdout], [stderr]) +AT_CHECK([ovs-vsctl show], [], [stdout]) + +dnl Execute testpmd in background +on_exit "pkill -f -x -9 'tail -f /dev/null'" +AT_CHECK([echo "show device info all" > CMDFILE]) +AT_CHECK([echo "show port 0 tx_offload capabilities" >> CMDFILE]) +tail -f /dev/null | testpmd --no-pci \ + --socket-mem="$(cat NUMA_NODE)" --socket-limit="$(cat NUMA_NODE)" \ + --vdev="net_virtio_user,path=$OVS_RUNDIR/dpdkvhostclient0,server=1" \ + --vdev="net_tap0,iface=tap0" --file-prefix page0 \ + --single-file-segments -- --cmdline-file=CMDFILE \ + -a >$OVS_RUNDIR/testpmd-dpdkvhostuserclient0.log 2>&1 & + +dnl Give settling time to the testpmd processes - NOTE: this is bad form. +sleep 10 + +dnl Check whether TSO is turned off (host side) +AT_CHECK([awk '/negotiated Virtio features/ {a=$NF} END{print a}' \ + $OVS_RUNDIR/ovs-vswitchd.log],[],[stdout]) +AT_CHECK([printf "%X" $(( $(cat stdout) & ((1<<0)|(1<<11)|(1<<12)) ))],[],[0]) + +dnl Check whether TSO is turned off (guest side) +AT_CHECK([awk 'BEGIN{n=0} /Per Port/ && /(TCP|UDP)_CKSUM/ && /TCP_TSO/ {n++} END{printf n}' \ + $OVS_RUNDIR/testpmd-dpdkvhostuserclient0.log],[0],[0]) + +dnl Clean up the testpmd now +pkill -f -x -9 'tail -f /dev/null' + +dnl Clean up +AT_CHECK([ovs-vsctl del-port br10 dpdkvhostuserclient0], [], [stdout], [stderr]) + +dnl Add vhostuser port (server mode) +AT_CHECK([ovs-vsctl add-port br10 dpdkvhostuser0 -- set Interface dpdkvhostuser0 \ + type=dpdkvhostuser], [], + [stdout], [stderr]) +AT_CHECK([ovs-vsctl show], [], [stdout]) + +dnl Execute testpmd in background +on_exit "pkill -f -x -9 'tail -f /dev/null'" +AT_CHECK([echo "show device info all" > CMDFILE]) +AT_CHECK([echo "show port 0 tx_offload capabilities" >> CMDFILE]) +tail -f /dev/null | testpmd --no-pci \ + --socket-mem="$(cat NUMA_NODE)" --socket-limit="$(cat NUMA_NODE)" \ + --vdev="net_virtio_user,path=$OVS_RUNDIR/dpdkvhostuser0" \ + --vdev="net_tap0,iface=tap0" --file-prefix page0 \ + --single-file-segments -- --cmdline-file=CMDFILE \ + -a >$OVS_RUNDIR/testpmd-dpdkvhostuser0.log 2>&1 & + +dnl Give settling time to the testpmd processes - NOTE: this is bad form. +sleep 10 + +dnl Check whether TSO is turned off (host side) +AT_CHECK([awk '/negotiated Virtio features/ {a=$NF} END{print a}' \ + $OVS_RUNDIR/ovs-vswitchd.log],[],[stdout]) +AT_CHECK([printf "%X" $(( $(cat stdout) & ((1<<0)|(1<<11)|(1<<12)) ))],[],[0]) + +dnl Check whether TSO is turned off (guest side) +AT_CHECK([awk 'BEGIN{n=0} /Per Port/ && /(TCP|UDP)_CKSUM/ && /TCP_TSO/ {n++} END{printf n}' \ + $OVS_RUNDIR/testpmd-dpdkvhostuser0.log],[0],[0]) + +dnl Clean up the testpmd now +pkill -f -x -9 'tail -f /dev/null' + +dnl Clean up +AT_CHECK([ovs-vsctl del-port br10 dpdkvhostuser0], [], [stdout], [stderr]) +OVS_VSWITCHD_STOP() +OVS_DPDK_HUGEPAGE_CLEANUP([rtemap_*]) +OVS_DPDK_HUGEPAGE_CLEANUP([page0map_0]) +AT_CLEANUP +dnl --------------------------------------------------------------------------