From patchwork Fri Apr 20 13:46:27 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Marcin Rybka X-Patchwork-Id: 901943 X-Patchwork-Delegate: ian.stokes@intel.com Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=openvswitch.org (client-ip=140.211.169.12; helo=mail.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=intel.com Received: from mail.linuxfoundation.org (mail.linuxfoundation.org [140.211.169.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 40SHg62h83z9s08 for ; Sat, 21 Apr 2018 00:05:05 +1000 (AEST) Received: from mail.linux-foundation.org (localhost [127.0.0.1]) by mail.linuxfoundation.org (Postfix) with ESMTP id 2E43ED1B; Fri, 20 Apr 2018 14:05:04 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@mail.linuxfoundation.org Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 407D3D13 for ; Fri, 20 Apr 2018 14:05:03 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id F27EE6A2 for ; Fri, 20 Apr 2018 14:05:01 +0000 (UTC) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by orsmga102.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 20 Apr 2018 07:05:00 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.49,301,1520924400"; d="scan'208";a="47757099" Received: from irvmail001.ir.intel.com ([163.33.26.43]) by fmsmga004.fm.intel.com with ESMTP; 20 Apr 2018 07:04:59 -0700 Received: from silpixa00389819.ir.intel.com (silpixa00389819.ir.intel.com [10.237.219.152]) by irvmail001.ir.intel.com (8.14.3/8.13.6/MailSET/Hub) with ESMTP id w3KE4wuw003047; Fri, 20 Apr 2018 15:04:58 +0100 From: Marcin Rybka To: dev@openvswitch.org Date: Fri, 20 Apr 2018 14:46:27 +0100 Message-Id: <1524231987-75856-1-git-send-email-marcinx.rybka@intel.com> X-Mailer: git-send-email 1.8.3.1 X-Spam-Status: No, score=-6.9 required=5.0 tests=BAYES_00,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Cc: Marcin Rybka Subject: [ovs-dev] [PATCH v5] tests: Add system-dpdk-testsuite X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: ovs-dev-bounces@openvswitch.org Errors-To: ovs-dev-bounces@openvswitch.org New OVS-DPDK testsuite, which can be launched via `make check-dpdk`, tests OVS using a DPDK datapath. The testsuite contains already initial tests: 1. EAL init 2. Add standard DPDK PHY port 3. Add vhost-user-client port Signed-off-by: Marcin Rybka --- Ver.5 updates: - updated documentation and NEWS file Ver.4 updates: - now works with DPDK 17.11.1 - added entry in NEWS - better hugepages allocation --- Documentation/topics/testing.rst | 23 ++++++++++++++ NEWS | 3 ++ tests/automake.mk | 17 ++++++++++ tests/system-dpdk-macros.at | 56 +++++++++++++++++++++++++++++++++ tests/system-dpdk-testsuite.at | 25 +++++++++++++++ tests/system-dpdk.at | 67 ++++++++++++++++++++++++++++++++++++++++ 6 files changed, 191 insertions(+) create mode 100644 tests/system-dpdk-macros.at create mode 100644 tests/system-dpdk-testsuite.at create mode 100644 tests/system-dpdk.at diff --git a/Documentation/topics/testing.rst b/Documentation/topics/testing.rst index 5dcf446..325c106 100644 --- a/Documentation/topics/testing.rst +++ b/Documentation/topics/testing.rst @@ -297,6 +297,29 @@ To invoke the datapath testsuite with the userspace datapath, run:: The results of the testsuite are in ``tests/system-userspace-testsuite.dir``. +DPDK datapath +''''''''''''' + +To test :doc:`/intro/install/dpdk` (i.e., the build was configured with +``--with-dpdk``, the DPDK is installed), run the testsuite and generate +a report by using the ``check-dpdk`` target:: + + # make check-dpdk + +or if you are not a root, but a sudo user:: + + $ sudo -E make check-dpdk + +To see a list of all the available tests, run:: + + # make check-dpdk TESTSUITEFLAGS=--list + +These tests require a `DPDK supported NIC`_ and proper DPDK variables +(``DPDK_DIR`` and ``DPDK_BUILD``). Moreover you need to have root privileges, +load the required modules and bind the NIC to the DPDK-compatible driver. + +.. _DPDK supported NIC: http://dpdk.org/doc/nics + Kernel datapath ''''''''''''''' diff --git a/NEWS b/NEWS index cd4ffbb..f6ea0aa 100644 --- a/NEWS +++ b/NEWS @@ -23,6 +23,9 @@ Post-v2.9.0 other IPv4/IPv6-based protocols whenever a reject ACL rule is hit. * ACL match conditions can now match on Port_Groups as well as address sets that are automatically generated by Port_Groups. + - DPDK: + * New 'check-dpdk' Makefile target to run a new system testsuite. + See Testing topic for the details. v2.9.0 - 19 Feb 2018 -------------------- diff --git a/tests/automake.mk b/tests/automake.mk index 589263b..c420b29 100644 --- a/tests/automake.mk +++ b/tests/automake.mk @@ -5,10 +5,12 @@ EXTRA_DIST += \ $(SYSTEM_KMOD_TESTSUITE_AT) \ $(SYSTEM_USERSPACE_TESTSUITE_AT) \ $(SYSTEM_OFFLOADS_TESTSUITE_AT) \ + $(SYSTEM_DPDK_TESTSUITE_AT) \ $(TESTSUITE) \ $(SYSTEM_KMOD_TESTSUITE) \ $(SYSTEM_USERSPACE_TESTSUITE) \ $(SYSTEM_OFFLOADS_TESTSUITE) \ + $(SYSTEM_DPDK_TESTSUITE) \ tests/atlocal.in \ $(srcdir)/package.m4 \ $(srcdir)/tests/testsuite \ @@ -128,6 +130,12 @@ SYSTEM_OFFLOADS_TESTSUITE_AT = \ tests/system-offloads-traffic.at \ tests/system-offloads-testsuite.at +SYSTEM_DPDK_TESTSUITE_AT = \ + tests/system-common-macros.at \ + tests/system-dpdk-macros.at \ + tests/system-dpdk-testsuite.at \ + tests/system-dpdk.at + check_SCRIPTS += tests/atlocal TESTSUITE = $(srcdir)/tests/testsuite @@ -135,6 +143,7 @@ TESTSUITE_PATCH = $(srcdir)/tests/testsuite.patch SYSTEM_KMOD_TESTSUITE = $(srcdir)/tests/system-kmod-testsuite SYSTEM_USERSPACE_TESTSUITE = $(srcdir)/tests/system-userspace-testsuite SYSTEM_OFFLOADS_TESTSUITE = $(srcdir)/tests/system-offloads-testsuite +SYSTEM_DPDK_TESTSUITE = $(srcdir)/tests/system-dpdk-testsuite DISTCLEANFILES += tests/atconfig tests/atlocal AUTOTEST_PATH = utilities:vswitchd:ovsdb:vtep:tests:$(PTHREAD_WIN32_DIR_DLL):$(SSL_DIR):ovn/controller-vtep:ovn/northd:ovn/utilities:ovn/controller @@ -258,6 +267,10 @@ check-offloads: all set $(SHELL) '$(SYSTEM_OFFLOADS_TESTSUITE)' -C tests AUTOTEST_PATH='$(AUTOTEST_PATH)' $(TESTSUITEFLAGS) -j1; \ "$$@" || (test X'$(RECHECK)' = Xyes && "$$@" --recheck) +check-dpdk: all + set $(SHELL) '$(SYSTEM_DPDK_TESTSUITE)' -C tests AUTOTEST_PATH='$(AUTOTEST_PATH)' $(TESTSUITEFLAGS) -j1; \ + "$$@" || (test X'$(RECHECK)' = Xyes && "$$@" --recheck) + clean-local: test ! -f '$(TESTSUITE)' || $(SHELL) '$(TESTSUITE)' -C tests --clean @@ -286,6 +299,10 @@ $(SYSTEM_OFFLOADS_TESTSUITE): package.m4 $(SYSTEM_TESTSUITE_AT) $(SYSTEM_OFFLOAD $(AM_V_GEN)$(AUTOTEST) -I '$(srcdir)' -o $@.tmp $@.at $(AM_V_at)mv $@.tmp $@ +$(SYSTEM_DPDK_TESTSUITE): package.m4 $(SYSTEM_TESTSUITE_AT) $(SYSTEM_DPDK_TESTSUITE_AT) $(COMMON_MACROS_AT) + $(AM_V_GEN)$(AUTOTEST) -I '$(srcdir)' -o $@.tmp $@.at + $(AM_V_at)mv $@.tmp $@ + # The `:;' works around a Bash 3.2 bug when the output is not writeable. $(srcdir)/package.m4: $(top_srcdir)/configure.ac $(AM_V_GEN):;{ \ diff --git a/tests/system-dpdk-macros.at b/tests/system-dpdk-macros.at new file mode 100644 index 0000000..0762ee0 --- /dev/null +++ b/tests/system-dpdk-macros.at @@ -0,0 +1,56 @@ +# OVS_DPDK_PRE_CHECK() +# +# Check prerequisites for DPDK tests. Following settings are checked: +# - Hugepages +# - UIO driver +# +m4_define([OVS_DPDK_PRE_CHECK], + [dnl Check Hugepages + AT_CHECK([cat /proc/meminfo], [], [stdout]) + AT_CHECK([grep HugePages_ stdout], [], [stdout]) + AT_CHECK([mount], [], [stdout]) + AT_CHECK([grep 'hugetlbfs' stdout], [], [stdout], []) + + dnl Check if VFIO or UIO driver is loaded + AT_CHECK([lsmod | grep -E "igb_uio|vfio"], [], [stdout]) + + dnl Find PCI address candidate, skip if there is no DPDK-compatible NIC + AT_CHECK([$DPDK_DIR/usertools/dpdk-devbind.py -s | head -n +4 | tail -1], [], [stdout]) + AT_CHECK([cat stdout | cut -d" " -s -f1 > PCI_ADDR]) + AT_CHECK([test -s PCI_ADDR || exit 77]) +]) + + +# OVS_DPDK_START() +# +# Create an empty database and start ovsdb-server. Add special configuration +# dpdk-init to enable DPDK functionality. Start ovs-vswitchd connected to that +# database using system devices (no dummies). +# +m4_define([OVS_DPDK_START], + [dnl Create database. + AT_CHECK([touch .conf.db.~lock~]) + AT_CHECK([ovsdb-tool create conf.db $abs_top_srcdir/vswitchd/vswitch.ovsschema]) + + dnl Start ovsdb-server. + AT_CHECK([ovsdb-server --detach --no-chdir --pidfile --log-file --remote=punix:$OVS_RUNDIR/db.sock], [0], [stdout], [stderr]) + on_exit "kill `cat ovsdb-server.pid`" + AT_CHECK([[sed < stderr ' +/vlog|INFO|opened log file/d +/ovsdb_server|INFO|ovsdb-server (Open vSwitch)/d']]) + AT_CAPTURE_FILE([ovsdb-server.log]) + + dnl Initialize database. + AT_CHECK([ovs-vsctl --no-wait init]) + + dnl Enable DPDK functionality + AT_CHECK([ovs-vsctl --no-wait set Open_vSwitch . other_config:dpdk-init=true]) + AT_CHECK([lscpu], [], [stdout]) + AT_CHECK([cat stdout | grep "NUMA node(s)" | awk '{c=1; while (c++<$(3)) {printf "1024,"}; print "1024"}' > SOCKET_MEM]) + AT_CHECK([ovs-vsctl --no-wait set Open_vSwitch . other_config:dpdk-socket-mem="$(cat SOCKET_MEM)"]) + + dnl Start ovs-vswitchd. + AT_CHECK([ovs-vswitchd --detach --no-chdir --pidfile --log-file -vvconn -vofproto_dpif -vunixctl], [0], [stdout], [stderr]) + AT_CAPTURE_FILE([ovs-vswitchd.log]) + on_exit "kill_ovs_vswitchd `cat ovs-vswitchd.pid`" +]) diff --git a/tests/system-dpdk-testsuite.at b/tests/system-dpdk-testsuite.at new file mode 100644 index 0000000..382f09e --- /dev/null +++ b/tests/system-dpdk-testsuite.at @@ -0,0 +1,25 @@ +AT_INIT + +AT_COPYRIGHT([Copyright (c) 2017 Intel Corporation + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at: + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License.]) + +m4_ifdef([AT_COLOR_TESTS], [AT_COLOR_TESTS]) + +m4_include([tests/ovs-macros.at]) +m4_include([tests/ovsdb-macros.at]) +m4_include([tests/ofproto-macros.at]) +m4_include([tests/system-common-macros.at]) +m4_include([tests/system-dpdk-macros.at]) + +m4_include([tests/system-dpdk.at]) diff --git a/tests/system-dpdk.at b/tests/system-dpdk.at new file mode 100644 index 0000000..1f1839e --- /dev/null +++ b/tests/system-dpdk.at @@ -0,0 +1,67 @@ +AT_BANNER([OVS-DPDK unit tests]) + +dnl -------------------------------------------------------------------------- +dnl Check if EAL init is successfull +AT_SETUP([OVS-DPDK datapath - EAL init]) +AT_KEYWORDS([dpdk]) +dnl OVS_DPDK_PRE_CHECK() +OVS_DPDK_START() +AT_CHECK([grep "DPDK Enabled - initializing..." ovs-vswitchd.log], [], [stdout]) +AT_CHECK([grep "EAL" ovs-vswitchd.log], [], [stdout]) +AT_CHECK([grep "DPDK Enabled - initialized" ovs-vswitchd.log], [], [stdout]) +OVS_VSWITCHD_STOP("/Global register is changed during/d") +AT_CLEANUP +dnl -------------------------------------------------------------------------- + + + +dnl -------------------------------------------------------------------------- +dnl Add standard DPDK PHY port +AT_SETUP([OVS-DPDK datapath - add standard DPDK port]) +AT_KEYWORDS([dpdk]) + +OVS_DPDK_PRE_CHECK() +OVS_DPDK_START() + +dnl Add userspace bridge and attach it to OVS +AT_CHECK([ovs-vsctl add-br br10 -- set bridge br10 datapath_type=netdev]) +AT_CHECK([ovs-vsctl add-port br10 phy0 -- set Interface phy0 type=dpdk options:dpdk-devargs=$(cat PCI_ADDR)], [], [stdout], [stderr]) +AT_CHECK([ovs-vsctl show], [], [stdout]) +sleep 2 + +dnl Clean up +AT_CHECK([ovs-vsctl del-port br10 phy0], [], [stdout], [stderr]) +OVS_VSWITCHD_STOP("/does not exist. The Open vSwitch kernel module is probably not loaded./d +/Failed to enable flow control/d +/Global register is changed during/d") +AT_CLEANUP +dnl -------------------------------------------------------------------------- + + + +dnl -------------------------------------------------------------------------- +dnl Add vhost-user-client port +AT_SETUP([OVS-DPDK datapath - add vhost-user-client port]) +AT_KEYWORDS([dpdk]) + +OVS_DPDK_START() + +dnl Add userspace bridge and attach it to OVS +AT_CHECK([ovs-vsctl add-br br10 -- set bridge br10 datapath_type=netdev]) +AT_CHECK([ovs-vsctl add-port br10 dpdkvhostuserclient0 -- set Interface dpdkvhostuserclient0 type=dpdkvhostuserclient options:vhost-server-path=/tmp/dpdkvhostclient0], [], [stdout], [stderr]) +AT_CHECK([ovs-vsctl show], [], [stdout]) +sleep 2 + +dnl Parse log file +AT_CHECK([grep "VHOST_CONFIG: vhost-user client: socket created" ovs-vswitchd.log], [], [stdout]) +AT_CHECK([grep "vHost User device 'dpdkvhostuserclient0' created in 'client' mode, using client socket" ovs-vswitchd.log], [], [stdout]) +AT_CHECK([grep "VHOST_CONFIG: /tmp/dpdkvhostclient0: reconnecting..." ovs-vswitchd.log], [], [stdout]) + +dnl Clean up +AT_CHECK([ovs-vsctl del-port br10 dpdkvhostuserclient0], [], [stdout], [stderr]) +OVS_VSWITCHD_STOP("/does not exist. The Open vSwitch kernel module is probably not loaded./d +/Failed to enable flow control/d +/failed to connect to \/tmp\/dpdkvhostclient0: No such file or directory/d +/Global register is changed during/d") +AT_CLEANUP +dnl --------------------------------------------------------------------------