diff mbox series

[ovs-dev,v4] tests: Add system-dpdk-testsuite

Message ID 1523892968-112044-1-git-send-email-marcinx.rybka@intel.com
State Changes Requested
Delegated to: Ian Stokes
Headers show
Series [ovs-dev,v4] tests: Add system-dpdk-testsuite | expand

Commit Message

Marcin Rybka April 16, 2018, 3:36 p.m. UTC 
New OVS-DPDK testsuite, which can be launched via `make check-dpdk`,
tests OVS using a DPDK datapath. The testsuite contains already
initial tests:
 1. EAL init
 2. Add standard DPDK PHY port
 3. Add vhost-user-client port

Signed-off-by: Marcin Rybka <marcinx.rybka@intel.com>

---
Ver.4 updates:
 - now works with DPDK 17.11.1
 - added entry in NEWS
 - better hugepages allocation
---
 Documentation/topics/testing.rst | 19 ++++++++++++
 NEWS                             |  2 ++
 tests/automake.mk                | 17 ++++++++++
 tests/system-dpdk-macros.at      | 56 +++++++++++++++++++++++++++++++++
 tests/system-dpdk-testsuite.at   | 25 +++++++++++++++
 tests/system-dpdk.at             | 67 ++++++++++++++++++++++++++++++++++++++++
 6 files changed, 186 insertions(+)
 create mode 100644 tests/system-dpdk-macros.at
 create mode 100644 tests/system-dpdk-testsuite.at
 create mode 100644 tests/system-dpdk.at

Comments

Stokes, Ian April 17, 2018, 6:50 p.m. UTC | #1 
> To: dev@openvswitch.org
> Cc: Rybka, MarcinX <marcinx.rybka@intel.com>
> Subject: [ovs-dev] [PATCH v4] tests: Add system-dpdk-testsuite
> 
> New OVS-DPDK testsuite, which can be launched via `make check-dpdk`, tests
> OVS using a DPDK datapath. The testsuite contains already initial tests:
>  1. EAL init
>  2. Add standard DPDK PHY port
>  3. Add vhost-user-client port
> 
> Signed-off-by: Marcin Rybka <marcinx.rybka@intel.com>
> 

Thanks for this Marcin, comments inline below. 
> ---
> Ver.4 updates:
>  - now works with DPDK 17.11.1
>  - added entry in NEWS
>  - better hugepages allocation
> ---
>  Documentation/topics/testing.rst | 19 ++++++++++++
>  NEWS                             |  2 ++
>  tests/automake.mk                | 17 ++++++++++
>  tests/system-dpdk-macros.at      | 56 +++++++++++++++++++++++++++++++++
>  tests/system-dpdk-testsuite.at   | 25 +++++++++++++++
>  tests/system-dpdk.at             | 67
> ++++++++++++++++++++++++++++++++++++++++
>  6 files changed, 186 insertions(+)
>  create mode 100644 tests/system-dpdk-macros.at  create mode 100644
> tests/system-dpdk-testsuite.at  create mode 100644 tests/system-dpdk.at
> 
> diff --git a/Documentation/topics/testing.rst
> b/Documentation/topics/testing.rst
> index 5dcf446..bdc7225 100644
> --- a/Documentation/topics/testing.rst
> +++ b/Documentation/topics/testing.rst
> @@ -297,6 +297,25 @@ To invoke the datapath testsuite with the userspace
> datapath, run::
> 
>  The results of the testsuite are in ``tests/system-userspace-
> testsuite.dir``.
> 
> +DPDK datapath
> +'''''''''''''
> +
> +To test :doc:`/intro/install/dpdk` (i.e., the build was configured with
> +``--with-dpdk``, the DPDK is installed), run the testsuite and generate
> +a report by using the ``check-dpdk`` target::
> +
> +    $ make check-dpdk
> +
> +To see a list of all the available tests, run::
> +
> +    $ make check-dpdk TESTSUITEFLAGS=--list
> +
> +These tests require a `DPDK supported NIC`_ and proper DPDK variables
> +(``DPDK_DIR`` and ``DPDK_BUILD``). Moreover you need to have root
> +privileges, load the required modules and bind the NIC to the DPDK-
> compatible driver.
> +
> +.. _DPDK supported NIC: http://dpdk.org/doc/nics
> +
>  Kernel datapath
>  '''''''''''''''
> 
> diff --git a/NEWS b/NEWS
> index 58a7b58..7bad608 100644
> --- a/NEWS
> +++ b/NEWS
> @@ -21,6 +21,8 @@ Post-v2.9.0
>         and reply with a RST for TCP or ICMPv4/ICMPv6 unreachable message
> for
>         other IPv4/IPv6-based protocols whenever a reject ACL rule is hit.
>       * ACL match conditions can now match on Port_Groups.
> +   - New 'check-dpdk' Makefile target to run a new system testsuite.
> +     See Testing topic for the details.

I think this should be added under a specific DPDK header in NEWS.

- DPDK
  * Add 'check-dpdk' Makefile target to run dpdk specific testsuite.

Thoughts?

> 
>  v2.9.0 - 19 Feb 2018
>  --------------------
> diff --git a/tests/automake.mk b/tests/automake.mk index d9292e8..e52531a
> 100644
> --- a/tests/automake.mk
> +++ b/tests/automake.mk
> @@ -5,10 +5,12 @@ EXTRA_DIST += \
>  	$(SYSTEM_KMOD_TESTSUITE_AT) \
>  	$(SYSTEM_USERSPACE_TESTSUITE_AT) \
>  	$(SYSTEM_OFFLOADS_TESTSUITE_AT) \
> +	$(SYSTEM_DPDK_TESTSUITE_AT) \
>  	$(TESTSUITE) \
>  	$(SYSTEM_KMOD_TESTSUITE) \
>  	$(SYSTEM_USERSPACE_TESTSUITE) \
>  	$(SYSTEM_OFFLOADS_TESTSUITE) \
> +	$(SYSTEM_DPDK_TESTSUITE) \
>  	tests/atlocal.in \
>  	$(srcdir)/package.m4 \
>  	$(srcdir)/tests/testsuite \
> @@ -128,6 +130,12 @@ SYSTEM_OFFLOADS_TESTSUITE_AT = \
>  	tests/system-offloads-traffic.at \
>  	tests/system-offloads-testsuite.at
> 
> +SYSTEM_DPDK_TESTSUITE_AT = \
> +	tests/system-common-macros.at \
> +	tests/system-dpdk-macros.at \
> +	tests/system-dpdk-testsuite.at \
> +	tests/system-dpdk.at
> +
>  check_SCRIPTS += tests/atlocal
> 
>  TESTSUITE = $(srcdir)/tests/testsuite
> @@ -135,6 +143,7 @@ TESTSUITE_PATCH = $(srcdir)/tests/testsuite.patch
> SYSTEM_KMOD_TESTSUITE = $(srcdir)/tests/system-kmod-testsuite
>  SYSTEM_USERSPACE_TESTSUITE = $(srcdir)/tests/system-userspace-testsuite
>  SYSTEM_OFFLOADS_TESTSUITE = $(srcdir)/tests/system-offloads-testsuite
> +SYSTEM_DPDK_TESTSUITE = $(srcdir)/tests/system-dpdk-testsuite
>  DISTCLEANFILES += tests/atconfig tests/atlocal
> 
>  AUTOTEST_PATH =
> utilities:vswitchd:ovsdb:vtep:tests:$(PTHREAD_WIN32_DIR_DLL):$(SSL_DIR):ov
> n/controller-vtep:ovn/northd:ovn/utilities:ovn/controller
> @@ -258,6 +267,10 @@ check-offloads: all
>  	set $(SHELL) '$(SYSTEM_OFFLOADS_TESTSUITE)' -C tests
> AUTOTEST_PATH='$(AUTOTEST_PATH)' $(TESTSUITEFLAGS) -j1; \
>  	"$$@" || (test X'$(RECHECK)' = Xyes && "$$@" --recheck)
> 
> +check-dpdk: all
> +	set $(SHELL) '$(SYSTEM_DPDK_TESTSUITE)' -C tests
> AUTOTEST_PATH='$(AUTOTEST_PATH)' $(TESTSUITEFLAGS) -j1; \
> +	"$$@" || (test X'$(RECHECK)' = Xyes && "$$@" --recheck)
> +
>  clean-local:
>  	test ! -f '$(TESTSUITE)' || $(SHELL) '$(TESTSUITE)' -C tests --clean
> 
> @@ -286,6 +299,10 @@ $(SYSTEM_OFFLOADS_TESTSUITE): package.m4
> $(SYSTEM_TESTSUITE_AT) $(SYSTEM_OFFLOAD
>  	$(AM_V_GEN)$(AUTOTEST) -I '$(srcdir)' -o $@.tmp $@.at
>  	$(AM_V_at)mv $@.tmp $@
> 
> +$(SYSTEM_DPDK_TESTSUITE): package.m4 $(SYSTEM_TESTSUITE_AT)
> $(SYSTEM_DPDK_TESTSUITE_AT) $(COMMON_MACROS_AT)
> +	$(AM_V_GEN)$(AUTOTEST) -I '$(srcdir)' -o $@.tmp $@.at
> +	$(AM_V_at)mv $@.tmp $@
> +
>  # The `:;' works around a Bash 3.2 bug when the output is not writeable.
>  $(srcdir)/package.m4: $(top_srcdir)/configure.ac
>  	$(AM_V_GEN):;{ \
> diff --git a/tests/system-dpdk-macros.at b/tests/system-dpdk-macros.at new
> file mode 100644 index 0000000..0762ee0
> --- /dev/null
> +++ b/tests/system-dpdk-macros.at
> @@ -0,0 +1,56 @@
> +# OVS_DPDK_PRE_CHECK()
> +#
> +# Check prerequisites for DPDK tests. Following settings are checked:
> +#  - Hugepages
> +#  - UIO driver
> +#
> +m4_define([OVS_DPDK_PRE_CHECK],
> +  [dnl Check Hugepages
> +   AT_CHECK([cat /proc/meminfo], [], [stdout])
> +   AT_CHECK([grep HugePages_ stdout], [], [stdout])
> +   AT_CHECK([mount], [], [stdout])
> +   AT_CHECK([grep 'hugetlbfs' stdout], [], [stdout], [])
> +
> +   dnl Check if VFIO or UIO driver is loaded
> +   AT_CHECK([lsmod | grep -E "igb_uio|vfio"], [], [stdout])
> +
> +   dnl Find PCI address candidate, skip if there is no DPDK-compatible
> NIC
> +   AT_CHECK([$DPDK_DIR/usertools/dpdk-devbind.py -s | head -n +4 | tail -
> 1], [], [stdout])

In testing I found I had to explicitly set DPDK_DIR as part of /etc/environment.

Defining it as part of .profile or even exporting from the bash shell explicitly before running the tests would result in the DPDK_DIR being empty.

Not sure why this is. It could be specific to my setup. Have you thought about passing the DPDK_DIR as an optional argument as part of running ./configure rather than relying on environmental variable declarations?

> +   AT_CHECK([cat stdout | cut -d" " -s -f1 > PCI_ADDR])
> +   AT_CHECK([test -s PCI_ADDR || exit 77])
> +])
> +
> +
> +# OVS_DPDK_START()
> +#
> +# Create an empty database and start ovsdb-server. Add special
> +configuration # dpdk-init to enable DPDK functionality. Start
> +ovs-vswitchd connected to that # database using system devices (no
> dummies).
> +#
> +m4_define([OVS_DPDK_START],
> +  [dnl Create database.
> +   AT_CHECK([touch .conf.db.~lock~])
> +   AT_CHECK([ovsdb-tool create conf.db
> +$abs_top_srcdir/vswitchd/vswitch.ovsschema])
> +
> +   dnl Start ovsdb-server.
> +   AT_CHECK([ovsdb-server --detach --no-chdir --pidfile --log-file --
> remote=punix:$OVS_RUNDIR/db.sock], [0], [stdout], [stderr])
> +   on_exit "kill `cat ovsdb-server.pid`"
> +   AT_CHECK([[sed < stderr '
> +/vlog|INFO|opened log file/d
> +/ovsdb_server|INFO|ovsdb-server (Open vSwitch)/d']])
> +   AT_CAPTURE_FILE([ovsdb-server.log])
> +
> +   dnl Initialize database.
> +   AT_CHECK([ovs-vsctl --no-wait init])
> +
> +   dnl Enable DPDK functionality
> +   AT_CHECK([ovs-vsctl --no-wait set Open_vSwitch . other_config:dpdk-
> init=true])
> +   AT_CHECK([lscpu], [], [stdout])
> +   AT_CHECK([cat stdout | grep "NUMA node(s)" | awk '{c=1; while
> (c++<$(3)) {printf "1024,"}; print "1024"}' > SOCKET_MEM])
> +   AT_CHECK([ovs-vsctl --no-wait set Open_vSwitch .
> + other_config:dpdk-socket-mem="$(cat SOCKET_MEM)"])
> +
> +   dnl Start ovs-vswitchd.
> +   AT_CHECK([ovs-vswitchd --detach --no-chdir --pidfile --log-file -
> vvconn -vofproto_dpif -vunixctl], [0], [stdout], [stderr])
> +   AT_CAPTURE_FILE([ovs-vswitchd.log])
> +   on_exit "kill_ovs_vswitchd `cat ovs-vswitchd.pid`"
> +])
> diff --git a/tests/system-dpdk-testsuite.at b/tests/system-dpdk-
> testsuite.at new file mode 100644 index 0000000..382f09e
> --- /dev/null
> +++ b/tests/system-dpdk-testsuite.at
> @@ -0,0 +1,25 @@
> +AT_INIT
> +
> +AT_COPYRIGHT([Copyright (c) 2017 Intel Corporation
> +
> +Licensed under the Apache License, Version 2.0 (the "License"); you may
> +not use this file except in compliance with the License.
> +You may obtain a copy of the License at:
> +
> +    http://www.apache.org/licenses/LICENSE-2.0
> +
> +Unless required by applicable law or agreed to in writing, software
> +distributed under the License is distributed on an "AS IS" BASIS,
> +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
> +See the License for the specific language governing permissions and
> +limitations under the License.])
> +
> +m4_ifdef([AT_COLOR_TESTS], [AT_COLOR_TESTS])
> +
> +m4_include([tests/ovs-macros.at])
> +m4_include([tests/ovsdb-macros.at])
> +m4_include([tests/ofproto-macros.at])
> +m4_include([tests/system-common-macros.at])
> +m4_include([tests/system-dpdk-macros.at])
> +
> +m4_include([tests/system-dpdk.at])
> diff --git a/tests/system-dpdk.at b/tests/system-dpdk.at new file mode
> 100644 index 0000000..1f1839e
> --- /dev/null
> +++ b/tests/system-dpdk.at
> @@ -0,0 +1,67 @@
> +AT_BANNER([OVS-DPDK unit tests])
> +
> +dnl
> +-----------------------------------------------------------------------
> +---
> +dnl Check if EAL init is successfull
> +AT_SETUP([OVS-DPDK datapath - EAL init])
> +AT_KEYWORDS([dpdk])
> +dnl OVS_DPDK_PRE_CHECK()
> +OVS_DPDK_START()
> +AT_CHECK([grep "DPDK Enabled - initializing..." ovs-vswitchd.log], [],
> +[stdout]) AT_CHECK([grep "EAL" ovs-vswitchd.log], [], [stdout])
> +AT_CHECK([grep "DPDK Enabled - initialized" ovs-vswitchd.log], [],
> +[stdout]) OVS_VSWITCHD_STOP("/Global register is changed during/d")
> +AT_CLEANUP dnl
> +-----------------------------------------------------------------------
> +---
> +
> +
> +
> +dnl
> +-----------------------------------------------------------------------
> +---
> +dnl Add standard DPDK PHY port
> +AT_SETUP([OVS-DPDK datapath - add standard DPDK port])
> +AT_KEYWORDS([dpdk])
> +
> +OVS_DPDK_PRE_CHECK()
> +OVS_DPDK_START()
> +
> +dnl Add userspace bridge and attach it to OVS AT_CHECK([ovs-vsctl
> +add-br br10 -- set bridge br10 datapath_type=netdev])
> +AT_CHECK([ovs-vsctl add-port br10 phy0 -- set Interface phy0 type=dpdk
> +options:dpdk-devargs=$(cat PCI_ADDR)], [], [stdout], [stderr])
> +AT_CHECK([ovs-vsctl show], [], [stdout]) sleep 2
> +
> +dnl Clean up
> +AT_CHECK([ovs-vsctl del-port br10 phy0], [], [stdout], [stderr])
> +OVS_VSWITCHD_STOP("/does not exist. The Open vSwitch kernel module is
> +probably not loaded./d /Failed to enable flow control/d /Global
> +register is changed during/d") AT_CLEANUP dnl
> +-----------------------------------------------------------------------
> +---
> +
> +
> +
> +dnl
> +-----------------------------------------------------------------------
> +---
> +dnl Add vhost-user-client port
> +AT_SETUP([OVS-DPDK datapath - add vhost-user-client port])
> +AT_KEYWORDS([dpdk])
> +
> +OVS_DPDK_START()
> +
> +dnl Add userspace bridge and attach it to OVS AT_CHECK([ovs-vsctl
> +add-br br10 -- set bridge br10 datapath_type=netdev])
> +AT_CHECK([ovs-vsctl add-port br10 dpdkvhostuserclient0 -- set Interface
> +dpdkvhostuserclient0 type=dpdkvhostuserclient
> +options:vhost-server-path=/tmp/dpdkvhostclient0], [], [stdout],
> +[stderr]) AT_CHECK([ovs-vsctl show], [], [stdout]) sleep 2
> +
> +dnl Parse log file
> +AT_CHECK([grep "VHOST_CONFIG: vhost-user client: socket created"
> +ovs-vswitchd.log], [], [stdout]) AT_CHECK([grep "vHost User device
> +'dpdkvhostuserclient0' created in 'client' mode, using client socket"
> +ovs-vswitchd.log], [], [stdout]) AT_CHECK([grep "VHOST_CONFIG:
> +/tmp/dpdkvhostclient0: reconnecting..." ovs-vswitchd.log], [],
> +[stdout])
> +
> +dnl Clean up
> +AT_CHECK([ovs-vsctl del-port br10 dpdkvhostuserclient0], [], [stdout],
> +[stderr]) OVS_VSWITCHD_STOP("/does not exist. The Open vSwitch kernel
> +module is probably not loaded./d /Failed to enable flow control/d
> +/failed to connect to \/tmp\/dpdkvhostclient0: No such file or
> +directory/d /Global register is changed during/d") AT_CLEANUP dnl
> +-----------------------------------------------------------------------
> +---
> --
> 1.9.3
> 
> _______________________________________________
> dev mailing list
> dev@openvswitch.org
> https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Marcin Rybka April 18, 2018, 3:21 p.m. UTC | #2 
Hi Ian,

Appreciate your comments and findings. See my replies inline.


> -----Original Message-----
> From: Stokes, Ian
> Sent: Tuesday, April 17, 2018 8:50 PM
> To: Rybka, MarcinX <marcinx.rybka@intel.com>; dev@openvswitch.org
> Cc: Rybka, MarcinX <marcinx.rybka@intel.com>
> Subject: RE: [ovs-dev] [PATCH v4] tests: Add system-dpdk-testsuite
> 
> > To: dev@openvswitch.org
> > Cc: Rybka, MarcinX <marcinx.rybka@intel.com>
> > Subject: [ovs-dev] [PATCH v4] tests: Add system-dpdk-testsuite
> >
> > New OVS-DPDK testsuite, which can be launched via `make check-dpdk`,
> > tests OVS using a DPDK datapath. The testsuite contains already initial
> tests:
> >  1. EAL init
> >  2. Add standard DPDK PHY port
> >  3. Add vhost-user-client port
> >
> > Signed-off-by: Marcin Rybka <marcinx.rybka@intel.com>
> >
> 
> Thanks for this Marcin, comments inline below.
> > ---
> > Ver.4 updates:
> >  - now works with DPDK 17.11.1
> >  - added entry in NEWS
> >  - better hugepages allocation
> > ---
> >  Documentation/topics/testing.rst | 19 ++++++++++++
> >  NEWS                             |  2 ++
> >  tests/automake.mk                | 17 ++++++++++
> >  tests/system-dpdk-macros.at      | 56
> +++++++++++++++++++++++++++++++++
> >  tests/system-dpdk-testsuite.at   | 25 +++++++++++++++
> >  tests/system-dpdk.at             | 67
> > ++++++++++++++++++++++++++++++++++++++++
> >  6 files changed, 186 insertions(+)
> >  create mode 100644 tests/system-dpdk-macros.at  create mode 100644
> > tests/system-dpdk-testsuite.at  create mode 100644
> > tests/system-dpdk.at
> >
> > diff --git a/Documentation/topics/testing.rst
> > b/Documentation/topics/testing.rst
> > index 5dcf446..bdc7225 100644
> > --- a/Documentation/topics/testing.rst
> > +++ b/Documentation/topics/testing.rst
> > @@ -297,6 +297,25 @@ To invoke the datapath testsuite with the
> > userspace datapath, run::
> >
> >  The results of the testsuite are in ``tests/system-userspace-
> > testsuite.dir``.
> >
> > +DPDK datapath
> > +'''''''''''''
> > +
> > +To test :doc:`/intro/install/dpdk` (i.e., the build was configured
> > +with ``--with-dpdk``, the DPDK is installed), run the testsuite and
> > +generate a report by using the ``check-dpdk`` target::
> > +
> > +    $ make check-dpdk
> > +
> > +To see a list of all the available tests, run::
> > +
> > +    $ make check-dpdk TESTSUITEFLAGS=--list
> > +
> > +These tests require a `DPDK supported NIC`_ and proper DPDK variables
> > +(``DPDK_DIR`` and ``DPDK_BUILD``). Moreover you need to have root
> > +privileges, load the required modules and bind the NIC to the DPDK-
> > compatible driver.
> > +
> > +.. _DPDK supported NIC: http://dpdk.org/doc/nics
> > +
> >  Kernel datapath
> >  '''''''''''''''
> >
> > diff --git a/NEWS b/NEWS
> > index 58a7b58..7bad608 100644
> > --- a/NEWS
> > +++ b/NEWS
> > @@ -21,6 +21,8 @@ Post-v2.9.0
> >         and reply with a RST for TCP or ICMPv4/ICMPv6 unreachable
> > message for
> >         other IPv4/IPv6-based protocols whenever a reject ACL rule is hit.
> >       * ACL match conditions can now match on Port_Groups.
> > +   - New 'check-dpdk' Makefile target to run a new system testsuite.
> > +     See Testing topic for the details.
> 
> I think this should be added under a specific DPDK header in NEWS.
> 
> - DPDK
>   * Add 'check-dpdk' Makefile target to run dpdk specific testsuite.
> 
> Thoughts?

Yes, I can do like this. I was suggested by previous news about new testsuites.

> 
> >
> >  v2.9.0 - 19 Feb 2018
> >  --------------------
> > diff --git a/tests/automake.mk b/tests/automake.mk index
> > d9292e8..e52531a
> > 100644
> > --- a/tests/automake.mk
> > +++ b/tests/automake.mk
> > @@ -5,10 +5,12 @@ EXTRA_DIST += \
> >  	$(SYSTEM_KMOD_TESTSUITE_AT) \
> >  	$(SYSTEM_USERSPACE_TESTSUITE_AT) \
> >  	$(SYSTEM_OFFLOADS_TESTSUITE_AT) \
> > +	$(SYSTEM_DPDK_TESTSUITE_AT) \
> >  	$(TESTSUITE) \
> >  	$(SYSTEM_KMOD_TESTSUITE) \
> >  	$(SYSTEM_USERSPACE_TESTSUITE) \
> >  	$(SYSTEM_OFFLOADS_TESTSUITE) \
> > +	$(SYSTEM_DPDK_TESTSUITE) \
> >  	tests/atlocal.in \
> >  	$(srcdir)/package.m4 \
> >  	$(srcdir)/tests/testsuite \
> > @@ -128,6 +130,12 @@ SYSTEM_OFFLOADS_TESTSUITE_AT = \
> >  	tests/system-offloads-traffic.at \
> >  	tests/system-offloads-testsuite.at
> >
> > +SYSTEM_DPDK_TESTSUITE_AT = \
> > +	tests/system-common-macros.at \
> > +	tests/system-dpdk-macros.at \
> > +	tests/system-dpdk-testsuite.at \
> > +	tests/system-dpdk.at
> > +
> >  check_SCRIPTS += tests/atlocal
> >
> >  TESTSUITE = $(srcdir)/tests/testsuite @@ -135,6 +143,7 @@
> > TESTSUITE_PATCH = $(srcdir)/tests/testsuite.patch
> > SYSTEM_KMOD_TESTSUITE = $(srcdir)/tests/system-kmod-testsuite
> >  SYSTEM_USERSPACE_TESTSUITE =
> > $(srcdir)/tests/system-userspace-testsuite
> >  SYSTEM_OFFLOADS_TESTSUITE = $(srcdir)/tests/system-offloads-testsuite
> > +SYSTEM_DPDK_TESTSUITE = $(srcdir)/tests/system-dpdk-testsuite
> >  DISTCLEANFILES += tests/atconfig tests/atlocal
> >
> >  AUTOTEST_PATH =
> > utilities:vswitchd:ovsdb:vtep:tests:$(PTHREAD_WIN32_DIR_DLL):$(SSL_DIR
> > ):ov n/controller-vtep:ovn/northd:ovn/utilities:ovn/controller
> > @@ -258,6 +267,10 @@ check-offloads: all
> >  	set $(SHELL) '$(SYSTEM_OFFLOADS_TESTSUITE)' -C tests
> > AUTOTEST_PATH='$(AUTOTEST_PATH)' $(TESTSUITEFLAGS) -j1; \
> >  	"$$@" || (test X'$(RECHECK)' = Xyes && "$$@" --recheck)
> >
> > +check-dpdk: all
> > +	set $(SHELL) '$(SYSTEM_DPDK_TESTSUITE)' -C tests
> > AUTOTEST_PATH='$(AUTOTEST_PATH)' $(TESTSUITEFLAGS) -j1; \
> > +	"$$@" || (test X'$(RECHECK)' = Xyes && "$$@" --recheck)
> > +
> >  clean-local:
> >  	test ! -f '$(TESTSUITE)' || $(SHELL) '$(TESTSUITE)' -C tests --clean
> >
> > @@ -286,6 +299,10 @@ $(SYSTEM_OFFLOADS_TESTSUITE): package.m4
> > $(SYSTEM_TESTSUITE_AT) $(SYSTEM_OFFLOAD
> >  	$(AM_V_GEN)$(AUTOTEST) -I '$(srcdir)' -o $@.tmp $@.at
> >  	$(AM_V_at)mv $@.tmp $@
> >
> > +$(SYSTEM_DPDK_TESTSUITE): package.m4 $(SYSTEM_TESTSUITE_AT)
> > $(SYSTEM_DPDK_TESTSUITE_AT) $(COMMON_MACROS_AT)
> > +	$(AM_V_GEN)$(AUTOTEST) -I '$(srcdir)' -o $@.tmp $@.at
> > +	$(AM_V_at)mv $@.tmp $@
> > +
> >  # The `:;' works around a Bash 3.2 bug when the output is not writeable.
> >  $(srcdir)/package.m4: $(top_srcdir)/configure.ac
> >  	$(AM_V_GEN):;{ \
> > diff --git a/tests/system-dpdk-macros.at b/tests/system-dpdk-macros.at
> > new file mode 100644 index 0000000..0762ee0
> > --- /dev/null
> > +++ b/tests/system-dpdk-macros.at
> > @@ -0,0 +1,56 @@
> > +# OVS_DPDK_PRE_CHECK()
> > +#
> > +# Check prerequisites for DPDK tests. Following settings are checked:
> > +#  - Hugepages
> > +#  - UIO driver
> > +#
> > +m4_define([OVS_DPDK_PRE_CHECK],
> > +  [dnl Check Hugepages
> > +   AT_CHECK([cat /proc/meminfo], [], [stdout])
> > +   AT_CHECK([grep HugePages_ stdout], [], [stdout])
> > +   AT_CHECK([mount], [], [stdout])
> > +   AT_CHECK([grep 'hugetlbfs' stdout], [], [stdout], [])
> > +
> > +   dnl Check if VFIO or UIO driver is loaded
> > +   AT_CHECK([lsmod | grep -E "igb_uio|vfio"], [], [stdout])
> > +
> > +   dnl Find PCI address candidate, skip if there is no
> > + DPDK-compatible
> > NIC
> > +   AT_CHECK([$DPDK_DIR/usertools/dpdk-devbind.py -s | head -n +4 |
> > + tail -
> > 1], [], [stdout])
> 
> In testing I found I had to explicitly set DPDK_DIR as part of
> /etc/environment.
> 
> Defining it as part of .profile or even exporting from the bash shell explicitly
> before running the tests would result in the DPDK_DIR being empty.
> 
> Not sure why this is. It could be specific to my setup. Have you thought about
> passing the DPDK_DIR as an optional argument as part of running ./configure
> rather than relying on environmental variable declarations?

Bad news is that it looks like it's not working...
Good news is that I finally reproduced your issue (it took me quite a lot of time).
Working on that...

> 
> > +   AT_CHECK([cat stdout | cut -d" " -s -f1 > PCI_ADDR])
> > +   AT_CHECK([test -s PCI_ADDR || exit 77])
> > +])
> > +
> > +
> > +# OVS_DPDK_START()
> > +#
> > +# Create an empty database and start ovsdb-server. Add special
> > +configuration # dpdk-init to enable DPDK functionality. Start
> > +ovs-vswitchd connected to that # database using system devices (no
> > dummies).
> > +#
> > +m4_define([OVS_DPDK_START],
> > +  [dnl Create database.
> > +   AT_CHECK([touch .conf.db.~lock~])
> > +   AT_CHECK([ovsdb-tool create conf.db
> > +$abs_top_srcdir/vswitchd/vswitch.ovsschema])
> > +
> > +   dnl Start ovsdb-server.
> > +   AT_CHECK([ovsdb-server --detach --no-chdir --pidfile --log-file --
> > remote=punix:$OVS_RUNDIR/db.sock], [0], [stdout], [stderr])
> > +   on_exit "kill `cat ovsdb-server.pid`"
> > +   AT_CHECK([[sed < stderr '
> > +/vlog|INFO|opened log file/d
> > +/ovsdb_server|INFO|ovsdb-server (Open vSwitch)/d']])
> > +   AT_CAPTURE_FILE([ovsdb-server.log])
> > +
> > +   dnl Initialize database.
> > +   AT_CHECK([ovs-vsctl --no-wait init])
> > +
> > +   dnl Enable DPDK functionality
> > +   AT_CHECK([ovs-vsctl --no-wait set Open_vSwitch .
> > + other_config:dpdk-
> > init=true])
> > +   AT_CHECK([lscpu], [], [stdout])
> > +   AT_CHECK([cat stdout | grep "NUMA node(s)" | awk '{c=1; while
> > (c++<$(3)) {printf "1024,"}; print "1024"}' > SOCKET_MEM])
> > +   AT_CHECK([ovs-vsctl --no-wait set Open_vSwitch .
> > + other_config:dpdk-socket-mem="$(cat SOCKET_MEM)"])
> > +
> > +   dnl Start ovs-vswitchd.
> > +   AT_CHECK([ovs-vswitchd --detach --no-chdir --pidfile --log-file -
> > vvconn -vofproto_dpif -vunixctl], [0], [stdout], [stderr])
> > +   AT_CAPTURE_FILE([ovs-vswitchd.log])
> > +   on_exit "kill_ovs_vswitchd `cat ovs-vswitchd.pid`"
> > +])
> > diff --git a/tests/system-dpdk-testsuite.at b/tests/system-dpdk-
> > testsuite.at new file mode 100644 index 0000000..382f09e
> > --- /dev/null
> > +++ b/tests/system-dpdk-testsuite.at
> > @@ -0,0 +1,25 @@
> > +AT_INIT
> > +
> > +AT_COPYRIGHT([Copyright (c) 2017 Intel Corporation
> > +
> > +Licensed under the Apache License, Version 2.0 (the "License"); you
> > +may not use this file except in compliance with the License.
> > +You may obtain a copy of the License at:
> > +
> > +    http://www.apache.org/licenses/LICENSE-2.0
> > +
> > +Unless required by applicable law or agreed to in writing, software
> > +distributed under the License is distributed on an "AS IS" BASIS,
> > +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
> implied.
> > +See the License for the specific language governing permissions and
> > +limitations under the License.])
> > +
> > +m4_ifdef([AT_COLOR_TESTS], [AT_COLOR_TESTS])
> > +
> > +m4_include([tests/ovs-macros.at])
> > +m4_include([tests/ovsdb-macros.at])
> > +m4_include([tests/ofproto-macros.at])
> > +m4_include([tests/system-common-macros.at])
> > +m4_include([tests/system-dpdk-macros.at])
> > +
> > +m4_include([tests/system-dpdk.at])
> > diff --git a/tests/system-dpdk.at b/tests/system-dpdk.at new file mode
> > 100644 index 0000000..1f1839e
> > --- /dev/null
> > +++ b/tests/system-dpdk.at
> > @@ -0,0 +1,67 @@
> > +AT_BANNER([OVS-DPDK unit tests])
> > +
> > +dnl
> > +---------------------------------------------------------------------
> > +--
> > +---
> > +dnl Check if EAL init is successfull
> > +AT_SETUP([OVS-DPDK datapath - EAL init])
> > +AT_KEYWORDS([dpdk])
> > +dnl OVS_DPDK_PRE_CHECK()
> > +OVS_DPDK_START()
> > +AT_CHECK([grep "DPDK Enabled - initializing..." ovs-vswitchd.log],
> > +[],
> > +[stdout]) AT_CHECK([grep "EAL" ovs-vswitchd.log], [], [stdout])
> > +AT_CHECK([grep "DPDK Enabled - initialized" ovs-vswitchd.log], [],
> > +[stdout]) OVS_VSWITCHD_STOP("/Global register is changed during/d")
> > +AT_CLEANUP dnl
> > +---------------------------------------------------------------------
> > +--
> > +---
> > +
> > +
> > +
> > +dnl
> > +---------------------------------------------------------------------
> > +--
> > +---
> > +dnl Add standard DPDK PHY port
> > +AT_SETUP([OVS-DPDK datapath - add standard DPDK port])
> > +AT_KEYWORDS([dpdk])
> > +
> > +OVS_DPDK_PRE_CHECK()
> > +OVS_DPDK_START()
> > +
> > +dnl Add userspace bridge and attach it to OVS AT_CHECK([ovs-vsctl
> > +add-br br10 -- set bridge br10 datapath_type=netdev])
> > +AT_CHECK([ovs-vsctl add-port br10 phy0 -- set Interface phy0
> > +type=dpdk options:dpdk-devargs=$(cat PCI_ADDR)], [], [stdout],
> > +[stderr]) AT_CHECK([ovs-vsctl show], [], [stdout]) sleep 2
> > +
> > +dnl Clean up
> > +AT_CHECK([ovs-vsctl del-port br10 phy0], [], [stdout], [stderr])
> > +OVS_VSWITCHD_STOP("/does not exist. The Open vSwitch kernel
> module is
> > +probably not loaded./d /Failed to enable flow control/d /Global
> > +register is changed during/d") AT_CLEANUP dnl
> > +---------------------------------------------------------------------
> > +--
> > +---
> > +
> > +
> > +
> > +dnl
> > +---------------------------------------------------------------------
> > +--
> > +---
> > +dnl Add vhost-user-client port
> > +AT_SETUP([OVS-DPDK datapath - add vhost-user-client port])
> > +AT_KEYWORDS([dpdk])
> > +
> > +OVS_DPDK_START()
> > +
> > +dnl Add userspace bridge and attach it to OVS AT_CHECK([ovs-vsctl
> > +add-br br10 -- set bridge br10 datapath_type=netdev])
> > +AT_CHECK([ovs-vsctl add-port br10 dpdkvhostuserclient0 -- set
> > +Interface
> > +dpdkvhostuserclient0 type=dpdkvhostuserclient
> > +options:vhost-server-path=/tmp/dpdkvhostclient0], [], [stdout],
> > +[stderr]) AT_CHECK([ovs-vsctl show], [], [stdout]) sleep 2
> > +
> > +dnl Parse log file
> > +AT_CHECK([grep "VHOST_CONFIG: vhost-user client: socket created"
> > +ovs-vswitchd.log], [], [stdout]) AT_CHECK([grep "vHost User device
> > +'dpdkvhostuserclient0' created in 'client' mode, using client socket"
> > +ovs-vswitchd.log], [], [stdout]) AT_CHECK([grep "VHOST_CONFIG:
> > +/tmp/dpdkvhostclient0: reconnecting..." ovs-vswitchd.log], [],
> > +[stdout])
> > +
> > +dnl Clean up
> > +AT_CHECK([ovs-vsctl del-port br10 dpdkvhostuserclient0], [],
> > +[stdout],
> > +[stderr]) OVS_VSWITCHD_STOP("/does not exist. The Open vSwitch
> kernel
> > +module is probably not loaded./d /Failed to enable flow control/d
> > +/failed to connect to \/tmp\/dpdkvhostclient0: No such file or
> > +directory/d /Global register is changed during/d") AT_CLEANUP dnl
> > +---------------------------------------------------------------------
> > +--
> > +---
> > --
> > 1.9.3
> >
> > _______________________________________________
> > dev mailing list
> > dev@openvswitch.org
> > https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Stokes, Ian April 18, 2018, 3:44 p.m. UTC | #3 
> Hi Ian,
> 
> Appreciate your comments and findings. See my replies inline.
> 
> 
> > -----Original Message-----
> > From: Stokes, Ian
> > Sent: Tuesday, April 17, 2018 8:50 PM
> > To: Rybka, MarcinX <marcinx.rybka@intel.com>; dev@openvswitch.org
> > Cc: Rybka, MarcinX <marcinx.rybka@intel.com>
> > Subject: RE: [ovs-dev] [PATCH v4] tests: Add system-dpdk-testsuite
> >
> > > To: dev@openvswitch.org
> > > Cc: Rybka, MarcinX <marcinx.rybka@intel.com>
> > > Subject: [ovs-dev] [PATCH v4] tests: Add system-dpdk-testsuite
> > >
> > > New OVS-DPDK testsuite, which can be launched via `make check-dpdk`,
> > > tests OVS using a DPDK datapath. The testsuite contains already
> > > initial
> > tests:
> > >  1. EAL init
> > >  2. Add standard DPDK PHY port
> > >  3. Add vhost-user-client port
> > >
> > > Signed-off-by: Marcin Rybka <marcinx.rybka@intel.com>
> > >
> >
> > Thanks for this Marcin, comments inline below.
> > > ---
> > > Ver.4 updates:
> > >  - now works with DPDK 17.11.1
> > >  - added entry in NEWS
> > >  - better hugepages allocation
> > > ---
> > >  Documentation/topics/testing.rst | 19 ++++++++++++
> > >  NEWS                             |  2 ++
> > >  tests/automake.mk                | 17 ++++++++++
> > >  tests/system-dpdk-macros.at      | 56
> > +++++++++++++++++++++++++++++++++
> > >  tests/system-dpdk-testsuite.at   | 25 +++++++++++++++
> > >  tests/system-dpdk.at             | 67
> > > ++++++++++++++++++++++++++++++++++++++++
> > >  6 files changed, 186 insertions(+)
> > >  create mode 100644 tests/system-dpdk-macros.at  create mode 100644
> > > tests/system-dpdk-testsuite.at  create mode 100644
> > > tests/system-dpdk.at
> > >
> > > diff --git a/Documentation/topics/testing.rst
> > > b/Documentation/topics/testing.rst
> > > index 5dcf446..bdc7225 100644
> > > --- a/Documentation/topics/testing.rst
> > > +++ b/Documentation/topics/testing.rst
> > > @@ -297,6 +297,25 @@ To invoke the datapath testsuite with the
> > > userspace datapath, run::
> > >
> > >  The results of the testsuite are in ``tests/system-userspace-
> > > testsuite.dir``.
> > >
> > > +DPDK datapath
> > > +'''''''''''''
> > > +
> > > +To test :doc:`/intro/install/dpdk` (i.e., the build was configured
> > > +with ``--with-dpdk``, the DPDK is installed), run the testsuite and
> > > +generate a report by using the ``check-dpdk`` target::
> > > +
> > > +    $ make check-dpdk
> > > +
> > > +To see a list of all the available tests, run::
> > > +
> > > +    $ make check-dpdk TESTSUITEFLAGS=--list
> > > +
> > > +These tests require a `DPDK supported NIC`_ and proper DPDK
> > > +variables (``DPDK_DIR`` and ``DPDK_BUILD``). Moreover you need to
> > > +have root privileges, load the required modules and bind the NIC to
> > > +the DPDK-
> > > compatible driver.
> > > +
> > > +.. _DPDK supported NIC: http://dpdk.org/doc/nics
> > > +
> > >  Kernel datapath
> > >  '''''''''''''''
> > >
> > > diff --git a/NEWS b/NEWS
> > > index 58a7b58..7bad608 100644
> > > --- a/NEWS
> > > +++ b/NEWS
> > > @@ -21,6 +21,8 @@ Post-v2.9.0
> > >         and reply with a RST for TCP or ICMPv4/ICMPv6 unreachable
> > > message for
> > >         other IPv4/IPv6-based protocols whenever a reject ACL rule is
> hit.
> > >       * ACL match conditions can now match on Port_Groups.
> > > +   - New 'check-dpdk' Makefile target to run a new system testsuite.
> > > +     See Testing topic for the details.
> >
> > I think this should be added under a specific DPDK header in NEWS.
> >
> > - DPDK
> >   * Add 'check-dpdk' Makefile target to run dpdk specific testsuite.
> >
> > Thoughts?
> 
> Yes, I can do like this. I was suggested by previous news about new
> testsuites.

Sure, I guess in the past unit tests have been missing for DPDK specifically so it's nice to call it out under that header.

> 
> >
> > >
> > >  v2.9.0 - 19 Feb 2018
> > >  --------------------
> > > diff --git a/tests/automake.mk b/tests/automake.mk index
> > > d9292e8..e52531a
> > > 100644
> > > --- a/tests/automake.mk
> > > +++ b/tests/automake.mk
> > > @@ -5,10 +5,12 @@ EXTRA_DIST += \
> > >  	$(SYSTEM_KMOD_TESTSUITE_AT) \
> > >  	$(SYSTEM_USERSPACE_TESTSUITE_AT) \
> > >  	$(SYSTEM_OFFLOADS_TESTSUITE_AT) \
> > > +	$(SYSTEM_DPDK_TESTSUITE_AT) \
> > >  	$(TESTSUITE) \
> > >  	$(SYSTEM_KMOD_TESTSUITE) \
> > >  	$(SYSTEM_USERSPACE_TESTSUITE) \
> > >  	$(SYSTEM_OFFLOADS_TESTSUITE) \
> > > +	$(SYSTEM_DPDK_TESTSUITE) \
> > >  	tests/atlocal.in \
> > >  	$(srcdir)/package.m4 \
> > >  	$(srcdir)/tests/testsuite \
> > > @@ -128,6 +130,12 @@ SYSTEM_OFFLOADS_TESTSUITE_AT = \
> > >  	tests/system-offloads-traffic.at \
> > >  	tests/system-offloads-testsuite.at
> > >
> > > +SYSTEM_DPDK_TESTSUITE_AT = \
> > > +	tests/system-common-macros.at \
> > > +	tests/system-dpdk-macros.at \
> > > +	tests/system-dpdk-testsuite.at \
> > > +	tests/system-dpdk.at
> > > +
> > >  check_SCRIPTS += tests/atlocal
> > >
> > >  TESTSUITE = $(srcdir)/tests/testsuite @@ -135,6 +143,7 @@
> > > TESTSUITE_PATCH = $(srcdir)/tests/testsuite.patch
> > > SYSTEM_KMOD_TESTSUITE = $(srcdir)/tests/system-kmod-testsuite
> > >  SYSTEM_USERSPACE_TESTSUITE =
> > > $(srcdir)/tests/system-userspace-testsuite
> > >  SYSTEM_OFFLOADS_TESTSUITE =
> > > $(srcdir)/tests/system-offloads-testsuite
> > > +SYSTEM_DPDK_TESTSUITE = $(srcdir)/tests/system-dpdk-testsuite
> > >  DISTCLEANFILES += tests/atconfig tests/atlocal
> > >
> > >  AUTOTEST_PATH =
> > > utilities:vswitchd:ovsdb:vtep:tests:$(PTHREAD_WIN32_DIR_DLL):$(SSL_D
> > > IR ):ov n/controller-vtep:ovn/northd:ovn/utilities:ovn/controller
> > > @@ -258,6 +267,10 @@ check-offloads: all
> > >  	set $(SHELL) '$(SYSTEM_OFFLOADS_TESTSUITE)' -C tests
> > > AUTOTEST_PATH='$(AUTOTEST_PATH)' $(TESTSUITEFLAGS) -j1; \
> > >  	"$$@" || (test X'$(RECHECK)' = Xyes && "$$@" --recheck)
> > >
> > > +check-dpdk: all
> > > +	set $(SHELL) '$(SYSTEM_DPDK_TESTSUITE)' -C tests
> > > AUTOTEST_PATH='$(AUTOTEST_PATH)' $(TESTSUITEFLAGS) -j1; \
> > > +	"$$@" || (test X'$(RECHECK)' = Xyes && "$$@" --recheck)
> > > +
> > >  clean-local:
> > >  	test ! -f '$(TESTSUITE)' || $(SHELL) '$(TESTSUITE)' -C tests
> > > --clean
> > >
> > > @@ -286,6 +299,10 @@ $(SYSTEM_OFFLOADS_TESTSUITE): package.m4
> > > $(SYSTEM_TESTSUITE_AT) $(SYSTEM_OFFLOAD
> > >  	$(AM_V_GEN)$(AUTOTEST) -I '$(srcdir)' -o $@.tmp $@.at
> > >  	$(AM_V_at)mv $@.tmp $@
> > >
> > > +$(SYSTEM_DPDK_TESTSUITE): package.m4 $(SYSTEM_TESTSUITE_AT)
> > > $(SYSTEM_DPDK_TESTSUITE_AT) $(COMMON_MACROS_AT)
> > > +	$(AM_V_GEN)$(AUTOTEST) -I '$(srcdir)' -o $@.tmp $@.at
> > > +	$(AM_V_at)mv $@.tmp $@
> > > +
> > >  # The `:;' works around a Bash 3.2 bug when the output is not
> writeable.
> > >  $(srcdir)/package.m4: $(top_srcdir)/configure.ac
> > >  	$(AM_V_GEN):;{ \
> > > diff --git a/tests/system-dpdk-macros.at
> > > b/tests/system-dpdk-macros.at new file mode 100644 index
> > > 0000000..0762ee0
> > > --- /dev/null
> > > +++ b/tests/system-dpdk-macros.at
> > > @@ -0,0 +1,56 @@
> > > +# OVS_DPDK_PRE_CHECK()
> > > +#
> > > +# Check prerequisites for DPDK tests. Following settings are checked:
> > > +#  - Hugepages
> > > +#  - UIO driver
> > > +#
> > > +m4_define([OVS_DPDK_PRE_CHECK],
> > > +  [dnl Check Hugepages
> > > +   AT_CHECK([cat /proc/meminfo], [], [stdout])
> > > +   AT_CHECK([grep HugePages_ stdout], [], [stdout])
> > > +   AT_CHECK([mount], [], [stdout])
> > > +   AT_CHECK([grep 'hugetlbfs' stdout], [], [stdout], [])
> > > +
> > > +   dnl Check if VFIO or UIO driver is loaded
> > > +   AT_CHECK([lsmod | grep -E "igb_uio|vfio"], [], [stdout])
> > > +
> > > +   dnl Find PCI address candidate, skip if there is no
> > > + DPDK-compatible
> > > NIC
> > > +   AT_CHECK([$DPDK_DIR/usertools/dpdk-devbind.py -s | head -n +4 |
> > > + tail -
> > > 1], [], [stdout])
> >
> > In testing I found I had to explicitly set DPDK_DIR as part of
> > /etc/environment.
> >
> > Defining it as part of .profile or even exporting from the bash shell
> > explicitly before running the tests would result in the DPDK_DIR being
> empty.
> >
> > Not sure why this is. It could be specific to my setup. Have you
> > thought about passing the DPDK_DIR as an optional argument as part of
> > running ./configure rather than relying on environmental variable
> declarations?
> 
> Bad news is that it looks like it's not working...
> Good news is that I finally reproduced your issue (it took me quite a lot
> of time).
> Working on that...

Ok that's good to hear. If there's any testing I can help with let me know.

Ian
> 
> >
> > > +   AT_CHECK([cat stdout | cut -d" " -s -f1 > PCI_ADDR])
> > > +   AT_CHECK([test -s PCI_ADDR || exit 77])
> > > +])
> > > +
> > > +
> > > +# OVS_DPDK_START()
> > > +#
> > > +# Create an empty database and start ovsdb-server. Add special
> > > +configuration # dpdk-init to enable DPDK functionality. Start
> > > +ovs-vswitchd connected to that # database using system devices (no
> > > dummies).
> > > +#
> > > +m4_define([OVS_DPDK_START],
> > > +  [dnl Create database.
> > > +   AT_CHECK([touch .conf.db.~lock~])
> > > +   AT_CHECK([ovsdb-tool create conf.db
> > > +$abs_top_srcdir/vswitchd/vswitch.ovsschema])
> > > +
> > > +   dnl Start ovsdb-server.
> > > +   AT_CHECK([ovsdb-server --detach --no-chdir --pidfile --log-file
> > > + --
> > > remote=punix:$OVS_RUNDIR/db.sock], [0], [stdout], [stderr])
> > > +   on_exit "kill `cat ovsdb-server.pid`"
> > > +   AT_CHECK([[sed < stderr '
> > > +/vlog|INFO|opened log file/d
> > > +/ovsdb_server|INFO|ovsdb-server (Open vSwitch)/d']])
> > > +   AT_CAPTURE_FILE([ovsdb-server.log])
> > > +
> > > +   dnl Initialize database.
> > > +   AT_CHECK([ovs-vsctl --no-wait init])
> > > +
> > > +   dnl Enable DPDK functionality
> > > +   AT_CHECK([ovs-vsctl --no-wait set Open_vSwitch .
> > > + other_config:dpdk-
> > > init=true])
> > > +   AT_CHECK([lscpu], [], [stdout])
> > > +   AT_CHECK([cat stdout | grep "NUMA node(s)" | awk '{c=1; while
> > > (c++<$(3)) {printf "1024,"}; print "1024"}' > SOCKET_MEM])
> > > +   AT_CHECK([ovs-vsctl --no-wait set Open_vSwitch .
> > > + other_config:dpdk-socket-mem="$(cat SOCKET_MEM)"])
> > > +
> > > +   dnl Start ovs-vswitchd.
> > > +   AT_CHECK([ovs-vswitchd --detach --no-chdir --pidfile --log-file
> > > + -
> > > vvconn -vofproto_dpif -vunixctl], [0], [stdout], [stderr])
> > > +   AT_CAPTURE_FILE([ovs-vswitchd.log])
> > > +   on_exit "kill_ovs_vswitchd `cat ovs-vswitchd.pid`"
> > > +])
> > > diff --git a/tests/system-dpdk-testsuite.at b/tests/system-dpdk-
> > > testsuite.at new file mode 100644 index 0000000..382f09e
> > > --- /dev/null
> > > +++ b/tests/system-dpdk-testsuite.at
> > > @@ -0,0 +1,25 @@
> > > +AT_INIT
> > > +
> > > +AT_COPYRIGHT([Copyright (c) 2017 Intel Corporation
> > > +
> > > +Licensed under the Apache License, Version 2.0 (the "License"); you
> > > +may not use this file except in compliance with the License.
> > > +You may obtain a copy of the License at:
> > > +
> > > +    http://www.apache.org/licenses/LICENSE-2.0
> > > +
> > > +Unless required by applicable law or agreed to in writing, software
> > > +distributed under the License is distributed on an "AS IS" BASIS,
> > > +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
> > implied.
> > > +See the License for the specific language governing permissions and
> > > +limitations under the License.])
> > > +
> > > +m4_ifdef([AT_COLOR_TESTS], [AT_COLOR_TESTS])
> > > +
> > > +m4_include([tests/ovs-macros.at])
> > > +m4_include([tests/ovsdb-macros.at])
> > > +m4_include([tests/ofproto-macros.at])
> > > +m4_include([tests/system-common-macros.at])
> > > +m4_include([tests/system-dpdk-macros.at])
> > > +
> > > +m4_include([tests/system-dpdk.at])
> > > diff --git a/tests/system-dpdk.at b/tests/system-dpdk.at new file
> > > mode
> > > 100644 index 0000000..1f1839e
> > > --- /dev/null
> > > +++ b/tests/system-dpdk.at
> > > @@ -0,0 +1,67 @@
> > > +AT_BANNER([OVS-DPDK unit tests])
> > > +
> > > +dnl
> > > +-------------------------------------------------------------------
> > > +--
> > > +--
> > > +---
> > > +dnl Check if EAL init is successfull AT_SETUP([OVS-DPDK datapath -
> > > +EAL init])
> > > +AT_KEYWORDS([dpdk])
> > > +dnl OVS_DPDK_PRE_CHECK()
> > > +OVS_DPDK_START()
> > > +AT_CHECK([grep "DPDK Enabled - initializing..." ovs-vswitchd.log],
> > > +[],
> > > +[stdout]) AT_CHECK([grep "EAL" ovs-vswitchd.log], [], [stdout])
> > > +AT_CHECK([grep "DPDK Enabled - initialized" ovs-vswitchd.log], [],
> > > +[stdout]) OVS_VSWITCHD_STOP("/Global register is changed during/d")
> > > +AT_CLEANUP dnl
> > > +-------------------------------------------------------------------
> > > +--
> > > +--
> > > +---
> > > +
> > > +
> > > +
> > > +dnl
> > > +-------------------------------------------------------------------
> > > +--
> > > +--
> > > +---
> > > +dnl Add standard DPDK PHY port
> > > +AT_SETUP([OVS-DPDK datapath - add standard DPDK port])
> > > +AT_KEYWORDS([dpdk])
> > > +
> > > +OVS_DPDK_PRE_CHECK()
> > > +OVS_DPDK_START()
> > > +
> > > +dnl Add userspace bridge and attach it to OVS AT_CHECK([ovs-vsctl
> > > +add-br br10 -- set bridge br10 datapath_type=netdev])
> > > +AT_CHECK([ovs-vsctl add-port br10 phy0 -- set Interface phy0
> > > +type=dpdk options:dpdk-devargs=$(cat PCI_ADDR)], [], [stdout],
> > > +[stderr]) AT_CHECK([ovs-vsctl show], [], [stdout]) sleep 2
> > > +
> > > +dnl Clean up
> > > +AT_CHECK([ovs-vsctl del-port br10 phy0], [], [stdout], [stderr])
> > > +OVS_VSWITCHD_STOP("/does not exist. The Open vSwitch kernel
> > module is
> > > +probably not loaded./d /Failed to enable flow control/d /Global
> > > +register is changed during/d") AT_CLEANUP dnl
> > > +-------------------------------------------------------------------
> > > +--
> > > +--
> > > +---
> > > +
> > > +
> > > +
> > > +dnl
> > > +-------------------------------------------------------------------
> > > +--
> > > +--
> > > +---
> > > +dnl Add vhost-user-client port
> > > +AT_SETUP([OVS-DPDK datapath - add vhost-user-client port])
> > > +AT_KEYWORDS([dpdk])
> > > +
> > > +OVS_DPDK_START()
> > > +
> > > +dnl Add userspace bridge and attach it to OVS AT_CHECK([ovs-vsctl
> > > +add-br br10 -- set bridge br10 datapath_type=netdev])
> > > +AT_CHECK([ovs-vsctl add-port br10 dpdkvhostuserclient0 -- set
> > > +Interface
> > > +dpdkvhostuserclient0 type=dpdkvhostuserclient
> > > +options:vhost-server-path=/tmp/dpdkvhostclient0], [], [stdout],
> > > +[stderr]) AT_CHECK([ovs-vsctl show], [], [stdout]) sleep 2
> > > +
> > > +dnl Parse log file
> > > +AT_CHECK([grep "VHOST_CONFIG: vhost-user client: socket created"
> > > +ovs-vswitchd.log], [], [stdout]) AT_CHECK([grep "vHost User device
> > > +'dpdkvhostuserclient0' created in 'client' mode, using client socket"
> > > +ovs-vswitchd.log], [], [stdout]) AT_CHECK([grep "VHOST_CONFIG:
> > > +/tmp/dpdkvhostclient0: reconnecting..." ovs-vswitchd.log], [],
> > > +[stdout])
> > > +
> > > +dnl Clean up
> > > +AT_CHECK([ovs-vsctl del-port br10 dpdkvhostuserclient0], [],
> > > +[stdout],
> > > +[stderr]) OVS_VSWITCHD_STOP("/does not exist. The Open vSwitch
> > kernel
> > > +module is probably not loaded./d /Failed to enable flow control/d
> > > +/failed to connect to \/tmp\/dpdkvhostclient0: No such file or
> > > +directory/d /Global register is changed during/d") AT_CLEANUP dnl
> > > +-------------------------------------------------------------------
> > > +--
> > > +--
> > > +---
> > > --
> > > 1.9.3
> > >
> > > _______________________________________________
> > > dev mailing list
> > > dev@openvswitch.org
> > > https://mail.openvswitch.org/mailman/listinfo/ovs-dev
diff mbox series

Patch

diff --git a/Documentation/topics/testing.rst b/Documentation/topics/testing.rst
index 5dcf446..bdc7225 100644
--- a/Documentation/topics/testing.rst
+++ b/Documentation/topics/testing.rst
@@ -297,6 +297,25 @@  To invoke the datapath testsuite with the userspace datapath, run::
 
 The results of the testsuite are in ``tests/system-userspace-testsuite.dir``.
 
+DPDK datapath
+'''''''''''''
+
+To test :doc:`/intro/install/dpdk` (i.e., the build was configured with
+``--with-dpdk``, the DPDK is installed), run the testsuite and generate
+a report by using the ``check-dpdk`` target::
+
+    $ make check-dpdk
+
+To see a list of all the available tests, run::
+
+    $ make check-dpdk TESTSUITEFLAGS=--list
+
+These tests require a `DPDK supported NIC`_ and proper DPDK variables
+(``DPDK_DIR`` and ``DPDK_BUILD``). Moreover you need to have root privileges,
+load the required modules and bind the NIC to the DPDK-compatible driver.
+
+.. _DPDK supported NIC: http://dpdk.org/doc/nics
+
 Kernel datapath
 '''''''''''''''
 
diff --git a/NEWS b/NEWS
index 58a7b58..7bad608 100644
--- a/NEWS
+++ b/NEWS
@@ -21,6 +21,8 @@  Post-v2.9.0
        and reply with a RST for TCP or ICMPv4/ICMPv6 unreachable message for
        other IPv4/IPv6-based protocols whenever a reject ACL rule is hit.
      * ACL match conditions can now match on Port_Groups.
+   - New 'check-dpdk' Makefile target to run a new system testsuite.
+     See Testing topic for the details.
 
 v2.9.0 - 19 Feb 2018
 --------------------
diff --git a/tests/automake.mk b/tests/automake.mk
index d9292e8..e52531a 100644
--- a/tests/automake.mk
+++ b/tests/automake.mk
@@ -5,10 +5,12 @@  EXTRA_DIST += \
 	$(SYSTEM_KMOD_TESTSUITE_AT) \
 	$(SYSTEM_USERSPACE_TESTSUITE_AT) \
 	$(SYSTEM_OFFLOADS_TESTSUITE_AT) \
+	$(SYSTEM_DPDK_TESTSUITE_AT) \
 	$(TESTSUITE) \
 	$(SYSTEM_KMOD_TESTSUITE) \
 	$(SYSTEM_USERSPACE_TESTSUITE) \
 	$(SYSTEM_OFFLOADS_TESTSUITE) \
+	$(SYSTEM_DPDK_TESTSUITE) \
 	tests/atlocal.in \
 	$(srcdir)/package.m4 \
 	$(srcdir)/tests/testsuite \
@@ -128,6 +130,12 @@  SYSTEM_OFFLOADS_TESTSUITE_AT = \
 	tests/system-offloads-traffic.at \
 	tests/system-offloads-testsuite.at
 
+SYSTEM_DPDK_TESTSUITE_AT = \
+	tests/system-common-macros.at \
+	tests/system-dpdk-macros.at \
+	tests/system-dpdk-testsuite.at \
+	tests/system-dpdk.at
+
 check_SCRIPTS += tests/atlocal
 
 TESTSUITE = $(srcdir)/tests/testsuite
@@ -135,6 +143,7 @@  TESTSUITE_PATCH = $(srcdir)/tests/testsuite.patch
 SYSTEM_KMOD_TESTSUITE = $(srcdir)/tests/system-kmod-testsuite
 SYSTEM_USERSPACE_TESTSUITE = $(srcdir)/tests/system-userspace-testsuite
 SYSTEM_OFFLOADS_TESTSUITE = $(srcdir)/tests/system-offloads-testsuite
+SYSTEM_DPDK_TESTSUITE = $(srcdir)/tests/system-dpdk-testsuite
 DISTCLEANFILES += tests/atconfig tests/atlocal
 
 AUTOTEST_PATH = utilities:vswitchd:ovsdb:vtep:tests:$(PTHREAD_WIN32_DIR_DLL):$(SSL_DIR):ovn/controller-vtep:ovn/northd:ovn/utilities:ovn/controller
@@ -258,6 +267,10 @@  check-offloads: all
 	set $(SHELL) '$(SYSTEM_OFFLOADS_TESTSUITE)' -C tests  AUTOTEST_PATH='$(AUTOTEST_PATH)' $(TESTSUITEFLAGS) -j1; \
 	"$$@" || (test X'$(RECHECK)' = Xyes && "$$@" --recheck)
 
+check-dpdk: all
+	set $(SHELL) '$(SYSTEM_DPDK_TESTSUITE)' -C tests  AUTOTEST_PATH='$(AUTOTEST_PATH)' $(TESTSUITEFLAGS) -j1; \
+	"$$@" || (test X'$(RECHECK)' = Xyes && "$$@" --recheck)
+
 clean-local:
 	test ! -f '$(TESTSUITE)' || $(SHELL) '$(TESTSUITE)' -C tests --clean
 
@@ -286,6 +299,10 @@  $(SYSTEM_OFFLOADS_TESTSUITE): package.m4 $(SYSTEM_TESTSUITE_AT) $(SYSTEM_OFFLOAD
 	$(AM_V_GEN)$(AUTOTEST) -I '$(srcdir)' -o $@.tmp $@.at
 	$(AM_V_at)mv $@.tmp $@
 
+$(SYSTEM_DPDK_TESTSUITE): package.m4 $(SYSTEM_TESTSUITE_AT) $(SYSTEM_DPDK_TESTSUITE_AT) $(COMMON_MACROS_AT)
+	$(AM_V_GEN)$(AUTOTEST) -I '$(srcdir)' -o $@.tmp $@.at
+	$(AM_V_at)mv $@.tmp $@
+
 # The `:;' works around a Bash 3.2 bug when the output is not writeable.
 $(srcdir)/package.m4: $(top_srcdir)/configure.ac
 	$(AM_V_GEN):;{ \
diff --git a/tests/system-dpdk-macros.at b/tests/system-dpdk-macros.at
new file mode 100644
index 0000000..0762ee0
--- /dev/null
+++ b/tests/system-dpdk-macros.at
@@ -0,0 +1,56 @@ 
+# OVS_DPDK_PRE_CHECK()
+#
+# Check prerequisites for DPDK tests. Following settings are checked:
+#  - Hugepages
+#  - UIO driver
+#
+m4_define([OVS_DPDK_PRE_CHECK],
+  [dnl Check Hugepages
+   AT_CHECK([cat /proc/meminfo], [], [stdout])
+   AT_CHECK([grep HugePages_ stdout], [], [stdout])
+   AT_CHECK([mount], [], [stdout])
+   AT_CHECK([grep 'hugetlbfs' stdout], [], [stdout], [])
+
+   dnl Check if VFIO or UIO driver is loaded
+   AT_CHECK([lsmod | grep -E "igb_uio|vfio"], [], [stdout])
+
+   dnl Find PCI address candidate, skip if there is no DPDK-compatible NIC
+   AT_CHECK([$DPDK_DIR/usertools/dpdk-devbind.py -s | head -n +4 | tail -1], [], [stdout])
+   AT_CHECK([cat stdout | cut -d" " -s -f1 > PCI_ADDR])
+   AT_CHECK([test -s PCI_ADDR || exit 77])
+])
+
+
+# OVS_DPDK_START()
+#
+# Create an empty database and start ovsdb-server. Add special configuration
+# dpdk-init to enable DPDK functionality. Start ovs-vswitchd connected to that
+# database using system devices (no dummies).
+#
+m4_define([OVS_DPDK_START],
+  [dnl Create database.
+   AT_CHECK([touch .conf.db.~lock~])
+   AT_CHECK([ovsdb-tool create conf.db $abs_top_srcdir/vswitchd/vswitch.ovsschema])
+
+   dnl Start ovsdb-server.
+   AT_CHECK([ovsdb-server --detach --no-chdir --pidfile --log-file --remote=punix:$OVS_RUNDIR/db.sock], [0], [stdout], [stderr])
+   on_exit "kill `cat ovsdb-server.pid`"
+   AT_CHECK([[sed < stderr '
+/vlog|INFO|opened log file/d
+/ovsdb_server|INFO|ovsdb-server (Open vSwitch)/d']])
+   AT_CAPTURE_FILE([ovsdb-server.log])
+
+   dnl Initialize database.
+   AT_CHECK([ovs-vsctl --no-wait init])
+
+   dnl Enable DPDK functionality
+   AT_CHECK([ovs-vsctl --no-wait set Open_vSwitch . other_config:dpdk-init=true])
+   AT_CHECK([lscpu], [], [stdout])
+   AT_CHECK([cat stdout | grep "NUMA node(s)" | awk '{c=1; while (c++<$(3)) {printf "1024,"}; print "1024"}' > SOCKET_MEM])
+   AT_CHECK([ovs-vsctl --no-wait set Open_vSwitch . other_config:dpdk-socket-mem="$(cat SOCKET_MEM)"])
+
+   dnl Start ovs-vswitchd.
+   AT_CHECK([ovs-vswitchd --detach --no-chdir --pidfile --log-file -vvconn -vofproto_dpif -vunixctl], [0], [stdout], [stderr])
+   AT_CAPTURE_FILE([ovs-vswitchd.log])
+   on_exit "kill_ovs_vswitchd `cat ovs-vswitchd.pid`"
+])
diff --git a/tests/system-dpdk-testsuite.at b/tests/system-dpdk-testsuite.at
new file mode 100644
index 0000000..382f09e
--- /dev/null
+++ b/tests/system-dpdk-testsuite.at
@@ -0,0 +1,25 @@ 
+AT_INIT
+
+AT_COPYRIGHT([Copyright (c) 2017 Intel Corporation
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at:
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.])
+
+m4_ifdef([AT_COLOR_TESTS], [AT_COLOR_TESTS])
+
+m4_include([tests/ovs-macros.at])
+m4_include([tests/ovsdb-macros.at])
+m4_include([tests/ofproto-macros.at])
+m4_include([tests/system-common-macros.at])
+m4_include([tests/system-dpdk-macros.at])
+
+m4_include([tests/system-dpdk.at])
diff --git a/tests/system-dpdk.at b/tests/system-dpdk.at
new file mode 100644
index 0000000..1f1839e
--- /dev/null
+++ b/tests/system-dpdk.at
@@ -0,0 +1,67 @@ 
+AT_BANNER([OVS-DPDK unit tests])
+
+dnl --------------------------------------------------------------------------
+dnl Check if EAL init is successfull
+AT_SETUP([OVS-DPDK datapath - EAL init])
+AT_KEYWORDS([dpdk])
+dnl OVS_DPDK_PRE_CHECK()
+OVS_DPDK_START()
+AT_CHECK([grep "DPDK Enabled - initializing..." ovs-vswitchd.log], [], [stdout])
+AT_CHECK([grep "EAL" ovs-vswitchd.log], [], [stdout])
+AT_CHECK([grep "DPDK Enabled - initialized" ovs-vswitchd.log], [], [stdout])
+OVS_VSWITCHD_STOP("/Global register is changed during/d")
+AT_CLEANUP
+dnl --------------------------------------------------------------------------
+
+
+
+dnl --------------------------------------------------------------------------
+dnl Add standard DPDK PHY port
+AT_SETUP([OVS-DPDK datapath - add standard DPDK port])
+AT_KEYWORDS([dpdk])
+
+OVS_DPDK_PRE_CHECK()
+OVS_DPDK_START()
+
+dnl Add userspace bridge and attach it to OVS
+AT_CHECK([ovs-vsctl add-br br10 -- set bridge br10 datapath_type=netdev])
+AT_CHECK([ovs-vsctl add-port br10 phy0 -- set Interface phy0 type=dpdk options:dpdk-devargs=$(cat PCI_ADDR)], [], [stdout], [stderr])
+AT_CHECK([ovs-vsctl show], [], [stdout])
+sleep 2
+
+dnl Clean up
+AT_CHECK([ovs-vsctl del-port br10 phy0], [], [stdout], [stderr])
+OVS_VSWITCHD_STOP("/does not exist. The Open vSwitch kernel module is probably not loaded./d
+/Failed to enable flow control/d
+/Global register is changed during/d")
+AT_CLEANUP
+dnl --------------------------------------------------------------------------
+
+
+
+dnl --------------------------------------------------------------------------
+dnl Add vhost-user-client port
+AT_SETUP([OVS-DPDK datapath - add vhost-user-client port])
+AT_KEYWORDS([dpdk])
+
+OVS_DPDK_START()
+
+dnl Add userspace bridge and attach it to OVS
+AT_CHECK([ovs-vsctl add-br br10 -- set bridge br10 datapath_type=netdev])
+AT_CHECK([ovs-vsctl add-port br10 dpdkvhostuserclient0 -- set Interface dpdkvhostuserclient0 type=dpdkvhostuserclient options:vhost-server-path=/tmp/dpdkvhostclient0], [], [stdout], [stderr])
+AT_CHECK([ovs-vsctl show], [], [stdout])
+sleep 2
+
+dnl Parse log file
+AT_CHECK([grep "VHOST_CONFIG: vhost-user client: socket created" ovs-vswitchd.log], [], [stdout])
+AT_CHECK([grep "vHost User device 'dpdkvhostuserclient0' created in 'client' mode, using client socket" ovs-vswitchd.log], [], [stdout])
+AT_CHECK([grep "VHOST_CONFIG: /tmp/dpdkvhostclient0: reconnecting..." ovs-vswitchd.log], [], [stdout])
+
+dnl Clean up
+AT_CHECK([ovs-vsctl del-port br10 dpdkvhostuserclient0], [], [stdout], [stderr])
+OVS_VSWITCHD_STOP("/does not exist. The Open vSwitch kernel module is probably not loaded./d
+/Failed to enable flow control/d
+/failed to connect to \/tmp\/dpdkvhostclient0: No such file or directory/d
+/Global register is changed during/d")
+AT_CLEANUP
+dnl --------------------------------------------------------------------------