diff mbox series

[ovs-dev,v2,6/7] system-dpdk: Run traffic tests.

Message ID 20230823153419.241641-6-david.marchand@redhat.com
State Changes Requested
Headers show
Series [ovs-dev,v2,1/7] system-dpdk: Introduce helpers for testpmd. | expand

Checks

Context Check Description
ovsrobot/apply-robot success apply and check: success
ovsrobot/github-robot-_Build_and_Test fail github build: failed
ovsrobot/intel-ovs-compilation fail test: fail

Commit Message

David Marchand Aug. 23, 2023, 3:34 p.m. UTC
Integrate system-traffic.at tests as part of check-dpdk.

DPDK net/tap specific logs are added to the log exclusions.

Some tests that can't work with the userspace datapath are skipped by
overriding some OVS_CHECK_* macros.

ADD_VETH is implemented with a tap interface in the kernel, directly
polled by OVS.

Signed-off-by: David Marchand <david.marchand@redhat.com>
---
 tests/system-dpdk-macros.at    | 85 +++++++++++++++++++++++++++++++++-
 tests/system-dpdk-testsuite.at |  2 +
 tests/system-dpdk.at           |  3 --
 3 files changed, 86 insertions(+), 4 deletions(-)

Comments

David Marchand Aug. 24, 2023, 6:56 a.m. UTC | #1
On Wed, Aug 23, 2023 at 5:35 PM David Marchand
<david.marchand@redhat.com> wrote:
>
> Integrate system-traffic.at tests as part of check-dpdk.

CI (thinking of Intel robot) other than GHA might not be too happy
about this change.
It is hard to tell what fails in the report I received.

A safer approach is probably to define a new check-XXX target instead
of expanding check-dpdk.

>
> DPDK net/tap specific logs are added to the log exclusions.
>
> Some tests that can't work with the userspace datapath are skipped by
> overriding some OVS_CHECK_* macros.
>
> ADD_VETH is implemented with a tap interface in the kernel, directly
> polled by OVS.
>
> Signed-off-by: David Marchand <david.marchand@redhat.com>
Eelco Chaudron Aug. 24, 2023, 8:17 a.m. UTC | #2
On 24 Aug 2023, at 8:56, David Marchand wrote:

> On Wed, Aug 23, 2023 at 5:35 PM David Marchand
> <david.marchand@redhat.com> wrote:
>>
>> Integrate system-traffic.at tests as part of check-dpdk.
>
> CI (thinking of Intel robot) other than GHA might not be too happy
> about this change.
> It is hard to tell what fails in the report I received.
>
> A safer approach is probably to define a new check-XXX target instead
> of expanding check-dpdk.

To be honest, I think we already have too many tests, so I would not mind keeping it in one test and see if intel can make reporting better (maybe store the results somewhere).

I added the system-traffic.at a while back to the offload tests (and no one complained :).

//Eelco

PS: Will review the full series later this week.

>>
>> DPDK net/tap specific logs are added to the log exclusions.
>>
>> Some tests that can't work with the userspace datapath are skipped by
>> overriding some OVS_CHECK_* macros.
>>
>> ADD_VETH is implemented with a tap interface in the kernel, directly
>> polled by OVS.
>>
>> Signed-off-by: David Marchand <david.marchand@redhat.com>
>
>
> -- 
> David Marchand
Aaron Conole Aug. 24, 2023, 3:22 p.m. UTC | #3
David Marchand <david.marchand@redhat.com> writes:

> On Wed, Aug 23, 2023 at 5:35 PM David Marchand
> <david.marchand@redhat.com> wrote:
>>
>> Integrate system-traffic.at tests as part of check-dpdk.
>
> CI (thinking of Intel robot) other than GHA might not be too happy
> about this change.
> It is hard to tell what fails in the report I received.
>
> A safer approach is probably to define a new check-XXX target instead
> of expanding check-dpdk.

I'm not sure - at least for Intel, we can ask them to investigate.

I noticed that we didn't get any new results from intel after this
patch, so I wonder if this change broke something.

I also noticed that without the conntrack tests (7/7) this passed, but
with them, we got a failure report.  Is it possible that the tests are
taking too long (in which case, we need to investigate a way of
optimizing this).

>>
>> DPDK net/tap specific logs are added to the log exclusions.
>>
>> Some tests that can't work with the userspace datapath are skipped by
>> overriding some OVS_CHECK_* macros.
>>
>> ADD_VETH is implemented with a tap interface in the kernel, directly
>> polled by OVS.
>>
>> Signed-off-by: David Marchand <david.marchand@redhat.com>
Ilya Maximets Aug. 24, 2023, 5:52 p.m. UTC | #4
On 8/24/23 17:22, Aaron Conole wrote:
> David Marchand <david.marchand@redhat.com> writes:
> 
>> On Wed, Aug 23, 2023 at 5:35 PM David Marchand
>> <david.marchand@redhat.com> wrote:
>>>
>>> Integrate system-traffic.at tests as part of check-dpdk.
>>
>> CI (thinking of Intel robot) other than GHA might not be too happy
>> about this change.
>> It is hard to tell what fails in the report I received.
>>
>> A safer approach is probably to define a new check-XXX target instead
>> of expanding check-dpdk.
> 
> I'm not sure - at least for Intel, we can ask them to investigate.
> 
> I noticed that we didn't get any new results from intel after this
> patch, so I wonder if this change broke something.
> 
> I also noticed that without the conntrack tests (7/7) this passed, but
> with them, we got a failure report.  Is it possible that the tests are
> taking too long (in which case, we need to investigate a way of
> optimizing this).

I believe, all the tests failed because of this:

> 2023-08-23T22:23:10.040Z|00088|dpdk|ERR|tap_dev_configure(): net_taptap1: number of rx queues 2 must be equal to number of tx queues 3
> 2023-08-23T22:23:10.040Z|00089|dpdk|ERR|Port0 dev_configure = -1
> 2023-08-23T22:23:10.040Z|00090|netdev_dpdk|WARN|Interface ovs-tap1 eth_dev setup error Operation not permitted
> 2023-08-23T22:23:10.040Z|00091|netdev_dpdk|ERR|Interface ovs-tap1(rxq:2 txq:3 lsc interrupt mode:false) configure error: Operation not permitted
> 2023-08-23T22:23:10.040Z|00092|netdev_dpdk|INFO|ovs-tap1: rx-steering: default rss
> 2023-08-23T22:23:10.040Z|00093|dpif_netdev|ERR|Failed to set interface ovs-tap1 new configuration
> 2023-08-23T22:23:10.040Z|00094|dpif_netdev|INFO|Performing pmd to rx queue assignment using cycles algorithm.
> 2023-08-23T22:23:10.040Z|00095|dpif_netdev|INFO|Core 21 on numa node 0 assigned port 'dpdkvhostuser0' rx queue 0 (measured processing cycles 0).
> 2023-08-23T22:23:10.040Z|00096|dpif|WARN|netdev at ovs-netdev: failed to add ovs-tap1 as port: Invalid argument
> 2023-08-23T22:23:10.040Z|00097|bridge|WARN|could not add network device ovs-tap1 to ofproto (Invalid argument)

And failing tests typically take much longer, because we wait
for some traffic or some flows that never appear.

Configuration for net/tap DPDK ports should be managed more carefully
in the tests.  We will likely have to use dummy NUMA even.

The Lab seems to work fine.  Error reporting could be better,
but it's enough to identify the issue in this case.

We should also keep in mind that these tests should pass in the debian
autopkgtest environment:
  https://patchwork.ozlabs.org/project/openvswitch/patch/20230627101104.72417-2-frode.nordahl@canonical.com/
Even if we don't have this merged, this is what Ubuntu/Debian folks
are running internally.

Best regards, Ilya Maximets.

> 
>>>
>>> DPDK net/tap specific logs are added to the log exclusions.
>>>
>>> Some tests that can't work with the userspace datapath are skipped by
>>> overriding some OVS_CHECK_* macros.
>>>
>>> ADD_VETH is implemented with a tap interface in the kernel, directly
>>> polled by OVS.
>>>
>>> Signed-off-by: David Marchand <david.marchand@redhat.com>
>
Eelco Chaudron Aug. 25, 2023, 11:31 a.m. UTC | #5
On 23 Aug 2023, at 17:34, David Marchand wrote:

> Integrate system-traffic.at tests as part of check-dpdk.
>
> DPDK net/tap specific logs are added to the log exclusions.
>
> Some tests that can't work with the userspace datapath are skipped by
> overriding some OVS_CHECK_* macros.
>
> ADD_VETH is implemented with a tap interface in the kernel, directly
> polled by OVS.
>
> Signed-off-by: David Marchand <david.marchand@redhat.com>

I think the patch by itself is fine, and I ran it on my Ubuntu 22.04 test VM with the “netdev-dpdk: Disable net/tap Tx L4 checksum offloads” applied and it was working fine.

Anyhow I’ll include my Acked-by flag, assuming it will pass the autopkgtest mentioned by Ilya (guess you need another revision anyway :)


Acked-by: Eelco Chaudron <echaudro@redhat.com>

> ---
>  tests/system-dpdk-macros.at    | 85 +++++++++++++++++++++++++++++++++-
>  tests/system-dpdk-testsuite.at |  2 +
>  tests/system-dpdk.at           |  3 --
>  3 files changed, 86 insertions(+), 4 deletions(-)
>
> diff --git a/tests/system-dpdk-macros.at b/tests/system-dpdk-macros.at
> index a72133fae1..de126946a0 100644
> --- a/tests/system-dpdk-macros.at
> +++ b/tests/system-dpdk-macros.at
> @@ -87,7 +87,13 @@ $1";/does not exist. The Open vSwitch kernel module is probably not loaded./d
>  /EAL: No \(available\|free\) .*hugepages reported/d
>  /Failed to enable flow control/d
>  /Rx checksum offload is not supported on/d
> -/TELEMETRY: No legacy callbacks, legacy socket not created/d"])
> +/TELEMETRY: No legacy callbacks, legacy socket not created/d
> +/tap_nl_dump_ext_ack(): Specified qdisc kind is unknown/d
> +/qdisc_create_multiq(): Could not add multiq qdisc (2): No such file or directory/d
> +/eth_dev_tap_create(): .*: failed to create multiq qdisc./d
> +/eth_dev_tap_create():  Disabling rte flow support: No such file or directory/d
> +/tap_mp_req_on_rxtx(): Failed to send start req to secondary/d
> +/does not exist. The Open vSwitch kernel module is probably not loaded./d"])
>    ])
>
>
> @@ -124,3 +130,80 @@ m4_define([OVS_DPDK_STOP_TESTPMD],
>    [AT_CHECK([kill `cat testpmd.pid`])
>     OVS_WAIT([kill -0 `cat testpmd.pid`], [kill -9 `cat testpmd.pid`])
>    ])
> +
> +
> +# OVS_TRAFFIC_VSWITCHD_START([vsctl-args], [vsctl-output], [dbinit-aux-args])
> +#
> +# Creates a database and starts ovsdb-server, starts ovs-vswitchd
> +# connected to that database, calls ovs-vsctl to create a bridge named
> +# br0 with predictable settings, passing 'vsctl-args' as additional
> +# commands to ovs-vsctl.  If 'vsctl-args' causes ovs-vsctl to provide
> +# output (e.g. because it includes "create" commands) then 'vsctl-output'
> +# specifies the expected output after filtering through uuidfilt.
> +# 'dbinit-aux-args' are passed as additional commands to 'ovs-vsctl init'
> +# before starting ovs-vswitchd.
> +m4_define([OVS_TRAFFIC_VSWITCHD_START],
> +  [
> +   OVS_DPDK_PRE_CHECK()
> +   OVS_WAIT_WHILE([ip link show ovs-netdev])
> +   OVS_DPDK_START([], [--disable-system], [$3])
> +   dnl Add bridges, ports, etc.
> +   OVS_WAIT_WHILE([ip link show br0])
> +   AT_CHECK([ovs-vsctl -- _ADD_BR([br0]) -- $1 m4_if([$2], [], [], [| uuidfilt])], [0], [$2])
> +])
> +
> +
> +# OVS_TRAFFIC_VSWITCHD_STOP([ALLOWLIST], [extra_cmds])
> +#
> +# Gracefully stops ovs-vswitchd and ovsdb-server, checking their log files
> +# for messages with severity WARN or higher and signaling an error if any
> +# is present.  The optional ALLOWLIST may contain shell-quoted "sed"
> +# commands to delete any warnings that are actually expected, e.g.:
> +#
> +#   OVS_TRAFFIC_VSWITCHD_STOP(["/expected error/d"])
> +#
> +# 'extra_cmds' are shell commands to be executed after OVS_VSWITCHD_STOP() is
> +# invoked. They can be used to perform additional cleanups such as name space
> +# removal.
> +m4_define([OVS_TRAFFIC_VSWITCHD_STOP],
> +  [OVS_DPDK_STOP_VSWITCHD([$1])
> +   AT_CHECK([:; $2])
> +  ])
> +
> +
> +m4_define([OVS_CHECK_8021AD],
> +    [AT_SKIP_IF([:])])
> +
> +
> +m4_define([OVS_CHECK_TC_QDISC],
> +    [AT_SKIP_IF([:])])
> +
> +
> +m4_define([OVS_CHECK_TCPDUMP],
> +    [AT_SKIP_IF([:])])
> +
> +
> +# Fake a veth by creating a tap on kernel side and plug it in OVS using the
> +# net/tap DPDK driver.
> +m4_define([ADD_VETH],
> +    [ AT_CHECK([ovs-vsctl add-port $3 ovs-$1 -- \
> +                set interface ovs-$1 external-ids:iface-id="$1" -- \
> +                set interface ovs-$1 type=dpdk -- \
> +                set interface ovs-$1 options:n_rxq=2 -- \
> +                set interface ovs-$1 options:dpdk-devargs=net_tap$1,iface=$1])
> +      OVS_WAIT_UNTIL([ip link show dev $1 | grep -qw LOWER_UP])
> +      AT_CHECK([ip link set $1 netns $2])
> +      NS_CHECK_EXEC([$2], [ip addr add $4 dev $1 $7])
> +      NS_CHECK_EXEC([$2], [ip link set dev $1 up])
> +      if test -n "$5"; then
> +        NS_CHECK_EXEC([$2], [ip link set dev $1 address $5])
> +      fi
> +      if test -n "$6"; then
> +        NS_CHECK_EXEC([$2], [ip route add default via $6])
> +      fi
> +    ]
> +)
> +
> +
> +m4_define([CONFIGURE_VETH_OFFLOADS],
> +   [AT_CHECK([ethtool -K $1 tx off], [0], [ignore], [ignore])])
> diff --git a/tests/system-dpdk-testsuite.at b/tests/system-dpdk-testsuite.at
> index 382f09e9ff..f61fbf9212 100644
> --- a/tests/system-dpdk-testsuite.at
> +++ b/tests/system-dpdk-testsuite.at
> @@ -20,6 +20,8 @@ m4_include([tests/ovs-macros.at])
>  m4_include([tests/ovsdb-macros.at])
>  m4_include([tests/ofproto-macros.at])
>  m4_include([tests/system-common-macros.at])
> +m4_include([tests/system-userspace-macros.at])
>  m4_include([tests/system-dpdk-macros.at])
>
>  m4_include([tests/system-dpdk.at])
> +m4_include([tests/system-traffic.at])
> diff --git a/tests/system-dpdk.at b/tests/system-dpdk.at
> index e8a04d1d86..3e3223c02d 100644
> --- a/tests/system-dpdk.at
> +++ b/tests/system-dpdk.at
> @@ -1,6 +1,3 @@
> -m4_define([CONFIGURE_VETH_OFFLOADS],
> -   [AT_CHECK([ethtool -K $1 tx off], [0], [ignore], [ignore])])
> -
>  AT_BANNER([OVS-DPDK unit tests])
>
>  dnl CHECK_MEMPOOL_PARAM([mtu], [numa], [+line])
> -- 
> 2.41.0
David Marchand Aug. 25, 2023, 2:44 p.m. UTC | #6
On Thu, Aug 24, 2023 at 7:52 PM Ilya Maximets <i.maximets@ovn.org> wrote:
>
> On 8/24/23 17:22, Aaron Conole wrote:
> > David Marchand <david.marchand@redhat.com> writes:
> >
> >> On Wed, Aug 23, 2023 at 5:35 PM David Marchand
> >> <david.marchand@redhat.com> wrote:
> >>>
> >>> Integrate system-traffic.at tests as part of check-dpdk.
> >>
> >> CI (thinking of Intel robot) other than GHA might not be too happy
> >> about this change.
> >> It is hard to tell what fails in the report I received.
> >>
> >> A safer approach is probably to define a new check-XXX target instead
> >> of expanding check-dpdk.
> >
> > I'm not sure - at least for Intel, we can ask them to investigate.
> >
> > I noticed that we didn't get any new results from intel after this
> > patch, so I wonder if this change broke something.
> >
> > I also noticed that without the conntrack tests (7/7) this passed, but
> > with them, we got a failure report.  Is it possible that the tests are
> > taking too long (in which case, we need to investigate a way of
> > optimizing this).
>
> I believe, all the tests failed because of this:
>
> > 2023-08-23T22:23:10.040Z|00088|dpdk|ERR|tap_dev_configure(): net_taptap1: number of rx queues 2 must be equal to number of tx queues 3
> > 2023-08-23T22:23:10.040Z|00089|dpdk|ERR|Port0 dev_configure = -1
> > 2023-08-23T22:23:10.040Z|00090|netdev_dpdk|WARN|Interface ovs-tap1 eth_dev setup error Operation not permitted

Yes, it seems to be the reason though I did not understand why those
rxq/txq count were used.


> > 2023-08-23T22:23:10.040Z|00091|netdev_dpdk|ERR|Interface ovs-tap1(rxq:2 txq:3 lsc interrupt mode:false) configure error: Operation not permitted
> > 2023-08-23T22:23:10.040Z|00092|netdev_dpdk|INFO|ovs-tap1: rx-steering: default rss
> > 2023-08-23T22:23:10.040Z|00093|dpif_netdev|ERR|Failed to set interface ovs-tap1 new configuration
> > 2023-08-23T22:23:10.040Z|00094|dpif_netdev|INFO|Performing pmd to rx queue assignment using cycles algorithm.
> > 2023-08-23T22:23:10.040Z|00095|dpif_netdev|INFO|Core 21 on numa node 0 assigned port 'dpdkvhostuser0' rx queue 0 (measured processing cycles 0).
> > 2023-08-23T22:23:10.040Z|00096|dpif|WARN|netdev at ovs-netdev: failed to add ovs-tap1 as port: Invalid argument
> > 2023-08-23T22:23:10.040Z|00097|bridge|WARN|could not add network device ovs-tap1 to ofproto (Invalid argument)
>
> And failing tests typically take much longer, because we wait
> for some traffic or some flows that never appear.
>
> Configuration for net/tap DPDK ports should be managed more carefully
> in the tests.  We will likely have to use dummy NUMA even.

Oh that's the catch, numa... I reproduced in a vm and I have something
that seems to work, testing a bit more.


>
> The Lab seems to work fine.  Error reporting could be better,
> but it's enough to identify the issue in this case.
>
> We should also keep in mind that these tests should pass in the debian
> autopkgtest environment:
>   https://patchwork.ozlabs.org/project/openvswitch/patch/20230627101104.72417-2-frode.nordahl@canonical.com/
> Even if we don't have this merged, this is what Ubuntu/Debian folks
> are running internally.

Ok.
For now, I don't see what could be blocking, do you have any specific
problem in mind?
David Marchand Aug. 25, 2023, 2:47 p.m. UTC | #7
On Wed, Aug 23, 2023 at 5:35 PM David Marchand
<david.marchand@redhat.com> wrote:
> diff --git a/tests/system-dpdk-macros.at b/tests/system-dpdk-macros.at
> index a72133fae1..de126946a0 100644
> --- a/tests/system-dpdk-macros.at
> +++ b/tests/system-dpdk-macros.at
> @@ -87,7 +87,13 @@ $1";/does not exist. The Open vSwitch kernel module is probably not loaded./d
>  /EAL: No \(available\|free\) .*hugepages reported/d
>  /Failed to enable flow control/d
>  /Rx checksum offload is not supported on/d
> -/TELEMETRY: No legacy callbacks, legacy socket not created/d"])
> +/TELEMETRY: No legacy callbacks, legacy socket not created/d
> +/tap_nl_dump_ext_ack(): Specified qdisc kind is unknown/d
> +/qdisc_create_multiq(): Could not add multiq qdisc (2): No such file or directory/d
> +/eth_dev_tap_create(): .*: failed to create multiq qdisc./d
> +/eth_dev_tap_create():  Disabling rte flow support: No such file or directory/d
> +/tap_mp_req_on_rxtx(): Failed to send start req to secondary/d

Since those logs come from the traffic tests (especially the net/tap
logs), I think it is more appropriate to move those to
OVS_TRAFFIC_VSWITCHD_STOP.
Opinion?


> +/does not exist. The Open vSwitch kernel module is probably not loaded./d"])
>    ])
>
>
Eelco Chaudron Aug. 28, 2023, 7:52 a.m. UTC | #8
On 25 Aug 2023, at 16:47, David Marchand wrote:

> On Wed, Aug 23, 2023 at 5:35 PM David Marchand
> <david.marchand@redhat.com> wrote:
>> diff --git a/tests/system-dpdk-macros.at b/tests/system-dpdk-macros.at
>> index a72133fae1..de126946a0 100644
>> --- a/tests/system-dpdk-macros.at
>> +++ b/tests/system-dpdk-macros.at
>> @@ -87,7 +87,13 @@ $1";/does not exist. The Open vSwitch kernel module is probably not loaded./d
>>  /EAL: No \(available\|free\) .*hugepages reported/d
>>  /Failed to enable flow control/d
>>  /Rx checksum offload is not supported on/d
>> -/TELEMETRY: No legacy callbacks, legacy socket not created/d"])
>> +/TELEMETRY: No legacy callbacks, legacy socket not created/d
>> +/tap_nl_dump_ext_ack(): Specified qdisc kind is unknown/d
>> +/qdisc_create_multiq(): Could not add multiq qdisc (2): No such file or directory/d
>> +/eth_dev_tap_create(): .*: failed to create multiq qdisc./d
>> +/eth_dev_tap_create():  Disabling rte flow support: No such file or directory/d
>> +/tap_mp_req_on_rxtx(): Failed to send start req to secondary/d
>
> Since those logs come from the traffic tests (especially the net/tap
> logs), I think it is more appropriate to move those to
> OVS_TRAFFIC_VSWITCHD_STOP.
> Opinion?

Agreed, this would probably be better as the basic tests do not generate them.

//Eelco

>
>> +/does not exist. The Open vSwitch kernel module is probably not loaded./d"])
>>    ])
>>
>>
>
>
> -- 
> David Marchand
diff mbox series

Patch

diff --git a/tests/system-dpdk-macros.at b/tests/system-dpdk-macros.at
index a72133fae1..de126946a0 100644
--- a/tests/system-dpdk-macros.at
+++ b/tests/system-dpdk-macros.at
@@ -87,7 +87,13 @@  $1";/does not exist. The Open vSwitch kernel module is probably not loaded./d
 /EAL: No \(available\|free\) .*hugepages reported/d
 /Failed to enable flow control/d
 /Rx checksum offload is not supported on/d
-/TELEMETRY: No legacy callbacks, legacy socket not created/d"])
+/TELEMETRY: No legacy callbacks, legacy socket not created/d
+/tap_nl_dump_ext_ack(): Specified qdisc kind is unknown/d
+/qdisc_create_multiq(): Could not add multiq qdisc (2): No such file or directory/d
+/eth_dev_tap_create(): .*: failed to create multiq qdisc./d
+/eth_dev_tap_create():  Disabling rte flow support: No such file or directory/d
+/tap_mp_req_on_rxtx(): Failed to send start req to secondary/d
+/does not exist. The Open vSwitch kernel module is probably not loaded./d"])
   ])
 
 
@@ -124,3 +130,80 @@  m4_define([OVS_DPDK_STOP_TESTPMD],
   [AT_CHECK([kill `cat testpmd.pid`])
    OVS_WAIT([kill -0 `cat testpmd.pid`], [kill -9 `cat testpmd.pid`])
   ])
+
+
+# OVS_TRAFFIC_VSWITCHD_START([vsctl-args], [vsctl-output], [dbinit-aux-args])
+#
+# Creates a database and starts ovsdb-server, starts ovs-vswitchd
+# connected to that database, calls ovs-vsctl to create a bridge named
+# br0 with predictable settings, passing 'vsctl-args' as additional
+# commands to ovs-vsctl.  If 'vsctl-args' causes ovs-vsctl to provide
+# output (e.g. because it includes "create" commands) then 'vsctl-output'
+# specifies the expected output after filtering through uuidfilt.
+# 'dbinit-aux-args' are passed as additional commands to 'ovs-vsctl init'
+# before starting ovs-vswitchd.
+m4_define([OVS_TRAFFIC_VSWITCHD_START],
+  [
+   OVS_DPDK_PRE_CHECK()
+   OVS_WAIT_WHILE([ip link show ovs-netdev])
+   OVS_DPDK_START([], [--disable-system], [$3])
+   dnl Add bridges, ports, etc.
+   OVS_WAIT_WHILE([ip link show br0])
+   AT_CHECK([ovs-vsctl -- _ADD_BR([br0]) -- $1 m4_if([$2], [], [], [| uuidfilt])], [0], [$2])
+])
+
+
+# OVS_TRAFFIC_VSWITCHD_STOP([ALLOWLIST], [extra_cmds])
+#
+# Gracefully stops ovs-vswitchd and ovsdb-server, checking their log files
+# for messages with severity WARN or higher and signaling an error if any
+# is present.  The optional ALLOWLIST may contain shell-quoted "sed"
+# commands to delete any warnings that are actually expected, e.g.:
+#
+#   OVS_TRAFFIC_VSWITCHD_STOP(["/expected error/d"])
+#
+# 'extra_cmds' are shell commands to be executed after OVS_VSWITCHD_STOP() is
+# invoked. They can be used to perform additional cleanups such as name space
+# removal.
+m4_define([OVS_TRAFFIC_VSWITCHD_STOP],
+  [OVS_DPDK_STOP_VSWITCHD([$1])
+   AT_CHECK([:; $2])
+  ])
+
+
+m4_define([OVS_CHECK_8021AD],
+    [AT_SKIP_IF([:])])
+
+
+m4_define([OVS_CHECK_TC_QDISC],
+    [AT_SKIP_IF([:])])
+
+
+m4_define([OVS_CHECK_TCPDUMP],
+    [AT_SKIP_IF([:])])
+
+
+# Fake a veth by creating a tap on kernel side and plug it in OVS using the
+# net/tap DPDK driver.
+m4_define([ADD_VETH],
+    [ AT_CHECK([ovs-vsctl add-port $3 ovs-$1 -- \
+                set interface ovs-$1 external-ids:iface-id="$1" -- \
+                set interface ovs-$1 type=dpdk -- \
+                set interface ovs-$1 options:n_rxq=2 -- \
+                set interface ovs-$1 options:dpdk-devargs=net_tap$1,iface=$1])
+      OVS_WAIT_UNTIL([ip link show dev $1 | grep -qw LOWER_UP])
+      AT_CHECK([ip link set $1 netns $2])
+      NS_CHECK_EXEC([$2], [ip addr add $4 dev $1 $7])
+      NS_CHECK_EXEC([$2], [ip link set dev $1 up])
+      if test -n "$5"; then
+        NS_CHECK_EXEC([$2], [ip link set dev $1 address $5])
+      fi
+      if test -n "$6"; then
+        NS_CHECK_EXEC([$2], [ip route add default via $6])
+      fi
+    ]
+)
+
+
+m4_define([CONFIGURE_VETH_OFFLOADS],
+   [AT_CHECK([ethtool -K $1 tx off], [0], [ignore], [ignore])])
diff --git a/tests/system-dpdk-testsuite.at b/tests/system-dpdk-testsuite.at
index 382f09e9ff..f61fbf9212 100644
--- a/tests/system-dpdk-testsuite.at
+++ b/tests/system-dpdk-testsuite.at
@@ -20,6 +20,8 @@  m4_include([tests/ovs-macros.at])
 m4_include([tests/ovsdb-macros.at])
 m4_include([tests/ofproto-macros.at])
 m4_include([tests/system-common-macros.at])
+m4_include([tests/system-userspace-macros.at])
 m4_include([tests/system-dpdk-macros.at])
 
 m4_include([tests/system-dpdk.at])
+m4_include([tests/system-traffic.at])
diff --git a/tests/system-dpdk.at b/tests/system-dpdk.at
index e8a04d1d86..3e3223c02d 100644
--- a/tests/system-dpdk.at
+++ b/tests/system-dpdk.at
@@ -1,6 +1,3 @@ 
-m4_define([CONFIGURE_VETH_OFFLOADS],
-   [AT_CHECK([ethtool -K $1 tx off], [0], [ignore], [ignore])])
-
 AT_BANNER([OVS-DPDK unit tests])
 
 dnl CHECK_MEMPOOL_PARAM([mtu], [numa], [+line])