From patchwork Fri Feb 21 00:48:03 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?WWkgWWFuZyAo5p2o54eaKS3kupHmnI3liqHpm4blm6I=?= X-Patchwork-Id: 1241733 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.136; helo=silver.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=inspur.com Received: from silver.osuosl.org (smtp3.osuosl.org [140.211.166.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 48NtDh0J34z9sRG for ; Fri, 21 Feb 2020 11:51:00 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by silver.osuosl.org (Postfix) with ESMTP id 2C6A521553; Fri, 21 Feb 2020 00:50:58 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from silver.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 6xY424yrPb7s; Fri, 21 Feb 2020 00:50:54 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by silver.osuosl.org (Postfix) with ESMTP id 834BF1FDAF; Fri, 21 Feb 2020 00:50:54 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 6A0B4C1D81; Fri, 21 Feb 2020 00:50:54 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from silver.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by lists.linuxfoundation.org (Postfix) with ESMTP id ED4A3C013E for ; Fri, 21 Feb 2020 00:50:52 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by silver.osuosl.org (Postfix) with ESMTP id E96DE2040A for ; Fri, 21 Feb 2020 00:50:52 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from silver.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id xJhdIvgw0nZu for ; Fri, 21 Feb 2020 00:50:45 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.7.6 Received: from sg05.corpemail.net (sg05.corpemail.net [128.199.149.210]) by silver.osuosl.org (Postfix) with ESMTPS id B5F1F1FDAF for ; Fri, 21 Feb 2020 00:50:44 +0000 (UTC) Received: from unicom149.biz-email.net ([192.168.0.9]) by ssh247.corpemail.net ((Trust)) with ESMTP (SSL) id NSS00008; Fri, 21 Feb 2020 08:49:08 +0800 Received: from ssh248.corpemail.net ([192.168.0.136]) by unicom149.biz-email.net ((Outbound)) with ESMTP (SSL) id NSR00111; Fri, 21 Feb 2020 08:48:11 +0800 Received: from ([60.208.111.195]) by ssh248.corpemail.net (Antispam) with ASMTP (SSL) id NCR27706; Fri, 21 Feb 2020 08:48:06 +0800 Received: from jtjnmail201606.home.langchao.com (10.100.2.6) by jtjnmail201606.home.langchao.com (10.100.2.6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.1591.10; Fri, 21 Feb 2020 08:48:03 +0800 Received: from jtjnmail201606.home.langchao.com ([fe80::8583:33f:807a:3430]) by jtjnmail201606.home.langchao.com ([fe80::8583:33f:807a:3430%6]) with mapi id 15.01.1591.008; Fri, 21 Feb 2020 08:48:03 +0800 From: Yi Yang (???D)-?????????? To: "u9012063@gmail.com" , "dev@openvswitch.org" Thread-Topic: [ovs-dev] [PATCHv2 1/2] userspace: Enable TSO support for non-DPDK. Thread-Index: AQHV6CBVbbVcRGRMj0KoIBqTggtnC6gk0Hfw Date: Fri, 21 Feb 2020 00:48:03 +0000 Message-ID: <8ceb00fba7264d9bab150adacaa8d499@inspur.com> References: <1582225172-78476-1-git-send-email-u9012063@gmail.com> In-Reply-To: <1582225172-78476-1-git-send-email-u9012063@gmail.com> Accept-Language: zh-CN, en-US Content-Language: zh-CN X-MS-Has-Attach: yes X-MS-TNEF-Correlator: x-originating-ip: [10.100.1.52] MIME-Version: 1.0 X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Cc: "fbl@sysclose.org" , "i.maximets@ovn.org" Subject: [ovs-dev] =?gb2312?b?tPC4tDogIFtQQVRDSHYyIDEvMl0gdXNlcnNwYWNlOiBF?= =?gb2312?b?bmFibGUgVFNPIHN1cHBvcnQgZm9yIG5vbi1EUERLLg==?= X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" William, which kernel version did you use to test for this patch? I don't want to build a kernel if Ubuntu 16.04 kernel can work. -----邮件原件----- 发件人: dev [mailto:ovs-dev-bounces@openvswitch.org] 代表 William Tu 发送时间: 2020年2月21日 3:00 收件人: dev@openvswitch.org 抄送: fbl@sysclose.org; i.maximets@ovn.org 主题: [ovs-dev] [PATCHv2 1/2] userspace: Enable TSO support for non-DPDK. This patch enables TSO support for non-DPDK use cases, and also add check-system-tso testsuite. Before TSO, we have to disable checksum offload, allowing the kernel to calculate the TCP/UDP packet checsum. With TSO, we can skip the checksum validation by enabling checksum offload, and with large packet size, we see better performance. Consider container to container use cases: iperf3 -c (ns0) -> veth peer -> OVS -> veth peer -> iperf3 -s (ns1) And I got around 6Gbps, similar to TSO with DPDK-enabled. Tested-at: https://travis-ci.org/williamtu/ovs-travis/builds/653109097 Signed-off-by: William Tu --- v2: - add make check-system-tso test - combine logging for dpdk and non-dpdk - I'm surprised that most of the test cases passed. This is due to few tests using tcp/udp, so it does not trigger TSO. I saw only geneve/vxlan fails randomly, maybe we can check it later. --- lib/dp-packet.h | 95 ++++++++++++++++++++++++++----------------- lib/userspace-tso.c | 5 --- tests/.gitignore | 3 ++ tests/automake.mk | 15 +++++++ tests/system-tso-macros.at | 42 +++++++++++++++++++ tests/system-tso-testsuite.at | 26 ++++++++++++ 6 files changed, 143 insertions(+), 43 deletions(-) create mode 100644 tests/system-tso-macros.at create mode 100644 tests/system-tso-testsuite.at -- 2.7.4 diff --git a/lib/dp-packet.h b/lib/dp-packet.h index 9f8991faad52..6b90cec2afb4 100644 --- a/lib/dp-packet.h +++ b/lib/dp-packet.h @@ -53,7 +53,25 @@ enum OVS_PACKED_ENUM dp_packet_source { enum dp_packet_offload_mask { DP_PACKET_OL_RSS_HASH_MASK = 0x1, /* Is the 'rss_hash' valid? */ DP_PACKET_OL_FLOW_MARK_MASK = 0x2, /* Is the 'flow_mark' valid? */ + DP_PACKET_OL_RX_L4_CKSUM_BAD = 1 << 3, + DP_PACKET_OL_RX_IP_CKSUM_BAD = 1 << 4, + DP_PACKET_OL_RX_L4_CKSUM_GOOD = 1 << 5, + DP_PACKET_OL_RX_IP_CKSUM_GOOD = 1 << 6, + DP_PACKET_OL_TX_TCP_SEG = 1 << 7, + DP_PACKET_OL_TX_IPV4 = 1 << 8, + DP_PACKET_OL_TX_IPV6 = 1 << 9, + DP_PACKET_OL_TX_TCP_CKSUM = 1 << 10, + DP_PACKET_OL_TX_UDP_CKSUM = 1 << 11, + DP_PACKET_OL_TX_SCTP_CKSUM = 1 << 12, }; + +#define DP_PACKET_OL_TX_L4_MASK (DP_PACKET_OL_TX_TCP_CKSUM | \ + DP_PACKET_OL_TX_UDP_CKSUM | \ + DP_PACKET_OL_TX_SCTP_CKSUM) #define +DP_PACKET_OL_RX_IP_CKSUM_MASK (DP_PACKET_OL_RX_IP_CKSUM_GOOD | \ + DP_PACKET_OL_RX_IP_CKSUM_BAD) +#define DP_PACKET_OL_RX_L4_CKSUM_MASK (DP_PACKET_OL_RX_L4_CKSUM_GOOD | \ + DP_PACKET_OL_RX_L4_CKSUM_BAD) #else /* DPDK mbuf ol_flags that are not really an offload flags. These are mostly * related to mbuf memory layout and OVS should not touch/clear them. */ @@ -739,82 +757,79 @@ dp_packet_set_allocated(struct dp_packet *b, uint16_t s) b->allocated_ = s; } -/* There are no implementation when not DPDK enabled datapath. */ static inline bool -dp_packet_hwol_is_tso(const struct dp_packet *b OVS_UNUSED) +dp_packet_hwol_is_tso(const struct dp_packet *b) { - return false; + return !!(b->ol_flags & DP_PACKET_OL_TX_TCP_SEG); } -/* There are no implementation when not DPDK enabled datapath. */ static inline bool -dp_packet_hwol_is_ipv4(const struct dp_packet *b OVS_UNUSED) +dp_packet_hwol_is_ipv4(const struct dp_packet *b) { - return false; + return !!(b->ol_flags & DP_PACKET_OL_TX_IPV4); } -/* There are no implementation when not DPDK enabled datapath. */ static inline uint64_t -dp_packet_hwol_l4_mask(const struct dp_packet *b OVS_UNUSED) +dp_packet_hwol_l4_mask(const struct dp_packet *b) { - return 0; + return b->ol_flags & DP_PACKET_OL_TX_L4_MASK; } -/* There are no implementation when not DPDK enabled datapath. */ static inline bool -dp_packet_hwol_l4_is_tcp(const struct dp_packet *b OVS_UNUSED) +dp_packet_hwol_l4_is_tcp(const struct dp_packet *b) { - return false; + return (b->ol_flags & DP_PACKET_OL_TX_L4_MASK) == + DP_PACKET_OL_TX_TCP_CKSUM; } -/* There are no implementation when not DPDK enabled datapath. */ static inline bool -dp_packet_hwol_l4_is_udp(const struct dp_packet *b OVS_UNUSED) +dp_packet_hwol_l4_is_udp(const struct dp_packet *b) { - return false; + return (b->ol_flags & DP_PACKET_OL_TX_L4_MASK) == + DP_PACKET_OL_TX_UDP_CKSUM; } -/* There are no implementation when not DPDK enabled datapath. */ static inline bool -dp_packet_hwol_l4_is_sctp(const struct dp_packet *b OVS_UNUSED) +dp_packet_hwol_l4_is_sctp(const struct dp_packet *b) { - return false; + return (b->ol_flags & DP_PACKET_OL_TX_L4_MASK) == + DP_PACKET_OL_TX_SCTP_CKSUM; } -/* There are no implementation when not DPDK enabled datapath. */ static inline void -dp_packet_hwol_set_tx_ipv4(struct dp_packet *b OVS_UNUSED) +dp_packet_hwol_set_tx_ipv4(struct dp_packet *b) { + b->ol_flags |= DP_PACKET_OL_TX_IPV4; } -/* There are no implementation when not DPDK enabled datapath. */ static inline void -dp_packet_hwol_set_tx_ipv6(struct dp_packet *b OVS_UNUSED) +dp_packet_hwol_set_tx_ipv6(struct dp_packet *b) { + b->ol_flags |= DP_PACKET_OL_TX_IPV6; } -/* There are no implementation when not DPDK enabled datapath. */ static inline void -dp_packet_hwol_set_csum_tcp(struct dp_packet *b OVS_UNUSED) +dp_packet_hwol_set_csum_tcp(struct dp_packet *b) { + b->ol_flags |= DP_PACKET_OL_TX_TCP_CKSUM; } -/* There are no implementation when not DPDK enabled datapath. */ static inline void -dp_packet_hwol_set_csum_udp(struct dp_packet *b OVS_UNUSED) +dp_packet_hwol_set_csum_udp(struct dp_packet *b) { + b->ol_flags |= DP_PACKET_OL_TX_UDP_CKSUM; } -/* There are no implementation when not DPDK enabled datapath. */ static inline void -dp_packet_hwol_set_csum_sctp(struct dp_packet *b OVS_UNUSED) +dp_packet_hwol_set_csum_sctp(struct dp_packet *b) { + b->ol_flags |= DP_PACKET_OL_TX_SCTP_CKSUM; } -/* There are no implementation when not DPDK enabled datapath. */ static inline void -dp_packet_hwol_set_tcp_seg(struct dp_packet *b OVS_UNUSED) +dp_packet_hwol_set_tcp_seg(struct dp_packet *b) { + b->ol_flags |= DP_PACKET_OL_TX_TCP_SEG; } /* Returns the RSS hash of the packet 'p'. Note that the returned value is @@ -845,27 +860,31 @@ dp_packet_reset_offload(struct dp_packet *p) } static inline bool -dp_packet_ip_checksum_valid(const struct dp_packet *p OVS_UNUSED) +dp_packet_ip_checksum_valid(const struct dp_packet *p) { - return false; + return (p->ol_flags & DP_PACKET_OL_RX_IP_CKSUM_MASK) == + DP_PACKET_OL_RX_IP_CKSUM_GOOD; } static inline bool -dp_packet_ip_checksum_bad(const struct dp_packet *p OVS_UNUSED) +dp_packet_ip_checksum_bad(const struct dp_packet *p) { - return false; + return (p->ol_flags & DP_PACKET_OL_RX_IP_CKSUM_MASK) == + DP_PACKET_OL_RX_IP_CKSUM_BAD; } static inline bool -dp_packet_l4_checksum_valid(const struct dp_packet *p OVS_UNUSED) +dp_packet_l4_checksum_valid(const struct dp_packet *p) { - return false; + return (p->ol_flags & DP_PACKET_OL_RX_L4_CKSUM_MASK) == + DP_PACKET_OL_RX_L4_CKSUM_GOOD; } static inline bool -dp_packet_l4_checksum_bad(const struct dp_packet *p OVS_UNUSED) +dp_packet_l4_checksum_bad(const struct dp_packet *p) { - return false; + return (p->ol_flags & DP_PACKET_OL_RX_L4_CKSUM_MASK) == + DP_PACKET_OL_RX_L4_CKSUM_BAD; } static inline bool diff --git a/lib/userspace-tso.c b/lib/userspace-tso.c index 6a4a0149b7f5..f843c2a763ce 100644 --- a/lib/userspace-tso.c +++ b/lib/userspace-tso.c @@ -34,13 +34,8 @@ userspace_tso_init(const struct smap *ovs_other_config) static struct ovsthread_once once = OVSTHREAD_ONCE_INITIALIZER; if (ovsthread_once_start(&once)) { -#ifdef DPDK_NETDEV VLOG_INFO("Userspace TCP Segmentation Offloading support enabled"); userspace_tso = true; -#else - VLOG_WARN("Userspace TCP Segmentation Offloading can not be enabled" - "since OVS is built without DPDK support."); -#endif ovsthread_once_done(&once); } } diff --git a/tests/.gitignore b/tests/.gitignore index c5abb32d025a..d6591c3f5f8f 100644 --- a/tests/.gitignore +++ b/tests/.gitignore @@ -25,6 +25,9 @@ /system-userspace-testsuite /system-userspace-testsuite.dir/ /system-userspace-testsuite.log +/system-tso-testsuite +/system-tso-testsuite.dir/ +/system-tso-testsuite.log /system-offloads-testsuite /system-offloads-testsuite.dir/ /system-offloads-testsuite.log diff --git a/tests/automake.mk b/tests/automake.mk index 9c7ebdce9bf0..b8ddc069417e 100644 --- a/tests/automake.mk +++ b/tests/automake.mk @@ -4,6 +4,7 @@ EXTRA_DIST += \ $(SYSTEM_TESTSUITE_AT) \ $(SYSTEM_KMOD_TESTSUITE_AT) \ $(SYSTEM_USERSPACE_TESTSUITE_AT) \ + $(SYSTEM_TSO_TESTSUITE_AT) \ $(SYSTEM_AFXDP_TESTSUITE_AT) \ $(SYSTEM_OFFLOADS_TESTSUITE_AT) \ $(SYSTEM_DPDK_TESTSUITE_AT) \ @@ -11,6 +12,7 @@ EXTRA_DIST += \ $(TESTSUITE) \ $(SYSTEM_KMOD_TESTSUITE) \ $(SYSTEM_USERSPACE_TESTSUITE) \ + $(SYSTEM_TSO_TESTSUITE) \ $(SYSTEM_AFXDP_TESTSUITE) \ $(SYSTEM_OFFLOADS_TESTSUITE) \ $(SYSTEM_DPDK_TESTSUITE) \ @@ -154,6 +156,10 @@ SYSTEM_USERSPACE_TESTSUITE_AT = \ tests/system-userspace-macros.at \ tests/system-userspace-packet-type-aware.at +SYSTEM_TSO_TESTSUITE_AT = \ + tests/system-tso-testsuite.at \ + tests/system-tso-macros.at + SYSTEM_AFXDP_TESTSUITE_AT = \ tests/system-userspace-macros.at \ tests/system-afxdp-testsuite.at \ @@ -183,6 +189,7 @@ TESTSUITE = $(srcdir)/tests/testsuite TESTSUITE_PATCH = $(srcdir)/tests/testsuite.patch SYSTEM_KMOD_TESTSUITE = $(srcdir)/tests/system-kmod-testsuite SYSTEM_USERSPACE_TESTSUITE = $(srcdir)/tests/system-userspace-testsuite +SYSTEM_TSO_TESTSUITE = $(srcdir)/tests/system-tso-testsuite SYSTEM_AFXDP_TESTSUITE = $(srcdir)/tests/system-afxdp-testsuite SYSTEM_OFFLOADS_TESTSUITE = $(srcdir)/tests/system-offloads-testsuite SYSTEM_DPDK_TESTSUITE = $(srcdir)/tests/system-dpdk-testsuite @@ -326,6 +333,10 @@ check-system-userspace: all set $(SHELL) '$(SYSTEM_USERSPACE_TESTSUITE)' -C tests AUTOTEST_PATH='$(AUTOTEST_PATH)'; \ "$$@" $(TESTSUITEFLAGS) -j1 || (test X'$(RECHECK)' = Xyes && "$$@" --recheck) +check-system-tso: all + set $(SHELL) '$(SYSTEM_TSO_TESTSUITE)' -C tests AUTOTEST_PATH='$(AUTOTEST_PATH)'; \ + "$$@" $(TESTSUITEFLAGS) -j1 || (test X'$(RECHECK)' = Xyes && "$$@" +--recheck) + check-afxdp: all set $(SHELL) '$(SYSTEM_AFXDP_TESTSUITE)' -C tests AUTOTEST_PATH='$(AUTOTEST_PATH)' $(TESTSUITEFLAGS) -j1; \ "$$@" || (test X'$(RECHECK)' = Xyes && "$$@" --recheck) @@ -367,6 +378,10 @@ $(SYSTEM_USERSPACE_TESTSUITE): package.m4 $(SYSTEM_TESTSUITE_AT) $(SYSTEM_USERSP $(AM_V_GEN)$(AUTOTEST) -I '$(srcdir)' -o $@.tmp $@.at $(AM_V_at)mv $@.tmp $@ +$(SYSTEM_TSO_TESTSUITE): package.m4 $(SYSTEM_TESTSUITE_AT) $(SYSTEM_TSO_TESTSUITE_AT) $(COMMON_MACROS_AT) + $(AM_V_GEN)$(AUTOTEST) -I '$(srcdir)' -o $@.tmp $@.at + $(AM_V_at)mv $@.tmp $@ + $(SYSTEM_AFXDP_TESTSUITE): package.m4 $(SYSTEM_TESTSUITE_AT) $(SYSTEM_AFXDP_TESTSUITE_AT) $(COMMON_MACROS_AT) $(AM_V_GEN)$(AUTOTEST) -I '$(srcdir)' -o $@.tmp $@.at $(AM_V_at)mv $@.tmp $@ diff --git a/tests/system-tso-macros.at b/tests/system-tso-macros.at new file mode 100644 index 000000000000..b2dcc85440b4 --- /dev/null +++ b/tests/system-tso-macros.at @@ -0,0 +1,42 @@ +# _ADD_BR([name]) +# +# Expands into the proper ovs-vsctl commands to create a bridge with +the # appropriate type and properties m4_define([_ADD_BR], [[add-br $1 +-- set Bridge $1 datapath_type="netdev" +protocols=OpenFlow10,OpenFlow11,OpenFlow12,OpenFlow13,OpenFlow14,OpenFl +ow15 fail-mode=secure ]]) + +# OVS_TRAFFIC_VSWITCHD_START([vsctl-args], [vsctl-output], [=override]) +# # Creates a database and starts ovsdb-server, starts ovs-vswitchd # +connected to that database, calls ovs-vsctl to create a bridge named # +br0 with predictable settings, passing 'vsctl-args' as additional # +commands to ovs-vsctl. If 'vsctl-args' causes ovs-vsctl to provide # +output (e.g. because it includes "create" commands) then 'vsctl-output' +# specifies the expected output after filtering through uuidfilt. +m4_define([OVS_TRAFFIC_VSWITCHD_START], + [ + OVS_WAIT_WHILE([ip link show ovs-netdev]) + _OVS_VSWITCHD_START([--disable-system]) + dnl Add bridges, ports, etc. + OVS_WAIT_WHILE([ip link show br0]) + AT_CHECK([ovs-vsctl set Open_vSwitch . other_config:userspace-tso-enable=true]) + AT_CHECK([ovs-vsctl -- _ADD_BR([br0]) -- $1 m4_if([$2], [], [], [| +uuidfilt])], [0], [$2]) +]) + +# CONFIGURE_VETH_OFFLOADS([VETH]) +# +# Disable TX offloads for veths. The userspace datapath uses the +AF_PACKET # socket to receive packets for veths. Unfortunately, the +AF_PACKET socket # doesn't play well with offloads: +# 1. GSO packets are received without segmentation and therefore discarded. +# 2. Packets with offloaded partial checksum are received with the wrong +# checksum, therefore discarded by the receiver. +# +# By disabling tx offloads in the non-OVS side of the veth peer we make +sure # that the AF_PACKET socket will not receive bad packets. +# +# This is a workaround, and should be removed when offloads are +properly # supported in netdev-linux. +m4_define([CONFIGURE_VETH_OFFLOADS], + [AT_CHECK([ethtool -K $1 sg on], [0], [ignore], [ignore])] + [AT_CHECK([ethtool -K $1 tso on], [0], [ignore], [ignore])] +) diff --git a/tests/system-tso-testsuite.at b/tests/system-tso-testsuite.at new file mode 100644 index 000000000000..99d748006a86 --- /dev/null +++ b/tests/system-tso-testsuite.at @@ -0,0 +1,26 @@ +AT_INIT + +AT_COPYRIGHT([Copyright (c) 2020 VMware, Inc. + +Licensed under the Apache License, Version 2.0 (the "License"); you may +not use this file except in compliance with the License. +You may obtain a copy of the License at: + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License.]) + +m4_ifdef([AT_COLOR_TESTS], [AT_COLOR_TESTS]) + +m4_include([tests/ovs-macros.at]) +m4_include([tests/ovsdb-macros.at]) +m4_include([tests/ofproto-macros.at]) +m4_include([tests/system-common-macros.at]) +m4_include([tests/system-userspace-macros.at]) +m4_include([tests/system-tso-macros.at]) + +m4_include([tests/system-traffic.at])