From patchwork Mon Jun 19 09:30:51 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eelco Chaudron X-Patchwork-Id: 1796427 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::138; helo=smtp1.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=OXflfq0n; dkim-atps=neutral Received: from smtp1.osuosl.org (smtp1.osuosl.org [IPv6:2605:bc80:3010::138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4Ql4JS74rhz20Z9 for ; Mon, 19 Jun 2023 19:31:04 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 8CF8481DE9; Mon, 19 Jun 2023 09:31:02 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 8CF8481DE9 Authentication-Results: smtp1.osuosl.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=OXflfq0n X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id MZXGGIDTODM0; Mon, 19 Jun 2023 09:31:01 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp1.osuosl.org (Postfix) with ESMTPS id 49A1981DE1; Mon, 19 Jun 2023 09:31:00 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 49A1981DE1 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 2901FC007A; Mon, 19 Jun 2023 09:31:00 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp2.osuosl.org (smtp2.osuosl.org [IPv6:2605:bc80:3010::133]) by lists.linuxfoundation.org (Postfix) with ESMTP id 29CB4C0029 for ; Mon, 19 Jun 2023 09:30:59 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id EB0CA40497 for ; Mon, 19 Jun 2023 09:30:58 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org EB0CA40497 Authentication-Results: smtp2.osuosl.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=OXflfq0n X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id TpS7UuM-HS1O for ; Mon, 19 Jun 2023 09:30:57 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 57269403F9 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by smtp2.osuosl.org (Postfix) with ESMTPS id 57269403F9 for ; Mon, 19 Jun 2023 09:30:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1687167056; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=qncXraPM7mIssSnqYbh3s0b+uRG0imoB41DhulIoe1I=; b=OXflfq0neuPanhWrNpuRTcQirXHFqM5MNxP4jlSZmS4Xb8I+acGxjEBTslOCiPTfhsK+in ZygV8gCHUztqpgm7XfPFCgkk345JbYRAtelWsStO6PfCHqeFLEtNuzzId24abnyPfsfIMv eb1SdbB7wMiQhfmmkkyOxFdhnbhdo+g= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-270-JeshF_qiNsic69Cbmwv4Zg-1; Mon, 19 Jun 2023 05:30:54 -0400 X-MC-Unique: JeshF_qiNsic69Cbmwv4Zg-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 690913802267; Mon, 19 Jun 2023 09:30:54 +0000 (UTC) Received: from ebuild.redhat.com (unknown [10.39.195.16]) by smtp.corp.redhat.com (Postfix) with ESMTP id 9508840C20F5; Mon, 19 Jun 2023 09:30:53 +0000 (UTC) From: Eelco Chaudron To: dev@openvswitch.org Date: Mon, 19 Jun 2023 11:30:51 +0200 Message-Id: <168716703359.3825993.14983017346113283324.stgit@ebuild> User-Agent: StGit/1.5 MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.1 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Cc: david.marchand@redhat.com, i.maximets@ovn.org, dceara@redhat.com Subject: [ovs-dev] [PATCH ovn v3 1/2] tests: add make check-system-dpdk to test suite. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" Allow the ovn-system tests to run on the OVS-DPDK infrastructure, i.e., DPDK ports and mbuf memory. Co-authored-by: David Marchand Signed-off-by: David Marchand Signed-off-by: Eelco Chaudron Acked-by: Dumitru Ceara Acked-by: Ales Musil --- v2: No changes. v3: Add 'dpdk-extra="--log-level=pmd.*:error --no-pci"' to test initialization. tests/automake.mk | 23 +++++++- tests/ofproto-macros.at | 9 ++- tests/system-dpdk-macros.at | 109 ++++++++++++++++++++++++++++++++++++++++ tests/system-dpdk-testsuite.at | 25 +++++++++ tests/system-ovn.at | 11 ++++ 5 files changed, 167 insertions(+), 10 deletions(-) create mode 100644 tests/system-dpdk-macros.at create mode 100644 tests/system-dpdk-testsuite.at diff --git a/tests/automake.mk b/tests/automake.mk index fd5ee14af..eea0d00f4 100644 --- a/tests/automake.mk +++ b/tests/automake.mk @@ -2,11 +2,13 @@ EXTRA_DIST += \ $(COMMON_MACROS_AT) \ $(TESTSUITE_AT) \ $(SYSTEM_TESTSUITE_AT) \ + $(SYSTEM_DPDK_TESTSUITE_AT) \ $(SYSTEM_KMOD_TESTSUITE_AT) \ $(SYSTEM_USERSPACE_TESTSUITE_AT) \ $(PERF_TESTSUITE_AT) \ $(MULTINODE_TESTSUITE_AT) \ $(TESTSUITE) \ + $(SYSTEM_DPDK_TESTSUITE) \ $(SYSTEM_KMOD_TESTSUITE) \ $(SYSTEM_USERSPACE_TESTSUITE) \ $(PERF_TESTSUITE) \ @@ -44,20 +46,22 @@ TESTSUITE_AT = \ tests/ovn-ipsec.at \ tests/ovn-vif-plug.at +SYSTEM_DPDK_TESTSUITE_AT = \ + tests/system-dpdk-testsuite.at \ + tests/system-dpdk-macros.at + SYSTEM_KMOD_TESTSUITE_AT = \ - tests/system-common-macros.at \ + tests/system-kmod-macros.at \ tests/system-kmod-testsuite.at \ - tests/system-kmod-macros.at + tests/system-ovn-kmod.at SYSTEM_USERSPACE_TESTSUITE_AT = \ tests/system-userspace-testsuite.at \ - tests/system-ovn.at \ tests/system-userspace-macros.at SYSTEM_TESTSUITE_AT = \ tests/system-common-macros.at \ - tests/system-ovn.at \ - tests/system-ovn-kmod.at + tests/system-ovn.at PERF_TESTSUITE_AT = \ tests/perf-testsuite.at \ @@ -73,6 +77,7 @@ check_SCRIPTS += tests/atlocal TESTSUITE = $(srcdir)/tests/testsuite TESTSUITE_PATCH = $(srcdir)/tests/testsuite.patch TESTSUITE_DIR = $(abs_top_builddir)/tests/testsuite.dir +SYSTEM_DPDK_TESTSUITE = $(srcdir)/tests/system-dpdk-testsuite SYSTEM_KMOD_TESTSUITE = $(srcdir)/tests/system-kmod-testsuite SYSTEM_USERSPACE_TESTSUITE = $(srcdir)/tests/system-userspace-testsuite PERF_TESTSUITE = $(srcdir)/tests/perf-testsuite @@ -180,6 +185,10 @@ check-userspace-valgrind: all $(valgrind_wrappers) $(check_DATA) check-helgrind: all $(valgrind_wrappers) $(check_DATA) -$(SHELL) '$(TESTSUITE)' -C tests CHECK_VALGRIND=true VALGRIND='$(HELGRIND)' AUTOTEST_PATH='tests/valgrind:$(AUTOTEST_PATH)' -d $(TESTSUITEFLAGS) +check-system-dpdk: all + set $(SHELL) '$(SYSTEM_DPDK_TESTSUITE)' -C tests AUTOTEST_PATH='$(AUTOTEST_PATH)'; \ + $(SUDO) "$$@" $(TESTSUITEFLAGS) -j1 || (test X'$(RECHECK)' = Xyes && $(SUDO) "$$@" --recheck) + # Run kmod tests. Assume kernel modules has been installed or linked into the kernel check-kernel: all set $(SHELL) '$(SYSTEM_KMOD_TESTSUITE)' -C tests AUTOTEST_PATH='$(AUTOTEST_PATH)'; \ @@ -232,6 +241,10 @@ $(TESTSUITE): package.m4 $(TESTSUITE_AT) $(COMMON_MACROS_AT) $(AM_V_at)mv $@.tmp $@ endif +$(SYSTEM_DPDK_TESTSUITE): package.m4 $(SYSTEM_TESTSUITE_AT) $(SYSTEM_DPDK_TESTSUITE_AT) $(COMMON_MACROS_AT) + $(AM_V_GEN)$(AUTOTEST) -I '$(srcdir)' -o $@.tmp $@.at + $(AM_V_at)mv $@.tmp $@ + $(SYSTEM_KMOD_TESTSUITE): package.m4 $(SYSTEM_TESTSUITE_AT) $(SYSTEM_KMOD_TESTSUITE_AT) $(COMMON_MACROS_AT) $(AM_V_GEN)$(AUTOTEST) -I '$(srcdir)' -o $@.tmp $@.at $(AM_V_at)mv $@.tmp $@ diff --git a/tests/ofproto-macros.at b/tests/ofproto-macros.at index 2e0bbd20b..f4ebdafc7 100644 --- a/tests/ofproto-macros.at +++ b/tests/ofproto-macros.at @@ -218,11 +218,12 @@ m4_define([_OVS_VSWITCHD_START], /ofproto|INFO|using datapath ID/d /netdev_linux|INFO|.*device has unknown hardware address family/d /ofproto|INFO|datapath ID changed to fedcba9876543210/d -/dpdk|INFO|DPDK Disabled - Use other_config:dpdk-init to enable/d /netlink_socket|INFO|netlink: could not enable listening to all nsid/d /netdev: Flow API/d /probe tc:/d -/tc: Using policy/d']]) +/tc: Using policy/d +/dpdk|INFO|/d +/dpdk|WARN|/d']]) ]) # OVS_VSWITCHD_START([vsctl-args], [vsctl-output], [=override], @@ -247,7 +248,7 @@ m4_define([OVS_VSWITCHD_START], AT_CHECK([add_of_br 0 $1 m4_if([$2], [], [], [| uuidfilt])], [0], [$2]) ]) -# check_logs scans through all *.log files (except '*.log' and testsuite.log) +# check_logs scans through all *.log files (except '*.log' and '*testsuite.log') # and reports all WARN, ERR, EMER log entries. User can add custom sed filters # in $1. m4_divert_push([PREPARE_TESTS]) @@ -255,7 +256,7 @@ check_logs () { local logs for log in *.log; do case ${log} in # ( - '*.log'|testsuite.log) ;; # ( + '*.log'|*testsuite.log) ;; # ( *) logs="${logs} ${log}" ;; esac done diff --git a/tests/system-dpdk-macros.at b/tests/system-dpdk-macros.at new file mode 100644 index 000000000..8843baa25 --- /dev/null +++ b/tests/system-dpdk-macros.at @@ -0,0 +1,109 @@ +AT_COPYRIGHT([Copyright (c) 2023 Red Hat, Inc. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at: + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License.]) + +m4_include([tests/system-userspace-macros.at]) + + +# OVS_DPDK_PRE_CHECK() +# +# Check prerequisites for DPDK tests. Following settings are checked: +# - Hugepages +# +m4_define([OVS_DPDK_PRE_CHECK], + [dnl Check Hugepages + AT_CHECK([cat /proc/meminfo], [], [stdout]) + AT_SKIP_IF([grep -E 'HugePages_Free: *0' stdout], [], [stdout]) + AT_CHECK([mount], [], [stdout]) + AT_CHECK([grep 'hugetlbfs' stdout], [], [stdout], []) +]) + +m4_define([SYSTEM_DPDK_ALLOWED_LOGS],[ +]) + +# OVS_TRAFFIC_VSWITCHD_START([vsctl-args], [vsctl-output], [=override]) +# +# Creates a database and starts ovsdb-server, starts ovs-vswitchd +# connected to that database, calls ovs-vsctl to create a bridge named +# br0 with predictable settings, passing 'vsctl-args' as additional +# commands to ovs-vsctl. If 'vsctl-args' causes ovs-vsctl to provide +# output (e.g. because it includes "create" commands) then 'vsctl-output' +# specifies the expected output after filtering through uuidfilt. +m4_define([OVS_TRAFFIC_VSWITCHD_START], + [OVS_DPDK_PRE_CHECK() + OVS_WAIT_WHILE([ip link show ovs-netdev]) + _OVS_VSWITCHD_START([--disable-system], + [-- set Open_vSwitch . other_config:dpdk-init=true other_config:dpdk-extra="--log-level=pmd.*:error --no-pci"]) + dnl Add bridges, ports, etc. + OVS_WAIT_WHILE([ip link show br0]) + AT_CHECK([ovs-vsctl -- _ADD_BR([br0]) -- $1 m4_if([$2], [], [], [| uuidfilt])], [0], [$2]) +]) + +# OVS_TRAFFIC_VSWITCHD_STOP([WHITELIST], [extra_cmds]) +# +# Gracefully stops ovs-vswitchd and ovsdb-server, checking their log files +# for messages with severity WARN or higher and signaling an error if any +# is present. The optional WHITELIST may contain shell-quoted "sed" +# commands to delete any warnings that are actually expected, e.g.: +# +# OVS_TRAFFIC_VSWITCHD_STOP(["/expected error/d"]) +# +# 'extra_cmds' are shell commands to be executed afte OVS_VSWITCHD_STOP() is +# invoked. They can be used to perform additional cleanups such as name space +# removal. +m4_define([OVS_TRAFFIC_VSWITCHD_STOP], + [OVS_VSWITCHD_STOP([dnl +$1";/EAL: No \(available\|free\) .*hugepages reported/d +/TELEMETRY: No legacy callbacks, legacy socket not created/d +/dpif(revalidator.*)|WARN|netdev@ovs-netdev: failed to.*proto=2.*/d +/dpif_netdev(revalidator.*)|ERR|internal error parsing flow key.*proto=2.*/d +/dpif_netlink.*Generic Netlink family 'ovs_datapath' does not exist. The Open vSwitch kernel module is probably not loaded./d +/eth_dev_tap_create(): Disabling rte flow support: No such file or directory/d +/eth_dev_tap_create(): .*: failed to create multiq qdisc./d +/netdev_linux.*obtaining netdev stats via vport failed/d +/qdisc_create_multiq(): Could not add multiq qdisc (2): No such file or directory/d +/tap_mp_req_on_rxtx(): Failed to send start req to secondary/d +/tap_nl_dump_ext_ack(): Specified qdisc kind is unknown/d"]) + AT_CHECK([:; $2]) +]) + +# ADD_VETH([port], [namespace], [ovs-br], [ip_addr] [mac_addr], [gateway], +# [ip_addr_flags] [ip6_addr] [gateway6]) +# +# Fake a veth by creating a tap on kernel side and plug it in OVS using the +# net/tap DPDK driver. +# +m4_define([ADD_VETH], + [AT_CHECK([ovs-vsctl add-port $3 ovs-$1 -- \ + set interface ovs-$1 external-ids:iface-id="$1" -- \ + set interface ovs-$1 type=dpdk -- \ + set interface ovs-$1 options:n_rxq=2 -- \ + set interface ovs-$1 options:dpdk-devargs=net_tap$1,iface=$1]) + OVS_WAIT_UNTIL([ip link show dev $1 | grep -qw LOWER_UP]) + AT_CHECK([ip link set $1 netns $2]) + NS_CHECK_EXEC([$2], [ip addr add $4 dev $1 $7]) + NS_CHECK_EXEC([$2], [ip link set dev $1 up]) + if test -n "$5"; then + NS_CHECK_EXEC([$2], [ip link set dev $1 address $5]) + fi + if test -n "$6"; then + NS_CHECK_EXEC([$2], [ip route add $6 dev $1]) + NS_CHECK_EXEC([$2], [ip route add default via $6]) + fi + if test -n "$8"; then + NS_CHECK_EXEC([$2], [ip addr add $8 dev $1]) + fi + if test -n "$9"; then + NS_CHECK_EXEC([$2], [ip route add default via $9]) + fi +]) diff --git a/tests/system-dpdk-testsuite.at b/tests/system-dpdk-testsuite.at new file mode 100644 index 000000000..72ddc3913 --- /dev/null +++ b/tests/system-dpdk-testsuite.at @@ -0,0 +1,25 @@ +AT_INIT + +AT_COPYRIGHT([Copyright (c) 2023 Red Hat, Inc. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at: + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License.]) + +m4_ifdef([AT_COLOR_TESTS], [AT_COLOR_TESTS]) + +m4_include([tests/ovs-macros.at]) +m4_include([tests/ovsdb-macros.at]) +m4_include([tests/ofproto-macros.at]) +m4_include([tests/ovn-macros.at]) +m4_include([tests/system-common-macros.at]) +m4_include([tests/system-dpdk-macros.at]) +m4_include([tests/system-ovn.at]) diff --git a/tests/system-ovn.at b/tests/system-ovn.at index c2490008d..05c234edc 100644 --- a/tests/system-ovn.at +++ b/tests/system-ovn.at @@ -4189,7 +4189,7 @@ icmp,orig=(src=192.168.1.10,dst=192.168.1.20,id=,type=8,code=0),reply=( check ovs-vsctl clear interface ovs-p1 external_ids # check conntrack zone is flushed -check ovs-appctl dpctl/dump-conntrack zone=$zone_id +OVS_WAIT_WHILE([ovs-appctl dpctl/dump-conntrack zone=$zone_id | grep .]) OVS_APP_EXIT_AND_WAIT([ovn-controller]) @@ -6574,6 +6574,9 @@ ADD_NAMESPACES(public) ADD_VETH(public, public, br-public, "192.168.2.2/24", "f0:00:00:01:02:05") AT_CHECK([ovs-vsctl remove interface ovs-public external-ids iface-id=public]) +# Skip this test if the datapath interface does not support linux-htb +AT_SKIP_IF([! ovs-appctl qos/show-types ovs-public | grep -q linux-htb]) + ADD_NAMESPACES(ext) ADD_VETH(ext, ext, br-ext, "192.168.3.2/24", "f0:00:00:01:02:06") AT_CHECK([ovs-vsctl remove interface ovs-ext external-ids iface-id=ext]) @@ -11202,6 +11205,12 @@ ADD_VETH(bar1, bar1, br-int, "2002::2/64", "f0:00:00:01:02:05", \ ovn-nbctl lsp-add bar bar1 \ -- lsp-set-addresses bar1 "f0:00:00:01:02:05 192.168.2.2 2002::2" +# Warm up the datapath (needed to make the DPDK datapth happy) +NS_CHECK_EXEC([foo1], [ping -q -c 3 -i 0.3 -w 2 192.168.2.2 | FORMAT_PING], \ +[0], [dnl +3 packets transmitted, 3 received, 0% packet loss, time 0ms +]) + ovn-nbctl mirror-add mirror0 gre 1 to-lport 172.16.0.100 ovn-nbctl lsp-attach-mirror bar1 mirror0 From patchwork Mon Jun 19 09:32:15 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eelco Chaudron X-Patchwork-Id: 1796439 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::138; helo=smtp1.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=VOOx9ns7; dkim-atps=neutral Received: from smtp1.osuosl.org (smtp1.osuosl.org [IPv6:2605:bc80:3010::138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4Ql4L54q5hz214Z for ; Mon, 19 Jun 2023 19:32:29 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 8A9D881E21; Mon, 19 Jun 2023 09:32:27 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 8A9D881E21 Authentication-Results: smtp1.osuosl.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=VOOx9ns7 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id eCJn8a9IuV0d; Mon, 19 Jun 2023 09:32:26 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp1.osuosl.org (Postfix) with ESMTPS id 13B0681E26; Mon, 19 Jun 2023 09:32:25 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 13B0681E26 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id E14D6C007A; Mon, 19 Jun 2023 09:32:24 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp2.osuosl.org (smtp2.osuosl.org [IPv6:2605:bc80:3010::133]) by lists.linuxfoundation.org (Postfix) with ESMTP id B3C20C0029 for ; Mon, 19 Jun 2023 09:32:23 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 8197D40497 for ; Mon, 19 Jun 2023 09:32:23 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 8197D40497 Authentication-Results: smtp2.osuosl.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=VOOx9ns7 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id YuHNR4fpBWhP for ; Mon, 19 Jun 2023 09:32:22 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org D699740B78 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by smtp2.osuosl.org (Postfix) with ESMTPS id D699740B78 for ; Mon, 19 Jun 2023 09:32:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1687167140; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Id0cSWxEUFgsfOPDFvPSwuaeEq2HXPaRs9mWlMzVpRQ=; b=VOOx9ns7S+urSZjvYylHheIVE3qde/kmxmT/k7+jU4PPKruVsEZrVDbeJeFUCfHY9zRHzs FIkFz/IY3R1VDggPN71cZJX7N+baV5TobascAMFHupqd7OO1HXy0n+lElJCOgkx4iltUAB IZsHozPSwpRVuZABU8q82bCVYTZ+RHI= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-414-HS5FuJARM0ut9C5IMsqgaQ-1; Mon, 19 Jun 2023 05:32:19 -0400 X-MC-Unique: HS5FuJARM0ut9C5IMsqgaQ-1 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id EFF2785A5BB; Mon, 19 Jun 2023 09:32:18 +0000 (UTC) Received: from ebuild.redhat.com (unknown [10.39.195.16]) by smtp.corp.redhat.com (Postfix) with ESMTP id 1C88F446248; Mon, 19 Jun 2023 09:32:18 +0000 (UTC) From: Eelco Chaudron To: dev@openvswitch.org Date: Mon, 19 Jun 2023 11:32:15 +0200 Message-Id: <168716705958.3825993.6171159611541124989.stgit@ebuild> In-Reply-To: <168716703359.3825993.14983017346113283324.stgit@ebuild> References: <168716703359.3825993.14983017346113283324.stgit@ebuild> User-Agent: StGit/1.5 MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.9 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Cc: david.marchand@redhat.com, i.maximets@ovn.org, dceara@redhat.com Subject: [ovs-dev] [PATCH ovn v3 2/2] ci: Run the new check-system-dpdk tests as part of the ci. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" This patch includes changes made earlier by David in the ovs branch to cache the dpdk builds. Co-authored-by: David Marchand Signed-off-by: David Marchand Signed-off-by: Eelco Chaudron Acked-by: Dumitru Ceara Acked-by: Ales Musil --- v2: Replaced 'sleep 1' with '' after consulting with Dumitru. v3: No changes Note that I ran the full GitHub ci 20x on the v1 patchset. For v2, I did 10 runs of only the changed/fixed test. For v3, I only did 1 run. .ci/ci.sh | 12 ++++++- .ci/dpdk-build.sh | 54 ++++++++++++++++++++++++++++++ .ci/dpdk-prepare.sh | 11 ++++++ .ci/linux-build.sh | 49 ++++++++++++++++++++++++++- .github/workflows/test.yml | 80 ++++++++++++++++++++++++++++++++++++++++++++ Makefile.am | 2 + tests/system-ovn.at | 2 + 7 files changed, 207 insertions(+), 3 deletions(-) create mode 100755 .ci/dpdk-build.sh create mode 100755 .ci/dpdk-prepare.sh diff --git a/.ci/ci.sh b/.ci/ci.sh index 90942bab6..10f11939c 100755 --- a/.ci/ci.sh +++ b/.ci/ci.sh @@ -16,6 +16,7 @@ OVN_PATH=${OVN_PATH:-$PWD} OVS_PATH=${OVS_PATH:-$OVN_PATH/ovs} +DPDK_PATH=${DPDK_PATH:-$OVN_PATH/dpdk-dir} CONTAINER_CMD=${CONTAINER_CMD:-podman} CONTAINER_WORKSPACE="/workspace" CONTAINER_WORKDIR="/workspace/ovn-tmp" @@ -80,6 +81,10 @@ function copy_sources_to_workdir() { && \ cp -a $CONTAINER_WORKSPACE/ovs/. $CONTAINER_WORKDIR/ovs \ && \ + rm -rf $CONTAINER_WORKDIR/dpdk-dir \ + && \ + cp -a $CONTAINER_WORKSPACE/dpdk-dir/. $CONTAINER_WORKDIR/dpdk-dir \ + && \ git config --global --add safe.directory $CONTAINER_WORKDIR " } @@ -95,7 +100,7 @@ function run_tests() { cd $CONTAINER_WORKDIR \ && \ ARCH=$ARCH CC=$CC LIBS=$LIBS OPTS=$OPTS TESTSUITE=$TESTSUITE \ - TEST_RANGE=$TEST_RANGE SANITIZERS=$SANITIZERS \ + TEST_RANGE=$TEST_RANGE SANITIZERS=$SANITIZERS DPDK=$DPDK \ ./.ci/linux-build.sh " } @@ -148,12 +153,17 @@ if [ "$ARCH" = "aarch64" ]; then ASAN_OPTIONS="detect_leaks=0" fi +if [ -z "$DPDK" ]; then + mkdir -p "$DPDK_PATH" +fi + CONTAINER_ID="$($CONTAINER_CMD run --privileged -d \ --pids-limit=-1 \ --env ASAN_OPTIONS=$ASAN_OPTIONS \ -v /lib/modules/$(uname -r):/lib/modules/$(uname -r):ro \ -v $OVN_PATH:$CONTAINER_WORKSPACE/ovn:Z \ -v $OVS_PATH:$CONTAINER_WORKSPACE/ovs:Z \ + -v $DPDK_PATH:$CONTAINER_WORKSPACE/dpdk-dir:Z \ $IMAGE_NAME)" trap remove_container EXIT diff --git a/.ci/dpdk-build.sh b/.ci/dpdk-build.sh new file mode 100755 index 000000000..f44ac15b0 --- /dev/null +++ b/.ci/dpdk-build.sh @@ -0,0 +1,54 @@ +#!/bin/bash + +set -o errexit +set -x + +function build_dpdk() +{ + local VERSION_FILE="dpdk-dir/cached-version" + local DPDK_VER=$1 + local DPDK_OPTS="" + + rm -rf dpdk-dir + + if [ "${DPDK_VER##refs/*/}" != "${DPDK_VER}" ]; then + git clone --single-branch $DPDK_GIT dpdk-dir -b "${DPDK_VER##refs/*/}" + pushd dpdk-dir + git log -1 --oneline + else + wget https://fast.dpdk.org/rel/dpdk-$1.tar.xz + tar xvf dpdk-$1.tar.xz > /dev/null + DIR_NAME=$(tar -tf dpdk-$1.tar.xz | head -1 | cut -f1 -d"/") + mv ${DIR_NAME} dpdk-dir + pushd dpdk-dir + fi + + # Switching to 'default' machine to make dpdk-dir cache usable on + # different CPUs. We can't be sure that all CI machines are exactly same. + DPDK_OPTS="$DPDK_OPTS -Dmachine=default" + + # Disable building DPDK unit tests. Not needed for OVS build or tests. + DPDK_OPTS="$DPDK_OPTS -Dtests=false" + + # Disable DPDK developer mode, this results in less build checks and less + # meson verbose outputs. + DPDK_OPTS="$DPDK_OPTS -Ddeveloper_mode=disabled" + + # OVS compilation and the "ovn-system-dpdk" unit tests (run in the CI) + # only depend on virtio/tap drivers. + # We can disable all remaining drivers to save compilation time. + DPDK_OPTS="$DPDK_OPTS -Denable_drivers=net/null,net/tap,net/virtio" + + # Install DPDK using prefix. + DPDK_OPTS="$DPDK_OPTS --prefix=$(pwd)/build" + + meson $DPDK_OPTS build + ninja -C build + ninja -C build install + + echo "Installed DPDK in $(pwd)" + popd + echo "${DPDK_VER}" > ${VERSION_FILE} +} + +build_dpdk $DPDK_VER diff --git a/.ci/dpdk-prepare.sh b/.ci/dpdk-prepare.sh new file mode 100755 index 000000000..f7e6215dd --- /dev/null +++ b/.ci/dpdk-prepare.sh @@ -0,0 +1,11 @@ +#!/bin/bash + +set -ev + +# Installing wheel separately because it may be needed to build some +# of the packages during dependency backtracking and pip >= 22.0 will +# abort backtracking on build failures: +# https://github.com/pypa/pip/issues/10655 +pip3 install --disable-pip-version-check --user wheel +pip3 install --disable-pip-version-check --user pyelftools +pip3 install --user 'meson==0.53.2' diff --git a/.ci/linux-build.sh b/.ci/linux-build.sh index 907a0dc6c..5a79a52da 100755 --- a/.ci/linux-build.sh +++ b/.ci/linux-build.sh @@ -10,13 +10,54 @@ OVN_CFLAGS="" OPTS="$OPTS --enable-Werror" JOBS=${JOBS:-"-j4"} +function install_dpdk() +{ + local VERSION_FILE="dpdk-dir/cached-version" + local DPDK_LIB=$(pwd)/dpdk-dir/build/lib/x86_64-linux-gnu + + # Export the following path for pkg-config to find the .pc file. + export PKG_CONFIG_PATH=$DPDK_LIB/pkgconfig/:$PKG_CONFIG_PATH + + if [ ! -f "${VERSION_FILE}" ]; then + echo "Could not find DPDK in $(pwd)/dpdk-dir" + return 1 + fi + + # As we build inside a container we need to update the prefix. + sed -i -E "s|^prefix=.*|prefix=$(pwd)/dpdk-dir/build|" \ + "$DPDK_LIB/pkgconfig/libdpdk-libs.pc" + + # Update the library paths. + sudo ldconfig + echo "Found cached DPDK $(cat ${VERSION_FILE}) build in $(pwd)/dpdk-dir" +} + function configure_ovs() { + if [ "$DPDK" ]; then + # When DPDK is enabled, we need to build OVS twice. Once to have + # ovs-vswitchd with DPDK. But OVN does not like the OVS libraries to + # be compiled with DPDK enabled, hence we need a final clean build + # with this disabled. + install_dpdk + + pushd ovs + ./boot.sh && ./configure CFLAGS="${COMMON_CFLAGS}" --with-dpdk=static \ + $* || { cat config.log; exit 1; } + make $JOBS || { cat config.log; exit 1; } + cp vswitchd/ovs-vswitchd vswitchd/ovs-vswitchd_with_dpdk + popd + fi + pushd ovs ./boot.sh && ./configure CFLAGS="${COMMON_CFLAGS}" $* || \ - { cat config.log; exit 1; } + { cat config.log; exit 1; } make $JOBS || { cat config.log; exit 1; } popd + + if [ "$DPDK" ]; then + cp ovs/vswitchd/ovs-vswitchd_with_dpdk ovs/vswitchd/ovs-vswitchd + fi } function configure_ovn() @@ -104,6 +145,12 @@ if [ "$TESTSUITE" ]; then execute_system_tests "check-system-userspace" \ "system-userspace-testsuite.log" ;; + + "system-test-dpdk") + # The dpdk tests need huge page memory, so reserve some 2M pages. + sudo bash -c "echo 2048 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages" + execute_system_tests "check-system-dpdk" "system-dpdk-testsuite.log" + ;; esac else configure_ovn $OPTS diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml index 6e8eac8d1..fe2a14c40 100644 --- a/.github/workflows/test.yml +++ b/.github/workflows/test.yml @@ -12,11 +12,81 @@ concurrency: cancel-in-progress: true jobs: + build-dpdk: + env: + dependencies: gcc libnuma-dev ninja-build + CC: gcc + DPDK_GIT: https://dpdk.org/git/dpdk-stable + DPDK_VER: 22.11.1 + name: dpdk gcc + outputs: + dpdk_key: ${{ steps.gen_dpdk_key.outputs.key }} + runs-on: ubuntu-20.04 + timeout-minutes: 30 + + steps: + - name: checkout + uses: actions/checkout@v3 + + - name: update PATH + run: | + echo "$HOME/bin" >> $GITHUB_PATH + echo "$HOME/.local/bin" >> $GITHUB_PATH + + - name: create ci signature file for the dpdk cache key + # This will collect most of DPDK related lines, so hash will be different + # if something changed in a way we're building DPDK including DPDK_VER. + # This also allows us to use cache from any branch as long as version + # and a way we're building DPDK stays the same. + run: | + grep -irE 'RTE_|DPDK|meson|ninja' .ci/dpdk-* > dpdk-ci-signature + grep -rwE 'DPDK_GIT|DPDK_VER' .github/ >> dpdk-ci-signature + if [ "${DPDK_VER##refs/*/}" != "${DPDK_VER}" ]; then + git ls-remote --heads $DPDK_GIT $DPDK_VER >> dpdk-ci-signature + fi + cat dpdk-ci-signature + + - name: generate ci DPDK key + id: gen_dpdk_key + env: + ci_key: ${{ hashFiles('dpdk-ci-signature') }} + run: echo 'key=dpdk-${{ env.ci_key }}' >> $GITHUB_OUTPUT + + - name: cache + id: dpdk_cache + uses: actions/cache@v3 + with: + path: dpdk-dir + key: ${{ steps.gen_dpdk_key.outputs.key }} + + - name: set up python + if: steps.dpdk_cache.outputs.cache-hit != 'true' + uses: actions/setup-python@v4 + with: + python-version: '3.9' + + - name: update APT cache + if: steps.dpdk_cache.outputs.cache-hit != 'true' + run: sudo apt update || true + - name: install common dependencies + if: steps.dpdk_cache.outputs.cache-hit != 'true' + run: sudo apt install -y ${{ env.dependencies }} + + - name: prepare + if: steps.dpdk_cache.outputs.cache-hit != 'true' + run: ./.ci/dpdk-prepare.sh + + - name: build + if: steps.dpdk_cache.outputs.cache-hit != 'true' + run: ./.ci/dpdk-build.sh + build-linux: + needs: build-dpdk env: IMAGE_NAME: ghcr.io/ovn-org/ovn-tests:ubuntu ARCH: ${{ matrix.cfg.arch }} CC: ${{ matrix.cfg.compiler }} + DPDK: ${{ matrix.cfg.dpdk }} LIBS: ${{ matrix.cfg.libs }} OPTS: ${{ matrix.cfg.opts }} TESTSUITE: ${{ matrix.cfg.testsuite }} @@ -43,6 +113,9 @@ jobs: - { compiler: gcc, testsuite: test, libs: -ljemalloc, test_range: "-500" } - { compiler: gcc, testsuite: test, libs: -ljemalloc, test_range: "501-1000" } - { compiler: gcc, testsuite: test, libs: -ljemalloc, test_range: "1001-" } + - { compiler: gcc, testsuite: system-test-dpdk, dpdk: dpdk, test_range: "-100" } + - { compiler: gcc, testsuite: system-test-dpdk, dpdk: dpdk, test_range: "101-200" } + - { compiler: gcc, testsuite: system-test-dpdk, dpdk: dpdk, test_range: "201-" } - { compiler: gcc, testsuite: system-test-userspace, test_range: "-100" } - { compiler: gcc, testsuite: system-test-userspace, test_range: "101-200" } - { compiler: gcc, testsuite: system-test-userspace, test_range: "201-" } @@ -84,6 +157,13 @@ jobs: sort -V | tail -1) working-directory: ovs + - name: cache + if: matrix.cfg.dpdk != '' + uses: actions/cache@v3 + with: + path: dpdk-dir + key: ${{ needs.build-dpdk.outputs.dpdk_key }} + - name: build if: ${{ startsWith(matrix.cfg.testsuite, 'system-test') }} run: sudo -E ./.ci/ci.sh --archive-logs diff --git a/Makefile.am b/Makefile.am index f1bd72d94..27182c7bc 100644 --- a/Makefile.am +++ b/Makefile.am @@ -87,6 +87,8 @@ EXTRA_DIST = \ NOTICE \ .cirrus.yml \ .ci/ci.sh \ + .ci/dpdk-build.sh \ + .ci/dpdk-prepare.sh \ .ci/linux-build.sh \ .ci/osx-build.sh \ .ci/osx-prepare.sh \ diff --git a/tests/system-ovn.at b/tests/system-ovn.at index 05c234edc..a3f8fdaa9 100644 --- a/tests/system-ovn.at +++ b/tests/system-ovn.at @@ -4267,7 +4267,7 @@ done # Enable IGMP snooping on sw1. ovn-nbctl set Logical_Switch sw1 other_config:mcast_querier="false" ovn-nbctl set Logical_Switch sw1 other_config:mcast_snoop="true" - +ovn-nbctl --wait=hv sync group_v4="239.0.1.68" # Inject IGMP Join for v4 group on sw1-p1.