From patchwork Tue Nov 23 14:15:02 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Marchand X-Patchwork-Id: 1558593 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=FnwkE5Ts; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.138; helo=smtp1.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4Hz5m072xJz9sXS for ; Wed, 24 Nov 2021 01:16:16 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 0D99480F63; Tue, 23 Nov 2021 14:16:15 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id tGJHxZnO5v0L; Tue, 23 Nov 2021 14:16:14 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp1.osuosl.org (Postfix) with ESMTPS id 25FE680F1F; Tue, 23 Nov 2021 14:16:13 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id F2E35C001E; Tue, 23 Nov 2021 14:16:12 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp4.osuosl.org (smtp4.osuosl.org [IPv6:2605:bc80:3010::137]) by lists.linuxfoundation.org (Postfix) with ESMTP id A66FCC0012 for ; Tue, 23 Nov 2021 14:16:11 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 95D97403BD for ; Tue, 23 Nov 2021 14:16:11 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Authentication-Results: smtp4.osuosl.org (amavisd-new); dkim=pass (1024-bit key) header.d=redhat.com Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id NPBZpO-cg127 for ; Tue, 23 Nov 2021 14:16:09 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by smtp4.osuosl.org (Postfix) with ESMTPS id 8FDA9402DF for ; Tue, 23 Nov 2021 14:16:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1637676968; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=OIpk3O1Gvu3HQI3DhWPgt7jySIthqvfsjlqiRGZpL0Q=; b=FnwkE5TsU84JOJTgrcVYFVmycb2Cek/E82/8K96nCZNVT4b1698+SeYJEgKh43v9zI/C9b Amru7Abcv5tDGSC9pkyAtWBxkN0VtYeMKe9au6A+DZyPgWTSBudZFYSEnPN4IgqPNQ87g0 V31qVwyK0kfxkdRZbRl+0qpDXu4Jhlg= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-480-iU2ALVoDM8OvhFko03CgjQ-1; Tue, 23 Nov 2021 09:16:03 -0500 X-MC-Unique: iU2ALVoDM8OvhFko03CgjQ-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 0A02739388; Tue, 23 Nov 2021 14:16:02 +0000 (UTC) Received: from dmarchan.remote.csb (unknown [10.40.192.8]) by smtp.corp.redhat.com (Postfix) with ESMTP id DAE3B101E5BF; Tue, 23 Nov 2021 14:15:12 +0000 (UTC) From: David Marchand To: dev@openvswitch.org Date: Tue, 23 Nov 2021 15:15:02 +0100 Message-Id: <20211123141504.4498-1-david.marchand@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=david.marchand@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Cc: i.maximets@ovn.org, fbl@sysclose.org, maxime.coquelin@redhat.com, kumar.amber@intel.com Subject: [ovs-dev] [PATCH v2 1/3] system-dpdk: Refactor common logs matching. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" Move EAL logs and commonly ignored logs to a common macro. Remove obsolete ones (like i40e [1] and timer [2] logs). Extend regex on hugepage logs since a check on hugepages availability is already present on OVS side, and as a consequence, we don't care about the warnings on availability for certain hugepage size. Add logs checks for MFEX tests that were missing them. 1: https://git.dpdk.org/dpdk/commit/?id=a075ce2b3e8c 2: https://git.dpdk.org/dpdk/commit/?id=c1077933d45b Signed-off-by: David Marchand Acked-by: Eelco Chaudron --- tests/system-dpdk.at | 53 ++++++++++++++++++++++---------------------- 1 file changed, 27 insertions(+), 26 deletions(-) diff --git a/tests/system-dpdk.at b/tests/system-dpdk.at index e0e750fde5..2f45613171 100644 --- a/tests/system-dpdk.at +++ b/tests/system-dpdk.at @@ -3,6 +3,12 @@ m4_define([CONFIGURE_VETH_OFFLOADS], AT_BANNER([OVS-DPDK unit tests]) +m4_define([SYSTEM_DPDK_ALLOWED_LOGS],[ +\@does not exist. The Open vSwitch kernel module is probably not loaded.@d +\@EAL: Invalid NUMA socket, default to 0@d +\@EAL: No \(available\|free\) hugepages reported in hugepages-@d +]) + dnl -------------------------------------------------------------------------- dnl Check if EAL init is successful AT_SETUP([OVS-DPDK - EAL init]) @@ -12,10 +18,7 @@ OVS_DPDK_START() AT_CHECK([grep "DPDK Enabled - initializing..." ovs-vswitchd.log], [], [stdout]) AT_CHECK([grep "EAL" ovs-vswitchd.log], [], [stdout]) AT_CHECK([grep "DPDK Enabled - initialized" ovs-vswitchd.log], [], [stdout]) -OVS_VSWITCHD_STOP(["/Global register is changed during/d -/EAL: Invalid NUMA socket, default to 0/d -/EAL: WARNING: cpu flags constant_tsc=yes nonstop_tsc=no -> using unreliable clock cycles !/d -/EAL: No free hugepages reported in hugepages-1048576kB/d"]) +OVS_VSWITCHD_STOP("[SYSTEM_DPDK_ALLOWED_LOGS]") AT_CLEANUP dnl -------------------------------------------------------------------------- @@ -37,12 +40,7 @@ sleep 2 dnl Clean up AT_CHECK([ovs-vsctl del-port br10 phy0], [], [stdout], [stderr]) -OVS_VSWITCHD_STOP("/does not exist. The Open vSwitch kernel module is probably not loaded./d -/Failed to enable flow control/d -/Global register is changed during/d -/EAL: WARNING: cpu flags constant_tsc=yes nonstop_tsc=no -> using unreliable clock cycles !/d -/EAL: No free hugepages reported in hugepages-1048576kB/d -") +OVS_VSWITCHD_STOP("[SYSTEM_DPDK_ALLOWED_LOGS]") AT_CLEANUP dnl -------------------------------------------------------------------------- @@ -68,13 +66,10 @@ AT_CHECK([grep "VHOST_CONFIG: $OVS_RUNDIR/dpdkvhostclient0: reconnecting..." ovs dnl Clean up AT_CHECK([ovs-vsctl del-port br10 dpdkvhostuserclient0], [], [stdout], [stderr]) -OVS_VSWITCHD_STOP(["\@does not exist. The Open vSwitch kernel module is probably not loaded.@d +OVS_VSWITCHD_STOP("m4_join([], [SYSTEM_DPDK_ALLOWED_LOGS], [ \@Failed to enable flow control@d \@VHOST_CONFIG: failed to connect to $OVS_RUNDIR/dpdkvhostclient0: No such file or directory@d -\@Global register is changed during@d -\@EAL: Invalid NUMA socket, default to 0@d -\@EAL: WARNING: cpu flags constant_tsc=yes nonstop_tsc=no -> using unreliable clock cycles !@d -\@EAL: No free hugepages reported in hugepages-1048576kB@d"]) +])") AT_CLEANUP dnl -------------------------------------------------------------------------- @@ -144,16 +139,12 @@ pkill -f -x -9 'tail -f /dev/null' dnl Clean up AT_CHECK([ovs-vsctl del-port br10 dpdkvhostuser0], [], [stdout], [stderr]) -OVS_VSWITCHD_STOP(["\@does not exist. The Open vSwitch kernel module is probably not loaded.@d +OVS_VSWITCHD_STOP("m4_join([], [SYSTEM_DPDK_ALLOWED_LOGS], [ \@Failed to enable flow control@d \@VHOST_CONFIG: recvmsg failed@d \@VHOST_CONFIG: failed to connect to $OVS_RUNDIR/dpdkvhostuser0: No such file or directory@d -\@Global register is changed during@d \@dpdkvhostuser ports are considered deprecated; please migrate to dpdkvhostuserclient ports.@d -\@failed to enumerate system datapaths: No such file or directory@d -\@EAL: Invalid NUMA socket, default to 0@d -\@EAL: WARNING: cpu flags constant_tsc=yes nonstop_tsc=no -> using unreliable clock cycles !@d -\@EAL: No free hugepages reported in hugepages-1048576kB@d"]) +])") AT_CLEANUP dnl -------------------------------------------------------------------------- @@ -220,16 +211,13 @@ pkill -f -x -9 'tail -f /dev/null' dnl Clean up AT_CHECK([ovs-vsctl del-port br10 dpdkvhostuserclient0], [], [stdout], [stderr]) -OVS_VSWITCHD_STOP(["\@does not exist. The Open vSwitch kernel module is probably not loaded.@d +OVS_VSWITCHD_STOP("m4_join([], [SYSTEM_DPDK_ALLOWED_LOGS], [ \@Failed to enable flow control@d \@VHOST_CONFIG: recvmsg failed@d \@VHOST_CONFIG: failed to connect to $OVS_RUNDIR/dpdkvhostclient0: No such file or directory@d -\@Global register is changed during@d \@dpdkvhostuser ports are considered deprecated; please migrate to dpdkvhostuserclient ports.@d \@failed to enumerate system datapaths: No such file or directory@d -\@EAL: Invalid NUMA socket, default to 0@d -\@EAL: WARNING: cpu flags constant_tsc=yes nonstop_tsc=no -> using unreliable clock cycles !@d -\@EAL: No free hugepages reported in hugepages-1048576kB@d"]) +])") AT_CLEANUP dnl -------------------------------------------------------------------------- @@ -256,6 +244,7 @@ OVS_WAIT_UNTIL([test `ovs-vsctl get interface p1 statistics | grep -oP 'rx_packe dnl Clean up AT_CHECK([ovs-vsctl del-port br0 p1], [], [stdout], [stderr]) +OVS_VSWITCHD_STOP("[SYSTEM_DPDK_ALLOWED_LOGS]") AT_CLEANUP dnl -------------------------------------------------------------------------- @@ -283,6 +272,7 @@ OVS_WAIT_UNTIL([test `ovs-vsctl get interface p1 statistics | grep -oP 'rx_packe dnl Clean up AT_CHECK([ovs-vsctl del-port br0 p1], [], [stdout], [stderr]) +OVS_VSWITCHD_STOP("[SYSTEM_DPDK_ALLOWED_LOGS]") AT_CLEANUP dnl -------------------------------------------------------------------------- @@ -390,5 +380,16 @@ ovs-appctl: ovs-vswitchd: server returned an error dnl Clean up AT_CHECK([ovs-vsctl del-port br0 p1], [], [stdout], [stderr]) +OVS_VSWITCHD_STOP("m4_join([], [SYSTEM_DPDK_ALLOWED_LOGS], [ +\@Error: unknown argument 1.@d +\@Error: invalid study_pkt_cnt value: xyz.@d +\@Error: unknown argument abcd.@d +\@Error: -pmd option requires a thread id argument.@d +\@Error: invalid study_pkt_cnt value: abcd.@d +\@Error: miniflow extract parser not changed, PMD thread passed is not valid: 'zero'. Pass a valid pmd thread ID.@d +\@Error: no miniflow extract name provided. Output of miniflow-parser-get shows implementation list.@d +\@Error: unknown miniflow extract implementation superstudy.@d +\@Error: invalid study_pkt_cnt value: -pmd.@d +])") AT_CLEANUP dnl dnl -------------------------------------------------------------------------- From patchwork Tue Nov 23 14:15:03 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Marchand X-Patchwork-Id: 1558595 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=Z01gyjw6; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.136; helo=smtp3.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4Hz5mY2qBcz9sXS for ; Wed, 24 Nov 2021 01:16:45 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 40C70600CD; Tue, 23 Nov 2021 14:16:43 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id HlLlgSE24Dvd; Tue, 23 Nov 2021 14:16:41 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp3.osuosl.org (Postfix) with ESMTPS id F300E60585; Tue, 23 Nov 2021 14:16:40 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id C8023C001E; Tue, 23 Nov 2021 14:16:40 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 877F4C0012 for ; Tue, 23 Nov 2021 14:16:39 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 47F9F60797 for ; Tue, 23 Nov 2021 14:16:34 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ynikAt_bgXlu for ; Tue, 23 Nov 2021 14:16:33 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by smtp3.osuosl.org (Postfix) with ESMTPS id 1C830600CD for ; Tue, 23 Nov 2021 14:16:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1637676992; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=AZSEN5GB8+3VVrvqmwaRaBIgpLt7dtS+oURzy4dxVMo=; b=Z01gyjw6UmaRNVktxtgKJbNRchKHMFXzp6s7pBXeKx/5lwEGWKioHEGNuKUvPzkrWEDkWq SfBTAGC+d+U2/HgZi2JP6RwV64FZnsseLycDfhrcO+y/NWVRYNRD3VubN8JogUQ+JTj/33 xiTqXJi8EC+N6a/azPSnoVUb04o/osY= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-337-TlYYPtLdOVajbOES4XV1xg-1; Tue, 23 Nov 2021 09:16:26 -0500 X-MC-Unique: TlYYPtLdOVajbOES4XV1xg-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 1E6CB87D563; Tue, 23 Nov 2021 14:16:25 +0000 (UTC) Received: from dmarchan.remote.csb (unknown [10.40.192.8]) by smtp.corp.redhat.com (Postfix) with ESMTP id DB149100760F; Tue, 23 Nov 2021 14:16:08 +0000 (UTC) From: David Marchand To: dev@openvswitch.org Date: Tue, 23 Nov 2021 15:15:03 +0100 Message-Id: <20211123141504.4498-2-david.marchand@redhat.com> In-Reply-To: <20211123141504.4498-1-david.marchand@redhat.com> References: <20211123141504.4498-1-david.marchand@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=david.marchand@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Cc: i.maximets@ovn.org, fbl@sysclose.org, maxime.coquelin@redhat.com, kumar.amber@intel.com Subject: [ovs-dev] [PATCH v2 2/3] system-dpdk: Use dummy-pmd port for packet injection. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" net_pcap is not always available in DPDK (like, in a dev environment when you forgot to install the libpcap-devel). On the other hand, OVS already has its own way to inject packets into a bridge. Let's make use of it. This solution is slower than net/pcap DPDK, so lower the number of expected packets so that it runs in OVS_CTL_TIMEOUT seconds. While at it, convert "known" packets from pcap to scapy so that the injected packets can be updated without having to read/write a pcap file. Note: this change also (avoids/) fixes a python exception in PcapWriter with scapy 2.4.3 that comes from EPEL. Suggested-by: Ilya Maximets Signed-off-by: David Marchand --- Changes since v1: - renamed generator script, - decreased packet count for fuzzy test, - simplified wait expression for packet count, --- tests/automake.mk | 5 ++-- tests/genpkts.py | 56 ++++++++++++++++++++++++++++++++++++ tests/mfex_fuzzy.py | 32 --------------------- tests/pcap/mfex_test.pcap | Bin 416 -> 0 bytes tests/system-dpdk-macros.at | 4 +-- tests/system-dpdk.at | 30 +++++++++++++++---- 6 files changed, 84 insertions(+), 43 deletions(-) create mode 100755 tests/genpkts.py delete mode 100755 tests/mfex_fuzzy.py delete mode 100644 tests/pcap/mfex_test.pcap diff --git a/tests/automake.mk b/tests/automake.mk index 43731d0973..c519a5cb36 100644 --- a/tests/automake.mk +++ b/tests/automake.mk @@ -145,8 +145,7 @@ $(srcdir)/tests/fuzz-regression-list.at: tests/automake.mk EXTRA_DIST += $(MFEX_AUTOVALIDATOR_TESTS) MFEX_AUTOVALIDATOR_TESTS = \ - tests/pcap/mfex_test.pcap \ - tests/mfex_fuzzy.py + tests/genpkts.py OVSDB_CLUSTER_TESTSUITE_AT = \ tests/ovsdb-cluster-testsuite.at \ @@ -518,7 +517,7 @@ tests_test_type_props_SOURCES = tests/test-type-props.c CHECK_PYFILES = \ tests/appctl.py \ tests/flowgen.py \ - tests/mfex_fuzzy.py \ + tests/genpkts.py \ tests/ovsdb-monitor-sort.py \ tests/test-daemon.py \ tests/test-json.py \ diff --git a/tests/genpkts.py b/tests/genpkts.py new file mode 100755 index 0000000000..e243bb960f --- /dev/null +++ b/tests/genpkts.py @@ -0,0 +1,56 @@ +#!/usr/bin/python3 + +import sys + +from scapy.all import RandMAC, RandIP, RandIP6, RandShort, fuzz +from scapy.all import IPv6, Dot1Q, IP, Ether, UDP, TCP + +if len(sys.argv) < 2: + print('usage: {} packets_count [fuzz]'.format(sys.argv[0])) + sys.exit(1) + +tmpl = [] + +if len(sys.argv) == 2: + eth = Ether(dst='ff:ff:ff:ff:ff:ff') + vlan = eth / Dot1Q(vlan=1) + p = eth / IP() / TCP(sport=20, dport=80, flags='SA', window=8192) + tmpl += [p.build().hex()] + p = eth / IP() / UDP(sport=53, dport=53) + tmpl += [p.build().hex()] + p = eth / IP() / TCP(sport=20, dport=80, flags='S', window=8192) + tmpl += [p.build().hex()] + p = eth / IP() / UDP(sport=53, dport=53) + tmpl += [p.build().hex()] + p = vlan / IP() / UDP(sport=53, dport=53) + tmpl += [p.build().hex()] + p = vlan / IP() / TCP(sport=20, dport=80, flags='S', window=8192) + tmpl += [p.build().hex()] +elif sys.argv[2] == 'fuzz': + # Generate random protocol bases, use a fuzz() over the combined packet + # for full fuzzing. + eth = Ether(src=RandMAC(), dst=RandMAC()) + vlan = Dot1Q() + ipv4 = IP(src=RandIP(), dst=RandIP()) + ipv6 = IPv6(src=RandIP6(), dst=RandIP6()) + udp = UDP(dport=RandShort(), sport=RandShort()) + tcp = TCP(dport=RandShort(), sport=RandShort()) + + # IPv4 packets with fuzzing + tmpl += [fuzz(eth / ipv4 / udp).build().hex()] + tmpl += [fuzz(eth / ipv4 / tcp).build().hex()] + tmpl += [fuzz(eth / vlan / ipv4 / udp).build().hex()] + tmpl += [fuzz(eth / vlan / ipv4 / tcp).build().hex()] + + # IPv6 packets with fuzzing + tmpl += [fuzz(eth / ipv6 / udp).build().hex()] + tmpl += [fuzz(eth / ipv6 / tcp).build().hex()] + tmpl += [fuzz(eth / vlan / ipv6 / udp).build().hex()] + tmpl += [fuzz(eth / vlan / ipv6 / tcp).build().hex()] + +i = 0 +count = int(sys.argv[1]) +while count != 0: + print(tmpl[i % len(tmpl)]) + i += 1 + count -= 1 diff --git a/tests/mfex_fuzzy.py b/tests/mfex_fuzzy.py deleted file mode 100755 index 3efe1152da..0000000000 --- a/tests/mfex_fuzzy.py +++ /dev/null @@ -1,32 +0,0 @@ -#!/usr/bin/python3 - -import sys - -from scapy.all import RandMAC, RandIP, PcapWriter, RandIP6, RandShort, fuzz -from scapy.all import IPv6, Dot1Q, IP, Ether, UDP, TCP - -path = str(sys.argv[1]) + "/pcap/fuzzy.pcap" -pktdump = PcapWriter(path, append=False, sync=True) - -for i in range(0, 2000): - - # Generate random protocol bases, use a fuzz() over the combined packet - # for full fuzzing. - eth = Ether(src=RandMAC(), dst=RandMAC()) - vlan = Dot1Q() - ipv4 = IP(src=RandIP(), dst=RandIP()) - ipv6 = IPv6(src=RandIP6(), dst=RandIP6()) - udp = UDP(dport=RandShort(), sport=RandShort()) - tcp = TCP(dport=RandShort(), sport=RandShort()) - - # IPv4 packets with fuzzing - pktdump.write(fuzz(eth / ipv4 / udp)) - pktdump.write(fuzz(eth / ipv4 / tcp)) - pktdump.write(fuzz(eth / vlan / ipv4 / udp)) - pktdump.write(fuzz(eth / vlan / ipv4 / tcp)) - - # IPv6 packets with fuzzing - pktdump.write(fuzz(eth / ipv6 / udp)) - pktdump.write(fuzz(eth / ipv6 / tcp)) - pktdump.write(fuzz(eth / vlan / ipv6 / udp)) - pktdump.write(fuzz(eth / vlan / ipv6 / tcp)) diff --git a/tests/pcap/mfex_test.pcap b/tests/pcap/mfex_test.pcap deleted file mode 100644 index 1aac67b8d643ecb016c758cba4cc32212a80f52a..0000000000000000000000000000000000000000 GIT binary patch literal 0 HcmV?d00001 literal 416 zcmca|c+)~A1{MYw`2U}Qff2}QK`M68ITRa|G@yFii5$Gfk6YL%z>@uY&}o| z2s4N<1VH2&7y^V87$)XGOtD~MV$cFgfG~zBGGJ2#YtF$KST_NTIwYriok6N4Vm)gX-Q@c^{cp<7_5LgK^UuU{2>VS0RZ!RQ+EIW diff --git a/tests/system-dpdk-macros.at b/tests/system-dpdk-macros.at index c6708caaf0..b3614f0565 100644 --- a/tests/system-dpdk-macros.at +++ b/tests/system-dpdk-macros.at @@ -37,7 +37,7 @@ m4_define([OVS_DPDK_PRE_PHY_SKIP], # # Create an empty database and start ovsdb-server. Add special configuration # dpdk-init to enable DPDK functionality. Start ovs-vswitchd connected to that -# database using system devices (no dummies). +# database using system devices. # m4_define([OVS_DPDK_START], [dnl Create database. @@ -59,7 +59,7 @@ m4_define([OVS_DPDK_START], AT_CHECK([ovs-vsctl --no-wait set Open_vSwitch . other_config:dpdk-init=true]) dnl Start ovs-vswitchd. - AT_CHECK([ovs-vswitchd --detach --no-chdir --pidfile --log-file -vvconn -vofproto_dpif -vunixctl], [0], [stdout], [stderr]) + AT_CHECK([ovs-vswitchd --enable-dummy --detach --no-chdir --pidfile --log-file -vvconn -vofproto_dpif -vunixctl], [0], [stdout], [stderr]) AT_CAPTURE_FILE([ovs-vswitchd.log]) on_exit "kill_ovs_vswitchd `cat ovs-vswitchd.pid`" ]) diff --git a/tests/system-dpdk.at b/tests/system-dpdk.at index 2f45613171..7b5d063f62 100644 --- a/tests/system-dpdk.at +++ b/tests/system-dpdk.at @@ -230,17 +230,23 @@ OVS_DPDK_START() dnl Add userspace bridge and attach it to OVS AT_CHECK([ovs-vsctl add-br br0 -- set bridge br0 datapath_type=netdev]) -AT_CHECK([ovs-vsctl add-port br0 p1 -- set Interface p1 type=dpdk options:dpdk-devargs=net_pcap1,rx_pcap=$srcdir/pcap/mfex_test.pcap,infinite_rx=1], [], [stdout], [stderr]) +AT_CHECK([ovs-vsctl add-port br0 p1 -- set interface p1 type=dummy-pmd]) AT_CHECK([ovs-vsctl show], [], [stdout]) AT_SKIP_IF([! ovs-appctl dpif-netdev/miniflow-parser-get | sed 1,4d | grep "True"], [], [dnl ]) +on_exit "pkill -f -x -9 '$PYTHON3 $srcdir/genpkts.py -1'" +($PYTHON3 $srcdir/genpkts.py -1 | while read pkt; do + ovs-appctl netdev-dummy/receive p1 "$pkt" || break + done) & + AT_CHECK([ovs-appctl dpif-netdev/miniflow-parser-set autovalidator], [0], [dnl Miniflow extract implementation set to autovalidator. ]) -OVS_WAIT_UNTIL([test `ovs-vsctl get interface p1 statistics | grep -oP 'rx_packets=\s*\K\d+'` -ge 1000]) +OVS_WAIT_UNTIL([test `ovs-vsctl get interface p1 statistics:rx_packets` -ge 1000]) +pkill -f -x -9 '$PYTHON3 $srcdir/genpkts.py -1' dnl Clean up AT_CHECK([ovs-vsctl del-port br0 p1], [], [stdout], [stderr]) @@ -253,22 +259,27 @@ dnl Add standard DPDK PHY port AT_SETUP([OVS-DPDK - MFEX Autovalidator Fuzzy]) AT_KEYWORDS([dpdk]) AT_SKIP_IF([! $PYTHON3 -c "import scapy"], [], []) -AT_CHECK([$PYTHON3 $srcdir/mfex_fuzzy.py $srcdir], [], [stdout]) OVS_DPDK_START() dnl Add userspace bridge and attach it to OVS AT_CHECK([ovs-vsctl add-br br0 -- set bridge br0 datapath_type=netdev]) -AT_CHECK([ovs-vsctl add-port br0 p1 -- set Interface p1 type=dpdk options:dpdk-devargs=net_pcap1,rx_pcap=$srcdir/pcap/fuzzy.pcap,infinite_rx=1], [], [stdout], [stderr]) +AT_CHECK([ovs-vsctl add-port br0 p1 -- set interface p1 type=dummy-pmd]) AT_CHECK([ovs-vsctl show], [], [stdout]) AT_SKIP_IF([! ovs-appctl dpif-netdev/miniflow-parser-get | sed 1,4d | grep "True"], [], [dnl ]) +on_exit "pkill -f -x -9 '$PYTHON3 $srcdir/genpkts.py -1 fuzz'" +($PYTHON3 $srcdir/genpkts.py -1 fuzz | while read pkt; do + ovs-appctl netdev-dummy/receive p1 "$pkt" || break + done) & + AT_CHECK([ovs-appctl dpif-netdev/miniflow-parser-set autovalidator], [0], [dnl Miniflow extract implementation set to autovalidator. ]) -OVS_WAIT_UNTIL([test `ovs-vsctl get interface p1 statistics | grep -oP 'rx_packets=\s*\K\d+'` -ge 100000]) +OVS_WAIT_UNTIL([test `ovs-vsctl get interface p1 statistics:rx_packets` -ge 1000]) +pkill -f -x -9 '$PYTHON3 $srcdir/genpkts.py -1 fuzz' dnl Clean up AT_CHECK([ovs-vsctl del-port br0 p1], [], [stdout], [stderr]) @@ -283,9 +294,14 @@ OVS_DPDK_START() AT_CHECK([ovs-vsctl --no-wait set Open_vSwitch . other_config:pmd-cpu-mask=0xC]) dnl Add userspace bridge and attach it to OVS AT_CHECK([ovs-vsctl add-br br0 -- set bridge br0 datapath_type=netdev]) -AT_CHECK([ovs-vsctl add-port br0 p1 -- set Interface p1 type=dpdk options:dpdk-devargs=net_pcap1,rx_pcap=$srcdir/pcap/mfex_test.pcap,infinite_rx=1], [], [stdout], [stderr]) +AT_CHECK([ovs-vsctl add-port br0 p1 -- set interface p1 type=dummy-pmd]) AT_CHECK([ovs-vsctl show], [], [stdout]) +on_exit "pkill -f -x -9 '$PYTHON3 $srcdir/genpkts.py -1'" +($PYTHON3 $srcdir/genpkts.py -1 | while read pkt; do + ovs-appctl netdev-dummy/receive p1 "$pkt" || break + done) & + AT_CHECK([ovs-appctl dpif-netdev/miniflow-parser-set scalar 1], [2], [], [dnl Error: unknown argument 1. @@ -378,6 +394,8 @@ Error: invalid study_pkt_cnt value: -pmd. ovs-appctl: ovs-vswitchd: server returned an error ]) +pkill -f -x -9 '$PYTHON3 $srcdir/genpkts.py -1' + dnl Clean up AT_CHECK([ovs-vsctl del-port br0 p1], [], [stdout], [stderr]) OVS_VSWITCHD_STOP("m4_join([], [SYSTEM_DPDK_ALLOWED_LOGS], [ From patchwork Tue Nov 23 14:15:04 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Marchand X-Patchwork-Id: 1558596 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=LXO+Biut; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.138; helo=smtp1.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4Hz5mb2CHfz9sXS for ; Wed, 24 Nov 2021 01:16:47 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 838FA80EF9; Tue, 23 Nov 2021 14:16:45 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id riuuPXnp2byV; Tue, 23 Nov 2021 14:16:44 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp1.osuosl.org (Postfix) with ESMTPS id 7DC1780F02; Tue, 23 Nov 2021 14:16:43 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id B64C9C0037; Tue, 23 Nov 2021 14:16:42 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 15D74C0036 for ; Tue, 23 Nov 2021 14:16:41 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id BA6E26077A for ; Tue, 23 Nov 2021 14:16:35 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Authentication-Results: smtp3.osuosl.org (amavisd-new); dkim=pass (1024-bit key) header.d=redhat.com Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Jjh82eQyxGFi for ; Tue, 23 Nov 2021 14:16:34 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by smtp3.osuosl.org (Postfix) with ESMTPS id 7C34B600CD for ; Tue, 23 Nov 2021 14:16:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1637676993; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ylHabAaWX7+0jQn69jsWjCE//BZ330jF69c6Qdyyh9Q=; b=LXO+BiutDZeZQi6i58wpkk+7OU/iePvkTTJEHHnuf2Ndg/eMFsl/+ERnP/0J0+0oGZn8Bd eWo18fy5gBOQ+8EZ9qwTtgri8JTyFtOoRjmAfEt/RFG6C2XPiRt/ievIFYf44z/AqLJziB /BPmTABvXGG6CKppbeCYTMRpMoSRr5k= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-448-u0e0pGKLOhqGpQqMUha2bA-1; Tue, 23 Nov 2021 09:16:30 -0500 X-MC-Unique: u0e0pGKLOhqGpQqMUha2bA-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 20E7710168EE; Tue, 23 Nov 2021 14:16:29 +0000 (UTC) Received: from dmarchan.remote.csb (unknown [10.40.192.8]) by smtp.corp.redhat.com (Postfix) with ESMTP id 00CF91045E84; Tue, 23 Nov 2021 14:16:26 +0000 (UTC) From: David Marchand To: dev@openvswitch.org Date: Tue, 23 Nov 2021 15:15:04 +0100 Message-Id: <20211123141504.4498-3-david.marchand@redhat.com> In-Reply-To: <20211123141504.4498-1-david.marchand@redhat.com> References: <20211123141504.4498-1-david.marchand@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=david.marchand@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Cc: i.maximets@ovn.org, fbl@sysclose.org, maxime.coquelin@redhat.com, kumar.amber@intel.com Subject: [ovs-dev] [PATCH v2 3/3] system-dpdk: Test with mlx5 devices. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" The DPDK unit test only runs if vfio or igb_uio kernel module is loaded: on systems with only mlx5, this test is always skipped. Besides, the test tries to grab the first device listed by dpdk-devbind.py, regardless of the PCI device status regarding kmod binding. Remove dependency on this DPDK script and use a minimal script that reads PCI sysfs. This script is not perfect, as one can imagine PCI devices bound to vfio-pci for virtual machines. Add a new environment variable DPDK_PCI_ADDR for testers to select the PCI device of their liking. For consistency and grep, the temporary file PCI_ADDR is renamed to DPDK_PCI_ADDR. Note: with mlx5 devices, there is now more OVS/DPDK warnings to waive. Signed-off-by: David Marchand Acked-by: Eelco Chaudron --- Documentation/topics/testing.rst | 1 - tests/automake.mk | 1 + tests/system-dpdk-find-device.py | 44 ++++++++++++++++++++++++++++++++ tests/system-dpdk-macros.at | 10 ++------ tests/system-dpdk.at | 4 ++- 5 files changed, 50 insertions(+), 10 deletions(-) create mode 100755 tests/system-dpdk-find-device.py diff --git a/Documentation/topics/testing.rst b/Documentation/topics/testing.rst index 0ddcbd58ab..00f734a77a 100644 --- a/Documentation/topics/testing.rst +++ b/Documentation/topics/testing.rst @@ -343,7 +343,6 @@ To see a list of all the available tests, run:: These tests support a `DPDK supported NIC`_. The tests operate on a wider set of environments, for instance, when a virtual port is used. -They do require proper DPDK variables (``DPDK_DIR`` and ``DPDK_BUILD``). Moreover you need to have root privileges to load the required modules and to bind the NIC to the DPDK-compatible driver. diff --git a/tests/automake.mk b/tests/automake.mk index c519a5cb36..ead92ace0f 100644 --- a/tests/automake.mk +++ b/tests/automake.mk @@ -188,6 +188,7 @@ SYSTEM_OFFLOADS_TESTSUITE_AT = \ SYSTEM_DPDK_TESTSUITE_AT = \ tests/system-common-macros.at \ + tests/system-dpdk-find-device.py \ tests/system-dpdk-macros.at \ tests/system-dpdk-testsuite.at \ tests/system-dpdk.at diff --git a/tests/system-dpdk-find-device.py b/tests/system-dpdk-find-device.py new file mode 100755 index 0000000000..4253326e75 --- /dev/null +++ b/tests/system-dpdk-find-device.py @@ -0,0 +1,44 @@ +#!/usr/bin/env python3 +# Copyright (c) 2021 Red Hat, Inc. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + +from pathlib import Path +import os +import sys + +# The tester might want to select a PCI device, if so, trust it. +if 'DPDK_PCI_ADDR' in os.environ: + print(os.environ['DPDK_PCI_ADDR']) + sys.exit(0) + +mlx5_ib_available = Path('/sys/module/mlx5_ib').exists() + +for device in sorted(Path('/sys/bus/pci/devices').iterdir()): + class_path = device / 'class' + # Only consider Network class devices + if class_path.read_text().strip() != '0x020000': + continue + kmod_path = device / 'driver' / 'module' + kmod_name = kmod_path.resolve().name + # Devices of interest: + # - device is bound to vfio_pci or igb_uio, + # - device is bound to mlx5_core and mlx5 is loaded, + if (kmod_name not in ['vfio_pci', 'igb_uio'] and + (kmod_name not in ['mlx5_core'] or not mlx5_ib_available)): + continue + print(device.resolve().name) + sys.exit(0) + +sys.exit(1) diff --git a/tests/system-dpdk-macros.at b/tests/system-dpdk-macros.at index b3614f0565..066fae0ba5 100644 --- a/tests/system-dpdk-macros.at +++ b/tests/system-dpdk-macros.at @@ -22,14 +22,8 @@ m4_define([OVS_DPDK_PRE_PHY_SKIP], [dnl Perform the precheck OVS_DPDK_PRE_CHECK() - dnl Check if VFIO or UIO driver is loaded - AT_SKIP_IF([ ! (lsmod | grep -E "igb_uio|vfio") ], [], [stdout]) - - dnl Find PCI address candidate, skip if there is no DPDK-compatible NIC - AT_CHECK([$DPDK_DIR/usertools/dpdk-devbind.py -s | head -n +4 | tail -1], [], [stdout]) - AT_CHECK([cat stdout | cut -d" " -s -f1 > PCI_ADDR]) - AT_SKIP_IF([ ! test -s PCI_ADDR ]) - + dnl Check if a device is available for DPDK + AT_SKIP_IF([ ! $abs_top_srcdir/tests/system-dpdk-find-device.py > DPDK_PCI_ADDR]) ]) diff --git a/tests/system-dpdk.at b/tests/system-dpdk.at index 7b5d063f62..894e01316f 100644 --- a/tests/system-dpdk.at +++ b/tests/system-dpdk.at @@ -7,6 +7,8 @@ m4_define([SYSTEM_DPDK_ALLOWED_LOGS],[ \@does not exist. The Open vSwitch kernel module is probably not loaded.@d \@EAL: Invalid NUMA socket, default to 0@d \@EAL: No \(available\|free\) hugepages reported in hugepages-@d +\@Rx checksum offload is not supported on@d +\@does not support MTU configuration,@d ]) dnl -------------------------------------------------------------------------- @@ -34,7 +36,7 @@ OVS_DPDK_START() dnl Add userspace bridge and attach it to OVS AT_CHECK([ovs-vsctl add-br br10 -- set bridge br10 datapath_type=netdev]) -AT_CHECK([ovs-vsctl add-port br10 phy0 -- set Interface phy0 type=dpdk options:dpdk-devargs=$(cat PCI_ADDR)], [], [stdout], [stderr]) +AT_CHECK([ovs-vsctl add-port br10 phy0 -- set Interface phy0 type=dpdk options:dpdk-devargs=$(cat DPDK_PCI_ADDR)], [], [stdout], [stderr]) AT_CHECK([ovs-vsctl show], [], [stdout]) sleep 2