From patchwork Wed Aug 30 08:28:19 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Marchand X-Patchwork-Id: 1827601 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=YjX7fGS4; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.137; helo=smtp4.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp4.osuosl.org (smtp4.osuosl.org [140.211.166.137]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4RbHWR6yRrz1ygM for ; Wed, 30 Aug 2023 18:28:51 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 10E1841A06; Wed, 30 Aug 2023 08:28:50 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 10E1841A06 Authentication-Results: smtp4.osuosl.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=YjX7fGS4 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id p-jFeejSzrL1; Wed, 30 Aug 2023 08:28:48 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp4.osuosl.org (Postfix) with ESMTPS id 2C0A241B05; Wed, 30 Aug 2023 08:28:47 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 2C0A241B05 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id E964FC0039; Wed, 30 Aug 2023 08:28:46 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by lists.linuxfoundation.org (Postfix) with ESMTP id 11D04C0039 for ; Wed, 30 Aug 2023 08:28:46 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 2B29041692 for ; Wed, 30 Aug 2023 08:28:44 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 2B29041692 Authentication-Results: smtp2.osuosl.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=YjX7fGS4 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id BijYeDsaZhgx for ; Wed, 30 Aug 2023 08:28:43 +0000 (UTC) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by smtp2.osuosl.org (Postfix) with ESMTPS id C4D59415D1 for ; Wed, 30 Aug 2023 08:28:42 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org C4D59415D1 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1693384121; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=tTLb5iScoMYqwJs3ZlstUQdIFY17ch28GjmSdf/yESY=; b=YjX7fGS46XQ4YZ60GZhDoX42Z1ZVbIn1Qz0o4OMIA8lDppXNm65COI+zmyJCPVFcRGOI7z PbUehti6cNvELFBfywlYIGL1EwGbzVo9MP2PrFyJkCDoCQW6tC9x9Jtw1gxJ64kGChTXEF jKrhpIBkOnHAVhdK4sQYvKxdn/TpIc0= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-627-oV_1307LNfKDz3kiycazZw-1; Wed, 30 Aug 2023 04:28:38 -0400 X-MC-Unique: oV_1307LNfKDz3kiycazZw-1 Received: from smtp.corp.redhat.com (int-mx10.intmail.prod.int.rdu2.redhat.com [10.11.54.10]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 8A35E858290; Wed, 30 Aug 2023 08:28:38 +0000 (UTC) Received: from dmarchan.redhat.com (unknown [10.45.225.206]) by smtp.corp.redhat.com (Postfix) with ESMTP id 2D345401E63; Wed, 30 Aug 2023 08:28:37 +0000 (UTC) From: David Marchand To: dev@openvswitch.org Date: Wed, 30 Aug 2023 10:28:19 +0200 Message-ID: <20230830082822.2168900-4-david.marchand@redhat.com> In-Reply-To: <20230830082822.2168900-1-david.marchand@redhat.com> References: <20230830082822.2168900-1-david.marchand@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.10 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Cc: Ilya Maximets , Flavio Leitner Subject: [ovs-dev] [PATCH v4 4/7] ci: Run DPDK tests in GitHub Actions. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" Let's enhance our coverage in the CI and run DPDK system tests. A few DPDK drivers are enabled in DPDK compilation. Put DPDK build in $PATH for dpdk-testpmd to be available. sudo drops PATH= updates and -E alone does not seem to preserve this variable. Pass PATH=$PATH when running the tests, as a workaround. Since those tests are run as root, the collection of logs is updated accordingly. In GHA, only two cores are available but some test rely on testpmd using three lcores. Add a DPDK_EAL_OPTIONS environment variable and use it to map all testpmd lcores to core 1 (and leave core 0 alone for OVS main and PMD threads). Signed-off-by: David Marchand Acked-by: Aaron Conole Acked-by: Eelco Chaudron --- Changes since v1: - rebased after DPDK build has been moved out of linux-build.sh, - restored running "normal" checks in the DPDK jobs, --- .ci/dpdk-build.sh | 8 +++++--- .ci/linux-build.sh | 15 ++++++++++++++- .github/workflows/build-and-test.yml | 7 ++++--- tests/system-dpdk-macros.at | 2 +- 4 files changed, 24 insertions(+), 8 deletions(-) diff --git a/.ci/dpdk-build.sh b/.ci/dpdk-build.sh index 02dcefef61..a754809d3d 100755 --- a/.ci/dpdk-build.sh +++ b/.ci/dpdk-build.sh @@ -35,9 +35,11 @@ function build_dpdk() DPDK_OPTS="$DPDK_OPTS -Ddeveloper_mode=disabled" # OVS compilation and "normal" unit tests (run in the CI) do not depend on - # any DPDK driver being present. - # We can disable all drivers to save compilation time. - DPDK_OPTS="$DPDK_OPTS -Ddisable_drivers=*/*" + # any DPDK driver. + # check-dpdk unit tests requires testpmd and some net/ driver. + # We can disable all drivers but them, in order to save compilation time. + DPDK_OPTS="$DPDK_OPTS -Denable_apps=test-pmd" + DPDK_OPTS="$DPDK_OPTS -Denable_drivers=net/null,net/tap,net/virtio" # Install DPDK using prefix. DPDK_OPTS="$DPDK_OPTS --prefix=$(pwd)/build" diff --git a/.ci/linux-build.sh b/.ci/linux-build.sh index 8227a57487..aa2ecc5050 100755 --- a/.ci/linux-build.sh +++ b/.ci/linux-build.sh @@ -22,6 +22,9 @@ function install_dpdk() # Export the following path for pkg-config to find the .pc file. export PKG_CONFIG_PATH=$DPDK_LIB/pkgconfig/:$PKG_CONFIG_PATH + # Expose dpdk binaries. + export PATH=$(pwd)/dpdk-dir/build/bin:$PATH + if [ ! -f "${VERSION_FILE}" ]; then echo "Could not find DPDK in $(pwd)/dpdk-dir" return 1 @@ -113,7 +116,7 @@ fi OPTS="${EXTRA_OPTS} ${OPTS} $*" -if [ "$TESTSUITE" ]; then +if [ "$TESTSUITE" = 'test' ]; then # 'distcheck' will reconfigure with required options. # Now we only need to prepare the Makefile without sparse-wrapped CC. configure_ovs @@ -123,6 +126,16 @@ if [ "$TESTSUITE" ]; then TESTSUITEFLAGS=-j4 RECHECK=yes else build_ovs + for testsuite in $TESTSUITE; do + run_as_root= + if [ "${testsuite##*dpdk}" != "$testsuite" ]; then + sudo sh -c 'echo 1024 > /proc/sys/vm/nr_hugepages' || true + [ "$(cat /proc/sys/vm/nr_hugepages)" = '1024' ] + export DPDK_EAL_OPTIONS="--lcores 0@1,1@1,2@1" + run_as_root="sudo -E PATH=$PATH" + fi + $run_as_root make $testsuite TESTSUITEFLAGS=-j4 RECHECK=yes + done fi exit 0 diff --git a/.github/workflows/build-and-test.yml b/.github/workflows/build-and-test.yml index bc5494e863..4f62efb7c3 100644 --- a/.github/workflows/build-and-test.yml +++ b/.github/workflows/build-and-test.yml @@ -123,10 +123,10 @@ jobs: opts: --enable-shared - compiler: gcc - testsuite: test + testsuite: check check-dpdk dpdk: dpdk - compiler: clang - testsuite: test + testsuite: check check-dpdk dpdk: dpdk - compiler: gcc @@ -213,7 +213,8 @@ jobs: mkdir logs cp config.log ./logs/ cp -r ./*/_build/sub/tests/testsuite.* ./logs/ || true - tar -czvf logs.tgz logs/ + sudo cp -r ./tests/*testsuite.* ./logs/ || true + sudo tar -czvf logs.tgz logs/ - name: upload logs on failure if: failure() || cancelled() diff --git a/tests/system-dpdk-macros.at b/tests/system-dpdk-macros.at index c149b9ce70..9a6685fafd 100644 --- a/tests/system-dpdk-macros.at +++ b/tests/system-dpdk-macros.at @@ -92,7 +92,7 @@ m4_define([OVS_DPDK_CHECK_TESTPMD], # Start dpdk-testpmd in background. # m4_define([OVS_DPDK_START_TESTPMD], - [eal_options="--in-memory --single-file-segments --no-pci" + [eal_options="$DPDK_EAL_OPTIONS --in-memory --single-file-segments --no-pci" options="$1" [ "$options" != "${options%% -- *}" ] || options="$options -- " eal_options="$eal_options ${options%% -- *}"