From patchwork Fri Oct 27 15:31:12 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Marchand X-Patchwork-Id: 1856291 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=FVWMQSGX; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.133; helo=smtp2.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4SH68d2Q8Lz23jV for ; Sat, 28 Oct 2023 02:31:45 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 5520643516; Fri, 27 Oct 2023 15:31:42 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 5520643516 Authentication-Results: smtp2.osuosl.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=FVWMQSGX X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id SRpDMoVfNX8x; Fri, 27 Oct 2023 15:31:40 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp2.osuosl.org (Postfix) with ESMTPS id DC2F443527; Fri, 27 Oct 2023 15:31:38 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org DC2F443527 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 1A62FC0032; Fri, 27 Oct 2023 15:31:38 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp4.osuosl.org (smtp4.osuosl.org [IPv6:2605:bc80:3010::137]) by lists.linuxfoundation.org (Postfix) with ESMTP id 1BF10C0DDD for ; Fri, 27 Oct 2023 15:31:37 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id E8FAD4ED96 for ; Fri, 27 Oct 2023 15:31:36 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org E8FAD4ED96 Authentication-Results: smtp4.osuosl.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=FVWMQSGX X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id rnHwnsJ7Mj_B for ; Fri, 27 Oct 2023 15:31:36 +0000 (UTC) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by smtp4.osuosl.org (Postfix) with ESMTPS id 0D63E4E818 for ; Fri, 27 Oct 2023 15:31:35 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 0D63E4E818 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1698420695; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=sF9msUtU6uoSROGqtcZ4RPLwJrE2/T1o7Yn4Sc+otPU=; b=FVWMQSGXjCIABaWFvu0OTwY4/Di0ZEI22kJy34pKSGySqkGK2NslQZr9ghvMFUTNLZP5j1 iJ+6tK/b3N/dW3PmYkWdRzkWftZzb+mEnNL7eNuroz+Ri4rwv/NWxI9hoH+R/GdDepd4be UurGXAMtwS/LfuAtbBINnr8uvW5etSg= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-157-EC-GqOfOMhqxZlbL0UEfZQ-1; Fri, 27 Oct 2023 11:31:32 -0400 X-MC-Unique: EC-GqOfOMhqxZlbL0UEfZQ-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id CEFE13C0E678; Fri, 27 Oct 2023 15:31:31 +0000 (UTC) Received: from dmarchan.redhat.com (unknown [10.45.225.98]) by smtp.corp.redhat.com (Postfix) with ESMTP id 30DC55027; Fri, 27 Oct 2023 15:31:30 +0000 (UTC) From: David Marchand To: dev@openvswitch.org Date: Fri, 27 Oct 2023 17:31:12 +0200 Message-ID: <20231027153118.580349-3-david.marchand@redhat.com> In-Reply-To: <20231027153118.580349-1-david.marchand@redhat.com> References: <20231027153118.580349-1-david.marchand@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.5 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Cc: Ilya Maximets , Flavio Leitner Subject: [ovs-dev] [PATCH v8 3/9] ci: Run DPDK tests in GitHub Actions. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" Let's enhance our coverage in the CI and run DPDK system tests. A few DPDK drivers are enabled in DPDK compilation. Put DPDK build in $PATH for dpdk-testpmd to be available. sudo drops PATH= updates and -E alone does not seem to preserve this variable. Pass PATH=$PATH when running the tests, as a workaround. Since those tests are run as root, the collection of logs is updated accordingly. In GHA, only two cores are available but some test rely on testpmd using three lcores. Add a DPDK_EAL_OPTIONS environment variable and use it to map all testpmd lcores to core 1 (and leave core 0 alone for OVS main and PMD threads). Signed-off-by: David Marchand Acked-by: Aaron Conole Acked-by: Eelco Chaudron --- Changes since v1: - rebased after DPDK build has been moved out of linux-build.sh, - restored running "normal" checks in the DPDK jobs, --- .ci/dpdk-build.sh | 7 ++++--- .ci/linux-build.sh | 15 ++++++++++++++- .github/workflows/build-and-test.yml | 7 ++++--- tests/system-dpdk-macros.at | 2 +- 4 files changed, 23 insertions(+), 8 deletions(-) diff --git a/.ci/dpdk-build.sh b/.ci/dpdk-build.sh index 02dcefef61..35540f0694 100755 --- a/.ci/dpdk-build.sh +++ b/.ci/dpdk-build.sh @@ -35,9 +35,10 @@ function build_dpdk() DPDK_OPTS="$DPDK_OPTS -Ddeveloper_mode=disabled" # OVS compilation and "normal" unit tests (run in the CI) do not depend on - # any DPDK driver being present. - # We can disable all drivers to save compilation time. - DPDK_OPTS="$DPDK_OPTS -Ddisable_drivers=*/*" + # any DPDK driver. + # check-dpdk unit tests requires testpmd and some net/ driver. + DPDK_OPTS="$DPDK_OPTS -Denable_apps=test-pmd" + DPDK_OPTS="$DPDK_OPTS -Denable_drivers=net/null,net/tap,net/virtio" # Install DPDK using prefix. DPDK_OPTS="$DPDK_OPTS --prefix=$(pwd)/build" diff --git a/.ci/linux-build.sh b/.ci/linux-build.sh index 8227a57487..aa2ecc5050 100755 --- a/.ci/linux-build.sh +++ b/.ci/linux-build.sh @@ -22,6 +22,9 @@ function install_dpdk() # Export the following path for pkg-config to find the .pc file. export PKG_CONFIG_PATH=$DPDK_LIB/pkgconfig/:$PKG_CONFIG_PATH + # Expose dpdk binaries. + export PATH=$(pwd)/dpdk-dir/build/bin:$PATH + if [ ! -f "${VERSION_FILE}" ]; then echo "Could not find DPDK in $(pwd)/dpdk-dir" return 1 @@ -113,7 +116,7 @@ fi OPTS="${EXTRA_OPTS} ${OPTS} $*" -if [ "$TESTSUITE" ]; then +if [ "$TESTSUITE" = 'test' ]; then # 'distcheck' will reconfigure with required options. # Now we only need to prepare the Makefile without sparse-wrapped CC. configure_ovs @@ -123,6 +126,16 @@ if [ "$TESTSUITE" ]; then TESTSUITEFLAGS=-j4 RECHECK=yes else build_ovs + for testsuite in $TESTSUITE; do + run_as_root= + if [ "${testsuite##*dpdk}" != "$testsuite" ]; then + sudo sh -c 'echo 1024 > /proc/sys/vm/nr_hugepages' || true + [ "$(cat /proc/sys/vm/nr_hugepages)" = '1024' ] + export DPDK_EAL_OPTIONS="--lcores 0@1,1@1,2@1" + run_as_root="sudo -E PATH=$PATH" + fi + $run_as_root make $testsuite TESTSUITEFLAGS=-j4 RECHECK=yes + done fi exit 0 diff --git a/.github/workflows/build-and-test.yml b/.github/workflows/build-and-test.yml index bc5494e863..4f62efb7c3 100644 --- a/.github/workflows/build-and-test.yml +++ b/.github/workflows/build-and-test.yml @@ -123,10 +123,10 @@ jobs: opts: --enable-shared - compiler: gcc - testsuite: test + testsuite: check check-dpdk dpdk: dpdk - compiler: clang - testsuite: test + testsuite: check check-dpdk dpdk: dpdk - compiler: gcc @@ -213,7 +213,8 @@ jobs: mkdir logs cp config.log ./logs/ cp -r ./*/_build/sub/tests/testsuite.* ./logs/ || true - tar -czvf logs.tgz logs/ + sudo cp -r ./tests/*testsuite.* ./logs/ || true + sudo tar -czvf logs.tgz logs/ - name: upload logs on failure if: failure() || cancelled() diff --git a/tests/system-dpdk-macros.at b/tests/system-dpdk-macros.at index a176a57a4b..35d14bee8f 100644 --- a/tests/system-dpdk-macros.at +++ b/tests/system-dpdk-macros.at @@ -94,7 +94,7 @@ m4_define([OVS_DPDK_CHECK_TESTPMD], m4_define([OVS_DPDK_START_TESTPMD], [AT_CHECK([lscpu], [], [stdout]) AT_CHECK([cat stdout | grep "NUMA node(s)" | awk '{c=1; while (c++<$(3)) {printf "512,"}; print "512"}' > NUMA_NODE]) - eal_options="--in-memory --socket-mem="$(cat NUMA_NODE)" --single-file-segments --no-pci" + eal_options="$DPDK_EAL_OPTIONS --in-memory --socket-mem="$(cat NUMA_NODE)" --single-file-segments --no-pci" options="$1" test "$options" != "${options%% -- *}" || options="$options -- " eal_options="$eal_options ${options%% -- *}"