From patchwork Wed Nov 2 14:35:13 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ales Musil X-Patchwork-Id: 1698189 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::133; helo=smtp2.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=MVMQ5elR; dkim-atps=neutral Received: from smtp2.osuosl.org (smtp2.osuosl.org [IPv6:2605:bc80:3010::133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4N2TvJ09Vfz1yqS for ; Thu, 3 Nov 2022 01:35:23 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 0C87F401B7; Wed, 2 Nov 2022 14:35:22 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 0C87F401B7 Authentication-Results: smtp2.osuosl.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=MVMQ5elR X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ukOd3JZOMiWZ; Wed, 2 Nov 2022 14:35:21 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp2.osuosl.org (Postfix) with ESMTPS id 11F3E40012; Wed, 2 Nov 2022 14:35:20 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 11F3E40012 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id D0479C0033; Wed, 2 Nov 2022 14:35:19 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 002E0C002D for ; Wed, 2 Nov 2022 14:35:18 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id CF3F760E7F for ; Wed, 2 Nov 2022 14:35:18 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org CF3F760E7F Authentication-Results: smtp3.osuosl.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=MVMQ5elR X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id TuTqNZA5o5st for ; Wed, 2 Nov 2022 14:35:17 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 2DDC860B30 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by smtp3.osuosl.org (Postfix) with ESMTPS id 2DDC860B30 for ; Wed, 2 Nov 2022 14:35:17 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1667399716; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=iPnOwzaB93QmnWp3ftM79yykV280/gAJ5lpCNQ0DzBU=; b=MVMQ5elRR7W6JyhP8Y+MN0Fr9etsKQPb1AoLBMXGjaDesDRPC19EIe5+tycmVhvsL5hTe8 aHN8ARnCRLG8tSHu4M/C06bb+df6kR7+lQumv+mUKWvQwyUyn4C31QvvOz2FYT9qv2ZcC2 uhkVG0m6XDDTEf8OBVYr8EEJv3rEsYc= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-154-N7y8lFLjMsiiVeR9BP_cvA-1; Wed, 02 Nov 2022 10:35:15 -0400 X-MC-Unique: N7y8lFLjMsiiVeR9BP_cvA-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id E71AA101AA46 for ; Wed, 2 Nov 2022 14:35:14 +0000 (UTC) Received: from amusil.redhat.com (ovpn-195-18.brq.redhat.com [10.40.195.18]) by smtp.corp.redhat.com (Postfix) with ESMTP id 161E340C6E14; Wed, 2 Nov 2022 14:35:13 +0000 (UTC) From: Ales Musil To: dev@openvswitch.org Date: Wed, 2 Nov 2022 15:35:13 +0100 Message-Id: <20221102143513.274198-1-amusil@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.2 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Cc: dceara@redhat.com Subject: [ovs-dev] [PATCH ovn] ci: Update jobs to use numbers instead of test flags X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" To prevent some jobs not running after CI scripts updates use numbers instead of the test flags. This still allows us to use parallelization, but without worrying about skipping some tests by mistake. For "test" suites use 3 parts, with 1500 tests in mind. That should give as additional space for future tests. Currently, there is ~1200 tests. For sanitizers use 5 parts instead, as they are slower in general. For "system-test" use 3 parts, with 300 tests in mind. Currently, there is ~200 tests. In the end this patch reduces the number of jobs by 8 to 20, which is ok as there is a limit of 20 running in parallel [0]. [0] https://docs.github.com/en/actions/learn-github-actions/usage-limits-billing-and-administration#usage-limits Signed-off-by: Ales Musil --- .ci/linux-build.sh | 9 ++------ .github/workflows/test.yml | 44 ++++++++++++++++---------------------- 2 files changed, 20 insertions(+), 33 deletions(-) diff --git a/.ci/linux-build.sh b/.ci/linux-build.sh index 2b0782aea..dc1ca5240 100755 --- a/.ci/linux-build.sh +++ b/.ci/linux-build.sh @@ -47,15 +47,10 @@ else fi if [ "$TESTSUITE" ]; then - TESTSUITEFLAGS="" - if [[ ! -z $TESTSUITE_KW ]]; then - TESTSUITEFLAGS="-k $TESTSUITE_KW" - fi - if [ "$TESTSUITE" = "system-test" ]; then configure_ovn $OPTS make -j4 || { cat config.log; exit 1; } - if ! sudo make -j4 check-kernel TESTSUITEFLAGS="$TESTSUITEFLAGS" RECHECK=yes; then + if ! sudo make -j4 check-kernel TESTSUITEFLAGS="$TEST_RANGE" RECHECK=yes; then # system-kmod-testsuite.log is necessary for debugging. cat tests/system-kmod-testsuite.log exit 1 @@ -67,7 +62,7 @@ if [ "$TESTSUITE" ]; then export DISTCHECK_CONFIGURE_FLAGS="$OPTS" if ! make distcheck CFLAGS="${COMMON_CFLAGS} ${OVN_CFLAGS}" -j4 \ - TESTSUITEFLAGS="$TESTSUITEFLAGS -j4" RECHECK=yes + TESTSUITEFLAGS="-j4 $TEST_RANGE" RECHECK=yes then # testsuite.log is necessary for debugging. cat */_build/sub/tests/testsuite.log diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml index 7a59cd478..88c48dd2c 100644 --- a/.github/workflows/test.yml +++ b/.github/workflows/test.yml @@ -24,7 +24,7 @@ jobs: M32: ${{ matrix.cfg.m32 }} OPTS: ${{ matrix.cfg.opts }} TESTSUITE: ${{ matrix.cfg.testsuite }} - TESTSUITE_KW: ${{ matrix.cfg.testsuite_kw }} + TEST_RANGE: ${{ matrix.cfg.test_range }} SANITIZERS: ${{ matrix.cfg.sanitizers }} name: linux ${{ join(matrix.cfg.*, ' ') }} @@ -36,31 +36,23 @@ jobs: cfg: - { compiler: gcc, opts: --disable-ssl } - { compiler: clang, opts: --disable-ssl } - - { compiler: gcc, testsuite: test, testsuite_kw: "parallelization=yes,ovn_monitor_all=yes" } - - { compiler: gcc, testsuite: test, testsuite_kw: "parallelization=yes,ovn_monitor_all=no" } - - { compiler: gcc, testsuite: test, testsuite_kw: "parallelization=no,ovn_monitor_all=yes" } - - { compiler: gcc, testsuite: test, testsuite_kw: "parallelization=no,ovn_monitor_all=no" } - - { compiler: gcc, testsuite: test, testsuite_kw: "!ovn-northd" } - - { compiler: clang, testsuite: test, sanitizers: sanitizers, testsuite_kw: "parallelization=yes,ovn_monitor_all=yes" } - - { compiler: clang, testsuite: test, sanitizers: sanitizers, testsuite_kw: "parallelization=yes,ovn_monitor_all=no" } - - { compiler: clang, testsuite: test, sanitizers: sanitizers, testsuite_kw: "parallelization=no,ovn_monitor_all=yes" } - - { compiler: clang, testsuite: test, sanitizers: sanitizers, testsuite_kw: "parallelization=no,ovn_monitor_all=no" } - - { compiler: clang, testsuite: test, sanitizers: sanitizers, testsuite_kw: "!ovn-northd" } - - { compiler: gcc, testsuite: test, libs: -ljemalloc, testsuite_kw: "parallelization=yes,ovn_monitor_all=yes" } - - { compiler: gcc, testsuite: test, libs: -ljemalloc, testsuite_kw: "parallelization=yes,ovn_monitor_all=no" } - - { compiler: gcc, testsuite: test, libs: -ljemalloc, testsuite_kw: "parallelization=no,ovn_monitor_all=yes" } - - { compiler: gcc, testsuite: test, libs: -ljemalloc, testsuite_kw: "parallelization=no,ovn_monitor_all=no" } - - { compiler: gcc, testsuite: test, libs: -ljemalloc, testsuite_kw: "!ovn-northd" } - - { compiler: clang, testsuite: test, libs: -ljemalloc, testsuite_kw: "parallelization=yes,ovn_monitor_all=yes" } - - { compiler: clang, testsuite: test, libs: -ljemalloc, testsuite_kw: "parallelization=yes,ovn_monitor_all=no" } - - { compiler: clang, testsuite: test, libs: -ljemalloc, testsuite_kw: "parallelization=no,ovn_monitor_all=yes" } - - { compiler: clang, testsuite: test, libs: -ljemalloc, testsuite_kw: "parallelization=no,ovn_monitor_all=no" } - - { compiler: clang, testsuite: test, libs: -ljemalloc, testsuite_kw: "!ovn-northd" } - - { compiler: gcc, testsuite: system-test, testsuite_kw: "parallelization=yes,ovn_monitor_all=yes" } - - { compiler: gcc, testsuite: system-test, testsuite_kw: "parallelization=yes,ovn_monitor_all=no" } - - { compiler: gcc, testsuite: system-test, testsuite_kw: "parallelization=no,ovn_monitor_all=yes" } - - { compiler: gcc, testsuite: system-test, testsuite_kw: "parallelization=no,ovn_monitor_all=no" } - - { compiler: gcc, testsuite: system-test, testsuite_kw: "!ovn-northd" } + - { compiler: gcc, testsuite: test, test_range: "-500" } + - { compiler: gcc, testsuite: test, test_range: "501-1000" } + - { compiler: gcc, testsuite: test, test_range: "1001-" } + - { compiler: clang, testsuite: test, sanitizers: sanitizers, test_range: "-300" } + - { compiler: clang, testsuite: test, sanitizers: sanitizers, test_range: "301-600" } + - { compiler: clang, testsuite: test, sanitizers: sanitizers, test_range: "601-900" } + - { compiler: clang, testsuite: test, sanitizers: sanitizers, test_range: "901-1200" } + - { compiler: clang, testsuite: test, sanitizers: sanitizers, test_range: "1201-" } + - { compiler: gcc, testsuite: test, libs: -ljemalloc, test_range: "-500" } + - { compiler: gcc, testsuite: test, libs: -ljemalloc, test_range: "501-1000" } + - { compiler: gcc, testsuite: test, libs: -ljemalloc, test_range: "1001-" } + - { compiler: clang, testsuite: test, libs: -ljemalloc, test_range: "-500" } + - { compiler: clang, testsuite: test, libs: -ljemalloc, test_range: "501-1000" } + - { compiler: clang, testsuite: test, libs: -ljemalloc, test_range: "1001-" } + - { compiler: gcc, testsuite: system-test, test_range: "-100" } + - { compiler: gcc, testsuite: system-test, test_range: "101-200" } + - { compiler: gcc, testsuite: system-test, test_range: "201-" } - { compiler: gcc, m32: m32, opts: --disable-ssl} steps: