From patchwork Thu Feb 9 08:28:27 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Po-Hsu Lin X-Patchwork-Id: 1739833 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ubuntu.com (client-ip=91.189.94.19; helo=huckleberry.canonical.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=canonical.com header.i=@canonical.com header.a=rsa-sha256 header.s=20210705 header.b=Lle7J0V2; dkim-atps=neutral Received: from huckleberry.canonical.com (huckleberry.canonical.com [91.189.94.19]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4PC9591lclz23fc for ; Thu, 9 Feb 2023 19:29:15 +1100 (AEDT) Received: from localhost ([127.0.0.1] helo=huckleberry.canonical.com) by huckleberry.canonical.com with esmtp (Exim 4.86_2) (envelope-from ) id 1pQ2Ih-0006tk-Eu; Thu, 09 Feb 2023 08:29:03 +0000 Received: from smtp-relay-canonical-0.internal ([10.131.114.83] helo=smtp-relay-canonical-0.canonical.com) by huckleberry.canonical.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1pQ2If-0006se-8q for kernel-team@lists.ubuntu.com; Thu, 09 Feb 2023 08:29:01 +0000 Received: from localhost.localdomain (1.general.phlin.uk.vpn [10.172.194.38]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by smtp-relay-canonical-0.canonical.com (Postfix) with ESMTPSA id A32C83F3F7 for ; Thu, 9 Feb 2023 08:28:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=canonical.com; s=20210705; t=1675931339; bh=lMtjZpD1ukMsJbJ9jWRjRBk1KSoc5ykplCWd4dvZVsY=; h=From:To:Subject:Date:Message-Id:MIME-Version; b=Lle7J0V2aOFaRfv2FzOypNiStVMdk4axYaYKF8tn9xhN3Mc3r2Fb315QeJOILsivl kdP6+T/ieGhIJMKkPMd7yh9rL5IantdvYdvRtewA4zr8acIkjsIqBpOTYy3c2q155h FL39ypLFkfjqVLPsy1FX533tvxVWd642zhPETV/CjBAwB9Cp/oHTeIsa9v0Ghiqstt orPGKafTe8qaO2ADO+3uS0/duE5w/trtqm+JJr2Xxco9MIC0cn9yqTKI9bC3Dob9Zi z+/Xr0WsjPnIu+lHJ1ZxHwnlc0cmfilVDzkHJlcgvRCqJPg6v2w2L+0EEtlb9rFdyR dhGtfycNh3ISg== From: Po-Hsu Lin To: kernel-team@lists.ubuntu.com Subject: [F/J][SRU][PATCH 0/1] Fix return code for net/fcnal-test.sh test Date: Thu, 9 Feb 2023 16:28:27 +0800 Message-Id: <20230209082828.193762-1-po-hsu.lin@canonical.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" [Impact] The net/fcnal-test.sh on F/J won't return a non-zero value even with some sub test cases failed, here is an example on Jammy: # Tests passed: 857 # Tests failed: 5 ok 1 selftests: net: fcnal-test.sh Therefore it's marked as PASSED on our report and making it difficult to spot these failures: # With VRF # SYSCTL: net.ipv4.raw_l3mdev_accept=1 # TEST: ping local, VRF bind - VRF IP [FAIL] # TEST: ping local, device bind - ns-A IP [FAIL] # SYSCTL: net.ipv4.ping_group_range=0 2147483647 # SYSCTL: net.ipv4.raw_l3mdev_accept=1 # TEST: ping local, VRF bind - VRF IP [FAIL] # TEST: ping local, device bind - ns-A IP [FAIL] # With VRF # SYSCTL: net.ipv4.raw_l3mdev_accept=1 # SYSCTL: net.ipv4.ping_group_range=0 2147483647 # SYSCTL: net.ipv4.raw_l3mdev_accept=1 # TEST: ping out, vrf device+address bind - ns-B IPv6 LLA [FAIL] [Fix] * 0f8a3b48f9 selftests: net/fcnal-test.sh: add exit code This patch can be cherry-picked into Focal and Jammy kernel. We don't have this test in Bionic, and this patch has already landed on Kinetic and OEM-5.17 [Test] Run the patched test, the return value will be 1 whenever the number of failed test cases is not 0. [Where problems could occur] Change limited to testing tools, but we're expected to see new failures in Jammy. With bug 2006391 fixed later on, we will see failures reported in F/K as well. Li Zhijian (1): selftests: net/fcnal-test.sh: add exit code tools/testing/selftests/net/fcnal-test.sh | 8 ++++++++ 1 file changed, 8 insertions(+) Acked-by: Tim Gardner