| Message ID | 20230209082828.193762-1-po-hsu.lin@canonical.com |
|---|---|
| Headers | show |
| Series | Fix return code for net/fcnal-test.sh test | expand |
On 2/9/23 1:28 AM, Po-Hsu Lin wrote: > [Impact] > The net/fcnal-test.sh on F/J won't return a non-zero value even with > some sub test cases failed, here is an example on Jammy: > > # Tests passed: 857 > # Tests failed: 5 > ok 1 selftests: net: fcnal-test.sh > > Therefore it's marked as PASSED on our report and making it difficult > to spot these failures: > # With VRF > # SYSCTL: net.ipv4.raw_l3mdev_accept=1 > # TEST: ping local, VRF bind - VRF IP [FAIL] > # TEST: ping local, device bind - ns-A IP [FAIL] > # SYSCTL: net.ipv4.ping_group_range=0 2147483647 > # SYSCTL: net.ipv4.raw_l3mdev_accept=1 > # TEST: ping local, VRF bind - VRF IP [FAIL] > # TEST: ping local, device bind - ns-A IP [FAIL] > # With VRF > # SYSCTL: net.ipv4.raw_l3mdev_accept=1 > # SYSCTL: net.ipv4.ping_group_range=0 2147483647 > # SYSCTL: net.ipv4.raw_l3mdev_accept=1 > # TEST: ping out, vrf device+address bind - ns-B IPv6 LLA [FAIL] > > [Fix] > * 0f8a3b48f9 selftests: net/fcnal-test.sh: add exit code > > This patch can be cherry-picked into Focal and Jammy kernel. > We don't have this test in Bionic, and this patch has already landed > on Kinetic and OEM-5.17 > > [Test] > Run the patched test, the return value will be 1 whenever the number > of failed test cases is not 0. > > [Where problems could occur] > Change limited to testing tools, but we're expected to see new failures > in Jammy. With bug 2006391 fixed later on, we will see failures > reported in F/K as well. > > Li Zhijian (1): > selftests: net/fcnal-test.sh: add exit code > > tools/testing/selftests/net/fcnal-test.sh | 8 ++++++++ > 1 file changed, 8 insertions(+) > Acked-by: Tim Gardner <tim.gardner@canonical.com>
On 09.02.23 09:28, Po-Hsu Lin wrote: > [Impact] > The net/fcnal-test.sh on F/J won't return a non-zero value even with > some sub test cases failed, here is an example on Jammy: > > # Tests passed: 857 > # Tests failed: 5 > ok 1 selftests: net: fcnal-test.sh > > Therefore it's marked as PASSED on our report and making it difficult > to spot these failures: > # With VRF > # SYSCTL: net.ipv4.raw_l3mdev_accept=1 > # TEST: ping local, VRF bind - VRF IP [FAIL] > # TEST: ping local, device bind - ns-A IP [FAIL] > # SYSCTL: net.ipv4.ping_group_range=0 2147483647 > # SYSCTL: net.ipv4.raw_l3mdev_accept=1 > # TEST: ping local, VRF bind - VRF IP [FAIL] > # TEST: ping local, device bind - ns-A IP [FAIL] > # With VRF > # SYSCTL: net.ipv4.raw_l3mdev_accept=1 > # SYSCTL: net.ipv4.ping_group_range=0 2147483647 > # SYSCTL: net.ipv4.raw_l3mdev_accept=1 > # TEST: ping out, vrf device+address bind - ns-B IPv6 LLA [FAIL] > > [Fix] > * 0f8a3b48f9 selftests: net/fcnal-test.sh: add exit code > > This patch can be cherry-picked into Focal and Jammy kernel. > We don't have this test in Bionic, and this patch has already landed > on Kinetic and OEM-5.17 > > [Test] > Run the patched test, the return value will be 1 whenever the number > of failed test cases is not 0. > > [Where problems could occur] > Change limited to testing tools, but we're expected to see new failures > in Jammy. With bug 2006391 fixed later on, we will see failures > reported in F/K as well. > > Li Zhijian (1): > selftests: net/fcnal-test.sh: add exit code > > tools/testing/selftests/net/fcnal-test.sh | 8 ++++++++ > 1 file changed, 8 insertions(+) > Applied to jammy,focal:linux/master-next. Thanks. -Stefan