diff mbox series

[ovs-dev] ci: Run system tests in a separate namespace.

Message ID 20240122172307.3801928-1-i.maximets@ovn.org
State Accepted
Commit 432a0b93557331e8a5f6a66456f33efd49a2fc9b
Headers show
Series [ovs-dev] ci: Run system tests in a separate namespace. | expand

Checks

Context Check Description
ovsrobot/intel-ovs-compilation success test: success

Commit Message

Ilya Maximets Jan. 22, 2024, 5:23 p.m. UTC
GitHub runners use 10.1.0.0/16 network as their base network for eth0
interface.  That is causing random system test failures when unexpected
conntrack entries for this network are present, because our system
tests are mainly using 10.1.1.0/24 subnet for their test networks.

Run system tests in their own network namespace to avoid any unwanted
interference.

Ideally, we would run every single test in its own namespace, but that
is not a trivial change and will likely be hard to backport.  Still
worth investigating in the future.

Note: Layer3 tunnel tests with Bareudp ports rely on loopback to work,
but lo interface is down by default in new namespaces.  So, bringing
it up.  These tests are skipped in Ubuntu 22.04, because it doesn't
have bareudp support, but it's better to have the change anyway, so it
doesn't bite us in the future while upgrading the base image.

Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
---
 .ci/linux-build.sh | 4 ++++
 1 file changed, 4 insertions(+)

Comments

Eelco Chaudron Jan. 22, 2024, 6:57 p.m. UTC | #1
On 22 Jan 2024, at 18:23, Ilya Maximets wrote:

> GitHub runners use 10.1.0.0/16 network as their base network for eth0
> interface.  That is causing random system test failures when unexpected
> conntrack entries for this network are present, because our system
> tests are mainly using 10.1.1.0/24 subnet for their test networks.
>
> Run system tests in their own network namespace to avoid any unwanted
> interference.
>
> Ideally, we would run every single test in its own namespace, but that
> is not a trivial change and will likely be hard to backport.  Still
> worth investigating in the future.
>
> Note: Layer3 tunnel tests with Bareudp ports rely on loopback to work,
> but lo interface is down by default in new namespaces.  So, bringing
> it up.  These tests are skipped in Ubuntu 22.04, because it doesn't
> have bareudp support, but it's better to have the change anyway, so it
> doesn't bite us in the future while upgrading the base image.
>
> Signed-off-by: Ilya Maximets <i.maximets@ovn.org>

Thanks Ilya for posting this patch.

Acked-by: Eelco Chaudron <echaudro@redhat.com>
Eelco Chaudron Jan. 22, 2024, 7:31 p.m. UTC | #2
On 22 Jan 2024, at 18:23, Ilya Maximets wrote:

> GitHub runners use 10.1.0.0/16 network as their base network for eth0
> interface.  That is causing random system test failures when unexpected
> conntrack entries for this network are present, because our system
> tests are mainly using 10.1.1.0/24 subnet for their test networks.
>
> Run system tests in their own network namespace to avoid any unwanted
> interference.
>
> Ideally, we would run every single test in its own namespace, but that
> is not a trivial change and will likely be hard to backport.  Still
> worth investigating in the future.
>
> Note: Layer3 tunnel tests with Bareudp ports rely on loopback to work,
> but lo interface is down by default in new namespaces.  So, bringing
> it up.  These tests are skipped in Ubuntu 22.04, because it doesn't
> have bareudp support, but it's better to have the change anyway, so it
> doesn't bite us in the future while upgrading the base image.
>
> Signed-off-by: Ilya Maximets <i.maximets@ovn.org>

Applied to main and branch-3.3:

* 432a0b935 (origin/master) ci: Run system tests in a separate namespace.
* 2222dbb92 (origin/branch-3.3) ci: Run system tests in a separate namespace.

//Eelco
Simon Horman Jan. 22, 2024, 7:38 p.m. UTC | #3
On Mon, Jan 22, 2024 at 06:23:06PM +0100, Ilya Maximets wrote:
> GitHub runners use 10.1.0.0/16 network as their base network for eth0
> interface.  That is causing random system test failures when unexpected
> conntrack entries for this network are present, because our system
> tests are mainly using 10.1.1.0/24 subnet for their test networks.
> 
> Run system tests in their own network namespace to avoid any unwanted
> interference.
> 
> Ideally, we would run every single test in its own namespace, but that
> is not a trivial change and will likely be hard to backport.  Still
> worth investigating in the future.
> 
> Note: Layer3 tunnel tests with Bareudp ports rely on loopback to work,
> but lo interface is down by default in new namespaces.  So, bringing
> it up.  These tests are skipped in Ubuntu 22.04, because it doesn't
> have bareudp support, but it's better to have the change anyway, so it
> doesn't bite us in the future while upgrading the base image.
> 
> Signed-off-by: Ilya Maximets <i.maximets@ovn.org>

Acked-by: Simon Horman <horms@ovn.org>
diff mbox series

Patch

diff --git a/.ci/linux-build.sh b/.ci/linux-build.sh
index 7c2aebad8..bf9d6241d 100755
--- a/.ci/linux-build.sh
+++ b/.ci/linux-build.sh
@@ -157,6 +157,10 @@  else
         if [ "$testsuite" != "check" ] && \
            [ "$testsuite" != "check-ovsdb-cluster" ] ; then
             run_as_root="sudo -E PATH=$PATH GITHUB_ACTIONS=$GITHUB_ACTIONS"
+            sudo ip netns add ovs-system-test-ns
+            # Some system tests may rely on traffic loopback.
+            sudo ip -netns ovs-system-test-ns link set dev lo up
+            run_as_root="${run_as_root} ip netns exec ovs-system-test-ns"
         fi
         if [ "${testsuite##*dpdk}" != "$testsuite" ]; then
             sudo sh -c 'echo 1024 > /proc/sys/vm/nr_hugepages' || true