| Message ID | 5fc8206d-6755-43df-b307-eda1f4396b3c@ovn.org |
|---|---|
| State | Not Applicable |
| Headers | show |
| Series | [ovs-dev] Asan+Ubsan tests are crashing all the tests with new 20240310.1.0 GHA images | expand |
| Context | Check | Description |
|---|---|---|
| ovsrobot/apply-robot | warning | apply and check: warning |
| ovsrobot/github-robot-_Build_and_Test | fail | github build: failed |
| ovsrobot/intel-ovs-compilation | fail | test: fail |
On 3/11/24 20:47, Ilya Maximets wrote: > Something strange is happening. Every single test fails if > clang with asan+ubsan tests are running on 20240310.1.0 image. > > Example: https://github.com/ovsrobot/ovs/actions/runs/8237626849 > > Every built binary appears to be crashing. For exmple: > > # -*- compilation -*- > 740. json.at:261: testing exponent must contain at least one digit (1) - C ... > ../../../tests/json.at:261: printf %s "[1e]" > input > ../../../tests/json.at:261: ovstest test-json input > --- /dev/null 2024-03-11 19:04:04.198924076 +0000 > +++ /home/runner/work/ovs/ovs/openvswitch-3.3.90/_build/sub/tests/testsuite.dir/at-groups/740/stderr 2024-03-11 19:13:51.408292432 +0000 > @@ -0,0 +1 @@ > +/home/runner/work/ovs/ovs/openvswitch-3.3.90/_build/sub/tests/testsuite.dir/at-groups/740/test-source: line 29: 66606 Segmentation fault (core dumped) ovstest test-json input > stdout: > ../../../tests/json.at:261: exit code was 139, expected 1 > input: >> [1e]740. json.at:261: 740. exponent must contain at least one digit (1) - C (json.at:261): FAILED (json.at:261) > > > At this point, I don't know what is going on there. For now, filed an issue for github runner-images: https://github.com/actions/runner-images/issues/9491 > > If a job was lucky to get older 20240304 images, everything works fine. > > FYI. > > Best regards, Ilya Maximets.
On 3/11/24 21:00, Ilya Maximets wrote: > On 3/11/24 20:47, Ilya Maximets wrote: >> Something strange is happening. Every single test fails if >> clang with asan+ubsan tests are running on 20240310.1.0 image. >> >> Example: https://github.com/ovsrobot/ovs/actions/runs/8237626849 >> >> Every built binary appears to be crashing. For exmple: >> >> # -*- compilation -*- >> 740. json.at:261: testing exponent must contain at least one digit (1) - C ... >> ../../../tests/json.at:261: printf %s "[1e]" > input >> ../../../tests/json.at:261: ovstest test-json input >> --- /dev/null 2024-03-11 19:04:04.198924076 +0000 >> +++ /home/runner/work/ovs/ovs/openvswitch-3.3.90/_build/sub/tests/testsuite.dir/at-groups/740/stderr 2024-03-11 19:13:51.408292432 +0000 >> @@ -0,0 +1 @@ >> +/home/runner/work/ovs/ovs/openvswitch-3.3.90/_build/sub/tests/testsuite.dir/at-groups/740/test-source: line 29: 66606 Segmentation fault (core dumped) ovstest test-json input >> stdout: >> ../../../tests/json.at:261: exit code was 139, expected 1 >> input: >>> [1e]740. json.at:261: 740. exponent must contain at least one digit (1) - C (json.at:261): FAILED (json.at:261) >> >> >> At this point, I don't know what is going on there. > > For now, filed an issue for github runner-images: > https://github.com/actions/runner-images/issues/9491 Found the problem. Turned out to be an incompatibility with the high-entropy ASLR with a new kernel. More info here: https://github.com/actions/runner-images/issues/9491#issuecomment-1989718917 I'll post a patch with a workaround tomorrow if they will not fix the image quickly enough. > >> >> If a job was lucky to get older 20240304 images, everything works fine. >> >> FYI. >> >> Best regards, Ilya Maximets. >
--- /dev/null 2024-03-11 19:04:04.198924076 +0000 +++ /home/runner/work/ovs/ovs/openvswitch-3.3.90/_build/sub/tests/testsuite.dir/at-groups/740/stderr 2024-03-11 19:13:51.408292432 +0000 @@ -0,0 +1 @@ +/home/runner/work/ovs/ovs/openvswitch-3.3.90/_build/sub/tests/testsuite.dir/at-groups/740/test-source: line 29: 66606 Segmentation fault (core dumped) ovstest test-json input
Something strange is happening. Every single test fails if clang with asan+ubsan tests are running on 20240310.1.0 image. Example: https://github.com/ovsrobot/ovs/actions/runs/8237626849 Every built binary appears to be crashing. For exmple: # -*- compilation -*- 740. json.at:261: testing exponent must contain at least one digit (1) - C ... ../../../tests/json.at:261: printf %s "[1e]" > input ../../../tests/json.at:261: ovstest test-json input stdout: ../../../tests/json.at:261: exit code was 139, expected 1 input: > [1e]740. json.at:261: 740. exponent must contain at least one digit (1) - C (json.at:261): FAILED (json.at:261) At this point, I don't know what is going on there. If a job was lucky to get older 20240304 images, everything works fine. FYI. Best regards, Ilya Maximets.