diff mbox series

[RFC,1/1] support/testing/tests/package/test_zfs: fix uclibc test by using bleeding-edge toolchain

Message ID 20250506155716.213768-1-ju.o@free.fr
State New
Headers show
Series [RFC,1/1] support/testing/tests/package/test_zfs: fix uclibc test by using bleeding-edge toolchain | expand

Commit Message

Julien Olivain May 6, 2025, 3:57 p.m. UTC 
Since commit [1] which bumped Python from 3.12.9 to 3.13.2, the runtime
test tests.package.test_zfs.TestZfsUclibc is failing with the error:

    WARN: defconfig can't be used
          Missing: BR2_PACKAGE_PYTHON3=y

This is because Python 3.13 requires time64, and uclibc supports it only
with toolchain headers >= 5.1. The stable bootlin toolchain is based on
4.19 headers and is no longer sufficient.

This commit fixes the issue by updating the defconfig of the runtime
test, to use the bleeding-edge toolchain (which is based on 5.15
headers).

Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/9922591461

[1] https://gitlab.com/buildroot.org/buildroot/-/commit/d63e207eb869063f82c867658676c2903beb08cb

Cc: Vincent Fazio <vfazio@xes-inc.com>
Signed-off-by: Julien Olivain <ju.o@free.fr>
---
While testing this patch in
https://gitlab.com/jolivain/buildroot/-/jobs/9935681972
it appears that it does fix the initial build issue, but it also
reveal a new runtime issue. The Python interpreter (from the
arc_summary script) crashes with a message:

    Fatal Python error: _PyThreadState_Attach: non-NULL old thread state
    Python runtime state: preinitialized
    Thread 0xSegmentation fault

So even when applying this patch, the runtime test still fails (it
just goes a bit farther). The issue now seems to be only related to
python 3.13 and uclibc.

Does anybody have an idea?
---
 support/testing/tests/package/test_zfs.py | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

Comments

José Luis Salvador Rufo May 6, 2025, 4:41 p.m. UTC | #1 
Hello Julien,

First, thank you for your patch. It fixes the ZFS test compilation issue
correctly. :thumb_up:

Now, regarding the second issue; it seems that the python3 binary is
somehow broken. If you try to run /usr/bin/python3, it will raise the same
problem. I am currently debugging this binary.

Greetings.

El mar, 6 may 2025 a las 17:57, Julien Olivain (<ju.o@free.fr>) escribió:

> Since commit [1] which bumped Python from 3.12.9 to 3.13.2, the runtime
> test tests.package.test_zfs.TestZfsUclibc is failing with the error:
>
>     WARN: defconfig can't be used
>           Missing: BR2_PACKAGE_PYTHON3=y
>
> This is because Python 3.13 requires time64, and uclibc supports it only
> with toolchain headers >= 5.1. The stable bootlin toolchain is based on
> 4.19 headers and is no longer sufficient.
>
> This commit fixes the issue by updating the defconfig of the runtime
> test, to use the bleeding-edge toolchain (which is based on 5.15
> headers).
>
> Fixes:
> https://gitlab.com/buildroot.org/buildroot/-/jobs/9922591461
>
> [1]
> https://gitlab.com/buildroot.org/buildroot/-/commit/d63e207eb869063f82c867658676c2903beb08cb
>
> Cc: Vincent Fazio <vfazio@xes-inc.com>
> Signed-off-by: Julien Olivain <ju.o@free.fr>
> ---
> While testing this patch in
> https://gitlab.com/jolivain/buildroot/-/jobs/9935681972
> it appears that it does fix the initial build issue, but it also
> reveal a new runtime issue. The Python interpreter (from the
> arc_summary script) crashes with a message:
>
>     Fatal Python error: _PyThreadState_Attach: non-NULL old thread state
>     Python runtime state: preinitialized
>     Thread 0xSegmentation fault
>
> So even when applying this patch, the runtime test still fails (it
> just goes a bit farther). The issue now seems to be only related to
> python 3.13 and uclibc.
>
> Does anybody have an idea?
> ---
>  support/testing/tests/package/test_zfs.py | 7 ++++++-
>  1 file changed, 6 insertions(+), 1 deletion(-)
>
> diff --git a/support/testing/tests/package/test_zfs.py
> b/support/testing/tests/package/test_zfs.py
> index 1b907ac4de..5c8fc58ac5 100644
> --- a/support/testing/tests/package/test_zfs.py
> +++ b/support/testing/tests/package/test_zfs.py
> @@ -74,9 +74,14 @@ class TestZfsGlibc(TestZfsBase):
>
>
>  class TestZfsUclibc(TestZfsBase):
> +    # This test uses uClibc bleeding edge toolchain, because it
> +    # selects python3 which needs time64 since 3.13. time64 support in
> +    # uclibc requires toolchain headers >= 5.1. The bleeding edge
> +    # toolchains are based on 5.15 headers, while the stable
> +    # toolchains are on 4.19.
>      config = TestZfsBase.config + \
>          """
> -        BR2_TOOLCHAIN_EXTERNAL_BOOTLIN_AARCH64_UCLIBC_STABLE=y
> +        BR2_TOOLCHAIN_EXTERNAL_BOOTLIN_AARCH64_UCLIBC_BLEEDING_EDGE=y
>          """
>
>      def test_run(self):
> --
> 2.49.0
>
>
José Luis Salvador Rufo May 6, 2025, 5:57 p.m. UTC | #2 
Hello @Vincent Fazio <vfazio@xes-inc.com> I saw that you submitted a
patch a few months ago to fix the thread address error:
https://raw.githubusercontent.com/buildroot/buildroot/d63e207eb869063f82c867658676c2903beb08cb/package/python3/0009-Fix-thread-identifiers-on-32-bit-musl-builds.patch

It looks like we might be encountering the same issue again
(si_addr=0x7373616c6362760b).
Could this be another upstream issue? (see:
https://github.com/python/cpython/blob/main/Python/thread_pthread.h#L312)

What do you think?

Here is the strace output:


# strace /usr/bin/python3.13
execve("/usr/bin/python3.13", ["/usr/bin/python3.13"], 0xffffe7396230
/* 10 vars */) = 0
readlinkat(AT_FDCWD, "/proc/self/exe", "/usr/bin/python3.13", 4096) = 19
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
0) = 0xffff81806000
openat(AT_FDCWD, "/lib/libpython3.13.so.1.0", O_RDONLY) = -1 ENOENT
(No such file or directory)
openat(AT_FDCWD, "/usr/lib/libpython3.13.so.1.0", O_RDONLY) = 3
fstat(3, {st_mode=S_IFREG|0755, st_size=5357776, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
0) = 0xffff81805000
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0\267\0\1\0\0\0\0\0\0\0\0\0\0\0"...,
4096) = 4096
mmap(NULL, 5828608, PROT_NONE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) =
0xffff81276000
mmap(0xffff81276000, 4514232, PROT_READ|PROT_EXEC,
MAP_PRIVATE|MAP_FIXED, 3, 0) = 0xffff81276000
mmap(0xffff816c5000, 842184, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_FIXED, 3, 0x44e000) = 0xffff816c5000
mmap(0xffff81793000, 465896, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xffff81793000
close(3)                                = 0
munmap(0xffff81805000, 4096)            = 0
openat(AT_FDCWD, "/lib/libc.so.0", O_RDONLY) = 3
fstat(3, {st_mode=S_IFREG|0755, st_size=723704, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
0) = 0xffff81805000
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0\267\0\1\0\0\0\0\0\0\0\0\0\0\0"...,
4096) = 4096
mmap(NULL, 761856, PROT_NONE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xffff811bc000
mmap(0xffff811bc000, 714260, PROT_READ|PROT_EXEC,
MAP_PRIVATE|MAP_FIXED, 3, 0) = 0xffff811bc000
mmap(0xffff8126b000, 5324, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_FIXED, 3, 0xaf000) = 0xffff8126b000
mmap(0xffff8126d000, 34024, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xffff8126d000
close(3)                                = 0
munmap(0xffff81805000, 4096)            = 0
openat(AT_FDCWD, "/lib/libc.so.0", O_RDONLY) = 3
fstat(3, {st_mode=S_IFREG|0755, st_size=723704, ...}) = 0
close(3)                                = 0
newfstatat(AT_FDCWD, "/lib/ld-uClibc.so.0", {st_mode=S_IFREG|0755,
st_size=33928, ...}, 0) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
0) = 0xffff81805000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
0) = 0xffff811bb000
mprotect(0xaaaade621000, 4096, PROT_READ) = 0
mprotect(0xffff816c5000, 323584, PROT_READ) = 0
mprotect(0xffff8126b000, 4096, PROT_READ) = 0
mprotect(0xffff81812000, 4096, PROT_READ) = 0
set_tid_address(0xffff811bb0d0)         = 106
set_robust_list(0xffff811bb0e0, 24)     = 0
rt_sigaction(SIGRTMIN, {sa_handler=0xffff812224a4, sa_mask=[],
sa_flags=SA_SIGINFO}, NULL, 8) = 0
rt_sigaction(SIGRT_1, {sa_handler=0xffff812223bc, sa_mask=[],
sa_flags=SA_RESTART|SA_SIGINFO}, NULL, 8) = 0
rt_sigprocmask(SIG_UNBLOCK, [RTMIN RT_1], NULL, 8) = 0
prlimit64(0, RLIMIT_STACK, NULL, {rlim_cur=8192*1024,
rlim_max=RLIM64_INFINITY}) = 0
ioctl(0, TCGETS, {c_iflag=ICRNL|IXON|IXOFF,
c_oflag=NL0|CR0|TAB0|BS0|VT0|FF0|OPOST|ONLCR,
c_cflag=B38400|CS8|CREAD|HUPCL|CLOCAL,
c_lflag=ISIG|ICANON|ECHO|ECHOE|ECHOK|ECHOCTL|ECHOKE, ...}) = 0
ioctl(1, TCGETS, {c_iflag=ICRNL|IXON|IXOFF,
c_oflag=NL0|CR0|TAB0|BS0|VT0|FF0|OPOST|ONLCR,
c_cflag=B38400|CS8|CREAD|HUPCL|CLOCAL,
c_lflag=ISIG|ICANON|ECHO|ECHOE|ECHOK|ECHOCTL|ECHOKE, ...}) = 0
getrandom("\xf2\x3f\xe8\x25\xff\x8b\xe6\xb2\xdf\x1b\xc2\x96\x20\x6c\x16\x55\xae\x06\x7a\xcb\xb5\x99\xbc\xf9\x3c\x21\xfd\x05\x52\xae\xff\x7b",
32, GRND_NONBLOCK) = 32
brk(NULL)                               = 0xaaaade623000
brk(0xaaaade624000)                     = 0xaaaade624000
openat(AT_FDCWD, "/proc/sys/vm/overcommit_memory", O_RDONLY) = 3
read(3, "0\n", 32)                      = 2
close(3)                                = 0
clock_gettime(CLOCK_MONOTONIC, {tv_sec=17, tv_nsec=16727584}) = 0
clock_gettime(CLOCK_MONOTONIC, {tv_sec=17, tv_nsec=24778944}) = 0
clock_gettime(CLOCK_MONOTONIC, {tv_sec=17, tv_nsec=25320784}) = 0
getrandom("\x90\xcb\x78\x23\xee\xdc\x38\x0b\x74\x1a\xf8\x3b\xb2\x54\xfe\xf0\x89\x10\x13\x08\x25\x8b\x45\x98",
24, GRND_NONBLOCK) = 24
gettid()                                = 106
write(2, "Fatal Python error: ", 20Fatal Python error: )    = 20
write(2, "_PyThreadState_Attach", 21_PyThreadState_Attach)   = 21
write(2, ": ", 2: )                       = 2
write(2, "non-NULL old thread state", 25non-NULL old thread state) = 25
write(2, "\n", 1
)                       = 1
write(2, "Python runtime state: ", 22Python runtime state: )  = 22
write(2, "preinitialized", 14preinitialized)          = 14
write(2, "\n", 1
)                       = 1
write(2, "\n", 1
)                       = 1
write(2, "Thread 0x", 9Thread 0x)                = 9
--- SIGSEGV {si_signo=SIGSEGV, si_code=SEGV_MAPERR,
si_addr=0x7373616c6362760b} ---
+++ killed by SIGSEGV +++
Segmentation fault
#

El mar, 6 may 2025 a las 18:41, José Luis Salvador Rufo
(<salvador.joseluis@gmail.com>) escribió:
>
> Hello Julien,
>
> First, thank you for your patch. It fixes the ZFS test compilation issue correctly. :thumb_up:
>
> Now, regarding the second issue; it seems that the python3 binary is somehow broken. If you try to run /usr/bin/python3, it will raise the same problem. I am currently debugging this binary.
>
> Greetings.
>
> El mar, 6 may 2025 a las 17:57, Julien Olivain (<ju.o@free.fr>) escribió:
>>
>> Since commit [1] which bumped Python from 3.12.9 to 3.13.2, the runtime
>> test tests.package.test_zfs.TestZfsUclibc is failing with the error:
>>
>>     WARN: defconfig can't be used
>>           Missing: BR2_PACKAGE_PYTHON3=y
>>
>> This is because Python 3.13 requires time64, and uclibc supports it only
>> with toolchain headers >= 5.1. The stable bootlin toolchain is based on
>> 4.19 headers and is no longer sufficient.
>>
>> This commit fixes the issue by updating the defconfig of the runtime
>> test, to use the bleeding-edge toolchain (which is based on 5.15
>> headers).
>>
>> Fixes:
>> https://gitlab.com/buildroot.org/buildroot/-/jobs/9922591461
>>
>> [1] https://gitlab.com/buildroot.org/buildroot/-/commit/d63e207eb869063f82c867658676c2903beb08cb
>>
>> Cc: Vincent Fazio <vfazio@xes-inc.com>
>> Signed-off-by: Julien Olivain <ju.o@free.fr>
>> ---
>> While testing this patch in
>> https://gitlab.com/jolivain/buildroot/-/jobs/9935681972
>> it appears that it does fix the initial build issue, but it also
>> reveal a new runtime issue. The Python interpreter (from the
>> arc_summary script) crashes with a message:
>>
>>     Fatal Python error: _PyThreadState_Attach: non-NULL old thread state
>>     Python runtime state: preinitialized
>>     Thread 0xSegmentation fault
>>
>> So even when applying this patch, the runtime test still fails (it
>> just goes a bit farther). The issue now seems to be only related to
>> python 3.13 and uclibc.
>>
>> Does anybody have an idea?
>> ---
>>  support/testing/tests/package/test_zfs.py | 7 ++++++-
>>  1 file changed, 6 insertions(+), 1 deletion(-)
>>
>> diff --git a/support/testing/tests/package/test_zfs.py b/support/testing/tests/package/test_zfs.py
>> index 1b907ac4de..5c8fc58ac5 100644
>> --- a/support/testing/tests/package/test_zfs.py
>> +++ b/support/testing/tests/package/test_zfs.py
>> @@ -74,9 +74,14 @@ class TestZfsGlibc(TestZfsBase):
>>
>>
>>  class TestZfsUclibc(TestZfsBase):
>> +    # This test uses uClibc bleeding edge toolchain, because it
>> +    # selects python3 which needs time64 since 3.13. time64 support in
>> +    # uclibc requires toolchain headers >= 5.1. The bleeding edge
>> +    # toolchains are based on 5.15 headers, while the stable
>> +    # toolchains are on 4.19.
>>      config = TestZfsBase.config + \
>>          """
>> -        BR2_TOOLCHAIN_EXTERNAL_BOOTLIN_AARCH64_UCLIBC_STABLE=y
>> +        BR2_TOOLCHAIN_EXTERNAL_BOOTLIN_AARCH64_UCLIBC_BLEEDING_EDGE=y
>>          """
>>
>>      def test_run(self):
>> --
>> 2.49.0
>>
diff mbox series

Patch

diff --git a/support/testing/tests/package/test_zfs.py b/support/testing/tests/package/test_zfs.py
index 1b907ac4de..5c8fc58ac5 100644
--- a/support/testing/tests/package/test_zfs.py
+++ b/support/testing/tests/package/test_zfs.py
@@ -74,9 +74,14 @@  class TestZfsGlibc(TestZfsBase):
 
 
 class TestZfsUclibc(TestZfsBase):
+    # This test uses uClibc bleeding edge toolchain, because it
+    # selects python3 which needs time64 since 3.13. time64 support in
+    # uclibc requires toolchain headers >= 5.1. The bleeding edge
+    # toolchains are based on 5.15 headers, while the stable
+    # toolchains are on 4.19.
     config = TestZfsBase.config + \
         """
-        BR2_TOOLCHAIN_EXTERNAL_BOOTLIN_AARCH64_UCLIBC_STABLE=y
+        BR2_TOOLCHAIN_EXTERNAL_BOOTLIN_AARCH64_UCLIBC_BLEEDING_EDGE=y
         """
 
     def test_run(self):