diff mbox series

[1/1] support/testing/infra/basetest.py: use Bootlin toolchain bleeding edge 2022.08-1

Message ID 20230330112750.1710771-1-sebastian.weyer@smile.fr
State Superseded
Headers show
Series [1/1] support/testing/infra/basetest.py: use Bootlin toolchain bleeding edge 2022.08-1 | expand

Commit Message

Sebastian Weyer March 30, 2023, 11:27 a.m. UTC 
This patch updates the default toolchain used for runtime tests. The
last time this toolchain was updated was in commit
0207a65323fc73fe9570f3d7e03bbfed4ab6d477 3 years ago. Since then,
multiple things have changed:

Firstly, it used uclibc-ng as the libc whereas since commit
4057e36ca9665edd5248512e4edba2c243b8f4be, glibc is used as the default
library.

And secondly, since commit 531b2a10cda6bae78ecc141c9fe48f2024f71dbc, buildroot
dropped the support for gcc 8 and it cannot be built internally anymore.

This new toolchain bleeding edge 2022.08-1 is based on gcc 12.2.0, linux
headers 5.4.212, glibc 2.35 and binutils 2.39.
It is now required to add enable BR2_TOOLCHAIN_USES_GLIBC and
BR2_TOOLCHAIN_EXTERNAL_CUSTOM_GLIBC since uclibc-ng is still the
standard choice when using an external toolchain. If one were to change
this, it would break the builds for people using a uclibc toolchain with
a recent version of buildroot. BR2_TOOLCHAIN_EXTERNAL_LOCALE doesn't
need to be enabled anymore since it is implicit in glibc.
BR2_TOOLCHAIN_EXTERNAL_INET_RPC needs to be disabled since support has
been dropped by glibc but it is selected by default when choosing glibc.

The previous toolchain bleeding edge 2018.11-1 is based on gcc 8.2.0,
linux headers 4.14.80, uclibc 1.0.30 and binutils 2.31.1

Tested with gitlab:
https://gitlab.com/DocSepp/buildroot/-/pipelines/821745386/

Due to gitlab constraints it's not possible to check all the tests. Most
of the tests that did not succeed didn't because of such constraints.
The remaining fails are missing python or perl modules, timeouts because
the test takes more than 3h, tests.package.test_kexec.TestKexec  is
executed with a property that is unknown to qemu and several tests use a
different toolchain.

I noticed 4 tests, TestPython3Py, TestPython3Pyc, TestPython3PyPyc and
TestTmux that failed due to locale and libc.so.1 not found errors. I
verified that they did run with the previous toolchain and subsequently
introduced changes in their configs so the tests succeed with the new
toolchain.

Signed-off-by: Sebastian Weyer <sebastian.weyer@smile.fr>
---
 support/testing/infra/basetest.py            | 10 ++++++----
 support/testing/tests/package/test_python.py |  2 +-
 support/testing/tests/package/test_tmux.py   |  1 +
 3 files changed, 8 insertions(+), 5 deletions(-)

Comments

Romain Naour June 6, 2023, 8:50 p.m. UTC | #1 
Hello Sebastian,

Le 30/03/2023 à 13:27, Sebastian Weyer a écrit :
> This patch updates the default toolchain used for runtime tests. The
> last time this toolchain was updated was in commit
> 0207a65323fc73fe9570f3d7e03bbfed4ab6d477 3 years ago. Since then,
> multiple things have changed:
> 
> Firstly, it used uclibc-ng as the libc whereas since commit
> 4057e36ca9665edd5248512e4edba2c243b8f4be, glibc is used as the default
> library.
> 
> And secondly, since commit 531b2a10cda6bae78ecc141c9fe48f2024f71dbc, buildroot
> dropped the support for gcc 8 and it cannot be built internally anymore.
> 
> This new toolchain bleeding edge 2022.08-1 is based on gcc 12.2.0, linux
> headers 5.4.212, glibc 2.35 and binutils 2.39.

I'm not sure we really want to use the bleeding edge toolchain for the
testsuite. Please use the stable one that provide gcc 11 (the current gcc
default version).

> It is now required to add enable BR2_TOOLCHAIN_USES_GLIBC and
> BR2_TOOLCHAIN_EXTERNAL_CUSTOM_GLIBC since uclibc-ng is still the
> standard choice when using an external toolchain. If one were to change
> this, it would break the builds for people using a uclibc toolchain with
> a recent version of buildroot. BR2_TOOLCHAIN_EXTERNAL_LOCALE doesn't
> need to be enabled anymore since it is implicit in glibc.
> BR2_TOOLCHAIN_EXTERNAL_INET_RPC needs to be disabled since support has
> been dropped by glibc but it is selected by default when choosing glibc.

IIRC last time the BASIC_TOOLCHAIN_CONFIG was updated (2020.04), the Bootlin
toolchains were not packaged as external toolchain (added in 2020.08).

That's why at that time we used BR2_TOOLCHAIN_EXTERNAL_CUSTOM_GLIBC but now you
can use:

     BR2_arm=y
     BR2_TOOLCHAIN_EXTERNAL=y
     BR2_TOOLCHAIN_EXTERNAL_BOOTLIN=y
     BR2_TOOLCHAIN_EXTERNAL_BOOTLIN_ARMV5_EABI_GLIBC_STABLE=y

> 
> The previous toolchain bleeding edge 2018.11-1 is based on gcc 8.2.0,
> linux headers 4.14.80, uclibc 1.0.30 and binutils 2.31.1

If you look closer, you will notice that some tests are using explicitely a
glibc toolchain:

 class TestPythonPy3Autobahn(TestPythonPackageBase):
     __test__ = True
    # Need to use a different toolchain than the default due to
    # python-cryptography using Rust (not available with uclibc)
    config = \
         """
         BR2_arm=y
         BR2_TOOLCHAIN_EXTERNAL=y
         BR2_TOOLCHAIN_EXTERNAL_BOOTLIN=y
         BR2_TOOLCHAIN_EXTERNAL_BOOTLIN_ARMV5_EABI_GLIBC_STABLE=y

now you can simply use the toolchain provided your update :)

class TestPythonPy3Autobahn(TestPythonPackageBase):
     __test__ = True
     config = infra.basetest.BASIC_TOOLCHAIN_CONFIG + \
         """

When the test must be built with an uClibc-ng toolchain, you can do
    config = \
         """
         BR2_arm=y
         BR2_TOOLCHAIN_EXTERNAL=y
         BR2_TOOLCHAIN_EXTERNAL_BOOTLIN=y
         BR2_TOOLCHAIN_EXTERNAL_BOOTLIN_ARMV5_EABI_UCLIBC_STABLE=y

Some test are named with the libc implementation (TestOpenSshuClibc) but
since you switch to glibc, you have to update the toolchain configuration.
Otherwise a test TestOpenSshuClibc will be built with a Glibc toolchain.

Be careful with the rootfs size, glibc toolchain are "slightly" bigger that
uclibc toolchain, you may have to increase BR2_TARGET_ROOTFS_EXT2_SIZE.

> 
> Tested with gitlab:
> https://gitlab.com/DocSepp/buildroot/-/pipelines/821745386/
> 
> Due to gitlab constraints it's not possible to check all the tests. Most
> of the tests that did not succeed didn't because of such constraints.
> The remaining fails are missing python or perl modules, timeouts because
> the test takes more than 3h, tests.package.test_kexec.TestKexec  is
> executed with a property that is unknown to qemu and several tests use a
> different toolchain.
> 
> I noticed 4 tests, TestPython3Py, TestPython3Pyc, TestPython3PyPyc and
> TestTmux that failed due to locale and libc.so.1 not found errors. I
> verified that they did run with the previous toolchain and subsequently
> introduced changes in their configs so the tests succeed with the new
> toolchain.

I would suggest to explain why we have to update some tests due to the glibc
switch. The libc.so.1 update is easy but TestTmux update that now requires a
locale is not.

Best regards,
Romain

> 
> Signed-off-by: Sebastian Weyer <sebastian.weyer@smile.fr>
> ---
>  support/testing/infra/basetest.py            | 10 ++++++----
>  support/testing/tests/package/test_python.py |  2 +-
>  support/testing/tests/package/test_tmux.py   |  1 +
>  3 files changed, 8 insertions(+), 5 deletions(-)
> 
> diff --git a/support/testing/infra/basetest.py b/support/testing/infra/basetest.py
> index 670c7213d6..586b51e56e 100644
> --- a/support/testing/infra/basetest.py
> +++ b/support/testing/infra/basetest.py
> @@ -10,13 +10,15 @@ BASIC_TOOLCHAIN_CONFIG = \
>      BR2_arm=y
>      BR2_TOOLCHAIN_EXTERNAL=y
>      BR2_TOOLCHAIN_EXTERNAL_CUSTOM=y
> +    BR2_TOOLCHAIN_USES_GLIBC=y
> +    BR2_TOOLCHAIN_EXTERNAL_CUSTOM_GLIBC=y
>      BR2_TOOLCHAIN_EXTERNAL_DOWNLOAD=y
> -    BR2_TOOLCHAIN_EXTERNAL_URL="https://toolchains.bootlin.com/downloads/releases/toolchains/armv5-eabi/tarballs/armv5-eabi--uclibc--bleeding-edge-2018.11-1.tar.bz2"
> -    BR2_TOOLCHAIN_EXTERNAL_GCC_8=y
> -    BR2_TOOLCHAIN_EXTERNAL_HEADERS_4_14=y
> -    BR2_TOOLCHAIN_EXTERNAL_LOCALE=y
> +    BR2_TOOLCHAIN_EXTERNAL_URL="https://toolchains.bootlin.com/downloads/releases/toolchains/armv5-eabi/tarballs/armv5-eabi--glibc--bleeding-edge-2022.08-1.tar.bz2"
> +    BR2_TOOLCHAIN_EXTERNAL_GCC_12=y
> +    BR2_TOOLCHAIN_EXTERNAL_HEADERS_5_4=y
>      BR2_TOOLCHAIN_HAS_THREADS_DEBUG=y
>      BR2_TOOLCHAIN_EXTERNAL_CXX=y
> +    # BR2_TOOLCHAIN_EXTERNAL_INET_RPC is not set
>      """
>  
>  MINIMAL_CONFIG = \
> diff --git a/support/testing/tests/package/test_python.py b/support/testing/tests/package/test_python.py
> index d6f69a83c7..c9ecf9acda 100644
> --- a/support/testing/tests/package/test_python.py
> +++ b/support/testing/tests/package/test_python.py
> @@ -29,7 +29,7 @@ class TestPythonBase(infra.basetest.BRTest):
>      def libc_time_test(self, timeout=-1):
>          cmd = self.interpreter + " -c '"
>          cmd += "import ctypes;"
> -        cmd += "libc = ctypes.cdll.LoadLibrary(\"libc.so.1\");"
> +        cmd += "libc = ctypes.cdll.LoadLibrary(\"libc.so.6\");"
>          cmd += "print(libc.time(None))'"
>          self.assertRunOk(cmd, timeout)
>  
> diff --git a/support/testing/tests/package/test_tmux.py b/support/testing/tests/package/test_tmux.py
> index 7a10b7a2b0..f2c620d5ca 100644
> --- a/support/testing/tests/package/test_tmux.py
> +++ b/support/testing/tests/package/test_tmux.py
> @@ -9,6 +9,7 @@ class TestTmux(infra.basetest.BRTest):
>          BR2_PACKAGE_TMUX=y
>          BR2_TARGET_ROOTFS_CPIO=y
>          # BR2_TARGET_ROOTFS_TAR is not set
> +        BR2_GENERATE_LOCALE="C.UTF-8"
>          """
>  
>      def test_run(self):
Arnout Vandecappelle June 18, 2023, 12:48 p.m. UTC | #2 
On 30/03/2023 13:27, Sebastian Weyer wrote:
> This patch updates the default toolchain used for runtime tests. The
> last time this toolchain was updated was in commit
> 0207a65323fc73fe9570f3d7e03bbfed4ab6d477 3 years ago. Since then,
> multiple things have changed:
> 
> Firstly, it used uclibc-ng as the libc whereas since commit
> 4057e36ca9665edd5248512e4edba2c243b8f4be, glibc is used as the default
> library.

  IMHO, that's not a good reason to switch to uclibc for the tests as well. I 
think it's good that we do runtime tests with a compiler that is a bit different 
from the standard that is probably already used by the upstream package.


> And secondly, since commit 531b2a10cda6bae78ecc141c9fe48f2024f71dbc, buildroot
> dropped the support for gcc 8 and it cannot be built internally anymore.

  That's also not a good reason to switch to a different gcc for the runtime 
tests. But here, there's no reason to stick to GCC 8 either, so OK.


> This new toolchain bleeding edge 2022.08-1 is based on gcc 12.2.0, linux

  Like Romain, I don't think it's a good idea to use bleeding edge here.

> headers 5.4.212, glibc 2.35 and binutils 2.39.
> It is now required to add enable BR2_TOOLCHAIN_USES_GLIBC and
> BR2_TOOLCHAIN_EXTERNAL_CUSTOM_GLIBC since uclibc-ng is still the

  Actually, regardless of the default, I think it's a good idea to explicitly 
specify this.


  Regards,
  Arnout

> standard choice when using an external toolchain. If one were to change
> this, it would break the builds for people using a uclibc toolchain with
> a recent version of buildroot. BR2_TOOLCHAIN_EXTERNAL_LOCALE doesn't
> need to be enabled anymore since it is implicit in glibc.
> BR2_TOOLCHAIN_EXTERNAL_INET_RPC needs to be disabled since support has
> been dropped by glibc but it is selected by default when choosing glibc.
> 
> The previous toolchain bleeding edge 2018.11-1 is based on gcc 8.2.0,
> linux headers 4.14.80, uclibc 1.0.30 and binutils 2.31.1
> 
> Tested with gitlab:
> https://gitlab.com/DocSepp/buildroot/-/pipelines/821745386/
> 
> Due to gitlab constraints it's not possible to check all the tests. Most
> of the tests that did not succeed didn't because of such constraints.
> The remaining fails are missing python or perl modules, timeouts because
> the test takes more than 3h, tests.package.test_kexec.TestKexec  is
> executed with a property that is unknown to qemu and several tests use a
> different toolchain.
> 
> I noticed 4 tests, TestPython3Py, TestPython3Pyc, TestPython3PyPyc and
> TestTmux that failed due to locale and libc.so.1 not found errors. I
> verified that they did run with the previous toolchain and subsequently
> introduced changes in their configs so the tests succeed with the new
> toolchain.
> 
> Signed-off-by: Sebastian Weyer <sebastian.weyer@smile.fr>
> ---
>   support/testing/infra/basetest.py            | 10 ++++++----
>   support/testing/tests/package/test_python.py |  2 +-
>   support/testing/tests/package/test_tmux.py   |  1 +
>   3 files changed, 8 insertions(+), 5 deletions(-)
> 
> diff --git a/support/testing/infra/basetest.py b/support/testing/infra/basetest.py
> index 670c7213d6..586b51e56e 100644
> --- a/support/testing/infra/basetest.py
> +++ b/support/testing/infra/basetest.py
> @@ -10,13 +10,15 @@ BASIC_TOOLCHAIN_CONFIG = \
>       BR2_arm=y
>       BR2_TOOLCHAIN_EXTERNAL=y
>       BR2_TOOLCHAIN_EXTERNAL_CUSTOM=y
> +    BR2_TOOLCHAIN_USES_GLIBC=y
> +    BR2_TOOLCHAIN_EXTERNAL_CUSTOM_GLIBC=y
>       BR2_TOOLCHAIN_EXTERNAL_DOWNLOAD=y
> -    BR2_TOOLCHAIN_EXTERNAL_URL="https://toolchains.bootlin.com/downloads/releases/toolchains/armv5-eabi/tarballs/armv5-eabi--uclibc--bleeding-edge-2018.11-1.tar.bz2"
> -    BR2_TOOLCHAIN_EXTERNAL_GCC_8=y
> -    BR2_TOOLCHAIN_EXTERNAL_HEADERS_4_14=y
> -    BR2_TOOLCHAIN_EXTERNAL_LOCALE=y
> +    BR2_TOOLCHAIN_EXTERNAL_URL="https://toolchains.bootlin.com/downloads/releases/toolchains/armv5-eabi/tarballs/armv5-eabi--glibc--bleeding-edge-2022.08-1.tar.bz2"
> +    BR2_TOOLCHAIN_EXTERNAL_GCC_12=y
> +    BR2_TOOLCHAIN_EXTERNAL_HEADERS_5_4=y
>       BR2_TOOLCHAIN_HAS_THREADS_DEBUG=y
>       BR2_TOOLCHAIN_EXTERNAL_CXX=y
> +    # BR2_TOOLCHAIN_EXTERNAL_INET_RPC is not set
>       """
>   
>   MINIMAL_CONFIG = \
> diff --git a/support/testing/tests/package/test_python.py b/support/testing/tests/package/test_python.py
> index d6f69a83c7..c9ecf9acda 100644
> --- a/support/testing/tests/package/test_python.py
> +++ b/support/testing/tests/package/test_python.py
> @@ -29,7 +29,7 @@ class TestPythonBase(infra.basetest.BRTest):
>       def libc_time_test(self, timeout=-1):
>           cmd = self.interpreter + " -c '"
>           cmd += "import ctypes;"
> -        cmd += "libc = ctypes.cdll.LoadLibrary(\"libc.so.1\");"
> +        cmd += "libc = ctypes.cdll.LoadLibrary(\"libc.so.6\");"
>           cmd += "print(libc.time(None))'"
>           self.assertRunOk(cmd, timeout)
>   
> diff --git a/support/testing/tests/package/test_tmux.py b/support/testing/tests/package/test_tmux.py
> index 7a10b7a2b0..f2c620d5ca 100644
> --- a/support/testing/tests/package/test_tmux.py
> +++ b/support/testing/tests/package/test_tmux.py
> @@ -9,6 +9,7 @@ class TestTmux(infra.basetest.BRTest):
>           BR2_PACKAGE_TMUX=y
>           BR2_TARGET_ROOTFS_CPIO=y
>           # BR2_TARGET_ROOTFS_TAR is not set
> +        BR2_GENERATE_LOCALE="C.UTF-8"
>           """
>   
>       def test_run(self):
Romain Naour June 18, 2023, 9:31 p.m. UTC | #3 
Hello Arnout, Sebastian, All,

Le 18/06/2023 à 14:48, Arnout Vandecappelle via buildroot a écrit :
> 
> 
> On 30/03/2023 13:27, Sebastian Weyer wrote:
>> This patch updates the default toolchain used for runtime tests. The
>> last time this toolchain was updated was in commit
>> 0207a65323fc73fe9570f3d7e03bbfed4ab6d477 3 years ago. Since then,
>> multiple things have changed:
>>
>> Firstly, it used uclibc-ng as the libc whereas since commit
>> 4057e36ca9665edd5248512e4edba2c243b8f4be, glibc is used as the default
>> library.
> 
>  IMHO, that's not a good reason to switch to uclibc for the tests as well. I
> think it's good that we do runtime tests with a compiler that is a bit different
> from the standard that is probably already used by the upstream package.

From 4057e36ca9665edd5248512e4edba2c243b8f4be
"The vast majority of Buildroot users are quite certainly running the resulting
system on a reasonably powerful platform, with significant amount of RAM and
storage. In this context, the benefits of uClibc are no longer that much
relevant, and glibc causes less "troubles"."

I'm fully agree with the comment and we should use in the testsuite what the
vast majority of Buildroot users uses in their project. I'm not sure how many
embedded project are still using uClibc-ng nowadays when glibc or musl is available.

Note: Yocto (openembedded-core) removed uClibc support back in 2016, uClibc-ng
was never supported.
https://github.com/openembedded/openembedded-core/commit/ff1599149942af1c36280abd4f1ed3878aaa62eb

Also the glibc project is really active and evolve quite significantly between
each release. We shouln't expect everything is working with glibc especially
since we are using the latest release while most of upstream packages may have
been developped against a previous glibc version.

Here are the numbers of commits betwen uClibc-ng, glibc and musl since the
2015-02-02 (the date of the first uClibc-ng release).

uClibc-ng:
git rev-list v1.0.0.. --count
1015

glibc:
git rev-list glibc-2.21.. --count
11677

musl:
git rev-list v1.1.6.. --count
2053

Switching to glibc toolchain by default also simplify writing tests for packages
that are not abailble on uClibc-ng.

> 
> 
>> And secondly, since commit 531b2a10cda6bae78ecc141c9fe48f2024f71dbc, buildroot
>> dropped the support for gcc 8 and it cannot be built internally anymore.
> 
>  That's also not a good reason to switch to a different gcc for the runtime
> tests. But here, there's no reason to stick to GCC 8 either, so OK.
> 
> 
>> This new toolchain bleeding edge 2022.08-1 is based on gcc 12.2.0, linux
> 
>  Like Romain, I don't think it's a good idea to use bleeding edge here.
> 
>> headers 5.4.212, glibc 2.35 and binutils 2.39.
>> It is now required to add enable BR2_TOOLCHAIN_USES_GLIBC and
>> BR2_TOOLCHAIN_EXTERNAL_CUSTOM_GLIBC since uclibc-ng is still the
> 
>  Actually, regardless of the default, I think it's a good idea to explicitly
> specify this.

I already sent an updated version of this series with some changes:
http://patchwork.ozlabs.org/project/buildroot/list/?series=358853

Best regards,
Romain


> 
> 
>  Regards,
>  Arnout
> 
>> standard choice when using an external toolchain. If one were to change
>> this, it would break the builds for people using a uclibc toolchain with
>> a recent version of buildroot. BR2_TOOLCHAIN_EXTERNAL_LOCALE doesn't
>> need to be enabled anymore since it is implicit in glibc.
>> BR2_TOOLCHAIN_EXTERNAL_INET_RPC needs to be disabled since support has
>> been dropped by glibc but it is selected by default when choosing glibc.
>>
>> The previous toolchain bleeding edge 2018.11-1 is based on gcc 8.2.0,
>> linux headers 4.14.80, uclibc 1.0.30 and binutils 2.31.1
>>
>> Tested with gitlab:
>> https://gitlab.com/DocSepp/buildroot/-/pipelines/821745386/
>>
>> Due to gitlab constraints it's not possible to check all the tests. Most
>> of the tests that did not succeed didn't because of such constraints.
>> The remaining fails are missing python or perl modules, timeouts because
>> the test takes more than 3h, tests.package.test_kexec.TestKexec  is
>> executed with a property that is unknown to qemu and several tests use a
>> different toolchain.
>>
>> I noticed 4 tests, TestPython3Py, TestPython3Pyc, TestPython3PyPyc and
>> TestTmux that failed due to locale and libc.so.1 not found errors. I
>> verified that they did run with the previous toolchain and subsequently
>> introduced changes in their configs so the tests succeed with the new
>> toolchain.
>>
>> Signed-off-by: Sebastian Weyer <sebastian.weyer@smile.fr>
>> ---
>>   support/testing/infra/basetest.py            | 10 ++++++----
>>   support/testing/tests/package/test_python.py |  2 +-
>>   support/testing/tests/package/test_tmux.py   |  1 +
>>   3 files changed, 8 insertions(+), 5 deletions(-)
>>
>> diff --git a/support/testing/infra/basetest.py
>> b/support/testing/infra/basetest.py
>> index 670c7213d6..586b51e56e 100644
>> --- a/support/testing/infra/basetest.py
>> +++ b/support/testing/infra/basetest.py
>> @@ -10,13 +10,15 @@ BASIC_TOOLCHAIN_CONFIG = \
>>       BR2_arm=y
>>       BR2_TOOLCHAIN_EXTERNAL=y
>>       BR2_TOOLCHAIN_EXTERNAL_CUSTOM=y
>> +    BR2_TOOLCHAIN_USES_GLIBC=y
>> +    BR2_TOOLCHAIN_EXTERNAL_CUSTOM_GLIBC=y
>>       BR2_TOOLCHAIN_EXTERNAL_DOWNLOAD=y
>> -   
>> BR2_TOOLCHAIN_EXTERNAL_URL="https://toolchains.bootlin.com/downloads/releases/toolchains/armv5-eabi/tarballs/armv5-eabi--uclibc--bleeding-edge-2018.11-1.tar.bz2"
>> -    BR2_TOOLCHAIN_EXTERNAL_GCC_8=y
>> -    BR2_TOOLCHAIN_EXTERNAL_HEADERS_4_14=y
>> -    BR2_TOOLCHAIN_EXTERNAL_LOCALE=y
>> +   
>> BR2_TOOLCHAIN_EXTERNAL_URL="https://toolchains.bootlin.com/downloads/releases/toolchains/armv5-eabi/tarballs/armv5-eabi--glibc--bleeding-edge-2022.08-1.tar.bz2"
>> +    BR2_TOOLCHAIN_EXTERNAL_GCC_12=y
>> +    BR2_TOOLCHAIN_EXTERNAL_HEADERS_5_4=y
>>       BR2_TOOLCHAIN_HAS_THREADS_DEBUG=y
>>       BR2_TOOLCHAIN_EXTERNAL_CXX=y
>> +    # BR2_TOOLCHAIN_EXTERNAL_INET_RPC is not set
>>       """
>>     MINIMAL_CONFIG = \
>> diff --git a/support/testing/tests/package/test_python.py
>> b/support/testing/tests/package/test_python.py
>> index d6f69a83c7..c9ecf9acda 100644
>> --- a/support/testing/tests/package/test_python.py
>> +++ b/support/testing/tests/package/test_python.py
>> @@ -29,7 +29,7 @@ class TestPythonBase(infra.basetest.BRTest):
>>       def libc_time_test(self, timeout=-1):
>>           cmd = self.interpreter + " -c '"
>>           cmd += "import ctypes;"
>> -        cmd += "libc = ctypes.cdll.LoadLibrary(\"libc.so.1\");"
>> +        cmd += "libc = ctypes.cdll.LoadLibrary(\"libc.so.6\");"
>>           cmd += "print(libc.time(None))'"
>>           self.assertRunOk(cmd, timeout)
>>   diff --git a/support/testing/tests/package/test_tmux.py
>> b/support/testing/tests/package/test_tmux.py
>> index 7a10b7a2b0..f2c620d5ca 100644
>> --- a/support/testing/tests/package/test_tmux.py
>> +++ b/support/testing/tests/package/test_tmux.py
>> @@ -9,6 +9,7 @@ class TestTmux(infra.basetest.BRTest):
>>           BR2_PACKAGE_TMUX=y
>>           BR2_TARGET_ROOTFS_CPIO=y
>>           # BR2_TARGET_ROOTFS_TAR is not set
>> +        BR2_GENERATE_LOCALE="C.UTF-8"
>>           """
>>         def test_run(self):
> _______________________________________________
> buildroot mailing list
> buildroot@buildroot.org
> https://lists.buildroot.org/mailman/listinfo/buildroot
Thomas Petazzoni July 30, 2023, 9:40 p.m. UTC | #4 
On Thu, 30 Mar 2023 13:27:49 +0200
Sebastian Weyer <sebastian.weyer@smile.fr> wrote:

> This patch updates the default toolchain used for runtime tests. The
> last time this toolchain was updated was in commit
> 0207a65323fc73fe9570f3d7e03bbfed4ab6d477 3 years ago. Since then,
> multiple things have changed:
> 
> Firstly, it used uclibc-ng as the libc whereas since commit
> 4057e36ca9665edd5248512e4edba2c243b8f4be, glibc is used as the default
> library.
> 
> And secondly, since commit 531b2a10cda6bae78ecc141c9fe48f2024f71dbc, buildroot
> dropped the support for gcc 8 and it cannot be built internally anymore.
> 
> This new toolchain bleeding edge 2022.08-1 is based on gcc 12.2.0, linux
> headers 5.4.212, glibc 2.35 and binutils 2.39.
> It is now required to add enable BR2_TOOLCHAIN_USES_GLIBC and
> BR2_TOOLCHAIN_EXTERNAL_CUSTOM_GLIBC since uclibc-ng is still the
> standard choice when using an external toolchain. If one were to change
> this, it would break the builds for people using a uclibc toolchain with
> a recent version of buildroot. BR2_TOOLCHAIN_EXTERNAL_LOCALE doesn't
> need to be enabled anymore since it is implicit in glibc.
> BR2_TOOLCHAIN_EXTERNAL_INET_RPC needs to be disabled since support has
> been dropped by glibc but it is selected by default when choosing glibc.
> 
> The previous toolchain bleeding edge 2018.11-1 is based on gcc 8.2.0,
> linux headers 4.14.80, uclibc 1.0.30 and binutils 2.31.1
> 
> Tested with gitlab:
> https://gitlab.com/DocSepp/buildroot/-/pipelines/821745386/
> 
> Due to gitlab constraints it's not possible to check all the tests. Most
> of the tests that did not succeed didn't because of such constraints.
> The remaining fails are missing python or perl modules, timeouts because
> the test takes more than 3h, tests.package.test_kexec.TestKexec  is
> executed with a property that is unknown to qemu and several tests use a
> different toolchain.
> 
> I noticed 4 tests, TestPython3Py, TestPython3Pyc, TestPython3PyPyc and
> TestTmux that failed due to locale and libc.so.1 not found errors. I
> verified that they did run with the previous toolchain and subsequently
> introduced changes in their configs so the tests succeed with the new
> toolchain.
> 
> Signed-off-by: Sebastian Weyer <sebastian.weyer@smile.fr>
> ---
>  support/testing/infra/basetest.py            | 10 ++++++----
>  support/testing/tests/package/test_python.py |  2 +-
>  support/testing/tests/package/test_tmux.py   |  1 +
>  3 files changed, 8 insertions(+), 5 deletions(-)

I've marked this patch as Superseded, as you did another iteration that
used the stable Bootlin toolchain, based on the feedback.

Best regards,

Thomas
diff mbox series

Patch

diff --git a/support/testing/infra/basetest.py b/support/testing/infra/basetest.py
index 670c7213d6..586b51e56e 100644
--- a/support/testing/infra/basetest.py
+++ b/support/testing/infra/basetest.py
@@ -10,13 +10,15 @@  BASIC_TOOLCHAIN_CONFIG = \
     BR2_arm=y
     BR2_TOOLCHAIN_EXTERNAL=y
     BR2_TOOLCHAIN_EXTERNAL_CUSTOM=y
+    BR2_TOOLCHAIN_USES_GLIBC=y
+    BR2_TOOLCHAIN_EXTERNAL_CUSTOM_GLIBC=y
     BR2_TOOLCHAIN_EXTERNAL_DOWNLOAD=y
-    BR2_TOOLCHAIN_EXTERNAL_URL="https://toolchains.bootlin.com/downloads/releases/toolchains/armv5-eabi/tarballs/armv5-eabi--uclibc--bleeding-edge-2018.11-1.tar.bz2"
-    BR2_TOOLCHAIN_EXTERNAL_GCC_8=y
-    BR2_TOOLCHAIN_EXTERNAL_HEADERS_4_14=y
-    BR2_TOOLCHAIN_EXTERNAL_LOCALE=y
+    BR2_TOOLCHAIN_EXTERNAL_URL="https://toolchains.bootlin.com/downloads/releases/toolchains/armv5-eabi/tarballs/armv5-eabi--glibc--bleeding-edge-2022.08-1.tar.bz2"
+    BR2_TOOLCHAIN_EXTERNAL_GCC_12=y
+    BR2_TOOLCHAIN_EXTERNAL_HEADERS_5_4=y
     BR2_TOOLCHAIN_HAS_THREADS_DEBUG=y
     BR2_TOOLCHAIN_EXTERNAL_CXX=y
+    # BR2_TOOLCHAIN_EXTERNAL_INET_RPC is not set
     """
 
 MINIMAL_CONFIG = \
diff --git a/support/testing/tests/package/test_python.py b/support/testing/tests/package/test_python.py
index d6f69a83c7..c9ecf9acda 100644
--- a/support/testing/tests/package/test_python.py
+++ b/support/testing/tests/package/test_python.py
@@ -29,7 +29,7 @@  class TestPythonBase(infra.basetest.BRTest):
     def libc_time_test(self, timeout=-1):
         cmd = self.interpreter + " -c '"
         cmd += "import ctypes;"
-        cmd += "libc = ctypes.cdll.LoadLibrary(\"libc.so.1\");"
+        cmd += "libc = ctypes.cdll.LoadLibrary(\"libc.so.6\");"
         cmd += "print(libc.time(None))'"
         self.assertRunOk(cmd, timeout)
 
diff --git a/support/testing/tests/package/test_tmux.py b/support/testing/tests/package/test_tmux.py
index 7a10b7a2b0..f2c620d5ca 100644
--- a/support/testing/tests/package/test_tmux.py
+++ b/support/testing/tests/package/test_tmux.py
@@ -9,6 +9,7 @@  class TestTmux(infra.basetest.BRTest):
         BR2_PACKAGE_TMUX=y
         BR2_TARGET_ROOTFS_CPIO=y
         # BR2_TARGET_ROOTFS_TAR is not set
+        BR2_GENERATE_LOCALE="C.UTF-8"
         """
 
     def test_run(self):