[1/1] support/testing: test_rust.py: Add test to check correct vendoring

Currently the tests TestRust and TestRustBin do check for vendoring by
requiring package ripgrep to be built but only if the download
directory is already empty, otherwise the existing contents of the download
directory will be reused and therefore not be redownloaded.

This new test will only verify that the required packages are downloaded
and vendored correctly without doing a runtime test. It does so by setting a
path to a folder "dl" inside the build directory (output-directory/testname/)
and then setting the environment variable BR2_DL_DIR to this path before the
build starts. This code was essentially copied from the file
test_gitforge.py which was added in commit
1ca6ab6ace3c1f9edfbfd16b57d1280b3ea2c6f5

We want the package ripgrep to be built since it requires vendoring
directly. Additionally we want the package python-cryptography to be
built because it has rust dependencies and therefore indirectly also requires
vendoring.

Signed-off-by: Sebastian Weyer <sebastian.weyer@smile.fr>
---
 support/testing/tests/package/test_rust.py | 42 ++++++++++++++++++++++
 1 file changed, 42 insertions(+)

Sebastian, All,

On 2023-02-07 17:50 +0100, Sebastian Weyer spake thusly:
> Currently the tests TestRust and TestRustBin do check for vendoring by
> requiring package ripgrep to be built but only if the download
> directory is already empty, otherwise the existing contents of the download
> directory will be reused and therefore not be redownloaded.
> 
> This new test will only verify that the required packages are downloaded
> and vendored correctly without doing a runtime test. It does so by setting a
> path to a folder "dl" inside the build directory (output-directory/testname/)
> and then setting the environment variable BR2_DL_DIR to this path before the
> build starts. This code was essentially copied from the file
> test_gitforge.py which was added in commit
> 1ca6ab6ace3c1f9edfbfd16b57d1280b3ea2c6f5

There is a missing piece in this otherwise detailed commit log: why set
BR2_DL_DIR in the environment, rather than set it in the config options?

The reason is that a user may already have BR2_DL_DIR set in their own
envronment, and that would override the dl dir we set in the config
options.

No need to respin, that can be fixed when applying.

Regards,
Yann E. MORIN.

> We want the package ripgrep to be built since it requires vendoring
> directly. Additionally we want the package python-cryptography to be
> built because it has rust dependencies and therefore indirectly also requires
> vendoring.
> 
> Signed-off-by: Sebastian Weyer <sebastian.weyer@smile.fr>
> ---
>  support/testing/tests/package/test_rust.py | 42 ++++++++++++++++++++++
>  1 file changed, 42 insertions(+)
> 
> diff --git a/support/testing/tests/package/test_rust.py b/support/testing/tests/package/test_rust.py
> index fb9dd2d916..763125ca20 100644
> --- a/support/testing/tests/package/test_rust.py
> +++ b/support/testing/tests/package/test_rust.py
> @@ -1,4 +1,5 @@
>  import os
> +import shutil
>  
>  import infra.basetest
>  
> @@ -54,3 +55,44 @@ class TestRust(TestRustBase):
>      def test_run(self):
>          self.login()
>          self.assertRunOk("rg Buildroot /etc/issue")
> +
> +
> +class TestRustVendoring(infra.basetest.BRConfigTest):
> +    config = \
> +        """
> +        BR2_arm=y
> +        BR2_cortex_a9=y
> +        BR2_ARM_ENABLE_NEON=y
> +        BR2_ARM_ENABLE_VFP=y
> +        BR2_TOOLCHAIN_EXTERNAL=y
> +        # BR2_TARGET_ROOTFS_TAR is not set
> +        BR2_PACKAGE_HOST_RUSTC=y
> +        BR2_PACKAGE_RIPGREP=y
> +        BR2_PACKAGE_PYTHON3=y
> +        BR2_PACKAGE_PYTHON_CRYPTOGRAPHY=y
> +        BR2_BACKUP_SITE=""
> +        """
> +
> +    def setUp(self):
> +        super(TestRustVendoring, self).setUp()
> +
> +    def tearDown(self):
> +        self.show_msg("Cleaning up")
> +        if self.b and not self.keepbuilds:
> +            self.b.delete()
> +
> +    def check_download(self, package):
> +        # store downloaded tarball inside the output dir so the test infra
> +        # cleans it up at the end
> +        dl_dir = os.path.join(self.builddir, "dl")
> +        # enforce we test the download
> +        if os.path.exists(dl_dir):
> +            shutil.rmtree(dl_dir)
> +        env = {"BR2_DL_DIR": dl_dir}
> +        self.b.build(["{}-dirclean".format(package),
> +                      "{}-legal-info".format(package)],
> +                     env)
> +
> +    def test_run(self):
> +        self.check_download("ripgrep")
> +        self.check_download("python-cryptography")
> -- 
> 2.25.1
> 
> _______________________________________________
> buildroot mailing list
> buildroot@buildroot.org
> https://lists.buildroot.org/mailman/listinfo/buildroot

Hello Yann, hello all,

I just wanted to ping this again since the patch seemed to be fine and it
hasn't been applied in 2 months.
Thank you :)

Regards,
Sebastian

On Tue, Feb 7, 2023 at 6:03 PM Yann E. MORIN <yann.morin.1998@free.fr>
wrote:

> Sebastian, All,
>
> On 2023-02-07 17:50 +0100, Sebastian Weyer spake thusly:
> > Currently the tests TestRust and TestRustBin do check for vendoring by
> > requiring package ripgrep to be built but only if the download
> > directory is already empty, otherwise the existing contents of the
> download
> > directory will be reused and therefore not be redownloaded.
> >
> > This new test will only verify that the required packages are downloaded
> > and vendored correctly without doing a runtime test. It does so by
> setting a
> > path to a folder "dl" inside the build directory
> (output-directory/testname/)
> > and then setting the environment variable BR2_DL_DIR to this path before
> the
> > build starts. This code was essentially copied from the file
> > test_gitforge.py which was added in commit
> > 1ca6ab6ace3c1f9edfbfd16b57d1280b3ea2c6f5
>
> There is a missing piece in this otherwise detailed commit log: why set
> BR2_DL_DIR in the environment, rather than set it in the config options?
>
> The reason is that a user may already have BR2_DL_DIR set in their own
> envronment, and that would override the dl dir we set in the config
> options.
>
> No need to respin, that can be fixed when applying.
>
> Regards,
> Yann E. MORIN.
>
> > We want the package ripgrep to be built since it requires vendoring
> > directly. Additionally we want the package python-cryptography to be
> > built because it has rust dependencies and therefore indirectly also
> requires
> > vendoring.
> >
> > Signed-off-by: Sebastian Weyer <sebastian.weyer@smile.fr>
> > ---
> >  support/testing/tests/package/test_rust.py | 42 ++++++++++++++++++++++
> >  1 file changed, 42 insertions(+)
> >
> > diff --git a/support/testing/tests/package/test_rust.py
> b/support/testing/tests/package/test_rust.py
> > index fb9dd2d916..763125ca20 100644
> > --- a/support/testing/tests/package/test_rust.py
> > +++ b/support/testing/tests/package/test_rust.py
> > @@ -1,4 +1,5 @@
> >  import os
> > +import shutil
> >
> >  import infra.basetest
> >
> > @@ -54,3 +55,44 @@ class TestRust(TestRustBase):
> >      def test_run(self):
> >          self.login()
> >          self.assertRunOk("rg Buildroot /etc/issue")
> > +
> > +
> > +class TestRustVendoring(infra.basetest.BRConfigTest):
> > +    config = \
> > +        """
> > +        BR2_arm=y
> > +        BR2_cortex_a9=y
> > +        BR2_ARM_ENABLE_NEON=y
> > +        BR2_ARM_ENABLE_VFP=y
> > +        BR2_TOOLCHAIN_EXTERNAL=y
> > +        # BR2_TARGET_ROOTFS_TAR is not set
> > +        BR2_PACKAGE_HOST_RUSTC=y
> > +        BR2_PACKAGE_RIPGREP=y
> > +        BR2_PACKAGE_PYTHON3=y
> > +        BR2_PACKAGE_PYTHON_CRYPTOGRAPHY=y
> > +        BR2_BACKUP_SITE=""
> > +        """
> > +
> > +    def setUp(self):
> > +        super(TestRustVendoring, self).setUp()
> > +
> > +    def tearDown(self):
> > +        self.show_msg("Cleaning up")
> > +        if self.b and not self.keepbuilds:
> > +            self.b.delete()
> > +
> > +    def check_download(self, package):
> > +        # store downloaded tarball inside the output dir so the test
> infra
> > +        # cleans it up at the end
> > +        dl_dir = os.path.join(self.builddir, "dl")
> > +        # enforce we test the download
> > +        if os.path.exists(dl_dir):
> > +            shutil.rmtree(dl_dir)
> > +        env = {"BR2_DL_DIR": dl_dir}
> > +        self.b.build(["{}-dirclean".format(package),
> > +                      "{}-legal-info".format(package)],
> > +                     env)
> > +
> > +    def test_run(self):
> > +        self.check_download("ripgrep")
> > +        self.check_download("python-cryptography")
> > --
> > 2.25.1
> >
> > _______________________________________________
> > buildroot mailing list
> > buildroot@buildroot.org
> > https://lists.buildroot.org/mailman/listinfo/buildroot
>
> --
>
> .-----------------.--------------------.------------------.--------------------.
> |  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics'
> conspiracy: |
> | +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___
>      |
> | +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is
> no  |
> | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v
>  conspiracy.  |
>
> '------------------------------^-------^------------------^--------------------'
>