mbox series

[v9,0/8] Add support for the Rust programming

Message ID 20171228155146.18193-1-eric.le.bihan.dev@free.fr
Headers show
Series Add support for the Rust programming | expand

Message

Eric Le Bihan Dec. 28, 2017, 3:51 p.m. UTC
This series adds support for the Rust programming language by adding the
following packages:

 - rustc: a virtual package for the Rust compiler.
 - rust-bin: provides a pre-built version of rustc.
 - cargo-bin: provides a pre-built version of Rust package manager.
 - rust: builds rustc from source.
 - cargo: builds Rust package manager from source.

Only the host variants are provided.

The rustc virtual package is inspired by the mysql one.

v8 -> v9:

 - replace giant patch by post-extract hook
 - drop rust-cargo virtual package
 - explicitly enable SSL support in host-libcurl

v7 -> v8:

 - replace cargo-bootstrap by cargo-bin
 - drop rust-bootstrap and use rust-bin instead
 - rust-bin and rust versions are the same
 - cargo-bin and cargo versions are the same
 - bump cargo to 0.23.0
 - bump rust to 1.22.1
 - add rust-cargo virtual package

v6 -> v7:

 - add cargo 0.20.0
 - bump rust to 1.19.0
 - bump rust-bootstrap to 1.18.0
 - bump cargo-bootstrap to 0.19.0
 - bump rust-bin to 1.19.0
 - fix mips64 target name in rustc

v5 -> v6:

 - bump rust to 1.18.0
 - bump rust-bootstrap to 1.17.0
 - bump rust-bin to 1.18.0

v4 -> v5:

 - add rustc virtual package
 - add rust-bin, provider for rustc
 - rework rust to be a provider for rustc
 - rework some commit messages

v3 -> v4:

 - bump rust to 1.16.0
 - bump rust-bootstrap to 1.15.1
 - add cargo-bootstrap
 - use built-in target specifications
 - drop external Python script to generate target specifications
 - enable support for PowerPC
 - expose host variant in menuconfig

v2 -> v3:

 - bump rust to 1.10.0
 - rework and bump rust-bootstrap to 1.9.0
 - host-rust requires at least GCC 4.7 because of LLVM (suggested by R. Naour)
 - rust requires a glibc-based cross-compiler
 - rust requires GCC 5.x or above for Aarch64

v1 -> v2:

 - bump rust to version 1.9.0.
 - drop patch for bzip2 support in host-python.
 - add package for jemalloc.
 - add dependency on host being a x86 machine.
 - add dependency on toolchain.
 - use dedicated package to provide bootstrapping binary: rust-bootstrap.
 - let ./configure find out host/build on its own.
 - remove entry from configuration menu.

Eric Le Bihan (8):
  rustc: new virtual package
  rust-bin: new package
  cargo-bin: new package
  rust: new package
  libssh2: add host variant
  libhttpparser: add host variant
  libcurl: add host variant
  cargo: new package

 DEVELOPERS                             |  4 ++
 package/Config.in.host                 |  2 +
 package/cargo-bin/cargo-bin.hash       |  4 ++
 package/cargo-bin/cargo-bin.mk         | 14 +++++
 package/cargo/Config.in.host           |  8 +++
 package/cargo/cargo.hash               |  4 ++
 package/cargo/cargo.mk                 | 96 ++++++++++++++++++++++++++++++++++
 package/cargo/config.in                |  2 +
 package/libcurl/libcurl.mk             | 14 +++++
 package/libhttpparser/libhttpparser.mk |  9 ++++
 package/libssh2/libssh2.mk             |  6 +++
 package/rust-bin/rust-bin.hash         | 30 +++++++++++
 package/rust-bin/rust-bin.mk           | 63 ++++++++++++++++++++++
 package/rust/rust.hash                 |  2 +
 package/rust/rust.mk                   | 82 +++++++++++++++++++++++++++++
 package/rustc/Config.in.host           | 62 ++++++++++++++++++++++
 package/rustc/rustc.mk                 | 31 +++++++++++
 17 files changed, 433 insertions(+)
 create mode 100644 package/cargo-bin/cargo-bin.hash
 create mode 100644 package/cargo-bin/cargo-bin.mk
 create mode 100644 package/cargo/Config.in.host
 create mode 100644 package/cargo/cargo.hash
 create mode 100644 package/cargo/cargo.mk
 create mode 100644 package/cargo/config.in
 create mode 100644 package/rust-bin/rust-bin.hash
 create mode 100644 package/rust-bin/rust-bin.mk
 create mode 100644 package/rust/rust.hash
 create mode 100644 package/rust/rust.mk
 create mode 100644 package/rustc/Config.in.host
 create mode 100644 package/rustc/rustc.mk

--
2.14.3

Comments

Thomas Petazzoni Dec. 28, 2017, 4:21 p.m. UTC | #1
Hello,

On Thu, 28 Dec 2017 16:51:38 +0100, Eric Le Bihan wrote:
> This series adds support for the Rust programming language by adding the
> following packages:
> 
>  - rustc: a virtual package for the Rust compiler.
>  - rust-bin: provides a pre-built version of rustc.
>  - cargo-bin: provides a pre-built version of Rust package manager.
>  - rust: builds rustc from source.
>  - cargo: builds Rust package manager from source.

One question: with your series, one can therefore chose whether he
wants to build its own Rust compiler and standard library ("rust"
package) or use the pre-built Rust compiler and standard library
("rustc-bin" package) ?

In other words, does this means that there are two options:

 - User chooses rustc-bin, every Rust package is built using the
   pre-built compiler, and the pre-built standard library is used.

 - User chooses rust. In this case, rust is built using rustc-bin, but
   every other Rust package will be built with the Rust compiler we
   have built from source, and will be using the Rust standard library
   we have built from source.

Is this correct ?

(I'm just trying to understand the overall series.)

Thanks!

Thomas
Eric Le Bihan Dec. 28, 2017, 5:52 p.m. UTC | #2
Hi!

On 17-12-28 17:21:29, Thomas Petazzoni wrote:
> Hello,
>
> On Thu, 28 Dec 2017 16:51:38 +0100, Eric Le Bihan wrote:
> > This series adds support for the Rust programming language by adding the
> > following packages:
> >
> >  - rustc: a virtual package for the Rust compiler.
> >  - rust-bin: provides a pre-built version of rustc.
> >  - cargo-bin: provides a pre-built version of Rust package manager.
> >  - rust: builds rustc from source.
> >  - cargo: builds Rust package manager from source.
>
> One question: with your series, one can therefore chose whether he
> wants to build its own Rust compiler and standard library ("rust"
> package) or use the pre-built Rust compiler and standard library
> ("rustc-bin" package) ?
>
> In other words, does this means that there are two options:
>
>  - User chooses rustc-bin, every Rust package is built using the
>    pre-built compiler, and the pre-built standard library is used.
>
>  - User chooses rust. In this case, rust is built using rustc-bin, but
>    every other Rust package will be built with the Rust compiler we
>    have built from source, and will be using the Rust standard library
>    we have built from source.
>
> Is this correct ?
>
> (I'm just trying to understand the overall series.)

Yes, this is correct. The introduction of a virtual package for the Rust
compiler was suggested in the mail thread about version v4 of this
series [1], as building the compiler from source takes time and using the
available pre-built version would speed things up (as done for GCC).

[1] http://lists.busybox.net/pipermail/buildroot/2017-April/189451.html

Regards,

--
ELB
Thomas Petazzoni Dec. 28, 2017, 9:36 p.m. UTC | #3
Hello,

On Thu, 28 Dec 2017 18:52:56 +0100, Eric Le Bihan wrote:

> Yes, this is correct. The introduction of a virtual package for the Rust
> compiler was suggested in the mail thread about version v4 of this
> series [1], as building the compiler from source takes time and using the
> available pre-built version would speed things up (as done for GCC).

OK, thanks for the additional explanation, makes sense.

I reviewed the different patches in your series, and I think it's
really close to a state where it can be merged. If you can take care of
the few comments/questions in the near future, it would definitely
help, as I would still have the context in mind.

Two other things that would be nice to do is:

 - Add two test cases in support/testing/. One that builds a system with
   rust-bin, builds a hello world program in Rust, and run it on the
   target. And another that does the same, but by building the rust
   compiler from source. Perhaps additional test cases can be added for
   Cargo, but I'm not sure how it's supposed to be used.

 - Add a short section in the documentation that explains how the Rust
   support in Buildroot is working. This would help people trying to do
   stuff with Rust to understand how things work.

Thanks!

Thomas
Eric Le Bihan Jan. 2, 2018, 5:51 p.m. UTC | #4
Hi!

On 17-12-28 22:36:29, Thomas Petazzoni wrote:
> I reviewed the different patches in your series, and I think it's
> really close to a state where it can be merged. If you can take care of
> the few comments/questions in the near future, it would definitely
> help, as I would still have the context in mind.
>
> Two other things that would be nice to do is:
>
>  - Add two test cases in support/testing/. One that builds a system with
>    rust-bin, builds a hello world program in Rust, and run it on the
>    target. And another that does the same, but by building the rust
>    compiler from source. Perhaps additional test cases can be added for
>    Cargo, but I'm not sure how it's supposed to be used.
>
>  - Add a short section in the documentation that explains how the Rust
>    support in Buildroot is working. This would help people trying to do
>    stuff with Rust to understand how things work.

I'll add these. Thanks for the review!

Regards,

--
ELB
Eric Le Bihan Jan. 18, 2018, 7:48 a.m. UTC | #5
Hi!

On 2017-12-28 22:36, Thomas Petazzoni wrote:
>
> Two other things that would be nice to do is:
>
>  - Add two test cases in support/testing/. One that builds a system with
>    rust-bin, builds a hello world program in Rust, and run it on the
>    target. And another that does the same, but by building the rust
>    compiler from source. Perhaps additional test cases can be added for
>    Cargo, but I'm not sure how it's supposed to be used.

Is there a glibc-based toolchain suitable for arm vexpress available on
http://autobuild.buildroot.org/toolchains/tarballs/?

I see br-aarch64-glibc-2017.05-1078-g95b1dae.tar.bz2, but there is no
support for Aarch64 in testing and no artefacts are available.

Regards,

--
ELB
Thomas Petazzoni Jan. 18, 2018, 8:03 a.m. UTC | #6
Hello Eric,

On Thu, 18 Jan 2018 08:48:50 +0100, Eric Le Bihan wrote:

> On 2017-12-28 22:36, Thomas Petazzoni wrote:
> >
> > Two other things that would be nice to do is:
> >
> >  - Add two test cases in support/testing/. One that builds a system with
> >    rust-bin, builds a hello world program in Rust, and run it on the
> >    target. And another that does the same, but by building the rust
> >    compiler from source. Perhaps additional test cases can be added for
> >    Cargo, but I'm not sure how it's supposed to be used.  
> 
> Is there a glibc-based toolchain suitable for arm vexpress available on
> http://autobuild.buildroot.org/toolchains/tarballs/?

Use the Linaro ARM toolchain ?

> I see br-aarch64-glibc-2017.05-1078-g95b1dae.tar.bz2, but there is no
> support for Aarch64 in testing

I'm not sure what you mean by "no support for AArch64 in testing". The
testing infrastructure can test any arbitrary configuration, so there
is nothing like "support for architecture <foo> in testing". Perhaps
you are confused by BASIC_TOOLCHAIN_CONFIG being an ARM toolchain
configuration. But this BASIC_TOOLCHAIN_CONFIG is not mandatory, it is
just one toolchain configuration that you can use if your test doesn't
really care about the architecture/toolchain.

We have tests like support/testing/tests/fs/test_iso9660.py that don't
use BASIC_TOOLCHAIN_CONFIG.

> and no artefacts are available.

I'm not sure what "artefacts" you are looking for here.

If you need an ARM or AArcH64 glibc toolchain, just use the Linaro
external toolchains I'd say.

Thanks!

Thomas
Eric Le Bihan Jan. 18, 2018, 10:41 p.m. UTC | #7
Hi!

On 2018-01-18 09:03, Thomas Petazzoni wrote:
> Hello Eric,
>
> On Thu, 18 Jan 2018 08:48:50 +0100, Eric Le Bihan wrote:
>
> > I see br-aarch64-glibc-2017.05-1078-g95b1dae.tar.bz2, but there is no
> > support for Aarch64 in testing
>
> I'm not sure what you mean by "no support for AArch64 in testing". The
> testing infrastructure can test any arbitrary configuration, so there
> is nothing like "support for architecture <foo> in testing". Perhaps
> you are confused by BASIC_TOOLCHAIN_CONFIG being an ARM toolchain
> configuration. But this BASIC_TOOLCHAIN_CONFIG is not mandatory, it is
> just one toolchain configuration that you can use if your test doesn't
> really care about the architecture/toolchain.
>
> We have tests like support/testing/tests/fs/test_iso9660.py that don't
> use BASIC_TOOLCHAIN_CONFIG.
>

Looking at the existing tests, I thought it was a requirement to use
toolchains hosted at http://autobuild.buildroot.org/.

> > and no artefacts are available.
>
> I'm not sure what "artefacts" you are looking for here.

I was referring to ARTIFACTS_URL, where ARM versatile/vexpress
kernels are available. As I thought I should have used
br-aarch64-glibc-2017.05-1078-g95b1dae.tar.bz2 to get a glibc-based
toolchain, adding Aarch64 support to the emulator seemed necessary.

I'll go with the Linaro toolchain, using ARM vexpress as target.

Thanks for the clarification.

Regards,

--
ELB
Thomas Petazzoni Jan. 19, 2018, 8:03 a.m. UTC | #8
Hello,

On Thu, 18 Jan 2018 23:41:58 +0100, Eric Le Bihan wrote:

> > We have tests like support/testing/tests/fs/test_iso9660.py that don't
> > use BASIC_TOOLCHAIN_CONFIG.
> 
> Looking at the existing tests, I thought it was a requirement to use
> toolchains hosted at http://autobuild.buildroot.org/.

Not at all. We do have some tests that don't use those toolchains. Look
at support/testing/tests/boot/test_atf.py, the tests use Linaro
toolchains.

> > I'm not sure what "artefacts" you are looking for here.  
> 
> I was referring to ARTIFACTS_URL, where ARM versatile/vexpress
> kernels are available. As I thought I should have used
> br-aarch64-glibc-2017.05-1078-g95b1dae.tar.bz2 to get a glibc-based
> toolchain, adding Aarch64 support to the emulator seemed necessary.
> 
> I'll go with the Linaro toolchain, using ARM vexpress as target.

If you want to do runtime test on an AArch64 platform without building
the kernel as part of your test configuration, then yes, we would have
to add an artifact on http://autobuild.buildroot.net/artefacts/. I can
definitely do that if needed.

I'm not super happy with this artefacts solution, but I wanted to avoid
putting all those binary files under version control inside the
Buildroot repository. If people have better proposals, I'm definitely
interested.

Best regards,

Thomas
Arnout Vandecappelle Jan. 22, 2018, 9:57 p.m. UTC | #9
On 19-01-18 09:03, Thomas Petazzoni wrote:
> Hello,
> 
> On Thu, 18 Jan 2018 23:41:58 +0100, Eric Le Bihan wrote:
> 
>>> We have tests like support/testing/tests/fs/test_iso9660.py that don't
>>> use BASIC_TOOLCHAIN_CONFIG.
>>
>> Looking at the existing tests, I thought it was a requirement to use
>> toolchains hosted at http://autobuild.buildroot.org/.
> 
> Not at all. We do have some tests that don't use those toolchains. Look
> at support/testing/tests/boot/test_atf.py, the tests use Linaro
> toolchains.
> 
>>> I'm not sure what "artefacts" you are looking for here.  
>>
>> I was referring to ARTIFACTS_URL, where ARM versatile/vexpress
>> kernels are available. As I thought I should have used
>> br-aarch64-glibc-2017.05-1078-g95b1dae.tar.bz2 to get a glibc-based
>> toolchain, adding Aarch64 support to the emulator seemed necessary.
>>
>> I'll go with the Linaro toolchain, using ARM vexpress as target.
> 
> If you want to do runtime test on an AArch64 platform without building
> the kernel as part of your test configuration, then yes, we would have
> to add an artifact on http://autobuild.buildroot.net/artefacts/. I can
> definitely do that if needed.
> 
> I'm not super happy with this artefacts solution, but I wanted to avoid
> putting all those binary files under version control inside the
> Buildroot repository. If people have better proposals, I'm definitely
> interested.

 We could have the artefacts produced by gitlab-CI, and use the gitlab pipeline
feature to make sure the most recently built one is used. Well, that's the
theory, I haven't looked at the details :-)

 Regards,
 Arnout