mbox series

[0/2] Add bcachefs-tools package

Message ID 20240408013859.732937-1-matt.stav.taylor@gmail.com
Headers show
Series Add bcachefs-tools package | expand

Message

Matt Staveley-Taylor April 8, 2024, 1:38 a.m. UTC 
This patch series adds a new bcachefs-tools package: the userspace tools
for the bcachefs filesystem [1].

The tarballs are only provided with zstd compression, so we first add
support for decompressing these in the package infrastructure.

Note that this does *not* teach buildroot how to format bcachefs
filesystems, as bcachefs-tools currently lacks the ability to do this
without root privileges (see [2]). I have local patches ready for when
this feature is implemented.

This package depends on host-clang to provide libclang, which is used to
generate C <-> Rust bindings. libudev is also required, which means
systemd is needed. Unfortunately these large dependencies seem to result
in utils/test-pkg -p bcachefs-tools -a reporting all toolchains SKIPPED.

(Apologies to Thomas -- re-sending because I wasn't subscribed to the
mailing list and my mail was rejected :)

[1]: https://bcachefs.org/
[2]: https://github.com/koverstreet/bcachefs-tools/issues/164

Matt Staveley-Taylor (2):
  package: add support for extracting zstd archives
  package/bcachefs-tools: new package

 Config.in                                  |  7 ++++++
 DEVELOPERS                                 |  3 +++
 Makefile                                   |  1 +
 package/Config.in                          |  1 +
 package/bcachefs-tools/Config.in           | 26 ++++++++++++++++++++++
 package/bcachefs-tools/bcachefs-tools.hash |  2 ++
 package/bcachefs-tools/bcachefs-tools.mk   | 23 +++++++++++++++++++
 package/pkg-utils.mk                       |  2 ++
 support/dependencies/check-host-zstd.mk    |  4 ++++
 support/dependencies/check-host-zstd.sh    | 14 ++++++++++++
 10 files changed, 83 insertions(+)
 create mode 100644 package/bcachefs-tools/Config.in
 create mode 100644 package/bcachefs-tools/bcachefs-tools.hash
 create mode 100644 package/bcachefs-tools/bcachefs-tools.mk
 create mode 100644 support/dependencies/check-host-zstd.mk
 create mode 100755 support/dependencies/check-host-zstd.sh

Comments

Thomas Petazzoni April 8, 2024, 8:54 p.m. UTC | #1 
Hello Matt,

On Mon,  8 Apr 2024 02:38:43 +0100
Matt Staveley-Taylor <matt.stav.taylor@gmail.com> wrote:

> This patch series adds a new bcachefs-tools package: the userspace tools
> for the bcachefs filesystem [1].

Thanks a lot for working on this!

> This package depends on host-clang to provide libclang, which is used to
> generate C <-> Rust bindings.

But do we need to unconditionally compile the C <-> Rust bindings?

Best regards,

Thomas
Matt Staveley-Taylor April 9, 2024, 10:43 p.m. UTC | #2 
On Mon Apr 8, 2024 at 9:54 PM BST, Thomas Petazzoni wrote:
> On Mon,  8 Apr 2024 02:38:43 +0100
> Matt Staveley-Taylor <matt.stav.taylor@gmail.com> wrote:
> > This package depends on host-clang to provide libclang, which is used to
> > generate C <-> Rust bindings.
>
> But do we need to unconditionally compile the C <-> Rust bindings?

The main binary and some of the subcommands are written in Rust, but
most of them are written in C. The Rust main calls into the C
subcommands using a tool called bindgen which generates Rust glue to
interface with the C code.

So unfortunately I don't see a good way to avoid building host-clang
here.

Matt
Thomas Petazzoni April 10, 2024, 9:30 a.m. UTC | #3 
Hello Matt,

On Tue, 09 Apr 2024 23:43:29 +0100
"Matt Staveley-Taylor" <matt.stav.taylor@gmail.com> wrote:

> The main binary and some of the subcommands are written in Rust, but
> most of them are written in C. The Rust main calls into the C
> subcommands using a tool called bindgen which generates Rust glue to
> interface with the C code.
> 
> So unfortunately I don't see a good way to avoid building host-clang
> here.

Indeed, not much we can do then. Thing is that host-clang is extremely
long to build, obviously...

Best regards,

Thomas
Matt Staveley-Taylor April 11, 2024, 1:32 a.m. UTC | #4 
On Wed Apr 10, 2024 at 10:30 AM BST, Thomas Petazzoni wrote:
> Indeed, not much we can do then. Thing is that host-clang is extremely
> long to build, obviously...

I noticed that host-rust downloads a pre-built binary from somewhere. Is
there a possibility to do the same for host-clang? Or how do you want to
proceed here?

Matt
Thomas Petazzoni April 11, 2024, 12:48 p.m. UTC | #5 
Hello Matt,

On Thu, 11 Apr 2024 02:32:24 +0100
"Matt Staveley-Taylor" <matt.stav.taylor@gmail.com> wrote:

> I noticed that host-rust downloads a pre-built binary from somewhere. Is
> there a possibility to do the same for host-clang? Or how do you want to
> proceed here?

It seems that
https://github.com/llvm/llvm-project/releases/tag/llvmorg-18.1.1 has
pre-built binaries, but not for the very common x86-64 linux host
platform, which seems odd? Apparently, those binaries are made by
volunteers. Not sure how much we want to use those binaries.

Thomas
Matt Staveley-Taylor April 11, 2024, 10:51 p.m. UTC | #6 
On Thu Apr 11, 2024 at 1:48 PM BST, Thomas Petazzoni wrote:
> It seems that
> https://github.com/llvm/llvm-project/releases/tag/llvmorg-18.1.1 has
> pre-built binaries, but not for the very common x86-64 linux host
> platform, which seems odd? Apparently, those binaries are made by
> volunteers. Not sure how much we want to use those binaries.

Yeah, agreed. Perhaps there is some Buildroot infrastructure we can
leverage for building these ourselves?

Failing that -- is there actually a problem with merging the patch
as-is? I can see a couple of other packages that already depend on
host-clang.

Matt