mbox series

[00/20,v2] support/download: allow downloading unlocked cargo packages (branch yem/cargo-unchained)

Message ID cover.1717779951.git.yann.morin.1998@free.fr
Headers show
Series support/download: allow downloading unlocked cargo packages (branch yem/cargo-unchained) | expand

Message

Yann E. MORIN June 7, 2024, 5:05 p.m. UTC
Hello All!

Some maintainers refuse to commit a Cargo.lock in their repository [0],
so there are cargo packages that are unlocked.

However, we currently require that a Cargo.lock be present: we call
"cargo venodor --locked"; if there is no Cargo.lock, cargo fails.

This series adds support for downloading unlocked cargo packages; the
series is articulated in three steps:

 1. preparatory cleanups (shelcheck, readability...)

 2. extend the download infra to pass arbitrary options to the
    post-process helpers; use that to pass the path of the manifest
    instead of using an environment variable

 3. add support for unlocked cargo packages by passing a new option to
    the cargo post-process helper; add documentation and check-package.

 4. (off-by-one error) add an RFC-style package to demonstrate the
    unlocked cargo package.

Unlocked packages can be handled in two ways: totally unlocked, in which
case they are not reproducible, or by providing a Cargo.lock in the
Buildroot package directory, in which case it is reproducible, but there
is a little bit more complexity to it, so this is implemented separately
in case it is deemed too heavy-weight.

Regards,
Yann E. MORIN.


----------------------------------------------------------------
Yann E. MORIN (20):
      package/pkg-download: symplify calling parameters
      package/pkg-generic: split long, unreadable DOWNLOAD line
      utils/check-package: add option to run the test-suite
      utils/checkpackagelib: instruct shellcheck to follow included files
      support/download/dl-wrapper: fix shellcheck errors
      support/download/cargo-post-process: fix shellcheck
      support/download: move tracing functions to helpers
      support/download: pass post-process options to post-process helper
      support/download: accept manifest path as a post-process option
      pakcage/pkg-cargo: use post-process opts to pass manifest path
      package/pkg-python: use post-process opts to pass Cargo manifest path
      package/python-bcrypt: use post-process opts to pass Cargo manifest path
      support/download: no longer support BR_CARGO_MANIFEST_PATH in cargo post-process
      docs/manual: rephrase end of vendoring section
      package/pkg-cargo: add support for unlocked packages
      utils/checkpackagelib: add test for default _CARGO_LOCKED
      RFC: package/anvil: new package
      package/pkg-utils: accept pacakge-specific archive format version
      package/pkg-cargo: allow unlocked packages to provide local Cargo.lock
      RFC: package/anvil: add local Cargo.lock

 .checkpackageignore                    |    2 -
 docs/manual/adding-packages-cargo.adoc |   34 +-
 package/Config.in                      |    1 +
 package/anvil/Cargo.lock               | 3464 ++++++++++++++++++++++++++++++++
 package/anvil/Config.in                |   20 +
 package/anvil/anvil.hash               |    2 +
 package/anvil/anvil.mk                 |   31 +
 package/pkg-cargo.mk                   |   31 +-
 package/pkg-download.mk                |   33 +-
 package/pkg-generic.mk                 |   12 +-
 package/pkg-python.mk                  |    6 +-
 package/pkg-utils.mk                   |    3 +-
 package/python-bcrypt/python-bcrypt.mk |    5 +-
 support/download/cargo-post-process    |   62 +-
 support/download/dl-wrapper            |   32 +-
 support/download/helpers               |    4 +
 utils/check-package                    |   10 +
 utils/checkpackagelib/lib_mk.py        |    8 +
 utils/checkpackagelib/readme.txt       |    1 +
 utils/checkpackagelib/test_lib_mk.py   |   18 +
 utils/checkpackagelib/tool.py          |    2 +-
 21 files changed, 3728 insertions(+), 53 deletions(-)
 create mode 100644 package/anvil/Cargo.lock
 create mode 100644 package/anvil/Config.in
 create mode 100644 package/anvil/anvil.hash
 create mode 100644 package/anvil/anvil.mk

--
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

Comments

Arnout Vandecappelle July 14, 2024, 8:59 a.m. UTC | #1
Hi Yann, all,

On 07/06/2024 19:05, Yann E. MORIN wrote:
> Hello All!
> 
> Some maintainers refuse to commit a Cargo.lock in their repository [0],
> so there are cargo packages that are unlocked.
> 
> However, we currently require that a Cargo.lock be present: we call
> "cargo venodor --locked"; if there is no Cargo.lock, cargo fails.
> 
> This series adds support for downloading unlocked cargo packages; the
> series is articulated in three steps:
> 
>   1. preparatory cleanups (shelcheck, readability...)
> 
>   2. extend the download infra to pass arbitrary options to the
>      post-process helpers; use that to pass the path of the manifest
>      instead of using an environment variable
> 
>   3. add support for unlocked cargo packages by passing a new option to
>      the cargo post-process helper; add documentation and check-package.

  I don't see why we want this in the first place... Why not simply _require_ a 
Cargo.lock file to be present, and just add the option of including our own?

> 
>   4. (off-by-one error) add an RFC-style package to demonstrate the
>      unlocked cargo package.
> 
> Unlocked packages can be handled in two ways: totally unlocked, in which
> case they are not reproducible, or by providing a Cargo.lock in the
> Buildroot package directory, in which case it is reproducible, but there
> is a little bit more complexity to it, so this is implemented separately
> in case it is deemed too heavy-weight.
> 
> Regards,
> Yann E. MORIN.
> 
> 
> ----------------------------------------------------------------
> Yann E. MORIN (20):
>        package/pkg-download: symplify calling parameters
>        package/pkg-generic: split long, unreadable DOWNLOAD line
>        utils/check-package: add option to run the test-suite
>        utils/checkpackagelib: instruct shellcheck to follow included files
>        support/download/dl-wrapper: fix shellcheck errors
>        support/download/cargo-post-process: fix shellcheck
>        support/download: move tracing functions to helpers
>        support/download: pass post-process options to post-process helper
>        support/download: accept manifest path as a post-process option
>        pakcage/pkg-cargo: use post-process opts to pass manifest path
>        package/pkg-python: use post-process opts to pass Cargo manifest path
>        package/python-bcrypt: use post-process opts to pass Cargo manifest path
>        support/download: no longer support BR_CARGO_MANIFEST_PATH in cargo post-process
>        docs/manual: rephrase end of vendoring section

  Applied these to master, except for python-bcrypt as explained separately.

>        package/pkg-cargo: add support for unlocked packages
>        utils/checkpackagelib: add test for default _CARGO_LOCKED
>        RFC: package/anvil: new package
>        package/pkg-utils: accept pacakge-specific archive format version
>        package/pkg-cargo: allow unlocked packages to provide local Cargo.lock
>        RFC: package/anvil: add local Cargo.lock

  All these are Changes Requested. I'll comment more in some individual patches.

  Regards,
  Arnout

> 
>   .checkpackageignore                    |    2 -
>   docs/manual/adding-packages-cargo.adoc |   34 +-
>   package/Config.in                      |    1 +
>   package/anvil/Cargo.lock               | 3464 ++++++++++++++++++++++++++++++++
>   package/anvil/Config.in                |   20 +
>   package/anvil/anvil.hash               |    2 +
>   package/anvil/anvil.mk                 |   31 +
>   package/pkg-cargo.mk                   |   31 +-
>   package/pkg-download.mk                |   33 +-
>   package/pkg-generic.mk                 |   12 +-
>   package/pkg-python.mk                  |    6 +-
>   package/pkg-utils.mk                   |    3 +-
>   package/python-bcrypt/python-bcrypt.mk |    5 +-
>   support/download/cargo-post-process    |   62 +-
>   support/download/dl-wrapper            |   32 +-
>   support/download/helpers               |    4 +
>   utils/check-package                    |   10 +
>   utils/checkpackagelib/lib_mk.py        |    8 +
>   utils/checkpackagelib/readme.txt       |    1 +
>   utils/checkpackagelib/test_lib_mk.py   |   18 +
>   utils/checkpackagelib/tool.py          |    2 +-
>   21 files changed, 3728 insertions(+), 53 deletions(-)
>   create mode 100644 package/anvil/Cargo.lock
>   create mode 100644 package/anvil/Config.in
>   create mode 100644 package/anvil/anvil.hash
>   create mode 100644 package/anvil/anvil.mk
> 
> --
> .-----------------.--------------------.------------------.--------------------.
> |  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
> | +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
> | +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
> | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
> '------------------------------^-------^------------------^--------------------'
> _______________________________________________
> buildroot mailing list
> buildroot@buildroot.org
> https://lists.buildroot.org/mailman/listinfo/buildroot