| Message ID | 20240207020239.120806-2-dimitri.ledkov@canonical.com |
|---|---|
| State | New |
| Headers | show |
| Series | Switch to 3.0 dpkg source format with verified orig tarball | expand |
Dimitri John Ledkov wrote: > This patch switches packaging to 3.0 dpkg source formats. Using native > (no orig-tarball) for RC releases, and quilt (with orig tarball) for > released kernels. > > This is a minimal conversion to the new formats. The immediate benefit > is ability to use xz compression for the source tarballs, GPG verified > upstream tarballs, and simplify kernel patching for those that are > rebuilding Ubuntu kernels. > > Once this is applied we will then be able to further simplify our > packaging by collapsing debian.* directories under debian dir, and > significantly reducing - if not eliminating our reconstruct scripts. > > Signed-off-by: Dimitri John Ledkov <dimitri.ledkov@canonical.com> > --- > debian/rules.d/1-maintainer.mk | 2 ++ > debian/source/format | 2 +- > debian/source/local-options | 12 ++++++++++++ > 3 files changed, 15 insertions(+), 1 deletion(-) > create mode 100644 debian/source/local-options > > diff --git a/debian/rules.d/1-maintainer.mk b/debian/rules.d/1-maintainer.mk > index dfcd814e2d..51eae55f78 100644 > --- a/debian/rules.d/1-maintainer.mk > +++ b/debian/rules.d/1-maintainer.mk > @@ -114,8 +114,10 @@ autoreconstruct: > # No need for reconstruct for -rc kernels since we don't upload an > # orig tarball, so just remove it. > if grep -q "^EXTRAVERSION = -rc[0-9]\+$$" Makefile; then \ > + echo "3.0 (native)" > debian/source/format; \ > echo "exit 0" >$(DEBIAN)/reconstruct; \ > else \ > + echo "3.0 (quilt)" > debian/source/format; \ > $(DROOT)/scripts/misc/gen-auto-reconstruct $(upstream_tag) $(DEBIAN)/reconstruct $(DROOT)/source/options; \ > fi > > diff --git a/debian/source/format b/debian/source/format > index d3827e75a5..89ae9db8f8 100644 > --- a/debian/source/format > +++ b/debian/source/format > @@ -1 +1 @@ > -1.0 > +3.0 (native) > diff --git a/debian/source/local-options b/debian/source/local-options > new file mode 100644 > index 0000000000..d77c6f2631 > --- /dev/null > +++ b/debian/source/local-options > @@ -0,0 +1,12 @@ > +# Do not ingore removed files and include them in the automatically *ignore > +# generated patch > +include-removal > +# Do not try to prepare the build tree by applying patches which are > +# apparently unapplied > +no-preparation > +# Use debian-changes patch name without version number > +single-debian-patch > +# Do not try to unapply patches > +no-unapply-patches > +# Automatically add to quilt series the autogenerated patch > +auto-commit > -- > 2.34.1 > > > -- > kernel-team mailing list > kernel-team@lists.ubuntu.com > https://lists.ubuntu.com/mailman/listinfo/kernel-team
On Wed, Feb 7, 2024 at 11:02 AM Dimitri John Ledkov <dimitri.ledkov@canonical.com> wrote: > > This patch switches packaging to 3.0 dpkg source formats. Using native > (no orig-tarball) for RC releases, and quilt (with orig tarball) for > released kernels. > > This is a minimal conversion to the new formats. The immediate benefit > is ability to use xz compression for the source tarballs, GPG verified > upstream tarballs, and simplify kernel patching for those that are > rebuilding Ubuntu kernels. > > Once this is applied we will then be able to further simplify our > packaging by collapsing debian.* directories under debian dir, and > significantly reducing - if not eliminating our reconstruct scripts. > > Signed-off-by: Dimitri John Ledkov <dimitri.ledkov@canonical.com> > --- > debian/rules.d/1-maintainer.mk | 2 ++ > debian/source/format | 2 +- > debian/source/local-options | 12 ++++++++++++ > 3 files changed, 15 insertions(+), 1 deletion(-) > create mode 100644 debian/source/local-options > > diff --git a/debian/rules.d/1-maintainer.mk b/debian/rules.d/1-maintainer.mk > index dfcd814e2d..51eae55f78 100644 > --- a/debian/rules.d/1-maintainer.mk > +++ b/debian/rules.d/1-maintainer.mk > @@ -114,8 +114,10 @@ autoreconstruct: > # No need for reconstruct for -rc kernels since we don't upload an > # orig tarball, so just remove it. > if grep -q "^EXTRAVERSION = -rc[0-9]\+$$" Makefile; then \ > + echo "3.0 (native)" > debian/source/format; \ > echo "exit 0" >$(DEBIAN)/reconstruct; \ > else \ > + echo "3.0 (quilt)" > debian/source/format; \ > $(DROOT)/scripts/misc/gen-auto-reconstruct $(upstream_tag) $(DEBIAN)/reconstruct $(DROOT)/source/options; \ > fi > > diff --git a/debian/source/format b/debian/source/format > index d3827e75a5..89ae9db8f8 100644 > --- a/debian/source/format > +++ b/debian/source/format > @@ -1 +1 @@ > -1.0 > +3.0 (native) debian/source/format is version-controlled and then recreated by debian/rules.d/1-maintainer.mk You should do either of them, not both. > diff --git a/debian/source/local-options b/debian/source/local-options > new file mode 100644 > index 0000000000..d77c6f2631 > --- /dev/null > +++ b/debian/source/local-options > @@ -0,0 +1,12 @@ > +# Do not ingore removed files and include them in the automatically > +# generated patch > +include-removal > +# Do not try to prepare the build tree by applying patches which are > +# apparently unapplied > +no-preparation > +# Use debian-changes patch name without version number > +single-debian-patch > +# Do not try to unapply patches > +no-unapply-patches > +# Automatically add to quilt series the autogenerated patch > +auto-commit This is a good progress, although I am not a big fan of auto-commit, which can include garbage files into the source file if you missed to run 'git clean' in advance. Also, it does not handle addition of symlinks as far as I understood.
On Wed, 14 Feb 2024, 06:13 Masahiro Yamada, <masahiro.yamada@canonical.com> wrote: > On Wed, Feb 7, 2024 at 11:02 AM Dimitri John Ledkov > <dimitri.ledkov@canonical.com> wrote: > > > > This patch switches packaging to 3.0 dpkg source formats. Using native > > (no orig-tarball) for RC releases, and quilt (with orig tarball) for > > released kernels. > > > > This is a minimal conversion to the new formats. The immediate benefit > > is ability to use xz compression for the source tarballs, GPG verified > > upstream tarballs, and simplify kernel patching for those that are > > rebuilding Ubuntu kernels. > > > > Once this is applied we will then be able to further simplify our > > packaging by collapsing debian.* directories under debian dir, and > > significantly reducing - if not eliminating our reconstruct scripts. > > > > Signed-off-by: Dimitri John Ledkov <dimitri.ledkov@canonical.com> > > --- > > debian/rules.d/1-maintainer.mk | 2 ++ > > debian/source/format | 2 +- > > debian/source/local-options | 12 ++++++++++++ > > 3 files changed, 15 insertions(+), 1 deletion(-) > > create mode 100644 debian/source/local-options > > > > diff --git a/debian/rules.d/1-maintainer.mk b/debian/rules.d/ > 1-maintainer.mk > > index dfcd814e2d..51eae55f78 100644 > > --- a/debian/rules.d/1-maintainer.mk > > +++ b/debian/rules.d/1-maintainer.mk > > @@ -114,8 +114,10 @@ autoreconstruct: > > # No need for reconstruct for -rc kernels since we don't upload > an > > # orig tarball, so just remove it. > > if grep -q "^EXTRAVERSION = -rc[0-9]\+$$" Makefile; then \ > > + echo "3.0 (native)" > debian/source/format; \ > > echo "exit 0" >$(DEBIAN)/reconstruct; \ > > else \ > > + echo "3.0 (quilt)" > debian/source/format; \ > > $(DROOT)/scripts/misc/gen-auto-reconstruct > $(upstream_tag) $(DEBIAN)/reconstruct $(DROOT)/source/options; \ > > fi > > > > diff --git a/debian/source/format b/debian/source/format > > index d3827e75a5..89ae9db8f8 100644 > > --- a/debian/source/format > > +++ b/debian/source/format > > @@ -1 +1 @@ > > -1.0 > > +3.0 (native) > > > > debian/source/format is version-controlled > and then recreated by debian/rules.d/1-maintainer.mk > > You should do either of them, not both. > This is intentional, to ensure it is spotted in the git history, but also is a no-op reproducibly recreated upon calls to cranky close. Specifically I want it to be automatic when rcN cranks of linux-unstable kernel flip to released tarball, and again to rcN for new major. Or when released kernel packaging is used in mainline builds generates native tarball (although maybe do mainline builds should actually force native always?! Or always fetch the x.y.z tarball?!), and when derivatives are rebased things are generated correctly. In noble cycle we will be rapidly cranking all derivatives as RC kernels, and a random cranky rebase will switch all kernels to released tarballs. Whilst linux-unstable squad knows those transitions, I am hoping this explicit auto-generated and VCS committed source format will help crankers with "why did last upload just worked without a tarball, and this one now didn't". Hence I want this to be automatically updated and committed as part of cranky close invocation (similar to changelogs files, reconstruct script). Those two are also both generated and committed, and are entangled as required for source package generation. (Well our changelogs are weird as we have N of them from parent kernels, current kernel and ephemeral under debian/ dir used for build 🫠) Maybe that's overthinking it. Because it is trivially reproducible with and without git history. And actually would help with enforcement to call clean. > > diff --git a/debian/source/local-options b/debian/source/local-options > > new file mode 100644 > > index 0000000000..d77c6f2631 > > --- /dev/null > > +++ b/debian/source/local-options > > @@ -0,0 +1,12 @@ > > +# Do not ingore removed files and include them in the automatically > > +# generated patch > > +include-removal > > +# Do not try to prepare the build tree by applying patches which are > > +# apparently unapplied > > +no-preparation > > +# Use debian-changes patch name without version number > > +single-debian-patch > > +# Do not try to unapply patches > > +no-unapply-patches > > +# Automatically add to quilt series the autogenerated patch > > +auto-commit > > This is a good progress, although I am not a big fan of auto-commit, > which can include garbage files into the source file if you missed > to run 'git clean' in advance. > True. Instead of single-debian-parch and auto-commit we can generate equivalent of that, strictly based on VCS committed changes, as part of cranky close, just like when we generate the reconstruct script. (And not commit the generated patch to VCS). Then if there is any uncommited garbage, the source package build will fail verbosily complaining about everything. > Also, it does not handle addition of symlinks > as far as I understood. > Yes, that remains as actions done in the generated and committed reconstruct script. Pending git-patch syntax for that, to be supported by gnu-patch and/or dpkg implementation of quilt or some such. Just like for example binaries & some executable permissions are already supported "just like that". >
On Wed, Feb 14, 2024 at 3:47 PM Dimitri John Ledkov <dimitri.ledkov@canonical.com> wrote: > > > > On Wed, 14 Feb 2024, 06:13 Masahiro Yamada, <masahiro.yamada@canonical.com> wrote: >> >> On Wed, Feb 7, 2024 at 11:02 AM Dimitri John Ledkov >> <dimitri.ledkov@canonical.com> wrote: >> > >> > This patch switches packaging to 3.0 dpkg source formats. Using native >> > (no orig-tarball) for RC releases, and quilt (with orig tarball) for >> > released kernels. >> > >> > This is a minimal conversion to the new formats. The immediate benefit >> > is ability to use xz compression for the source tarballs, GPG verified >> > upstream tarballs, and simplify kernel patching for those that are >> > rebuilding Ubuntu kernels. >> > >> > Once this is applied we will then be able to further simplify our >> > packaging by collapsing debian.* directories under debian dir, and >> > significantly reducing - if not eliminating our reconstruct scripts. >> > >> > Signed-off-by: Dimitri John Ledkov <dimitri.ledkov@canonical.com> >> > --- >> > debian/rules.d/1-maintainer.mk | 2 ++ >> > debian/source/format | 2 +- >> > debian/source/local-options | 12 ++++++++++++ >> > 3 files changed, 15 insertions(+), 1 deletion(-) >> > create mode 100644 debian/source/local-options >> > >> > diff --git a/debian/rules.d/1-maintainer.mk b/debian/rules.d/1-maintainer.mk >> > index dfcd814e2d..51eae55f78 100644 >> > --- a/debian/rules.d/1-maintainer.mk >> > +++ b/debian/rules.d/1-maintainer.mk >> > @@ -114,8 +114,10 @@ autoreconstruct: >> > # No need for reconstruct for -rc kernels since we don't upload an >> > # orig tarball, so just remove it. >> > if grep -q "^EXTRAVERSION = -rc[0-9]\+$$" Makefile; then \ >> > + echo "3.0 (native)" > debian/source/format; \ >> > echo "exit 0" >$(DEBIAN)/reconstruct; \ >> > else \ >> > + echo "3.0 (quilt)" > debian/source/format; \ >> > $(DROOT)/scripts/misc/gen-auto-reconstruct $(upstream_tag) $(DEBIAN)/reconstruct $(DROOT)/source/options; \ >> > fi >> > >> > diff --git a/debian/source/format b/debian/source/format >> > index d3827e75a5..89ae9db8f8 100644 >> > --- a/debian/source/format >> > +++ b/debian/source/format >> > @@ -1 +1 @@ >> > -1.0 >> > +3.0 (native) >> >> >> >> debian/source/format is version-controlled >> and then recreated by debian/rules.d/1-maintainer.mk >> >> You should do either of them, not both. > > > This is intentional, to ensure it is spotted in the git history, but also is a no-op reproducibly recreated upon calls to cranky close. > > Specifically I want it to be automatic when rcN cranks of linux-unstable kernel flip to released tarball, and again to rcN for new major. Or when released kernel packaging is used in mainline builds generates native tarball (although maybe do mainline builds should actually force native always?! Or always fetch the x.y.z tarball?!), and when derivatives are rebased things are generated correctly. > > In noble cycle we will be rapidly cranking all derivatives as RC kernels, and a random cranky rebase will switch all kernels to released tarballs. Whilst linux-unstable squad knows those transitions, I am hoping this explicit auto-generated and VCS committed source format will help crankers with "why did last upload just worked without a tarball, and this one now didn't". > > Hence I want this to be automatically updated and committed as part of cranky close invocation (similar to changelogs files, reconstruct script). Those two are also both generated and committed, and are entangled as required for source package generation. > (Well our changelogs are weird as we have N of them from parent kernels, current kernel and ephemeral under debian/ dir used for build 🫠) > > Maybe that's overthinking it. Because it is trivially reproducible with and without git history. And actually would help with enforcement to call clean. > > >> >> > diff --git a/debian/source/local-options b/debian/source/local-options >> > new file mode 100644 >> > index 0000000000..d77c6f2631 >> > --- /dev/null >> > +++ b/debian/source/local-options >> > @@ -0,0 +1,12 @@ >> > +# Do not ingore removed files and include them in the automatically >> > +# generated patch >> > +include-removal >> > +# Do not try to prepare the build tree by applying patches which are >> > +# apparently unapplied >> > +no-preparation >> > +# Use debian-changes patch name without version number >> > +single-debian-patch >> > +# Do not try to unapply patches >> > +no-unapply-patches >> > +# Automatically add to quilt series the autogenerated patch >> > +auto-commit >> >> This is a good progress, although I am not a big fan of auto-commit, >> which can include garbage files into the source file if you missed >> to run 'git clean' in advance. > > > > True. Instead of single-debian-parch and auto-commit we can generate equivalent of that, strictly based on VCS committed changes, as part of cranky close, just like when we generate the reconstruct script. (And not commit the generated patch to VCS). > > Then if there is any uncommited garbage, the source package build will fail verbosily complaining about everything. > > >> >> Also, it does not handle addition of symlinks >> as far as I understood. > > > > Yes, that remains as actions done in the generated and committed reconstruct script. Pending git-patch syntax for that, to be supported by gnu-patch and/or dpkg implementation of quilt or some such. Just like for example binaries & some executable permissions are already supported "just like that". Setting minutes aside, we had better merge it rather than losing it, because nobody may commit such a change after you leave. Acked-by: Masahiro Yamada <masahiro.yamada@canonical.com>
diff --git a/debian/rules.d/1-maintainer.mk b/debian/rules.d/1-maintainer.mk index dfcd814e2d..51eae55f78 100644 --- a/debian/rules.d/1-maintainer.mk +++ b/debian/rules.d/1-maintainer.mk @@ -114,8 +114,10 @@ autoreconstruct: # No need for reconstruct for -rc kernels since we don't upload an # orig tarball, so just remove it. if grep -q "^EXTRAVERSION = -rc[0-9]\+$$" Makefile; then \ + echo "3.0 (native)" > debian/source/format; \ echo "exit 0" >$(DEBIAN)/reconstruct; \ else \ + echo "3.0 (quilt)" > debian/source/format; \ $(DROOT)/scripts/misc/gen-auto-reconstruct $(upstream_tag) $(DEBIAN)/reconstruct $(DROOT)/source/options; \ fi diff --git a/debian/source/format b/debian/source/format index d3827e75a5..89ae9db8f8 100644 --- a/debian/source/format +++ b/debian/source/format @@ -1 +1 @@ -1.0 +3.0 (native) diff --git a/debian/source/local-options b/debian/source/local-options new file mode 100644 index 0000000000..d77c6f2631 --- /dev/null +++ b/debian/source/local-options @@ -0,0 +1,12 @@ +# Do not ingore removed files and include them in the automatically +# generated patch +include-removal +# Do not try to prepare the build tree by applying patches which are +# apparently unapplied +no-preparation +# Use debian-changes patch name without version number +single-debian-patch +# Do not try to unapply patches +no-unapply-patches +# Automatically add to quilt series the autogenerated patch +auto-commit
This patch switches packaging to 3.0 dpkg source formats. Using native (no orig-tarball) for RC releases, and quilt (with orig tarball) for released kernels. This is a minimal conversion to the new formats. The immediate benefit is ability to use xz compression for the source tarballs, GPG verified upstream tarballs, and simplify kernel patching for those that are rebuilding Ubuntu kernels. Once this is applied we will then be able to further simplify our packaging by collapsing debian.* directories under debian dir, and significantly reducing - if not eliminating our reconstruct scripts. Signed-off-by: Dimitri John Ledkov <dimitri.ledkov@canonical.com> --- debian/rules.d/1-maintainer.mk | 2 ++ debian/source/format | 2 +- debian/source/local-options | 12 ++++++++++++ 3 files changed, 15 insertions(+), 1 deletion(-) create mode 100644 debian/source/local-options