| Message ID | 20210226163318.3215416-1-apw@canonical.com |
|---|---|
| State | New |
| Headers | show |
| Series | [SRU,bionic:linux,1/1] UBUNTU: [Packaging] sync dkms-build to updated API | expand |
On 2021-02-26 16:33:18 , Andy Whitcroft wrote: > We have two forms of dkms-build* in use in our package in various series. > These have different APIs, from 18.10 we add an additional parameter > to allow handling of debug packages. Having two incompatible APIs for > dkms-build significantly complicates the application of fixes for this > scripting, and thus complicates maintenance of both the kernel and LRM. > Switch 18.04 to the new debug aware API. > > We also have some additional functionality for handling debug packages > for DKMS builds. In later series we use this to supply ZFS debug > packages (though this patch set does not enable those). > > Finally we have two different versions of fix-filenames which the newer > of which contains an alignment fix and a cleaner pad character. Also > sync this while we are at it. > > Contains the bulk of the following commits: > > UBUNTU: [Debian]: Remove hardcoded $(pkgdir) in debug symbols handling > UBUNTU: [Debian]: Handle debug symbols for modules in extras too > UBUNTU: [Debian]: Check/link modules with debug symbols after DKMS modules > UBUNTU: [Debian]: Warn about modules without debug symbols > UBUNTU: [Debian]: dkms-build: new parameter for debug package directory > UBUNTU: [Debian]: dkms-build: Move zfs special-casing into configure script > UBUNTU: [Debian]: dkms-build: Avoid executing post-processor scripts twice > UBUNTU: [Packaging] dkms-build -- remove retpoline data from final binary packages > UBUNTU: [Packaging] dkms-build--nvidia* -- check gcc version against primary build > UBUNTU: [Packaging] fix-filenames -- handle exact string removal > UBUNTU: [Packaging] dkms-build--nvidia-N -- remove GCC versions > > BugLink: https://bugs.launchpad.net/bugs/1917034 > Signed-off-by: Andy Whitcroft <apw@canonical.com> Acked-by: Kelsey Skunberg <kelsey.skunberg@canonical.com> > --- > debian/rules.d/2-binary-arch.mk | 49 +++++++++++++---------- > debian/scripts/dkms-build | 60 +++++++++++++++++++++++++++-- > debian/scripts/dkms-build--nvidia-N | 38 +++++++++++++++--- > debian/scripts/fix-filenames.c | 13 +++++-- > 4 files changed, 129 insertions(+), 31 deletions(-) > > diff --git a/debian/rules.d/2-binary-arch.mk b/debian/rules.d/2-binary-arch.mk > index a8a1b9f593b0..db990d5bee3e 100644 > --- a/debian/rules.d/2-binary-arch.mk > +++ b/debian/rules.d/2-binary-arch.mk > @@ -60,13 +60,13 @@ define build_dkms_sign = > ) > endef > define build_dkms = > - $(SHELL) $(DROOT)/scripts/dkms-build $(dkms_dir) $(abi_release)-$* '$(call build_dkms_sign,$(builddir)/build-$*)' $(1) $(2) $(3) $(4) > + CROSS_COMPILE=$(CROSS_COMPILE) $(SHELL) $(DROOT)/scripts/dkms-build $(dkms_dir) $(abi_release)-$* '$(call build_dkms_sign,$(builddir)/build-$*)' $(1) $(2) $(3) $(4) $(5) > endef > > # nvidia_build_payload 450 450 450_450.102.04-0ubuntu0.20.04.1 > # nvidia_build_payload 450-server 450srv 50.102.04-0ubuntu0.20.04.1 > define nvidia_build_payload = > - $(call build_dkms, $(bldinfo_pkg_name)-$*, $(pkgdir_bldinfo)/usr/lib/linux/$(abi_release)-$*/signatures, nvidia-$(2), pool/restricted/n/nvidia-graphics-drivers-$(1)/nvidia-kernel-source-$(1)_$(3)_$(arch).deb pool/restricted/n/nvidia-graphics-drivers-$(1)/nvidia-dkms-$(1)_$(3)_$(arch).deb) > + $(call build_dkms, $(bldinfo_pkg_name)-$*, $(pkgdir_bldinfo)/usr/lib/linux/$(abi_release)-$*/signatures, "", nvidia-$(2), pool/restricted/n/nvidia-graphics-drivers-$(1)/nvidia-kernel-source-$(1)_$(3)_$(arch).deb pool/restricted/n/nvidia-graphics-drivers-$(1)/nvidia-dkms-$(1)_$(3)_$(arch).deb) > endef > # nvidia_build 450 > # nvidia_build 450-server > @@ -283,21 +283,7 @@ ifneq ($(skipdbg),true) > $(dbgpkgdir)/usr/lib/debug/boot/vmlinux-$(abi_release)-$* > $(build_cd) $(kmake) $(build_O) modules_install $(vdso) \ > INSTALL_MOD_PATH=$(dbgpkgdir)/usr/lib/debug > - # Add .gnu_debuglink sections to each stripped .ko > - # pointing to unstripped verson > - find $(pkgdir) -name '*.ko' | sed 's|$(pkgdir)||'| while read module ; do \ > - if [[ -f "$(dbgpkgdir)/usr/lib/debug/$$module" ]] ; then \ > - $(CROSS_COMPILE)objcopy \ > - --add-gnu-debuglink=$(dbgpkgdir)/usr/lib/debug/$$module \ > - $(pkgdir)/$$module; \ > - if grep -q CONFIG_MODULE_SIG=y $(builddir)/build-$*/.config; then \ > - $(builddir)/build-$*/scripts/sign-file $(MODHASHALGO) \ > - $(MODSECKEY) \ > - $(MODPUBKEY) \ > - $(pkgdir)/$$module; \ > - fi; \ > - fi; \ > - done > + # Add .gnu_debuglink sections only after all/DKMS modules are built. > rm -f $(dbgpkgdir)/usr/lib/debug/lib/modules/$(abi_release)-$*/build > rm -f $(dbgpkgdir)/usr/lib/debug/lib/modules/$(abi_release)-$*/source > rm -f $(dbgpkgdir)/usr/lib/debug/lib/modules/$(abi_release)-$*/modules.* > @@ -410,10 +396,10 @@ endif > install -d $(dkms_dir) $(dkms_dir)/headers $(dkms_dir)/build $(dkms_dir)/source > cp -rp "$(hdrdir)" "$(indep_hdrdir)" "$(dkms_dir)/headers" > > - $(if $(filter true,$(enable_zfs)),$(call build_dkms, $(mods_pkg_name)-$*, $(pkgdir)/lib/modules/$(abi_release)-$*/kernel, spl, pool/universe/s/spl-linux/spl-dkms_$(dkms_spl_linux_version)_all.deb)) > - $(if $(filter true,$(enable_zfs)),$(call build_dkms, $(mods_pkg_name)-$*, $(pkgdir)/lib/modules/$(abi_release)-$*/kernel, zfs, pool/universe/z/zfs-linux/zfs-dkms_$(dkms_zfs_linux_version)_all.deb)) > + $(if $(filter true,$(enable_zfs)),$(call build_dkms, $(mods_pkg_name)-$*, $(pkgdir)/lib/modules/$(abi_release)-$*/kernel, "", spl, pool/universe/s/spl-linux/spl-dkms_$(dkms_spl_linux_version)_all.deb)) > + $(if $(filter true,$(enable_zfs)),$(call build_dkms, $(mods_pkg_name)-$*, $(pkgdir)/lib/modules/$(abi_release)-$*/kernel, "", zfs, pool/universe/z/zfs-linux/zfs-dkms_$(dkms_zfs_linux_version)_all.deb)) > > - $(if $(filter true,$(do_dkms_wireguard)),$(call build_dkms, $(mods_pkg_name)-$*, $(pkgdir)/lib/modules/$(abi_release)-$*/kernel, wireguard, pool/universe/w/wireguard-linux-compat/wireguard-dkms_$(dkms_wireguard_version)_all.deb)) > + $(if $(filter true,$(do_dkms_wireguard)),$(call build_dkms, $(mods_pkg_name)-$*, $(pkgdir)/lib/modules/$(abi_release)-$*/kernel, "", wireguard, pool/universe/w/wireguard-linux-compat/wireguard-dkms_$(dkms_wireguard_version)_all.deb)) > > ifeq ($(do_dkms_nvidia),true) > $(foreach series,$(nvidia_desktop_series),$(call nvidia_build,$(series))) > @@ -422,6 +408,29 @@ ifeq ($(do_dkms_nvidia_server),true) > $(foreach series,$(nvidia_server_series),$(call nvidia_build,$(series))) > endif > > +ifneq ($(skipdbg),true) > + # Add .gnu_debuglink sections to each stripped .ko > + # pointing to unstripped verson > + find $(pkgdir) \ > + $(if $(filter true,$(do_extras_package)),$(pkgdir_ex)) \ > + -name '*.ko' | while read path_module ; do \ > + module="/lib/modules/$${path_module#*/lib/modules/}"; \ > + if [[ -f "$(dbgpkgdir)/usr/lib/debug/$$module" ]] ; then \ > + $(CROSS_COMPILE)objcopy \ > + --add-gnu-debuglink=$(dbgpkgdir)/usr/lib/debug/$$module \ > + $$path_module; \ > + if grep -q CONFIG_MODULE_SIG=y $(builddir)/build-$*/.config; then \ > + $(builddir)/build-$*/scripts/sign-file $(MODHASHALGO) \ > + $(MODSECKEY) \ > + $(MODPUBKEY) \ > + $$path_module; \ > + fi; \ > + else \ > + echo "WARNING: Missing debug symbols for module '$$module'."; \ > + fi; \ > + done > +endif > + > # Build the final ABI information. > install -d $(abidir) > sed -e 's/^\(.\+\)[[:space:]]\+\(.\+\)[[:space:]]\(.\+\)$$/\3 \2 \1/' \ > diff --git a/debian/scripts/dkms-build b/debian/scripts/dkms-build > index 5e5cef9bd3ec..c1269c01953a 100755 > --- a/debian/scripts/dkms-build > +++ b/debian/scripts/dkms-build > @@ -6,8 +6,9 @@ abi_flavour="$2" > sign="$3" > pkgname="$4" > pkgdir="$5" > -package="$6" > -shift 6 > +dbgpkgdir="$6" > +package="$7" > +shift 7 > > here=$(dirname "$(readlink -f "${0}")") > > @@ -104,6 +105,30 @@ done > # package is installed directly, but not so much if we build it out > # of the normal location. > sed -i -e '/^PACKAGE_NAME=/ s/#//g' "$dkms_conf" > + > +# Run any dkms-package specfic configuration steps > +dkms_config_specific="$srcdir/$0-configure--$package" > +dkms_config_generic=$(echo "$dkms_config_specific" | sed -e 's/-[0-9][0-9]*$/-N/') > +for dkms_config in "$dkms_config_specific" "$dkms_config_generic" > +do > + if [ -z "$dkms_config" -o ! -e "$dkms_config" ]; then > + continue > + fi > + echo "II: dkms-build-configure $(basename "$dkms_config") found, executing" > + $SHELL "$dkms_config" \ > + "$srcdir" \ > + "$dkms_conf" \ > + "$dkms_dir" \ > + "$abi_flavour" \ > + "$sign" \ > + "$pkgname" \ > + "$pkgdir" \ > + "$dbgpkgdir" \ > + "$package" \ > + "$@" || exit 1 > + break > +done > + > cat - <<'EOF' >>"$dkms_conf" > POST_BUILD="ubuntu-save-objects ${dkms_tree}/${PACKAGE_NAME}/${PACKAGE_VERSION}/build ${dkms_tree}/${PACKAGE_NAME}/${PACKAGE_VERSION}/objects $POST_BUILD" > EOF > @@ -150,6 +175,34 @@ sed -e "s@$dkms_dir@<<DKMSDIR>>@g" <"$log" > # If this build failed then exit here. > [ "$rc" != 0 ] && exit "$rc" > > +# Install the modules with debug symbols we possibly built, > +# and strip the original modules for the next install step. > +if [ -n "$dbgpkgdir" ]; then > + dbgpkgdir="$dbgpkgdir/$package" > + echo "II: dkms-build installing $package into $dbgpkgdir (debug symbols)" > + install -d "$dbgpkgdir" > + find "$dkms_dir/build/$dkms_package/$dkms_version/$abi_version" -name \*.ko | > + while read module; do > + vmodule=$( basename "$module" ) > + > + # Check for '.debug_info' section in order to copy module. > + # Useful if debug symbols are requested but not built for > + # any reason (including not yet supported by DKMS package). > + # Strip module just in case even if section isn't present. > + if ${CROSS_COMPILE}objdump -h -j '.debug_info' "$module" >/dev/null 2>&1 > + then > + echo "copying $vmodule" > + cp "$module" "$dbgpkgdir" > + else > + echo "ignoring $vmodule (missing debug symbols)" > + fi > + > + # Just 'strip -g' as '/usr/sbin/dkms' does. > + echo "stripping $vmodule" > + strip -g "$module" > + done > +fi > + > # Install and optionally sign the modules we have built. > pkgdir="$pkgdir/$package" > echo "II: dkms-build installing $package into $pkgdir" > @@ -183,7 +236,7 @@ dkms_build_specific="$srcdir/$0--$package" > dkms_build_generic=$(echo "$dkms_build_specific" | sed -n -e 's/-[0-9][0-9]*[a-z]*$/-N/p') > for dkms_build in "$dkms_build_specific" "$dkms_build_generic" > do > - if [ ! -e "$dkms_build" ]; then > + if [ -z "$dkms_build" -o ! -e "$dkms_build" ]; then > continue > fi > echo "II: dkms-build override $(basename "$dkms_build") found, executing" > @@ -196,6 +249,7 @@ do > "$sign" \ > "$pkgname" \ > "$pkgdir" \ > + "$dbgpkgdir" \ > "$package" \ > "$@" || exit 1 > break > diff --git a/debian/scripts/dkms-build--nvidia-N b/debian/scripts/dkms-build--nvidia-N > index 3759232549f1..bdec78233b43 100755 > --- a/debian/scripts/dkms-build--nvidia-N > +++ b/debian/scripts/dkms-build--nvidia-N > @@ -10,14 +10,29 @@ abi_flavour="$2" > sign="$3" > pkgname="$4" > pkgdir="$5" > -package="$6" > -shift 6 > +dbgpkgdir="$6" > +package="$7" > +shift 7 > > build="$( dirname "$objects" )/build" > > -# Copy over the objects ready for reconstruction. > +# Copy over the objects ready for reconstruction. The objects copy contains > +# the *.o files and the *.o-ur* retpoline files to allow the kernel to track > +# any retpoline sequences therein. For our purposes we only want the *.o > +# files, elide the rest. > mkdir -p "$pkgdir/bits/scripts" > -cp -rp "$objects"/* "$pkgdir/bits" > +( > + gcc_variant1=$(gcc --version | head -1 | sed -e 's/^gcc/GCC:/') > + gcc_variant2=$(gcc --version | head -1 | sed -e 's/^\(gcc\) \((.*)\) \(.*\)$/\1 version \3 \2/') > + cd "$objects" || exit 1 > + find -name \*.o | \ > + while read file > + do > + cp --parents "$file" "$pkgdir/bits" > + "$srcdir/debian/scripts/fix-filenames" "$pkgdir/bits/$file" "$gcc_variant1" > + "$srcdir/debian/scripts/fix-filenames" "$pkgdir/bits/$file" "$gcc_variant2" > + done > +) > > # Install the support files we need. > cp "$srcdir/scripts/module-common.lds" "$pkgdir/bits/scripts" > @@ -45,9 +60,22 @@ grep /usr/bin/ld.bfd "$log" | sed -e "s@$build/@@g" \ > elif [ "$sign" = "--lrm" ]; then > # We are in LRM build the package a copy in any signatures we can > # find for them. These will be added after linking. > + base="/usr/lib/linux/$abi_flavour" > + > + # Check the GCC version we are using against that used in the kernel > + # NOTE: that we treat this as only a warning, as if the binaries did come > + # out differently then we will actually > + echo "II: checking gcc version ..." > + cat "$base/compiler" > + gcc --version > + gcc_was=$(cat "$base/compiler" | sed -e 's/^GCC:/gcc/') > + gcc_is=$(gcc --version | head -1) > + if [ "$gcc_was" != "$gcc_is" ]; then > + echo "WW: gcc version missmatch between linux and linux-restricted-modules" > + echo "WW: was: $gcc_was is: $gcc_is" > + fi > > # Apply any local signatures. > - base="/usr/lib/linux/$abi_flavour" > echo "II: adding signatures from $base ..." > cp "$base/signatures/$package/"*".ko.sig" "$pkgdir/bits" > sha256sum -c "$base/signatures/$package/SHA256SUMS" || exit 1 > diff --git a/debian/scripts/fix-filenames.c b/debian/scripts/fix-filenames.c > index e58dc72f8f94..9bd144a6efa8 100644 > --- a/debian/scripts/fix-filenames.c > +++ b/debian/scripts/fix-filenames.c > @@ -48,7 +48,6 @@ main(int argc, char *argv[]) > exit(1); > } > size = in_info.st_size; > - printf("%s %ld bytes\n", in_name + prefix_len + 1, size); > > in = mmap((void *)0, size, PROT_READ|PROT_WRITE, MAP_SHARED, in_fd, (off_t)0); > if (!in) { > @@ -61,13 +60,21 @@ main(int argc, char *argv[]) > continue; > if (strncmp(in, prefix, prefix_len) != 0) > continue; > - length = strlen(in + prefix_len + 1) + 1; > + /* In the case of an exact match there there is nothing to move. */ > + if (in[prefix_len] == '\0') > + length = 0; > + /* If this is a filename, strip the leading slash. */ > + else if (in[prefix_len] == '/') > + length = strlen(in + prefix_len + 1) + 1; > + /* Otherwise just keep the suffix. */ > + else > + length = strlen(in + prefix_len) + 1; > > /* > * Copy the suffix portion down to the start and clear > * the remainder of the space to 0. > */ > memmove(in, in + prefix_len + 1, length); > - memset(in + length, '_', prefix_len); > + memset(in + length, '\0', prefix_len); > } > } > -- > 2.29.2 > > > -- > kernel-team mailing list > kernel-team@lists.ubuntu.com > https://lists.ubuntu.com/mailman/listinfo/kernel-team
On 26.02.21 17:33, Andy Whitcroft wrote: > We have two forms of dkms-build* in use in our package in various series. > These have different APIs, from 18.10 we add an additional parameter > to allow handling of debug packages. Having two incompatible APIs for > dkms-build significantly complicates the application of fixes for this > scripting, and thus complicates maintenance of both the kernel and LRM. > Switch 18.04 to the new debug aware API. > > We also have some additional functionality for handling debug packages > for DKMS builds. In later series we use this to supply ZFS debug > packages (though this patch set does not enable those). > > Finally we have two different versions of fix-filenames which the newer > of which contains an alignment fix and a cleaner pad character. Also > sync this while we are at it. > > Contains the bulk of the following commits: > > UBUNTU: [Debian]: Remove hardcoded $(pkgdir) in debug symbols handling > UBUNTU: [Debian]: Handle debug symbols for modules in extras too > UBUNTU: [Debian]: Check/link modules with debug symbols after DKMS modules > UBUNTU: [Debian]: Warn about modules without debug symbols > UBUNTU: [Debian]: dkms-build: new parameter for debug package directory > UBUNTU: [Debian]: dkms-build: Move zfs special-casing into configure script > UBUNTU: [Debian]: dkms-build: Avoid executing post-processor scripts twice > UBUNTU: [Packaging] dkms-build -- remove retpoline data from final binary packages > UBUNTU: [Packaging] dkms-build--nvidia* -- check gcc version against primary build > UBUNTU: [Packaging] fix-filenames -- handle exact string removal > UBUNTU: [Packaging] dkms-build--nvidia-N -- remove GCC versions > > BugLink: https://bugs.launchpad.net/bugs/1917034 > Signed-off-by: Andy Whitcroft <apw@canonical.com> Acked-by: Stefan Bader <stefan.bader@canonical.com> > --- Even in its condensed form I find it hard to review the code formally (due to size) but it should be obviously work or not once applied, so lets do that. -Stefan > debian/rules.d/2-binary-arch.mk | 49 +++++++++++++---------- > debian/scripts/dkms-build | 60 +++++++++++++++++++++++++++-- > debian/scripts/dkms-build--nvidia-N | 38 +++++++++++++++--- > debian/scripts/fix-filenames.c | 13 +++++-- > 4 files changed, 129 insertions(+), 31 deletions(-) > > diff --git a/debian/rules.d/2-binary-arch.mk b/debian/rules.d/2-binary-arch.mk > index a8a1b9f593b0..db990d5bee3e 100644 > --- a/debian/rules.d/2-binary-arch.mk > +++ b/debian/rules.d/2-binary-arch.mk > @@ -60,13 +60,13 @@ define build_dkms_sign = > ) > endef > define build_dkms = > - $(SHELL) $(DROOT)/scripts/dkms-build $(dkms_dir) $(abi_release)-$* '$(call build_dkms_sign,$(builddir)/build-$*)' $(1) $(2) $(3) $(4) > + CROSS_COMPILE=$(CROSS_COMPILE) $(SHELL) $(DROOT)/scripts/dkms-build $(dkms_dir) $(abi_release)-$* '$(call build_dkms_sign,$(builddir)/build-$*)' $(1) $(2) $(3) $(4) $(5) > endef > > # nvidia_build_payload 450 450 450_450.102.04-0ubuntu0.20.04.1 > # nvidia_build_payload 450-server 450srv 50.102.04-0ubuntu0.20.04.1 > define nvidia_build_payload = > - $(call build_dkms, $(bldinfo_pkg_name)-$*, $(pkgdir_bldinfo)/usr/lib/linux/$(abi_release)-$*/signatures, nvidia-$(2), pool/restricted/n/nvidia-graphics-drivers-$(1)/nvidia-kernel-source-$(1)_$(3)_$(arch).deb pool/restricted/n/nvidia-graphics-drivers-$(1)/nvidia-dkms-$(1)_$(3)_$(arch).deb) > + $(call build_dkms, $(bldinfo_pkg_name)-$*, $(pkgdir_bldinfo)/usr/lib/linux/$(abi_release)-$*/signatures, "", nvidia-$(2), pool/restricted/n/nvidia-graphics-drivers-$(1)/nvidia-kernel-source-$(1)_$(3)_$(arch).deb pool/restricted/n/nvidia-graphics-drivers-$(1)/nvidia-dkms-$(1)_$(3)_$(arch).deb) > endef > # nvidia_build 450 > # nvidia_build 450-server > @@ -283,21 +283,7 @@ ifneq ($(skipdbg),true) > $(dbgpkgdir)/usr/lib/debug/boot/vmlinux-$(abi_release)-$* > $(build_cd) $(kmake) $(build_O) modules_install $(vdso) \ > INSTALL_MOD_PATH=$(dbgpkgdir)/usr/lib/debug > - # Add .gnu_debuglink sections to each stripped .ko > - # pointing to unstripped verson > - find $(pkgdir) -name '*.ko' | sed 's|$(pkgdir)||'| while read module ; do \ > - if [[ -f "$(dbgpkgdir)/usr/lib/debug/$$module" ]] ; then \ > - $(CROSS_COMPILE)objcopy \ > - --add-gnu-debuglink=$(dbgpkgdir)/usr/lib/debug/$$module \ > - $(pkgdir)/$$module; \ > - if grep -q CONFIG_MODULE_SIG=y $(builddir)/build-$*/.config; then \ > - $(builddir)/build-$*/scripts/sign-file $(MODHASHALGO) \ > - $(MODSECKEY) \ > - $(MODPUBKEY) \ > - $(pkgdir)/$$module; \ > - fi; \ > - fi; \ > - done > + # Add .gnu_debuglink sections only after all/DKMS modules are built. > rm -f $(dbgpkgdir)/usr/lib/debug/lib/modules/$(abi_release)-$*/build > rm -f $(dbgpkgdir)/usr/lib/debug/lib/modules/$(abi_release)-$*/source > rm -f $(dbgpkgdir)/usr/lib/debug/lib/modules/$(abi_release)-$*/modules.* > @@ -410,10 +396,10 @@ endif > install -d $(dkms_dir) $(dkms_dir)/headers $(dkms_dir)/build $(dkms_dir)/source > cp -rp "$(hdrdir)" "$(indep_hdrdir)" "$(dkms_dir)/headers" > > - $(if $(filter true,$(enable_zfs)),$(call build_dkms, $(mods_pkg_name)-$*, $(pkgdir)/lib/modules/$(abi_release)-$*/kernel, spl, pool/universe/s/spl-linux/spl-dkms_$(dkms_spl_linux_version)_all.deb)) > - $(if $(filter true,$(enable_zfs)),$(call build_dkms, $(mods_pkg_name)-$*, $(pkgdir)/lib/modules/$(abi_release)-$*/kernel, zfs, pool/universe/z/zfs-linux/zfs-dkms_$(dkms_zfs_linux_version)_all.deb)) > + $(if $(filter true,$(enable_zfs)),$(call build_dkms, $(mods_pkg_name)-$*, $(pkgdir)/lib/modules/$(abi_release)-$*/kernel, "", spl, pool/universe/s/spl-linux/spl-dkms_$(dkms_spl_linux_version)_all.deb)) > + $(if $(filter true,$(enable_zfs)),$(call build_dkms, $(mods_pkg_name)-$*, $(pkgdir)/lib/modules/$(abi_release)-$*/kernel, "", zfs, pool/universe/z/zfs-linux/zfs-dkms_$(dkms_zfs_linux_version)_all.deb)) > > - $(if $(filter true,$(do_dkms_wireguard)),$(call build_dkms, $(mods_pkg_name)-$*, $(pkgdir)/lib/modules/$(abi_release)-$*/kernel, wireguard, pool/universe/w/wireguard-linux-compat/wireguard-dkms_$(dkms_wireguard_version)_all.deb)) > + $(if $(filter true,$(do_dkms_wireguard)),$(call build_dkms, $(mods_pkg_name)-$*, $(pkgdir)/lib/modules/$(abi_release)-$*/kernel, "", wireguard, pool/universe/w/wireguard-linux-compat/wireguard-dkms_$(dkms_wireguard_version)_all.deb)) > > ifeq ($(do_dkms_nvidia),true) > $(foreach series,$(nvidia_desktop_series),$(call nvidia_build,$(series))) > @@ -422,6 +408,29 @@ ifeq ($(do_dkms_nvidia_server),true) > $(foreach series,$(nvidia_server_series),$(call nvidia_build,$(series))) > endif > > +ifneq ($(skipdbg),true) > + # Add .gnu_debuglink sections to each stripped .ko > + # pointing to unstripped verson > + find $(pkgdir) \ > + $(if $(filter true,$(do_extras_package)),$(pkgdir_ex)) \ > + -name '*.ko' | while read path_module ; do \ > + module="/lib/modules/$${path_module#*/lib/modules/}"; \ > + if [[ -f "$(dbgpkgdir)/usr/lib/debug/$$module" ]] ; then \ > + $(CROSS_COMPILE)objcopy \ > + --add-gnu-debuglink=$(dbgpkgdir)/usr/lib/debug/$$module \ > + $$path_module; \ > + if grep -q CONFIG_MODULE_SIG=y $(builddir)/build-$*/.config; then \ > + $(builddir)/build-$*/scripts/sign-file $(MODHASHALGO) \ > + $(MODSECKEY) \ > + $(MODPUBKEY) \ > + $$path_module; \ > + fi; \ > + else \ > + echo "WARNING: Missing debug symbols for module '$$module'."; \ > + fi; \ > + done > +endif > + > # Build the final ABI information. > install -d $(abidir) > sed -e 's/^\(.\+\)[[:space:]]\+\(.\+\)[[:space:]]\(.\+\)$$/\3 \2 \1/' \ > diff --git a/debian/scripts/dkms-build b/debian/scripts/dkms-build > index 5e5cef9bd3ec..c1269c01953a 100755 > --- a/debian/scripts/dkms-build > +++ b/debian/scripts/dkms-build > @@ -6,8 +6,9 @@ abi_flavour="$2" > sign="$3" > pkgname="$4" > pkgdir="$5" > -package="$6" > -shift 6 > +dbgpkgdir="$6" > +package="$7" > +shift 7 > > here=$(dirname "$(readlink -f "${0}")") > > @@ -104,6 +105,30 @@ done > # package is installed directly, but not so much if we build it out > # of the normal location. > sed -i -e '/^PACKAGE_NAME=/ s/#//g' "$dkms_conf" > + > +# Run any dkms-package specfic configuration steps > +dkms_config_specific="$srcdir/$0-configure--$package" > +dkms_config_generic=$(echo "$dkms_config_specific" | sed -e 's/-[0-9][0-9]*$/-N/') > +for dkms_config in "$dkms_config_specific" "$dkms_config_generic" > +do > + if [ -z "$dkms_config" -o ! -e "$dkms_config" ]; then > + continue > + fi > + echo "II: dkms-build-configure $(basename "$dkms_config") found, executing" > + $SHELL "$dkms_config" \ > + "$srcdir" \ > + "$dkms_conf" \ > + "$dkms_dir" \ > + "$abi_flavour" \ > + "$sign" \ > + "$pkgname" \ > + "$pkgdir" \ > + "$dbgpkgdir" \ > + "$package" \ > + "$@" || exit 1 > + break > +done > + > cat - <<'EOF' >>"$dkms_conf" > POST_BUILD="ubuntu-save-objects ${dkms_tree}/${PACKAGE_NAME}/${PACKAGE_VERSION}/build ${dkms_tree}/${PACKAGE_NAME}/${PACKAGE_VERSION}/objects $POST_BUILD" > EOF > @@ -150,6 +175,34 @@ sed -e "s@$dkms_dir@<<DKMSDIR>>@g" <"$log" > # If this build failed then exit here. > [ "$rc" != 0 ] && exit "$rc" > > +# Install the modules with debug symbols we possibly built, > +# and strip the original modules for the next install step. > +if [ -n "$dbgpkgdir" ]; then > + dbgpkgdir="$dbgpkgdir/$package" > + echo "II: dkms-build installing $package into $dbgpkgdir (debug symbols)" > + install -d "$dbgpkgdir" > + find "$dkms_dir/build/$dkms_package/$dkms_version/$abi_version" -name \*.ko | > + while read module; do > + vmodule=$( basename "$module" ) > + > + # Check for '.debug_info' section in order to copy module. > + # Useful if debug symbols are requested but not built for > + # any reason (including not yet supported by DKMS package). > + # Strip module just in case even if section isn't present. > + if ${CROSS_COMPILE}objdump -h -j '.debug_info' "$module" >/dev/null 2>&1 > + then > + echo "copying $vmodule" > + cp "$module" "$dbgpkgdir" > + else > + echo "ignoring $vmodule (missing debug symbols)" > + fi > + > + # Just 'strip -g' as '/usr/sbin/dkms' does. > + echo "stripping $vmodule" > + strip -g "$module" > + done > +fi > + > # Install and optionally sign the modules we have built. > pkgdir="$pkgdir/$package" > echo "II: dkms-build installing $package into $pkgdir" > @@ -183,7 +236,7 @@ dkms_build_specific="$srcdir/$0--$package" > dkms_build_generic=$(echo "$dkms_build_specific" | sed -n -e 's/-[0-9][0-9]*[a-z]*$/-N/p') > for dkms_build in "$dkms_build_specific" "$dkms_build_generic" > do > - if [ ! -e "$dkms_build" ]; then > + if [ -z "$dkms_build" -o ! -e "$dkms_build" ]; then > continue > fi > echo "II: dkms-build override $(basename "$dkms_build") found, executing" > @@ -196,6 +249,7 @@ do > "$sign" \ > "$pkgname" \ > "$pkgdir" \ > + "$dbgpkgdir" \ > "$package" \ > "$@" || exit 1 > break > diff --git a/debian/scripts/dkms-build--nvidia-N b/debian/scripts/dkms-build--nvidia-N > index 3759232549f1..bdec78233b43 100755 > --- a/debian/scripts/dkms-build--nvidia-N > +++ b/debian/scripts/dkms-build--nvidia-N > @@ -10,14 +10,29 @@ abi_flavour="$2" > sign="$3" > pkgname="$4" > pkgdir="$5" > -package="$6" > -shift 6 > +dbgpkgdir="$6" > +package="$7" > +shift 7 > > build="$( dirname "$objects" )/build" > > -# Copy over the objects ready for reconstruction. > +# Copy over the objects ready for reconstruction. The objects copy contains > +# the *.o files and the *.o-ur* retpoline files to allow the kernel to track > +# any retpoline sequences therein. For our purposes we only want the *.o > +# files, elide the rest. > mkdir -p "$pkgdir/bits/scripts" > -cp -rp "$objects"/* "$pkgdir/bits" > +( > + gcc_variant1=$(gcc --version | head -1 | sed -e 's/^gcc/GCC:/') > + gcc_variant2=$(gcc --version | head -1 | sed -e 's/^\(gcc\) \((.*)\) \(.*\)$/\1 version \3 \2/') > + cd "$objects" || exit 1 > + find -name \*.o | \ > + while read file > + do > + cp --parents "$file" "$pkgdir/bits" > + "$srcdir/debian/scripts/fix-filenames" "$pkgdir/bits/$file" "$gcc_variant1" > + "$srcdir/debian/scripts/fix-filenames" "$pkgdir/bits/$file" "$gcc_variant2" > + done > +) > > # Install the support files we need. > cp "$srcdir/scripts/module-common.lds" "$pkgdir/bits/scripts" > @@ -45,9 +60,22 @@ grep /usr/bin/ld.bfd "$log" | sed -e "s@$build/@@g" \ > elif [ "$sign" = "--lrm" ]; then > # We are in LRM build the package a copy in any signatures we can > # find for them. These will be added after linking. > + base="/usr/lib/linux/$abi_flavour" > + > + # Check the GCC version we are using against that used in the kernel > + # NOTE: that we treat this as only a warning, as if the binaries did come > + # out differently then we will actually > + echo "II: checking gcc version ..." > + cat "$base/compiler" > + gcc --version > + gcc_was=$(cat "$base/compiler" | sed -e 's/^GCC:/gcc/') > + gcc_is=$(gcc --version | head -1) > + if [ "$gcc_was" != "$gcc_is" ]; then > + echo "WW: gcc version missmatch between linux and linux-restricted-modules" > + echo "WW: was: $gcc_was is: $gcc_is" > + fi > > # Apply any local signatures. > - base="/usr/lib/linux/$abi_flavour" > echo "II: adding signatures from $base ..." > cp "$base/signatures/$package/"*".ko.sig" "$pkgdir/bits" > sha256sum -c "$base/signatures/$package/SHA256SUMS" || exit 1 > diff --git a/debian/scripts/fix-filenames.c b/debian/scripts/fix-filenames.c > index e58dc72f8f94..9bd144a6efa8 100644 > --- a/debian/scripts/fix-filenames.c > +++ b/debian/scripts/fix-filenames.c > @@ -48,7 +48,6 @@ main(int argc, char *argv[]) > exit(1); > } > size = in_info.st_size; > - printf("%s %ld bytes\n", in_name + prefix_len + 1, size); > > in = mmap((void *)0, size, PROT_READ|PROT_WRITE, MAP_SHARED, in_fd, (off_t)0); > if (!in) { > @@ -61,13 +60,21 @@ main(int argc, char *argv[]) > continue; > if (strncmp(in, prefix, prefix_len) != 0) > continue; > - length = strlen(in + prefix_len + 1) + 1; > + /* In the case of an exact match there there is nothing to move. */ > + if (in[prefix_len] == '\0') > + length = 0; > + /* If this is a filename, strip the leading slash. */ > + else if (in[prefix_len] == '/') > + length = strlen(in + prefix_len + 1) + 1; > + /* Otherwise just keep the suffix. */ > + else > + length = strlen(in + prefix_len) + 1; > > /* > * Copy the suffix portion down to the start and clear > * the remainder of the space to 0. > */ > memmove(in, in + prefix_len + 1, length); > - memset(in + length, '_', prefix_len); > + memset(in + length, '\0', prefix_len); > } > } >
On 26.02.21 17:33, Andy Whitcroft wrote: > We have two forms of dkms-build* in use in our package in various series. > These have different APIs, from 18.10 we add an additional parameter > to allow handling of debug packages. Having two incompatible APIs for > dkms-build significantly complicates the application of fixes for this > scripting, and thus complicates maintenance of both the kernel and LRM. > Switch 18.04 to the new debug aware API. > > We also have some additional functionality for handling debug packages > for DKMS builds. In later series we use this to supply ZFS debug > packages (though this patch set does not enable those). > > Finally we have two different versions of fix-filenames which the newer > of which contains an alignment fix and a cleaner pad character. Also > sync this while we are at it. > > Contains the bulk of the following commits: > > UBUNTU: [Debian]: Remove hardcoded $(pkgdir) in debug symbols handling > UBUNTU: [Debian]: Handle debug symbols for modules in extras too > UBUNTU: [Debian]: Check/link modules with debug symbols after DKMS modules > UBUNTU: [Debian]: Warn about modules without debug symbols > UBUNTU: [Debian]: dkms-build: new parameter for debug package directory > UBUNTU: [Debian]: dkms-build: Move zfs special-casing into configure script > UBUNTU: [Debian]: dkms-build: Avoid executing post-processor scripts twice > UBUNTU: [Packaging] dkms-build -- remove retpoline data from final binary packages > UBUNTU: [Packaging] dkms-build--nvidia* -- check gcc version against primary build > UBUNTU: [Packaging] fix-filenames -- handle exact string removal > UBUNTU: [Packaging] dkms-build--nvidia-N -- remove GCC versions > > BugLink: https://bugs.launchpad.net/bugs/1917034 > Signed-off-by: Andy Whitcroft <apw@canonical.com> > --- Applied to bionic:linux/master-next. Thanks. -Stefan > debian/rules.d/2-binary-arch.mk | 49 +++++++++++++---------- > debian/scripts/dkms-build | 60 +++++++++++++++++++++++++++-- > debian/scripts/dkms-build--nvidia-N | 38 +++++++++++++++--- > debian/scripts/fix-filenames.c | 13 +++++-- > 4 files changed, 129 insertions(+), 31 deletions(-) > > diff --git a/debian/rules.d/2-binary-arch.mk b/debian/rules.d/2-binary-arch.mk > index a8a1b9f593b0..db990d5bee3e 100644 > --- a/debian/rules.d/2-binary-arch.mk > +++ b/debian/rules.d/2-binary-arch.mk > @@ -60,13 +60,13 @@ define build_dkms_sign = > ) > endef > define build_dkms = > - $(SHELL) $(DROOT)/scripts/dkms-build $(dkms_dir) $(abi_release)-$* '$(call build_dkms_sign,$(builddir)/build-$*)' $(1) $(2) $(3) $(4) > + CROSS_COMPILE=$(CROSS_COMPILE) $(SHELL) $(DROOT)/scripts/dkms-build $(dkms_dir) $(abi_release)-$* '$(call build_dkms_sign,$(builddir)/build-$*)' $(1) $(2) $(3) $(4) $(5) > endef > > # nvidia_build_payload 450 450 450_450.102.04-0ubuntu0.20.04.1 > # nvidia_build_payload 450-server 450srv 50.102.04-0ubuntu0.20.04.1 > define nvidia_build_payload = > - $(call build_dkms, $(bldinfo_pkg_name)-$*, $(pkgdir_bldinfo)/usr/lib/linux/$(abi_release)-$*/signatures, nvidia-$(2), pool/restricted/n/nvidia-graphics-drivers-$(1)/nvidia-kernel-source-$(1)_$(3)_$(arch).deb pool/restricted/n/nvidia-graphics-drivers-$(1)/nvidia-dkms-$(1)_$(3)_$(arch).deb) > + $(call build_dkms, $(bldinfo_pkg_name)-$*, $(pkgdir_bldinfo)/usr/lib/linux/$(abi_release)-$*/signatures, "", nvidia-$(2), pool/restricted/n/nvidia-graphics-drivers-$(1)/nvidia-kernel-source-$(1)_$(3)_$(arch).deb pool/restricted/n/nvidia-graphics-drivers-$(1)/nvidia-dkms-$(1)_$(3)_$(arch).deb) > endef > # nvidia_build 450 > # nvidia_build 450-server > @@ -283,21 +283,7 @@ ifneq ($(skipdbg),true) > $(dbgpkgdir)/usr/lib/debug/boot/vmlinux-$(abi_release)-$* > $(build_cd) $(kmake) $(build_O) modules_install $(vdso) \ > INSTALL_MOD_PATH=$(dbgpkgdir)/usr/lib/debug > - # Add .gnu_debuglink sections to each stripped .ko > - # pointing to unstripped verson > - find $(pkgdir) -name '*.ko' | sed 's|$(pkgdir)||'| while read module ; do \ > - if [[ -f "$(dbgpkgdir)/usr/lib/debug/$$module" ]] ; then \ > - $(CROSS_COMPILE)objcopy \ > - --add-gnu-debuglink=$(dbgpkgdir)/usr/lib/debug/$$module \ > - $(pkgdir)/$$module; \ > - if grep -q CONFIG_MODULE_SIG=y $(builddir)/build-$*/.config; then \ > - $(builddir)/build-$*/scripts/sign-file $(MODHASHALGO) \ > - $(MODSECKEY) \ > - $(MODPUBKEY) \ > - $(pkgdir)/$$module; \ > - fi; \ > - fi; \ > - done > + # Add .gnu_debuglink sections only after all/DKMS modules are built. > rm -f $(dbgpkgdir)/usr/lib/debug/lib/modules/$(abi_release)-$*/build > rm -f $(dbgpkgdir)/usr/lib/debug/lib/modules/$(abi_release)-$*/source > rm -f $(dbgpkgdir)/usr/lib/debug/lib/modules/$(abi_release)-$*/modules.* > @@ -410,10 +396,10 @@ endif > install -d $(dkms_dir) $(dkms_dir)/headers $(dkms_dir)/build $(dkms_dir)/source > cp -rp "$(hdrdir)" "$(indep_hdrdir)" "$(dkms_dir)/headers" > > - $(if $(filter true,$(enable_zfs)),$(call build_dkms, $(mods_pkg_name)-$*, $(pkgdir)/lib/modules/$(abi_release)-$*/kernel, spl, pool/universe/s/spl-linux/spl-dkms_$(dkms_spl_linux_version)_all.deb)) > - $(if $(filter true,$(enable_zfs)),$(call build_dkms, $(mods_pkg_name)-$*, $(pkgdir)/lib/modules/$(abi_release)-$*/kernel, zfs, pool/universe/z/zfs-linux/zfs-dkms_$(dkms_zfs_linux_version)_all.deb)) > + $(if $(filter true,$(enable_zfs)),$(call build_dkms, $(mods_pkg_name)-$*, $(pkgdir)/lib/modules/$(abi_release)-$*/kernel, "", spl, pool/universe/s/spl-linux/spl-dkms_$(dkms_spl_linux_version)_all.deb)) > + $(if $(filter true,$(enable_zfs)),$(call build_dkms, $(mods_pkg_name)-$*, $(pkgdir)/lib/modules/$(abi_release)-$*/kernel, "", zfs, pool/universe/z/zfs-linux/zfs-dkms_$(dkms_zfs_linux_version)_all.deb)) > > - $(if $(filter true,$(do_dkms_wireguard)),$(call build_dkms, $(mods_pkg_name)-$*, $(pkgdir)/lib/modules/$(abi_release)-$*/kernel, wireguard, pool/universe/w/wireguard-linux-compat/wireguard-dkms_$(dkms_wireguard_version)_all.deb)) > + $(if $(filter true,$(do_dkms_wireguard)),$(call build_dkms, $(mods_pkg_name)-$*, $(pkgdir)/lib/modules/$(abi_release)-$*/kernel, "", wireguard, pool/universe/w/wireguard-linux-compat/wireguard-dkms_$(dkms_wireguard_version)_all.deb)) > > ifeq ($(do_dkms_nvidia),true) > $(foreach series,$(nvidia_desktop_series),$(call nvidia_build,$(series))) > @@ -422,6 +408,29 @@ ifeq ($(do_dkms_nvidia_server),true) > $(foreach series,$(nvidia_server_series),$(call nvidia_build,$(series))) > endif > > +ifneq ($(skipdbg),true) > + # Add .gnu_debuglink sections to each stripped .ko > + # pointing to unstripped verson > + find $(pkgdir) \ > + $(if $(filter true,$(do_extras_package)),$(pkgdir_ex)) \ > + -name '*.ko' | while read path_module ; do \ > + module="/lib/modules/$${path_module#*/lib/modules/}"; \ > + if [[ -f "$(dbgpkgdir)/usr/lib/debug/$$module" ]] ; then \ > + $(CROSS_COMPILE)objcopy \ > + --add-gnu-debuglink=$(dbgpkgdir)/usr/lib/debug/$$module \ > + $$path_module; \ > + if grep -q CONFIG_MODULE_SIG=y $(builddir)/build-$*/.config; then \ > + $(builddir)/build-$*/scripts/sign-file $(MODHASHALGO) \ > + $(MODSECKEY) \ > + $(MODPUBKEY) \ > + $$path_module; \ > + fi; \ > + else \ > + echo "WARNING: Missing debug symbols for module '$$module'."; \ > + fi; \ > + done > +endif > + > # Build the final ABI information. > install -d $(abidir) > sed -e 's/^\(.\+\)[[:space:]]\+\(.\+\)[[:space:]]\(.\+\)$$/\3 \2 \1/' \ > diff --git a/debian/scripts/dkms-build b/debian/scripts/dkms-build > index 5e5cef9bd3ec..c1269c01953a 100755 > --- a/debian/scripts/dkms-build > +++ b/debian/scripts/dkms-build > @@ -6,8 +6,9 @@ abi_flavour="$2" > sign="$3" > pkgname="$4" > pkgdir="$5" > -package="$6" > -shift 6 > +dbgpkgdir="$6" > +package="$7" > +shift 7 > > here=$(dirname "$(readlink -f "${0}")") > > @@ -104,6 +105,30 @@ done > # package is installed directly, but not so much if we build it out > # of the normal location. > sed -i -e '/^PACKAGE_NAME=/ s/#//g' "$dkms_conf" > + > +# Run any dkms-package specfic configuration steps > +dkms_config_specific="$srcdir/$0-configure--$package" > +dkms_config_generic=$(echo "$dkms_config_specific" | sed -e 's/-[0-9][0-9]*$/-N/') > +for dkms_config in "$dkms_config_specific" "$dkms_config_generic" > +do > + if [ -z "$dkms_config" -o ! -e "$dkms_config" ]; then > + continue > + fi > + echo "II: dkms-build-configure $(basename "$dkms_config") found, executing" > + $SHELL "$dkms_config" \ > + "$srcdir" \ > + "$dkms_conf" \ > + "$dkms_dir" \ > + "$abi_flavour" \ > + "$sign" \ > + "$pkgname" \ > + "$pkgdir" \ > + "$dbgpkgdir" \ > + "$package" \ > + "$@" || exit 1 > + break > +done > + > cat - <<'EOF' >>"$dkms_conf" > POST_BUILD="ubuntu-save-objects ${dkms_tree}/${PACKAGE_NAME}/${PACKAGE_VERSION}/build ${dkms_tree}/${PACKAGE_NAME}/${PACKAGE_VERSION}/objects $POST_BUILD" > EOF > @@ -150,6 +175,34 @@ sed -e "s@$dkms_dir@<<DKMSDIR>>@g" <"$log" > # If this build failed then exit here. > [ "$rc" != 0 ] && exit "$rc" > > +# Install the modules with debug symbols we possibly built, > +# and strip the original modules for the next install step. > +if [ -n "$dbgpkgdir" ]; then > + dbgpkgdir="$dbgpkgdir/$package" > + echo "II: dkms-build installing $package into $dbgpkgdir (debug symbols)" > + install -d "$dbgpkgdir" > + find "$dkms_dir/build/$dkms_package/$dkms_version/$abi_version" -name \*.ko | > + while read module; do > + vmodule=$( basename "$module" ) > + > + # Check for '.debug_info' section in order to copy module. > + # Useful if debug symbols are requested but not built for > + # any reason (including not yet supported by DKMS package). > + # Strip module just in case even if section isn't present. > + if ${CROSS_COMPILE}objdump -h -j '.debug_info' "$module" >/dev/null 2>&1 > + then > + echo "copying $vmodule" > + cp "$module" "$dbgpkgdir" > + else > + echo "ignoring $vmodule (missing debug symbols)" > + fi > + > + # Just 'strip -g' as '/usr/sbin/dkms' does. > + echo "stripping $vmodule" > + strip -g "$module" > + done > +fi > + > # Install and optionally sign the modules we have built. > pkgdir="$pkgdir/$package" > echo "II: dkms-build installing $package into $pkgdir" > @@ -183,7 +236,7 @@ dkms_build_specific="$srcdir/$0--$package" > dkms_build_generic=$(echo "$dkms_build_specific" | sed -n -e 's/-[0-9][0-9]*[a-z]*$/-N/p') > for dkms_build in "$dkms_build_specific" "$dkms_build_generic" > do > - if [ ! -e "$dkms_build" ]; then > + if [ -z "$dkms_build" -o ! -e "$dkms_build" ]; then > continue > fi > echo "II: dkms-build override $(basename "$dkms_build") found, executing" > @@ -196,6 +249,7 @@ do > "$sign" \ > "$pkgname" \ > "$pkgdir" \ > + "$dbgpkgdir" \ > "$package" \ > "$@" || exit 1 > break > diff --git a/debian/scripts/dkms-build--nvidia-N b/debian/scripts/dkms-build--nvidia-N > index 3759232549f1..bdec78233b43 100755 > --- a/debian/scripts/dkms-build--nvidia-N > +++ b/debian/scripts/dkms-build--nvidia-N > @@ -10,14 +10,29 @@ abi_flavour="$2" > sign="$3" > pkgname="$4" > pkgdir="$5" > -package="$6" > -shift 6 > +dbgpkgdir="$6" > +package="$7" > +shift 7 > > build="$( dirname "$objects" )/build" > > -# Copy over the objects ready for reconstruction. > +# Copy over the objects ready for reconstruction. The objects copy contains > +# the *.o files and the *.o-ur* retpoline files to allow the kernel to track > +# any retpoline sequences therein. For our purposes we only want the *.o > +# files, elide the rest. > mkdir -p "$pkgdir/bits/scripts" > -cp -rp "$objects"/* "$pkgdir/bits" > +( > + gcc_variant1=$(gcc --version | head -1 | sed -e 's/^gcc/GCC:/') > + gcc_variant2=$(gcc --version | head -1 | sed -e 's/^\(gcc\) \((.*)\) \(.*\)$/\1 version \3 \2/') > + cd "$objects" || exit 1 > + find -name \*.o | \ > + while read file > + do > + cp --parents "$file" "$pkgdir/bits" > + "$srcdir/debian/scripts/fix-filenames" "$pkgdir/bits/$file" "$gcc_variant1" > + "$srcdir/debian/scripts/fix-filenames" "$pkgdir/bits/$file" "$gcc_variant2" > + done > +) > > # Install the support files we need. > cp "$srcdir/scripts/module-common.lds" "$pkgdir/bits/scripts" > @@ -45,9 +60,22 @@ grep /usr/bin/ld.bfd "$log" | sed -e "s@$build/@@g" \ > elif [ "$sign" = "--lrm" ]; then > # We are in LRM build the package a copy in any signatures we can > # find for them. These will be added after linking. > + base="/usr/lib/linux/$abi_flavour" > + > + # Check the GCC version we are using against that used in the kernel > + # NOTE: that we treat this as only a warning, as if the binaries did come > + # out differently then we will actually > + echo "II: checking gcc version ..." > + cat "$base/compiler" > + gcc --version > + gcc_was=$(cat "$base/compiler" | sed -e 's/^GCC:/gcc/') > + gcc_is=$(gcc --version | head -1) > + if [ "$gcc_was" != "$gcc_is" ]; then > + echo "WW: gcc version missmatch between linux and linux-restricted-modules" > + echo "WW: was: $gcc_was is: $gcc_is" > + fi > > # Apply any local signatures. > - base="/usr/lib/linux/$abi_flavour" > echo "II: adding signatures from $base ..." > cp "$base/signatures/$package/"*".ko.sig" "$pkgdir/bits" > sha256sum -c "$base/signatures/$package/SHA256SUMS" || exit 1 > diff --git a/debian/scripts/fix-filenames.c b/debian/scripts/fix-filenames.c > index e58dc72f8f94..9bd144a6efa8 100644 > --- a/debian/scripts/fix-filenames.c > +++ b/debian/scripts/fix-filenames.c > @@ -48,7 +48,6 @@ main(int argc, char *argv[]) > exit(1); > } > size = in_info.st_size; > - printf("%s %ld bytes\n", in_name + prefix_len + 1, size); > > in = mmap((void *)0, size, PROT_READ|PROT_WRITE, MAP_SHARED, in_fd, (off_t)0); > if (!in) { > @@ -61,13 +60,21 @@ main(int argc, char *argv[]) > continue; > if (strncmp(in, prefix, prefix_len) != 0) > continue; > - length = strlen(in + prefix_len + 1) + 1; > + /* In the case of an exact match there there is nothing to move. */ > + if (in[prefix_len] == '\0') > + length = 0; > + /* If this is a filename, strip the leading slash. */ > + else if (in[prefix_len] == '/') > + length = strlen(in + prefix_len + 1) + 1; > + /* Otherwise just keep the suffix. */ > + else > + length = strlen(in + prefix_len) + 1; > > /* > * Copy the suffix portion down to the start and clear > * the remainder of the space to 0. > */ > memmove(in, in + prefix_len + 1, length); > - memset(in + length, '_', prefix_len); > + memset(in + length, '\0', prefix_len); > } > } >
diff --git a/debian/rules.d/2-binary-arch.mk b/debian/rules.d/2-binary-arch.mk index a8a1b9f593b0..db990d5bee3e 100644 --- a/debian/rules.d/2-binary-arch.mk +++ b/debian/rules.d/2-binary-arch.mk @@ -60,13 +60,13 @@ define build_dkms_sign = ) endef define build_dkms = - $(SHELL) $(DROOT)/scripts/dkms-build $(dkms_dir) $(abi_release)-$* '$(call build_dkms_sign,$(builddir)/build-$*)' $(1) $(2) $(3) $(4) + CROSS_COMPILE=$(CROSS_COMPILE) $(SHELL) $(DROOT)/scripts/dkms-build $(dkms_dir) $(abi_release)-$* '$(call build_dkms_sign,$(builddir)/build-$*)' $(1) $(2) $(3) $(4) $(5) endef # nvidia_build_payload 450 450 450_450.102.04-0ubuntu0.20.04.1 # nvidia_build_payload 450-server 450srv 50.102.04-0ubuntu0.20.04.1 define nvidia_build_payload = - $(call build_dkms, $(bldinfo_pkg_name)-$*, $(pkgdir_bldinfo)/usr/lib/linux/$(abi_release)-$*/signatures, nvidia-$(2), pool/restricted/n/nvidia-graphics-drivers-$(1)/nvidia-kernel-source-$(1)_$(3)_$(arch).deb pool/restricted/n/nvidia-graphics-drivers-$(1)/nvidia-dkms-$(1)_$(3)_$(arch).deb) + $(call build_dkms, $(bldinfo_pkg_name)-$*, $(pkgdir_bldinfo)/usr/lib/linux/$(abi_release)-$*/signatures, "", nvidia-$(2), pool/restricted/n/nvidia-graphics-drivers-$(1)/nvidia-kernel-source-$(1)_$(3)_$(arch).deb pool/restricted/n/nvidia-graphics-drivers-$(1)/nvidia-dkms-$(1)_$(3)_$(arch).deb) endef # nvidia_build 450 # nvidia_build 450-server @@ -283,21 +283,7 @@ ifneq ($(skipdbg),true) $(dbgpkgdir)/usr/lib/debug/boot/vmlinux-$(abi_release)-$* $(build_cd) $(kmake) $(build_O) modules_install $(vdso) \ INSTALL_MOD_PATH=$(dbgpkgdir)/usr/lib/debug - # Add .gnu_debuglink sections to each stripped .ko - # pointing to unstripped verson - find $(pkgdir) -name '*.ko' | sed 's|$(pkgdir)||'| while read module ; do \ - if [[ -f "$(dbgpkgdir)/usr/lib/debug/$$module" ]] ; then \ - $(CROSS_COMPILE)objcopy \ - --add-gnu-debuglink=$(dbgpkgdir)/usr/lib/debug/$$module \ - $(pkgdir)/$$module; \ - if grep -q CONFIG_MODULE_SIG=y $(builddir)/build-$*/.config; then \ - $(builddir)/build-$*/scripts/sign-file $(MODHASHALGO) \ - $(MODSECKEY) \ - $(MODPUBKEY) \ - $(pkgdir)/$$module; \ - fi; \ - fi; \ - done + # Add .gnu_debuglink sections only after all/DKMS modules are built. rm -f $(dbgpkgdir)/usr/lib/debug/lib/modules/$(abi_release)-$*/build rm -f $(dbgpkgdir)/usr/lib/debug/lib/modules/$(abi_release)-$*/source rm -f $(dbgpkgdir)/usr/lib/debug/lib/modules/$(abi_release)-$*/modules.* @@ -410,10 +396,10 @@ endif install -d $(dkms_dir) $(dkms_dir)/headers $(dkms_dir)/build $(dkms_dir)/source cp -rp "$(hdrdir)" "$(indep_hdrdir)" "$(dkms_dir)/headers" - $(if $(filter true,$(enable_zfs)),$(call build_dkms, $(mods_pkg_name)-$*, $(pkgdir)/lib/modules/$(abi_release)-$*/kernel, spl, pool/universe/s/spl-linux/spl-dkms_$(dkms_spl_linux_version)_all.deb)) - $(if $(filter true,$(enable_zfs)),$(call build_dkms, $(mods_pkg_name)-$*, $(pkgdir)/lib/modules/$(abi_release)-$*/kernel, zfs, pool/universe/z/zfs-linux/zfs-dkms_$(dkms_zfs_linux_version)_all.deb)) + $(if $(filter true,$(enable_zfs)),$(call build_dkms, $(mods_pkg_name)-$*, $(pkgdir)/lib/modules/$(abi_release)-$*/kernel, "", spl, pool/universe/s/spl-linux/spl-dkms_$(dkms_spl_linux_version)_all.deb)) + $(if $(filter true,$(enable_zfs)),$(call build_dkms, $(mods_pkg_name)-$*, $(pkgdir)/lib/modules/$(abi_release)-$*/kernel, "", zfs, pool/universe/z/zfs-linux/zfs-dkms_$(dkms_zfs_linux_version)_all.deb)) - $(if $(filter true,$(do_dkms_wireguard)),$(call build_dkms, $(mods_pkg_name)-$*, $(pkgdir)/lib/modules/$(abi_release)-$*/kernel, wireguard, pool/universe/w/wireguard-linux-compat/wireguard-dkms_$(dkms_wireguard_version)_all.deb)) + $(if $(filter true,$(do_dkms_wireguard)),$(call build_dkms, $(mods_pkg_name)-$*, $(pkgdir)/lib/modules/$(abi_release)-$*/kernel, "", wireguard, pool/universe/w/wireguard-linux-compat/wireguard-dkms_$(dkms_wireguard_version)_all.deb)) ifeq ($(do_dkms_nvidia),true) $(foreach series,$(nvidia_desktop_series),$(call nvidia_build,$(series))) @@ -422,6 +408,29 @@ ifeq ($(do_dkms_nvidia_server),true) $(foreach series,$(nvidia_server_series),$(call nvidia_build,$(series))) endif +ifneq ($(skipdbg),true) + # Add .gnu_debuglink sections to each stripped .ko + # pointing to unstripped verson + find $(pkgdir) \ + $(if $(filter true,$(do_extras_package)),$(pkgdir_ex)) \ + -name '*.ko' | while read path_module ; do \ + module="/lib/modules/$${path_module#*/lib/modules/}"; \ + if [[ -f "$(dbgpkgdir)/usr/lib/debug/$$module" ]] ; then \ + $(CROSS_COMPILE)objcopy \ + --add-gnu-debuglink=$(dbgpkgdir)/usr/lib/debug/$$module \ + $$path_module; \ + if grep -q CONFIG_MODULE_SIG=y $(builddir)/build-$*/.config; then \ + $(builddir)/build-$*/scripts/sign-file $(MODHASHALGO) \ + $(MODSECKEY) \ + $(MODPUBKEY) \ + $$path_module; \ + fi; \ + else \ + echo "WARNING: Missing debug symbols for module '$$module'."; \ + fi; \ + done +endif + # Build the final ABI information. install -d $(abidir) sed -e 's/^\(.\+\)[[:space:]]\+\(.\+\)[[:space:]]\(.\+\)$$/\3 \2 \1/' \ diff --git a/debian/scripts/dkms-build b/debian/scripts/dkms-build index 5e5cef9bd3ec..c1269c01953a 100755 --- a/debian/scripts/dkms-build +++ b/debian/scripts/dkms-build @@ -6,8 +6,9 @@ abi_flavour="$2" sign="$3" pkgname="$4" pkgdir="$5" -package="$6" -shift 6 +dbgpkgdir="$6" +package="$7" +shift 7 here=$(dirname "$(readlink -f "${0}")") @@ -104,6 +105,30 @@ done # package is installed directly, but not so much if we build it out # of the normal location. sed -i -e '/^PACKAGE_NAME=/ s/#//g' "$dkms_conf" + +# Run any dkms-package specfic configuration steps +dkms_config_specific="$srcdir/$0-configure--$package" +dkms_config_generic=$(echo "$dkms_config_specific" | sed -e 's/-[0-9][0-9]*$/-N/') +for dkms_config in "$dkms_config_specific" "$dkms_config_generic" +do + if [ -z "$dkms_config" -o ! -e "$dkms_config" ]; then + continue + fi + echo "II: dkms-build-configure $(basename "$dkms_config") found, executing" + $SHELL "$dkms_config" \ + "$srcdir" \ + "$dkms_conf" \ + "$dkms_dir" \ + "$abi_flavour" \ + "$sign" \ + "$pkgname" \ + "$pkgdir" \ + "$dbgpkgdir" \ + "$package" \ + "$@" || exit 1 + break +done + cat - <<'EOF' >>"$dkms_conf" POST_BUILD="ubuntu-save-objects ${dkms_tree}/${PACKAGE_NAME}/${PACKAGE_VERSION}/build ${dkms_tree}/${PACKAGE_NAME}/${PACKAGE_VERSION}/objects $POST_BUILD" EOF @@ -150,6 +175,34 @@ sed -e "s@$dkms_dir@<<DKMSDIR>>@g" <"$log" # If this build failed then exit here. [ "$rc" != 0 ] && exit "$rc" +# Install the modules with debug symbols we possibly built, +# and strip the original modules for the next install step. +if [ -n "$dbgpkgdir" ]; then + dbgpkgdir="$dbgpkgdir/$package" + echo "II: dkms-build installing $package into $dbgpkgdir (debug symbols)" + install -d "$dbgpkgdir" + find "$dkms_dir/build/$dkms_package/$dkms_version/$abi_version" -name \*.ko | + while read module; do + vmodule=$( basename "$module" ) + + # Check for '.debug_info' section in order to copy module. + # Useful if debug symbols are requested but not built for + # any reason (including not yet supported by DKMS package). + # Strip module just in case even if section isn't present. + if ${CROSS_COMPILE}objdump -h -j '.debug_info' "$module" >/dev/null 2>&1 + then + echo "copying $vmodule" + cp "$module" "$dbgpkgdir" + else + echo "ignoring $vmodule (missing debug symbols)" + fi + + # Just 'strip -g' as '/usr/sbin/dkms' does. + echo "stripping $vmodule" + strip -g "$module" + done +fi + # Install and optionally sign the modules we have built. pkgdir="$pkgdir/$package" echo "II: dkms-build installing $package into $pkgdir" @@ -183,7 +236,7 @@ dkms_build_specific="$srcdir/$0--$package" dkms_build_generic=$(echo "$dkms_build_specific" | sed -n -e 's/-[0-9][0-9]*[a-z]*$/-N/p') for dkms_build in "$dkms_build_specific" "$dkms_build_generic" do - if [ ! -e "$dkms_build" ]; then + if [ -z "$dkms_build" -o ! -e "$dkms_build" ]; then continue fi echo "II: dkms-build override $(basename "$dkms_build") found, executing" @@ -196,6 +249,7 @@ do "$sign" \ "$pkgname" \ "$pkgdir" \ + "$dbgpkgdir" \ "$package" \ "$@" || exit 1 break diff --git a/debian/scripts/dkms-build--nvidia-N b/debian/scripts/dkms-build--nvidia-N index 3759232549f1..bdec78233b43 100755 --- a/debian/scripts/dkms-build--nvidia-N +++ b/debian/scripts/dkms-build--nvidia-N @@ -10,14 +10,29 @@ abi_flavour="$2" sign="$3" pkgname="$4" pkgdir="$5" -package="$6" -shift 6 +dbgpkgdir="$6" +package="$7" +shift 7 build="$( dirname "$objects" )/build" -# Copy over the objects ready for reconstruction. +# Copy over the objects ready for reconstruction. The objects copy contains +# the *.o files and the *.o-ur* retpoline files to allow the kernel to track +# any retpoline sequences therein. For our purposes we only want the *.o +# files, elide the rest. mkdir -p "$pkgdir/bits/scripts" -cp -rp "$objects"/* "$pkgdir/bits" +( + gcc_variant1=$(gcc --version | head -1 | sed -e 's/^gcc/GCC:/') + gcc_variant2=$(gcc --version | head -1 | sed -e 's/^\(gcc\) \((.*)\) \(.*\)$/\1 version \3 \2/') + cd "$objects" || exit 1 + find -name \*.o | \ + while read file + do + cp --parents "$file" "$pkgdir/bits" + "$srcdir/debian/scripts/fix-filenames" "$pkgdir/bits/$file" "$gcc_variant1" + "$srcdir/debian/scripts/fix-filenames" "$pkgdir/bits/$file" "$gcc_variant2" + done +) # Install the support files we need. cp "$srcdir/scripts/module-common.lds" "$pkgdir/bits/scripts" @@ -45,9 +60,22 @@ grep /usr/bin/ld.bfd "$log" | sed -e "s@$build/@@g" \ elif [ "$sign" = "--lrm" ]; then # We are in LRM build the package a copy in any signatures we can # find for them. These will be added after linking. + base="/usr/lib/linux/$abi_flavour" + + # Check the GCC version we are using against that used in the kernel + # NOTE: that we treat this as only a warning, as if the binaries did come + # out differently then we will actually + echo "II: checking gcc version ..." + cat "$base/compiler" + gcc --version + gcc_was=$(cat "$base/compiler" | sed -e 's/^GCC:/gcc/') + gcc_is=$(gcc --version | head -1) + if [ "$gcc_was" != "$gcc_is" ]; then + echo "WW: gcc version missmatch between linux and linux-restricted-modules" + echo "WW: was: $gcc_was is: $gcc_is" + fi # Apply any local signatures. - base="/usr/lib/linux/$abi_flavour" echo "II: adding signatures from $base ..." cp "$base/signatures/$package/"*".ko.sig" "$pkgdir/bits" sha256sum -c "$base/signatures/$package/SHA256SUMS" || exit 1 diff --git a/debian/scripts/fix-filenames.c b/debian/scripts/fix-filenames.c index e58dc72f8f94..9bd144a6efa8 100644 --- a/debian/scripts/fix-filenames.c +++ b/debian/scripts/fix-filenames.c @@ -48,7 +48,6 @@ main(int argc, char *argv[]) exit(1); } size = in_info.st_size; - printf("%s %ld bytes\n", in_name + prefix_len + 1, size); in = mmap((void *)0, size, PROT_READ|PROT_WRITE, MAP_SHARED, in_fd, (off_t)0); if (!in) { @@ -61,13 +60,21 @@ main(int argc, char *argv[]) continue; if (strncmp(in, prefix, prefix_len) != 0) continue; - length = strlen(in + prefix_len + 1) + 1; + /* In the case of an exact match there there is nothing to move. */ + if (in[prefix_len] == '\0') + length = 0; + /* If this is a filename, strip the leading slash. */ + else if (in[prefix_len] == '/') + length = strlen(in + prefix_len + 1) + 1; + /* Otherwise just keep the suffix. */ + else + length = strlen(in + prefix_len) + 1; /* * Copy the suffix portion down to the start and clear * the remainder of the space to 0. */ memmove(in, in + prefix_len + 1, length); - memset(in + length, '_', prefix_len); + memset(in + length, '\0', prefix_len); } }
We have two forms of dkms-build* in use in our package in various series. These have different APIs, from 18.10 we add an additional parameter to allow handling of debug packages. Having two incompatible APIs for dkms-build significantly complicates the application of fixes for this scripting, and thus complicates maintenance of both the kernel and LRM. Switch 18.04 to the new debug aware API. We also have some additional functionality for handling debug packages for DKMS builds. In later series we use this to supply ZFS debug packages (though this patch set does not enable those). Finally we have two different versions of fix-filenames which the newer of which contains an alignment fix and a cleaner pad character. Also sync this while we are at it. Contains the bulk of the following commits: UBUNTU: [Debian]: Remove hardcoded $(pkgdir) in debug symbols handling UBUNTU: [Debian]: Handle debug symbols for modules in extras too UBUNTU: [Debian]: Check/link modules with debug symbols after DKMS modules UBUNTU: [Debian]: Warn about modules without debug symbols UBUNTU: [Debian]: dkms-build: new parameter for debug package directory UBUNTU: [Debian]: dkms-build: Move zfs special-casing into configure script UBUNTU: [Debian]: dkms-build: Avoid executing post-processor scripts twice UBUNTU: [Packaging] dkms-build -- remove retpoline data from final binary packages UBUNTU: [Packaging] dkms-build--nvidia* -- check gcc version against primary build UBUNTU: [Packaging] fix-filenames -- handle exact string removal UBUNTU: [Packaging] dkms-build--nvidia-N -- remove GCC versions BugLink: https://bugs.launchpad.net/bugs/1917034 Signed-off-by: Andy Whitcroft <apw@canonical.com> --- debian/rules.d/2-binary-arch.mk | 49 +++++++++++++---------- debian/scripts/dkms-build | 60 +++++++++++++++++++++++++++-- debian/scripts/dkms-build--nvidia-N | 38 +++++++++++++++--- debian/scripts/fix-filenames.c | 13 +++++-- 4 files changed, 129 insertions(+), 31 deletions(-)