From patchwork Fri Feb 26 16:33:18 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andy Whitcroft X-Patchwork-Id: 1445002 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (no SPF record) smtp.mailfrom=lists.ubuntu.com (client-ip=91.189.94.19; helo=huckleberry.canonical.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=) Received: from huckleberry.canonical.com (huckleberry.canonical.com [91.189.94.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4DnFZv69w9z9sTD; Sat, 27 Feb 2021 03:33:27 +1100 (AEDT) Received: from localhost ([127.0.0.1] helo=huckleberry.canonical.com) by huckleberry.canonical.com with esmtp (Exim 4.86_2) (envelope-from ) id 1lFg3P-0006Lr-AI; Fri, 26 Feb 2021 16:33:23 +0000 Received: from youngberry.canonical.com ([91.189.89.112]) by huckleberry.canonical.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1lFg3N-0006Ll-GQ for kernel-team@lists.ubuntu.com; Fri, 26 Feb 2021 16:33:21 +0000 Received: from mail-wr1-f72.google.com ([209.85.221.72]) by youngberry.canonical.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1lFg3N-0005us-8c for kernel-team@lists.ubuntu.com; Fri, 26 Feb 2021 16:33:21 +0000 Received: by mail-wr1-f72.google.com with SMTP id x9so1272023wro.9 for ; Fri, 26 Feb 2021 08:33:21 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=RPmaPdFcBs9djqlPi5vif7rLj7knIJm/m2stIo70MIU=; b=MXAG5D4Bd9dA4fwkDCcVJdX2vAA42MvYjvArQTMbP1BGmv6pPhOTwpdMUb5FMCBxLt mH0OFN/g/wSu/q3JOYxwPyQIybXCzTABY3NfzPAyYuJymYJ9WKBWQzRqKIbt2+DoQa1D Qim0QdSrbnS2DHuU4ZPNdju5XNXJ7erBweSMjpZC5GFtduUh+AbYfyaQqhlPrQ5T8RBS BZseRzDgG/QnlhLPo5aEUrEr43ZFrf7+/BIHra/MQwmHligz7utKh69LDhqBV8RfiryB X0li5UHxupfSX3F56yxJ1BxakVVFiSVgglx3oPkgf/uKdV23JmG1W6frxXcDJkdpQfW6 jxLg== X-Gm-Message-State: AOAM533kQl7I4wZWdwAlaOPXeHdp5+efpgacvHZRXTnc4F+tP+x6kT29 38KnhVSjwx6VjBbky7MX6MQK2AdluAR/XpsAhv+z6D93oghMqKfN4rOtHGxdsMT4FgEWx+BZB3N t+hgla8AFsQO5ahxh/jnTzFfLpFRqlEHWNupfbh7p1w== X-Received: by 2002:a5d:6686:: with SMTP id l6mr3976246wru.417.1614357200099; Fri, 26 Feb 2021 08:33:20 -0800 (PST) X-Google-Smtp-Source: ABdhPJwgXim4gPsNQRBdbR3X7v3ikttNM9guQ41yoCk1tf8rqrKPsQ+4eF3GDB8sMduyQR23BBlWZA== X-Received: by 2002:a5d:6686:: with SMTP id l6mr3976215wru.417.1614357199725; Fri, 26 Feb 2021 08:33:19 -0800 (PST) Received: from localhost ([2001:470:6973:2:3ae8:b49d:2db1:89da]) by smtp.gmail.com with ESMTPSA id b8sm5573551wrq.74.2021.02.26.08.33.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 26 Feb 2021 08:33:19 -0800 (PST) From: Andy Whitcroft To: kernel-team@lists.ubuntu.com Subject: [SRU bionic:linux] [1/1] UBUNTU: [Packaging] sync dkms-build to updated API Date: Fri, 26 Feb 2021 16:33:18 +0000 Message-Id: <20210226163318.3215416-1-apw@canonical.com> X-Mailer: git-send-email 2.29.2 MIME-Version: 1.0 X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Andy Whitcroft Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" We have two forms of dkms-build* in use in our package in various series. These have different APIs, from 18.10 we add an additional parameter to allow handling of debug packages. Having two incompatible APIs for dkms-build significantly complicates the application of fixes for this scripting, and thus complicates maintenance of both the kernel and LRM. Switch 18.04 to the new debug aware API. We also have some additional functionality for handling debug packages for DKMS builds. In later series we use this to supply ZFS debug packages (though this patch set does not enable those). Finally we have two different versions of fix-filenames which the newer of which contains an alignment fix and a cleaner pad character. Also sync this while we are at it. Contains the bulk of the following commits: UBUNTU: [Debian]: Remove hardcoded $(pkgdir) in debug symbols handling UBUNTU: [Debian]: Handle debug symbols for modules in extras too UBUNTU: [Debian]: Check/link modules with debug symbols after DKMS modules UBUNTU: [Debian]: Warn about modules without debug symbols UBUNTU: [Debian]: dkms-build: new parameter for debug package directory UBUNTU: [Debian]: dkms-build: Move zfs special-casing into configure script UBUNTU: [Debian]: dkms-build: Avoid executing post-processor scripts twice UBUNTU: [Packaging] dkms-build -- remove retpoline data from final binary packages UBUNTU: [Packaging] dkms-build--nvidia* -- check gcc version against primary build UBUNTU: [Packaging] fix-filenames -- handle exact string removal UBUNTU: [Packaging] dkms-build--nvidia-N -- remove GCC versions BugLink: https://bugs.launchpad.net/bugs/1917034 Signed-off-by: Andy Whitcroft Acked-by: Kelsey Skunberg Acked-by: Stefan Bader --- debian/rules.d/2-binary-arch.mk | 49 +++++++++++++---------- debian/scripts/dkms-build | 60 +++++++++++++++++++++++++++-- debian/scripts/dkms-build--nvidia-N | 38 +++++++++++++++--- debian/scripts/fix-filenames.c | 13 +++++-- 4 files changed, 129 insertions(+), 31 deletions(-) diff --git a/debian/rules.d/2-binary-arch.mk b/debian/rules.d/2-binary-arch.mk index a8a1b9f593b0..db990d5bee3e 100644 --- a/debian/rules.d/2-binary-arch.mk +++ b/debian/rules.d/2-binary-arch.mk @@ -60,13 +60,13 @@ define build_dkms_sign = ) endef define build_dkms = - $(SHELL) $(DROOT)/scripts/dkms-build $(dkms_dir) $(abi_release)-$* '$(call build_dkms_sign,$(builddir)/build-$*)' $(1) $(2) $(3) $(4) + CROSS_COMPILE=$(CROSS_COMPILE) $(SHELL) $(DROOT)/scripts/dkms-build $(dkms_dir) $(abi_release)-$* '$(call build_dkms_sign,$(builddir)/build-$*)' $(1) $(2) $(3) $(4) $(5) endef # nvidia_build_payload 450 450 450_450.102.04-0ubuntu0.20.04.1 # nvidia_build_payload 450-server 450srv 50.102.04-0ubuntu0.20.04.1 define nvidia_build_payload = - $(call build_dkms, $(bldinfo_pkg_name)-$*, $(pkgdir_bldinfo)/usr/lib/linux/$(abi_release)-$*/signatures, nvidia-$(2), pool/restricted/n/nvidia-graphics-drivers-$(1)/nvidia-kernel-source-$(1)_$(3)_$(arch).deb pool/restricted/n/nvidia-graphics-drivers-$(1)/nvidia-dkms-$(1)_$(3)_$(arch).deb) + $(call build_dkms, $(bldinfo_pkg_name)-$*, $(pkgdir_bldinfo)/usr/lib/linux/$(abi_release)-$*/signatures, "", nvidia-$(2), pool/restricted/n/nvidia-graphics-drivers-$(1)/nvidia-kernel-source-$(1)_$(3)_$(arch).deb pool/restricted/n/nvidia-graphics-drivers-$(1)/nvidia-dkms-$(1)_$(3)_$(arch).deb) endef # nvidia_build 450 # nvidia_build 450-server @@ -283,21 +283,7 @@ ifneq ($(skipdbg),true) $(dbgpkgdir)/usr/lib/debug/boot/vmlinux-$(abi_release)-$* $(build_cd) $(kmake) $(build_O) modules_install $(vdso) \ INSTALL_MOD_PATH=$(dbgpkgdir)/usr/lib/debug - # Add .gnu_debuglink sections to each stripped .ko - # pointing to unstripped verson - find $(pkgdir) -name '*.ko' | sed 's|$(pkgdir)||'| while read module ; do \ - if [[ -f "$(dbgpkgdir)/usr/lib/debug/$$module" ]] ; then \ - $(CROSS_COMPILE)objcopy \ - --add-gnu-debuglink=$(dbgpkgdir)/usr/lib/debug/$$module \ - $(pkgdir)/$$module; \ - if grep -q CONFIG_MODULE_SIG=y $(builddir)/build-$*/.config; then \ - $(builddir)/build-$*/scripts/sign-file $(MODHASHALGO) \ - $(MODSECKEY) \ - $(MODPUBKEY) \ - $(pkgdir)/$$module; \ - fi; \ - fi; \ - done + # Add .gnu_debuglink sections only after all/DKMS modules are built. rm -f $(dbgpkgdir)/usr/lib/debug/lib/modules/$(abi_release)-$*/build rm -f $(dbgpkgdir)/usr/lib/debug/lib/modules/$(abi_release)-$*/source rm -f $(dbgpkgdir)/usr/lib/debug/lib/modules/$(abi_release)-$*/modules.* @@ -410,10 +396,10 @@ endif install -d $(dkms_dir) $(dkms_dir)/headers $(dkms_dir)/build $(dkms_dir)/source cp -rp "$(hdrdir)" "$(indep_hdrdir)" "$(dkms_dir)/headers" - $(if $(filter true,$(enable_zfs)),$(call build_dkms, $(mods_pkg_name)-$*, $(pkgdir)/lib/modules/$(abi_release)-$*/kernel, spl, pool/universe/s/spl-linux/spl-dkms_$(dkms_spl_linux_version)_all.deb)) - $(if $(filter true,$(enable_zfs)),$(call build_dkms, $(mods_pkg_name)-$*, $(pkgdir)/lib/modules/$(abi_release)-$*/kernel, zfs, pool/universe/z/zfs-linux/zfs-dkms_$(dkms_zfs_linux_version)_all.deb)) + $(if $(filter true,$(enable_zfs)),$(call build_dkms, $(mods_pkg_name)-$*, $(pkgdir)/lib/modules/$(abi_release)-$*/kernel, "", spl, pool/universe/s/spl-linux/spl-dkms_$(dkms_spl_linux_version)_all.deb)) + $(if $(filter true,$(enable_zfs)),$(call build_dkms, $(mods_pkg_name)-$*, $(pkgdir)/lib/modules/$(abi_release)-$*/kernel, "", zfs, pool/universe/z/zfs-linux/zfs-dkms_$(dkms_zfs_linux_version)_all.deb)) - $(if $(filter true,$(do_dkms_wireguard)),$(call build_dkms, $(mods_pkg_name)-$*, $(pkgdir)/lib/modules/$(abi_release)-$*/kernel, wireguard, pool/universe/w/wireguard-linux-compat/wireguard-dkms_$(dkms_wireguard_version)_all.deb)) + $(if $(filter true,$(do_dkms_wireguard)),$(call build_dkms, $(mods_pkg_name)-$*, $(pkgdir)/lib/modules/$(abi_release)-$*/kernel, "", wireguard, pool/universe/w/wireguard-linux-compat/wireguard-dkms_$(dkms_wireguard_version)_all.deb)) ifeq ($(do_dkms_nvidia),true) $(foreach series,$(nvidia_desktop_series),$(call nvidia_build,$(series))) @@ -422,6 +408,29 @@ ifeq ($(do_dkms_nvidia_server),true) $(foreach series,$(nvidia_server_series),$(call nvidia_build,$(series))) endif +ifneq ($(skipdbg),true) + # Add .gnu_debuglink sections to each stripped .ko + # pointing to unstripped verson + find $(pkgdir) \ + $(if $(filter true,$(do_extras_package)),$(pkgdir_ex)) \ + -name '*.ko' | while read path_module ; do \ + module="/lib/modules/$${path_module#*/lib/modules/}"; \ + if [[ -f "$(dbgpkgdir)/usr/lib/debug/$$module" ]] ; then \ + $(CROSS_COMPILE)objcopy \ + --add-gnu-debuglink=$(dbgpkgdir)/usr/lib/debug/$$module \ + $$path_module; \ + if grep -q CONFIG_MODULE_SIG=y $(builddir)/build-$*/.config; then \ + $(builddir)/build-$*/scripts/sign-file $(MODHASHALGO) \ + $(MODSECKEY) \ + $(MODPUBKEY) \ + $$path_module; \ + fi; \ + else \ + echo "WARNING: Missing debug symbols for module '$$module'."; \ + fi; \ + done +endif + # Build the final ABI information. install -d $(abidir) sed -e 's/^\(.\+\)[[:space:]]\+\(.\+\)[[:space:]]\(.\+\)$$/\3 \2 \1/' \ diff --git a/debian/scripts/dkms-build b/debian/scripts/dkms-build index 5e5cef9bd3ec..c1269c01953a 100755 --- a/debian/scripts/dkms-build +++ b/debian/scripts/dkms-build @@ -6,8 +6,9 @@ abi_flavour="$2" sign="$3" pkgname="$4" pkgdir="$5" -package="$6" -shift 6 +dbgpkgdir="$6" +package="$7" +shift 7 here=$(dirname "$(readlink -f "${0}")") @@ -104,6 +105,30 @@ done # package is installed directly, but not so much if we build it out # of the normal location. sed -i -e '/^PACKAGE_NAME=/ s/#//g' "$dkms_conf" + +# Run any dkms-package specfic configuration steps +dkms_config_specific="$srcdir/$0-configure--$package" +dkms_config_generic=$(echo "$dkms_config_specific" | sed -e 's/-[0-9][0-9]*$/-N/') +for dkms_config in "$dkms_config_specific" "$dkms_config_generic" +do + if [ -z "$dkms_config" -o ! -e "$dkms_config" ]; then + continue + fi + echo "II: dkms-build-configure $(basename "$dkms_config") found, executing" + $SHELL "$dkms_config" \ + "$srcdir" \ + "$dkms_conf" \ + "$dkms_dir" \ + "$abi_flavour" \ + "$sign" \ + "$pkgname" \ + "$pkgdir" \ + "$dbgpkgdir" \ + "$package" \ + "$@" || exit 1 + break +done + cat - <<'EOF' >>"$dkms_conf" POST_BUILD="ubuntu-save-objects ${dkms_tree}/${PACKAGE_NAME}/${PACKAGE_VERSION}/build ${dkms_tree}/${PACKAGE_NAME}/${PACKAGE_VERSION}/objects $POST_BUILD" EOF @@ -150,6 +175,34 @@ sed -e "s@$dkms_dir@<>@g" <"$log" # If this build failed then exit here. [ "$rc" != 0 ] && exit "$rc" +# Install the modules with debug symbols we possibly built, +# and strip the original modules for the next install step. +if [ -n "$dbgpkgdir" ]; then + dbgpkgdir="$dbgpkgdir/$package" + echo "II: dkms-build installing $package into $dbgpkgdir (debug symbols)" + install -d "$dbgpkgdir" + find "$dkms_dir/build/$dkms_package/$dkms_version/$abi_version" -name \*.ko | + while read module; do + vmodule=$( basename "$module" ) + + # Check for '.debug_info' section in order to copy module. + # Useful if debug symbols are requested but not built for + # any reason (including not yet supported by DKMS package). + # Strip module just in case even if section isn't present. + if ${CROSS_COMPILE}objdump -h -j '.debug_info' "$module" >/dev/null 2>&1 + then + echo "copying $vmodule" + cp "$module" "$dbgpkgdir" + else + echo "ignoring $vmodule (missing debug symbols)" + fi + + # Just 'strip -g' as '/usr/sbin/dkms' does. + echo "stripping $vmodule" + strip -g "$module" + done +fi + # Install and optionally sign the modules we have built. pkgdir="$pkgdir/$package" echo "II: dkms-build installing $package into $pkgdir" @@ -183,7 +236,7 @@ dkms_build_specific="$srcdir/$0--$package" dkms_build_generic=$(echo "$dkms_build_specific" | sed -n -e 's/-[0-9][0-9]*[a-z]*$/-N/p') for dkms_build in "$dkms_build_specific" "$dkms_build_generic" do - if [ ! -e "$dkms_build" ]; then + if [ -z "$dkms_build" -o ! -e "$dkms_build" ]; then continue fi echo "II: dkms-build override $(basename "$dkms_build") found, executing" @@ -196,6 +249,7 @@ do "$sign" \ "$pkgname" \ "$pkgdir" \ + "$dbgpkgdir" \ "$package" \ "$@" || exit 1 break diff --git a/debian/scripts/dkms-build--nvidia-N b/debian/scripts/dkms-build--nvidia-N index 3759232549f1..bdec78233b43 100755 --- a/debian/scripts/dkms-build--nvidia-N +++ b/debian/scripts/dkms-build--nvidia-N @@ -10,14 +10,29 @@ abi_flavour="$2" sign="$3" pkgname="$4" pkgdir="$5" -package="$6" -shift 6 +dbgpkgdir="$6" +package="$7" +shift 7 build="$( dirname "$objects" )/build" -# Copy over the objects ready for reconstruction. +# Copy over the objects ready for reconstruction. The objects copy contains +# the *.o files and the *.o-ur* retpoline files to allow the kernel to track +# any retpoline sequences therein. For our purposes we only want the *.o +# files, elide the rest. mkdir -p "$pkgdir/bits/scripts" -cp -rp "$objects"/* "$pkgdir/bits" +( + gcc_variant1=$(gcc --version | head -1 | sed -e 's/^gcc/GCC:/') + gcc_variant2=$(gcc --version | head -1 | sed -e 's/^\(gcc\) \((.*)\) \(.*\)$/\1 version \3 \2/') + cd "$objects" || exit 1 + find -name \*.o | \ + while read file + do + cp --parents "$file" "$pkgdir/bits" + "$srcdir/debian/scripts/fix-filenames" "$pkgdir/bits/$file" "$gcc_variant1" + "$srcdir/debian/scripts/fix-filenames" "$pkgdir/bits/$file" "$gcc_variant2" + done +) # Install the support files we need. cp "$srcdir/scripts/module-common.lds" "$pkgdir/bits/scripts" @@ -45,9 +60,22 @@ grep /usr/bin/ld.bfd "$log" | sed -e "s@$build/@@g" \ elif [ "$sign" = "--lrm" ]; then # We are in LRM build the package a copy in any signatures we can # find for them. These will be added after linking. + base="/usr/lib/linux/$abi_flavour" + + # Check the GCC version we are using against that used in the kernel + # NOTE: that we treat this as only a warning, as if the binaries did come + # out differently then we will actually + echo "II: checking gcc version ..." + cat "$base/compiler" + gcc --version + gcc_was=$(cat "$base/compiler" | sed -e 's/^GCC:/gcc/') + gcc_is=$(gcc --version | head -1) + if [ "$gcc_was" != "$gcc_is" ]; then + echo "WW: gcc version missmatch between linux and linux-restricted-modules" + echo "WW: was: $gcc_was is: $gcc_is" + fi # Apply any local signatures. - base="/usr/lib/linux/$abi_flavour" echo "II: adding signatures from $base ..." cp "$base/signatures/$package/"*".ko.sig" "$pkgdir/bits" sha256sum -c "$base/signatures/$package/SHA256SUMS" || exit 1 diff --git a/debian/scripts/fix-filenames.c b/debian/scripts/fix-filenames.c index e58dc72f8f94..9bd144a6efa8 100644 --- a/debian/scripts/fix-filenames.c +++ b/debian/scripts/fix-filenames.c @@ -48,7 +48,6 @@ main(int argc, char *argv[]) exit(1); } size = in_info.st_size; - printf("%s %ld bytes\n", in_name + prefix_len + 1, size); in = mmap((void *)0, size, PROT_READ|PROT_WRITE, MAP_SHARED, in_fd, (off_t)0); if (!in) { @@ -61,13 +60,21 @@ main(int argc, char *argv[]) continue; if (strncmp(in, prefix, prefix_len) != 0) continue; - length = strlen(in + prefix_len + 1) + 1; + /* In the case of an exact match there there is nothing to move. */ + if (in[prefix_len] == '\0') + length = 0; + /* If this is a filename, strip the leading slash. */ + else if (in[prefix_len] == '/') + length = strlen(in + prefix_len + 1) + 1; + /* Otherwise just keep the suffix. */ + else + length = strlen(in + prefix_len) + 1; /* * Copy the suffix portion down to the start and clear * the remainder of the space to 0. */ memmove(in, in + prefix_len + 1, length); - memset(in + length, '_', prefix_len); + memset(in + length, '\0', prefix_len); } }