From patchwork Wed May 25 11:23:04 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dimitri John Ledkov X-Patchwork-Id: 1635388 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=canonical.com header.i=@canonical.com header.a=rsa-sha256 header.s=20210705 header.b=RgpvDn90; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ubuntu.com (client-ip=91.189.94.19; helo=huckleberry.canonical.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=) Received: from huckleberry.canonical.com (huckleberry.canonical.com [91.189.94.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4L7TG54p62z9sGS for ; Wed, 25 May 2022 21:23:25 +1000 (AEST) Received: from localhost ([127.0.0.1] helo=huckleberry.canonical.com) by huckleberry.canonical.com with esmtp (Exim 4.86_2) (envelope-from ) id 1ntp6m-0006n9-Kw; Wed, 25 May 2022 11:23:20 +0000 Received: from smtp-relay-internal-0.internal ([10.131.114.225] helo=smtp-relay-internal-0.canonical.com) by huckleberry.canonical.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1ntp6j-0006m1-LC for kernel-team@lists.ubuntu.com; Wed, 25 May 2022 11:23:17 +0000 Received: from mail-wm1-f72.google.com (mail-wm1-f72.google.com [209.85.128.72]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by smtp-relay-internal-0.canonical.com (Postfix) with ESMTPS id 63A693F19E for ; Wed, 25 May 2022 11:23:17 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=canonical.com; s=20210705; t=1653477797; bh=9cYA2pz7eRUnNdMFckN2zxxi6Vt2DqfHLxbC81In1+4=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=RgpvDn90F2APSL17am97p32hrE7/f/NMy1qPHYJRRxwhOpjrIi+7MUIstdyyIjnWf TC1NTkHHD6qtB68xkkW03Vtk/AQ+n/CeIWbzpVzxkS1EakDyTmDNBkQco6umBLR04q K9repBCrCbG9d10rr6IGv3mAxGw/QZECu2pnzXZO6eOsf3P+DngaJy04imjHbnY6ay fr8jPfH7RwQR9VjQpSFOY2hE/RdDJ8gw+72nZvOouEZYf9pmUAVRcvnK7BWSDzPzuV Vq+zJ+OgTwahYGVh4yC+dxd/oYXxTSiB6ykUmzWu6sYK2RhFW5T4AavYS4Fyp0y/WY j01my5hIXZ1Pg== Received: by mail-wm1-f72.google.com with SMTP id i1-20020a05600c354100b003976fc71579so680240wmq.8 for ; Wed, 25 May 2022 04:23:17 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=9cYA2pz7eRUnNdMFckN2zxxi6Vt2DqfHLxbC81In1+4=; b=ySVsOfJVu+pqIXzhxUNkM4mQ9WwDFnmo2mhWpdzYPIWhJtdir7wW47okRATjHALw42 blVKIoniRAU/L/YF6NwTsdvhW9P6syIgSXaArgk10GTTHJ0J/0VDbwZtw838XYZaBkyN Qm5I1X2rR7DQLP4hwKbPllwtzoySAcnXTQJ8Lq5hlq1nRBT0jV0+2DeKopdrQlIU3pGs fSvipDdPzCAdKJTavHkN4yH3uRU5wsDaeHMk2jGdelbVDMAisCOc/kDmgAQEvxRwxGZc P/m0HewR9urK/KZ7Vp2as0gDXuKRTNMDWCOXI4no7iVMpWDWHWS8wld4ZIjwVeUNxSaD VHwg== X-Gm-Message-State: AOAM530gQMQzycaYUK7GJfwb1kEwdAea/ENQwhF1VWHUYopu0jOHGbE7 nEYQD9oNeskB8OlZRmP4KaJM2GlxAg1S5IEWQY9uJvbkdm40COk3eIArwDMTU2wZb/xnNp7rR2O pqXqE9j2CgtGgIw3zeSVD/h3WzgpiRs0qfUeAYmHqlw== X-Received: by 2002:a1c:4342:0:b0:397:5c97:7147 with SMTP id q63-20020a1c4342000000b003975c977147mr6781709wma.83.1653477796481; Wed, 25 May 2022 04:23:16 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxqy/eAbVsRv7JP1wVKBE4iT8/Sg0ATLOjI1vw6cRFwhtxp71sW08eAxyL1jWhfypdPVfCGGA== X-Received: by 2002:a1c:4342:0:b0:397:5c97:7147 with SMTP id q63-20020a1c4342000000b003975c977147mr6781682wma.83.1653477796095; Wed, 25 May 2022 04:23:16 -0700 (PDT) Received: from localhost ([2a01:4b00:85fd:d700:d9f3:d3bd:637:900c]) by smtp.gmail.com with ESMTPSA id q21-20020a1ce915000000b0039768532c79sm1863910wmc.6.2022.05.25.04.23.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 25 May 2022 04:23:15 -0700 (PDT) From: Dimitri John Ledkov To: kernel-team@lists.ubuntu.com Subject: [SRU][JAMMY][PATCH v5 1/2] UBUNTU: [Packaging] support standalone dkms module builds Date: Wed, 25 May 2022 12:23:04 +0100 Message-Id: <20220525112305.39148-2-dimitri.ledkov@canonical.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220525112305.39148-1-dimitri.ledkov@canonical.com> References: <20220525112305.39148-1-dimitri.ledkov@canonical.com> MIME-Version: 1.0 X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" From: "You-Sheng Yang (vicamo)" BugLink: https://bugs.launchpad.net/bugs/1969434 Signed-off-by: You-Sheng Yang (vicamo) Signed-off-by: Dimitri John Ledkov --- debian/control.d/flavour-module.stub | 19 +++++++++++ debian/rules | 25 +++++++-------- debian/rules.d/0-common-vars.mk | 47 ++++++++++++++++++++++++++++ debian/rules.d/2-binary-arch.mk | 36 ++++++++++++++++----- debian/scripts/control-create | 27 ++++++++++++++++ 5 files changed, 133 insertions(+), 21 deletions(-) create mode 100644 debian/control.d/flavour-module.stub diff --git a/debian/control.d/flavour-module.stub b/debian/control.d/flavour-module.stub new file mode 100644 index 0000000000..2810f83bb3 --- /dev/null +++ b/debian/control.d/flavour-module.stub @@ -0,0 +1,19 @@ + +Package: linux-modules-MODULE-PKGVER-ABINUM-FLAVOUR +Build-Profiles: +Architecture: ARCH +Section: kernel +Priority: optional +Depends: + ${misc:Depends}, + linux-image-PKGVER-ABINUM-FLAVOUR | linux-image-unsigned-PKGVER-ABINUM-FLAVOUR, +Built-Using: ${linux:BuiltUsing} +Description: Linux kernel MODULE modules for version PKGVER-ABINUM + This package provides the Linux kernel MODULE modules for version + PKGVER-ABINUM. + . + You likely do not want to install this package directly. Instead, install the + one of the linux-modules-MODULE-FLAVOUR* meta-packages, + which will ensure that upgrades work correctly, and that supporting packages are + also installed. + diff --git a/debian/rules b/debian/rules index eca69cdc98..540e7a3cfc 100755 --- a/debian/rules +++ b/debian/rules @@ -40,6 +40,11 @@ do_tools_host?=false do_tools_perf_jvmti?=false do_enforce_all?=false +# Default settings for DKMS modules. +$(foreach _m,$(all_dkms_modules), \ + $(eval do_$(_m) ?= $(if $(filter $(_m),$(filter-out $(dkms_exclude),$(dkms_include) $(subst any,$(_m),$(subst $(arch),$(_m),$(dkms_$(_m)_archs))))),true,false)) \ +) + # Don't build tools in a cross compile environment. ifneq ($(DEB_HOST_ARCH),$(DEB_BUILD_ARCH)) do_tools=false @@ -74,8 +79,7 @@ endif # - disable dkms builds as the versions used may have been deleted ifneq ($(filter autopkgtest,$(DEB_BUILD_PROFILES)),) flavours := $(firstword $(flavours)) - do_zfs=false - do_v4l2loopback=false + $(foreach _m,$(all_dkms_modules),$(eval do_$(_m) = false)) endif # Being used to build a mainline build -- turn off things which do not work. @@ -97,13 +101,11 @@ ifneq ($(do_tools),true) do_tools_host= endif -ifeq ($(do_zfs),false) - do_zfs_disable:=$(shell for m in $$(cat $(DROOT)/zfs-modules.ignore); do grep -qxF $$m $(prev_abidir)/../modules.ignore 2>/dev/null || echo $$m >> $(prev_abidir)/../modules.ignore; done) -endif - -ifeq ($(do_v4l2loopback),false) - do_v4l2loopback_disable:=$(shell for m in $$(cat $(DROOT)/v4l2loopback-modules.ignore); do grep -qxF $$m $(prev_abidir)/../modules.ignore 2>/dev/null || echo $$m >> $(prev_abidir)/../modules.ignore; done) -endif +$(foreach _m,$(all_built-in_dkms_modules), \ + $(if $(filter true,$(do_$(_m))),, \ + $(eval do_$(_m)_disable := $$(shell for m in $$$$(cat $(DROOT)/$(_m)-modules.ignore); do grep -qxF $$$$m $(prev_abidir)/../modules.ignore 2>/dev/null || echo $$$$m >> $(prev_abidir)/../modules.ignore; done)) \ + ) \ +) ifeq ($(do_dkms_wireguard),false) do_wireguard_disable:=$(shell for m in $$(cat $(DROOT)/wireguard-modules.ignore); do grep -qxF $$m $(prev_abidir)/../modules.ignore 2>/dev/null || echo $$m >> $(prev_abidir)/../modules.ignore; done) @@ -112,10 +114,6 @@ endif # Either tools package needs the common source preparation do_any_tools=$(sort $(filter-out false,$(do_linux_tools) $(do_cloud_tools))) -# Versions of dkms packages. -dkms_zfs_linux_version=$(shell gawk '/^zfs-linux / { print $$2; }' debian/dkms-versions) -dkms_v4l2loopback_version=$(shell gawk '/^v4l2loopback / { print $$2; }' debian/dkms-versions) - # NVIDIA DKMS package gross series split into desktop and server. nvidia_desktop_series=$(shell sed -n -e 's/^nvidia-graphics-drivers-\([0-9][0-9]*\) .*/\1/p' debian/dkms-versions) nvidia_server_series=$(shell sed -n -e 's/^nvidia-graphics-drivers-\([0-9][0-9]*-server\) .*/\1/p' debian/dkms-versions) @@ -182,6 +180,7 @@ endif $(DEBIAN)/control.stub: \ $(DROOT)/scripts/control-create \ $(control_files) \ + $(DROOT)/control.d/flavour-module.stub \ $(DEBIAN)/changelog \ $(wildcard $(DEBIAN)/control.d/* $(DEBIAN)/sub-flavours/*.vars) for i in $(control_files); do \ diff --git a/debian/rules.d/0-common-vars.mk b/debian/rules.d/0-common-vars.mk index 78f8a0f16e..2f27d1710e 100644 --- a/debian/rules.d/0-common-vars.mk +++ b/debian/rules.d/0-common-vars.mk @@ -285,3 +285,50 @@ custom_override = \ # selftests that Ubuntu cares about ubuntu_selftests = breakpoints cpu-hotplug efivarfs memfd memory-hotplug mount net ptrace seccomp timers powerpc user ftrace + +# DKMS +all_dkms_modules = + +subst_paired = $(subst $(firstword $(subst =, ,$(1))),$(lastword $(subst =, ,$(1))),$(2)) +recursive_call = $(if $(2),$(call recursive_call,$(1),$(wordlist 2,$(words $(2)),$(2)),$(call $(1),$(firstword $(2)),$(3))),$(3)) + +$(foreach _line,$(shell gawk '{ OFS = "!"; $$1 = $$1; print }' $(DROOT)/dkms-versions), \ + $(eval _params = $(subst !, ,$(_line))) \ + $(eval _deb_pkgname = $(firstword $(_params))) \ + $(eval _deb_version = $(word 2,$(_params))) \ + $(if $(filter modulename=%,$(_params)), \ + $(eval _m = $(word 2,$(subst =, ,$(filter modulename=%,$(_params))))) \ + , \ + $(info modulename for $(_deb_pkgname) not specified in dkms-versions. Assume $(_deb_pkgname).) \ + $(eval _m = $(_deb_pkgname)) \ + ) \ + $(eval all_dkms_modules += $(_m)) \ + $(eval dkms_$(_m)_version = $(_deb_version)) \ + $(foreach _p,$(patsubst debpath=%,%,$(filter debpath=%,$(_params))), \ + $(eval dkms_$(_m)_debpath += $(strip \ + $(call recursive_call,subst_paired, \ + %module%=$(_m) \ + %package%=$(_deb_pkgname) \ + %version%=$(lastword $(subst :, ,$(_deb_version))) \ + , \ + $(_p) \ + ) \ + )) \ + ) \ + $(if $(dkms_$(_m)_debpath),,$(error debpath for $(_deb_pkgname) not specified.)) \ + $(if $(filter arch=%,$(_params)), \ + $(eval dkms_$(_m)_archs = $(patsubst arch=%,%,$(filter arch=%,$(_params)))) \ + , \ + $(eval dkms_$(_m)_archs = any) \ + ) \ + $(eval dkms_$(_m)_rprovides = $(patsubst rprovides=%,%,$(filter rprovides=%,$(_params)))) \ + $(eval dkms_$(_m)_type = $(word 1,$(patsubst type=%,%,$(filter type=%,$(_params))) built-in)) \ + $(eval all_$(dkms_$(_m)_type)_dkms_modules += $(_m)) \ + $(if $(filter standalone,$(dkms_$(_m)_type)), \ + $(eval dkms_$(_m)_pkg_name = linux-modules-$(_m)-$(abi_release)) \ + $(eval dkms_$(_m)_subdir = ubuntu) \ + , \ + $(eval dkms_$(_m)_pkg_name = $(mods_pkg_name)) \ + $(eval dkms_$(_m)_subdir = kernel) \ + ) \ +) diff --git a/debian/rules.d/2-binary-arch.mk b/debian/rules.d/2-binary-arch.mk index c4906036ce..24a395d310 100644 --- a/debian/rules.d/2-binary-arch.mk +++ b/debian/rules.d/2-binary-arch.mk @@ -117,8 +117,10 @@ $(stampdir)/stamp-install-%: MODSECKEY=$(builddir)/build-$*/certs/signing_key.pe $(stampdir)/stamp-install-%: MODPUBKEY=$(builddir)/build-$*/certs/signing_key.x509 $(stampdir)/stamp-install-%: build_dir=$(builddir)/build-$* $(stampdir)/stamp-install-%: dkms_dir=$(call dkms_dir_prefix,$(builddir)/build-$*) -$(stampdir)/stamp-install-%: enable_zfs = $(call custom_override,do_zfs,$*) -$(stampdir)/stamp-install-%: enable_v4l2loopback = $(call custom_override,do_v4l2loopback,$*) +$(foreach _m,$(all_dkms_modules), \ + $(eval $$(stampdir)/stamp-install-%: enable_$(_m) = $$(filter true,$$(call custom_override,do_$(_m),$$*))) \ + $(eval $$(stampdir)/stamp-install-%: dkms_$(_m)_pkgdir = $$(CURDIR)/debian/$(dkms_$(_m)_pkg_name)-$$*) \ +) $(stampdir)/stamp-install-%: dbgpkgdir_dkms = $(if $(filter true,$(skipdbg)),"",$(dbgpkgdir)/usr/lib/debug/lib/modules/$(abi_release)-$*/kernel) $(stampdir)/stamp-install-%: $(stampdir)/stamp-build-% $(stampdir)/stamp-install-headers @echo Debug: $@ kernel_file $(kernel_file) kernfile $(kernfile) install_file $(install_file) instfile $(instfile) @@ -126,6 +128,9 @@ $(stampdir)/stamp-install-%: $(stampdir)/stamp-build-% $(stampdir)/stamp-install dh_prep -p$(bin_pkg_name)-$* dh_prep -p$(mods_pkg_name)-$* dh_prep -p$(hdrs_pkg_name)-$* + $(foreach _m,$(all_standalone_dkms_modules), \ + $(if $(enable_$(_m)),dh_prep -p$(dkms_$(_m)_pkg_name)-$*;)\ + ) ifneq ($(skipdbg),true) dh_prep -p$(bin_pkg_name)-$*-dbgsym endif @@ -263,6 +268,12 @@ ifeq ($(do_extras_package),true) $(call install_control,$(mods_extra_pkg_name)-$*,extra,postinst postrm); \ fi endif + $(foreach _m,$(all_standalone_dkms_modules), \ + $(if $(enable_$(_m)), \ + install -d $(dkms_$(_m)_pkgdir)/usr/lib/linux/triggers; \ + $(call install_control,$(dkms_$(_m)_pkg_name)-$*,extra,postinst postrm); \ + ) \ + ) # Install the full changelog. ifeq ($(do_doc_package),true) @@ -412,8 +423,11 @@ endif install -d $(dkms_dir) $(dkms_dir)/headers $(dkms_dir)/build $(dkms_dir)/source cp -rp "$(hdrdir)" "$(indep_hdrdir)" "$(dkms_dir)/headers" - $(if $(filter true,$(enable_zfs)),$(call build_dkms, $(mods_pkg_name)-$*, $(pkgdir)/lib/modules/$(abi_release)-$*/kernel, $(dbgpkgdir_dkms), zfs, pool/universe/z/zfs-linux/zfs-dkms_$(dkms_zfs_linux_version)_all.deb)) - $(if $(filter true,$(enable_v4l2loopback)),$(call build_dkms, $(mods_pkg_name)-$*, $(pkgdir)/lib/modules/$(abi_release)-$*/kernel, $(dbgpkgdir_dkms), v4l2loopback, pool/universe/v/v4l2loopback/v4l2loopback-dkms_$(dkms_v4l2loopback_version)_all.deb)) + $(foreach _m,$(all_dkms_modules), \ + $(if $(enable_$(_m)), \ + $(call build_dkms,$(dkms_$(_m)_pkg_name)-$*,$(dkms_$(_m)_pkgdir)/lib/modules/$(abi_release)-$*/$(dkms_$(_m)_subdir),$(dbgpkgdir_dkms),$(_m),$(dkms_$(_m)_debpath)); \ + ) \ + ) ifneq ($(skipdbg),true) @@ -563,7 +577,7 @@ define dh_all dh_shlibdeps -p$(1) $(shlibdeps_opts) dh_installdeb -p$(1) dh_installdebconf -p$(1) - $(lockme) dh_gencontrol -p$(1) -- -Vlinux:rprovides='$(rprovides)' + $(lockme) dh_gencontrol -p$(1) -- -Vlinux:rprovides='$(rprovides)' $(2) dh_md5sums -p$(1) dh_builddeb -p$(1) endef @@ -572,7 +586,7 @@ define newline endef define dh_all_inline - $(subst ${newline},; \${newline},$(call dh_all,$(1))) + $(subst ${newline},; \${newline},$(call dh_all,$(1),$(2))) endef binary-arch-headers: install-arch-headers @@ -597,8 +611,10 @@ binary-%: dbgpkg = $(bin_pkg_name)-$*-dbgsym binary-%: dbgpkgdir = $(CURDIR)/debian/$(bin_pkg_name)-$*-dbgsym binary-%: pkgtools = $(tools_flavour_pkg_name)-$* binary-%: pkgcloud = $(cloud_flavour_pkg_name)-$* -binary-%: rprovides = $(if $(filter true,$(call custom_override,do_zfs,$*)),spl-modules$(comma) spl-dkms$(comma) zfs-modules$(comma) zfs-dkms$(comma)) \ - $(if $(filter true,$(call custom_override,do_v4l2loopback,$*)),v4l2loopback-modules$(comma) v4l2loopback-dkms$(comma)) +$(foreach _m,$(all_dkms_modules), \ + $(eval binary-%: enable_$(_m) = $$(filter true,$$(call custom_override,do_$(_m),$$*))) \ +) +binary-%: rprovides = $(foreach _m,$(all_built-in_dkms_modules),$(if $(enable_$(_m)),$(foreach _r,$(dkms_$(_m)_rprovides),$(_r)$(comma) ))) binary-%: target_flavour = $* binary-%: checks-% @echo Debug: $@ @@ -623,6 +639,10 @@ ifeq ($(do_extras_package),true) endif endif + $(foreach _m,$(all_standalone_dkms_modules), \ + $(if $(enable_$(_m)),$(call dh_all,$(dkms_$(_m)_pkg_name)-$*);)\ + ) + $(call dh_all,$(pkgbldinfo)) $(call dh_all,$(pkghdr)) diff --git a/debian/scripts/control-create b/debian/scripts/control-create index 2ab9e49751..5d0e3e4655 100755 --- a/debian/scripts/control-create +++ b/debian/scripts/control-create @@ -38,3 +38,30 @@ cat $stub | grep -v '^#' | sed \ -e "s#=SIGN-ME-PKG=#$sign_me_pkg#g" \ -e "s#=SIGN-ME-TXT=#$sign_me_txt#g" \ -e "s#=SIGN-PEER-PKG=#$sign_peer_pkg#g" + +while read package version extras +do + module="$package" + module_type= + + # Module arch parameters are skipped here, so a package section will + # be generated for each flavour, and its Architecture will be set to + # all architectures with that flavour. Even that is being generated, + # it doesn't follow all of them will be built. That's to work-around + # dkms_exclude/dkms_include that manipulates supported architectures + # in $(DEBIAN)/rules.d/$(arch).mk. + for param in $extras; do + case "$param" in + modulename=*) module="${param#modulename=}" ;; + type=*) module_type="${param#type=}" ;; + *) continue ;; + esac + done + + [ "$module_type" = "standalone" ] || continue + + cat debian/control.d/flavour-module.stub | grep -v '^#' | sed \ + -e "s#ARCH#$arch#g" \ + -e "s#MODULE#$module#g" \ + -e "s#FLAVOUR#$flavour#g" +done <"debian/dkms-versions"