From patchwork Mon Mar 8 15:00:02 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andy Whitcroft X-Patchwork-Id: 1449152 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (no SPF record) smtp.mailfrom=lists.ubuntu.com (client-ip=91.189.94.19; helo=huckleberry.canonical.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=) Received: from huckleberry.canonical.com (huckleberry.canonical.com [91.189.94.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4DvM2y3Lc0z9sWY; Tue, 9 Mar 2021 02:00:26 +1100 (AEDT) Received: from localhost ([127.0.0.1] helo=huckleberry.canonical.com) by huckleberry.canonical.com with esmtp (Exim 4.86_2) (envelope-from ) id 1lJHMr-00052J-6S; Mon, 08 Mar 2021 15:00:21 +0000 Received: from youngberry.canonical.com ([91.189.89.112]) by huckleberry.canonical.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1lJHMo-0004zL-AB for kernel-team@lists.ubuntu.com; Mon, 08 Mar 2021 15:00:18 +0000 Received: from mail-wm1-f72.google.com ([209.85.128.72]) by youngberry.canonical.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1lJHMo-00064a-1q for kernel-team@lists.ubuntu.com; Mon, 08 Mar 2021 15:00:18 +0000 Received: by mail-wm1-f72.google.com with SMTP id n17so2144542wmi.2 for ; Mon, 08 Mar 2021 07:00:18 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=4SSPu3SHjIHWN3QYmd8vTIGEs4TTboH2x9MJVbQvha0=; b=eFITjD+WoAM2/34qNxKsBfW9sYVe5dIun00sZrxxGUZJ09I7ZAhSNT46U9xevd589c NoqY54d5KoWU7GUF41cNkTnE2krqbnieR6tL/BcMyRrya+qQASRAnwIQD8AdgaLQQ8OS yW1eLtP8aprs0zwQq0EgyKl9iNexhsL6KzLeZL+4QTLVpaPdwSnEbYV3OtP5a8Bmkoae zDeECYV3oqp3vXVDJe+zRjiNOzJBt9lKod3ZRAMKToxVj/WZ7Om6ueC7SCQWwTnG411d kQ9wfmriStKqLV6A7lrajYp1Do8ZJSf6MMMcCiFYywzX82UiMl9dKYb3MEweB8XtlI3B Ejbw== X-Gm-Message-State: AOAM533qW6FUSrKmH8joYWNhIpcipLe4jllV1JwzUAHVW79CgAAZLA+L 0WPcBlGNKuIYtO4rsM0SDZ9jdckQG0vNRjclpZB5UGZsa2jqGlaJvWqsrY1SxVqi92YzprEUaA7 BfWbOxJMh4PcdVwJiM/bF30TNFRT4OaPXiY3TvY24NA== X-Received: by 2002:a05:600c:2053:: with SMTP id p19mr22519786wmg.87.1615215617123; Mon, 08 Mar 2021 07:00:17 -0800 (PST) X-Google-Smtp-Source: ABdhPJzEl4Ym+FkwOxoE7b7Gtmx8hfQtDJeGCrQRgFEXAz7Ry7lG6Ck4uMhOW3iOwk7SyMofvJD5sA== X-Received: by 2002:a05:600c:2053:: with SMTP id p19mr22519758wmg.87.1615215616844; Mon, 08 Mar 2021 07:00:16 -0800 (PST) Received: from localhost ([2001:470:6973:2:2843:61fd:9f2a:3b98]) by smtp.gmail.com with ESMTPSA id h62sm19595988wmf.37.2021.03.08.07.00.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 08 Mar 2021 07:00:16 -0800 (PST) From: Andy Whitcroft To: kernel-team@lists.ubuntu.com Subject: [PATCH 7/9] UBUNTU: [Packaging] linux-restricted-signatures -- publish clean signatures Date: Mon, 8 Mar 2021 15:00:02 +0000 Message-Id: <20210308150004.1746089-8-apw@canonical.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210308150004.1746089-1-apw@canonical.com> References: <20210308150004.1746089-1-apw@canonical.com> MIME-Version: 1.0 X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Andy Whitcroft Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" Consume the signatures produced by the signing service from the signing custom binary upload and express those in a linux-signatures-nvidia-* package. These are split by flavour for simplicity of consumption by linux-restricted-modules; consumed via a simple runtime depedency. BugLink: https://bugs.launchpad.net/bugs/1918134 Signed-off-by: Andy Whitcroft --- debian/rules.lrs | 12 +++ debian/scripts/gen-rules | 1 + debian/scripts/gen-rules.lrs | 100 +++++++++++++++++++ download-signed | 183 +++++++++++++++++++++++++++++++++++ 4 files changed, 296 insertions(+) create mode 100755 debian/rules.lrs create mode 100755 debian/scripts/gen-rules.lrs create mode 100755 download-signed diff --git a/debian/rules.lrs b/debian/rules.lrs new file mode 100755 index 0000000..1c22d8e --- /dev/null +++ b/debian/rules.lrs @@ -0,0 +1,12 @@ +##export DH_VERBOSE := 1 + +arch = $(shell dpkg-architecture -qDEB_HOST_ARCH) + +%: + dh $@ + +override_dh_auto_build: + ./download-signed "$(src_genr_package)" "$(src_genr_version)" "$(src_genr_package)" + +override_dh_auto_install: nvidia-$(arch) + dh_install diff --git a/debian/scripts/gen-rules b/debian/scripts/gen-rules index 8952f4b..4894481 100755 --- a/debian/scripts/gen-rules +++ b/debian/scripts/gen-rules @@ -3,6 +3,7 @@ src_package=$(LC_ALL=C dpkg-parsechangelog -SSource) case "$src_package" in linux-restricted-generate*) pkg='lrg' ;; +linux-restricted-signature*) pkg='lrs' ;; linux-restricted-modules*) pkg='lrm' ;; esac diff --git a/debian/scripts/gen-rules.lrs b/debian/scripts/gen-rules.lrs new file mode 100755 index 0000000..8298b0e --- /dev/null +++ b/debian/scripts/gen-rules.lrs @@ -0,0 +1,100 @@ +#!/bin/bash + +# Pick out relevant version and package information including our predecessor +# packages: linux-restricted-generate -> linux-restricted-signatures -> linux-restricted-modules +src_package=$(LC_ALL=C dpkg-parsechangelog -SSource) +src_version=$(LC_ALL=C dpkg-parsechangelog -SVersion) +src_abi=$(echo "${src_version}" | sed -ne 's/\([0-9]*\.[0-9]*\.[0-9]*\-[0-9]*\)\..*/\1/p') +src_series=$(LC_ALL=C dpkg-parsechangelog -SDistribution | sed -e 's/-\(security\|updates\|proposed\)$//') + +# linux/5.8.0-41.46 +src_main_package=$(echo "${src_package}" | sed -e 's/-restricted-signature//') +src_main_version=$(echo ${src_version} | sed -e 's/+[0-9][0-9\.]*$//') + +# linux-restricted-generate/5.8.0-41.46[+1] +src_genr_package=$(echo "${src_package}" | sed -e 's/-restricted-signatures/-restricted-generate/') +src_genr_version=${src_version} + +# linux-restricted-signatures/5.8.0-41.46[+1] + +# linux-restricted-modules/5.8.0-41.46[+1] + +cat - "debian/rules.lrs" >"debian/rules.gen" <"debian/control.interlock-up" + +build_archs= +while read command flavour archs +do + case "$command" in + build) ;; + *) continue ;; + esac + + for arch in $archs + do + case " $build_archs " in + *\ $arch\ *) ;; + *) build_archs="$buildarchs $arch" ;; + esac + done +done <"debian/package.config" + +while read command flavour archs +do + case "$command" in + build) ;; + *) continue ;; + esac + + targets=$(echo "$archs" | sed -e 's/\>"debian/rules.gen" <>debian/linux-signatures-nvidia-${src_abi}-${flavour}.install; \ + done +EOL + + cat - >>"debian/control.signatures" < +Architecture: ${archs} +Section: kernel +Priority: optional +Built-Using: \${linux:BuiltUsing} +Description: Linux kernel signatures for nvidia modules for version ${src_abi}-${flavour} + This package contains the Linux kernel signatures nvidia modules for + version ${src_abi}-${flavour}. + . + You likely do not want to install this package directly. +EOL +done <"debian/package.config" + +cat "debian/control.common" "debian/control.signatures" | sed \ + -e "/@BUILD-INTERLOCK@/{" \ + -e " r debian/control.interlock-up" \ + -e " d" \ + -e " }" \ + -e "s/@SRCPKGNAME@/${src_package}/g" \ + -e "s/@ABI@/${src_abi}/g" \ + -e "s/@SERIES@/${src_series}/g" \ + >"debian/control" + +rm -f "debian/control.interlock-up" "debian/control.signatures" diff --git a/download-signed b/download-signed new file mode 100755 index 0000000..0793696 --- /dev/null +++ b/download-signed @@ -0,0 +1,183 @@ +#! /usr/bin/python3 + +import hashlib +import argparse +import os +import re +import sys +import tarfile +from urllib import request +from urllib.error import HTTPError +from urllib.parse import ( + urlparse, + urlunparse, + ) + +import apt + +# package_name: package containing the objects we signed +# package_version: package version containing the objects we signed +# src_package: source package name in dists +# signed_type: 'signed' or 'uefi' schema in the url + +parser = argparse.ArgumentParser() +parser.add_argument( + "package_name", + help="package containining the objects we signed") +parser.add_argument( + "package_version", + help="package version containing the objects we signed, or 'current'") +parser.add_argument( + "src_package", + help="source package name in dists") +parser.add_argument( + "signed_type", + nargs='?', + default='signed', + help="subdirectory type in the url, 'signed' or 'uefi'") +args = parser.parse_args() + + +class SignedDownloader: + """Download a block of signed information from dists. + + Find a block of signed information as published in dists/*/signed + and download the contents. Use the contained checksum files to + identify the members and to validate them once downloaded. + """ + + def __init__(self, package_name, package_version, src_package, signed_type='signed'): + self.package_name = package_name + self.package_version = package_version + self.src_package = src_package + + # Find the package in the available archive repositories. Use a _binary_ + # package name and version to locate the appropriate archive. Then use the + # URI there to look for and find the appropriate binary. + cache = apt.Cache() + + self.package = None + if self.package_version == "current": + self.package = cache[package_name].candidate + else: + for version in cache[package_name].versions: + if version.version == self.package_version: + self.package = version + break + + if not self.package: + raise KeyError("{0}: package version not found".format(self.package_name)) + + origin = self.package.origins[0] + pool_parsed = urlparse(self.package.uri) + self.package_dir = "%s/%s/%s/%s-%s/%s/" % ( + origin.archive, 'main', signed_type, + self.src_package, self.package.architecture, self.package_version) + + # Prepare the master url stem and pull out any username/password. If present + # replace the default opener with one which offers that password. + dists_parsed_master = list(pool_parsed) + if '@' in dists_parsed_master[1]: + (username_password, host) = pool_parsed[1].split('@', 1) + (username, password) = username_password.split(':', 1) + + dists_parsed_master[1] = host + + # Work out the authentication domain. + domain_parsed = [ dists_parsed_master[0], dists_parsed_master[1], '/', None, None, None ] + auth_uri = urlunparse(domain_parsed) + + # create a password manager + password_mgr = request.HTTPPasswordMgrWithDefaultRealm() + + # Add the username and password. + # If we knew the realm, we could use it instead of None. + password_mgr.add_password(None, auth_uri, username, password) + + handler = request.HTTPBasicAuthHandler(password_mgr) + + # create "opener" (OpenerDirector instance) + opener = request.build_opener(handler) + + # Now all calls to urllib.request.urlopen use our opener. + request.install_opener(opener) + + self.dists_parsed = dists_parsed_master + + def download_one(self, member, filename, hash_factory=None): + directory = os.path.dirname(filename) + if not os.path.exists(directory): + os.makedirs(directory) + + dists_parsed = list(self.dists_parsed) + dists_parsed[2] = re.sub(r"/pool/.*", "/dists/" + self.package_dir + \ + member, dists_parsed[2]) + dists_uri = urlunparse(dists_parsed) + + print("Downloading %s ... " % dists_uri, end='') + sys.stdout.flush() + try: + with request.urlopen(dists_uri) as dists, open(filename, "wb") as out: + hashobj = None + if hash_factory: + hashobj = hash_factory() + for chunk in iter(lambda: dists.read(256 * 1024), b''): + if hashobj: + hashobj.update(chunk) + out.write(chunk) + checksum = True + if hashobj: + checksum = hashobj.hexdigest() + except HTTPError as e: + if e.code == 404: + print("not found") + else: + raise + else: + print("found") + return checksum + return None + + def download(self, base): + """Download an entire signed result from dists.""" + + # Download the checksums and use that to download the contents. + sums = 'SHA256SUMS' + sums_local = os.path.join(base, self.package_version, sums) + if not self.download_one(sums, sums_local): + print('download-signed: {0}: not found'.format(sums)) + sys.exit(1) + + # Read the checksum file and download the files it mentions. + here = os.path.abspath(base) + with open(sums_local) as sfd: + for line in sfd: + line = line.strip() + (checksum_expected, member) = (line[0:64], line[66:]) + filename = os.path.abspath(os.path.join(base, self.package_version, member)) + if not filename.startswith(here): + print('download-signed: {0}: member outside output directory'.format(member)) + sys.exit(1) + + # Download and checksum this member. + checksum_actual = self.download_one(member, filename, hashlib.sha256) + if checksum_expected != checksum_actual: + print('download-signed: {0}: member checksum invalid'.format(member)) + sys.exit(1) + + # If this is a tarball result then extract it. + here = os.path.abspath(os.path.join(base, self.package_version)) + tarball_filename = os.path.join(base, self.package_version, 'signed.tar.gz') + if os.path.exists(tarball_filename): + with tarfile.open(tarball_filename) as tarball: + for tarinfo in tarball: + if not filename.startswith(here): + print('download-signed: {0}: tarball member outside output directory'.format(member)) + sys.exit(1) + for tarinfo in tarball: + print('Extracting {0} ...'.format(tarinfo.name)) + tarball.extract(tarinfo, base) + + +downloader = SignedDownloader(**vars(args)) +downloader.download('.')