From patchwork Fri Mar 1 14:09:06 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andy Whitcroft X-Patchwork-Id: 1906725 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ubuntu.com (client-ip=185.125.189.65; helo=lists.ubuntu.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=patchwork.ozlabs.org) Received: from lists.ubuntu.com (lists.ubuntu.com [185.125.189.65]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4TmVN71rxbz23fC for ; Sat, 2 Mar 2024 01:09:59 +1100 (AEDT) Received: from localhost ([127.0.0.1] helo=lists.ubuntu.com) by lists.ubuntu.com with esmtp (Exim 4.86_2) (envelope-from ) id 1rg3a8-0005el-Kl; Fri, 01 Mar 2024 14:09:48 +0000 Received: from smtp-relay-internal-0.internal ([10.131.114.225] helo=smtp-relay-internal-0.canonical.com) by lists.ubuntu.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1rg3Zh-0005YU-RT for kernel-team@lists.ubuntu.com; Fri, 01 Mar 2024 14:09:21 +0000 Received: from mail-lf1-f71.google.com (mail-lf1-f71.google.com [209.85.167.71]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by smtp-relay-internal-0.canonical.com (Postfix) with ESMTPS id 849893F275 for ; Fri, 1 Mar 2024 14:09:20 +0000 (UTC) Received: by mail-lf1-f71.google.com with SMTP id 2adb3069b0e04-5131aa087beso1683190e87.2 for ; Fri, 01 Mar 2024 06:09:20 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709302155; x=1709906955; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=yUyqGj7RAoioQSQsHj7vbQPyecG2BTU6cpXQOnQ+uv8=; b=CSgsmL940pBTQyhPQoMTGaeB6KTfSAH4htYbh7uhb34wTVAj02DgjeOKBQOgkFJ31F dxFXlfrppalE6B537pfTOKWbx9bzJo6xJbvvViMwns2iBUB/XqDwR2Bnd1efFiq6V+AU cfkETMlWWNqI2E8fzhG/fZtKF4BgzKG/dTjOHQ9v2YBiaBp2JGuYuZ+EP3SZ22GsloGC MUDJUu5nAtX8NrMbrKTlS9qEBlX3ovotjHPSFIU0YWvzSq3cj3cp744u5RtCuofhebDb Z00ovtVzH+KVqcqvPTQMcJKAUcML/VmtJyHeiHubPt9Ppkit6fMyPaXYbpFEZyBwWG3L X2PA== X-Gm-Message-State: AOJu0YxwIPp+2GquIQlx2SckoxipKcRUy9O+Jt8xfmDQmzSnk5acvraj LD/1/i4xz9WuEoMMPLc7dzzReK1aq6ubh/VjTbGUEzWOKhy8jNOwi/aXzPYIKeg3Mi2DeHVWjXH PL5TqDz0imh9s6g5FK4T2kubHI5gNJeKbPU3duM5YiIhFcfIxahmVI8Wu/ISzD4AJXg/ZJ81bjr syjCbVAFSTDg== X-Received: by 2002:a05:6512:b86:b0:512:fad9:6a5b with SMTP id b6-20020a0565120b8600b00512fad96a5bmr1826935lfv.36.1709302155440; Fri, 01 Mar 2024 06:09:15 -0800 (PST) X-Google-Smtp-Source: AGHT+IHpQPgZcaSfWiUblmBFJT72trkUDx7f0CWMRM4bUbUzHw1FnATeTUUjbDkYnLspEzazoDm94w== X-Received: by 2002:a05:6512:b86:b0:512:fad9:6a5b with SMTP id b6-20020a0565120b8600b00512fad96a5bmr1826916lfv.36.1709302155015; Fri, 01 Mar 2024 06:09:15 -0800 (PST) Received: from localhost ([149.102.2.190]) by smtp.gmail.com with ESMTPSA id 17-20020a05600c229100b00412b2afb2c8sm8444174wmf.26.2024.03.01.06.09.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 01 Mar 2024 06:09:14 -0800 (PST) From: Andy Whitcroft To: kernel-team@lists.ubuntu.com Subject: [xenial/master-next 1/1] UBUNTU: [Packaging] Remove in-tree abi checks Date: Fri, 1 Mar 2024 14:09:06 +0000 Message-ID: <20240301140906.3634335-6-apw@canonical.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240301140906.3634335-1-apw@canonical.com> References: <20240301140906.3634335-1-apw@canonical.com> MIME-Version: 1.0 X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Andy Whitcroft Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" linux-buildinfo packages are now externally compared by swm, with results approving or rejecting updates based on the stable tracker. Those checks also allow hints and overrides to accept intentional changes. Also these are done on the correct pair-wise comparisons, especially when two streams are being cranked. The above eliminates the need to identify previous build abi, download, extract it, vendor it in, and assert it at build time. BugLink: https://bugs.launchpad.net/bugs/2055686 Signed-off-by: Dimitri John Ledkov Signed-off-by: Andy Whitcroft --- debian/rules | 8 +- debian/rules.d/0-common-vars.mk | 1 - debian/rules.d/4-checks.mk | 14 +- debian/scripts/checks/abi-check | 139 ------------------ debian/scripts/checks/final-checks | 44 ------ debian/scripts/checks/module-check | 131 ----------------- debian/scripts/misc/getabis | 217 ----------------------------- 7 files changed, 2 insertions(+), 552 deletions(-) delete mode 100755 debian/scripts/checks/abi-check delete mode 100755 debian/scripts/checks/module-check delete mode 100755 debian/scripts/misc/getabis --- debian/rules | 10 +- debian/rules.d/0-common-vars.mk | 1 - debian/rules.d/4-checks.mk | 20 +-- debian/scripts/abi-check | 210 -------------------------------- debian/scripts/module-check | 120 ------------------ debian/scripts/retpoline-check | 47 ------- 6 files changed, 2 insertions(+), 406 deletions(-) delete mode 100755 debian/scripts/abi-check delete mode 100755 debian/scripts/module-check delete mode 100755 debian/scripts/retpoline-check diff --git a/debian/rules b/debian/rules index c4a855f203cd..b4589d6a6cd5 100755 --- a/debian/rules +++ b/debian/rules @@ -89,14 +89,6 @@ ifneq ($(do_tools),true) do_tools_host= endif -ifeq ($(do_zfs),false) - do_zfs_disable:=$(shell cat $(DROOT)/zfs-modules.ignore >>$(prev_abidir)/../modules.ignore) -endif - -ifeq ($(do_dkms_wireguard),false) - do_wireguard_disable:=$(shell cat $(DROOT)/wireguard-modules.ignore >>$(prev_abidir)/../modules.ignore) -endif - # Either tools package needs the common source preparation do_any_tools=$(sort $(filter-out false,$(do_linux_tools) $(do_cloud_tools))) @@ -126,7 +118,7 @@ clean: debian/control rm -f $(DEBIAN)/d-i/firmware/kernel-image # normal build junk - rm -rf $(DEBIAN)/abi/$(release)-$(revision) + rm -rf $(DEBIAN)/abi rm -rf $(builddir) rm -f $(stampdir)/stamp-* rm -rf $(DEBIAN)/linux-* diff --git a/debian/rules.d/0-common-vars.mk b/debian/rules.d/0-common-vars.mk index 87cbdccb5a1d..49c6f79abd0e 100644 --- a/debian/rules.d/0-common-vars.mk +++ b/debian/rules.d/0-common-vars.mk @@ -118,7 +118,6 @@ ifneq ($(DEB_BUILD_GNU_TYPE),$(DEB_HOST_GNU_TYPE)) endif abidir := $(CURDIR)/$(DEBIAN)/abi/$(release)-$(revision)/$(arch) -prev_abidir := $(CURDIR)/$(DEBIAN)/abi/$(release)-$(prev_revision)/$(arch) commonconfdir := $(CURDIR)/$(DEBIAN)/config archconfdir := $(CURDIR)/$(DEBIAN)/config/$(arch) sharedconfdir := $(CURDIR)/debian.master/config diff --git a/debian/rules.d/4-checks.mk b/debian/rules.d/4-checks.mk index 3ce7e6e6f8e2..78a19dbee39f 100644 --- a/debian/rules.d/4-checks.mk +++ b/debian/rules.d/4-checks.mk @@ -1,22 +1,4 @@ -# Check ABI for package against last release (if not same abinum) -abi-check-%: install-% - @echo Debug: $@ - @perl -f $(DROOT)/scripts/abi-check "$*" "$(prev_abinum)" "$(abinum)" \ - "$(prev_abidir)" "$(abidir)" "$(skipabi)" - -# Check the module list against the last release (always) -module-check-%: install-% - @echo Debug: $@ - @perl -f $(DROOT)/scripts/module-check "$*" \ - "$(prev_abidir)" "$(abidir)" $(skipmodule) - -# Check the reptoline jmp/call functions against the last release. -retpoline-check-%: install-% - @echo Debug: $@ - $(SHELL) $(DROOT)/scripts/retpoline-check "$*" \ - "$(prev_abidir)" "$(abidir)" "$(skipretpoline)" "$(builddir)/build-$*" - -checks-%: module-check-% abi-check-% retpoline-check-% +checks-%: @echo Debug: $@ # Check the config against the known options list. diff --git a/debian/scripts/abi-check b/debian/scripts/abi-check deleted file mode 100755 index c7a02c5589af..000000000000 --- a/debian/scripts/abi-check +++ /dev/null @@ -1,210 +0,0 @@ -#!/usr/bin/perl -w - -my $flavour = shift; -my $prev_abinum = shift; -my $abinum = shift; -my $prev_abidir = shift; -my $abidir = shift; -my $skipabi = shift; - -my $fail_exit = 1; -my $EE = "EE:"; -my $errors = 0; -my $abiskip = 0; - -my $count; - -print "II: Checking ABI for $flavour...\n"; - -if (-f "$prev_abidir/ignore" - or -f "$prev_abidir/$flavour.ignore" or "$skipabi" eq "true") { - print "WW: Explicitly asked to ignore ABI, running in no-fail mode\n"; - $fail_exit = 0; - $abiskip = 1; - $EE = "WW:"; -} - -if ($prev_abinum != $abinum) { - print "II: Different ABI's, running in no-fail mode\n"; - $fail_exit = 0; - $EE = "WW:"; -} - -if (not -f "$abidir/$flavour" or not -f "$prev_abidir/$flavour") { - print "EE: Previous or current ABI file missing!\n"; - print " $abidir/$flavour\n" if not -f "$abidir/$flavour"; - print " $prev_abidir/$flavour\n" if not -f "$prev_abidir/$flavour"; - - # Exit if the ABI files are missing, but return status based on whether - # skip ABI was indicated. - if ("$abiskip" eq "1") { - exit(0); - } else { - exit(1); - } -} - -my %symbols; -my %symbols_ignore; -my %modules_ignore; -my %module_syms; - -# See if we have any ignores -my $ignore = 0; -print " Reading symbols/modules to ignore..."; - -for $file ("$prev_abidir/../blacklist", "$prev_abidir/../../perm-blacklist") { - if (-f $file) { - open(IGNORE, "< $file") or - die "Could not open $file"; - while () { - chomp; - if ($_ =~ m/M: (.*)/) { - $modules_ignore{$1} = 1; - } else { - $symbols_ignore{$_} = 1; - } - $ignore++; - } - close(IGNORE); - } -} -print "read $ignore symbols/modules.\n"; - -sub is_ignored($$) { - my ($mod, $sym) = @_; - - die "Missing module name in is_ignored()" if not defined($mod); - die "Missing symbol name in is_ignored()" if not defined($sym); - - if (defined($symbols_ignore{$sym}) or defined($modules_ignore{$mod})) { - return 1; - } - return 0; -} - -# Read new syms first -print " Reading new symbols ($abinum)..."; -$count = 0; -open(NEW, "< $abidir/$flavour") or - die "Could not open $abidir/$flavour"; -while () { - chomp; - m/^(\S+)\s(.+)\s(0x[0-9a-f]+)\s(.+)$/; - $symbols{$4}{'type'} = $1; - $symbols{$4}{'loc'} = $2; - $symbols{$4}{'hash'} = $3; - $module_syms{$2} = 0; - $count++; -} -close(NEW); -print "read $count symbols.\n"; - -# Now the old symbols, checking for missing ones -print " Reading old symbols ($prev_abinum)..."; -$count = 0; -open(OLD, "< $prev_abidir/$flavour") or - die "Could not open $prev_abidir/$flavour"; -while () { - chomp; - m/^(\S+)\s(.+)\s(0x[0-9a-f]+)\s(.+)$/; - $symbols{$4}{'old_type'} = $1; - $symbols{$4}{'old_loc'} = $2; - $symbols{$4}{'old_hash'} = $3; - $count++; -} -close(OLD); - -print "read $count symbols.\n"; - -print "II: Checking for missing symbols in new ABI..."; -$count = 0; -foreach $sym (keys(%symbols)) { - if (!defined($symbols{$sym}{'type'})) { - print "\n" if not $count; - printf(" MISS : %s%s\n", $sym, - is_ignored($symbols{$sym}{'old_loc'}, $sym) ? " (ignored)" : ""); - $count++ if !is_ignored($symbols{$sym}{'old_loc'}, $sym); - } -} -print " " if $count; -print "found $count missing symbols\n"; -if ($count) { - print "$EE Symbols gone missing (what did you do!?!)\n"; - $errors++; -} - - -print "II: Checking for new symbols in new ABI..."; -$count = 0; -foreach $sym (keys(%symbols)) { - if (!defined($symbols{$sym}{'old_type'})) { - print "\n" if not $count; - print " NEW : $sym\n"; - $count++; - } -} -print " " if $count; -print "found $count new symbols\n"; -if ($count and $prev_abinum == $abinum) { - print "WW: Found new symbols within same ABI. Not recommended\n"; -} - -print "II: Checking for changes to ABI...\n"; -$count = 0; -my $moved = 0; -my $changed_type = 0; -my $changed_hash = 0; -foreach $sym (keys(%symbols)) { - if (!defined($symbols{$sym}{'old_type'}) or - !defined($symbols{$sym}{'type'})) { - next; - } - - # Changes in location don't hurt us, but log it anyway - if ($symbols{$sym}{'loc'} ne $symbols{$sym}{'old_loc'}) { - printf(" MOVE : %-40s : %s => %s\n", $sym, $symbols{$sym}{'old_loc'}, - $symbols{$sym}{'loc'}); - $moved++; - } - - # Changes to export type are only bad if new type isn't - # EXPORT_SYMBOL. Changing things to GPL are bad. - if ($symbols{$sym}{'type'} ne $symbols{$sym}{'old_type'}) { - printf(" TYPE : %-40s : %s => %s%s\n", $sym, $symbols{$sym}{'old_type'}. - $symbols{$sym}{'type'}, is_ignored($symbols{$sym}{'loc'}, $sym) - ? " (ignored)" : ""); - $changed_type++ if $symbols{$sym}{'type'} ne "EXPORT_SYMBOL" - and !is_ignored($symbols{$sym}{'loc'}, $sym); - } - - # Changes to the hash are always bad - if ($symbols{$sym}{'hash'} ne $symbols{$sym}{'old_hash'}) { - printf(" HASH : %-40s : %s => %s%s\n", $sym, $symbols{$sym}{'old_hash'}, - $symbols{$sym}{'hash'}, is_ignored($symbols{$sym}{'loc'}, $sym) - ? " (ignored)" : ""); - $changed_hash++ if !is_ignored($symbols{$sym}{'loc'}, $sym); - $module_syms{$symbols{$sym}{'loc'}}++; - } -} - -print "WW: $moved symbols changed location\n" if $moved; -print "$EE $changed_type symbols changed export type and weren't ignored\n" if $changed_type; -print "$EE $changed_hash symbols changed hash and weren't ignored\n" if $changed_hash; - -$errors++ if $changed_hash or $changed_type; -if ($changed_hash) { - print "II: Module hash change summary...\n"; - foreach $mod (sort { $module_syms{$b} <=> $module_syms{$a} } keys %module_syms) { - next if ! $module_syms{$mod}; - printf(" %-40s: %d\n", $mod, $module_syms{$mod}); - } -} - -print "II: Done\n"; - -if ($errors) { - exit($fail_exit); -} else { - exit(0); -} diff --git a/debian/scripts/module-check b/debian/scripts/module-check deleted file mode 100755 index c754ea368cfb..000000000000 --- a/debian/scripts/module-check +++ /dev/null @@ -1,120 +0,0 @@ -#!/usr/bin/perl -w - -$flavour = shift; -$prev_abidir = shift; -$abidir = shift; -$skipmodule = shift; - -print "II: Checking modules for $flavour..."; - -if (-f "$prev_abidir/ignore.modules" - or -f "$prev_abidir/$flavour.ignore.modules") { - print "explicitly ignoring modules\n"; - exit(0); -} - -if (not -f "$abidir/$flavour.modules" or not -f - "$prev_abidir/$flavour.modules") { - print "previous or current modules file missing!\n"; - print " $abidir/$flavour.modules\n"; - print " $prev_abidir/$flavour.modules\n"; - if (defined($skipmodule)) { - exit(0); - } else { - exit(1); - } -} - -print "\n"; - -my %modules; -my %modules_ignore; -my $missing = 0; -my $new = 0; -my $errors = 0; - -# See if we have any ignores -if (-f "$prev_abidir/../modules.ignore") { - my $ignore = 0; - open(IGNORE, "< $prev_abidir/../modules.ignore") or - die "Could not open $prev_abidir/../modules.ignore"; - print " reading modules to ignore..."; - while () { - chomp; - next if /\s*#/; - $modules_ignore{$_} = 1; - $ignore++; - } - close(IGNORE); - print "read $ignore modules.\n"; -} - -# Read new modules first -print " reading new modules..."; -$new_count = 0; -open(NEW, "< $abidir/$flavour.modules") or - die "Could not open $abidir/$flavour.modules"; -while () { - chomp; - $modules{$_} = 1; - $new_count++; -} -close(NEW); -print "read $new_count modules.\n"; - -# Now the old modules, checking for missing ones -print " reading old modules..."; -$old_count = 0; -open(OLD, "< $prev_abidir/$flavour.modules") or - die "Could not open $prev_abidir/$flavour.modules"; -while () { - chomp; - if (not defined($modules{$_})) { - print "\n" if not $missing; - $missing++; - if (not defined($modules_ignore{$_})) { - print " MISS: $_\n"; - $errors++; - } else { - print " MISS: $_ (ignored)\n"; - } - } else { - $modules{$_}++; - } - $old_count++; -} -close(OLD); -# Check for new modules -foreach $mod (keys(%modules)) { - if ($modules{$mod} < 2) { - print "\n" if not $missing and not $new; - print " NEW : $mod\n"; - $new++; - } -} -if ($new or $missing) { - print " read $old_count modules : new($new) missing($missing)\n"; -} else { - print "read $old_count modules.\n"; -} - - -# Let's see where we stand... -if ($errors) { - if (defined($skipmodule)) { - print "WW: Explicitly asked to ignore failures (probably not good)\n"; - } else { - print "EE: Missing modules (start begging for mercy)\n"; - exit 1 - } -} - -if ($new) { - print "II: New modules (you've been busy, wipe the poop off your nose)\n"; -} else { - print "II: No new modules (hope you're happy, slacker)\n"; -} - -print "II: Done\n"; - -exit(0); diff --git a/debian/scripts/retpoline-check b/debian/scripts/retpoline-check deleted file mode 100755 index 7dbfb303360b..000000000000 --- a/debian/scripts/retpoline-check +++ /dev/null @@ -1,47 +0,0 @@ -#!/bin/bash - -flavour="$1" -prev_abidir="$2" -curr_abidir="$3" -skipretpoline="$4" - -echo "II: Checking retpoline indirections for $flavour..."; - -if [ "$skipretpoline" = 'true' ]; then - echo "manual request ignoring retpoline delta" -fi - -if [ -f "$prev_abidir/ignore.retpoline" -o \ - -f "$prev_abidir/$flavour.ignore.retpoline" ]; then - echo "explicitly ignoring retpoline delta" - skipretpoline='true' -fi - -prev="$prev_abidir/$flavour.retpoline" -curr="$curr_abidir/$flavour.retpoline" -if [ ! -f "$prev" ]; then - echo "previous retpoline file missing!" - echo " $prev" - prev="/dev/null" -fi -if [ ! -f "$curr" ]; then - echo "current retpoline file missing!" - echo " $curr" - curr="/dev/null" -fi - -echo "II: retpoline delta in this package..." -rc=0 -diff -u "$prev" "$curr" || true -count=$( diff -u "$prev" "$curr" | grep '^+[^+]' | wc -l ) -if [ "$count" != 0 ]; then - rc=1 - echo "WW: $count new retpoline sequences detected" -fi - -echo "II: Done"; -if [ "$skipretpoline" = 'true' -a "$rc" -ne 0 ]; then - echo "II: ignoring errors" - exit 0 -fi -exit "$rc"