From patchwork Fri Mar 1 14:09:02 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andy Whitcroft X-Patchwork-Id: 1906723 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ubuntu.com (client-ip=185.125.189.65; helo=lists.ubuntu.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=patchwork.ozlabs.org) Received: from lists.ubuntu.com (lists.ubuntu.com [185.125.189.65]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4TmVMr55wbz23fC for ; Sat, 2 Mar 2024 01:09:44 +1100 (AEDT) Received: from localhost ([127.0.0.1] helo=lists.ubuntu.com) by lists.ubuntu.com with esmtp (Exim 4.86_2) (envelope-from ) id 1rg3Zh-0005YV-Cj; Fri, 01 Mar 2024 14:09:21 +0000 Received: from smtp-relay-internal-1.internal ([10.131.114.114] helo=smtp-relay-internal-1.canonical.com) by lists.ubuntu.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1rg3ZW-0005XP-RP for kernel-team@lists.ubuntu.com; Fri, 01 Mar 2024 14:09:10 +0000 Received: from mail-lj1-f199.google.com (mail-lj1-f199.google.com [209.85.208.199]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by smtp-relay-internal-1.canonical.com (Postfix) with ESMTPS id 790383F27F for ; Fri, 1 Mar 2024 14:09:10 +0000 (UTC) Received: by mail-lj1-f199.google.com with SMTP id 38308e7fff4ca-2d2af8849b1so12859571fa.3 for ; Fri, 01 Mar 2024 06:09:10 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709302149; x=1709906949; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=IWNdvSNlGwXrzCiwa502KKpUmRY+R688tdnINCS0BCI=; b=MaSTIYVBoIIEhfLvXMsgvDJ3Eti+Prk9kqYtPuwTBdJEhhrJfvz2HuMDtyaie9QyP7 cFY+wzu/VsAY9w/3N0Xu5Q1IhYJz0DIM6JNhTeR+dMdm6/eST1bEHi0P9El6l2MCnywH WRQ9hunHuv4zsOHxXTaurImPQRKsgEC6KK4yWT/PeHfunrpjypAejSgVLLdelvwo89Rq 4w59Ye24HiGnTU72lNjRIdPF8PzVHdI4ZGbnCqb9X9i5aBMbUyao8AdQWFq3IIfM7RZw 8K/h3bIOGHUWZWgjzHBraTcjtbM59ChknIzueSNb5JZXCGVZp6b8bvijjCtFpGWqI5sR i1vA== X-Gm-Message-State: AOJu0Yxl0J2b5tNgaL2TvdcXL8S1cz0p3HiLGFid/338Hxj+6zyPRVno Nrwfm+ztvuHuM03FFCEVyhnJkIWIZk6LviObPCg7NXrNiDIT3KyNMGZqSqtKNSiHnp8Tsr4b7vA mZMg1ZTM/k6WVHgIZBsZP0aDuKttAG2NIspfgqblGO68lQ6Q9gAkVER6di/BgzuzrjjeDxe5On0 r6aal37OMMyg== X-Received: by 2002:a2e:3a08:0:b0:2d2:9b43:8d48 with SMTP id h8-20020a2e3a08000000b002d29b438d48mr1395608lja.39.1709302149328; Fri, 01 Mar 2024 06:09:09 -0800 (PST) X-Google-Smtp-Source: AGHT+IEbA8npDv0nq2hIjEmnERZIeJws4lU281Ev8zkQeUtYTu8DTVSw5VqSGPs4tT41+vAX0gZgdg== X-Received: by 2002:a2e:3a08:0:b0:2d2:9b43:8d48 with SMTP id h8-20020a2e3a08000000b002d29b438d48mr1395588lja.39.1709302148925; Fri, 01 Mar 2024 06:09:08 -0800 (PST) Received: from localhost ([149.102.2.190]) by smtp.gmail.com with ESMTPSA id j11-20020a5d604b000000b0033d9ee09b7asm4638292wrt.107.2024.03.01.06.09.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 01 Mar 2024 06:09:08 -0800 (PST) From: Andy Whitcroft To: kernel-team@lists.ubuntu.com Subject: [bionic/master-next 1/1] UBUNTU: [Packaging] Remove in-tree abi checks Date: Fri, 1 Mar 2024 14:09:02 +0000 Message-ID: <20240301140906.3634335-2-apw@canonical.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240301140906.3634335-1-apw@canonical.com> References: <20240301140906.3634335-1-apw@canonical.com> MIME-Version: 1.0 X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Andy Whitcroft Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" linux-buildinfo packages are now externally compared by swm, with results approving or rejecting updates based on the stable tracker. Those checks also allow hints and overrides to accept intentional changes. Also these are done on the correct pair-wise comparisons, especially when two streams are being cranked. The above eliminates the need to identify previous build abi, download, extract it, vendor it in, and assert it at build time. BugLink: https://bugs.launchpad.net/bugs/2055686 Signed-off-by: Dimitri John Ledkov Signed-off-by: Andy Whitcroft Acked-by: Roxana Nicolescu --- debian/rules | 10 +- debian/rules.d/0-common-vars.mk | 1 - debian/rules.d/4-checks.mk | 20 +-- debian/scripts/abi-check | 210 -------------------------------- debian/scripts/module-check | 120 ------------------ debian/scripts/retpoline-check | 47 ------- 6 files changed, 2 insertions(+), 406 deletions(-) delete mode 100755 debian/scripts/abi-check delete mode 100755 debian/scripts/module-check delete mode 100755 debian/scripts/retpoline-check diff --git a/debian/rules b/debian/rules index d742490a6341..ad69b020a4d1 100755 --- a/debian/rules +++ b/debian/rules @@ -93,14 +93,6 @@ ifneq ($(do_tools),true) do_tools_host= endif -ifeq ($(do_zfs),false) - do_zfs_disable:=$(shell cat $(DROOT)/zfs-modules.ignore >>$(prev_abidir)/../modules.ignore) -endif - -ifeq ($(do_dkms_wireguard),false) - do_wireguard_disable:=$(shell cat $(DROOT)/wireguard-modules.ignore >>$(prev_abidir)/../modules.ignore) -endif - # Either tools package needs the common source preparation do_any_tools=$(sort $(filter-out false,$(do_linux_tools) $(do_cloud_tools))) @@ -137,7 +129,7 @@ clean: debian/control debian/canonical-certs.pem debian/canonical-revoked-certs. rm -f $(DEBIAN)/d-i/firmware/$(arch)/kernel-image # normal build junk - rm -rf $(DEBIAN)/abi/$(release)-$(revision) + rm -rf $(DEBIAN)/abi rm -rf $(builddir) rm -f $(stampdir)/stamp-* rm -rf $(DEBIAN)/linux-* diff --git a/debian/rules.d/0-common-vars.mk b/debian/rules.d/0-common-vars.mk index ed8a4a360e26..6157b32bcc93 100644 --- a/debian/rules.d/0-common-vars.mk +++ b/debian/rules.d/0-common-vars.mk @@ -118,7 +118,6 @@ ifneq ($(DEB_BUILD_GNU_TYPE),$(DEB_HOST_GNU_TYPE)) endif abidir := $(CURDIR)/$(DEBIAN)/abi/$(release)-$(revision)/$(arch) -prev_abidir := $(CURDIR)/$(DEBIAN)/abi/$(release)-$(prev_revision)/$(arch) commonconfdir := $(CURDIR)/$(DEBIAN)/config archconfdir := $(CURDIR)/$(DEBIAN)/config/$(arch) sharedconfdir := $(CURDIR)/debian.master/config diff --git a/debian/rules.d/4-checks.mk b/debian/rules.d/4-checks.mk index 3ce7e6e6f8e2..78a19dbee39f 100644 --- a/debian/rules.d/4-checks.mk +++ b/debian/rules.d/4-checks.mk @@ -1,22 +1,4 @@ -# Check ABI for package against last release (if not same abinum) -abi-check-%: install-% - @echo Debug: $@ - @perl -f $(DROOT)/scripts/abi-check "$*" "$(prev_abinum)" "$(abinum)" \ - "$(prev_abidir)" "$(abidir)" "$(skipabi)" - -# Check the module list against the last release (always) -module-check-%: install-% - @echo Debug: $@ - @perl -f $(DROOT)/scripts/module-check "$*" \ - "$(prev_abidir)" "$(abidir)" $(skipmodule) - -# Check the reptoline jmp/call functions against the last release. -retpoline-check-%: install-% - @echo Debug: $@ - $(SHELL) $(DROOT)/scripts/retpoline-check "$*" \ - "$(prev_abidir)" "$(abidir)" "$(skipretpoline)" "$(builddir)/build-$*" - -checks-%: module-check-% abi-check-% retpoline-check-% +checks-%: @echo Debug: $@ # Check the config against the known options list. diff --git a/debian/scripts/abi-check b/debian/scripts/abi-check deleted file mode 100755 index c7a02c5589af..000000000000 --- a/debian/scripts/abi-check +++ /dev/null @@ -1,210 +0,0 @@ -#!/usr/bin/perl -w - -my $flavour = shift; -my $prev_abinum = shift; -my $abinum = shift; -my $prev_abidir = shift; -my $abidir = shift; -my $skipabi = shift; - -my $fail_exit = 1; -my $EE = "EE:"; -my $errors = 0; -my $abiskip = 0; - -my $count; - -print "II: Checking ABI for $flavour...\n"; - -if (-f "$prev_abidir/ignore" - or -f "$prev_abidir/$flavour.ignore" or "$skipabi" eq "true") { - print "WW: Explicitly asked to ignore ABI, running in no-fail mode\n"; - $fail_exit = 0; - $abiskip = 1; - $EE = "WW:"; -} - -if ($prev_abinum != $abinum) { - print "II: Different ABI's, running in no-fail mode\n"; - $fail_exit = 0; - $EE = "WW:"; -} - -if (not -f "$abidir/$flavour" or not -f "$prev_abidir/$flavour") { - print "EE: Previous or current ABI file missing!\n"; - print " $abidir/$flavour\n" if not -f "$abidir/$flavour"; - print " $prev_abidir/$flavour\n" if not -f "$prev_abidir/$flavour"; - - # Exit if the ABI files are missing, but return status based on whether - # skip ABI was indicated. - if ("$abiskip" eq "1") { - exit(0); - } else { - exit(1); - } -} - -my %symbols; -my %symbols_ignore; -my %modules_ignore; -my %module_syms; - -# See if we have any ignores -my $ignore = 0; -print " Reading symbols/modules to ignore..."; - -for $file ("$prev_abidir/../blacklist", "$prev_abidir/../../perm-blacklist") { - if (-f $file) { - open(IGNORE, "< $file") or - die "Could not open $file"; - while () { - chomp; - if ($_ =~ m/M: (.*)/) { - $modules_ignore{$1} = 1; - } else { - $symbols_ignore{$_} = 1; - } - $ignore++; - } - close(IGNORE); - } -} -print "read $ignore symbols/modules.\n"; - -sub is_ignored($$) { - my ($mod, $sym) = @_; - - die "Missing module name in is_ignored()" if not defined($mod); - die "Missing symbol name in is_ignored()" if not defined($sym); - - if (defined($symbols_ignore{$sym}) or defined($modules_ignore{$mod})) { - return 1; - } - return 0; -} - -# Read new syms first -print " Reading new symbols ($abinum)..."; -$count = 0; -open(NEW, "< $abidir/$flavour") or - die "Could not open $abidir/$flavour"; -while () { - chomp; - m/^(\S+)\s(.+)\s(0x[0-9a-f]+)\s(.+)$/; - $symbols{$4}{'type'} = $1; - $symbols{$4}{'loc'} = $2; - $symbols{$4}{'hash'} = $3; - $module_syms{$2} = 0; - $count++; -} -close(NEW); -print "read $count symbols.\n"; - -# Now the old symbols, checking for missing ones -print " Reading old symbols ($prev_abinum)..."; -$count = 0; -open(OLD, "< $prev_abidir/$flavour") or - die "Could not open $prev_abidir/$flavour"; -while () { - chomp; - m/^(\S+)\s(.+)\s(0x[0-9a-f]+)\s(.+)$/; - $symbols{$4}{'old_type'} = $1; - $symbols{$4}{'old_loc'} = $2; - $symbols{$4}{'old_hash'} = $3; - $count++; -} -close(OLD); - -print "read $count symbols.\n"; - -print "II: Checking for missing symbols in new ABI..."; -$count = 0; -foreach $sym (keys(%symbols)) { - if (!defined($symbols{$sym}{'type'})) { - print "\n" if not $count; - printf(" MISS : %s%s\n", $sym, - is_ignored($symbols{$sym}{'old_loc'}, $sym) ? " (ignored)" : ""); - $count++ if !is_ignored($symbols{$sym}{'old_loc'}, $sym); - } -} -print " " if $count; -print "found $count missing symbols\n"; -if ($count) { - print "$EE Symbols gone missing (what did you do!?!)\n"; - $errors++; -} - - -print "II: Checking for new symbols in new ABI..."; -$count = 0; -foreach $sym (keys(%symbols)) { - if (!defined($symbols{$sym}{'old_type'})) { - print "\n" if not $count; - print " NEW : $sym\n"; - $count++; - } -} -print " " if $count; -print "found $count new symbols\n"; -if ($count and $prev_abinum == $abinum) { - print "WW: Found new symbols within same ABI. Not recommended\n"; -} - -print "II: Checking for changes to ABI...\n"; -$count = 0; -my $moved = 0; -my $changed_type = 0; -my $changed_hash = 0; -foreach $sym (keys(%symbols)) { - if (!defined($symbols{$sym}{'old_type'}) or - !defined($symbols{$sym}{'type'})) { - next; - } - - # Changes in location don't hurt us, but log it anyway - if ($symbols{$sym}{'loc'} ne $symbols{$sym}{'old_loc'}) { - printf(" MOVE : %-40s : %s => %s\n", $sym, $symbols{$sym}{'old_loc'}, - $symbols{$sym}{'loc'}); - $moved++; - } - - # Changes to export type are only bad if new type isn't - # EXPORT_SYMBOL. Changing things to GPL are bad. - if ($symbols{$sym}{'type'} ne $symbols{$sym}{'old_type'}) { - printf(" TYPE : %-40s : %s => %s%s\n", $sym, $symbols{$sym}{'old_type'}. - $symbols{$sym}{'type'}, is_ignored($symbols{$sym}{'loc'}, $sym) - ? " (ignored)" : ""); - $changed_type++ if $symbols{$sym}{'type'} ne "EXPORT_SYMBOL" - and !is_ignored($symbols{$sym}{'loc'}, $sym); - } - - # Changes to the hash are always bad - if ($symbols{$sym}{'hash'} ne $symbols{$sym}{'old_hash'}) { - printf(" HASH : %-40s : %s => %s%s\n", $sym, $symbols{$sym}{'old_hash'}, - $symbols{$sym}{'hash'}, is_ignored($symbols{$sym}{'loc'}, $sym) - ? " (ignored)" : ""); - $changed_hash++ if !is_ignored($symbols{$sym}{'loc'}, $sym); - $module_syms{$symbols{$sym}{'loc'}}++; - } -} - -print "WW: $moved symbols changed location\n" if $moved; -print "$EE $changed_type symbols changed export type and weren't ignored\n" if $changed_type; -print "$EE $changed_hash symbols changed hash and weren't ignored\n" if $changed_hash; - -$errors++ if $changed_hash or $changed_type; -if ($changed_hash) { - print "II: Module hash change summary...\n"; - foreach $mod (sort { $module_syms{$b} <=> $module_syms{$a} } keys %module_syms) { - next if ! $module_syms{$mod}; - printf(" %-40s: %d\n", $mod, $module_syms{$mod}); - } -} - -print "II: Done\n"; - -if ($errors) { - exit($fail_exit); -} else { - exit(0); -} diff --git a/debian/scripts/module-check b/debian/scripts/module-check deleted file mode 100755 index c754ea368cfb..000000000000 --- a/debian/scripts/module-check +++ /dev/null @@ -1,120 +0,0 @@ -#!/usr/bin/perl -w - -$flavour = shift; -$prev_abidir = shift; -$abidir = shift; -$skipmodule = shift; - -print "II: Checking modules for $flavour..."; - -if (-f "$prev_abidir/ignore.modules" - or -f "$prev_abidir/$flavour.ignore.modules") { - print "explicitly ignoring modules\n"; - exit(0); -} - -if (not -f "$abidir/$flavour.modules" or not -f - "$prev_abidir/$flavour.modules") { - print "previous or current modules file missing!\n"; - print " $abidir/$flavour.modules\n"; - print " $prev_abidir/$flavour.modules\n"; - if (defined($skipmodule)) { - exit(0); - } else { - exit(1); - } -} - -print "\n"; - -my %modules; -my %modules_ignore; -my $missing = 0; -my $new = 0; -my $errors = 0; - -# See if we have any ignores -if (-f "$prev_abidir/../modules.ignore") { - my $ignore = 0; - open(IGNORE, "< $prev_abidir/../modules.ignore") or - die "Could not open $prev_abidir/../modules.ignore"; - print " reading modules to ignore..."; - while () { - chomp; - next if /\s*#/; - $modules_ignore{$_} = 1; - $ignore++; - } - close(IGNORE); - print "read $ignore modules.\n"; -} - -# Read new modules first -print " reading new modules..."; -$new_count = 0; -open(NEW, "< $abidir/$flavour.modules") or - die "Could not open $abidir/$flavour.modules"; -while () { - chomp; - $modules{$_} = 1; - $new_count++; -} -close(NEW); -print "read $new_count modules.\n"; - -# Now the old modules, checking for missing ones -print " reading old modules..."; -$old_count = 0; -open(OLD, "< $prev_abidir/$flavour.modules") or - die "Could not open $prev_abidir/$flavour.modules"; -while () { - chomp; - if (not defined($modules{$_})) { - print "\n" if not $missing; - $missing++; - if (not defined($modules_ignore{$_})) { - print " MISS: $_\n"; - $errors++; - } else { - print " MISS: $_ (ignored)\n"; - } - } else { - $modules{$_}++; - } - $old_count++; -} -close(OLD); -# Check for new modules -foreach $mod (keys(%modules)) { - if ($modules{$mod} < 2) { - print "\n" if not $missing and not $new; - print " NEW : $mod\n"; - $new++; - } -} -if ($new or $missing) { - print " read $old_count modules : new($new) missing($missing)\n"; -} else { - print "read $old_count modules.\n"; -} - - -# Let's see where we stand... -if ($errors) { - if (defined($skipmodule)) { - print "WW: Explicitly asked to ignore failures (probably not good)\n"; - } else { - print "EE: Missing modules (start begging for mercy)\n"; - exit 1 - } -} - -if ($new) { - print "II: New modules (you've been busy, wipe the poop off your nose)\n"; -} else { - print "II: No new modules (hope you're happy, slacker)\n"; -} - -print "II: Done\n"; - -exit(0); diff --git a/debian/scripts/retpoline-check b/debian/scripts/retpoline-check deleted file mode 100755 index 7dbfb303360b..000000000000 --- a/debian/scripts/retpoline-check +++ /dev/null @@ -1,47 +0,0 @@ -#!/bin/bash - -flavour="$1" -prev_abidir="$2" -curr_abidir="$3" -skipretpoline="$4" - -echo "II: Checking retpoline indirections for $flavour..."; - -if [ "$skipretpoline" = 'true' ]; then - echo "manual request ignoring retpoline delta" -fi - -if [ -f "$prev_abidir/ignore.retpoline" -o \ - -f "$prev_abidir/$flavour.ignore.retpoline" ]; then - echo "explicitly ignoring retpoline delta" - skipretpoline='true' -fi - -prev="$prev_abidir/$flavour.retpoline" -curr="$curr_abidir/$flavour.retpoline" -if [ ! -f "$prev" ]; then - echo "previous retpoline file missing!" - echo " $prev" - prev="/dev/null" -fi -if [ ! -f "$curr" ]; then - echo "current retpoline file missing!" - echo " $curr" - curr="/dev/null" -fi - -echo "II: retpoline delta in this package..." -rc=0 -diff -u "$prev" "$curr" || true -count=$( diff -u "$prev" "$curr" | grep '^+[^+]' | wc -l ) -if [ "$count" != 0 ]; then - rc=1 - echo "WW: $count new retpoline sequences detected" -fi - -echo "II: Done"; -if [ "$skipretpoline" = 'true' -a "$rc" -ne 0 ]; then - echo "II: ignoring errors" - exit 0 -fi -exit "$rc"