new file mode 100644
@@ -0,0 +1,185 @@
+==================
+Config Annotations
+==================
+
+:Author: Andrea Righi
+
+Overview
+========
+
+Each Ubuntu kernel needs to maintain its own .config for each supported
+architecture and each flavour.
+
+Every time a new patch is applied or a kernel is rebased on top of a new
+one, we need to update the .config's accordingly (config options can be
+added, removed and also renamed).
+
+So, we need to make sure that some critical config options are always
+matching the desired value in order to have a functional kernel.
+
+State of the art
+================
+
+At the moment configs are maintained as a set of Kconfig chunks (inside
+`debian.<kernel>/config/`): a global one, plus per-arch / per-flavour
+chunks.
+
+In addition to that, we need to maintain also a file called
+'annotations'; the purpose of this file is to make sure that some
+critical config options are not silently removed or changed when the
+real .config is re-generated (for example after a rebase or after
+applying a new set of patches).
+
+The main problem with this approach is that, often, we have duplicate
+information that is stored both in the Kconfig chunks *and* in the
+annotations files and, at the same time, the whole .config's information
+is distributed between Kconfig chunks and annotations, making it hard to
+maintain, review and manage in general.
+
+Proposed solution
+=================
+
+The proposed solution is to store all the config information into the
+"annotations" format and get rid of the config chunks (basically the
+real .config's can be produced "compiling" annotations).
+
+Implementation
+==============
+
+To help the management of the annotations an helper script is provided
+(`debian/scripts/misc/annotations`):
+
+```
+usage: annotations [-h] [--version] [--file FILE] [--arch ARCH] [--flavour FLAVOUR] [--config CONFIG]
+ (--query | --export | --import FILE | --update FILE | --check FILE)
+
+Manage Ubuntu kernel .config and annotations
+
+options:
+ -h, --help show this help message and exit
+ --version, -v show program's version number and exit
+ --file FILE, -f FILE Pass annotations or .config file to be parsed
+ --arch ARCH, -a ARCH Select architecture
+ --flavour FLAVOUR, -l FLAVOUR
+ Select flavour (default is "generic")
+ --config CONFIG, -c CONFIG
+ Select a specific config option
+
+Action:
+ --query, -q Query annotations
+ --export, -e Convert annotations to .config format
+ --import FILE, -i FILE
+ Import a full .config for a specific arch and flavour into annotations
+ --update FILE, -u FILE
+ Import a partial .config into annotations (only resync configs specified in FILE)
+ --check FILE, -k FILE
+ Validate kernel .config with annotations
+```
+
+This script allows to query config settings (per arch/flavour/config),
+export them into the Kconfig format (generating the real .config files)
+and check if the final .config matches the rules defined in the
+annotations.
+
+Examples (annotations is defined as an alias to `debian/scripts/annotations`):
+
+ - Show settings for `CONFIG_DEBUG_INFO_BTF` for master kernel across all the
+ supported architectures and flavours:
+
+```
+$ annotations --query --config CONFIG_DEBUG_INFO_BTF
+{
+ "policy": {
+ "amd64": "y",
+ "arm64": "y",
+ "armhf": "n",
+ "ppc64el": "y",
+ "riscv64": "y",
+ "s390x": "y"
+ },
+ "note": "'Needs newer pahole for armhf'"
+}
+```
+
+ - Dump kernel .config for arm64 and flavour generic-64k:
+
+```
+$ annotations --arch arm64 --flavour generic-64k --export
+CONFIG_DEBUG_FS=y
+CONFIG_DEBUG_KERNEL=y
+CONFIG_COMPAT=y
+...
+```
+
+ - Update annotations file with a new kernel .config for amd64 flavour
+ generic:
+
+```
+$ annotations --arch amd64 --flavour generic --import build/.config
+```
+
+Moreover, an additional kernelconfig commands are provided
+(via debian/rules targets):
+ - `migrateconfigs`: automatically merge all the previous configs into
+ annotations (local changes still need to be committed)
+
+Annotations headers
+===================
+
+The main annotations file should contain a header to define the architectures
+and flavours that are supported.
+
+Here is the format of the header for the generic kernel:
+```
+# Menu: HEADER
+# FORMAT: 4
+# ARCH: amd64 arm64 armhf ppc64el riscv64 s390x
+# FLAVOUR: amd64-generic arm64-generic arm64-generic-64k armhf-generic armhf-generic-lpae ppc64el-generic riscv64-generic s390x-generic
+
+```
+
+Example header of a derivative (linux-aws):
+```
+# Menu: HEADER
+# FORMAT: 4
+# ARCH: amd64 arm64
+# FLAVOUR: amd64-aws arm64-aws
+# FLAVOUR_DEP: {'amd64-aws': 'amd64-generic', 'arm64-aws': 'arm64-generic'}
+
+include "../../debian.master/config/annotations"
+
+# Below you can define only the specific linux-aws configs that differ from linux generic
+
+```
+
+Pros and Cons
+=============
+
+ Pros:
+ - avoid duplicate information in .config's and annotations
+ - allow to easily define groups of config settings (for a specific
+ environment or feature, such as annotations.clouds, annotations.ubuntu,
+ annotations.snapd, etc.)
+ - config options are more accessible, easy to change and review
+ - we can easily document how config options are managed (and external
+ contributors won't be discouraged anymore when they need to to change a
+ config option)
+
+ Cons:
+ - potential regressions: the new tool/scripts can have potential bugs,
+ so we could experience regressions due to some missed config changes
+ - kernel team need to understand the new process (even if everything
+ is transparent, kernel cranking process is the same, there might be
+ corner cases that need to be addressed and resolved manually)
+
+TODO
+====
+
+ - Migrate all flavour and arch definitions into annotations (rather
+ than having this information defined in multiple places inside
+ debian/scripts); right now this information is "partially" migrated,
+ meaning that we need to define arches and flavours in the headers
+ section of annotations (so that the annotations tool can figure out
+ the list of supported arches and flavours), but arches and flavours
+ are still defined elsewhere, ideally we would like to have arches and
+ flavours defined only in one place: annotations.
@@ -1,7 +1,7 @@
# The following targets are for the maintainer only! do not run if you don't
# know what they do.
-.PHONY: printenv updateconfigs printchanges insertchanges startnewrelease diffupstream help autoreconstruct finalchecks
+.PHONY: printenv updateconfigs migrateconfigs printchanges insertchanges startnewrelease diffupstream help autoreconstruct finalchecks
help:
@echo "These are the targets in addition to the normal $(DEBIAN) ones:"
@@ -11,6 +11,7 @@ help:
@echo " updateconfigs : Update core arch configs"
@echo
@echo " editconfigs : Update core arch configs interractively"
+ @echo " migrateconfigs : Automatically import old configs into annotations"
@echo " genconfigs : Generate core arch configs in CONFIGS/*"
@echo
@echo " printchanges : Print the current changelog entries (from git)"
@@ -40,8 +41,25 @@ printdebian:
updateconfigs defaultconfigs editconfigs genconfigs dumpconfigs:
dh_testdir;
- $(SHELL) $(DROOT)/scripts/misc/kernelconfig $@ "$(do_enforce_all)"
+ kmake='$(kmake)' skip_checks=$(do_skip_checks) conc_level=$(conc_level) \
+ $(SHELL) $(DROOT)/scripts/misc/kernelconfig $@
+ @rm -rf build
+
+migrateconfigs:
+ifneq ($(wildcard $(DEBIAN)/config/config.common.ubuntu),)
+ dh_testdir
+ conc_level=$(conc_level) $(SHELL) $(DROOT)/scripts/misc/old-kernelconfig genconfigs
rm -rf build
+ mkdir build
+ mv $(DEBIAN)/config/annotations build/.annotations
+ mv $(DEBIAN)/config/README.rst build/.README.rst 2>/dev/null || true
+ rm -rf $(DEBIAN)/config
+ mkdir -p $(DEBIAN)/config
+ debian/scripts/misc/migrate-annotations < build/.annotations > $(DEBIAN)/config/annotations
+ mv build/.README.rst $(DEBIAN)/config/README.rst 2>/dev/null || true
+ rm -rf build
+ kmake='$(kmake)' conc_level=$(conc_level) $(SHELL) $(DROOT)/scripts/misc/kernelconfig updateconfigs
+endif
printenv:
dh_testdir
@@ -24,13 +24,18 @@ $(stampdir)/stamp-prepare-%: config-prepare-check-%
@echo Debug: $@
@touch $@
$(stampdir)/stamp-prepare-tree-%: target_flavour = $*
-$(stampdir)/stamp-prepare-tree-%: $(commonconfdir)/config.common.$(family) $(archconfdir)/config.common.$(arch) $(archconfdir)/config.flavour.% debian/scripts/fix-filenames
+$(stampdir)/stamp-prepare-tree-%: debian/scripts/fix-filenames
@echo Debug: $@
install -d $(builddir)/build-$*
touch $(builddir)/build-$*/ubuntu-build
[ "$(do_full_source)" != 'true' ] && true || \
rsync -a --exclude debian --exclude debian.master --exclude $(DEBIAN) * $(builddir)/build-$*
- cat $(wordlist 1,3,$^) | sed -e 's/.*CONFIG_VERSION_SIGNATURE.*/CONFIG_VERSION_SIGNATURE="Ubuntu $(release)-$(revision)-$* $(raw_kernelversion)"/' > $(builddir)/build-$*/.config
+ if [ -e $(commonconfdir)/config.common.ubuntu ]; then \
+ cat $(commonconfdir)/config.common.ubuntu $(archconfdir)/config.common.$(arch) $(archconfdir)/config.flavour.$(target_flavour) > $(builddir)/build-$*/.config; \
+ else \
+ python3 debian/scripts/misc/annotations --export --arch $(arch) --flavour $(target_flavour) | sed -e 's/.*CONFIG_VERSION_SIGNATURE.*/CONFIG_VERSION_SIGNATURE="Ubuntu $(release)-$(revision)-$* $(raw_kernelversion)"/' > $(builddir)/build-$*/.config; \
+ fi
+ sed -i 's/.*CONFIG_VERSION_SIGNATURE.*/CONFIG_VERSION_SIGNATURE="Ubuntu $(release)-$(revision)-$* $(raw_kernelversion)"/' $(builddir)/build-$*/.config
[ "$(do_odm_drivers)" = 'true' ] && true || \
sed -ie 's/.*CONFIG_UBUNTU_ODM_DRIVERS.*/# CONFIG_UBUNTU_ODM_DRIVERS is not set/' \
$(builddir)/build-$*/.config
@@ -29,7 +29,11 @@ checks-%: module-check-% module-signature-check-% abi-check-% retpoline-check-%
# Check the config against the known options list.
config-prepare-check-%: $(stampdir)/stamp-prepare-tree-%
@echo Debug: $@
- @perl -f $(DROOT)/scripts/config-check \
- $(builddir)/build-$*/.config "$(arch)" "$*" "$(commonconfdir)" \
- "$(skipconfig)" "$(do_enforce_all)"
-
+ if [ -e $(commonconfdir)/config.common.ubuntu ]; then \
+ @perl -f $(DROOT)/scripts/config-check \
+ $(builddir)/build-$*/.config "$(arch)" "$*" "$(commonconfdir)" \
+ "$(skipconfig)" "$(do_enforce_all)" \
+ else \
+ python3 $(DROOT)/scripts/misc/annotations -f $(commonconfdir)/annotations \
+ --arch $(arch) --flavour $* --check $(builddir)/build-$*/.config; \
+ fi
new file mode 100755
@@ -0,0 +1,274 @@
+#!/usr/bin/env python3
+# -*- mode: python -*-
+# Manage Ubuntu kernel .config and annotations
+# Copyright © 2022 Canonical Ltd.
+
+import sys
+sys.dont_write_bytecode = True
+import os
+import argparse
+import json
+from signal import signal, SIGPIPE, SIG_DFL
+
+from kconfig.annotations import Annotation, KConfig
+
+VERSION = '0.1'
+
+SKIP_CONFIGS = (
+ # CONFIG_VERSION_SIGNATURE is dynamically set during the build
+ 'CONFIG_VERSION_SIGNATURE',
+ # Allow to use a different versions of toolchain tools
+ 'CONFIG_GCC_VERSION',
+ 'CONFIG_CC_VERSION_TEXT',
+ 'CONFIG_AS_VERSION',
+ 'CONFIG_LD_VERSION',
+ 'CONFIG_LLD_VERSION',
+ 'CONFIG_CLANG_VERSION',
+ 'CONFIG_PAHOLE_VERSION',
+ 'CONFIG_RUSTC_VERSION_TEXT',
+ 'CONFIG_BINDGEN_VERSION_TEXT',
+)
+
+
+def make_parser():
+ parser = argparse.ArgumentParser(
+ description='Manage Ubuntu kernel .config and annotations',
+ )
+ parser.add_argument('--version', '-v', action='version', version=f'%(prog)s {VERSION}')
+
+ parser.add_argument('--file', '-f', action='store',
+ help='Pass annotations or .config file to be parsed')
+ parser.add_argument('--arch', '-a', action='store',
+ help='Select architecture')
+ parser.add_argument('--flavour', '-l', action='store',
+ help='Select flavour (default is "generic")')
+ parser.add_argument('--config', '-c', action='store',
+ help='Select a specific config option')
+ parser.add_argument('--query', '-q', action='store_true',
+ help='Query annotations')
+ parser.add_argument('--note', '-n', action='store',
+ help='Write a specific note to a config option in annotations')
+ parser.add_argument('--autocomplete', action='store_true',
+ help='Enable config bash autocomplete: `source <(annotations --autocomplete)`')
+ parser.add_argument('--source', '-t', action='store_true',
+ help='Jump to a config definition in the kernel source code')
+
+ ga = parser.add_argument_group(title='Action').add_mutually_exclusive_group(required=False)
+ ga.add_argument('--write', '-w', action='store',
+ metavar='VALUE', dest='value',
+ help='Set a specific config value in annotations (use \'null\' to remove)')
+ ga.add_argument('--export', '-e', action='store_true',
+ help='Convert annotations to .config format')
+ ga.add_argument('--import', '-i', action='store',
+ metavar="FILE", dest='import_file',
+ help='Import a full .config for a specific arch and flavour into annotations')
+ ga.add_argument('--update', '-u', action='store',
+ metavar="FILE", dest='update_file',
+ help='Import a partial .config into annotations (only resync configs specified in FILE)')
+ ga.add_argument('--check', '-k', action='store',
+ metavar="FILE", dest='check_file',
+ help='Validate kernel .config with annotations')
+ return parser
+
+
+_ARGPARSER = make_parser()
+
+
+def arg_fail(message):
+ print(message)
+ _ARGPARSER.print_usage()
+ sys.exit(1)
+
+
+def print_result(config, res):
+ if res is not None and config not in res:
+ res = {config or '*': res}
+ print(json.dumps(res, indent=4))
+
+
+def do_query(args):
+ if args.arch is None and args.flavour is not None:
+ arg_fail('error: --flavour requires --arch')
+ a = Annotation(args.file)
+ res = a.search_config(config=args.config, arch=args.arch, flavour=args.flavour)
+ print_result(args.config, res)
+
+
+def do_autocomplete(args):
+ a = Annotation(args.file)
+ res = (c.removeprefix('CONFIG_') for c in a.search_config())
+ res_str = ' '.join(res)
+ print(f'complete -W "{res_str}" annotations')
+
+
+def do_source(args):
+ if args.config is None:
+ arg_fail('error: --source requires --config')
+ if not os.path.exists('tags'):
+ print('tags not found in the current directory, try: `make tags`')
+ sys.exit(1)
+ os.system(f'vim -t {args.config}')
+
+
+def do_note(args):
+ if args.config is None:
+ arg_fail('error: --note requires --config')
+
+ # Set the note in annotations
+ a = Annotation(args.file)
+ a.set(args.config, note=args.note)
+
+ # Save back to annotations
+ a.save(args.file)
+
+ # Query and print back the value
+ a = Annotation(args.file)
+ res = a.search_config(config=args.config)
+ print_result(args.config, res)
+
+
+def do_write(args):
+ if args.config is None:
+ arg_fail('error: --write requires --config')
+
+ # Set the value in annotations ('null' means remove)
+ a = Annotation(args.file)
+ if args.value == 'null':
+ a.remove(args.config, arch=args.arch, flavour=args.flavour)
+ else:
+ a.set(args.config, arch=args.arch, flavour=args.flavour, value=args.value, note=args.note)
+
+ # Save back to annotations
+ a.save(args.file)
+
+ # Query and print back the value
+ a = Annotation(args.file)
+ res = a.search_config(config=args.config)
+ print_result(args.config, res)
+
+
+def do_export(args):
+ if args.arch is None:
+ arg_fail('error: --export requires --arch')
+ a = Annotation(args.file)
+ conf = a.search_config(config=args.config, arch=args.arch, flavour=args.flavour)
+ if conf:
+ print(a.to_config(conf))
+
+
+def do_import(args):
+ if args.arch is None:
+ arg_fail('error: --arch is required with --import')
+ if args.flavour is None:
+ arg_fail('error: --flavour is required with --import')
+ if args.config is not None:
+ arg_fail('error: --config cannot be used with --import (try --update)')
+
+ # Merge with the current annotations
+ a = Annotation(args.file)
+ c = KConfig(args.import_file)
+ a.update(c, arch=args.arch, flavour=args.flavour)
+
+ # Save back to annotations
+ a.save(args.file)
+
+
+def do_update(args):
+ if args.arch is None:
+ arg_fail('error: --arch is required with --update')
+
+ # Merge with the current annotations
+ a = Annotation(args.file)
+ c = KConfig(args.update_file)
+ if args.config is None:
+ configs = list(set(c.config.keys()) - set(SKIP_CONFIGS))
+ if configs:
+ a.update(c, arch=args.arch, flavour=args.flavour, configs=configs)
+
+ # Save back to annotations
+ a.save(args.file)
+
+
+def do_check(args):
+ # Determine arch and flavour
+ if args.arch is None:
+ arg_fail('error: --arch is required with --check')
+
+ print(f"check-config: loading annotations from {args.file}")
+ total = good = ret = 0
+
+ # Load annotations settings
+ a = Annotation(args.file)
+ a_configs = a.search_config(arch=args.arch, flavour=args.flavour).keys()
+
+ # Parse target .config
+ c = KConfig(args.check_file)
+ c_configs = c.config.keys()
+
+ # Validate .config against annotations
+ for conf in sorted(a_configs | c_configs):
+ if conf in SKIP_CONFIGS:
+ continue
+ entry = a.search_config(config=conf, arch=args.arch, flavour=args.flavour)
+ expected = entry[conf] if entry else '-'
+ value = c.config[conf] if conf in c.config else '-'
+ if value != expected:
+ policy = a.config[conf] if conf in a.config else 'undefined'
+ if 'policy' in policy:
+ policy = f"policy<{policy['policy']}>"
+ print(f"check-config: FAIL: ({value} != {expected}): {conf} {policy})")
+ ret = 1
+ else:
+ good += 1
+ total += 1
+
+ print(f"check-config: {good}/{total} checks passed -- exit {ret}")
+ sys.exit(ret)
+
+
+def autodetect_annotations(args):
+ if args.file:
+ return
+ # If --file/-f isn't specified try to automatically determine the right
+ # location of the annotations file looking at debian/debian.env.
+ try:
+ with open('debian/debian.env', 'rt', encoding='utf-8') as fd:
+ args.file = fd.read().rstrip().split('=')[1] + '/config/annotations'
+ except (FileNotFoundError, IndexError):
+ arg_fail('error: could not determine DEBDIR, try using: --file/-f')
+
+
+def main():
+ # Prevent broken pipe errors when showing output in pipe to other tools
+ # (less for example)
+ signal(SIGPIPE, SIG_DFL)
+
+ # Main annotations program
+ args = _ARGPARSER.parse_args()
+ autodetect_annotations(args)
+
+ if args.config and not args.config.startswith('CONFIG_'):
+ args.config = 'CONFIG_' + args.config
+
+ if args.value:
+ do_write(args)
+ elif args.note:
+ do_note(args)
+ elif args.export:
+ do_export(args)
+ elif args.import_file:
+ do_import(args)
+ elif args.update_file:
+ do_update(args)
+ elif args.check_file:
+ do_check(args)
+ elif args.autocomplete:
+ do_autocomplete(args)
+ elif args.source:
+ do_source(args)
+ else:
+ do_query(args)
+
+
+if __name__ == '__main__':
+ main()
@@ -56,15 +56,17 @@ then
failure "$abi_version ABI version mismatch ($abi != $version)"
fi
-if [ -d debian/certs ]; then
- if ! grep -q '^CONFIG_SYSTEM_TRUSTED_KEYS="debian/canonical-certs.pem"$' $debian/config/config.common.ubuntu; then
- failure "'CONFIG_SYSTEM_TRUSTED_KEYS="debian/canonical-certs.pem"' is required"
+if [ -e $debian/config/config.common.ubuntu ]; then
+ if [ -d debian/certs ]; then
+ if ! grep -q '^CONFIG_SYSTEM_TRUSTED_KEYS="debian/canonical-certs.pem"$' $debian/config/config.common.ubuntu; then
+ failure "'CONFIG_SYSTEM_TRUSTED_KEYS="debian/canonical-certs.pem"' is required"
+ fi
fi
-fi
-if [ -d debian/revoked-certs ]; then
- if ! grep -q '^CONFIG_SYSTEM_REVOCATION_KEYS="debian/canonical-revoked-certs.pem"$' $debian/config/config.common.ubuntu; then
- failure "'CONFIG_SYSTEM_REVOCATION_KEYS="debian/canonical-revoked-certs.pem"' is required"
+ if [ -d debian/revoked-certs ]; then
+ if ! grep -q '^CONFIG_SYSTEM_REVOCATION_KEYS="debian/canonical-revoked-certs.pem"$' $debian/config/config.common.ubuntu; then
+ failure "'CONFIG_SYSTEM_REVOCATION_KEYS="debian/canonical-revoked-certs.pem"' is required"
+ fi
fi
fi
new file mode 100644
new file mode 100644
@@ -0,0 +1,416 @@
+#!/usr/bin/env python
+# -*- mode: python -*-
+# python module to manage Ubuntu kernel .config and annotations
+# Copyright © 2022 Canonical Ltd.
+
+import json
+import re
+import shutil
+import tempfile
+
+from abc import abstractmethod
+from ast import literal_eval
+from os.path import dirname, abspath
+
+
+class Config():
+ def __init__(self, fname):
+ """
+ Basic configuration file object
+ """
+ self.fname = fname
+ self.config = {}
+
+ raw_data = self._load(fname)
+ self._parse(raw_data)
+
+ @staticmethod
+ def _load(fname: str) -> str:
+ with open(fname, 'rt', encoding='utf-8') as fd:
+ data = fd.read()
+ return data.rstrip()
+
+ def __str__(self):
+ """ Return a JSON representation of the config """
+ return json.dumps(self.config, indent=4)
+
+ @abstractmethod
+ def _parse(self, data: str):
+ pass
+
+
+class KConfig(Config):
+ """
+ Parse a .config file, individual config options can be accessed via
+ .config[<CONFIG_OPTION>]
+ """
+ def _parse(self, data: str):
+ self.config = {}
+ for line in data.splitlines():
+ m = re.match(r'^# (CONFIG_.*) is not set$', line)
+ if m:
+ self.config[m.group(1)] = literal_eval("'n'")
+ continue
+ m = re.match(r'^(CONFIG_[A-Za-z0-9_]+)=(.*)$', line)
+ if m:
+ self.config[m.group(1)] = literal_eval("'" + m.group(2) + "'")
+ continue
+
+
+class Annotation(Config):
+ """
+ Parse body of annotations file
+ """
+ def _parse_body(self, data: str, parent=True):
+ for line in data.splitlines():
+ # Replace tabs with spaces, squeeze multiple into singles and
+ # remove leading and trailing spaces
+ line = line.replace('\t', ' ')
+ line = re.sub(r' +', ' ', line)
+ line = line.strip()
+
+ # Ignore empty lines
+ if not line:
+ continue
+
+ # Catpure flavors of included files
+ if line.startswith('# FLAVOUR: '):
+ self.include_flavour += line.split(' ')[2:]
+ continue
+
+ # Ignore comments
+ if line.startswith('#'):
+ continue
+
+ # Handle includes (recursively)
+ m = re.match(r'^include\s+"?([^"]*)"?', line)
+ if m:
+ if parent:
+ self.include.append(m.group(1))
+ include_fname = dirname(abspath(self.fname)) + '/' + m.group(1)
+ include_data = self._load(include_fname)
+ self._parse_body(include_data, parent=False)
+ continue
+
+ # Handle policy and note lines
+ if re.match(r'.* (policy|note)<', line):
+ try:
+ conf = line.split(' ')[0]
+ if conf in self.config:
+ entry = self.config[conf]
+ else:
+ entry = {'policy': {}}
+
+ match = False
+ m = re.match(r'.* policy<(.*?)>', line)
+ if m:
+ match = True
+ entry['policy'] |= literal_eval(m.group(1))
+
+ m = re.match(r'.* note<(.*?)>', line)
+ if m:
+ entry['oneline'] = match
+ match = True
+ entry['note'] = "'" + m.group(1).replace("'", '') + "'"
+
+ if not match:
+ raise Exception('syntax error')
+ self.config[conf] = entry
+ except Exception as e:
+ raise Exception(str(e) + f', line = {line}') from e
+ continue
+
+ # Invalid line
+ raise Exception(f'invalid line: {line}')
+
+ def _parse(self, data: str):
+ """
+ Parse main annotations file, individual config options can be accessed
+ via self.config[<CONFIG_OPTION>]
+ """
+ self.config = {}
+ self.arch = []
+ self.flavour = []
+ self.flavour_dep = {}
+ self.include = []
+ self.header = ''
+ self.include_flavour = []
+
+ # Parse header (only main header will considered, headers in includes
+ # will be treated as comments)
+ for line in data.splitlines():
+ if re.match(r'^#.*', line):
+ m = re.match(r'^# ARCH: (.*)', line)
+ if m:
+ self.arch = list(m.group(1).split(' '))
+ m = re.match(r'^# FLAVOUR: (.*)', line)
+ if m:
+ self.flavour = list(m.group(1).split(' '))
+ m = re.match(r'^# FLAVOUR_DEP: (.*)', line)
+ if m:
+ self.flavour_dep = literal_eval(m.group(1))
+ self.header += line + "\n"
+ else:
+ break
+
+ # Parse body (handle includes recursively)
+ self._parse_body(data)
+
+ # Sanity check: Verify that all FLAVOUR_DEP flavors are valid
+ for src, tgt in self.flavour_dep.items():
+ if src not in self.flavour:
+ raise Exception(f'Invalid source flavour in FLAVOUR_DEP: {src}')
+ if tgt not in self.include_flavour:
+ raise Exception(f'Invalid target flavour in FLAVOUR_DEP: {tgt}')
+
+ def _remove_entry(self, config: str):
+ if self.config[config]:
+ del self.config[config]
+
+ def remove(self, config: str, arch: str = None, flavour: str = None):
+ if config not in self.config:
+ return
+ if arch is not None:
+ if flavour is not None:
+ flavour = f'{arch}-{flavour}'
+ else:
+ flavour = arch
+ del self.config[config]['policy'][flavour]
+ if not self.config[config]['policy']:
+ self._remove_entry(config)
+ else:
+ self._remove_entry(config)
+
+ def set(self, config: str, arch: str = None, flavour: str = None,
+ value: str = None, note: str = None):
+ if value is not None:
+ if config not in self.config:
+ self.config[config] = {'policy': {}}
+ if arch is not None:
+ if flavour is not None:
+ flavour = f'{arch}-{flavour}'
+ else:
+ flavour = arch
+ self.config[config]['policy'][flavour] = value
+ else:
+ for a in self.arch:
+ self.config[config]['policy'][a] = value
+ if note is not None:
+ self.config[config]['note'] = "'" + note.replace("'", '') + "'"
+
+ def update(self, c: KConfig, arch: str, flavour: str = None, configs: list = None):
+ """ Merge configs from a Kconfig object into Annotation object """
+
+ # Determine if we need to import all configs or a single config
+ if not configs:
+ configs = c.config.keys()
+ configs |= self.search_config(arch=arch, flavour=flavour).keys()
+
+ # Import configs from the Kconfig object into Annotations
+ if flavour is not None:
+ flavour = arch + f'-{flavour}'
+ else:
+ flavour = arch
+ for conf in configs:
+ if conf in c.config:
+ val = c.config[conf]
+ else:
+ val = '-'
+ if conf in self.config:
+ if 'policy' in self.config[conf]:
+ self.config[conf]['policy'][flavour] = val
+ else:
+ self.config[conf]['policy'] = {flavour: val}
+ else:
+ self.config[conf] = {'policy': {flavour: val}}
+
+ def _compact(self):
+ # Try to remove redundant settings: if the config value of a flavour is
+ # the same as the one of the main arch simply drop it.
+ for conf in self.config.copy():
+ if 'policy' not in self.config[conf]:
+ continue
+ for flavour in self.flavour:
+ if flavour not in self.config[conf]['policy']:
+ continue
+ m = re.match(r'^(.*?)-(.*)$', flavour)
+ if not m:
+ continue
+ arch = m.group(1)
+ if arch in self.config[conf]['policy']:
+ if self.config[conf]['policy'][flavour] == self.config[conf]['policy'][arch]:
+ del self.config[conf]['policy'][flavour]
+ continue
+ if flavour not in self.flavour_dep:
+ continue
+ generic = self.flavour_dep[flavour]
+ if generic in self.config[conf]['policy']:
+ if self.config[conf]['policy'][flavour] == self.config[conf]['policy'][generic]:
+ del self.config[conf]['policy'][flavour]
+ continue
+ # Remove rules for flavours / arches that are not supported (not
+ # listed in the annotations header).
+ for flavour in self.config[conf]['policy'].copy():
+ if flavour not in list(set(self.arch + self.flavour)):
+ del self.config[conf]['policy'][flavour]
+ # Remove configs that are all undefined across all arches/flavours
+ # (unless we have includes)
+ if not self.include:
+ if 'policy' in self.config[conf]:
+ if list(set(self.config[conf]['policy'].values())) == ['-']:
+ self.config[conf]['policy'] = {}
+ # Drop empty rules
+ if not self.config[conf]['policy']:
+ del self.config[conf]
+ else:
+ # Compact same value across all flavour within the same arch
+ for arch in self.arch:
+ arch_flavours = [i for i in self.flavour if i.startswith(arch)]
+ value = None
+ for flavour in arch_flavours:
+ if flavour not in self.config[conf]['policy']:
+ break
+ if value is None:
+ value = self.config[conf]['policy'][flavour]
+ elif value != self.config[conf]['policy'][flavour]:
+ break
+ else:
+ for flavour in arch_flavours:
+ del self.config[conf]['policy'][flavour]
+ self.config[conf]['policy'][arch] = value
+ # After the first round of compaction we may end up having configs that
+ # are undefined across all arches, so do another round of compaction to
+ # drop these settings that are not needed anymore
+ # (unless we have includes).
+ if not self.include:
+ for conf in self.config.copy():
+ # Remove configs that are all undefined across all arches/flavours
+ if 'policy' in self.config[conf]:
+ if list(set(self.config[conf]['policy'].values())) == ['-']:
+ self.config[conf]['policy'] = {}
+ # Drop empty rules
+ if not self.config[conf]['policy']:
+ del self.config[conf]
+
+ @staticmethod
+ def _sorted(config):
+ """ Sort configs alphabetically but return configs with a note first """
+ w_note = []
+ wo_note = []
+ for c in sorted(config):
+ if 'note' in config[c]:
+ w_note.append(c)
+ else:
+ wo_note.append(c)
+ return w_note + wo_note
+
+ def save(self, fname: str):
+ """ Save annotations data to the annotation file """
+ # Compact annotations structure
+ self._compact()
+
+ # Save annotations to disk
+ with tempfile.NamedTemporaryFile(mode='w+t', delete=False) as tmp:
+ # Write header
+ tmp.write(self.header + '\n')
+
+ # Write includes
+ for i in self.include:
+ tmp.write(f'include "{i}"\n')
+ if self.include:
+ tmp.write("\n")
+
+ # Write config annotations and notes
+ tmp.flush()
+ shutil.copy(tmp.name, fname)
+ tmp_a = Annotation(fname)
+
+ # Only save local differences (preserve includes)
+ marker = False
+ for conf in self._sorted(self.config):
+ new_val = self.config[conf]
+ if 'policy' not in new_val:
+ continue
+
+ # If new_val is a subset of old_val, skip it
+ old_val = tmp_a.config.get(conf)
+ if old_val and 'policy' in old_val:
+ if old_val['policy'] == old_val['policy'] | new_val['policy']:
+ continue
+
+ # Write out the policy (and note) line(s)
+ val = dict(sorted(new_val['policy'].items()))
+ line = f"{conf : <47} policy<{val}>"
+ if 'note' in new_val:
+ val = new_val['note']
+ if new_val.get('oneline', False):
+ # Single line
+ line += f' note<{val}>'
+ else:
+ # Separate policy and note lines,
+ # followed by an empty line
+ line += f'\n{conf : <47} note<{val}>\n'
+ elif not marker:
+ # Write out a marker indicating the start of annotations
+ # without notes
+ tmp.write('\n# ---- Annotations without notes ----\n\n')
+ marker = True
+ tmp.write(line + "\n")
+
+ # Replace annotations with the updated version
+ tmp.flush()
+ shutil.move(tmp.name, fname)
+
+ def search_config(self, config: str = None, arch: str = None, flavour: str = None) -> dict:
+ """ Return config value of a specific config option or architecture """
+ if flavour is None:
+ flavour = 'generic'
+ flavour = f'{arch}-{flavour}'
+ if flavour in self.flavour_dep:
+ generic = self.flavour_dep[flavour]
+ else:
+ generic = flavour
+ if config is None and arch is None:
+ # Get all config options for all architectures
+ return self.config
+ if config is None and arch is not None:
+ # Get config options of a specific architecture
+ ret = {}
+ for c, val in self.config.items():
+ if 'policy' not in val:
+ continue
+ if flavour in val['policy']:
+ ret[c] = val['policy'][flavour]
+ elif generic != flavour and generic in val['policy']:
+ ret[c] = val['policy'][generic]
+ elif arch in val['policy']:
+ ret[c] = val['policy'][arch]
+ return ret
+ if config is not None and arch is None:
+ # Get a specific config option for all architectures
+ return self.config[config] if config in self.config else None
+ if config is not None and arch is not None:
+ # Get a specific config option for a specific architecture
+ if config in self.config:
+ if 'policy' in self.config[config]:
+ if flavour in self.config[config]['policy']:
+ return {config: self.config[config]['policy'][flavour]}
+ if generic != flavour and generic in self.config[config]['policy']:
+ return {config: self.config[config]['policy'][generic]}
+ if arch in self.config[config]['policy']:
+ return {config: self.config[config]['policy'][arch]}
+ return None
+
+ @staticmethod
+ def to_config(data: dict) -> str:
+ """ Convert annotations data to .config format """
+ s = ''
+ for c in data:
+ v = data[c]
+ if v == 'n':
+ s += f"# {c} is not set\n"
+ elif v == '-':
+ pass
+ else:
+ s += f"{c}={v}\n"
+ return s.rstrip()
@@ -1,207 +1,177 @@
-#!/bin/bash
-
-. debian/debian.env
+#!/bin/bash -u
+#
+# Manage kernel config annotations
+#
+# Supported environment variales:
+# conc_level : Concurrency level for upstream make (-jX)
+# skip_checks : Skip config checks if set to 'true'
+# gcc : Default gcc to use (mandatory)
+#
+
+function cleanup()
+{
+ rm -rf build "${TMP_DIR}"
+}
+
+# We have to be in the top level Ubuntu kernel source directory
+if ! [ -e debian/debian.env ] ; then
+ echo "ERROR: This is not an Ubuntu kernel source directory" >&2
+ exit 1
+fi
-# Script to merge all configs and run 'make syncconfig' on it to wade out bad juju.
-# Then split the configs into distro-commmon and flavour-specific parts
+if [ -z "${gcc:-}" ] ; then
+ gcc=gcc
+fi
-# We have to be in the top level kernel source directory
-if [ ! -f MAINTAINERS ] || [ ! -f Makefile ]; then
- echo "This does not appear to be the kernel source directory." 1>&2
- exit 1
+if [ ${#} -ne 1 ] ; then
+ echo "Usage: $0 updateconfigs|defaultconfigs|genconfigs|editconfigs"
+ exit 2
fi
-mode=${1:?"Usage: $0 (oldconfig|editconfig) [do_enforce_all]"}
-do_enforce_all=${2:-0}
-yes=0
-case "$mode" in
- update*configs) mode='syncconfig' ;;
- default*configs) mode='oldconfig'; yes=1 ;;
- edit*configs) ;; # All is good
- gen*configs) mode='genconfigs' ;; # All is good
- dump*configs) mode='config'; yes=1 ;;
- *) echo "$0 called with invalid mode" 1>&2
- exit 1 ;;
+mode=${1}
+
+case "${mode}" in
+ updateconfigs) target="syncconfig" ;;
+ defaultconfigs) target="olddefconfig" ;;
+ genconfigs) target="oldconfig" ;;
+ editconfigs) ;; # Target is set later based on user input
+ *) echo "ERROR: Invalid mode: ${1}" >&2
+ exit 1 ;;
esac
-kerneldir="`pwd`"
-confdir="$kerneldir/${DEBIAN}/config"
-variant="$2"
-. $DEBIAN/etc/kernelconfig
+. debian/debian.env
-bindir="`pwd`/${DROOT}/scripts/misc"
-common_conf="$confdir/config.common.$family"
-tmpdir=`mktemp -d`
-mkdir "$tmpdir/CONFIGS"
+annotations_file=${DEBIAN}/config/annotations
+warning_partial=()
-if [ "$mode" = "genconfigs" ]; then
- keep=1
- mode="oldconfig"
- test -d CONFIGS || mkdir CONFIGS
-fi
+TMP_DIR=$(mktemp -d)
+trap cleanup EXIT
-warning_partial=
+# Use annotations to generate configs
+FLAVOURS=$(sed -ne 's/^# FLAVOUR: //p' "${annotations_file}")
-for arch in $archs; do
- rm -rf build
- mkdir build
+for arch_flavour in ${FLAVOURS} ; do
+ arch=${arch_flavour%%-*}
+ flavour=${arch_flavour#*-}
+ tmp_conf_file=${TMP_DIR}/${arch}-config.flavour.${flavour}
# Map debian archs to kernel archs
- case "$arch" in
- ppc64|ppc64el) kernarch="powerpc" ;;
- amd64) kernarch="x86_64" ;;
- lpia) kernarch="x86" ;;
- sparc) kernarch="sparc64" ;;
- armel|armhf) kernarch="arm" ;;
- s390x) kernarch="s390" ;;
- riscv64) kernarch="riscv" ;;
- *) kernarch="$arch" ;;
+ case "${arch}" in
+ amd64) kern_arch="x86_64" ;;
+ arm64) kern_arch="arm64" ;;
+ armhf) kern_arch="arm" ;;
+ ppc64el) kern_arch="powerpc" ;;
+ riscv64) kern_arch="riscv" ;;
+ s390x) kern_arch="s390" ;;
+ *) echo "WARNING: Unsupported architecture: ${arch}"
+ warning_partial+=("${arch}")
+ continue ;;
esac
# Determine cross toolchain to use for Kconfig compiler tests
- cross_compile=""
- deb_build_arch=$(dpkg-architecture -qDEB_BUILD_ARCH -a$arch 2>/dev/null)
- deb_host_arch=$(dpkg-architecture -qDEB_HOST_ARCH -a$arch 2>/dev/null)
- [ $deb_build_arch != $deb_host_arch ] && cross_compile="$(dpkg-architecture -qDEB_HOST_GNU_TYPE -a$arch 2>/dev/null)-"
-
- # Environment variables for 'make *config'. We omit CROSS_COMPILE
- # for i386 since it is no longer supported after 19.04, however
- # we maintain the configs for hwe.
- modify_config=true
- env="ARCH=$kernarch DEB_ARCH=$arch"
- compiler_path=$(which "${cross_compile}gcc" || true)
- if [ "$compiler_path" != '' ]; then
- env="$env CROSS_COMPILE=$cross_compile"
- else
- echo "WARNING: ${cross_compile}gcc not installed"
- modify_config=
- warning_partial="$warning_partial $arch"
+ cross_compile="$(dpkg-architecture -qDEB_HOST_GNU_TYPE -a"${arch}" 2>/dev/null)-"
+
+ # Arch-specific compiler, if any
+ arch_gcc=$(cat <<EOF | make -s -f - all
+include ${DEBIAN}/rules.d/${arch}.mk
+all:
+ @echo \$(if \$(gcc),\$(gcc),${gcc})
+EOF
+ )
+ gcc_path=$(which "${cross_compile}${arch_gcc}" || true)
+ if [ -z "${gcc_path}" ] ; then
+ echo "WARNING: ${cross_compile}${arch_gcc} not installed"
+ warning_partial+=("${arch}")
+ continue
fi
- archconfdir=$confdir/$arch
- flavourconfigs=$(cd $archconfdir && ls config.flavour.*)
-
- # Merge configs
- # We merge config.common.ubuntu + config.common.<arch> +
- # config.flavour.<flavour>
-
- for config in $flavourconfigs; do
- fullconf="$tmpdir/$arch-$config-full"
- case $config in
- *)
- : >"$fullconf"
- if [ -f $common_conf ]; then
- cat $common_conf >> "$fullconf"
- fi
- if [ -f $archconfdir/config.common.$arch ]; then
- cat $archconfdir/config.common.$arch >> "$fullconf"
- fi
- cat "$archconfdir/$config" >>"$fullconf"
- if [ -f $confdir/OVERRIDES ]; then
- cat $confdir/OVERRIDES >> "$fullconf"
- fi
- ;;
- esac
- done
+ if [ "${mode}" = "editconfigs" ] ; then
+ while true ; do
+ echo -n "Do you want to edit configs for ${arch}-${flavour}? [Y/n] "
+ read -r choice
+ case "${choice,,}" in
+ y|"") target="menuconfig" ; break ;;
+ n) target="syncconfig" ; break ;;
+ esac
+ done
+ fi
- for config in $flavourconfigs; do
- if [ -f $archconfdir/$config ]; then
- fullconf="$tmpdir/$arch-$config-full"
- cat "$fullconf" > build/.config
- # Call oldconfig or menuconfig
- if [ "$modify_config" ]; then
- case "$mode" in
- editconfigs)
- # Interactively edit config parameters
- while : ; do
- echo -n "Do you want to edit config: $arch/$config? [Y/n] "
- read choice
- case "$choice" in
- y* | Y* | "" )
- make O=`pwd`/build $env menuconfig
- break ;;
- n* | N* )
- # 'syncconfig' prevents
- # errors for '-' options set
- # in common config fragments
- make O=`pwd`/build $env syncconfig
- break ;;
- *)
- echo "Entry not valid"
- esac
- done
- ;;
- *)
- echo "* Run $mode (yes=$yes) on $arch/$config ..."
- if [ "$yes" -eq 1 ]; then
- yes "" | make O=`pwd`/build $env "$mode"
- else
- make O=`pwd`/build $env "$mode"
- fi ;;
- esac
- fi
- cat build/.config > $archconfdir/$config
- [ "$modify_config" ] && cat build/.config >"$tmpdir/CONFIGS/$arch-$config"
- if [ "$keep" = "1" ]; then
- cat build/.config > CONFIGS/$arch-$config
- fi
- else
- echo "!! Config not found $archconfdir/$config..."
- fi
- done
+ rm -rf build
+ mkdir build
- echo "Running splitconfig.pl for $arch"
+ # Generate .config from annotations
+ python3 debian/scripts/misc/annotations -f "${annotations_file}" \
+ --arch "${arch}" --flavour "${flavour}" --export > build/.config
+
+ # Environment variables for 'make *config'
+ env=(ARCH="${kern_arch}"
+ DEB_ARCH="${arch}"
+ CROSS_COMPILE="${cross_compile}"
+ CC="${gcc_path}")
+
+ # Concurrency level
+ if [ -n "${conc_level:-}" ] ; then
+ env+=("${conc_level}")
+ fi
+
+ # Call config target
echo
+ echo "* Run ${target} on ${arch}/${flavour} ..."
+ ${kmake} O=build "${env[@]}" "${target}"
- # Can we make this more robust by avoiding $tmpdir completely?
- # This approach was used for now because I didn't want to change
- # splitconfig.pl
- (cd $archconfdir; $bindir/splitconfig.pl config.flavour.*; mv config.common \
- config.common.$arch; cp config.common.$arch $tmpdir)
+ # Move config for further processing
+ mv build/.config "${tmp_conf_file}"
done
-rm -f $common_conf
-
-# Now run splitconfig.pl on all the config.common.<arch> copied to
-# $tmpdir
-(cd $tmpdir; $bindir/splitconfig.pl *)
-(
- cd $confdir;
- rm -f *-full
- grep -v 'is UNMERGABLE' <$tmpdir/config.common >$common_conf
- for arch in $archs; do
- grep -v 'is UNMERGABLE' <$tmpdir/config.common.$arch \
- >$arch/config.common.$arch
- done
-)
-
-echo ""
-echo "Running config-check for all configurations ..."
-echo ""
-fail=0
-for arch in $archs; do
- archconfdir=$confdir/$arch
- flavourconfigs=$(cd $archconfdir && ls config.flavour.*)
- for config in $flavourconfigs; do
- flavour="${config##*.}"
- if [ -f $archconfdir/$config ]; then
- fullconf="$tmpdir/CONFIGS/$arch-$config"
- [ ! -f "$fullconf" ] && continue
- "$bindir/../config-check" "$fullconf" "$arch" "$flavour" "$confdir" "0" "$do_enforce_all" || let "fail=$fail+1"
- fi
+rc=0
+
+if [ "${skip_checks:-}" = "true" ] ; then
+ echo
+ echo "Skipping config-check (skip_checks=${skip_checks}) ..."
+else
+ echo
+ echo "Running config-check for all configurations ..."
+ fail=0
+ for arch_flavour in ${FLAVOURS} ; do
+ arch=${arch_flavour%%-*}
+ flavour=${arch_flavour#*-}
+ tmp_conf_file=${TMP_DIR}/${arch}-config.flavour.${flavour}
+
+ echo
+ echo "* Run config-check for ${arch}-${flavour} ..."
+ python3 debian/scripts/misc/annotations -f "${annotations_file}" \
+ --arch "${arch}" --flavour "${flavour}" --check "${tmp_conf_file}" || \
+ fail=$((fail + 1))
done
-done
-if [ "$fail" != 0 ]; then
- echo ""
- echo "*** ERROR: $fail config-check failures detected"
- echo ""
+ if [ ${fail} -gt 0 ] ; then
+ rc=1
+ echo "ERROR: ${fail} config-check failures detected" >&2
+ fi
fi
-rm -rf build
+if [ ${#warning_partial[@]} -gt 0 ] ; then
+ rc=1
+ echo "ERROR: Config operation not applied to all architectures (skipped ${warning_partial[*]})" >&2
+fi
-if [ "$warning_partial" ]; then
- echo ""
- echo "WARNING: configuration operation applied only to a subset of architectures (skipped$warning_partial)" 1>&2
- echo ""
+# Recreate the annotations file
+if [ "${mode}" = "genconfigs" ] ; then
+ rm -rf CONFIGS
+ mv "${TMP_DIR}" CONFIGS
+else
+ echo
+ echo "Importing all configurations ..."
+ echo
+ for arch_flavour in ${FLAVOURS} ; do
+ arch=${arch_flavour%%-*}
+ flavour=${arch_flavour#*-}
+ tmp_conf_file=${TMP_DIR}/${arch}-config.flavour.${flavour}
+
+ echo "* Import configs for ${arch}-${flavour} ..."
+ python3 debian/scripts/misc/annotations -f "${annotations_file}" \
+ --arch "${arch}" --flavour "${flavour}" --import "${tmp_conf_file}"
+ done
fi
+
+exit "${rc}"
new file mode 100755
@@ -0,0 +1,35 @@
+#!/bin/bash
+
+. debian/debian.env
+
+# We have to be in the top level kernel source directory
+if [ ! -f MAINTAINERS ] || [ ! -f Makefile ]; then
+ echo "This does not appear to be the kernel source directory." 1>&2
+ exit 1
+fi
+
+. ${DEBIAN}/etc/kernelconfig
+
+ARCH=$(echo $archs | tr " " "\n" | sort -u)
+FLAVOUR=$(for arch in ${ARCH}; do
+ flavours=$(sed -ne 's/^flavours\s*=\s*\(.*\)$/\1/p' ${DEBIAN}/rules.d/${arch}.mk)
+ for flavour in ${flavours}; do
+ echo ${arch}-${flavour}
+ done
+done | tr " " "\n" | sort -u)
+
+while read line; do
+ (echo $line | grep -q '^#') || break
+done
+
+cat << EOF
+# Menu: HEADER
+# FORMAT: 4
+# ARCH: $(echo ${ARCH})
+# FLAVOUR: $(echo ${FLAVOUR})
+
+EOF
+
+while read line; do
+ echo ${line}
+done
new file mode 100644
@@ -0,0 +1,207 @@
+#!/bin/bash
+
+. debian/debian.env
+
+# Script to merge all configs and run 'make syncconfig' on it to wade out bad juju.
+# Then split the configs into distro-commmon and flavour-specific parts
+
+# We have to be in the top level kernel source directory
+if [ ! -f MAINTAINERS ] || [ ! -f Makefile ]; then
+ echo "This does not appear to be the kernel source directory." 1>&2
+ exit 1
+fi
+
+mode=${1:?"Usage: $0 (oldconfig|editconfig) [do_enforce_all]"}
+do_enforce_all=${2:-0}
+yes=0
+case "$mode" in
+ update*configs) mode='syncconfig' ;;
+ default*configs) mode='oldconfig'; yes=1 ;;
+ edit*configs) ;; # All is good
+ gen*configs) mode='genconfigs' ;; # All is good
+ dump*configs) mode='config'; yes=1 ;;
+ *) echo "$0 called with invalid mode" 1>&2
+ exit 1 ;;
+esac
+kerneldir="`pwd`"
+confdir="$kerneldir/${DEBIAN}/config"
+variant="$2"
+
+. $DEBIAN/etc/kernelconfig
+
+bindir="`pwd`/${DROOT}/scripts/misc"
+common_conf="$confdir/config.common.$family"
+tmpdir=`mktemp -d`
+mkdir "$tmpdir/CONFIGS"
+
+if [ "$mode" = "genconfigs" ]; then
+ keep=1
+ mode="oldconfig"
+ test -d CONFIGS || mkdir CONFIGS
+fi
+
+warning_partial=
+
+for arch in $archs; do
+ rm -rf build
+ mkdir build
+
+ # Map debian archs to kernel archs
+ case "$arch" in
+ ppc64|ppc64el) kernarch="powerpc" ;;
+ amd64) kernarch="x86_64" ;;
+ lpia) kernarch="x86" ;;
+ sparc) kernarch="sparc64" ;;
+ armel|armhf) kernarch="arm" ;;
+ s390x) kernarch="s390" ;;
+ riscv64) kernarch="riscv" ;;
+ *) kernarch="$arch" ;;
+ esac
+
+ # Determine cross toolchain to use for Kconfig compiler tests
+ cross_compile=""
+ deb_build_arch=$(dpkg-architecture -qDEB_BUILD_ARCH -a$arch 2>/dev/null)
+ deb_host_arch=$(dpkg-architecture -qDEB_HOST_ARCH -a$arch 2>/dev/null)
+ [ $deb_build_arch != $deb_host_arch ] && cross_compile="$(dpkg-architecture -qDEB_HOST_GNU_TYPE -a$arch 2>/dev/null)-"
+
+ # Environment variables for 'make *config'. We omit CROSS_COMPILE
+ # for i386 since it is no longer supported after 19.04, however
+ # we maintain the configs for hwe.
+ modify_config=true
+ env="ARCH=$kernarch DEB_ARCH=$arch"
+ compiler_path=$(which "${cross_compile}gcc" || true)
+ if [ "$compiler_path" != '' ]; then
+ env="$env CROSS_COMPILE=$cross_compile"
+ else
+ echo "WARNING: ${cross_compile}gcc not installed"
+ modify_config=
+ warning_partial="$warning_partial $arch"
+ fi
+
+ archconfdir=$confdir/$arch
+ flavourconfigs=$(cd $archconfdir && ls config.flavour.*)
+
+ # Merge configs
+ # We merge config.common.ubuntu + config.common.<arch> +
+ # config.flavour.<flavour>
+
+ for config in $flavourconfigs; do
+ fullconf="$tmpdir/$arch-$config-full"
+ case $config in
+ *)
+ : >"$fullconf"
+ if [ -f $common_conf ]; then
+ cat $common_conf >> "$fullconf"
+ fi
+ if [ -f $archconfdir/config.common.$arch ]; then
+ cat $archconfdir/config.common.$arch >> "$fullconf"
+ fi
+ cat "$archconfdir/$config" >>"$fullconf"
+ if [ -f $confdir/OVERRIDES ]; then
+ cat $confdir/OVERRIDES >> "$fullconf"
+ fi
+ ;;
+ esac
+ done
+
+ for config in $flavourconfigs; do
+ if [ -f $archconfdir/$config ]; then
+ fullconf="$tmpdir/$arch-$config-full"
+ cat "$fullconf" > build/.config
+ # Call oldconfig or menuconfig
+ if [ "$modify_config" ]; then
+ case "$mode" in
+ editconfigs)
+ # Interactively edit config parameters
+ while : ; do
+ echo -n "Do you want to edit config: $arch/$config? [Y/n] "
+ read choice
+ case "$choice" in
+ y* | Y* | "" )
+ make O=`pwd`/build $env menuconfig
+ break ;;
+ n* | N* )
+ # 'syncconfig' prevents
+ # errors for '-' options set
+ # in common config fragments
+ make O=`pwd`/build $env syncconfig
+ break ;;
+ *)
+ echo "Entry not valid"
+ esac
+ done
+ ;;
+ *)
+ echo "* Run $mode (yes=$yes) on $arch/$config ..."
+ if [ "$yes" -eq 1 ]; then
+ yes "" | make O=`pwd`/build $env "$mode"
+ else
+ make O=`pwd`/build $env "$mode"
+ fi ;;
+ esac
+ fi
+ cat build/.config > $archconfdir/$config
+ [ "$modify_config" ] && cat build/.config >"$tmpdir/CONFIGS/$arch-$config"
+ if [ "$keep" = "1" ]; then
+ cat build/.config > CONFIGS/$arch-$config
+ fi
+ else
+ echo "!! Config not found $archconfdir/$config..."
+ fi
+ done
+
+ echo "Running splitconfig.pl for $arch"
+ echo
+
+ # Can we make this more robust by avoiding $tmpdir completely?
+ # This approach was used for now because I didn't want to change
+ # splitconfig.pl
+ (cd $archconfdir; $bindir/splitconfig.pl config.flavour.*; mv config.common \
+ config.common.$arch; cp config.common.$arch $tmpdir)
+done
+
+rm -f $common_conf
+
+# Now run splitconfig.pl on all the config.common.<arch> copied to
+# $tmpdir
+(cd $tmpdir; $bindir/splitconfig.pl *)
+(
+ cd $confdir;
+ rm -f *-full
+ grep -v 'is UNMERGABLE' <$tmpdir/config.common >$common_conf
+ for arch in $archs; do
+ grep -v 'is UNMERGABLE' <$tmpdir/config.common.$arch \
+ >$arch/config.common.$arch
+ done
+)
+
+echo ""
+echo "Running config-check for all configurations ..."
+echo ""
+fail=0
+for arch in $archs; do
+ archconfdir=$confdir/$arch
+ flavourconfigs=$(cd $archconfdir && ls config.flavour.*)
+ for config in $flavourconfigs; do
+ flavour="${config##*.}"
+ if [ -f $archconfdir/$config ]; then
+ fullconf="$tmpdir/CONFIGS/$arch-$config"
+ [ ! -f "$fullconf" ] && continue
+ "$bindir/../config-check" "$fullconf" "$arch" "$flavour" "$confdir" "0" "$do_enforce_all" || let "fail=$fail+1"
+ fi
+ done
+done
+
+if [ "$fail" != 0 ]; then
+ echo ""
+ echo "*** ERROR: $fail config-check failures detected"
+ echo ""
+fi
+
+rm -rf build
+
+if [ "$warning_partial" ]; then
+ echo ""
+ echo "WARNING: configuration operation applied only to a subset of architectures (skipped$warning_partial)" 1>&2
+ echo ""
+fi
@@ -8,4 +8,8 @@ else
arch := $(ARCH)
endif
config:
- cat debian.$(branch)/config/config.common.ubuntu debian.$(branch)/config/$(arch)/config.common.$(arch) debian.$(branch)/config/$(arch)/config.flavour.$(flavour) >.config
+ if [ -e debian/config/config.common.ubuntu ]; then \
+ cat debian.$(branch)/config/config.common.ubuntu debian.$(branch)/config/$(arch)/config.common.$(arch) debian.$(branch)/config/$(arch)/config.flavour.$(flavour) >.config; \
+ else \
+ python3 debian/scripts/misc/annotations --export --arch $(arch) --flavour $(flavour) >.config; \
+ fi
BugLink: https://bugs.launchpad.net/bugs/2019000 Import the scripts to support the new annotations model introduced with lunar. This allows to get rid of the duplicated annotations + config chunks configuration and use the annotations-only model. Signed-off-by: Andrea Righi <andrea.righi@canonical.com> --- debian.master/config/README.rst | 185 +++++++++ debian/rules.d/1-maintainer.mk | 22 +- debian/rules.d/2-binary-arch.mk | 9 +- debian/rules.d/4-checks.mk | 12 +- debian/scripts/misc/annotations | 274 ++++++++++++++ debian/scripts/misc/final-checks | 16 +- debian/scripts/misc/kconfig/__init__.py | 0 debian/scripts/misc/kconfig/annotations.py | 416 +++++++++++++++++++++ debian/scripts/misc/kernelconfig | 326 ++++++++-------- debian/scripts/misc/migrate-annotations | 35 ++ debian/scripts/misc/old-kernelconfig | 207 ++++++++++ debian/snapcraft.mk | 6 +- 12 files changed, 1314 insertions(+), 194 deletions(-) create mode 100644 debian.master/config/README.rst create mode 100755 debian/scripts/misc/annotations create mode 100644 debian/scripts/misc/kconfig/__init__.py create mode 100644 debian/scripts/misc/kconfig/annotations.py create mode 100755 debian/scripts/misc/migrate-annotations create mode 100644 debian/scripts/misc/old-kernelconfig