From patchwork Thu Nov 19 14:53:54 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Petazzoni X-Patchwork-Id: 1403075 Return-Path: X-Original-To: incoming-buildroot@patchwork.ozlabs.org Delivered-To: patchwork-incoming-buildroot@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=busybox.net (client-ip=140.211.166.138; helo=whitealder.osuosl.org; envelope-from=buildroot-bounces@busybox.net; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=bootlin.com Received: from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4CcN4K0cmqz9sT6 for ; Fri, 20 Nov 2020 01:54:25 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by whitealder.osuosl.org (Postfix) with ESMTP id 6A88F86E15; Thu, 19 Nov 2020 14:54:23 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from whitealder.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id mIUdZ8v4ZwXi; Thu, 19 Nov 2020 14:54:16 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by whitealder.osuosl.org (Postfix) with ESMTP id 2F2CC86E3E; Thu, 19 Nov 2020 14:54:16 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from silver.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by ash.osuosl.org (Postfix) with ESMTP id 76C1C1BF57F for ; Thu, 19 Nov 2020 14:54:12 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by silver.osuosl.org (Postfix) with ESMTP id 61EBD204E2 for ; Thu, 19 Nov 2020 14:54:12 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from silver.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id VE2FRkT+gH0P for ; Thu, 19 Nov 2020 14:54:10 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from relay3-d.mail.gandi.net (relay3-d.mail.gandi.net [217.70.183.195]) by silver.osuosl.org (Postfix) with ESMTPS id D2A4B2043B for ; Thu, 19 Nov 2020 14:54:09 +0000 (UTC) X-Originating-IP: 86.201.80.25 Received: from localhost (lfbn-tou-1-236-25.w86-201.abo.wanadoo.fr [86.201.80.25]) (Authenticated sender: thomas.petazzoni@bootlin.com) by relay3-d.mail.gandi.net (Postfix) with ESMTPSA id AA1FC60002; Thu, 19 Nov 2020 14:54:05 +0000 (UTC) From: Thomas Petazzoni To: Buildroot List Date: Thu, 19 Nov 2020 15:53:54 +0100 Message-Id: <20201119145354.1175043-4-thomas.petazzoni@bootlin.com> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20201119145354.1175043-1-thomas.petazzoni@bootlin.com> References: <20201119145354.1175043-1-thomas.petazzoni@bootlin.com> MIME-Version: 1.0 Subject: [Buildroot] [PATCH 4/4] utils/getdeveloperlib.py: use relative paths for files X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Matt Weber , Heiko Thiery , Thomas Petazzoni , Titouan Christophe Errors-To: buildroot-bounces@busybox.net Sender: "buildroot" Using absolute paths within getdeveloperlib isn't very sensible, it makes a lot more sense to handle everything as relative paths from the top-level Buildroot source directory. parse_developers() is changed to no longer take the base path as argument: it is automatically calculated based on the location of utils/getdeveloperlib.py. Then, the rest of the logic is adjusted to use relative paths, and prepend them with the base "brpath" when needed. This commit allows pkg-stats to report correct developers information even when executed from an out of tree directory. Before this patch: $ ~/buildroot/support/scripts/pkg-stats -p ipmitool --json out.json $ cat out.json | jq '.packages.ipmitool.developers' [] $ cat out.json | jq '.defconfigs.stm32f469_disco' { "name": "stm32f469_disco", "path": "configs/stm32f469_disco_defconfig", "developers": [] } After this patch: $ ~/buildroot/support/scripts/pkg-stats -p ipmitool --json out.json $ cat out.json | jq '.packages.ipmitool.developers' [ "Floris Bos ", "Heiko Thiery " ] $ cat out.json | jq '.defconfigs.stm32f469_disco' { "name": "stm32f469_disco", "path": "configs/stm32f469_disco_defconfig", "developers": [ "Christophe Priouzeau " ] } Signed-off-by: Thomas Petazzoni Tested-by: Heiko Thiery --- support/scripts/pkg-stats | 2 +- utils/get-developers | 5 ----- utils/getdeveloperlib.py | 26 ++++++++++++-------------- 3 files changed, 13 insertions(+), 20 deletions(-) diff --git a/support/scripts/pkg-stats b/support/scripts/pkg-stats index e472b67784..f6460cf6f2 100755 --- a/support/scripts/pkg-stats +++ b/support/scripts/pkg-stats @@ -969,7 +969,7 @@ def __main__(): print("Build package list ...") packages = get_pkglist(args.npackages, package_list) print("Getting developers ...") - developers = parse_developers(brpath) + developers = parse_developers() print("Build defconfig list ...") defconfigs = get_defconfig_list() for d in defconfigs: diff --git a/utils/get-developers b/utils/get-developers index 20272ed60b..e027c26562 100755 --- a/utils/get-developers +++ b/utils/get-developers @@ -45,10 +45,6 @@ def __main__(): print("No action specified") return - # getdeveloperlib expects to be executed from the toplevel buildroot - # directory, which is one level up from this script - os.chdir(os.path.join(os.path.dirname(os.path.realpath(__file__)), '..')) - devs = getdeveloperlib.parse_developers() if devs is None: sys.exit(1) @@ -75,7 +71,6 @@ def __main__(): # Handle the files action if args.files is not None: - args.files = [os.path.abspath(f) for f in args.files] for dev in devs: for f in args.files: if dev.hasfile(f): diff --git a/utils/getdeveloperlib.py b/utils/getdeveloperlib.py index f57f41887b..d7a90457ed 100644 --- a/utils/getdeveloperlib.py +++ b/utils/getdeveloperlib.py @@ -6,6 +6,8 @@ import subprocess import sys import unittest +brpath = os.path.normpath(os.path.join(os.path.dirname(__file__), "..")) + # # Patch parsing functions # @@ -94,14 +96,14 @@ def get_all_test_cases(suite): yield (suite.__module__, suite.__class__.__name__) -def list_unittests(path): +def list_unittests(): """Use the unittest module to retreive all test cases from a given directory""" loader = unittest.TestLoader() - suite = loader.discover(path) + suite = loader.discover(os.path.join(brpath, "support", "testing")) tests = {} for module, test in get_all_test_cases(suite): - module_path = os.path.join(path, *module.split('.')) + module_path = os.path.join("support", "testing", *module.split('.')) tests.setdefault(module_path, []).append('%s.%s' % (module, test)) return tests @@ -124,9 +126,7 @@ class Developer: self.defconfigs = parse_developer_defconfigs(files) def hasfile(self, f): - f = os.path.abspath(f) for fs in self.files: - fs = os.path.abspath(fs) if f.startswith(fs): return True return False @@ -158,7 +158,7 @@ def parse_developer_packages(fnames): patterns, and return a list of those packages.""" packages = set() for fname in fnames: - for root, dirs, files in os.walk(fname): + for root, dirs, files in os.walk(os.path.join(brpath, fname)): for f in files: path = os.path.join(root, f) if fname_get_package_infra(path): @@ -223,7 +223,7 @@ def parse_developer_runtime_tests(fnames): # List all files recursively for fname in fnames: if os.path.isdir(fname): - for root, _dirs, files in os.walk(fname): + for root, _dirs, files in os.walk(os.path.join(brpath, fname)): all_files += [os.path.join(root, f) for f in files] else: all_files.append(fname) @@ -237,15 +237,13 @@ def parse_developer_runtime_tests(fnames): return runtimes -def parse_developers(basepath=None): +def parse_developers(): """Parse the DEVELOPERS file and return a list of Developer objects.""" developers = [] linen = 0 - if basepath is None: - basepath = os.getcwd() global unittests - unittests = list_unittests(os.path.join(basepath, 'support/testing')) - with open(os.path.join(basepath, "DEVELOPERS"), "r") as f: + unittests = list_unittests() + with open(os.path.join(brpath, "DEVELOPERS"), "r") as f: files = [] name = None for line in f: @@ -259,11 +257,11 @@ def parse_developers(basepath=None): name = line[2:].strip() elif line.startswith("F:"): fname = line[2:].strip() - dev_files = glob.glob(os.path.join(basepath, fname)) + dev_files = glob.glob(os.path.join(brpath, fname)) if len(dev_files) == 0: print("WARNING: '%s' doesn't match any file" % fname, file=sys.stderr) - files += dev_files + files += [os.path.relpath(f, brpath) for f in dev_files] elif line == "": if not name: continue