From patchwork Sun Jul 2 14:25:26 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1802323 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=85.214.62.61; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.a=rsa-sha256 header.s=google header.b=UunnSINe; dkim-atps=neutral Received: from phobos.denx.de (phobos.denx.de [85.214.62.61]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QvBGH1P68z20Pj for ; Mon, 3 Jul 2023 00:27:19 +1000 (AEST) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id 29E69863A4; Sun, 2 Jul 2023 16:27:11 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="UunnSINe"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id 91B4786330; Sun, 2 Jul 2023 16:27:09 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.2 Received: from mail-il1-x136.google.com (mail-il1-x136.google.com [IPv6:2607:f8b0:4864:20::136]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id 0683B86202 for ; Sun, 2 Jul 2023 16:27:07 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=sjg@chromium.org Received: by mail-il1-x136.google.com with SMTP id e9e14a558f8ab-345c1ed760aso20192775ab.1 for ; Sun, 02 Jul 2023 07:27:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1688308025; x=1690900025; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=atj1cveUekwv39zqDQjL/F84U2n1/Zqx+DWpewvfc08=; b=UunnSINeDSp0UIqlU7MCMyqtFcloyKxYmvn4eH2hl6YQsOhbB/HY61xBpRyI1fYUW1 KuiXABQvSc1utAlgRr1W29ZONrtGaul623hEASjCOGoZ3eP9QOuJ1qRO7p4Qy5XqHYxt +obcEBjxZEO6ZtnlS2l4lUws6vwxu87/kqTqo= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1688308025; x=1690900025; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=atj1cveUekwv39zqDQjL/F84U2n1/Zqx+DWpewvfc08=; b=lZPdOozlaBNoMKs8Q87SxXTQ8cQn0g6yIAdtzjPtEM7OHBm0nZGArcsLFvOXF6bz9s 7g4JpQcOeXHOeKbIpCvOUJF1yh024/ZsoBmmkF7sU+FveLGj/s64QOfNoi/csbIuD87a rmZw3TUnJFi9zbZiGmgFFJovtLT8dstyuKvWq+sfheZGkLjxBynz5rDdV8rUgwMqbs67 4miUKcIOFuT+ZWU5pGB4wlu3u4XhIym8k4fmP/xayg/DfAInlMTqsYakJDsGP5frxOSa YOkIHyD74KPJnzR8d2L01GusQEBRwee4V7LtVXfiYQ/rsmXZ/Ufdjpi59ju+9kBp/PwF 52Dw== X-Gm-Message-State: ABy/qLZhk/iWRd0VsanYn/jW2mn09IqScKvTxIiwf2P2MhduBqkj9gUl eGAjhZrOgq0DvJmvASka9a3aUKXxYPdODO3sXOw= X-Google-Smtp-Source: APBJJlGCNjpFd5emkfnHT7rW7kA2uJxvVvLV91b4JABCWsZ1J9G15U6W7ZLB79tRMhgzy8+K5SkCeQ== X-Received: by 2002:a92:d48e:0:b0:345:aba5:3777 with SMTP id p14-20020a92d48e000000b00345aba53777mr7909371ilg.25.1688308025594; Sun, 02 Jul 2023 07:27:05 -0700 (PDT) Received: from sjg1.lan (c-73-14-173-85.hsd1.co.comcast.net. [73.14.173.85]) by smtp.gmail.com with ESMTPSA id c13-20020a02c9cd000000b0042af5268e4asm2934713jap.58.2023.07.02.07.27.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 02 Jul 2023 07:27:05 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Cc: Simon Glass Subject: [PATCH 01/58] buildman: Tidy up pylint warnings in main Date: Sun, 2 Jul 2023 15:25:26 +0100 Message-ID: <20230702142639.1249681-2-sjg@chromium.org> X-Mailer: git-send-email 2.41.0.255.g8b1d071c50-goog In-Reply-To: <20230702142639.1249681-1-sjg@chromium.org> References: <20230702142639.1249681-1-sjg@chromium.org> MIME-Version: 1.0 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.8 at phobos.denx.de X-Virus-Status: Clean Tidy up the various pylint warnings in module 'main'. Signed-off-by: Simon Glass --- tools/buildman/main.py | 23 +++++++++++------------ 1 file changed, 11 insertions(+), 12 deletions(-) diff --git a/tools/buildman/main.py b/tools/buildman/main.py index a2ffbc9073e7..ca1beb098292 100755 --- a/tools/buildman/main.py +++ b/tools/buildman/main.py @@ -6,29 +6,21 @@ """See README for more information""" -import doctest -import multiprocessing import os -import re import sys # Bring in the patman libraries +# pylint: disable=C0413 our_path = os.path.dirname(os.path.realpath(__file__)) sys.path.insert(1, os.path.join(our_path, '..')) # Our modules -from buildman import board from buildman import bsettings -from buildman import builder from buildman import cmdline from buildman import control -from buildman import toolchain -from patman import patchstream -from patman import gitutil -from u_boot_pylib import terminal from u_boot_pylib import test_util -def RunTests(skip_net_tests, verbose, args): +def run_tests(skip_net_tests, verbose, args): """Run the buildman tests Args: @@ -36,9 +28,11 @@ def RunTests(skip_net_tests, verbose, args): verbosity (int): Verbosity level to use (0-4) args (list of str): List of tests to run, empty to run all """ + # These imports are here since tests are not available when buildman is + # installed as a Python module + # pylint: disable=C0415 from buildman import func_test from buildman import test - import doctest test_name = args and args[0] or None if skip_net_tests: @@ -54,6 +48,11 @@ def RunTests(skip_net_tests, verbose, args): return (0 if result.wasSuccessful() else 1) def run_buildman(): + """Run bulidman + + This is the main program. It collects arguments and runs either the tests or + the control module. + """ options, args = cmdline.ParseArgs() if not options.debug: @@ -61,7 +60,7 @@ def run_buildman(): # Run our meagre tests if cmdline.HAS_TESTS and options.test: - RunTests(options.skip_net_tests, options.verbose, args) + run_tests(options.skip_net_tests, options.verbose, args) # Build selected commits for selected boards else: From patchwork Sun Jul 2 14:25:27 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1802325 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=85.214.62.61; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.a=rsa-sha256 header.s=google header.b=LH6dYTNe; dkim-atps=neutral Received: from phobos.denx.de (phobos.denx.de [85.214.62.61]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QvBGf2dGbz20Pj for ; Mon, 3 Jul 2023 00:27:38 +1000 (AEST) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id 179DC86335; Sun, 2 Jul 2023 16:27:16 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="LH6dYTNe"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id 9CBEF86330; Sun, 2 Jul 2023 16:27:13 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.2 Received: from mail-io1-xd36.google.com (mail-io1-xd36.google.com [IPv6:2607:f8b0:4864:20::d36]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id 85CE68637D for ; Sun, 2 Jul 2023 16:27:08 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=sjg@chromium.org Received: by mail-io1-xd36.google.com with SMTP id ca18e2360f4ac-777a6ebb542so147262139f.0 for ; Sun, 02 Jul 2023 07:27:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1688308027; x=1690900027; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ZHHF0pfdGV4dFqCK0dN4+TDtZWPKdsaKJZEYNr6Iy08=; b=LH6dYTNesCgyh6mS+u1YJZwWWZPrNm45kIzjiYulOfiYNTObJrzeXpcDws4aYK++rZ +fqpZC0dq9ucEVmB7y8WEZhPU1Kstyf1HHcQ+QmWqrGyNFVbf9GAbIOTi4u2weN9ffcM m8on1Wx+KtC9v+yZIDQBwpZK6BtdK/mALq0Kw= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1688308027; x=1690900027; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ZHHF0pfdGV4dFqCK0dN4+TDtZWPKdsaKJZEYNr6Iy08=; b=JWHX5BS08AxPRHPDzdACIgWep4ZfbEwDgqHTSaOiAAL7a8xp7uRr2Z6Zp1+ZRvD59X fc99x9DK4xAQrXFlELqiEmrH+fFppn+HvlGh98hD7cA/NMqu3uUTDDP9m0S9gZuQlG69 BeE5kXFnCyzmI592C2uLPpcvw5B+dpZRr3uwwVL2xh6bXPZG++JqWkp6e5hhRG04Cej5 04mQ7KTBUca+UqpnNvoiHeQctu4afCqZu7Cnbdc8H0mfqplzu0ETKmACHwyWGX26uLdf GtgNwWoTKvrrSyDp4Tz2y3ZalxmSIfip3WmUj2uFe7YpSATbRUfu28u9Arcgyi8DJczA hThQ== X-Gm-Message-State: AC+VfDxaud3/gl2Gels0b+j/oPsi8pI41Byz0Uh5dOoDp1JJZKQBxAb9 1jyPNIx4/oYSwgKeiQfZ526c57zWI34HN/QtIVI= X-Google-Smtp-Source: ACHHUZ54AZ4NpJvJrcTxKtSzfc7h21NGzyNVl4QdEFBqzuPMzhurGbjixvuAaw5L6T8wv8UpgyxOtw== X-Received: by 2002:a6b:f90d:0:b0:780:d338:5373 with SMTP id j13-20020a6bf90d000000b00780d3385373mr7726117iog.15.1688308026931; Sun, 02 Jul 2023 07:27:06 -0700 (PDT) Received: from sjg1.lan (c-73-14-173-85.hsd1.co.comcast.net. [73.14.173.85]) by smtp.gmail.com with ESMTPSA id c13-20020a02c9cd000000b0042af5268e4asm2934713jap.58.2023.07.02.07.27.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 02 Jul 2023 07:27:06 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Cc: Simon Glass Subject: [PATCH 02/58] buildman: Convert camel case in control.py Date: Sun, 2 Jul 2023 15:25:27 +0100 Message-ID: <20230702142639.1249681-3-sjg@chromium.org> X-Mailer: git-send-email 2.41.0.255.g8b1d071c50-goog In-Reply-To: <20230702142639.1249681-1-sjg@chromium.org> References: <20230702142639.1249681-1-sjg@chromium.org> MIME-Version: 1.0 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.8 at phobos.denx.de X-Virus-Status: Clean Convert this file to snake case and update all files which use it. Signed-off-by: Simon Glass --- tools/buildman/control.py | 31 ++++++++++++++++++------------- tools/buildman/func_test.py | 7 ++++--- tools/buildman/main.py | 2 +- tools/buildman/test.py | 2 +- 4 files changed, 24 insertions(+), 18 deletions(-) diff --git a/tools/buildman/control.py b/tools/buildman/control.py index b8286e185411..bfb02834e8fb 100644 --- a/tools/buildman/control.py +++ b/tools/buildman/control.py @@ -2,6 +2,11 @@ # Copyright (c) 2013 The Chromium OS Authors. # +"""Control module for buildman + +This holds the main control logic for buildman, when not running tests. +""" + import multiprocessing try: import importlib.resources @@ -25,11 +30,11 @@ from u_boot_pylib import terminal from u_boot_pylib import tools from u_boot_pylib.terminal import tprint -def GetPlural(count): +def get_plural(count): """Returns a plural 's' if count is not 1""" return 's' if count != 1 else '' -def GetActionSummary(is_summary, commits, selected, options): +def get_action_summary(is_summary, commits, selected, options): """Return a string summarising the intended action. Returns: @@ -38,18 +43,18 @@ def GetActionSummary(is_summary, commits, selected, options): if commits: count = len(commits) count = (count + options.step - 1) // options.step - commit_str = '%d commit%s' % (count, GetPlural(count)) + commit_str = '%d commit%s' % (count, get_plural(count)) else: commit_str = 'current source' str = '%s %s for %d boards' % ( 'Summary of' if is_summary else 'Building', commit_str, len(selected)) str += ' (%d thread%s, %d job%s per thread)' % (options.threads, - GetPlural(options.threads), options.jobs, GetPlural(options.jobs)) + get_plural(options.threads), options.jobs, get_plural(options.jobs)) return str -def ShowActions(series, why_selected, boards_selected, builder, options, - board_warnings): +def show_actions(series, why_selected, boards_selected, builder, options, + board_warnings): """Display a list of actions that we would take, if not a dry run. Args: @@ -72,7 +77,7 @@ def ShowActions(series, why_selected, boards_selected, builder, options, commits = series.commits else: commits = None - print(GetActionSummary(False, commits, boards_selected, + print(get_action_summary(False, commits, boards_selected, options)) print('Build directory: %s' % builder.base_dir) if commits: @@ -92,7 +97,7 @@ def ShowActions(series, why_selected, boards_selected, builder, options, for warning in board_warnings: print(col.build(col.YELLOW, warning)) -def ShowToolchainPrefix(brds, toolchains): +def show_toolchain_prefix(brds, toolchains): """Show information about a the tool chain used by one or more boards The function checks that all boards use the same toolchain, then prints @@ -133,8 +138,8 @@ def get_allow_missing(opt_allow, opt_no_allow, num_selected, has_branch): allow_missing = False return allow_missing -def DoBuildman(options, args, toolchains=None, make_func=None, brds=None, - clean_dir=False, test_thread_exceptions=False): +def do_buildman(options, args, toolchains=None, make_func=None, brds=None, + clean_dir=False, test_thread_exceptions=False): """The main control code for buildman Args: @@ -237,7 +242,7 @@ def DoBuildman(options, args, toolchains=None, make_func=None, brds=None, sys.exit(col.build(col.RED, 'No matching boards found')) if options.print_prefix: - err = ShowToolchainPrefix(brds, toolchains) + err = show_toolchain_prefix(brds, toolchains) if err: sys.exit(col.build(col.RED, err)) return 0 @@ -373,7 +378,7 @@ def DoBuildman(options, args, toolchains=None, make_func=None, brds=None, # For a dry run, just show our actions as a sanity check if options.dry_run: - ShowActions(series, why_selected, selected, builder, options, + show_actions(series, why_selected, selected, builder, options, board_warnings) else: builder.force_build = options.force_build @@ -393,7 +398,7 @@ def DoBuildman(options, args, toolchains=None, make_func=None, brds=None, commits = None if not options.ide: - tprint(GetActionSummary(options.summary, commits, board_selected, + tprint(get_action_summary(options.summary, commits, board_selected, options)) # We can't show function sizes without board details at present diff --git a/tools/buildman/func_test.py b/tools/buildman/func_test.py index 08b2714b7be2..052153043b21 100644 --- a/tools/buildman/func_test.py +++ b/tools/buildman/func_test.py @@ -247,9 +247,10 @@ class TestFunctional(unittest.TestCase): options, args = cmdline.ParseArgs() if brds == False: brds = self._boards - result = control.DoBuildman(options, args, toolchains=self._toolchains, - make_func=self._HandleMake, brds=brds, clean_dir=clean_dir, - test_thread_exceptions=test_thread_exceptions) + result = control.do_buildman( + options, args, toolchains=self._toolchains, + make_func=self._HandleMake, brds=brds, clean_dir=clean_dir, + test_thread_exceptions=test_thread_exceptions) if get_builder: self._builder = control.builder return result diff --git a/tools/buildman/main.py b/tools/buildman/main.py index ca1beb098292..9c84e16e7df0 100755 --- a/tools/buildman/main.py +++ b/tools/buildman/main.py @@ -65,7 +65,7 @@ def run_buildman(): # Build selected commits for selected boards else: bsettings.Setup(options.config_file) - ret_code = control.DoBuildman(options, args) + ret_code = control.do_buildman(options, args) sys.exit(ret_code) diff --git a/tools/buildman/test.py b/tools/buildman/test.py index 9fa6445b798f..7eb25aa80eba 100644 --- a/tools/buildman/test.py +++ b/tools/buildman/test.py @@ -465,7 +465,7 @@ class TestBuild(unittest.TestCase): options.show_errors = False options.keep_outputs = False args = ['tegra20'] - control.DoBuildman(options, args) + control.do_buildman(options, args) def testBoardSingle(self): """Test single board selection""" From patchwork Sun Jul 2 14:25:28 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1802324 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=2a01:238:438b:c500:173d:9f52:ddab:ee01; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.a=rsa-sha256 header.s=google header.b=faOwcLeT; dkim-atps=neutral Received: from phobos.denx.de (phobos.denx.de [IPv6:2a01:238:438b:c500:173d:9f52:ddab:ee01]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QvBGS1WT8z20Pj for ; Mon, 3 Jul 2023 00:27:28 +1000 (AEST) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id DC26786330; Sun, 2 Jul 2023 16:27:13 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="faOwcLeT"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id 044D286372; Sun, 2 Jul 2023 16:27:13 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.2 Received: from mail-io1-xd31.google.com (mail-io1-xd31.google.com [IPv6:2607:f8b0:4864:20::d31]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id 4C9528638B for ; Sun, 2 Jul 2023 16:27:09 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=sjg@chromium.org Received: by mail-io1-xd31.google.com with SMTP id ca18e2360f4ac-78666994bc2so35066239f.0 for ; Sun, 02 Jul 2023 07:27:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1688308028; x=1690900028; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=qjQEcKOZ8qveTSEaaYduU7U/2DnBLEWr9jc6rmt2ybE=; b=faOwcLeTloaj/b09AOEPF6h+UOpfKeztJsemXAF/a3+piOEo6915nh1V7KLJ/Fyqo0 /U3yCv+fJhg/wF3FU6HyHpdyvWUv/cZTozXsOPaxSSPsirdOC/0TjyT0Sr9Rx/7C1CvG 7iVzxGFDHrV/FFcbudgRZ7R+EPuXzCrUhQp0M= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1688308028; x=1690900028; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=qjQEcKOZ8qveTSEaaYduU7U/2DnBLEWr9jc6rmt2ybE=; b=CZa4DDFHb8vH+02i1VW82T8zJ6qTnNZjb8RO9sTQGG91i4WR8b52uRsBp49tLmEBv/ dca9aQ9SXxmlPAQDBTBjc6BwbW5TjfDJ9g7BmIHHO55T4dQgypMNiy08OwEfS0zxHq/M 5LiEBmXY2TMm8XTPSIXEKfAdMVG+sDKOxWQTF16p8SSBFZCgK5mYK7CmsxHbePSEhlXn hpcJ+PQzteIylnqDgbrz7JzeTm840oftPiC+3DXz0Z6sN5w69TKA0Hg1cwzsCm/5bYoU VVcZESHG2USE1ZKYDv10ylt1T7tIibedi/MHe6JIXPw0xxtcRc1At0d30WyK8zcJKe3o 25aw== X-Gm-Message-State: AC+VfDxgllEvPUQWQ9WW+Ws5ytiDufyCNefy/SwshfxAFLAMyYDGppFk 3x2oKM4rHTQsq6tmucQ02aVo4h2EUNf5P5nbLeg= X-Google-Smtp-Source: ACHHUZ5o1HUq9aTAqdPH7+oq6elCWUAP5tp8baet+9F58Z9Nz7wLzbLRy1VTrw3IkmoY4Mihe6/1Bg== X-Received: by 2002:a5e:c910:0:b0:783:58f4:2e2e with SMTP id z16-20020a5ec910000000b0078358f42e2emr8275181iol.0.1688308027744; Sun, 02 Jul 2023 07:27:07 -0700 (PDT) Received: from sjg1.lan (c-73-14-173-85.hsd1.co.comcast.net. [73.14.173.85]) by smtp.gmail.com with ESMTPSA id c13-20020a02c9cd000000b0042af5268e4asm2934713jap.58.2023.07.02.07.27.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 02 Jul 2023 07:27:07 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Cc: Simon Glass Subject: [PATCH 03/58] buildman: Fix most pylint warnings in control Date: Sun, 2 Jul 2023 15:25:28 +0100 Message-ID: <20230702142639.1249681-4-sjg@chromium.org> X-Mailer: git-send-email 2.41.0.255.g8b1d071c50-goog In-Reply-To: <20230702142639.1249681-1-sjg@chromium.org> References: <20230702142639.1249681-1-sjg@chromium.org> MIME-Version: 1.0 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.8 at phobos.denx.de X-Virus-Status: Clean Tidy up the easier-to-fix pylint warnings in module 'control'. Signed-off-by: Simon Glass --- tools/buildman/control.py | 119 +++++++++++++++++++++--------------- tools/buildman/func_test.py | 2 +- 2 files changed, 70 insertions(+), 51 deletions(-) diff --git a/tools/buildman/control.py b/tools/buildman/control.py index bfb02834e8fb..c890f778f501 100644 --- a/tools/buildman/control.py +++ b/tools/buildman/control.py @@ -15,7 +15,6 @@ except ImportError: import importlib_resources import os import shutil -import subprocess import sys from buildman import boards @@ -30,6 +29,8 @@ from u_boot_pylib import terminal from u_boot_pylib import tools from u_boot_pylib.terminal import tprint +TEST_BUILDER = None + def get_plural(count): """Returns a plural 's' if count is not 1""" return 's' if count != 1 else '' @@ -43,17 +44,17 @@ def get_action_summary(is_summary, commits, selected, options): if commits: count = len(commits) count = (count + options.step - 1) // options.step - commit_str = '%d commit%s' % (count, get_plural(count)) + commit_str = f'{count} commit{get_plural(count)}' else: commit_str = 'current source' - str = '%s %s for %d boards' % ( - 'Summary of' if is_summary else 'Building', commit_str, - len(selected)) - str += ' (%d thread%s, %d job%s per thread)' % (options.threads, - get_plural(options.threads), options.jobs, get_plural(options.jobs)) - return str - -def show_actions(series, why_selected, boards_selected, builder, options, + msg = (f"{'Summary of' if is_summary else 'Building'} " + f'{commit_str} for {len(selected)} boards') + msg += (f' ({options.threads} thread{get_plural(options.threads)}, ' + f'{options.jobs} job{get_plural(options.jobs)} per thread)') + return msg + +# pylint: disable=R0913 +def show_actions(series, why_selected, boards_selected, bldr, options, board_warnings): """Display a list of actions that we would take, if not a dry run. @@ -66,7 +67,7 @@ def show_actions(series, why_selected, boards_selected, builder, options, the value would be a list of board names. boards_selected: Dict of selected boards, key is target name, value is Board object - builder: The builder that will be used to build the commits + bldr: The builder that will be used to build the commits options: Command line options object board_warnings: List of warnings obtained from board selected """ @@ -79,7 +80,7 @@ def show_actions(series, why_selected, boards_selected, builder, options, commits = None print(get_action_summary(False, commits, boards_selected, options)) - print('Build directory: %s' % builder.base_dir) + print(f'Build directory: {bldr.base_dir}') if commits: for upto in range(0, len(series.commits), options.step): commit = series.commits[upto] @@ -88,11 +89,11 @@ def show_actions(series, why_selected, boards_selected, builder, options, print() for arg in why_selected: if arg != 'all': - print(arg, ': %d boards' % len(why_selected[arg])) + print(arg, f': {len(why_selected[arg])} boards') if options.verbose: - print(' %s' % ' '.join(why_selected[arg])) - print(('Total boards to build for each commit: %d\n' % - len(why_selected['all']))) + print(f" {' '.join(why_selected[arg])}") + print('Total boards to build for each ' + f"commit: {len(why_selected['all'])}\n") if board_warnings: for warning in board_warnings: print(col.build(col.YELLOW, warning)) @@ -116,12 +117,26 @@ def show_toolchain_prefix(brds, toolchains): tc_set.add(toolchains.Select(brd.arch)) if len(tc_set) != 1: return 'Supplied boards must share one toolchain' - return False - tc = tc_set.pop() - print(tc.GetEnvArgs(toolchain.VAR_CROSS_COMPILE)) + tchain = tc_set.pop() + print(tchain.GetEnvArgs(toolchain.VAR_CROSS_COMPILE)) return None def get_allow_missing(opt_allow, opt_no_allow, num_selected, has_branch): + """Figure out whether to allow external blobs + + Uses the allow-missing setting and the provided arguments to decide whether + missing external blobs should be allowed + + Args: + opt_allow (bool): True if --allow-missing flag is set + opt_no_allow (bool): True if --no-allow-missing flag is set + num_selected (int): Number of selected board + has_branch (bool): True if a git branch (to build) has been provided + + Returns: + bool: True to allow missing external blobs, False to produce an error if + external blobs are used + """ allow_missing = False am_setting = bsettings.GetGlobalItemValue('allow-missing') if am_setting: @@ -159,7 +174,8 @@ def do_buildman(options, args, toolchains=None, make_func=None, brds=None, raise an exception instead of reporting their result. This simulates a failure in the code somewhere """ - global builder + # Used so testing can obtain the builder: pylint: disable=W0603 + global TEST_BUILDER if options.full_help: with importlib.resources.path('buildman', 'README.rst') as readme: @@ -178,22 +194,23 @@ def do_buildman(options, args, toolchains=None, make_func=None, brds=None, if options.fetch_arch: if options.fetch_arch == 'list': sorted_list = toolchains.ListArchs() - print(col.build(col.BLUE, 'Available architectures: %s\n' % - ' '.join(sorted_list))) - return 0 - else: - fetch_arch = options.fetch_arch - if fetch_arch == 'all': - fetch_arch = ','.join(toolchains.ListArchs()) - print(col.build(col.CYAN, '\nDownloading toolchains: %s' % - fetch_arch)) - for arch in fetch_arch.split(','): - print() - ret = toolchains.FetchAndInstall(arch) - if ret: - return ret + print(col.build( + col.BLUE, + f"Available architectures: {' '.join(sorted_list)}\n")) return 0 + fetch_arch = options.fetch_arch + if fetch_arch == 'all': + fetch_arch = ','.join(toolchains.ListArchs()) + print(col.build(col.CYAN, + f'\nDownloading toolchains: {fetch_arch}')) + for arch in fetch_arch.split(','): + print() + ret = toolchains.FetchAndInstall(arch) + if ret: + return ret + return 0 + if no_toolchains: toolchains.GetSettings() toolchains.Scan(options.list_tool_chains and options.verbose) @@ -216,12 +233,13 @@ def do_buildman(options, args, toolchains=None, make_func=None, brds=None, board_file = options.regen_board_list brds = boards.Boards() - ok = brds.ensure_board_list(board_file, - options.threads or multiprocessing.cpu_count(), - force=options.regen_board_list, - quiet=not options.verbose) + okay = brds.ensure_board_list( + board_file, + options.threads or multiprocessing.cpu_count(), + force=options.regen_board_list, + quiet=not options.verbose) if options.regen_board_list: - return 0 if ok else 2 + return 0 if okay else 2 brds.read_boards(board_file) exclude = [] @@ -231,14 +249,14 @@ def do_buildman(options, args, toolchains=None, make_func=None, brds=None, if options.boards: requested_boards = [] - for b in options.boards: - requested_boards += b.split(',') + for brd in options.boards: + requested_boards += brd.split(',') else: requested_boards = None why_selected, board_warnings = brds.select_boards(args, exclude, requested_boards) selected = brds.get_selected() - if not len(selected): + if not selected: sys.exit(col.build(col.RED, 'No matching boards found')) if options.print_prefix: @@ -265,15 +283,15 @@ def do_buildman(options, args, toolchains=None, make_func=None, brds=None, if count is None: sys.exit(col.build(col.RED, msg)) elif count == 0: - sys.exit(col.build(col.RED, "Range '%s' has no commits" % - options.branch)) + sys.exit(col.build(col.RED, + f"Range '{options.branch}' has no commits")) if msg: print(col.build(col.YELLOW, msg)) count += 1 # Build upstream commit also if not count: - msg = ("No commits found to process in branch '%s': " - "set branch's upstream or use -c flag" % options.branch) + msg = (f"No commits found to process in branch '{options.branch}': " + "set branch's upstream or use -c flag") sys.exit(col.build(col.RED, msg)) if options.work_in_output: if len(selected) != 1: @@ -372,6 +390,7 @@ def do_buildman(options, args, toolchains=None, make_func=None, brds=None, adjust_cfg=adjust_cfg, allow_missing=allow_missing, no_lto=options.no_lto, reproducible_builds=options.reproducible_builds) + TEST_BUILDER = builder builder.force_config_on_failure = not options.quick if make_func: builder.do_make = make_func @@ -392,8 +411,8 @@ def do_buildman(options, args, toolchains=None, make_func=None, brds=None, if series: commits = series.commits # Number the commits for test purposes - for commit in range(len(commits)): - commits[commit].sequence = commit + for i, commit in enumerate(commits): + commit.sequence = i else: commits = None @@ -416,8 +435,8 @@ def do_buildman(options, args, toolchains=None, make_func=None, brds=None, commits, board_selected, options.keep_outputs, options.verbose) if excs: return 102 - elif fail: + if fail: return 100 - elif warned and not options.ignore_warnings: + if warned and not options.ignore_warnings: return 101 return 0 diff --git a/tools/buildman/func_test.py b/tools/buildman/func_test.py index 052153043b21..d9b4c41ec68c 100644 --- a/tools/buildman/func_test.py +++ b/tools/buildman/func_test.py @@ -252,7 +252,7 @@ class TestFunctional(unittest.TestCase): make_func=self._HandleMake, brds=brds, clean_dir=clean_dir, test_thread_exceptions=test_thread_exceptions) if get_builder: - self._builder = control.builder + self._builder = control.TEST_BUILDER return result def testFullHelp(self): From patchwork Sun Jul 2 14:25:29 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1802326 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=2a01:238:438b:c500:173d:9f52:ddab:ee01; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.a=rsa-sha256 header.s=google header.b=Y66vGzxe; dkim-atps=neutral Received: from phobos.denx.de (phobos.denx.de [IPv6:2a01:238:438b:c500:173d:9f52:ddab:ee01]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QvBGr4Xlkz20Pj for ; Mon, 3 Jul 2023 00:27:48 +1000 (AEST) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id C13F28638D; Sun, 2 Jul 2023 16:27:17 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="Y66vGzxe"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id B080F86342; Sun, 2 Jul 2023 16:27:13 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.2 Received: from mail-io1-xd2d.google.com (mail-io1-xd2d.google.com [IPv6:2607:f8b0:4864:20::d2d]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id D9DA88633D for ; Sun, 2 Jul 2023 16:27:09 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=sjg@chromium.org Received: by mail-io1-xd2d.google.com with SMTP id ca18e2360f4ac-78363cc070aso186523139f.1 for ; Sun, 02 Jul 2023 07:27:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1688308028; x=1690900028; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=yUB+dy7kwTZKpWmp8662hWKVCmMrg7MOiKhwg+uRtBU=; b=Y66vGzxeB/w2qGReouJrWSB3UijgHUvrOgRHpABuzaPQYHweDmRcWKVaMBgsnjeabM Ch5PXicbWVQJAm0ScbEpFWBfH9a9Gw9avg9TRsHBZVczBddZ4Sk9HRn0oeg9mNXU2PHN BBKKjsQpXgb7aUAq+zam75kpJ1/0Bx3KC5nek= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1688308028; x=1690900028; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=yUB+dy7kwTZKpWmp8662hWKVCmMrg7MOiKhwg+uRtBU=; b=Z4Zrspgt/Q2ih5uHTOEj7ooQYyAU9iSUzQ9d14m6/KUskqtsZdRYyfMBhhyn9TDOoV s66OoJLVW300J1uUqInOSOz+LpCci6y5nSVGj/56ri+RYNsCU/xKuRnKywyT/tQA3k/I jw92kFXjB0Cg/9E6QB4YYoiSFU/92PJQsBbUqdogdC8AHheKYTzJXu+YNuvdX9+9Qvt/ 2bXJpf/0PWYo+k7C9O2KGsIOSd/uWvnmGKwMTWV1YYByGMzNGXoaky2FUXyuzXHFtGZl blWqrWnwxDtkGbbEsDijScd/m5DFFfxT6g0OLcV8eA1LGsWxVRr58PsY3Y1xy/THVjf4 OW0A== X-Gm-Message-State: AC+VfDwOr4iVKK3wm0vxd8NkOYSq5XbyY0yQLWjDS/Cfquw1G/TkMSEY SBqqOOv3rlIqb9XYtzsox2lcTGIH/B0Fk85NQ7o= X-Google-Smtp-Source: ACHHUZ43zlMiOHr7Aby+AgY0EV8xkD3Rdq7HMY7dW2saRNGeM+pAsk7SNPxbD+3nmsxO0m7LlTKrSQ== X-Received: by 2002:a6b:e308:0:b0:786:2878:9593 with SMTP id u8-20020a6be308000000b0078628789593mr9204719ioc.0.1688308028435; Sun, 02 Jul 2023 07:27:08 -0700 (PDT) Received: from sjg1.lan (c-73-14-173-85.hsd1.co.comcast.net. [73.14.173.85]) by smtp.gmail.com with ESMTPSA id c13-20020a02c9cd000000b0042af5268e4asm2934713jap.58.2023.07.02.07.27.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 02 Jul 2023 07:27:08 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Cc: Simon Glass Subject: [PATCH 04/58] buildman: Move full-help processing to main Date: Sun, 2 Jul 2023 15:25:29 +0100 Message-ID: <20230702142639.1249681-5-sjg@chromium.org> X-Mailer: git-send-email 2.41.0.255.g8b1d071c50-goog In-Reply-To: <20230702142639.1249681-1-sjg@chromium.org> References: <20230702142639.1249681-1-sjg@chromium.org> MIME-Version: 1.0 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.8 at phobos.denx.de X-Virus-Status: Clean This does not need any of the control features. Move it out of main to reduce the size of the do_buildman() function. For Python 3.6 the -H feature will not work, but this does not seem to be a huge problem, as it dates from 2016. Signed-off-by: Simon Glass --- tools/buildman/control.py | 11 ----------- tools/buildman/main.py | 9 +++++++++ 2 files changed, 9 insertions(+), 11 deletions(-) diff --git a/tools/buildman/control.py b/tools/buildman/control.py index c890f778f501..b27a6b08477f 100644 --- a/tools/buildman/control.py +++ b/tools/buildman/control.py @@ -8,11 +8,6 @@ This holds the main control logic for buildman, when not running tests. """ import multiprocessing -try: - import importlib.resources -except ImportError: - # for Python 3.6 - import importlib_resources import os import shutil import sys @@ -26,7 +21,6 @@ from patman import gitutil from patman import patchstream from u_boot_pylib import command from u_boot_pylib import terminal -from u_boot_pylib import tools from u_boot_pylib.terminal import tprint TEST_BUILDER = None @@ -177,11 +171,6 @@ def do_buildman(options, args, toolchains=None, make_func=None, brds=None, # Used so testing can obtain the builder: pylint: disable=W0603 global TEST_BUILDER - if options.full_help: - with importlib.resources.path('buildman', 'README.rst') as readme: - tools.print_full_help(str(readme)) - return 0 - gitutil.setup() col = terminal.Color() diff --git a/tools/buildman/main.py b/tools/buildman/main.py index 9c84e16e7df0..2253401709a8 100755 --- a/tools/buildman/main.py +++ b/tools/buildman/main.py @@ -6,6 +6,11 @@ """See README for more information""" +try: + from importlib.resources import files +except ImportError: + # for Python 3.6 + import importlib_resources import os import sys @@ -19,6 +24,7 @@ from buildman import bsettings from buildman import cmdline from buildman import control from u_boot_pylib import test_util +from u_boot_pylib import tools def run_tests(skip_net_tests, verbose, args): """Run the buildman tests @@ -62,6 +68,9 @@ def run_buildman(): if cmdline.HAS_TESTS and options.test: run_tests(options.skip_net_tests, options.verbose, args) + elif options.full_help: + tools.print_full_help(str(files('buildman').joinpath('README.rst'))) + # Build selected commits for selected boards else: bsettings.Setup(options.config_file) From patchwork Sun Jul 2 14:25:30 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1802328 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=2a01:238:438b:c500:173d:9f52:ddab:ee01; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.a=rsa-sha256 header.s=google header.b=VRt1G7ga; dkim-atps=neutral Received: from phobos.denx.de (phobos.denx.de [IPv6:2a01:238:438b:c500:173d:9f52:ddab:ee01]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QvBHH4rg3z20Pj for ; Mon, 3 Jul 2023 00:28:11 +1000 (AEST) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id 1B864863C2; Sun, 2 Jul 2023 16:27:23 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="VRt1G7ga"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id 4599E8638A; Sun, 2 Jul 2023 16:27:15 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.2 Received: from mail-il1-x132.google.com (mail-il1-x132.google.com [IPv6:2607:f8b0:4864:20::132]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id E1B898622C for ; Sun, 2 Jul 2023 16:27:10 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=sjg@chromium.org Received: by mail-il1-x132.google.com with SMTP id e9e14a558f8ab-345a4efb66dso14964435ab.3 for ; Sun, 02 Jul 2023 07:27:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1688308029; x=1690900029; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=kwXtr/ImRHyf9Tr1iv55piEL5oh0W6RebFHQXm69gDE=; b=VRt1G7gadoUqsKyvE73iPGD7+plKQMt8jXg2vb6748FLWhbbGW9q/DGDS6KRI+KlN1 dLlojg/TvxzHO/YLYiXxzC5MIxxVo5v3UH24YYwJAMarYe+jzG3oMB8iMqCi77wLTjHP RfsDGJKuPDAig+LAnmNPBCZ5AA+Tpw7qbiT6E= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1688308029; x=1690900029; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=kwXtr/ImRHyf9Tr1iv55piEL5oh0W6RebFHQXm69gDE=; b=E2QgHkcdXjNHB2IJQLudjtw/x2BkMAILxZrs/pIwfAGb5Iqt6dDw6gMqJXDhaGNQvx MfV+o+sFX0fCR97bxHF5+yiqrUq0qcUvXx3ydL1cIbFCGjQhx3thCa7eG2WM/HBkCOTG 9GbO2G/yVAQnykM+BoyBsTMloy6OV3m9GQ2+nM97dtUM+iXZbiBTLDF0lLDIuHLegkqB U5diiw0WEgpeeHy9uKDeKy/Akx/DLF1zjuG1T3REq9EXcu5HjwyJT8+qWDLbHfcviIj9 AhpBvlHfBTszifQLrtXsj6wiGSXeVjaEmFIQ37pq6t+R2NDdoB/QjLDDYgNSsqRzd2jI uBHg== X-Gm-Message-State: AC+VfDwawgJSYd9KO0sILI4KsXl5lduGNHXRx8rno+wGQWpviuY24uLg rJeGaaj+RxWU1PPBpgfkw9RESMhZEY1zSv0YY68= X-Google-Smtp-Source: ACHHUZ51ALE+iIWRlpDuTat7CB5EJNx42rQ68Um+Q6uRT97i4C/xwVflX0Mh4+APruAuJxklKfN4+g== X-Received: by 2002:a5e:a706:0:b0:786:2b08:709d with SMTP id b6-20020a5ea706000000b007862b08709dmr9434160iod.13.1688308029460; Sun, 02 Jul 2023 07:27:09 -0700 (PDT) Received: from sjg1.lan (c-73-14-173-85.hsd1.co.comcast.net. [73.14.173.85]) by smtp.gmail.com with ESMTPSA id c13-20020a02c9cd000000b0042af5268e4asm2934713jap.58.2023.07.02.07.27.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 02 Jul 2023 07:27:09 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Cc: Simon Glass Subject: [PATCH 05/58] buildman: Move series calculations into a separate function Date: Sun, 2 Jul 2023 15:25:30 +0100 Message-ID: <20230702142639.1249681-6-sjg@chromium.org> X-Mailer: git-send-email 2.41.0.255.g8b1d071c50-goog In-Reply-To: <20230702142639.1249681-1-sjg@chromium.org> References: <20230702142639.1249681-1-sjg@chromium.org> MIME-Version: 1.0 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.8 at phobos.denx.de X-Virus-Status: Clean Reduce the size of the do_buildman() function a little by moving the code that figures out the series into a separate function. Signed-off-by: Simon Glass --- tools/buildman/control.py | 95 +++++++++++++++++++++++---------------- 1 file changed, 56 insertions(+), 39 deletions(-) diff --git a/tools/buildman/control.py b/tools/buildman/control.py index b27a6b08477f..7a81c2ced8b3 100644 --- a/tools/buildman/control.py +++ b/tools/buildman/control.py @@ -147,6 +147,51 @@ def get_allow_missing(opt_allow, opt_no_allow, num_selected, has_branch): allow_missing = False return allow_missing + +def determine_series(count, has_range, branch, git_dir): + """Determine the series which is to be built, if any + + Args: + count (int): Number of commits in branch + has_range (bool): True if a range of commits ('xx..yy') is being built + branch (str): Name of branch to build, or None if none + git_dir (str): Git directory to use, e.g. './.git' + + Returns: + Series: Series to build, or None for none + + Read the metadata from the commits. First look at the upstream commit, + then the ones in the branch. We would like to do something like + upstream/master~..branch but that isn't possible if upstream/master is + a merge commit (it will list all the commits that form part of the + merge) + + Conflicting tags are not a problem for buildman, since it does not use + them. For example, Series-version is not useful for buildman. On the + other hand conflicting tags will cause an error. So allow later tags + to overwrite earlier ones by setting allow_overwrite=True + """ + if branch: + if count == -1: + if has_range: + range_expr = branch + else: + range_expr = gitutil.get_range_in_branch(git_dir, branch) + upstream_commit = gitutil.get_upstream(git_dir, branch) + series = patchstream.get_metadata_for_list(upstream_commit, + git_dir, 1, series=None, allow_overwrite=True) + + series = patchstream.get_metadata_for_list(range_expr, + git_dir, None, series, allow_overwrite=True) + else: + # Honour the count + series = patchstream.get_metadata_for_list(branch, + git_dir, count, series=None, allow_overwrite=True) + else: + series = None + return series + + def do_buildman(options, args, toolchains=None, make_func=None, brds=None, clean_dir=False, test_thread_exceptions=False): """The main control code for buildman @@ -174,7 +219,7 @@ def do_buildman(options, args, toolchains=None, make_func=None, brds=None, gitutil.setup() col = terminal.Color() - options.git_dir = os.path.join(options.git, '.git') + git_dir = os.path.join(options.git, '.git') no_toolchains = toolchains is None if no_toolchains: @@ -264,11 +309,11 @@ def do_buildman(options, args, toolchains=None, make_func=None, brds=None, count = 1 else: if has_range: - count, msg = gitutil.count_commits_in_range(options.git_dir, - options.branch) + count, msg = gitutil.count_commits_in_range(git_dir, + options.branch) else: - count, msg = gitutil.count_commits_in_branch(options.git_dir, - options.branch) + count, msg = gitutil.count_commits_in_branch(git_dir, + options.branch) if count is None: sys.exit(col.build(col.RED, msg)) elif count == 0: @@ -290,39 +335,11 @@ def do_buildman(options, args, toolchains=None, make_func=None, brds=None, sys.exit(col.build(col.RED, '-w can only be used with a single commit')) - # Read the metadata from the commits. First look at the upstream commit, - # then the ones in the branch. We would like to do something like - # upstream/master~..branch but that isn't possible if upstream/master is - # a merge commit (it will list all the commits that form part of the - # merge) - # Conflicting tags are not a problem for buildman, since it does not use - # them. For example, Series-version is not useful for buildman. On the - # other hand conflicting tags will cause an error. So allow later tags - # to overwrite earlier ones by setting allow_overwrite=True - if options.branch: - if count == -1: - if has_range: - range_expr = options.branch - else: - range_expr = gitutil.get_range_in_branch(options.git_dir, - options.branch) - upstream_commit = gitutil.get_upstream(options.git_dir, - options.branch) - series = patchstream.get_metadata_for_list(upstream_commit, - options.git_dir, 1, series=None, allow_overwrite=True) - - series = patchstream.get_metadata_for_list(range_expr, - options.git_dir, None, series, allow_overwrite=True) - else: - # Honour the count - series = patchstream.get_metadata_for_list(options.branch, - options.git_dir, count, series=None, allow_overwrite=True) - else: - series = None - if not options.dry_run: - options.verbose = True - if not options.summary: - options.show_errors = True + series = determine_series(count, has_range, options.branch, git_dir) + if not series and not options.dry_run: + options.verbose = True + if not options.summary: + options.show_errors = True # By default we have one thread per CPU. But if there are not enough jobs # we can have fewer threads and use a high '-j' value for make. @@ -364,7 +381,7 @@ def do_buildman(options, args, toolchains=None, make_func=None, brds=None, else: adjust_cfg['LOCALVERSION_AUTO'] = '~' - builder = Builder(toolchains, output_dir, options.git_dir, + builder = Builder(toolchains, output_dir, git_dir, options.threads, options.jobs, gnu_make=gnu_make, checkout=True, show_unknown=options.show_unknown, step=options.step, no_subdirs=options.no_subdirs, full_path=options.full_path, From patchwork Sun Jul 2 14:25:31 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1802327 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=2a01:238:438b:c500:173d:9f52:ddab:ee01; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.a=rsa-sha256 header.s=google header.b=X3HmP00v; dkim-atps=neutral Received: from phobos.denx.de (phobos.denx.de [IPv6:2a01:238:438b:c500:173d:9f52:ddab:ee01]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QvBH40M67z20Pj for ; Mon, 3 Jul 2023 00:28:00 +1000 (AEST) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id 47431865B0; Sun, 2 Jul 2023 16:27:20 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="X3HmP00v"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id 20E708637D; Sun, 2 Jul 2023 16:27:15 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.2 Received: from mail-io1-xd35.google.com (mail-io1-xd35.google.com [IPv6:2607:f8b0:4864:20::d35]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id 0D2C4863BA for ; Sun, 2 Jul 2023 16:27:12 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=sjg@chromium.org Received: by mail-io1-xd35.google.com with SMTP id ca18e2360f4ac-785d738d3feso87723939f.0 for ; Sun, 02 Jul 2023 07:27:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1688308030; x=1690900030; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=kM1Gv2d8tVqnzQO3ZRO1HLrnMuPIZetVRqilsx9el7Q=; b=X3HmP00vGXBFmjFTOXSs1HPnepypk9FpbuM5rkiTmXILiV9QrBR32u3MdlV98GhRGA hSd+DjCtY/QPAH64oN6M+Qm+qiZIlxkYeBS6lO7VoOSkb5LiYXyBoE+kRIe0hUPshvJn EyKxedtwNTmMKyRdFdhQROpJ+nMyC5jTmNYac= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1688308030; x=1690900030; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=kM1Gv2d8tVqnzQO3ZRO1HLrnMuPIZetVRqilsx9el7Q=; b=jGiAL8HI5n+icWWOyUMcf5vYUwrj/5RMq+6Bq3Oakz2OxhyBkPzZIp5AQL8EoIQhso 8/jnP9wj+I/+BZEpykHX7eoFnP0bg4/1cL4bKDu0+8Ex2X7lh/hoN2cH/bSHCP++HfqK v3IOznYLc0guXHfh++ie/vHoXbq43IFYXan/P0ihcwIk9NBR+ySLv3GarLc4RoSG2W0R ZRoJNWsydVn5/h4hkyP9YJue31yDGtYjw047xlRQGsDgNnOr0vB87q8l/gc2X40DWXqv qeNcl1OvU8kvPaseZbVH3krCPs7lnXt+As63u3O+19eDSzWIn+tTuDYf3VrCsYIyfrZ6 YZKg== X-Gm-Message-State: AC+VfDzpPhydV/mIURZqLYTz9c0yHUSR+bWlCE/W2knaHlCXKs+YAdBZ vnxRY9UFpFZmWXkgWfzhnDr1v/R+U78wjF89D7U= X-Google-Smtp-Source: ACHHUZ751db22xD3ry8gb9UrmWvANgt7abCOewutpRjq2YSaQTp4kSfmqK4c0R93LOWHfZNm3KUF7Q== X-Received: by 2002:a05:6602:29a2:b0:780:ba85:f475 with SMTP id u2-20020a05660229a200b00780ba85f475mr9982206ios.3.1688308030616; Sun, 02 Jul 2023 07:27:10 -0700 (PDT) Received: from sjg1.lan (c-73-14-173-85.hsd1.co.comcast.net. [73.14.173.85]) by smtp.gmail.com with ESMTPSA id c13-20020a02c9cd000000b0042af5268e4asm2934713jap.58.2023.07.02.07.27.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 02 Jul 2023 07:27:10 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Cc: Simon Glass Subject: [PATCH 06/58] buildman: Move fetch-arch code into a separate function Date: Sun, 2 Jul 2023 15:25:31 +0100 Message-ID: <20230702142639.1249681-7-sjg@chromium.org> X-Mailer: git-send-email 2.41.0.255.g8b1d071c50-goog In-Reply-To: <20230702142639.1249681-1-sjg@chromium.org> References: <20230702142639.1249681-1-sjg@chromium.org> MIME-Version: 1.0 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.8 at phobos.denx.de X-Virus-Status: Clean Reduce the size of the do_buildman() function a little by moving the code that handles --fetch-arch into a separate function. Signed-off-by: Simon Glass --- tools/buildman/control.py | 49 +++++++++++++++++++++++++-------------- 1 file changed, 31 insertions(+), 18 deletions(-) diff --git a/tools/buildman/control.py b/tools/buildman/control.py index 7a81c2ced8b3..fd136e54baf9 100644 --- a/tools/buildman/control.py +++ b/tools/buildman/control.py @@ -192,6 +192,36 @@ def determine_series(count, has_range, branch, git_dir): return series +def do_fetch_arch(toolchains, col, fetch_arch): + """Handle the --fetch-arch option + + Args: + toolchains (Toolchains): Tool chains to use + col (terminal.Color): Color object to build + fetch_arch (str): Argument passed to the --fetch-arch option + + Returns: + int: Return code for buildman + """ + if fetch_arch == 'list': + sorted_list = toolchains.ListArchs() + print(col.build( + col.BLUE, + f"Available architectures: {' '.join(sorted_list)}\n")) + return 0 + + if fetch_arch == 'all': + fetch_arch = ','.join(toolchains.ListArchs()) + print(col.build(col.CYAN, + f'\nDownloading toolchains: {fetch_arch}')) + for arch in fetch_arch.split(','): + print() + ret = toolchains.FetchAndInstall(arch) + if ret: + return ret + return 0 + + def do_buildman(options, args, toolchains=None, make_func=None, brds=None, clean_dir=False, test_thread_exceptions=False): """The main control code for buildman @@ -226,24 +256,7 @@ def do_buildman(options, args, toolchains=None, make_func=None, brds=None, toolchains = toolchain.Toolchains(options.override_toolchain) if options.fetch_arch: - if options.fetch_arch == 'list': - sorted_list = toolchains.ListArchs() - print(col.build( - col.BLUE, - f"Available architectures: {' '.join(sorted_list)}\n")) - return 0 - - fetch_arch = options.fetch_arch - if fetch_arch == 'all': - fetch_arch = ','.join(toolchains.ListArchs()) - print(col.build(col.CYAN, - f'\nDownloading toolchains: {fetch_arch}')) - for arch in fetch_arch.split(','): - print() - ret = toolchains.FetchAndInstall(arch) - if ret: - return ret - return 0 + return do_fetch_arch(toolchains, col, options.fetch_arch) if no_toolchains: toolchains.GetSettings() From patchwork Sun Jul 2 14:25:32 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1802329 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=85.214.62.61; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.a=rsa-sha256 header.s=google header.b=MhS8Hu/M; dkim-atps=neutral Received: from phobos.denx.de (phobos.denx.de [85.214.62.61]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QvBHT6W4Bz20Pj for ; Mon, 3 Jul 2023 00:28:21 +1000 (AEST) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id 275C5865B8; Sun, 2 Jul 2023 16:27:25 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="MhS8Hu/M"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id 6AAA68622C; Sun, 2 Jul 2023 16:27:15 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.2 Received: from mail-io1-xd30.google.com (mail-io1-xd30.google.com [IPv6:2607:f8b0:4864:20::d30]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id F407E86335 for ; Sun, 2 Jul 2023 16:27:12 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=sjg@chromium.org Received: by mail-io1-xd30.google.com with SMTP id ca18e2360f4ac-783698a37beso192564439f.0 for ; Sun, 02 Jul 2023 07:27:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1688308031; x=1690900031; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=eZs6aIJNuJg/M7RhNK1XBKlxXR7xBWWZwCRKl1uMMsA=; b=MhS8Hu/McqSczDQOaJvvET0i0o+5bQOUXWe6KxiOZGVd5d0MRYufMsBhMaFR+PG+tx b+vFl3HVgsvQbL6TyO4OZiUYS0x6AEKNvI7tXX8NzriPjmHPnnCUIY+h1KAaTdTTcnLd MC/01sCD0sJ//MSOorvTmalAiqNkNm2pXIOsc= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1688308031; x=1690900031; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=eZs6aIJNuJg/M7RhNK1XBKlxXR7xBWWZwCRKl1uMMsA=; b=DmbwNTatv3Rlcq/9oh9W2WFE8yAO3b7I3U958cEtxRepbxY22/Er0BOqB1X5m6IAO3 jsjlattWQbJ/MxlR8wFw8a9xXU0bpEp4xOn1y0s8EmmfGeCK6RtLiMM2QRcCmZQSc0QF bqIvIlYRa9vfHzgIR9hb4aQ8fmMfJfA56NPWN4/BU4rlGs4Hya05VEjhCv4KvC8HaeTp EzBtvkhIr2l+hyWYr22V3n+/ad9bwHJ8bZe8IdsvP+jszAlJLMRg9WQfjX8cGQEyS5l8 xQkcPtX+HIaHfamslVi+fpAvuZiitHKj22beSAp2pmW/E7dv8Juc6nD7GjaO3TZMG+uI atIg== X-Gm-Message-State: AC+VfDwOXAR1X4qo2XBAyfkFqEcreE/EqEsmj2PbIvFVqCFmndHaugc0 IPs9qV1x5qivmeYP0tZqCXFRA5Zx6goqPvd8x2M= X-Google-Smtp-Source: ACHHUZ76HYRSXB0TM0gbpvxg0Gtel5ENYOf2HlxuUgOquMb4xFkXo53tt2X9E8XkLCP6020VKKnxPA== X-Received: by 2002:a05:6602:2146:b0:780:bf50:32c9 with SMTP id y6-20020a056602214600b00780bf5032c9mr9146770ioy.15.1688308031664; Sun, 02 Jul 2023 07:27:11 -0700 (PDT) Received: from sjg1.lan (c-73-14-173-85.hsd1.co.comcast.net. [73.14.173.85]) by smtp.gmail.com with ESMTPSA id c13-20020a02c9cd000000b0042af5268e4asm2934713jap.58.2023.07.02.07.27.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 02 Jul 2023 07:27:11 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Cc: Simon Glass Subject: [PATCH 07/58] buildman: Move logic for -A up a little Date: Sun, 2 Jul 2023 15:25:32 +0100 Message-ID: <20230702142639.1249681-8-sjg@chromium.org> X-Mailer: git-send-email 2.41.0.255.g8b1d071c50-goog In-Reply-To: <20230702142639.1249681-1-sjg@chromium.org> References: <20230702142639.1249681-1-sjg@chromium.org> MIME-Version: 1.0 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.8 at phobos.denx.de X-Virus-Status: Clean Move this up to just after the boards are calculated, so we can refactor the code now below this function. Signed-off-by: Simon Glass --- tools/buildman/control.py | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/tools/buildman/control.py b/tools/buildman/control.py index fd136e54baf9..cde705c1c956 100644 --- a/tools/buildman/control.py +++ b/tools/buildman/control.py @@ -289,6 +289,12 @@ def do_buildman(options, args, toolchains=None, make_func=None, brds=None, return 0 if okay else 2 brds.read_boards(board_file) + if options.print_prefix: + err = show_toolchain_prefix(brds, toolchains) + if err: + sys.exit(col.build(col.RED, err)) + return 0 + exclude = [] if options.exclude: for arg in options.exclude: @@ -306,12 +312,6 @@ def do_buildman(options, args, toolchains=None, make_func=None, brds=None, if not selected: sys.exit(col.build(col.RED, 'No matching boards found')) - if options.print_prefix: - err = show_toolchain_prefix(brds, toolchains) - if err: - sys.exit(col.build(col.RED, err)) - return 0 - # Work out how many commits to build. We want to build everything on the # branch. We also build the upstream commit as a control so we can see # problems introduced by the first commit on the branch. From patchwork Sun Jul 2 14:25:33 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1802330 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=2a01:238:438b:c500:173d:9f52:ddab:ee01; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.a=rsa-sha256 header.s=google header.b=Rs1Rgwa8; dkim-atps=neutral Received: from phobos.denx.de (phobos.denx.de [IPv6:2a01:238:438b:c500:173d:9f52:ddab:ee01]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QvBHg1Gd3z20Pj for ; Mon, 3 Jul 2023 00:28:31 +1000 (AEST) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id 55951865C2; Sun, 2 Jul 2023 16:27:27 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="Rs1Rgwa8"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id 451AB8633D; Sun, 2 Jul 2023 16:27:16 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.2 Received: from mail-io1-xd36.google.com (mail-io1-xd36.google.com [IPv6:2607:f8b0:4864:20::d36]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id AF57386339 for ; Sun, 2 Jul 2023 16:27:13 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=sjg@chromium.org Received: by mail-io1-xd36.google.com with SMTP id ca18e2360f4ac-785d738d3feso87724139f.0 for ; Sun, 02 Jul 2023 07:27:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1688308032; x=1690900032; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Z0biPI9jKyt3o5AlE4R2wAHGl86paiq0usw0QDAOuHU=; b=Rs1Rgwa8zKBC4+6uvAWz1acVd7KCxoDhZEInnP1jSdtPCw7KpeAQ+NMh9aXufqsR8r 0l+lzv2kPSWPp2WJOrxu1t3tujZzo1ZSe8rQOBBaAmJQpsMybwWmEsK7nFQkK1/cpuEh gDfWODYfwv6xNNpniSgYZ2SZxFjDlYFPmAfQk= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1688308032; x=1690900032; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Z0biPI9jKyt3o5AlE4R2wAHGl86paiq0usw0QDAOuHU=; b=MUiMpEGVWufLCI1UM6rt5WtHLnt/Fyu3cCoXLm8T2RZc880SC67ObkOD9OR7Vx6+eS WfExXwBE4BkqWT1fddhiP7DlZ1+Ztmn4Lei5ShiKX6Rn0AYvMlVOu/59evitor/tco5b ZrV4iesyGo/l6SwMv7YpM7yH6v1vhMFgrBD17abPlUBX+7sVKqQ5m6RKXEE8Xnivf5+0 O6BELURhslYYum9rAc4uEOKVT6oVZx76nwEjBrL1M+mdl45zW0TfmSZkcPCLJLuxgRD0 5fowkgusPdvmaB081b3Nc0ItVNeWDiQQlDzE5wnmqenMKmZcDkWcHN48aJsT+PX/tUA1 ViBA== X-Gm-Message-State: AC+VfDyOoFi2UYQtJ7uKf6OY/Z3HpSUTFZPgeKOqg5eH+LX/UPCukjBI KjnZDBB7BOziLnFy2mHnQIPjiJ8S9l9v52kkhF8= X-Google-Smtp-Source: ACHHUZ46CpJ8awAoHEPUwbY0oTrwPKrGvmzAwE3HTtta6+ZSj5YtSA1cH+aNUVPs40LFg9zkN6bzBQ== X-Received: by 2002:a05:6602:3e90:b0:780:c92c:38ed with SMTP id el16-20020a0566023e9000b00780c92c38edmr9783244iob.0.1688308032335; Sun, 02 Jul 2023 07:27:12 -0700 (PDT) Received: from sjg1.lan (c-73-14-173-85.hsd1.co.comcast.net. [73.14.173.85]) by smtp.gmail.com with ESMTPSA id c13-20020a02c9cd000000b0042af5268e4asm2934713jap.58.2023.07.02.07.27.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 02 Jul 2023 07:27:12 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Cc: Simon Glass Subject: [PATCH 08/58] buildman: Drop use of builder in show_actions() Date: Sun, 2 Jul 2023 15:25:33 +0100 Message-ID: <20230702142639.1249681-9-sjg@chromium.org> X-Mailer: git-send-email 2.41.0.255.g8b1d071c50-goog In-Reply-To: <20230702142639.1249681-1-sjg@chromium.org> References: <20230702142639.1249681-1-sjg@chromium.org> MIME-Version: 1.0 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.8 at phobos.denx.de X-Virus-Status: Clean This function only needs the output directory from the builder. This is passed into the builder, so just pass the same value to show_actions(). The avoids needing a builder to call show_actions(). Signed-off-by: Simon Glass --- tools/buildman/control.py | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/tools/buildman/control.py b/tools/buildman/control.py index cde705c1c956..8ecd124fdcf3 100644 --- a/tools/buildman/control.py +++ b/tools/buildman/control.py @@ -48,7 +48,7 @@ def get_action_summary(is_summary, commits, selected, options): return msg # pylint: disable=R0913 -def show_actions(series, why_selected, boards_selected, bldr, options, +def show_actions(series, why_selected, boards_selected, output_dir, options, board_warnings): """Display a list of actions that we would take, if not a dry run. @@ -61,7 +61,7 @@ def show_actions(series, why_selected, boards_selected, bldr, options, the value would be a list of board names. boards_selected: Dict of selected boards, key is target name, value is Board object - bldr: The builder that will be used to build the commits + output_dir (str): Output directory for builder options: Command line options object board_warnings: List of warnings obtained from board selected """ @@ -74,7 +74,7 @@ def show_actions(series, why_selected, boards_selected, bldr, options, commits = None print(get_action_summary(False, commits, boards_selected, options)) - print(f'Build directory: {bldr.base_dir}') + print(f'Build directory: {output_dir}') if commits: for upto in range(0, len(series.commits), options.step): commit = series.commits[upto] @@ -416,7 +416,7 @@ def do_buildman(options, args, toolchains=None, make_func=None, brds=None, # For a dry run, just show our actions as a sanity check if options.dry_run: - show_actions(series, why_selected, selected, builder, options, + show_actions(series, why_selected, selected, output_dir, options, board_warnings) else: builder.force_build = options.force_build From patchwork Sun Jul 2 14:25:34 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1802331 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=85.214.62.61; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.a=rsa-sha256 header.s=google header.b=nCfgjmII; dkim-atps=neutral Received: from phobos.denx.de (phobos.denx.de [85.214.62.61]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QvBHq4ybWz20Pj for ; Mon, 3 Jul 2023 00:28:39 +1000 (AEST) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id 0C92186424; Sun, 2 Jul 2023 16:27:29 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="nCfgjmII"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id DF6C4863C2; Sun, 2 Jul 2023 16:27:17 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.2 Received: from mail-io1-xd31.google.com (mail-io1-xd31.google.com [IPv6:2607:f8b0:4864:20::d31]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id 1CDB186372 for ; Sun, 2 Jul 2023 16:27:15 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=sjg@chromium.org Received: by mail-io1-xd31.google.com with SMTP id ca18e2360f4ac-783698a37beso192565639f.0 for ; Sun, 02 Jul 2023 07:27:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1688308033; x=1690900033; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=qdvmuDITtFBM4LMjMUMIzBJbIdx13csRqyjHv93XYWo=; b=nCfgjmIIgVTlmsggl7DYVdmYGhrDsnLCQNknAdOcVPWGUrs5xcirsVXirg4dPURjBZ CnF4T3pZ/JUA10j1I/WQRcsHPATfFKPfQVH4zIqZ50hnLdthDkbNvVBUSJWkf0LGSk4B R1lEvdjyHFSZI31p5N1IacC55jLs9c4CFShaE= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1688308033; x=1690900033; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=qdvmuDITtFBM4LMjMUMIzBJbIdx13csRqyjHv93XYWo=; b=ITuEYDHBs0lPGdo/kLCnLbj8XVjUC9QOgh7dNiXRCPDCCfOYhgQyVP8QZEXCum4c77 k6lAW01cTSf4fVm6eC0YrYGcmhi7amT5j/i8FUFlM9pq++kHgOxanZSKe/XUVSclPZ7j +/m66rAOyUZkqKWB218u+4fmHwirDETEGdp0sEXRNGUjU2yB3j6trFI2JeFHKtG7fTgd Q/0zfFDzl+EO2Uf9rHcbHbXjY91BFVPdvew7RxMZqGiJRSIWrdewI6uy1z8WBrxD1WpZ JENQhRFWOFzSVb5RdkA5T5nXdjS6sTdjbglfnpM+OTG78j8DVg6rj/tJEv33ZdgoE7Pk ktRg== X-Gm-Message-State: AC+VfDzq5k/H/S7myzjtnhk3+Lwgh0TfVxwS0w1dGZRkkdW7tFc80vYa U/pBSald+xNBV2cNdbpMA/0vky89nZYWdmCZuTk= X-Google-Smtp-Source: ACHHUZ4+ftC25m+V/KAs2dWQt9B7L854Ozt6oTcoaG4QcfCp3qS/6djWq+qd64CM7Nb5EcUfDUb2/Q== X-Received: by 2002:a5d:8045:0:b0:783:5452:e335 with SMTP id b5-20020a5d8045000000b007835452e335mr8782701ior.6.1688308033747; Sun, 02 Jul 2023 07:27:13 -0700 (PDT) Received: from sjg1.lan (c-73-14-173-85.hsd1.co.comcast.net. [73.14.173.85]) by smtp.gmail.com with ESMTPSA id c13-20020a02c9cd000000b0042af5268e4asm2934713jap.58.2023.07.02.07.27.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 02 Jul 2023 07:27:13 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Cc: Simon Glass Subject: [PATCH 09/58] buildman: Move dry-run handling higher in do_buildman() Date: Sun, 2 Jul 2023 15:25:34 +0100 Message-ID: <20230702142639.1249681-10-sjg@chromium.org> X-Mailer: git-send-email 2.41.0.255.g8b1d071c50-goog In-Reply-To: <20230702142639.1249681-1-sjg@chromium.org> References: <20230702142639.1249681-1-sjg@chromium.org> MIME-Version: 1.0 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.8 at phobos.denx.de X-Virus-Status: Clean Move this up above where the builder is created, since it no-longer makes use of the builder. Signed-off-by: Simon Glass --- tools/buildman/control.py | 88 ++++++++++++++++++++------------------- 1 file changed, 45 insertions(+), 43 deletions(-) diff --git a/tools/buildman/control.py b/tools/buildman/control.py index 8ecd124fdcf3..9a53d70f4344 100644 --- a/tools/buildman/control.py +++ b/tools/buildman/control.py @@ -384,6 +384,13 @@ def do_buildman(options, args, toolchains=None, make_func=None, brds=None, output_dir = os.path.join(options.output_dir, dirname) if clean_dir and os.path.exists(output_dir): shutil.rmtree(output_dir) + + # For a dry run, just show our actions as a sanity check + if options.dry_run: + show_actions(series, why_selected, selected, output_dir, options, + board_warnings) + return 0 + adjust_cfg = cfgutil.convert_list_to_dict(options.adjust_cfg) # Drop LOCALVERSION_AUTO since it changes the version string on every commit @@ -414,48 +421,43 @@ def do_buildman(options, args, toolchains=None, make_func=None, brds=None, if make_func: builder.do_make = make_func - # For a dry run, just show our actions as a sanity check - if options.dry_run: - show_actions(series, why_selected, selected, output_dir, options, - board_warnings) + builder.force_build = options.force_build + builder.force_build_failures = options.force_build_failures + builder.force_reconfig = options.force_reconfig + builder.in_tree = options.in_tree + + # Work out which boards to build + board_selected = brds.get_selected_dict() + + if series: + commits = series.commits + # Number the commits for test purposes + for i, commit in enumerate(commits): + commit.sequence = i else: - builder.force_build = options.force_build - builder.force_build_failures = options.force_build_failures - builder.force_reconfig = options.force_reconfig - builder.in_tree = options.in_tree - - # Work out which boards to build - board_selected = brds.get_selected_dict() - - if series: - commits = series.commits - # Number the commits for test purposes - for i, commit in enumerate(commits): - commit.sequence = i - else: - commits = None - - if not options.ide: - tprint(get_action_summary(options.summary, commits, board_selected, - options)) - - # We can't show function sizes without board details at present - if options.show_bloat: - options.show_detail = True - builder.SetDisplayOptions( - options.show_errors, options.show_sizes, options.show_detail, - options.show_bloat, options.list_error_boards, options.show_config, - options.show_environment, options.filter_dtb_warnings, - options.filter_migration_warnings, options.ide) - if options.summary: - builder.ShowSummary(commits, board_selected) - else: - fail, warned, excs = builder.BuildBoards( - commits, board_selected, options.keep_outputs, options.verbose) - if excs: - return 102 - if fail: - return 100 - if warned and not options.ignore_warnings: - return 101 + commits = None + + if not options.ide: + tprint(get_action_summary(options.summary, commits, board_selected, + options)) + + # We can't show function sizes without board details at present + if options.show_bloat: + options.show_detail = True + builder.SetDisplayOptions( + options.show_errors, options.show_sizes, options.show_detail, + options.show_bloat, options.list_error_boards, options.show_config, + options.show_environment, options.filter_dtb_warnings, + options.filter_migration_warnings, options.ide) + if options.summary: + builder.ShowSummary(commits, board_selected) + else: + fail, warned, excs = builder.BuildBoards( + commits, board_selected, options.keep_outputs, options.verbose) + if excs: + return 102 + if fail: + return 100 + if warned and not options.ignore_warnings: + return 101 return 0 From patchwork Sun Jul 2 14:25:35 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1802332 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=2a01:238:438b:c500:173d:9f52:ddab:ee01; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.a=rsa-sha256 header.s=google header.b=fgn3xOQo; dkim-atps=neutral Received: from phobos.denx.de (phobos.denx.de [IPv6:2a01:238:438b:c500:173d:9f52:ddab:ee01]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QvBJ145ZGz20Pj for ; Mon, 3 Jul 2023 00:28:49 +1000 (AEST) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id E9146865ED; Sun, 2 Jul 2023 16:27:30 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="fgn3xOQo"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id AE8C6865B0; Sun, 2 Jul 2023 16:27:19 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.2 Received: from mail-il1-x12b.google.com (mail-il1-x12b.google.com [IPv6:2607:f8b0:4864:20::12b]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id 2347286346 for ; Sun, 2 Jul 2023 16:27:17 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=sjg@chromium.org Received: by mail-il1-x12b.google.com with SMTP id e9e14a558f8ab-3457157a164so15110755ab.1 for ; Sun, 02 Jul 2023 07:27:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1688308035; x=1690900035; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=rFhNe0vUY01RtCWbG5U9P5DJhFOIb6OlF9P8VDxZcTg=; b=fgn3xOQou0fDrgCJQRyWFqNb51W9aIqzookICk1idVPbXdyKP77deXOpjloA4/kQSq XaJt0RzKYuTVG6nTML9MMc5r4BcmZchH9TwzO1NZ4xY2no3GZopNX2xa44uA2PbSnwwW buBYd+gRnbLpsoSxRxmdfyAs5UeJjOVtxdL0w= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1688308035; x=1690900035; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=rFhNe0vUY01RtCWbG5U9P5DJhFOIb6OlF9P8VDxZcTg=; b=TOK3MceCvwS3O244lALYpCpaqsdIy3kggVA48VMuFo90T7pER/dMyc9E5ccqs42Afz FaLMn5QMs5DKjp5NowNfVTOgD5CbvbwymOz23ttsjc82FpZvoDYgvNJTQctmqHBEa33O sSJ/bI4IHetM6cLOGMDtzlRTHL/B9unrQnRUEL/AmCgx2LdyX1YaKEDmKvvQOtOLufji AfLv547Ki9tKu10w9KxLPqgXx6XUdBbPdmBM7BD8wrk6iF4+yz8i7HaRRt9Ew8aRyGIX r0nq8Ujhok6rTZyfzdtdHIpj4Z3i5io5Cp1zartjNhI3NBn/bGCRpHSZzFt6D5B1DoKb mm0A== X-Gm-Message-State: ABy/qLYLsQguLx+j1NfaQERmK+AMJBtJX8vL3xDM7a6AbJMkKiXkwWCs xthOTdgylMUk0S4mLcIiVUX1QeN77hLvsDehQuU= X-Google-Smtp-Source: APBJJlG1mNuf14wLQwdYXyyzW4dORSuNarZMruxzQmu54n2GBa/jTiEl8uVTHs5xY0awbz8raWU/MA== X-Received: by 2002:a05:6e02:60c:b0:345:d79d:228f with SMTP id t12-20020a056e02060c00b00345d79d228fmr7025078ils.29.1688308035687; Sun, 02 Jul 2023 07:27:15 -0700 (PDT) Received: from sjg1.lan (c-73-14-173-85.hsd1.co.comcast.net. [73.14.173.85]) by smtp.gmail.com with ESMTPSA id c13-20020a02c9cd000000b0042af5268e4asm2934713jap.58.2023.07.02.07.27.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 02 Jul 2023 07:27:15 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Cc: Simon Glass Subject: [PATCH 10/58] buildman: Move board-selection code into a function Date: Sun, 2 Jul 2023 15:25:35 +0100 Message-ID: <20230702142639.1249681-11-sjg@chromium.org> X-Mailer: git-send-email 2.41.0.255.g8b1d071c50-goog In-Reply-To: <20230702142639.1249681-1-sjg@chromium.org> References: <20230702142639.1249681-1-sjg@chromium.org> MIME-Version: 1.0 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.8 at phobos.denx.de X-Virus-Status: Clean Create a new determine_boards() function to hold the code which selects which boards to build. Signed-off-by: Simon Glass --- tools/buildman/control.py | 59 ++++++++++++++++++++++++++++----------- 1 file changed, 43 insertions(+), 16 deletions(-) diff --git a/tools/buildman/control.py b/tools/buildman/control.py index 9a53d70f4344..4055e7aee6ce 100644 --- a/tools/buildman/control.py +++ b/tools/buildman/control.py @@ -222,6 +222,47 @@ def do_fetch_arch(toolchains, col, fetch_arch): return 0 +def determine_boards(brds, args, col, opt_boards, exclude): + """Determine which boards to build + + Each element of args and exclude can refer to a board name, arch or SoC + + Args: + brds (Boards): Boards object + args (list of str): Arguments describing boards to build + col (Terminal.Color): Color object + opt_boards (list of str): Specific boards to build, or None for all + exclude (list of str): Arguments describing boards to exclude + + Returns: + tuple: + list of Board: List of Board objects that are marked selected + why_selected: Dictionary where each key is a buildman argument + provided by the user, and the value is the list of boards + brought in by that argument. For example, 'arm' might bring + in 400 boards, so in this case the key would be 'arm' and + the value would be a list of board names. + board_warnings: List of warnings obtained from board selected + """ + exclude = [] + if exclude: + for arg in exclude: + exclude += arg.split(',') + + if opt_boards: + requested_boards = [] + for brd in opt_boards: + requested_boards += brd.split(',') + else: + requested_boards = None + why_selected, board_warnings = brds.select_boards(args, exclude, + requested_boards) + selected = brds.get_selected() + if not selected: + sys.exit(col.build(col.RED, 'No matching boards found')) + return selected, why_selected, board_warnings + + def do_buildman(options, args, toolchains=None, make_func=None, brds=None, clean_dir=False, test_thread_exceptions=False): """The main control code for buildman @@ -295,22 +336,8 @@ def do_buildman(options, args, toolchains=None, make_func=None, brds=None, sys.exit(col.build(col.RED, err)) return 0 - exclude = [] - if options.exclude: - for arg in options.exclude: - exclude += arg.split(',') - - if options.boards: - requested_boards = [] - for brd in options.boards: - requested_boards += brd.split(',') - else: - requested_boards = None - why_selected, board_warnings = brds.select_boards(args, exclude, - requested_boards) - selected = brds.get_selected() - if not selected: - sys.exit(col.build(col.RED, 'No matching boards found')) + selected, why_selected, board_warnings = determine_boards( + brds, args, col, options.boards, options.exclude) # Work out how many commits to build. We want to build everything on the # branch. We also build the upstream commit as a control so we can see From patchwork Sun Jul 2 14:25:36 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1802333 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=2a01:238:438b:c500:173d:9f52:ddab:ee01; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.a=rsa-sha256 header.s=google header.b=fw6E6h7P; dkim-atps=neutral Received: from phobos.denx.de (phobos.denx.de [IPv6:2a01:238:438b:c500:173d:9f52:ddab:ee01]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QvBJC5197z20Pj for ; Mon, 3 Jul 2023 00:28:59 +1000 (AEST) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id AC17F865E2; Sun, 2 Jul 2023 16:27:32 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="fw6E6h7P"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id A7F59865BA; Sun, 2 Jul 2023 16:27:20 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.2 Received: from mail-il1-x12a.google.com (mail-il1-x12a.google.com [IPv6:2607:f8b0:4864:20::12a]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id D1609863A5 for ; Sun, 2 Jul 2023 16:27:17 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=sjg@chromium.org Received: by mail-il1-x12a.google.com with SMTP id e9e14a558f8ab-345f4a3ff76so9197725ab.2 for ; Sun, 02 Jul 2023 07:27:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1688308036; x=1690900036; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=2Z+6EYosec3C3qyZ9MSsl/0fBqkj72zUKuM8mjm1h3I=; b=fw6E6h7P2f8P1bY/cuISNDc4uFzZ6mROeTDUhuTSlUHTuF8kUg1JxdBPMwccuZRM0d 1N4UOt8L5CSEFTfDKl30bUhWrMDRi3JWNTgw3jjxy/BABWFBJOTv5PqrVU7Fr7Sh0DWq iRsTSUbhFAnxIyey82iAmydA7CxcdZaFQ+Bt4= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1688308036; x=1690900036; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=2Z+6EYosec3C3qyZ9MSsl/0fBqkj72zUKuM8mjm1h3I=; b=TrSOITBXLMrn+3D0LflF6VbYtAlcRGNybSq5gKcWkS8ApuqNpmhLPNkkwYnx5NHNAr IXm2q980yHegHdl5yQivizUbUNv/OZ+1T0jW0sJJgK0AooRopn1jRfIALaWMZz6wNwrD PIN0rttvikf5J/qw2qV2gKiIbY+geZqXD4eY0ci/yo85AXQM2J649TRAOy/G4floY0dw oLxB6N0Im7pDpxbPz2JA0IsgwNGe0PWyNy/8rSaOyDvciXR58Ss3qGmpjhQwr8tw1Bf3 W0I+m9Jy3KcsgDZ/n2Ng3047qiJ9O0t4jG9oJR9RRfKvISZwRxg0X83WO58PUDdPwNoc 4/8g== X-Gm-Message-State: ABy/qLbrbdBl37GVrRiRRHHTmuMqxpxkJNqxDPIoTzWoHmNMmLIrach0 07Qx98LEoon8pkGkU8dXGNaqWwNBBDY0biKmAoM= X-Google-Smtp-Source: APBJJlEb9S2vt0uDHYkpliqiuMuZ1ILQKZ+8Px9f7YO3bDwNq2/6Lj5bQHLUWHgKePZlvRlNN6kk3w== X-Received: by 2002:a92:d185:0:b0:346:e96:7cbd with SMTP id z5-20020a92d185000000b003460e967cbdmr359343ilz.19.1688308036416; Sun, 02 Jul 2023 07:27:16 -0700 (PDT) Received: from sjg1.lan (c-73-14-173-85.hsd1.co.comcast.net. [73.14.173.85]) by smtp.gmail.com with ESMTPSA id c13-20020a02c9cd000000b0042af5268e4asm2934713jap.58.2023.07.02.07.27.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 02 Jul 2023 07:27:16 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Cc: Simon Glass Subject: [PATCH 11/58] bulidman: Move more code to determine_series() Date: Sun, 2 Jul 2023 15:25:36 +0100 Message-ID: <20230702142639.1249681-12-sjg@chromium.org> X-Mailer: git-send-email 2.41.0.255.g8b1d071c50-goog In-Reply-To: <20230702142639.1249681-1-sjg@chromium.org> References: <20230702142639.1249681-1-sjg@chromium.org> MIME-Version: 1.0 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.8 at phobos.denx.de X-Virus-Status: Clean Move some more series-related code here, to reduce the size of the main function. Signed-off-by: Simon Glass --- tools/buildman/control.py | 82 ++++++++++++++++++++------------------- 1 file changed, 42 insertions(+), 40 deletions(-) diff --git a/tools/buildman/control.py b/tools/buildman/control.py index 4055e7aee6ce..b88862a22c4a 100644 --- a/tools/buildman/control.py +++ b/tools/buildman/control.py @@ -148,14 +148,17 @@ def get_allow_missing(opt_allow, opt_no_allow, num_selected, has_branch): return allow_missing -def determine_series(count, has_range, branch, git_dir): +def determine_series(selected, col, git_dir, count, branch, work_in_output): """Determine the series which is to be built, if any Args: + selected (list of Board(: List of Board objects that are marked + selected + col (Terminal.Color): Color object to use + git_dir (str): Git directory to use, e.g. './.git' count (int): Number of commits in branch - has_range (bool): True if a range of commits ('xx..yy') is being built branch (str): Name of branch to build, or None if none - git_dir (str): Git directory to use, e.g. './.git' + work_in_output (bool): True to work in the output directory Returns: Series: Series to build, or None for none @@ -171,6 +174,40 @@ def determine_series(count, has_range, branch, git_dir): other hand conflicting tags will cause an error. So allow later tags to overwrite earlier ones by setting allow_overwrite=True """ + + # Work out how many commits to build. We want to build everything on the + # branch. We also build the upstream commit as a control so we can see + # problems introduced by the first commit on the branch. + has_range = branch and '..' in branch + if count == -1: + if not branch: + count = 1 + else: + if has_range: + count, msg = gitutil.count_commits_in_range(git_dir, branch) + else: + count, msg = gitutil.count_commits_in_branch(git_dir, branch) + if count is None: + sys.exit(col.build(col.RED, msg)) + elif count == 0: + sys.exit(col.build(col.RED, + f"Range '{branch}' has no commits")) + if msg: + print(col.build(col.YELLOW, msg)) + count += 1 # Build upstream commit also + + if not count: + msg = (f"No commits found to process in branch '{branch}': " + "set branch's upstream or use -c flag") + sys.exit(col.build(col.RED, msg)) + if work_in_output: + if len(selected) != 1: + sys.exit(col.build(col.RED, + '-w can only be used with a single board')) + if count != 1: + sys.exit(col.build(col.RED, + '-w can only be used with a single commit')) + if branch: if count == -1: if has_range: @@ -339,43 +376,8 @@ def do_buildman(options, args, toolchains=None, make_func=None, brds=None, selected, why_selected, board_warnings = determine_boards( brds, args, col, options.boards, options.exclude) - # Work out how many commits to build. We want to build everything on the - # branch. We also build the upstream commit as a control so we can see - # problems introduced by the first commit on the branch. - count = options.count - has_range = options.branch and '..' in options.branch - if count == -1: - if not options.branch: - count = 1 - else: - if has_range: - count, msg = gitutil.count_commits_in_range(git_dir, - options.branch) - else: - count, msg = gitutil.count_commits_in_branch(git_dir, - options.branch) - if count is None: - sys.exit(col.build(col.RED, msg)) - elif count == 0: - sys.exit(col.build(col.RED, - f"Range '{options.branch}' has no commits")) - if msg: - print(col.build(col.YELLOW, msg)) - count += 1 # Build upstream commit also - - if not count: - msg = (f"No commits found to process in branch '{options.branch}': " - "set branch's upstream or use -c flag") - sys.exit(col.build(col.RED, msg)) - if options.work_in_output: - if len(selected) != 1: - sys.exit(col.build(col.RED, - '-w can only be used with a single board')) - if count != 1: - sys.exit(col.build(col.RED, - '-w can only be used with a single commit')) - - series = determine_series(count, has_range, options.branch, git_dir) + series = determine_series(selected, col, git_dir, options.count, + options.branch, options.work_in_output) if not series and not options.dry_run: options.verbose = True if not options.summary: From patchwork Sun Jul 2 14:25:37 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1802334 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=2a01:238:438b:c500:173d:9f52:ddab:ee01; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.a=rsa-sha256 header.s=google header.b=Tpq2f1XR; dkim-atps=neutral Received: from phobos.denx.de (phobos.denx.de [IPv6:2a01:238:438b:c500:173d:9f52:ddab:ee01]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QvBJR1bhtz20Pj for ; Mon, 3 Jul 2023 00:29:11 +1000 (AEST) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id 0ABE28639C; Sun, 2 Jul 2023 16:27:35 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="Tpq2f1XR"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id E96F28638B; Sun, 2 Jul 2023 16:27:22 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.2 Received: from mail-il1-x12b.google.com (mail-il1-x12b.google.com [IPv6:2607:f8b0:4864:20::12b]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id 738C8863BA for ; Sun, 2 Jul 2023 16:27:18 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=sjg@chromium.org Received: by mail-il1-x12b.google.com with SMTP id e9e14a558f8ab-345a8a78bcfso15054175ab.2 for ; Sun, 02 Jul 2023 07:27:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1688308037; x=1690900037; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=SVxwlGCys4CU2LVTGck7E/eKxaoPMgueRcaJhS3lidw=; b=Tpq2f1XRzCgWZ9L6pFXvefx+h7oO8AkovWd+QvfdKKpGPkcMAOjpsM7+TNULRmIOUi lMmiLXuwpkwHNhEQyJgUQzl+KTyX7mFCLK7Wd0PhkIV0qGEW3iP4r3c2kHOLQelx0Ocv Z9+bfN+D95Y5Fss22td3ddClGs3To4NVT+KN0= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1688308037; x=1690900037; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=SVxwlGCys4CU2LVTGck7E/eKxaoPMgueRcaJhS3lidw=; b=XXNC0Sn3s36oXNZwygFu/6dWCagqPiFdludQzvwuB2oqk8wb4luFL+kK5aVvw2oJae /5L9meUex4EbpQhXtwwCnxeODbSTut77ulnEOieJDXFf4dbVPgeHGxmsfgTFU5GNTR02 uWUwECCBR6VkUpmHQRWgcsH5Jab1if+gvEqYXXQ20jnyB0yC/inLRdEra9B7WerVP5W5 CoLnKg0dh42dtHsvPDMnK8XbRr/qcFCHFyESDmfwaB0SAkNnaD+pCQx7lU4OCZNbuGHz JhQ9sIASGymUL4QjIDMbDvdhxYf2tMBIzshtHJ5Fsi0VqqQ1gaYo179fIGYO7c0Sog6S 18wQ== X-Gm-Message-State: ABy/qLZ7puDGayHewpQlFH113ZFd979Os+N8s5vLcqMBblSpf/BqLUfG knigKuhaPcCMSyJrnCbrGG/pIViN88Ta6ADPzXM= X-Google-Smtp-Source: APBJJlEJXHyXCuz3JnciKej2Tb0DB8U86l8wBF5dlaAYRcQixWPco2V8Oug3UyJZAijtJpXNZEVqkg== X-Received: by 2002:a92:d692:0:b0:346:9e1:bb10 with SMTP id p18-20020a92d692000000b0034609e1bb10mr1159576iln.8.1688308037102; Sun, 02 Jul 2023 07:27:17 -0700 (PDT) Received: from sjg1.lan (c-73-14-173-85.hsd1.co.comcast.net. [73.14.173.85]) by smtp.gmail.com with ESMTPSA id c13-20020a02c9cd000000b0042af5268e4asm2934713jap.58.2023.07.02.07.27.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 02 Jul 2023 07:27:16 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Cc: Simon Glass Subject: [PATCH 12/58] buildman: Move Boards-object code into a function Date: Sun, 2 Jul 2023 15:25:37 +0100 Message-ID: <20230702142639.1249681-13-sjg@chromium.org> X-Mailer: git-send-email 2.41.0.255.g8b1d071c50-goog In-Reply-To: <20230702142639.1249681-1-sjg@chromium.org> References: <20230702142639.1249681-1-sjg@chromium.org> MIME-Version: 1.0 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.8 at phobos.denx.de X-Virus-Status: Clean Move the code which obtains a Boards object into its own function, to reduce the size of the main function. Signed-off-by: Simon Glass --- tools/buildman/control.py | 54 ++++++++++++++++++++++++++++----------- 1 file changed, 39 insertions(+), 15 deletions(-) diff --git a/tools/buildman/control.py b/tools/buildman/control.py index b88862a22c4a..a409c4882972 100644 --- a/tools/buildman/control.py +++ b/tools/buildman/control.py @@ -259,6 +259,41 @@ def do_fetch_arch(toolchains, col, fetch_arch): return 0 +def get_boards_obj(output_dir, regen_board_list, threads, verbose): + """Object the Boards object to use + + Creates the output directory and ensures there is a boards.cfg file, then + read it in. + + Args: + output_dir (str): Output directory to use + regen_board_list (bool): True to just regenerate the board list + threads (int or None): Number of threads to use to create boards file + verbose (bool): False to suppress output from boards-file generation + + Returns: + Either: + int: Operation completed and buildman should exit with exit code + Boards: Boards object to use + """ + if not os.path.exists(output_dir): + os.makedirs(output_dir) + board_file = os.path.join(output_dir, 'boards.cfg') + if regen_board_list and regen_board_list != '-': + board_file = regen_board_list + + brds = boards.Boards() + okay = brds.ensure_board_list( + board_file, + threads or multiprocessing.cpu_count(), + force=regen_board_list, + quiet=not verbose) + if regen_board_list: + return 0 if okay else 2 + brds.read_boards(board_file) + return brds + + def determine_boards(brds, args, col, opt_boards, exclude): """Determine which boards to build @@ -351,21 +386,10 @@ def do_buildman(options, args, toolchains=None, make_func=None, brds=None, # Work out what subset of the boards we are building if not brds: - if not os.path.exists(options.output_dir): - os.makedirs(options.output_dir) - board_file = os.path.join(options.output_dir, 'boards.cfg') - if options.regen_board_list and options.regen_board_list != '-': - board_file = options.regen_board_list - - brds = boards.Boards() - okay = brds.ensure_board_list( - board_file, - options.threads or multiprocessing.cpu_count(), - force=options.regen_board_list, - quiet=not options.verbose) - if options.regen_board_list: - return 0 if okay else 2 - brds.read_boards(board_file) + brds = get_boards_obj(options.output_dir, options.regen_board_list, + options.threads, options.verbose) + if isinstance(brds, int): + return brds if options.print_prefix: err = show_toolchain_prefix(brds, toolchains) From patchwork Sun Jul 2 14:25:38 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1802335 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=85.214.62.61; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.a=rsa-sha256 header.s=google header.b=FzSmZ5bH; dkim-atps=neutral Received: from phobos.denx.de (phobos.denx.de [85.214.62.61]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QvBJd0bLXz20Pj for ; Mon, 3 Jul 2023 00:29:21 +1000 (AEST) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id 2102586605; Sun, 2 Jul 2023 16:27:37 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="FzSmZ5bH"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id 61D4686424; Sun, 2 Jul 2023 16:27:23 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.2 Received: from mail-io1-xd2b.google.com (mail-io1-xd2b.google.com [IPv6:2607:f8b0:4864:20::d2b]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id 897BA86442 for ; Sun, 2 Jul 2023 16:27:19 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=sjg@chromium.org Received: by mail-io1-xd2b.google.com with SMTP id ca18e2360f4ac-77b00bb3fd6so82083539f.1 for ; Sun, 02 Jul 2023 07:27:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1688308038; x=1690900038; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=zUehbxG1YBZlOiVenAnnHWyXbyjomOG5J9E0NRb7ZaA=; b=FzSmZ5bHdwxVJqV1XwCQahmFZpEgvx3+CCLEsq5+qXB0xowCkYWCp0goDAMHyJkOa+ Y9kqT0UkLg4rEKuGfLuUPQSc2uS7Th8Ppwo2SxOaB9yrBIykhBL04iVlX89D1WTWSN78 i+LXLfknoLJcyQbjOmFj9kOatFxbKACwQB+8o= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1688308038; x=1690900038; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=zUehbxG1YBZlOiVenAnnHWyXbyjomOG5J9E0NRb7ZaA=; b=Ox+Xr8wfFx+r4PyzPxXiC/Pmy6PmhVPcRLrBt0Da+XWW+ls350tDqFhv0M+0Ct9UGt qKjjcLiEfCt+CWgRfn/SfYAT7OWcUnPxv51SjlBcBnGtwSVxi78twpquB96B9THfvMjZ J5eOE9gO9wQG0ID77st4OQZ1h03iuRLnd9ZrX7+03W3u5SYZw23oUWV/oXm7xtNzmN6S S9oHmR31ZuTEwlx4D4ST0ls+KWJOEoHNN3cI2/LjIH9zgdPUAYU0YkVYX/oFm9Sqd8dI aTqLaOi6DObNWKdESkgTItW6XZKB+iLJDeilN9lhJlmj6Xc/aMVGMpCZIT3TAm64aW05 voWg== X-Gm-Message-State: AC+VfDwqOW7ukOXrhqOUxxI9YmSb2ahwhg9iD+FiNi0a33uErMkzpHUh WF6TZtOiXVHQkkru/C63LL4hHPaWwtJwirLtefg= X-Google-Smtp-Source: ACHHUZ44hdpp1YFgznnJTHCkWOKD1YsFKEAPmOE+A4fW+bzDPs7fuQ541S2xiD14mN3g+u9gGtjt9Q== X-Received: by 2002:a05:6e02:5c7:b0:33f:eb60:519d with SMTP id l7-20020a056e0205c700b0033feb60519dmr8588849ils.2.1688308038123; Sun, 02 Jul 2023 07:27:18 -0700 (PDT) Received: from sjg1.lan (c-73-14-173-85.hsd1.co.comcast.net. [73.14.173.85]) by smtp.gmail.com with ESMTPSA id c13-20020a02c9cd000000b0042af5268e4asm2934713jap.58.2023.07.02.07.27.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 02 Jul 2023 07:27:17 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Cc: Simon Glass Subject: [PATCH 13/58] bulidman: Move toolchain handling to a functoin Date: Sun, 2 Jul 2023 15:25:38 +0100 Message-ID: <20230702142639.1249681-14-sjg@chromium.org> X-Mailer: git-send-email 2.41.0.255.g8b1d071c50-goog In-Reply-To: <20230702142639.1249681-1-sjg@chromium.org> References: <20230702142639.1249681-1-sjg@chromium.org> MIME-Version: 1.0 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.8 at phobos.denx.de X-Virus-Status: Clean Move the code for dealing with toolchains out into its own function, to reduce the size of the main function. Signed-off-by: Simon Glass --- tools/buildman/control.py | 53 ++++++++++++++++++++++++++++----------- 1 file changed, 38 insertions(+), 15 deletions(-) diff --git a/tools/buildman/control.py b/tools/buildman/control.py index a409c4882972..13fc4799cf8b 100644 --- a/tools/buildman/control.py +++ b/tools/buildman/control.py @@ -259,6 +259,41 @@ def do_fetch_arch(toolchains, col, fetch_arch): return 0 +def get_toolchains(toolchains, col, override_toolchain, fetch_arch, + list_tool_chains, verbose): + """Get toolchains object to use + + Args: + toolchains (Toolchains or None): Toolchains to use. If None, then a + Toolchains object will be created and scanned + col (Terminal.Color): Color object + override_toolchain (str or None): Override value for toolchain, or None + fetch_arch (bool): True to fetch the toolchain for the architectures + list_tool_chains (bool): True to list all tool chains + verbose (bool): True for verbose output when listing toolchains + +Returns: + Either: + int: Operation completed and buildman should exit with exit code + Toolchains: Toolchains object to use + """ + no_toolchains = toolchains is None + if no_toolchains: + toolchains = toolchain.Toolchains(override_toolchain) + + if fetch_arch: + return do_fetch_arch(toolchains, col, fetch_arch) + + if no_toolchains: + toolchains.GetSettings() + toolchains.Scan(list_tool_chains and verbose) + if list_tool_chains: + toolchains.List() + print() + return 0 + return toolchains + + def get_boards_obj(output_dir, regen_board_list, threads, verbose): """Object the Boards object to use @@ -364,21 +399,9 @@ def do_buildman(options, args, toolchains=None, make_func=None, brds=None, git_dir = os.path.join(options.git, '.git') - no_toolchains = toolchains is None - if no_toolchains: - toolchains = toolchain.Toolchains(options.override_toolchain) - - if options.fetch_arch: - return do_fetch_arch(toolchains, col, options.fetch_arch) - - if no_toolchains: - toolchains.GetSettings() - toolchains.Scan(options.list_tool_chains and options.verbose) - if options.list_tool_chains: - toolchains.List() - print() - return 0 - + toolchains = get_toolchains(toolchains, col, options.override_toolchain, + options.fetch_arch, options.list_tool_chains, + options.verbose) if not options.output_dir: if options.work_in_output: sys.exit(col.build(col.RED, '-w requires that you specify -o')) From patchwork Sun Jul 2 14:25:39 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1802336 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=85.214.62.61; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.a=rsa-sha256 header.s=google header.b=cLSthFUE; dkim-atps=neutral Received: from phobos.denx.de (phobos.denx.de [85.214.62.61]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QvBJp4p0Kz20Pj for ; Mon, 3 Jul 2023 00:29:30 +1000 (AEST) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id 20C2C865EB; Sun, 2 Jul 2023 16:27:39 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="cLSthFUE"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id 1AE73865B4; Sun, 2 Jul 2023 16:27:25 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.2 Received: from mail-il1-x129.google.com (mail-il1-x129.google.com [IPv6:2607:f8b0:4864:20::129]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id F3D17865BC for ; Sun, 2 Jul 2023 16:27:20 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=sjg@chromium.org Received: by mail-il1-x129.google.com with SMTP id e9e14a558f8ab-345a32f3eeaso8775365ab.1 for ; Sun, 02 Jul 2023 07:27:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1688308039; x=1690900039; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=MtxE0/Y5d/WLDZye7PIosMrkh2CwuxLjtnrVV7cx0K8=; b=cLSthFUEE8g4BVqtgAm2SM/LUn0eydrEIssK+PpMOXgkWFose9K2NRlqgWTHpFTJQD 3ZstQkZcvKc4KxIYv5Pk+sfLkcpnk8rRKiqsE/vU54OoV8LaaIfVjS5s11rc1SMp/AEg XB50UFIZywgREzFyEVOIYjYOvCcvkC3AYbz/4= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1688308039; x=1690900039; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=MtxE0/Y5d/WLDZye7PIosMrkh2CwuxLjtnrVV7cx0K8=; b=GT9mWM7+vCAKtZ0JbLohAI4XAasnWv9+GloQ1rQRmQ7npvPCs59Kdpq8WPnLRELACE PQ//x+J9FbmkND/7P+7dhj4yJw4WBYjNKXsrkSJCgoFydQADFxkzdYgdeVPTyuouERT2 XDUuoTR3/MRehvnsJ88EkrNM7gaVCGPESUv3tdNkjZ9+11CuWgLQu/6E7zAMkQloGUiu TQ2jUR0Wd77R6b4umWpuVJekV+1Yy5bMr2BCjBAwQ/1GXQSncXf9McVJU540ZHm+3idA b/M3wV6ympRPc1eld/eymuy9ZtCQHVksTMxBrvNCFQeQ17FvSaaQ2aVtGHj+sFgFq6wU 5AKA== X-Gm-Message-State: AC+VfDxgY69Xpbf9V7oSb8PSSDSSyo8wMaivuLx4k7ErZprF72qWFEVk XaeLU5Z1WazChDBMWHAi47hqKlL1WQygGAWDS0c= X-Google-Smtp-Source: ACHHUZ4xu/CberXzkxim094KAnjK4GhYdQeaN0s1gN12N3tCThApZkfUSkN1nqUncKsGRmwWWIftzA== X-Received: by 2002:a05:6e02:1688:b0:343:ef5e:8286 with SMTP id f8-20020a056e02168800b00343ef5e8286mr10871619ila.7.1688308039530; Sun, 02 Jul 2023 07:27:19 -0700 (PDT) Received: from sjg1.lan (c-73-14-173-85.hsd1.co.comcast.net. [73.14.173.85]) by smtp.gmail.com with ESMTPSA id c13-20020a02c9cd000000b0042af5268e4asm2934713jap.58.2023.07.02.07.27.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 02 Jul 2023 07:27:19 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Cc: Simon Glass Subject: [PATCH 14/58] bulldman: Set up output_dir earlier Date: Sun, 2 Jul 2023 15:25:39 +0100 Message-ID: <20230702142639.1249681-15-sjg@chromium.org> X-Mailer: git-send-email 2.41.0.255.g8b1d071c50-goog In-Reply-To: <20230702142639.1249681-1-sjg@chromium.org> References: <20230702142639.1249681-1-sjg@chromium.org> MIME-Version: 1.0 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.8 at phobos.denx.de X-Virus-Status: Clean Set up output_dir at the start of the main function, instead of updating the options.output_dir option. Signed-off-by: Simon Glass --- tools/buildman/control.py | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/tools/buildman/control.py b/tools/buildman/control.py index 13fc4799cf8b..86b00e5e5ae4 100644 --- a/tools/buildman/control.py +++ b/tools/buildman/control.py @@ -402,14 +402,15 @@ def do_buildman(options, args, toolchains=None, make_func=None, brds=None, toolchains = get_toolchains(toolchains, col, options.override_toolchain, options.fetch_arch, options.list_tool_chains, options.verbose) - if not options.output_dir: + output_dir = options.output_dir + if not output_dir: if options.work_in_output: sys.exit(col.build(col.RED, '-w requires that you specify -o')) - options.output_dir = '..' + output_dir = '..' # Work out what subset of the boards we are building if not brds: - brds = get_boards_obj(options.output_dir, options.regen_board_list, + brds = get_boards_obj(output_dir, options.regen_board_list, options.threads, options.verbose) if isinstance(brds, int): return brds @@ -451,13 +452,12 @@ def do_buildman(options, args, toolchains=None, make_func=None, brds=None, options.branch) # Create a new builder with the selected options. - output_dir = options.output_dir if options.branch: dirname = options.branch.replace('/', '_') # As a special case allow the board directory to be placed in the # output directory itself rather than any subdirectory. if not options.no_subdirs: - output_dir = os.path.join(options.output_dir, dirname) + output_dir = os.path.join(output_dir, dirname) if clean_dir and os.path.exists(output_dir): shutil.rmtree(output_dir) From patchwork Sun Jul 2 14:25:40 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1802337 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=2a01:238:438b:c500:173d:9f52:ddab:ee01; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.a=rsa-sha256 header.s=google header.b=bovXL7PM; dkim-atps=neutral Received: from phobos.denx.de (phobos.denx.de [IPv6:2a01:238:438b:c500:173d:9f52:ddab:ee01]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QvBK0352Pz20Pj for ; Mon, 3 Jul 2023 00:29:40 +1000 (AEST) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id 4F89F865D7; Sun, 2 Jul 2023 16:27:41 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="bovXL7PM"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id 52F2D863BA; Sun, 2 Jul 2023 16:27:25 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.2 Received: from mail-il1-x134.google.com (mail-il1-x134.google.com [IPv6:2607:f8b0:4864:20::134]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id 9012A865C2 for ; Sun, 2 Jul 2023 16:27:21 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=sjg@chromium.org Received: by mail-il1-x134.google.com with SMTP id e9e14a558f8ab-345a4efb66dso14964815ab.3 for ; Sun, 02 Jul 2023 07:27:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1688308040; x=1690900040; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=oYKCg+s1Q9v8dHl/pJfKp96MJCwRylgOkNg1rC2pRvk=; b=bovXL7PM8ZjDjy1OaDVPvqoerb+5CP45Zcb1DlRAaLfEGU6H4hf4XSEEXsnWKLT4mZ 9s2ndLBjWzeFZH09ej9FPUH4nGKGErnIvj5Bd6RTQGENu4Mtwvq5OYleKxLUeWpl6JML m/Q2P90+99KIsRg4/PdSZ4FA2ugjwE7jKtB50= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1688308040; x=1690900040; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=oYKCg+s1Q9v8dHl/pJfKp96MJCwRylgOkNg1rC2pRvk=; b=iCtfLjVhSCCqYC/aRVlABBqJdGIsWKP1R3IUzzdNxQSTp4vhg7Eb3WAT05rpJQuSoN wE20wSd7IVAu2MZ6ZFcbb3lqXlCT3EzU7b+AxoF7ABXBxmf08dNmqSyf99Cvoygo/J6w azKS0scpLymzGMXvMUEYKfCZAVyWoyp62Y1GUr/IBCtZk+IJDaur+gXOoK8HNk454QBr t0GHgaO0ZYuqLddAeHnJLnb0YcBOSJA1hfU3u38po5V56jcX6jjGdpcsjXQm2O1QCYRz cU8fosdByxGS9o+CZFtNjRbXNqh1KmAO/VLX9YC67shNQlSCHGGQnpmoPLwvhse3VqJf SnPg== X-Gm-Message-State: ABy/qLZtXFbyWoaHOfu/F0YkbCcg92w6NWOWQ6OXP2t9FWbPSzqSzqBX mS/xyyBpnXtE0HRhP7mqyktu4cWXmnLwQZa/i8I= X-Google-Smtp-Source: APBJJlET6Z8WcA68TTLpa81MOjkU5+XN8RwILDJGZNbhxfYZT6CUjZvuO7aAukF8Wqhdtc/eYkHfkQ== X-Received: by 2002:a92:4b0e:0:b0:345:a01b:45a with SMTP id m14-20020a924b0e000000b00345a01b045amr8093912ilg.19.1688308040178; Sun, 02 Jul 2023 07:27:20 -0700 (PDT) Received: from sjg1.lan (c-73-14-173-85.hsd1.co.comcast.net. [73.14.173.85]) by smtp.gmail.com with ESMTPSA id c13-20020a02c9cd000000b0042af5268e4asm2934713jap.58.2023.07.02.07.27.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 02 Jul 2023 07:27:20 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Cc: Simon Glass Subject: [PATCH 15/58] buildman: Move output-file setup into one place Date: Sun, 2 Jul 2023 15:25:40 +0100 Message-ID: <20230702142639.1249681-16-sjg@chromium.org> X-Mailer: git-send-email 2.41.0.255.g8b1d071c50-goog In-Reply-To: <20230702142639.1249681-1-sjg@chromium.org> References: <20230702142639.1249681-1-sjg@chromium.org> MIME-Version: 1.0 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.8 at phobos.denx.de X-Virus-Status: Clean Collect the two parts of the output-file handling into single place. Signed-off-by: Simon Glass --- tools/buildman/control.py | 15 +++++++-------- 1 file changed, 7 insertions(+), 8 deletions(-) diff --git a/tools/buildman/control.py b/tools/buildman/control.py index 86b00e5e5ae4..200f4eb898ac 100644 --- a/tools/buildman/control.py +++ b/tools/buildman/control.py @@ -407,6 +407,13 @@ def do_buildman(options, args, toolchains=None, make_func=None, brds=None, if options.work_in_output: sys.exit(col.build(col.RED, '-w requires that you specify -o')) output_dir = '..' + if options.branch and not options.no_subdirs: + # As a special case allow the board directory to be placed in the + # output directory itself rather than any subdirectory. + dirname = options.branch.replace('/', '_') + output_dir = os.path.join(output_dir, dirname) + if clean_dir and os.path.exists(output_dir): + shutil.rmtree(output_dir) # Work out what subset of the boards we are building if not brds: @@ -452,14 +459,6 @@ def do_buildman(options, args, toolchains=None, make_func=None, brds=None, options.branch) # Create a new builder with the selected options. - if options.branch: - dirname = options.branch.replace('/', '_') - # As a special case allow the board directory to be placed in the - # output directory itself rather than any subdirectory. - if not options.no_subdirs: - output_dir = os.path.join(output_dir, dirname) - if clean_dir and os.path.exists(output_dir): - shutil.rmtree(output_dir) # For a dry run, just show our actions as a sanity check if options.dry_run: From patchwork Sun Jul 2 14:25:41 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1802338 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=2a01:238:438b:c500:173d:9f52:ddab:ee01; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.a=rsa-sha256 header.s=google header.b=T9BB8oBE; dkim-atps=neutral Received: from phobos.denx.de (phobos.denx.de [IPv6:2a01:238:438b:c500:173d:9f52:ddab:ee01]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QvBK95WSwz20Pj for ; Mon, 3 Jul 2023 00:29:49 +1000 (AEST) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id 45C8E86664; Sun, 2 Jul 2023 16:27:43 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="T9BB8oBE"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id A23DF865C1; Sun, 2 Jul 2023 16:27:25 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.2 Received: from mail-il1-x136.google.com (mail-il1-x136.google.com [IPv6:2607:f8b0:4864:20::136]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id 50EE0863C7 for ; Sun, 2 Jul 2023 16:27:22 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=sjg@chromium.org Received: by mail-il1-x136.google.com with SMTP id e9e14a558f8ab-345e55a62d8so11524785ab.3 for ; Sun, 02 Jul 2023 07:27:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1688308041; x=1690900041; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=oyhTb2FNJHwlVTmM6J/bd9jsGveDQdamujp+9182JD4=; b=T9BB8oBEH4qmWvqHFTMRmaW2YjNtGEM7pvQUzJQOX1++FRmy/37YLesKxVcA1MEXLo C6sJceyjgeOgxFuXbHOowEOrmRxGjdLII3Cb7Qodq5U+l0FqGwj+t7Jevb1CTpViK9Zc O3TROKbxbo1GgGB1PvWbWMVfYI+Hi/3thl7yI= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1688308041; x=1690900041; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=oyhTb2FNJHwlVTmM6J/bd9jsGveDQdamujp+9182JD4=; b=jzW5RTVsbkKA+iI2hj7BkR1OBaOnARRVMeUqdrLtYkENhf5Loioc2yBKJV+L2a4naD zjOoeR0z5ZN9AiA8vbsFnYDuOD98qzXq3XmZWMkrIapMnKcqArhtMdAtw9ftfOnMeYuJ MjmG0Fj+vz6YjuUFdA8B+x0/6TEzQTGXxbJkYDr5FcbJo3ZYbwoZef54tpBrlxhZYYTq PXyRyBp9LJtdzdHmQjnz2mJDyEl4uyAO+YO7vt2bg0VjNBEghnP/lSEzPTOtBR0aHMYC ssXK6HcyfUsFtgjdUKHrQ9g2No91vhbfDbgzCvzDSERv4v+GROoJhEPp4LctIb+YTgOZ BbTQ== X-Gm-Message-State: ABy/qLbvtlLJc0lPrR9yHkwlOHEK/sbdRKLpi71J0Tg9CSxbnlV7Ghjt CxkRlQ5PESvSwXk0uefR+vR3KHltuCg0z2zL5uk= X-Google-Smtp-Source: APBJJlFFuDoHU8D2xgdstS39FhTpJ7lOcyFxa+jW/N+609CMfWUDDBiUVDypkEFtpGK32ohV2QeYxw== X-Received: by 2002:a92:d605:0:b0:345:e443:d7ec with SMTP id w5-20020a92d605000000b00345e443d7ecmr6816974ilm.18.1688308040924; Sun, 02 Jul 2023 07:27:20 -0700 (PDT) Received: from sjg1.lan (c-73-14-173-85.hsd1.co.comcast.net. [73.14.173.85]) by smtp.gmail.com with ESMTPSA id c13-20020a02c9cd000000b0042af5268e4asm2934713jap.58.2023.07.02.07.27.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 02 Jul 2023 07:27:20 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Cc: Simon Glass Subject: [PATCH 16/58] buildman: Pass option values to get_action_summary() Date: Sun, 2 Jul 2023 15:25:41 +0100 Message-ID: <20230702142639.1249681-17-sjg@chromium.org> X-Mailer: git-send-email 2.41.0.255.g8b1d071c50-goog In-Reply-To: <20230702142639.1249681-1-sjg@chromium.org> References: <20230702142639.1249681-1-sjg@chromium.org> MIME-Version: 1.0 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.8 at phobos.denx.de X-Virus-Status: Clean Pass in the individual values rather than the whole options object, so we can see what is needed. Signed-off-by: Simon Glass --- tools/buildman/control.py | 22 +++++++++++++++------- 1 file changed, 15 insertions(+), 7 deletions(-) diff --git a/tools/buildman/control.py b/tools/buildman/control.py index 200f4eb898ac..07c9921b9e7f 100644 --- a/tools/buildman/control.py +++ b/tools/buildman/control.py @@ -29,22 +29,30 @@ def get_plural(count): """Returns a plural 's' if count is not 1""" return 's' if count != 1 else '' -def get_action_summary(is_summary, commits, selected, options): +def get_action_summary(is_summary, commits, selected, step, threads, jobs): """Return a string summarising the intended action. + Args: + is_summary (bool): True if this is a summary (otherwise it is building) + commits (list): List of commits being built + selected (list of Board): List of Board objects that are marked + step (int): Step increment through commits + threads (int): Number of processor threads being used + jobs (int): Number of jobs to build at once + Returns: Summary string. """ if commits: count = len(commits) - count = (count + options.step - 1) // options.step + count = (count + step - 1) // step commit_str = f'{count} commit{get_plural(count)}' else: commit_str = 'current source' msg = (f"{'Summary of' if is_summary else 'Building'} " f'{commit_str} for {len(selected)} boards') - msg += (f' ({options.threads} thread{get_plural(options.threads)}, ' - f'{options.jobs} job{get_plural(options.jobs)} per thread)') + msg += (f' ({threads} thread{get_plural(threads)}, ' + f'{jobs} job{get_plural(jobs)} per thread)') return msg # pylint: disable=R0913 @@ -73,7 +81,7 @@ def show_actions(series, why_selected, boards_selected, output_dir, options, else: commits = None print(get_action_summary(False, commits, boards_selected, - options)) + options.step, options.threads, options.jobs)) print(f'Build directory: {output_dir}') if commits: for upto in range(0, len(series.commits), options.step): @@ -152,7 +160,7 @@ def determine_series(selected, col, git_dir, count, branch, work_in_output): """Determine the series which is to be built, if any Args: - selected (list of Board(: List of Board objects that are marked + selected (list of Board): List of Board objects that are marked selected col (Terminal.Color): Color object to use git_dir (str): Git directory to use, e.g. './.git' @@ -514,7 +522,7 @@ def do_buildman(options, args, toolchains=None, make_func=None, brds=None, if not options.ide: tprint(get_action_summary(options.summary, commits, board_selected, - options)) + options.step, options.threads, options.jobs)) # We can't show function sizes without board details at present if options.show_bloat: From patchwork Sun Jul 2 14:25:42 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1802339 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=85.214.62.61; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.a=rsa-sha256 header.s=google header.b=hn09NLIb; dkim-atps=neutral Received: from phobos.denx.de (phobos.denx.de [85.214.62.61]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QvBKL3bB1z20Pj for ; Mon, 3 Jul 2023 00:29:58 +1000 (AEST) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id AC472866A7; Sun, 2 Jul 2023 16:27:45 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="hn09NLIb"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id 9273A865C1; Sun, 2 Jul 2023 16:27:26 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.2 Received: from mail-io1-xd2d.google.com (mail-io1-xd2d.google.com [IPv6:2607:f8b0:4864:20::d2d]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id A53BF8638A for ; Sun, 2 Jul 2023 16:27:23 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=sjg@chromium.org Received: by mail-io1-xd2d.google.com with SMTP id ca18e2360f4ac-78372b896d0so144903339f.2 for ; Sun, 02 Jul 2023 07:27:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1688308042; x=1690900042; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=1vBrRKUrqOxwpD96aDNDljL9cI1E6ElDm9Lk+9P2kik=; b=hn09NLIbnLFBu/qY8e9fUuEptKmnPH6WUqb4Nwa6KhFFg3TS1oCOgwScrpSIpB1UV4 hQhGwrDa7jNNQMcSW02py/S6qmAFuEfagTD2xNxmL3vcTnrx7AY/WbUvdwMPzqJQirVw T/2NJ8v87TpSjIHKTPS1GsYPNoXqBh5ZHHLww= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1688308042; x=1690900042; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=1vBrRKUrqOxwpD96aDNDljL9cI1E6ElDm9Lk+9P2kik=; b=ZWupM74wexWS8YLBIHvtB8pNppMFhBPvPhxIJBXZPZ99Iw5pCUUHL0AhSf+jBecmNh nUt/SpmSUt/KHjDT2rl9dBzVrr7PMZK5qrDO3q/7uMS5oktSPk2M7mlTNtc19cfv3Vkl 06uaRJtVatSzR6oLRrmEdSmBGw0WFxN5/O7kWeLPDAFJHvV07rj29u4m+Nh2q8jBOpb+ ETAkbrC6dAMOSBIJZ3NO7mchwcNMoYFUV6pphUv+cdhQ7UeFMFAPITQApBaas4uu/Jih o2cUp/Zr+dKCumYInvf2e8K4JZpgi+z8FaUwGTqzNTOUUbsnPdDOfenYFVstnqpB0qlW zBqw== X-Gm-Message-State: ABy/qLY2N8YZpvoT88h/FqNl+Ca4J+GrrzdUAa1EWa6WUkbYpPNDyqAF gqstNu2htO7xkUxIm3q82A6G2/UYoCD07y8fOnQ= X-Google-Smtp-Source: APBJJlH7jHIA+N6RmxzLQGuBPyvEln7HyDQaz6Bg6X1CECTvaBUoAvRFQvU0uqcBJlZTzgQjcvwsoA== X-Received: by 2002:a92:d349:0:b0:345:b53b:de98 with SMTP id a9-20020a92d349000000b00345b53bde98mr7713253ilh.26.1688308042212; Sun, 02 Jul 2023 07:27:22 -0700 (PDT) Received: from sjg1.lan (c-73-14-173-85.hsd1.co.comcast.net. [73.14.173.85]) by smtp.gmail.com with ESMTPSA id c13-20020a02c9cd000000b0042af5268e4asm2934713jap.58.2023.07.02.07.27.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 02 Jul 2023 07:27:22 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Cc: Simon Glass Subject: [PATCH 17/58] buildman: Pass option values to show_actions() Date: Sun, 2 Jul 2023 15:25:42 +0100 Message-ID: <20230702142639.1249681-18-sjg@chromium.org> X-Mailer: git-send-email 2.41.0.255.g8b1d071c50-goog In-Reply-To: <20230702142639.1249681-1-sjg@chromium.org> References: <20230702142639.1249681-1-sjg@chromium.org> MIME-Version: 1.0 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.8 at phobos.denx.de X-Virus-Status: Clean Pass in the individual values rather than the whole options object, so we can see what is needed. Signed-off-by: Simon Glass --- tools/buildman/control.py | 22 +++++++++++++--------- 1 file changed, 13 insertions(+), 9 deletions(-) diff --git a/tools/buildman/control.py b/tools/buildman/control.py index 07c9921b9e7f..b15ab8270e9d 100644 --- a/tools/buildman/control.py +++ b/tools/buildman/control.py @@ -56,8 +56,8 @@ def get_action_summary(is_summary, commits, selected, step, threads, jobs): return msg # pylint: disable=R0913 -def show_actions(series, why_selected, boards_selected, output_dir, options, - board_warnings): +def show_actions(series, why_selected, boards_selected, output_dir, + board_warnings, step, threads, jobs, verbose): """Display a list of actions that we would take, if not a dry run. Args: @@ -70,8 +70,11 @@ def show_actions(series, why_selected, boards_selected, output_dir, options, boards_selected: Dict of selected boards, key is target name, value is Board object output_dir (str): Output directory for builder - options: Command line options object board_warnings: List of warnings obtained from board selected + step (int): Step increment through commits + threads (int): Number of processor threads being used + jobs (int): Number of jobs to build at once + verbose (bool): True to indicate why each board was selected """ col = terminal.Color() print('Dry run, so not doing much. But I would do this:') @@ -80,11 +83,11 @@ def show_actions(series, why_selected, boards_selected, output_dir, options, commits = series.commits else: commits = None - print(get_action_summary(False, commits, boards_selected, - options.step, options.threads, options.jobs)) + print(get_action_summary(False, commits, boards_selected, step, threads, + jobs)) print(f'Build directory: {output_dir}') if commits: - for upto in range(0, len(series.commits), options.step): + for upto in range(0, len(series.commits), step): commit = series.commits[upto] print(' ', col.build(col.YELLOW, commit.hash[:8], bright=False), end=' ') print(commit.subject) @@ -92,7 +95,7 @@ def show_actions(series, why_selected, boards_selected, output_dir, options, for arg in why_selected: if arg != 'all': print(arg, f': {len(why_selected[arg])} boards') - if options.verbose: + if verbose: print(f" {' '.join(why_selected[arg])}") print('Total boards to build for each ' f"commit: {len(why_selected['all'])}\n") @@ -470,8 +473,9 @@ def do_buildman(options, args, toolchains=None, make_func=None, brds=None, # For a dry run, just show our actions as a sanity check if options.dry_run: - show_actions(series, why_selected, selected, output_dir, options, - board_warnings) + show_actions(series, why_selected, selected, output_dir, board_warnings, + options.step, options.threads, options.jobs, + options.verbose) return 0 adjust_cfg = cfgutil.convert_list_to_dict(options.adjust_cfg) From patchwork Sun Jul 2 14:25:43 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1802340 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=2a01:238:438b:c500:173d:9f52:ddab:ee01; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.a=rsa-sha256 header.s=google header.b=FxfmbGz3; dkim-atps=neutral Received: from phobos.denx.de (phobos.denx.de [IPv6:2a01:238:438b:c500:173d:9f52:ddab:ee01]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QvBKZ0sg8z242R for ; Mon, 3 Jul 2023 00:30:10 +1000 (AEST) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id 0A7C686308; Sun, 2 Jul 2023 16:27:48 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="FxfmbGz3"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id 54C59865D1; Sun, 2 Jul 2023 16:27:28 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.2 Received: from mail-io1-xd2c.google.com (mail-io1-xd2c.google.com [IPv6:2607:f8b0:4864:20::d2c]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id 83415865BC for ; Sun, 2 Jul 2023 16:27:25 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=sjg@chromium.org Received: by mail-io1-xd2c.google.com with SMTP id ca18e2360f4ac-78362f57500so191901639f.3 for ; Sun, 02 Jul 2023 07:27:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1688308044; x=1690900044; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=QRCLsKZiWP6idkVV4wEROuSJvyosn2bpCTV9K0bYtWs=; b=FxfmbGz3LkZZuPWY3rERY+iDaDx5rYGTKV+P+PGfDVwCk7pr2U4rGvnUePSK8jOe0t 0NWvCoRYOvz0ojg/g8ILc28VPuOkR9scJD7QnxSZBsAC1VQCsn3NSUpB3nSJ7etfQPeF OUAMN5KVP6+3e4kvDtjWOK0VmstL0nAHZ6TuA= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1688308044; x=1690900044; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=QRCLsKZiWP6idkVV4wEROuSJvyosn2bpCTV9K0bYtWs=; b=a9Iu7hnR2RL3/MZo2PHpTDigIU94QkbW9E8w6KZv6xPxS4LNCZpIh8jFdoXGVFQfpR XKg8GujhyVWceoDXsXImjzAQgCPlJBGy4lrrGfk3yk0odgiZST7UnWRDOdk1+UuHwoPD 66NRUoYMsXZFMyHOo4F5ZyeRqP05OUq2fPdSmxoLGTz33Kb76gvjkfMr2DOo1zuss+tS St8OG3L7Ttu80rHTrQfugNCN1rTAZinqqdj5GrY/bnJD7hlHjO82rUTQpQOfygG7FT5F Ejd1paNpWoQnE883j1ebyoRL06Rl4YS05bpFbIzJlH1rToBd5f//OOGrvr/T15CSunVJ jpXQ== X-Gm-Message-State: AC+VfDy62l+qoXnFM+mptEhGrMoRJ8IZC/B6GIVp/5+T+Xc9TvMP0Tso Xi416OQl7LgXXmY+UHhyycQKXFa1CtxMex29fgw= X-Google-Smtp-Source: ACHHUZ6ABULLeB3w1lVfi1ZlXrPHzXDdwC3ZArFg0o4SSWbCtvUmcbU5pV0oaEO1/T0GbUEkCem7Eg== X-Received: by 2002:a6b:db07:0:b0:786:25cf:421f with SMTP id t7-20020a6bdb07000000b0078625cf421fmr8937899ioc.19.1688308044128; Sun, 02 Jul 2023 07:27:24 -0700 (PDT) Received: from sjg1.lan (c-73-14-173-85.hsd1.co.comcast.net. [73.14.173.85]) by smtp.gmail.com with ESMTPSA id c13-20020a02c9cd000000b0042af5268e4asm2934713jap.58.2023.07.02.07.27.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 02 Jul 2023 07:27:23 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Cc: Simon Glass Subject: [PATCH 18/58] buildman: Build option-adjusting into a function Date: Sun, 2 Jul 2023 15:25:43 +0100 Message-ID: <20230702142639.1249681-19-sjg@chromium.org> X-Mailer: git-send-email 2.41.0.255.g8b1d071c50-goog In-Reply-To: <20230702142639.1249681-1-sjg@chromium.org> References: <20230702142639.1249681-1-sjg@chromium.org> MIME-Version: 1.0 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.8 at phobos.denx.de X-Virus-Status: Clean Create a separate function to adjust options. Also move show_actions() up as far as we can in the function. Signed-off-by: Simon Glass --- tools/buildman/control.py | 53 ++++++++++++++++++++++++--------------- 1 file changed, 33 insertions(+), 20 deletions(-) diff --git a/tools/buildman/control.py b/tools/buildman/control.py index b15ab8270e9d..7c56a0f8ed40 100644 --- a/tools/buildman/control.py +++ b/tools/buildman/control.py @@ -381,6 +381,32 @@ def determine_boards(brds, args, col, opt_boards, exclude): return selected, why_selected, board_warnings +def adjust_options(options, series, selected): + """Adjust options according to various constraints + + Updates verbose, show_errors, threads, jobs and step + + Args: + options (Options): Options object to adjust + series (Series): Series being built / summarised + selected (list of Board): List of Board objects that are marked + """ + if not series and not options.dry_run: + options.verbose = True + if not options.summary: + options.show_errors = True + + # By default we have one thread per CPU. But if there are not enough jobs + # we can have fewer threads and use a high '-j' value for make. + if options.threads is None: + options.threads = min(multiprocessing.cpu_count(), len(selected)) + if not options.jobs: + options.jobs = max(1, (multiprocessing.cpu_count() + + len(selected) - 1) // len(selected)) + + if not options.step: + options.step = len(series.commits) - 1 + def do_buildman(options, args, toolchains=None, make_func=None, brds=None, clean_dir=False, test_thread_exceptions=False): """The main control code for buildman @@ -444,21 +470,15 @@ def do_buildman(options, args, toolchains=None, make_func=None, brds=None, series = determine_series(selected, col, git_dir, options.count, options.branch, options.work_in_output) - if not series and not options.dry_run: - options.verbose = True - if not options.summary: - options.show_errors = True - # By default we have one thread per CPU. But if there are not enough jobs - # we can have fewer threads and use a high '-j' value for make. - if options.threads is None: - options.threads = min(multiprocessing.cpu_count(), len(selected)) - if not options.jobs: - options.jobs = max(1, (multiprocessing.cpu_count() + - len(selected) - 1) // len(selected)) + adjust_options(options, series, selected) - if not options.step: - options.step = len(series.commits) - 1 + # For a dry run, just show our actions as a sanity check + if options.dry_run: + show_actions(series, why_selected, selected, output_dir, board_warnings, + options.step, options.threads, options.jobs, + options.verbose) + return 0 gnu_make = command.output(os.path.join(options.git, 'scripts/show-gnu-make'), raise_on_error=False).rstrip() @@ -471,13 +491,6 @@ def do_buildman(options, args, toolchains=None, make_func=None, brds=None, # Create a new builder with the selected options. - # For a dry run, just show our actions as a sanity check - if options.dry_run: - show_actions(series, why_selected, selected, output_dir, board_warnings, - options.step, options.threads, options.jobs, - options.verbose) - return 0 - adjust_cfg = cfgutil.convert_list_to_dict(options.adjust_cfg) # Drop LOCALVERSION_AUTO since it changes the version string on every commit From patchwork Sun Jul 2 14:25:44 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1802341 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=2a01:238:438b:c500:173d:9f52:ddab:ee01; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.a=rsa-sha256 header.s=google header.b=HySJ57kV; dkim-atps=neutral Received: from phobos.denx.de (phobos.denx.de [IPv6:2a01:238:438b:c500:173d:9f52:ddab:ee01]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QvBKk6MFdz20ZF for ; Mon, 3 Jul 2023 00:30:18 +1000 (AEST) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id 6AACF86609; Sun, 2 Jul 2023 16:27:50 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="HySJ57kV"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id B52EE863DB; Sun, 2 Jul 2023 16:27:28 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.2 Received: from mail-io1-xd32.google.com (mail-io1-xd32.google.com [IPv6:2607:f8b0:4864:20::d32]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id 23CC6863C7 for ; Sun, 2 Jul 2023 16:27:26 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=sjg@chromium.org Received: by mail-io1-xd32.google.com with SMTP id ca18e2360f4ac-78625caa702so146241339f.1 for ; Sun, 02 Jul 2023 07:27:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1688308044; x=1690900044; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=nw5zVdtJl9puVq6H9xp/HgA7NEfzJK1al4l8HBtEbd8=; b=HySJ57kVtdrQPMstERE+4UivM6Spaket5HnmVs8OOvBVdzRFQyZxWJzCWHFp6c5tOB XU52IYGBG8QblzymZ0VqrE0gPTxpjd7XMlhc8rlfUWrctDyWvoAaitWqbuy//LDkNJoB BE9YB5awkOtRvpKhnei7n3k+H/zQySoPrW97g= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1688308044; x=1690900044; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=nw5zVdtJl9puVq6H9xp/HgA7NEfzJK1al4l8HBtEbd8=; b=Tt+qjDJfhdZdipXogEg69zgMpKlMGiSHFMHKjNSthVOZxHuC+7Dwr8eUtLIJE+KXS/ KH5QvArPTZukr3b4QAJOGoWPLPkCeJQARb4Q2MrsqTgeFmyQkhX8VJ8zopOPZl4E4qQp lYP6Clm79K2P+vrokanbCMPw6AOguKTHyh+cuRABw0Vk7RRQoQfi6mHUf74nM+guYCoi 8wFDrx77gz2ipArNABlQMacf5xO8aImRON5sqmP+fMP1ut4fOCS/EEOj5HT7AYJ0AwgV Jg9E+LRzr11ZO5iKwBNEsOIR5srOlwEqlSMTPwVJ/5GAEa67RRJV5zxvXH9wsnWN9ofb NrVg== X-Gm-Message-State: ABy/qLZESqVtc7ONchoGCV7+fe16RoUiiT8hUAwn2uOFmZQNt9cAW35j 5uZjX9MXS7snl3o71jOS63mnAuxokCP1HdwZFPw= X-Google-Smtp-Source: APBJJlFozVCy/yfzNTSZSncy29M+M9n1nYMvJwkkBv0z+OmLuGescWHLncesh+bJCHxUS2NBc0kuDg== X-Received: by 2002:a92:d349:0:b0:345:7e1d:ab9d with SMTP id a9-20020a92d349000000b003457e1dab9dmr7817095ilh.24.1688308044738; Sun, 02 Jul 2023 07:27:24 -0700 (PDT) Received: from sjg1.lan (c-73-14-173-85.hsd1.co.comcast.net. [73.14.173.85]) by smtp.gmail.com with ESMTPSA id c13-20020a02c9cd000000b0042af5268e4asm2934713jap.58.2023.07.02.07.27.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 02 Jul 2023 07:27:24 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Cc: Simon Glass Subject: [PATCH 19/58] buildman: Move counting of commits into a function Date: Sun, 2 Jul 2023 15:25:44 +0100 Message-ID: <20230702142639.1249681-20-sjg@chromium.org> X-Mailer: git-send-email 2.41.0.255.g8b1d071c50-goog In-Reply-To: <20230702142639.1249681-1-sjg@chromium.org> References: <20230702142639.1249681-1-sjg@chromium.org> MIME-Version: 1.0 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.8 at phobos.denx.de X-Virus-Status: Clean Move this code into a separate function to avoid a pylint warning in determine_series(). Signed-off-by: Simon Glass --- tools/buildman/control.py | 63 +++++++++++++++++++++++++-------------- 1 file changed, 41 insertions(+), 22 deletions(-) diff --git a/tools/buildman/control.py b/tools/buildman/control.py index 7c56a0f8ed40..c6abf96a4a14 100644 --- a/tools/buildman/control.py +++ b/tools/buildman/control.py @@ -159,6 +159,46 @@ def get_allow_missing(opt_allow, opt_no_allow, num_selected, has_branch): return allow_missing +def count_commits(branch, count, col, git_dir): + """Could the number of commits in the branch/ranch being built + + Args: + branch (str): Name of branch to build, or None if none + count (int): Number of commits to build, or -1 for all + col (Terminal.Color): Color object to use + git_dir (str): Git directory to use, e.g. './.git' + + Returns: + tuple: + Number of commits being built + True if the 'branch' string contains a range rather than a simple + name + """ + has_range = branch and '..' in branch + if count == -1: + if not branch: + count = 1 + else: + if has_range: + count, msg = gitutil.count_commits_in_range(git_dir, branch) + else: + count, msg = gitutil.count_commits_in_branch(git_dir, branch) + if count is None: + sys.exit(col.build(col.RED, msg)) + elif count == 0: + sys.exit(col.build(col.RED, + f"Range '{branch}' has no commits")) + if msg: + print(col.build(col.YELLOW, msg)) + count += 1 # Build upstream commit also + + if not count: + msg = (f"No commits found to process in branch '{branch}': " + "set branch's upstream or use -c flag") + sys.exit(col.build(col.RED, msg)) + return count, has_range + + def determine_series(selected, col, git_dir, count, branch, work_in_output): """Determine the series which is to be built, if any @@ -189,28 +229,7 @@ def determine_series(selected, col, git_dir, count, branch, work_in_output): # Work out how many commits to build. We want to build everything on the # branch. We also build the upstream commit as a control so we can see # problems introduced by the first commit on the branch. - has_range = branch and '..' in branch - if count == -1: - if not branch: - count = 1 - else: - if has_range: - count, msg = gitutil.count_commits_in_range(git_dir, branch) - else: - count, msg = gitutil.count_commits_in_branch(git_dir, branch) - if count is None: - sys.exit(col.build(col.RED, msg)) - elif count == 0: - sys.exit(col.build(col.RED, - f"Range '{branch}' has no commits")) - if msg: - print(col.build(col.YELLOW, msg)) - count += 1 # Build upstream commit also - - if not count: - msg = (f"No commits found to process in branch '{branch}': " - "set branch's upstream or use -c flag") - sys.exit(col.build(col.RED, msg)) + count, has_range = count_commits(branch, count, col, git_dir) if work_in_output: if len(selected) != 1: sys.exit(col.build(col.RED, From patchwork Sun Jul 2 14:25:45 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1802342 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=2a01:238:438b:c500:173d:9f52:ddab:ee01; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.a=rsa-sha256 header.s=google header.b=R3IM2cBO; dkim-atps=neutral Received: from phobos.denx.de (phobos.denx.de [IPv6:2a01:238:438b:c500:173d:9f52:ddab:ee01]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QvBKv3N6dz20ZF for ; Mon, 3 Jul 2023 00:30:27 +1000 (AEST) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id D798D866C2; Sun, 2 Jul 2023 16:27:51 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="R3IM2cBO"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id 4408C865DB; Sun, 2 Jul 2023 16:27:30 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.2 Received: from mail-il1-x133.google.com (mail-il1-x133.google.com [IPv6:2607:f8b0:4864:20::133]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id 8E4AF865C6 for ; Sun, 2 Jul 2023 16:27:27 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=sjg@chromium.org Received: by mail-il1-x133.google.com with SMTP id e9e14a558f8ab-345d3c10bdfso14994355ab.2 for ; Sun, 02 Jul 2023 07:27:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1688308046; x=1690900046; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=JDKbIA0SgWMfvkogW1JeuZOfx9nAv+cbU6iLAy3Hju4=; b=R3IM2cBO+ko/sxeECPPsYze1075VD0Cw+Mrtq0FoCYKZ9cN+odlSUQqWEPlerEwCak 8ugOcuDnDgq6AHaBjI78So88GZ+G4NTi7xy0lB+ZScySJ8sN8BEukj0y4tX5vvWQec0Y NcuU2VSOjmLRQyLZYKiKxOh4Tw8/SOlq53VPQ= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1688308046; x=1690900046; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=JDKbIA0SgWMfvkogW1JeuZOfx9nAv+cbU6iLAy3Hju4=; b=TFpTWWpDI/j1xRAmVS4Qsk0DbyE++4eIJSO3R8810CnGhURRHhhbBbjBySeroQGzun Kway8zrYBUcSOMjg/7sB7QOqWRjBEcSIdb6S8DPFm0MgLXPxyAo0eCH63IIE89/H29t1 VCybSye/iXrFsIf54DfEJKhZORWP7tSL8gdx6w3R7/PPmtiIfKkpXdQNCNsLUtOZUyng MBsiH1KxeST8jQIMZR+03WVppD/k3OIZnV+9L/mXxpQ1Zt78+8T51x6uKhIagMm5XXDs Dp7WeMnBiQXKG2yhHs2yajZiMnje4YGrmHlEeTsU9i93rd82vYzjST81Q9dssb8O5hJc 5tAQ== X-Gm-Message-State: ABy/qLZXQwlwd5+8i4jieRWoIFd/R8NzU46RKdKZPd7DfuTz52dv5oRQ GYVgb0dmAF7P/JlpTrvfOdUlUyCDPH53Cxjvb7Y= X-Google-Smtp-Source: APBJJlGwONyZUr27TVFTpJGRvYmSasa0TLcVdIedd+UGhTI0Rj573P8sq2FM6G2sZzk2wX3eXYkgZw== X-Received: by 2002:a92:d381:0:b0:340:79ff:c1b6 with SMTP id o1-20020a92d381000000b0034079ffc1b6mr7672930ilo.4.1688308046116; Sun, 02 Jul 2023 07:27:26 -0700 (PDT) Received: from sjg1.lan (c-73-14-173-85.hsd1.co.comcast.net. [73.14.173.85]) by smtp.gmail.com with ESMTPSA id c13-20020a02c9cd000000b0042af5268e4asm2934713jap.58.2023.07.02.07.27.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 02 Jul 2023 07:27:25 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Cc: Simon Glass Subject: [PATCH 20/58] buildman: Move setting up the output dir into a function Date: Sun, 2 Jul 2023 15:25:45 +0100 Message-ID: <20230702142639.1249681-21-sjg@chromium.org> X-Mailer: git-send-email 2.41.0.255.g8b1d071c50-goog In-Reply-To: <20230702142639.1249681-1-sjg@chromium.org> References: <20230702142639.1249681-1-sjg@chromium.org> MIME-Version: 1.0 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.8 at phobos.denx.de X-Virus-Status: Clean Move this code into a separate function to reduce the size of the main do_buildman() directory. Signed-off-by: Simon Glass --- tools/buildman/control.py | 45 ++++++++++++++++++++++++++++----------- 1 file changed, 33 insertions(+), 12 deletions(-) diff --git a/tools/buildman/control.py b/tools/buildman/control.py index c6abf96a4a14..5cb0c3f24465 100644 --- a/tools/buildman/control.py +++ b/tools/buildman/control.py @@ -426,6 +426,36 @@ def adjust_options(options, series, selected): if not options.step: options.step = len(series.commits) - 1 + +def setup_output_dir(output_dir, work_in_output, branch, no_subdirs, col, + clean_dir): + """Set up the output directory + + Args: + output_dir (str): Output directory provided by the user, or None if none + work_in_output (bool): True to work in the output directory + branch (str): Name of branch to build, or None if none + no_subdirs (bool): True to put the output in the top-level output dir + clean_dir: Used for tests only, indicates that the existing output_dir + should be removed before starting the build + + Returns: + str: Updated output directory pathname + """ + if not output_dir: + if work_in_output: + sys.exit(col.build(col.RED, '-w requires that you specify -o')) + output_dir = '..' + if branch and not no_subdirs: + # As a special case allow the board directory to be placed in the + # output directory itself rather than any subdirectory. + dirname = branch.replace('/', '_') + output_dir = os.path.join(output_dir, dirname) + if clean_dir and os.path.exists(output_dir): + shutil.rmtree(output_dir) + return output_dir + + def do_buildman(options, args, toolchains=None, make_func=None, brds=None, clean_dir=False, test_thread_exceptions=False): """The main control code for buildman @@ -458,18 +488,9 @@ def do_buildman(options, args, toolchains=None, make_func=None, brds=None, toolchains = get_toolchains(toolchains, col, options.override_toolchain, options.fetch_arch, options.list_tool_chains, options.verbose) - output_dir = options.output_dir - if not output_dir: - if options.work_in_output: - sys.exit(col.build(col.RED, '-w requires that you specify -o')) - output_dir = '..' - if options.branch and not options.no_subdirs: - # As a special case allow the board directory to be placed in the - # output directory itself rather than any subdirectory. - dirname = options.branch.replace('/', '_') - output_dir = os.path.join(output_dir, dirname) - if clean_dir and os.path.exists(output_dir): - shutil.rmtree(output_dir) + output_dir = setup_output_dir( + options.output_dir, options.work_in_output, options.branch, + options.no_subdirs, col, clean_dir) # Work out what subset of the boards we are building if not brds: From patchwork Sun Jul 2 14:25:46 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1802343 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=2a01:238:438b:c500:173d:9f52:ddab:ee01; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.a=rsa-sha256 header.s=google header.b=D5Cz6ush; dkim-atps=neutral Received: from phobos.denx.de (phobos.denx.de [IPv6:2a01:238:438b:c500:173d:9f52:ddab:ee01]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QvBL40DXXz20ZF for ; Mon, 3 Jul 2023 00:30:36 +1000 (AEST) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id E64A086767; Sun, 2 Jul 2023 16:27:53 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="D5Cz6ush"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id 7A06C865F6; Sun, 2 Jul 2023 16:27:31 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.2 Received: from mail-il1-x133.google.com (mail-il1-x133.google.com [IPv6:2607:f8b0:4864:20::133]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id 0F204865B4 for ; Sun, 2 Jul 2023 16:27:29 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=sjg@chromium.org Received: by mail-il1-x133.google.com with SMTP id e9e14a558f8ab-3425eb6865eso8689175ab.1 for ; Sun, 02 Jul 2023 07:27:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1688308047; x=1690900047; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=NeAN6fLaCb0FjHZaPiIoGJxABt8kFxorOo9NcV1Yrk0=; b=D5Cz6ushN+pSM2fQBlBZW6uAe9jU2QxITCr26k4qWlZYyUya3QXd/S6BtWiEKOIvQW pE1N4prQWEQUZn0RgUNNrumr7q6AysshRuSxgx/DPJKCEphnHIA8gVX/D3xBUSfGfsEy SwDCDORf2LwnOYBLrLUv2nXAoO3jY7ZX+btYU= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1688308047; x=1690900047; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=NeAN6fLaCb0FjHZaPiIoGJxABt8kFxorOo9NcV1Yrk0=; b=H3PVQvYErpRXa2Ncjsri76fMhd0RdEK3GtOyzLsucJFxrSp+Khoa8yor9zsQX/Ar4x tiacOKWajZg0adhloXznE8W+xu101lzNlDPXT0pheCPuEJtXpe9OFW+QPEysZtIEAg0n whMePY9B3zrkj+uCYjtVdx39yDPt1Ml/QSFBQWJEMkatFHxpifioz6fFBSiDdWnlHzUR Yz2o23iZZ4BNGzSisvO5QP7MBgjbvLkNF5iZRMYYjEAKifrAYj5YSDo1wFaWYJxs3pE3 S/pRBR13Z/opTj3nz3lEkqAPC6GzSwn2r4BPVS8rYwvaybpF+x9dY5tsbs9LRuQ0oEHk ef+w== X-Gm-Message-State: AC+VfDw1XkRMyx87WGi1JmLw2iZPTMJvGD6cuiOAwcjgL/laEp0z8scU Ox6AnQEV20yP2wl203huAcG3cJ9L2JmqRKz6q6U= X-Google-Smtp-Source: ACHHUZ7wzTzXhRR2vQpud1f30EH8RyRzI10xvprplha5kY/C5wcyLtuc8NRL+plGUMwhpPQHp4r6mw== X-Received: by 2002:a05:6e02:cb3:b0:345:abfe:e136 with SMTP id 19-20020a056e020cb300b00345abfee136mr8380149ilg.9.1688308047687; Sun, 02 Jul 2023 07:27:27 -0700 (PDT) Received: from sjg1.lan (c-73-14-173-85.hsd1.co.comcast.net. [73.14.173.85]) by smtp.gmail.com with ESMTPSA id c13-20020a02c9cd000000b0042af5268e4asm2934713jap.58.2023.07.02.07.27.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 02 Jul 2023 07:27:27 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Cc: Simon Glass Subject: [PATCH 21/58] buildman: Move commit numbering into determine_series() Date: Sun, 2 Jul 2023 15:25:46 +0100 Message-ID: <20230702142639.1249681-22-sjg@chromium.org> X-Mailer: git-send-email 2.41.0.255.g8b1d071c50-goog In-Reply-To: <20230702142639.1249681-1-sjg@chromium.org> References: <20230702142639.1249681-1-sjg@chromium.org> MIME-Version: 1.0 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.8 at phobos.denx.de X-Virus-Status: Clean Commits are numbered for use in tests. Do this in determine_series() since it is already dealing with the series. Signed-off-by: Simon Glass --- tools/buildman/control.py | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/tools/buildman/control.py b/tools/buildman/control.py index 5cb0c3f24465..f2e1cfa88e94 100644 --- a/tools/buildman/control.py +++ b/tools/buildman/control.py @@ -202,6 +202,9 @@ def count_commits(branch, count, col, git_dir): def determine_series(selected, col, git_dir, count, branch, work_in_output): """Determine the series which is to be built, if any + If there is a series, the commits in that series are numbered by setting + their sequence value (starting from 0). This is used by tests. + Args: selected (list of Board): List of Board objects that are marked selected @@ -254,6 +257,10 @@ def determine_series(selected, col, git_dir, count, branch, work_in_output): # Honour the count series = patchstream.get_metadata_for_list(branch, git_dir, count, series=None, allow_overwrite=True) + + # Number the commits for test purposes + for i, commit in enumerate(series.commits): + commit.sequence = i else: series = None return series @@ -571,9 +578,6 @@ def do_buildman(options, args, toolchains=None, make_func=None, brds=None, if series: commits = series.commits - # Number the commits for test purposes - for i, commit in enumerate(commits): - commit.sequence = i else: commits = None From patchwork Sun Jul 2 14:25:47 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1802344 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=85.214.62.61; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.a=rsa-sha256 header.s=google header.b=ZOI4sBF+; dkim-atps=neutral Received: from phobos.denx.de (phobos.denx.de [85.214.62.61]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QvBLD3j1dz20ZF for ; Mon, 3 Jul 2023 00:30:44 +1000 (AEST) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id E387186762; Sun, 2 Jul 2023 16:27:56 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="ZOI4sBF+"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id 77ACF865D4; Sun, 2 Jul 2023 16:27:32 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.2 Received: from mail-il1-x12c.google.com (mail-il1-x12c.google.com [IPv6:2607:f8b0:4864:20::12c]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id D559B865D7 for ; Sun, 2 Jul 2023 16:27:29 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=sjg@chromium.org Received: by mail-il1-x12c.google.com with SMTP id e9e14a558f8ab-345a8a78bcfso15054305ab.2 for ; Sun, 02 Jul 2023 07:27:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1688308048; x=1690900048; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=qyF7fQadN8ti/wNBQ0MVvHFp/vnXhAhaznlHoTWUDeQ=; b=ZOI4sBF+Cj624dJU3b+WtAEaEQSibQyrdkg9Af6mPYJDeehfw8aLz7o4qpxmAPfkzl mE8pfGB2/G/9MgWn1Oo8TrKde83mJSW0F/XGgffztG+H0du+VsK6+siz79qGNKMS120o 5bi2DdGqiaFjuQiK3xkMw8ZahIhZ7QRjtLM0U= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1688308048; x=1690900048; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=qyF7fQadN8ti/wNBQ0MVvHFp/vnXhAhaznlHoTWUDeQ=; b=Qjpt/fe6OHlNUIpyQnxuvqkBlR7kTL/yCHgMvj8M86TuMR0J44Q0ttAcluoMeAzdAY M981+/UWRlHXsQ8JPwUADWNu3+NvcNc4IFXj/i6dXKnF/rkoiyTxpgbz9ZKyq6YT+R0J izpDP/rVhm9ifpf3fFL8Pi++6vH0r9pkPBDKXGuyLqM/FFHg28u3fSTxn1/TqlE5A7ra gO2lDkzvMwrsXaltkVwAxfMTrLYjp2Jpa2Br/k2BHA1I4TOlmDWkqcAjtzRJmwrmStoX coAgh051TBYnvh+LS2ka0R0E4NPd0cVFgzCMVjCw9JfapCdb1vXBdcgJtPujVXQCY8Z0 EyeA== X-Gm-Message-State: ABy/qLZWr1zb31Km3yrumoTqHEAMJEzLYS8zLd7nfdT/JOpBTTuGUwL+ pmLgITCKEli8Hv04vfHcUnRpvFfOZx+7cgdyv90= X-Google-Smtp-Source: APBJJlF5MtPq5YevmgnKzzNw9ThshuiJSVv56RsgcLZ7bjkp96Mrb3S1cOYmNA+7gChNb0YB5yu+5A== X-Received: by 2002:a92:daca:0:b0:345:ac89:6c42 with SMTP id o10-20020a92daca000000b00345ac896c42mr7470631ilq.2.1688308048458; Sun, 02 Jul 2023 07:27:28 -0700 (PDT) Received: from sjg1.lan (c-73-14-173-85.hsd1.co.comcast.net. [73.14.173.85]) by smtp.gmail.com with ESMTPSA id c13-20020a02c9cd000000b0042af5268e4asm2934713jap.58.2023.07.02.07.27.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 02 Jul 2023 07:27:28 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Cc: Simon Glass Subject: [PATCH 22/58] buildman: Avoid too many returns in do_buildman() Date: Sun, 2 Jul 2023 15:25:47 +0100 Message-ID: <20230702142639.1249681-23-sjg@chromium.org> X-Mailer: git-send-email 2.41.0.255.g8b1d071c50-goog In-Reply-To: <20230702142639.1249681-1-sjg@chromium.org> References: <20230702142639.1249681-1-sjg@chromium.org> MIME-Version: 1.0 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.8 at phobos.denx.de X-Virus-Status: Clean Fix the pylint warning by using a variable instead of lots of 'return' statements. Signed-off-by: Simon Glass --- tools/buildman/control.py | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/tools/buildman/control.py b/tools/buildman/control.py index f2e1cfa88e94..3b07a94ecbac 100644 --- a/tools/buildman/control.py +++ b/tools/buildman/control.py @@ -593,15 +593,16 @@ def do_buildman(options, args, toolchains=None, make_func=None, brds=None, options.show_bloat, options.list_error_boards, options.show_config, options.show_environment, options.filter_dtb_warnings, options.filter_migration_warnings, options.ide) + retval = 0 if options.summary: builder.ShowSummary(commits, board_selected) else: fail, warned, excs = builder.BuildBoards( commits, board_selected, options.keep_outputs, options.verbose) if excs: - return 102 + retval = 102 if fail: - return 100 + retval = 100 if warned and not options.ignore_warnings: - return 101 - return 0 + retval = 101 + return retval From patchwork Sun Jul 2 14:25:48 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1802345 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=85.214.62.61; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.a=rsa-sha256 header.s=google header.b=Qk/L552w; dkim-atps=neutral Received: from phobos.denx.de (phobos.denx.de [85.214.62.61]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QvBLQ3rCRz20ZF for ; Mon, 3 Jul 2023 00:30:54 +1000 (AEST) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id E7C8486785; Sun, 2 Jul 2023 16:27:58 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="Qk/L552w"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id 33C69865DD; Sun, 2 Jul 2023 16:27:34 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.2 Received: from mail-il1-x12b.google.com (mail-il1-x12b.google.com [IPv6:2607:f8b0:4864:20::12b]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id 90450865DB for ; Sun, 2 Jul 2023 16:27:30 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=sjg@chromium.org Received: by mail-il1-x12b.google.com with SMTP id e9e14a558f8ab-341ff6251f2so15130665ab.0 for ; Sun, 02 Jul 2023 07:27:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1688308049; x=1690900049; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=nlccf+cELnaflHtxEHqr40NLNXcidUusShHE72GYr7Q=; b=Qk/L552w1TIFd/SF0CkUO2qniyOc4/fIFOTy/nOaF2gNW9r4OQUhbJta7YFcFwri6j h5d9Rs2Raig8bp2jEcxBEUdmkVZvXPHpw/JUY6ambkMRu8HG+zFrg8x3RBJ465lB53Vk zvppU37pWbUGDSmIAmTWqD5xgER30IwEV0lLw= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1688308049; x=1690900049; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=nlccf+cELnaflHtxEHqr40NLNXcidUusShHE72GYr7Q=; b=b56g7jR7M0ITPiU9L87s3MVaNNdUlKpKad+24arAk+DmgSYZwuCKhggtS3iooRfHPl rwLZ/qM+YyuMurzzH08hVd63HAXpSDaur9LU5rpCAjDUkM0VjkRPP/MXTTUNcw6YhIdI cPBYmYh7c+qlxNCYRppQA6u38OH8j9wenabPkmti9ZAur9sOmvi1ClOhE48Phe+75Lt7 nOpmVq05eved8Bw/FcjzvsGLlm04cZxpfMFiH7k1t0P5R/yxbqJstve2RZE3LulEUQrt 3Z/AwWU9e9Vg+JWvzq8Ey565EWBIQavd85HQFf192hblNHON0ecgbGTlsCRJaBKF6PYP +zEA== X-Gm-Message-State: ABy/qLYJKzpLiUhnUZPzUgJt+N7P0OlRu4Pmn/XJT0VoyQix31ZD2ATh C4EK7/5I5fJMCjvLcFPBl5jCQNslvbhhy+Y4uJE= X-Google-Smtp-Source: APBJJlFXUY/hq0gjXl5NjVA4ChsDHeZTqyaNSVWTU3itwKhuilmI7yPUa9kotCIobxKwdAOHaAripQ== X-Received: by 2002:a92:d24a:0:b0:345:ffad:49e1 with SMTP id v10-20020a92d24a000000b00345ffad49e1mr3158387ilg.1.1688308049158; Sun, 02 Jul 2023 07:27:29 -0700 (PDT) Received: from sjg1.lan (c-73-14-173-85.hsd1.co.comcast.net. [73.14.173.85]) by smtp.gmail.com with ESMTPSA id c13-20020a02c9cd000000b0042af5268e4asm2934713jap.58.2023.07.02.07.27.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 02 Jul 2023 07:27:28 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Cc: Simon Glass Subject: [PATCH 23/58] buildman: Move remaining builder properties to constructor Date: Sun, 2 Jul 2023 15:25:48 +0100 Message-ID: <20230702142639.1249681-24-sjg@chromium.org> X-Mailer: git-send-email 2.41.0.255.g8b1d071c50-goog In-Reply-To: <20230702142639.1249681-1-sjg@chromium.org> References: <20230702142639.1249681-1-sjg@chromium.org> MIME-Version: 1.0 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.8 at phobos.denx.de X-Virus-Status: Clean Do these all in the constructor, so it is consistent. Move the stray builder comment into the correct place. Signed-off-by: Simon Glass --- tools/buildman/builder.py | 25 ++++++++++++++++++------- tools/buildman/control.py | 18 +++++++----------- 2 files changed, 25 insertions(+), 18 deletions(-) diff --git a/tools/buildman/builder.py b/tools/buildman/builder.py index d81752e99438..cb3628a8a085 100644 --- a/tools/buildman/builder.py +++ b/tools/buildman/builder.py @@ -255,7 +255,10 @@ class Builder: config_only=False, squash_config_y=False, warnings_as_errors=False, work_in_output=False, test_thread_exceptions=False, adjust_cfg=None, - allow_missing=False, no_lto=False, reproducible_builds=False): + allow_missing=False, no_lto=False, reproducible_builds=False, + force_build=False, force_build_failures=False, + force_reconfig=False, in_tree=False, + force_config_on_failure=False, make_func=None): """Create a new Builder object Args: @@ -295,7 +298,14 @@ class Builder: a string Kconfig allow_missing: Run build with BINMAN_ALLOW_MISSING=1 no_lto (bool): True to set the NO_LTO flag when building - + force_build (bool): Rebuild even commits that are already built + force_build_failures (bool): Rebuild commits that have not been + built, or failed to build + force_reconfig (bool): Reconfigure on each commit + in_tree (bool): Bulid in tree instead of out-of-tree + force_config_on_failure (bool): Reconfigure the build before + retrying a failed build + make_func (function): Function to call to run 'make' """ self.toolchains = toolchains self.base_dir = base_dir @@ -304,7 +314,7 @@ class Builder: else: self._working_dir = os.path.join(base_dir, '.bm-work') self.threads = [] - self.do_make = self.Make + self.do_make = make_func or self.Make self.gnu_make = gnu_make self.checkout = checkout self.num_threads = num_threads @@ -318,11 +328,7 @@ class Builder: self._complete_delay = None self._next_delay_update = datetime.now() self._start_time = datetime.now() - self.force_config_on_failure = True - self.force_build_failures = False - self.force_reconfig = False self._step = step - self.in_tree = False self._error_lines = 0 self.no_subdirs = no_subdirs self.full_path = full_path @@ -336,6 +342,11 @@ class Builder: self._ide = False self.no_lto = no_lto self.reproducible_builds = reproducible_builds + self.force_build = force_build + self.force_build_failures = force_build_failures + self.force_reconfig = force_reconfig + self.in_tree = in_tree + self.force_config_on_failure = force_config_on_failure if not self.squash_config_y: self.config_filenames += EXTRA_CONFIG_FILENAMES diff --git a/tools/buildman/control.py b/tools/buildman/control.py index 3b07a94ecbac..c0274f5463dc 100644 --- a/tools/buildman/control.py +++ b/tools/buildman/control.py @@ -536,8 +536,6 @@ def do_buildman(options, args, toolchains=None, make_func=None, brds=None, options.no_allow_missing, len(selected), options.branch) - # Create a new builder with the selected options. - adjust_cfg = cfgutil.convert_list_to_dict(options.adjust_cfg) # Drop LOCALVERSION_AUTO since it changes the version string on every commit @@ -548,6 +546,7 @@ def do_buildman(options, args, toolchains=None, make_func=None, brds=None, else: adjust_cfg['LOCALVERSION_AUTO'] = '~' + # Create a new builder with the selected options builder = Builder(toolchains, output_dir, git_dir, options.threads, options.jobs, gnu_make=gnu_make, checkout=True, show_unknown=options.show_unknown, step=options.step, @@ -562,16 +561,13 @@ def do_buildman(options, args, toolchains=None, make_func=None, brds=None, test_thread_exceptions=test_thread_exceptions, adjust_cfg=adjust_cfg, allow_missing=allow_missing, no_lto=options.no_lto, - reproducible_builds=options.reproducible_builds) + reproducible_builds=options.reproducible_builds, + force_build = options.force_build, + force_build_failures = options.force_build_failures, + force_reconfig = options.force_reconfig, in_tree = options.in_tree, + force_config_on_failure=not options.quick, make_func=make_func) + TEST_BUILDER = builder - builder.force_config_on_failure = not options.quick - if make_func: - builder.do_make = make_func - - builder.force_build = options.force_build - builder.force_build_failures = options.force_build_failures - builder.force_reconfig = options.force_reconfig - builder.in_tree = options.in_tree # Work out which boards to build board_selected = brds.get_selected_dict() From patchwork Sun Jul 2 14:25:49 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1802346 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=2a01:238:438b:c500:173d:9f52:ddab:ee01; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.a=rsa-sha256 header.s=google header.b=Y9VhhB0A; dkim-atps=neutral Received: from phobos.denx.de (phobos.denx.de [IPv6:2a01:238:438b:c500:173d:9f52:ddab:ee01]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QvBLc42V1z20ZF for ; Mon, 3 Jul 2023 00:31:04 +1000 (AEST) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id A50D086772; Sun, 2 Jul 2023 16:28:00 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="Y9VhhB0A"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id 615D0865DE; Sun, 2 Jul 2023 16:27:34 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.2 Received: from mail-il1-x12d.google.com (mail-il1-x12d.google.com [IPv6:2607:f8b0:4864:20::12d]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id 30153865F2 for ; Sun, 2 Jul 2023 16:27:31 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=sjg@chromium.org Received: by mail-il1-x12d.google.com with SMTP id e9e14a558f8ab-3457157a164so15111005ab.1 for ; Sun, 02 Jul 2023 07:27:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1688308050; x=1690900050; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=4vEF+pnNDAfMa/8rCFJ95YvWfmhUialp1lIv0Gbu4Wk=; b=Y9VhhB0A3lRzuGzpiL9JEhK5xXrbrcOHDFmfpe6mUjneoxYWysIs2PtAHwIxUBNlN4 ikw8zvtqVXYORhYcsgYgKAa/+CrKPec2y73zg0JBUBPYMDGg04INcLCLEk/2cr4IDajp +yQ+K2Uot3w5pGZn5vdwDA3t5W70BiE4rwpBk= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1688308050; x=1690900050; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=4vEF+pnNDAfMa/8rCFJ95YvWfmhUialp1lIv0Gbu4Wk=; b=H1jBoH8cILl2vltKh9kWpeFCZJg+EFVg94uXXD91k9wDMNTeff+vS4sWiBjpiI1u8l V6D0QKKA98mmnrgwWAOOlZQEy689Ep3of6J755rKabDre5FiVURWwCH1/ewqjKnpCq/l sv9iThrcrRlIOHIgoq9A42V9wgVKP0W/q6FddN7AH0QRKLGXeUR0Ak5LLb1cdaGVdYj1 6GO9bkxOdEKo264njZZCl612adf9C644aQ+yw9yU4Llc7rtGlSWCsZEJD14tfsbKeiTc YXu4SXeFdZaIDNrnt5xzwYwZO/uHsA4rBBZ9rYEIRHE8oJC0bY3vV1UNe705MYdY9wm/ lA4w== X-Gm-Message-State: ABy/qLa+sLFrZx0ZMLGJ4utl7qP7Wb+1nTfA+HGIGHjUGgq7ay/rfckh KVlEfwCrLcBQyT+1FNPYyN+8VbX41WnRsuSz4nc= X-Google-Smtp-Source: APBJJlFzZHUAKGBE5iW/i5788qogVcpHZqDJ0WhPNKJIDieGurYG2ks07Jnn2Hi4LW/Ky+3xQZqPyw== X-Received: by 2002:a92:d2c8:0:b0:345:70b9:5171 with SMTP id w8-20020a92d2c8000000b0034570b95171mr7557947ilg.20.1688308049837; Sun, 02 Jul 2023 07:27:29 -0700 (PDT) Received: from sjg1.lan (c-73-14-173-85.hsd1.co.comcast.net. [73.14.173.85]) by smtp.gmail.com with ESMTPSA id c13-20020a02c9cd000000b0042af5268e4asm2934713jap.58.2023.07.02.07.27.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 02 Jul 2023 07:27:29 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Cc: Simon Glass Subject: [PATCH 24/58] buildman: Tweak commits and show_bloat Date: Sun, 2 Jul 2023 15:25:49 +0100 Message-ID: <20230702142639.1249681-25-sjg@chromium.org> X-Mailer: git-send-email 2.41.0.255.g8b1d071c50-goog In-Reply-To: <20230702142639.1249681-1-sjg@chromium.org> References: <20230702142639.1249681-1-sjg@chromium.org> MIME-Version: 1.0 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.8 at phobos.denx.de X-Virus-Status: Clean Move setting of show_bloat to adjust_options() and adjust how the commits variable is set. Together these remove the pylint warning about too many statements. Signed-off-by: Simon Glass --- tools/buildman/control.py | 13 +++++-------- 1 file changed, 5 insertions(+), 8 deletions(-) diff --git a/tools/buildman/control.py b/tools/buildman/control.py index c0274f5463dc..a0b5033bc878 100644 --- a/tools/buildman/control.py +++ b/tools/buildman/control.py @@ -433,6 +433,10 @@ def adjust_options(options, series, selected): if not options.step: options.step = len(series.commits) - 1 + # We can't show function sizes without board details at present + if options.show_bloat: + options.show_detail = True + def setup_output_dir(output_dir, work_in_output, branch, no_subdirs, col, clean_dir): @@ -572,18 +576,11 @@ def do_buildman(options, args, toolchains=None, make_func=None, brds=None, # Work out which boards to build board_selected = brds.get_selected_dict() - if series: - commits = series.commits - else: - commits = None - + commits = series.commits if series else None if not options.ide: tprint(get_action_summary(options.summary, commits, board_selected, options.step, options.threads, options.jobs)) - # We can't show function sizes without board details at present - if options.show_bloat: - options.show_detail = True builder.SetDisplayOptions( options.show_errors, options.show_sizes, options.show_detail, options.show_bloat, options.list_error_boards, options.show_config, From patchwork Sun Jul 2 14:25:50 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1802347 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=2a01:238:438b:c500:173d:9f52:ddab:ee01; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.a=rsa-sha256 header.s=google header.b=cNjleuiv; dkim-atps=neutral Received: from phobos.denx.de (phobos.denx.de [IPv6:2a01:238:438b:c500:173d:9f52:ddab:ee01]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QvBLp0W39z20ZF for ; Mon, 3 Jul 2023 00:31:14 +1000 (AEST) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id 5A4A0863B2; Sun, 2 Jul 2023 16:28:05 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="cNjleuiv"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id D80D4865E6; Sun, 2 Jul 2023 16:27:35 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.2 Received: from mail-io1-xd29.google.com (mail-io1-xd29.google.com [IPv6:2607:f8b0:4864:20::d29]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id 6B358865D5 for ; Sun, 2 Jul 2023 16:27:32 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=sjg@chromium.org Received: by mail-io1-xd29.google.com with SMTP id ca18e2360f4ac-785cbc5bfd2so186003039f.2 for ; Sun, 02 Jul 2023 07:27:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1688308051; x=1690900051; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=AijNCtb7IL1GYHA+IknmARruPM8h0wEsUmS+TTL6yqc=; b=cNjleuivo5P7jqzG5RF5y7aeUIHiqiktzaAgDRBRZIlyOwPklHtE9Dof6gzgpxsy9/ 4seslaDN9PXmRMWse/Og1lxa30mSyLCvcpVYY4jUmrdS3LJfpVHWR+cWpBzxDvA8nmPM y69z6pFhxiiFCwXQYJQDnv6D/tVHR0y/N8l0s= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1688308051; x=1690900051; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=AijNCtb7IL1GYHA+IknmARruPM8h0wEsUmS+TTL6yqc=; b=iIygHAwH2bHuDzOKF/qBy+WrnFqBvrqROIqIN603THQAm1vxHPlU9YVa0p1zXvks9J Vt6NBjux5NduuMJu6dMiNCj65LuyBAkY4sLSfWY9G9hbjY78r9uCLMC8vZvrgBog8cuU cbuQy/mZbZAyKr2tbWtfX/avRkGRHVI5hoCzGWmHbMFpv/rMYnHbwPpvjZXPSqfMeJ45 pa3RUD84YsCXCra5rFDk1cM9I7s8U4w0J0FVcECvQ21XduomI/R1XB4EqjEh/Y5VIcqL xmM9YJzXCQlBEtNVuIWE9PqDZJjE8c/DA1e5feJ8FU9fudlklv9U/di2+FOJfT0fjXaf 681g== X-Gm-Message-State: AC+VfDy1CAp8DRGOWZYrbOwbKkRkCcGRxiD8n9MZgQjEDjFBIHlrXINJ 9VO1s5a9uvA76e7OnLcsCET+q9fYXnD3RGQ+BB0= X-Google-Smtp-Source: ACHHUZ4fwVO9ljXibTzSV+/xDk+k32O+UxIItPugnpxKk5pondRO8+ywN/BfUJGjbuSLABSrwXr/Rg== X-Received: by 2002:a6b:dc11:0:b0:77e:3e85:34ee with SMTP id s17-20020a6bdc11000000b0077e3e8534eemr8067174ioc.13.1688308051052; Sun, 02 Jul 2023 07:27:31 -0700 (PDT) Received: from sjg1.lan (c-73-14-173-85.hsd1.co.comcast.net. [73.14.173.85]) by smtp.gmail.com with ESMTPSA id c13-20020a02c9cd000000b0042af5268e4asm2934713jap.58.2023.07.02.07.27.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 02 Jul 2023 07:27:30 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Cc: Simon Glass Subject: [PATCH 25/58] buildman: Moving running of the builder into a function Date: Sun, 2 Jul 2023 15:25:50 +0100 Message-ID: <20230702142639.1249681-26-sjg@chromium.org> X-Mailer: git-send-email 2.41.0.255.g8b1d071c50-goog In-Reply-To: <20230702142639.1249681-1-sjg@chromium.org> References: <20230702142639.1249681-1-sjg@chromium.org> MIME-Version: 1.0 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.8 at phobos.denx.de X-Virus-Status: Clean Move this code into a new function. This removes the pylint warning about too many branches. Signed-off-by: Simon Glass --- tools/buildman/control.py | 56 ++++++++++++++++++++++++--------------- 1 file changed, 35 insertions(+), 21 deletions(-) diff --git a/tools/buildman/control.py b/tools/buildman/control.py index a0b5033bc878..3c5b7128bf20 100644 --- a/tools/buildman/control.py +++ b/tools/buildman/control.py @@ -466,6 +466,40 @@ def setup_output_dir(output_dir, work_in_output, branch, no_subdirs, col, shutil.rmtree(output_dir) return output_dir +def run_builder(builder, commits, board_selected, options): + """Run the builder or show the summary + + Args: + commits (list of Commit): List of commits being built, None if no branch + boards_selected (dict): Dict of selected boards: + key: target name + value: Board object + options (Options): Options to use + + Returns: + int: Return code for buildman + """ + if not options.ide: + tprint(get_action_summary(options.summary, commits, board_selected, + options.step, options.threads, options.jobs)) + + builder.SetDisplayOptions( + options.show_errors, options.show_sizes, options.show_detail, + options.show_bloat, options.list_error_boards, options.show_config, + options.show_environment, options.filter_dtb_warnings, + options.filter_migration_warnings, options.ide) + if options.summary: + builder.ShowSummary(commits, board_selected) + else: + fail, warned, excs = builder.BuildBoards( + commits, board_selected, options.keep_outputs, options.verbose) + if excs: + return 102 + if fail: + return 100 + if warned and not options.ignore_warnings: + return 101 + return 0 def do_buildman(options, args, toolchains=None, make_func=None, brds=None, clean_dir=False, test_thread_exceptions=False): @@ -577,25 +611,5 @@ def do_buildman(options, args, toolchains=None, make_func=None, brds=None, board_selected = brds.get_selected_dict() commits = series.commits if series else None - if not options.ide: - tprint(get_action_summary(options.summary, commits, board_selected, - options.step, options.threads, options.jobs)) - - builder.SetDisplayOptions( - options.show_errors, options.show_sizes, options.show_detail, - options.show_bloat, options.list_error_boards, options.show_config, - options.show_environment, options.filter_dtb_warnings, - options.filter_migration_warnings, options.ide) - retval = 0 - if options.summary: - builder.ShowSummary(commits, board_selected) - else: - fail, warned, excs = builder.BuildBoards( - commits, board_selected, options.keep_outputs, options.verbose) - if excs: - retval = 102 - if fail: - retval = 100 - if warned and not options.ignore_warnings: - retval = 101 + retval = run_builder(builder, commits, board_selected, options) return retval From patchwork Sun Jul 2 14:25:51 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1802349 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=85.214.62.61; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.a=rsa-sha256 header.s=google header.b=mdHoeCRT; dkim-atps=neutral Received: from phobos.denx.de (phobos.denx.de [85.214.62.61]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QvBM941CCz20ZF for ; Mon, 3 Jul 2023 00:31:33 +1000 (AEST) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id A66DF86760; Sun, 2 Jul 2023 16:28:10 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="mdHoeCRT"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id 75C18865EB; Sun, 2 Jul 2023 16:27:37 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.2 Received: from mail-il1-x12c.google.com (mail-il1-x12c.google.com [IPv6:2607:f8b0:4864:20::12c]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id E093C865D8 for ; Sun, 2 Jul 2023 16:27:33 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=sjg@chromium.org Received: by mail-il1-x12c.google.com with SMTP id e9e14a558f8ab-345fe8d09d3so2077295ab.0 for ; Sun, 02 Jul 2023 07:27:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1688308052; x=1690900052; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=4+Szv1Af0AFs1ZF7rASw83DHphF07FEuLuWEZ402+Hw=; b=mdHoeCRTmV093pYRjI40Ss7Zj6H6/cahq9X3Dt1mH6J15SHHf0q/WCI8bYloPBEhhz gjg+/14f/7P4mb0F0ium42T2KefYk/ijg48jGBuLQYQZKaMvmiHZHCILWUCAbqBNz48/ /imJT0rKpqnFIU3RyfjoSNn4z8igTA3aUHpfg= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1688308052; x=1690900052; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=4+Szv1Af0AFs1ZF7rASw83DHphF07FEuLuWEZ402+Hw=; b=SmYsT2K8Ota5UqQ4RNVFJBnMZPGaoespOkoX0qEOyd/4DoxcnM1ZPZL56G2fucQYvX /X/Y1VNeNuaeIq9OAVQE6qcenkCCw1zOuffA4Bk4g6vVi3WyEW6bHh2qGWGC7u9MHEeg w7IN8qM4Kd0UfLpL07fSLmoY5MGU3Ks8H7S855zallqo9QQWOxjD+G8cdRe1rSAL2muq 8S7rY6rOs+zWip7XkDJi5w5RkUVEBWdf2w/GsK3Es7fdFZUJ++D6j/jdEQQ/hvzvOXXN Oiha92uZiHCT/7dHu0LxZCOoMcrOcf5V4yybbpwEL+0U0vra9z7r80hi7R1Bmqwzdras GXAQ== X-Gm-Message-State: ABy/qLbQMc4ROiZzPRb4HdHOTZyZcFT7V7Z/kLjjdAfndVamVVjEk8GK CMUAIphAOMAMD6JwW3EAvT74910HHzuIsIFGFdI= X-Google-Smtp-Source: APBJJlEof+w8r8jEUCi8BFFnz8BWE4jViwgJ/1e05UhVBGx06M4eCi8fyFXQweknOqSeHvgRzsl51Q== X-Received: by 2002:a05:6e02:974:b0:345:fe2d:56f with SMTP id q20-20020a056e02097400b00345fe2d056fmr2798497ilt.14.1688308052523; Sun, 02 Jul 2023 07:27:32 -0700 (PDT) Received: from sjg1.lan (c-73-14-173-85.hsd1.co.comcast.net. [73.14.173.85]) by smtp.gmail.com with ESMTPSA id c13-20020a02c9cd000000b0042af5268e4asm2934713jap.58.2023.07.02.07.27.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 02 Jul 2023 07:27:32 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Cc: Simon Glass Subject: [PATCH 26/58] buildman: Drop some unnecessary variables Date: Sun, 2 Jul 2023 15:25:51 +0100 Message-ID: <20230702142639.1249681-27-sjg@chromium.org> X-Mailer: git-send-email 2.41.0.255.g8b1d071c50-goog In-Reply-To: <20230702142639.1249681-1-sjg@chromium.org> References: <20230702142639.1249681-1-sjg@chromium.org> MIME-Version: 1.0 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.8 at phobos.denx.de X-Virus-Status: Clean Drop some variables at the end of the do_bulidman() function. Signed-off-by: Simon Glass --- tools/buildman/control.py | 8 ++------ 1 file changed, 2 insertions(+), 6 deletions(-) diff --git a/tools/buildman/control.py b/tools/buildman/control.py index 3c5b7128bf20..db8377e9ac42 100644 --- a/tools/buildman/control.py +++ b/tools/buildman/control.py @@ -607,9 +607,5 @@ def do_buildman(options, args, toolchains=None, make_func=None, brds=None, TEST_BUILDER = builder - # Work out which boards to build - board_selected = brds.get_selected_dict() - - commits = series.commits if series else None - retval = run_builder(builder, commits, board_selected, options) - return retval + return run_builder(builder, series.commits if series else None, + brds.get_selected_dict(), options) From patchwork Sun Jul 2 14:25:52 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1802348 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=2a01:238:438b:c500:173d:9f52:ddab:ee01; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.a=rsa-sha256 header.s=google header.b=Rq+nZB97; dkim-atps=neutral Received: from phobos.denx.de (phobos.denx.de [IPv6:2a01:238:438b:c500:173d:9f52:ddab:ee01]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QvBLz60p8z20ZF for ; Mon, 3 Jul 2023 00:31:23 +1000 (AEST) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id E69CC866CA; Sun, 2 Jul 2023 16:28:08 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="Rq+nZB97"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id 3200C86609; Sun, 2 Jul 2023 16:27:37 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.2 Received: from mail-il1-x136.google.com (mail-il1-x136.google.com [IPv6:2607:f8b0:4864:20::136]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id 96469865DD for ; Sun, 2 Jul 2023 16:27:34 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=sjg@chromium.org Received: by mail-il1-x136.google.com with SMTP id e9e14a558f8ab-3460b67fdd8so382835ab.0 for ; Sun, 02 Jul 2023 07:27:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1688308053; x=1690900053; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=nw/4RtbFzNXzTlUZC45AlZldOgmlJjZGNbVcbIwrU58=; b=Rq+nZB97fzKCUdP0JPlhDJmYijN5lTsCcvFR1o4gd0rHq4NcmbysgStv+YPWbrXGji 1/IWnb7S0gQUQoEjfAANIbfIAi9yCkK9jrwE6MC/CCk/tURJiWfeQ2LUDNJPK4OdOdSY 3WFLyXmeH5QuxGM5u1oouNki2JZvLmWADvE0w= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1688308053; x=1690900053; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=nw/4RtbFzNXzTlUZC45AlZldOgmlJjZGNbVcbIwrU58=; b=AKfCxzJXYbMVC48VIQnxAeG9RUoMLGQfdEkWm3DADxG9cwaN86A3Oe+1duWbsp4EIL ZcIYARqVspF/qyWjFeoor3QLhokge2Iyh5s3xQvZrUBtFKgwvCTJHnkY7x1mgyehyCgT r630dk77U2RxsjCscIDP8a6UBKkTsZhcyCSEfOp0X615HVk8aQaSEIMVCFLtmu1SNOlE I0dU4oEG7MRqJZjGAueyajs5TkOdBMKPNisIFBRpnfmBEBrewwQhzahyEybUxniPwctS j2OiMesaYesqlJwwNHmpVWE2bAAHZfJ6Y1JvHZUOre7g8qz5ceNT5cAae8ByHrKg6hld HznA== X-Gm-Message-State: ABy/qLb36T79BD8Qdz9pSJaCvzZPYe+3LXFZicDQRyI3YN3IWe9q6TRy 7k+pNjbEXLZCXSaRix5orb851Cfs3MCZhWYngLw= X-Google-Smtp-Source: APBJJlH9Lk751uohQZgZp5YzTMYQoCT9B8G+PNMOzyykXcGHjrCJuNtFebs9arY0gXdAWLB6fgCWQw== X-Received: by 2002:a92:d98d:0:b0:345:d977:5057 with SMTP id r13-20020a92d98d000000b00345d9775057mr6888343iln.4.1688308053203; Sun, 02 Jul 2023 07:27:33 -0700 (PDT) Received: from sjg1.lan (c-73-14-173-85.hsd1.co.comcast.net. [73.14.173.85]) by smtp.gmail.com with ESMTPSA id c13-20020a02c9cd000000b0042af5268e4asm2934713jap.58.2023.07.02.07.27.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 02 Jul 2023 07:27:33 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Cc: Simon Glass Subject: [PATCH 27/58] buildman: Adjust show_toolchain_prefix() to not return Date: Sun, 2 Jul 2023 15:25:52 +0100 Message-ID: <20230702142639.1249681-28-sjg@chromium.org> X-Mailer: git-send-email 2.41.0.255.g8b1d071c50-goog In-Reply-To: <20230702142639.1249681-1-sjg@chromium.org> References: <20230702142639.1249681-1-sjg@chromium.org> MIME-Version: 1.0 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.8 at phobos.denx.de X-Virus-Status: Clean This function does not need to return. Simplify the code by exiting immediately. Signed-off-by: Simon Glass --- tools/buildman/control.py | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-) diff --git a/tools/buildman/control.py b/tools/buildman/control.py index db8377e9ac42..d520e01334ce 100644 --- a/tools/buildman/control.py +++ b/tools/buildman/control.py @@ -121,10 +121,9 @@ def show_toolchain_prefix(brds, toolchains): for brd in board_selected.values(): tc_set.add(toolchains.Select(brd.arch)) if len(tc_set) != 1: - return 'Supplied boards must share one toolchain' + sys.exit('Supplied boards must share one toolchain') tchain = tc_set.pop() print(tchain.GetEnvArgs(toolchain.VAR_CROSS_COMPILE)) - return None def get_allow_missing(opt_allow, opt_no_allow, num_selected, has_branch): """Figure out whether to allow external blobs @@ -545,9 +544,7 @@ def do_buildman(options, args, toolchains=None, make_func=None, brds=None, return brds if options.print_prefix: - err = show_toolchain_prefix(brds, toolchains) - if err: - sys.exit(col.build(col.RED, err)) + show_toolchain_prefix(brds, toolchains) return 0 selected, why_selected, board_warnings = determine_boards( From patchwork Sun Jul 2 14:25:53 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1802350 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=2a01:238:438b:c500:173d:9f52:ddab:ee01; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.a=rsa-sha256 header.s=google header.b=NrVuxiNN; dkim-atps=neutral Received: from phobos.denx.de (phobos.denx.de [IPv6:2a01:238:438b:c500:173d:9f52:ddab:ee01]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QvBML6WfDz20ZF for ; Mon, 3 Jul 2023 00:31:42 +1000 (AEST) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id A77A98677F; Sun, 2 Jul 2023 16:28:13 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="NrVuxiNN"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id B88E5865DB; Sun, 2 Jul 2023 16:27:37 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.2 Received: from mail-il1-x134.google.com (mail-il1-x134.google.com [IPv6:2607:f8b0:4864:20::134]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id 37E96865D7 for ; Sun, 2 Jul 2023 16:27:35 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=sjg@chromium.org Received: by mail-il1-x134.google.com with SMTP id e9e14a558f8ab-34574eb05f4so15642805ab.0 for ; Sun, 02 Jul 2023 07:27:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1688308054; x=1690900054; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=G9qBH1w+spO04WGv7dBtppcYGlx3uddwoCzO1NuXsv0=; b=NrVuxiNN+B3eUo5N1U6fys+2g97N566EUgHMAmtj1c/v+Nkil+aIaS8PSbDhqjlAG/ STtSK+H0t7dWjMqHT20ThXTiGm1OsJPadRXm7MG1nN2RowhCzqnAf1yuUHqVXcSBaVV8 c6dv6l7v5eF+PNlnzGvdJWU27OYGNZ7KdV8yM= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1688308054; x=1690900054; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=G9qBH1w+spO04WGv7dBtppcYGlx3uddwoCzO1NuXsv0=; b=UGNYoXlbcrIwlhAacjoqK27L91algLveKKvA31tCnSGGNd7Qoi7lNdvRcZ17kv0jzM uozahEuG/LMP9WuSkTenz/aE4xkabHu21j9U3GaKWUdO0zmwOkQGUPL9fh1Z+FBNf9+j m4TRKNU83DQ6KBMRKdfoZ6PAPnFDGSpBvPN2I5NQCRqa7drENegCR1VC7gKhf5eDHdBR K1rIimv5AvYkzSd63ucgxPNO+LIMFRzkoXW/VFbpou4M7yhtdAfTP7LFMOeTzqvoxfXV Hw6ftPE4ep452AKQ4sGUUHsDB1oGeHdYGvVHqrNEHOajlrzoHs2VoXIiNJIkcX/3yvRE MqFg== X-Gm-Message-State: ABy/qLbuzCirE4/FsXya1n/dIly3Px690a4vUADYzrNke2baMeLwX1u/ ypWG5qnpb92VT/R2qPs1Kc7UbnZb9vBFjH4rQqg= X-Google-Smtp-Source: APBJJlH//wcIvbR9Sd215WsEP50/rIRVtZ7Kbf+z0ybW4Ao3KJKZAZ5k6GFou3P9ErkR8NfJ/5mCSA== X-Received: by 2002:a92:d682:0:b0:346:34f:beb2 with SMTP id p2-20020a92d682000000b00346034fbeb2mr2435666iln.5.1688308053854; Sun, 02 Jul 2023 07:27:33 -0700 (PDT) Received: from sjg1.lan (c-73-14-173-85.hsd1.co.comcast.net. [73.14.173.85]) by smtp.gmail.com with ESMTPSA id c13-20020a02c9cd000000b0042af5268e4asm2934713jap.58.2023.07.02.07.27.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 02 Jul 2023 07:27:33 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Cc: Simon Glass Subject: [PATCH 28/58] buildman: Move checking for make into run_builder() Date: Sun, 2 Jul 2023 15:25:53 +0100 Message-ID: <20230702142639.1249681-29-sjg@chromium.org> X-Mailer: git-send-email 2.41.0.255.g8b1d071c50-goog In-Reply-To: <20230702142639.1249681-1-sjg@chromium.org> References: <20230702142639.1249681-1-sjg@chromium.org> MIME-Version: 1.0 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.8 at phobos.denx.de X-Virus-Status: Clean This is not needed until the builder is run. Move it there to reduce the size of the do_buildman() function. Signed-off-by: Simon Glass --- tools/buildman/control.py | 14 ++++++++------ 1 file changed, 8 insertions(+), 6 deletions(-) diff --git a/tools/buildman/control.py b/tools/buildman/control.py index d520e01334ce..a1dc5334193c 100644 --- a/tools/buildman/control.py +++ b/tools/buildman/control.py @@ -465,6 +465,7 @@ def setup_output_dir(output_dir, work_in_output, branch, no_subdirs, col, shutil.rmtree(output_dir) return output_dir + def run_builder(builder, commits, board_selected, options): """Run the builder or show the summary @@ -478,6 +479,12 @@ def run_builder(builder, commits, board_selected, options): Returns: int: Return code for buildman """ + gnu_make = command.output(os.path.join(options.git, + 'scripts/show-gnu-make'), raise_on_error=False).rstrip() + if not gnu_make: + sys.exit('GNU Make not found') + builder.gnu_make = gnu_make + if not options.ide: tprint(get_action_summary(options.summary, commits, board_selected, options.step, options.threads, options.jobs)) @@ -562,11 +569,6 @@ def do_buildman(options, args, toolchains=None, make_func=None, brds=None, options.verbose) return 0 - gnu_make = command.output(os.path.join(options.git, - 'scripts/show-gnu-make'), raise_on_error=False).rstrip() - if not gnu_make: - sys.exit('GNU Make not found') - allow_missing = get_allow_missing(options.allow_missing, options.no_allow_missing, len(selected), options.branch) @@ -583,7 +585,7 @@ def do_buildman(options, args, toolchains=None, make_func=None, brds=None, # Create a new builder with the selected options builder = Builder(toolchains, output_dir, git_dir, - options.threads, options.jobs, gnu_make=gnu_make, checkout=True, + options.threads, options.jobs, checkout=True, show_unknown=options.show_unknown, step=options.step, no_subdirs=options.no_subdirs, full_path=options.full_path, verbose_build=options.verbose_build, From patchwork Sun Jul 2 14:25:54 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1802351 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=2a01:238:438b:c500:173d:9f52:ddab:ee01; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.a=rsa-sha256 header.s=google header.b=Qa/eGyoB; dkim-atps=neutral Received: from phobos.denx.de (phobos.denx.de [IPv6:2a01:238:438b:c500:173d:9f52:ddab:ee01]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QvBMW62p9z20ZF for ; Mon, 3 Jul 2023 00:31:51 +1000 (AEST) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id E4E3A8678F; Sun, 2 Jul 2023 16:28:15 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="Qa/eGyoB"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id C4EEF865DF; Sun, 2 Jul 2023 16:27:38 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.2 Received: from mail-il1-x136.google.com (mail-il1-x136.google.com [IPv6:2607:f8b0:4864:20::136]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id 08949865D5 for ; Sun, 2 Jul 2023 16:27:36 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=sjg@chromium.org Received: by mail-il1-x136.google.com with SMTP id e9e14a558f8ab-345f50577d3so12011525ab.2 for ; Sun, 02 Jul 2023 07:27:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1688308054; x=1690900054; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=oYE9ccRovKAq4ZRq9Uf7HvUEa7vyHWNyWcZkmcvn6fY=; b=Qa/eGyoBo5nl+3k3y6ryA7QN8nr4MC1QmK7H+o1MouSBuSi75zszXRaxJ7nDmacDge uCoc9GSvwBAf1AlACZmOMg6mP6Vg4CWUzhjoQHHkFMnLHLQ1bWlkyVewrHDhzaownMJF HS7gDwSi01qtD18Vo2yyT47ykV1Jz5y0VNCXA= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1688308054; x=1690900054; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=oYE9ccRovKAq4ZRq9Uf7HvUEa7vyHWNyWcZkmcvn6fY=; b=YIwDSpzwD0TlB+5DdNcalW+6ddCmKQRr/cNDyBJwU/AKBq/aOo3qdNwmDJjitkafX9 qbAvGQ3WtvCOnF+veQ0dca4NuZKBixXGqeXHAF74MWcLs7wyGAlMBnXlELWU7JJP8awr DcMzEqu1I5E/60oQfrZPvppnm9SeHs/occX5CHAUjGZyCgMq2+ZOLtQtiL3QxMbL22MV YAXAQYGhTYTeZzGxYEGNZGCltFQ9nPfLN87OeA7BvnaGW4aC6NHMLWsb4+X8nX9SXTrA rXzfzH+X9LtNPfPAwOVlwrVX1eJgP3JiWMBV3j1JDLqusXp/1qNn0kL6/TMt7jx6BMGs mV2g== X-Gm-Message-State: ABy/qLagl3ZWryRhe3wcwQ6HNeVH/jF6ejF9q3iYkTjHgP6YpZ+iPvG0 VWDy80oOIiUEPYRvtRAiyOKvB1uV6XpQ/NRCJa8= X-Google-Smtp-Source: APBJJlHJO9uWJg9FVrHRDRqnKWMUlU0ErziqWvWvxJevkaBMMiAs9NJkAZ2UsHmLSrFvA6q/T/Ca7A== X-Received: by 2002:a92:c051:0:b0:345:fbdc:bb78 with SMTP id o17-20020a92c051000000b00345fbdcbb78mr4663542ilf.29.1688308054727; Sun, 02 Jul 2023 07:27:34 -0700 (PDT) Received: from sjg1.lan (c-73-14-173-85.hsd1.co.comcast.net. [73.14.173.85]) by smtp.gmail.com with ESMTPSA id c13-20020a02c9cd000000b0042af5268e4asm2934713jap.58.2023.07.02.07.27.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 02 Jul 2023 07:27:34 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Cc: Simon Glass Subject: [PATCH 29/58] buildman: Move getting the adjust_cfg into run_builder() Date: Sun, 2 Jul 2023 15:25:54 +0100 Message-ID: <20230702142639.1249681-30-sjg@chromium.org> X-Mailer: git-send-email 2.41.0.255.g8b1d071c50-goog In-Reply-To: <20230702142639.1249681-1-sjg@chromium.org> References: <20230702142639.1249681-1-sjg@chromium.org> MIME-Version: 1.0 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.8 at phobos.denx.de X-Virus-Status: Clean Move this into its own function to reduce the size of do_buildman(). Signed-off-by: Simon Glass --- tools/buildman/control.py | 38 +++++++++++++++++++++++++++----------- 1 file changed, 27 insertions(+), 11 deletions(-) diff --git a/tools/buildman/control.py b/tools/buildman/control.py index a1dc5334193c..e60f4f156abd 100644 --- a/tools/buildman/control.py +++ b/tools/buildman/control.py @@ -507,6 +507,31 @@ def run_builder(builder, commits, board_selected, options): return 101 return 0 + +def calc_adjust_cfg(adjust_cfg, reproducible_builds): + """Calculate the value to use for adjust_cfg + + Args: + adjust_cfg (list of str): List of configuration changes. See cfgutil for + details + reproducible_builds (bool): True to adjust the configuration to get + reproduceable builds + + Returns: + adjust_cfg (list of str): List of configuration changes + """ + adjust_cfg = cfgutil.convert_list_to_dict(adjust_cfg) + + # Drop LOCALVERSION_AUTO since it changes the version string on every commit + if reproducible_builds: + # If these are mentioned, leave the local version alone + if 'LOCALVERSION' in adjust_cfg or 'LOCALVERSION_AUTO' in adjust_cfg: + print('Not dropping LOCALVERSION_AUTO for reproducible build') + else: + adjust_cfg['LOCALVERSION_AUTO'] = '~' + return adjust_cfg + + def do_buildman(options, args, toolchains=None, make_func=None, brds=None, clean_dir=False, test_thread_exceptions=False): """The main control code for buildman @@ -573,16 +598,6 @@ def do_buildman(options, args, toolchains=None, make_func=None, brds=None, options.no_allow_missing, len(selected), options.branch) - adjust_cfg = cfgutil.convert_list_to_dict(options.adjust_cfg) - - # Drop LOCALVERSION_AUTO since it changes the version string on every commit - if options.reproducible_builds: - # If these are mentioned, leave the local version alone - if 'LOCALVERSION' in adjust_cfg or 'LOCALVERSION_AUTO' in adjust_cfg: - print('Not dropping LOCALVERSION_AUTO for reproducible build') - else: - adjust_cfg['LOCALVERSION_AUTO'] = '~' - # Create a new builder with the selected options builder = Builder(toolchains, output_dir, git_dir, options.threads, options.jobs, checkout=True, @@ -596,7 +611,8 @@ def do_buildman(options, args, toolchains=None, make_func=None, brds=None, warnings_as_errors=options.warnings_as_errors, work_in_output=options.work_in_output, test_thread_exceptions=test_thread_exceptions, - adjust_cfg=adjust_cfg, + adjust_cfg=calc_adjust_cfg(options.adjust_cfg, + options.reproducible_builds), allow_missing=allow_missing, no_lto=options.no_lto, reproducible_builds=options.reproducible_builds, force_build = options.force_build, From patchwork Sun Jul 2 14:25:55 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1802352 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=85.214.62.61; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.a=rsa-sha256 header.s=google header.b=diab9k8r; dkim-atps=neutral Received: from phobos.denx.de (phobos.denx.de [85.214.62.61]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QvBMh4zdhz20ZF for ; Mon, 3 Jul 2023 00:32:00 +1000 (AEST) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id 94EE68677B; Sun, 2 Jul 2023 16:28:17 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="diab9k8r"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id 7E3FC865DF; Sun, 2 Jul 2023 16:27:39 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.2 Received: from mail-io1-xd35.google.com (mail-io1-xd35.google.com [IPv6:2607:f8b0:4864:20::d35]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id 023E8865F9 for ; Sun, 2 Jul 2023 16:27:37 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=sjg@chromium.org Received: by mail-io1-xd35.google.com with SMTP id ca18e2360f4ac-77acb04309dso191369739f.2 for ; Sun, 02 Jul 2023 07:27:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1688308055; x=1690900055; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ykS9blNosFe0E6Dd8mdgNJEN/sh3e/4bzTsIlyP0+Rk=; b=diab9k8rt2Te2Yz/QM1R6WKpGCfY5GKLk2rHAj11KSW/zLx8wuPsTnFX4YWZqV/gqL cQck6kvj1DSEWdTztkk8qVRP+fZdpseM1RnpmhsD9OhmpSQxu117itEZkBooq7UhSbsm 6Hbqiqw7IYMbcN4qVgHxQw29JT+d6oY7Xd/X0= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1688308055; x=1690900055; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ykS9blNosFe0E6Dd8mdgNJEN/sh3e/4bzTsIlyP0+Rk=; b=G6OdyZ+Uy20lr1Pcd0HtUbNmCTzB6wrfrZyx/l+VksBNTJXBfzh1AJc2/0dF8Tj0Ay yOgruAtXmEXjjzvl/VNWBdhO8Bmd8UGoh1Tq4v5lmr7u1ZudgLz2E2ULHUFdO8ba68dX y7Z4e++N8E+ub5vIF4ui0zqkDSKXqhSwkuXougL79rCt1OxNhfcF7paOryecEaaF0eoN 0QailLfdoT5lg63di25QsZ+SZh/EGzH5X9uI0JC3JBLtSmLA38VYPUjnrowY9VBrV+Sd SqASXOrWMrbDrHOkhVLDLYS3HaYEyGSvg9U11+3OwhgqSSNKJt+7OgZ8Bi2xk60K5nFK 2Mwg== X-Gm-Message-State: AC+VfDwkOPsm/tW2LZee1f6A8jTS8zgftuPOXZ9nHn3Cvix0x8nxEyWT +KnV2b++Dye6jFeshfBWZdMTDBiOvWSbzb1FSk8= X-Google-Smtp-Source: ACHHUZ4AH9jiJfEVGqiR06AhJ2GfHP2/1+LJxj/wTE/6zn6aWVWV0vIxNiuzyPcovbB9gd1yYIyEXg== X-Received: by 2002:a5e:d603:0:b0:786:4198:3c2d with SMTP id w3-20020a5ed603000000b0078641983c2dmr8580527iom.1.1688308055674; Sun, 02 Jul 2023 07:27:35 -0700 (PDT) Received: from sjg1.lan (c-73-14-173-85.hsd1.co.comcast.net. [73.14.173.85]) by smtp.gmail.com with ESMTPSA id c13-20020a02c9cd000000b0042af5268e4asm2934713jap.58.2023.07.02.07.27.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 02 Jul 2023 07:27:35 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Cc: Simon Glass Subject: [PATCH 30/58] buildman: Use get_alow_missing() directly to avoid var Date: Sun, 2 Jul 2023 15:25:55 +0100 Message-ID: <20230702142639.1249681-31-sjg@chromium.org> X-Mailer: git-send-email 2.41.0.255.g8b1d071c50-goog In-Reply-To: <20230702142639.1249681-1-sjg@chromium.org> References: <20230702142639.1249681-1-sjg@chromium.org> MIME-Version: 1.0 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.8 at phobos.denx.de X-Virus-Status: Clean Avoid an unnecessary local variable by moving this code to a function. This fixes the pylint warning about too many local variables. Signed-off-by: Simon Glass --- tools/buildman/control.py | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/tools/buildman/control.py b/tools/buildman/control.py index e60f4f156abd..612045bc89d5 100644 --- a/tools/buildman/control.py +++ b/tools/buildman/control.py @@ -594,10 +594,6 @@ def do_buildman(options, args, toolchains=None, make_func=None, brds=None, options.verbose) return 0 - allow_missing = get_allow_missing(options.allow_missing, - options.no_allow_missing, len(selected), - options.branch) - # Create a new builder with the selected options builder = Builder(toolchains, output_dir, git_dir, options.threads, options.jobs, checkout=True, @@ -613,7 +609,10 @@ def do_buildman(options, args, toolchains=None, make_func=None, brds=None, test_thread_exceptions=test_thread_exceptions, adjust_cfg=calc_adjust_cfg(options.adjust_cfg, options.reproducible_builds), - allow_missing=allow_missing, no_lto=options.no_lto, + allow_missing=get_allow_missing(options.allow_missing, + options.no_allow_missing, + len(selected), options.branch), + no_lto=options.no_lto, reproducible_builds=options.reproducible_builds, force_build = options.force_build, force_build_failures = options.force_build_failures, From patchwork Sun Jul 2 14:25:56 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1802353 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=85.214.62.61; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.a=rsa-sha256 header.s=google header.b=CNIa1rj1; dkim-atps=neutral Received: from phobos.denx.de (phobos.denx.de [85.214.62.61]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QvBMt21mRz20ZF for ; Mon, 3 Jul 2023 00:32:10 +1000 (AEST) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id DEBD086778; Sun, 2 Jul 2023 16:28:19 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="CNIa1rj1"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id 28BCC866A8; Sun, 2 Jul 2023 16:27:41 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.2 Received: from mail-io1-xd30.google.com (mail-io1-xd30.google.com [IPv6:2607:f8b0:4864:20::d30]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id 7B44B865D7 for ; Sun, 2 Jul 2023 16:27:38 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=sjg@chromium.org Received: by mail-io1-xd30.google.com with SMTP id ca18e2360f4ac-78372b896d0so144905139f.2 for ; Sun, 02 Jul 2023 07:27:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1688308057; x=1690900057; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=vgUuT5hbTddRqGZV3Qi4gx20QLpmbe0h8+JNitLWQaA=; b=CNIa1rj1APaEV2dY1zMPlfTjJZe8f5AfHF3GQVw6ci+0rH803RhvkxkKyerxPPkt85 g+38hnZKZ00NxOPJ/WAb9/OUXF8t1ZDD2k4w8Ka2+vfDCMUdKkShybR4i8ZiZ1fECEaq e9FH5emzPIN1Uw/xh9YsR7ThmI+vLGD9xRSzg= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1688308057; x=1690900057; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=vgUuT5hbTddRqGZV3Qi4gx20QLpmbe0h8+JNitLWQaA=; b=VOX03IhFP+CkpWSfCNhusXJDJ5kB+yFrcootGYkhZ8Dw8A7kBZdA8KPD3P2LXm0VhM 1SB8ExECihJASV9IRxutrTvTZ3jd7T1+tKpXVC0FcX0+DepOgBv+xKDczRGDmaXn4QFI w3/52rksSf16Gzc/N2U28DyP3b9IY6EMEr6lqamRuAoWrUPzpKhFZiNwj9dVUubANsUm NVv380celpKyCajF+vSg/feJkm1hBe+GGjaSohr4NjQH4SE4XUJR5f9thPiGdgHLlHYF PCvZVU9fPSeDL89yOLUc9zEyJCZ5Cy2nsHq5A23kd/u/ZxLXDf1sDMXlSOlYFgvTaUH+ pOsw== X-Gm-Message-State: AC+VfDzbV6F3FO44sagoV+V+Ec7uX/nZxmxY0KqXSVsJHBRdKGxDeK2t CriYy1yacneoa1WkpZIyf42SHRGHuGqB2MuvPlU= X-Google-Smtp-Source: ACHHUZ4QI+lCoaNQKgXBgJhyKGny9OWWqJjgb9rWk7wqwG3STU0qB9ykbuKsnXgsFb+p6z1IAlq99Q== X-Received: by 2002:a6b:7e01:0:b0:785:fbe8:1da0 with SMTP id i1-20020a6b7e01000000b00785fbe81da0mr8215345iom.15.1688308057134; Sun, 02 Jul 2023 07:27:37 -0700 (PDT) Received: from sjg1.lan (c-73-14-173-85.hsd1.co.comcast.net. [73.14.173.85]) by smtp.gmail.com with ESMTPSA id c13-20020a02c9cd000000b0042af5268e4asm2934713jap.58.2023.07.02.07.27.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 02 Jul 2023 07:27:36 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Cc: Simon Glass Subject: [PATCH 31/58] buildman: Create a function to get number of built commits Date: Sun, 2 Jul 2023 15:25:56 +0100 Message-ID: <20230702142639.1249681-32-sjg@chromium.org> X-Mailer: git-send-email 2.41.0.255.g8b1d071c50-goog In-Reply-To: <20230702142639.1249681-1-sjg@chromium.org> References: <20230702142639.1249681-1-sjg@chromium.org> MIME-Version: 1.0 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.8 at phobos.denx.de X-Virus-Status: Clean Move this code into a function. This removes the last pylint error in the control module. Signed-off-by: Simon Glass --- tools/buildman/control.py | 34 +++++++++++++++++++++++++--------- 1 file changed, 25 insertions(+), 9 deletions(-) diff --git a/tools/buildman/control.py b/tools/buildman/control.py index 612045bc89d5..346a40f26e9e 100644 --- a/tools/buildman/control.py +++ b/tools/buildman/control.py @@ -29,7 +29,24 @@ def get_plural(count): """Returns a plural 's' if count is not 1""" return 's' if count != 1 else '' -def get_action_summary(is_summary, commits, selected, step, threads, jobs): + +def count_build_commits(commits, step): + """Calculate the number of commits to be built + + Args: + commits (list of Commit): Commits to build or None + step (int): Step value for commits, typically 1 + + Returns: + Number of commits that will be built + """ + if commits: + count = len(commits) + return (count + step - 1) // step + return 0 + + +def get_action_summary(is_summary, commit_count, selected, threads, jobs): """Return a string summarising the intended action. Args: @@ -43,10 +60,8 @@ def get_action_summary(is_summary, commits, selected, step, threads, jobs): Returns: Summary string. """ - if commits: - count = len(commits) - count = (count + step - 1) // step - commit_str = f'{count} commit{get_plural(count)}' + if commit_count: + commit_str = f'{commit_count} commit{get_plural(commit_count)}' else: commit_str = 'current source' msg = (f"{'Summary of' if is_summary else 'Building'} " @@ -83,8 +98,8 @@ def show_actions(series, why_selected, boards_selected, output_dir, commits = series.commits else: commits = None - print(get_action_summary(False, commits, boards_selected, step, threads, - jobs)) + print(get_action_summary(False, count_build_commits(commits, step), + boards_selected, threads, jobs)) print(f'Build directory: {output_dir}') if commits: for upto in range(0, len(series.commits), step): @@ -486,8 +501,9 @@ def run_builder(builder, commits, board_selected, options): builder.gnu_make = gnu_make if not options.ide: - tprint(get_action_summary(options.summary, commits, board_selected, - options.step, options.threads, options.jobs)) + commit_count = count_build_commits(commits, options.step) + tprint(get_action_summary(options.summary, commit_count, board_selected, + options.threads, options.jobs)) builder.SetDisplayOptions( options.show_errors, options.show_sizes, options.show_detail, From patchwork Sun Jul 2 14:25:57 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1802354 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=85.214.62.61; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.a=rsa-sha256 header.s=google header.b=TOSoThfm; dkim-atps=neutral Received: from phobos.denx.de (phobos.denx.de [85.214.62.61]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QvBN36TWZz20ZF for ; Mon, 3 Jul 2023 00:32:19 +1000 (AEST) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id 1B80386793; Sun, 2 Jul 2023 16:28:22 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="TOSoThfm"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id EB823866A8; Sun, 2 Jul 2023 16:27:41 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.2 Received: from mail-il1-x12c.google.com (mail-il1-x12c.google.com [IPv6:2607:f8b0:4864:20::12c]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id 2399686609 for ; Sun, 2 Jul 2023 16:27:39 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=sjg@chromium.org Received: by mail-il1-x12c.google.com with SMTP id e9e14a558f8ab-341ff6251f2so15130825ab.0 for ; Sun, 02 Jul 2023 07:27:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1688308058; x=1690900058; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=WpI2rJmQPl49viKJb7HnujkwaatLp7HydvZ+eklC9i0=; b=TOSoThfmiXevWlH7LmDtQNbYPxGRCIrpKfRQOz8NcnioCQYqzMoOHb6UmnZB+nrQRe nR6n4SBh74inANx8piPaqGA9Vv+PaPY6yxQEQQuGpbW8I+UFJClEL0qJ7UFqnqQMlFEV B4OU6leP6t0dzxdynpTYCAt+UmAFLBtFEaP6A= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1688308058; x=1690900058; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=WpI2rJmQPl49viKJb7HnujkwaatLp7HydvZ+eklC9i0=; b=CIuS0F4WKBS4g3B7GNFIaGGOESQMbnIuGTKNNP3Ac+5PblkNCXrp3c3t3j9O9UBJqg QbguQKgT3nlZ6xQG43wvAY/I0pbRj0G/rJK3Yk5J3n5RL05Y9QWUijQM3GGni9lLRMDO fOpX0MCRBurmBOVl51NnmKzOcaKMXDOhaTtX9TJtazZRoYi3QnUi3IwN0kH7ElZTDUOX ijONGIDwgVwyjLtbjCMcJf8ck+rGnW429siMr4DA5nEwYGzg/t96+egVK9fKaMMuxCbo k4+FyQzaKqeDnlSuen/tpfcE02pT+rXW3Y7IVeCXQpssdNckHAQaaEXMVMy8wXjTbNwR kilQ== X-Gm-Message-State: ABy/qLZfzX7pOQ34UBvdr3JWo9gZu3hTfLTKpB/5h4fp2QDfVOyuq9bM Q5SwO3m9DN2NAMStwcB5o4voQT0KwX2XjP4dU/M= X-Google-Smtp-Source: APBJJlGyTheh9oSRIwCqud7kyRMOYs8cmlkHLoyvv3/wOvAftWW75Y5arHyvYksTW5kZkbxL4IUoKA== X-Received: by 2002:a92:cecf:0:b0:345:c2f2:97a3 with SMTP id z15-20020a92cecf000000b00345c2f297a3mr7304446ilq.9.1688308057803; Sun, 02 Jul 2023 07:27:37 -0700 (PDT) Received: from sjg1.lan (c-73-14-173-85.hsd1.co.comcast.net. [73.14.173.85]) by smtp.gmail.com with ESMTPSA id c13-20020a02c9cd000000b0042af5268e4asm2934713jap.58.2023.07.02.07.27.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 02 Jul 2023 07:27:37 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Cc: Simon Glass Subject: [PATCH 32/58] buildman: Convert camel case in cmdline.py Date: Sun, 2 Jul 2023 15:25:57 +0100 Message-ID: <20230702142639.1249681-33-sjg@chromium.org> X-Mailer: git-send-email 2.41.0.255.g8b1d071c50-goog In-Reply-To: <20230702142639.1249681-1-sjg@chromium.org> References: <20230702142639.1249681-1-sjg@chromium.org> MIME-Version: 1.0 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.8 at phobos.denx.de X-Virus-Status: Clean Convert this file to snake case and update all files which use it. Signed-off-by: Simon Glass --- tools/buildman/cmdline.py | 2 +- tools/buildman/func_test.py | 2 +- tools/buildman/main.py | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/tools/buildman/cmdline.py b/tools/buildman/cmdline.py index ea43685e394d..3066a0b38bbf 100644 --- a/tools/buildman/cmdline.py +++ b/tools/buildman/cmdline.py @@ -9,7 +9,7 @@ import pathlib BUILDMAN_DIR = pathlib.Path(__file__).parent HAS_TESTS = os.path.exists(BUILDMAN_DIR / "test.py") -def ParseArgs(): +def parse_args(): """Parse command line arguments from sys.argv[] Returns: diff --git a/tools/buildman/func_test.py b/tools/buildman/func_test.py index d9b4c41ec68c..ad48e3b2159f 100644 --- a/tools/buildman/func_test.py +++ b/tools/buildman/func_test.py @@ -244,7 +244,7 @@ class TestFunctional(unittest.TestCase): result code from buildman """ sys.argv = [sys.argv[0]] + list(args) - options, args = cmdline.ParseArgs() + options, args = cmdline.parse_args() if brds == False: brds = self._boards result = control.do_buildman( diff --git a/tools/buildman/main.py b/tools/buildman/main.py index 2253401709a8..0f711c8653b3 100755 --- a/tools/buildman/main.py +++ b/tools/buildman/main.py @@ -59,7 +59,7 @@ def run_buildman(): This is the main program. It collects arguments and runs either the tests or the control module. """ - options, args = cmdline.ParseArgs() + options, args = cmdline.parse_args() if not options.debug: sys.tracebacklimit = 0 From patchwork Sun Jul 2 14:25:58 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1802355 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=2a01:238:438b:c500:173d:9f52:ddab:ee01; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.a=rsa-sha256 header.s=google header.b=SQ1PS9TM; dkim-atps=neutral Received: from phobos.denx.de (phobos.denx.de [IPv6:2a01:238:438b:c500:173d:9f52:ddab:ee01]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QvBND717Fz20ZF for ; Mon, 3 Jul 2023 00:32:28 +1000 (AEST) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id C0813867A2; Sun, 2 Jul 2023 16:28:24 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="SQ1PS9TM"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id 5D03886609; Sun, 2 Jul 2023 16:27:42 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.2 Received: from mail-io1-xd36.google.com (mail-io1-xd36.google.com [IPv6:2607:f8b0:4864:20::d36]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id C93DF865F9 for ; Sun, 2 Jul 2023 16:27:39 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=sjg@chromium.org Received: by mail-io1-xd36.google.com with SMTP id ca18e2360f4ac-77acb04309dso191372039f.2 for ; Sun, 02 Jul 2023 07:27:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1688308058; x=1690900058; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=zOHJVkChmJpv4L0GXH874Xo3teEgNz9BxkAGLgFBgVQ=; b=SQ1PS9TMSm5NGibRrvdqttyXnQohRhfX3ciXdraBosubGv3zoX9TGV0LaBXex/t+u+ E9IX2ASoE3v0yTCXjVAExghUuHegVMAB+C8YjjBlIULuuLyfYcDdaQb0aEDuexaK9y3B pn+1BHuUDljPfd5jy8ACXQ44Y8jBYMFnyIjbI= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1688308058; x=1690900058; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=zOHJVkChmJpv4L0GXH874Xo3teEgNz9BxkAGLgFBgVQ=; b=ING+5qKYtSNMhto0eqYRDo+QugkioJRWSIpSMMb85e283NlJf+AooWUdGIr9n4LbKl ANWcu/TvkH3LW83WGyYX+I7V7lTNKKPtTpn7jxdzLdHYfcadOZHn+MueGMbH23EucX7z KThsd13LmI1OvqcyI/meoDNihq5AwOtuaTwz+MgvtMz32EmUDvYw0/MlvICf/q11NSf2 npR45e2nLGwBxAa4NdRf+ZaFaWWzSwyrsRWYdFAh/R8kpnmCxL+h4tSqwyoB5h1WWp+/ IdzP0I5L8c0tAx+Ol+J5lIr5GglipDN1bZuliXX4dz2fxQBDQuLRNg0wB69A7XbL5XzD NLpQ== X-Gm-Message-State: ABy/qLY84/L0F7BZ3F6T95qwC+Ffm/eVNcGKyI37e/u5Ye/714gyDa70 vrz0l9M/AAYQnq3ZHaf37o5CavRqK8BTMBHmrOA= X-Google-Smtp-Source: APBJJlFgNd0WZcqOYAVXMY+TON6Q75Csw0hYU7GRw0apbj2SZgr/9bNzewLjaK9usz3Srgo/wwtP2w== X-Received: by 2002:a92:cb47:0:b0:345:a3c6:87b6 with SMTP id f7-20020a92cb47000000b00345a3c687b6mr7428759ilq.32.1688308058462; Sun, 02 Jul 2023 07:27:38 -0700 (PDT) Received: from sjg1.lan (c-73-14-173-85.hsd1.co.comcast.net. [73.14.173.85]) by smtp.gmail.com with ESMTPSA id c13-20020a02c9cd000000b0042af5268e4asm2934713jap.58.2023.07.02.07.27.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 02 Jul 2023 07:27:38 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Cc: Simon Glass Subject: [PATCH 33/58] buildman: Correct most pylint warnings in cmdline Date: Sun, 2 Jul 2023 15:25:58 +0100 Message-ID: <20230702142639.1249681-34-sjg@chromium.org> X-Mailer: git-send-email 2.41.0.255.g8b1d071c50-goog In-Reply-To: <20230702142639.1249681-1-sjg@chromium.org> References: <20230702142639.1249681-1-sjg@chromium.org> MIME-Version: 1.0 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.8 at phobos.denx.de X-Virus-Status: Clean Tidu up warnings in this file. Signed-off-by: Simon Glass --- tools/buildman/cmdline.py | 15 +++++++++++---- 1 file changed, 11 insertions(+), 4 deletions(-) diff --git a/tools/buildman/cmdline.py b/tools/buildman/cmdline.py index 3066a0b38bbf..91571a5adb1d 100644 --- a/tools/buildman/cmdline.py +++ b/tools/buildman/cmdline.py @@ -2,6 +2,11 @@ # Copyright (c) 2014 Google, Inc # +"""Handles parsing of buildman arguments + +This creates the argument parser and uses it to parse the arguments passed in +""" + from optparse import OptionParser import os import pathlib @@ -71,7 +76,8 @@ def parse_args(): parser.add_option('-k', '--keep-outputs', action='store_true', default=False, help='Keep all build output files (e.g. binaries)') parser.add_option('-K', '--show-config', action='store_true', - default=False, help='Show configuration changes in summary (both board config files and Kconfig)') + default=False, + help='Show configuration changes in summary (both board config files and Kconfig)') parser.add_option('--preserve-config-y', action='store_true', default=False, help="Don't convert y to 1 in configs") parser.add_option('-l', '--list-error-boards', action='store_true', @@ -84,14 +90,15 @@ def parse_args(): default=False, help="Run 'make mrproper before reconfiguring") parser.add_option( '-M', '--allow-missing', action='store_true', default=False, - help='Tell binman to allow missing blobs and generate fake ones as needed'), + help='Tell binman to allow missing blobs and generate fake ones as needed') parser.add_option( '--no-allow-missing', action='store_true', default=False, - help='Disable telling binman to allow missing blobs'), + help='Disable telling binman to allow missing blobs') parser.add_option('-n', '--dry-run', action='store_true', dest='dry_run', default=False, help="Do a dry run (describe actions, but do nothing)") parser.add_option('-N', '--no-subdirs', action='store_true', dest='no_subdirs', - default=False, help="Don't create subdirectories when building current source for a single board") + default=False, + help="Don't create subdirectories when building current source for a single board") parser.add_option('-o', '--output-dir', type='string', dest='output_dir', help='Directory where all builds happen and buildman has its workspace (default is ../)') parser.add_option('-O', '--override-toolchain', type='string', From patchwork Sun Jul 2 14:25:59 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1802356 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=2a01:238:438b:c500:173d:9f52:ddab:ee01; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.a=rsa-sha256 header.s=google header.b=OtidgUd3; dkim-atps=neutral Received: from phobos.denx.de (phobos.denx.de [IPv6:2a01:238:438b:c500:173d:9f52:ddab:ee01]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QvBNQ1CjPz20ZF for ; Mon, 3 Jul 2023 00:32:38 +1000 (AEST) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id 06D0D8679D; Sun, 2 Jul 2023 16:28:27 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="OtidgUd3"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id 79F868665C; Sun, 2 Jul 2023 16:27:44 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.2 Received: from mail-io1-xd29.google.com (mail-io1-xd29.google.com [IPv6:2607:f8b0:4864:20::d29]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id A2EBF86684 for ; Sun, 2 Jul 2023 16:27:40 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=sjg@chromium.org Received: by mail-io1-xd29.google.com with SMTP id ca18e2360f4ac-785d738d3feso87728039f.0 for ; Sun, 02 Jul 2023 07:27:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1688308059; x=1690900059; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=4SH/gIFi5lGKzjcRLb6JNy1Y2051WCHvfJO6Vo/mT90=; b=OtidgUd3iXU7eX09NZIADqLkGOVMRKPqCTrwa49pnvvIJA6Ncw261sJcejQEucxpcR dVoPw8isBBoNd0rDoXA7NNqezpsnfEVsE9sQRYDNbJEciLqoWI/iAsEjvUjQSQa+3Es8 T44b4cespjXfWZ+aezPRYoQxevm8M+1RwjQrg= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1688308059; x=1690900059; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=4SH/gIFi5lGKzjcRLb6JNy1Y2051WCHvfJO6Vo/mT90=; b=euElse6dLrSHzr4+RvB+9ObCPhRhXnDu7VbvLiFjpGNAtniyRiYvq6O2RXRpcfhu1A +351dxxyXvkkL448nWLymwEI2vmEpeZeA9tgH+SPCt5mNW1ZoXJQxWEdDrxG8ax3Xras V9s8typJArSkMfuEoR1uIz6au7tEzUSHbrJvMZgy8PHmonXv2ujgnm07Rqsah3XrQljy dl0EV6aV8otzzEoswnViTAvZ6T7IhFkmDuJCV6l5fqNJAAlyUwe+qs0TDMPcHWNzr6Sz 2IDtDsT8t/mzy5xwwTQmSOFeDGC49BQYGZNUCg+kqlBDMNRzPT43bcmcpP/0JrvjPb2E 8pWQ== X-Gm-Message-State: AC+VfDzlhJPiLQ4KcxuL5tmiltFK5dzKZN5AJPzlqKaklJhwNpp9Y/Gj f0Rs2hoj+sCyKq67cq8/FbRHdcIuC/xOVsodFKA= X-Google-Smtp-Source: ACHHUZ5GZwLLIYuRI79q/sgyp23SZXpeE6GKuVcirBHTZx+pSGfuIkDch8Gb6iZFCDUuDOTxvTowSw== X-Received: by 2002:a05:6602:1224:b0:783:7453:e9c with SMTP id z4-20020a056602122400b0078374530e9cmr9243432iot.4.1688308059262; Sun, 02 Jul 2023 07:27:39 -0700 (PDT) Received: from sjg1.lan (c-73-14-173-85.hsd1.co.comcast.net. [73.14.173.85]) by smtp.gmail.com with ESMTPSA id c13-20020a02c9cd000000b0042af5268e4asm2934713jap.58.2023.07.02.07.27.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 02 Jul 2023 07:27:39 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Cc: Simon Glass Subject: [PATCH 34/58] buildman: Add a test for --boards Date: Sun, 2 Jul 2023 15:25:59 +0100 Message-ID: <20230702142639.1249681-35-sjg@chromium.org> X-Mailer: git-send-email 2.41.0.255.g8b1d071c50-goog In-Reply-To: <20230702142639.1249681-1-sjg@chromium.org> References: <20230702142639.1249681-1-sjg@chromium.org> MIME-Version: 1.0 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.8 at phobos.denx.de X-Virus-Status: Clean Add a simple functional test for the --boards option. Fix the example in the docs while we are here. Also improve the docs for Builder.count so it is clearer what it contains. Signed-off-by: Simon Glass --- tools/buildman/builder.py | 3 ++- tools/buildman/buildman.rst | 2 +- tools/buildman/func_test.py | 11 +++++++++++ 3 files changed, 14 insertions(+), 2 deletions(-) diff --git a/tools/buildman/builder.py b/tools/buildman/builder.py index cb3628a8a085..620b7b8c31a8 100644 --- a/tools/buildman/builder.py +++ b/tools/buildman/builder.py @@ -163,7 +163,8 @@ class Builder: checkout: True to check out source, False to skip that step. This is used for testing. col: terminal.Color() object - count: Number of commits to build + count: Total number of commits to build, which is the number of commits + multiplied by the number of boards do_make: Method to call to invoke Make fail: Number of builds that failed due to error force_build: Force building even if a build already exists diff --git a/tools/buildman/buildman.rst b/tools/buildman/buildman.rst index b28aea477dfe..d8c3957097e8 100644 --- a/tools/buildman/buildman.rst +++ b/tools/buildman/buildman.rst @@ -159,7 +159,7 @@ on the command line: .. code-block:: bash - buildman --boards sandbox,snow --boards + buildman --boards sandbox,snow --boards firefly-rk3399 It is convenient to use the -n option to see what will be built based on the subset given. Use -v as well to get an actual list of boards. diff --git a/tools/buildman/func_test.py b/tools/buildman/func_test.py index ad48e3b2159f..966ad178a902 100644 --- a/tools/buildman/func_test.py +++ b/tools/buildman/func_test.py @@ -792,3 +792,14 @@ CONFIG_LOCALVERSION=y os.remove(outfile) result = self._RunControl('-R', outfile, brds=None, get_builder=False) self.assertTrue(os.path.exists(outfile)) + + def testSingleBoards(self): + """Test building single boards""" + self._RunControl('--boards', 'board1') + self.assertEqual(1, self._builder.count) + + self._RunControl('--boards', 'board1', '--boards', 'board2') + self.assertEqual(2, self._builder.count) + + self._RunControl('--boards', 'board1,board2', '--boards', 'board4') + self.assertEqual(3, self._builder.count) From patchwork Sun Jul 2 14:26:00 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1802359 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=2a01:238:438b:c500:173d:9f52:ddab:ee01; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.a=rsa-sha256 header.s=google header.b=ofa4UIL7; dkim-atps=neutral Received: from phobos.denx.de (phobos.denx.de [IPv6:2a01:238:438b:c500:173d:9f52:ddab:ee01]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QvBNz3xPhz20ZF for ; Mon, 3 Jul 2023 00:33:07 +1000 (AEST) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id F1FDB867B1; Sun, 2 Jul 2023 16:28:31 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="ofa4UIL7"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id CD5BD866CA; Sun, 2 Jul 2023 16:27:47 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.2 Received: from mail-io1-xd33.google.com (mail-io1-xd33.google.com [IPv6:2607:f8b0:4864:20::d33]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id E92DA866B3 for ; Sun, 2 Jul 2023 16:27:41 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=sjg@chromium.org Received: by mail-io1-xd33.google.com with SMTP id ca18e2360f4ac-78666994bc2so35070839f.0 for ; Sun, 02 Jul 2023 07:27:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1688308060; x=1690900060; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=8mam/754xZTGx1QR11mh7Q7opSVHfY+qlytrd+YWG7k=; b=ofa4UIL79omGFSuIQqoJ0jJZwKUEByqeV+oO9mh+Xc+71WiShMu8pxH/WLW7dRN0F8 SvZgPsg1VWwoVDPUT3zhHz3f0bmu63aY/YD+5QWZyQzsC8sCVW2VqmurOfpEAL2B2mAM buI8KT3OtaOJaNeCdrbEEYgkKCXzBmS5d4ihA= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1688308060; x=1690900060; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=8mam/754xZTGx1QR11mh7Q7opSVHfY+qlytrd+YWG7k=; b=QNObTS5v2cqrs6Ptm7gTB3ZWX5oclvgm0ooIs/R7VB8ryy13dIu4dF75a0ZarHJw2b Q3749w7Wx44a4ijCRgJC3nAnLnPcbuTps9BkLBHTfoF7L5ROyls01nwQk3Ns4Wh2Ljvk zgb/Acvqb7D+YMPRXIiRF0pJXcwzJ7ThekuIJvShZwUGS0S8EMZPghhjaM0IZlJ7yO8L AWxdBIMcMP6Livydkj3ZODhasYS4dQmy4JQwBR1EFje7ZJKWXskdgI1BcbaV7svcQuWZ wUVFfODzdzqKYiKeyh8RiffzSFCEhPvIhb3hM90T75xmkQnWyw3vwLiQBTlo9kNvKY8F m3YA== X-Gm-Message-State: AC+VfDxO6JUc7XuYdzWFG4lDUmwToBDhv2Ahk42PXjwfJ1uyUvQpcRQu Pc09cnPcyIdgQ9mb6PjZj4hPCimmpazlRG3B/vA= X-Google-Smtp-Source: ACHHUZ6zxuenwP9tcZ7KG1ALATIVYAvp6tWYbt0TXsg63ySLHn93CXvTcLm4LVMFAjclQ3HswlS3WA== X-Received: by 2002:a05:6602:21cc:b0:783:5c26:db82 with SMTP id c12-20020a05660221cc00b007835c26db82mr8872231ioc.5.1688308060286; Sun, 02 Jul 2023 07:27:40 -0700 (PDT) Received: from sjg1.lan (c-73-14-173-85.hsd1.co.comcast.net. [73.14.173.85]) by smtp.gmail.com with ESMTPSA id c13-20020a02c9cd000000b0042af5268e4asm2934713jap.58.2023.07.02.07.27.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 02 Jul 2023 07:27:40 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Cc: Simon Glass Subject: [PATCH 35/58] buildman: Convert to argparse Date: Sun, 2 Jul 2023 15:26:00 +0100 Message-ID: <20230702142639.1249681-36-sjg@chromium.org> X-Mailer: git-send-email 2.41.0.255.g8b1d071c50-goog In-Reply-To: <20230702142639.1249681-1-sjg@chromium.org> References: <20230702142639.1249681-1-sjg@chromium.org> MIME-Version: 1.0 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.8 at phobos.denx.de X-Virus-Status: Clean Use argparse to parse the arguments, since OptionParser is deprecated now. Signed-off-by: Simon Glass --- tools/buildman/cmdline.py | 125 ++++++++++++++++---------------- tools/buildman/control.py | 140 ++++++++++++++++++------------------ tools/buildman/func_test.py | 6 +- tools/buildman/main.py | 16 ++--- 4 files changed, 145 insertions(+), 142 deletions(-) diff --git a/tools/buildman/cmdline.py b/tools/buildman/cmdline.py index 91571a5adb1d..6b9c1db14af7 100644 --- a/tools/buildman/cmdline.py +++ b/tools/buildman/cmdline.py @@ -7,7 +7,7 @@ This creates the argument parser and uses it to parse the arguments passed in """ -from optparse import OptionParser +import argparse import os import pathlib @@ -22,135 +22,138 @@ def parse_args(): options: command line options args: command lin arguments """ - parser = OptionParser() - parser.add_option('-a', '--adjust-cfg', type=str, action='append', + epilog = """ [list of target/arch/cpu/board/vendor/soc to build] + + Build U-Boot for all commits in a branch. Use -n to do a dry run""" + + parser = argparse.ArgumentParser(epilog=epilog) + parser.add_argument('-a', '--adjust-cfg', type=str, action='append', help='Adjust the Kconfig settings in .config before building') - parser.add_option('-A', '--print-prefix', action='store_true', + parser.add_argument('-A', '--print-prefix', action='store_true', help='Print the tool-chain prefix for a board (CROSS_COMPILE=)') - parser.add_option('-b', '--branch', type='string', + parser.add_argument('-b', '--branch', type=str, help='Branch name to build, or range of commits to build') - parser.add_option('-B', '--bloat', dest='show_bloat', + parser.add_argument('-B', '--bloat', dest='show_bloat', action='store_true', default=False, help='Show changes in function code size for each board') - parser.add_option('--boards', type='string', action='append', + parser.add_argument('--boards', type=str, action='append', help='List of board names to build separated by comma') - parser.add_option('-c', '--count', dest='count', type='int', + parser.add_argument('-c', '--count', dest='count', type=int, default=-1, help='Run build on the top n commits') - parser.add_option('-C', '--force-reconfig', dest='force_reconfig', + parser.add_argument('-C', '--force-reconfig', dest='force_reconfig', action='store_true', default=False, help='Reconfigure for every commit (disable incremental build)') - parser.add_option('-d', '--detail', dest='show_detail', + parser.add_argument('-d', '--detail', dest='show_detail', action='store_true', default=False, help='Show detailed size delta for each board in the -S summary') - parser.add_option('-D', '--config-only', action='store_true', default=False, + parser.add_argument('-D', '--config-only', action='store_true', + default=False, help="Don't build, just configure each commit") - parser.add_option('--debug', action='store_true', + parser.add_argument('--debug', action='store_true', help='Enabling debugging (provides a full traceback on error)') - parser.add_option('-e', '--show_errors', action='store_true', + parser.add_argument('-e', '--show_errors', action='store_true', default=False, help='Show errors and warnings') - parser.add_option('-E', '--warnings-as-errors', action='store_true', + parser.add_argument('-E', '--warnings-as-errors', action='store_true', default=False, help='Treat all compiler warnings as errors') - parser.add_option('-f', '--force-build', dest='force_build', + parser.add_argument('-f', '--force-build', dest='force_build', action='store_true', default=False, help='Force build of boards even if already built') - parser.add_option('-F', '--force-build-failures', dest='force_build_failures', + parser.add_argument('-F', '--force-build-failures', dest='force_build_failures', action='store_true', default=False, help='Force build of previously-failed build') - parser.add_option('--fetch-arch', type='string', + parser.add_argument('--fetch-arch', type=str, help="Fetch a toolchain for architecture FETCH_ARCH ('list' to list)." ' You can also fetch several toolchains separate by comma, or' " 'all' to download all") - parser.add_option('-g', '--git', type='string', + parser.add_argument('-g', '--git', type=str, help='Git repo containing branch to build', default='.') - parser.add_option('-G', '--config-file', type='string', + parser.add_argument('-G', '--config-file', type=str, help='Path to buildman config file', default='') - parser.add_option('-H', '--full-help', action='store_true', dest='full_help', + parser.add_argument('-H', '--full-help', action='store_true', dest='full_help', default=False, help='Display the README file') - parser.add_option('-i', '--in-tree', dest='in_tree', + parser.add_argument('-i', '--in-tree', dest='in_tree', action='store_true', default=False, help='Build in the source tree instead of a separate directory') - parser.add_option('-I', '--ide', action='store_true', default=False, + parser.add_argument('-I', '--ide', action='store_true', default=False, help='Create build output that can be parsed by an IDE') - parser.add_option('-j', '--jobs', dest='jobs', type='int', + parser.add_argument('-j', '--jobs', dest='jobs', type=int, default=None, help='Number of jobs to run at once (passed to make)') - parser.add_option('-k', '--keep-outputs', action='store_true', + parser.add_argument('-k', '--keep-outputs', action='store_true', default=False, help='Keep all build output files (e.g. binaries)') - parser.add_option('-K', '--show-config', action='store_true', + parser.add_argument('-K', '--show-config', action='store_true', default=False, help='Show configuration changes in summary (both board config files and Kconfig)') - parser.add_option('--preserve-config-y', action='store_true', + parser.add_argument('--preserve-config-y', action='store_true', default=False, help="Don't convert y to 1 in configs") - parser.add_option('-l', '--list-error-boards', action='store_true', + parser.add_argument('-l', '--list-error-boards', action='store_true', default=False, help='Show a list of boards next to each error/warning') - parser.add_option('-L', '--no-lto', action='store_true', + parser.add_argument('-L', '--no-lto', action='store_true', default=False, help='Disable Link-time Optimisation (LTO) for builds') - parser.add_option('--list-tool-chains', action='store_true', default=False, + parser.add_argument('--list-tool-chains', action='store_true', default=False, help='List available tool chains (use -v to see probing detail)') - parser.add_option('-m', '--mrproper', action='store_true', + parser.add_argument('-m', '--mrproper', action='store_true', default=False, help="Run 'make mrproper before reconfiguring") - parser.add_option( + parser.add_argument( '-M', '--allow-missing', action='store_true', default=False, help='Tell binman to allow missing blobs and generate fake ones as needed') - parser.add_option( + parser.add_argument( '--no-allow-missing', action='store_true', default=False, help='Disable telling binman to allow missing blobs') - parser.add_option('-n', '--dry-run', action='store_true', dest='dry_run', + parser.add_argument('-n', '--dry-run', action='store_true', dest='dry_run', default=False, help="Do a dry run (describe actions, but do nothing)") - parser.add_option('-N', '--no-subdirs', action='store_true', dest='no_subdirs', + parser.add_argument('-N', '--no-subdirs', action='store_true', dest='no_subdirs', default=False, help="Don't create subdirectories when building current source for a single board") - parser.add_option('-o', '--output-dir', type='string', dest='output_dir', + parser.add_argument('-o', '--output-dir', type=str, dest='output_dir', help='Directory where all builds happen and buildman has its workspace (default is ../)') - parser.add_option('-O', '--override-toolchain', type='string', + parser.add_argument('-O', '--override-toolchain', type=str, help="Override host toochain to use for sandbox (e.g. 'clang-7')") - parser.add_option('-Q', '--quick', action='store_true', + parser.add_argument('-Q', '--quick', action='store_true', default=False, help='Do a rough build, with limited warning resolution') - parser.add_option('-p', '--full-path', action='store_true', + parser.add_argument('-p', '--full-path', action='store_true', default=False, help="Use full toolchain path in CROSS_COMPILE") - parser.add_option('-P', '--per-board-out-dir', action='store_true', + parser.add_argument('-P', '--per-board-out-dir', action='store_true', default=False, help="Use an O= (output) directory per board rather than per thread") - parser.add_option('-r', '--reproducible-builds', action='store_true', + parser.add_argument('-r', '--reproducible-builds', action='store_true', help='Set SOURCE_DATE_EPOCH=0 to suuport a reproducible build') - parser.add_option('-R', '--regen-board-list', type='string', + parser.add_argument('-R', '--regen-board-list', type=str, help='Force regeneration of the list of boards, like the old boards.cfg file') - parser.add_option('-s', '--summary', action='store_true', + parser.add_argument('-s', '--summary', action='store_true', default=False, help='Show a build summary') - parser.add_option('-S', '--show-sizes', action='store_true', + parser.add_argument('-S', '--show-sizes', action='store_true', default=False, help='Show image size variation in summary') - parser.add_option('--step', type='int', + parser.add_argument('--step', type=int, default=1, help='Only build every n commits (0=just first and last)') if HAS_TESTS: - parser.add_option('--skip-net-tests', action='store_true', default=False, + parser.add_argument('--skip-net-tests', action='store_true', default=False, help='Skip tests which need the network') - parser.add_option('-t', '--test', action='store_true', dest='test', + parser.add_argument('-t', '--test', action='store_true', dest='test', default=False, help='run tests') - parser.add_option('-T', '--threads', type='int', + parser.add_argument('-T', '--threads', type=int, default=None, help='Number of builder threads to use (0=single-thread)') - parser.add_option('-u', '--show_unknown', action='store_true', + parser.add_argument('-u', '--show_unknown', action='store_true', default=False, help='Show boards with unknown build result') - parser.add_option('-U', '--show-environment', action='store_true', + parser.add_argument('-U', '--show-environment', action='store_true', default=False, help='Show environment changes in summary') - parser.add_option('-v', '--verbose', action='store_true', + parser.add_argument('-v', '--verbose', action='store_true', default=False, help='Show build results while the build progresses') - parser.add_option('-V', '--verbose-build', action='store_true', + parser.add_argument('-V', '--verbose-build', action='store_true', default=False, help='Run make with V=1, logging all output') - parser.add_option('-w', '--work-in-output', action='store_true', + parser.add_argument('-w', '--work-in-output', action='store_true', default=False, help='Use the output directory as the work directory') - parser.add_option('-W', '--ignore-warnings', action='store_true', + parser.add_argument('-W', '--ignore-warnings', action='store_true', default=False, help='Return success even if there are warnings') - parser.add_option('-x', '--exclude', dest='exclude', - type='string', action='append', + parser.add_argument('-x', '--exclude', dest='exclude', + type=str, action='append', help='Specify a list of boards to exclude, separated by comma') - parser.add_option('-y', '--filter-dtb-warnings', action='store_true', + parser.add_argument('-y', '--filter-dtb-warnings', action='store_true', default=False, help='Filter out device-tree-compiler warnings from output') - parser.add_option('-Y', '--filter-migration-warnings', action='store_true', + parser.add_argument('-Y', '--filter-migration-warnings', action='store_true', default=False, help='Filter out migration warnings from output') - - parser.usage += """ [list of target/arch/cpu/board/vendor/soc to build] - - Build U-Boot for all commits in a branch. Use -n to do a dry run""" + parser.add_argument('terms', type=str, nargs='*', + help='Board / SoC names to build') return parser.parse_args() diff --git a/tools/buildman/control.py b/tools/buildman/control.py index 346a40f26e9e..fa0fd616fe2d 100644 --- a/tools/buildman/control.py +++ b/tools/buildman/control.py @@ -421,35 +421,35 @@ def determine_boards(brds, args, col, opt_boards, exclude): return selected, why_selected, board_warnings -def adjust_options(options, series, selected): - """Adjust options according to various constraints +def adjust_args(args, series, selected): + """Adjust arguments according to various constraints Updates verbose, show_errors, threads, jobs and step Args: - options (Options): Options object to adjust + args (Namespace): Namespace object to adjust series (Series): Series being built / summarised selected (list of Board): List of Board objects that are marked """ - if not series and not options.dry_run: - options.verbose = True - if not options.summary: - options.show_errors = True + if not series and not args.dry_run: + args.verbose = True + if not args.summary: + args.show_errors = True # By default we have one thread per CPU. But if there are not enough jobs # we can have fewer threads and use a high '-j' value for make. - if options.threads is None: - options.threads = min(multiprocessing.cpu_count(), len(selected)) - if not options.jobs: - options.jobs = max(1, (multiprocessing.cpu_count() + + if args.threads is None: + args.threads = min(multiprocessing.cpu_count(), len(selected)) + if not args.jobs: + args.jobs = max(1, (multiprocessing.cpu_count() + len(selected) - 1) // len(selected)) - if not options.step: - options.step = len(series.commits) - 1 + if not args.step: + args.step = len(series.commits) - 1 # We can't show function sizes without board details at present - if options.show_bloat: - options.show_detail = True + if args.show_bloat: + args.show_detail = True def setup_output_dir(output_dir, work_in_output, branch, no_subdirs, col, @@ -481,7 +481,7 @@ def setup_output_dir(output_dir, work_in_output, branch, no_subdirs, col, return output_dir -def run_builder(builder, commits, board_selected, options): +def run_builder(builder, commits, board_selected, args): """Run the builder or show the summary Args: @@ -489,37 +489,37 @@ def run_builder(builder, commits, board_selected, options): boards_selected (dict): Dict of selected boards: key: target name value: Board object - options (Options): Options to use + args (Namespace): Namespace to use Returns: int: Return code for buildman """ - gnu_make = command.output(os.path.join(options.git, + gnu_make = command.output(os.path.join(args.git, 'scripts/show-gnu-make'), raise_on_error=False).rstrip() if not gnu_make: sys.exit('GNU Make not found') builder.gnu_make = gnu_make - if not options.ide: - commit_count = count_build_commits(commits, options.step) - tprint(get_action_summary(options.summary, commit_count, board_selected, - options.threads, options.jobs)) + if not args.ide: + commit_count = count_build_commits(commits, args.step) + tprint(get_action_summary(args.summary, commit_count, board_selected, + args.threads, args.jobs)) builder.SetDisplayOptions( - options.show_errors, options.show_sizes, options.show_detail, - options.show_bloat, options.list_error_boards, options.show_config, - options.show_environment, options.filter_dtb_warnings, - options.filter_migration_warnings, options.ide) - if options.summary: + args.show_errors, args.show_sizes, args.show_detail, + args.show_bloat, args.list_error_boards, args.show_config, + args.show_environment, args.filter_dtb_warnings, + args.filter_migration_warnings, args.ide) + if args.summary: builder.ShowSummary(commits, board_selected) else: fail, warned, excs = builder.BuildBoards( - commits, board_selected, options.keep_outputs, options.verbose) + commits, board_selected, args.keep_outputs, args.verbose) if excs: return 102 if fail: return 100 - if warned and not options.ignore_warnings: + if warned and not args.ignore_warnings: return 101 return 0 @@ -548,12 +548,12 @@ def calc_adjust_cfg(adjust_cfg, reproducible_builds): return adjust_cfg -def do_buildman(options, args, toolchains=None, make_func=None, brds=None, +def do_buildman(args, toolchains=None, make_func=None, brds=None, clean_dir=False, test_thread_exceptions=False): """The main control code for buildman Args: - options: Command line options object + args: ArgumentParser object args: Command line arguments (list of strings) toolchains: Toolchains to use - this should be a Toolchains() object. If None, then it will be created and scanned @@ -575,67 +575,67 @@ def do_buildman(options, args, toolchains=None, make_func=None, brds=None, gitutil.setup() col = terminal.Color() - git_dir = os.path.join(options.git, '.git') + git_dir = os.path.join(args.git, '.git') - toolchains = get_toolchains(toolchains, col, options.override_toolchain, - options.fetch_arch, options.list_tool_chains, - options.verbose) + toolchains = get_toolchains(toolchains, col, args.override_toolchain, + args.fetch_arch, args.list_tool_chains, + args.verbose) output_dir = setup_output_dir( - options.output_dir, options.work_in_output, options.branch, - options.no_subdirs, col, clean_dir) + args.output_dir, args.work_in_output, args.branch, + args.no_subdirs, col, clean_dir) # Work out what subset of the boards we are building if not brds: - brds = get_boards_obj(output_dir, options.regen_board_list, - options.threads, options.verbose) + brds = get_boards_obj(output_dir, args.regen_board_list, + args.threads, args.verbose) if isinstance(brds, int): return brds - if options.print_prefix: + if args.print_prefix: show_toolchain_prefix(brds, toolchains) return 0 selected, why_selected, board_warnings = determine_boards( - brds, args, col, options.boards, options.exclude) + brds, args.terms, col, args.boards, args.exclude) - series = determine_series(selected, col, git_dir, options.count, - options.branch, options.work_in_output) + series = determine_series(selected, col, git_dir, args.count, + args.branch, args.work_in_output) - adjust_options(options, series, selected) + adjust_args(args, series, selected) # For a dry run, just show our actions as a sanity check - if options.dry_run: + if args.dry_run: show_actions(series, why_selected, selected, output_dir, board_warnings, - options.step, options.threads, options.jobs, - options.verbose) + args.step, args.threads, args.jobs, + args.verbose) return 0 - # Create a new builder with the selected options + # Create a new builder with the selected args builder = Builder(toolchains, output_dir, git_dir, - options.threads, options.jobs, checkout=True, - show_unknown=options.show_unknown, step=options.step, - no_subdirs=options.no_subdirs, full_path=options.full_path, - verbose_build=options.verbose_build, - mrproper=options.mrproper, - per_board_out_dir=options.per_board_out_dir, - config_only=options.config_only, - squash_config_y=not options.preserve_config_y, - warnings_as_errors=options.warnings_as_errors, - work_in_output=options.work_in_output, + args.threads, args.jobs, checkout=True, + show_unknown=args.show_unknown, step=args.step, + no_subdirs=args.no_subdirs, full_path=args.full_path, + verbose_build=args.verbose_build, + mrproper=args.mrproper, + per_board_out_dir=args.per_board_out_dir, + config_only=args.config_only, + squash_config_y=not args.preserve_config_y, + warnings_as_errors=args.warnings_as_errors, + work_in_output=args.work_in_output, test_thread_exceptions=test_thread_exceptions, - adjust_cfg=calc_adjust_cfg(options.adjust_cfg, - options.reproducible_builds), - allow_missing=get_allow_missing(options.allow_missing, - options.no_allow_missing, - len(selected), options.branch), - no_lto=options.no_lto, - reproducible_builds=options.reproducible_builds, - force_build = options.force_build, - force_build_failures = options.force_build_failures, - force_reconfig = options.force_reconfig, in_tree = options.in_tree, - force_config_on_failure=not options.quick, make_func=make_func) + adjust_cfg=calc_adjust_cfg(args.adjust_cfg, + args.reproducible_builds), + allow_missing=get_allow_missing(args.allow_missing, + args.no_allow_missing, + len(selected), args.branch), + no_lto=args.no_lto, + reproducible_builds=args.reproducible_builds, + force_build = args.force_build, + force_build_failures = args.force_build_failures, + force_reconfig = args.force_reconfig, in_tree = args.in_tree, + force_config_on_failure=not args.quick, make_func=make_func) TEST_BUILDER = builder return run_builder(builder, series.commits if series else None, - brds.get_selected_dict(), options) + brds.get_selected_dict(), args) diff --git a/tools/buildman/func_test.py b/tools/buildman/func_test.py index 966ad178a902..5cfe99bf5f6b 100644 --- a/tools/buildman/func_test.py +++ b/tools/buildman/func_test.py @@ -244,12 +244,12 @@ class TestFunctional(unittest.TestCase): result code from buildman """ sys.argv = [sys.argv[0]] + list(args) - options, args = cmdline.parse_args() + args = cmdline.parse_args() if brds == False: brds = self._boards result = control.do_buildman( - options, args, toolchains=self._toolchains, - make_func=self._HandleMake, brds=brds, clean_dir=clean_dir, + args, toolchains=self._toolchains, make_func=self._HandleMake, + brds=brds, clean_dir=clean_dir, test_thread_exceptions=test_thread_exceptions) if get_builder: self._builder = control.TEST_BUILDER diff --git a/tools/buildman/main.py b/tools/buildman/main.py index 0f711c8653b3..8ce4d8678abc 100755 --- a/tools/buildman/main.py +++ b/tools/buildman/main.py @@ -40,7 +40,7 @@ def run_tests(skip_net_tests, verbose, args): from buildman import func_test from buildman import test - test_name = args and args[0] or None + test_name = args.terms and args.terms[0] or None if skip_net_tests: test.use_network = False @@ -59,22 +59,22 @@ def run_buildman(): This is the main program. It collects arguments and runs either the tests or the control module. """ - options, args = cmdline.parse_args() + args = cmdline.parse_args() - if not options.debug: + if not args.debug: sys.tracebacklimit = 0 # Run our meagre tests - if cmdline.HAS_TESTS and options.test: - run_tests(options.skip_net_tests, options.verbose, args) + if cmdline.HAS_TESTS and args.test: + run_tests(args.skip_net_tests, args.verbose, args) - elif options.full_help: + elif args.full_help: tools.print_full_help(str(files('buildman').joinpath('README.rst'))) # Build selected commits for selected boards else: - bsettings.Setup(options.config_file) - ret_code = control.do_buildman(options, args) + bsettings.Setup(args.config_file) + ret_code = control.do_buildman(args) sys.exit(ret_code) From patchwork Sun Jul 2 14:26:01 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1802357 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=2a01:238:438b:c500:173d:9f52:ddab:ee01; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.a=rsa-sha256 header.s=google header.b=ReBK5PZC; dkim-atps=neutral Received: from phobos.denx.de (phobos.denx.de [IPv6:2a01:238:438b:c500:173d:9f52:ddab:ee01]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QvBNb6l65z20ZF for ; Mon, 3 Jul 2023 00:32:47 +1000 (AEST) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id 400ED86777; Sun, 2 Jul 2023 16:28:29 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="ReBK5PZC"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id BDF91866C2; Sun, 2 Jul 2023 16:27:46 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.2 Received: from mail-io1-xd29.google.com (mail-io1-xd29.google.com [IPv6:2607:f8b0:4864:20::d29]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id 193FA86609 for ; Sun, 2 Jul 2023 16:27:43 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=sjg@chromium.org Received: by mail-io1-xd29.google.com with SMTP id ca18e2360f4ac-7836164a08aso191764439f.1 for ; Sun, 02 Jul 2023 07:27:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1688308061; x=1690900061; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=VEdsLiW2J27LwM1T9zW2FryXgikAmNtHFKCPe/sbe5o=; b=ReBK5PZCnLvQI3yvYYe8VntlOiRujODcbh36Vo4wiyN+kZsUnHQpKqVJT1PpF6lsq9 L7LtFnG41OGRHDpzUWrtAUKVBHckHpbA5QVioKpikiZpQd5OwAJyjhckW2c/mWqCuz/a N55oNmXLBOlR386bBiLp4/DTiaiYk3ba9w1jM= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1688308061; x=1690900061; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=VEdsLiW2J27LwM1T9zW2FryXgikAmNtHFKCPe/sbe5o=; b=QKZ6Z++v9VrBPQvFY6EdSOjuudPDGBghroeiOYVSOHO/zBG0Qr2WeWvo1Dt2fGzO6M NRaVV7I9wKOS/rVr/W+LyAupouIXNxuYavAwYFGtecS4Ax3mD42RW1sXHuYdFAVYqrE0 gf5bnEK2dbJlm+03ebkERFSobRxZ2xPUxr0AbJZvqsqA6NlTdLeJnWjOnasD0q1VQh8g w0lBl9n7a54k6z6BYimU2uPdvEUbqPoxkR2Dx+GDqxpFupK4nxc7bMh0F3afHzWb8+8z 7OFtGyep6A+gFeoe4Ui+xmO0oLyLFlnnDFp6F80bqMNaievTZ0JhQJ18ZsjApl64CsjB ifvA== X-Gm-Message-State: ABy/qLaez5mOMocAvDddfo7OHzv1H7biduJ0QqwohgeAdANBTwCIigXJ D3HcPfAgjS0UujuEW4PQ8kVfBAU63kb8e9zED40= X-Google-Smtp-Source: APBJJlEZNxHsLgyEILdo7P1Of6wXaTbgScvXopNZFH+VtwRbU+i931gcDAUrw/6dIJVvWsIEmqpEog== X-Received: by 2002:a92:d2c9:0:b0:340:aeac:456d with SMTP id w9-20020a92d2c9000000b00340aeac456dmr7346246ilg.18.1688308061677; Sun, 02 Jul 2023 07:27:41 -0700 (PDT) Received: from sjg1.lan (c-73-14-173-85.hsd1.co.comcast.net. [73.14.173.85]) by smtp.gmail.com with ESMTPSA id c13-20020a02c9cd000000b0042af5268e4asm2934713jap.58.2023.07.02.07.27.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 02 Jul 2023 07:27:41 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Cc: Simon Glass Subject: [PATCH 36/58] buildman: Convert camel case in bsettings.py Date: Sun, 2 Jul 2023 15:26:01 +0100 Message-ID: <20230702142639.1249681-37-sjg@chromium.org> X-Mailer: git-send-email 2.41.0.255.g8b1d071c50-goog In-Reply-To: <20230702142639.1249681-1-sjg@chromium.org> References: <20230702142639.1249681-1-sjg@chromium.org> MIME-Version: 1.0 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.8 at phobos.denx.de X-Virus-Status: Clean Convert this file to snake case and update all files which use it. Signed-off-by: Simon Glass --- tools/buildman/bsettings.py | 14 +++++++------- tools/buildman/control.py | 2 +- tools/buildman/func_test.py | 12 ++++++------ tools/buildman/main.py | 2 +- tools/buildman/test.py | 4 ++-- tools/buildman/toolchain.py | 14 +++++++------- tools/moveconfig.py | 2 +- 7 files changed, 25 insertions(+), 25 deletions(-) diff --git a/tools/buildman/bsettings.py b/tools/buildman/bsettings.py index 0eb894a558c1..612ec0c28464 100644 --- a/tools/buildman/bsettings.py +++ b/tools/buildman/bsettings.py @@ -7,7 +7,7 @@ import io config_fname = None -def Setup(fname=''): +def setup(fname=''): """Set up the buildman settings module by reading config files Args: @@ -23,15 +23,15 @@ def Setup(fname=''): config_fname = '%s/.buildman' % os.getenv('HOME') if not os.path.exists(config_fname): print('No config file found ~/.buildman\nCreating one...\n') - CreateBuildmanConfigFile(config_fname) + create_buildman_config_file(config_fname) print('To install tool chains, please use the --fetch-arch option') if config_fname: settings.read(config_fname) -def AddFile(data): +def add_file(data): settings.readfp(io.StringIO(data)) -def GetItems(section): +def get_items(section): """Get the items from a section of the config. Args: @@ -47,7 +47,7 @@ def GetItems(section): except: raise -def GetGlobalItemValue(name): +def get_global_item_value(name): """Get an item from the 'global' section of the config. Args: @@ -58,7 +58,7 @@ def GetGlobalItemValue(name): """ return settings.get('global', name, fallback=None) -def SetItem(section, tag, value): +def set_item(section, tag, value): """Set an item and write it back to the settings file""" global settings global config_fname @@ -68,7 +68,7 @@ def SetItem(section, tag, value): with open(config_fname, 'w') as fd: settings.write(fd) -def CreateBuildmanConfigFile(config_fname): +def create_buildman_config_file(config_fname): """Creates a new config file with no tool chain information. Args: diff --git a/tools/buildman/control.py b/tools/buildman/control.py index fa0fd616fe2d..d51b5fb0ad4a 100644 --- a/tools/buildman/control.py +++ b/tools/buildman/control.py @@ -157,7 +157,7 @@ def get_allow_missing(opt_allow, opt_no_allow, num_selected, has_branch): external blobs are used """ allow_missing = False - am_setting = bsettings.GetGlobalItemValue('allow-missing') + am_setting = bsettings.get_global_item_value('allow-missing') if am_setting: if am_setting == 'always': allow_missing = True diff --git a/tools/buildman/func_test.py b/tools/buildman/func_test.py index 5cfe99bf5f6b..5646b59f162d 100644 --- a/tools/buildman/func_test.py +++ b/tools/buildman/func_test.py @@ -184,8 +184,8 @@ class TestFunctional(unittest.TestCase): self._buildman_pathname = sys.argv[0] self._buildman_dir = os.path.dirname(os.path.realpath(sys.argv[0])) command.test_result = self._HandleCommand - bsettings.Setup(None) - bsettings.AddFile(settings_data) + bsettings.setup(None) + bsettings.add_file(settings_data) self.setupToolchains() self._toolchains.Add('arm-gcc', test=False) self._toolchains.Add('powerpc-gcc', test=False) @@ -691,7 +691,7 @@ Some images are invalid''' def testBlobSettingsAlways(self): """Test the 'always' policy""" - bsettings.SetItem('global', 'allow-missing', 'always') + bsettings.set_item('global', 'allow-missing', 'always') self.assertEqual(True, control.get_allow_missing(False, False, 1, False)) self.assertEqual(False, @@ -699,7 +699,7 @@ Some images are invalid''' def testBlobSettingsBranch(self): """Test the 'branch' policy""" - bsettings.SetItem('global', 'allow-missing', 'branch') + bsettings.set_item('global', 'allow-missing', 'branch') self.assertEqual(False, control.get_allow_missing(False, False, 1, False)) self.assertEqual(True, @@ -709,7 +709,7 @@ Some images are invalid''' def testBlobSettingsMultiple(self): """Test the 'multiple' policy""" - bsettings.SetItem('global', 'allow-missing', 'multiple') + bsettings.set_item('global', 'allow-missing', 'multiple') self.assertEqual(False, control.get_allow_missing(False, False, 1, False)) self.assertEqual(True, @@ -719,7 +719,7 @@ Some images are invalid''' def testBlobSettingsBranchMultiple(self): """Test the 'branch multiple' policy""" - bsettings.SetItem('global', 'allow-missing', 'branch multiple') + bsettings.set_item('global', 'allow-missing', 'branch multiple') self.assertEqual(False, control.get_allow_missing(False, False, 1, False)) self.assertEqual(True, diff --git a/tools/buildman/main.py b/tools/buildman/main.py index 8ce4d8678abc..593911353101 100755 --- a/tools/buildman/main.py +++ b/tools/buildman/main.py @@ -73,7 +73,7 @@ def run_buildman(): # Build selected commits for selected boards else: - bsettings.Setup(args.config_file) + bsettings.setup(args.config_file) ret_code = control.do_buildman(args) sys.exit(ret_code) diff --git a/tools/buildman/test.py b/tools/buildman/test.py index 7eb25aa80eba..3f55035ea2e4 100644 --- a/tools/buildman/test.py +++ b/tools/buildman/test.py @@ -138,8 +138,8 @@ class TestBuild(unittest.TestCase): self.brds.select_boards([]) # Add some test settings - bsettings.Setup(None) - bsettings.AddFile(settings_data) + bsettings.setup(None) + bsettings.add_file(settings_data) # Set up the toolchains self.toolchains = toolchain.Toolchains() diff --git a/tools/buildman/toolchain.py b/tools/buildman/toolchain.py index 0ecd8458b912..57bf6149ca61 100644 --- a/tools/buildman/toolchain.py +++ b/tools/buildman/toolchain.py @@ -139,7 +139,7 @@ class Toolchain: """Get toolchain wrapper from the setting file. """ value = '' - for name, value in bsettings.GetItems('toolchain-wrapper'): + for name, value in bsettings.get_items('toolchain-wrapper'): if not value: print("Warning: Wrapper not found") if value: @@ -249,7 +249,7 @@ class Toolchains: self.prefixes = {} self.paths = [] self.override_toolchain = override_toolchain - self._make_flags = dict(bsettings.GetItems('make-flags')) + self._make_flags = dict(bsettings.get_items('make-flags')) def GetPathList(self, show_warning=True): """Get a list of available toolchain paths @@ -261,7 +261,7 @@ class Toolchains: List of strings, each a path to a toolchain mentioned in the [toolchain] section of the settings file. """ - toolchains = bsettings.GetItems('toolchain') + toolchains = bsettings.get_items('toolchain') if show_warning and not toolchains: print(("Warning: No tool chains. Please run 'buildman " "--fetch-arch all' to download all available toolchains, or " @@ -283,7 +283,7 @@ class Toolchains: Args: show_warning: True to show a warning if there are no tool chains. """ - self.prefixes = bsettings.GetItems('toolchain-prefix') + self.prefixes = bsettings.get_items('toolchain-prefix') self.paths += self.GetPathList(show_warning) def Add(self, fname, test=True, verbose=False, priority=PRIORITY_CALC, @@ -399,7 +399,7 @@ class Toolchains: returns: toolchain object, or None if none found """ - for tag, value in bsettings.GetItems('toolchain-alias'): + for tag, value in bsettings.get_items('toolchain-alias'): if arch == tag: for alias in value.split(): if alias in self.toolchains: @@ -421,7 +421,7 @@ class Toolchains: Returns: Resolved string - >>> bsettings.Setup(None) + >>> bsettings.setup(None) >>> tcs = Toolchains() >>> tcs.Add('fred', False) >>> var_dict = {'oblique' : 'OBLIQUE', 'first' : 'fi${second}rst', \ @@ -598,5 +598,5 @@ class Toolchains: if not self.TestSettingsHasPath(dirpath): print(("Adding 'download' to config file '%s'" % bsettings.config_fname)) - bsettings.SetItem('toolchain', 'download', '%s/*/*' % dest) + bsettings.set_item('toolchain', 'download', '%s/*/*' % dest) return 0 diff --git a/tools/moveconfig.py b/tools/moveconfig.py index c4d72ede3684..6cbecc3d5c80 100755 --- a/tools/moveconfig.py +++ b/tools/moveconfig.py @@ -2037,7 +2037,7 @@ doc/develop/moveconfig.rst for documentation.''' if not args.cleanup_headers_only: check_clean_directory() - bsettings.Setup('') + bsettings.setup('') toolchains = toolchain.Toolchains() toolchains.GetSettings() toolchains.Scan(verbose=False) From patchwork Sun Jul 2 14:26:02 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1802360 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=2a01:238:438b:c500:173d:9f52:ddab:ee01; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.a=rsa-sha256 header.s=google header.b=DMZzq90l; dkim-atps=neutral Received: from phobos.denx.de (phobos.denx.de [IPv6:2a01:238:438b:c500:173d:9f52:ddab:ee01]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QvBP93pMWz20ZF for ; Mon, 3 Jul 2023 00:33:17 +1000 (AEST) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id 0931A867C2; Sun, 2 Jul 2023 16:28:34 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="DMZzq90l"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id 194DB8675B; Sun, 2 Jul 2023 16:27:50 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.2 Received: from mail-io1-xd2b.google.com (mail-io1-xd2b.google.com [IPv6:2607:f8b0:4864:20::d2b]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id 4E5D8865D5 for ; Sun, 2 Jul 2023 16:27:44 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=sjg@chromium.org Received: by mail-io1-xd2b.google.com with SMTP id ca18e2360f4ac-777a6ebb542so147267739f.0 for ; Sun, 02 Jul 2023 07:27:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1688308063; x=1690900063; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=JCe/HMgBMPLbD6aDynNXb0mqb6cz/H0vovbv1F5503s=; b=DMZzq90lsrgAMDDt7GBLH6S69bgBuWy/c+izzKJ1RcDHWokA3FccbR96SGAH0jgwRl in/rRNjy9T/YRsXwHORu+xZJXDAsjcRL4jM1JjQMAWw2vawgDDtO4xcBM60UQBHQhMhV LcVivaHY2QOdhk0cgF4mwitCqR5pn3FwBbKhI= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1688308063; x=1690900063; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=JCe/HMgBMPLbD6aDynNXb0mqb6cz/H0vovbv1F5503s=; b=Pp+Z0SaEgMNmpxosr+IgA9DOq46ATQ2/2AlbM4h8AZFHW+y3+olWqHqAfRbOy0xpsg wmyOLv4wCTFRLaJSdEM02SP4hM5EL0lxCW5EN0YbQDQnp+uyCuz8ZHscTqkxoaNZs9mz nrNJHNB8zjk1qiBBq9+Z4/ifr6wIHZ7b7oiCE8pSI4o8rWhcGY2B+z7xN2Ufj63GPGYX IcEEY4RBpIPzj6xzt2fevO/MUVnzmKhP9eqPHzCvAmLsHI68GLdLht26DScrubLDrcWL D8REYQ8E30ONxdt9y/mHZSWqWw2ExXMXxjjc/9RtrxQMan2egVmIa0b8X6umlrKLP9HL HkPA== X-Gm-Message-State: AC+VfDw0QEWaWrANMzeCA/ssGiLR3U0bHJxLS1z58mqi4sUO9z0EfyoG ukMGWqRNZnEiaQjY862njPWeocTyQMfUzjiZ9Rs= X-Google-Smtp-Source: ACHHUZ6Obj7MjBZ01RtVKTJdVUAPh+eTmCs0pm/Ujmds6yjtp3alMisoshxSnvSYhm6h13UwrdkwCA== X-Received: by 2002:a6b:fc10:0:b0:785:ccac:a4db with SMTP id r16-20020a6bfc10000000b00785ccaca4dbmr8018137ioh.2.1688308062376; Sun, 02 Jul 2023 07:27:42 -0700 (PDT) Received: from sjg1.lan (c-73-14-173-85.hsd1.co.comcast.net. [73.14.173.85]) by smtp.gmail.com with ESMTPSA id c13-20020a02c9cd000000b0042af5268e4asm2934713jap.58.2023.07.02.07.27.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 02 Jul 2023 07:27:42 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Cc: Simon Glass Subject: [PATCH 37/58] buildman: Convert camel case in builder.py Date: Sun, 2 Jul 2023 15:26:02 +0100 Message-ID: <20230702142639.1249681-38-sjg@chromium.org> X-Mailer: git-send-email 2.41.0.255.g8b1d071c50-goog In-Reply-To: <20230702142639.1249681-1-sjg@chromium.org> References: <20230702142639.1249681-1-sjg@chromium.org> MIME-Version: 1.0 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.8 at phobos.denx.de X-Virus-Status: Clean Convert this file to snake case and update all files which use it. Signed-off-by: Simon Glass --- tools/buildman/builder.py | 228 ++++++++++++++++---------------- tools/buildman/builderthread.py | 26 ++-- tools/buildman/control.py | 13 +- tools/buildman/func_test.py | 2 +- tools/buildman/test.py | 22 +-- 5 files changed, 145 insertions(+), 146 deletions(-) diff --git a/tools/buildman/builder.py b/tools/buildman/builder.py index 620b7b8c31a8..80c05fbd1e73 100644 --- a/tools/buildman/builder.py +++ b/tools/buildman/builder.py @@ -134,7 +134,7 @@ class Config: for fname in config_filename: self.config[fname] = {} - def Add(self, fname, key, value): + def add(self, fname, key, value): self.config[fname][key] = value def __hash__(self): @@ -151,7 +151,7 @@ class Environment: self.target = target self.environment = {} - def Add(self, key, value): + def add(self, key, value): self.environment[key] = value class Builder: @@ -315,7 +315,7 @@ class Builder: else: self._working_dir = os.path.join(base_dir, '.bm-work') self.threads = [] - self.do_make = make_func or self.Make + self.do_make = make_func or self.make self.gnu_make = gnu_make self.checkout = checkout self.num_threads = num_threads @@ -401,7 +401,7 @@ class Builder: def signal_handler(self, signal, frame): sys.exit(1) - def SetDisplayOptions(self, show_errors=False, show_sizes=False, + def set_display_options(self, show_errors=False, show_sizes=False, show_detail=False, show_bloat=False, list_error_boards=False, show_config=False, show_environment=False, filter_dtb_warnings=False, @@ -434,7 +434,7 @@ class Builder: self._filter_migration_warnings = filter_migration_warnings self._ide = ide - def _AddTimestamp(self): + def _add_timestamp(self): """Add a new timestamp to the list and record the build period. The build period is the length of time taken to perform a single @@ -463,14 +463,14 @@ class Builder: self._timestamps.popleft() count -= 1 - def SelectCommit(self, commit, checkout=True): + def select_commit(self, commit, checkout=True): """Checkout the selected commit for this build """ self.commit = commit if checkout and self.checkout: gitutil.checkout(commit.hash) - def Make(self, commit, brd, stage, cwd, *args, **kwargs): + def make(self, commit, brd, stage, cwd, *args, **kwargs): """Run make Args: @@ -515,7 +515,7 @@ class Builder: result.combined = '%s\n' % (' '.join(cmd)) + result.combined return result - def ProcessResult(self, result): + def process_result(self, result): """Process the result of a build, showing progress information Args: @@ -536,8 +536,8 @@ class Builder: if self._verbose: terminal.print_clear() boards_selected = {target : result.brd} - self.ResetResultSummary(boards_selected) - self.ProduceResultSummary(result.commit_upto, self.commits, + self.reset_result_summary(boards_selected) + self.produce_result_summary(result.commit_upto, self.commits, boards_selected) else: target = '(starting)' @@ -556,7 +556,7 @@ class Builder: line += ' ' * 8 # Add our current completion time estimate - self._AddTimestamp() + self._add_timestamp() if self._complete_delay: line += '%s : ' % self._complete_delay @@ -565,7 +565,7 @@ class Builder: terminal.print_clear() tprint(line, newline=False, limit_to_line=True) - def _GetOutputDir(self, commit_upto): + def _get_output_dir(self, commit_upto): """Get the name of the output directory for a commit number The output directory is typically ...//. @@ -580,7 +580,7 @@ class Builder: if self.commits: commit = self.commits[commit_upto] subject = commit.subject.translate(trans_valid_chars) - # See _GetOutputSpaceRemovals() which parses this name + # See _get_output_space_removals() which parses this name commit_dir = ('%02d_g%s_%s' % (commit_upto + 1, commit.hash, subject[:20])) elif not self.no_subdirs: @@ -589,7 +589,7 @@ class Builder: return self.base_dir return os.path.join(self.base_dir, commit_dir) - def GetBuildDir(self, commit_upto, target): + def get_build_dir(self, commit_upto, target): """Get the name of the build directory for a commit number The build directory is typically ...///. @@ -598,30 +598,30 @@ class Builder: commit_upto: Commit number to use (0..self.count-1) target: Target name """ - output_dir = self._GetOutputDir(commit_upto) + output_dir = self._get_output_dir(commit_upto) if self.work_in_output: return output_dir return os.path.join(output_dir, target) - def GetDoneFile(self, commit_upto, target): + def get_done_file(self, commit_upto, target): """Get the name of the done file for a commit number Args: commit_upto: Commit number to use (0..self.count-1) target: Target name """ - return os.path.join(self.GetBuildDir(commit_upto, target), 'done') + return os.path.join(self.get_build_dir(commit_upto, target), 'done') - def GetSizesFile(self, commit_upto, target): + def get_sizes_file(self, commit_upto, target): """Get the name of the sizes file for a commit number Args: commit_upto: Commit number to use (0..self.count-1) target: Target name """ - return os.path.join(self.GetBuildDir(commit_upto, target), 'sizes') + return os.path.join(self.get_build_dir(commit_upto, target), 'sizes') - def GetFuncSizesFile(self, commit_upto, target, elf_fname): + def get_func_sizes_file(self, commit_upto, target, elf_fname): """Get the name of the funcsizes file for a commit number and ELF file Args: @@ -629,10 +629,10 @@ class Builder: target: Target name elf_fname: Filename of elf image """ - return os.path.join(self.GetBuildDir(commit_upto, target), + return os.path.join(self.get_build_dir(commit_upto, target), '%s.sizes' % elf_fname.replace('/', '-')) - def GetObjdumpFile(self, commit_upto, target, elf_fname): + def get_objdump_file(self, commit_upto, target, elf_fname): """Get the name of the objdump file for a commit number and ELF file Args: @@ -640,20 +640,20 @@ class Builder: target: Target name elf_fname: Filename of elf image """ - return os.path.join(self.GetBuildDir(commit_upto, target), + return os.path.join(self.get_build_dir(commit_upto, target), '%s.objdump' % elf_fname.replace('/', '-')) - def GetErrFile(self, commit_upto, target): + def get_err_file(self, commit_upto, target): """Get the name of the err file for a commit number Args: commit_upto: Commit number to use (0..self.count-1) target: Target name """ - output_dir = self.GetBuildDir(commit_upto, target) + output_dir = self.get_build_dir(commit_upto, target) return os.path.join(output_dir, 'err') - def FilterErrors(self, lines): + def filter_errors(self, lines): """Filter out errors in which we have no interest We should probably use map(). @@ -676,7 +676,7 @@ class Builder: out_lines.append(line) return out_lines - def ReadFuncSizes(self, fname, fd): + def read_func_sizes(self, fname, fd): """Read function sizes from the output of 'nm' Args: @@ -700,7 +700,7 @@ class Builder: sym[name] = sym.get(name, 0) + int(size, 16) return sym - def _ProcessConfig(self, fname): + def _process_config(self, fname): """Read in a .config, autoconf.mk or autoconf.h file This function handles all config file types. It ignores comments and @@ -737,7 +737,7 @@ class Builder: config[key] = value return config - def _ProcessEnvironment(self, fname): + def _process_environment(self, fname): """Read in a uboot.env file This function reads in environment variables from a file. @@ -762,7 +762,7 @@ class Builder: pass return environment - def GetBuildOutcome(self, commit_upto, target, read_func_sizes, + def get_build_outcome(self, commit_upto, target, read_func_sizes, read_config, read_environment): """Work out the outcome of a build. @@ -776,8 +776,8 @@ class Builder: Returns: Outcome object """ - done_file = self.GetDoneFile(commit_upto, target) - sizes_file = self.GetSizesFile(commit_upto, target) + done_file = self.get_done_file(commit_upto, target) + sizes_file = self.get_sizes_file(commit_upto, target) sizes = {} func_sizes = {} config = {} @@ -791,10 +791,10 @@ class Builder: # Try a rebuild return_code = 1 err_lines = [] - err_file = self.GetErrFile(commit_upto, target) + err_file = self.get_err_file(commit_upto, target) if os.path.exists(err_file): with open(err_file, 'r') as fd: - err_lines = self.FilterErrors(fd.readlines()) + err_lines = self.filter_errors(fd.readlines()) # Decide whether the build was ok, failed or created warnings if return_code: @@ -823,30 +823,30 @@ class Builder: sizes[values[5]] = size_dict if read_func_sizes: - pattern = self.GetFuncSizesFile(commit_upto, target, '*') + pattern = self.get_func_sizes_file(commit_upto, target, '*') for fname in glob.glob(pattern): with open(fname, 'r') as fd: dict_name = os.path.basename(fname).replace('.sizes', '') - func_sizes[dict_name] = self.ReadFuncSizes(fname, fd) + func_sizes[dict_name] = self.read_func_sizes(fname, fd) if read_config: - output_dir = self.GetBuildDir(commit_upto, target) + output_dir = self.get_build_dir(commit_upto, target) for name in self.config_filenames: fname = os.path.join(output_dir, name) - config[name] = self._ProcessConfig(fname) + config[name] = self._process_config(fname) if read_environment: - output_dir = self.GetBuildDir(commit_upto, target) + output_dir = self.get_build_dir(commit_upto, target) fname = os.path.join(output_dir, 'uboot.env') - environment = self._ProcessEnvironment(fname) + environment = self._process_environment(fname) return Builder.Outcome(rc, err_lines, sizes, func_sizes, config, environment) return Builder.Outcome(OUTCOME_UNKNOWN, [], {}, {}, {}, {}) - def GetResultSummary(self, boards_selected, commit_upto, read_func_sizes, + def get_result_summary(self, boards_selected, commit_upto, read_func_sizes, read_config, read_environment): """Calculate a summary of the results of building a commit. @@ -877,7 +877,7 @@ class Builder: key: environment variable value: value of environment variable """ - def AddLine(lines_summary, lines_boards, line, board): + def add_line(lines_summary, lines_boards, line, board): line = line.rstrip() if line in lines_boards: lines_boards[line].append(board) @@ -894,7 +894,7 @@ class Builder: environment = {} for brd in boards_selected.values(): - outcome = self.GetBuildOutcome(commit_upto, brd.target, + outcome = self.get_build_outcome(commit_upto, brd.target, read_func_sizes, read_config, read_environment) board_dict[brd.target] = outcome @@ -911,15 +911,15 @@ class Builder: is_note = self._re_note.match(line) if is_warning or (last_was_warning and is_note): if last_func: - AddLine(warn_lines_summary, warn_lines_boards, + add_line(warn_lines_summary, warn_lines_boards, last_func, brd) - AddLine(warn_lines_summary, warn_lines_boards, + add_line(warn_lines_summary, warn_lines_boards, line, brd) else: if last_func: - AddLine(err_lines_summary, err_lines_boards, + add_line(err_lines_summary, err_lines_boards, last_func, brd) - AddLine(err_lines_summary, err_lines_boards, + add_line(err_lines_summary, err_lines_boards, line, brd) last_was_warning = is_warning last_func = None @@ -927,19 +927,19 @@ class Builder: for fname in self.config_filenames: if outcome.config: for key, value in outcome.config[fname].items(): - tconfig.Add(fname, key, value) + tconfig.add(fname, key, value) config[brd.target] = tconfig tenvironment = Environment(brd.target) if outcome.environment: for key, value in outcome.environment.items(): - tenvironment.Add(key, value) + tenvironment.add(key, value) environment[brd.target] = tenvironment return (board_dict, err_lines_summary, err_lines_boards, warn_lines_summary, warn_lines_boards, config, environment) - def AddOutcome(self, board_dict, arch_list, changes, char, color): + def add_outcome(self, board_dict, arch_list, changes, char, color): """Add an output to our list of outcomes for each architecture This simple function adds failing boards (changes) to the @@ -969,19 +969,19 @@ class Builder: arch_list[arch] += str - def ColourNum(self, num): + def colour_num(self, num): color = self.col.RED if num > 0 else self.col.GREEN if num == 0: return '0' return self.col.build(color, str(num)) - def ResetResultSummary(self, board_selected): + def reset_result_summary(self, board_selected): """Reset the results summary ready for use. Set up the base board list to be all those selected, and set the error lines to empty. - Following this, calls to PrintResultSummary() will use this + Following this, calls to print_result_summary() will use this information to work out what has changed. Args: @@ -998,7 +998,7 @@ class Builder: self._base_config = None self._base_environment = None - def PrintFuncSizeDetail(self, fname, old, new): + def print_func_size_detail(self, fname, old, new): grow, shrink, add, remove, up, down = 0, 0, 0, 0, 0, 0 delta, common = [], {} @@ -1032,7 +1032,7 @@ class Builder: args = [add, -remove, grow, -shrink, up, -down, up - down] if max(args) == 0 and min(args) == 0: return - args = [self.ColourNum(x) for x in args] + args = [self.colour_num(x) for x in args] indent = ' ' * 15 tprint('%s%s: add: %s/%s, grow: %s/%s bytes: %s/%s (%s)' % tuple([indent, self.col.build(self.col.YELLOW, fname)] + args)) @@ -1046,7 +1046,7 @@ class Builder: tprint(msg, colour=color) - def PrintSizeDetail(self, target_list, show_bloat): + def print_size_detail(self, target_list, show_bloat): """Show details size information for each board Args: @@ -1079,12 +1079,12 @@ class Builder: outcome = result['_outcome'] base_outcome = self._base_board_dict[target] for fname in outcome.func_sizes: - self.PrintFuncSizeDetail(fname, + self.print_func_size_detail(fname, base_outcome.func_sizes[fname], outcome.func_sizes[fname]) - def PrintSizeSummary(self, board_selected, board_dict, show_detail, + def print_size_summary(self, board_selected, board_dict, show_detail, show_bloat): """Print a summary of image sizes broken down by section. @@ -1185,10 +1185,10 @@ class Builder: if printed_arch: tprint() if show_detail: - self.PrintSizeDetail(target_list, show_bloat) + self.print_size_detail(target_list, show_bloat) - def PrintResultSummary(self, board_selected, board_dict, err_lines, + def print_result_summary(self, board_selected, board_dict, err_lines, err_line_boards, warn_lines, warn_line_boards, config, environment, show_sizes, show_detail, show_bloat, show_config, show_environment): @@ -1224,7 +1224,7 @@ class Builder: show_config: Show config changes show_environment: Show environment changes """ - def _BoardList(line, line_boards): + def _board_list(line, line_boards): """Helper function to get a line of boards containing a line Args: @@ -1243,7 +1243,7 @@ class Builder: board_set.add(brd) return brds - def _CalcErrorDelta(base_lines, base_line_boards, lines, line_boards, + def _calc_error_delta(base_lines, base_line_boards, lines, line_boards, char): """Calculate the required output based on changes in errors @@ -1267,17 +1267,17 @@ class Builder: worse_lines = [] for line in lines: if line not in base_lines: - errline = ErrLine(char + '+', _BoardList(line, line_boards), + errline = ErrLine(char + '+', _board_list(line, line_boards), line) worse_lines.append(errline) for line in base_lines: if line not in lines: errline = ErrLine(char + '-', - _BoardList(line, base_line_boards), line) + _board_list(line, base_line_boards), line) better_lines.append(errline) return better_lines, worse_lines - def _CalcConfig(delta, name, config): + def _calc_config(delta, name, config): """Calculate configuration changes Args: @@ -1295,7 +1295,7 @@ class Builder: out += '%s=%s ' % (key, config[key]) return '%s %s: %s' % (delta, name, out) - def _AddConfig(lines, name, config_plus, config_minus, config_change): + def _add_config(lines, name, config_plus, config_minus, config_change): """Add changes in configuration to a list Args: @@ -1312,13 +1312,13 @@ class Builder: value: config value """ if config_plus: - lines.append(_CalcConfig('+', name, config_plus)) + lines.append(_calc_config('+', name, config_plus)) if config_minus: - lines.append(_CalcConfig('-', name, config_minus)) + lines.append(_calc_config('-', name, config_minus)) if config_change: - lines.append(_CalcConfig('c', name, config_change)) + lines.append(_calc_config('c', name, config_change)) - def _OutputConfigInfo(lines): + def _output_config_info(lines): for line in lines: if not line: continue @@ -1330,7 +1330,7 @@ class Builder: col = self.col.YELLOW tprint(' ' + line, newline=True, colour=col) - def _OutputErrLines(err_lines, colour): + def _output_err_lines(err_lines, colour): """Output the line of error/warning lines, if not empty Also increments self._error_lines if err_lines not empty @@ -1388,9 +1388,9 @@ class Builder: new_boards.append(target) # Get a list of errors and warnings that have appeared, and disappeared - better_err, worse_err = _CalcErrorDelta(self._base_err_lines, + better_err, worse_err = _calc_error_delta(self._base_err_lines, self._base_err_line_boards, err_lines, err_line_boards, '') - better_warn, worse_warn = _CalcErrorDelta(self._base_warn_lines, + better_warn, worse_warn = _calc_error_delta(self._base_warn_lines, self._base_warn_line_boards, warn_lines, warn_line_boards, 'w') # For the IDE mode, print out all the output @@ -1403,26 +1403,26 @@ class Builder: elif any((ok_boards, warn_boards, err_boards, unknown_boards, new_boards, worse_err, better_err, worse_warn, better_warn)): arch_list = {} - self.AddOutcome(board_selected, arch_list, ok_boards, '', + self.add_outcome(board_selected, arch_list, ok_boards, '', self.col.GREEN) - self.AddOutcome(board_selected, arch_list, warn_boards, 'w+', + self.add_outcome(board_selected, arch_list, warn_boards, 'w+', self.col.YELLOW) - self.AddOutcome(board_selected, arch_list, err_boards, '+', + self.add_outcome(board_selected, arch_list, err_boards, '+', self.col.RED) - self.AddOutcome(board_selected, arch_list, new_boards, '*', self.col.BLUE) + self.add_outcome(board_selected, arch_list, new_boards, '*', self.col.BLUE) if self._show_unknown: - self.AddOutcome(board_selected, arch_list, unknown_boards, '?', + self.add_outcome(board_selected, arch_list, unknown_boards, '?', self.col.MAGENTA) for arch, target_list in arch_list.items(): tprint('%10s: %s' % (arch, target_list)) self._error_lines += 1 - _OutputErrLines(better_err, colour=self.col.GREEN) - _OutputErrLines(worse_err, colour=self.col.RED) - _OutputErrLines(better_warn, colour=self.col.CYAN) - _OutputErrLines(worse_warn, colour=self.col.YELLOW) + _output_err_lines(better_err, colour=self.col.GREEN) + _output_err_lines(worse_err, colour=self.col.RED) + _output_err_lines(better_warn, colour=self.col.CYAN) + _output_err_lines(worse_warn, colour=self.col.YELLOW) if show_sizes: - self.PrintSizeSummary(board_selected, board_dict, show_detail, + self.print_size_summary(board_selected, board_dict, show_detail, show_bloat) if show_environment and self._base_environment: @@ -1450,10 +1450,10 @@ class Builder: desc = '%s -> %s' % (value, new_value) environment_change[key] = desc - _AddConfig(lines, target, environment_plus, environment_minus, + _add_config(lines, target, environment_plus, environment_minus, environment_change) - _OutputConfigInfo(lines) + _output_config_info(lines) if show_config and self._base_config: summary = {} @@ -1516,9 +1516,9 @@ class Builder: arch_config_minus[arch][name].update(config_minus) arch_config_change[arch][name].update(config_change) - _AddConfig(lines, name, config_plus, config_minus, + _add_config(lines, name, config_plus, config_minus, config_change) - _AddConfig(lines, 'all', all_config_plus, all_config_minus, + _add_config(lines, 'all', all_config_plus, all_config_minus, all_config_change) summary[target] = '\n'.join(lines) @@ -1538,20 +1538,20 @@ class Builder: all_plus.update(arch_config_plus[arch][name]) all_minus.update(arch_config_minus[arch][name]) all_change.update(arch_config_change[arch][name]) - _AddConfig(lines, name, arch_config_plus[arch][name], + _add_config(lines, name, arch_config_plus[arch][name], arch_config_minus[arch][name], arch_config_change[arch][name]) - _AddConfig(lines, 'all', all_plus, all_minus, all_change) + _add_config(lines, 'all', all_plus, all_minus, all_change) #arch_summary[target] = '\n'.join(lines) if lines: tprint('%s:' % arch) - _OutputConfigInfo(lines) + _output_config_info(lines) for lines, targets in lines_by_target.items(): if not lines: continue tprint('%s :' % ' '.join(sorted(targets))) - _OutputConfigInfo(lines.split('\n')) + _output_config_info(lines.split('\n')) # Save our updated information for the next call to this function @@ -1572,9 +1572,9 @@ class Builder: tprint("Boards not built (%d): %s" % (len(not_built), ', '.join(not_built))) - def ProduceResultSummary(self, commit_upto, commits, board_selected): + def produce_result_summary(self, commit_upto, commits, board_selected): (board_dict, err_lines, err_line_boards, warn_lines, - warn_line_boards, config, environment) = self.GetResultSummary( + warn_line_boards, config, environment) = self.get_result_summary( board_selected, commit_upto, read_func_sizes=self._show_bloat, read_config=self._show_config, @@ -1583,13 +1583,13 @@ class Builder: msg = '%02d: %s' % (commit_upto + 1, commits[commit_upto].subject) tprint(msg, colour=self.col.BLUE) - self.PrintResultSummary(board_selected, board_dict, + self.print_result_summary(board_selected, board_dict, err_lines if self._show_errors else [], err_line_boards, warn_lines if self._show_errors else [], warn_line_boards, config, environment, self._show_sizes, self._show_detail, self._show_bloat, self._show_config, self._show_environment) - def ShowSummary(self, commits, board_selected): + def show_summary(self, commits, board_selected): """Show a build summary for U-Boot for a given board list. Reset the result summary, then repeatedly call GetResultSummary on @@ -1601,16 +1601,16 @@ class Builder: """ self.commit_count = len(commits) if commits else 1 self.commits = commits - self.ResetResultSummary(board_selected) + self.reset_result_summary(board_selected) self._error_lines = 0 for commit_upto in range(0, self.commit_count, self._step): - self.ProduceResultSummary(commit_upto, commits, board_selected) + self.produce_result_summary(commit_upto, commits, board_selected) if not self._error_lines: tprint('(no errors to report)', colour=self.col.GREEN) - def SetupBuild(self, board_selected, commits): + def setup_build(self, board_selected, commits): """Set up ready to start a build. Args: @@ -1623,7 +1623,7 @@ class Builder: self.upto = self.warned = self.fail = 0 self._timestamps = collections.deque() - def GetThreadDir(self, thread_num): + def get_thread_dir(self, thread_num): """Get the directory path to the working dir for a thread. Args: @@ -1634,7 +1634,7 @@ class Builder: return self._working_dir return os.path.join(self._working_dir, '%02d' % max(thread_num, 0)) - def _PrepareThread(self, thread_num, setup_git): + def _prepare_thread(self, thread_num, setup_git): """Prepare the working directory for a thread. This clones or fetches the repo into the thread's work directory. @@ -1647,7 +1647,7 @@ class Builder: 'clone' to set up a git clone 'worktree' to set up a git worktree """ - thread_dir = self.GetThreadDir(thread_num) + thread_dir = self.get_thread_dir(thread_num) builderthread.Mkdir(thread_dir) git_dir = os.path.join(thread_dir, '.git') @@ -1684,7 +1684,7 @@ class Builder: else: raise ValueError("Can't setup git repo with %s." % setup_git) - def _PrepareWorkingSpace(self, max_threads, setup_git): + def _prepare_working_space(self, max_threads, setup_git): """Prepare the working directory for use. Set up the git repo for each thread. Creates a linked working tree @@ -1710,14 +1710,14 @@ class Builder: # Always do at least one thread for thread in range(max(max_threads, 1)): - self._PrepareThread(thread, setup_git) + self._prepare_thread(thread, setup_git) - def _GetOutputSpaceRemovals(self): + def _get_output_space_removals(self): """Get the output directories ready to receive files. Figure out what needs to be deleted in the output directory before it can be used. We only delete old buildman directories which have the - expected name pattern. See _GetOutputDir(). + expected name pattern. See _get_output_dir(). Returns: List of full paths of directories to remove @@ -1726,7 +1726,7 @@ class Builder: return dir_list = [] for commit_upto in range(self.commit_count): - dir_list.append(self._GetOutputDir(commit_upto)) + dir_list.append(self._get_output_dir(commit_upto)) to_remove = [] for dirname in glob.glob(os.path.join(self.base_dir, '*')): @@ -1737,14 +1737,14 @@ class Builder: to_remove.append(dirname) return to_remove - def _PrepareOutputSpace(self): + def _prepare_output_space(self): """Get the output directories ready to receive files. We delete any output directories which look like ones we need to create. Having left over directories is confusing when the user wants to check the output manually. """ - to_remove = self._GetOutputSpaceRemovals() + to_remove = self._get_output_space_removals() if to_remove: tprint('Removing %d old build directories...' % len(to_remove), newline=False) @@ -1752,7 +1752,7 @@ class Builder: shutil.rmtree(dirname) terminal.print_clear() - def BuildBoards(self, commits, board_selected, keep_outputs, verbose): + def build_boards(self, commits, board_selected, keep_outputs, verbose): """Build all commits for a list of boards Args: @@ -1771,15 +1771,15 @@ class Builder: self.commits = commits self._verbose = verbose - self.ResetResultSummary(board_selected) + self.reset_result_summary(board_selected) builderthread.Mkdir(self.base_dir, parents = True) - self._PrepareWorkingSpace(min(self.num_threads, len(board_selected)), + self._prepare_working_space(min(self.num_threads, len(board_selected)), commits is not None) - self._PrepareOutputSpace() + self._prepare_output_space() if not self._ide: tprint('\rStarting build...', newline=False) - self.SetupBuild(board_selected, commits) - self.ProcessResult(None) + self.setup_build(board_selected, commits) + self.process_result(None) self.thread_exceptions = [] # Create jobs to build all commits for each board for brd in board_selected.values(): diff --git a/tools/buildman/builderthread.py b/tools/buildman/builderthread.py index 635865c21c85..5f1200ae890f 100644 --- a/tools/buildman/builderthread.py +++ b/tools/buildman/builderthread.py @@ -77,7 +77,7 @@ class ResultThread(threading.Thread): """ while True: result = self.builder.out_queue.get() - self.builder.ProcessResult(result) + self.builder.process_result(result) self.builder.out_queue.task_done() @@ -176,7 +176,7 @@ class BuilderThread(threading.Thread): out_dir = os.path.join(work_dir, out_rel_dir) # Check if the job was already completed last time - done_file = self.builder.GetDoneFile(commit_upto, brd.target) + done_file = self.builder.get_done_file(commit_upto, brd.target) result.already_done = os.path.exists(done_file) will_build = (force_build or force_build_failures or not result.already_done) @@ -194,7 +194,7 @@ class BuilderThread(threading.Thread): if result.return_code == RETURN_CODE_RETRY: will_build = True elif will_build: - err_file = self.builder.GetErrFile(commit_upto, brd.target) + err_file = self.builder.get_err_file(commit_upto, brd.target) if os.path.exists(err_file) and os.stat(err_file).st_size: result.stderr = 'bad' elif not force_build: @@ -270,7 +270,7 @@ class BuilderThread(threading.Thread): # SPL image. If we don't remove it (i.e. see do_config and # self.mrproper below) then it will appear to be the output of # this build, even if it does not produce SPL images. - build_dir = self.builder.GetBuildDir(commit_upto, brd.target) + build_dir = self.builder.get_build_dir(commit_upto, brd.target) for elf in BASE_ELF_FILENAMES: fname = os.path.join(out_dir, elf) if os.path.exists(fname): @@ -345,9 +345,9 @@ class BuilderThread(threading.Thread): return # Write the output and stderr - output_dir = self.builder._GetOutputDir(result.commit_upto) + output_dir = self.builder._get_output_dir(result.commit_upto) Mkdir(output_dir) - build_dir = self.builder.GetBuildDir(result.commit_upto, + build_dir = self.builder.get_build_dir(result.commit_upto, result.brd.target) Mkdir(build_dir) @@ -356,7 +356,7 @@ class BuilderThread(threading.Thread): if result.stdout: fd.write(result.stdout) - errfile = self.builder.GetErrFile(result.commit_upto, + errfile = self.builder.get_err_file(result.commit_upto, result.brd.target) if result.stderr: with open(errfile, 'w') as fd: @@ -370,7 +370,7 @@ class BuilderThread(threading.Thread): if result.toolchain: # Write the build result and toolchain information. - done_file = self.builder.GetDoneFile(result.commit_upto, + done_file = self.builder.get_done_file(result.commit_upto, result.brd.target) with open(done_file, 'w') as fd: if maybe_aborted: @@ -403,7 +403,7 @@ class BuilderThread(threading.Thread): capture_stderr=True, cwd=result.out_dir, raise_on_error=False, env=env) if nm_result.stdout: - nm = self.builder.GetFuncSizesFile(result.commit_upto, + nm = self.builder.get_func_sizes_file(result.commit_upto, result.brd.target, fname) with open(nm, 'w') as fd: print(nm_result.stdout, end=' ', file=fd) @@ -414,7 +414,7 @@ class BuilderThread(threading.Thread): raise_on_error=False, env=env) rodata_size = '' if dump_result.stdout: - objdump = self.builder.GetObjdumpFile(result.commit_upto, + objdump = self.builder.get_objdump_file(result.commit_upto, result.brd.target, fname) with open(objdump, 'w') as fd: print(dump_result.stdout, end=' ', file=fd) @@ -447,7 +447,7 @@ class BuilderThread(threading.Thread): # adds an additional hex value at the end of each line for the # rodata size if len(lines): - sizes = self.builder.GetSizesFile(result.commit_upto, + sizes = self.builder.get_sizes_file(result.commit_upto, result.brd.target) with open(sizes, 'w') as fd: print('\n'.join(lines), file=fd) @@ -502,7 +502,7 @@ class BuilderThread(threading.Thread): if self.thread_num != -1: self.builder.out_queue.put(result) else: - self.builder.ProcessResult(result) + self.builder.process_result(result) def RunJob(self, job): """Run a single job @@ -516,7 +516,7 @@ class BuilderThread(threading.Thread): List of Result objects """ brd = job.brd - work_dir = self.builder.GetThreadDir(self.thread_num) + work_dir = self.builder.get_thread_dir(self.thread_num) self.toolchain = None if job.commits: # Run 'make board_defconfig' on the first commit diff --git a/tools/buildman/control.py b/tools/buildman/control.py index d51b5fb0ad4a..8706c8ea636f 100644 --- a/tools/buildman/control.py +++ b/tools/buildman/control.py @@ -505,15 +505,14 @@ def run_builder(builder, commits, board_selected, args): tprint(get_action_summary(args.summary, commit_count, board_selected, args.threads, args.jobs)) - builder.SetDisplayOptions( - args.show_errors, args.show_sizes, args.show_detail, - args.show_bloat, args.list_error_boards, args.show_config, - args.show_environment, args.filter_dtb_warnings, - args.filter_migration_warnings, args.ide) + builder.set_display_options( + args.show_errors, args.show_sizes, args.show_detail, args.show_bloat, + args.list_error_boards, args.show_config, args.show_environment, + args.filter_dtb_warnings, args.filter_migration_warnings, args.ide) if args.summary: - builder.ShowSummary(commits, board_selected) + builder.show_summary(commits, board_selected) else: - fail, warned, excs = builder.BuildBoards( + fail, warned, excs = builder.build_boards( commits, board_selected, args.keep_outputs, args.verbose) if excs: return 102 diff --git a/tools/buildman/func_test.py b/tools/buildman/func_test.py index 5646b59f162d..b9f99c76cdb4 100644 --- a/tools/buildman/func_test.py +++ b/tools/buildman/func_test.py @@ -501,7 +501,7 @@ Some images are invalid''' for commit in range(self._commits): for brd in self._boards.get_list(): if brd.arch != 'sandbox': - errfile = self._builder.GetErrFile(commit, brd.target) + errfile = self._builder.get_err_file(commit, brd.target) fd = open(errfile) self.assertEqual(fd.readlines(), ['No tool chain for %s\n' % brd.arch]) diff --git a/tools/buildman/test.py b/tools/buildman/test.py index 3f55035ea2e4..7ac67260c337 100644 --- a/tools/buildman/test.py +++ b/tools/buildman/test.py @@ -208,8 +208,8 @@ class TestBuild(unittest.TestCase): # Build the boards for the pre-defined commits and warnings/errors # associated with each. This calls our Make() to inject the fake output. - build.BuildBoards(self.commits, board_selected, keep_outputs=False, - verbose=False) + build.build_boards(self.commits, board_selected, keep_outputs=False, + verbose=False) lines = terminal.get_print_test_lines() count = 0 for line in lines: @@ -219,8 +219,8 @@ class TestBuild(unittest.TestCase): # We should get two starting messages, an update for every commit built # and a summary message self.assertEqual(count, len(commits) * len(BOARDS) + 3) - build.SetDisplayOptions(**kwdisplay_args); - build.ShowSummary(self.commits, board_selected) + build.set_display_options(**kwdisplay_args); + build.show_summary(self.commits, board_selected) if echo_lines: terminal.echo_print_test_lines() return iter(terminal.get_print_test_lines()) @@ -528,17 +528,17 @@ class TestBuild(unittest.TestCase): 'sandbox']), ({'all': ['board4'], 'sandbox': ['board4']}, [])) def CheckDirs(self, build, dirname): - self.assertEqual('base%s' % dirname, build._GetOutputDir(1)) + self.assertEqual('base%s' % dirname, build._get_output_dir(1)) self.assertEqual('base%s/fred' % dirname, - build.GetBuildDir(1, 'fred')) + build.get_build_dir(1, 'fred')) self.assertEqual('base%s/fred/done' % dirname, - build.GetDoneFile(1, 'fred')) + build.get_done_file(1, 'fred')) self.assertEqual('base%s/fred/u-boot.sizes' % dirname, - build.GetFuncSizesFile(1, 'fred', 'u-boot')) + build.get_func_sizes_file(1, 'fred', 'u-boot')) self.assertEqual('base%s/fred/u-boot.objdump' % dirname, - build.GetObjdumpFile(1, 'fred', 'u-boot')) + build.get_objdump_file(1, 'fred', 'u-boot')) self.assertEqual('base%s/fred/err' % dirname, - build.GetErrFile(1, 'fred')) + build.get_err_file(1, 'fred')) def testOutputDir(self): build = builder.Builder(self.toolchains, BASE_DIR, None, 1, 2, @@ -622,7 +622,7 @@ class TestBuild(unittest.TestCase): build = builder.Builder(self.toolchains, base_dir, None, 1, 2) build.commits = self.commits build.commit_count = len(commits) - result = set(build._GetOutputSpaceRemovals()) + result = set(build._get_output_space_removals()) expected = set([os.path.join(base_dir, f) for f in to_remove]) self.assertEqual(expected, result) From patchwork Sun Jul 2 14:26:03 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1802358 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=85.214.62.61; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.a=rsa-sha256 header.s=google header.b=YPspZgpT; dkim-atps=neutral Received: from phobos.denx.de (phobos.denx.de [85.214.62.61]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QvBNn2NdKz20ZF for ; Mon, 3 Jul 2023 00:32:57 +1000 (AEST) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id 7738F867A7; Sun, 2 Jul 2023 16:28:30 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="YPspZgpT"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id 3B866866CD; Sun, 2 Jul 2023 16:27:47 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.2 Received: from mail-io1-xd2b.google.com (mail-io1-xd2b.google.com [IPv6:2607:f8b0:4864:20::d2b]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id 74DF986668 for ; Sun, 2 Jul 2023 16:27:44 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=sjg@chromium.org Received: by mail-io1-xd2b.google.com with SMTP id ca18e2360f4ac-7836164a08aso191765639f.1 for ; Sun, 02 Jul 2023 07:27:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1688308063; x=1690900063; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=EIDUgejbaFq546nO6pNG5Soa7vwzz7le1GGTc3uBBHI=; b=YPspZgpTQXs6zY0yJF84ICjRsa6uOXle87Ec8WnxB8SnZ1hp5g7k8YOw8yZ1fFFhWe bA3bH338pa5RbtAk+ti/o2jTFm0K9vWT6YLJFP8rxDXolEEBTfUduSM2FIWtoHIYomJI WXzzPsF8QEXvlsznhfxfjkZ8s3/fP+2DIZX20= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1688308063; x=1690900063; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=EIDUgejbaFq546nO6pNG5Soa7vwzz7le1GGTc3uBBHI=; b=ULAzROjC0i4aOA5X0wpph6aAyitdFa3NX3laOiQNp4pmhkYeMMiwkOitl36puQKuoa 2kcM1H144BAUSEeVR0LLsyku33BnciiWsSmHKMfR4mRP6XFFSraRkDpHWYzrXs3NzLOr lcopON+fbbG9mQgOVCkNsHYmktj1OA9ax8F+kL7Xzc3OPghXS9MCIGYXhEheqKH0zEDS g+yWIHuTALgPYbKtmR+tu4eA9Zl24lvIQ8U7PCMkNtmUwUVCRc8ZhnD1wO6T+UjHyQkv uOq8W1ja6WTZMhQXu6W077OFbuGEV/4FReXq1W37PsxBgNMd/jO07xWq3qPdDfcR0iLr djSQ== X-Gm-Message-State: ABy/qLb68VIi/wpuET86OmoVP2RWsn8vH81M3/uexPYElP9Rq4P3Z6X/ jnimf8uqdtZKeuvxBh47tf1Fg45lo0syMr1dnRk= X-Google-Smtp-Source: APBJJlFDBriCBuRKhEH95WsZ5x8FW53kPCypm69zLx64TjC2oEIM1eRABo4TyDdiQ0aRWUZCMvZYaQ== X-Received: by 2002:a05:6e02:792:b0:345:f017:2371 with SMTP id q18-20020a056e02079200b00345f0172371mr6423048ils.9.1688308063133; Sun, 02 Jul 2023 07:27:43 -0700 (PDT) Received: from sjg1.lan (c-73-14-173-85.hsd1.co.comcast.net. [73.14.173.85]) by smtp.gmail.com with ESMTPSA id c13-20020a02c9cd000000b0042af5268e4asm2934713jap.58.2023.07.02.07.27.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 02 Jul 2023 07:27:42 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Cc: Simon Glass Subject: [PATCH 38/58] buildman: Split parser creation in two Date: Sun, 2 Jul 2023 15:26:03 +0100 Message-ID: <20230702142639.1249681-39-sjg@chromium.org> X-Mailer: git-send-email 2.41.0.255.g8b1d071c50-goog In-Reply-To: <20230702142639.1249681-1-sjg@chromium.org> References: <20230702142639.1249681-1-sjg@chromium.org> MIME-Version: 1.0 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.8 at phobos.denx.de X-Virus-Status: Clean Split this into two functions to avoid a warning about too many statements. Signed-off-by: Simon Glass --- tools/buildman/cmdline.py | 44 +++++++++++++++++++++++++++++---------- 1 file changed, 33 insertions(+), 11 deletions(-) diff --git a/tools/buildman/cmdline.py b/tools/buildman/cmdline.py index 6b9c1db14af7..e295c7aef1a0 100644 --- a/tools/buildman/cmdline.py +++ b/tools/buildman/cmdline.py @@ -14,19 +14,14 @@ import pathlib BUILDMAN_DIR = pathlib.Path(__file__).parent HAS_TESTS = os.path.exists(BUILDMAN_DIR / "test.py") -def parse_args(): - """Parse command line arguments from sys.argv[] - - Returns: - tuple containing: - options: command line options - args: command lin arguments - """ - epilog = """ [list of target/arch/cpu/board/vendor/soc to build] +def add_upto_m(parser): + """Add arguments up to 'M' - Build U-Boot for all commits in a branch. Use -n to do a dry run""" + Args: + parser (ArgumentParser): Parse to add to - parser = argparse.ArgumentParser(epilog=epilog) + This is split out to avoid having too many statements in one function + """ parser.add_argument('-a', '--adjust-cfg', type=str, action='append', help='Adjust the Kconfig settings in .config before building') parser.add_argument('-A', '--print-prefix', action='store_true', @@ -104,6 +99,16 @@ def parse_args(): parser.add_argument('-N', '--no-subdirs', action='store_true', dest='no_subdirs', default=False, help="Don't create subdirectories when building current source for a single board") + + +def add_after_m(parser): + """Add arguments after 'M' + + Args: + parser (ArgumentParser): Parse to add to + + This is split out to avoid having too many statements in one function + """ parser.add_argument('-o', '--output-dir', type=str, dest='output_dir', help='Directory where all builds happen and buildman has its workspace (default is ../)') parser.add_argument('-O', '--override-toolchain', type=str, @@ -153,6 +158,23 @@ def parse_args(): parser.add_argument('-Y', '--filter-migration-warnings', action='store_true', default=False, help='Filter out migration warnings from output') + + +def parse_args(): + """Parse command line arguments from sys.argv[] + + Returns: + tuple containing: + options: command line options + args: command lin arguments + """ + epilog = """ [list of target/arch/cpu/board/vendor/soc to build] + + Build U-Boot for all commits in a branch. Use -n to do a dry run""" + + parser = argparse.ArgumentParser(epilog=epilog) + add_upto_m(parser) + add_after_m(parser) parser.add_argument('terms', type=str, nargs='*', help='Board / SoC names to build') From patchwork Sun Jul 2 14:26:04 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1802363 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=2a01:238:438b:c500:173d:9f52:ddab:ee01; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.a=rsa-sha256 header.s=google header.b=f0kMl5sX; dkim-atps=neutral Received: from phobos.denx.de (phobos.denx.de [IPv6:2a01:238:438b:c500:173d:9f52:ddab:ee01]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QvBPk3H1pz20ZF for ; Mon, 3 Jul 2023 00:33:46 +1000 (AEST) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id DE0A9867BE; Sun, 2 Jul 2023 16:28:38 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="f0kMl5sX"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id 0F960866B3; Sun, 2 Jul 2023 16:27:51 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.2 Received: from mail-il1-x136.google.com (mail-il1-x136.google.com [IPv6:2607:f8b0:4864:20::136]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id 8107886653 for ; Sun, 2 Jul 2023 16:27:45 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=sjg@chromium.org Received: by mail-il1-x136.google.com with SMTP id e9e14a558f8ab-3457a3ada84so19876555ab.1 for ; Sun, 02 Jul 2023 07:27:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1688308064; x=1690900064; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Ya1waf2Xk4UInTh08EGWJ1azt2gjXi8A1FrYX3fEMT4=; b=f0kMl5sXX11hWzq1WMC/qaNiH1iaUsrlLLWrl+Cmg4U4yX8d35Ip1c7bXhTxo5Fscv vksePcBFXFYCgtOnf/pi71VvURbkiaeQ3ph6y/BVSmHNDbORczuNRXzhcWPk1gzkJhVa kO58Q3yUsD1Guqr2fn78HpiqpKgv2/ybVRa5I= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1688308064; x=1690900064; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Ya1waf2Xk4UInTh08EGWJ1azt2gjXi8A1FrYX3fEMT4=; b=JAJw4AXiR1GSGQUDrKR4vfKV55lUn7Ai7pe3ek+RMW9kyFZLI3kibx/bAaP2+wxk5a fsQ1u4OLvSygOpLH1eBGT8ESB+XUGrOWp68rEb+NWbsS5s+y8I8BMj7f2vuB1fQS7P+h O7qEwXOtNi0oNucKSdbDvG7BWPcg8iiSzOMuWdTO0H6NPJwudXkhRDNmDUlIAhJsoZcL AQiJFW3ONHwU1zOrZbYlSrYAAy16BDsjlOYeZB48aHGOYJ0vKx8ukGhhnhhlVBoe7PE7 bQtggk8I0DZ9uDe3jS9WJ8me+AQzZUdMXqEIy3NBB5aoEhIkLKlABQjQ+FeGsr/kqEuv ehvQ== X-Gm-Message-State: ABy/qLa+Z2Ir/rXwdEJCPorFTLxLTQXEi6mu+SZJLRE3th4SUXm4jHhf jn4g/vwqWf+4O7Pq+6meyVUl8HvjgLD45izYYRI= X-Google-Smtp-Source: APBJJlH4vKKL1j/x5GeXXInfJTe/qeeTSOcq5MBKMD7fX6s8osKxCZuMaCbXp9NlzXt9UoIRcfFupw== X-Received: by 2002:a92:c942:0:b0:346:9f6:31a2 with SMTP id i2-20020a92c942000000b0034609f631a2mr1014152ilq.1.1688308064080; Sun, 02 Jul 2023 07:27:44 -0700 (PDT) Received: from sjg1.lan (c-73-14-173-85.hsd1.co.comcast.net. [73.14.173.85]) by smtp.gmail.com with ESMTPSA id c13-20020a02c9cd000000b0042af5268e4asm2934713jap.58.2023.07.02.07.27.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 02 Jul 2023 07:27:43 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Cc: Simon Glass Subject: [PATCH 39/58] buildman: Convert camel case in builderthread.py Date: Sun, 2 Jul 2023 15:26:04 +0100 Message-ID: <20230702142639.1249681-40-sjg@chromium.org> X-Mailer: git-send-email 2.41.0.255.g8b1d071c50-goog In-Reply-To: <20230702142639.1249681-1-sjg@chromium.org> References: <20230702142639.1249681-1-sjg@chromium.org> MIME-Version: 1.0 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.8 at phobos.denx.de X-Virus-Status: Clean Convert this file to snake case and update all files which use it. Signed-off-by: Simon Glass --- tools/buildman/builder.py | 8 +++--- tools/buildman/builderthread.py | 50 ++++++++++++++++----------------- 2 files changed, 29 insertions(+), 29 deletions(-) diff --git a/tools/buildman/builder.py b/tools/buildman/builder.py index 80c05fbd1e73..4cc266ff3049 100644 --- a/tools/buildman/builder.py +++ b/tools/buildman/builder.py @@ -1648,7 +1648,7 @@ class Builder: 'worktree' to set up a git worktree """ thread_dir = self.get_thread_dir(thread_num) - builderthread.Mkdir(thread_dir) + builderthread.mkdir(thread_dir) git_dir = os.path.join(thread_dir, '.git') # Create a worktree or a git repo clone for this thread if it @@ -1696,7 +1696,7 @@ class Builder: work setup_git: True to set up a git worktree or a git clone """ - builderthread.Mkdir(self._working_dir) + builderthread.mkdir(self._working_dir) if setup_git and self.git_dir: src_dir = os.path.abspath(self.git_dir) if gitutil.check_worktree_is_available(src_dir): @@ -1772,7 +1772,7 @@ class Builder: self._verbose = verbose self.reset_result_summary(board_selected) - builderthread.Mkdir(self.base_dir, parents = True) + builderthread.mkdir(self.base_dir, parents = True) self._prepare_working_space(min(self.num_threads, len(board_selected)), commits is not None) self._prepare_output_space() @@ -1793,7 +1793,7 @@ class Builder: if self.num_threads: self.queue.put(job) else: - self._single_builder.RunJob(job) + self._single_builder.run_job(job) if self.num_threads: term = threading.Thread(target=self.queue.join) diff --git a/tools/buildman/builderthread.py b/tools/buildman/builderthread.py index 5f1200ae890f..3f52d95fed9f 100644 --- a/tools/buildman/builderthread.py +++ b/tools/buildman/builderthread.py @@ -16,7 +16,7 @@ from u_boot_pylib import command RETURN_CODE_RETRY = -1 BASE_ELF_FILENAMES = ['u-boot', 'spl/u-boot-spl', 'tpl/u-boot-tpl'] -def Mkdir(dirname, parents = False): +def mkdir(dirname, parents = False): """Make a directory if it doesn't already exist. Args: @@ -108,7 +108,7 @@ class BuilderThread(threading.Thread): self.per_board_out_dir = per_board_out_dir self.test_exception = test_exception - def Make(self, commit, brd, stage, cwd, *args, **kwargs): + def make(self, commit, brd, stage, cwd, *args, **kwargs): """Run 'make' on a particular commit and board. The source code will already be checked out, so the 'commit' @@ -130,7 +130,7 @@ class BuilderThread(threading.Thread): return self.builder.do_make(commit, brd, stage, cwd, *args, **kwargs) - def RunCommit(self, commit_upto, brd, work_dir, do_config, config_only, + def run_commit(self, commit_upto, brd, work_dir, do_config, config_only, force_build, force_build_failures, work_in_output, adjust_cfg): """Build a particular commit. @@ -163,7 +163,7 @@ class BuilderThread(threading.Thread): - boolean indicating whether 'make config' is still needed """ # Create a default result - it will be overwritte by the call to - # self.Make() below, in the event that we do a build. + # self.make() below, in the event that we do a build. result = command.CommandResult() result.return_code = 0 if work_in_output or self.builder.in_tree: @@ -226,7 +226,7 @@ class BuilderThread(threading.Thread): # Set up the environment and command line env = self.toolchain.MakeEnvironment(self.builder.full_path) - Mkdir(out_dir) + mkdir(out_dir) args = [] cwd = work_dir src_dir = os.path.realpath(work_dir) @@ -282,12 +282,12 @@ class BuilderThread(threading.Thread): if do_config or adjust_cfg: config_out = '' if self.mrproper: - result = self.Make(commit, brd, 'mrproper', cwd, + result = self.make(commit, brd, 'mrproper', cwd, 'mrproper', *args, env=env) config_out += result.combined cmd_list.append([self.builder.gnu_make, 'mrproper', *args]) - result = self.Make(commit, brd, 'config', cwd, + result = self.make(commit, brd, 'config', cwd, *(args + config_args), env=env) cmd_list.append([self.builder.gnu_make] + args + config_args) @@ -298,7 +298,7 @@ class BuilderThread(threading.Thread): if result.return_code == 0: if config_only: args.append('cfg') - result = self.Make(commit, brd, 'build', cwd, *args, + result = self.make(commit, brd, 'build', cwd, *args, env=env) cmd_list.append([self.builder.gnu_make] + args) if (result.return_code == 2 and @@ -326,7 +326,7 @@ class BuilderThread(threading.Thread): result.out_dir = out_dir return result, do_config - def _WriteResult(self, result, keep_outputs, work_in_output): + def _write_result(self, result, keep_outputs, work_in_output): """Write a built result to the output directory. Args: @@ -346,10 +346,10 @@ class BuilderThread(threading.Thread): # Write the output and stderr output_dir = self.builder._get_output_dir(result.commit_upto) - Mkdir(output_dir) + mkdir(output_dir) build_dir = self.builder.get_build_dir(result.commit_upto, result.brd.target) - Mkdir(build_dir) + mkdir(build_dir) outfile = os.path.join(build_dir, 'log') with open(outfile, 'w') as fd: @@ -440,7 +440,7 @@ class BuilderThread(threading.Thread): raise_on_error=False, env=env) ubootenv = os.path.join(result.out_dir, 'uboot.env') if not work_in_output: - self.CopyFiles(result.out_dir, build_dir, '', ['uboot.env']) + self.copy_files(result.out_dir, build_dir, '', ['uboot.env']) # Write out the image sizes file. This is similar to the output # of binutil's 'size' utility, but it omits the header line and @@ -455,7 +455,7 @@ class BuilderThread(threading.Thread): if not work_in_output: # Write out the configuration files, with a special case for SPL for dirname in ['', 'spl', 'tpl']: - self.CopyFiles( + self.copy_files( result.out_dir, build_dir, dirname, ['u-boot.cfg', 'spl/u-boot-spl.cfg', 'tpl/u-boot-tpl.cfg', '.config', 'include/autoconf.mk', @@ -463,12 +463,12 @@ class BuilderThread(threading.Thread): # Now write the actual build output if keep_outputs: - self.CopyFiles( + self.copy_files( result.out_dir, build_dir, '', ['u-boot*', '*.bin', '*.map', '*.img', 'MLO', 'SPL', 'include/autoconf.mk', 'spl/u-boot-spl*']) - def CopyFiles(self, out_dir, build_dir, dirname, patterns): + def copy_files(self, out_dir, build_dir, dirname, patterns): """Copy files from the build directory to the output. Args: @@ -488,7 +488,7 @@ class BuilderThread(threading.Thread): target = '%s-%s%s' % (base, dirname, ext) shutil.copy(fname, os.path.join(build_dir, target)) - def _SendResult(self, result): + def _send_result(self, result): """Send a result to the builder for processing Args: @@ -504,7 +504,7 @@ class BuilderThread(threading.Thread): else: self.builder.process_result(result) - def RunJob(self, job): + def run_job(self, job): """Run a single job A job consists of a building a list of commits for a particular board. @@ -524,7 +524,7 @@ class BuilderThread(threading.Thread): commit_upto = 0 force_build = False for commit_upto in range(0, len(job.commits), job.step): - result, request_config = self.RunCommit(commit_upto, brd, + result, request_config = self.run_commit(commit_upto, brd, work_dir, do_config, self.builder.config_only, force_build or self.builder.force_build, self.builder.force_build_failures, @@ -535,7 +535,7 @@ class BuilderThread(threading.Thread): # If our incremental build failed, try building again # with a reconfig. if self.builder.force_config_on_failure: - result, request_config = self.RunCommit(commit_upto, + result, request_config = self.run_commit(commit_upto, brd, work_dir, True, False, True, False, job.work_in_output, job.adjust_cfg) did_config = True @@ -576,17 +576,17 @@ class BuilderThread(threading.Thread): raise ValueError('Interrupt') # We have the build results, so output the result - self._WriteResult(result, job.keep_outputs, job.work_in_output) - self._SendResult(result) + self._write_result(result, job.keep_outputs, job.work_in_output) + self._send_result(result) else: # Just build the currently checked-out build - result, request_config = self.RunCommit(None, brd, work_dir, True, + result, request_config = self.run_commit(None, brd, work_dir, True, self.builder.config_only, True, self.builder.force_build_failures, job.work_in_output, job.adjust_cfg) result.commit_upto = 0 - self._WriteResult(result, job.keep_outputs, job.work_in_output) - self._SendResult(result) + self._write_result(result, job.keep_outputs, job.work_in_output) + self._send_result(result) def run(self): """Our thread's run function @@ -597,7 +597,7 @@ class BuilderThread(threading.Thread): while True: job = self.builder.queue.get() try: - self.RunJob(job) + self.run_job(job) except Exception as e: print('Thread exception (use -T0 to run without threads):', e) self.builder.thread_exceptions.append(e) From patchwork Sun Jul 2 14:26:05 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1802361 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=2a01:238:438b:c500:173d:9f52:ddab:ee01; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.a=rsa-sha256 header.s=google header.b=Us0hd6Di; dkim-atps=neutral Received: from phobos.denx.de (phobos.denx.de [IPv6:2a01:238:438b:c500:173d:9f52:ddab:ee01]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QvBPN1XvVz20ZF for ; Mon, 3 Jul 2023 00:33:28 +1000 (AEST) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id 699B0867A5; Sun, 2 Jul 2023 16:28:35 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="Us0hd6Di"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id 4C8AA865D5; Sun, 2 Jul 2023 16:27:50 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.2 Received: from mail-io1-xd2b.google.com (mail-io1-xd2b.google.com [IPv6:2607:f8b0:4864:20::d2b]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id C318F86609 for ; Sun, 2 Jul 2023 16:27:46 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=sjg@chromium.org Received: by mail-io1-xd2b.google.com with SMTP id ca18e2360f4ac-777a4c8e8f4so191552839f.3 for ; Sun, 02 Jul 2023 07:27:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1688308065; x=1690900065; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=9dCJ/Y6l6hcAXNTl7ccEL5DEsXRV2rh/vkCKFkZiY3c=; b=Us0hd6DiGX9b5Z6M28Nxjy55yqux1KMhMWzrd7hL8g5e3mYp+4BN2pfXc/7Qw3yn34 G+bTIVgzyvx2zrarN0n/qlQLkhz6gw0+kcvumj3PsPpwT1FHiuhf8wtTjmSQ/70rMbHX 6RsJ/bAuAbXJcdc+jNTHU/qBTrYP0bF149KMc= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1688308065; x=1690900065; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=9dCJ/Y6l6hcAXNTl7ccEL5DEsXRV2rh/vkCKFkZiY3c=; b=j0Vig1uNBmsUffrnlvm5WXNkf7iJnpuVI1F43cREkNEF1BBR+QcOlm7TmQzblf3sMk z42D9SINOqwlPNfFAyr+xOmIwzq1gsrT9OoOWZ6+RT5dIyxjVCCnBNiNwkMtTIeMi4PD iutYJqwnIHTh1v+t69N0ZsujjW0mgMztQTxnxgb0tRr/xhwAe5vtsyKILt4p9dhb41FK vwRp0V/8d7IwqRT5sPxch4kFatVoZi4LfliAagU+IWQIy3iXWDKSU3v2QqnP5SsSI4Xu guO4JZnGrXkqPsYEeh0TSlx7NZCV5rYeNnmUZ3/ObnNOBubGTTCIXwkKL/WHQhbYwe1E YY1g== X-Gm-Message-State: AC+VfDy9foD353/2qW2LkB1O1E69JOcMdaN7w7JGhZt4PPrIrUb2XKY8 idfCClhjta57qCpTNqKrl/BYA4/XY91fgRm9zJc= X-Google-Smtp-Source: ACHHUZ6WJxnDXpqkIJrWi/EQw5DDwPUP19an8Tm6Mupelmf1R4CC1J5Sqka0hUGsruWSR/LrE/HV0A== X-Received: by 2002:a6b:7b4b:0:b0:783:6c18:1e9e with SMTP id m11-20020a6b7b4b000000b007836c181e9emr9698131iop.12.1688308065299; Sun, 02 Jul 2023 07:27:45 -0700 (PDT) Received: from sjg1.lan (c-73-14-173-85.hsd1.co.comcast.net. [73.14.173.85]) by smtp.gmail.com with ESMTPSA id c13-20020a02c9cd000000b0042af5268e4asm2934713jap.58.2023.07.02.07.27.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 02 Jul 2023 07:27:45 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Cc: Simon Glass Subject: [PATCH 40/58] buildman: Correct most pylint warnings in builderthread Date: Sun, 2 Jul 2023 15:26:05 +0100 Message-ID: <20230702142639.1249681-41-sjg@chromium.org> X-Mailer: git-send-email 2.41.0.255.g8b1d071c50-goog In-Reply-To: <20230702142639.1249681-1-sjg@chromium.org> References: <20230702142639.1249681-1-sjg@chromium.org> MIME-Version: 1.0 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.8 at phobos.denx.de X-Virus-Status: Clean Fix the easy warnings in this file. Signed-off-by: Simon Glass --- tools/buildman/builderthread.py | 95 ++++++++++++++++++--------------- 1 file changed, 51 insertions(+), 44 deletions(-) diff --git a/tools/buildman/builderthread.py b/tools/buildman/builderthread.py index 3f52d95fed9f..179fc477c008 100644 --- a/tools/buildman/builderthread.py +++ b/tools/buildman/builderthread.py @@ -2,6 +2,12 @@ # Copyright (c) 2014 Google, Inc # +"""Implementation the bulider threads + +This module provides the BuilderThread class, which handles calling the builder +based on the jobs provided. +""" + import errno import glob import os @@ -30,12 +36,12 @@ def mkdir(dirname, parents = False): except OSError as err: if err.errno == errno.EEXIST: if os.path.realpath('.') == os.path.realpath(dirname): - print("Cannot create the current working directory '%s'!" % dirname) + print(f"Cannot create the current working directory '{dirname}'!") sys.exit(1) - pass else: raise +# pylint: disable=R0903 class BuilderJob: """Holds information about a job to be performed by a thread @@ -107,6 +113,7 @@ class BuilderThread(threading.Thread): self.mrproper = mrproper self.per_board_out_dir = per_board_out_dir self.test_exception = test_exception + self.toolchain = None def make(self, commit, brd, stage, cwd, *args, **kwargs): """Run 'make' on a particular commit and board. @@ -182,9 +189,9 @@ class BuilderThread(threading.Thread): not result.already_done) if result.already_done: # Get the return code from that build and use it - with open(done_file, 'r') as fd: + with open(done_file, 'r', encoding='utf-8') as outf: try: - result.return_code = int(fd.readline()) + result.return_code = int(outf.readline()) except ValueError: # The file may be empty due to running out of disk space. # Try a rebuild @@ -240,11 +247,11 @@ class BuilderThread(threading.Thread): # Symlinks can confuse U-Boot's Makefile since # we may use '..' in our path, so remove them. out_dir = os.path.realpath(out_dir) - args.append('O=%s' % out_dir) + args.append(f'O={out_dir}') cwd = None src_dir = os.getcwd() else: - args.append('O=%s' % out_rel_dir) + args.append(f'O={out_rel_dir}') if self.builder.verbose_build: args.append('V=1') else: @@ -260,7 +267,7 @@ class BuilderThread(threading.Thread): args.append('NO_LTO=1') if self.builder.reproducible_builds: args.append('SOURCE_DATE_EPOCH=0') - config_args = ['%s_defconfig' % brd.target] + config_args = [f'{brd.target}_defconfig'] config_out = '' args.extend(self.builder.toolchains.GetMakeArguments(brd)) args.extend(self.toolchain.MakeArgs()) @@ -270,7 +277,6 @@ class BuilderThread(threading.Thread): # SPL image. If we don't remove it (i.e. see do_config and # self.mrproper below) then it will appear to be the output of # this build, even if it does not produce SPL images. - build_dir = self.builder.get_build_dir(commit_upto, brd.target) for elf in BASE_ELF_FILENAMES: fname = os.path.join(out_dir, elf) if os.path.exists(fname): @@ -317,7 +323,7 @@ class BuilderThread(threading.Thread): result.cmd_list = cmd_list else: result.return_code = 1 - result.stderr = 'No tool chain for %s\n' % brd.arch + result.stderr = f'No tool chain for {brd.arch}\n' result.already_done = False result.toolchain = self.toolchain @@ -352,15 +358,15 @@ class BuilderThread(threading.Thread): mkdir(build_dir) outfile = os.path.join(build_dir, 'log') - with open(outfile, 'w') as fd: + with open(outfile, 'w', encoding='utf-8') as outf: if result.stdout: - fd.write(result.stdout) + outf.write(result.stdout) errfile = self.builder.get_err_file(result.commit_upto, result.brd.target) if result.stderr: - with open(errfile, 'w') as fd: - fd.write(result.stderr) + with open(errfile, 'w', encoding='utf-8') as outf: + outf.write(result.stderr) elif os.path.exists(errfile): os.remove(errfile) @@ -372,43 +378,44 @@ class BuilderThread(threading.Thread): # Write the build result and toolchain information. done_file = self.builder.get_done_file(result.commit_upto, result.brd.target) - with open(done_file, 'w') as fd: + with open(done_file, 'w', encoding='utf-8') as outf: if maybe_aborted: # Special code to indicate we need to retry - fd.write('%s' % RETURN_CODE_RETRY) + outf.write(f'{RETURN_CODE_RETRY}') else: - fd.write('%s' % result.return_code) - with open(os.path.join(build_dir, 'toolchain'), 'w') as fd: - print('gcc', result.toolchain.gcc, file=fd) - print('path', result.toolchain.path, file=fd) - print('cross', result.toolchain.cross, file=fd) - print('arch', result.toolchain.arch, file=fd) - fd.write('%s' % result.return_code) + outf.write(f'{result.return_code}') + with open(os.path.join(build_dir, 'toolchain'), 'w', + encoding='utf-8') as outf: + print('gcc', result.toolchain.gcc, file=outf) + print('path', result.toolchain.path, file=outf) + print('cross', result.toolchain.cross, file=outf) + print('arch', result.toolchain.arch, file=outf) + outf.write(f'{result.return_code}') # Write out the image and function size information and an objdump env = result.toolchain.MakeEnvironment(self.builder.full_path) - with open(os.path.join(build_dir, 'out-env'), 'wb') as fd: + with open(os.path.join(build_dir, 'out-env'), 'wb') as outf: for var in sorted(env.keys()): - fd.write(b'%s="%s"' % (var, env[var])) + outf.write(b'%s="%s"' % (var, env[var])) with open(os.path.join(build_dir, 'out-cmd'), 'w', - encoding='utf-8') as fd: + encoding='utf-8') as outf: for cmd in result.cmd_list: - print(' '.join(cmd), file=fd) + print(' '.join(cmd), file=outf) lines = [] for fname in BASE_ELF_FILENAMES: - cmd = ['%snm' % self.toolchain.cross, '--size-sort', fname] + cmd = [f'{self.toolchain.cross}nm', '--size-sort', fname] nm_result = command.run_pipe([cmd], capture=True, capture_stderr=True, cwd=result.out_dir, raise_on_error=False, env=env) if nm_result.stdout: - nm = self.builder.get_func_sizes_file(result.commit_upto, - result.brd.target, fname) - with open(nm, 'w') as fd: - print(nm_result.stdout, end=' ', file=fd) + nm_fname = self.builder.get_func_sizes_file( + result.commit_upto, result.brd.target, fname) + with open(nm_fname, 'w', encoding='utf-8') as outf: + print(nm_result.stdout, end=' ', file=outf) - cmd = ['%sobjdump' % self.toolchain.cross, '-h', fname] + cmd = [f'{self.toolchain.cross}objdump', '-h', fname] dump_result = command.run_pipe([cmd], capture=True, capture_stderr=True, cwd=result.out_dir, raise_on_error=False, env=env) @@ -416,14 +423,14 @@ class BuilderThread(threading.Thread): if dump_result.stdout: objdump = self.builder.get_objdump_file(result.commit_upto, result.brd.target, fname) - with open(objdump, 'w') as fd: - print(dump_result.stdout, end=' ', file=fd) + with open(objdump, 'w', encoding='utf-8') as outf: + print(dump_result.stdout, end=' ', file=outf) for line in dump_result.stdout.splitlines(): fields = line.split() if len(fields) > 5 and fields[1] == '.rodata': rodata_size = fields[2] - cmd = ['%ssize' % self.toolchain.cross, fname] + cmd = [f'{self.toolchain.cross}size', fname] size_result = command.run_pipe([cmd], capture=True, capture_stderr=True, cwd=result.out_dir, raise_on_error=False, env=env) @@ -432,13 +439,12 @@ class BuilderThread(threading.Thread): rodata_size) # Extract the environment from U-Boot and dump it out - cmd = ['%sobjcopy' % self.toolchain.cross, '-O', 'binary', + cmd = [f'{self.toolchain.cross}objcopy', '-O', 'binary', '-j', '.rodata.default_environment', 'env/built-in.o', 'uboot.env'] command.run_pipe([cmd], capture=True, capture_stderr=True, cwd=result.out_dir, raise_on_error=False, env=env) - ubootenv = os.path.join(result.out_dir, 'uboot.env') if not work_in_output: self.copy_files(result.out_dir, build_dir, '', ['uboot.env']) @@ -446,11 +452,11 @@ class BuilderThread(threading.Thread): # of binutil's 'size' utility, but it omits the header line and # adds an additional hex value at the end of each line for the # rodata size - if len(lines): + if lines: sizes = self.builder.get_sizes_file(result.commit_upto, result.brd.target) - with open(sizes, 'w') as fd: - print('\n'.join(lines), file=fd) + with open(sizes, 'w', encoding='utf-8') as outf: + print('\n'.join(lines), file=outf) if not work_in_output: # Write out the configuration files, with a special case for SPL @@ -485,7 +491,7 @@ class BuilderThread(threading.Thread): if dirname: base, ext = os.path.splitext(target) if ext: - target = '%s-%s%s' % (base, dirname, ext) + target = f'{base}-{dirname}{ext}' shutil.copy(fname, os.path.join(build_dir, target)) def _send_result(self, result): @@ -598,7 +604,8 @@ class BuilderThread(threading.Thread): job = self.builder.queue.get() try: self.run_job(job) - except Exception as e: - print('Thread exception (use -T0 to run without threads):', e) - self.builder.thread_exceptions.append(e) + except Exception as exc: + print('Thread exception (use -T0 to run without threads):', + exc) + self.builder.thread_exceptions.append(exc) self.builder.queue.task_done() From patchwork Sun Jul 2 14:26:06 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1802362 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=2a01:238:438b:c500:173d:9f52:ddab:ee01; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.a=rsa-sha256 header.s=google header.b=knyFEUIU; dkim-atps=neutral Received: from phobos.denx.de (phobos.denx.de [IPv6:2a01:238:438b:c500:173d:9f52:ddab:ee01]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QvBPY54J7z20ZF for ; Mon, 3 Jul 2023 00:33:37 +1000 (AEST) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id 54AB9867C0; Sun, 2 Jul 2023 16:28:37 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="knyFEUIU"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id 660F5865F9; Sun, 2 Jul 2023 16:27:50 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.2 Received: from mail-io1-xd2f.google.com (mail-io1-xd2f.google.com [IPv6:2607:f8b0:4864:20::d2f]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id 99015866C2 for ; Sun, 2 Jul 2023 16:27:47 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=sjg@chromium.org Received: by mail-io1-xd2f.google.com with SMTP id ca18e2360f4ac-7837329a00aso140742739f.2 for ; Sun, 02 Jul 2023 07:27:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1688308066; x=1690900066; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=EhxzkjBv8s0V3P1XVv1nVj6SgGqc/5tZiXtKROk2LSE=; b=knyFEUIUatBpq1b3mT6iCJ0nA/9TltSLwqrdMANIKD6d5JX98i8zcG3BYI95Jb3hiB 9RJWPi3SrMssMnx4IDU8wDMPHl24Em1E7omAAQHEwzp8W5p4M5yQRm1kn/WJn7I5rCiD vjueqeuSqML5W2s9yrQCHgHcNOOslEJe1sYxg= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1688308066; x=1690900066; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=EhxzkjBv8s0V3P1XVv1nVj6SgGqc/5tZiXtKROk2LSE=; b=iQN5VgcxiUCSFR4U2JmBoOhvCX9qmv4yypb++ZHtxvD5Z+PPJU51gJxO1WjCVEij/R 3ImaLGbuVihz4P0c/YbOB5HbDryW98OP57PleO83kZuYlbM0VBB0bXWrMxkqYmIimGM4 v1XsThZ66FmBzJIH0zfb6cQkdTKQbS09KKY7kUPb57nHNkA31e3i9/cjE7vPLpyHRfDX vFPiNuA+nxKKTer0sZemYF962/j0bow27b+44pa6IdpcTgZgN78W8Pf/wp1KZxF1QW2I AIZLDAnlG7xW1cva6m2E5HhrztmNUsenUl0tcJEW3SSp4oOSS5Q13SSwf4AlytPhqS4b eNIQ== X-Gm-Message-State: ABy/qLYQHC5QzO8rcX8Awu9du7ua4gjNFqEix6BAxr5Wbd/+AnyVxn7w I7s2f99LpiBOpwGIxdnhjhohi4lV8jlijWGkhpI= X-Google-Smtp-Source: APBJJlEv5jnuEoFFfz23w1P3yZgFTKqM956815fp2ewWLNCLCtLXuxC1l2hqtpAyN+V6eP1DQH9g7g== X-Received: by 2002:a92:ce0c:0:b0:345:a629:83ff with SMTP id b12-20020a92ce0c000000b00345a62983ffmr8297854ilo.17.1688308066223; Sun, 02 Jul 2023 07:27:46 -0700 (PDT) Received: from sjg1.lan (c-73-14-173-85.hsd1.co.comcast.net. [73.14.173.85]) by smtp.gmail.com with ESMTPSA id c13-20020a02c9cd000000b0042af5268e4asm2934713jap.58.2023.07.02.07.27.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 02 Jul 2023 07:27:46 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Cc: Simon Glass Subject: [PATCH 41/58] buildman: Export _get_output_dir() to avoid warnings Date: Sun, 2 Jul 2023 15:26:06 +0100 Message-ID: <20230702142639.1249681-42-sjg@chromium.org> X-Mailer: git-send-email 2.41.0.255.g8b1d071c50-goog In-Reply-To: <20230702142639.1249681-1-sjg@chromium.org> References: <20230702142639.1249681-1-sjg@chromium.org> MIME-Version: 1.0 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.8 at phobos.denx.de X-Virus-Status: Clean Make this a public memory since it is used outside the class. Signed-off-by: Simon Glass --- tools/buildman/builder.py | 8 ++++---- tools/buildman/builderthread.py | 2 +- tools/buildman/test.py | 2 +- 3 files changed, 6 insertions(+), 6 deletions(-) diff --git a/tools/buildman/builder.py b/tools/buildman/builder.py index 4cc266ff3049..ecbd368c47a5 100644 --- a/tools/buildman/builder.py +++ b/tools/buildman/builder.py @@ -565,7 +565,7 @@ class Builder: terminal.print_clear() tprint(line, newline=False, limit_to_line=True) - def _get_output_dir(self, commit_upto): + def get_output_dir(self, commit_upto): """Get the name of the output directory for a commit number The output directory is typically ...//. @@ -598,7 +598,7 @@ class Builder: commit_upto: Commit number to use (0..self.count-1) target: Target name """ - output_dir = self._get_output_dir(commit_upto) + output_dir = self.get_output_dir(commit_upto) if self.work_in_output: return output_dir return os.path.join(output_dir, target) @@ -1717,7 +1717,7 @@ class Builder: Figure out what needs to be deleted in the output directory before it can be used. We only delete old buildman directories which have the - expected name pattern. See _get_output_dir(). + expected name pattern. See get_output_dir(). Returns: List of full paths of directories to remove @@ -1726,7 +1726,7 @@ class Builder: return dir_list = [] for commit_upto in range(self.commit_count): - dir_list.append(self._get_output_dir(commit_upto)) + dir_list.append(self.get_output_dir(commit_upto)) to_remove = [] for dirname in glob.glob(os.path.join(self.base_dir, '*')): diff --git a/tools/buildman/builderthread.py b/tools/buildman/builderthread.py index 179fc477c008..45ae6edf9f48 100644 --- a/tools/buildman/builderthread.py +++ b/tools/buildman/builderthread.py @@ -351,7 +351,7 @@ class BuilderThread(threading.Thread): return # Write the output and stderr - output_dir = self.builder._get_output_dir(result.commit_upto) + output_dir = self.builder.get_output_dir(result.commit_upto) mkdir(output_dir) build_dir = self.builder.get_build_dir(result.commit_upto, result.brd.target) diff --git a/tools/buildman/test.py b/tools/buildman/test.py index 7ac67260c337..bdd3d84158a6 100644 --- a/tools/buildman/test.py +++ b/tools/buildman/test.py @@ -528,7 +528,7 @@ class TestBuild(unittest.TestCase): 'sandbox']), ({'all': ['board4'], 'sandbox': ['board4']}, [])) def CheckDirs(self, build, dirname): - self.assertEqual('base%s' % dirname, build._get_output_dir(1)) + self.assertEqual('base%s' % dirname, build.get_output_dir(1)) self.assertEqual('base%s/fred' % dirname, build.get_build_dir(1, 'fred')) self.assertEqual('base%s/fred/done' % dirname, From patchwork Sun Jul 2 14:26:07 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1802366 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=2a01:238:438b:c500:173d:9f52:ddab:ee01; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.a=rsa-sha256 header.s=google header.b=UEi62PHh; dkim-atps=neutral Received: from phobos.denx.de (phobos.denx.de [IPv6:2a01:238:438b:c500:173d:9f52:ddab:ee01]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QvBQG1fzLz20ZF for ; Mon, 3 Jul 2023 00:34:14 +1000 (AEST) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id 3DA4F867DF; Sun, 2 Jul 2023 16:28:43 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="UEi62PHh"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id 41FB88676D; Sun, 2 Jul 2023 16:27:55 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.2 Received: from mail-il1-x12a.google.com (mail-il1-x12a.google.com [IPv6:2607:f8b0:4864:20::12a]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id 34F90866CD for ; Sun, 2 Jul 2023 16:27:48 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=sjg@chromium.org Received: by mail-il1-x12a.google.com with SMTP id e9e14a558f8ab-345db6c12b4so13773075ab.0 for ; Sun, 02 Jul 2023 07:27:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1688308067; x=1690900067; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Hh59DKxIH5g5H8R013s5JBbXae0Hl71uaSXyDx5JpBI=; b=UEi62PHhIjLWzvfe0+vpgGU2zjZAFdEpVCNxlNU8R95vYEWZWM93L78XOZjzQJhMC8 EOPpD1rgKGBwDnOLd0UsGfmNGanEHCkitli0nuBE14Nt+JthugxajxgcWqUmeB1qWZY7 pzUCNG84/2f8pgw8UThbtdj2nbRZqYCdeDZeI= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1688308067; x=1690900067; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Hh59DKxIH5g5H8R013s5JBbXae0Hl71uaSXyDx5JpBI=; b=YMurfQ1AP1kzJr0WEvnTDNQHp0F36aTQEBNX2S2bUw8cAtvXJaPIqnuvAlfUzkhvlW yLTwQ6kT5F3jh0cxd02i3RTZAWKq3F1CkhFmH14favqZ2JpokSnVejWOIY//lLtf3jzR 2XkpQVWAhWu2EAYs67bTgLy8SBU+JJcfSEfqFs7Zxd/eLMpGxvL02Mnr3ZU9qCUsXrq7 Zc02RoitAqVu6JUv+r/5gb7obemfC6M6mUi+zJ7jDZ4RNxTCEOhBOKuFYMagbWfQYvFl 9nC8sCaBEFyCA8TPjuEeuyw0aU47IlMbWC3udAJAc2WWJVkcBvBsV4XHoIk87MtsVXL0 +1Vg== X-Gm-Message-State: ABy/qLaPrSEohc8dRM/PAQmRyzTPv7T9nbepVMa86bidfom/Ihd5p+bJ 4IcJ7Mz2nH6jH0bSpr7SEdwwsLRvKL9Zc7OAQng= X-Google-Smtp-Source: APBJJlF26xduKyduYnW5309ynNS49raQovY/fw0UPvhz5Vws3hpiEZjejDEdjDUpstteVE5x+PCFwg== X-Received: by 2002:a92:d0c3:0:b0:345:cdc7:1083 with SMTP id y3-20020a92d0c3000000b00345cdc71083mr7567967ila.32.1688308066845; Sun, 02 Jul 2023 07:27:46 -0700 (PDT) Received: from sjg1.lan (c-73-14-173-85.hsd1.co.comcast.net. [73.14.173.85]) by smtp.gmail.com with ESMTPSA id c13-20020a02c9cd000000b0042af5268e4asm2934713jap.58.2023.07.02.07.27.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 02 Jul 2023 07:27:46 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Cc: Simon Glass Subject: [PATCH 42/58] buildman: Correct invalid use of out_dir variable Date: Sun, 2 Jul 2023 15:26:07 +0100 Message-ID: <20230702142639.1249681-43-sjg@chromium.org> X-Mailer: git-send-email 2.41.0.255.g8b1d071c50-goog In-Reply-To: <20230702142639.1249681-1-sjg@chromium.org> References: <20230702142639.1249681-1-sjg@chromium.org> MIME-Version: 1.0 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.8 at phobos.denx.de X-Virus-Status: Clean This variable has a different meaning in the outer scrop. Use a different name to avoid confusion, or bugs. Signed-off-by: Simon Glass --- tools/buildman/builderthread.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tools/buildman/builderthread.py b/tools/buildman/builderthread.py index 45ae6edf9f48..f110137e8f6e 100644 --- a/tools/buildman/builderthread.py +++ b/tools/buildman/builderthread.py @@ -246,8 +246,8 @@ class BuilderThread(threading.Thread): # # Symlinks can confuse U-Boot's Makefile since # we may use '..' in our path, so remove them. - out_dir = os.path.realpath(out_dir) - args.append(f'O={out_dir}') + real_dir = os.path.realpath(out_dir) + args.append(f'O={real_dir}') cwd = None src_dir = os.getcwd() else: From patchwork Sun Jul 2 14:26:08 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1802378 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=2a01:238:438b:c500:173d:9f52:ddab:ee01; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.a=rsa-sha256 header.s=google header.b=nqglaB9Y; dkim-atps=neutral Received: from phobos.denx.de (phobos.denx.de [IPv6:2a01:238:438b:c500:173d:9f52:ddab:ee01]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QvBSM0LsWz242R for ; Mon, 3 Jul 2023 00:36:03 +1000 (AEST) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id C557986812; Sun, 2 Jul 2023 16:29:00 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="nqglaB9Y"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id 7A1A0866CA; Sun, 2 Jul 2023 16:28:07 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE, T_SCC_BODY_TEXT_LINE,T_SPF_TEMPERROR autolearn=ham autolearn_force=no version=3.4.2 Received: from mail-il1-x12b.google.com (mail-il1-x12b.google.com [IPv6:2607:f8b0:4864:20::12b]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id EBCA9866D4 for ; Sun, 2 Jul 2023 16:27:48 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=sjg@chromium.org Received: by mail-il1-x12b.google.com with SMTP id e9e14a558f8ab-345db6c12b4so13773115ab.0 for ; Sun, 02 Jul 2023 07:27:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1688308067; x=1690900067; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=krjhK4SH6BrJweB0L0WMLvZa/97F2suDD94p2dNLnjg=; b=nqglaB9YwTvKYlPzGizgHJVQjTkWDm45cm8N9rHYZkiU/al/X6t1zxMH+2UjvRJGag AFntKStncF8vK4r0CRQ2FB7s6qIOpjBAxnZTefATyEFh8dA5VJ6NKnUd7s6sLA8/FCl5 sttM8hmgI0OXRhIl77PK5nHSg2JQNovQxWhoA= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1688308067; x=1690900067; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=krjhK4SH6BrJweB0L0WMLvZa/97F2suDD94p2dNLnjg=; b=l6zArqaB2W1ubweE3S/BPsJbXhT3/ETtSDKFflZ49jkg+5dm+N57yqS7PRt120wIkO CrlSP3/AWuqC94lJVq7Xfsd4Q5/gxilWk6wBV4uANruv/WV6SRTdPometb4ijfHRYmrM ppBh4q9d1POZkJ/ktCihM0CU7gmmjuXlJb5wRtJW22pNbET45TCoc3/Zb8MW8f+JwG/e Fao+xNFwM72qmnlanoCsV+gswPcSAMVPrHBBhd6+tjzIXUM7vxGWwQiylcIJ4fp1dJWZ koDUVYobeo78kl/0kvWnWUpX5rCiOPslCngwaBoOKNs8JMyfaei624HqoWSQjZ6YmZLX hhqQ== X-Gm-Message-State: AC+VfDywoKmFUs+AMuWwF0tJP3Vd5f721Xxr8SCvpACB2ONECFGvRrJa ZW8NggCqRiBGtgXE9FLFjvMzZtXyFiTRFvdX3cY= X-Google-Smtp-Source: ACHHUZ70kRjpjuZk6om5g1Nba0sTiJhKihuEwl+Fl6H9ohAwwI2ckpRaFrNrfudcy2WDE07D1ufqOg== X-Received: by 2002:a6b:dc05:0:b0:783:31bf:7d8b with SMTP id s5-20020a6bdc05000000b0078331bf7d8bmr9016117ioc.4.1688308067632; Sun, 02 Jul 2023 07:27:47 -0700 (PDT) Received: from sjg1.lan (c-73-14-173-85.hsd1.co.comcast.net. [73.14.173.85]) by smtp.gmail.com with ESMTPSA id c13-20020a02c9cd000000b0042af5268e4asm2934713jap.58.2023.07.02.07.27.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 02 Jul 2023 07:27:47 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Cc: Simon Glass Subject: [PATCH 43/58] buildman: Drop unnecessary assignment of config_out Date: Sun, 2 Jul 2023 15:26:08 +0100 Message-ID: <20230702142639.1249681-44-sjg@chromium.org> X-Mailer: git-send-email 2.41.0.255.g8b1d071c50-goog In-Reply-To: <20230702142639.1249681-1-sjg@chromium.org> References: <20230702142639.1249681-1-sjg@chromium.org> MIME-Version: 1.0 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.8 at phobos.denx.de X-Virus-Status: Clean This is already set up earlier in the function, so drop the extra assignment. Signed-off-by: Simon Glass --- tools/buildman/builderthread.py | 1 - 1 file changed, 1 deletion(-) diff --git a/tools/buildman/builderthread.py b/tools/buildman/builderthread.py index f110137e8f6e..ad12e9ede7f4 100644 --- a/tools/buildman/builderthread.py +++ b/tools/buildman/builderthread.py @@ -286,7 +286,6 @@ class BuilderThread(threading.Thread): cfg_file = os.path.join(out_dir, '.config') cmd_list = [] if do_config or adjust_cfg: - config_out = '' if self.mrproper: result = self.make(commit, brd, 'mrproper', cwd, 'mrproper', *args, env=env) From patchwork Sun Jul 2 14:26:09 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1802372 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=85.214.62.61; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.a=rsa-sha256 header.s=google header.b=HLn9pV0I; dkim-atps=neutral Received: from phobos.denx.de (phobos.denx.de [85.214.62.61]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QvBRJ39N9z20Pj for ; Mon, 3 Jul 2023 00:35:08 +1000 (AEST) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id 26BB9867F9; Sun, 2 Jul 2023 16:28:52 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="HLn9pV0I"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id F039586784; Sun, 2 Jul 2023 16:27:58 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.2 Received: from mail-io1-xd33.google.com (mail-io1-xd33.google.com [IPv6:2607:f8b0:4864:20::d33]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id 15CE386751 for ; Sun, 2 Jul 2023 16:27:50 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=sjg@chromium.org Received: by mail-io1-xd33.google.com with SMTP id ca18e2360f4ac-783544a1c90so191301139f.1 for ; Sun, 02 Jul 2023 07:27:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1688308068; x=1690900068; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=MGN8ONa5Ed3L63Mmx4csUE55Vqza1DCu9qLA3WTJpBg=; b=HLn9pV0IKRG++2a8Jr1Gs5aUsOCDZ9yLvdd/rsAQlladPjPEC93p0/Aum3UQcBzvhD rfKmZNVQuMRdSF7jOw8+HgUUc1Yxw8AaoyE6YUzZ+5LloHzr0UznKqTv24xaXqinpZzU o7Qi9ZcWrdJwodO4JZAjUiqu6ymr5JGOs76vs= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1688308068; x=1690900068; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=MGN8ONa5Ed3L63Mmx4csUE55Vqza1DCu9qLA3WTJpBg=; b=fkNy+/3cCI47akBGlTuknuql1eqiOz/e+lSD6658daZg0vuaxpMVr3rIjAtXkczzpA wumcSOgVKLvlP95qf7aIY8LlEbO+oz+X8+p5pgNg0K9f5vlXudXsO+Vy29ZG48CjGyH4 L4T9iuhd2wVBLlj0E4IQCzFQD0ftDMpkTu9gUWu7KtsbH0l9ZxS6qMsmhdh91EzOP2sq tvS/eqZP/K2mg8ayA4gVqISwlaGU9hSnNEHqSZ3ESoXugPRr/YSKyWPFQNEJnl3J30vO dgDuRRW6moeq+i5Kp1B+1r30Pd+tDIcAvLnYyB+MWgWVS2bRFF7EIQfDK2lkIXYAZ/zZ 6wxQ== X-Gm-Message-State: AC+VfDwVpwV/0LkYaJMgM4wqmH28/V57kPdrpyPxF0zKz5BbkwkoCNhC WHyjUjQynUwNA3F7Bwh8//gOlRrzkRrcHROLIM0= X-Google-Smtp-Source: ACHHUZ6i9lhXLKTe22P7XTdf9ACDRoxWwZC0fi25MqABQLFzeZpAL/COXyTpeVvGQbPrVq62VOTGKw== X-Received: by 2002:a05:6602:2146:b0:780:bf50:32c9 with SMTP id y6-20020a056602214600b00780bf5032c9mr9147645ioy.15.1688308068734; Sun, 02 Jul 2023 07:27:48 -0700 (PDT) Received: from sjg1.lan (c-73-14-173-85.hsd1.co.comcast.net. [73.14.173.85]) by smtp.gmail.com with ESMTPSA id c13-20020a02c9cd000000b0042af5268e4asm2934713jap.58.2023.07.02.07.27.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 02 Jul 2023 07:27:48 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Cc: Simon Glass Subject: [PATCH 44/58] buildman: Start a function to set up the make arguments Date: Sun, 2 Jul 2023 15:26:09 +0100 Message-ID: <20230702142639.1249681-45-sjg@chromium.org> X-Mailer: git-send-email 2.41.0.255.g8b1d071c50-goog In-Reply-To: <20230702142639.1249681-1-sjg@chromium.org> References: <20230702142639.1249681-1-sjg@chromium.org> MIME-Version: 1.0 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.8 at phobos.denx.de X-Virus-Status: Clean Move some of this code into a new funciion, to help reduce the size of the run_commits() function. Signed-off-by: Simon Glass --- tools/buildman/builderthread.py | 38 ++++++++++++++++++++------------- 1 file changed, 23 insertions(+), 15 deletions(-) diff --git a/tools/buildman/builderthread.py b/tools/buildman/builderthread.py index ad12e9ede7f4..47ebf4dcdd90 100644 --- a/tools/buildman/builderthread.py +++ b/tools/buildman/builderthread.py @@ -137,6 +137,28 @@ class BuilderThread(threading.Thread): return self.builder.do_make(commit, brd, stage, cwd, *args, **kwargs) + def _build_args(self, args): + """Set up arguments to the args list based on the settings + + Args: + args (list of str): List of string arguments to add things to + """ + if self.builder.verbose_build: + args.append('V=1') + else: + args.append('-s') + if self.builder.num_jobs is not None: + args.extend(['-j', str(self.builder.num_jobs)]) + if self.builder.warnings_as_errors: + args.append('KCFLAGS=-Werror') + args.append('HOSTCFLAGS=-Werror') + if self.builder.allow_missing: + args.append('BINMAN_ALLOW_MISSING=1') + if self.builder.no_lto: + args.append('NO_LTO=1') + if self.builder.reproducible_builds: + args.append('SOURCE_DATE_EPOCH=0') + def run_commit(self, commit_upto, brd, work_dir, do_config, config_only, force_build, force_build_failures, work_in_output, adjust_cfg): @@ -252,21 +274,7 @@ class BuilderThread(threading.Thread): src_dir = os.getcwd() else: args.append(f'O={out_rel_dir}') - if self.builder.verbose_build: - args.append('V=1') - else: - args.append('-s') - if self.builder.num_jobs is not None: - args.extend(['-j', str(self.builder.num_jobs)]) - if self.builder.warnings_as_errors: - args.append('KCFLAGS=-Werror') - args.append('HOSTCFLAGS=-Werror') - if self.builder.allow_missing: - args.append('BINMAN_ALLOW_MISSING=1') - if self.builder.no_lto: - args.append('NO_LTO=1') - if self.builder.reproducible_builds: - args.append('SOURCE_DATE_EPOCH=0') + self._build_args(args) config_args = [f'{brd.target}_defconfig'] config_out = '' args.extend(self.builder.toolchains.GetMakeArguments(brd)) From patchwork Sun Jul 2 14:26:10 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1802364 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=2a01:238:438b:c500:173d:9f52:ddab:ee01; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.a=rsa-sha256 header.s=google header.b=Dg4fF1mF; dkim-atps=neutral Received: from phobos.denx.de (phobos.denx.de [IPv6:2a01:238:438b:c500:173d:9f52:ddab:ee01]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QvBPw0wcyz20ZF for ; Mon, 3 Jul 2023 00:33:56 +1000 (AEST) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id 1DC03867CB; Sun, 2 Jul 2023 16:28:40 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="Dg4fF1mF"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id 7E82B8676E; Sun, 2 Jul 2023 16:27:53 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.2 Received: from mail-il1-x129.google.com (mail-il1-x129.google.com [IPv6:2607:f8b0:4864:20::129]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id DAC34865F9 for ; Sun, 2 Jul 2023 16:27:50 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=sjg@chromium.org Received: by mail-il1-x129.google.com with SMTP id e9e14a558f8ab-345a4efb66dso14965665ab.3 for ; Sun, 02 Jul 2023 07:27:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1688308069; x=1690900069; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=y65d5YCyAzMraKCLui0Waf7Zv83/jqbK36lK9NihegM=; b=Dg4fF1mF6d+xB9IReMkyaI9oxNLnsbM0+aF/3M+Y/sizP6Ntmyan6LJDKguhMwvryK 422lf6W0+mkSMWcUWWf89TX7s7BZkkgToHYLqsypBPGdG82tEcFG7GuHPndub4pSauwV 2M0UrPhty4GtUh8ji2PyCca50sgtC7NkVu3Ss= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1688308069; x=1690900069; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=y65d5YCyAzMraKCLui0Waf7Zv83/jqbK36lK9NihegM=; b=KQ334lG59HMbzZeFmc7Sm0YzEjlcsI1YqiCXU9UpV2cv4IXJOBQZWx9NbR0gZhLRTT WX481hl9TxZZg3c0a72YigTlcaSobdeyr74mE1v1S1tzxyobTnEbZL4m8VPn6oLTyPYy KVYSfe3br5wa/stRR+qJICTt9QFTOd7kiEKBR19n9AOVl34zSRjqB8OB4ne5c1zI40eD R48GIK4wy+hON+tlvsaHnrgm9UeaxWLsLHZvjklvkY/K43xvMPorV6tBX551dIDTtk/6 n1pLgeeiJ04tky421Pu5YhF0MNfeNobKWvwKREj4fPfdWSGJ82OuFBKw1OkprBtkEWJ9 NWeQ== X-Gm-Message-State: ABy/qLbJ9nAbnX1Ur19Gf6hWoY8uMOiH3JZnMmBZGXjFGPiukKf9W8C5 H+75ytXeT1HIojxnzRAubk39IzqWvQquErXhHvA= X-Google-Smtp-Source: APBJJlGk7CfOLYryOXdWQFkQQrucYjjAlg66elXjt5eQfUDywBcMYqksU8DL6qfX74pIC+SoqV6/Jg== X-Received: by 2002:a05:6e02:108:b0:346:de0:e9c0 with SMTP id t8-20020a056e02010800b003460de0e9c0mr407173ilm.23.1688308069509; Sun, 02 Jul 2023 07:27:49 -0700 (PDT) Received: from sjg1.lan (c-73-14-173-85.hsd1.co.comcast.net. [73.14.173.85]) by smtp.gmail.com with ESMTPSA id c13-20020a02c9cd000000b0042af5268e4asm2934713jap.58.2023.07.02.07.27.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 02 Jul 2023 07:27:49 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Cc: Simon Glass Subject: [PATCH 45/58] buildman: Move setting of toolchain arguments to _build_args() Date: Sun, 2 Jul 2023 15:26:10 +0100 Message-ID: <20230702142639.1249681-46-sjg@chromium.org> X-Mailer: git-send-email 2.41.0.255.g8b1d071c50-goog In-Reply-To: <20230702142639.1249681-1-sjg@chromium.org> References: <20230702142639.1249681-1-sjg@chromium.org> MIME-Version: 1.0 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.8 at phobos.denx.de X-Virus-Status: Clean Move a few more pieces to this new function. Signed-off-by: Simon Glass --- tools/buildman/builderthread.py | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/tools/buildman/builderthread.py b/tools/buildman/builderthread.py index 47ebf4dcdd90..e7d9a29d03ef 100644 --- a/tools/buildman/builderthread.py +++ b/tools/buildman/builderthread.py @@ -137,11 +137,12 @@ class BuilderThread(threading.Thread): return self.builder.do_make(commit, brd, stage, cwd, *args, **kwargs) - def _build_args(self, args): + def _build_args(self, args, brd): """Set up arguments to the args list based on the settings Args: args (list of str): List of string arguments to add things to + brd (Board): Board to create arguments for """ if self.builder.verbose_build: args.append('V=1') @@ -158,6 +159,8 @@ class BuilderThread(threading.Thread): args.append('NO_LTO=1') if self.builder.reproducible_builds: args.append('SOURCE_DATE_EPOCH=0') + args.extend(self.builder.toolchains.GetMakeArguments(brd)) + args.extend(self.toolchain.MakeArgs()) def run_commit(self, commit_upto, brd, work_dir, do_config, config_only, force_build, force_build_failures, work_in_output, @@ -274,11 +277,9 @@ class BuilderThread(threading.Thread): src_dir = os.getcwd() else: args.append(f'O={out_rel_dir}') - self._build_args(args) + self._build_args(args, brd) config_args = [f'{brd.target}_defconfig'] config_out = '' - args.extend(self.builder.toolchains.GetMakeArguments(brd)) - args.extend(self.toolchain.MakeArgs()) # Remove any output targets. Since we use a build directory that # was previously used by another board, it may have produced an From patchwork Sun Jul 2 14:26:11 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1802365 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=85.214.62.61; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.a=rsa-sha256 header.s=google header.b=kOjmB9VN; dkim-atps=neutral Received: from phobos.denx.de (phobos.denx.de [85.214.62.61]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QvBQ54TFmz20ZF for ; Mon, 3 Jul 2023 00:34:05 +1000 (AEST) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id 89BE0867CE; Sun, 2 Jul 2023 16:28:41 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="kOjmB9VN"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id 581D28676D; Sun, 2 Jul 2023 16:27:54 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.2 Received: from mail-il1-x135.google.com (mail-il1-x135.google.com [IPv6:2607:f8b0:4864:20::135]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id 851C6866B3 for ; Sun, 2 Jul 2023 16:27:51 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=sjg@chromium.org Received: by mail-il1-x135.google.com with SMTP id e9e14a558f8ab-3459baa237bso19671315ab.3 for ; Sun, 02 Jul 2023 07:27:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1688308070; x=1690900070; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=pIk8Q9KgugKbnd3oiGcivufI/+Q/xCukmBg5XzsMqeA=; b=kOjmB9VN84vL2sJsXLFJvCNZTBa+9PgrEcRXhOcvyf2WtxdTlEZfI64c9gkJYMV5if DcB7uKeOkb3H0sNXy8b3jDXNemmqr7TWZCYmGM/AEx6iKwDnpWT9dRvARq7h/m8B0+lo TtRaIFzT2o/67QW6KbCmboCY4iBSYHmN/E8dM= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1688308070; x=1690900070; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=pIk8Q9KgugKbnd3oiGcivufI/+Q/xCukmBg5XzsMqeA=; b=FfiAmVofNqBJJ4vT1r6jogSK6YpIL6Ge57YzihwsSYlFFWfGWklluls5GA8fFIiPua Wp+WQ80HKLwUwAWv7CHr6WGQeK/rIYyBf2iFQvOXgRqKE8Nf2rElTlyOyJCymQymkwJ0 +VX26rocDKsSB0a/+pwFYrUstjAlBYbXlztXRfov7rFl8zcYRrq31JTQtE+pVbNWwvzv KlPqEjHdjTYU3/Z21/fan3UK1iIO2Q/1LG6+RuUJKiDa2i62pO+4HPsfkai0ek8qVWW8 9bBFxsjrTkPhUvrx7QVlBFKH3KY6I4m9RDRVeRF/Zb3nvfA7WfsU1cVUp1ijR6T7Zd/9 zBRw== X-Gm-Message-State: ABy/qLbNAtZkCUyG1ZCqPypc2Nm00L/Bvgq0kJs1HUGmk7dDdxSLtKdq Z+ns7kKDSTzYRx1omdmilwBisoor9w1GKmEjdZo= X-Google-Smtp-Source: APBJJlG/FGFUrB9KpjDhVp6vPka6ym/XFBwi8cagDFhJnOTbM0i9Z85JtpPG8dwFHNbd2IYkXw+sWA== X-Received: by 2002:a92:d710:0:b0:345:d326:9c7f with SMTP id m16-20020a92d710000000b00345d3269c7fmr7278340iln.29.1688308070123; Sun, 02 Jul 2023 07:27:50 -0700 (PDT) Received: from sjg1.lan (c-73-14-173-85.hsd1.co.comcast.net. [73.14.173.85]) by smtp.gmail.com with ESMTPSA id c13-20020a02c9cd000000b0042af5268e4asm2934713jap.58.2023.07.02.07.27.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 02 Jul 2023 07:27:49 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Cc: Simon Glass Subject: [PATCH 46/58] buildman: Move more things into _build_args() Date: Sun, 2 Jul 2023 15:26:11 +0100 Message-ID: <20230702142639.1249681-47-sjg@chromium.org> X-Mailer: git-send-email 2.41.0.255.g8b1d071c50-goog In-Reply-To: <20230702142639.1249681-1-sjg@chromium.org> References: <20230702142639.1249681-1-sjg@chromium.org> MIME-Version: 1.0 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.8 at phobos.denx.de X-Virus-Status: Clean Move more of the argument-building code into this function. Fix a missing assignment for out_rel_dir too. Rename the function since it now builds all the arguments. Signed-off-by: Simon Glass --- tools/buildman/builderthread.py | 55 ++++++++++++++++++++------------- 1 file changed, 34 insertions(+), 21 deletions(-) diff --git a/tools/buildman/builderthread.py b/tools/buildman/builderthread.py index e7d9a29d03ef..42af5197dace 100644 --- a/tools/buildman/builderthread.py +++ b/tools/buildman/builderthread.py @@ -137,13 +137,40 @@ class BuilderThread(threading.Thread): return self.builder.do_make(commit, brd, stage, cwd, *args, **kwargs) - def _build_args(self, args, brd): + def _build_args(self, brd, out_dir, out_rel_dir, work_dir, commit_upto): """Set up arguments to the args list based on the settings Args: - args (list of str): List of string arguments to add things to brd (Board): Board to create arguments for + out_dir (str): Path to output directory containing the files + out_rel_dir (str): Output directory relative to the current dir + work_dir (str): Directory to which the source will be checked out + commit_upto (int): Commit number to build (0...n-1) + + Returns: + tuple: + list of str: Arguments to pass to make + str: Current working directory, or None if no commit + str: Source directory (typically the work directory) """ + args = [] + cwd = work_dir + src_dir = os.path.realpath(work_dir) + if not self.builder.in_tree: + if commit_upto is None: + # In this case we are building in the original source directory + # (i.e. the current directory where buildman is invoked. The + # output directory is set to this thread's selected work + # directory. + # + # Symlinks can confuse U-Boot's Makefile since we may use '..' + # in our path, so remove them. + real_dir = os.path.realpath(out_dir) + args.append(f'O={real_dir}') + cwd = None + src_dir = os.getcwd() + else: + args.append(f'O={out_rel_dir}') if self.builder.verbose_build: args.append('V=1') else: @@ -161,6 +188,7 @@ class BuilderThread(threading.Thread): args.append('SOURCE_DATE_EPOCH=0') args.extend(self.builder.toolchains.GetMakeArguments(brd)) args.extend(self.toolchain.MakeArgs()) + return args, cwd, src_dir def run_commit(self, commit_upto, brd, work_dir, do_config, config_only, force_build, force_build_failures, work_in_output, @@ -199,6 +227,7 @@ class BuilderThread(threading.Thread): result = command.CommandResult() result.return_code = 0 if work_in_output or self.builder.in_tree: + out_rel_dir = None out_dir = work_dir else: if self.per_board_out_dir: @@ -259,25 +288,9 @@ class BuilderThread(threading.Thread): # Set up the environment and command line env = self.toolchain.MakeEnvironment(self.builder.full_path) mkdir(out_dir) - args = [] - cwd = work_dir - src_dir = os.path.realpath(work_dir) - if not self.builder.in_tree: - if commit_upto is None: - # In this case we are building in the original source - # directory (i.e. the current directory where buildman - # is invoked. The output directory is set to this - # thread's selected work directory. - # - # Symlinks can confuse U-Boot's Makefile since - # we may use '..' in our path, so remove them. - real_dir = os.path.realpath(out_dir) - args.append(f'O={real_dir}') - cwd = None - src_dir = os.getcwd() - else: - args.append(f'O={out_rel_dir}') - self._build_args(args, brd) + + args, cwd, src_dir = self._build_args(brd, out_dir, out_rel_dir, + work_dir, commit_upto) config_args = [f'{brd.target}_defconfig'] config_out = '' From patchwork Sun Jul 2 14:26:12 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1802367 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=2a01:238:438b:c500:173d:9f52:ddab:ee01; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.a=rsa-sha256 header.s=google header.b=WdY3/fVv; dkim-atps=neutral Received: from phobos.denx.de (phobos.denx.de [IPv6:2a01:238:438b:c500:173d:9f52:ddab:ee01]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QvBQQ4sckz20ZF for ; Mon, 3 Jul 2023 00:34:22 +1000 (AEST) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id D8CC7867D5; Sun, 2 Jul 2023 16:28:44 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="WdY3/fVv"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id DE00786774; Sun, 2 Jul 2023 16:27:55 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.2 Received: from mail-io1-xd36.google.com (mail-io1-xd36.google.com [IPv6:2607:f8b0:4864:20::d36]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id 14BB8866DF for ; Sun, 2 Jul 2023 16:27:52 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=sjg@chromium.org Received: by mail-io1-xd36.google.com with SMTP id ca18e2360f4ac-77acb944bdfso192422839f.0 for ; Sun, 02 Jul 2023 07:27:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1688308070; x=1690900070; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=4uiClRU3vCY5wpZJ7GZfLeevB3M8y8UuRJoW7A2l8cg=; b=WdY3/fVvcga56CgHHbF6BQVLGx01THt76SS2Bg9B4rSmv5AZiB2otVW6pwhGfcL8DX 0+6vqclDvNUXsSWiZPyh10BEm9wGqOyy0iYCTYxX4qCybZq229mnv5jfLJlh5Xpx5hy9 9wLu03DLhUzLIfECM6r5tXsbytTLhvZxKNV50= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1688308070; x=1690900070; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=4uiClRU3vCY5wpZJ7GZfLeevB3M8y8UuRJoW7A2l8cg=; b=P6RkqvEK9s4mGkpbMCydixyQ8E7JfS7tbwC65dSIPsgRph8oRAt0XFQFhLZIl86kFr aEOWa3GcXRzrhcFaThIQ4xCl+vcGhWLs73GAJmLm4VJ04MgG4ARETaUAMIYTbt20mnjW ML2Z9JJrI00JEXQloZHuqJxFW7BWcmpES3Ju2xgpagpRSrwSMqT8E8LTNwPvOdW1EnRX Tsyzasxal45IsbLGkkjaNvgIIu3hz4vL7uN3Roy5esPXsbXCpeE9dWL6Otc7KjiZ9lFl o766aAEGS090LiATR3xThsrM2dX0EB6zbPTInuyMXHLpn+R0jmaOmKf6byCQjLLX8YJc gJaA== X-Gm-Message-State: AC+VfDzgrNAhQUHgaH29068Rxl9K59hTy4W09C6DpxhFS0iAUdcqDSl9 W50cZ+cAzAnX62hzmHIYvQ1WgndzAdTkBUysWT4= X-Google-Smtp-Source: ACHHUZ5qDxA3Q79gEYP5q5rGmbaqh8VX4aqWvJJha7F5FbILC7ATGli+9izKwO2qHgDzlCCc9zSm6w== X-Received: by 2002:a6b:630e:0:b0:780:ce72:ac55 with SMTP id p14-20020a6b630e000000b00780ce72ac55mr9126173iog.10.1688308070728; Sun, 02 Jul 2023 07:27:50 -0700 (PDT) Received: from sjg1.lan (c-73-14-173-85.hsd1.co.comcast.net. [73.14.173.85]) by smtp.gmail.com with ESMTPSA id c13-20020a02c9cd000000b0042af5268e4asm2934713jap.58.2023.07.02.07.27.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 02 Jul 2023 07:27:50 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Cc: Simon Glass Subject: [PATCH 47/58] buildman: Convert config_out to string IO Date: Sun, 2 Jul 2023 15:26:12 +0100 Message-ID: <20230702142639.1249681-48-sjg@chromium.org> X-Mailer: git-send-email 2.41.0.255.g8b1d071c50-goog In-Reply-To: <20230702142639.1249681-1-sjg@chromium.org> References: <20230702142639.1249681-1-sjg@chromium.org> MIME-Version: 1.0 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.8 at phobos.denx.de X-Virus-Status: Clean This is probably a little more efficient and it allows passing the object to another function to write data. Convert config_out to use a string I/O device. Signed-off-by: Simon Glass --- tools/buildman/builderthread.py | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/tools/buildman/builderthread.py b/tools/buildman/builderthread.py index 42af5197dace..9c2938d1b071 100644 --- a/tools/buildman/builderthread.py +++ b/tools/buildman/builderthread.py @@ -10,6 +10,7 @@ based on the jobs provided. import errno import glob +import io import os import shutil import sys @@ -292,7 +293,7 @@ class BuilderThread(threading.Thread): args, cwd, src_dir = self._build_args(brd, out_dir, out_rel_dir, work_dir, commit_upto) config_args = [f'{brd.target}_defconfig'] - config_out = '' + config_out = io.StringIO() # Remove any output targets. Since we use a build directory that # was previously used by another board, it may have produced an @@ -311,14 +312,14 @@ class BuilderThread(threading.Thread): if self.mrproper: result = self.make(commit, brd, 'mrproper', cwd, 'mrproper', *args, env=env) - config_out += result.combined + config_out.write(result.combined) cmd_list.append([self.builder.gnu_make, 'mrproper', *args]) result = self.make(commit, brd, 'config', cwd, *(args + config_args), env=env) cmd_list.append([self.builder.gnu_make] + args + config_args) - config_out += result.combined + config_out.write(result.combined) do_config = False # No need to configure next time if adjust_cfg: cfgutil.adjust_cfg_file(cfg_file, adjust_cfg) @@ -340,7 +341,7 @@ class BuilderThread(threading.Thread): result.return_code = 1 result.stderr = result.stderr.replace(src_dir + '/', '') if self.builder.verbose_build: - result.stdout = config_out + result.stdout + result.stdout = config_out.getvalue() + result.stdout result.cmd_list = cmd_list else: result.return_code = 1 From patchwork Sun Jul 2 14:26:13 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1802368 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=2a01:238:438b:c500:173d:9f52:ddab:ee01; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.a=rsa-sha256 header.s=google header.b=XsVezIVP; dkim-atps=neutral Received: from phobos.denx.de (phobos.denx.de [IPv6:2a01:238:438b:c500:173d:9f52:ddab:ee01]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QvBQb1t2Xz20ZF for ; Mon, 3 Jul 2023 00:34:31 +1000 (AEST) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id 49247867D2; Sun, 2 Jul 2023 16:28:46 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="XsVezIVP"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id 6063C8677D; Sun, 2 Jul 2023 16:27:56 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.2 Received: from mail-io1-xd2b.google.com (mail-io1-xd2b.google.com [IPv6:2607:f8b0:4864:20::d2b]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id 9E43886760 for ; Sun, 2 Jul 2023 16:27:52 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=sjg@chromium.org Received: by mail-io1-xd2b.google.com with SMTP id ca18e2360f4ac-785ccf19489so185823639f.3 for ; Sun, 02 Jul 2023 07:27:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1688308071; x=1690900071; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=7t2ZoWnaPYaRrxX5PkGZ0YRRZoVm7wet0s6Y+QjVhQk=; b=XsVezIVPpNSPGzliSRnplajRbmC43W9VKqPB6FGg/ThWbMgiYBP+F6UCSFwR2wBP5c Q6dFy/RFdaFwNhcwpTj1XPCq9w1UmALYa9F766CVJFmXB9QCXvqhn/U/d9S5uIl62mI/ KHo49y7+th5dEuFfeoHptPb26BWPGY06rwksc= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1688308071; x=1690900071; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=7t2ZoWnaPYaRrxX5PkGZ0YRRZoVm7wet0s6Y+QjVhQk=; b=llRg+XEKmCTQ3hO5HauqmNp3jMnqDriqSLALWL4YlWsWO+MzX0o0JhjviKn8D1Z9GS lKpahZH7loS+vT5VHybYUTfyA9QrogQUwSrW0b+5SLKGv87d1NI9bQlg62FOXlwmpfkP wjjWKsV3sC2kY6TnjgD9i/OZyP8h3168gkKBy8/bTNqes9mbkJcKVxtuGu3fHysnzN95 yOZ6Rg1ADwvKbYIFNf/2nLmDi5qX7tW5F25fBD2ucmxMcMS4mLCx494qbXWmfw2ynwR8 JRCpxFCia55QoKhknOHIlnJIl+uGFpzMJF+K0d0ONz+Sww3uyN3/xxuzNWu+Jw0AEHHR sGsQ== X-Gm-Message-State: AC+VfDzne6GKKno89g0ErlnR/r7gJ0RRpOE80XtjyqG8iA/34fANgFDD uQDhmfyCa35KTRiB+/FfEbpPlcrcFBCRuiQ4B/o= X-Google-Smtp-Source: ACHHUZ7R8gV7FP65zUI9e+KiARdISKaycj2693jiDAit8AhII6m4ecvOdsmjy0eZnrwwXZmtQDcrJQ== X-Received: by 2002:a6b:5b03:0:b0:786:28b4:43a5 with SMTP id v3-20020a6b5b03000000b0078628b443a5mr8632760ioh.4.1688308071325; Sun, 02 Jul 2023 07:27:51 -0700 (PDT) Received: from sjg1.lan (c-73-14-173-85.hsd1.co.comcast.net. [73.14.173.85]) by smtp.gmail.com with ESMTPSA id c13-20020a02c9cd000000b0042af5268e4asm2934713jap.58.2023.07.02.07.27.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 02 Jul 2023 07:27:51 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Cc: Simon Glass Subject: [PATCH 48/58] buildman: Move reconfigure code into its own function Date: Sun, 2 Jul 2023 15:26:13 +0100 Message-ID: <20230702142639.1249681-49-sjg@chromium.org> X-Mailer: git-send-email 2.41.0.255.g8b1d071c50-goog In-Reply-To: <20230702142639.1249681-1-sjg@chromium.org> References: <20230702142639.1249681-1-sjg@chromium.org> MIME-Version: 1.0 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.8 at phobos.denx.de X-Virus-Status: Clean Split this into its own function so reduce the size of run_commit(). Signed-off-by: Simon Glass --- tools/buildman/builderthread.py | 41 ++++++++++++++++++++++++--------- 1 file changed, 30 insertions(+), 11 deletions(-) diff --git a/tools/buildman/builderthread.py b/tools/buildman/builderthread.py index 9c2938d1b071..80fca2a61bb3 100644 --- a/tools/buildman/builderthread.py +++ b/tools/buildman/builderthread.py @@ -191,6 +191,33 @@ class BuilderThread(threading.Thread): args.extend(self.toolchain.MakeArgs()) return args, cwd, src_dir + def _reconfigure(self, commit, brd, cwd, args, env, config_args, config_out, + cmd_list): + """Reconfigure the build + + Args: + commit (Commit): Commit only being built + brd (Board): Board being built + cwd (str): Current working directory + args (list of str): Arguments to pass to make + env (dict): Environment strings + config_args (list of str): defconfig arg for this board + cmd_list (list of str): List to add the commands to, for logging + + Returns: + CommandResult object + """ + if self.mrproper: + result = self.make(commit, brd, 'mrproper', cwd, 'mrproper', *args, + env=env) + config_out.write(result.combined) + cmd_list.append([self.builder.gnu_make, 'mrproper', *args]) + result = self.make(commit, brd, 'config', cwd, *(args + config_args), + env=env) + cmd_list.append([self.builder.gnu_make] + args + config_args) + config_out.write(result.combined) + return result + def run_commit(self, commit_upto, brd, work_dir, do_config, config_only, force_build, force_build_failures, work_in_output, adjust_cfg): @@ -309,17 +336,9 @@ class BuilderThread(threading.Thread): cfg_file = os.path.join(out_dir, '.config') cmd_list = [] if do_config or adjust_cfg: - if self.mrproper: - result = self.make(commit, brd, 'mrproper', cwd, - 'mrproper', *args, env=env) - config_out.write(result.combined) - cmd_list.append([self.builder.gnu_make, 'mrproper', - *args]) - result = self.make(commit, brd, 'config', cwd, - *(args + config_args), env=env) - cmd_list.append([self.builder.gnu_make] + args + - config_args) - config_out.write(result.combined) + result = self._reconfigure( + commit, brd, cwd, args, env, config_args, config_out, + cmd_list) do_config = False # No need to configure next time if adjust_cfg: cfgutil.adjust_cfg_file(cfg_file, adjust_cfg) From patchwork Sun Jul 2 14:26:14 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1802369 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=2a01:238:438b:c500:173d:9f52:ddab:ee01; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.a=rsa-sha256 header.s=google header.b=SVkN8ExQ; dkim-atps=neutral Received: from phobos.denx.de (phobos.denx.de [IPv6:2a01:238:438b:c500:173d:9f52:ddab:ee01]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QvBQm3LL5z20ZF for ; Mon, 3 Jul 2023 00:34:40 +1000 (AEST) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id 99499867E3; Sun, 2 Jul 2023 16:28:47 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="SVkN8ExQ"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id 4519A863B2; Sun, 2 Jul 2023 16:27:57 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.2 Received: from mail-il1-x12d.google.com (mail-il1-x12d.google.com [IPv6:2607:f8b0:4864:20::12d]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id 3D76286763 for ; Sun, 2 Jul 2023 16:27:53 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=sjg@chromium.org Received: by mail-il1-x12d.google.com with SMTP id e9e14a558f8ab-345f50577d3so12012835ab.2 for ; Sun, 02 Jul 2023 07:27:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1688308072; x=1690900072; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=UukLfnuqurZoxvsml+Ou+/BuklL3bokQddLELdJowJ8=; b=SVkN8ExQre+tV3dCBrhlgDa3ZrC1TwrM70jFg1l1ktBf55g8GEa5T8XoGy8D2XAHRP 0ns2X84gukKknOCRFx6DSapd9kxfs7hLBKyEdybuu7FyZNKHFpWizconB/NvF5SVh9Tl k2VNi4dMKD146TsMrs5RKhQ3l6qp9Z6c04MxE= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1688308072; x=1690900072; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=UukLfnuqurZoxvsml+Ou+/BuklL3bokQddLELdJowJ8=; b=bPljFFSanJuyFdW31M1fni0Jyd2ZhQEAzEgnjfFu1awlNwwCyS+X0B4yu5K6WvOCZF Y2XXT9TblI3MjvEgF4enccZEx8uuAONsMwni1lsvFqKLUTh+hISpviU4h97HK1kkV3+x hlV+I3SoXu3fh46QsGOkwYmPHldLb1xKWOc5iFfRDOmCYOFk12EqM3eV8TW1AjFrH0Sx 10otMYzwKYACpzhDE10iWj1yuyDuo4Qn7E/r1R2qjYzyp6Eqje9r4fgAlhplj8SNLvv0 pZNOsw0HjNNxCdFe/AU2SE+cuaUV19MbtnhR60KcjY0bk0s3ad/kTxWjy4wtpFL2ze23 vxvA== X-Gm-Message-State: ABy/qLYQJWLOOx2wvhGSYIU9moRi9eArHi5OF8p1feKptvAEOk4maN3n J5/MtC1/YUtwpsccV3zP081DTR06yJCZQx6+RTA= X-Google-Smtp-Source: APBJJlE2MUFW8EQoGmEjkPiQa44FSRR16iHcngs7hlZ0RRfOrNV8Bng9U8FURhzwGchHrnZQX9nR0Q== X-Received: by 2002:a92:ce4c:0:b0:345:9798:9f7 with SMTP id a12-20020a92ce4c000000b00345979809f7mr7554318ilr.14.1688308071910; Sun, 02 Jul 2023 07:27:51 -0700 (PDT) Received: from sjg1.lan (c-73-14-173-85.hsd1.co.comcast.net. [73.14.173.85]) by smtp.gmail.com with ESMTPSA id c13-20020a02c9cd000000b0042af5268e4asm2934713jap.58.2023.07.02.07.27.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 02 Jul 2023 07:27:51 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Cc: Simon Glass Subject: [PATCH 49/58] buildman: Move bulid code into its own function Date: Sun, 2 Jul 2023 15:26:14 +0100 Message-ID: <20230702142639.1249681-50-sjg@chromium.org> X-Mailer: git-send-email 2.41.0.255.g8b1d071c50-goog In-Reply-To: <20230702142639.1249681-1-sjg@chromium.org> References: <20230702142639.1249681-1-sjg@chromium.org> MIME-Version: 1.0 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.8 at phobos.denx.de X-Virus-Status: Clean Split this into its own function so reduce the size of run_commit(). Signed-off-by: Simon Glass --- tools/buildman/builderthread.py | 40 ++++++++++++++++++++++++--------- 1 file changed, 30 insertions(+), 10 deletions(-) diff --git a/tools/buildman/builderthread.py b/tools/buildman/builderthread.py index 80fca2a61bb3..2d54e6283023 100644 --- a/tools/buildman/builderthread.py +++ b/tools/buildman/builderthread.py @@ -218,6 +218,32 @@ class BuilderThread(threading.Thread): config_out.write(result.combined) return result + def _build(self, commit, brd, cwd, args, env, cmd_list, config_only): + """Perform the build + + Args: + commit (Commit): Commit only being built + brd (Board): Board being built + cwd (str): Current working directory + args (list of str): Arguments to pass to make + env (dict): Environment strings + cmd_list (list of str): List to add the commands to, for logging + config_only (bool): True if this is a config-only build (using the + 'make cfg' target) + + Returns: + CommandResult object + """ + if config_only: + args.append('cfg') + result = self.make(commit, brd, 'build', cwd, *args, env=env) + cmd_list.append([self.builder.gnu_make] + args) + if (result.return_code == 2 and + ('Some images are invalid' in result.stderr)): + # This is handled later by the check for output in stderr + result.return_code = 0 + return result + def run_commit(self, commit_upto, brd, work_dir, do_config, config_only, force_build, force_build_failures, work_in_output, adjust_cfg): @@ -342,17 +368,11 @@ class BuilderThread(threading.Thread): do_config = False # No need to configure next time if adjust_cfg: cfgutil.adjust_cfg_file(cfg_file, adjust_cfg) + + # Now do the build, if everything looks OK if result.return_code == 0: - if config_only: - args.append('cfg') - result = self.make(commit, brd, 'build', cwd, *args, - env=env) - cmd_list.append([self.builder.gnu_make] + args) - if (result.return_code == 2 and - ('Some images are invalid' in result.stderr)): - # This is handled later by the check for output in - # stderr - result.return_code = 0 + result = self._build(commit, brd, cwd, args, env, cmd_list, + config_only) if adjust_cfg: errs = cfgutil.check_cfg_file(cfg_file, adjust_cfg) if errs: From patchwork Sun Jul 2 14:26:15 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1802370 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=85.214.62.61; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.a=rsa-sha256 header.s=google header.b=GwqeDphk; dkim-atps=neutral Received: from phobos.denx.de (phobos.denx.de [85.214.62.61]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QvBQx1G0xz20ZF for ; Mon, 3 Jul 2023 00:34:49 +1000 (AEST) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id 2B857867E7; Sun, 2 Jul 2023 16:28:49 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="GwqeDphk"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id 160998677F; Sun, 2 Jul 2023 16:27:58 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.2 Received: from mail-io1-xd30.google.com (mail-io1-xd30.google.com [IPv6:2607:f8b0:4864:20::d30]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id DB82486768 for ; Sun, 2 Jul 2023 16:27:53 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=sjg@chromium.org Received: by mail-io1-xd30.google.com with SMTP id ca18e2360f4ac-7836164a08aso191772439f.1 for ; Sun, 02 Jul 2023 07:27:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1688308072; x=1690900072; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=N9iNWTR5WVxiGrNxeWswWCVheKolnNfYe+KWHHSJupg=; b=GwqeDphkzgFQjlrq/6DTkvwRRphMdvIdhcJIi2cE/gccHODK0KlTfY2wFSFcip9mxB 0UeOIeiQ+00Rmrzm11RtKqTnQMNrlCrPXY5kjMjZTw1x5nPR96d9ajbk2RM6PIsX639E cbNjDdj+/K4i/6nf/nPoqqJvswrcidMFvcsYE= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1688308072; x=1690900072; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=N9iNWTR5WVxiGrNxeWswWCVheKolnNfYe+KWHHSJupg=; b=OW0Z1Z14juC7h9PAJ0h/P8i/IsF4Mjw9nLmRkMJK0h/m4pn00B+34OXX7oLqI0cqM2 qieBWzj4bvLvjinVSQcHdoVu80F0d/HI1ID9cMaYPEcJdPPtSqcuzoLRgw182VNzeC4D We/aL60h91l7NCB5j0sVTB+vEFJaS29hSM+beDxh/HJpydOn/f4ebmz/qDLfMlSo/c/J TbLAmhwfhIhqdhU8ZhdXljUNR6U8nOJTjqnH2IVHexUSxG+/e1L8S74ydR52b1l8Y4xG XIkdaFSoFFUhTQ+KvjRw0xFs1as88nH3V2xOFrK9feWP8sVRn5iKCmlBJNJj/vw5Txg1 9b8g== X-Gm-Message-State: AC+VfDwpuEUr3OOPUARJz/743GjJIOSUkjIwmBLs/wtPhfglA6oxgPcV eKDiUTASFCZkcVNEkz7eqoETj28JTbKqyvQW/hU= X-Google-Smtp-Source: ACHHUZ6GqlTJW8OY6DvjyERrOf/sgBPI2v9k12t2Z2Ef/HpItiQwAajXIKX10wgCj7a/qWIkbJ6RWA== X-Received: by 2002:a05:6602:276b:b0:786:3f74:c244 with SMTP id l11-20020a056602276b00b007863f74c244mr9397522ioe.17.1688308072537; Sun, 02 Jul 2023 07:27:52 -0700 (PDT) Received: from sjg1.lan (c-73-14-173-85.hsd1.co.comcast.net. [73.14.173.85]) by smtp.gmail.com with ESMTPSA id c13-20020a02c9cd000000b0042af5268e4asm2934713jap.58.2023.07.02.07.27.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 02 Jul 2023 07:27:52 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Cc: Simon Glass Subject: [PATCH 50/58] buildman: Move reading of the done file into a function Date: Sun, 2 Jul 2023 15:26:15 +0100 Message-ID: <20230702142639.1249681-51-sjg@chromium.org> X-Mailer: git-send-email 2.41.0.255.g8b1d071c50-goog In-Reply-To: <20230702142639.1249681-1-sjg@chromium.org> References: <20230702142639.1249681-1-sjg@chromium.org> MIME-Version: 1.0 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.8 at phobos.denx.de X-Virus-Status: Clean Move this logic into its own function to reduce the size of the run_commt() function. Signed-off-by: Simon Glass --- tools/buildman/builderthread.py | 66 +++++++++++++++++++++------------ 1 file changed, 42 insertions(+), 24 deletions(-) diff --git a/tools/buildman/builderthread.py b/tools/buildman/builderthread.py index 2d54e6283023..b4891059b6db 100644 --- a/tools/buildman/builderthread.py +++ b/tools/buildman/builderthread.py @@ -244,6 +244,46 @@ class BuilderThread(threading.Thread): result.return_code = 0 return result + def _read_done_file(self, commit_upto, brd, result, force_build, + force_build_failures): + """Check the 'done' file and see if this commit should be built + + Args: + commit (Commit): Commit only being built + brd (Board): Board being built + result (CommandResult): result object to update + force_build (bool): Force a build even if one was previously done + force_build_failures (bool): Force a bulid if the previous result + showed failure + + Returns: + bool: True if build should be built + """ + done_file = self.builder.get_done_file(commit_upto, brd.target) + result.already_done = os.path.exists(done_file) + will_build = (force_build or force_build_failures or + not result.already_done) + if result.already_done: + with open(done_file, 'r', encoding='utf-8') as outf: + try: + result.return_code = int(outf.readline()) + except ValueError: + # The file may be empty due to running out of disk space. + # Try a rebuild + result.return_code = RETURN_CODE_RETRY + + # Check the signal that the build needs to be retried + if result.return_code == RETURN_CODE_RETRY: + will_build = True + elif will_build: + err_file = self.builder.get_err_file(commit_upto, brd.target) + if os.path.exists(err_file) and os.stat(err_file).st_size: + result.stderr = 'bad' + elif not force_build: + # The build passed, so no need to build it again + will_build = False + return will_build + def run_commit(self, commit_upto, brd, work_dir, do_config, config_only, force_build, force_build_failures, work_in_output, adjust_cfg): @@ -291,30 +331,8 @@ class BuilderThread(threading.Thread): out_dir = os.path.join(work_dir, out_rel_dir) # Check if the job was already completed last time - done_file = self.builder.get_done_file(commit_upto, brd.target) - result.already_done = os.path.exists(done_file) - will_build = (force_build or force_build_failures or - not result.already_done) - if result.already_done: - # Get the return code from that build and use it - with open(done_file, 'r', encoding='utf-8') as outf: - try: - result.return_code = int(outf.readline()) - except ValueError: - # The file may be empty due to running out of disk space. - # Try a rebuild - result.return_code = RETURN_CODE_RETRY - - # Check the signal that the build needs to be retried - if result.return_code == RETURN_CODE_RETRY: - will_build = True - elif will_build: - err_file = self.builder.get_err_file(commit_upto, brd.target) - if os.path.exists(err_file) and os.stat(err_file).st_size: - result.stderr = 'bad' - elif not force_build: - # The build passed, so no need to build it again - will_build = False + will_build = self._read_done_file(commit_upto, brd, result, force_build, + force_build_failures) if will_build: # We are going to have to build it. First, get a toolchain From patchwork Sun Jul 2 14:26:16 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1802371 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=2a01:238:438b:c500:173d:9f52:ddab:ee01; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.a=rsa-sha256 header.s=google header.b=lAq5vZOy; dkim-atps=neutral Received: from phobos.denx.de (phobos.denx.de [IPv6:2a01:238:438b:c500:173d:9f52:ddab:ee01]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QvBR56hQ6z20ZF for ; Mon, 3 Jul 2023 00:34:57 +1000 (AEST) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id 8DF7F867ED; Sun, 2 Jul 2023 16:28:50 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="lAq5vZOy"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id 6CFA586781; Sun, 2 Jul 2023 16:27:58 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.2 Received: from mail-io1-xd30.google.com (mail-io1-xd30.google.com [IPv6:2607:f8b0:4864:20::d30]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id 7EAC0866B3 for ; Sun, 2 Jul 2023 16:27:54 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=sjg@chromium.org Received: by mail-io1-xd30.google.com with SMTP id ca18e2360f4ac-785cbc5bfd2so186020439f.2 for ; Sun, 02 Jul 2023 07:27:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1688308073; x=1690900073; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=1eCNAk1C2ueIYAlC2zsNuH5ky21pRII/Luv5TcjfGus=; b=lAq5vZOynUhG4nxtKp/nt+168L46OI796k1hAvxdY+BeKH2bicRP4WPU7K5lBkqVKl s3VTJ/dZ3IRGuVPrKQo8RGXDNnhOJ22gtbsKjTXRaaRcqijKj2aM1huFbJ6e9NO8vL1g OmkT06EXpMdUJ9+nf6C8ZFu4o2j5H8cqSGB80= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1688308073; x=1690900073; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=1eCNAk1C2ueIYAlC2zsNuH5ky21pRII/Luv5TcjfGus=; b=dUJ4gvPt3OV3+WzvAJi2xWmoIOHwMwjvXKpCI1MZnkIXd+tYCPINoXgsUTzJfMcuWi sOa2aAmEx+uXb7tSyB0/swxhL8cKE2mT+MuHMA9GQpPA0fe8PZZrZKA6A08APBenX00l qMdkxVH2wIcYNKt9JWOtG5GdizIRzaecdW5ShV/o2rwqoDiVN/Yd5y7G5xq19g5WpC3c y7sKOfejrHSpomoKakbXwxtLzBdg8JUUalnox61UwwhIaHamzAgR3P0JtQO6vma6QCaS wB0VzbV+vjWQ6vSSmpa9s5rlFh9o2rmeJgwX3g1rJgSx9C7dLhWss2c3bHJkCGptFNP0 tSoA== X-Gm-Message-State: AC+VfDzR3fXAd2f4FcZ43B4EC1raUJvuFCPv/R8Mw5rMGqJYg//wcUZW apnlqR4wKPmBgOgILqj7zFSRDdlUp2F3XvBMen4= X-Google-Smtp-Source: ACHHUZ7CeuU+gBj6iij2hrF+V/Kb/n98X+8IkepTdegpvBXp4dKFmbJa+pZMEi2H1RcZbUGYpfeEzQ== X-Received: by 2002:a5d:9ad0:0:b0:776:fa15:2898 with SMTP id x16-20020a5d9ad0000000b00776fa152898mr8139750ion.5.1688308073157; Sun, 02 Jul 2023 07:27:53 -0700 (PDT) Received: from sjg1.lan (c-73-14-173-85.hsd1.co.comcast.net. [73.14.173.85]) by smtp.gmail.com with ESMTPSA id c13-20020a02c9cd000000b0042af5268e4asm2934713jap.58.2023.07.02.07.27.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 02 Jul 2023 07:27:52 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Cc: Simon Glass Subject: [PATCH 51/58] buildman: Move code to remove old outputs Date: Sun, 2 Jul 2023 15:26:16 +0100 Message-ID: <20230702142639.1249681-52-sjg@chromium.org> X-Mailer: git-send-email 2.41.0.255.g8b1d071c50-goog In-Reply-To: <20230702142639.1249681-1-sjg@chromium.org> References: <20230702142639.1249681-1-sjg@chromium.org> MIME-Version: 1.0 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.8 at phobos.denx.de X-Virus-Status: Clean Put this in its own function to reduce the size of the run_commit() function. Signed-off-by: Simon Glass --- tools/buildman/builderthread.py | 28 +++++++++++++++++++--------- 1 file changed, 19 insertions(+), 9 deletions(-) diff --git a/tools/buildman/builderthread.py b/tools/buildman/builderthread.py index b4891059b6db..4abad86ebc7b 100644 --- a/tools/buildman/builderthread.py +++ b/tools/buildman/builderthread.py @@ -42,6 +42,24 @@ def mkdir(dirname, parents = False): else: raise + +def _remove_old_outputs(out_dir): + """Remove any old output-target files + + Args: + out_dir (str): Output directory for the build + + Since we use a build directory that was previously used by another + board, it may have produced an SPL image. If we don't remove it (i.e. + see do_config and self.mrproper below) then it will appear to be the + output of this build, even if it does not produce SPL images. + """ + for elf in BASE_ELF_FILENAMES: + fname = os.path.join(out_dir, elf) + if os.path.exists(fname): + os.remove(fname) + + # pylint: disable=R0903 class BuilderJob: """Holds information about a job to be performed by a thread @@ -366,15 +384,7 @@ class BuilderThread(threading.Thread): config_args = [f'{brd.target}_defconfig'] config_out = io.StringIO() - # Remove any output targets. Since we use a build directory that - # was previously used by another board, it may have produced an - # SPL image. If we don't remove it (i.e. see do_config and - # self.mrproper below) then it will appear to be the output of - # this build, even if it does not produce SPL images. - for elf in BASE_ELF_FILENAMES: - fname = os.path.join(out_dir, elf) - if os.path.exists(fname): - os.remove(fname) + _remove_old_outputs(out_dir) # If we need to reconfigure, do that now cfg_file = os.path.join(out_dir, '.config') From patchwork Sun Jul 2 14:26:17 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1802374 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=2a01:238:438b:c500:173d:9f52:ddab:ee01; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.a=rsa-sha256 header.s=google header.b=WKKCwfd2; dkim-atps=neutral Received: from phobos.denx.de (phobos.denx.de [IPv6:2a01:238:438b:c500:173d:9f52:ddab:ee01]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QvBRf20RBz242R for ; Mon, 3 Jul 2023 00:35:26 +1000 (AEST) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id ED3AA867F3; Sun, 2 Jul 2023 16:28:54 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="WKKCwfd2"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id 264EB8677F; Sun, 2 Jul 2023 16:28:02 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.2 Received: from mail-il1-x12e.google.com (mail-il1-x12e.google.com [IPv6:2607:f8b0:4864:20::12e]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id 5800786775 for ; Sun, 2 Jul 2023 16:27:55 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=sjg@chromium.org Received: by mail-il1-x12e.google.com with SMTP id e9e14a558f8ab-345a8a78bcfso15054765ab.2 for ; Sun, 02 Jul 2023 07:27:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1688308074; x=1690900074; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=GiOH/R8yqwbny8sk4uUX+WVS9Esh6jrUshHAJ4SlLWI=; b=WKKCwfd2u7ZExF0Vl6zFC/p/kA4UUSFok0Js8zsUatvKJiMUUEjo2xOainDm4l/5zr 4P3cVB+RHV19vvE8HDKFNjqz4EkOgySHNRE+VQaCmGdInahwQS0Fp3C1p1a5UwE5y3/B DUz15oZVvYRpP8ZZdD7+2XEeksIC/qodtOreI= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1688308074; x=1690900074; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=GiOH/R8yqwbny8sk4uUX+WVS9Esh6jrUshHAJ4SlLWI=; b=GsyVObzEcGmtmiePXw/23otLEYGtt6+DAXzZk1Lx2IkVc9wmqtjKJnkTPm3HWrDM6N gxv5dG4cCuk5g4rkXmnW5KD5lqCJkt1udN66jBrUHXr9M7O+hoZKTr7sKWwTZ14Zduvh Vlk8a7F48Fm7AZQKbsUhMJh6ohJHQfeCJqZZ0v2VG7lILXi5U905EfDklGC0bqH0hqET 3O6kFofGwhBBmQ5SFoqhsJ4/rBggq57KpGfWz2FTU9yw7En8BJsjO09UtrTLgDwmZDJ/ hKDRhfUlGKYlji/rZuuUlCoPumJmLuVtIO//jfyGLTbDkWCjNH2qMFKW6wY/chWHmNcO botQ== X-Gm-Message-State: ABy/qLZWTFkWmXnkybg/+2T8VRL2SHEmQ2s30Gf177u6iH5f1eE3fuvj XD6OAIp3NE3FMC+iSNa5PG3vWY9kGX304qnZg+A= X-Google-Smtp-Source: APBJJlFJFttwT9dEN+Btk+HbHbl9GRKBTLSiVqa5mFnGq3+jZWeou7qj7ShXFpCV+yZDj4pyUTwC0Q== X-Received: by 2002:a92:d48e:0:b0:345:6ffa:63c5 with SMTP id p14-20020a92d48e000000b003456ffa63c5mr6801991ilg.32.1688308073817; Sun, 02 Jul 2023 07:27:53 -0700 (PDT) Received: from sjg1.lan (c-73-14-173-85.hsd1.co.comcast.net. [73.14.173.85]) by smtp.gmail.com with ESMTPSA id c13-20020a02c9cd000000b0042af5268e4asm2934713jap.58.2023.07.02.07.27.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 02 Jul 2023 07:27:53 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Cc: Simon Glass Subject: [PATCH 52/58] buildman: Move code to decide output dirs Date: Sun, 2 Jul 2023 15:26:17 +0100 Message-ID: <20230702142639.1249681-53-sjg@chromium.org> X-Mailer: git-send-email 2.41.0.255.g8b1d071c50-goog In-Reply-To: <20230702142639.1249681-1-sjg@chromium.org> References: <20230702142639.1249681-1-sjg@chromium.org> MIME-Version: 1.0 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.8 at phobos.denx.de X-Virus-Status: Clean Put this in its own function to reduce the size of the run_commit() function. Signed-off-by: Simon Glass --- tools/buildman/builderthread.py | 34 ++++++++++++++++++++++++--------- 1 file changed, 25 insertions(+), 9 deletions(-) diff --git a/tools/buildman/builderthread.py b/tools/buildman/builderthread.py index 4abad86ebc7b..78405956ef51 100644 --- a/tools/buildman/builderthread.py +++ b/tools/buildman/builderthread.py @@ -302,6 +302,30 @@ class BuilderThread(threading.Thread): will_build = False return will_build + def _decide_dirs(self, brd, work_dir, work_in_output): + """Decide the output directory to use + + Args: + work_dir (str): Directory to which the source will be checked out + work_in_output (bool): Use the output directory as the work + directory and don't write to a separate output directory. + + Returns: + tuple: + out_dir (str): Output directory for the build + out_rel_dir (str): Output directory relatie to the current dir + """ + if work_in_output or self.builder.in_tree: + out_rel_dir = None + out_dir = work_dir + else: + if self.per_board_out_dir: + out_rel_dir = os.path.join('..', brd.target) + else: + out_rel_dir = 'build' + out_dir = os.path.join(work_dir, out_rel_dir) + return out_dir, out_rel_dir + def run_commit(self, commit_upto, brd, work_dir, do_config, config_only, force_build, force_build_failures, work_in_output, adjust_cfg): @@ -338,15 +362,7 @@ class BuilderThread(threading.Thread): # self.make() below, in the event that we do a build. result = command.CommandResult() result.return_code = 0 - if work_in_output or self.builder.in_tree: - out_rel_dir = None - out_dir = work_dir - else: - if self.per_board_out_dir: - out_rel_dir = os.path.join('..', brd.target) - else: - out_rel_dir = 'build' - out_dir = os.path.join(work_dir, out_rel_dir) + out_dir, out_rel_dir = self._decide_dirs(brd, work_dir, work_in_output) # Check if the job was already completed last time will_build = self._read_done_file(commit_upto, brd, result, force_build, From patchwork Sun Jul 2 14:26:18 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1802373 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=2a01:238:438b:c500:173d:9f52:ddab:ee01; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.a=rsa-sha256 header.s=google header.b=F5Su8owc; dkim-atps=neutral Received: from phobos.denx.de (phobos.denx.de [IPv6:2a01:238:438b:c500:173d:9f52:ddab:ee01]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QvBRT3bS6z242R for ; Mon, 3 Jul 2023 00:35:17 +1000 (AEST) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id 7763186801; Sun, 2 Jul 2023 16:28:53 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="F5Su8owc"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id CE45B866CA; Sun, 2 Jul 2023 16:28:02 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.2 Received: from mail-il1-x12a.google.com (mail-il1-x12a.google.com [IPv6:2607:f8b0:4864:20::12a]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id C926B86773 for ; Sun, 2 Jul 2023 16:27:55 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=sjg@chromium.org Received: by mail-il1-x12a.google.com with SMTP id e9e14a558f8ab-3460b67fdd8so383205ab.0 for ; Sun, 02 Jul 2023 07:27:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1688308074; x=1690900074; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=SZlBJqdJ2OABecGMzp217FHoJzNGCYV0bZidErPAkIM=; b=F5Su8owcENf+JNobsSGrddlKTqHDNsD81WU/oLbPQxa4mj+1AUEvjEmT942ph/d5mL PNsXiJOYCjT8Fp7kexKAjoZND0l/jLhZPc0LK/X1jfS1AgUAleVLEFOARg33mdOb0DGp fF1FZxyFHRvwiVa9/jNbhelwGoD3n3wlJysNM= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1688308074; x=1690900074; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=SZlBJqdJ2OABecGMzp217FHoJzNGCYV0bZidErPAkIM=; b=fxWB2xjbdwmyMGew8gd6v/MCM7qGq80zbXKqCZ7uomkQjDs4bCcqoZ5HErYnGm2Qyt VQNFJFOS5FndSMe/7Y7XipZ//W/s9BuRCqD4u7JajHIkZvE1I7BmsqFTkK7vOk13xRhI 5/urKerc4KnLDfNAF77Vf2trCVL3D4lO/vcdR1F1MzyZHoCAnMyRHRu8InwPh/Oi8DNj Y3Exc7GFPL3yseXbtcR4ybPtMRrJlNSrSKCrtG96IJtNmilYfQYbqi3UxeRGmibEdcjr S6BLQI/u7PPLl4krflv8qeWyKHp1+MBDCVfypmFHBwlgV1erLgZSeMNH0m476gLMN/DS Nzyg== X-Gm-Message-State: ABy/qLZ3J1Vl69Mt3VXHcP9tDC2Vr77Iu4qHpAoSXION3lOgI2tpxmNH cKh3yfjetZKkMHWdsDsa43nl/pXjQWR6ss9pxr8= X-Google-Smtp-Source: APBJJlFDqpFnxkGywcroUF/pacS0n+gsiNQAvY18KfG+CtfxzARHCBWer0maZ3tZxTmMS3pkjT8WWQ== X-Received: by 2002:a92:ce92:0:b0:345:775f:1a2f with SMTP id r18-20020a92ce92000000b00345775f1a2fmr7199393ilo.14.1688308074432; Sun, 02 Jul 2023 07:27:54 -0700 (PDT) Received: from sjg1.lan (c-73-14-173-85.hsd1.co.comcast.net. [73.14.173.85]) by smtp.gmail.com with ESMTPSA id c13-20020a02c9cd000000b0042af5268e4asm2934713jap.58.2023.07.02.07.27.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 02 Jul 2023 07:27:54 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Cc: Simon Glass Subject: [PATCH 53/58] buildman: Move checkout code to a separate function Date: Sun, 2 Jul 2023 15:26:18 +0100 Message-ID: <20230702142639.1249681-54-sjg@chromium.org> X-Mailer: git-send-email 2.41.0.255.g8b1d071c50-goog In-Reply-To: <20230702142639.1249681-1-sjg@chromium.org> References: <20230702142639.1249681-1-sjg@chromium.org> MIME-Version: 1.0 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.8 at phobos.denx.de X-Virus-Status: Clean Put this in its own function to reduce the size of the run_commit() function Signed-off-by: Simon Glass --- tools/buildman/builderthread.py | 30 +++++++++++++++++++++--------- 1 file changed, 21 insertions(+), 9 deletions(-) diff --git a/tools/buildman/builderthread.py b/tools/buildman/builderthread.py index 78405956ef51..0c73b8646b1c 100644 --- a/tools/buildman/builderthread.py +++ b/tools/buildman/builderthread.py @@ -326,6 +326,26 @@ class BuilderThread(threading.Thread): out_dir = os.path.join(work_dir, out_rel_dir) return out_dir, out_rel_dir + def _checkout(self, commit_upto, work_dir): + """Checkout the right commit + + Args: + commit_upto (int): Commit number to build (0...n-1) + work_dir (str): Directory to which the source will be checked out + + Returns: + Commit: Commit being built, or 'current' for current source + """ + if self.builder.commits: + commit = self.builder.commits[commit_upto] + if self.builder.checkout: + git_dir = os.path.join(work_dir, '.git') + gitutil.checkout(commit.hash, git_dir, work_dir, force=True) + else: + commit = 'current' + return commit + + def run_commit(self, commit_upto, brd, work_dir, do_config, config_only, force_build, force_build_failures, work_in_output, adjust_cfg): @@ -381,15 +401,7 @@ class BuilderThread(threading.Thread): # to be reported. if self.toolchain: - # Checkout the right commit - if self.builder.commits: - commit = self.builder.commits[commit_upto] - if self.builder.checkout: - git_dir = os.path.join(work_dir, '.git') - gitutil.checkout(commit.hash, git_dir, work_dir, - force=True) - else: - commit = 'current' + commit = self._checkout(commit_upto, work_dir) # Set up the environment and command line env = self.toolchain.MakeEnvironment(self.builder.full_path) From patchwork Sun Jul 2 14:26:19 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1802379 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=2a01:238:438b:c500:173d:9f52:ddab:ee01; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.a=rsa-sha256 header.s=google header.b=nmlzoGNV; dkim-atps=neutral Received: from phobos.denx.de (phobos.denx.de [IPv6:2a01:238:438b:c500:173d:9f52:ddab:ee01]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QvBSX2Ks6z242R for ; Mon, 3 Jul 2023 00:36:12 +1000 (AEST) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id 3570786816; Sun, 2 Jul 2023 16:29:08 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="nmlzoGNV"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id 5ECBD86342; Sun, 2 Jul 2023 16:28:08 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.2 Received: from mail-il1-x12b.google.com (mail-il1-x12b.google.com [IPv6:2607:f8b0:4864:20::12b]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id 7C35C86653 for ; Sun, 2 Jul 2023 16:27:56 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=sjg@chromium.org Received: by mail-il1-x12b.google.com with SMTP id e9e14a558f8ab-3460b67fdd8so383215ab.0 for ; Sun, 02 Jul 2023 07:27:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1688308075; x=1690900075; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=lMc6yCRuPIGqwDlgawCIyHBxTKhieIZ0kC2XlyrGvXI=; b=nmlzoGNVd78tgtciOrRj+fjv8DYX6tLQ5XfuYFnq3X0hakZTmwTXSwNoC1jKECR+MS bpSFKZTvyTq0Jj2M+hrfChPfdUumwebwhT3Xwy3R0AjHNl/6RDP9hqo7x546Lu0N+IhA 6ysUb7XrUyVo452z6T7CWsiS31444Xqb05ImA= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1688308075; x=1690900075; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=lMc6yCRuPIGqwDlgawCIyHBxTKhieIZ0kC2XlyrGvXI=; b=kSmB/AwhDaNI5gGTTRC5Lrxjd+GaU2AtU2DG8LjoS71G8mXOeSNZHqNbgkjvmXyqUj MpgKjikkKSA+1jXz8Q5sRaO0njNFjrZJDMXIXtwvTiDGhx8RBxWBJ+Xv22x0h8gGeEX0 ahK7rN1RRVw+PtKxLNiAFs49Hw8yImNOeM+o3FKABpk8/lWTeAHYpCt4zkX9hL4gKLOC bomDTtAovUhuhkP7p9yZA6+m5FZMbgLk0qT1CaPOicAHaD1FyWSDIh4QQ9xlIhkGSPJR 4pzq3zPo9eIOsMAftcLy/ju6PLOvBpusLCfWX4XpHEnsVEn/fINB+zj+0mA2tkzyZM3J 3g+A== X-Gm-Message-State: ABy/qLZrz6Kv5UmgmVZPYfjq1hvgfqvVK8w/U0wnIQnEBA9zLs+1pi/a nXQd48vmGocRYrD3YnL73eXxSkhC37m9yeUNoXs= X-Google-Smtp-Source: APBJJlEspyd1ie7QWXPxL/YAq/2NutKG3obx9j0dQa1c36EhQf+qVetavX5vhvaCEjvMse/97eng7g== X-Received: by 2002:a92:c904:0:b0:345:ad81:ecaf with SMTP id t4-20020a92c904000000b00345ad81ecafmr8121968ilp.3.1688308075059; Sun, 02 Jul 2023 07:27:55 -0700 (PDT) Received: from sjg1.lan (c-73-14-173-85.hsd1.co.comcast.net. [73.14.173.85]) by smtp.gmail.com with ESMTPSA id c13-20020a02c9cd000000b0042af5268e4asm2934713jap.58.2023.07.02.07.27.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 02 Jul 2023 07:27:54 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Cc: Simon Glass Subject: [PATCH 54/58] buildman: Create a function to handle config and build Date: Sun, 2 Jul 2023 15:26:19 +0100 Message-ID: <20230702142639.1249681-55-sjg@chromium.org> X-Mailer: git-send-email 2.41.0.255.g8b1d071c50-goog In-Reply-To: <20230702142639.1249681-1-sjg@chromium.org> References: <20230702142639.1249681-1-sjg@chromium.org> MIME-Version: 1.0 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.8 at phobos.denx.de X-Virus-Status: Clean Move this code into a _config_and_build() function, so reduce the size of run_commit(). Signed-off-by: Simon Glass --- tools/buildman/builderthread.py | 97 +++++++++++++++++++++------------ 1 file changed, 61 insertions(+), 36 deletions(-) diff --git a/tools/buildman/builderthread.py b/tools/buildman/builderthread.py index 0c73b8646b1c..aa4a9d9919c6 100644 --- a/tools/buildman/builderthread.py +++ b/tools/buildman/builderthread.py @@ -345,6 +345,64 @@ class BuilderThread(threading.Thread): commit = 'current' return commit + def _config_and_build(self, commit_upto, brd, work_dir, do_config, + config_only, adjust_cfg, commit, out_dir, out_rel_dir, + result): + """Do the build, configuring first if necessary + + Args: + commit_upto (int): Commit number to build (0...n-1) + brd (Board): Board to create arguments for + work_dir (str): Directory to which the source will be checked out + do_config (bool): True to run a make _defconfig on the source + config_only (bool): Only configure the source, do not build it + adjust_cfg (list of str): See the cfgutil module and run_commit() + commit (Commit): Commit only being built + out_dir (str): Output directory for the build + out_rel_dir (str): Output directory relatie to the current dir + result (CommandResult): Previous result + + Returns: + tuple: + result (CommandResult): Result of the build + do_config (bool): indicates whether 'make config' is needed on + the next incremental build + """ + # Set up the environment and command line + env = self.toolchain.MakeEnvironment(self.builder.full_path) + mkdir(out_dir) + + args, cwd, src_dir = self._build_args(brd, out_dir, out_rel_dir, + work_dir, commit_upto) + config_args = [f'{brd.target}_defconfig'] + config_out = io.StringIO() + + _remove_old_outputs(out_dir) + + # If we need to reconfigure, do that now + cfg_file = os.path.join(out_dir, '.config') + cmd_list = [] + if do_config or adjust_cfg: + result = self._reconfigure( + commit, brd, cwd, args, env, config_args, config_out, cmd_list) + do_config = False # No need to configure next time + if adjust_cfg: + cfgutil.adjust_cfg_file(cfg_file, adjust_cfg) + + # Now do the build, if everything looks OK + if result.return_code == 0: + result = self._build(commit, brd, cwd, args, env, cmd_list, + config_only) + if adjust_cfg: + errs = cfgutil.check_cfg_file(cfg_file, adjust_cfg) + if errs: + result.stderr += errs + result.return_code = 1 + result.stderr = result.stderr.replace(src_dir + '/', '') + if self.builder.verbose_build: + result.stdout = config_out.getvalue() + result.stdout + result.cmd_list = cmd_list + return result, do_config def run_commit(self, commit_upto, brd, work_dir, do_config, config_only, force_build, force_build_failures, work_in_output, @@ -402,42 +460,9 @@ class BuilderThread(threading.Thread): if self.toolchain: commit = self._checkout(commit_upto, work_dir) - - # Set up the environment and command line - env = self.toolchain.MakeEnvironment(self.builder.full_path) - mkdir(out_dir) - - args, cwd, src_dir = self._build_args(brd, out_dir, out_rel_dir, - work_dir, commit_upto) - config_args = [f'{brd.target}_defconfig'] - config_out = io.StringIO() - - _remove_old_outputs(out_dir) - - # If we need to reconfigure, do that now - cfg_file = os.path.join(out_dir, '.config') - cmd_list = [] - if do_config or adjust_cfg: - result = self._reconfigure( - commit, brd, cwd, args, env, config_args, config_out, - cmd_list) - do_config = False # No need to configure next time - if adjust_cfg: - cfgutil.adjust_cfg_file(cfg_file, adjust_cfg) - - # Now do the build, if everything looks OK - if result.return_code == 0: - result = self._build(commit, brd, cwd, args, env, cmd_list, - config_only) - if adjust_cfg: - errs = cfgutil.check_cfg_file(cfg_file, adjust_cfg) - if errs: - result.stderr += errs - result.return_code = 1 - result.stderr = result.stderr.replace(src_dir + '/', '') - if self.builder.verbose_build: - result.stdout = config_out.getvalue() + result.stdout - result.cmd_list = cmd_list + result, do_config = self._config_and_build( + commit_upto, brd, work_dir, do_config, config_only, + adjust_cfg, commit, out_dir, out_rel_dir, result) else: result.return_code = 1 result.stderr = f'No tool chain for {brd.arch}\n' From patchwork Sun Jul 2 14:26:20 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1802376 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=85.214.62.61; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.a=rsa-sha256 header.s=google header.b=KaWuBwOL; dkim-atps=neutral Received: from phobos.denx.de (phobos.denx.de [85.214.62.61]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QvBS016tMz242R for ; Mon, 3 Jul 2023 00:35:44 +1000 (AEST) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id C4C8B86806; Sun, 2 Jul 2023 16:28:57 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="KaWuBwOL"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id 6DA1F86773; Sun, 2 Jul 2023 16:28:06 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.2 Received: from mail-il1-x12e.google.com (mail-il1-x12e.google.com [IPv6:2607:f8b0:4864:20::12e]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id 137EB8677D for ; Sun, 2 Jul 2023 16:27:57 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=sjg@chromium.org Received: by mail-il1-x12e.google.com with SMTP id e9e14a558f8ab-3460aee1d57so2536305ab.0 for ; Sun, 02 Jul 2023 07:27:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1688308075; x=1690900075; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=40aqU+nsXYzII56iKAAqUv2mR9PRr8J8SHMD+F8/15o=; b=KaWuBwOLuxohLopxehCHe4JLRHT/QRHB4aanm9KZ6SCTmvSIeL2/SCpasOZQRU7lFi DLPIJCVBy7xszWCyo3MpqvdkuPVMeNemS3uEpiEvdYDbFrC1rqeAYj6Iyf94zx0sd7AB RMqOIX+QZml/D6DeclayD8k6QO8ih5rBNGXQQ= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1688308075; x=1690900075; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=40aqU+nsXYzII56iKAAqUv2mR9PRr8J8SHMD+F8/15o=; b=NTQbCGygOapKmS0uFJ7qAWzgO+pjdKn+Xvu87lHJCelpnX6ozixSdgZbuTlDFZfPrz hf/GTK2Ks/ch8fyOaKY88uX1m1kjmWwq97vacN5A8JFn2dIXrjsymIJH+VKTJKmgc2h1 lOLlKpfmSiHy5YdsVt1FOc7ca814An76Jvw2JLZF/+jBUbHV6L0uPinIpnMtfphPpV8n Pko/SJkM92Xy2udA/zMLDZydBacW7pXhTzmq9kryNWKe4Yaw3tdp9MwN0CjL5EzIJbhl UmEGEM8pDZjvp39CwIq0SXTFd+iWYK6ePnhLtg9z1gfTdDofKN777VwnjIGq+EnPvuI8 Sp+Q== X-Gm-Message-State: ABy/qLYNOkyapjE3fLic3c+Kf+HcIqSmoGH/BHWxOE008rZE7746Uz5y pvWDyF7do6VNLav9iAFY+T3LYFLx7JD98swemUM= X-Google-Smtp-Source: APBJJlFQY1bsI3gxKBWMsqGd3nJv3byBfH7lhkKk3Dl66RdPAbxr6ynO/58/JuSpVy+nefrcNRiB0g== X-Received: by 2002:a92:ce85:0:b0:33f:ff90:15b4 with SMTP id r5-20020a92ce85000000b0033fff9015b4mr9043058ilo.3.1688308075737; Sun, 02 Jul 2023 07:27:55 -0700 (PDT) Received: from sjg1.lan (c-73-14-173-85.hsd1.co.comcast.net. [73.14.173.85]) by smtp.gmail.com with ESMTPSA id c13-20020a02c9cd000000b0042af5268e4asm2934713jap.58.2023.07.02.07.27.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 02 Jul 2023 07:27:55 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Cc: Simon Glass Subject: [PATCH 55/58] buildman: Avoid passing result into _read_done_file() Date: Sun, 2 Jul 2023 15:26:20 +0100 Message-ID: <20230702142639.1249681-56-sjg@chromium.org> X-Mailer: git-send-email 2.41.0.255.g8b1d071c50-goog In-Reply-To: <20230702142639.1249681-1-sjg@chromium.org> References: <20230702142639.1249681-1-sjg@chromium.org> MIME-Version: 1.0 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.8 at phobos.denx.de X-Virus-Status: Clean Move the creating of the result object into the function which sets it up, to simplify the code. Signed-off-by: Simon Glass --- tools/buildman/builderthread.py | 19 +++++++++++-------- 1 file changed, 11 insertions(+), 8 deletions(-) diff --git a/tools/buildman/builderthread.py b/tools/buildman/builderthread.py index aa4a9d9919c6..d3912390bc4f 100644 --- a/tools/buildman/builderthread.py +++ b/tools/buildman/builderthread.py @@ -262,21 +262,26 @@ class BuilderThread(threading.Thread): result.return_code = 0 return result - def _read_done_file(self, commit_upto, brd, result, force_build, + def _read_done_file(self, commit_upto, brd, force_build, force_build_failures): """Check the 'done' file and see if this commit should be built Args: commit (Commit): Commit only being built brd (Board): Board being built - result (CommandResult): result object to update force_build (bool): Force a build even if one was previously done force_build_failures (bool): Force a bulid if the previous result showed failure Returns: - bool: True if build should be built + tuple: + bool: True if build should be built + CommandResult: if there was a previous run: + - already_done set to True + - return_code set to return code + - result.stderr set to 'bad' if stderr output was recorded """ + result = command.CommandResult() done_file = self.builder.get_done_file(commit_upto, brd.target) result.already_done = os.path.exists(done_file) will_build = (force_build or force_build_failures or @@ -300,7 +305,7 @@ class BuilderThread(threading.Thread): elif not force_build: # The build passed, so no need to build it again will_build = False - return will_build + return will_build, result def _decide_dirs(self, brd, work_dir, work_in_output): """Decide the output directory to use @@ -438,13 +443,11 @@ class BuilderThread(threading.Thread): """ # Create a default result - it will be overwritte by the call to # self.make() below, in the event that we do a build. - result = command.CommandResult() - result.return_code = 0 out_dir, out_rel_dir = self._decide_dirs(brd, work_dir, work_in_output) # Check if the job was already completed last time - will_build = self._read_done_file(commit_upto, brd, result, force_build, - force_build_failures) + will_build, result = self._read_done_file(commit_upto, brd, force_build, + force_build_failures) if will_build: # We are going to have to build it. First, get a toolchain From patchwork Sun Jul 2 14:26:21 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1802375 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=85.214.62.61; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.a=rsa-sha256 header.s=google header.b=Bga7W4Rd; dkim-atps=neutral Received: from phobos.denx.de (phobos.denx.de [85.214.62.61]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QvBRp6r8cz242R for ; Mon, 3 Jul 2023 00:35:34 +1000 (AEST) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id 98421867F5; Sun, 2 Jul 2023 16:28:56 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="Bga7W4Rd"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id 4F434866CA; Sun, 2 Jul 2023 16:28:05 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.2 Received: from mail-il1-x136.google.com (mail-il1-x136.google.com [IPv6:2607:f8b0:4864:20::136]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id A3A4A8677E for ; Sun, 2 Jul 2023 16:27:57 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=sjg@chromium.org Received: by mail-il1-x136.google.com with SMTP id e9e14a558f8ab-3456fda4ed1so20158945ab.0 for ; Sun, 02 Jul 2023 07:27:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1688308076; x=1690900076; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=QO8XjRR4bdHLo+D/BvPxVRFmArhXtcXQ8iFuWlIWaSg=; b=Bga7W4RdrxLIbFw57B0DcI8gGJuZj1NxOTLod5pPQlGGpq0pnyU9DQQ49Vsi1Mb0Sv 2xRYQjjVZsVOOJvhLlWENQEfdtewHDLfN8vi6mb8Oe74jonl819cfDSruo+RqUxEe6hw 4y6m4gYk7b3lCmxpgQh8P9azzCVgH5NJjaY08= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1688308076; x=1690900076; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=QO8XjRR4bdHLo+D/BvPxVRFmArhXtcXQ8iFuWlIWaSg=; b=C/7EnJKbpCvXjfXqOdxD6iXiHDiYW/bI4674bt4e6Bh5fgzhA7e8uDcKaKvwX+gkvJ 5Z2HQmE/g7u27E0PJqDUIHbrCE2DfdFniywcvMvZ1zjhLbXSVb/44g9348/MTGj1HvPd 43EnymDU1BVfQfXTd2SAiND/027+NrYYM6GbmTxOeJEadg4tQoig/yju9f7IUvIoutdx iskndEERjuLMxKUPf9IRkJO6FZTDStDvKMw5VmHMMimFfG7Q5BCvd9KrJxgI1BdoOVZt WqncN52knvfnSjxPIF4vl4wynbY//dafzyhAn8KXzjZFiu2A20VESUkIEQE7pu8xUhuA giUg== X-Gm-Message-State: ABy/qLbndurewmBXRGfxcgmfeLCYUZMgJJmwGKm+CWH7KWhYCwnThT57 xqJPfSMH2k2n1oP5/BH08ggguaVc+dNGZEXL5K4= X-Google-Smtp-Source: APBJJlF+xa4YNiqitTmCGfapzKCtgiOPMVt9p/nZYP1OiMF9lQsEl3VatBNSYR0yQ+SjXHitKnOBHA== X-Received: by 2002:a92:c608:0:b0:345:b536:61f with SMTP id p8-20020a92c608000000b00345b536061fmr8106831ilm.31.1688308076356; Sun, 02 Jul 2023 07:27:56 -0700 (PDT) Received: from sjg1.lan (c-73-14-173-85.hsd1.co.comcast.net. [73.14.173.85]) by smtp.gmail.com with ESMTPSA id c13-20020a02c9cd000000b0042af5268e4asm2934713jap.58.2023.07.02.07.27.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 02 Jul 2023 07:27:56 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Cc: Simon Glass Subject: [PATCH 56/58] buildman: Tidy up reporting of a toolchain error Date: Sun, 2 Jul 2023 15:26:21 +0100 Message-ID: <20230702142639.1249681-57-sjg@chromium.org> X-Mailer: git-send-email 2.41.0.255.g8b1d071c50-goog In-Reply-To: <20230702142639.1249681-1-sjg@chromium.org> References: <20230702142639.1249681-1-sjg@chromium.org> MIME-Version: 1.0 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.8 at phobos.denx.de X-Virus-Status: Clean Provide the text of the exception when something goes wrong. Signed-off-by: Simon Glass --- tools/buildman/builderthread.py | 7 +------ tools/buildman/func_test.py | 6 ++++-- 2 files changed, 5 insertions(+), 8 deletions(-) diff --git a/tools/buildman/builderthread.py b/tools/buildman/builderthread.py index d3912390bc4f..043e92b6d92d 100644 --- a/tools/buildman/builderthread.py +++ b/tools/buildman/builderthread.py @@ -457,18 +457,13 @@ class BuilderThread(threading.Thread): except ValueError as err: result.return_code = 10 result.stdout = '' - result.stderr = str(err) - # TODO(sjg@chromium.org): This gets swallowed, but needs - # to be reported. + result.stderr = f'Tool chain error for {brd.arch}: {str(err)}' if self.toolchain: commit = self._checkout(commit_upto, work_dir) result, do_config = self._config_and_build( commit_upto, brd, work_dir, do_config, config_only, adjust_cfg, commit, out_dir, out_rel_dir, result) - else: - result.return_code = 1 - result.stderr = f'No tool chain for {brd.arch}\n' result.already_done = False result.toolchain = self.toolchain diff --git a/tools/buildman/func_test.py b/tools/buildman/func_test.py index b9f99c76cdb4..dbaca7fd1094 100644 --- a/tools/buildman/func_test.py +++ b/tools/buildman/func_test.py @@ -503,8 +503,10 @@ Some images are invalid''' if brd.arch != 'sandbox': errfile = self._builder.get_err_file(commit, brd.target) fd = open(errfile) - self.assertEqual(fd.readlines(), - ['No tool chain for %s\n' % brd.arch]) + self.assertEqual( + fd.readlines(), + [f'Tool chain error for {brd.arch}: ' + f"No tool chain found for arch '{brd.arch}'"]) fd.close() def testBranch(self): From patchwork Sun Jul 2 14:26:22 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1802377 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=2a01:238:438b:c500:173d:9f52:ddab:ee01; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.a=rsa-sha256 header.s=google header.b=aavlQuIY; dkim-atps=neutral Received: from phobos.denx.de (phobos.denx.de [IPv6:2a01:238:438b:c500:173d:9f52:ddab:ee01]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QvBS932Vvz242R for ; Mon, 3 Jul 2023 00:35:53 +1000 (AEST) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id 2B97B8680C; Sun, 2 Jul 2023 16:28:59 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="aavlQuIY"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id B2399866CA; Sun, 2 Jul 2023 16:28:06 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.2 Received: from mail-io1-xd34.google.com (mail-io1-xd34.google.com [IPv6:2607:f8b0:4864:20::d34]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id 79170866B3 for ; Sun, 2 Jul 2023 16:27:58 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=sjg@chromium.org Received: by mail-io1-xd34.google.com with SMTP id ca18e2360f4ac-78666994bc2so35073939f.0 for ; Sun, 02 Jul 2023 07:27:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1688308077; x=1690900077; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=jvusXz0bAYoYZaFmwQwGX8hxpxWx2MRC/OcVGurJB48=; b=aavlQuIYneU4E4kK15b+saNNFkUwot+sUh2LpNhpFdY1bLpKrzWM+S2lqaJ+Fc46rS SjOcJSXbtcQ+IdSmSL9G0C3vn/S80h8vB6U/uUOZIjTSPjsFsq9dFsAlbGEU0oYqv3ML zNpogJEWOxZXnOjP1t7kDMAcLDb6JzF/6t3Yw= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1688308077; x=1690900077; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=jvusXz0bAYoYZaFmwQwGX8hxpxWx2MRC/OcVGurJB48=; b=RPsJOTSsVmlEsqZwY6vD8xJwlW0Ya4Jty2vl6l99qdMcQzVobHDhgzFavuJlUpcnkL Otsrjbyka6pkKaGTsORmFNtheVwjR9FQgJF6YoYbHe0CXQBgRzdBYM4GvlVEKjHQlr5V Hrvy2IQXD3S7l+InzLH2LiMvP2MQ2V8fxEqH73M1LcF6eFzaqgWIoIe7iEu/1a7AC72Y EkGjRTJ/LsoObk2u9T4ivGtnkn878l7phF5S/5KDLq2pdigjmkqiH2hDO7KEM1uj3Sto rlxHW69t7sKkjwsdIPiDXf+fhGwHOToz0IiU51mtNRjdl7Ym5RLfargmUSget/WOont6 WSqw== X-Gm-Message-State: AC+VfDzc/d0kRn+lwtSAtMUQbGHDI12Yhsf3pvL3qhg+4ls2MzSTUIR1 686GNz0DOWfm0paEUYaO765HcIIElzyEYxz4pO0= X-Google-Smtp-Source: ACHHUZ6NXlFccFHgL3RZNoca4fhBQhXk3eek5RNeKYRG70BEySAJE/uslD8kBb2hfYZfB9DezGj6Gg== X-Received: by 2002:a5e:c241:0:b0:783:4e11:76d7 with SMTP id w1-20020a5ec241000000b007834e1176d7mr7540295iop.21.1688308076964; Sun, 02 Jul 2023 07:27:56 -0700 (PDT) Received: from sjg1.lan (c-73-14-173-85.hsd1.co.comcast.net. [73.14.173.85]) by smtp.gmail.com with ESMTPSA id c13-20020a02c9cd000000b0042af5268e4asm2934713jap.58.2023.07.02.07.27.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 02 Jul 2023 07:27:56 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Cc: Simon Glass Subject: [PATCH 57/58] buildman: Tidy up some comments in builderthread Date: Sun, 2 Jul 2023 15:26:22 +0100 Message-ID: <20230702142639.1249681-58-sjg@chromium.org> X-Mailer: git-send-email 2.41.0.255.g8b1d071c50-goog In-Reply-To: <20230702142639.1249681-1-sjg@chromium.org> References: <20230702142639.1249681-1-sjg@chromium.org> MIME-Version: 1.0 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.8 at phobos.denx.de X-Virus-Status: Clean Make sure all functions have full argument documentation. Signed-off-by: Simon Glass --- tools/buildman/builderthread.py | 66 ++++++++++++++++++--------------- 1 file changed, 36 insertions(+), 30 deletions(-) diff --git a/tools/buildman/builderthread.py b/tools/buildman/builderthread.py index 043e92b6d92d..ed84a0f6baa3 100644 --- a/tools/buildman/builderthread.py +++ b/tools/buildman/builderthread.py @@ -23,11 +23,15 @@ from u_boot_pylib import command RETURN_CODE_RETRY = -1 BASE_ELF_FILENAMES = ['u-boot', 'spl/u-boot-spl', 'tpl/u-boot-tpl'] -def mkdir(dirname, parents = False): +def mkdir(dirname, parents=False): """Make a directory if it doesn't already exist. Args: - dirname: Directory to create + dirname (str): Directory to create + parents (bool): True to also make parent directories + + Raises: + OSError: File already exists """ try: if parents: @@ -141,14 +145,16 @@ class BuilderThread(threading.Thread): argument is only for information. Args: - commit: Commit object that is being built - brd: Board object that is being built - stage: Stage of the build. Valid stages are: + commit (Commit): Commit that is being built + brd (Board): Board that is being built + stage (str): Stage of the build. Valid stages are: mrproper - can be called to clean source config - called to configure for a board build - the main make invocation - it does the build - args: A list of arguments to pass to 'make' - kwargs: A list of keyword arguments to pass to command.run_pipe() + cwd (str): Working directory to set, or None to leave it alone + *args (list of str): Arguments to pass to 'make' + **kwargs (dict): A list of keyword arguments to pass to + command.run_pipe() Returns: CommandResult object @@ -418,16 +424,16 @@ class BuilderThread(threading.Thread): the build and just return the previously-saved results. Args: - commit_upto: Commit number to build (0...n-1) - brd: Board object to build - work_dir: Directory to which the source will be checked out - do_config: True to run a make _defconfig on the source - config_only: Only configure the source, do not build it - force_build: Force a build even if one was previously done - force_build_failures: Force a bulid if the previous result showed - failure - work_in_output: Use the output directory as the work directory and - don't write to a separate output directory. + commit_upto (int): Commit number to build (0...n-1) + brd (Board): Board to build + work_dir (str): Directory to which the source will be checked out + do_config (bool): True to run a make _defconfig on the source + config_only (bool): Only configure the source, do not build it + force_build (bool): Force a build even if one was previously done + force_build_failures (bool): Force a bulid if the previous result + showed failure + work_in_output (bool) : Use the output directory as the work + directory and don't write to a separate output directory. adjust_cfg (list of str): List of changes to make to .config file before building. Each is one of (where C is either CONFIG_xxx or just xxx): @@ -476,11 +482,11 @@ class BuilderThread(threading.Thread): """Write a built result to the output directory. Args: - result: CommandResult object containing result to write - keep_outputs: True to store the output binaries, False + result (CommandResult): result to write + keep_outputs (bool): True to store the output binaries, False to delete them - work_in_output: Use the output directory as the work directory and - don't write to a separate output directory. + work_in_output (bool): Use the output directory as the work + directory and don't write to a separate output directory. """ # If we think this might have been aborted with Ctrl-C, record the # failure but not that we are 'done' with this board. A retry may fix @@ -618,10 +624,10 @@ class BuilderThread(threading.Thread): """Copy files from the build directory to the output. Args: - out_dir: Path to output directory containing the files - build_dir: Place to copy the files - dirname: Source directory, '' for normal U-Boot, 'spl' for SPL - patterns: A list of filenames (strings) to copy, each relative + out_dir (str): Path to output directory containing the files + build_dir (str): Place to copy the files + dirname (str): Source directory, '' for normal U-Boot, 'spl' for SPL + patterns (list of str): A list of filenames to copy, each relative to the build directory """ for pattern in patterns: @@ -638,10 +644,10 @@ class BuilderThread(threading.Thread): """Send a result to the builder for processing Args: - result: CommandResult object containing the results of the build + result (CommandResult): results of the build Raises: - ValueError if self.test_exception is true (for testing) + ValueError: self.test_exception is true (for testing) """ if self.test_exception: raise ValueError('test exception') @@ -656,10 +662,10 @@ class BuilderThread(threading.Thread): A job consists of a building a list of commits for a particular board. Args: - job: Job to build + job (Job): Job to build - Returns: - List of Result objects + Raises: + ValueError: Thread was interrupted """ brd = job.brd work_dir = self.builder.get_thread_dir(self.thread_num) From patchwork Sun Jul 2 14:26:23 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1802380 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=2a01:238:438b:c500:173d:9f52:ddab:ee01; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.a=rsa-sha256 header.s=google header.b=LW3Z5/kw; dkim-atps=neutral Received: from phobos.denx.de (phobos.denx.de [IPv6:2a01:238:438b:c500:173d:9f52:ddab:ee01]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QvBSj5Rg7z242R for ; Mon, 3 Jul 2023 00:36:21 +1000 (AEST) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id 3744B865B5; Sun, 2 Jul 2023 16:29:11 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="LW3Z5/kw"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id 97AC8866CA; Sun, 2 Jul 2023 16:28:08 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.2 Received: from mail-io1-xd32.google.com (mail-io1-xd32.google.com [IPv6:2607:f8b0:4864:20::d32]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id 1A0DC866CD for ; Sun, 2 Jul 2023 16:27:59 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=sjg@chromium.org Received: by mail-io1-xd32.google.com with SMTP id ca18e2360f4ac-78362f57500so191927939f.3 for ; Sun, 02 Jul 2023 07:27:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1688308077; x=1690900077; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=0GgqjnLJEg8M/ZLAZ01XZdAgIZm3XX5Kn9cAWb0YNTM=; b=LW3Z5/kwFl3vr8ndYomTbGz0m5Jb8VCGCaf7L6J9pG9TC88NtTEsLnmhiCrAvJkVsX 0t2bSTIjFwM3i/X30rN/utAyyTvx73pf/UYijwqail2h4iz9Og8ksrUd5VRQNyF33ZOC 0hy9/D8qzQnVqI4xgSN9NoOzRqjwAZUEaDeOA= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1688308077; x=1690900077; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=0GgqjnLJEg8M/ZLAZ01XZdAgIZm3XX5Kn9cAWb0YNTM=; b=dHr9NvzZaM4WHgEfRK9+qFW7hq2jcBwFprRUKtniDPITYZu+a1PorvPSoVgnmHeFv0 J08RFJ7BBxKuoH9KpOf//elcnHlzijXVuYs8LnplLF8UTEBvyNiTpjkHkSj3yeVubbcx Sd+HfqJ1ni2ph7qVdmok1MPPpwGAk/q/3xgX3mlk8+164NHx81Pl0aG/Q4ZPeQmUpYtN 41ZKZuLY2og4R1OfAAd5jEIfoW8c6x0704eyTjedP+dLPlJPnGqxPtCb0Fx9Xejv/up7 Y/VcFKMsF4ZlRviwGIcZdfAkz/WTPHprS4bGHMkCUCLQBVoDNKkRttApk8ew4CQN2ajn X5tA== X-Gm-Message-State: ABy/qLasHTckfK+SVSUiHcrXxO4CjsB9gNki1zyLO9VuJ9anBQ7vwKvm Xxv0tdT9FzP/BcmhhVyOohapht9CGZY09qR3wRA= X-Google-Smtp-Source: APBJJlGaxxtSk73fMjN7RB5/MCDJqdD2yihRoAnQwt7Pkp7eWCh2P0pyhl13YDJyi1CkiEra5OHSQg== X-Received: by 2002:a92:c608:0:b0:33b:dcbf:e711 with SMTP id p8-20020a92c608000000b0033bdcbfe711mr8750652ilm.8.1688308077603; Sun, 02 Jul 2023 07:27:57 -0700 (PDT) Received: from sjg1.lan (c-73-14-173-85.hsd1.co.comcast.net. [73.14.173.85]) by smtp.gmail.com with ESMTPSA id c13-20020a02c9cd000000b0042af5268e4asm2934713jap.58.2023.07.02.07.27.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 02 Jul 2023 07:27:57 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Cc: Simon Glass Subject: [PATCH 58/58] buildman: Move copy_files() out ot BuilderThread class Date: Sun, 2 Jul 2023 15:26:23 +0100 Message-ID: <20230702142639.1249681-59-sjg@chromium.org> X-Mailer: git-send-email 2.41.0.255.g8b1d071c50-goog In-Reply-To: <20230702142639.1249681-1-sjg@chromium.org> References: <20230702142639.1249681-1-sjg@chromium.org> MIME-Version: 1.0 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.8 at phobos.denx.de X-Virus-Status: Clean This does not need to be in the class. Move it out to avoid a pylint warning. Signed-off-by: Simon Glass --- tools/buildman/builderthread.py | 47 +++++++++++++++++---------------- 1 file changed, 24 insertions(+), 23 deletions(-) diff --git a/tools/buildman/builderthread.py b/tools/buildman/builderthread.py index ed84a0f6baa3..25f460c207db 100644 --- a/tools/buildman/builderthread.py +++ b/tools/buildman/builderthread.py @@ -64,6 +64,27 @@ def _remove_old_outputs(out_dir): os.remove(fname) +def copy_files(out_dir, build_dir, dirname, patterns): + """Copy files from the build directory to the output. + + Args: + out_dir (str): Path to output directory containing the files + build_dir (str): Place to copy the files + dirname (str): Source directory, '' for normal U-Boot, 'spl' for SPL + patterns (list of str): A list of filenames to copy, each relative + to the build directory + """ + for pattern in patterns: + file_list = glob.glob(os.path.join(out_dir, dirname, pattern)) + for fname in file_list: + target = os.path.basename(fname) + if dirname: + base, ext = os.path.splitext(target) + if ext: + target = f'{base}-{dirname}{ext}' + shutil.copy(fname, os.path.join(build_dir, target)) + + # pylint: disable=R0903 class BuilderJob: """Holds information about a job to be performed by a thread @@ -592,7 +613,7 @@ class BuilderThread(threading.Thread): capture_stderr=True, cwd=result.out_dir, raise_on_error=False, env=env) if not work_in_output: - self.copy_files(result.out_dir, build_dir, '', ['uboot.env']) + copy_files(result.out_dir, build_dir, '', ['uboot.env']) # Write out the image sizes file. This is similar to the output # of binutil's 'size' utility, but it omits the header line and @@ -607,7 +628,7 @@ class BuilderThread(threading.Thread): if not work_in_output: # Write out the configuration files, with a special case for SPL for dirname in ['', 'spl', 'tpl']: - self.copy_files( + copy_files( result.out_dir, build_dir, dirname, ['u-boot.cfg', 'spl/u-boot-spl.cfg', 'tpl/u-boot-tpl.cfg', '.config', 'include/autoconf.mk', @@ -615,31 +636,11 @@ class BuilderThread(threading.Thread): # Now write the actual build output if keep_outputs: - self.copy_files( + copy_files( result.out_dir, build_dir, '', ['u-boot*', '*.bin', '*.map', '*.img', 'MLO', 'SPL', 'include/autoconf.mk', 'spl/u-boot-spl*']) - def copy_files(self, out_dir, build_dir, dirname, patterns): - """Copy files from the build directory to the output. - - Args: - out_dir (str): Path to output directory containing the files - build_dir (str): Place to copy the files - dirname (str): Source directory, '' for normal U-Boot, 'spl' for SPL - patterns (list of str): A list of filenames to copy, each relative - to the build directory - """ - for pattern in patterns: - file_list = glob.glob(os.path.join(out_dir, dirname, pattern)) - for fname in file_list: - target = os.path.basename(fname) - if dirname: - base, ext = os.path.splitext(target) - if ext: - target = f'{base}-{dirname}{ext}' - shutil.copy(fname, os.path.join(build_dir, target)) - def _send_result(self, result): """Send a result to the builder for processing