From patchwork Sun Feb 19 16:31:20 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1744709 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=2a01:238:438b:c500:173d:9f52:ddab:ee01; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.a=rsa-sha256 header.s=google header.b=Mruy3VyM; dkim-atps=neutral Received: from phobos.denx.de (phobos.denx.de [IPv6:2a01:238:438b:c500:173d:9f52:ddab:ee01]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4PKWKv0y7Jz23r4 for ; Mon, 20 Feb 2023 03:32:19 +1100 (AEDT) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id 531F98575D; Sun, 19 Feb 2023 17:31:57 +0100 (CET) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="Mruy3VyM"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id 2282885778; Sun, 19 Feb 2023 17:31:36 +0100 (CET) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.2 Received: from mail-il1-x12a.google.com (mail-il1-x12a.google.com [IPv6:2607:f8b0:4864:20::12a]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id 3D48A83623 for ; Sun, 19 Feb 2023 17:31:33 +0100 (CET) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=sjg@chromium.org Received: by mail-il1-x12a.google.com with SMTP id u6so189772ilk.12 for ; Sun, 19 Feb 2023 08:31:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=D/YltqFElwEw7qONMMWVKvgUMqixkjamrq1eD9R7sqY=; b=Mruy3VyMv6kaJb0TVZlp5IiueMr8stLzkgfFe91cbSliTXHMHCgWj72dWMBWWKqlpy i0opbKhyB5XxUoLbWZ4w6BX/ChXFWXeDpk6PfKqzIPdx6uot5xmMlWlgZTiWD5o9RXiB TJPV7hY+SZeJTB+sokvu3shXEOvcYG0AgAlBo= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=D/YltqFElwEw7qONMMWVKvgUMqixkjamrq1eD9R7sqY=; b=WC4LoPkjxcLu7uwibGraOFwP6H7VkVgeNvCMaK8BvoXaWEBUJ9evpOGxERqK7WmTFQ m4lvCyHbxQEmd+YdgmB2tQRDYKJb2wwZdSPlVPWXN2ZnZI6y0TyWj1IE5niHvQN/VW7B BVQzXigTfIEqjMw5ZyIY17eebl1BYNvuq7gQtT4e7Ju740HaXEmCv/efmiPitQ54+3Mt djWYUbplr8L9gLJhfchsnV5Y56tvO6l+bHeklIAXm3lYBCU8PnAYLQRooDWPnwRvJBfC GWqG/lXRcum4934ueGxFud43pjZ3Z3RnoHrWBC3vHn2aB2+8LUqS6fqRZKTDhT67MWBN y5NA== X-Gm-Message-State: AO0yUKU9ADRPLzQA/tjhjbF4T8tQ6Ihuas7guW9qS/sw2PS2UYwZUs5O lxmWCSoXt51qYYe8/jeJcUaDDQUDXoNRm6Lc X-Google-Smtp-Source: AK7set9CoRQCg5CMXUeVZl91fobeXIXpRYI9OuPJ+xtC6qKQrwg4RGIRjCRB5M3OoVlm+Qx+s+2axg== X-Received: by 2002:a92:7a0a:0:b0:315:9749:7a25 with SMTP id v10-20020a927a0a000000b0031597497a25mr388316ilc.17.1676824291697; Sun, 19 Feb 2023 08:31:31 -0800 (PST) Received: from sjg1.roam.corp.google.com (c-73-14-173-85.hsd1.co.comcast.net. [73.14.173.85]) by smtp.gmail.com with ESMTPSA id l22-20020a02a896000000b003a0565a5750sm3211922jam.119.2023.02.19.08.31.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 19 Feb 2023 08:31:31 -0800 (PST) From: Simon Glass To: U-Boot Mailing List Cc: Doug Anderson , Simon Glass , Maxim Cournoyer Subject: [PATCH 1/4] patman: Drop an uncorrect comment about git am Date: Sun, 19 Feb 2023 09:31:20 -0700 Message-Id: <20230219163124.1797834-2-sjg@chromium.org> X-Mailer: git-send-email 2.39.2.637.g21b0678d19-goog In-Reply-To: <20230219163124.1797834-1-sjg@chromium.org> References: <20230219163124.1797834-1-sjg@chromium.org> MIME-Version: 1.0 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.6 at phobos.denx.de X-Virus-Status: Clean Patman does not do this anymore, so drop the comment. Signed-off-by: Simon Glass --- tools/patmanu/control.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tools/patmanu/control.py b/tools/patmanu/control.py index b1e23870d9d..6ff94776da9 100644 --- a/tools/patmanu/control.py +++ b/tools/patmanu/control.py @@ -85,7 +85,7 @@ def check_patches(series, patch_files, run_checkpatch, verbose, use_tree): # Do a few checks on the series series.DoChecks() - # Check the patches, and run them through 'git am' just to be sure + # Check the patches if run_checkpatch: ok = checkpatch.check_patches(verbose, patch_files, use_tree) else: From patchwork Sun Feb 19 16:31:21 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1744712 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=2a01:238:438b:c500:173d:9f52:ddab:ee01; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.a=rsa-sha256 header.s=google header.b=O+J112aZ; dkim-atps=neutral Received: from phobos.denx.de (phobos.denx.de [IPv6:2a01:238:438b:c500:173d:9f52:ddab:ee01]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4PKWLm6Vvnz23r4 for ; Mon, 20 Feb 2023 03:33:04 +1100 (AEDT) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id ADDE18547B; Sun, 19 Feb 2023 17:33:01 +0100 (CET) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="O+J112aZ"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id 41B4283623; Sun, 19 Feb 2023 17:31:43 +0100 (CET) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.2 Received: from mail-il1-x12c.google.com (mail-il1-x12c.google.com [IPv6:2607:f8b0:4864:20::12c]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id E39968572C for ; Sun, 19 Feb 2023 17:31:33 +0100 (CET) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=sjg@chromium.org Received: by mail-il1-x12c.google.com with SMTP id u6so189776ilk.12 for ; Sun, 19 Feb 2023 08:31:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=zIju0ShcJn0VKmPcgLllzXyZUOex/lpEn2xL8o9GJwc=; b=O+J112aZ9v2i7GryVw5tGOjvcNDKl/NQwbkkm7hG7Sy69FojOHcvmu/TYrBwdDW1p0 nThlcwyV9iQLKEpzWr/OmtFU6wLeOTfoViadI2r/N8mpVwgRKqiyARcJo9dmL+Ltc6xN m5tkrnwLtEYnl9VPIyAy/Kz4ZFpTnhAvxKJys= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=zIju0ShcJn0VKmPcgLllzXyZUOex/lpEn2xL8o9GJwc=; b=Z+0Khj+D0QMy4d1TutchVbhlb7nFwbPIQgBtls6nRykpYVZ3Uby3bk8NFvrzq9KEm5 N6eEgUrJZRpQ/X06nffsGx75upB16n12rAV0YN6YExKslldHAE0tyNzEOOBj5y988fBZ lTC4Jj6nkEB3w98zcjhIBc6vN9Fd7SFpTiRZ8kPLIc7Vh7PhwrBBsqzCxkauUAWVAN6g iiMGhEAsADo8HmNU8WN5CE4zLRz+4e+yGfB4N2ULfXavl/2z474OBf70hDjkGf+Bju/J h9xUtjaWd2lCMbuB/rRziM3WtgyrD5y74f7na4JEyWnY4iVm/xBpW1n2i4nc2rWlEkbA 6OgA== X-Gm-Message-State: AO0yUKUOZ9FLZ5WRjD0m7T+9l8butKa2YPTNEpXmSgwc/H7ioFn18ls9 LQtPSUlSH6mrGNndkT90V+verkGHr82mG9Y3 X-Google-Smtp-Source: AK7set8jbIC5h1ran0r8GvA4lDUzlxTmY1TZXi+OjaCqqMdR6+47G+O9VKzlYs1vQDIn3ibr3GOIcQ== X-Received: by 2002:a05:6e02:1d10:b0:315:7911:748a with SMTP id i16-20020a056e021d1000b003157911748amr618129ila.1.1676824292429; Sun, 19 Feb 2023 08:31:32 -0800 (PST) Received: from sjg1.roam.corp.google.com (c-73-14-173-85.hsd1.co.comcast.net. [73.14.173.85]) by smtp.gmail.com with ESMTPSA id l22-20020a02a896000000b003a0565a5750sm3211922jam.119.2023.02.19.08.31.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 19 Feb 2023 08:31:32 -0800 (PST) From: Simon Glass To: U-Boot Mailing List Cc: Doug Anderson , Simon Glass , Maxim Cournoyer Subject: [PATCH 2/4] patman: Refactor MakeCcFile() into two functions Date: Sun, 19 Feb 2023 09:31:21 -0700 Message-Id: <20230219163124.1797834-3-sjg@chromium.org> X-Mailer: git-send-email 2.39.2.637.g21b0678d19-goog In-Reply-To: <20230219163124.1797834-1-sjg@chromium.org> References: <20230219163124.1797834-1-sjg@chromium.org> MIME-Version: 1.0 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.6 at phobos.denx.de X-Virus-Status: Clean This function is quite long. Moving the handling of a commit into a separate function. This will make it easier to do the work in parallel. Update function comments while we are here. Signed-off-by: Simon Glass --- tools/patmanu/series.py | 80 ++++++++++++++++++++++++++++------------- 1 file changed, 56 insertions(+), 24 deletions(-) diff --git a/tools/patmanu/series.py b/tools/patmanu/series.py index f2d415609d2..849733a9d29 100644 --- a/tools/patmanu/series.py +++ b/tools/patmanu/series.py @@ -234,6 +234,48 @@ class Series(dict): str = 'Change log exists, but no version is set' print(col.build(col.RED, str)) + def GetCcForCommit(self, commit, process_tags, warn_on_error, + add_maintainers, limit, get_maintainer_script, + all_skips): + """Get the email CCs to use with a particular commit + + Uses subject tags and get_maintainers.pl script to find people to cc + on a patch + + Args: + commit (Commit): Commit to process + process_tags (bool): Process tags as if they were aliases + warn_on_error (bool): True to print a warning when an alias fails to + match, False to ignore it. + add_maintainers (bool or list of str): Either: + True/False to call the get_maintainers to CC maintainers + List of maintainers to include (for testing) + limit (int): Limit the length of the Cc list (None if no limit) + get_maintainer_script (str): The file name of the get_maintainer.pl + script (or compatible). + all_skips (set of str): Set of bouncing email address that were + dropped from the output + + Returns: + list of str: List of email addresses to cc + """ + cc = [] + if process_tags: + cc += gitutil.build_email_list(commit.tags, + warn_on_error=warn_on_error) + cc += gitutil.build_email_list(commit.cc_list, + warn_on_error=warn_on_error) + if type(add_maintainers) == type(cc): + cc += add_maintainers + elif add_maintainers: + cc += get_maintainer.get_maintainer(get_maintainer_script, + commit.patch) + all_skips |= set(cc) & set(settings.bounces) + cc = list(set(cc) - set(settings.bounces)) + if limit is not None: + cc = cc[:limit] + return cc + def MakeCcFile(self, process_tags, cover_fname, warn_on_error, add_maintainers, limit, get_maintainer_script): """Make a cc file for us to use for per-commit Cc automation @@ -241,15 +283,15 @@ class Series(dict): Also stores in self._generated_cc to make ShowActions() faster. Args: - process_tags: Process tags as if they were aliases - cover_fname: If non-None the name of the cover letter. - warn_on_error: True to print a warning when an alias fails to match, - False to ignore it. - add_maintainers: Either: + process_tags (bool): Process tags as if they were aliases + cover_fname (str): If non-None the name of the cover letter. + warn_on_error (bool): True to print a warning when an alias fails to + match, False to ignore it. + add_maintainers (bool or list of str): Either: True/False to call the get_maintainers to CC maintainers List of maintainers to include (for testing) - limit: Limit the length of the Cc list (None if no limit) - get_maintainer_script: The file name of the get_maintainer.pl + limit (int): Limit the length of the Cc list (None if no limit) + get_maintainer_script (str): The file name of the get_maintainer.pl script (or compatible). Return: Filename of temp file created @@ -259,28 +301,18 @@ class Series(dict): fname = '/tmp/patman.%d' % os.getpid() fd = open(fname, 'w', encoding='utf-8') all_ccs = [] + all_skips = set() for commit in self.commits: - cc = [] - if process_tags: - cc += gitutil.build_email_list(commit.tags, - warn_on_error=warn_on_error) - cc += gitutil.build_email_list(commit.cc_list, - warn_on_error=warn_on_error) - if type(add_maintainers) == type(cc): - cc += add_maintainers - elif add_maintainers: - - cc += get_maintainer.get_maintainer(get_maintainer_script, - commit.patch) - for x in set(cc) & set(settings.bounces): - print(col.build(col.YELLOW, 'Skipping "%s"' % x)) - cc = list(set(cc) - set(settings.bounces)) - if limit is not None: - cc = cc[:limit] + cc = self.GetCcForCommit(commit, process_tags, warn_on_error, + add_maintainers, limit, + get_maintainer_script, all_skips) all_ccs += cc print(commit.patch, '\0'.join(sorted(set(cc))), file=fd) self._generated_cc[commit.patch] = cc + for x in sorted(list(all_skips)): + print(col.build(col.YELLOW, 'Skipping "{x}"')) + if cover_fname: cover_cc = gitutil.build_email_list(self.get('cover_cc', '')) cover_cc = list(set(cover_cc + all_ccs)) From patchwork Sun Feb 19 16:31:22 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1744710 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=2a01:238:438b:c500:173d:9f52:ddab:ee01; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.a=rsa-sha256 header.s=google header.b=n1tVCNX3; dkim-atps=neutral Received: from phobos.denx.de (phobos.denx.de [IPv6:2a01:238:438b:c500:173d:9f52:ddab:ee01]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4PKWL32ZrWz23r4 for ; Mon, 20 Feb 2023 03:32:27 +1100 (AEDT) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id B39FC8584B; Sun, 19 Feb 2023 17:32:15 +0100 (CET) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="n1tVCNX3"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id B1BE8857B1; Sun, 19 Feb 2023 17:31:39 +0100 (CET) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.2 Received: from mail-io1-xd2f.google.com (mail-io1-xd2f.google.com [IPv6:2607:f8b0:4864:20::d2f]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id 8F38785748 for ; Sun, 19 Feb 2023 17:31:34 +0100 (CET) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=sjg@chromium.org Received: by mail-io1-xd2f.google.com with SMTP id c21so410626iob.12 for ; Sun, 19 Feb 2023 08:31:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=9ti6p4iSys/lgo6Zj2NCyKEfIj5qwaef2ZWpsBKTWo8=; b=n1tVCNX3P/DLC+3wQDNkmjzCn0i088RHoql7AdKAHtjx7XB1U4Jh2XxX1kU/t/DhnH nUS+RGTz/hnPMmIb1r346krT/GJNMtN96OAS/1l3lOpRsFQLp6e5Ur19HQH8d8OJjp2B HmLLVxKjGJP5GAqjeSKS7015MOOsApf91mGVo= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=9ti6p4iSys/lgo6Zj2NCyKEfIj5qwaef2ZWpsBKTWo8=; b=Xhxr8GbRswSSN+RRt62xe4ahQYlbwe/aPCVek2jrxmRvs2zOY7K2vVzjOCJvH41UIa /A6b7xlQMCTUgOCs/z8PMP/WiFGS0CEO+EqrZCGLv1bQ6hOiWhn4ven+mBdNH6FfrJG3 6vAfzqhPrGccg3IadQe/SFcI7HEs6cZGOXWvJR4XAwFWhHVvsNcDs5SQbqhanlV+btuy UzQv30u/9QbcVyMtl9rH+XnKjNTHDXLxsB/znaMGMl9b2voPzeA9zlf9VSKJ69E/pXAr B4d6/ycL2E7eKPuxhKHfhtRivFVgw9EtpxdVBZW3qPcNT2hgWY3bphq+g9xrLskxev1X I5Ow== X-Gm-Message-State: AO0yUKUl539YJynPgBXtTCslnpjoPtozY0sBOBuWstX/Y6wh6vXskXZC 4TXh+zVt98OiDRx2imYFjWqQ3x65wTG6ZN6v X-Google-Smtp-Source: AK7set8TGRy5SSyxLFUPcoOGBAQWaO5XWBosf/g4MLWRIrOxq8TI1S6mVpKRuR8UOcJssoZiIQuAAw== X-Received: by 2002:a5e:dc05:0:b0:719:62d5:ed6c with SMTP id b5-20020a5edc05000000b0071962d5ed6cmr5280771iok.18.1676824293133; Sun, 19 Feb 2023 08:31:33 -0800 (PST) Received: from sjg1.roam.corp.google.com (c-73-14-173-85.hsd1.co.comcast.net. [73.14.173.85]) by smtp.gmail.com with ESMTPSA id l22-20020a02a896000000b003a0565a5750sm3211922jam.119.2023.02.19.08.31.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 19 Feb 2023 08:31:32 -0800 (PST) From: Simon Glass To: U-Boot Mailing List Cc: Doug Anderson , Simon Glass , Maxim Cournoyer Subject: [PATCH 3/4] patman: Run get_maintainer.pl in parallel Date: Sun, 19 Feb 2023 09:31:22 -0700 Message-Id: <20230219163124.1797834-4-sjg@chromium.org> X-Mailer: git-send-email 2.39.2.637.g21b0678d19-goog In-Reply-To: <20230219163124.1797834-1-sjg@chromium.org> References: <20230219163124.1797834-1-sjg@chromium.org> MIME-Version: 1.0 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.6 at phobos.denx.de X-Virus-Status: Clean This script can take ages on some series. Try to limit the time by using threads. If a few stubborn patches remain, show progress so the user has some idea what is going on. Signed-off-by: Simon Glass --- tools/patmanu/func_test.py | 2 ++ tools/patmanu/series.py | 33 ++++++++++++++++++++++++++++++--- 2 files changed, 32 insertions(+), 3 deletions(-) diff --git a/tools/patmanu/func_test.py b/tools/patmanu/func_test.py index 238fd5b6100..48109ae5725 100644 --- a/tools/patmanu/func_test.py +++ b/tools/patmanu/func_test.py @@ -240,6 +240,8 @@ class TestFunctional(unittest.TestCase): self.assertEqual('Change log missing for v3', next(lines)) self.assertEqual('Change log for unknown version v4', next(lines)) self.assertEqual("Alias 'pci' not found", next(lines)) + while next(lines) != 'Cc processing complete': + pass self.assertIn('Dry run', next(lines)) self.assertEqual('', next(lines)) self.assertIn('Send a total of %d patches' % count, next(lines)) diff --git a/tools/patmanu/series.py b/tools/patmanu/series.py index 849733a9d29..c8b09dc24fd 100644 --- a/tools/patmanu/series.py +++ b/tools/patmanu/series.py @@ -5,8 +5,11 @@ from __future__ import print_function import collections +import concurrent.futures import itertools import os +import sys +import time from patmanu import get_maintainer from patmanu import gitutil @@ -302,10 +305,34 @@ class Series(dict): fd = open(fname, 'w', encoding='utf-8') all_ccs = [] all_skips = set() + with concurrent.futures.ThreadPoolExecutor(max_workers=16) as executor: + for i, commit in enumerate(self.commits): + commit.seq = i + commit.future = executor.submit( + self.GetCcForCommit, commit, process_tags, warn_on_error, + add_maintainers, limit, get_maintainer_script, all_skips) + + # Show progress any commits that are taking forever + lastlen = 0 + while True: + left = [commit for commit in self.commits + if not commit.future.done()] + if not left: + break + names = ', '.join(f'{c.seq + 1}:{c.subject}' + for c in left[:2]) + out = f'\r{len(left)} remaining: {names}'[:79] + spaces = ' ' * (lastlen - len(out)) + if lastlen: # Don't print anything the first time + print(out, spaces, end='') + sys.stdout.flush() + lastlen = len(out) + time.sleep(.25) + print(f'\rdone{" " * lastlen}\r', end='') + print('Cc processing complete') + for commit in self.commits: - cc = self.GetCcForCommit(commit, process_tags, warn_on_error, - add_maintainers, limit, - get_maintainer_script, all_skips) + cc = commit.future.result() all_ccs += cc print(commit.patch, '\0'.join(sorted(set(cc))), file=fd) self._generated_cc[commit.patch] = cc From patchwork Sun Feb 19 16:31:23 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1744711 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=2a01:238:438b:c500:173d:9f52:ddab:ee01; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.a=rsa-sha256 header.s=google header.b=iYwsi/0c; dkim-atps=neutral Received: from phobos.denx.de (phobos.denx.de [IPv6:2a01:238:438b:c500:173d:9f52:ddab:ee01]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4PKWLN57J1z23r4 for ; Mon, 20 Feb 2023 03:32:44 +1100 (AEDT) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id AFD1E857E0; Sun, 19 Feb 2023 17:32:26 +0100 (CET) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="iYwsi/0c"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id 1248180ADF; Sun, 19 Feb 2023 17:31:40 +0100 (CET) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.2 Received: from mail-io1-xd32.google.com (mail-io1-xd32.google.com [IPv6:2607:f8b0:4864:20::d32]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id 412EA83523 for ; Sun, 19 Feb 2023 17:31:35 +0100 (CET) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=sjg@chromium.org Received: by mail-io1-xd32.google.com with SMTP id w22so160266ioc.0 for ; Sun, 19 Feb 2023 08:31:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=dGbPmHUhs/LgKdn/Ul+bgNnBHKG4/8HvJMrnJGj15/U=; b=iYwsi/0cSScyhvuHueZ9cDL1mRCzqbFJRKM+4qTwHiE3G4oD1VqybRtu3xZnXRxXO2 buKJPBA7WkqvZt/BGQJf4gj+ul5fzPM5BFE84Q26oqKjrwRjtbEZBRSOE7EaakVrzLIq ff+5/fsT9coc/0WzFg8w6/axgEofFtIGPal3s= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=dGbPmHUhs/LgKdn/Ul+bgNnBHKG4/8HvJMrnJGj15/U=; b=N8B/QdVoqKMwiwdwAFXgp+ihB7ShoI+3MH7Zqt7UQBe+XsWQlUBVHw61Zkz0/UhGAM 5Wz8ZiNUr2cYkj34J2tDWbjPeseGK0P2VIQd23VxLwcD+O1pSm7p/bRij3aPyJPNb//N F8Zby+sjnDLiFshbSjCl0oScfILg37mwO9fjSymnnQ5n2XkxoOfAXxDsrFe+m0gvi5bA iW4iuOMy1N9d9HWfpwIeypYYP2CFcFfQ51bq4gTGqDlnKbK1Pojy7uV5BbvI1uXkehhn laS98Wcezg9feP1xx+gaP4vy8WrdeHrXKsNOIobDgzg8CInaWczr6OMsaoJVRRcJR3uH UWNw== X-Gm-Message-State: AO0yUKUJhrGiY1INXrplYEYKZ7ILYh+KMHxwZ0DV+Ff+yirKS7Rbnj2u KMe1kQDDncuhtZy7GVUYEmU9oPUWldjvBOxw X-Google-Smtp-Source: AK7set+hlHVPQ3gRzOM9VHT17crLkLAtuXN3+bnw7TAAbHZn0Ys/IZrIWjc99+lFrUocu5PcK0luHA== X-Received: by 2002:a05:6602:22c4:b0:704:d17c:7dc1 with SMTP id e4-20020a05660222c400b00704d17c7dc1mr6072367ioe.11.1676824293852; Sun, 19 Feb 2023 08:31:33 -0800 (PST) Received: from sjg1.roam.corp.google.com (c-73-14-173-85.hsd1.co.comcast.net. [73.14.173.85]) by smtp.gmail.com with ESMTPSA id l22-20020a02a896000000b003a0565a5750sm3211922jam.119.2023.02.19.08.31.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 19 Feb 2023 08:31:33 -0800 (PST) From: Simon Glass To: U-Boot Mailing List Cc: Doug Anderson , Simon Glass , Maxim Cournoyer Subject: [PATCH 4/4] patman: Check patches in parallel Date: Sun, 19 Feb 2023 09:31:23 -0700 Message-Id: <20230219163124.1797834-5-sjg@chromium.org> X-Mailer: git-send-email 2.39.2.637.g21b0678d19-goog In-Reply-To: <20230219163124.1797834-1-sjg@chromium.org> References: <20230219163124.1797834-1-sjg@chromium.org> MIME-Version: 1.0 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.6 at phobos.denx.de X-Virus-Status: Clean For large series this can take a while. Run checkpatch in parallel to try to reduce the time. The checkpatch information is still reported in sequential order, so a very slow patch at the start can still slow things down. But overall this gives good results. Signed-off-by: Simon Glass --- tools/patmanu/checkpatch.py | 46 +++++++++++++++++++++---------------- 1 file changed, 26 insertions(+), 20 deletions(-) diff --git a/tools/patmanu/checkpatch.py b/tools/patmanu/checkpatch.py index 55d962f536f..1317f50b464 100644 --- a/tools/patmanu/checkpatch.py +++ b/tools/patmanu/checkpatch.py @@ -3,6 +3,7 @@ # import collections +import concurrent.futures import os import re import sys @@ -244,26 +245,31 @@ def check_patches(verbose, args, use_tree): error_count, warning_count, check_count = 0, 0, 0 col = terminal.Color() - for fname in args: - result = check_patch(fname, verbose, use_tree=use_tree) - if not result.ok: - error_count += result.errors - warning_count += result.warnings - check_count += result.checks - print('%d errors, %d warnings, %d checks for %s:' % (result.errors, - result.warnings, result.checks, col.build(col.BLUE, fname))) - if (len(result.problems) != result.errors + result.warnings + - result.checks): - print("Internal error: some problems lost") - # Python seems to get confused by this - # pylint: disable=E1133 - for item in result.problems: - sys.stderr.write( - get_warning_msg(col, item.get('type', ''), - item.get('file', ''), - item.get('line', 0), item.get('msg', 'message'))) - print - #print(stdout) + with concurrent.futures.ThreadPoolExecutor(max_workers=16) as executor: + futures = [] + for fname in args: + f = executor.submit(check_patch, fname, verbose, use_tree=use_tree) + futures.append(f) + + for fname, f in zip(args, futures): + result = f.result() + if not result.ok: + error_count += result.errors + warning_count += result.warnings + check_count += result.checks + print('%d errors, %d warnings, %d checks for %s:' % (result.errors, + result.warnings, result.checks, col.build(col.BLUE, fname))) + if (len(result.problems) != result.errors + result.warnings + + result.checks): + print("Internal error: some problems lost") + # Python seems to get confused by this + # pylint: disable=E1133 + for item in result.problems: + sys.stderr.write( + get_warning_msg(col, item.get('type', ''), + item.get('file', ''), + item.get('line', 0), item.get('msg', 'message'))) + print if error_count or warning_count or check_count: str = 'checkpatch.pl found %d error(s), %d warning(s), %d checks(s)' color = col.GREEN