contrib/gcc-changelog/git_update_version.py: Improve diagnostic (was: [Patch] contrib/gcc-changelog/git_update_version.py: Add ignore commit, improve diagnostic)

Hmm, there were now two daily bumps:

Date:   Mon May 20 00:16:30 2024 +0000

Date:   Sun May 19 18:15:28 2024 +0000

I really wonder why.

I guess, the 'ignore commit' is hence not needed – but I think the 
improved diagnostic part still makes sense.

See updated patch.

On May 19, 24 Tobias Burnus wrote:
> I noticed that the last bump happened on Thursday.
>
> * * *
>
> The error is according to
> https://gcc.gnu.org/pipermail/gccadmin/2024q2/021298.html
>
> 2024-05-19 00:17:28,643:INFO:root:cannot find a ChangeLog location in 
> message
>
> That's the commit
> ---------------
>     Revert "Revert: "Enable prange support.""
>
>     This reverts commit d7bb8eaade3cd3aa70715c8567b4d7b08098e699 and 
> enables prange
>     support again.
> ---------------
>
> * * * The attached patch adds this commit to the ignore list and helps 
> with the diagnosis by showing the failing hash in the error message.
>
> OK for mainline?
>
> Post commit: Can someone install the new version + fix the ChangeLog 
> for the ignored commit?
>
> * * *
>
> What I do not understand: Why does this commit get applied? I do see 
> for both
>
> contrib/gcc-changelog/git_check_commit.py -v -p 
> da73261ce7731be7f2b164f1db796878cdc23365
>
> and
>
> contrib/gcc-changelog/git_email.py 
> 0001-Revert-Revert-Enable-prange-support.patch the error above. - And 
> I do not understand why it made it past the commit check but now fails?
>
> Likewise for8057f9aa1f7e70490064de796d7a8d42d446caf8
>
> Does the commit hook use an older version of the check scripts? Does 
> it ignore the errors? Or what goes wrong here? Any idea?

Tobias

On Mon, May 20, 2024 at 08:31:02AM +0200, Tobias Burnus wrote:
> Hmm, there were now two daily bumps:
> 
> Date:   Mon May 20 00:16:30 2024 +0000
> 
> Date:   Sun May 19 18:15:28 2024 +0000
> 
> I really wonder why.

Because I've done it by hand.
I have in ~gccadmin a gcc-changelog copy and adjusted update_version_git
script which doesn't use contrib/gcc-changelog subdirectory from the
checkout it makes but from the ~gccadmin directory, because I don't want to
constantly try to add some commit number to IGNORED_COMMITS, see that it
either works or doesn't (I think sometimes it needs the hash of the revert
commit, at other times the commit hash referenced in the revert commit)
or that further ones are needed.

> From f56b1764f2b5c2c83c6852607405e5be0a763a2c Mon Sep 17 00:00:00 2001
> From: Tobias Burnus <tburnus@baylibre.com>
> Date: Sun, 19 May 2024 08:17:42 +0200
> Subject: [PATCH] contrib/gcc-changelog/git_update_version.py: Improve diagnostic
> 
> contrib/ChangeLog:
> 
>         * gcc-changelog/git_update_version.py (prepend_to_changelog_files): Output

8 spaces rather than tab

> 	git hash in case errors occurred.
> 
> diff --git a/contrib/gcc-changelog/git_update_version.py b/contrib/gcc-changelog/git_update_version.py
> index 24f6c43d0b2..ec0151b83fe 100755
> --- a/contrib/gcc-changelog/git_update_version.py
> +++ b/contrib/gcc-changelog/git_update_version.py
> @@ -58,6 +58,7 @@ def read_timestamp(path):
>  
>  def prepend_to_changelog_files(repo, folder, git_commit, add_to_git):
>      if not git_commit.success:
> +        logging.info(f"While processing {git_commit.info.hexsha}:")
>          for error in git_commit.errors:
>              logging.info(error)
>          raise AssertionError()

So, your commit is useful part of it, I'm already using something similar in
my hack (just was doing it for even successful commits, but I think your
patch is better).
And, I think best would be if update_version_git script simply
accepted a list of ignored commits from the command line too,
passed it to the git_update_version.py script and that one
added those to IGNORED_COMMITS.
Because typically if the DATESTAMP/ChangeLog updates gets stuck,
one doesn't just adjust IGNORED_COMMITS and wait up to another
day to see if it worked, but runs the script by hand to make sure
it works.

--- gcc-checkout/contrib/gcc-changelog/git_update_version.py	2024-05-13 16:52:57.890151748 +0000
+++ gcc-changelog/git_update_version.py	2024-05-19 18:13:44.953648834 +0000
@@ -41,7 +41,21 @@ IGNORED_COMMITS = (
         '040e5b0edbca861196d9e2ea2af5e805769c8d5d',
         '8057f9aa1f7e70490064de796d7a8d42d446caf8',
         '109f1b28fc94c93096506e3df0c25e331cef19d0',
-        '39f81924d88e3cc197fc3df74204c9b5e01e12f7')
+        '39f81924d88e3cc197fc3df74204c9b5e01e12f7',
+        'd7bb8eaade3cd3aa70715c8567b4d7b08098e699',
+        '89feb3557a018893cfe50c2e07f91559bd3cde2b',
+        'ccf8d3e3d26c6ba3d5e11fffeed8d64018e9c060',
+        'e0c52905f666e3d23881f82dbf39466a24f009f4',
+        'b38472ffc1e631bd357573b44d956ce16d94e666',
+        'a0b13d0860848dd5f2876897ada1e22e4e681e91',
+        'b8c772cae97b54386f7853edf0f9897012bfa90b',
+        '810d35a7e054bcbb5b66d2e5924428e445f5fba9',
+        '0df1ee083434ac00ecb19582b1e5b25e105981b2',
+        '2c688f6afce4cbb414f5baab1199cd525f309fca',
+        '60dcb710b6b4aa22ea96abc8df6dfe9067f3d7fe',
+        '44968a0e00f656e9bb3e504bb2fa1a8282002015',
+        'd7bb8eaade3cd3aa70715c8567b4d7b08098e699',
+        'da73261ce7731be7f2b164f1db796878cdc23365')
 
 FORMAT = '%(asctime)s:%(levelname)s:%(name)s:%(message)s'
 logging.basicConfig(level=logging.INFO, format=FORMAT,
@@ -125,6 +139,7 @@ def update_current_branch(ref_name):
                                       % (commit.hexsha, head.hexsha), ref_name)
         commits = [c for c in commits if c.info.hexsha not in IGNORED_COMMITS]
         for git_commit in reversed(commits):
+            logging.info('trying %s', git_commit.info.hexsha)
             prepend_to_changelog_files(repo, args.git_path, git_commit,
                                        not args.dry_mode)
         if args.dry_mode:

	Jakub

Hi Jakub,

Jakub Jelinek wrote:
> On Mon, May 20, 2024 at 08:31:02AM +0200, Tobias Burnus wrote:
>> Hmm, there were now two daily bumps: [...] I really wonder why.
> Because I've done it by hand.

Okay, that explains it.

I still do not understand why it slipped through at the first place; I 
tried old versions down to r12-709-g772e5e82e3114f and it still FAIL for 
the invalid commit ("ERR: cannot find a ChangeLog location in message").

Thus, I wonder whether the commit hook is active at all?!?

> I have in ~gccadmin a gcc-changelog copy and adjusted update_version_git
> script which doesn't use contrib/gcc-changelog subdirectory from the
> checkout it makes but from the ~gccadmin directory,
[...]
> I'm already using something similar in
> my hack (just was doing it for even successful commits, but I think your
> patch is better).
> And, I think best would be if update_version_git script simply
> accepted a list of ignored commits from the command line too,
> passed it to the git_update_version.py script and that one
> added those to IGNORED_COMMITS.

Updated version:

* Uses my diagnostic

* Adds an -i/--ignore argument for commits. Permits to use '-i hash1  -i 
hash2' but also '-i hash1,hash2' or '-i "hash1 hash2'

* I changed the global variable to lower case as Python's style guide 
states that all uppercase variables is for constants.

* The '=None' matches one of the current usages (no argument passed); 
hence, it is now explicit and 'pylint' is happy.

OK for mainline?

Tobias

PS: I have not updated the hashes. If needed/wanted, I leave that to 
you, Jakub.

On Tue, May 21, 2024 at 09:36:05AM +0200, Tobias Burnus wrote:
> Jakub Jelinek wrote:
> > On Mon, May 20, 2024 at 08:31:02AM +0200, Tobias Burnus wrote:
> > > Hmm, there were now two daily bumps: [...] I really wonder why.
> > Because I've done it by hand.
> 
> Okay, that explains it.
> 
> I still do not understand why it slipped through at the first place; I tried
> old versions down to r12-709-g772e5e82e3114f and it still FAIL for the
> invalid commit ("ERR: cannot find a ChangeLog location in message").
> 
> Thus, I wonder whether the commit hook is active at all?!?

They are.  But
https://github.com/AdaCore/git-hooks/blob/master/hooks/updates/__init__.py#L836
with
https://github.com/AdaCore/git-hooks/blob/master/hooks/updates/commits.py#L230
bypasses all commits which contain just 3 magic words in a row.
And because that part is owned by AdaCore hooks, not the GCC customizations,
not sure what to do about that.

> > I have in ~gccadmin a gcc-changelog copy and adjusted update_version_git
> > script which doesn't use contrib/gcc-changelog subdirectory from the
> > checkout it makes but from the ~gccadmin directory,
> [...]
> > I'm already using something similar in
> > my hack (just was doing it for even successful commits, but I think your
> > patch is better).
> > And, I think best would be if update_version_git script simply
> > accepted a list of ignored commits from the command line too,
> > passed it to the git_update_version.py script and that one
> > added those to IGNORED_COMMITS.
> 
> Updated version:
> 
> * Uses my diagnostic
> 
> * Adds an -i/--ignore argument for commits. Permits to use '-i hash1  -i
> hash2' but also '-i hash1,hash2' or '-i "hash1 hash2'
> 
> * I changed the global variable to lower case as Python's style guide states
> that all uppercase variables is for constants.
> 
> * The '=None' matches one of the current usages (no argument passed); hence,
> it is now explicit and 'pylint' is happy.
> 
> OK for mainline?

Yes, thanks.

> PS: I have not updated the hashes. If needed/wanted, I leave that to you,
> Jakub.

Once some commit is ignored, we won't be processing it anymore, so I think
the -i option is all we need.

	Jakub