| Message ID | 20240116111213.42899-1-iain@sandoe.co.uk |
|---|---|
| State | New |
| Headers | show |
| Series | testsuite, jit: Stabilize error output. | expand |
gentle ping, with the increasing use of CI, it seems an idea to tackle this sooner rather than later. thanks Iain > On 16 Jan 2024, at 11:12, Iain Sandoe <iains.gcc@gmail.com> wrote: > > Tested on x86_64, i686 Darwin, x86_64 Linux, > OK for trunk? When? > thanks > Iain > > --- 8< --- > > Currently when a test fails, we print out a lot of information, > this includes items that are not stable between invocations (e.g. > the PID for the executable). That makes automated comparisons > between test runs flag any persistent fails as new ones each time > which is not usually what is wanted. > > This patch amends the error output to drop the variable portion > of the message and retain items that should only change if the > failure mode changes. > > gcc/testsuite/ChangeLog: > > * jit.dg/jit.exp: Filter error output to remove per-run > variable content. > > Signed-off-by: Iain Sandoe <iain@sandoe.co.uk> > --- > gcc/testsuite/jit.dg/jit.exp | 21 +++++++++++++++------ > 1 file changed, 15 insertions(+), 6 deletions(-) > > diff --git a/gcc/testsuite/jit.dg/jit.exp b/gcc/testsuite/jit.dg/jit.exp > index 286cfa8192a..893ff5f6dd0 100644 > --- a/gcc/testsuite/jit.dg/jit.exp > +++ b/gcc/testsuite/jit.dg/jit.exp > @@ -94,25 +94,34 @@ proc parse_valgrind_logfile {name logfile} { > # unexpected exits. > > proc verify_exit_status { executable wres } { > - lassign $wres pid spawnid os_error_flag value > + set extra [lassign $wres pid spawnid os_error_flag value] > verbose "pid: $pid" 3 > verbose "spawnid: $spawnid" 3 > verbose "os_error_flag: $os_error_flag" 3 > verbose "value: $value" 3 > > # Detect segfaults etc: > - if { [llength $wres] > 4 } { > - if { [lindex $wres 4] == "CHILDKILLED" } { > - fail "$executable killed: $wres" > + set len [llength $extra] > + if { $len >= 1 } { > + if { [lindex $extra 0] == "CHILDKILLED" } { > + set reason "Unknown Reason" > + set detail "No Details" > + if { $len >= 2 } { > + set reason [lindex $extra 1] > + if { $len >= 3 } { > + set detail [lindex $extra 2] > + } > + } > + fail "$executable killed: $reason $detail" > return > } > } > if { $os_error_flag != 0 } { > - fail "$executable: OS error: $wres" > + fail "$executable: OS error: $os_error_flag $extra" > return > } > if { $value != 0 } { > - fail "$executable: non-zero exit code: $wres" > + fail "$executable: non-zero exit code: $value $extra" > return > } > pass "$executable exited cleanly" > -- > 2.39.2 (Apple Git-143) >
On Tue, 2024-01-16 at 11:12 +0000, Iain Sandoe wrote: > Tested on x86_64, i686 Darwin, x86_64 Linux, > OK for trunk? When? > thanks > Iain Thanks; looks good to me for trunk. Given that the scope is just the jit testsuite and that you've tested it on 3 configurations (and presumably made use of this when debugging the other issue), I think this is OK to go in now. Dave > > --- 8< --- > > Currently when a test fails, we print out a lot of information, > this includes items that are not stable between invocations (e.g. > the PID for the executable). That makes automated comparisons > between test runs flag any persistent fails as new ones each time > which is not usually what is wanted. > > This patch amends the error output to drop the variable portion > of the message and retain items that should only change if the > failure mode changes. > > gcc/testsuite/ChangeLog: > > * jit.dg/jit.exp: Filter error output to remove per-run > variable content. > > Signed-off-by: Iain Sandoe <iain@sandoe.co.uk> > --- > gcc/testsuite/jit.dg/jit.exp | 21 +++++++++++++++------ > 1 file changed, 15 insertions(+), 6 deletions(-) > > diff --git a/gcc/testsuite/jit.dg/jit.exp > b/gcc/testsuite/jit.dg/jit.exp > index 286cfa8192a..893ff5f6dd0 100644 > --- a/gcc/testsuite/jit.dg/jit.exp > +++ b/gcc/testsuite/jit.dg/jit.exp > @@ -94,25 +94,34 @@ proc parse_valgrind_logfile {name logfile} { > # unexpected exits. > > proc verify_exit_status { executable wres } { > - lassign $wres pid spawnid os_error_flag value > + set extra [lassign $wres pid spawnid os_error_flag value] > verbose "pid: $pid" 3 > verbose "spawnid: $spawnid" 3 > verbose "os_error_flag: $os_error_flag" 3 > verbose "value: $value" 3 > > # Detect segfaults etc: > - if { [llength $wres] > 4 } { > - if { [lindex $wres 4] == "CHILDKILLED" } { > - fail "$executable killed: $wres" > + set len [llength $extra] > + if { $len >= 1 } { > + if { [lindex $extra 0] == "CHILDKILLED" } { > + set reason "Unknown Reason" > + set detail "No Details" > + if { $len >= 2 } { > + set reason [lindex $extra 1] > + if { $len >= 3 } { > + set detail [lindex $extra 2] > + } > + } > + fail "$executable killed: $reason $detail" > return > } > } > if { $os_error_flag != 0 } { > - fail "$executable: OS error: $wres" > + fail "$executable: OS error: $os_error_flag $extra" > return > } > if { $value != 0 } { > - fail "$executable: non-zero exit code: $wres" > + fail "$executable: non-zero exit code: $value $extra" > return > } > pass "$executable exited cleanly"
diff --git a/gcc/testsuite/jit.dg/jit.exp b/gcc/testsuite/jit.dg/jit.exp index 286cfa8192a..893ff5f6dd0 100644 --- a/gcc/testsuite/jit.dg/jit.exp +++ b/gcc/testsuite/jit.dg/jit.exp @@ -94,25 +94,34 @@ proc parse_valgrind_logfile {name logfile} { # unexpected exits. proc verify_exit_status { executable wres } { - lassign $wres pid spawnid os_error_flag value + set extra [lassign $wres pid spawnid os_error_flag value] verbose "pid: $pid" 3 verbose "spawnid: $spawnid" 3 verbose "os_error_flag: $os_error_flag" 3 verbose "value: $value" 3 # Detect segfaults etc: - if { [llength $wres] > 4 } { - if { [lindex $wres 4] == "CHILDKILLED" } { - fail "$executable killed: $wres" + set len [llength $extra] + if { $len >= 1 } { + if { [lindex $extra 0] == "CHILDKILLED" } { + set reason "Unknown Reason" + set detail "No Details" + if { $len >= 2 } { + set reason [lindex $extra 1] + if { $len >= 3 } { + set detail [lindex $extra 2] + } + } + fail "$executable killed: $reason $detail" return } } if { $os_error_flag != 0 } { - fail "$executable: OS error: $wres" + fail "$executable: OS error: $os_error_flag $extra" return } if { $value != 0 } { - fail "$executable: non-zero exit code: $wres" + fail "$executable: non-zero exit code: $value $extra" return } pass "$executable exited cleanly"
Tested on x86_64, i686 Darwin, x86_64 Linux, OK for trunk? When? thanks Iain --- 8< --- Currently when a test fails, we print out a lot of information, this includes items that are not stable between invocations (e.g. the PID for the executable). That makes automated comparisons between test runs flag any persistent fails as new ones each time which is not usually what is wanted. This patch amends the error output to drop the variable portion of the message and retain items that should only change if the failure mode changes. gcc/testsuite/ChangeLog: * jit.dg/jit.exp: Filter error output to remove per-run variable content. Signed-off-by: Iain Sandoe <iain@sandoe.co.uk> --- gcc/testsuite/jit.dg/jit.exp | 21 +++++++++++++++------ 1 file changed, 15 insertions(+), 6 deletions(-)