| Message ID | 20200720111036.jnvu6qu3z4sbj2ns@arm.com |
|---|---|
| State | New |
| Headers | show |
| Series | jit: Fix random truncation of testsuite output | expand |
On Mon, 2020-07-20 at 12:10 +0100, Alex Coplan wrote: > Hello, > > This patch fixes a bug in jit.exp which causes the DejaGnu output of > the > libgccjit testsuite to be nondeterministically truncated. This bug > was > copied from DejaGnu's own implementation of the host_execute > function. > See the upstream bug report [0] where the maintainers point out that > the > regex patterns in host_execute should (but don't currently) > explicitly > match newlines to avoid relying on DejaGnu not reading more than one > line of the output (which is not guaranteed). > > To reproduce the bug, run: > > $ make check-jit RUNTESTFLAGS="jit.exp=test-arith-overflow.c" > $ grep -v iteration testsuite/jit/jit.sum > > and you should see some lines that have been truncated (I see the > word > iteration partially or fully truncated). Alternatively, simply run > the > testsuite twice (saving a copy of testsuite/jit/jit.sum from the > first > run) and diff the two jit.sum files to observe the random truncations > to > the output. > > This patch should make it easier to test jit patches in the future, > since it makes it possible to reliably compare the output of two > jit.sum > files (as with the other tests in GCC). > > Testing: > * Ran the testsuite before and after the patch, observing that the > only > differences in jit.sum were in test-threads.c (nondeterministic > test) > and where the truncated output from the first run was no longer > truncated. > * Ran the testsuite twice after the patch, observing that the only > differences in jit.sum between the two runs were in test- > threads.c. > > OK for master? > > Thanks, > Alex > > --- > > 2020-07-20 Alex Coplan <alex.coplan@arm.com> > > gcc/testsuite/ChangeLog: > > * jit.dg/jit.exp (fixed_host_execute): Fix regex patterns to > always explicitly match newlines. > > > [0] : https://debbugs.gnu.org/cgi/bugreport.cgi?bug=42399 ` Thanks for chasing this up. This looks a lot like the issues being tracked in PR jit/69435, so please add that to the ChangeLog when committing. OK for master. Dave
diff --git a/gcc/testsuite/jit.dg/jit.exp b/gcc/testsuite/jit.dg/jit.exp index 2f54681713b..2d8c884b6b8 100644 --- a/gcc/testsuite/jit.dg/jit.exp +++ b/gcc/testsuite/jit.dg/jit.exp @@ -202,37 +202,37 @@ proc fixed_host_execute {args} { set timetol 0 exp_continue } - -re "^$prefix\tNOTE:${text}*" { + -re "^$prefix\tNOTE:\[^\r\n\]+\r\n" { regsub "\[\n\r\t\]*NOTE: $text\r\n" $expect_out(0,string) "" output - set output [string range $output 6 end] + set output [string range $output 6 end-2] verbose "$output" 2 set timetol 0 exp_continue } - -re "^$prefix\tPASSED:${text}*" { + -re "^$prefix\tPASSED:\[^\r\n\]+\r\n" { regsub "\[\n\r\t\]*PASSED: $text\r\n" $expect_out(0,string) "" output - set output [string range $output 8 end] + set output [string range $output 8 end-2] pass "$output" set timetol 0 exp_continue } - -re "^$prefix\tFAILED:${text}*" { + -re "^$prefix\tFAILED:\[^\r\n\]+\r\n" { regsub "\[\n\r\t\]*FAILED: $text\r\n" $expect_out(0,string) "" output - set output [string range $output 8 end] + set output [string range $output 8 end-2] fail "$output" set timetol 0 exp_continue } - -re "^$prefix\tUNTESTED:${text}*" { + -re "^$prefix\tUNTESTED:\[^\r\n\]+\r\n" { regsub "\[\n\r\t\]*TESTED: $text\r\n" $expect_out(0,string) "" output - set output [string range $output 8 end] + set output [string range $output 8 end-2] untested "$output" set timetol 0 exp_continue } - -re "^$prefix\tUNRESOLVED:${text}*" { + -re "^$prefix\tUNRESOLVED:\[^\r\n\]+\r\n" { regsub "\[\n\r\t\]*UNRESOLVED: $text\r\n" $expect_out(0,string) "" output - set output [string range $output 8 end] + set output [string range $output 8 end-2] unresolved "$output" set timetol 0 exp_continue