Patchwork increase timeout in simulate-thread gdb test

login
register
mail settings
Submitter Jack Howarth
Date Dec. 7, 2011, 6:43 p.m.
Message ID <20111207184343.GA2527@bromo.med.uc.edu>
Download mbox | patch
Permalink /patch/130010/
State New
Headers show

Comments

Jack Howarth - Dec. 7, 2011, 6:43 p.m.
Currently we are failing...

FAIL: gcc.dg/simulate-thread/atomic-load-int128.c  -O1 -g  thread simulation test
FAIL: gcc.dg/simulate-thread/atomic-load-int128.c  -O2 -g  thread simulation test
FAIL: gcc.dg/simulate-thread/atomic-load-int128.c  -O3 -g  thread simulation test
FAIL: gcc.dg/simulate-thread/atomic-load-int128.c  -Os -g  thread simulation test

on x86_64-apple-darwin11 due to the 10 second timeout in simulate-thread of
gcc/testsuite/lib/gcc-simulate-thread.exp. Increasing this timeout to 20 seconds
eliminates the failures (as these test take ~16 seconds on x86_64-apple-darwin11).
Okay for gcc trunk?
         Jack

gcc/testsuite/

2011-12-07  Jack Howarth <howarth@bromo.med.uc.edu>

	* lib/gcc-simulate-thread.exp (simulate-thread): Increase timeout
	to 20 seconds.
Uros Bizjak - Dec. 7, 2011, 6:47 p.m.
On Wed, Dec 7, 2011 at 7:43 PM, Jack Howarth <howarth@bromo.med.uc.edu> wrote:
> Currently we are failing...
>
> FAIL: gcc.dg/simulate-thread/atomic-load-int128.c  -O1 -g  thread simulation test
> FAIL: gcc.dg/simulate-thread/atomic-load-int128.c  -O2 -g  thread simulation test
> FAIL: gcc.dg/simulate-thread/atomic-load-int128.c  -O3 -g  thread simulation test
> FAIL: gcc.dg/simulate-thread/atomic-load-int128.c  -Os -g  thread simulation test
>
> on x86_64-apple-darwin11 due to the 10 second timeout in simulate-thread of
> gcc/testsuite/lib/gcc-simulate-thread.exp. Increasing this timeout to 20 seconds
> eliminates the failures (as these test take ~16 seconds on x86_64-apple-darwin11).
> Okay for gcc trunk?

As said elsewhere, this will double the amount of already large
logfile in case of failed test.

Do we really need such detailed log?

Uros.
IainS - Dec. 7, 2011, 6:58 p.m.
On 7 Dec 2011, at 18:47, Uros Bizjak wrote:

> On Wed, Dec 7, 2011 at 7:43 PM, Jack Howarth  
> <howarth@bromo.med.uc.edu> wrote:
>> Currently we are failing...
>>
>> FAIL: gcc.dg/simulate-thread/atomic-load-int128.c  -O1 -g  thread  
>> simulation test
>> FAIL: gcc.dg/simulate-thread/atomic-load-int128.c  -O2 -g  thread  
>> simulation test
>> FAIL: gcc.dg/simulate-thread/atomic-load-int128.c  -O3 -g  thread  
>> simulation test
>> FAIL: gcc.dg/simulate-thread/atomic-load-int128.c  -Os -g  thread  
>> simulation test
>>
>> on x86_64-apple-darwin11 due to the 10 second timeout in simulate- 
>> thread of
>> gcc/testsuite/lib/gcc-simulate-thread.exp. Increasing this timeout  
>> to 20 seconds
>> eliminates the failures (as these test take ~16 seconds on x86_64- 
>> apple-darwin11).
>> Okay for gcc trunk?

if it's only one test can't you use { dg-timeout-factor 2.0 .... ?

> As said elsewhere, this will double the amount of already large
> logfile in case of failed test.
>
> Do we really need such detailed log?

anything to optimize what's in the logs would be welcome in debugging

Iain
Uros Bizjak - Dec. 7, 2011, 7:09 p.m.
On Wed, Dec 7, 2011 at 7:58 PM, Iain Sandoe
<developer@sandoe-acoustics.co.uk> wrote:

>>> Currently we are failing...
>>>
>>> FAIL: gcc.dg/simulate-thread/atomic-load-int128.c  -O1 -g  thread
>>> simulation test
>>> FAIL: gcc.dg/simulate-thread/atomic-load-int128.c  -O2 -g  thread
>>> simulation test
>>> FAIL: gcc.dg/simulate-thread/atomic-load-int128.c  -O3 -g  thread
>>> simulation test
>>> FAIL: gcc.dg/simulate-thread/atomic-load-int128.c  -Os -g  thread
>>> simulation test
>>>
>>> on x86_64-apple-darwin11 due to the 10 second timeout in simulate-thread
>>> of
>>> gcc/testsuite/lib/gcc-simulate-thread.exp. Increasing this timeout to 20
>>> seconds
>>> eliminates the failures (as these test take ~16 seconds on
>>> x86_64-apple-darwin11).
>>> Okay for gcc trunk?
>
>
> if it's only one test can't you use { dg-timeout-factor 2.0 .... ?
>
>
>> As said elsewhere, this will double the amount of already large
>> logfile in case of failed test.
>>
>> Do we really need such detailed log?
>
>
> anything to optimize what's in the logs would be welcome in debugging

I fully agree, but it is trivial to re-run the test in the debugger
outside the testsuite run. IMO, logging a couple of lines for
execution of every instruction (in a loop!) is a bit excessive.

Uros.
Jack Howarth - Feb. 8, 2012, 12:25 a.m.
On Wed, Dec 07, 2011 at 08:09:06PM +0100, Uros Bizjak wrote:
> On Wed, Dec 7, 2011 at 7:58 PM, Iain Sandoe
> <developer@sandoe-acoustics.co.uk> wrote:
> 
> >>> Currently we are failing...
> >>>
> >>> FAIL: gcc.dg/simulate-thread/atomic-load-int128.c  -O1 -g  thread
> >>> simulation test
> >>> FAIL: gcc.dg/simulate-thread/atomic-load-int128.c  -O2 -g  thread
> >>> simulation test
> >>> FAIL: gcc.dg/simulate-thread/atomic-load-int128.c  -O3 -g  thread
> >>> simulation test
> >>> FAIL: gcc.dg/simulate-thread/atomic-load-int128.c  -Os -g  thread
> >>> simulation test
> >>>
> >>> on x86_64-apple-darwin11 due to the 10 second timeout in simulate-thread
> >>> of
> >>> gcc/testsuite/lib/gcc-simulate-thread.exp. Increasing this timeout to 20
> >>> seconds
> >>> eliminates the failures (as these test take ~16 seconds on
> >>> x86_64-apple-darwin11).
> >>> Okay for gcc trunk?
> >
> >
> > if it's only one test can't you use { dg-timeout-factor 2.0 .... ?
> >
> >
> >> As said elsewhere, this will double the amount of already large
> >> logfile in case of failed test.
> >>
> >> Do we really need such detailed log?
> >
> >
> > anything to optimize what's in the logs would be welcome in debugging
> 
> I fully agree, but it is trivial to re-run the test in the debugger
> outside the testsuite run. IMO, logging a couple of lines for
> execution of every instruction (in a loop!) is a bit excessive.
> 
> Uros.

    Any chance we can get some sort of fix into FSF gcc 4.7 for this issue?
FYI, it appears that the current setup of gcc-simulate-thread.exp doesn't honor
the use of { dg-timeout-factor 2.0 } as written. Only manually increasing the
time to 20, per the originally proposed patch, works.
                   Jack
Mike Stump - Feb. 8, 2012, 12:55 a.m.
On Dec 7, 2011, at 10:43 AM, Jack Howarth wrote:
> Currently we are failing...
> 
> FAIL: gcc.dg/simulate-thread/atomic-load-int128.c  -O1 -g  thread simulation test
> FAIL: gcc.dg/simulate-thread/atomic-load-int128.c  -O2 -g  thread simulation test
> FAIL: gcc.dg/simulate-thread/atomic-load-int128.c  -O3 -g  thread simulation test
> FAIL: gcc.dg/simulate-thread/atomic-load-int128.c  -Os -g  thread simulation test
> 
> on x86_64-apple-darwin11 due to the 10 second timeout in simulate-thread of
> gcc/testsuite/lib/gcc-simulate-thread.exp. Increasing this timeout to 20 seconds
> eliminates the failures (as these test take ~16 seconds on x86_64-apple-darwin11).
> Okay for gcc trunk?

Ok.  Ok for 4.7.
Andrew MacLeod - Feb. 8, 2012, 1:11 p.m.
On 02/07/2012 07:55 PM, Mike Stump wrote:
> On Dec 7, 2011, at 10:43 AM, Jack Howarth wrote:
>> Currently we are failing...
>>
>> FAIL: gcc.dg/simulate-thread/atomic-load-int128.c  -O1 -g  thread simulation test
>> FAIL: gcc.dg/simulate-thread/atomic-load-int128.c  -O2 -g  thread simulation test
>> FAIL: gcc.dg/simulate-thread/atomic-load-int128.c  -O3 -g  thread simulation test
>> FAIL: gcc.dg/simulate-thread/atomic-load-int128.c  -Os -g  thread simulation test
>>
>> on x86_64-apple-darwin11 due to the 10 second timeout in simulate-thread of
>> gcc/testsuite/lib/gcc-simulate-thread.exp. Increasing this timeout to 20 seconds
>> eliminates the failures (as these test take ~16 seconds on x86_64-apple-darwin11).
>> Okay for gcc trunk?
> Ok.  Ok for 4.7.
And I just recently noticed armv7 was failing *all* the simulate-thread 
tests for the same reason.  20 seconds appears to make it pass all tests 
there as well.

On a different note, shouldn't these time out's be reported as 
UNRESOLVED rather than fails?  Seems like the framework isn't reporting 
properly.

Andrew
Richard Guenther - Feb. 8, 2012, 1:33 p.m.
On Wed, Feb 8, 2012 at 2:11 PM, Andrew MacLeod <amacleod@redhat.com> wrote:
> On 02/07/2012 07:55 PM, Mike Stump wrote:
>>
>> On Dec 7, 2011, at 10:43 AM, Jack Howarth wrote:
>>>
>>> Currently we are failing...
>>>
>>> FAIL: gcc.dg/simulate-thread/atomic-load-int128.c  -O1 -g  thread
>>> simulation test
>>> FAIL: gcc.dg/simulate-thread/atomic-load-int128.c  -O2 -g  thread
>>> simulation test
>>> FAIL: gcc.dg/simulate-thread/atomic-load-int128.c  -O3 -g  thread
>>> simulation test
>>> FAIL: gcc.dg/simulate-thread/atomic-load-int128.c  -Os -g  thread
>>> simulation test
>>>
>>> on x86_64-apple-darwin11 due to the 10 second timeout in simulate-thread
>>> of
>>> gcc/testsuite/lib/gcc-simulate-thread.exp. Increasing this timeout to 20
>>> seconds
>>> eliminates the failures (as these test take ~16 seconds on
>>> x86_64-apple-darwin11).
>>> Okay for gcc trunk?
>>
>> Ok.  Ok for 4.7.
>
> And I just recently noticed armv7 was failing *all* the simulate-thread
> tests for the same reason.  20 seconds appears to make it pass all tests
> there as well.
>
> On a different note, shouldn't these time out's be reported as UNRESOLVED
> rather than fails?  Seems like the framework isn't reporting properly.

Well - it depends.  You don't know whether the test will eventually terminate,
but yes, you could interpret "UNRESOLVED" as exactly what that is.  A
definite finishes-never would be a FAIL of course.  The question is on which
side to err.

Richard.
IainS - Feb. 8, 2012, 1:37 p.m.
On 8 Feb 2012, at 13:33, Richard Guenther wrote:

> On Wed, Feb 8, 2012 at 2:11 PM, Andrew MacLeod <amacleod@redhat.com>  
> wrote:
>> On 02/07/2012 07:55 PM, Mike Stump wrote:
>>>
>>> On Dec 7, 2011, at 10:43 AM, Jack Howarth wrote:
>>>>
>>>> Currently we are failing...
>>>>
>>>> FAIL: gcc.dg/simulate-thread/atomic-load-int128.c  -O1 -g  thread
>>>> simulation test
>>>> FAIL: gcc.dg/simulate-thread/atomic-load-int128.c  -O2 -g  thread
>>>> simulation test
>>>> FAIL: gcc.dg/simulate-thread/atomic-load-int128.c  -O3 -g  thread
>>>> simulation test
>>>> FAIL: gcc.dg/simulate-thread/atomic-load-int128.c  -Os -g  thread
>>>> simulation test
>>>>
>>>> on x86_64-apple-darwin11 due to the 10 second timeout in simulate- 
>>>> thread
>>>> of
>>>> gcc/testsuite/lib/gcc-simulate-thread.exp. Increasing this  
>>>> timeout to 20
>>>> seconds
>>>> eliminates the failures (as these test take ~16 seconds on
>>>> x86_64-apple-darwin11).
>>>> Okay for gcc trunk?
>>>
>>> Ok.  Ok for 4.7.
>>
>> And I just recently noticed armv7 was failing *all* the simulate- 
>> thread
>> tests for the same reason.  20 seconds appears to make it pass all  
>> tests
>> there as well.
>>
>> On a different note, shouldn't these time out's be reported as  
>> UNRESOLVED
>> rather than fails?  Seems like the framework isn't reporting  
>> properly.
>
> Well - it depends.  You don't know whether the test will eventually  
> terminate,
> but yes, you could interpret "UNRESOLVED" as exactly what that is.  A
> definite finishes-never would be a FAIL of course.  The question is  
> on which
> side to err.

There was also some discussion about reducing the amount of log-file  
output (which might help too).
Did that ever get anywhere/who would be willing to decide what could  
be dropped from the output?
Iain
Andrew MacLeod - Feb. 8, 2012, 2 p.m.
On 02/08/2012 08:37 AM, Iain Sandoe wrote:
>
>> Well - it depends.  You don't know whether the test will eventually 
>> terminate,
>> but yes, you could interpret "UNRESOLVED" as exactly what that is.  A
>> definite finishes-never would be a FAIL of course.  The question is 
>> on which
>> side to err.
>
>
> There was also some discussion about reducing the amount of log-file 
> output (which might help too).
> Did that ever get anywhere/who would be willing to decide what could 
> be dropped from the output?
> Iain
>

For a start we could comment out the
    disp/i $pc
command in the simulate-thread.gdb script file.   I'd leave it there as 
a comment to make it easy to turn back on when needed.

You can still see the source line being executed, and you generally need 
to take the test case and hand run it to see whats going on anyway.   
That cuts the log file in about half.

Andrew
Jack Howarth - Feb. 8, 2012, 2:27 p.m.
On Wed, Feb 08, 2012 at 09:00:00AM -0500, Andrew MacLeod wrote:
> On 02/08/2012 08:37 AM, Iain Sandoe wrote:
>>
>>> Well - it depends.  You don't know whether the test will eventually  
>>> terminate,
>>> but yes, you could interpret "UNRESOLVED" as exactly what that is.  A
>>> definite finishes-never would be a FAIL of course.  The question is  
>>> on which
>>> side to err.
>>
>>
>> There was also some discussion about reducing the amount of log-file  
>> output (which might help too).
>> Did that ever get anywhere/who would be willing to decide what could  
>> be dropped from the output?
>> Iain
>>
>
> For a start we could comment out the
>    disp/i $pc
> command in the simulate-thread.gdb script file.   I'd leave it there as  
> a comment to make it easy to turn back on when needed.

Andrew,
   Eliminating the "disp/i $pc" in the simulate-thread.gdb script file
is insufficient to reduce the execution time below 10 seconds. What is 
the next thing we can prune?
           Jack

>
> You can still see the source line being executed, and you generally need  
> to take the test case and hand run it to see whats going on anyway.    
> That cuts the log file in about half.
>
> Andrew
>
Andrew MacLeod - Feb. 8, 2012, 2:47 p.m.
On 02/08/2012 09:27 AM, Jack Howarth wrote:
> On Wed, Feb 08, 2012 at 09:00:00AM -0500, Andrew MacLeod wrote:
>> On 02/08/2012 08:37 AM, Iain Sandoe wrote:
>>>> Well - it depends.  You don't know whether the test will eventually
>>>> terminate,
>>>> but yes, you could interpret "UNRESOLVED" as exactly what that is.  A
>>>> definite finishes-never would be a FAIL of course.  The question is
>>>> on which
>>>> side to err.
>>>
>>> There was also some discussion about reducing the amount of log-file
>>> output (which might help too).
>>> Did that ever get anywhere/who would be willing to decide what could
>>> be dropped from the output?
>>> Iain
>>>
>> For a start we could comment out the
>>     disp/i $pc
>> command in the simulate-thread.gdb script file.   I'd leave it there as
>> a comment to make it easy to turn back on when needed.
> Andrew,
>     Eliminating the "disp/i $pc" in the simulate-thread.gdb script file
> is insufficient to reduce the execution time below 10 seconds. What is
> the next thing we can prune?
>             Jack
no, it wont change the time, just reduce the size of the log file.

The only way to reduce the time below 10 seconds would be to change the 
test case to run less time, but I'd rather leave them as they are. They 
did catch one very tricky case for me.

I propose increasing the time to 20 seconds and reduce the log file.  I 
believe the timeout as made really short because of the size of the log 
file when the timeout was needed. I htink it was an arbitrary number.   
Doubling the execution time and halving the size of the log file should 
put us about where we were.

Im even fine abandoning most of the log file if need be as well... Its 
really pass or fail that I care about and a fail usually requires hand 
investigation anyway

The other thing we can do is to only run the test with -O0 and -O3.   
right now they run with -O0, -O1, -O2, -O3, -Os.  I don't think the 
other options really add much.

Andrew
Andrew MacLeod - Feb. 8, 2012, 4:02 p.m.
On 02/08/2012 09:47 AM, Andrew MacLeod wrote:
>
> I propose increasing the time to 20 seconds and reduce the log file.  
> I believe the timeout as made really short because of the size of the 
> log file when the timeout was needed. I htink it was an arbitrary 
> number.   Doubling the execution time and halving the size of the log 
> file should put us about where we were.
>
> Im even fine abandoning most of the log file if need be as well... Its 
> really pass or fail that I care about and a fail usually requires hand 
> investigation anyway
>
> The other thing we can do is to only run the test with -O0 and -O3.   
> right now they run with -O0, -O1, -O2, -O3, -Os.  I don't think the 
> other options really add much.
>
So something like the following changes:


I'm running a testsuite run on x86-64 right now, I'll try arm as well 
shortly.     Presuming that it runs properly, is this OK for mainline?

Andrew
2012-02-08  Jack Howarth <howarth@bromo.med.uc.edu>

	* lib/gcc-simulate-thread.exp (simulate-thread): Increase timeout
	  to 20 seconds.

2012-02-08  Andrew MacLeod  <amacleod@redhat.com>

	* gcc.dg/simulate-thread/simulate-thread.exp: Use only -O0 and -O3.
	* gcc.dg/simulate-thread/simulate-thread.gdb: Don't display every 
	machine instuction in the log.

Index: lib/gcc-simulate-thread.exp
===================================================================
*** lib/gcc-simulate-thread.exp	(revision 183969)
--- lib/gcc-simulate-thread.exp	(working copy)
*************** proc simulate-thread { args } {
*** 56,63 ****
  
      set gdb_worked 0
  
!     # Set timeout to 10 seconds due to huge amount of generated log.
!     remote_expect target 10 {
  	# Too old GDB
  	-re "Unhandled dwarf expression|Error in sourced command file" {
  	    unsupported "$testcase $message"
--- 56,63 ----
  
      set gdb_worked 0
  
!     # Set timeout to 20 seconds due to huge amount of generated log.
!     remote_expect target 20 {
  	# Too old GDB
  	-re "Unhandled dwarf expression|Error in sourced command file" {
  	    unsupported "$testcase $message"
Index: gcc.dg/simulate-thread/simulate-thread.exp
===================================================================
*** gcc.dg/simulate-thread/simulate-thread.exp	(revision 183969)
--- gcc.dg/simulate-thread/simulate-thread.exp	(working copy)
*************** dg-init
*** 24,33 ****
  torture-init
  set-torture-options [list \
  	{ -O0 -g } \
! 	{ -O1 -g } \
! 	{ -O2 -g } \
! 	{ -O3 -g } \
! 	{ -Os -g } ]
  
  if [gdb-exists] {
    gcc-dg-runtest [lsort [glob -nocomplain $srcdir/$subdir/*.c]] ""
--- 24,30 ----
  torture-init
  set-torture-options [list \
  	{ -O0 -g } \
! 	{ -O3 -g } ]
  
  if [gdb-exists] {
    gcc-dg-runtest [lsort [glob -nocomplain $srcdir/$subdir/*.c]] ""
Index: gcc.dg/simulate-thread/simulate-thread.gdb
===================================================================
*** gcc.dg/simulate-thread/simulate-thread.gdb	(revision 183969)
--- gcc.dg/simulate-thread/simulate-thread.gdb	(working copy)
***************
*** 1,6 ****
  set height 0
  break simulate_thread_main
! disp/i $pc
  run
  
  set $ret = 0
--- 1,6 ----
  set height 0
  break simulate_thread_main
! # disp/i $pc
  run
  
  set $ret = 0
Jack Howarth - Feb. 8, 2012, 4:43 p.m.
On Wed, Feb 08, 2012 at 11:02:02AM -0500, Andrew MacLeod wrote:
> On 02/08/2012 09:47 AM, Andrew MacLeod wrote:
>>
>> I propose increasing the time to 20 seconds and reduce the log file.   
>> I believe the timeout as made really short because of the size of the  
>> log file when the timeout was needed. I htink it was an arbitrary  
>> number.   Doubling the execution time and halving the size of the log  
>> file should put us about where we were.
>>
>> Im even fine abandoning most of the log file if need be as well... Its  
>> really pass or fail that I care about and a fail usually requires hand  
>> investigation anyway
>>
>> The other thing we can do is to only run the test with -O0 and -O3.    
>> right now they run with -O0, -O1, -O2, -O3, -Os.  I don't think the  
>> other options really add much.
>>
> So something like the following changes:
>
>
> I'm running a testsuite run on x86-64 right now, I'll try arm as well  
> shortly.     Presuming that it runs properly, is this OK for mainline?

Andrew,
   I can confirm that this eliminates the unexpected failures in
simulate-thread.exp on x86_64-apple-darwin11...

Native configuration is x86_64-apple-darwin11.3.0

		=== gcc tests ===


Running target unix/-m32

		=== gcc Summary for unix/-m32 ===

# of expected passes		49
# of unsupported tests		7

Running target unix/-m64

		=== gcc Summary for unix/-m64 ===

# of expected passes		57
# of unsupported tests		3

		=== gcc Summary ===

# of expected passes		106
# of unsupported tests		10
/sw/src/fink.build/gcc47-4.7.0-1/darwin_objdir/gcc/xgcc  version 4.7.0 20120208 (experimental) (GCC) 

Compiler version: 4.7.0 20120208 (experimental) (GCC) 
Platform: x86_64-apple-darwin11.3.0
configure flags: --prefix=/sw --prefix=/sw/lib/gcc4.7 --mandir=/sw/share/man --infodir=/sw/lib/gcc4.7/info --with-build-config=bootstrap-lto --enable-stage1-languages=c,lto --enable-languages=c,c++,fortran,lto,objc,obj-c++,java --with-gmp=/sw --with-libiconv-prefix=/sw --with-ppl=/sw --with-cloog=/sw --with-mpc=/sw --with-system-zlib --x-includes=/usr/X11R6/include --x-libraries=/usr/X11R6/lib --program-suffix=-fsf-4.7 --enable-checking=release --enable-cloog-backend=isl

Thanks for the fix.
          Jack
>
> Andrew
>

> 2012-02-08  Jack Howarth <howarth@bromo.med.uc.edu>
> 
> 	* lib/gcc-simulate-thread.exp (simulate-thread): Increase timeout
> 	  to 20 seconds.
> 
> 2012-02-08  Andrew MacLeod  <amacleod@redhat.com>
> 
> 	* gcc.dg/simulate-thread/simulate-thread.exp: Use only -O0 and -O3.
> 	* gcc.dg/simulate-thread/simulate-thread.gdb: Don't display every 
> 	machine instuction in the log.
> 
> Index: lib/gcc-simulate-thread.exp
> ===================================================================
> *** lib/gcc-simulate-thread.exp	(revision 183969)
> --- lib/gcc-simulate-thread.exp	(working copy)
> *************** proc simulate-thread { args } {
> *** 56,63 ****
>   
>       set gdb_worked 0
>   
> !     # Set timeout to 10 seconds due to huge amount of generated log.
> !     remote_expect target 10 {
>   	# Too old GDB
>   	-re "Unhandled dwarf expression|Error in sourced command file" {
>   	    unsupported "$testcase $message"
> --- 56,63 ----
>   
>       set gdb_worked 0
>   
> !     # Set timeout to 20 seconds due to huge amount of generated log.
> !     remote_expect target 20 {
>   	# Too old GDB
>   	-re "Unhandled dwarf expression|Error in sourced command file" {
>   	    unsupported "$testcase $message"
> Index: gcc.dg/simulate-thread/simulate-thread.exp
> ===================================================================
> *** gcc.dg/simulate-thread/simulate-thread.exp	(revision 183969)
> --- gcc.dg/simulate-thread/simulate-thread.exp	(working copy)
> *************** dg-init
> *** 24,33 ****
>   torture-init
>   set-torture-options [list \
>   	{ -O0 -g } \
> ! 	{ -O1 -g } \
> ! 	{ -O2 -g } \
> ! 	{ -O3 -g } \
> ! 	{ -Os -g } ]
>   
>   if [gdb-exists] {
>     gcc-dg-runtest [lsort [glob -nocomplain $srcdir/$subdir/*.c]] ""
> --- 24,30 ----
>   torture-init
>   set-torture-options [list \
>   	{ -O0 -g } \
> ! 	{ -O3 -g } ]
>   
>   if [gdb-exists] {
>     gcc-dg-runtest [lsort [glob -nocomplain $srcdir/$subdir/*.c]] ""
> Index: gcc.dg/simulate-thread/simulate-thread.gdb
> ===================================================================
> *** gcc.dg/simulate-thread/simulate-thread.gdb	(revision 183969)
> --- gcc.dg/simulate-thread/simulate-thread.gdb	(working copy)
> ***************
> *** 1,6 ****
>   set height 0
>   break simulate_thread_main
> ! disp/i $pc
>   run
>   
>   set $ret = 0
> --- 1,6 ----
>   set height 0
>   break simulate_thread_main
> ! # disp/i $pc
>   run
>   
>   set $ret = 0
Andrew MacLeod - Feb. 8, 2012, 5:08 p.m.
On 02/08/2012 11:43 AM, Jack Howarth wrote:
>>
>> So something like the following changes:
>>
>>
>> I'm running a testsuite run on x86-64 right now, I'll try arm as well
>> shortly.     Presuming that it runs properly, is this OK for mainline?
> Andrew,
>     I can confirm that this eliminates the unexpected failures in
> simulate-thread.exp on x86_64-apple-darwin11...

This also resolved my arm tests, as well as works fine on x86-64.    OK 
for mainline?

Andrew
Mike Stump - Feb. 8, 2012, 9:22 p.m.
On Feb 8, 2012, at 5:11 AM, Andrew MacLeod wrote:
> On 02/07/2012 07:55 PM, Mike Stump wrote:
>> On Dec 7, 2011, at 10:43 AM, Jack Howarth wrote:
>>> Currently we are failing...
>>> 
>>> FAIL: gcc.dg/simulate-thread/atomic-load-int128.c  -O1 -g  thread simulation test
>>> FAIL: gcc.dg/simulate-thread/atomic-load-int128.c  -O2 -g  thread simulation test
>>> FAIL: gcc.dg/simulate-thread/atomic-load-int128.c  -O3 -g  thread simulation test
>>> FAIL: gcc.dg/simulate-thread/atomic-load-int128.c  -Os -g  thread simulation test
>>> 
>>> on x86_64-apple-darwin11 due to the 10 second timeout in simulate-thread of
>>> gcc/testsuite/lib/gcc-simulate-thread.exp. Increasing this timeout to 20 seconds
>>> eliminates the failures (as these test take ~16 seconds on x86_64-apple-darwin11).
>>> Okay for gcc trunk?
>> Ok.  Ok for 4.7.
> And I just recently noticed armv7 was failing *all* the simulate-thread tests for the same reason.  20 seconds appears to make it pass all tests there as well.

In the context of recent, reasonably fast machine, no timeout should be within 10x of failing because of the timeout.  Having a 20 second timeout, when 18 are required, means failures, if you merely have 2 test suites running at the same time.  If it takes 18 seconds, the timeout should be 180 seconds, minimum.  :-(  I'd welcome a simulate threads person re-engineering what the numbers should be.

> On a different note, shouldn't these time out's be reported as UNRESOLVED rather than fails?  Seems like the framework isn't reporting properly.

No.  An infinite loop is failure.  The timeouts are not supported to be close.
Mike Stump - Feb. 8, 2012, 9:23 p.m.
On Feb 8, 2012, at 8:02 AM, Andrew MacLeod wrote:
> Presuming that it runs properly, is this OK for mainline?

Ok.
Andrew MacLeod - Feb. 8, 2012, 9:32 p.m.
On 02/08/2012 04:22 PM, Mike Stump wrote:
> On Feb 8, 2012, at 5:11 AM, Andrew MacLeod wrote:
>> On 02/07/2012 07:55 PM, Mike Stump wrote:
>>> On Dec 7, 2011, at 10:43 AM, Jack Howarth wrote:
>>>> Currently we are failing...
>>>>
>>>> FAIL: gcc.dg/simulate-thread/atomic-load-int128.c  -O1 -g  thread simulation test
>>>> FAIL: gcc.dg/simulate-thread/atomic-load-int128.c  -O2 -g  thread simulation test
>>>> FAIL: gcc.dg/simulate-thread/atomic-load-int128.c  -O3 -g  thread simulation test
>>>> FAIL: gcc.dg/simulate-thread/atomic-load-int128.c  -Os -g  thread simulation test
>>>>
>>>> on x86_64-apple-darwin11 due to the 10 second timeout in simulate-thread of
>>>> gcc/testsuite/lib/gcc-simulate-thread.exp. Increasing this timeout to 20 seconds
>>>> eliminates the failures (as these test take ~16 seconds on x86_64-apple-darwin11).
>>>> Okay for gcc trunk?
>>> Ok.  Ok for 4.7.
>> And I just recently noticed armv7 was failing *all* the simulate-thread tests for the same reason.  20 seconds appears to make it pass all tests there as well.
> In the context of recent, reasonably fast machine, no timeout should be within 10x of failing because of the timeout.  Having a 20 second timeout, when 18 are required, means failures, if you merely have 2 test suites running at the same time.  If it takes 18 seconds, the timeout should be 180 seconds, minimum.  :-(  I'd welcome a simulate threads person re-engineering what the numbers should be.
well, originally there wasn't a special number.  Someone added the 10 
seconds because most of the testcases are short, and they generated  a 
*lot* of log. If an infinite loop did happen (which it did), the 300 
second timeout could generate a huge log file that fills the 
filesystem.  I think there was a defect for that...  Anyway, I think 10 
seconds just came out of someones imagination, but I'm not sure.  Was 
that you Aldy?

Given thats the case, I'd be more tempted long term to simply disable 
the line by line output into the log file...  I assume thats possible. 
then we dont need any special time outs.  If a failure needs 
investigation, you look at it by hand.

Andrew
Aldy Hernandez - Feb. 9, 2012, 2:12 p.m.
> I think there was a defect for that... Anyway, I think 10 seconds just
> came out of someones imagination, but I'm not sure. Was that you Aldy?

I really can't remember, but it's possible.

>
> Given thats the case, I'd be more tempted long term to simply disable
> the line by line output into the log file... I assume thats possible.
> then we dont need any special time outs. If a failure needs
> investigation, you look at it by hand.
>
> Andrew
Uros Bizjak - Feb. 9, 2012, 2:38 p.m.
On Thu, Feb 9, 2012 at 3:12 PM, Aldy Hernandez <aldyh@redhat.com> wrote:

>> I think there was a defect for that... Anyway, I think 10 seconds just
>> came out of someones imagination, but I'm not sure. Was that you Aldy?
>
>
> I really can't remember, but it's possible.

It was me, and the sole reason was that timeout didn't worked and the
log filled the file system. After timeout functionality was fixed, the
timeout was forced to 10 seconds. It is an arbitrary number.

Uros.
Andrew MacLeod - Feb. 9, 2012, 2:52 p.m.
On 02/09/2012 09:38 AM, Uros Bizjak wrote:
> On Thu, Feb 9, 2012 at 3:12 PM, Aldy Hernandez<aldyh@redhat.com>  wrote:
>
> It was me, and the sole reason was that timeout didn't worked and the
> log filled the file system. After timeout functionality was fixed, the
> timeout was forced to 10 seconds. It is an arbitrary number.
>
I suspected as much.  Long term I think we should remove the line by 
line log info as the default.

Andrew

Patch

Index: gcc/testsuite/lib/gcc-simulate-thread.exp
===================================================================
--- gcc/testsuite/lib/gcc-simulate-thread.exp	(revision 182083)
+++ gcc/testsuite/lib/gcc-simulate-thread.exp	(working copy)
@@ -56,8 +56,8 @@  proc simulate-thread { args } {
 
     set gdb_worked 0
 
-    # Set timeout to 10 seconds due to huge amount of generated log.
-    remote_expect target 10 {
+    # Set timeout to 20 seconds due to huge amount of generated log.
+    remote_expect target 20 {
 	# Too old GDB
 	-re "Unhandled dwarf expression|Error in sourced command file" {
 	    unsupported "$testcase $message"