diff mbox

stdlib/tst-strtod-overflow: Bump timeout up yet more

Message ID alpine.DEB.1.10.1406230137220.25395@tp.orcam.me.uk
State New
Headers show

Commit Message

Maciej W. Rozycki June 23, 2014, 12:54 a.m. UTC
Hi,

 In our routine testing I observed that stdlib/tst-strtod-overflow is very 
slow, especially on targets using soft-float or QEMU (where soft-float is 
used internally), enough to time out even on slow boards we have that have 
TIMEOUTFACTOR already bumped from the default of 1 up to 75.

 No other test case requires such a long timeout -- all the other 
succeeding cases fit within their timeouts scaled by TIMEOUTFACTOR on 
these boards.  As such I think it's counter-productive to require 
TIMEOUTFACTOR to be set as high as 450 globally for this lone outlier as 
the value affects overall testing duration where there are test cases that 
genuinely time out due to a defect.  Therefore I propose the following 
change that makes stdlib/tst-strtod-overflow pass on these slow boards 
with TIMEOUTFACTOR of 75.

 OK to apply?

2014-06-23  Maciej W. Rozycki  <macro@codesourcery.com>

	* stdlib/tst-strtod-overflow.c (TIMEOUT): Bump up to 30.

  Maciej

Comments

Maciej W. Rozycki June 30, 2014, 9:19 a.m. UTC | #1
On Mon, 23 Jun 2014, Maciej W. Rozycki wrote:

>  In our routine testing I observed that stdlib/tst-strtod-overflow is very 
> slow, especially on targets using soft-float or QEMU (where soft-float is 
> used internally), enough to time out even on slow boards we have that have 
> TIMEOUTFACTOR already bumped from the default of 1 up to 75.
> 
>  No other test case requires such a long timeout -- all the other 
> succeeding cases fit within their timeouts scaled by TIMEOUTFACTOR on 
> these boards.  As such I think it's counter-productive to require 
> TIMEOUTFACTOR to be set as high as 450 globally for this lone outlier as 
> the value affects overall testing duration where there are test cases that 
> genuinely time out due to a defect.  Therefore I propose the following 
> change that makes stdlib/tst-strtod-overflow pass on these slow boards 
> with TIMEOUTFACTOR of 75.

 Ping!

  Maciej
Adhemerval Zanella June 30, 2014, 11:16 a.m. UTC | #2
On 30-06-2014 06:19, Maciej W. Rozycki wrote:
> On Mon, 23 Jun 2014, Maciej W. Rozycki wrote:
>
>>  In our routine testing I observed that stdlib/tst-strtod-overflow is very 
>> slow, especially on targets using soft-float or QEMU (where soft-float is 
>> used internally), enough to time out even on slow boards we have that have 
>> TIMEOUTFACTOR already bumped from the default of 1 up to 75.
>>
>>  No other test case requires such a long timeout -- all the other 
>> succeeding cases fit within their timeouts scaled by TIMEOUTFACTOR on 
>> these boards.  As such I think it's counter-productive to require 
>> TIMEOUTFACTOR to be set as high as 450 globally for this lone outlier as 
>> the value affects overall testing duration where there are test cases that 
>> genuinely time out due to a defect.  Therefore I propose the following 
>> change that makes stdlib/tst-strtod-overflow pass on these slow boards 
>> with TIMEOUTFACTOR of 75.
>  Ping!
>
>   Maciej
>
I don't have any objection to the patch, the explanation seems fair enough.
Maciej W. Rozycki July 1, 2014, 11:04 a.m. UTC | #3
On Mon, 30 Jun 2014, Adhemerval Zanella wrote:

> >>  In our routine testing I observed that stdlib/tst-strtod-overflow is very 
> >> slow, especially on targets using soft-float or QEMU (where soft-float is 
> >> used internally), enough to time out even on slow boards we have that have 
> >> TIMEOUTFACTOR already bumped from the default of 1 up to 75.
> >>
> >>  No other test case requires such a long timeout -- all the other 
> >> succeeding cases fit within their timeouts scaled by TIMEOUTFACTOR on 
> >> these boards.  As such I think it's counter-productive to require 
> >> TIMEOUTFACTOR to be set as high as 450 globally for this lone outlier as 
> >> the value affects overall testing duration where there are test cases that 
> >> genuinely time out due to a defect.  Therefore I propose the following 
> >> change that makes stdlib/tst-strtod-overflow pass on these slow boards 
> >> with TIMEOUTFACTOR of 75.
> >  Ping!
> >
> I don't have any objection to the patch, the explanation seems fair enough. 

 Thanks for your input.  Any other comments, anyone, or shall I treat it 
as the consensus?

  Maciej
Will Newton July 1, 2014, 11:13 a.m. UTC | #4
On 1 July 2014 12:04, Maciej W. Rozycki <macro@codesourcery.com> wrote:
> On Mon, 30 Jun 2014, Adhemerval Zanella wrote:
>
>> >>  In our routine testing I observed that stdlib/tst-strtod-overflow is very
>> >> slow, especially on targets using soft-float or QEMU (where soft-float is
>> >> used internally), enough to time out even on slow boards we have that have
>> >> TIMEOUTFACTOR already bumped from the default of 1 up to 75.
>> >>
>> >>  No other test case requires such a long timeout -- all the other
>> >> succeeding cases fit within their timeouts scaled by TIMEOUTFACTOR on
>> >> these boards.  As such I think it's counter-productive to require
>> >> TIMEOUTFACTOR to be set as high as 450 globally for this lone outlier as
>> >> the value affects overall testing duration where there are test cases that
>> >> genuinely time out due to a defect.  Therefore I propose the following
>> >> change that makes stdlib/tst-strtod-overflow pass on these slow boards
>> >> with TIMEOUTFACTOR of 75.
>> >  Ping!
>> >
>> I don't have any objection to the patch, the explanation seems fair enough.
>
>  Thanks for your input.  Any other comments, anyone, or shall I treat it
> as the consensus?

The change looks ok to me too.
Maciej W. Rozycki July 3, 2014, 7:13 a.m. UTC | #5
On Tue, 1 Jul 2014, Will Newton wrote:

> >> >>  In our routine testing I observed that stdlib/tst-strtod-overflow is very
> >> >> slow, especially on targets using soft-float or QEMU (where soft-float is
> >> >> used internally), enough to time out even on slow boards we have that have
> >> >> TIMEOUTFACTOR already bumped from the default of 1 up to 75.
> >> >>
> >> >>  No other test case requires such a long timeout -- all the other
> >> >> succeeding cases fit within their timeouts scaled by TIMEOUTFACTOR on
> >> >> these boards.  As such I think it's counter-productive to require
> >> >> TIMEOUTFACTOR to be set as high as 450 globally for this lone outlier as
> >> >> the value affects overall testing duration where there are test cases that
> >> >> genuinely time out due to a defect.  Therefore I propose the following
> >> >> change that makes stdlib/tst-strtod-overflow pass on these slow boards
> >> >> with TIMEOUTFACTOR of 75.
> >> >  Ping!
> >> >
> >> I don't have any objection to the patch, the explanation seems fair enough.
> >
> >  Thanks for your input.  Any other comments, anyone, or shall I treat it
> > as the consensus?
> 
> The change looks ok to me too.

 Applied now, thanks!

  Maciej
diff mbox

Patch

Index: glibc-fsf-trunk-quilt/stdlib/tst-strtod-overflow.c
===================================================================
--- glibc-fsf-trunk-quilt.orig/stdlib/tst-strtod-overflow.c	2014-06-17 15:50:29.991694082 +0100
+++ glibc-fsf-trunk-quilt/stdlib/tst-strtod-overflow.c	2014-06-23 01:49:28.082011090 +0100
@@ -45,5 +45,5 @@  do_test (void)
 }
 
 #define TEST_FUNCTION do_test ()
-#define TIMEOUT 5
+#define TIMEOUT 30
 #include "../test-skeleton.c"