[RFC,v2] Test coverage for --param boundary values

On 07/21/2016 08:53 PM, Jeff Law wrote:
> On 07/18/2016 01:53 AM, Martin Liška wrote:
>> You are right, I was inspired by what we do for GCC plugins in:
>> gcc/testsuite/lib/plugin-support.exp
>>
>> where we have following comment:
>>     # Note that the plugin test support currently only works when the GCC
>>     # build tree is available. (We make sure that is the case in plugin.exp.)
>>     # Once we have figured out how/where to package/install GCC header files
>>     # for general plugin support, we should modify the following include paths
>>     # accordingly.
>>
>> Well, I can imaging a guard which will test whether "$objdir/../../params.options" file exits,
>> and if so, then the tests are executed? Is it acceptable approach?
> Yes.  That's acceptable to me.
> 
>>
>>> >
>>>> >> +set text [read $fd]
>>>> >> +close $fd
>>>> >> +
>>>> >> +# Main loop.
>>>> >> +foreach params [split $text "\n"] {
>>>> >> +    set parts [split $params "="]
>>>> >> +    set name [string trim [lindex $parts 0] '"']
>>>> >> +    set values [split [lindex $parts 1] ","]
>>>> >> +    if { [llength $values] == 3 } {
>>>> >> +    set default [lindex $values 0]
>>>> >> +    set min [lindex $values 1]
>>>> >> +    set max [lindex $values 2]
>>>> >> +    set int_max "INT_MAX"
>>>> >> +
>>>> >> +    if { $min != -1 } {
>>>> >> +        param_run_test $name $min
>>>> >> +    }
>>>> >> +    if { $max != $min && $max > 0 && $max != $int_max } {
>>>> >> +        param_run_test $name $max
>>>> >> +    }
>>>> >> +    }
>>>> >> +    if { [llength $values] == 5 } {
>>>> >> +    foreach v $values {
>>>> >> +        param_run_test $name $v
>>>> >> +    }
>>>> >> +    }
>>>> >> +}
>>> >
>>> > This produces ambiguous test result lines:
>>> >
>>> >     Running /scratch/tschwing/nvidiak20-2/gcc/trunk-light/source-gcc/gcc/testsuite/gcc.dg/params/params.exp ...
>>> >     PASS: gcc.dg/params/blocksort-part.c (test for excess errors)
>>> >     PASS: gcc.dg/params/blocksort-part.c (test for excess errors)
>>> >     PASS: gcc.dg/params/blocksort-part.c (test for excess errors)
>>> >     PASS: gcc.dg/params/blocksort-part.c (test for excess errors)
>>> >     [many more]
>>> >
>>> > OK to fix this as follows:
>>> >
>>> >     PASS: gcc.dg/params/blocksort-part.c -O3 --param predictable-branch-outcome=0 (test for excess errors)
>>> >     PASS: gcc.dg/params/blocksort-part.c -O3 --param predictable-branch-outcome=50 (test for excess errors)
>>> >     PASS: gcc.dg/params/blocksort-part.c -O3 --param inline-min-speedup=0 (test for excess errors)
>>> >     PASS: gcc.dg/params/blocksort-part.c -O3 --param max-inline-insns-single=0 (test for excess errors)
>>> >     [...]
>>> >
>>> > commit ae350ce9af062d7d2c05225280596b1bc982728b
>>> > Author: Thomas Schwinge <thomas@codesourcery.com>
>>> > Date:   Fri Jul 15 09:13:19 2016 +0200
>>> >
>>> >     Disambiguate gcc.dg/params/params.exp test results
>>> >
>>> >         gcc/testsuite/
>>> >         * gcc.dg/params/params.exp (param_run_test): Swap flags and
>>> >         default-extra-flags arguments when calling dg-runtest.
>>> > ---
>>> >  gcc/testsuite/gcc.dg/params/params.exp |    2 +-
>>> >  1 file changed, 1 insertion(+), 1 deletion(-)
>>> >
>>> > diff --git gcc/testsuite/gcc.dg/params/params.exp gcc/testsuite/gcc.dg/params/params.exp
>>> > index f2f397e..dd82a5d 100644
>>> > --- gcc/testsuite/gcc.dg/params/params.exp
>>> > +++ gcc/testsuite/gcc.dg/params/params.exp
>>> > @@ -28,7 +28,7 @@ proc param_run_test { param_name param_value } {
>>> >      global srcdir
>>> >      global subdir
>>> >
>>> > -    dg-runtest $srcdir/$subdir/blocksort-part.c "" "-O3 --param $param_name=$param_value"
>>> > +    dg-runtest $srcdir/$subdir/blocksort-part.c "-O3 --param $param_name=$param_value" ""
>>> >  }
>>> >
>>> >  set fd [open "$objdir/../../params.options" r]
>>> >
>>> >
>>> > Grüße
>>> >  Thomas
>>> >
>> Thanks for the patch, that's probably the same problem that Andreas spotted.
>> I'm sending candidate patch.
>>
>> Martin
>>
>>
>> 0001-Fix-params.exp-test-cases.patch
>>
>>
>> From d5e3c5cb20306352af7f45cd827669b7ff51a709 Mon Sep 17 00:00:00 2001
>> From: marxin <mliska@suse.cz>
>> Date: Mon, 18 Jul 2016 09:50:12 +0200
>> Subject: [PATCH] Fix params.exp test-cases
>>
>> gcc/testsuite/ChangeLog:
>>
>> 2016-07-18  Thomas Schwinge  <thomas@codesourcery.com>
>>     Martin Liska  <mliska@suse.cz>
>>
>>     * gcc.dg/params/LICENSE: Add missing LICENSE that is mentioned
>>     in a source file.
>>     * gcc.dg/params/README: Same for the README.
>>     * gcc.dg/params/params.exp (param_run_test): Swap flags and
>>     default-extra-flags arguments when calling dg-runtest.
> OK.
> 
> jeff
> 

Thanks.

I'm sending the version of patch I've just installed as r238633.

Martin

[RFC,v2] Test coverage for --param boundary values

Commit Message

Patch