[pph] Test cleanup (issue4572050)

Update the test suite to reflect recent fixes and merges from trunk.

The following changes happened in this patch.

Conversion of a compilation failure to an assembly diff failure:

    remove the dg comments and add the comment // pph asm xdiff

Conversion of a ICE to a BOGUS

    change the dg-xfail-if line from ICE to BOGUS
    change the dg-bogus line from the ICE message to the BOGUS message
    remove the dg-prune-output lines corresponding to the ICE
        (dg knows to remove them for errors, but not for ICEs)

Update line numbers from a merge (or other edits to regular sources)
    change the line number from old to new as reflected in the test log

The output of make check filtered through the following sed command
should yield no FAIL and no XPASS.

sed -e '
	/-fpph-map=pph.map/	! {
		/^XPASS: .*test for bogus messages/	d
		/^XPASS: .*test for excess errors/	d
	}

	/^#/	p
	/^ERROR: /	p
	/^XFAIL: /	p
	/^XPASS: /	p
	/^FAIL: /	p

	d
'

The same command works to filter the log file to its essential record
of pass/fail.  In particular, it enables listing XFAILs that need work.



--
This patch is available for review at http://codereview.appspot.com/4572050

Hey,

so I really like the new clean testing system, so that we always
quickly see what we fixed/broked with a local change.

One problem now though: `// pph asm xdiff`, only flags for asm diffs,
but those could be different diffs after a change (for the better or
worse) and this won't be caught. It's probably hard to get something
precise on this, but maybe we could simply add the # of lines of diff
expected, e.g. `// pph asm xdiff 32`. Then we XFAIL if the number of
expected lines in the diff match, but actually fail if the number of
lines in the diff is now different.

I'm not very familiar with dg.. Is that doable? Would be very helpful
at this stage.

Gab

On Wed, Jun 8, 2011 at 1:44 PM, Lawrence Crowl <crowl@google.com> wrote:
> Update the test suite to reflect recent fixes and merges from trunk.
>
> The following changes happened in this patch.
>
> Conversion of a compilation failure to an assembly diff failure:
>
>    remove the dg comments and add the comment // pph asm xdiff
>
> Conversion of a ICE to a BOGUS
>
>    change the dg-xfail-if line from ICE to BOGUS
>    change the dg-bogus line from the ICE message to the BOGUS message
>    remove the dg-prune-output lines corresponding to the ICE
>        (dg knows to remove them for errors, but not for ICEs)
>
> Update line numbers from a merge (or other edits to regular sources)
>    change the line number from old to new as reflected in the test log
>
> The output of make check filtered through the following sed command
> should yield no FAIL and no XPASS.
>
> sed -e '
>        /-fpph-map=pph.map/     ! {
>                /^XPASS: .*test for bogus messages/     d
>                /^XPASS: .*test for excess errors/      d
>        }
>
>        /^#/    p
>        /^ERROR: /      p
>        /^XFAIL: /      p
>        /^XPASS: /      p
>        /^FAIL: /       p
>
>        d
> '
>
> The same command works to filter the log file to its essential record
> of pass/fail.  In particular, it enables listing XFAILs that need work.
>
>
> Index: gcc/testsuite/ChangeLog.pph
>
> 2011-06-08  Lawrence Crowl <crowl@google.com>
>
>        * lib/dg-pph.exp (dg-pph-pos): Stop redundantly reporting a missing
>        pph assembly as an xfail when its compile xfails.
>        * g++.dg/pph/x1typerefs.cc: Replace ICE xfail with a bogus error xfail.
>        * g++.dg/pph/x1tmplclass.cc: Likewise.
>        * g++.dg/pph/x1autometh.cc: Replace ICE xfail with an assembly diff
>        xfail.
>        * g++.dg/pph/x1special.cc: Remove ICE xfail.
>        * g++.dg/pph/x1functions.cc: Correct xfail to BOGUS rather than ERROR.
>        Remove leftover pruning from previous ICE.
>        * g++.dg/pph/x1template.cc: Update line number of ICE.
>        * g++.dg/pph/x1variables.cc: Update line number of BOGUS.
>
>
> Index: gcc/testsuite/lib/dg-pph.exp
> ===================================================================
> --- gcc/testsuite/lib/dg-pph.exp        (revision 174789)
> +++ gcc/testsuite/lib/dg-pph.exp        (working copy)
> @@ -94,9 +94,7 @@ proc dg-pph-pos { subdir test options ma
>     # Quit if it did not compile successfully.
>     if { ![file_on_host exists "$bname.s"] } {
>        # Expect assembly to be missing when the compile is an expected fail.
> -       if { [llength [grep $test "dg-xfail-if.*-fpph-map"]] } {
> -           xfail "$nshort $options (pph assembly missing)"
> -       } else {
> +       if { ![llength [grep $test "dg-xfail-if.*-fpph-map"]] } {
>            fail "$nshort $options (pph assembly missing)"
>        }
>        return
> Index: gcc/testsuite/g++.dg/pph/x1typerefs.cc
> ===================================================================
> --- gcc/testsuite/g++.dg/pph/x1typerefs.cc      (revision 174789)
> +++ gcc/testsuite/g++.dg/pph/x1typerefs.cc      (working copy)
> @@ -1,6 +1,5 @@
> -// { dg-xfail-if "ICE" { "*-*-*" } { "-fpph-map=pph.map" } }
> -// { dg-bogus "c1typerefs.h:11:18: internal compiler error: Segmentation fault" "" { xfail *-*-* } 0 }
> -// { dg-prune-output "In file included from " }
> +// { dg-xfail-if "BOGUS" { "*-*-*" } { "-fpph-map=pph.map" } }
> +// { dg-bogus "c1typerefs.h:11:18: error: cannot convert 'const std::type_info.' to 'const std::type_info.' in initialization" "" { xfail *-*-* } 0 }
>
>  #include "x1typerefs.h"
>
> Index: gcc/testsuite/g++.dg/pph/x1autometh.cc
> ===================================================================
> --- gcc/testsuite/g++.dg/pph/x1autometh.cc      (revision 174789)
> +++ gcc/testsuite/g++.dg/pph/x1autometh.cc      (working copy)
> @@ -1,6 +1,4 @@
> -// { dg-xfail-if "ICE" { "*-*-*" } { "-fpph-map=pph.map" } }
> -// { dg-bogus "x1autometh.h:8:1: internal compiler error: Segmentation fault" "" { xfail *-*-* } 0 }
> -// { dg-prune-output "In file included from " }
> +// pph asm xdiff
>
>  #include "x1autometh.h"
>
> Index: gcc/testsuite/g++.dg/pph/x1special.cc
> ===================================================================
> --- gcc/testsuite/g++.dg/pph/x1special.cc       (revision 174789)
> +++ gcc/testsuite/g++.dg/pph/x1special.cc       (working copy)
> @@ -1,7 +1,3 @@
> -// { dg-xfail-if "ICE" { "*-*-*" } { "-fpph-map=pph.map" } }
> -// { dg-bogus "x1special.h:10:5: internal compiler error: Segmentation fault" "" { xfail *-*-* } 0 }
> -// { dg-prune-output "In file included from " }
> -
>  #include "x1special.h"
>
>  B b(1);
> Index: gcc/testsuite/g++.dg/pph/x1functions.cc
> ===================================================================
> --- gcc/testsuite/g++.dg/pph/x1functions.cc     (revision 174789)
> +++ gcc/testsuite/g++.dg/pph/x1functions.cc     (working copy)
> @@ -1,8 +1,5 @@
> -// { dg-xfail-if "ERROR" { "*-*-*" } { "-fpph-map=pph.map" } }
> +// { dg-xfail-if "BOGUS" { "*-*-*" } { "-fpph-map=pph.map" } }
>  // { dg-bogus "'mbr_decl_inline' was not declared in this scope" "" { xfail *-*-* } 0 }
> -// { dg-prune-output "In file included from " }
> -// { dg-prune-output "In member function " }
> -// { dg-prune-output "At global scope:" }
>
>  #include "x1functions.h"
>
> Index: gcc/testsuite/g++.dg/pph/x1template.cc
> ===================================================================
> --- gcc/testsuite/g++.dg/pph/x1template.cc      (revision 174789)
> +++ gcc/testsuite/g++.dg/pph/x1template.cc      (working copy)
> @@ -1,5 +1,5 @@
>  // { dg-xfail-if "ICE" { "*-*-*" } { "-fpph-map=pph.map" } }
> -// { dg-bogus "x1template.h:18:13: internal compiler error: in resume_scope, at cp/name-lookup.c:1567" "" { xfail *-*-* } 0 }
> +// { dg-bogus "x1template.h:18:13: internal compiler error: in resume_scope, at cp/name-lookup.c:1569" "" { xfail *-*-* } 0 }
>  // { dg-prune-output "In file included from " }
>
>  #include "x1template.h"
> Index: gcc/testsuite/g++.dg/pph/x1tmplclass.cc
> ===================================================================
> --- gcc/testsuite/g++.dg/pph/x1tmplclass.cc     (revision 174789)
> +++ gcc/testsuite/g++.dg/pph/x1tmplclass.cc     (working copy)
> @@ -1,6 +1,5 @@
> -// { dg-xfail-if "ICE" { "*-*-*" } { "-fpph-map=pph.map" } }
> -// { dg-bogus "x1tmplclass.h:14:17: internal compiler error: Segmentation fault" "" { xfail *-*-* } 0 }
> -// { dg-prune-output "In file included from " }
> +// { dg-xfail-if "BOGUS" { "*-*-*" } { "-fpph-map=pph.map" } }
> +// { dg-bogus "x1tmplclass.h:14:5: error: specializing member .wrapper<char>::cache. requires .template<>. syntax" "" { xfail *-*-* } 0 }
>
>  #include "x1tmplclass.h"
>
> Index: gcc/testsuite/g++.dg/pph/x1variables.cc
> ===================================================================
> --- gcc/testsuite/g++.dg/pph/x1variables.cc     (revision 174789)
> +++ gcc/testsuite/g++.dg/pph/x1variables.cc     (working copy)
> @@ -1,7 +1,6 @@
>  // { dg-xfail-if "BOGUS" { "*-*-*" } { "-fpph-map=pph.map" } }
> -// { dg-bogus "c1variables.h:4:8: error: 'int D::mbr_uninit_plain' is not a static member of 'struct D'" "" { xfail *-*-* } 0 }
> -// { dg-bogus "c1variables.h:5:14: error: 'const int D::mbr_init_const' is not a static member of 'struct D'" "" { xfail *-*-* } 0 }
> -// { dg-prune-output "In file included from " }
> +// { dg-bogus "c1variables.h:5:8: error: 'int D::mbr_uninit_plain' is not a static member of 'struct D'" "" { xfail *-*-* } 0 }
> +// { dg-bogus "c1variables.h:6:14: error: 'const int D::mbr_init_const' is not a static member of 'struct D'" "" { xfail *-*-* } 0 }
>
>  #include "x1variables.h"
>
>
> --
> This patch is available for review at http://codereview.appspot.com/4572050
>

On 7/1/11, Gabriel Charette <gchare@google.com> wrote:
> One problem now though: `// pph asm xdiff`, only flags for asm diffs,
> but those could be different diffs after a change (for the better or
> worse) and this won't be caught. It's probably hard to get something
> precise on this, but maybe we could simply add the # of lines of diff
> expected, e.g. `// pph asm xdiff 32`. Then we XFAIL if the number of
> expected lines in the diff match, but actually fail if the number of
> lines in the diff is now different.
>
> I'm not very familiar with dg.. Is that doable? Would be very helpful
> at this stage.

That looks easy enough.  I need to finish the current test stuff
before I get to that though.

After having a look at how pph.exp works last Friday I think I could
do this myself easily enough.

Or are you still modifying the tests and want me to avoid touching this for now?

Gab

On Fri, Jul 1, 2011 at 5:51 PM, Lawrence Crowl <crowl@google.com> wrote:
> On 7/1/11, Gabriel Charette <gchare@google.com> wrote:
>> One problem now though: `// pph asm xdiff`, only flags for asm diffs,
>> but those could be different diffs after a change (for the better or
>> worse) and this won't be caught. It's probably hard to get something
>> precise on this, but maybe we could simply add the # of lines of diff
>> expected, e.g. `// pph asm xdiff 32`. Then we XFAIL if the number of
>> expected lines in the diff match, but actually fail if the number of
>> lines in the diff is now different.
>>
>> I'm not very familiar with dg.. Is that doable? Would be very helpful
>> at this stage.
>
> That looks easy enough.  I need to finish the current test stuff
> before I get to that though.
>
> --
> Lawrence Crowl
>