diff mbox

[build,testsuite,v3] Increase gcc, g++, gfortran and libstdc++-v3 testsuite parallelism

Message ID yddhbjt40ty.fsf@manam.CeBiTec.Uni-Bielefeld.DE
State New
Headers show

Commit Message

Rainer Orth July 21, 2010, 9:54 a.m. UTC
[Please keep me on the Cc:, I'm not subscribed to libstdc++.  Thanks.]

make check times on slow machines with many cores (like the
UltraSPARC-T1/T2) is currently limited by too little parallelism in the
testsuite.  E.g., on a Sun Fire T5220 with a single UltraSPARC-T2 CPU
(1.2 GHz, 8 cores with 8 strands/hw threads each), you can easily run
bootstrap with make -j128 and the build completes in about 2 1/2 hours.
A subsequent make -j128 -k check (both 32 and 64-bit) takes 6 hours,
while even on my laptop (Core2 Duo T7500, 2.2 GHz, 2 cores), build and
test times are roughly identical.  This is not only a SPARC problem,
though: with 8-socket 8-core Nehalem EX machines coming, we have 64
cores per machine, doubling that with HT, so gcc testing won't exploit
the available parallelism.

As a first step to improve things, I've increased parallelism in the
testsuites with the largest number of tests (using wc -l <testsuite>.sum
as an estimate):

    2361 gcc/testsuite/ada/acats/acats.sum				 1
   49546 gcc/testsuite/g++/g++.sum					20
  145926 gcc/testsuite/gcc/gcc.sum					60
   74594 gcc/testsuite/gfortran/gfortran.sum				30
    1673 gcc/testsuite/gnat/gnat.sum					 1
    4079 gcc/testsuite/objc/objc.sum					 2
    5287 i386-pc-solaris2.11/libffi/testsuite/libffi.sum		 2
    5020 i386-pc-solaris2.11/libgomp/testsuite/libgomp.sum		 2
    5196 i386-pc-solaris2.11/libjava/testsuite/libjava.sum		 2
   14841 i386-pc-solaris2.11/libstdc++-v3/testsuite/libstdc++.sum	 6
  308523 total

With 300+ ktests total and a goal of spreading them about 128 jobs, we
get 2410 tests for each, and I've used that number as a guideline.  The
third column in the table above gives the number of jobs necessary to
achieve that goal.  One could go even further here considering there are
machines like the Sun Fire T5440 with 4 US-T2 CPUs, yielding 256 cores
and thus 512 jobs, but while single-cpu T1/T2 machines are readily
available, the latter are probably too expensive and I fear that further
increasing parallelism right now might add too much overhead on small
machines.

While using the test counts as estimates worked reasonably well in most
cases, it broke down in two cases: 

* The libstdc++ tests take much more time per test, so I had to increase
  parallelism considerably beyond the first estimate.

* The gcc.c-torture/compile/limits-fnargs.c test case alone takes 16+
  minutes to compile, the test is run 8 times with different
  optimization levels and the number is doubled again by testing both
  the 32 and 64-bit multilibs, so this single test case alone takes more
  than 4 hours to complete.  This is way over the top, so I've halved
  the number of function arguments.  This way, it still takes 3+
  minutes, which is bad, but manageable.

There are still some testsuites that run excessively long (last column
is runtime in seconds):

g++23.sep             6665
libstdc++13.sep       6920
libstdc++12.sep       7191
libjava               7994
gcc16.sep             8212
libstdc++15.sep       8466
libgomp               8615

I've so far refrained from parallelizing the libjava and libgomp
testsuites runs, and some of the libstdc++ test names use very long
common prefixes.  Still, the new state is far better than the current
one, so we might incrementally improve things if libstdc++ or the serial
testsuite runs begin to further dominate the make check times.

The changes below achive this and still try to keep the patterns
manageable.  Here are the make -j<2 * ncpu> -k check times for two tests
(T5220 as above, X7350 is a Sun Fire X4450 with 4 x 2.93 GHz Xeon X7350):

CPU		mainline	parallel

X7350	-j32	 63 min		 56 min
US-T2   -j128	358 min         143 min  

As you can see, on faster CPUs the change is in the noise (or the
parallel test is actually faster), while on the T2 there's a factor of 2.5
speedup.

mail-report.log is identical between both runs.

There's one partially related change included: even before this change,
there could be a strange double summary line in mail-report.log:

                === libstdc++ Summary ===

# of expected passes            13934
# of unexpected failures        8
# of unexpected successes       2
# of expected failures          118
# of unsupported tests          732

                === libstdc++ Summary ===

I could trace this to the fact that currently no libstdc++ ABI baselines
exist for Solaris 2 (being worked on), thus runtest abi.exp produces

[...]
                === libstdc++ Summary for unix ===

Running target unix/-m64
Running /vol/gcc/src/hg/trunk/local/libstdc++-v3/testsuite/libstdc++-abi/abi.exp ...

                === libstdc++ Summary for unix/-m64 ===


                === libstdc++ Summary ===


This situation is likely to occur more often in the future when runtest
is run with a .exp file that doesn't match any tests on a given platform
(like i386.exp on SPARC).  The fix is trivial: only emit the summary
line if any of the test counts is non-zero.

Ok for mainline and the 4.4 and 4.5 branches after some soaktime?

Thanks.
	Rainer


2010-03-07  Rainer Orth  <ro@CeBiTec.Uni-Bielefeld.DE>

	contrib:
	* dg-extract-results.sh (total.awk): Only print combined summary
	for non-zero total.

	gcc:
	* Makefile.in (check_gcc_parallelize): Increase parallelism.
	(check_p_numbers): Extend to 70.

	gcc/cp:
	* Make-lang.in (check_g++_parallelize): Increase parallelism.

	gcc/fortran:
	* Make-lang.in (check_gfortran_parallelize): Increase parallelism.

	gcc/testsuite:
	* gcc.c-torture/compile/limits-fnargs.c (PAR4HALF): Define.
	(func): Replace PAR4 by PAR4HALF.
	(ARG4HALF): Define.
	(caller): Call func with ARG4HALF.

	libstdc++-v3:
	* testsuite/Makefile.am (check_DEJAGNU_normal_targets): Extend to 17.
	Increase parallelism.
	* testsuite/Makefile.in: Regenerate.

Comments

Jakub Jelinek July 21, 2010, 10:43 a.m. UTC | #1
On Wed, Jul 21, 2010 at 11:54:17AM +0200, Rainer Orth wrote:
> [Please keep me on the Cc:, I'm not subscribed to libstdc++.  Thanks.]

I don't think we want to backport this to 4.4 or 4.5.

There are reasons why I've tried not to split the testing into too many
pieces, there is some constant cost needed to start runtest and e.g. compile
testcases that are used in the various dejagnu guards.

E.g.

+                     execute.exp=execute/2000* \                                                                                                  
+                     execute.exp=execute/2001* \                                                                                                  
+                     execute.exp=execute/2002* \                                                                                                  
+                     execute.exp=execute/2003* \                                                                                                  
+                     execute.exp=execute/2004* \                                                                                                  
+                     execute.exp=execute/2005* \                                                                                                  
+                     execute.exp=execute/2006* \                                                                                                  
+                     execute.exp=execute/2007* \                                                                                                  
+                     execute.exp=execute/2008* \                                                                                                  
+                     execute.exp=execute/2009* \                                                                                                  
+                     execute.exp=execute/2010* \                                                                                                  
+                     execute.exp=execute/201\[1-9\]* \                                                                                            
+                     execute.exp=execute/20\[2-9\]* \                                                                                             
+                     execute.exp=execute/2\[1-9\]* \                                                                                              
+                     execute.exp=execute/\[a-eA-E\]* \                                                                                            
+                     execute.exp=execute/\[f-mF-M\]* \                                                                                            
+                     execute.exp=execute/\[n-pN-P\]* \                                                                                            
+                     execute.exp=execute/\[q-zQ-Z\]* \                                                                                            

is IMHO way too small division.  The test counts for 20* are:
2000 88
2001 52
2002 72
2003 60
2004 56
2005 37
2006 16
2007 26
2008 19
2009 10
2010  5

so, while it might be appropriate to run 2000, 2001, 2002, 2003 and 2004
tests in separate jobs, you might as well run say 2005-2009 in another one
(or two).  On the other side, you don't mention guality.exp at all, and that
one takes several minutes to complete even on fast hw.

Much better than counting number of tests in some cases is to look at
...exp completed in ... seconds
lines (preferrably from a fast box than very slow one).

Anyway, even if you just count tests, start with counting the tests done
by that job and divide, decide into how many subjobs to split (the current
tuning was tuned for make -j24 check to make -j48 check, though the
testsuite changed a little since it was split, by adding new tests and by
adding -flto/-fwhopr testing) and then try to keep jobs roughly the same if
easily possible.  I don't think subdividing current jobs to more than say 5
or so parts is a good idea.  So say for execute.exp (or compile.exp) try
to have roughly the same sized chunks.  execute.exp has currently 1122 tests
and is ATM run as two parallel jobs, one with 2*.c (419 tests) and the rest.
So, you should AIM at roughly 100 tests in each group.  Perhaps
200\[069\]*, 200\[14\]*, 200\[27\]*, 200\[358\]*, 20\[1-9\]*,
2\[1-9\]*.
From the other former subjob, there are currently 286 9*.c tests, 188 pr*.c
tests and 229 other tests.  So you want to split the 9* group into say 3,
pr*.c into 2 and the other tests into 2 pieces.

	Jakub
Benjamin Kosnik July 21, 2010, 7:36 p.m. UTC | #2
On Wed, 21 Jul 2010 11:54:17 +0200
Rainer Orth <ro@CeBiTec.Uni-Bielefeld.DE> wrote:

> There's one partially related change included: even before this
> change, there could be a strange double summary line in
> mail-report.log:
> 
>                 === libstdc++ Summary ===
> 
> # of expected passes            13934
> # of unexpected failures        8
> # of unexpected successes       2
> # of expected failures          118
> # of unsupported tests          732
> 
>                 === libstdc++ Summary ===
> 
> I could trace this to the fact that currently no libstdc++ ABI
> baselines exist for Solaris 2 (being worked on), thus runtest abi.exp
> produces
> 
> [...]
>                 === libstdc++ Summary for unix ===
> 
> Running target unix/-m64
> Running /vol/gcc/src/hg/trunk/local/libstdc++-v3/testsuite/libstdc++-abi/abi.exp ...
> 
>                 === libstdc++ Summary for unix/-m64 ===
> 
> 
>                 === libstdc++ Summary ===
> 
> 
> This situation is likely to occur more often in the future when
> runtest is run with a .exp file that doesn't match any tests on a
> given platform (like i386.exp on SPARC).  The fix is trivial: only
> emit the summary line if any of the test counts is non-zero.

Sounds great to me. Feel free to check this part in as-is.

-benjamin
Mike Stump July 22, 2010, 2:35 a.m. UTC | #3
On Jul 21, 2010, at 2:54 AM, Rainer Orth wrote:
> make check times on slow machines with many cores (like the
> UltraSPARC-T1/T2) is currently limited by too little parallelism in the
> testsuite.

Thanks for your efforts.
diff mbox

Patch

diff -r cbdbde9aae00 contrib/dg-extract-results.sh
--- a/contrib/dg-extract-results.sh	Mon Jul 19 11:13:06 2010 +0200
+++ b/contrib/dg-extract-results.sh	Tue Jul 20 20:51:39 2010 +0200
@@ -401,7 +401,7 @@ 
 /^# of unresolved testcases/	{ unrescnt += \$5 }
 /^# of unsupported tests/	{ unsupcnt += \$5 }
 END {
-  printf ("\n\t\t=== %s Summary ===\n\n", tool)
+  if (passcnt + failcnt + xpasscnt + xfailcnt + untstcnt + unrescnt + unsupcnt != 0) printf ("\n\t\t=== %s Summary ===\n\n", tool)
   if (passcnt != 0) printf ("# of expected passes\t\t%d\n", passcnt)
   if (failcnt != 0) printf ("# of unexpected failures\t%d\n", failcnt)
   if (xpasscnt != 0) printf ("# of unexpected successes\t%d\n", xpasscnt)
diff -r cbdbde9aae00 gcc/Makefile.in
--- a/gcc/Makefile.in	Mon Jul 19 11:13:06 2010 +0200
+++ b/gcc/Makefile.in	Tue Jul 20 20:51:39 2010 +0200
@@ -494,14 +494,75 @@ 
 # very long running, but still should be split of from the check-parallel-$lang
 # remaining tests runtest invocation), they should be concatenated with commas.
 # Note that [a-zA-Z] wildcards need to have []s prefixed with \ (needed
-# by tcl) and as the *.exp arguments are mached both as is and with
+# by tcl) and as the *.exp arguments are matched both as is and with
 # */ prefixed to it in runtest_file_p, it is usually desirable to include
 # a subdirectory name.
-check_gcc_parallelize=execute.exp=execute/2* \
-		      execute.exp=execute/\[013-9a-zA-Z\]* \
-		      compile.exp dg.exp \
-		      dg-torture.exp,builtins.exp \
-		      struct-layout-1.exp,unsorted.exp,stackalign.exp,i386.exp
+check_gcc_parallelize=compile.exp=compile/\[013-9\]* \
+		      compile.exp=compile/2000* \
+		      compile.exp=compile/2001* \
+		      compile.exp=compile/2002* \
+		      compile.exp=compile/2003* \
+		      compile.exp=compile/2004* \
+		      compile.exp=compile/2005* \
+		      compile.exp=compile/2006* \
+		      compile.exp=compile/2007* \
+		      compile.exp=compile/2008* \
+		      compile.exp=compile/2009* \
+		      compile.exp=compile/2010* \
+		      compile.exp=compile/201\[1-9\]* \
+		      compile.exp=compile/20\[2-9\]* \
+		      compile.exp=compile/2\[1-9\]* \
+		      compile.exp=compile/\[a-oA-O\]* \
+		      compile.exp=compile/\[pP\]* \
+		      compile.exp=compile/\[q-zQ-Z\]* \
+		      execute.exp=execute/\[013-9\]* \
+		      execute.exp=execute/2000* \
+		      execute.exp=execute/2001* \
+		      execute.exp=execute/2002* \
+		      execute.exp=execute/2003* \
+		      execute.exp=execute/2004* \
+		      execute.exp=execute/2005* \
+		      execute.exp=execute/2006* \
+		      execute.exp=execute/2007* \
+		      execute.exp=execute/2008* \
+		      execute.exp=execute/2009* \
+		      execute.exp=execute/2010* \
+		      execute.exp=execute/201\[1-9\]* \
+		      execute.exp=execute/20\[2-9\]* \
+		      execute.exp=execute/2\[1-9\]* \
+		      execute.exp=execute/\[a-eA-E\]* \
+		      execute.exp=execute/\[f-mF-M\]* \
+		      execute.exp=execute/\[n-pN-P\]* \
+		      execute.exp=execute/\[q-zQ-Z\]* \
+		      unsorted.exp=unsorted/\[0-9a-cA-C\]* \
+		      unsorted.exp=unsorted/\[dD\]* \
+		      unsorted.exp=unsorted/\[e-tE-T\]* \
+		      unsorted.exp=unsorted/\[u-zU-Z\]* \
+		      dg.exp=gcc.dg/\[0-9a-bA-B\]* \
+		      dg.exp=gcc.dg/\[cC\]* \
+		      dg.exp=gcc.dg/\[d-iD-I\]* \
+		      dg.exp=gcc.dg/\[j-nJ-N\]* \
+		      dg.exp=gcc.dg/\[o-pO-P\]* \
+		      dg.exp=gcc.dg/\[q-tQ-T\]* \
+		      dg.exp=gcc.dg/\[u-zU-Z\]* \
+		      dg.exp=c-c++-common/* \
+		      dg-torture.exp=torture/\[0-9a-bA-B\]* \
+		      dg-torture.exp=torture/\[c-oC-O\]* \
+		      dg-torture.exp=torture/\[pP\]* \
+		      dg-torture.exp=torture/\[q-zQ-Z\]* \
+		      builtins.exp \
+		      cpp.exp \
+		      debug.exp \
+		      format.exp \
+		      i386.exp \
+		      ieee.exp \
+		      noncompile.exp \
+		      pch.exp \
+		      stackalign.exp \
+		      struct-layout-1.exp \
+		      test-framework.exp \
+		      tree-ssa.exp \
+		      vect.exp
 lang_opt_files=@lang_opt_files@ $(srcdir)/c-family/c.opt $(srcdir)/common.opt
 lang_specs_files=@lang_specs_files@
 lang_tree_files=@lang_tree_files@
@@ -4892,7 +4953,13 @@ 
 check_p_subno=$(word 2,$(subst _, ,$*))
 check_p_comma=,
 check_p_subwork=$(subst $(check_p_comma), ,$(if $(check_p_subno),$(word $(check_p_subno),$(check_p_vars))))
-check_p_numbers=1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
+check_p_numbers=1 2 3 4 5 6 7 8 9 10 \
+	11 12 13 14 15 16 17 18 19 20 \
+	21 22 23 24 25 26 27 28 29 30 \
+	31 32 33 34 35 36 37 38 39 40 \
+	41 42 43 44 45 46 47 48 49 50 \
+	51 52 53 54 55 56 57 58 59 60 \
+	61 62 63 64 65 66 67 68 69 70
 check_p_subdir=$(subst _,,$*)
 check_p_subdirs=$(wordlist 1,$(words $(check_$*_parallelize)),$(check_p_numbers))
 
diff -r cbdbde9aae00 gcc/cp/Make-lang.in
--- a/gcc/cp/Make-lang.in	Mon Jul 19 11:13:06 2010 +0200
+++ b/gcc/cp/Make-lang.in	Tue Jul 20 20:51:39 2010 +0200
@@ -143,7 +143,29 @@ 
 lang_checks += check-g++
 lang_checks_parallelized += check-g++
 # For description see comment above check_gcc_parallelize in gcc/Makefile.in.
-check_g++_parallelize = old-deja.exp dg.exp
+check_g++_parallelize = old-deja.exp=g++.old-deja/g++.\[a-b\]* \
+			old-deja.exp=g++.old-deja/g++.\[c-k\]* \
+			old-deja.exp=g++.old-deja/g++.\[l-m\]* \
+			old-deja.exp=g++.old-deja/g++.\[n-o\]* \
+			old-deja.exp=g++.old-deja/g++.\[p-r\]* \
+			old-deja.exp=g++.old-deja/g++.\[s-z\]* \
+			debug.exp \
+			dg.exp=c-c++-common/* \
+			dg.exp=g++.dg/\[0-9a-bA-B\]* \
+			dg.exp=g++.dg/\[cC\]\[a-pA-P\]* \
+			dg.exp=g++.dg/\[cC\]\[q-zQ-Z0-9_\]* \
+			dg.exp=g++.dg/\[dD\]* \
+			dg.exp=g++.dg/\[eE\]* \
+			dg.exp=g++.dg/\[f-kF-K\]* \
+			dg.exp=g++.dg/\[l-nL-N\]* \
+			dg.exp=g++.dg/\[oO\]* \
+			dg.exp=g++.dg/\[pP\]* \
+			dg.exp=g++.dg/\[q-sQ-S\]* \
+			dg.exp=g++.dg/\[tT\]\[a-ea-E\]* \
+			dg.exp=g++.dg/\[tT\]\[f-oF-O\]* \
+			dg.exp=g++.dg/\[tT\]\[p-zP-Z0-9_\]* \
+			dg.exp=g++.dg/\[u-zU-Z\]* \
+			dg-torture.exp
 
 #
 # Install hooks:
diff -r cbdbde9aae00 gcc/fortran/Make-lang.in
--- a/gcc/fortran/Make-lang.in	Mon Jul 19 11:13:06 2010 +0200
+++ b/gcc/fortran/Make-lang.in	Tue Jul 20 20:51:39 2010 +0200
@@ -170,9 +170,40 @@ 
 lang_checks += check-gfortran
 lang_checks_parallelized += check-gfortran
 # For description see comment above check_gcc_parallelize in gcc/Makefile.in.
-check_gfortran_parallelize = dg.exp=gfortran.dg/\[a-cA-C\]* \
-			     dg.exp=gfortran.dg/\[d-mD-M\]* \
-			     dg.exp=gfortran.dg/\[n-zN-Z0-9\]*
+check_gfortran_parallelize = dg.exp=gfortran.dg/\[aA\]\[a-lA-L\]* \
+			     dg.exp=gfortran.dg/\[aA\]\[m-rM-R\]* \
+			     dg.exp=gfortran.dg/\[aA\]\[s-zS-Z0-9_\]* \
+			     dg.exp=gfortran.dg/\[bB\]\[a-nA-N\]* \
+			     dg.exp=gfortran.dg/\[bB\]\[o-zO-Z0-9_\]* \
+			     dg.exp=gfortran.dg/\[cC\]\[a-mA-M\]* \
+			     dg.exp=gfortran.dg/\[cC\]\[n-zN-Z0-9_\]* \
+			     dg.exp=gfortran.dg/\[dD\]\[a-mA-M\]* \
+			     dg.exp=gfortran.dg/\[dD\]\[n-zN-Z0-9_\]* \
+			     dg.exp=gfortran.dg/\[eE\]* \
+			     dg.exp=gfortran.dg/\[fF\]* \
+			     dg.exp=gfortran.dg/\[gG\]\[a-zA-Z\]* \
+			     dg.exp=gfortran.dg/\[gG\]\[0-9_\]* \
+			     dg.exp=gfortran.dg/\[hH\]* \
+			     dg.exp=gfortran.dg/\[iI\]\[a-mA-M\]* \
+			     dg.exp=gfortran.dg/\[iI\]\[nN\]* \
+			     dg.exp=gfortran.dg/\[iI\]\[o-zO-Z0-9_\]* \
+			     dg.exp=gfortran.dg/\[j-lJ-L\]* \
+			     dg.exp=gfortran.dg/\[mM\]* \
+			     dg.exp=gfortran.dg/\[nN\]* \
+			     dg.exp=gfortran.dg/\[oO\]* \
+			     dg.exp=gfortran.dg/\[pP\]\[a-qA-Q\]* \
+			     dg.exp=gfortran.dg/\[pP\]\[r-zR-Z0-9_\]* \
+			     dg.exp=gfortran.dg/\[q-rQ-R\]* \
+			     dg.exp=gfortran.dg/\[sS\]* \
+			     dg.exp=gfortran.dg/\[tT\]* \
+			     dg.exp=gfortran.dg/\[u-vU-V\]* \
+			     dg.exp=gfortran.dg/\[wW\]* \
+			     dg.exp=gfortran.dg/\[x-zX-Z0-9_\]* \
+			     compile.exp \
+			     execute.exp=execute/\[a-eA-E\]* \
+			     execute.exp=execute/\[f-iF-I\]* \
+			     execute.exp=execute/\[j-tJ-T\]* \
+			     execute.exp=execute/\[u-zU-Z0-9_\]*
 
 # GFORTRAN documentation.
 GFORTRAN_TEXI = \
diff -r cbdbde9aae00 gcc/testsuite/gcc.c-torture/compile/limits-fnargs.c
--- a/gcc/testsuite/gcc.c-torture/compile/limits-fnargs.c	Mon Jul 19 11:13:06 2010 +0200
+++ b/gcc/testsuite/gcc.c-torture/compile/limits-fnargs.c	Tue Jul 20 20:51:39 2010 +0200
@@ -4,19 +4,21 @@ 
 #define PAR2 PAR1, PAR1, PAR1, PAR1, PAR1, PAR1, PAR1, PAR1, PAR1, PAR1
 #define PAR3 PAR2, PAR2, PAR2, PAR2, PAR2, PAR2, PAR2, PAR2, PAR2, PAR2
 #define PAR4 PAR3, PAR3, PAR3, PAR3, PAR3, PAR3, PAR3, PAR3, PAR3, PAR3
+#define PAR4HALF PAR3, PAR3, PAR3, PAR3, PAR3
 #define PAR5 PAR4, PAR4, PAR4, PAR4, PAR4, PAR4, PAR4, PAR4, PAR4, PAR4
 #define PAR6 PAR5, PAR5, PAR5, PAR5, PAR5, PAR5, PAR5, PAR5, PAR5, PAR5
 
-extern void func (PAR4);
+extern void func (PAR4HALF);
 
 #define ARG1 0,1,2,3,4,5,6,7,8,9
 #define ARG2 ARG1, ARG1, ARG1, ARG1, ARG1, ARG1, ARG1, ARG1, ARG1, ARG1
 #define ARG3 ARG2, ARG2, ARG2, ARG2, ARG2, ARG2, ARG2, ARG2, ARG2, ARG2
 #define ARG4 ARG3, ARG3, ARG3, ARG3, ARG3, ARG3, ARG3, ARG3, ARG3, ARG3
+#define ARG4HALF ARG3, ARG3, ARG3, ARG3, ARG3
 #define ARG5 ARG4, ARG4, ARG4, ARG4, ARG4, ARG4, ARG4, ARG4, ARG4, ARG4
 #define ARG5HALF ARG5, ARG5, ARG5, ARG5, ARG5
 
 void caller(void)
 {
-  func (ARG4);
+  func (ARG4HALF);
 }
diff -r cbdbde9aae00 libstdc++-v3/testsuite/Makefile.am
--- a/libstdc++-v3/testsuite/Makefile.am	Mon Jul 19 11:13:06 2010 +0200
+++ b/libstdc++-v3/testsuite/Makefile.am	Tue Jul 20 20:51:39 2010 +0200
@@ -100,7 +100,7 @@ 
 	@test ! -f $*/site.exp || mv $*/site.exp $*/site.bak
 	@mv $*/site.exp.tmp $*/site.exp
 
-check_DEJAGNU_normal_targets = $(patsubst %,check-DEJAGNUnormal%,0 1 2 3)
+check_DEJAGNU_normal_targets = $(patsubst %,check-DEJAGNUnormal%,0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17)
 $(check_DEJAGNU_normal_targets): check-DEJAGNUnormal%: normal%/site.exp
 
 # Run the testsuite in normal mode.
@@ -110,8 +110,15 @@ 
 	if [ -z "$*$(filter-out --target_board=%, $(RUNTESTFLAGS))" ] \
 	    && [ "$(filter -j, $(MFLAGS))" = "-j" ]; then \
 	  $(MAKE) $(AM_MAKEFLAGS) check-DEJAGNUnormal0 check-DEJAGNUnormal1 \
-		  check-DEJAGNUnormal2 check-DEJAGNUnormal3; \
-	  for idx in 0 1 2 3; do \
+		  check-DEJAGNUnormal2 check-DEJAGNUnormal3 \
+		  check-DEJAGNUnormal4 check-DEJAGNUnormal5 \
+		  check-DEJAGNUnormal6 check-DEJAGNUnormal7 \
+		  check-DEJAGNUnormal8 check-DEJAGNUnormal9 \
+		  check-DEJAGNUnormal10 check-DEJAGNUnormal11 \
+		  check-DEJAGNUnormal12 check-DEJAGNUnormal13 \
+		  check-DEJAGNUnormal14 check-DEJAGNUnormal15 \
+		  check-DEJAGNUnormal16 check-DEJAGNUnormal17; \
+	  for idx in 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17; do \
 	    mv -f normal$$idx/libstdc++.sum normal$$idx/libstdc++.sum.sep; \
 	    mv -f normal$$idx/libstdc++.log normal$$idx/libstdc++.log.sep; \
 	  done; \
@@ -120,11 +127,26 @@ 
 	  $(SHELL) $(srcdir)/../../contrib/dg-extract-results.sh \
 	    libstdc++.sum.sep normal0/libstdc++.sum.sep \
 	    normal1/libstdc++.sum.sep normal2/libstdc++.sum.sep \
-	    normal3/libstdc++.sum.sep > libstdc++.sum; \
+	    normal3/libstdc++.sum.sep normal4/libstdc++.sum.sep \
+	    normal5/libstdc++.sum.sep normal6/libstdc++.sum.sep \
+	    normal7/libstdc++.sum.sep normal8/libstdc++.sum.sep \
+	    normal9/libstdc++.sum.sep normal10/libstdc++.sum.sep \
+	    normal11/libstdc++.sum.sep normal12/libstdc++.sum.sep \
+	    normal13/libstdc++.sum.sep normal14/libstdc++.sum.sep \
+	    normal15/libstdc++.sum.sep normal16/libstdc++.sum.sep \
+	    normal17/libstdc++.sum.sep \
+	    normal14/libstdc++.sum.sep > libstdc++.sum; \
 	  $(SHELL) $(srcdir)/../../contrib/dg-extract-results.sh -L \
 	    libstdc++.log.sep normal0/libstdc++.log.sep \
 	    normal1/libstdc++.log.sep normal2/libstdc++.log.sep \
-	    normal3/libstdc++.log.sep > libstdc++.log; \
+	    normal3/libstdc++.log.sep normal4/libstdc++.log.sep \
+	    normal5/libstdc++.log.sep normal6/libstdc++.log.sep \
+	    normal7/libstdc++.log.sep normal8/libstdc++.log.sep \
+	    normal9/libstdc++.log.sep normal10/libstdc++.log.sep \
+	    normal11/libstdc++.log.sep normal12/libstdc++.log.sep \
+	    normal13/libstdc++.log.sep normal14/libstdc++.log.sep \
+	    normal15/libstdc++.log.sep normal16/libstdc++.log.sep \
+	    normal17/libstdc++.log.sep > libstdc++.log; \
 	  exit 0; \
 	fi; \
 	srcdir=`$(am__cd) $(srcdir) && pwd`; export srcdir; \
@@ -140,13 +162,41 @@ 
 			$(RUNTESTFLAGS) abi.exp; \
 	    else echo "WARNING: could not find \`runtest'" 1>&2; :;\
 	    fi; \
-	    dirs="`cd $$srcdir; echo [013-9][0-9]_*/* [ab]* de* [ep]*/*`";; \
+	    dirs="`cd $$srcdir; echo [01][0-9]_*/*`";; \
 	  normal1) \
-	    dirs="`cd $$srcdir; echo 2[0-2]_*/*`";; \
+	    dirs="`cd $$srcdir; echo [3-9][0-9]_*/*`";; \
 	  normal2) \
-	    dirs="`cd $$srcdir; echo 2[4-9]_*/*`";; \
+	    dirs="`cd $$srcdir; echo 20_*/*`";; \
 	  normal3) \
-	    dirs="`cd $$srcdir; echo 23_*/* t*/*`";; \
+	    dirs="`cd $$srcdir; echo 21_*/*`";; \
+	  normal4) \
+	    dirs="`cd $$srcdir; echo 22_*/*`";; \
+	  normal5) \
+	    dirs="`cd $$srcdir; echo 23_*/[0-9a-hA-H]*`";; \
+	  normal6) \
+	    dirs="`cd $$srcdir; echo 23_*/[i-mI-M]*`";; \
+	  normal7) \
+	    dirs="`cd $$srcdir; echo 23_*/[n-uN-U]*`";; \
+	  normal8) \
+	    dirs="`cd $$srcdir; echo 23_*/[v-zV-Z]*`";; \
+	  normal9) \
+	    dirs="`cd $$srcdir; echo 24_*/*`";; \
+	  normal10) \
+	    dirs="`cd $$srcdir; echo 25_*/*`";; \
+	  normal11) \
+	    dirs="`cd $$srcdir; echo 26_*/*`";; \
+	  normal12) \
+	    dirs="`cd $$srcdir; echo 27_*/*`";; \
+	  normal13) \
+	    dirs="`cd $$srcdir; echo 2[89]_*/*`";; \
+	  normal14) \
+	    dirs="`cd $$srcdir; echo [ab]* de*`";; \
+	  normal15) \
+	    dirs="`cd $$srcdir; echo e*/*`";; \
+	  normal16) \
+	    dirs="`cd $$srcdir; echo p*/*`";; \
+	  normal17) \
+	    dirs="`cd $$srcdir; echo t*/*`";; \
 	esac; \
 	if [ -n "$*" ]; then cd "$*"; fi; \
 	if $(SHELL) -c "$$runtest --version" > /dev/null 2>&1; then \