From patchwork Wed Jul 21 09:54:17 2010 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rainer Orth X-Patchwork-Id: 59428 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from sourceware.org (server1.sourceware.org [209.132.180.131]) by ozlabs.org (Postfix) with SMTP id 875761007D1 for ; Wed, 21 Jul 2010 19:54:48 +1000 (EST) Received: (qmail 26910 invoked by alias); 21 Jul 2010 09:54:46 -0000 Received: (qmail 26885 invoked by uid 22791); 21 Jul 2010 09:54:40 -0000 X-SWARE-Spam-Status: No, hits=-0.9 required=5.0 tests=AWL, BAYES_20, TW_BD, TW_DB, TW_IB, TW_TC, T_RP_MATCHES_RCVD X-Spam-Check-By: sourceware.org Received: from snape.CeBiTec.Uni-Bielefeld.DE (HELO smtp-relay.CeBiTec.Uni-Bielefeld.DE) (129.70.160.84) by sourceware.org (qpsmtpd/0.43rc1) with ESMTP; Wed, 21 Jul 2010 09:54:32 +0000 Received: from localhost (localhost.CeBiTec.Uni-Bielefeld.DE [127.0.0.1]) by smtp-relay.CeBiTec.Uni-Bielefeld.DE (Postfix) with ESMTP id 14B0AC91; Wed, 21 Jul 2010 11:54:30 +0200 (CEST) Received: from smtp-relay.CeBiTec.Uni-Bielefeld.DE ([127.0.0.1]) by localhost (malfoy.CeBiTec.Uni-Bielefeld.DE [127.0.0.1]) (amavisd-new, port 10024) with LMTP id xTSBcPanjyMK; Wed, 21 Jul 2010 11:54:19 +0200 (CEST) Received: from manam.CeBiTec.Uni-Bielefeld.DE (manam.CeBiTec.Uni-Bielefeld.DE [129.70.161.120]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by smtp-relay.CeBiTec.Uni-Bielefeld.DE (Postfix) with ESMTPS id E0350C8C; Wed, 21 Jul 2010 11:54:18 +0200 (CEST) Received: (from ro@localhost) by manam.CeBiTec.Uni-Bielefeld.DE (8.14.3+Sun/8.14.3/Submit) id o6L9sH4d012454; Wed, 21 Jul 2010 11:54:17 +0200 (MEST) From: Rainer Orth To: gcc-patches@gcc.gnu.org Cc: libstdc++@gcc.gnu.org, Janis Johnson , Paolo Bonzini , Jakub Jelinek Subject: [build, testsuite, v3] Increase gcc, g++, gfortran and libstdc++-v3 testsuite parallelism Date: Wed, 21 Jul 2010 11:54:17 +0200 Message-ID: User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.2.50 (usg-unix-v) MIME-Version: 1.0 X-IsSubscribed: yes Mailing-List: contact gcc-patches-help@gcc.gnu.org; run by ezmlm Precedence: bulk List-Id: List-Unsubscribe: List-Archive: List-Post: List-Help: Sender: gcc-patches-owner@gcc.gnu.org Delivered-To: mailing list gcc-patches@gcc.gnu.org [Please keep me on the Cc:, I'm not subscribed to libstdc++. Thanks.] make check times on slow machines with many cores (like the UltraSPARC-T1/T2) is currently limited by too little parallelism in the testsuite. E.g., on a Sun Fire T5220 with a single UltraSPARC-T2 CPU (1.2 GHz, 8 cores with 8 strands/hw threads each), you can easily run bootstrap with make -j128 and the build completes in about 2 1/2 hours. A subsequent make -j128 -k check (both 32 and 64-bit) takes 6 hours, while even on my laptop (Core2 Duo T7500, 2.2 GHz, 2 cores), build and test times are roughly identical. This is not only a SPARC problem, though: with 8-socket 8-core Nehalem EX machines coming, we have 64 cores per machine, doubling that with HT, so gcc testing won't exploit the available parallelism. As a first step to improve things, I've increased parallelism in the testsuites with the largest number of tests (using wc -l .sum as an estimate): 2361 gcc/testsuite/ada/acats/acats.sum 1 49546 gcc/testsuite/g++/g++.sum 20 145926 gcc/testsuite/gcc/gcc.sum 60 74594 gcc/testsuite/gfortran/gfortran.sum 30 1673 gcc/testsuite/gnat/gnat.sum 1 4079 gcc/testsuite/objc/objc.sum 2 5287 i386-pc-solaris2.11/libffi/testsuite/libffi.sum 2 5020 i386-pc-solaris2.11/libgomp/testsuite/libgomp.sum 2 5196 i386-pc-solaris2.11/libjava/testsuite/libjava.sum 2 14841 i386-pc-solaris2.11/libstdc++-v3/testsuite/libstdc++.sum 6 308523 total With 300+ ktests total and a goal of spreading them about 128 jobs, we get 2410 tests for each, and I've used that number as a guideline. The third column in the table above gives the number of jobs necessary to achieve that goal. One could go even further here considering there are machines like the Sun Fire T5440 with 4 US-T2 CPUs, yielding 256 cores and thus 512 jobs, but while single-cpu T1/T2 machines are readily available, the latter are probably too expensive and I fear that further increasing parallelism right now might add too much overhead on small machines. While using the test counts as estimates worked reasonably well in most cases, it broke down in two cases: * The libstdc++ tests take much more time per test, so I had to increase parallelism considerably beyond the first estimate. * The gcc.c-torture/compile/limits-fnargs.c test case alone takes 16+ minutes to compile, the test is run 8 times with different optimization levels and the number is doubled again by testing both the 32 and 64-bit multilibs, so this single test case alone takes more than 4 hours to complete. This is way over the top, so I've halved the number of function arguments. This way, it still takes 3+ minutes, which is bad, but manageable. There are still some testsuites that run excessively long (last column is runtime in seconds): g++23.sep 6665 libstdc++13.sep 6920 libstdc++12.sep 7191 libjava 7994 gcc16.sep 8212 libstdc++15.sep 8466 libgomp 8615 I've so far refrained from parallelizing the libjava and libgomp testsuites runs, and some of the libstdc++ test names use very long common prefixes. Still, the new state is far better than the current one, so we might incrementally improve things if libstdc++ or the serial testsuite runs begin to further dominate the make check times. The changes below achive this and still try to keep the patterns manageable. Here are the make -j<2 * ncpu> -k check times for two tests (T5220 as above, X7350 is a Sun Fire X4450 with 4 x 2.93 GHz Xeon X7350): CPU mainline parallel X7350 -j32 63 min 56 min US-T2 -j128 358 min 143 min As you can see, on faster CPUs the change is in the noise (or the parallel test is actually faster), while on the T2 there's a factor of 2.5 speedup. mail-report.log is identical between both runs. There's one partially related change included: even before this change, there could be a strange double summary line in mail-report.log: === libstdc++ Summary === # of expected passes 13934 # of unexpected failures 8 # of unexpected successes 2 # of expected failures 118 # of unsupported tests 732 === libstdc++ Summary === I could trace this to the fact that currently no libstdc++ ABI baselines exist for Solaris 2 (being worked on), thus runtest abi.exp produces [...] === libstdc++ Summary for unix === Running target unix/-m64 Running /vol/gcc/src/hg/trunk/local/libstdc++-v3/testsuite/libstdc++-abi/abi.exp ... === libstdc++ Summary for unix/-m64 === === libstdc++ Summary === This situation is likely to occur more often in the future when runtest is run with a .exp file that doesn't match any tests on a given platform (like i386.exp on SPARC). The fix is trivial: only emit the summary line if any of the test counts is non-zero. Ok for mainline and the 4.4 and 4.5 branches after some soaktime? Thanks. Rainer 2010-03-07 Rainer Orth contrib: * dg-extract-results.sh (total.awk): Only print combined summary for non-zero total. gcc: * Makefile.in (check_gcc_parallelize): Increase parallelism. (check_p_numbers): Extend to 70. gcc/cp: * Make-lang.in (check_g++_parallelize): Increase parallelism. gcc/fortran: * Make-lang.in (check_gfortran_parallelize): Increase parallelism. gcc/testsuite: * gcc.c-torture/compile/limits-fnargs.c (PAR4HALF): Define. (func): Replace PAR4 by PAR4HALF. (ARG4HALF): Define. (caller): Call func with ARG4HALF. libstdc++-v3: * testsuite/Makefile.am (check_DEJAGNU_normal_targets): Extend to 17. Increase parallelism. * testsuite/Makefile.in: Regenerate. diff -r cbdbde9aae00 contrib/dg-extract-results.sh --- a/contrib/dg-extract-results.sh Mon Jul 19 11:13:06 2010 +0200 +++ b/contrib/dg-extract-results.sh Tue Jul 20 20:51:39 2010 +0200 @@ -401,7 +401,7 @@ /^# of unresolved testcases/ { unrescnt += \$5 } /^# of unsupported tests/ { unsupcnt += \$5 } END { - printf ("\n\t\t=== %s Summary ===\n\n", tool) + if (passcnt + failcnt + xpasscnt + xfailcnt + untstcnt + unrescnt + unsupcnt != 0) printf ("\n\t\t=== %s Summary ===\n\n", tool) if (passcnt != 0) printf ("# of expected passes\t\t%d\n", passcnt) if (failcnt != 0) printf ("# of unexpected failures\t%d\n", failcnt) if (xpasscnt != 0) printf ("# of unexpected successes\t%d\n", xpasscnt) diff -r cbdbde9aae00 gcc/Makefile.in --- a/gcc/Makefile.in Mon Jul 19 11:13:06 2010 +0200 +++ b/gcc/Makefile.in Tue Jul 20 20:51:39 2010 +0200 @@ -494,14 +494,75 @@ # very long running, but still should be split of from the check-parallel-$lang # remaining tests runtest invocation), they should be concatenated with commas. # Note that [a-zA-Z] wildcards need to have []s prefixed with \ (needed -# by tcl) and as the *.exp arguments are mached both as is and with +# by tcl) and as the *.exp arguments are matched both as is and with # */ prefixed to it in runtest_file_p, it is usually desirable to include # a subdirectory name. -check_gcc_parallelize=execute.exp=execute/2* \ - execute.exp=execute/\[013-9a-zA-Z\]* \ - compile.exp dg.exp \ - dg-torture.exp,builtins.exp \ - struct-layout-1.exp,unsorted.exp,stackalign.exp,i386.exp +check_gcc_parallelize=compile.exp=compile/\[013-9\]* \ + compile.exp=compile/2000* \ + compile.exp=compile/2001* \ + compile.exp=compile/2002* \ + compile.exp=compile/2003* \ + compile.exp=compile/2004* \ + compile.exp=compile/2005* \ + compile.exp=compile/2006* \ + compile.exp=compile/2007* \ + compile.exp=compile/2008* \ + compile.exp=compile/2009* \ + compile.exp=compile/2010* \ + compile.exp=compile/201\[1-9\]* \ + compile.exp=compile/20\[2-9\]* \ + compile.exp=compile/2\[1-9\]* \ + compile.exp=compile/\[a-oA-O\]* \ + compile.exp=compile/\[pP\]* \ + compile.exp=compile/\[q-zQ-Z\]* \ + execute.exp=execute/\[013-9\]* \ + execute.exp=execute/2000* \ + execute.exp=execute/2001* \ + execute.exp=execute/2002* \ + execute.exp=execute/2003* \ + execute.exp=execute/2004* \ + execute.exp=execute/2005* \ + execute.exp=execute/2006* \ + execute.exp=execute/2007* \ + execute.exp=execute/2008* \ + execute.exp=execute/2009* \ + execute.exp=execute/2010* \ + execute.exp=execute/201\[1-9\]* \ + execute.exp=execute/20\[2-9\]* \ + execute.exp=execute/2\[1-9\]* \ + execute.exp=execute/\[a-eA-E\]* \ + execute.exp=execute/\[f-mF-M\]* \ + execute.exp=execute/\[n-pN-P\]* \ + execute.exp=execute/\[q-zQ-Z\]* \ + unsorted.exp=unsorted/\[0-9a-cA-C\]* \ + unsorted.exp=unsorted/\[dD\]* \ + unsorted.exp=unsorted/\[e-tE-T\]* \ + unsorted.exp=unsorted/\[u-zU-Z\]* \ + dg.exp=gcc.dg/\[0-9a-bA-B\]* \ + dg.exp=gcc.dg/\[cC\]* \ + dg.exp=gcc.dg/\[d-iD-I\]* \ + dg.exp=gcc.dg/\[j-nJ-N\]* \ + dg.exp=gcc.dg/\[o-pO-P\]* \ + dg.exp=gcc.dg/\[q-tQ-T\]* \ + dg.exp=gcc.dg/\[u-zU-Z\]* \ + dg.exp=c-c++-common/* \ + dg-torture.exp=torture/\[0-9a-bA-B\]* \ + dg-torture.exp=torture/\[c-oC-O\]* \ + dg-torture.exp=torture/\[pP\]* \ + dg-torture.exp=torture/\[q-zQ-Z\]* \ + builtins.exp \ + cpp.exp \ + debug.exp \ + format.exp \ + i386.exp \ + ieee.exp \ + noncompile.exp \ + pch.exp \ + stackalign.exp \ + struct-layout-1.exp \ + test-framework.exp \ + tree-ssa.exp \ + vect.exp lang_opt_files=@lang_opt_files@ $(srcdir)/c-family/c.opt $(srcdir)/common.opt lang_specs_files=@lang_specs_files@ lang_tree_files=@lang_tree_files@ @@ -4892,7 +4953,13 @@ check_p_subno=$(word 2,$(subst _, ,$*)) check_p_comma=, check_p_subwork=$(subst $(check_p_comma), ,$(if $(check_p_subno),$(word $(check_p_subno),$(check_p_vars)))) -check_p_numbers=1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 +check_p_numbers=1 2 3 4 5 6 7 8 9 10 \ + 11 12 13 14 15 16 17 18 19 20 \ + 21 22 23 24 25 26 27 28 29 30 \ + 31 32 33 34 35 36 37 38 39 40 \ + 41 42 43 44 45 46 47 48 49 50 \ + 51 52 53 54 55 56 57 58 59 60 \ + 61 62 63 64 65 66 67 68 69 70 check_p_subdir=$(subst _,,$*) check_p_subdirs=$(wordlist 1,$(words $(check_$*_parallelize)),$(check_p_numbers)) diff -r cbdbde9aae00 gcc/cp/Make-lang.in --- a/gcc/cp/Make-lang.in Mon Jul 19 11:13:06 2010 +0200 +++ b/gcc/cp/Make-lang.in Tue Jul 20 20:51:39 2010 +0200 @@ -143,7 +143,29 @@ lang_checks += check-g++ lang_checks_parallelized += check-g++ # For description see comment above check_gcc_parallelize in gcc/Makefile.in. -check_g++_parallelize = old-deja.exp dg.exp +check_g++_parallelize = old-deja.exp=g++.old-deja/g++.\[a-b\]* \ + old-deja.exp=g++.old-deja/g++.\[c-k\]* \ + old-deja.exp=g++.old-deja/g++.\[l-m\]* \ + old-deja.exp=g++.old-deja/g++.\[n-o\]* \ + old-deja.exp=g++.old-deja/g++.\[p-r\]* \ + old-deja.exp=g++.old-deja/g++.\[s-z\]* \ + debug.exp \ + dg.exp=c-c++-common/* \ + dg.exp=g++.dg/\[0-9a-bA-B\]* \ + dg.exp=g++.dg/\[cC\]\[a-pA-P\]* \ + dg.exp=g++.dg/\[cC\]\[q-zQ-Z0-9_\]* \ + dg.exp=g++.dg/\[dD\]* \ + dg.exp=g++.dg/\[eE\]* \ + dg.exp=g++.dg/\[f-kF-K\]* \ + dg.exp=g++.dg/\[l-nL-N\]* \ + dg.exp=g++.dg/\[oO\]* \ + dg.exp=g++.dg/\[pP\]* \ + dg.exp=g++.dg/\[q-sQ-S\]* \ + dg.exp=g++.dg/\[tT\]\[a-ea-E\]* \ + dg.exp=g++.dg/\[tT\]\[f-oF-O\]* \ + dg.exp=g++.dg/\[tT\]\[p-zP-Z0-9_\]* \ + dg.exp=g++.dg/\[u-zU-Z\]* \ + dg-torture.exp # # Install hooks: diff -r cbdbde9aae00 gcc/fortran/Make-lang.in --- a/gcc/fortran/Make-lang.in Mon Jul 19 11:13:06 2010 +0200 +++ b/gcc/fortran/Make-lang.in Tue Jul 20 20:51:39 2010 +0200 @@ -170,9 +170,40 @@ lang_checks += check-gfortran lang_checks_parallelized += check-gfortran # For description see comment above check_gcc_parallelize in gcc/Makefile.in. -check_gfortran_parallelize = dg.exp=gfortran.dg/\[a-cA-C\]* \ - dg.exp=gfortran.dg/\[d-mD-M\]* \ - dg.exp=gfortran.dg/\[n-zN-Z0-9\]* +check_gfortran_parallelize = dg.exp=gfortran.dg/\[aA\]\[a-lA-L\]* \ + dg.exp=gfortran.dg/\[aA\]\[m-rM-R\]* \ + dg.exp=gfortran.dg/\[aA\]\[s-zS-Z0-9_\]* \ + dg.exp=gfortran.dg/\[bB\]\[a-nA-N\]* \ + dg.exp=gfortran.dg/\[bB\]\[o-zO-Z0-9_\]* \ + dg.exp=gfortran.dg/\[cC\]\[a-mA-M\]* \ + dg.exp=gfortran.dg/\[cC\]\[n-zN-Z0-9_\]* \ + dg.exp=gfortran.dg/\[dD\]\[a-mA-M\]* \ + dg.exp=gfortran.dg/\[dD\]\[n-zN-Z0-9_\]* \ + dg.exp=gfortran.dg/\[eE\]* \ + dg.exp=gfortran.dg/\[fF\]* \ + dg.exp=gfortran.dg/\[gG\]\[a-zA-Z\]* \ + dg.exp=gfortran.dg/\[gG\]\[0-9_\]* \ + dg.exp=gfortran.dg/\[hH\]* \ + dg.exp=gfortran.dg/\[iI\]\[a-mA-M\]* \ + dg.exp=gfortran.dg/\[iI\]\[nN\]* \ + dg.exp=gfortran.dg/\[iI\]\[o-zO-Z0-9_\]* \ + dg.exp=gfortran.dg/\[j-lJ-L\]* \ + dg.exp=gfortran.dg/\[mM\]* \ + dg.exp=gfortran.dg/\[nN\]* \ + dg.exp=gfortran.dg/\[oO\]* \ + dg.exp=gfortran.dg/\[pP\]\[a-qA-Q\]* \ + dg.exp=gfortran.dg/\[pP\]\[r-zR-Z0-9_\]* \ + dg.exp=gfortran.dg/\[q-rQ-R\]* \ + dg.exp=gfortran.dg/\[sS\]* \ + dg.exp=gfortran.dg/\[tT\]* \ + dg.exp=gfortran.dg/\[u-vU-V\]* \ + dg.exp=gfortran.dg/\[wW\]* \ + dg.exp=gfortran.dg/\[x-zX-Z0-9_\]* \ + compile.exp \ + execute.exp=execute/\[a-eA-E\]* \ + execute.exp=execute/\[f-iF-I\]* \ + execute.exp=execute/\[j-tJ-T\]* \ + execute.exp=execute/\[u-zU-Z0-9_\]* # GFORTRAN documentation. GFORTRAN_TEXI = \ diff -r cbdbde9aae00 gcc/testsuite/gcc.c-torture/compile/limits-fnargs.c --- a/gcc/testsuite/gcc.c-torture/compile/limits-fnargs.c Mon Jul 19 11:13:06 2010 +0200 +++ b/gcc/testsuite/gcc.c-torture/compile/limits-fnargs.c Tue Jul 20 20:51:39 2010 +0200 @@ -4,19 +4,21 @@ #define PAR2 PAR1, PAR1, PAR1, PAR1, PAR1, PAR1, PAR1, PAR1, PAR1, PAR1 #define PAR3 PAR2, PAR2, PAR2, PAR2, PAR2, PAR2, PAR2, PAR2, PAR2, PAR2 #define PAR4 PAR3, PAR3, PAR3, PAR3, PAR3, PAR3, PAR3, PAR3, PAR3, PAR3 +#define PAR4HALF PAR3, PAR3, PAR3, PAR3, PAR3 #define PAR5 PAR4, PAR4, PAR4, PAR4, PAR4, PAR4, PAR4, PAR4, PAR4, PAR4 #define PAR6 PAR5, PAR5, PAR5, PAR5, PAR5, PAR5, PAR5, PAR5, PAR5, PAR5 -extern void func (PAR4); +extern void func (PAR4HALF); #define ARG1 0,1,2,3,4,5,6,7,8,9 #define ARG2 ARG1, ARG1, ARG1, ARG1, ARG1, ARG1, ARG1, ARG1, ARG1, ARG1 #define ARG3 ARG2, ARG2, ARG2, ARG2, ARG2, ARG2, ARG2, ARG2, ARG2, ARG2 #define ARG4 ARG3, ARG3, ARG3, ARG3, ARG3, ARG3, ARG3, ARG3, ARG3, ARG3 +#define ARG4HALF ARG3, ARG3, ARG3, ARG3, ARG3 #define ARG5 ARG4, ARG4, ARG4, ARG4, ARG4, ARG4, ARG4, ARG4, ARG4, ARG4 #define ARG5HALF ARG5, ARG5, ARG5, ARG5, ARG5 void caller(void) { - func (ARG4); + func (ARG4HALF); } diff -r cbdbde9aae00 libstdc++-v3/testsuite/Makefile.am --- a/libstdc++-v3/testsuite/Makefile.am Mon Jul 19 11:13:06 2010 +0200 +++ b/libstdc++-v3/testsuite/Makefile.am Tue Jul 20 20:51:39 2010 +0200 @@ -100,7 +100,7 @@ @test ! -f $*/site.exp || mv $*/site.exp $*/site.bak @mv $*/site.exp.tmp $*/site.exp -check_DEJAGNU_normal_targets = $(patsubst %,check-DEJAGNUnormal%,0 1 2 3) +check_DEJAGNU_normal_targets = $(patsubst %,check-DEJAGNUnormal%,0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17) $(check_DEJAGNU_normal_targets): check-DEJAGNUnormal%: normal%/site.exp # Run the testsuite in normal mode. @@ -110,8 +110,15 @@ if [ -z "$*$(filter-out --target_board=%, $(RUNTESTFLAGS))" ] \ && [ "$(filter -j, $(MFLAGS))" = "-j" ]; then \ $(MAKE) $(AM_MAKEFLAGS) check-DEJAGNUnormal0 check-DEJAGNUnormal1 \ - check-DEJAGNUnormal2 check-DEJAGNUnormal3; \ - for idx in 0 1 2 3; do \ + check-DEJAGNUnormal2 check-DEJAGNUnormal3 \ + check-DEJAGNUnormal4 check-DEJAGNUnormal5 \ + check-DEJAGNUnormal6 check-DEJAGNUnormal7 \ + check-DEJAGNUnormal8 check-DEJAGNUnormal9 \ + check-DEJAGNUnormal10 check-DEJAGNUnormal11 \ + check-DEJAGNUnormal12 check-DEJAGNUnormal13 \ + check-DEJAGNUnormal14 check-DEJAGNUnormal15 \ + check-DEJAGNUnormal16 check-DEJAGNUnormal17; \ + for idx in 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17; do \ mv -f normal$$idx/libstdc++.sum normal$$idx/libstdc++.sum.sep; \ mv -f normal$$idx/libstdc++.log normal$$idx/libstdc++.log.sep; \ done; \ @@ -120,11 +127,26 @@ $(SHELL) $(srcdir)/../../contrib/dg-extract-results.sh \ libstdc++.sum.sep normal0/libstdc++.sum.sep \ normal1/libstdc++.sum.sep normal2/libstdc++.sum.sep \ - normal3/libstdc++.sum.sep > libstdc++.sum; \ + normal3/libstdc++.sum.sep normal4/libstdc++.sum.sep \ + normal5/libstdc++.sum.sep normal6/libstdc++.sum.sep \ + normal7/libstdc++.sum.sep normal8/libstdc++.sum.sep \ + normal9/libstdc++.sum.sep normal10/libstdc++.sum.sep \ + normal11/libstdc++.sum.sep normal12/libstdc++.sum.sep \ + normal13/libstdc++.sum.sep normal14/libstdc++.sum.sep \ + normal15/libstdc++.sum.sep normal16/libstdc++.sum.sep \ + normal17/libstdc++.sum.sep \ + normal14/libstdc++.sum.sep > libstdc++.sum; \ $(SHELL) $(srcdir)/../../contrib/dg-extract-results.sh -L \ libstdc++.log.sep normal0/libstdc++.log.sep \ normal1/libstdc++.log.sep normal2/libstdc++.log.sep \ - normal3/libstdc++.log.sep > libstdc++.log; \ + normal3/libstdc++.log.sep normal4/libstdc++.log.sep \ + normal5/libstdc++.log.sep normal6/libstdc++.log.sep \ + normal7/libstdc++.log.sep normal8/libstdc++.log.sep \ + normal9/libstdc++.log.sep normal10/libstdc++.log.sep \ + normal11/libstdc++.log.sep normal12/libstdc++.log.sep \ + normal13/libstdc++.log.sep normal14/libstdc++.log.sep \ + normal15/libstdc++.log.sep normal16/libstdc++.log.sep \ + normal17/libstdc++.log.sep > libstdc++.log; \ exit 0; \ fi; \ srcdir=`$(am__cd) $(srcdir) && pwd`; export srcdir; \ @@ -140,13 +162,41 @@ $(RUNTESTFLAGS) abi.exp; \ else echo "WARNING: could not find \`runtest'" 1>&2; :;\ fi; \ - dirs="`cd $$srcdir; echo [013-9][0-9]_*/* [ab]* de* [ep]*/*`";; \ + dirs="`cd $$srcdir; echo [01][0-9]_*/*`";; \ normal1) \ - dirs="`cd $$srcdir; echo 2[0-2]_*/*`";; \ + dirs="`cd $$srcdir; echo [3-9][0-9]_*/*`";; \ normal2) \ - dirs="`cd $$srcdir; echo 2[4-9]_*/*`";; \ + dirs="`cd $$srcdir; echo 20_*/*`";; \ normal3) \ - dirs="`cd $$srcdir; echo 23_*/* t*/*`";; \ + dirs="`cd $$srcdir; echo 21_*/*`";; \ + normal4) \ + dirs="`cd $$srcdir; echo 22_*/*`";; \ + normal5) \ + dirs="`cd $$srcdir; echo 23_*/[0-9a-hA-H]*`";; \ + normal6) \ + dirs="`cd $$srcdir; echo 23_*/[i-mI-M]*`";; \ + normal7) \ + dirs="`cd $$srcdir; echo 23_*/[n-uN-U]*`";; \ + normal8) \ + dirs="`cd $$srcdir; echo 23_*/[v-zV-Z]*`";; \ + normal9) \ + dirs="`cd $$srcdir; echo 24_*/*`";; \ + normal10) \ + dirs="`cd $$srcdir; echo 25_*/*`";; \ + normal11) \ + dirs="`cd $$srcdir; echo 26_*/*`";; \ + normal12) \ + dirs="`cd $$srcdir; echo 27_*/*`";; \ + normal13) \ + dirs="`cd $$srcdir; echo 2[89]_*/*`";; \ + normal14) \ + dirs="`cd $$srcdir; echo [ab]* de*`";; \ + normal15) \ + dirs="`cd $$srcdir; echo e*/*`";; \ + normal16) \ + dirs="`cd $$srcdir; echo p*/*`";; \ + normal17) \ + dirs="`cd $$srcdir; echo t*/*`";; \ esac; \ if [ -n "$*" ]; then cd "$*"; fi; \ if $(SHELL) -c "$$runtest --version" > /dev/null 2>&1; then \