diff mbox

gcc parallel make check

Message ID 20140915160549.GM17454@tucnak.redhat.com
State New
Headers show

Commit Message

Jakub Jelinek Sept. 15, 2014, 4:05 p.m. UTC
On Fri, Sep 12, 2014 at 04:42:25PM -0700, Mike Stump wrote:
> On Sep 12, 2014, at 9:32 AM, Jakub Jelinek <jakub@redhat.com> wrote:
> > Here is my latest version of the patch.
> 
> I did a timing test:

Here is an updated version.
Changes since last version:
1) acats parallelized the same way (just, because it is in shell,
   using mkdir instead of open with O_EXCL|O_CREAT);
   also, as acats has pretty significant initial setup time
   (up to a minute or so on not really fast box), I'm now
   creating the /support stuff before spawning the parallel
   jobs and let the parallel jobs use the same shared support
   directory
2) I'm now using addprefix instead of patsubst where appropriate
3) I'm using $(if ...) instead of $(or ...) to make it usable
   with make 3.80 (3.81 already supports or)
4) parallelization is performed for any kinds of RUNTESTFLAGS arguments now
5) struct-layout-1.exp apparently doesn't have to be performed serially,
   and for gnu-encoding.exp I've used similar change as for go-test.exp
6) in libstdc++, abi.exp, pretty-printers.exp and xmethods.exp are performed
   together with conformance.exp, so again parallelization for any
   RUNTESTFLAGS flags; abi.exp and xmethods.exp are serially tested
   by the first runtest instance to encounter them

Regtested on x86_64-linux, without the patch toplevel make -k check
took 8hrs3minutes (don't have time data for that run), without the
patch toplevel make -j48 -k check took:
real    40m21.984s
user    341m51.675s
sys     112m46.993s
and with the patch make -j48 -k check took:
real    32m22.066s
user    355m1.788s
sys     117m5.809s
I saw over 45 jobs running pretty much as the point where all the
testing was done, and test_summary run from the non-parallel testing
is the same as test_summary from the -j48 testing with the patch.
Is this version ok for trunk?

2014-09-14  Jakub Jelinek  <jakub@redhat.com>

gcc/
	* Makefile.in (dg_target_exps): Remove.
	(check_gcc_parallelize): Change to just an upper bound number.
	(check-%-subtargets): Always print the non-parallelized goals.
	(check_p_vars, check_p_comma, check_p_subwork): Remove.
	(check_p_count, check_p_numbers0, check_p_numbers1, check_p_numbers2,
	check_p_numbers3, check_p_numbers4, check_p_numbers5,
	check_p_numbers6): New variables.
	(check_p_numbers): Set to sequence from 1 to 9999.
	(check_p_subdirs): Set to sequence from 1 to minimum of
	$(check_p_count) and either GCC_TEST_PARALLEL_SLOTS env var if set,
	or 128.
	(check-%, check-parallel-%): Rewritten so that for parallelized
	testing each job runs all the *.exp files, with
	GCC_RUNTEST_PARALLELIZE_DIR set in environment.
gcc/go/
	* Make-lang.in (check_go_parallelize): Change to just an upper bound
	number.
gcc/fortran/
	* Make-lang.in (check_gfortran_parallelize): Change to just an upper
	bound number.
gcc/cp/
	* Make-lang.in (check_g++_parallelize): Change to just an upper bound
	number.
gcc/objc/
	* Make-lang.in (check_objc_parallelize): Change to just an upper
	bound number.
gcc/ada/
	* gcc-interface/Make-lang.in (check_acats_numbers0,
	check_acats_numbers1, check_acats_numbers2, check_acats_numbers3,
	check_acats_numbers4, check_acats_numbers5, check_acats_numbers6,
	check_acats_numbers, check_acats_subdirs): New variables.
	(check_acats_targets): Use $(check_acats_subdirs).
	(check-acats, check-acats%): Rewritten so that for parallelized
	testing each job runs all the chapters files, with
	GCC_RUNTEST_PARALLELIZE_DIR set in environment.  Prepare the support
	directory sequentially and share it.
	(check-acats-subtargets): Always print just check-acats.
gcc/testsuite/
	* lib/gcc-defs.exp (gcc_parallel_test_run_p,
	gcc_parallel_test_enable): New procedures.  If
	GCC_RUNTEST_PARALLELIZE_DIR is set in environment, override
	runtest_file_p to invoke also gcc_parallel_test_run_p.
	* g++.dg/guality/guality.exp (check_guality): Save/restore
	test_counts array around the body of the procedure.
	* gcc.dg/guality/guality.exp (check_guality): Likewise.
	* g++.dg/plugin/plugin.exp: Run all the tests serially
	by the first parallel runtest encountering it.
	* gcc.dg/plugin/plugin.exp: Likewise.
	* gcc.misc-tests/matrix1.exp: Likewise.
	* gcc.misc-tests/dhry.exp: Likewise.
	* gcc.misc-tests/acker1.exp: Likewise.
	* gcc.misc-tests/linkage.exp: Likewise.
	* gcc.misc-tests/mg.exp: Likewise.
	* gcc.misc-tests/mg-2.exp: Likewise.
	* gcc.misc-tests/sort2.exp: Likewise.
	* gcc.misc-tests/sieve.exp: Likewise.
	* gcc.misc-tests/options.exp: Likewise.
	* gcc.misc-tests/help.exp: Likewise.
	* go.test/go-test.exp (go-gc-tests): Use
	gcc_parallel_test_enable {0, 1} around all handling of
	each test.
	* objc.dg/gnu-encoding/gnu-encoding.exp: Likewise.
	* ada/acats/run_all.sh: Parallelize using mkdir inside of
	a shared directory.  Avoid forking and expr uses if
	shell supports $(()).
libstdc++-v3/
	* testsuite/Makefile.am (check_p_numbers0, check_p_numbers1,
	check_p_numbers2, check_p_numbers3, check_p_numbers4,
	check_p_numbers5, check_p_numbers6, check_p_numbers,
	check_p_subdirs): New variables.
	(check_DEJAGNU_normal_targets): Use check_p_subdirs.
	(check-DEJAGNU): Rewritten so that for parallelized
	testing each job runs all the *.exp files, with
	GCC_RUNTEST_PARALLELIZE_DIR set in environment.
	* testsuite/Makefile.in: Regenerated.
	* testsuite/lib/libstdc++.exp (gcc_parallel_test_run_p,
	gcc_parallel_test_enable): New procedures.  If
	GCC_RUNTEST_PARALLELIZE_DIR is set in environment, override
	runtest_file_p to invoke also gcc_parallel_test_run_p.
	* testsuite/libstdc++-abi/abi.exp: Run all the tests serially
	by the first parallel runtest encountering it.  Fix up path
	of the extract_symvers script.
	* testsuite/libstdc++-xmethods/xmethods.exp: Run all the tests
	serially by the first parallel runtest encountering it.  Run
	dg-finish even in case of error.



	Jakub

Comments

Mike Stump Sept. 15, 2014, 5:44 p.m. UTC | #1
On Sep 15, 2014, at 9:05 AM, Jakub Jelinek <jakub@redhat.com> wrote:

All the updates sound good.

> Regtested on x86_64-linux, without the patch toplevel make -k check
> took 8hrs3minutes (don't have time data for that run),

This confuses me, but, no matter.  Isn’t 8hrs time data?  :-)

> patch toplevel make -j48 -k check took:
> real    40m21.984s
> user    341m51.675s
> sys     112m46.993s
> and with the patch make -j48 -k check took:
> real    32m22.066s
> user    355m1.788s
> sys     117m5.809s

These numbers are useful to try and ensure the overhead (scaling factor) is reasonable, thanks.

> Is this version ok for trunk?

Ok.

Thanks for all your work.
Richard Biener Sept. 16, 2014, 9:20 a.m. UTC | #2
On Mon, Sep 15, 2014 at 7:44 PM, Mike Stump <mikestump@comcast.net> wrote:
> On Sep 15, 2014, at 9:05 AM, Jakub Jelinek <jakub@redhat.com> wrote:
>
> All the updates sound good.
>
>> Regtested on x86_64-linux, without the patch toplevel make -k check
>> took 8hrs3minutes (don't have time data for that run),
>
> This confuses me, but, no matter.  Isn’t 8hrs time data?  :-)
>
>> patch toplevel make -j48 -k check took:
>> real    40m21.984s
>> user    341m51.675s
>> sys     112m46.993s
>> and with the patch make -j48 -k check took:
>> real    32m22.066s
>> user    355m1.788s
>> sys     117m5.809s
>
> These numbers are useful to try and ensure the overhead (scaling factor) is reasonable, thanks.

A nice improvement indeed.  The patched result is 15 times faster
than the serial unpatched run.  So there is room for improvement
(I wouldn't say the scaling factor is reasonable - with accounting for
overhead I'd expect it should be possible to arrive at a factor of 32
here at least).

So - where's the "serial" parts of the testing run?

Thanks for the improvements btw!

Richard.

>> Is this version ok for trunk?
>
> Ok.
>
> Thanks for all your work.
Jakub Jelinek Sept. 16, 2014, 9:28 a.m. UTC | #3
On Tue, Sep 16, 2014 at 11:20:37AM +0200, Richard Biener wrote:
> > This confuses me, but, no matter.  Isn’t 8hrs time data?  :-)

It is, but not time(1) data, just wall clock computed from subtracting
mtimes of my make check output log and make -j48 bootstrap log.

> >> patch toplevel make -j48 -k check took:
> >> real    40m21.984s
> >> user    341m51.675s
> >> sys     112m46.993s
> >> and with the patch make -j48 -k check took:
> >> real    32m22.066s
> >> user    355m1.788s
> >> sys     117m5.809s
> >
> > These numbers are useful to try and ensure the overhead (scaling factor) is reasonable, thanks.
> 
> A nice improvement indeed.  The patched result is 15 times faster
> than the serial unpatched run.  So there is room for improvement

Note, the box used was oldish AMD 16-core, no ht, box, haven't tried it on anything
more parallel, also it was normal hard disk, etc.  No idea whether anything
from this is relevant to that though.
Some CPU time goes into the expect processes, I can retry the build tonight
and grab also time(1) info from make -k check to see the user/sys times for
serial testing.

	Jakub
Richard Biener Sept. 16, 2014, 9:48 a.m. UTC | #4
On Tue, Sep 16, 2014 at 11:28 AM, Jakub Jelinek <jakub@redhat.com> wrote:
> On Tue, Sep 16, 2014 at 11:20:37AM +0200, Richard Biener wrote:
>> > This confuses me, but, no matter.  Isn’t 8hrs time data?  :-)
>
> It is, but not time(1) data, just wall clock computed from subtracting
> mtimes of my make check output log and make -j48 bootstrap log.
>
>> >> patch toplevel make -j48 -k check took:
>> >> real    40m21.984s
>> >> user    341m51.675s
>> >> sys     112m46.993s
>> >> and with the patch make -j48 -k check took:
>> >> real    32m22.066s
>> >> user    355m1.788s
>> >> sys     117m5.809s
>> >
>> > These numbers are useful to try and ensure the overhead (scaling factor) is reasonable, thanks.
>>
>> A nice improvement indeed.  The patched result is 15 times faster
>> than the serial unpatched run.  So there is room for improvement
>
> Note, the box used was oldish AMD 16-core, no ht, box, haven't tried it on anything

Ah, I assumed -j48 testing means you have 48 cores.  I usually test
with -j12 on my 6-core HT-enabled box.  A factor 15 scaling for 16
CPUs is of course close to the best we can achieve.

Richard.

> more parallel, also it was normal hard disk, etc.  No idea whether anything
> from this is relevant to that though.
> Some CPU time goes into the expect processes, I can retry the build tonight
> and grab also time(1) info from make -k check to see the user/sys times for
> serial testing.
>
>         Jakub
VandeVondele Joost Sept. 16, 2014, 4:04 p.m. UTC | #5
>> > These numbers are useful to try and ensure the overhead (scaling factor) is reasonable, thanks.
>>
>> A nice improvement indeed.  The patched result is 15 times faster
>> than the serial unpatched run.  So there is room for improvement
>
> Note, the box used was oldish AMD 16-core, no ht, box, haven't tried it on anything

on a 32 core box, no ht, I see these timings:

time make -j32 -k check >& log.check32 ; time make -j8 -k check >& log.check8

real	18m14.562s
user	260m21.578s
sys	264m26.042s

real	41m33.210s
user	233m4.563s
sys	72m11.429s

so it is not quite reaching the ideal 4x speedup. Counting the number of 'expect' processes they are nicely at around 32 and 8 for the full test, with only a very short tail near the end. So, there might be some overhead somewhere. Total user time is similar, but time in sys goes up.
Christophe Lyon Oct. 10, 2014, 2:09 p.m. UTC | #6
Hi Jakub,


On 15 September 2014 18:05, Jakub Jelinek <jakub@redhat.com> wrote:
[...]
>  # For parallelized check-% targets, this decides whether parallelization
>  # is desirable (if -jN is used and RUNTESTFLAGS doesn't contain anything
>  # but optional --target_board or --extra_opts arguments).  If desirable,
>  # recursive make is run with check-parallel-$lang{,1,2,3,4,5} etc. goals,
>  # which can be executed in parallel, as they are run in separate directories.
> -# check-parallel-$lang{1,2,3,4,5} etc. goals invoke runtest with the longest
> -# running *.exp files from the testsuite, as determined by check_$lang_parallelize
> -# variable.  The check-parallel-$lang goal in that case invokes runtest with
> -# all the remaining *.exp files not handled by the separate goals.
> +# check-parallel-$lang{,1,2,3,4,5} etc. goals invoke runtest with
> +# GCC_RUNTEST_PARALLELIZE_DIR var in the environment and runtest_file_p
> +# dejaGNU procedure is overridden to additionally synchronize through
> +# a $lang-parallel directory which tests will be run by which runtest instance.
>  # Afterwards contrib/dg-extract-results.sh is used to merge the sum and log
>  # files.  If parallelization isn't desirable, only one recursive make
>  # is run with check-parallel-$lang goal and check_$lang_parallelize variable
> @@ -3662,76 +3645,60 @@ check_p_subdirs=$(wordlist 1,$(words $(c
>  # to lang_checks_parallelized variable and define check_$lang_parallelize
>  # variable (see above check_gcc_parallelize description).
>  $(lang_checks_parallelized): check-% : site.exp
> -       @if [ -z "$(filter-out --target_board=%,$(filter-out --extra_opts%,$(RUNTESTFLAGS)))" ] \

Since you removed this test, the comment above is not longer accurate:
setting RUNTESTFLAGS to whatever value no longer disables
parallelization.

Which leads me to discuss a bug I faced after you committed this
change: I am testing a patch which bring a series of new tests.
$ RUNTESTFLAGS=my.exp make -jN check (in fact the 'make -j' is
embedded in a larger build script)

my.exp contains the following construct which is often used in the testsuite:
==========
foreach src [lsort [glob -nocomplain $srcdir/$subdir/*.c]] {
    # If we're only testing specific files and this isn't one of them,
skip it.
    if ![runtest_file_p $runtests $src] then {
        continue
    }
    c-torture-execute $src $additional_flags
    gcc-dg-runtest $src "" $additional_flags
}
==========
Note that gcc-dg-runtest calls runtest_file_p too.

What I observed is that if I use -j1, all my .c files get tested,
while with N>2 some of them are silently skipped.

It took me a while to figure out that it's because gcc-dg-runtest
calls runtest_file_p, which means that runtest_file_p is called twice
when the 1st invocation returns 1, and only once when the 1st
invocation returns 0.

For example, if we have pid0, pid1 the concurrent runtest processes,
and file0.c, file1.c, .... the testcases, then:
* pid0 decides to keep file0.c file1.c file2.c file3.c file4.c. Since
the above loop calls runtest_file_p twice for each, we reach the
"minor" counter of 10.
* in the mean time, pid1 decides to skip file0.c, file1.c ... file9.c
since it calls runtest_file_p only once for each
* pid1 increments its parallel counter to 1, and create the new testing subdir
* pid1 decides to keep file10, file11, file12, file13 and file14
(again, 2 calls to runtest_file_p per testcase)
* pid0 increments its parallel counter to 1, and decides it has to skip it
* pid0 thus decides to skip file5, file6, file7, ... file14, calling
runtest_file_p once for each
* etc...

In the end, we have ignored file5...file9

I'm not sure why you have made special cases for some of the existing
*.exp when you forced them to disable parallelization.
Was it to handle such cases?

I'm not sure about the next step:
- should I modify my .exp file?
- should you modify gcc_parallel_test_run_p?

Even if I have to modify my .exp file, I think this is error prone,
and others could introduce a similar construct in the future.

Thanks,

Christophe.
Jakub Jelinek Oct. 10, 2014, 2:19 p.m. UTC | #7
On Fri, Oct 10, 2014 at 04:09:39PM +0200, Christophe Lyon wrote:
> my.exp contains the following construct which is often used in the testsuite:
> ==========
> foreach src [lsort [glob -nocomplain $srcdir/$subdir/*.c]] {
>     # If we're only testing specific files and this isn't one of them,
> skip it.
>     if ![runtest_file_p $runtests $src] then {
>         continue
>     }
>     c-torture-execute $src $additional_flags
>     gcc-dg-runtest $src "" $additional_flags
> }
> ==========
> Note that gcc-dg-runtest calls runtest_file_p too.

Such my.exp is invalid, you need to guarantee gcc_parallel_test_run_p
is run the same number of times in all instances unless
gcc_parallel_test_enable has been disabled.

See the patches I've posted when adding the fine-grained parallelization,
e.g. go testsuite has been fixed that way, etc.
So, in your above example, you'd need:
    gcc_parallel_test_enable 0
line before c-torture-execute and
    gcc_parallel_test_enable 1
line after gcc-dg-runtest.  That way, if runtest_file_p says the test should
be scheduled by current instance, all the subtests will be run there.

If my.exp is part of gcc/testsuite, I'm sorry for missing it, if it is
elsewhere, just fix it up.

Note, there are #verbose lines in gcc_parallel_test_run_p, you can uncomment
them and through sed on the log files verify that each instance performs the
same parallelization checks (same strings).

	Jakub
Christophe Lyon Oct. 10, 2014, 2:50 p.m. UTC | #8
On 10 October 2014 16:19, Jakub Jelinek <jakub@redhat.com> wrote:
> On Fri, Oct 10, 2014 at 04:09:39PM +0200, Christophe Lyon wrote:
>> my.exp contains the following construct which is often used in the testsuite:
>> ==========
>> foreach src [lsort [glob -nocomplain $srcdir/$subdir/*.c]] {
>>     # If we're only testing specific files and this isn't one of them,
>> skip it.
>>     if ![runtest_file_p $runtests $src] then {
>>         continue
>>     }
>>     c-torture-execute $src $additional_flags
>>     gcc-dg-runtest $src "" $additional_flags
>> }
>> ==========
>> Note that gcc-dg-runtest calls runtest_file_p too.
>
> Such my.exp is invalid, you need to guarantee gcc_parallel_test_run_p
> is run the same number of times in all instances unless
> gcc_parallel_test_enable has been disabled.

Thanks for your prompt answer.

Is this documented somewhere, so that such cases do not happen in the future?


> See the patches I've posted when adding the fine-grained parallelization,
> e.g. go testsuite has been fixed that way, etc.
> So, in your above example, you'd need:
>     gcc_parallel_test_enable 0
> line before c-torture-execute and
>     gcc_parallel_test_enable 1
> line after gcc-dg-runtest.  That way, if runtest_file_p says the test should
> be scheduled by current instance, all the subtests will be run there.
>
> If my.exp is part of gcc/testsuite, I'm sorry for missing it, if it is
> elsewhere, just fix it up.

It's in a patch which has been under review for quite some time
(started before your change), that's why you missed it.

> Note, there are #verbose lines in gcc_parallel_test_run_p, you can uncomment
> them and through sed on the log files verify that each instance performs the
> same parallelization checks (same strings).
Yep, I saw those and also added other traces of my own :-)


What about my remark about:
>  # For parallelized check-% targets, this decides whether parallelization
>  # is desirable (if -jN is used and RUNTESTFLAGS doesn't contain anything
>  # but optional --target_board or --extra_opts arguments).  If desirable,
I think it should be removed from gcc/Makefile.in

Thanks,

Christophe.
Jakub Jelinek Oct. 10, 2014, 2:54 p.m. UTC | #9
On Fri, Oct 10, 2014 at 04:50:47PM +0200, Christophe Lyon wrote:
> On 10 October 2014 16:19, Jakub Jelinek <jakub@redhat.com> wrote:
> > On Fri, Oct 10, 2014 at 04:09:39PM +0200, Christophe Lyon wrote:
> >> my.exp contains the following construct which is often used in the testsuite:
> >> ==========
> >> foreach src [lsort [glob -nocomplain $srcdir/$subdir/*.c]] {
> >>     # If we're only testing specific files and this isn't one of them,
> >> skip it.
> >>     if ![runtest_file_p $runtests $src] then {
> >>         continue
> >>     }
> >>     c-torture-execute $src $additional_flags
> >>     gcc-dg-runtest $src "" $additional_flags
> >> }
> >> ==========
> >> Note that gcc-dg-runtest calls runtest_file_p too.
> >
> > Such my.exp is invalid, you need to guarantee gcc_parallel_test_run_p
> > is run the same number of times in all instances unless
> > gcc_parallel_test_enable has been disabled.
> 
> Thanks for your prompt answer.
> 
> Is this documented somewhere, so that such cases do not happen in the future?

Feel free to submit a documentation patch.

> It's in a patch which has been under review for quite some time
> (started before your change), that's why you missed it.

Ah, ok.

> What about my remark about:
> >  # For parallelized check-% targets, this decides whether parallelization
> >  # is desirable (if -jN is used and RUNTESTFLAGS doesn't contain anything
> >  # but optional --target_board or --extra_opts arguments).  If desirable,
> I think it should be removed from gcc/Makefile.in

Only the " and RUNTESTFLAGS ... arguments" part of that.  Patch preapproved.

	Jakub
diff mbox

Patch

--- gcc/Makefile.in.jj	2014-09-14 21:09:04.612498827 +0200
+++ gcc/Makefile.in	2014-09-14 21:30:27.509141045 +0200
@@ -513,34 +513,10 @@  xm_include_list=@xm_include_list@
 xm_defines=@xm_defines@
 lang_checks=
 lang_checks_parallelized=
-dg_target_exps:=aarch64.exp,alpha.exp,arm.exp,avr.exp,bfin.exp,cris.exp
-dg_target_exps:=$(dg_target_exps),epiphany.exp,frv.exp,i386.exp,ia64.exp
-dg_target_exps:=$(dg_target_exps),m68k.exp,microblaze.exp,mips.exp,powerpc.exp
-dg_target_exps:=$(dg_target_exps),rx.exp,s390.exp,sh.exp,sparc.exp,spu.exp
-dg_target_exps:=$(dg_target_exps),tic6x.exp,xstormy16.exp
-# This lists a couple of test files that take most time during check-gcc.
-# When doing parallelized check-gcc, these can run in parallel with the
-# remaining tests.  Each word in this variable stands for work for one
-# make goal and one extra make goal is added to handle all the *.exp
-# files not handled explicitly already.  If multiple *.exp files
-# should be run in the same runtest invocation (usually if they aren't
-# very long running, but still should be split of from the check-parallel-$lang
-# remaining tests runtest invocation), they should be concatenated with commas.
-# Note that [a-zA-Z] wildcards need to have []s prefixed with \ (needed
-# by tcl) and as the *.exp arguments are mached both as is and with
-# */ prefixed to it in runtest_file_p, it is usually desirable to include
-# a subdirectory name.
-check_gcc_parallelize=execute.exp=execute/2* \
-		      execute.exp=execute/\[013-9a-fA-F\]* \
-		      execute.exp=execute/\[pP\]*,dg.exp \
-		      execute.exp=execute/\[g-oq-zG-OQ-Z\]*,compile.exp=compile/2* \
-		      compile.exp=compile/\[9pP\]*,builtins.exp \
-		      compile.exp=compile/\[013-8a-oq-zA-OQ-Z\]* \
-		      dg-torture.exp,ieee.exp \
-		      vect.exp,unsorted.exp \
-		      guality.exp \
-		      struct-layout-1.exp,stackalign.exp \
-		      $(dg_target_exps)
+# Upper limit to which it is useful to parallelize this lang target.
+# It doesn't make sense to try e.g. 128 goals for small testsuites
+# like objc or go.
+check_gcc_parallelize=10000
 lang_opt_files=@lang_opt_files@ $(srcdir)/c-family/c.opt $(srcdir)/common.opt
 lang_specs_files=@lang_specs_files@
 lang_tree_files=@lang_tree_files@
@@ -3631,27 +3607,34 @@  $(filter-out $(lang_checks_parallelized)
 	    export TCL_LIBRARY ; fi ; \
 	$(RUNTEST) --tool $* $(RUNTESTFLAGS))
 
-$(patsubst %,%-subtargets,$(filter-out $(lang_checks_parallelized),$(lang_checks))): check-%-subtargets:
+$(patsubst %,%-subtargets,$(lang_checks)): check-%-subtargets:
 	@echo check-$*
 
 check_p_tool=$(firstword $(subst _, ,$*))
-check_p_vars=$(check_$(check_p_tool)_parallelize)
+check_p_count=$(check_$(check_p_tool)_parallelize)
 check_p_subno=$(word 2,$(subst _, ,$*))
-check_p_comma=,
-check_p_subwork=$(subst $(check_p_comma), ,$(if $(check_p_subno),$(word $(check_p_subno),$(check_p_vars))))
-check_p_numbers=1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
+check_p_numbers0:=1 2 3 4 5 6 7 8 9
+check_p_numbers1:=0 $(check_p_numbers0)
+check_p_numbers2:=$(foreach i,$(check_p_numbers0),$(addprefix $(i),$(check_p_numbers1)))
+check_p_numbers3:=$(addprefix 0,$(check_p_numbers1)) $(check_p_numbers2)
+check_p_numbers4:=$(foreach i,$(check_p_numbers0),$(addprefix $(i),$(check_p_numbers3)))
+check_p_numbers5:=$(addprefix 0,$(check_p_numbers3)) $(check_p_numbers4)
+check_p_numbers6:=$(foreach i,$(check_p_numbers0),$(addprefix $(i),$(check_p_numbers5)))
+check_p_numbers:=$(check_p_numbers0) $(check_p_numbers2) $(check_p_numbers4) $(check_p_numbers6)
 check_p_subdir=$(subst _,,$*)
-check_p_subdirs=$(wordlist 1,$(words $(check_$*_parallelize)),$(check_p_numbers))
+check_p_subdirs=$(wordlist 1,$(check_p_count),$(wordlist 1, \
+		$(if $(GCC_TEST_PARALLEL_SLOTS),$(GCC_TEST_PARALLEL_SLOTS),128), \
+		$(check_p_numbers)))
 
 # For parallelized check-% targets, this decides whether parallelization
 # is desirable (if -jN is used and RUNTESTFLAGS doesn't contain anything
 # but optional --target_board or --extra_opts arguments).  If desirable,
 # recursive make is run with check-parallel-$lang{,1,2,3,4,5} etc. goals,
 # which can be executed in parallel, as they are run in separate directories.
-# check-parallel-$lang{1,2,3,4,5} etc. goals invoke runtest with the longest
-# running *.exp files from the testsuite, as determined by check_$lang_parallelize
-# variable.  The check-parallel-$lang goal in that case invokes runtest with
-# all the remaining *.exp files not handled by the separate goals.
+# check-parallel-$lang{,1,2,3,4,5} etc. goals invoke runtest with
+# GCC_RUNTEST_PARALLELIZE_DIR var in the environment and runtest_file_p
+# dejaGNU procedure is overridden to additionally synchronize through
+# a $lang-parallel directory which tests will be run by which runtest instance.
 # Afterwards contrib/dg-extract-results.sh is used to merge the sum and log
 # files.  If parallelization isn't desirable, only one recursive make
 # is run with check-parallel-$lang goal and check_$lang_parallelize variable
@@ -3662,76 +3645,60 @@  check_p_subdirs=$(wordlist 1,$(words $(c
 # to lang_checks_parallelized variable and define check_$lang_parallelize
 # variable (see above check_gcc_parallelize description).
 $(lang_checks_parallelized): check-% : site.exp
-	@if [ -z "$(filter-out --target_board=%,$(filter-out --extra_opts%,$(RUNTESTFLAGS)))" ] \
-	    && [ "$(filter -j, $(MFLAGS))" = "-j" ]; then \
+	-rm -rf $(TESTSUITEDIR)/$*-parallel
+	@if [ "$(filter -j, $(MFLAGS))" = "-j" ]; then \
+	  test -d $(TESTSUITEDIR) || mkdir $(TESTSUITEDIR) || true; \
+	  test -d $(TESTSUITEDIR)/$*-parallel || mkdir $(TESTSUITEDIR)/$*-parallel || true; \
+	  GCC_RUNTEST_PARALLELIZE_DIR=`${PWD_COMMAND}`/$(TESTSUITEDIR)/$(check_p_tool)-parallel ; \
+	  export GCC_RUNTEST_PARALLELIZE_DIR ; \
 	  $(MAKE) TESTSUITEDIR="$(TESTSUITEDIR)" RUNTESTFLAGS="$(RUNTESTFLAGS)" \
 	    check-parallel-$* \
 	    $(patsubst %,check-parallel-$*_%, $(check_p_subdirs)); \
-	  for file in $(TESTSUITEDIR)/$*/$* \
-		      $(patsubst %,$(TESTSUITEDIR)/$*%/$*,$(check_p_subdirs));\
+	  sums= ; logs= ; \
+	  for dir in $(TESTSUITEDIR)/$* \
+		     $(patsubst %,$(TESTSUITEDIR)/$*%,$(check_p_subdirs));\
 	  do \
-	    mv -f $$file.sum $$file.sum.sep; mv -f $$file.log $$file.log.sep; \
+	    if [ -d $$dir ]; then \
+	      mv -f $$dir/$*.sum $$dir/$*.sum.sep; mv -f $$dir/$*.log $$dir/$*.log.sep; \
+	      sums="$$sums $$dir/$*.sum.sep"; logs="$$logs $$dir/$*.log.sep"; \
+	    fi; \
 	  done; \
-	  $(SHELL) $(srcdir)/../contrib/dg-extract-results.sh \
-	    $(TESTSUITEDIR)/$*/$*.sum.sep \
-	    $(patsubst %,$(TESTSUITEDIR)/$*%/$*.sum.sep,$(check_p_subdirs)) \
+	  $(SHELL) $(srcdir)/../contrib/dg-extract-results.sh $$sums \
 	    > $(TESTSUITEDIR)/$*/$*.sum; \
-	  $(SHELL) $(srcdir)/../contrib/dg-extract-results.sh -L \
-	    $(TESTSUITEDIR)/$*/$*.log.sep \
-	    $(patsubst %,$(TESTSUITEDIR)/$*%/$*.log.sep,$(check_p_subdirs)) \
+	  $(SHELL) $(srcdir)/../contrib/dg-extract-results.sh -L $$logs \
 	    > $(TESTSUITEDIR)/$*/$*.log; \
+	  rm -rf $(TESTSUITEDIR)/$*-parallel || true; \
 	else \
 	  $(MAKE) TESTSUITEDIR="$(TESTSUITEDIR)" RUNTESTFLAGS="$(RUNTESTFLAGS)" \
 	    check_$*_parallelize= check-parallel-$*; \
 	fi
 
-# Just print the parallelized subtargets for those that want to split
-# the testing across machines.
-$(patsubst %,%-subtargets,$(lang_checks_parallelized)): check-%-subtargets:
-	@echo check-parallel-$* \
-	  $(patsubst %,check-parallel-$*_%, $(check_p_subdirs))
-
-# In the if [ -n "$(check_p_subno)" ] case runtest should be given the name of
-# the given *.exp file(s).  See comment above check_gcc_parallelize variable
-# for details on the content of these variables.
-#
-# In the elif [ -n "$(check_p_vars)" ] case runtest should be given
-# names of all the *.exp files for this tool that aren't already handled by
-# other goals.  First it finds all the *.exp files for this tool, then
-# prunes those already specified in check_$lang_parallelize or duplicates.
-#
-# Otherwise check-$lang isn't parallelized and runtest is invoked just with
-# the $(RUNTESTFLAGS) arguments.
 check-parallel-% : site.exp
 	-test -d plugin || mkdir plugin
 	-test -d $(TESTSUITEDIR) || mkdir $(TESTSUITEDIR)
 	test -d $(TESTSUITEDIR)/$(check_p_subdir) || mkdir $(TESTSUITEDIR)/$(check_p_subdir)
 	-(rootme=`${PWD_COMMAND}`; export rootme; \
 	srcdir=`cd ${srcdir}; ${PWD_COMMAND}` ; export srcdir ; \
-	cd $(TESTSUITEDIR)/$(check_p_subdir); \
-	rm -f tmp-site.exp; \
-	sed '/set tmpdir/ s|testsuite$$|$(TESTSUITEDIR)/$(check_p_subdir)|' \
+	if [ -n "$(check_p_subno)" ] \
+	   && [ -n "$$GCC_RUNTEST_PARALLELIZE_DIR" ] \
+	   && [ -f $(TESTSUITEDIR)/$(check_p_tool)-parallel/finished ]; then \
+	  rm -rf $(TESTSUITEDIR)/$(check_p_subdir); \
+	else \
+	  cd $(TESTSUITEDIR)/$(check_p_subdir); \
+	  rm -f tmp-site.exp; \
+	  sed '/set tmpdir/ s|testsuite$$|$(TESTSUITEDIR)/$(check_p_subdir)|' \
 		< ../../site.exp > tmp-site.exp; \
-	$(SHELL) $${srcdir}/../move-if-change tmp-site.exp site.exp; \
-	EXPECT=${EXPECT} ; export EXPECT ; \
-	if [ -f $${rootme}/../expect/expect ] ; then  \
-	   TCL_LIBRARY=`cd .. ; cd $${srcdir}/../tcl/library ; ${PWD_COMMAND}` ; \
-	    export TCL_LIBRARY ; fi ; \
-	runtestflags= ; \
-	if [ -n "$(check_p_subno)" ] ; then \
-	  runtestflags="$(check_p_subwork)"; \
-	elif [ -n "$(check_p_vars)" ] ; then \
-	  parts="`echo ' $(strip $(subst $(check_p_comma), ,$(check_p_vars))) ' \
-		  | sed 's/=[^ ]* / /g'`"; \
-	  for part in `find $$srcdir/testsuite/$(check_p_tool)* -name \*.exp` ; do \
-	    part=`basename $$part` ; \
-	    case " $$parts $$runtestflags " in \
-	      *" $$part "*) ;; \
-	      *) runtestflags="$$runtestflags $$part" ;; \
-	    esac ; \
-	  done ; \
-	fi ; \
-	$(RUNTEST) --tool $(check_p_tool) $(RUNTESTFLAGS) $$runtestflags)
+	  $(SHELL) $${srcdir}/../move-if-change tmp-site.exp site.exp; \
+	  EXPECT=${EXPECT} ; export EXPECT ; \
+	  if [ -f $${rootme}/../expect/expect ] ; then  \
+	    TCL_LIBRARY=`cd .. ; cd $${srcdir}/../tcl/library ; ${PWD_COMMAND}` ; \
+	    export TCL_LIBRARY ; \
+	  fi ; \
+	  $(RUNTEST) --tool $(check_p_tool) $(RUNTESTFLAGS); \
+	  if [ -n "$$GCC_RUNTEST_PARALLELIZE_DIR" ] ; then \
+	    touch $${rootme}/$(TESTSUITEDIR)/$(check_p_tool)-parallel/finished; \
+	  fi ; \
+	fi )
 
 # QMTest targets
 
--- gcc/go/Make-lang.in.jj	2014-09-14 21:09:05.744493198 +0200
+++ gcc/go/Make-lang.in	2014-09-14 21:25:17.767673249 +0200
@@ -131,11 +131,7 @@  go.srcman: doc/gccgo.1
 
 lang_checks += check-go
 lang_checks_parallelized += check-go
-check_go_parallelize = go-test.exp=*/test/\[0-57-9a-bd-hj-qs-zA-Z\]* \
-		       go-test.exp=*/test/c* \
-		       go-test.exp=*/test/i* \
-		       go-test.exp=*/test/r* \
-		       go-test.exp=*/test/6*
+check_go_parallelize = 10
 
 # Install hooks.
 
--- gcc/fortran/Make-lang.in.jj	2014-09-14 21:09:04.560499080 +0200
+++ gcc/fortran/Make-lang.in	2014-09-14 21:25:17.768673222 +0200
@@ -168,12 +168,7 @@  check-fortran-subtargets : check-gfortra
 lang_checks += check-gfortran
 lang_checks_parallelized += check-gfortran
 # For description see comment above check_gcc_parallelize in gcc/Makefile.in.
-check_gfortran_parallelize = dg.exp=gfortran.dg/\[adAD\]* \
-			     dg.exp=gfortran.dg/\[bcBC\]* \
-			     dg.exp=gfortran.dg/\[nopNOP\]* \
-			     dg.exp=gfortran.dg/\[isuvISUV\]* \
-			     dg.exp=gfortran.dg/\[efhkqrxzEFHKQRXZ\]* \
-			     dg.exp=gfortran.dg/\[0-9gjlmtwyGJLMTWY\]*
+check_gfortran_parallelize = 10000
 
 # GFORTRAN documentation.
 GFORTRAN_TEXI = \
--- gcc/cp/Make-lang.in.jj	2014-09-14 21:09:05.729493272 +0200
+++ gcc/cp/Make-lang.in	2014-09-14 21:25:17.768673222 +0200
@@ -156,11 +156,7 @@  check-c++-subtargets : check-g++-subtarg
 lang_checks += check-g++
 lang_checks_parallelized += check-g++
 # For description see comment above check_gcc_parallelize in gcc/Makefile.in.
-check_g++_parallelize = old-deja.exp \
-	dg.exp=g++.dg/[0-9A-Za-bd-su-z]* \
-	dg.exp=g++.dg/[ct]* \
-	dg.exp=c-c++-common/*,dg-torture.exp
-
+check_g++_parallelize = 10000
 #
 # Install hooks:
 # cc1plus is installed elsewhere as part of $(COMPILERS).
--- gcc/ada/gcc-interface/Make-lang.in.jj	2014-08-04 10:43:27.000000000 +0200
+++ gcc/ada/gcc-interface/Make-lang.in	2014-09-15 14:38:30.250714169 +0200
@@ -862,7 +862,18 @@  check-ada-subtargets: check-acats-subtar
 
 ACATSDIR = $(TESTSUITEDIR)/ada/acats
 
-check_acats_targets = $(patsubst %,check-acats%, 0 1 2)
+check_acats_numbers0:=1 2 3 4 5 6 7 8 9
+check_acats_numbers1:=0 $(check_acats_numbers0)
+check_acats_numbers2:=$(foreach i,$(check_acats_numbers0),$(addprefix $(i),$(check_acats_numbers1)))
+check_acats_numbers3:=$(addprefix 0,$(check_acats_numbers1)) $(check_acats_numbers2)
+check_acats_numbers4:=$(foreach i,$(check_acats_numbers0),$(addprefix $(i),$(check_acats_numbers3)))
+check_acats_numbers5:=$(addprefix 0,$(check_acats_numbers3)) $(check_acats_numbers4)
+check_acats_numbers6:=$(foreach i,$(check_acats_numbers0),$(addprefix $(i),$(check_acats_numbers5)))
+check_acats_numbers:=$(check_acats_numbers0) $(check_acats_numbers2) $(check_acats_numbers4) $(check_acats_numbers6)
+check_acats_subdirs=$(wordlist 1, \
+		    $(if $(GCC_TEST_PARALLEL_SLOTS),$(GCC_TEST_PARALLEL_SLOTS),128), \
+		    $(check_acats_numbers))
+check_acats_targets = $(addprefix check-acats, $(check_acats_subdirs))
 
 check-acats:
 	@test -d $(ACATSDIR) || mkdir -p $(ACATSDIR); \
@@ -870,35 +881,45 @@  check-acats:
 	EXPECT=$(EXPECT); export EXPECT; \
 	if [ -z "$(CHAPTERS)" ] && [ "$(filter -j, $(MFLAGS))" = "-j" ]; \
 	then \
+	  rm -rf $(ACATSDIR)-parallel; \
+	  mkdir $(ACATSDIR)-parallel; \
+	  ( testdir=`cd ${srcdir}/${ACATSDIR} && ${PWD_COMMAND}`; \
+	    export testdir; \
+	    cd $(ACATSDIR) && $(SHELL) $${testdir}/run_acats NONE ) \
+	    || exit 1; \
+	  GCC_RUNTEST_PARALLELIZE_DIR=$$rootme/$(ACATSDIR)-parallel; \
+	  export GCC_RUNTEST_PARALLELIZE_DIR; \
 	  $(MAKE) $(check_acats_targets); \
-	  for idx in 0 1 2; do \
+	  rm -rf $(ACATSDIR)-parallel; \
+	  acats_sums= ; acats_logs= ; \
+	  for idx in "" $(check_acats_subdirs); do \
+	    [ -d $(ACATSDIR)$$idx ] || continue; \
 	    mv -f $(ACATSDIR)$$idx/acats.sum $(ACATSDIR)$$idx/acats.sum.sep; \
 	    mv -f $(ACATSDIR)$$idx/acats.log $(ACATSDIR)$$idx/acats.log.sep; \
+	    acats_sums="$$acats_sums $(ACATSDIR)$$idx/acats.sum.sep"; \
+	    acats_logs="$$acats_logs $(ACATSDIR)$$idx/acats.log.sep"; \
 	  done; \
 	  $(SHELL) $(srcdir)/../contrib/dg-extract-results.sh \
-	    $(ACATSDIR)0/acats.sum.sep $(ACATSDIR)1/acats.sum.sep \
-	    $(ACATSDIR)2/acats.sum.sep > $(ACATSDIR)/acats.sum; \
+	    $$acats_sums > $(ACATSDIR)/acats.sum; \
 	  $(SHELL) $(srcdir)/../contrib/dg-extract-results.sh -L \
-	    $(ACATSDIR)0/acats.log.sep $(ACATSDIR)1/acats.log.sep \
-	    $(ACATSDIR)2/acats.log.sep > $(ACATSDIR)/acats.log; \
+	    $$acats_logs > $(ACATSDIR)/acats.log; \
 	  exit 0; \
 	fi; \
 	testdir=`cd ${srcdir}/${ACATSDIR} && ${PWD_COMMAND}`; \
 	export testdir; cd $(ACATSDIR) && $(SHELL) $${testdir}/run_acats $(CHAPTERS)
 
 check-acats-subtargets:
-	@echo $(check_acats_targets)
+	@echo check-acats
 
 # Parallelized check-acats
 $(check_acats_targets): check-acats%:
+	if [ -f $$GCC_RUNTEST_PARALLELIZE_DIR/finished ]; then \
+	  rm -rf $(ACATSDIR)$*; exit 0; \
+	fi; \
 	test -d $(ACATSDIR)$* || mkdir -p $(ACATSDIR)$*; \
 	testdir=`cd ${srcdir}/${ACATSDIR} && ${PWD_COMMAND}`; \
-	case "$*" in \
-	  0) chapters="`cd $$testdir/tests; echo [a-b]* c[0-4]*`";; \
-	  1) chapters="`cd $$testdir/tests; echo c[5-9ab]*`";; \
-	  2) chapters="`cd $$testdir/tests; echo c[c-z]* [d-z]*`";; \
-	esac; \
-	export testdir; cd $(ACATSDIR)$* && $(SHELL) $${testdir}/run_acats $$chapters
+	export testdir; cd $(ACATSDIR)$* && $(SHELL) $${testdir}/run_acats
+	touch $$GCC_RUNTEST_PARALLELIZE_DIR/finished
 
 .PHONY: check-acats $(check_acats_targets)
 
--- gcc/testsuite/gcc.misc-tests/sieve.exp.jj	2014-09-14 21:09:05.668493574 +0200
+++ gcc/testsuite/gcc.misc-tests/sieve.exp	2014-09-14 21:25:17.773673082 +0200
@@ -20,9 +20,21 @@  if { ![info exists PERF_TEST] || "$PERF_
 }
 
 load_lib mike-gcc.exp
+load_lib gcc-defs.exp
+
+# These tests don't run runtest_file_p consistently if it
+# doesn't return the same values, so disable parallelization
+# of this *.exp file.  The first parallel runtest to reach
+# this will run all the tests serially.
+if ![gcc_parallel_test_run_p sieve] {
+    return
+}
+gcc_parallel_test_enable 0
 
 prebase
 set actions run
 set compiler_output "^$"
 set program_output "^$"
 postbase sieve.c $run $groups
+
+gcc_parallel_test_enable 1
--- gcc/testsuite/gcc.misc-tests/sort2.exp.jj	2014-09-14 21:09:05.524494295 +0200
+++ gcc/testsuite/gcc.misc-tests/sort2.exp	2014-09-14 21:25:17.773673082 +0200
@@ -20,9 +20,21 @@  if { ![info exists PERF_TEST] || "$PERF_
 }
 
 load_lib mike-gcc.exp
+load_lib gcc-defs.exp
+
+# These tests don't run runtest_file_p consistently if it
+# doesn't return the same values, so disable parallelization
+# of this *.exp file.  The first parallel runtest to reach
+# this will run all the tests serially.
+if ![gcc_parallel_test_run_p sort2] {
+    return
+}
+gcc_parallel_test_enable 0
 
 prebase
 set actions run
 set compiler_output "^$"
 set program_output "^$"
 postbase sort2.c $run $groups
+
+gcc_parallel_test_enable 1
--- gcc/testsuite/gcc.misc-tests/matrix1.exp.jj	2014-09-14 21:09:05.544494191 +0200
+++ gcc/testsuite/gcc.misc-tests/matrix1.exp	2014-09-14 21:25:17.772673116 +0200
@@ -20,9 +20,21 @@  if { ![info exists PERF_TEST] || "$PERF_
 }
 
 load_lib mike-gcc.exp
+load_lib gcc-defs.exp
+
+# These tests don't run runtest_file_p consistently if it
+# doesn't return the same values, so disable parallelization
+# of this *.exp file.  The first parallel runtest to reach
+# this will run all the tests serially.
+if ![gcc_parallel_test_run_p matrix1] {
+    return
+}
+gcc_parallel_test_enable 0
 
 prebase
 set actions run
 set compiler_output "^$"
 set program_output "^$"
 postbase matrix1.c $run $groups
+
+gcc_parallel_test_enable 1
--- gcc/testsuite/gcc.misc-tests/dhry.exp.jj	2014-09-14 21:09:05.596493933 +0200
+++ gcc/testsuite/gcc.misc-tests/dhry.exp	2014-09-14 21:25:17.772673116 +0200
@@ -20,9 +20,21 @@  if { ![info exists PERF_TEST] || "$PERF_
 }
 
 load_lib mike-gcc.exp
+load_lib gcc-defs.exp
+
+# These tests don't run runtest_file_p consistently if it
+# doesn't return the same values, so disable parallelization
+# of this *.exp file.  The first parallel runtest to reach
+# this will run all the tests serially.
+if ![gcc_parallel_test_run_p dhry] {
+    return
+}
+gcc_parallel_test_enable 0
 
 prebase
 set actions run
 set compiler_output "^$"
 set program_output "^$"
 postbase dhry.c $run $groups
+
+gcc_parallel_test_enable 1
--- gcc/testsuite/gcc.misc-tests/mg.exp.jj	2014-09-14 21:09:05.580494013 +0200
+++ gcc/testsuite/gcc.misc-tests/mg.exp	2014-09-14 21:25:17.772673116 +0200
@@ -17,8 +17,20 @@ 
 # Test the -MG flag.
 
 load_lib mike-gcc.exp
+load_lib gcc-defs.exp
+
+# These tests don't run runtest_file_p consistently if it
+# doesn't return the same values, so disable parallelization
+# of this *.exp file.  The first parallel runtest to reach
+# this will run all the tests serially.
+if ![gcc_parallel_test_run_p mg] {
+    return
+}
+gcc_parallel_test_enable 0
 
 prebase
 set actions none-of-the-above
 set compiler_output "mg.o ?: .*mg.c \[ \\\\\n\]*nonexist.h"
 postbase mg.c "" "" "-MM -MG"
+
+gcc_parallel_test_enable 1
--- gcc/testsuite/gcc.misc-tests/options.exp.jj	2014-09-14 21:09:05.563494097 +0200
+++ gcc/testsuite/gcc.misc-tests/options.exp	2014-09-14 21:25:17.773673082 +0200
@@ -19,6 +19,17 @@ 
 # match the patterns COMPILER_PATTERN, AS_PATTERN and LD_PATTERN,
 # respectively.
 
+load_lib gcc-defs.exp
+
+# These tests don't run runtest_file_p consistently if it
+# doesn't return the same values, so disable parallelization
+# of this *.exp file.  The first parallel runtest to reach
+# this will run all the tests serially.
+if ![gcc_parallel_test_run_p options] {
+    return
+}
+gcc_parallel_test_enable 0
+
 proc check_for_all_options {language gcc_options compiler_pattern as_pattern ld_pattern} {
     set filename test-[pid]
     set fd [open $filename.c w]
@@ -52,3 +63,5 @@  proc check_for_all_options {language gcc
 }
 
 check_for_all_options c {--coverage} {-fprofile-arcs -ftest-coverage} {} {-lgcov}
+
+gcc_parallel_test_enable 1
--- gcc/testsuite/gcc.misc-tests/help.exp.jj	2014-09-14 21:09:05.555494137 +0200
+++ gcc/testsuite/gcc.misc-tests/help.exp	2014-09-14 21:25:17.774673051 +0200
@@ -18,6 +18,16 @@ 
 # documented in --help, and that the various --help* options work.
 
 load_lib options.exp
+load_lib gcc-defs.exp
+
+# These tests don't run runtest_file_p consistently if it
+# doesn't return the same values, so disable parallelization
+# of this *.exp file.  The first parallel runtest to reach
+# this will run all the tests serially.
+if ![gcc_parallel_test_run_p help] {
+    return
+}
+gcc_parallel_test_enable 0
 
 # Document --version.  Ideally, there should be no undocumented switches
 # in --help.
@@ -78,3 +88,5 @@  check_for_options c "--help=joined,^sepa
 check_for_options c "--help=joined,undocumented" "" "" ""
 # Listing only excludes gives empty results.
 check_for_options c "--help=^joined,^separate" "" "" ""
+
+gcc_parallel_test_enable 1
--- gcc/testsuite/gcc.misc-tests/linkage.exp.jj	2014-09-14 21:09:05.624493790 +0200
+++ gcc/testsuite/gcc.misc-tests/linkage.exp	2014-09-14 21:25:17.772673116 +0200
@@ -18,6 +18,17 @@ 
 # was written by Rob Savoye. (rob@cygnus.com)
 # All the other tests driven by that file have since been moved elsewhere.
 
+load_lib gcc-defs.exp
+
+# These tests don't run runtest_file_p consistently if it
+# doesn't return the same values, so disable parallelization
+# of this *.exp file.  The first parallel runtest to reach
+# this will run all the tests serially.
+if ![gcc_parallel_test_run_p linkage] {
+    return
+}
+gcc_parallel_test_enable 0
+
 if { [isnative] && ![is_remote host] } then {
     set lines [gcc_target_compile "$srcdir/$subdir/linkage-x.c" "linkage-x.o" object {additional_flags="-w"}]
     if ![string match "" $lines] then {
@@ -117,3 +128,5 @@  if { [isnative] && ![is_remote host] } t
 	file delete "linkage-x.o"
     }
 }
+
+gcc_parallel_test_enable 1
--- gcc/testsuite/gcc.misc-tests/acker1.exp.jj	2014-09-14 21:09:05.586493983 +0200
+++ gcc/testsuite/gcc.misc-tests/acker1.exp	2014-09-14 21:25:17.772673116 +0200
@@ -20,9 +20,21 @@  if { ![info exists PERF_TEST] || "$PERF_
 }
 
 load_lib mike-gcc.exp
+load_lib gcc-defs.exp
+
+# These tests don't run runtest_file_p consistently if it
+# doesn't return the same values, so disable parallelization
+# of this *.exp file.  The first parallel runtest to reach
+# this will run all the tests serially.
+if ![gcc_parallel_test_run_p acker1] {
+    return
+}
+gcc_parallel_test_enable 0
 
 prebase
 set actions run
 set compiler_output "^$"
 set program_output "^$"
 postbase acker1.c $run $groups
+
+gcc_parallel_test_enable 1
--- gcc/testsuite/gcc.misc-tests/mg-2.exp.jj	2014-09-14 21:09:05.570494062 +0200
+++ gcc/testsuite/gcc.misc-tests/mg-2.exp	2014-09-14 21:25:17.773673082 +0200
@@ -17,8 +17,20 @@ 
 # Test the -MG flag with a system header file.
 
 load_lib mike-gcc.exp
+load_lib gcc-defs.exp
+
+# These tests don't run runtest_file_p consistently if it
+# doesn't return the same values, so disable parallelization
+# of this *.exp file.  The first parallel runtest to reach
+# this will run all the tests serially.
+if ![gcc_parallel_test_run_p mg-2] {
+    return
+}
+gcc_parallel_test_enable 0
 
 prebase
 set actions none-of-the-above
 set compiler_output "mg-2.o ?: .*mg-2.c \[ \\\\\n\]*nonexist.h"
 postbase mg-2.c "" "" "-MM -MG"
+
+gcc_parallel_test_enable 1
--- gcc/testsuite/g++.dg/guality/guality.exp.jj	2014-09-14 21:09:04.767498056 +0200
+++ gcc/testsuite/g++.dg/guality/guality.exp	2014-09-14 21:25:17.768673222 +0200
@@ -14,6 +14,11 @@  if { [istarget "powerpc-ibm-aix*"] } {
 }
 
 proc check_guality {args} {
+    # Don't count check_guality as PASS, or FAIL etc., that would make
+    # the total PASS count dependent on how many parallel runtest invocations
+    # ran guality.exp.  So save the counts first and restore them afterwards.
+    global test_counts
+    array set saved_test_counts [array get test_counts]
     set result [eval check_compile guality_check executable $args "-g -O0"]
     set lines [lindex $result 0]
     set output [lindex $result 1]
@@ -23,6 +28,7 @@  proc check_guality {args} {
       set ret [string match "*1 PASS, 0 FAIL, 0 UNRESOLVED*" $execout]
     }
     remote_file build delete $output
+    array get test_counts [array get saved_test_counts]
     return $ret
 }
 
--- gcc/testsuite/g++.dg/plugin/plugin.exp.jj	2014-09-14 21:09:04.824497773 +0200
+++ gcc/testsuite/g++.dg/plugin/plugin.exp	2014-09-14 21:25:17.769673197 +0200
@@ -44,6 +44,15 @@  if $tracelevel then {
 # Load support procs.
 load_lib plugin-support.exp
 
+# These tests don't run runtest_file_p consistently if it
+# doesn't return the same values, so disable parallelization
+# of this *.exp file.  The first parallel runtest to reach
+# this will run all the tests serially.
+if ![gcc_parallel_test_run_p plugin] {
+    return
+}
+gcc_parallel_test_enable 0
+
 # Specify the plugin source file and the associated test files in a list.
 # plugin_test_list={ {plugin1 test1 test2 ...} {plugin2 test1 ...} ... }
 set plugin_test_list [list \
@@ -68,3 +77,5 @@  foreach plugin_test $plugin_test_list {
     set plugin_input_tests [lreplace $plugin_test 0 0]
     plugin-test-execute $plugin_src $plugin_input_tests
 }
+
+gcc_parallel_test_enable 1
--- gcc/testsuite/gcc.dg/guality/guality.exp.jj	2014-09-14 21:09:05.362495088 +0200
+++ gcc/testsuite/gcc.dg/guality/guality.exp	2014-09-14 21:25:17.769673197 +0200
@@ -14,6 +14,11 @@  if { [istarget "powerpc-ibm-aix*"] } {
 }
 
 proc check_guality {args} {
+    # Don't count check_guality as PASS, or FAIL etc., that would make
+    # the total PASS count dependent on how many parallel runtest invocations
+    # ran guality.exp.  So save the counts first and restore them afterwards.
+    global test_counts
+    array set saved_test_counts [array get test_counts]
     set result [eval check_compile guality_check executable $args "-g -O0"]
     set lines [lindex $result 0]
     set output [lindex $result 1]
@@ -23,6 +28,7 @@  proc check_guality {args} {
       set ret [string match "*1 PASS, 0 FAIL, 0 UNRESOLVED*" $execout]
     }
     remote_file build delete $output
+    array set test_counts [array get saved_test_counts]
     return $ret
 }
 
--- gcc/testsuite/gcc.dg/plugin/plugin.exp.jj	2014-09-14 21:09:05.470494558 +0200
+++ gcc/testsuite/gcc.dg/plugin/plugin.exp	2014-09-14 21:25:17.770673173 +0200
@@ -44,6 +44,15 @@  if $tracelevel then {
 # Load support procs.
 load_lib plugin-support.exp
 
+# These tests don't run runtest_file_p consistently if it
+# doesn't return the same values, so disable parallelization
+# of this *.exp file.  The first parallel runtest to reach
+# this will run all the tests serially.
+if ![gcc_parallel_test_run_p plugin] {
+    return
+}
+gcc_parallel_test_enable 0
+
 # Specify the plugin source file and the associated test files in a list.
 # plugin_test_list={ {plugin1 test1 test2 ...} {plugin2 test1 ...} ... }
 set plugin_test_list [list \
@@ -80,3 +89,5 @@  dg-runtest [lsort [glob -nocomplain $src
 
 # All done.
 dg-finish
+
+gcc_parallel_test_enable 1
--- gcc/testsuite/go.test/go-test.exp.jj	2014-09-14 21:09:05.292495444 +0200
+++ gcc/testsuite/go.test/go-test.exp	2014-09-14 21:25:17.770673173 +0200
@@ -489,6 +489,12 @@  proc go-gc-tests { } {
 
 	close $fd
 
+	# runtest_file_p is already run above, and the code below can run
+	# runtest_file_p again, make sure everything for this test is
+	# performed if the above runtest_file_p decided this runtest
+	# instance should execute the test
+	gcc_parallel_test_enable 0
+
 	set go_compile_args ""
 	set go_execute_args ""
 	if { [regexp "// run (\[^|&>2\].*)\$" $test_line match progargs] \
@@ -1164,6 +1170,7 @@  proc go-gc-tests { } {
 	set go_compile_args ""
 	set go_execute_args ""
         set TORTURE_OPTIONS [list { -O2 -g }]
+	gcc_parallel_test_enable 1
     }
 
     set dg-do-what-default ${saved-dg-do-what-default}
--- gcc/testsuite/ada/acats/run_all.sh.jj	2013-06-01 10:28:28.100138045 +0200
+++ gcc/testsuite/ada/acats/run_all.sh	2014-09-15 15:33:15.793502393 +0200
@@ -18,6 +18,21 @@  target_run () {
 
 # End of customization section.
 
+# Perform arithmetic evaluation on the ARGs, and store the result in the
+# global $as_val. Take advantage of shells that can avoid forks. The arguments
+# must be portable across $(()) and expr.
+if (eval "test \$(( 1 + 1 )) = 2") 2>/dev/null; then :
+  eval 'as_fn_arith ()
+  {
+    as_val=$(( $* ))
+  }'
+else
+  as_fn_arith ()
+  {
+    as_val=`expr "$@" || test $? -eq 1`
+  }
+fi # as_fn_arith
+
 display_noeol () {
   printf "$@"
   printf "$@" >> $dir/acats.sum
@@ -90,6 +105,18 @@  display `type gnatmake`
 gnatls -v >> $dir/acats.log
 display ""
 
+if [ -n "$GCC_RUNTEST_PARALLELIZE_DIR" ]; then
+  dir_support=$dir/../acats/support
+
+  rm -rf $dir/run
+  mv $dir/tests $dir/tests.$$ 2> /dev/null
+  rm -rf $dir/tests.$$ &
+  mkdir -p $dir/run
+
+  cp -pr $dir/../acats/tests $dir/
+else
+  dir_support=$dir/support
+
 display "		=== acats support ==="
 display_noeol "Generating support files..."
 
@@ -186,6 +213,9 @@  target_gnatmake -c -gnato -gnatE *.adb >
 
 display " done."
 display ""
+
+fi
+
 display "		=== acats tests ==="
 
 if [ $# -eq 0 ]; then
@@ -197,8 +227,14 @@  fi
 glob_countn=0
 glob_countok=0
 glob_countu=0
+par_count=0
+par_countm=0
+par_last=
 
 for chapter in $chapters; do
+   # Used to generate support once and finish after that.
+   [ "$chapter" = "NONE" ] && continue
+
    display Running chapter $chapter ...
 
    if [ ! -d $dir/tests/$chapter ]; then
@@ -209,13 +245,43 @@  for chapter in $chapters; do
 
    cd $dir/tests/$chapter
    ls *.a *.ada *.adt *.am *.dep 2> /dev/null | sed -e 's/\(.*\)\..*/\1/g' | \
-   cut -c1-7 | sort | uniq | comm -23 - $dir/support/norun.lst \
+   cut -c1-7 | sort | uniq | comm -23 - $dir_support/norun.lst \
      > $dir/tests/$chapter/${chapter}.lst 
    countn=`wc -l < $dir/tests/$chapter/${chapter}.lst`
-   glob_countn=`expr $glob_countn + $countn`
-   counti=0
+   as_fn_arith $glob_countn + $countn
+   glob_countn=$as_val
    for i in `cat $dir/tests/$chapter/${chapter}.lst`; do 
-      counti=`expr $counti + 1`
+
+      # If running multiple run_all.sh jobs in parallel, decide
+      # if we should run this test in the current instance.
+      if [ -n "$GCC_RUNTEST_PARALLELIZE_DIR" ]; then
+	 case "$i" in
+	    # Ugh, some tests have inter-test dependencies, those
+	    # tests have to be scheduled on the same parallel instance
+	    # as previous test.
+	    ce2108f | ce2108h | ce3112d) ;;
+	    # All others can be hopefully scheduled freely.
+	    *)
+	       as_fn_arith $par_countm + 1
+	       par_countm=$as_val
+	       [ $par_countm -eq 10 ] && par_countm=0
+	       if [ $par_countm -eq 1 ]; then
+		  as_fn_arith $par_count + 1
+		  par_count=$as_val
+		  if mkdir $GCC_RUNTEST_PARALLELIZE_DIR/$par_count; then
+		     par_last=1
+		  else
+		     par_last=
+		  fi
+	       fi;;
+	 esac
+	 if [ -z "$par_last" ]; then
+	    as_fn_arith $glob_countn - 1
+	    glob_countn=$as_val
+	    continue
+	 fi
+      fi
+
       extraflags="-gnat95"
       grep $i $testdir/overflow.lst > /dev/null 2>&1
       if [ $? -eq 0 ]; then
@@ -254,7 +320,7 @@  for chapter in $chapters; do
       echo "BUILD $main" >> $dir/acats.log
       EXTERNAL_OBJECTS=""
       case $i in
-        cxb30*) EXTERNAL_OBJECTS="$dir/support/cxb30040.o $dir/support/cxb30060.o $dir/support/cxb30130.o $dir/support/cxb30131.o";;
+        cxb30*) EXTERNAL_OBJECTS="$dir_support/cxb30040.o $dir_support/cxb30060.o $dir_support/cxb30130.o $dir_support/cxb30131.o";;
         ca1020e) rm -f ca1020e_func1.adb ca1020e_func2.adb ca1020e_proc1.adb ca1020e_proc2.adb > /dev/null 2>&1;;
         ca14028) rm -f ca14028_func2.ads ca14028_func3.ads ca14028_proc1.ads ca14028_proc3.ads > /dev/null 2>&1;;
       esac
@@ -265,7 +331,7 @@  for chapter in $chapters; do
          continue
       fi
 
-      target_gnatmake $extraflags -I$dir/support $main >> $dir/acats.log 2>&1
+      target_gnatmake $extraflags -I$dir_support $main >> $dir/acats.log 2>&1
       if [ $? -ne 0 ]; then
          display "FAIL:	$i"
          failed="${failed}${i} "
@@ -290,12 +356,15 @@  for chapter in $chapters; do
             failed="${failed}${i} "
          else
             log "UNSUPPORTED:	$i"
-            glob_countn=`expr $glob_countn - 1`
-            glob_countu=`expr $glob_countu + 1`
+            as_fn_arith $glob_countn - 1
+            glob_countn=$as_val
+            as_fn_arith $glob_countu + 1
+            glob_countu=$as_val
          fi
       else
          log "PASS:	$i"
-         glob_countok=`expr $glob_countok + 1`
+         as_fn_arith $glob_countok + 1
+         glob_countok=$as_val
       fi
       clean_dir
    done
--- gcc/testsuite/lib/gcc-defs.exp.jj	2014-09-14 21:09:04.880497500 +0200
+++ gcc/testsuite/lib/gcc-defs.exp	2014-09-14 22:10:41.116165494 +0200
@@ -188,6 +188,81 @@  if { [info procs runtest_file_p] == "" }
     }
 }
 
+if { [info exists env(GCC_RUNTEST_PARALLELIZE_DIR)] \
+     && [info procs runtest_file_p] != [list] \
+     && [info procs gcc_parallelize_saved_runtest_file_p] == [list] } then {
+    global gcc_runtest_parallelize_counter
+    global gcc_runtest_parallelize_counter_minor
+    global gcc_runtest_parallelize_enable
+    global gcc_runtest_parallelize_dir
+    global gcc_runtest_parallelize_last
+
+    set gcc_runtest_parallelize_counter 0
+    set gcc_runtest_parallelize_counter_minor 0
+    set gcc_runtest_parallelize_enable 1
+    set gcc_runtest_parallelize_dir [getenv GCC_RUNTEST_PARALLELIZE_DIR]
+    set gcc_runtest_parallelize_last 0
+
+    proc gcc_parallel_test_run_p { testcase } {
+	global gcc_runtest_parallelize_counter
+	global gcc_runtest_parallelize_counter_minor
+	global gcc_runtest_parallelize_enable
+	global gcc_runtest_parallelize_dir
+	global gcc_runtest_parallelize_last
+
+	if { $gcc_runtest_parallelize_enable == 0 } {
+	    return 1
+	}
+
+	# Only test the filesystem every 10th iteration
+	incr gcc_runtest_parallelize_counter_minor
+	if { $gcc_runtest_parallelize_counter_minor == 10 } {
+	    set gcc_runtest_parallelize_counter_minor 0
+	}
+	if { $gcc_runtest_parallelize_counter_minor != 1 } {
+	    #verbose -log "gcc_parallel_test_run_p $testcase $gcc_runtest_parallelize_counter $gcc_runtest_parallelize_last"
+	    return $gcc_runtest_parallelize_last
+	}
+
+	set path $gcc_runtest_parallelize_dir/$gcc_runtest_parallelize_counter
+
+	if {![catch {open $path {RDWR CREAT EXCL} 0600} fd]} {
+	    close $fd
+	    set gcc_runtest_parallelize_last 1
+	    #verbose -log "gcc_parallel_test_run_p $testcase $gcc_runtest_parallelize_counter 1"
+	    incr gcc_runtest_parallelize_counter
+	    return 1
+	}
+	set gcc_runtest_parallelize_last 0
+	#verbose -log "gcc_parallel_test_run_p $testcase $gcc_runtest_parallelize_counter 0"
+	incr gcc_runtest_parallelize_counter
+	return 0
+    }
+
+    proc gcc_parallel_test_enable { val } {
+	global gcc_runtest_parallelize_enable
+	set gcc_runtest_parallelize_enable $val
+    }
+
+    rename runtest_file_p gcc_parallelize_saved_runtest_file_p
+    proc runtest_file_p { runtests testcase } {
+	if ![gcc_parallelize_saved_runtest_file_p $runtests $testcase] {
+	    return 0
+	}
+	return [gcc_parallel_test_run_p $testcase]
+    }
+
+} else {
+
+    proc gcc_parallel_test_run_p { testcase } {
+	return 1
+    }
+
+    proc gcc_parallel_test_enable { val } {
+    }
+
+}
+
 # Like dg-options, but adds to the default options rather than replacing them.
 
 proc dg-additional-options { args } {
--- gcc/testsuite/objc.dg/gnu-encoding/gnu-encoding.exp.jj	2014-09-14 21:09:05.237495674 +0200
+++ gcc/testsuite/objc.dg/gnu-encoding/gnu-encoding.exp	2014-09-14 22:09:58.788381473 +0200
@@ -58,7 +58,9 @@  if { $status == 0 } then {
                 continue
             }
 
+	    gcc_parallel_test_enable 0
             dg-runtest $src "" $DEFAULT_CFLAGS
+	    gcc_parallel_test_enable 1
         }
     } else {
         warning "Could not execute objc.dg/gnu-encoding/struct-layout-encoding-1 generator"
--- gcc/objc/Make-lang.in.jj	2014-09-14 21:09:05.807492885 +0200
+++ gcc/objc/Make-lang.in	2014-09-14 21:25:17.771673151 +0200
@@ -96,7 +96,7 @@  lang_checks += check-objc
 # The following allows you to do 'make check-objc -j2'.  The
 # execute.exp tests will be run in parallel with all the other ones.
 lang_checks_parallelized += check-objc
-check_objc_parallelize = gnu-encoding.exp execute.exp exceptions.exp
+check_objc_parallelize = 6
 
 #
 # Install hooks:
--- libstdc++-v3/testsuite/libstdc++-abi/abi.exp.jj	2014-05-20 16:36:47.000000000 +0200
+++ libstdc++-v3/testsuite/libstdc++-abi/abi.exp	2014-09-15 14:23:06.308253078 +0200
@@ -18,6 +18,12 @@ 
 # this test. Or, hey, if we don't support this kind of symbol
 # versioning test: don't run it.
 
+# Only run whole abi.exp in one instance, serially.
+if ![gcc_parallel_test_run_p abi] {
+    return
+}
+gcc_parallel_test_enable 0
+
 if { [string match "*-*-darwin*" $target_triplet] } {
     set lib $blddir/src/.libs/libstdc++.dylib
 } else { 
@@ -30,6 +36,7 @@  v3-build_support
 if { (${v3-symver} == 0) || ![info exists baseline_dir] \
 	 || ![file exists $baseline_dir] \
 	 || ![file exists $lib] } {
+    gcc_parallel_test_enable 1
     return
 }
 
@@ -42,12 +49,13 @@  if ![file exists $baseline_file] {
     set baseline_file [file join $baseline_dir "baseline_symbols.txt"]
 }
 if ![file exists $baseline_file] {
+    gcc_parallel_test_enable 1
     return
 }
 send_log "Checking $lib against $baseline_file\n"
 
 # Figure out what symbols are defined by the active build of the library.
-remote_exec "build" "$objdir/../scripts/extract_symvers" \
+remote_exec "build" "$blddir/scripts/extract_symvers" \
     [list $lib "current_symbols.txt"]
 
 # Build the abi_check program.
@@ -62,3 +70,5 @@  set result [${tool}_load "./abi_check" \
 		[list "--check-verbose" "current_symbols.txt" \
 		     "baseline_symbols.txt"]]
 [lindex $result 0] "libstdc++-abi/abi_check"
+
+gcc_parallel_test_enable 1
--- libstdc++-v3/testsuite/libstdc++-xmethods/xmethods.exp.jj	2014-09-10 21:57:14.000000000 +0200
+++ libstdc++-v3/testsuite/libstdc++-xmethods/xmethods.exp	2014-09-15 15:31:50.277960368 +0200
@@ -16,6 +16,12 @@ 
 
 load_lib gdb-test.exp
 
+# Only run whole xmethods.exp in one instance, serially.
+if ![gcc_parallel_test_run_p xmethods] {
+    return
+}
+gcc_parallel_test_enable 0
+
 dg-init
 v3-build_support
 
@@ -30,7 +36,9 @@  if ![info exists ::env(GUALITY_GDB_NAME)
 }
 
 if {! [gdb_version_check_xmethods]} {
+    dg-finish
     unsupported "xmethods.exp"
+    gcc_parallel_test_enable 1
     return
 }
 
@@ -49,3 +57,4 @@  if [info exists guality_gdb_name] {
 }
 
 dg-finish
+gcc_parallel_test_enable 1
--- libstdc++-v3/testsuite/Makefile.am.jj	2014-09-14 21:08:57.067536354 +0200
+++ libstdc++-v3/testsuite/Makefile.am	2014-09-15 15:27:57.632110277 +0200
@@ -101,26 +101,37 @@  new-abi-baseline:
 	@test ! -f $*/site.exp || mv $*/site.exp $*/site.bak
 	@mv $*/site.exp.tmp $*/site.exp
 
-check_DEJAGNU_normal_targets = $(patsubst %,check-DEJAGNUnormal%,0 1 2 3 4 5 6 7 8 9 10)
+check_p_numbers0:=1 2 3 4 5 6 7 8 9
+check_p_numbers1:=0 $(check_p_numbers0)
+check_p_numbers2:=$(foreach i,$(check_p_numbers0),$(addprefix $(i),$(check_p_numbers1)))
+check_p_numbers3:=$(addprefix 0,$(check_p_numbers1)) $(check_p_numbers2)
+check_p_numbers4:=$(foreach i,$(check_p_numbers0),$(addprefix $(i),$(check_p_numbers3)))
+check_p_numbers5:=$(addprefix 0,$(check_p_numbers3)) $(check_p_numbers4)
+check_p_numbers6:=$(foreach i,$(check_p_numbers0),$(addprefix $(i),$(check_p_numbers5)))
+check_p_numbers:=$(check_p_numbers0) $(check_p_numbers2) $(check_p_numbers4) $(check_p_numbers6)
+check_p_subdirs=$(wordlist 1,$(if $(GCC_TEST_PARALLEL_SLOTS),$(GCC_TEST_PARALLEL_SLOTS),128),$(check_p_numbers))
+check_DEJAGNU_normal_targets = $(addprefix check-DEJAGNUnormal,$(check_p_subdirs))
 $(check_DEJAGNU_normal_targets): check-DEJAGNUnormal%: normal%/site.exp
 
 # Run the testsuite in normal mode.
 check-DEJAGNU $(check_DEJAGNU_normal_targets): check-DEJAGNU%: site.exp
 	AR="$(AR)"; export AR; \
 	RANLIB="$(RANLIB)"; export RANLIB; \
-	if [ -z "$*$(filter-out --target_board=%, $(RUNTESTFLAGS))" ] \
-	    && [ "$(filter -j, $(MFLAGS))" = "-j" ]; then \
+	if [ -z "$*" ] && [ "$(filter -j, $(MFLAGS))" = "-j" ]; then \
+	  rm -rf normal-parallel || true; \
+	  mkdir normal-parallel; \
 	  $(MAKE) $(AM_MAKEFLAGS) $(check_DEJAGNU_normal_targets); \
-	  for idx in 0 1 2 3 4 5 6 7 8 9 10; do \
-	    mv -f normal$$idx/libstdc++.sum normal$$idx/libstdc++.sum.sep; \
-	    mv -f normal$$idx/libstdc++.log normal$$idx/libstdc++.log.sep; \
+	  rm -rf normal-parallel || true; \
+	  for idx in $(check_p_subdirs); do \
+	    if [ -d normal$$idx ]; then \
+	      mv -f normal$$idx/libstdc++.sum normal$$idx/libstdc++.sum.sep; \
+	      mv -f normal$$idx/libstdc++.log normal$$idx/libstdc++.log.sep; \
+	    fi; \
 	  done; \
-	  mv -f libstdc++.sum libstdc++.sum.sep; \
-	  mv -f libstdc++.log libstdc++.log.sep; \
 	  $(SHELL) $(srcdir)/../../contrib/dg-extract-results.sh \
-	    libstdc++.sum.sep normal[0-9]*/libstdc++.sum.sep > libstdc++.sum; \
+	    normal[0-9]*/libstdc++.sum.sep > libstdc++.sum; \
 	  $(SHELL) $(srcdir)/../../contrib/dg-extract-results.sh -L \
-	    libstdc++.log.sep normal[0-9]*/libstdc++.log.sep > libstdc++.log; \
+	    normal[0-9]*/libstdc++.log.sep > libstdc++.log; \
 	  exit 0; \
 	fi; \
 	srcdir=`$(am__cd) $(srcdir) && pwd`; export srcdir; \
@@ -128,47 +139,20 @@  check-DEJAGNU $(check_DEJAGNU_normal_tar
 	runtest=$(RUNTEST); \
 	if [ -z "$$runtest" ]; then runtest=runtest; fi; \
 	tool=libstdc++; \
-	dirs=; \
-	case "$*" in \
-	  normal0) \
-	    if $(SHELL) -c "$$runtest --version" > /dev/null 2>&1; then \
-	      $$runtest $(AM_RUNTESTFLAGS) $(RUNTESTDEFAULTFLAGS) \
-			$(RUNTESTFLAGS) abi.exp prettyprinters.exp; \
-	    else echo "WARNING: could not find \`runtest'" 1>&2; :;\
-	    fi; \
-	    dirs="`cd $$srcdir; echo [013-9][0-9]_*/*`";; \
-	  normal1) \
-	    dirs="`cd $$srcdir; echo [ab]* de* [ep]*/*`";; \
-	  normal2) \
-	    dirs="`cd $$srcdir; echo 2[01]_*/*`";; \
-	  normal3) \
-	    dirs="`cd $$srcdir; echo 22_*/*`";; \
-	  normal4) \
-	    dirs="`cd $$srcdir; echo 23_*/[a-km-tw-z]*`";; \
-	  normal5) \
-	    dirs="`cd $$srcdir; echo 23_*/[luv]*`";; \
-	  normal6) \
-	    dirs="`cd $$srcdir; echo 2[459]_*/*`";; \
-	  normal7) \
-	    dirs="`cd $$srcdir; echo 26_*/* 28_*/[c-z]*`";; \
-	  normal8) \
-	    dirs="`cd $$srcdir; echo 27_*/*`";; \
-	  normal9) \
-	    dirs="`cd $$srcdir; echo 28_*/[ab]*`";; \
-	  normal10) \
-	    dirs="`cd $$srcdir; echo t*/*`";; \
-	esac; \
-	if [ -n "$*" ]; then cd "$*"; fi; \
+	if [ -n "$*" ]; then \
+	  if [ -f normal-parallel/finished ]; then rm -rf "$*"; exit 0; fi; \
+	  GCC_RUNTEST_PARALLELIZE_DIR=`${PWD_COMMAND}`/normal-parallel; \
+	  export GCC_RUNTEST_PARALLELIZE_DIR; \
+	  cd "$*"; \
+	fi; \
 	if $(SHELL) -c "$$runtest --version" > /dev/null 2>&1; then \
-	  if [ -n "$$dirs" ]; then \
-	    $$runtest $(AM_RUNTESTFLAGS) $(RUNTESTDEFAULTFLAGS) \
-		    $(RUNTESTFLAGS) \
-		    "conformance.exp=`echo $$dirs | sed 's/ /* /g;s/$$/*/'`"; \
-	  else \
-	    $$runtest $(AM_RUNTESTFLAGS) $(RUNTESTDEFAULTFLAGS) \
+	  $$runtest $(AM_RUNTESTFLAGS) $(RUNTESTDEFAULTFLAGS) \
 		    $(RUNTESTFLAGS); \
+	  if [ -n "$*" ]; then \
+	    touch $$GCC_RUNTEST_PARALLELIZE_DIR/finished; \
 	  fi; \
-	else echo "WARNING: could not find \`runtest'" 1>&2; :;\
+	else \
+	  echo "WARNING: could not find \`runtest'" 1>&2; :;\
 	fi
 
 check-am:
--- libstdc++-v3/testsuite/lib/libstdc++.exp.jj	2014-09-14 21:08:57.278535267 +0200
+++ libstdc++-v3/testsuite/lib/libstdc++.exp	2014-09-15 13:42:23.822366707 +0200
@@ -1823,3 +1823,78 @@  proc check_v3_target_little_endian { } {
 }
 
 set additional_prunes ""
+
+if { [info exists env(GCC_RUNTEST_PARALLELIZE_DIR)] \
+     && [info procs runtest_file_p] != [list] \
+     && [info procs gcc_parallelize_saved_runtest_file_p] == [list] } then {
+    global gcc_runtest_parallelize_counter
+    global gcc_runtest_parallelize_counter_minor
+    global gcc_runtest_parallelize_enable
+    global gcc_runtest_parallelize_dir
+    global gcc_runtest_parallelize_last
+
+    set gcc_runtest_parallelize_counter 0
+    set gcc_runtest_parallelize_counter_minor 0
+    set gcc_runtest_parallelize_enable 1
+    set gcc_runtest_parallelize_dir [getenv GCC_RUNTEST_PARALLELIZE_DIR]
+    set gcc_runtest_parallelize_last 0
+
+    proc gcc_parallel_test_run_p { testcase } {
+	global gcc_runtest_parallelize_counter
+	global gcc_runtest_parallelize_counter_minor
+	global gcc_runtest_parallelize_enable
+	global gcc_runtest_parallelize_dir
+	global gcc_runtest_parallelize_last
+
+	if { $gcc_runtest_parallelize_enable == 0 } {
+	    return 1
+	}
+
+	# Only test the filesystem every 10th iteration
+	incr gcc_runtest_parallelize_counter_minor
+	if { $gcc_runtest_parallelize_counter_minor == 10 } {
+	    set gcc_runtest_parallelize_counter_minor 0
+	}
+	if { $gcc_runtest_parallelize_counter_minor != 1 } {
+	    #verbose -log "gcc_parallel_test_run_p $testcase $gcc_runtest_parallelize_counter $gcc_runtest_parallelize_last"
+	    return $gcc_runtest_parallelize_last
+	}
+
+	set path $gcc_runtest_parallelize_dir/$gcc_runtest_parallelize_counter
+
+	if {![catch {open $path {RDWR CREAT EXCL} 0600} fd]} {
+	    close $fd
+	    set gcc_runtest_parallelize_last 1
+	    #verbose -log "gcc_parallel_test_run_p $testcase $gcc_runtest_parallelize_counter 1"
+	    incr gcc_runtest_parallelize_counter
+	    return 1
+	}
+	set gcc_runtest_parallelize_last 0
+	#verbose -log "gcc_parallel_test_run_p $testcase $gcc_runtest_parallelize_counter 0"
+	incr gcc_runtest_parallelize_counter
+	return 0
+    }
+
+    proc gcc_parallel_test_enable { val } {
+	global gcc_runtest_parallelize_enable
+	set gcc_runtest_parallelize_enable $val
+    }
+
+    rename runtest_file_p gcc_parallelize_saved_runtest_file_p
+    proc runtest_file_p { runtests testcase } {
+	if ![gcc_parallelize_saved_runtest_file_p $runtests $testcase] {
+	    return 0
+	}
+	return [gcc_parallel_test_run_p $testcase]
+    }
+
+} else {
+
+    proc gcc_parallel_test_run_p { testcase } {
+	return 1
+    }
+
+    proc gcc_parallel_test_enable { val } {
+    }
+
+}
--- libstdc++-v3/testsuite/Makefile.in.jj	2014-09-14 21:08:57.170535802 +0200
+++ libstdc++-v3/testsuite/Makefile.in	2014-09-15 15:29:57.323521363 +0200
@@ -301,7 +301,16 @@  lists_of_files = \
 
 extract_symvers = $(glibcxx_builddir)/scripts/extract_symvers
 baseline_subdir := $(shell $(CXX) $(baseline_subdir_switch))
-check_DEJAGNU_normal_targets = $(patsubst %,check-DEJAGNUnormal%,0 1 2 3 4 5 6 7 8 9 10)
+check_p_numbers0:=1 2 3 4 5 6 7 8 9
+check_p_numbers1:=0 $(check_p_numbers0)
+check_p_numbers2:=$(foreach i,$(check_p_numbers0),$(addprefix $(i),$(check_p_numbers1)))
+check_p_numbers3:=$(addprefix 0,$(check_p_numbers1)) $(check_p_numbers2)
+check_p_numbers4:=$(foreach i,$(check_p_numbers0),$(addprefix $(i),$(check_p_numbers3)))
+check_p_numbers5:=$(addprefix 0,$(check_p_numbers3)) $(check_p_numbers4)
+check_p_numbers6:=$(foreach i,$(check_p_numbers0),$(addprefix $(i),$(check_p_numbers5)))
+check_p_numbers:=$(check_p_numbers0) $(check_p_numbers2) $(check_p_numbers4) $(check_p_numbers6)
+check_p_subdirs=$(wordlist 1,$(if $(GCC_TEST_PARALLEL_SLOTS),$(GCC_TEST_PARALLEL_SLOTS),128),$(check_p_numbers))
+check_DEJAGNU_normal_targets = $(addprefix check-DEJAGNUnormal,$(check_p_subdirs))
 
 # Runs the testsuite, but in compile only mode.
 # Can be used to test sources with non-GNU FE's at various warning
@@ -559,19 +568,21 @@  $(check_DEJAGNU_normal_targets): check-D
 check-DEJAGNU $(check_DEJAGNU_normal_targets): check-DEJAGNU%: site.exp
 	AR="$(AR)"; export AR; \
 	RANLIB="$(RANLIB)"; export RANLIB; \
-	if [ -z "$*$(filter-out --target_board=%, $(RUNTESTFLAGS))" ] \
-	    && [ "$(filter -j, $(MFLAGS))" = "-j" ]; then \
+	if [ -z "$*" ] && [ "$(filter -j, $(MFLAGS))" = "-j" ]; then \
+	  rm -rf normal-parallel || true; \
+	  mkdir normal-parallel; \
 	  $(MAKE) $(AM_MAKEFLAGS) $(check_DEJAGNU_normal_targets); \
-	  for idx in 0 1 2 3 4 5 6 7 8 9 10; do \
-	    mv -f normal$$idx/libstdc++.sum normal$$idx/libstdc++.sum.sep; \
-	    mv -f normal$$idx/libstdc++.log normal$$idx/libstdc++.log.sep; \
+	  rm -rf normal-parallel || true; \
+	  for idx in $(check_p_subdirs); do \
+	    if [ -d normal$$idx ]; then \
+	      mv -f normal$$idx/libstdc++.sum normal$$idx/libstdc++.sum.sep; \
+	      mv -f normal$$idx/libstdc++.log normal$$idx/libstdc++.log.sep; \
+	    fi; \
 	  done; \
-	  mv -f libstdc++.sum libstdc++.sum.sep; \
-	  mv -f libstdc++.log libstdc++.log.sep; \
 	  $(SHELL) $(srcdir)/../../contrib/dg-extract-results.sh \
-	    libstdc++.sum.sep normal[0-9]*/libstdc++.sum.sep > libstdc++.sum; \
+	    normal[0-9]*/libstdc++.sum.sep > libstdc++.sum; \
 	  $(SHELL) $(srcdir)/../../contrib/dg-extract-results.sh -L \
-	    libstdc++.log.sep normal[0-9]*/libstdc++.log.sep > libstdc++.log; \
+	    normal[0-9]*/libstdc++.log.sep > libstdc++.log; \
 	  exit 0; \
 	fi; \
 	srcdir=`$(am__cd) $(srcdir) && pwd`; export srcdir; \
@@ -579,47 +590,20 @@  check-DEJAGNU $(check_DEJAGNU_normal_tar
 	runtest=$(RUNTEST); \
 	if [ -z "$$runtest" ]; then runtest=runtest; fi; \
 	tool=libstdc++; \
-	dirs=; \
-	case "$*" in \
-	  normal0) \
-	    if $(SHELL) -c "$$runtest --version" > /dev/null 2>&1; then \
-	      $$runtest $(AM_RUNTESTFLAGS) $(RUNTESTDEFAULTFLAGS) \
-			$(RUNTESTFLAGS) abi.exp prettyprinters.exp; \
-	    else echo "WARNING: could not find \`runtest'" 1>&2; :;\
-	    fi; \
-	    dirs="`cd $$srcdir; echo [013-9][0-9]_*/*`";; \
-	  normal1) \
-	    dirs="`cd $$srcdir; echo [ab]* de* [ep]*/*`";; \
-	  normal2) \
-	    dirs="`cd $$srcdir; echo 2[01]_*/*`";; \
-	  normal3) \
-	    dirs="`cd $$srcdir; echo 22_*/*`";; \
-	  normal4) \
-	    dirs="`cd $$srcdir; echo 23_*/[a-km-tw-z]*`";; \
-	  normal5) \
-	    dirs="`cd $$srcdir; echo 23_*/[luv]*`";; \
-	  normal6) \
-	    dirs="`cd $$srcdir; echo 2[459]_*/*`";; \
-	  normal7) \
-	    dirs="`cd $$srcdir; echo 26_*/* 28_*/[c-z]*`";; \
-	  normal8) \
-	    dirs="`cd $$srcdir; echo 27_*/*`";; \
-	  normal9) \
-	    dirs="`cd $$srcdir; echo 28_*/[ab]*`";; \
-	  normal10) \
-	    dirs="`cd $$srcdir; echo t*/*`";; \
-	esac; \
-	if [ -n "$*" ]; then cd "$*"; fi; \
+	if [ -n "$*" ]; then \
+	  if [ -f normal-parallel/finished ]; then rm -rf "$*"; exit 0; fi; \
+	  GCC_RUNTEST_PARALLELIZE_DIR=`${PWD_COMMAND}`/normal-parallel; \
+	  export GCC_RUNTEST_PARALLELIZE_DIR; \
+	  cd "$*"; \
+	fi; \
 	if $(SHELL) -c "$$runtest --version" > /dev/null 2>&1; then \
-	  if [ -n "$$dirs" ]; then \
-	    $$runtest $(AM_RUNTESTFLAGS) $(RUNTESTDEFAULTFLAGS) \
-		    $(RUNTESTFLAGS) \
-		    "conformance.exp=`echo $$dirs | sed 's/ /* /g;s/$$/*/'`"; \
-	  else \
-	    $$runtest $(AM_RUNTESTFLAGS) $(RUNTESTDEFAULTFLAGS) \
+	  $$runtest $(AM_RUNTESTFLAGS) $(RUNTESTDEFAULTFLAGS) \
 		    $(RUNTESTFLAGS); \
+	  if [ -n "$*" ]; then \
+	    touch $$GCC_RUNTEST_PARALLELIZE_DIR/finished; \
 	  fi; \
-	else echo "WARNING: could not find \`runtest'" 1>&2; :;\
+	else \
+	  echo "WARNING: could not find \`runtest'" 1>&2; :;\
 	fi
 
 check-am: