| Message ID | 5270E0CD.9040600@net-b.de |
|---|---|
| State | New |
| Headers | show |
Hello Everyone, What I ideally wanted to do with my testsuite files was that I want all the Cilk keywords test to compile no matter what the architecture is, but it should only run in certain architectures where the runtime is enabled (this is known statically and thus the testsuite doesn't have to do anything to figure it out.). Can someone please tell me how do I do this? Thanks, Balaji V. Iyer. > -----Original Message----- > From: Tobias Burnus [mailto:burnus@net-b.de] > Sent: Wednesday, October 30, 2013 6:35 AM > To: gcc patches; Iyer, Balaji V > Subject: Testsuite / Cilk Plus: Include library path in compile flags in gcc.dg/cilk- > plus/cilk-plus.exp > > Without that patch, which I have copied from asan-dg.exp, I get tons of failures > because "ld" cannot find libcilkrts. > > OK for committal? > > Tobias
On 10/30/13 09:09, Iyer, Balaji V wrote: > Hello Everyone, What I ideally wanted to do with my testsuite files > was that I want all the Cilk keywords test to compile no matter what > the architecture is, but it should only run in certain architectures > where the runtime is enabled (this is known statically and thus the > testsuite doesn't have to do anything to figure it out.). Can someone > please tell me how do I do this? I can't recall a similar situation off the top of my head. Are you using the dg framework? Can you have multiple dg-do directives? ie, /* { dg-do compile } */ /* { dg-do run { target i?86-*-* x86-64-*-* } } */ ? That'd be my best guess. jeff
> -----Original Message----- > From: Jeff Law [mailto:law@redhat.com] > Sent: Wednesday, October 30, 2013 1:08 PM > To: Iyer, Balaji V; Tobias Burnus; gcc patches > Subject: Re: Testsuite / Cilk Plus: Include library path in compile flags in > gcc.dg/cilk-plus/cilk-plus.exp > > On 10/30/13 09:09, Iyer, Balaji V wrote: > > Hello Everyone, What I ideally wanted to do with my testsuite files > > was that I want all the Cilk keywords test to compile no matter what > > the architecture is, but it should only run in certain architectures > > where the runtime is enabled (this is known statically and thus the > > testsuite doesn't have to do anything to figure it out.). Can someone > > please tell me how do I do this? > > I can't recall a similar situation off the top of my head. Are you using the dg > framework? Can you have multiple dg-do directives? > > ie, > /* { dg-do compile } */ > /* { dg-do run { target i?86-*-* x86-64-*-* } } */ > Yes, I am using the dg-framework. I tried that couple months back, and from what it looked like, it was replacing the compile command with the run command, and then on non-x86 architectures, it was just ignoring the file... But, will try it again to see if it works. > ? > > That'd be my best guess. > > jeff
On Wed, 30 Oct 2013, Jeff Law wrote: > /* { dg-do compile } */ > /* { dg-do run { target i?86-*-* x86-64-*-* } } */ But with an effective-target keyword cilkplusrts or similar, rather than hardcoding the same list of targets in lots of places, please.
> -----Original Message----- > From: Joseph Myers [mailto:joseph@codesourcery.com] > Sent: Wednesday, October 30, 2013 1:15 PM > To: Jeff Law > Cc: Iyer, Balaji V; Tobias Burnus; gcc patches > Subject: Re: Testsuite / Cilk Plus: Include library path in compile flags in > gcc.dg/cilk-plus/cilk-plus.exp > > On Wed, 30 Oct 2013, Jeff Law wrote: > > > /* { dg-do compile } */ > > /* { dg-do run { target i?86-*-* x86-64-*-* } } */ > > But with an effective-target keyword cilkplusrts or similar, rather than > hardcoding the same list of targets in lots of places, please. > Wow, I didn't know you could do that. Do you have an example where it is done that I can model after? > -- > Joseph S. Myers > joseph@codesourcery.com
On 10/30/13 11:12, Iyer, Balaji V wrote: >> -----Original Message----- >> From: Jeff Law [mailto:law@redhat.com] >> Sent: Wednesday, October 30, 2013 1:08 PM >> To: Iyer, Balaji V; Tobias Burnus; gcc patches >> Subject: Re: Testsuite / Cilk Plus: Include library path in compile flags in >> gcc.dg/cilk-plus/cilk-plus.exp >> >> On 10/30/13 09:09, Iyer, Balaji V wrote: >>> Hello Everyone, What I ideally wanted to do with my testsuite files >>> was that I want all the Cilk keywords test to compile no matter what >>> the architecture is, but it should only run in certain architectures >>> where the runtime is enabled (this is known statically and thus the >>> testsuite doesn't have to do anything to figure it out.). Can someone >>> please tell me how do I do this? >> >> I can't recall a similar situation off the top of my head. Are you using the dg >> framework? Can you have multiple dg-do directives? >> >> ie, >> /* { dg-do compile } */ >> /* { dg-do run { target i?86-*-* x86-64-*-* } } */ >> > > Yes, I am using the dg-framework. > > I tried that couple months back, and from what it looked like, it was replacing the compile command with the run command, and then on non-x86 architectures, it was just ignoring the file... > > But, will try it again to see if it works. Hmm, if that's the case, you may need distinct tests. Judicious use of #include files would avoid unnecessary duplication. jeff
On 10/30/13 11:16, Iyer, Balaji V wrote: > > >> -----Original Message----- >> From: Joseph Myers [mailto:joseph@codesourcery.com] >> Sent: Wednesday, October 30, 2013 1:15 PM >> To: Jeff Law >> Cc: Iyer, Balaji V; Tobias Burnus; gcc patches >> Subject: Re: Testsuite / Cilk Plus: Include library path in compile flags in >> gcc.dg/cilk-plus/cilk-plus.exp >> >> On Wed, 30 Oct 2013, Jeff Law wrote: >> >>> /* { dg-do compile } */ >>> /* { dg-do run { target i?86-*-* x86-64-*-* } } */ >> >> But with an effective-target keyword cilkplusrts or similar, rather than >> hardcoding the same list of targets in lots of places, please. >> > > Wow, I didn't know you could do that. Do you have an example where it is done that I can model after? Look at testsuite/lib/target-support.exp for check_*. Once you see one or two, you can search in testsuite/gcc.dg for examples. jeff
Iyer, Balaji V wrote: > What I ideally wanted to do with my testsuite files was that I want > all the Cilk keywords test to compile no matter what the architecture > is, but it should only run in certain architectures where the runtime > is enabled (this is known statically and thus the testsuite doesn't > have to do anything to figure it out.). Can someone please tell me how > do I do this? Which is a bit orthogonal to my patch, which helps that the tests pass on systems which are supported. – Thus, let's start with the question what do you think of that patch? On the other hand, one could use the existence of libcilkrts* as detected by the patch to decide whether to link or not: If the library is there, one can link – if not found, it is unlikely to work (unless it is, e.g. found in /usr/lib). * * * Regarding compile vs. run: I think one possibility would be to have no "dg-do" in the files and simply change the default to compile or run, depending whether the architecture is supported. On the other hand, that can be confusing as an explicit "dg-do run" will break it on some systems. * * * Actually, I was wondering whether -fcilkplus should always automatically link libcilkrts – akin to -fopenmp which links libgomp. Currently, one has to specify it manually.* Or are the features which do not need libcilkplus common enough that one doesn't always want to link it? [For OpenMP, GCC will have -fopenmp-simd [patch posted, 1], which doesn't link libgomp. I could imagine that one would like to have Cilk Plus' "#pragma simd" and the array syntax without enabling threads (cilk_sync, cilk_spawn, cilk_for, reducers – and thus libcilkrts linkage).] Tobias * libgomp is handled via spec files – including one which adds "librt" when libgomp is linked. [1] -fopenmp-simd: http://gcc.gnu.org/ml/gcc-patches/2013-10/msg02275.html
> * * * > > Actually, I was wondering whether -fcilkplus should always automatically link > libcilkrts - akin to -fopenmp which links libgomp. Currently, one has to specify it > manually.* Yes, I would like that to happen. Do you have any pointers about how to do that?
Iyer, Balaji V wrote: >> * * * >> >> Actually, I was wondering whether -fcilkplus should always automatically link >> libcilkrts - akin to -fopenmp which links libgomp. Currently, one has to specify it >> manually.* > Yes, I would like that to happen. Do you have any pointers about how to do that? Well, if you need some special libraries like librt, see: libgomp/libgomp.spec.in libgomp/configure.ac (search for "link_gomp"). Otherwise, simply have a look at gcc/gcc.c – search for "fopenmp" (which adds -pthread). Tobias
On 10/30/13 13:27, Iyer, Balaji V wrote: >> * * * >> >> Actually, I was wondering whether -fcilkplus should always automatically link >> libcilkrts - akin to -fopenmp which links libgomp. Currently, one has to specify it >> manually.* > > Yes, I would like that to happen. Do you have any pointers about how to do that? LINK_SPEC and its friends. jeff
On 10/30/13 04:34, Tobias Burnus wrote: > Without that patch, which I have copied from asan-dg.exp, I get tons of > failures because "ld" cannot find libcilkrts. > > OK for committal? > > Tobias > > cilk.diff > > > 2013-10-30 Tobias Burnus<burnus@net-b.de> > > * gcc.dg/cilk-plus/cilk-plus.exp: Add the libcilkrts library > path to the compile flags. Yea, seems good to me. g++.dg/cilk-plus/cilk-plus.exp may have the same problem (haven't looked). jeff
On Wed, 30 Oct 2013, Tobias Burnus wrote: > On the other hand, one could use the existence of libcilkrts* as detected by > the patch to decide whether to link or not: If the library is there, one can > link – if not found, it is unlikely to work (unless it is, e.g. found in > /usr/lib). The way to detect it is to try to link with it, rather than just looking for it in the build tree. If you just look for it in the build tree and assume it's missing if not found there, you break installed testing. See Andrew's recent patch fixing this for asan/ubsan testing (or what I did for tests linking with libatomic on the C11-atomic branch).
Hi! On 30 Oct 2013, at 20:55, Joseph S. Myers wrote: > On Wed, 30 Oct 2013, Tobias Burnus wrote: > >> On the other hand, one could use the existence of libcilkrts* as detected by >> the patch to decide whether to link or not: If the library is there, one can >> link – if not found, it is unlikely to work (unless it is, e.g. found in >> /usr/lib). > > The way to detect it is to try to link with it, rather than just looking > for it in the build tree. If you just look for it in the build tree and > assume it's missing if not found there, you break installed testing. See > Andrew's recent patch fixing this for asan/ubsan testing (or what I did > for tests linking with libatomic on the C11-atomic branch). In addition to other posters, a few other observations. 1). the c execute tests all fail on Darwin (because nothing is calling set_ld_library_path_env_vars). Note that this could cause particularly subtle issues, if one builds consecutive versions of the compiler with the same install path (and install the compiler after testing). Since the new compiler could end up testing the previously installed shlib [at least on Darwin]. A minimal fix for the execute tests (in-tree testing) is to add set_ld_library_path_env_vars after the ld_library_path is decided (although this does not fix installed testing, of course). 2). The C tests are extremely time-consuming (m32/m64) with NO -jxx on the c/l: x86_64-linux (32 cores 2.8G Xeon) : 36m/421m/6m x86_64-darwin12 (8 cores 2.8G Xeon) : 45m/309m/2m. c.f. the g++ tests which take ~2m wall clock. this might be nothing to do with cilk, but it certainly warrants investigation. 3). The "torture" mixes you have are not obvious (at least to me): * there are a lot of tests where you do "-O0" and "" * also you test "-O2 -ftree-vectorize" (but not -O2) * but -O3 with / without -ftree-vectorize. * you test a few LTO combinations for C but none for C++ * you test C with/without -std=c99. * but don't test C++ with -std=c++98 and -std=c++11. There might be perfectly good reasons for these; but, as I say, they are not obvious ( at least to me ;) ) and probably warrant a comment in the .exp as to the rationale for the choices. ======== I'm attaching three .exp files which might form the basis of an alternate implementation of the testing - they fix the execute tests on darwin and work for installed testing too. However, given that Q2 and Q3 above might involve some changes, there are probably additional modifications needed. in the new .exps, the c++ tests do -std=c++98 and -std=c++11, but otherwise do the same torture mix as you have, so would also need adjustment once the questions are resolved. cilk-dg.exp => gcc/testsuite/lib/cilk-dg.exp cilk-plus.exp => gcc/testsuite/gcc.dg/cilk-plus/cilk-plus.exp cxx-cilk-plus.exp => gcc/testsuite/g++.dg/cilk-plus/cilk-plus.exp rename / use / ignore - as appropriate ;) cheers, Iain
Hi Iain,
I have prefixed all my replies below with "BVI:"
Thanks,
Balaji V. Iyer.
From: Iain Sandoe [mailto:iain@codesourcery.com]
Sent: Sunday, November 3, 2013 11:01 AM
To: Iyer, Balaji V
Cc: Joseph S. Myers; Tobias Burnus; gcc patches
Subject: Re: Testsuite / Cilk Plus: Include library path in compile flags in gcc.dg/cilk-plus/cilk-plus.exp
Hi!
On 30 Oct 2013, at 20:55, Joseph S. Myers wrote:
> On Wed, 30 Oct 2013, Tobias Burnus wrote:
>
>> On the other hand, one could use the existence of libcilkrts* as detected by
>> the patch to decide whether to link or not: If the library is there, one can
>> link - if not found, it is unlikely to work (unless it is, e.g. found in
>> /usr/lib).
>
> The way to detect it is to try to link with it, rather than just looking
> for it in the build tree. If you just look for it in the build tree and
> assume it's missing if not found there, you break installed testing. See
> Andrew's recent patch fixing this for asan/ubsan testing (or what I did
> for tests linking with libatomic on the C11-atomic branch).
In addition to other posters, a few other observations.
1). the c execute tests all fail on Darwin (because nothing is calling set_ld_library_path_env_vars).
Note that this could cause particularly subtle issues, if one builds consecutive versions of the compiler with the same install path (and install the compiler after testing). Since the new compiler could end up testing the previously installed shlib [at least on Darwin].
A minimal fix for the execute tests (in-tree testing) is to add set_ld_library_path_env_vars after the ld_library_path is decided (although this does not fix installed testing, of course).
BVI: Thanks for letting me know about this. I will try to test this in
2). The C tests are extremely time-consuming (m32/m64) with NO -jxx on the c/l:
x86_64-linux (32 cores 2.8G Xeon) : 36m/421m/6m
x86_64-darwin12 (8 cores 2.8G Xeon) : 45m/309m/2m.
c.f. the g++ tests which take ~2m wall clock.
BVI: yes, I know about this. It is because some of the Cilk tests are run for higher iterations to make it force a steal. This is one of the main place were runtime bugs could occur. My original solution was to put them all in separate directories with separate scripts to run them, but someone in GCC community did not like that idea. This way, when people compile with -jN, they will be run in parallel. If that is a good idea, I can look into reorganizing them.
this might be nothing to do with cilk, but it certainly warrants investigation.
3). The "torture" mixes you have are not obvious (at least to me):
* there are a lot of tests where you do "-O0" and ""
* also you test "-O2 -ftree-vectorize" (but not -O2)
* but -O3 with / without -ftree-vectorize.
* you test a few LTO combinations for C but none for C++
* you test C with/without -std=c99.
* but don't test C++ with -std=c++98 and -std=c++11.
There might be perfectly good reasons for these; but, as I say, they are not obvious ( at least to me ;) ) and probably warrant a comment in the .exp as to the rationale for the choices.
BVI: Thanks for looking into this for me. Originally (when I had submitted Array Notations), I had used the torture frame work. Someone in GCC community asked me not to use them. I am testing them using several flags because these are the flags many of our branch users have reported using. I will be happy to add a comment in .exp file stating so.
========
I'm attaching three .exp files which might form the basis of an alternate implementation of the testing - they fix the execute tests on darwin and work for installed testing too.
However, given that Q2 and Q3 above might involve some changes, there are probably additional modifications needed.
in the new .exps, the c++ tests do -std=c++98 and -std=c++11,
but otherwise do the same torture mix as you have, so would also need adjustment once the questions are resolved.
BVI: I have tests that use C++11 specific code (e.g. lamba functions) and I have called these flags inside the code using /* dg-options */
cilk-dg.exp => gcc/testsuite/lib/cilk-dg.exp
cilk-plus.exp => gcc/testsuite/gcc.dg/cilk-plus/cilk-plus.exp
cxx-cilk-plus.exp => gcc/testsuite/g++.dg/cilk-plus/cilk-plus.exp
rename / use / ignore - as appropriate ;)
BVI: Thanks again for working on this.
cheers,
Iain
hi Balaji sorry for a short reply, but I'm traveling this week, On 4 Nov 2013, at 23:02, Iyer, Balaji V wrote: > 2). The C tests are extremely time-consuming (m32/m64) with NO -jxx on the c/l: > > x86_64-linux (32 cores 2.8G Xeon) : 36m/421m/6m > x86_64-darwin12 (8 cores 2.8G Xeon) : 45m/309m/2m. > > c.f. the g++ tests which take ~2m wall clock. > > BVI: yes, I know about this. It is because some of the Cilk tests are run for higher iterations to make it force a steal. This is one of the main place were runtime bugs could occur. My original solution was to put them all in separate directories with separate scripts to run them, but someone in GCC community did not like that idea. This way, when people compile with -jN, they will be run in parallel. If that is a good idea, I can look into reorganising them. total CPU time consumed is relevant - folks can already get parallel testing (different suites for different languages will run in parallel). However, GCC is tested on a wide variety of hardware, e.g. even I use from core duo - 32core x86_64. There are facilities to identify tests as "expensive" - and testing that might add hundreds of minutes to the cycle would come into that category (at least, IMO). There should be some way to "opt in" if the tests are specifically of interest. sorry again for brevity, Iain
Hi! On Mon, Nov 04, 2013 at 11:02:13PM +0000, Iyer, Balaji V wrote: See my other mail on this topic, the number of different option combinations is just excessive and must be reduced to a reasonable minimum. > 2). The C tests are extremely time-consuming (m32/m64) with NO -jxx on the c/l: > > x86_64-linux (32 cores 2.8G Xeon) : 36m/421m/6m > x86_64-darwin12 (8 cores 2.8G Xeon) : 45m/309m/2m. > > c.f. the g++ tests which take ~2m wall clock. > > BVI: yes, I know about this. It is because some of the Cilk tests are run for higher iterations to make it force a steal. This is one of the main place were runtime bugs could occur. My original solution was to put them all in separate directories with separate scripts to run them, but someone in GCC community did not like that idea. This way, when people compile with -jN, they will be run in parallel. If that is a good idea, I can look into reorganizing them. > > this might be nothing to do with cilk, but it certainly warrants investigation. Tests that many seconds or more on fast machines, especially if you run them 25 times, are simply not appropriate for gcc testsuite, at least not by default. We have run_expensive_tests (from GCC_RUN_EXPENSIVE_TESTS in environment), which can be used say for: /* { dg-additional-options "-DEXPENSIVE" { target run_expensive_tests } } */ or // { dg-additional-options "-DASAN_AVOID_EXPENSIVE_TESTS=1" { target { ! run_expensive_tests } } } or /* { dg-skip-if "" { ! run_expensive_tests } { "*" } { "-O2" } } */ (just grep for it), but still it shouldn't be prohibitively slow. Remember what is slow on fast machines might turn into days on really slow machines. Say, if all you are looking for is look for library synchronization issues, I guess all optimization levels still result in similar pattern of library calls, so even for run_expensive_tests you could use higher iteration count for a single optimization level (say -O2) and for all others just use smaller iteration count. Jakub
On Tue, Nov 05, 2013 at 12:21:04PM +0100, Jakub Jelinek wrote: > Tests that many seconds or more on fast machines, especially if you run them > 25 times, are simply not appropriate for gcc testsuite, at least not by > default. We have run_expensive_tests (from GCC_RUN_EXPENSIVE_TESTS in > environment), which can be used say for: > /* { dg-additional-options "-DEXPENSIVE" { target run_expensive_tests } } */ > or > // { dg-additional-options "-DASAN_AVOID_EXPENSIVE_TESTS=1" { target { ! run_expensive_tests } } } > or > /* { dg-skip-if "" { ! run_expensive_tests } { "*" } { "-O2" } } */ > (just grep for it), but still it shouldn't be prohibitively slow. > Remember what is slow on fast machines might turn into days on really slow > machines. Say, if all you are looking for is look for library > synchronization issues, I guess all optimization levels still result in > similar pattern of library calls, so even for run_expensive_tests you could > use higher iteration count for a single optimization level (say -O2) > and for all others just use smaller iteration count. Oh, another thing, runtime tests should abort () or __builtin_abort () on failure, rather then just exit with non-zero status. Also, I wonder about the fib_* tests, fib_serial calls fib rather than fib_serial, so effectively the only difference between calling fib_serial and fib is just the outermost iteration. Also, are you really sure you have to call fib with all values from 0 up to 40? Isn't it enough to just call fib (40) once and compare that with precomputed fib (40) number? Because, when a single test takes about 2 minutes to run, it is really too much, and very much unnecessarily so. Jakub
2013-10-30 Tobias Burnus <burnus@net-b.de> * gcc.dg/cilk-plus/cilk-plus.exp: Add the libcilkrts library path to the compile flags. diff --git a/gcc/testsuite/gcc.dg/cilk-plus/cilk-plus.exp b/gcc/testsuite/gcc.dg/cilk-plus/cilk-plus.exp index a8f00d4..0a9d19b 100644 --- a/gcc/testsuite/gcc.dg/cilk-plus/cilk-plus.exp +++ b/gcc/testsuite/gcc.dg/cilk-plus/cilk-plus.exp @@ -26,7 +26,24 @@ if { ![check_effective_target_cilkplus] } { verbose "$tool $libdir" 1 set library_var [get_multilibs] # Pointing the ld_library_path to the Cilk Runtime library binaries. -set ld_library_path "${library_var}/libcilkrts/.libs" +if { $gccpath != "" } { + if { [file exists "${gccpath}/libcilkrts/.libs/libcilkrts.a"] + || [file exists "${gccpath}/libcilkrts/.libs/libcilkrts.${shlib_ext}"] } { + append flags " -B${gccpath}/libcilkrts/ " + append flags " -L${gccpath}/libcilkrts/.libs " + append ld_library_path ":${gccpath}/libcilkrts/.libs" + } +} else { + global tool_root_dir + + set libcilkrts [lookfor_file ${tool_root_dir} libcilkrts] + if { $libcilkrts != "" } { + append flags "-L${libcilkrts} " + append ld_library_path ":${libcilkrts}" + } +} + +set_ld_library_path_env_vars dg-init dg-runtest [lsort [glob -nocomplain $srcdir/c-c++-common/cilk-plus/AN/*.c]] " -fcilkplus" " "