Patchwork [testsuite,ada,build] Allow for further differences in type output in run_acats

login
register
mail settings
Submitter Rainer Orth
Date Dec. 1, 2010, 2:44 p.m.
Message ID <yddipzd1riv.fsf@manam.CeBiTec.Uni-Bielefeld.DE>
Download mbox | patch
Permalink /patch/73823/
State New
Headers show

Comments

Rainer Orth - Dec. 1, 2010, 2:44 p.m.
I've just noticed that the ACATS tests weren't run on IRIX 6.5 with
/bin/ksh as CONFIG_SHELL.  It turned out that the which() shell function
in ada/acats/run_acats still isn't robust enough.  type -p gnatchop
would e.g. print

gnatchop is a tracked alias for /vol/gcc-4.4/bin/gnatchop

under some circumstances, and /bin/ksh on Tru64 UNIX V5.1B gives

gnatchop is /vol/gcc-4.4/bin/gnatchop

despite the -p.  This is all a total mess and calls for moving ACATS to
Dejagnu instead.  I really hope to tackle this for 4.7.

For the time being, I've made the following changes to make which() more
robust:

* Save the output of type -p/type/whence in a variable and only
  emit/process it if successful, otherwise we use the awk exit code, not
  that from type et al.

* I'm no longer relying on a particular type etc. output format, but
  instead just extract the last output field with awk.

I've been able run make check-acats on mips-sgi-irix6.5 with a previous
version of this patch, and the final one below has been tested with the
following script

#!/bin/sh
which_old () {
    type -p $* 2>/dev/null && return 0
    type $* 2>/dev/null | awk '{print $3}' && return 0
    whence $* 2>/dev/null && return 0
    return 1
}

which_new () {
    path=`type -p $* 2>/dev/null` && { echo $path | awk '{print $NF}'; return 0; }
    path=`type $* 2>/dev/null` && { echo $path | awk '{print $NF}'; return 0; }
    path=`whence $* 2>/dev/null` && { echo $path; return 0; }
    return 1
}

old_gnatchop=`which_old gnatchop`
old_gnatdir=`dirname $old_gnatchop`

echo "old:
$old_gnatchop
$old_gnatdir"

new_gnatchop=`which_new gnatchop`
new_gnatdir=`dirname $new_gnatchop`
echo "new:
$new_gnatchop
$new_gnatdir"

with /bin/sh, /bin/ksh and bash on mips-sgi-irix5.3, mips-sgi-irix6.5,
alpha-dec-osf4.0f, alpha-dec-osf5.1b, sparc-sun-solaris2.8,
sparc-sun-solaris2.10 and sparc-sun-solaris2.11.

Ok for mainline and the 4.4, 4.5 branches after testing/soaktime on
mainline?

Thanks.
        Rainer


2010-11-30  Rainer Orth  <ro@CeBiTec.Uni-Bielefeld.DE>

	gcc/testsuite:
	* ada/acats/run_acats (which): Assign output to temporary
	variable, only use if successful.
	Use last field of type output.
Paolo Bonzini - Dec. 1, 2010, 2:47 p.m.
On 12/01/2010 03:44 PM, Rainer Orth wrote:
> diff -r 48851c3efb44 gcc/testsuite/ada/acats/run_acats
> --- a/gcc/testsuite/ada/acats/run_acats	Fri Nov 26 17:38:20 2010 +0000
> +++ b/gcc/testsuite/ada/acats/run_acats	Wed Dec 01 15:34:05 2010 +0100
> @@ -14,9 +14,9 @@
>   # Fall back to whence which ksh88 and ksh93 provide, but bash does not.
>
>   which () {
> -    type -p $* 2>/dev/null&&  return 0
> -    type $* 2>/dev/null | awk '{print $3}'&&  return 0
> -    whence $* 2>/dev/null&&  return 0
> +    path=`type -p $* 2>/dev/null`&&  { echo $path | awk '{print $NF}'; return 0; }
> +    path=`type $* 2>/dev/null`&&  { echo $path | awk '{print $NF}'; return 0; }
> +    path=`whence $* 2>/dev/null`&&  { echo $path; return 0; }
>       return 1
>   }
>

Not entirely my field, but okay if no one complains in 48 hours.

Paolo
Arnaud Charlet - Dec. 1, 2010, 3 p.m.
> Ok for mainline and the 4.4, 4.5 branches after testing/soaktime on
> mainline?

Looks OK to me.
Ralf Wildenhues - Dec. 2, 2010, 7:39 a.m.
* Rainer Orth wrote on Wed, Dec 01, 2010 at 03:44:56PM CET:
> I've just noticed that the ACATS tests weren't run on IRIX 6.5 with
> /bin/ksh as CONFIG_SHELL.  It turned out that the which() shell function
> in ada/acats/run_acats still isn't robust enough.  type -p gnatchop
> would e.g. print

Why don't you simply use the code which Autoconf provides for
AC_CHECK_PROG or AC_PATH_PROG?  Portable, stable, time-tested.

Cheers,
Ralf
Rainer Orth - Dec. 2, 2010, 6:35 p.m.
Ralf Wildenhues <Ralf.Wildenhues@gmx.de> writes:

> * Rainer Orth wrote on Wed, Dec 01, 2010 at 03:44:56PM CET:
>> I've just noticed that the ACATS tests weren't run on IRIX 6.5 with
>> /bin/ksh as CONFIG_SHELL.  It turned out that the which() shell function
>> in ada/acats/run_acats still isn't robust enough.  type -p gnatchop
>> would e.g. print
>
> Why don't you simply use the code which Autoconf provides for
> AC_CHECK_PROG or AC_PATH_PROG?  Portable, stable, time-tested.

You'd probably have to ask the author of the original shell-based ACATS
testsuite.  IMO the whole thing is so riddled with problems (both
portability and functionality, like no multilib support and copying the
whole testsuite into the build tree for every parallel invocation) that
any time spent improving it in its current form is wasted ;-(

I see absolutely no reason to have a non-Dejagnu based testsuite in
tree; this only creates integration problems of all sorts.  Given that
we have the gnat.dg testsuite now, it shouldn't be too hard to extend
that to ACATS.

	Rainer
Laurent GUERBY - Dec. 13, 2010, 5:33 p.m.
On Thu, 2010-12-02 at 19:35 +0100, Rainer Orth wrote:
> Ralf Wildenhues <Ralf.Wildenhues@gmx.de> writes:
> 
> > * Rainer Orth wrote on Wed, Dec 01, 2010 at 03:44:56PM CET:
> >> I've just noticed that the ACATS tests weren't run on IRIX 6.5 with
> >> /bin/ksh as CONFIG_SHELL.  It turned out that the which() shell function
> >> in ada/acats/run_acats still isn't robust enough.  type -p gnatchop
> >> would e.g. print
> >
> > Why don't you simply use the code which Autoconf provides for
> > AC_CHECK_PROG or AC_PATH_PROG?  Portable, stable, time-tested.
> 
> You'd probably have to ask the author of the original shell-based ACATS
> testsuite.  IMO the whole thing is so riddled with problems (both
> portability and functionality, like no multilib support and copying the
> whole testsuite into the build tree for every parallel invocation) that
> any time spent improving it in its current form is wasted ;-(
> 
> I see absolutely no reason to have a non-Dejagnu based testsuite in
> tree; this only creates integration problems of all sorts.  Given that
> we have the gnat.dg testsuite now, it shouldn't be too hard to extend
> that to ACATS.

Hi,

That was my first look but since no one ever responded to my dejagnu/GCC
questions at the time I ended up writing a simple shell script to run
ACATS, I'm happy to see dejagnu expert stepping in to fix this :). 

The only specific thing about ACATS is that it has support files that
need to be generated and some source compiled before compiling and
running any ACATS test. The ACATS test execution output is very regular
so it should be easy to determine test status as is done in the shell
script. There a few tests that need specific compiler flags but they're
also all explicitely listed in the script.

Sincerely,

Laurent
Rainer Orth - Dec. 13, 2010, 7:11 p.m.
Hi Laurent,

> That was my first look but since no one ever responded to my dejagnu/GCC
> questions at the time I ended up writing a simple shell script to run
> ACATS, I'm happy to see dejagnu expert stepping in to fix this :). 

I'm far from a DejaGnu expert (hardly anyone is, it seems, but perhaps
Janis will return soon to fill the void :-), but plan to look at a
couple of issues anyway, and the hard lifting has already been done with
the gnat.dg testsuite.

> The only specific thing about ACATS is that it has support files that
> need to be generated and some source compiled before compiling and
> running any ACATS test. The ACATS test execution output is very regular

Indeed, and I plan to look into doing this once from make instead of
several times for each parallel check-acats sub-run.  It seems like a
total waste of time to do this more than once per multilib.

> so it should be easy to determine test status as is done in the shell
> script. There a few tests that need specific compiler flags but they're
> also all explicitely listed in the script.

While this could be done with dg-* directives, I'm currently relucatant
to change the ACATS sources at all, so I'll probably stay with the *.lst
files for that purpose.

	Rainer
Laurent GUERBY - Dec. 13, 2010, 7:27 p.m.
On Mon, 2010-12-13 at 20:11 +0100, Rainer Orth wrote:
> While this could be done with dg-* directives, I'm currently relucatant
> to change the ACATS sources at all, so I'll probably stay with the *.lst
> files for that purpose.

Some ACATS source files are already modified, mainly to change the
hardcoded delay statement values in tasking tests (which were typically
in minutes ...) to something tweakable globally.

Adding dejagnu directives to ACATS sources is really not an issue
if you feel it's the right technical solution.

Also I haven't followed upstream ACATS changes in a while, I'll open
a PR for this next year if no ones beats me to it.

Laurent
Joseph S. Myers - Dec. 13, 2010, 10:05 p.m.
On Mon, 13 Dec 2010, Rainer Orth wrote:

> > The only specific thing about ACATS is that it has support files that
> > need to be generated and some source compiled before compiling and
> > running any ACATS test. The ACATS test execution output is very regular
> 
> Indeed, and I plan to look into doing this once from make instead of
> several times for each parallel check-acats sub-run.  It seems like a
> total waste of time to do this more than once per multilib.

Building testsuite support files from make is a bad idea; a key advantage 
of using DejaGnu is that you can test an installed toolchain by creating a 
suitable site.exp and using "runtest --tool whatever" without needing a 
build tree for a particular toolchain component at all, and building 
support files other than from the .exp files causes problems for that.
Rainer Orth - Dec. 14, 2010, 6:08 p.m.
Laurent GUERBY <laurent@guerby.net> writes:

> On Mon, 2010-12-13 at 20:11 +0100, Rainer Orth wrote:
>> While this could be done with dg-* directives, I'm currently relucatant
>> to change the ACATS sources at all, so I'll probably stay with the *.lst
>> files for that purpose.
>
> Some ACATS source files are already modified, mainly to change the
> hardcoded delay statement values in tasking tests (which were typically
> in minutes ...) to something tweakable globally.
>
> Adding dejagnu directives to ACATS sources is really not an issue
> if you feel it's the right technical solution.

It's certainly easier to use the common mechanism than having to look up
each file in a separate file to determine if to add or emit specific
options or stuff like that.

> Also I haven't followed upstream ACATS changes in a while, I'll open
> a PR for this next year if no ones beats me to it.

Fine, thanks.

	Rainer
Rainer Orth - Dec. 14, 2010, 6:10 p.m.
"Joseph S. Myers" <joseph@codesourcery.com> writes:

> On Mon, 13 Dec 2010, Rainer Orth wrote:
>
>> > The only specific thing about ACATS is that it has support files that
>> > need to be generated and some source compiled before compiling and
>> > running any ACATS test. The ACATS test execution output is very regular
>> 
>> Indeed, and I plan to look into doing this once from make instead of
>> several times for each parallel check-acats sub-run.  It seems like a
>> total waste of time to do this more than once per multilib.
>
> Building testsuite support files from make is a bad idea; a key advantage 
> of using DejaGnu is that you can test an installed toolchain by creating a 
> suitable site.exp and using "runtest --tool whatever" without needing a 
> build tree for a particular toolchain component at all, and building 
> support files other than from the .exp files causes problems for that.

Ok, I see.  It still seems quite wasteful to build the support libs once
per parallel testsuite run instead of once per multilib, especially if
building them is expensive, as in libstdc++ and ACATS.  I expect this to
be doable once per multilib instead of the current variant.

	Rainer

Patch

diff -r 48851c3efb44 gcc/testsuite/ada/acats/run_acats
--- a/gcc/testsuite/ada/acats/run_acats	Fri Nov 26 17:38:20 2010 +0000
+++ b/gcc/testsuite/ada/acats/run_acats	Wed Dec 01 15:34:05 2010 +0100
@@ -14,9 +14,9 @@ 
 # Fall back to whence which ksh88 and ksh93 provide, but bash does not.
 
 which () {
-    type -p $* 2>/dev/null && return 0
-    type $* 2>/dev/null | awk '{print $3}' && return 0
-    whence $* 2>/dev/null && return 0
+    path=`type -p $* 2>/dev/null` && { echo $path | awk '{print $NF}'; return 0; }
+    path=`type $* 2>/dev/null` && { echo $path | awk '{print $NF}'; return 0; }
+    path=`whence $* 2>/dev/null` && { echo $path; return 0; }
     return 1
 }