[05/05] Add command-line option-parsing to jit testcases

Add command-line option-parsing to the testcases, so that we can
manipulate them without needing a recompile (e.g. varying
optimization levels etc).

This uses getopt_long, which is a GNU extension to libc.  Is that
acceptable?

Implement a --num-iterations option, to override the default of 5.

When running tests under valgrind, pass in "--num-iterations=1",
speeding up the tests by a factor of 5.

gcc/testsuite/ChangeLog:
	* jit.dg/harness.h (num_iterations): New variable.
        (parse_options): New function.
        (main): Use "num_iterations", rather than hardcoding 5.
	* jit.dg/jit.exp (fixed_host_execute): When running tests under
	valgrind, pass in "--num-iterations=1".
---
 gcc/testsuite/jit.dg/harness.h | 46 +++++++++++++++++++++++++++++++++++++++---
 gcc/testsuite/jit.dg/jit.exp   |  5 +++++
 2 files changed, 48 insertions(+), 3 deletions(-)

On Tue, 2014-11-25 at 20:34 -0500, David Malcolm wrote:
> Add command-line option-parsing to the testcases, so that we can
> manipulate them without needing a recompile (e.g. varying
> optimization levels etc).
> 
> This uses getopt_long, which is a GNU extension to libc.  Is that
> acceptable?

Ping.  Specifically, is it acceptable to use getopt_long within the jit
testcases, or should I find another way to do this (e.g. just getopt)?

Sorry that it wasn't obvious that I was asking for review on this one.

> Implement a --num-iterations option, to override the default of 5.
> 
> When running tests under valgrind, pass in "--num-iterations=1",
> speeding up the tests by a factor of 5.
> 
> gcc/testsuite/ChangeLog:
> 	* jit.dg/harness.h (num_iterations): New variable.
>         (parse_options): New function.
>         (main): Use "num_iterations", rather than hardcoding 5.
> 	* jit.dg/jit.exp (fixed_host_execute): When running tests under
> 	valgrind, pass in "--num-iterations=1".
> ---
>  gcc/testsuite/jit.dg/harness.h | 46 +++++++++++++++++++++++++++++++++++++++---
>  gcc/testsuite/jit.dg/jit.exp   |  5 +++++
>  2 files changed, 48 insertions(+), 3 deletions(-)
> 
> diff --git a/gcc/testsuite/jit.dg/harness.h b/gcc/testsuite/jit.dg/harness.h
> index bff64de..a30b66e 100644
> --- a/gcc/testsuite/jit.dg/harness.h
> +++ b/gcc/testsuite/jit.dg/harness.h
> @@ -247,17 +247,57 @@ extract_progname (const char *argv0)
>  }
>  
>  #ifndef TEST_PROVIDES_MAIN
> +
> +/* Option parsing, for varying how we run the built testcases
> +   (e.g. when poking at things in the debugger).  */
> +
> +#include <getopt.h>
> +int num_iterations = 5;
> +
> +static void
> +parse_options (int argc, char **argv)
> +{
> +  while (1)
> +    {
> +      static struct option long_options[] =
> +	{
> +	  {"num-iterations", required_argument, 0, 'i'},
> +	  {0, 0, 0, 0}
> +	};
> +
> +      int option_index = 0;
> +      /* getopt_long is a GNU extension to libc.  */
> +      int c = getopt_long (argc, argv, "i:",
> +			   long_options, &option_index);
> +      if (c == -1)
> +	break;
> +
> +      switch (c)
> +	{
> +	case 'i':
> +	  num_iterations = atoi (optarg);
> +	  break;
> +	default:
> +	  fprintf (stderr, "Usage: %s [--num-iterations NUM]\n",
> +		   argv[0]);
> +	  exit(EXIT_FAILURE);
> +	}
> +    }
> +}
> +
>  int
>  main (int argc, char **argv)
>  {
>    int i;
>  
> -  for (i = 1; i <= 5; i++)
> +  parse_options (argc, argv);
> +
> +  for (i = 1; i <= num_iterations; i++)
>      {
>        snprintf (test, sizeof (test),
>  		"%s iteration %d of %d",
> -                extract_progname (argv[0]),
> -                i, 5);
> +		extract_progname (argv[0]),
> +		i, num_iterations);
>  
>        //printf ("ITERATION %d\n", i);
>        test_jit (argv[0], NULL);
> diff --git a/gcc/testsuite/jit.dg/jit.exp b/gcc/testsuite/jit.dg/jit.exp
> index a37ccc7..438aabd 100644
> --- a/gcc/testsuite/jit.dg/jit.exp
> +++ b/gcc/testsuite/jit.dg/jit.exp
> @@ -157,6 +157,11 @@ proc fixed_host_execute {args} {
>  	set valgrind_params {"valgrind"}
>  	lappend valgrind_params "--leak-check=full"
>  	lappend valgrind_params "--log-file=${valgrind_logfile}"
> +	# When running under valgrind, speed things up by only running one
> +	# in-process iteration of the testcase, rather than the default of 5.
> +	# Only testcases that use the "main" from harness.h
> +	# (#ifndef TEST_PROVIDES_MAIN) will respond to this option.
> +	lappend params "--num-iterations=1"
>      } else {
>  	set valgrind_params {}
>      }

On Dec 8, 2014, at 12:44 PM, David Malcolm <dmalcolm@redhat.com> wrote:
> On Tue, 2014-11-25 at 20:34 -0500, David Malcolm wrote:
>> Add command-line option-parsing to the testcases, so that we can
>> manipulate them without needing a recompile (e.g. varying
>> optimization levels etc).
>> 
>> This uses getopt_long, which is a GNU extension to libc.  Is that
>> acceptable?
> 
> Ping.  Specifically, is it acceptable to use getopt_long within the jit
> testcases, or should I find another way to do this (e.g. just getopt)?

So, the standard by which we measure, does it kill building or testing of ada on mingwin?  If it does, then no, not acceptable.  I’d like to think there is nothing you can do in jit.exp that could do that.  So, from this perspective, yeah, feel free to do what you want.  Git it done first.  The second person that wants to port your code to a new machine (a different machine) will trip over all the bad things you did, and someone will then have to fix them.

If you only use what gcc already sues, you will be portable to everything gcc is portable to.  If you use GNU extensions to libc, well, that isn’t portable enough in general.  Heck, even what’s in libc isn’t portable enough, that’s half the reason why we have autoconf in the first place.

If jit is on by default everywhere, then you need to be portable everywhere.  If only on for linux, then well, it already has GNU extensions in libc.  I don’t know if it is on by default and you didn’t say, so, hard to comment on it.

On Mon, 2014-12-08 at 13:57 -0800, Mike Stump wrote:
> On Dec 8, 2014, at 12:44 PM, David Malcolm <dmalcolm@redhat.com>
> wrote:
> > On Tue, 2014-11-25 at 20:34 -0500, David Malcolm wrote:
> >> Add command-line option-parsing to the testcases, so that we can
> >> manipulate them without needing a recompile (e.g. varying
> >> optimization levels etc).
> >> 
> >> This uses getopt_long, which is a GNU extension to libc.  Is that
> >> acceptable?
> > 
> > Ping.  Specifically, is it acceptable to use getopt_long within the
> jit
> > testcases, or should I find another way to do this (e.g. just
> getopt)?
> 
> So, the standard by which we measure, does it kill building or testing
> of ada on mingwin?  If it does, then no, not acceptable.  I’d like to
> think there is nothing you can do in jit.exp that could do that.  So,
> from this perspective, yeah, feel free to do what you want.  Git it
> done first.  The second person that wants to port your code to a new
> machine (a different machine) will trip over all the bad things you
> did, and someone will then have to fix them.
> 
> If you only use what gcc already sues, you will be portable to
> everything gcc is portable to.  If you use GNU extensions to libc,
> well, that isn’t portable enough in general.  Heck, even what’s in
> libc isn’t portable enough, that’s half the reason why we have
> autoconf in the first place.
> 
> If jit is on by default everywhere, then you need to be portable
> everywhere.  If only on for linux, then well, it already has GNU
> extensions in libc.  I don’t know if it is on by default and you
> didn’t say, so, hard to comment on it.

Thanks.

The only stuff I'm using getopt_long for is to make the binaries built
by jit.exp be more flexible e.g. so that I can turn down the number of
iterations they run when running under valgrind (and potentially other
tweaks, so e.g. I can experiment with them under gdb without having to
recompile them)

Hence I think we can simply fall back to ignoring argv on hosts that
don't support getopt_long; it should merely make the testsuite less
flexible.  Not sure how best to encode such a test though - check for it
in jit.exp, or in configure, I suppose.

Dave

On Dec 8, 2014, at 5:29 PM, David Malcolm <dmalcolm@redhat.com> wrote:
> The only stuff I'm using getopt_long for is to make the binaries built
> by jit.exp be more flexible e.g. so that I can turn down the number of
> iterations they run when running under valgrind (and potentially other
> tweaks, so e.g. I can experiment with them under gdb without having to
> recompile them)
> 
> Hence I think we can simply fall back to ignoring argv on hosts that
> don't support getopt_long; it should merely make the testsuite less
> flexible.  Not sure how best to encode such a test though - check for it
> in jit.exp, or in configure, I suppose.

I’d just check in jit.exp.  See target-supports.exp for ideas.