[WIKI] Replace delta with C-Vise (and C-Reduce)

Hello.

I've spent quite some time working of a super-parallel reduction tool
and I would like to promote it ;) Moreover, delta website is down and
it should be replaced: [1].

There's updated wording of the following WIKI page:
https://gcc.gnu.org/wiki/A_guide_to_testcase_reduction

For all delta lovers who don't like C-Vise (or C-Reduce), I recommend
cvise-delta tool. It works the same as delta, but in a super-parallel way.

The tool is available on openSUSE and Gentoo Linux. Thanks to Matthias,
the tool will be available soon on Ubuntu and Debian. The RedHat port
is work-in-progress.

Thoughts?
Martin

[1] http://delta.tigris.org/

Hi,

On Thu, May 28 2020, Martin Liška wrote:
> Hello.
>
> I've spent quite some time working of a super-parallel reduction tool
> and I would like to promote it ;) Moreover, delta website is down and
> it should be replaced: [1].
>
> There's updated wording of the following WIKI page:
> https://gcc.gnu.org/wiki/A_guide_to_testcase_reduction
>
> For all delta lovers who don't like C-Vise (or C-Reduce), I recommend
> cvise-delta tool. It works the same as delta, but in a super-parallel way.
>
> The tool is available on openSUSE and Gentoo Linux. Thanks to Matthias,
> the tool will be available soon on Ubuntu and Debian. The RedHat port
> is work-in-progress.
>
> Thoughts?

I don't think you need to seek approval to edit wiki pages and putting
c-vise instructions at the top of that page is definitely the right
thing to do.

On the other hand, I would not remove the delta and multidelta sections
but rather move them to the bottom of the page.  The instructions may
still be useful on various ancient and non-Linux systems.

Thanks,

Martin


> Martin
>
> [1] http://delta.tigris.org/
> From 4f9bb31b435d0e60ea7cdccd575aa590c943f516 Mon Sep 17 00:00:00 2001
> From: Martin Liska <mliska@suse.cz>
> Date: Thu, 28 May 2020 12:54:21 +0200
> Subject: [PATCH] Replace usage of delta (and multidelta) with C-Vise and
>  C-Reduce.
>
> ---
>  guide_to_test_case.txt | 110 ++++++++---------------------------------
>  1 file changed, 20 insertions(+), 90 deletions(-)
>
> diff --git a/guide_to_test_case.txt b/guide_to_test_case.txt
> index 7926b28..cd66c15 100644
> --- a/guide_to_test_case.txt
> +++ b/guide_to_test_case.txt
> @@ -5,9 +5,7 @@
>  Our [[https://gcc.gnu.org/bugs.html|bug reporting instructions]] ask that a bug report include the preprocessed version of the file that triggers the bug. There are several [[https://gcc.gnu.org/bugs/minimize.html|methods to minimise a testcase]].
>  
>  This page ought to serve as an introduction to automatic testcase reduction using
> -the [[http://delta.tigris.org/|Delta]] or [[http://embed.cs.utah.edu/creduce/|C-Reduce]] tools.
> -
> -For '''Fortran''' there is a [[https://gcc.gnu.org/ml/gcc/2009-10/msg00618.html|patched version of Delta]] which takes `subroutine`/`do`/`if` boundaries into account.
> +[[https://github.com/marxin/cvise|C-Vise]] tools or [[http://embed.cs.utah.edu/creduce/|C-Reduce]] tools.
>  
>  == Simple ICE reduction ==
>  
> @@ -23,19 +21,15 @@ For '''Fortran''' there is a [[https://gcc.gnu.org/ml/gcc/2009-10/msg00618.html|
>    > mv testcase.x testcase.i
>  }}}
>  
> - The Delta tool requires us to create a script that exits with status zero in
> + The C-Vise (or C-Reduce) tool requires us to create a script that exits with status zero in
>   case of the intermediate reduced testcase still is a testcase for what we got
>   it in the first place (the same ICE is produced).  A sample script may look
> - like (the testcase filename is passed as argument to the script)
> + like
>  
>  
>  {{{
>    #!/bin/sh
> -  gcc -c -O -Wfatal-errors $1 2>&1 | grep 'internal compiler error: in typeck.c:2534'
> -  if ! test $? = 0; then
> -   exit 1
> -  fi
> -  exit 0
> +  gcc -c -O -Wfatal-errors testcase.i 2>&1 | grep 'internal compiler error: in typeck.c:2534'
>  }}}
>  
>   Note the {{{-Wfatal-errors}}} option can greatly speed up reducing large testcases,
> @@ -46,92 +40,35 @@ For '''Fortran''' there is a [[https://gcc.gnu.org/ml/gcc/2009-10/msg00618.html|
>   You should be able to verify your script by invoking it with the unreduced testcase.
>   Try if it has zero exit code.
>  
> - Now we can invoke Delta to have it reduce the testcase using the script we just
> + Now we can invoke C-Vise (or C-Reduce) to have it reduce the testcase using the script we just
>   wrote
>  
> -
>  {{{
> -  > ~/bin/delta -test=check.sh -suffix=.i -cp_minimal=testcase-min.i testcase.i
> +  > cvise check.sh testcase.i
>  }}}
>  
>   This will reduce the testcase until no single line can be removed from it without
>   the check.sh script failing to identify it as a valid testcase.
>  
> -== Using topformflat ==
> -
> - The way delta reduces a testcase by removing complete lines often conflicts with
> - the syntactic structure of a C/C++ testcase.  To make testcase reduction faster
> - and more accurate there exists the topformflat tool in the Delta distribution
> - that puts syntactically related tokens on one line, thereby making it possible
> - to, f.i. restrict reduction to whole-function removal in a first step.  Basically
> - you can control the nesting level up to which tokens are put to separate lines
> - where a level of zero is all toplevel constructs onto a line on their own, level
> - one would be each statement of a toplevel function on a separate line.
> -
> - Reducing a big C++ testcase one usually starts with level zero, increasing it
> - until Delta no longer can reduce the testcase further (due to the line-oriented
> - reduction it may be worthwhile to start over with level zero again and iterate
> - until there's no further reduction).  An improved topformflat was posted at
> - [[https://gcc.gnu.org/ml/gcc-patches/2005-08/msg01503.html]] where you can additionally
> - specify if you want to ignore namespace and extern "C" as a nesting construct by
> - specifying a second command line argument to topformflat.
> -
> -
> -{{{
> -  > ~/bin/topformflat 0 x < testcase.i > testcase.0x.i
> -}}}
> -
> -
> -== Using multidelta ==
> -
> -All the above can be simplified by using the '''multidelta''' tool that comes with the [[http://delta.tigris.org/|Delta distribution]]. The only differences is that the script should be able to be called without parameters. In the above example, the new script would be:
> -
> -{{{
> -  #!/bin/bash
> -  TESTCASE=${1:-testcase.i}
> -  gcc -c -O -Wfatal-errors -w $TESTCASE 2>&1 | grep -q 'internal compiler error: in typeck.c:2534'
> -  if ! test $? = 0; then
> -   exit 1
> -  fi
> -  exit 0
> -}}}
> -
>  The fastest is this script, the fastest the reduction will be:
>  
>   * Avoid optimizing {{{-O0}}} if it is a bug in the front-ends.
>   * If the bug is not triggered by a warning, disable all warnings {{{-w}}}.
>   * If the bug is an ICE that also happens with {{{-fpermissive}}}, then use it.
> - * If the bug is not in the preprocessor, use {{{-fpreprocessed}}}, otherwhise use {{{-E}}}. 
> + * If the bug is not in the preprocessor, use {{{-fpreprocessed}}}, otherwhise use {{{-E}}}.
>   * Use {{{-S}}} to avoid assembling.
>   * {{{-o /dev/null}}} if you do not need to analyze the output files.
>   * It may be slightly faster to invoke {{{cc1}}} and {{{cc1plus}}} directly if the bug is not in the driver.
>   * For grepping, it is faster to not use regular expressions and invoke {{{fgrep}}} directly. It is also faster to use the option "--quiet".
>  
>  
> -Then, you may run multidelta with the command:
> -
> -{{{
> -multidelta -level=2 ./check.sh testcase.i &> /dev/null
> -}}}
> -
> -Where {{{-level=}}} is the level of topformflat (see above). Check the progress in the files {{{log}}} and {{{multidelta.log}}}. Multidelta modifies the input file, but it creates a backup {{{.bak}}} file). The last successful reduction is always stored in {{{testcase.i.ok}}}.
> -
> -== Using C-Reduce ==
> +== Using C-Vise and C-Reduce ==
>  
> -Instead of using delta or multidelta it is also possible to run '''C-Reduce''' on your
> -testcase. This reducer specifically targets C and C++ code and makes
> +C-Vise and C-Reduce provide much faster reduction than the original delta tool.
> +This reducer specifically targets C and C++ code and makes
>  coordinated changes across the whole program: removing an array dimension,
> -removing a function argument, reordering function calls, etc. It will first automatically
> -run a few passes that are identical to multidelta, so there is no need to run
> -delta or mulitdelta first. Creduce runs in parallel by default if multiple cores are available.
> -
> -C-Reduce can be found here: [[http://embed.cs.utah.edu/creduce/]]
> -
> -You may run C-Reduce with the same script that multidelta uses above:
> -
> -{{{
> -creduce check.sh testcase.i
> -}}}
> +removing a function argument, reordering function calls, etc.
> +C-Vise runs in parallel by default if multiple cores are available.
>  
>  == Reducing "works with -O, doesn't work with -O2" type bugs ==
>  
> @@ -172,17 +109,14 @@ creduce check.sh testcase.i
>  
>   If you need to reduce testcases for LTO bugs that happen at link stage you have the problem
>   that many object files (and thus source files) can be involved.  If the bug is an internal
> - compiler error you can bisect the files needed to trigger the bug using delta.  From the
> - lto1 invocation command extract the list-of-objects file (@/tmp/ccXXXXX) [use "-v -Wl,-debug" to find that line]; change spaces to newlines in there and prepend the current working directory name.  Then use delta
> - on that file with a check script similar to
> + compiler error you can bisect the files needed to trigger the bug using cvise-delta
> + (provided by C-Vise package).  From the lto1 invocation command extract the list-of-objects file (@/tmp/ccXXXXX)
> + [use "-v -Wl,-debug" to find that line]; change spaces to newlines in there and prepend the current working directory name.
> + Then use C-Vise (or C-Reduce) on that file with a check script similar to
>  
>  {{{
>    #!/bin/sh
> -
>    /path/to/lto1 -o /dev/null @$1 rest-of-your-options 2>&1 | grep '...the ICE...'
> -  if ! test $? = 0; then
> -    exit 1
> -  fi
>  }}}
>  
>    Now generate preprocessed source for the files required to build the remaining object files.
> @@ -192,19 +126,19 @@ creduce check.sh testcase.i
>    gcc -r -nostdlib preprocessed-inputs rest-of-your-options
>  }}}
>  
> -  And the preprocessed files can now be individually reduced with delta.  Be careful with
> +  And the preprocessed files can now be individually reduced with C-Vise.  Be careful with
>    creating invalid testcases here though.  Safe reduction can be done on topflatform level 0
>    and level 0 with ignoring namespaces.  This is to avoid differences in type layout for
>    same types in different files.
>  
> -  <<BR>>Alternatively creduce multi-file reduction may be used: 
> +  <<BR>>Alternatively C-Vise multi-file reduction may be used:
>  {{{
> -  creduce ./check.sh file1.ii file2.ii file3.ii ...
> +  cvise ./check.sh file1.ii file2.ii file3.ii ...
>  }}}
>  
>  == Further hints ==
>  
> - Sometimes it may be advisable to preserve parts of the testcase completely to make delta
> + Sometimes it may be advisable to preserve parts of the testcase completely to make C-Vise (or C-Reduce)
>   not reduce the testcase to nonsense still passing your script.  This is also useful if
>   you just want the smallest possible self-contained source for a function to ease analyzing
>   whatever you are interested in.  Split your testcase into two parts, one to be reduced and
> @@ -216,10 +150,6 @@ creduce check.sh testcase.i
>   #!/bin/sh
>   cat $1 ../../tail.i > x.i
>   gcc -S x.i -Wfatal-errors
> - if ! test "$?" = "0"; then
> -   exit 1
> - fi
> - exit 0
>  }}}
>  
>   You can even check for the same asm created in the above scheme, but you need to carefully
> -- 
> 2.26.2

Hi Martin,

On 5/28/20 1:17 PM, Martin Jambor wrote:
> On Thu, May 28 2020, Martin Liška wrote:
>> Hello.
>>
>> I've spent quite some time working of a super-parallel reduction tool
>> and I would like to promote it ;) Moreover, delta website is down and
>> it should be replaced: [1].
It is not completely clear to me whether C-Vise also works with Fortran;
if not, could you add a reference early in the page to point for Fortran
to the delta section? If C-Vise does work with Fortran, I still concur
with Martin's suggestion:
> On the other hand, I would not remove the delta and multidelta sections
> but rather move them to the bottom of the page.  The instructions may
> still be useful on various ancient and non-Linux systems.
Tobias
-----------------
Mentor Graphics (Deutschland) GmbH, Arnulfstraße 201, 80634 München / Germany
Registergericht München HRB 106955, Geschäftsführer: Thomas Heurung, Alexander Walter

On 5/28/20 1:17 PM, Martin Jambor wrote:
> I don't think you need to seek approval to edit wiki pages and putting
> c-vise instructions at the top of that page is definitely the right
> thing to do.

All right.

> 
> On the other hand, I would not remove the delta and multidelta sections
> but rather move them to the bottom of the page.  The instructions may
> still be useful on various ancient and non-Linux systems.

I moved the section to Legacy tools and I've just updated the wiki page.

Thanks,
Martin

On 5/28/20 1:26 PM, Tobias Burnus wrote:
> It is not completely clear to me whether C-Vise also works with Fortran;

Yes, it works. Some C/C++-related passes are skipped, but it works.

Martin