diff mbox

ubsan: Do not run the testsuite if ubsan does not work at all

Message ID 794f294d0fe7dd3f95e2b05875619307538a9b7f.1419462987.git.segher@kernel.crashing.org
State New
Headers show

Commit Message

Segher Boessenkool Dec. 24, 2014, 11:21 p.m. UTC
I normally build with --disable-libsanitizer, because the sanitizers
testresults are very unreproducable, so just annoying noise.  This however
makes most (all?) ubsan testcases fail, since they want to load a shared
library that does not exist.

This patch disables the ubsan testcases if a trivial program does not
run when compiled with ubsan.

Tested on powerpc64-linux, -m32,-m32/-mpowerpc64,-m64,-m64/-mlra;
5452 failures gone, no other changes.

Okay for mainline?


Segher


2014-12-24  Segher Boessenkool  <segher@kernel.crashing.org>

gcc/testsuite/
	* lib/ubsan-dg.exp (check_effective_target_fsanitize_undefined):
	Check if testcases run without errors, not just if they compile.

---
 gcc/testsuite/lib/ubsan-dg.exp | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

Comments

Mike Stump Dec. 27, 2014, 4:09 p.m. UTC | #1
On Dec 24, 2014, at 3:21 PM, Segher Boessenkool <segher@kernel.crashing.org> wrote:
> I normally build with --disable-libsanitizer, because the sanitizers
> testresults are very unreproducable, so just annoying noise.  This however
> makes most (all?) ubsan testcases fail, since they want to load a shared
> library that does not exist.
> 
> This patch disables the ubsan testcases if a trivial program does not
> run when compiled with ubsan.
> 
> Tested on powerpc64-linux, -m32,-m32/-mpowerpc64,-m64,-m64/-mlra;
> 5452 failures gone, no other changes.
> 
> Okay for mainline?

So, I was hoping the ubsan people would have weighed in...

Ok.  Please watch for nits from them.

Please consider pulling the check out and putting it in at the top of the ubsan.exp type of files.

I suspect there is little need to check this more than once per language or so.
Segher Boessenkool Dec. 27, 2014, 4:11 p.m. UTC | #2
On Sat, Dec 27, 2014 at 08:09:47AM -0800, Mike Stump wrote:
> > I normally build with --disable-libsanitizer, because the sanitizers
> > testresults are very unreproducable, so just annoying noise.  This however
> > makes most (all?) ubsan testcases fail, since they want to load a shared
> > library that does not exist.
> > 
> > This patch disables the ubsan testcases if a trivial program does not
> > run when compiled with ubsan.
> > 
> > Tested on powerpc64-linux, -m32,-m32/-mpowerpc64,-m64,-m64/-mlra;
> > 5452 failures gone, no other changes.
> > 
> > Okay for mainline?
> 
> So, I was hoping the ubsan people would have weighed in...

Holidays...

> Ok.  Please watch for nits from them.
> 
> Please consider pulling the check out and putting it in at the top of the ubsan.exp type of files.
> 
> I suspect there is little need to check this more than once per language or so.

It already is cached, but yeah I see what you mean.  Something for the
ubsan people to handle, perhaps?  Or someone else who can actually write
dejagnu things :-)


Segher
Segher Boessenkool Dec. 27, 2014, 4:33 p.m. UTC | #3
On Sat, Dec 27, 2014 at 10:11:59AM -0600, Segher Boessenkool wrote:
> > Please consider pulling the check out and putting it in at the top of the ubsan.exp type of files.
> > 
> > I suspect there is little need to check this more than once per language or so.

Actually, that is already how it is done :-)


Segher
Mike Stump Dec. 27, 2014, 5:34 p.m. UTC | #4
On Dec 27, 2014, at 8:33 AM, Segher Boessenkool <segher@kernel.crashing.org> wrote:
> On Sat, Dec 27, 2014 at 10:11:59AM -0600, Segher Boessenkool wrote:
>>> Please consider pulling the check out and putting it in at the top of the ubsan.exp type of files.
>>> 
>>> I suspect there is little need to check this more than once per language or so.
> 
> Actually, that is already how it is done :-)

No, examine i386.exp for the line at the top that does this:

# Exit immediately if this isn't a x86 target.                                                                                                       
if { ![istarget i?86*-*-*] && ![istarget x86_64-*-*] } then {
  return
}

Here, you see that if there are 100,000 x86 test cases, this is checked once and only about 2 lines of tcl are run.  In yours, if there were 100,000 ubsan test cases, you would discover millions of lines run, not 2.  millions is slightly more than 2.
Segher Boessenkool Dec. 27, 2014, 5:56 p.m. UTC | #5
On Sat, Dec 27, 2014 at 09:34:44AM -0800, Mike Stump wrote:
> >>> Please consider pulling the check out and putting it in at the top of the ubsan.exp type of files.
> >>> 
> >>> I suspect there is little need to check this more than once per language or so.
> > 
> > Actually, that is already how it is done :-)
> 
> No, examine i386.exp for the line at the top that does this:
> 
> # Exit immediately if this isn't a x86 target.                                                                                                       
> if { ![istarget i?86*-*-*] && ![istarget x86_64-*-*] } then {
>   return
> }
> 
> Here, you see that if there are 100,000 x86 test cases, this is checked once and only about 2 lines of tcl are run.  In yours, if there were 100,000 ubsan test cases, you would discover millions of lines run, not 2.  millions is slightly more than 2.

From gcc.dg/ubsan/ubsan.exp:

# Main loop.
if [check_effective_target_fsanitize_undefined] {
  gcc-dg-runtest [lsort [glob -nocomplain $srcdir/$subdir/*.c $srcdir/c-c++-common/ubsan/*.c]] "" ""
}

That only runs once, doesn't it?  Am I missing something?

There are some init things before it, but they may be needed for the
check itself.


Segher
Mike Stump Dec. 28, 2014, 7:43 a.m. UTC | #6
On Dec 27, 2014, at 9:56 AM, Segher Boessenkool <segher@kernel.crashing.org> wrote:
> From gcc.dg/ubsan/ubsan.exp:
> 
> # Main loop.
> if [check_effective_target_fsanitize_undefined] {

> That only runs once, doesn't it?

Yes, it does.  Perfect.
diff mbox

Patch

diff --git a/gcc/testsuite/lib/ubsan-dg.exp b/gcc/testsuite/lib/ubsan-dg.exp
index 3bfdcc8..cea3e0e 100644
--- a/gcc/testsuite/lib/ubsan-dg.exp
+++ b/gcc/testsuite/lib/ubsan-dg.exp
@@ -18,7 +18,7 @@ 
 # code, 0 otherwise.
 
 proc check_effective_target_fsanitize_undefined {} {
-    return [check_no_compiler_messages fsanitize_undefined executable {
+    return [check_runtime fsanitize_undefined {
 	int main (void) { return 0; }
     } "-fsanitize=undefined"]
 }