diff mbox

[ubsan] Add testsuite

Message ID 20130715174859.GL3697@redhat.com
State New
Headers show

Commit Message

Marek Polacek July 15, 2013, 5:48 p.m. UTC 
Ubsan testsuite is something we've been missing for some time now, so
this patch adds it.  Fortunately the dejagnu part was
quite simple, since ubsan doesn't need similar tweaks as asan does.
But I had to tweak gcc.c to include -lubsan.
The tests are testing pretty basic stuff, however, in LLVM testsuite they
don't check much more.  Maybe we should test also stuff like ++a << (v * x)
etc.?

Tested with
make check RUNTESTFLAGS='--target_board=unix\{-m32,-m64\} ubsan.exp'

Any comments before I put this on ubsan branch?

2013-07-15  Marek Polacek  <polacek@redhat.com>

	* gcc.c (ADD_STATIC_LIBUBSAN_LIBS): Define.
	(LIBUBSAN_SPEC): Likewise.
	(LIBUBSAN_EARLY_SPEC): Likewise.
	(SANITIZER_SPEC): Handle libubsan.
	(SANITIZER_EARLY_SPEC): Likewise.

testsuite/
	* lib/ubsan-dg.exp: New file.
	* g++.dg/ubsan/ubsan.exp: New file.
	* gcc.dg/ubsan/ubsan.exp: New file.
	* g++.dg/ubsan/cxx11-shift-1.C: New test.
	* g++.dg/ubsan/cxx11-shift-2.C: New test.
	* c-c++-common/ubsan/div-by-zero-3.c: New test.
	* c-c++-common/ubsan/div-by-zero-1.c: New test.
	* c-c++-common/ubsan/div-by-zero-4.c: New test.
	* c-c++-common/ubsan/shift-3.c: New test.
	* c-c++-common/ubsan/unreachable-1.c: New test.
	* c-c++-common/ubsan/shift-1.c: New test.
	* c-c++-common/ubsan/shift-2.c: New test.
	* c-c++-common/ubsan/div-by-zero-2.c: New test.
	* gcc.dg/ubsan/c99-shift-2.c: New test.
	* gcc.dg/ubsan/c99-shift-1.c: New test.


	Marek

Comments

Jakub Jelinek July 15, 2013, 11:27 p.m. UTC | #1 
On Mon, Jul 15, 2013 at 07:48:59PM +0200, Marek Polacek wrote:
> Ubsan testsuite is something we've been missing for some time now, so
> this patch adds it.  Fortunately the dejagnu part was
> quite simple, since ubsan doesn't need similar tweaks as asan does.
> But I had to tweak gcc.c to include -lubsan.

Looks good to me.

> The tests are testing pretty basic stuff, however, in LLVM testsuite they
> don't check much more.  Maybe we should test also stuff like ++a << (v * x)
> etc.?

Yeah, more tests would be helpful.  Also, I'd add to the tests where you use
explicit constants in the operation (where we can always (and probably do)
warn at compile time already) tests where it clearly can't be a compile time
decided whether to warn or not and needs to be deferred to runtime.
So say
volatile int shiftcount = 152;
... 1 << shiftcount;
and stuff like that compared to
... 1 << 152;
Side-effects in the expressions also won't hurt, different types too
(so that you check the different value encoding).  E.g. int will be passed
by value on both -m32 and -m64, while long long will be passed by reference
for -m32.

Anyway, this can be done incrementally.

	Jakub
Marek Polacek July 15, 2013, 11:33 p.m. UTC | #2 
On Tue, Jul 16, 2013 at 01:27:00AM +0200, Jakub Jelinek wrote:
> On Mon, Jul 15, 2013 at 07:48:59PM +0200, Marek Polacek wrote:
> > Ubsan testsuite is something we've been missing for some time now, so
> > this patch adds it.  Fortunately the dejagnu part was
> > quite simple, since ubsan doesn't need similar tweaks as asan does.
> > But I had to tweak gcc.c to include -lubsan.
> 
> Looks good to me.

Thanks, commited to ubsan branch.

> > The tests are testing pretty basic stuff, however, in LLVM testsuite they
> > don't check much more.  Maybe we should test also stuff like ++a << (v * x)
> > etc.?
> 
> Yeah, more tests would be helpful.  Also, I'd add to the tests where you use
> explicit constants in the operation (where we can always (and probably do)
> warn at compile time already) tests where it clearly can't be a compile time
> decided whether to warn or not and needs to be deferred to runtime.
> So say
> volatile int shiftcount = 152;
> ... 1 << shiftcount;
> and stuff like that compared to
> ... 1 << 152;
> Side-effects in the expressions also won't hurt, different types too
> (so that you check the different value encoding).  E.g. int will be passed
> by value on both -m32 and -m64, while long long will be passed by reference
> for -m32.
> 
> Anyway, this can be done incrementally.

Ok, will add something like that.

	Marek
Bernhard Reutner-Fischer July 16, 2013, 6:04 a.m. UTC | #3 
+# ubsan_finish -- called at the start of each subdir of tests

s/the start/the end/
Thanks,


Sent with AquaMail for Android
http://www.aqua-mail.com
Marek Polacek July 18, 2013, 9 p.m. UTC | #4 
On Tue, Jul 16, 2013 at 08:04:14AM +0200, Bernhard Reutner-Fischer wrote:
> +# ubsan_finish -- called at the start of each subdir of tests
> 
> s/the start/the end/

You're attentive, will fix.  Thanks,

	Marek
diff mbox

Patch

--- gcc/gcc.c.mp	2013-07-15 04:30:13.038677937 +0200
+++ gcc/gcc.c	2013-07-15 04:42:32.257592503 +0200
@@ -591,6 +591,28 @@  proper position among the other output f
 #define LIBTSAN_EARLY_SPEC ""
 #endif
 
+#ifndef LIBUBSAN_SPEC
+#ifdef STATIC_LIBUBSAN_LIBS
+#define ADD_STATIC_LIBUBSAN_LIBS \
+  " %{static-libubsan:" STATIC_LIBUBSAN_LIBS "}"
+#else
+#define ADD_STATIC_LIBUBSAN_LIBS
+#endif
+#ifdef LIBUBSAN_EARLY_SPEC
+#define LIBUBSAN_SPEC ADD_STATIC_LIBUBSAN_LIBS
+#elif defined(HAVE_LD_STATIC_DYNAMIC)
+#define LIBUBSAN_SPEC "%{static-libubsan:" LD_STATIC_OPTION \
+		     "} -lubsan %{static-libubsan:" LD_DYNAMIC_OPTION "}" \
+		     ADD_STATIC_LIBUBSAN_LIBS
+#else
+#define LIBUBSAN_SPEC "-lubsan" ADD_STATIC_LIBUBSAN_LIBS
+#endif
+#endif
+
+#ifndef LIBUBSAN_EARLY_SPEC
+#define LIBUBSAN_EARLY_SPEC ""
+#endif
+
 /* config.h can define LIBGCC_SPEC to override how and when libgcc.a is
    included.  */
 #ifndef LIBGCC_SPEC
@@ -714,7 +736,8 @@  proper position among the other output f
 #ifndef SANITIZER_EARLY_SPEC
 #define SANITIZER_EARLY_SPEC "\
 %{!nostdlib:%{!nodefaultlibs:%{%:sanitize(address):" LIBASAN_EARLY_SPEC "} \
-    %{%:sanitize(thread):" LIBTSAN_EARLY_SPEC "}}}"
+    %{%:sanitize(thread):" LIBTSAN_EARLY_SPEC "} \
+    %{%:sanitize(undefined):" LIBUBSAN_EARLY_SPEC "}}}"
 #endif
 
 /* Linker command line options for -fsanitize= late on the command line.  */
@@ -724,7 +747,8 @@  proper position among the other output f
     %{static:%ecannot specify -static with -fsanitize=address}\
     %{%:sanitize(thread):%e-fsanitize=address is incompatible with -fsanitize=thread}}\
     %{%:sanitize(thread):" LIBTSAN_SPEC "\
-    %{!pie:%{!shared:%e-fsanitize=thread linking must be done with -pie or -shared}}}}}"
+    %{!pie:%{!shared:%e-fsanitize=thread linking must be done with -pie or -shared}}}\
+    %{%:sanitize(undefined):" LIBUBSAN_SPEC "}}}"
 #endif
 
 /* -u* was put back because both BSD and SysV seem to support it.  */
--- gcc/testsuite/lib/ubsan-dg.exp.mp	2013-07-14 20:19:05.445091076 +0200
+++ gcc/testsuite/lib/ubsan-dg.exp	2013-07-15 04:25:00.476445749 +0200
@@ -0,0 +1,104 @@ 
+# Copyright (C) 2013 Free Software Foundation, Inc.
+
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with GCC; see the file COPYING3.  If not see
+# <http://www.gnu.org/licenses/>.
+
+#
+# ubsan_link_flags -- compute library path and flags to find libubsan.
+# (originally from g++.exp)
+#
+
+proc ubsan_link_flags { paths } {
+    global srcdir
+    global ld_library_path
+    global shlib_ext
+
+    set gccpath ${paths}
+    set flags ""
+
+    set shlib_ext [get_shlib_extension]
+
+    if { $gccpath != "" } {
+      if { [file exists "${gccpath}/libsanitizer/ubsan/.libs/libubsan.a"]
+	   || [file exists "${gccpath}/libsanitizer/ubsan/.libs/libubsan.${shlib_ext}"] } {
+	  append flags " -B${gccpath}/libsanitizer/ubsan/ "
+	  append flags " -L${gccpath}/libsanitizer/ubsan/.libs"
+	  append ld_library_path ":${gccpath}/libsanitizer/ubsan/.libs"
+      }
+    } else {
+      global tool_root_dir
+
+      set libubsan [lookfor_file ${tool_root_dir} libubsan]
+      if { $libubsan != "" } {
+	  append flags "-L${libubsan} "
+	  append ld_library_path ":${libubsan}"
+      }
+    }
+
+    set_ld_library_path_env_vars
+
+    return "$flags"
+}
+
+#
+# ubsan_init -- called at the start of each subdir of tests
+#
+
+proc ubsan_init { args } {
+    global TEST_ALWAYS_FLAGS
+    global ALWAYS_CXXFLAGS
+    global TOOL_OPTIONS
+    global ubsan_saved_TEST_ALWAYS_FLAGS
+
+    set link_flags ""
+    if ![is_remote host] {
+	if [info exists TOOL_OPTIONS] {
+	    set link_flags "[ubsan_link_flags [get_multilibs ${TOOL_OPTIONS}]]"
+	} else {
+	    set link_flags "[ubsan_link_flags [get_multilibs]]"
+	}
+    }
+
+    if [info exists TEST_ALWAYS_FLAGS] {
+	set ubsan_saved_TEST_ALWAYS_FLAGS $TEST_ALWAYS_FLAGS
+    }
+    if [info exists ALWAYS_CXXFLAGS] {
+	set ALWAYS_CXXFLAGS [concat "{ldflags=$link_flags}" $ALWAYS_CXXFLAGS]
+    } else {
+	if [info exists TEST_ALWAYS_FLAGS] {
+	    set TEST_ALWAYS_FLAGS "$link_flags $TEST_ALWAYS_FLAGS"
+	} else {
+	    set TEST_ALWAYS_FLAGS "$link_flags"
+	}
+    }
+    if { $link_flags != "" } {
+	return 1
+    }
+    return 0
+}
+
+#
+# ubsan_finish -- called at the start of each subdir of tests
+#
+
+proc ubsan_finish { args } {
+    global TEST_ALWAYS_FLAGS
+    global ubsan_saved_TEST_ALWAYS_FLAGS
+
+    if [info exists ubsan_saved_TEST_ALWAYS_FLAGS] {
+	set TEST_ALWAYS_FLAGS $ubsan_saved_TEST_ALWAYS_FLAGS
+    } else {
+	unset TEST_ALWAYS_FLAGS
+    }
+}
--- gcc/testsuite/g++.dg/ubsan/ubsan.exp.mp	2013-07-15 04:23:57.350195021 +0200
+++ gcc/testsuite/g++.dg/ubsan/ubsan.exp	2013-07-15 04:25:41.662610652 +0200
@@ -0,0 +1,34 @@ 
+# Copyright (C) 2013 Free Software Foundation, Inc.
+#
+# This file is part of GCC.
+#
+# GCC is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 3, or (at your option)
+# any later version.
+#
+# GCC is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with GCC; see the file COPYING3.  If not see
+# <http://www.gnu.org/licenses/>.
+
+# Load support procs.
+load_lib g++-dg.exp
+load_lib ubsan-dg.exp
+
+# Initialize `dg'.
+dg-init
+if [ubsan_init] {
+
+# Main loop.
+gcc-dg-runtest [lsort [glob -nocomplain $srcdir/$subdir/*.C $srcdir/c-c++-common/ubsan/*.c]] ""
+
+}
+
+# All done.
+ubsan_finish
+dg-finish
--- gcc/testsuite/g++.dg/ubsan/cxx11-shift-1.C.mp	2013-07-15 08:50:26.786626970 +0200
+++ gcc/testsuite/g++.dg/ubsan/cxx11-shift-1.C	2013-07-15 08:50:42.852684617 +0200
@@ -0,0 +1,9 @@ 
+/* { dg-do run } */
+/* { dg-options "-fsanitize=shift -w -std=c++11" } */
+
+int
+main (void)
+{
+  int a = 1;
+  a <<= 31;
+}
--- gcc/testsuite/g++.dg/ubsan/cxx11-shift-2.C.mp	2013-07-15 08:58:40.648302392 +0200
+++ gcc/testsuite/g++.dg/ubsan/cxx11-shift-2.C	2013-07-15 09:00:08.169601202 +0200
@@ -0,0 +1,10 @@ 
+/* { dg-do run } */
+/* { dg-options "-fsanitize=shift -w -std=c++11" } */
+
+int
+main (void)
+{
+  int a = -42;
+  a <<= 1;
+}
+/* { dg-output "left shift of negative value -42" } */
--- gcc/testsuite/c-c++-common/ubsan/div-by-zero-3.c.mp	2013-07-15 05:06:26.069422162 +0200
+++ gcc/testsuite/c-c++-common/ubsan/div-by-zero-3.c	2013-07-15 08:00:25.517263218 +0200
@@ -0,0 +1,12 @@ 
+/* { dg-do run } */
+/* { dg-options "-fsanitize=integer-divide-by-zero -Wno-overflow" } */
+
+#include <limits.h>
+
+int
+main (void)
+{
+  INT_MIN / -1;
+  return 0;
+}
+ /* { dg-output "division of -2147483648 by -1 cannot be represented in type int" } */
--- gcc/testsuite/c-c++-common/ubsan/div-by-zero-1.c.mp	2013-07-15 04:56:34.640124598 +0200
+++ gcc/testsuite/c-c++-common/ubsan/div-by-zero-1.c	2013-07-15 07:59:58.713250028 +0200
@@ -0,0 +1,10 @@ 
+/* { dg-do run } */
+/* { dg-options "-fsanitize=integer-divide-by-zero -Wno-div-by-zero" } */
+
+int
+main (void)
+{
+  0 / 0;
+  return 0;
+}
+ /* { dg-output "division by zero" } */
--- gcc/testsuite/c-c++-common/ubsan/div-by-zero-4.c.mp	2013-07-15 05:06:32.331446996 +0200
+++ gcc/testsuite/c-c++-common/ubsan/div-by-zero-4.c	2013-07-15 05:26:00.450043589 +0200
@@ -0,0 +1,11 @@ 
+/* { dg-do run } */
+/* { dg-options "-fsanitize=integer-divide-by-zero -Wno-overflow" } */
+
+#include <limits.h>
+
+int
+main (void)
+{
+  /* This should not fail.  */
+  return (unsigned int) INT_MIN / -1;
+}
--- gcc/testsuite/c-c++-common/ubsan/shift-3.c.mp	2013-07-15 08:55:51.786698656 +0200
+++ gcc/testsuite/c-c++-common/ubsan/shift-3.c	2013-07-15 09:06:12.325948616 +0200
@@ -0,0 +1,11 @@ 
+/* { dg-do run } */
+/* { dg-options "-fsanitize=shift -w" } */
+
+int
+main (void)
+{
+  unsigned int a = 1;
+  a <<= 31;
+  a <<= 1;
+  return 0;
+}
--- gcc/testsuite/c-c++-common/ubsan/unreachable-1.c.mp	2013-07-15 04:23:03.799985724 +0200
+++ gcc/testsuite/c-c++-common/ubsan/unreachable-1.c	2013-07-15 08:08:26.949168293 +0200
@@ -0,0 +1,10 @@ 
+/* { dg-do run } */
+/* { dg-options "-fsanitize=unreachable" } */
+/* { dg-shouldfail "ubsan" } */
+
+int
+main (void)
+{
+  __builtin_unreachable ();
+}
+ /* { dg-output "execution reached a __builtin_unreachable\\(\\) call" } */
--- gcc/testsuite/c-c++-common/ubsan/shift-1.c.mp	2013-07-15 05:30:50.662201212 +0200
+++ gcc/testsuite/c-c++-common/ubsan/shift-1.c	2013-07-15 08:55:12.915542820 +0200
@@ -0,0 +1,11 @@ 
+/* { dg-do run } */
+/* { dg-options "-fsanitize=shift -w" } */
+
+int
+main (void)
+{
+  int a = 1;
+  a <<= 152;
+  return 0;
+}
+/* { dg-output "shift exponent 152 is too large for \[^\n\r]*-bit type int" } */
--- gcc/testsuite/c-c++-common/ubsan/shift-2.c.mp	2013-07-15 06:36:50.754358414 +0200
+++ gcc/testsuite/c-c++-common/ubsan/shift-2.c	2013-07-15 07:59:13.770070848 +0200
@@ -0,0 +1,11 @@ 
+/* { dg-do run } */
+/* { dg-options "-fsanitize=shift -w" } */
+
+int
+main (void)
+{
+  int a = 1;
+  a <<= -3;
+  return 0;
+}
+/* { dg-output "shift exponent -3 is negative" } */
--- gcc/testsuite/c-c++-common/ubsan/div-by-zero-2.c.mp	2013-07-15 05:02:33.252507624 +0200
+++ gcc/testsuite/c-c++-common/ubsan/div-by-zero-2.c	2013-07-15 08:00:10.452296766 +0200
@@ -0,0 +1,10 @@ 
+/* { dg-do run } */
+/* { dg-options "-fsanitize=integer-divide-by-zero -Wno-div-by-zero" } */
+
+int
+main (void)
+{
+  1UL / 0;
+  return 0;
+}
+ /* { dg-output "division by zero" } */
--- gcc/testsuite/gcc.dg/ubsan/c99-shift-2.c.mp	2013-07-15 08:47:01.177890666 +0200
+++ gcc/testsuite/gcc.dg/ubsan/c99-shift-2.c	2013-07-15 08:48:39.085243951 +0200
@@ -0,0 +1,10 @@ 
+/* { dg-do run } */
+/* { dg-options "-fsanitize=shift -w -std=c99" } */
+
+int
+main (void)
+{
+  int a = 1;
+  a <<= 31;
+}
+/* { dg-output "left shift of 1 by 31 places cannot be represented in type int" } */
--- gcc/testsuite/gcc.dg/ubsan/ubsan.exp.mp	2013-07-14 20:17:20.856743632 +0200
+++ gcc/testsuite/gcc.dg/ubsan/ubsan.exp	2013-07-15 01:44:18.316194533 +0200
@@ -0,0 +1,36 @@ 
+# Copyright (C) 2013 Free Software Foundation, Inc.
+#
+# This file is part of GCC.
+#
+# GCC is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 3, or (at your option)
+# any later version.
+#
+# GCC is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with GCC; see the file COPYING3.  If not see
+# <http://www.gnu.org/licenses/>.
+
+# GCC testsuite that uses the `dg.exp' driver.
+
+# Load support procs.
+load_lib gcc-dg.exp
+load_lib ubsan-dg.exp
+
+# Initialize `dg'.
+dg-init
+if [ubsan_init] {
+
+# Main loop.
+gcc-dg-runtest [lsort [glob -nocomplain $srcdir/$subdir/*.c $srcdir/c-c++-common/ubsan/*.c]] ""
+
+}
+
+# All done.
+ubsan_finish
+dg-finish
--- gcc/testsuite/gcc.dg/ubsan/c99-shift-1.c.mp	2013-07-15 08:42:03.829843562 +0200
+++ gcc/testsuite/gcc.dg/ubsan/c99-shift-1.c	2013-07-15 08:43:20.769105362 +0200
@@ -0,0 +1,10 @@ 
+/* { dg-do run } */
+/* { dg-options "-fsanitize=shift -w -std=c99" } */
+
+int
+main (void)
+{
+  int a = -42;
+  a << 1;
+}
+/* { dg-output "left shift of negative value -42" } */