From patchwork Fri Nov 23 14:05:38 2012 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jakub Jelinek X-Patchwork-Id: 201327 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from sourceware.org (server1.sourceware.org [209.132.180.131]) by ozlabs.org (Postfix) with SMTP id 29DCF2C008C for ; Sat, 24 Nov 2012 01:06:29 +1100 (EST) Comment: DKIM? See http://www.dkim.org DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d=gcc.gnu.org; s=default; x=1354284390; h=Comment: DomainKey-Signature:Received:Received:Received:Received:Received: Received:Received:Date:From:To:Cc:Subject:Message-ID:Reply-To: MIME-Version:Content-Type:Content-Disposition:User-Agent: Mailing-List:Precedence:List-Id:List-Unsubscribe:List-Archive: List-Post:List-Help:Sender:Delivered-To; bh=/dktDhfVo1cxZ0C7tMhe Boi3VYk=; b=OJtEig/YXfiFMJGk3vH2bGbronWZK/lEkj8Gz0EYjZ9J1ZkiC7/Q kUPD4sip5CriXQvNTyjKBGSOhRnj5hg0caccT7R3YqKT/5MqCf19pkmVw4nX0jUc 6/rAMA0SL14WPBb7jWY+1gKQtph3qot6h7LFrYbrdxcHDAXhaG33Ed8= Comment: DomainKeys? See http://antispam.yahoo.com/domainkeys DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=default; d=gcc.gnu.org; h=Received:Received:X-SWARE-Spam-Status:X-Spam-Check-By:Received:Received:Received:Received:Received:Date:From:To:Cc:Subject:Message-ID:Reply-To:MIME-Version:Content-Type:Content-Disposition:User-Agent:X-IsSubscribed:Mailing-List:Precedence:List-Id:List-Unsubscribe:List-Archive:List-Post:List-Help:Sender:Delivered-To; b=Pu6JKwE1pJ2dZH6HVZQo4gd8BF2bu2+wUrzcMrirOA91zhzsCvaStC55YGYfQD 8mkxAuWR+EB0jB1Qn4Z1Rr4QpH9vkHL+O0Ic2Dmc6D3IHPmNLHhY6nHfYRKViD9i VXK0n0yiKTYdFSti1eGHWUXys0CldXfePCFFOL6yENVA4=; Received: (qmail 18923 invoked by alias); 23 Nov 2012 14:06:19 -0000 Received: (qmail 18911 invoked by uid 22791); 23 Nov 2012 14:06:16 -0000 X-SWARE-Spam-Status: No, hits=-6.5 required=5.0 tests=AWL, BAYES_00, KHOP_RCVD_UNTRUST, RCVD_IN_DNSWL_HI, RCVD_IN_HOSTKARMA_W, RP_MATCHES_RCVD, SPF_HELO_PASS, TW_TM, TW_VP, UPPERCASE_50_75 X-Spam-Check-By: sourceware.org Received: from mx1.redhat.com (HELO mx1.redhat.com) (209.132.183.28) by sourceware.org (qpsmtpd/0.43rc1) with ESMTP; Fri, 23 Nov 2012 14:05:42 +0000 Received: from int-mx09.intmail.prod.int.phx2.redhat.com (int-mx09.intmail.prod.int.phx2.redhat.com [10.5.11.22]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id qANE5f6b013168 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Fri, 23 Nov 2012 09:05:41 -0500 Received: from zalov.redhat.com (vpn1-7-208.ams2.redhat.com [10.36.7.208]) by int-mx09.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id qANE5dcZ012784 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Fri, 23 Nov 2012 09:05:41 -0500 Received: from zalov.cz (localhost [127.0.0.1]) by zalov.redhat.com (8.14.5/8.14.5) with ESMTP id qANE5dJH012167; Fri, 23 Nov 2012 15:05:39 +0100 Received: (from jakub@localhost) by zalov.cz (8.14.5/8.14.5/Submit) id qANE5cdo012166; Fri, 23 Nov 2012 15:05:38 +0100 Date: Fri, 23 Nov 2012 15:05:38 +0100 From: Jakub Jelinek To: Dmitry Vyukov Cc: Kostya Serebryany , Dodji Seketeli , gcc-patches@gcc.gnu.org Subject: [tsan] Instrument atomics Message-ID: <20121123140538.GX2315@tucnak.redhat.com> Reply-To: Jakub Jelinek MIME-Version: 1.0 Content-Disposition: inline User-Agent: Mutt/1.5.21 (2010-09-15) X-IsSubscribed: yes Mailing-List: contact gcc-patches-help@gcc.gnu.org; run by ezmlm Precedence: bulk List-Id: List-Unsubscribe: List-Archive: List-Post: List-Help: Sender: gcc-patches-owner@gcc.gnu.org Delivered-To: mailing list gcc-patches@gcc.gnu.org Hi! This patch attempts to instrument __atomic_* and __sync_* builtins. Unfortunately for none of the builtins there is 1:1 mapping to the tsan replacements, tsan uses weirdo memory model encoding (instead of values from 0 to 5 apparently 1 << 0 to 1 << 5, as if one could have more than one memory model at the same time), so even for the easy cases GCC has to transform them. More importantly, there is no way to pass through __ATOMIC_HLE_ACQUIRE/__ATOMIC_HLE_RELEASE. Right now the patch just gives up and doesn't replace anything if e.g. __atomic_store_n (p, 0, __ATOMIC_HLE_RELEASE | __ATOMIC_RELEASE); is seen, perhaps one could ignore the hle bits which are just an optimization, but there is no way to find out in the generic code whether the extra bits are just an optimization or change the behavior of the builtin. Another issue is that libtsan apparently internally uses just the deprecated __sync_* builtins (or no builtins at all for load/store). That in some cases works (because __sync_* is usually equivalent to __atomic_* with __ATOMIC_SEQ_CST memmodel), but for the load/store cases and for atomic exchange it doesn't (the former don't provide any barrier, the latter in __sync_lock_test_and_set is only __ATOMIC_ACQUIRE barrier). Can libtsan at least conditionally, when built with GCC 4.7 or later, use __atomic_* builtins instead of __sync_*? One still probably has to use __ATOMIC_SEQ_CST model anyway, otherwise there would need to be a switch based on the mem model, as only constant arguments to __atomic_* builtins do something other than sequential consistency. Another issue is that there are no fetch + nand (or nand + fetch) tsan entrypoints, I could transform those into a loop using cas, but that looks overkill, then it would be better to change libtsan. The most important problem is that __tsan_atomic*_compare_exchange has just a single memory model argument, while the __atomic_* builtins have two (one for success, another one for failure). Right now the patch just ignores the failure memory model, but that might actually lead into not detecting a race in a program when it should have been detected (note the failure memory model can't be stronger than success memory model). Would it be possible to add another argument to those, and use the failure mode instead of success mode if the atomic operation fails? Apparently there are also no entry points for op and fetch (as opposed to fetch and op), but the patch translates those into the corresponding fetch and op libcalls with + val (etc.) on the result. Oh, and there are no 16-byte atomics provided by libtsan, the library would need to be built with -mcx16 on x86_64 for that and have the additional entry points for unsigned __int128. The patch doesn't touch the 16-byte atomics, leaves them as is. This patch is on top of the patch from yesterday which introduced use of BUILT_IN_* functions in asan and builds them if the FE doesn't. I've used the attached testcase to verify it compiles, and quickly skimmed the changes it is doing, but haven't actually tested it more than that. As for local statics mentioned in the PR too, those are in GCC handled by __cxa_guard_{acquire,release,abort} library functions, so either libtsan would need to intercept those calls, or we'd need to add some instrumentation before and/or after that (what would be needed?). And, I'm also wondering about string/memory builtins asan is instrumenting, wouldn't it be useful to instrument those similarly for tsan (say also just the first byte accessed and last one)? I know you are overriding most of them in libtsan, but some of them are often expanded inline (e.g. memcpy, strlen, etc.) and thus the libtsan replacements aren't executed. Or better yet, is there some libtsan entry point to report memory access range? 2012-11-23 Jakub Jelinek PR sanitizer/55439 * Makefile.in (tsan.o): Depend on tree-ssa-propagate.h. * sanitizer.def: Add __tsan_atomic* builtins. * asan.c (initialize_sanitizer_builtins): Adjust to also initialize __tsan_atomic* builtins. * tsan.c: Include tree-ssa-propagate.h. (enum tsan_atomic_action): New enum. (tsan_atomic_table): New table. (instrument_builtin_call): New function. (instrument_gimple): Take pointer to gimple_stmt_iterator instead of gimple_stmt_iterator. Call instrument_builtin_call on builtin call stmts. (instrument_memory_accesses): Adjust instrument_gimple caller. * builtin-types.def (BT_FN_BOOL_VPTR_PTR_I1_INT, BT_FN_BOOL_VPTR_PTR_I2_INT, BT_FN_BOOL_VPTR_PTR_I4_INT, BT_FN_BOOL_VPTR_PTR_I8_INT): New. Jakub extern void bar (void *); template __attribute__((noinline, noclone)) T foo (T *p, bool var) { T x = 0, y; x += __sync_fetch_and_add (p, 1); bar ((void *) p); x += __sync_fetch_and_sub (p, 2); bar ((void *) p); x += __sync_fetch_and_and (p, 7); bar ((void *) p); x += __sync_fetch_and_or (p, 1); bar ((void *) p); x += __sync_fetch_and_xor (p, 2); bar ((void *) p); x += __sync_fetch_and_nand (p, 4); bar ((void *) p); x += __sync_add_and_fetch (p, 1); bar ((void *) p); x += __sync_sub_and_fetch (p, 2); bar ((void *) p); x += __sync_and_and_fetch (p, 7); bar ((void *) p); x += __sync_or_and_fetch (p, 1); bar ((void *) p); x += __sync_xor_and_fetch (p, 2); bar ((void *) p); x += __sync_nand_and_fetch (p, 4); bar ((void *) p); x += __sync_bool_compare_and_swap (p, 7, 3); bar ((void *) p); x += __sync_val_compare_and_swap (p, 7, 3); bar ((void *) p); x += __sync_lock_test_and_set (p, 9); bar ((void *) p); __sync_lock_release (p); bar ((void *) p); __sync_synchronize (); bar ((void *) p); x += __atomic_exchange_n (p, 9, __ATOMIC_RELAXED); bar ((void *) p); x += __atomic_exchange_n (p, 9, __ATOMIC_SEQ_CST); bar ((void *) p); x += __atomic_load_n (p, __ATOMIC_RELAXED); bar ((void *) p); x += __atomic_load_n (p, __ATOMIC_SEQ_CST); bar ((void *) p); y = 6; x += __atomic_compare_exchange_n (p, &y, 8, true, __ATOMIC_RELAXED, __ATOMIC_RELAXED); bar ((void *) p); y = 6; x += __atomic_compare_exchange_n (p, &y, 8, false, __ATOMIC_RELAXED, __ATOMIC_RELAXED); bar ((void *) p); y = 6; x += __atomic_compare_exchange_n (p, &y, 8, var, __ATOMIC_RELAXED, __ATOMIC_RELAXED); bar ((void *) p); y = 6; x += __atomic_compare_exchange_n (p, &y, 8, true, __ATOMIC_SEQ_CST, __ATOMIC_SEQ_CST); bar ((void *) p); y = 6; x += __atomic_compare_exchange_n (p, &y, 8, false, __ATOMIC_SEQ_CST, __ATOMIC_SEQ_CST); bar ((void *) p); y = 6; x += __atomic_compare_exchange_n (p, &y, 8, var, __ATOMIC_SEQ_CST, __ATOMIC_SEQ_CST); bar ((void *) p); __atomic_store_n (p, 9, __ATOMIC_RELAXED); bar ((void *) p); __atomic_store_n (p, 9, __ATOMIC_SEQ_CST); bar ((void *) p); x += __atomic_add_fetch (p, 1, __ATOMIC_RELAXED); bar ((void *) p); x += __atomic_add_fetch (p, 1, __ATOMIC_SEQ_CST); bar ((void *) p); x += __atomic_sub_fetch (p, 2, __ATOMIC_RELAXED); bar ((void *) p); x += __atomic_sub_fetch (p, 2, __ATOMIC_SEQ_CST); bar ((void *) p); x += __atomic_and_fetch (p, 7, __ATOMIC_RELAXED); bar ((void *) p); x += __atomic_and_fetch (p, 7, __ATOMIC_SEQ_CST); bar ((void *) p); x += __atomic_or_fetch (p, 1, __ATOMIC_RELAXED); bar ((void *) p); x += __atomic_or_fetch (p, 1, __ATOMIC_SEQ_CST); bar ((void *) p); x += __atomic_xor_fetch (p, 2, __ATOMIC_RELAXED); bar ((void *) p); x += __atomic_xor_fetch (p, 2, __ATOMIC_SEQ_CST); bar ((void *) p); x += __atomic_nand_fetch (p, 4, __ATOMIC_RELAXED); bar ((void *) p); x += __atomic_nand_fetch (p, 4, __ATOMIC_SEQ_CST); bar ((void *) p); x += __atomic_fetch_add (p, 1, __ATOMIC_RELAXED); bar ((void *) p); x += __atomic_fetch_add (p, 1, __ATOMIC_SEQ_CST); bar ((void *) p); x += __atomic_fetch_sub (p, 2, __ATOMIC_RELAXED); bar ((void *) p); x += __atomic_fetch_sub (p, 2, __ATOMIC_SEQ_CST); bar ((void *) p); x += __atomic_fetch_and (p, 7, __ATOMIC_RELAXED); bar ((void *) p); x += __atomic_fetch_and (p, 7, __ATOMIC_SEQ_CST); bar ((void *) p); x += __atomic_fetch_or (p, 1, __ATOMIC_RELAXED); bar ((void *) p); x += __atomic_fetch_or (p, 1, __ATOMIC_SEQ_CST); bar ((void *) p); x += __atomic_fetch_xor (p, 2, __ATOMIC_RELAXED); bar ((void *) p); x += __atomic_fetch_xor (p, 2, __ATOMIC_SEQ_CST); bar ((void *) p); x += __atomic_fetch_nand (p, 4, __ATOMIC_RELAXED); bar ((void *) p); x += __atomic_fetch_nand (p, 4, __ATOMIC_SEQ_CST); bar ((void *) p); __atomic_thread_fence (__ATOMIC_RELAXED); bar ((void *) p); __atomic_thread_fence (__ATOMIC_SEQ_CST); bar ((void *) p); __atomic_signal_fence (__ATOMIC_RELAXED); bar ((void *) p); __atomic_signal_fence (__ATOMIC_SEQ_CST); bar ((void *) p); return x; } void baz (bool var) { unsigned char c = 0; unsigned short s = 0; unsigned int i = 0; unsigned long l = 0; unsigned __int128 q = 0; foo (&c, var); foo (&s, var); foo (&i, var); foo (&l, var); foo (&q, var); } --- gcc/Makefile.in.jj 2012-11-23 10:31:37.861377311 +0100 +++ gcc/Makefile.in 2012-11-23 13:36:00.578761997 +0100 @@ -2234,7 +2234,8 @@ tsan.o : $(CONFIG_H) $(SYSTEM_H) $(TREE_ $(TM_H) coretypes.h $(TREE_DUMP_H) $(TREE_PASS_H) $(CGRAPH_H) $(GGC_H) \ $(BASIC_BLOCK_H) $(FLAGS_H) $(FUNCTION_H) \ $(TM_P_H) $(TREE_FLOW_H) $(DIAGNOSTIC_CORE_H) $(GIMPLE_H) tree-iterator.h \ - intl.h cfghooks.h output.h options.h c-family/c-common.h tsan.h asan.h + intl.h cfghooks.h output.h options.h c-family/c-common.h tsan.h asan.h \ + tree-ssa-propagate.h tree-ssa-tail-merge.o: tree-ssa-tail-merge.c \ $(SYSTEM_H) $(CONFIG_H) coretypes.h $(TM_H) $(BITMAP_H) \ $(FLAGS_H) $(TM_P_H) $(BASIC_BLOCK_H) \ --- gcc/sanitizer.def.jj 2012-11-23 10:31:37.859377232 +0100 +++ gcc/sanitizer.def 2012-11-23 13:36:00.576761947 +0100 @@ -57,3 +57,148 @@ DEF_SANITIZER_BUILTIN(BUILT_IN_TSAN_WRIT BT_FN_VOID_PTR, ATTR_NOTHROW_LEAF_LIST) DEF_SANITIZER_BUILTIN(BUILT_IN_TSAN_WRITE16, "__tsan_write16", BT_FN_VOID_PTR, ATTR_NOTHROW_LEAF_LIST) + +DEF_SANITIZER_BUILTIN(BUILT_IN_TSAN_ATOMIC8_LOAD, + "__tsan_atomic8_load", + BT_FN_I1_CONST_VPTR_INT, ATTR_NOTHROW_LEAF_LIST) +DEF_SANITIZER_BUILTIN(BUILT_IN_TSAN_ATOMIC16_LOAD, + "__tsan_atomic16_load", + BT_FN_I2_CONST_VPTR_INT, ATTR_NOTHROW_LEAF_LIST) +DEF_SANITIZER_BUILTIN(BUILT_IN_TSAN_ATOMIC32_LOAD, + "__tsan_atomic32_load", + BT_FN_I4_CONST_VPTR_INT, ATTR_NOTHROW_LEAF_LIST) +DEF_SANITIZER_BUILTIN(BUILT_IN_TSAN_ATOMIC64_LOAD, + "__tsan_atomic64_load", + BT_FN_I8_CONST_VPTR_INT, ATTR_NOTHROW_LEAF_LIST) + +DEF_SANITIZER_BUILTIN(BUILT_IN_TSAN_ATOMIC8_STORE, + "__tsan_atomic8_store", + BT_FN_VOID_VPTR_I1_INT, ATTR_NOTHROW_LEAF_LIST) +DEF_SANITIZER_BUILTIN(BUILT_IN_TSAN_ATOMIC16_STORE, + "__tsan_atomic16_store", + BT_FN_VOID_VPTR_I2_INT, ATTR_NOTHROW_LEAF_LIST) +DEF_SANITIZER_BUILTIN(BUILT_IN_TSAN_ATOMIC32_STORE, + "__tsan_atomic32_store", + BT_FN_VOID_VPTR_I4_INT, ATTR_NOTHROW_LEAF_LIST) +DEF_SANITIZER_BUILTIN(BUILT_IN_TSAN_ATOMIC64_STORE, + "__tsan_atomic64_store", + BT_FN_VOID_VPTR_I8_INT, ATTR_NOTHROW_LEAF_LIST) + +DEF_SANITIZER_BUILTIN(BUILT_IN_TSAN_ATOMIC8_EXCHANGE, + "__tsan_atomic8_exchange", + BT_FN_I1_VPTR_I1_INT, ATTR_NOTHROW_LEAF_LIST) +DEF_SANITIZER_BUILTIN(BUILT_IN_TSAN_ATOMIC16_EXCHANGE, + "__tsan_atomic16_exchange", + BT_FN_I2_VPTR_I2_INT, ATTR_NOTHROW_LEAF_LIST) +DEF_SANITIZER_BUILTIN(BUILT_IN_TSAN_ATOMIC32_EXCHANGE, + "__tsan_atomic32_exchange", + BT_FN_I4_VPTR_I4_INT, ATTR_NOTHROW_LEAF_LIST) +DEF_SANITIZER_BUILTIN(BUILT_IN_TSAN_ATOMIC64_EXCHANGE, + "__tsan_atomic64_exchange", + BT_FN_I8_VPTR_I8_INT, ATTR_NOTHROW_LEAF_LIST) + +DEF_SANITIZER_BUILTIN(BUILT_IN_TSAN_ATOMIC8_FETCH_ADD, + "__tsan_atomic8_fetch_add", + BT_FN_I1_VPTR_I1_INT, ATTR_NOTHROW_LEAF_LIST) +DEF_SANITIZER_BUILTIN(BUILT_IN_TSAN_ATOMIC16_FETCH_ADD, + "__tsan_atomic16_fetch_add", + BT_FN_I2_VPTR_I2_INT, ATTR_NOTHROW_LEAF_LIST) +DEF_SANITIZER_BUILTIN(BUILT_IN_TSAN_ATOMIC32_FETCH_ADD, + "__tsan_atomic32_fetch_add", + BT_FN_I4_VPTR_I4_INT, ATTR_NOTHROW_LEAF_LIST) +DEF_SANITIZER_BUILTIN(BUILT_IN_TSAN_ATOMIC64_FETCH_ADD, + "__tsan_atomic64_fetch_add", + BT_FN_I8_VPTR_I8_INT, ATTR_NOTHROW_LEAF_LIST) + +DEF_SANITIZER_BUILTIN(BUILT_IN_TSAN_ATOMIC8_FETCH_SUB, + "__tsan_atomic8_fetch_sub", + BT_FN_I1_VPTR_I1_INT, ATTR_NOTHROW_LEAF_LIST) +DEF_SANITIZER_BUILTIN(BUILT_IN_TSAN_ATOMIC16_FETCH_SUB, + "__tsan_atomic16_fetch_sub", + BT_FN_I2_VPTR_I2_INT, ATTR_NOTHROW_LEAF_LIST) +DEF_SANITIZER_BUILTIN(BUILT_IN_TSAN_ATOMIC32_FETCH_SUB, + "__tsan_atomic32_fetch_sub", + BT_FN_I4_VPTR_I4_INT, ATTR_NOTHROW_LEAF_LIST) +DEF_SANITIZER_BUILTIN(BUILT_IN_TSAN_ATOMIC64_FETCH_SUB, + "__tsan_atomic64_fetch_sub", + BT_FN_I8_VPTR_I8_INT, ATTR_NOTHROW_LEAF_LIST) + +DEF_SANITIZER_BUILTIN(BUILT_IN_TSAN_ATOMIC8_FETCH_AND, + "__tsan_atomic8_fetch_and", + BT_FN_I1_VPTR_I1_INT, ATTR_NOTHROW_LEAF_LIST) +DEF_SANITIZER_BUILTIN(BUILT_IN_TSAN_ATOMIC16_FETCH_AND, + "__tsan_atomic16_fetch_and", + BT_FN_I2_VPTR_I2_INT, ATTR_NOTHROW_LEAF_LIST) +DEF_SANITIZER_BUILTIN(BUILT_IN_TSAN_ATOMIC32_FETCH_AND, + "__tsan_atomic32_fetch_and", + BT_FN_I4_VPTR_I4_INT, ATTR_NOTHROW_LEAF_LIST) +DEF_SANITIZER_BUILTIN(BUILT_IN_TSAN_ATOMIC64_FETCH_AND, + "__tsan_atomic64_fetch_and", + BT_FN_I8_VPTR_I8_INT, ATTR_NOTHROW_LEAF_LIST) + +DEF_SANITIZER_BUILTIN(BUILT_IN_TSAN_ATOMIC8_FETCH_OR, + "__tsan_atomic8_fetch_or", + BT_FN_I1_VPTR_I1_INT, ATTR_NOTHROW_LEAF_LIST) +DEF_SANITIZER_BUILTIN(BUILT_IN_TSAN_ATOMIC16_FETCH_OR, + "__tsan_atomic16_fetch_or", + BT_FN_I2_VPTR_I2_INT, ATTR_NOTHROW_LEAF_LIST) +DEF_SANITIZER_BUILTIN(BUILT_IN_TSAN_ATOMIC32_FETCH_OR, + "__tsan_atomic32_fetch_or", + BT_FN_I4_VPTR_I4_INT, ATTR_NOTHROW_LEAF_LIST) +DEF_SANITIZER_BUILTIN(BUILT_IN_TSAN_ATOMIC64_FETCH_OR, + "__tsan_atomic64_fetch_or", + BT_FN_I8_VPTR_I8_INT, ATTR_NOTHROW_LEAF_LIST) + +DEF_SANITIZER_BUILTIN(BUILT_IN_TSAN_ATOMIC8_FETCH_XOR, + "__tsan_atomic8_fetch_xor", + BT_FN_I1_VPTR_I1_INT, ATTR_NOTHROW_LEAF_LIST) +DEF_SANITIZER_BUILTIN(BUILT_IN_TSAN_ATOMIC16_FETCH_XOR, + "__tsan_atomic16_fetch_xor", + BT_FN_I2_VPTR_I2_INT, ATTR_NOTHROW_LEAF_LIST) +DEF_SANITIZER_BUILTIN(BUILT_IN_TSAN_ATOMIC32_FETCH_XOR, + "__tsan_atomic32_fetch_xor", + BT_FN_I4_VPTR_I4_INT, ATTR_NOTHROW_LEAF_LIST) +DEF_SANITIZER_BUILTIN(BUILT_IN_TSAN_ATOMIC64_FETCH_XOR, + "__tsan_atomic64_fetch_xor", + BT_FN_I8_VPTR_I8_INT, ATTR_NOTHROW_LEAF_LIST) + +DEF_SANITIZER_BUILTIN(BUILT_IN_TSAN_ATOMIC8_COMPARE_EXCHANGE_STRONG, + "__tsan_atomic8_compare_exchange_strong", + BT_FN_BOOL_VPTR_PTR_I1_INT, + ATTR_NOTHROW_LEAF_LIST) +DEF_SANITIZER_BUILTIN(BUILT_IN_TSAN_ATOMIC16_COMPARE_EXCHANGE_STRONG, + "__tsan_atomic16_compare_exchange_strong", + BT_FN_BOOL_VPTR_PTR_I2_INT, + ATTR_NOTHROW_LEAF_LIST) +DEF_SANITIZER_BUILTIN(BUILT_IN_TSAN_ATOMIC32_COMPARE_EXCHANGE_STRONG, + "__tsan_atomic32_compare_exchange_strong", + BT_FN_BOOL_VPTR_PTR_I4_INT, + ATTR_NOTHROW_LEAF_LIST) +DEF_SANITIZER_BUILTIN(BUILT_IN_TSAN_ATOMIC64_COMPARE_EXCHANGE_STRONG, + "__tsan_atomic64_compare_exchange_strong", + BT_FN_BOOL_VPTR_PTR_I8_INT, + ATTR_NOTHROW_LEAF_LIST) + +DEF_SANITIZER_BUILTIN(BUILT_IN_TSAN_ATOMIC8_COMPARE_EXCHANGE_WEAK, + "__tsan_atomic8_compare_exchange_weak", + BT_FN_BOOL_VPTR_PTR_I1_INT, + ATTR_NOTHROW_LEAF_LIST) +DEF_SANITIZER_BUILTIN(BUILT_IN_TSAN_ATOMIC16_COMPARE_EXCHANGE_WEAK, + "__tsan_atomic16_compare_exchange_weak", + BT_FN_BOOL_VPTR_PTR_I2_INT, + ATTR_NOTHROW_LEAF_LIST) +DEF_SANITIZER_BUILTIN(BUILT_IN_TSAN_ATOMIC32_COMPARE_EXCHANGE_WEAK, + "__tsan_atomic32_compare_exchange_weak", + BT_FN_BOOL_VPTR_PTR_I4_INT, + ATTR_NOTHROW_LEAF_LIST) +DEF_SANITIZER_BUILTIN(BUILT_IN_TSAN_ATOMIC64_COMPARE_EXCHANGE_WEAK, + "__tsan_atomic64_compare_exchange_weak", + BT_FN_BOOL_VPTR_PTR_I8_INT, + ATTR_NOTHROW_LEAF_LIST) + +DEF_SANITIZER_BUILTIN(BUILT_IN_TSAN_ATOMIC_THREAD_FENCE, + "__tsan_atomic_thread_fence", + BT_FN_VOID_INT, ATTR_NOTHROW_LEAF_LIST) +DEF_SANITIZER_BUILTIN(BUILT_IN_TSAN_ATOMIC_SIGNAL_FENCE, + "__tsan_atomic_signal_fence", + BT_FN_VOID_INT, ATTR_NOTHROW_LEAF_LIST) --- gcc/asan.c.jj 2012-11-23 10:31:37.863377378 +0100 +++ gcc/asan.c 2012-11-23 13:36:00.579762019 +0100 @@ -1484,6 +1484,53 @@ initialize_sanitizer_builtins (void) = build_function_type_list (void_type_node, ptr_type_node, build_nonstandard_integer_type (POINTER_SIZE, 1), NULL_TREE); + tree BT_FN_VOID_INT + = build_function_type_list (void_type_node, integer_type_node, NULL_TREE); + tree BT_FN_BOOL_VPTR_PTR_IX_INT[4]; + tree BT_FN_IX_CONST_VPTR_INT[4]; + tree BT_FN_IX_VPTR_IX_INT[4]; + tree BT_FN_VOID_VPTR_IX_INT[4]; + tree vptr + = build_pointer_type (build_qualified_type (void_type_node, + TYPE_QUAL_VOLATILE)); + tree cvptr + = build_pointer_type (build_qualified_type (void_type_node, + TYPE_QUAL_VOLATILE + |TYPE_QUAL_CONST)); + tree boolt + = lang_hooks.types.type_for_size (BOOL_TYPE_SIZE, 1); + int i; + for (i = 0; i < 4; i++) + { + tree ix = build_nonstandard_integer_type (BITS_PER_UNIT * (1 << i), 1); + BT_FN_BOOL_VPTR_PTR_IX_INT[i] + = build_function_type_list (boolt, vptr, ptr_type_node, ix, + integer_type_node, NULL_TREE); + BT_FN_IX_CONST_VPTR_INT[i] + = build_function_type_list (ix, cvptr, integer_type_node, NULL_TREE); + BT_FN_IX_VPTR_IX_INT[i] + = build_function_type_list (ix, vptr, ix, integer_type_node, + NULL_TREE); + BT_FN_VOID_VPTR_IX_INT[i] + = build_function_type_list (void_type_node, vptr, ix, + integer_type_node, NULL_TREE); + } +#define BT_FN_BOOL_VPTR_PTR_I1_INT BT_FN_BOOL_VPTR_PTR_IX_INT[0] +#define BT_FN_I1_CONST_VPTR_INT BT_FN_IX_CONST_VPTR_INT[0] +#define BT_FN_I1_VPTR_I1_INT BT_FN_IX_VPTR_IX_INT[0] +#define BT_FN_VOID_VPTR_I1_INT BT_FN_VOID_VPTR_IX_INT[0] +#define BT_FN_BOOL_VPTR_PTR_I2_INT BT_FN_BOOL_VPTR_PTR_IX_INT[1] +#define BT_FN_I2_CONST_VPTR_INT BT_FN_IX_CONST_VPTR_INT[1] +#define BT_FN_I2_VPTR_I2_INT BT_FN_IX_VPTR_IX_INT[1] +#define BT_FN_VOID_VPTR_I2_INT BT_FN_VOID_VPTR_IX_INT[1] +#define BT_FN_BOOL_VPTR_PTR_I4_INT BT_FN_BOOL_VPTR_PTR_IX_INT[2] +#define BT_FN_I4_CONST_VPTR_INT BT_FN_IX_CONST_VPTR_INT[2] +#define BT_FN_I4_VPTR_I4_INT BT_FN_IX_VPTR_IX_INT[2] +#define BT_FN_VOID_VPTR_I4_INT BT_FN_VOID_VPTR_IX_INT[2] +#define BT_FN_BOOL_VPTR_PTR_I8_INT BT_FN_BOOL_VPTR_PTR_IX_INT[3] +#define BT_FN_I8_CONST_VPTR_INT BT_FN_IX_CONST_VPTR_INT[3] +#define BT_FN_I8_VPTR_I8_INT BT_FN_IX_VPTR_IX_INT[3] +#define BT_FN_VOID_VPTR_I8_INT BT_FN_VOID_VPTR_IX_INT[3] #undef ATTR_NOTHROW_LEAF_LIST #define ATTR_NOTHROW_LEAF_LIST ECF_NOTHROW | ECF_LEAF #undef ATTR_NORETURN_NOTHROW_LEAF_LIST --- gcc/tsan.c.jj 2012-11-23 13:35:06.448082211 +0100 +++ gcc/tsan.c 2012-11-23 13:50:49.750579441 +0100 @@ -37,6 +37,7 @@ along with GCC; see the file COPYING3. #include "target.h" #include "cgraph.h" #include "diagnostic.h" +#include "tree-ssa-propagate.h" #include "tsan.h" #include "asan.h" @@ -216,33 +217,370 @@ instrument_expr (gimple_stmt_iterator gs return true; } +/* Actions for sync/atomic builtin transformations. */ +enum tsan_atomic_action +{ + adjust_last, add_seq_cst, add_acquire, weak_cas, strong_cas, + bool_cas, val_cas, lock_release, fetch_op, fetch_op_seq_cst +}; + +/* Table how to map sync/atomic builtins to their corresponding + tsan equivalents. */ +static struct tsan_map_atomic +{ + enum built_in_function fcode, tsan_fcode; + enum tsan_atomic_action action; + enum tree_code code; +} tsan_atomic_table[] = +{ +#define TRANSFORM(fcode, tsan_fcode, action, code) \ + { BUILT_IN_##fcode, BUILT_IN_##tsan_fcode, action, code } +#define ADJUST_LAST(fcode, tsan_fcode) \ + TRANSFORM (fcode, tsan_fcode, adjust_last, ERROR_MARK) +#define ADD_SEQ_CST(fcode, tsan_fcode) \ + TRANSFORM (fcode, tsan_fcode, add_seq_cst, ERROR_MARK) +#define ADD_ACQUIRE(fcode, tsan_fcode) \ + TRANSFORM (fcode, tsan_fcode, add_acquire, ERROR_MARK) +#define WEAK_CAS(fcode, tsan_fcode) \ + TRANSFORM (fcode, tsan_fcode, weak_cas, ERROR_MARK) +#define STRONG_CAS(fcode, tsan_fcode) \ + TRANSFORM (fcode, tsan_fcode, strong_cas, ERROR_MARK) +#define BOOL_CAS(fcode, tsan_fcode) \ + TRANSFORM (fcode, tsan_fcode, bool_cas, ERROR_MARK) +#define VAL_CAS(fcode, tsan_fcode) \ + TRANSFORM (fcode, tsan_fcode, val_cas, ERROR_MARK) +#define LOCK_RELEASE(fcode, tsan_fcode) \ + TRANSFORM (fcode, tsan_fcode, lock_release, ERROR_MARK) +#define FETCH_OP(fcode, tsan_fcode, code) \ + TRANSFORM (fcode, tsan_fcode, fetch_op, code) +#define FETCH_OPS(fcode, tsan_fcode, code) \ + TRANSFORM (fcode, tsan_fcode, fetch_op_seq_cst, code) + + ADJUST_LAST (ATOMIC_LOAD_1, TSAN_ATOMIC8_LOAD), + ADJUST_LAST (ATOMIC_LOAD_2, TSAN_ATOMIC16_LOAD), + ADJUST_LAST (ATOMIC_LOAD_4, TSAN_ATOMIC32_LOAD), + ADJUST_LAST (ATOMIC_LOAD_8, TSAN_ATOMIC64_LOAD), + ADJUST_LAST (ATOMIC_STORE_1, TSAN_ATOMIC8_STORE), + ADJUST_LAST (ATOMIC_STORE_2, TSAN_ATOMIC16_STORE), + ADJUST_LAST (ATOMIC_STORE_4, TSAN_ATOMIC32_STORE), + ADJUST_LAST (ATOMIC_STORE_8, TSAN_ATOMIC64_STORE), + ADJUST_LAST (ATOMIC_EXCHANGE_1, TSAN_ATOMIC8_EXCHANGE), + ADJUST_LAST (ATOMIC_EXCHANGE_2, TSAN_ATOMIC16_EXCHANGE), + ADJUST_LAST (ATOMIC_EXCHANGE_4, TSAN_ATOMIC32_EXCHANGE), + ADJUST_LAST (ATOMIC_EXCHANGE_8, TSAN_ATOMIC64_EXCHANGE), + ADJUST_LAST (ATOMIC_FETCH_ADD_1, TSAN_ATOMIC8_FETCH_ADD), + ADJUST_LAST (ATOMIC_FETCH_ADD_2, TSAN_ATOMIC16_FETCH_ADD), + ADJUST_LAST (ATOMIC_FETCH_ADD_4, TSAN_ATOMIC32_FETCH_ADD), + ADJUST_LAST (ATOMIC_FETCH_ADD_8, TSAN_ATOMIC64_FETCH_ADD), + ADJUST_LAST (ATOMIC_FETCH_SUB_1, TSAN_ATOMIC8_FETCH_SUB), + ADJUST_LAST (ATOMIC_FETCH_SUB_2, TSAN_ATOMIC16_FETCH_SUB), + ADJUST_LAST (ATOMIC_FETCH_SUB_4, TSAN_ATOMIC32_FETCH_SUB), + ADJUST_LAST (ATOMIC_FETCH_SUB_8, TSAN_ATOMIC64_FETCH_SUB), + ADJUST_LAST (ATOMIC_FETCH_AND_1, TSAN_ATOMIC8_FETCH_AND), + ADJUST_LAST (ATOMIC_FETCH_AND_2, TSAN_ATOMIC16_FETCH_AND), + ADJUST_LAST (ATOMIC_FETCH_AND_4, TSAN_ATOMIC32_FETCH_AND), + ADJUST_LAST (ATOMIC_FETCH_AND_8, TSAN_ATOMIC64_FETCH_AND), + ADJUST_LAST (ATOMIC_FETCH_OR_1, TSAN_ATOMIC8_FETCH_OR), + ADJUST_LAST (ATOMIC_FETCH_OR_2, TSAN_ATOMIC16_FETCH_OR), + ADJUST_LAST (ATOMIC_FETCH_OR_4, TSAN_ATOMIC32_FETCH_OR), + ADJUST_LAST (ATOMIC_FETCH_OR_8, TSAN_ATOMIC64_FETCH_OR), + ADJUST_LAST (ATOMIC_FETCH_XOR_1, TSAN_ATOMIC8_FETCH_XOR), + ADJUST_LAST (ATOMIC_FETCH_XOR_2, TSAN_ATOMIC16_FETCH_XOR), + ADJUST_LAST (ATOMIC_FETCH_XOR_4, TSAN_ATOMIC32_FETCH_XOR), + ADJUST_LAST (ATOMIC_FETCH_XOR_8, TSAN_ATOMIC64_FETCH_XOR), + + ADJUST_LAST (ATOMIC_THREAD_FENCE, TSAN_ATOMIC_THREAD_FENCE), + ADJUST_LAST (ATOMIC_SIGNAL_FENCE, TSAN_ATOMIC_SIGNAL_FENCE), + + FETCH_OP (ATOMIC_ADD_FETCH_1, TSAN_ATOMIC8_FETCH_ADD, PLUS_EXPR), + FETCH_OP (ATOMIC_ADD_FETCH_2, TSAN_ATOMIC16_FETCH_ADD, PLUS_EXPR), + FETCH_OP (ATOMIC_ADD_FETCH_4, TSAN_ATOMIC32_FETCH_ADD, PLUS_EXPR), + FETCH_OP (ATOMIC_ADD_FETCH_8, TSAN_ATOMIC64_FETCH_ADD, PLUS_EXPR), + FETCH_OP (ATOMIC_SUB_FETCH_1, TSAN_ATOMIC8_FETCH_SUB, MINUS_EXPR), + FETCH_OP (ATOMIC_SUB_FETCH_2, TSAN_ATOMIC16_FETCH_SUB, MINUS_EXPR), + FETCH_OP (ATOMIC_SUB_FETCH_4, TSAN_ATOMIC32_FETCH_SUB, MINUS_EXPR), + FETCH_OP (ATOMIC_SUB_FETCH_8, TSAN_ATOMIC64_FETCH_SUB, MINUS_EXPR), + FETCH_OP (ATOMIC_AND_FETCH_1, TSAN_ATOMIC8_FETCH_AND, BIT_AND_EXPR), + FETCH_OP (ATOMIC_AND_FETCH_2, TSAN_ATOMIC16_FETCH_AND, BIT_AND_EXPR), + FETCH_OP (ATOMIC_AND_FETCH_4, TSAN_ATOMIC32_FETCH_AND, BIT_AND_EXPR), + FETCH_OP (ATOMIC_AND_FETCH_8, TSAN_ATOMIC64_FETCH_AND, BIT_AND_EXPR), + FETCH_OP (ATOMIC_OR_FETCH_1, TSAN_ATOMIC8_FETCH_OR, BIT_IOR_EXPR), + FETCH_OP (ATOMIC_OR_FETCH_2, TSAN_ATOMIC16_FETCH_OR, BIT_IOR_EXPR), + FETCH_OP (ATOMIC_OR_FETCH_4, TSAN_ATOMIC32_FETCH_OR, BIT_IOR_EXPR), + FETCH_OP (ATOMIC_OR_FETCH_8, TSAN_ATOMIC64_FETCH_OR, BIT_IOR_EXPR), + FETCH_OP (ATOMIC_XOR_FETCH_1, TSAN_ATOMIC8_FETCH_XOR, BIT_XOR_EXPR), + FETCH_OP (ATOMIC_XOR_FETCH_2, TSAN_ATOMIC16_FETCH_XOR, BIT_XOR_EXPR), + FETCH_OP (ATOMIC_XOR_FETCH_4, TSAN_ATOMIC32_FETCH_XOR, BIT_XOR_EXPR), + FETCH_OP (ATOMIC_XOR_FETCH_8, TSAN_ATOMIC64_FETCH_XOR, BIT_XOR_EXPR), + + ADD_ACQUIRE (SYNC_LOCK_TEST_AND_SET_1, TSAN_ATOMIC8_EXCHANGE), + ADD_ACQUIRE (SYNC_LOCK_TEST_AND_SET_2, TSAN_ATOMIC16_EXCHANGE), + ADD_ACQUIRE (SYNC_LOCK_TEST_AND_SET_4, TSAN_ATOMIC32_EXCHANGE), + ADD_ACQUIRE (SYNC_LOCK_TEST_AND_SET_8, TSAN_ATOMIC64_EXCHANGE), + + ADD_SEQ_CST (SYNC_FETCH_AND_ADD_1, TSAN_ATOMIC8_FETCH_ADD), + ADD_SEQ_CST (SYNC_FETCH_AND_ADD_2, TSAN_ATOMIC16_FETCH_ADD), + ADD_SEQ_CST (SYNC_FETCH_AND_ADD_4, TSAN_ATOMIC32_FETCH_ADD), + ADD_SEQ_CST (SYNC_FETCH_AND_ADD_8, TSAN_ATOMIC64_FETCH_ADD), + ADD_SEQ_CST (SYNC_FETCH_AND_SUB_1, TSAN_ATOMIC8_FETCH_SUB), + ADD_SEQ_CST (SYNC_FETCH_AND_SUB_2, TSAN_ATOMIC16_FETCH_SUB), + ADD_SEQ_CST (SYNC_FETCH_AND_SUB_4, TSAN_ATOMIC32_FETCH_SUB), + ADD_SEQ_CST (SYNC_FETCH_AND_SUB_8, TSAN_ATOMIC64_FETCH_SUB), + ADD_SEQ_CST (SYNC_FETCH_AND_AND_1, TSAN_ATOMIC8_FETCH_AND), + ADD_SEQ_CST (SYNC_FETCH_AND_AND_2, TSAN_ATOMIC16_FETCH_AND), + ADD_SEQ_CST (SYNC_FETCH_AND_AND_4, TSAN_ATOMIC32_FETCH_AND), + ADD_SEQ_CST (SYNC_FETCH_AND_AND_8, TSAN_ATOMIC64_FETCH_AND), + ADD_SEQ_CST (SYNC_FETCH_AND_OR_1, TSAN_ATOMIC8_FETCH_OR), + ADD_SEQ_CST (SYNC_FETCH_AND_OR_2, TSAN_ATOMIC16_FETCH_OR), + ADD_SEQ_CST (SYNC_FETCH_AND_OR_4, TSAN_ATOMIC32_FETCH_OR), + ADD_SEQ_CST (SYNC_FETCH_AND_OR_8, TSAN_ATOMIC64_FETCH_OR), + ADD_SEQ_CST (SYNC_FETCH_AND_XOR_1, TSAN_ATOMIC8_FETCH_XOR), + ADD_SEQ_CST (SYNC_FETCH_AND_XOR_2, TSAN_ATOMIC16_FETCH_XOR), + ADD_SEQ_CST (SYNC_FETCH_AND_XOR_4, TSAN_ATOMIC32_FETCH_XOR), + ADD_SEQ_CST (SYNC_FETCH_AND_XOR_8, TSAN_ATOMIC64_FETCH_XOR), + + ADD_SEQ_CST (SYNC_SYNCHRONIZE, TSAN_ATOMIC_THREAD_FENCE), + + FETCH_OPS (SYNC_ADD_AND_FETCH_1, TSAN_ATOMIC8_FETCH_ADD, PLUS_EXPR), + FETCH_OPS (SYNC_ADD_AND_FETCH_2, TSAN_ATOMIC16_FETCH_ADD, PLUS_EXPR), + FETCH_OPS (SYNC_ADD_AND_FETCH_4, TSAN_ATOMIC32_FETCH_ADD, PLUS_EXPR), + FETCH_OPS (SYNC_ADD_AND_FETCH_8, TSAN_ATOMIC64_FETCH_ADD, PLUS_EXPR), + FETCH_OPS (SYNC_SUB_AND_FETCH_1, TSAN_ATOMIC8_FETCH_SUB, MINUS_EXPR), + FETCH_OPS (SYNC_SUB_AND_FETCH_2, TSAN_ATOMIC16_FETCH_SUB, MINUS_EXPR), + FETCH_OPS (SYNC_SUB_AND_FETCH_4, TSAN_ATOMIC32_FETCH_SUB, MINUS_EXPR), + FETCH_OPS (SYNC_SUB_AND_FETCH_8, TSAN_ATOMIC64_FETCH_SUB, MINUS_EXPR), + FETCH_OPS (SYNC_AND_AND_FETCH_1, TSAN_ATOMIC8_FETCH_AND, BIT_AND_EXPR), + FETCH_OPS (SYNC_AND_AND_FETCH_2, TSAN_ATOMIC16_FETCH_AND, BIT_AND_EXPR), + FETCH_OPS (SYNC_AND_AND_FETCH_4, TSAN_ATOMIC32_FETCH_AND, BIT_AND_EXPR), + FETCH_OPS (SYNC_AND_AND_FETCH_8, TSAN_ATOMIC64_FETCH_AND, BIT_AND_EXPR), + FETCH_OPS (SYNC_OR_AND_FETCH_1, TSAN_ATOMIC8_FETCH_OR, BIT_IOR_EXPR), + FETCH_OPS (SYNC_OR_AND_FETCH_2, TSAN_ATOMIC16_FETCH_OR, BIT_IOR_EXPR), + FETCH_OPS (SYNC_OR_AND_FETCH_4, TSAN_ATOMIC32_FETCH_OR, BIT_IOR_EXPR), + FETCH_OPS (SYNC_OR_AND_FETCH_8, TSAN_ATOMIC64_FETCH_OR, BIT_IOR_EXPR), + FETCH_OPS (SYNC_XOR_AND_FETCH_1, TSAN_ATOMIC8_FETCH_XOR, BIT_XOR_EXPR), + FETCH_OPS (SYNC_XOR_AND_FETCH_2, TSAN_ATOMIC16_FETCH_XOR, BIT_XOR_EXPR), + FETCH_OPS (SYNC_XOR_AND_FETCH_4, TSAN_ATOMIC32_FETCH_XOR, BIT_XOR_EXPR), + FETCH_OPS (SYNC_XOR_AND_FETCH_8, TSAN_ATOMIC64_FETCH_XOR, BIT_XOR_EXPR), + + WEAK_CAS (ATOMIC_COMPARE_EXCHANGE_1, TSAN_ATOMIC8_COMPARE_EXCHANGE_WEAK), + WEAK_CAS (ATOMIC_COMPARE_EXCHANGE_2, TSAN_ATOMIC16_COMPARE_EXCHANGE_WEAK), + WEAK_CAS (ATOMIC_COMPARE_EXCHANGE_4, TSAN_ATOMIC32_COMPARE_EXCHANGE_WEAK), + WEAK_CAS (ATOMIC_COMPARE_EXCHANGE_8, TSAN_ATOMIC64_COMPARE_EXCHANGE_WEAK), + + STRONG_CAS (ATOMIC_COMPARE_EXCHANGE_1, TSAN_ATOMIC8_COMPARE_EXCHANGE_STRONG), + STRONG_CAS (ATOMIC_COMPARE_EXCHANGE_2, + TSAN_ATOMIC16_COMPARE_EXCHANGE_STRONG), + STRONG_CAS (ATOMIC_COMPARE_EXCHANGE_4, + TSAN_ATOMIC32_COMPARE_EXCHANGE_STRONG), + STRONG_CAS (ATOMIC_COMPARE_EXCHANGE_8, + TSAN_ATOMIC64_COMPARE_EXCHANGE_STRONG), + + BOOL_CAS (SYNC_BOOL_COMPARE_AND_SWAP_1, + TSAN_ATOMIC8_COMPARE_EXCHANGE_STRONG), + BOOL_CAS (SYNC_BOOL_COMPARE_AND_SWAP_2, + TSAN_ATOMIC16_COMPARE_EXCHANGE_STRONG), + BOOL_CAS (SYNC_BOOL_COMPARE_AND_SWAP_4, + TSAN_ATOMIC32_COMPARE_EXCHANGE_STRONG), + BOOL_CAS (SYNC_BOOL_COMPARE_AND_SWAP_8, + TSAN_ATOMIC64_COMPARE_EXCHANGE_STRONG), + + VAL_CAS (SYNC_VAL_COMPARE_AND_SWAP_1, TSAN_ATOMIC8_COMPARE_EXCHANGE_STRONG), + VAL_CAS (SYNC_VAL_COMPARE_AND_SWAP_2, TSAN_ATOMIC16_COMPARE_EXCHANGE_STRONG), + VAL_CAS (SYNC_VAL_COMPARE_AND_SWAP_4, TSAN_ATOMIC32_COMPARE_EXCHANGE_STRONG), + VAL_CAS (SYNC_VAL_COMPARE_AND_SWAP_8, TSAN_ATOMIC64_COMPARE_EXCHANGE_STRONG), + + LOCK_RELEASE (SYNC_LOCK_RELEASE_1, TSAN_ATOMIC8_STORE), + LOCK_RELEASE (SYNC_LOCK_RELEASE_2, TSAN_ATOMIC16_STORE), + LOCK_RELEASE (SYNC_LOCK_RELEASE_4, TSAN_ATOMIC32_STORE), + LOCK_RELEASE (SYNC_LOCK_RELEASE_8, TSAN_ATOMIC64_STORE) +}; + +/* Instrument an atomic builtin. */ + +static void +instrument_builtin_call (gimple_stmt_iterator *gsi) +{ + gimple stmt = gsi_stmt (*gsi), g; + tree callee = gimple_call_fndecl (stmt), last_arg, args[6], t, lhs; + enum built_in_function fcode = DECL_FUNCTION_CODE (callee); + unsigned int i, num = gimple_call_num_args (stmt), j; + for (j = 0; j < 6 && j < num; j++) + args[j] = gimple_call_arg (stmt, j); + for (i = 0; i < ARRAY_SIZE (tsan_atomic_table); i++) + if (fcode != tsan_atomic_table[i].fcode) + continue; + else + { + tree decl = builtin_decl_implicit (tsan_atomic_table[i].tsan_fcode); + if (decl == NULL_TREE) + return; + switch (tsan_atomic_table[i].action) + { + case adjust_last: + case fetch_op: + last_arg = gimple_call_arg (stmt, num - 1); + if (!host_integerp (last_arg, 1) + || (unsigned HOST_WIDE_INT) tree_low_cst (last_arg, 1) + > MEMMODEL_SEQ_CST) + return; + gimple_call_set_fndecl (stmt, decl); + gimple_call_set_arg (stmt, num - 1, + build_int_cst (NULL_TREE, + 1 << tree_low_cst (last_arg, + 1))); + update_stmt (stmt); + if (tsan_atomic_table[i].action == fetch_op) + { + args[1] = gimple_call_arg (stmt, 1); + goto adjust_result; + } + return; + case add_seq_cst: + case add_acquire: + case fetch_op_seq_cst: + gcc_assert (num <= 2); + for (j = 0; j < num; j++) + args[j] = gimple_call_arg (stmt, j); + for (; j < 2; j++) + args[j] = NULL_TREE; + args[num] = build_int_cst (NULL_TREE, + 1 << (tsan_atomic_table[i].action + != add_acquire + ? MEMMODEL_SEQ_CST + : MEMMODEL_ACQUIRE)); + update_gimple_call (gsi, decl, num + 1, args[0], args[1], args[2]); + stmt = gsi_stmt (*gsi); + if (tsan_atomic_table[i].action == fetch_op_seq_cst) + { + adjust_result: + lhs = gimple_call_lhs (stmt); + if (lhs == NULL_TREE) + return; + if (!useless_type_conversion_p (TREE_TYPE (lhs), + TREE_TYPE (args[1]))) + { + tree var = make_ssa_name (TREE_TYPE (lhs), NULL); + g = gimple_build_assign_with_ops (NOP_EXPR, var, + args[1], NULL_TREE); + gsi_insert_after (gsi, g, GSI_NEW_STMT); + args[1] = var; + } + gimple_call_set_lhs (stmt, + make_ssa_name (TREE_TYPE (lhs), NULL)); + g = gimple_build_assign_with_ops (tsan_atomic_table[i].code, + lhs, gimple_call_lhs (stmt), + args[1]); + update_stmt (stmt); + gsi_insert_after (gsi, g, GSI_NEW_STMT); + } + return; + case weak_cas: + if (!integer_nonzerop (gimple_call_arg (stmt, 3))) + continue; + /* FALLTHRU */ + case strong_cas: + gcc_assert (num == 6); + for (j = 0; j < 6; j++) + args[j] = gimple_call_arg (stmt, j); + if (!host_integerp (args[4], 1) + || (unsigned HOST_WIDE_INT) tree_low_cst (args[4], 1) + > MEMMODEL_SEQ_CST) + return; + update_gimple_call (gsi, decl, 4, args[0], args[1], args[2], + build_int_cst (NULL_TREE, + 1 << tree_low_cst (args[4], + 1))); + return; + case bool_cas: + case val_cas: + gcc_assert (num == 3); + for (j = 0; j < 3; j++) + args[j] = gimple_call_arg (stmt, j); + t = TYPE_ARG_TYPES (TREE_TYPE (decl)); + t = TREE_VALUE (TREE_CHAIN (TREE_CHAIN (t))); + t = create_tmp_var (t, NULL); + mark_addressable (t); + if (!useless_type_conversion_p (TREE_TYPE (t), + TREE_TYPE (args[1]))) + { + g = gimple_build_assign_with_ops (NOP_EXPR, + make_ssa_name (TREE_TYPE (t), + NULL), + args[1], NULL_TREE); + gsi_insert_before (gsi, g, GSI_SAME_STMT); + args[1] = gimple_assign_lhs (g); + } + g = gimple_build_assign (t, args[1]); + gsi_insert_before (gsi, g, GSI_SAME_STMT); + lhs = gimple_call_lhs (stmt); + update_gimple_call (gsi, decl, 4, args[0], + build_fold_addr_expr (t), args[2], + build_int_cst (NULL_TREE, + 1 << MEMMODEL_SEQ_CST)); + if (tsan_atomic_table[i].action == val_cas && lhs) + { + tree cond; + stmt = gsi_stmt (*gsi); + g = gimple_build_assign (make_ssa_name (TREE_TYPE (t), NULL), + t); + gsi_insert_after (gsi, g, GSI_NEW_STMT); + t = make_ssa_name (TREE_TYPE (TREE_TYPE (decl)), stmt); + cond = build2 (NE_EXPR, boolean_type_node, t, + build_int_cst (TREE_TYPE (t), 0)); + g = gimple_build_assign_with_ops (COND_EXPR, lhs, cond, + args[1], + gimple_assign_lhs (g)); + gimple_call_set_lhs (stmt, t); + update_stmt (stmt); + gsi_insert_after (gsi, g, GSI_NEW_STMT); + } + return; + case lock_release: + gcc_assert (num == 1); + t = TYPE_ARG_TYPES (TREE_TYPE (decl)); + t = TREE_VALUE (TREE_CHAIN (t)); + update_gimple_call (gsi, decl, 3, gimple_call_arg (stmt, 0), + build_int_cst (t, 0), + build_int_cst (NULL_TREE, + 1 << MEMMODEL_RELEASE)); + return; + default: + continue; + } + } +} + /* Instruments the gimple pointed to by GSI. Return true if func entry/exit should be instrumented. */ static bool -instrument_gimple (gimple_stmt_iterator gsi) +instrument_gimple (gimple_stmt_iterator *gsi) { gimple stmt; tree rhs, lhs; bool instrumented = false; - stmt = gsi_stmt (gsi); + stmt = gsi_stmt (*gsi); if (is_gimple_call (stmt) && (gimple_call_fndecl (stmt) != builtin_decl_implicit (BUILT_IN_TSAN_INIT))) - return true; + { + if (is_gimple_builtin_call (stmt)) + instrument_builtin_call (gsi); + return true; + } else if (is_gimple_assign (stmt) && !gimple_clobber_p (stmt)) { if (gimple_store_p (stmt)) { lhs = gimple_assign_lhs (stmt); - instrumented = instrument_expr (gsi, lhs, true); + instrumented = instrument_expr (*gsi, lhs, true); } if (gimple_assign_load_p (stmt)) { rhs = gimple_assign_rhs1 (stmt); - instrumented = instrument_expr (gsi, rhs, false); + instrumented = instrument_expr (*gsi, rhs, false); } } return instrumented; @@ -260,7 +598,7 @@ instrument_memory_accesses (void) FOR_EACH_BB (bb) for (gsi = gsi_start_bb (bb); !gsi_end_p (gsi); gsi_next (&gsi)) - fentry_exit_instrument |= instrument_gimple (gsi); + fentry_exit_instrument |= instrument_gimple (&gsi); return fentry_exit_instrument; } --- gcc/builtin-types.def.jj 2012-11-23 10:31:37.866377460 +0100 +++ gcc/builtin-types.def 2012-11-23 13:36:00.576761947 +0100 @@ -447,6 +447,14 @@ DEF_FUNCTION_TYPE_4 (BT_FN_VOID_SIZE_VPT BT_VOLATILE_PTR, BT_PTR, BT_INT) DEF_FUNCTION_TYPE_4 (BT_FN_VOID_SIZE_CONST_VPTR_PTR_INT, BT_VOID, BT_SIZE, BT_CONST_VOLATILE_PTR, BT_PTR, BT_INT) +DEF_FUNCTION_TYPE_4 (BT_FN_BOOL_VPTR_PTR_I1_INT, + BT_BOOL, BT_VOLATILE_PTR, BT_PTR, BT_I1, BT_INT) +DEF_FUNCTION_TYPE_4 (BT_FN_BOOL_VPTR_PTR_I2_INT, + BT_BOOL, BT_VOLATILE_PTR, BT_PTR, BT_I2, BT_INT) +DEF_FUNCTION_TYPE_4 (BT_FN_BOOL_VPTR_PTR_I4_INT, + BT_BOOL, BT_VOLATILE_PTR, BT_PTR, BT_I4, BT_INT) +DEF_FUNCTION_TYPE_4 (BT_FN_BOOL_VPTR_PTR_I8_INT, + BT_BOOL, BT_VOLATILE_PTR, BT_PTR, BT_I8, BT_INT) DEF_FUNCTION_TYPE_5 (BT_FN_INT_STRING_INT_SIZE_CONST_STRING_VALIST_ARG, BT_INT, BT_STRING, BT_INT, BT_SIZE, BT_CONST_STRING,