From patchwork Wed Jun 15 07:23:00 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jakub Jelinek X-Patchwork-Id: 1643503 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=gcc.gnu.org header.i=@gcc.gnu.org header.a=rsa-sha256 header.s=default header.b=DZcpdI1r; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gcc.gnu.org (client-ip=8.43.85.97; helo=sourceware.org; envelope-from=gcc-patches-bounces+incoming=patchwork.ozlabs.org@gcc.gnu.org; receiver=) Received: from sourceware.org (server2.sourceware.org [8.43.85.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4LNGxf5N30z9vGB for ; Wed, 15 Jun 2022 17:23:33 +1000 (AEST) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id AB721385740B for ; Wed, 15 Jun 2022 07:23:29 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org AB721385740B DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gcc.gnu.org; s=default; t=1655277809; bh=VxW9OGJhFg4+7maZs/jPn/RxzHUucKCgaKz+8KekE/c=; h=Date:To:Subject:List-Id:List-Unsubscribe:List-Archive:List-Post: List-Help:List-Subscribe:From:Reply-To:Cc:From; b=DZcpdI1rYgrNdt4HzyAJFnEZnz3XJIVGUWr+P81E9TwzZnR/w8b2jRwRs/f1hD2ZA rtMpk1plsoAGoak1LmBt0r4eXYfE9OB5DpqFsk+eZnYiNcqJtQnCTIXcte6pqUWjAM 7J+HpFk0fljA5+wU5Q19MYUdiin6Hs2e5VfKpwIs= X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by sourceware.org (Postfix) with ESMTPS id 670B53858C54 for ; Wed, 15 Jun 2022 07:23:09 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 670B53858C54 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-21-v_OGTZi_MD-bYEbWB1P5pg-1; Wed, 15 Jun 2022 03:23:05 -0400 X-MC-Unique: v_OGTZi_MD-bYEbWB1P5pg-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id C67B8803520; Wed, 15 Jun 2022 07:23:04 +0000 (UTC) Received: from tucnak.zalov.cz (unknown [10.39.192.11]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 7E8B91415107; Wed, 15 Jun 2022 07:23:04 +0000 (UTC) Received: from tucnak.zalov.cz (localhost [127.0.0.1]) by tucnak.zalov.cz (8.17.1/8.17.1) with ESMTPS id 25F7N1L61109384 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NOT); Wed, 15 Jun 2022 09:23:01 +0200 Received: (from jakub@localhost) by tucnak.zalov.cz (8.17.1/8.17.1/Submit) id 25F7N1Dh1109383; Wed, 15 Jun 2022 09:23:01 +0200 Date: Wed, 15 Jun 2022 09:23:00 +0200 To: Richard Biener , Jeff Law Subject: [PATCH] expand: Fix up IFN_ATOMIC_{BIT*,*CMP_0} expansion [PR105951] Message-ID: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.85 on 10.11.54.7 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Disposition: inline X-Spam-Status: No, score=-4.2 required=5.0 tests=BAYES_00, DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, KAM_SHORT, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_NONE, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Jakub Jelinek via Gcc-patches From: Jakub Jelinek Reply-To: Jakub Jelinek Cc: gcc-patches@gcc.gnu.org Errors-To: gcc-patches-bounces+incoming=patchwork.ozlabs.org@gcc.gnu.org Sender: "Gcc-patches" Hi! Both IFN_ATOMIC_BIT_TEST_AND_* and IFN_ATOMIC_*_FETCH_CMP_0 ifns are matched if their corresponding optab is implemented for the particular mode. The fact that those optabs are implemented doesn't guarantee they will succeed though, they can just FAIL in their expansion. The expansion in that case uses expand_atomic_fetch_op as fallback, but as has been reported and and can be reproduced on the testcases, even those can fail and we didn't have any fallback after that. For IFN_ATOMIC_BIT_TEST_AND_* we actually have such calls. One is done whenever we lost lhs of the ifn at some point in between matching it in tree-ssa-ccp.cc and expansion. The following patch for that case just falls through and expands as if there was a lhs, creates a temporary for it. For the other expand_atomic_fetch_op call in the same expander and for the only expand_atomic_fetch_op call in the other, this falls back the hard way, by constructing a CALL_EXPR to the call from which the ifn has been matched and expanding that. Either it is lucky and manages to expand inline, or it emits a libatomic API call. Bootstrapped/regtested on x86_64-linux and i686-linux, ok for trunk? 2022-06-15 Jakub Jelinek PR middle-end/105951 * builtins.cc (expand_ifn_atomic_bit_test_and): If expand_atomic_fetch_op fails for the lhs == NULL_TREE case, fall through into the optab code with gen_reg_rtx (mode) as target. If second expand_atomic_fetch_op fails, construct a CALL_EXPR and expand that. (expand_ifn_atomic_op_fetch_cmp_0): If expand_atomic_fetch_op fails, construct a CALL_EXPR and expand that. * gcc.target/i386/pr105951-1.c: New test. * gcc.target/i386/pr105951-2.c: New test. Jakub --- gcc/builtins.cc.jj 2022-05-16 11:14:57.499428916 +0200 +++ gcc/builtins.cc 2022-06-14 18:30:35.564288606 +0200 @@ -6250,15 +6250,19 @@ expand_ifn_atomic_bit_test_and (gcall *c if (lhs == NULL_TREE) { - val = expand_simple_binop (mode, ASHIFT, const1_rtx, - val, NULL_RTX, true, OPTAB_DIRECT); + rtx val2 = expand_simple_binop (mode, ASHIFT, const1_rtx, + val, NULL_RTX, true, OPTAB_DIRECT); if (code == AND) - val = expand_simple_unop (mode, NOT, val, NULL_RTX, true); - expand_atomic_fetch_op (const0_rtx, mem, val, code, model, false); - return; + val2 = expand_simple_unop (mode, NOT, val2, NULL_RTX, true); + if (expand_atomic_fetch_op (const0_rtx, mem, val2, code, model, false)) + return; } - rtx target = expand_expr (lhs, NULL_RTX, VOIDmode, EXPAND_WRITE); + rtx target; + if (lhs) + target = expand_expr (lhs, NULL_RTX, VOIDmode, EXPAND_WRITE); + else + target = gen_reg_rtx (mode); enum insn_code icode = direct_optab_handler (optab, mode); gcc_assert (icode != CODE_FOR_nothing); create_output_operand (&ops[0], target, mode); @@ -6277,6 +6281,51 @@ expand_ifn_atomic_bit_test_and (gcall *c val = expand_simple_unop (mode, NOT, val, NULL_RTX, true); rtx result = expand_atomic_fetch_op (gen_reg_rtx (mode), mem, val, code, model, false); + if (!result) + { + built_in_function fcode; + switch (code) + { + case IOR: + if (model != MEMMODEL_SYNC_SEQ_CST) + fcode = BUILT_IN_ATOMIC_FETCH_OR_1; + else + fcode = BUILT_IN_SYNC_FETCH_AND_OR_1; + break; + case XOR: + if (model != MEMMODEL_SYNC_SEQ_CST) + fcode = BUILT_IN_ATOMIC_FETCH_XOR_1; + else + fcode = BUILT_IN_SYNC_FETCH_AND_XOR_1; + break; + case AND: + if (model != MEMMODEL_SYNC_SEQ_CST) + fcode = BUILT_IN_ATOMIC_FETCH_AND_1; + else + fcode = BUILT_IN_SYNC_FETCH_AND_AND_1; + break; + default: + gcc_unreachable (); + } + unsigned int bytes_log2 + = exact_log2 (GET_MODE_SIZE (mode).to_constant ()); + gcc_assert (bytes_log2 < 5); + fcode = (built_in_function) (fcode + bytes_log2); + tree fndecl = builtin_decl_explicit (fcode); + tree type = TREE_TYPE (TREE_TYPE (fndecl)); + tree tcall = build1 (ADDR_EXPR, build_pointer_type (TREE_TYPE (fndecl)), + fndecl); + tree exp = build_call_nary (type, tcall, + 2 + (model != MEMMODEL_SYNC_SEQ_CST), ptr, + make_tree (type, val), + model != MEMMODEL_SYNC_SEQ_CST + ? gimple_call_arg (call, 3) + : integer_zero_node); + result = expand_builtin (exp, gen_reg_rtx (mode), NULL_RTX, + mode, !lhs); + } + if (!lhs) + return; if (integer_onep (flag)) { result = expand_simple_binop (mode, ASHIFTRT, result, bitval, @@ -6369,6 +6418,62 @@ expand_ifn_atomic_op_fetch_cmp_0 (gcall rtx result = expand_atomic_fetch_op (gen_reg_rtx (mode), mem, op, code, model, true); + if (!result) + { + built_in_function fcode; + switch (code) + { + case PLUS: + if (model != MEMMODEL_SYNC_SEQ_CST) + fcode = BUILT_IN_ATOMIC_ADD_FETCH_1; + else + fcode = BUILT_IN_SYNC_ADD_AND_FETCH_1; + break; + case MINUS: + if (model != MEMMODEL_SYNC_SEQ_CST) + fcode = BUILT_IN_ATOMIC_SUB_FETCH_1; + else + fcode = BUILT_IN_SYNC_SUB_AND_FETCH_1; + break; + case IOR: + if (model != MEMMODEL_SYNC_SEQ_CST) + fcode = BUILT_IN_ATOMIC_OR_FETCH_1; + else + fcode = BUILT_IN_SYNC_OR_AND_FETCH_1; + break; + case XOR: + if (model != MEMMODEL_SYNC_SEQ_CST) + fcode = BUILT_IN_ATOMIC_XOR_FETCH_1; + else + fcode = BUILT_IN_SYNC_XOR_AND_FETCH_1; + break; + case AND: + if (model != MEMMODEL_SYNC_SEQ_CST) + fcode = BUILT_IN_ATOMIC_AND_FETCH_1; + else + fcode = BUILT_IN_SYNC_AND_AND_FETCH_1; + break; + default: + gcc_unreachable (); + } + unsigned int bytes_log2 + = exact_log2 (GET_MODE_SIZE (mode).to_constant ()); + gcc_assert (bytes_log2 < 5); + fcode = (built_in_function) (fcode + bytes_log2); + tree fndecl = builtin_decl_explicit (fcode); + tree type = TREE_TYPE (TREE_TYPE (fndecl)); + tree tcall = build1 (ADDR_EXPR, build_pointer_type (TREE_TYPE (fndecl)), + fndecl); + tree exp = build_call_nary (type, tcall, + 2 + (model != MEMMODEL_SYNC_SEQ_CST), ptr, + arg, + model != MEMMODEL_SYNC_SEQ_CST + ? gimple_call_arg (call, 3) + : integer_zero_node); + result = expand_builtin (exp, gen_reg_rtx (mode), NULL_RTX, + mode, !lhs); + } + if (lhs) { result = emit_store_flag_force (target, comp, result, const0_rtx, mode, --- gcc/testsuite/gcc.target/i386/pr105951-1.c.jj 2022-06-14 19:06:09.452825662 +0200 +++ gcc/testsuite/gcc.target/i386/pr105951-1.c 2022-06-14 19:05:59.630924070 +0200 @@ -0,0 +1,5 @@ +/* PR middle-end/105951 */ +/* { dg-do compile { target ia32 } } */ +/* { dg-options "-O2 -march=i386" } */ + +#include "pr98737-2.c" --- gcc/testsuite/gcc.target/i386/pr105951-2.c.jj 2022-06-14 19:06:16.607753976 +0200 +++ gcc/testsuite/gcc.target/i386/pr105951-2.c 2022-06-14 19:06:22.394695990 +0200 @@ -0,0 +1,5 @@ +/* PR middle-end/105951 */ +/* { dg-do compile { target ia32 } } */ +/* { dg-options "-O2 -march=i386" } */ + +#include "pr98737-4.c"