From patchwork Mon Jul 4 10:41:12 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Biener X-Patchwork-Id: 1651931 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=gcc.gnu.org header.i=@gcc.gnu.org header.a=rsa-sha256 header.s=default header.b=IDou8r1N; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gcc.gnu.org (client-ip=8.43.85.97; helo=sourceware.org; envelope-from=gcc-patches-bounces+incoming=patchwork.ozlabs.org@gcc.gnu.org; receiver=) Received: from sourceware.org (server2.sourceware.org [8.43.85.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4Lc2RQ4420z9s5V for ; Mon, 4 Jul 2022 20:41:37 +1000 (AEST) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 28DEE3857B8E for ; Mon, 4 Jul 2022 10:41:34 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 28DEE3857B8E DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gcc.gnu.org; s=default; t=1656931294; bh=fKWuv5AHQKSfHMJ2C5le04R+9eZRE4uXkEIQ/JtaG0c=; h=Date:To:Subject:List-Id:List-Unsubscribe:List-Archive:List-Post: List-Help:List-Subscribe:From:Reply-To:Cc:From; b=IDou8r1NFgIMJ/Yixp1NmDkOm/+cx6oR4xzsmIdZgIYTDKy8AUDWFX5e+Z6EVI62z VUaFvZtkz+3Mq/MBiftnIE7SrrfqGMR8IfQ8ICulBir3ltcEZAZRszrGRqWRj3YjNy E8bXhNoNNDYN6bbJbosTQ65iMJXPKIK4VdjDr0d0= X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by sourceware.org (Postfix) with ESMTPS id 8853D3858C52 for ; Mon, 4 Jul 2022 10:41:14 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 8853D3858C52 Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 5509222744; Mon, 4 Jul 2022 10:41:13 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 3CF051342C; Mon, 4 Jul 2022 10:41:13 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id 74OmDcnDwmJYegAAMHmgww (envelope-from ); Mon, 04 Jul 2022 10:41:13 +0000 Date: Mon, 4 Jul 2022 12:41:12 +0200 (CEST) To: gcc-patches@gcc.gnu.org Subject: [PATCH] Keep virtual SSA up-to-date in vectorizer MIME-Version: 1.0 Message-Id: <20220704104113.3CF051342C@imap2.suse-dmz.suse.de> X-Spam-Status: No, score=-11.7 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, SPF_HELO_NONE, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Richard Biener via Gcc-patches From: Richard Biener Reply-To: Richard Biener Cc: richard.sandiford@arm.com Errors-To: gcc-patches-bounces+incoming=patchwork.ozlabs.org@gcc.gnu.org Sender: "Gcc-patches" The following removes a FIXME where we fail(ed) to keep virtual SSA up-to-date, patching up the remaining two cases I managed to trigger. I've left an assert so that we pick up cases arising for the cases I wasn't able to trigger. Bootstrapped and tested on x86_64-unknown-linux-gnu, I've also built SPEC CPU 2017 with and without LTO and zen2 arch. I eventually do expect fallout, so sorry for that in advance. Pushed to trunk. Richard. 2022-07-04 Richard Biener * tree-vect-loop-manip.cc (vect_do_peeling): Assert that no SSA update is needed instead of updating virtual SSA form. * tree-vect-stmts.cc (vectorizable_load): For hoisted invariant load use the loop entry virtual use. For emulated gather loads use the virtual use of the original stmt like vect_finish_stmt_generation would do. --- gcc/tree-vect-loop-manip.cc | 11 ++++------- gcc/tree-vect-stmts.cc | 15 ++++++++++++--- 2 files changed, 16 insertions(+), 10 deletions(-) diff --git a/gcc/tree-vect-loop-manip.cc b/gcc/tree-vect-loop-manip.cc index ae5533e0f68..81e29d564d6 100644 --- a/gcc/tree-vect-loop-manip.cc +++ b/gcc/tree-vect-loop-manip.cc @@ -2683,14 +2683,11 @@ vect_do_peeling (loop_vec_info loop_vinfo, tree niters, tree nitersm1, class loop *first_loop = loop; bool irred_flag = loop_preheader_edge (loop)->flags & EDGE_IRREDUCIBLE_LOOP; - /* We might have a queued need to update virtual SSA form. As we - delete the update SSA machinery below after doing a regular + /* Historically we might have a queued need to update virtual SSA form. + As we delete the update SSA machinery below after doing a regular incremental SSA update during loop copying make sure we don't - lose that fact. - ??? Needing to update virtual SSA form by renaming is unfortunate - but not all of the vectorizer code inserting new loads / stores - properly assigns virtual operands to those statements. */ - update_ssa (TODO_update_ssa_only_virtuals); + lose that fact. */ + gcc_assert (!need_ssa_update_p (cfun)); create_lcssa_for_virtual_phi (loop); diff --git a/gcc/tree-vect-stmts.cc b/gcc/tree-vect-stmts.cc index 346d8ce2804..d6a6fe3fb38 100644 --- a/gcc/tree-vect-stmts.cc +++ b/gcc/tree-vect-stmts.cc @@ -9024,9 +9024,16 @@ vectorizable_load (vec_info *vinfo, "hoisting out of the vectorized loop: %G", stmt); scalar_dest = copy_ssa_name (scalar_dest); tree rhs = unshare_expr (gimple_assign_rhs1 (stmt)); - gsi_insert_on_edge_immediate - (loop_preheader_edge (loop), - gimple_build_assign (scalar_dest, rhs)); + edge pe = loop_preheader_edge (loop); + gphi *vphi = get_virtual_phi (loop->header); + tree vuse; + if (vphi) + vuse = PHI_ARG_DEF_FROM_EDGE (vphi, pe); + else + vuse = gimple_vuse (gsi_stmt (*gsi)); + gimple *new_stmt = gimple_build_assign (scalar_dest, rhs); + gimple_set_vuse (new_stmt, vuse); + gsi_insert_on_edge_immediate (pe, new_stmt); } /* These copies are all equivalent, but currently the representation requires a separate STMT_VINFO_VEC_STMT for each one. */ @@ -9769,6 +9776,8 @@ vectorizable_load (vec_info *vinfo, tree ref = build2 (MEM_REF, ltype, ptr, build_int_cst (ref_type, 0)); new_stmt = gimple_build_assign (elt, ref); + gimple_set_vuse (new_stmt, + gimple_vuse (gsi_stmt (*gsi))); gimple_seq_add_stmt (&stmts, new_stmt); CONSTRUCTOR_APPEND_ELT (ctor_elts, NULL_TREE, elt); }