From patchwork Tue Aug 31 09:25:40 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Biener X-Patchwork-Id: 1522510 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gcc.gnu.org (client-ip=2620:52:3:1:0:246e:9693:128c; helo=sourceware.org; envelope-from=gcc-patches-bounces+incoming=patchwork.ozlabs.org@gcc.gnu.org; receiver=) Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=gcc.gnu.org header.i=@gcc.gnu.org header.a=rsa-sha256 header.s=default header.b=mrOI1wwd; dkim-atps=neutral Received: from sourceware.org (server2.sourceware.org [IPv6:2620:52:3:1:0:246e:9693:128c]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4GzMJZ5936z9sRN for ; Tue, 31 Aug 2021 19:26:37 +1000 (AEST) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 211ED3858414 for ; Tue, 31 Aug 2021 09:26:32 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 211ED3858414 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gcc.gnu.org; s=default; t=1630401992; bh=mM9s72M+/vw3bbOYn9Y7UGe8FEY7vWivc88yx8MxCxY=; h=Date:To:Subject:List-Id:List-Unsubscribe:List-Archive:List-Post: List-Help:List-Subscribe:From:Reply-To:Cc:From; b=mrOI1wwd2+Hlio4ORvNun6OFRMn3NbVnsI8m9IYiPpVRU2P0PtKxjjetFrT845ZAM kIB5VN7ONt6tu3jUoob+YyydZ1in21SsIMaWlAKHhEE10AFvUTO7ThIicQzn9UdlNK wXt4Vvd0REdq+MUD7XiFkRmeRBR2w8gPO5qjycdo= X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by sourceware.org (Postfix) with ESMTPS id 258BF3858408 for ; Tue, 31 Aug 2021 09:25:42 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 258BF3858408 Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id EF82722089; Tue, 31 Aug 2021 09:25:40 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id D334C13AC7; Tue, 31 Aug 2021 09:25:40 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id vaE0MpT1LWEwRQAAMHmgww (envelope-from ); Tue, 31 Aug 2021 09:25:40 +0000 Date: Tue, 31 Aug 2021 11:25:40 +0200 (CEST) To: gcc-patches@gcc.gnu.org Subject: [PATCH] tree-optimization/102139 - fix SLP DR base alignment Message-ID: MIME-Version: 1.0 X-Spam-Status: No, score=-11.5 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Richard Biener via Gcc-patches From: Richard Biener Reply-To: Richard Biener Cc: richard.sandiford@arm.com Errors-To: gcc-patches-bounces+incoming=patchwork.ozlabs.org@gcc.gnu.org Sender: "Gcc-patches" When doing whole-function SLP we have to make sure the recorded base alignments we compute as the maximum alignment seen for a base anywhere in the function is actually valid at the point we want to make use of it. To make this work we now record the stmt the alignment was derived from in addition to the DRs innermost behavior and we use a dominance check to verify the recorded info is valid when doing BB vectorization. Note this leaves a small(?) hole for the case where we have sth like unaligned DR call (); // does not return aligned DR since we'll derive an aligned access for the earlier DR but the later DR is never actually reached since the call does not return. To plug this hole one option (for the easy backporting) would be to simply not use the base-alignment recording at all. Alternatively we'd have to store the dataref grouping 'id' somewhere in the DR itself and use that to handle this particular case. For optimal handling we'd need the ability to record different base alignments based on context, we could hash on the BB and at query time walk immediate dominators to find the "best" base alignment. But I'd rather leave such improvement for trunk. Any opinions? The issue looks quite serious and IMHO warrants a timely fix, even if partial - I'm not sure how often the same-BB case would trigger. Bootstrapped on x86_64-unknown-linux-gnu, testing in progress. Thanks, Richard. 2021-08-31 Richard Biener PR tree-optimization/102139 * tree-vectorizer.h (vec_base_alignments): Adjust hash-map type to record a std::pair of the stmt and the innermost loop behavior. * tree-vect-data-refs.c (vect_record_base_alignment): Adjust. (vect_compute_data_ref_alignment): Verify the recorded base alignment can be used. --- gcc/tree-vect-data-refs.c | 19 ++++++++++++------- gcc/tree-vectorizer.h | 7 ++++--- 2 files changed, 16 insertions(+), 10 deletions(-) diff --git a/gcc/tree-vect-data-refs.c b/gcc/tree-vect-data-refs.c index 37f46d1aaa3..e2549811961 100644 --- a/gcc/tree-vect-data-refs.c +++ b/gcc/tree-vect-data-refs.c @@ -895,11 +895,11 @@ vect_record_base_alignment (vec_info *vinfo, stmt_vec_info stmt_info, innermost_loop_behavior *drb) { bool existed; - innermost_loop_behavior *&entry + std::pair &entry = vinfo->base_alignments.get_or_insert (drb->base_address, &existed); - if (!existed || entry->base_alignment < drb->base_alignment) + if (!existed || entry.second->base_alignment < drb->base_alignment) { - entry = drb; + entry = std::make_pair (stmt_info->stmt, drb); if (dump_enabled_p ()) dump_printf_loc (MSG_NOTE, vect_location, "recording new base alignment for %T\n" @@ -1060,11 +1060,16 @@ vect_compute_data_ref_alignment (vec_info *vinfo, dr_vec_info *dr_info) /* Calculate the maximum of the pooled base address alignment and the alignment that we can compute for DR itself. */ - innermost_loop_behavior **entry = base_alignments->get (drb->base_address); - if (entry && base_alignment < (*entry)->base_alignment) + std::pair *entry + = base_alignments->get (drb->base_address); + if (entry + && base_alignment < (*entry).second->base_alignment + && (loop_vinfo + || dominated_by_p (CDI_DOMINATORS, gimple_bb (stmt_info->stmt), + gimple_bb (entry->first)))) { - base_alignment = (*entry)->base_alignment; - base_misalignment = (*entry)->base_misalignment; + base_alignment = entry->second->base_alignment; + base_misalignment = entry->second->base_misalignment; } if (drb->offset_alignment < vect_align_c diff --git a/gcc/tree-vectorizer.h b/gcc/tree-vectorizer.h index 72e018e8eac..8db642c7dc3 100644 --- a/gcc/tree-vectorizer.h +++ b/gcc/tree-vectorizer.h @@ -106,10 +106,11 @@ struct stmt_info_for_cost { typedef vec stmt_vector_for_cost; -/* Maps base addresses to an innermost_loop_behavior that gives the maximum - known alignment for that base. */ +/* Maps base addresses to an innermost_loop_behavior and the stmt it was + derived from that gives the maximum known alignment for that base. */ typedef hash_map vec_base_alignments; + std::pair > + vec_base_alignments; /************************************************************************ SLP