From patchwork Thu Jan 16 18:32:32 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jakub Jelinek X-Patchwork-Id: 311819 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from sourceware.org (server1.sourceware.org [209.132.180.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 414702C009A for ; Fri, 17 Jan 2014 05:32:45 +1100 (EST) DomainKey-Signature: a=rsa-sha1; c=nofws; d=gcc.gnu.org; h=list-id :list-unsubscribe:list-archive:list-post:list-help:sender:date :from:to:cc:subject:message-id:reply-to:references:mime-version :content-type:in-reply-to; q=dns; s=default; b=L9rGFcLdnyCMTcyI9 L83ICPs8NjNb30dNgACNghPzT7Lb2lWkHGKOXJ9Qt3b7f+Y1aHQkgj9CffMlUyxG Lk9VtrHHIDvMbW9kZ/D74PGid8aUlfRkmNKI0KCtqMdVOtl/6/Kjbu1oMljK9U+G MKsY/j5F0N7D2FqVLkKq2DLGuA= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=gcc.gnu.org; h=list-id :list-unsubscribe:list-archive:list-post:list-help:sender:date :from:to:cc:subject:message-id:reply-to:references:mime-version :content-type:in-reply-to; s=default; bh=nvznaXzeNX/3maMgscjxI5a TgzM=; b=reZt01XxfkxWE+yVt+omS6W5bLRjMKrrGNX1PNw8Rd+rrgReyL2qMvg 0t+aMOyodASKTa50RD78CZlWNFM63ZwBf+3F7L2s8rMRHaUbccMnU746aWTWEVAT 1Ex/2wmV43BJwo2KSMQ/6QvpQJYkLTbcKPsdCeBPtLdDM6DrJkYQ= Received: (qmail 24958 invoked by alias); 16 Jan 2014 18:32:38 -0000 Mailing-List: contact gcc-patches-help@gcc.gnu.org; run by ezmlm Precedence: bulk List-Id: List-Unsubscribe: List-Archive: List-Post: List-Help: Sender: gcc-patches-owner@gcc.gnu.org Delivered-To: mailing list gcc-patches@gcc.gnu.org Received: (qmail 24937 invoked by uid 89); 16 Jan 2014 18:32:38 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=-3.8 required=5.0 tests=AWL, BAYES_00, RP_MATCHES_RCVD, SPF_HELO_PASS, SPF_PASS autolearn=ham version=3.3.2 X-HELO: mx1.redhat.com Received: from mx1.redhat.com (HELO mx1.redhat.com) (209.132.183.28) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with ESMTP; Thu, 16 Jan 2014 18:32:37 +0000 Received: from int-mx09.intmail.prod.int.phx2.redhat.com (int-mx09.intmail.prod.int.phx2.redhat.com [10.5.11.22]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id s0GIWZbb005346 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Thu, 16 Jan 2014 13:32:35 -0500 Received: from tucnak.zalov.cz (vpn1-4-28.ams2.redhat.com [10.36.4.28]) by int-mx09.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id s0GIWXlA021655 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 16 Jan 2014 13:32:35 -0500 Received: from tucnak.zalov.cz (localhost [127.0.0.1]) by tucnak.zalov.cz (8.14.7/8.14.7) with ESMTP id s0GIWWaE028317; Thu, 16 Jan 2014 19:32:33 +0100 Received: (from jakub@localhost) by tucnak.zalov.cz (8.14.7/8.14.7/Submit) id s0GIWWFN028316; Thu, 16 Jan 2014 19:32:32 +0100 Date: Thu, 16 Jan 2014 19:32:32 +0100 From: Jakub Jelinek To: Richard Biener Cc: gcc-patches@gcc.gnu.org Subject: Re: [PATCH] Fix PR59822 Message-ID: <20140116183232.GU892@tucnak.redhat.com> Reply-To: Jakub Jelinek References: <20140115173414.GD892@tucnak.redhat.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) X-IsSubscribed: yes On Thu, Jan 16, 2014 at 09:40:24AM +0100, Richard Biener wrote: > That should be enough I think (we can't hoist PHIs anyway). Ok, here is what I've finally successfully bootstrapped/regtested on x86_64-linux and i686-linux, ok for trunk? 2014-01-16 Jakub Jelinek * tree-vectorizer.h (struct _loop_vec_info): Add no_data_dependencies field. (LOOP_VINFO_NO_DATA_DEPENDENCIES): Define. * tree-vect-data-refs.c (vect_analyze_data_ref_dependence): Clear it when not giving up or versioning for alias only because of loop->safelen. (vect_analyze_data_ref_dependences): Set to true. * tree-vect-stmts.c (hoist_defs_of_uses): Return false if def_stmt is a GIMPLE_PHI. (vectorizable_load): Use LOOP_VINFO_NO_DATA_DEPENDENCIES instead of LOOP_REQUIRES_VERSIONING_FOR_ALIAS, add && !nested_in_vect_loop to the condition. Jakub --- gcc/tree-vectorizer.h.jj 2014-01-15 08:11:18.903449194 +0100 +++ gcc/tree-vectorizer.h 2014-01-16 16:01:48.892476417 +0100 @@ -347,6 +347,25 @@ typedef struct _loop_vec_info { fix it up. */ bool operands_swapped; + /* True if there are no loop carried data dependencies in the loop. + If loop->safelen <= 1, then this is always true, either the loop + didn't have any loop carried data dependencies, or the loop is being + vectorized guarded with some runtime alias checks, or couldn't + be vectorized at all, but then this field shouldn't be used. + For loop->safelen >= 2, the user has asserted that there are no + backward dependencies, but there still could be loop carried forward + dependencies in such loops. This flag will be false if normal + vectorizer data dependency analysis would fail or require versioning + for alias, but because of loop->safelen >= 2 it has been vectorized + even without versioning for alias. E.g. in: + #pragma omp simd + for (int i = 0; i < m; i++) + a[i] = a[i + k] * c; + (or #pragma simd or #pragma ivdep) we can vectorize this and it will + DTRT even for k > 0 && k < m, but without safelen we would not + vectorize this, so this field would be false. */ + bool no_data_dependencies; + /* If if-conversion versioned this loop before conversion, this is the loop version without if-conversion. */ struct loop *scalar_loop; @@ -385,6 +404,7 @@ typedef struct _loop_vec_info { #define LOOP_VINFO_PEELING_FOR_GAPS(L) (L)->peeling_for_gaps #define LOOP_VINFO_OPERANDS_SWAPPED(L) (L)->operands_swapped #define LOOP_VINFO_PEELING_FOR_NITER(L) (L)->peeling_for_niter +#define LOOP_VINFO_NO_DATA_DEPENDENCIES(L) (L)->no_data_dependencies #define LOOP_VINFO_SCALAR_LOOP(L) (L)->scalar_loop #define LOOP_REQUIRES_VERSIONING_FOR_ALIGNMENT(L) \ --- gcc/tree-vect-data-refs.c.jj 2014-01-15 08:11:22.593430265 +0100 +++ gcc/tree-vect-data-refs.c 2014-01-16 16:01:48.916476273 +0100 @@ -244,6 +244,7 @@ vect_analyze_data_ref_dependence (struct { if (loop->safelen < *max_vf) *max_vf = loop->safelen; + LOOP_VINFO_NO_DATA_DEPENDENCIES (loop_vinfo) = false; return false; } @@ -291,6 +292,7 @@ vect_analyze_data_ref_dependence (struct { if (loop->safelen < *max_vf) *max_vf = loop->safelen; + LOOP_VINFO_NO_DATA_DEPENDENCIES (loop_vinfo) = false; return false; } @@ -447,6 +449,7 @@ vect_analyze_data_ref_dependences (loop_ dump_printf_loc (MSG_NOTE, vect_location, "=== vect_analyze_data_ref_dependences ===\n"); + LOOP_VINFO_NO_DATA_DEPENDENCIES (loop_vinfo) = true; if (!compute_all_dependences (LOOP_VINFO_DATAREFS (loop_vinfo), &LOOP_VINFO_DDRS (loop_vinfo), LOOP_VINFO_LOOP_NEST (loop_vinfo), true)) --- gcc/tree-vect-stmts.c.jj 2014-01-16 09:41:19.534630044 +0100 +++ gcc/tree-vect-stmts.c 2014-01-16 16:03:50.948856799 +0100 @@ -5504,6 +5504,8 @@ hoist_defs_of_uses (gimple stmt, struct dependencies within them. */ tree op2; ssa_op_iter i2; + if (gimple_code (def_stmt) == GIMPLE_PHI) + return false; FOR_EACH_SSA_TREE_OPERAND (op2, def_stmt, i2, SSA_OP_USE) { gimple def_stmt2 = SSA_NAME_DEF_STMT (op2); @@ -6434,10 +6436,12 @@ vectorizable_load (gimple stmt, gimple_s if (inv_p && !bb_vinfo) { gcc_assert (!grouped_load); - /* If we have versioned for aliasing then we are sure - this is a loop invariant load and thus we can insert - it on the preheader edge. */ - if (LOOP_REQUIRES_VERSIONING_FOR_ALIAS (loop_vinfo) + /* If we have versioned for aliasing or the loop doesn't + have any data dependencies that would preclude this, + then we are sure this is a loop invariant load and + thus we can insert it on the preheader edge. */ + if (LOOP_VINFO_NO_DATA_DEPENDENCIES (loop_vinfo) + && !nested_in_vect_loop && hoist_defs_of_uses (stmt, loop)) { if (dump_enabled_p ())