From patchwork Fri Sep 20 12:56:46 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Biener X-Patchwork-Id: 1165168 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=gcc.gnu.org (client-ip=209.132.180.131; helo=sourceware.org; envelope-from=gcc-patches-return-509357-incoming=patchwork.ozlabs.org@gcc.gnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=suse.de Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=gcc.gnu.org header.i=@gcc.gnu.org header.b="uJQKymnl"; dkim-atps=neutral Received: from sourceware.org (server1.sourceware.org [209.132.180.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 46ZYdR5GN5z9s00 for ; Fri, 20 Sep 2019 22:56:59 +1000 (AEST) DomainKey-Signature: a=rsa-sha1; c=nofws; d=gcc.gnu.org; h=list-id :list-unsubscribe:list-archive:list-post:list-help:sender:date :from:to:subject:message-id:mime-version:content-type; q=dns; s= default; b=oXmum6Cxec0GUteqNlgm311Iu2SOaeungJgS/T7ItpYYSjwPZj+lj 9haN4gGGugcJHQIem7pMrYbDi/iJOw4JJypYsrM2lcGC48x5344uXMCNpA2ElWM8 wfB6mBa6nlw7rcY8Y1YD/bV0PFs4qCjLOr27GTIfGHEk4x41BOj/xs= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=gcc.gnu.org; h=list-id :list-unsubscribe:list-archive:list-post:list-help:sender:date :from:to:subject:message-id:mime-version:content-type; s= default; bh=jyRzMLMIas0OwUUGsHz/NCDLF5s=; b=uJQKymnlZQeUQaHj7NKC ymaT8AB4Zy+rEsB5+hgMAPbnh73uK2eaRS8d888CjOxyGOZwC5gTYStFuygoK/oM 7ziIHPOxbVorieDUE87XrgCd4OTBHaL5uOcOWGtX67L/o6VdhwQcIY89R6hKotx/ VT9hUWsLasl0virk4/YIDvg= Received: (qmail 18558 invoked by alias); 20 Sep 2019 12:56:52 -0000 Mailing-List: contact gcc-patches-help@gcc.gnu.org; run by ezmlm Precedence: bulk List-Id: List-Unsubscribe: List-Archive: List-Post: List-Help: Sender: gcc-patches-owner@gcc.gnu.org Delivered-To: mailing list gcc-patches@gcc.gnu.org Received: (qmail 18547 invoked by uid 89); 20 Sep 2019 12:56:51 -0000 Authentication-Results: sourceware.org; auth=none X-Spam-SWARE-Status: No, score=-10.7 required=5.0 tests=AWL, BAYES_00, GIT_PATCH_2, GIT_PATCH_3, SPF_PASS autolearn=ham version=3.3.1 spammy= X-HELO: mx1.suse.de Received: from mx2.suse.de (HELO mx1.suse.de) (195.135.220.15) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with ESMTP; Fri, 20 Sep 2019 12:56:50 +0000 Received: from relay2.suse.de (unknown [195.135.220.254]) by mx1.suse.de (Postfix) with ESMTP id 22F3FAB9D for ; Fri, 20 Sep 2019 12:56:47 +0000 (UTC) Date: Fri, 20 Sep 2019 14:56:46 +0200 (CEST) From: Richard Biener To: gcc-patches@gcc.gnu.org Subject: [PATCH] Some vect_create_epilog_for_reduction refactoring Message-ID: User-Agent: Alpine 2.21 (LSU 202 2017-01-01) MIME-Version: 1.0 Plus simplification in get_initial_def_for_reduction, avoiding some of the + 0 adjustments I've seen created. Bootstrapped on x86_64-unknown-linux-gnu, testing in progress. Richard. 2019-09-20 Richard Biener * tree-vect-loop.c (get_initial_def_for_reduction): Simplify, avoid adjusting by + 0 or * 1. (vect_create_epilog_for_reduction): Get reduction code only when necessary. Deal with adjustment_def only when necessary. Index: gcc/tree-vect-loop.c =================================================================== --- gcc/tree-vect-loop.c (revision 275997) +++ gcc/tree-vect-loop.c (working copy) @@ -4003,8 +4003,10 @@ get_initial_def_for_reduction (stmt_vec_ gcc_assert (nested_in_vect_loop_p (loop, stmt_vinfo) || loop == (gimple_bb (stmt_vinfo->stmt))->loop_father); - vect_reduction_type reduction_type - = STMT_VINFO_VEC_REDUCTION_TYPE (stmt_vinfo); + /* ADJUSTMENT_DEF is NULL when called from + vect_create_epilog_for_reduction to vectorize double reduction. */ + if (adjustment_def) + *adjustment_def = NULL; switch (code) { @@ -4018,11 +4020,6 @@ get_initial_def_for_reduction (stmt_vec_ case MULT_EXPR: case BIT_AND_EXPR: { - /* ADJUSTMENT_DEF is NULL when called from - vect_create_epilog_for_reduction to vectorize double reduction. */ - if (adjustment_def) - *adjustment_def = init_val; - if (code == MULT_EXPR) { real_init_val = dconst1; @@ -4037,10 +4034,14 @@ get_initial_def_for_reduction (stmt_vec_ else def_for_init = build_int_cst (scalar_type, int_init_val); - if (adjustment_def) - /* Option1: the first element is '0' or '1' as well. */ - init_def = gimple_build_vector_from_val (&stmts, vectype, - def_for_init); + if (adjustment_def || operand_equal_p (def_for_init, init_val, 0)) + { + /* Option1: the first element is '0' or '1' as well. */ + if (!operand_equal_p (def_for_init, init_val, 0)) + *adjustment_def = init_val; + init_def = gimple_build_vector_from_val (&stmts, vectype, + def_for_init); + } else if (!TYPE_VECTOR_SUBPARTS (vectype).is_constant ()) { /* Option2 (variable length): the first element is INIT_VAL. */ @@ -4064,16 +4065,8 @@ get_initial_def_for_reduction (stmt_vec_ case MAX_EXPR: case COND_EXPR: { - if (adjustment_def) - { - *adjustment_def = NULL_TREE; - if (reduction_type != COND_REDUCTION - && reduction_type != EXTRACT_LAST_REDUCTION) - { - init_def = vect_get_vec_def_for_operand (init_val, stmt_vinfo); - break; - } - } + gcc_assert (useless_type_conversion_p (TREE_TYPE (vectype), + TREE_TYPE (init_val))); init_val = gimple_convert (&stmts, TREE_TYPE (vectype), init_val); init_def = gimple_build_vector_from_val (&stmts, vectype, init_val); } @@ -4304,7 +4297,6 @@ vect_create_epilog_for_reduction (vecstmt); gimple *exit_phi; tree bitsize; tree adjustment_def = NULL; @@ -4645,12 +4637,6 @@ vect_create_epilog_for_reduction (vecstmt); - /* For MINUS_EXPR the initial vector is [init_val,0,...,0], therefore, - partial results are added and not subtracted. */ - if (code == MINUS_EXPR) - code = PLUS_EXPR; scalar_dest = gimple_assign_lhs (orig_stmt_info->stmt); scalar_type = TREE_TYPE (scalar_dest); @@ -4665,7 +4651,14 @@ vect_create_epilog_for_reduction (vecstmt); + /* For MINUS_EXPR the initial vector is [init_val,0,...,0], therefore, + partial results are added and not subtracted. */ + if (code == MINUS_EXPR) + code = PLUS_EXPR; /* SLP reduction without reduction chain, e.g., # a1 = phi @@ -5352,12 +5345,7 @@ vect_create_epilog_for_reduction (vecinner; - + /* 2.5 Adjust the final result by the initial value of the reduction variable. (When such adjustment is not needed, then 'adjustment_def' is zero). For example, if code is PLUS we create: @@ -5401,6 +5389,10 @@ vect_finalize_reduction: new_phis[0] = epilog_stmt; } + } + + if (double_reduc) + loop = loop->inner; /* 2.6 Handle the loop-exit phis. Replace the uses of scalar loop-exit phis with new adjusted scalar results, i.e., replace use