From patchwork Thu Jun 1 04:36:17 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?b?6ZKf5bGF5ZOy?= X-Patchwork-Id: 1788768 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gcc.gnu.org (client-ip=2620:52:3:1:0:246e:9693:128c; helo=sourceware.org; envelope-from=gcc-patches-bounces+incoming=patchwork.ozlabs.org@gcc.gnu.org; receiver=) Received: from sourceware.org (server2.sourceware.org [IPv6:2620:52:3:1:0:246e:9693:128c]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QWtd80V9zz20Q4 for ; Thu, 1 Jun 2023 14:36:43 +1000 (AEST) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id C8B913857729 for ; Thu, 1 Jun 2023 04:36:41 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from smtpbg156.qq.com (smtpbg156.qq.com [15.184.82.18]) by sourceware.org (Postfix) with ESMTPS id 430743858D20 for ; Thu, 1 Jun 2023 04:36:27 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 430743858D20 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=rivai.ai Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=rivai.ai X-QQ-mid: bizesmtp64t1685594180t0o8n2a8 Received: from server1.localdomain ( [58.60.1.22]) by bizesmtp.qq.com (ESMTP) with id ; Thu, 01 Jun 2023 12:36:19 +0800 (CST) X-QQ-SSF: 01400000000000F0R000000A0000000 X-QQ-FEAT: 1eo3oJ4iu3BKPJe5uBs1wn0CITmWcQT/mpkyk2k+6Qunn7a7mhS6Rcq/KT7Ei 9OKOEfX+2gXPbVe2LtkYUGqOKxTHc7uyKNoBST4c9Pgrf39b1FXBNROfyCuqlMs2xSf28iB rgtatmim/6o1W2V48fnLVyAzdjzvFMoUVMxg4AVDOD4F74YmgXZu72co/xXAjyeaV09bVX1 LPw2Bf1WmWmnEyYqAhi4wSK8jw5CbLAc+Uio0rREGknWaWN6919ZcPMmdcKXdjBY6D2FcDL HoE0gUn4b4ag0TD+ketBKKS06MFOzYHjysHE379e1lCQGOcPmdkaVcE9MB2OgpTRdlS0f9m kwRWraF2M2pUlEQwz515bMNDyx1ndHyt8QNlUt8Yhddzneut9UVt19s7rJAxdijRCaZpvHX 72k673wiZqs= X-QQ-GoodBg: 2 X-BIZMAIL-ID: 12377322005757806926 From: juzhe.zhong@rivai.ai To: gcc-patches@gcc.gnu.org Cc: richard.sandiford@arm.com, rguenther@suse.de, linkw@linux.ibm.com, Ju-Zhe Zhong Subject: [PATCH V3] VECT: Change flow of decrement IV Date: Thu, 1 Jun 2023 12:36:17 +0800 Message-Id: <20230601043617.173986-1-juzhe.zhong@rivai.ai> X-Mailer: git-send-email 2.36.1 MIME-Version: 1.0 X-QQ-SENDSIZE: 520 Feedback-ID: bizesmtp:rivai.ai:qybglogicsvrgz:qybglogicsvrgz7a-one-0 X-Spam-Status: No, score=-10.3 required=5.0 tests=BAYES_00, GIT_PATCH_0, KAM_DMARC_STATUS, RCVD_IN_BARRACUDACENTRAL, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H5, RCVD_IN_MSPIKE_WL, SPF_HELO_PASS, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces+incoming=patchwork.ozlabs.org@gcc.gnu.org Sender: "Gcc-patches" From: Ju-Zhe Zhong Follow Richi's suggestion, I change current decrement IV flow from: do { remain -= MIN (vf, remain); } while (remain != 0); into: do { old_remain = remain; len = MIN (vf, remain); remain -= vf; } while (old_remain >= vf); to enhance SCEV. Include fixes from kewen. This patch will need to wait for Kewen's test feedback. Testing on X86 is on-going Co-Authored by: Kewen Lin PR tree-optimization/109971 gcc/ChangeLog: * tree-vect-loop-manip.cc (vect_set_loop_controls_directly): Change decrement IV flow. (vect_set_loop_condition_partial_vectors): Ditto. --- gcc/tree-vect-loop-manip.cc | 36 +++++++++++++++++++++++++----------- 1 file changed, 25 insertions(+), 11 deletions(-) diff --git a/gcc/tree-vect-loop-manip.cc b/gcc/tree-vect-loop-manip.cc index acf3642ceb2..3f735945e67 100644 --- a/gcc/tree-vect-loop-manip.cc +++ b/gcc/tree-vect-loop-manip.cc @@ -483,7 +483,7 @@ vect_set_loop_controls_directly (class loop *loop, loop_vec_info loop_vinfo, gimple_stmt_iterator loop_cond_gsi, rgroup_controls *rgc, tree niters, tree niters_skip, bool might_wrap_p, - tree *iv_step) + tree *iv_step, tree *compare_step) { tree compare_type = LOOP_VINFO_RGROUP_COMPARE_TYPE (loop_vinfo); tree iv_type = LOOP_VINFO_RGROUP_IV_TYPE (loop_vinfo); @@ -538,9 +538,9 @@ vect_set_loop_controls_directly (class loop *loop, loop_vec_info loop_vinfo, ... vect__4.8_28 = .LEN_LOAD (_17, 32B, _36, 0); ... - ivtmp_35 = ivtmp_9 - _36; + ivtmp_35 = ivtmp_9 - POLY_INT_CST [4, 4]; ... - if (ivtmp_35 != 0) + if (ivtmp_9 > POLY_INT_CST [4, 4]) goto ; [83.33%] else goto ; [16.67%] @@ -549,13 +549,15 @@ vect_set_loop_controls_directly (class loop *loop, loop_vec_info loop_vinfo, tree step = rgc->controls.length () == 1 ? rgc->controls[0] : make_ssa_name (iv_type); /* Create decrement IV. */ - create_iv (nitems_total, MINUS_EXPR, step, NULL_TREE, loop, &incr_gsi, - insert_after, &index_before_incr, &index_after_incr); + create_iv (nitems_total, MINUS_EXPR, nitems_step, NULL_TREE, loop, + &incr_gsi, insert_after, &index_before_incr, + &index_after_incr); gimple_seq_add_stmt (header_seq, gimple_build_assign (step, MIN_EXPR, index_before_incr, nitems_step)); *iv_step = step; - return index_after_incr; + *compare_step = nitems_step; + return index_before_incr; } /* Create increment IV. */ @@ -825,6 +827,7 @@ vect_set_loop_condition_partial_vectors (class loop *loop, arbitrarily pick the last. */ tree test_ctrl = NULL_TREE; tree iv_step = NULL_TREE; + tree compare_step = NULL_TREE; rgroup_controls *rgc; rgroup_controls *iv_rgc = nullptr; unsigned int i; @@ -861,7 +864,7 @@ vect_set_loop_condition_partial_vectors (class loop *loop, &preheader_seq, &header_seq, loop_cond_gsi, rgc, niters, niters_skip, might_wrap_p, - &iv_step); + &iv_step, &compare_step); iv_rgc = rgc; } @@ -884,10 +887,21 @@ vect_set_loop_condition_partial_vectors (class loop *loop, /* Get a boolean result that tells us whether to iterate. */ edge exit_edge = single_exit (loop); - tree_code code = (exit_edge->flags & EDGE_TRUE_VALUE) ? EQ_EXPR : NE_EXPR; - tree zero_ctrl = build_zero_cst (TREE_TYPE (test_ctrl)); - gcond *cond_stmt = gimple_build_cond (code, test_ctrl, zero_ctrl, - NULL_TREE, NULL_TREE); + gcond *cond_stmt; + if (LOOP_VINFO_USING_DECREMENTING_IV_P (loop_vinfo)) + { + gcc_assert (compare_step); + tree_code code = (exit_edge->flags & EDGE_TRUE_VALUE) ? LE_EXPR : GT_EXPR; + cond_stmt = gimple_build_cond (code, test_ctrl, compare_step, NULL_TREE, + NULL_TREE); + } + else + { + tree_code code = (exit_edge->flags & EDGE_TRUE_VALUE) ? EQ_EXPR : NE_EXPR; + tree zero_ctrl = build_zero_cst (TREE_TYPE (test_ctrl)); + cond_stmt + = gimple_build_cond (code, test_ctrl, zero_ctrl, NULL_TREE, NULL_TREE); + } gsi_insert_before (&loop_cond_gsi, cond_stmt, GSI_SAME_STMT); /* The loop iterates (NITERS - 1) / VF + 1 times.