From patchwork Fri Jul 3 09:43:26 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Biener X-Patchwork-Id: 1322267 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gcc.gnu.org (client-ip=2620:52:3:1:0:246e:9693:128c; helo=sourceware.org; envelope-from=gcc-patches-bounces@gcc.gnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=suse.de Received: from sourceware.org (server2.sourceware.org [IPv6:2620:52:3:1:0:246e:9693:128c]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 49yqlq269Tz9sPF for ; Fri, 3 Jul 2020 19:43:33 +1000 (AEST) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id BDF393861916; Fri, 3 Jul 2020 09:43:29 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from mx2.suse.de (mx2.suse.de [195.135.220.15]) by sourceware.org (Postfix) with ESMTPS id C07033858D34 for ; Fri, 3 Jul 2020 09:43:27 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org C07033858D34 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=suse.de Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=rguenther@suse.de X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (unknown [195.135.221.27]) by mx2.suse.de (Postfix) with ESMTP id D7DACAD78 for ; Fri, 3 Jul 2020 09:43:26 +0000 (UTC) Date: Fri, 3 Jul 2020 11:43:26 +0200 (CEST) From: Richard Biener To: gcc-patches@gcc.gnu.org Subject: [PATCH] fix scalar BB vectorization costing Message-ID: User-Agent: Alpine 2.21 (LSU 202 2017-01-01) MIME-Version: 1.0 X-Spam-Status: No, score=-10.3 required=5.0 tests=BAYES_00, GIT_PATCH_0, KAM_DMARC_STATUS, RCVD_IN_MSPIKE_H3, RCVD_IN_MSPIKE_WL, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces@gcc.gnu.org Sender: "Gcc-patches" We were costing the scalar pattern stmts rather than the scalar original stmt and also not appropriately looking at the pattern stmt for whether the stmt is vectorized. Bootstrap and regtest running on x86_64-unknown-linux-gnu. 2020-07-03 Richard Biener * tree-vect-slp.c (vect_bb_slp_scalar_cost): Cost the original non-pattern stmts, look at the pattern stmt vectorization status. * gcc.dg/vect/costmodel/x86_64/costmodel-vect-slp-2.c: New testcase. --- .../costmodel/x86_64/costmodel-vect-slp-2.c | 14 ++++++++++ gcc/tree-vect-slp.c | 26 ++++++++++--------- 2 files changed, 28 insertions(+), 12 deletions(-) create mode 100644 gcc/testsuite/gcc.dg/vect/costmodel/x86_64/costmodel-vect-slp-2.c diff --git a/gcc/testsuite/gcc.dg/vect/costmodel/x86_64/costmodel-vect-slp-2.c b/gcc/testsuite/gcc.dg/vect/costmodel/x86_64/costmodel-vect-slp-2.c new file mode 100644 index 00000000000..1b7ac34ccaa --- /dev/null +++ b/gcc/testsuite/gcc.dg/vect/costmodel/x86_64/costmodel-vect-slp-2.c @@ -0,0 +1,14 @@ +/* { dg-do compile } */ +/* { dg-additional-options "-fdump-tree-slp-details" } */ + +int a[4], b[4]; +void foo() +{ + a[0] = b[0] / 7; + a[1] = b[1] / 7; + a[2] = b[2] / 7; + a[3] = b[3] / 7; +} + +/* We should cost the original division stmt, not the scalar pattern stmts. */ +/* { dg-final { scan-tree-dump-times " / 7 1 times scalar_stmt costs" 4 "slp2" } } */ diff --git a/gcc/tree-vect-slp.c b/gcc/tree-vect-slp.c index 727eba0b12f..33fc87a9f86 100644 --- a/gcc/tree-vect-slp.c +++ b/gcc/tree-vect-slp.c @@ -3039,7 +3039,6 @@ vect_bb_slp_scalar_cost (vec_info *vinfo, FOR_EACH_VEC_ELT (SLP_TREE_SCALAR_STMTS (node), i, stmt_info) { - gimple *stmt = stmt_info->stmt; ssa_op_iter op_iter; def_operand_p def_p; @@ -3051,7 +3050,9 @@ vect_bb_slp_scalar_cost (vec_info *vinfo, required defs in the SLP children in the scalar cost. This way we make the vectorization more costly when compared to the scalar cost. */ - FOR_EACH_SSA_DEF_OPERAND (def_p, stmt, op_iter, SSA_OP_DEF) + stmt_vec_info orig_stmt_info = vect_orig_stmt (stmt_info); + gimple *orig_stmt = orig_stmt_info->stmt; + FOR_EACH_SSA_DEF_OPERAND (def_p, orig_stmt, op_iter, SSA_OP_DEF) { imm_use_iterator use_iter; gimple *use_stmt; @@ -3059,7 +3060,8 @@ vect_bb_slp_scalar_cost (vec_info *vinfo, if (!is_gimple_debug (use_stmt)) { stmt_vec_info use_stmt_info = vinfo->lookup_stmt (use_stmt); - if (!use_stmt_info || !PURE_SLP_STMT (use_stmt_info)) + if (!use_stmt_info + || !PURE_SLP_STMT (vect_stmt_to_vectorize (use_stmt_info))) { (*life)[i] = true; BREAK_FROM_IMM_USE_STMT (use_iter); @@ -3070,23 +3072,23 @@ vect_bb_slp_scalar_cost (vec_info *vinfo, continue; /* Count scalar stmts only once. */ - if (gimple_visited_p (stmt)) + if (gimple_visited_p (orig_stmt)) continue; - gimple_set_visited (stmt, true); + gimple_set_visited (orig_stmt, true); vect_cost_for_stmt kind; - if (STMT_VINFO_DATA_REF (stmt_info)) - { - if (DR_IS_READ (STMT_VINFO_DATA_REF (stmt_info))) + if (STMT_VINFO_DATA_REF (orig_stmt_info)) + { + if (DR_IS_READ (STMT_VINFO_DATA_REF (orig_stmt_info))) kind = scalar_load; - else + else kind = scalar_store; - } - else if (vect_nop_conversion_p (stmt_info)) + } + else if (vect_nop_conversion_p (orig_stmt_info)) continue; else kind = scalar_stmt; - record_stmt_cost (cost_vec, 1, kind, stmt_info, 0, vect_body); + record_stmt_cost (cost_vec, 1, kind, orig_stmt_info, 0, vect_body); } auto_vec subtree_life;