From patchwork Sun Nov 3 22:47:12 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Hubicka X-Patchwork-Id: 1188718 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gcc.gnu.org (client-ip=209.132.180.131; helo=sourceware.org; envelope-from=gcc-patches-return-512284-incoming=patchwork.ozlabs.org@gcc.gnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=ucw.cz Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=gcc.gnu.org header.i=@gcc.gnu.org header.b="AJ5gwTRz"; dkim-atps=neutral Received: from sourceware.org (server1.sourceware.org [209.132.180.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 475rfY0BpNz9sP6 for ; Mon, 4 Nov 2019 09:47:30 +1100 (AEDT) DomainKey-Signature: a=rsa-sha1; c=nofws; d=gcc.gnu.org; h=list-id :list-unsubscribe:list-archive:list-post:list-help:sender:date :from:to:subject:message-id:mime-version:content-type; q=dns; s= default; b=Lx/6LVyesnOjwVCgot/CtIb4xis0hegHYzD8y7q0L9hLz60hJ8NdJ UVHzLq5gEz5M+CIGZ++fB7RWqfj3YemRrv7Y6lSJVDgbcgw/qukmfPsNNoFcJhD6 sAdsG8fwD8W21B0jvAqpv3VzFXOPcgSZqWynT6fYkKidX/c0Pxt9gw= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=gcc.gnu.org; h=list-id :list-unsubscribe:list-archive:list-post:list-help:sender:date :from:to:subject:message-id:mime-version:content-type; s= default; bh=7cn9CYgu5qrQmIrbFM14Nm1IB/E=; b=AJ5gwTRz2XYxFCw+gfzS wqgRsnd6pFbUdm4V2Rv9w3vCXSRgUJuufn0GGAK3XOklifend8ppNqozp0kPQXYm E/BP0XpShBsbECqN6qwrIq1jQtRA9IpFUbHXYOmZ+DFWwRO7T15iMeBB920Kiza4 f4kX3mpWx5pj/Le9t/pegZU= Received: (qmail 86510 invoked by alias); 3 Nov 2019 22:47:22 -0000 Mailing-List: contact gcc-patches-help@gcc.gnu.org; run by ezmlm Precedence: bulk List-Id: List-Unsubscribe: List-Archive: List-Post: List-Help: Sender: gcc-patches-owner@gcc.gnu.org Delivered-To: mailing list gcc-patches@gcc.gnu.org Received: (qmail 86499 invoked by uid 89); 3 Nov 2019 22:47:22 -0000 Authentication-Results: sourceware.org; auth=none X-Spam-SWARE-Status: No, score=-12.5 required=5.0 tests=AWL, BAYES_00, GIT_PATCH_1, GIT_PATCH_2, GIT_PATCH_3, KAM_ASCII_DIVIDERS autolearn=ham version=3.3.1 spammy=double-checking, doublechecking, Watch, fbi X-HELO: nikam.ms.mff.cuni.cz Received: from nikam.ms.mff.cuni.cz (HELO nikam.ms.mff.cuni.cz) (195.113.20.16) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with ESMTP; Sun, 03 Nov 2019 22:47:15 +0000 Received: by nikam.ms.mff.cuni.cz (Postfix, from userid 16202) id 55D53282BE5; Sun, 3 Nov 2019 23:47:12 +0100 (CET) Date: Sun, 3 Nov 2019 23:47:12 +0100 From: Jan Hubicka To: gcc-patches@gcc.gnu.org, mjambor@suse.cz Subject: Improve effectivity of ipa_context_cache Message-ID: <20191103224712.ndzyxu6cn3jt3enu@kam.mff.cuni.cz> MIME-Version: 1.0 Content-Disposition: inline User-Agent: NeoMutt/20170113 (1.7.2) Hi, this, somewhat verbose, patch make ipa-context-cache to only store the information that is actually used by the size/time estimation. In particular we now track what parameters are used in ipa predicates, what parameters are used as targets of indirect calls and what parameters are used for polymorphic calls. So we have 3 new flags or every parameter of function: unsigned used_by_ipa_predicates : 1; unsigned used_by_indirect_call : 1; unsigned used_by_polymorphic_call : 1; This greatly improves effectivity of the cache (from 51% to 75%) and saves a lot of allocations because only info needed is actually stored and compared for equality. As followup I also plan to track what parameters are used by jump function and drop useless jump function prior inlining. That should save some memory and avoid extra legwork updating the info while we know it is not being useful. Martin, I would welcome double-checking that I did not missed any update. Basically the flags are: 1) set by functions recording the indirect call and recording the condition for predicates 2) updated by functions manipulating these once function is inlined 3) set by stream-in code (rather than being streamed as well, since it is trivial to re-compute). Bootstrapped/regtested x86_64-linux, comitted. Honza * ipa-fnsummary.c (set_cond_stmt_execution_predicate, set_switch_stmt_execution_predicate, compute_bb_predicates, will_be_nonconstant_expr_predicate, phi_result_unknown_predicate, analyze_function_body): Pass arround params summary. (ipa_call_context::duplicate_from): New comment; only duplicate useful values. (ipa_call_context::equal_to): Only compare useful values. (remap_edge_summaries): Pass params_summary. (remap_hint_predicate): Likewise. (ipa_merge_fn_summary_after_inlining): Likewise. (inline_read_section): Initialize params summary used flags. * ipa-predicate.c (predicate::remap_after_inlining): Pass around param_summary. (add_condition): Initialized used params summary flags. * ipa-predicate.h (inline_param_summary::equals_to): Make const. (inline_param_summary::useless_p): New predicate. (remap_after_inlining, add_condition): Update prototype * ipa-prop.c (ipa_populate_param_decls): Watch overflow in move_cost. (ipa_note_param_call): Add parameter POLYMORPHIC; update params summaries. (ipa_analyze_indirect_call_uses): Update use of ipa_note_param_call. (ipa_analyze_virtual_call_uses): Likewise. (update_indirect_edges_after_inlining): Update param summaries. (ipa_print_node_params): Print used flags. (ipa_read_indirect_edge_info): Update param summareis. * ipa-prop.h (ipa_param_descriptor): Add used_by_ipa_predicates, used_by_indirect_call and used_by_polymorphic_call. (ipa_set_param_used_by_ipa_predicates, ipa_set_param_used_by_indirect_call, ipa_set_param_used_by_polymorphic_call, ipa_is_param_used_by_ipa_predicates, ipa_is_param_used_by_indirect_call, ipa_is_param_used_by_polymorphic_call): New inline functions. Index: ipa-fnsummary.c =================================================================== --- ipa-fnsummary.c (revision 277757) +++ ipa-fnsummary.c (working copy) @@ -1312,6 +1312,7 @@ fail: static void set_cond_stmt_execution_predicate (struct ipa_func_body_info *fbi, class ipa_fn_summary *summary, + class ipa_node_params *params_summary, basic_block bb) { gimple *last; @@ -1354,7 +1355,8 @@ set_cond_stmt_execution_predicate (struc && !dominated_by_p (CDI_POST_DOMINATORS, bb, e->dest)) { predicate p - = add_condition (summary, index, param_type, &aggpos, + = add_condition (summary, params_summary, index, + param_type, &aggpos, this_code, gimple_cond_rhs (last), param_ops); e->aux = edge_predicate_pool.allocate (); *(predicate *) e->aux = p; @@ -1387,7 +1389,8 @@ set_cond_stmt_execution_predicate (struc return; FOR_EACH_EDGE (e, ei, bb->succs) if (e->flags & EDGE_FALSE_VALUE) { - predicate p = add_condition (summary, index, param_type, &aggpos, + predicate p = add_condition (summary, params_summary, index, + param_type, &aggpos, predicate::is_not_constant, NULL_TREE); e->aux = edge_predicate_pool.allocate (); *(predicate *) e->aux = p; @@ -1401,6 +1404,7 @@ set_cond_stmt_execution_predicate (struc static void set_switch_stmt_execution_predicate (struct ipa_func_body_info *fbi, class ipa_fn_summary *summary, + class ipa_node_params *params_summary, basic_block bb) { gimple *lastg; @@ -1470,15 +1474,15 @@ set_switch_stmt_execution_predicate (str if (dominated_by_p (CDI_POST_DOMINATORS, bb, e->dest)) p = true; else if (min == max) - p = add_condition (summary, index, param_type, &aggpos, EQ_EXPR, - min, param_ops); + p = add_condition (summary, params_summary, index, param_type, + &aggpos, EQ_EXPR, min, param_ops); else { predicate p1, p2; - p1 = add_condition (summary, index, param_type, &aggpos, GE_EXPR, - min, param_ops); - p2 = add_condition (summary, index, param_type, &aggpos, LE_EXPR, - max, param_ops); + p1 = add_condition (summary, params_summary, index, param_type, + &aggpos, GE_EXPR, min, param_ops); + p2 = add_condition (summary, params_summary,index, param_type, + &aggpos, LE_EXPR, max, param_ops); p = p1 & p2; } *(class predicate *) e->aux @@ -1559,7 +1563,8 @@ set_switch_stmt_execution_predicate (str tree max = ranges[i].second; if (min == max) - p_seg &= add_condition (summary, index, param_type, &aggpos, NE_EXPR, + p_seg &= add_condition (summary, params_summary, index, + param_type, &aggpos, NE_EXPR, min, param_ops); else { @@ -1567,7 +1572,8 @@ set_switch_stmt_execution_predicate (str of switch index. */ if (wi::lt_p (vr_wmin, wi::to_wide (min), TYPE_SIGN (type))) { - p_seg &= add_condition (summary, index, param_type, &aggpos, + p_seg &= add_condition (summary, params_summary, index, + param_type, &aggpos, LT_EXPR, min, param_ops); p_all = p_all.or_with (summary->conds, p_seg); } @@ -1580,7 +1586,8 @@ set_switch_stmt_execution_predicate (str break; } - p_seg = add_condition (summary, index, param_type, &aggpos, GT_EXPR, + p_seg = add_condition (summary, params_summary, index, + param_type, &aggpos, GT_EXPR, max, param_ops); } } @@ -1599,7 +1606,8 @@ set_switch_stmt_execution_predicate (str static void compute_bb_predicates (struct ipa_func_body_info *fbi, struct cgraph_node *node, - class ipa_fn_summary *summary) + class ipa_fn_summary *summary, + class ipa_node_params *params_summary) { struct function *my_function = DECL_STRUCT_FUNCTION (node->decl); bool done = false; @@ -1607,8 +1615,8 @@ compute_bb_predicates (struct ipa_func_b FOR_EACH_BB_FN (bb, my_function) { - set_cond_stmt_execution_predicate (fbi, summary, bb); - set_switch_stmt_execution_predicate (fbi, summary, bb); + set_cond_stmt_execution_predicate (fbi, summary, params_summary, bb); + set_switch_stmt_execution_predicate (fbi, summary, params_summary, bb); } /* Entry block is always executable. */ @@ -1701,6 +1709,7 @@ compute_bb_predicates (struct ipa_func_b static predicate will_be_nonconstant_expr_predicate (ipa_func_body_info *fbi, class ipa_fn_summary *summary, + class ipa_node_params *params_summary, tree expr, vec nonconstant_names) { @@ -1712,7 +1721,7 @@ will_be_nonconstant_expr_predicate (ipa_ parm = unmodified_parm (fbi, NULL, expr, NULL); if (parm && (index = ipa_get_param_decl_index (fbi->info, parm)) >= 0) - return add_condition (summary, index, TREE_TYPE (parm), NULL, + return add_condition (summary, params_summary, index, TREE_TYPE (parm), NULL, predicate::changed, NULL_TREE); if (is_gimple_min_invariant (expr)) return false; @@ -1722,6 +1731,7 @@ will_be_nonconstant_expr_predicate (ipa_ { predicate p1 = will_be_nonconstant_expr_predicate (fbi, summary, + params_summary, TREE_OPERAND (expr, 0), nonconstant_names); if (p1 == true) @@ -1729,6 +1739,7 @@ will_be_nonconstant_expr_predicate (ipa_ predicate p2 = will_be_nonconstant_expr_predicate (fbi, summary, + params_summary, TREE_OPERAND (expr, 1), nonconstant_names); return p1.or_with (summary->conds, p2); @@ -1737,6 +1748,7 @@ will_be_nonconstant_expr_predicate (ipa_ { predicate p1 = will_be_nonconstant_expr_predicate (fbi, summary, + params_summary, TREE_OPERAND (expr, 0), nonconstant_names); if (p1 == true) @@ -1744,12 +1756,14 @@ will_be_nonconstant_expr_predicate (ipa_ predicate p2 = will_be_nonconstant_expr_predicate (fbi, summary, + params_summary, TREE_OPERAND (expr, 1), nonconstant_names); if (p2 == true) return p2; p1 = p1.or_with (summary->conds, p2); p2 = will_be_nonconstant_expr_predicate (fbi, summary, + params_summary, TREE_OPERAND (expr, 2), nonconstant_names); return p2.or_with (summary->conds, p1); @@ -1771,6 +1785,7 @@ will_be_nonconstant_expr_predicate (ipa_ static predicate will_be_nonconstant_predicate (struct ipa_func_body_info *fbi, class ipa_fn_summary *summary, + class ipa_node_params *params_summary, gimple *stmt, vec nonconstant_names) { @@ -1828,7 +1843,8 @@ will_be_nonconstant_predicate (struct ip if (is_load) op_non_const = - add_condition (summary, base_index, param_type, &aggpos, + add_condition (summary, params_summary, + base_index, param_type, &aggpos, predicate::changed, NULL_TREE); else op_non_const = false; @@ -1840,7 +1856,8 @@ will_be_nonconstant_predicate (struct ip if (parm && (index = ipa_get_param_decl_index (fbi->info, parm)) >= 0) { if (index != base_index) - p = add_condition (summary, index, TREE_TYPE (parm), NULL, + p = add_condition (summary, params_summary, index, + TREE_TYPE (parm), NULL, predicate::changed, NULL_TREE); else continue; @@ -2027,7 +2044,9 @@ param_change_prob (ipa_func_body_info *f static bool phi_result_unknown_predicate (ipa_func_body_info *fbi, - ipa_fn_summary *summary, basic_block bb, + ipa_fn_summary *summary, + class ipa_node_params *params_summary, + basic_block bb, predicate *p, vec nonconstant_names) { @@ -2071,7 +2090,7 @@ phi_result_unknown_predicate (ipa_func_b || !is_gimple_ip_invariant (gimple_cond_rhs (stmt))) return false; - *p = will_be_nonconstant_expr_predicate (fbi, summary, + *p = will_be_nonconstant_expr_predicate (fbi, summary, params_summary, gimple_cond_lhs (stmt), nonconstant_names); if (*p == true) @@ -2264,6 +2283,7 @@ analyze_function_body (struct cgraph_nod struct function *my_function = DECL_STRUCT_FUNCTION (node->decl); sreal freq; class ipa_fn_summary *info = ipa_fn_summaries->get_create (node); + class ipa_node_params *params_summary = early ? NULL : IPA_NODE_REF (node); predicate bb_predicate; struct ipa_func_body_info fbi; vec nonconstant_names = vNULL; @@ -2329,7 +2349,7 @@ analyze_function_body (struct cgraph_nod bb_predicate); if (fbi.info) - compute_bb_predicates (&fbi, node, info); + compute_bb_predicates (&fbi, node, info, params_summary); order = XNEWVEC (int, n_basic_blocks_for_fn (cfun)); nblocks = pre_and_rev_post_order_compute (NULL, order, false); for (n = 0; n < nblocks; n++) @@ -2371,7 +2391,9 @@ analyze_function_body (struct cgraph_nod gsi_next (&bsi)) { if (first_phi - && !phi_result_unknown_predicate (&fbi, info, bb, + && !phi_result_unknown_predicate (&fbi, info, + params_summary, + bb, &phi_predicate, nonconstant_names)) break; @@ -2469,7 +2491,7 @@ analyze_function_body (struct cgraph_nod just maximum of the possible paths. */ if (fbi.info) will_be_nonconstant - = will_be_nonconstant_predicate (&fbi, info, + = will_be_nonconstant_predicate (&fbi, info, params_summary, stmt, nonconstant_names); else will_be_nonconstant = true; @@ -2536,7 +2558,8 @@ analyze_function_body (struct cgraph_nod predicate p = bb_predicate; if (fbi.info) p = p & will_be_nonconstant_expr_predicate - (&fbi, info, TREE_OPERAND (op, 1), + (&fbi, info, params_summary, + TREE_OPERAND (op, 1), nonconstant_names); if (p != false) { @@ -2581,6 +2604,7 @@ analyze_function_body (struct cgraph_nod { predicate will_be_nonconstant = will_be_nonconstant_expr_predicate (&fbi, info, + params_summary, niter_desc.niter, nonconstant_names); if (will_be_nonconstant != true) @@ -2625,7 +2649,9 @@ analyze_function_body (struct cgraph_nod continue; predicate will_be_nonconstant - = will_be_nonconstant_expr_predicate (&fbi, info, iv.step, + = will_be_nonconstant_expr_predicate (&fbi, info, + params_summary, + iv.step, nonconstant_names); if (will_be_nonconstant != true) will_be_nonconstant = bb_predicate & will_be_nonconstant; @@ -2964,29 +2990,73 @@ ipa_call_context::ipa_call_context (cgra { } +/* Set THIS to be a duplicate of CTX. Copy all relevant info. */ + void ipa_call_context::duplicate_from (const ipa_call_context &ctx) { m_node = ctx.m_node; m_possible_truths = ctx.m_possible_truths; m_nonspec_possible_truths = ctx.m_nonspec_possible_truths; + class ipa_node_params *params_summary = IPA_NODE_REF (m_node); + unsigned int nargs = ipa_get_param_count (params_summary); + m_inline_param_summary = vNULL; + /* Copy the info only if there is at least one useful entry. */ if (ctx.m_inline_param_summary.exists ()) - m_inline_param_summary = ctx.m_inline_param_summary.copy (); - else - m_inline_param_summary = vNULL; + { + unsigned int n = MIN (ctx.m_inline_param_summary.length (), nargs); + + for (unsigned int i = 0; i < n; i++) + if (ipa_is_param_used_by_ipa_predicates (params_summary, i) + && !ctx.m_inline_param_summary[i].useless_p ()) + { + m_inline_param_summary + = ctx.m_inline_param_summary.copy (); + break; + } + } + m_known_vals = vNULL; if (ctx.m_known_vals.exists ()) - m_known_vals = ctx.m_known_vals.copy (); - else - m_known_vals = vNULL; + { + unsigned int n = MIN (ctx.m_known_vals.length (), nargs); + + for (unsigned int i = 0; i < n; i++) + if (ipa_is_param_used_by_indirect_call (params_summary, i) + && ctx.m_known_vals[i]) + { + m_known_vals = ctx.m_known_vals.copy (); + break; + } + } + + m_known_contexts = vNULL; if (ctx.m_known_contexts.exists ()) - m_known_contexts = ctx.m_known_contexts.copy (); - else - m_known_contexts = vNULL; + { + unsigned int n = MIN (ctx.m_known_contexts.length (), nargs); + + for (unsigned int i = 0; i < n; i++) + if (ipa_is_param_used_by_polymorphic_call (params_summary, i) + && !ctx.m_known_contexts[i].useless_p ()) + { + m_known_contexts = ctx.m_known_contexts.copy (); + break; + } + } + + m_known_aggs = vNULL; if (ctx.m_known_aggs.exists ()) - m_known_aggs = ctx.m_known_aggs.copy (); - else - m_known_aggs = vNULL; + { + unsigned int n = MIN (ctx.m_known_aggs.length (), nargs); + + for (unsigned int i = 0; i < n; i++) + if (ipa_is_param_used_by_indirect_call (params_summary, i) + && ctx.m_known_aggs[i]) + { + m_known_aggs = ctx.m_known_aggs.copy (); + break; + } + } } /* Release memory used by known_vals/contexts/aggs vectors. @@ -3016,49 +3086,107 @@ ipa_call_context::equal_to (const ipa_ca || m_possible_truths != ctx.m_possible_truths || m_nonspec_possible_truths != ctx.m_nonspec_possible_truths) return false; - if (m_inline_param_summary.exists () != ctx.m_inline_param_summary.exists () - || m_known_vals.exists () != ctx.m_known_vals.exists() - || m_known_contexts.exists () != ctx.m_known_contexts.exists () - || m_known_aggs.exists () != ctx.m_known_aggs.exists ()) - return false; - if (m_inline_param_summary.exists ()) + + class ipa_node_params *params_summary = IPA_NODE_REF (m_node); + unsigned int nargs = ipa_get_param_count (params_summary); + + if (m_inline_param_summary.exists () || ctx.m_inline_param_summary.exists ()) { - if (m_inline_param_summary.length () != ctx.m_inline_param_summary.length ()) - return false; - for (unsigned int i = 0; i < m_inline_param_summary.length (); i++) - if (!m_inline_param_summary[i].equal_to (ctx.m_inline_param_summary[i])) - return false; + for (unsigned int i = 0; i < nargs; i++) + { + if (!ipa_is_param_used_by_ipa_predicates (params_summary, i)) + continue; + if (i >= m_inline_param_summary.length () + || m_inline_param_summary[i].useless_p ()) + { + if (i < ctx.m_inline_param_summary.length () + && !ctx.m_inline_param_summary[i].useless_p ()) + return false; + continue; + } + if (i >= ctx.m_inline_param_summary.length () + || ctx.m_inline_param_summary[i].useless_p ()) + { + if (i < m_inline_param_summary.length () + && !m_inline_param_summary[i].useless_p ()) + return false; + continue; + } + if (!m_inline_param_summary[i].equal_to + (ctx.m_inline_param_summary[i])) + return false; + } } - if (m_known_vals.exists ()) + if (m_known_vals.exists () || ctx.m_known_vals.exists ()) { - if (m_known_vals.length () != ctx.m_known_vals.length ()) - return false; - for (unsigned int i = 0; i < m_known_vals.length (); i++) + for (unsigned int i = 0; i < nargs; i++) { - tree t1 = m_known_vals[i]; - tree t2 = ctx.m_known_vals[i]; - - if (t1 != t2 - && (!t1 || !t2 || !operand_equal_p (m_known_vals[i], - ctx.m_known_vals[i], 0))) + if (!ipa_is_param_used_by_indirect_call (params_summary, i)) + continue; + if (i >= m_known_vals.length () || !m_known_vals[i]) + { + if (i < ctx.m_known_vals.length () && ctx.m_known_vals[i]) + return false; + continue; + } + if (i >= ctx.m_known_vals.length () || !ctx.m_known_vals[i]) + { + if (i < m_known_vals.length () && m_known_vals[i]) + return false; + continue; + } + if (m_known_vals[i] != ctx.m_known_vals[i]) return false; } } - if (m_known_contexts.exists ()) + if (m_known_contexts.exists () || ctx.m_known_contexts.exists ()) { - if (m_known_contexts.length () != ctx.m_known_contexts.length ()) - return false; - for (unsigned int i = 0; i < m_known_contexts.length (); i++) - if (!m_known_contexts[i].equal_to (ctx.m_known_contexts[i])) - return false; + for (unsigned int i = 0; i < nargs; i++) + { + if (!ipa_is_param_used_by_polymorphic_call (params_summary, i)) + continue; + if (i >= m_known_contexts.length () + || m_known_contexts[i].useless_p ()) + { + if (i < ctx.m_known_contexts.length () + && !ctx.m_known_contexts[i].useless_p ()) + return false; + continue; + } + if (i >= ctx.m_known_contexts.length () + || ctx.m_known_contexts[i].useless_p ()) + { + if (i < m_known_contexts.length () + && !m_known_contexts[i].useless_p ()) + return false; + continue; + } + if (!m_known_contexts[i].equal_to + (ctx.m_known_contexts[i])) + return false; + } } - if (m_known_aggs.exists ()) + if (m_known_aggs.exists () || ctx.m_known_aggs.exists ()) { - if (m_known_aggs.length () != ctx.m_known_aggs.length ()) - return false; - for (unsigned int i = 0; i < m_known_aggs.length (); i++) - if (!m_known_aggs[i]->equal_to (*ctx.m_known_aggs[i])) - return false; + for (unsigned int i = 0; i < nargs; i++) + { + if (!ipa_is_param_used_by_indirect_call (params_summary, i)) + continue; + if (i >= m_known_aggs.length () || !m_known_aggs[i]) + { + if (i < ctx.m_known_aggs.length () && ctx.m_known_aggs[i]) + return false; + continue; + } + if (i >= ctx.m_known_aggs.length () || !ctx.m_known_aggs[i]) + { + if (i < m_known_aggs.length () && m_known_aggs[i]) + return false; + continue; + } + if (m_known_aggs[i] != ctx.m_known_aggs[i]) + return false; + } } return true; } @@ -3319,6 +3447,7 @@ static void remap_edge_summaries (struct cgraph_edge *inlined_edge, struct cgraph_node *node, class ipa_fn_summary *info, + class ipa_node_params *params_summary, class ipa_fn_summary *callee_info, vec operand_map, vec offset_map, @@ -3339,7 +3468,8 @@ remap_edge_summaries (struct cgraph_edge if (es->predicate) { p = es->predicate->remap_after_inlining - (info, callee_info, operand_map, + (info, params_summary, + callee_info, operand_map, offset_map, possible_truths, *toplev_predicate); edge_set_predicate (e, &p); @@ -3348,7 +3478,8 @@ remap_edge_summaries (struct cgraph_edge edge_set_predicate (e, toplev_predicate); } else - remap_edge_summaries (inlined_edge, e->callee, info, callee_info, + remap_edge_summaries (inlined_edge, e->callee, info, + params_summary, callee_info, operand_map, offset_map, possible_truths, toplev_predicate); } @@ -3362,7 +3493,8 @@ remap_edge_summaries (struct cgraph_edge if (es->predicate) { p = es->predicate->remap_after_inlining - (info, callee_info, operand_map, offset_map, + (info, params_summary, + callee_info, operand_map, offset_map, possible_truths, *toplev_predicate); edge_set_predicate (e, &p); } @@ -3375,6 +3507,7 @@ remap_edge_summaries (struct cgraph_edge static void remap_hint_predicate (class ipa_fn_summary *info, + class ipa_node_params *params_summary, class ipa_fn_summary *callee_info, predicate **hint, vec operand_map, @@ -3387,7 +3520,7 @@ remap_hint_predicate (class ipa_fn_summa if (!*hint) return; p = (*hint)->remap_after_inlining - (info, callee_info, + (info, params_summary, callee_info, operand_map, offset_map, possible_truths, *toplev_predicate); if (p != false && p != true) @@ -3415,6 +3548,8 @@ ipa_merge_fn_summary_after_inlining (str int i; predicate toplev_predicate; class ipa_call_summary *es = ipa_call_summaries->get (edge); + class ipa_node_params *params_summary = (ipa_node_params_sum + ? IPA_NODE_REF (to) : NULL); if (es->predicate) toplev_predicate = *es->predicate; @@ -3461,19 +3596,21 @@ ipa_merge_fn_summary_after_inlining (str } } operand_map[i] = map; - gcc_assert (map < ipa_get_param_count (IPA_NODE_REF (to))); + gcc_assert (map < ipa_get_param_count (params_summary)); } } for (i = 0; vec_safe_iterate (callee_info->size_time_table, i, &e); i++) { predicate p; p = e->exec_predicate.remap_after_inlining - (info, callee_info, operand_map, + (info, params_summary, + callee_info, operand_map, offset_map, clause, toplev_predicate); predicate nonconstp; nonconstp = e->nonconst_predicate.remap_after_inlining - (info, callee_info, operand_map, + (info, params_summary, + callee_info, operand_map, offset_map, clause, toplev_predicate); if (p != false && nonconstp != false) @@ -3491,12 +3628,13 @@ ipa_merge_fn_summary_after_inlining (str info->account_size_time (e->size, add_time, p, nonconstp); } } - remap_edge_summaries (edge, edge->callee, info, callee_info, operand_map, + remap_edge_summaries (edge, edge->callee, info, params_summary, + callee_info, operand_map, offset_map, clause, &toplev_predicate); - remap_hint_predicate (info, callee_info, + remap_hint_predicate (info, params_summary, callee_info, &callee_info->loop_iterations, operand_map, offset_map, clause, &toplev_predicate); - remap_hint_predicate (info, callee_info, + remap_hint_predicate (info, params_summary, callee_info, &callee_info->loop_stride, operand_map, offset_map, clause, &toplev_predicate); @@ -3687,6 +3825,7 @@ inline_read_section (struct lto_file_dec unsigned int index; struct cgraph_node *node; class ipa_fn_summary *info; + class ipa_node_params *params_summary; class ipa_size_summary *size_info; lto_symtab_encoder_t encoder; struct bitpack_d bp; @@ -3698,6 +3837,7 @@ inline_read_section (struct lto_file_dec node = dyn_cast (lto_symtab_encoder_deref (encoder, index)); info = node->prevailing_p () ? ipa_fn_summaries->get_create (node) : NULL; + params_summary = node->prevailing_p () ? IPA_NODE_REF (node) : NULL; size_info = node->prevailing_p () ? ipa_size_summaries->get_create (node) : NULL; @@ -3746,6 +3886,9 @@ inline_read_section (struct lto_file_dec c.param_ops = NULL; if (info) vec_safe_reserve_exact (c.param_ops, count3); + if (params_summary) + ipa_set_param_used_by_ipa_predicates + (params_summary, c.operand_num, true); for (k = 0; k < count3; k++) { struct expr_eval_op op; Index: ipa-predicate.c =================================================================== --- ipa-predicate.c (revision 277756) +++ ipa-predicate.c (working copy) @@ -505,6 +505,7 @@ predicate::remap_after_duplication (clau predicate predicate::remap_after_inlining (class ipa_fn_summary *info, + class ipa_node_params *params_summary, class ipa_fn_summary *callee_info, vec operand_map, vec offset_map, @@ -566,7 +567,7 @@ predicate::remap_after_inlining (class i ap.offset = c->offset + offset_delta; ap.agg_contents = c->agg_contents; ap.by_ref = c->by_ref; - cond_predicate = add_condition (info, + cond_predicate = add_condition (info, params_summary, operand_map[c->operand_num], c->type, &ap, c->code, c->val, c->param_ops); @@ -629,7 +630,9 @@ predicate::stream_out (struct output_blo aggregate. */ predicate -add_condition (class ipa_fn_summary *summary, int operand_num, +add_condition (class ipa_fn_summary *summary, + class ipa_node_params *params_summary, + int operand_num, tree type, struct agg_position_info *aggpos, enum tree_code code, tree val, expr_eval_ops param_ops) { @@ -640,6 +643,9 @@ add_condition (class ipa_fn_summary *sum bool agg_contents, by_ref; expr_eval_op *op; + if (params_summary) + ipa_set_param_used_by_ipa_predicates (params_summary, operand_num, true); + if (aggpos) { offset = aggpos->offset; Index: ipa-predicate.h =================================================================== --- ipa-predicate.h (revision 277757) +++ ipa-predicate.h (working copy) @@ -77,10 +77,14 @@ struct inline_param_summary Value 0 is reserved for compile time invariants. */ int change_prob; - bool equal_to (const inline_param_summary &other) + bool equal_to (const inline_param_summary &other) const { return change_prob == other.change_prob; } + bool useless_p (void) const + { + return change_prob == REG_BR_PROB_BASE; + } }; typedef vec *conditions; @@ -233,6 +237,7 @@ public: /* Return predicate equal to THIS after inlining. */ predicate remap_after_inlining (class ipa_fn_summary *, + class ipa_node_params *params_summary, class ipa_fn_summary *, vec, vec, clause_t, const predicate &); @@ -254,7 +259,9 @@ private: }; void dump_condition (FILE *f, conditions conditions, int cond); -predicate add_condition (class ipa_fn_summary *summary, int operand_num, +predicate add_condition (class ipa_fn_summary *summary, + class ipa_node_params *params_summary, + int operand_num, tree type, struct agg_position_info *aggpos, enum tree_code code, tree val, expr_eval_ops param_ops = NULL); Index: ipa-prop.c =================================================================== --- ipa-prop.c (revision 277757) +++ ipa-prop.c (working copy) @@ -227,8 +227,10 @@ ipa_populate_param_decls (struct cgraph_ for (parm = fnargs; parm; parm = DECL_CHAIN (parm)) { descriptors[param_num].decl_or_type = parm; - descriptors[param_num].move_cost = estimate_move_cost (TREE_TYPE (parm), - true); + unsigned int cost = estimate_move_cost (TREE_TYPE (parm), true); + descriptors[param_num].move_cost = cost; + /* Watch overflow, move_cost is a bitfield. */ + gcc_checking_assert (cost == descriptors[param_num].move_cost); param_num++; } } @@ -2116,11 +2118,12 @@ ipa_is_ssa_with_stmt_def (tree t) /* Find the indirect call graph edge corresponding to STMT and mark it as a call to a parameter number PARAM_INDEX. NODE is the caller. Return the - indirect call graph edge. */ + indirect call graph edge. + If POLYMORPHIC is true record is as a destination of polymorphic call. */ static struct cgraph_edge * ipa_note_param_call (struct cgraph_node *node, int param_index, - gcall *stmt) + gcall *stmt, bool polymorphic) { struct cgraph_edge *cs; @@ -2129,6 +2132,11 @@ ipa_note_param_call (struct cgraph_node cs->indirect_info->agg_contents = 0; cs->indirect_info->member_ptr = 0; cs->indirect_info->guaranteed_unmodified = 0; + ipa_set_param_used_by_indirect_call (IPA_NODE_REF (node), + param_index, true); + if (cs->indirect_info->polymorphic || polymorphic) + ipa_set_param_used_by_polymorphic_call + (IPA_NODE_REF (node), param_index, true); return cs; } @@ -2204,7 +2212,7 @@ ipa_analyze_indirect_call_uses (struct i tree var = SSA_NAME_VAR (target); int index = ipa_get_param_decl_index (info, var); if (index >= 0) - ipa_note_param_call (fbi->node, index, call); + ipa_note_param_call (fbi->node, index, call, false); return; } @@ -2216,7 +2224,8 @@ ipa_analyze_indirect_call_uses (struct i gimple_assign_rhs1 (def), &index, &offset, NULL, &by_ref, &guaranteed_unmodified)) { - struct cgraph_edge *cs = ipa_note_param_call (fbi->node, index, call); + struct cgraph_edge *cs = ipa_note_param_call (fbi->node, index, + call, false); cs->indirect_info->offset = offset; cs->indirect_info->agg_contents = 1; cs->indirect_info->by_ref = by_ref; @@ -2317,7 +2326,8 @@ ipa_analyze_indirect_call_uses (struct i if (index >= 0 && parm_preserved_before_stmt_p (fbi, index, call, rec)) { - struct cgraph_edge *cs = ipa_note_param_call (fbi->node, index, call); + struct cgraph_edge *cs = ipa_note_param_call (fbi->node, index, + call, false); cs->indirect_info->offset = offset; cs->indirect_info->agg_contents = 1; cs->indirect_info->member_ptr = 1; @@ -2377,7 +2387,8 @@ ipa_analyze_virtual_call_uses (struct ip return; } - struct cgraph_edge *cs = ipa_note_param_call (fbi->node, index, call); + struct cgraph_edge *cs = ipa_note_param_call (fbi->node, index, + call, true); class cgraph_indirect_call_info *ii = cs->indirect_info; ii->offset = anc_offset; ii->otr_token = tree_to_uhwi (OBJ_TYPE_REF_TOKEN (target)); @@ -3510,6 +3521,11 @@ update_indirect_edges_after_inlining (st if (ici->polymorphic && !ipa_get_jf_pass_through_type_preserved (jfunc)) ici->vptr_changed = true; + ipa_set_param_used_by_indirect_call (new_root_info, + ici->param_index, true); + if (ici->polymorphic) + ipa_set_param_used_by_polymorphic_call (new_root_info, + ici->param_index, true); } } else if (jfunc->type == IPA_JF_ANCESTOR) @@ -4055,6 +4071,12 @@ ipa_print_node_params (FILE *f, struct c ipa_dump_param (f, info, i); if (ipa_is_param_used (info, i)) fprintf (f, " used"); + if (ipa_is_param_used_by_ipa_predicates (info, i)) + fprintf (f, " used_by_ipa_predicates"); + if (ipa_is_param_used_by_indirect_call (info, i)) + fprintf (f, " used_by_indirect_call"); + if (ipa_is_param_used_by_polymorphic_call (info, i)) + fprintf (f, " used_by_polymorphic_call"); c = ipa_get_controlled_uses (info, i); if (c == IPA_UNDESCRIBED_USE) fprintf (f, " undescribed_use"); @@ -4331,7 +4353,8 @@ ipa_write_indirect_edge_info (struct out static void ipa_read_indirect_edge_info (class lto_input_block *ib, class data_in *data_in, - struct cgraph_edge *cs) + struct cgraph_edge *cs, + class ipa_node_params *info) { class cgraph_indirect_call_info *ii = cs->indirect_info; struct bitpack_d bp; @@ -4354,6 +4377,14 @@ ipa_read_indirect_edge_info (class lto_i ii->otr_type = stream_read_tree (ib, data_in); ii->context.stream_in (ib, data_in); } + if (info && ii->param_index >= 0) + { + if (ii->polymorphic) + ipa_set_param_used_by_polymorphic_call (info, + ii->param_index , true); + ipa_set_param_used_by_indirect_call (info, + ii->param_index, true); + } } /* Stream out NODE info to OB. */ @@ -4523,7 +4554,7 @@ ipa_read_node_info (class lto_input_bloc for (e = node->indirect_calls; e; e = e->next_callee) { ipa_read_edge_info (ib, data_in, e, prevails); - ipa_read_indirect_edge_info (ib, data_in, e); + ipa_read_indirect_edge_info (ib, data_in, e, info); } } Index: ipa-prop.h =================================================================== --- ipa-prop.h (revision 277757) +++ ipa-prop.h (working copy) @@ -333,9 +333,12 @@ struct GTY(()) ipa_param_descriptor says how many there are. If any use could not be described by means of ipa-prop structures, this is IPA_UNDESCRIBED_USE. */ int controlled_uses; - unsigned int move_cost : 31; + unsigned int move_cost : 28; /* The parameter is used. */ unsigned used : 1; + unsigned used_by_ipa_predicates : 1; + unsigned used_by_indirect_call : 1; + unsigned used_by_polymorphic_call : 1; }; /* ipa_node_params stores information related to formal parameters of functions @@ -519,6 +522,36 @@ ipa_set_param_used (class ipa_node_param (*info->descriptors)[i].used = val; } +/* Set the used_by_ipa_predicates flag corresponding to the Ith formal + parameter of the function associated with INFO to VAL. */ + +static inline void +ipa_set_param_used_by_ipa_predicates (class ipa_node_params *info, int i, bool val) +{ + gcc_checking_assert (info->descriptors); + (*info->descriptors)[i].used_by_ipa_predicates = val; +} + +/* Set the used_by_indirect_call flag corresponding to the Ith formal + parameter of the function associated with INFO to VAL. */ + +static inline void +ipa_set_param_used_by_indirect_call (class ipa_node_params *info, int i, bool val) +{ + gcc_checking_assert (info->descriptors); + (*info->descriptors)[i].used_by_indirect_call = val; +} + +/* Set the .used_by_polymorphic_call flag corresponding to the Ith formal + parameter of the function associated with INFO to VAL. */ + +static inline void +ipa_set_param_used_by_polymorphic_call (class ipa_node_params *info, int i, bool val) +{ + gcc_checking_assert (info->descriptors); + (*info->descriptors)[i].used_by_polymorphic_call = val; +} + /* Return how many uses described by ipa-prop a parameter has or IPA_UNDESCRIBED_USE if there is a use that is not described by these structures. */ @@ -550,6 +583,36 @@ ipa_is_param_used (class ipa_node_params return (*info->descriptors)[i].used; } +/* Return the used_by_ipa_predicates flag corresponding to the Ith formal + parameter of the function associated with INFO. */ + +static inline bool +ipa_is_param_used_by_ipa_predicates (class ipa_node_params *info, int i) +{ + gcc_checking_assert (info->descriptors); + return (*info->descriptors)[i].used_by_ipa_predicates; +} + +/* Return the used_by_indirect_call flag corresponding to the Ith formal + parameter of the function associated with INFO. */ + +static inline bool +ipa_is_param_used_by_indirect_call (class ipa_node_params *info, int i) +{ + gcc_checking_assert (info->descriptors); + return (*info->descriptors)[i].used_by_indirect_call; +} + +/* Return the used_by_polymorphic_call flag corresponding to the Ith formal + parameter of the function associated with INFO. */ + +static inline bool +ipa_is_param_used_by_polymorphic_call (class ipa_node_params *info, int i) +{ + gcc_checking_assert (info->descriptors); + return (*info->descriptors)[i].used_by_polymorphic_call; +} + /* Information about replacements done in aggregates for a given node (each node has its linked list). */ struct GTY(()) ipa_agg_replacement_value