| Message ID | 20130430142157.5C2EC804CE@tjsboxrox.mtv.corp.google.com |
|---|---|
| State | New |
| Headers | show |
On Tue, Apr 30, 2013 at 4:21 PM, Teresa Johnson <tejohnson@google.com> wrote: > Follow-on patch to r197595 to complete the replacement of truncating divides > in profile scaling code with rounding divide equivalents using helper routines > in basic-block.h. See http://gcc.gnu.org/ml/gcc-patches/2013-04/msg00321.html > for background. > > In addition to bootstrap and profiledbootstrap builds and tests (with and > without LTO), I built and tested performance of the SPEC cpu2006 benchmarks > with FDO on a Nehalem system. I didn't see any performance changes that > looked significant. > > Ok for trunk? Ok. Thanks, Richard. > Thanks, > Teresa > > 2013-04-30 Teresa Johnson <tejohnson@google.com> > > * loop-unswitch.c (unswitch_loop): Use helper routines with rounding > divides. > * cfg.c (update_bb_profile_for_threading): Ditto. > * tree-inline.c (copy_bb): Ditto. > (copy_edges_for_bb): Ditto. > (initialize_cfun): Ditto. > (copy_cfg_body): Ditto. > (expand_call_inline): Ditto. > * ipa-inline-analysis.c (estimate_edge_size_and_time): Ditto. > (estimate_node_size_and_time): Ditto. > (inline_merge_summary): Ditto. > * cgraphclones.c (cgraph_clone_edge): Ditto. > (cgraph_clone_node): Ditto. > * sched-rgn.c (compute_dom_prob_ps): Ditto. > (compute_trg_info): Ditto. > > Index: loop-unswitch.c > =================================================================== > --- loop-unswitch.c (revision 198416) > +++ loop-unswitch.c (working copy) > @@ -436,12 +436,10 @@ unswitch_loop (struct loop *loop, basic_block unsw > emit_insn_after (seq, BB_END (switch_bb)); > e = make_edge (switch_bb, true_edge->dest, 0); > e->probability = prob; > - /* Update to use apply_probability(). */ > - e->count = latch_edge->count * prob / REG_BR_PROB_BASE; > + e->count = apply_probability (latch_edge->count, prob); > e = make_edge (switch_bb, FALLTHRU_EDGE (unswitch_on)->dest, EDGE_FALLTHRU); > e->probability = false_edge->probability; > - /* Update to use apply_probability(). */ > - e->count = latch_edge->count * (false_edge->probability) / REG_BR_PROB_BASE; > + e->count = apply_probability (latch_edge->count, false_edge->probability); > > if (irred_flag) > { > Index: cfg.c > =================================================================== > --- cfg.c (revision 198416) > +++ cfg.c (working copy) > @@ -848,8 +848,7 @@ update_bb_profile_for_threading (basic_block bb, i > /* Compute the probability of TAKEN_EDGE being reached via threaded edge. > Watch for overflows. */ > if (bb->frequency) > - /* Update to use GCOV_COMPUTE_SCALE. */ > - prob = edge_frequency * REG_BR_PROB_BASE / bb->frequency; > + prob = GCOV_COMPUTE_SCALE (edge_frequency, bb->frequency); > else > prob = 0; > if (prob > taken_edge->probability) > Index: tree-inline.c > =================================================================== > --- tree-inline.c (revision 198416) > +++ tree-inline.c (working copy) > @@ -1519,13 +1519,11 @@ copy_bb (copy_body_data *id, basic_block bb, int f > basic_block_info automatically. */ > copy_basic_block = create_basic_block (NULL, (void *) 0, > (basic_block) prev->aux); > - /* Update to use apply_scale(). */ > - copy_basic_block->count = bb->count * count_scale / REG_BR_PROB_BASE; > + copy_basic_block->count = apply_scale (bb->count, count_scale); > > /* We are going to rebuild frequencies from scratch. These values > have just small importance to drive canonicalize_loop_headers. */ > - /* Update to use EDGE_FREQUENCY. */ > - freq = ((gcov_type)bb->frequency * frequency_scale / REG_BR_PROB_BASE); > + freq = apply_scale ((gcov_type)bb->frequency, frequency_scale); > > /* We recompute frequencies after inlining, so this is quite safe. */ > if (freq > BB_FREQ_MAX) > @@ -1891,8 +1889,7 @@ copy_edges_for_bb (basic_block bb, gcov_type count > && old_edge->dest->aux != EXIT_BLOCK_PTR) > flags |= EDGE_FALLTHRU; > new_edge = make_edge (new_bb, (basic_block) old_edge->dest->aux, flags); > - /* Update to use apply_scale(). */ > - new_edge->count = old_edge->count * count_scale / REG_BR_PROB_BASE; > + new_edge->count = apply_scale (old_edge->count, count_scale); > new_edge->probability = old_edge->probability; > } > > @@ -2066,10 +2063,10 @@ initialize_cfun (tree new_fndecl, tree callee_fnde > struct function *src_cfun = DECL_STRUCT_FUNCTION (callee_fndecl); > gcov_type count_scale; > > - /* Update to use GCOV_COMPUTE_SCALE. */ > if (ENTRY_BLOCK_PTR_FOR_FUNCTION (src_cfun)->count) > - count_scale = (REG_BR_PROB_BASE * count > - / ENTRY_BLOCK_PTR_FOR_FUNCTION (src_cfun)->count); > + count_scale > + = GCOV_COMPUTE_SCALE (count, > + ENTRY_BLOCK_PTR_FOR_FUNCTION (src_cfun)->count); > else > count_scale = REG_BR_PROB_BASE; > > @@ -2253,10 +2250,10 @@ copy_cfg_body (copy_body_data * id, gcov_type coun > int incoming_frequency = 0; > gcov_type incoming_count = 0; > > - /* Update to use GCOV_COMPUTE_SCALE. */ > if (ENTRY_BLOCK_PTR_FOR_FUNCTION (src_cfun)->count) > - count_scale = (REG_BR_PROB_BASE * count > - / ENTRY_BLOCK_PTR_FOR_FUNCTION (src_cfun)->count); > + count_scale > + = GCOV_COMPUTE_SCALE (count, > + ENTRY_BLOCK_PTR_FOR_FUNCTION (src_cfun)->count); > else > count_scale = REG_BR_PROB_BASE; > > @@ -2278,11 +2275,9 @@ copy_cfg_body (copy_body_data * id, gcov_type coun > incoming_frequency += EDGE_FREQUENCY (e); > incoming_count += e->count; > } > - /* Update to use apply_scale(). */ > - incoming_count = incoming_count * count_scale / REG_BR_PROB_BASE; > - /* Update to use EDGE_FREQUENCY. */ > + incoming_count = apply_scale (incoming_count, count_scale); > incoming_frequency > - = incoming_frequency * frequency_scale / REG_BR_PROB_BASE; > + = apply_scale ((gcov_type)incoming_frequency, frequency_scale); > ENTRY_BLOCK_PTR->count = incoming_count; > ENTRY_BLOCK_PTR->frequency = incoming_frequency; > } > @@ -4114,8 +4109,7 @@ expand_call_inline (basic_block bb, gimple stmt, c > a self-referential call; if we're calling ourselves, we need to > duplicate our body before altering anything. */ > copy_body (id, bb->count, > - /* Update to use GCOV_COMPUTE_SCALE. */ > - cg_edge->frequency * REG_BR_PROB_BASE / CGRAPH_FREQ_BASE, > + GCOV_COMPUTE_SCALE (cg_edge->frequency, CGRAPH_FREQ_BASE), > bb, return_block, NULL, NULL); > > /* Reset the escaped solution. */ > Index: ipa-inline-analysis.c > =================================================================== > --- ipa-inline-analysis.c (revision 198416) > +++ ipa-inline-analysis.c (working copy) > @@ -2790,8 +2790,7 @@ estimate_edge_size_and_time (struct cgraph_edge *e > && hints && cgraph_maybe_hot_edge_p (e)) > *hints |= INLINE_HINT_indirect_call; > *size += call_size * INLINE_SIZE_SCALE; > - /* Update to use apply_probability(). */ > - *time += call_time * prob / REG_BR_PROB_BASE > + *time += apply_probability ((gcov_type) call_time, prob) > * e->frequency * (INLINE_TIME_SCALE / CGRAPH_FREQ_BASE); > if (*time > MAX_TIME * INLINE_TIME_SCALE) > *time = MAX_TIME * INLINE_TIME_SCALE; > @@ -2901,8 +2900,7 @@ estimate_node_size_and_time (struct cgraph_node *n > inline_param_summary); > gcc_checking_assert (prob >= 0); > gcc_checking_assert (prob <= REG_BR_PROB_BASE); > - /* Update to use apply_probability(). */ > - time += ((gcov_type) e->time * prob) / REG_BR_PROB_BASE; > + time += apply_probability ((gcov_type) e->time, prob); > } > if (time > MAX_TIME * INLINE_TIME_SCALE) > time = MAX_TIME * INLINE_TIME_SCALE; > @@ -3311,8 +3309,7 @@ inline_merge_summary (struct cgraph_edge *edge) > int prob = predicate_probability (callee_info->conds, > &e->predicate, > clause, es->param); > - /* Update to use apply_probability(). */ > - add_time = ((gcov_type) add_time * prob) / REG_BR_PROB_BASE; > + add_time = apply_probability ((gcov_type) add_time, prob); > if (add_time > MAX_TIME * INLINE_TIME_SCALE) > add_time = MAX_TIME * INLINE_TIME_SCALE; > if (prob != REG_BR_PROB_BASE > Index: cgraphclones.c > =================================================================== > --- cgraphclones.c (revision 198416) > +++ cgraphclones.c (working copy) > @@ -102,8 +102,7 @@ cgraph_clone_edge (struct cgraph_edge *e, struct c > int freq_scale, bool update_original) > { > struct cgraph_edge *new_edge; > - /* Update this to use GCOV_COMPUTE_SCALE. */ > - gcov_type count = e->count * count_scale / REG_BR_PROB_BASE; > + gcov_type count = apply_probability (e->count, count_scale); > gcov_type freq; > > /* We do not want to ignore loop nest after frequency drops to 0. */ > @@ -205,8 +204,7 @@ cgraph_clone_node (struct cgraph_node *n, tree dec > if (new_node->count > n->count) > count_scale = REG_BR_PROB_BASE; > else > - /* Update to use GCOV_COMPUTE_SCALE. */ > - count_scale = new_node->count * REG_BR_PROB_BASE / n->count; > + count_scale = GCOV_COMPUTE_SCALE (new_node->count, n->count); > } > else > count_scale = 0; > Index: sched-rgn.c > =================================================================== > --- sched-rgn.c (revision 198416) > +++ sched-rgn.c (working copy) > @@ -1441,8 +1441,7 @@ compute_dom_prob_ps (int bb) > FOR_EACH_EDGE (out_edge, out_ei, in_edge->src->succs) > bitmap_set_bit (pot_split[bb], EDGE_TO_BIT (out_edge)); > > - /* Update to use apply_probability(). */ > - prob[bb] += ((prob[pred_bb] * in_edge->probability) / REG_BR_PROB_BASE); > + prob[bb] += combine_probabilities (prob[pred_bb], in_edge->probability); > } > > bitmap_set_bit (dom[bb], bb); > @@ -1515,8 +1514,7 @@ compute_trg_info (int trg) > int tf = prob[trg], cf = prob[i]; > > /* In CFGs with low probability edges TF can possibly be zero. */ > - /* Update to use GCOV_COMPUTE_SCALE. */ > - sp->src_prob = (tf ? ((cf * REG_BR_PROB_BASE) / tf) : 0); > + sp->src_prob = (tf ? GCOV_COMPUTE_SCALE (cf, tf) : 0); > sp->is_valid = (sp->src_prob >= min_spec_prob); > } > > > -- > This patch is available for review at http://codereview.appspot.com/9050044
Index: loop-unswitch.c =================================================================== --- loop-unswitch.c (revision 198416) +++ loop-unswitch.c (working copy) @@ -436,12 +436,10 @@ unswitch_loop (struct loop *loop, basic_block unsw emit_insn_after (seq, BB_END (switch_bb)); e = make_edge (switch_bb, true_edge->dest, 0); e->probability = prob; - /* Update to use apply_probability(). */ - e->count = latch_edge->count * prob / REG_BR_PROB_BASE; + e->count = apply_probability (latch_edge->count, prob); e = make_edge (switch_bb, FALLTHRU_EDGE (unswitch_on)->dest, EDGE_FALLTHRU); e->probability = false_edge->probability; - /* Update to use apply_probability(). */ - e->count = latch_edge->count * (false_edge->probability) / REG_BR_PROB_BASE; + e->count = apply_probability (latch_edge->count, false_edge->probability); if (irred_flag) { Index: cfg.c =================================================================== --- cfg.c (revision 198416) +++ cfg.c (working copy) @@ -848,8 +848,7 @@ update_bb_profile_for_threading (basic_block bb, i /* Compute the probability of TAKEN_EDGE being reached via threaded edge. Watch for overflows. */ if (bb->frequency) - /* Update to use GCOV_COMPUTE_SCALE. */ - prob = edge_frequency * REG_BR_PROB_BASE / bb->frequency; + prob = GCOV_COMPUTE_SCALE (edge_frequency, bb->frequency); else prob = 0; if (prob > taken_edge->probability) Index: tree-inline.c =================================================================== --- tree-inline.c (revision 198416) +++ tree-inline.c (working copy) @@ -1519,13 +1519,11 @@ copy_bb (copy_body_data *id, basic_block bb, int f basic_block_info automatically. */ copy_basic_block = create_basic_block (NULL, (void *) 0, (basic_block) prev->aux); - /* Update to use apply_scale(). */ - copy_basic_block->count = bb->count * count_scale / REG_BR_PROB_BASE; + copy_basic_block->count = apply_scale (bb->count, count_scale); /* We are going to rebuild frequencies from scratch. These values have just small importance to drive canonicalize_loop_headers. */ - /* Update to use EDGE_FREQUENCY. */ - freq = ((gcov_type)bb->frequency * frequency_scale / REG_BR_PROB_BASE); + freq = apply_scale ((gcov_type)bb->frequency, frequency_scale); /* We recompute frequencies after inlining, so this is quite safe. */ if (freq > BB_FREQ_MAX) @@ -1891,8 +1889,7 @@ copy_edges_for_bb (basic_block bb, gcov_type count && old_edge->dest->aux != EXIT_BLOCK_PTR) flags |= EDGE_FALLTHRU; new_edge = make_edge (new_bb, (basic_block) old_edge->dest->aux, flags); - /* Update to use apply_scale(). */ - new_edge->count = old_edge->count * count_scale / REG_BR_PROB_BASE; + new_edge->count = apply_scale (old_edge->count, count_scale); new_edge->probability = old_edge->probability; } @@ -2066,10 +2063,10 @@ initialize_cfun (tree new_fndecl, tree callee_fnde struct function *src_cfun = DECL_STRUCT_FUNCTION (callee_fndecl); gcov_type count_scale; - /* Update to use GCOV_COMPUTE_SCALE. */ if (ENTRY_BLOCK_PTR_FOR_FUNCTION (src_cfun)->count) - count_scale = (REG_BR_PROB_BASE * count - / ENTRY_BLOCK_PTR_FOR_FUNCTION (src_cfun)->count); + count_scale + = GCOV_COMPUTE_SCALE (count, + ENTRY_BLOCK_PTR_FOR_FUNCTION (src_cfun)->count); else count_scale = REG_BR_PROB_BASE; @@ -2253,10 +2250,10 @@ copy_cfg_body (copy_body_data * id, gcov_type coun int incoming_frequency = 0; gcov_type incoming_count = 0; - /* Update to use GCOV_COMPUTE_SCALE. */ if (ENTRY_BLOCK_PTR_FOR_FUNCTION (src_cfun)->count) - count_scale = (REG_BR_PROB_BASE * count - / ENTRY_BLOCK_PTR_FOR_FUNCTION (src_cfun)->count); + count_scale + = GCOV_COMPUTE_SCALE (count, + ENTRY_BLOCK_PTR_FOR_FUNCTION (src_cfun)->count); else count_scale = REG_BR_PROB_BASE; @@ -2278,11 +2275,9 @@ copy_cfg_body (copy_body_data * id, gcov_type coun incoming_frequency += EDGE_FREQUENCY (e); incoming_count += e->count; } - /* Update to use apply_scale(). */ - incoming_count = incoming_count * count_scale / REG_BR_PROB_BASE; - /* Update to use EDGE_FREQUENCY. */ + incoming_count = apply_scale (incoming_count, count_scale); incoming_frequency - = incoming_frequency * frequency_scale / REG_BR_PROB_BASE; + = apply_scale ((gcov_type)incoming_frequency, frequency_scale); ENTRY_BLOCK_PTR->count = incoming_count; ENTRY_BLOCK_PTR->frequency = incoming_frequency; } @@ -4114,8 +4109,7 @@ expand_call_inline (basic_block bb, gimple stmt, c a self-referential call; if we're calling ourselves, we need to duplicate our body before altering anything. */ copy_body (id, bb->count, - /* Update to use GCOV_COMPUTE_SCALE. */ - cg_edge->frequency * REG_BR_PROB_BASE / CGRAPH_FREQ_BASE, + GCOV_COMPUTE_SCALE (cg_edge->frequency, CGRAPH_FREQ_BASE), bb, return_block, NULL, NULL); /* Reset the escaped solution. */ Index: ipa-inline-analysis.c =================================================================== --- ipa-inline-analysis.c (revision 198416) +++ ipa-inline-analysis.c (working copy) @@ -2790,8 +2790,7 @@ estimate_edge_size_and_time (struct cgraph_edge *e && hints && cgraph_maybe_hot_edge_p (e)) *hints |= INLINE_HINT_indirect_call; *size += call_size * INLINE_SIZE_SCALE; - /* Update to use apply_probability(). */ - *time += call_time * prob / REG_BR_PROB_BASE + *time += apply_probability ((gcov_type) call_time, prob) * e->frequency * (INLINE_TIME_SCALE / CGRAPH_FREQ_BASE); if (*time > MAX_TIME * INLINE_TIME_SCALE) *time = MAX_TIME * INLINE_TIME_SCALE; @@ -2901,8 +2900,7 @@ estimate_node_size_and_time (struct cgraph_node *n inline_param_summary); gcc_checking_assert (prob >= 0); gcc_checking_assert (prob <= REG_BR_PROB_BASE); - /* Update to use apply_probability(). */ - time += ((gcov_type) e->time * prob) / REG_BR_PROB_BASE; + time += apply_probability ((gcov_type) e->time, prob); } if (time > MAX_TIME * INLINE_TIME_SCALE) time = MAX_TIME * INLINE_TIME_SCALE; @@ -3311,8 +3309,7 @@ inline_merge_summary (struct cgraph_edge *edge) int prob = predicate_probability (callee_info->conds, &e->predicate, clause, es->param); - /* Update to use apply_probability(). */ - add_time = ((gcov_type) add_time * prob) / REG_BR_PROB_BASE; + add_time = apply_probability ((gcov_type) add_time, prob); if (add_time > MAX_TIME * INLINE_TIME_SCALE) add_time = MAX_TIME * INLINE_TIME_SCALE; if (prob != REG_BR_PROB_BASE Index: cgraphclones.c =================================================================== --- cgraphclones.c (revision 198416) +++ cgraphclones.c (working copy) @@ -102,8 +102,7 @@ cgraph_clone_edge (struct cgraph_edge *e, struct c int freq_scale, bool update_original) { struct cgraph_edge *new_edge; - /* Update this to use GCOV_COMPUTE_SCALE. */ - gcov_type count = e->count * count_scale / REG_BR_PROB_BASE; + gcov_type count = apply_probability (e->count, count_scale); gcov_type freq; /* We do not want to ignore loop nest after frequency drops to 0. */ @@ -205,8 +204,7 @@ cgraph_clone_node (struct cgraph_node *n, tree dec if (new_node->count > n->count) count_scale = REG_BR_PROB_BASE; else - /* Update to use GCOV_COMPUTE_SCALE. */ - count_scale = new_node->count * REG_BR_PROB_BASE / n->count; + count_scale = GCOV_COMPUTE_SCALE (new_node->count, n->count); } else count_scale = 0; Index: sched-rgn.c =================================================================== --- sched-rgn.c (revision 198416) +++ sched-rgn.c (working copy) @@ -1441,8 +1441,7 @@ compute_dom_prob_ps (int bb) FOR_EACH_EDGE (out_edge, out_ei, in_edge->src->succs) bitmap_set_bit (pot_split[bb], EDGE_TO_BIT (out_edge)); - /* Update to use apply_probability(). */ - prob[bb] += ((prob[pred_bb] * in_edge->probability) / REG_BR_PROB_BASE); + prob[bb] += combine_probabilities (prob[pred_bb], in_edge->probability); } bitmap_set_bit (dom[bb], bb); @@ -1515,8 +1514,7 @@ compute_trg_info (int trg) int tf = prob[trg], cf = prob[i]; /* In CFGs with low probability edges TF can possibly be zero. */ - /* Update to use GCOV_COMPUTE_SCALE. */ - sp->src_prob = (tf ? ((cf * REG_BR_PROB_BASE) / tf) : 0); + sp->src_prob = (tf ? GCOV_COMPUTE_SCALE (cf, tf) : 0); sp->is_valid = (sp->src_prob >= min_spec_prob); }
Follow-on patch to r197595 to complete the replacement of truncating divides in profile scaling code with rounding divide equivalents using helper routines in basic-block.h. See http://gcc.gnu.org/ml/gcc-patches/2013-04/msg00321.html for background. In addition to bootstrap and profiledbootstrap builds and tests (with and without LTO), I built and tested performance of the SPEC cpu2006 benchmarks with FDO on a Nehalem system. I didn't see any performance changes that looked significant. Ok for trunk? Thanks, Teresa 2013-04-30 Teresa Johnson <tejohnson@google.com> * loop-unswitch.c (unswitch_loop): Use helper routines with rounding divides. * cfg.c (update_bb_profile_for_threading): Ditto. * tree-inline.c (copy_bb): Ditto. (copy_edges_for_bb): Ditto. (initialize_cfun): Ditto. (copy_cfg_body): Ditto. (expand_call_inline): Ditto. * ipa-inline-analysis.c (estimate_edge_size_and_time): Ditto. (estimate_node_size_and_time): Ditto. (inline_merge_summary): Ditto. * cgraphclones.c (cgraph_clone_edge): Ditto. (cgraph_clone_node): Ditto. * sched-rgn.c (compute_dom_prob_ps): Ditto. (compute_trg_info): Ditto. -- This patch is available for review at http://codereview.appspot.com/9050044