diff mbox

New IPA-CP with real function cloning

Message ID 20110623192404.GC2736@virgil.arch.suse.de
State New
Headers show

Commit Message

Martin Jambor June 23, 2011, 7:24 p.m. UTC
Hi,

this is an updated version of a patch I've posted here before:

http://gcc.gnu.org/ml/gcc-patches/2011-06/msg01161.html and
http://gcc.gnu.org/ml/gcc-patches/2011-06/msg00557.html

It is a new IPA-CP that now works well with the alias infrastructure
Honza introduced over the weekend.  It applies to trunk revision
175247.

Everything in the previous email messages still applies.  It
bootstraps, profile-bootstraps and tests well and Honza even tried
LTO-building Firefox with (a previous version of) it and it survived.
It brings about big speedup on c-ray, SPEC 2006 numbers are within
noise.

This is indeed the pass I have briefly introduced it at the Gathering
in London this weekend.

As in the previous case, below you'll find the new ipa-prop.h and
ipa-cp.c, the actual context diff is attached.  I believe it is more
convenient that way.

Thanks a lot for any comments,

Martin


2011-06-22  Martin Jambor  <mjambor@suse.cz>

	* ipa-prop.h: Include alloc-pool.h.
	(ipa_lattice_type): Removed.
	(ipcp_value_source): New type.
	(ipcp_value): Likewise.
	(ipcp_values_pool): Declare.
	(ipcp_sources_pool): Likewise.
	(ipa_param_descriptor): Removed.
	(ipcp_lattice): Removed fileds type and constant. Added fields decl,
	values, values_count, contains_variable, bottom, used and virt_call.
	(ipa_node_params): New fields lattices, known_vals,
	clone_for_all_contexts and noe dead, removed fields params and
	count_scale.
	(ipa_get_param): Updated.
	(ipa_param_cannot_devirtualize_p): Removed.
	(ipa_param_types_vec_empty): Likewise.
	(ipa_edge_args): New field next_edge_clone.
	(ipa_func_list): Removed.
	(ipa_init_func_list): Removed declaration.
	(ipa_push_func_to_list_1): Likewise.
	(ipa_pop_func_from_list): Likewise.
	(ipa_push_func_to_list): Removed.
	(ipa_lattice_from_jfunc): Remove declaration.
	(ipa_get_jf_pass_through_result): Declare.
	(ipa_get_jf_ancestor_result): Likewise.
	(ipa_value_from_jfunc): Likewise.
	(ipa_get_lattice): Update.
	(ipa_lat_is_single_const): New function.
	* ipa-prop.c (ipa_push_func_to_list_1): Removed.
	(ipa_init_func_list): Likewise.
	(ipa_pop_func_from_list): Likewise.
	(ipa_get_param_decl_index): Fix coding style.
	(ipa_populate_param_decls): Update to use new lattices.
	(ipa_initialize_node_params): Likewise.
	(visit_ref_for_mod_analysis): Likewise.
	(ipa_analyze_params_uses): Likewise.
	(ipa_free_node_params_substructures): Likewise.
	(ipa_edge_duplication_hook): Add the new edge to the list of edge
	clones.
	(ipa_node_duplication_hook): Update to use new lattices.
	(ipa_free_all_structures_after_ipa_cp): Free alloc pools.
	(ipa_free_all_structures_after_iinln): Likewise.
	(ipa_write_node_info): Update to use new lattices.
	(ipa_read_node_info): Likewise.
	(ipa_get_jf_pass_through_result): New function.
	(ipa_get_jf_ancestor_result): Likewise.
	(ipa_value_from_jfunc): Likewise.
	(ipa_cst_from_jfunc): Reimplemented using ipa_value_from_jfunc.
	* ipa-cp.c: Reimplemented.
	* params.def (PARAM_DEVIRT_TYPE_LIST_SIZE): Removed.
	(PARAM_IPA_CP_VALUE_LIST_SIZE): New parameter.
	* Makefile.in (IPA_PROP_H): Added alloc-pool.h to dependencies.

	* doc/invoke.texi (devirt-type-list-size): Removed description.
	(ipa-cp-value-list-size): Added description.

	* testsuite/gcc.dg/ipa/ipa-1.c: Updated testcase dump scan.
	* testsuite/gcc.dg/ipa/ipa-2.c: Likewise.
	* testsuite/gcc.dg/ipa/ipa-3.c: Likewise and made functions static.
	* testsuite/gcc.dg/ipa/ipa-4.c: Updated testcase dump scan.
	* testsuite/gcc.dg/ipa/ipa-5.c: Likewise.
	* testsuite/gcc.dg/ipa/ipa-7.c: Xfail test.
	* testsuite/gcc.dg/ipa/ipa-8.c: Updated testcase dump scan.
	* testsuite/gcc.dg/ipa/ipacost-1.c: Likewise.
	* testsuite/gcc.dg/ipa/ipacost-2.c: Likewise.
	* testsuite/gcc.dg/ipa/ipcp-1.c: New test.
	* testsuite/gcc.dg/ipa/ipcp-2.c: Likewise.
	* testsuite/gcc.dg/tree-ssa/ipa-cp-1.c: Updated testcase.

==================== ipa-prop.h ====================
/* Interprocedural analyses.
   Copyright (C) 2005, 2007, 2008, 2009, 2010
   Free Software Foundation, Inc.

This file is part of GCC.

GCC is free software; you can redistribute it and/or modify it under
the terms of the GNU General Public License as published by the Free
Software Foundation; either version 3, or (at your option) any later
version.

GCC is distributed in the hope that it will be useful, but WITHOUT ANY
WARRANTY; without even the implied warranty of MERCHANTABILITY or
FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License
for more details.

You should have received a copy of the GNU General Public License
along with GCC; see the file COPYING3.  If not see
<http://www.gnu.org/licenses/>.  */

#ifndef IPA_PROP_H
#define IPA_PROP_H

#include "tree.h"
#include "vec.h"
#include "cgraph.h"
#include "gimple.h"
#include "alloc-pool.h"

/* The following definitions and interfaces are used by
   interprocedural analyses or parameters.  */

/* ipa-prop.c stuff (ipa-cp, indirect inlining):  */

/* A jump function for a callsite represents the values passed as actual
   arguments of the callsite. There are three main types of values :

   Pass-through - the caller's formal parameter is passed as an actual
                  argument, possibly one simple operation performed on it.
   Constant     - a constant (is_gimple_ip_invariant)is passed as an actual
                  argument.
   Unknown      - neither of the above.

   IPA_JF_CONST_MEMBER_PTR stands for C++ member pointers, it is a special
   constant in this regard.  Other constants are represented with IPA_JF_CONST.

   IPA_JF_ANCESTOR is a special pass-through jump function, which means that
   the result is an address of a part of the object pointed to by the formal
   parameter to which the function refers.  It is mainly intended to represent
   getting addresses of of ancestor fields in C++
   (e.g. &this_1(D)->D.1766.D.1756).  Note that if the original pointer is
   NULL, ancestor jump function must behave like a simple pass-through.

   Other pass-through functions can either simply pass on an unchanged formal
   parameter or can apply one simple binary operation to it (such jump
   functions are called polynomial).

   IPA_JF_KNOWN_TYPE is a special type of an "unknown" function that applies
   only to pointer parameters.  It means that even though we cannot prove that
   the passed value is an interprocedural constant, we still know the exact
   type of the containing object which may be valuable for devirtualization.

   Jump functions are computed in ipa-prop.c by function
   update_call_notes_after_inlining.  Some information can be lost and jump
   functions degraded accordingly when inlining, see
   update_call_notes_after_inlining in the same file.  */

enum jump_func_type
{
  IPA_JF_UNKNOWN = 0,  /* newly allocated and zeroed jump functions default */
  IPA_JF_KNOWN_TYPE,        /* represented by field base_binfo */
  IPA_JF_CONST,             /* represented by field costant */
  IPA_JF_CONST_MEMBER_PTR,  /* represented by field member_cst */
  IPA_JF_PASS_THROUGH,	    /* represented by field pass_through */
  IPA_JF_ANCESTOR	    /* represented by field ancestor */
};

/* Structure holding data required to describe a pass-through jump function.  */

struct GTY(()) ipa_pass_through_data
{
  /* If an operation is to be performed on the original parameter, this is the
     second (constant) operand.  */
  tree operand;
  /* Number of the caller's formal parameter being passed.  */
  int formal_id;
  /* Operation that is performed on the argument before it is passed on.
     NOP_EXPR means no operation.  Otherwise oper must be a simple binary
     arithmetic operation where the caller's parameter is the first operand and
     operand field from this structure is the second one.  */
  enum tree_code operation;
};

/* Structure holding data required to describe an ancestor pass-through
   jump function.  */

struct GTY(()) ipa_ancestor_jf_data
{
  /* Offset of the field representing the ancestor.  */
  HOST_WIDE_INT offset;
  /* TYpe of the result.  */
  tree type;
  /* Number of the caller's formal parameter being passed.  */
  int formal_id;
};

/* Structure holding a C++ member pointer constant.  Holds a pointer to the
   method and delta offset.  */
struct GTY(()) ipa_member_ptr_cst
{
  tree pfn;
  tree delta;
};

/* A jump function for a callsite represents the values passed as actual
   arguments of the callsite. See enum jump_func_type for the various
   types of jump functions supported.  */
struct GTY (()) ipa_jump_func
{
  enum jump_func_type type;
  /* Represents a value of a jump function.  pass_through is used only in jump
     function context.  constant represents the actual constant in constant jump
     functions and member_cst holds constant c++ member functions.  */
  union jump_func_value
  {
    tree GTY ((tag ("IPA_JF_KNOWN_TYPE"))) base_binfo;
    tree GTY ((tag ("IPA_JF_CONST"))) constant;
    struct ipa_member_ptr_cst GTY ((tag ("IPA_JF_CONST_MEMBER_PTR"))) member_cst;
    struct ipa_pass_through_data GTY ((tag ("IPA_JF_PASS_THROUGH"))) pass_through;
    struct ipa_ancestor_jf_data GTY ((tag ("IPA_JF_ANCESTOR"))) ancestor;
  } GTY ((desc ("%1.type"))) value;
};

struct ipcp_value;

/* Describes a particular source for an IPA-CP value.  */

struct ipcp_value_source
{
  /* The incoming edge that brought the value.  */
  struct cgraph_edge *cs;
  /* If the jump function that resulted into his value was a pass-through or an
     ancestor, this is the ipcp_value of the caller from which the described
     value has been derived.  Otherwise it is NULL.  */
  struct ipcp_value *val;
  /* Next pointer in a linked list of sources of a value.  */
  struct ipcp_value_source *next;
  /* If the jump function that resulted into his value was a pass-through or an
     ancestor, this is the index of the parameter of the caller the jump
     function references.  */
  int index;
};

/* Describes one particular value stored in struct ipcp_lattice.  */

struct ipcp_value
{
  /* The actual value for the given parameter.  This is either an IPA invariant
     or a TREE_BINFO describing a type that can be used for
     devirtualization.  */
  tree value;
  /* The list of sources from which this value originates.  */
  struct ipcp_value_source *sources;
  /* Next pointers in a linked list of all values in a lattice.  */
  struct ipcp_value *next;
  /* Next pointers in a linked list of values in a strongly connected component
     of values. */
  struct ipcp_value *scc_next;
  /* Next pointers in a linked list of SCCs of values sorted topologically
     according their sources.  */
  struct ipcp_value  *topo_next;
  /* A specialized node created for this value, NULL if none has been (so far)
     created.  */
  struct cgraph_node *spec_node;
  /* Depth first search number and low link for topological sorting of
     values.  */
  int dfs, low_link;
  /* Time benefit and size cost that specializing the function for this value
     would bring about in this function alone.  */
  int local_time_benefit, local_size_cost;
  /* Time benefit and size cost that specializing the function for this value
     can bring about in it's callees (transitively).  */
  int prop_time_benefit, prop_size_cost;
  /* True if this valye is currently on the topo-sort stack.  */
  bool on_stack;
};

extern alloc_pool ipcp_values_pool;
extern alloc_pool ipcp_sources_pool;

/* Lattice describing potential values of a formal parameter of a function and
   some of their other properties.  */

struct ipcp_lattice
{
  /* PARAM_DECL of this parameter.  */
  tree decl;

  /* The list of known values and types in this lattice.  */
  struct ipcp_value *values;
  /* Number of known values and types in this lattice.  */
  int values_count;
  /* The lattice contains a variable component  (in addition to values).  */
  bool contains_variable;
  /* The value of the lattice is bottom (i.e. variable and unusable for any
     propagation).  */
  bool bottom;
  /* The parameter is used.  */
  bool used;
  /* There is a virtual call based on this parameter.  */
  bool virt_call;
};

/* ipa_node_params stores information related to formal parameters of functions
   and some other information for interprocedural passes that operate on
   parameters (such as ipa-cp).  */
struct ipa_node_params
{
  /* Pointer to an array of structures describing individual formal
     parameters.  */
  struct ipcp_lattice *lattices;
  /* Only for versioned nodes this field would not be NULL,
     it points to the node that IPA cp cloned from.  */
  struct cgraph_node *ipcp_orig_node;
  /* Number of formal parameters of this function.  When set to 0, this
     function's parameters would not be analyzed by IPA CP.  */

  /* If this node is an ipa-cp clone, these are the known values that describe
     what it has been specialized for.  */
  VEC (tree, heap) *known_vals;
  int param_count;
  /* Whether this function is called with variable number of actual
     arguments.  */
  unsigned called_with_var_arguments : 1;
  /* Set when it is possible to create specialized versions of this node.  */
  unsigned node_versionable : 1;
  /* Whether the param uses analysis has already been performed.  */
  unsigned uses_analysis_done : 1;
  /* Whether the function is enqueued in ipa-cp propagation stack.  */
  unsigned node_enqueued : 1;
  /* Whether we should create a specialized version based on values that are
     known to be constant in all contexts.  */
  unsigned clone_for_all_contexts : 1;
  /* Node has been completely replaced by clones and will be removed after
     ipa-cp is finished.  */
  unsigned node_dead : 1;
};

/* ipa_node_params access functions.  Please use these to access fields that
   are or will be shared among various passes.  */

/* Set the number of formal parameters. */

static inline void
ipa_set_param_count (struct ipa_node_params *info, int count)
{
  info->param_count = count;
}

/* Return the number of formal parameters. */

static inline int
ipa_get_param_count (struct ipa_node_params *info)
{
  return info->param_count;
}

/* Return the declaration of Ith formal parameter of the function corresponding
   to INFO.  Note there is no setter function as this array is built just once
   using ipa_initialize_node_params. */

static inline tree
ipa_get_param (struct ipa_node_params *info, int i)
{
  gcc_assert (i >= 0 && i <= info->param_count);
  return info->lattices[i].decl;
}

/* Return the used flag corresponding to the Ith formal parameter of
   the function associated with INFO.  */

static inline bool
ipa_is_param_used (struct ipa_node_params *info, int i)
{
  gcc_assert (i >= 0 && i <= info->param_count);
  return info->lattices[i].used;
}

/* Flag this node as having callers with variable number of arguments.  */

static inline void
ipa_set_called_with_variable_arg (struct ipa_node_params *info)
{
  info->called_with_var_arguments = 1;
}

/* Have we detected this node was called with variable number of arguments? */

static inline bool
ipa_is_called_with_var_arguments (struct ipa_node_params *info)
{
  return info->called_with_var_arguments;
}



/* ipa_edge_args stores information related to a callsite and particularly its
   arguments.  It can be accessed by the IPA_EDGE_REF macro.  */
typedef struct GTY(()) ipa_edge_args
{
  /* Next pointer in a linked list of clones of the same function.  */
  struct cgraph_edge *next_edge_clone;

  /* Number of actual arguments in this callsite.  When set to 0,
     this callsite's parameters would not be analyzed by the different
     stages of IPA CP.  */
  int argument_count;
  /* Array of the callsite's jump function of each parameter.  */
  struct ipa_jump_func GTY ((length ("%h.argument_count"))) *jump_functions;
} ipa_edge_args_t;

/* ipa_edge_args access functions.  Please use these to access fields that
   are or will be shared among various passes.  */

/* Set the number of actual arguments. */

static inline void
ipa_set_cs_argument_count (struct ipa_edge_args *args, int count)
{
  args->argument_count = count;
}

/* Return the number of actual arguments. */

static inline int
ipa_get_cs_argument_count (struct ipa_edge_args *args)
{
  return args->argument_count;
}

/* Returns a pointer to the jump function for the ith argument.  Please note
   there is no setter function as jump functions are all set up in
   ipa_compute_jump_functions. */

static inline struct ipa_jump_func *
ipa_get_ith_jump_func (struct ipa_edge_args *args, int i)
{
  gcc_assert (i >= 0 && i <= args->argument_count);
  return &args->jump_functions[i];
}

/* Vectors need to have typedefs of structures.  */
typedef struct ipa_node_params ipa_node_params_t;

/* Types of vectors holding the infos.  */
DEF_VEC_O (ipa_node_params_t);
DEF_VEC_ALLOC_O (ipa_node_params_t, heap);
DEF_VEC_O (ipa_edge_args_t);
DEF_VEC_ALLOC_O (ipa_edge_args_t, gc);

/* Vector where the parameter infos are actually stored. */
extern VEC (ipa_node_params_t, heap) *ipa_node_params_vector;
/* Vector where the parameter infos are actually stored. */
extern GTY(()) VEC (ipa_edge_args_t, gc) *ipa_edge_args_vector;

/* Return the associated parameter/argument info corresponding to the given
   node/edge.  */
#define IPA_NODE_REF(NODE) (VEC_index (ipa_node_params_t, \
				       ipa_node_params_vector, (NODE)->uid))
#define IPA_EDGE_REF(EDGE) (VEC_index (ipa_edge_args_t, \
				       ipa_edge_args_vector, (EDGE)->uid))
/* This macro checks validity of index returned by
   ipa_get_param_decl_index function.  */
#define IS_VALID_JUMP_FUNC_INDEX(I) ((I) != -1)

/* Creating and freeing ipa_node_params and ipa_edge_args.  */
void ipa_create_all_node_params (void);
void ipa_create_all_edge_args (void);
void ipa_free_edge_args_substructures (struct ipa_edge_args *);
void ipa_free_node_params_substructures (struct ipa_node_params *);
void ipa_free_all_node_params (void);
void ipa_free_all_edge_args (void);
void ipa_create_all_structures_for_iinln (void);
void ipa_free_all_structures_after_ipa_cp (void);
void ipa_free_all_structures_after_iinln (void);
void ipa_register_cgraph_hooks (void);

/* This function ensures the array of node param infos is big enough to
   accommodate a structure for all nodes and reallocates it if not.  */

static inline void
ipa_check_create_node_params (void)
{
  if (!ipa_node_params_vector)
    ipa_node_params_vector = VEC_alloc (ipa_node_params_t, heap,
					cgraph_max_uid);

  if (VEC_length (ipa_node_params_t, ipa_node_params_vector)
      <= (unsigned) cgraph_max_uid)
    VEC_safe_grow_cleared (ipa_node_params_t, heap,
			   ipa_node_params_vector, cgraph_max_uid + 1);
}

/* This function ensures the array of edge arguments infos is big enough to
   accommodate a structure for all edges and reallocates it if not.  */

static inline void
ipa_check_create_edge_args (void)
{
  if (!ipa_edge_args_vector)
    ipa_edge_args_vector = VEC_alloc (ipa_edge_args_t, gc,
				      cgraph_edge_max_uid);

  if (VEC_length (ipa_edge_args_t, ipa_edge_args_vector)
      <=  (unsigned) cgraph_edge_max_uid)
    VEC_safe_grow_cleared (ipa_edge_args_t, gc, ipa_edge_args_vector,
			   cgraph_edge_max_uid + 1);
}

/* Returns true if the array of edge infos is large enough to accommodate an
   info for EDGE.  The main purpose of this function is that debug dumping
   function can check info availability without causing reallocations.  */

static inline bool
ipa_edge_args_info_available_for_edge_p (struct cgraph_edge *edge)
{
  return ((unsigned) edge->uid < VEC_length (ipa_edge_args_t,
					     ipa_edge_args_vector));
}

/* Function formal parameters related computations.  */
void ipa_initialize_node_params (struct cgraph_node *node);
bool ipa_propagate_indirect_call_infos (struct cgraph_edge *cs,
					VEC (cgraph_edge_p, heap) **new_edges);

/* Indirect edge and binfo processing.  */
struct cgraph_edge *ipa_make_edge_direct_to_target (struct cgraph_edge *, tree,
						    tree);

/* Functions related to both.  */
void ipa_analyze_node (struct cgraph_node *);

/* Debugging interface.  */
void ipa_print_node_params (FILE *, struct cgraph_node *node);
void ipa_print_all_params (FILE *);
void ipa_print_node_jump_functions (FILE *f, struct cgraph_node *node);
void ipa_print_all_jump_functions (FILE * f);

/* Structure to describe transformations of formal parameters and actual
   arguments.  Each instance describes one new parameter and they are meant to
   be stored in a vector.  Additionally, most users will probably want to store
   adjustments about parameters that are being removed altogether so that SSA
   names belonging to them can be replaced by SSA names of an artificial
   variable.  */
struct ipa_parm_adjustment
{
  /* The original PARM_DECL itself, helpful for processing of the body of the
     function itself.  Intended for traversing function bodies.
     ipa_modify_formal_parameters, ipa_modify_call_arguments and
     ipa_combine_adjustments ignore this and use base_index.
     ipa_modify_formal_parameters actually sets this.  */
  tree base;

  /* Type of the new parameter.  However, if by_ref is true, the real type will
     be a pointer to this type.  */
  tree type;

  /* Alias refrerence type to be used in MEM_REFs when adjusting caller
     arguments.  */
  tree alias_ptr_type;

  /* The new declaration when creating/replacing a parameter.  Created by
     ipa_modify_formal_parameters, useful for functions modifying the body
     accordingly. */
  tree reduction;

  /* New declaration of a substitute variable that we may use to replace all
     non-default-def ssa names when a parm decl is going away.  */
  tree new_ssa_base;

  /* If non-NULL and the original parameter is to be removed (copy_param below
     is NULL), this is going to be its nonlocalized vars value.  */
  tree nonlocal_value;

  /* Offset into the original parameter (for the cases when the new parameter
     is a component of an original one).  */
  HOST_WIDE_INT offset;

  /* Zero based index of the original parameter this one is based on.  (ATM
     there is no way to insert a new parameter out of the blue because there is
     no need but if it arises the code can be easily exteded to do so.)  */
  int base_index;

  /* This new parameter is an unmodified parameter at index base_index. */
  unsigned copy_param : 1;

  /* This adjustment describes a parameter that is about to be removed
     completely.  Most users will probably need to book keep those so that they
     don't leave behinfd any non default def ssa names belonging to them.  */
  unsigned remove_param : 1;

  /* The parameter is to be passed by reference.  */
  unsigned by_ref : 1;
};

typedef struct ipa_parm_adjustment ipa_parm_adjustment_t;
DEF_VEC_O (ipa_parm_adjustment_t);
DEF_VEC_ALLOC_O (ipa_parm_adjustment_t, heap);

typedef VEC (ipa_parm_adjustment_t, heap) *ipa_parm_adjustment_vec;

VEC(tree, heap) *ipa_get_vector_of_formal_parms (tree fndecl);
void ipa_modify_formal_parameters (tree fndecl, ipa_parm_adjustment_vec,
				   const char *);
void ipa_modify_call_arguments (struct cgraph_edge *, gimple,
				ipa_parm_adjustment_vec);
ipa_parm_adjustment_vec ipa_combine_adjustments (ipa_parm_adjustment_vec,
						 ipa_parm_adjustment_vec);
void ipa_dump_param_adjustments (FILE *, ipa_parm_adjustment_vec, tree);

void ipa_prop_write_jump_functions (cgraph_node_set set);
void ipa_prop_read_jump_functions (void);
void ipa_update_after_lto_read (void);
int ipa_get_param_decl_index (struct ipa_node_params *, tree);
tree ipa_get_jf_pass_through_result (struct ipa_jump_func *, tree);
tree ipa_get_jf_ancestor_result (struct ipa_jump_func *, tree);
tree ipa_value_from_jfunc (struct ipa_node_params *info,
			   struct ipa_jump_func *jfunc);
tree ipa_cst_from_jfunc (struct ipa_node_params *info,
			 struct ipa_jump_func *jfunc);


/* From tree-sra.c:  */
tree build_ref_for_offset (location_t, tree, HOST_WIDE_INT, tree,
			   gimple_stmt_iterator *, bool);

/* Return the lattice corresponding to the Ith formal parameter of the function
   described by INFO.  */
static inline struct ipcp_lattice *
ipa_get_lattice (struct ipa_node_params *info, int i)
{
  gcc_assert (i >= 0 && i <= info->param_count);
  return &(info->lattices[i]);
}

/* Return whether LAT is a lattice with a single constant and without an
   undefined value.  */

static inline bool
ipa_lat_is_single_const (struct ipcp_lattice *lat)
{
  if (lat->bottom
      || lat->contains_variable
      || lat->values_count != 1)
    return false;
  else
    return true;
}

#endif /* IPA_PROP_H */

==================== ipa-cp.c ====================

/* Interprocedural constant propagation
   Copyright (C) 2005, 2006, 2007, 2008, 2009, 2010, 2011
   Free Software Foundation, Inc.
   Contributed by Razya Ladelsky <RAZYA@il.ibm.com>

This file is part of GCC.

GCC is free software; you can redistribute it and/or modify it under
the terms of the GNU General Public License as published by the Free
Software Foundation; either version 3, or (at your option) any later
version.

GCC is distributed in the hope that it will be useful, but WITHOUT ANY
WARRANTY; without even the implied warranty of MERCHANTABILITY or
FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License
for more details.

You should have received a copy of the GNU General Public License
along with GCC; see the file COPYING3.  If not see
<http://www.gnu.org/licenses/>.  */

/* Interprocedural constant propagation (IPA-CP).

   The goal of this transformation is to

   1) discover functions which are always invoked with some arguments with the
      same known constant values and modify the functions so that the
      subsequent optimizations can take advantage of the knowledge, and

   2) create specialized versions of functions transformed in this way if
      some parameters are known constants only in certain contexts but the
      estimated tradeoff between speedup and cost size is deemed good.

   The algorithm also propagates types and attempts to perform type based
   devirtualization.  Types are propagated much like constants.

   The algorithm basically consists of three stages.  In the first, functions
   are analyzed one at a time and jump functions are constructed for all known
   call-sites.  In the second phase, the pass propagates information from the
   jump functions across the call to reveal what values are available at what
   call sites, performs estimations of effects of known values on functions and
   their callees, and finally decides what specialized extra versions should be
   created.  In the third, the special versions materialize and appropriate
   calls are redirected.

   The algorithm used is to a certain extent based on "Interprocedural Constant
   Propagation", by David Callahan, Keith D Cooper, Ken Kennedy, Linda Torczon,
   Comp86, pg 152-161 and "A Methodology for Procedure Cloning" by Keith D
   Cooper, Mary W. Hall, and Ken Kennedy.


   First stage - intraprocedural analysis
   =======================================

   This phase computes jump_function and modification flags.

   A jump function for a call-site represents the values passed as an actual
   arguments of a given call-site. In principle, there are three types of
   values:

   Pass through - the caller's formal parameter is passed as an actual
                  argument, plus an operation on it can be performed.
   Constant - a constant is passed as an actual argument.
   Unknown - neither of the above.

   All jump function types are described in detail in ipa-prop.h, together with
   the data structures that represent them and methods of accessing them.

   ipcp_generate_summary() is the main function of the first stage.

   Second stage - interprocedural analysis
   ========================================

   This stage is itself divided into two phases.  In the first, we propagate
   known values over the call graph, in the second, we make cloning decisions.

   First, we traverse the functions topologically from callers to callees and,
   for each strongly connected component (SCC), we propagate constants
   according to previously computed jump functions.  We also record what known
   values depend on other known values and estimate local effects.  Finally, we
   propagate cumulative information about these effects from dependant values
   to those on which they depend.

   Second, we again traverse the call graph in the same topological order and
   make clones for functions which we know are called with the same values in
   all contexts and decide about extra specialized clones of functions just for
   some contexts - these decisions are based on both local estimates and
   cumulative estimates propagated from callees.

   ipcp_propagate_stage() and ipcp_decision_stage() together constitute the
   third stage.

   Third phase - materialization of clones, call statement updates.
   ============================================

   This stage is currently performed by call graph code (mainly in cgraphunit.c
   and tree-inline.c) according to instructions inserted to the call graph by
   the second stage.  */

#include "config.h"
#include "system.h"
#include "coretypes.h"
#include "tree.h"
#include "target.h"
#include "gimple.h"
#include "cgraph.h"
#include "ipa-prop.h"
#include "tree-flow.h"
#include "tree-pass.h"
#include "flags.h"
#include "timevar.h"
#include "diagnostic.h"
#include "tree-pretty-print.h"
#include "tree-dump.h"
#include "tree-inline.h"
#include "fibheap.h"
#include "params.h"
#include "ipa-inline.h"
#include "ipa-utils.h"

/* Maximal count found in program.  */

static gcov_type max_count;

/* Original overall size of the program.  */

static long overall_size, max_new_size;

/* Allocation pools for values and their sources.  */

alloc_pool ipcp_values_pool;
alloc_pool ipcp_sources_pool;

/* Head of the linked list of topologically sorted values. */

static struct ipcp_value *values_topo;

/* Return true iff the CS is an edge within a strongly connected component as
   computed by ipa_reduced_postorder.  */

static inline bool
edge_within_scc (struct cgraph_edge *cs)
{
  struct ipa_dfs_info *caller_dfs = (struct ipa_dfs_info *) cs->caller->aux;
  struct ipa_dfs_info *callee_dfs;
  struct cgraph_node *callee = cgraph_function_node (cs->callee, NULL);

  callee_dfs = (struct ipa_dfs_info *) callee->aux;
  return (caller_dfs
	  && callee_dfs
	  && caller_dfs->scc_no == callee_dfs->scc_no);
}

/* Print V which is extracted from a value in a lattice to F.  */

static void
print_ipcp_constant_value (FILE * f, tree v)
{
  if (TREE_CODE (v) == TREE_BINFO)
    {
      fprintf (f, "BINFO ");
      print_generic_expr (f, BINFO_TYPE (v), 0);
    }
  else if (TREE_CODE (v) == ADDR_EXPR
	   && TREE_CODE (TREE_OPERAND (v, 0)) == CONST_DECL)
    {
      fprintf (f, "& ");
      print_generic_expr (f, DECL_INITIAL (TREE_OPERAND (v, 0)), 0);
    }
  else
    print_generic_expr (f, v, 0);
}

/* Print all ipcp_lattices of all functions to F.  */

static void
print_all_lattices (FILE * f, bool dump_sources, bool dump_benefits)
{
  struct cgraph_node *node;
  int i, count;

  fprintf (f, "\nLattices:\n");
  FOR_EACH_FUNCTION_WITH_GIMPLE_BODY (node)
    {
      struct ipa_node_params *info;

      info = IPA_NODE_REF (node);
      fprintf (f, "  Node: %s/%i:\n", cgraph_node_name (node), node->uid);
      count = ipa_get_param_count (info);
      for (i = 0; i < count; i++)
	{
	  struct ipcp_lattice *lat = ipa_get_lattice (info, i);
	  struct ipcp_value *val;
	  bool prev = false;

	  fprintf (f, "    param [%d]: ", i);
	  if (lat->bottom)
	    {
	      fprintf (f, "BOTTOM\n");
	      continue;
	    }

	  if (!lat->values_count && !lat->contains_variable)
	    {
	      fprintf (f, "TOP\n");
	      continue;
	    }

	  if (lat->contains_variable)
	    {
	      fprintf (f, "VARIABLE");
	      prev = true;
	      if (dump_benefits)
		fprintf (f, "\n");
	    }

	  for (val = lat->values; val; val = val->next)
	    {
	      if (dump_benefits && prev)
		fprintf (f, "               ");
	      else if (!dump_benefits && prev)
		fprintf (f, ", ");
	      else
		prev = true;

	      print_ipcp_constant_value (f, val->value);

	      if (dump_sources)
		{
		  struct ipcp_value_source *s;

		  fprintf (f, " [from:");
		  for (s = val->sources; s; s = s->next)
		    fprintf (f, " %i(%i)", s->cs->caller->uid,s->cs->frequency);
		  fprintf (f, "]");
		}

	      if (dump_benefits)
		fprintf (f, " [loc_time: %i, loc_size: %i, "
			 "prop_time: %i, prop_size: %i]\n",
			 val->local_time_benefit, val->local_size_cost,
			 val->prop_time_benefit, val->prop_size_cost);
	    }
	  if (!dump_benefits)
	    fprintf (f, "\n");
	}
    }
}

/* Determine whether it is at all technically possible to create clones of NODE
   and store this information in the ipa_node_params structure associated
   with NODE.  */

static void
determine_versionability (struct cgraph_node *node)
{
  struct cgraph_edge *edge;
  bool res = true;

  /* There are a number of generic reasons functions cannot be versioned.  We
     also cannot remove parameters if there are type attributes such as fnspec
     present.  */
  if (node->alias || node->thunk.thunk_p
      || !inline_summary (node)->versionable
      || TYPE_ATTRIBUTES (TREE_TYPE (node->decl))
      || cgraph_function_body_availability (node) <= AVAIL_OVERWRITABLE)
    res =  false;
  else
    /* Removing arguments doesn't work if the function takes varargs
       or use __builtin_apply_args.
       FIXME: handle this together with can_change_signature flag.  */
    for (edge = node->callees; edge; edge = edge->next_callee)
      {
	tree t = edge->callee->decl;
	if (DECL_BUILT_IN_CLASS (t) == BUILT_IN_NORMAL
	    && (DECL_FUNCTION_CODE (t) == BUILT_IN_APPLY_ARGS
		|| DECL_FUNCTION_CODE (t) == BUILT_IN_VA_START))
	  {
	    res = false;
	    break;
	  };
      }

  if (!res && dump_file && node->alias && node->thunk.thunk_p)
    fprintf (dump_file, "Function %s/%i is not versionable.\n",
	     cgraph_node_name (node), node->uid);

  IPA_NODE_REF (node)->node_versionable = res;
}

/* Return true if it is at all technically possible to create clones of a
   NODE.  */

static bool
ipcp_versionable_function_p (struct cgraph_node *node)
{
  return IPA_NODE_REF (node)->node_versionable;
}

/* Structure holding accumulated information about callers of a node.  */

struct caller_statistics
{
  gcov_type count_sum;
  int n_calls, n_hot_calls, freq_sum;
};

/* Initialize fields of STAT to zeroes.  */

static inline void
init_caller_stats (struct caller_statistics *stats)
{
  stats->count_sum = 0;
  stats->n_calls = 0;
  stats->n_hot_calls = 0;
  stats->freq_sum = 0;
}

/* Worker callback of cgraph_for_node_and_aliases accumulating statistics of
   non-thunk incoming edges to NODE.  */

static bool
gather_caller_stats (struct cgraph_node *node, void *data)
{
  struct caller_statistics *stats = (struct caller_statistics *) data;
  struct cgraph_edge *cs;

  for (cs = node->callers; cs; cs = cs->next_caller)
    if (cs->caller->thunk.thunk_p)
      gather_caller_stats (cs->caller, stats);
    else
      {
	stats->count_sum += cs->count;
	stats->freq_sum += cs->frequency;
	stats->n_calls++;
	if (cgraph_maybe_hot_edge_p (cs))
	  stats->n_hot_calls ++;
      }
  return false;

}

/* Return true if this NODE is viable candidate for cloning.  */

static bool
ipcp_cloning_candidate_p (struct cgraph_node *node)
{
  struct caller_statistics stats;

  if (!cgraph_function_with_gimple_body_p (node))
    return false;

  if (!flag_ipa_cp_clone)
    {
      if (dump_file)
        fprintf (dump_file, "Not considering %s for cloning; "
		 "-fipa-cp-clone disabled.\n",
 	         cgraph_node_name (node));
      return false;
    }

  if (!optimize_function_for_speed_p (DECL_STRUCT_FUNCTION (node->decl)))
    {
      if (dump_file)
        fprintf (dump_file, "Not considering %s for cloning; "
		 "optimizing it for size.\n",
 	         cgraph_node_name (node));
      return false;
    }

  init_caller_stats (&stats);
  cgraph_for_node_and_aliases (node, gather_caller_stats, &stats, false);

  if (inline_summary (node)->self_size < stats.n_calls)
    {
      if (dump_file)
        fprintf (dump_file, "Considering %s for cloning; code might shrink.\n",
 	         cgraph_node_name (node));
      return true;
    }

  /* When profile is available and function is hot, propagate into it even if
     calls seems cold; constant propagation can improve function's speed
     significantly.  */
  if (max_count)
    {
      if (stats.count_sum > node->count * 90 / 100)
	{
	  if (dump_file)
	    fprintf (dump_file, "Considering %s for cloning; "
		     "usually called directly.\n",
		     cgraph_node_name (node));
	  return true;
        }
    }
  if (!stats.n_hot_calls)
    {
      if (dump_file)
	fprintf (dump_file, "Not considering %s for cloning; no hot calls.\n",
		 cgraph_node_name (node));
      return false;
    }
  if (dump_file)
    fprintf (dump_file, "Considering %s for cloning.\n",
	     cgraph_node_name (node));
  return true;
}

/* Arrays representing a topological ordering of call graph nodes and a stack
   of noes used during constant propagation.  */

struct topo_info
{
  struct cgraph_node **order;
  struct cgraph_node **stack;
  int nnodes, stack_top;
};

/* Allocate the arrays in TOPO and topologically sort the nodes into order.  */

static void
build_topo_info (struct topo_info *topo)
{
  topo->order = XCNEWVEC (struct cgraph_node *, cgraph_n_nodes);
  topo->stack = XCNEWVEC (struct cgraph_node *, cgraph_n_nodes);
  topo->stack_top = 0;
  topo->nnodes = ipa_reduced_postorder (topo->order, true, true, NULL);
}

/* Free information about strongly connected components and the arrays in
   TOPO.  */

static void
free_topo_info (struct topo_info *topo)
{
  ipa_free_postorder_info ();
  free (topo->order);
  free (topo->stack);
}

/* Add NODE to the stack in TOPO, unless it is already there.  */

static inline void
push_node_to_stack (struct topo_info *topo, struct cgraph_node *node)
{
  struct ipa_node_params *info = IPA_NODE_REF (node);
  if (info->node_enqueued)
    return;
  info->node_enqueued = 1;
  topo->stack[topo->stack_top++] = node;
}

/* Pop a node from the stack in TOPO and return it or return NULL if the stack
   is empty.  */

static struct cgraph_node *
pop_node_from_stack (struct topo_info *topo)
{
  if (topo->stack_top)
    {
      struct cgraph_node *node;
      topo->stack_top--;
      node = topo->stack[topo->stack_top];
      IPA_NODE_REF (node)->node_enqueued = 0;
      return node;
    }
  else
    return NULL;
}

/* Set lattice LAT to bottom and return true if it previously was not set as
   such.  */

static inline bool
set_lattice_to_bottom (struct ipcp_lattice *lat)
{
  bool ret = !lat->bottom;
  lat->bottom = true;
  return ret;
}

/* Mark lattice as containing an unknown value and return true if it previously
   was not marked as such.  */

static inline bool
set_lattice_contains_variable (struct ipcp_lattice *lat)
{
  bool ret = !lat->contains_variable;
  lat->contains_variable = true;
  return ret;
}

/* Initialize ipcp_lattices.  */

static void
initialize_node_lattices (struct cgraph_node *node)
{
  struct ipa_node_params *info = IPA_NODE_REF (node);
  struct cgraph_edge *ie;
  bool disable = false, variable = false;
  int i;

  /* FIXME: Can we clobber only the first argument of thunks?  */
  if (node->alias || node->thunk.thunk_p
      || ipa_is_called_with_var_arguments (info))
    disable = true;
  else if (!node->local.local)
    {
      /* When cloning is allowed, we can assume that externally visible
	 functions are not called.  We will compensate this by cloning
	 later.  */
      if (ipcp_versionable_function_p (node)
	  && ipcp_cloning_candidate_p (node))
	variable = true;
      else
	disable = true;
    }

  if (disable || variable)
    {
      for (i = 0; i < ipa_get_param_count (info) ; i++)
	{
	  struct ipcp_lattice *lat = ipa_get_lattice (info, i);
	  if (disable)
	    set_lattice_to_bottom (lat);
	  else
	    set_lattice_contains_variable (lat);
	}
      if (dump_file && (dump_flags & TDF_DETAILS)
	  && node->alias && node->thunk.thunk_p)
	fprintf (dump_file, "Marking all lattices of %s/%i as %s\n",
		 cgraph_node_name (node), node->uid,
		 disable ? "BOTTOM" : "VARIABLE");
    }

  for (ie = node->indirect_calls; ie; ie = ie->next_callee)
    if (ie->indirect_info->polymorphic)
      {
	gcc_checking_assert (ie->indirect_info->param_index >= 0);
	ipa_get_lattice (info, ie->indirect_info->param_index)->virt_call = 1;
      }
}

/* If checking is enabled, verify that no lattice is in the TOP state, i.e. not
   bottom, not containing a variable component and without any known value at
   the same time.  */

static void
verify_propagated_values (void)
{
#ifdef ENABLE_CHECKING
  struct cgraph_node *node;

  FOR_EACH_DEFINED_FUNCTION (node)
    {
      struct ipa_node_params *info = IPA_NODE_REF (node);
      int i, count = ipa_get_param_count (info);

      for (i = 0; i < count; i++)
	{
	  struct ipcp_lattice *lat = ipa_get_lattice (info, i);

	  if (!lat->bottom
	      && !lat->contains_variable
	      && lat->values_count == 0)
	    {
	      if (dump_file)
		{
		  fprintf (dump_file, "\nIPA lattices after constant "
			   "propagation:\n");
		  print_all_lattices (dump_file, true, false);
		}

	      gcc_unreachable ();
	    }
	}
    }
#endif
}

/* Return true iff X and Y should be considered equal values by IPA-CP.  */

static bool
values_equal_for_ipcp_p (tree x, tree y)
{
  gcc_checking_assert (x != NULL_TREE && y != NULL_TREE);

  if (x == y)
    return true;

  if (TREE_CODE (x) == TREE_BINFO || TREE_CODE (y) == TREE_BINFO)
    return false;

  if (TREE_CODE (x) ==  ADDR_EXPR
      && TREE_CODE (y) ==  ADDR_EXPR
      && TREE_CODE (TREE_OPERAND (x, 0)) == CONST_DECL
      && TREE_CODE (TREE_OPERAND (y, 0)) == CONST_DECL)
    return operand_equal_p (DECL_INITIAL (TREE_OPERAND (x, 0)),
			    DECL_INITIAL (TREE_OPERAND (y, 0)), 0);
  else
    return operand_equal_p (x, y, 0);
}

/* Add a new value source to VAL, marking that a value comes from edge CS and
   (if the underlying jump function is a pass-through or an ancestor one) from
   a caller value SRC_VAL of a caller parameter described by SRC_INDEX.  */

static void
add_value_source (struct ipcp_value *val, struct cgraph_edge *cs,
		  struct ipcp_value *src_val, int src_idx)
{
  struct ipcp_value_source *src;

  src = (struct ipcp_value_source *) pool_alloc (ipcp_sources_pool);
  src->cs = cs;
  src->val = src_val;
  src->index = src_idx;

  src->next = val->sources;
  val->sources = src;
}


/* Try to add NEWVAL to LAT, potentially creating a new struct ipcp_value for
   it.  CS, SRC_VAL and SRC_INDEX are meant for add_value_source and have the
   same meaning.  */

static bool
add_value_to_lattice (struct ipcp_lattice *lat, tree newval,
		      struct cgraph_edge *cs, struct ipcp_value *src_val,
		      int src_idx)
{
  struct ipcp_value *val;

  if (lat->bottom)
    return false;


  for (val = lat->values; val; val = val->next)
    if (values_equal_for_ipcp_p (val->value, newval))
      {
	if (edge_within_scc (cs))
	  {
	    struct ipcp_value_source *s;
	    for (s = val->sources; s ; s = s->next)
	      if (s->cs == cs)
		break;
	    if (s)
	      return false;
	  }

	add_value_source (val, cs, src_val, src_idx);
	return false;
      }

  if (lat->values_count == PARAM_VALUE (PARAM_IPA_CP_VALUE_LIST_SIZE))
    {
      /* We can only free sources, not the values themselves, because sources
	 of other values in this this SCC might point to them.   */
      for (val = lat->values; val; val = val->next)
	{
	  while (val->sources)
	    {
	      struct ipcp_value_source *src = val->sources;
	      val->sources = src->next;
	      pool_free (ipcp_sources_pool, src);
	    }
	}

      lat->values = NULL;
      return set_lattice_to_bottom (lat);
    }

  lat->values_count++;
  val = (struct ipcp_value *) pool_alloc (ipcp_values_pool);
  memset (val, 0, sizeof (*val));

  add_value_source (val, cs, src_val, src_idx);
  val->value = newval;
  val->next = lat->values;
  lat->values = val;
  return true;
}

/* Propagate values through a pass-through jump function JFUNC associated with
   edge CS, taking values from SRC_LAT and putting them into DEST_LAT.  SRC_IDX
   is the index of the source parameter.  */

static bool
propagate_vals_accross_pass_through (struct cgraph_edge *cs,
				     struct ipa_jump_func *jfunc,
				     struct ipcp_lattice *src_lat,
				     struct ipcp_lattice *dest_lat,
				     int src_idx)
{
  struct ipcp_value *src_val;
  bool ret = false;

  if (jfunc->value.pass_through.operation == NOP_EXPR)
    for (src_val = src_lat->values; src_val; src_val = src_val->next)
      ret |= add_value_to_lattice (dest_lat, src_val->value, cs,
				   src_val, src_idx);
  else if (edge_within_scc (cs))
    ret = set_lattice_contains_variable (dest_lat);
  else
    for (src_val = src_lat->values; src_val; src_val = src_val->next)
      {
	tree cstval = src_val->value;

	if (TREE_CODE (cstval) == TREE_BINFO)
	  {
	    ret |= set_lattice_contains_variable (dest_lat);
	    continue;
	  }
	cstval = ipa_get_jf_pass_through_result (jfunc, cstval);

	if (cstval)
	  ret |= add_value_to_lattice (dest_lat, cstval, cs, src_val, src_idx);
	else
	  ret |= set_lattice_contains_variable (dest_lat);
      }

  return ret;
}

/* Propagate values through an ancestor jump function JFUNC associated with
   edge CS, taking values from SRC_LAT and putting them into DEST_LAT.  SRC_IDX
   is the index of the source parameter.  */

static bool
propagate_vals_accross_ancestor (struct cgraph_edge *cs,
				 struct ipa_jump_func *jfunc,
				 struct ipcp_lattice *src_lat,
				 struct ipcp_lattice *dest_lat,
				 int src_idx)
{
  struct ipcp_value *src_val;
  bool ret = false;

  if (edge_within_scc (cs))
    return set_lattice_contains_variable (dest_lat);

  for (src_val = src_lat->values; src_val; src_val = src_val->next)
    {
      tree t = src_val->value;

      if (TREE_CODE (t) == TREE_BINFO)
	t = get_binfo_at_offset (t, jfunc->value.ancestor.offset,
				 jfunc->value.ancestor.type);
      else
	t = ipa_get_jf_ancestor_result (jfunc, t);

      if (t)
	ret |= add_value_to_lattice (dest_lat, t, cs, src_val, src_idx);
      else
	ret |= set_lattice_contains_variable (dest_lat);
    }

  return ret;
}

/* Propagate values across jump function JFUNC that is associated with edge CS
   and put the values into DEST_LAT.  */

static bool
propagate_accross_jump_function (struct cgraph_edge *cs,
				 struct ipa_jump_func *jfunc,
				 struct ipcp_lattice *dest_lat)
{
  if (dest_lat->bottom)
    return false;

  if (jfunc->type == IPA_JF_CONST
      || jfunc->type == IPA_JF_KNOWN_TYPE)
    {
      tree val;

      if (jfunc->type == IPA_JF_KNOWN_TYPE)
	val = jfunc->value.base_binfo;
      else
	val = jfunc->value.constant;
      return add_value_to_lattice (dest_lat, val, cs, NULL, 0);
    }
  else if (jfunc->type == IPA_JF_PASS_THROUGH
	   || jfunc->type == IPA_JF_ANCESTOR)
    {
      struct ipa_node_params *caller_info = IPA_NODE_REF (cs->caller);
      struct ipcp_lattice *src_lat;
      int src_idx;
      bool ret;

      if (jfunc->type == IPA_JF_PASS_THROUGH)
	src_idx = jfunc->value.pass_through.formal_id;
      else
	src_idx = jfunc->value.ancestor.formal_id;

      src_lat = ipa_get_lattice (caller_info, src_idx);
      if (src_lat->bottom)
	return set_lattice_contains_variable (dest_lat);

      /* If we would need to clone the caller and cannot, do not propagate.  */
      if (!ipcp_versionable_function_p (cs->caller)
	  && (src_lat->contains_variable
	      || (src_lat->values_count > 1)))
	return set_lattice_contains_variable (dest_lat);

      if (jfunc->type == IPA_JF_PASS_THROUGH)
	ret = propagate_vals_accross_pass_through (cs, jfunc, src_lat,
						   dest_lat, src_idx);
      else
	ret = propagate_vals_accross_ancestor (cs, jfunc, src_lat, dest_lat,
					       src_idx);

      if (src_lat->contains_variable)
	ret |= set_lattice_contains_variable (dest_lat);

      return ret;
    }

  /* TODO: We currently do not handle member method pointers in IPA-CP (we only
     use it for indirect inlining), we should propagate them too.  */
  return set_lattice_contains_variable (dest_lat);
}

/* Propagate constants from the caller to the callee of CS.  INFO describes the
   caller.  */

static bool
propagate_constants_accross_call (struct cgraph_edge *cs)
{
  struct ipa_node_params *callee_info;
  enum availability availability;
  struct cgraph_node *callee;
  struct ipa_edge_args *args;
  bool ret = false;
  int i, count;

  callee = cgraph_function_or_thunk_node (cs->callee, &availability);
  if (!callee || !cgraph_function_with_gimple_body_p (callee))
    return false;

  callee_info = IPA_NODE_REF (callee);
  if (ipa_is_called_with_var_arguments (callee_info))
    return false;

  args = IPA_EDGE_REF (cs);
  count = ipa_get_cs_argument_count (args);
  for (i = 0; i < count; i++)
    {
      struct ipa_jump_func *jump_func = ipa_get_ith_jump_func (args, i);
      struct ipcp_lattice *dest_lat = ipa_get_lattice (callee_info, i);

      if (availability == AVAIL_OVERWRITABLE)
	ret |= set_lattice_contains_variable (dest_lat);
      else
	ret |= propagate_accross_jump_function (cs, jump_func, dest_lat);
    }
  return ret;
}

/* If an indirect edge IE can be turned into a direct one based on KNOWN_VALS
   (which can contain both constants and binfos) or KNOWN_BINFOS (which can be
   NULL) return the destination.  If simple thunk delta must be applied too,
   store it to DELTA.  */

static tree
get_indirect_edge_target (struct cgraph_edge *ie, tree *delta,
			  VEC (tree, heap) *known_vals,
			  VEC (tree, heap) *known_binfos)
{
  int param_index = ie->indirect_info->param_index;
  HOST_WIDE_INT token, anc_offset;
  tree otr_type;
  tree t;

  if (param_index == -1)
    return NULL_TREE;

  if (!ie->indirect_info->polymorphic)
    {
      tree t = VEC_index (tree, known_vals, param_index);
      if (t &&
	  TREE_CODE (t) == ADDR_EXPR
	  && TREE_CODE (TREE_OPERAND (t, 0)) == FUNCTION_DECL)
	{
	  *delta = NULL_TREE;
	  return TREE_OPERAND (t, 0);
	}
      else
	return NULL_TREE;
    }

  token = ie->indirect_info->otr_token;
  anc_offset = ie->indirect_info->anc_offset;
  otr_type = ie->indirect_info->otr_type;

  t = VEC_index (tree, known_vals, param_index);
  if (!t && known_binfos)
    t = VEC_index (tree, known_binfos, param_index);
  if (!t)
    return NULL_TREE;

  if (TREE_CODE (t) != TREE_BINFO)
    {
      tree binfo;
      binfo = gimple_extract_devirt_binfo_from_cst (t);
      if (!binfo)
	return NULL_TREE;
      binfo = get_binfo_at_offset (binfo, anc_offset, otr_type);
      if (!binfo)
	return NULL_TREE;
      return gimple_get_virt_method_for_binfo (token, binfo, delta);
    }
  else
    {
      tree binfo;

      binfo = get_binfo_at_offset (t, anc_offset, otr_type);
      if (!binfo)
	return NULL_TREE;
      return gimple_get_virt_method_for_binfo (token, binfo, delta);
    }
}

/* Calculate devirtualization time bonus for NODE, assuming we know KNOWN_CSTS
   and KNOWN_BINFOS.  */

static int
devirtualization_time_bonus (struct cgraph_node *node,
			     VEC (tree, heap) *known_csts,
			     VEC (tree, heap) *known_binfos)
{
  struct cgraph_edge *ie;
  int res = 0;

  for (ie = node->indirect_calls; ie; ie = ie->next_callee)
    {
      struct cgraph_node *callee;
      struct inline_summary *isummary;
      tree delta, target;

      target = get_indirect_edge_target (ie, &delta, known_csts, known_binfos);
      if (!target)
	continue;

      /* Only bare minimum benefit for clearly un-inlineable targets.  */
      res += 1;
      callee = cgraph_get_node (target);
      if (!callee)
	continue;

      /* FIXME: The values below need re-considering and perhaps also
	 integrating into the cost metrics, at lest in some very basic way.  */
      isummary = inline_summary (callee);
      if (isummary->size <= MAX_INLINE_INSNS_AUTO / 4)
	res += 31;
      else if (isummary->size <= MAX_INLINE_INSNS_AUTO / 2)
	res += 15;
      else if (isummary->size <= MAX_INLINE_INSNS_AUTO
	       || DECL_DECLARED_INLINE_P (callee->decl))
	res += 7;
    }

  return res;
}

/* Return true if cloning NODE is a good idea, given the estimated TIME_BENEFIT
   and SIZE_COST and with the sum of frequencies of incoming edges to the
   potential new clone in FREQUENCIES.  */

static bool
good_cloning_opportunity_p (struct cgraph_node *node, int time_benefit,
			    int freq_sum, gcov_type count_sum, int size_cost)
{
  if (time_benefit == 0
      || !flag_ipa_cp_clone
      || !optimize_function_for_speed_p (DECL_STRUCT_FUNCTION (node->decl)))
    return false;

  gcc_checking_assert (size_cost >= 0);

  /* FIXME:  These decisions need tuning.  */
  if (max_count)
    {
      int evaluation, factor = (count_sum * 1000) / max_count;

      evaluation = (time_benefit * factor) / size_cost;

      if (dump_file && (dump_flags & TDF_DETAILS))
	fprintf (dump_file, "     good_cloning_opportunity_p (time: %i, "
		 "size: %i, count_sum: " HOST_WIDE_INT_PRINT_DEC
		 ") -> evaluation: %i, threshold: %i\n",
		 time_benefit, size_cost, (HOST_WIDE_INT) count_sum,
		 evaluation, 500);

      return evaluation > 500;
    }
  else
    {
      int evaluation = (time_benefit * freq_sum) / size_cost;

      if (dump_file && (dump_flags & TDF_DETAILS))
	fprintf (dump_file, "     good_cloning_opportunity_p (time: %i, "
		 "size: %i, freq_sum: %i) -> evaluation: %i, threshold: %i\n",
		 time_benefit, size_cost, freq_sum, evaluation,
		 CGRAPH_FREQ_BASE /2);

      return evaluation > (CGRAPH_FREQ_BASE /2);
    }
}


/* Allocate KNOWN_CSTS and KNOWN_BINFOS and populate them with values of
   parameters that are known independent of the context.  INFO describes the
   function.  If REMOVABLE_PARAMS is non-NULL, store the number o removable
   parameters in it.  */

static bool
gather_context_independent_values (struct ipa_node_params *info,
				   VEC (tree, heap) **known_csts,
				   VEC (tree, heap) **known_binfos,
				   int *removable_params)
{
  int i, count = ipa_get_param_count (info);
  int rparams = 0;
  bool ret = false;

  *known_csts = NULL;
  *known_binfos = NULL;
  VEC_safe_grow_cleared (tree, heap, *known_csts, count);
  VEC_safe_grow_cleared (tree, heap, *known_binfos, count);

  for (i = 0; i < count ; i++)
    {
      struct ipcp_lattice *lat = ipa_get_lattice (info, i);

      if (ipa_lat_is_single_const (lat))
	{
	  struct ipcp_value *val = lat->values;
	  if (TREE_CODE (val->value) != TREE_BINFO)
	    {
	      VEC_replace (tree, *known_csts, i, val->value);
	      rparams++;
	      ret = true;
	    }
	  else if (lat->virt_call)
	    {
	      VEC_replace (tree, *known_binfos, i, val->value);
	      ret = true;
	    }
	  else if (!ipa_is_param_used (info, i))
	    rparams++;
	}
      else if (!ipa_is_param_used (info, i))
	rparams++;
    }

  if (removable_params)
    *removable_params = rparams;
  return ret;
}

/* Iterate over known values of parameters of NODE and estimate the local
   effects in terms of time and size they have.  */

static void
estimate_local_effects (struct cgraph_node *node)
{
  struct ipa_node_params *info = IPA_NODE_REF (node);
  int i, count = ipa_get_param_count (info);
  VEC (tree, heap) *known_csts, *known_binfos;
  bool always_const;
  int base_time = inline_summary (node)->time;
  int removable_params;

  if (!count || !ipcp_versionable_function_p (node))
    return;

  if (dump_file && (dump_flags & TDF_DETAILS))
    fprintf (dump_file, "\nEstimating effects for %s/%i, base_time: %i.\n",
	     cgraph_node_name (node), node->uid, base_time);

  always_const = gather_context_independent_values (info, &known_csts,
						    &known_binfos,
						    &removable_params);
  if (always_const)
    {
      struct caller_statistics stats;
      int time, size;

      init_caller_stats (&stats);
      cgraph_for_node_and_aliases (node, gather_caller_stats, &stats, false);
      estimate_ipcp_clone_size_and_time (node, known_csts, &size, &time);
      time -= devirtualization_time_bonus (node, known_csts, known_binfos);
      size -= stats.n_calls * removable_params;

      if (dump_file)
	fprintf (dump_file, " * context independent values, size: %i, "
		 "time_benefit: %i\n", size, base_time - time);

      if (size <= 0
	  || cgraph_will_be_removed_from_program_if_no_direct_calls (node))
	{
	  info->clone_for_all_contexts = true;
	  base_time = time;

	  if (dump_file)
	    fprintf (dump_file, "     Decided to specialize for all "
		     "known contexts, code not going to grow.\n");
	}
      else if (good_cloning_opportunity_p (node, base_time - time,
					   stats.freq_sum, stats.count_sum,
					   size))
	{
	  if (size + overall_size <= max_new_size)
	    {
	      info->clone_for_all_contexts = true;
	      base_time = time;
	      overall_size += size;

	      if (dump_file)
		fprintf (dump_file, "     Decided to specialize for all "
			 "known contexts, growth deemed beneficial.\n");
	    }
	  else if (dump_file && (dump_flags & TDF_DETAILS))
	    fprintf (dump_file, "   Not cloning for all contexts because "
		     "max_new_size would be reached with %li.\n",
		     size + overall_size);
	}
    }

  for (i = 0; i < count ; i++)
    {
      struct ipcp_lattice *lat = ipa_get_lattice (info, i);
      struct ipcp_value *val;

      if (lat->bottom
	  || !lat->values
	  || VEC_index (tree, known_csts, i)
	  || VEC_index (tree, known_binfos, i))
	continue;

      for (val = lat->values; val; val = val->next)
	{
	  int time, size, time_benefit;

	  if (TREE_CODE (val->value) != TREE_BINFO)
	    {
	      VEC_replace (tree, known_csts, i, val->value);
	      VEC_replace (tree, known_binfos, i, NULL_TREE);
	    }
	  else if (lat->virt_call)
	    {
	      VEC_replace (tree, known_csts, i, NULL_TREE);
	      VEC_replace (tree, known_binfos, i, val->value);
	    }
	  else
	    continue;

	  estimate_ipcp_clone_size_and_time (node, known_csts, &size, &time);
	  time_benefit = base_time - time
	    + devirtualization_time_bonus (node, known_csts, known_binfos);

	  if (dump_file && (dump_flags & TDF_DETAILS))
	    {
	      fprintf (dump_file, " - estimates for value ");
	      print_ipcp_constant_value (dump_file, val->value);
	      fprintf (dump_file, " for parameter ");
	      print_generic_expr (dump_file, ipa_get_param (info, i), 0);
	      fprintf (dump_file, ": time_benefit: %i, size: %i\n",
		       time_benefit, size);
	    }

	  val->local_time_benefit = time_benefit;
	  val->local_size_cost = size;
	}
    }

  VEC_free (tree, heap, known_csts);
  VEC_free (tree, heap, known_binfos);
}


/* Add value CUR_VAL and all yet-unsorted values it is dependent on to the
   topological sort of values.  */

static void
add_val_to_toposort (struct ipcp_value *cur_val)
{
  static int dfs_counter = 0;
  static struct ipcp_value *stack;
  struct ipcp_value_source *src;

  if (cur_val->dfs)
    return;

  dfs_counter++;
  cur_val->dfs = dfs_counter;
  cur_val->low_link = dfs_counter;

  cur_val->topo_next = stack;
  stack = cur_val;
  cur_val->on_stack = true;

  for (src = cur_val->sources; src; src = src->next)
    if (src->val)
      {
	if (src->val->dfs == 0)
	  {
	    add_val_to_toposort (src->val);
	    if (src->val->low_link < cur_val->low_link)
	      cur_val->low_link = src->val->low_link;
	  }
	else if (src->val->on_stack
		 && src->val->dfs < cur_val->low_link)
	  cur_val->low_link = src->val->dfs;
      }

  if (cur_val->dfs == cur_val->low_link)
    {
      struct ipcp_value *v, *scc_list = NULL;

      do
	{
	  v = stack;
	  stack = v->topo_next;
	  v->on_stack = false;

	  v->scc_next = scc_list;
	  scc_list = v;
	}
      while (v != cur_val);

      cur_val->topo_next = values_topo;
      values_topo = cur_val;
    }
}

/* Add all values in lattices associated with NODE to the topological sort if
   they are not there yet.  */

static void
add_all_node_vals_to_toposort (struct cgraph_node *node)
{
  struct ipa_node_params *info = IPA_NODE_REF (node);
  int i, count = ipa_get_param_count (info);

  for (i = 0; i < count ; i++)
    {
      struct ipcp_lattice *lat = ipa_get_lattice (info, i);
      struct ipcp_value *val;

      if (lat->bottom || !lat->values)
	continue;
      for (val = lat->values; val; val = val->next)
	add_val_to_toposort (val);
    }
}

/* One pass of constants propagation along the call graph edges, from callers
   to callees (requires topological ordering in TOPO), iterate over strongly
   connected components.  */

static void
propagate_constants_topo (struct topo_info *topo)
{
  int i;

  for (i = topo->nnodes - 1; i >= 0; i--)
    {
      struct cgraph_node *v, *node = topo->order[i];
      struct ipa_dfs_info *node_dfs_info;

      if (!cgraph_function_with_gimple_body_p (node))
	continue;

      node_dfs_info = (struct ipa_dfs_info *) node->aux;
      /* First, iteratively propagate within the strongly connected component
	 until all lattices stabilize.  */
      v = node_dfs_info->next_cycle;
      while (v)
	{
	  push_node_to_stack (topo, v);
	  v = ((struct ipa_dfs_info *) v->aux)->next_cycle;
	}

      v = node;
      while (v)
	{
	  struct cgraph_edge *cs;

	  for (cs = v->callees; cs; cs = cs->next_callee)
	    if (edge_within_scc (cs)
		&& propagate_constants_accross_call (cs))
	      push_node_to_stack (topo, cs->callee);
	  v = pop_node_from_stack (topo);
	}

      /* Afterwards, propagate along edges leading out of the SCC, calculates
	 the local effects of the discovered constants and all valid values to
	 their topological sort.  */
      v = node;
      while (v)
	{
	  struct cgraph_edge *cs;

	  estimate_local_effects (v);
	  add_all_node_vals_to_toposort (v);
	  for (cs = v->callees; cs; cs = cs->next_callee)
	    if (!edge_within_scc (cs))
	      propagate_constants_accross_call (cs);

	  v = ((struct ipa_dfs_info *) v->aux)->next_cycle;
	}
    }
}

/* Propagate the estimated effects of individual values along the topological
   from the dependant values to those they depend on.  */

static void
propagate_effects (void)
{
  struct ipcp_value *base;

  for (base = values_topo; base; base = base->topo_next)
    {
      struct ipcp_value_source *src;
      struct ipcp_value *val;
      int time = 0, size = 0;

      for (val = base; val; val = val->scc_next)
	{
	  time += val->local_time_benefit + val->prop_time_benefit;
	  size += val->local_size_cost + val->prop_size_cost;
	}

      for (val = base; val; val = val->scc_next)
	for (src = val->sources; src; src = src->next)
	  if (src->val
	      && cgraph_maybe_hot_edge_p (src->cs))
	    {
	      src->val->prop_time_benefit += time;
	      src->val->prop_size_cost += size;
	    }
    }
}


/* Propagate constants, binfos and their effects from the summaries
   interprocedurally.  */

static void
ipcp_propagate_stage (struct topo_info *topo)
{
  struct cgraph_node *node;

  if (dump_file)
    fprintf (dump_file, "\n Propagating constants:\n\n");

  if (in_lto_p)
    ipa_update_after_lto_read ();

  FOR_EACH_DEFINED_FUNCTION (node)
  {
    determine_versionability (node);
    initialize_node_lattices (node);
    if (node->count > max_count)
      max_count = node->count;
    overall_size += inline_summary (node)->self_size;
  }

  max_new_size = overall_size;
  if (max_new_size < PARAM_VALUE (PARAM_LARGE_UNIT_INSNS))
    max_new_size = PARAM_VALUE (PARAM_LARGE_UNIT_INSNS);
  max_new_size += max_new_size * PARAM_VALUE (PARAM_IPCP_UNIT_GROWTH) / 100 + 1;

  if (dump_file)
    fprintf (dump_file, "\noverall_size: %li, max_new_size: %li\n",
	     overall_size, max_new_size);

  propagate_constants_topo (topo);
  verify_propagated_values ();
  propagate_effects ();

  if (dump_file)
    {
      fprintf (dump_file, "\nIPA lattices after all propagation:\n");
      print_all_lattices (dump_file, (dump_flags & TDF_DETAILS), true);
    }
}

/* Discover newly direct outgoing edges from NODE which is a new clone with
   known KNOWN_VALS and make them direct.  */

static void
ipcp_discover_new_direct_edges (struct cgraph_node *node,
				VEC (tree, heap) *known_vals)
{
  struct cgraph_edge *ie, *next_ie;

  for (ie = node->indirect_calls; ie; ie = next_ie)
    {
      tree delta, target;

      next_ie = ie->next_callee;
      target = get_indirect_edge_target (ie, &delta, known_vals, NULL);
      if (target)
	ipa_make_edge_direct_to_target (ie, target, delta);
    }
}

/* Return true if edge CS does bring about the value described by SRC.  */

static bool
cgraph_edge_brings_value_p (struct cgraph_edge *cs,
			    struct ipcp_value_source *src)
{
  struct ipa_node_params *caller_info = IPA_NODE_REF (cs->caller);

  if (IPA_NODE_REF (cs->callee)->ipcp_orig_node
      || caller_info->node_dead)
    return false;
  if (!src->val)
    return true;

  if (caller_info->ipcp_orig_node)
    {
      tree t = VEC_index (tree, caller_info->known_vals, src->index);
      return (t != NULL_TREE
	      && values_equal_for_ipcp_p (src->val->value, t));
    }
  else
    {
      struct ipcp_lattice *lat = ipa_get_lattice (caller_info, src->index);
      if (ipa_lat_is_single_const (lat)
	  && values_equal_for_ipcp_p (src->val->value, lat->values->value))
	return true;
      else
	return false;
    }
}

/* Get the next clone in the linked list of clones of an edge.  */

static inline struct cgraph_edge *
get_next_cgraph_edge_clone (struct cgraph_edge *cs)
{
  struct ipa_edge_args *args = IPA_EDGE_REF (cs);
  return args->next_edge_clone;
}

/* Given VAL, iterate over all its sources and if they still hold, add their
   edge frequency and their number into *FREQUENCY and *CALLER_COUNT
   respectively.  */

static bool
get_info_about_necessary_edges (struct ipcp_value *val, int *freq_sum,
				gcov_type *count_sum, int *caller_count)
{
  struct ipcp_value_source *src;
  int freq = 0, count = 0;
  gcov_type cnt = 0;
  bool hot = false;

  for (src = val->sources; src; src = src->next)
    {
      struct cgraph_edge *cs = src->cs;
      while (cs)
	{
	  if (cgraph_edge_brings_value_p (cs, src))
	    {
	      count++;
	      freq += cs->frequency;
	      cnt += cs->count;
	      hot |= cgraph_maybe_hot_edge_p (cs);
	    }
	  cs = get_next_cgraph_edge_clone (cs);
	}
    }

  *freq_sum = freq;
  *count_sum = cnt;
  *caller_count = count;
  return hot;
}

/* Return a vector of incoming edges that do bring value VAL.  It is assumed
   their number is known and equal to CALLER_COUNT.  */

static VEC (cgraph_edge_p,heap) *
gather_edges_for_value (struct ipcp_value *val, int caller_count)
{
  struct ipcp_value_source *src;
  VEC (cgraph_edge_p,heap) *ret;

  ret = VEC_alloc (cgraph_edge_p, heap, caller_count);
  for (src = val->sources; src; src = src->next)
    {
      struct cgraph_edge *cs = src->cs;
      while (cs)
	{
	  if (cgraph_edge_brings_value_p (cs, src))
	    VEC_quick_push (cgraph_edge_p, ret, cs);
	  cs = get_next_cgraph_edge_clone (cs);
	}
    }

  return ret;
}

/* Construct a replacement map for a know VALUE for a formal parameter PARAM.
   Return it or NULL if for some reason it cannot be created.  */

static struct ipa_replace_map *
get_replacement_map (tree value, tree parm)
{
  tree req_type = TREE_TYPE (parm);
  struct ipa_replace_map *replace_map;

  if (!useless_type_conversion_p (req_type, TREE_TYPE (value)))
    {
      if (fold_convertible_p (req_type, value))
	value = fold_build1 (NOP_EXPR, req_type, value);
      else if (TYPE_SIZE (req_type) == TYPE_SIZE (TREE_TYPE (value)))
	value = fold_build1 (VIEW_CONVERT_EXPR, req_type, value);
      else
	{
	  if (dump_file)
	    {
	      fprintf (dump_file, "    const ");
	      print_generic_expr (dump_file, value, 0);
	      fprintf (dump_file, "  can't be converted to param ");
	      print_generic_expr (dump_file, parm, 0);
	      fprintf (dump_file, "\n");
	    }
	  return NULL;
	}
    }

  replace_map = ggc_alloc_ipa_replace_map ();
  if (dump_file)
    {
      fprintf (dump_file, "    replacing param ");
      print_generic_expr (dump_file, parm, 0);
      fprintf (dump_file, " with const ");
      print_generic_expr (dump_file, value, 0);
      fprintf (dump_file, "\n");
    }
  replace_map->old_tree = parm;
  replace_map->new_tree = value;
  replace_map->replace_p = true;
  replace_map->ref_p = false;

  return replace_map;
}

/* Dump new profiling counts */

static void
dump_profile_updates (struct cgraph_node *orig_node,
		      struct cgraph_node *new_node)
{
  struct cgraph_edge *cs;

  fprintf (dump_file, "    setting count of the specialized node to "
	   HOST_WIDE_INT_PRINT_DEC "\n", (HOST_WIDE_INT) new_node->count);
  for (cs = new_node->callees; cs ; cs = cs->next_callee)
    fprintf (dump_file, "      edge to %s has count "
	     HOST_WIDE_INT_PRINT_DEC "\n",
	     cgraph_node_name (cs->callee), (HOST_WIDE_INT) cs->count);

  fprintf (dump_file, "    setting count of the original node to "
	   HOST_WIDE_INT_PRINT_DEC "\n", (HOST_WIDE_INT) orig_node->count);
  for (cs = orig_node->callees; cs ; cs = cs->next_callee)
    fprintf (dump_file, "      edge to %s is left with "
	     HOST_WIDE_INT_PRINT_DEC "\n",
	     cgraph_node_name (cs->callee), (HOST_WIDE_INT) cs->count);
}


/* After a specialized NEW_NODE version of ORIG_NODE has been created, update
   their profile information to reflect this.  */

static void
update_profiling_info (struct cgraph_node *orig_node,
		       struct cgraph_node *new_node)
{
  struct cgraph_edge *cs;
  struct caller_statistics stats;
  gcov_type new_sum, orig_sum;
  gcov_type remainder, orig_node_count = orig_node->count;

  if (orig_node_count == 0)
    return;

  init_caller_stats (&stats);
  cgraph_for_node_and_aliases (orig_node, gather_caller_stats, &stats, false);
  orig_sum = stats.count_sum;
  init_caller_stats (&stats);
  cgraph_for_node_and_aliases (new_node, gather_caller_stats, &stats, false);
  new_sum = stats.count_sum;

  if (orig_node_count < orig_sum + new_sum)
    {
      if (dump_file)
	fprintf (dump_file, "    Problem: node %s/%i has too low count "
		 HOST_WIDE_INT_PRINT_DEC " while the sum of incoming "
		 "counts is " HOST_WIDE_INT_PRINT_DEC "\n",
		 cgraph_node_name (orig_node), orig_node->uid,
		 (HOST_WIDE_INT) orig_node_count,
		 (HOST_WIDE_INT) (orig_sum + new_sum));

      orig_node_count = (orig_sum + new_sum) * 12 / 10;
      if (dump_file)
	fprintf (dump_file, "      proceeding by pretending it was "
		 HOST_WIDE_INT_PRINT_DEC "\n",
		 (HOST_WIDE_INT) orig_node_count);
    }

  new_node->count = new_sum;
  remainder = orig_node_count - new_sum;
  orig_node->count = remainder;

  for (cs = new_node->callees; cs ; cs = cs->next_callee)
    if (cs->frequency)
      cs->count = cs->count * new_sum / orig_node_count;
    else
      cs->count = 0;

  for (cs = orig_node->callees; cs ; cs = cs->next_callee)
    cs->count = cs->count * remainder / orig_node_count;

  if (dump_file)
    dump_profile_updates (orig_node, new_node);
}

/* Update the respective profile of specialized NEW_NODE and the original
   ORIG_NODE after additional edges with cumulative count sum REDIRECTED_SUM
   have been redirected to the specialized version.  */

static void
update_specialized_profile (struct cgraph_node *new_node,
			    struct cgraph_node *orig_node,
			    gcov_type redirected_sum)
{
  struct cgraph_edge *cs;
  gcov_type new_node_count, orig_node_count = orig_node->count;

  if (dump_file)
    fprintf (dump_file, "    the sum of counts of redirected  edges is "
	     HOST_WIDE_INT_PRINT_DEC "\n", (HOST_WIDE_INT) redirected_sum);
  if (orig_node_count == 0)
    return;

  gcc_assert (orig_node_count >= redirected_sum);

  new_node_count = new_node->count;
  new_node->count += redirected_sum;
  orig_node->count -= redirected_sum;

  for (cs = new_node->callees; cs ; cs = cs->next_callee)
    if (cs->frequency)
      cs->count += cs->count * redirected_sum / new_node_count;
    else
      cs->count = 0;

  for (cs = orig_node->callees; cs ; cs = cs->next_callee)
    {
      gcov_type dec = cs->count * redirected_sum / orig_node_count;
      if (dec < cs->count)
	cs->count -= dec;
      else
	cs->count = 0;
    }

  if (dump_file)
    dump_profile_updates (orig_node, new_node);
}

/* Create a specialized version of NODE with known constants and types of
   parameters in KNOWN_VALS and redirect all edges in CALLERS to it.  */

static struct cgraph_node *
create_specialized_node (struct cgraph_node *node,
			 VEC (tree, heap) *known_vals,
			 VEC (cgraph_edge_p,heap) *callers)
{
  struct ipa_node_params *new_info, *info = IPA_NODE_REF (node);
  VEC (ipa_replace_map_p,gc)* replace_trees = NULL;
  struct cgraph_node *new_node;
  int i, count = ipa_get_param_count (info);
  bitmap args_to_skip;

  gcc_assert (!info->ipcp_orig_node);

  if (node->local.can_change_signature)
    {
      args_to_skip = BITMAP_GGC_ALLOC ();
      for (i = 0; i < count; i++)
	{
	  tree t = VEC_index (tree, known_vals, i);

	  if ((t && TREE_CODE (t) != TREE_BINFO)
	      || !ipa_is_param_used (info, i))
	    bitmap_set_bit (args_to_skip, i);
	}
    }
  else
    args_to_skip = NULL;

  for (i = 0; i < count ; i++)
    {
      tree t = VEC_index (tree, known_vals, i);
      if (t && TREE_CODE (t) != TREE_BINFO)
	{
	  struct ipa_replace_map *replace_map;

	  replace_map = get_replacement_map (t, ipa_get_param (info, i));
	  if (replace_map)
	    VEC_safe_push (ipa_replace_map_p, gc, replace_trees, replace_map);
	}
    }

  new_node = cgraph_create_virtual_clone (node, callers, replace_trees,
					  args_to_skip, "constprop");
  if (dump_file && (dump_flags & TDF_DETAILS))
    fprintf (dump_file, "     the new node is %s/%i.\n",
	     cgraph_node_name (new_node), new_node->uid);
  gcc_checking_assert (ipa_node_params_vector
		       && (VEC_length (ipa_node_params_t,
				       ipa_node_params_vector)
			   > (unsigned) cgraph_max_uid));
  update_profiling_info (node, new_node);
  new_info = IPA_NODE_REF (new_node);
  gcc_checking_assert (new_info->lattices);
  new_info->ipcp_orig_node = node;
  new_info->known_vals = known_vals;

  ipcp_discover_new_direct_edges (new_node, known_vals);

  VEC_free (cgraph_edge_p, heap, callers);
  return new_node;
}

/* Given a NODE, and a subset of its CALLERS, try to populate blanks slots in
   KNOWN_VALS with constants and types that are also known for all of the
   CALLERS.  */

static void
find_more_values_for_callers_subset (struct cgraph_node *node,
				     VEC (tree, heap) *known_vals,
				     VEC (cgraph_edge_p,heap) *callers)
{
  struct ipa_node_params *info = IPA_NODE_REF (node);
  int i, count = ipa_get_param_count (info);

  for (i = 0; i < count ; i++)
    {
      struct cgraph_edge *cs;
      tree newval = NULL_TREE;
      int j;

      if (ipa_get_lattice (info, i)->bottom
	  || VEC_index (tree, known_vals, i))
	continue;

      FOR_EACH_VEC_ELT (cgraph_edge_p, callers, j, cs)
	{
	  struct ipa_jump_func *jump_func;
	  tree t;

	  jump_func = ipa_get_ith_jump_func (IPA_EDGE_REF (cs), i);

	  t = ipa_value_from_jfunc (IPA_NODE_REF (cs->caller), jump_func);
	  if (!t
	      || (newval
		  && !values_equal_for_ipcp_p (t, newval)))
	    {
	      newval = NULL_TREE;
	      break;
	    }
	  else
	    newval = t;
	}

      if (newval)
	{
	  if (dump_file && (dump_flags & TDF_DETAILS))
	    {
	      fprintf (dump_file, "    adding an extra known value ");
	      print_ipcp_constant_value (dump_file, newval);
	      fprintf (dump_file, " for parameter ");
	      print_generic_expr (dump_file, ipa_get_param (info, i), 0);
	      fprintf (dump_file, "\n");
	    }

	  VEC_replace (tree, known_vals, i, newval);
	}
    }
}

/* Given an original NODE and a VAL for which we have already created a
   specialized clone, look whether there are incoming edges that still lead
   into the old node but now also bring the requested value and also conform to
   all other criteria such that they can be redirected the the special node.
   This function can therefore redirect the final edge in a SCC.  */

static void
perhaps_add_new_callers (struct cgraph_node *node, struct ipcp_value *val)
{
  struct ipa_node_params *dest_info = IPA_NODE_REF (val->spec_node);
  struct ipcp_value_source *src;
  int count = ipa_get_param_count (dest_info);
  gcov_type redirected_sum = 0;

  for (src = val->sources; src; src = src->next)
    {
      struct cgraph_edge *cs = src->cs;
      while (cs)
	{
	  enum availability availability;
	  bool insufficient = false;

	  if (cgraph_function_or_thunk_node (cs->callee, &availability) == node
	      && availability > AVAIL_OVERWRITABLE
	      && cgraph_edge_brings_value_p (cs, src))
	    {
	      struct ipa_node_params *caller_info;
	      struct ipa_edge_args *args;
	      int i;

	      caller_info = IPA_NODE_REF (cs->caller);
	      args = IPA_EDGE_REF (cs);
	      for (i = 0; i < count; i++)
		{
		  struct ipa_jump_func *jump_func;
		  tree val, t;

		  val = VEC_index (tree, dest_info->known_vals, i);
		  if (!val)
		    continue;

		  jump_func = ipa_get_ith_jump_func (args, i);
		  t = ipa_value_from_jfunc (caller_info, jump_func);
		  if (!t || !values_equal_for_ipcp_p (val, t))
		    {
		      insufficient = true;
		      break;
		    }
		}

	      if (!insufficient)
		{
		  if (dump_file)
		    fprintf (dump_file, " - adding an extra caller %s/%i"
			     " of %s/%i\n",
			     cgraph_node_name (cs->caller), cs->caller->uid,
			     cgraph_node_name (val->spec_node),
			     val->spec_node->uid);

		  cgraph_redirect_edge_callee (cs, val->spec_node);
		  redirected_sum += cs->count;
		}
	    }
	  cs = get_next_cgraph_edge_clone (cs);
	}
    }

  if (redirected_sum)
    update_specialized_profile (val->spec_node, node, redirected_sum);
}


/* Copy KNOWN_BINFOS to KNOWN_VALS.  */

static void
move_binfos_to_values (VEC (tree, heap) *known_vals,
		       VEC (tree, heap) *known_binfos)
{
  tree t;
  int i;

  for (i = 0; VEC_iterate (tree, known_binfos, i, t); i++)
    if (t)
      VEC_replace (tree, known_vals, i, t);
}


/* Decide whether and what specialized clones of NODE should be created.  */

static bool
decide_whether_version_node (struct cgraph_node *node)
{
  struct ipa_node_params *info = IPA_NODE_REF (node);
  int i, count = ipa_get_param_count (info);
  VEC (tree, heap) *known_csts, *known_binfos;
  bool ret = false;

  if (count == 0)
    return false;

  if (dump_file && (dump_flags & TDF_DETAILS))
    fprintf (dump_file, "\nEvaluating opportunities for %s/%i.\n",
	     cgraph_node_name (node), node->uid);

  gather_context_independent_values (info, &known_csts, &known_binfos,
				     NULL);

  for (i = 0; i < count ; i++)
    {
      struct ipcp_lattice *lat = ipa_get_lattice (info, i);
      struct ipcp_value *val;

      if (lat->bottom
	  || VEC_index (tree, known_csts, i)
	  || VEC_index (tree, known_binfos, i))
	continue;

      for (val = lat->values; val; val = val->next)
	{
	  int freq_sum, caller_count;
	  gcov_type count_sum;
	  VEC (cgraph_edge_p, heap) *callers;
	  VEC (tree, heap) *kv;

	  if (val->spec_node)
	    {
	      perhaps_add_new_callers (node, val);
	      continue;
	    }
	  else if (val->local_size_cost + overall_size > max_new_size)
	    {
	      if (dump_file && (dump_flags & TDF_DETAILS))
		fprintf (dump_file, "   Ignoring candidate value because "
			 "max_new_size would be reached with %li.\n",
			 val->local_size_cost + overall_size);
	      continue;
	    }
	  else if (!get_info_about_necessary_edges (val, &freq_sum, &count_sum,
						    &caller_count))
	    continue;

	  if (dump_file && (dump_flags & TDF_DETAILS))
	    {
	      fprintf (dump_file, " - considering value ");
	      print_ipcp_constant_value (dump_file, val->value);
	      fprintf (dump_file, " for parameter ");
	      print_generic_expr (dump_file, ipa_get_param (info, i), 0);
	      fprintf (dump_file, " (caller_count: %i)\n", caller_count);
	    }


	  if (!good_cloning_opportunity_p (node, val->local_time_benefit,
					   freq_sum, count_sum,
					   val->local_size_cost)
	      && !good_cloning_opportunity_p (node,
					      val->local_time_benefit
					      + val->prop_time_benefit,
					      freq_sum, count_sum,
					      val->local_size_cost
					      + val->prop_size_cost))
	    continue;

	  if (dump_file)
	    fprintf (dump_file, "  Creating a specialized node of %s/%i.\n",
		     cgraph_node_name (node), node->uid);

	  callers = gather_edges_for_value (val, caller_count);
	  kv = VEC_copy (tree, heap, known_csts);
	  move_binfos_to_values (kv, known_binfos);
	  VEC_replace (tree, kv, i, val->value);
	  find_more_values_for_callers_subset (node, kv, callers);
	  val->spec_node = create_specialized_node (node, kv, callers);
	  overall_size += val->local_size_cost;
	  info = IPA_NODE_REF (node);

	  /* TODO: If for some lattice there is only one other known value
	     left, make a special node for it too. */
	  ret = true;

	  VEC_replace (tree, kv, i, val->value);
	}
    }

  if (info->clone_for_all_contexts)
    {
      VEC (cgraph_edge_p, heap) *callers;

      if (dump_file)
	fprintf (dump_file, " * Creating a specialized node of %s/%i "
		 "for all contexts.\n", cgraph_node_name (node), node->uid);

      callers = collect_callers_of_node (node);
      move_binfos_to_values (known_csts, known_binfos);
      create_specialized_node (node, known_csts, callers);
      info = IPA_NODE_REF (node);
      info->clone_for_all_contexts = false;
      ret = true;
    }
  else
    VEC_free (tree, heap, known_csts);

  VEC_free (tree, heap, known_binfos);
  return ret;
}

/* Transitively mark all callees of NODE within the same SCC as not dead.  */

static void
spread_undeadness (struct cgraph_node *node)
{
  struct cgraph_edge *cs;

  for (cs = node->callees; cs; cs = cs->next_callee)
    if (edge_within_scc (cs))
      {
	struct cgraph_node *callee;
	struct ipa_node_params *info;

	callee = cgraph_function_or_thunk_node (cs->callee, NULL);
	info = IPA_NODE_REF (callee);

	if (info->node_dead)
	  {
	    info->node_dead = 0;
	    spread_undeadness (callee);
	  }
      }
}

/* Return true if NODE has a caller from outside of its SCC that is not
   dead.  Worker callback for cgraph_for_node_and_aliases.  */

static bool
has_unded_caller_from_outside_scc_p (struct cgraph_node *node,
				     void *data ATTRIBUTE_UNUSED)
{
  struct cgraph_edge *cs;

  for (cs = node->callers; cs; cs = cs->next_caller)
    if (cs->caller->thunk.thunk_p
	&& has_unded_caller_from_outside_scc_p (cs->caller, NULL))
      return true;
    else if (!edge_within_scc (cs)
	     && !IPA_NODE_REF (cs->caller)->node_dead)
      return true;
  return false;
}


/* Identify nodes within the same SCC as NODE which are no longer needed
   because of new clones and will be removed as unreachable.  */

static void
identify_dead_nodes (struct cgraph_node *node)
{
  struct cgraph_node *v;
  for (v = node; v ; v = ((struct ipa_dfs_info *) v->aux)->next_cycle)
    if (cgraph_will_be_removed_from_program_if_no_direct_calls (v)
	&& !cgraph_for_node_and_aliases (v,
					 has_unded_caller_from_outside_scc_p,
					 NULL, true))
      IPA_NODE_REF (v)->node_dead = 1;

  for (v = node; v ; v = ((struct ipa_dfs_info *) v->aux)->next_cycle)
    if (!IPA_NODE_REF (v)->node_dead)
      spread_undeadness (v);

  if (dump_file && (dump_flags & TDF_DETAILS))
    {
      for (v = node; v ; v = ((struct ipa_dfs_info *) v->aux)->next_cycle)
	if (IPA_NODE_REF (v)->node_dead)
	  fprintf (dump_file, "  Marking node as dead: %s/%i.\n",
		   cgraph_node_name (v), v->uid);
    }
}

/* The decision stage.  Iterate over the topological order of call graph nodes
   TOPO and make specialized clones if deemed beneficial.  */

static void
ipcp_decision_stage (struct topo_info *topo)
{
  int i;

  if (dump_file)
    fprintf (dump_file, "\nIPA decision stage:\n\n");

  for (i = topo->nnodes - 1; i >= 0; i--)
    {
      struct cgraph_node *node = topo->order[i];
      bool change = false, iterate = true;

      while (iterate)
	{
	  struct cgraph_node *v;
	  iterate = false;
	  for (v = node; v ; v = ((struct ipa_dfs_info *) v->aux)->next_cycle)
	    if (cgraph_function_with_gimple_body_p (v)
		&& ipcp_versionable_function_p (v))
	      iterate |= decide_whether_version_node (v);

	  change |= iterate;
	}
      if (change)
	identify_dead_nodes (node);
    }
}

/* The IPCP driver.  */

static unsigned int
ipcp_driver (void)
{
  struct topo_info topo;

  cgraph_remove_unreachable_nodes (true,dump_file);
  ipa_check_create_node_params ();
  ipa_check_create_edge_args ();
  ipcp_values_pool = create_alloc_pool ("IPA-CP values",
					sizeof (struct ipcp_value), 32);
  ipcp_sources_pool = create_alloc_pool ("IPA-CP value sources",
					 sizeof (struct ipcp_value_source), 64);

  if (dump_file)
    {
      fprintf (dump_file, "\nIPA structures before propagation:\n");
      if (dump_flags & TDF_DETAILS)
        ipa_print_all_params (dump_file);
      ipa_print_all_jump_functions (dump_file);
    }

  build_topo_info (&topo);
  /* 2. Do the interprocedural propagation.  */
  ipcp_propagate_stage (&topo);
  /* 3. Decide what constant propagation and cloning should be performed.  */
  ipcp_decision_stage (&topo);

  /* Free all IPCP structures.  */
  free_topo_info (&topo);

  ipa_free_all_structures_after_ipa_cp ();
  if (dump_file)
    fprintf (dump_file, "\nIPA constant propagation end\n");
  return 0;
}

/* Initialization and computation of IPCP data structures.  This is the initial
   intraprocedural analysis of functions, which gathers information to be
   propagated later on.  */

static void
ipcp_generate_summary (void)
{
  struct cgraph_node *node;

  if (dump_file)
    fprintf (dump_file, "\nIPA constant propagation start:\n");
  ipa_register_cgraph_hooks ();

  /* FIXME: We could propagate through thunks happily and we could be
     even able to clone them, if needed.  Do that later.  */
  FOR_EACH_FUNCTION_WITH_GIMPLE_BODY (node)
      {
	/* Unreachable nodes should have been eliminated before ipcp.  */
	gcc_assert (node->needed || node->reachable);

	inline_summary (node)->versionable = tree_versionable_function_p (node->decl);
	ipa_analyze_node (node);
      }
}

/* Write ipcp summary for nodes in SET.  */

static void
ipcp_write_summary (cgraph_node_set set,
		    varpool_node_set vset ATTRIBUTE_UNUSED)
{
  ipa_prop_write_jump_functions (set);
}

/* Read ipcp summary.  */

static void
ipcp_read_summary (void)
{
  ipa_prop_read_jump_functions ();
}

/* Gate for IPCP optimization.  */

static bool
cgraph_gate_cp (void)
{
  /* FIXME: We should remove the optimize check after we ensure we never run
     IPA passes when not optimizing.  */
  return flag_ipa_cp && optimize;
}

struct ipa_opt_pass_d pass_ipa_cp =
{
 {
  IPA_PASS,
  "cp",				/* name */
  cgraph_gate_cp,		/* gate */
  ipcp_driver,			/* execute */
  NULL,				/* sub */
  NULL,				/* next */
  0,				/* static_pass_number */
  TV_IPA_CONSTANT_PROP,		/* tv_id */
  0,				/* properties_required */
  0,				/* properties_provided */
  0,				/* properties_destroyed */
  0,				/* todo_flags_start */
  TODO_dump_cgraph |
  TODO_remove_functions | TODO_ggc_collect /* todo_flags_finish */
 },
 ipcp_generate_summary,			/* generate_summary */
 ipcp_write_summary,			/* write_summary */
 ipcp_read_summary,			/* read_summary */
 NULL,					/* write_optimization_summary */
 NULL,					/* read_optimization_summary */
 NULL,			 		/* stmt_fixup */
 0,					/* TODOs */
 NULL,					/* function_transform */
 NULL,					/* variable_transform */
};

Comments

Jan Hubicka July 7, 2011, 4:03 p.m. UTC | #1
Hi,
patch is long, so let me review it in more passes.
> 
> 
> 2011-06-22  Martin Jambor  <mjambor@suse.cz>
> 
> 	* ipa-prop.h: Include alloc-pool.h.
> 	(ipa_lattice_type): Removed.
> 	(ipcp_value_source): New type.
> 	(ipcp_value): Likewise.
> 	(ipcp_values_pool): Declare.
> 	(ipcp_sources_pool): Likewise.
> 	(ipa_param_descriptor): Removed.
> 	(ipcp_lattice): Removed fileds type and constant. Added fields decl,
> 	values, values_count, contains_variable, bottom, used and virt_call.
> 	(ipa_node_params): New fields lattices, known_vals,
> 	clone_for_all_contexts and noe dead, removed fields params and
> 	count_scale.
> 	(ipa_get_param): Updated.
> 	(ipa_param_cannot_devirtualize_p): Removed.
> 	(ipa_param_types_vec_empty): Likewise.
> 	(ipa_edge_args): New field next_edge_clone.
> 	(ipa_func_list): Removed.
> 	(ipa_init_func_list): Removed declaration.
> 	(ipa_push_func_to_list_1): Likewise.
> 	(ipa_pop_func_from_list): Likewise.
> 	(ipa_push_func_to_list): Removed.
> 	(ipa_lattice_from_jfunc): Remove declaration.
> 	(ipa_get_jf_pass_through_result): Declare.
> 	(ipa_get_jf_ancestor_result): Likewise.
> 	(ipa_value_from_jfunc): Likewise.
> 	(ipa_get_lattice): Update.
> 	(ipa_lat_is_single_const): New function.
> 	* ipa-prop.c (ipa_push_func_to_list_1): Removed.
> 	(ipa_init_func_list): Likewise.
> 	(ipa_pop_func_from_list): Likewise.
> 	(ipa_get_param_decl_index): Fix coding style.
> 	(ipa_populate_param_decls): Update to use new lattices.
> 	(ipa_initialize_node_params): Likewise.
> 	(visit_ref_for_mod_analysis): Likewise.
> 	(ipa_analyze_params_uses): Likewise.
> 	(ipa_free_node_params_substructures): Likewise.
> 	(ipa_edge_duplication_hook): Add the new edge to the list of edge
> 	clones.
> 	(ipa_node_duplication_hook): Update to use new lattices.
> 	(ipa_free_all_structures_after_ipa_cp): Free alloc pools.
> 	(ipa_free_all_structures_after_iinln): Likewise.
> 	(ipa_write_node_info): Update to use new lattices.
> 	(ipa_read_node_info): Likewise.
> 	(ipa_get_jf_pass_through_result): New function.
> 	(ipa_get_jf_ancestor_result): Likewise.
> 	(ipa_value_from_jfunc): Likewise.
> 	(ipa_cst_from_jfunc): Reimplemented using ipa_value_from_jfunc.
> 	* ipa-cp.c: Reimplemented.
> 	* params.def (PARAM_DEVIRT_TYPE_LIST_SIZE): Removed.
> 	(PARAM_IPA_CP_VALUE_LIST_SIZE): New parameter.
> 	* Makefile.in (IPA_PROP_H): Added alloc-pool.h to dependencies.
> 
> 	* doc/invoke.texi (devirt-type-list-size): Removed description.
> 	(ipa-cp-value-list-size): Added description.
> 
> 	* testsuite/gcc.dg/ipa/ipa-1.c: Updated testcase dump scan.
> 	* testsuite/gcc.dg/ipa/ipa-2.c: Likewise.
> 	* testsuite/gcc.dg/ipa/ipa-3.c: Likewise and made functions static.
> 	* testsuite/gcc.dg/ipa/ipa-4.c: Updated testcase dump scan.
> 	* testsuite/gcc.dg/ipa/ipa-5.c: Likewise.
> 	* testsuite/gcc.dg/ipa/ipa-7.c: Xfail test.
> 	* testsuite/gcc.dg/ipa/ipa-8.c: Updated testcase dump scan.
> 	* testsuite/gcc.dg/ipa/ipacost-1.c: Likewise.
> 	* testsuite/gcc.dg/ipa/ipacost-2.c: Likewise.
> 	* testsuite/gcc.dg/ipa/ipcp-1.c: New test.
> 	* testsuite/gcc.dg/ipa/ipcp-2.c: Likewise.
> 	* testsuite/gcc.dg/tree-ssa/ipa-cp-1.c: Updated testcase.

> /* Interprocedural analyses.
>    Copyright (C) 2005, 2007, 2008, 2009, 2010
2011
>    Free Software Foundation, Inc.
> 
> 
> /* The following definitions and interfaces are used by
>    interprocedural analyses or parameters.  */
> 
> /* ipa-prop.c stuff (ipa-cp, indirect inlining):  */

I was bit thinking about it and probably we could make ipa-prop
and ipa-inline-analysis to be stand alone analysis passes, instead of
something called either from inliner or ipa-cp analysis stage. But
that could be done incrementally.

> 
> /* A jump function for a callsite represents the values passed as actual
>    arguments of the callsite. There are three main types of values :
> 
>    Pass-through - the caller's formal parameter is passed as an actual
>                   argument, possibly one simple operation performed on it.
>    Constant     - a constant (is_gimple_ip_invariant)is passed as an actual
>                   argument.
>    Unknown      - neither of the above.
> 
>    IPA_JF_CONST_MEMBER_PTR stands for C++ member pointers, it is a special
>    constant in this regard.  Other constants are represented with IPA_JF_CONST.

While we are at docs, I would bit expand. It seems to me that for someone not familiar
with the concept is not clear at all why member pointers are special.
(i.e. explain that they are non-gimple-regs etc.)

> 
>    IPA_JF_ANCESTOR is a special pass-through jump function, which means that
>    the result is an address of a part of the object pointed to by the formal
>    parameter to which the function refers.  It is mainly intended to represent
>    getting addresses of of ancestor fields in C++
>    (e.g. &this_1(D)->D.1766.D.1756).  Note that if the original pointer is
>    NULL, ancestor jump function must behave like a simple pass-through.
> 
>    Other pass-through functions can either simply pass on an unchanged formal
>    parameter or can apply one simple binary operation to it (such jump
>    functions are called polynomial).
> 
>    IPA_JF_KNOWN_TYPE is a special type of an "unknown" function that applies
>    only to pointer parameters.  It means that even though we cannot prove that
>    the passed value is an interprocedural constant, we still know the exact
>    type of the containing object which may be valuable for devirtualization.
> 
>    Jump functions are computed in ipa-prop.c by function
>    update_call_notes_after_inlining.  Some information can be lost and jump
>    functions degraded accordingly when inlining, see
>    update_call_notes_after_inlining in the same file.  */

I would add pointer to the original Callahan at al ipa-cp paper.
I think it is cited from ipa-cp, so just copy it from there.

> 
> /* Structure holding data required to describe a pass-through jump function.  */
> 
> struct GTY(()) ipa_pass_through_data
> {
>   /* If an operation is to be performed on the original parameter, this is the
>      second (constant) operand.  */
>   tree operand;
>   /* Number of the caller's formal parameter being passed.  */
>   int formal_id;

I probably should use this in ipa-inline-analsysi where I call it for some reason operand_num :)

> 
> struct ipcp_value;

I wonder if the jump functions used by several passes and ipcp internal types both has o go into
the same header?

>   /* Time benefit and size cost that specializing the function for this value
>      can bring about in it's callees (transitively).  */
>   int prop_time_benefit, prop_size_cost;
>   /* True if this valye is currently on the topo-sort stack.  */
value ;)
> /* Lattice describing potential values of a formal parameter of a function and
>    some of their other properties.  */
> 
> struct ipcp_lattice
> {
>   /* PARAM_DECL of this parameter.  */
>   tree decl;
Why we need param de cl here, when almost everything seems to be using those formal_ids?

Please add comment how the latice looks like, it is not quite ovious i.e. what is top etc.
> 
>   /* The list of known values and types in this lattice.  */
>   struct ipcp_value *values;
>   /* Number of known values and types in this lattice.  */
>   int values_count;
>   /* The lattice contains a variable component  (in addition to values).  */
>   bool contains_variable;
>   /* The value of the lattice is bottom (i.e. variable and unusable for any
>      propagation).  */
>   bool bottom;
>   /* The parameter is used.  */
>   bool used;
>   /* There is a virtual call based on this parameter.  */
>   bool virt_call;
> };
> 
> /* ipa_node_params stores information related to formal parameters of functions
>    and some other information for interprocedural passes that operate on
>    parameters (such as ipa-cp).  */
> struct ipa_node_params

Hmm, I see, this is why we need to mix the ipacp and ipa-prop together.  Probably it would be
easier if we had ipa_prop_node_params and ipacp_node-params
(and I call it summary in ipa-inline that I think is sor of common way to call this)
> {
>   /* Pointer to an array of structures describing individual formal
>      parameters.  */
>   struct ipcp_lattice *lattices;

Hmm, how we get here around the need to mark this GTY(). I.e are we sure that all the known_vals
must be referneced from elsewhere at ggc time?
I would also slowly switch those things to VECtors..

>   /* Only for versioned nodes this field would not be NULL,
>      it points to the node that IPA cp cloned from.  */
>   struct cgraph_node *ipcp_orig_node;
Why not use node->clone_of here?
> 
> /* ipa_node_params access functions.  Please use these to access fields that
>    are or will be shared among various passes.  */
> 
> /* Set the number of formal parameters. */
> 
> static inline void
> ipa_set_param_count (struct ipa_node_params *info, int count)
> {
>   info->param_count = count;
> }
> 
> /* Return the number of formal parameters. */
> 
> static inline int
> ipa_get_param_count (struct ipa_node_params *info)
> {
>   return info->param_count;
> }
I see this is the need for DECL pointer in lattice.  I think the ipcp latice is separate thing
from the parameter summary (that is also used by ipa-inline-analysis and friends) so I would
make this separate VECtor with ipa_set_param_count growing the vector to proper size
and get_param_count using VEC_length
> 
> /* Return the declaration of Ith formal parameter of the function corresponding
>    to INFO.  Note there is no setter function as this array is built just once
>    using ipa_initialize_node_params. */
> 
> static inline tree
> ipa_get_param (struct ipa_node_params *info, int i)
> {
>   gcc_assert (i >= 0 && i <= info->param_count);
>   return info->lattices[i].decl;
> }
> 
> /* Return the used flag corresponding to the Ith formal parameter of
>    the function associated with INFO.  */
> 
> static inline bool
> ipa_is_param_used (struct ipa_node_params *info, int i)
> {
>   gcc_assert (i >= 0 && i <= info->param_count);
>   return info->lattices[i].used;
> }
Similarly for the USED flag, should probably go into VECtor along with the DECL pointer.
> 
> /* ipa_edge_args stores information related to a callsite and particularly its
>    arguments.  It can be accessed by the IPA_EDGE_REF macro.  */
> typedef struct GTY(()) ipa_edge_args

probably edge_summary would be my preferred name.

> {
>   /* Next pointer in a linked list of clones of the same function.  */
>   struct cgraph_edge *next_edge_clone;

What this is needed for?
> 
>   /* Number of actual arguments in this callsite.  When set to 0,
>      this callsite's parameters would not be analyzed by the different
>      stages of IPA CP.  */
>   int argument_count;
>   /* Array of the callsite's jump function of each parameter.  */
>   struct ipa_jump_func GTY ((length ("%h.argument_count"))) *jump_functions;

Would be prettier as VECtor.

> } ipa_edge_args_t;
> 
> /* This function ensures the array of node param infos is big enough to
>    accommodate a structure for all nodes and reallocates it if not.  */
> 
> static inline void
> ipa_check_create_node_params (void)

this name never realy worked for me (i.e. what is it checking?).
What about ipa_alloc_node_params or ipa_alloc_resize_node_params?


I realize that most of changes I sugested above are to the existing ipa-prop
headers.  I would like to see them done, it depends what seems least painful
for you - i.e. do them as part of your patch, do them beforehand or
incrementally after the new ipa-cp is comitted to mainline.

In particular I would like to see the ipa-cp data (used at propagation stage)
separated from function summaries (i.e. jump functions, param counts and decl/used
info).

Things will be more moudlar that way and we save some memory, too.

> /* Interprocedural constant propagation
>    Copyright (C) 2005, 2006, 2007, 2008, 2009, 2010, 2011
>    Free Software Foundation, Inc.
>    Contributed by Razya Ladelsky <RAZYA@il.ibm.com>
Since it is complette rewrite, I would add you there ;)
> /* Interprocedural constant propagation (IPA-CP).
> 
>    The goal of this transformation is to
> 
>    1) discover functions which are always invoked with some arguments with the
>       same known constant values and modify the functions so that the
>       subsequent optimizations can take advantage of the knowledge, and
> 
>    2) create specialized versions of functions transformed in this way if
>       some parameters are known constants only in certain contexts but the
>       estimated tradeoff between speedup and cost size is deemed good.
> 
>    The algorithm also propagates types and attempts to perform type based
>    devirtualization.  Types are propagated much like constants.

I would add that the algorithm is not doing only the traditional ipa-cp propagation,
but also handles partial specialization and propagate known types for devirtualization.
>    First stage - intraprocedural analysis
>    =======================================
> 
>    This phase computes jump_function and modification flags.
> 
>    A jump function for a call-site represents the values passed as an actual
>    arguments of a given call-site. In principle, there are three types of
>    values:
> 
>    Pass through - the caller's formal parameter is passed as an actual
>                   argument, plus an operation on it can be performed.
>    Constant - a constant is passed as an actual argument.
>    Unknown - neither of the above.

Propbably should document the new fancy types of jump functions.
Also mention that return functions are not handled, yet.
> 
>    Second stage - interprocedural analysis
>    ========================================
> 
>    This stage is itself divided into two phases.  In the first, we propagate
>    known values over the call graph, in the second, we make cloning decisions.

I would also explicitely mention that this stage is done differently from the
classical ipa-cp paper.
> /* Return true iff the CS is an edge within a strongly connected component as
>    computed by ipa_reduced_postorder.  */
> 
> static inline bool
> edge_within_scc (struct cgraph_edge *cs)
> {
>   struct ipa_dfs_info *caller_dfs = (struct ipa_dfs_info *) cs->caller->aux;
>   struct ipa_dfs_info *callee_dfs;
>   struct cgraph_node *callee = cgraph_function_node (cs->callee, NULL);

since thunks are not transparent for you (and eventually we will want to have
jump functions for those; passthrough in all but first argument and jump
function representing the thunk adjustment), you want cgraph_function_or_thunk
node.
> 
> /* Print V which is extracted from a value in a lattice to F.  */
> 
> static void
> print_ipcp_constant_value (FILE * f, tree v)
> {
>   if (TREE_CODE (v) == TREE_BINFO)
>     {
>       fprintf (f, "BINFO ");
>       print_generic_expr (f, BINFO_TYPE (v), 0);

I wish we had some resonable way to visualise the BINFOs...
It was always completely unclear what really is in them. But this is independent problem.
> 
> /* Determine whether it is at all technically possible to create clones of NODE
>    and store this information in the ipa_node_params structure associated
>    with NODE.  */
> 
> static void
> determine_versionability (struct cgraph_node *node)
> {
>   struct cgraph_edge *edge;
>   bool res = true;
> 
>   /* There are a number of generic reasons functions cannot be versioned.  We
>      also cannot remove parameters if there are type attributes such as fnspec
>      present.  */
>   if (node->alias || node->thunk.thunk_p

We can version alias if its destination is versionable (i.e. get function_or_thunk node)
also thunks should be versionable but they are not at the moment, inline summary will
however have versionable flag cleared.

>       || !inline_summary (node)->versionable
>       || TYPE_ATTRIBUTES (TREE_TYPE (node->decl))

This is getting an issue for Fortran that attach the function arguments quite commonly,
perhaps we should start moving ahead on this and ruling out only the arguments that
gets can't be preserved.
Also you need to test attributes of DECL itself.

I think this all should be part of can_change_signature_p code, since we can version
with all attributes I can thunk of when we don't change signature.

>       || cgraph_function_body_availability (node) <= AVAIL_OVERWRITABLE)
>     res =  false;
>   else
>     /* Removing arguments doesn't work if the function takes varargs
>        or use __builtin_apply_args.
>        FIXME: handle this together with can_change_signature flag.  */
>     for (edge = node->callees; edge; edge = edge->next_callee)
>       {
> 	tree t = edge->callee->decl;
> 	if (DECL_BUILT_IN_CLASS (t) == BUILT_IN_NORMAL
> 	    && (DECL_FUNCTION_CODE (t) == BUILT_IN_APPLY_ARGS
> 		|| DECL_FUNCTION_CODE (t) == BUILT_IN_VA_START))

can_change_sigunature will also handle apply_args.
Add VA_START code there, too.  For the other use of this flag (in i386) VA_START
rules out the change, too, but so far we explicitely tested that in the backend.

Iguess with these changes, inline_summary (node)->versionable should coincide
with IPA_NODE_REF (node)->node_versionable making this whole code unnecesary
(it was supposed to work this way, I am not sure why we now have to versionable
flags).
> 
> /* Worker callback of cgraph_for_node_and_aliases accumulating statistics of
>    non-thunk incoming edges to NODE.  */
> 
> static bool
> gather_caller_stats (struct cgraph_node *node, void *data)
> {
>   struct caller_statistics *stats = (struct caller_statistics *) data;
>   struct cgraph_edge *cs;
> 
>   for (cs = node->callers; cs; cs = cs->next_caller)
>     if (cs->caller->thunk.thunk_p)
>       gather_caller_stats (cs->caller, stats);
>     else
>       {
> 	stats->count_sum += cs->count;
> 	stats->freq_sum += cs->frequency;
> 	stats->n_calls++;
> 	if (cgraph_maybe_hot_edge_p (cs))
> 	  stats->n_hot_calls ++;
>       }
>   return false;
Why you compute count_sum instead of using node->count?  Is it because you are
interested in local calls only?
>   if (!optimize_function_for_speed_p (DECL_STRUCT_FUNCTION (node->decl)))

When optimizing for size, we could still consider clonning the function if amount
of code saved at call sites is bigger than the function body..
Not really important however.
>     {
>       if (dump_file)
>         fprintf (dump_file, "Not considering %s for cloning; "
> 		 "optimizing it for size.\n",
>  	         cgraph_node_name (node));
>       return false;
>     }
> 
>   init_caller_stats (&stats);
>   cgraph_for_node_and_aliases (node, gather_caller_stats, &stats, false);
If you use for_node_thunk_and_aliases you can remove the recursion you do above
and it will work for aliases of thunk correctly, too ;)
> /* Allocate the arrays in TOPO and topologically sort the nodes into order.  */
> 
> static void
> build_topo_info (struct topo_info *topo)
toporder is probably more clear about what it means ;)
> /* Initialize ipcp_lattices.  */
> 
> static void
> initialize_node_lattices (struct cgraph_node *node)
> {
>   struct ipa_node_params *info = IPA_NODE_REF (node);
>   struct cgraph_edge *ie;
>   bool disable = false, variable = false;
>   int i;
> 
>   /* FIXME: Can we clobber only the first argument of thunks?  */
>   if (node->alias || node->thunk.thunk_p

I think sane thing to do here is to not have latices for aliases, since you skip
them during propagatoin, but have latices for thinks, since they should have their
own jump functions.

You can probably add sanity check into lattice accesor function that is is never
given an alias and skip the code disabling ipa-cp bellow leaving the data
uninitialized.

>       || ipa_is_called_with_var_arguments (info))

I always wondered if we need this, given that we check for use of stdarg.  Sure
writing vararg function without actually using the arguments is sort of stupid,
but it seems that the code should just work.

(at least with one bounds check in propagation for case function is called with
too few argumetns)

Also naturally propagating into stdarg functions is possible.

OK, rest later.

Honza
Martin Jambor July 8, 2011, 5 p.m. UTC | #2
Hi,

On Thu, Jul 07, 2011 at 06:03:07PM +0200, Jan Hubicka wrote:
> Hi,
> patch is long, so let me review it in more passes.

Fair enough.

> > 
> > 
> > 2011-06-22  Martin Jambor  <mjambor@suse.cz>
> > 
> > 	* ipa-prop.h: Include alloc-pool.h.
> > 	(ipa_lattice_type): Removed.
> > 	(ipcp_value_source): New type.
> > 	(ipcp_value): Likewise.
> > 	(ipcp_values_pool): Declare.
> > 	(ipcp_sources_pool): Likewise.
> > 	(ipa_param_descriptor): Removed.
> > 	(ipcp_lattice): Removed fileds type and constant. Added fields decl,
> > 	values, values_count, contains_variable, bottom, used and virt_call.
> > 	(ipa_node_params): New fields lattices, known_vals,
> > 	clone_for_all_contexts and noe dead, removed fields params and
> > 	count_scale.
> > 	(ipa_get_param): Updated.
> > 	(ipa_param_cannot_devirtualize_p): Removed.
> > 	(ipa_param_types_vec_empty): Likewise.
> > 	(ipa_edge_args): New field next_edge_clone.
> > 	(ipa_func_list): Removed.
> > 	(ipa_init_func_list): Removed declaration.
> > 	(ipa_push_func_to_list_1): Likewise.
> > 	(ipa_pop_func_from_list): Likewise.
> > 	(ipa_push_func_to_list): Removed.
> > 	(ipa_lattice_from_jfunc): Remove declaration.
> > 	(ipa_get_jf_pass_through_result): Declare.
> > 	(ipa_get_jf_ancestor_result): Likewise.
> > 	(ipa_value_from_jfunc): Likewise.
> > 	(ipa_get_lattice): Update.
> > 	(ipa_lat_is_single_const): New function.
> > 	* ipa-prop.c (ipa_push_func_to_list_1): Removed.
> > 	(ipa_init_func_list): Likewise.
> > 	(ipa_pop_func_from_list): Likewise.
> > 	(ipa_get_param_decl_index): Fix coding style.
> > 	(ipa_populate_param_decls): Update to use new lattices.
> > 	(ipa_initialize_node_params): Likewise.
> > 	(visit_ref_for_mod_analysis): Likewise.
> > 	(ipa_analyze_params_uses): Likewise.
> > 	(ipa_free_node_params_substructures): Likewise.
> > 	(ipa_edge_duplication_hook): Add the new edge to the list of edge
> > 	clones.
> > 	(ipa_node_duplication_hook): Update to use new lattices.
> > 	(ipa_free_all_structures_after_ipa_cp): Free alloc pools.
> > 	(ipa_free_all_structures_after_iinln): Likewise.
> > 	(ipa_write_node_info): Update to use new lattices.
> > 	(ipa_read_node_info): Likewise.
> > 	(ipa_get_jf_pass_through_result): New function.
> > 	(ipa_get_jf_ancestor_result): Likewise.
> > 	(ipa_value_from_jfunc): Likewise.
> > 	(ipa_cst_from_jfunc): Reimplemented using ipa_value_from_jfunc.
> > 	* ipa-cp.c: Reimplemented.
> > 	* params.def (PARAM_DEVIRT_TYPE_LIST_SIZE): Removed.
> > 	(PARAM_IPA_CP_VALUE_LIST_SIZE): New parameter.
> > 	* Makefile.in (IPA_PROP_H): Added alloc-pool.h to dependencies.
> > 
> > 	* doc/invoke.texi (devirt-type-list-size): Removed description.
> > 	(ipa-cp-value-list-size): Added description.
> > 
> > 	* testsuite/gcc.dg/ipa/ipa-1.c: Updated testcase dump scan.
> > 	* testsuite/gcc.dg/ipa/ipa-2.c: Likewise.
> > 	* testsuite/gcc.dg/ipa/ipa-3.c: Likewise and made functions static.
> > 	* testsuite/gcc.dg/ipa/ipa-4.c: Updated testcase dump scan.
> > 	* testsuite/gcc.dg/ipa/ipa-5.c: Likewise.
> > 	* testsuite/gcc.dg/ipa/ipa-7.c: Xfail test.
> > 	* testsuite/gcc.dg/ipa/ipa-8.c: Updated testcase dump scan.
> > 	* testsuite/gcc.dg/ipa/ipacost-1.c: Likewise.
> > 	* testsuite/gcc.dg/ipa/ipacost-2.c: Likewise.
> > 	* testsuite/gcc.dg/ipa/ipcp-1.c: New test.
> > 	* testsuite/gcc.dg/ipa/ipcp-2.c: Likewise.
> > 	* testsuite/gcc.dg/tree-ssa/ipa-cp-1.c: Updated testcase.
> 
> > /* Interprocedural analyses.
> >    Copyright (C) 2005, 2007, 2008, 2009, 2010
> 2011
> >    Free Software Foundation, Inc.
> > 
> > 
> > /* The following definitions and interfaces are used by
> >    interprocedural analyses or parameters.  */
> > 
> > /* ipa-prop.c stuff (ipa-cp, indirect inlining):  */
> 
> I was bit thinking about it and probably we could make ipa-prop
> and ipa-inline-analysis to be stand alone analysis passes, instead of
> something called either from inliner or ipa-cp analysis stage. But
> that could be done incrementally.

As I said in the first introductory mail, the summary generation part
is not really affected by this patch in any serious way.

> 
> > 
> > /* A jump function for a callsite represents the values passed as actual
> >    arguments of the callsite. There are three main types of values :
> > 
> >    Pass-through - the caller's formal parameter is passed as an actual
> >                   argument, possibly one simple operation performed on it.
> >    Constant     - a constant (is_gimple_ip_invariant)is passed as an actual
> >                   argument.
> >    Unknown      - neither of the above.
> > 
> >    IPA_JF_CONST_MEMBER_PTR stands for C++ member pointers, it is a special
> >    constant in this regard.  Other constants are represented with IPA_JF_CONST.
> 
> While we are at docs, I would bit expand. It seems to me that for someone not familiar
> with the concept is not clear at all why member pointers are special.
> (i.e. explain that they are non-gimple-regs etc.)

OK

> 
> > 
> >    IPA_JF_ANCESTOR is a special pass-through jump function, which means that
> >    the result is an address of a part of the object pointed to by the formal
> >    parameter to which the function refers.  It is mainly intended to represent
> >    getting addresses of of ancestor fields in C++
> >    (e.g. &this_1(D)->D.1766.D.1756).  Note that if the original pointer is
> >    NULL, ancestor jump function must behave like a simple pass-through.
> > 
> >    Other pass-through functions can either simply pass on an unchanged formal
> >    parameter or can apply one simple binary operation to it (such jump
> >    functions are called polynomial).
> > 
> >    IPA_JF_KNOWN_TYPE is a special type of an "unknown" function that applies
> >    only to pointer parameters.  It means that even though we cannot prove that
> >    the passed value is an interprocedural constant, we still know the exact
> >    type of the containing object which may be valuable for devirtualization.
> > 
> >    Jump functions are computed in ipa-prop.c by function
> >    update_call_notes_after_inlining.  Some information can be lost and jump
> >    functions degraded accordingly when inlining, see
> >    update_call_notes_after_inlining in the same file.  */
> 
> I would add pointer to the original Callahan at al ipa-cp paper.
> I think it is cited from ipa-cp, so just copy it from there.

Well, why not.

> 
> > 
> > /* Structure holding data required to describe a pass-through jump function.  */
> > 
> > struct GTY(()) ipa_pass_through_data
> > {
> >   /* If an operation is to be performed on the original parameter, this is the
> >      second (constant) operand.  */
> >   tree operand;
> >   /* Number of the caller's formal parameter being passed.  */
> >   int formal_id;
> 
> I probably should use this in ipa-inline-analsysi where I call it for some reason operand_num :)

So far I have resisted the urge to rename this but it pre-dates my
involvement with gcc.  I'd like it to be called parm_index but since
we might want to use it also for global variables and we might need
something more complex for also handling parts of aggregates, I left
it for later.

> 
> > 
> > struct ipcp_value;
> 
> I wonder if the jump functions used by several passes and ipcp
> internal types both has o go into the same header?

Well, I originally wanted ipa-prop to provide services to the outside
world like ipa-inline-analysis.c and to have ipa-cp self-contained.
But I guess the data separation is more important.  So if we move
ipa_cst_from_jfunc (and its 3 friends) to ipa-cp and move
ipcp_lattice.decl and ipcp_lattice.used to a special structure, I
might move ipcp_value_source, ipcp_value, and ipcp_lattice altogether
to ipa-cp.

At the moment I'm not sure whether I want to do this as a followup
patch or incorporate it in the changes.  I think I'll start with the
latter and revert to the former if it is too invasive to parts which
have not been touched so far by the change.

> 
> >   /* Time benefit and size cost that specializing the function for this value
> >      can bring about in it's callees (transitively).  */
> >   int prop_time_benefit, prop_size_cost;
> >   /* True if this value is currently on the topo-sort stack.  */
> value ;)
> > /* Lattice describing potential values of a formal parameter of a function and
> >    some of their other properties.  */
> > 
> > struct ipcp_lattice
> > {
> >   /* PARAM_DECL of this parameter.  */
> >   tree decl;

> Why we need param de cl here, when almost everything seems to be
> > using those formal_ids?

To look up the index when you have a decl, for replacement maps and
for dumping, and there may be other uses.

> 
> Please add comment how the latice looks like, it is not quite ovious
> i.e. what is top etc.

OK

> > 
> >   /* The list of known values and types in this lattice.  */
> >   struct ipcp_value *values;
> >   /* Number of known values and types in this lattice.  */
> >   int values_count;
> >   /* The lattice contains a variable component  (in addition to values).  */
> >   bool contains_variable;
> >   /* The value of the lattice is bottom (i.e. variable and unusable for any
> >      propagation).  */
> >   bool bottom;
> >   /* The parameter is used.  */
> >   bool used;
> >   /* There is a virtual call based on this parameter.  */
> >   bool virt_call;
> > };
> > 
> > /* ipa_node_params stores information related to formal parameters of functions
> >    and some other information for interprocedural passes that operate on
> >    parameters (such as ipa-cp).  */
> > struct ipa_node_params
> 
> Hmm, I see, this is why we need to mix the ipacp and ipa-prop together.  Probably it would be
> easier if we had ipa_prop_node_params and ipacp_node-params
> (and I call it summary in ipa-inline that I think is sor of common
> way to call this)

Yes, something like that would be part of the plan to split struct
ipcp_lattice into two.  I know that info and IPA_NODE_REF are not the
best names but I'd like to avoid calling everything a summary :-)

> > {
> >   /* Pointer to an array of structures describing individual formal
> >      parameters.  */
> >   struct ipcp_lattice *lattices;
> 
> Hmm, how we get here around the need to mark this GTY(). I.e are we sure that all the known_vals
> must be referneced from elsewhere at ggc time?

(Scalar) constants that are results of arithmetic jump functions may
not be referenced from elsewhere, everything else is referenced from
the jump functions.  If it is a problem it is already present in the
current IPA-CP.  ipa_node_params and lattices are not GTYed there
either.

> I would also slowly switch those things to VECtors..

Perhaps, but individual lattices are and always have been accessed
through ipa_get_lattice which checks bounds and so there's no big
reason to do that.

> 
> >   /* Only for versioned nodes this field would not be NULL,
> >      it points to the node that IPA cp cloned from.  */
> >   struct cgraph_node *ipcp_orig_node;
> Why not use node->clone_of here?

That would not work if the node was a clone created by some other
pass.  I need to differentiate between clones I create because they do
not have lattices but do have the exact values for individual
parameters in known_vals (which is NULL otherwise).  I should probably
use a flag though, the code I ended up only checks it for NULL anyway.

> > 
> > /* ipa_node_params access functions.  Please use these to access fields that
> >    are or will be shared among various passes.  */
> > 
> > /* Set the number of formal parameters. */
> > 
> > static inline void
> > ipa_set_param_count (struct ipa_node_params *info, int count)
> > {
> >   info->param_count = count;
> > }
> > 
> > /* Return the number of formal parameters. */
> > 
> > static inline int
> > ipa_get_param_count (struct ipa_node_params *info)
> > {
> >   return info->param_count;
> > }

> I see this is the need for DECL pointer in lattice.  I think the
> ipcp latice is separate thing from the parameter summary (that is
> also used by ipa-inline-analysis and friends) so I would make this
> separate VECtor with ipa_set_param_count growing the vector to
> proper size and get_param_count using VEC_length

I'll separate the lattice bits.

> > 
> > /* Return the declaration of Ith formal parameter of the function corresponding
> >    to INFO.  Note there is no setter function as this array is built just once
> >    using ipa_initialize_node_params. */
> > 
> > static inline tree
> > ipa_get_param (struct ipa_node_params *info, int i)
> > {
> >   gcc_assert (i >= 0 && i <= info->param_count);
> >   return info->lattices[i].decl;
> > }
> > 
> > /* Return the used flag corresponding to the Ith formal parameter of
> >    the function associated with INFO.  */
> > 
> > static inline bool
> > ipa_is_param_used (struct ipa_node_params *info, int i)
> > {
> >   gcc_assert (i >= 0 && i <= info->param_count);
> >   return info->lattices[i].used;
> > }

> Similarly for the USED flag, should probably go into VECtor along
> with the DECL pointer.

Right.

> > 
> > /* ipa_edge_args stores information related to a callsite and particularly its
> >    arguments.  It can be accessed by the IPA_EDGE_REF macro.  */
> > typedef struct GTY(()) ipa_edge_args
> 
> probably edge_summary would be my preferred name.

Ugh, this is the current name, we may change it later.  In any event
the name should probably tell that the summary is about parameters.

> 
> > {
> >   /* Next pointer in a linked list of clones of the same function.  */
> >   struct cgraph_edge *next_edge_clone;
> 
> What this is needed for?

For get_info_about_necessary_edges and gather_edges_for_value.  If a
value comes from a node and that is itself cloned, it now may come
from two nodes and so when I iterate over potential sources, I have to
include all such edges.  This is their linked list.

This field could be put into a separate structure because it is not
really part of the summary, it is needed only when the clones are
being created.  I'll do that too.

> > 
> >   /* Number of actual arguments in this callsite.  When set to 0,
> >      this callsite's parameters would not be analyzed by the different
> >      stages of IPA CP.  */
> >   int argument_count;
> >   /* Array of the callsite's jump function of each parameter.  */
> >   struct ipa_jump_func GTY ((length ("%h.argument_count"))) *jump_functions;
> 
> Would be prettier as VECtor.

The patch does not touch places which manipulate jump functions and
I'd prefer to keep it that way.

> 
> > } ipa_edge_args_t;
> > 
> > /* This function ensures the array of node param infos is big enough to
> >    accommodate a structure for all nodes and reallocates it if not.  */
> > 
> > static inline void
> > ipa_check_create_node_params (void)
> 
> this name never realy worked for me (i.e. what is it checking?).
> What about ipa_alloc_node_params or ipa_alloc_resize_node_params?
>

The idea behind the name was that it only allocates memory if
necessary.  But I'd be happy with any of the two suggestions.  But
again, preferably in a separate patch.
 
> 
> I realize that most of changes I sugested above are to the existing ipa-prop
> headers.  I would like to see them done, it depends what seems least painful
> for you - i.e. do them as part of your patch, do them beforehand or
> incrementally after the new ipa-cp is comitted to mainline.
> 
> In particular I would like to see the ipa-cp data (used at propagation stage)
> separated from function summaries (i.e. jump functions, param counts and decl/used
> info).

I will try to do this in this patch because I'm changing the
structures anyway.  Changing names of already existing things or
converting already existing arrays to vectors should wait.

> > /* Interprocedural constant propagation
> >    Copyright (C) 2005, 2006, 2007, 2008, 2009, 2010, 2011
> >    Free Software Foundation, Inc.
> >    Contributed by Razya Ladelsky <RAZYA@il.ibm.com>
> Since it is complette rewrite, I would add you there ;)

OK :-)

> > /* Interprocedural constant propagation (IPA-CP).
> > 
> >    The goal of this transformation is to
> > 
> >    1) discover functions which are always invoked with some arguments with the
> >       same known constant values and modify the functions so that the
> >       subsequent optimizations can take advantage of the knowledge, and
> > 
> >    2) create specialized versions of functions transformed in this way if
> >       some parameters are known constants only in certain contexts but the
> >       estimated tradeoff between speedup and cost size is deemed good.
> > 
> >    The algorithm also propagates types and attempts to perform type based
> >    devirtualization.  Types are propagated much like constants.
> 
> I would add that the algorithm is not doing only the traditional ipa-cp propagation,
> but also handles partial specialization and propagate known types
> for devirtualization.

I might call point 2 "partial specialization" but otherwise I believe
the comments above already state exactly that... do they not?

> >    First stage - intraprocedural analysis
> >    =======================================
> > 
> >    This phase computes jump_function and modification flags.
> > 
> >    A jump function for a call-site represents the values passed as an actual
> >    arguments of a given call-site. In principle, there are three types of
> >    values:
> > 
> >    Pass through - the caller's formal parameter is passed as an actual
> >                   argument, plus an operation on it can be performed.
> >    Constant - a constant is passed as an actual argument.
> >    Unknown - neither of the above.
> 
> Propbably should document the new fancy types of jump functions.
> Also mention that return functions are not handled, yet.

They are documented in ipa-prop.h and the comment refers the reader
there (just after the part quoted above).  I think it's better to
describe it at one place.

> > 
> >    Second stage - interprocedural analysis
> >    ========================================
> > 
> >    This stage is itself divided into two phases.  In the first, we propagate
> >    known values over the call graph, in the second, we make cloning decisions.
> 
> I would also explicitely mention that this stage is done differently from the
> classical ipa-cp paper.

OK. (But I reference two papers, we differ from both but much less
from the second one).

> > /* Return true iff the CS is an edge within a strongly connected component as
> >    computed by ipa_reduced_postorder.  */
> > 
> > static inline bool
> > edge_within_scc (struct cgraph_edge *cs)
> > {
> >   struct ipa_dfs_info *caller_dfs = (struct ipa_dfs_info *) cs->caller->aux;
> >   struct ipa_dfs_info *callee_dfs;
> >   struct cgraph_node *callee = cgraph_function_node (cs->callee, NULL);
> 
> since thunks are not transparent for you (and eventually we will want to have
> jump functions for those; passthrough in all but first argument and jump
> function representing the thunk adjustment), you want cgraph_function_or_thunk
> node.

True.

> > 
> > /* Print V which is extracted from a value in a lattice to F.  */
> > 
> > static void
> > print_ipcp_constant_value (FILE * f, tree v)
> > {
> >   if (TREE_CODE (v) == TREE_BINFO)
> >     {
> >       fprintf (f, "BINFO ");
> >       print_generic_expr (f, BINFO_TYPE (v), 0);
> 
> I wish we had some resonable way to visualise the BINFOs...
> It was always completely unclear what really is in them. But this is independent problem.
> > 
> > /* Determine whether it is at all technically possible to create clones of NODE
> >    and store this information in the ipa_node_params structure associated
> >    with NODE.  */
> > 
> > static void
> > determine_versionability (struct cgraph_node *node)
> > {
> >   struct cgraph_edge *edge;
> >   bool res = true;
> > 
> >   /* There are a number of generic reasons functions cannot be versioned.  We
> >      also cannot remove parameters if there are type attributes such as fnspec
> >      present.  */
> >   if (node->alias || node->thunk.thunk_p
> 
> We can version alias if its destination is versionable (i.e. get function_or_thunk node)
> also thunks should be versionable but they are not at the moment, inline summary will
> however have versionable flag cleared.

True.  I considered that but decided to allow their processing later
on, so that we can bisect potential problems to allowing that.

> 
> >       || !inline_summary (node)->versionable
> >       || TYPE_ATTRIBUTES (TREE_TYPE (node->decl))
> 
> This is getting an issue for Fortran that attach the function arguments quite commonly,
> perhaps we should start moving ahead on this and ruling out only the arguments that
> gets can't be preserved.

Yes, for example in 433.milc basically all the functions are
considered not versionable because of this.  I also thought of not
removing parameters for such functions.

> Also you need to test attributes of DECL itself.

Really?  We are not doing it now, nether for IPA-CP nor for IPA-SRA
(which is good at triggering problems with these).

> 
> I think this all should be part of can_change_signature_p code, since we can version
> with all attributes I can thunk of when we don't change signature.
> 
> >       || cgraph_function_body_availability (node) <= AVAIL_OVERWRITABLE)
> >     res =  false;
> >   else
> >     /* Removing arguments doesn't work if the function takes varargs
> >        or use __builtin_apply_args.
> >        FIXME: handle this together with can_change_signature flag.  */
> >     for (edge = node->callees; edge; edge = edge->next_callee)
> >       {
> > 	tree t = edge->callee->decl;
> > 	if (DECL_BUILT_IN_CLASS (t) == BUILT_IN_NORMAL
> > 	    && (DECL_FUNCTION_CODE (t) == BUILT_IN_APPLY_ARGS
> > 		|| DECL_FUNCTION_CODE (t) == BUILT_IN_VA_START))
> 
> can_change_sigunature will also handle apply_args.
> Add VA_START code there, too.  For the other use of this flag (in i386) VA_START
> rules out the change, too, but so far we explicitely tested that in the backend.
> 
> Iguess with these changes, inline_summary (node)->versionable should coincide
> with IPA_NODE_REF (node)->node_versionable making this whole code unnecesary
> (it was supposed to work this way, I am not sure why we now have to versionable
> flags).

That would be nice.  I was wondering why the difference between the
two.  Again, I am yet to decide whether this should be done as a
followup or within this change.

> > 
> > /* Worker callback of cgraph_for_node_and_aliases accumulating statistics of
> >    non-thunk incoming edges to NODE.  */
> > 
> > static bool
> > gather_caller_stats (struct cgraph_node *node, void *data)
> > {
> >   struct caller_statistics *stats = (struct caller_statistics *) data;
> >   struct cgraph_edge *cs;
> > 
> >   for (cs = node->callers; cs; cs = cs->next_caller)
> >     if (cs->caller->thunk.thunk_p)
> >       gather_caller_stats (cs->caller, stats);
> >     else
> >       {
> > 	stats->count_sum += cs->count;
> > 	stats->freq_sum += cs->frequency;
> > 	stats->n_calls++;
> > 	if (cgraph_maybe_hot_edge_p (cs))
> > 	  stats->n_hot_calls ++;
> >       }
> >   return false;


> Why you compute count_sum instead of using node->count?  Is it because you are
> interested in local calls only?

I'm only interested in known calls, that I can redirect, i.e. in "local"
direct ones.

> >   if (!optimize_function_for_speed_p (DECL_STRUCT_FUNCTION (node->decl)))
> 
> When optimizing for size, we could still consider clonning the function if amount
> of code saved at call sites is bigger than the function body..
> Not really important however.

And we do it if the parameters are constants in all contexts.

> >     {
> >       if (dump_file)
> >         fprintf (dump_file, "Not considering %s for cloning; "
> > 		 "optimizing it for size.\n",
> >  	         cgraph_node_name (node));
> >       return false;
> >     }
> > 
> >   init_caller_stats (&stats);
> >   cgraph_for_node_and_aliases (node, gather_caller_stats, &stats, false);
> If you use for_node_thunk_and_aliases you can remove the recursion you do above
> and it will work for aliases of thunk correctly, too ;)

But I wouldn't be able to check the edge for hotness.

> > /* Allocate the arrays in TOPO and topologically sort the nodes into order.  */
> > 
> > static void
> > build_topo_info (struct topo_info *topo)
> toporder is probably more clear about what it means ;)

OK

> > /* Initialize ipcp_lattices.  */
> > 
> > static void
> > initialize_node_lattices (struct cgraph_node *node)
> > {
> >   struct ipa_node_params *info = IPA_NODE_REF (node);
> >   struct cgraph_edge *ie;
> >   bool disable = false, variable = false;
> >   int i;
> > 
> >   /* FIXME: Can we clobber only the first argument of thunks?  */
> >   if (node->alias || node->thunk.thunk_p
> 
> I think sane thing to do here is to not have latices for aliases, since you skip
> them during propagatoin, but have latices for thinks, since they should have their
> own jump functions.

After they are separate, we may try to not have them at all.

> 
> You can probably add sanity check into lattice accesor function that
> is is never given an alias and skip the code disabling ipa-cp bellow
> leaving the data uninitialized.
> 
> >       || ipa_is_called_with_var_arguments (info))
> 
> I always wondered if we need this, given that we check for use of
> stdarg.  Sure writing vararg function without actually using the
> arguments is sort of stupid, but it seems that the code should just
> work.

I have seen ICEs because of this when the user passed an incorrect
number of parameters even though the functions was not vararg.  And I
think that wasn't even with LTO.  But a flag saying "just ignore this
for whatever unspecified reason" might be better, yes.

> 
> (at least with one bounds check in propagation for case function is called with
> too few argumetns)

I don't understand the above.

> 
> Also naturally propagating into stdarg functions is possible.
> 

Sure.  But not really important to worry about it now, is it?

Thanks a lot.  I'll try to coma up with the structure reorganization
soon.

Martin
Jan Hubicka July 8, 2011, 6:24 p.m. UTC | #3
> > > /* Structure holding data required to describe a pass-through jump function.  */
> > > 
> > > struct GTY(()) ipa_pass_through_data
> > > {
> > >   /* If an operation is to be performed on the original parameter, this is the
> > >      second (constant) operand.  */
> > >   tree operand;
> > >   /* Number of the caller's formal parameter being passed.  */
> > >   int formal_id;
> > 
> > I probably should use this in ipa-inline-analsysi where I call it for some reason operand_num :)
> 
> So far I have resisted the urge to rename this but it pre-dates my
> involvement with gcc.  I'd like it to be called parm_index but since
> we might want to use it also for global variables and we might need
> something more complex for also handling parts of aggregates, I left
> it for later.

parm_index sounds to me good, too.  formal_id is the name used in paper
so it makes sense, but its meaning is unobvious.
We could rename it later, together with ipa-inline-analysis one
(I think consistency here is more important).

Yep, we will have to see how the jfuncs will look like once they
reffer to global vars and parts of agregates..
> 
> > 
> > > 
> > > struct ipcp_value;
> > 
> > I wonder if the jump functions used by several passes and ipcp
> > internal types both has o go into the same header?
> 
> Well, I originally wanted ipa-prop to provide services to the outside
> world like ipa-inline-analysis.c and to have ipa-cp self-contained.
> But I guess the data separation is more important.  So if we move
> ipa_cst_from_jfunc (and its 3 friends) to ipa-cp and move
> ipcp_lattice.decl and ipcp_lattice.used to a special structure, I
> might move ipcp_value_source, ipcp_value, and ipcp_lattice altogether
> to ipa-cp.

I also think that having ipa-prop as a generic module for propagating&
looking into args is the goal.
Separating datastructures form ipa-cp definitely makes this more obvious.

I've done that to the inliner, too. inline_summary is now all about the
function body size/time estimates and the inliner heuristics do have
their own datastructures in their own space.
> 
> At the moment I'm not sure whether I want to do this as a followup
> patch or incorporate it in the changes.  I think I'll start with the
> latter and revert to the former if it is too invasive to parts which
> have not been touched so far by the change.

Lets see, I am happy with the patch going in with current organization of
datastructures if we move into privatizing it later.
> > > {
> > >   /* Pointer to an array of structures describing individual formal
> > >      parameters.  */
> > >   struct ipcp_lattice *lattices;
> > 
> > Hmm, how we get here around the need to mark this GTY(). I.e are we sure that all the known_vals
> > must be referneced from elsewhere at ggc time?
> 
> (Scalar) constants that are results of arithmetic jump functions may
> not be referenced from elsewhere, everything else is referenced from
> the jump functions.  If it is a problem it is already present in the
> current IPA-CP.  ipa_node_params and lattices are not GTYed there
> either.

Hmm, I guess it is not really problem only because the lattices are used
only in ipa-cp so the values do not really live across GGC call.

Well, this will be solved by separating out the ipa-cp datastructures, so
it is not a problem.
> 
> > I would also slowly switch those things to VECtors..
> 
> Perhaps, but individual lattices are and always have been accessed
> through ipa_get_lattice which checks bounds and so there's no big
> reason to do that.

Yep, I added the bounds check when I was debugging.  Not big deal,
just we sort of do have agreement using our VECtor API where it fits..
> 
> > 
> > >   /* Only for versioned nodes this field would not be NULL,
> > >      it points to the node that IPA cp cloned from.  */
> > >   struct cgraph_node *ipcp_orig_node;
> > Why not use node->clone_of here?
> 
> That would not work if the node was a clone created by some other
> pass.  I need to differentiate between clones I create because they do
> not have lattices but do have the exact values for individual
> parameters in known_vals (which is NULL otherwise).  I should probably
> use a flag though, the code I ended up only checks it for NULL anyway.

Hmm, OK, either flag or keeping the pointer is fine.
> > > 
> > > /* ipa_edge_args stores information related to a callsite and particularly its
> > >    arguments.  It can be accessed by the IPA_EDGE_REF macro.  */
> > > typedef struct GTY(()) ipa_edge_args
> > 
> > probably edge_summary would be my preferred name.
> 
> Ugh, this is the current name, we may change it later.  In any event
> the name should probably tell that the summary is about parameters.

Hmm, OK, it is not bad name after all.
> 
> > 
> > > {
> > >   /* Next pointer in a linked list of clones of the same function.  */
> > >   struct cgraph_edge *next_edge_clone;
> > 
> > What this is needed for?
> 
> For get_info_about_necessary_edges and gather_edges_for_value.  If a
> value comes from a node and that is itself cloned, it now may come
> from two nodes and so when I iterate over potential sources, I have to
> include all such edges.  This is their linked list.

OK, I see.
> 
> This field could be put into a separate structure because it is not
> really part of the summary, it is needed only when the clones are
> being created.  I'll do that too.
> 
> > > 
> > >   /* Number of actual arguments in this callsite.  When set to 0,
> > >      this callsite's parameters would not be analyzed by the different
> > >      stages of IPA CP.  */
> > >   int argument_count;
> > >   /* Array of the callsite's jump function of each parameter.  */
> > >   struct ipa_jump_func GTY ((length ("%h.argument_count"))) *jump_functions;
> > 
> > Would be prettier as VECtor.
> 
> The patch does not touch places which manipulate jump functions and
> I'd prefer to keep it that way.

Yep, lets do that incrementally.
> > >    The goal of this transformation is to
> > > 
> > >    1) discover functions which are always invoked with some arguments with the
> > >       same known constant values and modify the functions so that the
> > >       subsequent optimizations can take advantage of the knowledge, and
> > > 
> > >    2) create specialized versions of functions transformed in this way if
> > >       some parameters are known constants only in certain contexts but the
> > >       estimated tradeoff between speedup and cost size is deemed good.
> > > 
> > >    The algorithm also propagates types and attempts to perform type based
> > >    devirtualization.  Types are propagated much like constants.
> > 
> > I would add that the algorithm is not doing only the traditional ipa-cp propagation,
> > but also handles partial specialization and propagate known types
> > for devirtualization.
> 
> I might call point 2 "partial specialization" but otherwise I believe
> the comments above already state exactly that... do they not?

Hmm, I guess it does..
> > 
> > >       || !inline_summary (node)->versionable
> > >       || TYPE_ATTRIBUTES (TREE_TYPE (node->decl))
> > 
> > This is getting an issue for Fortran that attach the function arguments quite commonly,
> > perhaps we should start moving ahead on this and ruling out only the arguments that
> > gets can't be preserved.
> 
> Yes, for example in 433.milc basically all the functions are
> considered not versionable because of this.  I also thought of not
> removing parameters for such functions.
> 
> > Also you need to test attributes of DECL itself.
> 
> Really?  We are not doing it now, nether for IPA-CP nor for IPA-SRA
> (which is good at triggering problems with these).

Hmm, all the ipa-inline code checks DECL_ATTRIBUTES only.  I believe the type
attributes ends up being copied to decl attributes but not vice versa.
I guess the testcase should be

__attribute__ ((really_bad_attribute))
function (int param)
{
  use param
}

and then let ipa-cp to invent param is a constant.
> 
> > 
> > I think this all should be part of can_change_signature_p code, since we can version
> > with all attributes I can thunk of when we don't change signature.
> > 
> > >       || cgraph_function_body_availability (node) <= AVAIL_OVERWRITABLE)
> > >     res =  false;
> > >   else
> > >     /* Removing arguments doesn't work if the function takes varargs
> > >        or use __builtin_apply_args.
> > >        FIXME: handle this together with can_change_signature flag.  */
> > >     for (edge = node->callees; edge; edge = edge->next_callee)
> > >       {
> > > 	tree t = edge->callee->decl;
> > > 	if (DECL_BUILT_IN_CLASS (t) == BUILT_IN_NORMAL
> > > 	    && (DECL_FUNCTION_CODE (t) == BUILT_IN_APPLY_ARGS
> > > 		|| DECL_FUNCTION_CODE (t) == BUILT_IN_VA_START))
> > 
> > can_change_sigunature will also handle apply_args.
> > Add VA_START code there, too.  For the other use of this flag (in i386) VA_START
> > rules out the change, too, but so far we explicitely tested that in the backend.
> > 
> > Iguess with these changes, inline_summary (node)->versionable should coincide
> > with IPA_NODE_REF (node)->node_versionable making this whole code unnecesary
> > (it was supposed to work this way, I am not sure why we now have to versionable
> > flags).
> 
> That would be nice.  I was wondering why the difference between the
> two.  Again, I am yet to decide whether this should be done as a
> followup or within this change.

OK.  
> > Why you compute count_sum instead of using node->count?  Is it because you are
> > interested in local calls only?
> 
> I'm only interested in known calls, that I can redirect, i.e. in "local"
> direct ones.
OK
> > If you use for_node_thunk_and_aliases you can remove the recursion you do above
> > and it will work for aliases of thunk correctly, too ;)
> 
> But I wouldn't be able to check the edge for hotness.

BTW currently the edges from thunks miss any profile info.
(i.e. it will be 0). I guess we ought to make ipa-profile pass to estimate those
(it is difficult ot measure count of an alias).

If you walk only hot edges, then you need to make your function descent into
both alias refs and thunks calls, or the aliases of thunks will not be seen
then.
> > I always wondered if we need this, given that we check for use of
> > stdarg.  Sure writing vararg function without actually using the
> > arguments is sort of stupid, but it seems that the code should just
> > work.
> 
> I have seen ICEs because of this when the user passed an incorrect
> number of parameters even though the functions was not vararg.  And I
> think that wasn't even with LTO.  But a flag saying "just ignore this
> for whatever unspecified reason" might be better, yes.

Yep, K&R programs can mismatch parameters.  In ipa-inline I however simply bound
check and assume nothing on the missing parameter. I think you can easilly do
that in the propagation stage, too.
> 
> > 
> > (at least with one bounds check in propagation for case function is called with
> > too few argumetns)
> 
> I don't understand the above.
> 
> > 
> > Also naturally propagating into stdarg functions is possible.
> > 
> 
> Sure.  But not really important to worry about it now, is it?
Yep, not big deal.

Honza
> 
> Thanks a lot.  I'll try to coma up with the structure reorganization
> soon.
> 
> Martin
Jan Hubicka July 10, 2011, 5:04 p.m. UTC | #4
> 
> /* If checking is enabled, verify that no lattice is in the TOP state, i.e. not
>    bottom, not containing a variable component and without any known value at
>    the same time.  */
> 
> static void
> verify_propagated_values (void)
> {
> #ifdef ENABLE_CHECKING

Hmm, would not be better to keep this function around to be called from debugger, like
other verifiers do?
>   struct cgraph_node *node;
> 
>   FOR_EACH_DEFINED_FUNCTION (node)
>     {
>       struct ipa_node_params *info = IPA_NODE_REF (node);
>       int i, count = ipa_get_param_count (info);
> 
>       for (i = 0; i < count; i++)
> 	{
> 	  struct ipcp_lattice *lat = ipa_get_lattice (info, i);
> 
> 	  if (!lat->bottom
> 	      && !lat->contains_variable
> 	      && lat->values_count == 0)
> 	    {
> 	      if (dump_file)
> 		{
> 		  fprintf (dump_file, "\nIPA lattices after constant "
> 			   "propagation:\n");
> 		  print_all_lattices (dump_file, true, false);
> 		}
> 
> 	      gcc_unreachable ();
> 	    }
> 	}
>     }
> #endif
> }
> 
> /* Propagate values through a pass-through jump function JFUNC associated with
>    edge CS, taking values from SRC_LAT and putting them into DEST_LAT.  SRC_IDX
>    is the index of the source parameter.  */
> 
> static bool
> propagate_vals_accross_pass_through (struct cgraph_edge *cs,
> 				     struct ipa_jump_func *jfunc,
> 				     struct ipcp_lattice *src_lat,
> 				     struct ipcp_lattice *dest_lat,
> 				     int src_idx)
> {
>   struct ipcp_value *src_val;
>   bool ret = false;
> 
>   if (jfunc->value.pass_through.operation == NOP_EXPR)
>     for (src_val = src_lat->values; src_val; src_val = src_val->next)
>       ret |= add_value_to_lattice (dest_lat, src_val->value, cs,
> 				   src_val, src_idx);
>   else if (edge_within_scc (cs))
>     ret = set_lattice_contains_variable (dest_lat);

Hmm, is the reason for not using artimetic within SCC documented somewhere?
It seems like code someone will eventually run into and remove without much of consideration.
> 
> /* Calculate devirtualization time bonus for NODE, assuming we know KNOWN_CSTS
>    and KNOWN_BINFOS.  */
> 
> static int
> devirtualization_time_bonus (struct cgraph_node *node,
> 			     VEC (tree, heap) *known_csts,
> 			     VEC (tree, heap) *known_binfos)

Eventually it would be nice to make this common with inliner analysis.  We want to 
increase inlining limits here too....
>       /* Only bare minimum benefit for clearly un-inlineable targets.  */
>       res += 1;
>       callee = cgraph_get_node (target);
>       if (!callee)
> 	continue;

Hmm, when you test it here, you might probably ask if callee is analyzed and inlinable then ;)
> 
> /* Return true if cloning NODE is a good idea, given the estimated TIME_BENEFIT
>    and SIZE_COST and with the sum of frequencies of incoming edges to the
>    potential new clone in FREQUENCIES.  */
> 
> static bool
> good_cloning_opportunity_p (struct cgraph_node *node, int time_benefit,
> 			    int freq_sum, gcov_type count_sum, int size_cost)
> {
>   if (time_benefit == 0
>       || !flag_ipa_cp_clone
>       || !optimize_function_for_speed_p (DECL_STRUCT_FUNCTION (node->decl)))
>     return false;

Still I think cloning oppurtunity is good if the saving on call size reduce enough
to pay back for function body duplication.
It would probably make sense then to create simple wrapper functions instead of
duplicating the function body.

> 
>   gcc_checking_assert (size_cost >= 0);
> 
>   /* FIXME:  These decisions need tuning.  */
>   if (max_count)
>     {
>       int evaluation, factor = (count_sum * 1000) / max_count;
> 
>       evaluation = (time_benefit * factor) / size_cost;
> 
>       if (dump_file && (dump_flags & TDF_DETAILS))
> 	fprintf (dump_file, "     good_cloning_opportunity_p (time: %i, "
> 		 "size: %i, count_sum: " HOST_WIDE_INT_PRINT_DEC
> 		 ") -> evaluation: %i, threshold: %i\n",
> 		 time_benefit, size_cost, (HOST_WIDE_INT) count_sum,
> 		 evaluation, 500);
> 
>       return evaluation > 500;

Hmm, the magic numbers looks a bit scary... putting size and time into fraction
causes problem that the units are not really related.

But we will see.  I guess we will need the 500 as --param value however.

We probably want ipa-profile to collect expected running time of the unit
and then we can do something like computing relative speedup of unit versus
relative growth of it that is sort of better defined  and closer to
temperature metrics. (but of course off reality)
>     }
>   else
>     {
>       int evaluation = (time_benefit * freq_sum) / size_cost;
> 
>       if (dump_file && (dump_flags & TDF_DETAILS))
> 	fprintf (dump_file, "     good_cloning_opportunity_p (time: %i, "
> 		 "size: %i, freq_sum: %i) -> evaluation: %i, threshold: %i\n",
> 		 time_benefit, size_cost, freq_sum, evaluation,
> 		 CGRAPH_FREQ_BASE /2);
> 
>       return evaluation > (CGRAPH_FREQ_BASE /2);

Similarly here, the fraction is quite crazy, but don't have much ideas how to
handle it better.

>       init_caller_stats (&stats);
>       cgraph_for_node_and_aliases (node, gather_caller_stats, &stats, false);
>       estimate_ipcp_clone_size_and_time (node, known_csts, &size, &time);
>       time -= devirtualization_time_bonus (node, known_csts, known_binfos);
>       size -= stats.n_calls * removable_params;

You may probably use stame logic as when we estimate call size.
time also benefits from caller getting smaller... so I think you should subtract
sum of move costs of the removable params.
> 
> /* Update the respective profile of specialized NEW_NODE and the original
>    ORIG_NODE after additional edges with cumulative count sum REDIRECTED_SUM
>    have been redirected to the specialized version.  */
> 
> static void
> update_specialized_profile (struct cgraph_node *new_node,
> 			    struct cgraph_node *orig_node,
> 			    gcov_type redirected_sum)
> {
>   struct cgraph_edge *cs;
>   gcov_type new_node_count, orig_node_count = orig_node->count;
> 
>   if (dump_file)
>     fprintf (dump_file, "    the sum of counts of redirected  edges is "
> 	     HOST_WIDE_INT_PRINT_DEC "\n", (HOST_WIDE_INT) redirected_sum);
>   if (orig_node_count == 0)
>     return;
> 
>   gcc_assert (orig_node_count >= redirected_sum);
> 
>   new_node_count = new_node->count;
>   new_node->count += redirected_sum;
>   orig_node->count -= redirected_sum;
> 
>   for (cs = new_node->callees; cs ; cs = cs->next_callee)
>     if (cs->frequency)
>       cs->count += cs->count * redirected_sum / new_node_count;
>     else
>       cs->count = 0;
> 
>   for (cs = orig_node->callees; cs ; cs = cs->next_callee)
>     {
>       gcov_type dec = cs->count * redirected_sum / orig_node_count;
Are you sure we can't divide by 0 here or get to negative numbers for insane profiles?
>       if (dec < cs->count)
> 	cs->count -= dec;
>       else
> 	cs->count = 0;
>     }
> 
>   if (dump_file)
>     dump_profile_updates (orig_node, new_node);
> }
> 
>   if (node->local.can_change_signature)
>     {
>       args_to_skip = BITMAP_GGC_ALLOC ();
>       for (i = 0; i < count; i++)
> 	{
> 	  tree t = VEC_index (tree, known_vals, i);
> 
> 	  if ((t && TREE_CODE (t) != TREE_BINFO)
> 	      || !ipa_is_param_used (info, i))
> 	    bitmap_set_bit (args_to_skip, i);
> 	}
>     }
>   else
>     args_to_skip = NULL;
When we can't change signature, still we can set is_parm_unused flag for the callee
to aid later optimizers.

Rest of patch looks OK. It definitely reads better than previous ipa-cp.c ;)
I suppose we will need to get some experience with the logic deciding whether to clone..
Honza
Martin Jambor July 14, 2011, 1:46 p.m. UTC | #5
Hi,

I'll send a new version of IPA-CP incorporating most of the feedback
in a new thread but let me also comment on some of the points here:

On Fri, Jul 08, 2011 at 08:24:31PM +0200, Jan Hubicka wrote:
> > > > {
> > > >   /* Pointer to an array of structures describing individual formal
> > > >      parameters.  */
> > > >   struct ipcp_lattice *lattices;
> > > 
> > > Hmm, how we get here around the need to mark this GTY(). I.e are we sure that all the known_vals
> > > must be referneced from elsewhere at ggc time?
> > 
> > (Scalar) constants that are results of arithmetic jump functions may
> > not be referenced from elsewhere, everything else is referenced from
> > the jump functions.  If it is a problem it is already present in the
> > current IPA-CP.  ipa_node_params and lattices are not GTYed there
> > either.
> 
> Hmm, I guess it is not really problem only because the lattices are used
> only in ipa-cp so the values do not really live across GGC call.

Well, technically they survive until after inlining (because of
indirect inlining which also derives information from the lattices
corresponding to node->inlined_to node.  Results of arithmetic
functions are not going to be accessed during inlining when compiling
any reasonable program but...

> > > > 
> > > > /* ipa_edge_args stores information related to a callsite and particularly its
> > > >    arguments.  It can be accessed by the IPA_EDGE_REF macro.  */
> > > > typedef struct GTY(()) ipa_edge_args
> > > 
> > > probably edge_summary would be my preferred name.
> > 
> > Ugh, this is the current name, we may change it later.  In any event
> > the name should probably tell that the summary is about parameters.
> 
> Hmm, OK, it is not bad name after all.

:-)

> > > 
> > > >       || !inline_summary (node)->versionable
> > > >       || TYPE_ATTRIBUTES (TREE_TYPE (node->decl))
> > > 
> > > This is getting an issue for Fortran that attach the function arguments quite commonly,
> > > perhaps we should start moving ahead on this and ruling out only the arguments that
> > > gets can't be preserved.
> > 
> > Yes, for example in 433.milc basically all the functions are
> > considered not versionable because of this.  I also thought of not
> > removing parameters for such functions.
> > 
> > > Also you need to test attributes of DECL itself.
> > 
> > Really?  We are not doing it now, nether for IPA-CP nor for IPA-SRA
> > (which is good at triggering problems with these).
> 
> Hmm, all the ipa-inline code checks DECL_ATTRIBUTES only.  I believe the type
> attributes ends up being copied to decl attributes but not vice versa.
> I guess the testcase should be
> 
> __attribute__ ((really_bad_attribute))
> function (int param)
> {
>   use param
> }
> 
> and then let ipa-cp to invent param is a constant.

what would be such a "really_bad_attribute" ? 

> > 
> > > 
> > > I think this all should be part of can_change_signature_p code, since we can version
> > > with all attributes I can thunk of when we don't change signature.
> > > 
> > > >       || cgraph_function_body_availability (node) <= AVAIL_OVERWRITABLE)
> > > >     res =  false;
> > > >   else
> > > >     /* Removing arguments doesn't work if the function takes varargs
> > > >        or use __builtin_apply_args.
> > > >        FIXME: handle this together with can_change_signature flag.  */
> > > >     for (edge = node->callees; edge; edge = edge->next_callee)
> > > >       {
> > > > 	tree t = edge->callee->decl;
> > > > 	if (DECL_BUILT_IN_CLASS (t) == BUILT_IN_NORMAL
> > > > 	    && (DECL_FUNCTION_CODE (t) == BUILT_IN_APPLY_ARGS
> > > > 		|| DECL_FUNCTION_CODE (t) == BUILT_IN_VA_START))
> > > 
> > > can_change_sigunature will also handle apply_args.
> > > Add VA_START code there, too.  For the other use of this flag (in i386) VA_START

The last one already is VA_START... or do you mean a different one?

> > > rules out the change, too, but so far we explicitely tested that in the backend.
> > > 
> > > Iguess with these changes, inline_summary (node)->versionable should coincide
> > > with IPA_NODE_REF (node)->node_versionable making this whole code unnecesary
> > > (it was supposed to work this way, I am not sure why we now have to versionable
> > > flags).
> > 
> > That would be nice.  I was wondering why the difference between the
> > two.  Again, I am yet to decide whether this should be done as a
> > followup or within this change.
> 
> OK.  

BTW, it will be a followup change.

> > > If you use for_node_thunk_and_aliases you can remove the recursion you do above
> > > and it will work for aliases of thunk correctly, too ;)
> > 
> > But I wouldn't be able to check the edge for hotness.
> 
> BTW currently the edges from thunks miss any profile info.
> (i.e. it will be 0). I guess we ought to make ipa-profile pass to estimate those
> (it is difficult ot measure count of an alias).
> 

I'm not really looking at the edges from thunks to the actual
function.  OTOH, I assume that edges to a thunk do have a count and
look at that.

> If you walk only hot edges, then you need to make your function descent into
> both alias refs and thunks calls, or the aliases of thunks will not be seen
> then.

Well, looking at bits of the patch again now, aliases to thunks might
indeed be a problem for a few pieces of it.  I'll send the patch
nevertheless and ponder about this problem later.

Thanks,

Martin
Martin Jambor July 14, 2011, 3:31 p.m. UTC | #6
Hi,

like with the previous mail, I'll reply to the comments here and then
post a new version of the patch in a separate thread.

On Sun, Jul 10, 2011 at 07:04:21PM +0200, Jan Hubicka wrote:
> > 
> > /* If checking is enabled, verify that no lattice is in the TOP state, i.e. not
> >    bottom, not containing a variable component and without any known value at
> >    the same time.  */
> > 
> > static void
> > verify_propagated_values (void)
> > {
> > #ifdef ENABLE_CHECKING
> 
> Hmm, would not be better to keep this function around to be called from debugger, like
> other verifiers do?

OK.

> >   struct cgraph_node *node;
> > 
> >   FOR_EACH_DEFINED_FUNCTION (node)
> >     {
> >       struct ipa_node_params *info = IPA_NODE_REF (node);
> >       int i, count = ipa_get_param_count (info);
> > 
> >       for (i = 0; i < count; i++)
> > 	{
> > 	  struct ipcp_lattice *lat = ipa_get_lattice (info, i);
> > 
> > 	  if (!lat->bottom
> > 	      && !lat->contains_variable
> > 	      && lat->values_count == 0)
> > 	    {
> > 	      if (dump_file)
> > 		{
> > 		  fprintf (dump_file, "\nIPA lattices after constant "
> > 			   "propagation:\n");
> > 		  print_all_lattices (dump_file, true, false);
> > 		}
> > 
> > 	      gcc_unreachable ();
> > 	    }
> > 	}
> >     }
> > #endif
> > }
> > 
> > /* Propagate values through a pass-through jump function JFUNC associated with
> >    edge CS, taking values from SRC_LAT and putting them into DEST_LAT.  SRC_IDX
> >    is the index of the source parameter.  */
> > 
> > static bool
> > propagate_vals_accross_pass_through (struct cgraph_edge *cs,
> > 				     struct ipa_jump_func *jfunc,
> > 				     struct ipcp_lattice *src_lat,
> > 				     struct ipcp_lattice *dest_lat,
> > 				     int src_idx)
> > {
> >   struct ipcp_value *src_val;
> >   bool ret = false;
> > 
> >   if (jfunc->value.pass_through.operation == NOP_EXPR)
> >     for (src_val = src_lat->values; src_val; src_val = src_val->next)
> >       ret |= add_value_to_lattice (dest_lat, src_val->value, cs,
> > 				   src_val, src_idx);
> >   else if (edge_within_scc (cs))
> >     ret = set_lattice_contains_variable (dest_lat);
> 
> Hmm, is the reason for not using artimetic within SCC documented somewhere?
> It seems like code someone will eventually run into and remove without much of consideration.

I added a comment.

> > 
> > /* Calculate devirtualization time bonus for NODE, assuming we know KNOWN_CSTS
> >    and KNOWN_BINFOS.  */
> > 
> > static int
> > devirtualization_time_bonus (struct cgraph_node *node,
> > 			     VEC (tree, heap) *known_csts,
> > 			     VEC (tree, heap) *known_binfos)
> 
> Eventually it would be nice to make this common with inliner analysis.  We want to 
> increase inlining limits here too....

True.

> >       /* Only bare minimum benefit for clearly un-inlineable targets.  */
> >       res += 1;
> >       callee = cgraph_get_node (target);
> >       if (!callee)
> > 	continue;
> 
> Hmm, when you test it here, you might probably ask if callee is analyzed and inlinable then ;)

Good point, did that.

> > 
> > /* Return true if cloning NODE is a good idea, given the estimated TIME_BENEFIT
> >    and SIZE_COST and with the sum of frequencies of incoming edges to the
> >    potential new clone in FREQUENCIES.  */
> > 
> > static bool
> > good_cloning_opportunity_p (struct cgraph_node *node, int time_benefit,
> > 			    int freq_sum, gcov_type count_sum, int size_cost)
> > {
> >   if (time_benefit == 0
> >       || !flag_ipa_cp_clone
> >       || !optimize_function_for_speed_p (DECL_STRUCT_FUNCTION (node->decl)))
> >     return false;
> 
> Still I think cloning oppurtunity is good if the saving on call size reduce enough
> to pay back for function body duplication.
> It would probably make sense then to create simple wrapper functions instead of
> duplicating the function body.

I was already doing that for considering IPA-CP opportunities for all
known contexts - and the effect is more profound still now when we
consider moving costs.

On the other side, when opportunities for function specialization are
concerned, size is always considered a cost and never a benefit when
doing the calculations.  This simplifies a the code a bit because the
size savings are on the side of the caller and so it depends on the
number of call sites that actually get redirected.  That is not known
when estimating size but only we did specialization decisions for all
the callers and so it would need to be an extra thing stored along
values.  I think that since such calls should be picked up by
inlining, it's not really worth the added data and complexity.  But I
could add it later, sure.

> 
> > 
> >   gcc_checking_assert (size_cost >= 0);
> > 
> >   /* FIXME:  These decisions need tuning.  */
> >   if (max_count)
> >     {
> >       int evaluation, factor = (count_sum * 1000) / max_count;
> > 
> >       evaluation = (time_benefit * factor) / size_cost;
> > 
> >       if (dump_file && (dump_flags & TDF_DETAILS))
> > 	fprintf (dump_file, "     good_cloning_opportunity_p (time: %i, "
> > 		 "size: %i, count_sum: " HOST_WIDE_INT_PRINT_DEC
> > 		 ") -> evaluation: %i, threshold: %i\n",
> > 		 time_benefit, size_cost, (HOST_WIDE_INT) count_sum,
> > 		 evaluation, 500);
> > 
> >       return evaluation > 500;
> 
> Hmm, the magic numbers looks a bit scary... putting size and time into fraction
> causes problem that the units are not really related.
> 
> But we will see.  I guess we will need the 500 as --param value
> however.

I made it a parameter.  Because both the condition for profiling and
for profile estimates has the same 500 value as threshold, I added
only one for both.  We may want to split it up into two separate ones.

> 
> We probably want ipa-profile to collect expected running time of the unit
> and then we can do something like computing relative speedup of unit versus
> relative growth of it that is sort of better defined  and closer to
> temperature metrics. (but of course off reality)

Still better than what the attempts now.

> >     }
> >   else
> >     {
> >       int evaluation = (time_benefit * freq_sum) / size_cost;
> > 
> >       if (dump_file && (dump_flags & TDF_DETAILS))
> > 	fprintf (dump_file, "     good_cloning_opportunity_p (time: %i, "
> > 		 "size: %i, freq_sum: %i) -> evaluation: %i, threshold: %i\n",
> > 		 time_benefit, size_cost, freq_sum, evaluation,
> > 		 CGRAPH_FREQ_BASE /2);
> > 
> >       return evaluation > (CGRAPH_FREQ_BASE /2);
> 
> Similarly here, the fraction is quite crazy, but don't have much ideas how to
> handle it better.

It is, however despite I have spent quite a lot of time looking for
what it gets wrong and trying some modifications but did not really
come up with anything very useful.  Perhaps I'll want to multiply the
size by some small factor(s) for large functions (still a fair bit
larger than the small function inlining limit).

> 
> >       init_caller_stats (&stats);
> >       cgraph_for_node_and_aliases (node, gather_caller_stats, &stats, false);
> >       estimate_ipcp_clone_size_and_time (node, known_csts, &size, &time);
> >       time -= devirtualization_time_bonus (node, known_csts, known_binfos);
> >       size -= stats.n_calls * removable_params;
> 
> You may probably use stame logic as when we estimate call size.
> time also benefits from caller getting smaller... so I think you should subtract
> sum of move costs of the removable params.

OK, I did that, I hope I got it right, please re-check though.

> > 
> > /* Update the respective profile of specialized NEW_NODE and the original
> >    ORIG_NODE after additional edges with cumulative count sum REDIRECTED_SUM
> >    have been redirected to the specialized version.  */
> > 
> > static void
> > update_specialized_profile (struct cgraph_node *new_node,
> > 			    struct cgraph_node *orig_node,
> > 			    gcov_type redirected_sum)
> > {
> >   struct cgraph_edge *cs;
> >   gcov_type new_node_count, orig_node_count = orig_node->count;
> > 
> >   if (dump_file)
> >     fprintf (dump_file, "    the sum of counts of redirected  edges is "
> > 	     HOST_WIDE_INT_PRINT_DEC "\n", (HOST_WIDE_INT) redirected_sum);
> >   if (orig_node_count == 0)
> >     return;
> > 
> >   gcc_assert (orig_node_count >= redirected_sum);
> > 
> >   new_node_count = new_node->count;
> >   new_node->count += redirected_sum;
> >   orig_node->count -= redirected_sum;
> > 
> >   for (cs = new_node->callees; cs ; cs = cs->next_callee)
> >     if (cs->frequency)
> >       cs->count += cs->count * redirected_sum / new_node_count;
> >     else
> >       cs->count = 0;
> > 
> >   for (cs = orig_node->callees; cs ; cs = cs->next_callee)
> >     {
> >       gcov_type dec = cs->count * redirected_sum / orig_node_count;
> Are you sure we can't divide by 0 here or get to negative numbers for insane profiles?

A few lines above there is a check for (orig_node_count == 0).  As far
as insane profiles are concerned, these are identified and dealt with
when specialized clones are first created and should not be a problem
any more when additional callers are redirected.

> >       if (dec < cs->count)
> > 	cs->count -= dec;
> >       else
> > 	cs->count = 0;
> >     }
> > 
> >   if (dump_file)
> >     dump_profile_updates (orig_node, new_node);
> > }
> > 
> >   if (node->local.can_change_signature)
> >     {
> >       args_to_skip = BITMAP_GGC_ALLOC ();
> >       for (i = 0; i < count; i++)
> > 	{
> > 	  tree t = VEC_index (tree, known_vals, i);
> > 
> > 	  if ((t && TREE_CODE (t) != TREE_BINFO)
> > 	      || !ipa_is_param_used (info, i))
> > 	    bitmap_set_bit (args_to_skip, i);
> > 	}
> >     }
> >   else
> >     args_to_skip = NULL;
> When we can't change signature, still we can set is_parm_unused flag for the callee
> to aid later optimizers.

I assume I can re-use the node->local.can_change_signature flag?  Is
that supposed to be set at any given place or can IPA-CP do it on its own?

> 
> Rest of patch looks OK. It definitely reads better than previous ipa-cp.c ;)
> I suppose we will need to get some experience with the logic deciding whether to clone..


Thanks, I'll post the current version in a moment.

Martin
Jan Hubicka July 14, 2011, 4:11 p.m. UTC | #7
> > >       if (dec < cs->count)
> > > 	cs->count -= dec;
> > >       else
> > > 	cs->count = 0;
> > >     }
> > > 
> > >   if (dump_file)
> > >     dump_profile_updates (orig_node, new_node);
> > > }
> > > 
> > >   if (node->local.can_change_signature)
> > >     {
> > >       args_to_skip = BITMAP_GGC_ALLOC ();
> > >       for (i = 0; i < count; i++)
> > > 	{
> > > 	  tree t = VEC_index (tree, known_vals, i);
> > > 
> > > 	  if ((t && TREE_CODE (t) != TREE_BINFO)
> > > 	      || !ipa_is_param_used (info, i))
> > > 	    bitmap_set_bit (args_to_skip, i);
> > > 	}
> > >     }
> > >   else
> > >     args_to_skip = NULL;
> > When we can't change signature, still we can set is_parm_unused flag for the callee
> > to aid later optimizers.
> 
> I assume I can re-use the node->local.can_change_signature flag?  Is
> that supposed to be set at any given place or can IPA-CP do it on its own?

can_change_signature is currently used by i386 backend and it is set by inliner.
I plan to move it to visibility pass at the time local functions are dentified.
So yes, you can assume it is set and up to date at the time IPA-CP is run.

Honza
> 
> > 
> > Rest of patch looks OK. It definitely reads better than previous ipa-cp.c ;)
> > I suppose we will need to get some experience with the logic deciding whether to clone..
> 
> 
> Thanks, I'll post the current version in a moment.
> 
> Martin
Jan Hubicka July 14, 2011, 9:28 p.m. UTC | #8
> 
> Well, technically they survive until after inlining (because of
> indirect inlining which also derives information from the lattices
> corresponding to node->inlined_to node.  Results of arithmetic
> functions are not going to be accessed during inlining when compiling
> any reasonable program but...

Hmm, this sounds bad.  We should move it to GTY then incrementally.  I however
though that indirect inlining looks only at jump functions, not at lattices?
> > 
> > __attribute__ ((really_bad_attribute))
> > function (int param)
> > {
> >   use param
> > }
> > 
> > and then let ipa-cp to invent param is a constant.
> 
> what would be such a "really_bad_attribute" ? 

Well, we need to go through the attribute list and prepare list of "bad guys"
instead of forbidding any attribute.
Obviously we should not give up on "pure" attribute, for example.

Another class of attributes are those referring to function arguments that needs
updating if they are still in use after clonning. I think this was original reason
for adding the check.

I am not sure if we have attributes that should prevent clonning completely.
> > > > 
> > > > can_change_sigunature will also handle apply_args.
> > > > Add VA_START code there, too.  For the other use of this flag (in i386) VA_START
> 
> The last one already is VA_START... or do you mean a different one?

I meant the code in ipa-inline-analysis.c doing the same checks but skiiping va_start since
i386 backend tests it by itself.
> > BTW currently the edges from thunks miss any profile info.
> > (i.e. it will be 0). I guess we ought to make ipa-profile pass to estimate those
> > (it is difficult ot measure count of an alias).
> > 
> 
> I'm not really looking at the edges from thunks to the actual
> function.  OTOH, I assume that edges to a thunk do have a count and
> look at that.

They do have (unless they are thunk to thunk edges), but in any case we ought to
regularize things here, sooner or later someone will get confused with counts
missing in the callgraph.
> 
> > If you walk only hot edges, then you need to make your function descent into
> > both alias refs and thunks calls, or the aliases of thunks will not be seen
> > then.
> 
> Well, looking at bits of the patch again now, aliases to thunks might
> indeed be a problem for a few pieces of it.  I'll send the patch
> nevertheless and ponder about this problem later.

Hmm, please do :)
I will look at the updated patch.

Honza
> 
> Thanks,
> 
> Martin
Martin Jambor July 15, 2011, 1:08 p.m. UTC | #9
Hi,

On Thu, Jul 14, 2011 at 11:28:32PM +0200, Jan Hubicka wrote:
> > 
> > Well, technically they survive until after inlining (because of
> > indirect inlining which also derives information from the lattices
> > corresponding to node->inlined_to node.  Results of arithmetic
> > functions are not going to be accessed during inlining when compiling
> > any reasonable program but...
> 
> Hmm, this sounds bad.  We should move it to GTY then incrementally.  I however
> though that indirect inlining looks only at jump functions, not at lattices?

You are right, that is however basically an omission of mine - it was
supposed to handle the case below but I do not look into the
lattices.  I will fix that later.

Nevertheless, the calls to ipa_cst_from_jfunc from
ipa-inline-analyzis.c do access the values of lattices.  Potentially
all of them.  So I will GTY lattices too.

Thanks,

Martin


/* Verify that simple indirect calls are inlined even without early
   inlining..  */
/* { dg-do compile } */
/* { dg-options "-O3 -fdump-ipa-inline -fno-early-inlining"  } */

extern void non_existent(int);

int __attribute__ ((noinline,noclone)) get_input(void)
{
  return 1;
}

static void hooray ()
{
  non_existent (1);
}

void hip2 (void (*g)())
{
  non_existent (2);
  g ();
}

static void
__attribute__ ((noinline))
hip1 (void (*f)(void (*)()), void (*g)())
{
  non_existent (2);
  f (g);
}

int main (int argc, int *argv[])
{
  int i;

  for (i = 0; i < get_input (); i++)
    hip1 (hip2, hooray);
  return 0;
}

/* { dg-final { scan-ipa-dump "hooray\[^\\n\]*inline copy in main" "inline"  } } */
/* { dg-final { scan-ipa-dump "hip2\[^\\n\]*inline copy in main" "inline"  } } */
/* { dg-final { cleanup-ipa-dump "inline" } } */
diff mbox

Patch

2011-06-22  Martin Jambor  <mjambor@suse.cz>

	* ipa-prop.h: Include alloc-pool.h.
	(ipa_lattice_type): Removed.
	(ipcp_value_source): New type.
	(ipcp_value): Likewise.
	(ipcp_values_pool): Declare.
	(ipcp_sources_pool): Likewise.
	(ipa_param_descriptor): Removed.
	(ipcp_lattice): Removed fileds type and constant. Added fields decl,
	values, values_count, contains_variable, bottom, used and virt_call.
	(ipa_node_params): New fields lattices, known_vals,
	clone_for_all_contexts and noe dead, removed fields params and
	count_scale.
	(ipa_get_param): Updated.
	(ipa_param_cannot_devirtualize_p): Removed.
	(ipa_param_types_vec_empty): Likewise.
	(ipa_edge_args): New field next_edge_clone.
	(ipa_func_list): Removed.
	(ipa_init_func_list): Removed declaration.
	(ipa_push_func_to_list_1): Likewise.
	(ipa_pop_func_from_list): Likewise.
	(ipa_push_func_to_list): Removed.
	(ipa_lattice_from_jfunc): Remove declaration.
	(ipa_get_jf_pass_through_result): Declare.
	(ipa_get_jf_ancestor_result): Likewise.
	(ipa_value_from_jfunc): Likewise.
	(ipa_get_lattice): Update.
	(ipa_lat_is_single_const): New function.
	* ipa-prop.c (ipa_push_func_to_list_1): Removed.
	(ipa_init_func_list): Likewise.
	(ipa_pop_func_from_list): Likewise.
	(ipa_get_param_decl_index): Fix coding style.
	(ipa_populate_param_decls): Update to use new lattices.
	(ipa_initialize_node_params): Likewise.
	(visit_ref_for_mod_analysis): Likewise.
	(ipa_analyze_params_uses): Likewise.
	(ipa_free_node_params_substructures): Likewise.
	(ipa_edge_duplication_hook): Add the new edge to the list of edge
	clones.
	(ipa_node_duplication_hook): Update to use new lattices.
	(ipa_free_all_structures_after_ipa_cp): Free alloc pools.
	(ipa_free_all_structures_after_iinln): Likewise.
	(ipa_write_node_info): Update to use new lattices.
	(ipa_read_node_info): Likewise.
	(ipa_get_jf_pass_through_result): New function.
	(ipa_get_jf_ancestor_result): Likewise.
	(ipa_value_from_jfunc): Likewise.
	(ipa_cst_from_jfunc): Reimplemented using ipa_value_from_jfunc.
	* ipa-cp.c: Reimplemented.
	* params.def (PARAM_DEVIRT_TYPE_LIST_SIZE): Removed.
	(PARAM_IPA_CP_VALUE_LIST_SIZE): New parameter.
	* Makefile.in (IPA_PROP_H): Added alloc-pool.h to dependencies.

	* doc/invoke.texi (devirt-type-list-size): Removed description.
	(ipa-cp-value-list-size): Added description.

	* testsuite/gcc.dg/ipa/ipa-1.c: Updated testcase dump scan.
	* testsuite/gcc.dg/ipa/ipa-2.c: Likewise.
	* testsuite/gcc.dg/ipa/ipa-3.c: Likewise and made functions static.
	* testsuite/gcc.dg/ipa/ipa-4.c: Updated testcase dump scan.
	* testsuite/gcc.dg/ipa/ipa-5.c: Likewise.
	* testsuite/gcc.dg/ipa/ipa-7.c: Xfail test.
	* testsuite/gcc.dg/ipa/ipa-8.c: Updated testcase dump scan.
	* testsuite/gcc.dg/ipa/ipacost-1.c: Likewise.
	* testsuite/gcc.dg/ipa/ipacost-2.c: Likewise.
	* testsuite/gcc.dg/ipa/ipcp-1.c: New test.
	* testsuite/gcc.dg/ipa/ipcp-2.c: Likewise.
	* testsuite/gcc.dg/tree-ssa/ipa-cp-1.c: Updated testcase.


Index: src/gcc/ipa-prop.h
===================================================================
*** src.orig/gcc/ipa-prop.h
--- src/gcc/ipa-prop.h
*************** along with GCC; see the file COPYING3.
*** 25,30 ****
--- 25,31 ----
  #include "vec.h"
  #include "cgraph.h"
  #include "gimple.h"
+ #include "alloc-pool.h"
  
  /* The following definitions and interfaces are used by
     interprocedural analyses or parameters.  */
*************** struct GTY (()) ipa_jump_func
*** 130,176 ****
    } GTY ((desc ("%1.type"))) value;
  };
  
! /* All formal parameters in the program have a lattice associated with it
!    computed by the interprocedural stage of IPCP.
!    There are three main values of the lattice:
!    IPA_TOP - unknown,
!    IPA_BOTTOM - variable,
!    IPA_CONST_VALUE - simple scalar constant,
! 
!    We also use this type to propagate types accross the call graph for the
!    purpose of devirtualization.  In that case, IPA_CONST_VALUE denotes a known
!    type, rather than a constant.  */
! enum ipa_lattice_type
! {
!   IPA_BOTTOM,
!   IPA_CONST_VALUE,
!   IPA_TOP
! };
  
! /* All formal parameters in the program have a cval computed by
!    the interprocedural stage of IPCP. See enum ipa_lattice_type for
!    the various types of lattices supported */
! struct ipcp_lattice
  {
!   enum ipa_lattice_type type;
!   tree constant;
  };
  
! /* Structure describing a single formal parameter.  */
! struct ipa_param_descriptor
  {
-   /* IPA-CP lattice.  */
-   struct ipcp_lattice ipcp_lattice;
    /* PARAM_DECL of this parameter.  */
    tree decl;
!   /* Vector of BINFOs of types that this argument might encounter.  NULL
!      basically means a top value, bottom is marked by the cannot_devirtualize
!      flag below.*/
!   VEC (tree, heap) *types;
    /* The parameter is used.  */
!   unsigned used : 1;
!   /* Set when parameter type cannot be used for devirtualization.  */
!   unsigned cannot_devirtualize : 1;
  };
  
  /* ipa_node_params stores information related to formal parameters of functions
--- 131,214 ----
    } GTY ((desc ("%1.type"))) value;
  };
  
! struct ipcp_value;
  
! /* Describes a particular source for an IPA-CP value.  */
! 
! struct ipcp_value_source
  {
!   /* The incoming edge that brought the value.  */
!   struct cgraph_edge *cs;
!   /* If the jump function that resulted into his value was a pass-through or an
!      ancestor, this is the ipcp_value of the caller from which the described
!      value has been derived.  Otherwise it is NULL.  */
!   struct ipcp_value *val;
!   /* Next pointer in a linked list of sources of a value.  */
!   struct ipcp_value_source *next;
!   /* If the jump function that resulted into his value was a pass-through or an
!      ancestor, this is the index of the parameter of the caller the jump
!      function references.  */
!   int index;
! };
! 
! /* Describes one particular value stored in struct ipcp_lattice.  */
! 
! struct ipcp_value
! {
!   /* The actual value for the given parameter.  This is either an IPA invariant
!      or a TREE_BINFO describing a type that can be used for
!      devirtualization.  */
!   tree value;
!   /* The list of sources from which this value originates.  */
!   struct ipcp_value_source *sources;
!   /* Next pointers in a linked list of all values in a lattice.  */
!   struct ipcp_value *next;
!   /* Next pointers in a linked list of values in a strongly connected component
!      of values. */
!   struct ipcp_value *scc_next;
!   /* Next pointers in a linked list of SCCs of values sorted topologically
!      according their sources.  */
!   struct ipcp_value  *topo_next;
!   /* A specialized node created for this value, NULL if none has been (so far)
!      created.  */
!   struct cgraph_node *spec_node;
!   /* Depth first search number and low link for topological sorting of
!      values.  */
!   int dfs, low_link;
!   /* Time benefit and size cost that specializing the function for this value
!      would bring about in this function alone.  */
!   int local_time_benefit, local_size_cost;
!   /* Time benefit and size cost that specializing the function for this value
!      can bring about in it's callees (transitively).  */
!   int prop_time_benefit, prop_size_cost;
!   /* True if this valye is currently on the topo-sort stack.  */
!   bool on_stack;
  };
  
! extern alloc_pool ipcp_values_pool;
! extern alloc_pool ipcp_sources_pool;
! 
! /* Lattice describing potential values of a formal parameter of a function and
!    some of their other properties.  */
! 
! struct ipcp_lattice
  {
    /* PARAM_DECL of this parameter.  */
    tree decl;
! 
!   /* The list of known values and types in this lattice.  */
!   struct ipcp_value *values;
!   /* Number of known values and types in this lattice.  */
!   int values_count;
!   /* The lattice contains a variable component  (in addition to values).  */
!   bool contains_variable;
!   /* The value of the lattice is bottom (i.e. variable and unusable for any
!      propagation).  */
!   bool bottom;
    /* The parameter is used.  */
!   bool used;
!   /* There is a virtual call based on this parameter.  */
!   bool virt_call;
  };
  
  /* ipa_node_params stores information related to formal parameters of functions
*************** struct ipa_param_descriptor
*** 178,205 ****
     parameters (such as ipa-cp).  */
  struct ipa_node_params
  {
    /* Number of formal parameters of this function.  When set to 0, this
       function's parameters would not be analyzed by IPA CP.  */
    int param_count;
    /* Whether this function is called with variable number of actual
       arguments.  */
    unsigned called_with_var_arguments : 1;
    /* Whether the param uses analysis has already been performed.  */
    unsigned uses_analysis_done : 1;
!   /* Whether the function is enqueued in an ipa_func_list.  */
    unsigned node_enqueued : 1;
!   /* Pointer to an array of structures describing individual formal
!      parameters.  */
!   struct ipa_param_descriptor *params;
!   /* Only for versioned nodes this field would not be NULL,
!      it points to the node that IPA cp cloned from.  */
!   struct cgraph_node *ipcp_orig_node;
!   /* Meaningful only for original functions.  Expresses the
!      ratio between the direct calls and sum of all invocations of
!      this function (given by profiling info).  It is used to calculate
!      the profiling information of the original function and the versioned
!      one.  */
!   gcov_type count_scale;
  };
  
  /* ipa_node_params access functions.  Please use these to access fields that
--- 216,249 ----
     parameters (such as ipa-cp).  */
  struct ipa_node_params
  {
+   /* Pointer to an array of structures describing individual formal
+      parameters.  */
+   struct ipcp_lattice *lattices;
+   /* Only for versioned nodes this field would not be NULL,
+      it points to the node that IPA cp cloned from.  */
+   struct cgraph_node *ipcp_orig_node;
    /* Number of formal parameters of this function.  When set to 0, this
       function's parameters would not be analyzed by IPA CP.  */
+ 
+   /* If this node is an ipa-cp clone, these are the known values that describe
+      what it has been specialized for.  */
+   VEC (tree, heap) *known_vals;
    int param_count;
    /* Whether this function is called with variable number of actual
       arguments.  */
    unsigned called_with_var_arguments : 1;
+   /* Set when it is possible to create specialized versions of this node.  */
+   unsigned node_versionable : 1;
    /* Whether the param uses analysis has already been performed.  */
    unsigned uses_analysis_done : 1;
!   /* Whether the function is enqueued in ipa-cp propagation stack.  */
    unsigned node_enqueued : 1;
!   /* Whether we should create a specialized version based on values that are
!      known to be constant in all contexts.  */
!   unsigned clone_for_all_contexts : 1;
!   /* Node has been completely replaced by clones and will be removed after
!      ipa-cp is finished.  */
!   unsigned node_dead : 1;
  };
  
  /* ipa_node_params access functions.  Please use these to access fields that
*************** static inline tree
*** 229,235 ****
  ipa_get_param (struct ipa_node_params *info, int i)
  {
    gcc_assert (i >= 0 && i <= info->param_count);
!   return info->params[i].decl;
  }
  
  /* Return the used flag corresponding to the Ith formal parameter of
--- 273,279 ----
  ipa_get_param (struct ipa_node_params *info, int i)
  {
    gcc_assert (i >= 0 && i <= info->param_count);
!   return info->lattices[i].decl;
  }
  
  /* Return the used flag corresponding to the Ith formal parameter of
*************** static inline bool
*** 239,266 ****
  ipa_is_param_used (struct ipa_node_params *info, int i)
  {
    gcc_assert (i >= 0 && i <= info->param_count);
!   return info->params[i].used;
! }
! 
! /* Return the cannot_devirtualize flag corresponding to the Ith formal
!    parameter of the function associated with INFO.  The corresponding function
!    to set the flag is ipa_set_param_cannot_devirtualize.  */
! 
! static inline bool
! ipa_param_cannot_devirtualize_p (struct ipa_node_params *info, int i)
! {
!   gcc_assert (i >= 0 && i <= info->param_count);
!   return info->params[i].cannot_devirtualize;
! }
! 
! /* Return true iff the vector of possible types of the Ith formal parameter of
!    the function associated with INFO is empty.  */
! 
! static inline bool
! ipa_param_types_vec_empty (struct ipa_node_params *info, int i)
! {
!   gcc_assert (i >= 0 && i <= info->param_count);
!   return info->params[i].types == NULL;
  }
  
  /* Flag this node as having callers with variable number of arguments.  */
--- 283,289 ----
  ipa_is_param_used (struct ipa_node_params *info, int i)
  {
    gcc_assert (i >= 0 && i <= info->param_count);
!   return info->lattices[i].used;
  }
  
  /* Flag this node as having callers with variable number of arguments.  */
*************** ipa_is_called_with_var_arguments (struct
*** 285,290 ****
--- 308,316 ----
     arguments.  It can be accessed by the IPA_EDGE_REF macro.  */
  typedef struct GTY(()) ipa_edge_args
  {
+   /* Next pointer in a linked list of clones of the same function.  */
+   struct cgraph_edge *next_edge_clone;
+ 
    /* Number of actual arguments in this callsite.  When set to 0,
       this callsite's parameters would not be analyzed by the different
       stages of IPA CP.  */
*************** ipa_edge_args_info_available_for_edge_p
*** 402,434 ****
  					     ipa_edge_args_vector));
  }
  
- /* A function list element.  It is used to create a temporary worklist used in
-    the propagation stage of IPCP. (can be used for more IPA optimizations)  */
- struct ipa_func_list
- {
-   struct cgraph_node *node;
-   struct ipa_func_list *next;
- };
- 
- /* ipa_func_list interface.  */
- struct ipa_func_list *ipa_init_func_list (void);
- void ipa_push_func_to_list_1 (struct ipa_func_list **, struct cgraph_node *,
- 			      struct ipa_node_params *);
- struct cgraph_node *ipa_pop_func_from_list (struct ipa_func_list **);
- 
- /* Add cgraph NODE to the worklist WL if it is not already in one.  */
- 
- static inline void
- ipa_push_func_to_list (struct ipa_func_list **wl, struct cgraph_node *node)
- {
-   struct ipa_node_params *info = IPA_NODE_REF (node);
- 
-   if (!info->node_enqueued)
-     ipa_push_func_to_list_1 (wl, node, info);
- }
- 
- void ipa_analyze_node (struct cgraph_node *);
- 
  /* Function formal parameters related computations.  */
  void ipa_initialize_node_params (struct cgraph_node *node);
  bool ipa_propagate_indirect_call_infos (struct cgraph_edge *cs,
--- 428,433 ----
*************** bool ipa_propagate_indirect_call_infos (
*** 438,443 ****
--- 437,444 ----
  struct cgraph_edge *ipa_make_edge_direct_to_target (struct cgraph_edge *, tree,
  						    tree);
  
+ /* Functions related to both.  */
+ void ipa_analyze_node (struct cgraph_node *);
  
  /* Debugging interface.  */
  void ipa_print_node_params (FILE *, struct cgraph_node *node);
*************** void ipa_prop_write_jump_functions (cgra
*** 521,529 ****
  void ipa_prop_read_jump_functions (void);
  void ipa_update_after_lto_read (void);
  int ipa_get_param_decl_index (struct ipa_node_params *, tree);
! void ipa_lattice_from_jfunc (struct ipa_node_params *info,
! 			     struct ipcp_lattice *lat,
! 			     struct ipa_jump_func *jfunc);
  tree ipa_cst_from_jfunc (struct ipa_node_params *info,
  			 struct ipa_jump_func *jfunc);
  
--- 522,531 ----
  void ipa_prop_read_jump_functions (void);
  void ipa_update_after_lto_read (void);
  int ipa_get_param_decl_index (struct ipa_node_params *, tree);
! tree ipa_get_jf_pass_through_result (struct ipa_jump_func *, tree);
! tree ipa_get_jf_ancestor_result (struct ipa_jump_func *, tree);
! tree ipa_value_from_jfunc (struct ipa_node_params *info,
! 			   struct ipa_jump_func *jfunc);
  tree ipa_cst_from_jfunc (struct ipa_node_params *info,
  			 struct ipa_jump_func *jfunc);
  
*************** static inline struct ipcp_lattice *
*** 538,544 ****
  ipa_get_lattice (struct ipa_node_params *info, int i)
  {
    gcc_assert (i >= 0 && i <= info->param_count);
!   return &(info->params[i].ipcp_lattice);
  }
  
  #endif /* IPA_PROP_H */
--- 540,560 ----
  ipa_get_lattice (struct ipa_node_params *info, int i)
  {
    gcc_assert (i >= 0 && i <= info->param_count);
!   return &(info->lattices[i]);
! }
! 
! /* Return whether LAT is a lattice with a single constant and without an
!    undefined value.  */
! 
! static inline bool
! ipa_lat_is_single_const (struct ipcp_lattice *lat)
! {
!   if (lat->bottom
!       || lat->contains_variable
!       || lat->values_count != 1)
!     return false;
!   else
!     return true;
  }
  
  #endif /* IPA_PROP_H */
Index: src/gcc/ipa-prop.c
===================================================================
*** src.orig/gcc/ipa-prop.c
--- src/gcc/ipa-prop.c
*************** static struct cgraph_2edge_hook_list *ed
*** 65,129 ****
  static struct cgraph_2node_hook_list *node_duplication_hook_holder;
  static struct cgraph_node_hook_list *function_insertion_hook_holder;
  
- /* Add cgraph NODE described by INFO to the worklist WL regardless of whether
-    it is in one or not.  It should almost never be used directly, as opposed to
-    ipa_push_func_to_list.  */
- 
- void
- ipa_push_func_to_list_1 (struct ipa_func_list **wl,
- 			 struct cgraph_node *node,
- 			 struct ipa_node_params *info)
- {
-   struct ipa_func_list *temp;
- 
-   info->node_enqueued = 1;
-   temp = XCNEW (struct ipa_func_list);
-   temp->node = node;
-   temp->next = *wl;
-   *wl = temp;
- }
- 
- /* Initialize worklist to contain all functions.  */
- 
- struct ipa_func_list *
- ipa_init_func_list (void)
- {
-   struct cgraph_node *node;
-   struct ipa_func_list * wl;
- 
-   wl = NULL;
-   for (node = cgraph_nodes; node; node = node->next)
-     if (node->analyzed && !node->alias)
-       {
- 	struct ipa_node_params *info = IPA_NODE_REF (node);
- 	/* Unreachable nodes should have been eliminated before ipcp and
- 	   inlining.  */
- 	gcc_assert (node->needed || node->reachable);
- 	ipa_push_func_to_list_1 (&wl, node, info);
-       }
- 
-   return wl;
- }
- 
- /* Remove a function from the worklist WL and return it.  */
- 
- struct cgraph_node *
- ipa_pop_func_from_list (struct ipa_func_list **wl)
- {
-   struct ipa_node_params *info;
-   struct ipa_func_list *first;
-   struct cgraph_node *node;
- 
-   first = *wl;
-   *wl = (*wl)->next;
-   node = first->node;
-   free (first);
- 
-   info = IPA_NODE_REF (node);
-   info->node_enqueued = 0;
-   return node;
- }
- 
  /* Return index of the formal whose tree is PTREE in function which corresponds
     to INFO.  */
  
--- 65,70 ----
*************** ipa_get_param_decl_index (struct ipa_nod
*** 134,140 ****
  
    count = ipa_get_param_count (info);
    for (i = 0; i < count; i++)
!     if (ipa_get_param(info, i) == ptree)
        return i;
  
    return -1;
--- 75,81 ----
  
    count = ipa_get_param_count (info);
    for (i = 0; i < count; i++)
!     if (ipa_get_param (info, i) == ptree)
        return i;
  
    return -1;
*************** ipa_populate_param_decls (struct cgraph_
*** 157,163 ****
    param_num = 0;
    for (parm = fnargs; parm; parm = DECL_CHAIN (parm))
      {
!       info->params[param_num].decl = parm;
        param_num++;
      }
  }
--- 98,104 ----
    param_num = 0;
    for (parm = fnargs; parm; parm = DECL_CHAIN (parm))
      {
!       info->lattices[param_num].decl = parm;
        param_num++;
      }
  }
*************** ipa_initialize_node_params (struct cgrap
*** 198,208 ****
  {
    struct ipa_node_params *info = IPA_NODE_REF (node);
  
!   if (!info->params)
      {
        ipa_count_formal_params (node, info);
!       info->params = XCNEWVEC (struct ipa_param_descriptor,
! 				    ipa_get_param_count (info));
        ipa_populate_param_decls (node, info);
      }
  }
--- 139,149 ----
  {
    struct ipa_node_params *info = IPA_NODE_REF (node);
  
!   if (!info->lattices)
      {
        ipa_count_formal_params (node, info);
!       info->lattices = XCNEWVEC (struct ipcp_lattice,
! 				 ipa_get_param_count (info));
        ipa_populate_param_decls (node, info);
      }
  }
*************** visit_ref_for_mod_analysis (gimple stmt
*** 1497,1503 ****
      {
        int index = ipa_get_param_decl_index (info, op);
        gcc_assert (index >= 0);
!       info->params[index].used = true;
      }
  
    return false;
--- 1438,1444 ----
      {
        int index = ipa_get_param_decl_index (info, op);
        gcc_assert (index >= 0);
!       info->lattices[index].used = true;
      }
  
    return false;
*************** ipa_analyze_params_uses (struct cgraph_n
*** 1529,1535 ****
  	 the flag during modification analysis.  */
        if (is_gimple_reg (parm)
  	  && gimple_default_def (DECL_STRUCT_FUNCTION (node->decl), parm))
! 	info->params[i].used = true;
      }
  
    func = DECL_STRUCT_FUNCTION (decl);
--- 1470,1476 ----
  	 the flag during modification analysis.  */
        if (is_gimple_reg (parm)
  	  && gimple_default_def (DECL_STRUCT_FUNCTION (node->decl), parm))
! 	info->lattices[i].used = true;
      }
  
    func = DECL_STRUCT_FUNCTION (decl);
*************** ipa_free_all_edge_args (void)
*** 1936,1943 ****
  void
  ipa_free_node_params_substructures (struct ipa_node_params *info)
  {
!   free (info->params);
! 
    memset (info, 0, sizeof (*info));
  }
  
--- 1877,1886 ----
  void
  ipa_free_node_params_substructures (struct ipa_node_params *info)
  {
!   free (info->lattices);
!   /* Lattice values and their sources are deallocated with their alocation
!      pool.  */
!   VEC_free (tree, heap, info->known_vals);
    memset (info, 0, sizeof (*info));
  }
  
*************** ipa_edge_duplication_hook (struct cgraph
*** 2025,2030 ****
--- 1968,1977 ----
  
    arg_count = ipa_get_cs_argument_count (old_args);
    ipa_set_cs_argument_count (new_args, arg_count);
+ 
+   new_args->next_edge_clone = old_args->next_edge_clone;
+   old_args->next_edge_clone = dst;
+ 
    new_args->jump_functions =
      duplicate_ipa_jump_func_array (old_args->jump_functions, arg_count);
  
*************** ipa_node_duplication_hook (struct cgraph
*** 2048,2061 ****
    param_count = ipa_get_param_count (old_info);
  
    ipa_set_param_count (new_info, param_count);
!   new_info->params = (struct ipa_param_descriptor *)
!     duplicate_array (old_info->params,
! 		     sizeof (struct ipa_param_descriptor) * param_count);
    for (i = 0; i < param_count; i++)
!     new_info->params[i].types = VEC_copy (tree, heap,
!  					  old_info->params[i].types);
    new_info->ipcp_orig_node = old_info->ipcp_orig_node;
-   new_info->count_scale = old_info->count_scale;
  
    new_info->called_with_var_arguments = old_info->called_with_var_arguments;
    new_info->uses_analysis_done = old_info->uses_analysis_done;
--- 1995,2006 ----
    param_count = ipa_get_param_count (old_info);
  
    ipa_set_param_count (new_info, param_count);
!   new_info->lattices = (struct ipcp_lattice *)
!     duplicate_array (old_info->lattices,
! 		     sizeof (struct ipcp_lattice) * param_count);
    for (i = 0; i < param_count; i++)
!     new_info->lattices[i].values = old_info->lattices[i].values;
    new_info->ipcp_orig_node = old_info->ipcp_orig_node;
  
    new_info->called_with_var_arguments = old_info->called_with_var_arguments;
    new_info->uses_analysis_done = old_info->uses_analysis_done;
*************** ipa_free_all_structures_after_ipa_cp (vo
*** 2127,2132 ****
--- 2072,2079 ----
      {
        ipa_free_all_edge_args ();
        ipa_free_all_node_params ();
+       free_alloc_pool (ipcp_sources_pool);
+       free_alloc_pool (ipcp_values_pool);
        ipa_unregister_cgraph_hooks ();
      }
  }
*************** ipa_free_all_structures_after_iinln (voi
*** 2142,2147 ****
--- 2089,2098 ----
    ipa_free_all_edge_args ();
    ipa_free_all_node_params ();
    ipa_unregister_cgraph_hooks ();
+   if (ipcp_sources_pool)
+     free_alloc_pool (ipcp_sources_pool);
+   if (ipcp_values_pool)
+     free_alloc_pool (ipcp_values_pool);
  }
  
  /* Print ipa_tree_map data structures of all functions in the
*************** ipa_write_node_info (struct output_block
*** 2822,2828 ****
    gcc_assert (!info->node_enqueued);
    gcc_assert (!info->ipcp_orig_node);
    for (j = 0; j < ipa_get_param_count (info); j++)
!     bp_pack_value (&bp, info->params[j].used, 1);
    lto_output_bitpack (&bp);
    for (e = node->callees; e; e = e->next_callee)
      {
--- 2773,2779 ----
    gcc_assert (!info->node_enqueued);
    gcc_assert (!info->ipcp_orig_node);
    for (j = 0; j < ipa_get_param_count (info); j++)
!     bp_pack_value (&bp, info->lattices[j].used, 1);
    lto_output_bitpack (&bp);
    for (e = node->callees; e; e = e->next_callee)
      {
*************** ipa_read_node_info (struct lto_input_blo
*** 2863,2869 ****
      info->uses_analysis_done = true;
    info->node_enqueued = false;
    for (k = 0; k < ipa_get_param_count (info); k++)
!     info->params[k].used = bp_unpack_value (&bp, 1);
    for (e = node->callees; e; e = e->next_callee)
      {
        struct ipa_edge_args *args = IPA_EDGE_REF (e);
--- 2814,2820 ----
      info->uses_analysis_done = true;
    info->node_enqueued = false;
    for (k = 0; k < ipa_get_param_count (info); k++)
!     info->lattices[k].used = bp_unpack_value (&bp, 1);
    for (e = node->callees; e; e = e->next_callee)
      {
        struct ipa_edge_args *args = IPA_EDGE_REF (e);
*************** ipa_update_after_lto_read (void)
*** 3027,3094 ****
  	}
  }
  
! /* Given the jump function JFUNC, compute the lattice LAT that describes the
!    value coming down the callsite. INFO describes the caller node so that
!    pass-through jump functions can be evaluated.  */
  
! void
! ipa_lattice_from_jfunc (struct ipa_node_params *info, struct ipcp_lattice *lat,
! 			 struct ipa_jump_func *jfunc)
  {
!   if (jfunc->type == IPA_JF_CONST)
      {
!       lat->type = IPA_CONST_VALUE;
!       lat->constant = jfunc->value.constant;
      }
!   else if (jfunc->type == IPA_JF_PASS_THROUGH)
      {
!       struct ipcp_lattice *caller_lat;
!       tree cst;
  
!       caller_lat = ipa_get_lattice (info, jfunc->value.pass_through.formal_id);
!       lat->type = caller_lat->type;
!       if (caller_lat->type != IPA_CONST_VALUE)
! 	return;
!       cst = caller_lat->constant;
  
!       if (jfunc->value.pass_through.operation != NOP_EXPR)
  	{
! 	  tree restype;
! 	  if (TREE_CODE_CLASS (jfunc->value.pass_through.operation)
! 	      == tcc_comparison)
! 	    restype = boolean_type_node;
  	  else
! 	    restype = TREE_TYPE (cst);
! 	  cst = fold_binary (jfunc->value.pass_through.operation,
! 			     restype, cst, jfunc->value.pass_through.operand);
! 	}
!       if (!cst || !is_gimple_ip_invariant (cst))
! 	lat->type = IPA_BOTTOM;
!       lat->constant = cst;
!     }
!   else if (jfunc->type == IPA_JF_ANCESTOR)
!     {
!       struct ipcp_lattice *caller_lat;
!       tree t;
! 
!       caller_lat = ipa_get_lattice (info, jfunc->value.ancestor.formal_id);
!       lat->type = caller_lat->type;
!       if (caller_lat->type != IPA_CONST_VALUE)
! 	return;
!       if (TREE_CODE (caller_lat->constant) != ADDR_EXPR)
  	{
! 	  /* This can happen when the constant is a NULL pointer.  */
! 	  lat->type = IPA_BOTTOM;
! 	  return;
  	}
-       t = TREE_OPERAND (caller_lat->constant, 0);
-       t = build_ref_for_offset (EXPR_LOCATION (t), t,
- 				jfunc->value.ancestor.offset,
- 				jfunc->value.ancestor.type, NULL, false);
-       lat->constant = build_fold_addr_expr (t);
      }
    else
!     lat->type = IPA_BOTTOM;
  }
  
  /* Determine whether JFUNC evaluates to a constant and if so, return it.
--- 2978,3084 ----
  	}
  }
  
! /* Return the result of a (possibly arithmetic) pass through jump function
!    JFUNC on the constant value INPUT.  Return NULL_TREE if that cannot be
!    determined or itself is considered an interprocedural invariant.  */
  
! tree
! ipa_get_jf_pass_through_result (struct ipa_jump_func *jfunc, tree input)
  {
!   tree restype, res;
! 
!   gcc_checking_assert (is_gimple_ip_invariant (input));
!   if (jfunc->value.pass_through.operation == NOP_EXPR)
!     return input;
! 
!   if (TREE_CODE_CLASS (jfunc->value.pass_through.operation)
!       == tcc_comparison)
!     restype = boolean_type_node;
!   else
!     restype = TREE_TYPE (input);
!   res = fold_binary (jfunc->value.pass_through.operation, restype,
! 		     input, jfunc->value.pass_through.operand);
! 
!   if (res && !is_gimple_ip_invariant (res))
!     return NULL_TREE;
! 
!   return res;
! }
! 
! /* Return the result of an ancestor jump function JFUNC on te constant value
!    INPUT.  Return NULL_TREE if that cannot be determined.  */
! 
! tree
! ipa_get_jf_ancestor_result (struct ipa_jump_func *jfunc, tree input)
! {
!   if (TREE_CODE (input) == ADDR_EXPR)
      {
!       tree t = TREE_OPERAND (input, 0);
!       t = build_ref_for_offset (EXPR_LOCATION (t), t,
! 				jfunc->value.ancestor.offset,
! 				jfunc->value.ancestor.type, NULL, false);
!       return build_fold_addr_expr (t);
      }
!   else
!     return NULL_TREE;
! }
! 
! /* Determine whether JFUNC evaluates to a known value (that is either a
!    constant or a binfo) and if so, return it.  Otherwise return NULL. INFO
!    describes the caller node so that pass-through jump functions can be
!    evaluated.  */
! 
! tree
! ipa_value_from_jfunc (struct ipa_node_params *info, struct ipa_jump_func *jfunc)
! {
!   if (jfunc->type == IPA_JF_CONST)
!     return jfunc->value.constant;
!   else if (jfunc->type == IPA_JF_KNOWN_TYPE)
!     return jfunc->value.base_binfo;
!   else if (jfunc->type == IPA_JF_PASS_THROUGH
! 	   || jfunc->type == IPA_JF_ANCESTOR)
      {
!       tree input;
!       int idx;
  
!       if (jfunc->type == IPA_JF_PASS_THROUGH)
! 	idx = jfunc->value.pass_through.formal_id;
!       else
! 	idx = jfunc->value.ancestor.formal_id;
  
!       if (info->ipcp_orig_node)
! 	input = VEC_index (tree, info->known_vals, idx);
!       else
  	{
! 	  struct ipcp_lattice *lat = ipa_get_lattice (info, idx);
! 	  if (!ipa_lat_is_single_const (lat))
! 	    return NULL_TREE;
! 	  input = lat->values->value;
! 	}
! 
!       if (!input)
! 	return NULL_TREE;
! 
!       if (jfunc->type == IPA_JF_PASS_THROUGH)
! 	{
! 	  if (jfunc->value.pass_through.operation == NOP_EXPR)
! 	    return input;
! 	  else if (TREE_CODE (input) == TREE_BINFO)
! 	    return NULL_TREE;
  	  else
! 	    return ipa_get_jf_pass_through_result (jfunc, input);
! 	}
!       else
  	{
! 	  if (TREE_CODE (input) == TREE_BINFO)
! 	    return get_binfo_at_offset (input, jfunc->value.ancestor.offset,
! 					jfunc->value.ancestor.type);
! 	  else
! 	    return ipa_get_jf_ancestor_result (jfunc, input);
  	}
      }
    else
!     return NULL_TREE;
  }
  
  /* Determine whether JFUNC evaluates to a constant and if so, return it.
*************** ipa_lattice_from_jfunc (struct ipa_node_
*** 3098,3108 ****
  tree
  ipa_cst_from_jfunc (struct ipa_node_params *info, struct ipa_jump_func *jfunc)
  {
!   struct ipcp_lattice lat;
  
!   ipa_lattice_from_jfunc (info, &lat, jfunc);
!   if (lat.type == IPA_CONST_VALUE)
!     return lat.constant;
!   else
      return NULL_TREE;
  }
--- 3088,3097 ----
  tree
  ipa_cst_from_jfunc (struct ipa_node_params *info, struct ipa_jump_func *jfunc)
  {
!   tree res = ipa_value_from_jfunc (info, jfunc);
  
!   if (res && TREE_CODE (res) == TREE_BINFO)
      return NULL_TREE;
+   else
+     return res;
  }
Index: src/gcc/ipa-cp.c
===================================================================
*** src.orig/gcc/ipa-cp.c
--- src/gcc/ipa-cp.c
*************** You should have received a copy of the G
*** 19,134 ****
  along with GCC; see the file COPYING3.  If not see
  <http://www.gnu.org/licenses/>.  */
  
! /* Interprocedural constant propagation.  The aim of interprocedural constant
!    propagation (IPCP) is to find which function's argument has the same
!    constant value in each invocation throughout the whole program. For example,
!    consider the following program:
! 
!    int g (int y)
!    {
!      printf ("value is %d",y);
!    }
! 
!    int f (int x)
!    {
!      g (x);
!    }
! 
!    int h (int y)
!    {
!      g (y);
!    }
! 
!    void main (void)
!    {
!      f (3);
!      h (3);
!    }
! 
! 
!    The IPCP algorithm will find that g's formal argument y is always called
!    with the value 3.
! 
!    The algorithm used is based on "Interprocedural Constant Propagation", by
!    David Callahan, Keith D Cooper, Ken Kennedy, Linda Torczon, Comp86, pg
!    152-161
  
-    The optimization is divided into three stages:
  
     First stage - intraprocedural analysis
     =======================================
     This phase computes jump_function and modification flags.
  
!    A jump function for a callsite represents the values passed as an actual
!    arguments of a given callsite. There are three types of values:
!    Pass through - the caller's formal parameter is passed as an actual argument.
     Constant - a constant is passed as an actual argument.
     Unknown - neither of the above.
  
!    The jump function info, ipa_jump_func, is stored in ipa_edge_args
!    structure (defined in ipa_prop.h and pointed to by cgraph_node->aux)
!    modified_flags are defined in ipa_node_params structure
!    (defined in ipa_prop.h and pointed to by cgraph_edge->aux).
  
!    -ipcp_generate_summary() is the first stage driver.
  
     Second stage - interprocedural analysis
     ========================================
-    This phase does the interprocedural constant propagation.
-    It computes lattices for all formal parameters in the program
-    and their value that may be:
-    TOP - unknown.
-    BOTTOM - non constant.
-    CONSTANT - constant value.
- 
-    Lattice describing a formal parameter p will have a constant value if all
-    callsites invoking this function have the same constant value passed to p.
  
!    The lattices are stored in ipcp_lattice which is itself in ipa_node_params
!    structure (defined in ipa_prop.h and pointed to by cgraph_edge->aux).
  
!    -ipcp_iterate_stage() is the second stage driver.
  
!    Third phase - transformation of function code
     ============================================
-    Propagates the constant-valued formals into the function.
-    For each function whose parameters are constants, we create its clone.
  
!    Then we process the clone in two ways:
!    1. We insert an assignment statement 'parameter = const' at the beginning
!       of the cloned function.
!    2. For read-only parameters that do not live in memory, we replace all their
!       uses with the constant.
! 
!    We also need to modify some callsites to call the cloned functions instead
!    of the original ones.  For a callsite passing an argument found to be a
!    constant by IPCP, there are two different cases to handle:
!    1. A constant is passed as an argument.  In this case the callsite in the
!       should be redirected to call the cloned callee.
!    2. A parameter (of the caller) passed as an argument (pass through
!       argument).  In such cases both the caller and the callee have clones and
!       only the callsite in the cloned caller is redirected to call to the
!       cloned callee.
! 
!    This update is done in two steps: First all cloned functions are created
!    during a traversal of the call graph, during which all callsites are
!    redirected to call the cloned function.  Then the callsites are traversed
!    and many calls redirected back to fit the description above.
! 
!    -ipcp_insert_stage() is the third phase driver.
! 
! 
!    This pass also performs devirtualization - turns virtual calls into direct
!    ones if it can prove that all invocations of the function call the same
!    callee.  This is achieved by building a list of all base types (actually,
!    their BINFOs) that individual parameters can have in an iterative matter
!    just like propagating scalar constants and then examining whether virtual
!    calls which take a parameter as their object fold to the same target for all
!    these types.  If we cannot enumerate all types or there is a type which does
!    not have any BINFO associated with it, cannot_devirtualize of the associated
!    parameter descriptor is set which is an equivalent of BOTTOM lattice value
!    in standard IPA constant propagation.
! */
  
  #include "config.h"
  #include "system.h"
--- 19,101 ----
  along with GCC; see the file COPYING3.  If not see
  <http://www.gnu.org/licenses/>.  */
  
! /* Interprocedural constant propagation (IPA-CP).
! 
!    The goal of this transformation is to
! 
!    1) discover functions which are always invoked with some arguments with the
!       same known constant values and modify the functions so that the
!       subsequent optimizations can take advantage of the knowledge, and
! 
!    2) create specialized versions of functions transformed in this way if
!       some parameters are known constants only in certain contexts but the
!       estimated tradeoff between speedup and cost size is deemed good.
! 
!    The algorithm also propagates types and attempts to perform type based
!    devirtualization.  Types are propagated much like constants.
! 
!    The algorithm basically consists of three stages.  In the first, functions
!    are analyzed one at a time and jump functions are constructed for all known
!    call-sites.  In the second phase, the pass propagates information from the
!    jump functions across the call to reveal what values are available at what
!    call sites, performs estimations of effects of known values on functions and
!    their callees, and finally decides what specialized extra versions should be
!    created.  In the third, the special versions materialize and appropriate
!    calls are redirected.
! 
!    The algorithm used is to a certain extent based on "Interprocedural Constant
!    Propagation", by David Callahan, Keith D Cooper, Ken Kennedy, Linda Torczon,
!    Comp86, pg 152-161 and "A Methodology for Procedure Cloning" by Keith D
!    Cooper, Mary W. Hall, and Ken Kennedy.
  
  
     First stage - intraprocedural analysis
     =======================================
+ 
     This phase computes jump_function and modification flags.
  
!    A jump function for a call-site represents the values passed as an actual
!    arguments of a given call-site. In principle, there are three types of
!    values:
! 
!    Pass through - the caller's formal parameter is passed as an actual
!                   argument, plus an operation on it can be performed.
     Constant - a constant is passed as an actual argument.
     Unknown - neither of the above.
  
!    All jump function types are described in detail in ipa-prop.h, together with
!    the data structures that represent them and methods of accessing them.
  
!    ipcp_generate_summary() is the main function of the first stage.
  
     Second stage - interprocedural analysis
     ========================================
  
!    This stage is itself divided into two phases.  In the first, we propagate
!    known values over the call graph, in the second, we make cloning decisions.
  
!    First, we traverse the functions topologically from callers to callees and,
!    for each strongly connected component (SCC), we propagate constants
!    according to previously computed jump functions.  We also record what known
!    values depend on other known values and estimate local effects.  Finally, we
!    propagate cumulative information about these effects from dependant values
!    to those on which they depend.
! 
!    Second, we again traverse the call graph in the same topological order and
!    make clones for functions which we know are called with the same values in
!    all contexts and decide about extra specialized clones of functions just for
!    some contexts - these decisions are based on both local estimates and
!    cumulative estimates propagated from callees.
  
!    ipcp_propagate_stage() and ipcp_decision_stage() together constitute the
!    third stage.
! 
!    Third phase - materialization of clones, call statement updates.
     ============================================
  
!    This stage is currently performed by call graph code (mainly in cgraphunit.c
!    and tree-inline.c) according to instructions inserted to the call graph by
!    the second stage.  */
  
  #include "config.h"
  #include "system.h"
*************** along with GCC; see the file COPYING3.
*** 149,465 ****
  #include "fibheap.h"
  #include "params.h"
  #include "ipa-inline.h"
! 
! /* Number of functions identified as candidates for cloning. When not cloning
!    we can simplify iterate stage not forcing it to go through the decision
!    on what is profitable and what not.  */
! static int n_cloning_candidates;
  
  /* Maximal count found in program.  */
  static gcov_type max_count;
  
! /* Cgraph nodes that has been completely replaced by cloning during iterate
!  * stage and will be removed after ipcp is finished.  */
! static bitmap dead_nodes;
  
! static void ipcp_print_profile_data (FILE *);
! static void ipcp_function_scale_print (FILE *);
  
! /* Get the original node field of ipa_node_params associated with node NODE.  */
! static inline struct cgraph_node *
! ipcp_get_orig_node (struct cgraph_node *node)
! {
!   return IPA_NODE_REF (node)->ipcp_orig_node;
! }
  
! /* Return true if NODE describes a cloned/versioned function.  */
! static inline bool
! ipcp_node_is_clone (struct cgraph_node *node)
! {
!   return (ipcp_get_orig_node (node) != NULL);
! }
  
! /* Create ipa_node_params and its data structures for NEW_NODE.  Set ORIG_NODE
!    as the ipcp_orig_node field in ipa_node_params.  */
! static void
! ipcp_init_cloned_node (struct cgraph_node *orig_node,
! 		       struct cgraph_node *new_node)
! {
!   gcc_checking_assert (ipa_node_params_vector
! 		       && (VEC_length (ipa_node_params_t,
! 				       ipa_node_params_vector)
! 			   > (unsigned) cgraph_max_uid));
!   gcc_checking_assert (IPA_NODE_REF (new_node)->params);
!   IPA_NODE_REF (new_node)->ipcp_orig_node = orig_node;
! }
  
! /* Return scale for NODE.  */
! static inline gcov_type
! ipcp_get_node_scale (struct cgraph_node *node)
! {
!   return IPA_NODE_REF (node)->count_scale;
! }
  
! /* Set COUNT as scale for NODE.  */
! static inline void
! ipcp_set_node_scale (struct cgraph_node *node, gcov_type count)
! {
!   IPA_NODE_REF (node)->count_scale = count;
! }
  
- /* Return whether LAT is a constant lattice.  */
  static inline bool
! ipcp_lat_is_const (struct ipcp_lattice *lat)
  {
!   if (lat->type == IPA_CONST_VALUE)
!     return true;
!   else
!     return false;
! }
  
! /* Return whether LAT is a constant lattice that ipa-cp can actually insert
!    into the code (i.e. constants excluding member pointers and pointers).  */
! static inline bool
! ipcp_lat_is_insertable (struct ipcp_lattice *lat)
! {
!   return lat->type == IPA_CONST_VALUE;
  }
  
! /* Return true if LAT1 and LAT2 are equal.  */
! static inline bool
! ipcp_lats_are_equal (struct ipcp_lattice *lat1, struct ipcp_lattice *lat2)
! {
!   gcc_assert (ipcp_lat_is_const (lat1) && ipcp_lat_is_const (lat2));
!   if (lat1->type != lat2->type)
!     return false;
! 
!   if (TREE_CODE (lat1->constant) ==  ADDR_EXPR
!       && TREE_CODE (lat2->constant) ==  ADDR_EXPR
!       && TREE_CODE (TREE_OPERAND (lat1->constant, 0)) == CONST_DECL
!       && TREE_CODE (TREE_OPERAND (lat2->constant, 0)) == CONST_DECL)
!     return operand_equal_p (DECL_INITIAL (TREE_OPERAND (lat1->constant, 0)),
! 			    DECL_INITIAL (TREE_OPERAND (lat2->constant, 0)), 0);
!   else
!     return operand_equal_p (lat1->constant, lat2->constant, 0);
! }
  
- /* Compute Meet arithmetics:
-    Meet (IPA_BOTTOM, x) = IPA_BOTTOM
-    Meet (IPA_TOP,x) = x
-    Meet (const_a,const_b) = IPA_BOTTOM,  if const_a != const_b.
-    MEET (const_a,const_b) = const_a, if const_a == const_b.*/
  static void
! ipa_lattice_meet (struct ipcp_lattice *res, struct ipcp_lattice *lat1,
! 		  struct ipcp_lattice *lat2)
  {
!   if (lat1->type == IPA_BOTTOM || lat2->type == IPA_BOTTOM)
!     {
!       res->type = IPA_BOTTOM;
!       return;
!     }
!   if (lat1->type == IPA_TOP)
!     {
!       res->type = lat2->type;
!       res->constant = lat2->constant;
!       return;
!     }
!   if (lat2->type == IPA_TOP)
!     {
!       res->type = lat1->type;
!       res->constant = lat1->constant;
!       return;
!     }
!   if (!ipcp_lats_are_equal (lat1, lat2))
      {
!       res->type = IPA_BOTTOM;
!       return;
      }
!   res->type = lat1->type;
!   res->constant = lat1->constant;
! }
! 
! /* True when OLD_LAT and NEW_LAT values are not the same.  */
! 
! static bool
! ipcp_lattice_changed (struct ipcp_lattice *old_lat,
! 		      struct ipcp_lattice *new_lat)
! {
!   if (old_lat->type == new_lat->type)
      {
!       if (!ipcp_lat_is_const (old_lat))
! 	return false;
!       if (ipcp_lats_are_equal (old_lat, new_lat))
! 	return false;
      }
!   return true;
  }
  
  /* Print all ipcp_lattices of all functions to F.  */
  static void
! ipcp_print_all_lattices (FILE * f)
  {
    struct cgraph_node *node;
    int i, count;
  
!   fprintf (f, "\nLattice:\n");
!   for (node = cgraph_nodes; node; node = node->next)
      {
        struct ipa_node_params *info;
  
-       if (!node->analyzed)
- 	continue;
        info = IPA_NODE_REF (node);
!       fprintf (f, "  Node: %s:\n", cgraph_node_name (node));
        count = ipa_get_param_count (info);
        for (i = 0; i < count; i++)
  	{
  	  struct ipcp_lattice *lat = ipa_get_lattice (info, i);
  
  	  fprintf (f, "    param [%d]: ", i);
! 	  if (lat->type == IPA_CONST_VALUE)
  	    {
! 	      tree cst = lat->constant;
! 	      fprintf (f, "type is CONST ");
! 	      print_generic_expr (f, cst, 0);
! 	      if (TREE_CODE (cst) == ADDR_EXPR
! 		  && TREE_CODE (TREE_OPERAND (cst, 0)) == CONST_DECL)
  		{
! 		  fprintf (f, " -> ");
! 		  print_generic_expr (f, DECL_INITIAL (TREE_OPERAND (cst, 0)),
! 						       0);
  		}
  	    }
! 	  else if (lat->type == IPA_TOP)
! 	    fprintf (f, "type is TOP");
! 	  else
! 	    fprintf (f, "type is BOTTOM");
! 	  if (ipa_param_cannot_devirtualize_p (info, i))
! 	    fprintf (f, " - cannot_devirtualize set\n");
! 	  else if (ipa_param_types_vec_empty (info, i))
! 	    fprintf (f, " - type list empty\n");
! 	  else
  	    fprintf (f, "\n");
  	}
      }
  }
  
! /* Return true if ipcp algorithms would allow cloning NODE.  */
  
! static bool
! ipcp_versionable_function_p (struct cgraph_node *node)
  {
    struct cgraph_edge *edge;
! 
!   /* We always version the actual function and redirect through the aliases.  */
!   if (node->alias)
!     return false;
! 
!   /* We don't know how to clone thunks.  */
!   if (node->thunk.thunk_p)
!     return false;
  
    /* There are a number of generic reasons functions cannot be versioned.  We
       also cannot remove parameters if there are type attributes such as fnspec
       present.  */
!   if (!inline_summary (node)->versionable
!       || TYPE_ATTRIBUTES (TREE_TYPE (node->decl)))
!     return false;
  
!   /* Removing arguments doesn't work if the function takes varargs
!      or use __builtin_apply_args. 
!      FIXME: handle this together with can_change_signature flag.  */
!   for (edge = node->callees; edge; edge = edge->next_callee)
!     {
!       tree t = edge->callee->decl;
!       if (DECL_BUILT_IN_CLASS (t) == BUILT_IN_NORMAL
! 	  && (DECL_FUNCTION_CODE (t) == BUILT_IN_APPLY_ARGS
! 	     || DECL_FUNCTION_CODE (t) == BUILT_IN_VA_START))
! 	return false;
!     }
  
-   return true;
  }
  
  /* Return true if this NODE is viable candidate for cloning.  */
  static bool
  ipcp_cloning_candidate_p (struct cgraph_node *node)
  {
!   int n_calls = 0;
!   int n_hot_calls = 0;
!   gcov_type direct_call_sum = 0;
!   struct cgraph_edge *e;
! 
!   /* We look through aliases, so we clone the aliased function instead.  */
!   if (node->alias)
!     return false;
  
!   /* We never clone functions that are not visible from outside.
!      FIXME: in future we should clone such functions when they are called with
!      different constants, but current ipcp implementation is not good on this.
!      */
!   if (cgraph_only_called_directly_p (node) || !node->analyzed)
      return false;
  
!   /* When function address is taken, we are pretty sure it will be called in hidden way.  */
!   if (node->address_taken)
!     {
!       if (dump_file)
!         fprintf (dump_file, "Not considering %s for cloning; address is taken.\n",
!  	         cgraph_node_name (node));
!       return false;
!     }
! 
!   if (cgraph_function_body_availability (node) <= AVAIL_OVERWRITABLE)
!     {
!       if (dump_file)
!         fprintf (dump_file, "Not considering %s for cloning; body is overwritable.\n",
!  	         cgraph_node_name (node));
!       return false;
!     }
!   if (!ipcp_versionable_function_p (node))
      {
        if (dump_file)
!         fprintf (dump_file, "Not considering %s for cloning; body is not versionable.\n",
   	         cgraph_node_name (node));
        return false;
      }
-   for (e = node->callers; e; e = e->next_caller)
-     {
-       direct_call_sum += e->count;
-       n_calls ++;
-       if (cgraph_maybe_hot_edge_p (e))
- 	n_hot_calls ++;
-     }
  
!   if (!n_calls)
      {
        if (dump_file)
!         fprintf (dump_file, "Not considering %s for cloning; no direct calls.\n",
   	         cgraph_node_name (node));
        return false;
      }
-   if (inline_summary (node)->self_size < n_calls)
-     {
-       if (dump_file)
-         fprintf (dump_file, "Considering %s for cloning; code would shrink.\n",
-  	         cgraph_node_name (node));
-       return true;
-     }
  
!   if (!flag_ipa_cp_clone)
!     {
!       if (dump_file)
!         fprintf (dump_file, "Not considering %s for cloning; -fipa-cp-clone disabled.\n",
!  	         cgraph_node_name (node));
!       return false;
!     }
  
!   if (!optimize_function_for_speed_p (DECL_STRUCT_FUNCTION (node->decl)))
      {
        if (dump_file)
!         fprintf (dump_file, "Not considering %s for cloning; optimizing it for size.\n",
   	         cgraph_node_name (node));
!       return false;
      }
  
    /* When profile is available and function is hot, propagate into it even if
--- 116,382 ----
  #include "fibheap.h"
  #include "params.h"
  #include "ipa-inline.h"
! #include "ipa-utils.h"
  
  /* Maximal count found in program.  */
+ 
  static gcov_type max_count;
  
! /* Original overall size of the program.  */
  
! static long overall_size, max_new_size;
  
! /* Allocation pools for values and their sources.  */
  
! alloc_pool ipcp_values_pool;
! alloc_pool ipcp_sources_pool;
  
! /* Head of the linked list of topologically sorted values. */
  
! static struct ipcp_value *values_topo;
  
! /* Return true iff the CS is an edge within a strongly connected component as
!    computed by ipa_reduced_postorder.  */
  
  static inline bool
! edge_within_scc (struct cgraph_edge *cs)
  {
!   struct ipa_dfs_info *caller_dfs = (struct ipa_dfs_info *) cs->caller->aux;
!   struct ipa_dfs_info *callee_dfs;
!   struct cgraph_node *callee = cgraph_function_node (cs->callee, NULL);
  
!   callee_dfs = (struct ipa_dfs_info *) callee->aux;
!   return (caller_dfs
! 	  && callee_dfs
! 	  && caller_dfs->scc_no == callee_dfs->scc_no);
  }
  
! /* Print V which is extracted from a value in a lattice to F.  */
  
  static void
! print_ipcp_constant_value (FILE * f, tree v)
  {
!   if (TREE_CODE (v) == TREE_BINFO)
      {
!       fprintf (f, "BINFO ");
!       print_generic_expr (f, BINFO_TYPE (v), 0);
      }
!   else if (TREE_CODE (v) == ADDR_EXPR
! 	   && TREE_CODE (TREE_OPERAND (v, 0)) == CONST_DECL)
      {
!       fprintf (f, "& ");
!       print_generic_expr (f, DECL_INITIAL (TREE_OPERAND (v, 0)), 0);
      }
!   else
!     print_generic_expr (f, v, 0);
  }
  
  /* Print all ipcp_lattices of all functions to F.  */
+ 
  static void
! print_all_lattices (FILE * f, bool dump_sources, bool dump_benefits)
  {
    struct cgraph_node *node;
    int i, count;
  
!   fprintf (f, "\nLattices:\n");
!   FOR_EACH_FUNCTION_WITH_GIMPLE_BODY (node)
      {
        struct ipa_node_params *info;
  
        info = IPA_NODE_REF (node);
!       fprintf (f, "  Node: %s/%i:\n", cgraph_node_name (node), node->uid);
        count = ipa_get_param_count (info);
        for (i = 0; i < count; i++)
  	{
  	  struct ipcp_lattice *lat = ipa_get_lattice (info, i);
+ 	  struct ipcp_value *val;
+ 	  bool prev = false;
  
  	  fprintf (f, "    param [%d]: ", i);
! 	  if (lat->bottom)
! 	    {
! 	      fprintf (f, "BOTTOM\n");
! 	      continue;
! 	    }
! 
! 	  if (!lat->values_count && !lat->contains_variable)
! 	    {
! 	      fprintf (f, "TOP\n");
! 	      continue;
! 	    }
! 
! 	  if (lat->contains_variable)
! 	    {
! 	      fprintf (f, "VARIABLE");
! 	      prev = true;
! 	      if (dump_benefits)
! 		fprintf (f, "\n");
! 	    }
! 
! 	  for (val = lat->values; val; val = val->next)
  	    {
! 	      if (dump_benefits && prev)
! 		fprintf (f, "               ");
! 	      else if (!dump_benefits && prev)
! 		fprintf (f, ", ");
! 	      else
! 		prev = true;
! 
! 	      print_ipcp_constant_value (f, val->value);
! 
! 	      if (dump_sources)
  		{
! 		  struct ipcp_value_source *s;
! 
! 		  fprintf (f, " [from:");
! 		  for (s = val->sources; s; s = s->next)
! 		    fprintf (f, " %i(%i)", s->cs->caller->uid,s->cs->frequency);
! 		  fprintf (f, "]");
  		}
+ 
+ 	      if (dump_benefits)
+ 		fprintf (f, " [loc_time: %i, loc_size: %i, "
+ 			 "prop_time: %i, prop_size: %i]\n",
+ 			 val->local_time_benefit, val->local_size_cost,
+ 			 val->prop_time_benefit, val->prop_size_cost);
  	    }
! 	  if (!dump_benefits)
  	    fprintf (f, "\n");
  	}
      }
  }
  
! /* Determine whether it is at all technically possible to create clones of NODE
!    and store this information in the ipa_node_params structure associated
!    with NODE.  */
  
! static void
! determine_versionability (struct cgraph_node *node)
  {
    struct cgraph_edge *edge;
!   bool res = true;
  
    /* There are a number of generic reasons functions cannot be versioned.  We
       also cannot remove parameters if there are type attributes such as fnspec
       present.  */
!   if (node->alias || node->thunk.thunk_p
!       || !inline_summary (node)->versionable
!       || TYPE_ATTRIBUTES (TREE_TYPE (node->decl))
!       || cgraph_function_body_availability (node) <= AVAIL_OVERWRITABLE)
!     res =  false;
!   else
!     /* Removing arguments doesn't work if the function takes varargs
!        or use __builtin_apply_args.
!        FIXME: handle this together with can_change_signature flag.  */
!     for (edge = node->callees; edge; edge = edge->next_callee)
!       {
! 	tree t = edge->callee->decl;
! 	if (DECL_BUILT_IN_CLASS (t) == BUILT_IN_NORMAL
! 	    && (DECL_FUNCTION_CODE (t) == BUILT_IN_APPLY_ARGS
! 		|| DECL_FUNCTION_CODE (t) == BUILT_IN_VA_START))
! 	  {
! 	    res = false;
! 	    break;
! 	  };
!       }
  
!   if (!res && dump_file && node->alias && node->thunk.thunk_p)
!     fprintf (dump_file, "Function %s/%i is not versionable.\n",
! 	     cgraph_node_name (node), node->uid);
! 
!   IPA_NODE_REF (node)->node_versionable = res;
! }
! 
! /* Return true if it is at all technically possible to create clones of a
!    NODE.  */
! 
! static bool
! ipcp_versionable_function_p (struct cgraph_node *node)
! {
!   return IPA_NODE_REF (node)->node_versionable;
! }
! 
! /* Structure holding accumulated information about callers of a node.  */
! 
! struct caller_statistics
! {
!   gcov_type count_sum;
!   int n_calls, n_hot_calls, freq_sum;
! };
! 
! /* Initialize fields of STAT to zeroes.  */
! 
! static inline void
! init_caller_stats (struct caller_statistics *stats)
! {
!   stats->count_sum = 0;
!   stats->n_calls = 0;
!   stats->n_hot_calls = 0;
!   stats->freq_sum = 0;
! }
! 
! /* Worker callback of cgraph_for_node_and_aliases accumulating statistics of
!    non-thunk incoming edges to NODE.  */
! 
! static bool
! gather_caller_stats (struct cgraph_node *node, void *data)
! {
!   struct caller_statistics *stats = (struct caller_statistics *) data;
!   struct cgraph_edge *cs;
! 
!   for (cs = node->callers; cs; cs = cs->next_caller)
!     if (cs->caller->thunk.thunk_p)
!       gather_caller_stats (cs->caller, stats);
!     else
!       {
! 	stats->count_sum += cs->count;
! 	stats->freq_sum += cs->frequency;
! 	stats->n_calls++;
! 	if (cgraph_maybe_hot_edge_p (cs))
! 	  stats->n_hot_calls ++;
!       }
!   return false;
  
  }
  
  /* Return true if this NODE is viable candidate for cloning.  */
+ 
  static bool
  ipcp_cloning_candidate_p (struct cgraph_node *node)
  {
!   struct caller_statistics stats;
  
!   if (!cgraph_function_with_gimple_body_p (node))
      return false;
  
!   if (!flag_ipa_cp_clone)
      {
        if (dump_file)
!         fprintf (dump_file, "Not considering %s for cloning; "
! 		 "-fipa-cp-clone disabled.\n",
   	         cgraph_node_name (node));
        return false;
      }
  
!   if (!optimize_function_for_speed_p (DECL_STRUCT_FUNCTION (node->decl)))
      {
        if (dump_file)
!         fprintf (dump_file, "Not considering %s for cloning; "
! 		 "optimizing it for size.\n",
   	         cgraph_node_name (node));
        return false;
      }
  
!   init_caller_stats (&stats);
!   cgraph_for_node_and_aliases (node, gather_caller_stats, &stats, false);
  
!   if (inline_summary (node)->self_size < stats.n_calls)
      {
        if (dump_file)
!         fprintf (dump_file, "Considering %s for cloning; code might shrink.\n",
   	         cgraph_node_name (node));
!       return true;
      }
  
    /* When profile is available and function is hot, propagate into it even if
*************** ipcp_cloning_candidate_p (struct cgraph_
*** 467,481 ****
       significantly.  */
    if (max_count)
      {
!       if (direct_call_sum > node->count * 90 / 100)
  	{
  	  if (dump_file)
! 	    fprintf (dump_file, "Considering %s for cloning; usually called directly.\n",
  		     cgraph_node_name (node));
  	  return true;
          }
      }
!   if (!n_hot_calls)
      {
        if (dump_file)
  	fprintf (dump_file, "Not considering %s for cloning; no hot calls.\n",
--- 384,399 ----
       significantly.  */
    if (max_count)
      {
!       if (stats.count_sum > node->count * 90 / 100)
  	{
  	  if (dump_file)
! 	    fprintf (dump_file, "Considering %s for cloning; "
! 		     "usually called directly.\n",
  		     cgraph_node_name (node));
  	  return true;
          }
      }
!   if (!stats.n_hot_calls)
      {
        if (dump_file)
  	fprintf (dump_file, "Not considering %s for cloning; no hot calls.\n",
*************** ipcp_cloning_candidate_p (struct cgraph_
*** 488,1499 ****
    return true;
  }
  
! /* Mark parameter with index I of function described by INFO as unsuitable for
!    devirtualization.  Return true if it has already been marked so.  */
  
! static bool
! ipa_set_param_cannot_devirtualize (struct ipa_node_params *info, int i)
  {
!   bool ret = info->params[i].cannot_devirtualize;
!   info->params[i].cannot_devirtualize = true;
!   if (info->params[i].types)
!     VEC_free (tree, heap, info->params[i].types);
!   return ret;
! }
  
- /* Initialize ipcp_lattices array.  The lattices corresponding to supported
-    types (integers, real types and Fortran constants defined as const_decls)
-    are initialized to IPA_TOP, the rest of them to IPA_BOTTOM.  */
  static void
! ipcp_initialize_node_lattices (struct cgraph_node *node)
  {
!   int i;
!   struct ipa_node_params *info = IPA_NODE_REF (node);
!   enum ipa_lattice_type type;
  
!   if (ipa_is_called_with_var_arguments (info))
!     type = IPA_BOTTOM;
!   /* We don't know how to clone thunks even when they are local.  */
!   else if (node->local.local
! 	   && !node->thunk.thunk_p)
!     type = IPA_TOP;
!   /* When cloning is allowed, we can assume that externally visible functions
!      are not called.  We will compensate this by cloning later.  */
!   else if (ipcp_cloning_candidate_p (node))
!     type = IPA_TOP, n_cloning_candidates ++;
!   else
!     type = IPA_BOTTOM;
  
!   for (i = 0; i < ipa_get_param_count (info) ; i++)
!     {
!       ipa_get_lattice (info, i)->type = type;
!       if (type == IPA_BOTTOM)
! 	ipa_set_param_cannot_devirtualize (info, i);
!     }
  }
  
! /* Build a constant tree with type TREE_TYPE and value according to LAT.
!    Return the tree, or, if it is not possible to convert such value
!    to TREE_TYPE, NULL.  */
! static tree
! build_const_val (struct ipcp_lattice *lat, tree tree_type)
  {
!   tree val;
  
!   gcc_assert (ipcp_lat_is_const (lat));
!   val = lat->constant;
  
!   if (!useless_type_conversion_p (tree_type, TREE_TYPE (val)))
      {
!       if (fold_convertible_p (tree_type, val))
! 	return fold_build1 (NOP_EXPR, tree_type, val);
!       else if (TYPE_SIZE (tree_type) == TYPE_SIZE (TREE_TYPE (val)))
! 	return fold_build1 (VIEW_CONVERT_EXPR, tree_type, val);
!       else
! 	return NULL;
      }
!   return val;
  }
  
! /* Compute the proper scale for NODE.  It is the ratio between the number of
!    direct calls (represented on the incoming cgraph_edges) and sum of all
!    invocations of NODE (represented as count in cgraph_node).
  
!    FIXME: This code is wrong.  Since the callers can be also clones and
!    the clones are not scaled yet, the sums gets unrealistically high.
!    To properly compute the counts, we would need to do propagation across
!    callgraph (as external call to A might imply call to non-cloned B
!    if A's clone calls cloned B).  */
! static void
! ipcp_compute_node_scale (struct cgraph_node *node)
  {
!   gcov_type sum;
!   struct cgraph_edge *cs;
! 
!   sum = 0;
!   /* Compute sum of all counts of callers. */
!   for (cs = node->callers; cs != NULL; cs = cs->next_caller)
!     sum += cs->count;
!   /* Work around the unrealistically high sum problem.  We just don't want
!      the non-cloned body to have negative or very low frequency.  Since
!      majority of execution time will be spent in clones anyway, this should
!      give good enough profile.  */
!   if (sum > node->count * 9 / 10)
!     sum = node->count * 9 / 10;
!   if (node->count == 0)
!     ipcp_set_node_scale (node, 0);
!   else
!     ipcp_set_node_scale (node, sum * REG_BR_PROB_BASE / node->count);
  }
  
! /* Return true if there are some formal parameters whose value is IPA_TOP (in
!    the whole compilation unit).  Change their values to IPA_BOTTOM, since they
!    most probably get their values from outside of this compilation unit.  */
! static bool
! ipcp_change_tops_to_bottom (void)
! {
!   int i, count;
!   struct cgraph_node *node;
!   bool prop_again;
  
!   prop_again = false;
!   for (node = cgraph_nodes; node; node = node->next)
!     if (!node->alias)
!       {
! 	struct ipa_node_params *info = IPA_NODE_REF (node);
! 	count = ipa_get_param_count (info);
! 	for (i = 0; i < count; i++)
! 	  {
! 	    struct ipcp_lattice *lat = ipa_get_lattice (info, i);
! 	    if (lat->type == IPA_TOP)
! 	      {
! 		prop_again = true;
! 		if (dump_file)
! 		  {
! 		    fprintf (dump_file, "Forcing param ");
! 		    print_generic_expr (dump_file, ipa_get_param (info, i), 0);
! 		    fprintf (dump_file, " of node %s to bottom.\n",
! 			     cgraph_node_name (node));
! 		  }
! 		lat->type = IPA_BOTTOM;
! 	      }
! 	    if (!ipa_param_cannot_devirtualize_p (info, i)
! 		&& ipa_param_types_vec_empty (info, i))
! 	      {
! 		prop_again = true;
! 		ipa_set_param_cannot_devirtualize (info, i);
! 		if (dump_file)
! 		  {
! 		    fprintf (dump_file, "Marking param ");
! 		    print_generic_expr (dump_file, ipa_get_param (info, i), 0);
! 		    fprintf (dump_file, " of node %s as unusable for "
! 			     "devirtualization.\n",
! 			     cgraph_node_name (node));
! 		  }
! 	      }
! 	  }
!       }
!   return prop_again;
  }
  
! /* Insert BINFO to the list of known types of parameter number I of the
!    function described by CALLEE_INFO.  Return true iff the type information
!    associated with the callee parameter changed in any way.  */
  
! static bool
! ipcp_add_param_type (struct ipa_node_params *callee_info, int i, tree binfo)
  {
!   int j, count;
  
!   if (ipa_param_cannot_devirtualize_p (callee_info, i))
!     return false;
  
!   if (callee_info->params[i].types)
      {
!       count = VEC_length (tree, callee_info->params[i].types);
!       for (j = 0; j < count; j++)
! 	if (VEC_index (tree, callee_info->params[i].types, j) == binfo)
! 	  return false;
      }
  
!   if (VEC_length (tree, callee_info->params[i].types)
!       == (unsigned) PARAM_VALUE (PARAM_DEVIRT_TYPE_LIST_SIZE))
!     return !ipa_set_param_cannot_devirtualize (callee_info, i);
! 
!   VEC_safe_push (tree, heap, callee_info->params[i].types, binfo);
!   return true;
  }
  
! /* Copy known types information for parameter number CALLEE_IDX of CALLEE_INFO
!    from a parameter of CALLER_INFO as described by JF.  Return true iff the
!    type information changed in any way.  JF must be a pass-through or an
!    ancestor jump function.  */
  
! static bool
! ipcp_copy_types (struct ipa_node_params *caller_info,
! 		 struct ipa_node_params *callee_info,
! 		 int callee_idx, struct ipa_jump_func *jf)
  {
!   int caller_idx, j, count;
!   bool res;
! 
!   if (ipa_param_cannot_devirtualize_p (callee_info, callee_idx))
!     return false;
! 
!   if (jf->type == IPA_JF_PASS_THROUGH)
!     {
!       if (jf->value.pass_through.operation != NOP_EXPR)
! 	{
! 	  ipa_set_param_cannot_devirtualize (callee_info, callee_idx);
! 	  return true;
! 	}
!       caller_idx = jf->value.pass_through.formal_id;
!     }
!   else
!     caller_idx = jf->value.ancestor.formal_id;
  
!   if (ipa_param_cannot_devirtualize_p (caller_info, caller_idx))
      {
!       ipa_set_param_cannot_devirtualize (callee_info, callee_idx);
!       return true;
!     }
! 
!   if (!caller_info->params[caller_idx].types)
!     return false;
  
!   res = false;
!   count = VEC_length (tree, caller_info->params[caller_idx].types);
!   for (j = 0; j < count; j++)
!     {
!       tree binfo = VEC_index (tree, caller_info->params[caller_idx].types, j);
!       if (jf->type == IPA_JF_ANCESTOR)
  	{
! 	  binfo = get_binfo_at_offset (binfo, jf->value.ancestor.offset,
! 				       jf->value.ancestor.type);
! 	  if (!binfo)
  	    {
! 	      ipa_set_param_cannot_devirtualize (callee_info, callee_idx);
! 	      return true;
  	    }
  	}
-       res |= ipcp_add_param_type (callee_info, callee_idx, binfo);
      }
!   return res;
  }
  
! /* Propagate type information for parameter of CALLEE_INFO number I as
!    described by JF.  CALLER_INFO describes the caller.  Return true iff the
!    type information changed in any way.  */
  
  static bool
! ipcp_propagate_types (struct ipa_node_params *caller_info,
! 		      struct ipa_node_params *callee_info,
! 		      struct ipa_jump_func *jf, int i)
  {
!   switch (jf->type)
!     {
!     case IPA_JF_UNKNOWN:
!     case IPA_JF_CONST_MEMBER_PTR:
!     case IPA_JF_CONST:
!       break;
  
!     case IPA_JF_KNOWN_TYPE:
!       return ipcp_add_param_type (callee_info, i, jf->value.base_binfo);
  
!     case IPA_JF_PASS_THROUGH:
!     case IPA_JF_ANCESTOR:
!       return ipcp_copy_types (caller_info, callee_info, i, jf);
!     }
  
!   /* If we reach this we cannot use this parameter for devirtualization.  */
!   return !ipa_set_param_cannot_devirtualize (callee_info, i);
  }
  
! /* Interprocedural analysis. The algorithm propagates constants from the
!    caller's parameters to the callee's arguments.  */
  static void
! ipcp_propagate_stage (void)
  {
!   int i;
!   struct ipcp_lattice inc_lat = { IPA_BOTTOM, NULL };
!   struct ipcp_lattice new_lat = { IPA_BOTTOM, NULL };
!   struct ipcp_lattice *dest_lat;
!   struct cgraph_edge *cs;
!   struct ipa_jump_func *jump_func;
!   struct ipa_func_list *wl;
!   int count;
! 
!   ipa_check_create_node_params ();
!   ipa_check_create_edge_args ();
  
!   /* Initialize worklist to contain all functions.  */
!   wl = ipa_init_func_list ();
!   while (wl)
!     {
!       struct cgraph_node *node = ipa_pop_func_from_list (&wl);
!       struct ipa_node_params *info = IPA_NODE_REF (node);
  
!       for (cs = node->callees; cs; cs = cs->next_callee)
! 	{
! 	  struct cgraph_node *callee = cgraph_function_or_thunk_node (cs->callee, NULL);
! 	  struct ipa_node_params *callee_info = IPA_NODE_REF (callee);
! 	  struct ipa_edge_args *args = IPA_EDGE_REF (cs);
! 
! 	  if (ipa_is_called_with_var_arguments (callee_info)
! 	      || !cs->callee->analyzed
! 	      || ipa_is_called_with_var_arguments (callee_info))
! 	    continue;
  
- 	  count = ipa_get_cs_argument_count (args);
- 	  for (i = 0; i < count; i++)
- 	    {
- 	      jump_func = ipa_get_ith_jump_func (args, i);
- 	      ipa_lattice_from_jfunc (info, &inc_lat, jump_func);
- 	      dest_lat = ipa_get_lattice (callee_info, i);
- 	      ipa_lattice_meet (&new_lat, &inc_lat, dest_lat);
- 	      if (ipcp_lattice_changed (&new_lat, dest_lat))
- 		{
- 		  dest_lat->type = new_lat.type;
- 		  dest_lat->constant = new_lat.constant;
- 		  ipa_push_func_to_list (&wl, callee);
- 		}
  
! 	      if (ipcp_propagate_types (info, callee_info, jump_func, i))
! 		ipa_push_func_to_list (&wl, callee);
! 	    }
! 	}
!     }
! }
  
! /* Call the constant propagation algorithm and re-call it if necessary
!    (if there are undetermined values left).  */
! static void
! ipcp_iterate_stage (void)
  {
!   struct cgraph_node *node;
!   n_cloning_candidates = 0;
  
!   if (dump_file)
!     fprintf (dump_file, "\nIPA iterate stage:\n\n");
  
-   if (in_lto_p)
-     ipa_update_after_lto_read ();
  
!   for (node = cgraph_nodes; node; node = node->next)
!     if (!node->alias)
        {
! 	ipcp_initialize_node_lattices (node);
! 	ipcp_compute_node_scale (node);
!       }
!   if (dump_file && (dump_flags & TDF_DETAILS))
!     {
!       ipcp_print_all_lattices (dump_file);
!       ipcp_function_scale_print (dump_file);
      }
  
!   ipcp_propagate_stage ();
!   if (ipcp_change_tops_to_bottom ())
!     /* Some lattices have changed from IPA_TOP to IPA_BOTTOM.
!        This change should be propagated.  */
      {
!       gcc_assert (n_cloning_candidates);
!       ipcp_propagate_stage ();
      }
!   if (dump_file)
      {
!       fprintf (dump_file, "\nIPA lattices after propagation:\n");
!       ipcp_print_all_lattices (dump_file);
!       if (dump_flags & TDF_DETAILS)
!         ipcp_print_profile_data (dump_file);
      }
  }
  
! /* Check conditions to forbid constant insertion to function described by
!    NODE.  */
! static inline bool
! ipcp_node_modifiable_p (struct cgraph_node *node)
  {
!   /* Once we will be able to do in-place replacement, we can be more
!      lax here.  */
!   return ipcp_versionable_function_p (node);
  }
  
! /* Print count scale data structures.  */
! static void
! ipcp_function_scale_print (FILE * f)
  {
!   struct cgraph_node *node;
  
!   for (node = cgraph_nodes; node; node = node->next)
      {
!       if (!node->analyzed)
  	continue;
!       fprintf (f, "printing scale for %s: ", cgraph_node_name (node));
!       fprintf (f, "value is  " HOST_WIDE_INT_PRINT_DEC
! 	       "  \n", (HOST_WIDE_INT) ipcp_get_node_scale (node));
      }
  }
  
! /* Print counts of all cgraph nodes.  */
  static void
! ipcp_print_func_profile_counts (FILE * f)
  {
!   struct cgraph_node *node;
  
!   for (node = cgraph_nodes; node; node = node->next)
      {
!       fprintf (f, "function %s: ", cgraph_node_name (node));
!       fprintf (f, "count is  " HOST_WIDE_INT_PRINT_DEC
! 	       "  \n", (HOST_WIDE_INT) node->count);
      }
  }
  
! /* Print counts of all cgraph edges.  */
  static void
! ipcp_print_call_profile_counts (FILE * f)
  {
!   struct cgraph_node *node;
!   struct cgraph_edge *cs;
  
!   for (node = cgraph_nodes; node; node = node->next)
      {
!       for (cs = node->callees; cs; cs = cs->next_callee)
  	{
! 	  fprintf (f, "%s -> %s ", cgraph_node_name (cs->caller),
! 		   cgraph_node_name (cs->callee));
! 	  fprintf (f, "count is  " HOST_WIDE_INT_PRINT_DEC "  \n",
! 		   (HOST_WIDE_INT) cs->count);
  	}
      }
  }
  
! /* Print profile info for all functions.  */
  static void
! ipcp_print_profile_data (FILE * f)
  {
!   fprintf (f, "\nNODE COUNTS :\n");
!   ipcp_print_func_profile_counts (f);
!   fprintf (f, "\nCS COUNTS stage:\n");
!   ipcp_print_call_profile_counts (f);
  }
  
! /* Build and initialize ipa_replace_map struct according to LAT. This struct is
!    processed by versioning, which operates according to the flags set.
!    PARM_TREE is the formal parameter found to be constant.  LAT represents the
!    constant.  */
! static struct ipa_replace_map *
! ipcp_create_replace_map (tree parm_tree, struct ipcp_lattice *lat)
  {
!   struct ipa_replace_map *replace_map;
!   tree const_val;
  
!   const_val = build_const_val (lat, TREE_TYPE (parm_tree));
!   if (const_val == NULL_TREE)
      {
!       if (dump_file)
  	{
! 	  fprintf (dump_file, "  const ");
! 	  print_generic_expr (dump_file, lat->constant, 0);
! 	  fprintf (dump_file, "  can't be converted to param ");
! 	  print_generic_expr (dump_file, parm_tree, 0);
! 	  fprintf (dump_file, "\n");
  	}
-       return NULL;
      }
!   replace_map = ggc_alloc_ipa_replace_map ();
    if (dump_file)
      {
!       fprintf (dump_file, "  replacing param ");
!       print_generic_expr (dump_file, parm_tree, 0);
!       fprintf (dump_file, " with const ");
!       print_generic_expr (dump_file, const_val, 0);
!       fprintf (dump_file, "\n");
      }
!   replace_map->old_tree = parm_tree;
!   replace_map->new_tree = const_val;
!   replace_map->replace_p = true;
!   replace_map->ref_p = false;
  
!   return replace_map;
  }
  
! /* Return true if this callsite should be redirected to the original callee
!    (instead of the cloned one).  */
  static bool
! ipcp_need_redirect_p (struct cgraph_edge *cs)
  {
!   struct ipa_node_params *orig_callee_info;
!   int i, count;
!   struct cgraph_node *node = cgraph_function_or_thunk_node (cs->callee, NULL);
!   struct cgraph_node *orig;
  
!   if (!n_cloning_candidates)
      return false;
! 
!   /* We can't redirect anything in thunks, yet.  */
!   if (cs->caller->thunk.thunk_p)
      return true;
  
!   if ((orig = ipcp_get_orig_node (node)) != NULL)
!     node = orig;
!   if (ipcp_get_orig_node (cs->caller))
!     return false;
  
!   orig_callee_info = IPA_NODE_REF (node);
!   count = ipa_get_param_count (orig_callee_info);
!   for (i = 0; i < count; i++)
      {
!       struct ipcp_lattice *lat = ipa_get_lattice (orig_callee_info, i);
!       struct ipa_jump_func *jump_func;
! 
!       jump_func = ipa_get_ith_jump_func (IPA_EDGE_REF (cs), i);
!       if ((ipcp_lat_is_const (lat)
! 	   && jump_func->type != IPA_JF_CONST)
! 	  || (!ipa_param_cannot_devirtualize_p (orig_callee_info, i)
! 	      && !ipa_param_types_vec_empty (orig_callee_info, i)
! 	      && jump_func->type != IPA_JF_CONST
! 	      && jump_func->type != IPA_JF_KNOWN_TYPE))
! 	return true;
      }
  
!   return false;
  }
  
! /* Fix the callsites and the call graph after function cloning was done.  */
  static void
! ipcp_update_callgraph (void)
  {
!   struct cgraph_node *node;
  
!   for (node = cgraph_nodes; node; node = node->next)
!     if (node->analyzed && ipcp_node_is_clone (node))
!       {
! 	bitmap args_to_skip = NULL;
! 	struct cgraph_node *orig_node = ipcp_get_orig_node (node);
!         struct ipa_node_params *info = IPA_NODE_REF (orig_node);
!         int i, count = ipa_get_param_count (info);
!         struct cgraph_edge *cs, *next;
  
- 	if (node->local.can_change_signature)
- 	  {
- 	    args_to_skip = BITMAP_ALLOC (NULL);
- 	    for (i = 0; i < count; i++)
- 	      {
- 		struct ipcp_lattice *lat = ipa_get_lattice (info, i);
- 
- 		/* We can proactively remove obviously unused arguments.  */
- 		if (!ipa_is_param_used (info, i))
- 		  {
- 		    bitmap_set_bit (args_to_skip, i);
- 		    continue;
- 		  }
  
! 		if (lat->type == IPA_CONST_VALUE)
! 		  bitmap_set_bit (args_to_skip, i);
! 	      }
! 	  }
! 	for (cs = node->callers; cs; cs = next)
! 	  {
! 	    next = cs->next_caller;
! 	    if (!ipcp_node_is_clone (cs->caller) && ipcp_need_redirect_p (cs))
! 	      {
! 		if (dump_file)
! 		  fprintf (dump_file, "Redirecting edge %s/%i -> %s/%i "
! 			   "back to %s/%i.",
! 			   cgraph_node_name (cs->caller), cs->caller->uid,
! 			   cgraph_node_name (cs->callee), cs->callee->uid,
! 			   cgraph_node_name (orig_node), orig_node->uid);
! 		cgraph_redirect_edge_callee (cs, orig_node);
! 	      }
! 	  }
!       }
! }
  
- /* Update profiling info for versioned functions and the functions they were
-    versioned from.  */
  static void
! ipcp_update_profiling (void)
  {
-   struct cgraph_node *node, *orig_node;
-   gcov_type scale, scale_complement;
    struct cgraph_edge *cs;
  
!   for (node = cgraph_nodes; node; node = node->next)
      {
!       if (ipcp_node_is_clone (node))
! 	{
! 	  orig_node = ipcp_get_orig_node (node);
! 	  scale = ipcp_get_node_scale (orig_node);
! 	  node->count = orig_node->count * scale / REG_BR_PROB_BASE;
! 	  scale_complement = REG_BR_PROB_BASE - scale;
  
! 	  gcc_assert (scale_complement >= 0);
! 	  orig_node->count =
! 	    orig_node->count * scale_complement / REG_BR_PROB_BASE;
! 	  for (cs = node->callees; cs; cs = cs->next_callee)
! 	    cs->count = cs->count * scale / REG_BR_PROB_BASE;
! 	  for (cs = orig_node->callees; cs; cs = cs->next_callee)
! 	    cs->count = cs->count * scale_complement / REG_BR_PROB_BASE;
! 	}
      }
  }
  
! /* If NODE was cloned, how much would program grow? */
! static long
! ipcp_estimate_growth (struct cgraph_node *node)
  {
    struct cgraph_edge *cs;
!   int redirectable_node_callers = 0;
!   int removable_args = 0;
!   bool need_original
!      = !cgraph_will_be_removed_from_program_if_no_direct_calls (node);
!   VEC (tree, heap) *known_vals = NULL;
!   struct ipa_node_params *info;
!   int i, count;
!   int growth;
  
!   for (cs = node->callers; cs != NULL; cs = cs->next_caller)
!     if (cs->caller == node || !ipcp_need_redirect_p (cs))
!       redirectable_node_callers++;
      else
!       need_original = true;
  
!   /* If we will be able to fully replace original node, we never increase
!      program size.  */
!   if (!need_original)
!     return 0;
! 
!   info = IPA_NODE_REF (node);
!   count = ipa_get_param_count (info);
!   VEC_safe_grow_cleared (tree, heap, known_vals, count);
!   if (node->local.can_change_signature)
!     for (i = 0; i < count; i++)
!       {
! 	struct ipcp_lattice *lat = ipa_get_lattice (info, i);
  
! 	/* We can proactively remove obviously unused arguments.  */
! 	if (!ipa_is_param_used (info, i))
! 	  removable_args++;
  
! 	if (lat->type == IPA_CONST_VALUE)
! 	  {
! 	    removable_args++;
! 	    VEC_replace (tree, known_vals, i, lat->constant);
! 	  }
!       }
  
!   /* We make just very simple estimate of savings for removal of operand from
!      call site.  Precise cost is difficult to get, as our size metric counts
!      constants and moves as free.  Generally we are looking for cases that
!      small function is called very many times.  */
!   estimate_ipcp_clone_size_and_time (node, known_vals, &growth, NULL);
!   VEC_free (tree, heap, known_vals);
!   growth = growth
!   	   - removable_args * redirectable_node_callers;
!   if (growth < 0)
!     return 0;
!   return growth;
! }
  
  
! /* Estimate cost of cloning NODE.  */
! static long
! ipcp_estimate_cloning_cost (struct cgraph_node *node)
! {
!   int freq_sum = 1;
!   gcov_type count_sum = 1;
!   struct cgraph_edge *e;
!   int cost;
  
!   cost = ipcp_estimate_growth (node) * 1000;
!   if (!cost)
      {
!       if (dump_file)
!         fprintf (dump_file, "Versioning of %s will save code size\n",
! 	         cgraph_node_name (node));
!       return 0;
      }
  
!   for (e = node->callers; e; e = e->next_caller)
!     if (!bitmap_bit_p (dead_nodes, e->caller->uid)
!         && !ipcp_need_redirect_p (e))
!       {
! 	count_sum += e->count;
! 	freq_sum += e->frequency + 1;
!       }
  
!   if (max_count)
!     cost /= count_sum * 1000 / max_count + 1;
!   else
!     cost /= freq_sum * 1000 / REG_BR_PROB_BASE + 1;
!   if (dump_file)
!     fprintf (dump_file, "Cost of versioning %s is %i, (size: %i, freq: %i)\n",
!              cgraph_node_name (node), cost, inline_summary (node)->self_size,
! 	     freq_sum);
!   return cost + 1;
  }
  
! /* Walk indirect calls of NODE and if any polymorphic can be turned into a
!    direct one now, do so.  */
  
  static void
! ipcp_process_devirtualization_opportunities (struct cgraph_node *node)
  {
    struct ipa_node_params *info = IPA_NODE_REF (node);
!   struct cgraph_edge *ie, *next_ie;
  
!   for (ie = node->indirect_calls; ie; ie = next_ie)
      {
!       int param_index;
!       HOST_WIDE_INT token, anc_offset;
!       tree target, delta, otr_type;
!       struct ipcp_lattice *lat;
  
!       next_ie = ie->next_callee;
!       if (!ie->indirect_info->polymorphic)
! 	continue;
!       param_index = ie->indirect_info->param_index;
!       if (param_index == -1)
  	continue;
  
!       lat = ipa_get_lattice (info, param_index);
!       token = ie->indirect_info->otr_token;
!       anc_offset = ie->indirect_info->anc_offset;
!       otr_type = ie->indirect_info->otr_type;
!       target = NULL_TREE;
!       if (lat->type == IPA_CONST_VALUE)
  	{
! 	  tree binfo = gimple_extract_devirt_binfo_from_cst (lat->constant);
! 	  if (!binfo)
! 	    continue;
! 	  binfo = get_binfo_at_offset (binfo, anc_offset, otr_type);
! 	  if (!binfo)
! 	    continue;
! 	  target = gimple_get_virt_method_for_binfo (token, binfo, &delta);
! 	}
!       else
! 	{
! 	  int  types_count, j;
  
! 	  if (ipa_param_cannot_devirtualize_p (info, param_index)
! 	      || ipa_param_types_vec_empty (info, param_index))
! 	    continue;
  
! 	  types_count = VEC_length (tree, info->params[param_index].types);
! 	  for (j = 0; j < types_count; j++)
  	    {
! 	      tree binfo = VEC_index (tree, info->params[param_index].types, j);
! 	      tree d, t;
! 
! 	      binfo = get_binfo_at_offset (binfo, anc_offset, otr_type);
! 	      if (!binfo)
! 		{
! 		  target = NULL_TREE;
! 		  break;
! 		}
! 
! 	      t = gimple_get_virt_method_for_binfo (token, binfo, &d);
! 	      if (!t)
! 		{
! 		  target = NULL_TREE;
! 		  break;
! 		}
! 	      else if (!target)
! 		{
! 		  target = t;
! 		  delta = d;
! 		}
! 	      else if (target != t || !tree_int_cst_equal (delta, d))
! 		{
! 		  target = NULL_TREE;
! 		  break;
! 		}
  	    }
  	}
  
!       if (target)
! 	ipa_make_edge_direct_to_target (ie, target, delta);
!     }
! }
! 
! /* Return number of live constant parameters.  */
! static int
! ipcp_const_param_count (struct cgraph_node *node)
! {
!   int const_param = 0;
!   struct ipa_node_params *info = IPA_NODE_REF (node);
!   int count = ipa_get_param_count (info);
!   int i;
  
!   for (i = 0; i < count; i++)
!     {
!       struct ipcp_lattice *lat = ipa_get_lattice (info, i);
!       if ((ipcp_lat_is_insertable (lat)
! 	  /* Do not count obviously unused arguments.  */
! 	   && ipa_is_param_used (info, i))
! 	  || (!ipa_param_cannot_devirtualize_p (info, i)
! 	      && !ipa_param_types_vec_empty (info, i)))
! 	const_param++;
      }
-   return const_param;
  }
  
! /* Given that a formal parameter of NODE given by INDEX is known to be constant
!    CST, try to find any indirect edges that can be made direct and make them
!    so.  Note that INDEX is the number the parameter at the time of analyzing
!    parameter uses and parameter removals should not be considered for it.  (In
!    fact, the parameter itself has just been removed.)  */
  
  static void
! ipcp_discover_new_direct_edges (struct cgraph_node *node, int index, tree cst)
  {
!   struct cgraph_edge *ie, *next_ie;
  
!   for (ie = node->indirect_calls; ie; ie = next_ie)
      {
!       struct cgraph_indirect_call_info *ici = ie->indirect_info;
  
!       next_ie = ie->next_callee;
!       if (ici->param_index != index
! 	  || ici->polymorphic)
! 	continue;
  
!       ipa_make_edge_direct_to_target (ie, cst, NULL_TREE);
      }
  }
  
  
! /* Propagate the constant parameters found by ipcp_iterate_stage()
!    to the function's code.  */
  static void
! ipcp_insert_stage (void)
  {
!   struct cgraph_node *node, *node1 = NULL;
    int i;
-   VEC (cgraph_edge_p, heap) * redirect_callers;
-   VEC (ipa_replace_map_p,gc)* replace_trees;
-   int count;
-   tree parm_tree;
-   struct ipa_replace_map *replace_param;
-   fibheap_t heap;
-   long overall_size = 0, new_size = 0;
-   long max_new_size;
- 
-   ipa_check_create_node_params ();
-   ipa_check_create_edge_args ();
-   if (dump_file)
-     fprintf (dump_file, "\nIPA insert stage:\n\n");
  
!   dead_nodes = BITMAP_ALLOC (NULL);
  
-   FOR_EACH_FUNCTION_WITH_GIMPLE_BODY (node)
-     {
-       if (node->count > max_count)
- 	max_count = node->count;
-       overall_size += inline_summary (node)->self_size;
-     }
  
!   max_new_size = overall_size;
!   if (max_new_size < PARAM_VALUE (PARAM_LARGE_UNIT_INSNS))
!     max_new_size = PARAM_VALUE (PARAM_LARGE_UNIT_INSNS);
!   max_new_size = max_new_size * PARAM_VALUE (PARAM_IPCP_UNIT_GROWTH) / 100 + 1;
  
!   /* First collect all functions we proved to have constant arguments to
!      heap.  */
!   heap = fibheap_new ();
!   for (node = cgraph_nodes; node; node = node->next)
!     {
!       struct ipa_node_params *info;
!       /* Propagation of the constant is forbidden in certain conditions.  */
!       if (!node->analyzed || !ipcp_node_modifiable_p (node))
! 	  continue;
!       info = IPA_NODE_REF (node);
!       if (ipa_is_called_with_var_arguments (info))
! 	continue;
!       if (ipcp_const_param_count (node))
! 	node->aux = fibheap_insert (heap, ipcp_estimate_cloning_cost (node),
! 				    node);
!      }
! 
!   /* Now clone in priority order until code size growth limits are met or
!      heap is emptied.  */
!   while (!fibheap_empty (heap))
!     {
!       struct ipa_node_params *info;
!       int growth = 0;
!       bitmap args_to_skip;
!       struct cgraph_edge *cs;
  
!       node = (struct cgraph_node *)fibheap_extract_min (heap);
!       node->aux = NULL;
!       if (dump_file)
! 	fprintf (dump_file, "considering function %s\n",
! 		 cgraph_node_name (node));
  
!       growth = ipcp_estimate_growth (node);
  
!       if (new_size + growth > max_new_size)
! 	break;
!       if (growth
! 	  && cgraph_optimize_for_size_p (node))
! 	{
! 	  if (dump_file)
! 	    fprintf (dump_file, "Not versioning, cold code would grow");
! 	  continue;
! 	}
  
!       info = IPA_NODE_REF (node);
!       count = ipa_get_param_count (info);
  
!       replace_trees = VEC_alloc (ipa_replace_map_p, gc, 1);
  
!       if (node->local.can_change_signature)
! 	args_to_skip = BITMAP_GGC_ALLOC ();
!       else
! 	args_to_skip = NULL;
!       for (i = 0; i < count; i++)
  	{
! 	  struct ipcp_lattice *lat = ipa_get_lattice (info, i);
! 	  parm_tree = ipa_get_param (info, i);
  
! 	  /* We can proactively remove obviously unused arguments.  */
! 	  if (!ipa_is_param_used (info, i))
  	    {
! 	      if (args_to_skip)
! 	        bitmap_set_bit (args_to_skip, i);
  	      continue;
  	    }
  
! 	  if (lat->type == IPA_CONST_VALUE)
  	    {
! 	      replace_param =
! 		ipcp_create_replace_map (parm_tree, lat);
! 	      if (replace_param == NULL)
! 		break;
! 	      VEC_safe_push (ipa_replace_map_p, gc, replace_trees, replace_param);
! 	      if (args_to_skip)
! 	        bitmap_set_bit (args_to_skip, i);
  	    }
! 	}
!       if (i < count)
! 	{
  	  if (dump_file)
! 	    fprintf (dump_file, "Not versioning, some parameters couldn't be replaced");
! 	  continue;
! 	}
  
!       new_size += growth;
  
!       /* Look if original function becomes dead after cloning.  */
!       for (cs = node->callers; cs != NULL; cs = cs->next_caller)
! 	if (cs->caller == node || ipcp_need_redirect_p (cs))
! 	  break;
!       if (!cs && cgraph_will_be_removed_from_program_if_no_direct_calls (node))
! 	bitmap_set_bit (dead_nodes, node->uid);
! 
!       redirect_callers = collect_callers_of_node (node);
! 
!       /* Redirecting all the callers of the node to the
!          new versioned node.  */
!       node1 =
! 	cgraph_create_virtual_clone (node, redirect_callers, replace_trees,
! 				     args_to_skip, "constprop");
!       args_to_skip = NULL;
!       VEC_free (cgraph_edge_p, heap, redirect_callers);
!       replace_trees = NULL;
  
!       if (node1 == NULL)
! 	continue;
!       ipcp_process_devirtualization_opportunities (node1);
  
        if (dump_file)
! 	fprintf (dump_file, "versioned function %s with growth %i, overall %i\n",
! 		 cgraph_node_name (node), (int)growth, (int)new_size);
!       ipcp_init_cloned_node (node, node1);
  
        info = IPA_NODE_REF (node);
!       for (i = 0; i < count; i++)
! 	{
! 	  struct ipcp_lattice *lat = ipa_get_lattice (info, i);
! 	  if (lat->type == IPA_CONST_VALUE)
! 	    ipcp_discover_new_direct_edges (node1, i, lat->constant);
!         }
  
!       if (dump_file)
! 	dump_function_to_file (node1->decl, dump_file, dump_flags);
  
!       for (cs = node->callees; cs; cs = cs->next_callee)
! 	{
! 	  struct cgraph_node *callee = cgraph_function_or_thunk_node (cs->callee, NULL);
! 	  if (callee->aux)
! 	    {
! 	      fibheap_delete_node (heap, (fibnode_t) callee->aux);
! 	      callee->aux = fibheap_insert (heap,
! 					    ipcp_estimate_cloning_cost (callee),
! 					    callee);
! 	    }
! 	}
      }
  
!   while (!fibheap_empty (heap))
      {
!       if (dump_file)
! 	fprintf (dump_file, "skipping function %s\n",
! 		 cgraph_node_name (node));
!       node = (struct cgraph_node *) fibheap_extract_min (heap);
!       node->aux = NULL;
      }
-   fibheap_delete (heap);
-   BITMAP_FREE (dead_nodes);
-   ipcp_update_callgraph ();
-   ipcp_update_profiling ();
  }
  
  /* The IPCP driver.  */
  static unsigned int
  ipcp_driver (void)
  {
    cgraph_remove_unreachable_nodes (true,dump_file);
    if (dump_file)
      {
        fprintf (dump_file, "\nIPA structures before propagation:\n");
--- 406,2128 ----
    return true;
  }
  
! /* Arrays representing a topological ordering of call graph nodes and a stack
!    of noes used during constant propagation.  */
  
! struct topo_info
  {
!   struct cgraph_node **order;
!   struct cgraph_node **stack;
!   int nnodes, stack_top;
! };
! 
! /* Allocate the arrays in TOPO and topologically sort the nodes into order.  */
  
  static void
! build_topo_info (struct topo_info *topo)
  {
!   topo->order = XCNEWVEC (struct cgraph_node *, cgraph_n_nodes);
!   topo->stack = XCNEWVEC (struct cgraph_node *, cgraph_n_nodes);
!   topo->stack_top = 0;
!   topo->nnodes = ipa_reduced_postorder (topo->order, true, true, NULL);
! }
  
! /* Free information about strongly connected components and the arrays in
!    TOPO.  */
  
! static void
! free_topo_info (struct topo_info *topo)
! {
!   ipa_free_postorder_info ();
!   free (topo->order);
!   free (topo->stack);
  }
  
! /* Add NODE to the stack in TOPO, unless it is already there.  */
! 
! static inline void
! push_node_to_stack (struct topo_info *topo, struct cgraph_node *node)
  {
!   struct ipa_node_params *info = IPA_NODE_REF (node);
!   if (info->node_enqueued)
!     return;
!   info->node_enqueued = 1;
!   topo->stack[topo->stack_top++] = node;
! }
  
! /* Pop a node from the stack in TOPO and return it or return NULL if the stack
!    is empty.  */
  
! static struct cgraph_node *
! pop_node_from_stack (struct topo_info *topo)
! {
!   if (topo->stack_top)
      {
!       struct cgraph_node *node;
!       topo->stack_top--;
!       node = topo->stack[topo->stack_top];
!       IPA_NODE_REF (node)->node_enqueued = 0;
!       return node;
      }
!   else
!     return NULL;
  }
  
! /* Set lattice LAT to bottom and return true if it previously was not set as
!    such.  */
  
! static inline bool
! set_lattice_to_bottom (struct ipcp_lattice *lat)
  {
!   bool ret = !lat->bottom;
!   lat->bottom = true;
!   return ret;
  }
  
! /* Mark lattice as containing an unknown value and return true if it previously
!    was not marked as such.  */
  
! static inline bool
! set_lattice_contains_variable (struct ipcp_lattice *lat)
! {
!   bool ret = !lat->contains_variable;
!   lat->contains_variable = true;
!   return ret;
  }
  
! /* Initialize ipcp_lattices.  */
  
! static void
! initialize_node_lattices (struct cgraph_node *node)
  {
!   struct ipa_node_params *info = IPA_NODE_REF (node);
!   struct cgraph_edge *ie;
!   bool disable = false, variable = false;
!   int i;
  
!   /* FIXME: Can we clobber only the first argument of thunks?  */
!   if (node->alias || node->thunk.thunk_p
!       || ipa_is_called_with_var_arguments (info))
!     disable = true;
!   else if (!node->local.local)
!     {
!       /* When cloning is allowed, we can assume that externally visible
! 	 functions are not called.  We will compensate this by cloning
! 	 later.  */
!       if (ipcp_versionable_function_p (node)
! 	  && ipcp_cloning_candidate_p (node))
! 	variable = true;
!       else
! 	disable = true;
!     }
  
!   if (disable || variable)
      {
!       for (i = 0; i < ipa_get_param_count (info) ; i++)
! 	{
! 	  struct ipcp_lattice *lat = ipa_get_lattice (info, i);
! 	  if (disable)
! 	    set_lattice_to_bottom (lat);
! 	  else
! 	    set_lattice_contains_variable (lat);
! 	}
!       if (dump_file && (dump_flags & TDF_DETAILS)
! 	  && node->alias && node->thunk.thunk_p)
! 	fprintf (dump_file, "Marking all lattices of %s/%i as %s\n",
! 		 cgraph_node_name (node), node->uid,
! 		 disable ? "BOTTOM" : "VARIABLE");
      }
  
!   for (ie = node->indirect_calls; ie; ie = ie->next_callee)
!     if (ie->indirect_info->polymorphic)
!       {
! 	gcc_checking_assert (ie->indirect_info->param_index >= 0);
! 	ipa_get_lattice (info, ie->indirect_info->param_index)->virt_call = 1;
!       }
  }
  
! /* If checking is enabled, verify that no lattice is in the TOP state, i.e. not
!    bottom, not containing a variable component and without any known value at
!    the same time.  */
  
! static void
! verify_propagated_values (void)
  {
! #ifdef ENABLE_CHECKING
!   struct cgraph_node *node;
  
!   FOR_EACH_DEFINED_FUNCTION (node)
      {
!       struct ipa_node_params *info = IPA_NODE_REF (node);
!       int i, count = ipa_get_param_count (info);
  
!       for (i = 0; i < count; i++)
  	{
! 	  struct ipcp_lattice *lat = ipa_get_lattice (info, i);
! 
! 	  if (!lat->bottom
! 	      && !lat->contains_variable
! 	      && lat->values_count == 0)
  	    {
! 	      if (dump_file)
! 		{
! 		  fprintf (dump_file, "\nIPA lattices after constant "
! 			   "propagation:\n");
! 		  print_all_lattices (dump_file, true, false);
! 		}
! 
! 	      gcc_unreachable ();
  	    }
  	}
      }
! #endif
  }
  
! /* Return true iff X and Y should be considered equal values by IPA-CP.  */
  
  static bool
! values_equal_for_ipcp_p (tree x, tree y)
  {
!   gcc_checking_assert (x != NULL_TREE && y != NULL_TREE);
  
!   if (x == y)
!     return true;
  
!   if (TREE_CODE (x) == TREE_BINFO || TREE_CODE (y) == TREE_BINFO)
!     return false;
  
!   if (TREE_CODE (x) ==  ADDR_EXPR
!       && TREE_CODE (y) ==  ADDR_EXPR
!       && TREE_CODE (TREE_OPERAND (x, 0)) == CONST_DECL
!       && TREE_CODE (TREE_OPERAND (y, 0)) == CONST_DECL)
!     return operand_equal_p (DECL_INITIAL (TREE_OPERAND (x, 0)),
! 			    DECL_INITIAL (TREE_OPERAND (y, 0)), 0);
!   else
!     return operand_equal_p (x, y, 0);
  }
  
! /* Add a new value source to VAL, marking that a value comes from edge CS and
!    (if the underlying jump function is a pass-through or an ancestor one) from
!    a caller value SRC_VAL of a caller parameter described by SRC_INDEX.  */
! 
  static void
! add_value_source (struct ipcp_value *val, struct cgraph_edge *cs,
! 		  struct ipcp_value *src_val, int src_idx)
  {
!   struct ipcp_value_source *src;
  
!   src = (struct ipcp_value_source *) pool_alloc (ipcp_sources_pool);
!   src->cs = cs;
!   src->val = src_val;
!   src->index = src_idx;
  
!   src->next = val->sources;
!   val->sources = src;
! }
  
  
! /* Try to add NEWVAL to LAT, potentially creating a new struct ipcp_value for
!    it.  CS, SRC_VAL and SRC_INDEX are meant for add_value_source and have the
!    same meaning.  */
  
! static bool
! add_value_to_lattice (struct ipcp_lattice *lat, tree newval,
! 		      struct cgraph_edge *cs, struct ipcp_value *src_val,
! 		      int src_idx)
  {
!   struct ipcp_value *val;
  
!   if (lat->bottom)
!     return false;
  
  
!   for (val = lat->values; val; val = val->next)
!     if (values_equal_for_ipcp_p (val->value, newval))
        {
! 	if (edge_within_scc (cs))
! 	  {
! 	    struct ipcp_value_source *s;
! 	    for (s = val->sources; s ; s = s->next)
! 	      if (s->cs == cs)
! 		break;
! 	    if (s)
! 	      return false;
! 	  }
! 
! 	add_value_source (val, cs, src_val, src_idx);
! 	return false;
!       }
! 
!   if (lat->values_count == PARAM_VALUE (PARAM_IPA_CP_VALUE_LIST_SIZE))
!     {
!       /* We can only free sources, not the values themselves, because sources
! 	 of other values in this this SCC might point to them.   */
!       for (val = lat->values; val; val = val->next)
! 	{
! 	  while (val->sources)
! 	    {
! 	      struct ipcp_value_source *src = val->sources;
! 	      val->sources = src->next;
! 	      pool_free (ipcp_sources_pool, src);
! 	    }
! 	}
! 
!       lat->values = NULL;
!       return set_lattice_to_bottom (lat);
      }
  
!   lat->values_count++;
!   val = (struct ipcp_value *) pool_alloc (ipcp_values_pool);
!   memset (val, 0, sizeof (*val));
! 
!   add_value_source (val, cs, src_val, src_idx);
!   val->value = newval;
!   val->next = lat->values;
!   lat->values = val;
!   return true;
! }
! 
! /* Propagate values through a pass-through jump function JFUNC associated with
!    edge CS, taking values from SRC_LAT and putting them into DEST_LAT.  SRC_IDX
!    is the index of the source parameter.  */
! 
! static bool
! propagate_vals_accross_pass_through (struct cgraph_edge *cs,
! 				     struct ipa_jump_func *jfunc,
! 				     struct ipcp_lattice *src_lat,
! 				     struct ipcp_lattice *dest_lat,
! 				     int src_idx)
! {
!   struct ipcp_value *src_val;
!   bool ret = false;
! 
!   if (jfunc->value.pass_through.operation == NOP_EXPR)
!     for (src_val = src_lat->values; src_val; src_val = src_val->next)
!       ret |= add_value_to_lattice (dest_lat, src_val->value, cs,
! 				   src_val, src_idx);
!   else if (edge_within_scc (cs))
!     ret = set_lattice_contains_variable (dest_lat);
!   else
!     for (src_val = src_lat->values; src_val; src_val = src_val->next)
!       {
! 	tree cstval = src_val->value;
! 
! 	if (TREE_CODE (cstval) == TREE_BINFO)
! 	  {
! 	    ret |= set_lattice_contains_variable (dest_lat);
! 	    continue;
! 	  }
! 	cstval = ipa_get_jf_pass_through_result (jfunc, cstval);
! 
! 	if (cstval)
! 	  ret |= add_value_to_lattice (dest_lat, cstval, cs, src_val, src_idx);
! 	else
! 	  ret |= set_lattice_contains_variable (dest_lat);
!       }
! 
!   return ret;
! }
! 
! /* Propagate values through an ancestor jump function JFUNC associated with
!    edge CS, taking values from SRC_LAT and putting them into DEST_LAT.  SRC_IDX
!    is the index of the source parameter.  */
! 
! static bool
! propagate_vals_accross_ancestor (struct cgraph_edge *cs,
! 				 struct ipa_jump_func *jfunc,
! 				 struct ipcp_lattice *src_lat,
! 				 struct ipcp_lattice *dest_lat,
! 				 int src_idx)
! {
!   struct ipcp_value *src_val;
!   bool ret = false;
! 
!   if (edge_within_scc (cs))
!     return set_lattice_contains_variable (dest_lat);
! 
!   for (src_val = src_lat->values; src_val; src_val = src_val->next)
      {
!       tree t = src_val->value;
! 
!       if (TREE_CODE (t) == TREE_BINFO)
! 	t = get_binfo_at_offset (t, jfunc->value.ancestor.offset,
! 				 jfunc->value.ancestor.type);
!       else
! 	t = ipa_get_jf_ancestor_result (jfunc, t);
! 
!       if (t)
! 	ret |= add_value_to_lattice (dest_lat, t, cs, src_val, src_idx);
!       else
! 	ret |= set_lattice_contains_variable (dest_lat);
      }
! 
!   return ret;
! }
! 
! /* Propagate values across jump function JFUNC that is associated with edge CS
!    and put the values into DEST_LAT.  */
! 
! static bool
! propagate_accross_jump_function (struct cgraph_edge *cs,
! 				 struct ipa_jump_func *jfunc,
! 				 struct ipcp_lattice *dest_lat)
! {
!   if (dest_lat->bottom)
!     return false;
! 
!   if (jfunc->type == IPA_JF_CONST
!       || jfunc->type == IPA_JF_KNOWN_TYPE)
      {
!       tree val;
! 
!       if (jfunc->type == IPA_JF_KNOWN_TYPE)
! 	val = jfunc->value.base_binfo;
!       else
! 	val = jfunc->value.constant;
!       return add_value_to_lattice (dest_lat, val, cs, NULL, 0);
!     }
!   else if (jfunc->type == IPA_JF_PASS_THROUGH
! 	   || jfunc->type == IPA_JF_ANCESTOR)
!     {
!       struct ipa_node_params *caller_info = IPA_NODE_REF (cs->caller);
!       struct ipcp_lattice *src_lat;
!       int src_idx;
!       bool ret;
! 
!       if (jfunc->type == IPA_JF_PASS_THROUGH)
! 	src_idx = jfunc->value.pass_through.formal_id;
!       else
! 	src_idx = jfunc->value.ancestor.formal_id;
! 
!       src_lat = ipa_get_lattice (caller_info, src_idx);
!       if (src_lat->bottom)
! 	return set_lattice_contains_variable (dest_lat);
! 
!       /* If we would need to clone the caller and cannot, do not propagate.  */
!       if (!ipcp_versionable_function_p (cs->caller)
! 	  && (src_lat->contains_variable
! 	      || (src_lat->values_count > 1)))
! 	return set_lattice_contains_variable (dest_lat);
! 
!       if (jfunc->type == IPA_JF_PASS_THROUGH)
! 	ret = propagate_vals_accross_pass_through (cs, jfunc, src_lat,
! 						   dest_lat, src_idx);
!       else
! 	ret = propagate_vals_accross_ancestor (cs, jfunc, src_lat, dest_lat,
! 					       src_idx);
! 
!       if (src_lat->contains_variable)
! 	ret |= set_lattice_contains_variable (dest_lat);
! 
!       return ret;
      }
+ 
+   /* TODO: We currently do not handle member method pointers in IPA-CP (we only
+      use it for indirect inlining), we should propagate them too.  */
+   return set_lattice_contains_variable (dest_lat);
  }
  
! /* Propagate constants from the caller to the callee of CS.  INFO describes the
!    caller.  */
! 
! static bool
! propagate_constants_accross_call (struct cgraph_edge *cs)
  {
!   struct ipa_node_params *callee_info;
!   enum availability availability;
!   struct cgraph_node *callee;
!   struct ipa_edge_args *args;
!   bool ret = false;
!   int i, count;
! 
!   callee = cgraph_function_or_thunk_node (cs->callee, &availability);
!   if (!callee || !cgraph_function_with_gimple_body_p (callee))
!     return false;
! 
!   callee_info = IPA_NODE_REF (callee);
!   if (ipa_is_called_with_var_arguments (callee_info))
!     return false;
! 
!   args = IPA_EDGE_REF (cs);
!   count = ipa_get_cs_argument_count (args);
!   for (i = 0; i < count; i++)
!     {
!       struct ipa_jump_func *jump_func = ipa_get_ith_jump_func (args, i);
!       struct ipcp_lattice *dest_lat = ipa_get_lattice (callee_info, i);
! 
!       if (availability == AVAIL_OVERWRITABLE)
! 	ret |= set_lattice_contains_variable (dest_lat);
!       else
! 	ret |= propagate_accross_jump_function (cs, jump_func, dest_lat);
!     }
!   return ret;
  }
  
! /* If an indirect edge IE can be turned into a direct one based on KNOWN_VALS
!    (which can contain both constants and binfos) or KNOWN_BINFOS (which can be
!    NULL) return the destination.  If simple thunk delta must be applied too,
!    store it to DELTA.  */
! 
! static tree
! get_indirect_edge_target (struct cgraph_edge *ie, tree *delta,
! 			  VEC (tree, heap) *known_vals,
! 			  VEC (tree, heap) *known_binfos)
! {
!   int param_index = ie->indirect_info->param_index;
!   HOST_WIDE_INT token, anc_offset;
!   tree otr_type;
!   tree t;
! 
!   if (param_index == -1)
!     return NULL_TREE;
! 
!   if (!ie->indirect_info->polymorphic)
!     {
!       tree t = VEC_index (tree, known_vals, param_index);
!       if (t &&
! 	  TREE_CODE (t) == ADDR_EXPR
! 	  && TREE_CODE (TREE_OPERAND (t, 0)) == FUNCTION_DECL)
! 	{
! 	  *delta = NULL_TREE;
! 	  return TREE_OPERAND (t, 0);
! 	}
!       else
! 	return NULL_TREE;
!     }
! 
!   token = ie->indirect_info->otr_token;
!   anc_offset = ie->indirect_info->anc_offset;
!   otr_type = ie->indirect_info->otr_type;
! 
!   t = VEC_index (tree, known_vals, param_index);
!   if (!t && known_binfos)
!     t = VEC_index (tree, known_binfos, param_index);
!   if (!t)
!     return NULL_TREE;
! 
!   if (TREE_CODE (t) != TREE_BINFO)
!     {
!       tree binfo;
!       binfo = gimple_extract_devirt_binfo_from_cst (t);
!       if (!binfo)
! 	return NULL_TREE;
!       binfo = get_binfo_at_offset (binfo, anc_offset, otr_type);
!       if (!binfo)
! 	return NULL_TREE;
!       return gimple_get_virt_method_for_binfo (token, binfo, delta);
!     }
!   else
!     {
!       tree binfo;
! 
!       binfo = get_binfo_at_offset (t, anc_offset, otr_type);
!       if (!binfo)
! 	return NULL_TREE;
!       return gimple_get_virt_method_for_binfo (token, binfo, delta);
!     }
! }
! 
! /* Calculate devirtualization time bonus for NODE, assuming we know KNOWN_CSTS
!    and KNOWN_BINFOS.  */
! 
! static int
! devirtualization_time_bonus (struct cgraph_node *node,
! 			     VEC (tree, heap) *known_csts,
! 			     VEC (tree, heap) *known_binfos)
  {
!   struct cgraph_edge *ie;
!   int res = 0;
  
!   for (ie = node->indirect_calls; ie; ie = ie->next_callee)
      {
!       struct cgraph_node *callee;
!       struct inline_summary *isummary;
!       tree delta, target;
! 
!       target = get_indirect_edge_target (ie, &delta, known_csts, known_binfos);
!       if (!target)
! 	continue;
! 
!       /* Only bare minimum benefit for clearly un-inlineable targets.  */
!       res += 1;
!       callee = cgraph_get_node (target);
!       if (!callee)
  	continue;
! 
!       /* FIXME: The values below need re-considering and perhaps also
! 	 integrating into the cost metrics, at lest in some very basic way.  */
!       isummary = inline_summary (callee);
!       if (isummary->size <= MAX_INLINE_INSNS_AUTO / 4)
! 	res += 31;
!       else if (isummary->size <= MAX_INLINE_INSNS_AUTO / 2)
! 	res += 15;
!       else if (isummary->size <= MAX_INLINE_INSNS_AUTO
! 	       || DECL_DECLARED_INLINE_P (callee->decl))
! 	res += 7;
      }
+ 
+   return res;
  }
  
! /* Return true if cloning NODE is a good idea, given the estimated TIME_BENEFIT
!    and SIZE_COST and with the sum of frequencies of incoming edges to the
!    potential new clone in FREQUENCIES.  */
! 
! static bool
! good_cloning_opportunity_p (struct cgraph_node *node, int time_benefit,
! 			    int freq_sum, gcov_type count_sum, int size_cost)
! {
!   if (time_benefit == 0
!       || !flag_ipa_cp_clone
!       || !optimize_function_for_speed_p (DECL_STRUCT_FUNCTION (node->decl)))
!     return false;
! 
!   gcc_checking_assert (size_cost >= 0);
! 
!   /* FIXME:  These decisions need tuning.  */
!   if (max_count)
!     {
!       int evaluation, factor = (count_sum * 1000) / max_count;
! 
!       evaluation = (time_benefit * factor) / size_cost;
! 
!       if (dump_file && (dump_flags & TDF_DETAILS))
! 	fprintf (dump_file, "     good_cloning_opportunity_p (time: %i, "
! 		 "size: %i, count_sum: " HOST_WIDE_INT_PRINT_DEC
! 		 ") -> evaluation: %i, threshold: %i\n",
! 		 time_benefit, size_cost, (HOST_WIDE_INT) count_sum,
! 		 evaluation, 500);
! 
!       return evaluation > 500;
!     }
!   else
!     {
!       int evaluation = (time_benefit * freq_sum) / size_cost;
! 
!       if (dump_file && (dump_flags & TDF_DETAILS))
! 	fprintf (dump_file, "     good_cloning_opportunity_p (time: %i, "
! 		 "size: %i, freq_sum: %i) -> evaluation: %i, threshold: %i\n",
! 		 time_benefit, size_cost, freq_sum, evaluation,
! 		 CGRAPH_FREQ_BASE /2);
! 
!       return evaluation > (CGRAPH_FREQ_BASE /2);
!     }
! }
! 
! 
! /* Allocate KNOWN_CSTS and KNOWN_BINFOS and populate them with values of
!    parameters that are known independent of the context.  INFO describes the
!    function.  If REMOVABLE_PARAMS is non-NULL, store the number o removable
!    parameters in it.  */
! 
! static bool
! gather_context_independent_values (struct ipa_node_params *info,
! 				   VEC (tree, heap) **known_csts,
! 				   VEC (tree, heap) **known_binfos,
! 				   int *removable_params)
! {
!   int i, count = ipa_get_param_count (info);
!   int rparams = 0;
!   bool ret = false;
! 
!   *known_csts = NULL;
!   *known_binfos = NULL;
!   VEC_safe_grow_cleared (tree, heap, *known_csts, count);
!   VEC_safe_grow_cleared (tree, heap, *known_binfos, count);
! 
!   for (i = 0; i < count ; i++)
!     {
!       struct ipcp_lattice *lat = ipa_get_lattice (info, i);
! 
!       if (ipa_lat_is_single_const (lat))
! 	{
! 	  struct ipcp_value *val = lat->values;
! 	  if (TREE_CODE (val->value) != TREE_BINFO)
! 	    {
! 	      VEC_replace (tree, *known_csts, i, val->value);
! 	      rparams++;
! 	      ret = true;
! 	    }
! 	  else if (lat->virt_call)
! 	    {
! 	      VEC_replace (tree, *known_binfos, i, val->value);
! 	      ret = true;
! 	    }
! 	  else if (!ipa_is_param_used (info, i))
! 	    rparams++;
! 	}
!       else if (!ipa_is_param_used (info, i))
! 	rparams++;
!     }
! 
!   if (removable_params)
!     *removable_params = rparams;
!   return ret;
! }
! 
! /* Iterate over known values of parameters of NODE and estimate the local
!    effects in terms of time and size they have.  */
! 
  static void
! estimate_local_effects (struct cgraph_node *node)
  {
!   struct ipa_node_params *info = IPA_NODE_REF (node);
!   int i, count = ipa_get_param_count (info);
!   VEC (tree, heap) *known_csts, *known_binfos;
!   bool always_const;
!   int base_time = inline_summary (node)->time;
!   int removable_params;
! 
!   if (!count || !ipcp_versionable_function_p (node))
!     return;
  
!   if (dump_file && (dump_flags & TDF_DETAILS))
!     fprintf (dump_file, "\nEstimating effects for %s/%i, base_time: %i.\n",
! 	     cgraph_node_name (node), node->uid, base_time);
! 
!   always_const = gather_context_independent_values (info, &known_csts,
! 						    &known_binfos,
! 						    &removable_params);
!   if (always_const)
!     {
!       struct caller_statistics stats;
!       int time, size;
! 
!       init_caller_stats (&stats);
!       cgraph_for_node_and_aliases (node, gather_caller_stats, &stats, false);
!       estimate_ipcp_clone_size_and_time (node, known_csts, &size, &time);
!       time -= devirtualization_time_bonus (node, known_csts, known_binfos);
!       size -= stats.n_calls * removable_params;
! 
!       if (dump_file)
! 	fprintf (dump_file, " * context independent values, size: %i, "
! 		 "time_benefit: %i\n", size, base_time - time);
! 
!       if (size <= 0
! 	  || cgraph_will_be_removed_from_program_if_no_direct_calls (node))
! 	{
! 	  info->clone_for_all_contexts = true;
! 	  base_time = time;
! 
! 	  if (dump_file)
! 	    fprintf (dump_file, "     Decided to specialize for all "
! 		     "known contexts, code not going to grow.\n");
! 	}
!       else if (good_cloning_opportunity_p (node, base_time - time,
! 					   stats.freq_sum, stats.count_sum,
! 					   size))
! 	{
! 	  if (size + overall_size <= max_new_size)
! 	    {
! 	      info->clone_for_all_contexts = true;
! 	      base_time = time;
! 	      overall_size += size;
! 
! 	      if (dump_file)
! 		fprintf (dump_file, "     Decided to specialize for all "
! 			 "known contexts, growth deemed beneficial.\n");
! 	    }
! 	  else if (dump_file && (dump_flags & TDF_DETAILS))
! 	    fprintf (dump_file, "   Not cloning for all contexts because "
! 		     "max_new_size would be reached with %li.\n",
! 		     size + overall_size);
! 	}
!     }
! 
!   for (i = 0; i < count ; i++)
      {
!       struct ipcp_lattice *lat = ipa_get_lattice (info, i);
!       struct ipcp_value *val;
! 
!       if (lat->bottom
! 	  || !lat->values
! 	  || VEC_index (tree, known_csts, i)
! 	  || VEC_index (tree, known_binfos, i))
! 	continue;
! 
!       for (val = lat->values; val; val = val->next)
! 	{
! 	  int time, size, time_benefit;
! 
! 	  if (TREE_CODE (val->value) != TREE_BINFO)
! 	    {
! 	      VEC_replace (tree, known_csts, i, val->value);
! 	      VEC_replace (tree, known_binfos, i, NULL_TREE);
! 	    }
! 	  else if (lat->virt_call)
! 	    {
! 	      VEC_replace (tree, known_csts, i, NULL_TREE);
! 	      VEC_replace (tree, known_binfos, i, val->value);
! 	    }
! 	  else
! 	    continue;
! 
! 	  estimate_ipcp_clone_size_and_time (node, known_csts, &size, &time);
! 	  time_benefit = base_time - time
! 	    + devirtualization_time_bonus (node, known_csts, known_binfos);
! 
! 	  if (dump_file && (dump_flags & TDF_DETAILS))
! 	    {
! 	      fprintf (dump_file, " - estimates for value ");
! 	      print_ipcp_constant_value (dump_file, val->value);
! 	      fprintf (dump_file, " for parameter ");
! 	      print_generic_expr (dump_file, ipa_get_param (info, i), 0);
! 	      fprintf (dump_file, ": time_benefit: %i, size: %i\n",
! 		       time_benefit, size);
! 	    }
! 
! 	  val->local_time_benefit = time_benefit;
! 	  val->local_size_cost = size;
! 	}
      }
+ 
+   VEC_free (tree, heap, known_csts);
+   VEC_free (tree, heap, known_binfos);
  }
  
! 
! /* Add value CUR_VAL and all yet-unsorted values it is dependent on to the
!    topological sort of values.  */
! 
  static void
! add_val_to_toposort (struct ipcp_value *cur_val)
  {
!   static int dfs_counter = 0;
!   static struct ipcp_value *stack;
!   struct ipcp_value_source *src;
! 
!   if (cur_val->dfs)
!     return;
! 
!   dfs_counter++;
!   cur_val->dfs = dfs_counter;
!   cur_val->low_link = dfs_counter;
! 
!   cur_val->topo_next = stack;
!   stack = cur_val;
!   cur_val->on_stack = true;
  
!   for (src = cur_val->sources; src; src = src->next)
!     if (src->val)
!       {
! 	if (src->val->dfs == 0)
! 	  {
! 	    add_val_to_toposort (src->val);
! 	    if (src->val->low_link < cur_val->low_link)
! 	      cur_val->low_link = src->val->low_link;
! 	  }
! 	else if (src->val->on_stack
! 		 && src->val->dfs < cur_val->low_link)
! 	  cur_val->low_link = src->val->dfs;
!       }
! 
!   if (cur_val->dfs == cur_val->low_link)
      {
!       struct ipcp_value *v, *scc_list = NULL;
! 
!       do
  	{
! 	  v = stack;
! 	  stack = v->topo_next;
! 	  v->on_stack = false;
! 
! 	  v->scc_next = scc_list;
! 	  scc_list = v;
  	}
+       while (v != cur_val);
+ 
+       cur_val->topo_next = values_topo;
+       values_topo = cur_val;
      }
  }
  
! /* Add all values in lattices associated with NODE to the topological sort if
!    they are not there yet.  */
! 
  static void
! add_all_node_vals_to_toposort (struct cgraph_node *node)
  {
!   struct ipa_node_params *info = IPA_NODE_REF (node);
!   int i, count = ipa_get_param_count (info);
! 
!   for (i = 0; i < count ; i++)
!     {
!       struct ipcp_lattice *lat = ipa_get_lattice (info, i);
!       struct ipcp_value *val;
! 
!       if (lat->bottom || !lat->values)
! 	continue;
!       for (val = lat->values; val; val = val->next)
! 	add_val_to_toposort (val);
!     }
  }
  
! /* One pass of constants propagation along the call graph edges, from callers
!    to callees (requires topological ordering in TOPO), iterate over strongly
!    connected components.  */
! 
! static void
! propagate_constants_topo (struct topo_info *topo)
  {
!   int i;
  
!   for (i = topo->nnodes - 1; i >= 0; i--)
      {
!       struct cgraph_node *v, *node = topo->order[i];
!       struct ipa_dfs_info *node_dfs_info;
! 
!       if (!cgraph_function_with_gimple_body_p (node))
! 	continue;
! 
!       node_dfs_info = (struct ipa_dfs_info *) node->aux;
!       /* First, iteratively propagate within the strongly connected component
! 	 until all lattices stabilize.  */
!       v = node_dfs_info->next_cycle;
!       while (v)
! 	{
! 	  push_node_to_stack (topo, v);
! 	  v = ((struct ipa_dfs_info *) v->aux)->next_cycle;
! 	}
! 
!       v = node;
!       while (v)
! 	{
! 	  struct cgraph_edge *cs;
! 
! 	  for (cs = v->callees; cs; cs = cs->next_callee)
! 	    if (edge_within_scc (cs)
! 		&& propagate_constants_accross_call (cs))
! 	      push_node_to_stack (topo, cs->callee);
! 	  v = pop_node_from_stack (topo);
! 	}
! 
!       /* Afterwards, propagate along edges leading out of the SCC, calculates
! 	 the local effects of the discovered constants and all valid values to
! 	 their topological sort.  */
!       v = node;
!       while (v)
  	{
! 	  struct cgraph_edge *cs;
! 
! 	  estimate_local_effects (v);
! 	  add_all_node_vals_to_toposort (v);
! 	  for (cs = v->callees; cs; cs = cs->next_callee)
! 	    if (!edge_within_scc (cs))
! 	      propagate_constants_accross_call (cs);
! 
! 	  v = ((struct ipa_dfs_info *) v->aux)->next_cycle;
  	}
      }
! }
! 
! /* Propagate the estimated effects of individual values along the topological
!    from the dependant values to those they depend on.  */
! 
! static void
! propagate_effects (void)
! {
!   struct ipcp_value *base;
! 
!   for (base = values_topo; base; base = base->topo_next)
!     {
!       struct ipcp_value_source *src;
!       struct ipcp_value *val;
!       int time = 0, size = 0;
! 
!       for (val = base; val; val = val->scc_next)
! 	{
! 	  time += val->local_time_benefit + val->prop_time_benefit;
! 	  size += val->local_size_cost + val->prop_size_cost;
! 	}
! 
!       for (val = base; val; val = val->scc_next)
! 	for (src = val->sources; src; src = src->next)
! 	  if (src->val
! 	      && cgraph_maybe_hot_edge_p (src->cs))
! 	    {
! 	      src->val->prop_time_benefit += time;
! 	      src->val->prop_size_cost += size;
! 	    }
!     }
! }
! 
! 
! /* Propagate constants, binfos and their effects from the summaries
!    interprocedurally.  */
! 
! static void
! ipcp_propagate_stage (struct topo_info *topo)
! {
!   struct cgraph_node *node;
! 
!   if (dump_file)
!     fprintf (dump_file, "\n Propagating constants:\n\n");
! 
!   if (in_lto_p)
!     ipa_update_after_lto_read ();
! 
!   FOR_EACH_DEFINED_FUNCTION (node)
!   {
!     determine_versionability (node);
!     initialize_node_lattices (node);
!     if (node->count > max_count)
!       max_count = node->count;
!     overall_size += inline_summary (node)->self_size;
!   }
! 
!   max_new_size = overall_size;
!   if (max_new_size < PARAM_VALUE (PARAM_LARGE_UNIT_INSNS))
!     max_new_size = PARAM_VALUE (PARAM_LARGE_UNIT_INSNS);
!   max_new_size += max_new_size * PARAM_VALUE (PARAM_IPCP_UNIT_GROWTH) / 100 + 1;
! 
!   if (dump_file)
!     fprintf (dump_file, "\noverall_size: %li, max_new_size: %li\n",
! 	     overall_size, max_new_size);
! 
!   propagate_constants_topo (topo);
!   verify_propagated_values ();
!   propagate_effects ();
! 
    if (dump_file)
      {
!       fprintf (dump_file, "\nIPA lattices after all propagation:\n");
!       print_all_lattices (dump_file, (dump_flags & TDF_DETAILS), true);
      }
! }
  
! /* Discover newly direct outgoing edges from NODE which is a new clone with
!    known KNOWN_VALS and make them direct.  */
! 
! static void
! ipcp_discover_new_direct_edges (struct cgraph_node *node,
! 				VEC (tree, heap) *known_vals)
! {
!   struct cgraph_edge *ie, *next_ie;
! 
!   for (ie = node->indirect_calls; ie; ie = next_ie)
!     {
!       tree delta, target;
! 
!       next_ie = ie->next_callee;
!       target = get_indirect_edge_target (ie, &delta, known_vals, NULL);
!       if (target)
! 	ipa_make_edge_direct_to_target (ie, target, delta);
!     }
  }
  
! /* Return true if edge CS does bring about the value described by SRC.  */
! 
  static bool
! cgraph_edge_brings_value_p (struct cgraph_edge *cs,
! 			    struct ipcp_value_source *src)
  {
!   struct ipa_node_params *caller_info = IPA_NODE_REF (cs->caller);
  
!   if (IPA_NODE_REF (cs->callee)->ipcp_orig_node
!       || caller_info->node_dead)
      return false;
!   if (!src->val)
      return true;
  
!   if (caller_info->ipcp_orig_node)
!     {
!       tree t = VEC_index (tree, caller_info->known_vals, src->index);
!       return (t != NULL_TREE
! 	      && values_equal_for_ipcp_p (src->val->value, t));
!     }
!   else
!     {
!       struct ipcp_lattice *lat = ipa_get_lattice (caller_info, src->index);
!       if (ipa_lat_is_single_const (lat)
! 	  && values_equal_for_ipcp_p (src->val->value, lat->values->value))
! 	return true;
!       else
! 	return false;
!     }
! }
! 
! /* Get the next clone in the linked list of clones of an edge.  */
! 
! static inline struct cgraph_edge *
! get_next_cgraph_edge_clone (struct cgraph_edge *cs)
! {
!   struct ipa_edge_args *args = IPA_EDGE_REF (cs);
!   return args->next_edge_clone;
! }
! 
! /* Given VAL, iterate over all its sources and if they still hold, add their
!    edge frequency and their number into *FREQUENCY and *CALLER_COUNT
!    respectively.  */
! 
! static bool
! get_info_about_necessary_edges (struct ipcp_value *val, int *freq_sum,
! 				gcov_type *count_sum, int *caller_count)
! {
!   struct ipcp_value_source *src;
!   int freq = 0, count = 0;
!   gcov_type cnt = 0;
!   bool hot = false;
! 
!   for (src = val->sources; src; src = src->next)
!     {
!       struct cgraph_edge *cs = src->cs;
!       while (cs)
! 	{
! 	  if (cgraph_edge_brings_value_p (cs, src))
! 	    {
! 	      count++;
! 	      freq += cs->frequency;
! 	      cnt += cs->count;
! 	      hot |= cgraph_maybe_hot_edge_p (cs);
! 	    }
! 	  cs = get_next_cgraph_edge_clone (cs);
! 	}
!     }
! 
!   *freq_sum = freq;
!   *count_sum = cnt;
!   *caller_count = count;
!   return hot;
! }
! 
! /* Return a vector of incoming edges that do bring value VAL.  It is assumed
!    their number is known and equal to CALLER_COUNT.  */
! 
! static VEC (cgraph_edge_p,heap) *
! gather_edges_for_value (struct ipcp_value *val, int caller_count)
! {
!   struct ipcp_value_source *src;
!   VEC (cgraph_edge_p,heap) *ret;
! 
!   ret = VEC_alloc (cgraph_edge_p, heap, caller_count);
!   for (src = val->sources; src; src = src->next)
!     {
!       struct cgraph_edge *cs = src->cs;
!       while (cs)
! 	{
! 	  if (cgraph_edge_brings_value_p (cs, src))
! 	    VEC_quick_push (cgraph_edge_p, ret, cs);
! 	  cs = get_next_cgraph_edge_clone (cs);
! 	}
!     }
! 
!   return ret;
! }
! 
! /* Construct a replacement map for a know VALUE for a formal parameter PARAM.
!    Return it or NULL if for some reason it cannot be created.  */
! 
! static struct ipa_replace_map *
! get_replacement_map (tree value, tree parm)
! {
!   tree req_type = TREE_TYPE (parm);
!   struct ipa_replace_map *replace_map;
! 
!   if (!useless_type_conversion_p (req_type, TREE_TYPE (value)))
!     {
!       if (fold_convertible_p (req_type, value))
! 	value = fold_build1 (NOP_EXPR, req_type, value);
!       else if (TYPE_SIZE (req_type) == TYPE_SIZE (TREE_TYPE (value)))
! 	value = fold_build1 (VIEW_CONVERT_EXPR, req_type, value);
!       else
! 	{
! 	  if (dump_file)
! 	    {
! 	      fprintf (dump_file, "    const ");
! 	      print_generic_expr (dump_file, value, 0);
! 	      fprintf (dump_file, "  can't be converted to param ");
! 	      print_generic_expr (dump_file, parm, 0);
! 	      fprintf (dump_file, "\n");
! 	    }
! 	  return NULL;
! 	}
!     }
  
!   replace_map = ggc_alloc_ipa_replace_map ();
!   if (dump_file)
      {
!       fprintf (dump_file, "    replacing param ");
!       print_generic_expr (dump_file, parm, 0);
!       fprintf (dump_file, " with const ");
!       print_generic_expr (dump_file, value, 0);
!       fprintf (dump_file, "\n");
      }
+   replace_map->old_tree = parm;
+   replace_map->new_tree = value;
+   replace_map->replace_p = true;
+   replace_map->ref_p = false;
  
!   return replace_map;
  }
  
! /* Dump new profiling counts */
! 
  static void
! dump_profile_updates (struct cgraph_node *orig_node,
! 		      struct cgraph_node *new_node)
  {
!   struct cgraph_edge *cs;
  
!   fprintf (dump_file, "    setting count of the specialized node to "
! 	   HOST_WIDE_INT_PRINT_DEC "\n", (HOST_WIDE_INT) new_node->count);
!   for (cs = new_node->callees; cs ; cs = cs->next_callee)
!     fprintf (dump_file, "      edge to %s has count "
! 	     HOST_WIDE_INT_PRINT_DEC "\n",
! 	     cgraph_node_name (cs->callee), (HOST_WIDE_INT) cs->count);
! 
!   fprintf (dump_file, "    setting count of the original node to "
! 	   HOST_WIDE_INT_PRINT_DEC "\n", (HOST_WIDE_INT) orig_node->count);
!   for (cs = orig_node->callees; cs ; cs = cs->next_callee)
!     fprintf (dump_file, "      edge to %s is left with "
! 	     HOST_WIDE_INT_PRINT_DEC "\n",
! 	     cgraph_node_name (cs->callee), (HOST_WIDE_INT) cs->count);
! }
  
  
! /* After a specialized NEW_NODE version of ORIG_NODE has been created, update
!    their profile information to reflect this.  */
  
  static void
! update_profiling_info (struct cgraph_node *orig_node,
! 		       struct cgraph_node *new_node)
  {
    struct cgraph_edge *cs;
+   struct caller_statistics stats;
+   gcov_type new_sum, orig_sum;
+   gcov_type remainder, orig_node_count = orig_node->count;
+ 
+   if (orig_node_count == 0)
+     return;
+ 
+   init_caller_stats (&stats);
+   cgraph_for_node_and_aliases (orig_node, gather_caller_stats, &stats, false);
+   orig_sum = stats.count_sum;
+   init_caller_stats (&stats);
+   cgraph_for_node_and_aliases (new_node, gather_caller_stats, &stats, false);
+   new_sum = stats.count_sum;
  
!   if (orig_node_count < orig_sum + new_sum)
      {
!       if (dump_file)
! 	fprintf (dump_file, "    Problem: node %s/%i has too low count "
! 		 HOST_WIDE_INT_PRINT_DEC " while the sum of incoming "
! 		 "counts is " HOST_WIDE_INT_PRINT_DEC "\n",
! 		 cgraph_node_name (orig_node), orig_node->uid,
! 		 (HOST_WIDE_INT) orig_node_count,
! 		 (HOST_WIDE_INT) (orig_sum + new_sum));
  
!       orig_node_count = (orig_sum + new_sum) * 12 / 10;
!       if (dump_file)
! 	fprintf (dump_file, "      proceeding by pretending it was "
! 		 HOST_WIDE_INT_PRINT_DEC "\n",
! 		 (HOST_WIDE_INT) orig_node_count);
      }
+ 
+   new_node->count = new_sum;
+   remainder = orig_node_count - new_sum;
+   orig_node->count = remainder;
+ 
+   for (cs = new_node->callees; cs ; cs = cs->next_callee)
+     if (cs->frequency)
+       cs->count = cs->count * new_sum / orig_node_count;
+     else
+       cs->count = 0;
+ 
+   for (cs = orig_node->callees; cs ; cs = cs->next_callee)
+     cs->count = cs->count * remainder / orig_node_count;
+ 
+   if (dump_file)
+     dump_profile_updates (orig_node, new_node);
  }
  
! /* Update the respective profile of specialized NEW_NODE and the original
!    ORIG_NODE after additional edges with cumulative count sum REDIRECTED_SUM
!    have been redirected to the specialized version.  */
! 
! static void
! update_specialized_profile (struct cgraph_node *new_node,
! 			    struct cgraph_node *orig_node,
! 			    gcov_type redirected_sum)
  {
    struct cgraph_edge *cs;
!   gcov_type new_node_count, orig_node_count = orig_node->count;
  
!   if (dump_file)
!     fprintf (dump_file, "    the sum of counts of redirected  edges is "
! 	     HOST_WIDE_INT_PRINT_DEC "\n", (HOST_WIDE_INT) redirected_sum);
!   if (orig_node_count == 0)
!     return;
! 
!   gcc_assert (orig_node_count >= redirected_sum);
! 
!   new_node_count = new_node->count;
!   new_node->count += redirected_sum;
!   orig_node->count -= redirected_sum;
! 
!   for (cs = new_node->callees; cs ; cs = cs->next_callee)
!     if (cs->frequency)
!       cs->count += cs->count * redirected_sum / new_node_count;
      else
!       cs->count = 0;
  
!   for (cs = orig_node->callees; cs ; cs = cs->next_callee)
!     {
!       gcov_type dec = cs->count * redirected_sum / orig_node_count;
!       if (dec < cs->count)
! 	cs->count -= dec;
!       else
! 	cs->count = 0;
!     }
  
!   if (dump_file)
!     dump_profile_updates (orig_node, new_node);
! }
  
! /* Create a specialized version of NODE with known constants and types of
!    parameters in KNOWN_VALS and redirect all edges in CALLERS to it.  */
  
! static struct cgraph_node *
! create_specialized_node (struct cgraph_node *node,
! 			 VEC (tree, heap) *known_vals,
! 			 VEC (cgraph_edge_p,heap) *callers)
! {
!   struct ipa_node_params *new_info, *info = IPA_NODE_REF (node);
!   VEC (ipa_replace_map_p,gc)* replace_trees = NULL;
!   struct cgraph_node *new_node;
!   int i, count = ipa_get_param_count (info);
!   bitmap args_to_skip;
  
+   gcc_assert (!info->ipcp_orig_node);
+ 
+   if (node->local.can_change_signature)
+     {
+       args_to_skip = BITMAP_GGC_ALLOC ();
+       for (i = 0; i < count; i++)
+ 	{
+ 	  tree t = VEC_index (tree, known_vals, i);
  
! 	  if ((t && TREE_CODE (t) != TREE_BINFO)
! 	      || !ipa_is_param_used (info, i))
! 	    bitmap_set_bit (args_to_skip, i);
! 	}
!     }
!   else
!     args_to_skip = NULL;
  
!   for (i = 0; i < count ; i++)
      {
!       tree t = VEC_index (tree, known_vals, i);
!       if (t && TREE_CODE (t) != TREE_BINFO)
! 	{
! 	  struct ipa_replace_map *replace_map;
! 
! 	  replace_map = get_replacement_map (t, ipa_get_param (info, i));
! 	  if (replace_map)
! 	    VEC_safe_push (ipa_replace_map_p, gc, replace_trees, replace_map);
! 	}
      }
  
!   new_node = cgraph_create_virtual_clone (node, callers, replace_trees,
! 					  args_to_skip, "constprop");
!   if (dump_file && (dump_flags & TDF_DETAILS))
!     fprintf (dump_file, "     the new node is %s/%i.\n",
! 	     cgraph_node_name (new_node), new_node->uid);
!   gcc_checking_assert (ipa_node_params_vector
! 		       && (VEC_length (ipa_node_params_t,
! 				       ipa_node_params_vector)
! 			   > (unsigned) cgraph_max_uid));
!   update_profiling_info (node, new_node);
!   new_info = IPA_NODE_REF (new_node);
!   gcc_checking_assert (new_info->lattices);
!   new_info->ipcp_orig_node = node;
!   new_info->known_vals = known_vals;
  
!   ipcp_discover_new_direct_edges (new_node, known_vals);
! 
!   VEC_free (cgraph_edge_p, heap, callers);
!   return new_node;
  }
  
! /* Given a NODE, and a subset of its CALLERS, try to populate blanks slots in
!    KNOWN_VALS with constants and types that are also known for all of the
!    CALLERS.  */
  
  static void
! find_more_values_for_callers_subset (struct cgraph_node *node,
! 				     VEC (tree, heap) *known_vals,
! 				     VEC (cgraph_edge_p,heap) *callers)
  {
    struct ipa_node_params *info = IPA_NODE_REF (node);
!   int i, count = ipa_get_param_count (info);
  
!   for (i = 0; i < count ; i++)
      {
!       struct cgraph_edge *cs;
!       tree newval = NULL_TREE;
!       int j;
  
!       if (ipa_get_lattice (info, i)->bottom
! 	  || VEC_index (tree, known_vals, i))
  	continue;
  
!       FOR_EACH_VEC_ELT (cgraph_edge_p, callers, j, cs)
  	{
! 	  struct ipa_jump_func *jump_func;
! 	  tree t;
  
! 	  jump_func = ipa_get_ith_jump_func (IPA_EDGE_REF (cs), i);
  
! 	  t = ipa_value_from_jfunc (IPA_NODE_REF (cs->caller), jump_func);
! 	  if (!t
! 	      || (newval
! 		  && !values_equal_for_ipcp_p (t, newval)))
  	    {
! 	      newval = NULL_TREE;
! 	      break;
  	    }
+ 	  else
+ 	    newval = t;
  	}
  
!       if (newval)
! 	{
! 	  if (dump_file && (dump_flags & TDF_DETAILS))
! 	    {
! 	      fprintf (dump_file, "    adding an extra known value ");
! 	      print_ipcp_constant_value (dump_file, newval);
! 	      fprintf (dump_file, " for parameter ");
! 	      print_generic_expr (dump_file, ipa_get_param (info, i), 0);
! 	      fprintf (dump_file, "\n");
! 	    }
  
! 	  VEC_replace (tree, known_vals, i, newval);
! 	}
      }
  }
  
! /* Given an original NODE and a VAL for which we have already created a
!    specialized clone, look whether there are incoming edges that still lead
!    into the old node but now also bring the requested value and also conform to
!    all other criteria such that they can be redirected the the special node.
!    This function can therefore redirect the final edge in a SCC.  */
  
  static void
! perhaps_add_new_callers (struct cgraph_node *node, struct ipcp_value *val)
  {
!   struct ipa_node_params *dest_info = IPA_NODE_REF (val->spec_node);
!   struct ipcp_value_source *src;
!   int count = ipa_get_param_count (dest_info);
!   gcov_type redirected_sum = 0;
  
!   for (src = val->sources; src; src = src->next)
      {
!       struct cgraph_edge *cs = src->cs;
!       while (cs)
! 	{
! 	  enum availability availability;
! 	  bool insufficient = false;
  
! 	  if (cgraph_function_or_thunk_node (cs->callee, &availability) == node
! 	      && availability > AVAIL_OVERWRITABLE
! 	      && cgraph_edge_brings_value_p (cs, src))
! 	    {
! 	      struct ipa_node_params *caller_info;
! 	      struct ipa_edge_args *args;
! 	      int i;
! 
! 	      caller_info = IPA_NODE_REF (cs->caller);
! 	      args = IPA_EDGE_REF (cs);
! 	      for (i = 0; i < count; i++)
! 		{
! 		  struct ipa_jump_func *jump_func;
! 		  tree val, t;
  
! 		  val = VEC_index (tree, dest_info->known_vals, i);
! 		  if (!val)
! 		    continue;
! 
! 		  jump_func = ipa_get_ith_jump_func (args, i);
! 		  t = ipa_value_from_jfunc (caller_info, jump_func);
! 		  if (!t || !values_equal_for_ipcp_p (val, t))
! 		    {
! 		      insufficient = true;
! 		      break;
! 		    }
! 		}
! 
! 	      if (!insufficient)
! 		{
! 		  if (dump_file)
! 		    fprintf (dump_file, " - adding an extra caller %s/%i"
! 			     " of %s/%i\n",
! 			     cgraph_node_name (cs->caller), cs->caller->uid,
! 			     cgraph_node_name (val->spec_node),
! 			     val->spec_node->uid);
! 
! 		  cgraph_redirect_edge_callee (cs, val->spec_node);
! 		  redirected_sum += cs->count;
! 		}
! 	    }
! 	  cs = get_next_cgraph_edge_clone (cs);
! 	}
      }
+ 
+   if (redirected_sum)
+     update_specialized_profile (val->spec_node, node, redirected_sum);
  }
  
  
! /* Copy KNOWN_BINFOS to KNOWN_VALS.  */
! 
  static void
! move_binfos_to_values (VEC (tree, heap) *known_vals,
! 		       VEC (tree, heap) *known_binfos)
  {
!   tree t;
    int i;
  
!   for (i = 0; VEC_iterate (tree, known_binfos, i, t); i++)
!     if (t)
!       VEC_replace (tree, known_vals, i, t);
! }
  
  
! /* Decide whether and what specialized clones of NODE should be created.  */
  
! static bool
! decide_whether_version_node (struct cgraph_node *node)
! {
!   struct ipa_node_params *info = IPA_NODE_REF (node);
!   int i, count = ipa_get_param_count (info);
!   VEC (tree, heap) *known_csts, *known_binfos;
!   bool ret = false;
  
!   if (count == 0)
!     return false;
  
!   if (dump_file && (dump_flags & TDF_DETAILS))
!     fprintf (dump_file, "\nEvaluating opportunities for %s/%i.\n",
! 	     cgraph_node_name (node), node->uid);
  
!   gather_context_independent_values (info, &known_csts, &known_binfos,
! 				     NULL);
  
!   for (i = 0; i < count ; i++)
!     {
!       struct ipcp_lattice *lat = ipa_get_lattice (info, i);
!       struct ipcp_value *val;
  
!       if (lat->bottom
! 	  || VEC_index (tree, known_csts, i)
! 	  || VEC_index (tree, known_binfos, i))
! 	continue;
  
!       for (val = lat->values; val; val = val->next)
  	{
! 	  int freq_sum, caller_count;
! 	  gcov_type count_sum;
! 	  VEC (cgraph_edge_p, heap) *callers;
! 	  VEC (tree, heap) *kv;
  
! 	  if (val->spec_node)
! 	    {
! 	      perhaps_add_new_callers (node, val);
! 	      continue;
! 	    }
! 	  else if (val->local_size_cost + overall_size > max_new_size)
  	    {
! 	      if (dump_file && (dump_flags & TDF_DETAILS))
! 		fprintf (dump_file, "   Ignoring candidate value because "
! 			 "max_new_size would be reached with %li.\n",
! 			 val->local_size_cost + overall_size);
  	      continue;
  	    }
+ 	  else if (!get_info_about_necessary_edges (val, &freq_sum, &count_sum,
+ 						    &caller_count))
+ 	    continue;
  
! 	  if (dump_file && (dump_flags & TDF_DETAILS))
  	    {
! 	      fprintf (dump_file, " - considering value ");
! 	      print_ipcp_constant_value (dump_file, val->value);
! 	      fprintf (dump_file, " for parameter ");
! 	      print_generic_expr (dump_file, ipa_get_param (info, i), 0);
! 	      fprintf (dump_file, " (caller_count: %i)\n", caller_count);
  	    }
! 
! 
! 	  if (!good_cloning_opportunity_p (node, val->local_time_benefit,
! 					   freq_sum, count_sum,
! 					   val->local_size_cost)
! 	      && !good_cloning_opportunity_p (node,
! 					      val->local_time_benefit
! 					      + val->prop_time_benefit,
! 					      freq_sum, count_sum,
! 					      val->local_size_cost
! 					      + val->prop_size_cost))
! 	    continue;
! 
  	  if (dump_file)
! 	    fprintf (dump_file, "  Creating a specialized node of %s/%i.\n",
! 		     cgraph_node_name (node), node->uid);
  
! 	  callers = gather_edges_for_value (val, caller_count);
! 	  kv = VEC_copy (tree, heap, known_csts);
! 	  move_binfos_to_values (kv, known_binfos);
! 	  VEC_replace (tree, kv, i, val->value);
! 	  find_more_values_for_callers_subset (node, kv, callers);
! 	  val->spec_node = create_specialized_node (node, kv, callers);
! 	  overall_size += val->local_size_cost;
! 	  info = IPA_NODE_REF (node);
! 
! 	  /* TODO: If for some lattice there is only one other known value
! 	     left, make a special node for it too. */
! 	  ret = true;
  
! 	  VEC_replace (tree, kv, i, val->value);
! 	}
!     }
  
!   if (info->clone_for_all_contexts)
!     {
!       VEC (cgraph_edge_p, heap) *callers;
  
        if (dump_file)
! 	fprintf (dump_file, " * Creating a specialized node of %s/%i "
! 		 "for all contexts.\n", cgraph_node_name (node), node->uid);
  
+       callers = collect_callers_of_node (node);
+       move_binfos_to_values (known_csts, known_binfos);
+       create_specialized_node (node, known_csts, callers);
        info = IPA_NODE_REF (node);
!       info->clone_for_all_contexts = false;
!       ret = true;
!     }
!   else
!     VEC_free (tree, heap, known_csts);
  
!   VEC_free (tree, heap, known_binfos);
!   return ret;
! }
  
! /* Transitively mark all callees of NODE within the same SCC as not dead.  */
! 
! static void
! spread_undeadness (struct cgraph_node *node)
! {
!   struct cgraph_edge *cs;
! 
!   for (cs = node->callees; cs; cs = cs->next_callee)
!     if (edge_within_scc (cs))
!       {
! 	struct cgraph_node *callee;
! 	struct ipa_node_params *info;
! 
! 	callee = cgraph_function_or_thunk_node (cs->callee, NULL);
! 	info = IPA_NODE_REF (callee);
! 
! 	if (info->node_dead)
! 	  {
! 	    info->node_dead = 0;
! 	    spread_undeadness (callee);
! 	  }
!       }
! }
! 
! /* Return true if NODE has a caller from outside of its SCC that is not
!    dead.  Worker callback for cgraph_for_node_and_aliases.  */
! 
! static bool
! has_unded_caller_from_outside_scc_p (struct cgraph_node *node,
! 				     void *data ATTRIBUTE_UNUSED)
! {
!   struct cgraph_edge *cs;
! 
!   for (cs = node->callers; cs; cs = cs->next_caller)
!     if (cs->caller->thunk.thunk_p
! 	&& has_unded_caller_from_outside_scc_p (cs->caller, NULL))
!       return true;
!     else if (!edge_within_scc (cs)
! 	     && !IPA_NODE_REF (cs->caller)->node_dead)
!       return true;
!   return false;
! }
! 
! 
! /* Identify nodes within the same SCC as NODE which are no longer needed
!    because of new clones and will be removed as unreachable.  */
! 
! static void
! identify_dead_nodes (struct cgraph_node *node)
! {
!   struct cgraph_node *v;
!   for (v = node; v ; v = ((struct ipa_dfs_info *) v->aux)->next_cycle)
!     if (cgraph_will_be_removed_from_program_if_no_direct_calls (v)
! 	&& !cgraph_for_node_and_aliases (v,
! 					 has_unded_caller_from_outside_scc_p,
! 					 NULL, true))
!       IPA_NODE_REF (v)->node_dead = 1;
! 
!   for (v = node; v ; v = ((struct ipa_dfs_info *) v->aux)->next_cycle)
!     if (!IPA_NODE_REF (v)->node_dead)
!       spread_undeadness (v);
! 
!   if (dump_file && (dump_flags & TDF_DETAILS))
!     {
!       for (v = node; v ; v = ((struct ipa_dfs_info *) v->aux)->next_cycle)
! 	if (IPA_NODE_REF (v)->node_dead)
! 	  fprintf (dump_file, "  Marking node as dead: %s/%i.\n",
! 		   cgraph_node_name (v), v->uid);
      }
+ }
+ 
+ /* The decision stage.  Iterate over the topological order of call graph nodes
+    TOPO and make specialized clones if deemed beneficial.  */
+ 
+ static void
+ ipcp_decision_stage (struct topo_info *topo)
+ {
+   int i;
  
!   if (dump_file)
!     fprintf (dump_file, "\nIPA decision stage:\n\n");
! 
!   for (i = topo->nnodes - 1; i >= 0; i--)
      {
!       struct cgraph_node *node = topo->order[i];
!       bool change = false, iterate = true;
! 
!       while (iterate)
! 	{
! 	  struct cgraph_node *v;
! 	  iterate = false;
! 	  for (v = node; v ; v = ((struct ipa_dfs_info *) v->aux)->next_cycle)
! 	    if (cgraph_function_with_gimple_body_p (v)
! 		&& ipcp_versionable_function_p (v))
! 	      iterate |= decide_whether_version_node (v);
! 
! 	  change |= iterate;
! 	}
!       if (change)
! 	identify_dead_nodes (node);
      }
  }
  
  /* The IPCP driver.  */
+ 
  static unsigned int
  ipcp_driver (void)
  {
+   struct topo_info topo;
+ 
    cgraph_remove_unreachable_nodes (true,dump_file);
+   ipa_check_create_node_params ();
+   ipa_check_create_edge_args ();
+   ipcp_values_pool = create_alloc_pool ("IPA-CP values",
+ 					sizeof (struct ipcp_value), 32);
+   ipcp_sources_pool = create_alloc_pool ("IPA-CP value sources",
+ 					 sizeof (struct ipcp_value_source), 64);
+ 
    if (dump_file)
      {
        fprintf (dump_file, "\nIPA structures before propagation:\n");
*************** ipcp_driver (void)
*** 1501,1518 ****
          ipa_print_all_params (dump_file);
        ipa_print_all_jump_functions (dump_file);
      }
!   ipa_check_create_node_params ();
!   ipa_check_create_edge_args ();
    /* 2. Do the interprocedural propagation.  */
!   ipcp_iterate_stage ();
!   /* 3. Insert the constants found to the functions.  */
!   ipcp_insert_stage ();
!   if (dump_file && (dump_flags & TDF_DETAILS))
!     {
!       fprintf (dump_file, "\nProfiling info after insert stage:\n");
!       ipcp_print_profile_data (dump_file);
!     }
    /* Free all IPCP structures.  */
    ipa_free_all_structures_after_ipa_cp ();
    if (dump_file)
      fprintf (dump_file, "\nIPA constant propagation end\n");
--- 2130,2145 ----
          ipa_print_all_params (dump_file);
        ipa_print_all_jump_functions (dump_file);
      }
! 
!   build_topo_info (&topo);
    /* 2. Do the interprocedural propagation.  */
!   ipcp_propagate_stage (&topo);
!   /* 3. Decide what constant propagation and cloning should be performed.  */
!   ipcp_decision_stage (&topo);
! 
    /* Free all IPCP structures.  */
+   free_topo_info (&topo);
+ 
    ipa_free_all_structures_after_ipa_cp ();
    if (dump_file)
      fprintf (dump_file, "\nIPA constant propagation end\n");
*************** ipcp_generate_summary (void)
*** 1545,1550 ****
--- 2172,2178 ----
  }
  
  /* Write ipcp summary for nodes in SET.  */
+ 
  static void
  ipcp_write_summary (cgraph_node_set set,
  		    varpool_node_set vset ATTRIBUTE_UNUSED)
*************** ipcp_write_summary (cgraph_node_set set,
*** 1553,1558 ****
--- 2181,2187 ----
  }
  
  /* Read ipcp summary.  */
+ 
  static void
  ipcp_read_summary (void)
  {
*************** ipcp_read_summary (void)
*** 1560,1565 ****
--- 2189,2195 ----
  }
  
  /* Gate for IPCP optimization.  */
+ 
  static bool
  cgraph_gate_cp (void)
  {
Index: src/gcc/testsuite/gcc.dg/ipa/ipa-1.c
===================================================================
*** src.orig/gcc/testsuite/gcc.dg/ipa/ipa-1.c
--- src/gcc/testsuite/gcc.dg/ipa/ipa-1.c
*************** int main ()
*** 24,32 ****
  }
  
  
! /* { dg-final { scan-ipa-dump-times "versioned function" 2 "cp"  } } */
  /* { dg-final { scan-ipa-dump "replacing param a with const 7" "cp"  } } */
- /* { dg-final { scan-ipa-dump "replacing param b with const 7" "cp"  } } */
  /* { dg-final { cleanup-ipa-dump "cp" } } */
  
  
--- 24,31 ----
  }
  
  
! /* { dg-final { scan-ipa-dump "Creating a specialized node of f" "cp" } } */
  /* { dg-final { scan-ipa-dump "replacing param a with const 7" "cp"  } } */
  /* { dg-final { cleanup-ipa-dump "cp" } } */
  
  
Index: src/gcc/testsuite/gcc.dg/ipa/ipa-2.c
===================================================================
*** src.orig/gcc/testsuite/gcc.dg/ipa/ipa-2.c
--- src/gcc/testsuite/gcc.dg/ipa/ipa-2.c
*************** int main ()
*** 22,28 ****
  }
  
  
! /* { dg-final { scan-ipa-dump-times "versioned function" 2 "cp"  } } */
  /* { dg-final { scan-ipa-dump "replacing param a with const 7" "cp"  } } */
- /* { dg-final { scan-ipa-dump "replacing param c with const 3" "cp"  } } */
  /* { dg-final { cleanup-ipa-dump "cp" } } */
--- 22,27 ----
  }
  
  
! /* { dg-final { scan-ipa-dump "Creating a specialized node of f" "cp" } } */
  /* { dg-final { scan-ipa-dump "replacing param a with const 7" "cp"  } } */
  /* { dg-final { cleanup-ipa-dump "cp" } } */
Index: src/gcc/testsuite/gcc.dg/ipa/ipa-3.c
===================================================================
*** src.orig/gcc/testsuite/gcc.dg/ipa/ipa-3.c
--- src/gcc/testsuite/gcc.dg/ipa/ipa-3.c
***************
*** 7,18 ****
  
  #include <stdio.h>
  void t(void);
! int g (double b, double c)
  {
    t();
    return (int)(b+c);
  }
! int f (double a)
  {
    if (a > 0)
      g (a, 3.1);
--- 7,18 ----
  
  #include <stdio.h>
  void t(void);
! static int g (double b, double c)
  {
    t();
    return (int)(b+c);
  }
! static int f (double a)
  {
    if (a > 0)
      g (a, 3.1);
*************** int main ()
*** 28,35 ****
  }
  
  
! /* { dg-final { scan-ipa-dump-times "versioned function" 2 "cp"  } } */
  /* { dg-final { scan-ipa-dump "replacing param a with const 7" "cp"  } } */
  /* { dg-final { scan-ipa-dump "replacing param b with const 7" "cp"  } } */
  /* { dg-final { scan-ipa-dump "replacing param c with const 3" "cp"  } } */
  /* { dg-final { cleanup-ipa-dump "cp" } } */
--- 28,36 ----
  }
  
  
! /* { dg-final { scan-ipa-dump "Creating a specialized node of f" "cp" } } */
  /* { dg-final { scan-ipa-dump "replacing param a with const 7" "cp"  } } */
+ /* { dg-final { scan-ipa-dump "Creating a specialized node of g" "cp" } } */
  /* { dg-final { scan-ipa-dump "replacing param b with const 7" "cp"  } } */
  /* { dg-final { scan-ipa-dump "replacing param c with const 3" "cp"  } } */
  /* { dg-final { cleanup-ipa-dump "cp" } } */
Index: src/gcc/testsuite/gcc.dg/ipa/ipa-4.c
===================================================================
*** src.orig/gcc/testsuite/gcc.dg/ipa/ipa-4.c
--- src/gcc/testsuite/gcc.dg/ipa/ipa-4.c
*************** int main ()
*** 25,30 ****
  }
  
  
! /* { dg-final { scan-ipa-dump-times "versioned function" 1 "cp"  } } */
  /* { dg-final { scan-ipa-dump-times "replacing param a with const 7" 1 "cp"  } } */
  /* { dg-final { cleanup-ipa-dump "cp" } } */
--- 25,30 ----
  }
  
  
! /* { dg-final { scan-ipa-dump "Creating a specialized node of f" "cp"  } } */
  /* { dg-final { scan-ipa-dump-times "replacing param a with const 7" 1 "cp"  } } */
  /* { dg-final { cleanup-ipa-dump "cp" } } */
Index: src/gcc/testsuite/gcc.dg/ipa/ipa-5.c
===================================================================
*** src.orig/gcc/testsuite/gcc.dg/ipa/ipa-5.c
--- src/gcc/testsuite/gcc.dg/ipa/ipa-5.c
*************** int main ()
*** 26,33 ****
    return 0;
  }
  
! 
! /* { dg-final { scan-ipa-dump-times "versioned function" 2 "cp"  } } */
  /* { dg-final { scan-ipa-dump "replacing param c with const 3" "cp"  } } */
  /* { dg-final { scan-ipa-dump "replacing param a with const 7" "cp"  } } */
  /* { dg-final { cleanup-ipa-dump "cp" } } */
--- 26,32 ----
    return 0;
  }
  
! /* { dg-final { scan-ipa-dump-times "Creating a specialized node" 2 "cp"  } } */
  /* { dg-final { scan-ipa-dump "replacing param c with const 3" "cp"  } } */
  /* { dg-final { scan-ipa-dump "replacing param a with const 7" "cp"  } } */
  /* { dg-final { cleanup-ipa-dump "cp" } } */
Index: src/gcc/testsuite/gcc.dg/ipa/ipa-7.c
===================================================================
*** src.orig/gcc/testsuite/gcc.dg/ipa/ipa-7.c
--- src/gcc/testsuite/gcc.dg/ipa/ipa-7.c
*************** int main ()
*** 26,33 ****
  }
  
  
! /* { dg-final { scan-ipa-dump-times "versioned function" 1 "cp"  } } */
! /* { dg-final { scan-ipa-dump-times "replacing param a with const 7" 1 "cp"  } } */
  /* { dg-final { cleanup-ipa-dump "cp" } } */
  
  
--- 26,33 ----
  }
  
  
! /* { dg-final { scan-ipa-dump "Creating a specialized node of f" "cp" { xfail *-*-* } } } */
! /* { dg-final { scan-ipa-dump-times "replacing param . with const 7" 1 "cp" { xfail *-*-* } } } */
  /* { dg-final { cleanup-ipa-dump "cp" } } */
  
  
Index: src/gcc/testsuite/gcc.dg/ipa/ipa-8.c
===================================================================
*** src.orig/gcc/testsuite/gcc.dg/ipa/ipa-8.c
--- src/gcc/testsuite/gcc.dg/ipa/ipa-8.c
*************** int main ()
*** 22,29 ****
  }
  
  
! /* { dg-final { scan-ipa-dump-times "versioned function" 2 "cp"  } } */
  /* { dg-final { scan-ipa-dump "replacing param a with const 7" "cp"  } } */
  /* { dg-final { scan-ipa-dump "replacing param b with const 7" "cp"  } } */
  /* { dg-final { cleanup-ipa-dump "cp" } } */
  
--- 22,30 ----
  }
  
  
! /* { dg-final { scan-ipa-dump "Creating a specialized node of f" "cp" } } */
  /* { dg-final { scan-ipa-dump "replacing param a with const 7" "cp"  } } */
+ /* { dg-final { scan-ipa-dump "Creating a specialized node of g" "cp" } } */
  /* { dg-final { scan-ipa-dump "replacing param b with const 7" "cp"  } } */
  /* { dg-final { cleanup-ipa-dump "cp" } } */
  
Index: src/gcc/testsuite/gcc.dg/ipa/ipacost-1.c
===================================================================
*** src.orig/gcc/testsuite/gcc.dg/ipa/ipacost-1.c
--- src/gcc/testsuite/gcc.dg/ipa/ipacost-1.c
*************** main()
*** 51,60 ****
    i_can_not_be_propagated_fully2 (array);
  }
  
! /* { dg-final { scan-ipa-dump-times "versioned function i_can_be_propagated_fully2" 1 "cp"  } } */
! /* { dg-final { scan-ipa-dump-times "versioned function i_can_be_propagated_fully " 1 "cp"  } } */
! /* { dg-final { scan-ipa-dump-not "versioned function i_can_not_be_propagated_fully2" "cp"  } } */
! /* { dg-final { scan-ipa-dump-not "versioned function i_can_not_be_propagated_fully " "cp"  } } */
  /* { dg-final { scan-tree-dump-not "i_can_be_propagated_fully " "optimized"  } } */
  /* { dg-final { scan-tree-dump-not "i_can_be_propagated_fully2 " "optimized"  } } */
  /* { dg-final { cleanup-ipa-dump "cp" } } */
--- 51,60 ----
    i_can_not_be_propagated_fully2 (array);
  }
  
! /* { dg-final { scan-ipa-dump-times "Creating a specialized node of i_can_be_propagated_fully2" 1 "cp"  } } */
! /* { dg-final { scan-ipa-dump-times "Creating a specialized node of i_can_be_propagated_fully/" 1 "cp"  } } */
! /* { dg-final { scan-ipa-dump-not "Creating a specialized node of i_can_not_be_propagated_fully2" "cp"  } } */
! /* { dg-final { scan-ipa-dump-not "Creating a specialized node of i_can_not_be_propagated_fully/" "cp"  } } */
  /* { dg-final { scan-tree-dump-not "i_can_be_propagated_fully " "optimized"  } } */
  /* { dg-final { scan-tree-dump-not "i_can_be_propagated_fully2 " "optimized"  } } */
  /* { dg-final { cleanup-ipa-dump "cp" } } */
Index: src/gcc/testsuite/gcc.dg/ipa/ipacost-2.c
===================================================================
*** src.orig/gcc/testsuite/gcc.dg/ipa/ipacost-2.c
--- src/gcc/testsuite/gcc.dg/ipa/ipacost-2.c
*************** main()
*** 55,64 ****
    i_can_not_be_propagated_fully2 (array);
  }
  
! /* { dg-final { scan-ipa-dump-times "versioned function i_can_be_propagated_fully2" 1 "cp"  } } */
! /* { dg-final { scan-ipa-dump-times "versioned function i_can_be_propagated_fully " 1 "cp"  } } */
! /* { dg-final { scan-ipa-dump-times "versioned function i_can_not_be_propagated_fully2" 1 "cp"  } } */
! /* { dg-final { scan-ipa-dump-times "versioned function i_can_not_be_propagated_fully " 1 "cp"  } } */
  /* { dg-final { scan-tree-dump-not "i_can_be_propagated_fully \\(" "optimized"  } } */
  /* { dg-final { scan-tree-dump-not "i_can_be_propagated_fully2 \\(" "optimized"  } } */
  /* { dg-final { cleanup-ipa-dump "cp" } } */
--- 55,64 ----
    i_can_not_be_propagated_fully2 (array);
  }
  
! /* { dg-final { scan-ipa-dump-times "Creating a specialized node of i_can_be_propagated_fully2" 1 "cp"  } } */
! /* { dg-final { scan-ipa-dump-times "Creating a specialized node of i_can_be_propagated_fully/" 1 "cp"  } } */
! /* { dg-final { scan-ipa-dump-not "Creating a specialized node of i_can_not_be_propagated_fully2" "cp"  } } */
! /* { dg-final { scan-ipa-dump-not "Creating a specialized node of i_can_not_be_propagated_fully/" "cp"  } } */
  /* { dg-final { scan-tree-dump-not "i_can_be_propagated_fully \\(" "optimized"  } } */
  /* { dg-final { scan-tree-dump-not "i_can_be_propagated_fully2 \\(" "optimized"  } } */
  /* { dg-final { cleanup-ipa-dump "cp" } } */
Index: src/gcc/testsuite/gcc.dg/ipa/ipcp-1.c
===================================================================
*** /dev/null
--- src/gcc/testsuite/gcc.dg/ipa/ipcp-1.c
***************
*** 0 ****
--- 1,52 ----
+ /* Test that IPA-CP is able to figure out that poth parameters a are constant 7
+    even though f and h recursively call each other and specialize them
+    accordinly.  */
+ 
+ /* { dg-do compile } */
+ /* { dg-options "-O3 -fipa-cp -fipa-cp-clone -fdump-ipa-cp -fno-early-inlining"  } */
+ /* { dg-add-options bind_pic_locally } */
+ 
+ extern void use_stuff (int);
+ 
+ static
+ int g (int b, int c)
+ {
+   int i;
+ 
+   for (i = 0; i < b; i++)
+     use_stuff (c);
+ }
+ 
+ static void f (int a, int x, int z);
+ 
+ static void h (int z, int a)
+ {
+   use_stuff (z);
+   f (a, 9, 10);
+ 
+ }
+ 
+ static void
+ f (int a, int x, int z)
+ {
+   if (z > 1)
+     g (a, x);
+   else
+     h (5, a);
+ }
+ 
+ int
+ main (int argc, char *argv[])
+ {
+   int i;
+   for (i = 0; i < 100; i++)
+     f (7, 8, argc);
+   return 0;
+ }
+ 
+ 
+ /* { dg-final { scan-ipa-dump "Creating a specialized node of f.*for all contexts" "cp" } } */
+ /* { dg-final { scan-ipa-dump "replacing param a with const 7" "cp"  } } */
+ /* { dg-final { cleanup-ipa-dump "cp" } } */
+ 
+ 
Index: src/gcc/testsuite/gcc.dg/ipa/ipcp-2.c
===================================================================
*** /dev/null
--- src/gcc/testsuite/gcc.dg/ipa/ipcp-2.c
***************
*** 0 ****
--- 1,99 ----
+ /* { dg-do compile } */
+ /* { dg-options "-O3 -fipa-cp -fipa-cp-clone -fdump-ipa-cp -fno-early-inlining"  } */
+ /* { dg-add-options bind_pic_locally } */
+ 
+ extern int get_stuff (int);
+ extern void do_stuff (int);
+ extern void do_stuff2 (int);
+ extern void do_other_stuff (void);
+ extern int get_element (int, int, int);
+ extern int adjust (int, int, int, int);
+ 
+ extern int count;
+ 
+ int
+ foo (int s, int p)
+ {
+   int c, r = 0;
+ 
+   for (c = 0 ; c < count; c++)
+     {
+       r += get_stuff (s);
+       /* The following is just something big that can go away.  */
+       if (p != 0)
+ 	{
+ 	  int a[64][64];
+ 	  int i, j, k;
+ 
+ 	  for (i = 0; i < 64; i++)
+ 	    for (j = 0; j < 64; j++)
+ 	      a[i][j] = get_element (p + c, i, j);
+ 
+ 	  for (k = 0; k < 4; k++)
+ 	    {
+ 	      r = r / 2;
+ 
+ 	      for (i = 1; i < 63; i++)
+ 		for (j = 62; j > 0; j--)
+ 		  a[i][j] += adjust (a[i-1][j], a[i][j-1],
+ 				     a[i+1][j], a[i][j+1]);
+ 
+ 	      for (i = 4; i < 64; i += 4)
+ 		for (j = 4; j < 64; j += 4)
+ 		  r += a[i][j] / 4;
+ 	    }
+ 	}
+     }
+   return r;
+ }
+ 
+ int
+ bar (int p, int q)
+ {
+   if (q > 0)
+     do_stuff (q);
+   else
+     do_stuff (-q);
+ 
+   if (q % 2)
+     do_stuff2 (2 * q);
+   else
+     do_stuff2 (2 * (q + 1));
+ 
+   return foo (4, p);
+ }
+ 
+ int
+ bah (int p, int q)
+ {
+   int i, j;
+ 
+   while (q < -20)
+     q += get_stuff (-q);
+ 
+   for (i = 0; i < 36; i++)
+     for (j = 0; j < 36; j++)
+       do_stuff (get_stuff (q * i + 2));
+ 
+   bar (p, q);
+ }
+ 
+ int
+ top1 (int q)
+ {
+   do_other_stuff ();
+   return bah (0, q);
+ }
+ 
+ int
+ top2 (int q)
+ {
+   do_stuff (200);
+   do_other_stuff ();
+   return bah (16, q);
+ }
+ 
+ /* { dg-final { scan-ipa-dump-times "Creating a specialized node of foo" 1 "cp" } } */
+ /* { dg-final { scan-ipa-dump-times "replacing param p with const 0" 3 "cp"  } } */
+ /* { dg-final { scan-ipa-dump "replacing param s with const 4" "cp"  } } */
+ /* { dg-final { cleanup-ipa-dump "cp" } } */
Index: src/gcc/testsuite/gcc.dg/tree-ssa/ipa-cp-1.c
===================================================================
*** src.orig/gcc/testsuite/gcc.dg/tree-ssa/ipa-cp-1.c
--- src/gcc/testsuite/gcc.dg/tree-ssa/ipa-cp-1.c
***************
*** 5,13 ****
  int
  very_long_function(int a)
  {
!   return very_long_function (a)/4;
  }
! main()
  {
    very_long_function (1);
  }
--- 5,17 ----
  int
  very_long_function(int a)
  {
!   if (a > 0)
!     return 2 * a + very_long_function (a)/4;
!   else
!     return 2 * -a + very_long_function (a)/4;
  }
! 
! blah ()
  {
    very_long_function (1);
  }
Index: src/gcc/doc/invoke.texi
===================================================================
*** src.orig/gcc/doc/invoke.texi
--- src/gcc/doc/invoke.texi
*************** loop in the loop nest by a given number
*** 8999,9009 ****
  length can be changed using the @option{loop-block-tile-size}
  parameter.  The default value is 51 iterations.
  
! @item devirt-type-list-size
! IPA-CP attempts to track all possible types passed to a function's
! parameter in order to perform devirtualization.
! @option{devirt-type-list-size} is the maximum number of types it
! stores per a single formal parameter of a function.
  
  @item lto-partitions
  Specify desired number of partitions produced during WHOPR compilation.
--- 8999,9009 ----
  length can be changed using the @option{loop-block-tile-size}
  parameter.  The default value is 51 iterations.
  
! @item ipa-cp-value-list-size
! IPA-CP attempts to track all possible values and types passed to a function's
! parameter in order to propagate them and perform devirtualization.
! @option{ipa-cp-value-list-size} is the maximum number of values and types it
! stores per one formal parameter of a function.
  
  @item lto-partitions
  Specify desired number of partitions produced during WHOPR compilation.
Index: src/gcc/params.def
===================================================================
*** src.orig/gcc/params.def
--- src/gcc/params.def
*************** DEFPARAM (PARAM_IPA_SRA_PTR_GROWTH_FACTO
*** 860,869 ****
  	  "a pointer to an aggregate with",
  	  2, 0, 0)
  
! DEFPARAM (PARAM_DEVIRT_TYPE_LIST_SIZE,
! 	  "devirt-type-list-size",
! 	  "Maximum size of a type list associated with each parameter for "
! 	  "devirtualization",
  	  8, 0, 0)
  
  /* WHOPR partitioning configuration.  */
--- 860,869 ----
  	  "a pointer to an aggregate with",
  	  2, 0, 0)
  
! DEFPARAM (PARAM_IPA_CP_VALUE_LIST_SIZE,
! 	  "ipa-cp-value-list-size",
! 	  "Maximum size of a list of values associated with each parameter for "
! 	  "interprocedural constant propagation",
  	  8, 0, 0)
  
  /* WHOPR partitioning configuration.  */
Index: src/gcc/Makefile.in
===================================================================
*** src.orig/gcc/Makefile.in
--- src/gcc/Makefile.in
*************** LTO_STREAMER_H = lto-streamer.h $(LINKER
*** 1009,1015 ****
  		$(CGRAPH_H) $(VEC_H) vecprim.h $(TREE_H) $(GIMPLE_H) \
  		$(GCOV_IO_H)
  TREE_VECTORIZER_H = tree-vectorizer.h $(TREE_DATA_REF_H)
! IPA_PROP_H = ipa-prop.h $(TREE_H) $(VEC_H) $(CGRAPH_H) $(GIMPLE_H)
  GSTAB_H = gstab.h stab.def
  BITMAP_H = bitmap.h $(HASHTAB_H) statistics.h
  GCC_PLUGIN_H = gcc-plugin.h highlev-plugin-common.h $(CONFIG_H) $(SYSTEM_H) \
--- 1009,1015 ----
  		$(CGRAPH_H) $(VEC_H) vecprim.h $(TREE_H) $(GIMPLE_H) \
  		$(GCOV_IO_H)
  TREE_VECTORIZER_H = tree-vectorizer.h $(TREE_DATA_REF_H)
! IPA_PROP_H = ipa-prop.h $(TREE_H) $(VEC_H) $(CGRAPH_H) $(GIMPLE_H) alloc-pool.h
  GSTAB_H = gstab.h stab.def
  BITMAP_H = bitmap.h $(HASHTAB_H) statistics.h
  GCC_PLUGIN_H = gcc-plugin.h highlev-plugin-common.h $(CONFIG_H) $(SYSTEM_H) \