| Message ID | 20120702161230.GA9622@virgil.arch.suse.de |
|---|---|
| State | New |
| Headers | show |
On Mon, 2 Jul 2012, Martin Jambor wrote: > Hi, > > this is second version of patch introducing aggregate jump functions. > For various reasons, I have merged two patches and so this one also > implements indirect inlining based on them. One of the reasons is > that indirect inlining can lead to miscompilations (as opposed to > inlining heuristics) and so it's easier to make testcases for that. Finally taking my time to review this ... > The patch introduces two changes to jump function creation: > > 1) we gather information about known compile time IPA-invariants at > various offsets of an aggregate passed to a callee either by > reference (when the parameter is pointer) or by value (when it is > an aggregate). This patch simply scans the current BB backwards > from the call statement and examines what values are stored there. > This simple approach (in function determine_known_aggregate_parts) > is sufficient for Fortran array descripttors, I intend to make it > more sophisticated in future. It's probably also a good measure against compile-time issues. > When we do this, we also record the type through which data was > stored into the aggregate, which is either the type of a DECL or > type stored in the offset of a MEM_REF. This is slightly more > strict than necessary but I'd need that extra field there for jump > functions (see below). So for each { object, offset, data } triple you store the type of data (implicitely?) and the type of object? What you need is the type of the access (which should be compatible to the type of data which you have implicitely) and the type that determines the effective alias-set of the store (reference_alias_ptr_type). I don't see why you should special-case a DECL here. As for the object type, for MEM = CONST; useless_type_conversion_p (TREE_TYPE (MEM), TREE_TYPE (CONST)) holds, but as this function is not transitive the opposite relation needed for the corresponding load ... = MEM; in the called function may not hold (it's probably non-trivial to construct a testcase for this nowadays - maybe useless_type_conversion_p is sufficiently transitive these days, it just is designed not to be). > 2) Being able to conservatively but usefully recognize that an > aggregate (passed either by reference or a value) that we got from > a caller and pass it to a callee has not changed. > > This is slightly complex in cases where aggregates are passed by > reference. Because in gimple pointer types carry more-or-less no > information about the data it points to, we'd normally have to feed > AA with a type that aliases all in order to get conservatively > correct results. That might however be too conservative. Did you verify how "too conservative" this is? Note that if you have obj.a = 1; obj.b = 2; foo (&obj); then the two stores may have two different effective AA pointer types (unless you require that they have the same one during analysis and if not, fall back to a ref-all pointer) > We circumvent this problem by feeding AA the pointed-to type but also > verifying that the aggregate data was read with the same type > (modulo TYPE_MAIN_VARIANT and sub-structures). This consists of: > > a) checking that the alias type of memory reads from the aggregates > is the same as the declared pointed-to type of the parameter. > For example, the load in function g1 in testcase iinline-5.c > clearly does not pass this test. > > b) when aggregate values are propagated along a pass-through or > ancestor jump functions, we check that the declared types in > both caller and callee match and refuse to propagate if they > don't. The parameter passing in f2->g2 in the same testcase > does not pass this test. I think both a) and b) can be simplified by verifying that get_deref_alias_set returns the same alias-set for all involved AA pointer types. You can use any AA pointer type with the same get_deref_alias_set () alias-set for accesses. > c) before we propagate values into their eventual use, we check > that the type of the store matches the type of the use. This is > strictly necessary only when the jump function with values is a > result of update_jump_functions_after_inlining combining > aggregate values with a pass-through function, and the > pass-through component requires theat we do the type check (we > miscompile main->f2->g2 without this check). So this is about required value-conversions, right? Comparing access sizes (you have matching offsets) and on type incompatibility using a VIEW_CONVERT_EXPR should be able to relax this (if it is worth, works only for non-bitfield-like types). > So, for example, when we have functions A->B->C where A stores > something into a structure which is then passed to B, which passes > is to C where the data is read and called. We perform the > following: > > - We remember what we stored to the structure in A. > > - We use alias analysis with the type of the parameter to check > that the pointed-to structure is not modified in B. Hmm. So what happens to void baz (void *p) { if (*(float *)p != 1.) abort (); } void bar (void *p) { (float *)p = 1.; baz (p); } void foo () { int i = 2; bar (&i); } ? I suppose this is catched by c)? As I think you do not notice that bar() modifies i (possibly this is the reason why you needed c) in the first place?) In the GIMPLE memory model a store to a memory location may change its dynamic type (making a read of the old dynamic type undefined, so in the above case baz cannot use *(int *)p without invoking undefined behavior). With that said - what is "the type of the parameter" in bar you use that *p is not modified? I suppose it's the AA type stored in the jump-function for the parameter? > - We note the use in C and make sure that is is read with the > same alias type as is the declared type. We use AA with that > type to check that the structure is not modified before use in C. Why not use the AA type of the jump function? Ah - you do not have that readily available when you do local analysis and this is not only done at IPA transform time? > - If we first inline A->B and then B->C: In each step we move the > aggregate contents info but in each step we also check that the > type of stored information matches the formal parameter of > callee (this is important because we used that type for AA in > both B and C. Hmm, I see. Indeed somewhat kludgy, but I suppose changing inline decisions at IPA transform time would be bad ;) As above you could relax this by ensuring same alias-sets are used (hm, beware of mixing -fno-strict-aliasing and -fstrict-aliasing units ...). At least for AA type compatibility you can use TYPE_CANONICAL (TYPE_MAIN_VARIANT (t)). That should get you an effective alias-set handling with LTO (and without LTO not make a difference). Mind that you cannot store/combine alias-sets across WPA/LTRANS, they are re-computed. > - if we first inline B->C, we check that their parameters match > and if so, adjust the information about loads from the > structure to refer to parameters of B. Inlining A->B is then > handled like above. In both cases, the used AA type is > guaranteed to correspond to the load type (and store type too). > > The data structures can also represent such pass-through functions > with known listed exceptions but that is not currently implemented > and is left for later. I also understand that eventually the alias > vdef walking should be limited somehow (also, results should be > shared with inlining). > > The patch passes bootstrap and testing on x86_64-linux. > I'll be grateful for any comments and suggestions. Leaving the actual patch review until after your comments on the above. Thanks, Richard. > Martin > > > 2012-06-28 Martin Jambor <mjambor@suse.cz> > > * cgraph.h (cgraph_indirect_call_info): Field anc_offse renamd to > offset, updated all users. New field agg_contents. > * ipa-prop.h (jump_func_type): Removed IPA_JF_CONST_MEMBER_PTR. > (ipa_pass_through_data): New field agg_preserved. > (ipa_ancestor_jf_data): Likewise. > (ipa_member_ptr_cst): Removed. > (ipa_agg_jf_item): New type. > (ipa_agg_jump_function): Likewise. > (ipa_jump_func): New field agg. Removed field member_cst. > (ipa_get_jf_pass_through_agg_preserved): New function. > (ipa_get_jf_ancestor_agg_preserved): Likewise. > (ipa_get_jf_member_ptr_pfn): Removed. > (ipa_find_agg_cst_for_param): Declare. > (ipa_load_from_parm_agg): Likewise. > (ipa_jf_propagates_agg_p): Likewise. > > * ipa-prop.c (param_analysis_info): Fields modified and > visited_statements rename to parm_modified and parm_visited_statements > respectively, added fields ref_modified and ref_visited_statements. > (ipa_print_node_jump_functions_for_edge): Do not dumo const member > functions. Dump agg_preserved flags and aggregate jump functions. > (ipa_set_jf_simple_pass_through): Set also agg_preserved. > (ipa_set_ancestor_jf): Likewise. > (ipa_set_jf_arith_pass_through): Clear agg_preserved. > (ipa_set_jf_member_ptr_cst): Removed. > (is_parm_modified_before_stmt): Logic reversed, renamed to > is_parm_preserved_before_stmt. All callers updated. > (is_parm_ref_data_preserved): New function. > (is_parm_ref_data_preserved_call): Likewise. > (ipa_load_from_parm_agg_1): Likewise. > (ipa_load_from_parm_agg): Likewise. > (compute_complex_assign_jump_func): Check if aggregate contents are > preserved. > (compute_complex_ancestor_jump_func): Likewise. > (compute_scalar_jump_functions): Removed. > (type_like_member_ptr_p): Also check field position are known and > sane. > (compute_pass_through_member_ptrs): Removed. > (determine_cst_member_ptr): Likewise. > (init_ao_ref_for_byref_agg_jf): New function. > (ipa_known_agg_contents_list): New type. > (determine_known_aggregate_parts): New function. > (compute_cst_member_ptr_arguments): Removed. > (ipa_compute_jump_functions_for_edge): Compute all kinds of jump > functions (scalar, aggregate and member pointer). > (ipa_get_member_ptr_load_param): Incorporate into > ipa_get_stmt_member_ptr_load_param, also pass back an offset. > (ipa_note_param_call): Clear agg_contents. > (ipa_analyze_indirect_call_uses): Also look for simple pointers loaded > from aggregates. In such cases, store offset of the called field. > (ipa_analyze_node): Initialize new fields of param_analysis_info. > (ipa_agg_types_propagable_p): New function. > (node_parm_type): Likewise. > (inlined_params_propagable_p): Likewise. > (update_jump_functions_after_inlining): Handle aggregate contents. > (ipa_find_agg_cst_for_param): New function. > (try_make_edge_direct_simple_call): Handle called aggregate values. > (update_indirect_edges_after_inlining): Make sure aggregate preserving > jump functions comply with type compatibility requirements. > (ipa_edge_duplication_hook): Copy also aggregate jump functions. > (ipa_write_jump_function): Stream agg_preserved flags and aggregate > jump functions. Do not stream member pointer constant jump functions. > (ipa_read_jump_function): Likewise. > (ipa_write_indirect_edge_info): Stream new cgraph_indirect_call_info > fields. > (ipa_read_indirect_edge_info): Likewise. > > * testsuite/gcc.dg/ipa/iinline-4.c: New test. > * testsuite/gcc.dg/ipa/iinline-5.c: Likewise. > * testsuite/gcc.dg/ipa/iinline-6.c: Likewise. > * testsuite/gcc.dg/ipa/iinline-7.c: Likewise. > > > Index: src/gcc/ipa-prop.h > =================================================================== > --- src.orig/gcc/ipa-prop.h > +++ src/gcc/ipa-prop.h > @@ -44,10 +44,6 @@ along with GCC; see the file COPYING3. > argument. > Unknown - neither of the above. > > - IPA_JF_CONST_MEMBER_PTR stands for C++ member pointers, it is a special > - constant in this regard because it is in fact a structure consisting of two > - values. Other constants are represented with IPA_JF_CONST. > - > IPA_JF_ANCESTOR is a special pass-through jump function, which means that > the result is an address of a part of the object pointed to by the formal > parameter to which the function refers. It is mainly intended to represent > @@ -74,7 +70,6 @@ enum jump_func_type > IPA_JF_UNKNOWN = 0, /* newly allocated and zeroed jump functions default */ > IPA_JF_KNOWN_TYPE, /* represented by field known_type */ > IPA_JF_CONST, /* represented by field costant */ > - IPA_JF_CONST_MEMBER_PTR, /* represented by field member_cst */ > IPA_JF_PASS_THROUGH, /* represented by field pass_through */ > IPA_JF_ANCESTOR /* represented by field ancestor */ > }; > @@ -104,6 +99,13 @@ struct GTY(()) ipa_pass_through_data > arithmetic operation where the caller's parameter is the first operand and > operand field from this structure is the second one. */ > enum tree_code operation; > + /* When the passed value is a pointer or an aggregate passed by value, it is > + set to true only when we are certain that no write to the object it points > + to or the aggregate has occurred since the caller functions started > + execution, except for changes noted in the aggregate part of the jump > + function (see description of ipa_agg_jump_function). The flag is used > + only when this operation is NOP_EXPR. */ > + bool agg_preserved; > }; > > /* Structure holding data required to describe an ancestor pass-through > @@ -117,21 +119,57 @@ struct GTY(()) ipa_ancestor_jf_data > tree type; > /* Number of the caller's formal parameter being passed. */ > int formal_id; > + /* Flag with the same meaning like agg_preserve in ipa_pass_through_data. */ > + bool agg_preserved; > }; > > -/* Structure holding a C++ member pointer constant. Holds a pointer to the > - method and delta offset. */ > -struct GTY(()) ipa_member_ptr_cst > +/* An element in an aggegate part of a jump function describing a known value > + at a given offset. When the this is part of a pass-through jump function > + with agg_preserved set or an ancestor jump function with agg_preserved set, > + all unlisted positions are assumed to be preserved but the value can be a > + type node, which means that the particular piece (starting at offset and > + having the size of the type) is clobbered with an unknown value. When > + agg_preserved is false or the type of the containing jump function is > + different, all unlisted parts are assumed to be unknown and all values must > + fullfill is_gimple_ip_invariant. */ > + > +typedef struct GTY(()) ipa_agg_jf_item > { > - tree pfn; > - tree delta; > + /* The offset at which the known value is located within the aggregate. */ > + HOST_WIDE_INT offset; > + > + /* The known constant or type if this is a clobber. */ > + tree value; > +} ipa_agg_jf_item_t; > + > +DEF_VEC_O (ipa_agg_jf_item_t); > +DEF_VEC_ALLOC_O (ipa_agg_jf_item_t, gc); > + > +/* Aggregate jump function - i.e. description of contents of aggregates passed > + either by reference or value. */ > + > +struct GTY(()) ipa_agg_jump_function > +{ > + /* Description of the individual items. */ > + VEC (ipa_agg_jf_item_t, gc) *items; > + /* Type which was used to store the contents of the aggregate. Not > + meaningful if items is NULL. */ > + tree type; > }; > > +typedef struct ipa_agg_jump_function *ipa_agg_jump_function_p; > +DEF_VEC_P (ipa_agg_jump_function_p); > +DEF_VEC_ALLOC_P (ipa_agg_jump_function_p, heap); > + > /* A jump function for a callsite represents the values passed as actual > arguments of the callsite. See enum jump_func_type for the various > types of jump functions supported. */ > typedef struct GTY (()) ipa_jump_func > { > + /* Aggregate contants description. See struct ipa_agg_jump_function and its > + description. */ > + struct ipa_agg_jump_function agg; > + > enum jump_func_type type; > /* Represents a value of a jump function. pass_through is used only in jump > function context. constant represents the actual constant in constant jump > @@ -140,7 +178,6 @@ typedef struct GTY (()) ipa_jump_func > { > struct ipa_known_type_data GTY ((tag ("IPA_JF_KNOWN_TYPE"))) known_type; > tree GTY ((tag ("IPA_JF_CONST"))) constant; > - struct ipa_member_ptr_cst GTY ((tag ("IPA_JF_CONST_MEMBER_PTR"))) member_cst; > struct ipa_pass_through_data GTY ((tag ("IPA_JF_PASS_THROUGH"))) pass_through; > struct ipa_ancestor_jf_data GTY ((tag ("IPA_JF_ANCESTOR"))) ancestor; > } GTY ((desc ("%1.type"))) value; > @@ -214,6 +251,15 @@ ipa_get_jf_pass_through_operation (struc > return jfunc->value.pass_through.operation; > } > > +/* Return the agg_preserved flag of a pass through jump functin JFUNC. */ > + > +static inline bool > +ipa_get_jf_pass_through_agg_preserved (struct ipa_jump_func *jfunc) > +{ > + gcc_checking_assert (jfunc->type == IPA_JF_PASS_THROUGH); > + return jfunc->value.pass_through.agg_preserved; > +} > + > /* Return the offset of an ancestor jump function JFUNC. */ > > static inline HOST_WIDE_INT > @@ -242,13 +288,13 @@ ipa_get_jf_ancestor_formal_id (struct ip > return jfunc->value.ancestor.formal_id; > } > > -/* Return the pfn part of a member pointer constant jump function JFUNC. */ > +/* Return the agg_preserved flag of an ancestor jump functin JFUNC. */ > > -static inline tree > -ipa_get_jf_member_ptr_pfn (struct ipa_jump_func *jfunc) > +static inline bool > +ipa_get_jf_ancestor_agg_preserved (struct ipa_jump_func *jfunc) > { > - gcc_checking_assert (jfunc->type == IPA_JF_CONST_MEMBER_PTR); > - return jfunc->value.member_cst.pfn; > + gcc_checking_assert (jfunc->type == IPA_JF_ANCESTOR); > + return jfunc->value.ancestor.agg_preserved; > } > > /* Summary describing a single formal parameter. */ > @@ -456,6 +502,14 @@ struct cgraph_edge *ipa_make_edge_direct > /* Functions related to both. */ > void ipa_analyze_node (struct cgraph_node *); > > +/* Aggregate jump function related functions. */ > +tree ipa_find_agg_cst_for_param (struct ipa_agg_jump_function *, HOST_WIDE_INT, > + struct cgraph_node *, int); > +bool ipa_load_from_parm_agg (struct ipa_node_params *, gimple, tree, int *, > + HOST_WIDE_INT *); > +bool ipa_jf_propagates_agg_p (struct cgraph_edge *, struct ipa_jump_func *, > + int); > + > /* Debugging interface. */ > void ipa_print_node_params (FILE *, struct cgraph_node *node); > void ipa_print_all_params (FILE *); > Index: src/gcc/ipa-prop.c > =================================================================== > --- src.orig/gcc/ipa-prop.c > +++ src/gcc/ipa-prop.c > @@ -47,8 +47,9 @@ along with GCC; see the file COPYING3. > > struct param_analysis_info > { > - bool modified; > - bitmap visited_statements; > + bool parm_modified, ref_modified; > + bitmap parm_visited_statements, ref_visited_statements; > + tree deref; > }; > > /* Vector where the parameter infos are actually stored. */ > @@ -181,24 +182,21 @@ ipa_print_node_jump_functions_for_edge ( > } > fprintf (f, "\n"); > } > - else if (type == IPA_JF_CONST_MEMBER_PTR) > - { > - fprintf (f, "CONST MEMBER PTR: "); > - print_generic_expr (f, jump_func->value.member_cst.pfn, 0); > - fprintf (f, ", "); > - print_generic_expr (f, jump_func->value.member_cst.delta, 0); > - fprintf (f, "\n"); > - } > else if (type == IPA_JF_PASS_THROUGH) > { > fprintf (f, "PASS THROUGH: "); > - fprintf (f, "%d, op %s ", > + fprintf (f, "%d, op %s", > jump_func->value.pass_through.formal_id, > tree_code_name[(int) > jump_func->value.pass_through.operation]); > if (jump_func->value.pass_through.operation != NOP_EXPR) > - print_generic_expr (f, > - jump_func->value.pass_through.operand, 0); > + { > + fprintf (f, " "); > + print_generic_expr (f, > + jump_func->value.pass_through.operand, 0); > + } > + if (jump_func->value.pass_through.agg_preserved) > + fprintf (f, ", agg_preserved"); > fprintf (f, "\n"); > } > else if (type == IPA_JF_ANCESTOR) > @@ -208,8 +206,33 @@ ipa_print_node_jump_functions_for_edge ( > jump_func->value.ancestor.formal_id, > jump_func->value.ancestor.offset); > print_generic_expr (f, jump_func->value.ancestor.type, 0); > + if (jump_func->value.ancestor.agg_preserved) > + fprintf (f, ", agg_preserved"); > fprintf (f, "\n"); > } > + > + if (jump_func->agg.items) > + { > + struct ipa_agg_jf_item *item; > + int j; > + > + fprintf (f, " Aggregte contents:\n"); > + FOR_EACH_VEC_ELT (ipa_agg_jf_item_t, jump_func->agg.items, > + j, item) > + { > + fprintf (f, " offset: " HOST_WIDE_INT_PRINT_DEC ", ", > + item->offset); > + if (TYPE_P (item->value)) > + fprintf (f, "clobber of " HOST_WIDE_INT_PRINT_DEC " bits", > + tree_low_cst (TYPE_SIZE (item->value), 1)); > + else > + { > + fprintf (f, "cst: "); > + print_generic_expr (f, item->value, 0); > + } > + fprintf (f, "\n"); > + } > + } > } > } > > @@ -289,12 +312,14 @@ ipa_set_jf_constant (struct ipa_jump_fun > > /* Set JFUNC to be a simple pass-through jump function. */ > static void > -ipa_set_jf_simple_pass_through (struct ipa_jump_func *jfunc, int formal_id) > +ipa_set_jf_simple_pass_through (struct ipa_jump_func *jfunc, int formal_id, > + bool agg_preserved) > { > jfunc->type = IPA_JF_PASS_THROUGH; > jfunc->value.pass_through.operand = NULL_TREE; > jfunc->value.pass_through.formal_id = formal_id; > jfunc->value.pass_through.operation = NOP_EXPR; > + jfunc->value.pass_through.agg_preserved = agg_preserved; > } > > /* Set JFUNC to be an arithmetic pass through jump function. */ > @@ -307,30 +332,20 @@ ipa_set_jf_arith_pass_through (struct ip > jfunc->value.pass_through.operand = operand; > jfunc->value.pass_through.formal_id = formal_id; > jfunc->value.pass_through.operation = operation; > + jfunc->value.pass_through.agg_preserved = false; > } > > /* Set JFUNC to be an ancestor jump function. */ > > static void > ipa_set_ancestor_jf (struct ipa_jump_func *jfunc, HOST_WIDE_INT offset, > - tree type, int formal_id) > + tree type, int formal_id, bool agg_preserved) > { > jfunc->type = IPA_JF_ANCESTOR; > jfunc->value.ancestor.formal_id = formal_id; > jfunc->value.ancestor.offset = offset; > jfunc->value.ancestor.type = type; > -} > - > -/* Simple function filling in a member pointer constant jump function (with PFN > - and DELTA as the constant value) into JFUNC. */ > - > -static void > -ipa_set_jf_member_ptr_cst (struct ipa_jump_func *jfunc, > - tree pfn, tree delta) > -{ > - jfunc->type = IPA_JF_CONST_MEMBER_PTR; > - jfunc->value.member_cst.pfn = pfn; > - jfunc->value.member_cst.delta = delta; > + jfunc->value.ancestor.agg_preserved = agg_preserved; > } > > /* Structure to be passed in between detect_type_change and > @@ -584,30 +599,28 @@ mark_modified (ao_ref *ao ATTRIBUTE_UNUS > return true; > } > > -/* Return true if the formal parameter PARM might have been modified in this > +/* Return true if the formal parameter PARM is known not to be modified in this > function before reaching the statement STMT. PARM_AINFO is a pointer to a > structure containing temporary information about PARM. */ > > static bool > -is_parm_modified_before_stmt (struct param_analysis_info *parm_ainfo, > - gimple stmt, tree parm) > +is_parm_preserved_before_stmt (struct param_analysis_info *parm_ainfo, > + gimple stmt, tree parm) > { > bool modified = false; > ao_ref refd; > > - if (parm_ainfo->modified) > - return true; > + if (parm_ainfo && parm_ainfo->parm_modified) > + return false; > > gcc_checking_assert (gimple_vuse (stmt) != NULL_TREE); > ao_ref_init (&refd, parm); > walk_aliased_vdefs (&refd, gimple_vuse (stmt), mark_modified, > - &modified, &parm_ainfo->visited_statements); > - if (modified) > - { > - parm_ainfo->modified = true; > - return true; > - } > - return false; > + &modified, > + parm_ainfo ? &parm_ainfo->parm_visited_statements : NULL); > + if (parm_ainfo && modified) > + parm_ainfo->parm_modified = true; > + return !modified; > } > > /* If STMT is an assignment that loads a value from an parameter declaration, > @@ -631,12 +644,125 @@ load_from_unmodified_param (struct ipa_n > > index = ipa_get_param_decl_index (info, op1); > if (index < 0 > - || is_parm_modified_before_stmt (&parms_ainfo[index], stmt, op1)) > + || !is_parm_preserved_before_stmt (&parms_ainfo[index], stmt, op1)) > return -1; > > return index; > } > > +/* Return true if the data pointed to by REF is known to be unmodified in > + this function before reaching statement STMT. PARM_AINFO, if non-NULL, is a > + pointer to a structure containing temporary information about PARM. */ > + > +static bool > +is_parm_ref_data_preserved (struct param_analysis_info *parm_ainfo, > + gimple stmt, tree ref) > +{ > + bool modified = false; > + ao_ref refd; > + > + gcc_checking_assert (gimple_vuse (stmt)); > + if (parm_ainfo && parm_ainfo->ref_modified) > + return false; > + > + ao_ref_init (&refd, ref); > + walk_aliased_vdefs (&refd, gimple_vuse (stmt), mark_modified, &modified, > + parm_ainfo ? &parm_ainfo->ref_visited_statements : NULL); > + if (parm_ainfo && modified) > + parm_ainfo->ref_modified = true; > + return !modified; > +} > + > +/* Return true if the data pointed to by PARM is known to be unmodified in this > + function before reaching call statement CALL. PARM_AINFO is a pointer to a > + structure containing temporary information about PARM. We use the > + pointed-to type for TBAA, so make sure any consumer in any indirect calle > + uses that type to read the data (see ipa_agg_types_propagable_p). */ > + > +static bool > +is_parm_ref_data_preserved_call (struct param_analysis_info *parm_ainfo, > + gimple call, tree parm) > +{ > + /* It's unnecessary to calculate anything about memory contnets for a const > + function because it is not goin to use it. But do not cache the result > + either. Also, no such calculations for non-pointers. */ > + if (!gimple_vuse (call) > + || !POINTER_TYPE_P (TREE_TYPE (parm))) > + return false; > + > + if (!parm_ainfo->deref) > + /* Use of this alias type requires that we make sure the data we pass to > + the function is actually of the supposed type. But we do that both when > + gathering the contents of aggregates and when propagating. */ > + parm_ainfo->deref = build2 (MEM_REF, TREE_TYPE (TREE_TYPE (parm)), parm, > + build_int_cst (TREE_TYPE (parm), 0)); > + > + > + return is_parm_ref_data_preserved (parm_ainfo, call, parm_ainfo->deref); > +} > + > +/* Return true if we can prove that OP is a memory reference loading unmodified > + data from an aggregate passed as a parameter and if the aggregate is passed > + by reference, that the alias type of the load corresponds to the type of the > + formal parameter (so that we can rely on this type for TBAA in callers). > + INFO and PARMS_AINFO describe parameters of the current function (but the > + latter can be NULL), STMT is the load statement. If function returns true, > + *INDEX_P and *OFFSET_P is filled with the parameter index and offset within > + the aggregate respectively. */ > + > +static bool > +ipa_load_from_parm_agg_1 (struct ipa_node_params *info, > + struct param_analysis_info *parms_ainfo, gimple stmt, > + tree op, int *index_p, HOST_WIDE_INT *offset_p) > +{ > + HOST_WIDE_INT size, max_size; > + tree base = get_ref_base_and_extent (op, offset_p, &size, &max_size); > + > + if (max_size == -1 || max_size != size || *offset_p < 0) > + return false; > + > + if (DECL_P (base)) > + { > + int index = ipa_get_param_decl_index (info, base); > + if (index >= 0 > + && is_parm_preserved_before_stmt (parms_ainfo ? &parms_ainfo[index] > + : NULL, stmt, base)) > + { > + *index_p = index; > + return true; > + } > + } > + else if (TREE_CODE (base) == MEM_REF > + && TREE_CODE (TREE_OPERAND (base, 0)) == SSA_NAME > + && SSA_NAME_IS_DEFAULT_DEF (TREE_OPERAND (base, 0)) > + && integer_zerop (TREE_OPERAND (base, 1))) > + { > + tree parm = SSA_NAME_VAR (TREE_OPERAND (base, 0)); > + int index = ipa_get_param_decl_index (info, parm); > + if (index >= 0 > + && POINTER_TYPE_P (TREE_TYPE (parm)) > + && (TYPE_MAIN_VARIANT (TREE_TYPE (TREE_TYPE (TREE_OPERAND (base, 1)))) > + == TYPE_MAIN_VARIANT (TREE_TYPE (TREE_TYPE (parm)))) > + && is_parm_ref_data_preserved (parms_ainfo ? &parms_ainfo[index] > + : NULL, stmt, base)) > + { > + *index_p = index; > + return true; > + } > + } > + return false; > +} > + > +/* just like the previous function, just without the param_analysis_info > + pointer, for users outside of this file. */ > + > +bool > +ipa_load_from_parm_agg (struct ipa_node_params *info, gimple stmt, > + tree op, int *index_p, HOST_WIDE_INT *offset_p) > +{ > + return ipa_load_from_parm_agg_1 (info, NULL, stmt, op, index_p, offset_p); > +} > + > /* Given that an actual argument is an SSA_NAME (given in NAME) and is a result > of an assignment statement STMT, try to determine whether we are actually > handling any of the following cases and construct an appropriate jump > @@ -734,7 +860,11 @@ compute_complex_assign_jump_func (struct > } > else if (gimple_assign_single_p (stmt) > && !detect_type_change_ssa (tc_ssa, call, jfunc)) > - ipa_set_jf_simple_pass_through (jfunc, index); > + { > + bool agg_p = is_parm_ref_data_preserved_call (&parms_ainfo[index], > + call, tc_ssa); > + ipa_set_jf_simple_pass_through (jfunc, index, agg_p); > + } > return; > } > > @@ -760,7 +890,9 @@ compute_complex_assign_jump_func (struct > index = ipa_get_param_decl_index (info, SSA_NAME_VAR (ssa)); > if (index >= 0 > && !detect_type_change (op1, base, call, jfunc, offset)) > - ipa_set_ancestor_jf (jfunc, offset, TREE_TYPE (op1), index); > + ipa_set_ancestor_jf (jfunc, offset, TREE_TYPE (op1), index, > + is_parm_ref_data_preserved_call (&parms_ainfo[index], > + call, ssa)); > } > > /* Extract the base, offset and MEM_REF expression from a statement ASSIGN if > @@ -831,6 +963,7 @@ get_ancestor_addr_info (gimple assign, t > > static void > compute_complex_ancestor_jump_func (struct ipa_node_params *info, > + struct param_analysis_info *parms_ainfo, > struct ipa_jump_func *jfunc, > gimple call, gimple phi) > { > @@ -884,7 +1017,9 @@ compute_complex_ancestor_jump_func (stru > } > > if (!detect_type_change (obj, expr, call, jfunc, offset)) > - ipa_set_ancestor_jf (jfunc, offset, TREE_TYPE (obj), index); > + ipa_set_ancestor_jf (jfunc, offset, TREE_TYPE (obj), index, > + is_parm_ref_data_preserved_call (&parms_ainfo[index], > + call, parm)); > } > > /* Given OP which is passed as an actual argument to a called function, > @@ -919,55 +1054,6 @@ compute_known_type_jump_func (tree op, s > ipa_set_jf_known_type (jfunc, offset, TREE_TYPE (base), TREE_TYPE (op)); > } > > - > -/* Determine the jump functions of scalar arguments. Scalar means SSA names > - and constants of a number of selected types. INFO is the ipa_node_params > - structure associated with the caller, PARMS_AINFO describes state of > - analysis with respect to individual formal parameters. ARGS is the > - ipa_edge_args structure describing the callsite CALL which is the call > - statement being examined.*/ > - > -static void > -compute_scalar_jump_functions (struct ipa_node_params *info, > - struct param_analysis_info *parms_ainfo, > - struct ipa_edge_args *args, > - gimple call) > -{ > - tree arg; > - unsigned num = 0; > - > - for (num = 0; num < gimple_call_num_args (call); num++) > - { > - struct ipa_jump_func *jfunc = ipa_get_ith_jump_func (args, num); > - arg = gimple_call_arg (call, num); > - > - if (is_gimple_ip_invariant (arg)) > - ipa_set_jf_constant (jfunc, arg); > - else if (TREE_CODE (arg) == SSA_NAME) > - { > - if (SSA_NAME_IS_DEFAULT_DEF (arg)) > - { > - int index = ipa_get_param_decl_index (info, SSA_NAME_VAR (arg)); > - > - if (index >= 0 > - && !detect_type_change_ssa (arg, call, jfunc)) > - ipa_set_jf_simple_pass_through (jfunc, index); > - } > - else > - { > - gimple stmt = SSA_NAME_DEF_STMT (arg); > - if (is_gimple_assign (stmt)) > - compute_complex_assign_jump_func (info, parms_ainfo, jfunc, > - call, stmt, arg); > - else if (gimple_code (stmt) == GIMPLE_PHI) > - compute_complex_ancestor_jump_func (info, jfunc, call, stmt); > - } > - } > - else > - compute_known_type_jump_func (arg, jfunc, call); > - } > -} > - > /* Inspect the given TYPE and return true iff it has the same structure (the > same number of fields of the same types) as a C++ member pointer. If > METHOD_PTR and DELTA are non-NULL, store the trees representing the > @@ -983,14 +1069,16 @@ type_like_member_ptr_p (tree type, tree > > fld = TYPE_FIELDS (type); > if (!fld || !POINTER_TYPE_P (TREE_TYPE (fld)) > - || TREE_CODE (TREE_TYPE (TREE_TYPE (fld))) != METHOD_TYPE) > + || TREE_CODE (TREE_TYPE (TREE_TYPE (fld))) != METHOD_TYPE > + || !host_integerp (DECL_FIELD_OFFSET (fld), 1)) > return false; > > if (method_ptr) > *method_ptr = fld; > > fld = DECL_CHAIN (fld); > - if (!fld || INTEGRAL_TYPE_P (fld)) > + if (!fld || INTEGRAL_TYPE_P (fld) > + || !host_integerp (DECL_FIELD_OFFSET (fld), 1)) > return false; > if (delta) > *delta = fld; > @@ -1001,54 +1089,9 @@ type_like_member_ptr_p (tree type, tree > return true; > } > > -/* Go through arguments of the CALL and for every one that looks like a member > - pointer, check whether it can be safely declared pass-through and if so, > - mark that to the corresponding item of jump FUNCTIONS. Return true iff > - there are non-pass-through member pointers within the arguments. INFO > - describes formal parameters of the caller. PARMS_INFO is a pointer to a > - vector containing intermediate information about each formal parameter. */ > - > -static bool > -compute_pass_through_member_ptrs (struct ipa_node_params *info, > - struct param_analysis_info *parms_ainfo, > - struct ipa_edge_args *args, > - gimple call) > -{ > - bool undecided_members = false; > - unsigned num; > - tree arg; > - > - for (num = 0; num < gimple_call_num_args (call); num++) > - { > - arg = gimple_call_arg (call, num); > - > - if (type_like_member_ptr_p (TREE_TYPE (arg), NULL, NULL)) > - { > - if (TREE_CODE (arg) == PARM_DECL) > - { > - int index = ipa_get_param_decl_index (info, arg); > - > - gcc_assert (index >=0); > - if (!is_parm_modified_before_stmt (&parms_ainfo[index], call, > - arg)) > - { > - struct ipa_jump_func *jfunc = ipa_get_ith_jump_func (args, > - num); > - ipa_set_jf_simple_pass_through (jfunc, index); > - } > - else > - undecided_members = true; > - } > - else > - undecided_members = true; > - } > - } > - > - return undecided_members; > -} > - > /* If RHS is an SSA_NAME and it is defined by a simple copy assign statement, > - return the rhs of its defining statement. */ > + return the rhs of its defining statement. Otherwise return RHS as it > + is. */ > > static inline tree > get_ssa_def_if_simple_copy (tree rhs) > @@ -1065,104 +1108,233 @@ get_ssa_def_if_simple_copy (tree rhs) > return rhs; > } > > -/* Traverse statements from CALL backwards, scanning whether the argument ARG > - which is a member pointer is filled in with constant values. If it is, fill > - the jump function JFUNC in appropriately. METHOD_FIELD and DELTA_FIELD are > - fields of the record type of the member pointer. To give an example, we > - look for a pattern looking like the following: > - > - D.2515.__pfn ={v} printStuff; > - D.2515.__delta ={v} 0; > - i_1 = doprinting (D.2515); */ > +/* Helper for determine_known_aggregate_parts, initializes *R for an aggregate > + passed by reference based on BASE and with the given TYPE. */ > > static void > -determine_cst_member_ptr (gimple call, tree arg, tree method_field, > - tree delta_field, struct ipa_jump_func *jfunc) > +init_ao_ref_for_byref_agg_jf (ao_ref *r, tree base, tree type) > { > + static tree alias_all_type = NULL; > + > + if (!alias_all_type) > + alias_all_type = build_pointer_type_for_mode (char_type_node, > + ptr_mode, true); > + ao_ref_init (r, build2 (MEM_REF, type, base, > + build_int_cst (alias_all_type, 0))); > +} > + > +/* TODO: Turn this into a PARAM. */ > +#define IPA_MAX_AFF_JF_ITEMS 16 > + > +/* Simple linked list, describing known contents of an aggregate beforere > + call. */ > + > +struct ipa_known_agg_contents_list > +{ > + /* Offset and size of the described part of the aggregate. */ > + HOST_WIDE_INT offset, size; > + /* Known constant value or NULL if the contents is known to be unknown. */ > + tree constant; > + /* Pointer to the next structure in the list. */ > + struct ipa_known_agg_contents_list *next; > +}; > + > +/* Traverse statements from CALL backwards, scanning whether an aggregate given > + in ARG is filled in with constant values. ARG can either be an aggregate > + expression or a pointer to an aggregate. JFUNC is the jump function into > + which the constants are subsequently stored. */ > + > +static void > +determine_known_aggregate_parts (gimple call, tree arg, > + struct ipa_jump_func *jfunc) > +{ > + struct ipa_known_agg_contents_list *list = NULL; > + int item_count = 0, const_count = 0; > + HOST_WIDE_INT arg_offset, arg_size; > gimple_stmt_iterator gsi; > - tree method = NULL_TREE; > - tree delta = NULL_TREE; > + tree arg_base; > + bool by_ref; > + ao_ref r; > > - gsi = gsi_for_stmt (call); > + /* The function operates in three stages. First, we prepare r, arg_base and > + arg_offset based on what is actually passed as an actual argument. */ > + > + if (POINTER_TYPE_P (TREE_TYPE (arg))) > + { > + gcc_checking_assert (AGGREGATE_TYPE_P (TREE_TYPE (TREE_TYPE (arg)))); > + if (TREE_CODE (arg) == SSA_NAME) > + { > + if (!host_integerp (TYPE_SIZE (TREE_TYPE (TREE_TYPE (arg))), 1)) > + return; > + > + by_ref = true; > + arg_base = arg; > + arg_offset = 0; > + arg_size = tree_low_cst (TYPE_SIZE (TREE_TYPE (TREE_TYPE (arg))), 1); > + init_ao_ref_for_byref_agg_jf (&r, arg_base, > + TREE_TYPE (TREE_TYPE (arg))); > + } > + else if (TREE_CODE (arg) == ADDR_EXPR) > + { > + HOST_WIDE_INT arg_max_size; > + > + arg = TREE_OPERAND (arg, 0); > + arg_base = get_ref_base_and_extent (arg, &arg_offset, &arg_size, > + &arg_max_size); > + if (arg_max_size == -1 > + || arg_max_size != arg_size > + || arg_offset < 0) > + return; > + if (DECL_P (arg_base)) > + { > + by_ref = false; > + ao_ref_init (&r, arg); > + } > + else > + return; > + } > + else > + return; > + } > + else > + { > + HOST_WIDE_INT arg_max_size; > + > + gcc_checking_assert (AGGREGATE_TYPE_P (TREE_TYPE (arg))); > + > + by_ref = false; > + arg_base = get_ref_base_and_extent (arg, &arg_offset, &arg_size, > + &arg_max_size); > + if (arg_max_size == -1 > + || arg_max_size != arg_size > + || arg_offset < 0) > + return; > > + ao_ref_init (&r, arg); > + } > + > + /* Second stage walks back the BB, looks at individual statements and as long > + as it is confident of how the statements affect contents of the > + aggregates, it builds a sorted linked list of ipa_agg_jf_list structures > + describing it. */ > + gsi = gsi_for_stmt (call); > gsi_prev (&gsi); > for (; !gsi_end_p (gsi); gsi_prev (&gsi)) > { > + struct ipa_known_agg_contents_list *n, **p; > gimple stmt = gsi_stmt (gsi); > - tree lhs, rhs, fld; > + HOST_WIDE_INT lhs_offset, lhs_size, lhs_max_size; > + tree lhs, rhs, lhs_base; > + bool partial_overlap; > > - if (!stmt_may_clobber_ref_p (stmt, arg)) > + if (!stmt_may_clobber_ref_p_1 (stmt, &r)) > continue; > if (!gimple_assign_single_p (stmt)) > - return; > + break; > > lhs = gimple_assign_lhs (stmt); > rhs = gimple_assign_rhs1 (stmt); > + if (!is_gimple_reg_type (rhs)) > + break; > > - if (TREE_CODE (lhs) != COMPONENT_REF > - || TREE_OPERAND (lhs, 0) != arg) > - return; > + lhs_base = get_ref_base_and_extent (lhs, &lhs_offset, &lhs_size, > + &lhs_max_size); > + if (lhs_max_size == -1 > + || lhs_max_size != lhs_size > + || (lhs_offset < arg_offset > + && lhs_offset + lhs_size > arg_offset) > + || (lhs_offset < arg_offset + arg_size > + && lhs_offset + lhs_size > arg_offset + arg_size)) > + break; > > - fld = TREE_OPERAND (lhs, 1); > - if (!method && fld == method_field) > + if (by_ref) > { > - rhs = get_ssa_def_if_simple_copy (rhs); > - if (TREE_CODE (rhs) == ADDR_EXPR > - && TREE_CODE (TREE_OPERAND (rhs, 0)) == FUNCTION_DECL > - && TREE_CODE (TREE_TYPE (TREE_OPERAND (rhs, 0))) == METHOD_TYPE) > - { > - method = TREE_OPERAND (rhs, 0); > - if (delta) > - { > - ipa_set_jf_member_ptr_cst (jfunc, rhs, delta); > - return; > - } > - } > - else > - return; > + if (TREE_CODE (lhs_base) != MEM_REF > + || TREE_OPERAND (lhs_base, 0) != arg_base > + || !integer_zerop (TREE_OPERAND (lhs_base, 1)) > + /* We record the access type here and use it for checks later so > + make sure there is only one. */ > + || (TYPE_MAIN_VARIANT (TREE_TYPE (TREE_TYPE ( > + TREE_OPERAND (lhs_base, 1)))) > + != TYPE_MAIN_VARIANT (TREE_TYPE (TREE_TYPE (arg_base))))) > + break; > } > + else if (lhs_base != arg_base) > + break; > + > + if (lhs_offset + lhs_size < arg_offset > + || lhs_offset >= (arg_offset + arg_size)) > + continue; > > - if (!delta && fld == delta_field) > + partial_overlap = false; > + p = &list; > + while (*p && (*p)->offset < lhs_offset) > { > - rhs = get_ssa_def_if_simple_copy (rhs); > - if (TREE_CODE (rhs) == INTEGER_CST) > + if ((*p)->offset + (*p)->size > lhs_offset) > { > - delta = rhs; > - if (method) > - { > - ipa_set_jf_member_ptr_cst (jfunc, rhs, delta); > - return; > - } > + partial_overlap = true; > + break; > } > + p = &(*p)->next; > + } > + if (partial_overlap) > + break; > + if (*p && (*p)->offset < lhs_offset + lhs_size) > + { > + if ((*p)->offset == lhs_offset && (*p)->size == lhs_size) > + /* We already know this value is subsequently overwritten with > + something else. */ > + continue; > else > - return; > + /* Otherwise this is a partial overlap which we cannot > + represent. */ > + break; > } > - } > - > - return; > -} > > -/* Go through the arguments of the CALL and for every member pointer within > - tries determine whether it is a constant. If it is, create a corresponding > - constant jump function in FUNCTIONS which is an array of jump functions > - associated with the call. */ > - > -static void > -compute_cst_member_ptr_arguments (struct ipa_edge_args *args, > - gimple call) > -{ > - unsigned num; > - tree arg, method_field, delta_field; > - > - for (num = 0; num < gimple_call_num_args (call); num++) > - { > - struct ipa_jump_func *jfunc = ipa_get_ith_jump_func (args, num); > - arg = gimple_call_arg (call, num); > + rhs = get_ssa_def_if_simple_copy (rhs); > + n = XALLOCA (struct ipa_known_agg_contents_list); > + n->size = lhs_size; > + n->offset = lhs_offset; > + if (is_gimple_ip_invariant (rhs)) > + { > + n->constant = rhs; > + const_count++; > + } > + else > + n->constant = NULL_TREE; > + n->next = *p; > + *p = n; > + > + item_count++; > + if (const_count == IPA_MAX_AFF_JF_ITEMS > + || item_count == 2 * IPA_MAX_AFF_JF_ITEMS) > + break; > + } > > - if (jfunc->type == IPA_JF_UNKNOWN > - && type_like_member_ptr_p (TREE_TYPE (arg), &method_field, > - &delta_field)) > - determine_cst_member_ptr (call, arg, method_field, delta_field, jfunc); > + /* Third stage just goes over the list and creates an appropriate vector of > + ipa_agg_jf_item structures out of it, of sourse only if there are > + any known constants to begin with. */ > + > + if (const_count) > + { > + jfunc->agg.items = VEC_alloc (ipa_agg_jf_item_t, gc, const_count); > + if (POINTER_TYPE_P (TREE_TYPE (arg_base))) > + jfunc->agg.type = TREE_TYPE (TREE_TYPE (arg_base)); > + else > + jfunc->agg.type = TREE_TYPE (arg_base); > + gcc_checking_assert (AGGREGATE_TYPE_P (jfunc->agg.type)); > + while (list) > + { > + if (list->constant) > + { > + struct ipa_agg_jf_item *item; > + item = VEC_quick_push (ipa_agg_jf_item_t, > + jfunc->agg.items, NULL); > + item->offset = list->offset - arg_offset; > + item->value = list->constant; > + } > + list = list->next; > + } > } > } > > @@ -1177,23 +1349,71 @@ ipa_compute_jump_functions_for_edge (str > struct ipa_node_params *info = IPA_NODE_REF (cs->caller); > struct ipa_edge_args *args = IPA_EDGE_REF (cs); > gimple call = cs->call_stmt; > - int arg_num = gimple_call_num_args (call); > + int n, arg_num = gimple_call_num_args (call); > > if (arg_num == 0 || args->jump_functions) > return; > VEC_safe_grow_cleared (ipa_jump_func_t, gc, args->jump_functions, arg_num); > > - /* We will deal with constants and SSA scalars first: */ > - compute_scalar_jump_functions (info, parms_ainfo, args, call); > + for (n = 0; n < arg_num; n++) > + { > + struct ipa_jump_func *jfunc = ipa_get_ith_jump_func (args, n); > + tree arg = gimple_call_arg (call, n); > > - /* Let's check whether there are any potential member pointers and if so, > - whether we can determine their functions as pass_through. */ > - if (!compute_pass_through_member_ptrs (info, parms_ainfo, args, call)) > - return; > + if (is_gimple_ip_invariant (arg)) > + ipa_set_jf_constant (jfunc, arg); > + else if (!is_gimple_reg_type (TREE_TYPE (arg)) > + && TREE_CODE (arg) == PARM_DECL) > + { > + int index = ipa_get_param_decl_index (info, arg); > > - /* Finally, let's check whether we actually pass a new constant member > - pointer here... */ > - compute_cst_member_ptr_arguments (args, call); > + gcc_assert (index >=0); > + /* Aggregate passed by value, check for pass-through, otherwise we > + will attempt to fill in aggregate contents later in this > + function. */ > + if (is_parm_preserved_before_stmt (&parms_ainfo[index], call, arg)) > + { > + ipa_set_jf_simple_pass_through (jfunc, index, true); > + continue; > + } > + } > + else if (TREE_CODE (arg) == SSA_NAME) > + { > + if (SSA_NAME_IS_DEFAULT_DEF (arg)) > + { > + int index = ipa_get_param_decl_index (info, SSA_NAME_VAR (arg)); > + if (index >= 0 > + && !detect_type_change_ssa (arg, call, jfunc)) > + { > + bool agg_p; > + agg_p = is_parm_ref_data_preserved_call (&parms_ainfo[index], > + call, arg); > + ipa_set_jf_simple_pass_through (jfunc, index, agg_p); > + } > + } > + else > + { > + gimple stmt = SSA_NAME_DEF_STMT (arg); > + if (is_gimple_assign (stmt)) > + compute_complex_assign_jump_func (info, parms_ainfo, jfunc, > + call, stmt, arg); > + else if (gimple_code (stmt) == GIMPLE_PHI) > + compute_complex_ancestor_jump_func (info, parms_ainfo, jfunc, > + call, stmt); > + } > + } > + else > + compute_known_type_jump_func (arg, jfunc, call); > + > + if ((jfunc->type != IPA_JF_PASS_THROUGH > + || !ipa_get_jf_pass_through_agg_preserved (jfunc)) > + && (jfunc->type != IPA_JF_ANCESTOR > + || !ipa_get_jf_ancestor_agg_preserved (jfunc)) > + && (AGGREGATE_TYPE_P (TREE_TYPE (arg)) > + || (POINTER_TYPE_P (TREE_TYPE (arg)) > + && AGGREGATE_TYPE_P (TREE_TYPE (TREE_TYPE (arg)))))) > + determine_known_aggregate_parts (call, arg, jfunc); > + } > } > > /* Compute jump functions for all edges - both direct and indirect - outgoing > @@ -1220,16 +1440,22 @@ ipa_compute_jump_functions (struct cgrap > ipa_compute_jump_functions_for_edge (parms_ainfo, cs); > } > > -/* If RHS looks like a rhs of a statement loading pfn from a member > - pointer formal parameter, return the parameter, otherwise return > - NULL. If USE_DELTA, then we look for a use of the delta field > - rather than the pfn. */ > +/* If STMT looks like a statement loading a value from a member pointer formal > + parameter, return that parameter and store the offset of the field to > + *OFFSET_P, if it is non-NULL. Otherwise return NULL (but *OFFSET_P still > + might be clobbered). If USE_DELTA, then we look for a use of the delta > + field rather than the pfn. */ > > static tree > -ipa_get_member_ptr_load_param (tree rhs, bool use_delta) > +ipa_get_stmt_member_ptr_load_param (gimple stmt, bool use_delta, > + HOST_WIDE_INT *offset_p) > { > - tree rec, ref_field, ref_offset, fld, fld_offset, ptr_field, delta_field; > + tree rhs, rec, ref_field, ref_offset, fld, ptr_field, delta_field; > + > + if (!gimple_assign_single_p (stmt)) > + return NULL_TREE; > > + rhs = gimple_assign_rhs1 (stmt); > if (TREE_CODE (rhs) == COMPONENT_REF) > { > ref_field = TREE_OPERAND (rhs, 1); > @@ -1246,43 +1472,24 @@ ipa_get_member_ptr_load_param (tree rhs, > if (TREE_CODE (rec) != PARM_DECL > || !type_like_member_ptr_p (TREE_TYPE (rec), &ptr_field, &delta_field)) > return NULL_TREE; > - > ref_offset = TREE_OPERAND (rhs, 1); > > + if (use_delta) > + fld = delta_field; > + else > + fld = ptr_field; > + if (offset_p) > + *offset_p = int_bit_position (fld); > + > if (ref_field) > { > if (integer_nonzerop (ref_offset)) > return NULL_TREE; > - > - if (use_delta) > - fld = delta_field; > - else > - fld = ptr_field; > - > return ref_field == fld ? rec : NULL_TREE; > } > - > - if (use_delta) > - fld_offset = byte_position (delta_field); > else > - fld_offset = byte_position (ptr_field); > - > - return tree_int_cst_equal (ref_offset, fld_offset) ? rec : NULL_TREE; > -} > - > -/* If STMT looks like a statement loading a value from a member pointer formal > - parameter, this function returns that parameter. */ > - > -static tree > -ipa_get_stmt_member_ptr_load_param (gimple stmt, bool use_delta) > -{ > - tree rhs; > - > - if (!gimple_assign_single_p (stmt)) > - return NULL_TREE; > - > - rhs = gimple_assign_rhs1 (stmt); > - return ipa_get_member_ptr_load_param (rhs, use_delta); > + return tree_int_cst_equal (byte_position (fld), ref_offset) ? rec > + : NULL_TREE; > } > > /* Returns true iff T is an SSA_NAME defined by a statement. */ > @@ -1308,8 +1515,9 @@ ipa_note_param_call (struct cgraph_node > > cs = cgraph_edge (node, stmt); > cs->indirect_info->param_index = param_index; > - cs->indirect_info->anc_offset = 0; > + cs->indirect_info->offset = 0; > cs->indirect_info->polymorphic = 0; > + cs->indirect_info->agg_contents = 0; > return cs; > } > > @@ -1368,7 +1576,9 @@ ipa_note_param_call (struct cgraph_node > > return (S.*f)(4); > } > -*/ > + > + Moreover, the function also looks for called pointers loaded from aggregates > + passed by value or reference. */ > > static void > ipa_analyze_indirect_call_uses (struct cgraph_node *node, > @@ -1383,6 +1593,7 @@ ipa_analyze_indirect_call_uses (struct c > gimple branch; > int index; > basic_block bb, virt_bb, join; > + HOST_WIDE_INT offset; > > if (SSA_NAME_IS_DEFAULT_DEF (target)) > { > @@ -1393,20 +1604,25 @@ ipa_analyze_indirect_call_uses (struct c > return; > } > > + def = SSA_NAME_DEF_STMT (target); > + if (gimple_assign_single_p (def) > + && ipa_load_from_parm_agg_1 (info, parms_ainfo, def, > + gimple_assign_rhs1 (def), &index, &offset)) > + { > + struct cgraph_edge *cs = ipa_note_param_call (node, index, call); > + cs->indirect_info->offset = offset; > + cs->indirect_info->agg_contents = 1; > + return; > + } > + > /* Now we need to try to match the complex pattern of calling a member > pointer. */ > - > - if (!POINTER_TYPE_P (TREE_TYPE (target)) > + if (gimple_code (def) != GIMPLE_PHI > + || gimple_phi_num_args (def) != 2 > + || !POINTER_TYPE_P (TREE_TYPE (target)) > || TREE_CODE (TREE_TYPE (TREE_TYPE (target))) != METHOD_TYPE) > return; > > - def = SSA_NAME_DEF_STMT (target); > - if (gimple_code (def) != GIMPLE_PHI) > - return; > - > - if (gimple_phi_num_args (def) != 2) > - return; > - > /* First, we need to check whether one of these is a load from a member > pointer that is a parameter to this function. */ > n1 = PHI_ARG_DEF (def, 0); > @@ -1417,15 +1633,15 @@ ipa_analyze_indirect_call_uses (struct c > d2 = SSA_NAME_DEF_STMT (n2); > > join = gimple_bb (def); > - if ((rec = ipa_get_stmt_member_ptr_load_param (d1, false))) > + if ((rec = ipa_get_stmt_member_ptr_load_param (d1, false, &offset))) > { > - if (ipa_get_stmt_member_ptr_load_param (d2, false)) > + if (ipa_get_stmt_member_ptr_load_param (d2, false, NULL)) > return; > > bb = EDGE_PRED (join, 0)->src; > virt_bb = gimple_bb (d2); > } > - else if ((rec = ipa_get_stmt_member_ptr_load_param (d2, false))) > + else if ((rec = ipa_get_stmt_member_ptr_load_param (d2, false, &offset))) > { > bb = EDGE_PRED (join, 1)->src; > virt_bb = gimple_bb (d1); > @@ -1480,15 +1696,19 @@ ipa_analyze_indirect_call_uses (struct c > > rec2 = ipa_get_stmt_member_ptr_load_param (def, > (TARGET_PTRMEMFUNC_VBIT_LOCATION > - == ptrmemfunc_vbit_in_delta)); > - > + == ptrmemfunc_vbit_in_delta), > + NULL); > if (rec != rec2) > return; > > index = ipa_get_param_decl_index (info, rec); > - if (index >= 0 && !is_parm_modified_before_stmt (&parms_ainfo[index], > + if (index >= 0 && is_parm_preserved_before_stmt (&parms_ainfo[index], > call, rec)) > - ipa_note_param_call (node, index, call); > + { > + struct cgraph_edge *cs = ipa_note_param_call (node, index, call); > + cs->indirect_info->offset = offset; > + cs->indirect_info->agg_contents = 1; > + } > > return; > } > @@ -1543,7 +1763,7 @@ ipa_analyze_virtual_call_uses (struct cg > > cs = ipa_note_param_call (node, index, call); > ii = cs->indirect_info; > - ii->anc_offset = anc_offset; > + ii->offset = anc_offset; > ii->otr_token = tree_low_cst (OBJ_TYPE_REF_TOKEN (target), 1); > ii->otr_type = TREE_TYPE (TREE_TYPE (OBJ_TYPE_REF_OBJECT (target))); > ii->polymorphic = 1; > @@ -1685,8 +1905,12 @@ ipa_analyze_node (struct cgraph_node *no > ipa_compute_jump_functions (node, parms_ainfo); > > for (i = 0; i < param_count; i++) > - if (parms_ainfo[i].visited_statements) > - BITMAP_FREE (parms_ainfo[i].visited_statements); > + { > + if (parms_ainfo[i].parm_visited_statements) > + BITMAP_FREE (parms_ainfo[i].parm_visited_statements); > + if (parms_ainfo[i].ref_visited_statements) > + BITMAP_FREE (parms_ainfo[i].ref_visited_statements); > + } > > current_function_decl = NULL; > pop_cfun (); > @@ -1713,6 +1937,112 @@ combine_known_type_and_ancestor_jfs (str > combined_type); > } > > +/* Return true if we can still rely on our non-modification analysis > + (is_parm_preserved_before_stmt and especially is_parm_ref_data_preserved) if > + information about aggregate contents of type AGG can be propagated to or > + through a parameter of type TO. If OFFSET is zero, we look at the types as > + they are first. Then, if FROM is a record or a pointer to a record, we look > + for a TO-typed field at offset. */ > + > +static bool > +ipa_agg_types_propagable_p (tree from, tree to, HOST_WIDE_INT offset) > +{ > + gcc_checking_assert (TYPE_P (from) && TYPE_P (to)); > + if (POINTER_TYPE_P (from)) > + from = TREE_TYPE (from); > + if (POINTER_TYPE_P (to)) > + to = TREE_TYPE (to); > + > + if (offset == 0 && useless_type_conversion_p (to, from)) > + return true; > + > + /* If the above failed, we resort to finding a field within a structure, > + provided we are looking at one. */ > + if (TREE_CODE (from) != RECORD_TYPE) > + return false; > + > + while (true) > + { > + tree fld; > + > + for (fld = TYPE_FIELDS (from); fld; fld = DECL_CHAIN (fld)) > + { > + HOST_WIDE_INT pos, size; > + tree tr_size; > + > + if (TREE_CODE (fld) != FIELD_DECL) > + continue; > + > + pos = int_bit_position (fld); > + tr_size = DECL_SIZE (fld); > + if (!tr_size || !host_integerp (tr_size, 1)) > + continue; > + size = tree_low_cst (tr_size, 1); > + if (pos > offset > + || (pos + size) <= offset) > + continue; > + > + from = TREE_TYPE (fld); > + if (offset == pos > + && TYPE_MAIN_VARIANT (from) == TYPE_MAIN_VARIANT (to)) > + return true; > + > + if (TREE_CODE (from) == RECORD_TYPE) > + break; > + else > + return false; > + } > + if (!fld) > + return false; > + } > +} > + > +/* Return type of Ith parameter of NODE. */ > + > +static tree > +node_parm_type (struct cgraph_node *node, int i) > +{ > + return TREE_TYPE (ipa_get_param (IPA_NODE_REF (node), i)); > +} > + > +/* CS is an edge that is being currently inlined. Return false if parameter of > + the callee with index DST_IDX, which recieves its value via JF, fails to > + comply with aggregate type propagability criteria > + (ipa_agg_types_propagable_p). */ > + > +bool > +ipa_jf_propagates_agg_p (struct cgraph_edge *cs, > + struct ipa_jump_func *jf, int dst_idx) > +{ > + struct cgraph_node *new_root; > + > + new_root = cs->caller->global.inlined_to ? cs->caller->global.inlined_to > + : cs->caller; > + > + if (jf->type == IPA_JF_PASS_THROUGH) > + { > + tree t1, t2; > + if (!ipa_get_jf_pass_through_agg_preserved (jf)) > + return false; > + t1 = node_parm_type (new_root, ipa_get_jf_pass_through_formal_id (jf)); > + t2 = node_parm_type (cs->callee, dst_idx); > + return ipa_agg_types_propagable_p (t1, t2, 0); > + } > + else if (jf->type == IPA_JF_ANCESTOR) > + { > + tree t1, t2; > + if (!ipa_get_jf_ancestor_agg_preserved (jf)) > + return false; > + t1 = node_parm_type (new_root, ipa_get_jf_ancestor_formal_id (jf)); > + t2 = node_parm_type (cs->callee, dst_idx); > + return ipa_agg_types_propagable_p (t1, t2, > + ipa_get_jf_ancestor_offset (jf)); > + } > + > + return true; > +} > + > + > /* Update the jump functions associated with call graph edge E when the call > graph edge CS is being inlined, assuming that E->caller is already (possibly > indirectly) inlined into CS->callee and that E has not been inlined. */ > @@ -1733,26 +2063,55 @@ update_jump_functions_after_inlining (st > if (dst->type == IPA_JF_ANCESTOR) > { > struct ipa_jump_func *src; > + int dst_fid = dst->value.ancestor.formal_id; > > /* Variable number of arguments can cause havoc if we try to access > one that does not exist in the inlined edge. So make sure we > don't. */ > - if (dst->value.ancestor.formal_id >= ipa_get_cs_argument_count (top)) > + if (dst_fid >= ipa_get_cs_argument_count (top)) > { > dst->type = IPA_JF_UNKNOWN; > continue; > } > > - src = ipa_get_ith_jump_func (top, dst->value.ancestor.formal_id); > + src = ipa_get_ith_jump_func (top, dst_fid); > + > + if (dst->value.ancestor.agg_preserved && src->agg.items) > + { > + /* Currently we do not produce clobber aggregate jump functions, > + replace with merging when we do. */ > + gcc_assert (!dst->agg.items); > + > + if (ipa_agg_types_propagable_p (src->agg.type, > + dst->value.ancestor.type, > + dst->value.ancestor.offset)) > + { > + struct ipa_agg_jf_item *item; > + int j; > + > + dst->agg.items = VEC_copy (ipa_agg_jf_item_t, gc, > + src->agg.items); > + FOR_EACH_VEC_ELT (ipa_agg_jf_item_t, dst->agg.items, j, item) > + item->offset -= dst->value.ancestor.offset; > + dst->agg.type = dst->value.ancestor.type; > + } > + } > + > if (src->type == IPA_JF_KNOWN_TYPE) > combine_known_type_and_ancestor_jfs (src, dst); > else if (src->type == IPA_JF_PASS_THROUGH > && src->value.pass_through.operation == NOP_EXPR) > - dst->value.ancestor.formal_id = src->value.pass_through.formal_id; > + { > + dst->value.ancestor.formal_id = src->value.pass_through.formal_id; > + dst->value.ancestor.agg_preserved &= > + ipa_jf_propagates_agg_p (cs, src, dst_fid); > + } > else if (src->type == IPA_JF_ANCESTOR) > { > dst->value.ancestor.formal_id = src->value.ancestor.formal_id; > dst->value.ancestor.offset += src->value.ancestor.offset; > + dst->value.ancestor.agg_preserved &= > + ipa_jf_propagates_agg_p (cs, src, dst_fid); > } > else > dst->type = IPA_JF_UNKNOWN; > @@ -1766,9 +2125,34 @@ update_jump_functions_after_inlining (st > && (dst->value.pass_through.formal_id > < ipa_get_cs_argument_count (top))) > { > - src = ipa_get_ith_jump_func (top, > - dst->value.pass_through.formal_id); > - *dst = *src; > + bool agg_p; > + int dst_fid = dst->value.pass_through.formal_id; > + src = ipa_get_ith_jump_func (top, dst_fid); > + agg_p = dst->value.pass_through.agg_preserved; > + /* Currently we do not produce clobber aggregate jump functions, > + replace with merging when we do. */ > + gcc_assert (!agg_p || !dst->agg.items); > + > + dst->type = src->type; > + dst->value = src->value; > + > + if (agg_p && src->agg.items) > + { > + tree t = node_parm_type (cs->callee, dst_fid); > + > + if (ipa_agg_types_propagable_p (src->agg.type, t, 0)) > + { > + dst->agg.items = VEC_copy (ipa_agg_jf_item_t, gc, > + src->agg.items); > + dst->agg.type = src->agg.type; > + } > + } > + if (src->type == IPA_JF_PASS_THROUGH) > + dst->value.pass_through.agg_preserved = agg_p > + && ipa_jf_propagates_agg_p (cs, src, dst_fid); > + else if (src->type == IPA_JF_ANCESTOR) > + dst->value.ancestor.agg_preserved = agg_p > + && ipa_jf_propagates_agg_p (cs, src, dst_fid); > } > else > dst->type = IPA_JF_UNKNOWN; > @@ -1815,24 +2199,71 @@ ipa_make_edge_direct_to_target (struct c > return ie; > } > > +/* Provided that values from AGG can be propagated to parameter of NODE with > + index PARAM_INDEX so that is_parm_ref_data_preserved results are reliable > + and that there is a known constant value in AGG the given OFFSET, return > + the known constant value. */ > + > +tree > +ipa_find_agg_cst_for_param (struct ipa_agg_jump_function *agg, > + HOST_WIDE_INT offset, > + struct cgraph_node *node, > + int param_index) > +{ > + struct ipa_node_params *info = IPA_NODE_REF (node); > + struct ipa_agg_jf_item *item; > + tree t = TREE_TYPE (ipa_get_param (info, param_index)); > + int i; > + > + if (!agg->items > + || !ipa_agg_types_propagable_p (agg->type, t, 0)) > + return NULL; > + > + FOR_EACH_VEC_ELT (ipa_agg_jf_item_t, agg->items, i, item) > + { > + if (item->offset == offset) > + { > + /* Currently we do not have clobber values, return NULL for them once > + we do. */ > + gcc_checking_assert (is_gimple_ip_invariant (item->value)); > + return item->value; > + } > + else if (item->offset > offset) > + return NULL; > + } > + return NULL; > +} > + > /* Try to find a destination for indirect edge IE that corresponds to a simple > call or a call of a member function pointer and where the destination is a > - pointer formal parameter described by jump function JFUNC. If it can be > - determined, return the newly direct edge, otherwise return NULL. */ > + pointer formal parameter described by jump function JFUNC. CS is the edge > + that is currently being inlined. If it can be determined, return the newly > + direct edge, otherwise return NULL. */ > > static struct cgraph_edge * > -try_make_edge_direct_simple_call (struct cgraph_edge *ie, > - struct ipa_jump_func *jfunc) > +try_make_edge_direct_simple_call (struct cgraph_edge *cs, > + struct cgraph_edge *ie, > + struct ipa_jump_func *jfunc, int param_index) > { > tree target; > > - if (jfunc->type == IPA_JF_CONST) > - target = ipa_get_jf_constant (jfunc); > - else if (jfunc->type == IPA_JF_CONST_MEMBER_PTR) > - target = ipa_get_jf_member_ptr_pfn (jfunc); > - else > - return NULL; > + if (ie->indirect_info->agg_contents) > + { > + if (!ipa_jf_propagates_agg_p (cs, jfunc, param_index)) > + return NULL; > > + target = ipa_find_agg_cst_for_param (&jfunc->agg, > + ie->indirect_info->offset, > + cs->callee, param_index); > + if (!target) > + return NULL; > + } > + else > + { > + if (jfunc->type != IPA_JF_CONST) > + return NULL; > + target = ipa_get_jf_constant (jfunc); > + } > return ipa_make_edge_direct_to_target (ie, target); > } > > @@ -1853,7 +2284,7 @@ try_make_edge_direct_virtual_call (struc > binfo = TYPE_BINFO (ipa_get_jf_known_type_base_type (jfunc)); > gcc_checking_assert (binfo); > binfo = get_binfo_at_offset (binfo, ipa_get_jf_known_type_offset (jfunc) > - + ie->indirect_info->anc_offset, > + + ie->indirect_info->offset, > ie->indirect_info->otr_type); > if (binfo) > target = gimple_get_virt_method_for_binfo (ie->indirect_info->otr_token, > @@ -1889,6 +2320,7 @@ update_indirect_edges_after_inlining (st > { > struct cgraph_indirect_call_info *ici = ie->indirect_info; > struct ipa_jump_func *jfunc; > + int param_index; > > next_ie = ie->next_callee; > > @@ -1902,14 +2334,27 @@ update_indirect_edges_after_inlining (st > continue; > } > > - jfunc = ipa_get_ith_jump_func (top, ici->param_index); > + param_index = ici->param_index; > + jfunc = ipa_get_ith_jump_func (top, param_index); > if (jfunc->type == IPA_JF_PASS_THROUGH > && ipa_get_jf_pass_through_operation (jfunc) == NOP_EXPR) > - ici->param_index = ipa_get_jf_pass_through_formal_id (jfunc); > + { > + if (ici->agg_contents > + && !ipa_jf_propagates_agg_p (cs, jfunc, param_index)) > + ici->param_index = -1; > + else > + ici->param_index = ipa_get_jf_pass_through_formal_id (jfunc); > + } > else if (jfunc->type == IPA_JF_ANCESTOR) > { > - ici->param_index = ipa_get_jf_ancestor_formal_id (jfunc); > - ici->anc_offset += ipa_get_jf_ancestor_offset (jfunc); > + if (ici->agg_contents > + && !ipa_jf_propagates_agg_p (cs, jfunc, param_index)) > + ici->param_index = -1; > + else > + { > + ici->param_index = ipa_get_jf_ancestor_formal_id (jfunc); > + ici->offset += ipa_get_jf_ancestor_offset (jfunc); > + } > } > else > /* Either we can find a destination for this edge now or never. */ > @@ -1921,7 +2366,8 @@ update_indirect_edges_after_inlining (st > if (ici->polymorphic) > new_direct_edge = try_make_edge_direct_virtual_call (ie, jfunc); > else > - new_direct_edge = try_make_edge_direct_simple_call (ie, jfunc); > + new_direct_edge = try_make_edge_direct_simple_call (cs, ie, jfunc, > + param_index); > > if (new_direct_edge) > { > @@ -2077,13 +2523,14 @@ ipa_node_removal_hook (struct cgraph_nod > ipa_free_node_params_substructures (IPA_NODE_REF (node)); > } > > -/* Hook that is called by cgraph.c when a node is duplicated. */ > +/* Hook that is called by cgraph.c when an edge is duplicated. */ > > static void > ipa_edge_duplication_hook (struct cgraph_edge *src, struct cgraph_edge *dst, > __attribute__((unused)) void *data) > { > struct ipa_edge_args *old_args, *new_args; > + unsigned int i; > > ipa_check_create_edge_args (); > > @@ -2092,6 +2539,12 @@ ipa_edge_duplication_hook (struct cgraph > > new_args->jump_functions = VEC_copy (ipa_jump_func_t, gc, > old_args->jump_functions); > + > + for (i = 0; i < VEC_length (ipa_jump_func_t, old_args->jump_functions); i++) > + VEC_index (ipa_jump_func_t, new_args->jump_functions, i)->agg.items > + = VEC_copy (ipa_agg_jf_item_t, gc, > + VEC_index (ipa_jump_func_t, > + old_args->jump_functions, i)->agg.items); > } > > /* Hook that is called by cgraph.c when a node is duplicated. */ > @@ -2790,8 +3243,12 @@ static void > ipa_write_jump_function (struct output_block *ob, > struct ipa_jump_func *jump_func) > { > - streamer_write_uhwi (ob, jump_func->type); > + struct ipa_agg_jf_item *item; > + struct bitpack_d bp; > + unsigned count; > + int i; > > + streamer_write_uhwi (ob, jump_func->type); > switch (jump_func->type) > { > case IPA_JF_UNKNOWN: > @@ -2808,16 +3265,28 @@ ipa_write_jump_function (struct output_b > stream_write_tree (ob, jump_func->value.pass_through.operand, true); > streamer_write_uhwi (ob, jump_func->value.pass_through.formal_id); > streamer_write_uhwi (ob, jump_func->value.pass_through.operation); > + bp = bitpack_create (ob->main_stream); > + bp_pack_value (&bp, jump_func->value.pass_through.agg_preserved, 1); > + streamer_write_bitpack (&bp); > break; > case IPA_JF_ANCESTOR: > streamer_write_uhwi (ob, jump_func->value.ancestor.offset); > stream_write_tree (ob, jump_func->value.ancestor.type, true); > streamer_write_uhwi (ob, jump_func->value.ancestor.formal_id); > + bp = bitpack_create (ob->main_stream); > + bp_pack_value (&bp, jump_func->value.ancestor.agg_preserved, 1); > + streamer_write_bitpack (&bp); > break; > - case IPA_JF_CONST_MEMBER_PTR: > - stream_write_tree (ob, jump_func->value.member_cst.pfn, true); > - stream_write_tree (ob, jump_func->value.member_cst.delta, false); > - break; > + } > + > + count = VEC_length (ipa_agg_jf_item_t, jump_func->agg.items); > + streamer_write_uhwi (ob, count); > + if (count) > + stream_write_tree (ob, jump_func->agg.type, true); > + FOR_EACH_VEC_ELT (ipa_agg_jf_item_t, jump_func->agg.items, i, item) > + { > + streamer_write_uhwi (ob, item->offset); > + stream_write_tree (ob, item->value, true); > } > } > > @@ -2828,8 +3297,10 @@ ipa_read_jump_function (struct lto_input > struct ipa_jump_func *jump_func, > struct data_in *data_in) > { > - jump_func->type = (enum jump_func_type) streamer_read_uhwi (ib); > + struct bitpack_d bp; > + int i, count; > > + jump_func->type = (enum jump_func_type) streamer_read_uhwi (ib); > switch (jump_func->type) > { > case IPA_JF_UNKNOWN: > @@ -2848,16 +3319,29 @@ ipa_read_jump_function (struct lto_input > jump_func->value.pass_through.formal_id = streamer_read_uhwi (ib); > jump_func->value.pass_through.operation > = (enum tree_code) streamer_read_uhwi (ib); > + bp = streamer_read_bitpack (ib); > + jump_func->value.pass_through.agg_preserved = bp_unpack_value (&bp, 1); > break; > case IPA_JF_ANCESTOR: > jump_func->value.ancestor.offset = streamer_read_uhwi (ib); > jump_func->value.ancestor.type = stream_read_tree (ib, data_in); > jump_func->value.ancestor.formal_id = streamer_read_uhwi (ib); > + bp = streamer_read_bitpack (ib); > + jump_func->value.ancestor.agg_preserved = bp_unpack_value (&bp, 1); > break; > - case IPA_JF_CONST_MEMBER_PTR: > - jump_func->value.member_cst.pfn = stream_read_tree (ib, data_in); > - jump_func->value.member_cst.delta = stream_read_tree (ib, data_in); > - break; > + } > + > + count = streamer_read_uhwi (ib); > + jump_func->agg.items = VEC_alloc (ipa_agg_jf_item_t, gc, count); > + if (count) > + jump_func->agg.type = stream_read_tree (ib, data_in); > + for (i = 0; i < count; i++) > + { > + struct ipa_agg_jf_item *item = VEC_quick_push (ipa_agg_jf_item_t, > + jump_func->agg.items, NULL); > + > + item->offset = streamer_read_uhwi (ib); > + item->value = stream_read_tree (ib, data_in); > } > } > > @@ -2872,9 +3356,10 @@ ipa_write_indirect_edge_info (struct out > struct bitpack_d bp; > > streamer_write_hwi (ob, ii->param_index); > - streamer_write_hwi (ob, ii->anc_offset); > + streamer_write_hwi (ob, ii->offset); > bp = bitpack_create (ob->main_stream); > bp_pack_value (&bp, ii->polymorphic, 1); > + bp_pack_value (&bp, ii->agg_contents, 1); > streamer_write_bitpack (&bp); > > if (ii->polymorphic) > @@ -2896,9 +3381,10 @@ ipa_read_indirect_edge_info (struct lto_ > struct bitpack_d bp; > > ii->param_index = (int) streamer_read_hwi (ib); > - ii->anc_offset = (HOST_WIDE_INT) streamer_read_hwi (ib); > + ii->offset = (HOST_WIDE_INT) streamer_read_hwi (ib); > bp = streamer_read_bitpack (ib); > ii->polymorphic = bp_unpack_value (&bp, 1); > + ii->agg_contents = bp_unpack_value (&bp, 1); > if (ii->polymorphic) > { > ii->otr_token = (HOST_WIDE_INT) streamer_read_hwi (ib); > Index: src/gcc/cgraph.h > =================================================================== > --- src.orig/gcc/cgraph.h > +++ src/gcc/cgraph.h > @@ -338,9 +338,11 @@ typedef enum cgraph_inline_failed_enum { > > struct GTY(()) cgraph_indirect_call_info > { > - /* Offset accumulated from ancestor jump functions of inlined call graph > - edges. */ > - HOST_WIDE_INT anc_offset; > + /* When polymorphic is set, this field contains offset where the object which > + was actually used in the polymorphic resides within a larger structure. > + If agg_contents is set, the field contains the offset within the aggregate > + from which the address to call was loaded. */ > + HOST_WIDE_INT offset; > /* OBJ_TYPE_REF_TOKEN of a polymorphic call (if polymorphic is set). */ > HOST_WIDE_INT otr_token; > /* Type of the object from OBJ_TYPE_REF_OBJECT. */ > @@ -353,6 +355,9 @@ struct GTY(()) cgraph_indirect_call_info > /* Set when the call is a virtual call with the parameter being the > associated object pointer rather than a simple direct call. */ > unsigned polymorphic : 1; > + /* Set when the call is a call of a pointer loaded from contents of an > + aggregate at offset. */ > + unsigned agg_contents : 1; > }; > > struct GTY((chain_next ("%h.next_caller"), chain_prev ("%h.prev_caller"))) cgraph_edge { > Index: src/gcc/ipa-cp.c > =================================================================== > --- src.orig/gcc/ipa-cp.c > +++ src/gcc/ipa-cp.c > @@ -1110,7 +1110,7 @@ ipa_get_indirect_edge_target (struct cgr > } > > token = ie->indirect_info->otr_token; > - anc_offset = ie->indirect_info->anc_offset; > + anc_offset = ie->indirect_info->offset; > otr_type = ie->indirect_info->otr_type; > > t = VEC_index (tree, known_vals, param_index); > Index: src/gcc/testsuite/gcc.dg/ipa/iinline-4.c > =================================================================== > --- /dev/null > +++ src/gcc/testsuite/gcc.dg/ipa/iinline-4.c > @@ -0,0 +1,187 @@ > +/* Verify that simple indirect calls are inlined even without early > + inlining.. */ > +/* { dg-do compile } */ > +/* { dg-options "-O3 -fdump-ipa-inline -fno-early-inlining" } */ > + > +struct S > +{ > + int i; > + void (*f)(struct S *); > + unsigned u; > +}; > + > +struct U > +{ > + struct U *next; > + struct S s; > + short a[8]; > +}; > + > +extern void non_existent(struct S *p, int); > + > +/* ----- 1 ----- */ > + > +static void hooray1 (struct S *p) > +{ > + non_existent (p, 1); > +} > + > +static void hiphip1 (struct S *p) > +{ > + p->f (p); > +} > + > +int test1 (void) > +{ > + struct S s; > + s.i = 1234; > + s.f = hooray1; > + s.u = 1001; > + hiphip1 (&s); > + return 0; > +} > + > +/* ----- 2 ----- */ > + > +struct S *gp; > + > +static void hooray2 (struct S *p) > +{ > + non_existent (p, 2); > +} > + > +static void hip2 (struct S *p) > +{ > + p->f (p); > +} > + > +static void hiphip2 (struct S *p) > +{ > + hip2 (p); > +} > + > +int test2 (void) > +{ > + struct S *p = gp; > + p->i = 2341; > + p->f = hooray2; > + p->u = 1002; > + hiphip2 (p); > + return 0; > +} > + > +/* ----- 3 ----- */ > + > +static void hooray3 (struct S *p) > +{ > + non_existent (p, 2); > +} > + > +static void hip3 (struct S *p) > +{ > + p->f (p); > +} > + > +static __attribute__ ((flatten)) void hiphip3 (struct S *p) > +{ > + hip3 (p); > +} > + > +int test3 (void) > +{ > + struct S *p = gp; > + p->i = 2341; > + p->f = hooray3; > + p->u = 1003; > + hiphip3 (p); > + return 0; > +} > + > +/* ----- 4 ----- */ > + > +static void hooray4 (struct S *p) > +{ > + non_existent (p, 3); > +} > + > +static void hiphip4 (struct S s) > +{ > + s.f (&s); > +} > + > +int test4(void) > +{ > + struct S s; > + s.i = 3412; > + s.f = hooray4; > + s.u = 1004; > + hiphip4 (s); > + return 0; > +} > + > +/* ----- 5 ----- */ > + > +struct U *gu; > + > +static void hooray5 (struct S *p) > +{ > + non_existent (p, 5); > +} > + > +static void hip5 (struct S *p) > +{ > + p->f (p); > +} > + > +static void hiphip5 (struct U *u) > +{ > + hip5 (&u->s); > +} > + > +int test5 (void) > +{ > + struct U *u = gu; > + u->next = u; > + u->s.i = 9876; > + u->s.f = hooray5; > + u->s.u = 1005; > + hiphip5 (u); > + return 0; > +} > + > +/* ----- 6 ----- */ > + > +static void hooray6 (struct S *p) > +{ > + non_existent (p, 6); > +} > + > +static void hip6 (struct S *p) > +{ > + p->f (p); > +} > + > +static __attribute__ ((flatten)) void hiphip6 (struct U *u) > +{ > + hip6 (&u->s); > +} > + > +int test6 (void) > +{ > + struct U *u = gu; > + u->next = u; > + u->s.i = 9876; > + u->s.f = hooray6; > + u->s.u = 1006; > + hiphip6 (u); > + return 0; > +} > + > + > +/* { dg-final { scan-ipa-dump "hooray1\[^\\n\]*inline copy in test1" "inline" } } */ > +/* { dg-final { scan-ipa-dump "hooray2\[^\\n\]*inline copy in test2" "inline" } } */ > +/* { dg-final { scan-ipa-dump "hooray3\[^\\n\]*inline copy in test3" "inline" } } */ > +/* { dg-final { scan-ipa-dump "hooray4\[^\\n\]*inline copy in test4" "inline" } } */ > +/* { dg-final { scan-ipa-dump "hooray5\[^\\n\]*inline copy in test5" "inline" } } */ > +/* { dg-final { scan-ipa-dump "hooray6\[^\\n\]*inline copy in test6" "inline" } } */ > +/* { dg-final { cleanup-ipa-dump "inline" } } */ > Index: src/gcc/testsuite/gcc.dg/ipa/iinline-5.c > =================================================================== > --- /dev/null > +++ src/gcc/testsuite/gcc.dg/ipa/iinline-5.c > @@ -0,0 +1,124 @@ > +/* Verify that simple indirect calls are inlined even without early > + inlining.. */ > +/* { dg-do run } */ > +/* { dg-options "-O3 -fdump-ipa-inline -fno-early-inlining" } */ > + > +extern void abort (void); > + > +struct S > +{ > + int i; > + void (*f)(struct S *); > + int j,k,l; > +}; > + > +struct Z > +{ > + unsigned u; > + void (*f)(struct Z *, int); > + struct Z *next; > +}; > + > +static struct Z *gz; > +static struct S *gs; > +static int gr = 111; > +char gc[1024]; > + > +static __attribute__ ((noinline, noclone)) struct S * > +get_s (void) > +{ > + return (struct S *) &gc; > +} > + > + > +static void wrong_target_1 (struct S *s) > +{ > + abort (); > +} > + > +static void wrong_target_2 (struct S *s) > +{ > + abort (); > +} > + > +static void wrong_target_3 (struct S *s) > +{ > + abort (); > +} > + > +static void good_target (struct Z *z, int i) > +{ > + gr = 0; > +} > + > +static void good_target_3 (struct S *s) > +{ > + gr = 0; > +} > + > +static void g1 (struct S *s) > +{ > + struct Z *z = (struct Z*) s; > + z->f (z, 8); > +} > + > +static void f1 (struct S *s) > +{ > + gz->f = good_target; > + g1 (s); > +} > + > +static void g2 (struct Z *z) > +{ > + z->f (z, 8); > +} > + > +static void f2 (struct S *s) > +{ > + gz->f = good_target; > + g2 ((struct Z*) s); > +} > + > +static void g3 (struct S *s) > +{ > + s->f (s); > +} > + > +static void h3 (struct Z *z) > +{ > + gs->f = good_target_3; > + g3 ((struct S *) z); > +} > + > +static void f3 (struct S *s) > +{ > + h3 ((struct Z*) s); > +} > + > +int main (int argc, char **argv) > +{ > + struct S *s = get_s(); > + s->i = 5678; > + s->f = wrong_target_1; > + s->j = 1234; > + gz = (struct Z *) s; > + f1 (s); > + > + s = get_s(); > + gz = (struct Z *) s; > + s->i = 9999; > + s->f = wrong_target_1; > + f2 (s); > + > + s = get_s(); > + gs = s; > + s->i = 9999; > + s->f = wrong_target_3; > + f3 (s); > + > + return gr; > +} > + > + > +/* { dg-final { scan-ipa-dump-not "wrong_target\[^\\n\]*inline copy in" "inline" } } */ > +/* { dg-final { cleanup-ipa-dump "inline" } } */ > Index: src/gcc/testsuite/gcc.dg/ipa/iinline-6.c > =================================================================== > --- /dev/null > +++ src/gcc/testsuite/gcc.dg/ipa/iinline-6.c > @@ -0,0 +1,72 @@ > +/* Verify that simple indirect calls are inlined even without early > + inlining.. */ > +/* { dg-do run } */ > +/* { dg-options "-O3 -fdump-ipa-inline -fno-early-inlining" } */ > + > +extern void abort (void); > + > +struct S > +{ > + int i; > + void (*f)(struct S *); > + int j,k,l; > +}; > + > +struct Z > +{ > + unsigned u; > + void (*f)(struct Z *, int); > + struct Z *next; > +}; > + > +static struct S *gs; > +static int gr = 111; > +char gc[1024]; > + > +static __attribute__ ((noinline, noclone)) struct S * > +get_s (void) > +{ > + return (struct S *) &gc; > +} > + > +static void wrong_target (struct S *s) > +{ > + abort (); > +} > + > +static void good_target (struct S *s) > +{ > + gr = 0; > +} > + > +static void g1 (struct S *s) > +{ > + s->f (s); > +} > + > +static void f2 (struct Z *z) > +{ > + gs->f = good_target; > + g1 ((struct S *) z); > +} > + > +static inline __attribute__ ((flatten)) void f1 (struct S *s) > +{ > + f2 ((struct Z *) s); > +} > + > +int main (int argc, char **argv) > +{ > + struct S *s = get_s(); > + s->i = 5678; > + s->f = wrong_target; > + s->j = 1234; > + gs = s; > + f1 (s); > + > + return gr; > +} > + > + > +/* { dg-final { scan-ipa-dump-not "wrong_target\[^\\n\]*inline copy in" "inline" } } */ > +/* { dg-final { cleanup-ipa-dump "inline" } } */ > Index: src/gcc/testsuite/gcc.dg/ipa/iinline-7.c > =================================================================== > --- /dev/null > +++ src/gcc/testsuite/gcc.dg/ipa/iinline-7.c > @@ -0,0 +1,157 @@ > +/* Verify that simple indirect calls are inlined even without early > + inlining.. */ > +/* { dg-do run } */ > +/* { dg-options "-O3 -fdump-ipa-inline -fno-early-inlining" } */ > + > +extern void abort (void); > + > +struct S > +{ > + int i; > + void (*f)(struct S *); > + int j,k,l; > +}; > + > +struct U > +{ > + struct U *next; > + struct S s; > + short a[8]; > +}; > + > +struct Z > +{ > + unsigned u; > + void (*f)(struct Z *, int); > + struct Z *next; > +}; > + > +static struct Z *gz; > +static struct U *gu; > +static int gr = 111; > +char gc[1024]; > + > +static __attribute__ ((noinline, noclone)) struct U * > +get_u (void) > +{ > + return (struct U *) &gc; > +} > + > +static void wrong_target_1 (struct S *s) > +{ > + abort (); > +} > + > +static void wrong_target_2 (struct S *s) > +{ > + abort (); > +} > + > +static void wrong_target_3 (struct S *s) > +{ > + abort (); > +} > + > +static void wrong_target_4 (struct S *s) > +{ > + abort (); > +} > + > +static void good_target (struct Z *z, int i) > +{ > + gr = 0; > +} > + > +static void good_target_4 (struct S *s) > +{ > + gr = 0; > +} > + > +static void g1 (struct S *s) > +{ > + struct Z *z = (struct Z*) s; > + z->f (z, 8); > +} > + > +static void f1 (struct U *u) > +{ > + gz->f = good_target; > + g1 (&u->s); > +} > + > +static void g2 (struct Z *z) > +{ > + z->f (z, 8); > +} > + > +static void f2 (struct U *u) > +{ > + gz->f = good_target; > + g2 ((struct Z*) &u->s); > +} > + > +static void h3 (struct Z *z) > +{ > + z->f (z, 8); > +} > + > +static void g3 (struct S *s) > +{ > + h3 ((struct Z*) s); > +} > + > +static void f3 (struct U *u) > +{ > + gz->f = good_target; > + g3 (&u->s); > +} > + > +static void h4 (struct S *s) > +{ > + s->f (s); > +} > + > +static void g4 (struct U *u) > +{ > + h4 (&u->s); > +} > + > +static inline __attribute__ ((flatten)) void f4 (struct Z *z) > +{ > + gu->s.f = good_target_4; > + g4 ((struct U *) z); > +} > + > +int main (int argc, char **argv) > +{ > + struct U *u = get_u (); > + u->next = u; > + u->s.i = 5678; > + u->s.f = wrong_target_1; > + u->s.j = 1234; > + gz = (struct Z *) &u->s; > + f1 (u); > + > + u = get_u(); > + u->s.i = 9999; > + u->s.f = wrong_target_2; > + gz = (struct Z *) &u->s; > + f2 (u); > + > + u = get_u(); > + u->s.i = 9998; > + u->s.f = wrong_target_3; > + gz = (struct Z *) &u->s; > + f3 (u); > + > + u = get_u(); > + u->s.i = 9998; > + u->s.f = wrong_target_4; > + gu = u; > + f4 ((struct Z *) u); > + return gr; > +} > + > + > +/* { dg-final { scan-ipa-dump-not "wrong_target\[^\\n\]*inline copy in" "inline" } } */ > +/* { dg-final { cleanup-ipa-dump "inline" } } */ > >
Index: src/gcc/ipa-prop.h =================================================================== --- src.orig/gcc/ipa-prop.h +++ src/gcc/ipa-prop.h @@ -44,10 +44,6 @@ along with GCC; see the file COPYING3. argument. Unknown - neither of the above. - IPA_JF_CONST_MEMBER_PTR stands for C++ member pointers, it is a special - constant in this regard because it is in fact a structure consisting of two - values. Other constants are represented with IPA_JF_CONST. - IPA_JF_ANCESTOR is a special pass-through jump function, which means that the result is an address of a part of the object pointed to by the formal parameter to which the function refers. It is mainly intended to represent @@ -74,7 +70,6 @@ enum jump_func_type IPA_JF_UNKNOWN = 0, /* newly allocated and zeroed jump functions default */ IPA_JF_KNOWN_TYPE, /* represented by field known_type */ IPA_JF_CONST, /* represented by field costant */ - IPA_JF_CONST_MEMBER_PTR, /* represented by field member_cst */ IPA_JF_PASS_THROUGH, /* represented by field pass_through */ IPA_JF_ANCESTOR /* represented by field ancestor */ }; @@ -104,6 +99,13 @@ struct GTY(()) ipa_pass_through_data arithmetic operation where the caller's parameter is the first operand and operand field from this structure is the second one. */ enum tree_code operation; + /* When the passed value is a pointer or an aggregate passed by value, it is + set to true only when we are certain that no write to the object it points + to or the aggregate has occurred since the caller functions started + execution, except for changes noted in the aggregate part of the jump + function (see description of ipa_agg_jump_function). The flag is used + only when this operation is NOP_EXPR. */ + bool agg_preserved; }; /* Structure holding data required to describe an ancestor pass-through @@ -117,21 +119,57 @@ struct GTY(()) ipa_ancestor_jf_data tree type; /* Number of the caller's formal parameter being passed. */ int formal_id; + /* Flag with the same meaning like agg_preserve in ipa_pass_through_data. */ + bool agg_preserved; }; -/* Structure holding a C++ member pointer constant. Holds a pointer to the - method and delta offset. */ -struct GTY(()) ipa_member_ptr_cst +/* An element in an aggegate part of a jump function describing a known value + at a given offset. When the this is part of a pass-through jump function + with agg_preserved set or an ancestor jump function with agg_preserved set, + all unlisted positions are assumed to be preserved but the value can be a + type node, which means that the particular piece (starting at offset and + having the size of the type) is clobbered with an unknown value. When + agg_preserved is false or the type of the containing jump function is + different, all unlisted parts are assumed to be unknown and all values must + fullfill is_gimple_ip_invariant. */ + +typedef struct GTY(()) ipa_agg_jf_item { - tree pfn; - tree delta; + /* The offset at which the known value is located within the aggregate. */ + HOST_WIDE_INT offset; + + /* The known constant or type if this is a clobber. */ + tree value; +} ipa_agg_jf_item_t; + +DEF_VEC_O (ipa_agg_jf_item_t); +DEF_VEC_ALLOC_O (ipa_agg_jf_item_t, gc); + +/* Aggregate jump function - i.e. description of contents of aggregates passed + either by reference or value. */ + +struct GTY(()) ipa_agg_jump_function +{ + /* Description of the individual items. */ + VEC (ipa_agg_jf_item_t, gc) *items; + /* Type which was used to store the contents of the aggregate. Not + meaningful if items is NULL. */ + tree type; }; +typedef struct ipa_agg_jump_function *ipa_agg_jump_function_p; +DEF_VEC_P (ipa_agg_jump_function_p); +DEF_VEC_ALLOC_P (ipa_agg_jump_function_p, heap); + /* A jump function for a callsite represents the values passed as actual arguments of the callsite. See enum jump_func_type for the various types of jump functions supported. */ typedef struct GTY (()) ipa_jump_func { + /* Aggregate contants description. See struct ipa_agg_jump_function and its + description. */ + struct ipa_agg_jump_function agg; + enum jump_func_type type; /* Represents a value of a jump function. pass_through is used only in jump function context. constant represents the actual constant in constant jump @@ -140,7 +178,6 @@ typedef struct GTY (()) ipa_jump_func { struct ipa_known_type_data GTY ((tag ("IPA_JF_KNOWN_TYPE"))) known_type; tree GTY ((tag ("IPA_JF_CONST"))) constant; - struct ipa_member_ptr_cst GTY ((tag ("IPA_JF_CONST_MEMBER_PTR"))) member_cst; struct ipa_pass_through_data GTY ((tag ("IPA_JF_PASS_THROUGH"))) pass_through; struct ipa_ancestor_jf_data GTY ((tag ("IPA_JF_ANCESTOR"))) ancestor; } GTY ((desc ("%1.type"))) value; @@ -214,6 +251,15 @@ ipa_get_jf_pass_through_operation (struc return jfunc->value.pass_through.operation; } +/* Return the agg_preserved flag of a pass through jump functin JFUNC. */ + +static inline bool +ipa_get_jf_pass_through_agg_preserved (struct ipa_jump_func *jfunc) +{ + gcc_checking_assert (jfunc->type == IPA_JF_PASS_THROUGH); + return jfunc->value.pass_through.agg_preserved; +} + /* Return the offset of an ancestor jump function JFUNC. */ static inline HOST_WIDE_INT @@ -242,13 +288,13 @@ ipa_get_jf_ancestor_formal_id (struct ip return jfunc->value.ancestor.formal_id; } -/* Return the pfn part of a member pointer constant jump function JFUNC. */ +/* Return the agg_preserved flag of an ancestor jump functin JFUNC. */ -static inline tree -ipa_get_jf_member_ptr_pfn (struct ipa_jump_func *jfunc) +static inline bool +ipa_get_jf_ancestor_agg_preserved (struct ipa_jump_func *jfunc) { - gcc_checking_assert (jfunc->type == IPA_JF_CONST_MEMBER_PTR); - return jfunc->value.member_cst.pfn; + gcc_checking_assert (jfunc->type == IPA_JF_ANCESTOR); + return jfunc->value.ancestor.agg_preserved; } /* Summary describing a single formal parameter. */ @@ -456,6 +502,14 @@ struct cgraph_edge *ipa_make_edge_direct /* Functions related to both. */ void ipa_analyze_node (struct cgraph_node *); +/* Aggregate jump function related functions. */ +tree ipa_find_agg_cst_for_param (struct ipa_agg_jump_function *, HOST_WIDE_INT, + struct cgraph_node *, int); +bool ipa_load_from_parm_agg (struct ipa_node_params *, gimple, tree, int *, + HOST_WIDE_INT *); +bool ipa_jf_propagates_agg_p (struct cgraph_edge *, struct ipa_jump_func *, + int); + /* Debugging interface. */ void ipa_print_node_params (FILE *, struct cgraph_node *node); void ipa_print_all_params (FILE *); Index: src/gcc/ipa-prop.c =================================================================== --- src.orig/gcc/ipa-prop.c +++ src/gcc/ipa-prop.c @@ -47,8 +47,9 @@ along with GCC; see the file COPYING3. struct param_analysis_info { - bool modified; - bitmap visited_statements; + bool parm_modified, ref_modified; + bitmap parm_visited_statements, ref_visited_statements; + tree deref; }; /* Vector where the parameter infos are actually stored. */ @@ -181,24 +182,21 @@ ipa_print_node_jump_functions_for_edge ( } fprintf (f, "\n"); } - else if (type == IPA_JF_CONST_MEMBER_PTR) - { - fprintf (f, "CONST MEMBER PTR: "); - print_generic_expr (f, jump_func->value.member_cst.pfn, 0); - fprintf (f, ", "); - print_generic_expr (f, jump_func->value.member_cst.delta, 0); - fprintf (f, "\n"); - } else if (type == IPA_JF_PASS_THROUGH) { fprintf (f, "PASS THROUGH: "); - fprintf (f, "%d, op %s ", + fprintf (f, "%d, op %s", jump_func->value.pass_through.formal_id, tree_code_name[(int) jump_func->value.pass_through.operation]); if (jump_func->value.pass_through.operation != NOP_EXPR) - print_generic_expr (f, - jump_func->value.pass_through.operand, 0); + { + fprintf (f, " "); + print_generic_expr (f, + jump_func->value.pass_through.operand, 0); + } + if (jump_func->value.pass_through.agg_preserved) + fprintf (f, ", agg_preserved"); fprintf (f, "\n"); } else if (type == IPA_JF_ANCESTOR) @@ -208,8 +206,33 @@ ipa_print_node_jump_functions_for_edge ( jump_func->value.ancestor.formal_id, jump_func->value.ancestor.offset); print_generic_expr (f, jump_func->value.ancestor.type, 0); + if (jump_func->value.ancestor.agg_preserved) + fprintf (f, ", agg_preserved"); fprintf (f, "\n"); } + + if (jump_func->agg.items) + { + struct ipa_agg_jf_item *item; + int j; + + fprintf (f, " Aggregte contents:\n"); + FOR_EACH_VEC_ELT (ipa_agg_jf_item_t, jump_func->agg.items, + j, item) + { + fprintf (f, " offset: " HOST_WIDE_INT_PRINT_DEC ", ", + item->offset); + if (TYPE_P (item->value)) + fprintf (f, "clobber of " HOST_WIDE_INT_PRINT_DEC " bits", + tree_low_cst (TYPE_SIZE (item->value), 1)); + else + { + fprintf (f, "cst: "); + print_generic_expr (f, item->value, 0); + } + fprintf (f, "\n"); + } + } } } @@ -289,12 +312,14 @@ ipa_set_jf_constant (struct ipa_jump_fun /* Set JFUNC to be a simple pass-through jump function. */ static void -ipa_set_jf_simple_pass_through (struct ipa_jump_func *jfunc, int formal_id) +ipa_set_jf_simple_pass_through (struct ipa_jump_func *jfunc, int formal_id, + bool agg_preserved) { jfunc->type = IPA_JF_PASS_THROUGH; jfunc->value.pass_through.operand = NULL_TREE; jfunc->value.pass_through.formal_id = formal_id; jfunc->value.pass_through.operation = NOP_EXPR; + jfunc->value.pass_through.agg_preserved = agg_preserved; } /* Set JFUNC to be an arithmetic pass through jump function. */ @@ -307,30 +332,20 @@ ipa_set_jf_arith_pass_through (struct ip jfunc->value.pass_through.operand = operand; jfunc->value.pass_through.formal_id = formal_id; jfunc->value.pass_through.operation = operation; + jfunc->value.pass_through.agg_preserved = false; } /* Set JFUNC to be an ancestor jump function. */ static void ipa_set_ancestor_jf (struct ipa_jump_func *jfunc, HOST_WIDE_INT offset, - tree type, int formal_id) + tree type, int formal_id, bool agg_preserved) { jfunc->type = IPA_JF_ANCESTOR; jfunc->value.ancestor.formal_id = formal_id; jfunc->value.ancestor.offset = offset; jfunc->value.ancestor.type = type; -} - -/* Simple function filling in a member pointer constant jump function (with PFN - and DELTA as the constant value) into JFUNC. */ - -static void -ipa_set_jf_member_ptr_cst (struct ipa_jump_func *jfunc, - tree pfn, tree delta) -{ - jfunc->type = IPA_JF_CONST_MEMBER_PTR; - jfunc->value.member_cst.pfn = pfn; - jfunc->value.member_cst.delta = delta; + jfunc->value.ancestor.agg_preserved = agg_preserved; } /* Structure to be passed in between detect_type_change and @@ -584,30 +599,28 @@ mark_modified (ao_ref *ao ATTRIBUTE_UNUS return true; } -/* Return true if the formal parameter PARM might have been modified in this +/* Return true if the formal parameter PARM is known not to be modified in this function before reaching the statement STMT. PARM_AINFO is a pointer to a structure containing temporary information about PARM. */ static bool -is_parm_modified_before_stmt (struct param_analysis_info *parm_ainfo, - gimple stmt, tree parm) +is_parm_preserved_before_stmt (struct param_analysis_info *parm_ainfo, + gimple stmt, tree parm) { bool modified = false; ao_ref refd; - if (parm_ainfo->modified) - return true; + if (parm_ainfo && parm_ainfo->parm_modified) + return false; gcc_checking_assert (gimple_vuse (stmt) != NULL_TREE); ao_ref_init (&refd, parm); walk_aliased_vdefs (&refd, gimple_vuse (stmt), mark_modified, - &modified, &parm_ainfo->visited_statements); - if (modified) - { - parm_ainfo->modified = true; - return true; - } - return false; + &modified, + parm_ainfo ? &parm_ainfo->parm_visited_statements : NULL); + if (parm_ainfo && modified) + parm_ainfo->parm_modified = true; + return !modified; } /* If STMT is an assignment that loads a value from an parameter declaration, @@ -631,12 +644,125 @@ load_from_unmodified_param (struct ipa_n index = ipa_get_param_decl_index (info, op1); if (index < 0 - || is_parm_modified_before_stmt (&parms_ainfo[index], stmt, op1)) + || !is_parm_preserved_before_stmt (&parms_ainfo[index], stmt, op1)) return -1; return index; } +/* Return true if the data pointed to by REF is known to be unmodified in + this function before reaching statement STMT. PARM_AINFO, if non-NULL, is a + pointer to a structure containing temporary information about PARM. */ + +static bool +is_parm_ref_data_preserved (struct param_analysis_info *parm_ainfo, + gimple stmt, tree ref) +{ + bool modified = false; + ao_ref refd; + + gcc_checking_assert (gimple_vuse (stmt)); + if (parm_ainfo && parm_ainfo->ref_modified) + return false; + + ao_ref_init (&refd, ref); + walk_aliased_vdefs (&refd, gimple_vuse (stmt), mark_modified, &modified, + parm_ainfo ? &parm_ainfo->ref_visited_statements : NULL); + if (parm_ainfo && modified) + parm_ainfo->ref_modified = true; + return !modified; +} + +/* Return true if the data pointed to by PARM is known to be unmodified in this + function before reaching call statement CALL. PARM_AINFO is a pointer to a + structure containing temporary information about PARM. We use the + pointed-to type for TBAA, so make sure any consumer in any indirect calle + uses that type to read the data (see ipa_agg_types_propagable_p). */ + +static bool +is_parm_ref_data_preserved_call (struct param_analysis_info *parm_ainfo, + gimple call, tree parm) +{ + /* It's unnecessary to calculate anything about memory contnets for a const + function because it is not goin to use it. But do not cache the result + either. Also, no such calculations for non-pointers. */ + if (!gimple_vuse (call) + || !POINTER_TYPE_P (TREE_TYPE (parm))) + return false; + + if (!parm_ainfo->deref) + /* Use of this alias type requires that we make sure the data we pass to + the function is actually of the supposed type. But we do that both when + gathering the contents of aggregates and when propagating. */ + parm_ainfo->deref = build2 (MEM_REF, TREE_TYPE (TREE_TYPE (parm)), parm, + build_int_cst (TREE_TYPE (parm), 0)); + + + return is_parm_ref_data_preserved (parm_ainfo, call, parm_ainfo->deref); +} + +/* Return true if we can prove that OP is a memory reference loading unmodified + data from an aggregate passed as a parameter and if the aggregate is passed + by reference, that the alias type of the load corresponds to the type of the + formal parameter (so that we can rely on this type for TBAA in callers). + INFO and PARMS_AINFO describe parameters of the current function (but the + latter can be NULL), STMT is the load statement. If function returns true, + *INDEX_P and *OFFSET_P is filled with the parameter index and offset within + the aggregate respectively. */ + +static bool +ipa_load_from_parm_agg_1 (struct ipa_node_params *info, + struct param_analysis_info *parms_ainfo, gimple stmt, + tree op, int *index_p, HOST_WIDE_INT *offset_p) +{ + HOST_WIDE_INT size, max_size; + tree base = get_ref_base_and_extent (op, offset_p, &size, &max_size); + + if (max_size == -1 || max_size != size || *offset_p < 0) + return false; + + if (DECL_P (base)) + { + int index = ipa_get_param_decl_index (info, base); + if (index >= 0 + && is_parm_preserved_before_stmt (parms_ainfo ? &parms_ainfo[index] + : NULL, stmt, base)) + { + *index_p = index; + return true; + } + } + else if (TREE_CODE (base) == MEM_REF + && TREE_CODE (TREE_OPERAND (base, 0)) == SSA_NAME + && SSA_NAME_IS_DEFAULT_DEF (TREE_OPERAND (base, 0)) + && integer_zerop (TREE_OPERAND (base, 1))) + { + tree parm = SSA_NAME_VAR (TREE_OPERAND (base, 0)); + int index = ipa_get_param_decl_index (info, parm); + if (index >= 0 + && POINTER_TYPE_P (TREE_TYPE (parm)) + && (TYPE_MAIN_VARIANT (TREE_TYPE (TREE_TYPE (TREE_OPERAND (base, 1)))) + == TYPE_MAIN_VARIANT (TREE_TYPE (TREE_TYPE (parm)))) + && is_parm_ref_data_preserved (parms_ainfo ? &parms_ainfo[index] + : NULL, stmt, base)) + { + *index_p = index; + return true; + } + } + return false; +} + +/* just like the previous function, just without the param_analysis_info + pointer, for users outside of this file. */ + +bool +ipa_load_from_parm_agg (struct ipa_node_params *info, gimple stmt, + tree op, int *index_p, HOST_WIDE_INT *offset_p) +{ + return ipa_load_from_parm_agg_1 (info, NULL, stmt, op, index_p, offset_p); +} + /* Given that an actual argument is an SSA_NAME (given in NAME) and is a result of an assignment statement STMT, try to determine whether we are actually handling any of the following cases and construct an appropriate jump @@ -734,7 +860,11 @@ compute_complex_assign_jump_func (struct } else if (gimple_assign_single_p (stmt) && !detect_type_change_ssa (tc_ssa, call, jfunc)) - ipa_set_jf_simple_pass_through (jfunc, index); + { + bool agg_p = is_parm_ref_data_preserved_call (&parms_ainfo[index], + call, tc_ssa); + ipa_set_jf_simple_pass_through (jfunc, index, agg_p); + } return; } @@ -760,7 +890,9 @@ compute_complex_assign_jump_func (struct index = ipa_get_param_decl_index (info, SSA_NAME_VAR (ssa)); if (index >= 0 && !detect_type_change (op1, base, call, jfunc, offset)) - ipa_set_ancestor_jf (jfunc, offset, TREE_TYPE (op1), index); + ipa_set_ancestor_jf (jfunc, offset, TREE_TYPE (op1), index, + is_parm_ref_data_preserved_call (&parms_ainfo[index], + call, ssa)); } /* Extract the base, offset and MEM_REF expression from a statement ASSIGN if @@ -831,6 +963,7 @@ get_ancestor_addr_info (gimple assign, t static void compute_complex_ancestor_jump_func (struct ipa_node_params *info, + struct param_analysis_info *parms_ainfo, struct ipa_jump_func *jfunc, gimple call, gimple phi) { @@ -884,7 +1017,9 @@ compute_complex_ancestor_jump_func (stru } if (!detect_type_change (obj, expr, call, jfunc, offset)) - ipa_set_ancestor_jf (jfunc, offset, TREE_TYPE (obj), index); + ipa_set_ancestor_jf (jfunc, offset, TREE_TYPE (obj), index, + is_parm_ref_data_preserved_call (&parms_ainfo[index], + call, parm)); } /* Given OP which is passed as an actual argument to a called function, @@ -919,55 +1054,6 @@ compute_known_type_jump_func (tree op, s ipa_set_jf_known_type (jfunc, offset, TREE_TYPE (base), TREE_TYPE (op)); } - -/* Determine the jump functions of scalar arguments. Scalar means SSA names - and constants of a number of selected types. INFO is the ipa_node_params - structure associated with the caller, PARMS_AINFO describes state of - analysis with respect to individual formal parameters. ARGS is the - ipa_edge_args structure describing the callsite CALL which is the call - statement being examined.*/ - -static void -compute_scalar_jump_functions (struct ipa_node_params *info, - struct param_analysis_info *parms_ainfo, - struct ipa_edge_args *args, - gimple call) -{ - tree arg; - unsigned num = 0; - - for (num = 0; num < gimple_call_num_args (call); num++) - { - struct ipa_jump_func *jfunc = ipa_get_ith_jump_func (args, num); - arg = gimple_call_arg (call, num); - - if (is_gimple_ip_invariant (arg)) - ipa_set_jf_constant (jfunc, arg); - else if (TREE_CODE (arg) == SSA_NAME) - { - if (SSA_NAME_IS_DEFAULT_DEF (arg)) - { - int index = ipa_get_param_decl_index (info, SSA_NAME_VAR (arg)); - - if (index >= 0 - && !detect_type_change_ssa (arg, call, jfunc)) - ipa_set_jf_simple_pass_through (jfunc, index); - } - else - { - gimple stmt = SSA_NAME_DEF_STMT (arg); - if (is_gimple_assign (stmt)) - compute_complex_assign_jump_func (info, parms_ainfo, jfunc, - call, stmt, arg); - else if (gimple_code (stmt) == GIMPLE_PHI) - compute_complex_ancestor_jump_func (info, jfunc, call, stmt); - } - } - else - compute_known_type_jump_func (arg, jfunc, call); - } -} - /* Inspect the given TYPE and return true iff it has the same structure (the same number of fields of the same types) as a C++ member pointer. If METHOD_PTR and DELTA are non-NULL, store the trees representing the @@ -983,14 +1069,16 @@ type_like_member_ptr_p (tree type, tree fld = TYPE_FIELDS (type); if (!fld || !POINTER_TYPE_P (TREE_TYPE (fld)) - || TREE_CODE (TREE_TYPE (TREE_TYPE (fld))) != METHOD_TYPE) + || TREE_CODE (TREE_TYPE (TREE_TYPE (fld))) != METHOD_TYPE + || !host_integerp (DECL_FIELD_OFFSET (fld), 1)) return false; if (method_ptr) *method_ptr = fld; fld = DECL_CHAIN (fld); - if (!fld || INTEGRAL_TYPE_P (fld)) + if (!fld || INTEGRAL_TYPE_P (fld) + || !host_integerp (DECL_FIELD_OFFSET (fld), 1)) return false; if (delta) *delta = fld; @@ -1001,54 +1089,9 @@ type_like_member_ptr_p (tree type, tree return true; } -/* Go through arguments of the CALL and for every one that looks like a member - pointer, check whether it can be safely declared pass-through and if so, - mark that to the corresponding item of jump FUNCTIONS. Return true iff - there are non-pass-through member pointers within the arguments. INFO - describes formal parameters of the caller. PARMS_INFO is a pointer to a - vector containing intermediate information about each formal parameter. */ - -static bool -compute_pass_through_member_ptrs (struct ipa_node_params *info, - struct param_analysis_info *parms_ainfo, - struct ipa_edge_args *args, - gimple call) -{ - bool undecided_members = false; - unsigned num; - tree arg; - - for (num = 0; num < gimple_call_num_args (call); num++) - { - arg = gimple_call_arg (call, num); - - if (type_like_member_ptr_p (TREE_TYPE (arg), NULL, NULL)) - { - if (TREE_CODE (arg) == PARM_DECL) - { - int index = ipa_get_param_decl_index (info, arg); - - gcc_assert (index >=0); - if (!is_parm_modified_before_stmt (&parms_ainfo[index], call, - arg)) - { - struct ipa_jump_func *jfunc = ipa_get_ith_jump_func (args, - num); - ipa_set_jf_simple_pass_through (jfunc, index); - } - else - undecided_members = true; - } - else - undecided_members = true; - } - } - - return undecided_members; -} - /* If RHS is an SSA_NAME and it is defined by a simple copy assign statement, - return the rhs of its defining statement. */ + return the rhs of its defining statement. Otherwise return RHS as it + is. */ static inline tree get_ssa_def_if_simple_copy (tree rhs) @@ -1065,104 +1108,233 @@ get_ssa_def_if_simple_copy (tree rhs) return rhs; } -/* Traverse statements from CALL backwards, scanning whether the argument ARG - which is a member pointer is filled in with constant values. If it is, fill - the jump function JFUNC in appropriately. METHOD_FIELD and DELTA_FIELD are - fields of the record type of the member pointer. To give an example, we - look for a pattern looking like the following: - - D.2515.__pfn ={v} printStuff; - D.2515.__delta ={v} 0; - i_1 = doprinting (D.2515); */ +/* Helper for determine_known_aggregate_parts, initializes *R for an aggregate + passed by reference based on BASE and with the given TYPE. */ static void -determine_cst_member_ptr (gimple call, tree arg, tree method_field, - tree delta_field, struct ipa_jump_func *jfunc) +init_ao_ref_for_byref_agg_jf (ao_ref *r, tree base, tree type) { + static tree alias_all_type = NULL; + + if (!alias_all_type) + alias_all_type = build_pointer_type_for_mode (char_type_node, + ptr_mode, true); + ao_ref_init (r, build2 (MEM_REF, type, base, + build_int_cst (alias_all_type, 0))); +} + +/* TODO: Turn this into a PARAM. */ +#define IPA_MAX_AFF_JF_ITEMS 16 + +/* Simple linked list, describing known contents of an aggregate beforere + call. */ + +struct ipa_known_agg_contents_list +{ + /* Offset and size of the described part of the aggregate. */ + HOST_WIDE_INT offset, size; + /* Known constant value or NULL if the contents is known to be unknown. */ + tree constant; + /* Pointer to the next structure in the list. */ + struct ipa_known_agg_contents_list *next; +}; + +/* Traverse statements from CALL backwards, scanning whether an aggregate given + in ARG is filled in with constant values. ARG can either be an aggregate + expression or a pointer to an aggregate. JFUNC is the jump function into + which the constants are subsequently stored. */ + +static void +determine_known_aggregate_parts (gimple call, tree arg, + struct ipa_jump_func *jfunc) +{ + struct ipa_known_agg_contents_list *list = NULL; + int item_count = 0, const_count = 0; + HOST_WIDE_INT arg_offset, arg_size; gimple_stmt_iterator gsi; - tree method = NULL_TREE; - tree delta = NULL_TREE; + tree arg_base; + bool by_ref; + ao_ref r; - gsi = gsi_for_stmt (call); + /* The function operates in three stages. First, we prepare r, arg_base and + arg_offset based on what is actually passed as an actual argument. */ + + if (POINTER_TYPE_P (TREE_TYPE (arg))) + { + gcc_checking_assert (AGGREGATE_TYPE_P (TREE_TYPE (TREE_TYPE (arg)))); + if (TREE_CODE (arg) == SSA_NAME) + { + if (!host_integerp (TYPE_SIZE (TREE_TYPE (TREE_TYPE (arg))), 1)) + return; + + by_ref = true; + arg_base = arg; + arg_offset = 0; + arg_size = tree_low_cst (TYPE_SIZE (TREE_TYPE (TREE_TYPE (arg))), 1); + init_ao_ref_for_byref_agg_jf (&r, arg_base, + TREE_TYPE (TREE_TYPE (arg))); + } + else if (TREE_CODE (arg) == ADDR_EXPR) + { + HOST_WIDE_INT arg_max_size; + + arg = TREE_OPERAND (arg, 0); + arg_base = get_ref_base_and_extent (arg, &arg_offset, &arg_size, + &arg_max_size); + if (arg_max_size == -1 + || arg_max_size != arg_size + || arg_offset < 0) + return; + if (DECL_P (arg_base)) + { + by_ref = false; + ao_ref_init (&r, arg); + } + else + return; + } + else + return; + } + else + { + HOST_WIDE_INT arg_max_size; + + gcc_checking_assert (AGGREGATE_TYPE_P (TREE_TYPE (arg))); + + by_ref = false; + arg_base = get_ref_base_and_extent (arg, &arg_offset, &arg_size, + &arg_max_size); + if (arg_max_size == -1 + || arg_max_size != arg_size + || arg_offset < 0) + return; + ao_ref_init (&r, arg); + } + + /* Second stage walks back the BB, looks at individual statements and as long + as it is confident of how the statements affect contents of the + aggregates, it builds a sorted linked list of ipa_agg_jf_list structures + describing it. */ + gsi = gsi_for_stmt (call); gsi_prev (&gsi); for (; !gsi_end_p (gsi); gsi_prev (&gsi)) { + struct ipa_known_agg_contents_list *n, **p; gimple stmt = gsi_stmt (gsi); - tree lhs, rhs, fld; + HOST_WIDE_INT lhs_offset, lhs_size, lhs_max_size; + tree lhs, rhs, lhs_base; + bool partial_overlap; - if (!stmt_may_clobber_ref_p (stmt, arg)) + if (!stmt_may_clobber_ref_p_1 (stmt, &r)) continue; if (!gimple_assign_single_p (stmt)) - return; + break; lhs = gimple_assign_lhs (stmt); rhs = gimple_assign_rhs1 (stmt); + if (!is_gimple_reg_type (rhs)) + break; - if (TREE_CODE (lhs) != COMPONENT_REF - || TREE_OPERAND (lhs, 0) != arg) - return; + lhs_base = get_ref_base_and_extent (lhs, &lhs_offset, &lhs_size, + &lhs_max_size); + if (lhs_max_size == -1 + || lhs_max_size != lhs_size + || (lhs_offset < arg_offset + && lhs_offset + lhs_size > arg_offset) + || (lhs_offset < arg_offset + arg_size + && lhs_offset + lhs_size > arg_offset + arg_size)) + break; - fld = TREE_OPERAND (lhs, 1); - if (!method && fld == method_field) + if (by_ref) { - rhs = get_ssa_def_if_simple_copy (rhs); - if (TREE_CODE (rhs) == ADDR_EXPR - && TREE_CODE (TREE_OPERAND (rhs, 0)) == FUNCTION_DECL - && TREE_CODE (TREE_TYPE (TREE_OPERAND (rhs, 0))) == METHOD_TYPE) - { - method = TREE_OPERAND (rhs, 0); - if (delta) - { - ipa_set_jf_member_ptr_cst (jfunc, rhs, delta); - return; - } - } - else - return; + if (TREE_CODE (lhs_base) != MEM_REF + || TREE_OPERAND (lhs_base, 0) != arg_base + || !integer_zerop (TREE_OPERAND (lhs_base, 1)) + /* We record the access type here and use it for checks later so + make sure there is only one. */ + || (TYPE_MAIN_VARIANT (TREE_TYPE (TREE_TYPE ( + TREE_OPERAND (lhs_base, 1)))) + != TYPE_MAIN_VARIANT (TREE_TYPE (TREE_TYPE (arg_base))))) + break; } + else if (lhs_base != arg_base) + break; + + if (lhs_offset + lhs_size < arg_offset + || lhs_offset >= (arg_offset + arg_size)) + continue; - if (!delta && fld == delta_field) + partial_overlap = false; + p = &list; + while (*p && (*p)->offset < lhs_offset) { - rhs = get_ssa_def_if_simple_copy (rhs); - if (TREE_CODE (rhs) == INTEGER_CST) + if ((*p)->offset + (*p)->size > lhs_offset) { - delta = rhs; - if (method) - { - ipa_set_jf_member_ptr_cst (jfunc, rhs, delta); - return; - } + partial_overlap = true; + break; } + p = &(*p)->next; + } + if (partial_overlap) + break; + if (*p && (*p)->offset < lhs_offset + lhs_size) + { + if ((*p)->offset == lhs_offset && (*p)->size == lhs_size) + /* We already know this value is subsequently overwritten with + something else. */ + continue; else - return; + /* Otherwise this is a partial overlap which we cannot + represent. */ + break; } - } - - return; -} -/* Go through the arguments of the CALL and for every member pointer within - tries determine whether it is a constant. If it is, create a corresponding - constant jump function in FUNCTIONS which is an array of jump functions - associated with the call. */ - -static void -compute_cst_member_ptr_arguments (struct ipa_edge_args *args, - gimple call) -{ - unsigned num; - tree arg, method_field, delta_field; - - for (num = 0; num < gimple_call_num_args (call); num++) - { - struct ipa_jump_func *jfunc = ipa_get_ith_jump_func (args, num); - arg = gimple_call_arg (call, num); + rhs = get_ssa_def_if_simple_copy (rhs); + n = XALLOCA (struct ipa_known_agg_contents_list); + n->size = lhs_size; + n->offset = lhs_offset; + if (is_gimple_ip_invariant (rhs)) + { + n->constant = rhs; + const_count++; + } + else + n->constant = NULL_TREE; + n->next = *p; + *p = n; + + item_count++; + if (const_count == IPA_MAX_AFF_JF_ITEMS + || item_count == 2 * IPA_MAX_AFF_JF_ITEMS) + break; + } - if (jfunc->type == IPA_JF_UNKNOWN - && type_like_member_ptr_p (TREE_TYPE (arg), &method_field, - &delta_field)) - determine_cst_member_ptr (call, arg, method_field, delta_field, jfunc); + /* Third stage just goes over the list and creates an appropriate vector of + ipa_agg_jf_item structures out of it, of sourse only if there are + any known constants to begin with. */ + + if (const_count) + { + jfunc->agg.items = VEC_alloc (ipa_agg_jf_item_t, gc, const_count); + if (POINTER_TYPE_P (TREE_TYPE (arg_base))) + jfunc->agg.type = TREE_TYPE (TREE_TYPE (arg_base)); + else + jfunc->agg.type = TREE_TYPE (arg_base); + gcc_checking_assert (AGGREGATE_TYPE_P (jfunc->agg.type)); + while (list) + { + if (list->constant) + { + struct ipa_agg_jf_item *item; + item = VEC_quick_push (ipa_agg_jf_item_t, + jfunc->agg.items, NULL); + item->offset = list->offset - arg_offset; + item->value = list->constant; + } + list = list->next; + } } } @@ -1177,23 +1349,71 @@ ipa_compute_jump_functions_for_edge (str struct ipa_node_params *info = IPA_NODE_REF (cs->caller); struct ipa_edge_args *args = IPA_EDGE_REF (cs); gimple call = cs->call_stmt; - int arg_num = gimple_call_num_args (call); + int n, arg_num = gimple_call_num_args (call); if (arg_num == 0 || args->jump_functions) return; VEC_safe_grow_cleared (ipa_jump_func_t, gc, args->jump_functions, arg_num); - /* We will deal with constants and SSA scalars first: */ - compute_scalar_jump_functions (info, parms_ainfo, args, call); + for (n = 0; n < arg_num; n++) + { + struct ipa_jump_func *jfunc = ipa_get_ith_jump_func (args, n); + tree arg = gimple_call_arg (call, n); - /* Let's check whether there are any potential member pointers and if so, - whether we can determine their functions as pass_through. */ - if (!compute_pass_through_member_ptrs (info, parms_ainfo, args, call)) - return; + if (is_gimple_ip_invariant (arg)) + ipa_set_jf_constant (jfunc, arg); + else if (!is_gimple_reg_type (TREE_TYPE (arg)) + && TREE_CODE (arg) == PARM_DECL) + { + int index = ipa_get_param_decl_index (info, arg); - /* Finally, let's check whether we actually pass a new constant member - pointer here... */ - compute_cst_member_ptr_arguments (args, call); + gcc_assert (index >=0); + /* Aggregate passed by value, check for pass-through, otherwise we + will attempt to fill in aggregate contents later in this + function. */ + if (is_parm_preserved_before_stmt (&parms_ainfo[index], call, arg)) + { + ipa_set_jf_simple_pass_through (jfunc, index, true); + continue; + } + } + else if (TREE_CODE (arg) == SSA_NAME) + { + if (SSA_NAME_IS_DEFAULT_DEF (arg)) + { + int index = ipa_get_param_decl_index (info, SSA_NAME_VAR (arg)); + if (index >= 0 + && !detect_type_change_ssa (arg, call, jfunc)) + { + bool agg_p; + agg_p = is_parm_ref_data_preserved_call (&parms_ainfo[index], + call, arg); + ipa_set_jf_simple_pass_through (jfunc, index, agg_p); + } + } + else + { + gimple stmt = SSA_NAME_DEF_STMT (arg); + if (is_gimple_assign (stmt)) + compute_complex_assign_jump_func (info, parms_ainfo, jfunc, + call, stmt, arg); + else if (gimple_code (stmt) == GIMPLE_PHI) + compute_complex_ancestor_jump_func (info, parms_ainfo, jfunc, + call, stmt); + } + } + else + compute_known_type_jump_func (arg, jfunc, call); + + if ((jfunc->type != IPA_JF_PASS_THROUGH + || !ipa_get_jf_pass_through_agg_preserved (jfunc)) + && (jfunc->type != IPA_JF_ANCESTOR + || !ipa_get_jf_ancestor_agg_preserved (jfunc)) + && (AGGREGATE_TYPE_P (TREE_TYPE (arg)) + || (POINTER_TYPE_P (TREE_TYPE (arg)) + && AGGREGATE_TYPE_P (TREE_TYPE (TREE_TYPE (arg)))))) + determine_known_aggregate_parts (call, arg, jfunc); + } } /* Compute jump functions for all edges - both direct and indirect - outgoing @@ -1220,16 +1440,22 @@ ipa_compute_jump_functions (struct cgrap ipa_compute_jump_functions_for_edge (parms_ainfo, cs); } -/* If RHS looks like a rhs of a statement loading pfn from a member - pointer formal parameter, return the parameter, otherwise return - NULL. If USE_DELTA, then we look for a use of the delta field - rather than the pfn. */ +/* If STMT looks like a statement loading a value from a member pointer formal + parameter, return that parameter and store the offset of the field to + *OFFSET_P, if it is non-NULL. Otherwise return NULL (but *OFFSET_P still + might be clobbered). If USE_DELTA, then we look for a use of the delta + field rather than the pfn. */ static tree -ipa_get_member_ptr_load_param (tree rhs, bool use_delta) +ipa_get_stmt_member_ptr_load_param (gimple stmt, bool use_delta, + HOST_WIDE_INT *offset_p) { - tree rec, ref_field, ref_offset, fld, fld_offset, ptr_field, delta_field; + tree rhs, rec, ref_field, ref_offset, fld, ptr_field, delta_field; + + if (!gimple_assign_single_p (stmt)) + return NULL_TREE; + rhs = gimple_assign_rhs1 (stmt); if (TREE_CODE (rhs) == COMPONENT_REF) { ref_field = TREE_OPERAND (rhs, 1); @@ -1246,43 +1472,24 @@ ipa_get_member_ptr_load_param (tree rhs, if (TREE_CODE (rec) != PARM_DECL || !type_like_member_ptr_p (TREE_TYPE (rec), &ptr_field, &delta_field)) return NULL_TREE; - ref_offset = TREE_OPERAND (rhs, 1); + if (use_delta) + fld = delta_field; + else + fld = ptr_field; + if (offset_p) + *offset_p = int_bit_position (fld); + if (ref_field) { if (integer_nonzerop (ref_offset)) return NULL_TREE; - - if (use_delta) - fld = delta_field; - else - fld = ptr_field; - return ref_field == fld ? rec : NULL_TREE; } - - if (use_delta) - fld_offset = byte_position (delta_field); else - fld_offset = byte_position (ptr_field); - - return tree_int_cst_equal (ref_offset, fld_offset) ? rec : NULL_TREE; -} - -/* If STMT looks like a statement loading a value from a member pointer formal - parameter, this function returns that parameter. */ - -static tree -ipa_get_stmt_member_ptr_load_param (gimple stmt, bool use_delta) -{ - tree rhs; - - if (!gimple_assign_single_p (stmt)) - return NULL_TREE; - - rhs = gimple_assign_rhs1 (stmt); - return ipa_get_member_ptr_load_param (rhs, use_delta); + return tree_int_cst_equal (byte_position (fld), ref_offset) ? rec + : NULL_TREE; } /* Returns true iff T is an SSA_NAME defined by a statement. */ @@ -1308,8 +1515,9 @@ ipa_note_param_call (struct cgraph_node cs = cgraph_edge (node, stmt); cs->indirect_info->param_index = param_index; - cs->indirect_info->anc_offset = 0; + cs->indirect_info->offset = 0; cs->indirect_info->polymorphic = 0; + cs->indirect_info->agg_contents = 0; return cs; } @@ -1368,7 +1576,9 @@ ipa_note_param_call (struct cgraph_node return (S.*f)(4); } -*/ + + Moreover, the function also looks for called pointers loaded from aggregates + passed by value or reference. */ static void ipa_analyze_indirect_call_uses (struct cgraph_node *node, @@ -1383,6 +1593,7 @@ ipa_analyze_indirect_call_uses (struct c gimple branch; int index; basic_block bb, virt_bb, join; + HOST_WIDE_INT offset; if (SSA_NAME_IS_DEFAULT_DEF (target)) { @@ -1393,20 +1604,25 @@ ipa_analyze_indirect_call_uses (struct c return; } + def = SSA_NAME_DEF_STMT (target); + if (gimple_assign_single_p (def) + && ipa_load_from_parm_agg_1 (info, parms_ainfo, def, + gimple_assign_rhs1 (def), &index, &offset)) + { + struct cgraph_edge *cs = ipa_note_param_call (node, index, call); + cs->indirect_info->offset = offset; + cs->indirect_info->agg_contents = 1; + return; + } + /* Now we need to try to match the complex pattern of calling a member pointer. */ - - if (!POINTER_TYPE_P (TREE_TYPE (target)) + if (gimple_code (def) != GIMPLE_PHI + || gimple_phi_num_args (def) != 2 + || !POINTER_TYPE_P (TREE_TYPE (target)) || TREE_CODE (TREE_TYPE (TREE_TYPE (target))) != METHOD_TYPE) return; - def = SSA_NAME_DEF_STMT (target); - if (gimple_code (def) != GIMPLE_PHI) - return; - - if (gimple_phi_num_args (def) != 2) - return; - /* First, we need to check whether one of these is a load from a member pointer that is a parameter to this function. */ n1 = PHI_ARG_DEF (def, 0); @@ -1417,15 +1633,15 @@ ipa_analyze_indirect_call_uses (struct c d2 = SSA_NAME_DEF_STMT (n2); join = gimple_bb (def); - if ((rec = ipa_get_stmt_member_ptr_load_param (d1, false))) + if ((rec = ipa_get_stmt_member_ptr_load_param (d1, false, &offset))) { - if (ipa_get_stmt_member_ptr_load_param (d2, false)) + if (ipa_get_stmt_member_ptr_load_param (d2, false, NULL)) return; bb = EDGE_PRED (join, 0)->src; virt_bb = gimple_bb (d2); } - else if ((rec = ipa_get_stmt_member_ptr_load_param (d2, false))) + else if ((rec = ipa_get_stmt_member_ptr_load_param (d2, false, &offset))) { bb = EDGE_PRED (join, 1)->src; virt_bb = gimple_bb (d1); @@ -1480,15 +1696,19 @@ ipa_analyze_indirect_call_uses (struct c rec2 = ipa_get_stmt_member_ptr_load_param (def, (TARGET_PTRMEMFUNC_VBIT_LOCATION - == ptrmemfunc_vbit_in_delta)); - + == ptrmemfunc_vbit_in_delta), + NULL); if (rec != rec2) return; index = ipa_get_param_decl_index (info, rec); - if (index >= 0 && !is_parm_modified_before_stmt (&parms_ainfo[index], + if (index >= 0 && is_parm_preserved_before_stmt (&parms_ainfo[index], call, rec)) - ipa_note_param_call (node, index, call); + { + struct cgraph_edge *cs = ipa_note_param_call (node, index, call); + cs->indirect_info->offset = offset; + cs->indirect_info->agg_contents = 1; + } return; } @@ -1543,7 +1763,7 @@ ipa_analyze_virtual_call_uses (struct cg cs = ipa_note_param_call (node, index, call); ii = cs->indirect_info; - ii->anc_offset = anc_offset; + ii->offset = anc_offset; ii->otr_token = tree_low_cst (OBJ_TYPE_REF_TOKEN (target), 1); ii->otr_type = TREE_TYPE (TREE_TYPE (OBJ_TYPE_REF_OBJECT (target))); ii->polymorphic = 1; @@ -1685,8 +1905,12 @@ ipa_analyze_node (struct cgraph_node *no ipa_compute_jump_functions (node, parms_ainfo); for (i = 0; i < param_count; i++) - if (parms_ainfo[i].visited_statements) - BITMAP_FREE (parms_ainfo[i].visited_statements); + { + if (parms_ainfo[i].parm_visited_statements) + BITMAP_FREE (parms_ainfo[i].parm_visited_statements); + if (parms_ainfo[i].ref_visited_statements) + BITMAP_FREE (parms_ainfo[i].ref_visited_statements); + } current_function_decl = NULL; pop_cfun (); @@ -1713,6 +1937,112 @@ combine_known_type_and_ancestor_jfs (str combined_type); } +/* Return true if we can still rely on our non-modification analysis + (is_parm_preserved_before_stmt and especially is_parm_ref_data_preserved) if + information about aggregate contents of type AGG can be propagated to or + through a parameter of type TO. If OFFSET is zero, we look at the types as + they are first. Then, if FROM is a record or a pointer to a record, we look + for a TO-typed field at offset. */ + +static bool +ipa_agg_types_propagable_p (tree from, tree to, HOST_WIDE_INT offset) +{ + gcc_checking_assert (TYPE_P (from) && TYPE_P (to)); + if (POINTER_TYPE_P (from)) + from = TREE_TYPE (from); + if (POINTER_TYPE_P (to)) + to = TREE_TYPE (to); + + if (offset == 0 && useless_type_conversion_p (to, from)) + return true; + + /* If the above failed, we resort to finding a field within a structure, + provided we are looking at one. */ + if (TREE_CODE (from) != RECORD_TYPE) + return false; + + while (true) + { + tree fld; + + for (fld = TYPE_FIELDS (from); fld; fld = DECL_CHAIN (fld)) + { + HOST_WIDE_INT pos, size; + tree tr_size; + + if (TREE_CODE (fld) != FIELD_DECL) + continue; + + pos = int_bit_position (fld); + tr_size = DECL_SIZE (fld); + if (!tr_size || !host_integerp (tr_size, 1)) + continue; + size = tree_low_cst (tr_size, 1); + if (pos > offset + || (pos + size) <= offset) + continue; + + from = TREE_TYPE (fld); + if (offset == pos + && TYPE_MAIN_VARIANT (from) == TYPE_MAIN_VARIANT (to)) + return true; + + if (TREE_CODE (from) == RECORD_TYPE) + break; + else + return false; + } + if (!fld) + return false; + } +} + +/* Return type of Ith parameter of NODE. */ + +static tree +node_parm_type (struct cgraph_node *node, int i) +{ + return TREE_TYPE (ipa_get_param (IPA_NODE_REF (node), i)); +} + +/* CS is an edge that is being currently inlined. Return false if parameter of + the callee with index DST_IDX, which recieves its value via JF, fails to + comply with aggregate type propagability criteria + (ipa_agg_types_propagable_p). */ + +bool +ipa_jf_propagates_agg_p (struct cgraph_edge *cs, + struct ipa_jump_func *jf, int dst_idx) +{ + struct cgraph_node *new_root; + + new_root = cs->caller->global.inlined_to ? cs->caller->global.inlined_to + : cs->caller; + + if (jf->type == IPA_JF_PASS_THROUGH) + { + tree t1, t2; + if (!ipa_get_jf_pass_through_agg_preserved (jf)) + return false; + t1 = node_parm_type (new_root, ipa_get_jf_pass_through_formal_id (jf)); + t2 = node_parm_type (cs->callee, dst_idx); + return ipa_agg_types_propagable_p (t1, t2, 0); + } + else if (jf->type == IPA_JF_ANCESTOR) + { + tree t1, t2; + if (!ipa_get_jf_ancestor_agg_preserved (jf)) + return false; + t1 = node_parm_type (new_root, ipa_get_jf_ancestor_formal_id (jf)); + t2 = node_parm_type (cs->callee, dst_idx); + return ipa_agg_types_propagable_p (t1, t2, + ipa_get_jf_ancestor_offset (jf)); + } + + return true; +} + + /* Update the jump functions associated with call graph edge E when the call graph edge CS is being inlined, assuming that E->caller is already (possibly indirectly) inlined into CS->callee and that E has not been inlined. */ @@ -1733,26 +2063,55 @@ update_jump_functions_after_inlining (st if (dst->type == IPA_JF_ANCESTOR) { struct ipa_jump_func *src; + int dst_fid = dst->value.ancestor.formal_id; /* Variable number of arguments can cause havoc if we try to access one that does not exist in the inlined edge. So make sure we don't. */ - if (dst->value.ancestor.formal_id >= ipa_get_cs_argument_count (top)) + if (dst_fid >= ipa_get_cs_argument_count (top)) { dst->type = IPA_JF_UNKNOWN; continue; } - src = ipa_get_ith_jump_func (top, dst->value.ancestor.formal_id); + src = ipa_get_ith_jump_func (top, dst_fid); + + if (dst->value.ancestor.agg_preserved && src->agg.items) + { + /* Currently we do not produce clobber aggregate jump functions, + replace with merging when we do. */ + gcc_assert (!dst->agg.items); + + if (ipa_agg_types_propagable_p (src->agg.type, + dst->value.ancestor.type, + dst->value.ancestor.offset)) + { + struct ipa_agg_jf_item *item; + int j; + + dst->agg.items = VEC_copy (ipa_agg_jf_item_t, gc, + src->agg.items); + FOR_EACH_VEC_ELT (ipa_agg_jf_item_t, dst->agg.items, j, item) + item->offset -= dst->value.ancestor.offset; + dst->agg.type = dst->value.ancestor.type; + } + } + if (src->type == IPA_JF_KNOWN_TYPE) combine_known_type_and_ancestor_jfs (src, dst); else if (src->type == IPA_JF_PASS_THROUGH && src->value.pass_through.operation == NOP_EXPR) - dst->value.ancestor.formal_id = src->value.pass_through.formal_id; + { + dst->value.ancestor.formal_id = src->value.pass_through.formal_id; + dst->value.ancestor.agg_preserved &= + ipa_jf_propagates_agg_p (cs, src, dst_fid); + } else if (src->type == IPA_JF_ANCESTOR) { dst->value.ancestor.formal_id = src->value.ancestor.formal_id; dst->value.ancestor.offset += src->value.ancestor.offset; + dst->value.ancestor.agg_preserved &= + ipa_jf_propagates_agg_p (cs, src, dst_fid); } else dst->type = IPA_JF_UNKNOWN; @@ -1766,9 +2125,34 @@ update_jump_functions_after_inlining (st && (dst->value.pass_through.formal_id < ipa_get_cs_argument_count (top))) { - src = ipa_get_ith_jump_func (top, - dst->value.pass_through.formal_id); - *dst = *src; + bool agg_p; + int dst_fid = dst->value.pass_through.formal_id; + src = ipa_get_ith_jump_func (top, dst_fid); + agg_p = dst->value.pass_through.agg_preserved; + /* Currently we do not produce clobber aggregate jump functions, + replace with merging when we do. */ + gcc_assert (!agg_p || !dst->agg.items); + + dst->type = src->type; + dst->value = src->value; + + if (agg_p && src->agg.items) + { + tree t = node_parm_type (cs->callee, dst_fid); + + if (ipa_agg_types_propagable_p (src->agg.type, t, 0)) + { + dst->agg.items = VEC_copy (ipa_agg_jf_item_t, gc, + src->agg.items); + dst->agg.type = src->agg.type; + } + } + if (src->type == IPA_JF_PASS_THROUGH) + dst->value.pass_through.agg_preserved = agg_p + && ipa_jf_propagates_agg_p (cs, src, dst_fid); + else if (src->type == IPA_JF_ANCESTOR) + dst->value.ancestor.agg_preserved = agg_p + && ipa_jf_propagates_agg_p (cs, src, dst_fid); } else dst->type = IPA_JF_UNKNOWN; @@ -1815,24 +2199,71 @@ ipa_make_edge_direct_to_target (struct c return ie; } +/* Provided that values from AGG can be propagated to parameter of NODE with + index PARAM_INDEX so that is_parm_ref_data_preserved results are reliable + and that there is a known constant value in AGG the given OFFSET, return + the known constant value. */ + +tree +ipa_find_agg_cst_for_param (struct ipa_agg_jump_function *agg, + HOST_WIDE_INT offset, + struct cgraph_node *node, + int param_index) +{ + struct ipa_node_params *info = IPA_NODE_REF (node); + struct ipa_agg_jf_item *item; + tree t = TREE_TYPE (ipa_get_param (info, param_index)); + int i; + + if (!agg->items + || !ipa_agg_types_propagable_p (agg->type, t, 0)) + return NULL; + + FOR_EACH_VEC_ELT (ipa_agg_jf_item_t, agg->items, i, item) + { + if (item->offset == offset) + { + /* Currently we do not have clobber values, return NULL for them once + we do. */ + gcc_checking_assert (is_gimple_ip_invariant (item->value)); + return item->value; + } + else if (item->offset > offset) + return NULL; + } + return NULL; +} + /* Try to find a destination for indirect edge IE that corresponds to a simple call or a call of a member function pointer and where the destination is a - pointer formal parameter described by jump function JFUNC. If it can be - determined, return the newly direct edge, otherwise return NULL. */ + pointer formal parameter described by jump function JFUNC. CS is the edge + that is currently being inlined. If it can be determined, return the newly + direct edge, otherwise return NULL. */ static struct cgraph_edge * -try_make_edge_direct_simple_call (struct cgraph_edge *ie, - struct ipa_jump_func *jfunc) +try_make_edge_direct_simple_call (struct cgraph_edge *cs, + struct cgraph_edge *ie, + struct ipa_jump_func *jfunc, int param_index) { tree target; - if (jfunc->type == IPA_JF_CONST) - target = ipa_get_jf_constant (jfunc); - else if (jfunc->type == IPA_JF_CONST_MEMBER_PTR) - target = ipa_get_jf_member_ptr_pfn (jfunc); - else - return NULL; + if (ie->indirect_info->agg_contents) + { + if (!ipa_jf_propagates_agg_p (cs, jfunc, param_index)) + return NULL; + target = ipa_find_agg_cst_for_param (&jfunc->agg, + ie->indirect_info->offset, + cs->callee, param_index); + if (!target) + return NULL; + } + else + { + if (jfunc->type != IPA_JF_CONST) + return NULL; + target = ipa_get_jf_constant (jfunc); + } return ipa_make_edge_direct_to_target (ie, target); } @@ -1853,7 +2284,7 @@ try_make_edge_direct_virtual_call (struc binfo = TYPE_BINFO (ipa_get_jf_known_type_base_type (jfunc)); gcc_checking_assert (binfo); binfo = get_binfo_at_offset (binfo, ipa_get_jf_known_type_offset (jfunc) - + ie->indirect_info->anc_offset, + + ie->indirect_info->offset, ie->indirect_info->otr_type); if (binfo) target = gimple_get_virt_method_for_binfo (ie->indirect_info->otr_token, @@ -1889,6 +2320,7 @@ update_indirect_edges_after_inlining (st { struct cgraph_indirect_call_info *ici = ie->indirect_info; struct ipa_jump_func *jfunc; + int param_index; next_ie = ie->next_callee; @@ -1902,14 +2334,27 @@ update_indirect_edges_after_inlining (st continue; } - jfunc = ipa_get_ith_jump_func (top, ici->param_index); + param_index = ici->param_index; + jfunc = ipa_get_ith_jump_func (top, param_index); if (jfunc->type == IPA_JF_PASS_THROUGH && ipa_get_jf_pass_through_operation (jfunc) == NOP_EXPR) - ici->param_index = ipa_get_jf_pass_through_formal_id (jfunc); + { + if (ici->agg_contents + && !ipa_jf_propagates_agg_p (cs, jfunc, param_index)) + ici->param_index = -1; + else + ici->param_index = ipa_get_jf_pass_through_formal_id (jfunc); + } else if (jfunc->type == IPA_JF_ANCESTOR) { - ici->param_index = ipa_get_jf_ancestor_formal_id (jfunc); - ici->anc_offset += ipa_get_jf_ancestor_offset (jfunc); + if (ici->agg_contents + && !ipa_jf_propagates_agg_p (cs, jfunc, param_index)) + ici->param_index = -1; + else + { + ici->param_index = ipa_get_jf_ancestor_formal_id (jfunc); + ici->offset += ipa_get_jf_ancestor_offset (jfunc); + } } else /* Either we can find a destination for this edge now or never. */ @@ -1921,7 +2366,8 @@ update_indirect_edges_after_inlining (st if (ici->polymorphic) new_direct_edge = try_make_edge_direct_virtual_call (ie, jfunc); else - new_direct_edge = try_make_edge_direct_simple_call (ie, jfunc); + new_direct_edge = try_make_edge_direct_simple_call (cs, ie, jfunc, + param_index); if (new_direct_edge) { @@ -2077,13 +2523,14 @@ ipa_node_removal_hook (struct cgraph_nod ipa_free_node_params_substructures (IPA_NODE_REF (node)); } -/* Hook that is called by cgraph.c when a node is duplicated. */ +/* Hook that is called by cgraph.c when an edge is duplicated. */ static void ipa_edge_duplication_hook (struct cgraph_edge *src, struct cgraph_edge *dst, __attribute__((unused)) void *data) { struct ipa_edge_args *old_args, *new_args; + unsigned int i; ipa_check_create_edge_args (); @@ -2092,6 +2539,12 @@ ipa_edge_duplication_hook (struct cgraph new_args->jump_functions = VEC_copy (ipa_jump_func_t, gc, old_args->jump_functions); + + for (i = 0; i < VEC_length (ipa_jump_func_t, old_args->jump_functions); i++) + VEC_index (ipa_jump_func_t, new_args->jump_functions, i)->agg.items + = VEC_copy (ipa_agg_jf_item_t, gc, + VEC_index (ipa_jump_func_t, + old_args->jump_functions, i)->agg.items); } /* Hook that is called by cgraph.c when a node is duplicated. */ @@ -2790,8 +3243,12 @@ static void ipa_write_jump_function (struct output_block *ob, struct ipa_jump_func *jump_func) { - streamer_write_uhwi (ob, jump_func->type); + struct ipa_agg_jf_item *item; + struct bitpack_d bp; + unsigned count; + int i; + streamer_write_uhwi (ob, jump_func->type); switch (jump_func->type) { case IPA_JF_UNKNOWN: @@ -2808,16 +3265,28 @@ ipa_write_jump_function (struct output_b stream_write_tree (ob, jump_func->value.pass_through.operand, true); streamer_write_uhwi (ob, jump_func->value.pass_through.formal_id); streamer_write_uhwi (ob, jump_func->value.pass_through.operation); + bp = bitpack_create (ob->main_stream); + bp_pack_value (&bp, jump_func->value.pass_through.agg_preserved, 1); + streamer_write_bitpack (&bp); break; case IPA_JF_ANCESTOR: streamer_write_uhwi (ob, jump_func->value.ancestor.offset); stream_write_tree (ob, jump_func->value.ancestor.type, true); streamer_write_uhwi (ob, jump_func->value.ancestor.formal_id); + bp = bitpack_create (ob->main_stream); + bp_pack_value (&bp, jump_func->value.ancestor.agg_preserved, 1); + streamer_write_bitpack (&bp); break; - case IPA_JF_CONST_MEMBER_PTR: - stream_write_tree (ob, jump_func->value.member_cst.pfn, true); - stream_write_tree (ob, jump_func->value.member_cst.delta, false); - break; + } + + count = VEC_length (ipa_agg_jf_item_t, jump_func->agg.items); + streamer_write_uhwi (ob, count); + if (count) + stream_write_tree (ob, jump_func->agg.type, true); + FOR_EACH_VEC_ELT (ipa_agg_jf_item_t, jump_func->agg.items, i, item) + { + streamer_write_uhwi (ob, item->offset); + stream_write_tree (ob, item->value, true); } } @@ -2828,8 +3297,10 @@ ipa_read_jump_function (struct lto_input struct ipa_jump_func *jump_func, struct data_in *data_in) { - jump_func->type = (enum jump_func_type) streamer_read_uhwi (ib); + struct bitpack_d bp; + int i, count; + jump_func->type = (enum jump_func_type) streamer_read_uhwi (ib); switch (jump_func->type) { case IPA_JF_UNKNOWN: @@ -2848,16 +3319,29 @@ ipa_read_jump_function (struct lto_input jump_func->value.pass_through.formal_id = streamer_read_uhwi (ib); jump_func->value.pass_through.operation = (enum tree_code) streamer_read_uhwi (ib); + bp = streamer_read_bitpack (ib); + jump_func->value.pass_through.agg_preserved = bp_unpack_value (&bp, 1); break; case IPA_JF_ANCESTOR: jump_func->value.ancestor.offset = streamer_read_uhwi (ib); jump_func->value.ancestor.type = stream_read_tree (ib, data_in); jump_func->value.ancestor.formal_id = streamer_read_uhwi (ib); + bp = streamer_read_bitpack (ib); + jump_func->value.ancestor.agg_preserved = bp_unpack_value (&bp, 1); break; - case IPA_JF_CONST_MEMBER_PTR: - jump_func->value.member_cst.pfn = stream_read_tree (ib, data_in); - jump_func->value.member_cst.delta = stream_read_tree (ib, data_in); - break; + } + + count = streamer_read_uhwi (ib); + jump_func->agg.items = VEC_alloc (ipa_agg_jf_item_t, gc, count); + if (count) + jump_func->agg.type = stream_read_tree (ib, data_in); + for (i = 0; i < count; i++) + { + struct ipa_agg_jf_item *item = VEC_quick_push (ipa_agg_jf_item_t, + jump_func->agg.items, NULL); + + item->offset = streamer_read_uhwi (ib); + item->value = stream_read_tree (ib, data_in); } } @@ -2872,9 +3356,10 @@ ipa_write_indirect_edge_info (struct out struct bitpack_d bp; streamer_write_hwi (ob, ii->param_index); - streamer_write_hwi (ob, ii->anc_offset); + streamer_write_hwi (ob, ii->offset); bp = bitpack_create (ob->main_stream); bp_pack_value (&bp, ii->polymorphic, 1); + bp_pack_value (&bp, ii->agg_contents, 1); streamer_write_bitpack (&bp); if (ii->polymorphic) @@ -2896,9 +3381,10 @@ ipa_read_indirect_edge_info (struct lto_ struct bitpack_d bp; ii->param_index = (int) streamer_read_hwi (ib); - ii->anc_offset = (HOST_WIDE_INT) streamer_read_hwi (ib); + ii->offset = (HOST_WIDE_INT) streamer_read_hwi (ib); bp = streamer_read_bitpack (ib); ii->polymorphic = bp_unpack_value (&bp, 1); + ii->agg_contents = bp_unpack_value (&bp, 1); if (ii->polymorphic) { ii->otr_token = (HOST_WIDE_INT) streamer_read_hwi (ib); Index: src/gcc/cgraph.h =================================================================== --- src.orig/gcc/cgraph.h +++ src/gcc/cgraph.h @@ -338,9 +338,11 @@ typedef enum cgraph_inline_failed_enum { struct GTY(()) cgraph_indirect_call_info { - /* Offset accumulated from ancestor jump functions of inlined call graph - edges. */ - HOST_WIDE_INT anc_offset; + /* When polymorphic is set, this field contains offset where the object which + was actually used in the polymorphic resides within a larger structure. + If agg_contents is set, the field contains the offset within the aggregate + from which the address to call was loaded. */ + HOST_WIDE_INT offset; /* OBJ_TYPE_REF_TOKEN of a polymorphic call (if polymorphic is set). */ HOST_WIDE_INT otr_token; /* Type of the object from OBJ_TYPE_REF_OBJECT. */ @@ -353,6 +355,9 @@ struct GTY(()) cgraph_indirect_call_info /* Set when the call is a virtual call with the parameter being the associated object pointer rather than a simple direct call. */ unsigned polymorphic : 1; + /* Set when the call is a call of a pointer loaded from contents of an + aggregate at offset. */ + unsigned agg_contents : 1; }; struct GTY((chain_next ("%h.next_caller"), chain_prev ("%h.prev_caller"))) cgraph_edge { Index: src/gcc/ipa-cp.c =================================================================== --- src.orig/gcc/ipa-cp.c +++ src/gcc/ipa-cp.c @@ -1110,7 +1110,7 @@ ipa_get_indirect_edge_target (struct cgr } token = ie->indirect_info->otr_token; - anc_offset = ie->indirect_info->anc_offset; + anc_offset = ie->indirect_info->offset; otr_type = ie->indirect_info->otr_type; t = VEC_index (tree, known_vals, param_index); Index: src/gcc/testsuite/gcc.dg/ipa/iinline-4.c =================================================================== --- /dev/null +++ src/gcc/testsuite/gcc.dg/ipa/iinline-4.c @@ -0,0 +1,187 @@ +/* Verify that simple indirect calls are inlined even without early + inlining.. */ +/* { dg-do compile } */ +/* { dg-options "-O3 -fdump-ipa-inline -fno-early-inlining" } */ + +struct S +{ + int i; + void (*f)(struct S *); + unsigned u; +}; + +struct U +{ + struct U *next; + struct S s; + short a[8]; +}; + +extern void non_existent(struct S *p, int); + +/* ----- 1 ----- */ + +static void hooray1 (struct S *p) +{ + non_existent (p, 1); +} + +static void hiphip1 (struct S *p) +{ + p->f (p); +} + +int test1 (void) +{ + struct S s; + s.i = 1234; + s.f = hooray1; + s.u = 1001; + hiphip1 (&s); + return 0; +} + +/* ----- 2 ----- */ + +struct S *gp; + +static void hooray2 (struct S *p) +{ + non_existent (p, 2); +} + +static void hip2 (struct S *p) +{ + p->f (p); +} + +static void hiphip2 (struct S *p) +{ + hip2 (p); +} + +int test2 (void) +{ + struct S *p = gp; + p->i = 2341; + p->f = hooray2; + p->u = 1002; + hiphip2 (p); + return 0; +} + +/* ----- 3 ----- */ + +static void hooray3 (struct S *p) +{ + non_existent (p, 2); +} + +static void hip3 (struct S *p) +{ + p->f (p); +} + +static __attribute__ ((flatten)) void hiphip3 (struct S *p) +{ + hip3 (p); +} + +int test3 (void) +{ + struct S *p = gp; + p->i = 2341; + p->f = hooray3; + p->u = 1003; + hiphip3 (p); + return 0; +} + +/* ----- 4 ----- */ + +static void hooray4 (struct S *p) +{ + non_existent (p, 3); +} + +static void hiphip4 (struct S s) +{ + s.f (&s); +} + +int test4(void) +{ + struct S s; + s.i = 3412; + s.f = hooray4; + s.u = 1004; + hiphip4 (s); + return 0; +} + +/* ----- 5 ----- */ + +struct U *gu; + +static void hooray5 (struct S *p) +{ + non_existent (p, 5); +} + +static void hip5 (struct S *p) +{ + p->f (p); +} + +static void hiphip5 (struct U *u) +{ + hip5 (&u->s); +} + +int test5 (void) +{ + struct U *u = gu; + u->next = u; + u->s.i = 9876; + u->s.f = hooray5; + u->s.u = 1005; + hiphip5 (u); + return 0; +} + +/* ----- 6 ----- */ + +static void hooray6 (struct S *p) +{ + non_existent (p, 6); +} + +static void hip6 (struct S *p) +{ + p->f (p); +} + +static __attribute__ ((flatten)) void hiphip6 (struct U *u) +{ + hip6 (&u->s); +} + +int test6 (void) +{ + struct U *u = gu; + u->next = u; + u->s.i = 9876; + u->s.f = hooray6; + u->s.u = 1006; + hiphip6 (u); + return 0; +} + + +/* { dg-final { scan-ipa-dump "hooray1\[^\\n\]*inline copy in test1" "inline" } } */ +/* { dg-final { scan-ipa-dump "hooray2\[^\\n\]*inline copy in test2" "inline" } } */ +/* { dg-final { scan-ipa-dump "hooray3\[^\\n\]*inline copy in test3" "inline" } } */ +/* { dg-final { scan-ipa-dump "hooray4\[^\\n\]*inline copy in test4" "inline" } } */ +/* { dg-final { scan-ipa-dump "hooray5\[^\\n\]*inline copy in test5" "inline" } } */ +/* { dg-final { scan-ipa-dump "hooray6\[^\\n\]*inline copy in test6" "inline" } } */ +/* { dg-final { cleanup-ipa-dump "inline" } } */ Index: src/gcc/testsuite/gcc.dg/ipa/iinline-5.c =================================================================== --- /dev/null +++ src/gcc/testsuite/gcc.dg/ipa/iinline-5.c @@ -0,0 +1,124 @@ +/* Verify that simple indirect calls are inlined even without early + inlining.. */ +/* { dg-do run } */ +/* { dg-options "-O3 -fdump-ipa-inline -fno-early-inlining" } */ + +extern void abort (void); + +struct S +{ + int i; + void (*f)(struct S *); + int j,k,l; +}; + +struct Z +{ + unsigned u; + void (*f)(struct Z *, int); + struct Z *next; +}; + +static struct Z *gz; +static struct S *gs; +static int gr = 111; +char gc[1024]; + +static __attribute__ ((noinline, noclone)) struct S * +get_s (void) +{ + return (struct S *) &gc; +} + + +static void wrong_target_1 (struct S *s) +{ + abort (); +} + +static void wrong_target_2 (struct S *s) +{ + abort (); +} + +static void wrong_target_3 (struct S *s) +{ + abort (); +} + +static void good_target (struct Z *z, int i) +{ + gr = 0; +} + +static void good_target_3 (struct S *s) +{ + gr = 0; +} + +static void g1 (struct S *s) +{ + struct Z *z = (struct Z*) s; + z->f (z, 8); +} + +static void f1 (struct S *s) +{ + gz->f = good_target; + g1 (s); +} + +static void g2 (struct Z *z) +{ + z->f (z, 8); +} + +static void f2 (struct S *s) +{ + gz->f = good_target; + g2 ((struct Z*) s); +} + +static void g3 (struct S *s) +{ + s->f (s); +} + +static void h3 (struct Z *z) +{ + gs->f = good_target_3; + g3 ((struct S *) z); +} + +static void f3 (struct S *s) +{ + h3 ((struct Z*) s); +} + +int main (int argc, char **argv) +{ + struct S *s = get_s(); + s->i = 5678; + s->f = wrong_target_1; + s->j = 1234; + gz = (struct Z *) s; + f1 (s); + + s = get_s(); + gz = (struct Z *) s; + s->i = 9999; + s->f = wrong_target_1; + f2 (s); + + s = get_s(); + gs = s; + s->i = 9999; + s->f = wrong_target_3; + f3 (s); + + return gr; +} + + +/* { dg-final { scan-ipa-dump-not "wrong_target\[^\\n\]*inline copy in" "inline" } } */ +/* { dg-final { cleanup-ipa-dump "inline" } } */ Index: src/gcc/testsuite/gcc.dg/ipa/iinline-6.c =================================================================== --- /dev/null +++ src/gcc/testsuite/gcc.dg/ipa/iinline-6.c @@ -0,0 +1,72 @@ +/* Verify that simple indirect calls are inlined even without early + inlining.. */ +/* { dg-do run } */ +/* { dg-options "-O3 -fdump-ipa-inline -fno-early-inlining" } */ + +extern void abort (void); + +struct S +{ + int i; + void (*f)(struct S *); + int j,k,l; +}; + +struct Z +{ + unsigned u; + void (*f)(struct Z *, int); + struct Z *next; +}; + +static struct S *gs; +static int gr = 111; +char gc[1024]; + +static __attribute__ ((noinline, noclone)) struct S * +get_s (void) +{ + return (struct S *) &gc; +} + +static void wrong_target (struct S *s) +{ + abort (); +} + +static void good_target (struct S *s) +{ + gr = 0; +} + +static void g1 (struct S *s) +{ + s->f (s); +} + +static void f2 (struct Z *z) +{ + gs->f = good_target; + g1 ((struct S *) z); +} + +static inline __attribute__ ((flatten)) void f1 (struct S *s) +{ + f2 ((struct Z *) s); +} + +int main (int argc, char **argv) +{ + struct S *s = get_s(); + s->i = 5678; + s->f = wrong_target; + s->j = 1234; + gs = s; + f1 (s); + + return gr; +} + + +/* { dg-final { scan-ipa-dump-not "wrong_target\[^\\n\]*inline copy in" "inline" } } */ +/* { dg-final { cleanup-ipa-dump "inline" } } */ Index: src/gcc/testsuite/gcc.dg/ipa/iinline-7.c =================================================================== --- /dev/null +++ src/gcc/testsuite/gcc.dg/ipa/iinline-7.c @@ -0,0 +1,157 @@ +/* Verify that simple indirect calls are inlined even without early + inlining.. */ +/* { dg-do run } */ +/* { dg-options "-O3 -fdump-ipa-inline -fno-early-inlining" } */ + +extern void abort (void); + +struct S +{ + int i; + void (*f)(struct S *); + int j,k,l; +}; + +struct U +{ + struct U *next; + struct S s; + short a[8]; +}; + +struct Z +{ + unsigned u; + void (*f)(struct Z *, int); + struct Z *next; +}; + +static struct Z *gz; +static struct U *gu; +static int gr = 111; +char gc[1024]; + +static __attribute__ ((noinline, noclone)) struct U * +get_u (void) +{ + return (struct U *) &gc; +} + +static void wrong_target_1 (struct S *s) +{ + abort (); +} + +static void wrong_target_2 (struct S *s) +{ + abort (); +} + +static void wrong_target_3 (struct S *s) +{ + abort (); +} + +static void wrong_target_4 (struct S *s) +{ + abort (); +} + +static void good_target (struct Z *z, int i) +{ + gr = 0; +} + +static void good_target_4 (struct S *s) +{ + gr = 0; +} + +static void g1 (struct S *s) +{ + struct Z *z = (struct Z*) s; + z->f (z, 8); +} + +static void f1 (struct U *u) +{ + gz->f = good_target; + g1 (&u->s); +} + +static void g2 (struct Z *z) +{ + z->f (z, 8); +} + +static void f2 (struct U *u) +{ + gz->f = good_target; + g2 ((struct Z*) &u->s); +} + +static void h3 (struct Z *z) +{ + z->f (z, 8); +} + +static void g3 (struct S *s) +{ + h3 ((struct Z*) s); +} + +static void f3 (struct U *u) +{ + gz->f = good_target; + g3 (&u->s); +} + +static void h4 (struct S *s) +{ + s->f (s); +} + +static void g4 (struct U *u) +{ + h4 (&u->s); +} + +static inline __attribute__ ((flatten)) void f4 (struct Z *z) +{ + gu->s.f = good_target_4; + g4 ((struct U *) z); +} + +int main (int argc, char **argv) +{ + struct U *u = get_u (); + u->next = u; + u->s.i = 5678; + u->s.f = wrong_target_1; + u->s.j = 1234; + gz = (struct Z *) &u->s; + f1 (u); + + u = get_u(); + u->s.i = 9999; + u->s.f = wrong_target_2; + gz = (struct Z *) &u->s; + f2 (u); + + u = get_u(); + u->s.i = 9998; + u->s.f = wrong_target_3; + gz = (struct Z *) &u->s; + f3 (u); + + u = get_u(); + u->s.i = 9998; + u->s.f = wrong_target_4; + gu = u; + f4 ((struct Z *) u); + return gr; +} + + +/* { dg-final { scan-ipa-dump-not "wrong_target\[^\\n\]*inline copy in" "inline" } } */ +/* { dg-final { cleanup-ipa-dump "inline" } } */
Hi, this is second version of patch introducing aggregate jump functions. For various reasons, I have merged two patches and so this one also implements indirect inlining based on them. One of the reasons is that indirect inlining can lead to miscompilations (as opposed to inlining heuristics) and so it's easier to make testcases for that. The patch introduces two changes to jump function creation: 1) we gather information about known compile time IPA-invariants at various offsets of an aggregate passed to a callee either by reference (when the parameter is pointer) or by value (when it is an aggregate). This patch simply scans the current BB backwards from the call statement and examines what values are stored there. This simple approach (in function determine_known_aggregate_parts) is sufficient for Fortran array descripttors, I intend to make it more sophisticated in future. When we do this, we also record the type through which data was stored into the aggregate, which is either the type of a DECL or type stored in the offset of a MEM_REF. This is slightly more strict than necessary but I'd need that extra field there for jump functions (see below). 2) Being able to conservatively but usefully recognize that an aggregate (passed either by reference or a value) that we got from a caller and pass it to a callee has not changed. This is slightly complex in cases where aggregates are passed by reference. Because in gimple pointer types carry more-or-less no information about the data it points to, we'd normally have to feed AA with a type that aliases all in order to get conservatively correct results. That might however be too conservative. We circumvent this problem by feeding AA the pointed-to type but also verifying that the aggregate data was read with the same type (modulo TYPE_MAIN_VARIANT and sub-structures). This consists of: a) checking that the alias type of memory reads from the aggregates is the same as the declared pointed-to type of the parameter. For example, the load in function g1 in testcase iinline-5.c clearly does not pass this test. b) when aggregate values are propagated along a pass-through or ancestor jump functions, we check that the declared types in both caller and callee match and refuse to propagate if they don't. The parameter passing in f2->g2 in the same testcase does not pass this test. c) before we propagate values into their eventual use, we check that the type of the store matches the type of the use. This is strictly necessary only when the jump function with values is a result of update_jump_functions_after_inlining combining aggregate values with a pass-through function, and the pass-through component requires theat we do the type check (we miscompile main->f2->g2 without this check). So, for example, when we have functions A->B->C where A stores something into a structure which is then passed to B, which passes is to C where the data is read and called. We perform the following: - We remember what we stored to the structure in A. - We use alias analysis with the type of the parameter to check that the pointed-to structure is not modified in B. - We note the use in C and make sure that is is read with the same alias type as is the declared type. We use AA with that type to check that the structure is not modified before use in C. - If we first inline A->B and then B->C: In each step we move the aggregate contents info but in each step we also check that the type of stored information matches the formal parameter of callee (this is important because we used that type for AA in both B and C. - if we first inline B->C, we check that their parameters match and if so, adjust the information about loads from the structure to refer to parameters of B. Inlining A->B is then handled like above. In both cases, the used AA type is guaranteed to correspond to the load type (and store type too). The data structures can also represent such pass-through functions with known listed exceptions but that is not currently implemented and is left for later. I also understand that eventually the alias vdef walking should be limited somehow (also, results should be shared with inlining). The patch passes bootstrap and testing on x86_64-linux. I'll be grateful for any comments and suggestions. Martin 2012-06-28 Martin Jambor <mjambor@suse.cz> * cgraph.h (cgraph_indirect_call_info): Field anc_offse renamd to offset, updated all users. New field agg_contents. * ipa-prop.h (jump_func_type): Removed IPA_JF_CONST_MEMBER_PTR. (ipa_pass_through_data): New field agg_preserved. (ipa_ancestor_jf_data): Likewise. (ipa_member_ptr_cst): Removed. (ipa_agg_jf_item): New type. (ipa_agg_jump_function): Likewise. (ipa_jump_func): New field agg. Removed field member_cst. (ipa_get_jf_pass_through_agg_preserved): New function. (ipa_get_jf_ancestor_agg_preserved): Likewise. (ipa_get_jf_member_ptr_pfn): Removed. (ipa_find_agg_cst_for_param): Declare. (ipa_load_from_parm_agg): Likewise. (ipa_jf_propagates_agg_p): Likewise. * ipa-prop.c (param_analysis_info): Fields modified and visited_statements rename to parm_modified and parm_visited_statements respectively, added fields ref_modified and ref_visited_statements. (ipa_print_node_jump_functions_for_edge): Do not dumo const member functions. Dump agg_preserved flags and aggregate jump functions. (ipa_set_jf_simple_pass_through): Set also agg_preserved. (ipa_set_ancestor_jf): Likewise. (ipa_set_jf_arith_pass_through): Clear agg_preserved. (ipa_set_jf_member_ptr_cst): Removed. (is_parm_modified_before_stmt): Logic reversed, renamed to is_parm_preserved_before_stmt. All callers updated. (is_parm_ref_data_preserved): New function. (is_parm_ref_data_preserved_call): Likewise. (ipa_load_from_parm_agg_1): Likewise. (ipa_load_from_parm_agg): Likewise. (compute_complex_assign_jump_func): Check if aggregate contents are preserved. (compute_complex_ancestor_jump_func): Likewise. (compute_scalar_jump_functions): Removed. (type_like_member_ptr_p): Also check field position are known and sane. (compute_pass_through_member_ptrs): Removed. (determine_cst_member_ptr): Likewise. (init_ao_ref_for_byref_agg_jf): New function. (ipa_known_agg_contents_list): New type. (determine_known_aggregate_parts): New function. (compute_cst_member_ptr_arguments): Removed. (ipa_compute_jump_functions_for_edge): Compute all kinds of jump functions (scalar, aggregate and member pointer). (ipa_get_member_ptr_load_param): Incorporate into ipa_get_stmt_member_ptr_load_param, also pass back an offset. (ipa_note_param_call): Clear agg_contents. (ipa_analyze_indirect_call_uses): Also look for simple pointers loaded from aggregates. In such cases, store offset of the called field. (ipa_analyze_node): Initialize new fields of param_analysis_info. (ipa_agg_types_propagable_p): New function. (node_parm_type): Likewise. (inlined_params_propagable_p): Likewise. (update_jump_functions_after_inlining): Handle aggregate contents. (ipa_find_agg_cst_for_param): New function. (try_make_edge_direct_simple_call): Handle called aggregate values. (update_indirect_edges_after_inlining): Make sure aggregate preserving jump functions comply with type compatibility requirements. (ipa_edge_duplication_hook): Copy also aggregate jump functions. (ipa_write_jump_function): Stream agg_preserved flags and aggregate jump functions. Do not stream member pointer constant jump functions. (ipa_read_jump_function): Likewise. (ipa_write_indirect_edge_info): Stream new cgraph_indirect_call_info fields. (ipa_read_indirect_edge_info): Likewise. * testsuite/gcc.dg/ipa/iinline-4.c: New test. * testsuite/gcc.dg/ipa/iinline-5.c: Likewise. * testsuite/gcc.dg/ipa/iinline-6.c: Likewise. * testsuite/gcc.dg/ipa/iinline-7.c: Likewise.