Add genmatch support for internal functions

Richard Sandiford <richard.sandiford@arm.com> writes:
> This patch makes genmatch match calls based on combined_fn rather
> than built_in_function and extends the matching to internal functions.
> It also uses fold_const_call to fold the calls to a constant, rather
> than going through fold_builtin_n.
>
> In order to slightly simplify the code and remove potential
> ambiguity, the patch enforces lower case for tree codes
> (foo->FOO_EXPR), caps for functions (no built_in_hypot->BUILT_IN_HYPOT)
> and requires an exact match for user-defined identifiers.  The first two
> were already met in practice but there were a couple of cases where
> operator lists were defined in one case and used in another.
>
> Tested on x86_64-linux-gnu, aarch64-linux-gnu and arm-linux-gnueabi.
> OK to install?

The updated patch below adds the SCALAR_FLOAT_TYPE_P check discussed here:

    https://gcc.gnu.org/ml/gcc-patches/2015-11/msg01949.html

Tested on x86_64-linux-gnu, aarch64-linux-gnu and arm-linux-gnueabi.
OK to install?

Thanks,
Richard


gcc/
	* match.pd: Use HYPOT and COS rather than hypot and cos.
	Use CASE_CFN_* macros.  Guard log/exp folds with
	SCALAR_FLOAT_TYPE_P.
	* genmatch.c (internal_fn): New enum.
	(fn_id::fn): Change to an unsigned int.
	(fn_id::fn_id): Accept internal_fn too.
	(add_builtin): Rename to...
	(add_function): ...this and turn into a template.
	(get_operator): Only try one variation if the original name fails.
	Only add _EXPR if the original name was all lower case.
	Try converting internal and built-in function names to their
	CFN equivalents.
	(expr::gen_transform): Use maybe_build_call_expr_loc for generic.
	(dt_simplify::gen_1): Likewise.
	(dt_node::gen_kids_1): Use gimple_call_combined_fn for gimple
	and get_call_combined_fn for generic.
	(dt_simplify::gen): Use combined_fn as the type of fn_ids.
	(decision_tree::gen): Likewise.
	(main): Use lower case in the strings for {VIEW_,}CONVERT[012].
	Use add_function rather than add_builtin.  Register internal
	functions too.
	* generic-match-head.c: Include case-cfn-macros.h.
	* gimple-fold.c (replace_stmt_with_simplification): Use
	gimple_call_combined_fn to test whether we can keep an
	existing call.
	* gimple-match.h (code_helper): Replace built_in_function
	with combined_fn.
	* gimple-match-head.c: Include fold-const-call.h, internal-fn.h
	and case-fn-macros.h.
	(gimple_resimplify1): Use fold_const_call.
	(gimple_resimplify2, gimple_resimplify3): Likewise.
	(build_call_internal, build_call): New functions.
	(maybe_push_res_to_seq): Use them.
	(gimple_simplify): Use fold_const_call.  Set *rcode to a combined_fn
	rather than a built-in function.
	* tree.h (build_call_expr_internal_loc): Declare.
	(maybe_build_call_expr_loc): Likewise.
	* tree.c (build_call_expr_internal_loc_array): New function.
	(maybe_build_call_expr_loc): Likewise.

On Tue, Nov 17, 2015 at 9:52 AM, Richard Sandiford
<richard.sandiford@arm.com> wrote:
> Richard Sandiford <richard.sandiford@arm.com> writes:
>> This patch makes genmatch match calls based on combined_fn rather
>> than built_in_function and extends the matching to internal functions.
>> It also uses fold_const_call to fold the calls to a constant, rather
>> than going through fold_builtin_n.
>>
>> In order to slightly simplify the code and remove potential
>> ambiguity, the patch enforces lower case for tree codes
>> (foo->FOO_EXPR), caps for functions (no built_in_hypot->BUILT_IN_HYPOT)
>> and requires an exact match for user-defined identifiers.  The first two
>> were already met in practice but there were a couple of cases where
>> operator lists were defined in one case and used in another.
>>
>> Tested on x86_64-linux-gnu, aarch64-linux-gnu and arm-linux-gnueabi.
>> OK to install?
>
> The updated patch below adds the SCALAR_FLOAT_TYPE_P check discussed here:
>
>     https://gcc.gnu.org/ml/gcc-patches/2015-11/msg01949.html
>
> Tested on x86_64-linux-gnu, aarch64-linux-gnu and arm-linux-gnueabi.
> OK to install?

Ok.

Thanks,
Richard.

> Thanks,
> Richard
>
>
> gcc/
>         * match.pd: Use HYPOT and COS rather than hypot and cos.
>         Use CASE_CFN_* macros.  Guard log/exp folds with
>         SCALAR_FLOAT_TYPE_P.
>         * genmatch.c (internal_fn): New enum.
>         (fn_id::fn): Change to an unsigned int.
>         (fn_id::fn_id): Accept internal_fn too.
>         (add_builtin): Rename to...
>         (add_function): ...this and turn into a template.
>         (get_operator): Only try one variation if the original name fails.
>         Only add _EXPR if the original name was all lower case.
>         Try converting internal and built-in function names to their
>         CFN equivalents.
>         (expr::gen_transform): Use maybe_build_call_expr_loc for generic.
>         (dt_simplify::gen_1): Likewise.
>         (dt_node::gen_kids_1): Use gimple_call_combined_fn for gimple
>         and get_call_combined_fn for generic.
>         (dt_simplify::gen): Use combined_fn as the type of fn_ids.
>         (decision_tree::gen): Likewise.
>         (main): Use lower case in the strings for {VIEW_,}CONVERT[012].
>         Use add_function rather than add_builtin.  Register internal
>         functions too.
>         * generic-match-head.c: Include case-cfn-macros.h.
>         * gimple-fold.c (replace_stmt_with_simplification): Use
>         gimple_call_combined_fn to test whether we can keep an
>         existing call.
>         * gimple-match.h (code_helper): Replace built_in_function
>         with combined_fn.
>         * gimple-match-head.c: Include fold-const-call.h, internal-fn.h
>         and case-fn-macros.h.
>         (gimple_resimplify1): Use fold_const_call.
>         (gimple_resimplify2, gimple_resimplify3): Likewise.
>         (build_call_internal, build_call): New functions.
>         (maybe_push_res_to_seq): Use them.
>         (gimple_simplify): Use fold_const_call.  Set *rcode to a combined_fn
>         rather than a built-in function.
>         * tree.h (build_call_expr_internal_loc): Declare.
>         (maybe_build_call_expr_loc): Likewise.
>         * tree.c (build_call_expr_internal_loc_array): New function.
>         (maybe_build_call_expr_loc): Likewise.
>
> diff --git a/gcc/generic-match-head.c b/gcc/generic-match-head.c
> index f2e08ed..f55f91e 100644
> --- a/gcc/generic-match-head.c
> +++ b/gcc/generic-match-head.c
> @@ -32,6 +32,7 @@ along with GCC; see the file COPYING3.  If not see
>  #include "tree-dfa.h"
>  #include "builtins.h"
>  #include "dumpfile.h"
> +#include "case-cfn-macros.h"
>
>
>  /* Routine to determine if the types T1 and T2 are effectively
> diff --git a/gcc/genmatch.c b/gcc/genmatch.c
> index 9d74ed7..daa66d9 100644
> --- a/gcc/genmatch.c
> +++ b/gcc/genmatch.c
> @@ -230,6 +230,12 @@ enum built_in_function {
>  END_BUILTINS
>  };
>
> +#define DEF_INTERNAL_FN(CODE, FLAGS, FNSPEC) IFN_##CODE,
> +enum internal_fn {
> +#include "internal-fn.def"
> +  IFN_LAST
> +};
> +
>  /* Return true if CODE represents a commutative tree code.  Otherwise
>     return false.  */
>  bool
> @@ -341,13 +347,15 @@ struct operator_id : public id_base
>    const char *tcc;
>  };
>
> -/* Identifier that maps to a builtin function code.  */
> +/* Identifier that maps to a builtin or internal function code.  */
>
>  struct fn_id : public id_base
>  {
>    fn_id (enum built_in_function fn_, const char *id_)
>        : id_base (id_base::FN, id_), fn (fn_) {}
> -  enum built_in_function fn;
> +  fn_id (enum internal_fn fn_, const char *id_)
> +      : id_base (id_base::FN, id_), fn (int (END_BUILTINS) + int (fn_)) {}
> +  unsigned int fn;
>  };
>
>  struct simplify;
> @@ -447,10 +455,12 @@ add_operator (enum tree_code code, const char *id,
>    *slot = op;
>  }
>
> -/* Add a builtin identifier to the hash.  */
> +/* Add a built-in or internal function identifier to the hash.  ID is
> +   the name of its CFN_* enumeration value.  */
>
> +template <typename T>
>  static void
> -add_builtin (enum built_in_function code, const char *id)
> +add_function (T code, const char *id)
>  {
>    fn_id *fn = new fn_id (code, id);
>    id_base **slot = operators->find_slot_with_hash (fn, fn->hashval, INSERT);
> @@ -485,30 +495,32 @@ get_operator (const char *id)
>        return op;
>      }
>
> -  /* Try all-uppercase.  */
> -  char *id2 = xstrdup (id);
> -  for (unsigned i = 0; i < strlen (id2); ++i)
> -    id2[i] = TOUPPER (id2[i]);
> -  new (&tem) id_base (id_base::CODE, id2);
> -  op = operators->find_with_hash (&tem, tem.hashval);
> -  if (op)
> +  char *id2;
> +  bool all_upper = true;
> +  bool all_lower = true;
> +  for (unsigned int i = 0; id[i]; ++i)
> +    if (ISUPPER (id[i]))
> +      all_lower = false;
> +    else if (ISLOWER (id[i]))
> +      all_upper = false;
> +  if (all_lower)
>      {
> -      free (id2);
> -      return op;
> +      /* Try in caps with _EXPR appended.  */
> +      id2 = ACONCAT ((id, "_EXPR", NULL));
> +      for (unsigned int i = 0; id2[i]; ++i)
> +       id2[i] = TOUPPER (id2[i]);
>      }
> +  else if (all_upper && strncmp (id, "IFN_", 4) == 0)
> +    /* Try CFN_ instead of IFN_.  */
> +    id2 = ACONCAT (("CFN_", id + 4, NULL));
> +  else if (all_upper && strncmp (id, "BUILT_IN_", 9) == 0)
> +    /* Try prepending CFN_.  */
> +    id2 = ACONCAT (("CFN_", id, NULL));
> +  else
> +    return NULL;
>
> -  /* Try _EXPR appended.  */
> -  id2 = (char *)xrealloc (id2, strlen (id2) + sizeof ("_EXPR") + 1);
> -  strcat (id2, "_EXPR");
>    new (&tem) id_base (id_base::CODE, id2);
> -  op = operators->find_with_hash (&tem, tem.hashval);
> -  if (op)
> -    {
> -      free (id2);
> -      return op;
> -    }
> -
> -  return 0;
> +  return operators->find_with_hash (&tem, tem.hashval);
>  }
>
>  typedef hash_map<nofree_string_hash, unsigned> cid_map_t;
> @@ -2207,17 +2219,18 @@ expr::gen_transform (FILE *f, int indent, const char *dest, bool gimple,
>        else
>         {
>           fprintf_indent (f, indent, "{\n");
> -         fprintf_indent (f, indent, "  tree decl = builtin_decl_implicit (%s);\n",
> -                         opr_name);
> -         fprintf_indent (f, indent, "  if (!decl) return NULL_TREE;\n");
> -         fprintf_indent (f, indent, "  res = build_call_expr_loc (loc, "
> -                         "decl, %d", ops.length());
> +         fprintf_indent (f, indent, "  res = maybe_build_call_expr_loc (loc, "
> +                         "%s, %s, %d", opr_name, type, ops.length());
>         }
>        for (unsigned i = 0; i < ops.length (); ++i)
>         fprintf (f, ", ops%d[%u]", depth, i);
>        fprintf (f, ");\n");
>        if (opr->kind != id_base::CODE)
> -       fprintf_indent (f, indent, "}\n");
> +       {
> +         fprintf_indent (f, indent, "  if (!res)\n");
> +         fprintf_indent (f, indent, "    return NULL_TREE;\n");
> +         fprintf_indent (f, indent, "}\n");
> +       }
>        if (*opr == CONVERT_EXPR)
>         {
>           indent -= 2;
> @@ -2665,20 +2678,14 @@ dt_node::gen_kids_1 (FILE *f, int indent, bool gimple,
>        if (fns_len)
>         {
>           fprintf_indent (f, indent,
> -                         "%sif (gimple_call_builtin_p (def_stmt, BUILT_IN_NORMAL))\n",
> +                         "%sif (gcall *def = dyn_cast <gcall *>"
> +                         " (def_stmt))\n",
>                           exprs_len ? "else " : "");
>           fprintf_indent (f, indent,
> -                         "  {\n");
> -         fprintf_indent (f, indent,
> -                         "    gcall *def = as_a <gcall *> (def_stmt);\n");
> -         fprintf_indent (f, indent,
> -                         "    tree fndecl = gimple_call_fndecl (def);\n");
> -         fprintf_indent (f, indent,
> -                         "    switch (DECL_FUNCTION_CODE (fndecl))\n");
> -         fprintf_indent (f, indent,
> -                         "      {\n");
> +                         "  switch (gimple_call_combined_fn (def))\n");
>
> -         indent += 6;
> +         indent += 4;
> +         fprintf_indent (f, indent, "{\n");
>           for (unsigned i = 0; i < fns_len; ++i)
>             {
>               expr *e = as_a <expr *>(fns[i]->op);
> @@ -2691,8 +2698,7 @@ dt_node::gen_kids_1 (FILE *f, int indent, bool gimple,
>
>           fprintf_indent (f, indent, "default:;\n");
>           fprintf_indent (f, indent, "}\n");
> -         indent -= 6;
> -         fprintf_indent (f, indent, "  }\n");
> +         indent -= 4;
>         }
>
>        indent -= 6;
> @@ -2719,17 +2725,11 @@ dt_node::gen_kids_1 (FILE *f, int indent, bool gimple,
>        fprintf_indent (f, indent,
>                       "case CALL_EXPR:\n");
>        fprintf_indent (f, indent,
> -                     "  {\n");
> -      fprintf_indent (f, indent,
> -                     "    tree fndecl = get_callee_fndecl (%s);\n",
> +                     "  switch (get_call_combined_fn (%s))\n",
>                       kid_opname);
>        fprintf_indent (f, indent,
> -                     "    if (fndecl && DECL_BUILT_IN_CLASS (fndecl) == BUILT_IN_NORMAL)\n");
> -      fprintf_indent (f, indent,
> -                     "      switch (DECL_FUNCTION_CODE (fndecl))\n");
> -      fprintf_indent (f, indent,
> -                     "        {\n");
> -      indent += 8;
> +                     "    {\n");
> +      indent += 4;
>
>        for (unsigned j = 0; j < generic_fns.length (); ++j)
>         {
> @@ -2742,12 +2742,11 @@ dt_node::gen_kids_1 (FILE *f, int indent, bool gimple,
>           fprintf_indent (f, indent, "    break;\n");
>           fprintf_indent (f, indent, "  }\n");
>         }
> +      fprintf_indent (f, indent, "default:;\n");
>
> -      indent -= 8;
> -      fprintf_indent (f, indent, "          default:;\n");
> -      fprintf_indent (f, indent, "        }\n");
> -      fprintf_indent (f, indent, "    break;\n");
> -      fprintf_indent (f, indent, "  }\n");
> +      indent -= 4;
> +      fprintf_indent (f, indent, "    }\n");
> +      fprintf_indent (f, indent, "  break;\n");
>      }
>
>    /* Close switch (TREE_CODE ()).  */
> @@ -3108,24 +3107,18 @@ dt_simplify::gen_1 (FILE *f, int indent, bool gimple, operand *result)
>                                     *e->operation == CONVERT_EXPR
>                                     ? "NOP_EXPR" : e->operation->id);
>                   else
> -                   {
> -                     fprintf_indent (f, indent,
> -                                     "{\n");
> -                     fprintf_indent (f, indent,
> -                                     "  tree decl = builtin_decl_implicit (%s);\n",
> -                                     e->operation->id);
> -                     fprintf_indent (f, indent,
> -                                     "  if (!decl) return NULL_TREE;\n");
> -                     fprintf_indent (f, indent,
> -                                     "  res = build_call_expr_loc "
> -                                     "(loc, decl, %d",
> -                                     e->ops.length());
> -                   }
> +                   fprintf_indent (f, indent,
> +                                   "res = maybe_build_call_expr_loc (loc, "
> +                                   "%s, type, %d", e->operation->id,
> +                                   e->ops.length());
>                   for (unsigned j = 0; j < e->ops.length (); ++j)
>                     fprintf (f, ", res_op%d", j);
>                   fprintf (f, ");\n");
>                   if (!is_a <operator_id *> (opr))
> -                   fprintf_indent (f, indent, "}\n");
> +                   {
> +                     fprintf_indent (f, indent, "if (!res)\n");
> +                     fprintf_indent (f, indent, "  return NULL_TREE;\n");
> +                   }
>                 }
>             }
>         }
> @@ -3225,7 +3218,7 @@ dt_simplify::gen (FILE *f, int indent, bool gimple)
>                             s->for_subst_vec[i].first->id,
>                             s->for_subst_vec[i].second->id);
>           else if (is_a <fn_id *> (s->for_subst_vec[i].second))
> -           fprintf_indent (f, indent, "enum built_in_function %s = %s;\n",
> +           fprintf_indent (f, indent, "combined_fn %s = %s;\n",
>                             s->for_subst_vec[i].first->id,
>                             s->for_subst_vec[i].second->id);
>           else
> @@ -3380,7 +3373,7 @@ decision_tree::gen (FILE *f, bool gimple)
>             fprintf (f, ", enum tree_code ARG_UNUSED (%s)",
>                      s->s->s->for_subst_vec[i].first->id);
>           else if (is_a <fn_id *> (s->s->s->for_subst_vec[i].second))
> -           fprintf (f, ", enum built_in_function ARG_UNUSED (%s)",
> +           fprintf (f, ", combined_fn ARG_UNUSED (%s)",
>                      s->s->s->for_subst_vec[i].first->id);
>         }
>
> @@ -4603,12 +4596,12 @@ main (int argc, char **argv)
>    add_operator (SYM, # SYM, # TYPE, NARGS);
>  #define END_OF_BASE_TREE_CODES
>  #include "tree.def"
> -add_operator (CONVERT0, "CONVERT0", "tcc_unary", 1);
> -add_operator (CONVERT1, "CONVERT1", "tcc_unary", 1);
> -add_operator (CONVERT2, "CONVERT2", "tcc_unary", 1);
> -add_operator (VIEW_CONVERT0, "VIEW_CONVERT0", "tcc_unary", 1);
> -add_operator (VIEW_CONVERT1, "VIEW_CONVERT1", "tcc_unary", 1);
> -add_operator (VIEW_CONVERT2, "VIEW_CONVERT2", "tcc_unary", 1);
> +add_operator (CONVERT0, "convert0", "tcc_unary", 1);
> +add_operator (CONVERT1, "convert1", "tcc_unary", 1);
> +add_operator (CONVERT2, "convert2", "tcc_unary", 1);
> +add_operator (VIEW_CONVERT0, "view_convert0", "tcc_unary", 1);
> +add_operator (VIEW_CONVERT1, "view_convert1", "tcc_unary", 1);
> +add_operator (VIEW_CONVERT2, "view_convert2", "tcc_unary", 1);
>  #undef END_OF_BASE_TREE_CODES
>  #undef DEFTREECODE
>
> @@ -4616,9 +4609,13 @@ add_operator (VIEW_CONVERT2, "VIEW_CONVERT2", "tcc_unary", 1);
>       ???  Cannot use N (name) as that is targetm.emultls.get_address
>       for BUILT_IN_EMUTLS_GET_ADDRESS ... */
>  #define DEF_BUILTIN(ENUM, N, C, T, LT, B, F, NA, AT, IM, COND) \
> -  add_builtin (ENUM, # ENUM);
> +  add_function (ENUM, "CFN_" # ENUM);
>  #include "builtins.def"
>
> +#define DEF_INTERNAL_FN(CODE, NAME, FNSPEC) \
> +  add_function (IFN_##CODE, "CFN_" #CODE);
> +#include "internal-fn.def"
> +
>    /* Parse ahead!  */
>    parser p (r);
>
> diff --git a/gcc/gimple-fold.c b/gcc/gimple-fold.c
> index 436e29b..1ab20d1 100644
> --- a/gcc/gimple-fold.c
> +++ b/gcc/gimple-fold.c
> @@ -3369,7 +3369,7 @@ replace_stmt_with_simplification (gimple_stmt_iterator *gsi,
>         }
>      }
>    else if (rcode.is_fn_code ()
> -          && gimple_call_builtin_p (stmt, rcode))
> +          && gimple_call_combined_fn (stmt) == rcode)
>      {
>        unsigned i;
>        for (i = 0; i < gimple_call_num_args (stmt); ++i)
> diff --git a/gcc/gimple-match-head.c b/gcc/gimple-match-head.c
> index 030cc74..bdc1f98 100644
> --- a/gcc/gimple-match-head.c
> +++ b/gcc/gimple-match-head.c
> @@ -28,6 +28,7 @@ along with GCC; see the file COPYING3.  If not see
>  #include "ssa.h"
>  #include "cgraph.h"
>  #include "fold-const.h"
> +#include "fold-const-call.h"
>  #include "stor-layout.h"
>  #include "gimple-fold.h"
>  #include "calls.h"
> @@ -35,6 +36,8 @@ along with GCC; see the file COPYING3.  If not see
>  #include "builtins.h"
>  #include "gimple-match.h"
>  #include "tree-pass.h"
> +#include "internal-fn.h"
> +#include "case-cfn-macros.h"
>
>
>  /* Forward declarations of the private auto-generated matchers.
> @@ -81,19 +84,7 @@ gimple_resimplify1 (gimple_seq *seq,
>        if (res_code->is_tree_code ())
>         tem = const_unop (*res_code, type, res_ops[0]);
>        else
> -       {
> -         tree decl = builtin_decl_implicit (*res_code);
> -         if (decl)
> -           {
> -             tem = fold_builtin_n (UNKNOWN_LOCATION, decl, res_ops, 1, false);
> -             if (tem)
> -               {
> -                 /* fold_builtin_n wraps the result inside a NOP_EXPR.  */
> -                 STRIP_NOPS (tem);
> -                 tem = fold_convert (type, tem);
> -               }
> -           }
> -       }
> +       tem = fold_const_call (combined_fn (*res_code), type, res_ops[0]);
>        if (tem != NULL_TREE
>           && CONSTANT_CLASS_P (tem))
>         {
> @@ -137,19 +128,8 @@ gimple_resimplify2 (gimple_seq *seq,
>        if (res_code->is_tree_code ())
>         tem = const_binop (*res_code, type, res_ops[0], res_ops[1]);
>        else
> -       {
> -         tree decl = builtin_decl_implicit (*res_code);
> -         if (decl)
> -           {
> -             tem = fold_builtin_n (UNKNOWN_LOCATION, decl, res_ops, 2, false);
> -             if (tem)
> -               {
> -                 /* fold_builtin_n wraps the result inside a NOP_EXPR.  */
> -                 STRIP_NOPS (tem);
> -                 tem = fold_convert (type, tem);
> -               }
> -           }
> -       }
> +       tem = fold_const_call (combined_fn (*res_code), type,
> +                              res_ops[0], res_ops[1]);
>        if (tem != NULL_TREE
>           && CONSTANT_CLASS_P (tem))
>         {
> @@ -208,19 +188,8 @@ gimple_resimplify3 (gimple_seq *seq,
>         tem = fold_ternary/*_to_constant*/ (*res_code, type, res_ops[0],
>                                             res_ops[1], res_ops[2]);
>        else
> -       {
> -         tree decl = builtin_decl_implicit (*res_code);
> -         if (decl)
> -           {
> -             tem = fold_builtin_n (UNKNOWN_LOCATION, decl, res_ops, 3, false);
> -             if (tem)
> -               {
> -                 /* fold_builtin_n wraps the result inside a NOP_EXPR.  */
> -                 STRIP_NOPS (tem);
> -                 tem = fold_convert (type, tem);
> -               }
> -           }
> -       }
> +       tem = fold_const_call (combined_fn (*res_code), type,
> +                              res_ops[0], res_ops[1], res_ops[2]);
>        if (tem != NULL_TREE
>           && CONSTANT_CLASS_P (tem))
>         {
> @@ -282,6 +251,22 @@ maybe_build_generic_op (enum tree_code code, tree type,
>
>  tree (*mprts_hook) (code_helper, tree, tree *);
>
> +/* Try to build a call to FN with return type TYPE and the NARGS
> +   arguments given in OPS.  Return null if the target doesn't support
> +   the function.  */
> +
> +static gcall *
> +build_call_internal (internal_fn fn, tree type, unsigned int nargs, tree *ops)
> +{
> +  if (direct_internal_fn_p (fn))
> +    {
> +      tree_pair types = direct_internal_fn_types (fn, type, ops);
> +      if (!direct_internal_fn_supported_p (fn, types))
> +       return NULL;
> +    }
> +  return gimple_build_call_internal (fn, nargs, ops[0], ops[1], ops[2]);
> +}
> +
>  /* Push the exploded expression described by RCODE, TYPE and OPS
>     as a statement to SEQ if necessary and return a gimple value
>     denoting the value of the expression.  If RES is not NULL
> @@ -333,12 +318,7 @@ maybe_push_res_to_seq (code_helper rcode, tree type, tree *ops,
>      {
>        if (!seq)
>         return NULL_TREE;
> -      tree decl = builtin_decl_implicit (rcode);
> -      if (!decl)
> -       return NULL_TREE;
> -      /* We can't and should not emit calls to non-const functions.  */
> -      if (!(flags_from_decl_or_type (decl) & ECF_CONST))
> -       return NULL_TREE;
> +      combined_fn fn = rcode;
>        /* Play safe and do not allow abnormals to be mentioned in
>           newly created statements.  */
>        unsigned nargs;
> @@ -351,6 +331,28 @@ maybe_push_res_to_seq (code_helper rcode, tree type, tree *ops,
>             return NULL_TREE;
>         }
>        gcc_assert (nargs != 0);
> +      gcall *new_stmt = NULL;
> +      if (internal_fn_p (fn))
> +       {
> +         /* Generate the given function if we can.  */
> +         internal_fn ifn = as_internal_fn (fn);
> +         new_stmt = build_call_internal (ifn, type, nargs, ops);
> +         if (!new_stmt)
> +           return NULL_TREE;
> +       }
> +      else
> +       {
> +         /* Find the function we want to call.  */
> +         tree decl = builtin_decl_implicit (as_builtin_fn (fn));
> +         if (!decl)
> +           return NULL;
> +
> +         /* We can't and should not emit calls to non-const functions.  */
> +         if (!(flags_from_decl_or_type (decl) & ECF_CONST))
> +           return NULL;
> +
> +         new_stmt = gimple_build_call (decl, nargs, ops[0], ops[1], ops[2]);
> +       }
>        if (!res)
>         {
>           if (gimple_in_ssa_p (cfun))
> @@ -358,7 +360,6 @@ maybe_push_res_to_seq (code_helper rcode, tree type, tree *ops,
>           else
>             res = create_tmp_reg (type);
>         }
> -      gimple *new_stmt = gimple_build_call (decl, nargs, ops[0], ops[1], ops[2]);
>        gimple_call_set_lhs (new_stmt, res);
>        gimple_seq_add_stmt_without_update (seq, new_stmt);
>        return res;
> @@ -471,25 +472,15 @@ gimple_simplify (enum built_in_function fn, tree type,
>  {
>    if (constant_for_folding (arg0))
>      {
> -      tree decl = builtin_decl_implicit (fn);
> -      if (decl)
> -       {
> -         tree res = fold_builtin_n (UNKNOWN_LOCATION, decl, &arg0, 1, false);
> -         if (res)
> -           {
> -             /* fold_builtin_n wraps the result inside a NOP_EXPR.  */
> -             STRIP_NOPS (res);
> -             res = fold_convert (type, res);
> -             if (CONSTANT_CLASS_P (res))
> -               return res;
> -           }
> -       }
> +      tree res = fold_const_call (as_combined_fn (fn), type, arg0);
> +      if (res && CONSTANT_CLASS_P (res))
> +       return res;
>      }
>
>    code_helper rcode;
>    tree ops[3] = {};
>    if (!gimple_simplify (&rcode, ops, seq, valueize,
> -                       fn, type, arg0))
> +                       as_combined_fn (fn), type, arg0))
>      return NULL_TREE;
>    return maybe_push_res_to_seq (rcode, type, ops, seq);
>  }
> @@ -504,28 +495,15 @@ gimple_simplify (enum built_in_function fn, tree type,
>    if (constant_for_folding (arg0)
>        && constant_for_folding (arg1))
>      {
> -      tree decl = builtin_decl_implicit (fn);
> -      if (decl)
> -       {
> -         tree args[2];
> -         args[0] = arg0;
> -         args[1] = arg1;
> -         tree res = fold_builtin_n (UNKNOWN_LOCATION, decl, args, 2, false);
> -         if (res)
> -           {
> -             /* fold_builtin_n wraps the result inside a NOP_EXPR.  */
> -             STRIP_NOPS (res);
> -             res = fold_convert (type, res);
> -             if (CONSTANT_CLASS_P (res))
> -               return res;
> -           }
> -       }
> +      tree res = fold_const_call (as_combined_fn (fn), type, arg0, arg1);
> +      if (res && CONSTANT_CLASS_P (res))
> +       return res;
>      }
>
>    code_helper rcode;
>    tree ops[3] = {};
>    if (!gimple_simplify (&rcode, ops, seq, valueize,
> -                       fn, type, arg0, arg1))
> +                       as_combined_fn (fn), type, arg0, arg1))
>      return NULL_TREE;
>    return maybe_push_res_to_seq (rcode, type, ops, seq);
>  }
> @@ -541,29 +519,15 @@ gimple_simplify (enum built_in_function fn, tree type,
>        && constant_for_folding (arg1)
>        && constant_for_folding (arg2))
>      {
> -      tree decl = builtin_decl_implicit (fn);
> -      if (decl)
> -       {
> -         tree args[3];
> -         args[0] = arg0;
> -         args[1] = arg1;
> -         args[2] = arg2;
> -         tree res = fold_builtin_n (UNKNOWN_LOCATION, decl, args, 3, false);
> -         if (res)
> -           {
> -             /* fold_builtin_n wraps the result inside a NOP_EXPR.  */
> -             STRIP_NOPS (res);
> -             res = fold_convert (type, res);
> -             if (CONSTANT_CLASS_P (res))
> -               return res;
> -           }
> -       }
> +      tree res = fold_const_call (as_combined_fn (fn), type, arg0, arg1, arg2);
> +      if (res && CONSTANT_CLASS_P (res))
> +       return res;
>      }
>
>    code_helper rcode;
>    tree ops[3] = {};
>    if (!gimple_simplify (&rcode, ops, seq, valueize,
> -                       fn, type, arg0, arg1, arg2))
> +                       as_combined_fn (fn), type, arg0, arg1, arg2))
>      return NULL_TREE;
>    return maybe_push_res_to_seq (rcode, type, ops, seq);
>  }
> @@ -726,23 +690,29 @@ gimple_simplify (gimple *stmt,
>           && gimple_call_num_args (stmt) >= 1
>           && gimple_call_num_args (stmt) <= 3)
>         {
> -         tree fn = gimple_call_fn (stmt);
> -         /* ???  Internal function support missing.  */
> -         if (!fn)
> -           return false;
>           bool valueized = false;
> -         fn = do_valueize (fn, top_valueize, valueized);
> -         if (TREE_CODE (fn) != ADDR_EXPR
> -             || TREE_CODE (TREE_OPERAND (fn, 0)) != FUNCTION_DECL)
> -           return false;
> +         if (gimple_call_internal_p (stmt))
> +           *rcode = as_combined_fn (gimple_call_internal_fn (stmt));
> +         else
> +           {
> +             tree fn = gimple_call_fn (stmt);
> +             if (!fn)
> +               return false;
>
> -         tree decl = TREE_OPERAND (fn, 0);
> -         if (DECL_BUILT_IN_CLASS (decl) != BUILT_IN_NORMAL
> -             || !gimple_builtin_call_types_compatible_p (stmt, decl))
> -           return false;
> +             fn = do_valueize (fn, top_valueize, valueized);
> +             if (TREE_CODE (fn) != ADDR_EXPR
> +                 || TREE_CODE (TREE_OPERAND (fn, 0)) != FUNCTION_DECL)
> +               return false;
> +
> +             tree decl = TREE_OPERAND (fn, 0);
> +             if (DECL_BUILT_IN_CLASS (decl) != BUILT_IN_NORMAL
> +                 || !gimple_builtin_call_types_compatible_p (stmt, decl))
> +               return false;
> +
> +             *rcode = as_combined_fn (DECL_FUNCTION_CODE (decl));
> +           }
>
>           tree type = TREE_TYPE (gimple_call_lhs (stmt));
> -         *rcode = DECL_FUNCTION_CODE (decl);
>           for (unsigned i = 0; i < gimple_call_num_args (stmt); ++i)
>             {
>               tree arg = gimple_call_arg (stmt, i);
> diff --git a/gcc/gimple-match.h b/gcc/gimple-match.h
> index 632e9a5..1d5e444 100644
> --- a/gcc/gimple-match.h
> +++ b/gcc/gimple-match.h
> @@ -30,9 +30,9 @@ class code_helper
>  public:
>    code_helper () {}
>    code_helper (tree_code code) : rep ((int) code) {}
> -  code_helper (built_in_function fn) : rep (-(int) fn) {}
> +  code_helper (combined_fn fn) : rep (-(int) fn) {}
>    operator tree_code () const { return (tree_code) rep; }
> -  operator built_in_function () const { return (built_in_function) -rep; }
> +  operator combined_fn () const { return (combined_fn) -rep; }
>    bool is_tree_code () const { return rep > 0; }
>    bool is_fn_code () const { return rep < 0; }
>    int get_rep () const { return rep; }
> diff --git a/gcc/match.pd b/gcc/match.pd
> index d552beb..15bf2c9 100644
> --- a/gcc/match.pd
> +++ b/gcc/match.pd
> @@ -2510,32 +2510,33 @@ DEFINE_INT_AND_FLOAT_ROUND_FN (RINT)
>        exps (EXP2 EXP10 POW10 EXP  EXP10 POW10 EXP   EXP2)
>    (simplify
>     (logs (exps @0))
> -   (with {
> -     tree x;
> -     switch (exps)
> -       {
> -       CASE_FLT_FN (BUILT_IN_EXP):
> -         /* Prepare to do logN(exp(exponent)) -> exponent*logN(e).  */
> -        x = build_real_truncate (type, dconst_e ());
> -         break;
> -       CASE_FLT_FN (BUILT_IN_EXP2):
> -         /* Prepare to do logN(exp2(exponent)) -> exponent*logN(2).  */
> -         x = build_real (type, dconst2);
> -         break;
> -       CASE_FLT_FN (BUILT_IN_EXP10):
> -       CASE_FLT_FN (BUILT_IN_POW10):
> -        /* Prepare to do logN(exp10(exponent)) -> exponent*logN(10).  */
> -        {
> -          REAL_VALUE_TYPE dconst10;
> -          real_from_integer (&dconst10, VOIDmode, 10, SIGNED);
> -          x = build_real (type, dconst10);
> -        }
> -         break;
> -       default:
> -        gcc_unreachable ();
> -       }
> -     }
> -    (mult (logs { x; }) @0))))
> +   (if (SCALAR_FLOAT_TYPE_P (type))
> +    (with {
> +      tree x;
> +      switch (exps)
> +       {
> +       CASE_CFN_EXP:
> +         /* Prepare to do logN(exp(exponent)) -> exponent*logN(e).  */
> +         x = build_real_truncate (type, dconst_e ());
> +         break;
> +       CASE_CFN_EXP2:
> +         /* Prepare to do logN(exp2(exponent)) -> exponent*logN(2).  */
> +         x = build_real (type, dconst2);
> +         break;
> +       CASE_CFN_EXP10:
> +       CASE_CFN_POW10:
> +         /* Prepare to do logN(exp10(exponent)) -> exponent*logN(10).  */
> +         {
> +           REAL_VALUE_TYPE dconst10;
> +           real_from_integer (&dconst10, VOIDmode, 10, SIGNED);
> +           x = build_real (type, dconst10);
> +         }
> +         break;
> +       default:
> +         gcc_unreachable ();
> +       }
> +      }
> +     (mult (logs { x; }) @0)))))
>
>   (for logs (LOG LOG
>              LOG2 LOG2
> @@ -2543,23 +2544,24 @@ DEFINE_INT_AND_FLOAT_ROUND_FN (RINT)
>        exps (SQRT CBRT)
>    (simplify
>     (logs (exps @0))
> -   (with {
> -     tree x;
> -     switch (exps)
> -       {
> -       CASE_FLT_FN (BUILT_IN_SQRT):
> -        /* Prepare to do logN(sqrt(x)) -> 0.5*logN(x).  */
> -        x = build_real (type, dconsthalf);
> -         break;
> -       CASE_FLT_FN (BUILT_IN_CBRT):
> -        /* Prepare to do logN(cbrt(x)) -> (1/3)*logN(x).  */
> -         x = build_real_truncate (type, dconst_third ());
> -         break;
> -       default:
> -        gcc_unreachable ();
> -       }
> -     }
> -    (mult { x; } (logs @0)))))
> +   (if (SCALAR_FLOAT_TYPE_P (type))
> +    (with {
> +      tree x;
> +      switch (exps)
> +       {
> +       CASE_CFN_SQRT:
> +         /* Prepare to do logN(sqrt(x)) -> 0.5*logN(x).  */
> +         x = build_real (type, dconsthalf);
> +         break;
> +       CASE_CFN_CBRT:
> +         /* Prepare to do logN(cbrt(x)) -> (1/3)*logN(x).  */
> +         x = build_real_truncate (type, dconst_third ());
> +         break;
> +       default:
> +         gcc_unreachable ();
> +       }
> +      }
> +     (mult { x; } (logs @0))))))
>
>   /* logN(pow(x,exponent)) -> exponent*logN(x).  */
>   (for logs (LOG LOG2 LOG10)
> @@ -2616,7 +2618,7 @@ DEFINE_INT_AND_FLOAT_ROUND_FN (RINT)
>
>  /* hypot(x,0) and hypot(0,x) -> abs(x).  */
>  (simplify
> - (hypot:c @0 real_zerop@1)
> + (HYPOT:c @0 real_zerop@1)
>   (abs @0))
>
>  /* pow(1,x) -> 1.  */
> @@ -2684,7 +2686,7 @@ DEFINE_INT_AND_FLOAT_ROUND_FN (RINT)
>    (rdiv (SIN:s @0) (TAN:s @0))
>    (if (! HONOR_NANS (@0)
>         && ! HONOR_INFINITIES (@0))
> -   (cos @0)))
> +   (COS @0)))
>
>   /* Simplify tan(x) / sin(x) -> 1.0 / cos(x). */
>   (simplify
> diff --git a/gcc/tree.c b/gcc/tree.c
> index 29c5f4c..d5a71a3 100644
> --- a/gcc/tree.c
> +++ b/gcc/tree.c
> @@ -11063,6 +11063,22 @@ build_call_expr (tree fndecl, int n, ...)
>    return build_call_expr_loc_array (UNKNOWN_LOCATION, fndecl, n, argarray);
>  }
>
> +/* Build an internal call to IFN, with arguments ARGS[0:N-1] and with return
> +   type TYPE.  This is just like CALL_EXPR, except its CALL_EXPR_FN is NULL.
> +   It will get gimplified later into an ordinary internal function.  */
> +
> +tree
> +build_call_expr_internal_loc_array (location_t loc, internal_fn ifn,
> +                                   tree type, int n, const tree *args)
> +{
> +  tree t = build_call_1 (type, NULL_TREE, n);
> +  for (int i = 0; i < n; ++i)
> +    CALL_EXPR_ARG (t, i) = args[i];
> +  SET_EXPR_LOCATION (t, loc);
> +  CALL_EXPR_IFN (t) = ifn;
> +  return t;
> +}
> +
>  /* Build internal call expression.  This is just like CALL_EXPR, except
>     its CALL_EXPR_FN is NULL.  It will get gimplified later into ordinary
>     internal function.  */
> @@ -11072,16 +11088,52 @@ build_call_expr_internal_loc (location_t loc, enum internal_fn ifn,
>                               tree type, int n, ...)
>  {
>    va_list ap;
> +  tree *argarray = XALLOCAVEC (tree, n);
>    int i;
>
> -  tree fn = build_call_1 (type, NULL_TREE, n);
>    va_start (ap, n);
>    for (i = 0; i < n; i++)
> -    CALL_EXPR_ARG (fn, i) = va_arg (ap, tree);
> +    argarray[i] = va_arg (ap, tree);
>    va_end (ap);
> -  SET_EXPR_LOCATION (fn, loc);
> -  CALL_EXPR_IFN (fn) = ifn;
> -  return fn;
> +  return build_call_expr_internal_loc_array (loc, ifn, type, n, argarray);
> +}
> +
> +/* Return a function call to FN, if the target is guaranteed to support it,
> +   or null otherwise.
> +
> +   N is the number of arguments, passed in the "...", and TYPE is the
> +   type of the return value.  */
> +
> +tree
> +maybe_build_call_expr_loc (location_t loc, combined_fn fn, tree type,
> +                          int n, ...)
> +{
> +  va_list ap;
> +  tree *argarray = XALLOCAVEC (tree, n);
> +  int i;
> +
> +  va_start (ap, n);
> +  for (i = 0; i < n; i++)
> +    argarray[i] = va_arg (ap, tree);
> +  va_end (ap);
> +  if (internal_fn_p (fn))
> +    {
> +      internal_fn ifn = as_internal_fn (fn);
> +      if (direct_internal_fn_p (ifn))
> +       {
> +         tree_pair types = direct_internal_fn_types (ifn, type, argarray);
> +         if (!direct_internal_fn_supported_p (ifn, types))
> +           return NULL_TREE;
> +       }
> +      return build_call_expr_internal_loc_array (loc, ifn, type, n, argarray);
> +    }
> +  else
> +    {
> +      tree fndecl = builtin_decl_implicit (as_builtin_fn (fn));
> +      if (!fndecl)
> +       return NULL_TREE;
> +      return build_call_expr_loc_array (loc, fndecl, n, argarray);
> +    }
>  }
>
>  /* Create a new constant string literal and return a char* pointer to it.
> diff --git a/gcc/tree.h b/gcc/tree.h
> index 6a8354e..b9c400c 100644
> --- a/gcc/tree.h
> +++ b/gcc/tree.h
> @@ -3957,6 +3957,10 @@ extern tree build_call_expr_loc (location_t, tree, int, ...);
>  extern tree build_call_expr (tree, int, ...);
>  extern tree build_call_expr_internal_loc (location_t, enum internal_fn,
>                                           tree, int, ...);
> +extern tree build_call_expr_internal_loc (location_t, enum internal_fn,
> +                                         tree, int, tree *);
> +extern tree maybe_build_call_expr_loc (location_t, combined_fn, tree,
> +                                      int, ...);
>  extern tree build_string_literal (int, const char *);
>
>  /* Construct various nodes representing data types.  */
>