diff mbox

Cleaning up expand optabs code

Message ID g4k4fxzp0h.fsf@linaro.org
State New
Headers show

Commit Message

Richard Sandiford March 17, 2011, 4:32 p.m. UTC
This patch adds a few helper functions for dealing with optabs:

- insn_operand_matches (ICODE, OPNO, X)

  Return true if X is suitable for operand OPNO of instruction ICODE.

  [ I've deliberately left the mode out here.  We're testing whether
    something matches a match_operand (or suchlike) in the .md file.
    As far as recog is concerned, it's always the mode specified in
    the .md file that should be passed to the predicate, and I think
    that should be the rule for expand as well.

    Of course, in many cases, expand "knows" which mode the .md file
    is supposed to use.  But I think we should honour the .md-file
    mode even then, and abort where there's a mismatch.  We shouldn't
    just ignore what the .md file says.  I suppose this could expose
    a few .md-file bugs though...

    At the moment, some direct predicate calls are careful to use
    the match_operand mode, but others aren't. ]

- (maybe_)legitimize_insn_target (ICODE, OPNO, TARGET, MODE)

  Replaces the idiom:

      if (!TARGET
          || TARGET == const0_rtx
          || GET_MODE (TARGET) != MODE
          || !insn_operand_matches (ICODE, OPNO, TARGET))
        TARGET = gen_reg_rtx (...);

  and the various simplications of it.

  [ I think we should assert here that the new target really is OK,
    rather than passing it to the generator function regardless.
    Passing a bogus value would often trigger a later recog failure,
    but it might not be obvious at that stage where the instruction
    came from. ]

- (maybe_)legitimize_insn_source (ICODE, OPNO, SOURCE, MODE)

  Same idea for:

      if (!insn_operand_matches (ICODE, OPNO, SOURCE))
        SOURCE = copy_to_mode_reg (MODE, SOURCE);

  [ Here too I think we should assert that the new source is OK. ]

The main motivations are:

  * It should be simpler to add new optabs (I want to add a few more...)

  * I hope it'll be easier to avoid the kinds of bug that I came
    across while going through the optabs code (more below).

  * We could be more flexible in future.  E.g. I was told about a port
    in which several named patterns accept restricted memory operands.
    At the moment, I think you have to do something like:

    (define_expand "blah"
      [... (match_operand:M N "memory_operand") ...]
      "..."
      {
        if (!restricted_memory_operand (operands[N], M))
	  operands[N] = replace_equiv_address (operands[N],
					       force_reg (...));
      })

    (define_insn "*blah"
      [... (match_operand:M N "restricted_memory_operand") ...]
      ...

    It'd be nice if expand knew how to handle this automatically.
    It should be fairly easy now that predicates have a more
    declarative form.

A few notes:

- The new functions take insn_code arguments, so various bits of code
  that found it easier to use ints need to be changed to use the
  "proper" type instead.  This includes...

- prepare_operands.  I moved the declaration from expr.h to optabs.h,
  where we can be sure that insn_code is available.  This feels right
  anyway, since the function is defined in optabs.c.

  I've left this as a separate function for now because it can be called
  after reload as well.  I wanted to concentrate on expand in this patch.

- Parts of store_bit_field_1 used sequences rather than the usual
  delete_insns_since approach.  When using the new functions,
  the flow is easier with delete_insns_since.  Using delete_insns_since
  also avoids allocating unnecessary memory.

- If a movstrict expansion failed, the code would rightly try to delete
  any set-up instructions that had been created.  However, it stored the
  results of those set-up instructions in the function-wide "value" variable,
  so the rest of the function could end up using an uninitialised pseudo.

- The followuing functions didn't clean up any set-up instructions if
  the optab couldn't be used:

  - expand_builtin_strlen
  - emit_cstore
  - emit_storent_insn
  - expand_binop_directly (if a mode test failed)

  The insns would probably be deleted as dead by later optimisers,
  but it seems better to get rid of them straight away.

- Arguably the same sort of problem applied to:

  - expand_sync_operation
  - expand_sync_fetch_operation
  - expand_sync_lock_test_and_set

  although the effect here was simply that, if we forced something
  to a register for an insn that we ended up not being able to use,
  we'd keep it in that form anyway for the fallback code.

- The vec_extract code treated an output operand as an input:

      if (! (*insn_data[icode].operand[0].predicate) (dest, mode0))
	dest = copy_to_mode_reg (mode0, dest);

- I first tried to make insn_operand_matches an inline function,
  but I can't think of a good header file that is guaranteed to
  know about both recog.h and insn-codes.h.

The patch only tackles files in gcc/ itself.  If it's OK, I'll do a
follow-up for the backends.

Bootstrapped & regression-tested on x86_64-linux-gnu.  OK to install?

Richard


gcc/
	* expr.h (prepare_operand): Move to optabs.h.
	* optabs.h (emit_unop_insn): Change insn code parameter from "int"
	to "enum insn_code".
	(maybe_emit_unop_insn): Likewise.
	(prepare_operand): Likewise.  Move from expr.h.
	(insn_operand_matches): Declare.
	(maybe_legitimize_insn_target, legitimize_insn_target): Likewise.
	(maybe_legitimize_insn_source, legitimize_insn_source): Likewise.
	* builtins.c (expand_builtin_prefetch): Call convert_memory_address
	unconditionally.  Use legitimize_insn_source.
	(expand_builtin_interclass_mathfn): Use legitimize_insn_target.
	(expand_movstr): Likewise.   Use GEN_FCN.
	(expand_builtin___clear_cache): Use legitimize_insn_source.
	(expand_builtin_strlen): Likewise.  Clean up if the expander FAILs.
	(expand_builtin_lock_release): Likewise.
	* explow.c (probe_stack_range): Use legitimize_insn_source.
	(allocate_dynamic_stack_space): Likewise.  Call convert_to_mode
	unconditionally beforehand.
	* expmed.c (check_predicate_volatile_ok): Remove mode parameter.
	Use insn_operand_matches.
	(store_bit_field_1): Use insn_operand_matches and
	maybe_legitimize_insn_source.  Don't generate a sequences when
	tentatively expanding an instruction; use delete_insns_since instead.
	Localise the second operand to movstrict.  Update calls to
	check_predicate_volatile_ok.  Use "enum insn_code" for instruction
	codes.
	(extract_bit_field_1): Likewise, except for movstrict.
	Use maybe_legitimize_insn_target.
	(emit_cstore): Use insn_operand_matches and
	maybe_legitimize_insn_target.  Clean up if the expansion fails.
	* expr.c (init_expr_target): Use insn_operand_matches.
	(compress_float_constant): Likewise.
	(emit_block_move_via_movmem): Use insn_operand_matches and
	maybe_legitimize_insn_source.
	(set_storage_via_setmem): Likewise.
	(emit_storent_insn): Likewise.  Clean if up the expansion fails.
	(emit_single_push_insn): Use legitimize_insn_source.
	(expand_assignment): Likewise.
	(try_casesi): Likewise.
	* function.c (safe_insn_predicate): Use insn_operand_matches.
	(assign_parm_setup_reg): Likewise.
	* optabs.c (insn_operand_matches): New function.
	(maybe_legitimize_insn_target, legitimize_insn_target): Likewise.
	(maybe_legitimize_insn_source, legitimize_insn_source): Likewise.
	(expand_vec_shift_expr): Use legitimize_insn_source and
	legitimize_insn_target.
	(expand_vec_cond_expr): Likewise.
	(expand_widen_pattern_expr): Likewise.  Change icode to an insn_code.
	(expand_ternary_op): Likewise.
	(expand_unop_direct): Likewise.
	(maybe_emit_unop_insn): Likewise.
	(expand_binop_directly): Likewise.  Clean up if the mode isn't
	acceptable.
	(expand_twoval_unop): Change icode to an insn_code.
	Use insn_operand_matches and maybe_legitimize_insn_source.
	(expand_twoval_binop): Likewise.
	(expand_copysign_absneg): Change icode to an insn_code.
	(emit_unop_insn): Likewise.
	(emit_unop): Likewise.  Use insn_operand_matches.
	(can_compare_p): Likewise.
	(prepare_operand): Likewise.
	(gen_add2_insn): Likewise.
	(gen_add3_insn): Likewise.
	(have_add2_insn): Likewise.
	(gen_sub2_insn): Likewise.
	(gen_sub3_insn): Likewise.
	(have_sub2_insn): Likewise.
	(prepare_cmp_insn): Use insn_operand_matches.
	(emit_cmp_and_jump_insn_1): Likewise.
	(gen_cond_trap): Likewise.
	(emit_indirect_jump): Use legitimize_insn_source.
	(vector_compare_rtx): Likewise.
	(emit_conditional_move): Use maybe_legitimize_insn_source
	and maybe_legitimize_insn_target.
	(emit_conditional_add): Likewise.
	(expand_val_compare_and_swap_1): Likewise.  Clean up on failure.
	(expand_sync_operation): Likewise.  Use local variables for the
	coerced operands.
	(expand_sync_fetch_operation): Likewise.
	(expand_sync_lock_test_and_set): Likewise.
	* reload.c (find_reloads_address_1): Change icode to an insn_code.
	Use insn_operand_matches.
	* reload1.c (gen_reload): Likewise.
	* targhooks.c (default_secondary_reload): Use insn_operand_matches.

Comments

Richard Henderson March 17, 2011, 7:20 p.m. UTC | #1
On 03/17/2011 09:32 AM, Richard Sandiford wrote:
> This patch adds a few helper functions for dealing with optabs:
> 
> - insn_operand_matches (ICODE, OPNO, X)
> - (maybe_)legitimize_insn_target (ICODE, OPNO, TARGET, MODE)
> - (maybe_)legitimize_insn_source (ICODE, OPNO, SOURCE, MODE)

Cool.

Why pass in MODE in the later two cases?  Seems to me that your
argument for omitting it for insn_operand_matches is just as
compelling for the other functions...

> - I first tried to make insn_operand_matches an inline function,
>   but I can't think of a good header file that is guaranteed to
>   know about both recog.h and insn-codes.h.

Meh.  I can't imagine it mattering so much.  Anyway, an lto build
of gcc itself would fix that if needed, right?  ;-)

> -      char_rtx = const0_rtx;
> -      char_mode = insn_data[(int) icode].operand[2].mode;
> -      if (! (*insn_data[(int) icode].operand[2].predicate) (char_rtx,
> -							    char_mode))
> -	char_rtx = copy_to_mode_reg (char_mode, char_rtx);
> -
> -      pat = GEN_FCN (icode) (result, gen_rtx_MEM (BLKmode, src_reg),
> -			     char_rtx, GEN_INT (align));
> -      if (! pat)
> -	return NULL_RTX;
> +      if (!(char_rtx = maybe_legitimize_insn_source (icode, 2, const0_rtx,
> +						     VOIDmode))
> +	  || !(pat = GEN_FCN (icode) (result, gen_rtx_MEM (BLKmode, src_reg),
> +				      char_rtx, GEN_INT (align))))
> +	{
> +	  delete_insns_since (before_strlen);
> +	  return NULL_RTX;
> +	}
>        emit_insn (pat);

I'm not so fond of burying assignments into conditionals.  I know we
do it a lot in gcc, and it's a tad harder to avoid here than ...

> -      if (!insn_data[icode].operand[1].predicate (val, mode))
> -	val = force_reg (mode, val);
> -
> -      insn = GEN_FCN (icode) (mem, val);
> -      if (insn)
> +      start = get_last_insn ();
> +      if ((val = maybe_legitimize_insn_source (icode, 1, const0_rtx, mode))
> +	  && (insn = GEN_FCN (icode) (mem, val)))
>  	{
>  	  emit_insn (insn);
>  	  return;
>  	}
> +      delete_insns_since (start);

... here, where it's just as readable to write

	val = maybe_legitimize_insn_source (icode, 1, const0_rtx, mode);
	if (val)
	  {
	    insn = GEN_FCN (icode) (mem, val);
	    if (insn)
	      {
		emit_insn (insn);
		return;
	      }
	  }
	delete_insns_since (start);

> +  if ((temp = maybe_legitimize_insn_target (icode, 0, target, tmp_mode))
> +      && (xop0 = maybe_legitimize_insn_source (icode, 1, xop0, mode0))
> +      && (xop1 = maybe_legitimize_insn_source (icode, 2, xop1, mode1))
> +      && (pat = GEN_FCN (icode) (temp, xop0, xop1)))

Although, these patterns occur often enough that I wonder if there's a way
to tidy them up into a single function call.  We could either use a set of
functions like

    target = maybe_legitimize_emit_insn_ts (icode, target, src1);
    target = maybe_legitimize_emit_insn_tss (icode, target, src1, src2);

or have a single function with variadic source operands.  Probably the
separate functions is better for error checking within the compiler.  We
can validate the correct function was called for a given icode based on
the n_operands stored in the insn_data, and the compiler itself will make
sure that we didn't omit an argument for that function.

The interface I was considering here would validate the target and all
of the sources, emit the insn or delete_insns_since, and return the new
target or NULL if no insn was emitted.

All that said, I'm very much in favour of this cleanup.


r~
diff mbox

Patch

Index: gcc/expr.h
===================================================================
--- gcc/expr.h	2011-03-04 14:15:06.000000000 +0000
+++ gcc/expr.h	2011-03-16 15:44:16.000000000 +0000
@@ -173,9 +173,6 @@  extern rtx expand_simple_unop (enum mach
    perform the operation described by CODE and MODE.  */
 extern int have_insn_for (enum rtx_code, enum machine_mode);
 
-extern rtx prepare_operand (int, rtx, int, enum machine_mode, enum machine_mode,
-			    int);
-
 /* Emit code to make a call to a constant function or a library call.  */
 extern void emit_libcall_block (rtx, rtx, rtx, rtx);
 
Index: gcc/optabs.h
===================================================================
--- gcc/optabs.h	2011-03-16 09:02:36.000000000 +0000
+++ gcc/optabs.h	2011-03-16 15:44:38.000000000 +0000
@@ -791,8 +791,8 @@  extern rtx expand_copysign (rtx, rtx, rt
 
 /* Generate an instruction with a given INSN_CODE with an output and
    an input.  */
-extern void emit_unop_insn (int, rtx, rtx, enum rtx_code);
-extern bool maybe_emit_unop_insn (int, rtx, rtx, enum rtx_code);
+extern void emit_unop_insn (enum insn_code, rtx, rtx, enum rtx_code);
+extern bool maybe_emit_unop_insn (enum insn_code, rtx, rtx, enum rtx_code);
 
 /* An extra flag to control optab_for_tree_code's behavior.  This is needed to
    distinguish between machines with a vector shift that takes a scalar for the
@@ -923,4 +923,21 @@  set_direct_optab_handler (direct_optab o
 extern rtx optab_libfunc (optab optab, enum machine_mode mode);
 extern rtx convert_optab_libfunc (convert_optab optab, enum machine_mode mode1,
 			          enum machine_mode mode2);
+
+extern bool insn_operand_matches (enum insn_code icode, unsigned int opno,
+				  rtx operand);
+extern rtx maybe_legitimize_insn_target (enum insn_code icode,
+					 unsigned int opno, rtx target,
+					 enum machine_mode mode);
+extern rtx legitimize_insn_target (enum insn_code icode, unsigned int opno,
+				   rtx target, enum machine_mode mode);
+extern rtx maybe_legitimize_insn_source (enum insn_code icode,
+					 unsigned int opno, rtx source,
+					 enum machine_mode mode);
+extern rtx legitimize_insn_source (enum insn_code icode, unsigned int opno,
+				   rtx source, enum machine_mode mode);
+
+extern rtx prepare_operand (enum insn_code, rtx, int, enum machine_mode,
+			    enum machine_mode, int);
+
 #endif /* GCC_OPTABS_H */
Index: gcc/builtins.c
===================================================================
--- gcc/builtins.c	2011-03-16 09:02:40.000000000 +0000
+++ gcc/builtins.c	2011-03-17 09:22:38.000000000 +0000
@@ -1143,14 +1143,8 @@  expand_builtin_prefetch (tree exp)
 #ifdef HAVE_prefetch
   if (HAVE_prefetch)
     {
-      if ((! (*insn_data[(int) CODE_FOR_prefetch].operand[0].predicate)
-	     (op0,
-	      insn_data[(int) CODE_FOR_prefetch].operand[0].mode))
-	  || (GET_MODE (op0) != Pmode))
-	{
-	  op0 = convert_memory_address (Pmode, op0);
-	  op0 = force_reg (Pmode, op0);
-	}
+      op0 = convert_memory_address (Pmode, op0);
+      op0 = legitimize_insn_source (CODE_FOR_prefetch, 0, op0, Pmode);
       emit_insn (gen_prefetch (op0, op1, op2));
     }
 #endif
@@ -2434,13 +2428,8 @@  expand_builtin_interclass_mathfn (tree e
       rtx last = get_last_insn ();
       tree orig_arg = arg;
       /* Make a suitable register to place result in.  */
-      if (!target
-	  || GET_MODE (target) != TYPE_MODE (TREE_TYPE (exp))
-	  || !insn_data[icode].operand[0].predicate (target, GET_MODE (target)))
-         target = gen_reg_rtx (TYPE_MODE (TREE_TYPE (exp)));
-
-      gcc_assert (insn_data[icode].operand[0].predicate
-		  (target, GET_MODE (target)));
+      target = legitimize_insn_target (icode, 0, target,
+				       TYPE_MODE (TREE_TYPE (exp)));
 
       /* Wrap the computation of the argument in a SAVE_EXPR, as we may
 	 need to expand the argument again.  This way, we will not perform
@@ -3366,7 +3355,7 @@  expand_builtin_strlen (tree exp, rtx tar
       tree len;
       tree src = CALL_EXPR_ARG (exp, 0);
       rtx result, src_reg, char_rtx, before_strlen;
-      enum machine_mode insn_mode = target_mode, char_mode;
+      enum machine_mode insn_mode = target_mode;
       enum insn_code icode = CODE_FOR_nothing;
       unsigned int align;
 
@@ -3422,16 +3411,14 @@  expand_builtin_strlen (tree exp, rtx tar
 	 source operand later.  */
       before_strlen = get_last_insn ();
 
-      char_rtx = const0_rtx;
-      char_mode = insn_data[(int) icode].operand[2].mode;
-      if (! (*insn_data[(int) icode].operand[2].predicate) (char_rtx,
-							    char_mode))
-	char_rtx = copy_to_mode_reg (char_mode, char_rtx);
-
-      pat = GEN_FCN (icode) (result, gen_rtx_MEM (BLKmode, src_reg),
-			     char_rtx, GEN_INT (align));
-      if (! pat)
-	return NULL_RTX;
+      if (!(char_rtx = maybe_legitimize_insn_source (icode, 2, const0_rtx,
+						     VOIDmode))
+	  || !(pat = GEN_FCN (icode) (result, gen_rtx_MEM (BLKmode, src_reg),
+				      char_rtx, GEN_INT (align))))
+	{
+	  delete_insns_since (before_strlen);
+	  return NULL_RTX;
+	}
       emit_insn (pat);
 
       /* Now that we are assured of success, expand the source.  */
@@ -3678,14 +3665,12 @@  expand_movstr (tree dest, tree src, rtx 
   rtx dest_mem;
   rtx src_mem;
   rtx insn;
-  const struct insn_data_d * data;
 
   if (!HAVE_movstr)
     return NULL_RTX;
 
   dest_mem = get_memory_rtx (dest, NULL);
   src_mem = get_memory_rtx (src, NULL);
-  data = insn_data + CODE_FOR_movstr;
   if (!endp)
     {
       target = force_reg (Pmode, XEXP (dest_mem, 0));
@@ -3694,22 +3679,12 @@  expand_movstr (tree dest, tree src, rtx 
     }
   else
     {
-      if (target == 0
-	  || target == const0_rtx
-	  || ! (*data->operand[0].predicate) (target, Pmode))
-	{
-	  end = gen_reg_rtx (Pmode);
-	  if (target != const0_rtx)
-	    target = end;
-	}
-      else
-	end = target;
+      end = legitimize_insn_target (CODE_FOR_movstr, 0, target, Pmode);
+      if (target != const0_rtx)
+	target = end;
     }
 
-  if (data->operand[0].mode != VOIDmode)
-    end = gen_lowpart (data->operand[0].mode, end);
-
-  insn = data->genfun (end, dest_mem, src_mem);
+  insn = GEN_FCN (CODE_FOR_movstr) (end, dest_mem, src_mem);
 
   gcc_assert (insn);
 
@@ -5241,14 +5216,12 @@  expand_builtin___clear_cache (tree exp A
       begin = CALL_EXPR_ARG (exp, 0);
       begin_rtx = expand_expr (begin, NULL_RTX, Pmode, EXPAND_NORMAL);
       begin_rtx = convert_memory_address (Pmode, begin_rtx);
-      if (!insn_data[icode].operand[0].predicate (begin_rtx, Pmode))
-	begin_rtx = copy_to_mode_reg (Pmode, begin_rtx);
+      begin_rtx = legitimize_insn_source (icode, 0, begin_rtx, Pmode);
 
       end = CALL_EXPR_ARG (exp, 1);
       end_rtx = expand_expr (end, NULL_RTX, Pmode, EXPAND_NORMAL);
       end_rtx = convert_memory_address (Pmode, end_rtx);
-      if (!insn_data[icode].operand[1].predicate (end_rtx, Pmode))
-	end_rtx = copy_to_mode_reg (Pmode, end_rtx);
+      end_rtx = legitimize_insn_source (icode, 1, end_rtx, Pmode);
 
       emit_insn (gen_clear_cache (begin_rtx, end_rtx));
     }
@@ -5749,8 +5722,7 @@  expand_builtin_synchronize (void)
 expand_builtin_lock_release (enum machine_mode mode, tree exp)
 {
   enum insn_code icode;
-  rtx mem, insn;
-  rtx val = const0_rtx;
+  rtx mem, insn, start, val;
 
   /* Expand the operands.  */
   mem = get_builtin_sync_mem (CALL_EXPR_ARG (exp, 0), mode);
@@ -5759,21 +5731,20 @@  expand_builtin_lock_release (enum machin
   icode = direct_optab_handler (sync_lock_release_optab, mode);
   if (icode != CODE_FOR_nothing)
     {
-      if (!insn_data[icode].operand[1].predicate (val, mode))
-	val = force_reg (mode, val);
-
-      insn = GEN_FCN (icode) (mem, val);
-      if (insn)
+      start = get_last_insn ();
+      if ((val = maybe_legitimize_insn_source (icode, 1, const0_rtx, mode))
+	  && (insn = GEN_FCN (icode) (mem, val)))
 	{
 	  emit_insn (insn);
 	  return;
 	}
+      delete_insns_since (start);
     }
 
   /* Otherwise we can implement this operation by emitting a barrier
      followed by a store of zero.  */
   expand_builtin_synchronize ();
-  emit_move_insn (mem, val);
+  emit_move_insn (mem, const0_rtx);
 }
 
 /* Expand an expression EXP that calls a built-in function,
Index: gcc/explow.c
===================================================================
--- gcc/explow.c	2011-03-04 14:15:06.000000000 +0000
+++ gcc/explow.c	2011-03-16 11:19:28.000000000 +0000
@@ -1380,7 +1380,6 @@  allocate_dynamic_stack_space (rtx size, 
   if (HAVE_allocate_stack)
     {
       enum machine_mode mode = STACK_SIZE_MODE;
-      insn_operand_predicate_fn pred;
 
       /* We don't have to check against the predicate for operand 0 since
 	 TARGET is known to be a pseudo of the proper mode, which must
@@ -1388,11 +1387,8 @@  allocate_dynamic_stack_space (rtx size, 
 	 proper mode and validate.  */
       if (mode == VOIDmode)
 	mode = insn_data[(int) CODE_FOR_allocate_stack].operand[1].mode;
-
-      pred = insn_data[(int) CODE_FOR_allocate_stack].operand[1].predicate;
-      if (pred && ! ((*pred) (size, mode)))
-	size = copy_to_mode_reg (mode, convert_to_mode (mode, size, 1));
-
+      size = convert_to_mode (mode, size, 1);
+      size = legitimize_insn_source (CODE_FOR_allocate_stack, 1, size, mode);
       emit_insn (gen_allocate_stack (target, size));
     }
   else
@@ -1554,11 +1550,7 @@  probe_stack_range (HOST_WIDE_INT first, 
 				 gen_rtx_fmt_ee (STACK_GROW_OP, Pmode,
 					         stack_pointer_rtx,
 					         plus_constant (size, first)));
-      insn_operand_predicate_fn pred
-	= insn_data[(int) CODE_FOR_check_stack].operand[0].predicate;
-      if (pred && !((*pred) (addr, Pmode)))
-	addr = copy_to_mode_reg (Pmode, addr);
-
+      addr = legitimize_insn_source (CODE_FOR_check_stack, 0, addr, Pmode);
       emit_insn (gen_check_stack (addr));
     }
 #endif
Index: gcc/expmed.c
===================================================================
--- gcc/expmed.c	2011-03-04 14:15:06.000000000 +0000
+++ gcc/expmed.c	2011-03-17 10:38:39.000000000 +0000
@@ -324,18 +324,17 @@  mode_for_extraction (enum extraction_pat
   return data->operand[opno].mode;
 }
 
-/* Return true if X, of mode MODE, matches the predicate for operand
-   OPNO of instruction ICODE.  Allow volatile memories, regardless of
+/* Return true if X matches the predicate for operand OPNO of
+   instruction ICODE.  Allow volatile memories, regardless of
    the ambient volatile_ok setting.  */
 
 static bool
-check_predicate_volatile_ok (enum insn_code icode, int opno,
-			     rtx x, enum machine_mode mode)
+check_predicate_volatile_ok (enum insn_code icode, int opno, rtx x)
 {
   bool save_volatile_ok, result;
 
   save_volatile_ok = volatile_ok;
-  result = insn_data[(int) icode].operand[opno].predicate (x, mode);
+  result = insn_operand_matches (icode, opno, x);
   volatile_ok = save_volatile_ok;
   return result;
 }
@@ -407,38 +406,25 @@  store_bit_field_1 (rtx str_rtx, unsigned
     {
       enum machine_mode outermode = GET_MODE (op0);
       enum machine_mode innermode = GET_MODE_INNER (outermode);
-      int icode = (int) optab_handler (vec_set_optab, outermode);
+      enum insn_code icode = optab_handler (vec_set_optab, outermode);
       int pos = bitnum / GET_MODE_BITSIZE (innermode);
-      rtx rtxpos = GEN_INT (pos);
-      rtx src = value;
       rtx dest = op0;
-      rtx pat, seq;
-      enum machine_mode mode0 = insn_data[icode].operand[0].mode;
-      enum machine_mode mode1 = insn_data[icode].operand[1].mode;
-      enum machine_mode mode2 = insn_data[icode].operand[2].mode;
+      rtx rtxpos, src, start, pat;
 
-      start_sequence ();
-
-      if (! (*insn_data[icode].operand[1].predicate) (src, mode1))
-	src = copy_to_mode_reg (mode1, src);
-
-      if (! (*insn_data[icode].operand[2].predicate) (rtxpos, mode2))
-	rtxpos = copy_to_mode_reg (mode1, rtxpos);
+      start = get_last_insn ();
 
       /* We could handle this, but we should always be called with a pseudo
 	 for our targets and all insns should take them as outputs.  */
-      gcc_assert ((*insn_data[icode].operand[0].predicate) (dest, mode0)
-		  && (*insn_data[icode].operand[1].predicate) (src, mode1)
-		  && (*insn_data[icode].operand[2].predicate) (rtxpos, mode2));
-      pat = GEN_FCN (icode) (dest, src, rtxpos);
-      seq = get_insns ();
-      end_sequence ();
-      if (pat)
+      gcc_assert (insn_operand_matches (icode, 0, dest));
+      if ((src = maybe_legitimize_insn_source (icode, 1, value, innermode))
+	  && (rtxpos = maybe_legitimize_insn_source (icode, 2, GEN_INT (pos),
+						     VOIDmode))
+	  && (pat = GEN_FCN (icode) (dest, src, rtxpos)))
 	{
-	  emit_insn (seq);
 	  emit_insn (pat);
 	  return true;
 	}
+      delete_insns_since (start);
     }
 
   /* If the target is a register, overwriting the entire object, or storing
@@ -515,39 +501,37 @@  store_bit_field_1 (rtx str_rtx, unsigned
       && bitsize == GET_MODE_BITSIZE (fieldmode)
       && optab_handler (movstrict_optab, fieldmode) != CODE_FOR_nothing)
     {
-      int icode = optab_handler (movstrict_optab, fieldmode);
+      enum insn_code icode = optab_handler (movstrict_optab, fieldmode);
       rtx insn;
       rtx start = get_last_insn ();
       rtx arg0 = op0;
+      rtx arg1 = value;
 
-      /* Get appropriate low part of the value being stored.  */
-      if (CONST_INT_P (value) || REG_P (value))
-	value = gen_lowpart (fieldmode, value);
-      else if (!(GET_CODE (value) == SYMBOL_REF
-		 || GET_CODE (value) == LABEL_REF
-		 || GET_CODE (value) == CONST))
-	value = convert_to_mode (fieldmode, value, 0);
+      /* Get appropriate low part of the arg1 being stored.  */
+      if (CONST_INT_P (arg1) || REG_P (arg1))
+	arg1 = gen_lowpart (fieldmode, arg1);
+      else if (!(GET_CODE (arg1) == SYMBOL_REF
+		 || GET_CODE (arg1) == LABEL_REF
+		 || GET_CODE (arg1) == CONST))
+	arg1 = convert_to_mode (fieldmode, arg1, 0);
 
-      if (! (*insn_data[icode].operand[1].predicate) (value, fieldmode))
-	value = copy_to_mode_reg (fieldmode, value);
-
-      if (GET_CODE (op0) == SUBREG)
+      if (GET_CODE (arg0) == SUBREG)
 	{
 	  /* Else we've got some float mode source being extracted into
 	     a different float mode destination -- this combination of
 	     subregs results in Severe Tire Damage.  */
-	  gcc_assert (GET_MODE (SUBREG_REG (op0)) == fieldmode
+	  gcc_assert (GET_MODE (SUBREG_REG (arg0)) == fieldmode
 		      || GET_MODE_CLASS (fieldmode) == MODE_INT
 		      || GET_MODE_CLASS (fieldmode) == MODE_PARTIAL_INT);
-	  arg0 = SUBREG_REG (op0);
+	  arg0 = SUBREG_REG (arg0);
 	}
 
-      insn = (GEN_FCN (icode)
-		 (gen_rtx_SUBREG (fieldmode, arg0,
-				  (bitnum % BITS_PER_WORD) / BITS_PER_UNIT
-				  + (offset * UNITS_PER_WORD)),
-				  value));
-      if (insn)
+      if ((arg1 = maybe_legitimize_insn_source (icode, 1, arg1, fieldmode))
+	  && (insn = (GEN_FCN (icode)
+		      (gen_rtx_SUBREG (fieldmode, arg0,
+				       (bitnum % BITS_PER_WORD) / BITS_PER_UNIT
+				       + (offset * UNITS_PER_WORD)),
+		       arg1))))
 	{
 	  emit_insn (insn);
 	  return true;
@@ -654,9 +638,8 @@  store_bit_field_1 (rtx str_rtx, unsigned
       && GET_MODE_BITSIZE (op_mode) >= bitsize
       && ! ((REG_P (op0) || GET_CODE (op0) == SUBREG)
 	    && (bitsize + bitpos > GET_MODE_BITSIZE (op_mode)))
-      && insn_data[CODE_FOR_insv].operand[1].predicate (GEN_INT (bitsize),
-							VOIDmode)
-      && check_predicate_volatile_ok (CODE_FOR_insv, 0, op0, VOIDmode))
+      && insn_operand_matches (CODE_FOR_insv, 1, GEN_INT (bitsize))
+      && check_predicate_volatile_ok (CODE_FOR_insv, 0, op0))
     {
       int xbitpos = bitpos;
       rtx value1;
@@ -745,12 +728,10 @@  store_bit_field_1 (rtx str_rtx, unsigned
 
       /* If this machine's insv insists on a register,
 	 get VALUE1 into a register.  */
-      if (! ((*insn_data[(int) CODE_FOR_insv].operand[3].predicate)
-	     (value1, op_mode)))
-	value1 = force_reg (op_mode, value1);
-
-      pat = gen_insv (xop0, GEN_INT (bitsize), GEN_INT (xbitpos), value1);
-      if (pat)
+      if ((value1 = maybe_legitimize_insn_source (CODE_FOR_insv, 3,
+						  value1, op_mode))
+	  && (pat = gen_insv (xop0, GEN_INT (bitsize),
+			      GEN_INT (xbitpos), value1)))
 	{
 	  emit_insn (pat);
 
@@ -1237,49 +1218,23 @@  extract_bit_field_1 (rtx str_rtx, unsign
     {
       enum machine_mode outermode = GET_MODE (op0);
       enum machine_mode innermode = GET_MODE_INNER (outermode);
-      int icode = (int) optab_handler (vec_extract_optab, outermode);
+      enum insn_code icode = optab_handler (vec_extract_optab, outermode);
       unsigned HOST_WIDE_INT pos = bitnum / GET_MODE_BITSIZE (innermode);
-      rtx rtxpos = GEN_INT (pos);
-      rtx src = op0;
-      rtx dest = NULL, pat, seq;
-      enum machine_mode mode0 = insn_data[icode].operand[0].mode;
-      enum machine_mode mode1 = insn_data[icode].operand[1].mode;
-      enum machine_mode mode2 = insn_data[icode].operand[2].mode;
-
-      if (innermode == tmode || innermode == mode)
-	dest = target;
-
-      if (!dest)
-	dest = gen_reg_rtx (innermode);
-
-      start_sequence ();
-
-      if (! (*insn_data[icode].operand[0].predicate) (dest, mode0))
-	dest = copy_to_mode_reg (mode0, dest);
-
-      if (! (*insn_data[icode].operand[1].predicate) (src, mode1))
-	src = copy_to_mode_reg (mode1, src);
-
-      if (! (*insn_data[icode].operand[2].predicate) (rtxpos, mode2))
-	rtxpos = copy_to_mode_reg (mode1, rtxpos);
+      rtx dest, src, rtxpos, pat, last;
 
-      /* We could handle this, but we should always be called with a pseudo
-	 for our targets and all insns should take them as outputs.  */
-      gcc_assert ((*insn_data[icode].operand[0].predicate) (dest, mode0)
-		  && (*insn_data[icode].operand[1].predicate) (src, mode1)
-		  && (*insn_data[icode].operand[2].predicate) (rtxpos, mode2));
-
-      pat = GEN_FCN (icode) (dest, src, rtxpos);
-      seq = get_insns ();
-      end_sequence ();
-      if (pat)
+      last = get_last_insn ();
+      if ((dest = maybe_legitimize_insn_target (icode, 0, target, innermode))
+	  && (src = maybe_legitimize_insn_source (icode, 1, op0, outermode))
+	  && (rtxpos = maybe_legitimize_insn_source (icode, 2, GEN_INT (pos),
+						     VOIDmode))
+	  && (pat = GEN_FCN (icode) (dest, src, rtxpos)))
 	{
-	  emit_insn (seq);
 	  emit_insn (pat);
-      	  if (mode0 != mode)
+      	  if (GET_MODE (dest) != mode)
 	    return gen_lowpart (tmode, dest);
 	  return dest;
 	}
+      delete_insns_since (last);
     }
 
   /* Make sure we are playing with integral modes.  Pun with subregs
@@ -1518,7 +1473,7 @@  extract_bit_field_1 (rtx str_rtx, unsign
       && !(GET_CODE (op0) == SUBREG && GET_MODE (op0) != ext_mode)
       && !((REG_P (op0) || GET_CODE (op0) == SUBREG)
 	   && (bitsize + bitpos > GET_MODE_BITSIZE (ext_mode)))
-      && check_predicate_volatile_ok (icode, 1, op0, GET_MODE (op0)))
+      && check_predicate_volatile_ok (icode, 1, op0))
     {
       unsigned HOST_WIDE_INT xbitpos = bitpos, xoffset = offset;
       rtx bitsize_rtx, bitpos_rtx;
@@ -1570,18 +1525,13 @@  extract_bit_field_1 (rtx str_rtx, unsign
 	    xtarget = gen_reg_rtx (ext_mode);
 	}
 
-      /* If this machine's ext(z)v insists on a register target,
-	 make sure we have one.  */
-      if (!insn_data[(int) icode].operand[0].predicate (xtarget, ext_mode))
-	xtarget = gen_reg_rtx (ext_mode);
-
       bitsize_rtx = GEN_INT (bitsize);
       bitpos_rtx = GEN_INT (xbitpos);
 
-      pat = (unsignedp
-	     ? gen_extzv (xtarget, xop0, bitsize_rtx, bitpos_rtx)
-	     : gen_extv (xtarget, xop0, bitsize_rtx, bitpos_rtx));
-      if (pat)
+      if ((xtarget = maybe_legitimize_insn_target (icode, 0, xtarget, ext_mode))
+	  && (pat = (unsignedp
+		     ? gen_extzv (xtarget, xop0, bitsize_rtx, bitpos_rtx)
+		     : gen_extv (xtarget, xop0, bitsize_rtx, bitpos_rtx))))
 	{
 	  emit_insn (pat);
 	  if (xtarget == xspec_target)
@@ -5109,11 +5059,9 @@  emit_cstore (rtx target, enum insn_code 
   y = prepare_operand (icode, y, 3, mode, compare_mode, unsignedp);
   comparison = gen_rtx_fmt_ee (code, result_mode, x, y);
   if (!x || !y
-      || !insn_data[icode].operand[2].predicate
-	  (x, insn_data[icode].operand[2].mode)
-      || !insn_data[icode].operand[3].predicate
-	  (y, insn_data[icode].operand[3].mode)
-      || !insn_data[icode].operand[1].predicate (comparison, VOIDmode))
+      || !insn_operand_matches (icode, 2, x)
+      || !insn_operand_matches (icode, 3, y)
+      || !insn_operand_matches (icode, 1, comparison))
     {
       delete_insns_since (last);
       return NULL_RTX;
@@ -5124,15 +5072,14 @@  emit_cstore (rtx target, enum insn_code 
   if (!target)
     target = gen_reg_rtx (target_mode);
 
-  if (optimize
-      || !(insn_data[(int) icode].operand[0].predicate (target, result_mode)))
-    subtarget = gen_reg_rtx (result_mode);
-  else
-    subtarget = target;
-
-  pattern = GEN_FCN (icode) (subtarget, comparison, x, y);
-  if (!pattern)
-    return NULL_RTX;
+  if (!(subtarget = maybe_legitimize_insn_target (icode, 0,
+						  optimize ? NULL_RTX : target,
+						  result_mode))
+      || !(pattern = GEN_FCN (icode) (subtarget, comparison, x, y)))
+    {
+      delete_insns_since (last);
+      return NULL_RTX;
+    }
   emit_insn (pattern);
 
   /* If we are converting to a wider mode, first convert to
Index: gcc/expr.c
===================================================================
--- gcc/expr.c	2011-03-15 14:16:09.000000000 +0000
+++ gcc/expr.c	2011-03-16 16:57:21.000000000 +0000
@@ -286,7 +286,7 @@  init_expr_target (void)
 
 	  PUT_MODE (mem, srcmode);
 
-	  if ((*insn_data[ic].operand[1].predicate) (mem, srcmode))
+	  if (insn_operand_matches (ic, 1, mem))
 	    float_extend_from_mem[mode][srcmode] = true;
 	}
     }
@@ -1276,7 +1276,6 @@  emit_block_move_via_movmem (rtx x, rtx y
        mode = GET_MODE_WIDER_MODE (mode))
     {
       enum insn_code code = direct_optab_handler (movmem_optab, mode);
-      insn_operand_predicate_fn pred;
 
       if (code != CODE_FOR_nothing
 	  /* We don't need MODE to be narrower than BITS_PER_HOST_WIDE_INT
@@ -1287,42 +1286,32 @@  emit_block_move_via_movmem (rtx x, rtx y
 	       && ((unsigned HOST_WIDE_INT) INTVAL (size)
 		   <= (GET_MODE_MASK (mode) >> 1)))
 	      || GET_MODE_BITSIZE (mode) >= BITS_PER_WORD)
-	  && ((pred = insn_data[(int) code].operand[0].predicate) == 0
-	      || (*pred) (x, BLKmode))
-	  && ((pred = insn_data[(int) code].operand[1].predicate) == 0
-	      || (*pred) (y, BLKmode))
-	  && ((pred = insn_data[(int) code].operand[3].predicate) == 0
-	      || (*pred) (opalign, VOIDmode)))
+	  && insn_operand_matches (code, 0, x)
+	  && insn_operand_matches (code, 1, y)
+	  && insn_operand_matches (code, 3, opalign))
 	{
 	  rtx op2;
 	  rtx last = get_last_insn ();
 	  rtx pat;
 
-	  op2 = convert_to_mode (mode, size, 1);
-	  pred = insn_data[(int) code].operand[2].predicate;
-	  if (pred != 0 && ! (*pred) (op2, mode))
-	    op2 = copy_to_mode_reg (mode, op2);
-
 	  /* ??? When called via emit_block_move_for_call, it'd be
 	     nice if there were some way to inform the backend, so
 	     that it doesn't fail the expansion because it thinks
 	     emitting the libcall would be more efficient.  */
-
-	  if (insn_data[(int) code].n_operands == 4)
-	    pat = GEN_FCN ((int) code) (x, y, op2, opalign);
-	  else
-	    pat = GEN_FCN ((int) code) (x, y, op2, opalign,
-					GEN_INT (expected_align
-						 / BITS_PER_UNIT),
-					GEN_INT (expected_size));
-	  if (pat)
+	  op2 = convert_to_mode (mode, size, 1);
+	  if ((op2 = maybe_legitimize_insn_source (code, 2, op2, mode))
+	      && (pat = (insn_data[(int) code].n_operands == 4
+			 ? GEN_FCN ((int) code) (x, y, op2, opalign)
+			 : GEN_FCN ((int) code) (x, y, op2, opalign,
+						 GEN_INT (expected_align
+							  / BITS_PER_UNIT),
+						 GEN_INT (expected_size)))))
 	    {
 	      emit_insn (pat);
 	      volatile_ok = save_volatile_ok;
 	      return true;
 	    }
-	  else
-	    delete_insns_since (last);
+	  delete_insns_since (last);
 	}
     }
 
@@ -2715,7 +2704,6 @@  set_storage_via_setmem (rtx object, rtx 
        mode = GET_MODE_WIDER_MODE (mode))
     {
       enum insn_code code = direct_optab_handler (setmem_optab, mode);
-      insn_operand_predicate_fn pred;
 
       if (code != CODE_FOR_nothing
 	  /* We don't need MODE to be narrower than
@@ -2726,10 +2714,8 @@  set_storage_via_setmem (rtx object, rtx 
 	       && ((unsigned HOST_WIDE_INT) INTVAL (size)
 		   <= (GET_MODE_MASK (mode) >> 1)))
 	      || GET_MODE_BITSIZE (mode) >= BITS_PER_WORD)
-	  && ((pred = insn_data[(int) code].operand[0].predicate) == 0
-	      || (*pred) (object, BLKmode))
-	  && ((pred = insn_data[(int) code].operand[3].predicate) == 0
-	      || (*pred) (opalign, VOIDmode)))
+	  && insn_operand_matches (code, 0, object)
+	  && insn_operand_matches (code, 3, opalign))
 	{
 	  rtx opsize, opchar;
 	  enum machine_mode char_mode;
@@ -2737,34 +2723,28 @@  set_storage_via_setmem (rtx object, rtx 
 	  rtx pat;
 
 	  opsize = convert_to_mode (mode, size, 1);
-	  pred = insn_data[(int) code].operand[1].predicate;
-	  if (pred != 0 && ! (*pred) (opsize, mode))
-	    opsize = copy_to_mode_reg (mode, opsize);
 
 	  opchar = val;
 	  char_mode = insn_data[(int) code].operand[2].mode;
 	  if (char_mode != VOIDmode)
-	    {
-	      opchar = convert_to_mode (char_mode, opchar, 1);
-	      pred = insn_data[(int) code].operand[2].predicate;
-	      if (pred != 0 && ! (*pred) (opchar, char_mode))
-		opchar = copy_to_mode_reg (char_mode, opchar);
-	    }
+	    opchar = convert_to_mode (char_mode, opchar, 1);
 
-	  if (insn_data[(int) code].n_operands == 4)
-	    pat = GEN_FCN ((int) code) (object, opsize, opchar, opalign);
-	  else
-	    pat = GEN_FCN ((int) code) (object, opsize, opchar, opalign,
-					GEN_INT (expected_align
-						 / BITS_PER_UNIT),
-					GEN_INT (expected_size));
-	  if (pat)
+	  if ((opsize = maybe_legitimize_insn_source (code, 1, opsize, mode))
+	      && (opchar = maybe_legitimize_insn_source (code, 2, opchar,
+							 char_mode))
+	      && (pat = (insn_data[(int) code].n_operands == 4
+			 ? GEN_FCN ((int) code) (object, opsize,
+						 opchar, opalign)
+			 : GEN_FCN ((int) code) (object, opsize, opchar,
+						 opalign,
+						 GEN_INT (expected_align
+							  / BITS_PER_UNIT),
+						 GEN_INT (expected_size)))))
 	    {
 	      emit_insn (pat);
 	      return true;
 	    }
-	  else
-	    delete_insns_since (last);
+	  delete_insns_since (last);
 	}
     }
 
@@ -3446,7 +3426,7 @@  compress_float_constant (rtx x, rtx y)
 	{
 	  /* Skip if the target needs extra instructions to perform
 	     the extension.  */
-	  if (! (*insn_data[ic].operand[1].predicate) (trunc_y, srcmode))
+	  if (!insn_operand_matches (ic, 1, trunc_y))
 	    continue;
 	  /* This is valid, but may not be cheaper than the original. */
 	  newcost = rtx_cost (gen_rtx_FLOAT_EXTEND (dstmode, trunc_y), SET, speed);
@@ -3547,7 +3527,6 @@  emit_single_push_insn (enum machine_mode
   unsigned rounded_size = PUSH_ROUNDING (GET_MODE_SIZE (mode));
   rtx dest;
   enum insn_code icode;
-  insn_operand_predicate_fn pred;
 
   stack_pointer_delta += PUSH_ROUNDING (GET_MODE_SIZE (mode));
   /* If there is push pattern, use it.  Otherwise try old way of throwing
@@ -3555,9 +3534,7 @@  emit_single_push_insn (enum machine_mode
   icode = optab_handler (push_optab, mode);
   if (icode != CODE_FOR_nothing)
     {
-      if (((pred = insn_data[(int) icode].operand[0].predicate)
-	   && !((*pred) (x, mode))))
-	x = force_reg (mode, x);
+      x = legitimize_insn_source (icode, 0, x, mode);
       emit_insn (GEN_FCN (icode) (x));
       return;
     }
@@ -4121,7 +4098,8 @@  expand_assignment (tree to, tree from, b
   rtx to_rtx = 0;
   rtx result;
   enum machine_mode mode;
-  int align, icode;
+  int align;
+  enum insn_code icode;
 
   /* Don't crash if the lhs of the assignment was erroneous.  */
   if (TREE_CODE (to) == ERROR_MARK)
@@ -4144,7 +4122,7 @@  expand_assignment (tree to, tree from, b
       && ((icode = optab_handler (movmisalign_optab, mode))
 	  != CODE_FOR_nothing))
     {
-      enum machine_mode address_mode, op_mode1;
+      enum machine_mode address_mode;
       rtx insn, reg, op0, mem;
 
       reg = expand_expr (from, NULL_RTX, VOIDmode, EXPAND_NORMAL);
@@ -4186,11 +4164,7 @@  expand_assignment (tree to, tree from, b
       if (TREE_THIS_VOLATILE (to))
 	MEM_VOLATILE_P (mem) = 1;
 
-      op_mode1 = insn_data[icode].operand[1].mode;
-      if (! (*insn_data[icode].operand[1].predicate) (reg, op_mode1)
-	  && op_mode1 != VOIDmode)
-	reg = copy_to_mode_reg (op_mode1, reg);
-
+      reg = legitimize_insn_source (icode, 1, reg, mode);
       insn = GEN_FCN (icode) (mem, reg);
       /* The movmisalign<mode> pattern cannot fail, else the assignment would
          silently be omitted.  */
@@ -4457,31 +4431,25 @@  expand_assignment (tree to, tree from, b
 bool
 emit_storent_insn (rtx to, rtx from)
 {
-  enum machine_mode mode = GET_MODE (to), imode;
+  enum machine_mode mode = GET_MODE (to);
   enum insn_code code = optab_handler (storent_optab, mode);
-  rtx pattern;
+  rtx pattern, last;
 
   if (code == CODE_FOR_nothing)
     return false;
 
-  imode = insn_data[code].operand[0].mode;
-  if (!insn_data[code].operand[0].predicate (to, imode))
+  if (!insn_operand_matches (code, 0, to))
     return false;
 
-  imode = insn_data[code].operand[1].mode;
-  if (!insn_data[code].operand[1].predicate (from, imode))
+  last = get_last_insn ();
+  if ((from = maybe_legitimize_insn_source (code, 1, from, mode))
+      && (pattern = GEN_FCN (code) (to, from)))
     {
-      from = copy_to_mode_reg (imode, from);
-      if (!insn_data[code].operand[1].predicate (from, imode))
-	return false;
+      emit_insn (pattern);
+      return true;
     }
-
-  pattern = GEN_FCN (code) (to, from);
-  if (pattern == NULL_RTX)
-    return false;
-
-  emit_insn (pattern);
-  return true;
+  delete_insns_since (last);
+  return false;
 }
 
 /* Generate code for computing expression EXP,
@@ -10133,28 +10101,21 @@  try_casesi (tree index_type, tree index_
 
   do_pending_stack_adjust ();
 
-  op_mode = insn_data[(int) CODE_FOR_casesi].operand[0].mode;
-  if (! (*insn_data[(int) CODE_FOR_casesi].operand[0].predicate)
-      (index, op_mode))
-    index = copy_to_mode_reg (op_mode, index);
+  index = legitimize_insn_source (CODE_FOR_casesi, 0, index, VOIDmode);
 
   op1 = expand_normal (minval);
 
   op_mode = insn_data[(int) CODE_FOR_casesi].operand[1].mode;
   op1 = convert_modes (op_mode, TYPE_MODE (TREE_TYPE (minval)),
 		       op1, TYPE_UNSIGNED (TREE_TYPE (minval)));
-  if (! (*insn_data[(int) CODE_FOR_casesi].operand[1].predicate)
-      (op1, op_mode))
-    op1 = copy_to_mode_reg (op_mode, op1);
+  op1 = legitimize_insn_source (CODE_FOR_casesi, 1, op1, op_mode);
 
   op2 = expand_normal (range);
 
   op_mode = insn_data[(int) CODE_FOR_casesi].operand[2].mode;
   op2 = convert_modes (op_mode, TYPE_MODE (TREE_TYPE (range)),
 		       op2, TYPE_UNSIGNED (TREE_TYPE (range)));
-  if (! (*insn_data[(int) CODE_FOR_casesi].operand[2].predicate)
-      (op2, op_mode))
-    op2 = copy_to_mode_reg (op_mode, op2);
+  op2 = legitimize_insn_source (CODE_FOR_casesi, 2, op2, op_mode);
 
   emit_jump_insn (gen_casesi (index, op1, op2,
 			      table_label, !default_label
Index: gcc/function.c
===================================================================
--- gcc/function.c	2011-03-14 11:55:40.000000000 +0000
+++ gcc/function.c	2011-03-16 15:29:53.000000000 +0000
@@ -1493,16 +1493,7 @@  instantiate_virtual_regs_in_rtx (rtx *lo
 static int
 safe_insn_predicate (int code, int operand, rtx x)
 {
-  const struct insn_operand_data *op_data;
-
-  if (code < 0)
-    return true;
-
-  op_data = &insn_data[code].operand[operand];
-  if (op_data->predicate == NULL)
-    return true;
-
-  return op_data->predicate (x, op_data->mode);
+  return code < 0 || insn_operand_matches ((enum insn_code) code, operand, x);
 }
 
 /* A subroutine of instantiate_virtual_regs.  Instantiate any virtual
@@ -3013,8 +3004,8 @@  assign_parm_setup_reg (struct assign_par
       op0 = parmreg;
       op1 = validated_mem;
       if (icode != CODE_FOR_nothing
-	  && insn_data[icode].operand[0].predicate (op0, promoted_nominal_mode)
-	  && insn_data[icode].operand[1].predicate (op1, data->passed_mode))
+	  && insn_operand_matches (icode, 0, op0)
+	  && insn_operand_matches (icode, 1, op1))
 	{
 	  enum rtx_code code = unsignedp ? ZERO_EXTEND : SIGN_EXTEND;
 	  rtx insn, insns;
Index: gcc/optabs.c
===================================================================
--- gcc/optabs.c	2011-03-04 14:15:06.000000000 +0000
+++ gcc/optabs.c	2011-03-17 12:25:28.000000000 +0000
@@ -157,6 +157,113 @@  optab_libfunc (optab optab, enum machine
 }
 
 
+/* Return true if OPERAND is suitable for operand number OPNO of
+   instruction ICODE.  */
+
+bool
+insn_operand_matches (enum insn_code icode, unsigned int opno, rtx operand)
+{
+  return (!insn_data[(int) icode].operand[opno].predicate
+	  || (insn_data[(int) icode].operand[opno].predicate
+	      (operand, insn_data[(int) icode].operand[opno].mode)));
+}
+
+/* Try to create an rtx that is suitable for output operand OPNO of
+   instruction ICODE.  MODE is the mode of target that the caller requires,
+   or VOIDmode if it doesn't care.  Return the rtx on success or null on
+   failure.
+
+   If TARGET is nonnull and not const0_rtx, try to use that rtx as the
+   target if it meets all the requirements.  It is this function's
+   responsibility to check whether TARGET is consistent with MODE,
+   not the caller's.  */
+
+rtx
+maybe_legitimize_insn_target (enum insn_code icode, unsigned int opno,
+			      rtx target, enum machine_mode mode)
+{
+  if (!target
+      || target == const0_rtx
+      || (mode != VOIDmode && GET_MODE (target) != mode)
+      || !insn_operand_matches (icode, opno, target))
+    {
+      if (mode == VOIDmode)
+	{
+	  mode = insn_data[(int) icode].operand[opno].mode;
+	  gcc_assert (mode != VOIDmode);
+	}
+      target = gen_reg_rtx (mode);
+    }
+  if (!insn_operand_matches (icode, opno, target))
+    return NULL_RTX;
+  return target;
+}
+
+/* Like maybe_legitimize_target, but require the operation to succeed.  */
+
+rtx
+legitimize_insn_target (enum insn_code icode, unsigned int opno,
+			rtx target, enum machine_mode mode)
+{
+  target = maybe_legitimize_insn_target (icode, opno, target, mode);
+  gcc_assert (target);
+  return target;
+}
+
+/* Try to coerce SOURCE into a form that is suitable for input operand
+   OPNO of instruction ICODE.  MODE is the mode of operand that the
+   caller requires, or VOIDmode if it doesn't care.  SOURCE must be
+   consistent with MODE.
+
+   Return the new form of SOURCE on success and null on failure.  */
+
+rtx
+maybe_legitimize_insn_source (enum insn_code icode, unsigned int opno,
+			      rtx source, enum machine_mode mode)
+{
+  /* This function should never need to do any mode conversion.  */
+  if (mode == VOIDmode)
+    mode = GET_MODE (source);
+  else
+    gcc_assert (GET_MODE (source) == VOIDmode || GET_MODE (source) == mode);
+
+  if (!insn_operand_matches (icode, opno, source))
+    {
+      rtx start;
+
+      start = get_last_insn ();
+      /* If the caller has given a CONST_INT without specifying a MODE.
+	 we can only go further if the instruction pattern itself
+	 nominates a suitable mode.  */
+      if (mode == VOIDmode)
+	{
+	  gcc_assert (CONST_INT_P (source));
+	  mode = insn_data[(int) icode].operand[opno].mode;
+	  if (mode == VOIDmode
+	      || trunc_int_for_mode (INTVAL (source), mode) != INTVAL (source))
+	    return NULL_RTX;
+	}
+      source = copy_to_mode_reg (mode, source);
+      if (!insn_operand_matches (icode, opno, source))
+	{
+	  delete_insns_since (start);
+	  return NULL_RTX;
+	}
+    }
+  return source;
+}
+
+/* Like maybe_legitimize_insn_source, but require the coercion to succeed.  */
+
+rtx
+legitimize_insn_source (enum insn_code icode, unsigned int opno,
+			rtx source, enum machine_mode mode)
+{
+  source = maybe_legitimize_insn_source (icode, opno, source, mode);
+  gcc_assert (source);
+  return source;
+}
+
 /* Add a REG_EQUAL note to the last insn in INSNS.  TARGET is being set to
    the result of operation CODE applied to OP0 (and OP1 if it is a binary
    operation).
@@ -504,7 +611,7 @@  expand_widen_pattern_expr (sepops ops, r
   tree oprnd0, oprnd1, oprnd2;
   enum machine_mode wmode = VOIDmode, tmode0, tmode1 = VOIDmode;
   optab widen_pattern_optab;
-  int icode;
+  enum insn_code icode;
   enum machine_mode xmode0, xmode1 = VOIDmode, wxmode = VOIDmode;
   rtx temp;
   rtx pat;
@@ -517,18 +624,18 @@  expand_widen_pattern_expr (sepops ops, r
     optab_for_tree_code (ops->code, TREE_TYPE (oprnd0), optab_default);
   if (ops->code == WIDEN_MULT_PLUS_EXPR
       || ops->code == WIDEN_MULT_MINUS_EXPR)
-    icode = (int) optab_handler (widen_pattern_optab,
-				 TYPE_MODE (TREE_TYPE (ops->op2)));
+    icode = optab_handler (widen_pattern_optab,
+			   TYPE_MODE (TREE_TYPE (ops->op2)));
   else
-    icode = (int) optab_handler (widen_pattern_optab, tmode0);
+    icode = optab_handler (widen_pattern_optab, tmode0);
   gcc_assert (icode != CODE_FOR_nothing);
-  xmode0 = insn_data[icode].operand[1].mode;
+  xmode0 = insn_data[(int) icode].operand[1].mode;
 
   if (nops >= 2)
     {
       oprnd1 = ops->op1;
       tmode1 = TYPE_MODE (TREE_TYPE (oprnd1));
-      xmode1 = insn_data[icode].operand[2].mode;
+      xmode1 = insn_data[(int) icode].operand[2].mode;
     }
 
   /* The last operand is of a wider mode than the rest of the operands.  */
@@ -543,18 +650,13 @@  expand_widen_pattern_expr (sepops ops, r
       gcc_assert (op1);
       oprnd2 = ops->op2;
       wmode = TYPE_MODE (TREE_TYPE (oprnd2));
-      wxmode = insn_data[icode].operand[3].mode;
+      wxmode = insn_data[(int) icode].operand[3].mode;
     }
 
   if (!wide_op)
-    wmode = wxmode = insn_data[icode].operand[0].mode;
-
-  if (!target
-      || ! (*insn_data[icode].operand[0].predicate) (target, wmode))
-    temp = gen_reg_rtx (wmode);
-  else
-    temp = target;
+    wmode = wxmode = insn_data[(int) icode].operand[0].mode;
 
+  temp = legitimize_insn_target (icode, 0, target, wmode);
   xop0 = op0;
   xop1 = op1;
   wxop = wide_op;
@@ -591,22 +693,13 @@  expand_widen_pattern_expr (sepops ops, r
   /* Now, if insn's predicates don't allow our operands, put them into
      pseudo regs.  */
 
-  if (! (*insn_data[icode].operand[1].predicate) (xop0, xmode0)
-      && xmode0 != VOIDmode)
-    xop0 = copy_to_mode_reg (xmode0, xop0);
-
+  xop0 = legitimize_insn_source (icode, 1, xop0, xmode0);
   if (op1)
     {
-      if (! (*insn_data[icode].operand[2].predicate) (xop1, xmode1)
-          && xmode1 != VOIDmode)
-        xop1 = copy_to_mode_reg (xmode1, xop1);
-
+      xop1 = legitimize_insn_source (icode, 2, xop1, xmode1);
       if (wide_op)
         {
-          if (! (*insn_data[icode].operand[3].predicate) (wxop, wxmode)
-              && wxmode != VOIDmode)
-            wxop = copy_to_mode_reg (wxmode, wxop);
-
+	  wxop = legitimize_insn_source (icode, 3, wxop, wxmode);
           pat = GEN_FCN (icode) (temp, xop0, xop1, wxop);
         }
       else
@@ -616,10 +709,7 @@  expand_widen_pattern_expr (sepops ops, r
     {
       if (wide_op)
         {
-          if (! (*insn_data[icode].operand[2].predicate) (wxop, wxmode)
-              && wxmode != VOIDmode)
-            wxop = copy_to_mode_reg (wxmode, wxop);
-
+	  wxop = legitimize_insn_source (icode, 2, wxop, wxmode);
           pat = GEN_FCN (icode) (temp, xop0, wxop);
         }
       else
@@ -645,20 +735,17 @@  expand_widen_pattern_expr (sepops ops, r
 expand_ternary_op (enum machine_mode mode, optab ternary_optab, rtx op0,
 		   rtx op1, rtx op2, rtx target, int unsignedp)
 {
-  int icode = (int) optab_handler (ternary_optab, mode);
-  enum machine_mode mode0 = insn_data[icode].operand[1].mode;
-  enum machine_mode mode1 = insn_data[icode].operand[2].mode;
-  enum machine_mode mode2 = insn_data[icode].operand[3].mode;
+  enum insn_code icode = optab_handler (ternary_optab, mode);
+  enum machine_mode mode0 = insn_data[(int) icode].operand[1].mode;
+  enum machine_mode mode1 = insn_data[(int) icode].operand[2].mode;
+  enum machine_mode mode2 = insn_data[(int) icode].operand[3].mode;
   rtx temp;
   rtx pat;
   rtx xop0 = op0, xop1 = op1, xop2 = op2;
 
   gcc_assert (optab_handler (ternary_optab, mode) != CODE_FOR_nothing);
 
-  if (!target || !insn_data[icode].operand[0].predicate (target, mode))
-    temp = gen_reg_rtx (mode);
-  else
-    temp = target;
+  temp = legitimize_insn_target (icode, 0, target, mode);
 
   /* In case the insn wants input operands in modes different from
      those of the actual operands, convert the operands.  It would
@@ -690,18 +777,9 @@  expand_ternary_op (enum machine_mode mod
   /* Now, if insn's predicates don't allow our operands, put them into
      pseudo regs.  */
 
-  if (!insn_data[icode].operand[1].predicate (xop0, mode0)
-      && mode0 != VOIDmode)
-    xop0 = copy_to_mode_reg (mode0, xop0);
-
-  if (!insn_data[icode].operand[2].predicate (xop1, mode1)
-      && mode1 != VOIDmode)
-    xop1 = copy_to_mode_reg (mode1, xop1);
-
-  if (!insn_data[icode].operand[3].predicate (xop2, mode2)
-      && mode2 != VOIDmode)
-    xop2 = copy_to_mode_reg (mode2, xop2);
-
+  xop0 = legitimize_insn_source (icode, 1, xop0, mode0);
+  xop1 = legitimize_insn_source (icode, 2, xop1, mode1);
+  xop2 = legitimize_insn_source (icode, 3, xop2, mode2);
   pat = GEN_FCN (icode) (temp, xop0, xop1, xop2);
 
   emit_insn (pat);
@@ -753,8 +831,6 @@  expand_vec_shift_expr (sepops ops, rtx t
 {
   enum insn_code icode;
   rtx rtx_op1, rtx_op2;
-  enum machine_mode mode1;
-  enum machine_mode mode2;
   enum machine_mode mode = TYPE_MODE (ops->type);
   tree vec_oprnd = ops->op0;
   tree shift_oprnd = ops->op1;
@@ -776,22 +852,13 @@  expand_vec_shift_expr (sepops ops, rtx t
   icode = optab_handler (shift_optab, mode);
   gcc_assert (icode != CODE_FOR_nothing);
 
-  mode1 = insn_data[icode].operand[1].mode;
-  mode2 = insn_data[icode].operand[2].mode;
-
   rtx_op1 = expand_normal (vec_oprnd);
-  if (!(*insn_data[icode].operand[1].predicate) (rtx_op1, mode1)
-      && mode1 != VOIDmode)
-    rtx_op1 = force_reg (mode1, rtx_op1);
+  rtx_op1 = legitimize_insn_source (icode, 1, rtx_op1, VOIDmode);
 
   rtx_op2 = expand_normal (shift_oprnd);
-  if (!(*insn_data[icode].operand[2].predicate) (rtx_op2, mode2)
-      && mode2 != VOIDmode)
-    rtx_op2 = force_reg (mode2, rtx_op2);
+  rtx_op2 = legitimize_insn_source (icode, 2, rtx_op2, VOIDmode);
 
-  if (!target
-      || ! (*insn_data[icode].operand[0].predicate) (target, mode))
-    target = gen_reg_rtx (mode);
+  target = legitimize_insn_target (icode, 0, target, mode);
 
   /* Emit instruction */
   pat = GEN_FCN (icode) (target, rtx_op1, rtx_op2);
@@ -1389,9 +1456,9 @@  expand_binop_directly (enum machine_mode
 		       rtx target, int unsignedp, enum optab_methods methods,
 		       rtx last)
 {
-  int icode = (int) optab_handler (binoptab, mode);
-  enum machine_mode mode0 = insn_data[icode].operand[1].mode;
-  enum machine_mode mode1 = insn_data[icode].operand[2].mode;
+  enum insn_code icode = optab_handler (binoptab, mode);
+  enum machine_mode mode0 = insn_data[(int) icode].operand[1].mode;
+  enum machine_mode mode1 = insn_data[(int) icode].operand[2].mode;
   enum machine_mode tmp_mode;
   bool commutative_p;
   rtx pat;
@@ -1399,11 +1466,6 @@  expand_binop_directly (enum machine_mode
   rtx temp;
   rtx swap;
 
-  if (target)
-    temp = target;
-  else
-    temp = gen_reg_rtx (mode);
-
   /* If it is a commutative operator and the modes would match
      if we would swap the operands, we can save the conversions.  */
   commutative_p = commutative_optab_p (binoptab);
@@ -1456,14 +1518,6 @@  expand_binop_directly (enum machine_mode
   /* Now, if insn's predicates don't allow our operands, put them into
      pseudo regs.  */
 
-  if (!insn_data[icode].operand[1].predicate (xop0, mode0)
-      && mode0 != VOIDmode)
-    xop0 = copy_to_mode_reg (mode0, xop0);
-
-  if (!insn_data[icode].operand[2].predicate (xop1, mode1)
-      && mode1 != VOIDmode)
-    xop1 = copy_to_mode_reg (mode1, xop1);
-
   if (binoptab == vec_pack_trunc_optab
       || binoptab == vec_pack_usat_optab
       || binoptab == vec_pack_ssat_optab
@@ -1472,18 +1526,20 @@  expand_binop_directly (enum machine_mode
     {
       /* The mode of the result is different then the mode of the
 	 arguments.  */
-      tmp_mode = insn_data[icode].operand[0].mode;
+      tmp_mode = insn_data[(int) icode].operand[0].mode;
       if (GET_MODE_NUNITS (tmp_mode) != 2 * GET_MODE_NUNITS (mode))
-	return 0;
+	{
+	  delete_insns_since (last);
+	  return NULL_RTX;
+	}
     }
   else
     tmp_mode = mode;
 
-  if (!insn_data[icode].operand[0].predicate (temp, tmp_mode))
-    temp = gen_reg_rtx (tmp_mode);
-
-  pat = GEN_FCN (icode) (temp, xop0, xop1);
-  if (pat)
+  if ((temp = maybe_legitimize_insn_target (icode, 0, target, tmp_mode))
+      && (xop0 = maybe_legitimize_insn_source (icode, 1, xop0, mode0))
+      && (xop1 = maybe_legitimize_insn_source (icode, 2, xop1, mode1))
+      && (pat = GEN_FCN (icode) (temp, xop0, xop1)))
     {
       /* If PAT is composed of more than one insn, try to add an appropriate
 	 REG_EQUAL note to it.  If we can't because TEMP conflicts with an
@@ -2284,8 +2340,8 @@  expand_twoval_unop (optab unoptab, rtx o
 
   if (optab_handler (unoptab, mode) != CODE_FOR_nothing)
     {
-      int icode = (int) optab_handler (unoptab, mode);
-      enum machine_mode mode0 = insn_data[icode].operand[2].mode;
+      enum insn_code icode = optab_handler (unoptab, mode);
+      enum machine_mode mode0 = insn_data[(int) icode].operand[2].mode;
       rtx pat;
       rtx xop0 = op0;
 
@@ -2293,23 +2349,20 @@  expand_twoval_unop (optab unoptab, rtx o
 	  && GET_MODE (xop0) != mode0)
 	xop0 = convert_to_mode (mode0, xop0, unsignedp);
 
-      /* Now, if insn doesn't accept these operands, put them into pseudos.  */
-      if (!insn_data[icode].operand[2].predicate (xop0, mode0))
-	xop0 = copy_to_mode_reg (mode0, xop0);
-
-      /* We could handle this, but we should always be called with a pseudo
-	 for our targets and all insns should take them as outputs.  */
-      gcc_assert (insn_data[icode].operand[0].predicate (targ0, mode));
-      gcc_assert (insn_data[icode].operand[1].predicate (targ1, mode));
+      /* We could handle this, but we should always be called with
+	 a pseudo for our targets and all insns should take them as
+	 outputs.  */
+      gcc_assert (insn_operand_matches (icode, 0, targ0));
+      gcc_assert (insn_operand_matches (icode, 1, targ1));
 
-      pat = GEN_FCN (icode) (targ0, targ1, xop0);
-      if (pat)
+      /* Now, if insn doesn't accept these operands, put them into pseudos.  */
+      if ((xop0 = maybe_legitimize_insn_source (icode, 2, xop0, mode0))
+	  && (pat = GEN_FCN (icode) (targ0, targ1, xop0)))
 	{
 	  emit_insn (pat);
 	  return 1;
 	}
-      else
-	delete_insns_since (last);
+      delete_insns_since (last);
     }
 
   /* It can't be done in this mode.  Can we do it in a wider mode?  */
@@ -2376,9 +2429,9 @@  expand_twoval_binop (optab binoptab, rtx
 
   if (optab_handler (binoptab, mode) != CODE_FOR_nothing)
     {
-      int icode = (int) optab_handler (binoptab, mode);
-      enum machine_mode mode0 = insn_data[icode].operand[1].mode;
-      enum machine_mode mode1 = insn_data[icode].operand[2].mode;
+      enum insn_code icode = optab_handler (binoptab, mode);
+      enum machine_mode mode0 = insn_data[(int) icode].operand[1].mode;
+      enum machine_mode mode1 = insn_data[(int) icode].operand[2].mode;
       rtx pat;
       rtx xop0 = op0, xop1 = op1;
 
@@ -2406,26 +2459,20 @@  expand_twoval_binop (optab binoptab, rtx
 			      : mode,
 			      xop1, unsignedp);
 
-      /* Now, if insn doesn't accept these operands, put them into pseudos.  */
-      if (!insn_data[icode].operand[1].predicate (xop0, mode0))
-	xop0 = copy_to_mode_reg (mode0, xop0);
-
-      if (!insn_data[icode].operand[2].predicate (xop1, mode1))
-	xop1 = copy_to_mode_reg (mode1, xop1);
-
       /* We could handle this, but we should always be called with a pseudo
 	 for our targets and all insns should take them as outputs.  */
-      gcc_assert (insn_data[icode].operand[0].predicate (targ0, mode));
-      gcc_assert (insn_data[icode].operand[3].predicate (targ1, mode));
+      gcc_assert (insn_operand_matches (icode, 0, targ0));
+      gcc_assert (insn_operand_matches (icode, 3, targ1));
 
-      pat = GEN_FCN (icode) (targ0, xop0, xop1, targ1);
-      if (pat)
+      /* Now, if insn doesn't accept these operands, put them into pseudos.  */
+      if ((xop0 = maybe_legitimize_insn_source (icode, 1, xop0, mode0))
+	  && (xop1 = maybe_legitimize_insn_source (icode, 2, xop1, mode1))
+	  && (pat = GEN_FCN (icode) (targ0, xop0, xop1, targ1)))
 	{
 	  emit_insn (pat);
 	  return 1;
 	}
-      else
-	delete_insns_since (last);
+      delete_insns_since (last);
     }
 
   /* It can't be done in this mode.  Can we do it in a wider mode?  */
@@ -2985,31 +3032,20 @@  expand_unop_direct (enum machine_mode mo
 {
   if (optab_handler (unoptab, mode) != CODE_FOR_nothing)
     {
-      int icode = (int) optab_handler (unoptab, mode);
-      enum machine_mode mode0 = insn_data[icode].operand[1].mode;
+      enum insn_code icode = optab_handler (unoptab, mode);
+      enum machine_mode mode0 = insn_data[(int) icode].operand[1].mode;
       rtx xop0 = op0;
       rtx last = get_last_insn ();
       rtx pat, temp;
 
-      if (target)
-	temp = target;
-      else
-	temp = gen_reg_rtx (mode);
-
       if (GET_MODE (xop0) != VOIDmode
 	  && GET_MODE (xop0) != mode0)
 	xop0 = convert_to_mode (mode0, xop0, unsignedp);
 
       /* Now, if insn doesn't accept our operand, put it into a pseudo.  */
-
-      if (!insn_data[icode].operand[1].predicate (xop0, mode0))
-	xop0 = copy_to_mode_reg (mode0, xop0);
-
-      if (!insn_data[icode].operand[0].predicate (temp, mode))
-	temp = gen_reg_rtx (mode);
-
-      pat = GEN_FCN (icode) (temp, xop0);
-      if (pat)
+      if ((temp = maybe_legitimize_insn_target (icode, 0, target, mode))
+	  && (xop0 = maybe_legitimize_insn_source (icode, 1, xop0, mode0))
+	  && (pat = GEN_FCN (icode) (temp, xop0)))
 	{
 	  if (INSN_P (pat) && NEXT_INSN (pat) != NULL_RTX
 	      && ! add_equal_note (pat, temp, unoptab->code, xop0, NULL_RTX))
@@ -3022,8 +3058,7 @@  expand_unop_direct (enum machine_mode mo
 
 	  return temp;
 	}
-      else
-	delete_insns_since (last);
+      delete_insns_since (last);
     }
   return 0;
 }
@@ -3499,7 +3534,7 @@  expand_copysign_absneg (enum machine_mod
 		        int bitpos, bool op0_is_abs)
 {
   enum machine_mode imode;
-  int icode;
+  enum insn_code icode;
   rtx sign, label;
 
   if (target == op1)
@@ -3507,10 +3542,10 @@  expand_copysign_absneg (enum machine_mod
 
   /* Check if the back end provides an insn that handles signbit for the
      argument's mode. */
-  icode = (int) optab_handler (signbit_optab, mode);
+  icode = optab_handler (signbit_optab, mode);
   if (icode != CODE_FOR_nothing)
     {
-      imode = insn_data[icode].operand[0].mode;
+      imode = insn_data[(int) icode].operand[0].mode;
       sign = gen_reg_rtx (imode);
       emit_unop_insn (icode, sign, op1, UNKNOWN);
     }
@@ -3731,38 +3766,30 @@  expand_copysign (rtx op0, rtx op1, rtx t
    Return false if expansion failed.  */
 
 bool
-maybe_emit_unop_insn (int icode, rtx target, rtx op0, enum rtx_code code)
+maybe_emit_unop_insn (enum insn_code icode, rtx target, rtx op0,
+		      enum rtx_code code)
 {
   rtx temp;
-  enum machine_mode mode0 = insn_data[icode].operand[1].mode;
   rtx pat;
   rtx last = get_last_insn ();
 
-  temp = target;
-
   /* Now, if insn does not accept our operands, put them into pseudos.  */
-
-  if (!insn_data[icode].operand[1].predicate (op0, mode0))
-    op0 = copy_to_mode_reg (mode0, op0);
-
-  if (!insn_data[icode].operand[0].predicate (temp, GET_MODE (temp)))
-    temp = gen_reg_rtx (GET_MODE (temp));
-
-  pat = GEN_FCN (icode) (temp, op0);
-  if (!pat)
+  if ((temp = maybe_legitimize_insn_target (icode, 0, target,
+					    GET_MODE (target)))
+      && (op0 = maybe_legitimize_insn_source (icode, 1, op0, VOIDmode))
+      && (pat = GEN_FCN (icode) (temp, op0)))
     {
-      delete_insns_since (last);
-      return false;
-    }
+      if (INSN_P (pat) && NEXT_INSN (pat) != NULL_RTX && code != UNKNOWN)
+	add_equal_note (pat, temp, code, op0, NULL_RTX);
 
-  if (INSN_P (pat) && NEXT_INSN (pat) != NULL_RTX && code != UNKNOWN)
-    add_equal_note (pat, temp, code, op0, NULL_RTX);
-
-  emit_insn (pat);
+      emit_insn (pat);
 
-  if (temp != target)
-    emit_move_insn (target, temp);
-  return true;
+      if (temp != target)
+	emit_move_insn (target, temp);
+      return true;
+    }
+  delete_insns_since (last);
+  return false;
 }
 /* Generate an instruction whose insn-code is INSN_CODE,
    with two operands: an output TARGET and an input OP0.
@@ -3771,7 +3798,7 @@  maybe_emit_unop_insn (int icode, rtx tar
    the value that is stored into TARGET.  */
 
 void
-emit_unop_insn (int icode, rtx target, rtx op0, enum rtx_code code)
+emit_unop_insn (enum insn_code icode, rtx target, rtx op0, enum rtx_code code)
 {
   bool ok = maybe_emit_unop_insn (icode, target, op0, code);
   gcc_assert (ok);
@@ -3943,15 +3970,15 @@  can_compare_p (enum rtx_code code, enum 
   test = gen_rtx_fmt_ee (code, mode, const0_rtx, const0_rtx);
   do
     {
-      int icode;
+      enum insn_code icode;
 
       if (purpose == ccp_jump
           && (icode = optab_handler (cbranch_optab, mode)) != CODE_FOR_nothing
-          && insn_data[icode].operand[0].predicate (test, mode))
+          && insn_operand_matches (icode, 0, test))
         return 1;
       if (purpose == ccp_store_flag
           && (icode = optab_handler (cstore_optab, mode)) != CODE_FOR_nothing
-          && insn_data[icode].operand[1].predicate (test, mode))
+          && insn_operand_matches (icode, 1, test))
         return 1;
       if (purpose == ccp_cmov
 	  && optab_handler (cmov_optab, mode) != CODE_FOR_nothing)
@@ -4112,16 +4139,14 @@  prepare_cmp_insn (rtx x, rtx y, enum rtx
       enum insn_code icode;
       icode = optab_handler (cbranch_optab, cmp_mode);
       if (icode != CODE_FOR_nothing
-	  && insn_data[icode].operand[0].predicate (test, VOIDmode))
+	  && insn_operand_matches (icode, 0, test))
 	{
 	  rtx last = get_last_insn ();
 	  rtx op0 = prepare_operand (icode, x, 1, mode, cmp_mode, unsignedp);
 	  rtx op1 = prepare_operand (icode, y, 2, mode, cmp_mode, unsignedp);
 	  if (op0 && op1
-	      && insn_data[icode].operand[1].predicate
-		 (op0, insn_data[icode].operand[1].mode)
-	      && insn_data[icode].operand[2].predicate
-		 (op1, insn_data[icode].operand[2].mode))
+	      && insn_operand_matches (icode, 1, op0)
+	      && insn_operand_matches (icode, 2, op1))
 	    {
 	      XEXP (test, 0) = op0;
 	      XEXP (test, 1) = op1;
@@ -4200,18 +4225,17 @@  prepare_cmp_insn (rtx x, rtx y, enum rtx
    that it is accepted by the operand predicate.  Return the new value.  */
 
 rtx
-prepare_operand (int icode, rtx x, int opnum, enum machine_mode mode,
+prepare_operand (enum insn_code icode, rtx x, int opnum, enum machine_mode mode,
 		 enum machine_mode wider_mode, int unsignedp)
 {
   if (mode != wider_mode)
     x = convert_modes (wider_mode, mode, x, unsignedp);
 
-  if (!insn_data[icode].operand[opnum].predicate
-      (x, insn_data[icode].operand[opnum].mode))
+  if (!insn_operand_matches (icode, opnum, x))
     {
       if (reload_completed)
 	return NULL_RTX;
-      x = copy_to_mode_reg (insn_data[icode].operand[opnum].mode, x);
+      x = copy_to_mode_reg (insn_data[(int) icode].operand[opnum].mode, x);
     }
 
   return x;
@@ -4232,7 +4256,7 @@  emit_cmp_and_jump_insn_1 (rtx test, enum
   icode = optab_handler (cbranch_optab, optab_mode);
 
   gcc_assert (icode != CODE_FOR_nothing);
-  gcc_assert (insn_data[icode].operand[0].predicate (test, VOIDmode));
+  gcc_assert (insn_operand_matches (icode, 0, test));
   emit_jump_insn (GEN_FCN (icode) (test, XEXP (test, 0), XEXP (test, 1), label));
 }
 
@@ -4421,10 +4445,7 @@  prepare_float_lib_cmp (rtx x, rtx y, enu
 void
 emit_indirect_jump (rtx loc)
 {
-  if (!insn_data[(int) CODE_FOR_indirect_jump].operand[0].predicate
-      (loc, Pmode))
-    loc = copy_to_mode_reg (Pmode, loc);
-
+  loc = legitimize_insn_source (CODE_FOR_indirect_jump, 0, loc, Pmode);
   emit_jump_insn (gen_indirect_jump (loc));
   emit_barrier ();
 }
@@ -4497,21 +4518,13 @@  emit_conditional_move (rtx target, enum 
   if (!target)
     target = gen_reg_rtx (mode);
 
-  subtarget = target;
-
   /* If the insn doesn't accept these operands, put them in pseudos.  */
 
-  if (!insn_data[icode].operand[0].predicate
-      (subtarget, insn_data[icode].operand[0].mode))
-    subtarget = gen_reg_rtx (insn_data[icode].operand[0].mode);
-
-  if (!insn_data[icode].operand[2].predicate
-      (op2, insn_data[icode].operand[2].mode))
-    op2 = copy_to_mode_reg (insn_data[icode].operand[2].mode, op2);
-
-  if (!insn_data[icode].operand[3].predicate
-      (op3, insn_data[icode].operand[3].mode))
-    op3 = copy_to_mode_reg (insn_data[icode].operand[3].mode, op3);
+  if (!(subtarget = maybe_legitimize_insn_target (icode, 0, target, mode))
+      || !(op2 = maybe_legitimize_insn_source (icode, 2, op2, mode))
+      || !(op3 = maybe_legitimize_insn_source (icode, 3, op3, mode)))
+    /* The caller cleans up.  */
+    return NULL_RTX;
 
   /* Everything should now be in the suitable form.  */
 
@@ -4638,19 +4651,11 @@  emit_conditional_add (rtx target, enum r
 
   /* If the insn doesn't accept these operands, put them in pseudos.  */
 
-  if (!insn_data[icode].operand[0].predicate
-      (target, insn_data[icode].operand[0].mode))
-    subtarget = gen_reg_rtx (insn_data[icode].operand[0].mode);
-  else
-    subtarget = target;
-
-  if (!insn_data[icode].operand[2].predicate
-      (op2, insn_data[icode].operand[2].mode))
-    op2 = copy_to_mode_reg (insn_data[icode].operand[2].mode, op2);
-
-  if (!insn_data[icode].operand[3].predicate
-      (op3, insn_data[icode].operand[3].mode))
-    op3 = copy_to_mode_reg (insn_data[icode].operand[3].mode, op3);
+  if (!(subtarget = maybe_legitimize_insn_target (icode, 0, target, mode))
+      && !(op2 = maybe_legitimize_insn_source (icode, 2, op2, mode))
+      && !(op3 = maybe_legitimize_insn_source (icode, 3, op3, mode)))
+    /* The caller cleans up.  */
+    return NULL_RTX;
 
   /* Everything should now be in the suitable form.  */
 
@@ -4699,14 +4704,11 @@  emit_conditional_add (rtx target, enum r
 rtx
 gen_add2_insn (rtx x, rtx y)
 {
-  int icode = (int) optab_handler (add_optab, GET_MODE (x));
+  enum insn_code icode = optab_handler (add_optab, GET_MODE (x));
 
-  gcc_assert (insn_data[icode].operand[0].predicate
-	      (x, insn_data[icode].operand[0].mode));
-  gcc_assert (insn_data[icode].operand[1].predicate
-	      (x, insn_data[icode].operand[1].mode));
-  gcc_assert (insn_data[icode].operand[2].predicate
-	      (y, insn_data[icode].operand[2].mode));
+  gcc_assert (insn_operand_matches (icode, 0, x));
+  gcc_assert (insn_operand_matches (icode, 1, x));
+  gcc_assert (insn_operand_matches (icode, 2, y));
 
   return GEN_FCN (icode) (x, x, y);
 }
@@ -4717,15 +4719,12 @@  gen_add2_insn (rtx x, rtx y)
 rtx
 gen_add3_insn (rtx r0, rtx r1, rtx c)
 {
-  int icode = (int) optab_handler (add_optab, GET_MODE (r0));
+  enum insn_code icode = optab_handler (add_optab, GET_MODE (r0));
 
   if (icode == CODE_FOR_nothing
-      || !(insn_data[icode].operand[0].predicate
-	   (r0, insn_data[icode].operand[0].mode))
-      || !(insn_data[icode].operand[1].predicate
-	   (r1, insn_data[icode].operand[1].mode))
-      || !(insn_data[icode].operand[2].predicate
-	   (c, insn_data[icode].operand[2].mode)))
+      || !insn_operand_matches (icode, 0, r0)
+      || !insn_operand_matches (icode, 1, r1)
+      || !insn_operand_matches (icode, 2, c))
     return NULL_RTX;
 
   return GEN_FCN (icode) (r0, r1, c);
@@ -4734,21 +4733,18 @@  gen_add3_insn (rtx r0, rtx r1, rtx c)
 int
 have_add2_insn (rtx x, rtx y)
 {
-  int icode;
+  enum insn_code icode;
 
   gcc_assert (GET_MODE (x) != VOIDmode);
 
-  icode = (int) optab_handler (add_optab, GET_MODE (x));
+  icode = optab_handler (add_optab, GET_MODE (x));
 
   if (icode == CODE_FOR_nothing)
     return 0;
 
-  if (!(insn_data[icode].operand[0].predicate
-	(x, insn_data[icode].operand[0].mode))
-      || !(insn_data[icode].operand[1].predicate
-	   (x, insn_data[icode].operand[1].mode))
-      || !(insn_data[icode].operand[2].predicate
-	   (y, insn_data[icode].operand[2].mode)))
+  if (!insn_operand_matches (icode, 0, x)
+      || !insn_operand_matches (icode, 1, x)
+      || !insn_operand_matches (icode, 2, y))
     return 0;
 
   return 1;
@@ -4759,14 +4755,11 @@  have_add2_insn (rtx x, rtx y)
 rtx
 gen_sub2_insn (rtx x, rtx y)
 {
-  int icode = (int) optab_handler (sub_optab, GET_MODE (x));
+  enum insn_code icode = optab_handler (sub_optab, GET_MODE (x));
 
-  gcc_assert (insn_data[icode].operand[0].predicate
-	      (x, insn_data[icode].operand[0].mode));
-  gcc_assert (insn_data[icode].operand[1].predicate
-	      (x, insn_data[icode].operand[1].mode));
-  gcc_assert  (insn_data[icode].operand[2].predicate
-	       (y, insn_data[icode].operand[2].mode));
+  gcc_assert (insn_operand_matches (icode, 0, x));
+  gcc_assert (insn_operand_matches (icode, 1, x));
+  gcc_assert (insn_operand_matches (icode, 2, y));
 
   return GEN_FCN (icode) (x, x, y);
 }
@@ -4777,15 +4770,12 @@  gen_sub2_insn (rtx x, rtx y)
 rtx
 gen_sub3_insn (rtx r0, rtx r1, rtx c)
 {
-  int icode = (int) optab_handler (sub_optab, GET_MODE (r0));
+  enum insn_code icode = optab_handler (sub_optab, GET_MODE (r0));
 
   if (icode == CODE_FOR_nothing
-      || !(insn_data[icode].operand[0].predicate
-	   (r0, insn_data[icode].operand[0].mode))
-      || !(insn_data[icode].operand[1].predicate
-	   (r1, insn_data[icode].operand[1].mode))
-      || !(insn_data[icode].operand[2].predicate
-	   (c, insn_data[icode].operand[2].mode)))
+      || !insn_operand_matches (icode, 0, r0)
+      || !insn_operand_matches (icode, 1, r1)
+      || !insn_operand_matches (icode, 2, c))
     return NULL_RTX;
 
   return GEN_FCN (icode) (r0, r1, c);
@@ -4794,21 +4784,18 @@  gen_sub3_insn (rtx r0, rtx r1, rtx c)
 int
 have_sub2_insn (rtx x, rtx y)
 {
-  int icode;
+  enum insn_code icode;
 
   gcc_assert (GET_MODE (x) != VOIDmode);
 
-  icode = (int) optab_handler (sub_optab, GET_MODE (x));
+  icode = optab_handler (sub_optab, GET_MODE (x));
 
   if (icode == CODE_FOR_nothing)
     return 0;
 
-  if (!(insn_data[icode].operand[0].predicate
-	(x, insn_data[icode].operand[0].mode))
-      || !(insn_data[icode].operand[1].predicate
-	   (x, insn_data[icode].operand[1].mode))
-      || !(insn_data[icode].operand[2].predicate
-	   (y, insn_data[icode].operand[2].mode)))
+  if (!insn_operand_matches (icode, 0, x)
+      || !insn_operand_matches (icode, 1, x)
+      || !insn_operand_matches (icode, 2, y))
     return 0;
 
   return 1;
@@ -6643,8 +6630,7 @@  gen_cond_trap (enum rtx_code code, rtx o
     return 0;
 
   /* Some targets only accept a zero trap code.  */
-  if (insn_data[icode].operand[3].predicate
-      && !insn_data[icode].operand[3].predicate (tcode, VOIDmode))
+  if (!insn_operand_matches (icode, 3, tcode))
     return 0;
 
   do_pending_stack_adjust ();
@@ -6753,14 +6739,8 @@  vector_compare_rtx (tree cond, bool unsi
   rtx_op1 = expand_expr (t_op1, NULL_RTX, TYPE_MODE (TREE_TYPE (t_op1)),
 			 EXPAND_STACK_PARM);
 
-  if (!insn_data[icode].operand[4].predicate (rtx_op0, GET_MODE (rtx_op0))
-      && GET_MODE (rtx_op0) != VOIDmode)
-    rtx_op0 = force_reg (GET_MODE (rtx_op0), rtx_op0);
-
-  if (!insn_data[icode].operand[5].predicate (rtx_op1, GET_MODE (rtx_op1))
-      && GET_MODE (rtx_op1) != VOIDmode)
-    rtx_op1 = force_reg (GET_MODE (rtx_op1), rtx_op1);
-
+  rtx_op0 = legitimize_insn_source (icode, 4, rtx_op0, GET_MODE (rtx_op0));
+  rtx_op1 = legitimize_insn_source (icode, 5, rtx_op1, GET_MODE (rtx_op1));
   return gen_rtx_fmt_ee (rcode, VOIDmode, rtx_op0, rtx_op1);
 }
 
@@ -6805,8 +6785,7 @@  expand_vec_cond_expr (tree vec_cond_type
   if (icode == CODE_FOR_nothing)
     return 0;
 
-  if (!target || !insn_data[icode].operand[0].predicate (target, mode))
-    target = gen_reg_rtx (mode);
+  target = legitimize_insn_target (icode, 0, target, mode);
 
   /* Get comparison rtx.  First expand both cond expr operands.  */
   comparison = vector_compare_rtx (op0,
@@ -6814,15 +6793,8 @@  expand_vec_cond_expr (tree vec_cond_type
   cc_op0 = XEXP (comparison, 0);
   cc_op1 = XEXP (comparison, 1);
   /* Expand both operands and force them in reg, if required.  */
-  rtx_op1 = expand_normal (op1);
-  if (!insn_data[icode].operand[1].predicate (rtx_op1, mode)
-      && mode != VOIDmode)
-    rtx_op1 = force_reg (mode, rtx_op1);
-
-  rtx_op2 = expand_normal (op2);
-  if (!insn_data[icode].operand[2].predicate (rtx_op2, mode)
-      && mode != VOIDmode)
-    rtx_op2 = force_reg (mode, rtx_op2);
+  rtx_op1 = legitimize_insn_source (icode, 1, expand_normal (op1), mode);
+  rtx_op2 = legitimize_insn_source (icode, 2, expand_normal (op2), mode);
 
   /* Emit instruction! */
   emit_insn (GEN_FCN (icode) (target, rtx_op1, rtx_op2,
@@ -6843,27 +6815,26 @@  expand_val_compare_and_swap_1 (rtx mem, 
 			       rtx target, enum insn_code icode)
 {
   enum machine_mode mode = GET_MODE (mem);
-  rtx insn;
+  rtx pat, start;
 
-  if (!target || !insn_data[icode].operand[0].predicate (target, mode))
-    target = gen_reg_rtx (mode);
+  start = get_last_insn ();
 
   if (GET_MODE (old_val) != VOIDmode && GET_MODE (old_val) != mode)
     old_val = convert_modes (mode, GET_MODE (old_val), old_val, 1);
-  if (!insn_data[icode].operand[2].predicate (old_val, mode))
-    old_val = force_reg (mode, old_val);
 
   if (GET_MODE (new_val) != VOIDmode && GET_MODE (new_val) != mode)
     new_val = convert_modes (mode, GET_MODE (new_val), new_val, 1);
-  if (!insn_data[icode].operand[3].predicate (new_val, mode))
-    new_val = force_reg (mode, new_val);
 
-  insn = GEN_FCN (icode) (target, mem, old_val, new_val);
-  if (insn == NULL_RTX)
-    return NULL_RTX;
-  emit_insn (insn);
-
-  return target;
+  if ((target = maybe_legitimize_insn_target (icode, 0, target, mode))
+      && (old_val = maybe_legitimize_insn_source (icode, 2, old_val, mode))
+      && (new_val = maybe_legitimize_insn_source (icode, 3, new_val, mode))
+      && (pat = GEN_FCN (icode) (target, mem, old_val, new_val)))
+    {
+      emit_insn (pat);
+      return target;
+    }
+  delete_insns_since (start);
+  return NULL_RTX;
 }
 
 /* Expand a compare-and-swap operation and return its value.  */
@@ -7068,17 +7039,19 @@  expand_sync_operation (rtx mem, rtx val,
   /* Generate the direct operation, if present.  */
   if (icode != CODE_FOR_nothing)
     {
-      if (GET_MODE (val) != VOIDmode && GET_MODE (val) != mode)
-	val = convert_modes (mode, GET_MODE (val), val, 1);
-      if (!insn_data[icode].operand[1].predicate (val, mode))
-	val = force_reg (mode, val);
+      rtx start, op1;
 
-      insn = GEN_FCN (icode) (mem, val);
-      if (insn)
+      start = get_last_insn ();
+      op1 = val;
+      if (GET_MODE (op1) != VOIDmode && GET_MODE (op1) != mode)
+	op1 = convert_modes (mode, GET_MODE (op1), op1, 1);
+      if ((op1 = maybe_legitimize_insn_source (icode, 1, op1, mode))
+	  && (insn = GEN_FCN (icode) (mem, op1)))
 	{
 	  emit_insn (insn);
 	  return const0_rtx;
 	}
+      delete_insns_since (start);
     }
 
   /* Failing that, generate a compare-and-swap loop in which we perform the
@@ -7201,16 +7174,17 @@  expand_sync_fetch_operation (rtx mem, rt
   /* If we found something supported, great.  */
   if (icode != CODE_FOR_nothing)
     {
-      if (!target || !insn_data[icode].operand[0].predicate (target, mode))
-	target = gen_reg_rtx (mode);
+      rtx start, dest, op2;
 
-      if (GET_MODE (val) != VOIDmode && GET_MODE (val) != mode)
-	val = convert_modes (mode, GET_MODE (val), val, 1);
-      if (!insn_data[icode].operand[2].predicate (val, mode))
-	val = force_reg (mode, val);
+      start = get_last_insn ();
+
+      op2 = val;
+      if (GET_MODE (op2) != VOIDmode && GET_MODE (op2) != mode)
+	op2 = convert_modes (mode, GET_MODE (op2), op2, 1);
 
-      insn = GEN_FCN (icode) (target, mem, val);
-      if (insn)
+      if ((dest = maybe_legitimize_insn_target (icode, 0, target, mode))
+	  && (op2 = maybe_legitimize_insn_source (icode, 2, op2, mode))
+	  && (insn = GEN_FCN (icode) (dest, mem, op2)))
 	{
 	  emit_insn (insn);
 
@@ -7228,20 +7202,18 @@  expand_sync_fetch_operation (rtx mem, rt
 
 	      if (code == NOT)
 		{
-		  target = expand_simple_binop (mode, AND, target, val,
-						NULL_RTX, true,
-						OPTAB_LIB_WIDEN);
-		  target = expand_simple_unop (mode, code, target,
-					       NULL_RTX, true);
+		  dest = expand_simple_binop (mode, AND, dest, op2,
+					      NULL_RTX, true, OPTAB_LIB_WIDEN);
+		  dest = expand_simple_unop (mode, code, dest, NULL_RTX, true);
 		}
 	      else
-		target = expand_simple_binop (mode, code, target, val,
-					      NULL_RTX, true,
-					      OPTAB_LIB_WIDEN);
+		dest = expand_simple_binop (mode, code, dest, op2,
+					    NULL_RTX, true, OPTAB_LIB_WIDEN);
 	    }
 
-	  return target;
+	  return dest;
 	}
+      delete_insns_since (start);
     }
 
   /* Failing that, generate a compare-and-swap loop in which we perform the
@@ -7299,20 +7271,22 @@  expand_sync_lock_test_and_set (rtx mem, 
   icode = direct_optab_handler (sync_lock_test_and_set_optab, mode);
   if (icode != CODE_FOR_nothing)
     {
-      if (!target || !insn_data[icode].operand[0].predicate (target, mode))
-	target = gen_reg_rtx (mode);
+      rtx start, dest, op2;
 
-      if (GET_MODE (val) != VOIDmode && GET_MODE (val) != mode)
-	val = convert_modes (mode, GET_MODE (val), val, 1);
-      if (!insn_data[icode].operand[2].predicate (val, mode))
-	val = force_reg (mode, val);
+      start = get_last_insn ();
 
-      insn = GEN_FCN (icode) (target, mem, val);
-      if (insn)
+      op2 = val;
+      if (GET_MODE (op2) != VOIDmode && GET_MODE (op2) != mode)
+	op2 = convert_modes (mode, GET_MODE (op2), op2, 1);
+
+      if ((dest = maybe_legitimize_insn_target (icode, 0, target, mode))
+	  && (op2 = maybe_legitimize_insn_source (icode, 2, op2, mode))
+	  && (insn = GEN_FCN (icode) (dest, mem, op2)))
 	{
 	  emit_insn (insn);
-	  return target;
+	  return dest;
 	}
+      delete_insns_since (start);
     }
 
   /* Otherwise, use a compare-and-swap loop for the exchange.  */
Index: gcc/reload.c
===================================================================
--- gcc/reload.c	2011-02-28 10:29:00.000000000 +0000
+++ gcc/reload.c	2011-03-16 15:56:56.000000000 +0000
@@ -5819,17 +5819,15 @@  #define REG_OK_FOR_CONTEXT(CONTEXT, REGN
 	      rtx equiv = (MEM_P (XEXP (x, 0))
 			   ? XEXP (x, 0)
 			   : reg_equiv_mem[regno]);
-	      int icode = (int) optab_handler (add_optab, GET_MODE (x));
+	      enum insn_code icode = optab_handler (add_optab, GET_MODE (x));
 	      if (insn && NONJUMP_INSN_P (insn) && equiv
 		  && memory_operand (equiv, GET_MODE (equiv))
 #ifdef HAVE_cc0
 		  && ! sets_cc0_p (PATTERN (insn))
 #endif
 		  && ! (icode != CODE_FOR_nothing
-			&& ((*insn_data[icode].operand[0].predicate)
-			    (equiv, GET_MODE (x)))
-			&& ((*insn_data[icode].operand[1].predicate)
-			    (equiv, GET_MODE (x)))))
+			&& insn_operand_matches (icode, 0, equiv)
+			&& insn_operand_matches (icode, 1, equiv)))
 		{
 		  /* We use the original pseudo for loc, so that
 		     emit_reload_insns() knows which pseudo this
Index: gcc/reload1.c
===================================================================
--- gcc/reload1.c	2011-01-28 11:27:01.000000000 +0000
+++ gcc/reload1.c	2011-03-16 15:56:16.000000000 +0000
@@ -8479,7 +8479,7 @@  gen_reload (rtx out, rtx in, int opnum, 
 	 not valid than to dummy things up.  */
 
       rtx op0, op1, tem, insn;
-      int code;
+      enum insn_code code;
 
       op0 = find_replacement (&XEXP (in, 0));
       op1 = find_replacement (&XEXP (in, 1));
@@ -8517,14 +8517,13 @@  gen_reload (rtx out, rtx in, int opnum, 
 	 DEFINE_PEEPHOLE should be specified that recognizes the sequence
 	 we emit below.  */
 
-      code = (int) optab_handler (add_optab, GET_MODE (out));
+      code = optab_handler (add_optab, GET_MODE (out));
 
       if (CONSTANT_P (op1) || MEM_P (op1) || GET_CODE (op1) == SUBREG
 	  || (REG_P (op1)
 	      && REGNO (op1) >= FIRST_PSEUDO_REGISTER)
 	  || (code != CODE_FOR_nothing
-	      && ! ((*insn_data[code].operand[2].predicate)
-		    (op1, insn_data[code].operand[2].mode))))
+	      && !insn_operand_matches (code, 2, op1)))
 	tem = op0, op0 = op1, op1 = tem;
 
       gen_reload (out, op0, opnum, type);
Index: gcc/targhooks.c
===================================================================
--- gcc/targhooks.c	2011-01-28 11:27:01.000000000 +0000
+++ gcc/targhooks.c	2011-03-16 10:55:16.000000000 +0000
@@ -893,8 +893,7 @@  default_secondary_reload (bool in_p ATTR
 				reload_mode);
 
       if (icode != CODE_FOR_nothing
-	  && insn_data[(int) icode].operand[in_p].predicate
-	  && ! insn_data[(int) icode].operand[in_p].predicate (x, reload_mode))
+	  && !insn_operand_matches (icode, in_p, x))
 	icode = CODE_FOR_nothing;
       else if (icode != CODE_FOR_nothing)
 	{